2024-05-03T17:57:47.865 INFO:root:teuthology version: 0.0.1.dev271+g1ae7ad8 2024-05-03T17:57:47.865 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_rishabh@teuthology --verbose --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-quiesce/no 6-workunit/postgres}} --archive /home/teuthworker/archive/rishabh-2024-05-03_13:56:19-fs-wip-rishabh-testing-20240501.193033-testing-default-smithi/7687421 --name rishabh-2024-05-03_13:56:19-fs-wip-rishabh-testing-20240501.193033-testing-default-smithi -- /home/teuthworker/archive/rishabh-2024-05-03_13:56:19-fs-wip-rishabh-testing-20240501.193033-testing-default-smithi/7687421/orig.config.yaml 2024-05-03T17:57:47.925 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-03T17:57:48.018 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-05-03_13:56:19-fs-wip-rishabh-testing-20240501.193033-testing-default-smithi/7687421 branch: wip-rishabh-testing-20240501.193033 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-quiesce/no 6-workunit/postgres}} email: null first_in_suite: false job_id: '7687421' kernel: flavor: default kdb: true sha1: 806ec36b21129f113ce356c007dd59694d340b37 last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule name: rishabh-2024-05-03_13:56:19-fs-wip-rishabh-testing-20240501.193033-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-rishabh-testing-20240501.193033 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 1 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - slow metadata IO - SLOW_OPS - slow request sha1: 9a047fe7599a549042d06a3fb5b6813a203eabea subvols: create: 2 subvol_options: --size 25000000000 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 0 mountpoint: /tmp/cephfs check-counter: counters: mds: - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: 9a047fe7599a549042d06a3fb5b6813a203eabea kclient: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 1 mountpoint: /tmp/cephfs selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-testing-20240501.193033 sha1: 9a047fe7599a549042d06a3fb5b6813a203eabea owner: scheduled_rishabh@teuthology priority: 75 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 1698 sha1: 9a047fe7599a549042d06a3fb5b6813a203eabea sleep_before_teardown: 0 subset: 434/512 suite: fs suite_branch: wip-rishabh-testing-20240501.193033 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_9a047fe7599a549042d06a3fb5b6813a203eabea/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 9a047fe7599a549042d06a3fb5b6813a203eabea syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi033.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBB5MJOMfNJoeLFESwvDdxrMu7l9cNMT5vJSFUQKCxUFkVVMzEqy5eLVYWo1ZetgqZ8d3Qt486/dqubD7Z3OWnow= smithi099.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE0/eV2WlXCOuZQJIIYmNhDM4SW9D6/Cv73iMFFqFhh7V47S+fjkTdPCwI+HfXEZ8Z1AHD3Fxmvm1jd5rSPCK2E= smithi150.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKz4iJkWOPpLKXVRb/Cfm3Ff8016sVcAKpouIvI9VKzJdiqa4BBbDIZZakAfxD5pZng3tvc7s2gpr38Yqh0tpuI= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - ceph-fuse: null - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph config set mgr mgr/snap_schedule/log_level debug - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - exec: client.0: - sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/ - sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres - sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data - sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?" - sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local - sudo restorecon -R -v /tmp/cephfs/postgres - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf - sudo -u postgres -- postgresql-setup --initdb - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo systemctl start postgresql - sudo -u postgres -- pgbench -s 32 -i - sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900 - sudo systemctl stop postgresql - sudo ls -lZaR /tmp/cephfs/postgres/ teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 1ae7ad82388e92a475afff437d49054826c019a1 timestamp: 2024-05-03_13:56:19 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-05-03T17:57:48.019 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_9a047fe7599a549042d06a3fb5b6813a203eabea/qa; will attempt to use it 2024-05-03T17:57:48.019 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_9a047fe7599a549042d06a3fb5b6813a203eabea/qa/tasks 2024-05-03T17:57:48.020 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-03T17:57:48.021 INFO:teuthology.task.internal:Checking packages... 2024-05-03T17:57:48.045 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '9a047fe7599a549042d06a3fb5b6813a203eabea' 2024-05-03T17:57:48.045 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-03T17:57:48.045 INFO:teuthology.packaging:ref: None 2024-05-03T17:57:48.045 INFO:teuthology.packaging:tag: None 2024-05-03T17:57:48.045 INFO:teuthology.packaging:branch: wip-rishabh-testing-20240501.193033 2024-05-03T17:57:48.045 INFO:teuthology.packaging:sha1: 9a047fe7599a549042d06a3fb5b6813a203eabea 2024-05-03T17:57:48.045 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-rishabh-testing-20240501.193033 2024-05-03T17:57:48.184 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3450.g9a047fe7 2024-05-03T17:57:48.186 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-03T17:57:48.192 INFO:teuthology.task.internal:no buildpackages task found 2024-05-03T17:57:48.192 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-03T17:57:48.243 INFO:teuthology.task.internal:Saving configuration 2024-05-03T17:57:48.265 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-03T17:57:48.275 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-03T17:57:48.296 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi033.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-05-03_13:56:19-fs-wip-rishabh-testing-20240501.193033-testing-default-smithi/7687421', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-03 17:52:51.902249', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBB5MJOMfNJoeLFESwvDdxrMu7l9cNMT5vJSFUQKCxUFkVVMzEqy5eLVYWo1ZetgqZ8d3Qt486/dqubD7Z3OWnow='} 2024-05-03T17:57:48.318 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi099.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-05-03_13:56:19-fs-wip-rishabh-testing-20240501.193033-testing-default-smithi/7687421', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-03 17:52:51.903613', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE0/eV2WlXCOuZQJIIYmNhDM4SW9D6/Cv73iMFFqFhh7V47S+fjkTdPCwI+HfXEZ8Z1AHD3Fxmvm1jd5rSPCK2E='} 2024-05-03T17:57:48.339 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi150.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-05-03_13:56:19-fs-wip-rishabh-testing-20240501.193033-testing-default-smithi/7687421', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-03 17:52:51.904738', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKz4iJkWOPpLKXVRb/Cfm3Ff8016sVcAKpouIvI9VKzJdiqa4BBbDIZZakAfxD5pZng3tvc7s2gpr38Yqh0tpuI='} 2024-05-03T17:57:48.340 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-03T17:57:48.345 INFO:teuthology.task.internal:roles: ubuntu@smithi033.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-05-03T17:57:48.345 INFO:teuthology.task.internal:roles: ubuntu@smithi099.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-05-03T17:57:48.345 INFO:teuthology.task.internal:roles: ubuntu@smithi150.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-05-03T17:57:48.345 INFO:teuthology.run_tasks:Running task console_log... 2024-05-03T17:57:48.486 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fc010ac3160>, signals=[15]) 2024-05-03T17:57:48.486 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-03T17:57:48.492 INFO:teuthology.task.internal:Opening connections... 2024-05-03T17:57:48.492 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi033.front.sepia.ceph.com 2024-05-03T17:57:48.494 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-03T17:57:48.572 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi099.front.sepia.ceph.com 2024-05-03T17:57:48.573 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-03T17:57:48.647 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi150.front.sepia.ceph.com 2024-05-03T17:57:48.647 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-03T17:57:48.719 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-03T17:57:48.726 DEBUG:teuthology.orchestra.run.smithi033:> uname -m 2024-05-03T17:57:48.744 INFO:teuthology.orchestra.run.smithi033.stdout:x86_64 2024-05-03T17:57:48.744 DEBUG:teuthology.orchestra.run.smithi033:> cat /etc/os-release 2024-05-03T17:57:48.802 INFO:teuthology.orchestra.run.smithi033.stdout:NAME="CentOS Stream" 2024-05-03T17:57:48.802 INFO:teuthology.orchestra.run.smithi033.stdout:VERSION="9" 2024-05-03T17:57:48.802 INFO:teuthology.orchestra.run.smithi033.stdout:ID="centos" 2024-05-03T17:57:48.802 INFO:teuthology.orchestra.run.smithi033.stdout:ID_LIKE="rhel fedora" 2024-05-03T17:57:48.802 INFO:teuthology.orchestra.run.smithi033.stdout:VERSION_ID="9" 2024-05-03T17:57:48.802 INFO:teuthology.orchestra.run.smithi033.stdout:PLATFORM_ID="platform:el9" 2024-05-03T17:57:48.802 INFO:teuthology.orchestra.run.smithi033.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-03T17:57:48.803 INFO:teuthology.orchestra.run.smithi033.stdout:ANSI_COLOR="0;31" 2024-05-03T17:57:48.803 INFO:teuthology.orchestra.run.smithi033.stdout:LOGO="fedora-logo-icon" 2024-05-03T17:57:48.803 INFO:teuthology.orchestra.run.smithi033.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-03T17:57:48.803 INFO:teuthology.orchestra.run.smithi033.stdout:HOME_URL="https://centos.org/" 2024-05-03T17:57:48.803 INFO:teuthology.orchestra.run.smithi033.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-03T17:57:48.803 INFO:teuthology.orchestra.run.smithi033.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-03T17:57:48.803 INFO:teuthology.orchestra.run.smithi033.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-03T17:57:48.803 INFO:teuthology.lock.ops:Updating smithi033.front.sepia.ceph.com on lock server 2024-05-03T17:57:48.832 DEBUG:teuthology.orchestra.run.smithi099:> uname -m 2024-05-03T17:57:48.853 INFO:teuthology.orchestra.run.smithi099.stdout:x86_64 2024-05-03T17:57:48.853 DEBUG:teuthology.orchestra.run.smithi099:> cat /etc/os-release 2024-05-03T17:57:48.910 INFO:teuthology.orchestra.run.smithi099.stdout:NAME="CentOS Stream" 2024-05-03T17:57:48.911 INFO:teuthology.orchestra.run.smithi099.stdout:VERSION="9" 2024-05-03T17:57:48.911 INFO:teuthology.orchestra.run.smithi099.stdout:ID="centos" 2024-05-03T17:57:48.911 INFO:teuthology.orchestra.run.smithi099.stdout:ID_LIKE="rhel fedora" 2024-05-03T17:57:48.911 INFO:teuthology.orchestra.run.smithi099.stdout:VERSION_ID="9" 2024-05-03T17:57:48.911 INFO:teuthology.orchestra.run.smithi099.stdout:PLATFORM_ID="platform:el9" 2024-05-03T17:57:48.911 INFO:teuthology.orchestra.run.smithi099.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-03T17:57:48.911 INFO:teuthology.orchestra.run.smithi099.stdout:ANSI_COLOR="0;31" 2024-05-03T17:57:48.911 INFO:teuthology.orchestra.run.smithi099.stdout:LOGO="fedora-logo-icon" 2024-05-03T17:57:48.911 INFO:teuthology.orchestra.run.smithi099.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-03T17:57:48.911 INFO:teuthology.orchestra.run.smithi099.stdout:HOME_URL="https://centos.org/" 2024-05-03T17:57:48.911 INFO:teuthology.orchestra.run.smithi099.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-03T17:57:48.911 INFO:teuthology.orchestra.run.smithi099.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-03T17:57:48.911 INFO:teuthology.orchestra.run.smithi099.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-03T17:57:48.912 INFO:teuthology.lock.ops:Updating smithi099.front.sepia.ceph.com on lock server 2024-05-03T17:57:48.932 DEBUG:teuthology.orchestra.run.smithi150:> uname -m 2024-05-03T17:57:48.949 INFO:teuthology.orchestra.run.smithi150.stdout:x86_64 2024-05-03T17:57:48.950 DEBUG:teuthology.orchestra.run.smithi150:> cat /etc/os-release 2024-05-03T17:57:49.007 INFO:teuthology.orchestra.run.smithi150.stdout:NAME="CentOS Stream" 2024-05-03T17:57:49.007 INFO:teuthology.orchestra.run.smithi150.stdout:VERSION="9" 2024-05-03T17:57:49.007 INFO:teuthology.orchestra.run.smithi150.stdout:ID="centos" 2024-05-03T17:57:49.007 INFO:teuthology.orchestra.run.smithi150.stdout:ID_LIKE="rhel fedora" 2024-05-03T17:57:49.007 INFO:teuthology.orchestra.run.smithi150.stdout:VERSION_ID="9" 2024-05-03T17:57:49.007 INFO:teuthology.orchestra.run.smithi150.stdout:PLATFORM_ID="platform:el9" 2024-05-03T17:57:49.007 INFO:teuthology.orchestra.run.smithi150.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-03T17:57:49.007 INFO:teuthology.orchestra.run.smithi150.stdout:ANSI_COLOR="0;31" 2024-05-03T17:57:49.007 INFO:teuthology.orchestra.run.smithi150.stdout:LOGO="fedora-logo-icon" 2024-05-03T17:57:49.007 INFO:teuthology.orchestra.run.smithi150.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-03T17:57:49.007 INFO:teuthology.orchestra.run.smithi150.stdout:HOME_URL="https://centos.org/" 2024-05-03T17:57:49.007 INFO:teuthology.orchestra.run.smithi150.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-03T17:57:49.007 INFO:teuthology.orchestra.run.smithi150.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-03T17:57:49.007 INFO:teuthology.orchestra.run.smithi150.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-03T17:57:49.008 INFO:teuthology.lock.ops:Updating smithi150.front.sepia.ceph.com on lock server 2024-05-03T17:57:49.032 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-03T17:57:49.041 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-03T17:57:49.076 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-03T17:57:49.076 DEBUG:teuthology.orchestra.run.smithi033:> test '!' -e /home/ubuntu/cephtest 2024-05-03T17:57:49.078 DEBUG:teuthology.orchestra.run.smithi099:> test '!' -e /home/ubuntu/cephtest 2024-05-03T17:57:49.081 DEBUG:teuthology.orchestra.run.smithi150:> test '!' -e /home/ubuntu/cephtest 2024-05-03T17:57:49.097 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-03T17:57:49.109 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-03T17:57:49.109 DEBUG:teuthology.orchestra.run.smithi033:> test -z $(ls -A /var/lib/ceph) 2024-05-03T17:57:49.136 DEBUG:teuthology.orchestra.run.smithi099:> test -z $(ls -A /var/lib/ceph) 2024-05-03T17:57:49.138 DEBUG:teuthology.orchestra.run.smithi150:> test -z $(ls -A /var/lib/ceph) 2024-05-03T17:57:49.161 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-03T17:57:49.231 INFO:teuthology.run_tasks:Running task kernel... 2024-05-03T17:57:49.245 INFO:teuthology.task.kernel:normalize config orig: {'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'} 2024-05-03T17:57:49.245 INFO:teuthology.task.kernel:config {'mon.a': {'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}, 'mon.b': {'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}, 'mon.c': {'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}}, timeout 300 2024-05-03T17:57:49.245 DEBUG:teuthology.orchestra.run.smithi033:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-03T17:57:49.246 DEBUG:teuthology.orchestra.run.smithi099:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-03T17:57:49.246 DEBUG:teuthology.orchestra.run.smithi150:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-03T17:57:49.262 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-03T17:57:49.263 DEBUG:teuthology.task.kernel:sha1 for mon.c is 806ec36b21129f113ce356c007dd59694d340b37 2024-05-03T17:57:49.263 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-05-03T17:57:49.263 DEBUG:teuthology.orchestra.run.smithi150:> uname -r 2024-05-03T17:57:49.264 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-03T17:57:49.264 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-03T17:57:49.264 DEBUG:teuthology.task.kernel:sha1 for mon.a is 806ec36b21129f113ce356c007dd59694d340b37 2024-05-03T17:57:49.265 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-05-03T17:57:49.265 DEBUG:teuthology.orchestra.run.smithi033:> uname -r 2024-05-03T17:57:49.265 DEBUG:teuthology.task.kernel:sha1 for mon.b is 806ec36b21129f113ce356c007dd59694d340b37 2024-05-03T17:57:49.265 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-05-03T17:57:49.265 DEBUG:teuthology.orchestra.run.smithi099:> uname -r 2024-05-03T17:57:49.319 INFO:teuthology.orchestra.run.smithi150.stdout:5.14.0-437.el9.x86_64 2024-05-03T17:57:49.320 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-05-03T17:57:49.320 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-05-03T17:57:49.320 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-05-03T17:57:49.324 INFO:teuthology.orchestra.run.smithi033.stdout:5.14.0-437.el9.x86_64 2024-05-03T17:57:49.325 INFO:teuthology.orchestra.run.smithi099.stdout:5.14.0-437.el9.x86_64 2024-05-03T17:57:49.325 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-05-03T17:57:49.325 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-05-03T17:57:49.325 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-05-03T17:57:49.326 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-05-03T17:57:49.326 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-05-03T17:57:49.326 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-05-03T17:57:49.543 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum upgrade -y linux-firmware 2024-05-03T17:57:49.555 DEBUG:teuthology.orchestra.run.smithi033:> sudo yum upgrade -y linux-firmware 2024-05-03T17:57:49.557 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum upgrade -y linux-firmware 2024-05-03T17:57:51.936 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 9 - BaseOS 10 MB/s | 8.0 MB 00:00 2024-05-03T17:57:52.221 INFO:teuthology.orchestra.run.smithi099.stdout:CentOS Stream 9 - BaseOS 6.9 MB/s | 8.0 MB 00:01 2024-05-03T17:57:53.201 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - BaseOS 4.1 MB/s | 8.0 MB 00:01 2024-05-03T17:57:54.430 INFO:teuthology.orchestra.run.smithi099.stdout:CentOS Stream 9 - AppStream 16 MB/s | 19 MB 00:01 2024-05-03T17:57:55.602 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - AppStream 14 MB/s | 19 MB 00:01 2024-05-03T17:57:57.769 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 9 - AppStream 4.0 MB/s | 19 MB 00:04 2024-05-03T17:57:58.381 INFO:teuthology.orchestra.run.smithi099.stdout:CentOS Stream 9 - CRB 15 MB/s | 6.2 MB 00:00 2024-05-03T17:58:00.048 INFO:teuthology.orchestra.run.smithi099.stdout:CentOS Stream 9 - Extras packages 50 kB/s | 16 kB 00:00 2024-05-03T17:58:00.080 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - CRB 7.0 MB/s | 6.2 MB 00:00 2024-05-03T17:58:01.299 INFO:teuthology.orchestra.run.smithi099.stdout:Extra Packages for Enterprise Linux 19 MB/s | 21 MB 00:01 2024-05-03T17:58:02.070 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.2 MB 00:00 2024-05-03T17:58:02.836 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - Extras packages 11 kB/s | 16 kB 00:01 2024-05-03T17:58:03.670 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 9 - Extras packages 40 kB/s | 16 kB 00:00 2024-05-03T17:58:04.440 INFO:teuthology.orchestra.run.smithi150.stdout:Extra Packages for Enterprise Linux 14 MB/s | 21 MB 00:01 2024-05-03T17:58:05.541 INFO:teuthology.orchestra.run.smithi033.stdout:Extra Packages for Enterprise Linux 12 MB/s | 21 MB 00:01 2024-05-03T17:58:06.407 INFO:teuthology.orchestra.run.smithi099.stdout:lab-extras 14 kB/s | 1.7 kB 00:00 2024-05-03T17:58:08.389 INFO:teuthology.orchestra.run.smithi099.stdout:Dependencies resolved. 2024-05-03T17:58:08.390 INFO:teuthology.orchestra.run.smithi099.stdout:Nothing to do. 2024-05-03T17:58:08.390 INFO:teuthology.orchestra.run.smithi099.stdout:Complete! 2024-05-03T17:58:08.508 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on mon.b... 2024-05-03T17:58:08.508 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-05-03T17:58:08.744 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-05-03T17:58:08.744 DEBUG:teuthology.orchestra.run.smithi099:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-05-03T17:58:08.746 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi099.front.sepia.ceph.com to complete... 2024-05-03T17:58:09.562 INFO:teuthology.orchestra.run.smithi150.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2024-05-03T17:58:10.821 INFO:teuthology.orchestra.run.smithi033.stdout:lab-extras 34 kB/s | 1.7 kB 00:00 2024-05-03T17:58:11.563 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-05-03T17:58:11.564 INFO:teuthology.orchestra.run.smithi150.stdout:Nothing to do. 2024-05-03T17:58:11.564 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-05-03T17:58:11.671 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on mon.c... 2024-05-03T17:58:11.671 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-05-03T17:58:11.917 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-05-03T17:58:11.917 DEBUG:teuthology.orchestra.run.smithi150:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-05-03T17:58:11.920 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi150.front.sepia.ceph.com to complete... 2024-05-03T17:58:12.876 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies resolved. 2024-05-03T17:58:12.876 INFO:teuthology.orchestra.run.smithi033.stdout:Nothing to do. 2024-05-03T17:58:12.877 INFO:teuthology.orchestra.run.smithi033.stdout:Complete! 2024-05-03T17:58:12.978 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on mon.a... 2024-05-03T17:58:12.978 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-05-03T17:58:13.128 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-05-03T17:58:13.128 DEBUG:teuthology.orchestra.run.smithi033:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-05-03T17:58:13.131 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi033.front.sepia.ceph.com to complete... 2024-05-03T17:58:50.505 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03 17:58:50 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-05-03T17:58:50.506 INFO:teuthology.orchestra.run.smithi150.stderr:FINISHED --2024-05-03 17:58:50-- 2024-05-03T17:58:50.506 INFO:teuthology.orchestra.run.smithi150.stderr:Total wall clock time: 39s 2024-05-03T17:58:50.506 INFO:teuthology.orchestra.run.smithi150.stderr:Downloaded: 1 files, 341M in 38s (8.98 MB/s) 2024-05-03T17:58:50.509 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on mon.c... 2024-05-03T17:58:50.509 DEBUG:teuthology.orchestra.run.smithi150:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-05-03T17:58:52.614 INFO:teuthology.orchestra.run.smithi150.stdout:Verifying... ######################################## 2024-05-03T17:58:52.746 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing... ######################################## 2024-05-03T17:58:52.857 INFO:teuthology.orchestra.run.smithi150.stdout:Updating / installing... 2024-05-03T17:58:53.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03 17:58:53 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-05-03T17:58:53.638 INFO:teuthology.orchestra.run.smithi099.stderr:FINISHED --2024-05-03 17:58:53-- 2024-05-03T17:58:53.638 INFO:teuthology.orchestra.run.smithi099.stderr:Total wall clock time: 45s 2024-05-03T17:58:53.638 INFO:teuthology.orchestra.run.smithi099.stderr:Downloaded: 1 files, 341M in 45s (7.66 MB/s) 2024-05-03T17:58:53.641 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on mon.b... 2024-05-03T17:58:53.641 DEBUG:teuthology.orchestra.run.smithi099:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-05-03T17:58:55.722 INFO:teuthology.orchestra.run.smithi099.stdout:Verifying... ######################################## 2024-05-03T17:58:55.935 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing... ######################################## 2024-05-03T17:58:56.052 INFO:teuthology.orchestra.run.smithi099.stdout:Updating / installing... 2024-05-03T17:58:58.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03 17:58:58 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-05-03T17:58:58.448 INFO:teuthology.orchestra.run.smithi033.stderr:FINISHED --2024-05-03 17:58:58-- 2024-05-03T17:58:58.448 INFO:teuthology.orchestra.run.smithi033.stderr:Total wall clock time: 45s 2024-05-03T17:58:58.448 INFO:teuthology.orchestra.run.smithi033.stderr:Downloaded: 1 files, 341M in 44s (7.81 MB/s) 2024-05-03T17:58:58.451 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on mon.a... 2024-05-03T17:58:58.451 DEBUG:teuthology.orchestra.run.smithi033:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-05-03T17:59:00.634 INFO:teuthology.orchestra.run.smithi033.stdout:Verifying... ######################################## 2024-05-03T17:59:00.753 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing... ######################################## 2024-05-03T17:59:00.799 INFO:teuthology.orchestra.run.smithi150.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-05-03T17:59:00.886 INFO:teuthology.orchestra.run.smithi033.stdout:Updating / installing... 2024-05-03T17:59:03.975 INFO:teuthology.orchestra.run.smithi099.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-05-03T17:59:09.254 INFO:teuthology.orchestra.run.smithi033.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-05-03T17:59:37.982 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi150.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-05-03T17:59:37.982 DEBUG:teuthology.orchestra.run.smithi150:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-05-03T17:59:38.063 INFO:teuthology.orchestra.run.smithi150.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-05-03T17:59:38.063 INFO:teuthology.orchestra.run.smithi150.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-05-03T17:59:38.063 INFO:teuthology.orchestra.run.smithi150.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-05-03T17:59:38.063 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-05-03T17:59:38.064 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-05-03T17:59:38.064 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-05-03T17:59:38.064 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-05-03T17:59:38.064 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-05-03T17:59:38.064 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-05-03T17:59:38.064 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-05-03T17:59:38.064 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-05-03T17:59:38.064 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-05-03T17:59:38.064 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-05-03T17:59:38.064 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-05-03T17:59:38.064 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-05-03T17:59:38.064 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-05-03T17:59:38.064 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-05-03T17:59:38.064 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-05-03T17:59:38.065 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-05-03T17:59:38.065 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-05-03T17:59:38.065 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-05-03T17:59:38.065 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-05-03T17:59:38.065 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-05-03T17:59:38.065 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-05-03T17:59:38.065 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-05-03T17:59:38.065 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-05-03T17:59:38.065 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-05-03T17:59:38.065 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-05-03T17:59:38.065 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-05-03T17:59:38.065 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-05-03T17:59:38.065 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-05-03T17:59:38.065 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-05-03T17:59:38.066 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-05-03T17:59:38.066 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-05-03T17:59:38.066 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-05-03T17:59:38.066 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-05-03T17:59:38.066 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-05-03T17:59:38.066 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-05-03T17:59:38.066 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-05-03T17:59:38.066 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-05-03T17:59:38.066 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-05-03T17:59:38.066 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-05-03T17:59:38.066 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-05-03T17:59:38.066 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-05-03T17:59:38.067 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-05-03T17:59:38.067 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-05-03T17:59:38.067 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-05-03T17:59:38.067 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-05-03T17:59:38.067 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-05-03T17:59:38.067 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-05-03T17:59:38.067 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-05-03T17:59:38.067 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-05-03T17:59:38.067 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-05-03T17:59:38.067 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-05-03T17:59:38.067 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-05-03T17:59:38.068 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-05-03T17:59:38.068 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-05-03T17:59:38.068 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-05-03T17:59:38.068 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-05-03T17:59:38.068 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-05-03T17:59:38.068 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-05-03T17:59:38.068 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-05-03T17:59:38.068 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-05-03T17:59:38.068 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-05-03T17:59:38.068 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-05-03T17:59:38.068 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-05-03T17:59:38.068 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-05-03T17:59:38.068 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-05-03T17:59:38.069 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-05-03T17:59:38.069 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-05-03T17:59:38.069 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-05-03T17:59:38.069 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-05-03T17:59:38.069 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-05-03T17:59:38.069 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-05-03T17:59:38.069 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-05-03T17:59:38.069 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-05-03T17:59:38.069 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-05-03T17:59:38.069 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-05-03T17:59:38.069 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-05-03T17:59:38.069 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-05-03T17:59:38.069 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-05-03T17:59:38.069 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-05-03T17:59:38.070 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-05-03T17:59:38.070 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-05-03T17:59:38.070 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-05-03T17:59:38.070 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-05-03T17:59:38.070 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-05-03T17:59:38.070 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-05-03T17:59:38.070 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-05-03T17:59:38.070 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-05-03T17:59:38.070 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-05-03T17:59:38.070 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-05-03T17:59:38.070 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-05-03T17:59:38.070 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-05-03T17:59:38.070 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-05-03T17:59:38.071 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-05-03T17:59:38.071 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-05-03T17:59:38.071 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-05-03T17:59:38.071 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-05-03T17:59:38.071 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-05-03T17:59:38.071 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-05-03T17:59:38.071 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-05-03T17:59:38.071 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-05-03T17:59:38.071 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-05-03T17:59:38.071 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-05-03T17:59:38.071 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-05-03T17:59:38.071 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-05-03T17:59:38.071 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-05-03T17:59:38.072 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-05-03T17:59:38.072 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-05-03T17:59:38.072 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-05-03T17:59:38.072 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-05-03T17:59:38.072 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-05-03T17:59:38.072 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-05-03T17:59:38.072 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-05-03T17:59:38.072 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-05-03T17:59:38.072 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-05-03T17:59:38.072 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-05-03T17:59:38.072 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-05-03T17:59:38.072 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-05-03T17:59:38.072 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-05-03T17:59:38.072 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-05-03T17:59:38.073 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-05-03T17:59:38.073 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-05-03T17:59:38.073 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-05-03T17:59:38.073 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-05-03T17:59:38.073 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-05-03T17:59:38.073 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-05-03T17:59:38.073 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-05-03T17:59:38.073 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-05-03T17:59:38.073 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-05-03T17:59:38.073 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-05-03T17:59:38.073 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-05-03T17:59:38.073 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-05-03T17:59:38.073 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-05-03T17:59:38.073 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-05-03T17:59:38.074 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-05-03T17:59:38.074 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-05-03T17:59:38.074 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-05-03T17:59:38.074 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-05-03T17:59:38.074 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-05-03T17:59:38.074 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-05-03T17:59:38.074 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-05-03T17:59:38.074 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-05-03T17:59:38.074 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-05-03T17:59:38.074 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-05-03T17:59:38.074 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-05-03T17:59:38.074 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-05-03T17:59:38.074 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-05-03T17:59:38.074 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-05-03T17:59:38.075 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-05-03T17:59:38.075 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-05-03T17:59:38.075 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-05-03T17:59:38.075 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-05-03T17:59:38.075 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-05-03T17:59:38.075 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-05-03T17:59:38.075 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-05-03T17:59:38.075 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-05-03T17:59:38.075 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-05-03T17:59:38.075 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-05-03T17:59:38.075 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-05-03T17:59:38.075 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-05-03T17:59:38.075 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-05-03T17:59:38.075 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-05-03T17:59:38.076 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-05-03T17:59:38.076 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-05-03T17:59:38.076 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-05-03T17:59:38.076 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-05-03T17:59:38.076 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-05-03T17:59:38.076 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-05-03T17:59:38.076 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-05-03T17:59:38.076 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-05-03T17:59:38.076 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-05-03T17:59:38.076 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-05-03T17:59:38.076 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-05-03T17:59:38.076 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-05-03T17:59:38.076 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-05-03T17:59:38.076 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-05-03T17:59:38.077 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-05-03T17:59:38.077 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-05-03T17:59:38.077 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-05-03T17:59:38.077 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-05-03T17:59:38.077 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-05-03T17:59:38.077 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-05-03T17:59:38.077 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-05-03T17:59:38.077 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-05-03T17:59:38.077 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-05-03T17:59:38.077 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-05-03T17:59:38.077 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-05-03T17:59:38.077 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-05-03T17:59:38.077 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-05-03T17:59:38.078 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-05-03T17:59:38.078 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-05-03T17:59:38.078 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-05-03T17:59:38.078 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-05-03T17:59:38.078 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-05-03T17:59:38.078 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-05-03T17:59:38.078 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-05-03T17:59:38.078 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-05-03T17:59:38.078 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-05-03T17:59:38.078 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-05-03T17:59:38.078 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-05-03T17:59:38.078 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-05-03T17:59:38.078 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-05-03T17:59:38.078 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-05-03T17:59:38.078 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-05-03T17:59:38.079 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-05-03T17:59:38.079 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-05-03T17:59:38.079 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-05-03T17:59:38.079 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-05-03T17:59:38.079 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-05-03T17:59:38.079 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-05-03T17:59:38.079 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-05-03T17:59:38.079 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-05-03T17:59:38.079 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-05-03T17:59:38.079 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-05-03T17:59:38.079 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-05-03T17:59:38.079 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-05-03T17:59:38.079 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-05-03T17:59:38.080 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-05-03T17:59:38.080 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-05-03T17:59:38.080 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-05-03T17:59:38.080 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-05-03T17:59:38.080 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-05-03T17:59:38.080 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-05-03T17:59:38.080 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-05-03T17:59:38.080 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-05-03T17:59:38.080 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-05-03T17:59:38.080 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-05-03T17:59:38.080 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-05-03T17:59:38.080 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-05-03T17:59:38.080 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-05-03T17:59:38.080 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-05-03T17:59:38.080 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-05-03T17:59:38.080 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-05-03T17:59:38.081 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-05-03T17:59:38.081 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-05-03T17:59:38.081 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-05-03T17:59:38.081 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-05-03T17:59:38.081 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-05-03T17:59:38.081 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-05-03T17:59:38.081 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-05-03T17:59:38.081 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-05-03T17:59:38.081 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-05-03T17:59:38.081 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-05-03T17:59:38.081 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-05-03T17:59:38.081 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-05-03T17:59:38.081 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-05-03T17:59:38.081 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-05-03T17:59:38.081 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-05-03T17:59:38.082 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-05-03T17:59:38.082 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-05-03T17:59:38.082 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-05-03T17:59:38.082 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-05-03T17:59:38.082 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-05-03T17:59:38.082 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-05-03T17:59:38.082 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-05-03T17:59:38.082 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-05-03T17:59:38.082 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-05-03T17:59:38.082 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-05-03T17:59:38.082 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-05-03T17:59:38.082 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-05-03T17:59:38.082 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-05-03T17:59:38.082 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-05-03T17:59:38.083 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-05-03T17:59:38.083 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-05-03T17:59:38.083 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-05-03T17:59:38.083 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-05-03T17:59:38.083 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-05-03T17:59:38.083 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-05-03T17:59:38.083 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-05-03T17:59:38.083 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-05-03T17:59:38.083 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-05-03T17:59:38.083 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-05-03T17:59:38.083 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-05-03T17:59:38.083 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-05-03T17:59:38.083 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-05-03T17:59:38.083 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-05-03T17:59:38.084 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-05-03T17:59:38.084 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-05-03T17:59:38.084 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-05-03T17:59:38.084 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-05-03T17:59:38.084 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-05-03T17:59:38.084 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-05-03T17:59:38.084 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-05-03T17:59:38.084 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-05-03T17:59:38.084 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-05-03T17:59:38.084 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-05-03T17:59:38.084 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-05-03T17:59:38.084 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-05-03T17:59:38.084 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-05-03T17:59:38.084 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-05-03T17:59:38.084 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-05-03T17:59:38.085 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-05-03T17:59:38.085 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-05-03T17:59:38.085 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-05-03T17:59:38.085 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-05-03T17:59:38.085 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-05-03T17:59:38.085 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-05-03T17:59:38.085 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-05-03T17:59:38.085 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-05-03T17:59:38.085 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-05-03T17:59:38.085 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-05-03T17:59:38.085 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-05-03T17:59:38.085 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-05-03T17:59:38.085 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-05-03T17:59:38.085 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-05-03T17:59:38.085 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-05-03T17:59:38.085 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-05-03T17:59:38.086 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-05-03T17:59:38.086 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-05-03T17:59:38.086 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-05-03T17:59:38.086 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-05-03T17:59:38.086 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-05-03T17:59:38.086 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-05-03T17:59:38.086 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-05-03T17:59:38.086 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-05-03T17:59:38.086 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-05-03T17:59:38.086 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-05-03T17:59:38.086 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-05-03T17:59:38.086 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-05-03T17:59:38.086 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-05-03T17:59:38.086 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-05-03T17:59:38.086 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-05-03T17:59:38.087 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-05-03T17:59:38.087 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-05-03T17:59:38.087 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-05-03T17:59:38.087 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-05-03T17:59:38.087 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-05-03T17:59:38.087 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-05-03T17:59:38.087 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-05-03T17:59:38.087 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-05-03T17:59:38.087 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-05-03T17:59:38.087 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-05-03T17:59:38.087 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-05-03T17:59:38.087 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-05-03T17:59:38.087 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-05-03T17:59:38.087 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-05-03T17:59:38.087 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-05-03T17:59:38.088 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-05-03T17:59:38.088 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-05-03T17:59:38.088 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-05-03T17:59:38.088 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-05-03T17:59:38.088 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-05-03T17:59:38.088 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-05-03T17:59:38.088 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-05-03T17:59:38.088 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-05-03T17:59:38.088 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-05-03T17:59:38.088 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-05-03T17:59:38.088 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-05-03T17:59:38.088 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-05-03T17:59:38.088 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-05-03T17:59:38.088 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-05-03T17:59:38.088 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-05-03T17:59:38.089 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-05-03T17:59:38.089 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-05-03T17:59:38.089 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-05-03T17:59:38.089 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-05-03T17:59:38.089 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-05-03T17:59:38.089 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-05-03T17:59:38.089 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-05-03T17:59:38.089 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-05-03T17:59:38.089 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-05-03T17:59:38.089 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-05-03T17:59:38.089 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-05-03T17:59:38.089 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-05-03T17:59:38.089 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-05-03T17:59:38.089 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-05-03T17:59:38.090 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-05-03T17:59:38.090 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-05-03T17:59:38.090 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-05-03T17:59:38.090 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-05-03T17:59:38.090 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-05-03T17:59:38.090 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-05-03T17:59:38.090 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-05-03T17:59:38.090 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-05-03T17:59:38.090 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-05-03T17:59:38.090 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-05-03T17:59:38.090 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-05-03T17:59:38.090 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-05-03T17:59:38.090 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-05-03T17:59:38.091 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-05-03T17:59:38.091 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-05-03T17:59:38.091 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-05-03T17:59:38.091 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-05-03T17:59:38.091 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-05-03T17:59:38.091 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-05-03T17:59:38.091 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-05-03T17:59:38.091 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-05-03T17:59:38.091 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-05-03T17:59:38.091 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-05-03T17:59:38.091 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-05-03T17:59:38.091 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-05-03T17:59:38.091 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-05-03T17:59:38.091 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-05-03T17:59:38.092 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-05-03T17:59:38.092 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-05-03T17:59:38.092 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-05-03T17:59:38.092 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-05-03T17:59:38.092 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-05-03T17:59:38.092 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-05-03T17:59:38.092 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-05-03T17:59:38.092 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-05-03T17:59:38.092 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-05-03T17:59:38.092 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-05-03T17:59:38.092 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-05-03T17:59:38.092 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-05-03T17:59:38.092 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-05-03T17:59:38.092 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-05-03T17:59:38.093 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-05-03T17:59:38.093 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-05-03T17:59:38.093 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-05-03T17:59:38.093 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-05-03T17:59:38.093 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-05-03T17:59:38.093 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-05-03T17:59:38.093 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-05-03T17:59:38.093 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-05-03T17:59:38.093 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-05-03T17:59:38.093 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-05-03T17:59:38.093 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-05-03T17:59:38.093 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-05-03T17:59:38.093 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-05-03T17:59:38.094 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-05-03T17:59:38.094 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-05-03T17:59:38.094 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-05-03T17:59:38.094 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-05-03T17:59:38.094 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-05-03T17:59:38.094 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-05-03T17:59:38.094 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-05-03T17:59:38.094 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-05-03T17:59:38.094 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-05-03T17:59:38.094 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-05-03T17:59:38.094 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-05-03T17:59:38.094 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-05-03T17:59:38.094 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-05-03T17:59:38.094 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-05-03T17:59:38.095 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-05-03T17:59:38.095 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-05-03T17:59:38.095 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-05-03T17:59:38.095 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-05-03T17:59:38.095 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-05-03T17:59:38.095 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-05-03T17:59:38.095 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-05-03T17:59:38.095 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-05-03T17:59:38.095 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-05-03T17:59:38.095 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-05-03T17:59:38.095 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-05-03T17:59:38.095 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-05-03T17:59:38.095 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-05-03T17:59:38.095 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-05-03T17:59:38.096 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-05-03T17:59:38.096 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-05-03T17:59:38.096 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-05-03T17:59:38.096 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-05-03T17:59:38.096 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-05-03T17:59:38.096 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-05-03T17:59:38.096 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-05-03T17:59:38.096 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-05-03T17:59:38.096 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-05-03T17:59:38.096 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-05-03T17:59:38.096 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-05-03T17:59:38.096 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-05-03T17:59:38.096 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-05-03T17:59:38.096 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-05-03T17:59:38.096 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-05-03T17:59:38.097 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-05-03T17:59:38.097 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-05-03T17:59:38.097 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-05-03T17:59:38.097 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-05-03T17:59:38.097 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-05-03T17:59:38.097 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-05-03T17:59:38.097 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-05-03T17:59:38.097 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-05-03T17:59:38.097 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-05-03T17:59:38.097 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-05-03T17:59:38.097 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-05-03T17:59:38.097 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-05-03T17:59:38.097 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-05-03T17:59:38.097 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-05-03T17:59:38.098 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-05-03T17:59:38.098 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-05-03T17:59:38.098 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-05-03T17:59:38.098 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-05-03T17:59:38.098 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-05-03T17:59:38.098 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-05-03T17:59:38.098 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-05-03T17:59:38.098 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-05-03T17:59:38.098 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-05-03T17:59:38.098 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-05-03T17:59:38.098 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-05-03T17:59:38.098 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-05-03T17:59:38.098 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-05-03T17:59:38.098 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-05-03T17:59:38.099 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-05-03T17:59:38.099 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-05-03T17:59:38.099 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-05-03T17:59:38.099 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-05-03T17:59:38.099 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-05-03T17:59:38.099 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-05-03T17:59:38.099 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-05-03T17:59:38.099 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-05-03T17:59:38.099 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-05-03T17:59:38.099 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-05-03T17:59:38.099 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-05-03T17:59:38.099 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-05-03T17:59:38.099 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-05-03T17:59:38.100 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-05-03T17:59:38.100 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-05-03T17:59:38.100 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-05-03T17:59:38.100 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-05-03T17:59:38.100 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-05-03T17:59:38.100 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-05-03T17:59:38.100 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-05-03T17:59:38.100 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-05-03T17:59:38.100 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-05-03T17:59:38.100 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-05-03T17:59:38.100 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-05-03T17:59:38.100 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-05-03T17:59:38.100 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-05-03T17:59:38.100 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-05-03T17:59:38.100 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-05-03T17:59:38.101 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-05-03T17:59:38.101 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-05-03T17:59:38.101 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-05-03T17:59:38.101 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-05-03T17:59:38.101 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-05-03T17:59:38.101 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-05-03T17:59:38.101 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-05-03T17:59:38.101 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-05-03T17:59:38.101 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-05-03T17:59:38.101 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-05-03T17:59:38.101 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-05-03T17:59:38.101 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-05-03T17:59:38.101 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-05-03T17:59:38.101 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-05-03T17:59:38.101 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-05-03T17:59:38.102 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-05-03T17:59:38.102 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-05-03T17:59:38.102 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-05-03T17:59:38.102 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-05-03T17:59:38.102 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-05-03T17:59:38.102 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-05-03T17:59:38.102 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-05-03T17:59:38.102 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-05-03T17:59:38.102 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-05-03T17:59:38.102 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-05-03T17:59:38.102 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-05-03T17:59:38.102 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-05-03T17:59:38.102 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-05-03T17:59:38.103 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-05-03T17:59:38.103 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-05-03T17:59:38.103 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-05-03T17:59:38.103 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-05-03T17:59:38.103 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-05-03T17:59:38.103 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-05-03T17:59:38.103 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-05-03T17:59:38.103 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-05-03T17:59:38.103 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-05-03T17:59:38.103 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-05-03T17:59:38.103 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-05-03T17:59:38.103 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-05-03T17:59:38.103 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-05-03T17:59:38.103 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-05-03T17:59:38.104 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-05-03T17:59:38.104 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-05-03T17:59:38.104 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-05-03T17:59:38.104 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-05-03T17:59:38.104 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-05-03T17:59:38.104 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-05-03T17:59:38.104 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-05-03T17:59:38.104 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-05-03T17:59:38.104 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-05-03T17:59:38.104 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-05-03T17:59:38.104 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-05-03T17:59:38.104 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-05-03T17:59:38.104 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-05-03T17:59:38.104 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-05-03T17:59:38.105 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-05-03T17:59:38.105 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-05-03T17:59:38.105 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-05-03T17:59:38.105 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-05-03T17:59:38.105 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-05-03T17:59:38.105 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-05-03T17:59:38.105 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-05-03T17:59:38.105 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-05-03T17:59:38.105 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-05-03T17:59:38.105 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-05-03T17:59:38.105 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-05-03T17:59:38.105 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-05-03T17:59:38.105 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-05-03T17:59:38.105 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-05-03T17:59:38.106 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-05-03T17:59:38.106 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-05-03T17:59:38.106 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-05-03T17:59:38.106 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-05-03T17:59:38.106 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-05-03T17:59:38.106 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-05-03T17:59:38.106 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-05-03T17:59:38.106 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-05-03T17:59:38.106 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-05-03T17:59:38.106 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-05-03T17:59:38.106 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-05-03T17:59:38.106 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-05-03T17:59:38.106 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-05-03T17:59:38.106 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-05-03T17:59:38.106 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-05-03T17:59:38.106 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-05-03T17:59:38.107 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-05-03T17:59:38.107 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-05-03T17:59:38.107 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-05-03T17:59:38.107 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-05-03T17:59:38.107 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-05-03T17:59:38.107 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-05-03T17:59:38.107 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-05-03T17:59:38.107 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-05-03T17:59:38.107 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-05-03T17:59:38.107 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-05-03T17:59:38.107 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-05-03T17:59:38.107 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-05-03T17:59:38.107 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-05-03T17:59:38.107 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-05-03T17:59:38.107 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-05-03T17:59:38.108 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-05-03T17:59:38.108 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-05-03T17:59:38.108 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-05-03T17:59:38.108 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-05-03T17:59:38.108 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-05-03T17:59:38.108 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-05-03T17:59:38.108 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-05-03T17:59:38.108 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-05-03T17:59:38.108 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-05-03T17:59:38.108 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-05-03T17:59:38.108 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-05-03T17:59:38.108 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-05-03T17:59:38.108 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-05-03T17:59:38.108 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-05-03T17:59:38.108 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-05-03T17:59:38.109 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-05-03T17:59:38.109 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-05-03T17:59:38.109 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-05-03T17:59:38.109 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-05-03T17:59:38.109 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-05-03T17:59:38.109 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-05-03T17:59:38.109 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-05-03T17:59:38.109 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-05-03T17:59:38.109 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-05-03T17:59:38.109 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-05-03T17:59:38.109 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-05-03T17:59:38.109 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-05-03T17:59:38.109 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-05-03T17:59:38.109 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-05-03T17:59:38.109 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-05-03T17:59:38.110 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-05-03T17:59:38.110 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-05-03T17:59:38.110 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-05-03T17:59:38.110 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-05-03T17:59:38.110 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-05-03T17:59:38.110 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-05-03T17:59:38.110 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-05-03T17:59:38.110 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-05-03T17:59:38.110 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-05-03T17:59:38.110 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-05-03T17:59:38.110 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-05-03T17:59:38.110 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-05-03T17:59:38.110 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-05-03T17:59:38.111 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-05-03T17:59:38.111 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-05-03T17:59:38.111 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-05-03T17:59:38.111 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-05-03T17:59:38.111 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-05-03T17:59:38.111 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-05-03T17:59:38.111 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-05-03T17:59:38.111 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-05-03T17:59:38.111 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-05-03T17:59:38.111 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-05-03T17:59:38.111 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-05-03T17:59:38.111 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-05-03T17:59:38.111 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-05-03T17:59:38.111 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-05-03T17:59:38.111 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-05-03T17:59:38.111 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-05-03T17:59:38.112 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-05-03T17:59:38.112 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-05-03T17:59:38.112 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-05-03T17:59:38.112 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-05-03T17:59:38.112 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-05-03T17:59:38.112 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-05-03T17:59:38.112 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-05-03T17:59:38.112 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-05-03T17:59:38.112 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-05-03T17:59:38.112 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-05-03T17:59:38.112 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-05-03T17:59:38.112 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-05-03T17:59:38.112 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-05-03T17:59:38.112 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-05-03T17:59:38.113 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-05-03T17:59:38.113 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-05-03T17:59:38.113 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-05-03T17:59:38.113 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-05-03T17:59:38.113 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-05-03T17:59:38.113 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-05-03T17:59:38.113 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-05-03T17:59:38.113 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-05-03T17:59:38.113 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-05-03T17:59:38.113 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-05-03T17:59:38.113 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-05-03T17:59:38.113 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-05-03T17:59:38.113 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-05-03T17:59:38.113 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-05-03T17:59:38.114 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-05-03T17:59:38.114 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-05-03T17:59:38.114 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-05-03T17:59:38.114 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-05-03T17:59:38.114 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-05-03T17:59:38.114 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-05-03T17:59:38.114 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-05-03T17:59:38.114 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-05-03T17:59:38.114 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-05-03T17:59:38.114 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-05-03T17:59:38.114 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-05-03T17:59:38.114 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-05-03T17:59:38.114 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-05-03T17:59:38.114 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-05-03T17:59:38.115 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-05-03T17:59:38.115 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-05-03T17:59:38.115 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-05-03T17:59:38.115 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-05-03T17:59:38.115 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-05-03T17:59:38.115 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-05-03T17:59:38.115 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-05-03T17:59:38.115 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-05-03T17:59:38.115 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-05-03T17:59:38.115 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-05-03T17:59:38.115 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-05-03T17:59:38.115 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-05-03T17:59:38.115 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-05-03T17:59:38.116 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-05-03T17:59:38.116 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-05-03T17:59:38.116 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-05-03T17:59:38.116 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-05-03T17:59:38.116 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-05-03T17:59:38.116 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-05-03T17:59:38.116 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-05-03T17:59:38.116 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-05-03T17:59:38.116 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-05-03T17:59:38.116 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-05-03T17:59:38.116 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-05-03T17:59:38.116 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-05-03T17:59:38.116 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-05-03T17:59:38.116 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-05-03T17:59:38.116 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-05-03T17:59:38.117 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-05-03T17:59:38.117 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-05-03T17:59:38.117 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-05-03T17:59:38.117 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-05-03T17:59:38.117 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-05-03T17:59:38.117 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-05-03T17:59:38.117 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-05-03T17:59:38.117 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-05-03T17:59:38.117 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-05-03T17:59:38.117 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-05-03T17:59:38.117 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-05-03T17:59:38.117 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-05-03T17:59:38.117 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-05-03T17:59:38.117 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-05-03T17:59:38.118 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-05-03T17:59:38.118 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-05-03T17:59:38.118 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-05-03T17:59:38.118 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-05-03T17:59:38.118 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-05-03T17:59:38.118 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-05-03T17:59:38.118 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-05-03T17:59:38.118 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-05-03T17:59:38.118 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-05-03T17:59:38.118 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-05-03T17:59:38.118 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-05-03T17:59:38.118 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-05-03T17:59:38.118 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-05-03T17:59:38.118 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-05-03T17:59:38.119 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-05-03T17:59:38.119 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-05-03T17:59:38.119 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-05-03T17:59:38.119 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-05-03T17:59:38.119 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-05-03T17:59:38.119 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-05-03T17:59:38.119 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-05-03T17:59:38.119 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-05-03T17:59:38.119 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-05-03T17:59:38.119 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-05-03T17:59:38.119 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-05-03T17:59:38.119 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-05-03T17:59:38.119 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-05-03T17:59:38.119 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-05-03T17:59:38.119 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-05-03T17:59:38.120 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-05-03T17:59:38.120 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-05-03T17:59:38.120 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-05-03T17:59:38.120 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-05-03T17:59:38.120 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-05-03T17:59:38.120 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-05-03T17:59:38.120 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-05-03T17:59:38.120 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-05-03T17:59:38.120 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-05-03T17:59:38.120 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-05-03T17:59:38.120 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-05-03T17:59:38.120 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-05-03T17:59:38.120 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-05-03T17:59:38.121 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-05-03T17:59:38.121 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-05-03T17:59:38.121 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-05-03T17:59:38.121 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-05-03T17:59:38.121 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-05-03T17:59:38.121 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-05-03T17:59:38.121 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-05-03T17:59:38.121 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-05-03T17:59:38.121 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-05-03T17:59:38.121 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-05-03T17:59:38.121 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-05-03T17:59:38.121 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-05-03T17:59:38.121 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-05-03T17:59:38.121 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-05-03T17:59:38.122 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-05-03T17:59:38.122 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-05-03T17:59:38.122 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-05-03T17:59:38.122 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-05-03T17:59:38.122 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-05-03T17:59:38.122 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-05-03T17:59:38.122 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-05-03T17:59:38.122 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-05-03T17:59:38.122 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-05-03T17:59:38.122 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-05-03T17:59:38.122 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-05-03T17:59:38.122 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-05-03T17:59:38.122 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-05-03T17:59:38.123 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-05-03T17:59:38.123 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-05-03T17:59:38.123 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-05-03T17:59:38.123 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-05-03T17:59:38.123 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-05-03T17:59:38.123 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-05-03T17:59:38.123 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-05-03T17:59:38.123 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-05-03T17:59:38.123 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-05-03T17:59:38.123 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-05-03T17:59:38.123 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-05-03T17:59:38.123 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-05-03T17:59:38.123 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-05-03T17:59:38.124 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-05-03T17:59:38.124 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-05-03T17:59:38.124 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-05-03T17:59:38.124 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-05-03T17:59:38.124 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-05-03T17:59:38.124 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-05-03T17:59:38.124 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-05-03T17:59:38.124 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-05-03T17:59:38.124 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-05-03T17:59:38.124 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-05-03T17:59:38.124 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-05-03T17:59:38.124 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-05-03T17:59:38.124 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-05-03T17:59:38.124 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-05-03T17:59:38.125 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-05-03T17:59:38.125 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-05-03T17:59:38.125 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-05-03T17:59:38.125 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-05-03T17:59:38.125 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-05-03T17:59:38.125 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-05-03T17:59:38.125 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-05-03T17:59:38.125 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-05-03T17:59:38.125 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-05-03T17:59:38.125 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-05-03T17:59:38.125 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-05-03T17:59:38.125 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-05-03T17:59:38.125 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-05-03T17:59:38.125 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-05-03T17:59:38.125 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-05-03T17:59:38.125 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-05-03T17:59:38.126 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-05-03T17:59:38.126 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-05-03T17:59:38.126 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-05-03T17:59:38.126 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-05-03T17:59:38.126 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-05-03T17:59:38.126 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-05-03T17:59:38.126 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-05-03T17:59:38.126 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-05-03T17:59:38.126 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-05-03T17:59:38.126 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-05-03T17:59:38.126 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-05-03T17:59:38.126 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-05-03T17:59:38.126 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-05-03T17:59:38.126 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-05-03T17:59:38.126 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-05-03T17:59:38.127 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-05-03T17:59:38.127 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-05-03T17:59:38.127 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-05-03T17:59:38.127 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-05-03T17:59:38.127 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-05-03T17:59:38.127 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-05-03T17:59:38.127 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-05-03T17:59:38.127 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-05-03T17:59:38.127 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-05-03T17:59:38.127 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-05-03T17:59:38.127 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-05-03T17:59:38.127 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-05-03T17:59:38.127 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-05-03T17:59:38.127 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-05-03T17:59:38.127 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-05-03T17:59:38.128 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-05-03T17:59:38.128 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-05-03T17:59:38.128 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-05-03T17:59:38.128 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-05-03T17:59:38.128 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-05-03T17:59:38.128 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-05-03T17:59:38.128 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-05-03T17:59:38.128 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-05-03T17:59:38.128 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-05-03T17:59:38.128 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-05-03T17:59:38.128 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-05-03T17:59:38.128 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-05-03T17:59:38.128 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-05-03T17:59:38.129 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-05-03T17:59:38.129 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-05-03T17:59:38.129 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-05-03T17:59:38.129 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-05-03T17:59:38.129 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-05-03T17:59:38.129 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-05-03T17:59:38.129 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-05-03T17:59:38.129 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-05-03T17:59:38.129 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-05-03T17:59:38.129 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-05-03T17:59:38.129 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-05-03T17:59:38.129 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-05-03T17:59:38.129 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-05-03T17:59:38.129 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-05-03T17:59:38.129 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-05-03T17:59:38.130 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-05-03T17:59:38.130 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-05-03T17:59:38.130 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-05-03T17:59:38.130 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-05-03T17:59:38.130 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-05-03T17:59:38.130 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-05-03T17:59:38.130 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-05-03T17:59:38.130 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-05-03T17:59:38.130 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-05-03T17:59:38.130 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-05-03T17:59:38.130 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-05-03T17:59:38.130 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-05-03T17:59:38.130 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-05-03T17:59:38.130 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-05-03T17:59:38.131 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-05-03T17:59:38.131 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-05-03T17:59:38.131 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-05-03T17:59:38.131 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-05-03T17:59:38.131 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-05-03T17:59:38.131 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-05-03T17:59:38.131 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-05-03T17:59:38.131 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-05-03T17:59:38.131 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-05-03T17:59:38.131 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-05-03T17:59:38.131 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-05-03T17:59:38.131 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-05-03T17:59:38.131 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-05-03T17:59:38.131 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-05-03T17:59:38.131 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-05-03T17:59:38.132 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-05-03T17:59:38.132 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-05-03T17:59:38.132 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-05-03T17:59:38.132 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-05-03T17:59:38.132 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-05-03T17:59:38.132 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-05-03T17:59:38.132 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-05-03T17:59:38.132 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-05-03T17:59:38.132 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-05-03T17:59:38.132 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-05-03T17:59:38.132 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-05-03T17:59:38.132 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-05-03T17:59:38.132 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-05-03T17:59:38.132 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-05-03T17:59:38.132 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-05-03T17:59:38.133 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-05-03T17:59:38.133 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-05-03T17:59:38.133 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-05-03T17:59:38.133 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-05-03T17:59:38.133 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-05-03T17:59:38.133 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-05-03T17:59:38.133 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-05-03T17:59:38.133 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-05-03T17:59:38.133 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-05-03T17:59:38.133 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-05-03T17:59:38.133 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-05-03T17:59:38.133 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-05-03T17:59:38.133 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-05-03T17:59:38.133 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-05-03T17:59:38.133 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-05-03T17:59:38.134 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-05-03T17:59:38.134 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-05-03T17:59:38.134 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-05-03T17:59:38.134 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-05-03T17:59:38.134 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-05-03T17:59:38.134 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-05-03T17:59:38.134 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-05-03T17:59:38.134 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-05-03T17:59:38.134 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-05-03T17:59:38.134 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-05-03T17:59:38.134 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-05-03T17:59:38.134 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-05-03T17:59:38.134 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-05-03T17:59:38.134 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-05-03T17:59:38.135 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-05-03T17:59:38.135 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-05-03T17:59:38.135 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-05-03T17:59:38.135 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-05-03T17:59:38.135 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-05-03T17:59:38.135 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-05-03T17:59:38.135 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-05-03T17:59:38.135 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-05-03T17:59:38.135 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-05-03T17:59:38.135 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-05-03T17:59:38.135 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-05-03T17:59:38.135 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-05-03T17:59:38.135 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-05-03T17:59:38.135 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-05-03T17:59:38.135 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-05-03T17:59:38.136 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-05-03T17:59:38.136 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-05-03T17:59:38.136 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-05-03T17:59:38.136 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-05-03T17:59:38.136 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-05-03T17:59:38.136 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-05-03T17:59:38.136 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-05-03T17:59:38.136 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-05-03T17:59:38.136 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-05-03T17:59:38.136 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-05-03T17:59:38.136 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-05-03T17:59:38.136 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-05-03T17:59:38.136 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-05-03T17:59:38.136 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-05-03T17:59:38.136 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-05-03T17:59:38.137 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-05-03T17:59:38.137 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-05-03T17:59:38.137 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-05-03T17:59:38.137 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-05-03T17:59:38.137 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-05-03T17:59:38.137 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-05-03T17:59:38.137 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-05-03T17:59:38.137 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-05-03T17:59:38.137 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-05-03T17:59:38.137 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-05-03T17:59:38.137 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-05-03T17:59:38.137 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-05-03T17:59:38.137 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-05-03T17:59:38.137 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-05-03T17:59:38.137 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-05-03T17:59:38.138 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-05-03T17:59:38.138 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-05-03T17:59:38.138 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-05-03T17:59:38.138 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-05-03T17:59:38.138 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-05-03T17:59:38.138 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-05-03T17:59:38.138 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-05-03T17:59:38.138 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-05-03T17:59:38.138 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-05-03T17:59:38.138 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-05-03T17:59:38.138 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-05-03T17:59:38.138 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-05-03T17:59:38.138 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-05-03T17:59:38.138 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-05-03T17:59:38.139 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-05-03T17:59:38.139 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-05-03T17:59:38.139 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-05-03T17:59:38.139 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-05-03T17:59:38.139 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-05-03T17:59:38.139 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-05-03T17:59:38.139 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-05-03T17:59:38.139 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-05-03T17:59:38.139 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-05-03T17:59:38.139 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-05-03T17:59:38.139 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-05-03T17:59:38.139 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-05-03T17:59:38.139 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-05-03T17:59:38.139 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-05-03T17:59:38.140 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-05-03T17:59:38.140 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-05-03T17:59:38.140 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-05-03T17:59:38.140 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-05-03T17:59:38.140 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-05-03T17:59:38.140 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-05-03T17:59:38.140 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-05-03T17:59:38.140 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-05-03T17:59:38.140 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-05-03T17:59:38.140 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-05-03T17:59:38.140 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-05-03T17:59:38.140 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-05-03T17:59:38.140 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-05-03T17:59:38.140 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-05-03T17:59:38.140 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-05-03T17:59:38.140 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-05-03T17:59:38.141 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-05-03T17:59:38.141 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-05-03T17:59:38.141 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-05-03T17:59:38.141 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-05-03T17:59:38.141 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-05-03T17:59:38.141 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-05-03T17:59:38.141 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-05-03T17:59:38.141 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-05-03T17:59:38.141 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-05-03T17:59:38.141 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-05-03T17:59:38.141 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-05-03T17:59:38.141 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-05-03T17:59:38.141 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-05-03T17:59:38.141 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-05-03T17:59:38.142 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-05-03T17:59:38.142 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-05-03T17:59:38.142 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-05-03T17:59:38.142 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-05-03T17:59:38.142 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-05-03T17:59:38.142 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-05-03T17:59:38.142 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-05-03T17:59:38.142 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-05-03T17:59:38.142 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-05-03T17:59:38.142 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-05-03T17:59:38.142 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-05-03T17:59:38.142 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-05-03T17:59:38.142 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-05-03T17:59:38.143 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-05-03T17:59:38.143 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-05-03T17:59:38.143 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-05-03T17:59:38.143 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-05-03T17:59:38.143 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-05-03T17:59:38.143 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-05-03T17:59:38.143 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-05-03T17:59:38.143 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-05-03T17:59:38.143 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-05-03T17:59:38.143 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-05-03T17:59:38.143 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-05-03T17:59:38.143 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-05-03T17:59:38.143 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-05-03T17:59:38.144 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-05-03T17:59:38.144 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-05-03T17:59:38.144 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-05-03T17:59:38.144 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-05-03T17:59:38.144 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-05-03T17:59:38.144 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-05-03T17:59:38.144 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-05-03T17:59:38.144 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-05-03T17:59:38.144 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-05-03T17:59:38.144 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-05-03T17:59:38.144 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-05-03T17:59:38.144 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-05-03T17:59:38.144 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-05-03T17:59:38.145 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-05-03T17:59:38.145 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-05-03T17:59:38.145 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-05-03T17:59:38.145 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-05-03T17:59:38.145 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-05-03T17:59:38.145 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-05-03T17:59:38.145 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-05-03T17:59:38.145 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-05-03T17:59:38.145 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-05-03T17:59:38.145 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-05-03T17:59:38.145 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-05-03T17:59:38.145 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-05-03T17:59:38.145 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-05-03T17:59:38.145 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-05-03T17:59:38.145 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-05-03T17:59:38.145 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-05-03T17:59:38.146 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-05-03T17:59:38.146 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-05-03T17:59:38.146 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-05-03T17:59:38.146 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-05-03T17:59:38.146 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-05-03T17:59:38.146 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-05-03T17:59:38.146 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-05-03T17:59:38.146 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-05-03T17:59:38.146 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-05-03T17:59:38.146 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-05-03T17:59:38.146 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-05-03T17:59:38.146 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-05-03T17:59:38.146 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-05-03T17:59:38.146 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-05-03T17:59:38.146 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-05-03T17:59:38.147 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-05-03T17:59:38.147 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-05-03T17:59:38.147 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-05-03T17:59:38.147 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-05-03T17:59:38.147 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-05-03T17:59:38.147 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-05-03T17:59:38.147 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-05-03T17:59:38.147 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-05-03T17:59:38.147 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-05-03T17:59:38.147 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-05-03T17:59:38.147 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-05-03T17:59:38.147 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-05-03T17:59:38.147 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-05-03T17:59:38.147 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-05-03T17:59:38.147 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-05-03T17:59:38.148 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-05-03T17:59:38.148 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-05-03T17:59:38.148 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-05-03T17:59:38.148 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-05-03T17:59:38.148 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-05-03T17:59:38.148 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-05-03T17:59:38.148 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-05-03T17:59:38.148 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-05-03T17:59:38.148 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-05-03T17:59:38.148 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-05-03T17:59:38.148 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-05-03T17:59:38.148 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-05-03T17:59:38.148 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-05-03T17:59:38.148 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-05-03T17:59:38.148 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-05-03T17:59:38.148 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-05-03T17:59:38.149 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-05-03T17:59:38.149 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-05-03T17:59:38.149 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-05-03T17:59:38.149 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-05-03T17:59:38.149 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-05-03T17:59:38.149 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-05-03T17:59:38.149 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-05-03T17:59:38.149 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-05-03T17:59:38.149 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-05-03T17:59:38.149 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-05-03T17:59:38.149 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-05-03T17:59:38.149 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-05-03T17:59:38.149 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-05-03T17:59:38.149 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-05-03T17:59:38.149 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-05-03T17:59:38.149 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-05-03T17:59:38.150 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-05-03T17:59:38.150 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-05-03T17:59:38.150 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-05-03T17:59:38.150 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-05-03T17:59:38.150 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-05-03T17:59:38.150 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-05-03T17:59:38.150 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-05-03T17:59:38.150 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-05-03T17:59:38.150 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-05-03T17:59:38.150 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-05-03T17:59:38.150 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-05-03T17:59:38.150 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-05-03T17:59:38.150 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-05-03T17:59:38.150 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-05-03T17:59:38.151 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-05-03T17:59:38.151 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-05-03T17:59:38.151 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-05-03T17:59:38.151 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-05-03T17:59:38.151 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-05-03T17:59:38.151 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-05-03T17:59:38.151 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-05-03T17:59:38.151 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-05-03T17:59:38.151 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-05-03T17:59:38.151 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-05-03T17:59:38.151 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-05-03T17:59:38.151 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-05-03T17:59:38.151 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-05-03T17:59:38.151 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-05-03T17:59:38.151 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-05-03T17:59:38.152 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-05-03T17:59:38.152 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-05-03T17:59:38.152 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-05-03T17:59:38.152 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-05-03T17:59:38.152 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-05-03T17:59:38.152 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-05-03T17:59:38.152 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-05-03T17:59:38.152 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-05-03T17:59:38.152 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-05-03T17:59:38.152 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-05-03T17:59:38.152 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-05-03T17:59:38.152 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-05-03T17:59:38.152 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-05-03T17:59:38.152 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-05-03T17:59:38.153 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-05-03T17:59:38.153 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-05-03T17:59:38.153 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-05-03T17:59:38.153 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-05-03T17:59:38.153 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-05-03T17:59:38.153 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-05-03T17:59:38.153 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-05-03T17:59:38.153 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-05-03T17:59:38.153 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-05-03T17:59:38.153 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-05-03T17:59:38.153 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-05-03T17:59:38.153 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-05-03T17:59:38.153 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-05-03T17:59:38.153 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-05-03T17:59:38.153 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-05-03T17:59:38.153 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-05-03T17:59:38.154 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-05-03T17:59:38.154 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-05-03T17:59:38.154 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-05-03T17:59:38.154 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-05-03T17:59:38.154 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-05-03T17:59:38.154 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-05-03T17:59:38.154 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-05-03T17:59:38.154 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-05-03T17:59:38.154 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-05-03T17:59:38.154 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-05-03T17:59:38.154 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-05-03T17:59:38.154 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-05-03T17:59:38.154 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-05-03T17:59:38.154 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-05-03T17:59:38.155 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-05-03T17:59:38.155 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-05-03T17:59:38.155 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-05-03T17:59:38.155 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-05-03T17:59:38.155 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-05-03T17:59:38.155 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-05-03T17:59:38.155 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-05-03T17:59:38.155 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-05-03T17:59:38.155 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-05-03T17:59:38.155 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-05-03T17:59:38.155 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-05-03T17:59:38.155 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-05-03T17:59:38.155 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-05-03T17:59:38.155 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-05-03T17:59:38.156 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-05-03T17:59:38.156 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-05-03T17:59:38.156 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-05-03T17:59:38.156 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-05-03T17:59:38.156 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-05-03T17:59:38.156 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-05-03T17:59:38.156 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-05-03T17:59:38.156 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-05-03T17:59:38.156 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-05-03T17:59:38.156 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-05-03T17:59:38.156 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-05-03T17:59:38.156 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-05-03T17:59:38.156 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-05-03T17:59:38.156 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-05-03T17:59:38.157 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-05-03T17:59:38.157 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-05-03T17:59:38.157 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-05-03T17:59:38.157 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-05-03T17:59:38.157 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-05-03T17:59:38.157 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-05-03T17:59:38.157 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-05-03T17:59:38.157 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-05-03T17:59:38.157 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-05-03T17:59:38.157 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-05-03T17:59:38.157 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-05-03T17:59:38.157 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-05-03T17:59:38.157 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-05-03T17:59:38.157 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-05-03T17:59:38.158 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-05-03T17:59:38.158 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-05-03T17:59:38.158 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-05-03T17:59:38.158 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-05-03T17:59:38.158 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-05-03T17:59:38.158 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-05-03T17:59:38.158 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-05-03T17:59:38.158 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-05-03T17:59:38.158 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-05-03T17:59:38.158 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-05-03T17:59:38.158 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-05-03T17:59:38.158 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-05-03T17:59:38.158 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-05-03T17:59:38.158 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-05-03T17:59:38.159 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-05-03T17:59:38.159 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-05-03T17:59:38.159 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-05-03T17:59:38.159 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-05-03T17:59:38.159 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-05-03T17:59:38.159 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-05-03T17:59:38.159 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-05-03T17:59:38.159 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-05-03T17:59:38.159 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-05-03T17:59:38.159 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-05-03T17:59:38.159 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-05-03T17:59:38.159 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-05-03T17:59:38.159 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-05-03T17:59:38.159 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-05-03T17:59:38.160 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-05-03T17:59:38.160 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-05-03T17:59:38.160 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-05-03T17:59:38.160 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-05-03T17:59:38.160 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-05-03T17:59:38.160 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-05-03T17:59:38.160 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-05-03T17:59:38.160 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-05-03T17:59:38.160 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-05-03T17:59:38.160 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-05-03T17:59:38.160 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-05-03T17:59:38.160 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-05-03T17:59:38.160 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-05-03T17:59:38.161 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-05-03T17:59:38.161 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-05-03T17:59:38.161 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-05-03T17:59:38.161 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-05-03T17:59:38.161 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-05-03T17:59:38.161 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-05-03T17:59:38.161 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-05-03T17:59:38.161 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-05-03T17:59:38.161 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-05-03T17:59:38.161 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-05-03T17:59:38.161 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-05-03T17:59:38.161 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-05-03T17:59:38.161 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-05-03T17:59:38.161 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-05-03T17:59:38.161 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-05-03T17:59:38.162 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-05-03T17:59:38.162 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-05-03T17:59:38.162 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-05-03T17:59:38.162 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-05-03T17:59:38.162 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-05-03T17:59:38.162 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-05-03T17:59:38.162 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-05-03T17:59:38.162 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-05-03T17:59:38.162 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-05-03T17:59:38.162 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-05-03T17:59:38.162 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-05-03T17:59:38.162 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-05-03T17:59:38.162 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-05-03T17:59:38.162 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-05-03T17:59:38.163 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-05-03T17:59:38.163 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-05-03T17:59:38.163 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-05-03T17:59:38.163 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-05-03T17:59:38.163 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-05-03T17:59:38.163 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-05-03T17:59:38.163 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-05-03T17:59:38.163 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-05-03T17:59:38.163 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-05-03T17:59:38.163 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-05-03T17:59:38.163 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-05-03T17:59:38.163 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-05-03T17:59:38.163 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-05-03T17:59:38.163 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-05-03T17:59:38.164 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-05-03T17:59:38.164 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-05-03T17:59:38.164 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-05-03T17:59:38.164 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-05-03T17:59:38.164 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-05-03T17:59:38.164 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-05-03T17:59:38.164 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-05-03T17:59:38.164 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-05-03T17:59:38.164 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-05-03T17:59:38.164 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-05-03T17:59:38.164 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-05-03T17:59:38.164 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-05-03T17:59:38.164 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-05-03T17:59:38.164 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-05-03T17:59:38.164 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-05-03T17:59:38.165 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-05-03T17:59:38.165 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-05-03T17:59:38.165 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-05-03T17:59:38.165 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-05-03T17:59:38.165 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-05-03T17:59:38.165 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-05-03T17:59:38.165 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-05-03T17:59:38.165 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-05-03T17:59:38.165 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-05-03T17:59:38.165 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-05-03T17:59:38.165 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-05-03T17:59:38.165 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-05-03T17:59:38.165 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-05-03T17:59:38.165 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-05-03T17:59:38.166 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-05-03T17:59:38.166 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-05-03T17:59:38.166 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-05-03T17:59:38.166 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-05-03T17:59:38.166 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-05-03T17:59:38.166 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-05-03T17:59:38.166 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-05-03T17:59:38.166 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-05-03T17:59:38.166 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-05-03T17:59:38.166 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-05-03T17:59:38.166 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-05-03T17:59:38.166 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-05-03T17:59:38.166 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-05-03T17:59:38.166 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-05-03T17:59:38.167 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-05-03T17:59:38.167 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-05-03T17:59:38.167 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-05-03T17:59:38.167 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-05-03T17:59:38.167 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-05-03T17:59:38.167 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-05-03T17:59:38.167 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-05-03T17:59:38.167 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-05-03T17:59:38.167 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-05-03T17:59:38.167 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-05-03T17:59:38.167 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-05-03T17:59:38.167 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-05-03T17:59:38.167 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-05-03T17:59:38.167 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-05-03T17:59:38.167 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-05-03T17:59:38.168 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-05-03T17:59:38.168 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-05-03T17:59:38.168 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-05-03T17:59:38.168 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-05-03T17:59:38.168 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-05-03T17:59:38.168 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-05-03T17:59:38.168 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-05-03T17:59:38.168 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-05-03T17:59:38.168 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-05-03T17:59:38.168 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-05-03T17:59:38.168 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-05-03T17:59:38.168 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-05-03T17:59:38.168 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-05-03T17:59:38.168 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-05-03T17:59:38.169 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-05-03T17:59:38.169 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-05-03T17:59:38.169 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-05-03T17:59:38.169 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-05-03T17:59:38.169 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-05-03T17:59:38.169 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-05-03T17:59:38.169 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-05-03T17:59:38.169 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-05-03T17:59:38.169 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-05-03T17:59:38.169 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-05-03T17:59:38.169 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-05-03T17:59:38.169 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-05-03T17:59:38.169 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-05-03T17:59:38.169 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-05-03T17:59:38.169 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-05-03T17:59:38.170 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-05-03T17:59:38.170 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-05-03T17:59:38.170 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-05-03T17:59:38.170 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-05-03T17:59:38.170 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-05-03T17:59:38.170 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-05-03T17:59:38.170 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-05-03T17:59:38.170 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-05-03T17:59:38.170 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-05-03T17:59:38.170 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-05-03T17:59:38.170 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-05-03T17:59:38.170 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-05-03T17:59:38.170 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-05-03T17:59:38.170 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-05-03T17:59:38.171 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-05-03T17:59:38.171 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-05-03T17:59:38.171 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-05-03T17:59:38.171 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-05-03T17:59:38.171 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-05-03T17:59:38.171 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-05-03T17:59:38.171 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-05-03T17:59:38.171 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-05-03T17:59:38.171 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-05-03T17:59:38.171 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-05-03T17:59:38.171 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-05-03T17:59:38.171 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-05-03T17:59:38.171 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-05-03T17:59:38.172 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-05-03T17:59:38.172 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-05-03T17:59:38.172 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-05-03T17:59:38.172 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-05-03T17:59:38.172 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-05-03T17:59:38.172 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-05-03T17:59:38.172 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-05-03T17:59:38.172 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-05-03T17:59:38.172 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-05-03T17:59:38.172 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-05-03T17:59:38.172 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-05-03T17:59:38.172 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-05-03T17:59:38.172 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-05-03T17:59:38.172 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-05-03T17:59:38.172 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-05-03T17:59:38.173 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-05-03T17:59:38.173 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-05-03T17:59:38.173 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-05-03T17:59:38.173 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-05-03T17:59:38.173 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-05-03T17:59:38.173 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-05-03T17:59:38.173 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-05-03T17:59:38.173 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-05-03T17:59:38.173 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-05-03T17:59:38.173 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-05-03T17:59:38.173 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-05-03T17:59:38.173 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-05-03T17:59:38.173 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-05-03T17:59:38.173 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-05-03T17:59:38.174 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-05-03T17:59:38.174 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-05-03T17:59:38.174 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-05-03T17:59:38.174 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-05-03T17:59:38.174 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-05-03T17:59:38.174 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-05-03T17:59:38.174 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-05-03T17:59:38.174 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-05-03T17:59:38.174 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-05-03T17:59:38.174 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-05-03T17:59:38.174 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-05-03T17:59:38.174 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-05-03T17:59:38.174 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-05-03T17:59:38.175 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-05-03T17:59:38.175 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-05-03T17:59:38.175 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-05-03T17:59:38.175 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-05-03T17:59:38.175 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-05-03T17:59:38.175 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-05-03T17:59:38.175 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-05-03T17:59:38.175 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-05-03T17:59:38.175 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-05-03T17:59:38.175 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-05-03T17:59:38.175 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-05-03T17:59:38.175 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-05-03T17:59:38.175 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-05-03T17:59:38.175 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-05-03T17:59:38.176 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-05-03T17:59:38.176 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-05-03T17:59:38.176 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-05-03T17:59:38.176 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-05-03T17:59:38.176 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-05-03T17:59:38.176 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-05-03T17:59:38.176 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-05-03T17:59:38.176 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-05-03T17:59:38.176 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-05-03T17:59:38.176 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-05-03T17:59:38.176 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-05-03T17:59:38.176 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-05-03T17:59:38.176 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-05-03T17:59:38.176 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-05-03T17:59:38.176 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-05-03T17:59:38.177 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-05-03T17:59:38.177 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-05-03T17:59:38.177 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-05-03T17:59:38.177 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-05-03T17:59:38.177 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-05-03T17:59:38.177 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-05-03T17:59:38.177 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-05-03T17:59:38.177 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-05-03T17:59:38.177 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-05-03T17:59:38.177 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-05-03T17:59:38.177 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-05-03T17:59:38.177 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-05-03T17:59:38.177 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-05-03T17:59:38.177 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-05-03T17:59:38.178 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-05-03T17:59:38.178 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-05-03T17:59:38.178 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-05-03T17:59:38.178 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-05-03T17:59:38.178 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-05-03T17:59:38.178 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-05-03T17:59:38.178 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-05-03T17:59:38.178 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-05-03T17:59:38.178 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-05-03T17:59:38.178 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-05-03T17:59:38.178 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-05-03T17:59:38.178 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-05-03T17:59:38.178 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-05-03T17:59:38.178 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-05-03T17:59:38.179 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-05-03T17:59:38.179 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-05-03T17:59:38.179 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-05-03T17:59:38.179 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-05-03T17:59:38.179 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-05-03T17:59:38.179 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-05-03T17:59:38.179 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-05-03T17:59:38.179 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-05-03T17:59:38.179 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-05-03T17:59:38.179 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-05-03T17:59:38.179 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-05-03T17:59:38.179 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-05-03T17:59:38.179 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-05-03T17:59:38.179 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-05-03T17:59:38.180 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-05-03T17:59:38.180 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-05-03T17:59:38.180 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-05-03T17:59:38.180 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-05-03T17:59:38.180 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-05-03T17:59:38.180 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-05-03T17:59:38.180 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-05-03T17:59:38.180 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-05-03T17:59:38.180 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-05-03T17:59:38.180 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-05-03T17:59:38.180 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-05-03T17:59:38.180 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-05-03T17:59:38.180 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-05-03T17:59:38.180 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-05-03T17:59:38.180 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-05-03T17:59:38.181 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-05-03T17:59:38.181 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-05-03T17:59:38.181 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-05-03T17:59:38.181 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-05-03T17:59:38.181 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-05-03T17:59:38.181 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-05-03T17:59:38.181 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-05-03T17:59:38.181 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-05-03T17:59:38.181 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-05-03T17:59:38.181 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-05-03T17:59:38.181 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-05-03T17:59:38.181 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-05-03T17:59:38.181 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-05-03T17:59:38.182 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-05-03T17:59:38.182 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-05-03T17:59:38.182 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-05-03T17:59:38.182 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-05-03T17:59:38.182 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-05-03T17:59:38.182 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-05-03T17:59:38.182 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-05-03T17:59:38.182 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-05-03T17:59:38.182 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-05-03T17:59:38.182 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-05-03T17:59:38.182 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-05-03T17:59:38.182 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-05-03T17:59:38.182 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-05-03T17:59:38.182 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-05-03T17:59:38.183 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-05-03T17:59:38.183 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-05-03T17:59:38.183 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-05-03T17:59:38.183 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-05-03T17:59:38.183 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-05-03T17:59:38.183 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-05-03T17:59:38.183 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-05-03T17:59:38.183 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-05-03T17:59:38.183 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-05-03T17:59:38.183 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-05-03T17:59:38.183 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-05-03T17:59:38.183 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-05-03T17:59:38.183 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-05-03T17:59:38.183 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-05-03T17:59:38.184 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-05-03T17:59:38.184 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-05-03T17:59:38.184 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-05-03T17:59:38.184 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-05-03T17:59:38.184 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-05-03T17:59:38.184 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-05-03T17:59:38.184 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-05-03T17:59:38.184 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-05-03T17:59:38.184 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-05-03T17:59:38.184 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-05-03T17:59:38.184 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-05-03T17:59:38.184 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-05-03T17:59:38.184 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-05-03T17:59:38.184 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-05-03T17:59:38.185 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-05-03T17:59:38.185 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-05-03T17:59:38.185 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-05-03T17:59:38.185 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-05-03T17:59:38.185 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-05-03T17:59:38.185 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-05-03T17:59:38.185 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-05-03T17:59:38.185 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-05-03T17:59:38.185 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-05-03T17:59:38.185 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-05-03T17:59:38.185 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-05-03T17:59:38.185 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-05-03T17:59:38.185 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-05-03T17:59:38.185 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-05-03T17:59:38.186 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-05-03T17:59:38.186 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-05-03T17:59:38.186 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-05-03T17:59:38.186 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-05-03T17:59:38.186 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-05-03T17:59:38.186 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-05-03T17:59:38.186 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-05-03T17:59:38.186 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-05-03T17:59:38.186 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-05-03T17:59:38.186 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-05-03T17:59:38.186 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-05-03T17:59:38.186 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-05-03T17:59:38.186 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-05-03T17:59:38.186 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-05-03T17:59:38.187 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-05-03T17:59:38.187 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-05-03T17:59:38.187 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-05-03T17:59:38.187 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-05-03T17:59:38.187 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-05-03T17:59:38.187 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-05-03T17:59:38.187 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-05-03T17:59:38.187 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-05-03T17:59:38.187 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-05-03T17:59:38.187 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-05-03T17:59:38.187 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-05-03T17:59:38.187 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-05-03T17:59:38.187 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-05-03T17:59:38.187 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-05-03T17:59:38.187 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-05-03T17:59:38.188 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-05-03T17:59:38.188 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-05-03T17:59:38.188 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-05-03T17:59:38.188 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-05-03T17:59:38.188 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-05-03T17:59:38.188 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-05-03T17:59:38.188 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-05-03T17:59:38.188 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-05-03T17:59:38.188 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-05-03T17:59:38.188 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-05-03T17:59:38.188 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-05-03T17:59:38.188 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-05-03T17:59:38.188 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-05-03T17:59:38.188 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-05-03T17:59:38.188 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-05-03T17:59:38.189 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-05-03T17:59:38.189 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-05-03T17:59:38.189 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-05-03T17:59:38.189 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-05-03T17:59:38.189 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-05-03T17:59:38.189 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-05-03T17:59:38.189 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-05-03T17:59:38.189 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-05-03T17:59:38.189 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-05-03T17:59:38.189 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-05-03T17:59:38.189 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-05-03T17:59:38.189 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-05-03T17:59:38.189 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-05-03T17:59:38.189 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-05-03T17:59:38.189 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-05-03T17:59:38.190 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-05-03T17:59:38.190 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-05-03T17:59:38.190 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-05-03T17:59:38.190 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-05-03T17:59:38.190 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-05-03T17:59:38.190 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-05-03T17:59:38.190 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-05-03T17:59:38.190 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-05-03T17:59:38.190 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-05-03T17:59:38.190 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-05-03T17:59:38.190 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-05-03T17:59:38.190 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-05-03T17:59:38.190 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-05-03T17:59:38.190 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-05-03T17:59:38.190 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-05-03T17:59:38.191 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-05-03T17:59:38.191 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-05-03T17:59:38.191 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-05-03T17:59:38.191 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-05-03T17:59:38.191 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-05-03T17:59:38.191 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-05-03T17:59:38.191 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-05-03T17:59:38.191 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-05-03T17:59:38.191 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-05-03T17:59:38.191 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-05-03T17:59:38.191 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-05-03T17:59:38.191 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-05-03T17:59:38.191 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-05-03T17:59:38.191 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-05-03T17:59:38.192 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-05-03T17:59:38.192 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-05-03T17:59:38.192 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-05-03T17:59:38.192 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-05-03T17:59:38.192 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-05-03T17:59:38.192 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-05-03T17:59:38.192 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-05-03T17:59:38.192 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-05-03T17:59:38.192 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-05-03T17:59:38.192 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-05-03T17:59:38.192 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-05-03T17:59:38.192 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-05-03T17:59:38.192 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-05-03T17:59:38.192 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-05-03T17:59:38.193 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-05-03T17:59:38.193 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-05-03T17:59:38.193 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-05-03T17:59:38.193 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-05-03T17:59:38.193 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-05-03T17:59:38.193 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-05-03T17:59:38.193 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-05-03T17:59:38.193 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-05-03T17:59:38.193 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-05-03T17:59:38.193 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-05-03T17:59:38.193 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-05-03T17:59:38.193 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-05-03T17:59:38.193 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-05-03T17:59:38.193 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-05-03T17:59:38.194 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-05-03T17:59:38.194 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-05-03T17:59:38.194 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-05-03T17:59:38.194 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-05-03T17:59:38.194 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-05-03T17:59:38.194 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-05-03T17:59:38.194 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-05-03T17:59:38.194 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-05-03T17:59:38.194 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-05-03T17:59:38.194 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-05-03T17:59:38.194 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-05-03T17:59:38.194 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-05-03T17:59:38.194 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-05-03T17:59:38.195 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-05-03T17:59:38.195 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-05-03T17:59:38.195 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-05-03T17:59:38.195 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-05-03T17:59:38.195 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-05-03T17:59:38.195 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-05-03T17:59:38.195 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-05-03T17:59:38.195 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-05-03T17:59:38.195 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-05-03T17:59:38.195 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-05-03T17:59:38.195 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-05-03T17:59:38.195 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-05-03T17:59:38.195 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-05-03T17:59:38.195 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-05-03T17:59:38.195 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-05-03T17:59:38.196 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-05-03T17:59:38.196 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-05-03T17:59:38.196 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-05-03T17:59:38.196 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-05-03T17:59:38.196 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-05-03T17:59:38.196 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-05-03T17:59:38.196 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-05-03T17:59:38.196 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-05-03T17:59:38.196 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-05-03T17:59:38.196 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-05-03T17:59:38.196 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-05-03T17:59:38.196 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-05-03T17:59:38.196 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-05-03T17:59:38.196 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-05-03T17:59:38.196 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-05-03T17:59:38.197 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-05-03T17:59:38.197 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-05-03T17:59:38.197 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-05-03T17:59:38.197 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-05-03T17:59:38.197 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-05-03T17:59:38.197 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-05-03T17:59:38.197 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-05-03T17:59:38.197 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-05-03T17:59:38.197 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-05-03T17:59:38.197 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-05-03T17:59:38.197 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-05-03T17:59:38.197 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-05-03T17:59:38.197 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-05-03T17:59:38.197 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-05-03T17:59:38.198 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-05-03T17:59:38.198 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-05-03T17:59:38.198 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-05-03T17:59:38.198 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-05-03T17:59:38.198 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-05-03T17:59:38.198 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-05-03T17:59:38.198 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-05-03T17:59:38.198 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-05-03T17:59:38.198 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-05-03T17:59:38.198 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-05-03T17:59:38.198 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-05-03T17:59:38.198 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-05-03T17:59:38.198 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-05-03T17:59:38.198 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-05-03T17:59:38.199 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-05-03T17:59:38.199 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-05-03T17:59:38.199 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-05-03T17:59:38.199 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-05-03T17:59:38.199 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-05-03T17:59:38.199 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-05-03T17:59:38.199 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-05-03T17:59:38.199 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-05-03T17:59:38.199 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-05-03T17:59:38.199 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-05-03T17:59:38.199 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-05-03T17:59:38.199 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-05-03T17:59:38.199 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-05-03T17:59:38.199 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-05-03T17:59:38.200 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-05-03T17:59:38.200 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-05-03T17:59:38.200 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-05-03T17:59:38.200 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-05-03T17:59:38.200 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-05-03T17:59:38.200 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-05-03T17:59:38.200 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-05-03T17:59:38.200 INFO:teuthology.orchestra.run.smithi150.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-05-03T17:59:38.204 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-05-03T17:59:38.204 DEBUG:teuthology.orchestra.run.smithi150:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-05-03T17:59:38.227 INFO:teuthology.orchestra.run.smithi150.stdout:postinstall scriptlet (using /bin/sh): 2024-05-03T17:59:38.227 INFO:teuthology.orchestra.run.smithi150.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-05-03T17:59:38.227 INFO:teuthology.orchestra.run.smithi150.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-05-03T17:59:38.227 INFO:teuthology.orchestra.run.smithi150.stdout:fi 2024-05-03T17:59:38.227 INFO:teuthology.orchestra.run.smithi150.stdout:for file in vmlinuz System.map config; do 2024-05-03T17:59:38.227 INFO:teuthology.orchestra.run.smithi150.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-05-03T17:59:38.227 INFO:teuthology.orchestra.run.smithi150.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-05-03T17:59:38.227 INFO:teuthology.orchestra.run.smithi150.stdout: fi 2024-05-03T17:59:38.227 INFO:teuthology.orchestra.run.smithi150.stdout:done 2024-05-03T17:59:38.227 INFO:teuthology.orchestra.run.smithi150.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-05-03T17:59:38.227 INFO:teuthology.orchestra.run.smithi150.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-05-03T17:59:38.228 INFO:teuthology.orchestra.run.smithi150.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-05-03T17:59:38.228 INFO:teuthology.orchestra.run.smithi150.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-05-03T17:59:38.228 INFO:teuthology.orchestra.run.smithi150.stdout:fi 2024-05-03T17:59:38.228 INFO:teuthology.orchestra.run.smithi150.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-05-03T17:59:38.228 INFO:teuthology.orchestra.run.smithi150.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-05-03T17:59:38.228 INFO:teuthology.orchestra.run.smithi150.stdout:fi 2024-05-03T17:59:38.228 INFO:teuthology.orchestra.run.smithi150.stdout:preuninstall scriptlet (using /bin/sh): 2024-05-03T17:59:38.228 INFO:teuthology.orchestra.run.smithi150.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-05-03T17:59:38.228 INFO:teuthology.orchestra.run.smithi150.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-05-03T17:59:38.228 INFO:teuthology.orchestra.run.smithi150.stdout:fi 2024-05-03T17:59:38.229 DEBUG:teuthology.orchestra.run.smithi150:> sudo rpm -qi grub2-tools 2024-05-03T17:59:38.304 INFO:teuthology.orchestra.run.smithi150.stdout:Name : grub2-tools 2024-05-03T17:59:38.304 INFO:teuthology.orchestra.run.smithi150.stdout:Epoch : 1 2024-05-03T17:59:38.304 INFO:teuthology.orchestra.run.smithi150.stdout:Version : 2.06 2024-05-03T17:59:38.304 INFO:teuthology.orchestra.run.smithi150.stdout:Release : 68.el9 2024-05-03T17:59:38.304 INFO:teuthology.orchestra.run.smithi150.stdout:Architecture: x86_64 2024-05-03T17:59:38.304 INFO:teuthology.orchestra.run.smithi150.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-03T17:59:38.304 INFO:teuthology.orchestra.run.smithi150.stdout:Group : Unspecified 2024-05-03T17:59:38.304 INFO:teuthology.orchestra.run.smithi150.stdout:Size : 8264254 2024-05-03T17:59:38.304 INFO:teuthology.orchestra.run.smithi150.stdout:License : GPLv3+ 2024-05-03T17:59:38.304 INFO:teuthology.orchestra.run.smithi150.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-03T17:59:38.305 INFO:teuthology.orchestra.run.smithi150.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-03T17:59:38.305 INFO:teuthology.orchestra.run.smithi150.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-03T17:59:38.305 INFO:teuthology.orchestra.run.smithi150.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-03T17:59:38.305 INFO:teuthology.orchestra.run.smithi150.stdout:Packager : builder@centos.org 2024-05-03T17:59:38.305 INFO:teuthology.orchestra.run.smithi150.stdout:Vendor : CentOS 2024-05-03T17:59:38.305 INFO:teuthology.orchestra.run.smithi150.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-03T17:59:38.305 INFO:teuthology.orchestra.run.smithi150.stdout:Summary : Support tools for GRUB. 2024-05-03T17:59:38.305 INFO:teuthology.orchestra.run.smithi150.stdout:Description : 2024-05-03T17:59:38.305 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-03T17:59:38.305 INFO:teuthology.orchestra.run.smithi150.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-03T17:59:38.305 INFO:teuthology.orchestra.run.smithi150.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-03T17:59:38.305 INFO:teuthology.orchestra.run.smithi150.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-03T17:59:38.305 INFO:teuthology.orchestra.run.smithi150.stdout:hardware devices. 2024-05-03T17:59:38.305 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-03T17:59:38.305 INFO:teuthology.orchestra.run.smithi150.stdout:This subpackage provides tools for support of all platforms. 2024-05-03T17:59:38.307 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-03T17:59:38.307 INFO:teuthology.task.kernel:Updating grub on smithi150 to boot 6.9.0-rc3-g806ec36b2112 2024-05-03T17:59:38.307 DEBUG:teuthology.orchestra.run.smithi150:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-03T17:59:38.885 INFO:teuthology.orchestra.run.smithi150.stderr:Generating grub configuration file ... 2024-05-03T17:59:40.198 INFO:teuthology.orchestra.run.smithi150.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-03T17:59:40.227 INFO:teuthology.orchestra.run.smithi150.stderr:done 2024-05-03T17:59:40.229 DEBUG:teuthology.orchestra.run.smithi150:> mktemp 2024-05-03T17:59:40.246 INFO:teuthology.orchestra.run.smithi150.stdout:/tmp/tmp.hqyQl5diUW 2024-05-03T17:59:40.246 DEBUG:teuthology.orchestra.run.smithi150:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.hqyQl5diUW 2024-05-03T17:59:40.323 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod 0666 /tmp/tmp.hqyQl5diUW 2024-05-03T17:59:40.456 DEBUG:teuthology.orchestra.remote:smithi150:/tmp/tmp.hqyQl5diUW is 6KB 2024-05-03T17:59:40.476 DEBUG:teuthology.orchestra.run.smithi150:> rm -fr /tmp/tmp.hqyQl5diUW 2024-05-03T17:59:40.490 DEBUG:teuthology.orchestra.run.smithi150:> sudo /bin/ls /boot/loader/entries || true 2024-05-03T17:59:40.561 INFO:teuthology.orchestra.run.smithi150.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-03T17:59:40.562 INFO:teuthology.orchestra.run.smithi150.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-03T17:59:40.562 INFO:teuthology.orchestra.run.smithi150.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-03T17:59:40.562 INFO:teuthology.orchestra.run.smithi150.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-05-03T17:59:40.563 DEBUG:teuthology.orchestra.run.smithi150:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-05-03T17:59:40.724 DEBUG:teuthology.orchestra.run.smithi150:> sudo shutdown -r now 2024-05-03T17:59:41.742 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi099.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-05-03T17:59:41.742 DEBUG:teuthology.orchestra.run.smithi099:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-05-03T17:59:41.824 INFO:teuthology.orchestra.run.smithi099.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-05-03T17:59:41.824 INFO:teuthology.orchestra.run.smithi099.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-05-03T17:59:41.824 INFO:teuthology.orchestra.run.smithi099.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-05-03T17:59:41.825 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-05-03T17:59:41.825 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-05-03T17:59:41.825 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-05-03T17:59:41.825 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-05-03T17:59:41.825 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-05-03T17:59:41.825 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-05-03T17:59:41.825 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-05-03T17:59:41.825 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-05-03T17:59:41.825 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-05-03T17:59:41.825 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-05-03T17:59:41.825 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-05-03T17:59:41.825 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-05-03T17:59:41.825 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-05-03T17:59:41.826 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-05-03T17:59:41.826 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-05-03T17:59:41.826 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-05-03T17:59:41.826 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-05-03T17:59:41.826 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-05-03T17:59:41.826 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-05-03T17:59:41.826 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-05-03T17:59:41.826 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-05-03T17:59:41.826 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-05-03T17:59:41.826 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-05-03T17:59:41.827 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-05-03T17:59:41.827 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-05-03T17:59:41.827 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-05-03T17:59:41.827 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-05-03T17:59:41.827 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-05-03T17:59:41.827 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-05-03T17:59:41.827 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-05-03T17:59:41.827 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-05-03T17:59:41.827 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-05-03T17:59:41.827 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-05-03T17:59:41.827 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-05-03T17:59:41.827 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-05-03T17:59:41.827 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-05-03T17:59:41.828 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-05-03T17:59:41.828 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-05-03T17:59:41.828 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-05-03T17:59:41.828 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-05-03T17:59:41.828 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-05-03T17:59:41.828 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-05-03T17:59:41.828 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-05-03T17:59:41.828 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-05-03T17:59:41.828 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-05-03T17:59:41.828 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-05-03T17:59:41.828 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-05-03T17:59:41.828 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-05-03T17:59:41.828 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-05-03T17:59:41.829 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-05-03T17:59:41.829 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-05-03T17:59:41.829 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-05-03T17:59:41.829 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-05-03T17:59:41.829 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-05-03T17:59:41.829 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-05-03T17:59:41.829 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-05-03T17:59:41.829 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-05-03T17:59:41.829 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-05-03T17:59:41.829 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-05-03T17:59:41.829 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-05-03T17:59:41.829 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-05-03T17:59:41.829 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-05-03T17:59:41.829 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-05-03T17:59:41.830 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-05-03T17:59:41.830 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-05-03T17:59:41.830 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-05-03T17:59:41.830 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-05-03T17:59:41.830 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-05-03T17:59:41.830 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-05-03T17:59:41.830 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-05-03T17:59:41.830 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-05-03T17:59:41.830 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-05-03T17:59:41.830 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-05-03T17:59:41.830 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-05-03T17:59:41.830 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-05-03T17:59:41.830 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-05-03T17:59:41.831 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-05-03T17:59:41.831 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-05-03T17:59:41.831 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-05-03T17:59:41.831 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-05-03T17:59:41.831 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-05-03T17:59:41.831 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-05-03T17:59:41.831 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-05-03T17:59:41.831 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-05-03T17:59:41.831 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-05-03T17:59:41.831 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-05-03T17:59:41.831 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-05-03T17:59:41.832 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-05-03T17:59:41.832 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-05-03T17:59:41.832 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-05-03T17:59:41.832 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-05-03T17:59:41.832 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-05-03T17:59:41.832 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-05-03T17:59:41.832 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-05-03T17:59:41.832 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-05-03T17:59:41.832 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-05-03T17:59:41.832 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-05-03T17:59:41.832 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-05-03T17:59:41.832 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-05-03T17:59:41.833 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-05-03T17:59:41.833 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-05-03T17:59:41.833 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-05-03T17:59:41.833 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-05-03T17:59:41.833 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-05-03T17:59:41.833 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-05-03T17:59:41.833 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-05-03T17:59:41.833 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-05-03T17:59:41.833 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-05-03T17:59:41.833 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-05-03T17:59:41.833 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-05-03T17:59:41.833 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-05-03T17:59:41.833 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-05-03T17:59:41.834 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-05-03T17:59:41.834 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-05-03T17:59:41.834 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-05-03T17:59:41.834 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-05-03T17:59:41.834 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-05-03T17:59:41.834 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-05-03T17:59:41.834 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-05-03T17:59:41.834 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-05-03T17:59:41.834 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-05-03T17:59:41.834 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-05-03T17:59:41.834 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-05-03T17:59:41.834 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-05-03T17:59:41.835 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-05-03T17:59:41.835 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-05-03T17:59:41.835 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-05-03T17:59:41.835 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-05-03T17:59:41.835 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-05-03T17:59:41.835 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-05-03T17:59:41.835 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-05-03T17:59:41.835 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-05-03T17:59:41.835 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-05-03T17:59:41.835 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-05-03T17:59:41.835 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-05-03T17:59:41.835 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-05-03T17:59:41.835 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-05-03T17:59:41.835 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-05-03T17:59:41.835 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-05-03T17:59:41.836 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-05-03T17:59:41.836 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-05-03T17:59:41.836 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-05-03T17:59:41.836 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-05-03T17:59:41.836 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-05-03T17:59:41.836 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-05-03T17:59:41.836 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-05-03T17:59:41.836 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-05-03T17:59:41.836 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-05-03T17:59:41.836 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-05-03T17:59:41.836 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-05-03T17:59:41.836 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-05-03T17:59:41.836 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-05-03T17:59:41.837 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-05-03T17:59:41.837 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-05-03T17:59:41.837 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-05-03T17:59:41.837 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-05-03T17:59:41.837 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-05-03T17:59:41.837 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-05-03T17:59:41.837 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-05-03T17:59:41.837 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-05-03T17:59:41.837 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-05-03T17:59:41.837 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-05-03T17:59:41.837 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-05-03T17:59:41.837 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-05-03T17:59:41.837 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-05-03T17:59:41.837 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-05-03T17:59:41.838 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-05-03T17:59:41.838 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-05-03T17:59:41.838 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-05-03T17:59:41.838 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-05-03T17:59:41.838 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-05-03T17:59:41.838 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-05-03T17:59:41.838 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-05-03T17:59:41.838 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-05-03T17:59:41.838 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-05-03T17:59:41.838 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-05-03T17:59:41.838 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-05-03T17:59:41.838 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-05-03T17:59:41.838 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-05-03T17:59:41.838 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-05-03T17:59:41.839 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-05-03T17:59:41.839 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-05-03T17:59:41.839 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-05-03T17:59:41.839 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-05-03T17:59:41.839 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-05-03T17:59:41.839 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-05-03T17:59:41.839 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-05-03T17:59:41.839 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-05-03T17:59:41.839 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-05-03T17:59:41.839 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-05-03T17:59:41.839 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-05-03T17:59:41.839 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-05-03T17:59:41.839 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-05-03T17:59:41.839 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-05-03T17:59:41.840 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-05-03T17:59:41.840 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-05-03T17:59:41.840 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-05-03T17:59:41.840 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-05-03T17:59:41.840 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-05-03T17:59:41.840 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-05-03T17:59:41.840 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-05-03T17:59:41.840 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-05-03T17:59:41.840 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-05-03T17:59:41.840 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-05-03T17:59:41.840 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-05-03T17:59:41.840 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-05-03T17:59:41.840 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-05-03T17:59:41.841 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-05-03T17:59:41.841 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-05-03T17:59:41.841 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-05-03T17:59:41.841 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-05-03T17:59:41.841 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-05-03T17:59:41.841 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-05-03T17:59:41.841 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-05-03T17:59:41.841 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-05-03T17:59:41.841 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-05-03T17:59:41.841 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-05-03T17:59:41.841 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-05-03T17:59:41.841 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-05-03T17:59:41.841 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-05-03T17:59:41.841 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-05-03T17:59:41.841 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-05-03T17:59:41.842 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-05-03T17:59:41.842 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-05-03T17:59:41.842 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-05-03T17:59:41.842 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-05-03T17:59:41.842 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-05-03T17:59:41.842 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-05-03T17:59:41.842 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-05-03T17:59:41.842 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-05-03T17:59:41.842 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-05-03T17:59:41.842 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-05-03T17:59:41.842 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-05-03T17:59:41.842 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-05-03T17:59:41.842 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-05-03T17:59:41.843 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-05-03T17:59:41.843 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-05-03T17:59:41.843 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-05-03T17:59:41.843 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-05-03T17:59:41.843 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-05-03T17:59:41.843 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-05-03T17:59:41.843 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-05-03T17:59:41.843 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-05-03T17:59:41.843 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-05-03T17:59:41.843 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-05-03T17:59:41.843 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-05-03T17:59:41.843 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-05-03T17:59:41.843 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-05-03T17:59:41.843 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-05-03T17:59:41.843 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-05-03T17:59:41.844 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-05-03T17:59:41.844 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-05-03T17:59:41.844 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-05-03T17:59:41.844 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-05-03T17:59:41.844 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-05-03T17:59:41.844 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-05-03T17:59:41.844 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-05-03T17:59:41.844 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-05-03T17:59:41.844 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-05-03T17:59:41.844 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-05-03T17:59:41.844 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-05-03T17:59:41.844 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-05-03T17:59:41.844 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-05-03T17:59:41.844 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-05-03T17:59:41.845 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-05-03T17:59:41.845 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-05-03T17:59:41.845 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-05-03T17:59:41.845 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-05-03T17:59:41.845 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-05-03T17:59:41.845 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-05-03T17:59:41.845 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-05-03T17:59:41.845 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-05-03T17:59:41.845 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-05-03T17:59:41.845 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-05-03T17:59:41.845 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-05-03T17:59:41.845 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-05-03T17:59:41.845 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-05-03T17:59:41.845 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-05-03T17:59:41.845 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-05-03T17:59:41.845 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-05-03T17:59:41.846 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-05-03T17:59:41.846 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-05-03T17:59:41.846 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-05-03T17:59:41.846 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-05-03T17:59:41.846 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-05-03T17:59:41.846 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-05-03T17:59:41.846 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-05-03T17:59:41.846 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-05-03T17:59:41.846 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-05-03T17:59:41.846 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-05-03T17:59:41.846 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-05-03T17:59:41.846 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-05-03T17:59:41.846 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-05-03T17:59:41.846 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-05-03T17:59:41.847 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-05-03T17:59:41.847 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-05-03T17:59:41.847 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-05-03T17:59:41.847 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-05-03T17:59:41.847 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-05-03T17:59:41.847 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-05-03T17:59:41.847 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-05-03T17:59:41.847 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-05-03T17:59:41.847 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-05-03T17:59:41.847 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-05-03T17:59:41.847 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-05-03T17:59:41.847 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-05-03T17:59:41.847 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-05-03T17:59:41.847 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-05-03T17:59:41.848 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-05-03T17:59:41.848 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-05-03T17:59:41.848 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-05-03T17:59:41.848 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-05-03T17:59:41.848 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-05-03T17:59:41.848 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-05-03T17:59:41.848 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-05-03T17:59:41.848 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-05-03T17:59:41.848 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-05-03T17:59:41.848 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-05-03T17:59:41.848 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-05-03T17:59:41.848 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-05-03T17:59:41.849 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-05-03T17:59:41.849 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-05-03T17:59:41.849 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-05-03T17:59:41.849 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-05-03T17:59:41.849 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-05-03T17:59:41.849 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-05-03T17:59:41.849 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-05-03T17:59:41.849 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-05-03T17:59:41.849 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-05-03T17:59:41.849 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-05-03T17:59:41.849 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-05-03T17:59:41.849 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-05-03T17:59:41.849 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-05-03T17:59:41.849 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-05-03T17:59:41.850 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-05-03T17:59:41.850 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-05-03T17:59:41.850 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-05-03T17:59:41.850 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-05-03T17:59:41.850 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-05-03T17:59:41.850 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-05-03T17:59:41.850 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-05-03T17:59:41.850 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-05-03T17:59:41.850 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-05-03T17:59:41.850 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-05-03T17:59:41.850 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-05-03T17:59:41.850 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-05-03T17:59:41.851 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-05-03T17:59:41.851 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-05-03T17:59:41.851 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-05-03T17:59:41.851 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-05-03T17:59:41.851 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-05-03T17:59:41.851 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-05-03T17:59:41.851 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-05-03T17:59:41.851 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-05-03T17:59:41.851 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-05-03T17:59:41.851 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-05-03T17:59:41.851 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-05-03T17:59:41.851 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-05-03T17:59:41.851 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-05-03T17:59:41.851 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-05-03T17:59:41.851 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-05-03T17:59:41.852 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-05-03T17:59:41.852 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-05-03T17:59:41.852 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-05-03T17:59:41.852 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-05-03T17:59:41.852 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-05-03T17:59:41.852 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-05-03T17:59:41.852 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-05-03T17:59:41.852 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-05-03T17:59:41.852 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-05-03T17:59:41.852 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-05-03T17:59:41.852 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-05-03T17:59:41.852 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-05-03T17:59:41.852 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-05-03T17:59:41.852 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-05-03T17:59:41.852 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-05-03T17:59:41.852 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-05-03T17:59:41.853 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-05-03T17:59:41.853 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-05-03T17:59:41.853 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-05-03T17:59:41.853 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-05-03T17:59:41.853 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-05-03T17:59:41.853 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-05-03T17:59:41.853 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-05-03T17:59:41.853 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-05-03T17:59:41.853 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-05-03T17:59:41.853 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-05-03T17:59:41.853 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-05-03T17:59:41.853 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-05-03T17:59:41.853 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-05-03T17:59:41.853 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-05-03T17:59:41.853 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-05-03T17:59:41.854 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-05-03T17:59:41.854 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-05-03T17:59:41.854 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-05-03T17:59:41.854 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-05-03T17:59:41.854 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-05-03T17:59:41.854 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-05-03T17:59:41.854 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-05-03T17:59:41.854 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-05-03T17:59:41.854 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-05-03T17:59:41.854 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-05-03T17:59:41.854 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-05-03T17:59:41.854 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-05-03T17:59:41.854 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-05-03T17:59:41.854 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-05-03T17:59:41.855 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-05-03T17:59:41.855 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-05-03T17:59:41.855 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-05-03T17:59:41.855 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-05-03T17:59:41.855 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-05-03T17:59:41.855 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-05-03T17:59:41.855 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-05-03T17:59:41.855 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-05-03T17:59:41.855 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-05-03T17:59:41.855 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-05-03T17:59:41.855 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-05-03T17:59:41.855 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-05-03T17:59:41.855 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-05-03T17:59:41.855 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-05-03T17:59:41.855 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-05-03T17:59:41.855 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-05-03T17:59:41.856 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-05-03T17:59:41.856 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-05-03T17:59:41.856 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-05-03T17:59:41.856 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-05-03T17:59:41.856 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-05-03T17:59:41.856 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-05-03T17:59:41.856 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-05-03T17:59:41.856 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-05-03T17:59:41.856 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-05-03T17:59:41.856 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-05-03T17:59:41.856 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-05-03T17:59:41.856 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-05-03T17:59:41.856 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-05-03T17:59:41.857 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-05-03T17:59:41.857 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-05-03T17:59:41.857 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-05-03T17:59:41.857 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-05-03T17:59:41.857 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-05-03T17:59:41.857 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-05-03T17:59:41.857 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-05-03T17:59:41.857 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-05-03T17:59:41.857 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-05-03T17:59:41.857 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-05-03T17:59:41.857 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-05-03T17:59:41.857 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-05-03T17:59:41.857 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-05-03T17:59:41.857 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-05-03T17:59:41.858 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-05-03T17:59:41.858 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-05-03T17:59:41.858 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-05-03T17:59:41.858 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-05-03T17:59:41.858 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-05-03T17:59:41.858 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-05-03T17:59:41.858 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-05-03T17:59:41.858 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-05-03T17:59:41.858 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-05-03T17:59:41.858 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-05-03T17:59:41.859 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-05-03T17:59:41.859 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-05-03T17:59:41.859 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-05-03T17:59:41.860 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-05-03T17:59:41.860 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-05-03T17:59:41.860 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-05-03T17:59:41.860 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-05-03T17:59:41.860 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-05-03T17:59:41.860 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-05-03T17:59:41.860 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-05-03T17:59:41.860 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-05-03T17:59:41.860 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-05-03T17:59:41.860 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-05-03T17:59:41.860 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-05-03T17:59:41.860 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-05-03T17:59:41.860 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-05-03T17:59:41.861 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-05-03T17:59:41.861 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-05-03T17:59:41.861 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-05-03T17:59:41.861 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-05-03T17:59:41.861 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-05-03T17:59:41.861 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-05-03T17:59:41.861 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-05-03T17:59:41.861 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-05-03T17:59:41.861 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-05-03T17:59:41.861 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-05-03T17:59:41.861 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-05-03T17:59:41.861 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-05-03T17:59:41.861 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-05-03T17:59:41.862 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-05-03T17:59:41.862 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-05-03T17:59:41.862 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-05-03T17:59:41.862 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-05-03T17:59:41.862 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-05-03T17:59:41.862 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-05-03T17:59:41.862 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-05-03T17:59:41.862 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-05-03T17:59:41.862 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-05-03T17:59:41.862 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-05-03T17:59:41.862 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-05-03T17:59:41.862 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-05-03T17:59:41.862 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-05-03T17:59:41.863 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-05-03T17:59:41.863 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-05-03T17:59:41.863 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-05-03T17:59:41.863 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-05-03T17:59:41.863 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-05-03T17:59:41.863 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-05-03T17:59:41.864 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-05-03T17:59:41.864 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-05-03T17:59:41.864 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-05-03T17:59:41.864 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-05-03T17:59:41.864 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-05-03T17:59:41.864 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-05-03T17:59:41.864 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-05-03T17:59:41.864 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-05-03T17:59:41.864 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-05-03T17:59:41.864 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-05-03T17:59:41.864 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-05-03T17:59:41.864 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-05-03T17:59:41.864 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-05-03T17:59:41.864 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-05-03T17:59:41.864 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-05-03T17:59:41.865 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-05-03T17:59:41.865 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-05-03T17:59:41.865 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-05-03T17:59:41.865 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-05-03T17:59:41.865 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-05-03T17:59:41.865 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-05-03T17:59:41.865 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-05-03T17:59:41.865 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-05-03T17:59:41.865 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-05-03T17:59:41.865 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-05-03T17:59:41.868 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-05-03T17:59:41.868 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-05-03T17:59:41.868 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-05-03T17:59:41.868 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-05-03T17:59:41.868 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-05-03T17:59:41.868 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-05-03T17:59:41.868 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-05-03T17:59:41.868 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-05-03T17:59:41.868 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-05-03T17:59:41.868 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-05-03T17:59:41.868 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-05-03T17:59:41.868 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-05-03T17:59:41.868 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-05-03T17:59:41.869 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-05-03T17:59:41.869 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-05-03T17:59:41.869 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-05-03T17:59:41.869 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-05-03T17:59:41.869 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-05-03T17:59:41.869 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-05-03T17:59:41.869 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-05-03T17:59:41.869 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-05-03T17:59:41.869 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-05-03T17:59:41.869 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-05-03T17:59:41.869 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-05-03T17:59:41.869 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-05-03T17:59:41.869 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-05-03T17:59:41.869 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-05-03T17:59:41.870 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-05-03T17:59:41.870 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-05-03T17:59:41.870 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-05-03T17:59:41.870 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-05-03T17:59:41.870 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-05-03T17:59:41.870 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-05-03T17:59:41.870 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-05-03T17:59:41.870 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-05-03T17:59:41.870 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-05-03T17:59:41.870 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-05-03T17:59:41.870 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-05-03T17:59:41.870 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-05-03T17:59:41.870 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-05-03T17:59:41.871 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-05-03T17:59:41.871 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-05-03T17:59:41.871 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-05-03T17:59:41.871 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-05-03T17:59:41.871 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-05-03T17:59:41.871 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-05-03T17:59:41.871 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-05-03T17:59:41.871 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-05-03T17:59:41.871 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-05-03T17:59:41.871 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-05-03T17:59:41.871 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-05-03T17:59:41.871 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-05-03T17:59:41.871 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-05-03T17:59:41.872 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-05-03T17:59:41.872 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-05-03T17:59:41.872 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-05-03T17:59:41.872 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-05-03T17:59:41.872 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-05-03T17:59:41.872 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-05-03T17:59:41.872 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-05-03T17:59:41.872 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-05-03T17:59:41.872 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-05-03T17:59:41.872 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-05-03T17:59:41.872 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-05-03T17:59:41.872 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-05-03T17:59:41.872 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-05-03T17:59:41.872 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-05-03T17:59:41.873 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-05-03T17:59:41.873 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-05-03T17:59:41.873 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-05-03T17:59:41.873 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-05-03T17:59:41.873 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-05-03T17:59:41.873 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-05-03T17:59:41.873 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-05-03T17:59:41.873 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-05-03T17:59:41.873 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-05-03T17:59:41.873 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-05-03T17:59:41.873 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-05-03T17:59:41.873 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-05-03T17:59:41.873 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-05-03T17:59:41.873 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-05-03T17:59:41.874 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-05-03T17:59:41.874 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-05-03T17:59:41.874 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-05-03T17:59:41.874 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-05-03T17:59:41.874 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-05-03T17:59:41.874 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-05-03T17:59:41.874 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-05-03T17:59:41.874 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-05-03T17:59:41.874 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-05-03T17:59:41.874 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-05-03T17:59:41.874 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-05-03T17:59:41.874 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-05-03T17:59:41.874 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-05-03T17:59:41.874 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-05-03T17:59:41.874 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-05-03T17:59:41.875 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-05-03T17:59:41.875 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-05-03T17:59:41.875 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-05-03T17:59:41.875 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-05-03T17:59:41.875 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-05-03T17:59:41.875 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-05-03T17:59:41.875 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-05-03T17:59:41.875 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-05-03T17:59:41.875 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-05-03T17:59:41.875 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-05-03T17:59:41.875 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-05-03T17:59:41.875 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-05-03T17:59:41.875 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-05-03T17:59:41.875 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-05-03T17:59:41.876 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-05-03T17:59:41.876 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-05-03T17:59:41.876 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-05-03T17:59:41.876 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-05-03T17:59:41.876 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-05-03T17:59:41.876 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-05-03T17:59:41.876 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-05-03T17:59:41.876 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-05-03T17:59:41.876 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-05-03T17:59:41.876 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-05-03T17:59:41.876 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-05-03T17:59:41.876 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-05-03T17:59:41.876 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-05-03T17:59:41.877 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-05-03T17:59:41.877 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-05-03T17:59:41.877 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-05-03T17:59:41.877 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-05-03T17:59:41.877 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-05-03T17:59:41.877 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-05-03T17:59:41.877 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-05-03T17:59:41.877 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-05-03T17:59:41.877 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-05-03T17:59:41.877 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-05-03T17:59:41.877 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-05-03T17:59:41.877 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-05-03T17:59:41.877 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-05-03T17:59:41.877 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-05-03T17:59:41.877 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-05-03T17:59:41.878 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-05-03T17:59:41.878 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-05-03T17:59:41.878 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-05-03T17:59:41.878 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-05-03T17:59:41.878 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-05-03T17:59:41.878 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-05-03T17:59:41.878 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-05-03T17:59:41.878 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-05-03T17:59:41.878 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-05-03T17:59:41.878 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-05-03T17:59:41.878 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-05-03T17:59:41.878 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-05-03T17:59:41.878 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-05-03T17:59:41.878 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-05-03T17:59:41.879 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-05-03T17:59:41.879 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-05-03T17:59:41.879 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-05-03T17:59:41.879 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-05-03T17:59:41.879 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-05-03T17:59:41.879 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-05-03T17:59:41.879 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-05-03T17:59:41.879 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-05-03T17:59:41.879 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-05-03T17:59:41.879 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-05-03T17:59:41.879 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-05-03T17:59:41.879 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-05-03T17:59:41.879 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-05-03T17:59:41.879 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-05-03T17:59:41.880 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-05-03T17:59:41.880 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-05-03T17:59:41.880 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-05-03T17:59:41.880 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-05-03T17:59:41.880 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-05-03T17:59:41.880 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-05-03T17:59:41.880 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-05-03T17:59:41.880 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-05-03T17:59:41.880 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-05-03T17:59:41.880 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-05-03T17:59:41.880 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-05-03T17:59:41.880 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-05-03T17:59:41.880 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-05-03T17:59:41.880 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-05-03T17:59:41.881 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-05-03T17:59:41.881 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-05-03T17:59:41.881 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-05-03T17:59:41.881 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-05-03T17:59:41.881 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-05-03T17:59:41.881 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-05-03T17:59:41.881 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-05-03T17:59:41.881 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-05-03T17:59:41.881 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-05-03T17:59:41.881 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-05-03T17:59:41.881 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-05-03T17:59:41.881 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-05-03T17:59:41.881 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-05-03T17:59:41.881 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-05-03T17:59:41.882 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-05-03T17:59:41.882 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-05-03T17:59:41.882 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-05-03T17:59:41.882 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-05-03T17:59:41.882 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-05-03T17:59:41.882 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-05-03T17:59:41.882 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-05-03T17:59:41.882 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-05-03T17:59:41.882 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-05-03T17:59:41.882 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-05-03T17:59:41.882 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-05-03T17:59:41.882 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-05-03T17:59:41.882 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-05-03T17:59:41.882 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-05-03T17:59:41.882 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-05-03T17:59:41.883 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-05-03T17:59:41.883 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-05-03T17:59:41.883 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-05-03T17:59:41.883 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-05-03T17:59:41.883 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-05-03T17:59:41.883 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-05-03T17:59:41.883 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-05-03T17:59:41.883 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-05-03T17:59:41.883 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-05-03T17:59:41.883 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-05-03T17:59:41.883 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-05-03T17:59:41.883 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-05-03T17:59:41.883 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-05-03T17:59:41.883 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-05-03T17:59:41.883 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-05-03T17:59:41.884 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-05-03T17:59:41.884 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-05-03T17:59:41.884 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-05-03T17:59:41.884 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-05-03T17:59:41.884 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-05-03T17:59:41.884 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-05-03T17:59:41.884 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-05-03T17:59:41.884 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-05-03T17:59:41.884 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-05-03T17:59:41.884 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-05-03T17:59:41.884 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-05-03T17:59:41.884 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-05-03T17:59:41.884 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-05-03T17:59:41.884 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-05-03T17:59:41.884 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-05-03T17:59:41.885 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-05-03T17:59:41.885 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-05-03T17:59:41.885 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-05-03T17:59:41.885 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-05-03T17:59:41.885 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-05-03T17:59:41.885 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-05-03T17:59:41.885 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-05-03T17:59:41.885 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-05-03T17:59:41.885 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-05-03T17:59:41.885 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-05-03T17:59:41.885 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-05-03T17:59:41.885 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-05-03T17:59:41.885 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-05-03T17:59:41.885 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-05-03T17:59:41.885 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-05-03T17:59:41.886 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-05-03T17:59:41.886 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-05-03T17:59:41.886 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-05-03T17:59:41.886 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-05-03T17:59:41.886 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-05-03T17:59:41.886 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-05-03T17:59:41.886 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-05-03T17:59:41.886 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-05-03T17:59:41.886 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-05-03T17:59:41.886 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-05-03T17:59:41.886 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-05-03T17:59:41.886 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-05-03T17:59:41.886 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-05-03T17:59:41.887 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-05-03T17:59:41.887 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-05-03T17:59:41.887 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-05-03T17:59:41.887 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-05-03T17:59:41.887 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-05-03T17:59:41.887 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-05-03T17:59:41.887 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-05-03T17:59:41.887 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-05-03T17:59:41.887 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-05-03T17:59:41.887 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-05-03T17:59:41.887 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-05-03T17:59:41.887 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-05-03T17:59:41.887 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-05-03T17:59:41.887 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-05-03T17:59:41.887 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-05-03T17:59:41.887 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-05-03T17:59:41.888 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-05-03T17:59:41.888 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-05-03T17:59:41.888 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-05-03T17:59:41.888 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-05-03T17:59:41.888 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-05-03T17:59:41.888 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-05-03T17:59:41.888 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-05-03T17:59:41.888 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-05-03T17:59:41.888 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-05-03T17:59:41.888 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-05-03T17:59:41.888 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-05-03T17:59:41.888 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-05-03T17:59:41.888 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-05-03T17:59:41.888 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-05-03T17:59:41.889 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-05-03T17:59:41.889 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-05-03T17:59:41.889 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-05-03T17:59:41.889 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-05-03T17:59:41.889 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-05-03T17:59:41.889 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-05-03T17:59:41.889 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-05-03T17:59:41.889 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-05-03T17:59:41.889 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-05-03T17:59:41.889 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-05-03T17:59:41.889 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-05-03T17:59:41.889 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-05-03T17:59:41.889 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-05-03T17:59:41.889 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-05-03T17:59:41.889 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-05-03T17:59:41.890 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-05-03T17:59:41.890 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-05-03T17:59:41.890 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-05-03T17:59:41.890 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-05-03T17:59:41.890 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-05-03T17:59:41.890 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-05-03T17:59:41.890 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-05-03T17:59:41.890 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-05-03T17:59:41.890 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-05-03T17:59:41.890 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-05-03T17:59:41.890 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-05-03T17:59:41.890 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-05-03T17:59:41.890 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-05-03T17:59:41.891 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-05-03T17:59:41.891 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-05-03T17:59:41.891 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-05-03T17:59:41.891 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-05-03T17:59:41.891 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-05-03T17:59:41.891 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-05-03T17:59:41.891 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-05-03T17:59:41.891 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-05-03T17:59:41.891 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-05-03T17:59:41.891 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-05-03T17:59:41.891 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-05-03T17:59:41.891 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-05-03T17:59:41.891 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-05-03T17:59:41.891 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-05-03T17:59:41.891 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-05-03T17:59:41.892 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-05-03T17:59:41.892 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-05-03T17:59:41.892 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-05-03T17:59:41.892 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-05-03T17:59:41.892 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-05-03T17:59:41.892 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-05-03T17:59:41.892 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-05-03T17:59:41.892 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-05-03T17:59:41.892 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-05-03T17:59:41.892 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-05-03T17:59:41.892 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-05-03T17:59:41.892 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-05-03T17:59:41.892 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-05-03T17:59:41.892 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-05-03T17:59:41.892 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-05-03T17:59:41.893 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-05-03T17:59:41.893 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-05-03T17:59:41.893 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-05-03T17:59:41.893 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-05-03T17:59:41.893 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-05-03T17:59:41.893 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-05-03T17:59:41.893 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-05-03T17:59:41.893 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-05-03T17:59:41.893 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-05-03T17:59:41.893 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-05-03T17:59:41.893 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-05-03T17:59:41.893 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-05-03T17:59:41.893 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-05-03T17:59:41.893 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-05-03T17:59:41.894 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-05-03T17:59:41.894 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-05-03T17:59:41.894 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-05-03T17:59:41.894 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-05-03T17:59:41.894 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-05-03T17:59:41.894 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-05-03T17:59:41.894 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-05-03T17:59:41.894 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-05-03T17:59:41.894 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-05-03T17:59:41.894 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-05-03T17:59:41.894 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-05-03T17:59:41.894 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-05-03T17:59:41.894 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-05-03T17:59:41.895 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-05-03T17:59:41.895 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-05-03T17:59:41.895 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-05-03T17:59:41.895 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-05-03T17:59:41.895 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-05-03T17:59:41.895 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-05-03T17:59:41.895 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-05-03T17:59:41.895 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-05-03T17:59:41.895 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-05-03T17:59:41.895 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-05-03T17:59:41.895 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-05-03T17:59:41.895 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-05-03T17:59:41.895 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-05-03T17:59:41.896 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-05-03T17:59:41.896 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-05-03T17:59:41.896 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-05-03T17:59:41.896 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-05-03T17:59:41.896 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-05-03T17:59:41.896 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-05-03T17:59:41.896 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-05-03T17:59:41.896 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-05-03T17:59:41.896 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-05-03T17:59:41.896 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-05-03T17:59:41.896 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-05-03T17:59:41.896 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-05-03T17:59:41.896 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-05-03T17:59:41.896 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-05-03T17:59:41.897 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-05-03T17:59:41.897 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-05-03T17:59:41.897 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-05-03T17:59:41.897 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-05-03T17:59:41.897 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-05-03T17:59:41.897 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-05-03T17:59:41.897 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-05-03T17:59:41.897 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-05-03T17:59:41.897 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-05-03T17:59:41.897 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-05-03T17:59:41.897 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-05-03T17:59:41.897 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-05-03T17:59:41.897 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-05-03T17:59:41.897 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-05-03T17:59:41.897 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-05-03T17:59:41.897 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-05-03T17:59:41.898 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-05-03T17:59:41.898 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-05-03T17:59:41.898 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-05-03T17:59:41.898 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-05-03T17:59:41.898 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-05-03T17:59:41.898 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-05-03T17:59:41.898 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-05-03T17:59:41.898 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-05-03T17:59:41.898 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-05-03T17:59:41.898 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-05-03T17:59:41.898 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-05-03T17:59:41.898 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-05-03T17:59:41.898 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-05-03T17:59:41.898 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-05-03T17:59:41.898 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-05-03T17:59:41.899 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-05-03T17:59:41.899 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-05-03T17:59:41.899 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-05-03T17:59:41.899 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-05-03T17:59:41.899 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-05-03T17:59:41.899 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-05-03T17:59:41.899 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-05-03T17:59:41.899 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-05-03T17:59:41.899 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-05-03T17:59:41.899 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-05-03T17:59:41.899 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-05-03T17:59:41.899 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-05-03T17:59:41.899 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-05-03T17:59:41.899 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-05-03T17:59:41.899 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-05-03T17:59:41.899 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-05-03T17:59:41.900 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-05-03T17:59:41.900 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-05-03T17:59:41.900 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-05-03T17:59:41.900 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-05-03T17:59:41.900 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-05-03T17:59:41.900 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-05-03T17:59:41.900 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-05-03T17:59:41.900 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-05-03T17:59:41.900 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-05-03T17:59:41.900 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-05-03T17:59:41.900 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-05-03T17:59:41.900 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-05-03T17:59:41.900 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-05-03T17:59:41.901 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-05-03T17:59:41.901 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-05-03T17:59:41.901 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-05-03T17:59:41.901 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-05-03T17:59:41.901 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-05-03T17:59:41.901 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-05-03T17:59:41.901 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-05-03T17:59:41.901 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-05-03T17:59:41.901 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-05-03T17:59:41.901 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-05-03T17:59:41.901 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-05-03T17:59:41.901 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-05-03T17:59:41.901 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-05-03T17:59:41.901 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-05-03T17:59:41.901 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-05-03T17:59:41.902 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-05-03T17:59:41.902 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-05-03T17:59:41.902 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-05-03T17:59:41.902 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-05-03T17:59:41.902 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-05-03T17:59:41.902 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-05-03T17:59:41.902 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-05-03T17:59:41.902 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-05-03T17:59:41.902 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-05-03T17:59:41.902 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-05-03T17:59:41.902 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-05-03T17:59:41.902 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-05-03T17:59:41.902 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-05-03T17:59:41.902 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-05-03T17:59:41.903 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-05-03T17:59:41.903 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-05-03T17:59:41.903 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-05-03T17:59:41.903 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-05-03T17:59:41.903 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-05-03T17:59:41.903 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-05-03T17:59:41.903 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-05-03T17:59:41.903 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-05-03T17:59:41.903 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-05-03T17:59:41.903 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-05-03T17:59:41.903 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-05-03T17:59:41.903 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-05-03T17:59:41.903 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-05-03T17:59:41.903 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-05-03T17:59:41.904 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-05-03T17:59:41.904 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-05-03T17:59:41.904 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-05-03T17:59:41.904 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-05-03T17:59:41.904 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-05-03T17:59:41.904 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-05-03T17:59:41.904 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-05-03T17:59:41.904 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-05-03T17:59:41.904 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-05-03T17:59:41.904 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-05-03T17:59:41.904 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-05-03T17:59:41.904 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-05-03T17:59:41.904 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-05-03T17:59:41.904 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-05-03T17:59:41.904 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-05-03T17:59:41.905 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-05-03T17:59:41.905 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-05-03T17:59:41.905 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-05-03T17:59:41.905 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-05-03T17:59:41.905 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-05-03T17:59:41.905 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-05-03T17:59:41.905 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-05-03T17:59:41.905 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-05-03T17:59:41.905 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-05-03T17:59:41.905 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-05-03T17:59:41.905 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-05-03T17:59:41.905 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-05-03T17:59:41.905 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-05-03T17:59:41.905 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-05-03T17:59:41.906 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-05-03T17:59:41.906 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-05-03T17:59:41.906 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-05-03T17:59:41.906 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-05-03T17:59:41.906 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-05-03T17:59:41.906 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-05-03T17:59:41.906 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-05-03T17:59:41.906 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-05-03T17:59:41.906 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-05-03T17:59:41.906 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-05-03T17:59:41.906 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-05-03T17:59:41.906 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-05-03T17:59:41.906 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-05-03T17:59:41.906 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-05-03T17:59:41.907 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-05-03T17:59:41.907 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-05-03T17:59:41.907 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-05-03T17:59:41.907 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-05-03T17:59:41.907 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-05-03T17:59:41.907 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-05-03T17:59:41.907 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-05-03T17:59:41.907 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-05-03T17:59:41.907 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-05-03T17:59:41.907 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-05-03T17:59:41.907 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-05-03T17:59:41.907 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-05-03T17:59:41.907 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-05-03T17:59:41.907 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-05-03T17:59:41.907 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-05-03T17:59:41.907 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-05-03T17:59:41.908 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-05-03T17:59:41.908 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-05-03T17:59:41.908 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-05-03T17:59:41.908 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-05-03T17:59:41.908 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-05-03T17:59:41.908 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-05-03T17:59:41.908 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-05-03T17:59:41.908 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-05-03T17:59:41.908 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-05-03T17:59:41.908 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-05-03T17:59:41.908 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-05-03T17:59:41.908 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-05-03T17:59:41.908 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-05-03T17:59:41.908 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-05-03T17:59:41.909 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-05-03T17:59:41.909 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-05-03T17:59:41.909 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-05-03T17:59:41.909 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-05-03T17:59:41.909 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-05-03T17:59:41.909 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-05-03T17:59:41.909 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-05-03T17:59:41.909 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-05-03T17:59:41.909 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-05-03T17:59:41.909 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-05-03T17:59:41.909 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-05-03T17:59:41.909 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-05-03T17:59:41.909 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-05-03T17:59:41.909 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-05-03T17:59:41.909 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-05-03T17:59:41.910 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-05-03T17:59:41.910 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-05-03T17:59:41.910 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-05-03T17:59:41.910 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-05-03T17:59:41.910 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-05-03T17:59:41.910 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-05-03T17:59:41.910 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-05-03T17:59:41.910 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-05-03T17:59:41.910 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-05-03T17:59:41.910 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-05-03T17:59:41.910 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-05-03T17:59:41.910 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-05-03T17:59:41.910 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-05-03T17:59:41.910 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-05-03T17:59:41.911 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-05-03T17:59:41.911 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-05-03T17:59:41.911 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-05-03T17:59:41.911 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-05-03T17:59:41.911 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-05-03T17:59:41.911 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-05-03T17:59:41.911 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-05-03T17:59:41.911 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-05-03T17:59:41.911 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-05-03T17:59:41.911 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-05-03T17:59:41.911 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-05-03T17:59:41.911 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-05-03T17:59:41.911 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-05-03T17:59:41.911 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-05-03T17:59:41.912 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-05-03T17:59:41.912 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-05-03T17:59:41.912 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-05-03T17:59:41.912 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-05-03T17:59:41.912 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-05-03T17:59:41.912 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-05-03T17:59:41.912 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-05-03T17:59:41.912 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-05-03T17:59:41.912 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-05-03T17:59:41.912 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-05-03T17:59:41.912 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-05-03T17:59:41.912 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-05-03T17:59:41.912 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-05-03T17:59:41.913 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-05-03T17:59:41.913 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-05-03T17:59:41.913 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-05-03T17:59:41.913 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-05-03T17:59:41.913 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-05-03T17:59:41.913 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-05-03T17:59:41.913 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-05-03T17:59:41.913 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-05-03T17:59:41.913 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-05-03T17:59:41.913 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-05-03T17:59:41.913 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-05-03T17:59:41.913 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-05-03T17:59:41.913 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-05-03T17:59:41.913 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-05-03T17:59:41.914 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-05-03T17:59:41.914 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-05-03T17:59:41.914 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-05-03T17:59:41.914 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-05-03T17:59:41.914 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-05-03T17:59:41.914 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-05-03T17:59:41.914 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-05-03T17:59:41.914 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-05-03T17:59:41.914 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-05-03T17:59:41.914 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-05-03T17:59:41.914 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-05-03T17:59:41.914 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-05-03T17:59:41.914 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-05-03T17:59:41.915 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-05-03T17:59:41.915 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-05-03T17:59:41.915 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-05-03T17:59:41.915 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-05-03T17:59:41.915 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-05-03T17:59:41.915 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-05-03T17:59:41.915 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-05-03T17:59:41.915 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-05-03T17:59:41.915 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-05-03T17:59:41.915 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-05-03T17:59:41.915 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-05-03T17:59:41.915 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-05-03T17:59:41.915 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-05-03T17:59:41.915 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-05-03T17:59:41.915 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-05-03T17:59:41.915 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-05-03T17:59:41.916 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-05-03T17:59:41.916 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-05-03T17:59:41.916 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-05-03T17:59:41.916 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-05-03T17:59:41.916 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-05-03T17:59:41.916 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-05-03T17:59:41.916 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-05-03T17:59:41.916 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-05-03T17:59:41.916 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-05-03T17:59:41.916 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-05-03T17:59:41.916 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-05-03T17:59:41.916 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-05-03T17:59:41.916 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-05-03T17:59:41.916 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-05-03T17:59:41.916 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-05-03T17:59:41.917 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-05-03T17:59:41.917 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-05-03T17:59:41.917 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-05-03T17:59:41.917 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-05-03T17:59:41.917 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-05-03T17:59:41.917 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-05-03T17:59:41.917 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-05-03T17:59:41.917 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-05-03T17:59:41.917 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-05-03T17:59:41.917 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-05-03T17:59:41.917 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-05-03T17:59:41.917 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-05-03T17:59:41.917 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-05-03T17:59:41.917 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-05-03T17:59:41.917 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-05-03T17:59:41.918 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-05-03T17:59:41.918 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-05-03T17:59:41.918 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-05-03T17:59:41.918 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-05-03T17:59:41.918 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-05-03T17:59:41.918 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-05-03T17:59:41.918 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-05-03T17:59:41.918 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-05-03T17:59:41.918 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-05-03T17:59:41.918 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-05-03T17:59:41.918 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-05-03T17:59:41.918 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-05-03T17:59:41.918 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-05-03T17:59:41.918 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-05-03T17:59:41.919 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-05-03T17:59:41.919 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-05-03T17:59:41.919 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-05-03T17:59:41.919 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-05-03T17:59:41.919 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-05-03T17:59:41.919 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-05-03T17:59:41.919 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-05-03T17:59:41.919 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-05-03T17:59:41.919 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-05-03T17:59:41.919 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-05-03T17:59:41.919 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-05-03T17:59:41.919 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-05-03T17:59:41.919 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-05-03T17:59:41.919 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-05-03T17:59:41.920 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-05-03T17:59:41.920 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-05-03T17:59:41.920 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-05-03T17:59:41.920 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-05-03T17:59:41.920 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-05-03T17:59:41.920 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-05-03T17:59:41.920 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-05-03T17:59:41.920 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-05-03T17:59:41.920 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-05-03T17:59:41.920 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-05-03T17:59:41.920 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-05-03T17:59:41.920 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-05-03T17:59:41.920 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-05-03T17:59:41.920 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-05-03T17:59:41.921 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-05-03T17:59:41.921 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-05-03T17:59:41.921 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-05-03T17:59:41.921 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-05-03T17:59:41.921 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-05-03T17:59:41.921 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-05-03T17:59:41.921 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-05-03T17:59:41.921 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-05-03T17:59:41.921 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-05-03T17:59:41.921 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-05-03T17:59:41.921 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-05-03T17:59:41.921 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-05-03T17:59:41.921 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-05-03T17:59:41.921 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-05-03T17:59:41.922 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-05-03T17:59:41.922 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-05-03T17:59:41.922 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-05-03T17:59:41.922 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-05-03T17:59:41.922 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-05-03T17:59:41.922 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-05-03T17:59:41.922 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-05-03T17:59:41.922 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-05-03T17:59:41.922 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-05-03T17:59:41.922 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-05-03T17:59:41.922 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-05-03T17:59:41.922 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-05-03T17:59:41.922 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-05-03T17:59:41.922 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-05-03T17:59:41.923 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-05-03T17:59:41.923 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-05-03T17:59:41.923 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-05-03T17:59:41.923 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-05-03T17:59:41.923 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-05-03T17:59:41.923 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-05-03T17:59:41.923 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-05-03T17:59:41.923 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-05-03T17:59:41.923 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-05-03T17:59:41.923 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-05-03T17:59:41.923 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-05-03T17:59:41.923 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-05-03T17:59:41.923 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-05-03T17:59:41.923 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-05-03T17:59:41.924 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-05-03T17:59:41.924 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-05-03T17:59:41.924 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-05-03T17:59:41.924 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-05-03T17:59:41.924 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-05-03T17:59:41.924 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-05-03T17:59:41.924 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-05-03T17:59:41.924 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-05-03T17:59:41.924 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-05-03T17:59:41.924 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-05-03T17:59:41.924 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-05-03T17:59:41.924 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-05-03T17:59:41.924 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-05-03T17:59:41.924 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-05-03T17:59:41.925 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-05-03T17:59:41.925 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-05-03T17:59:41.925 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-05-03T17:59:41.925 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-05-03T17:59:41.925 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-05-03T17:59:41.925 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-05-03T17:59:41.925 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-05-03T17:59:41.925 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-05-03T17:59:41.925 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-05-03T17:59:41.925 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-05-03T17:59:41.925 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-05-03T17:59:41.925 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-05-03T17:59:41.925 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-05-03T17:59:41.925 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-05-03T17:59:41.925 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-05-03T17:59:41.926 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-05-03T17:59:41.926 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-05-03T17:59:41.926 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-05-03T17:59:41.926 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-05-03T17:59:41.926 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-05-03T17:59:41.926 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-05-03T17:59:41.926 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-05-03T17:59:41.926 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-05-03T17:59:41.926 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-05-03T17:59:41.926 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-05-03T17:59:41.926 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-05-03T17:59:41.926 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-05-03T17:59:41.926 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-05-03T17:59:41.927 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-05-03T17:59:41.927 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-05-03T17:59:41.927 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-05-03T17:59:41.927 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-05-03T17:59:41.927 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-05-03T17:59:41.927 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-05-03T17:59:41.927 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-05-03T17:59:41.927 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-05-03T17:59:41.927 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-05-03T17:59:41.927 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-05-03T17:59:41.927 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-05-03T17:59:41.927 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-05-03T17:59:41.927 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-05-03T17:59:41.927 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-05-03T17:59:41.927 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-05-03T17:59:41.928 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-05-03T17:59:41.928 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-05-03T17:59:41.928 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-05-03T17:59:41.928 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-05-03T17:59:41.928 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-05-03T17:59:41.928 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-05-03T17:59:41.928 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-05-03T17:59:41.928 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-05-03T17:59:41.928 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-05-03T17:59:41.928 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-05-03T17:59:41.928 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-05-03T17:59:41.928 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-05-03T17:59:41.928 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-05-03T17:59:41.928 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-05-03T17:59:41.929 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-05-03T17:59:41.929 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-05-03T17:59:41.929 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-05-03T17:59:41.929 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-05-03T17:59:41.929 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-05-03T17:59:41.929 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-05-03T17:59:41.929 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-05-03T17:59:41.929 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-05-03T17:59:41.929 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-05-03T17:59:41.929 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-05-03T17:59:41.929 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-05-03T17:59:41.929 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-05-03T17:59:41.929 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-05-03T17:59:41.930 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-05-03T17:59:41.930 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-05-03T17:59:41.930 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-05-03T17:59:41.930 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-05-03T17:59:41.930 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-05-03T17:59:41.930 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-05-03T17:59:41.930 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-05-03T17:59:41.930 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-05-03T17:59:41.930 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-05-03T17:59:41.930 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-05-03T17:59:41.930 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-05-03T17:59:41.930 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-05-03T17:59:41.930 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-05-03T17:59:41.930 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-05-03T17:59:41.931 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-05-03T17:59:41.931 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-05-03T17:59:41.931 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-05-03T17:59:41.931 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-05-03T17:59:41.931 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-05-03T17:59:41.931 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-05-03T17:59:41.931 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-05-03T17:59:41.931 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-05-03T17:59:41.931 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-05-03T17:59:41.931 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-05-03T17:59:41.931 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-05-03T17:59:41.931 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-05-03T17:59:41.931 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-05-03T17:59:41.931 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-05-03T17:59:41.932 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-05-03T17:59:41.932 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-05-03T17:59:41.932 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-05-03T17:59:41.932 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-05-03T17:59:41.932 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-05-03T17:59:41.932 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-05-03T17:59:41.932 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-05-03T17:59:41.932 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-05-03T17:59:41.932 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-05-03T17:59:41.932 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-05-03T17:59:41.932 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-05-03T17:59:41.932 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-05-03T17:59:41.932 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-05-03T17:59:41.932 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-05-03T17:59:41.933 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-05-03T17:59:41.933 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-05-03T17:59:41.933 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-05-03T17:59:41.933 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-05-03T17:59:41.933 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-05-03T17:59:41.933 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-05-03T17:59:41.933 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-05-03T17:59:41.933 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-05-03T17:59:41.933 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-05-03T17:59:41.933 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-05-03T17:59:41.933 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-05-03T17:59:41.933 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-05-03T17:59:41.933 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-05-03T17:59:41.933 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-05-03T17:59:41.934 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-05-03T17:59:41.934 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-05-03T17:59:41.934 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-05-03T17:59:41.934 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-05-03T17:59:41.934 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-05-03T17:59:41.934 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-05-03T17:59:41.934 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-05-03T17:59:41.934 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-05-03T17:59:41.934 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-05-03T17:59:41.934 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-05-03T17:59:41.934 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-05-03T17:59:41.934 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-05-03T17:59:41.934 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-05-03T17:59:41.934 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-05-03T17:59:41.935 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-05-03T17:59:41.935 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-05-03T17:59:41.935 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-05-03T17:59:41.935 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-05-03T17:59:41.935 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-05-03T17:59:41.935 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-05-03T17:59:41.935 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-05-03T17:59:41.935 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-05-03T17:59:41.935 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-05-03T17:59:41.935 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-05-03T17:59:41.935 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-05-03T17:59:41.935 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-05-03T17:59:41.935 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-05-03T17:59:41.935 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-05-03T17:59:41.936 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-05-03T17:59:41.936 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-05-03T17:59:41.936 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-05-03T17:59:41.936 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-05-03T17:59:41.936 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-05-03T17:59:41.936 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-05-03T17:59:41.936 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-05-03T17:59:41.936 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-05-03T17:59:41.936 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-05-03T17:59:41.936 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-05-03T17:59:41.936 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-05-03T17:59:41.936 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-05-03T17:59:41.936 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-05-03T17:59:41.936 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-05-03T17:59:41.936 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-05-03T17:59:41.937 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-05-03T17:59:41.937 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-05-03T17:59:41.937 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-05-03T17:59:41.937 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-05-03T17:59:41.937 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-05-03T17:59:41.937 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-05-03T17:59:41.937 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-05-03T17:59:41.937 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-05-03T17:59:41.937 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-05-03T17:59:41.937 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-05-03T17:59:41.937 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-05-03T17:59:41.937 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-05-03T17:59:41.937 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-05-03T17:59:41.937 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-05-03T17:59:41.937 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-05-03T17:59:41.938 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-05-03T17:59:41.938 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-05-03T17:59:41.938 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-05-03T17:59:41.938 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-05-03T17:59:41.938 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-05-03T17:59:41.938 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-05-03T17:59:41.938 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-05-03T17:59:41.938 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-05-03T17:59:41.938 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-05-03T17:59:41.938 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-05-03T17:59:41.938 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-05-03T17:59:41.938 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-05-03T17:59:41.938 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-05-03T17:59:41.938 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-05-03T17:59:41.939 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-05-03T17:59:41.939 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-05-03T17:59:41.939 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-05-03T17:59:41.939 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-05-03T17:59:41.939 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-05-03T17:59:41.939 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-05-03T17:59:41.939 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-05-03T17:59:41.939 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-05-03T17:59:41.939 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-05-03T17:59:41.939 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-05-03T17:59:41.939 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-05-03T17:59:41.939 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-05-03T17:59:41.939 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-05-03T17:59:41.939 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-05-03T17:59:41.939 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-05-03T17:59:41.939 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-05-03T17:59:41.940 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-05-03T17:59:41.940 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-05-03T17:59:41.940 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-05-03T17:59:41.940 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-05-03T17:59:41.940 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-05-03T17:59:41.940 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-05-03T17:59:41.940 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-05-03T17:59:41.940 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-05-03T17:59:41.940 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-05-03T17:59:41.940 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-05-03T17:59:41.940 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-05-03T17:59:41.940 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-05-03T17:59:41.940 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-05-03T17:59:41.940 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-05-03T17:59:41.940 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-05-03T17:59:41.940 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-05-03T17:59:41.941 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-05-03T17:59:41.941 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-05-03T17:59:41.941 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-05-03T17:59:41.941 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-05-03T17:59:41.941 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-05-03T17:59:41.941 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-05-03T17:59:41.941 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-05-03T17:59:41.941 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-05-03T17:59:41.941 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-05-03T17:59:41.941 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-05-03T17:59:41.941 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-05-03T17:59:41.941 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-05-03T17:59:41.941 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-05-03T17:59:41.941 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-05-03T17:59:41.941 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-05-03T17:59:41.942 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-05-03T17:59:41.942 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-05-03T17:59:41.942 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-05-03T17:59:41.942 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-05-03T17:59:41.942 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-05-03T17:59:41.942 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-05-03T17:59:41.942 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-05-03T17:59:41.942 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-05-03T17:59:41.942 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-05-03T17:59:41.942 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-05-03T17:59:41.942 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-05-03T17:59:41.942 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-05-03T17:59:41.942 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-05-03T17:59:41.942 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-05-03T17:59:41.943 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-05-03T17:59:41.943 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-05-03T17:59:41.943 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-05-03T17:59:41.943 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-05-03T17:59:41.943 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-05-03T17:59:41.943 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-05-03T17:59:41.943 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-05-03T17:59:41.943 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-05-03T17:59:41.943 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-05-03T17:59:41.943 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-05-03T17:59:41.943 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-05-03T17:59:41.943 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-05-03T17:59:41.943 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-05-03T17:59:41.943 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-05-03T17:59:41.943 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-05-03T17:59:41.943 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-05-03T17:59:41.944 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-05-03T17:59:41.944 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-05-03T17:59:41.944 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-05-03T17:59:41.944 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-05-03T17:59:41.944 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-05-03T17:59:41.944 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-05-03T17:59:41.944 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-05-03T17:59:41.944 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-05-03T17:59:41.944 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-05-03T17:59:41.944 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-05-03T17:59:41.944 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-05-03T17:59:41.944 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-05-03T17:59:41.944 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-05-03T17:59:41.944 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-05-03T17:59:41.945 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-05-03T17:59:41.945 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-05-03T17:59:41.945 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-05-03T17:59:41.945 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-05-03T17:59:41.945 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-05-03T17:59:41.945 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-05-03T17:59:41.945 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-05-03T17:59:41.945 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-05-03T17:59:41.945 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-05-03T17:59:41.945 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-05-03T17:59:41.945 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-05-03T17:59:41.945 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-05-03T17:59:41.945 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-05-03T17:59:41.945 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-05-03T17:59:41.945 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-05-03T17:59:41.946 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-05-03T17:59:41.946 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-05-03T17:59:41.946 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-05-03T17:59:41.946 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-05-03T17:59:41.946 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-05-03T17:59:41.946 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-05-03T17:59:41.946 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-05-03T17:59:41.946 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-05-03T17:59:41.946 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-05-03T17:59:41.946 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-05-03T17:59:41.946 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-05-03T17:59:41.946 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-05-03T17:59:41.946 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-05-03T17:59:41.946 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-05-03T17:59:41.946 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-05-03T17:59:41.947 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-05-03T17:59:41.947 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-05-03T17:59:41.947 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-05-03T17:59:41.947 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-05-03T17:59:41.947 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-05-03T17:59:41.947 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-05-03T17:59:41.947 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-05-03T17:59:41.947 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-05-03T17:59:41.947 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-05-03T17:59:41.947 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-05-03T17:59:41.947 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-05-03T17:59:41.947 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-05-03T17:59:41.947 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-05-03T17:59:41.947 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-05-03T17:59:41.947 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-05-03T17:59:41.947 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-05-03T17:59:41.948 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-05-03T17:59:41.948 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-05-03T17:59:41.948 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-05-03T17:59:41.948 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-05-03T17:59:41.948 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-05-03T17:59:41.948 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-05-03T17:59:41.948 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-05-03T17:59:41.948 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-05-03T17:59:41.948 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-05-03T17:59:41.948 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-05-03T17:59:41.948 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-05-03T17:59:41.948 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-05-03T17:59:41.948 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-05-03T17:59:41.948 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-05-03T17:59:41.948 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-05-03T17:59:41.948 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-05-03T17:59:41.949 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-05-03T17:59:41.949 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-05-03T17:59:41.949 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-05-03T17:59:41.949 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-05-03T17:59:41.949 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-05-03T17:59:41.949 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-05-03T17:59:41.949 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-05-03T17:59:41.949 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-05-03T17:59:41.949 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-05-03T17:59:41.949 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-05-03T17:59:41.949 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-05-03T17:59:41.949 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-05-03T17:59:41.949 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-05-03T17:59:41.949 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-05-03T17:59:41.949 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-05-03T17:59:41.949 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-05-03T17:59:41.950 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-05-03T17:59:41.950 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-05-03T17:59:41.950 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-05-03T17:59:41.950 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-05-03T17:59:41.950 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-05-03T17:59:41.950 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-05-03T17:59:41.950 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-05-03T17:59:41.950 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-05-03T17:59:41.950 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-05-03T17:59:41.950 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-05-03T17:59:41.950 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-05-03T17:59:41.950 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-05-03T17:59:41.950 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-05-03T17:59:41.951 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-05-03T17:59:41.951 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-05-03T17:59:41.951 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-05-03T17:59:41.951 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-05-03T17:59:41.951 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-05-03T17:59:41.951 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-05-03T17:59:41.951 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-05-03T17:59:41.951 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-05-03T17:59:41.951 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-05-03T17:59:41.951 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-05-03T17:59:41.951 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-05-03T17:59:41.951 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-05-03T17:59:41.951 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-05-03T17:59:41.951 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-05-03T17:59:41.952 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-05-03T17:59:41.952 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-05-03T17:59:41.952 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-05-03T17:59:41.952 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-05-03T17:59:41.952 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-05-03T17:59:41.952 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-05-03T17:59:41.952 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-05-03T17:59:41.952 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-05-03T17:59:41.952 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-05-03T17:59:41.952 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-05-03T17:59:41.952 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-05-03T17:59:41.952 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-05-03T17:59:41.952 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-05-03T17:59:41.952 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-05-03T17:59:41.952 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-05-03T17:59:41.953 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-05-03T17:59:41.953 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-05-03T17:59:41.953 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-05-03T17:59:41.953 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-05-03T17:59:41.953 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-05-03T17:59:41.953 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-05-03T17:59:41.953 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-05-03T17:59:41.953 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-05-03T17:59:41.953 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-05-03T17:59:41.953 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-05-03T17:59:41.953 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-05-03T17:59:41.953 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-05-03T17:59:41.953 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-05-03T17:59:41.953 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-05-03T17:59:41.954 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-05-03T17:59:41.954 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-05-03T17:59:41.954 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-05-03T17:59:41.954 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-05-03T17:59:41.954 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-05-03T17:59:41.954 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-05-03T17:59:41.954 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-05-03T17:59:41.954 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-05-03T17:59:41.954 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-05-03T17:59:41.954 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-05-03T17:59:41.954 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-05-03T17:59:41.954 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-05-03T17:59:41.954 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-05-03T17:59:41.954 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-05-03T17:59:41.955 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-05-03T17:59:41.955 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-05-03T17:59:41.955 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-05-03T17:59:41.955 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-05-03T17:59:41.955 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-05-03T17:59:41.955 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-05-03T17:59:41.955 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-05-03T17:59:41.955 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-05-03T17:59:41.955 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-05-03T17:59:41.955 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-05-03T17:59:41.955 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-05-03T17:59:41.955 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-05-03T17:59:41.955 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-05-03T17:59:41.955 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-05-03T17:59:41.955 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-05-03T17:59:41.956 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-05-03T17:59:41.956 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-05-03T17:59:41.956 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-05-03T17:59:41.956 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-05-03T17:59:41.956 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-05-03T17:59:41.956 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-05-03T17:59:41.956 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-05-03T17:59:41.956 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-05-03T17:59:41.956 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-05-03T17:59:41.956 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-05-03T17:59:41.956 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-05-03T17:59:41.956 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-05-03T17:59:41.956 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-05-03T17:59:41.956 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-05-03T17:59:41.956 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-05-03T17:59:41.956 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-05-03T17:59:41.957 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-05-03T17:59:41.957 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-05-03T17:59:41.957 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-05-03T17:59:41.957 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-05-03T17:59:41.957 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-05-03T17:59:41.957 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-05-03T17:59:41.957 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-05-03T17:59:41.957 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-05-03T17:59:41.957 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-05-03T17:59:41.957 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-05-03T17:59:41.957 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-05-03T17:59:41.957 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-05-03T17:59:41.957 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-05-03T17:59:41.957 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-05-03T17:59:41.958 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-05-03T17:59:41.958 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-05-03T17:59:41.958 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-05-03T17:59:41.958 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-05-03T17:59:41.958 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-05-03T17:59:41.958 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-05-03T17:59:41.958 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-05-03T17:59:41.958 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-05-03T17:59:41.958 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-05-03T17:59:41.958 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-05-03T17:59:41.958 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-05-03T17:59:41.958 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-05-03T17:59:41.958 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-05-03T17:59:41.958 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-05-03T17:59:41.959 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-05-03T17:59:41.959 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-05-03T17:59:41.959 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-05-03T17:59:41.959 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-05-03T17:59:41.959 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-05-03T17:59:41.959 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-05-03T17:59:41.959 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-05-03T17:59:41.959 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-05-03T17:59:41.959 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-05-03T17:59:41.959 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-05-03T17:59:41.959 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-05-03T17:59:41.959 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-05-03T17:59:41.959 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-05-03T17:59:41.959 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-05-03T17:59:41.960 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-05-03T17:59:41.960 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-05-03T17:59:41.960 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-05-03T17:59:41.960 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-05-03T17:59:41.960 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-05-03T17:59:41.960 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-05-03T17:59:41.960 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-05-03T17:59:41.960 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-05-03T17:59:41.960 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-05-03T17:59:41.960 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-05-03T17:59:41.960 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-05-03T17:59:41.960 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-05-03T17:59:41.961 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-05-03T17:59:41.961 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-05-03T17:59:41.961 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-05-03T17:59:41.961 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-05-03T17:59:41.961 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-05-03T17:59:41.961 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-05-03T17:59:41.961 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-05-03T17:59:41.961 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-05-03T17:59:41.961 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-05-03T17:59:41.961 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-05-03T17:59:41.961 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-05-03T17:59:41.961 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-05-03T17:59:41.961 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-05-03T17:59:41.961 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-05-03T17:59:41.962 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-05-03T17:59:41.962 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-05-03T17:59:41.962 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-05-03T17:59:41.962 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-05-03T17:59:41.962 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-05-03T17:59:41.962 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-05-03T17:59:41.962 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-05-03T17:59:41.962 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-05-03T17:59:41.962 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-05-03T17:59:41.962 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-05-03T17:59:41.962 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-05-03T17:59:41.962 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-05-03T17:59:41.962 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-05-03T17:59:41.962 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-05-03T17:59:41.963 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-05-03T17:59:41.963 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-05-03T17:59:41.963 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-05-03T17:59:41.963 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-05-03T17:59:41.963 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-05-03T17:59:41.963 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-05-03T17:59:41.963 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-05-03T17:59:41.963 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-05-03T17:59:41.963 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-05-03T17:59:41.963 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-05-03T17:59:41.963 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-05-03T17:59:41.963 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-05-03T17:59:41.963 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-05-03T17:59:41.963 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-05-03T17:59:41.963 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-05-03T17:59:41.964 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-05-03T17:59:41.964 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-05-03T17:59:41.964 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-05-03T17:59:41.964 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-05-03T17:59:41.964 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-05-03T17:59:41.964 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-05-03T17:59:41.964 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-05-03T17:59:41.964 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-05-03T17:59:41.964 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-05-03T17:59:41.964 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-05-03T17:59:41.964 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-05-03T17:59:41.964 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-05-03T17:59:41.964 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-05-03T17:59:41.964 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-05-03T17:59:41.964 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-05-03T17:59:41.965 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-05-03T17:59:41.965 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-05-03T17:59:41.965 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-05-03T17:59:41.965 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-05-03T17:59:41.965 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-05-03T17:59:41.965 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-05-03T17:59:41.965 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-05-03T17:59:41.965 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-05-03T17:59:41.965 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-05-03T17:59:41.965 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-05-03T17:59:41.965 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-05-03T17:59:41.965 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-05-03T17:59:41.965 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-05-03T17:59:41.965 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-05-03T17:59:41.965 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-05-03T17:59:41.965 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-05-03T17:59:41.966 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-05-03T17:59:41.966 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-05-03T17:59:41.966 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-05-03T17:59:41.966 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-05-03T17:59:41.969 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-05-03T17:59:41.969 DEBUG:teuthology.orchestra.run.smithi099:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-05-03T17:59:41.990 INFO:teuthology.orchestra.run.smithi099.stdout:postinstall scriptlet (using /bin/sh): 2024-05-03T17:59:41.990 INFO:teuthology.orchestra.run.smithi099.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-05-03T17:59:41.991 INFO:teuthology.orchestra.run.smithi099.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-05-03T17:59:41.991 INFO:teuthology.orchestra.run.smithi099.stdout:fi 2024-05-03T17:59:41.991 INFO:teuthology.orchestra.run.smithi099.stdout:for file in vmlinuz System.map config; do 2024-05-03T17:59:41.991 INFO:teuthology.orchestra.run.smithi099.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-05-03T17:59:41.991 INFO:teuthology.orchestra.run.smithi099.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-05-03T17:59:41.991 INFO:teuthology.orchestra.run.smithi099.stdout: fi 2024-05-03T17:59:41.991 INFO:teuthology.orchestra.run.smithi099.stdout:done 2024-05-03T17:59:41.991 INFO:teuthology.orchestra.run.smithi099.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-05-03T17:59:41.991 INFO:teuthology.orchestra.run.smithi099.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-05-03T17:59:41.991 INFO:teuthology.orchestra.run.smithi099.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-05-03T17:59:41.991 INFO:teuthology.orchestra.run.smithi099.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-05-03T17:59:41.991 INFO:teuthology.orchestra.run.smithi099.stdout:fi 2024-05-03T17:59:41.991 INFO:teuthology.orchestra.run.smithi099.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-05-03T17:59:41.991 INFO:teuthology.orchestra.run.smithi099.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-05-03T17:59:41.991 INFO:teuthology.orchestra.run.smithi099.stdout:fi 2024-05-03T17:59:41.992 INFO:teuthology.orchestra.run.smithi099.stdout:preuninstall scriptlet (using /bin/sh): 2024-05-03T17:59:41.992 INFO:teuthology.orchestra.run.smithi099.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-05-03T17:59:41.992 INFO:teuthology.orchestra.run.smithi099.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-05-03T17:59:41.992 INFO:teuthology.orchestra.run.smithi099.stdout:fi 2024-05-03T17:59:41.993 DEBUG:teuthology.orchestra.run.smithi099:> sudo rpm -qi grub2-tools 2024-05-03T17:59:42.072 INFO:teuthology.orchestra.run.smithi099.stdout:Name : grub2-tools 2024-05-03T17:59:42.072 INFO:teuthology.orchestra.run.smithi099.stdout:Epoch : 1 2024-05-03T17:59:42.072 INFO:teuthology.orchestra.run.smithi099.stdout:Version : 2.06 2024-05-03T17:59:42.072 INFO:teuthology.orchestra.run.smithi099.stdout:Release : 68.el9 2024-05-03T17:59:42.072 INFO:teuthology.orchestra.run.smithi099.stdout:Architecture: x86_64 2024-05-03T17:59:42.073 INFO:teuthology.orchestra.run.smithi099.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-03T17:59:42.073 INFO:teuthology.orchestra.run.smithi099.stdout:Group : Unspecified 2024-05-03T17:59:42.073 INFO:teuthology.orchestra.run.smithi099.stdout:Size : 8264254 2024-05-03T17:59:42.073 INFO:teuthology.orchestra.run.smithi099.stdout:License : GPLv3+ 2024-05-03T17:59:42.073 INFO:teuthology.orchestra.run.smithi099.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-03T17:59:42.073 INFO:teuthology.orchestra.run.smithi099.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-03T17:59:42.073 INFO:teuthology.orchestra.run.smithi099.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-03T17:59:42.073 INFO:teuthology.orchestra.run.smithi099.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-03T17:59:42.073 INFO:teuthology.orchestra.run.smithi099.stdout:Packager : builder@centos.org 2024-05-03T17:59:42.073 INFO:teuthology.orchestra.run.smithi099.stdout:Vendor : CentOS 2024-05-03T17:59:42.073 INFO:teuthology.orchestra.run.smithi099.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-03T17:59:42.073 INFO:teuthology.orchestra.run.smithi099.stdout:Summary : Support tools for GRUB. 2024-05-03T17:59:42.073 INFO:teuthology.orchestra.run.smithi099.stdout:Description : 2024-05-03T17:59:42.073 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-03T17:59:42.073 INFO:teuthology.orchestra.run.smithi099.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-03T17:59:42.074 INFO:teuthology.orchestra.run.smithi099.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-03T17:59:42.074 INFO:teuthology.orchestra.run.smithi099.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-03T17:59:42.074 INFO:teuthology.orchestra.run.smithi099.stdout:hardware devices. 2024-05-03T17:59:42.074 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-03T17:59:42.074 INFO:teuthology.orchestra.run.smithi099.stdout:This subpackage provides tools for support of all platforms. 2024-05-03T17:59:42.075 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-03T17:59:42.075 INFO:teuthology.task.kernel:Updating grub on smithi099 to boot 6.9.0-rc3-g806ec36b2112 2024-05-03T17:59:42.075 DEBUG:teuthology.orchestra.run.smithi099:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-03T17:59:42.631 INFO:teuthology.orchestra.run.smithi099.stderr:Generating grub configuration file ... 2024-05-03T17:59:43.968 INFO:teuthology.orchestra.run.smithi099.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-03T17:59:44.022 INFO:teuthology.orchestra.run.smithi099.stderr:done 2024-05-03T17:59:44.023 DEBUG:teuthology.orchestra.run.smithi099:> mktemp 2024-05-03T17:59:44.038 INFO:teuthology.orchestra.run.smithi099.stdout:/tmp/tmp.k19xknURAS 2024-05-03T17:59:44.039 DEBUG:teuthology.orchestra.run.smithi099:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.k19xknURAS 2024-05-03T17:59:44.106 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 0666 /tmp/tmp.k19xknURAS 2024-05-03T17:59:44.248 DEBUG:teuthology.orchestra.remote:smithi099:/tmp/tmp.k19xknURAS is 6KB 2024-05-03T17:59:44.297 DEBUG:teuthology.orchestra.run.smithi099:> rm -fr /tmp/tmp.k19xknURAS 2024-05-03T17:59:44.311 DEBUG:teuthology.orchestra.run.smithi099:> sudo /bin/ls /boot/loader/entries || true 2024-05-03T17:59:44.371 INFO:teuthology.orchestra.run.smithi099.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-03T17:59:44.372 INFO:teuthology.orchestra.run.smithi099.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-03T17:59:44.372 INFO:teuthology.orchestra.run.smithi099.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-03T17:59:44.372 INFO:teuthology.orchestra.run.smithi099.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-05-03T17:59:44.373 DEBUG:teuthology.orchestra.run.smithi099:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-05-03T17:59:44.517 DEBUG:teuthology.orchestra.run.smithi099:> sudo shutdown -r now 2024-05-03T17:59:48.303 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi033.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-05-03T17:59:48.303 DEBUG:teuthology.orchestra.run.smithi033:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-05-03T17:59:48.389 INFO:teuthology.orchestra.run.smithi033.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-05-03T17:59:48.390 INFO:teuthology.orchestra.run.smithi033.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-05-03T17:59:48.390 INFO:teuthology.orchestra.run.smithi033.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-05-03T17:59:48.390 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-05-03T17:59:48.390 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-05-03T17:59:48.390 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-05-03T17:59:48.390 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-05-03T17:59:48.390 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-05-03T17:59:48.390 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-05-03T17:59:48.390 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-05-03T17:59:48.390 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-05-03T17:59:48.390 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-05-03T17:59:48.390 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-05-03T17:59:48.391 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-05-03T17:59:48.391 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-05-03T17:59:48.391 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-05-03T17:59:48.391 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-05-03T17:59:48.391 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-05-03T17:59:48.391 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-05-03T17:59:48.391 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-05-03T17:59:48.391 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-05-03T17:59:48.391 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-05-03T17:59:48.391 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-05-03T17:59:48.391 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-05-03T17:59:48.391 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-05-03T17:59:48.391 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-05-03T17:59:48.391 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-05-03T17:59:48.391 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-05-03T17:59:48.392 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-05-03T17:59:48.392 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-05-03T17:59:48.392 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-05-03T17:59:48.392 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-05-03T17:59:48.392 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-05-03T17:59:48.392 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-05-03T17:59:48.392 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-05-03T17:59:48.392 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-05-03T17:59:48.392 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-05-03T17:59:48.392 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-05-03T17:59:48.392 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-05-03T17:59:48.392 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-05-03T17:59:48.392 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-05-03T17:59:48.392 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-05-03T17:59:48.392 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-05-03T17:59:48.393 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-05-03T17:59:48.393 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-05-03T17:59:48.393 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-05-03T17:59:48.393 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-05-03T17:59:48.393 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-05-03T17:59:48.393 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-05-03T17:59:48.393 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-05-03T17:59:48.393 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-05-03T17:59:48.393 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-05-03T17:59:48.393 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-05-03T17:59:48.393 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-05-03T17:59:48.393 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-05-03T17:59:48.393 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-05-03T17:59:48.393 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-05-03T17:59:48.393 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-05-03T17:59:48.393 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-05-03T17:59:48.394 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-05-03T17:59:48.394 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-05-03T17:59:48.394 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-05-03T17:59:48.394 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-05-03T17:59:48.394 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-05-03T17:59:48.394 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-05-03T17:59:48.394 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-05-03T17:59:48.394 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-05-03T17:59:48.394 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-05-03T17:59:48.394 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-05-03T17:59:48.394 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-05-03T17:59:48.394 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-05-03T17:59:48.394 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-05-03T17:59:48.395 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-05-03T17:59:48.395 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-05-03T17:59:48.395 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-05-03T17:59:48.395 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-05-03T17:59:48.395 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-05-03T17:59:48.395 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-05-03T17:59:48.395 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-05-03T17:59:48.395 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-05-03T17:59:48.395 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-05-03T17:59:48.395 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-05-03T17:59:48.395 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-05-03T17:59:48.395 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-05-03T17:59:48.395 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-05-03T17:59:48.395 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-05-03T17:59:48.396 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-05-03T17:59:48.396 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-05-03T17:59:48.396 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-05-03T17:59:48.396 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-05-03T17:59:48.396 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-05-03T17:59:48.396 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-05-03T17:59:48.396 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-05-03T17:59:48.396 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-05-03T17:59:48.396 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-05-03T17:59:48.396 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-05-03T17:59:48.396 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-05-03T17:59:48.396 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-05-03T17:59:48.396 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-05-03T17:59:48.397 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-05-03T17:59:48.397 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-05-03T17:59:48.397 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-05-03T17:59:48.397 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-05-03T17:59:48.397 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-05-03T17:59:48.397 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-05-03T17:59:48.397 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-05-03T17:59:48.397 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-05-03T17:59:48.397 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-05-03T17:59:48.397 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-05-03T17:59:48.397 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-05-03T17:59:48.397 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-05-03T17:59:48.397 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-05-03T17:59:48.397 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-05-03T17:59:48.398 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-05-03T17:59:48.398 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-05-03T17:59:48.398 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-05-03T17:59:48.398 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-05-03T17:59:48.398 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-05-03T17:59:48.398 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-05-03T17:59:48.398 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-05-03T17:59:48.398 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-05-03T17:59:48.398 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-05-03T17:59:48.398 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-05-03T17:59:48.398 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-05-03T17:59:48.398 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-05-03T17:59:48.398 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-05-03T17:59:48.399 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-05-03T17:59:48.399 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-05-03T17:59:48.399 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-05-03T17:59:48.399 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-05-03T17:59:48.399 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-05-03T17:59:48.399 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-05-03T17:59:48.399 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-05-03T17:59:48.399 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-05-03T17:59:48.399 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-05-03T17:59:48.399 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-05-03T17:59:48.399 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-05-03T17:59:48.399 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-05-03T17:59:48.399 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-05-03T17:59:48.400 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-05-03T17:59:48.400 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-05-03T17:59:48.400 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-05-03T17:59:48.400 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-05-03T17:59:48.400 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-05-03T17:59:48.400 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-05-03T17:59:48.400 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-05-03T17:59:48.400 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-05-03T17:59:48.400 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-05-03T17:59:48.400 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-05-03T17:59:48.400 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-05-03T17:59:48.400 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-05-03T17:59:48.400 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-05-03T17:59:48.400 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-05-03T17:59:48.400 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-05-03T17:59:48.400 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-05-03T17:59:48.401 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-05-03T17:59:48.401 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-05-03T17:59:48.401 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-05-03T17:59:48.401 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-05-03T17:59:48.401 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-05-03T17:59:48.401 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-05-03T17:59:48.401 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-05-03T17:59:48.401 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-05-03T17:59:48.401 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-05-03T17:59:48.401 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-05-03T17:59:48.401 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-05-03T17:59:48.401 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-05-03T17:59:48.401 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-05-03T17:59:48.401 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-05-03T17:59:48.402 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-05-03T17:59:48.402 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-05-03T17:59:48.402 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-05-03T17:59:48.402 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-05-03T17:59:48.402 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-05-03T17:59:48.402 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-05-03T17:59:48.402 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-05-03T17:59:48.402 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-05-03T17:59:48.402 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-05-03T17:59:48.402 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-05-03T17:59:48.402 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-05-03T17:59:48.402 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-05-03T17:59:48.403 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-05-03T17:59:48.403 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-05-03T17:59:48.403 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-05-03T17:59:48.403 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-05-03T17:59:48.403 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-05-03T17:59:48.403 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-05-03T17:59:48.403 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-05-03T17:59:48.403 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-05-03T17:59:48.403 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-05-03T17:59:48.403 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-05-03T17:59:48.403 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-05-03T17:59:48.403 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-05-03T17:59:48.403 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-05-03T17:59:48.403 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-05-03T17:59:48.403 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-05-03T17:59:48.404 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-05-03T17:59:48.404 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-05-03T17:59:48.404 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-05-03T17:59:48.404 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-05-03T17:59:48.404 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-05-03T17:59:48.404 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-05-03T17:59:48.404 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-05-03T17:59:48.404 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-05-03T17:59:48.404 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-05-03T17:59:48.404 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-05-03T17:59:48.404 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-05-03T17:59:48.404 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-05-03T17:59:48.404 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-05-03T17:59:48.404 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-05-03T17:59:48.405 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-05-03T17:59:48.405 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-05-03T17:59:48.405 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-05-03T17:59:48.405 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-05-03T17:59:48.405 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-05-03T17:59:48.405 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-05-03T17:59:48.405 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-05-03T17:59:48.405 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-05-03T17:59:48.405 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-05-03T17:59:48.405 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-05-03T17:59:48.405 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-05-03T17:59:48.405 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-05-03T17:59:48.405 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-05-03T17:59:48.406 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-05-03T17:59:48.406 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-05-03T17:59:48.406 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-05-03T17:59:48.406 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-05-03T17:59:48.406 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-05-03T17:59:48.406 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-05-03T17:59:48.406 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-05-03T17:59:48.406 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-05-03T17:59:48.406 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-05-03T17:59:48.406 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-05-03T17:59:48.406 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-05-03T17:59:48.406 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-05-03T17:59:48.407 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-05-03T17:59:48.407 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-05-03T17:59:48.407 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-05-03T17:59:48.407 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-05-03T17:59:48.407 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-05-03T17:59:48.407 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-05-03T17:59:48.407 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-05-03T17:59:48.407 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-05-03T17:59:48.407 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-05-03T17:59:48.407 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-05-03T17:59:48.407 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-05-03T17:59:48.407 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-05-03T17:59:48.408 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-05-03T17:59:48.408 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-05-03T17:59:48.408 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-05-03T17:59:48.408 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-05-03T17:59:48.408 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-05-03T17:59:48.408 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-05-03T17:59:48.408 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-05-03T17:59:48.408 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-05-03T17:59:48.408 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-05-03T17:59:48.408 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-05-03T17:59:48.408 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-05-03T17:59:48.408 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-05-03T17:59:48.408 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-05-03T17:59:48.408 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-05-03T17:59:48.409 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-05-03T17:59:48.409 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-05-03T17:59:48.409 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-05-03T17:59:48.409 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-05-03T17:59:48.409 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-05-03T17:59:48.409 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-05-03T17:59:48.409 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-05-03T17:59:48.409 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-05-03T17:59:48.409 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-05-03T17:59:48.409 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-05-03T17:59:48.409 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-05-03T17:59:48.409 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-05-03T17:59:48.409 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-05-03T17:59:48.409 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-05-03T17:59:48.409 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-05-03T17:59:48.410 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-05-03T17:59:48.410 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-05-03T17:59:48.410 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-05-03T17:59:48.410 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-05-03T17:59:48.410 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-05-03T17:59:48.410 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-05-03T17:59:48.410 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-05-03T17:59:48.410 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-05-03T17:59:48.410 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-05-03T17:59:48.410 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-05-03T17:59:48.410 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-05-03T17:59:48.410 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-05-03T17:59:48.411 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-05-03T17:59:48.411 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-05-03T17:59:48.411 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-05-03T17:59:48.411 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-05-03T17:59:48.411 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-05-03T17:59:48.411 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-05-03T17:59:48.411 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-05-03T17:59:48.411 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-05-03T17:59:48.411 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-05-03T17:59:48.411 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-05-03T17:59:48.411 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-05-03T17:59:48.411 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-05-03T17:59:48.411 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-05-03T17:59:48.411 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-05-03T17:59:48.412 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-05-03T17:59:48.412 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-05-03T17:59:48.412 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-05-03T17:59:48.412 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-05-03T17:59:48.412 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-05-03T17:59:48.412 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-05-03T17:59:48.412 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-05-03T17:59:48.412 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-05-03T17:59:48.412 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-05-03T17:59:48.412 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-05-03T17:59:48.412 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-05-03T17:59:48.412 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-05-03T17:59:48.412 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-05-03T17:59:48.413 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-05-03T17:59:48.413 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-05-03T17:59:48.413 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-05-03T17:59:48.413 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-05-03T17:59:48.413 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-05-03T17:59:48.413 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-05-03T17:59:48.413 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-05-03T17:59:48.413 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-05-03T17:59:48.413 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-05-03T17:59:48.413 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-05-03T17:59:48.413 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-05-03T17:59:48.413 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-05-03T17:59:48.413 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-05-03T17:59:48.413 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-05-03T17:59:48.414 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-05-03T17:59:48.414 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-05-03T17:59:48.414 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-05-03T17:59:48.414 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-05-03T17:59:48.414 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-05-03T17:59:48.414 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-05-03T17:59:48.414 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-05-03T17:59:48.414 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-05-03T17:59:48.414 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-05-03T17:59:48.414 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-05-03T17:59:48.414 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-05-03T17:59:48.414 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-05-03T17:59:48.414 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-05-03T17:59:48.414 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-05-03T17:59:48.415 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-05-03T17:59:48.415 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-05-03T17:59:48.415 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-05-03T17:59:48.415 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-05-03T17:59:48.415 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-05-03T17:59:48.415 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-05-03T17:59:48.415 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-05-03T17:59:48.415 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-05-03T17:59:48.415 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-05-03T17:59:48.415 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-05-03T17:59:48.415 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-05-03T17:59:48.415 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-05-03T17:59:48.415 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-05-03T17:59:48.415 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-05-03T17:59:48.415 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-05-03T17:59:48.416 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-05-03T17:59:48.416 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-05-03T17:59:48.416 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-05-03T17:59:48.416 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-05-03T17:59:48.416 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-05-03T17:59:48.416 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-05-03T17:59:48.416 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-05-03T17:59:48.416 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-05-03T17:59:48.416 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-05-03T17:59:48.416 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-05-03T17:59:48.416 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-05-03T17:59:48.416 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-05-03T17:59:48.416 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-05-03T17:59:48.416 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-05-03T17:59:48.416 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-05-03T17:59:48.417 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-05-03T17:59:48.417 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-05-03T17:59:48.417 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-05-03T17:59:48.417 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-05-03T17:59:48.417 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-05-03T17:59:48.417 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-05-03T17:59:48.417 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-05-03T17:59:48.417 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-05-03T17:59:48.417 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-05-03T17:59:48.417 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-05-03T17:59:48.417 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-05-03T17:59:48.417 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-05-03T17:59:48.417 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-05-03T17:59:48.418 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-05-03T17:59:48.418 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-05-03T17:59:48.418 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-05-03T17:59:48.418 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-05-03T17:59:48.418 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-05-03T17:59:48.418 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-05-03T17:59:48.418 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-05-03T17:59:48.418 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-05-03T17:59:48.418 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-05-03T17:59:48.418 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-05-03T17:59:48.418 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-05-03T17:59:48.418 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-05-03T17:59:48.418 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-05-03T17:59:48.418 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-05-03T17:59:48.419 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-05-03T17:59:48.419 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-05-03T17:59:48.419 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-05-03T17:59:48.419 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-05-03T17:59:48.419 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-05-03T17:59:48.419 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-05-03T17:59:48.419 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-05-03T17:59:48.419 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-05-03T17:59:48.419 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-05-03T17:59:48.419 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-05-03T17:59:48.419 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-05-03T17:59:48.419 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-05-03T17:59:48.419 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-05-03T17:59:48.420 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-05-03T17:59:48.420 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-05-03T17:59:48.420 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-05-03T17:59:48.420 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-05-03T17:59:48.420 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-05-03T17:59:48.420 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-05-03T17:59:48.420 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-05-03T17:59:48.420 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-05-03T17:59:48.420 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-05-03T17:59:48.420 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-05-03T17:59:48.420 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-05-03T17:59:48.420 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-05-03T17:59:48.420 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-05-03T17:59:48.421 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-05-03T17:59:48.421 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-05-03T17:59:48.421 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-05-03T17:59:48.421 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-05-03T17:59:48.421 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-05-03T17:59:48.421 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-05-03T17:59:48.421 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-05-03T17:59:48.421 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-05-03T17:59:48.421 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-05-03T17:59:48.421 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-05-03T17:59:48.421 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-05-03T17:59:48.421 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-05-03T17:59:48.421 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-05-03T17:59:48.421 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-05-03T17:59:48.421 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-05-03T17:59:48.421 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-05-03T17:59:48.422 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-05-03T17:59:48.422 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-05-03T17:59:48.422 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-05-03T17:59:48.422 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-05-03T17:59:48.422 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-05-03T17:59:48.422 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-05-03T17:59:48.422 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-05-03T17:59:48.422 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-05-03T17:59:48.422 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-05-03T17:59:48.422 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-05-03T17:59:48.422 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-05-03T17:59:48.422 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-05-03T17:59:48.422 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-05-03T17:59:48.422 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-05-03T17:59:48.423 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-05-03T17:59:48.423 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-05-03T17:59:48.423 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-05-03T17:59:48.423 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-05-03T17:59:48.423 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-05-03T17:59:48.423 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-05-03T17:59:48.423 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-05-03T17:59:48.423 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-05-03T17:59:48.423 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-05-03T17:59:48.423 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-05-03T17:59:48.423 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-05-03T17:59:48.423 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-05-03T17:59:48.423 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-05-03T17:59:48.423 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-05-03T17:59:48.423 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-05-03T17:59:48.423 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-05-03T17:59:48.424 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-05-03T17:59:48.424 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-05-03T17:59:48.424 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-05-03T17:59:48.424 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-05-03T17:59:48.424 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-05-03T17:59:48.424 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-05-03T17:59:48.424 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-05-03T17:59:48.424 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-05-03T17:59:48.424 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-05-03T17:59:48.424 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-05-03T17:59:48.424 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-05-03T17:59:48.424 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-05-03T17:59:48.424 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-05-03T17:59:48.424 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-05-03T17:59:48.425 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-05-03T17:59:48.425 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-05-03T17:59:48.425 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-05-03T17:59:48.425 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-05-03T17:59:48.425 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-05-03T17:59:48.425 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-05-03T17:59:48.425 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-05-03T17:59:48.425 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-05-03T17:59:48.425 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-05-03T17:59:48.425 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-05-03T17:59:48.425 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-05-03T17:59:48.425 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-05-03T17:59:48.425 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-05-03T17:59:48.425 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-05-03T17:59:48.425 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-05-03T17:59:48.425 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-05-03T17:59:48.426 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-05-03T17:59:48.426 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-05-03T17:59:48.426 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-05-03T17:59:48.426 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-05-03T17:59:48.426 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-05-03T17:59:48.426 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-05-03T17:59:48.426 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-05-03T17:59:48.426 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-05-03T17:59:48.426 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-05-03T17:59:48.426 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-05-03T17:59:48.426 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-05-03T17:59:48.426 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-05-03T17:59:48.426 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-05-03T17:59:48.426 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-05-03T17:59:48.427 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-05-03T17:59:48.427 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-05-03T17:59:48.427 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-05-03T17:59:48.427 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-05-03T17:59:48.427 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-05-03T17:59:48.427 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-05-03T17:59:48.427 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-05-03T17:59:48.427 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-05-03T17:59:48.427 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-05-03T17:59:48.427 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-05-03T17:59:48.427 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-05-03T17:59:48.427 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-05-03T17:59:48.427 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-05-03T17:59:48.428 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-05-03T17:59:48.428 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-05-03T17:59:48.428 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-05-03T17:59:48.428 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-05-03T17:59:48.428 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-05-03T17:59:48.428 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-05-03T17:59:48.428 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-05-03T17:59:48.428 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-05-03T17:59:48.428 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-05-03T17:59:48.428 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-05-03T17:59:48.428 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-05-03T17:59:48.428 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-05-03T17:59:48.428 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-05-03T17:59:48.429 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-05-03T17:59:48.429 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-05-03T17:59:48.429 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-05-03T17:59:48.429 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-05-03T17:59:48.429 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-05-03T17:59:48.429 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-05-03T17:59:48.429 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-05-03T17:59:48.429 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-05-03T17:59:48.429 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-05-03T17:59:48.429 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-05-03T17:59:48.429 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-05-03T17:59:48.429 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-05-03T17:59:48.429 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-05-03T17:59:48.430 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-05-03T17:59:48.430 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-05-03T17:59:48.430 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-05-03T17:59:48.430 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-05-03T17:59:48.430 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-05-03T17:59:48.430 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-05-03T17:59:48.430 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-05-03T17:59:48.430 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-05-03T17:59:48.430 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-05-03T17:59:48.430 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-05-03T17:59:48.430 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-05-03T17:59:48.430 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-05-03T17:59:48.430 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-05-03T17:59:48.430 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-05-03T17:59:48.430 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-05-03T17:59:48.431 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-05-03T17:59:48.431 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-05-03T17:59:48.431 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-05-03T17:59:48.431 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-05-03T17:59:48.431 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-05-03T17:59:48.431 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-05-03T17:59:48.431 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-05-03T17:59:48.431 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-05-03T17:59:48.431 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-05-03T17:59:48.431 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-05-03T17:59:48.431 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-05-03T17:59:48.431 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-05-03T17:59:48.431 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-05-03T17:59:48.432 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-05-03T17:59:48.432 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-05-03T17:59:48.432 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-05-03T17:59:48.432 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-05-03T17:59:48.432 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-05-03T17:59:48.432 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-05-03T17:59:48.432 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-05-03T17:59:48.432 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-05-03T17:59:48.432 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-05-03T17:59:48.432 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-05-03T17:59:48.432 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-05-03T17:59:48.433 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-05-03T17:59:48.433 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-05-03T17:59:48.433 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-05-03T17:59:48.433 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-05-03T17:59:48.433 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-05-03T17:59:48.433 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-05-03T17:59:48.433 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-05-03T17:59:48.433 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-05-03T17:59:48.433 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-05-03T17:59:48.433 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-05-03T17:59:48.433 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-05-03T17:59:48.433 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-05-03T17:59:48.434 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-05-03T17:59:48.434 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-05-03T17:59:48.434 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-05-03T17:59:48.434 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-05-03T17:59:48.434 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-05-03T17:59:48.434 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-05-03T17:59:48.434 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-05-03T17:59:48.434 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-05-03T17:59:48.434 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-05-03T17:59:48.434 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-05-03T17:59:48.434 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-05-03T17:59:48.434 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-05-03T17:59:48.434 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-05-03T17:59:48.435 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-05-03T17:59:48.435 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-05-03T17:59:48.435 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-05-03T17:59:48.435 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-05-03T17:59:48.435 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-05-03T17:59:48.435 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-05-03T17:59:48.435 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-05-03T17:59:48.435 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-05-03T17:59:48.435 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-05-03T17:59:48.435 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-05-03T17:59:48.435 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-05-03T17:59:48.435 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-05-03T17:59:48.435 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-05-03T17:59:48.435 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-05-03T17:59:48.436 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-05-03T17:59:48.436 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-05-03T17:59:48.436 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-05-03T17:59:48.436 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-05-03T17:59:48.436 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-05-03T17:59:48.436 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-05-03T17:59:48.436 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-05-03T17:59:48.436 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-05-03T17:59:48.436 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-05-03T17:59:48.436 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-05-03T17:59:48.436 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-05-03T17:59:48.436 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-05-03T17:59:48.436 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-05-03T17:59:48.436 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-05-03T17:59:48.436 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-05-03T17:59:48.437 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-05-03T17:59:48.437 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-05-03T17:59:48.437 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-05-03T17:59:48.437 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-05-03T17:59:48.437 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-05-03T17:59:48.437 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-05-03T17:59:48.437 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-05-03T17:59:48.437 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-05-03T17:59:48.437 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-05-03T17:59:48.437 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-05-03T17:59:48.437 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-05-03T17:59:48.437 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-05-03T17:59:48.437 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-05-03T17:59:48.438 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-05-03T17:59:48.438 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-05-03T17:59:48.438 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-05-03T17:59:48.438 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-05-03T17:59:48.438 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-05-03T17:59:48.438 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-05-03T17:59:48.438 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-05-03T17:59:48.438 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-05-03T17:59:48.438 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-05-03T17:59:48.438 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-05-03T17:59:48.438 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-05-03T17:59:48.438 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-05-03T17:59:48.439 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-05-03T17:59:48.439 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-05-03T17:59:48.439 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-05-03T17:59:48.439 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-05-03T17:59:48.439 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-05-03T17:59:48.439 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-05-03T17:59:48.439 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-05-03T17:59:48.439 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-05-03T17:59:48.439 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-05-03T17:59:48.439 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-05-03T17:59:48.439 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-05-03T17:59:48.439 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-05-03T17:59:48.439 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-05-03T17:59:48.439 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-05-03T17:59:48.439 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-05-03T17:59:48.440 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-05-03T17:59:48.440 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-05-03T17:59:48.440 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-05-03T17:59:48.440 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-05-03T17:59:48.440 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-05-03T17:59:48.440 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-05-03T17:59:48.440 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-05-03T17:59:48.440 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-05-03T17:59:48.440 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-05-03T17:59:48.440 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-05-03T17:59:48.440 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-05-03T17:59:48.440 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-05-03T17:59:48.440 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-05-03T17:59:48.440 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-05-03T17:59:48.441 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-05-03T17:59:48.441 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-05-03T17:59:48.441 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-05-03T17:59:48.441 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-05-03T17:59:48.441 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-05-03T17:59:48.441 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-05-03T17:59:48.441 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-05-03T17:59:48.441 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-05-03T17:59:48.441 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-05-03T17:59:48.441 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-05-03T17:59:48.441 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-05-03T17:59:48.441 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-05-03T17:59:48.441 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-05-03T17:59:48.441 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-05-03T17:59:48.441 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-05-03T17:59:48.441 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-05-03T17:59:48.442 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-05-03T17:59:48.442 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-05-03T17:59:48.442 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-05-03T17:59:48.442 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-05-03T17:59:48.442 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-05-03T17:59:48.442 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-05-03T17:59:48.442 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-05-03T17:59:48.442 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-05-03T17:59:48.442 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-05-03T17:59:48.442 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-05-03T17:59:48.442 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-05-03T17:59:48.442 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-05-03T17:59:48.442 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-05-03T17:59:48.443 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-05-03T17:59:48.443 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-05-03T17:59:48.443 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-05-03T17:59:48.443 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-05-03T17:59:48.443 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-05-03T17:59:48.443 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-05-03T17:59:48.443 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-05-03T17:59:48.443 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-05-03T17:59:48.443 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-05-03T17:59:48.443 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-05-03T17:59:48.443 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-05-03T17:59:48.443 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-05-03T17:59:48.443 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-05-03T17:59:48.443 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-05-03T17:59:48.443 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-05-03T17:59:48.444 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-05-03T17:59:48.444 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-05-03T17:59:48.444 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-05-03T17:59:48.444 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-05-03T17:59:48.444 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-05-03T17:59:48.444 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-05-03T17:59:48.444 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-05-03T17:59:48.444 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-05-03T17:59:48.444 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-05-03T17:59:48.444 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-05-03T17:59:48.444 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-05-03T17:59:48.444 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-05-03T17:59:48.444 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-05-03T17:59:48.444 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-05-03T17:59:48.444 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-05-03T17:59:48.444 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-05-03T17:59:48.445 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-05-03T17:59:48.445 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-05-03T17:59:48.445 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-05-03T17:59:48.445 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-05-03T17:59:48.445 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-05-03T17:59:48.445 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-05-03T17:59:48.445 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-05-03T17:59:48.445 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-05-03T17:59:48.445 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-05-03T17:59:48.445 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-05-03T17:59:48.445 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-05-03T17:59:48.445 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-05-03T17:59:48.445 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-05-03T17:59:48.445 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-05-03T17:59:48.445 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-05-03T17:59:48.446 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-05-03T17:59:48.446 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-05-03T17:59:48.446 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-05-03T17:59:48.446 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-05-03T17:59:48.446 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-05-03T17:59:48.446 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-05-03T17:59:48.446 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-05-03T17:59:48.446 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-05-03T17:59:48.446 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-05-03T17:59:48.446 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-05-03T17:59:48.446 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-05-03T17:59:48.446 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-05-03T17:59:48.446 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-05-03T17:59:48.446 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-05-03T17:59:48.447 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-05-03T17:59:48.447 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-05-03T17:59:48.447 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-05-03T17:59:48.447 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-05-03T17:59:48.447 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-05-03T17:59:48.447 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-05-03T17:59:48.447 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-05-03T17:59:48.447 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-05-03T17:59:48.447 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-05-03T17:59:48.447 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-05-03T17:59:48.447 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-05-03T17:59:48.447 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-05-03T17:59:48.447 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-05-03T17:59:48.447 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-05-03T17:59:48.447 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-05-03T17:59:48.448 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-05-03T17:59:48.448 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-05-03T17:59:48.448 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-05-03T17:59:48.448 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-05-03T17:59:48.448 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-05-03T17:59:48.448 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-05-03T17:59:48.448 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-05-03T17:59:48.448 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-05-03T17:59:48.448 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-05-03T17:59:48.448 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-05-03T17:59:48.448 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-05-03T17:59:48.448 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-05-03T17:59:48.448 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-05-03T17:59:48.448 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-05-03T17:59:48.448 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-05-03T17:59:48.449 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-05-03T17:59:48.449 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-05-03T17:59:48.449 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-05-03T17:59:48.449 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-05-03T17:59:48.449 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-05-03T17:59:48.449 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-05-03T17:59:48.449 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-05-03T17:59:48.449 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-05-03T17:59:48.449 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-05-03T17:59:48.449 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-05-03T17:59:48.449 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-05-03T17:59:48.449 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-05-03T17:59:48.449 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-05-03T17:59:48.449 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-05-03T17:59:48.449 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-05-03T17:59:48.449 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-05-03T17:59:48.450 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-05-03T17:59:48.450 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-05-03T17:59:48.450 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-05-03T17:59:48.450 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-05-03T17:59:48.450 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-05-03T17:59:48.450 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-05-03T17:59:48.450 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-05-03T17:59:48.450 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-05-03T17:59:48.450 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-05-03T17:59:48.450 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-05-03T17:59:48.450 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-05-03T17:59:48.450 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-05-03T17:59:48.450 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-05-03T17:59:48.450 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-05-03T17:59:48.451 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-05-03T17:59:48.451 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-05-03T17:59:48.451 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-05-03T17:59:48.451 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-05-03T17:59:48.451 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-05-03T17:59:48.451 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-05-03T17:59:48.451 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-05-03T17:59:48.451 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-05-03T17:59:48.451 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-05-03T17:59:48.451 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-05-03T17:59:48.451 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-05-03T17:59:48.451 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-05-03T17:59:48.451 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-05-03T17:59:48.451 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-05-03T17:59:48.452 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-05-03T17:59:48.452 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-05-03T17:59:48.452 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-05-03T17:59:48.452 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-05-03T17:59:48.452 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-05-03T17:59:48.452 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-05-03T17:59:48.452 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-05-03T17:59:48.452 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-05-03T17:59:48.452 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-05-03T17:59:48.452 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-05-03T17:59:48.452 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-05-03T17:59:48.452 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-05-03T17:59:48.452 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-05-03T17:59:48.452 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-05-03T17:59:48.453 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-05-03T17:59:48.453 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-05-03T17:59:48.453 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-05-03T17:59:48.453 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-05-03T17:59:48.453 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-05-03T17:59:48.453 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-05-03T17:59:48.453 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-05-03T17:59:48.453 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-05-03T17:59:48.453 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-05-03T17:59:48.453 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-05-03T17:59:48.453 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-05-03T17:59:48.453 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-05-03T17:59:48.453 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-05-03T17:59:48.453 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-05-03T17:59:48.453 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-05-03T17:59:48.454 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-05-03T17:59:48.454 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-05-03T17:59:48.454 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-05-03T17:59:48.454 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-05-03T17:59:48.454 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-05-03T17:59:48.454 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-05-03T17:59:48.454 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-05-03T17:59:48.454 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-05-03T17:59:48.454 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-05-03T17:59:48.454 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-05-03T17:59:48.454 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-05-03T17:59:48.454 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-05-03T17:59:48.454 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-05-03T17:59:48.455 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-05-03T17:59:48.455 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-05-03T17:59:48.455 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-05-03T17:59:48.455 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-05-03T17:59:48.455 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-05-03T17:59:48.455 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-05-03T17:59:48.455 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-05-03T17:59:48.455 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-05-03T17:59:48.455 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-05-03T17:59:48.455 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-05-03T17:59:48.455 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-05-03T17:59:48.455 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-05-03T17:59:48.455 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-05-03T17:59:48.455 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-05-03T17:59:48.455 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-05-03T17:59:48.456 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-05-03T17:59:48.456 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-05-03T17:59:48.456 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-05-03T17:59:48.456 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-05-03T17:59:48.456 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-05-03T17:59:48.456 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-05-03T17:59:48.456 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-05-03T17:59:48.456 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-05-03T17:59:48.456 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-05-03T17:59:48.456 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-05-03T17:59:48.456 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-05-03T17:59:48.456 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-05-03T17:59:48.456 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-05-03T17:59:48.456 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-05-03T17:59:48.457 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-05-03T17:59:48.457 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-05-03T17:59:48.457 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-05-03T17:59:48.457 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-05-03T17:59:48.457 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-05-03T17:59:48.457 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-05-03T17:59:48.457 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-05-03T17:59:48.457 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-05-03T17:59:48.457 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-05-03T17:59:48.457 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-05-03T17:59:48.457 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-05-03T17:59:48.457 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-05-03T17:59:48.457 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-05-03T17:59:48.457 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-05-03T17:59:48.457 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-05-03T17:59:48.458 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-05-03T17:59:48.458 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-05-03T17:59:48.458 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-05-03T17:59:48.458 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-05-03T17:59:48.458 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-05-03T17:59:48.458 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-05-03T17:59:48.458 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-05-03T17:59:48.458 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-05-03T17:59:48.458 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-05-03T17:59:48.458 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-05-03T17:59:48.458 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-05-03T17:59:48.458 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-05-03T17:59:48.458 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-05-03T17:59:48.459 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-05-03T17:59:48.459 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-05-03T17:59:48.459 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-05-03T17:59:48.459 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-05-03T17:59:48.459 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-05-03T17:59:48.459 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-05-03T17:59:48.459 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-05-03T17:59:48.459 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-05-03T17:59:48.459 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-05-03T17:59:48.459 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-05-03T17:59:48.459 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-05-03T17:59:48.459 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-05-03T17:59:48.459 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-05-03T17:59:48.460 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-05-03T17:59:48.460 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-05-03T17:59:48.460 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-05-03T17:59:48.460 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-05-03T17:59:48.460 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-05-03T17:59:48.460 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-05-03T17:59:48.460 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-05-03T17:59:48.460 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-05-03T17:59:48.460 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-05-03T17:59:48.460 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-05-03T17:59:48.460 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-05-03T17:59:48.460 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-05-03T17:59:48.460 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-05-03T17:59:48.461 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-05-03T17:59:48.461 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-05-03T17:59:48.461 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-05-03T17:59:48.461 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-05-03T17:59:48.461 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-05-03T17:59:48.461 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-05-03T17:59:48.461 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-05-03T17:59:48.461 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-05-03T17:59:48.461 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-05-03T17:59:48.461 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-05-03T17:59:48.461 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-05-03T17:59:48.461 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-05-03T17:59:48.461 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-05-03T17:59:48.461 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-05-03T17:59:48.461 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-05-03T17:59:48.462 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-05-03T17:59:48.462 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-05-03T17:59:48.462 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-05-03T17:59:48.462 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-05-03T17:59:48.462 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-05-03T17:59:48.462 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-05-03T17:59:48.462 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-05-03T17:59:48.462 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-05-03T17:59:48.462 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-05-03T17:59:48.462 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-05-03T17:59:48.462 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-05-03T17:59:48.462 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-05-03T17:59:48.462 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-05-03T17:59:48.462 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-05-03T17:59:48.463 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-05-03T17:59:48.463 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-05-03T17:59:48.463 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-05-03T17:59:48.463 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-05-03T17:59:48.463 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-05-03T17:59:48.463 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-05-03T17:59:48.463 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-05-03T17:59:48.463 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-05-03T17:59:48.463 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-05-03T17:59:48.463 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-05-03T17:59:48.463 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-05-03T17:59:48.463 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-05-03T17:59:48.463 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-05-03T17:59:48.463 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-05-03T17:59:48.464 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-05-03T17:59:48.464 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-05-03T17:59:48.464 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-05-03T17:59:48.464 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-05-03T17:59:48.464 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-05-03T17:59:48.464 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-05-03T17:59:48.464 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-05-03T17:59:48.464 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-05-03T17:59:48.464 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-05-03T17:59:48.464 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-05-03T17:59:48.464 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-05-03T17:59:48.464 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-05-03T17:59:48.464 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-05-03T17:59:48.464 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-05-03T17:59:48.464 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-05-03T17:59:48.465 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-05-03T17:59:48.465 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-05-03T17:59:48.465 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-05-03T17:59:48.465 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-05-03T17:59:48.465 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-05-03T17:59:48.465 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-05-03T17:59:48.465 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-05-03T17:59:48.465 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-05-03T17:59:48.465 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-05-03T17:59:48.465 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-05-03T17:59:48.465 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-05-03T17:59:48.465 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-05-03T17:59:48.465 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-05-03T17:59:48.465 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-05-03T17:59:48.465 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-05-03T17:59:48.466 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-05-03T17:59:48.466 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-05-03T17:59:48.466 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-05-03T17:59:48.466 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-05-03T17:59:48.466 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-05-03T17:59:48.466 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-05-03T17:59:48.466 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-05-03T17:59:48.466 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-05-03T17:59:48.466 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-05-03T17:59:48.466 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-05-03T17:59:48.466 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-05-03T17:59:48.466 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-05-03T17:59:48.466 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-05-03T17:59:48.466 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-05-03T17:59:48.467 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-05-03T17:59:48.467 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-05-03T17:59:48.467 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-05-03T17:59:48.467 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-05-03T17:59:48.467 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-05-03T17:59:48.467 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-05-03T17:59:48.467 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-05-03T17:59:48.467 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-05-03T17:59:48.467 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-05-03T17:59:48.467 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-05-03T17:59:48.467 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-05-03T17:59:48.467 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-05-03T17:59:48.467 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-05-03T17:59:48.467 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-05-03T17:59:48.467 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-05-03T17:59:48.468 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-05-03T17:59:48.468 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-05-03T17:59:48.468 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-05-03T17:59:48.468 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-05-03T17:59:48.468 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-05-03T17:59:48.468 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-05-03T17:59:48.468 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-05-03T17:59:48.468 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-05-03T17:59:48.468 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-05-03T17:59:48.468 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-05-03T17:59:48.468 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-05-03T17:59:48.468 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-05-03T17:59:48.468 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-05-03T17:59:48.468 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-05-03T17:59:48.468 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-05-03T17:59:48.469 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-05-03T17:59:48.469 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-05-03T17:59:48.469 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-05-03T17:59:48.469 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-05-03T17:59:48.469 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-05-03T17:59:48.469 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-05-03T17:59:48.469 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-05-03T17:59:48.469 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-05-03T17:59:48.469 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-05-03T17:59:48.469 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-05-03T17:59:48.469 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-05-03T17:59:48.469 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-05-03T17:59:48.469 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-05-03T17:59:48.469 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-05-03T17:59:48.469 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-05-03T17:59:48.470 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-05-03T17:59:48.470 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-05-03T17:59:48.470 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-05-03T17:59:48.470 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-05-03T17:59:48.470 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-05-03T17:59:48.470 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-05-03T17:59:48.470 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-05-03T17:59:48.470 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-05-03T17:59:48.470 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-05-03T17:59:48.470 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-05-03T17:59:48.470 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-05-03T17:59:48.470 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-05-03T17:59:48.470 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-05-03T17:59:48.471 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-05-03T17:59:48.471 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-05-03T17:59:48.471 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-05-03T17:59:48.471 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-05-03T17:59:48.471 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-05-03T17:59:48.471 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-05-03T17:59:48.471 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-05-03T17:59:48.471 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-05-03T17:59:48.471 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-05-03T17:59:48.471 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-05-03T17:59:48.471 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-05-03T17:59:48.471 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-05-03T17:59:48.471 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-05-03T17:59:48.471 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-05-03T17:59:48.472 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-05-03T17:59:48.472 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-05-03T17:59:48.472 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-05-03T17:59:48.472 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-05-03T17:59:48.472 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-05-03T17:59:48.472 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-05-03T17:59:48.472 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-05-03T17:59:48.472 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-05-03T17:59:48.472 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-05-03T17:59:48.472 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-05-03T17:59:48.472 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-05-03T17:59:48.472 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-05-03T17:59:48.472 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-05-03T17:59:48.472 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-05-03T17:59:48.473 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-05-03T17:59:48.473 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-05-03T17:59:48.473 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-05-03T17:59:48.473 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-05-03T17:59:48.473 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-05-03T17:59:48.473 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-05-03T17:59:48.473 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-05-03T17:59:48.473 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-05-03T17:59:48.473 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-05-03T17:59:48.473 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-05-03T17:59:48.473 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-05-03T17:59:48.473 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-05-03T17:59:48.473 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-05-03T17:59:48.473 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-05-03T17:59:48.474 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-05-03T17:59:48.474 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-05-03T17:59:48.474 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-05-03T17:59:48.474 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-05-03T17:59:48.474 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-05-03T17:59:48.474 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-05-03T17:59:48.474 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-05-03T17:59:48.474 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-05-03T17:59:48.474 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-05-03T17:59:48.474 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-05-03T17:59:48.474 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-05-03T17:59:48.474 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-05-03T17:59:48.474 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-05-03T17:59:48.475 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-05-03T17:59:48.475 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-05-03T17:59:48.475 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-05-03T17:59:48.475 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-05-03T17:59:48.475 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-05-03T17:59:48.475 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-05-03T17:59:48.475 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-05-03T17:59:48.475 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-05-03T17:59:48.475 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-05-03T17:59:48.475 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-05-03T17:59:48.475 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-05-03T17:59:48.475 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-05-03T17:59:48.475 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-05-03T17:59:48.475 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-05-03T17:59:48.475 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-05-03T17:59:48.475 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-05-03T17:59:48.476 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-05-03T17:59:48.476 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-05-03T17:59:48.476 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-05-03T17:59:48.476 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-05-03T17:59:48.476 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-05-03T17:59:48.476 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-05-03T17:59:48.476 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-05-03T17:59:48.476 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-05-03T17:59:48.476 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-05-03T17:59:48.476 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-05-03T17:59:48.476 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-05-03T17:59:48.476 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-05-03T17:59:48.476 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-05-03T17:59:48.476 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-05-03T17:59:48.476 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-05-03T17:59:48.476 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-05-03T17:59:48.477 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-05-03T17:59:48.477 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-05-03T17:59:48.477 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-05-03T17:59:48.477 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-05-03T17:59:48.477 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-05-03T17:59:48.477 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-05-03T17:59:48.477 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-05-03T17:59:48.477 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-05-03T17:59:48.477 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-05-03T17:59:48.477 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-05-03T17:59:48.477 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-05-03T17:59:48.477 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-05-03T17:59:48.477 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-05-03T17:59:48.477 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-05-03T17:59:48.477 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-05-03T17:59:48.477 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-05-03T17:59:48.478 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-05-03T17:59:48.478 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-05-03T17:59:48.478 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-05-03T17:59:48.478 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-05-03T17:59:48.478 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-05-03T17:59:48.478 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-05-03T17:59:48.478 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-05-03T17:59:48.478 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-05-03T17:59:48.478 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-05-03T17:59:48.478 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-05-03T17:59:48.478 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-05-03T17:59:48.478 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-05-03T17:59:48.478 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-05-03T17:59:48.479 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-05-03T17:59:48.479 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-05-03T17:59:48.479 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-05-03T17:59:48.479 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-05-03T17:59:48.479 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-05-03T17:59:48.479 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-05-03T17:59:48.479 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-05-03T17:59:48.479 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-05-03T17:59:48.479 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-05-03T17:59:48.479 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-05-03T17:59:48.479 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-05-03T17:59:48.479 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-05-03T17:59:48.479 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-05-03T17:59:48.479 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-05-03T17:59:48.480 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-05-03T17:59:48.480 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-05-03T17:59:48.480 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-05-03T17:59:48.480 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-05-03T17:59:48.480 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-05-03T17:59:48.480 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-05-03T17:59:48.480 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-05-03T17:59:48.480 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-05-03T17:59:48.480 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-05-03T17:59:48.480 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-05-03T17:59:48.480 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-05-03T17:59:48.480 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-05-03T17:59:48.480 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-05-03T17:59:48.480 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-05-03T17:59:48.480 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-05-03T17:59:48.480 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-05-03T17:59:48.481 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-05-03T17:59:48.481 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-05-03T17:59:48.481 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-05-03T17:59:48.481 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-05-03T17:59:48.481 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-05-03T17:59:48.481 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-05-03T17:59:48.481 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-05-03T17:59:48.481 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-05-03T17:59:48.481 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-05-03T17:59:48.481 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-05-03T17:59:48.481 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-05-03T17:59:48.481 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-05-03T17:59:48.481 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-05-03T17:59:48.481 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-05-03T17:59:48.481 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-05-03T17:59:48.482 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-05-03T17:59:48.482 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-05-03T17:59:48.482 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-05-03T17:59:48.482 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-05-03T17:59:48.482 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-05-03T17:59:48.482 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-05-03T17:59:48.482 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-05-03T17:59:48.482 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-05-03T17:59:48.482 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-05-03T17:59:48.482 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-05-03T17:59:48.482 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-05-03T17:59:48.482 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-05-03T17:59:48.482 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-05-03T17:59:48.483 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-05-03T17:59:48.483 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-05-03T17:59:48.483 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-05-03T17:59:48.483 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-05-03T17:59:48.483 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-05-03T17:59:48.483 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-05-03T17:59:48.483 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-05-03T17:59:48.483 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-05-03T17:59:48.483 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-05-03T17:59:48.483 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-05-03T17:59:48.483 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-05-03T17:59:48.483 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-05-03T17:59:48.483 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-05-03T17:59:48.483 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-05-03T17:59:48.483 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-05-03T17:59:48.483 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-05-03T17:59:48.484 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-05-03T17:59:48.484 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-05-03T17:59:48.484 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-05-03T17:59:48.484 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-05-03T17:59:48.484 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-05-03T17:59:48.484 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-05-03T17:59:48.484 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-05-03T17:59:48.484 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-05-03T17:59:48.484 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-05-03T17:59:48.484 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-05-03T17:59:48.484 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-05-03T17:59:48.484 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-05-03T17:59:48.484 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-05-03T17:59:48.484 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-05-03T17:59:48.484 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-05-03T17:59:48.485 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-05-03T17:59:48.485 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-05-03T17:59:48.485 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-05-03T17:59:48.485 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-05-03T17:59:48.485 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-05-03T17:59:48.485 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-05-03T17:59:48.485 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-05-03T17:59:48.485 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-05-03T17:59:48.485 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-05-03T17:59:48.485 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-05-03T17:59:48.485 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-05-03T17:59:48.485 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-05-03T17:59:48.485 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-05-03T17:59:48.485 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-05-03T17:59:48.485 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-05-03T17:59:48.485 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-05-03T17:59:48.485 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-05-03T17:59:48.486 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-05-03T17:59:48.486 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-05-03T17:59:48.486 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-05-03T17:59:48.486 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-05-03T17:59:48.486 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-05-03T17:59:48.486 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-05-03T17:59:48.486 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-05-03T17:59:48.486 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-05-03T17:59:48.486 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-05-03T17:59:48.486 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-05-03T17:59:48.486 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-05-03T17:59:48.486 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-05-03T17:59:48.486 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-05-03T17:59:48.486 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-05-03T17:59:48.486 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-05-03T17:59:48.487 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-05-03T17:59:48.487 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-05-03T17:59:48.487 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-05-03T17:59:48.487 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-05-03T17:59:48.487 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-05-03T17:59:48.487 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-05-03T17:59:48.487 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-05-03T17:59:48.487 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-05-03T17:59:48.487 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-05-03T17:59:48.487 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-05-03T17:59:48.487 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-05-03T17:59:48.487 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-05-03T17:59:48.487 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-05-03T17:59:48.487 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-05-03T17:59:48.488 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-05-03T17:59:48.488 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-05-03T17:59:48.488 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-05-03T17:59:48.488 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-05-03T17:59:48.488 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-05-03T17:59:48.488 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-05-03T17:59:48.488 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-05-03T17:59:48.488 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-05-03T17:59:48.488 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-05-03T17:59:48.488 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-05-03T17:59:48.488 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-05-03T17:59:48.488 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-05-03T17:59:48.488 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-05-03T17:59:48.489 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-05-03T17:59:48.489 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-05-03T17:59:48.489 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-05-03T17:59:48.489 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-05-03T17:59:48.489 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-05-03T17:59:48.489 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-05-03T17:59:48.489 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-05-03T17:59:48.489 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-05-03T17:59:48.489 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-05-03T17:59:48.489 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-05-03T17:59:48.489 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-05-03T17:59:48.489 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-05-03T17:59:48.489 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-05-03T17:59:48.489 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-05-03T17:59:48.489 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-05-03T17:59:48.490 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-05-03T17:59:48.490 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-05-03T17:59:48.490 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-05-03T17:59:48.490 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-05-03T17:59:48.490 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-05-03T17:59:48.490 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-05-03T17:59:48.490 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-05-03T17:59:48.490 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-05-03T17:59:48.490 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-05-03T17:59:48.490 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-05-03T17:59:48.490 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-05-03T17:59:48.490 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-05-03T17:59:48.490 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-05-03T17:59:48.490 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-05-03T17:59:48.490 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-05-03T17:59:48.491 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-05-03T17:59:48.491 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-05-03T17:59:48.491 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-05-03T17:59:48.491 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-05-03T17:59:48.491 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-05-03T17:59:48.491 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-05-03T17:59:48.491 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-05-03T17:59:48.491 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-05-03T17:59:48.491 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-05-03T17:59:48.491 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-05-03T17:59:48.491 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-05-03T17:59:48.491 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-05-03T17:59:48.491 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-05-03T17:59:48.491 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-05-03T17:59:48.491 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-05-03T17:59:48.492 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-05-03T17:59:48.492 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-05-03T17:59:48.492 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-05-03T17:59:48.492 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-05-03T17:59:48.492 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-05-03T17:59:48.492 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-05-03T17:59:48.492 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-05-03T17:59:48.492 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-05-03T17:59:48.492 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-05-03T17:59:48.492 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-05-03T17:59:48.492 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-05-03T17:59:48.492 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-05-03T17:59:48.492 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-05-03T17:59:48.492 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-05-03T17:59:48.493 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-05-03T17:59:48.493 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-05-03T17:59:48.493 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-05-03T17:59:48.493 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-05-03T17:59:48.493 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-05-03T17:59:48.493 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-05-03T17:59:48.493 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-05-03T17:59:48.493 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-05-03T17:59:48.493 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-05-03T17:59:48.493 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-05-03T17:59:48.493 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-05-03T17:59:48.493 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-05-03T17:59:48.493 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-05-03T17:59:48.493 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-05-03T17:59:48.493 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-05-03T17:59:48.494 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-05-03T17:59:48.494 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-05-03T17:59:48.494 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-05-03T17:59:48.494 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-05-03T17:59:48.494 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-05-03T17:59:48.494 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-05-03T17:59:48.494 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-05-03T17:59:48.494 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-05-03T17:59:48.494 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-05-03T17:59:48.494 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-05-03T17:59:48.494 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-05-03T17:59:48.494 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-05-03T17:59:48.494 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-05-03T17:59:48.494 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-05-03T17:59:48.495 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-05-03T17:59:48.495 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-05-03T17:59:48.495 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-05-03T17:59:48.495 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-05-03T17:59:48.495 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-05-03T17:59:48.495 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-05-03T17:59:48.495 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-05-03T17:59:48.495 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-05-03T17:59:48.495 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-05-03T17:59:48.495 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-05-03T17:59:48.495 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-05-03T17:59:48.495 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-05-03T17:59:48.495 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-05-03T17:59:48.495 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-05-03T17:59:48.496 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-05-03T17:59:48.496 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-05-03T17:59:48.496 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-05-03T17:59:48.496 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-05-03T17:59:48.496 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-05-03T17:59:48.496 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-05-03T17:59:48.496 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-05-03T17:59:48.496 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-05-03T17:59:48.496 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-05-03T17:59:48.496 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-05-03T17:59:48.496 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-05-03T17:59:48.496 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-05-03T17:59:48.497 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-05-03T17:59:48.497 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-05-03T17:59:48.497 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-05-03T17:59:48.497 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-05-03T17:59:48.497 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-05-03T17:59:48.497 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-05-03T17:59:48.497 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-05-03T17:59:48.497 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-05-03T17:59:48.497 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-05-03T17:59:48.497 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-05-03T17:59:48.497 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-05-03T17:59:48.497 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-05-03T17:59:48.497 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-05-03T17:59:48.497 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-05-03T17:59:48.498 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-05-03T17:59:48.498 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-05-03T17:59:48.498 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-05-03T17:59:48.498 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-05-03T17:59:48.498 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-05-03T17:59:48.498 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-05-03T17:59:48.498 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-05-03T17:59:48.498 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-05-03T17:59:48.498 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-05-03T17:59:48.498 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-05-03T17:59:48.498 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-05-03T17:59:48.498 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-05-03T17:59:48.498 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-05-03T17:59:48.498 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-05-03T17:59:48.498 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-05-03T17:59:48.499 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-05-03T17:59:48.499 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-05-03T17:59:48.499 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-05-03T17:59:48.499 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-05-03T17:59:48.499 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-05-03T17:59:48.499 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-05-03T17:59:48.499 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-05-03T17:59:48.499 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-05-03T17:59:48.499 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-05-03T17:59:48.499 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-05-03T17:59:48.499 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-05-03T17:59:48.499 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-05-03T17:59:48.500 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-05-03T17:59:48.500 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-05-03T17:59:48.500 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-05-03T17:59:48.500 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-05-03T17:59:48.500 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-05-03T17:59:48.500 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-05-03T17:59:48.500 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-05-03T17:59:48.500 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-05-03T17:59:48.500 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-05-03T17:59:48.500 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-05-03T17:59:48.500 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-05-03T17:59:48.500 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-05-03T17:59:48.500 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-05-03T17:59:48.500 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-05-03T17:59:48.500 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-05-03T17:59:48.501 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-05-03T17:59:48.501 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-05-03T17:59:48.501 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-05-03T17:59:48.501 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-05-03T17:59:48.501 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-05-03T17:59:48.501 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-05-03T17:59:48.501 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-05-03T17:59:48.501 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-05-03T17:59:48.501 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-05-03T17:59:48.501 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-05-03T17:59:48.501 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-05-03T17:59:48.501 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-05-03T17:59:48.501 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-05-03T17:59:48.501 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-05-03T17:59:48.502 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-05-03T17:59:48.502 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-05-03T17:59:48.502 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-05-03T17:59:48.502 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-05-03T17:59:48.502 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-05-03T17:59:48.502 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-05-03T17:59:48.502 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-05-03T17:59:48.502 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-05-03T17:59:48.502 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-05-03T17:59:48.502 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-05-03T17:59:48.502 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-05-03T17:59:48.502 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-05-03T17:59:48.502 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-05-03T17:59:48.502 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-05-03T17:59:48.503 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-05-03T17:59:48.503 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-05-03T17:59:48.503 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-05-03T17:59:48.503 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-05-03T17:59:48.503 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-05-03T17:59:48.503 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-05-03T17:59:48.503 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-05-03T17:59:48.503 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-05-03T17:59:48.503 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-05-03T17:59:48.503 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-05-03T17:59:48.503 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-05-03T17:59:48.503 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-05-03T17:59:48.503 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-05-03T17:59:48.503 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-05-03T17:59:48.504 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-05-03T17:59:48.504 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-05-03T17:59:48.504 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-05-03T17:59:48.504 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-05-03T17:59:48.504 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-05-03T17:59:48.504 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-05-03T17:59:48.504 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-05-03T17:59:48.504 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-05-03T17:59:48.504 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-05-03T17:59:48.504 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-05-03T17:59:48.504 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-05-03T17:59:48.504 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-05-03T17:59:48.504 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-05-03T17:59:48.504 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-05-03T17:59:48.504 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-05-03T17:59:48.505 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-05-03T17:59:48.505 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-05-03T17:59:48.505 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-05-03T17:59:48.505 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-05-03T17:59:48.505 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-05-03T17:59:48.505 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-05-03T17:59:48.505 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-05-03T17:59:48.505 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-05-03T17:59:48.505 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-05-03T17:59:48.505 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-05-03T17:59:48.505 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-05-03T17:59:48.505 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-05-03T17:59:48.505 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-05-03T17:59:48.505 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-05-03T17:59:48.505 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-05-03T17:59:48.506 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-05-03T17:59:48.506 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-05-03T17:59:48.506 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-05-03T17:59:48.506 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-05-03T17:59:48.506 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-05-03T17:59:48.506 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-05-03T17:59:48.506 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-05-03T17:59:48.506 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-05-03T17:59:48.506 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-05-03T17:59:48.506 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-05-03T17:59:48.506 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-05-03T17:59:48.506 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-05-03T17:59:48.506 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-05-03T17:59:48.506 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-05-03T17:59:48.507 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-05-03T17:59:48.507 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-05-03T17:59:48.507 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-05-03T17:59:48.507 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-05-03T17:59:48.507 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-05-03T17:59:48.507 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-05-03T17:59:48.507 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-05-03T17:59:48.507 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-05-03T17:59:48.507 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-05-03T17:59:48.507 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-05-03T17:59:48.507 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-05-03T17:59:48.507 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-05-03T17:59:48.507 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-05-03T17:59:48.507 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-05-03T17:59:48.507 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-05-03T17:59:48.508 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-05-03T17:59:48.508 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-05-03T17:59:48.508 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-05-03T17:59:48.508 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-05-03T17:59:48.508 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-05-03T17:59:48.508 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-05-03T17:59:48.508 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-05-03T17:59:48.508 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-05-03T17:59:48.508 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-05-03T17:59:48.508 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-05-03T17:59:48.508 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-05-03T17:59:48.508 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-05-03T17:59:48.508 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-05-03T17:59:48.508 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-05-03T17:59:48.509 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-05-03T17:59:48.509 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-05-03T17:59:48.509 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-05-03T17:59:48.509 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-05-03T17:59:48.509 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-05-03T17:59:48.509 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-05-03T17:59:48.509 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-05-03T17:59:48.509 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-05-03T17:59:48.509 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-05-03T17:59:48.509 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-05-03T17:59:48.509 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-05-03T17:59:48.509 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-05-03T17:59:48.509 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-05-03T17:59:48.509 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-05-03T17:59:48.510 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-05-03T17:59:48.510 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-05-03T17:59:48.510 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-05-03T17:59:48.510 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-05-03T17:59:48.510 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-05-03T17:59:48.510 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-05-03T17:59:48.510 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-05-03T17:59:48.510 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-05-03T17:59:48.510 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-05-03T17:59:48.510 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-05-03T17:59:48.510 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-05-03T17:59:48.510 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-05-03T17:59:48.510 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-05-03T17:59:48.510 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-05-03T17:59:48.510 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-05-03T17:59:48.511 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-05-03T17:59:48.511 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-05-03T17:59:48.511 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-05-03T17:59:48.511 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-05-03T17:59:48.511 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-05-03T17:59:48.511 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-05-03T17:59:48.511 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-05-03T17:59:48.511 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-05-03T17:59:48.511 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-05-03T17:59:48.511 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-05-03T17:59:48.511 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-05-03T17:59:48.511 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-05-03T17:59:48.511 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-05-03T17:59:48.511 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-05-03T17:59:48.512 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-05-03T17:59:48.512 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-05-03T17:59:48.512 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-05-03T17:59:48.512 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-05-03T17:59:48.512 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-05-03T17:59:48.512 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-05-03T17:59:48.512 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-05-03T17:59:48.512 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-05-03T17:59:48.512 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-05-03T17:59:48.512 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-05-03T17:59:48.512 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-05-03T17:59:48.512 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-05-03T17:59:48.512 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-05-03T17:59:48.512 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-05-03T17:59:48.512 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-05-03T17:59:48.513 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-05-03T17:59:48.513 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-05-03T17:59:48.513 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-05-03T17:59:48.513 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-05-03T17:59:48.513 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-05-03T17:59:48.513 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-05-03T17:59:48.513 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-05-03T17:59:48.513 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-05-03T17:59:48.513 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-05-03T17:59:48.513 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-05-03T17:59:48.513 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-05-03T17:59:48.513 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-05-03T17:59:48.513 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-05-03T17:59:48.513 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-05-03T17:59:48.514 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-05-03T17:59:48.514 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-05-03T17:59:48.514 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-05-03T17:59:48.514 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-05-03T17:59:48.514 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-05-03T17:59:48.514 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-05-03T17:59:48.514 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-05-03T17:59:48.514 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-05-03T17:59:48.514 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-05-03T17:59:48.514 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-05-03T17:59:48.514 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-05-03T17:59:48.514 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-05-03T17:59:48.514 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-05-03T17:59:48.514 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-05-03T17:59:48.515 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-05-03T17:59:48.515 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-05-03T17:59:48.515 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-05-03T17:59:48.515 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-05-03T17:59:48.515 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-05-03T17:59:48.515 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-05-03T17:59:48.515 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-05-03T17:59:48.515 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-05-03T17:59:48.515 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-05-03T17:59:48.515 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-05-03T17:59:48.515 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-05-03T17:59:48.515 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-05-03T17:59:48.515 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-05-03T17:59:48.515 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-05-03T17:59:48.515 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-05-03T17:59:48.516 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-05-03T17:59:48.516 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-05-03T17:59:48.516 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-05-03T17:59:48.516 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-05-03T17:59:48.516 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-05-03T17:59:48.516 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-05-03T17:59:48.516 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-05-03T17:59:48.516 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-05-03T17:59:48.516 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-05-03T17:59:48.516 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-05-03T17:59:48.516 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-05-03T17:59:48.516 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-05-03T17:59:48.516 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-05-03T17:59:48.517 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-05-03T17:59:48.517 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-05-03T17:59:48.517 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-05-03T17:59:48.517 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-05-03T17:59:48.517 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-05-03T17:59:48.517 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-05-03T17:59:48.517 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-05-03T17:59:48.517 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-05-03T17:59:48.517 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-05-03T17:59:48.517 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-05-03T17:59:48.517 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-05-03T17:59:48.517 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-05-03T17:59:48.517 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-05-03T17:59:48.517 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-05-03T17:59:48.517 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-05-03T17:59:48.518 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-05-03T17:59:48.518 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-05-03T17:59:48.518 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-05-03T17:59:48.518 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-05-03T17:59:48.518 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-05-03T17:59:48.518 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-05-03T17:59:48.518 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-05-03T17:59:48.518 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-05-03T17:59:48.518 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-05-03T17:59:48.518 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-05-03T17:59:48.518 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-05-03T17:59:48.518 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-05-03T17:59:48.518 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-05-03T17:59:48.518 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-05-03T17:59:48.519 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-05-03T17:59:48.519 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-05-03T17:59:48.519 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-05-03T17:59:48.519 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-05-03T17:59:48.519 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-05-03T17:59:48.519 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-05-03T17:59:48.519 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-05-03T17:59:48.519 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-05-03T17:59:48.519 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-05-03T17:59:48.519 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-05-03T17:59:48.519 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-05-03T17:59:48.519 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-05-03T17:59:48.520 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-05-03T17:59:48.520 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-05-03T17:59:48.520 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-05-03T17:59:48.520 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-05-03T17:59:48.520 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-05-03T17:59:48.520 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-05-03T17:59:48.520 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-05-03T17:59:48.520 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-05-03T17:59:48.520 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-05-03T17:59:48.520 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-05-03T17:59:48.520 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-05-03T17:59:48.520 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-05-03T17:59:48.521 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-05-03T17:59:48.521 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-05-03T17:59:48.521 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-05-03T17:59:48.521 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-05-03T17:59:48.521 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-05-03T17:59:48.521 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-05-03T17:59:48.521 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-05-03T17:59:48.521 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-05-03T17:59:48.521 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-05-03T17:59:48.521 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-05-03T17:59:48.521 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-05-03T17:59:48.521 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-05-03T17:59:48.521 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-05-03T17:59:48.521 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-05-03T17:59:48.522 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-05-03T17:59:48.522 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-05-03T17:59:48.522 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-05-03T17:59:48.522 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-05-03T17:59:48.522 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-05-03T17:59:48.522 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-05-03T17:59:48.522 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-05-03T17:59:48.522 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-05-03T17:59:48.522 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-05-03T17:59:48.522 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-05-03T17:59:48.522 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-05-03T17:59:48.522 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-05-03T17:59:48.522 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-05-03T17:59:48.522 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-05-03T17:59:48.523 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-05-03T17:59:48.523 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-05-03T17:59:48.523 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-05-03T17:59:48.523 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-05-03T17:59:48.523 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-05-03T17:59:48.523 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-05-03T17:59:48.523 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-05-03T17:59:48.523 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-05-03T17:59:48.523 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-05-03T17:59:48.523 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-05-03T17:59:48.523 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-05-03T17:59:48.523 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-05-03T17:59:48.523 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-05-03T17:59:48.524 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-05-03T17:59:48.524 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-05-03T17:59:48.524 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-05-03T17:59:48.524 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-05-03T17:59:48.524 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-05-03T17:59:48.524 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-05-03T17:59:48.524 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-05-03T17:59:48.524 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-05-03T17:59:48.524 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-05-03T17:59:48.524 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-05-03T17:59:48.524 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-05-03T17:59:48.524 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-05-03T17:59:48.524 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-05-03T17:59:48.524 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-05-03T17:59:48.524 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-05-03T17:59:48.525 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-05-03T17:59:48.525 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-05-03T17:59:48.525 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-05-03T17:59:48.525 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-05-03T17:59:48.525 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-05-03T17:59:48.525 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-05-03T17:59:48.525 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-05-03T17:59:48.525 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-05-03T17:59:48.525 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-05-03T17:59:48.525 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-05-03T17:59:48.525 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-05-03T17:59:48.525 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-05-03T17:59:48.525 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-05-03T17:59:48.525 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-05-03T17:59:48.525 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-05-03T17:59:48.526 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-05-03T17:59:48.526 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-05-03T17:59:48.526 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-05-03T17:59:48.526 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-05-03T17:59:48.526 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-05-03T17:59:48.526 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-05-03T17:59:48.526 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-05-03T17:59:48.526 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-05-03T17:59:48.526 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-05-03T17:59:48.526 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-05-03T17:59:48.526 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-05-03T17:59:48.526 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-05-03T17:59:48.526 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-05-03T17:59:48.526 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-05-03T17:59:48.527 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-05-03T17:59:48.527 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-05-03T17:59:48.527 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-05-03T17:59:48.527 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-05-03T17:59:48.527 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-05-03T17:59:48.527 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-05-03T17:59:48.527 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-05-03T17:59:48.527 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-05-03T17:59:48.527 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-05-03T17:59:48.527 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-05-03T17:59:48.527 INFO:teuthology.orchestra.run.smithi033.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-05-03T17:59:48.531 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-05-03T17:59:48.531 DEBUG:teuthology.orchestra.run.smithi033:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-05-03T17:59:48.554 INFO:teuthology.orchestra.run.smithi033.stdout:postinstall scriptlet (using /bin/sh): 2024-05-03T17:59:48.554 INFO:teuthology.orchestra.run.smithi033.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-05-03T17:59:48.554 INFO:teuthology.orchestra.run.smithi033.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-05-03T17:59:48.554 INFO:teuthology.orchestra.run.smithi033.stdout:fi 2024-05-03T17:59:48.555 INFO:teuthology.orchestra.run.smithi033.stdout:for file in vmlinuz System.map config; do 2024-05-03T17:59:48.555 INFO:teuthology.orchestra.run.smithi033.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-05-03T17:59:48.555 INFO:teuthology.orchestra.run.smithi033.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-05-03T17:59:48.555 INFO:teuthology.orchestra.run.smithi033.stdout: fi 2024-05-03T17:59:48.555 INFO:teuthology.orchestra.run.smithi033.stdout:done 2024-05-03T17:59:48.555 INFO:teuthology.orchestra.run.smithi033.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-05-03T17:59:48.555 INFO:teuthology.orchestra.run.smithi033.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-05-03T17:59:48.555 INFO:teuthology.orchestra.run.smithi033.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-05-03T17:59:48.555 INFO:teuthology.orchestra.run.smithi033.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-05-03T17:59:48.555 INFO:teuthology.orchestra.run.smithi033.stdout:fi 2024-05-03T17:59:48.555 INFO:teuthology.orchestra.run.smithi033.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-05-03T17:59:48.555 INFO:teuthology.orchestra.run.smithi033.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-05-03T17:59:48.555 INFO:teuthology.orchestra.run.smithi033.stdout:fi 2024-05-03T17:59:48.555 INFO:teuthology.orchestra.run.smithi033.stdout:preuninstall scriptlet (using /bin/sh): 2024-05-03T17:59:48.556 INFO:teuthology.orchestra.run.smithi033.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-05-03T17:59:48.556 INFO:teuthology.orchestra.run.smithi033.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-05-03T17:59:48.556 INFO:teuthology.orchestra.run.smithi033.stdout:fi 2024-05-03T17:59:48.556 DEBUG:teuthology.orchestra.run.smithi033:> sudo rpm -qi grub2-tools 2024-05-03T17:59:48.629 INFO:teuthology.orchestra.run.smithi033.stdout:Name : grub2-tools 2024-05-03T17:59:48.629 INFO:teuthology.orchestra.run.smithi033.stdout:Epoch : 1 2024-05-03T17:59:48.630 INFO:teuthology.orchestra.run.smithi033.stdout:Version : 2.06 2024-05-03T17:59:48.630 INFO:teuthology.orchestra.run.smithi033.stdout:Release : 68.el9 2024-05-03T17:59:48.630 INFO:teuthology.orchestra.run.smithi033.stdout:Architecture: x86_64 2024-05-03T17:59:48.630 INFO:teuthology.orchestra.run.smithi033.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-03T17:59:48.630 INFO:teuthology.orchestra.run.smithi033.stdout:Group : Unspecified 2024-05-03T17:59:48.630 INFO:teuthology.orchestra.run.smithi033.stdout:Size : 8264254 2024-05-03T17:59:48.630 INFO:teuthology.orchestra.run.smithi033.stdout:License : GPLv3+ 2024-05-03T17:59:48.630 INFO:teuthology.orchestra.run.smithi033.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-03T17:59:48.630 INFO:teuthology.orchestra.run.smithi033.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-03T17:59:48.630 INFO:teuthology.orchestra.run.smithi033.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-03T17:59:48.630 INFO:teuthology.orchestra.run.smithi033.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-03T17:59:48.630 INFO:teuthology.orchestra.run.smithi033.stdout:Packager : builder@centos.org 2024-05-03T17:59:48.630 INFO:teuthology.orchestra.run.smithi033.stdout:Vendor : CentOS 2024-05-03T17:59:48.631 INFO:teuthology.orchestra.run.smithi033.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-03T17:59:48.631 INFO:teuthology.orchestra.run.smithi033.stdout:Summary : Support tools for GRUB. 2024-05-03T17:59:48.631 INFO:teuthology.orchestra.run.smithi033.stdout:Description : 2024-05-03T17:59:48.631 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T17:59:48.631 INFO:teuthology.orchestra.run.smithi033.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-03T17:59:48.631 INFO:teuthology.orchestra.run.smithi033.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-03T17:59:48.631 INFO:teuthology.orchestra.run.smithi033.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-03T17:59:48.631 INFO:teuthology.orchestra.run.smithi033.stdout:hardware devices. 2024-05-03T17:59:48.631 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T17:59:48.631 INFO:teuthology.orchestra.run.smithi033.stdout:This subpackage provides tools for support of all platforms. 2024-05-03T17:59:48.632 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-03T17:59:48.632 INFO:teuthology.task.kernel:Updating grub on smithi033 to boot 6.9.0-rc3-g806ec36b2112 2024-05-03T17:59:48.632 DEBUG:teuthology.orchestra.run.smithi033:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-03T17:59:49.233 INFO:teuthology.orchestra.run.smithi033.stderr:Generating grub configuration file ... 2024-05-03T17:59:50.605 INFO:teuthology.orchestra.run.smithi033.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-03T17:59:50.639 INFO:teuthology.orchestra.run.smithi033.stderr:done 2024-05-03T17:59:50.641 DEBUG:teuthology.orchestra.run.smithi033:> mktemp 2024-05-03T17:59:50.682 INFO:teuthology.orchestra.run.smithi033.stdout:/tmp/tmp.Jjq527b1ue 2024-05-03T17:59:50.682 DEBUG:teuthology.orchestra.run.smithi033:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.Jjq527b1ue 2024-05-03T17:59:50.747 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 0666 /tmp/tmp.Jjq527b1ue 2024-05-03T17:59:50.884 DEBUG:teuthology.orchestra.remote:smithi033:/tmp/tmp.Jjq527b1ue is 6KB 2024-05-03T17:59:50.933 DEBUG:teuthology.orchestra.run.smithi033:> rm -fr /tmp/tmp.Jjq527b1ue 2024-05-03T17:59:50.948 DEBUG:teuthology.orchestra.run.smithi033:> sudo /bin/ls /boot/loader/entries || true 2024-05-03T17:59:51.010 INFO:teuthology.orchestra.run.smithi033.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-03T17:59:51.010 INFO:teuthology.orchestra.run.smithi033.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-03T17:59:51.010 INFO:teuthology.orchestra.run.smithi033.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-03T17:59:51.011 INFO:teuthology.orchestra.run.smithi033.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-05-03T17:59:51.012 DEBUG:teuthology.orchestra.run.smithi033:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-05-03T17:59:51.178 DEBUG:teuthology.orchestra.run.smithi033:> sudo shutdown -r now 2024-05-03T18:00:10.746 INFO:teuthology.task.kernel:Checking client mon.c for new kernel version... 2024-05-03T18:00:10.748 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi150.front.sepia.ceph.com' 2024-05-03T18:00:10.749 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-03T18:00:14.522 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-05-03T18:00:14.523 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi099.front.sepia.ceph.com' 2024-05-03T18:00:14.523 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-03T18:00:21.187 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-05-03T18:00:21.187 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi033.front.sepia.ceph.com' 2024-05-03T18:00:21.188 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-03T18:00:29.219 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.150 2024-05-03T18:00:33.058 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.99 2024-05-03T18:00:38.224 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi150.front.sepia.ceph.com' 2024-05-03T18:00:38.225 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-03T18:00:39.710 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.33 2024-05-03T18:00:41.282 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.150 2024-05-03T18:00:42.059 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi099.front.sepia.ceph.com' 2024-05-03T18:00:42.060 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-03T18:00:45.118 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.99 2024-05-03T18:00:48.714 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi033.front.sepia.ceph.com' 2024-05-03T18:00:48.715 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-03T18:00:53.288 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi150.front.sepia.ceph.com' 2024-05-03T18:00:53.289 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-03T18:00:57.122 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi099.front.sepia.ceph.com' 2024-05-03T18:00:57.123 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-03T18:01:07.102 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.33 2024-05-03T18:01:11.714 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.150 2024-05-03T18:01:15.550 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.99 2024-05-03T18:01:19.107 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi033.front.sepia.ceph.com' 2024-05-03T18:01:19.107 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-03T18:01:22.174 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.33 2024-05-03T18:01:26.750 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi150.front.sepia.ceph.com' 2024-05-03T18:01:26.751 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-03T18:01:29.822 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.150 2024-05-03T18:01:30.552 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi099.front.sepia.ceph.com' 2024-05-03T18:01:30.552 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-03T18:01:34.185 DEBUG:teuthology.orchestra.run.smithi099:> true 2024-05-03T18:01:34.779 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi099.front.sepia.ceph.com' 2024-05-03T18:01:34.779 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "6.9.0-rc3-g806ec36b2112"... 2024-05-03T18:01:34.779 DEBUG:teuthology.orchestra.run.smithi099:> uname -r 2024-05-03T18:01:34.819 INFO:teuthology.orchestra.run.smithi099.stdout:6.9.0-rc3-g806ec36b2112 2024-05-03T18:01:34.819 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-05-03T18:01:34.819 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-03T18:01:34.819 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-03T18:01:35.820 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-05-03T18:01:35.821 DEBUG:teuthology.orchestra.run.smithi099:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-03T18:01:36.023 INFO:teuthology.orchestra.run.smithi099.stdout:ttyS1 2024-05-03T18:01:36.083 DEBUG:teuthology.parallel:result is None 2024-05-03T18:01:37.175 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi033.front.sepia.ceph.com' 2024-05-03T18:01:37.176 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-03T18:01:37.799 DEBUG:teuthology.orchestra.run.smithi033:> true 2024-05-03T18:01:38.368 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi033.front.sepia.ceph.com' 2024-05-03T18:01:38.368 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "6.9.0-rc3-g806ec36b2112"... 2024-05-03T18:01:38.368 DEBUG:teuthology.orchestra.run.smithi033:> uname -r 2024-05-03T18:01:38.407 INFO:teuthology.orchestra.run.smithi033.stdout:6.9.0-rc3-g806ec36b2112 2024-05-03T18:01:38.407 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-05-03T18:01:38.407 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-03T18:01:38.407 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-03T18:01:39.408 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-05-03T18:01:39.408 DEBUG:teuthology.orchestra.run.smithi033:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-03T18:01:39.521 INFO:teuthology.orchestra.run.smithi033.stdout:ttyS1 2024-05-03T18:01:39.590 DEBUG:teuthology.parallel:result is None 2024-05-03T18:01:47.824 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi150.front.sepia.ceph.com' 2024-05-03T18:01:47.824 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-03T18:01:48.213 DEBUG:teuthology.orchestra.run.smithi150:> true 2024-05-03T18:01:48.823 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi150.front.sepia.ceph.com' 2024-05-03T18:01:48.823 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "6.9.0-rc3-g806ec36b2112"... 2024-05-03T18:01:48.823 DEBUG:teuthology.orchestra.run.smithi150:> uname -r 2024-05-03T18:01:48.864 INFO:teuthology.orchestra.run.smithi150.stdout:6.9.0-rc3-g806ec36b2112 2024-05-03T18:01:48.864 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-05-03T18:01:48.864 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-03T18:01:48.864 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-03T18:01:49.865 INFO:teuthology.task.kernel:Enabling kdb on mon.c... 2024-05-03T18:01:49.865 DEBUG:teuthology.orchestra.run.smithi150:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-03T18:01:49.978 INFO:teuthology.orchestra.run.smithi150.stdout:ttyS1 2024-05-03T18:01:50.036 DEBUG:teuthology.parallel:result is None 2024-05-03T18:01:50.036 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-03T18:01:50.043 INFO:teuthology.task.internal:Creating test directory... 2024-05-03T18:01:50.044 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-03T18:01:50.052 DEBUG:teuthology.orchestra.run.smithi099:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-03T18:01:50.069 DEBUG:teuthology.orchestra.run.smithi150:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-03T18:01:50.144 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-03T18:01:50.151 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-03T18:01:50.177 INFO:teuthology.task.internal:Creating archive directory... 2024-05-03T18:01:50.177 DEBUG:teuthology.orchestra.run.smithi033:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-03T18:01:50.185 DEBUG:teuthology.orchestra.run.smithi099:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-03T18:01:50.192 DEBUG:teuthology.orchestra.run.smithi150:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-03T18:01:50.277 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-03T18:01:50.284 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-03T18:01:50.284 DEBUG:teuthology.orchestra.run.smithi033:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-03T18:01:50.290 DEBUG:teuthology.orchestra.run.smithi099:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-03T18:01:50.303 DEBUG:teuthology.orchestra.run.smithi150:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-03T18:01:50.378 INFO:teuthology.orchestra.run.smithi033.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-03T18:01:50.389 INFO:teuthology.orchestra.run.smithi150.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-03T18:01:50.413 INFO:teuthology.orchestra.run.smithi033.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-03T18:01:50.418 INFO:teuthology.orchestra.run.smithi150.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-03T18:01:50.426 INFO:teuthology.orchestra.run.smithi099.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-03T18:01:50.474 INFO:teuthology.orchestra.run.smithi099.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-03T18:01:50.479 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-03T18:01:50.488 INFO:teuthology.task.internal:Configuring sudo... 2024-05-03T18:01:50.489 DEBUG:teuthology.orchestra.run.smithi033:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-03T18:01:50.496 DEBUG:teuthology.orchestra.run.smithi099:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-03T18:01:50.500 DEBUG:teuthology.orchestra.run.smithi150:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-03T18:01:50.585 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-03T18:01:50.595 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-03T18:01:50.596 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-03T18:01:50.603 DEBUG:teuthology.orchestra.run.smithi099:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-03T18:01:50.610 DEBUG:teuthology.orchestra.run.smithi150:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-03T18:01:50.663 DEBUG:teuthology.orchestra.run.smithi033:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-03T18:01:50.760 DEBUG:teuthology.orchestra.run.smithi033:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-03T18:01:50.864 DEBUG:teuthology.orchestra.run.smithi033:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-03T18:01:50.954 DEBUG:teuthology.orchestra.run.smithi033:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-03T18:01:51.047 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-03T18:01:51.047 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-03T18:01:51.138 DEBUG:teuthology.orchestra.run.smithi099:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-03T18:01:51.225 DEBUG:teuthology.orchestra.run.smithi099:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-03T18:01:51.317 DEBUG:teuthology.orchestra.run.smithi099:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-03T18:01:51.392 DEBUG:teuthology.orchestra.run.smithi099:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-03T18:01:51.458 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-05-03T18:01:51.458 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-03T18:01:51.523 DEBUG:teuthology.orchestra.run.smithi150:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-03T18:01:51.606 DEBUG:teuthology.orchestra.run.smithi150:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-03T18:01:51.688 DEBUG:teuthology.orchestra.run.smithi150:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-03T18:01:51.773 DEBUG:teuthology.orchestra.run.smithi150:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-03T18:01:51.857 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-05-03T18:01:51.858 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-03T18:01:51.938 DEBUG:teuthology.orchestra.run.smithi033:> sudo service rsyslog restart 2024-05-03T18:01:51.946 DEBUG:teuthology.orchestra.run.smithi099:> sudo service rsyslog restart 2024-05-03T18:01:51.952 DEBUG:teuthology.orchestra.run.smithi150:> sudo service rsyslog restart 2024-05-03T18:01:52.051 INFO:teuthology.orchestra.run.smithi099.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-03T18:01:52.055 INFO:teuthology.orchestra.run.smithi033.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-03T18:01:52.088 INFO:teuthology.orchestra.run.smithi150.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-03T18:01:52.598 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-03T18:01:52.608 INFO:teuthology.task.internal:Starting timer... 2024-05-03T18:01:52.608 INFO:teuthology.run_tasks:Running task pcp... 2024-05-03T18:01:52.653 INFO:teuthology.run_tasks:Running task selinux... 2024-05-03T18:01:52.702 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-05-03T18:01:52.703 DEBUG:teuthology.orchestra.run.smithi033:> sudo service auditd rotate 2024-05-03T18:01:52.848 INFO:teuthology.orchestra.run.smithi033.stdout:Rotating logs: 2024-05-03T18:01:52.854 DEBUG:teuthology.orchestra.run.smithi099:> sudo service auditd rotate 2024-05-03T18:01:53.005 INFO:teuthology.orchestra.run.smithi099.stdout:Rotating logs: 2024-05-03T18:01:53.013 DEBUG:teuthology.orchestra.run.smithi150:> sudo service auditd rotate 2024-05-03T18:01:53.166 INFO:teuthology.orchestra.run.smithi150.stdout:Rotating logs: 2024-05-03T18:01:53.171 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-03T18:01:53.171 DEBUG:teuthology.orchestra.run.smithi033:> /usr/sbin/getenforce 2024-05-03T18:01:53.233 INFO:teuthology.orchestra.run.smithi033.stdout:Permissive 2024-05-03T18:01:53.234 DEBUG:teuthology.orchestra.run.smithi099:> /usr/sbin/getenforce 2024-05-03T18:01:53.297 INFO:teuthology.orchestra.run.smithi099.stdout:Permissive 2024-05-03T18:01:53.297 DEBUG:teuthology.orchestra.run.smithi150:> /usr/sbin/getenforce 2024-05-03T18:01:53.361 INFO:teuthology.orchestra.run.smithi150.stdout:Permissive 2024-05-03T18:01:53.362 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi033.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi099.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi150.front.sepia.ceph.com': 'permissive'} 2024-05-03T18:01:53.362 DEBUG:teuthology.orchestra.run.smithi033:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-03T18:01:53.448 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-03T18:01:53.448 DEBUG:teuthology.orchestra.run.smithi099:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-03T18:01:53.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-03T18:01:53.535 DEBUG:teuthology.orchestra.run.smithi150:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-03T18:01:53.613 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-03T18:01:53.614 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-03T18:01:53.614 DEBUG:teuthology.orchestra.run.smithi033:> sudo /usr/sbin/setenforce permissive 2024-05-03T18:01:53.681 DEBUG:teuthology.orchestra.run.smithi099:> sudo /usr/sbin/setenforce permissive 2024-05-03T18:01:53.767 DEBUG:teuthology.orchestra.run.smithi150:> sudo /usr/sbin/setenforce permissive 2024-05-03T18:01:53.840 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-03T18:01:53.850 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-05-03T18:01:53.856 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-05-03T18:01:54.003 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-03T18:01:54.020 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-03T18:01:54.021 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi033.front.sepia.ceph.com,smithi099.front.sepia.ceph.com,smithi150.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-03T18:07:36.459 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi033.front.sepia.ceph.com'), Remote(name='ubuntu@smithi099.front.sepia.ceph.com'), Remote(name='ubuntu@smithi150.front.sepia.ceph.com')] 2024-05-03T18:07:36.460 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi033.front.sepia.ceph.com' 2024-05-03T18:07:36.461 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-03T18:07:36.571 DEBUG:teuthology.orchestra.run.smithi033:> true 2024-05-03T18:07:36.735 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi033.front.sepia.ceph.com' 2024-05-03T18:07:36.735 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi099.front.sepia.ceph.com' 2024-05-03T18:07:36.736 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-03T18:07:36.845 DEBUG:teuthology.orchestra.run.smithi099:> true 2024-05-03T18:07:36.997 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi099.front.sepia.ceph.com' 2024-05-03T18:07:36.997 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi150.front.sepia.ceph.com' 2024-05-03T18:07:36.998 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-03T18:07:37.103 DEBUG:teuthology.orchestra.run.smithi150:> true 2024-05-03T18:07:37.257 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi150.front.sepia.ceph.com' 2024-05-03T18:07:37.258 INFO:teuthology.run_tasks:Running task clock... 2024-05-03T18:07:37.268 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-03T18:07:37.269 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-03T18:07:37.269 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-03T18:07:37.277 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-03T18:07:37.277 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-03T18:07:37.284 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-03T18:07:37.284 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-03T18:07:37.370 INFO:teuthology.orchestra.run.smithi033.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-03T18:07:37.373 INFO:teuthology.orchestra.run.smithi150.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-03T18:07:37.373 INFO:teuthology.orchestra.run.smithi099.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-03T18:07:37.414 INFO:teuthology.orchestra.run.smithi150.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-03T18:07:37.416 INFO:teuthology.orchestra.run.smithi033.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-03T18:07:37.416 INFO:teuthology.orchestra.run.smithi099.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-03T18:07:37.486 INFO:teuthology.orchestra.run.smithi150.stderr:sudo: ntpd: command not found 2024-05-03T18:07:37.488 INFO:teuthology.orchestra.run.smithi099.stderr:sudo: ntpd: command not found 2024-05-03T18:07:37.496 INFO:teuthology.orchestra.run.smithi033.stderr:sudo: ntpd: command not found 2024-05-03T18:07:37.519 INFO:teuthology.orchestra.run.smithi150.stdout:506 Cannot talk to daemon 2024-05-03T18:07:37.519 INFO:teuthology.orchestra.run.smithi099.stdout:506 Cannot talk to daemon 2024-05-03T18:07:37.531 INFO:teuthology.orchestra.run.smithi033.stdout:506 Cannot talk to daemon 2024-05-03T18:07:37.558 INFO:teuthology.orchestra.run.smithi099.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-03T18:07:37.559 INFO:teuthology.orchestra.run.smithi150.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-03T18:07:37.576 INFO:teuthology.orchestra.run.smithi033.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-03T18:07:37.597 INFO:teuthology.orchestra.run.smithi099.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-03T18:07:37.600 INFO:teuthology.orchestra.run.smithi150.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-03T18:07:37.620 INFO:teuthology.orchestra.run.smithi033.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-03T18:07:37.728 INFO:teuthology.orchestra.run.smithi099.stderr:bash: line 1: ntpq: command not found 2024-05-03T18:07:37.737 INFO:teuthology.orchestra.run.smithi099.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-03T18:07:37.737 INFO:teuthology.orchestra.run.smithi099.stdout:=============================================================================== 2024-05-03T18:07:37.737 INFO:teuthology.orchestra.run.smithi099.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-03T18:07:37.737 INFO:teuthology.orchestra.run.smithi099.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-03T18:07:37.737 INFO:teuthology.orchestra.run.smithi099.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-03T18:07:37.737 INFO:teuthology.orchestra.run.smithi099.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-03T18:07:37.739 INFO:teuthology.orchestra.run.smithi150.stderr:bash: line 1: ntpq: command not found 2024-05-03T18:07:37.749 INFO:teuthology.orchestra.run.smithi150.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-03T18:07:37.749 INFO:teuthology.orchestra.run.smithi150.stdout:=============================================================================== 2024-05-03T18:07:37.749 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-03T18:07:37.749 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-03T18:07:37.749 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-03T18:07:37.749 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-03T18:07:37.761 INFO:teuthology.orchestra.run.smithi033.stderr:bash: line 1: ntpq: command not found 2024-05-03T18:07:37.770 INFO:teuthology.orchestra.run.smithi033.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-03T18:07:37.770 INFO:teuthology.orchestra.run.smithi033.stdout:=============================================================================== 2024-05-03T18:07:37.770 INFO:teuthology.orchestra.run.smithi033.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-03T18:07:37.770 INFO:teuthology.orchestra.run.smithi033.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-03T18:07:37.771 INFO:teuthology.orchestra.run.smithi033.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-03T18:07:37.771 INFO:teuthology.orchestra.run.smithi033.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-03T18:07:37.772 INFO:teuthology.run_tasks:Running task install... 2024-05-03T18:07:37.781 DEBUG:teuthology.task.install:project ceph 2024-05-03T18:07:37.781 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '9a047fe7599a549042d06a3fb5b6813a203eabea'}} 2024-05-03T18:07:37.781 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '9a047fe7599a549042d06a3fb5b6813a203eabea'} 2024-05-03T18:07:37.781 INFO:teuthology.task.install:Using flavor: default 2024-05-03T18:07:37.787 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-03T18:07:37.787 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-05-03T18:07:37.787 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '9a047fe7599a549042d06a3fb5b6813a203eabea', 'tag': None, 'wait_for_package': False} 2024-05-03T18:07:37.788 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9a047fe7599a549042d06a3fb5b6813a203eabea 2024-05-03T18:07:37.790 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '9a047fe7599a549042d06a3fb5b6813a203eabea', 'tag': None, 'wait_for_package': False} 2024-05-03T18:07:37.790 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9a047fe7599a549042d06a3fb5b6813a203eabea 2024-05-03T18:07:37.791 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '9a047fe7599a549042d06a3fb5b6813a203eabea', 'tag': None, 'wait_for_package': False} 2024-05-03T18:07:37.792 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9a047fe7599a549042d06a3fb5b6813a203eabea 2024-05-03T18:07:37.944 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240501.193033/9a047fe7599a549042d06a3fb5b6813a203eabea/centos/9/flavors/default/ 2024-05-03T18:07:37.944 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3450.g9a047fe7 2024-05-03T18:07:38.012 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240501.193033/9a047fe7599a549042d06a3fb5b6813a203eabea/centos/9/flavors/default/ 2024-05-03T18:07:38.012 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3450.g9a047fe7 2024-05-03T18:07:38.021 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240501.193033/9a047fe7599a549042d06a3fb5b6813a203eabea/centos/9/flavors/default/ 2024-05-03T18:07:38.022 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3450.g9a047fe7 2024-05-03T18:07:38.262 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240501.193033/9a047fe7599a549042d06a3fb5b6813a203eabea/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240501.193033/9a047fe7599a549042d06a3fb5b6813a203eabea/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240501.193033/9a047fe7599a549042d06a3fb5b6813a203eabea/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-03T18:07:38.263 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-05-03T18:07:38.263 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-03T18:07:38.353 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-03T18:07:38.353 DEBUG:teuthology.orchestra.run.smithi150:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/9a047fe7599a549042d06a3fb5b6813a203eabea/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-03T18:07:38.373 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240501.193033/9a047fe7599a549042d06a3fb5b6813a203eabea/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240501.193033/9a047fe7599a549042d06a3fb5b6813a203eabea/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240501.193033/9a047fe7599a549042d06a3fb5b6813a203eabea/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-03T18:07:38.373 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-05-03T18:07:38.373 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-03T18:07:38.428 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240501.193033/9a047fe7599a549042d06a3fb5b6813a203eabea/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240501.193033/9a047fe7599a549042d06a3fb5b6813a203eabea/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240501.193033/9a047fe7599a549042d06a3fb5b6813a203eabea/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-03T18:07:38.429 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-03T18:07:38.429 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-03T18:07:38.462 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-03T18:07:38.462 DEBUG:teuthology.orchestra.run.smithi099:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/9a047fe7599a549042d06a3fb5b6813a203eabea/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-03T18:07:38.482 DEBUG:teuthology.orchestra.run.smithi150:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-03T18:07:38.522 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-03T18:07:38.522 DEBUG:teuthology.orchestra.run.smithi033:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/9a047fe7599a549042d06a3fb5b6813a203eabea/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-03T18:07:38.589 DEBUG:teuthology.orchestra.run.smithi099:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-03T18:07:38.593 DEBUG:teuthology.orchestra.run.smithi150:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-03T18:07:38.659 DEBUG:teuthology.orchestra.run.smithi033:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-03T18:07:38.667 INFO:teuthology.orchestra.run.smithi150.stdout:check_obsoletes = 1 2024-05-03T18:07:38.671 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum clean all 2024-05-03T18:07:38.701 DEBUG:teuthology.orchestra.run.smithi099:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-03T18:07:38.778 DEBUG:teuthology.orchestra.run.smithi033:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-03T18:07:38.781 INFO:teuthology.orchestra.run.smithi099.stdout:check_obsoletes = 1 2024-05-03T18:07:38.785 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum clean all 2024-05-03T18:07:38.865 INFO:teuthology.orchestra.run.smithi033.stdout:check_obsoletes = 1 2024-05-03T18:07:38.870 DEBUG:teuthology.orchestra.run.smithi033:> sudo yum clean all 2024-05-03T18:07:38.996 INFO:teuthology.orchestra.run.smithi150.stdout:50 files removed 2024-05-03T18:07:39.035 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-03T18:07:39.099 INFO:teuthology.orchestra.run.smithi099.stdout:50 files removed 2024-05-03T18:07:39.137 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-03T18:07:39.205 INFO:teuthology.orchestra.run.smithi033.stdout:50 files removed 2024-05-03T18:07:39.247 DEBUG:teuthology.orchestra.run.smithi033:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-03T18:07:40.340 INFO:teuthology.orchestra.run.smithi150.stdout:ceph packages for x86_64 97 kB/s | 84 kB 00:00 2024-05-03T18:07:40.342 INFO:teuthology.orchestra.run.smithi099.stdout:ceph packages for x86_64 108 kB/s | 84 kB 00:00 2024-05-03T18:07:40.482 INFO:teuthology.orchestra.run.smithi033.stdout:ceph packages for x86_64 107 kB/s | 84 kB 00:00 2024-05-03T18:07:41.058 INFO:teuthology.orchestra.run.smithi150.stdout:ceph noarch packages 19 kB/s | 12 kB 00:00 2024-05-03T18:07:41.060 INFO:teuthology.orchestra.run.smithi099.stdout:ceph noarch packages 19 kB/s | 12 kB 00:00 2024-05-03T18:07:41.490 INFO:teuthology.orchestra.run.smithi033.stdout:ceph noarch packages 13 kB/s | 12 kB 00:00 2024-05-03T18:07:41.769 INFO:teuthology.orchestra.run.smithi099.stdout:ceph source packages 3.0 kB/s | 1.9 kB 00:00 2024-05-03T18:07:41.817 INFO:teuthology.orchestra.run.smithi150.stdout:ceph source packages 2.8 kB/s | 1.9 kB 00:00 2024-05-03T18:07:42.307 INFO:teuthology.orchestra.run.smithi033.stdout:ceph source packages 2.6 kB/s | 1.9 kB 00:00 2024-05-03T18:07:42.619 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.0 MB 00:00 2024-05-03T18:07:43.200 INFO:teuthology.orchestra.run.smithi099.stdout:CentOS Stream 9 - BaseOS 6.0 MB/s | 8.0 MB 00:01 2024-05-03T18:07:43.689 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 9 - BaseOS 6.2 MB/s | 8.0 MB 00:01 2024-05-03T18:07:44.608 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - AppStream 25 MB/s | 19 MB 00:00 2024-05-03T18:07:46.823 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 9 - AppStream 10 MB/s | 19 MB 00:01 2024-05-03T18:07:46.835 INFO:teuthology.orchestra.run.smithi099.stdout:CentOS Stream 9 - AppStream 7.8 MB/s | 19 MB 00:02 2024-05-03T18:07:49.642 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - CRB 5.2 MB/s | 6.2 MB 00:01 2024-05-03T18:07:51.360 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - Extras packages 43 kB/s | 16 kB 00:00 2024-05-03T18:07:52.277 INFO:teuthology.orchestra.run.smithi099.stdout:CentOS Stream 9 - CRB 3.9 MB/s | 6.2 MB 00:01 2024-05-03T18:07:53.132 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 9 - CRB 2.7 MB/s | 6.2 MB 00:02 2024-05-03T18:07:53.550 INFO:teuthology.orchestra.run.smithi150.stdout:Extra Packages for Enterprise Linux 10 MB/s | 21 MB 00:02 2024-05-03T18:07:54.119 INFO:teuthology.orchestra.run.smithi099.stdout:CentOS Stream 9 - Extras packages 32 kB/s | 16 kB 00:00 2024-05-03T18:07:54.858 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 9 - Extras packages 49 kB/s | 16 kB 00:00 2024-05-03T18:07:56.334 INFO:teuthology.orchestra.run.smithi099.stdout:Extra Packages for Enterprise Linux 10 MB/s | 21 MB 00:02 2024-05-03T18:07:56.699 INFO:teuthology.orchestra.run.smithi033.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-05-03T18:07:59.537 INFO:teuthology.orchestra.run.smithi150.stdout:lab-extras 26 kB/s | 1.7 kB 00:00 2024-05-03T18:08:01.963 INFO:teuthology.orchestra.run.smithi150.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-03T18:08:01.965 INFO:teuthology.orchestra.run.smithi150.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-03T18:08:01.985 INFO:teuthology.orchestra.run.smithi150.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-03T18:08:01.986 INFO:teuthology.orchestra.run.smithi150.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-03T18:08:01.987 INFO:teuthology.orchestra.run.smithi150.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-03T18:08:01.989 INFO:teuthology.orchestra.run.smithi150.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-03T18:08:01.991 INFO:teuthology.orchestra.run.smithi150.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-03T18:08:01.994 INFO:teuthology.orchestra.run.smithi150.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-03T18:08:01.995 INFO:teuthology.orchestra.run.smithi150.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-03T18:08:02.066 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-05-03T18:08:02.087 INFO:teuthology.orchestra.run.smithi150.stdout:======================================================================================= 2024-05-03T18:08:02.087 INFO:teuthology.orchestra.run.smithi150.stdout: Package Arch Version Repository Size 2024-05-03T18:08:02.087 INFO:teuthology.orchestra.run.smithi150.stdout:======================================================================================= 2024-05-03T18:08:02.087 INFO:teuthology.orchestra.run.smithi150.stdout:Installing: 2024-05-03T18:08:02.087 INFO:teuthology.orchestra.run.smithi150.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-03T18:08:02.088 INFO:teuthology.orchestra.run.smithi150.stdout: ceph x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 6.6 k 2024-05-03T18:08:02.088 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-base x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 5.4 M 2024-05-03T18:08:02.088 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-fuse x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 891 k 2024-05-03T18:08:02.088 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 145 k 2024-05-03T18:08:02.088 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 1.5 M 2024-05-03T18:08:02.088 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 142 k 2024-05-03T18:08:02.088 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 3.5 M 2024-05-03T18:08:02.088 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 7.4 M 2024-05-03T18:08:02.088 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-rook noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 49 k 2024-05-03T18:08:02.088 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-radosgw x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 11 M 2024-05-03T18:08:02.088 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-test x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 49 M 2024-05-03T18:08:02.088 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-volume noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 273 k 2024-05-03T18:08:02.088 INFO:teuthology.orchestra.run.smithi150.stdout: cephadm noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 763 k 2024-05-03T18:08:02.089 INFO:teuthology.orchestra.run.smithi150.stdout: cephfs-mirror x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 225 k 2024-05-03T18:08:02.089 INFO:teuthology.orchestra.run.smithi150.stdout: cephfs-shell x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 27 k 2024-05-03T18:08:02.089 INFO:teuthology.orchestra.run.smithi150.stdout: cephfs-top noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 18 k 2024-05-03T18:08:02.089 INFO:teuthology.orchestra.run.smithi150.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-03T18:08:02.089 INFO:teuthology.orchestra.run.smithi150.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-03T18:08:02.089 INFO:teuthology.orchestra.run.smithi150.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-03T18:08:02.089 INFO:teuthology.orchestra.run.smithi150.stdout: libcephfs-devel x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 32 k 2024-05-03T18:08:02.089 INFO:teuthology.orchestra.run.smithi150.stdout: libcephfs2 x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 747 k 2024-05-03T18:08:02.089 INFO:teuthology.orchestra.run.smithi150.stdout: librados-devel x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 127 k 2024-05-03T18:08:02.089 INFO:teuthology.orchestra.run.smithi150.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-03T18:08:02.089 INFO:teuthology.orchestra.run.smithi150.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-03T18:08:02.089 INFO:teuthology.orchestra.run.smithi150.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-03T18:08:02.090 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-03T18:08:02.090 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-03T18:08:02.090 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-03T18:08:02.090 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cephfs x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 161 k 2024-05-03T18:08:02.090 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rados x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 320 k 2024-05-03T18:08:02.090 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rbd x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 298 k 2024-05-03T18:08:02.090 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rgw x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 100 k 2024-05-03T18:08:02.090 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-fuse x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 85 k 2024-05-03T18:08:02.090 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-mirror x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 3.1 M 2024-05-03T18:08:02.090 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-nbd x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 170 k 2024-05-03T18:08:02.090 INFO:teuthology.orchestra.run.smithi150.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-03T18:08:02.090 INFO:teuthology.orchestra.run.smithi150.stdout:Upgrading: 2024-05-03T18:08:02.091 INFO:teuthology.orchestra.run.smithi150.stdout: librados2 x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 3.4 M 2024-05-03T18:08:02.091 INFO:teuthology.orchestra.run.smithi150.stdout: librbd1 x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 3.1 M 2024-05-03T18:08:02.091 INFO:teuthology.orchestra.run.smithi150.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-03T18:08:02.091 INFO:teuthology.orchestra.run.smithi150.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-03T18:08:02.091 INFO:teuthology.orchestra.run.smithi150.stdout:Installing dependencies: 2024-05-03T18:08:02.091 INFO:teuthology.orchestra.run.smithi150.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-03T18:08:02.091 INFO:teuthology.orchestra.run.smithi150.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-03T18:08:02.091 INFO:teuthology.orchestra.run.smithi150.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-03T18:08:02.091 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-common x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 21 M 2024-05-03T18:08:02.091 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 34 k 2024-05-03T18:08:02.091 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mds x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 2.4 M 2024-05-03T18:08:02.091 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 274 k 2024-05-03T18:08:02.091 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mon x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 4.7 M 2024-05-03T18:08:02.092 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-osd x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 17 M 2024-05-03T18:08:02.092 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 16 k 2024-05-03T18:08:02.092 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-selinux x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 25 k 2024-05-03T18:08:02.092 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-03T18:08:02.092 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-03T18:08:02.092 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-03T18:08:02.092 INFO:teuthology.orchestra.run.smithi150.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-03T18:08:02.092 INFO:teuthology.orchestra.run.smithi150.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-03T18:08:02.092 INFO:teuthology.orchestra.run.smithi150.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-03T18:08:02.092 INFO:teuthology.orchestra.run.smithi150.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-03T18:08:02.092 INFO:teuthology.orchestra.run.smithi150.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-03T18:08:02.092 INFO:teuthology.orchestra.run.smithi150.stdout: libcephsqlite x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 163 k 2024-05-03T18:08:02.092 INFO:teuthology.orchestra.run.smithi150.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-03T18:08:02.093 INFO:teuthology.orchestra.run.smithi150.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-03T18:08:02.093 INFO:teuthology.orchestra.run.smithi150.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-03T18:08:02.093 INFO:teuthology.orchestra.run.smithi150.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-03T18:08:02.093 INFO:teuthology.orchestra.run.smithi150.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-03T18:08:02.093 INFO:teuthology.orchestra.run.smithi150.stdout: libradosstriper1 x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 500 k 2024-05-03T18:08:02.093 INFO:teuthology.orchestra.run.smithi150.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-03T18:08:02.093 INFO:teuthology.orchestra.run.smithi150.stdout: librgw2 x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 5.4 M 2024-05-03T18:08:02.093 INFO:teuthology.orchestra.run.smithi150.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-03T18:08:02.093 INFO:teuthology.orchestra.run.smithi150.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-03T18:08:02.093 INFO:teuthology.orchestra.run.smithi150.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-03T18:08:02.093 INFO:teuthology.orchestra.run.smithi150.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-03T18:08:02.093 INFO:teuthology.orchestra.run.smithi150.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-03T18:08:02.093 INFO:teuthology.orchestra.run.smithi150.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-03T18:08:02.094 INFO:teuthology.orchestra.run.smithi150.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-03T18:08:02.094 INFO:teuthology.orchestra.run.smithi150.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-03T18:08:02.094 INFO:teuthology.orchestra.run.smithi150.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-03T18:08:02.094 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-03T18:08:02.094 INFO:teuthology.orchestra.run.smithi150.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-03T18:08:02.094 INFO:teuthology.orchestra.run.smithi150.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-03T18:08:02.094 INFO:teuthology.orchestra.run.smithi150.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-03T18:08:02.094 INFO:teuthology.orchestra.run.smithi150.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-03T18:08:02.094 INFO:teuthology.orchestra.run.smithi150.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-03T18:08:02.094 INFO:teuthology.orchestra.run.smithi150.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-03T18:08:02.094 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-03T18:08:02.094 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ceph-argparse x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 45 k 2024-05-03T18:08:02.095 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ceph-common x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 135 k 2024-05-03T18:08:02.095 INFO:teuthology.orchestra.run.smithi150.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-03T18:08:02.095 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-03T18:08:02.095 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-03T18:08:02.095 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-03T18:08:02.095 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-03T18:08:02.095 INFO:teuthology.orchestra.run.smithi150.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-03T18:08:02.095 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-03T18:08:02.095 INFO:teuthology.orchestra.run.smithi150.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-03T18:08:02.095 INFO:teuthology.orchestra.run.smithi150.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-03T18:08:02.095 INFO:teuthology.orchestra.run.smithi150.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-03T18:08:02.095 INFO:teuthology.orchestra.run.smithi150.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-03T18:08:02.096 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-03T18:08:02.096 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-03T18:08:02.096 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-03T18:08:02.096 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-03T18:08:02.096 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-03T18:08:02.096 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-03T18:08:02.096 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-03T18:08:02.096 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-03T18:08:02.096 INFO:teuthology.orchestra.run.smithi150.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-03T18:08:02.096 INFO:teuthology.orchestra.run.smithi150.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-03T18:08:02.096 INFO:teuthology.orchestra.run.smithi150.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-03T18:08:02.097 INFO:teuthology.orchestra.run.smithi150.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-03T18:08:02.097 INFO:teuthology.orchestra.run.smithi150.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-03T18:08:02.097 INFO:teuthology.orchestra.run.smithi150.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-03T18:08:02.097 INFO:teuthology.orchestra.run.smithi150.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-03T18:08:02.097 INFO:teuthology.orchestra.run.smithi150.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-03T18:08:02.097 INFO:teuthology.orchestra.run.smithi150.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-03T18:08:02.097 INFO:teuthology.orchestra.run.smithi150.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-03T18:08:02.097 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-03T18:08:02.097 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-03T18:08:02.097 INFO:teuthology.orchestra.run.smithi150.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-03T18:08:02.097 INFO:teuthology.orchestra.run.smithi150.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-03T18:08:02.097 INFO:teuthology.orchestra.run.smithi150.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-03T18:08:02.098 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-03T18:08:02.098 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-03T18:08:02.098 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-03T18:08:02.098 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-03T18:08:02.098 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-03T18:08:02.098 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-03T18:08:02.098 INFO:teuthology.orchestra.run.smithi150.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-03T18:08:02.098 INFO:teuthology.orchestra.run.smithi150.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-03T18:08:02.099 INFO:teuthology.orchestra.run.smithi150.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-03T18:08:02.099 INFO:teuthology.orchestra.run.smithi150.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-03T18:08:02.099 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-03T18:08:02.099 INFO:teuthology.orchestra.run.smithi150.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-03T18:08:02.099 INFO:teuthology.orchestra.run.smithi150.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-03T18:08:02.099 INFO:teuthology.orchestra.run.smithi150.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-03T18:08:02.099 INFO:teuthology.orchestra.run.smithi150.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-03T18:08:02.099 INFO:teuthology.orchestra.run.smithi150.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-03T18:08:02.099 INFO:teuthology.orchestra.run.smithi150.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-03T18:08:02.099 INFO:teuthology.orchestra.run.smithi150.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-03T18:08:02.100 INFO:teuthology.orchestra.run.smithi150.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-03T18:08:02.100 INFO:teuthology.orchestra.run.smithi150.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-03T18:08:02.100 INFO:teuthology.orchestra.run.smithi150.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-03T18:08:02.100 INFO:teuthology.orchestra.run.smithi150.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-03T18:08:02.100 INFO:teuthology.orchestra.run.smithi150.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-03T18:08:02.100 INFO:teuthology.orchestra.run.smithi150.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-03T18:08:02.100 INFO:teuthology.orchestra.run.smithi150.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-03T18:08:02.100 INFO:teuthology.orchestra.run.smithi150.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-03T18:08:02.100 INFO:teuthology.orchestra.run.smithi150.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-03T18:08:02.100 INFO:teuthology.orchestra.run.smithi150.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-03T18:08:02.101 INFO:teuthology.orchestra.run.smithi150.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-03T18:08:02.101 INFO:teuthology.orchestra.run.smithi150.stdout:Installing weak dependencies: 2024-05-03T18:08:02.101 INFO:teuthology.orchestra.run.smithi150.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-03T18:08:02.101 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-03T18:08:02.101 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-05-03T18:08:02.101 INFO:teuthology.orchestra.run.smithi150.stdout:======================================================================================= 2024-05-03T18:08:02.101 INFO:teuthology.orchestra.run.smithi150.stdout:Install 148 Packages 2024-05-03T18:08:02.101 INFO:teuthology.orchestra.run.smithi150.stdout:Upgrade 4 Packages 2024-05-03T18:08:02.101 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-03T18:08:02.102 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 222 M 2024-05-03T18:08:02.102 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-05-03T18:08:02.245 INFO:teuthology.orchestra.run.smithi099.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-05-03T18:08:02.767 INFO:teuthology.orchestra.run.smithi033.stdout:lab-extras 41 kB/s | 1.7 kB 00:00 2024-05-03T18:08:03.047 INFO:teuthology.orchestra.run.smithi150.stdout:(1/152): ceph-19.0.0-3450.g9a047fe7.el9.x86_64. 26 kB/s | 6.6 kB 00:00 2024-05-03T18:08:03.255 INFO:teuthology.orchestra.run.smithi150.stdout:(2/152): ceph-fuse-19.0.0-3450.g9a047fe7.el9.x8 4.2 MB/s | 891 kB 00:00 2024-05-03T18:08:03.322 INFO:teuthology.orchestra.run.smithi150.stdout:(3/152): ceph-immutable-object-cache-19.0.0-345 2.2 MB/s | 145 kB 00:00 2024-05-03T18:08:03.573 INFO:teuthology.orchestra.run.smithi150.stdout:(4/152): ceph-base-19.0.0-3450.g9a047fe7.el9.x8 6.9 MB/s | 5.4 MB 00:00 2024-05-03T18:08:03.681 INFO:teuthology.orchestra.run.smithi150.stdout:(5/152): ceph-mds-19.0.0-3450.g9a047fe7.el9.x86 6.6 MB/s | 2.4 MB 00:00 2024-05-03T18:08:03.765 INFO:teuthology.orchestra.run.smithi150.stdout:(6/152): ceph-mgr-19.0.0-3450.g9a047fe7.el9.x86 7.9 MB/s | 1.5 MB 00:00 2024-05-03T18:08:04.148 INFO:teuthology.orchestra.run.smithi150.stdout:(7/152): ceph-mon-19.0.0-3450.g9a047fe7.el9.x86 10 MB/s | 4.7 MB 00:00 2024-05-03T18:08:04.612 INFO:teuthology.orchestra.run.smithi099.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-03T18:08:04.614 INFO:teuthology.orchestra.run.smithi099.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-03T18:08:04.633 INFO:teuthology.orchestra.run.smithi099.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-03T18:08:04.634 INFO:teuthology.orchestra.run.smithi099.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-03T18:08:04.635 INFO:teuthology.orchestra.run.smithi099.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-03T18:08:04.636 INFO:teuthology.orchestra.run.smithi099.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-03T18:08:04.638 INFO:teuthology.orchestra.run.smithi099.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-03T18:08:04.642 INFO:teuthology.orchestra.run.smithi099.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-03T18:08:04.643 INFO:teuthology.orchestra.run.smithi099.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-03T18:08:04.713 INFO:teuthology.orchestra.run.smithi099.stdout:Dependencies resolved. 2024-05-03T18:08:04.734 INFO:teuthology.orchestra.run.smithi099.stdout:======================================================================================= 2024-05-03T18:08:04.734 INFO:teuthology.orchestra.run.smithi099.stdout: Package Arch Version Repository Size 2024-05-03T18:08:04.734 INFO:teuthology.orchestra.run.smithi099.stdout:======================================================================================= 2024-05-03T18:08:04.734 INFO:teuthology.orchestra.run.smithi099.stdout:Installing: 2024-05-03T18:08:04.734 INFO:teuthology.orchestra.run.smithi099.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-03T18:08:04.734 INFO:teuthology.orchestra.run.smithi099.stdout: ceph x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 6.6 k 2024-05-03T18:08:04.734 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-base x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 5.4 M 2024-05-03T18:08:04.734 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-fuse x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 891 k 2024-05-03T18:08:04.734 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 145 k 2024-05-03T18:08:04.734 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 1.5 M 2024-05-03T18:08:04.734 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 142 k 2024-05-03T18:08:04.734 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 3.5 M 2024-05-03T18:08:04.735 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 7.4 M 2024-05-03T18:08:04.735 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-rook noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 49 k 2024-05-03T18:08:04.735 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-radosgw x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 11 M 2024-05-03T18:08:04.735 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-test x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 49 M 2024-05-03T18:08:04.735 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-volume noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 273 k 2024-05-03T18:08:04.735 INFO:teuthology.orchestra.run.smithi099.stdout: cephadm noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 763 k 2024-05-03T18:08:04.735 INFO:teuthology.orchestra.run.smithi099.stdout: cephfs-mirror x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 225 k 2024-05-03T18:08:04.735 INFO:teuthology.orchestra.run.smithi099.stdout: cephfs-shell x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 27 k 2024-05-03T18:08:04.735 INFO:teuthology.orchestra.run.smithi099.stdout: cephfs-top noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 18 k 2024-05-03T18:08:04.735 INFO:teuthology.orchestra.run.smithi099.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-03T18:08:04.735 INFO:teuthology.orchestra.run.smithi099.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-03T18:08:04.735 INFO:teuthology.orchestra.run.smithi099.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-03T18:08:04.735 INFO:teuthology.orchestra.run.smithi099.stdout: libcephfs-devel x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 32 k 2024-05-03T18:08:04.735 INFO:teuthology.orchestra.run.smithi099.stdout: libcephfs2 x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 747 k 2024-05-03T18:08:04.736 INFO:teuthology.orchestra.run.smithi099.stdout: librados-devel x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 127 k 2024-05-03T18:08:04.736 INFO:teuthology.orchestra.run.smithi099.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-03T18:08:04.736 INFO:teuthology.orchestra.run.smithi099.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-03T18:08:04.736 INFO:teuthology.orchestra.run.smithi099.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-03T18:08:04.736 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-03T18:08:04.736 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-03T18:08:04.736 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-03T18:08:04.736 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cephfs x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 161 k 2024-05-03T18:08:04.736 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rados x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 320 k 2024-05-03T18:08:04.736 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rbd x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 298 k 2024-05-03T18:08:04.736 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rgw x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 100 k 2024-05-03T18:08:04.736 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-fuse x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 85 k 2024-05-03T18:08:04.736 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-mirror x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 3.1 M 2024-05-03T18:08:04.737 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-nbd x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 170 k 2024-05-03T18:08:04.737 INFO:teuthology.orchestra.run.smithi099.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-03T18:08:04.737 INFO:teuthology.orchestra.run.smithi099.stdout:Upgrading: 2024-05-03T18:08:04.737 INFO:teuthology.orchestra.run.smithi099.stdout: librados2 x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 3.4 M 2024-05-03T18:08:04.737 INFO:teuthology.orchestra.run.smithi099.stdout: librbd1 x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 3.1 M 2024-05-03T18:08:04.737 INFO:teuthology.orchestra.run.smithi099.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-03T18:08:04.737 INFO:teuthology.orchestra.run.smithi099.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-03T18:08:04.737 INFO:teuthology.orchestra.run.smithi099.stdout:Installing dependencies: 2024-05-03T18:08:04.737 INFO:teuthology.orchestra.run.smithi099.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-03T18:08:04.737 INFO:teuthology.orchestra.run.smithi099.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-03T18:08:04.737 INFO:teuthology.orchestra.run.smithi099.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-03T18:08:04.737 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-common x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 21 M 2024-05-03T18:08:04.737 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 34 k 2024-05-03T18:08:04.738 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mds x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 2.4 M 2024-05-03T18:08:04.738 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 274 k 2024-05-03T18:08:04.738 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mon x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 4.7 M 2024-05-03T18:08:04.738 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-osd x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 17 M 2024-05-03T18:08:04.738 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 16 k 2024-05-03T18:08:04.738 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-selinux x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 25 k 2024-05-03T18:08:04.738 INFO:teuthology.orchestra.run.smithi099.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-03T18:08:04.738 INFO:teuthology.orchestra.run.smithi099.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-03T18:08:04.738 INFO:teuthology.orchestra.run.smithi099.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-03T18:08:04.738 INFO:teuthology.orchestra.run.smithi099.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-03T18:08:04.738 INFO:teuthology.orchestra.run.smithi099.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-03T18:08:04.738 INFO:teuthology.orchestra.run.smithi099.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-03T18:08:04.738 INFO:teuthology.orchestra.run.smithi099.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-03T18:08:04.738 INFO:teuthology.orchestra.run.smithi099.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-03T18:08:04.739 INFO:teuthology.orchestra.run.smithi099.stdout: libcephsqlite x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 163 k 2024-05-03T18:08:04.739 INFO:teuthology.orchestra.run.smithi099.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-03T18:08:04.739 INFO:teuthology.orchestra.run.smithi099.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-03T18:08:04.739 INFO:teuthology.orchestra.run.smithi099.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-03T18:08:04.739 INFO:teuthology.orchestra.run.smithi099.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-03T18:08:04.739 INFO:teuthology.orchestra.run.smithi099.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-03T18:08:04.739 INFO:teuthology.orchestra.run.smithi099.stdout: libradosstriper1 x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 500 k 2024-05-03T18:08:04.739 INFO:teuthology.orchestra.run.smithi099.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-03T18:08:04.739 INFO:teuthology.orchestra.run.smithi099.stdout: librgw2 x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 5.4 M 2024-05-03T18:08:04.739 INFO:teuthology.orchestra.run.smithi099.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-03T18:08:04.739 INFO:teuthology.orchestra.run.smithi099.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-03T18:08:04.739 INFO:teuthology.orchestra.run.smithi099.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-03T18:08:04.739 INFO:teuthology.orchestra.run.smithi099.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-03T18:08:04.739 INFO:teuthology.orchestra.run.smithi099.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-03T18:08:04.739 INFO:teuthology.orchestra.run.smithi099.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-03T18:08:04.740 INFO:teuthology.orchestra.run.smithi099.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-03T18:08:04.740 INFO:teuthology.orchestra.run.smithi099.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-03T18:08:04.740 INFO:teuthology.orchestra.run.smithi099.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-03T18:08:04.740 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-03T18:08:04.740 INFO:teuthology.orchestra.run.smithi099.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-03T18:08:04.740 INFO:teuthology.orchestra.run.smithi099.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-03T18:08:04.740 INFO:teuthology.orchestra.run.smithi099.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-03T18:08:04.740 INFO:teuthology.orchestra.run.smithi099.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-03T18:08:04.740 INFO:teuthology.orchestra.run.smithi099.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-03T18:08:04.740 INFO:teuthology.orchestra.run.smithi099.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-03T18:08:04.740 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-03T18:08:04.740 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ceph-argparse x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 45 k 2024-05-03T18:08:04.740 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ceph-common x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 135 k 2024-05-03T18:08:04.740 INFO:teuthology.orchestra.run.smithi099.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-03T18:08:04.740 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-03T18:08:04.741 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-03T18:08:04.741 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-03T18:08:04.741 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-03T18:08:04.741 INFO:teuthology.orchestra.run.smithi099.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-03T18:08:04.741 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-03T18:08:04.741 INFO:teuthology.orchestra.run.smithi099.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-03T18:08:04.741 INFO:teuthology.orchestra.run.smithi099.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-03T18:08:04.741 INFO:teuthology.orchestra.run.smithi099.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-03T18:08:04.741 INFO:teuthology.orchestra.run.smithi099.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-03T18:08:04.741 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-03T18:08:04.741 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-03T18:08:04.741 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-03T18:08:04.741 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-03T18:08:04.742 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-03T18:08:04.742 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-03T18:08:04.742 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-03T18:08:04.742 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-03T18:08:04.742 INFO:teuthology.orchestra.run.smithi099.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-03T18:08:04.742 INFO:teuthology.orchestra.run.smithi099.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-03T18:08:04.742 INFO:teuthology.orchestra.run.smithi099.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-03T18:08:04.742 INFO:teuthology.orchestra.run.smithi099.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-03T18:08:04.742 INFO:teuthology.orchestra.run.smithi099.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-03T18:08:04.742 INFO:teuthology.orchestra.run.smithi099.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-03T18:08:04.742 INFO:teuthology.orchestra.run.smithi099.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-03T18:08:04.742 INFO:teuthology.orchestra.run.smithi099.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-03T18:08:04.742 INFO:teuthology.orchestra.run.smithi099.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-03T18:08:04.743 INFO:teuthology.orchestra.run.smithi099.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-03T18:08:04.743 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-03T18:08:04.743 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-03T18:08:04.743 INFO:teuthology.orchestra.run.smithi099.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-03T18:08:04.743 INFO:teuthology.orchestra.run.smithi099.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-03T18:08:04.743 INFO:teuthology.orchestra.run.smithi099.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-03T18:08:04.743 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-03T18:08:04.743 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-03T18:08:04.743 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-03T18:08:04.743 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-03T18:08:04.743 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-03T18:08:04.743 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-03T18:08:04.743 INFO:teuthology.orchestra.run.smithi099.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-03T18:08:04.743 INFO:teuthology.orchestra.run.smithi099.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-03T18:08:04.743 INFO:teuthology.orchestra.run.smithi099.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-03T18:08:04.744 INFO:teuthology.orchestra.run.smithi099.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-03T18:08:04.744 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-03T18:08:04.744 INFO:teuthology.orchestra.run.smithi099.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-03T18:08:04.744 INFO:teuthology.orchestra.run.smithi099.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-03T18:08:04.744 INFO:teuthology.orchestra.run.smithi099.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-03T18:08:04.744 INFO:teuthology.orchestra.run.smithi099.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-03T18:08:04.744 INFO:teuthology.orchestra.run.smithi099.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-03T18:08:04.744 INFO:teuthology.orchestra.run.smithi099.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-03T18:08:04.744 INFO:teuthology.orchestra.run.smithi099.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-03T18:08:04.744 INFO:teuthology.orchestra.run.smithi099.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-03T18:08:04.744 INFO:teuthology.orchestra.run.smithi099.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-03T18:08:04.744 INFO:teuthology.orchestra.run.smithi099.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-03T18:08:04.744 INFO:teuthology.orchestra.run.smithi099.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-03T18:08:04.745 INFO:teuthology.orchestra.run.smithi099.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-03T18:08:04.745 INFO:teuthology.orchestra.run.smithi099.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-03T18:08:04.745 INFO:teuthology.orchestra.run.smithi099.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-03T18:08:04.745 INFO:teuthology.orchestra.run.smithi099.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-03T18:08:04.745 INFO:teuthology.orchestra.run.smithi099.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-03T18:08:04.745 INFO:teuthology.orchestra.run.smithi099.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-03T18:08:04.745 INFO:teuthology.orchestra.run.smithi099.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-03T18:08:04.745 INFO:teuthology.orchestra.run.smithi099.stdout:Installing weak dependencies: 2024-05-03T18:08:04.745 INFO:teuthology.orchestra.run.smithi099.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-03T18:08:04.745 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-03T18:08:04.745 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction Summary 2024-05-03T18:08:04.745 INFO:teuthology.orchestra.run.smithi099.stdout:======================================================================================= 2024-05-03T18:08:04.745 INFO:teuthology.orchestra.run.smithi099.stdout:Install 148 Packages 2024-05-03T18:08:04.746 INFO:teuthology.orchestra.run.smithi099.stdout:Upgrade 4 Packages 2024-05-03T18:08:04.746 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-03T18:08:04.746 INFO:teuthology.orchestra.run.smithi099.stdout:Total download size: 222 M 2024-05-03T18:08:04.746 INFO:teuthology.orchestra.run.smithi099.stdout:Downloading Packages: 2024-05-03T18:08:05.220 INFO:teuthology.orchestra.run.smithi033.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-03T18:08:05.221 INFO:teuthology.orchestra.run.smithi033.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-03T18:08:05.243 INFO:teuthology.orchestra.run.smithi033.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-03T18:08:05.245 INFO:teuthology.orchestra.run.smithi033.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-03T18:08:05.246 INFO:teuthology.orchestra.run.smithi033.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-03T18:08:05.247 INFO:teuthology.orchestra.run.smithi033.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-03T18:08:05.250 INFO:teuthology.orchestra.run.smithi033.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-03T18:08:05.253 INFO:teuthology.orchestra.run.smithi033.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-03T18:08:05.255 INFO:teuthology.orchestra.run.smithi033.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-03T18:08:05.273 INFO:teuthology.orchestra.run.smithi150.stdout:(8/152): ceph-osd-19.0.0-3450.g9a047fe7.el9.x86 11 MB/s | 17 MB 00:01 2024-05-03T18:08:05.329 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies resolved. 2024-05-03T18:08:05.332 INFO:teuthology.orchestra.run.smithi150.stdout:(9/152): ceph-selinux-19.0.0-3450.g9a047fe7.el9 439 kB/s | 25 kB 00:00 2024-05-03T18:08:05.350 INFO:teuthology.orchestra.run.smithi033.stdout:======================================================================================= 2024-05-03T18:08:05.350 INFO:teuthology.orchestra.run.smithi033.stdout: Package Arch Version Repository Size 2024-05-03T18:08:05.351 INFO:teuthology.orchestra.run.smithi033.stdout:======================================================================================= 2024-05-03T18:08:05.351 INFO:teuthology.orchestra.run.smithi033.stdout:Installing: 2024-05-03T18:08:05.351 INFO:teuthology.orchestra.run.smithi033.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-03T18:08:05.351 INFO:teuthology.orchestra.run.smithi033.stdout: ceph x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 6.6 k 2024-05-03T18:08:05.351 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-base x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 5.4 M 2024-05-03T18:08:05.351 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-fuse x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 891 k 2024-05-03T18:08:05.351 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 145 k 2024-05-03T18:08:05.351 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 1.5 M 2024-05-03T18:08:05.351 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 142 k 2024-05-03T18:08:05.351 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 3.5 M 2024-05-03T18:08:05.351 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 7.4 M 2024-05-03T18:08:05.351 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-rook noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 49 k 2024-05-03T18:08:05.351 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-radosgw x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 11 M 2024-05-03T18:08:05.351 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-test x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 49 M 2024-05-03T18:08:05.352 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-volume noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 273 k 2024-05-03T18:08:05.352 INFO:teuthology.orchestra.run.smithi033.stdout: cephadm noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 763 k 2024-05-03T18:08:05.352 INFO:teuthology.orchestra.run.smithi033.stdout: cephfs-mirror x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 225 k 2024-05-03T18:08:05.352 INFO:teuthology.orchestra.run.smithi033.stdout: cephfs-shell x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 27 k 2024-05-03T18:08:05.352 INFO:teuthology.orchestra.run.smithi033.stdout: cephfs-top noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 18 k 2024-05-03T18:08:05.352 INFO:teuthology.orchestra.run.smithi033.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-03T18:08:05.352 INFO:teuthology.orchestra.run.smithi033.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-03T18:08:05.352 INFO:teuthology.orchestra.run.smithi033.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-03T18:08:05.352 INFO:teuthology.orchestra.run.smithi033.stdout: libcephfs-devel x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 32 k 2024-05-03T18:08:05.352 INFO:teuthology.orchestra.run.smithi033.stdout: libcephfs2 x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 747 k 2024-05-03T18:08:05.352 INFO:teuthology.orchestra.run.smithi033.stdout: librados-devel x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 127 k 2024-05-03T18:08:05.352 INFO:teuthology.orchestra.run.smithi033.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-03T18:08:05.352 INFO:teuthology.orchestra.run.smithi033.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-03T18:08:05.353 INFO:teuthology.orchestra.run.smithi033.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-03T18:08:05.353 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-03T18:08:05.353 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-03T18:08:05.353 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-03T18:08:05.353 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cephfs x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 161 k 2024-05-03T18:08:05.353 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rados x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 320 k 2024-05-03T18:08:05.353 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rbd x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 298 k 2024-05-03T18:08:05.353 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rgw x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 100 k 2024-05-03T18:08:05.353 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-fuse x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 85 k 2024-05-03T18:08:05.353 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-mirror x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 3.1 M 2024-05-03T18:08:05.353 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-nbd x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 170 k 2024-05-03T18:08:05.353 INFO:teuthology.orchestra.run.smithi033.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-03T18:08:05.353 INFO:teuthology.orchestra.run.smithi033.stdout:Upgrading: 2024-05-03T18:08:05.353 INFO:teuthology.orchestra.run.smithi033.stdout: librados2 x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 3.4 M 2024-05-03T18:08:05.354 INFO:teuthology.orchestra.run.smithi033.stdout: librbd1 x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 3.1 M 2024-05-03T18:08:05.354 INFO:teuthology.orchestra.run.smithi033.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-03T18:08:05.354 INFO:teuthology.orchestra.run.smithi033.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-03T18:08:05.354 INFO:teuthology.orchestra.run.smithi033.stdout:Installing dependencies: 2024-05-03T18:08:05.354 INFO:teuthology.orchestra.run.smithi033.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-03T18:08:05.354 INFO:teuthology.orchestra.run.smithi033.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-03T18:08:05.354 INFO:teuthology.orchestra.run.smithi033.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-03T18:08:05.354 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-common x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 21 M 2024-05-03T18:08:05.354 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 34 k 2024-05-03T18:08:05.354 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mds x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 2.4 M 2024-05-03T18:08:05.354 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 274 k 2024-05-03T18:08:05.354 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mon x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 4.7 M 2024-05-03T18:08:05.355 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-osd x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 17 M 2024-05-03T18:08:05.355 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 16 k 2024-05-03T18:08:05.355 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-selinux x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 25 k 2024-05-03T18:08:05.355 INFO:teuthology.orchestra.run.smithi033.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-03T18:08:05.355 INFO:teuthology.orchestra.run.smithi033.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-03T18:08:05.355 INFO:teuthology.orchestra.run.smithi033.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-03T18:08:05.355 INFO:teuthology.orchestra.run.smithi033.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-03T18:08:05.355 INFO:teuthology.orchestra.run.smithi033.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-03T18:08:05.355 INFO:teuthology.orchestra.run.smithi033.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-03T18:08:05.355 INFO:teuthology.orchestra.run.smithi033.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-03T18:08:05.355 INFO:teuthology.orchestra.run.smithi033.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-03T18:08:05.355 INFO:teuthology.orchestra.run.smithi033.stdout: libcephsqlite x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 163 k 2024-05-03T18:08:05.355 INFO:teuthology.orchestra.run.smithi033.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-03T18:08:05.355 INFO:teuthology.orchestra.run.smithi033.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-03T18:08:05.355 INFO:teuthology.orchestra.run.smithi033.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-03T18:08:05.356 INFO:teuthology.orchestra.run.smithi033.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-03T18:08:05.356 INFO:teuthology.orchestra.run.smithi033.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-03T18:08:05.356 INFO:teuthology.orchestra.run.smithi033.stdout: libradosstriper1 x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 500 k 2024-05-03T18:08:05.356 INFO:teuthology.orchestra.run.smithi033.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-03T18:08:05.356 INFO:teuthology.orchestra.run.smithi033.stdout: librgw2 x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 5.4 M 2024-05-03T18:08:05.356 INFO:teuthology.orchestra.run.smithi033.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-03T18:08:05.356 INFO:teuthology.orchestra.run.smithi033.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-03T18:08:05.356 INFO:teuthology.orchestra.run.smithi033.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-03T18:08:05.356 INFO:teuthology.orchestra.run.smithi033.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-03T18:08:05.356 INFO:teuthology.orchestra.run.smithi033.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-03T18:08:05.356 INFO:teuthology.orchestra.run.smithi033.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-03T18:08:05.356 INFO:teuthology.orchestra.run.smithi033.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-03T18:08:05.356 INFO:teuthology.orchestra.run.smithi033.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-03T18:08:05.356 INFO:teuthology.orchestra.run.smithi033.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-03T18:08:05.357 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-03T18:08:05.357 INFO:teuthology.orchestra.run.smithi033.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-03T18:08:05.357 INFO:teuthology.orchestra.run.smithi033.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-03T18:08:05.357 INFO:teuthology.orchestra.run.smithi033.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-03T18:08:05.357 INFO:teuthology.orchestra.run.smithi033.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-03T18:08:05.357 INFO:teuthology.orchestra.run.smithi033.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-03T18:08:05.357 INFO:teuthology.orchestra.run.smithi033.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-03T18:08:05.357 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-03T18:08:05.357 INFO:teuthology.orchestra.run.smithi033.stdout: python3-ceph-argparse x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 45 k 2024-05-03T18:08:05.357 INFO:teuthology.orchestra.run.smithi033.stdout: python3-ceph-common x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 135 k 2024-05-03T18:08:05.357 INFO:teuthology.orchestra.run.smithi033.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-03T18:08:05.357 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-03T18:08:05.357 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-03T18:08:05.357 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-03T18:08:05.358 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-03T18:08:05.358 INFO:teuthology.orchestra.run.smithi033.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-03T18:08:05.358 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-03T18:08:05.358 INFO:teuthology.orchestra.run.smithi033.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-03T18:08:05.358 INFO:teuthology.orchestra.run.smithi033.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-03T18:08:05.358 INFO:teuthology.orchestra.run.smithi033.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-03T18:08:05.358 INFO:teuthology.orchestra.run.smithi033.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-03T18:08:05.358 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-03T18:08:05.358 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-03T18:08:05.358 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-03T18:08:05.358 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-03T18:08:05.358 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-03T18:08:05.358 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-03T18:08:05.359 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-03T18:08:05.359 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-03T18:08:05.359 INFO:teuthology.orchestra.run.smithi033.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-03T18:08:05.359 INFO:teuthology.orchestra.run.smithi033.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-03T18:08:05.359 INFO:teuthology.orchestra.run.smithi033.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-03T18:08:05.359 INFO:teuthology.orchestra.run.smithi033.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-03T18:08:05.359 INFO:teuthology.orchestra.run.smithi033.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-03T18:08:05.359 INFO:teuthology.orchestra.run.smithi033.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-03T18:08:05.359 INFO:teuthology.orchestra.run.smithi033.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-03T18:08:05.359 INFO:teuthology.orchestra.run.smithi033.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-03T18:08:05.359 INFO:teuthology.orchestra.run.smithi033.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-03T18:08:05.359 INFO:teuthology.orchestra.run.smithi033.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-03T18:08:05.359 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-03T18:08:05.360 INFO:teuthology.orchestra.run.smithi033.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-03T18:08:05.360 INFO:teuthology.orchestra.run.smithi033.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-03T18:08:05.360 INFO:teuthology.orchestra.run.smithi033.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-03T18:08:05.360 INFO:teuthology.orchestra.run.smithi033.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-03T18:08:05.360 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-03T18:08:05.360 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-03T18:08:05.360 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-03T18:08:05.360 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-03T18:08:05.360 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-03T18:08:05.360 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-03T18:08:05.360 INFO:teuthology.orchestra.run.smithi033.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-03T18:08:05.360 INFO:teuthology.orchestra.run.smithi033.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-03T18:08:05.360 INFO:teuthology.orchestra.run.smithi033.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-03T18:08:05.360 INFO:teuthology.orchestra.run.smithi033.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-03T18:08:05.361 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-03T18:08:05.361 INFO:teuthology.orchestra.run.smithi033.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-03T18:08:05.361 INFO:teuthology.orchestra.run.smithi033.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-03T18:08:05.361 INFO:teuthology.orchestra.run.smithi033.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-03T18:08:05.361 INFO:teuthology.orchestra.run.smithi033.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-03T18:08:05.361 INFO:teuthology.orchestra.run.smithi033.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-03T18:08:05.361 INFO:teuthology.orchestra.run.smithi033.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-03T18:08:05.361 INFO:teuthology.orchestra.run.smithi033.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-03T18:08:05.361 INFO:teuthology.orchestra.run.smithi033.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-03T18:08:05.361 INFO:teuthology.orchestra.run.smithi033.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-03T18:08:05.361 INFO:teuthology.orchestra.run.smithi033.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-03T18:08:05.361 INFO:teuthology.orchestra.run.smithi033.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-03T18:08:05.361 INFO:teuthology.orchestra.run.smithi033.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-03T18:08:05.362 INFO:teuthology.orchestra.run.smithi033.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-03T18:08:05.362 INFO:teuthology.orchestra.run.smithi033.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-03T18:08:05.362 INFO:teuthology.orchestra.run.smithi033.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-03T18:08:05.362 INFO:teuthology.orchestra.run.smithi033.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-03T18:08:05.362 INFO:teuthology.orchestra.run.smithi033.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-03T18:08:05.362 INFO:teuthology.orchestra.run.smithi033.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-03T18:08:05.362 INFO:teuthology.orchestra.run.smithi033.stdout:Installing weak dependencies: 2024-05-03T18:08:05.362 INFO:teuthology.orchestra.run.smithi033.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-03T18:08:05.362 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:08:05.362 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction Summary 2024-05-03T18:08:05.362 INFO:teuthology.orchestra.run.smithi033.stdout:======================================================================================= 2024-05-03T18:08:05.362 INFO:teuthology.orchestra.run.smithi033.stdout:Install 148 Packages 2024-05-03T18:08:05.362 INFO:teuthology.orchestra.run.smithi033.stdout:Upgrade 4 Packages 2024-05-03T18:08:05.362 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:08:05.363 INFO:teuthology.orchestra.run.smithi033.stdout:Total download size: 222 M 2024-05-03T18:08:05.363 INFO:teuthology.orchestra.run.smithi033.stdout:Downloading Packages: 2024-05-03T18:08:05.508 INFO:teuthology.orchestra.run.smithi150.stdout:(10/152): ceph-radosgw-19.0.0-3450.g9a047fe7.el 7.8 MB/s | 11 MB 00:01 2024-05-03T18:08:05.600 INFO:teuthology.orchestra.run.smithi150.stdout:(11/152): cephfs-mirror-19.0.0-3450.g9a047fe7.e 2.4 MB/s | 225 kB 00:00 2024-05-03T18:08:05.631 INFO:teuthology.orchestra.run.smithi099.stdout:(1/152): ceph-19.0.0-3450.g9a047fe7.el9.x86_64. 19 kB/s | 6.6 kB 00:00 2024-05-03T18:08:05.691 INFO:teuthology.orchestra.run.smithi150.stdout:(12/152): cephfs-shell-19.0.0-3450.g9a047fe7.el 297 kB/s | 27 kB 00:00 2024-05-03T18:08:05.950 INFO:teuthology.orchestra.run.smithi150.stdout:(13/152): ceph-common-19.0.0-3450.g9a047fe7.el9 6.7 MB/s | 21 MB 00:03 2024-05-03T18:08:05.976 INFO:teuthology.orchestra.run.smithi150.stdout:(14/152): libcephfs-devel-19.0.0-3450.g9a047fe7 112 kB/s | 32 kB 00:00 2024-05-03T18:08:06.034 INFO:teuthology.orchestra.run.smithi150.stdout:(15/152): libcephsqlite-19.0.0-3450.g9a047fe7.e 2.8 MB/s | 163 kB 00:00 2024-05-03T18:08:06.093 INFO:teuthology.orchestra.run.smithi150.stdout:(16/152): libcephfs2-19.0.0-3450.g9a047fe7.el9. 5.2 MB/s | 747 kB 00:00 2024-05-03T18:08:06.126 INFO:teuthology.orchestra.run.smithi150.stdout:(17/152): librados-devel-19.0.0-3450.g9a047fe7. 1.4 MB/s | 127 kB 00:00 2024-05-03T18:08:06.227 INFO:teuthology.orchestra.run.smithi150.stdout:(18/152): libradosstriper1-19.0.0-3450.g9a047fe 3.7 MB/s | 500 kB 00:00 2024-05-03T18:08:06.362 INFO:teuthology.orchestra.run.smithi150.stdout:(19/152): python3-ceph-argparse-19.0.0-3450.g9a 337 kB/s | 45 kB 00:00 2024-05-03T18:08:06.431 INFO:teuthology.orchestra.run.smithi099.stdout:(2/152): ceph-fuse-19.0.0-3450.g9a047fe7.el9.x8 1.1 MB/s | 891 kB 00:00 2024-05-03T18:08:06.477 INFO:teuthology.orchestra.run.smithi150.stdout:(20/152): python3-ceph-common-19.0.0-3450.g9a04 1.2 MB/s | 135 kB 00:00 2024-05-03T18:08:06.586 INFO:teuthology.orchestra.run.smithi150.stdout:(21/152): python3-cephfs-19.0.0-3450.g9a047fe7. 1.5 MB/s | 161 kB 00:00 2024-05-03T18:08:06.623 INFO:teuthology.orchestra.run.smithi099.stdout:(3/152): ceph-immutable-object-cache-19.0.0-345 760 kB/s | 145 kB 00:00 2024-05-03T18:08:06.731 INFO:teuthology.orchestra.run.smithi150.stdout:(22/152): python3-rados-19.0.0-3450.g9a047fe7.e 2.2 MB/s | 320 kB 00:00 2024-05-03T18:08:06.861 INFO:teuthology.orchestra.run.smithi150.stdout:(23/152): python3-rbd-19.0.0-3450.g9a047fe7.el9 2.3 MB/s | 298 kB 00:00 2024-05-03T18:08:06.919 INFO:teuthology.orchestra.run.smithi033.stdout:(1/152): ceph-19.0.0-3450.g9a047fe7.el9.x86_64. 5.3 kB/s | 6.6 kB 00:01 2024-05-03T18:08:06.940 INFO:teuthology.orchestra.run.smithi099.stdout:(4/152): ceph-base-19.0.0-3450.g9a047fe7.el9.x8 3.2 MB/s | 5.4 MB 00:01 2024-05-03T18:08:06.961 INFO:teuthology.orchestra.run.smithi150.stdout:(24/152): python3-rgw-19.0.0-3450.g9a047fe7.el9 1.0 MB/s | 100 kB 00:00 2024-05-03T18:08:07.109 INFO:teuthology.orchestra.run.smithi150.stdout:(25/152): rbd-fuse-19.0.0-3450.g9a047fe7.el9.x8 584 kB/s | 85 kB 00:00 2024-05-03T18:08:07.238 INFO:teuthology.orchestra.run.smithi150.stdout:(26/152): librgw2-19.0.0-3450.g9a047fe7.el9.x86 4.8 MB/s | 5.4 MB 00:01 2024-05-03T18:08:07.385 INFO:teuthology.orchestra.run.smithi150.stdout:(27/152): rbd-nbd-19.0.0-3450.g9a047fe7.el9.x86 1.6 MB/s | 170 kB 00:00 2024-05-03T18:08:07.430 INFO:teuthology.orchestra.run.smithi150.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3450.g 412 kB/s | 34 kB 00:00 2024-05-03T18:08:07.481 INFO:teuthology.orchestra.run.smithi099.stdout:(5/152): ceph-mgr-19.0.0-3450.g9a047fe7.el9.x86 2.8 MB/s | 1.5 MB 00:00 2024-05-03T18:08:07.521 INFO:teuthology.orchestra.run.smithi150.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3450.g9a047fe 1.5 MB/s | 142 kB 00:00 2024-05-03T18:08:07.560 INFO:teuthology.orchestra.run.smithi033.stdout:(2/152): ceph-fuse-19.0.0-3450.g9a047fe7.el9.x8 1.4 MB/s | 891 kB 00:00 2024-05-03T18:08:07.686 INFO:teuthology.orchestra.run.smithi033.stdout:(3/152): ceph-immutable-object-cache-19.0.0-345 1.1 MB/s | 145 kB 00:00 2024-05-03T18:08:07.819 INFO:teuthology.orchestra.run.smithi033.stdout:(4/152): ceph-base-19.0.0-3450.g9a047fe7.el9.x8 2.5 MB/s | 5.4 MB 00:02 2024-05-03T18:08:07.872 INFO:teuthology.orchestra.run.smithi150.stdout:(30/152): rbd-mirror-19.0.0-3450.g9a047fe7.el9. 4.1 MB/s | 3.1 MB 00:00 2024-05-03T18:08:07.998 INFO:teuthology.orchestra.run.smithi099.stdout:(6/152): ceph-mds-19.0.0-3450.g9a047fe7.el9.x86 1.7 MB/s | 2.4 MB 00:01 2024-05-03T18:08:08.361 INFO:teuthology.orchestra.run.smithi033.stdout:(5/152): ceph-mgr-19.0.0-3450.g9a047fe7.el9.x86 2.8 MB/s | 1.5 MB 00:00 2024-05-03T18:08:08.397 INFO:teuthology.orchestra.run.smithi150.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3450.g9a047 4.0 MB/s | 3.5 MB 00:00 2024-05-03T18:08:08.503 INFO:teuthology.orchestra.run.smithi033.stdout:(6/152): ceph-mds-19.0.0-3450.g9a047fe7.el9.x86 2.9 MB/s | 2.4 MB 00:00 2024-05-03T18:08:08.545 INFO:teuthology.orchestra.run.smithi150.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3450.g9a 1.8 MB/s | 274 kB 00:00 2024-05-03T18:08:08.639 INFO:teuthology.orchestra.run.smithi150.stdout:(33/152): ceph-mgr-rook-19.0.0-3450.g9a047fe7.e 529 kB/s | 49 kB 00:00 2024-05-03T18:08:08.765 INFO:teuthology.orchestra.run.smithi150.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3450.g9 132 kB/s | 16 kB 00:00 2024-05-03T18:08:08.824 INFO:teuthology.orchestra.run.smithi099.stdout:(7/152): ceph-mon-19.0.0-3450.g9a047fe7.el9.x86 3.5 MB/s | 4.7 MB 00:01 2024-05-03T18:08:08.856 INFO:teuthology.orchestra.run.smithi150.stdout:(35/152): ceph-volume-19.0.0-3450.g9a047fe7.el9 2.9 MB/s | 273 kB 00:00 2024-05-03T18:08:09.407 INFO:teuthology.orchestra.run.smithi150.stdout:(36/152): cephadm-19.0.0-3450.g9a047fe7.el9.noa 1.4 MB/s | 763 kB 00:00 2024-05-03T18:08:09.490 INFO:teuthology.orchestra.run.smithi150.stdout:(37/152): cephfs-top-19.0.0-3450.g9a047fe7.el9. 222 kB/s | 18 kB 00:00 2024-05-03T18:08:09.724 INFO:teuthology.orchestra.run.smithi150.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 345 kB/s | 80 kB 00:00 2024-05-03T18:08:09.820 INFO:teuthology.orchestra.run.smithi033.stdout:(7/152): ceph-mon-19.0.0-3450.g9a047fe7.el9.x86 3.2 MB/s | 4.7 MB 00:01 2024-05-03T18:08:09.908 INFO:teuthology.orchestra.run.smithi150.stdout:(39/152): ceph-mgr-diskprediction-local-19.0.0- 3.6 MB/s | 7.4 MB 00:02 2024-05-03T18:08:09.983 INFO:teuthology.orchestra.run.smithi150.stdout:(40/152): libgfortran-11.4.1-3.el9.x86_64.rpm 3.0 MB/s | 803 kB 00:00 2024-05-03T18:08:10.125 INFO:teuthology.orchestra.run.smithi150.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.4 MB/s | 253 kB 00:00 2024-05-03T18:08:10.200 INFO:teuthology.orchestra.run.smithi150.stdout:(42/152): libquadmath-11.4.1-3.el9.x86_64.rpm 671 kB/s | 195 kB 00:00 2024-05-03T18:08:10.276 INFO:teuthology.orchestra.run.smithi150.stdout:(43/152): python3-cryptography-36.0.1-4.el9.x86 8.2 MB/s | 1.2 MB 00:00 2024-05-03T18:08:10.334 INFO:teuthology.orchestra.run.smithi150.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 803 kB/s | 106 kB 00:00 2024-05-03T18:08:10.385 INFO:teuthology.orchestra.run.smithi150.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 1.2 MB/s | 135 kB 00:00 2024-05-03T18:08:10.418 INFO:teuthology.orchestra.run.smithi150.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 1.5 MB/s | 125 kB 00:00 2024-05-03T18:08:10.477 INFO:teuthology.orchestra.run.smithi150.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 2.3 MB/s | 215 kB 00:00 2024-05-03T18:08:11.157 INFO:teuthology.orchestra.run.smithi099.stdout:(8/152): ceph-common-19.0.0-3450.g9a047fe7.el9. 3.6 MB/s | 21 MB 00:05 2024-05-03T18:08:11.261 INFO:teuthology.orchestra.run.smithi150.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 819 kB/s | 689 kB 00:00 2024-05-03T18:08:11.332 INFO:teuthology.orchestra.run.smithi099.stdout:(9/152): ceph-radosgw-19.0.0-3450.g9a047fe7.el9 4.2 MB/s | 11 MB 00:02 2024-05-03T18:08:11.344 INFO:teuthology.orchestra.run.smithi150.stdout:(49/152): bison-3.7.4-5.el9.x86_64.rpm 1.1 MB/s | 943 kB 00:00 2024-05-03T18:08:11.357 INFO:teuthology.orchestra.run.smithi099.stdout:(10/152): ceph-selinux-19.0.0-3450.g9a047fe7.el 126 kB/s | 25 kB 00:00 2024-05-03T18:08:11.403 INFO:teuthology.orchestra.run.smithi150.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 758 kB/s | 107 kB 00:00 2024-05-03T18:08:11.466 INFO:teuthology.orchestra.run.smithi099.stdout:(11/152): cephfs-mirror-19.0.0-3450.g9a047fe7.e 2.0 MB/s | 225 kB 00:00 2024-05-03T18:08:11.470 INFO:teuthology.orchestra.run.smithi150.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 172 kB/s | 21 kB 00:00 2024-05-03T18:08:11.528 INFO:teuthology.orchestra.run.smithi150.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 2.5 MB/s | 313 kB 00:00 2024-05-03T18:08:11.533 INFO:teuthology.orchestra.run.smithi099.stdout:(12/152): cephfs-shell-19.0.0-3450.g9a047fe7.el 410 kB/s | 27 kB 00:00 2024-05-03T18:08:11.587 INFO:teuthology.orchestra.run.smithi150.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 281 kB/s | 33 kB 00:00 2024-05-03T18:08:11.625 INFO:teuthology.orchestra.run.smithi099.stdout:(13/152): libcephfs-devel-19.0.0-3450.g9a047fe7 349 kB/s | 32 kB 00:00 2024-05-03T18:08:11.704 INFO:teuthology.orchestra.run.smithi150.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 157 kB/s | 18 kB 00:00 2024-05-03T18:08:11.829 INFO:teuthology.orchestra.run.smithi150.stdout:(55/152): libaio-devel-0.3.111-13.el9.x86_64.rp 90 kB/s | 11 kB 00:00 2024-05-03T18:08:11.859 INFO:teuthology.orchestra.run.smithi099.stdout:(14/152): libcephfs2-19.0.0-3450.g9a047fe7.el9. 3.1 MB/s | 747 kB 00:00 2024-05-03T18:08:11.963 INFO:teuthology.orchestra.run.smithi150.stdout:(56/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.2 MB/s | 160 kB 00:00 2024-05-03T18:08:12.079 INFO:teuthology.orchestra.run.smithi033.stdout:(8/152): ceph-radosgw-19.0.0-3450.g9a047fe7.el9 4.7 MB/s | 11 MB 00:02 2024-05-03T18:08:12.113 INFO:teuthology.orchestra.run.smithi150.stdout:(57/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 5.1 MB/s | 3.0 MB 00:00 2024-05-03T18:08:12.120 INFO:teuthology.orchestra.run.smithi033.stdout:(9/152): ceph-selinux-19.0.0-3450.g9a047fe7.el9 621 kB/s | 25 kB 00:00 2024-05-03T18:08:12.126 INFO:teuthology.orchestra.run.smithi099.stdout:(15/152): ceph-osd-19.0.0-3450.g9a047fe7.el9.x8 4.1 MB/s | 17 MB 00:04 2024-05-03T18:08:12.138 INFO:teuthology.orchestra.run.smithi150.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 259 kB/s | 45 kB 00:00 2024-05-03T18:08:12.159 INFO:teuthology.orchestra.run.smithi099.stdout:(16/152): libcephsqlite-19.0.0-3450.g9a047fe7.e 546 kB/s | 163 kB 00:00 2024-05-03T18:08:12.217 INFO:teuthology.orchestra.run.smithi099.stdout:(17/152): librados-devel-19.0.0-3450.g9a047fe7. 1.4 MB/s | 127 kB 00:00 2024-05-03T18:08:12.272 INFO:teuthology.orchestra.run.smithi150.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 4.1 MB/s | 662 kB 00:00 2024-05-03T18:08:12.322 INFO:teuthology.orchestra.run.smithi150.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 3.1 MB/s | 582 kB 00:00 2024-05-03T18:08:12.334 INFO:teuthology.orchestra.run.smithi099.stdout:(18/152): libradosstriper1-19.0.0-3450.g9a047fe 2.8 MB/s | 500 kB 00:00 2024-05-03T18:08:12.373 INFO:teuthology.orchestra.run.smithi150.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 254 kB/s | 25 kB 00:00 2024-05-03T18:08:12.393 INFO:teuthology.orchestra.run.smithi099.stdout:(19/152): python3-ceph-argparse-19.0.0-3450.g9a 781 kB/s | 45 kB 00:00 2024-05-03T18:08:12.461 INFO:teuthology.orchestra.run.smithi150.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 343 kB/s | 47 kB 00:00 2024-05-03T18:08:12.468 INFO:teuthology.orchestra.run.smithi099.stdout:(20/152): python3-ceph-common-19.0.0-3450.g9a04 1.8 MB/s | 135 kB 00:00 2024-05-03T18:08:12.506 INFO:teuthology.orchestra.run.smithi150.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.1 MB/s | 292 kB 00:00 2024-05-03T18:08:12.544 INFO:teuthology.orchestra.run.smithi099.stdout:(21/152): python3-cephfs-19.0.0-3450.g9a047fe7. 2.1 MB/s | 161 kB 00:00 2024-05-03T18:08:12.574 INFO:teuthology.orchestra.run.smithi150.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 1.7 MB/s | 188 kB 00:00 2024-05-03T18:08:12.624 INFO:teuthology.orchestra.run.smithi150.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 303 kB/s | 35 kB 00:00 2024-05-03T18:08:12.686 INFO:teuthology.orchestra.run.smithi099.stdout:(22/152): python3-rados-19.0.0-3450.g9a047fe7.e 2.2 MB/s | 320 kB 00:00 2024-05-03T18:08:12.811 INFO:teuthology.orchestra.run.smithi099.stdout:(23/152): python3-rbd-19.0.0-3450.g9a047fe7.el9 2.3 MB/s | 298 kB 00:00 2024-05-03T18:08:13.128 INFO:teuthology.orchestra.run.smithi099.stdout:(24/152): python3-rgw-19.0.0-3450.g9a047fe7.el9 316 kB/s | 100 kB 00:00 2024-05-03T18:08:13.204 INFO:teuthology.orchestra.run.smithi099.stdout:(25/152): rbd-fuse-19.0.0-3450.g9a047fe7.el9.x8 1.1 MB/s | 85 kB 00:00 2024-05-03T18:08:13.590 INFO:teuthology.orchestra.run.smithi033.stdout:(10/152): ceph-osd-19.0.0-3450.g9a047fe7.el9.x8 3.3 MB/s | 17 MB 00:05 2024-05-03T18:08:13.696 INFO:teuthology.orchestra.run.smithi099.stdout:(26/152): librgw2-19.0.0-3450.g9a047fe7.el9.x86 3.6 MB/s | 5.4 MB 00:01 2024-05-03T18:08:13.804 INFO:teuthology.orchestra.run.smithi099.stdout:(27/152): rbd-mirror-19.0.0-3450.g9a047fe7.el9. 5.2 MB/s | 3.1 MB 00:00 2024-05-03T18:08:13.838 INFO:teuthology.orchestra.run.smithi099.stdout:(28/152): rbd-nbd-19.0.0-3450.g9a047fe7.el9.x86 1.2 MB/s | 170 kB 00:00 2024-05-03T18:08:13.863 INFO:teuthology.orchestra.run.smithi099.stdout:(29/152): ceph-grafana-dashboards-19.0.0-3450.g 593 kB/s | 34 kB 00:00 2024-05-03T18:08:13.879 INFO:teuthology.orchestra.run.smithi033.stdout:(11/152): ceph-common-19.0.0-3450.g9a047fe7.el9 2.6 MB/s | 21 MB 00:08 2024-05-03T18:08:13.922 INFO:teuthology.orchestra.run.smithi099.stdout:(30/152): ceph-mgr-cephadm-19.0.0-3450.g9a047fe 1.7 MB/s | 142 kB 00:00 2024-05-03T18:08:13.938 INFO:teuthology.orchestra.run.smithi033.stdout:(12/152): cephfs-mirror-19.0.0-3450.g9a047fe7.e 644 kB/s | 225 kB 00:00 2024-05-03T18:08:13.963 INFO:teuthology.orchestra.run.smithi033.stdout:(13/152): cephfs-shell-19.0.0-3450.g9a047fe7.el 328 kB/s | 27 kB 00:00 2024-05-03T18:08:13.983 INFO:teuthology.orchestra.run.smithi150.stdout:(66/152): ceph-test-19.0.0-3450.g9a047fe7.el9.x 5.7 MB/s | 49 MB 00:08 2024-05-03T18:08:14.030 INFO:teuthology.orchestra.run.smithi033.stdout:(14/152): libcephfs-devel-19.0.0-3450.g9a047fe7 348 kB/s | 32 kB 00:00 2024-05-03T18:08:14.197 INFO:teuthology.orchestra.run.smithi033.stdout:(15/152): libcephsqlite-19.0.0-3450.g9a047fe7.e 984 kB/s | 163 kB 00:00 2024-05-03T18:08:14.347 INFO:teuthology.orchestra.run.smithi033.stdout:(16/152): librados-devel-19.0.0-3450.g9a047fe7. 851 kB/s | 127 kB 00:00 2024-05-03T18:08:14.439 INFO:teuthology.orchestra.run.smithi033.stdout:(17/152): libcephfs2-19.0.0-3450.g9a047fe7.el9. 1.5 MB/s | 747 kB 00:00 2024-05-03T18:08:14.482 INFO:teuthology.orchestra.run.smithi099.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3450.g9a047 5.7 MB/s | 3.5 MB 00:00 2024-05-03T18:08:14.572 INFO:teuthology.orchestra.run.smithi099.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3450.g9a 3.0 MB/s | 274 kB 00:00 2024-05-03T18:08:14.622 INFO:teuthology.orchestra.run.smithi099.stdout:(33/152): ceph-mgr-rook-19.0.0-3450.g9a047fe7.e 1.0 MB/s | 49 kB 00:00 2024-05-03T18:08:14.639 INFO:teuthology.orchestra.run.smithi033.stdout:(18/152): libradosstriper1-19.0.0-3450.g9a047fe 1.7 MB/s | 500 kB 00:00 2024-05-03T18:08:14.673 INFO:teuthology.orchestra.run.smithi099.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3450.g9 334 kB/s | 16 kB 00:00 2024-05-03T18:08:14.706 INFO:teuthology.orchestra.run.smithi033.stdout:(19/152): python3-ceph-argparse-19.0.0-3450.g9a 685 kB/s | 45 kB 00:00 2024-05-03T18:08:14.756 INFO:teuthology.orchestra.run.smithi099.stdout:(35/152): ceph-volume-19.0.0-3450.g9a047fe7.el9 3.2 MB/s | 273 kB 00:00 2024-05-03T18:08:14.848 INFO:teuthology.orchestra.run.smithi033.stdout:(20/152): python3-ceph-common-19.0.0-3450.g9a04 963 kB/s | 135 kB 00:00 2024-05-03T18:08:14.915 INFO:teuthology.orchestra.run.smithi099.stdout:(36/152): cephadm-19.0.0-3450.g9a047fe7.el9.noa 4.7 MB/s | 763 kB 00:00 2024-05-03T18:08:14.966 INFO:teuthology.orchestra.run.smithi150.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 1.6 MB/s | 1.6 MB 00:00 2024-05-03T18:08:14.974 INFO:teuthology.orchestra.run.smithi099.stdout:(37/152): cephfs-top-19.0.0-3450.g9a047fe7.el9. 317 kB/s | 18 kB 00:00 2024-05-03T18:08:15.023 INFO:teuthology.orchestra.run.smithi033.stdout:(21/152): python3-cephfs-19.0.0-3450.g9a047fe7. 927 kB/s | 161 kB 00:00 2024-05-03T18:08:15.108 INFO:teuthology.orchestra.run.smithi150.stdout:(68/152): postgresql-contrib-13.14-1.el9.x86_64 6.3 MB/s | 911 kB 00:00 2024-05-03T18:08:15.232 INFO:teuthology.orchestra.run.smithi033.stdout:(22/152): python3-rados-19.0.0-3450.g9a047fe7.e 1.5 MB/s | 320 kB 00:00 2024-05-03T18:08:15.242 INFO:teuthology.orchestra.run.smithi150.stdout:(69/152): postgresql-private-libs-13.14-1.el9.x 1.0 MB/s | 133 kB 00:00 2024-05-03T18:08:15.358 INFO:teuthology.orchestra.run.smithi099.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 210 kB/s | 80 kB 00:00 2024-05-03T18:08:15.524 INFO:teuthology.orchestra.run.smithi033.stdout:(23/152): python3-rbd-19.0.0-3450.g9a047fe7.el9 1.0 MB/s | 298 kB 00:00 2024-05-03T18:08:15.566 INFO:teuthology.orchestra.run.smithi099.stdout:(39/152): libgfortran-11.4.1-3.el9.x86_64.rpm 3.8 MB/s | 803 kB 00:00 2024-05-03T18:08:15.691 INFO:teuthology.orchestra.run.smithi033.stdout:(24/152): python3-rgw-19.0.0-3450.g9a047fe7.el9 602 kB/s | 100 kB 00:00 2024-05-03T18:08:15.700 INFO:teuthology.orchestra.run.smithi099.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 1.4 MB/s | 195 kB 00:00 2024-05-03T18:08:15.808 INFO:teuthology.orchestra.run.smithi033.stdout:(25/152): rbd-fuse-19.0.0-3450.g9a047fe7.el9.x8 737 kB/s | 85 kB 00:00 2024-05-03T18:08:15.884 INFO:teuthology.orchestra.run.smithi150.stdout:(70/152): postgresql-server-13.14-1.el9.x86_64. 9.0 MB/s | 5.8 MB 00:00 2024-05-03T18:08:15.906 INFO:teuthology.orchestra.run.smithi099.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.2 MB/s | 253 kB 00:00 2024-05-03T18:08:16.051 INFO:teuthology.orchestra.run.smithi150.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 6.1 MB/s | 1.0 MB 00:00 2024-05-03T18:08:16.176 INFO:teuthology.orchestra.run.smithi150.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 703 kB/s | 87 kB 00:00 2024-05-03T18:08:16.250 INFO:teuthology.orchestra.run.smithi099.stdout:(42/152): python3-cryptography-36.0.1-4.el9.x86 3.6 MB/s | 1.2 MB 00:00 2024-05-03T18:08:16.434 INFO:teuthology.orchestra.run.smithi099.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 585 kB/s | 106 kB 00:00 2024-05-03T18:08:16.584 INFO:teuthology.orchestra.run.smithi099.stdout:(44/152): python3-pycparser-2.20-6.el9.noarch.r 906 kB/s | 135 kB 00:00 2024-05-03T18:08:16.718 INFO:teuthology.orchestra.run.smithi099.stdout:(45/152): python3-requests-2.25.1-8.el9.noarch. 940 kB/s | 125 kB 00:00 2024-05-03T18:08:16.766 INFO:teuthology.orchestra.run.smithi033.stdout:(26/152): librgw2-19.0.0-3450.g9a047fe7.el9.x86 2.3 MB/s | 5.4 MB 00:02 2024-05-03T18:08:16.868 INFO:teuthology.orchestra.run.smithi099.stdout:(46/152): ceph-mgr-diskprediction-local-19.0.0- 2.5 MB/s | 7.4 MB 00:02 2024-05-03T18:08:16.900 INFO:teuthology.orchestra.run.smithi033.stdout:(27/152): rbd-nbd-19.0.0-3450.g9a047fe7.el9.x86 1.3 MB/s | 170 kB 00:00 2024-05-03T18:08:16.967 INFO:teuthology.orchestra.run.smithi033.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3450.g 518 kB/s | 34 kB 00:00 2024-05-03T18:08:17.059 INFO:teuthology.orchestra.run.smithi150.stdout:(73/152): python3-babel-2.9.1-2.el9.noarch.rpm 6.8 MB/s | 6.0 MB 00:00 2024-05-03T18:08:17.076 INFO:teuthology.orchestra.run.smithi033.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3450.g9a047fe 1.3 MB/s | 142 kB 00:00 2024-05-03T18:08:17.185 INFO:teuthology.orchestra.run.smithi150.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 1.9 MB/s | 244 kB 00:00 2024-05-03T18:08:17.302 INFO:teuthology.orchestra.run.smithi150.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 2.1 MB/s | 248 kB 00:00 2024-05-03T18:08:17.410 INFO:teuthology.orchestra.run.smithi150.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 238 kB/s | 26 kB 00:00 2024-05-03T18:08:17.435 INFO:teuthology.orchestra.run.smithi099.stdout:(47/152): ceph-test-19.0.0-3450.g9a047fe7.el9.x 8.1 MB/s | 49 MB 00:06 2024-05-03T18:08:17.469 INFO:teuthology.orchestra.run.smithi099.stdout:(48/152): python3-urllib3-1.26.5-5.el9.noarch.r 286 kB/s | 215 kB 00:00 2024-05-03T18:08:17.519 INFO:teuthology.orchestra.run.smithi150.stdout:(77/152): python3-jsonpointer-2.0-4.el9.noarch. 178 kB/s | 19 kB 00:00 2024-05-03T18:08:17.627 INFO:teuthology.orchestra.run.smithi150.stdout:(78/152): python3-mako-1.1.4-6.el9.noarch.rpm 1.6 MB/s | 172 kB 00:00 2024-05-03T18:08:17.745 INFO:teuthology.orchestra.run.smithi150.stdout:(79/152): python3-markupsafe-1.1.1-12.el9.x86_6 298 kB/s | 35 kB 00:00 2024-05-03T18:08:18.085 INFO:teuthology.orchestra.run.smithi033.stdout:(30/152): ceph-test-19.0.0-3450.g9a047fe7.el9.x 8.2 MB/s | 49 MB 00:05 2024-05-03T18:08:18.160 INFO:teuthology.orchestra.run.smithi033.stdout:(31/152): rbd-mirror-19.0.0-3450.g9a047fe7.el9. 1.3 MB/s | 3.1 MB 00:02 2024-05-03T18:08:18.343 INFO:teuthology.orchestra.run.smithi033.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3450.g9a 1.5 MB/s | 274 kB 00:00 2024-05-03T18:08:18.352 INFO:teuthology.orchestra.run.smithi099.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 121 kB/s | 107 kB 00:00 2024-05-03T18:08:18.410 INFO:teuthology.orchestra.run.smithi033.stdout:(33/152): ceph-mgr-rook-19.0.0-3450.g9a047fe7.e 750 kB/s | 49 kB 00:00 2024-05-03T18:08:18.436 INFO:teuthology.orchestra.run.smithi099.stdout:(50/152): automake-1.16.2-8.el9.noarch.rpm 440 kB/s | 689 kB 00:01 2024-05-03T18:08:18.477 INFO:teuthology.orchestra.run.smithi033.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3450.g9 249 kB/s | 16 kB 00:00 2024-05-03T18:08:18.570 INFO:teuthology.orchestra.run.smithi099.stdout:(51/152): bison-3.7.4-5.el9.x86_64.rpm 832 kB/s | 943 kB 00:01 2024-05-03T18:08:18.578 INFO:teuthology.orchestra.run.smithi033.stdout:(35/152): ceph-mgr-dashboard-19.0.0-3450.g9a047 2.4 MB/s | 3.5 MB 00:01 2024-05-03T18:08:18.603 INFO:teuthology.orchestra.run.smithi150.stdout:(80/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 5.9 MB/s | 5.1 MB 00:00 2024-05-03T18:08:18.695 INFO:teuthology.orchestra.run.smithi033.stdout:(36/152): ceph-volume-19.0.0-3450.g9a047fe7.el9 1.2 MB/s | 273 kB 00:00 2024-05-03T18:08:18.720 INFO:teuthology.orchestra.run.smithi150.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.1 MB/s | 250 kB 00:00 2024-05-03T18:08:18.828 INFO:teuthology.orchestra.run.smithi033.stdout:(37/152): ceph-mgr-diskprediction-local-19.0.0- 10 MB/s | 7.4 MB 00:00 2024-05-03T18:08:18.837 INFO:teuthology.orchestra.run.smithi150.stdout:(82/152): python3-oauthlib-3.1.1-5.el9.noarch.r 1.9 MB/s | 222 kB 00:00 2024-05-03T18:08:18.845 INFO:teuthology.orchestra.run.smithi033.stdout:(38/152): cephfs-top-19.0.0-3450.g9a047fe7.el9. 122 kB/s | 18 kB 00:00 2024-05-03T18:08:18.904 INFO:teuthology.orchestra.run.smithi033.stdout:(39/152): cephadm-19.0.0-3450.g9a047fe7.el9.noa 2.3 MB/s | 763 kB 00:00 2024-05-03T18:08:18.938 INFO:teuthology.orchestra.run.smithi033.stdout:(40/152): fuse-2.9.9-15.el9.x86_64.rpm 747 kB/s | 80 kB 00:00 2024-05-03T18:08:18.962 INFO:teuthology.orchestra.run.smithi150.stdout:(83/152): python3-packaging-20.9-5.el9.noarch.r 618 kB/s | 77 kB 00:00 2024-05-03T18:08:18.996 INFO:teuthology.orchestra.run.smithi033.stdout:(41/152): libgfortran-11.4.1-3.el9.x86_64.rpm 5.2 MB/s | 803 kB 00:00 2024-05-03T18:08:19.038 INFO:teuthology.orchestra.run.smithi033.stdout:(42/152): libquadmath-11.4.1-3.el9.x86_64.rpm 1.4 MB/s | 195 kB 00:00 2024-05-03T18:08:19.070 INFO:teuthology.orchestra.run.smithi099.stdout:(52/152): elfutils-libelf-devel-0.190-2.el9.x86 30 kB/s | 21 kB 00:00 2024-05-03T18:08:19.071 INFO:teuthology.orchestra.run.smithi150.stdout:(84/152): python3-prettytable-0.7.2-27.el9.noar 392 kB/s | 42 kB 00:00 2024-05-03T18:08:19.071 INFO:teuthology.orchestra.run.smithi033.stdout:(43/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.9 MB/s | 253 kB 00:00 2024-05-03T18:08:19.138 INFO:teuthology.orchestra.run.smithi033.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 1.0 MB/s | 106 kB 00:00 2024-05-03T18:08:19.155 INFO:teuthology.orchestra.run.smithi033.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 1.6 MB/s | 135 kB 00:00 2024-05-03T18:08:19.187 INFO:teuthology.orchestra.run.smithi150.stdout:(85/152): python3-protobuf-3.14.0-13.el9.noarch 2.3 MB/s | 269 kB 00:00 2024-05-03T18:08:19.206 INFO:teuthology.orchestra.run.smithi033.stdout:(46/152): python3-cryptography-36.0.1-4.el9.x86 5.9 MB/s | 1.2 MB 00:00 2024-05-03T18:08:19.248 INFO:teuthology.orchestra.run.smithi033.stdout:(47/152): python3-requests-2.25.1-8.el9.noarch. 1.1 MB/s | 125 kB 00:00 2024-05-03T18:08:19.270 INFO:teuthology.orchestra.run.smithi099.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 47 kB/s | 33 kB 00:00 2024-05-03T18:08:19.298 INFO:teuthology.orchestra.run.smithi033.stdout:(48/152): python3-urllib3-1.26.5-5.el9.noarch.r 1.5 MB/s | 215 kB 00:00 2024-05-03T18:08:19.412 INFO:teuthology.orchestra.run.smithi099.stdout:(54/152): flex-2.6.4-9.el9.x86_64.rpm 322 kB/s | 313 kB 00:00 2024-05-03T18:08:19.421 INFO:teuthology.orchestra.run.smithi150.stdout:(86/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 680 kB/s | 159 kB 00:00 2024-05-03T18:08:19.431 INFO:teuthology.orchestra.run.smithi033.stdout:(49/152): automake-1.16.2-8.el9.noarch.rpm 3.0 MB/s | 689 kB 00:00 2024-05-03T18:08:19.457 INFO:teuthology.orchestra.run.smithi033.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 678 kB/s | 107 kB 00:00 2024-05-03T18:08:19.482 INFO:teuthology.orchestra.run.smithi033.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 431 kB/s | 21 kB 00:00 2024-05-03T18:08:19.538 INFO:teuthology.orchestra.run.smithi150.stdout:(87/152): python3-pyasn1-modules-0.4.8-6.el9.no 2.4 MB/s | 279 kB 00:00 2024-05-03T18:08:19.541 INFO:teuthology.orchestra.run.smithi033.stdout:(52/152): bison-3.7.4-5.el9.x86_64.rpm 3.2 MB/s | 943 kB 00:00 2024-05-03T18:08:19.566 INFO:teuthology.orchestra.run.smithi033.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 395 kB/s | 33 kB 00:00 2024-05-03T18:08:19.599 INFO:teuthology.orchestra.run.smithi033.stdout:(54/152): flex-2.6.4-9.el9.x86_64.rpm 2.2 MB/s | 313 kB 00:00 2024-05-03T18:08:19.625 INFO:teuthology.orchestra.run.smithi033.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 314 kB/s | 18 kB 00:00 2024-05-03T18:08:19.658 INFO:teuthology.orchestra.run.smithi033.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 193 kB/s | 11 kB 00:00 2024-05-03T18:08:19.672 INFO:teuthology.orchestra.run.smithi150.stdout:(88/152): python3-pytz-2021.1-5.el9.noarch.rpm 384 kB/s | 51 kB 00:00 2024-05-03T18:08:19.692 INFO:teuthology.orchestra.run.smithi033.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.4 MB/s | 160 kB 00:00 2024-05-03T18:08:19.717 INFO:teuthology.orchestra.run.smithi033.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 786 kB/s | 45 kB 00:00 2024-05-03T18:08:19.768 INFO:teuthology.orchestra.run.smithi033.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 8.7 MB/s | 662 kB 00:00 2024-05-03T18:08:19.818 INFO:teuthology.orchestra.run.smithi033.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 5.7 MB/s | 582 kB 00:00 2024-05-03T18:08:19.843 INFO:teuthology.orchestra.run.smithi033.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 342 kB/s | 25 kB 00:00 2024-05-03T18:08:19.868 INFO:teuthology.orchestra.run.smithi033.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 960 kB/s | 47 kB 00:00 2024-05-03T18:08:19.879 INFO:teuthology.orchestra.run.smithi099.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 30 kB/s | 18 kB 00:00 2024-05-03T18:08:19.881 INFO:teuthology.orchestra.run.smithi150.stdout:(89/152): python3-requests-oauthlib-1.3.0-12.el 258 kB/s | 54 kB 00:00 2024-05-03T18:08:19.902 INFO:teuthology.orchestra.run.smithi033.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.0 MB/s | 292 kB 00:00 2024-05-03T18:08:19.936 INFO:teuthology.orchestra.run.smithi033.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 2.8 MB/s | 188 kB 00:00 2024-05-03T18:08:19.961 INFO:teuthology.orchestra.run.smithi033.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 607 kB/s | 35 kB 00:00 2024-05-03T18:08:20.038 INFO:teuthology.orchestra.run.smithi099.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 18 kB/s | 11 kB 00:00 2024-05-03T18:08:20.045 INFO:teuthology.orchestra.run.smithi033.stdout:(66/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 5.9 MB/s | 3.0 MB 00:00 2024-05-03T18:08:20.162 INFO:teuthology.orchestra.run.smithi033.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-05-03T18:08:20.279 INFO:teuthology.orchestra.run.smithi033.stdout:(68/152): openblas-openmp-0.3.21-2.el9.x86_64.r 14 MB/s | 4.7 MB 00:00 2024-05-03T18:08:20.371 INFO:teuthology.orchestra.run.smithi033.stdout:(69/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-05-03T18:08:20.404 INFO:teuthology.orchestra.run.smithi033.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 1.0 MB/s | 133 kB 00:00 2024-05-03T18:08:20.413 INFO:teuthology.orchestra.run.smithi099.stdout:(57/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.2 MB/s | 3.0 MB 00:01 2024-05-03T18:08:20.471 INFO:teuthology.orchestra.run.smithi033.stdout:(71/152): postgresql-contrib-13.14-1.el9.x86_64 2.9 MB/s | 911 kB 00:00 2024-05-03T18:08:20.538 INFO:teuthology.orchestra.run.smithi033.stdout:(72/152): protobuf-3.14.0-13.el9.x86_64.rpm 7.6 MB/s | 1.0 MB 00:00 2024-05-03T18:08:20.555 INFO:teuthology.orchestra.run.smithi033.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.0 MB/s | 87 kB 00:00 2024-05-03T18:08:20.647 INFO:teuthology.orchestra.run.smithi033.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 2.6 MB/s | 244 kB 00:00 2024-05-03T18:08:20.714 INFO:teuthology.orchestra.run.smithi033.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 3.7 MB/s | 248 kB 00:00 2024-05-03T18:08:20.805 INFO:teuthology.orchestra.run.smithi099.stdout:(58/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 173 kB/s | 160 kB 00:00 2024-05-03T18:08:20.823 INFO:teuthology.orchestra.run.smithi033.stdout:(76/152): postgresql-server-13.14-1.el9.x86_64. 13 MB/s | 5.8 MB 00:00 2024-05-03T18:08:20.830 INFO:teuthology.orchestra.run.smithi099.stdout:(59/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 57 kB/s | 45 kB 00:00 2024-05-03T18:08:20.840 INFO:teuthology.orchestra.run.smithi033.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 206 kB/s | 26 kB 00:00 2024-05-03T18:08:20.865 INFO:teuthology.orchestra.run.smithi033.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 468 kB/s | 19 kB 00:00 2024-05-03T18:08:20.898 INFO:teuthology.orchestra.run.smithi033.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-05-03T18:08:20.923 INFO:teuthology.orchestra.run.smithi033.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 602 kB/s | 35 kB 00:00 2024-05-03T18:08:20.989 INFO:teuthology.orchestra.run.smithi150.stdout:(90/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 542 kB/s | 4.4 MB 00:08 2024-05-03T18:08:21.049 INFO:teuthology.orchestra.run.smithi033.stdout:(81/152): python3-babel-2.9.1-2.el9.noarch.rpm 12 MB/s | 6.0 MB 00:00 2024-05-03T18:08:21.082 INFO:teuthology.orchestra.run.smithi033.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.6 MB/s | 250 kB 00:00 2024-05-03T18:08:21.106 INFO:teuthology.orchestra.run.smithi150.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 360 kB/s | 42 kB 00:00 2024-05-03T18:08:21.116 INFO:teuthology.orchestra.run.smithi033.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 3.3 MB/s | 222 kB 00:00 2024-05-03T18:08:21.150 INFO:teuthology.orchestra.run.smithi033.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.1 MB/s | 77 kB 00:00 2024-05-03T18:08:21.175 INFO:teuthology.orchestra.run.smithi033.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 733 kB/s | 42 kB 00:00 2024-05-03T18:08:21.208 INFO:teuthology.orchestra.run.smithi033.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 4.6 MB/s | 269 kB 00:00 2024-05-03T18:08:21.215 INFO:teuthology.orchestra.run.smithi150.stdout:(92/152): openblas-openmp-0.3.21-2.el9.x86_64.r 562 kB/s | 4.7 MB 00:08 2024-05-03T18:08:21.240 INFO:teuthology.orchestra.run.smithi150.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 333 kB/s | 44 kB 00:00 2024-05-03T18:08:21.242 INFO:teuthology.orchestra.run.smithi033.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 159 kB 00:00 2024-05-03T18:08:21.276 INFO:teuthology.orchestra.run.smithi033.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 279 kB 00:00 2024-05-03T18:08:21.301 INFO:teuthology.orchestra.run.smithi033.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 884 kB/s | 51 kB 00:00 2024-05-03T18:08:21.324 INFO:teuthology.orchestra.run.smithi150.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 2.0 MB/s | 221 kB 00:00 2024-05-03T18:08:21.326 INFO:teuthology.orchestra.run.smithi033.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 1.1 MB/s | 54 kB 00:00 2024-05-03T18:08:21.358 INFO:teuthology.orchestra.run.smithi150.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 309 kB/s | 36 kB 00:00 2024-05-03T18:08:21.376 INFO:teuthology.orchestra.run.smithi033.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 845 kB/s | 42 kB 00:00 2024-05-03T18:08:21.427 INFO:teuthology.orchestra.run.smithi033.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 900 kB/s | 44 kB 00:00 2024-05-03T18:08:21.450 INFO:teuthology.orchestra.run.smithi150.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 381 kB/s | 47 kB 00:00 2024-05-03T18:08:21.491 INFO:teuthology.orchestra.run.smithi150.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 2.2 MB/s | 305 kB 00:00 2024-05-03T18:08:21.539 INFO:teuthology.orchestra.run.smithi099.stdout:(60/152): librdkafka-1.6.1-102.el9.x86_64.rpm 589 kB/s | 662 kB 00:01 2024-05-03T18:08:21.558 INFO:teuthology.orchestra.run.smithi150.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 537 kB/s | 58 kB 00:00 2024-05-03T18:08:21.561 INFO:teuthology.orchestra.run.smithi033.stdout:(93/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 7.7 MB/s | 5.1 MB 00:00 2024-05-03T18:08:21.564 INFO:teuthology.orchestra.run.smithi099.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 35 kB/s | 25 kB 00:00 2024-05-03T18:08:21.592 INFO:teuthology.orchestra.run.smithi150.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 939 kB/s | 94 kB 00:00 2024-05-03T18:08:21.594 INFO:teuthology.orchestra.run.smithi033.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 1.3 MB/s | 221 kB 00:00 2024-05-03T18:08:21.620 INFO:teuthology.orchestra.run.smithi033.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 626 kB/s | 36 kB 00:00 2024-05-03T18:08:21.645 INFO:teuthology.orchestra.run.smithi033.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 959 kB/s | 47 kB 00:00 2024-05-03T18:08:21.668 INFO:teuthology.orchestra.run.smithi150.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 587 kB/s | 64 kB 00:00 2024-05-03T18:08:21.679 INFO:teuthology.orchestra.run.smithi033.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 5.1 MB/s | 305 kB 00:00 2024-05-03T18:08:21.701 INFO:teuthology.orchestra.run.smithi150.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 415 kB/s | 45 kB 00:00 2024-05-03T18:08:21.704 INFO:teuthology.orchestra.run.smithi033.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 1.0 MB/s | 58 kB 00:00 2024-05-03T18:08:21.729 INFO:teuthology.orchestra.run.smithi033.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.9 MB/s | 94 kB 00:00 2024-05-03T18:08:21.754 INFO:teuthology.orchestra.run.smithi033.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.3 MB/s | 64 kB 00:00 2024-05-03T18:08:21.780 INFO:teuthology.orchestra.run.smithi033.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 908 kB/s | 45 kB 00:00 2024-05-03T18:08:21.830 INFO:teuthology.orchestra.run.smithi033.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 300 kB/s | 22 kB 00:00 2024-05-03T18:08:21.906 INFO:teuthology.orchestra.run.smithi099.stdout:(62/152): libtool-2.4.6-45.el9.x86_64.rpm 530 kB/s | 582 kB 00:01 2024-05-03T18:08:21.926 INFO:teuthology.orchestra.run.smithi150.stdout:(102/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 7.9 MB/s | 16 MB 00:02 2024-05-03T18:08:21.952 INFO:teuthology.orchestra.run.smithi150.stdout:(103/152): lua-devel-5.4.4-4.el9.x86_64.rpm 79 kB/s | 22 kB 00:00 2024-05-03T18:08:22.081 INFO:teuthology.orchestra.run.smithi033.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 2.8 MB/s | 863 kB 00:00 2024-05-03T18:08:22.231 INFO:teuthology.orchestra.run.smithi099.stdout:(63/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 69 kB/s | 47 kB 00:00 2024-05-03T18:08:22.260 INFO:teuthology.orchestra.run.smithi150.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 1.5 MB/s | 863 kB 00:00 2024-05-03T18:08:22.381 INFO:teuthology.orchestra.run.smithi033.stdout:(104/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 15 MB/s | 16 MB 00:01 2024-05-03T18:08:22.498 INFO:teuthology.orchestra.run.smithi033.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 168 kB/s | 19 kB 00:00 2024-05-03T18:08:22.510 INFO:teuthology.orchestra.run.smithi150.stdout:(105/152): gperftools-libs-2.9.1-2.el9.x86_64.r 554 kB/s | 309 kB 00:00 2024-05-03T18:08:22.531 INFO:teuthology.orchestra.run.smithi033.stdout:(106/152): abseil-cpp-20211102.0-3.el9.x86_64.r 783 kB/s | 548 kB 00:00 2024-05-03T18:08:22.544 INFO:teuthology.orchestra.run.smithi150.stdout:(106/152): abseil-cpp-20211102.0-3.el9.x86_64.r 888 kB/s | 548 kB 00:00 2024-05-03T18:08:22.561 INFO:teuthology.orchestra.run.smithi150.stdout:(107/152): grpc-data-1.46.7-10.el9.noarch.rpm 65 kB/s | 19 kB 00:00 2024-05-03T18:08:22.598 INFO:teuthology.orchestra.run.smithi099.stdout:(64/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 283 kB/s | 292 kB 00:01 2024-05-03T18:08:22.611 INFO:teuthology.orchestra.run.smithi150.stdout:(108/152): libarrow-doc-9.0.0-11.el9.noarch.rpm 390 kB/s | 26 kB 00:00 2024-05-03T18:08:22.622 INFO:teuthology.orchestra.run.smithi033.stdout:(107/152): libarrow-doc-9.0.0-11.el9.noarch.rpm 291 kB/s | 26 kB 00:00 2024-05-03T18:08:22.657 INFO:teuthology.orchestra.run.smithi033.stdout:(108/152): gperftools-libs-2.9.1-2.el9.x86_64.r 537 kB/s | 309 kB 00:00 2024-05-03T18:08:22.687 INFO:teuthology.orchestra.run.smithi150.stdout:(109/152): libunwind-1.6.2-1.el9.x86_64.rpm 907 kB/s | 67 kB 00:00 2024-05-03T18:08:22.699 INFO:teuthology.orchestra.run.smithi033.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 639 kB/s | 49 kB 00:00 2024-05-03T18:08:22.712 INFO:teuthology.orchestra.run.smithi150.stdout:(110/152): liboath-2.6.7-2.el9.x86_64.rpm 324 kB/s | 49 kB 00:00 2024-05-03T18:08:22.749 INFO:teuthology.orchestra.run.smithi033.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 740 kB/s | 67 kB 00:00 2024-05-03T18:08:22.795 INFO:teuthology.orchestra.run.smithi150.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 1.4 MB/s | 151 kB 00:00 2024-05-03T18:08:22.833 INFO:teuthology.orchestra.run.smithi033.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 1.1 MB/s | 151 kB 00:00 2024-05-03T18:08:22.904 INFO:teuthology.orchestra.run.smithi150.stdout:(112/152): libarrow-9.0.0-11.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-05-03T18:08:22.932 INFO:teuthology.orchestra.run.smithi099.stdout:(65/152): lua-5.4.4-4.el9.x86_64.rpm 184 kB/s | 188 kB 00:01 2024-05-03T18:08:22.941 INFO:teuthology.orchestra.run.smithi033.stdout:(112/152): libarrow-9.0.0-11.el9.x86_64.rpm 10 MB/s | 4.4 MB 00:00 2024-05-03T18:08:22.954 INFO:teuthology.orchestra.run.smithi150.stdout:(113/152): python3-asyncssh-2.13.2-5.el9.noarch 3.4 MB/s | 548 kB 00:00 2024-05-03T18:08:22.988 INFO:teuthology.orchestra.run.smithi150.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 526 kB/s | 43 kB 00:00 2024-05-03T18:08:23.000 INFO:teuthology.orchestra.run.smithi033.stdout:(113/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 753 kB/s | 43 kB 00:00 2024-05-03T18:08:23.038 INFO:teuthology.orchestra.run.smithi150.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 389 kB/s | 32 kB 00:00 2024-05-03T18:08:23.050 INFO:teuthology.orchestra.run.smithi099.stdout:(66/152): openblas-0.3.21-2.el9.x86_64.rpm 43 kB/s | 35 kB 00:00 2024-05-03T18:08:23.059 INFO:teuthology.orchestra.run.smithi033.stdout:(114/152): python3-cachetools-4.2.4-1.el9.noarc 561 kB/s | 32 kB 00:00 2024-05-03T18:08:23.064 INFO:teuthology.orchestra.run.smithi150.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 201 kB/s | 15 kB 00:00 2024-05-03T18:08:23.097 INFO:teuthology.orchestra.run.smithi150.stdout:(117/152): parquet-libs-9.0.0-11.el9.x86_64.rpm 2.1 MB/s | 839 kB 00:00 2024-05-03T18:08:23.100 INFO:teuthology.orchestra.run.smithi033.stdout:(115/152): python3-asyncssh-2.13.2-5.el9.noarch 2.0 MB/s | 548 kB 00:00 2024-05-03T18:08:23.117 INFO:teuthology.orchestra.run.smithi033.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 259 kB/s | 15 kB 00:00 2024-05-03T18:08:23.131 INFO:teuthology.orchestra.run.smithi150.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.8 MB/s | 172 kB 00:00 2024-05-03T18:08:23.184 INFO:teuthology.orchestra.run.smithi033.stdout:(117/152): python3-cherrypy-18.6.1-2.el9.noarch 5.3 MB/s | 358 kB 00:00 2024-05-03T18:08:23.197 INFO:teuthology.orchestra.run.smithi150.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 2.7 MB/s | 358 kB 00:00 2024-05-03T18:08:23.218 INFO:teuthology.orchestra.run.smithi033.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.4 MB/s | 172 kB 00:00 2024-05-03T18:08:23.232 INFO:teuthology.orchestra.run.smithi150.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 354 kB/s | 35 kB 00:00 2024-05-03T18:08:23.251 INFO:teuthology.orchestra.run.smithi033.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.5 MB/s | 303 kB 00:00 2024-05-03T18:08:23.265 INFO:teuthology.orchestra.run.smithi150.stdout:(121/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.8 MB/s | 303 kB 00:00 2024-05-03T18:08:23.285 INFO:teuthology.orchestra.run.smithi033.stdout:(120/152): parquet-libs-9.0.0-11.el9.x86_64.rpm 1.5 MB/s | 839 kB 00:00 2024-05-03T18:08:23.299 INFO:teuthology.orchestra.run.smithi150.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 2.0 MB/s | 206 kB 00:00 2024-05-03T18:08:23.302 INFO:teuthology.orchestra.run.smithi033.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 425 kB/s | 35 kB 00:00 2024-05-03T18:08:23.336 INFO:teuthology.orchestra.run.smithi033.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 2.4 MB/s | 206 kB 00:00 2024-05-03T18:08:23.357 INFO:teuthology.orchestra.run.smithi150.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 1.5 MB/s | 144 kB 00:00 2024-05-03T18:08:23.383 INFO:teuthology.orchestra.run.smithi150.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 130 kB/s | 11 kB 00:00 2024-05-03T18:08:23.395 INFO:teuthology.orchestra.run.smithi033.stdout:(123/152): python3-jaraco-8.2.1-3.el9.noarch.rp 186 kB/s | 11 kB 00:00 2024-05-03T18:08:23.420 INFO:teuthology.orchestra.run.smithi033.stdout:(124/152): python3-grpcio-tools-1.46.7-10.el9.x 1.2 MB/s | 144 kB 00:00 2024-05-03T18:08:23.454 INFO:teuthology.orchestra.run.smithi033.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 306 kB/s | 18 kB 00:00 2024-05-03T18:08:23.458 INFO:teuthology.orchestra.run.smithi150.stdout:(125/152): python3-grpcio-1.46.7-10.el9.x86_64. 9.1 MB/s | 2.0 MB 00:00 2024-05-03T18:08:23.479 INFO:teuthology.orchestra.run.smithi033.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 400 kB/s | 23 kB 00:00 2024-05-03T18:08:23.483 INFO:teuthology.orchestra.run.smithi150.stdout:(126/152): python3-jaraco-classes-3.2.1-5.el9.n 142 kB/s | 18 kB 00:00 2024-05-03T18:08:23.508 INFO:teuthology.orchestra.run.smithi150.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 185 kB/s | 23 kB 00:00 2024-05-03T18:08:23.512 INFO:teuthology.orchestra.run.smithi033.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 337 kB/s | 19 kB 00:00 2024-05-03T18:08:23.537 INFO:teuthology.orchestra.run.smithi033.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 340 kB/s | 20 kB 00:00 2024-05-03T18:08:23.558 INFO:teuthology.orchestra.run.smithi150.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 196 kB/s | 19 kB 00:00 2024-05-03T18:08:23.592 INFO:teuthology.orchestra.run.smithi150.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 239 kB/s | 20 kB 00:00 2024-05-03T18:08:23.596 INFO:teuthology.orchestra.run.smithi033.stdout:(129/152): python3-kubernetes-26.1.0-2.el9.noar 12 MB/s | 1.0 MB 00:00 2024-05-03T18:08:23.622 INFO:teuthology.orchestra.run.smithi033.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 558 kB/s | 46 kB 00:00 2024-05-03T18:08:23.643 INFO:teuthology.orchestra.run.smithi150.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 551 kB/s | 46 kB 00:00 2024-05-03T18:08:23.655 INFO:teuthology.orchestra.run.smithi033.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 1.3 MB/s | 79 kB 00:00 2024-05-03T18:08:23.693 INFO:teuthology.orchestra.run.smithi150.stdout:(131/152): python3-kubernetes-26.1.0-2.el9.noar 5.6 MB/s | 1.0 MB 00:00 2024-05-03T18:08:23.718 INFO:teuthology.orchestra.run.smithi150.stdout:(132/152): python3-more-itertools-8.12.0-2.el9. 632 kB/s | 79 kB 00:00 2024-05-03T18:08:23.722 INFO:teuthology.orchestra.run.smithi033.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 579 kB/s | 58 kB 00:00 2024-05-03T18:08:23.743 INFO:teuthology.orchestra.run.smithi150.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 579 kB/s | 58 kB 00:00 2024-05-03T18:08:23.756 INFO:teuthology.orchestra.run.smithi033.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.7 MB/s | 272 kB 00:00 2024-05-03T18:08:23.785 INFO:teuthology.orchestra.run.smithi150.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.9 MB/s | 272 kB 00:00 2024-05-03T18:08:23.789 INFO:teuthology.orchestra.run.smithi033.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 249 kB/s | 16 kB 00:00 2024-05-03T18:08:23.814 INFO:teuthology.orchestra.run.smithi033.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-05-03T18:08:23.819 INFO:teuthology.orchestra.run.smithi150.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 221 kB/s | 16 kB 00:00 2024-05-03T18:08:23.844 INFO:teuthology.orchestra.run.smithi150.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 906 kB/s | 90 kB 00:00 2024-05-03T18:08:23.848 INFO:teuthology.orchestra.run.smithi033.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 488 kB/s | 28 kB 00:00 2024-05-03T18:08:23.869 INFO:teuthology.orchestra.run.smithi150.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 339 kB/s | 28 kB 00:00 2024-05-03T18:08:23.873 INFO:teuthology.orchestra.run.smithi033.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 530 kB/s | 31 kB 00:00 2024-05-03T18:08:23.932 INFO:teuthology.orchestra.run.smithi033.stdout:(138/152): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-05-03T18:08:23.945 INFO:teuthology.orchestra.run.smithi150.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 246 kB/s | 31 kB 00:00 2024-05-03T18:08:23.966 INFO:teuthology.orchestra.run.smithi033.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 1.6 MB/s | 188 kB 00:00 2024-05-03T18:08:23.970 INFO:teuthology.orchestra.run.smithi150.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 593 kB/s | 59 kB 00:00 2024-05-03T18:08:23.991 INFO:teuthology.orchestra.run.smithi033.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 618 kB/s | 36 kB 00:00 2024-05-03T18:08:24.004 INFO:teuthology.orchestra.run.smithi150.stdout:(140/152): python3-routes-2.5.1-5.el9.noarch.rp 1.2 MB/s | 188 kB 00:00 2024-05-03T18:08:24.033 INFO:teuthology.orchestra.run.smithi033.stdout:(141/152): python3-typing-extensions-4.4.0-2.el 772 kB/s | 51 kB 00:00 2024-05-03T18:08:24.046 INFO:teuthology.orchestra.run.smithi150.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 359 kB/s | 36 kB 00:00 2024-05-03T18:08:24.066 INFO:teuthology.orchestra.run.smithi033.stdout:(142/152): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-05-03T18:08:24.071 INFO:teuthology.orchestra.run.smithi150.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 510 kB/s | 51 kB 00:00 2024-05-03T18:08:24.107 INFO:teuthology.orchestra.run.smithi099.stdout:(67/152): openblas-openmp-0.3.21-2.el9.x86_64.r 3.1 MB/s | 4.7 MB 00:01 2024-05-03T18:08:24.113 INFO:teuthology.orchestra.run.smithi150.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-05-03T18:08:24.134 INFO:teuthology.orchestra.run.smithi033.stdout:(143/152): python3-grpcio-1.46.7-10.el9.x86_64. 2.4 MB/s | 2.0 MB 00:00 2024-05-03T18:08:24.146 INFO:teuthology.orchestra.run.smithi150.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 899 kB/s | 90 kB 00:00 2024-05-03T18:08:24.159 INFO:teuthology.orchestra.run.smithi033.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 719 kB/s | 90 kB 00:00 2024-05-03T18:08:24.180 INFO:teuthology.orchestra.run.smithi150.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 3.9 MB/s | 427 kB 00:00 2024-05-03T18:08:24.192 INFO:teuthology.orchestra.run.smithi033.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-05-03T18:08:24.205 INFO:teuthology.orchestra.run.smithi150.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 217 kB/s | 20 kB 00:00 2024-05-03T18:08:24.207 INFO:teuthology.orchestra.run.smithi099.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 1.4 MB/s | 1.6 MB 00:01 2024-05-03T18:08:24.218 INFO:teuthology.orchestra.run.smithi033.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 241 kB/s | 20 kB 00:00 2024-05-03T18:08:24.239 INFO:teuthology.orchestra.run.smithi150.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 2.0 MB/s | 192 kB 00:00 2024-05-03T18:08:24.285 INFO:teuthology.orchestra.run.smithi033.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 1.5 MB/s | 192 kB 00:00 2024-05-03T18:08:24.335 INFO:teuthology.orchestra.run.smithi033.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-05-03T18:08:24.399 INFO:teuthology.orchestra.run.smithi099.stdout:(69/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 3.2 MB/s | 4.4 MB 00:01 2024-05-03T18:08:24.471 INFO:teuthology.orchestra.run.smithi150.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 5.5 MB/s | 1.6 MB 00:00 2024-05-03T18:08:24.477 INFO:teuthology.orchestra.run.smithi033.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 9.6 MB/s | 1.3 MB 00:00 2024-05-03T18:08:24.678 INFO:teuthology.orchestra.run.smithi033.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 12 MB/s | 2.4 MB 00:00 2024-05-03T18:08:24.818 INFO:teuthology.orchestra.run.smithi150.stdout:(149/152): librbd1-19.0.0-3450.g9a047fe7.el9.x8 5.5 MB/s | 3.1 MB 00:00 2024-05-03T18:08:24.819 INFO:teuthology.orchestra.run.smithi033.stdout:(151/152): librbd1-19.0.0-3450.g9a047fe7.el9.x8 5.9 MB/s | 3.1 MB 00:00 2024-05-03T18:08:24.915 INFO:teuthology.orchestra.run.smithi150.stdout:(150/152): librados2-19.0.0-3450.g9a047fe7.el9. 4.8 MB/s | 3.4 MB 00:00 2024-05-03T18:08:25.040 INFO:teuthology.orchestra.run.smithi150.stdout:(151/152): openssl-3.2.1-1.el9.x86_64.rpm 2.4 MB/s | 1.3 MB 00:00 2024-05-03T18:08:25.183 INFO:teuthology.orchestra.run.smithi099.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 137 kB/s | 133 kB 00:00 2024-05-03T18:08:25.233 INFO:teuthology.orchestra.run.smithi099.stdout:(71/152): postgresql-contrib-13.14-1.el9.x86_64 810 kB/s | 911 kB 00:01 2024-05-03T18:08:25.257 INFO:teuthology.orchestra.run.smithi150.stdout:(152/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 5.4 MB/s | 2.4 MB 00:00 2024-05-03T18:08:25.273 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-05-03T18:08:25.274 INFO:teuthology.orchestra.run.smithi150.stdout:Total 9.6 MB/s | 222 MB 00:23 2024-05-03T18:08:25.761 INFO:teuthology.orchestra.run.smithi033.stdout:(152/152): librados2-19.0.0-3450.g9a047fe7.el9. 2.2 MB/s | 3.4 MB 00:01 2024-05-03T18:08:25.766 INFO:teuthology.orchestra.run.smithi033.stdout:-------------------------------------------------------------------------------- 2024-05-03T18:08:25.767 INFO:teuthology.orchestra.run.smithi033.stdout:Total 11 MB/s | 222 MB 00:20 2024-05-03T18:08:25.833 INFO:teuthology.orchestra.run.smithi099.stdout:(72/152): postgresql-server-13.14-1.el9.x86_64. 4.2 MB/s | 5.8 MB 00:01 2024-05-03T18:08:26.025 INFO:teuthology.orchestra.run.smithi099.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 111 kB/s | 87 kB 00:00 2024-05-03T18:08:26.267 INFO:teuthology.orchestra.run.smithi099.stdout:(74/152): protobuf-3.14.0-13.el9.x86_64.rpm 954 kB/s | 1.0 MB 00:01 2024-05-03T18:08:26.926 INFO:teuthology.orchestra.run.smithi099.stdout:(75/152): python3-devel-3.9.18-3.el9.x86_64.rpm 271 kB/s | 244 kB 00:00 2024-05-03T18:08:27.151 INFO:teuthology.orchestra.run.smithi099.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 281 kB/s | 248 kB 00:00 2024-05-03T18:08:27.325 INFO:teuthology.orchestra.run.smithi099.stdout:(77/152): python3-babel-2.9.1-2.el9.noarch.rpm 4.0 MB/s | 6.0 MB 00:01 2024-05-03T18:08:27.635 INFO:teuthology.orchestra.run.smithi099.stdout:(78/152): python3-jsonpatch-1.21-16.el9.noarch. 36 kB/s | 26 kB 00:00 2024-05-03T18:08:27.869 INFO:teuthology.orchestra.run.smithi099.stdout:(79/152): python3-jsonpointer-2.0-4.el9.noarch. 29 kB/s | 19 kB 00:00 2024-05-03T18:08:28.043 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-05-03T18:08:28.276 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-05-03T18:08:28.276 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-05-03T18:08:28.280 INFO:teuthology.orchestra.run.smithi099.stdout:(80/152): python3-mako-1.1.4-6.el9.noarch.rpm 181 kB/s | 172 kB 00:00 2024-05-03T18:08:28.311 INFO:teuthology.orchestra.run.smithi099.stdout:(81/152): python3-markupsafe-1.1.1-12.el9.x86_6 52 kB/s | 35 kB 00:00 2024-05-03T18:08:28.630 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction check 2024-05-03T18:08:28.880 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction check succeeded. 2024-05-03T18:08:28.880 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction test 2024-05-03T18:08:29.153 INFO:teuthology.orchestra.run.smithi099.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 287 kB/s | 250 kB 00:00 2024-05-03T18:08:29.186 INFO:teuthology.orchestra.run.smithi099.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 255 kB/s | 222 kB 00:00 2024-05-03T18:08:29.295 INFO:teuthology.orchestra.run.smithi099.stdout:(84/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 3.8 MB/s | 5.1 MB 00:01 2024-05-03T18:08:29.995 INFO:teuthology.orchestra.run.smithi099.stdout:(85/152): python3-packaging-20.9-5.el9.noarch.r 92 kB/s | 77 kB 00:00 2024-05-03T18:08:30.029 INFO:teuthology.orchestra.run.smithi099.stdout:(86/152): python3-prettytable-0.7.2-27.el9.noar 50 kB/s | 42 kB 00:00 2024-05-03T18:08:30.371 INFO:teuthology.orchestra.run.smithi099.stdout:(87/152): python3-protobuf-3.14.0-13.el9.noarch 251 kB/s | 269 kB 00:01 2024-05-03T18:08:30.838 INFO:teuthology.orchestra.run.smithi099.stdout:(88/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 189 kB/s | 159 kB 00:00 2024-05-03T18:08:30.917 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-05-03T18:08:30.918 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-05-03T18:08:31.038 INFO:teuthology.orchestra.run.smithi099.stdout:(89/152): python3-pyasn1-modules-0.4.8-6.el9.no 277 kB/s | 279 kB 00:01 2024-05-03T18:08:31.088 INFO:teuthology.orchestra.run.smithi099.stdout:(90/152): python3-pytz-2021.1-5.el9.noarch.rpm 71 kB/s | 51 kB 00:00 2024-05-03T18:08:31.553 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction test succeeded. 2024-05-03T18:08:31.553 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction 2024-05-03T18:08:31.705 INFO:teuthology.orchestra.run.smithi099.stdout:(91/152): python3-requests-oauthlib-1.3.0-12.el 62 kB/s | 54 kB 00:00 2024-05-03T18:08:31.805 INFO:teuthology.orchestra.run.smithi099.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 64 kB/s | 42 kB 00:00 2024-05-03T18:08:32.406 INFO:teuthology.orchestra.run.smithi099.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 63 kB/s | 44 kB 00:00 2024-05-03T18:08:32.873 INFO:teuthology.orchestra.run.smithi099.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 207 kB/s | 221 kB 00:01 2024-05-03T18:08:33.223 INFO:teuthology.orchestra.run.smithi099.stdout:(95/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 7.4 MB/s | 16 MB 00:02 2024-05-03T18:08:33.248 INFO:teuthology.orchestra.run.smithi099.stdout:(96/152): qatlib-service-24.02.0-1.el9.x86_64.r 43 kB/s | 36 kB 00:00 2024-05-03T18:08:33.831 INFO:teuthology.orchestra.run.smithi099.stdout:(97/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 50 kB/s | 47 kB 00:00 2024-05-03T18:08:34.018 INFO:teuthology.orchestra.run.smithi099.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 75 kB/s | 58 kB 00:00 2024-05-03T18:08:34.340 INFO:teuthology.orchestra.run.smithi099.stdout:(99/152): socat-1.7.4.1-5.el9.x86_64.rpm 273 kB/s | 305 kB 00:01 2024-05-03T18:08:34.708 INFO:teuthology.orchestra.run.smithi099.stdout:(100/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 107 kB/s | 94 kB 00:00 2024-05-03T18:08:34.740 INFO:teuthology.orchestra.run.smithi099.stdout:(101/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 88 kB/s | 64 kB 00:00 2024-05-03T18:08:34.966 INFO:teuthology.orchestra.run.smithi099.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 87 kB/s | 22 kB 00:00 2024-05-03T18:08:35.125 INFO:teuthology.orchestra.run.smithi099.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 2.2 MB/s | 863 kB 00:00 2024-05-03T18:08:35.150 INFO:teuthology.orchestra.run.smithi099.stdout:(104/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 55 kB/s | 45 kB 00:00 2024-05-03T18:08:35.325 INFO:teuthology.orchestra.run.smithi099.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 111 kB/s | 19 kB 00:00 2024-05-03T18:08:35.367 INFO:teuthology.orchestra.run.smithi099.stdout:(106/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.3 MB/s | 548 kB 00:00 2024-05-03T18:08:35.443 INFO:teuthology.orchestra.run.smithi099.stdout:(107/152): libarrow-doc-9.0.0-11.el9.noarch.rpm 350 kB/s | 26 kB 00:00 2024-05-03T18:08:35.476 INFO:teuthology.orchestra.run.smithi099.stdout:(108/152): gperftools-libs-2.9.1-2.el9.x86_64.r 887 kB/s | 309 kB 00:00 2024-05-03T18:08:35.518 INFO:teuthology.orchestra.run.smithi099.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 652 kB/s | 49 kB 00:00 2024-05-03T18:08:35.552 INFO:teuthology.orchestra.run.smithi099.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 903 kB/s | 67 kB 00:00 2024-05-03T18:08:35.594 INFO:teuthology.orchestra.run.smithi099.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 2.0 MB/s | 151 kB 00:00 2024-05-03T18:08:35.686 INFO:teuthology.orchestra.run.smithi099.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 5.9 MB/s | 548 kB 00:00 2024-05-03T18:08:35.736 INFO:teuthology.orchestra.run.smithi099.stdout:(113/152): parquet-libs-9.0.0-11.el9.x86_64.rpm 4.5 MB/s | 839 kB 00:00 2024-05-03T18:08:35.770 INFO:teuthology.orchestra.run.smithi099.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 524 kB/s | 43 kB 00:00 2024-05-03T18:08:35.803 INFO:teuthology.orchestra.run.smithi099.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 484 kB/s | 32 kB 00:00 2024-05-03T18:08:35.845 INFO:teuthology.orchestra.run.smithi099.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 201 kB/s | 15 kB 00:00 2024-05-03T18:08:35.897 INFO:teuthology.orchestra.run.smithi099.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.8 MB/s | 172 kB 00:00 2024-05-03T18:08:35.996 INFO:teuthology.orchestra.run.smithi099.stdout:(118/152): libarrow-9.0.0-11.el9.x86_64.rpm 6.6 MB/s | 4.4 MB 00:00 2024-05-03T18:08:36.030 INFO:teuthology.orchestra.run.smithi099.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 1.9 MB/s | 358 kB 00:00 2024-05-03T18:08:36.072 INFO:teuthology.orchestra.run.smithi099.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.7 MB/s | 303 kB 00:00 2024-05-03T18:08:36.097 INFO:teuthology.orchestra.run.smithi099.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 352 kB/s | 35 kB 00:00 2024-05-03T18:08:36.164 INFO:teuthology.orchestra.run.smithi099.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 1.5 MB/s | 206 kB 00:00 2024-05-03T18:08:36.289 INFO:teuthology.orchestra.run.smithi099.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 755 kB/s | 144 kB 00:00 2024-05-03T18:08:36.314 INFO:teuthology.orchestra.run.smithi099.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 71 kB/s | 11 kB 00:00 2024-05-03T18:08:36.415 INFO:teuthology.orchestra.run.smithi099.stdout:(125/152): python3-grpcio-1.46.7-10.el9.x86_64. 6.0 MB/s | 2.0 MB 00:00 2024-05-03T18:08:36.549 INFO:teuthology.orchestra.run.smithi099.stdout:(126/152): python3-jaraco-classes-3.2.1-5.el9.n 68 kB/s | 18 kB 00:00 2024-05-03T18:08:36.574 INFO:teuthology.orchestra.run.smithi099.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 90 kB/s | 23 kB 00:00 2024-05-03T18:08:36.599 INFO:teuthology.orchestra.run.smithi099.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 106 kB/s | 19 kB 00:00 2024-05-03T18:08:36.641 INFO:teuthology.orchestra.run.smithi099.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 216 kB/s | 20 kB 00:00 2024-05-03T18:08:36.683 INFO:teuthology.orchestra.run.smithi099.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 560 kB/s | 46 kB 00:00 2024-05-03T18:08:36.717 INFO:teuthology.orchestra.run.smithi099.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 1.0 MB/s | 79 kB 00:00 2024-05-03T18:08:36.767 INFO:teuthology.orchestra.run.smithi099.stdout:(132/152): python3-kubernetes-26.1.0-2.el9.noar 6.1 MB/s | 1.0 MB 00:00 2024-05-03T18:08:36.792 INFO:teuthology.orchestra.run.smithi099.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 534 kB/s | 58 kB 00:00 2024-05-03T18:08:36.826 INFO:teuthology.orchestra.run.smithi099.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.5 MB/s | 272 kB 00:00 2024-05-03T18:08:36.885 INFO:teuthology.orchestra.run.smithi099.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 141 kB/s | 16 kB 00:00 2024-05-03T18:08:36.910 INFO:teuthology.orchestra.run.smithi099.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 341 kB/s | 28 kB 00:00 2024-05-03T18:08:36.935 INFO:teuthology.orchestra.run.smithi099.stdout:(137/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 633 kB/s | 90 kB 00:00 2024-05-03T18:08:36.961 INFO:teuthology.orchestra.run.smithi099.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 413 kB/s | 31 kB 00:00 2024-05-03T18:08:36.994 INFO:teuthology.orchestra.run.smithi099.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 2.2 MB/s | 188 kB 00:00 2024-05-03T18:08:37.020 INFO:teuthology.orchestra.run.smithi099.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 711 kB/s | 59 kB 00:00 2024-05-03T18:08:37.045 INFO:teuthology.orchestra.run.smithi099.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 430 kB/s | 36 kB 00:00 2024-05-03T18:08:37.070 INFO:teuthology.orchestra.run.smithi099.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 683 kB/s | 51 kB 00:00 2024-05-03T18:08:37.208 INFO:teuthology.orchestra.run.smithi099.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 1.2 MB/s | 230 kB 00:00 2024-05-03T18:08:37.254 INFO:teuthology.orchestra.run.smithi099.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 430 kB/s | 90 kB 00:00 2024-05-03T18:08:37.280 INFO:teuthology.orchestra.run.smithi099.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 284 kB/s | 20 kB 00:00 2024-05-03T18:08:37.332 INFO:teuthology.orchestra.run.smithi099.stdout:(146/152): python3-werkzeug-2.0.3-3.el9.1.noarc 1.6 MB/s | 427 kB 00:00 2024-05-03T18:08:37.348 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-05-03T18:08:37.363 INFO:teuthology.orchestra.run.smithi099.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 1.7 MB/s | 192 kB 00:00 2024-05-03T18:08:37.447 INFO:teuthology.orchestra.run.smithi099.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 9.5 MB/s | 1.6 MB 00:00 2024-05-03T18:08:37.449 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-05-03T18:08:37.561 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-05-03T18:08:37.580 INFO:teuthology.orchestra.run.smithi099.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 10 MB/s | 1.3 MB 00:00 2024-05-03T18:08:37.670 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-05-03T18:08:37.726 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-05-03T18:08:37.756 INFO:teuthology.orchestra.run.smithi099.stdout:(150/152): librbd1-19.0.0-3450.g9a047fe7.el9.x8 8.0 MB/s | 3.1 MB 00:00 2024-05-03T18:08:37.758 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-05-03T18:08:37.856 INFO:teuthology.orchestra.run.smithi099.stdout:(151/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 8.7 MB/s | 2.4 MB 00:00 2024-05-03T18:08:37.913 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-05-03T18:08:38.326 INFO:teuthology.orchestra.run.smithi033.stdout: Preparing : 1/1 2024-05-03T18:08:38.368 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-05-03T18:08:38.435 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-05-03T18:08:38.481 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : librados2-2:19.0.0-3450.g9a047fe7.el9.x86_64 8/156 2024-05-03T18:08:38.509 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-05-03T18:08:38.524 INFO:teuthology.orchestra.run.smithi099.stdout:(152/152): librados2-19.0.0-3450.g9a047fe7.el9. 2.9 MB/s | 3.4 MB 00:01 2024-05-03T18:08:38.535 INFO:teuthology.orchestra.run.smithi099.stdout:-------------------------------------------------------------------------------- 2024-05-03T18:08:38.535 INFO:teuthology.orchestra.run.smithi099.stdout:Total 6.6 MB/s | 222 MB 00:33 2024-05-03T18:08:38.609 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-05-03T18:08:38.660 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-05-03T18:08:38.681 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librados2-2:19.0.0-3450.g9a047fe7.el9.x86_64 8/156 2024-05-03T18:08:38.694 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-05-03T18:08:38.716 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libcephfs2-2:19.0.0-3450.g9a047fe7.el9.x86_64 9/156 2024-05-03T18:08:38.809 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-05-03T18:08:38.842 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libcephfs2-2:19.0.0-3450.g9a047fe7.el9.x86_64 9/156 2024-05-03T18:08:38.890 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-rados-2:19.0.0-3450.g9a047fe7.el9.x86_64 10/156 2024-05-03T18:08:39.215 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-05-03T18:08:39.260 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-05-03T18:08:39.270 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-05-03T18:08:39.328 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libcephsqlite-2:19.0.0-3450.g9a047fe7.el9.x86_64 13/156 2024-05-03T18:08:39.384 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : librados2-2:19.0.0-3450.g9a047fe7.el9.x86_64 8/156 2024-05-03T18:08:39.464 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3450.g9a047fe7.el9.x86_64 13/156 2024-05-03T18:08:39.496 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libradosstriper1-2:19.0.0-3450.g9a047fe7.el9.x86 14/156 2024-05-03T18:08:39.581 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: librados2-2:19.0.0-3450.g9a047fe7.el9.x86_64 8/156 2024-05-03T18:08:39.582 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3450.g9a047fe7.el9.x86 14/156 2024-05-03T18:08:39.614 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-05-03T18:08:39.618 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libcephfs2-2:19.0.0-3450.g9a047fe7.el9.x86_64 9/156 2024-05-03T18:08:39.702 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-05-03T18:08:39.785 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: libcephfs2-2:19.0.0-3450.g9a047fe7.el9.x86_64 9/156 2024-05-03T18:08:39.828 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-rados-2:19.0.0-3450.g9a047fe7.el9.x86_64 10/156 2024-05-03T18:08:39.990 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-05-03T18:08:40.048 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-05-03T18:08:40.211 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-05-03T18:08:40.247 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-05-03T18:08:40.275 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-05-03T18:08:40.306 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libcephsqlite-2:19.0.0-3450.g9a047fe7.el9.x86_64 13/156 2024-05-03T18:08:40.374 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 18/156 2024-05-03T18:08:40.423 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-05-03T18:08:40.471 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-05-03T18:08:40.506 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3450.g9a047fe7.el9.x86_64 13/156 2024-05-03T18:08:40.532 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libradosstriper1-2:19.0.0-3450.g9a047fe7.el9.x86 14/156 2024-05-03T18:08:40.575 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-05-03T18:08:40.626 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-05-03T18:08:40.661 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3450.g9a047fe7.el9.x86 14/156 2024-05-03T18:08:40.689 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-05-03T18:08:40.691 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-05-03T18:08:40.756 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-05-03T18:08:40.775 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-05-03T18:08:40.797 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-05-03T18:08:40.916 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-05-03T18:08:40.987 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-05-03T18:08:41.039 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-05-03T18:08:41.076 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-05-03T18:08:41.123 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-05-03T18:08:41.134 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-05-03T18:08:41.240 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-05-03T18:08:41.276 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-ceph-common-2:19.0.0-3450.g9a047fe7.el9. 31/156 2024-05-03T18:08:41.336 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-05-03T18:08:41.340 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-ceph-argparse-2:19.0.0-3450.g9a047fe7.el 32/156 2024-05-03T18:08:41.378 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cephfs-2:19.0.0-3450.g9a047fe7.el9.x86_6 33/156 2024-05-03T18:08:41.397 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction check 2024-05-03T18:08:41.477 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 18/156 2024-05-03T18:08:41.515 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-05-03T18:08:41.517 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-05-03T18:08:41.566 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-05-03T18:08:41.578 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-05-03T18:08:41.630 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction check succeeded. 2024-05-03T18:08:41.630 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction test 2024-05-03T18:08:41.673 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-05-03T18:08:41.678 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-05-03T18:08:41.721 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-05-03T18:08:41.793 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-05-03T18:08:41.842 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-05-03T18:08:41.854 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-05-03T18:08:41.881 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-05-03T18:08:41.920 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-05-03T18:08:42.004 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-05-03T18:08:42.075 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-05-03T18:08:42.109 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-05-03T18:08:42.165 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-05-03T18:08:42.277 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-05-03T18:08:42.313 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-ceph-common-2:19.0.0-3450.g9a047fe7.el9. 31/156 2024-05-03T18:08:42.359 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-ceph-argparse-2:19.0.0-3450.g9a047fe7.el 32/156 2024-05-03T18:08:42.427 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cephfs-2:19.0.0-3450.g9a047fe7.el9.x86_6 33/156 2024-05-03T18:08:42.549 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-05-03T18:08:42.552 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-05-03T18:08:42.616 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-05-03T18:08:42.653 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-05-03T18:08:42.698 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-05-03T18:08:42.726 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-05-03T18:08:42.742 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-05-03T18:08:42.783 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : librados-devel-2:19.0.0-3450.g9a047fe7.el9.x86_6 43/156 2024-05-03T18:08:42.879 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-05-03T18:08:42.904 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-05-03T18:08:42.967 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-05-03T18:08:43.132 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-05-03T18:08:43.221 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-05-03T18:08:43.303 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-05-03T18:08:43.397 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-05-03T18:08:43.434 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-05-03T18:08:43.472 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-05-03T18:08:43.540 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-05-03T18:08:43.606 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-05-03T18:08:43.668 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-05-03T18:08:43.683 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-05-03T18:08:43.727 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-05-03T18:08:43.785 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-05-03T18:08:43.825 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-05-03T18:08:43.878 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-05-03T18:08:43.919 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : librados-devel-2:19.0.0-3450.g9a047fe7.el9.x86_6 43/156 2024-05-03T18:08:43.924 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-05-03T18:08:43.979 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-05-03T18:08:43.980 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-05-03T18:08:44.016 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-05-03T18:08:44.074 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-05-03T18:08:44.210 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-05-03T18:08:44.225 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction test succeeded. 2024-05-03T18:08:44.225 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction 2024-05-03T18:08:44.228 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-05-03T18:08:44.252 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-05-03T18:08:44.286 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-05-03T18:08:44.321 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-05-03T18:08:44.356 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-05-03T18:08:44.360 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-05-03T18:08:44.389 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 63/156 2024-05-03T18:08:44.458 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-05-03T18:08:44.494 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-05-03T18:08:44.533 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-05-03T18:08:44.577 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-05-03T18:08:44.633 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-05-03T18:08:44.685 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-05-03T18:08:44.751 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-05-03T18:08:44.780 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-05-03T18:08:44.815 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-05-03T18:08:44.821 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-05-03T18:08:44.863 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-05-03T18:08:44.912 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-05-03T18:08:44.974 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-05-03T18:08:45.026 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-05-03T18:08:45.051 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-05-03T18:08:45.081 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-05-03T18:08:45.109 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-05-03T18:08:45.118 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-05-03T18:08:45.155 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-05-03T18:08:45.177 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-05-03T18:08:45.205 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-05-03T18:08:45.277 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-05-03T18:08:45.321 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-05-03T18:08:45.327 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-05-03T18:08:45.369 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-05-03T18:08:45.419 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-05-03T18:08:45.459 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-05-03T18:08:45.492 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 63/156 2024-05-03T18:08:45.497 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-05-03T18:08:45.563 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-05-03T18:08:45.609 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-05-03T18:08:45.679 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-05-03T18:08:45.701 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-05-03T18:08:45.862 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-05-03T18:08:45.883 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-05-03T18:08:45.930 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-05-03T18:08:45.931 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-05-03T18:08:45.971 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-05-03T18:08:46.026 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-05-03T18:08:46.105 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-05-03T18:08:46.164 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-05-03T18:08:46.212 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-05-03T18:08:46.265 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-05-03T18:08:46.316 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-05-03T18:08:46.403 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-05-03T18:08:46.447 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-05-03T18:08:46.594 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-05-03T18:08:46.650 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-05-03T18:08:46.695 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-05-03T18:08:46.788 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-05-03T18:08:46.898 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-05-03T18:08:46.946 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-05-03T18:08:46.953 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-05-03T18:08:46.979 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-05-03T18:08:47.016 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-05-03T18:08:47.101 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-05-03T18:08:47.183 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-05-03T18:08:47.286 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-05-03T18:08:47.323 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-05-03T18:08:47.554 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-05-03T18:08:47.586 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-05-03T18:08:47.701 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-05-03T18:08:48.004 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-05-03T18:08:48.065 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-05-03T18:08:48.099 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-05-03T18:08:48.228 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-05-03T18:08:48.294 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-05-03T18:08:48.405 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-05-03T18:08:48.450 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-05-03T18:08:48.531 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-05-03T18:08:48.674 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-05-03T18:08:48.695 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-05-03T18:08:48.730 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-05-03T18:08:48.840 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-05-03T18:08:49.712 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-05-03T18:08:49.864 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-05-03T18:08:50.288 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-05-03T18:08:50.393 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-05-03T18:08:50.439 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-05-03T18:08:50.488 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-05-03T18:08:50.564 INFO:teuthology.orchestra.run.smithi099.stdout: Preparing : 1/1 2024-05-03T18:08:50.681 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-05-03T18:08:50.735 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-05-03T18:08:50.825 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-05-03T18:08:50.879 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-05-03T18:08:50.880 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-05-03T18:08:50.914 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-05-03T18:08:50.930 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : librbd1-2:19.0.0-3450.g9a047fe7.el9.x86_64 99/156 2024-05-03T18:08:51.054 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-05-03T18:08:51.069 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librbd1-2:19.0.0-3450.g9a047fe7.el9.x86_64 99/156 2024-05-03T18:08:51.104 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-rbd-2:19.0.0-3450.g9a047fe7.el9.x86_64 100/156 2024-05-03T18:08:51.160 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-05-03T18:08:51.494 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-05-03T18:08:51.521 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-05-03T18:08:51.619 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-05-03T18:08:51.629 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : librados2-2:19.0.0-3450.g9a047fe7.el9.x86_64 8/156 2024-05-03T18:08:51.671 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-05-03T18:08:51.704 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/156 2024-05-03T18:08:51.715 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-05-03T18:08:51.741 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : librgw2-2:19.0.0-3450.g9a047fe7.el9.x86_64 103/156 2024-05-03T18:08:51.804 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librados2-2:19.0.0-3450.g9a047fe7.el9.x86_64 8/156 2024-05-03T18:08:51.816 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librgw2-2:19.0.0-3450.g9a047fe7.el9.x86_64 103/156 2024-05-03T18:08:51.839 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libcephfs2-2:19.0.0-3450.g9a047fe7.el9.x86_64 9/156 2024-05-03T18:08:51.843 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-rgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 104/156 2024-05-03T18:08:51.965 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libcephfs2-2:19.0.0-3450.g9a047fe7.el9.x86_64 9/156 2024-05-03T18:08:51.996 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-rados-2:19.0.0-3450.g9a047fe7.el9.x86_64 10/156 2024-05-03T18:08:52.120 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-05-03T18:08:52.182 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : librbd1-2:19.0.0-3450.g9a047fe7.el9.x86_64 99/156 2024-05-03T18:08:52.313 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-05-03T18:08:52.316 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: librbd1-2:19.0.0-3450.g9a047fe7.el9.x86_64 99/156 2024-05-03T18:08:52.347 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-rbd-2:19.0.0-3450.g9a047fe7.el9.x86_64 100/156 2024-05-03T18:08:52.377 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-05-03T18:08:52.402 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libcephsqlite-2:19.0.0-3450.g9a047fe7.el9.x86_64 13/156 2024-05-03T18:08:52.407 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-05-03T18:08:52.550 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3450.g9a047fe7.el9.x86_64 13/156 2024-05-03T18:08:52.595 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libradosstriper1-2:19.0.0-3450.g9a047fe7.el9.x86 14/156 2024-05-03T18:08:52.665 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3450.g9a047fe7.el9.x86 14/156 2024-05-03T18:08:52.696 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-05-03T18:08:52.762 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-05-03T18:08:52.999 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/156 2024-05-03T18:08:53.043 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : librgw2-2:19.0.0-3450.g9a047fe7.el9.x86_64 103/156 2024-05-03T18:08:53.090 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-05-03T18:08:53.128 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: librgw2-2:19.0.0-3450.g9a047fe7.el9.x86_64 103/156 2024-05-03T18:08:53.153 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-rgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 104/156 2024-05-03T18:08:53.155 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-05-03T18:08:53.354 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-05-03T18:08:53.481 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 18/156 2024-05-03T18:08:53.547 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-05-03T18:08:53.592 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-05-03T18:08:53.692 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-05-03T18:08:53.743 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-05-03T18:08:53.798 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-05-03T18:08:53.845 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-05-03T18:08:53.878 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-05-03T18:08:53.992 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-05-03T18:08:54.065 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-05-03T18:08:54.114 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-05-03T18:08:54.173 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-05-03T18:08:54.280 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-05-03T18:08:54.317 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-ceph-common-2:19.0.0-3450.g9a047fe7.el9. 31/156 2024-05-03T18:08:54.370 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-ceph-argparse-2:19.0.0-3450.g9a047fe7.el 32/156 2024-05-03T18:08:54.408 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cephfs-2:19.0.0-3450.g9a047fe7.el9.x86_6 33/156 2024-05-03T18:08:54.505 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-common-2:19.0.0-3450.g9a047fe7.el9.x86_64 105/156 2024-05-03T18:08:54.527 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-05-03T18:08:54.559 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-common-2:19.0.0-3450.g9a047fe7.el9.x86_64 105/156 2024-05-03T18:08:54.585 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-05-03T18:08:54.680 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-05-03T18:08:54.863 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-05-03T18:08:54.931 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-05-03T18:08:55.227 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-common-2:19.0.0-3450.g9a047fe7.el9.x86_64 105/156 2024-05-03T18:08:55.293 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-base-2:19.0.0-3450.g9a047fe7.el9.x86_64 106/156 2024-05-03T18:08:55.471 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-base-2:19.0.0-3450.g9a047fe7.el9.x86_64 106/156 2024-05-03T18:08:55.471 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-03T18:08:55.471 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-03T18:08:55.471 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-03T18:08:55.511 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-selinux-2:19.0.0-3450.g9a047fe7.el9.x86_64 107/156 2024-05-03T18:08:55.606 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-05-03T18:08:55.711 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-05-03T18:08:55.756 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-05-03T18:08:55.803 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-05-03T18:08:55.844 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : librados-devel-2:19.0.0-3450.g9a047fe7.el9.x86_6 43/156 2024-05-03T18:08:55.949 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-common-2:19.0.0-3450.g9a047fe7.el9.x86_64 105/156 2024-05-03T18:08:55.956 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-05-03T18:08:55.991 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-common-2:19.0.0-3450.g9a047fe7.el9.x86_64 105/156 2024-05-03T18:08:56.195 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-05-03T18:08:56.251 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-05-03T18:08:56.323 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-05-03T18:08:56.411 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-05-03T18:08:56.459 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-05-03T18:08:56.497 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-05-03T18:08:56.564 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-05-03T18:08:56.673 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-05-03T18:08:56.693 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-common-2:19.0.0-3450.g9a047fe7.el9.x86_64 105/156 2024-05-03T18:08:56.742 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-05-03T18:08:56.753 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-base-2:19.0.0-3450.g9a047fe7.el9.x86_64 106/156 2024-05-03T18:08:56.808 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-05-03T18:08:56.911 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-base-2:19.0.0-3450.g9a047fe7.el9.x86_64 106/156 2024-05-03T18:08:56.911 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-03T18:08:56.911 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-03T18:08:56.911 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:08:56.953 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-selinux-2:19.0.0-3450.g9a047fe7.el9.x86_64 107/156 2024-05-03T18:08:57.045 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-05-03T18:08:57.105 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-05-03T18:08:57.142 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-05-03T18:08:57.198 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-05-03T18:08:57.327 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-05-03T18:08:57.375 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-05-03T18:08:57.422 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-05-03T18:08:57.457 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-05-03T18:08:57.490 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 63/156 2024-05-03T18:08:57.697 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-05-03T18:08:57.888 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-05-03T18:08:57.939 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-05-03T18:08:57.981 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-05-03T18:08:58.025 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-05-03T18:08:58.091 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-05-03T18:08:58.154 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-05-03T18:08:58.202 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-05-03T18:08:58.249 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-05-03T18:08:58.322 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-05-03T18:08:58.400 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-05-03T18:08:58.464 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-05-03T18:08:58.629 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-05-03T18:08:58.698 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-05-03T18:08:58.769 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-05-03T18:08:58.844 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-05-03T18:08:58.997 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-05-03T18:08:59.065 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-05-03T18:09:00.037 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-05-03T18:09:00.091 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-05-03T18:09:00.124 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-05-03T18:09:00.277 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-05-03T18:09:00.335 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-05-03T18:09:00.439 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-05-03T18:09:00.484 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-05-03T18:09:00.724 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-05-03T18:09:00.973 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-05-03T18:09:01.135 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-05-03T18:09:02.061 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-05-03T18:09:02.195 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-05-03T18:09:03.758 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-05-03T18:09:03.856 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-05-03T18:09:03.904 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-05-03T18:09:03.958 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-05-03T18:09:04.367 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-05-03T18:09:04.418 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : librbd1-2:19.0.0-3450.g9a047fe7.el9.x86_64 99/156 2024-05-03T18:09:04.532 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librbd1-2:19.0.0-3450.g9a047fe7.el9.x86_64 99/156 2024-05-03T18:09:04.566 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-rbd-2:19.0.0-3450.g9a047fe7.el9.x86_64 100/156 2024-05-03T18:09:04.623 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-05-03T18:09:05.166 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/156 2024-05-03T18:09:05.196 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : librgw2-2:19.0.0-3450.g9a047fe7.el9.x86_64 103/156 2024-05-03T18:09:05.279 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librgw2-2:19.0.0-3450.g9a047fe7.el9.x86_64 103/156 2024-05-03T18:09:05.315 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-rgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 104/156 2024-05-03T18:09:07.990 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-common-2:19.0.0-3450.g9a047fe7.el9.x86_64 105/156 2024-05-03T18:09:08.028 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-common-2:19.0.0-3450.g9a047fe7.el9.x86_64 105/156 2024-05-03T18:09:08.518 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3450.g9a047fe7.el9.x86_64 107/156 2024-05-03T18:09:08.519 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /sys 2024-05-03T18:09:08.519 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /proc 2024-05-03T18:09:08.519 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /mnt 2024-05-03T18:09:08.519 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /var/tmp 2024-05-03T18:09:08.519 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /home 2024-05-03T18:09:08.519 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /root 2024-05-03T18:09:08.519 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /tmp 2024-05-03T18:09:08.519 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-03T18:09:08.679 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-common-2:19.0.0-3450.g9a047fe7.el9.x86_64 105/156 2024-05-03T18:09:08.757 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-base-2:19.0.0-3450.g9a047fe7.el9.x86_64 106/156 2024-05-03T18:09:08.775 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-volume-2:19.0.0-3450.g9a047fe7.el9.noarch 108/156 2024-05-03T18:09:08.824 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-volume-2:19.0.0-3450.g9a047fe7.el9.noarch 108/156 2024-05-03T18:09:08.824 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T18:09:08.824 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-03T18:09:08.825 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-03T18:09:08.921 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-base-2:19.0.0-3450.g9a047fe7.el9.x86_64 106/156 2024-05-03T18:09:08.921 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-03T18:09:08.921 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-03T18:09:08.921 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-03T18:09:08.958 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-selinux-2:19.0.0-3450.g9a047fe7.el9.x86_64 107/156 2024-05-03T18:09:10.479 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-osd-2:19.0.0-3450.g9a047fe7.el9.x86_64 109/156 2024-05-03T18:09:10.557 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-osd-2:19.0.0-3450.g9a047fe7.el9.x86_64 109/156 2024-05-03T18:09:10.557 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T18:09:10.557 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-03T18:09:10.557 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-03T18:09:10.557 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-03T18:09:10.557 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-03T18:09:10.682 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3450.g9a047fe7.el9.x86_64 107/156 2024-05-03T18:09:10.682 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /sys 2024-05-03T18:09:10.683 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /proc 2024-05-03T18:09:10.683 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /mnt 2024-05-03T18:09:10.683 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /var/tmp 2024-05-03T18:09:10.683 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /home 2024-05-03T18:09:10.683 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /root 2024-05-03T18:09:10.683 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /tmp 2024-05-03T18:09:10.683 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:09:10.795 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mds-2:19.0.0-3450.g9a047fe7.el9.x86_64 110/156 2024-05-03T18:09:10.860 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mds-2:19.0.0-3450.g9a047fe7.el9.x86_64 110/156 2024-05-03T18:09:10.860 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T18:09:10.860 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-03T18:09:10.860 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-03T18:09:10.860 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-03T18:09:10.860 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-03T18:09:10.925 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-volume-2:19.0.0-3450.g9a047fe7.el9.noarch 108/156 2024-05-03T18:09:10.978 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-volume-2:19.0.0-3450.g9a047fe7.el9.noarch 108/156 2024-05-03T18:09:10.978 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T18:09:10.978 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-03T18:09:10.978 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:09:11.363 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mon-2:19.0.0-3450.g9a047fe7.el9.x86_64 111/156 2024-05-03T18:09:11.430 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mon-2:19.0.0-3450.g9a047fe7.el9.x86_64 111/156 2024-05-03T18:09:11.430 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T18:09:11.431 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-03T18:09:11.431 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-03T18:09:11.431 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-03T18:09:11.431 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-03T18:09:11.563 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-05-03T18:09:11.623 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-05-03T18:09:11.691 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-05-03T18:09:11.930 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-05-03T18:09:11.992 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-05-03T18:09:12.077 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-05-03T18:09:12.186 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-05-03T18:09:12.360 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-05-03T18:09:12.450 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-05-03T18:09:12.500 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-05-03T18:09:12.541 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-05-03T18:09:12.679 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-osd-2:19.0.0-3450.g9a047fe7.el9.x86_64 109/156 2024-05-03T18:09:12.772 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-osd-2:19.0.0-3450.g9a047fe7.el9.x86_64 109/156 2024-05-03T18:09:12.772 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T18:09:12.772 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-03T18:09:12.772 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-03T18:09:12.772 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-03T18:09:12.772 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:09:13.037 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mds-2:19.0.0-3450.g9a047fe7.el9.x86_64 110/156 2024-05-03T18:09:13.109 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mds-2:19.0.0-3450.g9a047fe7.el9.x86_64 110/156 2024-05-03T18:09:13.110 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T18:09:13.110 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-03T18:09:13.110 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-03T18:09:13.110 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-03T18:09:13.110 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:09:13.126 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: cephadm-2:19.0.0-3450.g9a047fe7.el9.noarch 123/156 2024-05-03T18:09:13.178 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : cephadm-2:19.0.0-3450.g9a047fe7.el9.noarch 123/156 2024-05-03T18:09:13.234 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3450.g9a047fe7.e 124/156 2024-05-03T18:09:13.343 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3450.g9a047fe7. 125/156 2024-05-03T18:09:13.370 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3450.g9a047fe7.el9.noa 126/156 2024-05-03T18:09:13.621 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mon-2:19.0.0-3450.g9a047fe7.el9.x86_64 111/156 2024-05-03T18:09:13.691 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mon-2:19.0.0-3450.g9a047fe7.el9.x86_64 111/156 2024-05-03T18:09:13.691 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T18:09:13.691 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-03T18:09:13.691 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-03T18:09:13.691 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-03T18:09:13.691 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:09:13.831 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-05-03T18:09:13.929 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-05-03T18:09:13.970 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-05-03T18:09:14.226 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-05-03T18:09:14.293 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-05-03T18:09:14.399 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-05-03T18:09:14.515 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-05-03T18:09:14.671 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3450.g9a047fe7.el9.noa 126/156 2024-05-03T18:09:14.688 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-05-03T18:09:14.705 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3450.g9a047fe7.el9.n 127/156 2024-05-03T18:09:14.750 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-05-03T18:09:14.792 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-05-03T18:09:14.832 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-05-03T18:09:15.449 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: cephadm-2:19.0.0-3450.g9a047fe7.el9.noarch 123/156 2024-05-03T18:09:15.497 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : cephadm-2:19.0.0-3450.g9a047fe7.el9.noarch 123/156 2024-05-03T18:09:15.554 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3450.g9a047fe7.e 124/156 2024-05-03T18:09:15.677 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3450.g9a047fe7. 125/156 2024-05-03T18:09:15.714 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3450.g9a047fe7.el9.noa 126/156 2024-05-03T18:09:15.808 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3450.g9a047fe7.el9.n 127/156 2024-05-03T18:09:15.831 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3450.g9a0 128/156 2024-05-03T18:09:15.987 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3450.g9a0 128/156 2024-05-03T18:09:16.170 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3450.g9a047fe7.el 129/156 2024-05-03T18:09:16.208 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-2:19.0.0-3450.g9a047fe7.el9.x86_64 130/156 2024-05-03T18:09:16.286 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3450.g9a047fe7.el9.x86_64 130/156 2024-05-03T18:09:16.286 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T18:09:16.287 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-03T18:09:16.287 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-03T18:09:16.287 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-03T18:09:16.287 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-03T18:09:16.358 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-rook-2:19.0.0-3450.g9a047fe7.el9.noarch 131/156 2024-05-03T18:09:16.396 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3450.g9a047fe7.el9.noarch 131/156 2024-05-03T18:09:16.538 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-2:19.0.0-3450.g9a047fe7.el9.x86_64 132/156 2024-05-03T18:09:16.611 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-fuse-2:19.0.0-3450.g9a047fe7.el9.x86_64 133/156 2024-05-03T18:09:17.082 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3450.g9a047fe7.el9.noa 126/156 2024-05-03T18:09:17.124 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3450.g9a047fe7.el9.n 127/156 2024-05-03T18:09:17.623 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-05-03T18:09:17.653 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-radosgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 135/156 2024-05-03T18:09:17.725 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 135/156 2024-05-03T18:09:17.725 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T18:09:17.725 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-03T18:09:17.725 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-03T18:09:17.726 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-03T18:09:17.726 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-03T18:09:17.786 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3450.g9a047 136/156 2024-05-03T18:09:17.878 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3450.g9a047 136/156 2024-05-03T18:09:17.878 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T18:09:17.878 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-03T18:09:17.878 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-03T18:09:17.954 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : cephfs-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 137/156 2024-05-03T18:09:18.054 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 137/156 2024-05-03T18:09:18.054 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T18:09:18.054 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-03T18:09:18.055 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-03T18:09:18.055 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-03T18:09:18.055 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-03T18:09:18.292 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3450.g9a047fe7.el9.n 127/156 2024-05-03T18:09:18.324 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3450.g9a0 128/156 2024-05-03T18:09:18.464 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : rbd-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 138/156 2024-05-03T18:09:18.490 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3450.g9a0 128/156 2024-05-03T18:09:18.553 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 138/156 2024-05-03T18:09:18.553 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T18:09:18.553 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-03T18:09:18.553 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-03T18:09:18.553 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-03T18:09:18.554 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-03T18:09:18.671 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3450.g9a047fe7.el 129/156 2024-05-03T18:09:18.701 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-2:19.0.0-3450.g9a047fe7.el9.x86_64 130/156 2024-05-03T18:09:18.776 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3450.g9a047fe7.el9.x86_64 130/156 2024-05-03T18:09:18.776 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T18:09:18.776 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-03T18:09:18.776 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-03T18:09:18.776 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-03T18:09:18.776 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:09:18.836 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-rook-2:19.0.0-3450.g9a047fe7.el9.noarch 131/156 2024-05-03T18:09:18.875 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3450.g9a047fe7.el9.noarch 131/156 2024-05-03T18:09:18.990 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-2:19.0.0-3450.g9a047fe7.el9.x86_64 132/156 2024-05-03T18:09:19.068 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-fuse-2:19.0.0-3450.g9a047fe7.el9.x86_64 133/156 2024-05-03T18:09:20.137 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-05-03T18:09:20.174 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-radosgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 135/156 2024-05-03T18:09:20.247 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 135/156 2024-05-03T18:09:20.247 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T18:09:20.247 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-03T18:09:20.247 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-03T18:09:20.247 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-03T18:09:20.247 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:09:20.297 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3450.g9a047 136/156 2024-05-03T18:09:20.379 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3450.g9a047 136/156 2024-05-03T18:09:20.379 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T18:09:20.379 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-03T18:09:20.379 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:09:20.465 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : cephfs-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 137/156 2024-05-03T18:09:20.574 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 137/156 2024-05-03T18:09:20.575 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T18:09:20.575 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-03T18:09:20.575 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-03T18:09:20.575 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-03T18:09:20.575 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:09:20.974 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : rbd-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 138/156 2024-05-03T18:09:21.057 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 138/156 2024-05-03T18:09:21.057 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T18:09:21.058 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-03T18:09:21.058 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-03T18:09:21.058 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-03T18:09:21.058 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:09:22.116 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3450.g9a047fe7.el9.x86_64 107/156 2024-05-03T18:09:22.116 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /sys 2024-05-03T18:09:22.116 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /proc 2024-05-03T18:09:22.116 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /mnt 2024-05-03T18:09:22.116 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /var/tmp 2024-05-03T18:09:22.116 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /home 2024-05-03T18:09:22.116 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /root 2024-05-03T18:09:22.117 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /tmp 2024-05-03T18:09:22.117 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-03T18:09:22.380 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-volume-2:19.0.0-3450.g9a047fe7.el9.noarch 108/156 2024-05-03T18:09:22.426 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-volume-2:19.0.0-3450.g9a047fe7.el9.noarch 108/156 2024-05-03T18:09:22.426 INFO:teuthology.orchestra.run.smithi099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T18:09:22.427 INFO:teuthology.orchestra.run.smithi099.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-03T18:09:22.427 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-03T18:09:23.662 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-test-2:19.0.0-3450.g9a047fe7.el9.x86_64 139/156 2024-05-03T18:09:23.718 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : rbd-fuse-2:19.0.0-3450.g9a047fe7.el9.x86_64 140/156 2024-05-03T18:09:23.856 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : rbd-nbd-2:19.0.0-3450.g9a047fe7.el9.x86_64 141/156 2024-05-03T18:09:23.901 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-05-03T18:09:23.948 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-05-03T18:09:24.044 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-osd-2:19.0.0-3450.g9a047fe7.el9.x86_64 109/156 2024-05-03T18:09:24.127 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-osd-2:19.0.0-3450.g9a047fe7.el9.x86_64 109/156 2024-05-03T18:09:24.127 INFO:teuthology.orchestra.run.smithi099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T18:09:24.127 INFO:teuthology.orchestra.run.smithi099.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-03T18:09:24.127 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-03T18:09:24.127 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-03T18:09:24.127 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-03T18:09:24.268 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : cephfs-shell-2:19.0.0-3450.g9a047fe7.el9.x86_64 144/156 2024-05-03T18:09:24.334 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-05-03T18:09:24.362 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libcephfs-devel-2:19.0.0-3450.g9a047fe7.el9.x86_ 146/156 2024-05-03T18:09:24.377 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mds-2:19.0.0-3450.g9a047fe7.el9.x86_64 110/156 2024-05-03T18:09:24.445 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mds-2:19.0.0-3450.g9a047fe7.el9.x86_64 110/156 2024-05-03T18:09:24.445 INFO:teuthology.orchestra.run.smithi099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T18:09:24.445 INFO:teuthology.orchestra.run.smithi099.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-03T18:09:24.446 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-03T18:09:24.446 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-03T18:09:24.446 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-03T18:09:24.928 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mon-2:19.0.0-3450.g9a047fe7.el9.x86_64 111/156 2024-05-03T18:09:24.996 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mon-2:19.0.0-3450.g9a047fe7.el9.x86_64 111/156 2024-05-03T18:09:24.996 INFO:teuthology.orchestra.run.smithi099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T18:09:24.996 INFO:teuthology.orchestra.run.smithi099.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-03T18:09:24.996 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-03T18:09:24.996 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-03T18:09:24.996 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-03T18:09:25.128 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-05-03T18:09:25.190 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-05-03T18:09:25.297 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-05-03T18:09:25.303 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-05-03T18:09:25.352 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-05-03T18:09:25.411 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-05-03T18:09:25.534 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-05-03T18:09:25.591 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-05-03T18:09:25.676 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-05-03T18:09:25.809 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-05-03T18:09:25.980 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-05-03T18:09:26.026 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-05-03T18:09:26.066 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-05-03T18:09:26.108 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-05-03T18:09:26.416 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-test-2:19.0.0-3450.g9a047fe7.el9.x86_64 139/156 2024-05-03T18:09:26.475 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : rbd-fuse-2:19.0.0-3450.g9a047fe7.el9.x86_64 140/156 2024-05-03T18:09:26.554 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : rbd-nbd-2:19.0.0-3450.g9a047fe7.el9.x86_64 141/156 2024-05-03T18:09:26.600 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-05-03T18:09:26.658 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-05-03T18:09:26.713 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : cephfs-top-2:19.0.0-3450.g9a047fe7.el9.noarch 148/156 2024-05-03T18:09:26.719 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: cephadm-2:19.0.0-3450.g9a047fe7.el9.noarch 123/156 2024-05-03T18:09:26.761 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : cephadm-2:19.0.0-3450.g9a047fe7.el9.noarch 123/156 2024-05-03T18:09:26.813 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3450.g9a047fe7.e 124/156 2024-05-03T18:09:26.817 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-05-03T18:09:26.899 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-05-03T18:09:26.912 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3450.g9a047fe7. 125/156 2024-05-03T18:09:26.949 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3450.g9a047fe7.el9.noa 126/156 2024-05-03T18:09:26.992 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : cephfs-shell-2:19.0.0-3450.g9a047fe7.el9.x86_64 144/156 2024-05-03T18:09:27.043 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-05-03T18:09:27.058 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-05-03T18:09:27.071 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libcephfs-devel-2:19.0.0-3450.g9a047fe7.el9.x86_ 146/156 2024-05-03T18:09:27.098 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-05-03T18:09:27.098 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-05-03T18:09:27.221 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-05-03T18:09:27.255 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-05-03T18:09:27.255 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-05-03T18:09:27.421 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-05-03T18:09:27.552 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-05-03T18:09:27.822 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-05-03T18:09:27.823 INFO:teuthology.orchestra.run.smithi150.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-05-03T18:09:27.823 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-03T18:09:27.934 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-05-03T18:09:27.986 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-05-03T18:09:28.052 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-05-03T18:09:28.259 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3450.g9a047fe7.el9.noa 126/156 2024-05-03T18:09:28.307 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3450.g9a047fe7.el9.n 127/156 2024-05-03T18:09:29.405 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3450.g9a047fe7.el9.n 127/156 2024-05-03T18:09:29.440 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3450.g9a0 128/156 2024-05-03T18:09:29.475 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : cephfs-top-2:19.0.0-3450.g9a047fe7.el9.noarch 148/156 2024-05-03T18:09:29.597 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3450.g9a0 128/156 2024-05-03T18:09:29.610 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-05-03T18:09:29.697 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-05-03T18:09:29.770 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3450.g9a047fe7.el 129/156 2024-05-03T18:09:29.809 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-2:19.0.0-3450.g9a047fe7.el9.x86_64 130/156 2024-05-03T18:09:29.878 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3450.g9a047fe7.el9.x86_64 130/156 2024-05-03T18:09:29.878 INFO:teuthology.orchestra.run.smithi099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T18:09:29.878 INFO:teuthology.orchestra.run.smithi099.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-03T18:09:29.878 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-03T18:09:29.879 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-03T18:09:29.879 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-03T18:09:29.916 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-05-03T18:09:29.943 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-rook-2:19.0.0-3450.g9a047fe7.el9.noarch 131/156 2024-05-03T18:09:29.967 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-05-03T18:09:29.968 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-05-03T18:09:29.979 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3450.g9a047fe7.el9.noarch 131/156 2024-05-03T18:09:30.085 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-05-03T18:09:30.132 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-05-03T18:09:30.133 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-05-03T18:09:30.135 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-2:19.0.0-3450.g9a047fe7.el9.x86_64 132/156 2024-05-03T18:09:30.212 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-fuse-2:19.0.0-3450.g9a047fe7.el9.x86_64 133/156 2024-05-03T18:09:30.272 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-05-03T18:09:30.407 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-05-03T18:09:30.708 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-05-03T18:09:30.708 INFO:teuthology.orchestra.run.smithi033.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-05-03T18:09:30.709 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:09:31.283 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-05-03T18:09:31.313 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-radosgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 135/156 2024-05-03T18:09:31.381 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 135/156 2024-05-03T18:09:31.381 INFO:teuthology.orchestra.run.smithi099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T18:09:31.381 INFO:teuthology.orchestra.run.smithi099.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-03T18:09:31.381 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-03T18:09:31.381 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-03T18:09:31.381 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-03T18:09:31.455 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3450.g9a047 136/156 2024-05-03T18:09:31.534 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3450.g9a047 136/156 2024-05-03T18:09:31.535 INFO:teuthology.orchestra.run.smithi099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T18:09:31.535 INFO:teuthology.orchestra.run.smithi099.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-03T18:09:31.535 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-03T18:09:31.631 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : cephfs-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 137/156 2024-05-03T18:09:31.706 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 137/156 2024-05-03T18:09:31.706 INFO:teuthology.orchestra.run.smithi099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T18:09:31.706 INFO:teuthology.orchestra.run.smithi099.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-03T18:09:31.707 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-03T18:09:31.707 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-03T18:09:31.707 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-03T18:09:32.081 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : rbd-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 138/156 2024-05-03T18:09:32.151 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 138/156 2024-05-03T18:09:32.151 INFO:teuthology.orchestra.run.smithi099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T18:09:32.151 INFO:teuthology.orchestra.run.smithi099.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-03T18:09:32.151 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-03T18:09:32.151 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-03T18:09:32.151 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-03T18:09:33.603 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-2:19.0.0-3450.g9a047fe7.el9.x86_64 1/156 2024-05-03T18:09:33.603 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-base-2:19.0.0-3450.g9a047fe7.el9.x86_64 2/156 2024-05-03T18:09:33.603 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-common-2:19.0.0-3450.g9a047fe7.el9.x86_64 3/156 2024-05-03T18:09:33.603 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-fuse-2:19.0.0-3450.g9a047fe7.el9.x86_64 4/156 2024-05-03T18:09:33.603 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3450.g9a047 5/156 2024-05-03T18:09:33.604 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mds-2:19.0.0-3450.g9a047fe7.el9.x86_64 6/156 2024-05-03T18:09:33.604 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-2:19.0.0-3450.g9a047fe7.el9.x86_64 7/156 2024-05-03T18:09:33.604 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mon-2:19.0.0-3450.g9a047fe7.el9.x86_64 8/156 2024-05-03T18:09:33.604 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-osd-2:19.0.0-3450.g9a047fe7.el9.x86_64 9/156 2024-05-03T18:09:33.604 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-radosgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 10/156 2024-05-03T18:09:33.604 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-selinux-2:19.0.0-3450.g9a047fe7.el9.x86_64 11/156 2024-05-03T18:09:33.604 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-test-2:19.0.0-3450.g9a047fe7.el9.x86_64 12/156 2024-05-03T18:09:33.604 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : cephfs-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 13/156 2024-05-03T18:09:33.604 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : cephfs-shell-2:19.0.0-3450.g9a047fe7.el9.x86_64 14/156 2024-05-03T18:09:33.604 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libcephfs-devel-2:19.0.0-3450.g9a047fe7.el9.x86_ 15/156 2024-05-03T18:09:33.604 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libcephfs2-2:19.0.0-3450.g9a047fe7.el9.x86_64 16/156 2024-05-03T18:09:33.604 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libcephsqlite-2:19.0.0-3450.g9a047fe7.el9.x86_64 17/156 2024-05-03T18:09:33.604 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librados-devel-2:19.0.0-3450.g9a047fe7.el9.x86_6 18/156 2024-05-03T18:09:33.605 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libradosstriper1-2:19.0.0-3450.g9a047fe7.el9.x86 19/156 2024-05-03T18:09:33.605 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librgw2-2:19.0.0-3450.g9a047fe7.el9.x86_64 20/156 2024-05-03T18:09:33.605 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3450.g9a047fe7.el 21/156 2024-05-03T18:09:33.605 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-ceph-common-2:19.0.0-3450.g9a047fe7.el9. 22/156 2024-05-03T18:09:33.605 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cephfs-2:19.0.0-3450.g9a047fe7.el9.x86_6 23/156 2024-05-03T18:09:33.605 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-rados-2:19.0.0-3450.g9a047fe7.el9.x86_64 24/156 2024-05-03T18:09:33.605 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-rbd-2:19.0.0-3450.g9a047fe7.el9.x86_64 25/156 2024-05-03T18:09:33.605 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-rgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 26/156 2024-05-03T18:09:33.605 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : rbd-fuse-2:19.0.0-3450.g9a047fe7.el9.x86_64 27/156 2024-05-03T18:09:33.605 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : rbd-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 28/156 2024-05-03T18:09:33.605 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : rbd-nbd-2:19.0.0-3450.g9a047fe7.el9.x86_64 29/156 2024-05-03T18:09:33.605 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3450.g9a047fe7. 30/156 2024-05-03T18:09:33.605 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3450.g9a047fe7.el9.noa 31/156 2024-05-03T18:09:33.605 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3450.g9a047fe7.el9.n 32/156 2024-05-03T18:09:33.606 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3450.g9a0 33/156 2024-05-03T18:09:33.606 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3450.g9a047fe7.el 34/156 2024-05-03T18:09:33.606 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3450.g9a047fe7.el9.noarch 35/156 2024-05-03T18:09:33.606 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3450.g9a047fe7.e 36/156 2024-05-03T18:09:33.606 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-volume-2:19.0.0-3450.g9a047fe7.el9.noarch 37/156 2024-05-03T18:09:33.606 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : cephadm-2:19.0.0-3450.g9a047fe7.el9.noarch 38/156 2024-05-03T18:09:33.606 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : cephfs-top-2:19.0.0-3450.g9a047fe7.el9.noarch 39/156 2024-05-03T18:09:33.606 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-05-03T18:09:33.606 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-05-03T18:09:33.606 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-05-03T18:09:33.606 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-05-03T18:09:33.606 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-05-03T18:09:33.606 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-05-03T18:09:33.606 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-05-03T18:09:33.607 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-05-03T18:09:33.607 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-05-03T18:09:33.607 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-05-03T18:09:33.607 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-05-03T18:09:33.607 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-05-03T18:09:33.607 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-05-03T18:09:33.607 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-05-03T18:09:33.607 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-05-03T18:09:33.607 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-05-03T18:09:33.607 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-05-03T18:09:33.607 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-05-03T18:09:33.607 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-05-03T18:09:33.607 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-05-03T18:09:33.608 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-05-03T18:09:33.608 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-05-03T18:09:33.608 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-05-03T18:09:33.608 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-05-03T18:09:33.608 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-05-03T18:09:33.608 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-05-03T18:09:33.608 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-05-03T18:09:33.608 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-05-03T18:09:33.608 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-05-03T18:09:33.608 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-05-03T18:09:33.608 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-05-03T18:09:33.608 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-05-03T18:09:33.608 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-05-03T18:09:33.608 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-05-03T18:09:33.608 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-05-03T18:09:33.609 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-05-03T18:09:33.609 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-05-03T18:09:33.609 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-05-03T18:09:33.609 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-05-03T18:09:33.609 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-05-03T18:09:33.609 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-05-03T18:09:33.609 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-05-03T18:09:33.609 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-05-03T18:09:33.609 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-05-03T18:09:33.609 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-05-03T18:09:33.609 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-05-03T18:09:33.609 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-05-03T18:09:33.609 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-05-03T18:09:33.609 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-05-03T18:09:33.609 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-05-03T18:09:33.609 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-05-03T18:09:33.610 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-05-03T18:09:33.610 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-05-03T18:09:33.610 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-05-03T18:09:33.610 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-05-03T18:09:33.610 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-05-03T18:09:33.610 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-05-03T18:09:33.610 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-05-03T18:09:33.610 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-05-03T18:09:33.610 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-05-03T18:09:33.610 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-05-03T18:09:33.610 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-05-03T18:09:33.610 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-05-03T18:09:33.610 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-05-03T18:09:33.610 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-05-03T18:09:33.611 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-05-03T18:09:33.611 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-05-03T18:09:33.611 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-05-03T18:09:33.611 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/156 2024-05-03T18:09:33.611 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/156 2024-05-03T18:09:33.611 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-05-03T18:09:33.611 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-05-03T18:09:33.611 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-05-03T18:09:33.611 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/156 2024-05-03T18:09:33.611 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-05-03T18:09:33.611 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-05-03T18:09:33.611 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-05-03T18:09:33.611 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-05-03T18:09:33.611 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-05-03T18:09:33.612 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-05-03T18:09:33.612 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-05-03T18:09:33.612 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-05-03T18:09:33.612 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-05-03T18:09:33.612 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-05-03T18:09:33.612 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-05-03T18:09:33.612 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-05-03T18:09:33.612 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-05-03T18:09:33.612 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-05-03T18:09:33.612 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-05-03T18:09:33.612 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-05-03T18:09:33.612 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-05-03T18:09:33.612 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-05-03T18:09:33.612 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-05-03T18:09:33.612 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-05-03T18:09:33.613 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-05-03T18:09:33.613 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-05-03T18:09:33.613 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-05-03T18:09:33.613 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-05-03T18:09:33.613 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-05-03T18:09:33.613 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-05-03T18:09:33.613 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-05-03T18:09:33.613 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-05-03T18:09:33.613 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-05-03T18:09:33.613 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-05-03T18:09:33.613 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-05-03T18:09:33.613 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-05-03T18:09:33.613 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-05-03T18:09:33.613 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-05-03T18:09:33.613 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-05-03T18:09:33.614 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librados2-2:19.0.0-3450.g9a047fe7.el9.x86_64 149/156 2024-05-03T18:09:33.614 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-05-03T18:09:33.614 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librbd1-2:19.0.0-3450.g9a047fe7.el9.x86_64 151/156 2024-05-03T18:09:33.614 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-05-03T18:09:33.614 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-05-03T18:09:33.614 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-05-03T18:09:33.614 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-05-03T18:09:36.158 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-05-03T18:09:36.158 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-03T18:09:36.158 INFO:teuthology.orchestra.run.smithi150.stdout:Upgraded: 2024-05-03T18:09:36.158 INFO:teuthology.orchestra.run.smithi150.stdout: librados2-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:09:36.159 INFO:teuthology.orchestra.run.smithi150.stdout: librbd1-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:09:36.159 INFO:teuthology.orchestra.run.smithi150.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-03T18:09:36.159 INFO:teuthology.orchestra.run.smithi150.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-03T18:09:36.159 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2024-05-03T18:09:36.159 INFO:teuthology.orchestra.run.smithi150.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-03T18:09:36.159 INFO:teuthology.orchestra.run.smithi150.stdout: automake-1.16.2-8.el9.noarch 2024-05-03T18:09:36.159 INFO:teuthology.orchestra.run.smithi150.stdout: bison-3.7.4-5.el9.x86_64 2024-05-03T18:09:36.159 INFO:teuthology.orchestra.run.smithi150.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-03T18:09:36.159 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:09:36.159 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-base-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:09:36.159 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-common-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:09:36.159 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-fuse-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:09:36.159 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-grafana-dashboards-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T18:09:36.159 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-immutable-object-cache-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:09:36.160 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mds-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:09:36.160 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:09:36.160 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-cephadm-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T18:09:36.160 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-dashboard-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T18:09:36.160 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T18:09:36.160 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-modules-core-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T18:09:36.160 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-rook-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T18:09:36.160 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mon-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:09:36.160 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-osd-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:09:36.160 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-prometheus-alerts-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T18:09:36.160 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-radosgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:09:36.160 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-selinux-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:09:36.160 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-test-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:09:36.161 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-volume-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T18:09:36.161 INFO:teuthology.orchestra.run.smithi150.stdout: cephadm-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T18:09:36.161 INFO:teuthology.orchestra.run.smithi150.stdout: cephfs-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:09:36.161 INFO:teuthology.orchestra.run.smithi150.stdout: cephfs-shell-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:09:36.161 INFO:teuthology.orchestra.run.smithi150.stdout: cephfs-top-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T18:09:36.161 INFO:teuthology.orchestra.run.smithi150.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-03T18:09:36.161 INFO:teuthology.orchestra.run.smithi150.stdout: flex-2.6.4-9.el9.x86_64 2024-05-03T18:09:36.161 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-03T18:09:36.161 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-03T18:09:36.161 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-03T18:09:36.161 INFO:teuthology.orchestra.run.smithi150.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-03T18:09:36.161 INFO:teuthology.orchestra.run.smithi150.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-03T18:09:36.161 INFO:teuthology.orchestra.run.smithi150.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-03T18:09:36.161 INFO:teuthology.orchestra.run.smithi150.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-03T18:09:36.162 INFO:teuthology.orchestra.run.smithi150.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-03T18:09:36.162 INFO:teuthology.orchestra.run.smithi150.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-03T18:09:36.162 INFO:teuthology.orchestra.run.smithi150.stdout: libcephfs-devel-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:09:36.162 INFO:teuthology.orchestra.run.smithi150.stdout: libcephfs2-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:09:36.162 INFO:teuthology.orchestra.run.smithi150.stdout: libcephsqlite-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:09:36.162 INFO:teuthology.orchestra.run.smithi150.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-03T18:09:36.162 INFO:teuthology.orchestra.run.smithi150.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-03T18:09:36.162 INFO:teuthology.orchestra.run.smithi150.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-03T18:09:36.162 INFO:teuthology.orchestra.run.smithi150.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-03T18:09:36.162 INFO:teuthology.orchestra.run.smithi150.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-03T18:09:36.162 INFO:teuthology.orchestra.run.smithi150.stdout: librados-devel-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:09:36.162 INFO:teuthology.orchestra.run.smithi150.stdout: libradosstriper1-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:09:36.163 INFO:teuthology.orchestra.run.smithi150.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-03T18:09:36.163 INFO:teuthology.orchestra.run.smithi150.stdout: librgw2-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:09:36.163 INFO:teuthology.orchestra.run.smithi150.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-03T18:09:36.163 INFO:teuthology.orchestra.run.smithi150.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-03T18:09:36.163 INFO:teuthology.orchestra.run.smithi150.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-03T18:09:36.163 INFO:teuthology.orchestra.run.smithi150.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-03T18:09:36.163 INFO:teuthology.orchestra.run.smithi150.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-03T18:09:36.163 INFO:teuthology.orchestra.run.smithi150.stdout: lua-5.4.4-4.el9.x86_64 2024-05-03T18:09:36.163 INFO:teuthology.orchestra.run.smithi150.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-03T18:09:36.163 INFO:teuthology.orchestra.run.smithi150.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-03T18:09:36.163 INFO:teuthology.orchestra.run.smithi150.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-03T18:09:36.163 INFO:teuthology.orchestra.run.smithi150.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-03T18:09:36.163 INFO:teuthology.orchestra.run.smithi150.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-03T18:09:36.163 INFO:teuthology.orchestra.run.smithi150.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-03T18:09:36.164 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-13.14-1.el9.x86_64 2024-05-03T18:09:36.164 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-03T18:09:36.164 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-03T18:09:36.164 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-03T18:09:36.164 INFO:teuthology.orchestra.run.smithi150.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-03T18:09:36.164 INFO:teuthology.orchestra.run.smithi150.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-03T18:09:36.164 INFO:teuthology.orchestra.run.smithi150.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-03T18:09:36.164 INFO:teuthology.orchestra.run.smithi150.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-03T18:09:36.164 INFO:teuthology.orchestra.run.smithi150.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-03T18:09:36.164 INFO:teuthology.orchestra.run.smithi150.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-03T18:09:36.164 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-03T18:09:36.164 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ceph-argparse-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:09:36.164 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ceph-common-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:09:36.164 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cephfs-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:09:36.164 INFO:teuthology.orchestra.run.smithi150.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-03T18:09:36.165 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-03T18:09:36.165 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-03T18:09:36.165 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-03T18:09:36.165 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-03T18:09:36.165 INFO:teuthology.orchestra.run.smithi150.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-03T18:09:36.165 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-03T18:09:36.165 INFO:teuthology.orchestra.run.smithi150.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-03T18:09:36.165 INFO:teuthology.orchestra.run.smithi150.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-03T18:09:36.165 INFO:teuthology.orchestra.run.smithi150.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-03T18:09:36.165 INFO:teuthology.orchestra.run.smithi150.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-03T18:09:36.165 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-03T18:09:36.165 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-03T18:09:36.165 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-03T18:09:36.165 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-03T18:09:36.166 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-03T18:09:36.166 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-03T18:09:36.166 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-03T18:09:36.166 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-03T18:09:36.166 INFO:teuthology.orchestra.run.smithi150.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-03T18:09:36.166 INFO:teuthology.orchestra.run.smithi150.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-03T18:09:36.166 INFO:teuthology.orchestra.run.smithi150.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-03T18:09:36.166 INFO:teuthology.orchestra.run.smithi150.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-03T18:09:36.166 INFO:teuthology.orchestra.run.smithi150.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-03T18:09:36.166 INFO:teuthology.orchestra.run.smithi150.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-03T18:09:36.166 INFO:teuthology.orchestra.run.smithi150.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-03T18:09:36.166 INFO:teuthology.orchestra.run.smithi150.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-03T18:09:36.166 INFO:teuthology.orchestra.run.smithi150.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-03T18:09:36.167 INFO:teuthology.orchestra.run.smithi150.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-03T18:09:36.167 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-03T18:09:36.167 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ply-3.11-14.el9.noarch 2024-05-03T18:09:36.167 INFO:teuthology.orchestra.run.smithi150.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-03T18:09:36.167 INFO:teuthology.orchestra.run.smithi150.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-03T18:09:36.167 INFO:teuthology.orchestra.run.smithi150.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-03T18:09:36.167 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-03T18:09:36.167 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-03T18:09:36.167 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-03T18:09:36.167 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-03T18:09:36.167 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-03T18:09:36.167 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-03T18:09:36.167 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rados-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:09:36.167 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rbd-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:09:36.168 INFO:teuthology.orchestra.run.smithi150.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-03T18:09:36.168 INFO:teuthology.orchestra.run.smithi150.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-03T18:09:36.168 INFO:teuthology.orchestra.run.smithi150.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-03T18:09:36.168 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:09:36.168 INFO:teuthology.orchestra.run.smithi150.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-03T18:09:36.168 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-03T18:09:36.168 INFO:teuthology.orchestra.run.smithi150.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-03T18:09:36.168 INFO:teuthology.orchestra.run.smithi150.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-03T18:09:36.168 INFO:teuthology.orchestra.run.smithi150.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-03T18:09:36.168 INFO:teuthology.orchestra.run.smithi150.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-03T18:09:36.168 INFO:teuthology.orchestra.run.smithi150.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-03T18:09:36.168 INFO:teuthology.orchestra.run.smithi150.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-03T18:09:36.168 INFO:teuthology.orchestra.run.smithi150.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-03T18:09:36.168 INFO:teuthology.orchestra.run.smithi150.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-03T18:09:36.169 INFO:teuthology.orchestra.run.smithi150.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-03T18:09:36.169 INFO:teuthology.orchestra.run.smithi150.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-03T18:09:36.169 INFO:teuthology.orchestra.run.smithi150.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-03T18:09:36.169 INFO:teuthology.orchestra.run.smithi150.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-03T18:09:36.169 INFO:teuthology.orchestra.run.smithi150.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-03T18:09:36.169 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-fuse-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:09:36.169 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:09:36.169 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-nbd-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:09:36.169 INFO:teuthology.orchestra.run.smithi150.stdout: re2-1:20211101-3.el9.x86_64 2024-05-03T18:09:36.169 INFO:teuthology.orchestra.run.smithi150.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-03T18:09:36.169 INFO:teuthology.orchestra.run.smithi150.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-03T18:09:36.169 INFO:teuthology.orchestra.run.smithi150.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-03T18:09:36.169 INFO:teuthology.orchestra.run.smithi150.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-03T18:09:36.170 INFO:teuthology.orchestra.run.smithi150.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-03T18:09:36.170 INFO:teuthology.orchestra.run.smithi150.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-03T18:09:36.170 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-03T18:09:36.170 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-05-03T18:09:36.737 DEBUG:teuthology.parallel:result is None 2024-05-03T18:09:36.894 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-2:19.0.0-3450.g9a047fe7.el9.x86_64 1/156 2024-05-03T18:09:36.894 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-base-2:19.0.0-3450.g9a047fe7.el9.x86_64 2/156 2024-05-03T18:09:36.894 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-common-2:19.0.0-3450.g9a047fe7.el9.x86_64 3/156 2024-05-03T18:09:36.894 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-fuse-2:19.0.0-3450.g9a047fe7.el9.x86_64 4/156 2024-05-03T18:09:36.894 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3450.g9a047 5/156 2024-05-03T18:09:36.894 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mds-2:19.0.0-3450.g9a047fe7.el9.x86_64 6/156 2024-05-03T18:09:36.894 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-2:19.0.0-3450.g9a047fe7.el9.x86_64 7/156 2024-05-03T18:09:36.894 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mon-2:19.0.0-3450.g9a047fe7.el9.x86_64 8/156 2024-05-03T18:09:36.894 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-osd-2:19.0.0-3450.g9a047fe7.el9.x86_64 9/156 2024-05-03T18:09:36.894 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-radosgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 10/156 2024-05-03T18:09:36.895 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-selinux-2:19.0.0-3450.g9a047fe7.el9.x86_64 11/156 2024-05-03T18:09:36.895 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-test-2:19.0.0-3450.g9a047fe7.el9.x86_64 12/156 2024-05-03T18:09:36.895 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : cephfs-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 13/156 2024-05-03T18:09:36.895 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : cephfs-shell-2:19.0.0-3450.g9a047fe7.el9.x86_64 14/156 2024-05-03T18:09:36.895 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libcephfs-devel-2:19.0.0-3450.g9a047fe7.el9.x86_ 15/156 2024-05-03T18:09:36.895 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libcephfs2-2:19.0.0-3450.g9a047fe7.el9.x86_64 16/156 2024-05-03T18:09:36.895 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libcephsqlite-2:19.0.0-3450.g9a047fe7.el9.x86_64 17/156 2024-05-03T18:09:36.895 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librados-devel-2:19.0.0-3450.g9a047fe7.el9.x86_6 18/156 2024-05-03T18:09:36.895 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libradosstriper1-2:19.0.0-3450.g9a047fe7.el9.x86 19/156 2024-05-03T18:09:36.895 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librgw2-2:19.0.0-3450.g9a047fe7.el9.x86_64 20/156 2024-05-03T18:09:36.895 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3450.g9a047fe7.el 21/156 2024-05-03T18:09:36.895 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-ceph-common-2:19.0.0-3450.g9a047fe7.el9. 22/156 2024-05-03T18:09:36.895 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cephfs-2:19.0.0-3450.g9a047fe7.el9.x86_6 23/156 2024-05-03T18:09:36.895 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-rados-2:19.0.0-3450.g9a047fe7.el9.x86_64 24/156 2024-05-03T18:09:36.896 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-rbd-2:19.0.0-3450.g9a047fe7.el9.x86_64 25/156 2024-05-03T18:09:36.896 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-rgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 26/156 2024-05-03T18:09:36.896 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : rbd-fuse-2:19.0.0-3450.g9a047fe7.el9.x86_64 27/156 2024-05-03T18:09:36.907 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : rbd-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 28/156 2024-05-03T18:09:36.908 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : rbd-nbd-2:19.0.0-3450.g9a047fe7.el9.x86_64 29/156 2024-05-03T18:09:36.908 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3450.g9a047fe7. 30/156 2024-05-03T18:09:36.908 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3450.g9a047fe7.el9.noa 31/156 2024-05-03T18:09:36.908 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3450.g9a047fe7.el9.n 32/156 2024-05-03T18:09:36.908 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3450.g9a0 33/156 2024-05-03T18:09:36.908 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3450.g9a047fe7.el 34/156 2024-05-03T18:09:36.908 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3450.g9a047fe7.el9.noarch 35/156 2024-05-03T18:09:36.908 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3450.g9a047fe7.e 36/156 2024-05-03T18:09:36.908 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-volume-2:19.0.0-3450.g9a047fe7.el9.noarch 37/156 2024-05-03T18:09:36.908 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : cephadm-2:19.0.0-3450.g9a047fe7.el9.noarch 38/156 2024-05-03T18:09:36.908 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : cephfs-top-2:19.0.0-3450.g9a047fe7.el9.noarch 39/156 2024-05-03T18:09:36.909 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-05-03T18:09:36.909 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-05-03T18:09:36.909 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-05-03T18:09:36.909 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-05-03T18:09:36.909 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-05-03T18:09:36.909 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-05-03T18:09:36.909 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-05-03T18:09:36.909 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-05-03T18:09:36.909 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-05-03T18:09:36.909 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-05-03T18:09:36.909 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-05-03T18:09:36.909 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-05-03T18:09:36.909 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-05-03T18:09:36.909 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-05-03T18:09:36.910 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-05-03T18:09:36.910 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-05-03T18:09:36.910 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-05-03T18:09:36.910 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-05-03T18:09:36.910 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-05-03T18:09:36.910 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-05-03T18:09:36.910 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-05-03T18:09:36.910 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-05-03T18:09:36.910 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-05-03T18:09:36.910 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-05-03T18:09:36.910 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-05-03T18:09:36.910 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-05-03T18:09:36.910 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-05-03T18:09:36.910 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-05-03T18:09:36.911 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-05-03T18:09:36.911 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-05-03T18:09:36.911 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-05-03T18:09:36.911 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-05-03T18:09:36.911 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-05-03T18:09:36.911 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-05-03T18:09:36.911 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-05-03T18:09:36.911 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-05-03T18:09:36.911 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-05-03T18:09:36.911 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-05-03T18:09:36.911 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-05-03T18:09:36.911 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-05-03T18:09:36.911 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-05-03T18:09:36.911 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-05-03T18:09:36.912 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-05-03T18:09:36.912 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-05-03T18:09:36.912 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-05-03T18:09:36.912 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-05-03T18:09:36.912 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-05-03T18:09:36.912 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-05-03T18:09:36.912 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-05-03T18:09:36.912 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-05-03T18:09:36.912 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-05-03T18:09:36.912 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-05-03T18:09:36.912 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-05-03T18:09:36.912 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-05-03T18:09:36.912 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-05-03T18:09:36.913 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-05-03T18:09:36.913 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-05-03T18:09:36.913 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-05-03T18:09:36.913 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-05-03T18:09:36.913 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-05-03T18:09:36.913 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-05-03T18:09:36.913 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-05-03T18:09:36.913 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-05-03T18:09:36.913 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-05-03T18:09:36.913 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-05-03T18:09:36.913 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-05-03T18:09:36.913 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-05-03T18:09:36.913 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-05-03T18:09:36.913 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/156 2024-05-03T18:09:36.913 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/156 2024-05-03T18:09:36.913 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-05-03T18:09:36.914 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-05-03T18:09:36.914 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-05-03T18:09:36.914 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/156 2024-05-03T18:09:36.914 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-05-03T18:09:36.914 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-05-03T18:09:36.914 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-05-03T18:09:36.914 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-05-03T18:09:36.914 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-05-03T18:09:36.914 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-05-03T18:09:36.914 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-05-03T18:09:36.914 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-05-03T18:09:36.914 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-05-03T18:09:36.914 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-05-03T18:09:36.914 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-05-03T18:09:36.915 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-05-03T18:09:36.915 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-05-03T18:09:36.915 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-05-03T18:09:36.915 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-05-03T18:09:36.915 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-05-03T18:09:36.915 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-05-03T18:09:36.915 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-05-03T18:09:36.915 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-05-03T18:09:36.915 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-05-03T18:09:36.915 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-05-03T18:09:36.915 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-05-03T18:09:36.915 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-05-03T18:09:36.915 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-05-03T18:09:36.915 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-05-03T18:09:36.915 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-05-03T18:09:36.916 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-05-03T18:09:36.916 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-05-03T18:09:36.916 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-05-03T18:09:36.916 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-05-03T18:09:36.916 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-05-03T18:09:36.916 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-05-03T18:09:36.916 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-05-03T18:09:36.916 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-05-03T18:09:36.916 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-05-03T18:09:36.916 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librados2-2:19.0.0-3450.g9a047fe7.el9.x86_64 149/156 2024-05-03T18:09:36.916 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-05-03T18:09:36.916 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librbd1-2:19.0.0-3450.g9a047fe7.el9.x86_64 151/156 2024-05-03T18:09:36.916 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-05-03T18:09:36.916 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-05-03T18:09:36.917 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-05-03T18:09:36.917 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-05-03T18:09:37.232 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-test-2:19.0.0-3450.g9a047fe7.el9.x86_64 139/156 2024-05-03T18:09:37.288 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : rbd-fuse-2:19.0.0-3450.g9a047fe7.el9.x86_64 140/156 2024-05-03T18:09:37.374 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : rbd-nbd-2:19.0.0-3450.g9a047fe7.el9.x86_64 141/156 2024-05-03T18:09:37.417 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-05-03T18:09:37.466 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-05-03T18:09:37.813 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : cephfs-shell-2:19.0.0-3450.g9a047fe7.el9.x86_64 144/156 2024-05-03T18:09:37.877 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-05-03T18:09:37.905 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libcephfs-devel-2:19.0.0-3450.g9a047fe7.el9.x86_ 146/156 2024-05-03T18:09:38.786 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-05-03T18:09:38.836 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-05-03T18:09:38.903 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-05-03T18:09:39.434 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-05-03T18:09:39.435 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:09:39.435 INFO:teuthology.orchestra.run.smithi033.stdout:Upgraded: 2024-05-03T18:09:39.435 INFO:teuthology.orchestra.run.smithi033.stdout: librados2-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:09:39.435 INFO:teuthology.orchestra.run.smithi033.stdout: librbd1-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:09:39.435 INFO:teuthology.orchestra.run.smithi033.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-03T18:09:39.435 INFO:teuthology.orchestra.run.smithi033.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-03T18:09:39.435 INFO:teuthology.orchestra.run.smithi033.stdout:Installed: 2024-05-03T18:09:39.435 INFO:teuthology.orchestra.run.smithi033.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-03T18:09:39.435 INFO:teuthology.orchestra.run.smithi033.stdout: automake-1.16.2-8.el9.noarch 2024-05-03T18:09:39.435 INFO:teuthology.orchestra.run.smithi033.stdout: bison-3.7.4-5.el9.x86_64 2024-05-03T18:09:39.435 INFO:teuthology.orchestra.run.smithi033.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-03T18:09:39.435 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:09:39.436 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-base-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:09:39.436 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-common-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:09:39.436 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-fuse-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:09:39.436 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-grafana-dashboards-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T18:09:39.436 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-immutable-object-cache-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:09:39.436 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mds-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:09:39.436 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:09:39.436 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-cephadm-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T18:09:39.436 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-dashboard-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T18:09:39.436 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T18:09:39.436 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-modules-core-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T18:09:39.436 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-rook-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T18:09:39.436 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mon-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:09:39.437 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-osd-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:09:39.437 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-prometheus-alerts-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T18:09:39.437 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-radosgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:09:39.437 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-selinux-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:09:39.437 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-test-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:09:39.437 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-volume-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T18:09:39.437 INFO:teuthology.orchestra.run.smithi033.stdout: cephadm-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T18:09:39.437 INFO:teuthology.orchestra.run.smithi033.stdout: cephfs-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:09:39.437 INFO:teuthology.orchestra.run.smithi033.stdout: cephfs-shell-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:09:39.437 INFO:teuthology.orchestra.run.smithi033.stdout: cephfs-top-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T18:09:39.437 INFO:teuthology.orchestra.run.smithi033.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-03T18:09:39.437 INFO:teuthology.orchestra.run.smithi033.stdout: flex-2.6.4-9.el9.x86_64 2024-05-03T18:09:39.437 INFO:teuthology.orchestra.run.smithi033.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-03T18:09:39.437 INFO:teuthology.orchestra.run.smithi033.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-03T18:09:39.438 INFO:teuthology.orchestra.run.smithi033.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-03T18:09:39.438 INFO:teuthology.orchestra.run.smithi033.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-03T18:09:39.438 INFO:teuthology.orchestra.run.smithi033.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-03T18:09:39.438 INFO:teuthology.orchestra.run.smithi033.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-03T18:09:39.438 INFO:teuthology.orchestra.run.smithi033.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-03T18:09:39.438 INFO:teuthology.orchestra.run.smithi033.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-03T18:09:39.438 INFO:teuthology.orchestra.run.smithi033.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-03T18:09:39.438 INFO:teuthology.orchestra.run.smithi033.stdout: libcephfs-devel-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:09:39.438 INFO:teuthology.orchestra.run.smithi033.stdout: libcephfs2-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:09:39.438 INFO:teuthology.orchestra.run.smithi033.stdout: libcephsqlite-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:09:39.438 INFO:teuthology.orchestra.run.smithi033.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-03T18:09:39.438 INFO:teuthology.orchestra.run.smithi033.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-03T18:09:39.438 INFO:teuthology.orchestra.run.smithi033.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-03T18:09:39.438 INFO:teuthology.orchestra.run.smithi033.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-03T18:09:39.438 INFO:teuthology.orchestra.run.smithi033.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-03T18:09:39.439 INFO:teuthology.orchestra.run.smithi033.stdout: librados-devel-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:09:39.439 INFO:teuthology.orchestra.run.smithi033.stdout: libradosstriper1-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:09:39.439 INFO:teuthology.orchestra.run.smithi033.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-03T18:09:39.439 INFO:teuthology.orchestra.run.smithi033.stdout: librgw2-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:09:39.439 INFO:teuthology.orchestra.run.smithi033.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-03T18:09:39.439 INFO:teuthology.orchestra.run.smithi033.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-03T18:09:39.439 INFO:teuthology.orchestra.run.smithi033.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-03T18:09:39.439 INFO:teuthology.orchestra.run.smithi033.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-03T18:09:39.439 INFO:teuthology.orchestra.run.smithi033.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-03T18:09:39.439 INFO:teuthology.orchestra.run.smithi033.stdout: lua-5.4.4-4.el9.x86_64 2024-05-03T18:09:39.439 INFO:teuthology.orchestra.run.smithi033.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-03T18:09:39.439 INFO:teuthology.orchestra.run.smithi033.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-03T18:09:39.439 INFO:teuthology.orchestra.run.smithi033.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-03T18:09:39.439 INFO:teuthology.orchestra.run.smithi033.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-03T18:09:39.440 INFO:teuthology.orchestra.run.smithi033.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-03T18:09:39.440 INFO:teuthology.orchestra.run.smithi033.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-03T18:09:39.440 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql-13.14-1.el9.x86_64 2024-05-03T18:09:39.440 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-03T18:09:39.440 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-03T18:09:39.440 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-03T18:09:39.440 INFO:teuthology.orchestra.run.smithi033.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-03T18:09:39.440 INFO:teuthology.orchestra.run.smithi033.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-03T18:09:39.440 INFO:teuthology.orchestra.run.smithi033.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-03T18:09:39.440 INFO:teuthology.orchestra.run.smithi033.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-03T18:09:39.440 INFO:teuthology.orchestra.run.smithi033.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-03T18:09:39.440 INFO:teuthology.orchestra.run.smithi033.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-03T18:09:39.440 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-03T18:09:39.441 INFO:teuthology.orchestra.run.smithi033.stdout: python3-ceph-argparse-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:09:39.441 INFO:teuthology.orchestra.run.smithi033.stdout: python3-ceph-common-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:09:39.441 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cephfs-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:09:39.441 INFO:teuthology.orchestra.run.smithi033.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-03T18:09:39.441 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-03T18:09:39.441 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-03T18:09:39.441 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-03T18:09:39.441 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-03T18:09:39.441 INFO:teuthology.orchestra.run.smithi033.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-03T18:09:39.441 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-03T18:09:39.441 INFO:teuthology.orchestra.run.smithi033.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-03T18:09:39.441 INFO:teuthology.orchestra.run.smithi033.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-03T18:09:39.442 INFO:teuthology.orchestra.run.smithi033.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-03T18:09:39.442 INFO:teuthology.orchestra.run.smithi033.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-03T18:09:39.442 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-03T18:09:39.442 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-03T18:09:39.442 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-03T18:09:39.442 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-03T18:09:39.442 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-03T18:09:39.442 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-03T18:09:39.442 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-03T18:09:39.442 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-03T18:09:39.442 INFO:teuthology.orchestra.run.smithi033.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-03T18:09:39.442 INFO:teuthology.orchestra.run.smithi033.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-03T18:09:39.442 INFO:teuthology.orchestra.run.smithi033.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-03T18:09:39.442 INFO:teuthology.orchestra.run.smithi033.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-03T18:09:39.443 INFO:teuthology.orchestra.run.smithi033.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-03T18:09:39.443 INFO:teuthology.orchestra.run.smithi033.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-03T18:09:39.443 INFO:teuthology.orchestra.run.smithi033.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-03T18:09:39.443 INFO:teuthology.orchestra.run.smithi033.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-03T18:09:39.443 INFO:teuthology.orchestra.run.smithi033.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-03T18:09:39.443 INFO:teuthology.orchestra.run.smithi033.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-03T18:09:39.443 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-03T18:09:39.444 INFO:teuthology.orchestra.run.smithi033.stdout: python3-ply-3.11-14.el9.noarch 2024-05-03T18:09:39.444 INFO:teuthology.orchestra.run.smithi033.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-03T18:09:39.444 INFO:teuthology.orchestra.run.smithi033.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-03T18:09:39.444 INFO:teuthology.orchestra.run.smithi033.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-03T18:09:39.444 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-03T18:09:39.444 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-03T18:09:39.444 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-03T18:09:39.444 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-03T18:09:39.444 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-03T18:09:39.445 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-03T18:09:39.445 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rados-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:09:39.445 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rbd-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:09:39.445 INFO:teuthology.orchestra.run.smithi033.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-03T18:09:39.445 INFO:teuthology.orchestra.run.smithi033.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-03T18:09:39.445 INFO:teuthology.orchestra.run.smithi033.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-03T18:09:39.445 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:09:39.445 INFO:teuthology.orchestra.run.smithi033.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-03T18:09:39.446 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-03T18:09:39.446 INFO:teuthology.orchestra.run.smithi033.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-03T18:09:39.446 INFO:teuthology.orchestra.run.smithi033.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-03T18:09:39.446 INFO:teuthology.orchestra.run.smithi033.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-03T18:09:39.446 INFO:teuthology.orchestra.run.smithi033.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-03T18:09:39.446 INFO:teuthology.orchestra.run.smithi033.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-03T18:09:39.446 INFO:teuthology.orchestra.run.smithi033.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-03T18:09:39.446 INFO:teuthology.orchestra.run.smithi033.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-03T18:09:39.446 INFO:teuthology.orchestra.run.smithi033.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-03T18:09:39.447 INFO:teuthology.orchestra.run.smithi033.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-03T18:09:39.447 INFO:teuthology.orchestra.run.smithi033.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-03T18:09:39.447 INFO:teuthology.orchestra.run.smithi033.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-03T18:09:39.447 INFO:teuthology.orchestra.run.smithi033.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-03T18:09:39.447 INFO:teuthology.orchestra.run.smithi033.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-03T18:09:39.447 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-fuse-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:09:39.447 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:09:39.447 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-nbd-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:09:39.447 INFO:teuthology.orchestra.run.smithi033.stdout: re2-1:20211101-3.el9.x86_64 2024-05-03T18:09:39.447 INFO:teuthology.orchestra.run.smithi033.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-03T18:09:39.447 INFO:teuthology.orchestra.run.smithi033.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-03T18:09:39.447 INFO:teuthology.orchestra.run.smithi033.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-03T18:09:39.447 INFO:teuthology.orchestra.run.smithi033.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-03T18:09:39.448 INFO:teuthology.orchestra.run.smithi033.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-03T18:09:39.448 INFO:teuthology.orchestra.run.smithi033.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-03T18:09:39.448 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:09:39.448 INFO:teuthology.orchestra.run.smithi033.stdout:Complete! 2024-05-03T18:09:40.051 DEBUG:teuthology.parallel:result is None 2024-05-03T18:09:40.234 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : cephfs-top-2:19.0.0-3450.g9a047fe7.el9.noarch 148/156 2024-05-03T18:09:40.335 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-05-03T18:09:40.414 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-05-03T18:09:40.601 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-05-03T18:09:40.641 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-05-03T18:09:40.641 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-05-03T18:09:40.767 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-05-03T18:09:40.823 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-05-03T18:09:40.824 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-05-03T18:09:40.997 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-05-03T18:09:41.127 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-05-03T18:09:41.418 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-05-03T18:09:41.418 INFO:teuthology.orchestra.run.smithi099.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-05-03T18:09:41.419 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-03T18:09:47.399 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-2:19.0.0-3450.g9a047fe7.el9.x86_64 1/156 2024-05-03T18:09:47.400 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-base-2:19.0.0-3450.g9a047fe7.el9.x86_64 2/156 2024-05-03T18:09:47.400 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-common-2:19.0.0-3450.g9a047fe7.el9.x86_64 3/156 2024-05-03T18:09:47.400 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-fuse-2:19.0.0-3450.g9a047fe7.el9.x86_64 4/156 2024-05-03T18:09:47.400 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3450.g9a047 5/156 2024-05-03T18:09:47.400 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mds-2:19.0.0-3450.g9a047fe7.el9.x86_64 6/156 2024-05-03T18:09:47.400 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-2:19.0.0-3450.g9a047fe7.el9.x86_64 7/156 2024-05-03T18:09:47.400 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mon-2:19.0.0-3450.g9a047fe7.el9.x86_64 8/156 2024-05-03T18:09:47.400 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-osd-2:19.0.0-3450.g9a047fe7.el9.x86_64 9/156 2024-05-03T18:09:47.400 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-radosgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 10/156 2024-05-03T18:09:47.400 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-selinux-2:19.0.0-3450.g9a047fe7.el9.x86_64 11/156 2024-05-03T18:09:47.400 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-test-2:19.0.0-3450.g9a047fe7.el9.x86_64 12/156 2024-05-03T18:09:47.400 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : cephfs-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 13/156 2024-05-03T18:09:47.400 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : cephfs-shell-2:19.0.0-3450.g9a047fe7.el9.x86_64 14/156 2024-05-03T18:09:47.401 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libcephfs-devel-2:19.0.0-3450.g9a047fe7.el9.x86_ 15/156 2024-05-03T18:09:47.401 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libcephfs2-2:19.0.0-3450.g9a047fe7.el9.x86_64 16/156 2024-05-03T18:09:47.401 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libcephsqlite-2:19.0.0-3450.g9a047fe7.el9.x86_64 17/156 2024-05-03T18:09:47.401 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librados-devel-2:19.0.0-3450.g9a047fe7.el9.x86_6 18/156 2024-05-03T18:09:47.401 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libradosstriper1-2:19.0.0-3450.g9a047fe7.el9.x86 19/156 2024-05-03T18:09:47.401 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librgw2-2:19.0.0-3450.g9a047fe7.el9.x86_64 20/156 2024-05-03T18:09:47.401 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3450.g9a047fe7.el 21/156 2024-05-03T18:09:47.401 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-ceph-common-2:19.0.0-3450.g9a047fe7.el9. 22/156 2024-05-03T18:09:47.401 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cephfs-2:19.0.0-3450.g9a047fe7.el9.x86_6 23/156 2024-05-03T18:09:47.401 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-rados-2:19.0.0-3450.g9a047fe7.el9.x86_64 24/156 2024-05-03T18:09:47.401 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-rbd-2:19.0.0-3450.g9a047fe7.el9.x86_64 25/156 2024-05-03T18:09:47.401 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-rgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 26/156 2024-05-03T18:09:47.401 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : rbd-fuse-2:19.0.0-3450.g9a047fe7.el9.x86_64 27/156 2024-05-03T18:09:47.401 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : rbd-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 28/156 2024-05-03T18:09:47.402 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : rbd-nbd-2:19.0.0-3450.g9a047fe7.el9.x86_64 29/156 2024-05-03T18:09:47.402 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3450.g9a047fe7. 30/156 2024-05-03T18:09:47.402 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3450.g9a047fe7.el9.noa 31/156 2024-05-03T18:09:47.402 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3450.g9a047fe7.el9.n 32/156 2024-05-03T18:09:47.402 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3450.g9a0 33/156 2024-05-03T18:09:47.402 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3450.g9a047fe7.el 34/156 2024-05-03T18:09:47.402 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3450.g9a047fe7.el9.noarch 35/156 2024-05-03T18:09:47.402 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3450.g9a047fe7.e 36/156 2024-05-03T18:09:47.402 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-volume-2:19.0.0-3450.g9a047fe7.el9.noarch 37/156 2024-05-03T18:09:47.402 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : cephadm-2:19.0.0-3450.g9a047fe7.el9.noarch 38/156 2024-05-03T18:09:47.402 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : cephfs-top-2:19.0.0-3450.g9a047fe7.el9.noarch 39/156 2024-05-03T18:09:47.402 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-05-03T18:09:47.402 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-05-03T18:09:47.402 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-05-03T18:09:47.403 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-05-03T18:09:47.403 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-05-03T18:09:47.403 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-05-03T18:09:47.403 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-05-03T18:09:47.403 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-05-03T18:09:47.403 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-05-03T18:09:47.403 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-05-03T18:09:47.403 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-05-03T18:09:47.403 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-05-03T18:09:47.403 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-05-03T18:09:47.403 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-05-03T18:09:47.403 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-05-03T18:09:47.403 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-05-03T18:09:47.404 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-05-03T18:09:47.404 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-05-03T18:09:47.404 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-05-03T18:09:47.404 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-05-03T18:09:47.404 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-05-03T18:09:47.404 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-05-03T18:09:47.404 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-05-03T18:09:47.404 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-05-03T18:09:47.404 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-05-03T18:09:47.404 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-05-03T18:09:47.404 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-05-03T18:09:47.404 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-05-03T18:09:47.404 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-05-03T18:09:47.405 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-05-03T18:09:47.405 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-05-03T18:09:47.405 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-05-03T18:09:47.405 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-05-03T18:09:47.405 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-05-03T18:09:47.405 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-05-03T18:09:47.405 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-05-03T18:09:47.405 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-05-03T18:09:47.405 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-05-03T18:09:47.405 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-05-03T18:09:47.405 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-05-03T18:09:47.405 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-05-03T18:09:47.405 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-05-03T18:09:47.406 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-05-03T18:09:47.406 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-05-03T18:09:47.406 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-05-03T18:09:47.406 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-05-03T18:09:47.406 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-05-03T18:09:47.406 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-05-03T18:09:47.406 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-05-03T18:09:47.406 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-05-03T18:09:47.406 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-05-03T18:09:47.406 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-05-03T18:09:47.406 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-05-03T18:09:47.406 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-05-03T18:09:47.406 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-05-03T18:09:47.407 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-05-03T18:09:47.407 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-05-03T18:09:47.407 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-05-03T18:09:47.407 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-05-03T18:09:47.407 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-05-03T18:09:47.407 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-05-03T18:09:47.407 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-05-03T18:09:47.407 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-05-03T18:09:47.407 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-05-03T18:09:47.407 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-05-03T18:09:47.407 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-05-03T18:09:47.407 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-05-03T18:09:47.407 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-05-03T18:09:47.408 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/156 2024-05-03T18:09:47.408 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/156 2024-05-03T18:09:47.408 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-05-03T18:09:47.408 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-05-03T18:09:47.408 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-05-03T18:09:47.408 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/156 2024-05-03T18:09:47.408 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-05-03T18:09:47.408 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-05-03T18:09:47.408 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-05-03T18:09:47.408 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-05-03T18:09:47.408 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-05-03T18:09:47.408 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-05-03T18:09:47.408 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-05-03T18:09:47.408 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-05-03T18:09:47.409 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-05-03T18:09:47.409 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-05-03T18:09:47.409 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-05-03T18:09:47.409 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-05-03T18:09:47.409 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-05-03T18:09:47.409 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-05-03T18:09:47.409 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-05-03T18:09:47.409 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-05-03T18:09:47.409 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-05-03T18:09:47.409 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-05-03T18:09:47.409 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-05-03T18:09:47.409 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-05-03T18:09:47.409 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-05-03T18:09:47.410 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-05-03T18:09:47.410 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-05-03T18:09:47.410 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-05-03T18:09:47.410 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-05-03T18:09:47.410 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-05-03T18:09:47.410 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-05-03T18:09:47.410 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-05-03T18:09:47.410 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-05-03T18:09:47.410 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-05-03T18:09:47.410 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-05-03T18:09:47.410 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-05-03T18:09:47.410 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-05-03T18:09:47.411 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-05-03T18:09:47.411 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-05-03T18:09:47.411 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librados2-2:19.0.0-3450.g9a047fe7.el9.x86_64 149/156 2024-05-03T18:09:47.411 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-05-03T18:09:47.411 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librbd1-2:19.0.0-3450.g9a047fe7.el9.x86_64 151/156 2024-05-03T18:09:47.411 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-05-03T18:09:47.411 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-05-03T18:09:47.411 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-05-03T18:09:47.411 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-05-03T18:09:49.845 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-05-03T18:09:49.845 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-03T18:09:49.845 INFO:teuthology.orchestra.run.smithi099.stdout:Upgraded: 2024-05-03T18:09:49.845 INFO:teuthology.orchestra.run.smithi099.stdout: librados2-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:09:49.845 INFO:teuthology.orchestra.run.smithi099.stdout: librbd1-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:09:49.845 INFO:teuthology.orchestra.run.smithi099.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-03T18:09:49.846 INFO:teuthology.orchestra.run.smithi099.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-03T18:09:49.846 INFO:teuthology.orchestra.run.smithi099.stdout:Installed: 2024-05-03T18:09:49.846 INFO:teuthology.orchestra.run.smithi099.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-03T18:09:49.846 INFO:teuthology.orchestra.run.smithi099.stdout: automake-1.16.2-8.el9.noarch 2024-05-03T18:09:49.846 INFO:teuthology.orchestra.run.smithi099.stdout: bison-3.7.4-5.el9.x86_64 2024-05-03T18:09:49.846 INFO:teuthology.orchestra.run.smithi099.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-03T18:09:49.846 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:09:49.846 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-base-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:09:49.846 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-common-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:09:49.846 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-fuse-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:09:49.846 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-grafana-dashboards-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T18:09:49.846 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-immutable-object-cache-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:09:49.847 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mds-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:09:49.847 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:09:49.847 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-cephadm-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T18:09:49.847 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-dashboard-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T18:09:49.847 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T18:09:49.847 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-modules-core-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T18:09:49.847 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-rook-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T18:09:49.847 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mon-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:09:49.847 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-osd-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:09:49.847 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-prometheus-alerts-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T18:09:49.847 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-radosgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:09:49.847 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-selinux-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:09:49.847 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-test-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:09:49.847 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-volume-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T18:09:49.847 INFO:teuthology.orchestra.run.smithi099.stdout: cephadm-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T18:09:49.848 INFO:teuthology.orchestra.run.smithi099.stdout: cephfs-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:09:49.848 INFO:teuthology.orchestra.run.smithi099.stdout: cephfs-shell-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:09:49.848 INFO:teuthology.orchestra.run.smithi099.stdout: cephfs-top-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T18:09:49.848 INFO:teuthology.orchestra.run.smithi099.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-03T18:09:49.848 INFO:teuthology.orchestra.run.smithi099.stdout: flex-2.6.4-9.el9.x86_64 2024-05-03T18:09:49.848 INFO:teuthology.orchestra.run.smithi099.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-03T18:09:49.848 INFO:teuthology.orchestra.run.smithi099.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-03T18:09:49.848 INFO:teuthology.orchestra.run.smithi099.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-03T18:09:49.848 INFO:teuthology.orchestra.run.smithi099.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-03T18:09:49.848 INFO:teuthology.orchestra.run.smithi099.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-03T18:09:49.848 INFO:teuthology.orchestra.run.smithi099.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-03T18:09:49.848 INFO:teuthology.orchestra.run.smithi099.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-03T18:09:49.848 INFO:teuthology.orchestra.run.smithi099.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-03T18:09:49.848 INFO:teuthology.orchestra.run.smithi099.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-03T18:09:49.849 INFO:teuthology.orchestra.run.smithi099.stdout: libcephfs-devel-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:09:49.849 INFO:teuthology.orchestra.run.smithi099.stdout: libcephfs2-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:09:49.849 INFO:teuthology.orchestra.run.smithi099.stdout: libcephsqlite-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:09:49.849 INFO:teuthology.orchestra.run.smithi099.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-03T18:09:49.849 INFO:teuthology.orchestra.run.smithi099.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-03T18:09:49.849 INFO:teuthology.orchestra.run.smithi099.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-03T18:09:49.849 INFO:teuthology.orchestra.run.smithi099.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-03T18:09:49.849 INFO:teuthology.orchestra.run.smithi099.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-03T18:09:49.849 INFO:teuthology.orchestra.run.smithi099.stdout: librados-devel-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:09:49.849 INFO:teuthology.orchestra.run.smithi099.stdout: libradosstriper1-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:09:49.849 INFO:teuthology.orchestra.run.smithi099.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-03T18:09:49.849 INFO:teuthology.orchestra.run.smithi099.stdout: librgw2-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:09:49.849 INFO:teuthology.orchestra.run.smithi099.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-03T18:09:49.849 INFO:teuthology.orchestra.run.smithi099.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-03T18:09:49.850 INFO:teuthology.orchestra.run.smithi099.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-03T18:09:49.850 INFO:teuthology.orchestra.run.smithi099.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-03T18:09:49.850 INFO:teuthology.orchestra.run.smithi099.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-03T18:09:49.850 INFO:teuthology.orchestra.run.smithi099.stdout: lua-5.4.4-4.el9.x86_64 2024-05-03T18:09:49.850 INFO:teuthology.orchestra.run.smithi099.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-03T18:09:49.850 INFO:teuthology.orchestra.run.smithi099.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-03T18:09:49.850 INFO:teuthology.orchestra.run.smithi099.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-03T18:09:49.850 INFO:teuthology.orchestra.run.smithi099.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-03T18:09:49.850 INFO:teuthology.orchestra.run.smithi099.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-03T18:09:49.850 INFO:teuthology.orchestra.run.smithi099.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-03T18:09:49.850 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql-13.14-1.el9.x86_64 2024-05-03T18:09:49.850 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-03T18:09:49.850 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-03T18:09:49.851 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-03T18:09:49.851 INFO:teuthology.orchestra.run.smithi099.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-03T18:09:49.851 INFO:teuthology.orchestra.run.smithi099.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-03T18:09:49.851 INFO:teuthology.orchestra.run.smithi099.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-03T18:09:49.851 INFO:teuthology.orchestra.run.smithi099.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-03T18:09:49.851 INFO:teuthology.orchestra.run.smithi099.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-03T18:09:49.851 INFO:teuthology.orchestra.run.smithi099.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-03T18:09:49.851 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-03T18:09:49.851 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ceph-argparse-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:09:49.851 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ceph-common-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:09:49.851 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cephfs-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:09:49.851 INFO:teuthology.orchestra.run.smithi099.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-03T18:09:49.852 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-03T18:09:49.852 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-03T18:09:49.852 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-03T18:09:49.852 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-03T18:09:49.852 INFO:teuthology.orchestra.run.smithi099.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-03T18:09:49.852 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-03T18:09:49.852 INFO:teuthology.orchestra.run.smithi099.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-03T18:09:49.852 INFO:teuthology.orchestra.run.smithi099.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-03T18:09:49.852 INFO:teuthology.orchestra.run.smithi099.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-03T18:09:49.852 INFO:teuthology.orchestra.run.smithi099.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-03T18:09:49.852 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-03T18:09:49.852 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-03T18:09:49.852 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-03T18:09:49.853 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-03T18:09:49.853 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-03T18:09:49.853 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-03T18:09:49.853 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-03T18:09:49.853 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-03T18:09:49.853 INFO:teuthology.orchestra.run.smithi099.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-03T18:09:49.853 INFO:teuthology.orchestra.run.smithi099.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-03T18:09:49.853 INFO:teuthology.orchestra.run.smithi099.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-03T18:09:49.853 INFO:teuthology.orchestra.run.smithi099.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-03T18:09:49.853 INFO:teuthology.orchestra.run.smithi099.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-03T18:09:49.853 INFO:teuthology.orchestra.run.smithi099.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-03T18:09:49.853 INFO:teuthology.orchestra.run.smithi099.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-03T18:09:49.853 INFO:teuthology.orchestra.run.smithi099.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-03T18:09:49.854 INFO:teuthology.orchestra.run.smithi099.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-03T18:09:49.854 INFO:teuthology.orchestra.run.smithi099.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-03T18:09:49.854 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-03T18:09:49.854 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ply-3.11-14.el9.noarch 2024-05-03T18:09:49.854 INFO:teuthology.orchestra.run.smithi099.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-03T18:09:49.854 INFO:teuthology.orchestra.run.smithi099.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-03T18:09:49.854 INFO:teuthology.orchestra.run.smithi099.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-03T18:09:49.854 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-03T18:09:49.854 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-03T18:09:49.855 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-03T18:09:49.855 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-03T18:09:49.855 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-03T18:09:49.855 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-03T18:09:49.855 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rados-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:09:49.855 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rbd-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:09:49.855 INFO:teuthology.orchestra.run.smithi099.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-03T18:09:49.856 INFO:teuthology.orchestra.run.smithi099.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-03T18:09:49.856 INFO:teuthology.orchestra.run.smithi099.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-03T18:09:49.856 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:09:49.856 INFO:teuthology.orchestra.run.smithi099.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-03T18:09:49.856 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-03T18:09:49.856 INFO:teuthology.orchestra.run.smithi099.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-03T18:09:49.856 INFO:teuthology.orchestra.run.smithi099.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-03T18:09:49.856 INFO:teuthology.orchestra.run.smithi099.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-03T18:09:49.856 INFO:teuthology.orchestra.run.smithi099.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-03T18:09:49.857 INFO:teuthology.orchestra.run.smithi099.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-03T18:09:49.857 INFO:teuthology.orchestra.run.smithi099.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-03T18:09:49.857 INFO:teuthology.orchestra.run.smithi099.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-03T18:09:49.857 INFO:teuthology.orchestra.run.smithi099.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-03T18:09:49.857 INFO:teuthology.orchestra.run.smithi099.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-03T18:09:49.857 INFO:teuthology.orchestra.run.smithi099.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-03T18:09:49.857 INFO:teuthology.orchestra.run.smithi099.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-03T18:09:49.857 INFO:teuthology.orchestra.run.smithi099.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-03T18:09:49.857 INFO:teuthology.orchestra.run.smithi099.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-03T18:09:49.857 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-fuse-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:09:49.857 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:09:49.857 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-nbd-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:09:49.857 INFO:teuthology.orchestra.run.smithi099.stdout: re2-1:20211101-3.el9.x86_64 2024-05-03T18:09:49.857 INFO:teuthology.orchestra.run.smithi099.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-03T18:09:49.858 INFO:teuthology.orchestra.run.smithi099.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-03T18:09:49.858 INFO:teuthology.orchestra.run.smithi099.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-03T18:09:49.858 INFO:teuthology.orchestra.run.smithi099.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-03T18:09:49.858 INFO:teuthology.orchestra.run.smithi099.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-03T18:09:49.858 INFO:teuthology.orchestra.run.smithi099.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-03T18:09:49.858 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-05-03T18:09:49.858 INFO:teuthology.orchestra.run.smithi099.stdout:Complete! 2024-05-03T18:09:50.473 DEBUG:teuthology.parallel:result is None 2024-05-03T18:09:50.474 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9a047fe7599a549042d06a3fb5b6813a203eabea 2024-05-03T18:09:50.620 DEBUG:teuthology.orchestra.run.smithi033:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-03T18:09:50.699 INFO:teuthology.orchestra.run.smithi033.stdout:19.0.0-3450.g9a047fe7.el9 2024-05-03T18:09:50.699 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3450.g9a047fe7.el9 2024-05-03T18:09:50.699 INFO:teuthology.task.install:The correct ceph version 19.0.0-3450.g9a047fe7 is installed. 2024-05-03T18:09:50.701 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9a047fe7599a549042d06a3fb5b6813a203eabea 2024-05-03T18:09:50.919 DEBUG:teuthology.orchestra.run.smithi099:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-03T18:09:50.973 INFO:teuthology.orchestra.run.smithi099.stdout:19.0.0-3450.g9a047fe7.el9 2024-05-03T18:09:50.973 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3450.g9a047fe7.el9 2024-05-03T18:09:50.973 INFO:teuthology.task.install:The correct ceph version 19.0.0-3450.g9a047fe7 is installed. 2024-05-03T18:09:50.975 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9a047fe7599a549042d06a3fb5b6813a203eabea 2024-05-03T18:09:51.121 DEBUG:teuthology.orchestra.run.smithi150:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-03T18:09:51.194 INFO:teuthology.orchestra.run.smithi150.stdout:19.0.0-3450.g9a047fe7.el9 2024-05-03T18:09:51.194 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3450.g9a047fe7.el9 2024-05-03T18:09:51.194 INFO:teuthology.task.install:The correct ceph version 19.0.0-3450.g9a047fe7 is installed. 2024-05-03T18:09:51.196 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-03T18:09:51.196 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-03T18:09:51.196 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-03T18:09:51.290 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-05-03T18:09:51.290 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-03T18:09:51.363 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-05-03T18:09:51.363 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-03T18:09:51.453 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-03T18:09:51.453 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-03T18:09:51.453 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/usr/bin/daemon-helper 2024-05-03T18:09:51.539 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-03T18:09:51.621 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-05-03T18:09:51.622 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/usr/bin/daemon-helper 2024-05-03T18:09:51.689 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-03T18:09:51.794 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-05-03T18:09:51.794 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/usr/bin/daemon-helper 2024-05-03T18:09:51.880 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-03T18:09:52.006 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-03T18:09:52.007 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-03T18:09:52.007 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-03T18:09:52.100 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-03T18:09:52.232 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-05-03T18:09:52.232 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-03T18:09:52.300 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-03T18:09:52.406 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-05-03T18:09:52.406 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-03T18:09:52.488 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-03T18:09:52.614 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-03T18:09:52.614 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-03T18:09:52.614 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/usr/bin/stdin-killer 2024-05-03T18:09:52.708 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-03T18:09:52.796 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-05-03T18:09:52.796 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/usr/bin/stdin-killer 2024-05-03T18:09:52.863 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-03T18:09:52.973 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-05-03T18:09:52.974 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/usr/bin/stdin-killer 2024-05-03T18:09:53.061 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-03T18:09:53.144 INFO:teuthology.run_tasks:Running task cephadm... 2024-05-03T18:09:53.235 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 1, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '9a047fe7599a549042d06a3fb5b6813a203eabea', 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-05-03T18:09:53.235 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea 2024-05-03T18:09:53.236 INFO:tasks.cephadm:Cluster fsid is 5703bba4-0978-11ef-bc97-c7b262605968 2024-05-03T18:09:53.236 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-05-03T18:09:53.236 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.33', 'mon.b': '172.21.15.99', 'mon.c': '172.21.15.150'} 2024-05-03T18:09:53.236 INFO:tasks.cephadm:First mon is mon.a on smithi033 2024-05-03T18:09:53.236 INFO:tasks.cephadm:First mgr is x 2024-05-03T18:09:53.236 INFO:tasks.cephadm:Normalizing hostnames... 2024-05-03T18:09:53.237 DEBUG:teuthology.orchestra.run.smithi033:> sudo hostname $(hostname -s) 2024-05-03T18:09:53.326 DEBUG:teuthology.orchestra.run.smithi099:> sudo hostname $(hostname -s) 2024-05-03T18:09:53.390 DEBUG:teuthology.orchestra.run.smithi150:> sudo hostname $(hostname -s) 2024-05-03T18:09:53.481 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-05-03T18:09:53.482 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9a047fe7599a549042d06a3fb5b6813a203eabea 2024-05-03T18:09:53.698 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '9a047fe7599a549042d06a3fb5b6813a203eabea', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/79034/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3450-g9a047fe7', 'node_name': '172.21.5.34+adami04', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3450.g9a047fe7'}, 'url': 'https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240501.193033/9a047fe7599a549042d06a3fb5b6813a203eabea/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-05-02 00:36:37.664792', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-rishabh-testing-20240501.193033', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/wip-rishabh-testing-20240501.193033/9a047fe7599a549042d06a3fb5b6813a203eabea/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-05-03T18:09:53.832 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref wip-rishabh-testing-20240501.193033, sha1 9a047fe7599a549042d06a3fb5b6813a203eabea from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=9a047fe7599a549042d06a3fb5b6813a203eabea 2024-05-03T18:09:53.833 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240501.193033/9a047fe7599a549042d06a3fb5b6813a203eabea/centos/9/x86_64/flavors/default/cephadm 2024-05-03T18:09:53.833 DEBUG:teuthology.orchestra.run.smithi033:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240501.193033/9a047fe7599a549042d06a3fb5b6813a203eabea/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-03T18:09:54.494 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-r--r--. 1 ubuntu ubuntu 782467 May 3 18:09 /home/ubuntu/cephtest/cephadm 2024-05-03T18:09:54.494 DEBUG:teuthology.orchestra.run.smithi099:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240501.193033/9a047fe7599a549042d06a3fb5b6813a203eabea/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-03T18:09:54.992 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r--. 1 ubuntu ubuntu 782467 May 3 18:09 /home/ubuntu/cephtest/cephadm 2024-05-03T18:09:54.993 DEBUG:teuthology.orchestra.run.smithi150:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240501.193033/9a047fe7599a549042d06a3fb5b6813a203eabea/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-03T18:09:55.477 INFO:teuthology.orchestra.run.smithi150.stdout:-rw-r--r--. 1 ubuntu ubuntu 782467 May 3 18:09 /home/ubuntu/cephtest/cephadm 2024-05-03T18:09:55.477 DEBUG:teuthology.orchestra.run.smithi033:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-03T18:09:55.539 DEBUG:teuthology.orchestra.run.smithi099:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-03T18:09:55.588 DEBUG:teuthology.orchestra.run.smithi150:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-03T18:09:55.661 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea on all hosts... 2024-05-03T18:09:55.661 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea pull 2024-05-03T18:09:55.669 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea pull 2024-05-03T18:09:55.674 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea pull 2024-05-03T18:09:57.680 INFO:teuthology.orchestra.run.smithi033.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea... 2024-05-03T18:09:57.719 INFO:teuthology.orchestra.run.smithi150.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea... 2024-05-03T18:09:57.837 INFO:teuthology.orchestra.run.smithi099.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea... 2024-05-03T18:10:34.589 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-05-03T18:10:34.589 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_version": "ceph version 19.0.0-3450-g9a047fe7 (9a047fe7599a549042d06a3fb5b6813a203eabea) squid (dev)", 2024-05-03T18:10:34.589 INFO:teuthology.orchestra.run.smithi150.stdout: "image_id": "ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012", 2024-05-03T18:10:34.589 INFO:teuthology.orchestra.run.smithi150.stdout: "repo_digests": [ 2024-05-03T18:10:34.589 INFO:teuthology.orchestra.run.smithi150.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd" 2024-05-03T18:10:34.589 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-05-03T18:10:34.589 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-05-03T18:10:37.624 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-05-03T18:10:37.625 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_version": "ceph version 19.0.0-3450-g9a047fe7 (9a047fe7599a549042d06a3fb5b6813a203eabea) squid (dev)", 2024-05-03T18:10:37.625 INFO:teuthology.orchestra.run.smithi099.stdout: "image_id": "ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012", 2024-05-03T18:10:37.625 INFO:teuthology.orchestra.run.smithi099.stdout: "repo_digests": [ 2024-05-03T18:10:37.625 INFO:teuthology.orchestra.run.smithi099.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd" 2024-05-03T18:10:37.625 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-05-03T18:10:37.625 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-05-03T18:10:47.648 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-03T18:10:47.648 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_version": "ceph version 19.0.0-3450-g9a047fe7 (9a047fe7599a549042d06a3fb5b6813a203eabea) squid (dev)", 2024-05-03T18:10:47.648 INFO:teuthology.orchestra.run.smithi033.stdout: "image_id": "ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012", 2024-05-03T18:10:47.648 INFO:teuthology.orchestra.run.smithi033.stdout: "repo_digests": [ 2024-05-03T18:10:47.648 INFO:teuthology.orchestra.run.smithi033.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd" 2024-05-03T18:10:47.648 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-03T18:10:47.648 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-03T18:10:47.693 DEBUG:teuthology.orchestra.run.smithi033:> sudo mkdir -p /etc/ceph 2024-05-03T18:10:47.782 DEBUG:teuthology.orchestra.run.smithi099:> sudo mkdir -p /etc/ceph 2024-05-03T18:10:47.865 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /etc/ceph 2024-05-03T18:10:47.942 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 777 /etc/ceph 2024-05-03T18:10:48.033 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 777 /etc/ceph 2024-05-03T18:10:48.117 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod 777 /etc/ceph 2024-05-03T18:10:48.205 INFO:tasks.cephadm:Writing seed config... 2024-05-03T18:10:48.206 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-05-03T18:10:48.206 INFO:tasks.cephadm: override: [client] debug client = 20 2024-05-03T18:10:48.206 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-05-03T18:10:48.206 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-05-03T18:10:48.206 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-05-03T18:10:48.207 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-05-03T18:10:48.207 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-05-03T18:10:48.207 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-05-03T18:10:48.207 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-05-03T18:10:48.207 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-05-03T18:10:48.207 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-05-03T18:10:48.207 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-05-03T18:10:48.207 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-05-03T18:10:48.207 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-05-03T18:10:48.207 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-05-03T18:10:48.207 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-05-03T18:10:48.207 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-05-03T18:10:48.207 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-05-03T18:10:48.207 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-05-03T18:10:48.207 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-05-03T18:10:48.208 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-05-03T18:10:48.208 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-05-03T18:10:48.208 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-05-03T18:10:48.208 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-05-03T18:10:48.208 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-05-03T18:10:48.208 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-05-03T18:10:48.208 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-05-03T18:10:48.208 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-05-03T18:10:48.208 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-05-03T18:10:48.208 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-05-03T18:10:48.208 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-05-03T18:10:48.208 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-05-03T18:10:48.208 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-05-03T18:10:48.208 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-05-03T18:10:48.208 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-05-03T18:10:48.209 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-05-03T18:10:48.209 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-05-03T18:10:48.209 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-05-03T18:10:48.209 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-05-03T18:10:48.209 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-05-03T18:10:48.209 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-05-03T18:10:48.209 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-05-03T18:10:48.209 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-05-03T18:10:48.209 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-05-03T18:10:48.210 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-03T18:10:48.210 DEBUG:teuthology.orchestra.run.smithi033:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-05-03T18:10:48.268 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 5703bba4-0978-11ef-bc97-c7b262605968 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-05-03T18:10:48.268 DEBUG:teuthology.orchestra.run.smithi033:mon.a> sudo journalctl -f -n 0 -u ceph-5703bba4-0978-11ef-bc97-c7b262605968@mon.a.service 2024-05-03T18:10:48.315 DEBUG:teuthology.orchestra.run.smithi033:mgr.x> sudo journalctl -f -n 0 -u ceph-5703bba4-0978-11ef-bc97-c7b262605968@mgr.x.service 2024-05-03T18:10:48.362 INFO:tasks.cephadm:Bootstrapping... 2024-05-03T18:10:48.363 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea -v bootstrap --fsid 5703bba4-0978-11ef-bc97-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.33 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-05-03T18:10:48.637 INFO:teuthology.orchestra.run.smithi033.stdout:-------------------------------------------------------------------------------- 2024-05-03T18:10:48.638 INFO:teuthology.orchestra.run.smithi033.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea', '-v', 'bootstrap', '--fsid', '5703bba4-0978-11ef-bc97-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.33', '--skip-admin-label'] 2024-05-03T18:10:48.638 INFO:teuthology.orchestra.run.smithi033.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-05-03T18:10:48.638 INFO:teuthology.orchestra.run.smithi033.stdout:Verifying podman|docker is present... 2024-05-03T18:10:48.699 INFO:teuthology.orchestra.run.smithi033.stdout:/bin/podman: stdout 4.9.4-dev 2024-05-03T18:10:48.699 INFO:teuthology.orchestra.run.smithi033.stdout:Verifying lvm2 is present... 2024-05-03T18:10:48.699 INFO:teuthology.orchestra.run.smithi033.stdout:Verifying time synchronization is in place... 2024-05-03T18:10:48.717 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-03T18:10:48.718 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-03T18:10:48.733 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-03T18:10:48.734 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout inactive 2024-05-03T18:10:48.751 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout enabled 2024-05-03T18:10:48.768 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout active 2024-05-03T18:10:48.768 INFO:teuthology.orchestra.run.smithi033.stdout:Unit chronyd.service is enabled and running 2024-05-03T18:10:48.768 INFO:teuthology.orchestra.run.smithi033.stdout:Repeating the final host check... 2024-05-03T18:10:48.822 INFO:teuthology.orchestra.run.smithi033.stdout:/bin/podman: stdout 4.9.4-dev 2024-05-03T18:10:48.822 INFO:teuthology.orchestra.run.smithi033.stdout:podman (/bin/podman) version 4.9.4 is present 2024-05-03T18:10:48.822 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl is present 2024-05-03T18:10:48.822 INFO:teuthology.orchestra.run.smithi033.stdout:lvcreate is present 2024-05-03T18:10:48.838 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-03T18:10:48.838 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-03T18:10:48.858 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-03T18:10:48.858 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout inactive 2024-05-03T18:10:48.884 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout enabled 2024-05-03T18:10:48.907 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout active 2024-05-03T18:10:48.907 INFO:teuthology.orchestra.run.smithi033.stdout:Unit chronyd.service is enabled and running 2024-05-03T18:10:48.907 INFO:teuthology.orchestra.run.smithi033.stdout:Host looks OK 2024-05-03T18:10:48.907 INFO:teuthology.orchestra.run.smithi033.stdout:Cluster fsid: 5703bba4-0978-11ef-bc97-c7b262605968 2024-05-03T18:10:48.908 INFO:teuthology.orchestra.run.smithi033.stdout:Acquiring lock 140111022120912 on /run/cephadm/5703bba4-0978-11ef-bc97-c7b262605968.lock 2024-05-03T18:10:48.908 INFO:teuthology.orchestra.run.smithi033.stdout:Lock 140111022120912 acquired on /run/cephadm/5703bba4-0978-11ef-bc97-c7b262605968.lock 2024-05-03T18:10:48.908 INFO:teuthology.orchestra.run.smithi033.stdout:Verifying IP 172.21.15.33 port 3300 ... 2024-05-03T18:10:48.909 INFO:teuthology.orchestra.run.smithi033.stdout:Verifying IP 172.21.15.33 port 6789 ... 2024-05-03T18:10:48.909 INFO:teuthology.orchestra.run.smithi033.stdout:Base mon IP(s) is [172.21.15.33:3300, 172.21.15.33:6789], mon addrv is [v2:172.21.15.33:3300,v1:172.21.15.33:6789] 2024-05-03T18:10:48.919 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.33 metric 100 2024-05-03T18:10:48.919 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.33 metric 100 2024-05-03T18:10:48.927 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-05-03T18:10:48.928 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-05-03T18:10:48.928 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-05-03T18:10:48.936 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-05-03T18:10:48.936 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-05-03T18:10:48.936 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-03T18:10:48.936 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-05-03T18:10:48.936 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/ip: stdout inet6 fe80::ae1f:6bff:fea5:d294/64 scope link noprefixroute 2024-05-03T18:10:48.936 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-03T18:10:48.937 INFO:teuthology.orchestra.run.smithi033.stdout:Mon IP `172.21.15.33` is in CIDR network `172.21.0.0/20` 2024-05-03T18:10:48.937 INFO:teuthology.orchestra.run.smithi033.stdout:Mon IP `172.21.15.33` is in CIDR network `172.21.0.0/20` 2024-05-03T18:10:48.938 INFO:teuthology.orchestra.run.smithi033.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-05-03T18:10:48.939 INFO:teuthology.orchestra.run.smithi033.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-05-03T18:10:48.940 INFO:teuthology.orchestra.run.smithi033.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea... 2024-05-03T18:10:49.662 INFO:teuthology.orchestra.run.smithi033.stdout:/bin/podman: stdout ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 2024-05-03T18:10:49.662 INFO:teuthology.orchestra.run.smithi033.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea... 2024-05-03T18:10:49.662 INFO:teuthology.orchestra.run.smithi033.stdout:/bin/podman: stderr Getting image source signatures 2024-05-03T18:10:49.662 INFO:teuthology.orchestra.run.smithi033.stdout:/bin/podman: stderr Copying blob sha256:8173fd91c96404ae0424bb08037c57749051dcbdfaeee9354a65e201dad7b0e2 2024-05-03T18:10:49.662 INFO:teuthology.orchestra.run.smithi033.stdout:/bin/podman: stderr Copying blob sha256:e2f2fd2adca92139da4b4b907fa90a6775d3eb4eddaa977c42ef37c66eea70bd 2024-05-03T18:10:49.662 INFO:teuthology.orchestra.run.smithi033.stdout:/bin/podman: stderr Copying config sha256:ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 2024-05-03T18:10:49.662 INFO:teuthology.orchestra.run.smithi033.stdout:/bin/podman: stderr Writing manifest to image destination 2024-05-03T18:10:50.997 INFO:teuthology.orchestra.run.smithi033.stdout:ceph: stdout ceph version 19.0.0-3450-g9a047fe7 (9a047fe7599a549042d06a3fb5b6813a203eabea) squid (dev) 2024-05-03T18:10:50.997 INFO:teuthology.orchestra.run.smithi033.stdout:Ceph version: ceph version 19.0.0-3450-g9a047fe7 (9a047fe7599a549042d06a3fb5b6813a203eabea) squid (dev) 2024-05-03T18:10:50.998 INFO:teuthology.orchestra.run.smithi033.stdout:Extracting ceph user uid/gid from container image... 2024-05-03T18:10:52.319 INFO:teuthology.orchestra.run.smithi033.stdout:stat: stdout 167 167 2024-05-03T18:10:52.320 INFO:teuthology.orchestra.run.smithi033.stdout:Creating initial keys... 2024-05-03T18:10:53.618 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph-authtool: stdout AQCsKDVm/570MxAAKxqJ2cfTtYMqDz6DFo63Jw== 2024-05-03T18:10:54.984 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph-authtool: stdout AQCuKDVmUKw9CBAAra1FeUBJQhbL7TJsdr/HxQ== 2024-05-03T18:10:56.147 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph-authtool: stdout AQCvKDVmWz/lGhAAFL/BkpGQOPfxQDKAjuJzLg== 2024-05-03T18:10:56.149 INFO:teuthology.orchestra.run.smithi033.stdout:Creating initial monmap... 2024-05-03T18:10:57.318 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-03T18:10:57.318 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-05-03T18:10:57.318 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 5703bba4-0978-11ef-bc97-c7b262605968 2024-05-03T18:10:57.319 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-03T18:10:57.319 INFO:teuthology.orchestra.run.smithi033.stdout:monmaptool for a [v2:172.21.15.33:3300,v1:172.21.15.33:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-03T18:10:57.319 INFO:teuthology.orchestra.run.smithi033.stdout:setting min_mon_release = quincy 2024-05-03T18:10:57.319 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/monmaptool: set fsid to 5703bba4-0978-11ef-bc97-c7b262605968 2024-05-03T18:10:57.319 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-03T18:10:57.319 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:10:57.319 INFO:teuthology.orchestra.run.smithi033.stdout:Creating mon... 2024-05-03T18:10:58.540 INFO:teuthology.orchestra.run.smithi033.stdout:create mon.a on 2024-05-03T18:10:58.906 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-05-03T18:10:59.153 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-05-03T18:10:59.445 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-5703bba4-0978-11ef-bc97-c7b262605968.target → /etc/systemd/system/ceph-5703bba4-0978-11ef-bc97-c7b262605968.target. 2024-05-03T18:10:59.445 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-5703bba4-0978-11ef-bc97-c7b262605968.target → /etc/systemd/system/ceph-5703bba4-0978-11ef-bc97-c7b262605968.target. 2024-05-03T18:10:59.839 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-5703bba4-0978-11ef-bc97-c7b262605968@mon.a 2024-05-03T18:10:59.839 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stderr Failed to reset failed state of unit ceph-5703bba4-0978-11ef-bc97-c7b262605968@mon.a.service: Unit ceph-5703bba4-0978-11ef-bc97-c7b262605968@mon.a.service not loaded. 2024-05-03T18:11:00.143 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-5703bba4-0978-11ef-bc97-c7b262605968.target.wants/ceph-5703bba4-0978-11ef-bc97-c7b262605968@mon.a.service → /etc/systemd/system/ceph-5703bba4-0978-11ef-bc97-c7b262605968@.service. 2024-05-03T18:11:00.809 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:00 smithi033 podman[27942]: 2024-05-03 18:11:00.564388741 +0000 UTC m=+0.145928576 container create 11ef764cbc21aedb439cef333999b5388e6089ed3af9eef696a2740983aecd4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mon-a, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-03T18:11:00.809 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:00 smithi033 podman[27942]: 2024-05-03 18:11:00.809567587 +0000 UTC m=+0.391107495 container init 11ef764cbc21aedb439cef333999b5388e6089ed3af9eef696a2740983aecd4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mon-a, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240430, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-05-03T18:11:00.931 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-03T18:11:00.931 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout disabled 2024-05-03T18:11:00.951 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-03T18:11:00.951 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout inactive 2024-05-03T18:11:00.951 INFO:teuthology.orchestra.run.smithi033.stdout:firewalld.service is not enabled 2024-05-03T18:11:00.951 INFO:teuthology.orchestra.run.smithi033.stdout:Not possible to enable service . firewalld.service is not available 2024-05-03T18:11:00.952 INFO:teuthology.orchestra.run.smithi033.stdout:Waiting for mon to start... 2024-05-03T18:11:00.952 INFO:teuthology.orchestra.run.smithi033.stdout:Waiting for mon... 2024-05-03T18:11:01.101 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:00 smithi033 podman[27942]: 2024-05-03 18:11:00.82092563 +0000 UTC m=+0.402465501 container start 11ef764cbc21aedb439cef333999b5388e6089ed3af9eef696a2740983aecd4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mon-a, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430) 2024-05-03T18:11:01.101 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:00 smithi033 bash[27942]: 11ef764cbc21aedb439cef333999b5388e6089ed3af9eef696a2740983aecd4d 2024-05-03T18:11:01.101 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:00 smithi033 ceph-mon[27960]: mkfs 5703bba4-0978-11ef-bc97-c7b262605968 2024-05-03T18:11:01.101 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:00 smithi033 ceph-mon[27960]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-03T18:11:01.101 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:00 smithi033 systemd[1]: Started Ceph mon.a for 5703bba4-0978-11ef-bc97-c7b262605968. 2024-05-03T18:11:02.215 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:01 smithi033 ceph-mon[27960]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-03T18:11:02.216 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:01 smithi033 ceph-mon[27960]: monmap epoch 1 2024-05-03T18:11:02.216 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:01 smithi033 ceph-mon[27960]: fsid 5703bba4-0978-11ef-bc97-c7b262605968 2024-05-03T18:11:02.216 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:01 smithi033 ceph-mon[27960]: last_changed 2024-05-03T18:10:56.604483+0000 2024-05-03T18:11:02.216 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:01 smithi033 ceph-mon[27960]: created 2024-05-03T18:10:56.604483+0000 2024-05-03T18:11:02.216 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:01 smithi033 ceph-mon[27960]: min_mon_release 19 (squid) 2024-05-03T18:11:02.216 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:01 smithi033 ceph-mon[27960]: election_strategy: 1 2024-05-03T18:11:02.217 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:01 smithi033 ceph-mon[27960]: 0: [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] mon.a 2024-05-03T18:11:02.217 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:01 smithi033 ceph-mon[27960]: fsmap 2024-05-03T18:11:02.217 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:01 smithi033 ceph-mon[27960]: osdmap e1: 0 total, 0 up, 0 in 2024-05-03T18:11:02.217 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:01 smithi033 ceph-mon[27960]: mgrmap e1: no daemons active 2024-05-03T18:11:02.217 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:01 smithi033 ceph-mon[27960]: from='client.? 172.21.15.33:0/1239215289' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-05-03T18:11:02.612 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout cluster: 2024-05-03T18:11:02.618 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout id: 5703bba4-0978-11ef-bc97-c7b262605968 2024-05-03T18:11:02.618 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-05-03T18:11:02.619 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-05-03T18:11:02.619 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout services: 2024-05-03T18:11:02.619 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.839309s) 2024-05-03T18:11:02.619 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-05-03T18:11:02.619 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-05-03T18:11:02.619 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-05-03T18:11:02.619 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout data: 2024-05-03T18:11:02.619 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-05-03T18:11:02.619 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-05-03T18:11:02.619 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-05-03T18:11:02.619 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout pgs: 2024-05-03T18:11:02.619 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-05-03T18:11:02.619 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:01.643+0000 7f8381e00700 1 Processor -- start 2024-05-03T18:11:02.620 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:01.643+0000 7f8381e00700 1 -- start start 2024-05-03T18:11:02.620 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:01.644+0000 7f8381e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f837c108c00 0x7f837c109020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:11:02.620 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:01.644+0000 7f8381e00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f837c1095f0 con 0x7f837c108c00 2024-05-03T18:11:02.620 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:01.645+0000 7f837aa00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f837c108c00 0x7f837c109020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:11:02.620 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:01.645+0000 7f837aa00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f837c108c00 0x7f837c109020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38418/0 (socket says 172.21.15.33:38418) 2024-05-03T18:11:02.620 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:01.645+0000 7f837aa00700 1 -- 172.21.15.33:0/3183067557 learned_addr learned my addr 172.21.15.33:0/3183067557 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:11:02.620 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:01.645+0000 7f837aa00700 1 -- 172.21.15.33:0/3183067557 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f837c109e00 con 0x7f837c108c00 2024-05-03T18:11:02.620 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:01.645+0000 7f837aa00700 1 --2- 172.21.15.33:0/3183067557 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f837c108c00 0x7f837c109020 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f836c0098a0 tx=0x7f836c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c9f569dcf6307b server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:11:02.620 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:01.646+0000 7f8379600700 1 -- 172.21.15.33:0/3183067557 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f836c004160 con 0x7f837c108c00 2024-05-03T18:11:02.620 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:01.646+0000 7f8379600700 1 -- 172.21.15.33:0/3183067557 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f836c004300 con 0x7f837c108c00 2024-05-03T18:11:02.620 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:01.646+0000 7f8379600700 1 -- 172.21.15.33:0/3183067557 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f836c0036a0 con 0x7f837c108c00 2024-05-03T18:11:02.620 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:01.646+0000 7f8381e00700 1 -- 172.21.15.33:0/3183067557 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f837c108c00 msgr2=0x7f837c109020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:11:02.620 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:01.646+0000 7f8381e00700 1 --2- 172.21.15.33:0/3183067557 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f837c108c00 0x7f837c109020 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f836c0098a0 tx=0x7f836c009bb0 comp rx=0 tx=0).stop 2024-05-03T18:11:02.620 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:01.646+0000 7f8381e00700 1 -- 172.21.15.33:0/3183067557 shutdown_connections 2024-05-03T18:11:02.621 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:01.646+0000 7f8381e00700 1 --2- 172.21.15.33:0/3183067557 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f837c108c00 0x7f837c109020 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:02.621 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:01.647+0000 7f8381e00700 1 -- 172.21.15.33:0/3183067557 >> 172.21.15.33:0/3183067557 conn(0x7f837c104200 msgr2=0x7f837c106620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:11:02.621 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:01.647+0000 7f8381e00700 1 -- 172.21.15.33:0/3183067557 shutdown_connections 2024-05-03T18:11:02.621 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:01.647+0000 7f8381e00700 1 -- 172.21.15.33:0/3183067557 wait complete. 2024-05-03T18:11:02.621 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:01.648+0000 7f8381e00700 1 Processor -- start 2024-05-03T18:11:02.621 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:01.648+0000 7f8381e00700 1 -- start start 2024-05-03T18:11:02.621 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:01.649+0000 7f8381e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f837c108c00 0x7f837c07eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:11:02.621 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:01.649+0000 7f8381e00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f837c10a330 con 0x7f837c108c00 2024-05-03T18:11:02.621 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:01.649+0000 7f837aa00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f837c108c00 0x7f837c07eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:11:02.621 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:01.649+0000 7f837aa00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f837c108c00 0x7f837c07eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38430/0 (socket says 172.21.15.33:38430) 2024-05-03T18:11:02.621 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:01.649+0000 7f837aa00700 1 -- 172.21.15.33:0/1239215289 learned_addr learned my addr 172.21.15.33:0/1239215289 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:11:02.621 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:01.650+0000 7f837aa00700 1 -- 172.21.15.33:0/1239215289 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f837c07f050 con 0x7f837c108c00 2024-05-03T18:11:02.621 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:01.650+0000 7f837aa00700 1 --2- 172.21.15.33:0/1239215289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f837c108c00 0x7f837c07eb10 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f836c000c00 tx=0x7f836c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:11:02.622 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:01.650+0000 7f8380a00700 1 -- 172.21.15.33:0/1239215289 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f836c003db0 con 0x7f837c108c00 2024-05-03T18:11:02.622 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:01.650+0000 7f8380a00700 1 -- 172.21.15.33:0/1239215289 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f836c011630 con 0x7f837c108c00 2024-05-03T18:11:02.622 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:01.650+0000 7f8380a00700 1 -- 172.21.15.33:0/1239215289 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f836c011950 con 0x7f837c108c00 2024-05-03T18:11:02.622 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:01.650+0000 7f8381e00700 1 -- 172.21.15.33:0/1239215289 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f837c07d240 con 0x7f837c108c00 2024-05-03T18:11:02.622 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:01.651+0000 7f8381e00700 1 -- 172.21.15.33:0/1239215289 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f837c07d780 con 0x7f837c108c00 2024-05-03T18:11:02.622 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:01.651+0000 7f8380a00700 1 -- 172.21.15.33:0/1239215289 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f836c01a460 con 0x7f837c108c00 2024-05-03T18:11:02.622 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:01.651+0000 7f8380a00700 1 -- 172.21.15.33:0/1239215289 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f836c01b780 con 0x7f837c108c00 2024-05-03T18:11:02.622 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:01.652+0000 7f8381e00700 1 -- 172.21.15.33:0/1239215289 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f837c064b00 con 0x7f837c108c00 2024-05-03T18:11:02.622 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:01.655+0000 7f8380a00700 1 -- 172.21.15.33:0/1239215289 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f836c031020 con 0x7f837c108c00 2024-05-03T18:11:02.622 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:01.735+0000 7f8381e00700 1 -- 172.21.15.33:0/1239215289 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f837c195c70 con 0x7f837c108c00 2024-05-03T18:11:02.622 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:01.736+0000 7f8380a00700 1 -- 172.21.15.33:0/1239215289 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f836c01b9d0 con 0x7f837c108c00 2024-05-03T18:11:02.622 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:01.738+0000 7f8381e00700 1 -- 172.21.15.33:0/1239215289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f837c108c00 msgr2=0x7f837c07eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:11:02.622 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:01.738+0000 7f8381e00700 1 --2- 172.21.15.33:0/1239215289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f837c108c00 0x7f837c07eb10 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f836c000c00 tx=0x7f836c004750 comp rx=0 tx=0).stop 2024-05-03T18:11:02.623 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:01.738+0000 7f8381e00700 1 -- 172.21.15.33:0/1239215289 shutdown_connections 2024-05-03T18:11:02.623 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:01.738+0000 7f8381e00700 1 --2- 172.21.15.33:0/1239215289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f837c108c00 0x7f837c07eb10 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:02.623 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:01.738+0000 7f8381e00700 1 -- 172.21.15.33:0/1239215289 >> 172.21.15.33:0/1239215289 conn(0x7f837c104200 msgr2=0x7f837c192fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:11:02.623 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:01.738+0000 7f8381e00700 1 -- 172.21.15.33:0/1239215289 shutdown_connections 2024-05-03T18:11:02.623 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:01.738+0000 7f8381e00700 1 -- 172.21.15.33:0/1239215289 wait complete. 2024-05-03T18:11:02.623 INFO:teuthology.orchestra.run.smithi033.stdout:mon is available 2024-05-03T18:11:02.623 INFO:teuthology.orchestra.run.smithi033.stdout:Assimilating anything we can from ceph.conf... 2024-05-03T18:11:03.482 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:03 smithi033 ceph-mon[27960]: from='client.? 172.21.15.33:0/3925261654' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-03T18:11:03.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:03 smithi033 ceph-mon[27960]: from='client.? 172.21.15.33:0/3925261654' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-05-03T18:11:03.927 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-05-03T18:11:03.927 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout [global] 2024-05-03T18:11:03.927 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout fsid = 5703bba4-0978-11ef-bc97-c7b262605968 2024-05-03T18:11:03.927 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-05-03T18:11:03.927 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.33:3300,v1:172.21.15.33:6789] 2024-05-03T18:11:03.927 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-03T18:11:03.928 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-03T18:11:03.928 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-03T18:11:03.929 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-03T18:11:03.929 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-05-03T18:11:03.929 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-03T18:11:03.929 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-03T18:11:03.929 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-05-03T18:11:03.929 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout [osd] 2024-05-03T18:11:03.929 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-03T18:11:03.930 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-03T18:11:03.930 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-03T18:11:03.930 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-03T18:11:03.930 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-03T18:11:03.930 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:03.209+0000 7fdf96000700 1 Processor -- start 2024-05-03T18:11:03.931 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:03.210+0000 7fdf96000700 1 -- start start 2024-05-03T18:11:03.931 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:03.210+0000 7fdf96000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf90108a20 0x7fdf90108e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:11:03.931 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:03.210+0000 7fdf96000700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fdf90109410 con 0x7fdf90108a20 2024-05-03T18:11:03.931 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:03.211+0000 7fdf94c00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf90108a20 0x7fdf90108e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:11:03.932 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:03.211+0000 7fdf94c00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf90108a20 0x7fdf90108e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38446/0 (socket says 172.21.15.33:38446) 2024-05-03T18:11:03.932 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:03.211+0000 7fdf94c00700 1 -- 172.21.15.33:0/2546609954 learned_addr learned my addr 172.21.15.33:0/2546609954 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:11:03.933 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:03.212+0000 7fdf94c00700 1 -- 172.21.15.33:0/2546609954 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf90109c20 con 0x7fdf90108a20 2024-05-03T18:11:03.933 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:03.212+0000 7fdf94c00700 1 --2- 172.21.15.33:0/2546609954 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf90108a20 0x7fdf90108e40 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fdf840098a0 tx=0x7fdf84009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0fa6a7dcce75aa server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:11:03.933 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:03.212+0000 7fdf8f400700 1 -- 172.21.15.33:0/2546609954 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf84004160 con 0x7fdf90108a20 2024-05-03T18:11:03.933 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:03.212+0000 7fdf8f400700 1 -- 172.21.15.33:0/2546609954 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf84004300 con 0x7fdf90108a20 2024-05-03T18:11:03.933 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:03.212+0000 7fdf8f400700 1 -- 172.21.15.33:0/2546609954 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf840036a0 con 0x7fdf90108a20 2024-05-03T18:11:03.933 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:03.213+0000 7fdf96000700 1 -- 172.21.15.33:0/2546609954 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf90108a20 msgr2=0x7fdf90108e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:11:03.933 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:03.213+0000 7fdf96000700 1 --2- 172.21.15.33:0/2546609954 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf90108a20 0x7fdf90108e40 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fdf840098a0 tx=0x7fdf84009bb0 comp rx=0 tx=0).stop 2024-05-03T18:11:03.934 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:03.213+0000 7fdf96000700 1 -- 172.21.15.33:0/2546609954 shutdown_connections 2024-05-03T18:11:03.934 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:03.213+0000 7fdf96000700 1 --2- 172.21.15.33:0/2546609954 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf90108a20 0x7fdf90108e40 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:03.935 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:03.213+0000 7fdf96000700 1 -- 172.21.15.33:0/2546609954 >> 172.21.15.33:0/2546609954 conn(0x7fdf90103fe0 msgr2=0x7fdf90106400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:11:03.935 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:03.213+0000 7fdf96000700 1 -- 172.21.15.33:0/2546609954 shutdown_connections 2024-05-03T18:11:03.935 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:03.214+0000 7fdf96000700 1 -- 172.21.15.33:0/2546609954 wait complete. 2024-05-03T18:11:03.935 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:03.215+0000 7fdf96000700 1 Processor -- start 2024-05-03T18:11:03.935 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:03.215+0000 7fdf96000700 1 -- start start 2024-05-03T18:11:03.936 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:03.215+0000 7fdf96000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf90108a20 0x7fdf9007eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:11:03.936 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:03.215+0000 7fdf96000700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9010a150 con 0x7fdf90108a20 2024-05-03T18:11:03.936 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:03.215+0000 7fdf94c00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf90108a20 0x7fdf9007eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:11:03.936 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:03.215+0000 7fdf94c00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf90108a20 0x7fdf9007eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38450/0 (socket says 172.21.15.33:38450) 2024-05-03T18:11:03.936 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:03.215+0000 7fdf94c00700 1 -- 172.21.15.33:0/3925261654 learned_addr learned my addr 172.21.15.33:0/3925261654 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:11:03.936 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:03.216+0000 7fdf94c00700 1 -- 172.21.15.33:0/3925261654 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf9007f050 con 0x7fdf90108a20 2024-05-03T18:11:03.937 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:03.216+0000 7fdf94c00700 1 --2- 172.21.15.33:0/3925261654 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf90108a20 0x7fdf9007eb10 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fdf84000c00 tx=0x7fdf84004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:11:03.937 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:03.216+0000 7fdf8d600700 1 -- 172.21.15.33:0/3925261654 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf84003db0 con 0x7fdf90108a20 2024-05-03T18:11:03.937 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:03.216+0000 7fdf8d600700 1 -- 172.21.15.33:0/3925261654 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf84011630 con 0x7fdf90108a20 2024-05-03T18:11:03.937 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:03.216+0000 7fdf8d600700 1 -- 172.21.15.33:0/3925261654 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf84011950 con 0x7fdf90108a20 2024-05-03T18:11:03.937 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:03.216+0000 7fdf96000700 1 -- 172.21.15.33:0/3925261654 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf9007d240 con 0x7fdf90108a20 2024-05-03T18:11:03.938 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:03.217+0000 7fdf96000700 1 -- 172.21.15.33:0/3925261654 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf9007d780 con 0x7fdf90108a20 2024-05-03T18:11:03.938 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:03.217+0000 7fdf8d600700 1 -- 172.21.15.33:0/3925261654 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fdf84011af0 con 0x7fdf90108a20 2024-05-03T18:11:03.938 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:03.217+0000 7fdf8d600700 1 -- 172.21.15.33:0/3925261654 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fdf8401c450 con 0x7fdf90108a20 2024-05-03T18:11:03.938 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:03.218+0000 7fdf96000700 1 -- 172.21.15.33:0/3925261654 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf90064b00 con 0x7fdf90108a20 2024-05-03T18:11:03.939 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:03.221+0000 7fdf8d600700 1 -- 172.21.15.33:0/3925261654 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fdf84032020 con 0x7fdf90108a20 2024-05-03T18:11:03.939 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:03.298+0000 7fdf96000700 1 -- 172.21.15.33:0/3925261654 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fdf901959d0 con 0x7fdf90108a20 2024-05-03T18:11:03.940 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:03.306+0000 7fdf8d600700 1 -- 172.21.15.33:0/3925261654 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+465 (secure 0 0 0) 0x7fdf84004260 con 0x7fdf90108a20 2024-05-03T18:11:03.940 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:03.308+0000 7fdf96000700 1 -- 172.21.15.33:0/3925261654 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf90108a20 msgr2=0x7fdf9007eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:11:03.940 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:03.308+0000 7fdf96000700 1 --2- 172.21.15.33:0/3925261654 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf90108a20 0x7fdf9007eb10 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fdf84000c00 tx=0x7fdf84004750 comp rx=0 tx=0).stop 2024-05-03T18:11:03.940 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:03.308+0000 7fdf96000700 1 -- 172.21.15.33:0/3925261654 shutdown_connections 2024-05-03T18:11:03.940 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:03.308+0000 7fdf96000700 1 --2- 172.21.15.33:0/3925261654 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf90108a20 0x7fdf9007eb10 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:03.940 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:03.308+0000 7fdf96000700 1 -- 172.21.15.33:0/3925261654 >> 172.21.15.33:0/3925261654 conn(0x7fdf90103fe0 msgr2=0x7fdf90192dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:11:03.941 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:03.308+0000 7fdf96000700 1 -- 172.21.15.33:0/3925261654 shutdown_connections 2024-05-03T18:11:03.941 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:03.308+0000 7fdf96000700 1 -- 172.21.15.33:0/3925261654 wait complete. 2024-05-03T18:11:03.941 INFO:teuthology.orchestra.run.smithi033.stdout:Generating new minimal ceph.conf... 2024-05-03T18:11:04.944 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:04 smithi033 ceph-mon[27960]: from='client.? 172.21.15.33:0/695353785' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:11:05.266 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:04.553+0000 7fcf9dc00700 1 Processor -- start 2024-05-03T18:11:05.266 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:04.553+0000 7fcf9dc00700 1 -- start start 2024-05-03T18:11:05.266 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:04.553+0000 7fcf9dc00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf9810fc70 0x7fcf98112060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:11:05.266 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:04.553+0000 7fcf9dc00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcf98083480 con 0x7fcf9810fc70 2024-05-03T18:11:05.266 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:04.554+0000 7fcf96a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf9810fc70 0x7fcf98112060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:11:05.267 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:04.554+0000 7fcf96a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf9810fc70 0x7fcf98112060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38460/0 (socket says 172.21.15.33:38460) 2024-05-03T18:11:05.267 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:04.554+0000 7fcf96a00700 1 -- 172.21.15.33:0/1781170254 learned_addr learned my addr 172.21.15.33:0/1781170254 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:11:05.267 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:04.554+0000 7fcf96a00700 1 -- 172.21.15.33:0/1781170254 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf981125a0 con 0x7fcf9810fc70 2024-05-03T18:11:05.267 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:04.555+0000 7fcf96a00700 1 --2- 172.21.15.33:0/1781170254 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf9810fc70 0x7fcf98112060 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fcf880098a0 tx=0x7fcf88009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3407b54bfc0104a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:11:05.267 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:04.555+0000 7fcf95600700 1 -- 172.21.15.33:0/1781170254 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcf88004160 con 0x7fcf9810fc70 2024-05-03T18:11:05.267 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:04.555+0000 7fcf95600700 1 -- 172.21.15.33:0/1781170254 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fcf88004300 con 0x7fcf9810fc70 2024-05-03T18:11:05.267 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:04.555+0000 7fcf95600700 1 -- 172.21.15.33:0/1781170254 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcf880119e0 con 0x7fcf9810fc70 2024-05-03T18:11:05.267 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:04.556+0000 7fcf9dc00700 1 -- 172.21.15.33:0/1781170254 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf9810fc70 msgr2=0x7fcf98112060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:11:05.267 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:04.556+0000 7fcf9dc00700 1 --2- 172.21.15.33:0/1781170254 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf9810fc70 0x7fcf98112060 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fcf880098a0 tx=0x7fcf88009bb0 comp rx=0 tx=0).stop 2024-05-03T18:11:05.267 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:04.556+0000 7fcf9dc00700 1 -- 172.21.15.33:0/1781170254 shutdown_connections 2024-05-03T18:11:05.267 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:04.556+0000 7fcf9dc00700 1 --2- 172.21.15.33:0/1781170254 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf9810fc70 0x7fcf98112060 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:05.267 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:04.556+0000 7fcf9dc00700 1 -- 172.21.15.33:0/1781170254 >> 172.21.15.33:0/1781170254 conn(0x7fcf9807e1e0 msgr2=0x7fcf98080620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:11:05.267 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:04.556+0000 7fcf9dc00700 1 -- 172.21.15.33:0/1781170254 shutdown_connections 2024-05-03T18:11:05.267 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:04.556+0000 7fcf9dc00700 1 -- 172.21.15.33:0/1781170254 wait complete. 2024-05-03T18:11:05.268 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:04.557+0000 7fcf9dc00700 1 Processor -- start 2024-05-03T18:11:05.268 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:04.557+0000 7fcf9dc00700 1 -- start start 2024-05-03T18:11:05.268 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:04.558+0000 7fcf9dc00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf9810fc70 0x7fcf981a3410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:11:05.268 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:04.558+0000 7fcf9dc00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcf98113560 con 0x7fcf9810fc70 2024-05-03T18:11:05.268 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:04.558+0000 7fcf96a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf9810fc70 0x7fcf981a3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:11:05.268 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:04.558+0000 7fcf96a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf9810fc70 0x7fcf981a3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38476/0 (socket says 172.21.15.33:38476) 2024-05-03T18:11:05.268 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:04.558+0000 7fcf96a00700 1 -- 172.21.15.33:0/695353785 learned_addr learned my addr 172.21.15.33:0/695353785 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:11:05.268 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:04.558+0000 7fcf96a00700 1 -- 172.21.15.33:0/695353785 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf981a3950 con 0x7fcf9810fc70 2024-05-03T18:11:05.268 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:04.559+0000 7fcf96a00700 1 --2- 172.21.15.33:0/695353785 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf9810fc70 0x7fcf981a3410 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fcf880116a0 tx=0x7fcf880046a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:11:05.268 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:04.559+0000 7fcf8f400700 1 -- 172.21.15.33:0/695353785 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcf88003b70 con 0x7fcf9810fc70 2024-05-03T18:11:05.268 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:04.559+0000 7fcf8f400700 1 -- 172.21.15.33:0/695353785 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fcf8801a8a0 con 0x7fcf9810fc70 2024-05-03T18:11:05.268 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:04.559+0000 7fcf9dc00700 1 -- 172.21.15.33:0/695353785 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf981a3be0 con 0x7fcf9810fc70 2024-05-03T18:11:05.268 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:04.559+0000 7fcf8f400700 1 -- 172.21.15.33:0/695353785 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcf88024890 con 0x7fcf9810fc70 2024-05-03T18:11:05.268 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:04.559+0000 7fcf9dc00700 1 -- 172.21.15.33:0/695353785 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf981a6880 con 0x7fcf9810fc70 2024-05-03T18:11:05.269 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:04.560+0000 7fcf8f400700 1 -- 172.21.15.33:0/695353785 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fcf88022020 con 0x7fcf9810fc70 2024-05-03T18:11:05.269 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:04.560+0000 7fcf8f400700 1 -- 172.21.15.33:0/695353785 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fcf880193c0 con 0x7fcf9810fc70 2024-05-03T18:11:05.269 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:04.561+0000 7fcf9dc00700 1 -- 172.21.15.33:0/695353785 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf98064b00 con 0x7fcf9810fc70 2024-05-03T18:11:05.269 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:04.563+0000 7fcf8f400700 1 -- 172.21.15.33:0/695353785 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fcf8801aa50 con 0x7fcf9810fc70 2024-05-03T18:11:05.269 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:04.642+0000 7fcf9dc00700 1 -- 172.21.15.33:0/695353785 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fcf9802e510 con 0x7fcf9810fc70 2024-05-03T18:11:05.269 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:04.643+0000 7fcf8f400700 1 -- 172.21.15.33:0/695353785 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7fcf8801a430 con 0x7fcf9810fc70 2024-05-03T18:11:05.269 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:04.645+0000 7fcf9dc00700 1 -- 172.21.15.33:0/695353785 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf9810fc70 msgr2=0x7fcf981a3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:11:05.269 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:04.645+0000 7fcf9dc00700 1 --2- 172.21.15.33:0/695353785 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf9810fc70 0x7fcf981a3410 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fcf880116a0 tx=0x7fcf880046a0 comp rx=0 tx=0).stop 2024-05-03T18:11:05.269 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:04.645+0000 7fcf9dc00700 1 -- 172.21.15.33:0/695353785 shutdown_connections 2024-05-03T18:11:05.269 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:04.645+0000 7fcf9dc00700 1 --2- 172.21.15.33:0/695353785 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf9810fc70 0x7fcf981a3410 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:05.269 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:04.645+0000 7fcf9dc00700 1 -- 172.21.15.33:0/695353785 >> 172.21.15.33:0/695353785 conn(0x7fcf9807e1e0 msgr2=0x7fcf9807eed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:11:05.269 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:04.645+0000 7fcf9dc00700 1 -- 172.21.15.33:0/695353785 shutdown_connections 2024-05-03T18:11:05.270 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:04.645+0000 7fcf9dc00700 1 -- 172.21.15.33:0/695353785 wait complete. 2024-05-03T18:11:05.270 INFO:teuthology.orchestra.run.smithi033.stdout:Restarting the monitor... 2024-05-03T18:11:05.732 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:05 smithi033 systemd[1]: Stopping Ceph mon.a for 5703bba4-0978-11ef-bc97-c7b262605968... 2024-05-03T18:11:09.233 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:08 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mon-a[27956]: 2024-05-03T18:11:08.749+0000 7fbce1200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T18:11:09.233 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:08 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mon-a[27956]: 2024-05-03T18:11:08.749+0000 7fbce1200700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-05-03T18:11:09.233 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:08 smithi033 podman[28203]: 2024-05-03 18:11:08.966193032 +0000 UTC m=+3.276925070 container died 11ef764cbc21aedb439cef333999b5388e6089ed3af9eef696a2740983aecd4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mon-a, ceph=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-05-03T18:11:15.317 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:15 smithi033 podman[28203]: 2024-05-03 18:11:15.133910408 +0000 UTC m=+9.444642504 container cleanup 11ef764cbc21aedb439cef333999b5388e6089ed3af9eef696a2740983aecd4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mon-a, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-05-03T18:11:15.317 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:15 smithi033 bash[28203]: ceph-5703bba4-0978-11ef-bc97-c7b262605968-mon-a 2024-05-03T18:11:15.732 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:15 smithi033 podman[28217]: 2024-05-03 18:11:15.318031075 +0000 UTC m=+6.336109403 container remove 11ef764cbc21aedb439cef333999b5388e6089ed3af9eef696a2740983aecd4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mon-a, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-03T18:11:16.232 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:15 smithi033 systemd[1]: ceph-5703bba4-0978-11ef-bc97-c7b262605968@mon.a.service: Deactivated successfully. 2024-05-03T18:11:16.232 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:15 smithi033 systemd[1]: Stopped Ceph mon.a for 5703bba4-0978-11ef-bc97-c7b262605968. 2024-05-03T18:11:16.232 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:15 smithi033 systemd[1]: ceph-5703bba4-0978-11ef-bc97-c7b262605968@mon.a.service: Consumed 2.152s CPU time. 2024-05-03T18:11:16.232 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:15 smithi033 systemd[1]: Starting Ceph mon.a for 5703bba4-0978-11ef-bc97-c7b262605968... 2024-05-03T18:11:16.554 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 podman[28310]: 2024-05-03 18:11:16.202526312 +0000 UTC m=+0.033164772 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea 2024-05-03T18:11:16.555 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 podman[28310]: 2024-05-03 18:11:16.327123457 +0000 UTC m=+0.157761912 container create b6180ce9a1aba09fc279a712a70e56313100bb29eaf8e9bc94fb031b5a595714 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mon-a, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe) 2024-05-03T18:11:16.555 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 podman[28310]: 2024-05-03 18:11:16.555620178 +0000 UTC m=+0.386258702 container init b6180ce9a1aba09fc279a712a70e56313100bb29eaf8e9bc94fb031b5a595714 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mon-a, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-03T18:11:16.664 INFO:teuthology.orchestra.run.smithi033.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-05-03T18:11:16.814 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 podman[28310]: 2024-05-03 18:11:16.567472153 +0000 UTC m=+0.398110643 container start b6180ce9a1aba09fc279a712a70e56313100bb29eaf8e9bc94fb031b5a595714 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mon-a, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS) 2024-05-03T18:11:16.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: set uid:gid to 167:167 (ceph:ceph) 2024-05-03T18:11:16.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: ceph version 19.0.0-3450-g9a047fe7 (9a047fe7599a549042d06a3fb5b6813a203eabea) squid (dev), process ceph-mon, pid 2 2024-05-03T18:11:16.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: pidfile_write: ignore empty --pid-file 2024-05-03T18:11:16.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: load: jerasure load: lrc 2024-05-03T18:11:16.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: RocksDB version: 7.9.2 2024-05-03T18:11:16.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Git sha 0 2024-05-03T18:11:16.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Compile date 2024-05-01 21:12:03 2024-05-03T18:11:16.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: DB SUMMARY 2024-05-03T18:11:16.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: DB Session ID: C59I57SQG62X5BH2BDVP 2024-05-03T18:11:16.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: CURRENT file: CURRENT 2024-05-03T18:11:16.816 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: IDENTITY file: IDENTITY 2024-05-03T18:11:16.816 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-05-03T18:11:16.816 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-05-03T18:11:16.816 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 103428 ; 2024-05-03T18:11:16.816 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.error_if_exists: 0 2024-05-03T18:11:16.816 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.create_if_missing: 0 2024-05-03T18:11:16.816 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.paranoid_checks: 1 2024-05-03T18:11:16.816 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-03T18:11:16.816 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-03T18:11:16.816 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-03T18:11:16.816 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.env: 0x56061546fcc0 2024-05-03T18:11:16.816 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.fs: PosixFileSystem 2024-05-03T18:11:16.816 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.info_log: 0x56062ef7fde0 2024-05-03T18:11:16.816 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.max_file_opening_threads: 16 2024-05-03T18:11:16.816 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.statistics: (nil) 2024-05-03T18:11:16.817 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.use_fsync: 0 2024-05-03T18:11:16.817 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.max_log_file_size: 0 2024-05-03T18:11:16.817 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-03T18:11:16.817 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-03T18:11:16.817 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.keep_log_file_num: 1000 2024-05-03T18:11:16.817 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.recycle_log_file_num: 0 2024-05-03T18:11:16.817 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.allow_fallocate: 1 2024-05-03T18:11:16.817 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.allow_mmap_reads: 0 2024-05-03T18:11:16.817 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.allow_mmap_writes: 0 2024-05-03T18:11:16.818 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.use_direct_reads: 0 2024-05-03T18:11:16.818 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-03T18:11:16.818 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.create_missing_column_families: 0 2024-05-03T18:11:16.818 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.db_log_dir: 2024-05-03T18:11:16.818 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.wal_dir: 2024-05-03T18:11:16.818 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-03T18:11:16.818 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-03T18:11:16.818 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-03T18:11:16.818 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-03T18:11:16.818 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-03T18:11:16.818 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-03T18:11:16.818 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.advise_random_on_open: 1 2024-05-03T18:11:16.818 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.db_write_buffer_size: 0 2024-05-03T18:11:16.819 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.write_buffer_manager: 0x56062ef510e0 2024-05-03T18:11:16.819 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-03T18:11:16.819 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-03T18:11:16.819 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-03T18:11:16.819 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.rate_limiter: (nil) 2024-05-03T18:11:16.819 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-03T18:11:16.819 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.wal_recovery_mode: 2 2024-05-03T18:11:16.819 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.enable_thread_tracking: 0 2024-05-03T18:11:16.819 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.enable_pipelined_write: 0 2024-05-03T18:11:16.819 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.unordered_write: 0 2024-05-03T18:11:16.819 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-03T18:11:16.819 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-03T18:11:16.819 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-03T18:11:16.820 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-03T18:11:16.820 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.row_cache: None 2024-05-03T18:11:16.820 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.wal_filter: None 2024-05-03T18:11:16.820 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-03T18:11:16.820 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.allow_ingest_behind: 0 2024-05-03T18:11:16.820 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.two_write_queues: 0 2024-05-03T18:11:16.820 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.manual_wal_flush: 0 2024-05-03T18:11:16.820 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.wal_compression: 0 2024-05-03T18:11:16.820 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.atomic_flush: 0 2024-05-03T18:11:16.820 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-03T18:11:16.820 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-03T18:11:16.820 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-03T18:11:16.821 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.log_readahead_size: 0 2024-05-03T18:11:16.821 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-03T18:11:16.821 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.best_efforts_recovery: 0 2024-05-03T18:11:16.821 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-03T18:11:16.821 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-03T18:11:16.821 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.allow_data_in_errors: 0 2024-05-03T18:11:16.821 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.db_host_id: __hostname__ 2024-05-03T18:11:16.821 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-03T18:11:16.821 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.max_background_jobs: 2 2024-05-03T18:11:16.821 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.max_background_compactions: -1 2024-05-03T18:11:16.821 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.max_subcompactions: 1 2024-05-03T18:11:16.822 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-03T18:11:16.822 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-03T18:11:16.822 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-03T18:11:16.822 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.max_total_wal_size: 0 2024-05-03T18:11:16.822 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-03T18:11:16.822 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-03T18:11:16.822 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-03T18:11:16.822 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-03T18:11:16.822 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.max_open_files: -1 2024-05-03T18:11:16.822 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.bytes_per_sync: 0 2024-05-03T18:11:16.822 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-03T18:11:16.822 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-03T18:11:16.822 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.compaction_readahead_size: 0 2024-05-03T18:11:16.822 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.max_background_flushes: -1 2024-05-03T18:11:16.823 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Compression algorithms supported: 2024-05-03T18:11:16.823 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-03T18:11:16.823 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: kZSTD supported: 0 2024-05-03T18:11:16.823 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: kXpressCompression supported: 0 2024-05-03T18:11:16.823 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: kLZ4HCCompression supported: 1 2024-05-03T18:11:16.823 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: kZlibCompression supported: 1 2024-05-03T18:11:16.823 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: kSnappyCompression supported: 1 2024-05-03T18:11:16.823 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: kLZ4Compression supported: 1 2024-05-03T18:11:16.823 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: kBZip2Compression supported: 0 2024-05-03T18:11:16.823 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-03T18:11:16.823 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-03T18:11:16.823 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-05-03T18:11:16.824 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-03T18:11:16.824 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-03T18:11:16.824 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.merge_operator: 2024-05-03T18:11:16.824 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.compaction_filter: None 2024-05-03T18:11:16.825 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.compaction_filter_factory: None 2024-05-03T18:11:16.825 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.sst_partitioner_factory: None 2024-05-03T18:11:16.825 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-03T18:11:16.825 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-03T18:11:16.825 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x56062ef7ffa0) 2024-05-03T18:11:16.825 INFO:journalctl@ceph.mon.a.smithi033.stdout: cache_index_and_filter_blocks: 1 2024-05-03T18:11:16.825 INFO:journalctl@ceph.mon.a.smithi033.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-03T18:11:16.825 INFO:journalctl@ceph.mon.a.smithi033.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-03T18:11:16.825 INFO:journalctl@ceph.mon.a.smithi033.stdout: pin_top_level_index_and_filter: 1 2024-05-03T18:11:16.825 INFO:journalctl@ceph.mon.a.smithi033.stdout: index_type: 0 2024-05-03T18:11:16.825 INFO:journalctl@ceph.mon.a.smithi033.stdout: data_block_index_type: 0 2024-05-03T18:11:16.825 INFO:journalctl@ceph.mon.a.smithi033.stdout: index_shortening: 1 2024-05-03T18:11:16.826 INFO:journalctl@ceph.mon.a.smithi033.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-03T18:11:16.826 INFO:journalctl@ceph.mon.a.smithi033.stdout: checksum: 4 2024-05-03T18:11:16.826 INFO:journalctl@ceph.mon.a.smithi033.stdout: no_block_cache: 0 2024-05-03T18:11:16.826 INFO:journalctl@ceph.mon.a.smithi033.stdout: block_cache: 0x56062ef831f0 2024-05-03T18:11:16.826 INFO:journalctl@ceph.mon.a.smithi033.stdout: block_cache_name: BinnedLRUCache 2024-05-03T18:11:16.826 INFO:journalctl@ceph.mon.a.smithi033.stdout: block_cache_options: 2024-05-03T18:11:16.826 INFO:journalctl@ceph.mon.a.smithi033.stdout: capacity : 536870912 2024-05-03T18:11:16.826 INFO:journalctl@ceph.mon.a.smithi033.stdout: num_shard_bits : 4 2024-05-03T18:11:16.826 INFO:journalctl@ceph.mon.a.smithi033.stdout: strict_capacity_limit : 0 2024-05-03T18:11:16.826 INFO:journalctl@ceph.mon.a.smithi033.stdout: high_pri_pool_ratio: 0.000 2024-05-03T18:11:16.826 INFO:journalctl@ceph.mon.a.smithi033.stdout: block_cache_compressed: (nil) 2024-05-03T18:11:16.826 INFO:journalctl@ceph.mon.a.smithi033.stdout: persistent_cache: (nil) 2024-05-03T18:11:16.827 INFO:journalctl@ceph.mon.a.smithi033.stdout: block_size: 4096 2024-05-03T18:11:16.827 INFO:journalctl@ceph.mon.a.smithi033.stdout: block_size_deviation: 10 2024-05-03T18:11:16.827 INFO:journalctl@ceph.mon.a.smithi033.stdout: block_restart_interval: 16 2024-05-03T18:11:16.827 INFO:journalctl@ceph.mon.a.smithi033.stdout: index_block_restart_interval: 1 2024-05-03T18:11:16.827 INFO:journalctl@ceph.mon.a.smithi033.stdout: metadata_block_size: 4096 2024-05-03T18:11:16.827 INFO:journalctl@ceph.mon.a.smithi033.stdout: partition_filters: 0 2024-05-03T18:11:16.827 INFO:journalctl@ceph.mon.a.smithi033.stdout: use_delta_encoding: 1 2024-05-03T18:11:16.827 INFO:journalctl@ceph.mon.a.smithi033.stdout: filter_policy: bloomfilter 2024-05-03T18:11:16.827 INFO:journalctl@ceph.mon.a.smithi033.stdout: whole_key_filtering: 1 2024-05-03T18:11:16.827 INFO:journalctl@ceph.mon.a.smithi033.stdout: verify_compression: 0 2024-05-03T18:11:16.827 INFO:journalctl@ceph.mon.a.smithi033.stdout: read_amp_bytes_per_bit: 0 2024-05-03T18:11:16.827 INFO:journalctl@ceph.mon.a.smithi033.stdout: format_version: 5 2024-05-03T18:11:16.827 INFO:journalctl@ceph.mon.a.smithi033.stdout: enable_index_compression: 1 2024-05-03T18:11:16.827 INFO:journalctl@ceph.mon.a.smithi033.stdout: block_align: 0 2024-05-03T18:11:16.828 INFO:journalctl@ceph.mon.a.smithi033.stdout: max_auto_readahead_size: 262144 2024-05-03T18:11:16.828 INFO:journalctl@ceph.mon.a.smithi033.stdout: prepopulate_block_cache: 0 2024-05-03T18:11:16.828 INFO:journalctl@ceph.mon.a.smithi033.stdout: initial_auto_readahead_size: 8192 2024-05-03T18:11:16.828 INFO:journalctl@ceph.mon.a.smithi033.stdout: num_file_reads_for_auto_readahead: 2 2024-05-03T18:11:16.828 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.write_buffer_size: 33554432 2024-05-03T18:11:16.828 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.max_write_buffer_number: 2 2024-05-03T18:11:16.828 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.compression: NoCompression 2024-05-03T18:11:16.828 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.bottommost_compression: Disabled 2024-05-03T18:11:16.828 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.prefix_extractor: nullptr 2024-05-03T18:11:16.828 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-03T18:11:16.828 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.num_levels: 7 2024-05-03T18:11:16.828 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-03T18:11:16.828 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-03T18:11:16.829 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-03T18:11:16.829 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-03T18:11:16.829 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-03T18:11:16.829 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-03T18:11:16.829 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-03T18:11:16.829 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-03T18:11:16.829 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-03T18:11:16.829 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-03T18:11:16.829 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-03T18:11:16.829 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-03T18:11:16.829 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-03T18:11:16.829 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.compression_opts.level: 32767 2024-05-03T18:11:16.829 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.compression_opts.strategy: 0 2024-05-03T18:11:16.829 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-03T18:11:16.830 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-03T18:11:16.830 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-03T18:11:16.830 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-03T18:11:16.830 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.compression_opts.enabled: false 2024-05-03T18:11:16.830 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-03T18:11:16.830 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-03T18:11:16.830 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-03T18:11:16.830 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-03T18:11:16.830 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.target_file_size_base: 67108864 2024-05-03T18:11:16.830 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-03T18:11:16.830 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-03T18:11:16.830 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-03T18:11:16.830 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-03T18:11:16.830 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-03T18:11:16.831 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-03T18:11:16.831 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-03T18:11:16.831 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-03T18:11:16.831 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-03T18:11:16.831 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-03T18:11:16.831 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-03T18:11:16.831 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-03T18:11:16.831 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-03T18:11:16.831 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-03T18:11:16.831 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.arena_block_size: 1048576 2024-05-03T18:11:16.831 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-03T18:11:16.831 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-03T18:11:16.831 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.disable_auto_compactions: 0 2024-05-03T18:11:16.832 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-03T18:11:16.832 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-03T18:11:16.832 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-03T18:11:16.832 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-03T18:11:16.832 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-03T18:11:16.832 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-03T18:11:16.832 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-03T18:11:16.832 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-03T18:11:16.832 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-03T18:11:16.832 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-03T18:11:16.832 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-05-03T18:11:16.832 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.inplace_update_support: 0 2024-05-03T18:11:16.832 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-03T18:11:16.832 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-03T18:11:16.833 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-03T18:11:16.833 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-03T18:11:16.833 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.bloom_locality: 0 2024-05-03T18:11:16.833 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.max_successive_merges: 0 2024-05-03T18:11:16.833 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-03T18:11:16.833 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.paranoid_file_checks: 0 2024-05-03T18:11:16.833 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.force_consistency_checks: 1 2024-05-03T18:11:16.833 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.report_bg_io_stats: 0 2024-05-03T18:11:16.833 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.ttl: 2592000 2024-05-03T18:11:16.833 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-03T18:11:16.833 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-03T18:11:16.833 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-03T18:11:16.833 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.enable_blob_files: false 2024-05-03T18:11:16.833 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.min_blob_size: 0 2024-05-03T18:11:16.834 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.blob_file_size: 268435456 2024-05-03T18:11:16.834 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-03T18:11:16.834 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-03T18:11:16.834 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-03T18:11:16.834 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-03T18:11:16.835 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-03T18:11:16.835 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.blob_file_starting_level: 0 2024-05-03T18:11:16.835 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-03T18:11:16.835 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-05-03T18:11:16.835 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-05-03T18:11:16.835 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 1bf94025-fead-4af2-95dd-a45b9f6fce84 2024-05-03T18:11:16.835 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714759876625280, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-05-03T18:11:16.835 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-05-03T18:11:16.835 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714759876627418, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 98719, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 320, "table_properties": {"data_size": 96672, "index_size": 303, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13762, "raw_average_key_size": 49, "raw_value_size": 89141, "raw_average_value_size": 319, "num_data_blocks": 12, "num_entries": 279, "num_filter_entries": 279, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714759876, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "1bf94025-fead-4af2-95dd-a45b9f6fce84", "db_session_id": "C59I57SQG62X5BH2BDVP", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-05-03T18:11:16.835 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714759876627624, "job": 1, "event": "recovery_finished"} 2024-05-03T18:11:16.836 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-05-03T18:11:16.836 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-03T18:11:16.836 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x56062f0a0000 2024-05-03T18:11:16.836 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: DB pointer 0x56062f08c000 2024-05-03T18:11:16.836 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-03T18:11:16.836 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-03T18:11:16.836 INFO:journalctl@ceph.mon.a.smithi033.stdout: ** DB Stats ** 2024-05-03T18:11:16.836 INFO:journalctl@ceph.mon.a.smithi033.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-03T18:11:16.836 INFO:journalctl@ceph.mon.a.smithi033.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-03T18:11:16.836 INFO:journalctl@ceph.mon.a.smithi033.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-03T18:11:16.836 INFO:journalctl@ceph.mon.a.smithi033.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-03T18:11:16.836 INFO:journalctl@ceph.mon.a.smithi033.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-03T18:11:16.837 INFO:journalctl@ceph.mon.a.smithi033.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-03T18:11:16.837 INFO:journalctl@ceph.mon.a.smithi033.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-03T18:11:16.837 INFO:journalctl@ceph.mon.a.smithi033.stdout: 2024-05-03T18:11:16.837 INFO:journalctl@ceph.mon.a.smithi033.stdout: ** Compaction Stats [default] ** 2024-05-03T18:11:16.837 INFO:journalctl@ceph.mon.a.smithi033.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-03T18:11:16.837 INFO:journalctl@ceph.mon.a.smithi033.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-03T18:11:16.837 INFO:journalctl@ceph.mon.a.smithi033.stdout: L0 2/0 98.27 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 66.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-03T18:11:16.837 INFO:journalctl@ceph.mon.a.smithi033.stdout: Sum 2/0 98.27 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 66.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-03T18:11:16.837 INFO:journalctl@ceph.mon.a.smithi033.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 66.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-03T18:11:16.837 INFO:journalctl@ceph.mon.a.smithi033.stdout: 2024-05-03T18:11:16.837 INFO:journalctl@ceph.mon.a.smithi033.stdout: ** Compaction Stats [default] ** 2024-05-03T18:11:16.837 INFO:journalctl@ceph.mon.a.smithi033.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-03T18:11:16.837 INFO:journalctl@ceph.mon.a.smithi033.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-03T18:11:16.837 INFO:journalctl@ceph.mon.a.smithi033.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 66.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-03T18:11:16.838 INFO:journalctl@ceph.mon.a.smithi033.stdout: 2024-05-03T18:11:16.838 INFO:journalctl@ceph.mon.a.smithi033.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-03T18:11:16.838 INFO:journalctl@ceph.mon.a.smithi033.stdout: 2024-05-03T18:11:16.838 INFO:journalctl@ceph.mon.a.smithi033.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-03T18:11:16.838 INFO:journalctl@ceph.mon.a.smithi033.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-03T18:11:16.838 INFO:journalctl@ceph.mon.a.smithi033.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-03T18:11:16.838 INFO:journalctl@ceph.mon.a.smithi033.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-03T18:11:16.838 INFO:journalctl@ceph.mon.a.smithi033.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-03T18:11:16.838 INFO:journalctl@ceph.mon.a.smithi033.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-03T18:11:16.838 INFO:journalctl@ceph.mon.a.smithi033.stdout: Cumulative compaction: 0.00 GB write, 13.82 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-03T18:11:16.838 INFO:journalctl@ceph.mon.a.smithi033.stdout: Interval compaction: 0.00 GB write, 13.82 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-03T18:11:16.838 INFO:journalctl@ceph.mon.a.smithi033.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-03T18:11:16.838 INFO:journalctl@ceph.mon.a.smithi033.stdout: Block cache BinnedLRUCache@0x56062ef831f0#2 capacity: 512.00 MB usage: 1.39 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.2e-05 secs_since: 0 2024-05-03T18:11:16.838 INFO:journalctl@ceph.mon.a.smithi033.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-05-03T18:11:16.839 INFO:journalctl@ceph.mon.a.smithi033.stdout: 2024-05-03T18:11:16.839 INFO:journalctl@ceph.mon.a.smithi033.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-03T18:11:16.839 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: starting mon.a rank 0 at public addrs [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] at bind addrs [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 5703bba4-0978-11ef-bc97-c7b262605968 2024-05-03T18:11:16.839 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: mon.a@-1(???) e1 preinit fsid 5703bba4-0978-11ef-bc97-c7b262605968 2024-05-03T18:11:16.839 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: mon.a@-1(???).mds e1 new map 2024-05-03T18:11:16.839 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: mon.a@-1(???).mds e1 print_map 2024-05-03T18:11:16.839 INFO:journalctl@ceph.mon.a.smithi033.stdout: e1 2024-05-03T18:11:16.839 INFO:journalctl@ceph.mon.a.smithi033.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-03T18:11:16.839 INFO:journalctl@ceph.mon.a.smithi033.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-03T18:11:16.839 INFO:journalctl@ceph.mon.a.smithi033.stdout: legacy client fscid: -1 2024-05-03T18:11:16.839 INFO:journalctl@ceph.mon.a.smithi033.stdout: 2024-05-03T18:11:16.839 INFO:journalctl@ceph.mon.a.smithi033.stdout: No filesystems configured 2024-05-03T18:11:16.840 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-03T18:11:16.840 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-03T18:11:16.840 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-03T18:11:16.840 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-03T18:11:16.840 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-05-03T18:11:16.840 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 bash[28310]: b6180ce9a1aba09fc279a712a70e56313100bb29eaf8e9bc94fb031b5a595714 2024-05-03T18:11:16.840 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 systemd[1]: Started Ceph mon.a for 5703bba4-0978-11ef-bc97-c7b262605968. 2024-05-03T18:11:16.840 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-03T18:11:16.840 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: monmap epoch 1 2024-05-03T18:11:16.841 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: fsid 5703bba4-0978-11ef-bc97-c7b262605968 2024-05-03T18:11:16.841 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: last_changed 2024-05-03T18:10:56.604483+0000 2024-05-03T18:11:16.841 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: created 2024-05-03T18:10:56.604483+0000 2024-05-03T18:11:16.841 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: min_mon_release 19 (squid) 2024-05-03T18:11:16.841 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: election_strategy: 1 2024-05-03T18:11:16.841 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: 0: [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] mon.a 2024-05-03T18:11:16.841 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: fsmap 2024-05-03T18:11:16.841 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: osdmap e1: 0 total, 0 up, 0 in 2024-05-03T18:11:16.841 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:16 smithi033 ceph-mon[28330]: mgrmap e1: no daemons active 2024-05-03T18:11:18.070 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:17.305+0000 7fa050e00700 1 Processor -- start 2024-05-03T18:11:18.071 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:17.305+0000 7fa050e00700 1 -- start start 2024-05-03T18:11:18.071 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:17.306+0000 7fa050e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa04c106eb0 0x7fa04c1072d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:11:18.072 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:17.306+0000 7fa050e00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa04c1078a0 con 0x7fa04c106eb0 2024-05-03T18:11:18.072 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:17.307+0000 7fa049e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa04c106eb0 0x7fa04c1072d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:11:18.072 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:17.307+0000 7fa049e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa04c106eb0 0x7fa04c1072d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35522/0 (socket says 172.21.15.33:35522) 2024-05-03T18:11:18.073 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:17.307+0000 7fa049e00700 1 -- 172.21.15.33:0/2606579455 learned_addr learned my addr 172.21.15.33:0/2606579455 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:11:18.073 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:17.307+0000 7fa049e00700 1 -- 172.21.15.33:0/2606579455 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa04c109e40 con 0x7fa04c106eb0 2024-05-03T18:11:18.073 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:17.308+0000 7fa049e00700 1 --2- 172.21.15.33:0/2606579455 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa04c106eb0 0x7fa04c1072d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fa03c0098a0 tx=0x7fa03c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=646523a6467bef35 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:11:18.073 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:17.308+0000 7fa049400700 1 -- 172.21.15.33:0/2606579455 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa03c004160 con 0x7fa04c106eb0 2024-05-03T18:11:18.074 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:17.308+0000 7fa049400700 1 -- 172.21.15.33:0/2606579455 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fa03c004300 con 0x7fa04c106eb0 2024-05-03T18:11:18.074 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:17.308+0000 7fa049400700 1 -- 172.21.15.33:0/2606579455 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa03c0119e0 con 0x7fa04c106eb0 2024-05-03T18:11:18.074 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:17.309+0000 7fa050e00700 1 -- 172.21.15.33:0/2606579455 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa04c106eb0 msgr2=0x7fa04c1072d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:11:18.074 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:17.309+0000 7fa050e00700 1 --2- 172.21.15.33:0/2606579455 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa04c106eb0 0x7fa04c1072d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fa03c0098a0 tx=0x7fa03c009bb0 comp rx=0 tx=0).stop 2024-05-03T18:11:18.074 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:17.309+0000 7fa050e00700 1 -- 172.21.15.33:0/2606579455 shutdown_connections 2024-05-03T18:11:18.074 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:17.309+0000 7fa050e00700 1 --2- 172.21.15.33:0/2606579455 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa04c106eb0 0x7fa04c1072d0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:18.075 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:17.309+0000 7fa050e00700 1 -- 172.21.15.33:0/2606579455 >> 172.21.15.33:0/2606579455 conn(0x7fa04c102840 msgr2=0x7fa04c104c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:11:18.075 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:17.309+0000 7fa050e00700 1 -- 172.21.15.33:0/2606579455 shutdown_connections 2024-05-03T18:11:18.075 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:17.309+0000 7fa050e00700 1 -- 172.21.15.33:0/2606579455 wait complete. 2024-05-03T18:11:18.076 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:17.310+0000 7fa050e00700 1 Processor -- start 2024-05-03T18:11:18.076 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:17.310+0000 7fa050e00700 1 -- start start 2024-05-03T18:11:18.076 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:17.311+0000 7fa050e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa04c106eb0 0x7fa04c19f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:11:18.077 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:17.311+0000 7fa050e00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa04c10d070 con 0x7fa04c106eb0 2024-05-03T18:11:18.077 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:17.311+0000 7fa049e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa04c106eb0 0x7fa04c19f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:11:18.077 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:17.311+0000 7fa049e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa04c106eb0 0x7fa04c19f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35524/0 (socket says 172.21.15.33:35524) 2024-05-03T18:11:18.078 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:17.311+0000 7fa049e00700 1 -- 172.21.15.33:0/256859015 learned_addr learned my addr 172.21.15.33:0/256859015 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:11:18.078 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:17.311+0000 7fa049e00700 1 -- 172.21.15.33:0/256859015 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa04c19f560 con 0x7fa04c106eb0 2024-05-03T18:11:18.078 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:17.312+0000 7fa049e00700 1 --2- 172.21.15.33:0/256859015 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa04c106eb0 0x7fa04c19f020 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fa03c006b20 tx=0x7fa03c011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:11:18.078 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:17.312+0000 7fa042a00700 1 -- 172.21.15.33:0/256859015 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa03c003a60 con 0x7fa04c106eb0 2024-05-03T18:11:18.078 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:17.312+0000 7fa042a00700 1 -- 172.21.15.33:0/256859015 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fa03c003c00 con 0x7fa04c106eb0 2024-05-03T18:11:18.078 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:17.312+0000 7fa042a00700 1 -- 172.21.15.33:0/256859015 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa03c024780 con 0x7fa04c106eb0 2024-05-03T18:11:18.078 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:17.312+0000 7fa050e00700 1 -- 172.21.15.33:0/256859015 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa04c19f7f0 con 0x7fa04c106eb0 2024-05-03T18:11:18.078 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:17.312+0000 7fa050e00700 1 -- 172.21.15.33:0/256859015 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa04c1a24e0 con 0x7fa04c106eb0 2024-05-03T18:11:18.078 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:17.313+0000 7fa042a00700 1 -- 172.21.15.33:0/256859015 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa03c022020 con 0x7fa04c106eb0 2024-05-03T18:11:18.081 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:17.313+0000 7fa042a00700 1 -- 172.21.15.33:0/256859015 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa03c02d9c0 con 0x7fa04c106eb0 2024-05-03T18:11:18.081 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:17.313+0000 7fa050e00700 1 -- 172.21.15.33:0/256859015 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa04c10d1f0 con 0x7fa04c106eb0 2024-05-03T18:11:18.081 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:17.317+0000 7fa042a00700 1 -- 172.21.15.33:0/256859015 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fa03c004300 con 0x7fa04c106eb0 2024-05-03T18:11:18.082 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:17.396+0000 7fa050e00700 1 -- 172.21.15.33:0/256859015 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7fa04c064b00 con 0x7fa04c106eb0 2024-05-03T18:11:18.082 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:17.400+0000 7fa042a00700 1 -- 172.21.15.33:0/256859015 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7fa03c02d450 con 0x7fa04c106eb0 2024-05-03T18:11:18.082 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:17.402+0000 7fa050e00700 1 -- 172.21.15.33:0/256859015 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa04c106eb0 msgr2=0x7fa04c19f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:11:18.082 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:17.402+0000 7fa050e00700 1 --2- 172.21.15.33:0/256859015 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa04c106eb0 0x7fa04c19f020 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fa03c006b20 tx=0x7fa03c011a60 comp rx=0 tx=0).stop 2024-05-03T18:11:18.082 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:17.402+0000 7fa050e00700 1 -- 172.21.15.33:0/256859015 shutdown_connections 2024-05-03T18:11:18.082 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:17.402+0000 7fa050e00700 1 --2- 172.21.15.33:0/256859015 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa04c106eb0 0x7fa04c19f020 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:18.082 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:17.402+0000 7fa050e00700 1 -- 172.21.15.33:0/256859015 >> 172.21.15.33:0/256859015 conn(0x7fa04c102840 msgr2=0x7fa04c103670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:11:18.082 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:17.402+0000 7fa050e00700 1 -- 172.21.15.33:0/256859015 shutdown_connections 2024-05-03T18:11:18.082 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:17.402+0000 7fa050e00700 1 -- 172.21.15.33:0/256859015 wait complete. 2024-05-03T18:11:18.082 INFO:teuthology.orchestra.run.smithi033.stdout:Wrote config to /etc/ceph/ceph.conf 2024-05-03T18:11:18.108 INFO:teuthology.orchestra.run.smithi033.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-05-03T18:11:18.109 INFO:teuthology.orchestra.run.smithi033.stdout:Creating mgr... 2024-05-03T18:11:18.110 INFO:teuthology.orchestra.run.smithi033.stdout:Verifying port 0.0.0.0:9283 ... 2024-05-03T18:11:18.112 INFO:teuthology.orchestra.run.smithi033.stdout:Verifying port 0.0.0.0:8765 ... 2024-05-03T18:11:18.556 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-5703bba4-0978-11ef-bc97-c7b262605968@mgr.x 2024-05-03T18:11:18.556 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stderr Failed to reset failed state of unit ceph-5703bba4-0978-11ef-bc97-c7b262605968@mgr.x.service: Unit ceph-5703bba4-0978-11ef-bc97-c7b262605968@mgr.x.service not loaded. 2024-05-03T18:11:18.732 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:18 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/256859015' entity='client.admin' 2024-05-03T18:11:18.826 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-5703bba4-0978-11ef-bc97-c7b262605968.target.wants/ceph-5703bba4-0978-11ef-bc97-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-5703bba4-0978-11ef-bc97-c7b262605968@.service. 2024-05-03T18:11:19.221 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:11:18 smithi033 systemd[1]: Starting Ceph mgr.x for 5703bba4-0978-11ef-bc97-c7b262605968... 2024-05-03T18:11:19.483 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:11:19 smithi033 podman[28557]: 2024-05-03 18:11:19.221824801 +0000 UTC m=+0.125527398 container create baf6dc704ff570c8ae8af8003fafac25f0d00e15950cade55fec01f17c100d56 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-03T18:11:19.483 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:11:19 smithi033 podman[28557]: 2024-05-03 18:11:19.133166521 +0000 UTC m=+0.036869077 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea 2024-05-03T18:11:19.483 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:11:19 smithi033 podman[28557]: 2024-05-03 18:11:19.483920482 +0000 UTC m=+0.387623081 container init baf6dc704ff570c8ae8af8003fafac25f0d00e15950cade55fec01f17c100d56 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-05-03T18:11:19.732 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:11:19 smithi033 podman[28557]: 2024-05-03 18:11:19.495665961 +0000 UTC m=+0.399368527 container start baf6dc704ff570c8ae8af8003fafac25f0d00e15950cade55fec01f17c100d56 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_CLEAN=True, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-05-03T18:11:19.884 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-03T18:11:19.884 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout disabled 2024-05-03T18:11:19.909 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-03T18:11:19.909 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout inactive 2024-05-03T18:11:19.909 INFO:teuthology.orchestra.run.smithi033.stdout:firewalld.service is not enabled 2024-05-03T18:11:19.910 INFO:teuthology.orchestra.run.smithi033.stdout:Not possible to enable service . firewalld.service is not available 2024-05-03T18:11:19.945 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-03T18:11:19.945 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout disabled 2024-05-03T18:11:19.971 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-03T18:11:19.972 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout inactive 2024-05-03T18:11:19.972 INFO:teuthology.orchestra.run.smithi033.stdout:firewalld.service is not enabled 2024-05-03T18:11:19.972 INFO:teuthology.orchestra.run.smithi033.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-05-03T18:11:19.972 INFO:teuthology.orchestra.run.smithi033.stdout:Waiting for mgr to start... 2024-05-03T18:11:19.973 INFO:teuthology.orchestra.run.smithi033.stdout:Waiting for mgr... 2024-05-03T18:11:20.134 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:11:19 smithi033 bash[28557]: baf6dc704ff570c8ae8af8003fafac25f0d00e15950cade55fec01f17c100d56 2024-05-03T18:11:20.135 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:11:19 smithi033 systemd[1]: Started Ceph mgr.x for 5703bba4-0978-11ef-bc97-c7b262605968. 2024-05-03T18:11:20.451 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:11:20 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:11:20.211+0000 7f07024ce200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-03T18:11:20.717 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:11:20 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:11:20.483+0000 7f07024ce200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-03T18:11:20.717 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:11:20 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:11:20.621+0000 7f07024ce200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-03T18:11:20.982 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:20 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/1527597456' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-03T18:11:21.342 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:11:21 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:11:21.049+0000 7f07024ce200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-03T18:11:21.342 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:11:21 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:11:21.184+0000 7f07024ce200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-03T18:11:21.717 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-05-03T18:11:21.720 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout { 2024-05-03T18:11:21.720 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "fsid": "5703bba4-0978-11ef-bc97-c7b262605968", 2024-05-03T18:11:21.720 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "health": { 2024-05-03T18:11:21.720 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-03T18:11:21.720 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-03T18:11:21.720 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-03T18:11:21.720 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-05-03T18:11:21.721 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-03T18:11:21.721 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-03T18:11:21.721 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 0 2024-05-03T18:11:21.721 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ], 2024-05-03T18:11:21.721 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-03T18:11:21.721 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "a" 2024-05-03T18:11:21.721 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ], 2024-05-03T18:11:21.721 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "quorum_age": 4, 2024-05-03T18:11:21.721 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-03T18:11:21.721 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-03T18:11:21.721 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-03T18:11:21.721 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-03T18:11:21.721 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-05-03T18:11:21.721 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-03T18:11:21.722 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-03T18:11:21.722 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-03T18:11:21.722 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-03T18:11:21.722 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-03T18:11:21.722 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-03T18:11:21.722 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-03T18:11:21.722 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-03T18:11:21.722 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-05-03T18:11:21.722 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-03T18:11:21.722 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-03T18:11:21.722 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-03T18:11:21.722 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-03T18:11:21.730 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-03T18:11:21.731 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-03T18:11:21.731 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-03T18:11:21.731 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-03T18:11:21.731 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-03T18:11:21.731 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-05-03T18:11:21.731 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-03T18:11:21.731 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-03T18:11:21.731 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-03T18:11:21.731 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-03T18:11:21.731 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-05-03T18:11:21.731 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-03T18:11:21.731 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-03T18:11:21.732 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-03T18:11:21.732 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-03T18:11:21.732 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-03T18:11:21.732 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-03T18:11:21.732 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "restful" 2024-05-03T18:11:21.732 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ], 2024-05-03T18:11:21.732 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-03T18:11:21.732 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-05-03T18:11:21.732 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-03T18:11:21.732 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-03T18:11:21.732 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "modified": "2024-05-03T18:11:00.900759+0000", 2024-05-03T18:11:21.732 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-03T18:11:21.732 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-05-03T18:11:21.732 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-03T18:11:21.733 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout } 2024-05-03T18:11:21.733 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:20.764+0000 7f5370200700 1 Processor -- start 2024-05-03T18:11:21.733 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:20.764+0000 7f5370200700 1 -- start start 2024-05-03T18:11:21.733 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:20.765+0000 7f5370200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5368108bf0 0x7f5368109010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:11:21.733 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:20.765+0000 7f5370200700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f53681095e0 con 0x7f5368108bf0 2024-05-03T18:11:21.733 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:20.765+0000 7f536d800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5368108bf0 0x7f5368109010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:11:21.733 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:20.766+0000 7f536d800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5368108bf0 0x7f5368109010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35542/0 (socket says 172.21.15.33:35542) 2024-05-03T18:11:21.733 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:20.766+0000 7f536d800700 1 -- 172.21.15.33:0/2751709042 learned_addr learned my addr 172.21.15.33:0/2751709042 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:11:21.733 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:20.766+0000 7f536d800700 1 -- 172.21.15.33:0/2751709042 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5368109da0 con 0x7f5368108bf0 2024-05-03T18:11:21.733 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:20.766+0000 7f536d800700 1 --2- 172.21.15.33:0/2751709042 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5368108bf0 0x7f5368109010 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f535c0098a0 tx=0x7f535c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=764a603740bdf820 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:11:21.733 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:20.767+0000 7f5367e00700 1 -- 172.21.15.33:0/2751709042 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f535c004160 con 0x7f5368108bf0 2024-05-03T18:11:21.733 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:20.767+0000 7f5367e00700 1 -- 172.21.15.33:0/2751709042 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f535c004300 con 0x7f5368108bf0 2024-05-03T18:11:21.733 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:20.767+0000 7f5367e00700 1 -- 172.21.15.33:0/2751709042 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f535c0119e0 con 0x7f5368108bf0 2024-05-03T18:11:21.734 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:20.767+0000 7f5370200700 1 -- 172.21.15.33:0/2751709042 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5368108bf0 msgr2=0x7f5368109010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:11:21.734 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:20.767+0000 7f5370200700 1 --2- 172.21.15.33:0/2751709042 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5368108bf0 0x7f5368109010 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f535c0098a0 tx=0x7f535c009bb0 comp rx=0 tx=0).stop 2024-05-03T18:11:21.734 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:20.768+0000 7f5370200700 1 -- 172.21.15.33:0/2751709042 shutdown_connections 2024-05-03T18:11:21.734 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:20.768+0000 7f5370200700 1 --2- 172.21.15.33:0/2751709042 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5368108bf0 0x7f5368109010 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:21.734 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:20.768+0000 7f5370200700 1 -- 172.21.15.33:0/2751709042 >> 172.21.15.33:0/2751709042 conn(0x7f53681041b0 msgr2=0x7f53681065d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:11:21.734 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:20.768+0000 7f5370200700 1 -- 172.21.15.33:0/2751709042 shutdown_connections 2024-05-03T18:11:21.734 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:20.768+0000 7f5370200700 1 -- 172.21.15.33:0/2751709042 wait complete. 2024-05-03T18:11:21.734 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:20.769+0000 7f5370200700 1 Processor -- start 2024-05-03T18:11:21.734 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:20.769+0000 7f5370200700 1 -- start start 2024-05-03T18:11:21.734 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:20.769+0000 7f5370200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5368108bf0 0x7f536819efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:11:21.734 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:20.769+0000 7f5370200700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f536810cfd0 con 0x7f5368108bf0 2024-05-03T18:11:21.734 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:20.769+0000 7f536d800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5368108bf0 0x7f536819efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:11:21.734 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:20.769+0000 7f536d800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5368108bf0 0x7f536819efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35554/0 (socket says 172.21.15.33:35554) 2024-05-03T18:11:21.735 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:20.769+0000 7f536d800700 1 -- 172.21.15.33:0/1527597456 learned_addr learned my addr 172.21.15.33:0/1527597456 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:11:21.735 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:20.770+0000 7f536d800700 1 -- 172.21.15.33:0/1527597456 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f536819f520 con 0x7f5368108bf0 2024-05-03T18:11:21.735 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:20.770+0000 7f536d800700 1 --2- 172.21.15.33:0/1527597456 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5368108bf0 0x7f536819efe0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f535c006b20 tx=0x7f535c011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:11:21.735 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:20.770+0000 7f5366000700 1 -- 172.21.15.33:0/1527597456 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f535c003a60 con 0x7f5368108bf0 2024-05-03T18:11:21.735 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:20.770+0000 7f5366000700 1 -- 172.21.15.33:0/1527597456 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f535c003c00 con 0x7f5368108bf0 2024-05-03T18:11:21.735 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:20.770+0000 7f5366000700 1 -- 172.21.15.33:0/1527597456 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f535c024780 con 0x7f5368108bf0 2024-05-03T18:11:21.735 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:20.770+0000 7f5370200700 1 -- 172.21.15.33:0/1527597456 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f536819f7b0 con 0x7f5368108bf0 2024-05-03T18:11:21.735 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:20.771+0000 7f5370200700 1 -- 172.21.15.33:0/1527597456 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53681a24a0 con 0x7f5368108bf0 2024-05-03T18:11:21.735 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:20.771+0000 7f5366000700 1 -- 172.21.15.33:0/1527597456 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f535c022020 con 0x7f5368108bf0 2024-05-03T18:11:21.735 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:20.771+0000 7f5366000700 1 -- 172.21.15.33:0/1527597456 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f535c02dde0 con 0x7f5368108bf0 2024-05-03T18:11:21.735 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:20.772+0000 7f5370200700 1 -- 172.21.15.33:0/1527597456 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5368064b00 con 0x7f5368108bf0 2024-05-03T18:11:21.735 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:20.775+0000 7f5366000700 1 -- 172.21.15.33:0/1527597456 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f535c004300 con 0x7f5368108bf0 2024-05-03T18:11:21.735 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:20.856+0000 7f5370200700 1 -- 172.21.15.33:0/1527597456 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f536802e510 con 0x7f5368108bf0 2024-05-03T18:11:21.735 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:20.857+0000 7f5366000700 1 -- 172.21.15.33:0/1527597456 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f535c01aba0 con 0x7f5368108bf0 2024-05-03T18:11:21.736 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:20.858+0000 7f5370200700 1 -- 172.21.15.33:0/1527597456 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5368108bf0 msgr2=0x7f536819efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:11:21.736 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:20.858+0000 7f5370200700 1 --2- 172.21.15.33:0/1527597456 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5368108bf0 0x7f536819efe0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f535c006b20 tx=0x7f535c011a60 comp rx=0 tx=0).stop 2024-05-03T18:11:21.736 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:20.858+0000 7f5370200700 1 -- 172.21.15.33:0/1527597456 shutdown_connections 2024-05-03T18:11:21.736 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:20.858+0000 7f5370200700 1 --2- 172.21.15.33:0/1527597456 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5368108bf0 0x7f536819efe0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:21.736 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:20.858+0000 7f5370200700 1 -- 172.21.15.33:0/1527597456 >> 172.21.15.33:0/1527597456 conn(0x7f53681041b0 msgr2=0x7f5368104ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:11:21.736 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:20.859+0000 7f5370200700 1 -- 172.21.15.33:0/1527597456 shutdown_connections 2024-05-03T18:11:21.736 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:20.859+0000 7f5370200700 1 -- 172.21.15.33:0/1527597456 wait complete. 2024-05-03T18:11:21.736 INFO:teuthology.orchestra.run.smithi033.stdout:mgr not available, waiting (1/15)... 2024-05-03T18:11:23.232 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:11:22 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:11:22.837+0000 7f07024ce200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-03T18:11:23.232 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:11:23 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:11:23.074+0000 7f07024ce200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-03T18:11:23.689 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:11:23 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:11:23.294+0000 7f07024ce200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-03T18:11:23.689 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:11:23 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:11:23.690+0000 7f07024ce200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-03T18:11:23.982 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:11:23 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:11:23.822+0000 7f07024ce200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-03T18:11:24.282 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:11:24 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:11:24.089+0000 7f07024ce200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-03T18:11:24.732 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:24 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/3765668413' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-03T18:11:24.732 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:11:24 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:11:24.399+0000 7f07024ce200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-03T18:11:25.150 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-05-03T18:11:25.150 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout { 2024-05-03T18:11:25.150 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "fsid": "5703bba4-0978-11ef-bc97-c7b262605968", 2024-05-03T18:11:25.150 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "health": { 2024-05-03T18:11:25.151 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-03T18:11:25.151 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-03T18:11:25.151 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-03T18:11:25.151 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-05-03T18:11:25.151 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-03T18:11:25.151 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-03T18:11:25.151 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 0 2024-05-03T18:11:25.151 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ], 2024-05-03T18:11:25.151 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-03T18:11:25.151 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "a" 2024-05-03T18:11:25.151 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ], 2024-05-03T18:11:25.151 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "quorum_age": 7, 2024-05-03T18:11:25.151 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-03T18:11:25.151 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-03T18:11:25.152 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-03T18:11:25.152 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-03T18:11:25.152 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-05-03T18:11:25.152 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-03T18:11:25.152 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-03T18:11:25.152 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-03T18:11:25.152 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-03T18:11:25.152 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-03T18:11:25.159 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-03T18:11:25.159 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-03T18:11:25.159 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-03T18:11:25.159 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-05-03T18:11:25.159 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-03T18:11:25.160 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-03T18:11:25.160 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-03T18:11:25.160 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-03T18:11:25.160 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-03T18:11:25.160 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-03T18:11:25.160 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-03T18:11:25.160 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-03T18:11:25.160 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-03T18:11:25.160 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-05-03T18:11:25.160 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-03T18:11:25.160 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-03T18:11:25.160 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-03T18:11:25.160 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-03T18:11:25.160 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-05-03T18:11:25.160 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-03T18:11:25.161 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-03T18:11:25.161 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-03T18:11:25.161 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-03T18:11:25.161 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-03T18:11:25.161 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-03T18:11:25.161 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "restful" 2024-05-03T18:11:25.161 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ], 2024-05-03T18:11:25.161 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-03T18:11:25.161 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-05-03T18:11:25.161 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-03T18:11:25.161 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-03T18:11:25.161 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "modified": "2024-05-03T18:11:00.900759+0000", 2024-05-03T18:11:25.161 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-03T18:11:25.161 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-05-03T18:11:25.162 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-03T18:11:25.162 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout } 2024-05-03T18:11:25.162 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:24.355+0000 7f8711a00700 1 Processor -- start 2024-05-03T18:11:25.162 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:24.356+0000 7f8711a00700 1 -- start start 2024-05-03T18:11:25.162 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:24.356+0000 7f8711a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f870c1096d0 0x7f870c10bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:11:25.162 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:24.356+0000 7f8711a00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f870c07cdf0 con 0x7f870c1096d0 2024-05-03T18:11:25.162 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:24.356+0000 7f870aa00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f870c1096d0 0x7f870c10bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:11:25.162 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:24.357+0000 7f870aa00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f870c1096d0 0x7f870c10bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35558/0 (socket says 172.21.15.33:35558) 2024-05-03T18:11:25.162 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:24.357+0000 7f870aa00700 1 -- 172.21.15.33:0/3099839567 learned_addr learned my addr 172.21.15.33:0/3099839567 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:11:25.162 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:24.357+0000 7f870aa00700 1 -- 172.21.15.33:0/3099839567 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f870c10c000 con 0x7f870c1096d0 2024-05-03T18:11:25.162 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:24.357+0000 7f870aa00700 1 --2- 172.21.15.33:0/3099839567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f870c1096d0 0x7f870c10bac0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f86fc0098a0 tx=0x7f86fc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faa67c08a59c48d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:11:25.162 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:24.357+0000 7f8709600700 1 -- 172.21.15.33:0/3099839567 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f86fc004160 con 0x7f870c1096d0 2024-05-03T18:11:25.162 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:24.358+0000 7f8709600700 1 -- 172.21.15.33:0/3099839567 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f86fc004300 con 0x7f870c1096d0 2024-05-03T18:11:25.162 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:24.358+0000 7f8709600700 1 -- 172.21.15.33:0/3099839567 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f86fc0119e0 con 0x7f870c1096d0 2024-05-03T18:11:25.163 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:24.358+0000 7f8711a00700 1 -- 172.21.15.33:0/3099839567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f870c1096d0 msgr2=0x7f870c10bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:11:25.163 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:24.358+0000 7f8711a00700 1 --2- 172.21.15.33:0/3099839567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f870c1096d0 0x7f870c10bac0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f86fc0098a0 tx=0x7f86fc009bb0 comp rx=0 tx=0).stop 2024-05-03T18:11:25.163 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:24.358+0000 7f8711a00700 1 -- 172.21.15.33:0/3099839567 shutdown_connections 2024-05-03T18:11:25.163 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:24.358+0000 7f8711a00700 1 --2- 172.21.15.33:0/3099839567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f870c1096d0 0x7f870c10bac0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:25.163 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:24.358+0000 7f8711a00700 1 -- 172.21.15.33:0/3099839567 >> 172.21.15.33:0/3099839567 conn(0x7f870c1030c0 msgr2=0x7f870c1054e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:11:25.163 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:24.358+0000 7f8711a00700 1 -- 172.21.15.33:0/3099839567 shutdown_connections 2024-05-03T18:11:25.163 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:24.358+0000 7f8711a00700 1 -- 172.21.15.33:0/3099839567 wait complete. 2024-05-03T18:11:25.163 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:24.359+0000 7f8711a00700 1 Processor -- start 2024-05-03T18:11:25.163 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:24.360+0000 7f8711a00700 1 -- start start 2024-05-03T18:11:25.163 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:24.360+0000 7f8711a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f870c1096d0 0x7f870c19cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:11:25.163 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:24.360+0000 7f8711a00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f870c10cf10 con 0x7f870c1096d0 2024-05-03T18:11:25.163 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:24.360+0000 7f870aa00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f870c1096d0 0x7f870c19cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:11:25.163 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:24.360+0000 7f870aa00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f870c1096d0 0x7f870c19cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35564/0 (socket says 172.21.15.33:35564) 2024-05-03T18:11:25.163 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:24.360+0000 7f870aa00700 1 -- 172.21.15.33:0/3765668413 learned_addr learned my addr 172.21.15.33:0/3765668413 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:11:25.164 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:24.361+0000 7f870aa00700 1 -- 172.21.15.33:0/3765668413 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f870c19d2e0 con 0x7f870c1096d0 2024-05-03T18:11:25.164 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:24.361+0000 7f870aa00700 1 --2- 172.21.15.33:0/3765668413 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f870c1096d0 0x7f870c19cda0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f86fc0116a0 tx=0x7f86fc0046a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:11:25.164 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:24.361+0000 7f8703400700 1 -- 172.21.15.33:0/3765668413 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f86fc003b70 con 0x7f870c1096d0 2024-05-03T18:11:25.164 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:24.361+0000 7f8703400700 1 -- 172.21.15.33:0/3765668413 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f86fc01a8a0 con 0x7f870c1096d0 2024-05-03T18:11:25.164 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:24.361+0000 7f8711a00700 1 -- 172.21.15.33:0/3765668413 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f870c19d570 con 0x7f870c1096d0 2024-05-03T18:11:25.164 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:24.361+0000 7f8703400700 1 -- 172.21.15.33:0/3765668413 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f86fc024890 con 0x7f870c1096d0 2024-05-03T18:11:25.164 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:24.361+0000 7f8711a00700 1 -- 172.21.15.33:0/3765668413 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f870c1a0260 con 0x7f870c1096d0 2024-05-03T18:11:25.164 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:24.362+0000 7f8703400700 1 -- 172.21.15.33:0/3765668413 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f86fc022020 con 0x7f870c1096d0 2024-05-03T18:11:25.164 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:24.362+0000 7f8703400700 1 -- 172.21.15.33:0/3765668413 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f86fc0193c0 con 0x7f870c1096d0 2024-05-03T18:11:25.164 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:24.362+0000 7f8711a00700 1 -- 172.21.15.33:0/3765668413 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f870c064b00 con 0x7f870c1096d0 2024-05-03T18:11:25.164 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:24.365+0000 7f8703400700 1 -- 172.21.15.33:0/3765668413 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f86fc0094f0 con 0x7f870c1096d0 2024-05-03T18:11:25.164 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:24.445+0000 7f8711a00700 1 -- 172.21.15.33:0/3765668413 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f870c02e510 con 0x7f870c1096d0 2024-05-03T18:11:25.164 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:24.446+0000 7f8703400700 1 -- 172.21.15.33:0/3765668413 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f86fc01aa50 con 0x7f870c1096d0 2024-05-03T18:11:25.164 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:24.447+0000 7f8711a00700 1 -- 172.21.15.33:0/3765668413 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f870c1096d0 msgr2=0x7f870c19cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:11:25.165 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:24.447+0000 7f8711a00700 1 --2- 172.21.15.33:0/3765668413 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f870c1096d0 0x7f870c19cda0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f86fc0116a0 tx=0x7f86fc0046a0 comp rx=0 tx=0).stop 2024-05-03T18:11:25.165 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:24.447+0000 7f8711a00700 1 -- 172.21.15.33:0/3765668413 shutdown_connections 2024-05-03T18:11:25.165 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:24.447+0000 7f8711a00700 1 --2- 172.21.15.33:0/3765668413 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f870c1096d0 0x7f870c19cda0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:25.165 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:24.447+0000 7f8711a00700 1 -- 172.21.15.33:0/3765668413 >> 172.21.15.33:0/3765668413 conn(0x7f870c1030c0 msgr2=0x7f870c103d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:11:25.165 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:24.448+0000 7f8711a00700 1 -- 172.21.15.33:0/3765668413 shutdown_connections 2024-05-03T18:11:25.165 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:24.448+0000 7f8711a00700 1 -- 172.21.15.33:0/3765668413 wait complete. 2024-05-03T18:11:25.165 INFO:teuthology.orchestra.run.smithi033.stdout:mgr not available, waiting (2/15)... 2024-05-03T18:11:25.482 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:11:25 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:11:25.248+0000 7f07024ce200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-03T18:11:25.982 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:11:25 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:11:25.553+0000 7f07024ce200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-03T18:11:26.482 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:11:26 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:11:26.129+0000 7f07024ce200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-03T18:11:26.482 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:11:26 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:11:26.261+0000 7f07024ce200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-03T18:11:27.207 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:11:26 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:11:26.736+0000 7f07024ce200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-03T18:11:27.207 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:11:27 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:11:27.208+0000 7f07024ce200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-03T18:11:27.483 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:11:27 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:11:27.346+0000 7f07024ce200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-03T18:11:27.769 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:11:27 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:11:27.684+0000 7f07024ce200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-03T18:11:28.232 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:11:27 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:11:27.875+0000 7f07024ce200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-03T18:11:28.232 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:27 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/3097304848' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-03T18:11:28.533 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:11:28 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:11:28.264+0000 7f07024ce200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-03T18:11:28.533 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:11:28 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:11:28.420+0000 7f07024ce200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-03T18:11:28.559 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-05-03T18:11:28.559 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout { 2024-05-03T18:11:28.559 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "fsid": "5703bba4-0978-11ef-bc97-c7b262605968", 2024-05-03T18:11:28.559 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "health": { 2024-05-03T18:11:28.559 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-03T18:11:28.559 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-03T18:11:28.560 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-03T18:11:28.560 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-05-03T18:11:28.560 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-03T18:11:28.560 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-03T18:11:28.560 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 0 2024-05-03T18:11:28.560 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ], 2024-05-03T18:11:28.560 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-03T18:11:28.560 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "a" 2024-05-03T18:11:28.560 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ], 2024-05-03T18:11:28.560 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "quorum_age": 11, 2024-05-03T18:11:28.560 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-03T18:11:28.560 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-03T18:11:28.560 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-03T18:11:28.560 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-03T18:11:28.561 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-05-03T18:11:28.561 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-03T18:11:28.561 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-03T18:11:28.561 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-03T18:11:28.561 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-03T18:11:28.561 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-03T18:11:28.561 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-03T18:11:28.561 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-03T18:11:28.561 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-03T18:11:28.561 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-05-03T18:11:28.561 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-03T18:11:28.561 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-03T18:11:28.561 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-03T18:11:28.561 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-03T18:11:28.562 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-03T18:11:28.562 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-03T18:11:28.562 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-03T18:11:28.562 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-03T18:11:28.562 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-03T18:11:28.562 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-05-03T18:11:28.562 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-03T18:11:28.562 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-03T18:11:28.562 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-03T18:11:28.562 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-03T18:11:28.562 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-05-03T18:11:28.562 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-03T18:11:28.563 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-03T18:11:28.563 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-03T18:11:28.563 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-03T18:11:28.563 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-03T18:11:28.563 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-03T18:11:28.563 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "restful" 2024-05-03T18:11:28.563 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ], 2024-05-03T18:11:28.563 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-03T18:11:28.563 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-05-03T18:11:28.563 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-03T18:11:28.563 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-03T18:11:28.563 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "modified": "2024-05-03T18:11:00.900759+0000", 2024-05-03T18:11:28.563 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-03T18:11:28.563 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-05-03T18:11:28.564 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-03T18:11:28.564 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout } 2024-05-03T18:11:28.564 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:27.770+0000 7f1e7ca00700 1 Processor -- start 2024-05-03T18:11:28.564 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:27.770+0000 7f1e7ca00700 1 -- start start 2024-05-03T18:11:28.564 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:27.771+0000 7f1e7ca00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e78108c40 0x7f1e78109060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:11:28.564 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:27.771+0000 7f1e7ca00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1e78109630 con 0x7f1e78108c40 2024-05-03T18:11:28.564 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:27.771+0000 7f1e75e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e78108c40 0x7f1e78109060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:11:28.564 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:27.771+0000 7f1e75e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e78108c40 0x7f1e78109060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45566/0 (socket says 172.21.15.33:45566) 2024-05-03T18:11:28.564 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:27.771+0000 7f1e75e00700 1 -- 172.21.15.33:0/2995615795 learned_addr learned my addr 172.21.15.33:0/2995615795 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:11:28.564 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:27.772+0000 7f1e75e00700 1 -- 172.21.15.33:0/2995615795 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e78109df0 con 0x7f1e78108c40 2024-05-03T18:11:28.564 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:27.772+0000 7f1e75e00700 1 --2- 172.21.15.33:0/2995615795 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e78108c40 0x7f1e78109060 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1e680098a0 tx=0x7f1e68009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5b31bd00d043148 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:11:28.564 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:27.772+0000 7f1e74a00700 1 -- 172.21.15.33:0/2995615795 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1e68004160 con 0x7f1e78108c40 2024-05-03T18:11:28.564 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:27.772+0000 7f1e74a00700 1 -- 172.21.15.33:0/2995615795 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f1e68004300 con 0x7f1e78108c40 2024-05-03T18:11:28.564 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:27.773+0000 7f1e74a00700 1 -- 172.21.15.33:0/2995615795 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1e680119e0 con 0x7f1e78108c40 2024-05-03T18:11:28.564 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:27.773+0000 7f1e7ca00700 1 -- 172.21.15.33:0/2995615795 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e78108c40 msgr2=0x7f1e78109060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:11:28.565 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:27.773+0000 7f1e7ca00700 1 --2- 172.21.15.33:0/2995615795 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e78108c40 0x7f1e78109060 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1e680098a0 tx=0x7f1e68009bb0 comp rx=0 tx=0).stop 2024-05-03T18:11:28.565 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:27.773+0000 7f1e7ca00700 1 -- 172.21.15.33:0/2995615795 shutdown_connections 2024-05-03T18:11:28.565 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:27.773+0000 7f1e7ca00700 1 --2- 172.21.15.33:0/2995615795 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e78108c40 0x7f1e78109060 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:28.565 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:27.773+0000 7f1e7ca00700 1 -- 172.21.15.33:0/2995615795 >> 172.21.15.33:0/2995615795 conn(0x7f1e78104220 msgr2=0x7f1e78106640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:11:28.565 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:27.774+0000 7f1e7ca00700 1 -- 172.21.15.33:0/2995615795 shutdown_connections 2024-05-03T18:11:28.565 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:27.774+0000 7f1e7ca00700 1 -- 172.21.15.33:0/2995615795 wait complete. 2024-05-03T18:11:28.565 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:27.775+0000 7f1e7ca00700 1 Processor -- start 2024-05-03T18:11:28.565 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:27.775+0000 7f1e7ca00700 1 -- start start 2024-05-03T18:11:28.565 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:27.775+0000 7f1e7ca00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e78108c40 0x7f1e7807eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:11:28.565 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:27.775+0000 7f1e7ca00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1e7810d020 con 0x7f1e78108c40 2024-05-03T18:11:28.565 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:27.775+0000 7f1e75e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e78108c40 0x7f1e7807eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:11:28.565 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:27.776+0000 7f1e75e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e78108c40 0x7f1e7807eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45568/0 (socket says 172.21.15.33:45568) 2024-05-03T18:11:28.565 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:27.776+0000 7f1e75e00700 1 -- 172.21.15.33:0/3097304848 learned_addr learned my addr 172.21.15.33:0/3097304848 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:11:28.565 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:27.776+0000 7f1e75e00700 1 -- 172.21.15.33:0/3097304848 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e7807f000 con 0x7f1e78108c40 2024-05-03T18:11:28.565 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:27.776+0000 7f1e75e00700 1 --2- 172.21.15.33:0/3097304848 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e78108c40 0x7f1e7807eac0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1e68006b20 tx=0x7f1e68011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:11:28.565 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:27.776+0000 7f1e6ea00700 1 -- 172.21.15.33:0/3097304848 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1e68003a60 con 0x7f1e78108c40 2024-05-03T18:11:28.566 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:27.777+0000 7f1e6ea00700 1 -- 172.21.15.33:0/3097304848 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f1e68003c00 con 0x7f1e78108c40 2024-05-03T18:11:28.566 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:27.777+0000 7f1e6ea00700 1 -- 172.21.15.33:0/3097304848 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1e68024780 con 0x7f1e78108c40 2024-05-03T18:11:28.566 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:27.777+0000 7f1e7ca00700 1 -- 172.21.15.33:0/3097304848 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e7807d240 con 0x7f1e78108c40 2024-05-03T18:11:28.566 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:27.777+0000 7f1e7ca00700 1 -- 172.21.15.33:0/3097304848 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e7807d720 con 0x7f1e78108c40 2024-05-03T18:11:28.566 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:27.777+0000 7f1e6ea00700 1 -- 172.21.15.33:0/3097304848 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1e68022020 con 0x7f1e78108c40 2024-05-03T18:11:28.566 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:27.777+0000 7f1e6ea00700 1 -- 172.21.15.33:0/3097304848 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1e680193c0 con 0x7f1e78108c40 2024-05-03T18:11:28.566 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:27.778+0000 7f1e7ca00700 1 -- 172.21.15.33:0/3097304848 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e78064b00 con 0x7f1e78108c40 2024-05-03T18:11:28.566 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:27.780+0000 7f1e6ea00700 1 -- 172.21.15.33:0/3097304848 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f1e6802dde0 con 0x7f1e78108c40 2024-05-03T18:11:28.566 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:27.861+0000 7f1e7ca00700 1 -- 172.21.15.33:0/3097304848 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f1e78196400 con 0x7f1e78108c40 2024-05-03T18:11:28.566 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:27.861+0000 7f1e6ea00700 1 -- 172.21.15.33:0/3097304848 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1240 (secure 0 0 0) 0x7f1e6801aba0 con 0x7f1e78108c40 2024-05-03T18:11:28.566 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:27.863+0000 7f1e7ca00700 1 -- 172.21.15.33:0/3097304848 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e78108c40 msgr2=0x7f1e7807eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:11:28.566 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:27.863+0000 7f1e7ca00700 1 --2- 172.21.15.33:0/3097304848 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e78108c40 0x7f1e7807eac0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1e68006b20 tx=0x7f1e68011a60 comp rx=0 tx=0).stop 2024-05-03T18:11:28.566 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:27.863+0000 7f1e7ca00700 1 -- 172.21.15.33:0/3097304848 shutdown_connections 2024-05-03T18:11:28.566 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:27.863+0000 7f1e7ca00700 1 --2- 172.21.15.33:0/3097304848 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e78108c40 0x7f1e7807eac0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:28.566 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:27.863+0000 7f1e7ca00700 1 -- 172.21.15.33:0/3097304848 >> 172.21.15.33:0/3097304848 conn(0x7f1e78104220 msgr2=0x7f1e78104f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:11:28.567 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:27.863+0000 7f1e7ca00700 1 -- 172.21.15.33:0/3097304848 shutdown_connections 2024-05-03T18:11:28.567 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:27.863+0000 7f1e7ca00700 1 -- 172.21.15.33:0/3097304848 wait complete. 2024-05-03T18:11:28.567 INFO:teuthology.orchestra.run.smithi033.stdout:mgr not available, waiting (3/15)... 2024-05-03T18:11:28.982 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:11:28 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:11:28.549+0000 7f07024ce200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-03T18:11:29.664 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:11:29 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:11:29.343+0000 7f07024ce200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-03T18:11:29.982 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:11:29 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:11:29.665+0000 7f07024ce200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-03T18:11:29.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:29 smithi033 ceph-mon[28330]: Activating manager daemon x 2024-05-03T18:11:29.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:29 smithi033 ceph-mon[28330]: mgrmap e2: x(active, starting, since 0.00985956s) 2024-05-03T18:11:29.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:29 smithi033 ceph-mon[28330]: from='mgr.14100 172.21.15.33:0/874034407' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-03T18:11:29.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:29 smithi033 ceph-mon[28330]: from='mgr.14100 172.21.15.33:0/874034407' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-03T18:11:29.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:29 smithi033 ceph-mon[28330]: from='mgr.14100 172.21.15.33:0/874034407' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-03T18:11:29.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:29 smithi033 ceph-mon[28330]: from='mgr.14100 172.21.15.33:0/874034407' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-03T18:11:29.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:29 smithi033 ceph-mon[28330]: from='mgr.14100 172.21.15.33:0/874034407' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-03T18:11:29.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:29 smithi033 ceph-mon[28330]: Manager daemon x is now available 2024-05-03T18:11:29.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:29 smithi033 ceph-mon[28330]: from='mgr.14100 172.21.15.33:0/874034407' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-03T18:11:29.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:29 smithi033 ceph-mon[28330]: from='mgr.14100 172.21.15.33:0/874034407' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-03T18:11:29.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:29 smithi033 ceph-mon[28330]: from='mgr.14100 172.21.15.33:0/874034407' entity='mgr.x' 2024-05-03T18:11:29.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:29 smithi033 ceph-mon[28330]: from='mgr.14100 172.21.15.33:0/874034407' entity='mgr.x' 2024-05-03T18:11:30.982 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:30 smithi033 ceph-mon[28330]: from='mgr.14100 172.21.15.33:0/874034407' entity='mgr.x' 2024-05-03T18:11:30.982 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:30 smithi033 ceph-mon[28330]: mgrmap e3: x(active, since 1.01768s) 2024-05-03T18:11:31.859 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:31 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/2010199628' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-03T18:11:32.220 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-05-03T18:11:32.220 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout { 2024-05-03T18:11:32.220 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "fsid": "5703bba4-0978-11ef-bc97-c7b262605968", 2024-05-03T18:11:32.220 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "health": { 2024-05-03T18:11:32.220 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-03T18:11:32.220 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-03T18:11:32.221 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-03T18:11:32.221 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-05-03T18:11:32.221 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-03T18:11:32.221 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-03T18:11:32.221 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 0 2024-05-03T18:11:32.221 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ], 2024-05-03T18:11:32.221 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-03T18:11:32.221 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "a" 2024-05-03T18:11:32.221 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ], 2024-05-03T18:11:32.221 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2024-05-03T18:11:32.221 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-03T18:11:32.221 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-03T18:11:32.221 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-03T18:11:32.221 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-03T18:11:32.221 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-05-03T18:11:32.221 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-03T18:11:32.221 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-03T18:11:32.222 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-03T18:11:32.222 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-03T18:11:32.222 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-03T18:11:32.229 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-03T18:11:32.230 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-03T18:11:32.230 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-03T18:11:32.230 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-05-03T18:11:32.230 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-03T18:11:32.230 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-03T18:11:32.230 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-03T18:11:32.230 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-03T18:11:32.230 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-03T18:11:32.230 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-03T18:11:32.230 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-03T18:11:32.230 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-03T18:11:32.230 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-03T18:11:32.230 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-05-03T18:11:32.230 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-03T18:11:32.231 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-03T18:11:32.231 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-03T18:11:32.231 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-03T18:11:32.231 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-05-03T18:11:32.231 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-03T18:11:32.231 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-03T18:11:32.231 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-03T18:11:32.231 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-03T18:11:32.231 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-03T18:11:32.231 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-03T18:11:32.231 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "restful" 2024-05-03T18:11:32.231 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ], 2024-05-03T18:11:32.231 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-03T18:11:32.231 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-05-03T18:11:32.231 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-03T18:11:32.231 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-03T18:11:32.232 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "modified": "2024-05-03T18:11:00.900759+0000", 2024-05-03T18:11:32.232 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-03T18:11:32.232 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-05-03T18:11:32.232 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-03T18:11:32.232 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout } 2024-05-03T18:11:32.232 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:31.165+0000 7fac80600700 1 Processor -- start 2024-05-03T18:11:32.232 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:31.165+0000 7fac80600700 1 -- start start 2024-05-03T18:11:32.232 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:31.166+0000 7fac80600700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac78109690 0x7fac7810ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:11:32.232 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:31.166+0000 7fac80600700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fac7807cdf0 con 0x7fac78109690 2024-05-03T18:11:32.232 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:31.166+0000 7fac7dc00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac78109690 0x7fac7810ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:11:32.232 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:31.166+0000 7fac7dc00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac78109690 0x7fac7810ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45624/0 (socket says 172.21.15.33:45624) 2024-05-03T18:11:32.232 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:31.166+0000 7fac7dc00700 1 -- 172.21.15.33:0/2069951567 learned_addr learned my addr 172.21.15.33:0/2069951567 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:11:32.232 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:31.167+0000 7fac7dc00700 1 -- 172.21.15.33:0/2069951567 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac7810bfc0 con 0x7fac78109690 2024-05-03T18:11:32.232 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:31.167+0000 7fac7dc00700 1 --2- 172.21.15.33:0/2069951567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac78109690 0x7fac7810ba80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fac6c0098a0 tx=0x7fac6c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ba3b68363b9c32b server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:11:32.232 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:31.167+0000 7fac77e00700 1 -- 172.21.15.33:0/2069951567 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac6c004160 con 0x7fac78109690 2024-05-03T18:11:32.232 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:31.167+0000 7fac77e00700 1 -- 172.21.15.33:0/2069951567 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fac6c004300 con 0x7fac78109690 2024-05-03T18:11:32.233 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:31.167+0000 7fac77e00700 1 -- 172.21.15.33:0/2069951567 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac6c0119e0 con 0x7fac78109690 2024-05-03T18:11:32.233 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:31.168+0000 7fac80600700 1 -- 172.21.15.33:0/2069951567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac78109690 msgr2=0x7fac7810ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:11:32.233 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:31.168+0000 7fac80600700 1 --2- 172.21.15.33:0/2069951567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac78109690 0x7fac7810ba80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fac6c0098a0 tx=0x7fac6c009bb0 comp rx=0 tx=0).stop 2024-05-03T18:11:32.233 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:31.168+0000 7fac80600700 1 -- 172.21.15.33:0/2069951567 shutdown_connections 2024-05-03T18:11:32.233 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:31.168+0000 7fac80600700 1 --2- 172.21.15.33:0/2069951567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac78109690 0x7fac7810ba80 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:32.233 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:31.168+0000 7fac80600700 1 -- 172.21.15.33:0/2069951567 >> 172.21.15.33:0/2069951567 conn(0x7fac781030a0 msgr2=0x7fac781054c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:11:32.233 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:31.168+0000 7fac80600700 1 -- 172.21.15.33:0/2069951567 shutdown_connections 2024-05-03T18:11:32.233 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:31.168+0000 7fac80600700 1 -- 172.21.15.33:0/2069951567 wait complete. 2024-05-03T18:11:32.233 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:31.169+0000 7fac80600700 1 Processor -- start 2024-05-03T18:11:32.233 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:31.169+0000 7fac80600700 1 -- start start 2024-05-03T18:11:32.233 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:31.170+0000 7fac80600700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac78109690 0x7fac78108f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:11:32.233 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:31.170+0000 7fac80600700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fac7810ced0 con 0x7fac78109690 2024-05-03T18:11:32.233 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:31.170+0000 7fac7dc00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac78109690 0x7fac78108f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:11:32.233 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:31.170+0000 7fac7dc00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac78109690 0x7fac78108f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45636/0 (socket says 172.21.15.33:45636) 2024-05-03T18:11:32.233 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:31.170+0000 7fac7dc00700 1 -- 172.21.15.33:0/2010199628 learned_addr learned my addr 172.21.15.33:0/2010199628 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:11:32.233 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:31.170+0000 7fac7dc00700 1 -- 172.21.15.33:0/2010199628 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac78109450 con 0x7fac78109690 2024-05-03T18:11:32.234 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:31.171+0000 7fac7dc00700 1 --2- 172.21.15.33:0/2010199628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac78109690 0x7fac78108f10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fac6c009130 tx=0x7fac6c011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:11:32.234 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:31.171+0000 7fac76000700 1 -- 172.21.15.33:0/2010199628 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac6c003b20 con 0x7fac78109690 2024-05-03T18:11:32.234 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:31.171+0000 7fac76000700 1 -- 172.21.15.33:0/2010199628 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fac6c01a8a0 con 0x7fac78109690 2024-05-03T18:11:32.234 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:31.171+0000 7fac80600700 1 -- 172.21.15.33:0/2010199628 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac78107640 con 0x7fac78109690 2024-05-03T18:11:32.234 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:31.171+0000 7fac76000700 1 -- 172.21.15.33:0/2010199628 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac6c024810 con 0x7fac78109690 2024-05-03T18:11:32.234 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:31.171+0000 7fac80600700 1 -- 172.21.15.33:0/2010199628 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac78107b80 con 0x7fac78109690 2024-05-03T18:11:32.234 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:31.172+0000 7fac80600700 1 -- 172.21.15.33:0/2010199628 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac78064b00 con 0x7fac78109690 2024-05-03T18:11:32.234 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:31.172+0000 7fac76000700 1 -- 172.21.15.33:0/2010199628 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 3) ==== 50105+0+0 (secure 0 0 0) 0x7fac6c022020 con 0x7fac78109690 2024-05-03T18:11:32.234 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:31.173+0000 7fac76000700 1 --2- 172.21.15.33:0/2010199628 >> [v2:172.21.15.33:6800/4007322071,v1:172.21.15.33:6801/4007322071] conn(0x7fac6003e3a0 0x7fac60040860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:11:32.234 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:31.173+0000 7fac76000700 1 -- 172.21.15.33:0/2010199628 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fac6c053340 con 0x7fac78109690 2024-05-03T18:11:32.234 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:31.173+0000 7fac7d200700 1 --2- 172.21.15.33:0/2010199628 >> [v2:172.21.15.33:6800/4007322071,v1:172.21.15.33:6801/4007322071] conn(0x7fac6003e3a0 0x7fac60040860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:11:32.234 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:31.174+0000 7fac7d200700 1 --2- 172.21.15.33:0/2010199628 >> [v2:172.21.15.33:6800/4007322071,v1:172.21.15.33:6801/4007322071] conn(0x7fac6003e3a0 0x7fac60040860 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fac64006fd0 tx=0x7fac64006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:11:32.234 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:31.180+0000 7fac76000700 1 -- 172.21.15.33:0/2010199628 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fac6c00f2e0 con 0x7fac78109690 2024-05-03T18:11:32.234 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:31.507+0000 7fac80600700 1 -- 172.21.15.33:0/2010199628 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fac78196510 con 0x7fac78109690 2024-05-03T18:11:32.234 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:31.508+0000 7fac76000700 1 -- 172.21.15.33:0/2010199628 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fac6c01ad70 con 0x7fac78109690 2024-05-03T18:11:32.235 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:31.511+0000 7fac80600700 1 -- 172.21.15.33:0/2010199628 >> [v2:172.21.15.33:6800/4007322071,v1:172.21.15.33:6801/4007322071] conn(0x7fac6003e3a0 msgr2=0x7fac60040860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:11:32.235 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:31.511+0000 7fac80600700 1 --2- 172.21.15.33:0/2010199628 >> [v2:172.21.15.33:6800/4007322071,v1:172.21.15.33:6801/4007322071] conn(0x7fac6003e3a0 0x7fac60040860 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fac64006fd0 tx=0x7fac64006e40 comp rx=0 tx=0).stop 2024-05-03T18:11:32.235 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:31.511+0000 7fac80600700 1 -- 172.21.15.33:0/2010199628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac78109690 msgr2=0x7fac78108f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:11:32.235 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:31.511+0000 7fac80600700 1 --2- 172.21.15.33:0/2010199628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac78109690 0x7fac78108f10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fac6c009130 tx=0x7fac6c011a60 comp rx=0 tx=0).stop 2024-05-03T18:11:32.235 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:31.511+0000 7fac80600700 1 -- 172.21.15.33:0/2010199628 shutdown_connections 2024-05-03T18:11:32.235 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:31.511+0000 7fac80600700 1 --2- 172.21.15.33:0/2010199628 >> [v2:172.21.15.33:6800/4007322071,v1:172.21.15.33:6801/4007322071] conn(0x7fac6003e3a0 0x7fac60040860 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:32.235 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:31.512+0000 7fac80600700 1 --2- 172.21.15.33:0/2010199628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac78109690 0x7fac78108f10 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:32.235 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:31.512+0000 7fac80600700 1 -- 172.21.15.33:0/2010199628 >> 172.21.15.33:0/2010199628 conn(0x7fac781030a0 msgr2=0x7fac78103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:11:32.235 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:31.512+0000 7fac80600700 1 -- 172.21.15.33:0/2010199628 shutdown_connections 2024-05-03T18:11:32.235 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:31.512+0000 7fac80600700 1 -- 172.21.15.33:0/2010199628 wait complete. 2024-05-03T18:11:32.235 INFO:teuthology.orchestra.run.smithi033.stdout:mgr is available 2024-05-03T18:11:32.840 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:32 smithi033 ceph-mon[28330]: mgrmap e4: x(active, since 2s) 2024-05-03T18:11:33.692 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-05-03T18:11:33.692 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout [global] 2024-05-03T18:11:33.692 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout fsid = 5703bba4-0978-11ef-bc97-c7b262605968 2024-05-03T18:11:33.692 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-05-03T18:11:33.692 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-03T18:11:33.692 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-03T18:11:33.692 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-03T18:11:33.692 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-03T18:11:33.693 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-05-03T18:11:33.693 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-03T18:11:33.693 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-03T18:11:33.693 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-05-03T18:11:33.693 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout [osd] 2024-05-03T18:11:33.693 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-03T18:11:33.693 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-03T18:11:33.693 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-03T18:11:33.693 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-03T18:11:33.693 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-03T18:11:33.693 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:32.841+0000 7fd284c00700 1 Processor -- start 2024-05-03T18:11:33.693 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:32.841+0000 7fd284c00700 1 -- start start 2024-05-03T18:11:33.693 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:32.842+0000 7fd284c00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2801072f0 0x7fd280107710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:11:33.693 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:32.842+0000 7fd284c00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd28007cdd0 con 0x7fd2801072f0 2024-05-03T18:11:33.693 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:32.842+0000 7fd27f400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2801072f0 0x7fd280107710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:11:33.693 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:32.842+0000 7fd27f400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2801072f0 0x7fd280107710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45646/0 (socket says 172.21.15.33:45646) 2024-05-03T18:11:33.694 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:32.842+0000 7fd27f400700 1 -- 172.21.15.33:0/3256546308 learned_addr learned my addr 172.21.15.33:0/3256546308 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:11:33.694 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:32.843+0000 7fd27f400700 1 -- 172.21.15.33:0/3256546308 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd280107c50 con 0x7fd2801072f0 2024-05-03T18:11:33.694 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:32.843+0000 7fd27f400700 1 --2- 172.21.15.33:0/3256546308 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2801072f0 0x7fd280107710 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd2700098a0 tx=0x7fd270009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e073d854293277c server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:11:33.694 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:32.844+0000 7fd27e000700 1 -- 172.21.15.33:0/3256546308 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd270004160 con 0x7fd2801072f0 2024-05-03T18:11:33.694 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:32.844+0000 7fd27e000700 1 -- 172.21.15.33:0/3256546308 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fd270004300 con 0x7fd2801072f0 2024-05-03T18:11:33.694 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:32.844+0000 7fd27e000700 1 -- 172.21.15.33:0/3256546308 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd2700119e0 con 0x7fd2801072f0 2024-05-03T18:11:33.694 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:32.844+0000 7fd284c00700 1 -- 172.21.15.33:0/3256546308 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2801072f0 msgr2=0x7fd280107710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:11:33.694 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:32.844+0000 7fd284c00700 1 --2- 172.21.15.33:0/3256546308 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2801072f0 0x7fd280107710 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd2700098a0 tx=0x7fd270009bb0 comp rx=0 tx=0).stop 2024-05-03T18:11:33.694 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:32.845+0000 7fd284c00700 1 -- 172.21.15.33:0/3256546308 shutdown_connections 2024-05-03T18:11:33.694 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:32.845+0000 7fd284c00700 1 --2- 172.21.15.33:0/3256546308 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2801072f0 0x7fd280107710 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:33.694 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:32.845+0000 7fd284c00700 1 -- 172.21.15.33:0/3256546308 >> 172.21.15.33:0/3256546308 conn(0x7fd280102ed0 msgr2=0x7fd2801052f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:11:33.694 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:32.845+0000 7fd284c00700 1 -- 172.21.15.33:0/3256546308 shutdown_connections 2024-05-03T18:11:33.694 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:32.845+0000 7fd284c00700 1 -- 172.21.15.33:0/3256546308 wait complete. 2024-05-03T18:11:33.695 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:32.846+0000 7fd284c00700 1 Processor -- start 2024-05-03T18:11:33.695 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:32.846+0000 7fd284c00700 1 -- start start 2024-05-03T18:11:33.695 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:32.846+0000 7fd284c00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2801072f0 0x7fd28019ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:11:33.695 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:32.846+0000 7fd284c00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd280108c20 con 0x7fd2801072f0 2024-05-03T18:11:33.695 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:32.847+0000 7fd27f400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2801072f0 0x7fd28019ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:11:33.695 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:32.847+0000 7fd27f400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2801072f0 0x7fd28019ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45656/0 (socket says 172.21.15.33:45656) 2024-05-03T18:11:33.695 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:32.847+0000 7fd27f400700 1 -- 172.21.15.33:0/3840077157 learned_addr learned my addr 172.21.15.33:0/3840077157 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:11:33.695 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:32.847+0000 7fd27f400700 1 -- 172.21.15.33:0/3840077157 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd28019efb0 con 0x7fd2801072f0 2024-05-03T18:11:33.695 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:32.847+0000 7fd27f400700 1 --2- 172.21.15.33:0/3840077157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2801072f0 0x7fd28019ea70 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd2700119b0 tx=0x7fd270011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:11:33.695 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:32.848+0000 7fd277e00700 1 -- 172.21.15.33:0/3840077157 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd2700039f0 con 0x7fd2801072f0 2024-05-03T18:11:33.695 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:32.848+0000 7fd277e00700 1 -- 172.21.15.33:0/3840077157 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fd270003b90 con 0x7fd2801072f0 2024-05-03T18:11:33.695 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:32.848+0000 7fd284c00700 1 -- 172.21.15.33:0/3840077157 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd28019f240 con 0x7fd2801072f0 2024-05-03T18:11:33.696 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:32.848+0000 7fd277e00700 1 -- 172.21.15.33:0/3840077157 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd27001a620 con 0x7fd2801072f0 2024-05-03T18:11:33.696 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:32.848+0000 7fd284c00700 1 -- 172.21.15.33:0/3840077157 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2801a1f30 con 0x7fd2801072f0 2024-05-03T18:11:33.696 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:32.849+0000 7fd284c00700 1 -- 172.21.15.33:0/3840077157 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd280064b00 con 0x7fd2801072f0 2024-05-03T18:11:33.696 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:32.849+0000 7fd277e00700 1 -- 172.21.15.33:0/3840077157 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7fd270024430 con 0x7fd2801072f0 2024-05-03T18:11:33.696 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:32.850+0000 7fd277e00700 1 --2- 172.21.15.33:0/3840077157 >> [v2:172.21.15.33:6800/4007322071,v1:172.21.15.33:6801/4007322071] conn(0x7fd26403e420 0x7fd2640408e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:11:33.696 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:32.850+0000 7fd277e00700 1 -- 172.21.15.33:0/3840077157 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd270053b50 con 0x7fd2801072f0 2024-05-03T18:11:33.696 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:32.850+0000 7fd27ea00700 1 --2- 172.21.15.33:0/3840077157 >> [v2:172.21.15.33:6800/4007322071,v1:172.21.15.33:6801/4007322071] conn(0x7fd26403e420 0x7fd2640408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:11:33.696 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:32.851+0000 7fd27ea00700 1 --2- 172.21.15.33:0/3840077157 >> [v2:172.21.15.33:6800/4007322071,v1:172.21.15.33:6801/4007322071] conn(0x7fd26403e420 0x7fd2640408e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd26c006fd0 tx=0x7fd26c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:11:33.696 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:32.855+0000 7fd277e00700 1 -- 172.21.15.33:0/3840077157 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fd2700172b0 con 0x7fd2801072f0 2024-05-03T18:11:33.696 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:33.097+0000 7fd284c00700 1 -- 172.21.15.33:0/3840077157 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fd2800008d0 con 0x7fd2801072f0 2024-05-03T18:11:33.696 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:33.100+0000 7fd277e00700 1 -- 172.21.15.33:0/3840077157 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+409 (secure 0 0 0) 0x7fd27000fdd0 con 0x7fd2801072f0 2024-05-03T18:11:33.696 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:33.103+0000 7fd284c00700 1 -- 172.21.15.33:0/3840077157 >> [v2:172.21.15.33:6800/4007322071,v1:172.21.15.33:6801/4007322071] conn(0x7fd26403e420 msgr2=0x7fd2640408e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:11:33.696 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:33.103+0000 7fd284c00700 1 --2- 172.21.15.33:0/3840077157 >> [v2:172.21.15.33:6800/4007322071,v1:172.21.15.33:6801/4007322071] conn(0x7fd26403e420 0x7fd2640408e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd26c006fd0 tx=0x7fd26c006e40 comp rx=0 tx=0).stop 2024-05-03T18:11:33.697 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:33.103+0000 7fd284c00700 1 -- 172.21.15.33:0/3840077157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2801072f0 msgr2=0x7fd28019ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:11:33.697 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:33.103+0000 7fd284c00700 1 --2- 172.21.15.33:0/3840077157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2801072f0 0x7fd28019ea70 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd2700119b0 tx=0x7fd270011a90 comp rx=0 tx=0).stop 2024-05-03T18:11:33.697 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:33.103+0000 7fd284c00700 1 -- 172.21.15.33:0/3840077157 shutdown_connections 2024-05-03T18:11:33.697 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:33.103+0000 7fd284c00700 1 --2- 172.21.15.33:0/3840077157 >> [v2:172.21.15.33:6800/4007322071,v1:172.21.15.33:6801/4007322071] conn(0x7fd26403e420 0x7fd2640408e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:33.697 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:33.103+0000 7fd284c00700 1 --2- 172.21.15.33:0/3840077157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2801072f0 0x7fd28019ea70 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:33.697 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:33.103+0000 7fd284c00700 1 -- 172.21.15.33:0/3840077157 >> 172.21.15.33:0/3840077157 conn(0x7fd280102ed0 msgr2=0x7fd280195710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:11:33.697 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:33.104+0000 7fd284c00700 1 -- 172.21.15.33:0/3840077157 shutdown_connections 2024-05-03T18:11:33.697 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:33.104+0000 7fd284c00700 1 -- 172.21.15.33:0/3840077157 wait complete. 2024-05-03T18:11:33.697 INFO:teuthology.orchestra.run.smithi033.stdout:Enabling cephadm module... 2024-05-03T18:11:33.974 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:33 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/3840077157' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-03T18:11:34.982 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:34 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/1197581295' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-05-03T18:11:34.982 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:11:34 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: ignoring --setuser ceph since I am not root 2024-05-03T18:11:34.982 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:11:34 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: ignoring --setgroup ceph since I am not root 2024-05-03T18:11:35.413 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:11:35 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:11:35.033+0000 7fb04c689200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-03T18:11:35.413 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:11:35 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:11:35.280+0000 7fb04c689200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-03T18:11:35.413 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:11:35 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:11:35.414+0000 7fb04c689200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-03T18:11:35.471 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:34.300+0000 7fd17ce00700 1 Processor -- start 2024-05-03T18:11:35.471 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:34.300+0000 7fd17ce00700 1 -- start start 2024-05-03T18:11:35.471 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:34.300+0000 7fd17ce00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd17810fc70 0x7fd178112060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:11:35.472 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:34.300+0000 7fd17ce00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd178083480 con 0x7fd17810fc70 2024-05-03T18:11:35.473 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:34.301+0000 7fd175e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd17810fc70 0x7fd178112060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:11:35.474 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:34.301+0000 7fd175e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd17810fc70 0x7fd178112060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45670/0 (socket says 172.21.15.33:45670) 2024-05-03T18:11:35.474 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:34.301+0000 7fd175e00700 1 -- 172.21.15.33:0/3431132989 learned_addr learned my addr 172.21.15.33:0/3431132989 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:11:35.474 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:34.301+0000 7fd175e00700 1 -- 172.21.15.33:0/3431132989 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1781125a0 con 0x7fd17810fc70 2024-05-03T18:11:35.474 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:34.302+0000 7fd175e00700 1 --2- 172.21.15.33:0/3431132989 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd17810fc70 0x7fd178112060 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd1680098a0 tx=0x7fd168009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f8dfe1f881699be server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:11:35.474 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:34.302+0000 7fd175400700 1 -- 172.21.15.33:0/3431132989 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd168004160 con 0x7fd17810fc70 2024-05-03T18:11:35.474 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:34.302+0000 7fd175400700 1 -- 172.21.15.33:0/3431132989 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fd168004300 con 0x7fd17810fc70 2024-05-03T18:11:35.474 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:34.302+0000 7fd175400700 1 -- 172.21.15.33:0/3431132989 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd1680119e0 con 0x7fd17810fc70 2024-05-03T18:11:35.474 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:34.303+0000 7fd17ce00700 1 -- 172.21.15.33:0/3431132989 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd17810fc70 msgr2=0x7fd178112060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:11:35.474 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:34.303+0000 7fd17ce00700 1 --2- 172.21.15.33:0/3431132989 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd17810fc70 0x7fd178112060 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd1680098a0 tx=0x7fd168009bb0 comp rx=0 tx=0).stop 2024-05-03T18:11:35.474 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:34.303+0000 7fd17ce00700 1 -- 172.21.15.33:0/3431132989 shutdown_connections 2024-05-03T18:11:35.474 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:34.303+0000 7fd17ce00700 1 --2- 172.21.15.33:0/3431132989 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd17810fc70 0x7fd178112060 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:35.474 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:34.303+0000 7fd17ce00700 1 -- 172.21.15.33:0/3431132989 >> 172.21.15.33:0/3431132989 conn(0x7fd17807e1e0 msgr2=0x7fd178080620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:11:35.474 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:34.303+0000 7fd17ce00700 1 -- 172.21.15.33:0/3431132989 shutdown_connections 2024-05-03T18:11:35.475 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:34.303+0000 7fd17ce00700 1 -- 172.21.15.33:0/3431132989 wait complete. 2024-05-03T18:11:35.477 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:34.304+0000 7fd17ce00700 1 Processor -- start 2024-05-03T18:11:35.477 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:34.304+0000 7fd17ce00700 1 -- start start 2024-05-03T18:11:35.477 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:34.305+0000 7fd17ce00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd17810fc70 0x7fd1781a3410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:11:35.477 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:34.305+0000 7fd17ce00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd178113560 con 0x7fd17810fc70 2024-05-03T18:11:35.478 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:34.305+0000 7fd175e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd17810fc70 0x7fd1781a3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:11:35.478 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:34.305+0000 7fd175e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd17810fc70 0x7fd1781a3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45680/0 (socket says 172.21.15.33:45680) 2024-05-03T18:11:35.478 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:34.305+0000 7fd175e00700 1 -- 172.21.15.33:0/1197581295 learned_addr learned my addr 172.21.15.33:0/1197581295 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:11:35.478 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:34.305+0000 7fd175e00700 1 -- 172.21.15.33:0/1197581295 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1781a3950 con 0x7fd17810fc70 2024-05-03T18:11:35.478 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:34.306+0000 7fd175e00700 1 --2- 172.21.15.33:0/1197581295 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd17810fc70 0x7fd1781a3410 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd1680116a0 tx=0x7fd1680046a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:11:35.478 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:34.306+0000 7fd16f400700 1 -- 172.21.15.33:0/1197581295 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd168003b70 con 0x7fd17810fc70 2024-05-03T18:11:35.478 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:34.306+0000 7fd16f400700 1 -- 172.21.15.33:0/1197581295 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fd16801a8a0 con 0x7fd17810fc70 2024-05-03T18:11:35.478 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:34.306+0000 7fd17ce00700 1 -- 172.21.15.33:0/1197581295 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1781a3be0 con 0x7fd17810fc70 2024-05-03T18:11:35.478 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:34.306+0000 7fd16f400700 1 -- 172.21.15.33:0/1197581295 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd168024890 con 0x7fd17810fc70 2024-05-03T18:11:35.478 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:34.306+0000 7fd17ce00700 1 -- 172.21.15.33:0/1197581295 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1781a6880 con 0x7fd17810fc70 2024-05-03T18:11:35.478 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:34.307+0000 7fd17ce00700 1 -- 172.21.15.33:0/1197581295 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd178064b00 con 0x7fd17810fc70 2024-05-03T18:11:35.478 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:34.307+0000 7fd16f400700 1 -- 172.21.15.33:0/1197581295 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7fd168022020 con 0x7fd17810fc70 2024-05-03T18:11:35.478 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:34.308+0000 7fd16f400700 1 --2- 172.21.15.33:0/1197581295 >> [v2:172.21.15.33:6800/4007322071,v1:172.21.15.33:6801/4007322071] conn(0x7fd15c03e4c0 0x7fd15c040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:11:35.479 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:34.308+0000 7fd16f400700 1 -- 172.21.15.33:0/1197581295 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd1680536e0 con 0x7fd17810fc70 2024-05-03T18:11:35.479 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:34.308+0000 7fd16d400700 1 --2- 172.21.15.33:0/1197581295 >> [v2:172.21.15.33:6800/4007322071,v1:172.21.15.33:6801/4007322071] conn(0x7fd15c03e4c0 0x7fd15c040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:11:35.479 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:34.309+0000 7fd16d400700 1 --2- 172.21.15.33:0/1197581295 >> [v2:172.21.15.33:6800/4007322071,v1:172.21.15.33:6801/4007322071] conn(0x7fd15c03e4c0 0x7fd15c040980 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd160006fd0 tx=0x7fd160006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:11:35.479 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:34.314+0000 7fd16f400700 1 -- 172.21.15.33:0/1197581295 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fd1680212c0 con 0x7fd17810fc70 2024-05-03T18:11:35.479 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:34.613+0000 7fd17ce00700 1 -- 172.21.15.33:0/1197581295 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7fd17802e510 con 0x7fd17810fc70 2024-05-03T18:11:35.479 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:34.744+0000 7fd16f400700 1 -- 172.21.15.33:0/1197581295 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7fd16800f2e0 con 0x7fd17810fc70 2024-05-03T18:11:35.479 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:34.749+0000 7fd16f400700 1 -- 172.21.15.33:0/1197581295 <== mon.0 v2:172.21.15.33:3300/0 8 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7fd168018530 con 0x7fd17810fc70 2024-05-03T18:11:35.479 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:34.749+0000 7fd17ce00700 1 -- 172.21.15.33:0/1197581295 >> [v2:172.21.15.33:6800/4007322071,v1:172.21.15.33:6801/4007322071] conn(0x7fd15c03e4c0 msgr2=0x7fd15c040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:11:35.479 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:34.749+0000 7fd17ce00700 1 --2- 172.21.15.33:0/1197581295 >> [v2:172.21.15.33:6800/4007322071,v1:172.21.15.33:6801/4007322071] conn(0x7fd15c03e4c0 0x7fd15c040980 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd160006fd0 tx=0x7fd160006e40 comp rx=0 tx=0).stop 2024-05-03T18:11:35.479 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:34.749+0000 7fd17ce00700 1 -- 172.21.15.33:0/1197581295 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd17810fc70 msgr2=0x7fd1781a3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:11:35.479 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:34.749+0000 7fd17ce00700 1 --2- 172.21.15.33:0/1197581295 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd17810fc70 0x7fd1781a3410 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd1680116a0 tx=0x7fd1680046a0 comp rx=0 tx=0).stop 2024-05-03T18:11:35.480 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:34.750+0000 7fd17ce00700 1 -- 172.21.15.33:0/1197581295 shutdown_connections 2024-05-03T18:11:35.480 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:34.750+0000 7fd17ce00700 1 --2- 172.21.15.33:0/1197581295 >> [v2:172.21.15.33:6800/4007322071,v1:172.21.15.33:6801/4007322071] conn(0x7fd15c03e4c0 0x7fd15c040980 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:35.480 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:34.750+0000 7fd17ce00700 1 --2- 172.21.15.33:0/1197581295 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd17810fc70 0x7fd1781a3410 secure :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd1680116a0 tx=0x7fd1680046a0 comp rx=0 tx=0).stop 2024-05-03T18:11:35.480 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:34.750+0000 7fd17ce00700 1 -- 172.21.15.33:0/1197581295 >> 172.21.15.33:0/1197581295 conn(0x7fd17807e1e0 msgr2=0x7fd17807eed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:11:35.480 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:34.750+0000 7fd17ce00700 1 -- 172.21.15.33:0/1197581295 shutdown_connections 2024-05-03T18:11:35.480 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:34.750+0000 7fd17ce00700 1 -- 172.21.15.33:0/1197581295 wait complete. 2024-05-03T18:11:36.023 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:35 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/1197581295' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-05-03T18:11:36.023 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:35 smithi033 ceph-mon[28330]: mgrmap e5: x(active, since 5s) 2024-05-03T18:11:36.023 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:11:35 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:11:35.837+0000 7fb04c689200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-03T18:11:36.023 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:11:35 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:11:35.955+0000 7fb04c689200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-03T18:11:36.982 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:36 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/1819144188' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-03T18:11:36.997 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout { 2024-05-03T18:11:36.997 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-05-03T18:11:36.997 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-03T18:11:36.997 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-03T18:11:36.997 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-03T18:11:36.997 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout } 2024-05-03T18:11:36.997 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:36.024+0000 7fb72ba00700 1 Processor -- start 2024-05-03T18:11:36.997 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:36.024+0000 7fb72ba00700 1 -- start start 2024-05-03T18:11:36.997 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:36.025+0000 7fb72ba00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb72407ee80 0x7fb72407d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:11:36.997 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:36.025+0000 7fb72ba00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb72407d8b0 con 0x7fb72407ee80 2024-05-03T18:11:36.997 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:36.025+0000 7fb729000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb72407ee80 0x7fb72407d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:11:36.997 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:36.025+0000 7fb729000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb72407ee80 0x7fb72407d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56310/0 (socket says 172.21.15.33:56310) 2024-05-03T18:11:36.997 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:36.025+0000 7fb729000700 1 -- 172.21.15.33:0/2421700877 learned_addr learned my addr 172.21.15.33:0/2421700877 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:11:36.998 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:36.026+0000 7fb729000700 1 -- 172.21.15.33:0/2421700877 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb72407da30 con 0x7fb72407ee80 2024-05-03T18:11:36.998 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:36.026+0000 7fb729000700 1 --2- 172.21.15.33:0/2421700877 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb72407ee80 0x7fb72407d2e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb7180098a0 tx=0x7fb718009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a908d35260d9c0e server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:11:36.998 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:36.027+0000 7fb723400700 1 -- 172.21.15.33:0/2421700877 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb718004160 con 0x7fb72407ee80 2024-05-03T18:11:36.998 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:36.027+0000 7fb723400700 1 -- 172.21.15.33:0/2421700877 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fb718004300 con 0x7fb72407ee80 2024-05-03T18:11:36.998 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:36.027+0000 7fb723400700 1 -- 172.21.15.33:0/2421700877 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb7180119e0 con 0x7fb72407ee80 2024-05-03T18:11:36.998 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:36.027+0000 7fb72ba00700 1 -- 172.21.15.33:0/2421700877 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb72407ee80 msgr2=0x7fb72407d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:11:36.998 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:36.027+0000 7fb72ba00700 1 --2- 172.21.15.33:0/2421700877 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb72407ee80 0x7fb72407d2e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb7180098a0 tx=0x7fb718009bb0 comp rx=0 tx=0).stop 2024-05-03T18:11:36.998 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:36.028+0000 7fb72ba00700 1 -- 172.21.15.33:0/2421700877 shutdown_connections 2024-05-03T18:11:36.998 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:36.028+0000 7fb72ba00700 1 --2- 172.21.15.33:0/2421700877 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb72407ee80 0x7fb72407d2e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:36.998 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:36.028+0000 7fb72ba00700 1 -- 172.21.15.33:0/2421700877 >> 172.21.15.33:0/2421700877 conn(0x7fb7241041b0 msgr2=0x7fb7241065d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:11:36.998 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:36.028+0000 7fb72ba00700 1 -- 172.21.15.33:0/2421700877 shutdown_connections 2024-05-03T18:11:36.998 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:36.028+0000 7fb72ba00700 1 -- 172.21.15.33:0/2421700877 wait complete. 2024-05-03T18:11:36.998 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:36.029+0000 7fb72ba00700 1 Processor -- start 2024-05-03T18:11:36.998 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:36.029+0000 7fb72ba00700 1 -- start start 2024-05-03T18:11:36.999 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:36.029+0000 7fb72ba00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb72407ee80 0x7fb72419ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:11:36.999 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:36.029+0000 7fb72ba00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb72410f150 con 0x7fb72407ee80 2024-05-03T18:11:36.999 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:36.029+0000 7fb729000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb72407ee80 0x7fb72419ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:11:36.999 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:36.029+0000 7fb729000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb72407ee80 0x7fb72419ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56324/0 (socket says 172.21.15.33:56324) 2024-05-03T18:11:36.999 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:36.029+0000 7fb729000700 1 -- 172.21.15.33:0/1819144188 learned_addr learned my addr 172.21.15.33:0/1819144188 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:11:36.999 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:36.030+0000 7fb729000700 1 -- 172.21.15.33:0/1819144188 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb72419f220 con 0x7fb72407ee80 2024-05-03T18:11:36.999 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:36.030+0000 7fb729000700 1 --2- 172.21.15.33:0/1819144188 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb72407ee80 0x7fb72419ece0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb7180119b0 tx=0x7fb718011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:11:36.999 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:36.030+0000 7fb721600700 1 -- 172.21.15.33:0/1819144188 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb718003b70 con 0x7fb72407ee80 2024-05-03T18:11:36.999 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:36.030+0000 7fb721600700 1 -- 172.21.15.33:0/1819144188 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fb71801a8a0 con 0x7fb72407ee80 2024-05-03T18:11:36.999 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:36.030+0000 7fb721600700 1 -- 172.21.15.33:0/1819144188 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb718024830 con 0x7fb72407ee80 2024-05-03T18:11:36.999 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:36.030+0000 7fb72ba00700 1 -- 172.21.15.33:0/1819144188 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb72419f4b0 con 0x7fb72407ee80 2024-05-03T18:11:36.999 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:36.031+0000 7fb72ba00700 1 -- 172.21.15.33:0/1819144188 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7241a2190 con 0x7fb72407ee80 2024-05-03T18:11:36.999 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:36.032+0000 7fb72ba00700 1 -- 172.21.15.33:0/1819144188 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb724064b00 con 0x7fb72407ee80 2024-05-03T18:11:36.999 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:36.032+0000 7fb721600700 1 -- 172.21.15.33:0/1819144188 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7fb718022020 con 0x7fb72407ee80 2024-05-03T18:11:36.999 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:36.032+0000 7fb721600700 1 --2- 172.21.15.33:0/1819144188 >> [v2:172.21.15.33:6800/4007322071,v1:172.21.15.33:6801/4007322071] conn(0x7fb70c03e470 0x7fb70c040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:11:37.000 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:36.033+0000 7fb721600700 1 -- 172.21.15.33:0/1819144188 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb718053690 con 0x7fb72407ee80 2024-05-03T18:11:37.000 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:36.033+0000 7fb723e00700 1 -- 172.21.15.33:0/1819144188 >> [v2:172.21.15.33:6800/4007322071,v1:172.21.15.33:6801/4007322071] conn(0x7fb70c03e470 msgr2=0x7fb70c040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/4007322071 2024-05-03T18:11:37.000 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:36.033+0000 7fb723e00700 1 --2- 172.21.15.33:0/1819144188 >> [v2:172.21.15.33:6800/4007322071,v1:172.21.15.33:6801/4007322071] conn(0x7fb70c03e470 0x7fb70c040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-03T18:11:37.000 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:36.038+0000 7fb721600700 1 -- 172.21.15.33:0/1819144188 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fb7180212c0 con 0x7fb72407ee80 2024-05-03T18:11:37.000 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:36.234+0000 7fb723e00700 1 -- 172.21.15.33:0/1819144188 >> [v2:172.21.15.33:6800/4007322071,v1:172.21.15.33:6801/4007322071] conn(0x7fb70c03e470 msgr2=0x7fb70c040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/4007322071 2024-05-03T18:11:37.000 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:36.234+0000 7fb723e00700 1 --2- 172.21.15.33:0/1819144188 >> [v2:172.21.15.33:6800/4007322071,v1:172.21.15.33:6801/4007322071] conn(0x7fb70c03e470 0x7fb70c040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-03T18:11:37.000 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:36.331+0000 7fb72ba00700 1 -- 172.21.15.33:0/1819144188 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fb7241a2360 con 0x7fb72407ee80 2024-05-03T18:11:37.000 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:36.332+0000 7fb721600700 1 -- 172.21.15.33:0/1819144188 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7fb7241a2360 con 0x7fb72407ee80 2024-05-03T18:11:37.000 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:36.335+0000 7fb72ba00700 1 -- 172.21.15.33:0/1819144188 >> [v2:172.21.15.33:6800/4007322071,v1:172.21.15.33:6801/4007322071] conn(0x7fb70c03e470 msgr2=0x7fb70c040930 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-03T18:11:37.000 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:36.335+0000 7fb72ba00700 1 --2- 172.21.15.33:0/1819144188 >> [v2:172.21.15.33:6800/4007322071,v1:172.21.15.33:6801/4007322071] conn(0x7fb70c03e470 0x7fb70c040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:37.000 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:36.335+0000 7fb72ba00700 1 -- 172.21.15.33:0/1819144188 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb72407ee80 msgr2=0x7fb72419ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:11:37.000 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:36.335+0000 7fb72ba00700 1 --2- 172.21.15.33:0/1819144188 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb72407ee80 0x7fb72419ece0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb7180119b0 tx=0x7fb718011a90 comp rx=0 tx=0).stop 2024-05-03T18:11:37.000 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:36.335+0000 7fb72ba00700 1 -- 172.21.15.33:0/1819144188 shutdown_connections 2024-05-03T18:11:37.000 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:36.335+0000 7fb72ba00700 1 --2- 172.21.15.33:0/1819144188 >> [v2:172.21.15.33:6800/4007322071,v1:172.21.15.33:6801/4007322071] conn(0x7fb70c03e470 0x7fb70c040930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:37.000 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:36.335+0000 7fb72ba00700 1 --2- 172.21.15.33:0/1819144188 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb72407ee80 0x7fb72419ece0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:37.001 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:36.335+0000 7fb72ba00700 1 -- 172.21.15.33:0/1819144188 >> 172.21.15.33:0/1819144188 conn(0x7fb7241041b0 msgr2=0x7fb724104d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:11:37.001 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:36.336+0000 7fb72ba00700 1 -- 172.21.15.33:0/1819144188 shutdown_connections 2024-05-03T18:11:37.001 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:36.336+0000 7fb72ba00700 1 -- 172.21.15.33:0/1819144188 wait complete. 2024-05-03T18:11:37.001 INFO:teuthology.orchestra.run.smithi033.stdout:Waiting for the mgr to restart... 2024-05-03T18:11:37.001 INFO:teuthology.orchestra.run.smithi033.stdout:Waiting for mgr epoch 5... 2024-05-03T18:11:37.814 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:11:37 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:11:37.585+0000 7fb04c689200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-03T18:11:38.215 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:11:37 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:11:37.820+0000 7fb04c689200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-03T18:11:38.215 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:11:38 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:11:38.040+0000 7fb04c689200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-03T18:11:38.482 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:11:38 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:11:38.434+0000 7fb04c689200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-03T18:11:38.817 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:11:38 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:11:38.561+0000 7fb04c689200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-03T18:11:39.079 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:11:38 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:11:38.819+0000 7fb04c689200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-03T18:11:39.482 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:11:39 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:11:39.080+0000 7fb04c689200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-03T18:11:40.187 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:11:39 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:11:39.893+0000 7fb04c689200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-03T18:11:40.482 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:11:40 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:11:40.188+0000 7fb04c689200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-03T18:11:40.890 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:11:40 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:11:40.759+0000 7fb04c689200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-03T18:11:41.232 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:11:40 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:11:40.891+0000 7fb04c689200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-03T18:11:41.732 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:11:41 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:11:41.368+0000 7fb04c689200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-03T18:11:42.232 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:11:41 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:11:41.839+0000 7fb04c689200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-03T18:11:42.233 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:11:41 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:11:41.966+0000 7fb04c689200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-03T18:11:42.732 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:11:42 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:11:42.299+0000 7fb04c689200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-03T18:11:42.733 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:11:42 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:11:42.483+0000 7fb04c689200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-03T18:11:43.148 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:11:42 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:11:42.864+0000 7fb04c689200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-03T18:11:43.149 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:11:43 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:11:43.021+0000 7fb04c689200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-03T18:11:43.149 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:11:43 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:11:43.149+0000 7fb04c689200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-03T18:11:44.232 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:11:43 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:11:43.932+0000 7fb04c689200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-03T18:11:44.732 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:11:44 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:11:44.236+0000 7fb04c689200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-03T18:11:44.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:44 smithi033 ceph-mon[28330]: Active manager daemon x restarted 2024-05-03T18:11:44.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:44 smithi033 ceph-mon[28330]: Activating manager daemon x 2024-05-03T18:11:44.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:44 smithi033 ceph-mon[28330]: osdmap e2: 0 total, 0 up, 0 in 2024-05-03T18:11:44.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:44 smithi033 ceph-mon[28330]: mgrmap e6: x(active, starting, since 0.0118635s) 2024-05-03T18:11:44.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:44 smithi033 ceph-mon[28330]: from='mgr.14120 172.21.15.33:0/3290632043' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-03T18:11:44.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:44 smithi033 ceph-mon[28330]: from='mgr.14120 172.21.15.33:0/3290632043' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-03T18:11:44.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:44 smithi033 ceph-mon[28330]: from='mgr.14120 172.21.15.33:0/3290632043' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-03T18:11:44.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:44 smithi033 ceph-mon[28330]: from='mgr.14120 172.21.15.33:0/3290632043' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-03T18:11:44.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:44 smithi033 ceph-mon[28330]: from='mgr.14120 172.21.15.33:0/3290632043' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-03T18:11:44.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:44 smithi033 ceph-mon[28330]: Manager daemon x is now available 2024-05-03T18:11:45.575 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:45 smithi033 ceph-mon[28330]: Found migration_current of "None". Setting to last migration. 2024-05-03T18:11:45.575 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:45 smithi033 ceph-mon[28330]: from='mgr.14120 172.21.15.33:0/3290632043' entity='mgr.x' 2024-05-03T18:11:45.575 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:45 smithi033 ceph-mon[28330]: from='mgr.14120 172.21.15.33:0/3290632043' entity='mgr.x' 2024-05-03T18:11:45.575 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:45 smithi033 ceph-mon[28330]: from='mgr.14120 172.21.15.33:0/3290632043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:11:45.575 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:45 smithi033 ceph-mon[28330]: from='mgr.14120 172.21.15.33:0/3290632043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:11:45.575 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:45 smithi033 ceph-mon[28330]: from='mgr.14120 172.21.15.33:0/3290632043' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-03T18:11:45.575 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:45 smithi033 ceph-mon[28330]: from='mgr.14120 172.21.15.33:0/3290632043' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-03T18:11:45.575 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:45 smithi033 ceph-mon[28330]: from='mgr.14120 172.21.15.33:0/3290632043' entity='mgr.x' 2024-05-03T18:11:45.575 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:45 smithi033 ceph-mon[28330]: from='mgr.14120 172.21.15.33:0/3290632043' entity='mgr.x' 2024-05-03T18:11:45.575 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:45 smithi033 ceph-mon[28330]: mgrmap e7: x(active, since 1.01957s) 2024-05-03T18:11:45.889 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout { 2024-05-03T18:11:45.889 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-05-03T18:11:45.889 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-03T18:11:45.889 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout } 2024-05-03T18:11:45.889 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:37.606+0000 7fef7b200700 1 Processor -- start 2024-05-03T18:11:45.889 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:37.606+0000 7fef7b200700 1 -- start start 2024-05-03T18:11:45.889 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:37.606+0000 7fef7b200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef7c107910 0x7fef7c107d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:11:45.889 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:37.606+0000 7fef7b200700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fef7c108300 con 0x7fef7c107910 2024-05-03T18:11:45.890 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:37.607+0000 7fef79e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef7c107910 0x7fef7c107d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:11:45.890 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:37.607+0000 7fef79e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef7c107910 0x7fef7c107d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56332/0 (socket says 172.21.15.33:56332) 2024-05-03T18:11:45.890 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:37.607+0000 7fef79e00700 1 -- 172.21.15.33:0/2243185310 learned_addr learned my addr 172.21.15.33:0/2243185310 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:11:45.890 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:37.607+0000 7fef79e00700 1 -- 172.21.15.33:0/2243185310 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef7c108b10 con 0x7fef7c107910 2024-05-03T18:11:45.890 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:37.607+0000 7fef79e00700 1 --2- 172.21.15.33:0/2243185310 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef7c107910 0x7fef7c107d30 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fef6c0098a0 tx=0x7fef6c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68d296d61010cfa0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:11:45.890 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:37.608+0000 7fef79400700 1 -- 172.21.15.33:0/2243185310 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fef6c004160 con 0x7fef7c107910 2024-05-03T18:11:45.890 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:37.608+0000 7fef79400700 1 -- 172.21.15.33:0/2243185310 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fef6c004300 con 0x7fef7c107910 2024-05-03T18:11:45.890 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:37.608+0000 7fef79400700 1 -- 172.21.15.33:0/2243185310 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fef6c0119e0 con 0x7fef7c107910 2024-05-03T18:11:45.890 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:37.608+0000 7fef7b200700 1 -- 172.21.15.33:0/2243185310 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef7c107910 msgr2=0x7fef7c107d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:11:45.890 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:37.608+0000 7fef7b200700 1 --2- 172.21.15.33:0/2243185310 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef7c107910 0x7fef7c107d30 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fef6c0098a0 tx=0x7fef6c009bb0 comp rx=0 tx=0).stop 2024-05-03T18:11:45.890 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:37.609+0000 7fef7b200700 1 -- 172.21.15.33:0/2243185310 shutdown_connections 2024-05-03T18:11:45.891 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:37.609+0000 7fef7b200700 1 --2- 172.21.15.33:0/2243185310 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef7c107910 0x7fef7c107d30 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:45.891 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:37.609+0000 7fef7b200700 1 -- 172.21.15.33:0/2243185310 >> 172.21.15.33:0/2243185310 conn(0x7fef7c102ed0 msgr2=0x7fef7c1052f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:11:45.891 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:37.609+0000 7fef7b200700 1 -- 172.21.15.33:0/2243185310 shutdown_connections 2024-05-03T18:11:45.891 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:37.609+0000 7fef7b200700 1 -- 172.21.15.33:0/2243185310 wait complete. 2024-05-03T18:11:45.891 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:37.610+0000 7fef7b200700 1 Processor -- start 2024-05-03T18:11:45.891 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:37.610+0000 7fef7b200700 1 -- start start 2024-05-03T18:11:45.891 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:37.610+0000 7fef7b200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef7c107910 0x7fef7c19eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:11:45.891 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:37.610+0000 7fef7b200700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fef7c10bd40 con 0x7fef7c107910 2024-05-03T18:11:45.891 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:37.611+0000 7fef79e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef7c107910 0x7fef7c19eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:11:45.891 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:37.611+0000 7fef79e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef7c107910 0x7fef7c19eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56336/0 (socket says 172.21.15.33:56336) 2024-05-03T18:11:45.891 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:37.611+0000 7fef79e00700 1 -- 172.21.15.33:0/1372026294 learned_addr learned my addr 172.21.15.33:0/1372026294 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:11:45.891 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:37.611+0000 7fef79e00700 1 -- 172.21.15.33:0/1372026294 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef7c19f0a0 con 0x7fef7c107910 2024-05-03T18:11:45.891 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:37.611+0000 7fef79e00700 1 --2- 172.21.15.33:0/1372026294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef7c107910 0x7fef7c19eb60 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fef6c006b20 tx=0x7fef6c011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:11:45.892 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:37.611+0000 7fef72a00700 1 -- 172.21.15.33:0/1372026294 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fef6c003a60 con 0x7fef7c107910 2024-05-03T18:11:45.892 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:37.612+0000 7fef72a00700 1 -- 172.21.15.33:0/1372026294 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fef6c003c00 con 0x7fef7c107910 2024-05-03T18:11:45.892 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:37.612+0000 7fef72a00700 1 -- 172.21.15.33:0/1372026294 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fef6c024780 con 0x7fef7c107910 2024-05-03T18:11:45.892 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:37.612+0000 7fef7b200700 1 -- 172.21.15.33:0/1372026294 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef7c19f330 con 0x7fef7c107910 2024-05-03T18:11:45.892 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:37.612+0000 7fef7b200700 1 -- 172.21.15.33:0/1372026294 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef7c1a2020 con 0x7fef7c107910 2024-05-03T18:11:45.892 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:37.613+0000 7fef72a00700 1 -- 172.21.15.33:0/1372026294 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7fef6c022020 con 0x7fef7c107910 2024-05-03T18:11:45.892 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:37.614+0000 7fef72a00700 1 --2- 172.21.15.33:0/1372026294 >> [v2:172.21.15.33:6800/4007322071,v1:172.21.15.33:6801/4007322071] conn(0x7fef6003e4c0 0x7fef60040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:11:45.892 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:37.614+0000 7fef72a00700 1 -- 172.21.15.33:0/1372026294 --> [v2:172.21.15.33:6800/4007322071,v1:172.21.15.33:6801/4007322071] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fef60041090 con 0x7fef6003e4c0 2024-05-03T18:11:45.892 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:37.614+0000 7fef72a00700 1 -- 172.21.15.33:0/1372026294 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fef6c0536b0 con 0x7fef7c107910 2024-05-03T18:11:45.892 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:37.614+0000 7fef73e00700 1 -- 172.21.15.33:0/1372026294 >> [v2:172.21.15.33:6800/4007322071,v1:172.21.15.33:6801/4007322071] conn(0x7fef6003e4c0 msgr2=0x7fef60040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/4007322071 2024-05-03T18:11:45.892 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:37.614+0000 7fef73e00700 1 --2- 172.21.15.33:0/1372026294 >> [v2:172.21.15.33:6800/4007322071,v1:172.21.15.33:6801/4007322071] conn(0x7fef6003e4c0 0x7fef60040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-03T18:11:45.892 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:37.815+0000 7fef73e00700 1 -- 172.21.15.33:0/1372026294 >> [v2:172.21.15.33:6800/4007322071,v1:172.21.15.33:6801/4007322071] conn(0x7fef6003e4c0 msgr2=0x7fef60040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/4007322071 2024-05-03T18:11:45.892 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:37.815+0000 7fef73e00700 1 --2- 172.21.15.33:0/1372026294 >> [v2:172.21.15.33:6800/4007322071,v1:172.21.15.33:6801/4007322071] conn(0x7fef6003e4c0 0x7fef60040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-03T18:11:45.893 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:38.216+0000 7fef73e00700 1 -- 172.21.15.33:0/1372026294 >> [v2:172.21.15.33:6800/4007322071,v1:172.21.15.33:6801/4007322071] conn(0x7fef6003e4c0 msgr2=0x7fef60040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/4007322071 2024-05-03T18:11:45.893 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:38.216+0000 7fef73e00700 1 --2- 172.21.15.33:0/1372026294 >> [v2:172.21.15.33:6800/4007322071,v1:172.21.15.33:6801/4007322071] conn(0x7fef6003e4c0 0x7fef60040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-03T18:11:45.893 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:39.017+0000 7fef73e00700 1 -- 172.21.15.33:0/1372026294 >> [v2:172.21.15.33:6800/4007322071,v1:172.21.15.33:6801/4007322071] conn(0x7fef6003e4c0 msgr2=0x7fef60040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/4007322071 2024-05-03T18:11:45.893 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:39.017+0000 7fef73e00700 1 --2- 172.21.15.33:0/1372026294 >> [v2:172.21.15.33:6800/4007322071,v1:172.21.15.33:6801/4007322071] conn(0x7fef6003e4c0 0x7fef60040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-03T18:11:45.893 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:40.619+0000 7fef73e00700 1 -- 172.21.15.33:0/1372026294 >> [v2:172.21.15.33:6800/4007322071,v1:172.21.15.33:6801/4007322071] conn(0x7fef6003e4c0 msgr2=0x7fef60040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/4007322071 2024-05-03T18:11:45.893 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:40.619+0000 7fef73e00700 1 --2- 172.21.15.33:0/1372026294 >> [v2:172.21.15.33:6800/4007322071,v1:172.21.15.33:6801/4007322071] conn(0x7fef6003e4c0 0x7fef60040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-03T18:11:45.893 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:43.822+0000 7fef73e00700 1 -- 172.21.15.33:0/1372026294 >> [v2:172.21.15.33:6800/4007322071,v1:172.21.15.33:6801/4007322071] conn(0x7fef6003e4c0 msgr2=0x7fef60040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/4007322071 2024-05-03T18:11:45.893 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:43.823+0000 7fef73e00700 1 --2- 172.21.15.33:0/1372026294 >> [v2:172.21.15.33:6800/4007322071,v1:172.21.15.33:6801/4007322071] conn(0x7fef6003e4c0 0x7fef60040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-05-03T18:11:45.893 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:44.248+0000 7fef72a00700 1 -- 172.21.15.33:0/1372026294 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mgrmap(e 6) ==== 49989+0+0 (secure 0 0 0) 0x7fef6c018d10 con 0x7fef7c107910 2024-05-03T18:11:45.893 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:44.248+0000 7fef72a00700 1 -- 172.21.15.33:0/1372026294 >> [v2:172.21.15.33:6800/4007322071,v1:172.21.15.33:6801/4007322071] conn(0x7fef6003e4c0 msgr2=0x7fef60040980 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-03T18:11:45.893 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:44.248+0000 7fef72a00700 1 --2- 172.21.15.33:0/1372026294 >> [v2:172.21.15.33:6800/4007322071,v1:172.21.15.33:6801/4007322071] conn(0x7fef6003e4c0 0x7fef60040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:45.893 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:45.252+0000 7fef72a00700 1 -- 172.21.15.33:0/1372026294 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 7) ==== 50116+0+0 (secure 0 0 0) 0x7fef6c0550a0 con 0x7fef7c107910 2024-05-03T18:11:45.893 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:45.253+0000 7fef72a00700 1 --2- 172.21.15.33:0/1372026294 >> [v2:172.21.15.33:6800/2830135294,v1:172.21.15.33:6801/2830135294] conn(0x7fef60041e00 0x7fef600441f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:11:45.893 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:45.253+0000 7fef72a00700 1 -- 172.21.15.33:0/1372026294 --> [v2:172.21.15.33:6800/2830135294,v1:172.21.15.33:6801/2830135294] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fef6c00f4c0 con 0x7fef60041e00 2024-05-03T18:11:45.894 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:45.255+0000 7fef73e00700 1 --2- 172.21.15.33:0/1372026294 >> [v2:172.21.15.33:6800/2830135294,v1:172.21.15.33:6801/2830135294] conn(0x7fef60041e00 0x7fef600441f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:11:45.894 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:45.258+0000 7fef73e00700 1 --2- 172.21.15.33:0/1372026294 >> [v2:172.21.15.33:6800/2830135294,v1:172.21.15.33:6801/2830135294] conn(0x7fef60041e00 0x7fef600441f0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fef64003a10 tx=0x7fef640092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:11:45.894 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:45.260+0000 7fef72a00700 1 -- 172.21.15.33:0/1372026294 <== mgr.14120 v2:172.21.15.33:6800/2830135294 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7fef6c00f4c0 con 0x7fef60041e00 2024-05-03T18:11:45.894 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:45.270+0000 7fef7b200700 1 -- 172.21.15.33:0/1372026294 --> [v2:172.21.15.33:6800/2830135294,v1:172.21.15.33:6801/2830135294] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fef7c064b00 con 0x7fef60041e00 2024-05-03T18:11:45.894 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:45.271+0000 7fef72a00700 1 -- 172.21.15.33:0/1372026294 <== mgr.14120 v2:172.21.15.33:6800/2830135294 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7fef7c064b00 con 0x7fef60041e00 2024-05-03T18:11:45.894 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:45.271+0000 7fef7b200700 1 -- 172.21.15.33:0/1372026294 >> [v2:172.21.15.33:6800/2830135294,v1:172.21.15.33:6801/2830135294] conn(0x7fef60041e00 msgr2=0x7fef600441f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:11:45.894 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:45.271+0000 7fef7b200700 1 --2- 172.21.15.33:0/1372026294 >> [v2:172.21.15.33:6800/2830135294,v1:172.21.15.33:6801/2830135294] conn(0x7fef60041e00 0x7fef600441f0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fef64003a10 tx=0x7fef640092b0 comp rx=0 tx=0).stop 2024-05-03T18:11:45.894 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:45.271+0000 7fef7b200700 1 -- 172.21.15.33:0/1372026294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef7c107910 msgr2=0x7fef7c19eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:11:45.894 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:45.271+0000 7fef7b200700 1 --2- 172.21.15.33:0/1372026294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef7c107910 0x7fef7c19eb60 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fef6c006b20 tx=0x7fef6c011a60 comp rx=0 tx=0).stop 2024-05-03T18:11:45.894 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:45.272+0000 7fef7b200700 1 -- 172.21.15.33:0/1372026294 shutdown_connections 2024-05-03T18:11:45.894 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:45.272+0000 7fef7b200700 1 --2- 172.21.15.33:0/1372026294 >> [v2:172.21.15.33:6800/2830135294,v1:172.21.15.33:6801/2830135294] conn(0x7fef60041e00 0x7fef600441f0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:45.894 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:45.272+0000 7fef7b200700 1 --2- 172.21.15.33:0/1372026294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef7c107910 0x7fef7c19eb60 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:45.894 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:45.272+0000 7fef7b200700 1 --2- 172.21.15.33:0/1372026294 >> [v2:172.21.15.33:6800/4007322071,v1:172.21.15.33:6801/4007322071] conn(0x7fef6003e4c0 0x7fef60040980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:45.895 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:45.272+0000 7fef7b200700 1 -- 172.21.15.33:0/1372026294 >> 172.21.15.33:0/1372026294 conn(0x7fef7c102ed0 msgr2=0x7fef7c103da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:11:45.895 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:45.272+0000 7fef7b200700 1 -- 172.21.15.33:0/1372026294 shutdown_connections 2024-05-03T18:11:45.895 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:45.272+0000 7fef7b200700 1 -- 172.21.15.33:0/1372026294 wait complete. 2024-05-03T18:11:45.895 INFO:teuthology.orchestra.run.smithi033.stdout:mgr epoch 5 is available 2024-05-03T18:11:45.895 INFO:teuthology.orchestra.run.smithi033.stdout:Setting orchestrator backend to cephadm... 2024-05-03T18:11:46.482 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:46 smithi033 ceph-mon[28330]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-03T18:11:46.482 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:46 smithi033 ceph-mon[28330]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-03T18:11:46.482 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:46 smithi033 ceph-mon[28330]: from='mgr.14120 172.21.15.33:0/3290632043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:11:47.403 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:47 smithi033 ceph-mon[28330]: [03/May/2024:18:11:45] ENGINE Bus STARTING 2024-05-03T18:11:47.404 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:47 smithi033 ceph-mon[28330]: [03/May/2024:18:11:46] ENGINE Serving on http://172.21.15.33:8765 2024-05-03T18:11:47.404 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:47 smithi033 ceph-mon[28330]: [03/May/2024:18:11:46] ENGINE Serving on https://172.21.15.33:7150 2024-05-03T18:11:47.404 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:47 smithi033 ceph-mon[28330]: [03/May/2024:18:11:46] ENGINE Bus STARTED 2024-05-03T18:11:47.404 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:47 smithi033 ceph-mon[28330]: mgrmap e8: x(active, since 2s) 2024-05-03T18:11:47.404 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:47 smithi033 ceph-mon[28330]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:11:47.404 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:47 smithi033 ceph-mon[28330]: from='mgr.14120 172.21.15.33:0/3290632043' entity='mgr.x' 2024-05-03T18:11:47.404 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:47 smithi033 ceph-mon[28330]: from='mgr.14120 172.21.15.33:0/3290632043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:11:47.418 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:46.514+0000 7f7ab1400700 1 Processor -- start 2024-05-03T18:11:47.419 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:46.514+0000 7f7ab1400700 1 -- start start 2024-05-03T18:11:47.419 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:46.515+0000 7f7ab1400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7aac10fc70 0x7f7aac112060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:11:47.419 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:46.515+0000 7f7ab1400700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7aac083480 con 0x7f7aac10fc70 2024-05-03T18:11:47.419 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:46.515+0000 7f7aaa800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7aac10fc70 0x7f7aac112060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:11:47.419 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:46.515+0000 7f7aaa800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7aac10fc70 0x7f7aac112060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55288/0 (socket says 172.21.15.33:55288) 2024-05-03T18:11:47.421 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:46.515+0000 7f7aaa800700 1 -- 172.21.15.33:0/3027864247 learned_addr learned my addr 172.21.15.33:0/3027864247 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:11:47.421 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:46.516+0000 7f7aaa800700 1 -- 172.21.15.33:0/3027864247 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7aac1125a0 con 0x7f7aac10fc70 2024-05-03T18:11:47.421 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:46.516+0000 7f7aaa800700 1 --2- 172.21.15.33:0/3027864247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7aac10fc70 0x7f7aac112060 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c0098a0 tx=0x7f7a9c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4038a9ad0f35db7b server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:11:47.421 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:46.516+0000 7f7aa9400700 1 -- 172.21.15.33:0/3027864247 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a9c004160 con 0x7f7aac10fc70 2024-05-03T18:11:47.421 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:46.517+0000 7f7aa9400700 1 -- 172.21.15.33:0/3027864247 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f7a9c004300 con 0x7f7aac10fc70 2024-05-03T18:11:47.422 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:46.517+0000 7f7aa9400700 1 -- 172.21.15.33:0/3027864247 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a9c0119e0 con 0x7f7aac10fc70 2024-05-03T18:11:47.422 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:46.517+0000 7f7ab1400700 1 -- 172.21.15.33:0/3027864247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7aac10fc70 msgr2=0x7f7aac112060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:11:47.422 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:46.517+0000 7f7ab1400700 1 --2- 172.21.15.33:0/3027864247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7aac10fc70 0x7f7aac112060 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c0098a0 tx=0x7f7a9c009bb0 comp rx=0 tx=0).stop 2024-05-03T18:11:47.422 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:46.517+0000 7f7ab1400700 1 -- 172.21.15.33:0/3027864247 shutdown_connections 2024-05-03T18:11:47.422 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:46.517+0000 7f7ab1400700 1 --2- 172.21.15.33:0/3027864247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7aac10fc70 0x7f7aac112060 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:47.422 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:46.517+0000 7f7ab1400700 1 -- 172.21.15.33:0/3027864247 >> 172.21.15.33:0/3027864247 conn(0x7f7aac07e1e0 msgr2=0x7f7aac080620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:11:47.424 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:46.517+0000 7f7ab1400700 1 -- 172.21.15.33:0/3027864247 shutdown_connections 2024-05-03T18:11:47.424 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:46.517+0000 7f7ab1400700 1 -- 172.21.15.33:0/3027864247 wait complete. 2024-05-03T18:11:47.424 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:46.518+0000 7f7ab1400700 1 Processor -- start 2024-05-03T18:11:47.425 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:46.519+0000 7f7ab1400700 1 -- start start 2024-05-03T18:11:47.425 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:46.519+0000 7f7ab1400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7aac10fc70 0x7f7aac1a3410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:11:47.425 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:46.519+0000 7f7ab1400700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7aac113560 con 0x7f7aac10fc70 2024-05-03T18:11:47.425 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:46.519+0000 7f7aaa800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7aac10fc70 0x7f7aac1a3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:11:47.425 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:46.519+0000 7f7aaa800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7aac10fc70 0x7f7aac1a3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55302/0 (socket says 172.21.15.33:55302) 2024-05-03T18:11:47.425 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:46.519+0000 7f7aaa800700 1 -- 172.21.15.33:0/3399846382 learned_addr learned my addr 172.21.15.33:0/3399846382 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:11:47.425 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:46.520+0000 7f7aaa800700 1 -- 172.21.15.33:0/3399846382 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7aac1a3950 con 0x7f7aac10fc70 2024-05-03T18:11:47.425 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:46.520+0000 7f7aaa800700 1 --2- 172.21.15.33:0/3399846382 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7aac10fc70 0x7f7aac1a3410 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c0116a0 tx=0x7f7a9c0046a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:11:47.425 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:46.520+0000 7f7aa3400700 1 -- 172.21.15.33:0/3399846382 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a9c003b70 con 0x7f7aac10fc70 2024-05-03T18:11:47.425 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:46.520+0000 7f7aa3400700 1 -- 172.21.15.33:0/3399846382 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f7a9c01a8a0 con 0x7f7aac10fc70 2024-05-03T18:11:47.425 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:46.520+0000 7f7aa3400700 1 -- 172.21.15.33:0/3399846382 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a9c024890 con 0x7f7aac10fc70 2024-05-03T18:11:47.425 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:46.520+0000 7f7ab1400700 1 -- 172.21.15.33:0/3399846382 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7aac1a3be0 con 0x7f7aac10fc70 2024-05-03T18:11:47.426 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:46.520+0000 7f7ab1400700 1 -- 172.21.15.33:0/3399846382 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7aac1a6880 con 0x7f7aac10fc70 2024-05-03T18:11:47.426 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:46.521+0000 7f7ab1400700 1 -- 172.21.15.33:0/3399846382 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7aac064b00 con 0x7f7aac10fc70 2024-05-03T18:11:47.426 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:46.522+0000 7f7aa3400700 1 -- 172.21.15.33:0/3399846382 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f7a9c022020 con 0x7f7aac10fc70 2024-05-03T18:11:47.428 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:46.522+0000 7f7aa3400700 1 --2- 172.21.15.33:0/3399846382 >> [v2:172.21.15.33:6800/2830135294,v1:172.21.15.33:6801/2830135294] conn(0x7f7a9003e4c0 0x7f7a90040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:11:47.428 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:46.522+0000 7f7aa3400700 1 -- 172.21.15.33:0/3399846382 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f7a9c053b40 con 0x7f7aac10fc70 2024-05-03T18:11:47.428 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:46.523+0000 7f7aa9e00700 1 --2- 172.21.15.33:0/3399846382 >> [v2:172.21.15.33:6800/2830135294,v1:172.21.15.33:6801/2830135294] conn(0x7f7a9003e4c0 0x7f7a90040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:11:47.428 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:46.523+0000 7f7aa9e00700 1 --2- 172.21.15.33:0/3399846382 >> [v2:172.21.15.33:6800/2830135294,v1:172.21.15.33:6801/2830135294] conn(0x7f7a9003e4c0 0x7f7a90040980 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7a98006fd0 tx=0x7f7a98006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:11:47.428 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:46.528+0000 7f7aa3400700 1 -- 172.21.15.33:0/3399846382 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f7a9c0212c0 con 0x7f7aac10fc70 2024-05-03T18:11:47.428 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:46.789+0000 7f7ab1400700 1 -- 172.21.15.33:0/3399846382 --> [v2:172.21.15.33:6800/2830135294,v1:172.21.15.33:6801/2830135294] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f7aac02e580 con 0x7f7a9003e4c0 2024-05-03T18:11:47.428 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:46.801+0000 7f7aa3400700 1 -- 172.21.15.33:0/3399846382 <== mgr.14120 v2:172.21.15.33:6800/2830135294 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7aac02e580 con 0x7f7a9003e4c0 2024-05-03T18:11:47.429 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:46.805+0000 7f7ab1400700 1 -- 172.21.15.33:0/3399846382 >> [v2:172.21.15.33:6800/2830135294,v1:172.21.15.33:6801/2830135294] conn(0x7f7a9003e4c0 msgr2=0x7f7a90040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:11:47.429 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:46.805+0000 7f7ab1400700 1 --2- 172.21.15.33:0/3399846382 >> [v2:172.21.15.33:6800/2830135294,v1:172.21.15.33:6801/2830135294] conn(0x7f7a9003e4c0 0x7f7a90040980 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7a98006fd0 tx=0x7f7a98006e40 comp rx=0 tx=0).stop 2024-05-03T18:11:47.429 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:46.805+0000 7f7ab1400700 1 -- 172.21.15.33:0/3399846382 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7aac10fc70 msgr2=0x7f7aac1a3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:11:47.429 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:46.805+0000 7f7ab1400700 1 --2- 172.21.15.33:0/3399846382 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7aac10fc70 0x7f7aac1a3410 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c0116a0 tx=0x7f7a9c0046a0 comp rx=0 tx=0).stop 2024-05-03T18:11:47.429 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:46.805+0000 7f7ab1400700 1 -- 172.21.15.33:0/3399846382 shutdown_connections 2024-05-03T18:11:47.429 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:46.805+0000 7f7ab1400700 1 --2- 172.21.15.33:0/3399846382 >> [v2:172.21.15.33:6800/2830135294,v1:172.21.15.33:6801/2830135294] conn(0x7f7a9003e4c0 0x7f7a90040980 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:47.429 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:46.805+0000 7f7ab1400700 1 --2- 172.21.15.33:0/3399846382 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7aac10fc70 0x7f7aac1a3410 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:47.429 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:46.805+0000 7f7ab1400700 1 -- 172.21.15.33:0/3399846382 >> 172.21.15.33:0/3399846382 conn(0x7f7aac07e1e0 msgr2=0x7f7aac07eed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:11:47.429 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:46.805+0000 7f7ab1400700 1 -- 172.21.15.33:0/3399846382 shutdown_connections 2024-05-03T18:11:47.429 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:46.805+0000 7f7ab1400700 1 -- 172.21.15.33:0/3399846382 wait complete. 2024-05-03T18:11:48.874 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout value unchanged 2024-05-03T18:11:48.880 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:48.016+0000 7f13fa200700 1 Processor -- start 2024-05-03T18:11:48.880 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:48.016+0000 7f13fa200700 1 -- start start 2024-05-03T18:11:48.880 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:48.016+0000 7f13fa200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13f407ee80 0x7f13f407d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:11:48.880 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:48.017+0000 7f13fa200700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f13f407d8b0 con 0x7f13f407ee80 2024-05-03T18:11:48.880 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:48.017+0000 7f13f3400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13f407ee80 0x7f13f407d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:11:48.880 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:48.017+0000 7f13f3400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13f407ee80 0x7f13f407d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55316/0 (socket says 172.21.15.33:55316) 2024-05-03T18:11:48.881 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:48.017+0000 7f13f3400700 1 -- 172.21.15.33:0/1987287045 learned_addr learned my addr 172.21.15.33:0/1987287045 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:11:48.881 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:48.018+0000 7f13f3400700 1 -- 172.21.15.33:0/1987287045 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13f407da30 con 0x7f13f407ee80 2024-05-03T18:11:48.881 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:48.018+0000 7f13f3400700 1 --2- 172.21.15.33:0/1987287045 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13f407ee80 0x7f13f407d2e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f13e40098a0 tx=0x7f13e4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f926d7aee59cb74a server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:11:48.881 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:48.018+0000 7f13f2000700 1 -- 172.21.15.33:0/1987287045 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f13e4004160 con 0x7f13f407ee80 2024-05-03T18:11:48.881 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:48.019+0000 7f13f2000700 1 -- 172.21.15.33:0/1987287045 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f13e4004300 con 0x7f13f407ee80 2024-05-03T18:11:48.881 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:48.019+0000 7f13f2000700 1 -- 172.21.15.33:0/1987287045 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f13e40119e0 con 0x7f13f407ee80 2024-05-03T18:11:48.881 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:48.019+0000 7f13fa200700 1 -- 172.21.15.33:0/1987287045 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13f407ee80 msgr2=0x7f13f407d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:11:48.881 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:48.019+0000 7f13fa200700 1 --2- 172.21.15.33:0/1987287045 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13f407ee80 0x7f13f407d2e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f13e40098a0 tx=0x7f13e4009bb0 comp rx=0 tx=0).stop 2024-05-03T18:11:48.881 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:48.019+0000 7f13fa200700 1 -- 172.21.15.33:0/1987287045 shutdown_connections 2024-05-03T18:11:48.881 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:48.019+0000 7f13fa200700 1 --2- 172.21.15.33:0/1987287045 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13f407ee80 0x7f13f407d2e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:48.881 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:48.019+0000 7f13fa200700 1 -- 172.21.15.33:0/1987287045 >> 172.21.15.33:0/1987287045 conn(0x7f13f4104180 msgr2=0x7f13f41065e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:11:48.881 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:48.020+0000 7f13fa200700 1 -- 172.21.15.33:0/1987287045 shutdown_connections 2024-05-03T18:11:48.881 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:48.020+0000 7f13fa200700 1 -- 172.21.15.33:0/1987287045 wait complete. 2024-05-03T18:11:48.881 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:48.021+0000 7f13fa200700 1 Processor -- start 2024-05-03T18:11:48.881 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:48.021+0000 7f13fa200700 1 -- start start 2024-05-03T18:11:48.882 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:48.021+0000 7f13fa200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13f407ee80 0x7f13f419f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:11:48.882 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:48.021+0000 7f13fa200700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f13f410f160 con 0x7f13f407ee80 2024-05-03T18:11:48.882 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:48.021+0000 7f13f3400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13f407ee80 0x7f13f419f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:11:48.882 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:48.022+0000 7f13f3400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13f407ee80 0x7f13f419f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55320/0 (socket says 172.21.15.33:55320) 2024-05-03T18:11:48.882 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:48.022+0000 7f13f3400700 1 -- 172.21.15.33:0/1426840820 learned_addr learned my addr 172.21.15.33:0/1426840820 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:11:48.882 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:48.022+0000 7f13f3400700 1 -- 172.21.15.33:0/1426840820 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13f419f580 con 0x7f13f407ee80 2024-05-03T18:11:48.882 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:48.022+0000 7f13f3400700 1 --2- 172.21.15.33:0/1426840820 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13f407ee80 0x7f13f419f040 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f13e40119b0 tx=0x7f13e4011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:11:48.882 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:48.023+0000 7f13ebe00700 1 -- 172.21.15.33:0/1426840820 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f13e4003b70 con 0x7f13f407ee80 2024-05-03T18:11:48.882 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:48.023+0000 7f13ebe00700 1 -- 172.21.15.33:0/1426840820 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f13e401a8a0 con 0x7f13f407ee80 2024-05-03T18:11:48.882 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:48.023+0000 7f13ebe00700 1 -- 172.21.15.33:0/1426840820 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f13e4024830 con 0x7f13f407ee80 2024-05-03T18:11:48.882 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:48.023+0000 7f13fa200700 1 -- 172.21.15.33:0/1426840820 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13f419f810 con 0x7f13f407ee80 2024-05-03T18:11:48.882 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:48.023+0000 7f13fa200700 1 -- 172.21.15.33:0/1426840820 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13f41a2500 con 0x7f13f407ee80 2024-05-03T18:11:48.882 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:48.024+0000 7f13ebe00700 1 -- 172.21.15.33:0/1426840820 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f13e4022020 con 0x7f13f407ee80 2024-05-03T18:11:48.883 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:48.024+0000 7f13fa200700 1 -- 172.21.15.33:0/1426840820 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13f4064b00 con 0x7f13f407ee80 2024-05-03T18:11:48.883 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:48.025+0000 7f13ebe00700 1 --2- 172.21.15.33:0/1426840820 >> [v2:172.21.15.33:6800/2830135294,v1:172.21.15.33:6801/2830135294] conn(0x7f13d803e470 0x7f13d8040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:11:48.883 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:48.025+0000 7f13ebe00700 1 -- 172.21.15.33:0/1426840820 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f13e4053a00 con 0x7f13f407ee80 2024-05-03T18:11:48.883 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:48.025+0000 7f13f2a00700 1 --2- 172.21.15.33:0/1426840820 >> [v2:172.21.15.33:6800/2830135294,v1:172.21.15.33:6801/2830135294] conn(0x7f13d803e470 0x7f13d8040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:11:48.883 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:48.026+0000 7f13f2a00700 1 --2- 172.21.15.33:0/1426840820 >> [v2:172.21.15.33:6800/2830135294,v1:172.21.15.33:6801/2830135294] conn(0x7f13d803e470 0x7f13d8040930 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f13e0006fd0 tx=0x7f13e0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:11:48.883 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:48.031+0000 7f13ebe00700 1 -- 172.21.15.33:0/1426840820 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f13e401aa50 con 0x7f13f407ee80 2024-05-03T18:11:48.883 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:48.279+0000 7f13fa200700 1 -- 172.21.15.33:0/1426840820 --> [v2:172.21.15.33:6800/2830135294,v1:172.21.15.33:6801/2830135294] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f13f402e580 con 0x7f13d803e470 2024-05-03T18:11:48.883 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:48.280+0000 7f13ebe00700 1 -- 172.21.15.33:0/1426840820 <== mgr.14120 v2:172.21.15.33:6800/2830135294 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f13f402e580 con 0x7f13d803e470 2024-05-03T18:11:48.883 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:48.283+0000 7f13fa200700 1 -- 172.21.15.33:0/1426840820 >> [v2:172.21.15.33:6800/2830135294,v1:172.21.15.33:6801/2830135294] conn(0x7f13d803e470 msgr2=0x7f13d8040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:11:48.883 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:48.283+0000 7f13fa200700 1 --2- 172.21.15.33:0/1426840820 >> [v2:172.21.15.33:6800/2830135294,v1:172.21.15.33:6801/2830135294] conn(0x7f13d803e470 0x7f13d8040930 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f13e0006fd0 tx=0x7f13e0006e40 comp rx=0 tx=0).stop 2024-05-03T18:11:48.883 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:48.283+0000 7f13fa200700 1 -- 172.21.15.33:0/1426840820 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13f407ee80 msgr2=0x7f13f419f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:11:48.883 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:48.283+0000 7f13fa200700 1 --2- 172.21.15.33:0/1426840820 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13f407ee80 0x7f13f419f040 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f13e40119b0 tx=0x7f13e4011a90 comp rx=0 tx=0).stop 2024-05-03T18:11:48.883 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:48.284+0000 7f13fa200700 1 -- 172.21.15.33:0/1426840820 shutdown_connections 2024-05-03T18:11:48.883 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:48.284+0000 7f13fa200700 1 --2- 172.21.15.33:0/1426840820 >> [v2:172.21.15.33:6800/2830135294,v1:172.21.15.33:6801/2830135294] conn(0x7f13d803e470 0x7f13d8040930 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:48.883 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:48.284+0000 7f13fa200700 1 --2- 172.21.15.33:0/1426840820 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13f407ee80 0x7f13f419f040 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:48.884 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:48.284+0000 7f13fa200700 1 -- 172.21.15.33:0/1426840820 >> 172.21.15.33:0/1426840820 conn(0x7f13f4104180 msgr2=0x7f13f4104fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:11:48.884 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:48.284+0000 7f13fa200700 1 -- 172.21.15.33:0/1426840820 shutdown_connections 2024-05-03T18:11:48.884 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:48.284+0000 7f13fa200700 1 -- 172.21.15.33:0/1426840820 wait complete. 2024-05-03T18:11:48.884 INFO:teuthology.orchestra.run.smithi033.stdout:Generating ssh key... 2024-05-03T18:11:49.155 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:48 smithi033 ceph-mon[28330]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:11:49.982 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:49 smithi033 ceph-mon[28330]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:11:49.982 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:49 smithi033 ceph-mon[28330]: Generating ssh key... 2024-05-03T18:11:50.484 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:11:50 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: Generating public/private rsa key pair. 2024-05-03T18:11:50.484 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:11:50 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: Your identification has been saved in /tmp/tmphg89rqz4/key. 2024-05-03T18:11:50.484 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:11:50 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: Your public key has been saved in /tmp/tmphg89rqz4/key.pub. 2024-05-03T18:11:50.484 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:11:50 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: The key fingerprint is: 2024-05-03T18:11:50.484 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:11:50 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: SHA256:H9I7f2kCB+7GEYQnCWCVjEcn3lsFzywtC7wcw7t47QI ceph-5703bba4-0978-11ef-bc97-c7b262605968 2024-05-03T18:11:50.484 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:11:50 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: The key's randomart image is: 2024-05-03T18:11:50.484 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:11:50 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: +---[RSA 3072]----+ 2024-05-03T18:11:50.484 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:11:50 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: | o*=o.o... | 2024-05-03T18:11:50.484 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:11:50 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: | ...+*+ o* | 2024-05-03T18:11:50.484 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:11:50 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: | .. B++ = | 2024-05-03T18:11:50.484 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:11:50 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: | . Xo+ | 2024-05-03T18:11:50.485 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:11:50 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: | S.+o | 2024-05-03T18:11:50.485 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:11:50 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: | E. =+o. | 2024-05-03T18:11:50.485 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:11:50 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: | ..oo=+ . | 2024-05-03T18:11:50.485 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:11:50 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: | ...+o. + | 2024-05-03T18:11:50.485 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:11:50 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: | o. .+ | 2024-05-03T18:11:50.485 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:11:50 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: +----[SHA256]-----+ 2024-05-03T18:11:50.896 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:49.435+0000 7ffbd7a00700 1 Processor -- start 2024-05-03T18:11:50.896 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:49.435+0000 7ffbd7a00700 1 -- start start 2024-05-03T18:11:50.896 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:49.435+0000 7ffbd7a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffbd007ee40 0x7ffbd007d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:11:50.896 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:49.435+0000 7ffbd7a00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd007d870 con 0x7ffbd007ee40 2024-05-03T18:11:50.896 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:49.436+0000 7ffbd5000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffbd007ee40 0x7ffbd007d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:11:50.897 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:49.436+0000 7ffbd5000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffbd007ee40 0x7ffbd007d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55332/0 (socket says 172.21.15.33:55332) 2024-05-03T18:11:50.897 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:49.436+0000 7ffbd5000700 1 -- 172.21.15.33:0/61081135 learned_addr learned my addr 172.21.15.33:0/61081135 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:11:50.897 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:49.436+0000 7ffbd5000700 1 -- 172.21.15.33:0/61081135 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbd007d9f0 con 0x7ffbd007ee40 2024-05-03T18:11:50.897 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:49.437+0000 7ffbd5000700 1 --2- 172.21.15.33:0/61081135 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffbd007ee40 0x7ffbd007d2a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ffbc40098a0 tx=0x7ffbc4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8f576f6f2d8897e server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:11:50.897 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:49.437+0000 7ffbcf400700 1 -- 172.21.15.33:0/61081135 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffbc4004160 con 0x7ffbd007ee40 2024-05-03T18:11:50.897 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:49.437+0000 7ffbcf400700 1 -- 172.21.15.33:0/61081135 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7ffbc4004300 con 0x7ffbd007ee40 2024-05-03T18:11:50.897 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:49.437+0000 7ffbcf400700 1 -- 172.21.15.33:0/61081135 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffbc40119e0 con 0x7ffbd007ee40 2024-05-03T18:11:50.897 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:49.437+0000 7ffbd7a00700 1 -- 172.21.15.33:0/61081135 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffbd007ee40 msgr2=0x7ffbd007d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:11:50.897 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:49.437+0000 7ffbd7a00700 1 --2- 172.21.15.33:0/61081135 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffbd007ee40 0x7ffbd007d2a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ffbc40098a0 tx=0x7ffbc4009bb0 comp rx=0 tx=0).stop 2024-05-03T18:11:50.897 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:49.438+0000 7ffbd7a00700 1 -- 172.21.15.33:0/61081135 shutdown_connections 2024-05-03T18:11:50.897 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:49.438+0000 7ffbd7a00700 1 --2- 172.21.15.33:0/61081135 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffbd007ee40 0x7ffbd007d2a0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:50.897 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:49.438+0000 7ffbd7a00700 1 -- 172.21.15.33:0/61081135 >> 172.21.15.33:0/61081135 conn(0x7ffbd0104180 msgr2=0x7ffbd01065e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:11:50.897 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:49.438+0000 7ffbd7a00700 1 -- 172.21.15.33:0/61081135 shutdown_connections 2024-05-03T18:11:50.897 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:49.438+0000 7ffbd7a00700 1 -- 172.21.15.33:0/61081135 wait complete. 2024-05-03T18:11:50.897 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:49.439+0000 7ffbd7a00700 1 Processor -- start 2024-05-03T18:11:50.898 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:49.439+0000 7ffbd7a00700 1 -- start start 2024-05-03T18:11:50.898 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:49.439+0000 7ffbd7a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffbd007ee40 0x7ffbd019f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:11:50.898 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:49.439+0000 7ffbd7a00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd010f150 con 0x7ffbd007ee40 2024-05-03T18:11:50.898 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:49.439+0000 7ffbd5000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffbd007ee40 0x7ffbd019f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:11:50.898 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:49.440+0000 7ffbd5000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffbd007ee40 0x7ffbd019f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55334/0 (socket says 172.21.15.33:55334) 2024-05-03T18:11:50.898 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:49.440+0000 7ffbd5000700 1 -- 172.21.15.33:0/2188677813 learned_addr learned my addr 172.21.15.33:0/2188677813 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:11:50.898 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:49.440+0000 7ffbd5000700 1 -- 172.21.15.33:0/2188677813 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbd019f560 con 0x7ffbd007ee40 2024-05-03T18:11:50.898 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:49.440+0000 7ffbd5000700 1 --2- 172.21.15.33:0/2188677813 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffbd007ee40 0x7ffbd019f020 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ffbc4006b20 tx=0x7ffbc4011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:11:50.898 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:49.440+0000 7ffbcd600700 1 -- 172.21.15.33:0/2188677813 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffbc4003a60 con 0x7ffbd007ee40 2024-05-03T18:11:50.898 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:49.440+0000 7ffbcd600700 1 -- 172.21.15.33:0/2188677813 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7ffbc4003c00 con 0x7ffbd007ee40 2024-05-03T18:11:50.898 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:49.440+0000 7ffbcd600700 1 -- 172.21.15.33:0/2188677813 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffbc4024780 con 0x7ffbd007ee40 2024-05-03T18:11:50.898 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:49.440+0000 7ffbd7a00700 1 -- 172.21.15.33:0/2188677813 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbd019f7f0 con 0x7ffbd007ee40 2024-05-03T18:11:50.898 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:49.441+0000 7ffbd7a00700 1 -- 172.21.15.33:0/2188677813 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbd01a24e0 con 0x7ffbd007ee40 2024-05-03T18:11:50.899 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:49.442+0000 7ffbd7a00700 1 -- 172.21.15.33:0/2188677813 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbd0064b00 con 0x7ffbd007ee40 2024-05-03T18:11:50.899 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:49.442+0000 7ffbcd600700 1 -- 172.21.15.33:0/2188677813 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7ffbc4022020 con 0x7ffbd007ee40 2024-05-03T18:11:50.899 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:49.442+0000 7ffbcd600700 1 --2- 172.21.15.33:0/2188677813 >> [v2:172.21.15.33:6800/2830135294,v1:172.21.15.33:6801/2830135294] conn(0x7ffbb803e4c0 0x7ffbb8040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:11:50.899 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:49.442+0000 7ffbcd600700 1 -- 172.21.15.33:0/2188677813 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ffbc4053a20 con 0x7ffbd007ee40 2024-05-03T18:11:50.899 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:49.443+0000 7ffbcfe00700 1 --2- 172.21.15.33:0/2188677813 >> [v2:172.21.15.33:6800/2830135294,v1:172.21.15.33:6801/2830135294] conn(0x7ffbb803e4c0 0x7ffbb8040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:11:50.899 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:49.443+0000 7ffbcfe00700 1 --2- 172.21.15.33:0/2188677813 >> [v2:172.21.15.33:6800/2830135294,v1:172.21.15.33:6801/2830135294] conn(0x7ffbb803e4c0 0x7ffbb8040980 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc006fd0 tx=0x7ffbbc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:11:50.899 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:49.448+0000 7ffbcd600700 1 -- 172.21.15.33:0/2188677813 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7ffbc40212c0 con 0x7ffbd007ee40 2024-05-03T18:11:50.899 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:49.681+0000 7ffbd7a00700 1 -- 172.21.15.33:0/2188677813 --> [v2:172.21.15.33:6800/2830135294,v1:172.21.15.33:6801/2830135294] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7ffbd002e580 con 0x7ffbb803e4c0 2024-05-03T18:11:50.899 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:50.187+0000 7ffbcd600700 1 -- 172.21.15.33:0/2188677813 <== mgr.14120 v2:172.21.15.33:6800/2830135294 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ffbd002e580 con 0x7ffbb803e4c0 2024-05-03T18:11:50.899 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:50.190+0000 7ffbd7a00700 1 -- 172.21.15.33:0/2188677813 >> [v2:172.21.15.33:6800/2830135294,v1:172.21.15.33:6801/2830135294] conn(0x7ffbb803e4c0 msgr2=0x7ffbb8040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:11:50.899 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:50.190+0000 7ffbd7a00700 1 --2- 172.21.15.33:0/2188677813 >> [v2:172.21.15.33:6800/2830135294,v1:172.21.15.33:6801/2830135294] conn(0x7ffbb803e4c0 0x7ffbb8040980 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc006fd0 tx=0x7ffbbc006e40 comp rx=0 tx=0).stop 2024-05-03T18:11:50.899 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:50.190+0000 7ffbd7a00700 1 -- 172.21.15.33:0/2188677813 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffbd007ee40 msgr2=0x7ffbd019f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:11:50.899 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:50.190+0000 7ffbd7a00700 1 --2- 172.21.15.33:0/2188677813 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffbd007ee40 0x7ffbd019f020 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ffbc4006b20 tx=0x7ffbc4011a60 comp rx=0 tx=0).stop 2024-05-03T18:11:50.900 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:50.190+0000 7ffbd7a00700 1 -- 172.21.15.33:0/2188677813 shutdown_connections 2024-05-03T18:11:50.900 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:50.190+0000 7ffbd7a00700 1 --2- 172.21.15.33:0/2188677813 >> [v2:172.21.15.33:6800/2830135294,v1:172.21.15.33:6801/2830135294] conn(0x7ffbb803e4c0 0x7ffbb8040980 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:50.900 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:50.190+0000 7ffbd7a00700 1 --2- 172.21.15.33:0/2188677813 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffbd007ee40 0x7ffbd019f020 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:50.900 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:50.190+0000 7ffbd7a00700 1 -- 172.21.15.33:0/2188677813 >> 172.21.15.33:0/2188677813 conn(0x7ffbd0104180 msgr2=0x7ffbd0104fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:11:50.900 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:50.191+0000 7ffbd7a00700 1 -- 172.21.15.33:0/2188677813 shutdown_connections 2024-05-03T18:11:50.900 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:50.191+0000 7ffbd7a00700 1 -- 172.21.15.33:0/2188677813 wait complete. 2024-05-03T18:11:51.466 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:51 smithi033 ceph-mon[28330]: from='mgr.14120 172.21.15.33:0/3290632043' entity='mgr.x' 2024-05-03T18:11:51.466 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:51 smithi033 ceph-mon[28330]: from='mgr.14120 172.21.15.33:0/3290632043' entity='mgr.x' 2024-05-03T18:11:52.380 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:52 smithi033 ceph-mon[28330]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:11:52.395 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC1lmQQo6k2LDBsuIGKtWYqKpOkX6uZZPnuLvYWFSsz1PyVFx7IxRC7vc5iYHtyToWFJMbLQ23lLsma4V9GWKwqbQ+/lYoz2DpkSfDeC3RHoa7d3Kh/Zql/VsFFsw0xH3gc85+iSKTMZUOGUxM34oiG+E1o6jbQ6bztCdkLBkdHQDkNXGOTIKYnguIBpUnU1tLVAjhrwqZrGI4EoKfQ0CYMAMKXMzXRMnsRJ/XLGoZTrDZJPbUWNxCU2qYcDsLoMpJiMxLuMitPg00+tYuzDEvm+3x+rxhF6z9p+iWWDpM9nNCiebNEQl8lRFjA5TweakujLZ53G/+OOqzHVYgA4UTdMtxCwoUpoUD1jmtaLu5lSbYaE7Zhqd9Ve1Kue14wh8DI8jxtKtl7+rxC4CDEFE1Snoq/ibCXvO0ROO7FWGATp6CNS29XvhxsMvbMxEXVmQss/faTrKDZ8LxN2qzmUv0qhTw2kUwSt4gVpJG7B5XgdOu0krKmKbYgubXDCIoaTCM= ceph-5703bba4-0978-11ef-bc97-c7b262605968 2024-05-03T18:11:52.396 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:51.466+0000 7f1106e00700 1 Processor -- start 2024-05-03T18:11:52.396 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:51.466+0000 7f1106e00700 1 -- start start 2024-05-03T18:11:52.396 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:51.466+0000 7f1106e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f110010fc70 0x7f1100112060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:11:52.396 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:51.466+0000 7f1106e00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1100083480 con 0x7f110010fc70 2024-05-03T18:11:52.397 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:51.467+0000 7f10ffe00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f110010fc70 0x7f1100112060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:11:52.397 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:51.467+0000 7f10ffe00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f110010fc70 0x7f1100112060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55350/0 (socket says 172.21.15.33:55350) 2024-05-03T18:11:52.397 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:51.467+0000 7f10ffe00700 1 -- 172.21.15.33:0/1445515098 learned_addr learned my addr 172.21.15.33:0/1445515098 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:11:52.397 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:51.467+0000 7f10ffe00700 1 -- 172.21.15.33:0/1445515098 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11001125a0 con 0x7f110010fc70 2024-05-03T18:11:52.397 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:51.468+0000 7f10ffe00700 1 --2- 172.21.15.33:0/1445515098 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f110010fc70 0x7f1100112060 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f10f00098a0 tx=0x7f10f0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da26becb62d8b995 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:11:52.397 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:51.468+0000 7f10fea00700 1 -- 172.21.15.33:0/1445515098 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f10f0004160 con 0x7f110010fc70 2024-05-03T18:11:52.399 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:51.468+0000 7f10fea00700 1 -- 172.21.15.33:0/1445515098 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f10f0004300 con 0x7f110010fc70 2024-05-03T18:11:52.399 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:51.468+0000 7f10fea00700 1 -- 172.21.15.33:0/1445515098 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f10f00119e0 con 0x7f110010fc70 2024-05-03T18:11:52.399 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:51.468+0000 7f1106e00700 1 -- 172.21.15.33:0/1445515098 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f110010fc70 msgr2=0x7f1100112060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:11:52.399 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:51.469+0000 7f1106e00700 1 --2- 172.21.15.33:0/1445515098 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f110010fc70 0x7f1100112060 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f10f00098a0 tx=0x7f10f0009bb0 comp rx=0 tx=0).stop 2024-05-03T18:11:52.399 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:51.469+0000 7f1106e00700 1 -- 172.21.15.33:0/1445515098 shutdown_connections 2024-05-03T18:11:52.399 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:51.469+0000 7f1106e00700 1 --2- 172.21.15.33:0/1445515098 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f110010fc70 0x7f1100112060 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:52.399 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:51.469+0000 7f1106e00700 1 -- 172.21.15.33:0/1445515098 >> 172.21.15.33:0/1445515098 conn(0x7f110007e1e0 msgr2=0x7f1100080620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:11:52.399 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:51.469+0000 7f1106e00700 1 -- 172.21.15.33:0/1445515098 shutdown_connections 2024-05-03T18:11:52.399 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:51.469+0000 7f1106e00700 1 -- 172.21.15.33:0/1445515098 wait complete. 2024-05-03T18:11:52.399 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:51.470+0000 7f1106e00700 1 Processor -- start 2024-05-03T18:11:52.399 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:51.470+0000 7f1106e00700 1 -- start start 2024-05-03T18:11:52.401 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:51.471+0000 7f1106e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f110010fc70 0x7f11001a3410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:11:52.401 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:51.471+0000 7f1106e00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1100113560 con 0x7f110010fc70 2024-05-03T18:11:52.401 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:51.471+0000 7f10ffe00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f110010fc70 0x7f11001a3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:11:52.401 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:51.471+0000 7f10ffe00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f110010fc70 0x7f11001a3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55362/0 (socket says 172.21.15.33:55362) 2024-05-03T18:11:52.401 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:51.471+0000 7f10ffe00700 1 -- 172.21.15.33:0/2863683582 learned_addr learned my addr 172.21.15.33:0/2863683582 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:11:52.402 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:51.471+0000 7f10ffe00700 1 -- 172.21.15.33:0/2863683582 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11001a3950 con 0x7f110010fc70 2024-05-03T18:11:52.402 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:51.472+0000 7f10ffe00700 1 --2- 172.21.15.33:0/2863683582 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f110010fc70 0x7f11001a3410 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f10f00116a0 tx=0x7f10f00046a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:11:52.402 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:51.472+0000 7f10fcc00700 1 -- 172.21.15.33:0/2863683582 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f10f0003b70 con 0x7f110010fc70 2024-05-03T18:11:52.402 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:51.472+0000 7f10fcc00700 1 -- 172.21.15.33:0/2863683582 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f10f001a8a0 con 0x7f110010fc70 2024-05-03T18:11:52.402 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:51.472+0000 7f10fcc00700 1 -- 172.21.15.33:0/2863683582 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f10f0024890 con 0x7f110010fc70 2024-05-03T18:11:52.402 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:51.472+0000 7f1106e00700 1 -- 172.21.15.33:0/2863683582 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11001a3be0 con 0x7f110010fc70 2024-05-03T18:11:52.402 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:51.472+0000 7f1106e00700 1 -- 172.21.15.33:0/2863683582 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11001a6880 con 0x7f110010fc70 2024-05-03T18:11:52.402 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:51.473+0000 7f1106e00700 1 -- 172.21.15.33:0/2863683582 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1100064b00 con 0x7f110010fc70 2024-05-03T18:11:52.402 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:51.474+0000 7f10fcc00700 1 -- 172.21.15.33:0/2863683582 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f10f0022020 con 0x7f110010fc70 2024-05-03T18:11:52.402 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:51.474+0000 7f10fcc00700 1 --2- 172.21.15.33:0/2863683582 >> [v2:172.21.15.33:6800/2830135294,v1:172.21.15.33:6801/2830135294] conn(0x7f10e403e4c0 0x7f10e4040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:11:52.404 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:51.474+0000 7f10fcc00700 1 -- 172.21.15.33:0/2863683582 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f10f0053b80 con 0x7f110010fc70 2024-05-03T18:11:52.404 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:51.474+0000 7f10ff400700 1 --2- 172.21.15.33:0/2863683582 >> [v2:172.21.15.33:6800/2830135294,v1:172.21.15.33:6801/2830135294] conn(0x7f10e403e4c0 0x7f10e4040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:11:52.404 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:51.475+0000 7f10ff400700 1 --2- 172.21.15.33:0/2863683582 >> [v2:172.21.15.33:6800/2830135294,v1:172.21.15.33:6801/2830135294] conn(0x7f10e403e4c0 0x7f10e4040980 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f10ec006fd0 tx=0x7f10ec006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:11:52.404 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:51.480+0000 7f10fcc00700 1 -- 172.21.15.33:0/2863683582 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f10f0017e90 con 0x7f110010fc70 2024-05-03T18:11:52.404 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:51.717+0000 7f1106e00700 1 -- 172.21.15.33:0/2863683582 --> [v2:172.21.15.33:6800/2830135294,v1:172.21.15.33:6801/2830135294] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f110002e580 con 0x7f10e403e4c0 2024-05-03T18:11:52.405 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:51.719+0000 7f10fcc00700 1 -- 172.21.15.33:0/2863683582 <== mgr.14120 v2:172.21.15.33:6800/2830135294 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f110002e580 con 0x7f10e403e4c0 2024-05-03T18:11:52.405 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:51.722+0000 7f1106e00700 1 -- 172.21.15.33:0/2863683582 >> [v2:172.21.15.33:6800/2830135294,v1:172.21.15.33:6801/2830135294] conn(0x7f10e403e4c0 msgr2=0x7f10e4040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:11:52.405 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:51.722+0000 7f1106e00700 1 --2- 172.21.15.33:0/2863683582 >> [v2:172.21.15.33:6800/2830135294,v1:172.21.15.33:6801/2830135294] conn(0x7f10e403e4c0 0x7f10e4040980 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f10ec006fd0 tx=0x7f10ec006e40 comp rx=0 tx=0).stop 2024-05-03T18:11:52.405 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:51.722+0000 7f1106e00700 1 -- 172.21.15.33:0/2863683582 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f110010fc70 msgr2=0x7f11001a3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:11:52.405 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:51.722+0000 7f1106e00700 1 --2- 172.21.15.33:0/2863683582 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f110010fc70 0x7f11001a3410 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f10f00116a0 tx=0x7f10f00046a0 comp rx=0 tx=0).stop 2024-05-03T18:11:52.405 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:51.723+0000 7f1106e00700 1 -- 172.21.15.33:0/2863683582 shutdown_connections 2024-05-03T18:11:52.405 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:51.723+0000 7f1106e00700 1 --2- 172.21.15.33:0/2863683582 >> [v2:172.21.15.33:6800/2830135294,v1:172.21.15.33:6801/2830135294] conn(0x7f10e403e4c0 0x7f10e4040980 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:52.405 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:51.723+0000 7f1106e00700 1 --2- 172.21.15.33:0/2863683582 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f110010fc70 0x7f11001a3410 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:11:52.405 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:51.723+0000 7f1106e00700 1 -- 172.21.15.33:0/2863683582 >> 172.21.15.33:0/2863683582 conn(0x7f110007e1e0 msgr2=0x7f110007eed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:11:52.405 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:51.723+0000 7f1106e00700 1 -- 172.21.15.33:0/2863683582 shutdown_connections 2024-05-03T18:11:52.405 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:51.723+0000 7f1106e00700 1 -- 172.21.15.33:0/2863683582 wait complete. 2024-05-03T18:11:52.406 INFO:teuthology.orchestra.run.smithi033.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-05-03T18:11:52.406 INFO:teuthology.orchestra.run.smithi033.stdout:Adding key to root@localhost authorized_keys... 2024-05-03T18:11:52.406 INFO:teuthology.orchestra.run.smithi033.stdout:Adding host smithi033... 2024-05-03T18:11:54.232 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:53 smithi033 ceph-mon[28330]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi033", "addr": "172.21.15.33", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:11:54.982 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:11:54 smithi033 ceph-mon[28330]: Deploying cephadm binary to smithi033 2024-05-03T18:12:00.422 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout Added host 'smithi033' with addr '172.21.15.33' 2024-05-03T18:12:00.422 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:53.112+0000 7f57bba00700 1 Processor -- start 2024-05-03T18:12:00.422 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:53.113+0000 7f57bba00700 1 -- start start 2024-05-03T18:12:00.422 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:53.113+0000 7f57bba00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57b4108bf0 0x7f57b4109010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:12:00.422 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:53.113+0000 7f57bba00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f57b41095e0 con 0x7f57b4108bf0 2024-05-03T18:12:00.423 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:53.113+0000 7f57b9000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57b4108bf0 0x7f57b4109010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:12:00.423 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:53.114+0000 7f57b9000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57b4108bf0 0x7f57b4109010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55366/0 (socket says 172.21.15.33:55366) 2024-05-03T18:12:00.423 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:53.114+0000 7f57b9000700 1 -- 172.21.15.33:0/1364714445 learned_addr learned my addr 172.21.15.33:0/1364714445 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:12:00.423 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:53.114+0000 7f57b9000700 1 -- 172.21.15.33:0/1364714445 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57b4109da0 con 0x7f57b4108bf0 2024-05-03T18:12:00.423 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:53.114+0000 7f57b9000700 1 --2- 172.21.15.33:0/1364714445 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57b4108bf0 0x7f57b4109010 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f57a80098a0 tx=0x7f57a8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36ae952ba4cd1c76 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:12:00.423 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:53.115+0000 7f57b3400700 1 -- 172.21.15.33:0/1364714445 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f57a8004160 con 0x7f57b4108bf0 2024-05-03T18:12:00.423 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:53.115+0000 7f57b3400700 1 -- 172.21.15.33:0/1364714445 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f57a8004300 con 0x7f57b4108bf0 2024-05-03T18:12:00.423 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:53.115+0000 7f57b3400700 1 -- 172.21.15.33:0/1364714445 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f57a80119e0 con 0x7f57b4108bf0 2024-05-03T18:12:00.423 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:53.115+0000 7f57bba00700 1 -- 172.21.15.33:0/1364714445 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57b4108bf0 msgr2=0x7f57b4109010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:12:00.423 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:53.115+0000 7f57bba00700 1 --2- 172.21.15.33:0/1364714445 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57b4108bf0 0x7f57b4109010 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f57a80098a0 tx=0x7f57a8009bb0 comp rx=0 tx=0).stop 2024-05-03T18:12:00.423 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:53.116+0000 7f57bba00700 1 -- 172.21.15.33:0/1364714445 shutdown_connections 2024-05-03T18:12:00.423 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:53.116+0000 7f57bba00700 1 --2- 172.21.15.33:0/1364714445 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57b4108bf0 0x7f57b4109010 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:12:00.423 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:53.116+0000 7f57bba00700 1 -- 172.21.15.33:0/1364714445 >> 172.21.15.33:0/1364714445 conn(0x7f57b41041b0 msgr2=0x7f57b41065d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:12:00.424 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:53.116+0000 7f57bba00700 1 -- 172.21.15.33:0/1364714445 shutdown_connections 2024-05-03T18:12:00.424 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:53.116+0000 7f57bba00700 1 -- 172.21.15.33:0/1364714445 wait complete. 2024-05-03T18:12:00.424 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:53.117+0000 7f57bba00700 1 Processor -- start 2024-05-03T18:12:00.424 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:53.117+0000 7f57bba00700 1 -- start start 2024-05-03T18:12:00.424 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:53.117+0000 7f57bba00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57b4108bf0 0x7f57b407eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:12:00.424 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:53.118+0000 7f57bba00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f57b410cfd0 con 0x7f57b4108bf0 2024-05-03T18:12:00.424 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:53.118+0000 7f57b9000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57b4108bf0 0x7f57b407eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:12:00.424 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:53.118+0000 7f57b9000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57b4108bf0 0x7f57b407eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55372/0 (socket says 172.21.15.33:55372) 2024-05-03T18:12:00.424 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:53.118+0000 7f57b9000700 1 -- 172.21.15.33:0/3402389607 learned_addr learned my addr 172.21.15.33:0/3402389607 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:12:00.424 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:53.118+0000 7f57b9000700 1 -- 172.21.15.33:0/3402389607 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57b407f050 con 0x7f57b4108bf0 2024-05-03T18:12:00.424 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:53.118+0000 7f57b9000700 1 --2- 172.21.15.33:0/3402389607 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57b4108bf0 0x7f57b407eb10 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f57a80119b0 tx=0x7f57a8011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:12:00.424 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:53.119+0000 7f57b1600700 1 -- 172.21.15.33:0/3402389607 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f57a8003b70 con 0x7f57b4108bf0 2024-05-03T18:12:00.424 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:53.119+0000 7f57b1600700 1 -- 172.21.15.33:0/3402389607 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f57a801a8a0 con 0x7f57b4108bf0 2024-05-03T18:12:00.425 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:53.119+0000 7f57b1600700 1 -- 172.21.15.33:0/3402389607 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f57a8024830 con 0x7f57b4108bf0 2024-05-03T18:12:00.425 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:53.119+0000 7f57bba00700 1 -- 172.21.15.33:0/3402389607 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57b407d240 con 0x7f57b4108bf0 2024-05-03T18:12:00.425 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:53.119+0000 7f57bba00700 1 -- 172.21.15.33:0/3402389607 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57b407d780 con 0x7f57b4108bf0 2024-05-03T18:12:00.425 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:53.120+0000 7f57b1600700 1 -- 172.21.15.33:0/3402389607 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f57a8022020 con 0x7f57b4108bf0 2024-05-03T18:12:00.425 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:53.120+0000 7f57bba00700 1 -- 172.21.15.33:0/3402389607 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57b4064b00 con 0x7f57b4108bf0 2024-05-03T18:12:00.425 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:53.121+0000 7f57b1600700 1 --2- 172.21.15.33:0/3402389607 >> [v2:172.21.15.33:6800/2830135294,v1:172.21.15.33:6801/2830135294] conn(0x7f579c03e470 0x7f579c040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:12:00.425 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:53.121+0000 7f57b1600700 1 -- 172.21.15.33:0/3402389607 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f57a80539e0 con 0x7f57b4108bf0 2024-05-03T18:12:00.425 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:53.121+0000 7f57b3e00700 1 --2- 172.21.15.33:0/3402389607 >> [v2:172.21.15.33:6800/2830135294,v1:172.21.15.33:6801/2830135294] conn(0x7f579c03e470 0x7f579c040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:12:00.425 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:53.122+0000 7f57b3e00700 1 --2- 172.21.15.33:0/3402389607 >> [v2:172.21.15.33:6800/2830135294,v1:172.21.15.33:6801/2830135294] conn(0x7f579c03e470 0x7f579c040930 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f57a0006fd0 tx=0x7f57a0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:12:00.425 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:53.126+0000 7f57b1600700 1 -- 172.21.15.33:0/3402389607 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f57a80212c0 con 0x7f57b4108bf0 2024-05-03T18:12:00.425 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:53.368+0000 7f57bba00700 1 -- 172.21.15.33:0/3402389607 --> [v2:172.21.15.33:6800/2830135294,v1:172.21.15.33:6801/2830135294] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi033", "addr": "172.21.15.33", "target": ["mon-mgr", ""]}) -- 0x7f57b4196540 con 0x7f579c03e470 2024-05-03T18:12:00.425 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:59.691+0000 7f57b1600700 1 -- 172.21.15.33:0/3402389607 <== mgr.14120 v2:172.21.15.33:6800/2830135294 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f57b4196540 con 0x7f579c03e470 2024-05-03T18:12:00.425 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:59.696+0000 7f57bba00700 1 -- 172.21.15.33:0/3402389607 >> [v2:172.21.15.33:6800/2830135294,v1:172.21.15.33:6801/2830135294] conn(0x7f579c03e470 msgr2=0x7f579c040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:12:00.425 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:59.696+0000 7f57bba00700 1 --2- 172.21.15.33:0/3402389607 >> [v2:172.21.15.33:6800/2830135294,v1:172.21.15.33:6801/2830135294] conn(0x7f579c03e470 0x7f579c040930 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f57a0006fd0 tx=0x7f57a0006e40 comp rx=0 tx=0).stop 2024-05-03T18:12:00.426 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:59.696+0000 7f57bba00700 1 -- 172.21.15.33:0/3402389607 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57b4108bf0 msgr2=0x7f57b407eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:12:00.426 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:59.696+0000 7f57bba00700 1 --2- 172.21.15.33:0/3402389607 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57b4108bf0 0x7f57b407eb10 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f57a80119b0 tx=0x7f57a8011a90 comp rx=0 tx=0).stop 2024-05-03T18:12:00.426 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:59.696+0000 7f57bba00700 1 -- 172.21.15.33:0/3402389607 shutdown_connections 2024-05-03T18:12:00.426 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:59.696+0000 7f57bba00700 1 --2- 172.21.15.33:0/3402389607 >> [v2:172.21.15.33:6800/2830135294,v1:172.21.15.33:6801/2830135294] conn(0x7f579c03e470 0x7f579c040930 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:12:00.426 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:59.696+0000 7f57bba00700 1 --2- 172.21.15.33:0/3402389607 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57b4108bf0 0x7f57b407eb10 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:12:00.426 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:59.696+0000 7f57bba00700 1 -- 172.21.15.33:0/3402389607 >> 172.21.15.33:0/3402389607 conn(0x7f57b41041b0 msgr2=0x7f57b4104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:12:00.426 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:59.697+0000 7f57bba00700 1 -- 172.21.15.33:0/3402389607 shutdown_connections 2024-05-03T18:12:00.426 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:11:59.697+0000 7f57bba00700 1 -- 172.21.15.33:0/3402389607 wait complete. 2024-05-03T18:12:00.426 INFO:teuthology.orchestra.run.smithi033.stdout:Deploying unmanaged mon service... 2024-05-03T18:12:00.938 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:12:00 smithi033 ceph-mon[28330]: from='mgr.14120 172.21.15.33:0/3290632043' entity='mgr.x' 2024-05-03T18:12:00.938 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:12:00 smithi033 ceph-mon[28330]: Added host smithi033 2024-05-03T18:12:00.938 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:12:00 smithi033 ceph-mon[28330]: from='mgr.14120 172.21.15.33:0/3290632043' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:12:02.644 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:12:02 smithi033 ceph-mon[28330]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:12:02.645 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:12:02 smithi033 ceph-mon[28330]: Saving service mon spec with placement count:5 2024-05-03T18:12:02.645 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:12:02 smithi033 ceph-mon[28330]: from='mgr.14120 172.21.15.33:0/3290632043' entity='mgr.x' 2024-05-03T18:12:03.026 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-05-03T18:12:03.026 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:01.102+0000 7f37dec00700 1 Processor -- start 2024-05-03T18:12:03.026 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:01.102+0000 7f37dec00700 1 -- start start 2024-05-03T18:12:03.026 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:01.103+0000 7f37dec00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37d810ae40 0x7f37d810b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:12:03.026 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:01.103+0000 7f37dec00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f37d810b830 con 0x7f37d810ae40 2024-05-03T18:12:03.026 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:01.104+0000 7f37d7e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37d810ae40 0x7f37d810b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:12:03.026 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:01.104+0000 7f37d7e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37d810ae40 0x7f37d810b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48150/0 (socket says 172.21.15.33:48150) 2024-05-03T18:12:03.026 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:01.104+0000 7f37d7e00700 1 -- 172.21.15.33:0/650790195 learned_addr learned my addr 172.21.15.33:0/650790195 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:12:03.026 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:01.105+0000 7f37d7e00700 1 -- 172.21.15.33:0/650790195 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37d810bff0 con 0x7f37d810ae40 2024-05-03T18:12:03.026 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:01.105+0000 7f37d7e00700 1 --2- 172.21.15.33:0/650790195 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37d810ae40 0x7f37d810b260 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f37c80098a0 tx=0x7f37c8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3da002e431bc8610 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:12:03.027 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:01.105+0000 7f37d6a00700 1 -- 172.21.15.33:0/650790195 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f37c8004160 con 0x7f37d810ae40 2024-05-03T18:12:03.027 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:01.105+0000 7f37d6a00700 1 -- 172.21.15.33:0/650790195 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f37c8004300 con 0x7f37d810ae40 2024-05-03T18:12:03.027 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:01.105+0000 7f37d6a00700 1 -- 172.21.15.33:0/650790195 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f37c80119e0 con 0x7f37d810ae40 2024-05-03T18:12:03.027 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:01.106+0000 7f37dec00700 1 -- 172.21.15.33:0/650790195 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37d810ae40 msgr2=0x7f37d810b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:12:03.027 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:01.106+0000 7f37dec00700 1 --2- 172.21.15.33:0/650790195 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37d810ae40 0x7f37d810b260 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f37c80098a0 tx=0x7f37c8009bb0 comp rx=0 tx=0).stop 2024-05-03T18:12:03.027 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:01.106+0000 7f37dec00700 1 -- 172.21.15.33:0/650790195 shutdown_connections 2024-05-03T18:12:03.027 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:01.106+0000 7f37dec00700 1 --2- 172.21.15.33:0/650790195 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37d810ae40 0x7f37d810b260 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:12:03.027 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:01.106+0000 7f37dec00700 1 -- 172.21.15.33:0/650790195 >> 172.21.15.33:0/650790195 conn(0x7f37d807e230 msgr2=0x7f37d807e640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:12:03.027 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:01.106+0000 7f37dec00700 1 -- 172.21.15.33:0/650790195 shutdown_connections 2024-05-03T18:12:03.027 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:01.106+0000 7f37dec00700 1 -- 172.21.15.33:0/650790195 wait complete. 2024-05-03T18:12:03.027 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:01.107+0000 7f37dec00700 1 Processor -- start 2024-05-03T18:12:03.027 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:01.108+0000 7f37dec00700 1 -- start start 2024-05-03T18:12:03.027 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:01.108+0000 7f37dec00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37d810ae40 0x7f37d819efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:12:03.027 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:01.108+0000 7f37dec00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f37d810f220 con 0x7f37d810ae40 2024-05-03T18:12:03.027 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:01.108+0000 7f37d7e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37d810ae40 0x7f37d819efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:12:03.027 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:01.108+0000 7f37d7e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37d810ae40 0x7f37d819efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48166/0 (socket says 172.21.15.33:48166) 2024-05-03T18:12:03.028 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:01.108+0000 7f37d7e00700 1 -- 172.21.15.33:0/3045755187 learned_addr learned my addr 172.21.15.33:0/3045755187 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:12:03.028 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:01.109+0000 7f37d7e00700 1 -- 172.21.15.33:0/3045755187 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37d819f510 con 0x7f37d810ae40 2024-05-03T18:12:03.028 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:01.109+0000 7f37d7e00700 1 --2- 172.21.15.33:0/3045755187 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37d810ae40 0x7f37d819efd0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f37c8006b20 tx=0x7f37c8011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:12:03.028 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:01.109+0000 7f37d4c00700 1 -- 172.21.15.33:0/3045755187 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f37c8003a60 con 0x7f37d810ae40 2024-05-03T18:12:03.028 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:01.109+0000 7f37d4c00700 1 -- 172.21.15.33:0/3045755187 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f37c8003c00 con 0x7f37d810ae40 2024-05-03T18:12:03.028 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:01.109+0000 7f37d4c00700 1 -- 172.21.15.33:0/3045755187 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f37c8024780 con 0x7f37d810ae40 2024-05-03T18:12:03.028 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:01.109+0000 7f37dec00700 1 -- 172.21.15.33:0/3045755187 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37d819f7a0 con 0x7f37d810ae40 2024-05-03T18:12:03.028 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:01.109+0000 7f37dec00700 1 -- 172.21.15.33:0/3045755187 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37d81a2490 con 0x7f37d810ae40 2024-05-03T18:12:03.028 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:01.111+0000 7f37dec00700 1 -- 172.21.15.33:0/3045755187 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37d8064b00 con 0x7f37d810ae40 2024-05-03T18:12:03.028 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:01.111+0000 7f37d4c00700 1 -- 172.21.15.33:0/3045755187 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f37c8022020 con 0x7f37d810ae40 2024-05-03T18:12:03.028 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:01.111+0000 7f37d4c00700 1 --2- 172.21.15.33:0/3045755187 >> [v2:172.21.15.33:6800/2830135294,v1:172.21.15.33:6801/2830135294] conn(0x7f37bc03e4c0 0x7f37bc040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:12:03.028 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:01.111+0000 7f37d4c00700 1 -- 172.21.15.33:0/3045755187 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f37c8053ac0 con 0x7f37d810ae40 2024-05-03T18:12:03.028 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:01.112+0000 7f37d7400700 1 --2- 172.21.15.33:0/3045755187 >> [v2:172.21.15.33:6800/2830135294,v1:172.21.15.33:6801/2830135294] conn(0x7f37bc03e4c0 0x7f37bc040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:12:03.028 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:01.112+0000 7f37d7400700 1 --2- 172.21.15.33:0/3045755187 >> [v2:172.21.15.33:6800/2830135294,v1:172.21.15.33:6801/2830135294] conn(0x7f37bc03e4c0 0x7f37bc040980 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f37c4006fd0 tx=0x7f37c4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:12:03.029 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:01.117+0000 7f37d4c00700 1 -- 172.21.15.33:0/3045755187 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f37c80212c0 con 0x7f37d810ae40 2024-05-03T18:12:03.029 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:01.423+0000 7f37dec00700 1 -- 172.21.15.33:0/3045755187 --> [v2:172.21.15.33:6800/2830135294,v1:172.21.15.33:6801/2830135294] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f37d802e580 con 0x7f37bc03e4c0 2024-05-03T18:12:03.029 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:01.435+0000 7f37d4c00700 1 -- 172.21.15.33:0/3045755187 <== mgr.14120 v2:172.21.15.33:6800/2830135294 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f37d802e580 con 0x7f37bc03e4c0 2024-05-03T18:12:03.029 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:01.438+0000 7f37dec00700 1 -- 172.21.15.33:0/3045755187 >> [v2:172.21.15.33:6800/2830135294,v1:172.21.15.33:6801/2830135294] conn(0x7f37bc03e4c0 msgr2=0x7f37bc040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:12:03.029 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:01.438+0000 7f37dec00700 1 --2- 172.21.15.33:0/3045755187 >> [v2:172.21.15.33:6800/2830135294,v1:172.21.15.33:6801/2830135294] conn(0x7f37bc03e4c0 0x7f37bc040980 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f37c4006fd0 tx=0x7f37c4006e40 comp rx=0 tx=0).stop 2024-05-03T18:12:03.029 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:01.438+0000 7f37dec00700 1 -- 172.21.15.33:0/3045755187 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37d810ae40 msgr2=0x7f37d819efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:12:03.029 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:01.438+0000 7f37dec00700 1 --2- 172.21.15.33:0/3045755187 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37d810ae40 0x7f37d819efd0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f37c8006b20 tx=0x7f37c8011a60 comp rx=0 tx=0).stop 2024-05-03T18:12:03.029 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:01.438+0000 7f37dec00700 1 -- 172.21.15.33:0/3045755187 shutdown_connections 2024-05-03T18:12:03.029 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:01.438+0000 7f37dec00700 1 --2- 172.21.15.33:0/3045755187 >> [v2:172.21.15.33:6800/2830135294,v1:172.21.15.33:6801/2830135294] conn(0x7f37bc03e4c0 0x7f37bc040980 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:12:03.029 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:01.438+0000 7f37dec00700 1 --2- 172.21.15.33:0/3045755187 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37d810ae40 0x7f37d819efd0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:12:03.029 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:01.438+0000 7f37dec00700 1 -- 172.21.15.33:0/3045755187 >> 172.21.15.33:0/3045755187 conn(0x7f37d807e230 msgr2=0x7f37d8108660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:12:03.029 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:01.438+0000 7f37dec00700 1 -- 172.21.15.33:0/3045755187 shutdown_connections 2024-05-03T18:12:03.029 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:01.439+0000 7f37dec00700 1 -- 172.21.15.33:0/3045755187 wait complete. 2024-05-03T18:12:03.029 INFO:teuthology.orchestra.run.smithi033.stdout:Deploying unmanaged mgr service... 2024-05-03T18:12:03.857 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:12:03 smithi033 ceph-mon[28330]: from='mgr.14120 172.21.15.33:0/3290632043' entity='mgr.x' 2024-05-03T18:12:03.857 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:12:03 smithi033 ceph-mon[28330]: from='mgr.14120 172.21.15.33:0/3290632043' entity='mgr.x' 2024-05-03T18:12:04.538 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-05-03T18:12:04.539 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:03.581+0000 7fa568e00700 1 Processor -- start 2024-05-03T18:12:04.539 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:03.581+0000 7fa568e00700 1 -- start start 2024-05-03T18:12:04.539 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:03.582+0000 7fa568e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa564107ac0 0x7fa564107ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:12:04.539 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:03.582+0000 7fa568e00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa5641084b0 con 0x7fa564107ac0 2024-05-03T18:12:04.539 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:03.582+0000 7fa561e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa564107ac0 0x7fa564107ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:12:04.542 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:03.582+0000 7fa561e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa564107ac0 0x7fa564107ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48182/0 (socket says 172.21.15.33:48182) 2024-05-03T18:12:04.542 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:03.582+0000 7fa561e00700 1 -- 172.21.15.33:0/715802905 learned_addr learned my addr 172.21.15.33:0/715802905 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:12:04.542 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:03.583+0000 7fa561e00700 1 -- 172.21.15.33:0/715802905 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa564108cc0 con 0x7fa564107ac0 2024-05-03T18:12:04.542 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:03.583+0000 7fa561e00700 1 --2- 172.21.15.33:0/715802905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa564107ac0 0x7fa564107ee0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa5540098a0 tx=0x7fa554009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=887499841c4543ab server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:12:04.542 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:03.583+0000 7fa560a00700 1 -- 172.21.15.33:0/715802905 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa554004160 con 0x7fa564107ac0 2024-05-03T18:12:04.542 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:03.584+0000 7fa560a00700 1 -- 172.21.15.33:0/715802905 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa554004300 con 0x7fa564107ac0 2024-05-03T18:12:04.542 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:03.584+0000 7fa560a00700 1 -- 172.21.15.33:0/715802905 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa554011960 con 0x7fa564107ac0 2024-05-03T18:12:04.542 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:03.584+0000 7fa568e00700 1 -- 172.21.15.33:0/715802905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa564107ac0 msgr2=0x7fa564107ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:12:04.542 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:03.584+0000 7fa568e00700 1 --2- 172.21.15.33:0/715802905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa564107ac0 0x7fa564107ee0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa5540098a0 tx=0x7fa554009bb0 comp rx=0 tx=0).stop 2024-05-03T18:12:04.542 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:03.585+0000 7fa568e00700 1 -- 172.21.15.33:0/715802905 shutdown_connections 2024-05-03T18:12:04.543 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:03.585+0000 7fa568e00700 1 --2- 172.21.15.33:0/715802905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa564107ac0 0x7fa564107ee0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:12:04.543 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:03.585+0000 7fa568e00700 1 -- 172.21.15.33:0/715802905 >> 172.21.15.33:0/715802905 conn(0x7fa5641030a0 msgr2=0x7fa5641054c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:12:04.543 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:03.585+0000 7fa568e00700 1 -- 172.21.15.33:0/715802905 shutdown_connections 2024-05-03T18:12:04.543 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:03.585+0000 7fa568e00700 1 -- 172.21.15.33:0/715802905 wait complete. 2024-05-03T18:12:04.543 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:03.586+0000 7fa568e00700 1 Processor -- start 2024-05-03T18:12:04.543 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:03.586+0000 7fa568e00700 1 -- start start 2024-05-03T18:12:04.543 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:03.586+0000 7fa568e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa564107ac0 0x7fa564192bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:12:04.543 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:03.586+0000 7fa568e00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa56410bef0 con 0x7fa564107ac0 2024-05-03T18:12:04.543 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:03.587+0000 7fa561e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa564107ac0 0x7fa564192bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:12:04.543 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:03.587+0000 7fa561e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa564107ac0 0x7fa564192bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48184/0 (socket says 172.21.15.33:48184) 2024-05-03T18:12:04.543 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:03.587+0000 7fa561e00700 1 -- 172.21.15.33:0/224653718 learned_addr learned my addr 172.21.15.33:0/224653718 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:12:04.543 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:03.587+0000 7fa561e00700 1 -- 172.21.15.33:0/224653718 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5641930f0 con 0x7fa564107ac0 2024-05-03T18:12:04.543 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:03.588+0000 7fa561e00700 1 --2- 172.21.15.33:0/224653718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa564107ac0 0x7fa564192bb0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa554011670 tx=0x7fa554011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:12:04.543 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:03.588+0000 7fa55aa00700 1 -- 172.21.15.33:0/224653718 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa554003a60 con 0x7fa564107ac0 2024-05-03T18:12:04.543 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:03.588+0000 7fa55aa00700 1 -- 172.21.15.33:0/224653718 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa55401b8d0 con 0x7fa564107ac0 2024-05-03T18:12:04.544 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:03.588+0000 7fa55aa00700 1 -- 172.21.15.33:0/224653718 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa554023790 con 0x7fa564107ac0 2024-05-03T18:12:04.544 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:03.588+0000 7fa568e00700 1 -- 172.21.15.33:0/224653718 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5641912e0 con 0x7fa564107ac0 2024-05-03T18:12:04.544 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:03.588+0000 7fa568e00700 1 -- 172.21.15.33:0/224653718 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa564191820 con 0x7fa564107ac0 2024-05-03T18:12:04.546 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:03.590+0000 7fa55aa00700 1 -- 172.21.15.33:0/224653718 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fa55401b440 con 0x7fa564107ac0 2024-05-03T18:12:04.546 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:03.590+0000 7fa568e00700 1 -- 172.21.15.33:0/224653718 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa56410c070 con 0x7fa564107ac0 2024-05-03T18:12:04.546 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:03.590+0000 7fa55aa00700 1 --2- 172.21.15.33:0/224653718 >> [v2:172.21.15.33:6800/2830135294,v1:172.21.15.33:6801/2830135294] conn(0x7fa54803e4c0 0x7fa548040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:12:04.546 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:03.590+0000 7fa55aa00700 1 -- 172.21.15.33:0/224653718 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa554053f50 con 0x7fa564107ac0 2024-05-03T18:12:04.546 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:03.591+0000 7fa561400700 1 --2- 172.21.15.33:0/224653718 >> [v2:172.21.15.33:6800/2830135294,v1:172.21.15.33:6801/2830135294] conn(0x7fa54803e4c0 0x7fa548040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:12:04.546 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:03.591+0000 7fa561400700 1 --2- 172.21.15.33:0/224653718 >> [v2:172.21.15.33:6800/2830135294,v1:172.21.15.33:6801/2830135294] conn(0x7fa54803e4c0 0x7fa548040980 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa550006fd0 tx=0x7fa550006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:12:04.546 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:03.597+0000 7fa55aa00700 1 -- 172.21.15.33:0/224653718 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fa55401f030 con 0x7fa564107ac0 2024-05-03T18:12:04.546 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:03.857+0000 7fa568e00700 1 -- 172.21.15.33:0/224653718 --> [v2:172.21.15.33:6800/2830135294,v1:172.21.15.33:6801/2830135294] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fa564196610 con 0x7fa54803e4c0 2024-05-03T18:12:04.546 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:03.864+0000 7fa55aa00700 1 -- 172.21.15.33:0/224653718 <== mgr.14120 v2:172.21.15.33:6800/2830135294 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fa564196610 con 0x7fa54803e4c0 2024-05-03T18:12:04.546 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:03.867+0000 7fa568e00700 1 -- 172.21.15.33:0/224653718 >> [v2:172.21.15.33:6800/2830135294,v1:172.21.15.33:6801/2830135294] conn(0x7fa54803e4c0 msgr2=0x7fa548040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:12:04.547 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:03.867+0000 7fa568e00700 1 --2- 172.21.15.33:0/224653718 >> [v2:172.21.15.33:6800/2830135294,v1:172.21.15.33:6801/2830135294] conn(0x7fa54803e4c0 0x7fa548040980 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa550006fd0 tx=0x7fa550006e40 comp rx=0 tx=0).stop 2024-05-03T18:12:04.547 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:03.867+0000 7fa568e00700 1 -- 172.21.15.33:0/224653718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa564107ac0 msgr2=0x7fa564192bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:12:04.547 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:03.867+0000 7fa568e00700 1 --2- 172.21.15.33:0/224653718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa564107ac0 0x7fa564192bb0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa554011670 tx=0x7fa554011a30 comp rx=0 tx=0).stop 2024-05-03T18:12:04.547 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:03.867+0000 7fa568e00700 1 -- 172.21.15.33:0/224653718 shutdown_connections 2024-05-03T18:12:04.547 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:03.867+0000 7fa568e00700 1 --2- 172.21.15.33:0/224653718 >> [v2:172.21.15.33:6800/2830135294,v1:172.21.15.33:6801/2830135294] conn(0x7fa54803e4c0 0x7fa548040980 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:12:04.547 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:03.867+0000 7fa568e00700 1 --2- 172.21.15.33:0/224653718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa564107ac0 0x7fa564192bb0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:12:04.547 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:03.867+0000 7fa568e00700 1 -- 172.21.15.33:0/224653718 >> 172.21.15.33:0/224653718 conn(0x7fa5641030a0 msgr2=0x7fa564103f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:12:04.547 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:03.868+0000 7fa568e00700 1 -- 172.21.15.33:0/224653718 shutdown_connections 2024-05-03T18:12:04.547 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:03.868+0000 7fa568e00700 1 -- 172.21.15.33:0/224653718 wait complete. 2024-05-03T18:12:05.154 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:12:04 smithi033 ceph-mon[28330]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:12:05.154 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:12:04 smithi033 ceph-mon[28330]: Saving service mgr spec with placement count:2 2024-05-03T18:12:05.154 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:12:04 smithi033 ceph-mon[28330]: from='mgr.14120 172.21.15.33:0/3290632043' entity='mgr.x' 2024-05-03T18:12:06.232 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:12:05 smithi033 ceph-mon[28330]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:12:06.233 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:12:05 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/3654134693' entity='client.admin' 2024-05-03T18:12:06.586 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:05.154+0000 7f6df3000700 1 Processor -- start 2024-05-03T18:12:06.586 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:05.154+0000 7f6df3000700 1 -- start start 2024-05-03T18:12:06.586 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:05.155+0000 7f6df3000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6dec109480 0x7f6dec10b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:12:06.586 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:05.155+0000 7f6df3000700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6dec07ce50 con 0x7f6dec109480 2024-05-03T18:12:06.587 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:05.155+0000 7f6df1c00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6dec109480 0x7f6dec10b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:12:06.587 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:05.155+0000 7f6df1c00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6dec109480 0x7f6dec10b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48200/0 (socket says 172.21.15.33:48200) 2024-05-03T18:12:06.587 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:05.155+0000 7f6df1c00700 1 -- 172.21.15.33:0/1114333082 learned_addr learned my addr 172.21.15.33:0/1114333082 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:12:06.587 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:05.156+0000 7f6df1c00700 1 -- 172.21.15.33:0/1114333082 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dec10bdb0 con 0x7f6dec109480 2024-05-03T18:12:06.587 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:05.156+0000 7f6df1c00700 1 --2- 172.21.15.33:0/1114333082 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6dec109480 0x7f6dec10b870 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6de00098a0 tx=0x7f6de0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a30e660f8078dbd server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:12:06.587 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:05.157+0000 7f6debe00700 1 -- 172.21.15.33:0/1114333082 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6de0004160 con 0x7f6dec109480 2024-05-03T18:12:06.587 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:05.157+0000 7f6debe00700 1 -- 172.21.15.33:0/1114333082 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6de0004300 con 0x7f6dec109480 2024-05-03T18:12:06.587 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:05.157+0000 7f6debe00700 1 -- 172.21.15.33:0/1114333082 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6de0011960 con 0x7f6dec109480 2024-05-03T18:12:06.587 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:05.157+0000 7f6df3000700 1 -- 172.21.15.33:0/1114333082 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6dec109480 msgr2=0x7f6dec10b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:12:06.587 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:05.157+0000 7f6df3000700 1 --2- 172.21.15.33:0/1114333082 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6dec109480 0x7f6dec10b870 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6de00098a0 tx=0x7f6de0009bb0 comp rx=0 tx=0).stop 2024-05-03T18:12:06.587 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:05.158+0000 7f6df3000700 1 -- 172.21.15.33:0/1114333082 shutdown_connections 2024-05-03T18:12:06.587 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:05.158+0000 7f6df3000700 1 --2- 172.21.15.33:0/1114333082 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6dec109480 0x7f6dec10b870 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:12:06.587 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:05.158+0000 7f6df3000700 1 -- 172.21.15.33:0/1114333082 >> 172.21.15.33:0/1114333082 conn(0x7f6dec102ed0 msgr2=0x7f6dec1052f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:12:06.587 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:05.158+0000 7f6df3000700 1 -- 172.21.15.33:0/1114333082 shutdown_connections 2024-05-03T18:12:06.587 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:05.158+0000 7f6df3000700 1 -- 172.21.15.33:0/1114333082 wait complete. 2024-05-03T18:12:06.588 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:05.159+0000 7f6df3000700 1 Processor -- start 2024-05-03T18:12:06.588 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:05.160+0000 7f6df3000700 1 -- start start 2024-05-03T18:12:06.588 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:05.160+0000 7f6df3000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6dec109480 0x7f6dec19eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:12:06.588 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:05.160+0000 7f6df3000700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6dec10cd60 con 0x7f6dec109480 2024-05-03T18:12:06.588 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:05.160+0000 7f6df1c00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6dec109480 0x7f6dec19eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:12:06.588 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:05.160+0000 7f6df1c00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6dec109480 0x7f6dec19eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48210/0 (socket says 172.21.15.33:48210) 2024-05-03T18:12:06.588 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:05.160+0000 7f6df1c00700 1 -- 172.21.15.33:0/3654134693 learned_addr learned my addr 172.21.15.33:0/3654134693 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:12:06.588 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:05.161+0000 7f6df1c00700 1 -- 172.21.15.33:0/3654134693 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dec19f080 con 0x7f6dec109480 2024-05-03T18:12:06.588 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:05.161+0000 7f6df1c00700 1 --2- 172.21.15.33:0/3654134693 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6dec109480 0x7f6dec19eb40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6de0011670 tx=0x7f6de0011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:12:06.588 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:05.161+0000 7f6dea000700 1 -- 172.21.15.33:0/3654134693 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6de001f070 con 0x7f6dec109480 2024-05-03T18:12:06.588 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:05.162+0000 7f6dea000700 1 -- 172.21.15.33:0/3654134693 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6de0003b20 con 0x7f6dec109480 2024-05-03T18:12:06.588 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:05.162+0000 7f6dea000700 1 -- 172.21.15.33:0/3654134693 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6de00195f0 con 0x7f6dec109480 2024-05-03T18:12:06.588 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:05.162+0000 7f6df3000700 1 -- 172.21.15.33:0/3654134693 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dec19f310 con 0x7f6dec109480 2024-05-03T18:12:06.588 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:05.162+0000 7f6df3000700 1 -- 172.21.15.33:0/3654134693 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dec1a2000 con 0x7f6dec109480 2024-05-03T18:12:06.589 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:05.163+0000 7f6df3000700 1 -- 172.21.15.33:0/3654134693 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6dec064b00 con 0x7f6dec109480 2024-05-03T18:12:06.589 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:05.163+0000 7f6dea000700 1 -- 172.21.15.33:0/3654134693 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f6de0022070 con 0x7f6dec109480 2024-05-03T18:12:06.589 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:05.164+0000 7f6dea000700 1 --2- 172.21.15.33:0/3654134693 >> [v2:172.21.15.33:6800/2830135294,v1:172.21.15.33:6801/2830135294] conn(0x7f6dd403e510 0x7f6dd40409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:12:06.589 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:05.164+0000 7f6dea000700 1 -- 172.21.15.33:0/3654134693 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f6de00539d0 con 0x7f6dec109480 2024-05-03T18:12:06.589 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:05.164+0000 7f6df1200700 1 --2- 172.21.15.33:0/3654134693 >> [v2:172.21.15.33:6800/2830135294,v1:172.21.15.33:6801/2830135294] conn(0x7f6dd403e510 0x7f6dd40409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:12:06.589 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:05.165+0000 7f6df1200700 1 --2- 172.21.15.33:0/3654134693 >> [v2:172.21.15.33:6800/2830135294,v1:172.21.15.33:6801/2830135294] conn(0x7f6dd403e510 0x7f6dd40409d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f6dd8006fd0 tx=0x7f6dd8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:12:06.589 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:05.171+0000 7f6dea000700 1 -- 172.21.15.33:0/3654134693 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f6de002cd00 con 0x7f6dec109480 2024-05-03T18:12:06.589 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:05.418+0000 7f6df3000700 1 -- 172.21.15.33:0/3654134693 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f6dec1a2580 con 0x7f6dec109480 2024-05-03T18:12:06.589 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:05.422+0000 7f6dea000700 1 -- 172.21.15.33:0/3654134693 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f6de001baf0 con 0x7f6dec109480 2024-05-03T18:12:06.589 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:05.426+0000 7f6df3000700 1 -- 172.21.15.33:0/3654134693 >> [v2:172.21.15.33:6800/2830135294,v1:172.21.15.33:6801/2830135294] conn(0x7f6dd403e510 msgr2=0x7f6dd40409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:12:06.589 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:05.426+0000 7f6df3000700 1 --2- 172.21.15.33:0/3654134693 >> [v2:172.21.15.33:6800/2830135294,v1:172.21.15.33:6801/2830135294] conn(0x7f6dd403e510 0x7f6dd40409d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f6dd8006fd0 tx=0x7f6dd8006e40 comp rx=0 tx=0).stop 2024-05-03T18:12:06.589 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:05.426+0000 7f6df3000700 1 -- 172.21.15.33:0/3654134693 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6dec109480 msgr2=0x7f6dec19eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:12:06.590 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:05.426+0000 7f6df3000700 1 --2- 172.21.15.33:0/3654134693 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6dec109480 0x7f6dec19eb40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6de0011670 tx=0x7f6de0011a30 comp rx=0 tx=0).stop 2024-05-03T18:12:06.590 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:05.426+0000 7f6df3000700 1 -- 172.21.15.33:0/3654134693 shutdown_connections 2024-05-03T18:12:06.590 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:05.426+0000 7f6df3000700 1 --2- 172.21.15.33:0/3654134693 >> [v2:172.21.15.33:6800/2830135294,v1:172.21.15.33:6801/2830135294] conn(0x7f6dd403e510 0x7f6dd40409d0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:12:06.590 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:05.426+0000 7f6df3000700 1 --2- 172.21.15.33:0/3654134693 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6dec109480 0x7f6dec19eb40 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:12:06.590 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:05.426+0000 7f6df3000700 1 -- 172.21.15.33:0/3654134693 >> 172.21.15.33:0/3654134693 conn(0x7f6dec102ed0 msgr2=0x7f6dec103960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:12:06.590 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:05.426+0000 7f6df3000700 1 -- 172.21.15.33:0/3654134693 shutdown_connections 2024-05-03T18:12:06.590 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:05.426+0000 7f6df3000700 1 -- 172.21.15.33:0/3654134693 wait complete. 2024-05-03T18:12:07.435 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:12:07 smithi033 ceph-mon[28330]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:12:07.435 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:12:07 smithi033 ceph-mon[28330]: from='mgr.14120 172.21.15.33:0/3290632043' entity='mgr.x' 2024-05-03T18:12:08.118 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:07.180+0000 7ff183400700 1 Processor -- start 2024-05-03T18:12:08.200 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:07.180+0000 7ff183400700 1 -- start start 2024-05-03T18:12:08.200 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:07.180+0000 7ff183400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff17c1074e0 0x7ff17c107900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:12:08.200 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:07.180+0000 7ff183400700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff17c07cdd0 con 0x7ff17c1074e0 2024-05-03T18:12:08.201 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:07.181+0000 7ff180a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff17c1074e0 0x7ff17c107900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:12:08.201 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:07.181+0000 7ff180a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff17c1074e0 0x7ff17c107900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59562/0 (socket says 172.21.15.33:59562) 2024-05-03T18:12:08.201 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:07.181+0000 7ff180a00700 1 -- 172.21.15.33:0/2997839756 learned_addr learned my addr 172.21.15.33:0/2997839756 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:12:08.201 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:07.181+0000 7ff180a00700 1 -- 172.21.15.33:0/2997839756 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff17c107e40 con 0x7ff17c1074e0 2024-05-03T18:12:08.201 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:07.182+0000 7ff180a00700 1 --2- 172.21.15.33:0/2997839756 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff17c1074e0 0x7ff17c107900 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff1700098a0 tx=0x7ff170009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b239bc50901064c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:12:08.201 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:07.182+0000 7ff17b400700 1 -- 172.21.15.33:0/2997839756 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff170004160 con 0x7ff17c1074e0 2024-05-03T18:12:08.201 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:07.182+0000 7ff17b400700 1 -- 172.21.15.33:0/2997839756 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff170004300 con 0x7ff17c1074e0 2024-05-03T18:12:08.201 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:07.182+0000 7ff17b400700 1 -- 172.21.15.33:0/2997839756 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff170011960 con 0x7ff17c1074e0 2024-05-03T18:12:08.201 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:07.183+0000 7ff183400700 1 -- 172.21.15.33:0/2997839756 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff17c1074e0 msgr2=0x7ff17c107900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:12:08.201 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:07.183+0000 7ff183400700 1 --2- 172.21.15.33:0/2997839756 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff17c1074e0 0x7ff17c107900 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff1700098a0 tx=0x7ff170009bb0 comp rx=0 tx=0).stop 2024-05-03T18:12:08.201 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:07.183+0000 7ff183400700 1 -- 172.21.15.33:0/2997839756 shutdown_connections 2024-05-03T18:12:08.201 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:07.183+0000 7ff183400700 1 --2- 172.21.15.33:0/2997839756 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff17c1074e0 0x7ff17c107900 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:12:08.201 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:07.183+0000 7ff183400700 1 -- 172.21.15.33:0/2997839756 >> 172.21.15.33:0/2997839756 conn(0x7ff17c1030a0 msgr2=0x7ff17c1054e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:12:08.201 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:07.183+0000 7ff183400700 1 -- 172.21.15.33:0/2997839756 shutdown_connections 2024-05-03T18:12:08.202 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:07.183+0000 7ff183400700 1 -- 172.21.15.33:0/2997839756 wait complete. 2024-05-03T18:12:08.202 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:07.184+0000 7ff183400700 1 Processor -- start 2024-05-03T18:12:08.202 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:07.185+0000 7ff183400700 1 -- start start 2024-05-03T18:12:08.202 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:07.185+0000 7ff183400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff17c1074e0 0x7ff17c19ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:12:08.202 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:07.185+0000 7ff183400700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff17c108e50 con 0x7ff17c1074e0 2024-05-03T18:12:08.202 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:07.185+0000 7ff180a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff17c1074e0 0x7ff17c19ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:12:08.202 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:07.185+0000 7ff180a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff17c1074e0 0x7ff17c19ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59570/0 (socket says 172.21.15.33:59570) 2024-05-03T18:12:08.202 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:07.185+0000 7ff180a00700 1 -- 172.21.15.33:0/451506626 learned_addr learned my addr 172.21.15.33:0/451506626 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:12:08.202 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:07.186+0000 7ff180a00700 1 -- 172.21.15.33:0/451506626 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff17c19f1b0 con 0x7ff17c1074e0 2024-05-03T18:12:08.202 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:07.186+0000 7ff180a00700 1 --2- 172.21.15.33:0/451506626 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff17c1074e0 0x7ff17c19ec70 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff170011670 tx=0x7ff170011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:12:08.202 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:07.186+0000 7ff179600700 1 -- 172.21.15.33:0/451506626 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff170003a60 con 0x7ff17c1074e0 2024-05-03T18:12:08.202 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:07.187+0000 7ff179600700 1 -- 172.21.15.33:0/451506626 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff17001b8d0 con 0x7ff17c1074e0 2024-05-03T18:12:08.202 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:07.187+0000 7ff183400700 1 -- 172.21.15.33:0/451506626 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff17c19f440 con 0x7ff17c1074e0 2024-05-03T18:12:08.203 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:07.187+0000 7ff179600700 1 -- 172.21.15.33:0/451506626 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff170023790 con 0x7ff17c1074e0 2024-05-03T18:12:08.203 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:07.187+0000 7ff183400700 1 -- 172.21.15.33:0/451506626 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff17c1a2130 con 0x7ff17c1074e0 2024-05-03T18:12:08.203 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:07.188+0000 7ff183400700 1 -- 172.21.15.33:0/451506626 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff17c064b00 con 0x7ff17c1074e0 2024-05-03T18:12:08.203 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:07.188+0000 7ff179600700 1 -- 172.21.15.33:0/451506626 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7ff17001b440 con 0x7ff17c1074e0 2024-05-03T18:12:08.203 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:07.189+0000 7ff179600700 1 --2- 172.21.15.33:0/451506626 >> [v2:172.21.15.33:6800/2830135294,v1:172.21.15.33:6801/2830135294] conn(0x7ff16403e470 0x7ff164040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:12:08.203 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:07.189+0000 7ff179600700 1 -- 172.21.15.33:0/451506626 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff170053f20 con 0x7ff17c1074e0 2024-05-03T18:12:08.203 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:07.189+0000 7ff17be00700 1 --2- 172.21.15.33:0/451506626 >> [v2:172.21.15.33:6800/2830135294,v1:172.21.15.33:6801/2830135294] conn(0x7ff16403e470 0x7ff164040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:12:08.203 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:07.190+0000 7ff17be00700 1 --2- 172.21.15.33:0/451506626 >> [v2:172.21.15.33:6800/2830135294,v1:172.21.15.33:6801/2830135294] conn(0x7ff16403e470 0x7ff164040930 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff16c006fd0 tx=0x7ff16c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:12:08.203 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:07.195+0000 7ff179600700 1 -- 172.21.15.33:0/451506626 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7ff17001f030 con 0x7ff17c1074e0 2024-05-03T18:12:08.203 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:07.435+0000 7ff183400700 1 -- 172.21.15.33:0/451506626 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7ff17c02e810 con 0x7ff17c1074e0 2024-05-03T18:12:08.203 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:07.440+0000 7ff179600700 1 -- 172.21.15.33:0/451506626 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7ff17001a3d0 con 0x7ff17c1074e0 2024-05-03T18:12:08.203 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:07.444+0000 7ff183400700 1 -- 172.21.15.33:0/451506626 >> [v2:172.21.15.33:6800/2830135294,v1:172.21.15.33:6801/2830135294] conn(0x7ff16403e470 msgr2=0x7ff164040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:12:08.203 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:07.444+0000 7ff183400700 1 --2- 172.21.15.33:0/451506626 >> [v2:172.21.15.33:6800/2830135294,v1:172.21.15.33:6801/2830135294] conn(0x7ff16403e470 0x7ff164040930 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff16c006fd0 tx=0x7ff16c006e40 comp rx=0 tx=0).stop 2024-05-03T18:12:08.203 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:07.444+0000 7ff183400700 1 -- 172.21.15.33:0/451506626 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff17c1074e0 msgr2=0x7ff17c19ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:12:08.204 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:07.444+0000 7ff183400700 1 --2- 172.21.15.33:0/451506626 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff17c1074e0 0x7ff17c19ec70 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff170011670 tx=0x7ff170011a30 comp rx=0 tx=0).stop 2024-05-03T18:12:08.204 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:07.445+0000 7ff183400700 1 -- 172.21.15.33:0/451506626 shutdown_connections 2024-05-03T18:12:08.204 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:07.445+0000 7ff183400700 1 --2- 172.21.15.33:0/451506626 >> [v2:172.21.15.33:6800/2830135294,v1:172.21.15.33:6801/2830135294] conn(0x7ff16403e470 0x7ff164040930 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:12:08.204 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:07.445+0000 7ff183400700 1 --2- 172.21.15.33:0/451506626 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff17c1074e0 0x7ff17c19ec70 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:12:08.204 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:07.445+0000 7ff183400700 1 -- 172.21.15.33:0/451506626 >> 172.21.15.33:0/451506626 conn(0x7ff17c1030a0 msgr2=0x7ff17c195900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:12:08.204 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:07.445+0000 7ff183400700 1 -- 172.21.15.33:0/451506626 shutdown_connections 2024-05-03T18:12:08.204 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:07.445+0000 7ff183400700 1 -- 172.21.15.33:0/451506626 wait complete. 2024-05-03T18:12:08.204 INFO:teuthology.orchestra.run.smithi033.stdout:Enabling the dashboard module... 2024-05-03T18:12:08.699 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:12:08 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/451506626' entity='client.admin' 2024-05-03T18:12:09.732 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:12:09 smithi033 ceph-mon[28330]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:12:09.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:12:09 smithi033 ceph-mon[28330]: from='mgr.14120 172.21.15.33:0/3290632043' entity='mgr.x' 2024-05-03T18:12:09.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:12:09 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/3750122490' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-05-03T18:12:09.733 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:12:09 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: ignoring --setuser ceph since I am not root 2024-05-03T18:12:09.733 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:12:09 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: ignoring --setgroup ceph since I am not root 2024-05-03T18:12:10.074 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:12:09 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:12:09.807+0000 7f2c70539200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-03T18:12:10.074 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:12:10 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:12:10.072+0000 7f2c70539200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-03T18:12:10.312 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:08.699+0000 7fd9fe800700 1 Processor -- start 2024-05-03T18:12:10.312 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:08.699+0000 7fd9fe800700 1 -- start start 2024-05-03T18:12:10.312 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:08.699+0000 7fd9fe800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9f807ee30 0x7fd9f807d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:12:10.312 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:08.699+0000 7fd9fe800700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f807d8b0 con 0x7fd9f807ee30 2024-05-03T18:12:10.312 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:08.700+0000 7fd9f7400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9f807ee30 0x7fd9f807d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:12:10.313 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:08.700+0000 7fd9f7400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9f807ee30 0x7fd9f807d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59574/0 (socket says 172.21.15.33:59574) 2024-05-03T18:12:10.313 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:08.700+0000 7fd9f7400700 1 -- 172.21.15.33:0/2908441003 learned_addr learned my addr 172.21.15.33:0/2908441003 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:12:10.313 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:08.700+0000 7fd9f7400700 1 -- 172.21.15.33:0/2908441003 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9f807da30 con 0x7fd9f807ee30 2024-05-03T18:12:10.313 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:08.701+0000 7fd9f7400700 1 --2- 172.21.15.33:0/2908441003 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9f807ee30 0x7fd9f807d2e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd9e80098a0 tx=0x7fd9e8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dbfab5aee19e93 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:12:10.313 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:08.701+0000 7fd9f6000700 1 -- 172.21.15.33:0/2908441003 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd9e8004160 con 0x7fd9f807ee30 2024-05-03T18:12:10.313 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:08.701+0000 7fd9f6000700 1 -- 172.21.15.33:0/2908441003 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd9e8004300 con 0x7fd9f807ee30 2024-05-03T18:12:10.313 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:08.701+0000 7fd9f6000700 1 -- 172.21.15.33:0/2908441003 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd9e8011960 con 0x7fd9f807ee30 2024-05-03T18:12:10.319 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:08.702+0000 7fd9fe800700 1 -- 172.21.15.33:0/2908441003 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9f807ee30 msgr2=0x7fd9f807d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:12:10.319 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:08.702+0000 7fd9fe800700 1 --2- 172.21.15.33:0/2908441003 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9f807ee30 0x7fd9f807d2e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd9e80098a0 tx=0x7fd9e8009bb0 comp rx=0 tx=0).stop 2024-05-03T18:12:10.319 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:08.702+0000 7fd9fe800700 1 -- 172.21.15.33:0/2908441003 shutdown_connections 2024-05-03T18:12:10.319 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:08.702+0000 7fd9fe800700 1 --2- 172.21.15.33:0/2908441003 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9f807ee30 0x7fd9f807d2e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:12:10.319 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:08.702+0000 7fd9fe800700 1 -- 172.21.15.33:0/2908441003 >> 172.21.15.33:0/2908441003 conn(0x7fd9f8104220 msgr2=0x7fd9f8106640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:12:10.319 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:08.702+0000 7fd9fe800700 1 -- 172.21.15.33:0/2908441003 shutdown_connections 2024-05-03T18:12:10.319 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:08.703+0000 7fd9fe800700 1 -- 172.21.15.33:0/2908441003 wait complete. 2024-05-03T18:12:10.319 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:08.704+0000 7fd9fe800700 1 Processor -- start 2024-05-03T18:12:10.319 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:08.704+0000 7fd9fe800700 1 -- start start 2024-05-03T18:12:10.319 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:08.704+0000 7fd9fe800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9f807ee30 0x7fd9f819efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:12:10.320 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:08.704+0000 7fd9fe800700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f810f1a0 con 0x7fd9f807ee30 2024-05-03T18:12:10.320 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:08.704+0000 7fd9f7400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9f807ee30 0x7fd9f819efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:12:10.320 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:08.704+0000 7fd9f7400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9f807ee30 0x7fd9f819efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59590/0 (socket says 172.21.15.33:59590) 2024-05-03T18:12:10.320 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:08.704+0000 7fd9f7400700 1 -- 172.21.15.33:0/3750122490 learned_addr learned my addr 172.21.15.33:0/3750122490 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:12:10.320 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:08.705+0000 7fd9f7400700 1 -- 172.21.15.33:0/3750122490 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9f819f4e0 con 0x7fd9f807ee30 2024-05-03T18:12:10.320 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:08.705+0000 7fd9f7400700 1 --2- 172.21.15.33:0/3750122490 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9f807ee30 0x7fd9f819efa0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd9e8011670 tx=0x7fd9e8011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:12:10.320 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:08.705+0000 7fd9efe00700 1 -- 172.21.15.33:0/3750122490 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd9e801f070 con 0x7fd9f807ee30 2024-05-03T18:12:10.320 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:08.705+0000 7fd9efe00700 1 -- 172.21.15.33:0/3750122490 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd9e8003b20 con 0x7fd9f807ee30 2024-05-03T18:12:10.320 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:08.705+0000 7fd9fe800700 1 -- 172.21.15.33:0/3750122490 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9f819f770 con 0x7fd9f807ee30 2024-05-03T18:12:10.320 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:08.706+0000 7fd9efe00700 1 -- 172.21.15.33:0/3750122490 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd9e80195f0 con 0x7fd9f807ee30 2024-05-03T18:12:10.320 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:08.706+0000 7fd9fe800700 1 -- 172.21.15.33:0/3750122490 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9f81a2460 con 0x7fd9f807ee30 2024-05-03T18:12:10.320 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:08.707+0000 7fd9fe800700 1 -- 172.21.15.33:0/3750122490 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9f8064b00 con 0x7fd9f807ee30 2024-05-03T18:12:10.320 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:08.707+0000 7fd9efe00700 1 -- 172.21.15.33:0/3750122490 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fd9e8022070 con 0x7fd9f807ee30 2024-05-03T18:12:10.321 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:08.708+0000 7fd9efe00700 1 --2- 172.21.15.33:0/3750122490 >> [v2:172.21.15.33:6800/2830135294,v1:172.21.15.33:6801/2830135294] conn(0x7fd9dc03e4c0 0x7fd9dc040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:12:10.321 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:08.708+0000 7fd9efe00700 1 -- 172.21.15.33:0/3750122490 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd9e80539b0 con 0x7fd9f807ee30 2024-05-03T18:12:10.321 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:08.708+0000 7fd9f6a00700 1 --2- 172.21.15.33:0/3750122490 >> [v2:172.21.15.33:6800/2830135294,v1:172.21.15.33:6801/2830135294] conn(0x7fd9dc03e4c0 0x7fd9dc040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:12:10.321 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:08.709+0000 7fd9f6a00700 1 --2- 172.21.15.33:0/3750122490 >> [v2:172.21.15.33:6800/2830135294,v1:172.21.15.33:6801/2830135294] conn(0x7fd9dc03e4c0 0x7fd9dc040980 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd9e4006fd0 tx=0x7fd9e4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:12:10.321 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:08.713+0000 7fd9efe00700 1 -- 172.21.15.33:0/3750122490 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fd9e802cd20 con 0x7fd9f807ee30 2024-05-03T18:12:10.321 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:09.020+0000 7fd9fe800700 1 -- 172.21.15.33:0/3750122490 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7fd9f802e510 con 0x7fd9f807ee30 2024-05-03T18:12:10.321 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:09.473+0000 7fd9efe00700 1 -- 172.21.15.33:0/3750122490 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7fd9e801b440 con 0x7fd9f807ee30 2024-05-03T18:12:10.321 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:09.475+0000 7fd9efe00700 1 -- 172.21.15.33:0/3750122490 <== mon.0 v2:172.21.15.33:3300/0 8 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7fd9e8053d10 con 0x7fd9f807ee30 2024-05-03T18:12:10.321 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:09.478+0000 7fd9fe800700 1 -- 172.21.15.33:0/3750122490 >> [v2:172.21.15.33:6800/2830135294,v1:172.21.15.33:6801/2830135294] conn(0x7fd9dc03e4c0 msgr2=0x7fd9dc040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:12:10.321 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:09.478+0000 7fd9fe800700 1 --2- 172.21.15.33:0/3750122490 >> [v2:172.21.15.33:6800/2830135294,v1:172.21.15.33:6801/2830135294] conn(0x7fd9dc03e4c0 0x7fd9dc040980 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd9e4006fd0 tx=0x7fd9e4006e40 comp rx=0 tx=0).stop 2024-05-03T18:12:10.321 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:09.478+0000 7fd9fe800700 1 -- 172.21.15.33:0/3750122490 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9f807ee30 msgr2=0x7fd9f819efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:12:10.321 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:09.478+0000 7fd9fe800700 1 --2- 172.21.15.33:0/3750122490 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9f807ee30 0x7fd9f819efa0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd9e8011670 tx=0x7fd9e8011a30 comp rx=0 tx=0).stop 2024-05-03T18:12:10.321 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:09.479+0000 7fd9fe800700 1 -- 172.21.15.33:0/3750122490 shutdown_connections 2024-05-03T18:12:10.322 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:09.479+0000 7fd9fe800700 1 --2- 172.21.15.33:0/3750122490 >> [v2:172.21.15.33:6800/2830135294,v1:172.21.15.33:6801/2830135294] conn(0x7fd9dc03e4c0 0x7fd9dc040980 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:12:10.322 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:09.479+0000 7fd9fe800700 1 --2- 172.21.15.33:0/3750122490 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9f807ee30 0x7fd9f819efa0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:12:10.322 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:09.479+0000 7fd9fe800700 1 -- 172.21.15.33:0/3750122490 >> 172.21.15.33:0/3750122490 conn(0x7fd9f8104220 msgr2=0x7fd9f8104ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:12:10.322 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:09.481+0000 7fd9fe800700 1 -- 172.21.15.33:0/3750122490 shutdown_connections 2024-05-03T18:12:10.322 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:09.482+0000 7fd9fe800700 1 -- 172.21.15.33:0/3750122490 wait complete. 2024-05-03T18:12:10.459 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:12:10 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:12:10.208+0000 7f2c70539200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-03T18:12:10.732 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:12:10 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/3750122490' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-05-03T18:12:10.732 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:12:10 smithi033 ceph-mon[28330]: mgrmap e9: x(active, since 25s) 2024-05-03T18:12:10.733 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:12:10 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:12:10.631+0000 7f2c70539200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-03T18:12:11.046 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:12:10 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:12:10.745+0000 7f2c70539200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-03T18:12:12.155 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:12:11 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/3918904440' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-03T18:12:12.467 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:12:12 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:12:12.368+0000 7f2c70539200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-03T18:12:12.482 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout { 2024-05-03T18:12:12.482 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-05-03T18:12:12.482 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-03T18:12:12.482 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-03T18:12:12.482 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-03T18:12:12.482 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout } 2024-05-03T18:12:12.482 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:11.486+0000 7fa285400700 1 Processor -- start 2024-05-03T18:12:12.483 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:11.486+0000 7fa285400700 1 -- start start 2024-05-03T18:12:12.483 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:11.486+0000 7fa285400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa280108a10 0x7fa280108e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:12:12.483 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:11.486+0000 7fa285400700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa280109400 con 0x7fa280108a10 2024-05-03T18:12:12.483 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:11.487+0000 7fa27fe00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa280108a10 0x7fa280108e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:12:12.483 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:11.487+0000 7fa27fe00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa280108a10 0x7fa280108e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59622/0 (socket says 172.21.15.33:59622) 2024-05-03T18:12:12.483 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:11.487+0000 7fa27fe00700 1 -- 172.21.15.33:0/876015717 learned_addr learned my addr 172.21.15.33:0/876015717 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:12:12.483 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:11.487+0000 7fa27fe00700 1 -- 172.21.15.33:0/876015717 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa280109bc0 con 0x7fa280108a10 2024-05-03T18:12:12.483 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:11.488+0000 7fa27fe00700 1 --2- 172.21.15.33:0/876015717 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa280108a10 0x7fa280108e30 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa2700098a0 tx=0x7fa270009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94e4c6bb9bd5c8f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:12:12.483 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:11.488+0000 7fa27ea00700 1 -- 172.21.15.33:0/876015717 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa270004160 con 0x7fa280108a10 2024-05-03T18:12:12.483 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:11.488+0000 7fa27ea00700 1 -- 172.21.15.33:0/876015717 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa270004300 con 0x7fa280108a10 2024-05-03T18:12:12.483 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:11.488+0000 7fa27ea00700 1 -- 172.21.15.33:0/876015717 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa270011960 con 0x7fa280108a10 2024-05-03T18:12:12.483 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:11.488+0000 7fa285400700 1 -- 172.21.15.33:0/876015717 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa280108a10 msgr2=0x7fa280108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:12:12.483 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:11.488+0000 7fa285400700 1 --2- 172.21.15.33:0/876015717 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa280108a10 0x7fa280108e30 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa2700098a0 tx=0x7fa270009bb0 comp rx=0 tx=0).stop 2024-05-03T18:12:12.483 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:11.489+0000 7fa285400700 1 -- 172.21.15.33:0/876015717 shutdown_connections 2024-05-03T18:12:12.484 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:11.489+0000 7fa285400700 1 --2- 172.21.15.33:0/876015717 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa280108a10 0x7fa280108e30 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:12:12.484 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:11.489+0000 7fa285400700 1 -- 172.21.15.33:0/876015717 >> 172.21.15.33:0/876015717 conn(0x7fa280103fb0 msgr2=0x7fa2801063f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:12:12.484 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:11.489+0000 7fa285400700 1 -- 172.21.15.33:0/876015717 shutdown_connections 2024-05-03T18:12:12.484 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:11.489+0000 7fa285400700 1 -- 172.21.15.33:0/876015717 wait complete. 2024-05-03T18:12:12.484 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:11.490+0000 7fa285400700 1 Processor -- start 2024-05-03T18:12:12.484 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:11.490+0000 7fa285400700 1 -- start start 2024-05-03T18:12:12.484 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:11.490+0000 7fa285400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa280108a10 0x7fa28019eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:12:12.484 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:11.490+0000 7fa285400700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa28010cdf0 con 0x7fa280108a10 2024-05-03T18:12:12.484 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:11.490+0000 7fa27fe00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa280108a10 0x7fa28019eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:12:12.484 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:11.491+0000 7fa27fe00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa280108a10 0x7fa28019eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59626/0 (socket says 172.21.15.33:59626) 2024-05-03T18:12:12.484 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:11.491+0000 7fa27fe00700 1 -- 172.21.15.33:0/3918904440 learned_addr learned my addr 172.21.15.33:0/3918904440 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:12:12.484 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:11.491+0000 7fa27fe00700 1 -- 172.21.15.33:0/3918904440 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa28019f030 con 0x7fa280108a10 2024-05-03T18:12:12.484 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:11.491+0000 7fa27fe00700 1 --2- 172.21.15.33:0/3918904440 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa280108a10 0x7fa28019eaf0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa270011670 tx=0x7fa270011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:12:12.484 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:11.491+0000 7fa27cc00700 1 -- 172.21.15.33:0/3918904440 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa270003a60 con 0x7fa280108a10 2024-05-03T18:12:12.484 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:11.491+0000 7fa27cc00700 1 -- 172.21.15.33:0/3918904440 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa27001b8d0 con 0x7fa280108a10 2024-05-03T18:12:12.485 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:11.491+0000 7fa285400700 1 -- 172.21.15.33:0/3918904440 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa28019f2c0 con 0x7fa280108a10 2024-05-03T18:12:12.485 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:11.491+0000 7fa27cc00700 1 -- 172.21.15.33:0/3918904440 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa270023790 con 0x7fa280108a10 2024-05-03T18:12:12.485 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:11.492+0000 7fa285400700 1 -- 172.21.15.33:0/3918904440 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2801a1fb0 con 0x7fa280108a10 2024-05-03T18:12:12.485 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:11.493+0000 7fa285400700 1 -- 172.21.15.33:0/3918904440 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa28010cf70 con 0x7fa280108a10 2024-05-03T18:12:12.485 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:11.493+0000 7fa27cc00700 1 -- 172.21.15.33:0/3918904440 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7fa27001b440 con 0x7fa280108a10 2024-05-03T18:12:12.485 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:11.494+0000 7fa27cc00700 1 --2- 172.21.15.33:0/3918904440 >> [v2:172.21.15.33:6800/2830135294,v1:172.21.15.33:6801/2830135294] conn(0x7fa26403e4c0 0x7fa264040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:12:12.485 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:11.494+0000 7fa27cc00700 1 -- 172.21.15.33:0/3918904440 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa270054020 con 0x7fa280108a10 2024-05-03T18:12:12.485 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:11.494+0000 7fa27f400700 1 -- 172.21.15.33:0/3918904440 >> [v2:172.21.15.33:6800/2830135294,v1:172.21.15.33:6801/2830135294] conn(0x7fa26403e4c0 msgr2=0x7fa264040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/2830135294 2024-05-03T18:12:12.485 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:11.494+0000 7fa27f400700 1 --2- 172.21.15.33:0/3918904440 >> [v2:172.21.15.33:6800/2830135294,v1:172.21.15.33:6801/2830135294] conn(0x7fa26403e4c0 0x7fa264040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-03T18:12:12.485 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:11.499+0000 7fa27cc00700 1 -- 172.21.15.33:0/3918904440 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fa27001f030 con 0x7fa280108a10 2024-05-03T18:12:12.485 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:11.695+0000 7fa27f400700 1 -- 172.21.15.33:0/3918904440 >> [v2:172.21.15.33:6800/2830135294,v1:172.21.15.33:6801/2830135294] conn(0x7fa26403e4c0 msgr2=0x7fa264040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/2830135294 2024-05-03T18:12:12.485 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:11.695+0000 7fa27f400700 1 --2- 172.21.15.33:0/3918904440 >> [v2:172.21.15.33:6800/2830135294,v1:172.21.15.33:6801/2830135294] conn(0x7fa26403e4c0 0x7fa264040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-03T18:12:12.485 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:11.802+0000 7fa285400700 1 -- 172.21.15.33:0/3918904440 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fa2801a2180 con 0x7fa280108a10 2024-05-03T18:12:12.485 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:11.803+0000 7fa27cc00700 1 -- 172.21.15.33:0/3918904440 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7fa270023a40 con 0x7fa280108a10 2024-05-03T18:12:12.486 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:11.806+0000 7fa285400700 1 -- 172.21.15.33:0/3918904440 >> [v2:172.21.15.33:6800/2830135294,v1:172.21.15.33:6801/2830135294] conn(0x7fa26403e4c0 msgr2=0x7fa264040980 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-03T18:12:12.486 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:11.806+0000 7fa285400700 1 --2- 172.21.15.33:0/3918904440 >> [v2:172.21.15.33:6800/2830135294,v1:172.21.15.33:6801/2830135294] conn(0x7fa26403e4c0 0x7fa264040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:12:12.486 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:11.806+0000 7fa285400700 1 -- 172.21.15.33:0/3918904440 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa280108a10 msgr2=0x7fa28019eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:12:12.486 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:11.806+0000 7fa285400700 1 --2- 172.21.15.33:0/3918904440 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa280108a10 0x7fa28019eaf0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa270011670 tx=0x7fa270011a30 comp rx=0 tx=0).stop 2024-05-03T18:12:12.486 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:11.806+0000 7fa285400700 1 -- 172.21.15.33:0/3918904440 shutdown_connections 2024-05-03T18:12:12.486 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:11.806+0000 7fa285400700 1 --2- 172.21.15.33:0/3918904440 >> [v2:172.21.15.33:6800/2830135294,v1:172.21.15.33:6801/2830135294] conn(0x7fa26403e4c0 0x7fa264040980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:12:12.486 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:11.806+0000 7fa285400700 1 --2- 172.21.15.33:0/3918904440 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa280108a10 0x7fa28019eaf0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:12:12.486 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:11.806+0000 7fa285400700 1 -- 172.21.15.33:0/3918904440 >> 172.21.15.33:0/3918904440 conn(0x7fa280103fb0 msgr2=0x7fa280104a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:12:12.486 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:11.806+0000 7fa285400700 1 -- 172.21.15.33:0/3918904440 shutdown_connections 2024-05-03T18:12:12.486 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:11.807+0000 7fa285400700 1 -- 172.21.15.33:0/3918904440 wait complete. 2024-05-03T18:12:12.486 INFO:teuthology.orchestra.run.smithi033.stdout:Waiting for the mgr to restart... 2024-05-03T18:12:12.486 INFO:teuthology.orchestra.run.smithi033.stdout:Waiting for mgr epoch 9... 2024-05-03T18:12:12.732 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:12:12 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:12:12.611+0000 7f2c70539200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-03T18:12:13.101 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:12:12 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:12:12.840+0000 7f2c70539200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-03T18:12:13.371 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:12:13 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:12:13.242+0000 7f2c70539200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-03T18:12:13.630 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:12:13 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:12:13.371+0000 7f2c70539200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-03T18:12:13.630 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:12:13 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:12:13.630+0000 7f2c70539200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-03T18:12:14.232 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:12:13 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:12:13.893+0000 7f2c70539200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-03T18:12:14.982 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:12:14 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:12:14.708+0000 7f2c70539200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-03T18:12:15.482 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:12:15 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:12:15.006+0000 7f2c70539200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-03T18:12:15.982 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:12:15 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:12:15.593+0000 7f2c70539200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-03T18:12:15.983 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:12:15 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:12:15.726+0000 7f2c70539200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-03T18:12:16.482 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:12:16 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:12:16.204+0000 7f2c70539200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-03T18:12:16.813 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:12:16 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:12:16.682+0000 7f2c70539200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-03T18:12:17.159 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:12:16 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:12:16.814+0000 7f2c70539200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-03T18:12:17.482 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:12:17 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:12:17.162+0000 7f2c70539200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-03T18:12:17.482 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:12:17 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:12:17.353+0000 7f2c70539200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-03T18:12:18.026 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:12:17 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:12:17.738+0000 7f2c70539200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-03T18:12:18.026 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:12:17 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:12:17.898+0000 7f2c70539200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-03T18:12:18.482 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:12:18 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:12:18.027+0000 7f2c70539200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-03T18:12:19.123 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:12:18 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:12:18.820+0000 7f2c70539200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-03T18:12:19.482 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:12:19 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:12:19.123+0000 7f2c70539200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-03T18:12:19.482 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:12:19 smithi033 ceph-mon[28330]: Active manager daemon x restarted 2024-05-03T18:12:19.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:12:19 smithi033 ceph-mon[28330]: Activating manager daemon x 2024-05-03T18:12:19.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:12:19 smithi033 ceph-mon[28330]: osdmap e3: 0 total, 0 up, 0 in 2024-05-03T18:12:19.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:12:19 smithi033 ceph-mon[28330]: mgrmap e10: x(active, starting, since 0.0112738s) 2024-05-03T18:12:19.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:12:19 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-03T18:12:19.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:12:19 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-03T18:12:19.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:12:19 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-03T18:12:19.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:12:19 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-03T18:12:19.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:12:19 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-03T18:12:19.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:12:19 smithi033 ceph-mon[28330]: Manager daemon x is now available 2024-05-03T18:12:20.482 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:12:20 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:12:20.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:12:20 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-03T18:12:20.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:12:20 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-03T18:12:20.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:12:20 smithi033 ceph-mon[28330]: [03/May/2024:18:12:19] ENGINE Bus STARTING 2024-05-03T18:12:20.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:12:20 smithi033 ceph-mon[28330]: mgrmap e11: x(active, since 1.01752s) 2024-05-03T18:12:20.750 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout { 2024-05-03T18:12:20.751 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-05-03T18:12:20.751 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-03T18:12:20.751 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout } 2024-05-03T18:12:20.752 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:13.100+0000 7efe5a200700 1 Processor -- start 2024-05-03T18:12:20.752 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:13.100+0000 7efe5a200700 1 -- start start 2024-05-03T18:12:20.752 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:13.101+0000 7efe5a200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe5407edd0 0x7efe5407d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:12:20.752 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:13.101+0000 7efe5a200700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7efe5407d7c0 con 0x7efe5407edd0 2024-05-03T18:12:20.752 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:13.101+0000 7efe53400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe5407edd0 0x7efe5407d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:12:20.752 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:13.102+0000 7efe53400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe5407edd0 0x7efe5407d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59634/0 (socket says 172.21.15.33:59634) 2024-05-03T18:12:20.752 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:13.102+0000 7efe53400700 1 -- 172.21.15.33:0/349365454 learned_addr learned my addr 172.21.15.33:0/349365454 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:12:20.752 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:13.102+0000 7efe53400700 1 -- 172.21.15.33:0/349365454 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe5407d940 con 0x7efe5407edd0 2024-05-03T18:12:20.752 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:13.102+0000 7efe53400700 1 --2- 172.21.15.33:0/349365454 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe5407edd0 0x7efe5407d280 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7efe440098a0 tx=0x7efe44009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eceda8b01a08980a server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:12:20.755 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:13.103+0000 7efe52000700 1 -- 172.21.15.33:0/349365454 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efe44004160 con 0x7efe5407edd0 2024-05-03T18:12:20.755 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:13.103+0000 7efe52000700 1 -- 172.21.15.33:0/349365454 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efe44004300 con 0x7efe5407edd0 2024-05-03T18:12:20.755 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:13.103+0000 7efe52000700 1 -- 172.21.15.33:0/349365454 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efe44011960 con 0x7efe5407edd0 2024-05-03T18:12:20.755 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:13.103+0000 7efe5a200700 1 -- 172.21.15.33:0/349365454 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe5407edd0 msgr2=0x7efe5407d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:12:20.755 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:13.103+0000 7efe5a200700 1 --2- 172.21.15.33:0/349365454 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe5407edd0 0x7efe5407d280 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7efe440098a0 tx=0x7efe44009bb0 comp rx=0 tx=0).stop 2024-05-03T18:12:20.755 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:13.104+0000 7efe5a200700 1 -- 172.21.15.33:0/349365454 shutdown_connections 2024-05-03T18:12:20.755 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:13.104+0000 7efe5a200700 1 --2- 172.21.15.33:0/349365454 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe5407edd0 0x7efe5407d280 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:12:20.755 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:13.104+0000 7efe5a200700 1 -- 172.21.15.33:0/349365454 >> 172.21.15.33:0/349365454 conn(0x7efe541041b0 msgr2=0x7efe541065d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:12:20.755 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:13.104+0000 7efe5a200700 1 -- 172.21.15.33:0/349365454 shutdown_connections 2024-05-03T18:12:20.755 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:13.104+0000 7efe5a200700 1 -- 172.21.15.33:0/349365454 wait complete. 2024-05-03T18:12:20.755 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:13.105+0000 7efe5a200700 1 Processor -- start 2024-05-03T18:12:20.755 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:13.105+0000 7efe5a200700 1 -- start start 2024-05-03T18:12:20.755 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:13.105+0000 7efe5a200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe5407edd0 0x7efe5419efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:12:20.755 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:13.105+0000 7efe5a200700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7efe5410f0f0 con 0x7efe5407edd0 2024-05-03T18:12:20.756 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:13.106+0000 7efe53400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe5407edd0 0x7efe5419efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:12:20.756 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:13.106+0000 7efe53400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe5407edd0 0x7efe5419efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59644/0 (socket says 172.21.15.33:59644) 2024-05-03T18:12:20.756 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:13.106+0000 7efe53400700 1 -- 172.21.15.33:0/598349429 learned_addr learned my addr 172.21.15.33:0/598349429 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:12:20.756 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:13.106+0000 7efe53400700 1 -- 172.21.15.33:0/598349429 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe5419f500 con 0x7efe5407edd0 2024-05-03T18:12:20.756 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:13.106+0000 7efe53400700 1 --2- 172.21.15.33:0/598349429 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe5407edd0 0x7efe5419efc0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7efe44011670 tx=0x7efe44011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:12:20.759 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:13.107+0000 7efe4be00700 1 -- 172.21.15.33:0/598349429 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efe4401f070 con 0x7efe5407edd0 2024-05-03T18:12:20.759 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:13.107+0000 7efe4be00700 1 -- 172.21.15.33:0/598349429 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efe44003b20 con 0x7efe5407edd0 2024-05-03T18:12:20.759 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:13.107+0000 7efe5a200700 1 -- 172.21.15.33:0/598349429 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe5419f790 con 0x7efe5407edd0 2024-05-03T18:12:20.760 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:13.107+0000 7efe4be00700 1 -- 172.21.15.33:0/598349429 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efe44019660 con 0x7efe5407edd0 2024-05-03T18:12:20.760 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:13.107+0000 7efe5a200700 1 -- 172.21.15.33:0/598349429 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe541a2480 con 0x7efe5407edd0 2024-05-03T18:12:20.760 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:13.108+0000 7efe4be00700 1 -- 172.21.15.33:0/598349429 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7efe44022070 con 0x7efe5407edd0 2024-05-03T18:12:20.760 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:13.109+0000 7efe4be00700 1 --2- 172.21.15.33:0/598349429 >> [v2:172.21.15.33:6800/2830135294,v1:172.21.15.33:6801/2830135294] conn(0x7efe3803e4c0 0x7efe38040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:12:20.760 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:13.109+0000 7efe4be00700 1 -- 172.21.15.33:0/598349429 --> [v2:172.21.15.33:6800/2830135294,v1:172.21.15.33:6801/2830135294] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7efe38041090 con 0x7efe3803e4c0 2024-05-03T18:12:20.760 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:13.109+0000 7efe4be00700 1 -- 172.21.15.33:0/598349429 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7efe44053b00 con 0x7efe5407edd0 2024-05-03T18:12:20.760 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:13.109+0000 7efe52a00700 1 -- 172.21.15.33:0/598349429 >> [v2:172.21.15.33:6800/2830135294,v1:172.21.15.33:6801/2830135294] conn(0x7efe3803e4c0 msgr2=0x7efe38040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/2830135294 2024-05-03T18:12:20.760 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:13.109+0000 7efe52a00700 1 --2- 172.21.15.33:0/598349429 >> [v2:172.21.15.33:6800/2830135294,v1:172.21.15.33:6801/2830135294] conn(0x7efe3803e4c0 0x7efe38040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-03T18:12:20.760 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:13.310+0000 7efe52a00700 1 -- 172.21.15.33:0/598349429 >> [v2:172.21.15.33:6800/2830135294,v1:172.21.15.33:6801/2830135294] conn(0x7efe3803e4c0 msgr2=0x7efe38040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/2830135294 2024-05-03T18:12:20.760 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:13.310+0000 7efe52a00700 1 --2- 172.21.15.33:0/598349429 >> [v2:172.21.15.33:6800/2830135294,v1:172.21.15.33:6801/2830135294] conn(0x7efe3803e4c0 0x7efe38040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-03T18:12:20.760 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:13.711+0000 7efe52a00700 1 -- 172.21.15.33:0/598349429 >> [v2:172.21.15.33:6800/2830135294,v1:172.21.15.33:6801/2830135294] conn(0x7efe3803e4c0 msgr2=0x7efe38040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/2830135294 2024-05-03T18:12:20.760 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:13.711+0000 7efe52a00700 1 --2- 172.21.15.33:0/598349429 >> [v2:172.21.15.33:6800/2830135294,v1:172.21.15.33:6801/2830135294] conn(0x7efe3803e4c0 0x7efe38040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-03T18:12:20.760 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:14.512+0000 7efe52a00700 1 -- 172.21.15.33:0/598349429 >> [v2:172.21.15.33:6800/2830135294,v1:172.21.15.33:6801/2830135294] conn(0x7efe3803e4c0 msgr2=0x7efe38040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/2830135294 2024-05-03T18:12:20.761 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:14.512+0000 7efe52a00700 1 --2- 172.21.15.33:0/598349429 >> [v2:172.21.15.33:6800/2830135294,v1:172.21.15.33:6801/2830135294] conn(0x7efe3803e4c0 0x7efe38040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-03T18:12:20.761 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:16.114+0000 7efe52a00700 1 -- 172.21.15.33:0/598349429 >> [v2:172.21.15.33:6800/2830135294,v1:172.21.15.33:6801/2830135294] conn(0x7efe3803e4c0 msgr2=0x7efe38040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/2830135294 2024-05-03T18:12:20.761 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:16.114+0000 7efe52a00700 1 --2- 172.21.15.33:0/598349429 >> [v2:172.21.15.33:6800/2830135294,v1:172.21.15.33:6801/2830135294] conn(0x7efe3803e4c0 0x7efe38040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-03T18:12:20.761 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:19.135+0000 7efe4be00700 1 -- 172.21.15.33:0/598349429 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mgrmap(e 10) ==== 50002+0+0 (secure 0 0 0) 0x7efe4401bb60 con 0x7efe5407edd0 2024-05-03T18:12:20.761 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:19.135+0000 7efe4be00700 1 -- 172.21.15.33:0/598349429 >> [v2:172.21.15.33:6800/2830135294,v1:172.21.15.33:6801/2830135294] conn(0x7efe3803e4c0 msgr2=0x7efe38040980 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-03T18:12:20.761 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:19.135+0000 7efe4be00700 1 --2- 172.21.15.33:0/598349429 >> [v2:172.21.15.33:6800/2830135294,v1:172.21.15.33:6801/2830135294] conn(0x7efe3803e4c0 0x7efe38040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:12:20.761 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:20.138+0000 7efe4be00700 1 -- 172.21.15.33:0/598349429 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 11) ==== 50129+0+0 (secure 0 0 0) 0x7efe44017d70 con 0x7efe5407edd0 2024-05-03T18:12:20.761 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:20.138+0000 7efe4be00700 1 --2- 172.21.15.33:0/598349429 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7efe38041f40 0x7efe38044330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:12:20.761 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:20.139+0000 7efe4be00700 1 -- 172.21.15.33:0/598349429 --> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7efe38041090 con 0x7efe38041f40 2024-05-03T18:12:20.761 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:20.140+0000 7efe52a00700 1 --2- 172.21.15.33:0/598349429 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7efe38041f40 0x7efe38044330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:12:20.761 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:20.141+0000 7efe52a00700 1 --2- 172.21.15.33:0/598349429 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7efe38041f40 0x7efe38044330 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7efe40003a10 tx=0x7efe400092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:12:20.761 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:20.142+0000 7efe4be00700 1 -- 172.21.15.33:0/598349429 <== mgr.14152 v2:172.21.15.33:6800/3642895501 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7efe38041090 con 0x7efe38041f40 2024-05-03T18:12:20.761 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:20.156+0000 7efe5a200700 1 -- 172.21.15.33:0/598349429 --> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7efe54064b00 con 0x7efe38041f40 2024-05-03T18:12:20.761 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:20.157+0000 7efe4be00700 1 -- 172.21.15.33:0/598349429 <== mgr.14152 v2:172.21.15.33:6800/3642895501 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7efe54064b00 con 0x7efe38041f40 2024-05-03T18:12:20.762 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:20.157+0000 7efe5a200700 1 -- 172.21.15.33:0/598349429 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7efe38041f40 msgr2=0x7efe38044330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:12:20.762 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:20.157+0000 7efe5a200700 1 --2- 172.21.15.33:0/598349429 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7efe38041f40 0x7efe38044330 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7efe40003a10 tx=0x7efe400092b0 comp rx=0 tx=0).stop 2024-05-03T18:12:20.762 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:20.157+0000 7efe5a200700 1 -- 172.21.15.33:0/598349429 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe5407edd0 msgr2=0x7efe5419efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:12:20.762 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:20.157+0000 7efe5a200700 1 --2- 172.21.15.33:0/598349429 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe5407edd0 0x7efe5419efc0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7efe44011670 tx=0x7efe44011aa0 comp rx=0 tx=0).stop 2024-05-03T18:12:20.762 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:20.157+0000 7efe5a200700 1 -- 172.21.15.33:0/598349429 shutdown_connections 2024-05-03T18:12:20.762 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:20.157+0000 7efe5a200700 1 --2- 172.21.15.33:0/598349429 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7efe38041f40 0x7efe38044330 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:12:20.763 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:20.157+0000 7efe5a200700 1 --2- 172.21.15.33:0/598349429 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe5407edd0 0x7efe5419efc0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:12:20.763 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:20.158+0000 7efe5a200700 1 --2- 172.21.15.33:0/598349429 >> [v2:172.21.15.33:6800/2830135294,v1:172.21.15.33:6801/2830135294] conn(0x7efe3803e4c0 0x7efe38040980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:12:20.763 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:20.158+0000 7efe5a200700 1 -- 172.21.15.33:0/598349429 >> 172.21.15.33:0/598349429 conn(0x7efe541041b0 msgr2=0x7efe54104ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:12:20.763 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:20.158+0000 7efe5a200700 1 -- 172.21.15.33:0/598349429 shutdown_connections 2024-05-03T18:12:20.763 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:20.158+0000 7efe5a200700 1 -- 172.21.15.33:0/598349429 wait complete. 2024-05-03T18:12:20.763 INFO:teuthology.orchestra.run.smithi033.stdout:mgr epoch 9 is available 2024-05-03T18:12:20.763 INFO:teuthology.orchestra.run.smithi033.stdout:Generating a dashboard self-signed certificate... 2024-05-03T18:12:21.376 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:12:21 smithi033 ceph-mon[28330]: [03/May/2024:18:12:19] ENGINE Serving on http://172.21.15.33:8765 2024-05-03T18:12:21.376 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:12:21 smithi033 ceph-mon[28330]: [03/May/2024:18:12:19] ENGINE Serving on https://172.21.15.33:7150 2024-05-03T18:12:21.376 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:12:21 smithi033 ceph-mon[28330]: [03/May/2024:18:12:19] ENGINE Bus STARTED 2024-05-03T18:12:21.377 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:12:21 smithi033 ceph-mon[28330]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-03T18:12:21.377 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:12:21 smithi033 ceph-mon[28330]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-03T18:12:22.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:12:22 smithi033 ceph-mon[28330]: mgrmap e12: x(active, since 2s) 2024-05-03T18:12:22.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:12:22 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:12:23.230 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-05-03T18:12:23.230 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:22.025+0000 7f4d0a600700 1 Processor -- start 2024-05-03T18:12:23.230 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:22.025+0000 7f4d0a600700 1 -- start start 2024-05-03T18:12:23.230 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:22.026+0000 7f4d0a600700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d0410ae40 0x7f4d0410b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:12:23.230 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:22.026+0000 7f4d0a600700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4d0410b830 con 0x7f4d0410ae40 2024-05-03T18:12:23.230 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:22.026+0000 7f4d03400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d0410ae40 0x7f4d0410b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:12:23.230 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:22.026+0000 7f4d03400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d0410ae40 0x7f4d0410b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44356/0 (socket says 172.21.15.33:44356) 2024-05-03T18:12:23.231 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:22.026+0000 7f4d03400700 1 -- 172.21.15.33:0/1448750075 learned_addr learned my addr 172.21.15.33:0/1448750075 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:12:23.231 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:22.026+0000 7f4d03400700 1 -- 172.21.15.33:0/1448750075 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d0410bff0 con 0x7f4d0410ae40 2024-05-03T18:12:23.231 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:22.027+0000 7f4d03400700 1 --2- 172.21.15.33:0/1448750075 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d0410ae40 0x7f4d0410b260 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4cf40098a0 tx=0x7f4cf4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df5cce928e8c6519 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:12:23.231 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:22.027+0000 7f4d02a00700 1 -- 172.21.15.33:0/1448750075 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4cf4004160 con 0x7f4d0410ae40 2024-05-03T18:12:23.231 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:22.027+0000 7f4d02a00700 1 -- 172.21.15.33:0/1448750075 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4cf4004300 con 0x7f4d0410ae40 2024-05-03T18:12:23.231 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:22.027+0000 7f4d02a00700 1 -- 172.21.15.33:0/1448750075 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4cf4011960 con 0x7f4d0410ae40 2024-05-03T18:12:23.231 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:22.028+0000 7f4d0a600700 1 -- 172.21.15.33:0/1448750075 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d0410ae40 msgr2=0x7f4d0410b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:12:23.231 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:22.028+0000 7f4d0a600700 1 --2- 172.21.15.33:0/1448750075 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d0410ae40 0x7f4d0410b260 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4cf40098a0 tx=0x7f4cf4009bb0 comp rx=0 tx=0).stop 2024-05-03T18:12:23.231 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:22.028+0000 7f4d0a600700 1 -- 172.21.15.33:0/1448750075 shutdown_connections 2024-05-03T18:12:23.231 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:22.028+0000 7f4d0a600700 1 --2- 172.21.15.33:0/1448750075 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d0410ae40 0x7f4d0410b260 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:12:23.231 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:22.028+0000 7f4d0a600700 1 -- 172.21.15.33:0/1448750075 >> 172.21.15.33:0/1448750075 conn(0x7f4d0407e230 msgr2=0x7f4d0407e640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:12:23.231 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:22.029+0000 7f4d0a600700 1 -- 172.21.15.33:0/1448750075 shutdown_connections 2024-05-03T18:12:23.231 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:22.029+0000 7f4d0a600700 1 -- 172.21.15.33:0/1448750075 wait complete. 2024-05-03T18:12:23.231 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:22.030+0000 7f4d0a600700 1 Processor -- start 2024-05-03T18:12:23.232 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:22.030+0000 7f4d0a600700 1 -- start start 2024-05-03T18:12:23.232 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:22.031+0000 7f4d0a600700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d0410ae40 0x7f4d0419eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:12:23.232 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:22.031+0000 7f4d0a600700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4d0410f220 con 0x7f4d0410ae40 2024-05-03T18:12:23.232 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:22.031+0000 7f4d03400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d0410ae40 0x7f4d0419eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:12:23.232 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:22.031+0000 7f4d03400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d0410ae40 0x7f4d0419eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44370/0 (socket says 172.21.15.33:44370) 2024-05-03T18:12:23.232 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:22.031+0000 7f4d03400700 1 -- 172.21.15.33:0/4152455107 learned_addr learned my addr 172.21.15.33:0/4152455107 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:12:23.232 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:22.031+0000 7f4d03400700 1 -- 172.21.15.33:0/4152455107 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d0419f530 con 0x7f4d0410ae40 2024-05-03T18:12:23.232 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:22.032+0000 7f4d03400700 1 --2- 172.21.15.33:0/4152455107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d0410ae40 0x7f4d0419eff0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4cf4011670 tx=0x7f4cf4011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:12:23.232 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:22.032+0000 7f4d00c00700 1 -- 172.21.15.33:0/4152455107 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4cf401f070 con 0x7f4d0410ae40 2024-05-03T18:12:23.232 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:22.032+0000 7f4d00c00700 1 -- 172.21.15.33:0/4152455107 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4cf4003b20 con 0x7f4d0410ae40 2024-05-03T18:12:23.232 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:22.032+0000 7f4d00c00700 1 -- 172.21.15.33:0/4152455107 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4cf4019580 con 0x7f4d0410ae40 2024-05-03T18:12:23.232 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:22.033+0000 7f4d0a600700 1 -- 172.21.15.33:0/4152455107 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d0419f7c0 con 0x7f4d0410ae40 2024-05-03T18:12:23.232 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:22.033+0000 7f4d0a600700 1 -- 172.21.15.33:0/4152455107 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d041a24b0 con 0x7f4d0410ae40 2024-05-03T18:12:23.232 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:22.034+0000 7f4d0a600700 1 -- 172.21.15.33:0/4152455107 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d0410f3a0 con 0x7f4d0410ae40 2024-05-03T18:12:23.233 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:22.035+0000 7f4d00c00700 1 -- 172.21.15.33:0/4152455107 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7f4cf4022070 con 0x7f4d0410ae40 2024-05-03T18:12:23.233 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:22.036+0000 7f4d00c00700 1 --2- 172.21.15.33:0/4152455107 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f4ce803e520 0x7f4ce80409e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:12:23.233 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:22.036+0000 7f4d00c00700 1 -- 172.21.15.33:0/4152455107 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f4cf40538e0 con 0x7f4d0410ae40 2024-05-03T18:12:23.233 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:22.036+0000 7f4cfbe00700 1 --2- 172.21.15.33:0/4152455107 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f4ce803e520 0x7f4ce80409e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:12:23.233 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:22.037+0000 7f4cfbe00700 1 --2- 172.21.15.33:0/4152455107 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f4ce803e520 0x7f4ce80409e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4cec006fd0 tx=0x7f4cec006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:12:23.233 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:22.042+0000 7f4d00c00700 1 -- 172.21.15.33:0/4152455107 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f4cf4029070 con 0x7f4d0410ae40 2024-05-03T18:12:23.233 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:22.294+0000 7f4d0a600700 1 -- 172.21.15.33:0/4152455107 --> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f4d0402e560 con 0x7f4ce803e520 2024-05-03T18:12:23.233 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:22.563+0000 7f4d00c00700 1 -- 172.21.15.33:0/4152455107 <== mgr.14152 v2:172.21.15.33:6800/3642895501 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f4d0402e560 con 0x7f4ce803e520 2024-05-03T18:12:23.233 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:22.566+0000 7f4d0a600700 1 -- 172.21.15.33:0/4152455107 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f4ce803e520 msgr2=0x7f4ce80409e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:12:23.233 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:22.566+0000 7f4d0a600700 1 --2- 172.21.15.33:0/4152455107 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f4ce803e520 0x7f4ce80409e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4cec006fd0 tx=0x7f4cec006e40 comp rx=0 tx=0).stop 2024-05-03T18:12:23.233 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:22.566+0000 7f4d0a600700 1 -- 172.21.15.33:0/4152455107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d0410ae40 msgr2=0x7f4d0419eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:12:23.233 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:22.566+0000 7f4d0a600700 1 --2- 172.21.15.33:0/4152455107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d0410ae40 0x7f4d0419eff0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4cf4011670 tx=0x7f4cf4011a30 comp rx=0 tx=0).stop 2024-05-03T18:12:23.233 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:22.566+0000 7f4d0a600700 1 -- 172.21.15.33:0/4152455107 shutdown_connections 2024-05-03T18:12:23.233 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:22.566+0000 7f4d0a600700 1 --2- 172.21.15.33:0/4152455107 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f4ce803e520 0x7f4ce80409e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:12:23.234 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:22.566+0000 7f4d0a600700 1 --2- 172.21.15.33:0/4152455107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d0410ae40 0x7f4d0419eff0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:12:23.234 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:22.566+0000 7f4d0a600700 1 -- 172.21.15.33:0/4152455107 >> 172.21.15.33:0/4152455107 conn(0x7f4d0407e230 msgr2=0x7f4d04108660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:12:23.234 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:22.566+0000 7f4d0a600700 1 -- 172.21.15.33:0/4152455107 shutdown_connections 2024-05-03T18:12:23.234 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:22.567+0000 7f4d0a600700 1 -- 172.21.15.33:0/4152455107 wait complete. 2024-05-03T18:12:23.234 INFO:teuthology.orchestra.run.smithi033.stdout:Creating initial admin user... 2024-05-03T18:12:23.812 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:12:23 smithi033 ceph-mon[28330]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:12:23.812 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:12:23 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:12:23.812 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:12:23 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:12:25.040 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$4u8NzN9vfy9Sj/yXUApvueBqrYX4mDooTO7Z5DLPZB1z4WkeYIRaS", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1714759944, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-05-03T18:12:25.040 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:23.813+0000 7f5a68a00700 1 Processor -- start 2024-05-03T18:12:25.040 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:23.813+0000 7f5a68a00700 1 -- start start 2024-05-03T18:12:25.040 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:23.813+0000 7f5a68a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5a64107ac0 0x7f5a64107ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:12:25.040 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:23.814+0000 7f5a68a00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5a641084b0 con 0x7f5a64107ac0 2024-05-03T18:12:25.040 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:23.814+0000 7f5a61e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5a64107ac0 0x7f5a64107ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:12:25.040 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:23.814+0000 7f5a61e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5a64107ac0 0x7f5a64107ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44372/0 (socket says 172.21.15.33:44372) 2024-05-03T18:12:25.041 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:23.814+0000 7f5a61e00700 1 -- 172.21.15.33:0/3533066538 learned_addr learned my addr 172.21.15.33:0/3533066538 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:12:25.041 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:23.815+0000 7f5a61e00700 1 -- 172.21.15.33:0/3533066538 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a64108cc0 con 0x7f5a64107ac0 2024-05-03T18:12:25.041 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:23.815+0000 7f5a61e00700 1 --2- 172.21.15.33:0/3533066538 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5a64107ac0 0x7f5a64107ee0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5a54009e80 tx=0x7f5a5400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e76be7251c96e90 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:12:25.041 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:23.815+0000 7f5a61400700 1 -- 172.21.15.33:0/3533066538 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5a54017070 con 0x7f5a64107ac0 2024-05-03T18:12:25.041 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:23.816+0000 7f5a61400700 1 -- 172.21.15.33:0/3533066538 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5a5400b8c0 con 0x7f5a64107ac0 2024-05-03T18:12:25.041 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:23.816+0000 7f5a61400700 1 -- 172.21.15.33:0/3533066538 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5a54004420 con 0x7f5a64107ac0 2024-05-03T18:12:25.041 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:23.817+0000 7f5a68a00700 1 -- 172.21.15.33:0/3533066538 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5a64107ac0 msgr2=0x7f5a64107ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:12:25.041 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:23.817+0000 7f5a68a00700 1 --2- 172.21.15.33:0/3533066538 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5a64107ac0 0x7f5a64107ee0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5a54009e80 tx=0x7f5a5400b330 comp rx=0 tx=0).stop 2024-05-03T18:12:25.041 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:23.817+0000 7f5a68a00700 1 -- 172.21.15.33:0/3533066538 shutdown_connections 2024-05-03T18:12:25.041 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:23.817+0000 7f5a68a00700 1 --2- 172.21.15.33:0/3533066538 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5a64107ac0 0x7f5a64107ee0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:12:25.041 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:23.817+0000 7f5a68a00700 1 -- 172.21.15.33:0/3533066538 >> 172.21.15.33:0/3533066538 conn(0x7f5a641030a0 msgr2=0x7f5a641054c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:12:25.042 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:23.817+0000 7f5a68a00700 1 -- 172.21.15.33:0/3533066538 shutdown_connections 2024-05-03T18:12:25.042 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:23.817+0000 7f5a68a00700 1 -- 172.21.15.33:0/3533066538 wait complete. 2024-05-03T18:12:25.042 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:23.818+0000 7f5a68a00700 1 Processor -- start 2024-05-03T18:12:25.042 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:23.818+0000 7f5a68a00700 1 -- start start 2024-05-03T18:12:25.042 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:23.818+0000 7f5a68a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5a64107ac0 0x7f5a6419f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:12:25.042 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:23.819+0000 7f5a68a00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5a6410bef0 con 0x7f5a64107ac0 2024-05-03T18:12:25.042 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:23.819+0000 7f5a61e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5a64107ac0 0x7f5a6419f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:12:25.042 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:23.819+0000 7f5a61e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5a64107ac0 0x7f5a6419f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44386/0 (socket says 172.21.15.33:44386) 2024-05-03T18:12:25.042 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:23.819+0000 7f5a61e00700 1 -- 172.21.15.33:0/3253791294 learned_addr learned my addr 172.21.15.33:0/3253791294 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:12:25.043 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:23.819+0000 7f5a61e00700 1 -- 172.21.15.33:0/3253791294 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a6419f5a0 con 0x7f5a64107ac0 2024-05-03T18:12:25.043 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:23.820+0000 7f5a61e00700 1 --2- 172.21.15.33:0/3253791294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5a64107ac0 0x7f5a6419f060 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5a54004000 tx=0x7f5a5400bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:12:25.043 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:23.820+0000 7f5a5b400700 1 -- 172.21.15.33:0/3253791294 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5a54020020 con 0x7f5a64107ac0 2024-05-03T18:12:25.043 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:23.820+0000 7f5a5b400700 1 -- 172.21.15.33:0/3253791294 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5a540128c0 con 0x7f5a64107ac0 2024-05-03T18:12:25.043 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:23.820+0000 7f5a68a00700 1 -- 172.21.15.33:0/3253791294 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a6419f830 con 0x7f5a64107ac0 2024-05-03T18:12:25.043 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:23.820+0000 7f5a5b400700 1 -- 172.21.15.33:0/3253791294 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5a54017040 con 0x7f5a64107ac0 2024-05-03T18:12:25.043 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:23.820+0000 7f5a68a00700 1 -- 172.21.15.33:0/3253791294 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a641a2520 con 0x7f5a64107ac0 2024-05-03T18:12:25.043 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:23.821+0000 7f5a5b400700 1 -- 172.21.15.33:0/3253791294 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7f5a54023020 con 0x7f5a64107ac0 2024-05-03T18:12:25.043 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:23.822+0000 7f5a68a00700 1 -- 172.21.15.33:0/3253791294 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a6410c070 con 0x7f5a64107ac0 2024-05-03T18:12:25.043 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:23.822+0000 7f5a5b400700 1 --2- 172.21.15.33:0/3253791294 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f5a4803e4c0 0x7f5a48040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:12:25.044 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:23.822+0000 7f5a5b400700 1 -- 172.21.15.33:0/3253791294 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f5a5400f050 con 0x7f5a64107ac0 2024-05-03T18:12:25.044 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:23.822+0000 7f5a59400700 1 --2- 172.21.15.33:0/3253791294 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f5a4803e4c0 0x7f5a48040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:12:25.044 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:23.823+0000 7f5a59400700 1 --2- 172.21.15.33:0/3253791294 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f5a4803e4c0 0x7f5a48040980 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c006fd0 tx=0x7f5a4c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:12:25.044 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:23.828+0000 7f5a5b400700 1 -- 172.21.15.33:0/3253791294 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f5a5402a020 con 0x7f5a64107ac0 2024-05-03T18:12:25.044 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:24.081+0000 7f5a68a00700 1 -- 172.21.15.33:0/3253791294 --> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f5a64197740 con 0x7f5a4803e4c0 2024-05-03T18:12:25.044 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:24.360+0000 7f5a5b400700 1 -- 172.21.15.33:0/3253791294 <== mgr.14152 v2:172.21.15.33:6800/3642895501 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f5a64197740 con 0x7f5a4803e4c0 2024-05-03T18:12:25.044 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:24.363+0000 7f5a68a00700 1 -- 172.21.15.33:0/3253791294 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f5a4803e4c0 msgr2=0x7f5a48040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:12:25.044 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:24.363+0000 7f5a68a00700 1 --2- 172.21.15.33:0/3253791294 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f5a4803e4c0 0x7f5a48040980 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c006fd0 tx=0x7f5a4c006e40 comp rx=0 tx=0).stop 2024-05-03T18:12:25.045 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:24.363+0000 7f5a68a00700 1 -- 172.21.15.33:0/3253791294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5a64107ac0 msgr2=0x7f5a6419f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:12:25.045 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:24.363+0000 7f5a68a00700 1 --2- 172.21.15.33:0/3253791294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5a64107ac0 0x7f5a6419f060 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5a54004000 tx=0x7f5a5400bd50 comp rx=0 tx=0).stop 2024-05-03T18:12:25.045 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:24.363+0000 7f5a68a00700 1 -- 172.21.15.33:0/3253791294 shutdown_connections 2024-05-03T18:12:25.045 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:24.363+0000 7f5a68a00700 1 --2- 172.21.15.33:0/3253791294 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f5a4803e4c0 0x7f5a48040980 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:12:25.045 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:24.363+0000 7f5a68a00700 1 --2- 172.21.15.33:0/3253791294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5a64107ac0 0x7f5a6419f060 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:12:25.045 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:24.363+0000 7f5a68a00700 1 -- 172.21.15.33:0/3253791294 >> 172.21.15.33:0/3253791294 conn(0x7f5a641030a0 msgr2=0x7f5a64103fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:12:25.045 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:24.363+0000 7f5a68a00700 1 -- 172.21.15.33:0/3253791294 shutdown_connections 2024-05-03T18:12:25.045 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:24.363+0000 7f5a68a00700 1 -- 172.21.15.33:0/3253791294 wait complete. 2024-05-03T18:12:25.045 INFO:teuthology.orchestra.run.smithi033.stdout:Fetching dashboard port number... 2024-05-03T18:12:25.361 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:12:25 smithi033 ceph-mon[28330]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:12:25.361 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:12:25 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:12:25.361 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:12:25 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:12:26.232 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:12:26 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/3252331647' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-05-03T18:12:27.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:12:27 smithi033 ceph-mon[28330]: mgrmap e13: x(active, since 6s) 2024-05-03T18:12:27.419 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 8443 2024-05-03T18:12:27.419 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:25.662+0000 7f8917e00700 1 Processor -- start 2024-05-03T18:12:27.419 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:25.662+0000 7f8917e00700 1 -- start start 2024-05-03T18:12:27.419 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:25.662+0000 7f8917e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8910111400 0x7f8910111820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:12:27.419 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:25.663+0000 7f8917e00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8910111df0 con 0x7f8910111400 2024-05-03T18:12:27.419 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:25.663+0000 7f8915400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8910111400 0x7f8910111820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:12:27.419 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:25.663+0000 7f8915400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8910111400 0x7f8910111820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44402/0 (socket says 172.21.15.33:44402) 2024-05-03T18:12:27.419 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:25.663+0000 7f8915400700 1 -- 172.21.15.33:0/3000208911 learned_addr learned my addr 172.21.15.33:0/3000208911 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:12:27.419 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:25.663+0000 7f8915400700 1 -- 172.21.15.33:0/3000208911 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8910112600 con 0x7f8910111400 2024-05-03T18:12:27.419 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:25.664+0000 7f8915400700 1 --2- 172.21.15.33:0/3000208911 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8910111400 0x7f8910111820 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f89040098a0 tx=0x7f8904009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfd4c12452585403 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:12:27.419 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:25.664+0000 7f890fe00700 1 -- 172.21.15.33:0/3000208911 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8904004160 con 0x7f8910111400 2024-05-03T18:12:27.420 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:25.664+0000 7f890fe00700 1 -- 172.21.15.33:0/3000208911 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8904004300 con 0x7f8910111400 2024-05-03T18:12:27.420 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:25.664+0000 7f890fe00700 1 -- 172.21.15.33:0/3000208911 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8904011960 con 0x7f8910111400 2024-05-03T18:12:27.420 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:25.665+0000 7f8917e00700 1 -- 172.21.15.33:0/3000208911 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8910111400 msgr2=0x7f8910111820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:12:27.420 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:25.665+0000 7f8917e00700 1 --2- 172.21.15.33:0/3000208911 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8910111400 0x7f8910111820 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f89040098a0 tx=0x7f8904009bb0 comp rx=0 tx=0).stop 2024-05-03T18:12:27.420 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:25.665+0000 7f8917e00700 1 -- 172.21.15.33:0/3000208911 shutdown_connections 2024-05-03T18:12:27.420 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:25.665+0000 7f8917e00700 1 --2- 172.21.15.33:0/3000208911 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8910111400 0x7f8910111820 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:12:27.420 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:25.665+0000 7f8917e00700 1 -- 172.21.15.33:0/3000208911 >> 172.21.15.33:0/3000208911 conn(0x7f89100714a0 msgr2=0x7f89100738c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:12:27.420 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:25.665+0000 7f8917e00700 1 -- 172.21.15.33:0/3000208911 shutdown_connections 2024-05-03T18:12:27.420 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:25.665+0000 7f8917e00700 1 -- 172.21.15.33:0/3000208911 wait complete. 2024-05-03T18:12:27.420 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:25.666+0000 7f8917e00700 1 Processor -- start 2024-05-03T18:12:27.420 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:25.666+0000 7f8917e00700 1 -- start start 2024-05-03T18:12:27.420 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:25.667+0000 7f8917e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8910111400 0x7f89101a74d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:12:27.420 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:25.667+0000 7f8917e00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8910115830 con 0x7f8910111400 2024-05-03T18:12:27.421 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:25.667+0000 7f8915400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8910111400 0x7f89101a74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:12:27.421 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:25.667+0000 7f8915400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8910111400 0x7f89101a74d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44416/0 (socket says 172.21.15.33:44416) 2024-05-03T18:12:27.421 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:25.667+0000 7f8915400700 1 -- 172.21.15.33:0/3252331647 learned_addr learned my addr 172.21.15.33:0/3252331647 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:12:27.421 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:25.668+0000 7f8915400700 1 -- 172.21.15.33:0/3252331647 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89101a7a10 con 0x7f8910111400 2024-05-03T18:12:27.421 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:25.668+0000 7f8915400700 1 --2- 172.21.15.33:0/3252331647 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8910111400 0x7f89101a74d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8904011670 tx=0x7f8904011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:12:27.421 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:25.668+0000 7f890e000700 1 -- 172.21.15.33:0/3252331647 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8904003aa0 con 0x7f8910111400 2024-05-03T18:12:27.421 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:25.668+0000 7f890e000700 1 -- 172.21.15.33:0/3252331647 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f890401b8d0 con 0x7f8910111400 2024-05-03T18:12:27.421 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:25.668+0000 7f890e000700 1 -- 172.21.15.33:0/3252331647 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f89040197f0 con 0x7f8910111400 2024-05-03T18:12:27.421 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:25.668+0000 7f8917e00700 1 -- 172.21.15.33:0/3252331647 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89101a7ca0 con 0x7f8910111400 2024-05-03T18:12:27.421 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:25.668+0000 7f8917e00700 1 -- 172.21.15.33:0/3252331647 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89101aa990 con 0x7f8910111400 2024-05-03T18:12:27.421 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:25.669+0000 7f8917e00700 1 -- 172.21.15.33:0/3252331647 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89101159b0 con 0x7f8910111400 2024-05-03T18:12:27.421 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:25.670+0000 7f890e000700 1 -- 172.21.15.33:0/3252331647 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7f890401b440 con 0x7f8910111400 2024-05-03T18:12:27.421 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:25.670+0000 7f890e000700 1 --2- 172.21.15.33:0/3252331647 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f88f803e4d0 0x7f88f8040990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:12:27.421 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:25.670+0000 7f890e000700 1 -- 172.21.15.33:0/3252331647 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f8904054040 con 0x7f8910111400 2024-05-03T18:12:27.422 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:25.670+0000 7f8914a00700 1 --2- 172.21.15.33:0/3252331647 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f88f803e4d0 0x7f88f8040990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:12:27.422 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:25.671+0000 7f8914a00700 1 --2- 172.21.15.33:0/3252331647 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f88f803e4d0 0x7f88f8040990 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f88fc006fd0 tx=0x7f88fc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:12:27.422 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:25.675+0000 7f890e000700 1 -- 172.21.15.33:0/3252331647 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f890401f030 con 0x7f8910111400 2024-05-03T18:12:27.422 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:25.915+0000 7f8917e00700 1 -- 172.21.15.33:0/3252331647 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f89101aab60 con 0x7f8910111400 2024-05-03T18:12:27.422 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:25.915+0000 7f890e000700 1 -- 172.21.15.33:0/3252331647 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f8904017e90 con 0x7f8910111400 2024-05-03T18:12:27.422 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:25.918+0000 7f8917e00700 1 -- 172.21.15.33:0/3252331647 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f88f803e4d0 msgr2=0x7f88f8040990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:12:27.422 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:25.918+0000 7f8917e00700 1 --2- 172.21.15.33:0/3252331647 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f88f803e4d0 0x7f88f8040990 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f88fc006fd0 tx=0x7f88fc006e40 comp rx=0 tx=0).stop 2024-05-03T18:12:27.422 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:25.918+0000 7f8917e00700 1 -- 172.21.15.33:0/3252331647 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8910111400 msgr2=0x7f89101a74d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:12:27.422 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:25.918+0000 7f8917e00700 1 --2- 172.21.15.33:0/3252331647 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8910111400 0x7f89101a74d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8904011670 tx=0x7f8904011a30 comp rx=0 tx=0).stop 2024-05-03T18:12:27.422 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:25.919+0000 7f8917e00700 1 -- 172.21.15.33:0/3252331647 shutdown_connections 2024-05-03T18:12:27.422 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:25.919+0000 7f8917e00700 1 --2- 172.21.15.33:0/3252331647 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f88f803e4d0 0x7f88f8040990 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:12:27.422 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:25.919+0000 7f8917e00700 1 --2- 172.21.15.33:0/3252331647 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8910111400 0x7f89101a74d0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:12:27.423 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:25.919+0000 7f8917e00700 1 -- 172.21.15.33:0/3252331647 >> 172.21.15.33:0/3252331647 conn(0x7f89100714a0 msgr2=0x7f8910072f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:12:27.423 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:25.919+0000 7f8917e00700 1 -- 172.21.15.33:0/3252331647 shutdown_connections 2024-05-03T18:12:27.423 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:25.919+0000 7f8917e00700 1 -- 172.21.15.33:0/3252331647 wait complete. 2024-05-03T18:12:27.447 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-03T18:12:27.447 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout disabled 2024-05-03T18:12:27.470 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-03T18:12:27.470 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout inactive 2024-05-03T18:12:27.470 INFO:teuthology.orchestra.run.smithi033.stdout:firewalld.service is not enabled 2024-05-03T18:12:27.470 INFO:teuthology.orchestra.run.smithi033.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-05-03T18:12:27.474 INFO:teuthology.orchestra.run.smithi033.stdout:Ceph Dashboard is now available at: 2024-05-03T18:12:27.475 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:12:27.475 INFO:teuthology.orchestra.run.smithi033.stdout: URL: https://smithi033.front.sepia.ceph.com:8443/ 2024-05-03T18:12:27.475 INFO:teuthology.orchestra.run.smithi033.stdout: User: admin 2024-05-03T18:12:27.475 INFO:teuthology.orchestra.run.smithi033.stdout: Password: vjfp2w1xc1 2024-05-03T18:12:27.475 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:12:27.475 INFO:teuthology.orchestra.run.smithi033.stdout:Saving cluster configuration to /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/config directory 2024-05-03T18:12:27.476 INFO:teuthology.orchestra.run.smithi033.stdout:Enabling autotune for osd_memory_target 2024-05-03T18:12:28.978 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:28.082+0000 7f456f000700 1 Processor -- start 2024-05-03T18:12:28.985 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:28.082+0000 7f456f000700 1 -- start start 2024-05-03T18:12:28.985 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:28.083+0000 7f456f000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4568109630 0x7f456810ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:12:28.985 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:28.083+0000 7f456f000700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f456807ce50 con 0x7f4568109630 2024-05-03T18:12:28.985 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:28.083+0000 7f4567e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4568109630 0x7f456810ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:12:28.985 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:28.083+0000 7f4567e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4568109630 0x7f456810ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58180/0 (socket says 172.21.15.33:58180) 2024-05-03T18:12:28.985 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:28.083+0000 7f4567e00700 1 -- 172.21.15.33:0/380249699 learned_addr learned my addr 172.21.15.33:0/380249699 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:12:28.985 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:28.084+0000 7f4567e00700 1 -- 172.21.15.33:0/380249699 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f456810bf60 con 0x7f4568109630 2024-05-03T18:12:28.985 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:28.084+0000 7f4567e00700 1 --2- 172.21.15.33:0/380249699 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4568109630 0x7f456810ba20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f45580098a0 tx=0x7f4558009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=546d3917cbcb2623 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:12:28.985 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:28.085+0000 7f4566a00700 1 -- 172.21.15.33:0/380249699 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4558004160 con 0x7f4568109630 2024-05-03T18:12:28.986 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:28.085+0000 7f4566a00700 1 -- 172.21.15.33:0/380249699 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4558004300 con 0x7f4568109630 2024-05-03T18:12:28.986 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:28.085+0000 7f4566a00700 1 -- 172.21.15.33:0/380249699 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4558011960 con 0x7f4568109630 2024-05-03T18:12:28.986 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:28.085+0000 7f456f000700 1 -- 172.21.15.33:0/380249699 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4568109630 msgr2=0x7f456810ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:12:28.986 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:28.085+0000 7f456f000700 1 --2- 172.21.15.33:0/380249699 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4568109630 0x7f456810ba20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f45580098a0 tx=0x7f4558009bb0 comp rx=0 tx=0).stop 2024-05-03T18:12:28.986 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:28.085+0000 7f456f000700 1 -- 172.21.15.33:0/380249699 shutdown_connections 2024-05-03T18:12:28.986 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:28.085+0000 7f456f000700 1 --2- 172.21.15.33:0/380249699 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4568109630 0x7f456810ba20 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:12:28.986 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:28.085+0000 7f456f000700 1 -- 172.21.15.33:0/380249699 >> 172.21.15.33:0/380249699 conn(0x7f45681030a0 msgr2=0x7f45681054c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:12:28.986 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:28.086+0000 7f456f000700 1 -- 172.21.15.33:0/380249699 shutdown_connections 2024-05-03T18:12:28.986 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:28.086+0000 7f456f000700 1 -- 172.21.15.33:0/380249699 wait complete. 2024-05-03T18:12:28.986 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:28.086+0000 7f456f000700 1 Processor -- start 2024-05-03T18:12:28.986 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:28.087+0000 7f456f000700 1 -- start start 2024-05-03T18:12:28.987 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:28.087+0000 7f456f000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4568109630 0x7f4568108eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:12:28.987 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:28.087+0000 7f456f000700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f456810cf10 con 0x7f4568109630 2024-05-03T18:12:28.987 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:28.087+0000 7f4567e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4568109630 0x7f4568108eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:12:28.987 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:28.087+0000 7f4567e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4568109630 0x7f4568108eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58192/0 (socket says 172.21.15.33:58192) 2024-05-03T18:12:28.987 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:28.087+0000 7f4567e00700 1 -- 172.21.15.33:0/1752880877 learned_addr learned my addr 172.21.15.33:0/1752880877 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:12:28.987 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:28.088+0000 7f4567e00700 1 -- 172.21.15.33:0/1752880877 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45681093f0 con 0x7f4568109630 2024-05-03T18:12:28.987 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:28.088+0000 7f4567e00700 1 --2- 172.21.15.33:0/1752880877 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4568109630 0x7f4568108eb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4558011670 tx=0x7f4558011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:12:28.987 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:28.088+0000 7f4564c00700 1 -- 172.21.15.33:0/1752880877 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4558003a60 con 0x7f4568109630 2024-05-03T18:12:28.987 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:28.088+0000 7f4564c00700 1 -- 172.21.15.33:0/1752880877 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f455801b8d0 con 0x7f4568109630 2024-05-03T18:12:28.987 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:28.088+0000 7f4564c00700 1 -- 172.21.15.33:0/1752880877 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4558023790 con 0x7f4568109630 2024-05-03T18:12:28.987 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:28.088+0000 7f456f000700 1 -- 172.21.15.33:0/1752880877 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45681075e0 con 0x7f4568109630 2024-05-03T18:12:28.988 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:28.089+0000 7f456f000700 1 -- 172.21.15.33:0/1752880877 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4568107b20 con 0x7f4568109630 2024-05-03T18:12:28.988 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:28.090+0000 7f4564c00700 1 -- 172.21.15.33:0/1752880877 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f455801b440 con 0x7f4568109630 2024-05-03T18:12:28.988 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:28.090+0000 7f456f000700 1 -- 172.21.15.33:0/1752880877 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4568064b00 con 0x7f4568109630 2024-05-03T18:12:28.988 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:28.090+0000 7f4564c00700 1 --2- 172.21.15.33:0/1752880877 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f454c03e560 0x7f454c040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:12:28.988 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:28.090+0000 7f4564c00700 1 -- 172.21.15.33:0/1752880877 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f4558054150 con 0x7f4568109630 2024-05-03T18:12:28.988 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:28.091+0000 7f4567400700 1 --2- 172.21.15.33:0/1752880877 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f454c03e560 0x7f454c040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:12:28.988 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:28.091+0000 7f4567400700 1 --2- 172.21.15.33:0/1752880877 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f454c03e560 0x7f454c040a20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4554006fd0 tx=0x7f4554006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:12:28.988 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:28.096+0000 7f4564c00700 1 -- 172.21.15.33:0/1752880877 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f455801f030 con 0x7f4568109630 2024-05-03T18:12:28.988 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:28.338+0000 7f456f000700 1 -- 172.21.15.33:0/1752880877 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7f4568196500 con 0x7f4568109630 2024-05-03T18:12:28.988 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:28.339+0000 7f4564c00700 1 -- 172.21.15.33:0/1752880877 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7f4558023a40 con 0x7f4568109630 2024-05-03T18:12:28.989 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:28.342+0000 7f456f000700 1 -- 172.21.15.33:0/1752880877 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f454c03e560 msgr2=0x7f454c040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:12:28.989 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:28.342+0000 7f456f000700 1 --2- 172.21.15.33:0/1752880877 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f454c03e560 0x7f454c040a20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4554006fd0 tx=0x7f4554006e40 comp rx=0 tx=0).stop 2024-05-03T18:12:28.989 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:28.342+0000 7f456f000700 1 -- 172.21.15.33:0/1752880877 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4568109630 msgr2=0x7f4568108eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:12:28.989 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:28.342+0000 7f456f000700 1 --2- 172.21.15.33:0/1752880877 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4568109630 0x7f4568108eb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4558011670 tx=0x7f4558011a30 comp rx=0 tx=0).stop 2024-05-03T18:12:28.989 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:28.342+0000 7f456f000700 1 -- 172.21.15.33:0/1752880877 shutdown_connections 2024-05-03T18:12:28.989 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:28.342+0000 7f456f000700 1 --2- 172.21.15.33:0/1752880877 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f454c03e560 0x7f454c040a20 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:12:28.989 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:28.342+0000 7f456f000700 1 --2- 172.21.15.33:0/1752880877 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4568109630 0x7f4568108eb0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:12:28.989 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:28.342+0000 7f456f000700 1 -- 172.21.15.33:0/1752880877 >> 172.21.15.33:0/1752880877 conn(0x7f45681030a0 msgr2=0x7f4568103d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:12:28.989 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:28.342+0000 7f456f000700 1 -- 172.21.15.33:0/1752880877 shutdown_connections 2024-05-03T18:12:28.989 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:28.342+0000 7f456f000700 1 -- 172.21.15.33:0/1752880877 wait complete. 2024-05-03T18:12:30.663 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:29.573+0000 7f8f7b400700 1 Processor -- start 2024-05-03T18:12:30.663 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:29.573+0000 7f8f7b400700 1 -- start start 2024-05-03T18:12:30.663 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:29.573+0000 7f8f7b400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f74108bf0 0x7f8f74109010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:12:30.663 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:29.574+0000 7f8f7b400700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8f741095e0 con 0x7f8f74108bf0 2024-05-03T18:12:30.663 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:29.574+0000 7f8f78a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f74108bf0 0x7f8f74109010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:12:30.663 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:29.574+0000 7f8f78a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f74108bf0 0x7f8f74109010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58208/0 (socket says 172.21.15.33:58208) 2024-05-03T18:12:30.663 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:29.574+0000 7f8f78a00700 1 -- 172.21.15.33:0/3289796916 learned_addr learned my addr 172.21.15.33:0/3289796916 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:12:30.664 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:29.575+0000 7f8f78a00700 1 -- 172.21.15.33:0/3289796916 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f74109da0 con 0x7f8f74108bf0 2024-05-03T18:12:30.664 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:29.575+0000 7f8f78a00700 1 --2- 172.21.15.33:0/3289796916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f74108bf0 0x7f8f74109010 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8f680098a0 tx=0x7f8f68009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51ceb3c8ebc12d9d server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:12:30.664 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:29.575+0000 7f8f73400700 1 -- 172.21.15.33:0/3289796916 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8f68004160 con 0x7f8f74108bf0 2024-05-03T18:12:30.664 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:29.575+0000 7f8f73400700 1 -- 172.21.15.33:0/3289796916 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8f68004300 con 0x7f8f74108bf0 2024-05-03T18:12:30.664 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:29.575+0000 7f8f73400700 1 -- 172.21.15.33:0/3289796916 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8f68011960 con 0x7f8f74108bf0 2024-05-03T18:12:30.664 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:29.576+0000 7f8f7b400700 1 -- 172.21.15.33:0/3289796916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f74108bf0 msgr2=0x7f8f74109010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:12:30.664 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:29.576+0000 7f8f7b400700 1 --2- 172.21.15.33:0/3289796916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f74108bf0 0x7f8f74109010 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8f680098a0 tx=0x7f8f68009bb0 comp rx=0 tx=0).stop 2024-05-03T18:12:30.664 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:29.576+0000 7f8f7b400700 1 -- 172.21.15.33:0/3289796916 shutdown_connections 2024-05-03T18:12:30.664 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:29.576+0000 7f8f7b400700 1 --2- 172.21.15.33:0/3289796916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f74108bf0 0x7f8f74109010 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:12:30.664 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:29.576+0000 7f8f7b400700 1 -- 172.21.15.33:0/3289796916 >> 172.21.15.33:0/3289796916 conn(0x7f8f741041b0 msgr2=0x7f8f741065d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:12:30.664 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:29.576+0000 7f8f7b400700 1 -- 172.21.15.33:0/3289796916 shutdown_connections 2024-05-03T18:12:30.664 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:29.576+0000 7f8f7b400700 1 -- 172.21.15.33:0/3289796916 wait complete. 2024-05-03T18:12:30.664 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:29.577+0000 7f8f7b400700 1 Processor -- start 2024-05-03T18:12:30.664 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:29.577+0000 7f8f7b400700 1 -- start start 2024-05-03T18:12:30.664 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:29.578+0000 7f8f7b400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f74108bf0 0x7f8f7419f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:12:30.664 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:29.578+0000 7f8f7b400700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8f7410cfd0 con 0x7f8f74108bf0 2024-05-03T18:12:30.665 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:29.578+0000 7f8f78a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f74108bf0 0x7f8f7419f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:12:30.665 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:29.578+0000 7f8f78a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f74108bf0 0x7f8f7419f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58222/0 (socket says 172.21.15.33:58222) 2024-05-03T18:12:30.665 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:29.578+0000 7f8f78a00700 1 -- 172.21.15.33:0/818555079 learned_addr learned my addr 172.21.15.33:0/818555079 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:12:30.665 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:29.578+0000 7f8f78a00700 1 -- 172.21.15.33:0/818555079 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f7419f570 con 0x7f8f74108bf0 2024-05-03T18:12:30.665 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:29.579+0000 7f8f78a00700 1 --2- 172.21.15.33:0/818555079 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f74108bf0 0x7f8f7419f030 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8f68011670 tx=0x7f8f68011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:12:30.665 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:29.579+0000 7f8f71600700 1 -- 172.21.15.33:0/818555079 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8f6801f070 con 0x7f8f74108bf0 2024-05-03T18:12:30.665 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:29.579+0000 7f8f71600700 1 -- 172.21.15.33:0/818555079 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8f68003b20 con 0x7f8f74108bf0 2024-05-03T18:12:30.665 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:29.579+0000 7f8f7b400700 1 -- 172.21.15.33:0/818555079 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f7419f800 con 0x7f8f74108bf0 2024-05-03T18:12:30.665 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:29.579+0000 7f8f71600700 1 -- 172.21.15.33:0/818555079 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8f68019580 con 0x7f8f74108bf0 2024-05-03T18:12:30.665 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:29.579+0000 7f8f7b400700 1 -- 172.21.15.33:0/818555079 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f741a24f0 con 0x7f8f74108bf0 2024-05-03T18:12:30.665 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:29.580+0000 7f8f71600700 1 -- 172.21.15.33:0/818555079 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f8f68022070 con 0x7f8f74108bf0 2024-05-03T18:12:30.665 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:29.580+0000 7f8f7b400700 1 -- 172.21.15.33:0/818555079 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f7410d150 con 0x7f8f74108bf0 2024-05-03T18:12:30.665 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:29.581+0000 7f8f71600700 1 --2- 172.21.15.33:0/818555079 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f8f5c03e5c0 0x7f8f5c040a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:12:30.666 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:29.581+0000 7f8f71600700 1 -- 172.21.15.33:0/818555079 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f8f68053bd0 con 0x7f8f74108bf0 2024-05-03T18:12:30.666 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:29.581+0000 7f8f73e00700 1 --2- 172.21.15.33:0/818555079 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f8f5c03e5c0 0x7f8f5c040a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:12:30.666 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:29.582+0000 7f8f73e00700 1 --2- 172.21.15.33:0/818555079 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f8f5c03e5c0 0x7f8f5c040a80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8f60006fd0 tx=0x7f8f60006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:12:30.666 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:29.587+0000 7f8f71600700 1 -- 172.21.15.33:0/818555079 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f8f6801ba80 con 0x7f8f74108bf0 2024-05-03T18:12:30.666 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:29.948+0000 7f8f7b400700 1 -- 172.21.15.33:0/818555079 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f8f7402e510 con 0x7f8f74108bf0 2024-05-03T18:12:30.666 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:29.951+0000 7f8f71600700 1 -- 172.21.15.33:0/818555079 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7f8f6802ca40 con 0x7f8f74108bf0 2024-05-03T18:12:30.666 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:29.954+0000 7f8f7b400700 1 -- 172.21.15.33:0/818555079 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f8f5c03e5c0 msgr2=0x7f8f5c040a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:12:30.666 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:29.954+0000 7f8f7b400700 1 --2- 172.21.15.33:0/818555079 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f8f5c03e5c0 0x7f8f5c040a80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8f60006fd0 tx=0x7f8f60006e40 comp rx=0 tx=0).stop 2024-05-03T18:12:30.666 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:29.954+0000 7f8f7b400700 1 -- 172.21.15.33:0/818555079 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f74108bf0 msgr2=0x7f8f7419f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:12:30.666 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:29.954+0000 7f8f7b400700 1 --2- 172.21.15.33:0/818555079 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f74108bf0 0x7f8f7419f030 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8f68011670 tx=0x7f8f68011a30 comp rx=0 tx=0).stop 2024-05-03T18:12:30.666 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:29.954+0000 7f8f7b400700 1 -- 172.21.15.33:0/818555079 shutdown_connections 2024-05-03T18:12:30.666 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:29.954+0000 7f8f7b400700 1 --2- 172.21.15.33:0/818555079 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f8f5c03e5c0 0x7f8f5c040a80 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:12:30.666 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:29.954+0000 7f8f7b400700 1 --2- 172.21.15.33:0/818555079 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f74108bf0 0x7f8f7419f030 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:12:30.667 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:29.954+0000 7f8f7b400700 1 -- 172.21.15.33:0/818555079 >> 172.21.15.33:0/818555079 conn(0x7f8f741041b0 msgr2=0x7f8f74104ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:12:30.667 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:29.954+0000 7f8f7b400700 1 -- 172.21.15.33:0/818555079 shutdown_connections 2024-05-03T18:12:30.667 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-03T18:12:29.955+0000 7f8f7b400700 1 -- 172.21.15.33:0/818555079 wait complete. 2024-05-03T18:12:30.667 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-05-03T18:12:30.667 INFO:teuthology.orchestra.run.smithi033.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-05-03T18:12:30.667 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:12:30.667 INFO:teuthology.orchestra.run.smithi033.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-05-03T18:12:30.667 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:12:30.667 INFO:teuthology.orchestra.run.smithi033.stdout:Or, if you are only running a single cluster on this host: 2024-05-03T18:12:30.667 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:12:30.667 INFO:teuthology.orchestra.run.smithi033.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-05-03T18:12:30.667 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:12:30.667 INFO:teuthology.orchestra.run.smithi033.stdout:Please consider enabling telemetry to help improve Ceph: 2024-05-03T18:12:30.668 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:12:30.668 INFO:teuthology.orchestra.run.smithi033.stdout: ceph telemetry on 2024-05-03T18:12:30.668 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:12:30.668 INFO:teuthology.orchestra.run.smithi033.stdout:For more information see: 2024-05-03T18:12:30.668 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:12:30.668 INFO:teuthology.orchestra.run.smithi033.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-05-03T18:12:30.668 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:12:30.668 INFO:teuthology.orchestra.run.smithi033.stdout:Bootstrap complete. 2024-05-03T18:12:30.729 INFO:tasks.cephadm:Fetching config... 2024-05-03T18:12:30.730 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-03T18:12:30.730 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-05-03T18:12:30.774 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-05-03T18:12:30.774 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-03T18:12:30.774 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-05-03T18:12:30.863 INFO:tasks.cephadm:Fetching mon keyring... 2024-05-03T18:12:30.863 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-03T18:12:30.864 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/keyring of=/dev/stdout 2024-05-03T18:12:30.982 INFO:tasks.cephadm:Fetching pub ssh key... 2024-05-03T18:12:30.982 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-03T18:12:30.982 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-05-03T18:12:31.081 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-05-03T18:12:31.081 DEBUG:teuthology.orchestra.run.smithi033:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC1lmQQo6k2LDBsuIGKtWYqKpOkX6uZZPnuLvYWFSsz1PyVFx7IxRC7vc5iYHtyToWFJMbLQ23lLsma4V9GWKwqbQ+/lYoz2DpkSfDeC3RHoa7d3Kh/Zql/VsFFsw0xH3gc85+iSKTMZUOGUxM34oiG+E1o6jbQ6bztCdkLBkdHQDkNXGOTIKYnguIBpUnU1tLVAjhrwqZrGI4EoKfQ0CYMAMKXMzXRMnsRJ/XLGoZTrDZJPbUWNxCU2qYcDsLoMpJiMxLuMitPg00+tYuzDEvm+3x+rxhF6z9p+iWWDpM9nNCiebNEQl8lRFjA5TweakujLZ53G/+OOqzHVYgA4UTdMtxCwoUpoUD1jmtaLu5lSbYaE7Zhqd9Ve1Kue14wh8DI8jxtKtl7+rxC4CDEFE1Snoq/ibCXvO0ROO7FWGATp6CNS29XvhxsMvbMxEXVmQss/faTrKDZ8LxN2qzmUv0qhTw2kUwSt4gVpJG7B5XgdOu0krKmKbYgubXDCIoaTCM= ceph-5703bba4-0978-11ef-bc97-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-03T18:12:31.188 INFO:teuthology.orchestra.run.smithi033.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC1lmQQo6k2LDBsuIGKtWYqKpOkX6uZZPnuLvYWFSsz1PyVFx7IxRC7vc5iYHtyToWFJMbLQ23lLsma4V9GWKwqbQ+/lYoz2DpkSfDeC3RHoa7d3Kh/Zql/VsFFsw0xH3gc85+iSKTMZUOGUxM34oiG+E1o6jbQ6bztCdkLBkdHQDkNXGOTIKYnguIBpUnU1tLVAjhrwqZrGI4EoKfQ0CYMAMKXMzXRMnsRJ/XLGoZTrDZJPbUWNxCU2qYcDsLoMpJiMxLuMitPg00+tYuzDEvm+3x+rxhF6z9p+iWWDpM9nNCiebNEQl8lRFjA5TweakujLZ53G/+OOqzHVYgA4UTdMtxCwoUpoUD1jmtaLu5lSbYaE7Zhqd9Ve1Kue14wh8DI8jxtKtl7+rxC4CDEFE1Snoq/ibCXvO0ROO7FWGATp6CNS29XvhxsMvbMxEXVmQss/faTrKDZ8LxN2qzmUv0qhTw2kUwSt4gVpJG7B5XgdOu0krKmKbYgubXDCIoaTCM= ceph-5703bba4-0978-11ef-bc97-c7b262605968 2024-05-03T18:12:31.209 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:12:30 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/818555079' entity='client.admin' 2024-05-03T18:12:31.221 DEBUG:teuthology.orchestra.run.smithi099:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC1lmQQo6k2LDBsuIGKtWYqKpOkX6uZZPnuLvYWFSsz1PyVFx7IxRC7vc5iYHtyToWFJMbLQ23lLsma4V9GWKwqbQ+/lYoz2DpkSfDeC3RHoa7d3Kh/Zql/VsFFsw0xH3gc85+iSKTMZUOGUxM34oiG+E1o6jbQ6bztCdkLBkdHQDkNXGOTIKYnguIBpUnU1tLVAjhrwqZrGI4EoKfQ0CYMAMKXMzXRMnsRJ/XLGoZTrDZJPbUWNxCU2qYcDsLoMpJiMxLuMitPg00+tYuzDEvm+3x+rxhF6z9p+iWWDpM9nNCiebNEQl8lRFjA5TweakujLZ53G/+OOqzHVYgA4UTdMtxCwoUpoUD1jmtaLu5lSbYaE7Zhqd9Ve1Kue14wh8DI8jxtKtl7+rxC4CDEFE1Snoq/ibCXvO0ROO7FWGATp6CNS29XvhxsMvbMxEXVmQss/faTrKDZ8LxN2qzmUv0qhTw2kUwSt4gVpJG7B5XgdOu0krKmKbYgubXDCIoaTCM= ceph-5703bba4-0978-11ef-bc97-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-03T18:12:31.334 INFO:teuthology.orchestra.run.smithi099.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC1lmQQo6k2LDBsuIGKtWYqKpOkX6uZZPnuLvYWFSsz1PyVFx7IxRC7vc5iYHtyToWFJMbLQ23lLsma4V9GWKwqbQ+/lYoz2DpkSfDeC3RHoa7d3Kh/Zql/VsFFsw0xH3gc85+iSKTMZUOGUxM34oiG+E1o6jbQ6bztCdkLBkdHQDkNXGOTIKYnguIBpUnU1tLVAjhrwqZrGI4EoKfQ0CYMAMKXMzXRMnsRJ/XLGoZTrDZJPbUWNxCU2qYcDsLoMpJiMxLuMitPg00+tYuzDEvm+3x+rxhF6z9p+iWWDpM9nNCiebNEQl8lRFjA5TweakujLZ53G/+OOqzHVYgA4UTdMtxCwoUpoUD1jmtaLu5lSbYaE7Zhqd9Ve1Kue14wh8DI8jxtKtl7+rxC4CDEFE1Snoq/ibCXvO0ROO7FWGATp6CNS29XvhxsMvbMxEXVmQss/faTrKDZ8LxN2qzmUv0qhTw2kUwSt4gVpJG7B5XgdOu0krKmKbYgubXDCIoaTCM= ceph-5703bba4-0978-11ef-bc97-c7b262605968 2024-05-03T18:12:31.368 DEBUG:teuthology.orchestra.run.smithi150:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC1lmQQo6k2LDBsuIGKtWYqKpOkX6uZZPnuLvYWFSsz1PyVFx7IxRC7vc5iYHtyToWFJMbLQ23lLsma4V9GWKwqbQ+/lYoz2DpkSfDeC3RHoa7d3Kh/Zql/VsFFsw0xH3gc85+iSKTMZUOGUxM34oiG+E1o6jbQ6bztCdkLBkdHQDkNXGOTIKYnguIBpUnU1tLVAjhrwqZrGI4EoKfQ0CYMAMKXMzXRMnsRJ/XLGoZTrDZJPbUWNxCU2qYcDsLoMpJiMxLuMitPg00+tYuzDEvm+3x+rxhF6z9p+iWWDpM9nNCiebNEQl8lRFjA5TweakujLZ53G/+OOqzHVYgA4UTdMtxCwoUpoUD1jmtaLu5lSbYaE7Zhqd9Ve1Kue14wh8DI8jxtKtl7+rxC4CDEFE1Snoq/ibCXvO0ROO7FWGATp6CNS29XvhxsMvbMxEXVmQss/faTrKDZ8LxN2qzmUv0qhTw2kUwSt4gVpJG7B5XgdOu0krKmKbYgubXDCIoaTCM= ceph-5703bba4-0978-11ef-bc97-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-03T18:12:31.486 INFO:teuthology.orchestra.run.smithi150.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC1lmQQo6k2LDBsuIGKtWYqKpOkX6uZZPnuLvYWFSsz1PyVFx7IxRC7vc5iYHtyToWFJMbLQ23lLsma4V9GWKwqbQ+/lYoz2DpkSfDeC3RHoa7d3Kh/Zql/VsFFsw0xH3gc85+iSKTMZUOGUxM34oiG+E1o6jbQ6bztCdkLBkdHQDkNXGOTIKYnguIBpUnU1tLVAjhrwqZrGI4EoKfQ0CYMAMKXMzXRMnsRJ/XLGoZTrDZJPbUWNxCU2qYcDsLoMpJiMxLuMitPg00+tYuzDEvm+3x+rxhF6z9p+iWWDpM9nNCiebNEQl8lRFjA5TweakujLZ53G/+OOqzHVYgA4UTdMtxCwoUpoUD1jmtaLu5lSbYaE7Zhqd9Ve1Kue14wh8DI8jxtKtl7+rxC4CDEFE1Snoq/ibCXvO0ROO7FWGATp6CNS29XvhxsMvbMxEXVmQss/faTrKDZ8LxN2qzmUv0qhTw2kUwSt4gVpJG7B5XgdOu0krKmKbYgubXDCIoaTCM= ceph-5703bba4-0978-11ef-bc97-c7b262605968 2024-05-03T18:12:31.537 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-05-03T18:12:31.962 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:12:35.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:35.141+0000 7f58dd200700 1 -- 172.21.15.33:0/2380415718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58d8101640 msgr2=0x7f58d8101a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:12:35.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:35.141+0000 7f58dd200700 1 --2- 172.21.15.33:0/2380415718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58d8101640 0x7f58d8101a60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f58c8009910 tx=0x7f58c8009c20 comp rx=0 tx=0).stop 2024-05-03T18:12:35.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:35.141+0000 7f58dd200700 1 -- 172.21.15.33:0/2380415718 shutdown_connections 2024-05-03T18:12:35.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:35.141+0000 7f58dd200700 1 --2- 172.21.15.33:0/2380415718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58d8101640 0x7f58d8101a60 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:12:35.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:35.141+0000 7f58dd200700 1 -- 172.21.15.33:0/2380415718 >> 172.21.15.33:0/2380415718 conn(0x7f58d80fd200 msgr2=0x7f58d80ff640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:12:35.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:35.141+0000 7f58dd200700 1 -- 172.21.15.33:0/2380415718 shutdown_connections 2024-05-03T18:12:35.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:35.141+0000 7f58dd200700 1 -- 172.21.15.33:0/2380415718 wait complete. 2024-05-03T18:12:35.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:35.143+0000 7f58dd200700 1 Processor -- start 2024-05-03T18:12:35.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:35.143+0000 7f58dd200700 1 -- start start 2024-05-03T18:12:35.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:35.143+0000 7f58dd200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58d8101640 0x7f58d819a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:12:35.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:35.143+0000 7f58dd200700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f58d810ad40 con 0x7f58d8101640 2024-05-03T18:12:35.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:35.143+0000 7f58d7400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58d8101640 0x7f58d819a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:12:35.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:35.143+0000 7f58d7400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58d8101640 0x7f58d819a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58242/0 (socket says 172.21.15.33:58242) 2024-05-03T18:12:35.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:35.144+0000 7f58d7400700 1 -- 172.21.15.33:0/3131332 learned_addr learned my addr 172.21.15.33:0/3131332 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:12:35.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:35.144+0000 7f58d7400700 1 -- 172.21.15.33:0/3131332 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58d819a9b0 con 0x7f58d8101640 2024-05-03T18:12:35.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:35.144+0000 7f58d7400700 1 --2- 172.21.15.33:0/3131332 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58d8101640 0x7f58d819a470 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f58c8000c00 tx=0x7f58c8005810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:12:35.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:35.145+0000 7f58cfe00700 1 -- 172.21.15.33:0/3131332 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f58c8027070 con 0x7f58d8101640 2024-05-03T18:12:35.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:35.145+0000 7f58cfe00700 1 -- 172.21.15.33:0/3131332 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f58c800f460 con 0x7f58d8101640 2024-05-03T18:12:35.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:35.145+0000 7f58cfe00700 1 -- 172.21.15.33:0/3131332 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f58c8017440 con 0x7f58d8101640 2024-05-03T18:12:35.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:35.145+0000 7f58dd200700 1 -- 172.21.15.33:0/3131332 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58d819ac40 con 0x7f58d8101640 2024-05-03T18:12:35.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:35.145+0000 7f58dd200700 1 -- 172.21.15.33:0/3131332 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58d819d930 con 0x7f58d8101640 2024-05-03T18:12:35.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:35.146+0000 7f58dd200700 1 -- 172.21.15.33:0/3131332 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58d810b4c0 con 0x7f58d8101640 2024-05-03T18:12:35.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:35.146+0000 7f58cfe00700 1 -- 172.21.15.33:0/3131332 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f58c802a070 con 0x7f58d8101640 2024-05-03T18:12:35.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:35.147+0000 7f58cfe00700 1 --2- 172.21.15.33:0/3131332 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f58bc03e570 0x7f58bc040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:12:35.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:35.147+0000 7f58cfe00700 1 -- 172.21.15.33:0/3131332 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f58c80539a0 con 0x7f58d8101640 2024-05-03T18:12:35.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:35.147+0000 7f58d6a00700 1 --2- 172.21.15.33:0/3131332 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f58bc03e570 0x7f58bc040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:12:35.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:35.148+0000 7f58d6a00700 1 --2- 172.21.15.33:0/3131332 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f58bc03e570 0x7f58bc040a30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f58c4006fd0 tx=0x7f58c4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:12:35.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:35.153+0000 7f58cfe00700 1 -- 172.21.15.33:0/3131332 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f58c8013070 con 0x7f58d8101640 2024-05-03T18:12:35.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:35.392+0000 7f58dd200700 1 -- 172.21.15.33:0/3131332 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f58d802e620 con 0x7f58d8101640 2024-05-03T18:12:35.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:35.396+0000 7f58cfe00700 1 -- 172.21.15.33:0/3131332 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f58c800d340 con 0x7f58d8101640 2024-05-03T18:12:35.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:35.399+0000 7f58dd200700 1 -- 172.21.15.33:0/3131332 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f58bc03e570 msgr2=0x7f58bc040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:12:35.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:35.399+0000 7f58dd200700 1 --2- 172.21.15.33:0/3131332 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f58bc03e570 0x7f58bc040a30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f58c4006fd0 tx=0x7f58c4006e40 comp rx=0 tx=0).stop 2024-05-03T18:12:35.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:35.399+0000 7f58dd200700 1 -- 172.21.15.33:0/3131332 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58d8101640 msgr2=0x7f58d819a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:12:35.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:35.399+0000 7f58dd200700 1 --2- 172.21.15.33:0/3131332 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58d8101640 0x7f58d819a470 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f58c8000c00 tx=0x7f58c8005810 comp rx=0 tx=0).stop 2024-05-03T18:12:35.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:35.400+0000 7f58dd200700 1 -- 172.21.15.33:0/3131332 shutdown_connections 2024-05-03T18:12:35.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:35.400+0000 7f58dd200700 1 --2- 172.21.15.33:0/3131332 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f58bc03e570 0x7f58bc040a30 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:12:35.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:35.400+0000 7f58dd200700 1 --2- 172.21.15.33:0/3131332 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58d8101640 0x7f58d819a470 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:12:35.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:35.400+0000 7f58dd200700 1 -- 172.21.15.33:0/3131332 >> 172.21.15.33:0/3131332 conn(0x7f58d80fd200 msgr2=0x7f58d80fddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:12:35.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:35.400+0000 7f58dd200700 1 -- 172.21.15.33:0/3131332 shutdown_connections 2024-05-03T18:12:35.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:35.400+0000 7f58dd200700 1 -- 172.21.15.33:0/3131332 wait complete. 2024-05-03T18:12:36.269 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-05-03T18:12:36.269 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-05-03T18:12:36.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:12:36 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/3131332' entity='client.admin' 2024-05-03T18:12:37.087 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:12:38.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:12:38 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:12:38.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:12:38 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:12:38.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:12:38 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi033", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:12:38.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:12:38 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:12:38.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:12:38 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:12:38.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:12:38 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:12:39.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:39.289+0000 7f8275400700 1 -- 172.21.15.33:0/1140701278 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8270109660 msgr2=0x7f8270109a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:12:39.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:39.289+0000 7f8275400700 1 --2- 172.21.15.33:0/1140701278 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8270109660 0x7f8270109a80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8260009910 tx=0x7f8260009c20 comp rx=0 tx=0).stop 2024-05-03T18:12:39.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:39.290+0000 7f8275400700 1 -- 172.21.15.33:0/1140701278 shutdown_connections 2024-05-03T18:12:39.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:39.290+0000 7f8275400700 1 --2- 172.21.15.33:0/1140701278 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8270109660 0x7f8270109a80 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:12:39.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:39.290+0000 7f8275400700 1 -- 172.21.15.33:0/1140701278 >> 172.21.15.33:0/1140701278 conn(0x7f82700784c0 msgr2=0x7f827007a920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:12:39.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:39.290+0000 7f8275400700 1 -- 172.21.15.33:0/1140701278 shutdown_connections 2024-05-03T18:12:39.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:39.290+0000 7f8275400700 1 -- 172.21.15.33:0/1140701278 wait complete. 2024-05-03T18:12:39.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:39.291+0000 7f8275400700 1 Processor -- start 2024-05-03T18:12:39.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:39.292+0000 7f8275400700 1 -- start start 2024-05-03T18:12:39.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:39.292+0000 7f8275400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8270109660 0x7f827019e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:12:39.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:39.292+0000 7f8275400700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f827010f0e0 con 0x7f8270109660 2024-05-03T18:12:39.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:39.292+0000 7f826fe00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8270109660 0x7f827019e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:12:39.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:39.292+0000 7f826fe00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8270109660 0x7f827019e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36316/0 (socket says 172.21.15.33:36316) 2024-05-03T18:12:39.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:39.292+0000 7f826fe00700 1 -- 172.21.15.33:0/767502790 learned_addr learned my addr 172.21.15.33:0/767502790 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:12:39.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:39.293+0000 7f826fe00700 1 -- 172.21.15.33:0/767502790 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f827019eda0 con 0x7f8270109660 2024-05-03T18:12:39.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:39.293+0000 7f826fe00700 1 --2- 172.21.15.33:0/767502790 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8270109660 0x7f827019e860 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8260000c00 tx=0x7f8260005810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:12:39.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:39.293+0000 7f826cc00700 1 -- 172.21.15.33:0/767502790 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8260027070 con 0x7f8270109660 2024-05-03T18:12:39.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:39.293+0000 7f826cc00700 1 -- 172.21.15.33:0/767502790 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f826000f460 con 0x7f8270109660 2024-05-03T18:12:39.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:39.293+0000 7f826cc00700 1 -- 172.21.15.33:0/767502790 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8260017440 con 0x7f8270109660 2024-05-03T18:12:39.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:39.293+0000 7f8275400700 1 -- 172.21.15.33:0/767502790 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f827019f030 con 0x7f8270109660 2024-05-03T18:12:39.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:39.293+0000 7f8275400700 1 -- 172.21.15.33:0/767502790 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82701a1d20 con 0x7f8270109660 2024-05-03T18:12:39.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:39.295+0000 7f826cc00700 1 -- 172.21.15.33:0/767502790 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f826002a070 con 0x7f8270109660 2024-05-03T18:12:39.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:39.295+0000 7f8275400700 1 -- 172.21.15.33:0/767502790 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f827010f860 con 0x7f8270109660 2024-05-03T18:12:39.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:39.295+0000 7f826cc00700 1 --2- 172.21.15.33:0/767502790 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f825403e570 0x7f8254040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:12:39.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:39.295+0000 7f826cc00700 1 -- 172.21.15.33:0/767502790 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f82600539a0 con 0x7f8270109660 2024-05-03T18:12:39.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:39.296+0000 7f826f400700 1 --2- 172.21.15.33:0/767502790 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f825403e570 0x7f8254040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:12:39.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:39.296+0000 7f826f400700 1 --2- 172.21.15.33:0/767502790 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f825403e570 0x7f8254040a30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f825c006fd0 tx=0x7f825c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:12:39.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:39.301+0000 7f826cc00700 1 -- 172.21.15.33:0/767502790 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f8260013070 con 0x7f8270109660 2024-05-03T18:12:39.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:39.546+0000 7f8275400700 1 -- 172.21.15.33:0/767502790 --> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f827002e620 con 0x7f825403e570 2024-05-03T18:12:39.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:39.555+0000 7f826cc00700 1 -- 172.21.15.33:0/767502790 <== mgr.14152 v2:172.21.15.33:6800/3642895501 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f827002e620 con 0x7f825403e570 2024-05-03T18:12:39.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:39.558+0000 7f8275400700 1 -- 172.21.15.33:0/767502790 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f825403e570 msgr2=0x7f8254040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:12:39.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:39.558+0000 7f8275400700 1 --2- 172.21.15.33:0/767502790 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f825403e570 0x7f8254040a30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f825c006fd0 tx=0x7f825c006e40 comp rx=0 tx=0).stop 2024-05-03T18:12:39.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:39.558+0000 7f8275400700 1 -- 172.21.15.33:0/767502790 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8270109660 msgr2=0x7f827019e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:12:39.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:39.558+0000 7f8275400700 1 --2- 172.21.15.33:0/767502790 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8270109660 0x7f827019e860 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8260000c00 tx=0x7f8260005810 comp rx=0 tx=0).stop 2024-05-03T18:12:39.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:39.558+0000 7f8275400700 1 -- 172.21.15.33:0/767502790 shutdown_connections 2024-05-03T18:12:39.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:39.558+0000 7f8275400700 1 --2- 172.21.15.33:0/767502790 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f825403e570 0x7f8254040a30 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:12:39.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:39.558+0000 7f8275400700 1 --2- 172.21.15.33:0/767502790 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8270109660 0x7f827019e860 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:12:39.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:39.558+0000 7f8275400700 1 -- 172.21.15.33:0/767502790 >> 172.21.15.33:0/767502790 conn(0x7f82700784c0 msgr2=0x7f8270079030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:12:39.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:39.558+0000 7f8275400700 1 -- 172.21.15.33:0/767502790 shutdown_connections 2024-05-03T18:12:39.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:39.558+0000 7f8275400700 1 -- 172.21.15.33:0/767502790 wait complete. 2024-05-03T18:12:40.312 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi099 2024-05-03T18:12:40.312 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-05-03T18:12:40.312 DEBUG:teuthology.orchestra.run.smithi099:> dd of=/etc/ceph/ceph.conf 2024-05-03T18:12:40.368 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-05-03T18:12:40.368 DEBUG:teuthology.orchestra.run.smithi099:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-03T18:12:40.462 INFO:tasks.cephadm:Adding host smithi099 to orchestrator... 2024-05-03T18:12:40.463 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph orch host add smithi099 2024-05-03T18:12:40.892 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:12:40.982 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:12:40 smithi033 ceph-mon[28330]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:12:40.982 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:12:40 smithi033 ceph-mon[28330]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:12:40.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:12:40 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:12:40.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:12:40 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:12:40.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:12:40 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:12:40.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:12:40 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:12:40.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:12:40 smithi033 ceph-mon[28330]: Updating smithi033:/etc/ceph/ceph.conf 2024-05-03T18:12:41.732 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:12:41 smithi033 ceph-mon[28330]: Updating smithi033:/var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-03T18:12:42.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:12:42 smithi033 ceph-mon[28330]: Updating smithi033:/etc/ceph/ceph.client.admin.keyring 2024-05-03T18:12:42.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:12:42 smithi033 ceph-mon[28330]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:12:42.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:12:42 smithi033 ceph-mon[28330]: Updating smithi033:/var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-03T18:12:42.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:12:42 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:12:42.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:12:42 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:12:42.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:12:42 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:12:42.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:42.945+0000 7fae0b000700 1 -- 172.21.15.33:0/918860457 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae04103110 msgr2=0x7fae041034f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:12:42.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:42.945+0000 7fae0b000700 1 --2- 172.21.15.33:0/918860457 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae04103110 0x7fae041034f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fadf4009910 tx=0x7fadf4009c20 comp rx=0 tx=0).stop 2024-05-03T18:12:42.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:42.945+0000 7fae0b000700 1 -- 172.21.15.33:0/918860457 shutdown_connections 2024-05-03T18:12:42.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:42.945+0000 7fae0b000700 1 --2- 172.21.15.33:0/918860457 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae04103110 0x7fae041034f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:12:42.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:42.945+0000 7fae0b000700 1 -- 172.21.15.33:0/918860457 >> 172.21.15.33:0/918860457 conn(0x7fae040fe9e0 msgr2=0x7fae04100e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:12:42.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:42.945+0000 7fae0b000700 1 -- 172.21.15.33:0/918860457 shutdown_connections 2024-05-03T18:12:42.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:42.946+0000 7fae0b000700 1 -- 172.21.15.33:0/918860457 wait complete. 2024-05-03T18:12:42.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:42.947+0000 7fae0b000700 1 Processor -- start 2024-05-03T18:12:42.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:42.947+0000 7fae0b000700 1 -- start start 2024-05-03T18:12:42.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:42.948+0000 7fae0b000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae04103110 0x7fae0419e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:12:42.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:42.948+0000 7fae0b000700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fae0410d240 con 0x7fae04103110 2024-05-03T18:12:42.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:42.948+0000 7fae03e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae04103110 0x7fae0419e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:12:42.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:42.948+0000 7fae03e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae04103110 0x7fae0419e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36338/0 (socket says 172.21.15.33:36338) 2024-05-03T18:12:42.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:42.948+0000 7fae03e00700 1 -- 172.21.15.33:0/4113506946 learned_addr learned my addr 172.21.15.33:0/4113506946 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:12:42.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:42.948+0000 7fae03e00700 1 -- 172.21.15.33:0/4113506946 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae0419e990 con 0x7fae04103110 2024-05-03T18:12:42.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:42.949+0000 7fae03e00700 1 --2- 172.21.15.33:0/4113506946 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae04103110 0x7fae0419e450 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fadf4005660 tx=0x7fadf4005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:12:42.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:42.949+0000 7fae01600700 1 -- 172.21.15.33:0/4113506946 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fadf4026070 con 0x7fae04103110 2024-05-03T18:12:42.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:42.949+0000 7fae01600700 1 -- 172.21.15.33:0/4113506946 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fadf400b6f0 con 0x7fae04103110 2024-05-03T18:12:42.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:42.949+0000 7fae01600700 1 -- 172.21.15.33:0/4113506946 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fadf40037b0 con 0x7fae04103110 2024-05-03T18:12:42.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:42.949+0000 7fae0b000700 1 -- 172.21.15.33:0/4113506946 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae0419ec80 con 0x7fae04103110 2024-05-03T18:12:42.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:42.949+0000 7fae0b000700 1 -- 172.21.15.33:0/4113506946 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae04198610 con 0x7fae04103110 2024-05-03T18:12:42.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:42.950+0000 7fae0b000700 1 -- 172.21.15.33:0/4113506946 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae041045f0 con 0x7fae04103110 2024-05-03T18:12:42.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:42.950+0000 7fae01600700 1 -- 172.21.15.33:0/4113506946 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fadf4029070 con 0x7fae04103110 2024-05-03T18:12:42.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:42.951+0000 7fae01600700 1 --2- 172.21.15.33:0/4113506946 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fade803e570 0x7fade8040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:12:42.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:42.951+0000 7fae01600700 1 -- 172.21.15.33:0/4113506946 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fadf401e070 con 0x7fae04103110 2024-05-03T18:12:42.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:42.951+0000 7fadfb400700 1 --2- 172.21.15.33:0/4113506946 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fade803e570 0x7fade8040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:12:42.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:42.952+0000 7fadfb400700 1 --2- 172.21.15.33:0/4113506946 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fade803e570 0x7fade8040a30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fadec00ad30 tx=0x7fadec0093f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:12:42.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:42.957+0000 7fae01600700 1 -- 172.21.15.33:0/4113506946 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fadf4013070 con 0x7fae04103110 2024-05-03T18:12:43.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:43.202+0000 7fae0b000700 1 -- 172.21.15.33:0/4113506946 --> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi099", "target": ["mon-mgr", ""]}) -- 0x7fae04199b40 con 0x7fade803e570 2024-05-03T18:12:44.982 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:12:44 smithi033 ceph-mon[28330]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:12:44.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:12:44 smithi033 ceph-mon[28330]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi099", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:12:45.982 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:12:45 smithi033 ceph-mon[28330]: Deploying cephadm binary to smithi099 2024-05-03T18:12:46.884 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:12:46 smithi033 ceph-mon[28330]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:12:48.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:12:48 smithi033 ceph-mon[28330]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:12:49.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:49.585+0000 7fae01600700 1 -- 172.21.15.33:0/4113506946 <== mgr.14152 v2:172.21.15.33:6800/3642895501 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7fae04199b40 con 0x7fade803e570 2024-05-03T18:12:49.586 INFO:teuthology.orchestra.run.smithi033.stdout:Added host 'smithi099' with addr '172.21.15.99' 2024-05-03T18:12:49.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:49.589+0000 7fae0b000700 1 -- 172.21.15.33:0/4113506946 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fade803e570 msgr2=0x7fade8040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:12:49.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:49.589+0000 7fae0b000700 1 --2- 172.21.15.33:0/4113506946 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fade803e570 0x7fade8040a30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fadec00ad30 tx=0x7fadec0093f0 comp rx=0 tx=0).stop 2024-05-03T18:12:49.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:49.589+0000 7fae0b000700 1 -- 172.21.15.33:0/4113506946 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae04103110 msgr2=0x7fae0419e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:12:49.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:49.589+0000 7fae0b000700 1 --2- 172.21.15.33:0/4113506946 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae04103110 0x7fae0419e450 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fadf4005660 tx=0x7fadf4005740 comp rx=0 tx=0).stop 2024-05-03T18:12:49.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:49.589+0000 7fae0b000700 1 -- 172.21.15.33:0/4113506946 shutdown_connections 2024-05-03T18:12:49.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:49.589+0000 7fae0b000700 1 --2- 172.21.15.33:0/4113506946 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fade803e570 0x7fade8040a30 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:12:49.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:49.589+0000 7fae0b000700 1 --2- 172.21.15.33:0/4113506946 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae04103110 0x7fae0419e450 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:12:49.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:49.589+0000 7fae0b000700 1 -- 172.21.15.33:0/4113506946 >> 172.21.15.33:0/4113506946 conn(0x7fae040fe9e0 msgr2=0x7fae04104ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:12:49.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:49.589+0000 7fae0b000700 1 -- 172.21.15.33:0/4113506946 shutdown_connections 2024-05-03T18:12:49.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:49.590+0000 7fae0b000700 1 -- 172.21.15.33:0/4113506946 wait complete. 2024-05-03T18:12:50.340 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph orch host ls --format=json 2024-05-03T18:12:50.809 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:12:50.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:12:50 smithi033 ceph-mon[28330]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:12:50.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:12:50 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:12:50.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:12:50 smithi033 ceph-mon[28330]: Added host smithi099 2024-05-03T18:12:50.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:12:50 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:12:50.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:12:50 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:12:52.559 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:12:52 smithi033 ceph-mon[28330]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:12:52.559 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:12:52 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:12:52.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:52.727+0000 7fc9d3e00700 1 -- 172.21.15.33:0/1024849344 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9cc103110 msgr2=0x7fc9cc1034f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:12:52.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:52.727+0000 7fc9d3e00700 1 --2- 172.21.15.33:0/1024849344 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9cc103110 0x7fc9cc1034f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc9c0009910 tx=0x7fc9c0009c20 comp rx=0 tx=0).stop 2024-05-03T18:12:52.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:52.727+0000 7fc9d3e00700 1 -- 172.21.15.33:0/1024849344 shutdown_connections 2024-05-03T18:12:52.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:52.727+0000 7fc9d3e00700 1 --2- 172.21.15.33:0/1024849344 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9cc103110 0x7fc9cc1034f0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:12:52.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:52.727+0000 7fc9d3e00700 1 -- 172.21.15.33:0/1024849344 >> 172.21.15.33:0/1024849344 conn(0x7fc9cc0fe9e0 msgr2=0x7fc9cc100e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:12:52.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:52.727+0000 7fc9d3e00700 1 -- 172.21.15.33:0/1024849344 shutdown_connections 2024-05-03T18:12:52.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:52.727+0000 7fc9d3e00700 1 -- 172.21.15.33:0/1024849344 wait complete. 2024-05-03T18:12:52.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:52.728+0000 7fc9d3e00700 1 Processor -- start 2024-05-03T18:12:52.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:52.728+0000 7fc9d3e00700 1 -- start start 2024-05-03T18:12:52.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:52.729+0000 7fc9d3e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9cc103110 0x7fc9cc0757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:12:52.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:52.729+0000 7fc9d3e00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc9cc10d240 con 0x7fc9cc103110 2024-05-03T18:12:52.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:52.729+0000 7fc9d1400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9cc103110 0x7fc9cc0757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:12:52.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:52.729+0000 7fc9d1400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9cc103110 0x7fc9cc0757a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34656/0 (socket says 172.21.15.33:34656) 2024-05-03T18:12:52.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:52.729+0000 7fc9d1400700 1 -- 172.21.15.33:0/2091736144 learned_addr learned my addr 172.21.15.33:0/2091736144 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:12:52.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:52.730+0000 7fc9d1400700 1 -- 172.21.15.33:0/2091736144 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9cc077690 con 0x7fc9cc103110 2024-05-03T18:12:52.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:52.730+0000 7fc9d1400700 1 --2- 172.21.15.33:0/2091736144 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9cc103110 0x7fc9cc0757a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc9c0006010 tx=0x7fc9c0005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:12:52.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:52.730+0000 7fc9ca000700 1 -- 172.21.15.33:0/2091736144 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc9c0026070 con 0x7fc9cc103110 2024-05-03T18:12:52.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:52.730+0000 7fc9ca000700 1 -- 172.21.15.33:0/2091736144 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc9c000b750 con 0x7fc9cc103110 2024-05-03T18:12:52.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:52.730+0000 7fc9ca000700 1 -- 172.21.15.33:0/2091736144 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc9c0003810 con 0x7fc9cc103110 2024-05-03T18:12:52.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:52.730+0000 7fc9d3e00700 1 -- 172.21.15.33:0/2091736144 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9cc075d40 con 0x7fc9cc103110 2024-05-03T18:12:52.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:52.731+0000 7fc9d3e00700 1 -- 172.21.15.33:0/2091736144 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9cc076200 con 0x7fc9cc103110 2024-05-03T18:12:52.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:52.732+0000 7fc9d3e00700 1 -- 172.21.15.33:0/2091736144 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9cc1045f0 con 0x7fc9cc103110 2024-05-03T18:12:52.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:52.732+0000 7fc9ca000700 1 -- 172.21.15.33:0/2091736144 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fc9c0029070 con 0x7fc9cc103110 2024-05-03T18:12:52.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:52.732+0000 7fc9ca000700 1 --2- 172.21.15.33:0/2091736144 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fc9b403e570 0x7fc9b4040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:12:52.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:52.733+0000 7fc9ca000700 1 -- 172.21.15.33:0/2091736144 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc9c001e070 con 0x7fc9cc103110 2024-05-03T18:12:52.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:52.733+0000 7fc9d0a00700 1 --2- 172.21.15.33:0/2091736144 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fc9b403e570 0x7fc9b4040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:12:52.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:52.734+0000 7fc9d0a00700 1 --2- 172.21.15.33:0/2091736144 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fc9b403e570 0x7fc9b4040a30 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc9b8006fd0 tx=0x7fc9b8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:12:52.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:52.739+0000 7fc9ca000700 1 -- 172.21.15.33:0/2091736144 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fc9c0013070 con 0x7fc9cc103110 2024-05-03T18:12:52.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:52.989+0000 7fc9d3e00700 1 -- 172.21.15.33:0/2091736144 --> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc9cc076cd0 con 0x7fc9b403e570 2024-05-03T18:12:52.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:52.991+0000 7fc9ca000700 1 -- 172.21.15.33:0/2091736144 <== mgr.14152 v2:172.21.15.33:6800/3642895501 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+159 (secure 0 0 0) 0x7fc9cc076cd0 con 0x7fc9b403e570 2024-05-03T18:12:52.991 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:12:52.991 INFO:teuthology.orchestra.run.smithi033.stdout:[{"addr": "172.21.15.33", "hostname": "smithi033", "labels": [], "status": ""}, {"addr": "172.21.15.99", "hostname": "smithi099", "labels": [], "status": ""}] 2024-05-03T18:12:52.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:52.994+0000 7fc9d3e00700 1 -- 172.21.15.33:0/2091736144 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fc9b403e570 msgr2=0x7fc9b4040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:12:52.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:52.994+0000 7fc9d3e00700 1 --2- 172.21.15.33:0/2091736144 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fc9b403e570 0x7fc9b4040a30 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc9b8006fd0 tx=0x7fc9b8006e40 comp rx=0 tx=0).stop 2024-05-03T18:12:52.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:52.994+0000 7fc9d3e00700 1 -- 172.21.15.33:0/2091736144 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9cc103110 msgr2=0x7fc9cc0757a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:12:52.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:52.994+0000 7fc9d3e00700 1 --2- 172.21.15.33:0/2091736144 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9cc103110 0x7fc9cc0757a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc9c0006010 tx=0x7fc9c0005710 comp rx=0 tx=0).stop 2024-05-03T18:12:52.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:52.994+0000 7fc9d3e00700 1 -- 172.21.15.33:0/2091736144 shutdown_connections 2024-05-03T18:12:52.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:52.994+0000 7fc9d3e00700 1 --2- 172.21.15.33:0/2091736144 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fc9b403e570 0x7fc9b4040a30 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:12:52.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:52.994+0000 7fc9d3e00700 1 --2- 172.21.15.33:0/2091736144 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9cc103110 0x7fc9cc0757a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:12:52.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:52.994+0000 7fc9d3e00700 1 -- 172.21.15.33:0/2091736144 >> 172.21.15.33:0/2091736144 conn(0x7fc9cc0fe9e0 msgr2=0x7fc9cc10acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:12:52.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:52.994+0000 7fc9d3e00700 1 -- 172.21.15.33:0/2091736144 shutdown_connections 2024-05-03T18:12:52.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:52.994+0000 7fc9d3e00700 1 -- 172.21.15.33:0/2091736144 wait complete. 2024-05-03T18:12:53.740 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi150 2024-05-03T18:12:53.740 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-05-03T18:12:53.740 DEBUG:teuthology.orchestra.run.smithi150:> dd of=/etc/ceph/ceph.conf 2024-05-03T18:12:53.799 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-05-03T18:12:53.800 DEBUG:teuthology.orchestra.run.smithi150:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-03T18:12:53.898 INFO:tasks.cephadm:Adding host smithi150 to orchestrator... 2024-05-03T18:12:53.898 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph orch host add smithi150 2024-05-03T18:12:54.359 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:12:54.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:12:54 smithi033 ceph-mon[28330]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-03T18:12:54.692 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:12:54 smithi033 ceph-mon[28330]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:12:54.692 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:12:54 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:12:56.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:56.310+0000 7fd320000700 1 -- 172.21.15.33:0/3274211896 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd318105320 msgr2=0x7fd318105700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:12:56.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:56.310+0000 7fd320000700 1 --2- 172.21.15.33:0/3274211896 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd318105320 0x7fd318105700 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd30c009910 tx=0x7fd30c009c20 comp rx=0 tx=0).stop 2024-05-03T18:12:56.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:56.311+0000 7fd320000700 1 -- 172.21.15.33:0/3274211896 shutdown_connections 2024-05-03T18:12:56.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:56.311+0000 7fd320000700 1 --2- 172.21.15.33:0/3274211896 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd318105320 0x7fd318105700 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:12:56.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:56.311+0000 7fd320000700 1 -- 172.21.15.33:0/3274211896 >> 172.21.15.33:0/3274211896 conn(0x7fd318100bd0 msgr2=0x7fd318102ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:12:56.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:56.311+0000 7fd320000700 1 -- 172.21.15.33:0/3274211896 shutdown_connections 2024-05-03T18:12:56.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:56.311+0000 7fd320000700 1 -- 172.21.15.33:0/3274211896 wait complete. 2024-05-03T18:12:56.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:56.312+0000 7fd320000700 1 Processor -- start 2024-05-03T18:12:56.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:56.312+0000 7fd320000700 1 -- start start 2024-05-03T18:12:56.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:56.313+0000 7fd320000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd318105320 0x7fd3181a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:12:56.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:56.313+0000 7fd320000700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd31810f450 con 0x7fd318105320 2024-05-03T18:12:56.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:56.313+0000 7fd31d600700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd318105320 0x7fd3181a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:12:56.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:56.313+0000 7fd31d600700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd318105320 0x7fd3181a06c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51256/0 (socket says 172.21.15.33:51256) 2024-05-03T18:12:56.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:56.313+0000 7fd31d600700 1 -- 172.21.15.33:0/596594135 learned_addr learned my addr 172.21.15.33:0/596594135 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:12:56.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:56.313+0000 7fd31d600700 1 -- 172.21.15.33:0/596594135 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3181a0c00 con 0x7fd318105320 2024-05-03T18:12:56.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:56.314+0000 7fd31d600700 1 --2- 172.21.15.33:0/596594135 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd318105320 0x7fd3181a06c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd30c006010 tx=0x7fd30c005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:12:56.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:56.314+0000 7fd316000700 1 -- 172.21.15.33:0/596594135 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd30c026070 con 0x7fd318105320 2024-05-03T18:12:56.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:56.314+0000 7fd316000700 1 -- 172.21.15.33:0/596594135 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd30c00b6f0 con 0x7fd318105320 2024-05-03T18:12:56.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:56.314+0000 7fd316000700 1 -- 172.21.15.33:0/596594135 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd30c003810 con 0x7fd318105320 2024-05-03T18:12:56.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:56.314+0000 7fd320000700 1 -- 172.21.15.33:0/596594135 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3181a0ef0 con 0x7fd318105320 2024-05-03T18:12:56.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:56.314+0000 7fd320000700 1 -- 172.21.15.33:0/596594135 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd31819a880 con 0x7fd318105320 2024-05-03T18:12:56.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:56.315+0000 7fd320000700 1 -- 172.21.15.33:0/596594135 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd318106800 con 0x7fd318105320 2024-05-03T18:12:56.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:56.315+0000 7fd316000700 1 -- 172.21.15.33:0/596594135 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fd30c029070 con 0x7fd318105320 2024-05-03T18:12:56.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:56.316+0000 7fd316000700 1 --2- 172.21.15.33:0/596594135 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fd30003e510 0x7fd3000409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:12:56.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:56.316+0000 7fd316000700 1 -- 172.21.15.33:0/596594135 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fd30c01e070 con 0x7fd318105320 2024-05-03T18:12:56.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:56.316+0000 7fd31cc00700 1 --2- 172.21.15.33:0/596594135 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fd30003e510 0x7fd3000409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:12:56.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:56.317+0000 7fd31cc00700 1 --2- 172.21.15.33:0/596594135 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fd30003e510 0x7fd3000409d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd304006fd0 tx=0x7fd304006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:12:56.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:56.322+0000 7fd316000700 1 -- 172.21.15.33:0/596594135 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fd30c013070 con 0x7fd318105320 2024-05-03T18:12:56.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:12:56.565+0000 7fd320000700 1 -- 172.21.15.33:0/596594135 --> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi150", "target": ["mon-mgr", ""]}) -- 0x7fd31819bec0 con 0x7fd30003e510 2024-05-03T18:12:56.732 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:12:56 smithi033 ceph-mon[28330]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:12:57.732 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:12:57 smithi033 ceph-mon[28330]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi150", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:12:58.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:12:58 smithi033 ceph-mon[28330]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:12:58.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:12:58 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:12:58.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:12:58 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:12:58.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:12:58 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:12:58.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:12:58 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:12:58.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:12:58 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi099", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:12:58.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:12:58 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:12:58.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:12:58 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:12:59.732 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:12:59 smithi033 ceph-mon[28330]: Deploying cephadm binary to smithi150 2024-05-03T18:12:59.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:12:59 smithi033 ceph-mon[28330]: Updating smithi099:/etc/ceph/ceph.conf 2024-05-03T18:13:00.732 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:00 smithi033 ceph-mon[28330]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:00.732 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:00 smithi033 ceph-mon[28330]: Updating smithi099:/var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-03T18:13:02.732 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:02 smithi033 ceph-mon[28330]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:03.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:03.022+0000 7fd316000700 1 -- 172.21.15.33:0/596594135 <== mgr.14152 v2:172.21.15.33:6800/3642895501 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fd31819bec0 con 0x7fd30003e510 2024-05-03T18:13:03.023 INFO:teuthology.orchestra.run.smithi033.stdout:Added host 'smithi150' with addr '172.21.15.150' 2024-05-03T18:13:03.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:03.026+0000 7fd320000700 1 -- 172.21.15.33:0/596594135 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fd30003e510 msgr2=0x7fd3000409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:13:03.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:03.026+0000 7fd320000700 1 --2- 172.21.15.33:0/596594135 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fd30003e510 0x7fd3000409d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd304006fd0 tx=0x7fd304006e40 comp rx=0 tx=0).stop 2024-05-03T18:13:03.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:03.026+0000 7fd320000700 1 -- 172.21.15.33:0/596594135 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd318105320 msgr2=0x7fd3181a06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:13:03.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:03.026+0000 7fd320000700 1 --2- 172.21.15.33:0/596594135 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd318105320 0x7fd3181a06c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd30c006010 tx=0x7fd30c005710 comp rx=0 tx=0).stop 2024-05-03T18:13:03.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:03.026+0000 7fd320000700 1 -- 172.21.15.33:0/596594135 shutdown_connections 2024-05-03T18:13:03.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:03.026+0000 7fd320000700 1 --2- 172.21.15.33:0/596594135 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fd30003e510 0x7fd3000409d0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:13:03.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:03.026+0000 7fd320000700 1 --2- 172.21.15.33:0/596594135 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd318105320 0x7fd3181a06c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:13:03.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:03.026+0000 7fd320000700 1 -- 172.21.15.33:0/596594135 >> 172.21.15.33:0/596594135 conn(0x7fd318100bd0 msgr2=0x7fd318106cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:13:03.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:03.027+0000 7fd320000700 1 -- 172.21.15.33:0/596594135 shutdown_connections 2024-05-03T18:13:03.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:03.027+0000 7fd320000700 1 -- 172.21.15.33:0/596594135 wait complete. 2024-05-03T18:13:03.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:03 smithi033 ceph-mon[28330]: Updating smithi099:/etc/ceph/ceph.client.admin.keyring 2024-05-03T18:13:03.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:03 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:03.814 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph orch host ls --format=json 2024-05-03T18:13:04.265 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:13:04.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:04 smithi033 ceph-mon[28330]: Added host smithi150 2024-05-03T18:13:04.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:04 smithi033 ceph-mon[28330]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:04.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:04 smithi033 ceph-mon[28330]: Updating smithi099:/var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-03T18:13:04.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:04 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:04.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:04 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:04.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:04 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:05.732 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:05 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:13:05.732 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:05 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:06.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:06.301+0000 7f96c9e00700 1 -- 172.21.15.33:0/3180857341 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96c40758d0 msgr2=0x7f96c4075cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:13:06.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:06.301+0000 7f96c9e00700 1 --2- 172.21.15.33:0/3180857341 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96c40758d0 0x7f96c4075cb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f96b8009910 tx=0x7f96b8009c20 comp rx=0 tx=0).stop 2024-05-03T18:13:06.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:06.302+0000 7f96c9e00700 1 -- 172.21.15.33:0/3180857341 shutdown_connections 2024-05-03T18:13:06.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:06.302+0000 7f96c9e00700 1 --2- 172.21.15.33:0/3180857341 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96c40758d0 0x7f96c4075cb0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:13:06.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:06.302+0000 7f96c9e00700 1 -- 172.21.15.33:0/3180857341 >> 172.21.15.33:0/3180857341 conn(0x7f96c40fe800 msgr2=0x7f96c4100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:13:06.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:06.302+0000 7f96c9e00700 1 -- 172.21.15.33:0/3180857341 shutdown_connections 2024-05-03T18:13:06.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:06.302+0000 7f96c9e00700 1 -- 172.21.15.33:0/3180857341 wait complete. 2024-05-03T18:13:06.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:06.303+0000 7f96c9e00700 1 Processor -- start 2024-05-03T18:13:06.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:06.303+0000 7f96c9e00700 1 -- start start 2024-05-03T18:13:06.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:06.304+0000 7f96c9e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96c41a07d0 0x7f96c41a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:13:06.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:06.304+0000 7f96c9e00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f96c410f300 con 0x7f96c41a07d0 2024-05-03T18:13:06.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:06.304+0000 7f96c8a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96c41a07d0 0x7f96c41a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:13:06.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:06.304+0000 7f96c8a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96c41a07d0 0x7f96c41a0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47476/0 (socket says 172.21.15.33:47476) 2024-05-03T18:13:06.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:06.304+0000 7f96c8a00700 1 -- 172.21.15.33:0/1484089731 learned_addr learned my addr 172.21.15.33:0/1484089731 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:13:06.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:06.305+0000 7f96c8a00700 1 -- 172.21.15.33:0/1484089731 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96c41a10f0 con 0x7f96c41a07d0 2024-05-03T18:13:06.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:06.305+0000 7f96c8a00700 1 --2- 172.21.15.33:0/1484089731 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96c41a07d0 0x7f96c41a0bb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f96b8005950 tx=0x7f96b8005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:13:06.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:06.305+0000 7f96c1600700 1 -- 172.21.15.33:0/1484089731 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96b8026070 con 0x7f96c41a07d0 2024-05-03T18:13:06.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:06.305+0000 7f96c1600700 1 -- 172.21.15.33:0/1484089731 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f96b800b750 con 0x7f96c41a07d0 2024-05-03T18:13:06.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:06.305+0000 7f96c1600700 1 -- 172.21.15.33:0/1484089731 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96b8003810 con 0x7f96c41a07d0 2024-05-03T18:13:06.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:06.306+0000 7f96c9e00700 1 -- 172.21.15.33:0/1484089731 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96c41a1380 con 0x7f96c41a07d0 2024-05-03T18:13:06.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:06.306+0000 7f96c9e00700 1 -- 172.21.15.33:0/1484089731 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96c419ab60 con 0x7f96c41a07d0 2024-05-03T18:13:06.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:06.307+0000 7f96c1600700 1 -- 172.21.15.33:0/1484089731 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f96b8029070 con 0x7f96c41a07d0 2024-05-03T18:13:06.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:06.307+0000 7f96c9e00700 1 -- 172.21.15.33:0/1484089731 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96c4076f80 con 0x7f96c41a07d0 2024-05-03T18:13:06.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:06.308+0000 7f96c1600700 1 --2- 172.21.15.33:0/1484089731 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f96ac03e570 0x7f96ac040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:13:06.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:06.308+0000 7f96c1600700 1 -- 172.21.15.33:0/1484089731 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f96b801e070 con 0x7f96c41a07d0 2024-05-03T18:13:06.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:06.308+0000 7f96c3e00700 1 --2- 172.21.15.33:0/1484089731 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f96ac03e570 0x7f96ac040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:13:06.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:06.309+0000 7f96c3e00700 1 --2- 172.21.15.33:0/1484089731 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f96ac03e570 0x7f96ac040a30 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f96b0006fd0 tx=0x7f96b0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:13:06.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:06.314+0000 7f96c1600700 1 -- 172.21.15.33:0/1484089731 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f96b8013070 con 0x7f96c41a07d0 2024-05-03T18:13:06.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:06.571+0000 7f96c9e00700 1 -- 172.21.15.33:0/1484089731 --> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f96c419ade0 con 0x7f96ac03e570 2024-05-03T18:13:06.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:06.573+0000 7f96c1600700 1 -- 172.21.15.33:0/1484089731 <== mgr.14152 v2:172.21.15.33:6800/3642895501 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+239 (secure 0 0 0) 0x7f96c419ade0 con 0x7f96ac03e570 2024-05-03T18:13:06.573 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:13:06.573 INFO:teuthology.orchestra.run.smithi033.stdout:[{"addr": "172.21.15.33", "hostname": "smithi033", "labels": [], "status": ""}, {"addr": "172.21.15.99", "hostname": "smithi099", "labels": [], "status": ""}, {"addr": "172.21.15.150", "hostname": "smithi150", "labels": [], "status": ""}] 2024-05-03T18:13:06.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:06.576+0000 7f96c9e00700 1 -- 172.21.15.33:0/1484089731 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f96ac03e570 msgr2=0x7f96ac040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:13:06.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:06.576+0000 7f96c9e00700 1 --2- 172.21.15.33:0/1484089731 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f96ac03e570 0x7f96ac040a30 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f96b0006fd0 tx=0x7f96b0006e40 comp rx=0 tx=0).stop 2024-05-03T18:13:06.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:06.576+0000 7f96c9e00700 1 -- 172.21.15.33:0/1484089731 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96c41a07d0 msgr2=0x7f96c41a0bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:13:06.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:06.576+0000 7f96c9e00700 1 --2- 172.21.15.33:0/1484089731 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96c41a07d0 0x7f96c41a0bb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f96b8005950 tx=0x7f96b8005710 comp rx=0 tx=0).stop 2024-05-03T18:13:06.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:06.576+0000 7f96c9e00700 1 -- 172.21.15.33:0/1484089731 shutdown_connections 2024-05-03T18:13:06.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:06.576+0000 7f96c9e00700 1 --2- 172.21.15.33:0/1484089731 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f96ac03e570 0x7f96ac040a30 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:13:06.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:06.576+0000 7f96c9e00700 1 --2- 172.21.15.33:0/1484089731 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96c41a07d0 0x7f96c41a0bb0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:13:06.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:06.576+0000 7f96c9e00700 1 -- 172.21.15.33:0/1484089731 >> 172.21.15.33:0/1484089731 conn(0x7f96c40fe800 msgr2=0x7f96c410c0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:13:06.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:06.576+0000 7f96c9e00700 1 -- 172.21.15.33:0/1484089731 shutdown_connections 2024-05-03T18:13:06.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:06.576+0000 7f96c9e00700 1 -- 172.21.15.33:0/1484089731 wait complete. 2024-05-03T18:13:06.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:06 smithi033 ceph-mon[28330]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:06.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:06 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:07.328 INFO:tasks.cephadm:Setting crush tunables to default 2024-05-03T18:13:07.328 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph osd crush tunables default 2024-05-03T18:13:07.732 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:07 smithi033 ceph-mon[28330]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-03T18:13:07.777 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:13:08.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:08 smithi033 ceph-mon[28330]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:08.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:08 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:09.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:09.738+0000 7f4433e00700 1 -- 172.21.15.33:0/782803900 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4434105170 msgr2=0x7f4434105550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:13:09.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:09.738+0000 7f4433e00700 1 --2- 172.21.15.33:0/782803900 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4434105170 0x7f4434105550 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4424009910 tx=0x7f4424009c20 comp rx=0 tx=0).stop 2024-05-03T18:13:09.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:09.738+0000 7f4433e00700 1 -- 172.21.15.33:0/782803900 shutdown_connections 2024-05-03T18:13:09.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:09.738+0000 7f4433e00700 1 --2- 172.21.15.33:0/782803900 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4434105170 0x7f4434105550 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:13:09.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:09.738+0000 7f4433e00700 1 -- 172.21.15.33:0/782803900 >> 172.21.15.33:0/782803900 conn(0x7f4434100a20 msgr2=0x7f4434102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:13:09.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:09.738+0000 7f4433e00700 1 -- 172.21.15.33:0/782803900 shutdown_connections 2024-05-03T18:13:09.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:09.738+0000 7f4433e00700 1 -- 172.21.15.33:0/782803900 wait complete. 2024-05-03T18:13:09.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:09.739+0000 7f4433e00700 1 Processor -- start 2024-05-03T18:13:09.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:09.739+0000 7f4433e00700 1 -- start start 2024-05-03T18:13:09.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:09.740+0000 7f4433e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4434105170 0x7f44341a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:13:09.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:09.740+0000 7f4433e00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f443410f2a0 con 0x7f4434105170 2024-05-03T18:13:09.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:09.740+0000 7f4432a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4434105170 0x7f44341a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:13:09.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:09.740+0000 7f4432a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4434105170 0x7f44341a0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47482/0 (socket says 172.21.15.33:47482) 2024-05-03T18:13:09.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:09.740+0000 7f4432a00700 1 -- 172.21.15.33:0/1639324663 learned_addr learned my addr 172.21.15.33:0/1639324663 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:13:09.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:09.740+0000 7f4432a00700 1 -- 172.21.15.33:0/1639324663 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44341a0b70 con 0x7f4434105170 2024-05-03T18:13:09.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:09.741+0000 7f4432a00700 1 --2- 172.21.15.33:0/1639324663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4434105170 0x7f44341a0630 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4424005660 tx=0x7f4424005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:13:09.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:09.741+0000 7f4438a00700 1 -- 172.21.15.33:0/1639324663 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4424026070 con 0x7f4434105170 2024-05-03T18:13:09.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:09.741+0000 7f4438a00700 1 -- 172.21.15.33:0/1639324663 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f442400b6f0 con 0x7f4434105170 2024-05-03T18:13:09.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:09.741+0000 7f4433e00700 1 -- 172.21.15.33:0/1639324663 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44341a0e60 con 0x7f4434105170 2024-05-03T18:13:09.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:09.741+0000 7f4438a00700 1 -- 172.21.15.33:0/1639324663 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f44240037b0 con 0x7f4434105170 2024-05-03T18:13:09.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:09.741+0000 7f4433e00700 1 -- 172.21.15.33:0/1639324663 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f443419a8d0 con 0x7f4434105170 2024-05-03T18:13:09.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:09.742+0000 7f4433e00700 1 -- 172.21.15.33:0/1639324663 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4434106650 con 0x7f4434105170 2024-05-03T18:13:09.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:09.742+0000 7f4438a00700 1 -- 172.21.15.33:0/1639324663 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f4424029070 con 0x7f4434105170 2024-05-03T18:13:09.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:09.743+0000 7f4438a00700 1 --2- 172.21.15.33:0/1639324663 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f441803e570 0x7f4418040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:13:09.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:09.743+0000 7f4438a00700 1 -- 172.21.15.33:0/1639324663 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f442401e070 con 0x7f4434105170 2024-05-03T18:13:09.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:09.743+0000 7f4432000700 1 --2- 172.21.15.33:0/1639324663 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f441803e570 0x7f4418040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:13:09.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:09.744+0000 7f4432000700 1 --2- 172.21.15.33:0/1639324663 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f441803e570 0x7f4418040a30 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4420006fd0 tx=0x7f4420006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:13:09.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:09.749+0000 7f4438a00700 1 -- 172.21.15.33:0/1639324663 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f4424013070 con 0x7f4434105170 2024-05-03T18:13:09.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:09.994+0000 7f4433e00700 1 -- 172.21.15.33:0/1639324663 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f443419bc30 con 0x7f4434105170 2024-05-03T18:13:10.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:10.477+0000 7f4438a00700 1 -- 172.21.15.33:0/1639324663 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f4424029360 con 0x7f4434105170 2024-05-03T18:13:10.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:10.482+0000 7f4433e00700 1 -- 172.21.15.33:0/1639324663 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f441803e570 msgr2=0x7f4418040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:13:10.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:10.482+0000 7f4433e00700 1 --2- 172.21.15.33:0/1639324663 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f441803e570 0x7f4418040a30 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4420006fd0 tx=0x7f4420006e40 comp rx=0 tx=0).stop 2024-05-03T18:13:10.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:10.482+0000 7f4433e00700 1 -- 172.21.15.33:0/1639324663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4434105170 msgr2=0x7f44341a0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:13:10.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:10.482+0000 7f4433e00700 1 --2- 172.21.15.33:0/1639324663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4434105170 0x7f44341a0630 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4424005660 tx=0x7f4424005740 comp rx=0 tx=0).stop 2024-05-03T18:13:10.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:10.482+0000 7f4433e00700 1 -- 172.21.15.33:0/1639324663 shutdown_connections 2024-05-03T18:13:10.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:10.482+0000 7f4433e00700 1 --2- 172.21.15.33:0/1639324663 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f441803e570 0x7f4418040a30 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:13:10.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:10.482+0000 7f4433e00700 1 --2- 172.21.15.33:0/1639324663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4434105170 0x7f44341a0630 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:13:10.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:10.482+0000 7f4433e00700 1 -- 172.21.15.33:0/1639324663 >> 172.21.15.33:0/1639324663 conn(0x7f4434100a20 msgr2=0x7f4434106b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:13:10.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:10.482+0000 7f4433e00700 1 -- 172.21.15.33:0/1639324663 shutdown_connections 2024-05-03T18:13:10.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:10.482+0000 7f4433e00700 1 -- 172.21.15.33:0/1639324663 wait complete. 2024-05-03T18:13:10.484 INFO:teuthology.orchestra.run.smithi033.stderr:adjusted tunables profile to default 2024-05-03T18:13:10.732 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:10 smithi033 ceph-mon[28330]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:10.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:10 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/1639324663' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-03T18:13:11.257 INFO:tasks.cephadm:Adding mon.a on smithi033 2024-05-03T18:13:11.257 INFO:tasks.cephadm:Adding mon.b on smithi099 2024-05-03T18:13:11.257 INFO:tasks.cephadm:Adding mon.c on smithi150 2024-05-03T18:13:11.257 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph orch apply mon '3;smithi033:172.21.15.33=a;smithi099:172.21.15.99=b;smithi150:172.21.15.150=c' 2024-05-03T18:13:11.664 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-03T18:13:11.775 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-03T18:13:11.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:11 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/1639324663' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-03T18:13:11.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:11 smithi033 ceph-mon[28330]: osdmap e4: 0 total, 0 up, 0 in 2024-05-03T18:13:12.982 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:12 smithi033 ceph-mon[28330]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:14.492 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:14.492+0000 7f62f6000700 1 -- 172.21.15.150:0/101286838 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62f0102f00 msgr2=0x7f62f0103320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:13:14.492 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:14.492+0000 7f62f6000700 1 --2- 172.21.15.150:0/101286838 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62f0102f00 0x7f62f0103320 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f62d8009910 tx=0x7f62d8009c20 comp rx=0 tx=0).stop 2024-05-03T18:13:14.492 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:14.492+0000 7f62f6000700 1 -- 172.21.15.150:0/101286838 shutdown_connections 2024-05-03T18:13:14.492 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:14.492+0000 7f62f6000700 1 --2- 172.21.15.150:0/101286838 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62f0102f00 0x7f62f0103320 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:13:14.492 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:14.492+0000 7f62f6000700 1 -- 172.21.15.150:0/101286838 >> 172.21.15.150:0/101286838 conn(0x7f62f00fe4a0 msgr2=0x7f62f01008e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:13:14.492 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:14.492+0000 7f62f6000700 1 -- 172.21.15.150:0/101286838 shutdown_connections 2024-05-03T18:13:14.492 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:14.492+0000 7f62f6000700 1 -- 172.21.15.150:0/101286838 wait complete. 2024-05-03T18:13:14.493 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:14.494+0000 7f62f6000700 1 Processor -- start 2024-05-03T18:13:14.493 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:14.494+0000 7f62f6000700 1 -- start start 2024-05-03T18:13:14.493 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:14.494+0000 7f62f6000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62f0102f00 0x7f62f0198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:13:14.494 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:14.494+0000 7f62f6000700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f62f0108c70 con 0x7f62f0102f00 2024-05-03T18:13:14.494 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:14.495+0000 7f62ef400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62f0102f00 0x7f62f0198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:13:14.494 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:14.495+0000 7f62ef400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62f0102f00 0x7f62f0198440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.150:48516/0 (socket says 172.21.15.150:48516) 2024-05-03T18:13:14.494 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:14.495+0000 7f62ef400700 1 -- 172.21.15.150:0/226561277 learned_addr learned my addr 172.21.15.150:0/226561277 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-03T18:13:14.495 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:14.496+0000 7f62ef400700 1 -- 172.21.15.150:0/226561277 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62f0198980 con 0x7f62f0102f00 2024-05-03T18:13:14.495 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:14.496+0000 7f62ef400700 1 --2- 172.21.15.150:0/226561277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62f0102f00 0x7f62f0198440 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f62d800b5c0 tx=0x7f62d8005810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:13:14.496 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:14.496+0000 7f62ecc00700 1 -- 172.21.15.150:0/226561277 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f62d8027070 con 0x7f62f0102f00 2024-05-03T18:13:14.496 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:14.496+0000 7f62ecc00700 1 -- 172.21.15.150:0/226561277 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f62d800f460 con 0x7f62f0102f00 2024-05-03T18:13:14.496 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:14.496+0000 7f62f6000700 1 -- 172.21.15.150:0/226561277 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62f0198c10 con 0x7f62f0102f00 2024-05-03T18:13:14.496 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:14.497+0000 7f62ecc00700 1 -- 172.21.15.150:0/226561277 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f62d80175d0 con 0x7f62f0102f00 2024-05-03T18:13:14.496 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:14.497+0000 7f62f6000700 1 -- 172.21.15.150:0/226561277 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62f019b900 con 0x7f62f0102f00 2024-05-03T18:13:14.497 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:14.498+0000 7f62f6000700 1 -- 172.21.15.150:0/226561277 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62f01093f0 con 0x7f62f0102f00 2024-05-03T18:13:14.497 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:14.498+0000 7f62ecc00700 1 -- 172.21.15.150:0/226561277 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f62d802a070 con 0x7f62f0102f00 2024-05-03T18:13:14.498 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:14.499+0000 7f62ecc00700 1 --2- 172.21.15.150:0/226561277 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f62d403e510 0x7f62d40409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:13:14.498 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:14.499+0000 7f62ecc00700 1 -- 172.21.15.150:0/226561277 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f62d8053d60 con 0x7f62f0102f00 2024-05-03T18:13:14.498 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:14.500+0000 7f62e7e00700 1 --2- 172.21.15.150:0/226561277 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f62d403e510 0x7f62d40409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:13:14.500 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:14.501+0000 7f62e7e00700 1 --2- 172.21.15.150:0/226561277 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f62d403e510 0x7f62d40409d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f62e0006fd0 tx=0x7f62e0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:13:14.514 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:14.515+0000 7f62ecc00700 1 -- 172.21.15.150:0/226561277 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f62d8013070 con 0x7f62f0102f00 2024-05-03T18:13:14.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:14 smithi033 ceph-mon[28330]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:14.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:14 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:14.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:14 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:14.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:14 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:14.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:14 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:14.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:14 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi150", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:13:14.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:14 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:13:14.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:14 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:13:14.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:14 smithi033 ceph-mon[28330]: Updating smithi150:/etc/ceph/ceph.conf 2024-05-03T18:13:14.765 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:14.766+0000 7f62f6000700 1 -- 172.21.15.150:0/226561277 --> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi033:172.21.15.33=a;smithi099:172.21.15.99=b;smithi150:172.21.15.150=c", "target": ["mon-mgr", ""]}) -- 0x7f62f002e7a0 con 0x7f62d403e510 2024-05-03T18:13:14.783 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:14.783+0000 7f62ecc00700 1 -- 172.21.15.150:0/226561277 <== mgr.14152 v2:172.21.15.33:6800/3642895501 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f62f002e7a0 con 0x7f62d403e510 2024-05-03T18:13:14.783 INFO:teuthology.orchestra.run.smithi150.stdout:Scheduled mon update... 2024-05-03T18:13:14.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:14.786+0000 7f62f6000700 1 -- 172.21.15.150:0/226561277 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f62d403e510 msgr2=0x7f62d40409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:13:14.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:14.786+0000 7f62f6000700 1 --2- 172.21.15.150:0/226561277 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f62d403e510 0x7f62d40409d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f62e0006fd0 tx=0x7f62e0006e40 comp rx=0 tx=0).stop 2024-05-03T18:13:14.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:14.786+0000 7f62f6000700 1 -- 172.21.15.150:0/226561277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62f0102f00 msgr2=0x7f62f0198440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:13:14.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:14.786+0000 7f62f6000700 1 --2- 172.21.15.150:0/226561277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62f0102f00 0x7f62f0198440 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f62d800b5c0 tx=0x7f62d8005810 comp rx=0 tx=0).stop 2024-05-03T18:13:14.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:14.786+0000 7f62f6000700 1 -- 172.21.15.150:0/226561277 shutdown_connections 2024-05-03T18:13:14.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:14.786+0000 7f62f6000700 1 --2- 172.21.15.150:0/226561277 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f62d403e510 0x7f62d40409d0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:13:14.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:14.787+0000 7f62f6000700 1 --2- 172.21.15.150:0/226561277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62f0102f00 0x7f62f0198440 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:13:14.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:14.787+0000 7f62f6000700 1 -- 172.21.15.150:0/226561277 >> 172.21.15.150:0/226561277 conn(0x7f62f00fe4a0 msgr2=0x7f62f00ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:13:14.786 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:14.787+0000 7f62f6000700 1 -- 172.21.15.150:0/226561277 shutdown_connections 2024-05-03T18:13:14.787 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:14.787+0000 7f62f6000700 1 -- 172.21.15.150:0/226561277 wait complete. 2024-05-03T18:13:15.625 DEBUG:teuthology.orchestra.run.smithi099:mon.b> sudo journalctl -f -n 0 -u ceph-5703bba4-0978-11ef-bc97-c7b262605968@mon.b.service 2024-05-03T18:13:15.633 DEBUG:teuthology.orchestra.run.smithi150:mon.c> sudo journalctl -f -n 0 -u ceph-5703bba4-0978-11ef-bc97-c7b262605968@mon.c.service 2024-05-03T18:13:15.638 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-03T18:13:15.638 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-03T18:13:15.732 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:15 smithi033 ceph-mon[28330]: Updating smithi150:/var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-03T18:13:15.732 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:15 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:16.057 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-03T18:13:16.635 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:16 smithi033 ceph-mon[28330]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi033:172.21.15.33=a;smithi099:172.21.15.99=b;smithi150:172.21.15.150=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:13:16.635 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:16 smithi033 ceph-mon[28330]: Saving service mon spec with placement smithi033:172.21.15.33=a;smithi099:172.21.15.99=b;smithi150:172.21.15.150=c;count:3 2024-05-03T18:13:16.635 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:16 smithi033 ceph-mon[28330]: Updating smithi150:/etc/ceph/ceph.client.admin.keyring 2024-05-03T18:13:16.635 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:16 smithi033 ceph-mon[28330]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:16.635 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:16 smithi033 ceph-mon[28330]: Updating smithi150:/var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-03T18:13:16.635 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:16 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:16.635 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:16 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:16.635 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:16 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:16.635 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:16 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-03T18:13:16.635 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:16 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:13:17.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:17 smithi033 ceph-mon[28330]: Deploying daemon mon.c on smithi150 2024-05-03T18:13:18.460 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:18.461+0000 7f3954000700 1 -- 172.21.15.150:0/3620423042 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f394c103110 msgr2=0x7f394c1034f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:13:18.461 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:18.461+0000 7f3954000700 1 --2- 172.21.15.150:0/3620423042 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f394c103110 0x7f394c1034f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3940009910 tx=0x7f3940009c20 comp rx=0 tx=0).stop 2024-05-03T18:13:18.461 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:18.461+0000 7f3954000700 1 -- 172.21.15.150:0/3620423042 shutdown_connections 2024-05-03T18:13:18.461 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:18.461+0000 7f3954000700 1 --2- 172.21.15.150:0/3620423042 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f394c103110 0x7f394c1034f0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:13:18.461 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:18.461+0000 7f3954000700 1 -- 172.21.15.150:0/3620423042 >> 172.21.15.150:0/3620423042 conn(0x7f394c0fe9e0 msgr2=0x7f394c100e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:13:18.461 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:18.461+0000 7f3954000700 1 -- 172.21.15.150:0/3620423042 shutdown_connections 2024-05-03T18:13:18.461 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:18.461+0000 7f3954000700 1 -- 172.21.15.150:0/3620423042 wait complete. 2024-05-03T18:13:18.462 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:18.462+0000 7f3954000700 1 Processor -- start 2024-05-03T18:13:18.462 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:18.462+0000 7f3954000700 1 -- start start 2024-05-03T18:13:18.462 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:18.462+0000 7f3954000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f394c103110 0x7f394c0757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:13:18.462 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:18.463+0000 7f3954000700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f394c10d240 con 0x7f394c103110 2024-05-03T18:13:18.462 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:18.463+0000 7f3951600700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f394c103110 0x7f394c0757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:13:18.462 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:18.463+0000 7f3951600700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f394c103110 0x7f394c0757a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.150:48526/0 (socket says 172.21.15.150:48526) 2024-05-03T18:13:18.463 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:18.463+0000 7f3951600700 1 -- 172.21.15.150:0/236891127 learned_addr learned my addr 172.21.15.150:0/236891127 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-03T18:13:18.463 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:18.464+0000 7f3951600700 1 -- 172.21.15.150:0/236891127 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f394c077690 con 0x7f394c103110 2024-05-03T18:13:18.463 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:18.464+0000 7f3951600700 1 --2- 172.21.15.150:0/236891127 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f394c103110 0x7f394c0757a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3940006010 tx=0x7f3940005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:13:18.464 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:18.465+0000 7f394a000700 1 -- 172.21.15.150:0/236891127 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3940026070 con 0x7f394c103110 2024-05-03T18:13:18.464 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:18.465+0000 7f3954000700 1 -- 172.21.15.150:0/236891127 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f394c075d40 con 0x7f394c103110 2024-05-03T18:13:18.464 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:18.465+0000 7f394a000700 1 -- 172.21.15.150:0/236891127 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f394000b750 con 0x7f394c103110 2024-05-03T18:13:18.464 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:18.465+0000 7f394a000700 1 -- 172.21.15.150:0/236891127 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3940003810 con 0x7f394c103110 2024-05-03T18:13:18.465 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:18.465+0000 7f3954000700 1 -- 172.21.15.150:0/236891127 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f394c076200 con 0x7f394c103110 2024-05-03T18:13:18.465 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:18.466+0000 7f3954000700 1 -- 172.21.15.150:0/236891127 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f394c1045f0 con 0x7f394c103110 2024-05-03T18:13:18.466 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:18.467+0000 7f394a000700 1 -- 172.21.15.150:0/236891127 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f3940029070 con 0x7f394c103110 2024-05-03T18:13:18.466 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:18.467+0000 7f394a000700 1 --2- 172.21.15.150:0/236891127 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f393403e510 0x7f39340409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:13:18.466 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:18.467+0000 7f394a000700 1 -- 172.21.15.150:0/236891127 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f394001e070 con 0x7f394c103110 2024-05-03T18:13:18.467 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:18.468+0000 7f3950c00700 1 --2- 172.21.15.150:0/236891127 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f393403e510 0x7f39340409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:13:18.468 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:18.469+0000 7f3950c00700 1 --2- 172.21.15.150:0/236891127 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f393403e510 0x7f39340409d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3938006fd0 tx=0x7f3938006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:13:18.482 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:18.483+0000 7f394a000700 1 -- 172.21.15.150:0/236891127 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f3940013070 con 0x7f394c103110 2024-05-03T18:13:18.732 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:18 smithi033 ceph-mon[28330]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:18.801 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:18.802+0000 7f3954000700 1 -- 172.21.15.150:0/236891127 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f394c076d70 con 0x7f394c103110 2024-05-03T18:13:18.802 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:18.803+0000 7f394a000700 1 -- 172.21.15.150:0/236891127 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f3940030020 con 0x7f394c103110 2024-05-03T18:13:18.803 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-03T18:13:18.803 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":1,"fsid":"5703bba4-0978-11ef-bc97-c7b262605968","modified":"2024-05-03T18:10:56.604483Z","created":"2024-05-03T18:10:56.604483Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:3300","nonce":0},{"type":"v1","addr":"172.21.15.33:6789","nonce":0}]},"addr":"172.21.15.33:6789/0","public_addr":"172.21.15.33:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-03T18:13:18.806 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:18.806+0000 7f3954000700 1 -- 172.21.15.150:0/236891127 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f393403e510 msgr2=0x7f39340409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:13:18.806 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:18.806+0000 7f3954000700 1 --2- 172.21.15.150:0/236891127 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f393403e510 0x7f39340409d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3938006fd0 tx=0x7f3938006e40 comp rx=0 tx=0).stop 2024-05-03T18:13:18.806 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:18.806+0000 7f3954000700 1 -- 172.21.15.150:0/236891127 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f394c103110 msgr2=0x7f394c0757a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:13:18.806 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:18.806+0000 7f3954000700 1 --2- 172.21.15.150:0/236891127 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f394c103110 0x7f394c0757a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3940006010 tx=0x7f3940005710 comp rx=0 tx=0).stop 2024-05-03T18:13:18.806 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:18.806+0000 7f3954000700 1 -- 172.21.15.150:0/236891127 shutdown_connections 2024-05-03T18:13:18.806 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:18.806+0000 7f3954000700 1 --2- 172.21.15.150:0/236891127 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f393403e510 0x7f39340409d0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:13:18.806 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:18.806+0000 7f3954000700 1 --2- 172.21.15.150:0/236891127 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f394c103110 0x7f394c0757a0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:13:18.806 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:18.806+0000 7f3954000700 1 -- 172.21.15.150:0/236891127 >> 172.21.15.150:0/236891127 conn(0x7f394c0fe9e0 msgr2=0x7f394c104ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:13:18.806 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:18.807+0000 7f3954000700 1 -- 172.21.15.150:0/236891127 shutdown_connections 2024-05-03T18:13:18.806 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:18.807+0000 7f3954000700 1 -- 172.21.15.150:0/236891127 wait complete. 2024-05-03T18:13:18.807 INFO:teuthology.orchestra.run.smithi150.stderr:dumped monmap epoch 1 2024-05-03T18:13:19.732 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:19 smithi033 ceph-mon[28330]: from='client.? 172.21.15.150:0/236891127' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-03T18:13:20.732 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:20 smithi033 ceph-mon[28330]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:20.810 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-03T18:13:20.811 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-03T18:13:21.379 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.c/config 2024-05-03T18:13:22.732 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:22 smithi033 ceph-mon[28330]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:24.058 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:23.984+0000 7fe111c00700 1 -- 172.21.15.150:0/915750382 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe10c105420 msgr2=0x7fe10c105800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:13:24.058 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:23.984+0000 7fe111c00700 1 --2- 172.21.15.150:0/915750382 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe10c105420 0x7fe10c105800 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc009910 tx=0x7fe0fc009c20 comp rx=0 tx=0).stop 2024-05-03T18:13:24.058 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:23.985+0000 7fe111c00700 1 -- 172.21.15.150:0/915750382 shutdown_connections 2024-05-03T18:13:24.058 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:23.985+0000 7fe111c00700 1 --2- 172.21.15.150:0/915750382 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe10c105420 0x7fe10c105800 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:13:24.058 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:23.985+0000 7fe111c00700 1 -- 172.21.15.150:0/915750382 >> 172.21.15.150:0/915750382 conn(0x7fe10c100cd0 msgr2=0x7fe10c1030f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:13:24.058 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:23.985+0000 7fe111c00700 1 -- 172.21.15.150:0/915750382 shutdown_connections 2024-05-03T18:13:24.058 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:23.985+0000 7fe111c00700 1 -- 172.21.15.150:0/915750382 wait complete. 2024-05-03T18:13:24.058 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:23.986+0000 7fe111c00700 1 Processor -- start 2024-05-03T18:13:24.058 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:23.986+0000 7fe111c00700 1 -- start start 2024-05-03T18:13:24.058 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:23.987+0000 7fe111c00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe10c105420 0x7fe10c1a07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:13:24.058 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:23.987+0000 7fe111c00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe10c10f550 con 0x7fe10c105420 2024-05-03T18:13:24.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:23.988+0000 7fe10aa00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe10c105420 0x7fe10c1a07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:13:24.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:23.988+0000 7fe10aa00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe10c105420 0x7fe10c1a07c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.150:43564/0 (socket says 172.21.15.150:43564) 2024-05-03T18:13:24.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:23.988+0000 7fe10aa00700 1 -- 172.21.15.150:0/4048611273 learned_addr learned my addr 172.21.15.150:0/4048611273 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-03T18:13:24.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:23.988+0000 7fe10aa00700 1 -- 172.21.15.150:0/4048611273 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe10c1a0d00 con 0x7fe10c105420 2024-05-03T18:13:24.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:23.989+0000 7fe10aa00700 1 --2- 172.21.15.150:0/4048611273 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe10c105420 0x7fe10c1a07c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc005660 tx=0x7fe0fc005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:13:24.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:23.989+0000 7fe103400700 1 -- 172.21.15.150:0/4048611273 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe0fc026070 con 0x7fe10c105420 2024-05-03T18:13:24.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:23.989+0000 7fe111c00700 1 -- 172.21.15.150:0/4048611273 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe10c1a0ff0 con 0x7fe10c105420 2024-05-03T18:13:24.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:23.990+0000 7fe103400700 1 -- 172.21.15.150:0/4048611273 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe0fc00b6f0 con 0x7fe10c105420 2024-05-03T18:13:24.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:23.990+0000 7fe103400700 1 -- 172.21.15.150:0/4048611273 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe0fc0037b0 con 0x7fe10c105420 2024-05-03T18:13:24.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:23.990+0000 7fe111c00700 1 -- 172.21.15.150:0/4048611273 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe10c19a980 con 0x7fe10c105420 2024-05-03T18:13:24.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:23.992+0000 7fe111c00700 1 -- 172.21.15.150:0/4048611273 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe10c106900 con 0x7fe10c105420 2024-05-03T18:13:24.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:23.992+0000 7fe103400700 1 -- 172.21.15.150:0/4048611273 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fe0fc029020 con 0x7fe10c105420 2024-05-03T18:13:24.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:23.993+0000 7fe103400700 1 --2- 172.21.15.150:0/4048611273 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fe0f003e510 0x7fe0f00409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:13:24.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:23.993+0000 7fe103400700 1 -- 172.21.15.150:0/4048611273 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fe0fc01e070 con 0x7fe10c105420 2024-05-03T18:13:24.059 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:23.993+0000 7fe10a000700 1 --2- 172.21.15.150:0/4048611273 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fe0f003e510 0x7fe0f00409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:13:24.060 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:23.995+0000 7fe10a000700 1 --2- 172.21.15.150:0/4048611273 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fe0f003e510 0x7fe0f00409d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe0f8006fd0 tx=0x7fe0f8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:13:24.060 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:24.009+0000 7fe103400700 1 -- 172.21.15.150:0/4048611273 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fe0fc013070 con 0x7fe10c105420 2024-05-03T18:13:24.248 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:24 smithi150 ceph-mon[32492]: mon.c@-1(synchronizing).mds e1 new map 2024-05-03T18:13:24.248 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:24 smithi150 ceph-mon[32492]: mon.c@-1(synchronizing).mds e1 print_map 2024-05-03T18:13:24.248 INFO:journalctl@ceph.mon.c.smithi150.stdout: e1 2024-05-03T18:13:24.248 INFO:journalctl@ceph.mon.c.smithi150.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-03T18:13:24.248 INFO:journalctl@ceph.mon.c.smithi150.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-03T18:13:24.249 INFO:journalctl@ceph.mon.c.smithi150.stdout: legacy client fscid: -1 2024-05-03T18:13:24.249 INFO:journalctl@ceph.mon.c.smithi150.stdout: 2024-05-03T18:13:24.249 INFO:journalctl@ceph.mon.c.smithi150.stdout: No filesystems configured 2024-05-03T18:13:24.249 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:24 smithi150 ceph-mon[32492]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-05-03T18:13:24.259 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:24.259+0000 7fe103400700 1 -- 172.21.15.150:0/4048611273 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fe0fc00e3e0 con 0x7fe10c105420 2024-05-03T18:13:24.348 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:24.348+0000 7fe111c00700 1 -- 172.21.15.150:0/4048611273 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fe10c19beb0 con 0x7fe10c105420 2024-05-03T18:13:24.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:24 smithi150 ceph-mon[32492]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-05-03T18:13:24.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:24 smithi150 ceph-mon[32492]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-05-03T18:13:24.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:24 smithi150 ceph-mon[32492]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-05-03T18:13:24.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:24 smithi150 ceph-mon[32492]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-05-03T18:13:24.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:24 smithi150 ceph-mon[32492]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-05-03T18:13:24.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:24 smithi150 ceph-mon[32492]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-03T18:13:24.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:24 smithi150 ceph-mon[32492]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-03T18:13:24.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:24 smithi150 ceph-mon[32492]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-03T18:13:24.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:24 smithi150 ceph-mon[32492]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-03T18:13:24.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:24 smithi150 ceph-mon[32492]: Updating smithi033:/etc/ceph/ceph.client.admin.keyring 2024-05-03T18:13:24.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:24 smithi150 ceph-mon[32492]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:24.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:24 smithi150 ceph-mon[32492]: Updating smithi033:/var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-03T18:13:24.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:24 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:24.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:24 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:24.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:24 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:24.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:24 smithi150 ceph-mon[32492]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:24.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:24 smithi150 ceph-mon[32492]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi099", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:13:24.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:24 smithi150 ceph-mon[32492]: Deploying cephadm binary to smithi099 2024-05-03T18:13:24.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:24 smithi150 ceph-mon[32492]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:24.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:24 smithi150 ceph-mon[32492]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:24.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:24 smithi150 ceph-mon[32492]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:24.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:24 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:24.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:24 smithi150 ceph-mon[32492]: Added host smithi099 2024-05-03T18:13:24.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:24 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:13:24.634 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:24 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:24.634 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:24 smithi150 ceph-mon[32492]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:24.634 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:24 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:24.634 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:24 smithi150 ceph-mon[32492]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-03T18:13:24.634 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:24 smithi150 ceph-mon[32492]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:24.634 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:24 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:24.634 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:24 smithi150 ceph-mon[32492]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:24.634 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:24 smithi150 ceph-mon[32492]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi150", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:13:24.634 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:24 smithi150 ceph-mon[32492]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:24.634 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:24 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:24.634 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:24 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:24.635 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:24 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:24.635 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:24 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:24.635 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:24 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi099", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:13:24.635 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:24 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:13:24.635 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:24 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:13:24.635 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:24 smithi150 ceph-mon[32492]: Deploying cephadm binary to smithi150 2024-05-03T18:13:24.636 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:24 smithi150 ceph-mon[32492]: Updating smithi099:/etc/ceph/ceph.conf 2024-05-03T18:13:24.636 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:24 smithi150 ceph-mon[32492]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:24.636 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:24 smithi150 ceph-mon[32492]: Updating smithi099:/var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-03T18:13:24.636 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:24 smithi150 ceph-mon[32492]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:24.636 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:24 smithi150 ceph-mon[32492]: Updating smithi099:/etc/ceph/ceph.client.admin.keyring 2024-05-03T18:13:24.636 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:24 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:24.636 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:24 smithi150 ceph-mon[32492]: Added host smithi150 2024-05-03T18:13:24.636 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:24 smithi150 ceph-mon[32492]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:24.636 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:24 smithi150 ceph-mon[32492]: Updating smithi099:/var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-03T18:13:24.636 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:24 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:24.636 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:24 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:24.636 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:24 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:24.636 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:24 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:13:24.637 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:24 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:24.637 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:24 smithi150 ceph-mon[32492]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:24.637 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:24 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:24.637 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:24 smithi150 ceph-mon[32492]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-03T18:13:24.637 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:24 smithi150 ceph-mon[32492]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:24.637 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:24 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:24.637 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:24 smithi150 ceph-mon[32492]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:24.637 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:24 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/1639324663' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-03T18:13:24.637 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:24 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/1639324663' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-03T18:13:24.637 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:24 smithi150 ceph-mon[32492]: osdmap e4: 0 total, 0 up, 0 in 2024-05-03T18:13:24.637 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:24 smithi150 ceph-mon[32492]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:24.637 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:24 smithi150 ceph-mon[32492]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:24.637 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:24 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:24.637 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:24 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:24.638 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:24 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:24.638 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:24 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:24.638 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:24 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi150", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:13:24.638 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:24 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:13:24.638 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:24 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:13:24.638 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:24 smithi150 ceph-mon[32492]: Updating smithi150:/etc/ceph/ceph.conf 2024-05-03T18:13:24.638 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:24 smithi150 ceph-mon[32492]: Updating smithi150:/var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-03T18:13:24.638 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:24 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:24.638 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:24 smithi150 ceph-mon[32492]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi033:172.21.15.33=a;smithi099:172.21.15.99=b;smithi150:172.21.15.150=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:13:24.638 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:24 smithi150 ceph-mon[32492]: Saving service mon spec with placement smithi033:172.21.15.33=a;smithi099:172.21.15.99=b;smithi150:172.21.15.150=c;count:3 2024-05-03T18:13:24.639 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:24 smithi150 ceph-mon[32492]: Updating smithi150:/etc/ceph/ceph.client.admin.keyring 2024-05-03T18:13:24.639 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:24 smithi150 ceph-mon[32492]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:24.639 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:24 smithi150 ceph-mon[32492]: Updating smithi150:/var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-03T18:13:24.639 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:24 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:24.639 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:24 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:24.639 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:24 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:24.639 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:24 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-03T18:13:24.639 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:24 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:13:24.639 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:24 smithi150 ceph-mon[32492]: Deploying daemon mon.c on smithi150 2024-05-03T18:13:24.639 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:24 smithi150 ceph-mon[32492]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:24.639 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:24 smithi150 ceph-mon[32492]: from='client.? 172.21.15.150:0/236891127' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-03T18:13:24.639 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:24 smithi150 ceph-mon[32492]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:24.639 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:24 smithi150 ceph-mon[32492]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:24.639 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:24 smithi150 ceph-mon[32492]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-05-03T18:13:24.640 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:24 smithi150 bash[32445]: a45cbbe8663d93fc5e0abb73897e4890ed593f639efa684967df7d9567772571 2024-05-03T18:13:24.640 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:24 smithi150 systemd[1]: Started Ceph mon.c for 5703bba4-0978-11ef-bc97-c7b262605968. 2024-05-03T18:13:29.288 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:29.288+0000 7fe103400700 1 -- 172.21.15.150:0/4048611273 <== mon.0 v2:172.21.15.33:3300/0 8 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7fe0fc0047c0 con 0x7fe10c105420 2024-05-03T18:13:29.289 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-03T18:13:29.289 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":2,"fsid":"5703bba4-0978-11ef-bc97-c7b262605968","modified":"2024-05-03T18:13:24.258653Z","created":"2024-05-03T18:10:56.604483Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:3300","nonce":0},{"type":"v1","addr":"172.21.15.33:6789","nonce":0}]},"addr":"172.21.15.33:6789/0","public_addr":"172.21.15.33:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:3300","nonce":0},{"type":"v1","addr":"172.21.15.150:6789","nonce":0}]},"addr":"172.21.15.150:6789/0","public_addr":"172.21.15.150:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-03T18:13:29.292 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:29.292+0000 7fe111c00700 1 -- 172.21.15.150:0/4048611273 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fe0f003e510 msgr2=0x7fe0f00409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:13:29.292 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:29.292+0000 7fe111c00700 1 --2- 172.21.15.150:0/4048611273 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fe0f003e510 0x7fe0f00409d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe0f8006fd0 tx=0x7fe0f8006e40 comp rx=0 tx=0).stop 2024-05-03T18:13:29.292 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:29.292+0000 7fe111c00700 1 -- 172.21.15.150:0/4048611273 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe10c105420 msgr2=0x7fe10c1a07c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:13:29.292 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:29.292+0000 7fe111c00700 1 --2- 172.21.15.150:0/4048611273 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe10c105420 0x7fe10c1a07c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc005660 tx=0x7fe0fc005740 comp rx=0 tx=0).stop 2024-05-03T18:13:29.292 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:29.293+0000 7fe111c00700 1 -- 172.21.15.150:0/4048611273 shutdown_connections 2024-05-03T18:13:29.292 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:29.293+0000 7fe111c00700 1 --2- 172.21.15.150:0/4048611273 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fe0f003e510 0x7fe0f00409d0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:13:29.293 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:29.293+0000 7fe111c00700 1 --2- 172.21.15.150:0/4048611273 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe10c105420 0x7fe10c1a07c0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:13:29.293 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:29.293+0000 7fe111c00700 1 -- 172.21.15.150:0/4048611273 >> 172.21.15.150:0/4048611273 conn(0x7fe10c100cd0 msgr2=0x7fe10c106df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:13:29.293 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:29.293+0000 7fe111c00700 1 -- 172.21.15.150:0/4048611273 shutdown_connections 2024-05-03T18:13:29.293 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:29.293+0000 7fe111c00700 1 -- 172.21.15.150:0/4048611273 wait complete. 2024-05-03T18:13:29.293 INFO:teuthology.orchestra.run.smithi150.stderr:dumped monmap epoch 2 2024-05-03T18:13:29.605 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:29 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T18:13:29.605 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:29 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-03T18:13:29.605 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:29 smithi150 ceph-mon[32492]: mon.a calling monitor election 2024-05-03T18:13:29.605 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:29 smithi150 ceph-mon[32492]: from='client.? 172.21.15.150:0/4048611273' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-03T18:13:29.606 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:29 smithi150 ceph-mon[32492]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:29.606 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:29 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T18:13:29.606 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:29 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T18:13:29.606 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:29 smithi150 ceph-mon[32492]: mon.c calling monitor election 2024-05-03T18:13:29.606 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:29 smithi150 ceph-mon[32492]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:29.606 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:29 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T18:13:29.606 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:29 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T18:13:29.606 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:29 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T18:13:29.606 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:29 smithi150 ceph-mon[32492]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-03T18:13:29.606 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:29 smithi150 ceph-mon[32492]: monmap epoch 2 2024-05-03T18:13:29.606 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:29 smithi150 ceph-mon[32492]: fsid 5703bba4-0978-11ef-bc97-c7b262605968 2024-05-03T18:13:29.606 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:29 smithi150 ceph-mon[32492]: last_changed 2024-05-03T18:13:24.258653+0000 2024-05-03T18:13:29.607 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:29 smithi150 ceph-mon[32492]: created 2024-05-03T18:10:56.604483+0000 2024-05-03T18:13:29.607 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:29 smithi150 ceph-mon[32492]: min_mon_release 19 (squid) 2024-05-03T18:13:29.607 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:29 smithi150 ceph-mon[32492]: election_strategy: 1 2024-05-03T18:13:29.607 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:29 smithi150 ceph-mon[32492]: 0: [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] mon.a 2024-05-03T18:13:29.607 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:29 smithi150 ceph-mon[32492]: 1: [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] mon.c 2024-05-03T18:13:29.607 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:29 smithi150 ceph-mon[32492]: fsmap 2024-05-03T18:13:29.607 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:29 smithi150 ceph-mon[32492]: osdmap e4: 0 total, 0 up, 0 in 2024-05-03T18:13:29.607 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:29 smithi150 ceph-mon[32492]: mgrmap e13: x(active, since 70s) 2024-05-03T18:13:29.607 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:29 smithi150 ceph-mon[32492]: overall HEALTH_OK 2024-05-03T18:13:29.607 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:29 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:29.607 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:29 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:29.608 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:29 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:29.608 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:29 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-03T18:13:29.608 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:29 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:13:29.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:29 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T18:13:29.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:29 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-03T18:13:29.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:29 smithi033 ceph-mon[28330]: mon.a calling monitor election 2024-05-03T18:13:29.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:29 smithi033 ceph-mon[28330]: from='client.? 172.21.15.150:0/4048611273' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-03T18:13:29.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:29 smithi033 ceph-mon[28330]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:29.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:29 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T18:13:29.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:29 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T18:13:29.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:29 smithi033 ceph-mon[28330]: mon.c calling monitor election 2024-05-03T18:13:29.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:29 smithi033 ceph-mon[28330]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:29.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:29 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T18:13:29.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:29 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T18:13:29.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:29 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T18:13:29.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:29 smithi033 ceph-mon[28330]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-03T18:13:29.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:29 smithi033 ceph-mon[28330]: monmap epoch 2 2024-05-03T18:13:29.735 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:29 smithi033 ceph-mon[28330]: fsid 5703bba4-0978-11ef-bc97-c7b262605968 2024-05-03T18:13:29.735 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:29 smithi033 ceph-mon[28330]: last_changed 2024-05-03T18:13:24.258653+0000 2024-05-03T18:13:29.735 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:29 smithi033 ceph-mon[28330]: created 2024-05-03T18:10:56.604483+0000 2024-05-03T18:13:29.735 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:29 smithi033 ceph-mon[28330]: min_mon_release 19 (squid) 2024-05-03T18:13:29.735 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:29 smithi033 ceph-mon[28330]: election_strategy: 1 2024-05-03T18:13:29.735 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:29 smithi033 ceph-mon[28330]: 0: [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] mon.a 2024-05-03T18:13:29.735 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:29 smithi033 ceph-mon[28330]: 1: [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] mon.c 2024-05-03T18:13:29.735 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:29 smithi033 ceph-mon[28330]: fsmap 2024-05-03T18:13:29.735 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:29 smithi033 ceph-mon[28330]: osdmap e4: 0 total, 0 up, 0 in 2024-05-03T18:13:29.735 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:29 smithi033 ceph-mon[28330]: mgrmap e13: x(active, since 70s) 2024-05-03T18:13:29.735 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:29 smithi033 ceph-mon[28330]: overall HEALTH_OK 2024-05-03T18:13:29.735 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:29 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:29.735 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:29 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:29.735 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:29 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:29.735 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:29 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-03T18:13:29.735 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:29 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:13:30.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:30 smithi150 ceph-mon[32492]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:30.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:30 smithi150 ceph-mon[32492]: Deploying daemon mon.b on smithi099 2024-05-03T18:13:30.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:30 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T18:13:30.732 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:30 smithi033 ceph-mon[28330]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:30.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:30 smithi033 ceph-mon[28330]: Deploying daemon mon.b on smithi099 2024-05-03T18:13:30.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:30 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T18:13:31.162 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-03T18:13:31.162 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-03T18:13:31.591 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.c/config 2024-05-03T18:13:31.733 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:13:31 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:13:31.260+0000 7f2c35400700 -1 mgr.server handle_report got status from non-daemon mon.c 2024-05-03T18:13:32.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:32 smithi150 ceph-mon[32492]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:32.732 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:32 smithi033 ceph-mon[28330]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:33.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:33.620+0000 7f3727200700 1 -- 172.21.15.150:0/1758531443 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3728101f40 msgr2=0x7f3728102320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:13:33.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:33.620+0000 7f3727200700 1 --2- 172.21.15.150:0/1758531443 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3728101f40 0x7f3728102320 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3718009910 tx=0x7f3718009c20 comp rx=0 tx=0).stop 2024-05-03T18:13:33.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:33.620+0000 7f3727200700 1 -- 172.21.15.150:0/1758531443 shutdown_connections 2024-05-03T18:13:33.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:33.620+0000 7f3727200700 1 --2- 172.21.15.150:0/1758531443 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3728101f40 0x7f3728102320 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:13:33.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:33.620+0000 7f3727200700 1 -- 172.21.15.150:0/1758531443 >> 172.21.15.150:0/1758531443 conn(0x7f37280fd810 msgr2=0x7f37280ffc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:13:33.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:33.620+0000 7f3727200700 1 -- 172.21.15.150:0/1758531443 shutdown_connections 2024-05-03T18:13:33.620 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:33.620+0000 7f3727200700 1 -- 172.21.15.150:0/1758531443 wait complete. 2024-05-03T18:13:33.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:33.621+0000 7f3727200700 1 Processor -- start 2024-05-03T18:13:33.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:33.622+0000 7f3727200700 1 -- start start 2024-05-03T18:13:33.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:33.622+0000 7f3727200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3728101f40 0x7f37281915d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:13:33.622 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:33.622+0000 7f3727200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3728195530 0x7f3728191b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:13:33.622 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:33.622+0000 7f3727200700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f372810c070 con 0x7f3728195530 2024-05-03T18:13:33.622 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:33.622+0000 7f3727200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f372810bef0 con 0x7f3728101f40 2024-05-03T18:13:33.623 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:33.622+0000 7f3725e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3728101f40 0x7f37281915d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:13:33.623 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:33.623+0000 7f3725e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3728101f40 0x7f37281915d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:49420/0 (socket says 172.21.15.150:49420) 2024-05-03T18:13:33.623 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:33.623+0000 7f3725400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3728195530 0x7f3728191b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:13:33.623 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:33.623+0000 7f3725e00700 1 -- 172.21.15.150:0/2146704541 learned_addr learned my addr 172.21.15.150:0/2146704541 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-03T18:13:33.623 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:33.623+0000 7f3725e00700 1 -- 172.21.15.150:0/2146704541 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3728101f40 msgr2=0x7f37281915d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-05-03T18:13:33.623 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:33.623+0000 7f3725e00700 1 -- 172.21.15.150:0/2146704541 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3728101f40 msgr2=0x7f37281915d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-03T18:13:33.623 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:33.623+0000 7f3725e00700 1 --2- 172.21.15.150:0/2146704541 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3728101f40 0x7f37281915d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-03T18:13:33.624 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:33.623+0000 7f3725e00700 1 --2- 172.21.15.150:0/2146704541 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3728101f40 0x7f37281915d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-03T18:13:33.624 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:33.623+0000 7f3725400700 1 -- 172.21.15.150:0/2146704541 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3728101f40 msgr2=0x7f37281915d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-03T18:13:33.624 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:33.623+0000 7f3725400700 1 --2- 172.21.15.150:0/2146704541 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3728101f40 0x7f37281915d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:13:33.624 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:33.623+0000 7f3725400700 1 -- 172.21.15.150:0/2146704541 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37281921c0 con 0x7f3728195530 2024-05-03T18:13:33.624 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:33.624+0000 7f3725400700 1 --2- 172.21.15.150:0/2146704541 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3728195530 0x7f3728191b10 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f371400b4d0 tx=0x7f371400b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:13:33.624 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:33.624+0000 7f371ea00700 1 -- 172.21.15.150:0/2146704541 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f3714011630 con 0x7f3728195530 2024-05-03T18:13:33.625 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:33.624+0000 7f3727200700 1 -- 172.21.15.150:0/2146704541 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37281a90e0 con 0x7f3728195530 2024-05-03T18:13:33.625 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:33.624+0000 7f3727200700 1 -- 172.21.15.150:0/2146704541 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37281a94e0 con 0x7f3728195530 2024-05-03T18:13:33.625 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:33.624+0000 7f371ea00700 1 -- 172.21.15.150:0/2146704541 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3714011c60 con 0x7f3728195530 2024-05-03T18:13:33.625 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:33.625+0000 7f371ea00700 1 -- 172.21.15.150:0/2146704541 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f371401acb0 con 0x7f3728195530 2024-05-03T18:13:33.625 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:33.625+0000 7f3727200700 1 -- 172.21.15.150:0/2146704541 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3708005330 con 0x7f3728195530 2024-05-03T18:13:33.626 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:33.627+0000 7f371ea00700 1 -- 172.21.15.150:0/2146704541 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f3714018020 con 0x7f3728195530 2024-05-03T18:13:33.626 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:33.627+0000 7f371ea00700 1 --2- 172.21.15.150:0/2146704541 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f370c03e2b0 0x7f370c040770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:13:33.626 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:33.627+0000 7f371ea00700 1 -- 172.21.15.150:0/2146704541 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f3714052290 con 0x7f3728195530 2024-05-03T18:13:33.627 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:33.628+0000 7f3725e00700 1 --2- 172.21.15.150:0/2146704541 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f370c03e2b0 0x7f370c040770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:13:33.628 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:33.629+0000 7f3725e00700 1 --2- 172.21.15.150:0/2146704541 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f370c03e2b0 0x7f370c040770 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f3718006010 tx=0x7f371800b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:13:33.642 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:33.643+0000 7f371ea00700 1 -- 172.21.15.150:0/2146704541 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f371401f020 con 0x7f3728195530 2024-05-03T18:13:33.959 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:33.959+0000 7f3727200700 1 -- 172.21.15.150:0/2146704541 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f37080058f0 con 0x7f3728195530 2024-05-03T18:13:33.960 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:33.961+0000 7f371ea00700 1 -- 172.21.15.150:0/2146704541 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f37140217a0 con 0x7f3728195530 2024-05-03T18:13:33.960 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-03T18:13:33.960 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":2,"fsid":"5703bba4-0978-11ef-bc97-c7b262605968","modified":"2024-05-03T18:13:24.258653Z","created":"2024-05-03T18:10:56.604483Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:3300","nonce":0},{"type":"v1","addr":"172.21.15.33:6789","nonce":0}]},"addr":"172.21.15.33:6789/0","public_addr":"172.21.15.33:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:3300","nonce":0},{"type":"v1","addr":"172.21.15.150:6789","nonce":0}]},"addr":"172.21.15.150:6789/0","public_addr":"172.21.15.150:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-03T18:13:33.963 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:33.963+0000 7f3727200700 1 -- 172.21.15.150:0/2146704541 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f370c03e2b0 msgr2=0x7f370c040770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:13:33.963 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:33.963+0000 7f3727200700 1 --2- 172.21.15.150:0/2146704541 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f370c03e2b0 0x7f370c040770 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f3718006010 tx=0x7f371800b540 comp rx=0 tx=0).stop 2024-05-03T18:13:33.963 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:33.963+0000 7f3727200700 1 -- 172.21.15.150:0/2146704541 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3728195530 msgr2=0x7f3728191b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:13:33.963 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:33.963+0000 7f3727200700 1 --2- 172.21.15.150:0/2146704541 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3728195530 0x7f3728191b10 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f371400b4d0 tx=0x7f371400b890 comp rx=0 tx=0).stop 2024-05-03T18:13:33.963 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:33.963+0000 7f3727200700 1 -- 172.21.15.150:0/2146704541 shutdown_connections 2024-05-03T18:13:33.963 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:33.963+0000 7f3727200700 1 --2- 172.21.15.150:0/2146704541 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f370c03e2b0 0x7f370c040770 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:13:33.963 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:33.963+0000 7f3727200700 1 --2- 172.21.15.150:0/2146704541 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3728101f40 0x7f37281915d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:13:33.964 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:33.963+0000 7f3727200700 1 --2- 172.21.15.150:0/2146704541 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3728195530 0x7f3728191b10 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:13:33.964 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:33.963+0000 7f3727200700 1 -- 172.21.15.150:0/2146704541 >> 172.21.15.150:0/2146704541 conn(0x7f37280fd810 msgr2=0x7f372806ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:13:33.964 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:33.964+0000 7f3727200700 1 -- 172.21.15.150:0/2146704541 shutdown_connections 2024-05-03T18:13:33.964 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:33.964+0000 7f3727200700 1 -- 172.21.15.150:0/2146704541 wait complete. 2024-05-03T18:13:33.964 INFO:teuthology.orchestra.run.smithi150.stderr:dumped monmap epoch 2 2024-05-03T18:13:34.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:34 smithi150 ceph-mon[32492]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:34.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:34 smithi150 ceph-mon[32492]: from='client.? 172.21.15.150:0/2146704541' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-03T18:13:34.679 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 podman[31640]: 2024-05-03 18:13:34.399969216 +0000 UTC m=+0.128785028 container create 7fd86560ad990d432f5e47fe88649ddb1df88194a39d20914ddf475590fe59fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mon-b, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-05-03T18:13:34.679 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 podman[31640]: 2024-05-03 18:13:34.678796894 +0000 UTC m=+0.407612693 container init 7fd86560ad990d432f5e47fe88649ddb1df88194a39d20914ddf475590fe59fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mon-b, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240430, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-05-03T18:13:34.732 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:34 smithi033 ceph-mon[28330]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:34.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:34 smithi033 ceph-mon[28330]: from='client.? 172.21.15.150:0/2146704541' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-03T18:13:35.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 podman[31640]: 2024-05-03 18:13:34.689194124 +0000 UTC m=+0.418009923 container start 7fd86560ad990d432f5e47fe88649ddb1df88194a39d20914ddf475590fe59fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mon-b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240430, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe) 2024-05-03T18:13:35.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: set uid:gid to 167:167 (ceph:ceph) 2024-05-03T18:13:35.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: ceph version 19.0.0-3450-g9a047fe7 (9a047fe7599a549042d06a3fb5b6813a203eabea) squid (dev), process ceph-mon, pid 2 2024-05-03T18:13:35.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: pidfile_write: ignore empty --pid-file 2024-05-03T18:13:35.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: load: jerasure load: lrc 2024-05-03T18:13:35.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: RocksDB version: 7.9.2 2024-05-03T18:13:35.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Git sha 0 2024-05-03T18:13:35.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Compile date 2024-05-01 21:12:03 2024-05-03T18:13:35.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: DB SUMMARY 2024-05-03T18:13:35.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: DB Session ID: 4D6FAMTQ578AP5KLEEAS 2024-05-03T18:13:35.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: CURRENT file: CURRENT 2024-05-03T18:13:35.047 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: IDENTITY file: IDENTITY 2024-05-03T18:13:35.047 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-05-03T18:13:35.047 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-05-03T18:13:35.047 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-05-03T18:13:35.047 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.error_if_exists: 0 2024-05-03T18:13:35.047 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.create_if_missing: 0 2024-05-03T18:13:35.047 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.paranoid_checks: 1 2024-05-03T18:13:35.047 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-03T18:13:35.047 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-03T18:13:35.047 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-03T18:13:35.047 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.env: 0x558a68d5acc0 2024-05-03T18:13:35.047 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.fs: PosixFileSystem 2024-05-03T18:13:35.047 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.info_log: 0x558a9c5b5de0 2024-05-03T18:13:35.047 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.max_file_opening_threads: 16 2024-05-03T18:13:35.048 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.statistics: (nil) 2024-05-03T18:13:35.048 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.use_fsync: 0 2024-05-03T18:13:35.048 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.max_log_file_size: 0 2024-05-03T18:13:35.048 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-03T18:13:35.048 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-03T18:13:35.048 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.keep_log_file_num: 1000 2024-05-03T18:13:35.048 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.recycle_log_file_num: 0 2024-05-03T18:13:35.048 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.allow_fallocate: 1 2024-05-03T18:13:35.049 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.allow_mmap_reads: 0 2024-05-03T18:13:35.049 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.allow_mmap_writes: 0 2024-05-03T18:13:35.049 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.use_direct_reads: 0 2024-05-03T18:13:35.049 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-03T18:13:35.049 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.create_missing_column_families: 0 2024-05-03T18:13:35.049 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.db_log_dir: 2024-05-03T18:13:35.049 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.wal_dir: 2024-05-03T18:13:35.049 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-03T18:13:35.049 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-03T18:13:35.050 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-03T18:13:35.050 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-03T18:13:35.050 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-03T18:13:35.050 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-03T18:13:35.050 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.advise_random_on_open: 1 2024-05-03T18:13:35.050 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.db_write_buffer_size: 0 2024-05-03T18:13:35.050 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.write_buffer_manager: 0x558a9c5870e0 2024-05-03T18:13:35.050 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-03T18:13:35.050 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-03T18:13:35.050 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-03T18:13:35.050 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.rate_limiter: (nil) 2024-05-03T18:13:35.050 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-03T18:13:35.050 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.wal_recovery_mode: 2 2024-05-03T18:13:35.051 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.enable_thread_tracking: 0 2024-05-03T18:13:35.051 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.enable_pipelined_write: 0 2024-05-03T18:13:35.051 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.unordered_write: 0 2024-05-03T18:13:35.051 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-03T18:13:35.051 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-03T18:13:35.051 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-03T18:13:35.051 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-03T18:13:35.051 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.row_cache: None 2024-05-03T18:13:35.051 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.wal_filter: None 2024-05-03T18:13:35.051 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-03T18:13:35.051 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.allow_ingest_behind: 0 2024-05-03T18:13:35.051 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.two_write_queues: 0 2024-05-03T18:13:35.051 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.manual_wal_flush: 0 2024-05-03T18:13:35.052 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.wal_compression: 0 2024-05-03T18:13:35.052 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.atomic_flush: 0 2024-05-03T18:13:35.052 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-03T18:13:35.052 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-03T18:13:35.052 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-03T18:13:35.052 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.log_readahead_size: 0 2024-05-03T18:13:35.052 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-03T18:13:35.052 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.best_efforts_recovery: 0 2024-05-03T18:13:35.052 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-03T18:13:35.052 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-03T18:13:35.052 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.allow_data_in_errors: 0 2024-05-03T18:13:35.052 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.db_host_id: __hostname__ 2024-05-03T18:13:35.052 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-03T18:13:35.053 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.max_background_jobs: 2 2024-05-03T18:13:35.053 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.max_background_compactions: -1 2024-05-03T18:13:35.053 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.max_subcompactions: 1 2024-05-03T18:13:35.053 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-03T18:13:35.053 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-03T18:13:35.053 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-03T18:13:35.053 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.max_total_wal_size: 0 2024-05-03T18:13:35.053 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-03T18:13:35.053 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-03T18:13:35.053 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-03T18:13:35.053 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-03T18:13:35.053 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.max_open_files: -1 2024-05-03T18:13:35.053 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.bytes_per_sync: 0 2024-05-03T18:13:35.053 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-03T18:13:35.054 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-03T18:13:35.054 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.compaction_readahead_size: 0 2024-05-03T18:13:35.054 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.max_background_flushes: -1 2024-05-03T18:13:35.054 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Compression algorithms supported: 2024-05-03T18:13:35.054 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-03T18:13:35.054 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: kZSTD supported: 0 2024-05-03T18:13:35.054 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: kXpressCompression supported: 0 2024-05-03T18:13:35.054 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: kLZ4HCCompression supported: 1 2024-05-03T18:13:35.054 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: kZlibCompression supported: 1 2024-05-03T18:13:35.054 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: kSnappyCompression supported: 1 2024-05-03T18:13:35.054 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: kLZ4Compression supported: 1 2024-05-03T18:13:35.054 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: kBZip2Compression supported: 0 2024-05-03T18:13:35.054 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-03T18:13:35.054 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-03T18:13:35.055 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-05-03T18:13:35.055 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-03T18:13:35.055 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-03T18:13:35.055 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.merge_operator: 2024-05-03T18:13:35.055 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.compaction_filter: None 2024-05-03T18:13:35.055 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.compaction_filter_factory: None 2024-05-03T18:13:35.055 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.sst_partitioner_factory: None 2024-05-03T18:13:35.055 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-03T18:13:35.055 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-03T18:13:35.055 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x558a9c5b5fa0) 2024-05-03T18:13:35.055 INFO:journalctl@ceph.mon.b.smithi099.stdout: cache_index_and_filter_blocks: 1 2024-05-03T18:13:35.055 INFO:journalctl@ceph.mon.b.smithi099.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-03T18:13:35.055 INFO:journalctl@ceph.mon.b.smithi099.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-03T18:13:35.056 INFO:journalctl@ceph.mon.b.smithi099.stdout: pin_top_level_index_and_filter: 1 2024-05-03T18:13:35.056 INFO:journalctl@ceph.mon.b.smithi099.stdout: index_type: 0 2024-05-03T18:13:35.056 INFO:journalctl@ceph.mon.b.smithi099.stdout: data_block_index_type: 0 2024-05-03T18:13:35.056 INFO:journalctl@ceph.mon.b.smithi099.stdout: index_shortening: 1 2024-05-03T18:13:35.056 INFO:journalctl@ceph.mon.b.smithi099.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-03T18:13:35.056 INFO:journalctl@ceph.mon.b.smithi099.stdout: checksum: 4 2024-05-03T18:13:35.056 INFO:journalctl@ceph.mon.b.smithi099.stdout: no_block_cache: 0 2024-05-03T18:13:35.056 INFO:journalctl@ceph.mon.b.smithi099.stdout: block_cache: 0x558a9c5b91f0 2024-05-03T18:13:35.056 INFO:journalctl@ceph.mon.b.smithi099.stdout: block_cache_name: BinnedLRUCache 2024-05-03T18:13:35.056 INFO:journalctl@ceph.mon.b.smithi099.stdout: block_cache_options: 2024-05-03T18:13:35.056 INFO:journalctl@ceph.mon.b.smithi099.stdout: capacity : 536870912 2024-05-03T18:13:35.056 INFO:journalctl@ceph.mon.b.smithi099.stdout: num_shard_bits : 4 2024-05-03T18:13:35.057 INFO:journalctl@ceph.mon.b.smithi099.stdout: strict_capacity_limit : 0 2024-05-03T18:13:35.057 INFO:journalctl@ceph.mon.b.smithi099.stdout: high_pri_pool_ratio: 0.000 2024-05-03T18:13:35.057 INFO:journalctl@ceph.mon.b.smithi099.stdout: block_cache_compressed: (nil) 2024-05-03T18:13:35.057 INFO:journalctl@ceph.mon.b.smithi099.stdout: persistent_cache: (nil) 2024-05-03T18:13:35.057 INFO:journalctl@ceph.mon.b.smithi099.stdout: block_size: 4096 2024-05-03T18:13:35.057 INFO:journalctl@ceph.mon.b.smithi099.stdout: block_size_deviation: 10 2024-05-03T18:13:35.057 INFO:journalctl@ceph.mon.b.smithi099.stdout: block_restart_interval: 16 2024-05-03T18:13:35.057 INFO:journalctl@ceph.mon.b.smithi099.stdout: index_block_restart_interval: 1 2024-05-03T18:13:35.057 INFO:journalctl@ceph.mon.b.smithi099.stdout: metadata_block_size: 4096 2024-05-03T18:13:35.057 INFO:journalctl@ceph.mon.b.smithi099.stdout: partition_filters: 0 2024-05-03T18:13:35.057 INFO:journalctl@ceph.mon.b.smithi099.stdout: use_delta_encoding: 1 2024-05-03T18:13:35.057 INFO:journalctl@ceph.mon.b.smithi099.stdout: filter_policy: bloomfilter 2024-05-03T18:13:35.057 INFO:journalctl@ceph.mon.b.smithi099.stdout: whole_key_filtering: 1 2024-05-03T18:13:35.057 INFO:journalctl@ceph.mon.b.smithi099.stdout: verify_compression: 0 2024-05-03T18:13:35.058 INFO:journalctl@ceph.mon.b.smithi099.stdout: read_amp_bytes_per_bit: 0 2024-05-03T18:13:35.058 INFO:journalctl@ceph.mon.b.smithi099.stdout: format_version: 5 2024-05-03T18:13:35.058 INFO:journalctl@ceph.mon.b.smithi099.stdout: enable_index_compression: 1 2024-05-03T18:13:35.058 INFO:journalctl@ceph.mon.b.smithi099.stdout: block_align: 0 2024-05-03T18:13:35.058 INFO:journalctl@ceph.mon.b.smithi099.stdout: max_auto_readahead_size: 262144 2024-05-03T18:13:35.058 INFO:journalctl@ceph.mon.b.smithi099.stdout: prepopulate_block_cache: 0 2024-05-03T18:13:35.058 INFO:journalctl@ceph.mon.b.smithi099.stdout: initial_auto_readahead_size: 8192 2024-05-03T18:13:35.058 INFO:journalctl@ceph.mon.b.smithi099.stdout: num_file_reads_for_auto_readahead: 2 2024-05-03T18:13:35.058 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.write_buffer_size: 33554432 2024-05-03T18:13:35.058 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.max_write_buffer_number: 2 2024-05-03T18:13:35.058 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.compression: NoCompression 2024-05-03T18:13:35.058 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.bottommost_compression: Disabled 2024-05-03T18:13:35.059 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.prefix_extractor: nullptr 2024-05-03T18:13:35.059 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-03T18:13:35.059 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.num_levels: 7 2024-05-03T18:13:35.059 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-03T18:13:35.060 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-03T18:13:35.061 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-03T18:13:35.061 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-03T18:13:35.061 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-03T18:13:35.061 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-03T18:13:35.061 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-03T18:13:35.061 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-03T18:13:35.061 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-03T18:13:35.061 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-03T18:13:35.061 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-03T18:13:35.061 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-03T18:13:35.061 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-03T18:13:35.061 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.compression_opts.level: 32767 2024-05-03T18:13:35.061 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.compression_opts.strategy: 0 2024-05-03T18:13:35.062 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-03T18:13:35.062 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-03T18:13:35.062 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-03T18:13:35.062 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-03T18:13:35.062 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.compression_opts.enabled: false 2024-05-03T18:13:35.062 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-03T18:13:35.062 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-03T18:13:35.062 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-03T18:13:35.062 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-03T18:13:35.062 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.target_file_size_base: 67108864 2024-05-03T18:13:35.062 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 bash[31640]: 7fd86560ad990d432f5e47fe88649ddb1df88194a39d20914ddf475590fe59fd 2024-05-03T18:13:35.062 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-03T18:13:35.063 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-03T18:13:35.063 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-03T18:13:35.063 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-03T18:13:35.063 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-03T18:13:35.063 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-03T18:13:35.063 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-03T18:13:35.063 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-03T18:13:35.063 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-03T18:13:35.063 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-03T18:13:35.063 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-03T18:13:35.063 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-03T18:13:35.063 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-03T18:13:35.063 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-03T18:13:35.063 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.arena_block_size: 1048576 2024-05-03T18:13:35.063 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-03T18:13:35.064 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-03T18:13:35.064 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.disable_auto_compactions: 0 2024-05-03T18:13:35.064 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-03T18:13:35.064 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-03T18:13:35.064 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-03T18:13:35.064 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-03T18:13:35.064 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-03T18:13:35.064 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-03T18:13:35.064 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-03T18:13:35.064 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-03T18:13:35.064 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-03T18:13:35.064 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-03T18:13:35.064 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-05-03T18:13:35.064 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.inplace_update_support: 0 2024-05-03T18:13:35.064 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-03T18:13:35.065 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-03T18:13:35.065 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-03T18:13:35.065 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-03T18:13:35.065 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.bloom_locality: 0 2024-05-03T18:13:35.065 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.max_successive_merges: 0 2024-05-03T18:13:35.065 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-03T18:13:35.065 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.paranoid_file_checks: 0 2024-05-03T18:13:35.065 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.force_consistency_checks: 1 2024-05-03T18:13:35.065 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.report_bg_io_stats: 0 2024-05-03T18:13:35.065 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.ttl: 2592000 2024-05-03T18:13:35.065 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-03T18:13:35.065 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-03T18:13:35.065 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-03T18:13:35.066 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.enable_blob_files: false 2024-05-03T18:13:35.066 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.min_blob_size: 0 2024-05-03T18:13:35.066 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.blob_file_size: 268435456 2024-05-03T18:13:35.066 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-03T18:13:35.066 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-03T18:13:35.066 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-03T18:13:35.066 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-03T18:13:35.066 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-03T18:13:35.066 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.blob_file_starting_level: 0 2024-05-03T18:13:35.066 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-03T18:13:35.066 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-05-03T18:13:35.066 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-05-03T18:13:35.066 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 928bd539-f89a-46a6-8cce-8c52095caaf9 2024-05-03T18:13:35.067 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714760014749418, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-05-03T18:13:35.067 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-05-03T18:13:35.067 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714760014750306, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714760014, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "928bd539-f89a-46a6-8cce-8c52095caaf9", "db_session_id": "4D6FAMTQ578AP5KLEEAS", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-05-03T18:13:35.067 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714760014750603, "job": 1, "event": "recovery_finished"} 2024-05-03T18:13:35.067 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-05-03T18:13:35.067 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-03T18:13:35.067 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x558a9c6d6000 2024-05-03T18:13:35.067 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: DB pointer 0x558a9c6c2000 2024-05-03T18:13:35.067 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-03T18:13:35.067 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-03T18:13:35.067 INFO:journalctl@ceph.mon.b.smithi099.stdout: ** DB Stats ** 2024-05-03T18:13:35.067 INFO:journalctl@ceph.mon.b.smithi099.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-03T18:13:35.067 INFO:journalctl@ceph.mon.b.smithi099.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-03T18:13:35.067 INFO:journalctl@ceph.mon.b.smithi099.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-03T18:13:35.067 INFO:journalctl@ceph.mon.b.smithi099.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-03T18:13:35.067 INFO:journalctl@ceph.mon.b.smithi099.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-03T18:13:35.068 INFO:journalctl@ceph.mon.b.smithi099.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-03T18:13:35.068 INFO:journalctl@ceph.mon.b.smithi099.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-03T18:13:35.068 INFO:journalctl@ceph.mon.b.smithi099.stdout: 2024-05-03T18:13:35.068 INFO:journalctl@ceph.mon.b.smithi099.stdout: ** Compaction Stats [default] ** 2024-05-03T18:13:35.068 INFO:journalctl@ceph.mon.b.smithi099.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-03T18:13:35.068 INFO:journalctl@ceph.mon.b.smithi099.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-03T18:13:35.068 INFO:journalctl@ceph.mon.b.smithi099.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-03T18:13:35.068 INFO:journalctl@ceph.mon.b.smithi099.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-03T18:13:35.068 INFO:journalctl@ceph.mon.b.smithi099.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-03T18:13:35.068 INFO:journalctl@ceph.mon.b.smithi099.stdout: 2024-05-03T18:13:35.068 INFO:journalctl@ceph.mon.b.smithi099.stdout: ** Compaction Stats [default] ** 2024-05-03T18:13:35.068 INFO:journalctl@ceph.mon.b.smithi099.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-03T18:13:35.068 INFO:journalctl@ceph.mon.b.smithi099.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-03T18:13:35.068 INFO:journalctl@ceph.mon.b.smithi099.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-03T18:13:35.069 INFO:journalctl@ceph.mon.b.smithi099.stdout: 2024-05-03T18:13:35.069 INFO:journalctl@ceph.mon.b.smithi099.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-03T18:13:35.069 INFO:journalctl@ceph.mon.b.smithi099.stdout: 2024-05-03T18:13:35.069 INFO:journalctl@ceph.mon.b.smithi099.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-03T18:13:35.069 INFO:journalctl@ceph.mon.b.smithi099.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-03T18:13:35.069 INFO:journalctl@ceph.mon.b.smithi099.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-03T18:13:35.069 INFO:journalctl@ceph.mon.b.smithi099.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-03T18:13:35.069 INFO:journalctl@ceph.mon.b.smithi099.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-03T18:13:35.069 INFO:journalctl@ceph.mon.b.smithi099.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-03T18:13:35.069 INFO:journalctl@ceph.mon.b.smithi099.stdout: Cumulative compaction: 0.00 GB write, 0.27 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-03T18:13:35.069 INFO:journalctl@ceph.mon.b.smithi099.stdout: Interval compaction: 0.00 GB write, 0.27 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-03T18:13:35.069 INFO:journalctl@ceph.mon.b.smithi099.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-03T18:13:35.069 INFO:journalctl@ceph.mon.b.smithi099.stdout: Block cache BinnedLRUCache@0x558a9c5b91f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.2e-05 secs_since: 0 2024-05-03T18:13:35.069 INFO:journalctl@ceph.mon.b.smithi099.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-05-03T18:13:35.069 INFO:journalctl@ceph.mon.b.smithi099.stdout: 2024-05-03T18:13:35.069 INFO:journalctl@ceph.mon.b.smithi099.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-03T18:13:35.070 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-05-03T18:13:35.070 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: using public_addr v2:172.21.15.99:0/0 -> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] 2024-05-03T18:13:35.070 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: starting mon.b rank -1 at public addrs [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] at bind addrs [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 5703bba4-0978-11ef-bc97-c7b262605968 2024-05-03T18:13:35.070 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: mon.b@-1(???) e0 preinit fsid 5703bba4-0978-11ef-bc97-c7b262605968 2024-05-03T18:13:35.070 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 systemd[1]: Started Ceph mon.b for 5703bba4-0978-11ef-bc97-c7b262605968. 2024-05-03T18:13:35.070 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: mon.b@-1(synchronizing).mds e1 new map 2024-05-03T18:13:35.070 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: mon.b@-1(synchronizing).mds e1 print_map 2024-05-03T18:13:35.070 INFO:journalctl@ceph.mon.b.smithi099.stdout: e1 2024-05-03T18:13:35.070 INFO:journalctl@ceph.mon.b.smithi099.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-03T18:13:35.070 INFO:journalctl@ceph.mon.b.smithi099.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-03T18:13:35.070 INFO:journalctl@ceph.mon.b.smithi099.stdout: legacy client fscid: -1 2024-05-03T18:13:35.070 INFO:journalctl@ceph.mon.b.smithi099.stdout: 2024-05-03T18:13:35.070 INFO:journalctl@ceph.mon.b.smithi099.stdout: No filesystems configured 2024-05-03T18:13:35.071 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-05-03T18:13:35.071 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-05-03T18:13:35.071 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-05-03T18:13:35.071 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-05-03T18:13:35.071 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-05-03T18:13:35.071 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-05-03T18:13:35.071 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-03T18:13:35.071 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-03T18:13:35.071 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-03T18:13:35.071 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-03T18:13:35.071 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: Updating smithi033:/etc/ceph/ceph.client.admin.keyring 2024-05-03T18:13:35.071 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:35.071 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: Updating smithi033:/var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-03T18:13:35.072 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:35.072 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:35.072 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:35.072 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:35.072 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi099", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:13:35.072 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: Deploying cephadm binary to smithi099 2024-05-03T18:13:35.072 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:35.072 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:35.072 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:35.072 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:35.072 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: Added host smithi099 2024-05-03T18:13:35.072 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:13:35.072 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:35.072 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:35.072 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:35.073 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-03T18:13:35.073 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:35.073 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:35.073 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:35.073 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi150", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:13:35.073 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:35.073 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:35.073 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:35.073 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:35.073 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:35.073 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi099", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:13:35.073 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:13:35.074 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:13:35.074 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: Deploying cephadm binary to smithi150 2024-05-03T18:13:35.074 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: Updating smithi099:/etc/ceph/ceph.conf 2024-05-03T18:13:35.074 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:35.074 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: Updating smithi099:/var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-03T18:13:35.074 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:35.074 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: Updating smithi099:/etc/ceph/ceph.client.admin.keyring 2024-05-03T18:13:35.074 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:35.074 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: Added host smithi150 2024-05-03T18:13:35.074 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:35.074 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: Updating smithi099:/var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-03T18:13:35.074 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:35.074 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:35.074 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:35.075 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:13:35.075 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:35.075 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:35.075 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:35.075 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-03T18:13:35.075 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:35.075 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:35.075 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:35.075 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/1639324663' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-03T18:13:35.075 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/1639324663' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-03T18:13:35.075 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: osdmap e4: 0 total, 0 up, 0 in 2024-05-03T18:13:35.075 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:35.075 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:35.075 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:35.076 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:35.076 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:35.076 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:35.076 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi150", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:13:35.076 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:13:35.076 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:13:35.076 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: Updating smithi150:/etc/ceph/ceph.conf 2024-05-03T18:13:35.076 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: Updating smithi150:/var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-03T18:13:35.076 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:35.076 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi033:172.21.15.33=a;smithi099:172.21.15.99=b;smithi150:172.21.15.150=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:13:35.076 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: Saving service mon spec with placement smithi033:172.21.15.33=a;smithi099:172.21.15.99=b;smithi150:172.21.15.150=c;count:3 2024-05-03T18:13:35.076 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: Updating smithi150:/etc/ceph/ceph.client.admin.keyring 2024-05-03T18:13:35.076 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:35.076 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: Updating smithi150:/var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-03T18:13:35.077 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:35.077 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:35.077 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:35.077 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-03T18:13:35.077 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:13:35.077 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: Deploying daemon mon.c on smithi150 2024-05-03T18:13:35.077 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:35.077 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: from='client.? 172.21.15.150:0/236891127' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-03T18:13:35.077 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:35.077 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:35.077 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T18:13:35.077 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-03T18:13:35.077 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: mon.a calling monitor election 2024-05-03T18:13:35.077 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: from='client.? 172.21.15.150:0/4048611273' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-03T18:13:35.078 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:35.078 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T18:13:35.078 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T18:13:35.078 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: mon.c calling monitor election 2024-05-03T18:13:35.078 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:35.078 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T18:13:35.078 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T18:13:35.078 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T18:13:35.078 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-03T18:13:35.078 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: monmap epoch 2 2024-05-03T18:13:35.078 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: fsid 5703bba4-0978-11ef-bc97-c7b262605968 2024-05-03T18:13:35.078 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: last_changed 2024-05-03T18:13:24.258653+0000 2024-05-03T18:13:35.078 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: created 2024-05-03T18:10:56.604483+0000 2024-05-03T18:13:35.078 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: min_mon_release 19 (squid) 2024-05-03T18:13:35.079 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: election_strategy: 1 2024-05-03T18:13:35.079 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: 0: [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] mon.a 2024-05-03T18:13:35.079 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: 1: [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] mon.c 2024-05-03T18:13:35.079 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: fsmap 2024-05-03T18:13:35.079 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: osdmap e4: 0 total, 0 up, 0 in 2024-05-03T18:13:35.079 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: mgrmap e13: x(active, since 70s) 2024-05-03T18:13:35.079 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: overall HEALTH_OK 2024-05-03T18:13:35.079 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:35.079 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:35.079 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:35.079 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-03T18:13:35.079 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:13:35.079 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:35.079 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: Deploying daemon mon.b on smithi099 2024-05-03T18:13:35.079 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T18:13:35.080 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:35.080 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:35.080 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: from='client.? 172.21.15.150:0/2146704541' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-03T18:13:35.080 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:34 smithi099 ceph-mon[31658]: mon.b@-1(synchronizing).paxosservice(auth 1..4) refresh upgraded, format 0 -> 3 2024-05-03T18:13:35.854 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-03T18:13:35.854 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-03T18:13:36.288 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.c/config 2024-05-03T18:13:40.233 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:39 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-03T18:13:40.233 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:39 smithi033 ceph-mon[28330]: mon.a calling monitor election 2024-05-03T18:13:40.233 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:39 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T18:13:40.233 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:39 smithi033 ceph-mon[28330]: mon.c calling monitor election 2024-05-03T18:13:40.233 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:39 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T18:13:40.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:39 smithi033 ceph-mon[28330]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:40.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:39 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T18:13:40.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:39 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T18:13:40.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:39 smithi033 ceph-mon[28330]: mon.b calling monitor election 2024-05-03T18:13:40.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:39 smithi033 ceph-mon[28330]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:40.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:39 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T18:13:40.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:39 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T18:13:40.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:39 smithi033 ceph-mon[28330]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:40.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:39 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T18:13:40.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:39 smithi033 ceph-mon[28330]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-03T18:13:40.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:39 smithi033 ceph-mon[28330]: monmap epoch 3 2024-05-03T18:13:40.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:39 smithi033 ceph-mon[28330]: fsid 5703bba4-0978-11ef-bc97-c7b262605968 2024-05-03T18:13:40.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:39 smithi033 ceph-mon[28330]: last_changed 2024-05-03T18:13:34.805521+0000 2024-05-03T18:13:40.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:39 smithi033 ceph-mon[28330]: created 2024-05-03T18:10:56.604483+0000 2024-05-03T18:13:40.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:39 smithi033 ceph-mon[28330]: min_mon_release 19 (squid) 2024-05-03T18:13:40.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:39 smithi033 ceph-mon[28330]: election_strategy: 1 2024-05-03T18:13:40.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:39 smithi033 ceph-mon[28330]: 0: [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] mon.a 2024-05-03T18:13:40.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:39 smithi033 ceph-mon[28330]: 1: [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] mon.c 2024-05-03T18:13:40.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:39 smithi033 ceph-mon[28330]: 2: [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] mon.b 2024-05-03T18:13:40.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:39 smithi033 ceph-mon[28330]: fsmap 2024-05-03T18:13:40.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:39 smithi033 ceph-mon[28330]: osdmap e4: 0 total, 0 up, 0 in 2024-05-03T18:13:40.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:39 smithi033 ceph-mon[28330]: mgrmap e13: x(active, since 80s) 2024-05-03T18:13:40.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:39 smithi033 ceph-mon[28330]: overall HEALTH_OK 2024-05-03T18:13:40.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:39 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:40.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:39 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:40.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:39 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-03T18:13:40.344 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:39 smithi099 ceph-mon[31658]: mon.a calling monitor election 2024-05-03T18:13:40.344 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:39 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T18:13:40.344 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:39 smithi099 ceph-mon[31658]: mon.c calling monitor election 2024-05-03T18:13:40.344 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:39 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T18:13:40.345 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:39 smithi099 ceph-mon[31658]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:40.345 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:39 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T18:13:40.345 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:39 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T18:13:40.345 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:39 smithi099 ceph-mon[31658]: mon.b calling monitor election 2024-05-03T18:13:40.345 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:39 smithi099 ceph-mon[31658]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:40.345 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:39 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T18:13:40.345 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:39 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T18:13:40.345 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:39 smithi099 ceph-mon[31658]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:40.345 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:39 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T18:13:40.345 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:39 smithi099 ceph-mon[31658]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-03T18:13:40.345 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:39 smithi099 ceph-mon[31658]: monmap epoch 3 2024-05-03T18:13:40.345 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:39 smithi099 ceph-mon[31658]: fsid 5703bba4-0978-11ef-bc97-c7b262605968 2024-05-03T18:13:40.345 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:39 smithi099 ceph-mon[31658]: last_changed 2024-05-03T18:13:34.805521+0000 2024-05-03T18:13:40.346 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:39 smithi099 ceph-mon[31658]: created 2024-05-03T18:10:56.604483+0000 2024-05-03T18:13:40.346 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:39 smithi099 ceph-mon[31658]: min_mon_release 19 (squid) 2024-05-03T18:13:40.346 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:39 smithi099 ceph-mon[31658]: election_strategy: 1 2024-05-03T18:13:40.346 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:39 smithi099 ceph-mon[31658]: 0: [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] mon.a 2024-05-03T18:13:40.346 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:39 smithi099 ceph-mon[31658]: 1: [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] mon.c 2024-05-03T18:13:40.346 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:39 smithi099 ceph-mon[31658]: 2: [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] mon.b 2024-05-03T18:13:40.346 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:39 smithi099 ceph-mon[31658]: fsmap 2024-05-03T18:13:40.346 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:39 smithi099 ceph-mon[31658]: osdmap e4: 0 total, 0 up, 0 in 2024-05-03T18:13:40.346 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:39 smithi099 ceph-mon[31658]: mgrmap e13: x(active, since 80s) 2024-05-03T18:13:40.346 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:39 smithi099 ceph-mon[31658]: overall HEALTH_OK 2024-05-03T18:13:40.346 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:39 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:40.347 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:39 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:40.381 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:39 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-03T18:13:40.381 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:39 smithi150 ceph-mon[32492]: mon.a calling monitor election 2024-05-03T18:13:40.381 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:39 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T18:13:40.381 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:39 smithi150 ceph-mon[32492]: mon.c calling monitor election 2024-05-03T18:13:40.381 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:39 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T18:13:40.381 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:39 smithi150 ceph-mon[32492]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:40.381 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:39 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T18:13:40.381 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:39 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T18:13:40.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:39 smithi150 ceph-mon[32492]: mon.b calling monitor election 2024-05-03T18:13:40.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:39 smithi150 ceph-mon[32492]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:40.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:39 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T18:13:40.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:39 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T18:13:40.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:39 smithi150 ceph-mon[32492]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:40.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:39 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T18:13:40.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:39 smithi150 ceph-mon[32492]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-03T18:13:40.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:39 smithi150 ceph-mon[32492]: monmap epoch 3 2024-05-03T18:13:40.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:39 smithi150 ceph-mon[32492]: fsid 5703bba4-0978-11ef-bc97-c7b262605968 2024-05-03T18:13:40.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:39 smithi150 ceph-mon[32492]: last_changed 2024-05-03T18:13:34.805521+0000 2024-05-03T18:13:40.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:39 smithi150 ceph-mon[32492]: created 2024-05-03T18:10:56.604483+0000 2024-05-03T18:13:40.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:39 smithi150 ceph-mon[32492]: min_mon_release 19 (squid) 2024-05-03T18:13:40.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:39 smithi150 ceph-mon[32492]: election_strategy: 1 2024-05-03T18:13:40.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:39 smithi150 ceph-mon[32492]: 0: [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] mon.a 2024-05-03T18:13:40.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:39 smithi150 ceph-mon[32492]: 1: [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] mon.c 2024-05-03T18:13:40.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:39 smithi150 ceph-mon[32492]: 2: [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] mon.b 2024-05-03T18:13:40.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:39 smithi150 ceph-mon[32492]: fsmap 2024-05-03T18:13:40.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:39 smithi150 ceph-mon[32492]: osdmap e4: 0 total, 0 up, 0 in 2024-05-03T18:13:40.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:39 smithi150 ceph-mon[32492]: mgrmap e13: x(active, since 80s) 2024-05-03T18:13:40.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:39 smithi150 ceph-mon[32492]: overall HEALTH_OK 2024-05-03T18:13:40.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:39 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:40.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:39 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:41.137 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:40 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:41.137 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:40 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:41.138 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:40 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:13:41.138 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:40 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T18:13:41.233 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:40 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:41.233 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:40 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:41.233 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:40 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:13:41.233 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:40 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T18:13:41.310 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:41.311+0000 7f5453e00700 1 -- 172.21.15.150:0/2232196199 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54380035c0 msgr2=0x7f5438005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:13:41.311 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:41.311+0000 7f5453e00700 1 --2- 172.21.15.150:0/2232196199 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54380035c0 0x7f5438005a50 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5454104f10 tx=0x7f544400feb0 comp rx=0 tx=0).stop 2024-05-03T18:13:41.311 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:41.311+0000 7f5453e00700 1 -- 172.21.15.150:0/2232196199 shutdown_connections 2024-05-03T18:13:41.311 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:41.311+0000 7f5453e00700 1 --2- 172.21.15.150:0/2232196199 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54380035c0 0x7f5438005a50 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:13:41.311 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:41.311+0000 7f5453e00700 1 -- 172.21.15.150:0/2232196199 >> 172.21.15.150:0/2232196199 conn(0x7f5454100af0 msgr2=0x7f5454102f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:13:41.311 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:40 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:41.311 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:40 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:41.311 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:40 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:13:41.311 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:40 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T18:13:41.312 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:41.311+0000 7f5453e00700 1 -- 172.21.15.150:0/2232196199 shutdown_connections 2024-05-03T18:13:41.312 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:41.311+0000 7f5453e00700 1 -- 172.21.15.150:0/2232196199 wait complete. 2024-05-03T18:13:41.313 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:41.313+0000 7f5453e00700 1 Processor -- start 2024-05-03T18:13:41.313 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:41.314+0000 7f5453e00700 1 -- start start 2024-05-03T18:13:41.313 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:41.314+0000 7f5453e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54380035c0 0x7f54541a06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:13:41.313 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:41.314+0000 7f5453e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f54541a0c20 0x7f545419a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:13:41.313 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:41.314+0000 7f5453e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f545419ae30 0x7f545419b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:13:41.314 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:41.314+0000 7f5453e00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f545410cde0 con 0x7f54380035c0 2024-05-03T18:13:41.314 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:41.314+0000 7f5453e00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f545410cc60 con 0x7f545419ae30 2024-05-03T18:13:41.314 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:41.314+0000 7f5453e00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f545410cf60 con 0x7f54541a0c20 2024-05-03T18:13:41.314 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:41.315+0000 7f5452a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54380035c0 0x7f54541a06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:13:41.314 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:41.315+0000 7f5452a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54380035c0 0x7f54541a06e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.150:54180/0 (socket says 172.21.15.150:54180) 2024-05-03T18:13:41.315 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:41.315+0000 7f5452a00700 1 -- 172.21.15.150:0/1222882412 learned_addr learned my addr 172.21.15.150:0/1222882412 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-03T18:13:41.315 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:41.315+0000 7f5452000700 1 --2- 172.21.15.150:0/1222882412 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f54541a0c20 0x7f545419a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:13:41.315 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:41.315+0000 7f5452a00700 1 -- 172.21.15.150:0/1222882412 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f54541a0c20 msgr2=0x7f545419a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:13:41.315 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:41.315+0000 7f5452a00700 1 --2- 172.21.15.150:0/1222882412 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f54541a0c20 0x7f545419a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:13:41.315 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:41.315+0000 7f5452a00700 1 -- 172.21.15.150:0/1222882412 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f545419ae30 msgr2=0x7f545419b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:13:41.315 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:41.315+0000 7f5453400700 1 --2- 172.21.15.150:0/1222882412 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f545419ae30 0x7f545419b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:13:41.315 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:41.316+0000 7f5452a00700 1 --2- 172.21.15.150:0/1222882412 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f545419ae30 0x7f545419b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:13:41.316 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:41.316+0000 7f5452a00700 1 -- 172.21.15.150:0/1222882412 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54541a75b0 con 0x7f54380035c0 2024-05-03T18:13:41.316 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:41.316+0000 7f5453400700 1 --2- 172.21.15.150:0/1222882412 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f545419ae30 0x7f545419b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:13:41.316 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:41.316+0000 7f5452a00700 1 --2- 172.21.15.150:0/1222882412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54380035c0 0x7f54541a06e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5444005b40 tx=0x7f54440139e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:13:41.316 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:41.316+0000 7f544b400700 1 -- 172.21.15.150:0/1222882412 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5444028020 con 0x7f54380035c0 2024-05-03T18:13:41.316 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:41.317+0000 7f5453e00700 1 -- 172.21.15.150:0/1222882412 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54541a7780 con 0x7f54380035c0 2024-05-03T18:13:41.317 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:41.317+0000 7f544b400700 1 -- 172.21.15.150:0/1222882412 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5444004d90 con 0x7f54380035c0 2024-05-03T18:13:41.317 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:41.317+0000 7f544b400700 1 -- 172.21.15.150:0/1222882412 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5444024400 con 0x7f54380035c0 2024-05-03T18:13:41.317 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:41.317+0000 7f5453e00700 1 -- 172.21.15.150:0/1222882412 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54541a7d10 con 0x7f54380035c0 2024-05-03T18:13:41.317 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:41.318+0000 7f5453e00700 1 -- 172.21.15.150:0/1222882412 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5454064b00 con 0x7f54380035c0 2024-05-03T18:13:41.318 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:41.319+0000 7f544b400700 1 -- 172.21.15.150:0/1222882412 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f544402b020 con 0x7f54380035c0 2024-05-03T18:13:41.318 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:41.319+0000 7f544b400700 1 --2- 172.21.15.150:0/1222882412 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f543403e9a0 0x7f5434040e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:13:41.318 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:41.319+0000 7f544b400700 1 -- 172.21.15.150:0/1222882412 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f5444054c10 con 0x7f54380035c0 2024-05-03T18:13:41.319 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:41.320+0000 7f5452000700 1 --2- 172.21.15.150:0/1222882412 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f543403e9a0 0x7f5434040e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:13:41.320 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:41.321+0000 7f5452000700 1 --2- 172.21.15.150:0/1222882412 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f543403e9a0 0x7f5434040e60 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5440009ad0 tx=0x7f5440009210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:13:41.334 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:41.335+0000 7f544b400700 1 -- 172.21.15.150:0/1222882412 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f544401ada0 con 0x7f54380035c0 2024-05-03T18:13:41.662 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:41.663+0000 7f5453e00700 1 -- 172.21.15.150:0/1222882412 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f545402e510 con 0x7f54380035c0 2024-05-03T18:13:41.665 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:41.666+0000 7f544b400700 1 -- 172.21.15.150:0/1222882412 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1281 (secure 0 0 0) 0x7f54440192f0 con 0x7f54380035c0 2024-05-03T18:13:41.665 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-05-03T18:13:41.665 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":3,"fsid":"5703bba4-0978-11ef-bc97-c7b262605968","modified":"2024-05-03T18:13:34.805521Z","created":"2024-05-03T18:10:56.604483Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:3300","nonce":0},{"type":"v1","addr":"172.21.15.33:6789","nonce":0}]},"addr":"172.21.15.33:6789/0","public_addr":"172.21.15.33:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:3300","nonce":0},{"type":"v1","addr":"172.21.15.150:6789","nonce":0}]},"addr":"172.21.15.150:6789/0","public_addr":"172.21.15.150:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:3300","nonce":0},{"type":"v1","addr":"172.21.15.99:6789","nonce":0}]},"addr":"172.21.15.99:6789/0","public_addr":"172.21.15.99:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-05-03T18:13:41.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:41.669+0000 7f5453e00700 1 -- 172.21.15.150:0/1222882412 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f543403e9a0 msgr2=0x7f5434040e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:13:41.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:41.669+0000 7f5453e00700 1 --2- 172.21.15.150:0/1222882412 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f543403e9a0 0x7f5434040e60 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5440009ad0 tx=0x7f5440009210 comp rx=0 tx=0).stop 2024-05-03T18:13:41.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:41.669+0000 7f5453e00700 1 -- 172.21.15.150:0/1222882412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54380035c0 msgr2=0x7f54541a06e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:13:41.669 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:41.669+0000 7f5453e00700 1 --2- 172.21.15.150:0/1222882412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54380035c0 0x7f54541a06e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5444005b40 tx=0x7f54440139e0 comp rx=0 tx=0).stop 2024-05-03T18:13:41.669 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:41.669+0000 7f5453e00700 1 -- 172.21.15.150:0/1222882412 shutdown_connections 2024-05-03T18:13:41.669 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:41.669+0000 7f5453e00700 1 --2- 172.21.15.150:0/1222882412 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f543403e9a0 0x7f5434040e60 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:13:41.669 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:41.669+0000 7f5453e00700 1 --2- 172.21.15.150:0/1222882412 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f545419ae30 0x7f545419b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:13:41.669 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:41.669+0000 7f5453e00700 1 --2- 172.21.15.150:0/1222882412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54380035c0 0x7f54541a06e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:13:41.669 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:41.669+0000 7f5453e00700 1 --2- 172.21.15.150:0/1222882412 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f54541a0c20 0x7f545419a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:13:41.669 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:41.669+0000 7f5453e00700 1 -- 172.21.15.150:0/1222882412 >> 172.21.15.150:0/1222882412 conn(0x7f5454100af0 msgr2=0x7f54541020d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:13:41.669 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:41.669+0000 7f5453e00700 1 -- 172.21.15.150:0/1222882412 shutdown_connections 2024-05-03T18:13:41.669 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:41.669+0000 7f5453e00700 1 -- 172.21.15.150:0/1222882412 wait complete. 2024-05-03T18:13:41.670 INFO:teuthology.orchestra.run.smithi150.stderr:dumped monmap epoch 3 2024-05-03T18:13:42.164 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:41 smithi150 ceph-mon[32492]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:42.164 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:41 smithi150 ceph-mon[32492]: from='client.? 172.21.15.150:0/1222882412' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-03T18:13:42.233 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:41 smithi033 ceph-mon[28330]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:42.233 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:41 smithi033 ceph-mon[28330]: from='client.? 172.21.15.150:0/1222882412' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-03T18:13:42.233 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:13:41 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:13:41.807+0000 7f2c35400700 -1 mgr.server handle_report got status from non-daemon mon.b 2024-05-03T18:13:42.294 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:41 smithi099 ceph-mon[31658]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:42.294 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:41 smithi099 ceph-mon[31658]: from='client.? 172.21.15.150:0/1222882412' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-03T18:13:42.900 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-05-03T18:13:42.900 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph config generate-minimal-conf 2024-05-03T18:13:43.131 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:43 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:43.131 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:43 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:43.131 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:43 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:43.131 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:43 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:43.131 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:43 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:13:43.131 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:43 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:13:43.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:43 smithi150 ceph-mon[32492]: Updating smithi033:/etc/ceph/ceph.conf 2024-05-03T18:13:43.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:43 smithi150 ceph-mon[32492]: Updating smithi099:/etc/ceph/ceph.conf 2024-05-03T18:13:43.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:43 smithi150 ceph-mon[32492]: Updating smithi150:/etc/ceph/ceph.conf 2024-05-03T18:13:43.233 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:43 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:43.233 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:43 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:43.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:43 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:43.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:43 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:43.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:43 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:13:43.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:43 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:13:43.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:43 smithi033 ceph-mon[28330]: Updating smithi033:/etc/ceph/ceph.conf 2024-05-03T18:13:43.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:43 smithi033 ceph-mon[28330]: Updating smithi099:/etc/ceph/ceph.conf 2024-05-03T18:13:43.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:43 smithi033 ceph-mon[28330]: Updating smithi150:/etc/ceph/ceph.conf 2024-05-03T18:13:43.346 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:13:43.544 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:43 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:43.544 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:43 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:43.544 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:43 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:43.544 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:43 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:43.544 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:43 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:13:43.544 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:43 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:13:43.544 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:43 smithi099 ceph-mon[31658]: Updating smithi033:/etc/ceph/ceph.conf 2024-05-03T18:13:43.544 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:43 smithi099 ceph-mon[31658]: Updating smithi099:/etc/ceph/ceph.conf 2024-05-03T18:13:43.544 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:43 smithi099 ceph-mon[31658]: Updating smithi150:/etc/ceph/ceph.conf 2024-05-03T18:13:44.233 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:44 smithi033 ceph-mon[28330]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:44.233 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:44 smithi033 ceph-mon[28330]: Updating smithi150:/var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-03T18:13:44.233 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:44 smithi033 ceph-mon[28330]: Updating smithi033:/var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-03T18:13:44.233 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:44 smithi033 ceph-mon[28330]: Updating smithi099:/var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-03T18:13:44.233 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:44 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:44.233 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:44 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:44.233 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:44 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:44.233 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:44 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:44.381 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:44 smithi150 ceph-mon[32492]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:44.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:44 smithi150 ceph-mon[32492]: Updating smithi150:/var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-03T18:13:44.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:44 smithi150 ceph-mon[32492]: Updating smithi033:/var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-03T18:13:44.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:44 smithi150 ceph-mon[32492]: Updating smithi099:/var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-03T18:13:44.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:44 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:44.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:44 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:44.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:44 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:44.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:44 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:44.544 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:44 smithi099 ceph-mon[31658]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:44.544 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:44 smithi099 ceph-mon[31658]: Updating smithi150:/var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-03T18:13:44.544 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:44 smithi099 ceph-mon[31658]: Updating smithi033:/var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-03T18:13:44.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:44 smithi099 ceph-mon[31658]: Updating smithi099:/var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-03T18:13:44.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:44 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:44.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:44 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:44.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:44 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:44.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:44 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:45.128 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:45 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:45.128 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:45 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:45.129 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:45 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:45.129 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:45 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:45.129 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:45 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:45.129 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:45 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:45.129 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:45 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:45.129 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:45 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:45.129 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:45 smithi033 ceph-mon[28330]: Reconfiguring mon.a (unknown last config time)... 2024-05-03T18:13:45.129 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:45 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-03T18:13:45.129 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:45 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-03T18:13:45.129 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:45 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:13:45.129 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:45 smithi033 ceph-mon[28330]: Reconfiguring daemon mon.a on smithi033 2024-05-03T18:13:45.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:45.293+0000 7f20c7200700 1 -- 172.21.15.33:0/3399815477 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20c80758d0 msgr2=0x7f20c8075cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:13:45.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:45.293+0000 7f20c7200700 1 --2- 172.21.15.33:0/3399815477 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20c80758d0 0x7f20c8075cb0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f20b8009910 tx=0x7f20b8009c20 comp rx=0 tx=0).stop 2024-05-03T18:13:45.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:45.293+0000 7f20c7200700 1 -- 172.21.15.33:0/3399815477 shutdown_connections 2024-05-03T18:13:45.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:45.293+0000 7f20c7200700 1 --2- 172.21.15.33:0/3399815477 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20c80758d0 0x7f20c8075cb0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:13:45.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:45.293+0000 7f20c7200700 1 -- 172.21.15.33:0/3399815477 >> 172.21.15.33:0/3399815477 conn(0x7f20c80fe800 msgr2=0x7f20c8100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:13:45.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:45.293+0000 7f20c7200700 1 -- 172.21.15.33:0/3399815477 shutdown_connections 2024-05-03T18:13:45.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:45.293+0000 7f20c7200700 1 -- 172.21.15.33:0/3399815477 wait complete. 2024-05-03T18:13:45.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:45.295+0000 7f20c7200700 1 Processor -- start 2024-05-03T18:13:45.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:45.295+0000 7f20c7200700 1 -- start start 2024-05-03T18:13:45.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:45.296+0000 7f20c7200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20c80758d0 0x7f20c81a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:13:45.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:45.296+0000 7f20c7200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20c81a0d20 0x7f20c819a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:13:45.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:45.296+0000 7f20c7200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20c819aee0 0x7f20c819b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:13:45.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:45.296+0000 7f20c7200700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f20c810f2b0 con 0x7f20c81a0d20 2024-05-03T18:13:45.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:45.296+0000 7f20c7200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f20c810f130 con 0x7f20c819aee0 2024-05-03T18:13:45.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:45.296+0000 7f20c7200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f20c810f430 con 0x7f20c80758d0 2024-05-03T18:13:45.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:45.296+0000 7f20c5400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20c81a0d20 0x7f20c819a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:13:45.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:45.296+0000 7f20c5400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20c81a0d20 0x7f20c819a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54568/0 (socket says 172.21.15.33:54568) 2024-05-03T18:13:45.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:45.296+0000 7f20c5400700 1 -- 172.21.15.33:0/4036891925 learned_addr learned my addr 172.21.15.33:0/4036891925 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:13:45.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:45.296+0000 7f20c5e00700 1 --2- 172.21.15.33:0/4036891925 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20c80758d0 0x7f20c81a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:13:45.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:45.297+0000 7f20c6800700 1 --2- 172.21.15.33:0/4036891925 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20c819aee0 0x7f20c819b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:13:45.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:45.297+0000 7f20c5400700 1 -- 172.21.15.33:0/4036891925 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20c80758d0 msgr2=0x7f20c81a07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:13:45.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:45.297+0000 7f20c5400700 1 --2- 172.21.15.33:0/4036891925 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20c80758d0 0x7f20c81a07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:13:45.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:45.297+0000 7f20c5400700 1 -- 172.21.15.33:0/4036891925 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20c819aee0 msgr2=0x7f20c819b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:13:45.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:45.297+0000 7f20c5400700 1 --2- 172.21.15.33:0/4036891925 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20c819aee0 0x7f20c819b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:13:45.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:45.297+0000 7f20c5400700 1 -- 172.21.15.33:0/4036891925 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20c81a76b0 con 0x7f20c81a0d20 2024-05-03T18:13:45.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:45.297+0000 7f20c5e00700 1 --2- 172.21.15.33:0/4036891925 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20c80758d0 0x7f20c81a07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:13:45.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:45.297+0000 7f20c5400700 1 --2- 172.21.15.33:0/4036891925 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20c81a0d20 0x7f20c819a9a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f20b400b780 tx=0x7f20b400bb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:13:45.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:45.297+0000 7f20bea00700 1 -- 172.21.15.33:0/4036891925 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b4004230 con 0x7f20c81a0d20 2024-05-03T18:13:45.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:45.298+0000 7f20bea00700 1 -- 172.21.15.33:0/4036891925 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f20b40043d0 con 0x7f20c81a0d20 2024-05-03T18:13:45.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:45.298+0000 7f20bea00700 1 -- 172.21.15.33:0/4036891925 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b4011ac0 con 0x7f20c81a0d20 2024-05-03T18:13:45.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:45.298+0000 7f20c7200700 1 -- 172.21.15.33:0/4036891925 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20c81a78e0 con 0x7f20c81a0d20 2024-05-03T18:13:45.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:45.298+0000 7f20c7200700 1 -- 172.21.15.33:0/4036891925 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20c81a7dd0 con 0x7f20c81a0d20 2024-05-03T18:13:45.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:45.299+0000 7f20c7200700 1 -- 172.21.15.33:0/4036891925 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20c8064b00 con 0x7f20c81a0d20 2024-05-03T18:13:45.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:45.300+0000 7f20bea00700 1 -- 172.21.15.33:0/4036891925 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f20b40036a0 con 0x7f20c81a0d20 2024-05-03T18:13:45.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:45.300+0000 7f20bea00700 1 --2- 172.21.15.33:0/4036891925 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f20ac03e9a0 0x7f20ac040e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:13:45.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:45.300+0000 7f20bea00700 1 -- 172.21.15.33:0/4036891925 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f20b4052480 con 0x7f20c81a0d20 2024-05-03T18:13:45.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:45.301+0000 7f20c5e00700 1 --2- 172.21.15.33:0/4036891925 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f20ac03e9a0 0x7f20ac040e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:13:45.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:45.301+0000 7f20c5e00700 1 --2- 172.21.15.33:0/4036891925 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f20ac03e9a0 0x7f20ac040e60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f20b800b5c0 tx=0x7f20b801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:13:45.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:45.306+0000 7f20bea00700 1 -- 172.21.15.33:0/4036891925 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f20b4020740 con 0x7f20c81a0d20 2024-05-03T18:13:45.381 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:45 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:45.381 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:45 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:45.381 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:45 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:45.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:45 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:45.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:45 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:45.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:45 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:45.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:45 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:45.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:45 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:45.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:45 smithi150 ceph-mon[32492]: Reconfiguring mon.a (unknown last config time)... 2024-05-03T18:13:45.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:45 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-03T18:13:45.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:45 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-03T18:13:45.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:45 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:13:45.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:45 smithi150 ceph-mon[32492]: Reconfiguring daemon mon.a on smithi033 2024-05-03T18:13:45.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:45.541+0000 7f20c7200700 1 -- 172.21.15.33:0/4036891925 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f20c802e6d0 con 0x7f20c81a0d20 2024-05-03T18:13:45.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:45.542+0000 7f20bea00700 1 -- 172.21.15.33:0/4036891925 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+273 (secure 0 0 0) 0x7f20b4003990 con 0x7f20c81a0d20 2024-05-03T18:13:45.541 INFO:teuthology.orchestra.run.smithi033.stdout:# minimal ceph.conf for 5703bba4-0978-11ef-bc97-c7b262605968 2024-05-03T18:13:45.541 INFO:teuthology.orchestra.run.smithi033.stdout:[global] 2024-05-03T18:13:45.541 INFO:teuthology.orchestra.run.smithi033.stdout: fsid = 5703bba4-0978-11ef-bc97-c7b262605968 2024-05-03T18:13:45.542 INFO:teuthology.orchestra.run.smithi033.stdout: mon_host = [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] 2024-05-03T18:13:45.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:45.544+0000 7f20c7200700 1 -- 172.21.15.33:0/4036891925 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f20ac03e9a0 msgr2=0x7f20ac040e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:13:45.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:45.544+0000 7f20c7200700 1 --2- 172.21.15.33:0/4036891925 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f20ac03e9a0 0x7f20ac040e60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f20b800b5c0 tx=0x7f20b801a040 comp rx=0 tx=0).stop 2024-05-03T18:13:45.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:45.544+0000 7f20c7200700 1 -- 172.21.15.33:0/4036891925 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20c81a0d20 msgr2=0x7f20c819a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:13:45.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:45.544+0000 7f20c7200700 1 --2- 172.21.15.33:0/4036891925 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20c81a0d20 0x7f20c819a9a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f20b400b780 tx=0x7f20b400bb40 comp rx=0 tx=0).stop 2024-05-03T18:13:45.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:45.544+0000 7f20c7200700 1 -- 172.21.15.33:0/4036891925 shutdown_connections 2024-05-03T18:13:45.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:45.544+0000 7f20c7200700 1 --2- 172.21.15.33:0/4036891925 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f20ac03e9a0 0x7f20ac040e60 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:13:45.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:45.544+0000 7f20c7200700 1 --2- 172.21.15.33:0/4036891925 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20c819aee0 0x7f20c819b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:13:45.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:45.544+0000 7f20c7200700 1 --2- 172.21.15.33:0/4036891925 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20c80758d0 0x7f20c81a07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:13:45.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:45.544+0000 7f20c7200700 1 --2- 172.21.15.33:0/4036891925 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20c81a0d20 0x7f20c819a9a0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:13:45.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:45.544+0000 7f20c7200700 1 -- 172.21.15.33:0/4036891925 >> 172.21.15.33:0/4036891925 conn(0x7f20c80fe800 msgr2=0x7f20c8108fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:13:45.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:45.545+0000 7f20c7200700 1 -- 172.21.15.33:0/4036891925 shutdown_connections 2024-05-03T18:13:45.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:13:45.545+0000 7f20c7200700 1 -- 172.21.15.33:0/4036891925 wait complete. 2024-05-03T18:13:45.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:45 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:45.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:45 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:45.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:45 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:45.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:45 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:45.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:45 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:45.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:45 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:45.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:45 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:45.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:45 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:45.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:45 smithi099 ceph-mon[31658]: Reconfiguring mon.a (unknown last config time)... 2024-05-03T18:13:45.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:45 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-03T18:13:45.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:45 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-03T18:13:45.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:45 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:13:45.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:45 smithi099 ceph-mon[31658]: Reconfiguring daemon mon.a on smithi033 2024-05-03T18:13:46.381 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:46 smithi150 ceph-mon[32492]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:46.381 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:46 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/4036891925' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:13:46.482 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:46 smithi033 ceph-mon[28330]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:46.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:46 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/4036891925' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:13:46.544 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:46 smithi099 ceph-mon[31658]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:46.544 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:46 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/4036891925' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:13:46.999 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-05-03T18:13:46.999 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-03T18:13:46.999 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/etc/ceph/ceph.conf 2024-05-03T18:13:47.091 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-03T18:13:47.091 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-03T18:13:47.229 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-05-03T18:13:47.229 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/etc/ceph/ceph.conf 2024-05-03T18:13:47.324 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-05-03T18:13:47.324 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-03T18:13:47.447 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-05-03T18:13:47.448 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/ceph/ceph.conf 2024-05-03T18:13:47.541 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-05-03T18:13:47.541 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-03T18:13:47.670 INFO:tasks.cephadm:Adding mgr.x on smithi033 2024-05-03T18:13:47.671 INFO:tasks.cephadm:Adding mgr.y on smithi099 2024-05-03T18:13:47.671 INFO:tasks.cephadm:Adding mgr.z on smithi150 2024-05-03T18:13:47.671 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph orch apply mgr '3;smithi033=x;smithi099=y;smithi150=z' 2024-05-03T18:13:48.144 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.c/config 2024-05-03T18:13:48.544 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:48 smithi099 ceph-mon[31658]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:48.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:48 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:48.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:48 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:48.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:48 smithi099 ceph-mon[31658]: Reconfiguring mon.b (monmap changed)... 2024-05-03T18:13:48.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:48 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-03T18:13:48.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:48 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-03T18:13:48.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:48 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:13:48.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:48 smithi099 ceph-mon[31658]: Reconfiguring daemon mon.b on smithi099 2024-05-03T18:13:48.561 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:48 smithi150 ceph-mon[32492]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:48.561 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:48 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:48.561 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:48 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:48.561 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:48 smithi150 ceph-mon[32492]: Reconfiguring mon.b (monmap changed)... 2024-05-03T18:13:48.561 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:48 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-03T18:13:48.561 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:48 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-03T18:13:48.561 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:48 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:13:48.561 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:48 smithi150 ceph-mon[32492]: Reconfiguring daemon mon.b on smithi099 2024-05-03T18:13:48.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:48 smithi033 ceph-mon[28330]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:48.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:48 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:48.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:48 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:48.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:48 smithi033 ceph-mon[28330]: Reconfiguring mon.b (monmap changed)... 2024-05-03T18:13:48.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:48 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-03T18:13:48.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:48 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-03T18:13:48.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:48 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:13:48.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:48 smithi033 ceph-mon[28330]: Reconfiguring daemon mon.b on smithi099 2024-05-03T18:13:50.143 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:50.144+0000 7f7e89200700 1 -- 172.21.15.150:0/461854502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e84103000 msgr2=0x7f7e84103420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:13:50.143 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:50.144+0000 7f7e89200700 1 --2- 172.21.15.150:0/461854502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e84103000 0x7f7e84103420 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7e74009910 tx=0x7f7e74009c20 comp rx=0 tx=0).stop 2024-05-03T18:13:50.143 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:50.144+0000 7f7e89200700 1 -- 172.21.15.150:0/461854502 shutdown_connections 2024-05-03T18:13:50.143 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:50.144+0000 7f7e89200700 1 --2- 172.21.15.150:0/461854502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e84103000 0x7f7e84103420 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:13:50.143 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:50.144+0000 7f7e89200700 1 -- 172.21.15.150:0/461854502 >> 172.21.15.150:0/461854502 conn(0x7f7e840fe5a0 msgr2=0x7f7e841009e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:13:50.143 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:50.144+0000 7f7e89200700 1 -- 172.21.15.150:0/461854502 shutdown_connections 2024-05-03T18:13:50.143 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:50.144+0000 7f7e89200700 1 -- 172.21.15.150:0/461854502 wait complete. 2024-05-03T18:13:50.145 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:50.145+0000 7f7e89200700 1 Processor -- start 2024-05-03T18:13:50.145 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:50.145+0000 7f7e89200700 1 -- start start 2024-05-03T18:13:50.145 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:50.146+0000 7f7e89200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e84103000 0x7f7e8419a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:13:50.145 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:50.146+0000 7f7e89200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e8419ae10 0x7f7e8419fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:13:50.145 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:50.146+0000 7f7e89200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7e8419b290 0x7f7e841a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:13:50.145 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:50.146+0000 7f7e89200700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7e84108da0 con 0x7f7e84103000 2024-05-03T18:13:50.145 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:50.146+0000 7f7e89200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7e84108c20 con 0x7f7e8419ae10 2024-05-03T18:13:50.145 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:50.146+0000 7f7e89200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f7e84108f20 con 0x7f7e8419b290 2024-05-03T18:13:50.145 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:50.146+0000 7f7e81e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e84103000 0x7f7e8419a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:13:50.146 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:50.146+0000 7f7e81e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e84103000 0x7f7e8419a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.150:42616/0 (socket says 172.21.15.150:42616) 2024-05-03T18:13:50.146 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:50.146+0000 7f7e81e00700 1 -- 172.21.15.150:0/2127058328 learned_addr learned my addr 172.21.15.150:0/2127058328 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-03T18:13:50.146 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:50.146+0000 7f7e81400700 1 --2- 172.21.15.150:0/2127058328 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e8419ae10 0x7f7e8419fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:13:50.146 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:50.147+0000 7f7e82800700 1 --2- 172.21.15.150:0/2127058328 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7e8419b290 0x7f7e841a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:13:50.147 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:50.147+0000 7f7e81e00700 1 -- 172.21.15.150:0/2127058328 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7e8419b290 msgr2=0x7f7e841a2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:13:50.147 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:50.147+0000 7f7e81e00700 1 --2- 172.21.15.150:0/2127058328 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7e8419b290 0x7f7e841a2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:13:50.147 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:50.147+0000 7f7e81e00700 1 -- 172.21.15.150:0/2127058328 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e8419ae10 msgr2=0x7f7e8419fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:13:50.147 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:50.147+0000 7f7e81e00700 1 --2- 172.21.15.150:0/2127058328 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e8419ae10 0x7f7e8419fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:13:50.147 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:50.147+0000 7f7e81e00700 1 -- 172.21.15.150:0/2127058328 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e841a2ce0 con 0x7f7e84103000 2024-05-03T18:13:50.147 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:50.148+0000 7f7e81e00700 1 --2- 172.21.15.150:0/2127058328 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e84103000 0x7f7e8419a8d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7e74005000 tx=0x7f7e74005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:13:50.147 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:50.148+0000 7f7e7aa00700 1 -- 172.21.15.150:0/2127058328 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e74027070 con 0x7f7e84103000 2024-05-03T18:13:50.147 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:50.148+0000 7f7e7aa00700 1 -- 172.21.15.150:0/2127058328 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7e7400bdf0 con 0x7f7e84103000 2024-05-03T18:13:50.147 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:50.148+0000 7f7e89200700 1 -- 172.21.15.150:0/2127058328 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e841a2f70 con 0x7f7e84103000 2024-05-03T18:13:50.148 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:50.148+0000 7f7e7aa00700 1 -- 172.21.15.150:0/2127058328 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e74022cf0 con 0x7f7e84103000 2024-05-03T18:13:50.148 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:50.148+0000 7f7e89200700 1 -- 172.21.15.150:0/2127058328 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e841a33e0 con 0x7f7e84103000 2024-05-03T18:13:50.148 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:50.149+0000 7f7e89200700 1 -- 172.21.15.150:0/2127058328 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e84064b00 con 0x7f7e84103000 2024-05-03T18:13:50.148 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:50.150+0000 7f7e7aa00700 1 -- 172.21.15.150:0/2127058328 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f7e7402a070 con 0x7f7e84103000 2024-05-03T18:13:50.149 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:50.150+0000 7f7e7aa00700 1 --2- 172.21.15.150:0/2127058328 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f7e6803e9a0 0x7f7e68040e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:13:50.149 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:50.151+0000 7f7e7aa00700 1 -- 172.21.15.150:0/2127058328 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f7e74053c40 con 0x7f7e84103000 2024-05-03T18:13:50.149 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:50.151+0000 7f7e81400700 1 --2- 172.21.15.150:0/2127058328 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f7e6803e9a0 0x7f7e68040e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:13:50.150 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:50.152+0000 7f7e81400700 1 --2- 172.21.15.150:0/2127058328 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f7e6803e9a0 0x7f7e68040e60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7e70009ad0 tx=0x7f7e70009210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:13:50.164 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:50.166+0000 7f7e7aa00700 1 -- 172.21.15.150:0/2127058328 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f7e74013070 con 0x7f7e84103000 2024-05-03T18:13:50.416 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:50.417+0000 7f7e89200700 1 -- 172.21.15.150:0/2127058328 --> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi033=x;smithi099=y;smithi150=z", "target": ["mon-mgr", ""]}) -- 0x7f7e840638e0 con 0x7f7e6803e9a0 2024-05-03T18:13:50.431 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:50.432+0000 7f7e7aa00700 1 -- 172.21.15.150:0/2127058328 <== mgr.14152 v2:172.21.15.33:6800/3642895501 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f7e840638e0 con 0x7f7e6803e9a0 2024-05-03T18:13:50.431 INFO:teuthology.orchestra.run.smithi150.stdout:Scheduled mgr update... 2024-05-03T18:13:50.435 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:50.436+0000 7f7e89200700 1 -- 172.21.15.150:0/2127058328 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f7e6803e9a0 msgr2=0x7f7e68040e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:13:50.435 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:50.436+0000 7f7e89200700 1 --2- 172.21.15.150:0/2127058328 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f7e6803e9a0 0x7f7e68040e60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7e70009ad0 tx=0x7f7e70009210 comp rx=0 tx=0).stop 2024-05-03T18:13:50.435 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:50.436+0000 7f7e89200700 1 -- 172.21.15.150:0/2127058328 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e84103000 msgr2=0x7f7e8419a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:13:50.435 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:50.436+0000 7f7e89200700 1 --2- 172.21.15.150:0/2127058328 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e84103000 0x7f7e8419a8d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7e74005000 tx=0x7f7e74005030 comp rx=0 tx=0).stop 2024-05-03T18:13:50.435 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:50.436+0000 7f7e89200700 1 -- 172.21.15.150:0/2127058328 shutdown_connections 2024-05-03T18:13:50.435 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:50.436+0000 7f7e89200700 1 --2- 172.21.15.150:0/2127058328 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7e8419b290 0x7f7e841a2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:13:50.436 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:50.436+0000 7f7e89200700 1 --2- 172.21.15.150:0/2127058328 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e84103000 0x7f7e8419a8d0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:13:50.436 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:50.436+0000 7f7e89200700 1 --2- 172.21.15.150:0/2127058328 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f7e6803e9a0 0x7f7e68040e60 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:13:50.436 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:50.436+0000 7f7e89200700 1 --2- 172.21.15.150:0/2127058328 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e8419ae10 0x7f7e8419fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:13:50.436 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:50.436+0000 7f7e89200700 1 -- 172.21.15.150:0/2127058328 >> 172.21.15.150:0/2127058328 conn(0x7f7e840fe5a0 msgr2=0x7f7e841009e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:13:50.436 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:50.436+0000 7f7e89200700 1 -- 172.21.15.150:0/2127058328 shutdown_connections 2024-05-03T18:13:50.436 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:13:50.436+0000 7f7e89200700 1 -- 172.21.15.150:0/2127058328 wait complete. 2024-05-03T18:13:50.784 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:50 smithi150 ceph-mon[32492]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:50.785 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:50 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:50.785 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:50 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:50.785 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:50 smithi150 ceph-mon[32492]: Reconfiguring mon.c (monmap changed)... 2024-05-03T18:13:50.785 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:50 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-03T18:13:50.785 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:50 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-03T18:13:50.785 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:50 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:13:50.785 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:50 smithi150 ceph-mon[32492]: Reconfiguring daemon mon.c on smithi150 2024-05-03T18:13:50.785 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:50 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:50.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:50 smithi033 ceph-mon[28330]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:50.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:50 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:50.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:50 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:50.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:50 smithi033 ceph-mon[28330]: Reconfiguring mon.c (monmap changed)... 2024-05-03T18:13:50.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:50 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-03T18:13:50.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:50 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-03T18:13:50.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:50 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:13:50.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:50 smithi033 ceph-mon[28330]: Reconfiguring daemon mon.c on smithi150 2024-05-03T18:13:50.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:50 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:51.044 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:50 smithi099 ceph-mon[31658]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:51.044 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:50 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:51.044 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:50 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:51.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:50 smithi099 ceph-mon[31658]: Reconfiguring mon.c (monmap changed)... 2024-05-03T18:13:51.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:50 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-03T18:13:51.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:50 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-03T18:13:51.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:50 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:13:51.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:50 smithi099 ceph-mon[31658]: Reconfiguring daemon mon.c on smithi150 2024-05-03T18:13:51.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:50 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:51.376 DEBUG:teuthology.orchestra.run.smithi099:mgr.y> sudo journalctl -f -n 0 -u ceph-5703bba4-0978-11ef-bc97-c7b262605968@mgr.y.service 2024-05-03T18:13:51.384 DEBUG:teuthology.orchestra.run.smithi150:mgr.z> sudo journalctl -f -n 0 -u ceph-5703bba4-0978-11ef-bc97-c7b262605968@mgr.z.service 2024-05-03T18:13:51.391 INFO:tasks.cephadm:Deploying OSDs... 2024-05-03T18:13:51.392 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-03T18:13:51.392 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/scratch_devs of=/dev/stdout 2024-05-03T18:13:51.454 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-03T18:13:51.455 DEBUG:teuthology.orchestra.run.smithi033:> stat /dev/vg_nvme/lv_1 2024-05-03T18:13:51.509 INFO:teuthology.orchestra.run.smithi033.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-03T18:13:51.509 INFO:teuthology.orchestra.run.smithi033.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-03T18:13:51.509 INFO:teuthology.orchestra.run.smithi033.stdout:Device: 6h/6d Inode: 633 Links: 1 2024-05-03T18:13:51.510 INFO:teuthology.orchestra.run.smithi033.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-03T18:13:51.510 INFO:teuthology.orchestra.run.smithi033.stdout:Context: system_u:object_r:device_t:s0 2024-05-03T18:13:51.510 INFO:teuthology.orchestra.run.smithi033.stdout:Access: 2024-05-03 18:12:35.778244706 +0000 2024-05-03T18:13:51.510 INFO:teuthology.orchestra.run.smithi033.stdout:Modify: 2024-05-03 18:07:08.219680861 +0000 2024-05-03T18:13:51.510 INFO:teuthology.orchestra.run.smithi033.stdout:Change: 2024-05-03 18:07:08.219680861 +0000 2024-05-03T18:13:51.510 INFO:teuthology.orchestra.run.smithi033.stdout: Birth: 2024-05-03 18:07:08.219680861 +0000 2024-05-03T18:13:51.510 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-03T18:13:51.589 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records in 2024-05-03T18:13:51.590 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records out 2024-05-03T18:13:51.590 INFO:teuthology.orchestra.run.smithi033.stderr:512 bytes copied, 0.000399315 s, 1.3 MB/s 2024-05-03T18:13:51.593 DEBUG:teuthology.orchestra.run.smithi033:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-03T18:13:51.648 DEBUG:teuthology.orchestra.run.smithi033:> stat /dev/vg_nvme/lv_2 2024-05-03T18:13:51.695 INFO:teuthology.orchestra.run.smithi033.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-03T18:13:51.695 INFO:teuthology.orchestra.run.smithi033.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-03T18:13:51.695 INFO:teuthology.orchestra.run.smithi033.stdout:Device: 6h/6d Inode: 650 Links: 1 2024-05-03T18:13:51.695 INFO:teuthology.orchestra.run.smithi033.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-03T18:13:51.696 INFO:teuthology.orchestra.run.smithi033.stdout:Context: system_u:object_r:device_t:s0 2024-05-03T18:13:51.696 INFO:teuthology.orchestra.run.smithi033.stdout:Access: 2024-05-03 18:12:35.866243208 +0000 2024-05-03T18:13:51.696 INFO:teuthology.orchestra.run.smithi033.stdout:Modify: 2024-05-03 18:07:08.802671075 +0000 2024-05-03T18:13:51.696 INFO:teuthology.orchestra.run.smithi033.stdout:Change: 2024-05-03 18:07:08.802671075 +0000 2024-05-03T18:13:51.696 INFO:teuthology.orchestra.run.smithi033.stdout: Birth: 2024-05-03 18:07:08.802671075 +0000 2024-05-03T18:13:51.696 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-03T18:13:51.794 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:51 smithi099 ceph-mon[31658]: from='client.14214 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi033=x;smithi099=y;smithi150=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:13:51.794 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:51 smithi099 ceph-mon[31658]: Saving service mgr spec with placement smithi033=x;smithi099=y;smithi150=z;count:3 2024-05-03T18:13:51.815 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:51 smithi150 ceph-mon[32492]: from='client.14214 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi033=x;smithi099=y;smithi150=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:13:51.815 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:51 smithi150 ceph-mon[32492]: Saving service mgr spec with placement smithi033=x;smithi099=y;smithi150=z;count:3 2024-05-03T18:13:51.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:51 smithi033 ceph-mon[28330]: from='client.14214 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi033=x;smithi099=y;smithi150=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:13:51.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:51 smithi033 ceph-mon[28330]: Saving service mgr spec with placement smithi033=x;smithi099=y;smithi150=z;count:3 2024-05-03T18:13:51.826 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records in 2024-05-03T18:13:51.826 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records out 2024-05-03T18:13:51.826 INFO:teuthology.orchestra.run.smithi033.stderr:512 bytes copied, 0.000422931 s, 1.2 MB/s 2024-05-03T18:13:51.831 DEBUG:teuthology.orchestra.run.smithi033:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-03T18:13:51.893 DEBUG:teuthology.orchestra.run.smithi033:> stat /dev/vg_nvme/lv_3 2024-05-03T18:13:51.950 INFO:teuthology.orchestra.run.smithi033.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-03T18:13:51.950 INFO:teuthology.orchestra.run.smithi033.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-03T18:13:51.950 INFO:teuthology.orchestra.run.smithi033.stdout:Device: 6h/6d Inode: 670 Links: 1 2024-05-03T18:13:51.950 INFO:teuthology.orchestra.run.smithi033.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-03T18:13:51.950 INFO:teuthology.orchestra.run.smithi033.stdout:Context: system_u:object_r:device_t:s0 2024-05-03T18:13:51.950 INFO:teuthology.orchestra.run.smithi033.stdout:Access: 2024-05-03 18:12:35.956241676 +0000 2024-05-03T18:13:51.950 INFO:teuthology.orchestra.run.smithi033.stdout:Modify: 2024-05-03 18:07:09.405660954 +0000 2024-05-03T18:13:51.950 INFO:teuthology.orchestra.run.smithi033.stdout:Change: 2024-05-03 18:07:09.405660954 +0000 2024-05-03T18:13:51.950 INFO:teuthology.orchestra.run.smithi033.stdout: Birth: 2024-05-03 18:07:09.405660954 +0000 2024-05-03T18:13:51.951 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-03T18:13:52.037 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records in 2024-05-03T18:13:52.038 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records out 2024-05-03T18:13:52.038 INFO:teuthology.orchestra.run.smithi033.stderr:512 bytes copied, 0.000412571 s, 1.2 MB/s 2024-05-03T18:13:52.041 DEBUG:teuthology.orchestra.run.smithi033:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-03T18:13:52.100 DEBUG:teuthology.orchestra.run.smithi033:> stat /dev/vg_nvme/lv_4 2024-05-03T18:13:52.152 INFO:teuthology.orchestra.run.smithi033.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-03T18:13:52.153 INFO:teuthology.orchestra.run.smithi033.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-03T18:13:52.153 INFO:teuthology.orchestra.run.smithi033.stdout:Device: 6h/6d Inode: 690 Links: 1 2024-05-03T18:13:52.153 INFO:teuthology.orchestra.run.smithi033.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-03T18:13:52.153 INFO:teuthology.orchestra.run.smithi033.stdout:Context: system_u:object_r:device_t:s0 2024-05-03T18:13:52.153 INFO:teuthology.orchestra.run.smithi033.stdout:Access: 2024-05-03 18:12:36.034240348 +0000 2024-05-03T18:13:52.153 INFO:teuthology.orchestra.run.smithi033.stdout:Modify: 2024-05-03 18:07:10.028650498 +0000 2024-05-03T18:13:52.153 INFO:teuthology.orchestra.run.smithi033.stdout:Change: 2024-05-03 18:07:10.028650498 +0000 2024-05-03T18:13:52.153 INFO:teuthology.orchestra.run.smithi033.stdout: Birth: 2024-05-03 18:07:10.028650498 +0000 2024-05-03T18:13:52.153 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-03T18:13:52.236 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records in 2024-05-03T18:13:52.237 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records out 2024-05-03T18:13:52.237 INFO:teuthology.orchestra.run.smithi033.stderr:512 bytes copied, 0.000330009 s, 1.6 MB/s 2024-05-03T18:13:52.240 DEBUG:teuthology.orchestra.run.smithi033:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-03T18:13:52.299 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-05-03T18:13:52.299 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/scratch_devs of=/dev/stdout 2024-05-03T18:13:52.356 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-03T18:13:52.356 DEBUG:teuthology.orchestra.run.smithi099:> stat /dev/vg_nvme/lv_1 2024-05-03T18:13:52.407 INFO:teuthology.orchestra.run.smithi099.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-03T18:13:52.408 INFO:teuthology.orchestra.run.smithi099.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-03T18:13:52.408 INFO:teuthology.orchestra.run.smithi099.stdout:Device: 6h/6d Inode: 631 Links: 1 2024-05-03T18:13:52.408 INFO:teuthology.orchestra.run.smithi099.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-03T18:13:52.408 INFO:teuthology.orchestra.run.smithi099.stdout:Context: system_u:object_r:device_t:s0 2024-05-03T18:13:52.408 INFO:teuthology.orchestra.run.smithi099.stdout:Access: 2024-05-03 18:13:33.855762402 +0000 2024-05-03T18:13:52.408 INFO:teuthology.orchestra.run.smithi099.stdout:Modify: 2024-05-03 18:06:56.887196241 +0000 2024-05-03T18:13:52.408 INFO:teuthology.orchestra.run.smithi099.stdout:Change: 2024-05-03 18:06:56.887196241 +0000 2024-05-03T18:13:52.408 INFO:teuthology.orchestra.run.smithi099.stdout: Birth: 2024-05-03 18:06:56.887196241 +0000 2024-05-03T18:13:52.408 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-03T18:13:52.482 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records in 2024-05-03T18:13:52.482 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records out 2024-05-03T18:13:52.482 INFO:teuthology.orchestra.run.smithi099.stderr:512 bytes copied, 0.000388794 s, 1.3 MB/s 2024-05-03T18:13:52.486 DEBUG:teuthology.orchestra.run.smithi099:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-03T18:13:52.542 DEBUG:teuthology.orchestra.run.smithi099:> stat /dev/vg_nvme/lv_2 2024-05-03T18:13:52.591 INFO:teuthology.orchestra.run.smithi099.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-03T18:13:52.591 INFO:teuthology.orchestra.run.smithi099.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-03T18:13:52.591 INFO:teuthology.orchestra.run.smithi099.stdout:Device: 6h/6d Inode: 651 Links: 1 2024-05-03T18:13:52.591 INFO:teuthology.orchestra.run.smithi099.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-03T18:13:52.591 INFO:teuthology.orchestra.run.smithi099.stdout:Context: system_u:object_r:device_t:s0 2024-05-03T18:13:52.591 INFO:teuthology.orchestra.run.smithi099.stdout:Access: 2024-05-03 18:13:33.855762402 +0000 2024-05-03T18:13:52.591 INFO:teuthology.orchestra.run.smithi099.stdout:Modify: 2024-05-03 18:06:57.478182757 +0000 2024-05-03T18:13:52.591 INFO:teuthology.orchestra.run.smithi099.stdout:Change: 2024-05-03 18:06:57.478182757 +0000 2024-05-03T18:13:52.591 INFO:teuthology.orchestra.run.smithi099.stdout: Birth: 2024-05-03 18:06:57.478182757 +0000 2024-05-03T18:13:52.591 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-03T18:13:52.659 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records in 2024-05-03T18:13:52.659 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records out 2024-05-03T18:13:52.659 INFO:teuthology.orchestra.run.smithi099.stderr:512 bytes copied, 0.000364931 s, 1.4 MB/s 2024-05-03T18:13:52.662 DEBUG:teuthology.orchestra.run.smithi099:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-03T18:13:52.715 DEBUG:teuthology.orchestra.run.smithi099:> stat /dev/vg_nvme/lv_3 2024-05-03T18:13:52.762 INFO:teuthology.orchestra.run.smithi099.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-03T18:13:52.762 INFO:teuthology.orchestra.run.smithi099.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-03T18:13:52.763 INFO:teuthology.orchestra.run.smithi099.stdout:Device: 6h/6d Inode: 670 Links: 1 2024-05-03T18:13:52.763 INFO:teuthology.orchestra.run.smithi099.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-03T18:13:52.763 INFO:teuthology.orchestra.run.smithi099.stdout:Context: system_u:object_r:device_t:s0 2024-05-03T18:13:52.763 INFO:teuthology.orchestra.run.smithi099.stdout:Access: 2024-05-03 18:13:33.856762380 +0000 2024-05-03T18:13:52.763 INFO:teuthology.orchestra.run.smithi099.stdout:Modify: 2024-05-03 18:06:58.059169501 +0000 2024-05-03T18:13:52.763 INFO:teuthology.orchestra.run.smithi099.stdout:Change: 2024-05-03 18:06:58.059169501 +0000 2024-05-03T18:13:52.763 INFO:teuthology.orchestra.run.smithi099.stdout: Birth: 2024-05-03 18:06:58.059169501 +0000 2024-05-03T18:13:52.763 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-03T18:13:52.794 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:52 smithi099 ceph-mon[31658]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:52.794 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:52 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:52.794 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:52 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:52.794 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:52 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:13:52.794 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:52 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:13:52.794 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:52 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:13:52.794 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:52 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:52.794 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:52 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-03T18:13:52.794 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:52 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-03T18:13:52.794 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:52 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-03T18:13:52.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:52 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:13:52.833 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records in 2024-05-03T18:13:52.833 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records out 2024-05-03T18:13:52.833 INFO:teuthology.orchestra.run.smithi099.stderr:512 bytes copied, 0.000361971 s, 1.4 MB/s 2024-05-03T18:13:52.836 DEBUG:teuthology.orchestra.run.smithi099:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-03T18:13:52.881 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:52 smithi150 ceph-mon[32492]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:52.881 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:52 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:52.881 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:52 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:52.881 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:52 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:13:52.881 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:52 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:13:52.881 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:52 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:13:52.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:52 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:52.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:52 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-03T18:13:52.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:52 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-03T18:13:52.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:52 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-03T18:13:52.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:52 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:13:52.934 DEBUG:teuthology.orchestra.run.smithi099:> stat /dev/vg_nvme/lv_4 2024-05-03T18:13:52.982 INFO:teuthology.orchestra.run.smithi099.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-03T18:13:52.982 INFO:teuthology.orchestra.run.smithi099.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-03T18:13:52.982 INFO:teuthology.orchestra.run.smithi099.stdout:Device: 6h/6d Inode: 688 Links: 1 2024-05-03T18:13:52.982 INFO:teuthology.orchestra.run.smithi099.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-03T18:13:52.982 INFO:teuthology.orchestra.run.smithi099.stdout:Context: system_u:object_r:device_t:s0 2024-05-03T18:13:52.982 INFO:teuthology.orchestra.run.smithi099.stdout:Access: 2024-05-03 18:13:33.857762357 +0000 2024-05-03T18:13:52.982 INFO:teuthology.orchestra.run.smithi099.stdout:Modify: 2024-05-03 18:06:58.676155425 +0000 2024-05-03T18:13:52.982 INFO:teuthology.orchestra.run.smithi099.stdout:Change: 2024-05-03 18:06:58.676155425 +0000 2024-05-03T18:13:52.982 INFO:teuthology.orchestra.run.smithi099.stdout: Birth: 2024-05-03 18:06:58.676155425 +0000 2024-05-03T18:13:52.983 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-03T18:13:52.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:52 smithi033 ceph-mon[28330]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:52.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:52 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:52.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:52 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:52.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:52 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:13:52.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:52 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:13:52.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:52 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:13:52.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:52 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:52.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:52 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-03T18:13:52.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:52 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-03T18:13:52.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:52 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-03T18:13:52.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:52 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:13:53.052 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records in 2024-05-03T18:13:53.052 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records out 2024-05-03T18:13:53.052 INFO:teuthology.orchestra.run.smithi099.stderr:512 bytes copied, 0.000434609 s, 1.2 MB/s 2024-05-03T18:13:53.056 DEBUG:teuthology.orchestra.run.smithi099:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-03T18:13:53.108 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-05-03T18:13:53.108 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/scratch_devs of=/dev/stdout 2024-05-03T18:13:53.151 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-03T18:13:53.151 DEBUG:teuthology.orchestra.run.smithi150:> stat /dev/vg_nvme/lv_1 2024-05-03T18:13:53.231 INFO:teuthology.orchestra.run.smithi150.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-03T18:13:53.232 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-03T18:13:53.232 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 6h/6d Inode: 633 Links: 1 2024-05-03T18:13:53.232 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-03T18:13:53.232 INFO:teuthology.orchestra.run.smithi150.stdout:Context: system_u:object_r:device_t:s0 2024-05-03T18:13:53.232 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2024-05-03 18:13:22.937575011 +0000 2024-05-03T18:13:53.232 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2024-05-03 18:06:58.233394113 +0000 2024-05-03T18:13:53.232 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2024-05-03 18:06:58.233394113 +0000 2024-05-03T18:13:53.232 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: 2024-05-03 18:06:58.233394113 +0000 2024-05-03T18:13:53.232 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-03T18:13:53.334 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records in 2024-05-03T18:13:53.334 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records out 2024-05-03T18:13:53.334 INFO:teuthology.orchestra.run.smithi150.stderr:512 bytes copied, 0.000356646 s, 1.4 MB/s 2024-05-03T18:13:53.337 DEBUG:teuthology.orchestra.run.smithi150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-03T18:13:53.383 DEBUG:teuthology.orchestra.run.smithi150:> stat /dev/vg_nvme/lv_2 2024-05-03T18:13:53.476 INFO:teuthology.orchestra.run.smithi150.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-03T18:13:53.476 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-03T18:13:53.476 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 6h/6d Inode: 653 Links: 1 2024-05-03T18:13:53.476 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-03T18:13:53.476 INFO:teuthology.orchestra.run.smithi150.stdout:Context: system_u:object_r:device_t:s0 2024-05-03T18:13:53.477 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2024-05-03 18:13:22.938574991 +0000 2024-05-03T18:13:53.477 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2024-05-03 18:06:58.838382432 +0000 2024-05-03T18:13:53.477 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2024-05-03 18:06:58.838382432 +0000 2024-05-03T18:13:53.477 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: 2024-05-03 18:06:58.838382432 +0000 2024-05-03T18:13:53.477 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-03T18:13:53.539 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records in 2024-05-03T18:13:53.539 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records out 2024-05-03T18:13:53.539 INFO:teuthology.orchestra.run.smithi150.stderr:512 bytes copied, 0.00029406 s, 1.7 MB/s 2024-05-03T18:13:53.542 DEBUG:teuthology.orchestra.run.smithi150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-03T18:13:53.585 DEBUG:teuthology.orchestra.run.smithi150:> stat /dev/vg_nvme/lv_3 2024-05-03T18:13:53.673 INFO:teuthology.orchestra.run.smithi150.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-03T18:13:53.673 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-03T18:13:53.673 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 6h/6d Inode: 670 Links: 1 2024-05-03T18:13:53.674 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-03T18:13:53.674 INFO:teuthology.orchestra.run.smithi150.stdout:Context: system_u:object_r:device_t:s0 2024-05-03T18:13:53.674 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2024-05-03 18:13:22.939574972 +0000 2024-05-03T18:13:53.674 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2024-05-03 18:06:59.409371407 +0000 2024-05-03T18:13:53.674 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2024-05-03 18:06:59.409371407 +0000 2024-05-03T18:13:53.674 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: 2024-05-03 18:06:59.409371407 +0000 2024-05-03T18:13:53.674 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-03T18:13:53.733 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records in 2024-05-03T18:13:53.733 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records out 2024-05-03T18:13:53.733 INFO:teuthology.orchestra.run.smithi150.stderr:512 bytes copied, 0.000287029 s, 1.8 MB/s 2024-05-03T18:13:53.736 DEBUG:teuthology.orchestra.run.smithi150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-03T18:13:53.778 DEBUG:teuthology.orchestra.run.smithi150:> stat /dev/vg_nvme/lv_4 2024-05-03T18:13:53.865 INFO:teuthology.orchestra.run.smithi150.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-03T18:13:53.866 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-03T18:13:53.866 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 6h/6d Inode: 690 Links: 1 2024-05-03T18:13:53.866 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-03T18:13:53.866 INFO:teuthology.orchestra.run.smithi150.stdout:Context: system_u:object_r:device_t:s0 2024-05-03T18:13:53.866 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2024-05-03 18:13:22.939574972 +0000 2024-05-03T18:13:53.866 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2024-05-03 18:07:00.017359654 +0000 2024-05-03T18:13:53.866 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2024-05-03 18:07:00.017359654 +0000 2024-05-03T18:13:53.866 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: 2024-05-03 18:07:00.017359654 +0000 2024-05-03T18:13:53.867 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-03T18:13:53.880 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:53 smithi150 ceph-mon[32492]: Deploying daemon mgr.z on smithi150 2024-05-03T18:13:53.957 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records in 2024-05-03T18:13:53.957 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records out 2024-05-03T18:13:53.957 INFO:teuthology.orchestra.run.smithi150.stderr:512 bytes copied, 0.000350995 s, 1.5 MB/s 2024-05-03T18:13:53.961 DEBUG:teuthology.orchestra.run.smithi150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-03T18:13:53.982 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:53 smithi033 ceph-mon[28330]: Deploying daemon mgr.z on smithi150 2024-05-03T18:13:54.013 INFO:tasks.cephadm:Deploying osd.0 on smithi033 with /dev/vg_nvme/lv_4... 2024-05-03T18:13:54.013 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-03T18:13:54.044 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:53 smithi099 ceph-mon[31658]: Deploying daemon mgr.z on smithi150 2024-05-03T18:13:54.453 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:13:54.881 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:54 smithi150 ceph-mon[32492]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:54.892 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:54 smithi033 ceph-mon[28330]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:55.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:54 smithi099 ceph-mon[31658]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:55.631 INFO:journalctl@ceph.mgr.z.smithi150.stdout:May 03 18:13:55 smithi150 systemd[1]: Starting Ceph mgr.z for 5703bba4-0978-11ef-bc97-c7b262605968... 2024-05-03T18:13:55.969 INFO:journalctl@ceph.mgr.z.smithi150.stdout:May 03 18:13:55 smithi150 podman[34347]: 2024-05-03 18:13:55.548297122 +0000 UTC m=+0.031755166 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T18:13:55.969 INFO:journalctl@ceph.mgr.z.smithi150.stdout:May 03 18:13:55 smithi150 podman[34347]: 2024-05-03 18:13:55.81092424 +0000 UTC m=+0.294382321 container create aa8e47cd8dde7fb55087c2c58b1e8296b5a03f8e0c48d945d25be78d1bd495d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-z, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240430, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-05-03T18:13:56.324 INFO:journalctl@ceph.mgr.z.smithi150.stdout:May 03 18:13:56 smithi150 podman[34347]: 2024-05-03 18:13:56.064327797 +0000 UTC m=+0.547785901 container init aa8e47cd8dde7fb55087c2c58b1e8296b5a03f8e0c48d945d25be78d1bd495d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-z, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe) 2024-05-03T18:13:56.324 INFO:journalctl@ceph.mgr.z.smithi150.stdout:May 03 18:13:56 smithi150 podman[34347]: 2024-05-03 18:13:56.07498413 +0000 UTC m=+0.558442241 container start aa8e47cd8dde7fb55087c2c58b1e8296b5a03f8e0c48d945d25be78d1bd495d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-z, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-05-03T18:13:56.324 INFO:journalctl@ceph.mgr.z.smithi150.stdout:May 03 18:13:56 smithi150 ceph-mgr[34366]: -- 172.21.15.150:0/1220180835 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x557153b744e0 con 0x557153b4ac00 2024-05-03T18:13:56.324 INFO:journalctl@ceph.mgr.z.smithi150.stdout:May 03 18:13:56 smithi150 ceph-mgr[34366]: -- 172.21.15.150:0/1220180835 <== mon.1 v2:172.21.15.150:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557153b765a0 con 0x557153b4ac00 2024-05-03T18:13:56.324 INFO:journalctl@ceph.mgr.z.smithi150.stdout:May 03 18:13:56 smithi150 bash[34347]: aa8e47cd8dde7fb55087c2c58b1e8296b5a03f8e0c48d945d25be78d1bd495d6 2024-05-03T18:13:56.324 INFO:journalctl@ceph.mgr.z.smithi150.stdout:May 03 18:13:56 smithi150 systemd[1]: Started Ceph mgr.z for 5703bba4-0978-11ef-bc97-c7b262605968. 2024-05-03T18:13:56.324 INFO:journalctl@ceph.mgr.z.smithi150.stdout:May 03 18:13:56 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-z[34362]: 2024-05-03T18:13:56.323+0000 7f9f26296200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-03T18:13:56.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:56 smithi033 ceph-mon[28330]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:56.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:56 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:56.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:56 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:56.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:56 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:56.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:56 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-03T18:13:56.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:56 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-03T18:13:56.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:56 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-03T18:13:56.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:56 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:13:56.881 INFO:journalctl@ceph.mgr.z.smithi150.stdout:May 03 18:13:56 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-z[34362]: 2024-05-03T18:13:56.598+0000 7f9f26296200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-03T18:13:56.881 INFO:journalctl@ceph.mgr.z.smithi150.stdout:May 03 18:13:56 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-z[34362]: 2024-05-03T18:13:56.738+0000 7f9f26296200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-03T18:13:56.881 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:56 smithi150 ceph-mon[32492]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:56.881 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:56 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:56.881 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:56 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:56.881 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:56 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:56.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:56 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-03T18:13:56.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:56 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-03T18:13:56.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:56 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-03T18:13:56.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:56 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:13:57.044 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:56 smithi099 ceph-mon[31658]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:57.044 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:56 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:57.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:56 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:57.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:56 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:13:57.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:56 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-03T18:13:57.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:56 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-03T18:13:57.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:56 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-03T18:13:57.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:56 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:13:57.357 INFO:teuthology.orchestra.run.smithi033.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-03T18:13:57.357 INFO:teuthology.orchestra.run.smithi033.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-03T18:13:57.357 INFO:teuthology.orchestra.run.smithi033.stderr: stderr: 10+0 records in 2024-05-03T18:13:57.357 INFO:teuthology.orchestra.run.smithi033.stderr:10+0 records out 2024-05-03T18:13:57.357 INFO:teuthology.orchestra.run.smithi033.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0383672 s, 273 MB/s 2024-05-03T18:13:57.357 INFO:teuthology.orchestra.run.smithi033.stderr:--> Zapping successful for: 2024-05-03T18:13:57.598 INFO:journalctl@ceph.mgr.z.smithi150.stdout:May 03 18:13:57 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-z[34362]: 2024-05-03T18:13:57.134+0000 7f9f26296200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-03T18:13:57.598 INFO:journalctl@ceph.mgr.z.smithi150.stdout:May 03 18:13:57 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-z[34362]: 2024-05-03T18:13:57.243+0000 7f9f26296200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-03T18:13:57.727 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:57 smithi099 ceph-mon[31658]: Deploying daemon mgr.y on smithi099 2024-05-03T18:13:57.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:57 smithi033 ceph-mon[28330]: Deploying daemon mgr.y on smithi099 2024-05-03T18:13:57.880 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:57 smithi150 ceph-mon[32492]: Deploying daemon mgr.y on smithi099 2024-05-03T18:13:58.174 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi033:vg_nvme/lv_4 2024-05-03T18:13:58.633 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:13:58.881 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:13:58 smithi150 ceph-mon[32492]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:58.881 INFO:journalctl@ceph.mgr.z.smithi150.stdout:May 03 18:13:58 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-z[34362]: 2024-05-03T18:13:58.801+0000 7f9f26296200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-03T18:13:58.966 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:13:58 smithi033 ceph-mon[28330]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:59.044 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:13:58 smithi099 ceph-mon[31658]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:13:59.381 INFO:journalctl@ceph.mgr.z.smithi150.stdout:May 03 18:13:59 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-z[34362]: 2024-05-03T18:13:59.027+0000 7f9f26296200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-03T18:13:59.381 INFO:journalctl@ceph.mgr.z.smithi150.stdout:May 03 18:13:59 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-z[34362]: 2024-05-03T18:13:59.240+0000 7f9f26296200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-03T18:13:59.737 INFO:journalctl@ceph.mgr.y.smithi099.stdout:May 03 18:13:59 smithi099 systemd[1]: Starting Ceph mgr.y for 5703bba4-0978-11ef-bc97-c7b262605968... 2024-05-03T18:13:59.881 INFO:journalctl@ceph.mgr.z.smithi150.stdout:May 03 18:13:59 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-z[34362]: 2024-05-03T18:13:59.619+0000 7f9f26296200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-03T18:13:59.881 INFO:journalctl@ceph.mgr.z.smithi150.stdout:May 03 18:13:59 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-z[34362]: 2024-05-03T18:13:59.738+0000 7f9f26296200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-03T18:14:00.027 INFO:journalctl@ceph.mgr.y.smithi099.stdout:May 03 18:13:59 smithi099 podman[33032]: 2024-05-03 18:13:59.642730262 +0000 UTC m=+0.036150042 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T18:14:00.027 INFO:journalctl@ceph.mgr.y.smithi099.stdout:May 03 18:13:59 smithi099 podman[33032]: 2024-05-03 18:13:59.755841511 +0000 UTC m=+0.149261321 container create 028f4cdf6284139251cf532de484f4338ba682fa738ca67d359627135bcbc7cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-y, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, ceph=True) 2024-05-03T18:14:00.027 INFO:journalctl@ceph.mgr.y.smithi099.stdout:May 03 18:14:00 smithi099 podman[33032]: 2024-05-03 18:14:00.027614618 +0000 UTC m=+0.421034462 container init 028f4cdf6284139251cf532de484f4338ba682fa738ca67d359627135bcbc7cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-y, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-05-03T18:14:00.277 INFO:journalctl@ceph.mgr.y.smithi099.stdout:May 03 18:14:00 smithi099 podman[33032]: 2024-05-03 18:14:00.038575366 +0000 UTC m=+0.431995176 container start 028f4cdf6284139251cf532de484f4338ba682fa738ca67d359627135bcbc7cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-y, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-05-03T18:14:00.277 INFO:journalctl@ceph.mgr.y.smithi099.stdout:May 03 18:14:00 smithi099 ceph-mgr[33051]: -- 172.21.15.99:0/2185349517 <== mon.2 v2:172.21.15.99:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5567bb22a5a0 con 0x5567bb1fec00 2024-05-03T18:14:00.277 INFO:journalctl@ceph.mgr.y.smithi099.stdout:May 03 18:14:00 smithi099 bash[33032]: 028f4cdf6284139251cf532de484f4338ba682fa738ca67d359627135bcbc7cc 2024-05-03T18:14:00.277 INFO:journalctl@ceph.mgr.y.smithi099.stdout:May 03 18:14:00 smithi099 systemd[1]: Started Ceph mgr.y for 5703bba4-0978-11ef-bc97-c7b262605968. 2024-05-03T18:14:00.381 INFO:journalctl@ceph.mgr.z.smithi150.stdout:May 03 18:13:59 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-z[34362]: 2024-05-03T18:13:59.989+0000 7f9f26296200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-03T18:14:00.381 INFO:journalctl@ceph.mgr.z.smithi150.stdout:May 03 18:14:00 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-z[34362]: 2024-05-03T18:14:00.238+0000 7f9f26296200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-03T18:14:00.536 INFO:journalctl@ceph.mgr.y.smithi099.stdout:May 03 18:14:00 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-y[33047]: 2024-05-03T18:14:00.276+0000 7ff73e552200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-03T18:14:00.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:14:00.669+0000 7f82aa800700 1 -- 172.21.15.33:0/400676351 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f82a4075f00 msgr2=0x7f82a4076380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:14:00.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:14:00.669+0000 7f82aa800700 1 --2- 172.21.15.33:0/400676351 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f82a4075f00 0x7f82a4076380 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f82900098f0 tx=0x7f8290009c00 comp rx=0 tx=0).stop 2024-05-03T18:14:00.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:14:00.669+0000 7f82aa800700 1 -- 172.21.15.33:0/400676351 shutdown_connections 2024-05-03T18:14:00.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:14:00.669+0000 7f82aa800700 1 --2- 172.21.15.33:0/400676351 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82a410b3d0 0x7f82a410d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:14:00.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:14:00.669+0000 7f82aa800700 1 --2- 172.21.15.33:0/400676351 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82a40774d0 0x7f82a4075930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:14:00.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:14:00.669+0000 7f82aa800700 1 --2- 172.21.15.33:0/400676351 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f82a4075f00 0x7f82a4076380 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:14:00.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:14:00.669+0000 7f82aa800700 1 -- 172.21.15.33:0/400676351 >> 172.21.15.33:0/400676351 conn(0x7f82a40feaa0 msgr2=0x7f82a4100ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:14:00.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:14:00.669+0000 7f82aa800700 1 -- 172.21.15.33:0/400676351 shutdown_connections 2024-05-03T18:14:00.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:14:00.669+0000 7f82aa800700 1 -- 172.21.15.33:0/400676351 wait complete. 2024-05-03T18:14:00.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:14:00.671+0000 7f82aa800700 1 Processor -- start 2024-05-03T18:14:00.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:14:00.671+0000 7f82aa800700 1 -- start start 2024-05-03T18:14:00.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:14:00.671+0000 7f82aa800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f82a4075f00 0x7f82a419b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:14:00.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:14:00.671+0000 7f82aa800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82a40774d0 0x7f82a419bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:14:00.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:14:00.671+0000 7f82aa800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82a410b3d0 0x7f82a41a2df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:14:00.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:14:00.671+0000 7f82aa800700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f82a41110b0 con 0x7f82a410b3d0 2024-05-03T18:14:00.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:14:00.671+0000 7f82aa800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f82a4110f30 con 0x7f82a40774d0 2024-05-03T18:14:00.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:14:00.671+0000 7f82aa800700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f82a4111230 con 0x7f82a4075f00 2024-05-03T18:14:00.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:14:00.672+0000 7f82a3400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f82a4075f00 0x7f82a419b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:14:00.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:14:00.672+0000 7f82a3400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f82a4075f00 0x7f82a419b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:44744/0 (socket says 172.21.15.33:44744) 2024-05-03T18:14:00.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:14:00.672+0000 7f82a3400700 1 -- 172.21.15.33:0/3775746636 learned_addr learned my addr 172.21.15.33:0/3775746636 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:14:00.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:14:00.672+0000 7f82a3e00700 1 --2- 172.21.15.33:0/3775746636 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82a410b3d0 0x7f82a41a2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:14:00.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:14:00.672+0000 7f82a2a00700 1 --2- 172.21.15.33:0/3775746636 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82a40774d0 0x7f82a419bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:14:00.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:14:00.672+0000 7f82a3e00700 1 -- 172.21.15.33:0/3775746636 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f82a4075f00 msgr2=0x7f82a419b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:14:00.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:14:00.672+0000 7f82a3e00700 1 --2- 172.21.15.33:0/3775746636 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f82a4075f00 0x7f82a419b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:14:00.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:14:00.672+0000 7f82a3e00700 1 -- 172.21.15.33:0/3775746636 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82a40774d0 msgr2=0x7f82a419bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:14:00.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:14:00.672+0000 7f82a3e00700 1 --2- 172.21.15.33:0/3775746636 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82a40774d0 0x7f82a419bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:14:00.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:14:00.672+0000 7f82a3e00700 1 -- 172.21.15.33:0/3775746636 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82a41a3490 con 0x7f82a410b3d0 2024-05-03T18:14:00.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:14:00.672+0000 7f82a3400700 1 --2- 172.21.15.33:0/3775746636 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f82a4075f00 0x7f82a419b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:14:00.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:14:00.673+0000 7f82a3e00700 1 --2- 172.21.15.33:0/3775746636 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82a410b3d0 0x7f82a41a2df0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f829c009f60 tx=0x7f829c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:14:00.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:14:00.673+0000 7f829be00700 1 -- 172.21.15.33:0/3775746636 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f829c0092e0 con 0x7f82a410b3d0 2024-05-03T18:14:00.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:14:00.673+0000 7f829be00700 1 -- 172.21.15.33:0/3775746636 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f829c004bb0 con 0x7f82a410b3d0 2024-05-03T18:14:00.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:14:00.673+0000 7f829be00700 1 -- 172.21.15.33:0/3775746636 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f829c004020 con 0x7f82a410b3d0 2024-05-03T18:14:00.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:14:00.673+0000 7f82aa800700 1 -- 172.21.15.33:0/3775746636 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82a41a3720 con 0x7f82a410b3d0 2024-05-03T18:14:00.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:14:00.673+0000 7f82aa800700 1 -- 172.21.15.33:0/3775746636 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82a41a3cb0 con 0x7f82a410b3d0 2024-05-03T18:14:00.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:14:00.675+0000 7f829be00700 1 -- 172.21.15.33:0/3775746636 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f829c01b030 con 0x7f82a410b3d0 2024-05-03T18:14:00.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:14:00.675+0000 7f82aa800700 1 -- 172.21.15.33:0/3775746636 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82a406a190 con 0x7f82a410b3d0 2024-05-03T18:14:00.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:14:00.675+0000 7f829be00700 1 --2- 172.21.15.33:0/3775746636 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f828803e750 0x7f8288040c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:14:00.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:14:00.675+0000 7f829be00700 1 -- 172.21.15.33:0/3775746636 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f829c052430 con 0x7f82a410b3d0 2024-05-03T18:14:00.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:14:00.676+0000 7f82a3400700 1 --2- 172.21.15.33:0/3775746636 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f828803e750 0x7f8288040c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:14:00.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:14:00.677+0000 7f82a3400700 1 --2- 172.21.15.33:0/3775746636 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f828803e750 0x7f8288040c10 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8294009750 tx=0x7f8294009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:14:00.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:14:00.681+0000 7f829be00700 1 -- 172.21.15.33:0/3775746636 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f829c022020 con 0x7f82a410b3d0 2024-05-03T18:14:00.794 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:00 smithi099 ceph-mon[31658]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:14:00.794 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:00 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:14:00.794 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:00 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:14:00.794 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:00 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:14:00.794 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:00 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:14:00.795 INFO:journalctl@ceph.mgr.y.smithi099.stdout:May 03 18:14:00 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-y[33047]: 2024-05-03T18:14:00.536+0000 7ff73e552200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-03T18:14:00.795 INFO:journalctl@ceph.mgr.y.smithi099.stdout:May 03 18:14:00 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-y[33047]: 2024-05-03T18:14:00.669+0000 7ff73e552200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-03T18:14:00.881 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:00 smithi150 ceph-mon[32492]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:14:00.881 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:00 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:14:00.881 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:00 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:14:00.881 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:00 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:14:00.881 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:00 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:14:00.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:14:00.934+0000 7f82aa800700 1 -- 172.21.15.33:0/3775746636 --> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f82a40639a0 con 0x7f828803e750 2024-05-03T18:14:00.934 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:00 smithi033 ceph-mon[28330]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:14:00.934 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:00 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:14:00.934 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:00 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:14:00.935 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:00 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:14:00.935 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:00 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:14:01.334 INFO:journalctl@ceph.mgr.z.smithi150.stdout:May 03 18:14:01 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-z[34362]: 2024-05-03T18:14:01.040+0000 7f9f26296200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-03T18:14:01.334 INFO:journalctl@ceph.mgr.z.smithi150.stdout:May 03 18:14:01 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-z[34362]: 2024-05-03T18:14:01.334+0000 7f9f26296200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-03T18:14:01.544 INFO:journalctl@ceph.mgr.y.smithi099.stdout:May 03 18:14:01 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-y[33047]: 2024-05-03T18:14:01.088+0000 7ff73e552200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-03T18:14:01.544 INFO:journalctl@ceph.mgr.y.smithi099.stdout:May 03 18:14:01 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-y[33047]: 2024-05-03T18:14:01.203+0000 7ff73e552200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-03T18:14:01.881 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:01 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:14:01.881 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:01 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T18:14:01.881 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:01 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T18:14:01.881 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:01 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:14:01.920 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:01 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:14:01.920 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:01 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T18:14:01.920 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:01 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T18:14:01.920 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:01 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:14:01.941 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:01 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:14:01.941 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:01 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T18:14:01.941 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:01 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T18:14:01.941 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:01 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:14:02.381 INFO:journalctl@ceph.mgr.z.smithi150.stdout:May 03 18:14:01 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-z[34362]: 2024-05-03T18:14:01.913+0000 7f9f26296200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-03T18:14:02.381 INFO:journalctl@ceph.mgr.z.smithi150.stdout:May 03 18:14:02 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-z[34362]: 2024-05-03T18:14:02.048+0000 7f9f26296200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-03T18:14:02.874 INFO:journalctl@ceph.mgr.z.smithi150.stdout:May 03 18:14:02 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-z[34362]: 2024-05-03T18:14:02.520+0000 7f9f26296200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-03T18:14:02.875 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:02 smithi150 ceph-mon[32492]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:14:02.875 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:02 smithi150 ceph-mon[32492]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:14:02.875 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:02 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:14:02.875 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:02 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:14:02.875 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:02 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:14:02.875 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:02 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:14:02.961 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:02 smithi033 ceph-mon[28330]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:14:02.961 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:02 smithi033 ceph-mon[28330]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:14:02.961 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:02 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:14:02.961 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:02 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:14:02.961 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:02 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:14:02.961 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:02 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:14:03.044 INFO:journalctl@ceph.mgr.y.smithi099.stdout:May 03 18:14:02 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-y[33047]: 2024-05-03T18:14:02.840+0000 7ff73e552200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-03T18:14:03.044 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:02 smithi099 ceph-mon[31658]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:14:03.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:02 smithi099 ceph-mon[31658]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:14:03.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:02 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:14:03.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:02 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:14:03.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:02 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:14:03.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:02 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:14:03.131 INFO:journalctl@ceph.mgr.z.smithi150.stdout:May 03 18:14:02 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-z[34362]: 2024-05-03T18:14:02.983+0000 7f9f26296200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-03T18:14:03.131 INFO:journalctl@ceph.mgr.z.smithi150.stdout:May 03 18:14:03 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-z[34362]: 2024-05-03T18:14:03.116+0000 7f9f26296200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-03T18:14:03.544 INFO:journalctl@ceph.mgr.y.smithi099.stdout:May 03 18:14:03 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-y[33047]: 2024-05-03T18:14:03.077+0000 7ff73e552200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-03T18:14:03.544 INFO:journalctl@ceph.mgr.y.smithi099.stdout:May 03 18:14:03 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-y[33047]: 2024-05-03T18:14:03.300+0000 7ff73e552200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-03T18:14:03.880 INFO:journalctl@ceph.mgr.z.smithi150.stdout:May 03 18:14:03 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-z[34362]: 2024-05-03T18:14:03.447+0000 7f9f26296200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-03T18:14:03.881 INFO:journalctl@ceph.mgr.z.smithi150.stdout:May 03 18:14:03 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-z[34362]: 2024-05-03T18:14:03.629+0000 7f9f26296200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-03T18:14:04.044 INFO:journalctl@ceph.mgr.y.smithi099.stdout:May 03 18:14:03 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-y[33047]: 2024-05-03T18:14:03.690+0000 7ff73e552200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-03T18:14:04.044 INFO:journalctl@ceph.mgr.y.smithi099.stdout:May 03 18:14:03 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-y[33047]: 2024-05-03T18:14:03.814+0000 7ff73e552200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-03T18:14:04.288 INFO:journalctl@ceph.mgr.z.smithi150.stdout:May 03 18:14:04 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-z[34362]: 2024-05-03T18:14:04.010+0000 7f9f26296200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-03T18:14:04.288 INFO:journalctl@ceph.mgr.z.smithi150.stdout:May 03 18:14:04 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-z[34362]: 2024-05-03T18:14:04.164+0000 7f9f26296200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-03T18:14:04.288 INFO:journalctl@ceph.mgr.z.smithi150.stdout:May 03 18:14:04 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-z[34362]: 2024-05-03T18:14:04.288+0000 7f9f26296200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-03T18:14:04.320 INFO:journalctl@ceph.mgr.y.smithi099.stdout:May 03 18:14:04 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-y[33047]: 2024-05-03T18:14:04.068+0000 7ff73e552200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-03T18:14:04.623 INFO:journalctl@ceph.mgr.y.smithi099.stdout:May 03 18:14:04 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-y[33047]: 2024-05-03T18:14:04.321+0000 7ff73e552200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-03T18:14:04.881 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:04 smithi150 ceph-mon[32492]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:14:04.881 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:04 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:14:04.881 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:04 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:14:04.881 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:04 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:14:04.881 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:04 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-03T18:14:04.881 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:04 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-03T18:14:04.881 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:04 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:14:04.881 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:04 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:14:04.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:04 smithi033 ceph-mon[28330]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:14:04.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:04 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:14:04.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:04 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:14:04.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:04 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:14:04.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:04 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-03T18:14:04.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:04 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-03T18:14:04.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:04 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:14:04.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:04 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:14:05.044 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:04 smithi099 ceph-mon[31658]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:14:05.044 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:04 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:14:05.044 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:04 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:14:05.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:04 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:14:05.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:04 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-03T18:14:05.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:04 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-03T18:14:05.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:04 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:14:05.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:04 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:14:05.316 INFO:journalctl@ceph.mgr.z.smithi150.stdout:May 03 18:14:05 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-z[34362]: 2024-05-03T18:14:05.037+0000 7f9f26296200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-03T18:14:05.410 INFO:journalctl@ceph.mgr.y.smithi099.stdout:May 03 18:14:05 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-y[33047]: 2024-05-03T18:14:05.123+0000 7ff73e552200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-03T18:14:05.624 INFO:journalctl@ceph.mgr.z.smithi150.stdout:May 03 18:14:05 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-z[34362]: 2024-05-03T18:14:05.317+0000 7f9f26296200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-03T18:14:05.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:05 smithi033 ceph-mon[28330]: Reconfiguring mgr.x (unknown last config time)... 2024-05-03T18:14:05.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:05 smithi033 ceph-mon[28330]: Reconfiguring daemon mgr.x on smithi033 2024-05-03T18:14:05.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:05 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/2442638948' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fdcc4e66-4ba0-4378-a9a4-7d907cd00b4e"}]: dispatch 2024-05-03T18:14:05.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:05 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/2442638948' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fdcc4e66-4ba0-4378-a9a4-7d907cd00b4e"}]': finished 2024-05-03T18:14:05.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:05 smithi033 ceph-mon[28330]: osdmap e5: 1 total, 0 up, 1 in 2024-05-03T18:14:05.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:05 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-03T18:14:05.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:05 smithi033 ceph-mon[28330]: Standby manager daemon z started 2024-05-03T18:14:05.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:05 smithi033 ceph-mon[28330]: from='mgr.? 172.21.15.150:0/308186434' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-03T18:14:05.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:05 smithi033 ceph-mon[28330]: from='mgr.? 172.21.15.150:0/308186434' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-03T18:14:05.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:05 smithi033 ceph-mon[28330]: from='mgr.? 172.21.15.150:0/308186434' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-03T18:14:05.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:05 smithi033 ceph-mon[28330]: from='mgr.? 172.21.15.150:0/308186434' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-03T18:14:05.794 INFO:journalctl@ceph.mgr.y.smithi099.stdout:May 03 18:14:05 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-y[33047]: 2024-05-03T18:14:05.407+0000 7ff73e552200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-03T18:14:05.794 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:05 smithi099 ceph-mon[31658]: Reconfiguring mgr.x (unknown last config time)... 2024-05-03T18:14:05.794 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:05 smithi099 ceph-mon[31658]: Reconfiguring daemon mgr.x on smithi033 2024-05-03T18:14:05.794 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:05 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/2442638948' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fdcc4e66-4ba0-4378-a9a4-7d907cd00b4e"}]: dispatch 2024-05-03T18:14:05.794 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:05 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/2442638948' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fdcc4e66-4ba0-4378-a9a4-7d907cd00b4e"}]': finished 2024-05-03T18:14:05.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:05 smithi099 ceph-mon[31658]: osdmap e5: 1 total, 0 up, 1 in 2024-05-03T18:14:05.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:05 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-03T18:14:05.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:05 smithi099 ceph-mon[31658]: Standby manager daemon z started 2024-05-03T18:14:05.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:05 smithi099 ceph-mon[31658]: from='mgr.? 172.21.15.150:0/308186434' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-03T18:14:05.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:05 smithi099 ceph-mon[31658]: from='mgr.? 172.21.15.150:0/308186434' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-03T18:14:05.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:05 smithi099 ceph-mon[31658]: from='mgr.? 172.21.15.150:0/308186434' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-03T18:14:05.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:05 smithi099 ceph-mon[31658]: from='mgr.? 172.21.15.150:0/308186434' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-03T18:14:05.881 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:05 smithi150 ceph-mon[32492]: Reconfiguring mgr.x (unknown last config time)... 2024-05-03T18:14:05.934 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:05 smithi150 ceph-mon[32492]: Reconfiguring daemon mgr.x on smithi033 2024-05-03T18:14:05.934 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:05 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/2442638948' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fdcc4e66-4ba0-4378-a9a4-7d907cd00b4e"}]: dispatch 2024-05-03T18:14:05.934 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:05 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/2442638948' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fdcc4e66-4ba0-4378-a9a4-7d907cd00b4e"}]': finished 2024-05-03T18:14:05.934 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:05 smithi150 ceph-mon[32492]: osdmap e5: 1 total, 0 up, 1 in 2024-05-03T18:14:05.934 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:05 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-03T18:14:05.934 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:05 smithi150 ceph-mon[32492]: Standby manager daemon z started 2024-05-03T18:14:05.934 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:05 smithi150 ceph-mon[32492]: from='mgr.? 172.21.15.150:0/308186434' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-03T18:14:05.934 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:05 smithi150 ceph-mon[32492]: from='mgr.? 172.21.15.150:0/308186434' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-03T18:14:05.934 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:05 smithi150 ceph-mon[32492]: from='mgr.? 172.21.15.150:0/308186434' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-03T18:14:05.935 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:05 smithi150 ceph-mon[32492]: from='mgr.? 172.21.15.150:0/308186434' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-03T18:14:05.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:14:05.887+0000 7f829be00700 1 -- 172.21.15.33:0/3775746636 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 14) ==== 99937+0+0 (secure 0 0 0) 0x7f829c012050 con 0x7f82a410b3d0 2024-05-03T18:14:06.294 INFO:journalctl@ceph.mgr.y.smithi099.stdout:May 03 18:14:05 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-y[33047]: 2024-05-03T18:14:05.967+0000 7ff73e552200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-03T18:14:06.294 INFO:journalctl@ceph.mgr.y.smithi099.stdout:May 03 18:14:06 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-y[33047]: 2024-05-03T18:14:06.093+0000 7ff73e552200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-03T18:14:06.881 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:06 smithi150 ceph-mon[32492]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:14:06.881 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:06 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/2650638352' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T18:14:06.881 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:06 smithi150 ceph-mon[32492]: mgrmap e14: x(active, since 106s), standbys: z 2024-05-03T18:14:06.881 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:06 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-03T18:14:06.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:06 smithi033 ceph-mon[28330]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:14:06.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:06 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/2650638352' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T18:14:06.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:06 smithi033 ceph-mon[28330]: mgrmap e14: x(active, since 106s), standbys: z 2024-05-03T18:14:06.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:06 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-03T18:14:07.010 INFO:journalctl@ceph.mgr.y.smithi099.stdout:May 03 18:14:06 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-y[33047]: 2024-05-03T18:14:06.561+0000 7ff73e552200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-03T18:14:07.010 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:06 smithi099 ceph-mon[31658]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:14:07.010 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:06 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/2650638352' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T18:14:07.010 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:06 smithi099 ceph-mon[31658]: mgrmap e14: x(active, since 106s), standbys: z 2024-05-03T18:14:07.010 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:06 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-03T18:14:07.294 INFO:journalctl@ceph.mgr.y.smithi099.stdout:May 03 18:14:07 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-y[33047]: 2024-05-03T18:14:07.010+0000 7ff73e552200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-03T18:14:07.294 INFO:journalctl@ceph.mgr.y.smithi099.stdout:May 03 18:14:07 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-y[33047]: 2024-05-03T18:14:07.133+0000 7ff73e552200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-03T18:14:07.794 INFO:journalctl@ceph.mgr.y.smithi099.stdout:May 03 18:14:07 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-y[33047]: 2024-05-03T18:14:07.453+0000 7ff73e552200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-03T18:14:07.794 INFO:journalctl@ceph.mgr.y.smithi099.stdout:May 03 18:14:07 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-y[33047]: 2024-05-03T18:14:07.629+0000 7ff73e552200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-03T18:14:08.277 INFO:journalctl@ceph.mgr.y.smithi099.stdout:May 03 18:14:08 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-y[33047]: 2024-05-03T18:14:08.003+0000 7ff73e552200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-03T18:14:08.277 INFO:journalctl@ceph.mgr.y.smithi099.stdout:May 03 18:14:08 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-y[33047]: 2024-05-03T18:14:08.152+0000 7ff73e552200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-03T18:14:08.277 INFO:journalctl@ceph.mgr.y.smithi099.stdout:May 03 18:14:08 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-y[33047]: 2024-05-03T18:14:08.276+0000 7ff73e552200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-03T18:14:08.982 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:08 smithi033 ceph-mon[28330]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:14:09.033 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:08 smithi099 ceph-mon[31658]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:14:09.131 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:08 smithi150 ceph-mon[32492]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:14:09.294 INFO:journalctl@ceph.mgr.y.smithi099.stdout:May 03 18:14:09 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-y[33047]: 2024-05-03T18:14:09.032+0000 7ff73e552200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-03T18:14:09.656 INFO:journalctl@ceph.mgr.y.smithi099.stdout:May 03 18:14:09 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-y[33047]: 2024-05-03T18:14:09.317+0000 7ff73e552200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-03T18:14:09.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:14:09.672+0000 7f829be00700 1 -- 172.21.15.33:0/3775746636 <== mon.0 v2:172.21.15.33:3300/0 8 ==== mgrmap(e 15) ==== 149596+0+0 (secure 0 0 0) 0x7f829c01c3c0 con 0x7f82a410b3d0 2024-05-03T18:14:09.844 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:09 smithi033 ceph-mon[28330]: Standby manager daemon y started 2024-05-03T18:14:09.844 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:09 smithi033 ceph-mon[28330]: from='mgr.? 172.21.15.99:0/790087822' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-03T18:14:09.844 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:09 smithi033 ceph-mon[28330]: from='mgr.? 172.21.15.99:0/790087822' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-03T18:14:09.844 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:09 smithi033 ceph-mon[28330]: from='mgr.? 172.21.15.99:0/790087822' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-03T18:14:09.844 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:09 smithi033 ceph-mon[28330]: from='mgr.? 172.21.15.99:0/790087822' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-03T18:14:10.044 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:09 smithi099 ceph-mon[31658]: Standby manager daemon y started 2024-05-03T18:14:10.044 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:09 smithi099 ceph-mon[31658]: from='mgr.? 172.21.15.99:0/790087822' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-03T18:14:10.044 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:09 smithi099 ceph-mon[31658]: from='mgr.? 172.21.15.99:0/790087822' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-03T18:14:10.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:09 smithi099 ceph-mon[31658]: from='mgr.? 172.21.15.99:0/790087822' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-03T18:14:10.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:09 smithi099 ceph-mon[31658]: from='mgr.? 172.21.15.99:0/790087822' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-03T18:14:10.131 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:09 smithi150 ceph-mon[32492]: Standby manager daemon y started 2024-05-03T18:14:10.131 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:09 smithi150 ceph-mon[32492]: from='mgr.? 172.21.15.99:0/790087822' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-03T18:14:10.131 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:09 smithi150 ceph-mon[32492]: from='mgr.? 172.21.15.99:0/790087822' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-03T18:14:10.131 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:09 smithi150 ceph-mon[32492]: from='mgr.? 172.21.15.99:0/790087822' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-03T18:14:10.131 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:09 smithi150 ceph-mon[32492]: from='mgr.? 172.21.15.99:0/790087822' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-03T18:14:10.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:10 smithi033 ceph-mon[28330]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:14:10.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:10 smithi033 ceph-mon[28330]: mgrmap e15: x(active, since 110s), standbys: z, y 2024-05-03T18:14:10.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:10 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-03T18:14:11.044 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:10 smithi099 ceph-mon[31658]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:14:11.044 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:10 smithi099 ceph-mon[31658]: mgrmap e15: x(active, since 110s), standbys: z, y 2024-05-03T18:14:11.044 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:10 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-03T18:14:11.131 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:10 smithi150 ceph-mon[32492]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:14:11.131 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:10 smithi150 ceph-mon[32492]: mgrmap e15: x(active, since 110s), standbys: z, y 2024-05-03T18:14:11.131 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:10 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-03T18:14:12.440 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:12 smithi033 ceph-mon[28330]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:14:12.440 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:12 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:14:12.440 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:12 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:14:12.440 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:12 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:14:12.544 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:12 smithi099 ceph-mon[31658]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:14:12.544 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:12 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:14:12.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:12 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:14:12.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:12 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:14:12.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:12 smithi150 ceph-mon[32492]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:14:12.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:12 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:14:12.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:12 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:14:12.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:12 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:14:13.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:13 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:14:13.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:13 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:14:13.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:13 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:14:13.537 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:13 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:14:13.538 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:13 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:14:13.538 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:13 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:14:13.544 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:13 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:14:13.544 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:13 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:14:13.544 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:13 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:14:14.881 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:14 smithi150 ceph-mon[32492]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:14:14.881 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:14 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-03T18:14:14.881 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:14 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:14:14.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:14 smithi033 ceph-mon[28330]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:14:14.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:14 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-03T18:14:14.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:14 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:14:15.044 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:14 smithi099 ceph-mon[31658]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:14:15.044 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:14 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-03T18:14:15.044 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:14 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:14:15.667 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:15 smithi033 ceph-mon[28330]: Deploying daemon osd.0 on smithi033 2024-05-03T18:14:15.881 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:15 smithi150 ceph-mon[32492]: Deploying daemon osd.0 on smithi033 2024-05-03T18:14:16.044 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:15 smithi099 ceph-mon[31658]: Deploying daemon osd.0 on smithi033 2024-05-03T18:14:16.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:16 smithi033 ceph-mon[28330]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:14:16.881 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:16 smithi150 ceph-mon[32492]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:14:17.044 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:16 smithi099 ceph-mon[31658]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:14:18.881 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:18 smithi150 ceph-mon[32492]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:14:18.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:18 smithi033 ceph-mon[28330]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:14:19.044 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:18 smithi099 ceph-mon[31658]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:14:20.881 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:20 smithi150 ceph-mon[32492]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:14:20.920 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:20 smithi033 ceph-mon[28330]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:14:21.044 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:20 smithi099 ceph-mon[31658]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:14:22.881 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:22 smithi150 ceph-mon[32492]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:14:22.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:22 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:14:22.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:22 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:14:22.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:22 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:14:22.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:22 smithi033 ceph-mon[28330]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:14:22.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:22 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:14:22.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:22 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:14:22.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:22 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:14:23.044 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:22 smithi099 ceph-mon[31658]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:14:23.044 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:22 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:14:23.044 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:22 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:14:23.044 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:22 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:14:24.881 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:24 smithi150 ceph-mon[32492]: pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:14:24.906 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:24 smithi033 ceph-mon[28330]: pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:14:25.044 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:24 smithi099 ceph-mon[31658]: pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:14:26.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:26 smithi150 ceph-mon[32492]: pgmap v58: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:14:26.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:26 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:14:26.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:26 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:14:26.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:26 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:14:26.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:26 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:14:26.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:26 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:14:26.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:26 smithi150 ceph-mon[32492]: from='osd.0 [v2:172.21.15.33:6802/637686669,v1:172.21.15.33:6803/637686669]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-03T18:14:26.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:26 smithi033 ceph-mon[28330]: pgmap v58: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:14:26.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:26 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:14:26.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:26 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:14:26.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:26 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:14:26.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:26 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:14:26.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:26 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:14:26.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:26 smithi033 ceph-mon[28330]: from='osd.0 [v2:172.21.15.33:6802/637686669,v1:172.21.15.33:6803/637686669]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-03T18:14:27.044 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:26 smithi099 ceph-mon[31658]: pgmap v58: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:14:27.044 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:26 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:14:27.044 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:26 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:14:27.044 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:26 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:14:27.044 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:26 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:14:27.044 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:26 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:14:27.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:26 smithi099 ceph-mon[31658]: from='osd.0 [v2:172.21.15.33:6802/637686669,v1:172.21.15.33:6803/637686669]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-03T18:14:27.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:14:27.292+0000 7f829be00700 1 -- 172.21.15.33:0/3775746636 <== mgr.14152 v2:172.21.15.33:6800/3642895501 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f82a40639a0 con 0x7f828803e750 2024-05-03T18:14:27.291 INFO:teuthology.orchestra.run.smithi033.stdout:Created osd(s) 0 on host 'smithi033' 2024-05-03T18:14:27.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:14:27.295+0000 7f82aa800700 1 -- 172.21.15.33:0/3775746636 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f828803e750 msgr2=0x7f8288040c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:14:27.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:14:27.295+0000 7f82aa800700 1 --2- 172.21.15.33:0/3775746636 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f828803e750 0x7f8288040c10 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8294009750 tx=0x7f8294009310 comp rx=0 tx=0).stop 2024-05-03T18:14:27.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:14:27.295+0000 7f82aa800700 1 -- 172.21.15.33:0/3775746636 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82a410b3d0 msgr2=0x7f82a41a2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:14:27.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:14:27.295+0000 7f82aa800700 1 --2- 172.21.15.33:0/3775746636 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82a410b3d0 0x7f82a41a2df0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f829c009f60 tx=0x7f829c009f90 comp rx=0 tx=0).stop 2024-05-03T18:14:27.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:14:27.296+0000 7f82aa800700 1 -- 172.21.15.33:0/3775746636 shutdown_connections 2024-05-03T18:14:27.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:14:27.296+0000 7f82aa800700 1 --2- 172.21.15.33:0/3775746636 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82a410b3d0 0x7f82a41a2df0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:14:27.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:14:27.296+0000 7f82aa800700 1 --2- 172.21.15.33:0/3775746636 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f82a4075f00 0x7f82a419b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:14:27.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:14:27.296+0000 7f82aa800700 1 --2- 172.21.15.33:0/3775746636 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f828803e750 0x7f8288040c10 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:14:27.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:14:27.296+0000 7f82aa800700 1 --2- 172.21.15.33:0/3775746636 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82a40774d0 0x7f82a419bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:14:27.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:14:27.296+0000 7f82aa800700 1 -- 172.21.15.33:0/3775746636 >> 172.21.15.33:0/3775746636 conn(0x7f82a40feaa0 msgr2=0x7f82a41072d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:14:27.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:14:27.296+0000 7f82aa800700 1 -- 172.21.15.33:0/3775746636 shutdown_connections 2024-05-03T18:14:27.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:14:27.296+0000 7f82aa800700 1 -- 172.21.15.33:0/3775746636 wait complete. 2024-05-03T18:14:27.986 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:27 smithi033 ceph-mon[28330]: from='osd.0 [v2:172.21.15.33:6802/637686669,v1:172.21.15.33:6803/637686669]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-03T18:14:27.986 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:27 smithi033 ceph-mon[28330]: osdmap e6: 1 total, 0 up, 1 in 2024-05-03T18:14:27.986 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:27 smithi033 ceph-mon[28330]: from='osd.0 [v2:172.21.15.33:6802/637686669,v1:172.21.15.33:6803/637686669]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-05-03T18:14:27.986 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:27 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-03T18:14:27.986 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:27 smithi033 ceph-mon[28330]: pgmap v60: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:14:27.986 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:27 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:14:27.986 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:27 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:14:27.986 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:27 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:14:28.041 DEBUG:teuthology.orchestra.run.smithi033:osd.0> sudo journalctl -f -n 0 -u ceph-5703bba4-0978-11ef-bc97-c7b262605968@osd.0.service 2024-05-03T18:14:28.045 INFO:tasks.cephadm:Deploying osd.1 on smithi099 with /dev/vg_nvme/lv_4... 2024-05-03T18:14:28.045 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-03T18:14:28.126 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:27 smithi099 ceph-mon[31658]: from='osd.0 [v2:172.21.15.33:6802/637686669,v1:172.21.15.33:6803/637686669]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-03T18:14:28.126 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:27 smithi099 ceph-mon[31658]: osdmap e6: 1 total, 0 up, 1 in 2024-05-03T18:14:28.126 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:27 smithi099 ceph-mon[31658]: from='osd.0 [v2:172.21.15.33:6802/637686669,v1:172.21.15.33:6803/637686669]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-05-03T18:14:28.127 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:27 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-03T18:14:28.127 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:27 smithi099 ceph-mon[31658]: pgmap v60: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:14:28.127 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:27 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:14:28.127 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:27 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:14:28.127 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:27 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:14:28.131 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:27 smithi150 ceph-mon[32492]: from='osd.0 [v2:172.21.15.33:6802/637686669,v1:172.21.15.33:6803/637686669]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-03T18:14:28.131 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:27 smithi150 ceph-mon[32492]: osdmap e6: 1 total, 0 up, 1 in 2024-05-03T18:14:28.131 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:27 smithi150 ceph-mon[32492]: from='osd.0 [v2:172.21.15.33:6802/637686669,v1:172.21.15.33:6803/637686669]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-05-03T18:14:28.131 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:27 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-03T18:14:28.131 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:27 smithi150 ceph-mon[32492]: pgmap v60: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:14:28.131 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:27 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:14:28.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:27 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:14:28.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:27 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:14:28.481 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.b/config 2024-05-03T18:14:28.946 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:28 smithi099 ceph-mon[31658]: from='osd.0 [v2:172.21.15.33:6802/637686669,v1:172.21.15.33:6803/637686669]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-05-03T18:14:28.946 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:28 smithi099 ceph-mon[31658]: osdmap e7: 1 total, 0 up, 1 in 2024-05-03T18:14:28.946 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:28 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-03T18:14:28.946 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:28 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-03T18:14:29.131 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:28 smithi150 ceph-mon[32492]: from='osd.0 [v2:172.21.15.33:6802/637686669,v1:172.21.15.33:6803/637686669]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-05-03T18:14:29.131 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:28 smithi150 ceph-mon[32492]: osdmap e7: 1 total, 0 up, 1 in 2024-05-03T18:14:29.131 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:28 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-03T18:14:29.131 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:28 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-03T18:14:29.233 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:28 smithi033 ceph-mon[28330]: from='osd.0 [v2:172.21.15.33:6802/637686669,v1:172.21.15.33:6803/637686669]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-05-03T18:14:29.233 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:28 smithi033 ceph-mon[28330]: osdmap e7: 1 total, 0 up, 1 in 2024-05-03T18:14:29.233 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:28 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-03T18:14:29.233 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:28 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-03T18:14:29.943 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:29 smithi033 ceph-mon[28330]: purged_snaps scrub starts 2024-05-03T18:14:29.943 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:29 smithi033 ceph-mon[28330]: purged_snaps scrub ok 2024-05-03T18:14:29.943 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:29 smithi033 ceph-mon[28330]: osd.0 [v2:172.21.15.33:6802/637686669,v1:172.21.15.33:6803/637686669] boot 2024-05-03T18:14:29.943 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:29 smithi033 ceph-mon[28330]: osdmap e8: 1 total, 1 up, 1 in 2024-05-03T18:14:29.943 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:29 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-03T18:14:29.943 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:29 smithi033 ceph-mon[28330]: pgmap v63: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:14:30.119 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:29 smithi099 ceph-mon[31658]: purged_snaps scrub starts 2024-05-03T18:14:30.119 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:29 smithi099 ceph-mon[31658]: purged_snaps scrub ok 2024-05-03T18:14:30.120 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:29 smithi099 ceph-mon[31658]: osd.0 [v2:172.21.15.33:6802/637686669,v1:172.21.15.33:6803/637686669] boot 2024-05-03T18:14:30.120 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:29 smithi099 ceph-mon[31658]: osdmap e8: 1 total, 1 up, 1 in 2024-05-03T18:14:30.120 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:29 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-03T18:14:30.120 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:29 smithi099 ceph-mon[31658]: pgmap v63: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:14:30.131 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:29 smithi150 ceph-mon[32492]: purged_snaps scrub starts 2024-05-03T18:14:30.131 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:29 smithi150 ceph-mon[32492]: purged_snaps scrub ok 2024-05-03T18:14:30.131 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:29 smithi150 ceph-mon[32492]: osd.0 [v2:172.21.15.33:6802/637686669,v1:172.21.15.33:6803/637686669] boot 2024-05-03T18:14:30.131 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:29 smithi150 ceph-mon[32492]: osdmap e8: 1 total, 1 up, 1 in 2024-05-03T18:14:30.131 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:29 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-03T18:14:30.131 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:29 smithi150 ceph-mon[32492]: pgmap v63: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:14:31.131 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:30 smithi150 ceph-mon[32492]: osdmap e9: 1 total, 1 up, 1 in 2024-05-03T18:14:31.233 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:30 smithi033 ceph-mon[28330]: osdmap e9: 1 total, 1 up, 1 in 2024-05-03T18:14:31.294 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:30 smithi099 ceph-mon[31658]: osdmap e9: 1 total, 1 up, 1 in 2024-05-03T18:14:31.379 INFO:teuthology.orchestra.run.smithi099.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-03T18:14:31.379 INFO:teuthology.orchestra.run.smithi099.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-03T18:14:31.379 INFO:teuthology.orchestra.run.smithi099.stderr: stderr: 10+0 records in 2024-05-03T18:14:31.379 INFO:teuthology.orchestra.run.smithi099.stderr:10+0 records out 2024-05-03T18:14:31.379 INFO:teuthology.orchestra.run.smithi099.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0355352 s, 295 MB/s 2024-05-03T18:14:31.379 INFO:teuthology.orchestra.run.smithi099.stderr:--> Zapping successful for: 2024-05-03T18:14:32.044 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:31 smithi099 ceph-mon[31658]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-03T18:14:32.131 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:31 smithi150 ceph-mon[32492]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-03T18:14:32.218 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi099:vg_nvme/lv_4 2024-05-03T18:14:32.232 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:31 smithi033 ceph-mon[28330]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-03T18:14:32.647 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.b/config 2024-05-03T18:14:34.133 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:33 smithi099 ceph-mon[31658]: Detected new or changed devices on smithi033 2024-05-03T18:14:34.134 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:33 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:14:34.134 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:33 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:14:34.134 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:33 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:14:34.134 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:33 smithi099 ceph-mon[31658]: Adjusting osd_memory_target on smithi033 to 17094M 2024-05-03T18:14:34.134 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:33 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:14:34.134 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:33 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:14:34.134 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:33 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:14:34.134 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:33 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:14:34.134 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:33 smithi099 ceph-mon[31658]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-03T18:14:34.233 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:33 smithi033 ceph-mon[28330]: Detected new or changed devices on smithi033 2024-05-03T18:14:34.233 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:33 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:14:34.233 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:33 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:14:34.233 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:33 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:14:34.233 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:33 smithi033 ceph-mon[28330]: Adjusting osd_memory_target on smithi033 to 17094M 2024-05-03T18:14:34.233 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:33 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:14:34.233 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:33 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:14:34.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:33 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:14:34.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:33 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:14:34.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:33 smithi033 ceph-mon[28330]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-03T18:14:34.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:33 smithi150 ceph-mon[32492]: Detected new or changed devices on smithi033 2024-05-03T18:14:34.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:33 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:14:34.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:33 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:14:34.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:33 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:14:34.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:33 smithi150 ceph-mon[32492]: Adjusting osd_memory_target on smithi033 to 17094M 2024-05-03T18:14:34.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:33 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:14:34.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:33 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:14:34.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:33 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:14:34.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:33 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:14:34.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:33 smithi150 ceph-mon[32492]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-03T18:14:34.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:14:34.651+0000 7fafb1200700 1 -- 172.21.15.99:0/2693355019 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fafac075f00 msgr2=0x7fafac076380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:14:34.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:14:34.651+0000 7fafb1200700 1 --2- 172.21.15.99:0/2693355019 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fafac075f00 0x7fafac076380 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7faf98009990 tx=0x7faf98009ca0 comp rx=0 tx=0).stop 2024-05-03T18:14:34.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:14:34.651+0000 7fafb1200700 1 -- 172.21.15.99:0/2693355019 shutdown_connections 2024-05-03T18:14:34.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:14:34.651+0000 7fafb1200700 1 --2- 172.21.15.99:0/2693355019 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fafac10b3e0 0x7fafac10d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:14:34.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:14:34.651+0000 7fafb1200700 1 --2- 172.21.15.99:0/2693355019 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fafac0774d0 0x7fafac075930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:14:34.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:14:34.651+0000 7fafb1200700 1 --2- 172.21.15.99:0/2693355019 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fafac075f00 0x7fafac076380 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:14:34.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:14:34.651+0000 7fafb1200700 1 -- 172.21.15.99:0/2693355019 >> 172.21.15.99:0/2693355019 conn(0x7fafac0feab0 msgr2=0x7fafac100ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:14:34.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:14:34.651+0000 7fafb1200700 1 -- 172.21.15.99:0/2693355019 shutdown_connections 2024-05-03T18:14:34.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:14:34.651+0000 7fafb1200700 1 -- 172.21.15.99:0/2693355019 wait complete. 2024-05-03T18:14:34.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:14:34.652+0000 7fafb1200700 1 Processor -- start 2024-05-03T18:14:34.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:14:34.652+0000 7fafb1200700 1 -- start start 2024-05-03T18:14:34.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:14:34.653+0000 7fafb1200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fafac075f00 0x7fafac19b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:14:34.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:14:34.653+0000 7fafb1200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fafac0774d0 0x7fafac19bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:14:34.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:14:34.653+0000 7fafb1200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fafac10b3e0 0x7fafac1a2d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:14:34.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:14:34.653+0000 7fafb1200700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fafac1110a0 con 0x7fafac10b3e0 2024-05-03T18:14:34.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:14:34.653+0000 7fafb1200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fafac110f20 con 0x7fafac0774d0 2024-05-03T18:14:34.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:14:34.653+0000 7fafb1200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fafac111220 con 0x7fafac075f00 2024-05-03T18:14:34.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:14:34.653+0000 7fafa9400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fafac0774d0 0x7fafac19bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:14:34.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:14:34.653+0000 7fafa9400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fafac0774d0 0x7fafac19bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44250/0 (socket says 172.21.15.99:44250) 2024-05-03T18:14:34.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:14:34.653+0000 7fafa9400700 1 -- 172.21.15.99:0/1825019602 learned_addr learned my addr 172.21.15.99:0/1825019602 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-03T18:14:34.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:14:34.653+0000 7fafa9e00700 1 --2- 172.21.15.99:0/1825019602 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fafac075f00 0x7fafac19b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:14:34.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:14:34.654+0000 7fafaa800700 1 --2- 172.21.15.99:0/1825019602 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fafac10b3e0 0x7fafac1a2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:14:34.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:14:34.654+0000 7fafa9400700 1 -- 172.21.15.99:0/1825019602 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fafac075f00 msgr2=0x7fafac19b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:14:34.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:14:34.654+0000 7fafa9400700 1 --2- 172.21.15.99:0/1825019602 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fafac075f00 0x7fafac19b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:14:34.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:14:34.654+0000 7fafa9400700 1 -- 172.21.15.99:0/1825019602 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fafac10b3e0 msgr2=0x7fafac1a2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:14:34.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:14:34.654+0000 7fafa9400700 1 --2- 172.21.15.99:0/1825019602 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fafac10b3e0 0x7fafac1a2d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:14:34.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:14:34.654+0000 7fafa9400700 1 -- 172.21.15.99:0/1825019602 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafac1a3420 con 0x7fafac0774d0 2024-05-03T18:14:34.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:14:34.654+0000 7fafa9400700 1 --2- 172.21.15.99:0/1825019602 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fafac0774d0 0x7fafac19bd00 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7faf98003820 tx=0x7faf980057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:14:34.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:14:34.655+0000 7fafa2a00700 1 -- 172.21.15.99:0/1825019602 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf98026070 con 0x7fafac0774d0 2024-05-03T18:14:34.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:14:34.655+0000 7fafa2a00700 1 -- 172.21.15.99:0/1825019602 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faf98003fe0 con 0x7fafac0774d0 2024-05-03T18:14:34.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:14:34.655+0000 7fafa2a00700 1 -- 172.21.15.99:0/1825019602 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf9800fd20 con 0x7fafac0774d0 2024-05-03T18:14:34.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:14:34.655+0000 7fafb1200700 1 -- 172.21.15.99:0/1825019602 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafac1a36b0 con 0x7fafac0774d0 2024-05-03T18:14:34.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:14:34.655+0000 7fafb1200700 1 -- 172.21.15.99:0/1825019602 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafac1a3ab0 con 0x7fafac0774d0 2024-05-03T18:14:34.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:14:34.656+0000 7fafb1200700 1 -- 172.21.15.99:0/1825019602 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafac06a190 con 0x7fafac0774d0 2024-05-03T18:14:34.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:14:34.658+0000 7fafa2a00700 1 -- 172.21.15.99:0/1825019602 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 15) ==== 149596+0+0 (secure 0 0 0) 0x7faf98029070 con 0x7fafac0774d0 2024-05-03T18:14:34.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:14:34.660+0000 7fafa2a00700 1 --2- 172.21.15.99:0/1825019602 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7faf900b2870 0x7faf900b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:14:34.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:14:34.660+0000 7fafa2a00700 1 -- 172.21.15.99:0/1825019602 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7faf980e1b80 con 0x7fafac0774d0 2024-05-03T18:14:34.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:14:34.661+0000 7fafa9e00700 1 --2- 172.21.15.99:0/1825019602 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7faf900b2870 0x7faf900b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:14:34.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:14:34.662+0000 7fafa9e00700 1 --2- 172.21.15.99:0/1825019602 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7faf900b2870 0x7faf900b4d30 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7faf9c005950 tx=0x7faf9c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:14:34.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:14:34.663+0000 7fafa2a00700 1 -- 172.21.15.99:0/1825019602 <== mon.2 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7faf98013070 con 0x7fafac0774d0 2024-05-03T18:14:34.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:14:34.907+0000 7fafb1200700 1 -- 172.21.15.99:0/1825019602 --> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fafac0638e0 con 0x7faf900b2870 2024-05-03T18:14:35.233 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:34 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T18:14:35.233 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:34 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T18:14:35.233 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:34 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:14:35.294 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:34 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T18:14:35.294 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:34 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T18:14:35.294 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:34 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:14:35.381 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:34 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T18:14:35.381 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:34 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T18:14:35.381 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:34 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:14:36.114 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:35 smithi099 ceph-mon[31658]: from='client.34139 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:14:36.114 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:35 smithi099 ceph-mon[31658]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-03T18:14:36.233 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:35 smithi033 ceph-mon[28330]: from='client.34139 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:14:36.233 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:35 smithi033 ceph-mon[28330]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-03T18:14:36.381 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:35 smithi150 ceph-mon[32492]: from='client.34139 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:14:36.381 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:35 smithi150 ceph-mon[32492]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-03T18:14:38.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:38 smithi033 ceph-mon[28330]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-03T18:14:38.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:38 smithi099 ceph-mon[31658]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-03T18:14:38.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:38 smithi150 ceph-mon[32492]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-03T18:14:39.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:39 smithi033 ceph-mon[28330]: from='client.? 172.21.15.99:0/3413653510' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cf78cad3-fc24-43f4-8784-21cabea7277c"}]: dispatch 2024-05-03T18:14:39.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:39 smithi033 ceph-mon[28330]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cf78cad3-fc24-43f4-8784-21cabea7277c"}]: dispatch 2024-05-03T18:14:39.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:39 smithi033 ceph-mon[28330]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cf78cad3-fc24-43f4-8784-21cabea7277c"}]': finished 2024-05-03T18:14:39.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:39 smithi033 ceph-mon[28330]: osdmap e10: 2 total, 1 up, 2 in 2024-05-03T18:14:39.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:39 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-03T18:14:39.544 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:39 smithi099 ceph-mon[31658]: from='client.? 172.21.15.99:0/3413653510' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cf78cad3-fc24-43f4-8784-21cabea7277c"}]: dispatch 2024-05-03T18:14:39.544 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:39 smithi099 ceph-mon[31658]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cf78cad3-fc24-43f4-8784-21cabea7277c"}]: dispatch 2024-05-03T18:14:39.544 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:39 smithi099 ceph-mon[31658]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cf78cad3-fc24-43f4-8784-21cabea7277c"}]': finished 2024-05-03T18:14:39.544 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:39 smithi099 ceph-mon[31658]: osdmap e10: 2 total, 1 up, 2 in 2024-05-03T18:14:39.544 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:39 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-03T18:14:39.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:39 smithi150 ceph-mon[32492]: from='client.? 172.21.15.99:0/3413653510' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cf78cad3-fc24-43f4-8784-21cabea7277c"}]: dispatch 2024-05-03T18:14:39.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:39 smithi150 ceph-mon[32492]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cf78cad3-fc24-43f4-8784-21cabea7277c"}]: dispatch 2024-05-03T18:14:39.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:39 smithi150 ceph-mon[32492]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cf78cad3-fc24-43f4-8784-21cabea7277c"}]': finished 2024-05-03T18:14:39.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:39 smithi150 ceph-mon[32492]: osdmap e10: 2 total, 1 up, 2 in 2024-05-03T18:14:39.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:39 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-03T18:14:40.544 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:40 smithi099 ceph-mon[31658]: pgmap v70: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-03T18:14:40.544 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:40 smithi099 ceph-mon[31658]: from='client.? 172.21.15.99:0/3693387985' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T18:14:40.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:40 smithi150 ceph-mon[32492]: pgmap v70: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-03T18:14:40.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:40 smithi150 ceph-mon[32492]: from='client.? 172.21.15.99:0/3693387985' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T18:14:40.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:40 smithi033 ceph-mon[28330]: pgmap v70: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-03T18:14:40.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:40 smithi033 ceph-mon[28330]: from='client.? 172.21.15.99:0/3693387985' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T18:14:42.294 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:42 smithi099 ceph-mon[31658]: pgmap v71: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-03T18:14:42.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:42 smithi150 ceph-mon[32492]: pgmap v71: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-03T18:14:42.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:42 smithi033 ceph-mon[28330]: pgmap v71: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-03T18:14:44.544 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:44 smithi099 ceph-mon[31658]: pgmap v72: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-03T18:14:44.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:44 smithi150 ceph-mon[32492]: pgmap v72: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-03T18:14:44.732 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:44 smithi033 ceph-mon[28330]: pgmap v72: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-03T18:14:46.544 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:46 smithi099 ceph-mon[31658]: pgmap v73: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-03T18:14:46.623 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:46 smithi033 ceph-mon[28330]: pgmap v73: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-03T18:14:46.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:46 smithi150 ceph-mon[32492]: pgmap v73: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-03T18:14:48.544 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:48 smithi099 ceph-mon[31658]: pgmap v74: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-03T18:14:48.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:48 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-03T18:14:48.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:48 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:14:48.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:48 smithi099 ceph-mon[31658]: Deploying daemon osd.1 on smithi099 2024-05-03T18:14:48.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:48 smithi150 ceph-mon[32492]: pgmap v74: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-03T18:14:48.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:48 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-03T18:14:48.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:48 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:14:48.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:48 smithi150 ceph-mon[32492]: Deploying daemon osd.1 on smithi099 2024-05-03T18:14:48.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:48 smithi033 ceph-mon[28330]: pgmap v74: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-03T18:14:48.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:48 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-03T18:14:48.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:48 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:14:48.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:48 smithi033 ceph-mon[28330]: Deploying daemon osd.1 on smithi099 2024-05-03T18:14:50.544 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:50 smithi099 ceph-mon[31658]: pgmap v75: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-03T18:14:50.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:50 smithi150 ceph-mon[32492]: pgmap v75: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-03T18:14:50.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:50 smithi033 ceph-mon[28330]: pgmap v75: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-03T18:14:52.346 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:52 smithi099 ceph-mon[31658]: pgmap v76: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-03T18:14:52.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:52 smithi150 ceph-mon[32492]: pgmap v76: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-03T18:14:52.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:52 smithi033 ceph-mon[28330]: pgmap v76: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-03T18:14:54.544 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:54 smithi099 ceph-mon[31658]: pgmap v77: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-03T18:14:54.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:54 smithi150 ceph-mon[32492]: pgmap v77: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-03T18:14:54.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:54 smithi033 ceph-mon[28330]: pgmap v77: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-03T18:14:56.794 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:56 smithi099 ceph-mon[31658]: pgmap v78: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-03T18:14:56.794 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:56 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:14:56.794 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:56 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:14:56.794 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:56 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:14:56.881 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:56 smithi150 ceph-mon[32492]: pgmap v78: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-03T18:14:56.881 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:56 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:14:56.881 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:56 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:14:56.881 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:56 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:14:56.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:56 smithi033 ceph-mon[28330]: pgmap v78: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-03T18:14:56.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:56 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:14:56.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:56 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:14:56.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:56 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:14:58.710 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:58 smithi099 ceph-mon[31658]: pgmap v79: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-03T18:14:58.881 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:58 smithi150 ceph-mon[32492]: pgmap v79: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-03T18:14:58.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:58 smithi033 ceph-mon[28330]: pgmap v79: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-03T18:15:00.214 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:59 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:15:00.214 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:59 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:15:00.214 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:59 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:15:00.214 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:59 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:15:00.214 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:59 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:15:00.214 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:59 smithi099 ceph-mon[31658]: pgmap v80: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-03T18:15:00.215 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:59 smithi099 ceph-mon[31658]: from='osd.1 [v2:172.21.15.99:6800/1536122841,v1:172.21.15.99:6801/1536122841]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-03T18:15:00.215 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:14:59 smithi099 ceph-mon[31658]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-03T18:15:00.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:15:00.321+0000 7fafa2a00700 1 -- 172.21.15.99:0/1825019602 <== mgr.14152 v2:172.21.15.33:6800/3642895501 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fafac0638e0 con 0x7faf900b2870 2024-05-03T18:15:00.321 INFO:teuthology.orchestra.run.smithi099.stdout:Created osd(s) 1 on host 'smithi099' 2024-05-03T18:15:00.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:15:00.324+0000 7fafb1200700 1 -- 172.21.15.99:0/1825019602 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7faf900b2870 msgr2=0x7faf900b4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:15:00.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:15:00.324+0000 7fafb1200700 1 --2- 172.21.15.99:0/1825019602 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7faf900b2870 0x7faf900b4d30 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7faf9c005950 tx=0x7faf9c00b410 comp rx=0 tx=0).stop 2024-05-03T18:15:00.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:15:00.324+0000 7fafb1200700 1 -- 172.21.15.99:0/1825019602 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fafac0774d0 msgr2=0x7fafac19bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:15:00.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:15:00.324+0000 7fafb1200700 1 --2- 172.21.15.99:0/1825019602 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fafac0774d0 0x7fafac19bd00 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7faf98003820 tx=0x7faf980057c0 comp rx=0 tx=0).stop 2024-05-03T18:15:00.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:15:00.325+0000 7fafb1200700 1 -- 172.21.15.99:0/1825019602 shutdown_connections 2024-05-03T18:15:00.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:15:00.325+0000 7fafb1200700 1 --2- 172.21.15.99:0/1825019602 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fafac10b3e0 0x7fafac1a2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:15:00.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:15:00.325+0000 7fafb1200700 1 --2- 172.21.15.99:0/1825019602 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fafac075f00 0x7fafac19b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:15:00.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:15:00.325+0000 7fafb1200700 1 --2- 172.21.15.99:0/1825019602 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7faf900b2870 0x7faf900b4d30 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:15:00.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:15:00.325+0000 7fafb1200700 1 --2- 172.21.15.99:0/1825019602 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fafac0774d0 0x7fafac19bd00 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:15:00.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:15:00.325+0000 7fafb1200700 1 -- 172.21.15.99:0/1825019602 >> 172.21.15.99:0/1825019602 conn(0x7fafac0feab0 msgr2=0x7fafac1072e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:15:00.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:15:00.325+0000 7fafb1200700 1 -- 172.21.15.99:0/1825019602 shutdown_connections 2024-05-03T18:15:00.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:15:00.325+0000 7fafb1200700 1 -- 172.21.15.99:0/1825019602 wait complete. 2024-05-03T18:15:00.381 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:59 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:15:00.381 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:59 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:15:00.381 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:59 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:15:00.381 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:59 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:15:00.381 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:59 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:15:00.381 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:59 smithi150 ceph-mon[32492]: pgmap v80: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-03T18:15:00.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:59 smithi150 ceph-mon[32492]: from='osd.1 [v2:172.21.15.99:6800/1536122841,v1:172.21.15.99:6801/1536122841]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-03T18:15:00.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:14:59 smithi150 ceph-mon[32492]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-03T18:15:00.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:59 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:15:00.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:59 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:15:00.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:59 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:15:00.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:59 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:15:00.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:59 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:15:00.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:59 smithi033 ceph-mon[28330]: pgmap v80: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-03T18:15:00.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:59 smithi033 ceph-mon[28330]: from='osd.1 [v2:172.21.15.99:6800/1536122841,v1:172.21.15.99:6801/1536122841]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-03T18:15:00.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:14:59 smithi033 ceph-mon[28330]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-03T18:15:01.114 DEBUG:teuthology.orchestra.run.smithi099:osd.1> sudo journalctl -f -n 0 -u ceph-5703bba4-0978-11ef-bc97-c7b262605968@osd.1.service 2024-05-03T18:15:01.119 INFO:tasks.cephadm:Deploying osd.2 on smithi150 with /dev/vg_nvme/lv_4... 2024-05-03T18:15:01.119 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-03T18:15:01.294 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:01 smithi099 ceph-mon[31658]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-03T18:15:01.294 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:01 smithi099 ceph-mon[31658]: osdmap e11: 2 total, 1 up, 2 in 2024-05-03T18:15:01.294 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:01 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-03T18:15:01.294 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:01 smithi099 ceph-mon[31658]: from='osd.1 [v2:172.21.15.99:6800/1536122841,v1:172.21.15.99:6801/1536122841]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-05-03T18:15:01.294 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:01 smithi099 ceph-mon[31658]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-05-03T18:15:01.294 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:01 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:15:01.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:01 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:15:01.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:01 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:15:01.381 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:01 smithi150 ceph-mon[32492]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-03T18:15:01.381 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:01 smithi150 ceph-mon[32492]: osdmap e11: 2 total, 1 up, 2 in 2024-05-03T18:15:01.381 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:01 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-03T18:15:01.381 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:01 smithi150 ceph-mon[32492]: from='osd.1 [v2:172.21.15.99:6800/1536122841,v1:172.21.15.99:6801/1536122841]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-05-03T18:15:01.381 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:01 smithi150 ceph-mon[32492]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-05-03T18:15:01.381 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:01 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:15:01.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:01 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:15:01.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:01 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:15:01.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:01 smithi033 ceph-mon[28330]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-03T18:15:01.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:01 smithi033 ceph-mon[28330]: osdmap e11: 2 total, 1 up, 2 in 2024-05-03T18:15:01.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:01 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-03T18:15:01.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:01 smithi033 ceph-mon[28330]: from='osd.1 [v2:172.21.15.99:6800/1536122841,v1:172.21.15.99:6801/1536122841]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-05-03T18:15:01.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:01 smithi033 ceph-mon[28330]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-05-03T18:15:01.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:01 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:15:01.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:01 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:15:01.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:01 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:15:01.552 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.c/config 2024-05-03T18:15:02.205 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:02 smithi099 ceph-mon[31658]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-05-03T18:15:02.205 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:02 smithi099 ceph-mon[31658]: osdmap e12: 2 total, 1 up, 2 in 2024-05-03T18:15:02.205 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:02 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-03T18:15:02.205 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:02 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-03T18:15:02.205 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:02 smithi099 ceph-mon[31658]: pgmap v83: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-03T18:15:02.205 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:02 smithi099 ceph-mon[31658]: from='osd.1 ' entity='osd.1' 2024-05-03T18:15:02.206 INFO:journalctl@ceph.osd.1.smithi099.stdout:May 03 18:15:01 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-1[36254]: 2024-05-03T18:15:01.888+0000 7fe34f800700 -1 osd.1 0 waiting for initial osdmap 2024-05-03T18:15:02.206 INFO:journalctl@ceph.osd.1.smithi099.stdout:May 03 18:15:01 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-1[36254]: 2024-05-03T18:15:01.913+0000 7fe345600700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-03T18:15:02.381 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:02 smithi150 ceph-mon[32492]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-05-03T18:15:02.381 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:02 smithi150 ceph-mon[32492]: osdmap e12: 2 total, 1 up, 2 in 2024-05-03T18:15:02.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:02 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-03T18:15:02.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:02 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-03T18:15:02.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:02 smithi150 ceph-mon[32492]: pgmap v83: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-03T18:15:02.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:02 smithi150 ceph-mon[32492]: from='osd.1 ' entity='osd.1' 2024-05-03T18:15:02.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:02 smithi033 ceph-mon[28330]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-05-03T18:15:02.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:02 smithi033 ceph-mon[28330]: osdmap e12: 2 total, 1 up, 2 in 2024-05-03T18:15:02.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:02 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-03T18:15:02.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:02 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-03T18:15:02.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:02 smithi033 ceph-mon[28330]: pgmap v83: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-03T18:15:02.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:02 smithi033 ceph-mon[28330]: from='osd.1 ' entity='osd.1' 2024-05-03T18:15:03.127 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:03 smithi099 ceph-mon[31658]: purged_snaps scrub starts 2024-05-03T18:15:03.127 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:03 smithi099 ceph-mon[31658]: purged_snaps scrub ok 2024-05-03T18:15:03.127 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:03 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-03T18:15:03.127 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:03 smithi099 ceph-mon[31658]: osd.1 [v2:172.21.15.99:6800/1536122841,v1:172.21.15.99:6801/1536122841] boot 2024-05-03T18:15:03.127 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:03 smithi099 ceph-mon[31658]: osdmap e13: 2 total, 2 up, 2 in 2024-05-03T18:15:03.127 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:03 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-03T18:15:03.372 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:03 smithi150 ceph-mon[32492]: purged_snaps scrub starts 2024-05-03T18:15:03.372 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:03 smithi150 ceph-mon[32492]: purged_snaps scrub ok 2024-05-03T18:15:03.372 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:03 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-03T18:15:03.373 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:03 smithi150 ceph-mon[32492]: osd.1 [v2:172.21.15.99:6800/1536122841,v1:172.21.15.99:6801/1536122841] boot 2024-05-03T18:15:03.373 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:03 smithi150 ceph-mon[32492]: osdmap e13: 2 total, 2 up, 2 in 2024-05-03T18:15:03.373 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:03 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-03T18:15:03.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:03 smithi033 ceph-mon[28330]: purged_snaps scrub starts 2024-05-03T18:15:03.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:03 smithi033 ceph-mon[28330]: purged_snaps scrub ok 2024-05-03T18:15:03.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:03 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-03T18:15:03.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:03 smithi033 ceph-mon[28330]: osd.1 [v2:172.21.15.99:6800/1536122841,v1:172.21.15.99:6801/1536122841] boot 2024-05-03T18:15:03.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:03 smithi033 ceph-mon[28330]: osdmap e13: 2 total, 2 up, 2 in 2024-05-03T18:15:03.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:03 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-03T18:15:04.275 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-03T18:15:04.276 INFO:teuthology.orchestra.run.smithi150.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-03T18:15:04.276 INFO:teuthology.orchestra.run.smithi150.stderr: stderr: 10+0 records in 2024-05-03T18:15:04.276 INFO:teuthology.orchestra.run.smithi150.stderr:10+0 records out 2024-05-03T18:15:04.276 INFO:teuthology.orchestra.run.smithi150.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0365067 s, 287 MB/s 2024-05-03T18:15:04.276 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping successful for: 2024-05-03T18:15:04.319 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:04 smithi150 ceph-mon[32492]: pgmap v85: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-03T18:15:04.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:04 smithi033 ceph-mon[28330]: pgmap v85: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-03T18:15:04.544 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:04 smithi099 ceph-mon[31658]: pgmap v85: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-03T18:15:05.073 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi150:vg_nvme/lv_4 2024-05-03T18:15:05.294 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:05 smithi099 ceph-mon[31658]: osdmap e14: 2 total, 2 up, 2 in 2024-05-03T18:15:05.381 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:05 smithi150 ceph-mon[32492]: osdmap e14: 2 total, 2 up, 2 in 2024-05-03T18:15:05.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:05 smithi033 ceph-mon[28330]: osdmap e14: 2 total, 2 up, 2 in 2024-05-03T18:15:05.492 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.c/config 2024-05-03T18:15:06.381 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:06 smithi150 ceph-mon[32492]: pgmap v87: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-03T18:15:06.381 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:06 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:15:06.381 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:06 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:15:06.381 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:06 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:15:06.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:06 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:15:06.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:06 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:15:06.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:06 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:15:06.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:06 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:15:06.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:06 smithi033 ceph-mon[28330]: pgmap v87: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-03T18:15:06.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:06 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:15:06.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:06 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:15:06.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:06 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:15:06.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:06 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:15:06.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:06 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:15:06.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:06 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:15:06.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:06 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:15:06.544 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:06 smithi099 ceph-mon[31658]: pgmap v87: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-03T18:15:06.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:06 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:15:06.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:06 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:15:06.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:06 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:15:06.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:06 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:15:06.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:06 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:15:06.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:06 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:15:06.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:06 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:15:07.381 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:07 smithi150 ceph-mon[32492]: Detected new or changed devices on smithi099 2024-05-03T18:15:07.381 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:07 smithi150 ceph-mon[32492]: Adjusting osd_memory_target on smithi099 to 17089M 2024-05-03T18:15:07.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:07 smithi033 ceph-mon[28330]: Detected new or changed devices on smithi099 2024-05-03T18:15:07.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:07 smithi033 ceph-mon[28330]: Adjusting osd_memory_target on smithi099 to 17089M 2024-05-03T18:15:07.544 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:07 smithi099 ceph-mon[31658]: Detected new or changed devices on smithi099 2024-05-03T18:15:07.544 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:07 smithi099 ceph-mon[31658]: Adjusting osd_memory_target on smithi099 to 17089M 2024-05-03T18:15:07.603 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:15:07.603+0000 7fb96b600700 1 -- 172.21.15.150:0/927535798 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb9641092a0 msgr2=0x7fb96410b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:15:07.604 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:15:07.603+0000 7fb96b600700 1 --2- 172.21.15.150:0/927535798 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb9641092a0 0x7fb96410b690 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb95c00b0a0 tx=0x7fb95c00b3b0 comp rx=0 tx=0).stop 2024-05-03T18:15:07.604 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:15:07.604+0000 7fb96b600700 1 -- 172.21.15.150:0/927535798 shutdown_connections 2024-05-03T18:15:07.604 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:15:07.604+0000 7fb96b600700 1 --2- 172.21.15.150:0/927535798 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb9641092a0 0x7fb96410b690 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:15:07.604 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:15:07.604+0000 7fb96b600700 1 --2- 172.21.15.150:0/927535798 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb964104040 0x7fb964106430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:15:07.604 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:15:07.604+0000 7fb96b600700 1 --2- 172.21.15.150:0/927535798 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb964106970 0x7fb964108d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:15:07.604 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:15:07.604+0000 7fb96b600700 1 -- 172.21.15.150:0/927535798 >> 172.21.15.150:0/927535798 conn(0x7fb9640fd990 msgr2=0x7fb9640ffdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:15:07.604 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:15:07.604+0000 7fb96b600700 1 -- 172.21.15.150:0/927535798 shutdown_connections 2024-05-03T18:15:07.604 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:15:07.604+0000 7fb96b600700 1 -- 172.21.15.150:0/927535798 wait complete. 2024-05-03T18:15:07.605 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:15:07.605+0000 7fb96b600700 1 Processor -- start 2024-05-03T18:15:07.606 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:15:07.605+0000 7fb96b600700 1 -- start start 2024-05-03T18:15:07.606 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:15:07.605+0000 7fb96b600700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb964104040 0x7fb964199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:15:07.606 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:15:07.605+0000 7fb96b600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb964106970 0x7fb9641999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:15:07.606 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:15:07.605+0000 7fb96b600700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9641092a0 0x7fb9641a0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:15:07.606 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:15:07.605+0000 7fb96b600700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb96410ee40 con 0x7fb9641092a0 2024-05-03T18:15:07.606 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:15:07.605+0000 7fb96b600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb96410ecc0 con 0x7fb964106970 2024-05-03T18:15:07.606 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:15:07.605+0000 7fb96b600700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb96410efc0 con 0x7fb964104040 2024-05-03T18:15:07.606 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:15:07.606+0000 7fb968c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb964104040 0x7fb964199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:15:07.606 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:15:07.606+0000 7fb968c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb964104040 0x7fb964199480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:37134/0 (socket says 172.21.15.150:37134) 2024-05-03T18:15:07.606 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:15:07.606+0000 7fb968c00700 1 -- 172.21.15.150:0/2949954177 learned_addr learned my addr 172.21.15.150:0/2949954177 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-03T18:15:07.606 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:15:07.606+0000 7fb969600700 1 --2- 172.21.15.150:0/2949954177 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9641092a0 0x7fb9641a0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:15:07.606 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:15:07.606+0000 7fb968c00700 1 -- 172.21.15.150:0/2949954177 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb964106970 msgr2=0x7fb9641999c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:15:07.606 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:15:07.606+0000 7fb968c00700 1 --2- 172.21.15.150:0/2949954177 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb964106970 0x7fb9641999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:15:07.607 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:15:07.606+0000 7fb968c00700 1 -- 172.21.15.150:0/2949954177 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9641092a0 msgr2=0x7fb9641a0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:15:07.607 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:15:07.606+0000 7fb968c00700 1 --2- 172.21.15.150:0/2949954177 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9641092a0 0x7fb9641a0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:15:07.607 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:15:07.606+0000 7fb968c00700 1 -- 172.21.15.150:0/2949954177 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9641a10e0 con 0x7fb964104040 2024-05-03T18:15:07.607 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:15:07.606+0000 7fb969600700 1 --2- 172.21.15.150:0/2949954177 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9641092a0 0x7fb9641a0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:15:07.608 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:15:07.607+0000 7fb968c00700 1 --2- 172.21.15.150:0/2949954177 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb964104040 0x7fb964199480 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb95800c9e0 tx=0x7fb95800cda0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:15:07.608 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:15:07.607+0000 7fb961600700 1 -- 172.21.15.150:0/2949954177 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb958007ac0 con 0x7fb964104040 2024-05-03T18:15:07.608 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:15:07.607+0000 7fb961600700 1 -- 172.21.15.150:0/2949954177 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb958007c60 con 0x7fb964104040 2024-05-03T18:15:07.608 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:15:07.607+0000 7fb961600700 1 -- 172.21.15.150:0/2949954177 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb958018660 con 0x7fb964104040 2024-05-03T18:15:07.608 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:15:07.607+0000 7fb96b600700 1 -- 172.21.15.150:0/2949954177 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9641a13d0 con 0x7fb964104040 2024-05-03T18:15:07.608 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:15:07.607+0000 7fb96b600700 1 -- 172.21.15.150:0/2949954177 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9641a1960 con 0x7fb964104040 2024-05-03T18:15:07.608 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:15:07.608+0000 7fb96b600700 1 -- 172.21.15.150:0/2949954177 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb96406a190 con 0x7fb964104040 2024-05-03T18:15:07.610 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:15:07.611+0000 7fb961600700 1 -- 172.21.15.150:0/2949954177 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 15) ==== 149596+0+0 (secure 0 0 0) 0x7fb95800f450 con 0x7fb964104040 2024-05-03T18:15:07.611 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:15:07.612+0000 7fb961600700 1 --2- 172.21.15.150:0/2949954177 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fb94c0b2870 0x7fb94c0b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:15:07.611 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:15:07.612+0000 7fb961600700 1 -- 172.21.15.150:0/2949954177 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7fb9580e0b40 con 0x7fb964104040 2024-05-03T18:15:07.612 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:15:07.613+0000 7fb963e00700 1 --2- 172.21.15.150:0/2949954177 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fb94c0b2870 0x7fb94c0b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:15:07.613 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:15:07.614+0000 7fb963e00700 1 --2- 172.21.15.150:0/2949954177 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fb94c0b2870 0x7fb94c0b4d30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb950005950 tx=0x7fb9500058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:15:07.615 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:15:07.616+0000 7fb961600700 1 -- 172.21.15.150:0/2949954177 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fb958054d70 con 0x7fb964104040 2024-05-03T18:15:07.861 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:15:07.861+0000 7fb96b600700 1 -- 172.21.15.150:0/2949954177 --> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fb9640638e0 con 0x7fb94c0b2870 2024-05-03T18:15:08.131 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:08 smithi150 ceph-mon[32492]: pgmap v88: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-03T18:15:08.131 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:08 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T18:15:08.131 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:08 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T18:15:08.131 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:08 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:15:08.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:08 smithi033 ceph-mon[28330]: pgmap v88: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-03T18:15:08.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:08 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T18:15:08.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:08 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T18:15:08.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:08 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:15:08.544 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:08 smithi099 ceph-mon[31658]: pgmap v88: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-03T18:15:08.544 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:08 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T18:15:08.544 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:08 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T18:15:08.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:08 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:15:09.381 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:09 smithi150 ceph-mon[32492]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:15:09.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:09 smithi033 ceph-mon[28330]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:15:09.544 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:09 smithi099 ceph-mon[31658]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:15:10.381 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:10 smithi150 ceph-mon[32492]: pgmap v89: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-03T18:15:10.446 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:10 smithi099 ceph-mon[31658]: pgmap v89: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-03T18:15:10.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:10 smithi033 ceph-mon[28330]: pgmap v89: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-03T18:15:12.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:12 smithi099 ceph-mon[31658]: pgmap v90: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-03T18:15:12.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:12 smithi099 ceph-mon[31658]: from='client.? 172.21.15.150:0/3577145556' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "59f066d7-626b-410a-8d67-daca6fbbd3ce"}]: dispatch 2024-05-03T18:15:12.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:12 smithi099 ceph-mon[31658]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "59f066d7-626b-410a-8d67-daca6fbbd3ce"}]: dispatch 2024-05-03T18:15:12.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:12 smithi099 ceph-mon[31658]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "59f066d7-626b-410a-8d67-daca6fbbd3ce"}]': finished 2024-05-03T18:15:12.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:12 smithi099 ceph-mon[31658]: osdmap e15: 3 total, 2 up, 3 in 2024-05-03T18:15:12.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:12 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-03T18:15:12.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:12 smithi150 ceph-mon[32492]: pgmap v90: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-03T18:15:12.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:12 smithi150 ceph-mon[32492]: from='client.? 172.21.15.150:0/3577145556' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "59f066d7-626b-410a-8d67-daca6fbbd3ce"}]: dispatch 2024-05-03T18:15:12.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:12 smithi150 ceph-mon[32492]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "59f066d7-626b-410a-8d67-daca6fbbd3ce"}]: dispatch 2024-05-03T18:15:12.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:12 smithi150 ceph-mon[32492]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "59f066d7-626b-410a-8d67-daca6fbbd3ce"}]': finished 2024-05-03T18:15:12.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:12 smithi150 ceph-mon[32492]: osdmap e15: 3 total, 2 up, 3 in 2024-05-03T18:15:12.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:12 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-03T18:15:12.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:12 smithi033 ceph-mon[28330]: pgmap v90: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-03T18:15:12.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:12 smithi033 ceph-mon[28330]: from='client.? 172.21.15.150:0/3577145556' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "59f066d7-626b-410a-8d67-daca6fbbd3ce"}]: dispatch 2024-05-03T18:15:12.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:12 smithi033 ceph-mon[28330]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "59f066d7-626b-410a-8d67-daca6fbbd3ce"}]: dispatch 2024-05-03T18:15:12.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:12 smithi033 ceph-mon[28330]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "59f066d7-626b-410a-8d67-daca6fbbd3ce"}]': finished 2024-05-03T18:15:12.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:12 smithi033 ceph-mon[28330]: osdmap e15: 3 total, 2 up, 3 in 2024-05-03T18:15:12.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:12 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-03T18:15:13.544 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:13 smithi099 ceph-mon[31658]: from='client.? 172.21.15.150:0/333282703' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T18:15:13.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:13 smithi150 ceph-mon[32492]: from='client.? 172.21.15.150:0/333282703' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T18:15:13.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:13 smithi033 ceph-mon[28330]: from='client.? 172.21.15.150:0/333282703' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T18:15:14.544 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:14 smithi099 ceph-mon[31658]: pgmap v92: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-03T18:15:14.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:14 smithi150 ceph-mon[32492]: pgmap v92: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-03T18:15:14.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:14 smithi033 ceph-mon[28330]: pgmap v92: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-03T18:15:16.381 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:16 smithi150 ceph-mon[32492]: pgmap v93: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-03T18:15:16.535 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:16 smithi033 ceph-mon[28330]: pgmap v93: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-03T18:15:16.544 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:16 smithi099 ceph-mon[31658]: pgmap v93: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-03T18:15:18.544 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:18 smithi099 ceph-mon[31658]: pgmap v94: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-03T18:15:18.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:18 smithi150 ceph-mon[32492]: pgmap v94: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-03T18:15:18.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:18 smithi033 ceph-mon[28330]: pgmap v94: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-03T18:15:20.495 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:20 smithi150 ceph-mon[32492]: pgmap v95: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-03T18:15:20.544 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:20 smithi099 ceph-mon[31658]: pgmap v95: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-03T18:15:20.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:20 smithi033 ceph-mon[28330]: pgmap v95: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-03T18:15:21.544 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:21 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-03T18:15:21.544 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:21 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:15:21.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:21 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-03T18:15:21.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:21 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:15:21.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:21 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-03T18:15:21.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:21 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:15:22.544 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:22 smithi099 ceph-mon[31658]: Deploying daemon osd.2 on smithi150 2024-05-03T18:15:22.544 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:22 smithi099 ceph-mon[31658]: pgmap v96: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-03T18:15:22.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:22 smithi150 ceph-mon[32492]: Deploying daemon osd.2 on smithi150 2024-05-03T18:15:22.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:22 smithi150 ceph-mon[32492]: pgmap v96: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-03T18:15:22.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:22 smithi033 ceph-mon[28330]: Deploying daemon osd.2 on smithi150 2024-05-03T18:15:22.749 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:22 smithi033 ceph-mon[28330]: pgmap v96: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-03T18:15:24.544 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:24 smithi099 ceph-mon[31658]: pgmap v97: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-03T18:15:24.572 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:24 smithi150 ceph-mon[32492]: pgmap v97: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-03T18:15:24.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:24 smithi033 ceph-mon[28330]: pgmap v97: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-03T18:15:26.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:26 smithi150 ceph-mon[32492]: pgmap v98: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-03T18:15:26.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:26 smithi033 ceph-mon[28330]: pgmap v98: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-03T18:15:26.794 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:26 smithi099 ceph-mon[31658]: pgmap v98: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-03T18:15:28.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:28 smithi150 ceph-mon[32492]: pgmap v99: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-03T18:15:28.663 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:28 smithi033 ceph-mon[28330]: pgmap v99: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-03T18:15:28.794 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:28 smithi099 ceph-mon[31658]: pgmap v99: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-03T18:15:30.381 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:30 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:15:30.381 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:30 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:15:30.381 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:30 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:15:30.381 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:30 smithi150 ceph-mon[32492]: pgmap v100: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-03T18:15:30.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:30 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:15:30.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:30 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:15:30.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:30 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:15:30.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:30 smithi033 ceph-mon[28330]: pgmap v100: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-03T18:15:30.544 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:30 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:15:30.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:30 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:15:30.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:30 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:15:30.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:30 smithi099 ceph-mon[31658]: pgmap v100: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-03T18:15:32.345 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:32 smithi150 ceph-mon[32492]: pgmap v101: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-03T18:15:32.544 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:32 smithi099 ceph-mon[31658]: pgmap v101: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-03T18:15:32.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:32 smithi033 ceph-mon[28330]: pgmap v101: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-03T18:15:33.381 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:33 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:15:33.381 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:33 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:15:33.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:33 smithi150 ceph-mon[32492]: from='osd.2 [v2:172.21.15.150:6800/1995659622,v1:172.21.15.150:6801/1995659622]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-03T18:15:33.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:33 smithi150 ceph-mon[32492]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-03T18:15:33.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:33 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:15:33.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:33 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:15:33.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:33 smithi033 ceph-mon[28330]: from='osd.2 [v2:172.21.15.150:6800/1995659622,v1:172.21.15.150:6801/1995659622]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-03T18:15:33.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:33 smithi033 ceph-mon[28330]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-03T18:15:33.794 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:33 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:15:33.794 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:33 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:15:33.794 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:33 smithi099 ceph-mon[31658]: from='osd.2 [v2:172.21.15.150:6800/1995659622,v1:172.21.15.150:6801/1995659622]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-03T18:15:33.794 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:33 smithi099 ceph-mon[31658]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-03T18:15:34.486 INFO:teuthology.orchestra.run.smithi150.stdout:Created osd(s) 2 on host 'smithi150' 2024-05-03T18:15:34.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:15:34.486+0000 7fb961600700 1 -- 172.21.15.150:0/2949954177 <== mgr.14152 v2:172.21.15.33:6800/3642895501 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb9640638e0 con 0x7fb94c0b2870 2024-05-03T18:15:34.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:15:34.490+0000 7fb96b600700 1 -- 172.21.15.150:0/2949954177 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fb94c0b2870 msgr2=0x7fb94c0b4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:15:34.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:15:34.490+0000 7fb96b600700 1 --2- 172.21.15.150:0/2949954177 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fb94c0b2870 0x7fb94c0b4d30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb950005950 tx=0x7fb9500058e0 comp rx=0 tx=0).stop 2024-05-03T18:15:34.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:15:34.491+0000 7fb96b600700 1 -- 172.21.15.150:0/2949954177 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb964104040 msgr2=0x7fb964199480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:15:34.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:15:34.491+0000 7fb96b600700 1 --2- 172.21.15.150:0/2949954177 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb964104040 0x7fb964199480 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb95800c9e0 tx=0x7fb95800cda0 comp rx=0 tx=0).stop 2024-05-03T18:15:34.490 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:15:34.491+0000 7fb96b600700 1 -- 172.21.15.150:0/2949954177 shutdown_connections 2024-05-03T18:15:34.490 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:15:34.491+0000 7fb96b600700 1 --2- 172.21.15.150:0/2949954177 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9641092a0 0x7fb9641a0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:15:34.490 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:15:34.491+0000 7fb96b600700 1 --2- 172.21.15.150:0/2949954177 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb964104040 0x7fb964199480 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:15:34.490 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:15:34.491+0000 7fb96b600700 1 --2- 172.21.15.150:0/2949954177 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fb94c0b2870 0x7fb94c0b4d30 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:15:34.490 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:15:34.491+0000 7fb96b600700 1 --2- 172.21.15.150:0/2949954177 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb964106970 0x7fb9641999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:15:34.491 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:15:34.491+0000 7fb96b600700 1 -- 172.21.15.150:0/2949954177 >> 172.21.15.150:0/2949954177 conn(0x7fb9640fd990 msgr2=0x7fb964104aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:15:34.491 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:15:34.491+0000 7fb96b600700 1 -- 172.21.15.150:0/2949954177 shutdown_connections 2024-05-03T18:15:34.491 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:15:34.491+0000 7fb96b600700 1 -- 172.21.15.150:0/2949954177 wait complete. 2024-05-03T18:15:34.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:34 smithi150 ceph-mon[32492]: pgmap v102: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-03T18:15:34.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:34 smithi150 ceph-mon[32492]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-03T18:15:34.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:34 smithi150 ceph-mon[32492]: osdmap e16: 3 total, 2 up, 3 in 2024-05-03T18:15:34.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:34 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-03T18:15:34.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:34 smithi150 ceph-mon[32492]: from='osd.2 [v2:172.21.15.150:6800/1995659622,v1:172.21.15.150:6801/1995659622]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-03T18:15:34.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:34 smithi150 ceph-mon[32492]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-03T18:15:34.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:34 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:15:34.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:34 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:15:34.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:34 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:15:34.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:34 smithi033 ceph-mon[28330]: pgmap v102: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-03T18:15:34.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:34 smithi033 ceph-mon[28330]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-03T18:15:34.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:34 smithi033 ceph-mon[28330]: osdmap e16: 3 total, 2 up, 3 in 2024-05-03T18:15:34.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:34 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-03T18:15:34.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:34 smithi033 ceph-mon[28330]: from='osd.2 [v2:172.21.15.150:6800/1995659622,v1:172.21.15.150:6801/1995659622]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-03T18:15:34.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:34 smithi033 ceph-mon[28330]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-03T18:15:34.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:34 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:15:34.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:34 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:15:34.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:34 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:15:34.794 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:34 smithi099 ceph-mon[31658]: pgmap v102: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-03T18:15:34.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:34 smithi099 ceph-mon[31658]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-03T18:15:34.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:34 smithi099 ceph-mon[31658]: osdmap e16: 3 total, 2 up, 3 in 2024-05-03T18:15:34.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:34 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-03T18:15:34.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:34 smithi099 ceph-mon[31658]: from='osd.2 [v2:172.21.15.150:6800/1995659622,v1:172.21.15.150:6801/1995659622]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-03T18:15:34.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:34 smithi099 ceph-mon[31658]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-03T18:15:34.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:34 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:15:34.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:34 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:15:34.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:34 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:15:35.335 DEBUG:teuthology.orchestra.run.smithi150:osd.2> sudo journalctl -f -n 0 -u ceph-5703bba4-0978-11ef-bc97-c7b262605968@osd.2.service 2024-05-03T18:15:35.339 INFO:tasks.cephadm:Deploying osd.3 on smithi033 with /dev/vg_nvme/lv_3... 2024-05-03T18:15:35.340 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-03T18:15:35.581 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:35 smithi150 ceph-mon[32492]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-05-03T18:15:35.581 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:35 smithi150 ceph-mon[32492]: osdmap e17: 3 total, 2 up, 3 in 2024-05-03T18:15:35.581 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:35 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-03T18:15:35.581 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:35 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-03T18:15:35.581 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:35 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:15:35.581 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:35 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:15:35.581 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:35 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:15:35.581 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:35 smithi150 ceph-mon[32492]: from='osd.2 ' entity='osd.2' 2024-05-03T18:15:35.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:35 smithi033 ceph-mon[28330]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-05-03T18:15:35.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:35 smithi033 ceph-mon[28330]: osdmap e17: 3 total, 2 up, 3 in 2024-05-03T18:15:35.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:35 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-03T18:15:35.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:35 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-03T18:15:35.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:35 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:15:35.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:35 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:15:35.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:35 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:15:35.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:35 smithi033 ceph-mon[28330]: from='osd.2 ' entity='osd.2' 2024-05-03T18:15:35.794 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:35 smithi099 ceph-mon[31658]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-05-03T18:15:35.794 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:35 smithi099 ceph-mon[31658]: osdmap e17: 3 total, 2 up, 3 in 2024-05-03T18:15:35.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:35 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-03T18:15:35.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:35 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-03T18:15:35.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:35 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:15:35.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:35 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:15:35.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:35 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:15:35.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:35 smithi099 ceph-mon[31658]: from='osd.2 ' entity='osd.2' 2024-05-03T18:15:35.801 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:15:36.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:36 smithi150 ceph-mon[32492]: purged_snaps scrub starts 2024-05-03T18:15:36.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:36 smithi150 ceph-mon[32492]: purged_snaps scrub ok 2024-05-03T18:15:36.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:36 smithi150 ceph-mon[32492]: pgmap v105: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-03T18:15:36.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:36 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-03T18:15:36.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:36 smithi150 ceph-mon[32492]: osd.2 [v2:172.21.15.150:6800/1995659622,v1:172.21.15.150:6801/1995659622] boot 2024-05-03T18:15:36.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:36 smithi150 ceph-mon[32492]: osdmap e18: 3 total, 3 up, 3 in 2024-05-03T18:15:36.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:36 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-03T18:15:36.702 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:36 smithi033 ceph-mon[28330]: purged_snaps scrub starts 2024-05-03T18:15:36.702 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:36 smithi033 ceph-mon[28330]: purged_snaps scrub ok 2024-05-03T18:15:36.702 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:36 smithi033 ceph-mon[28330]: pgmap v105: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-03T18:15:36.702 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:36 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-03T18:15:36.702 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:36 smithi033 ceph-mon[28330]: osd.2 [v2:172.21.15.150:6800/1995659622,v1:172.21.15.150:6801/1995659622] boot 2024-05-03T18:15:36.702 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:36 smithi033 ceph-mon[28330]: osdmap e18: 3 total, 3 up, 3 in 2024-05-03T18:15:36.702 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:36 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-03T18:15:36.794 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:36 smithi099 ceph-mon[31658]: purged_snaps scrub starts 2024-05-03T18:15:36.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:36 smithi099 ceph-mon[31658]: purged_snaps scrub ok 2024-05-03T18:15:36.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:36 smithi099 ceph-mon[31658]: pgmap v105: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-03T18:15:36.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:36 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-03T18:15:36.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:36 smithi099 ceph-mon[31658]: osd.2 [v2:172.21.15.150:6800/1995659622,v1:172.21.15.150:6801/1995659622] boot 2024-05-03T18:15:36.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:36 smithi099 ceph-mon[31658]: osdmap e18: 3 total, 3 up, 3 in 2024-05-03T18:15:36.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:36 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-03T18:15:37.562 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:37 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-03T18:15:37.657 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:37 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-03T18:15:37.794 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:37 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-03T18:15:38.627 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:38 smithi033 ceph-mon[28330]: pgmap v107: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-03T18:15:38.627 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:38 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-03T18:15:38.627 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:38 smithi033 ceph-mon[28330]: osdmap e19: 3 total, 3 up, 3 in 2024-05-03T18:15:38.627 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:38 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-03T18:15:38.627 INFO:teuthology.orchestra.run.smithi033.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-03T18:15:38.628 INFO:teuthology.orchestra.run.smithi033.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-03T18:15:38.628 INFO:teuthology.orchestra.run.smithi033.stderr: stderr: 10+0 records in 2024-05-03T18:15:38.628 INFO:teuthology.orchestra.run.smithi033.stderr:10+0 records out 2024-05-03T18:15:38.628 INFO:teuthology.orchestra.run.smithi033.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0593286 s, 177 MB/s 2024-05-03T18:15:38.628 INFO:teuthology.orchestra.run.smithi033.stderr:--> Zapping successful for: 2024-05-03T18:15:38.628 INFO:journalctl@ceph.osd.0.smithi033.stdout:May 03 18:15:38 smithi033 sudo[47286]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-03T18:15:38.628 INFO:journalctl@ceph.osd.0.smithi033.stdout:May 03 18:15:38 smithi033 sudo[47286]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-03T18:15:38.628 INFO:journalctl@ceph.osd.0.smithi033.stdout:May 03 18:15:38 smithi033 sudo[47286]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-03T18:15:38.628 INFO:journalctl@ceph.osd.0.smithi033.stdout:May 03 18:15:38 smithi033 sudo[47286]: pam_unix(sudo:session): session closed for user root 2024-05-03T18:15:38.628 INFO:journalctl@ceph.osd.0.smithi033.stdout:May 03 18:15:38 smithi033 sudo[47334]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-03T18:15:38.628 INFO:journalctl@ceph.osd.0.smithi033.stdout:May 03 18:15:38 smithi033 sudo[47334]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-03T18:15:38.629 INFO:journalctl@ceph.osd.0.smithi033.stdout:May 03 18:15:38 smithi033 sudo[47334]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-03T18:15:38.629 INFO:journalctl@ceph.osd.0.smithi033.stdout:May 03 18:15:38 smithi033 sudo[47334]: pam_unix(sudo:session): session closed for user root 2024-05-03T18:15:38.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:38 smithi150 ceph-mon[32492]: pgmap v107: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-03T18:15:38.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:38 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-03T18:15:38.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:38 smithi150 ceph-mon[32492]: osdmap e19: 3 total, 3 up, 3 in 2024-05-03T18:15:38.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:38 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-03T18:15:38.727 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:38 smithi099 ceph-mon[31658]: pgmap v107: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-03T18:15:38.727 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:38 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-03T18:15:38.728 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:38 smithi099 ceph-mon[31658]: osdmap e19: 3 total, 3 up, 3 in 2024-05-03T18:15:38.728 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:38 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-03T18:15:39.044 INFO:journalctl@ceph.osd.1.smithi099.stdout:May 03 18:15:38 smithi099 sudo[40273]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-03T18:15:39.045 INFO:journalctl@ceph.osd.1.smithi099.stdout:May 03 18:15:38 smithi099 sudo[40273]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-03T18:15:39.045 INFO:journalctl@ceph.osd.1.smithi099.stdout:May 03 18:15:38 smithi099 sudo[40273]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-03T18:15:39.045 INFO:journalctl@ceph.osd.1.smithi099.stdout:May 03 18:15:38 smithi099 sudo[40273]: pam_unix(sudo:session): session closed for user root 2024-05-03T18:15:39.045 INFO:journalctl@ceph.osd.1.smithi099.stdout:May 03 18:15:38 smithi099 sudo[40276]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-03T18:15:39.045 INFO:journalctl@ceph.osd.1.smithi099.stdout:May 03 18:15:38 smithi099 sudo[40276]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-03T18:15:39.045 INFO:journalctl@ceph.osd.1.smithi099.stdout:May 03 18:15:38 smithi099 sudo[40276]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-03T18:15:39.045 INFO:journalctl@ceph.osd.1.smithi099.stdout:May 03 18:15:38 smithi099 sudo[40276]: pam_unix(sudo:session): session closed for user root 2024-05-03T18:15:39.170 INFO:journalctl@ceph.osd.2.smithi150.stdout:May 03 18:15:38 smithi150 sudo[39299]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-03T18:15:39.170 INFO:journalctl@ceph.osd.2.smithi150.stdout:May 03 18:15:38 smithi150 sudo[39299]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-03T18:15:39.170 INFO:journalctl@ceph.osd.2.smithi150.stdout:May 03 18:15:38 smithi150 sudo[39299]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-03T18:15:39.170 INFO:journalctl@ceph.osd.2.smithi150.stdout:May 03 18:15:38 smithi150 sudo[39299]: pam_unix(sudo:session): session closed for user root 2024-05-03T18:15:39.170 INFO:journalctl@ceph.osd.2.smithi150.stdout:May 03 18:15:38 smithi150 sudo[39305]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-03T18:15:39.170 INFO:journalctl@ceph.osd.2.smithi150.stdout:May 03 18:15:38 smithi150 sudo[39305]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-03T18:15:39.170 INFO:journalctl@ceph.osd.2.smithi150.stdout:May 03 18:15:38 smithi150 sudo[39305]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-03T18:15:39.170 INFO:journalctl@ceph.osd.2.smithi150.stdout:May 03 18:15:38 smithi150 sudo[39305]: pam_unix(sudo:session): session closed for user root 2024-05-03T18:15:39.233 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:39 smithi033 sudo[47352]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-03T18:15:39.233 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:39 smithi033 sudo[47352]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-03T18:15:39.233 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:39 smithi033 sudo[47352]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-03T18:15:39.233 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:39 smithi033 sudo[47352]: pam_unix(sudo:session): session closed for user root 2024-05-03T18:15:39.233 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:39 smithi033 sudo[47358]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-03T18:15:39.233 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:39 smithi033 sudo[47358]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-03T18:15:39.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:39 smithi033 sudo[47358]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-03T18:15:39.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:39 smithi033 sudo[47358]: pam_unix(sudo:session): session closed for user root 2024-05-03T18:15:39.468 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi033:vg_nvme/lv_3 2024-05-03T18:15:39.537 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:39 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-03T18:15:39.537 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:39 smithi033 ceph-mon[28330]: osdmap e20: 3 total, 3 up, 3 in 2024-05-03T18:15:39.537 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:39 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:15:39.537 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:39 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:15:39.537 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:39 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:15:39.537 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:39 smithi033 ceph-mon[28330]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-03T18:15:39.537 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:39 smithi033 ceph-mon[28330]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-03T18:15:39.537 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:39 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-03T18:15:39.537 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:39 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T18:15:39.538 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:39 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T18:15:39.538 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:39 smithi033 ceph-mon[28330]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-03T18:15:39.538 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:39 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-03T18:15:39.538 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:39 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T18:15:39.538 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:39 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T18:15:39.538 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:39 smithi033 ceph-mon[28330]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-03T18:15:39.538 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:39 smithi033 ceph-mon[28330]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-03T18:15:39.538 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:39 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-03T18:15:39.538 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:39 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T18:15:39.538 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:39 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T18:15:39.538 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:39 smithi033 ceph-mon[28330]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-03T18:15:39.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:39 smithi099 sudo[40279]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-03T18:15:39.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:39 smithi099 sudo[40279]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-03T18:15:39.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:39 smithi099 sudo[40279]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-03T18:15:39.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:39 smithi099 sudo[40279]: pam_unix(sudo:session): session closed for user root 2024-05-03T18:15:39.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:39 smithi099 sudo[40282]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-03T18:15:39.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:39 smithi099 sudo[40282]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-03T18:15:39.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:39 smithi099 sudo[40282]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-03T18:15:39.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:39 smithi099 sudo[40282]: pam_unix(sudo:session): session closed for user root 2024-05-03T18:15:39.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:39 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-03T18:15:39.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:39 smithi099 ceph-mon[31658]: osdmap e20: 3 total, 3 up, 3 in 2024-05-03T18:15:39.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:39 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:15:39.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:39 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:15:39.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:39 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:15:39.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:39 smithi099 ceph-mon[31658]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-03T18:15:39.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:39 smithi099 ceph-mon[31658]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-03T18:15:39.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:39 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-03T18:15:39.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:39 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T18:15:39.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:39 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T18:15:39.547 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:39 smithi099 ceph-mon[31658]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-03T18:15:39.547 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:39 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-03T18:15:39.547 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:39 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T18:15:39.547 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:39 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T18:15:39.547 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:39 smithi099 ceph-mon[31658]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-03T18:15:39.547 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:39 smithi099 ceph-mon[31658]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-03T18:15:39.547 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:39 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-03T18:15:39.547 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:39 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T18:15:39.547 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:39 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T18:15:39.547 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:39 smithi099 ceph-mon[31658]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-03T18:15:39.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:39 smithi150 sudo[39423]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-03T18:15:39.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:39 smithi150 sudo[39423]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-03T18:15:39.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:39 smithi150 sudo[39423]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-03T18:15:39.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:39 smithi150 sudo[39423]: pam_unix(sudo:session): session closed for user root 2024-05-03T18:15:39.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:39 smithi150 sudo[39657]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-03T18:15:39.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:39 smithi150 sudo[39657]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-03T18:15:39.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:39 smithi150 sudo[39657]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-03T18:15:39.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:39 smithi150 sudo[39657]: pam_unix(sudo:session): session closed for user root 2024-05-03T18:15:39.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:39 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-03T18:15:39.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:39 smithi150 ceph-mon[32492]: osdmap e20: 3 total, 3 up, 3 in 2024-05-03T18:15:39.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:39 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:15:39.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:39 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:15:39.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:39 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:15:39.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:39 smithi150 ceph-mon[32492]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-03T18:15:39.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:39 smithi150 ceph-mon[32492]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-03T18:15:39.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:39 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-03T18:15:39.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:39 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T18:15:39.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:39 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T18:15:39.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:39 smithi150 ceph-mon[32492]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-03T18:15:39.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:39 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-03T18:15:39.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:39 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T18:15:39.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:39 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T18:15:39.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:39 smithi150 ceph-mon[32492]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-03T18:15:39.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:39 smithi150 ceph-mon[32492]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-03T18:15:39.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:39 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-03T18:15:39.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:39 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T18:15:39.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:39 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T18:15:39.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:39 smithi150 ceph-mon[32492]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-03T18:15:39.914 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:15:40.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:40 smithi033 ceph-mon[28330]: pgmap v110: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-03T18:15:40.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:40 smithi033 ceph-mon[28330]: osdmap e21: 3 total, 3 up, 3 in 2024-05-03T18:15:40.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:40 smithi033 ceph-mon[28330]: mgrmap e16: x(active, since 3m), standbys: z, y 2024-05-03T18:15:40.794 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:40 smithi099 ceph-mon[31658]: pgmap v110: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-03T18:15:40.794 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:40 smithi099 ceph-mon[31658]: osdmap e21: 3 total, 3 up, 3 in 2024-05-03T18:15:40.794 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:40 smithi099 ceph-mon[31658]: mgrmap e16: x(active, since 3m), standbys: z, y 2024-05-03T18:15:40.881 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:40 smithi150 ceph-mon[32492]: pgmap v110: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-03T18:15:40.881 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:40 smithi150 ceph-mon[32492]: osdmap e21: 3 total, 3 up, 3 in 2024-05-03T18:15:40.881 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:40 smithi150 ceph-mon[32492]: mgrmap e16: x(active, since 3m), standbys: z, y 2024-05-03T18:15:41.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:41 smithi033 ceph-mon[28330]: Detected new or changed devices on smithi150 2024-05-03T18:15:41.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:41 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:15:41.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:41 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:15:41.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:41 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:15:41.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:41 smithi033 ceph-mon[28330]: Adjusting osd_memory_target on smithi150 to 17089M 2024-05-03T18:15:41.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:41 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:15:41.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:41 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:15:41.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:41 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:15:41.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:41 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:15:41.881 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:41 smithi150 ceph-mon[32492]: Detected new or changed devices on smithi150 2024-05-03T18:15:41.881 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:41 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:15:41.881 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:41 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:15:41.881 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:41 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:15:41.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:41 smithi150 ceph-mon[32492]: Adjusting osd_memory_target on smithi150 to 17089M 2024-05-03T18:15:41.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:41 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:15:41.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:41 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:15:41.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:41 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:15:41.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:41 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:15:41.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:15:41.970+0000 7fbb37000700 1 -- 172.21.15.33:0/3497077189 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb301034f0 msgr2=0x7fbb30103910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:15:41.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:15:41.970+0000 7fbb37000700 1 --2- 172.21.15.33:0/3497077189 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb301034f0 0x7fbb30103910 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fbb200099c0 tx=0x7fbb20009cd0 comp rx=0 tx=0).stop 2024-05-03T18:15:41.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:15:41.970+0000 7fbb37000700 1 -- 172.21.15.33:0/3497077189 shutdown_connections 2024-05-03T18:15:41.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:15:41.970+0000 7fbb37000700 1 --2- 172.21.15.33:0/3497077189 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbb30105090 0x7fbb3010b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:15:41.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:15:41.970+0000 7fbb37000700 1 --2- 172.21.15.33:0/3497077189 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb301034f0 0x7fbb30103910 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:15:41.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:15:41.970+0000 7fbb37000700 1 --2- 172.21.15.33:0/3497077189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb301046f0 0x7fbb30104b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:15:41.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:15:41.970+0000 7fbb37000700 1 -- 172.21.15.33:0/3497077189 >> 172.21.15.33:0/3497077189 conn(0x7fbb300fea90 msgr2=0x7fbb30100ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:15:41.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:15:41.970+0000 7fbb37000700 1 -- 172.21.15.33:0/3497077189 shutdown_connections 2024-05-03T18:15:41.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:15:41.970+0000 7fbb37000700 1 -- 172.21.15.33:0/3497077189 wait complete. 2024-05-03T18:15:41.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:15:41.972+0000 7fbb37000700 1 Processor -- start 2024-05-03T18:15:41.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:15:41.972+0000 7fbb37000700 1 -- start start 2024-05-03T18:15:41.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:15:41.972+0000 7fbb37000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb301034f0 0x7fbb30077240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:15:41.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:15:41.972+0000 7fbb37000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbb301046f0 0x7fbb30075890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:15:41.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:15:41.972+0000 7fbb37000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb30105090 0x7fbb30075dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:15:41.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:15:41.972+0000 7fbb37000700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbb3010ee40 con 0x7fbb301034f0 2024-05-03T18:15:41.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:15:41.972+0000 7fbb37000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbb3010ecc0 con 0x7fbb30105090 2024-05-03T18:15:41.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:15:41.972+0000 7fbb37000700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fbb3010efc0 con 0x7fbb301046f0 2024-05-03T18:15:41.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:15:41.973+0000 7fbb2fe00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb301034f0 0x7fbb30077240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:15:41.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:15:41.973+0000 7fbb2fe00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb301034f0 0x7fbb30077240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34734/0 (socket says 172.21.15.33:34734) 2024-05-03T18:15:41.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:15:41.973+0000 7fbb2fe00700 1 -- 172.21.15.33:0/3867975128 learned_addr learned my addr 172.21.15.33:0/3867975128 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:15:41.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:15:41.973+0000 7fbb2f400700 1 --2- 172.21.15.33:0/3867975128 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbb301046f0 0x7fbb30075890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:15:41.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:15:41.973+0000 7fbb35000700 1 --2- 172.21.15.33:0/3867975128 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb30105090 0x7fbb30075dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:15:41.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:15:41.973+0000 7fbb2fe00700 1 -- 172.21.15.33:0/3867975128 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbb301046f0 msgr2=0x7fbb30075890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:15:41.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:15:41.973+0000 7fbb2fe00700 1 --2- 172.21.15.33:0/3867975128 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbb301046f0 0x7fbb30075890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:15:41.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:15:41.973+0000 7fbb2fe00700 1 -- 172.21.15.33:0/3867975128 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb30105090 msgr2=0x7fbb30075dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:15:41.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:15:41.973+0000 7fbb2fe00700 1 --2- 172.21.15.33:0/3867975128 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb30105090 0x7fbb30075dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:15:41.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:15:41.973+0000 7fbb2fe00700 1 -- 172.21.15.33:0/3867975128 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb30076680 con 0x7fbb301034f0 2024-05-03T18:15:41.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:15:41.974+0000 7fbb2fe00700 1 --2- 172.21.15.33:0/3867975128 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb301034f0 0x7fbb30077240 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fbb200036f0 tx=0x7fbb2000f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:15:41.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:15:41.974+0000 7fbb2cc00700 1 -- 172.21.15.33:0/3867975128 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb20009e60 con 0x7fbb301034f0 2024-05-03T18:15:41.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:15:41.974+0000 7fbb2cc00700 1 -- 172.21.15.33:0/3867975128 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbb20005940 con 0x7fbb301034f0 2024-05-03T18:15:41.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:15:41.974+0000 7fbb2cc00700 1 -- 172.21.15.33:0/3867975128 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb20005450 con 0x7fbb301034f0 2024-05-03T18:15:41.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:15:41.974+0000 7fbb37000700 1 -- 172.21.15.33:0/3867975128 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb301a36a0 con 0x7fbb301034f0 2024-05-03T18:15:41.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:15:41.974+0000 7fbb37000700 1 -- 172.21.15.33:0/3867975128 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb301a3b80 con 0x7fbb301034f0 2024-05-03T18:15:41.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:15:41.975+0000 7fbb37000700 1 -- 172.21.15.33:0/3867975128 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb3006a190 con 0x7fbb301034f0 2024-05-03T18:15:41.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:15:41.978+0000 7fbb2cc00700 1 -- 172.21.15.33:0/3867975128 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fbb2000fb80 con 0x7fbb301034f0 2024-05-03T18:15:41.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:15:41.979+0000 7fbb2cc00700 1 --2- 172.21.15.33:0/3867975128 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fbb140b2900 0x7fbb140b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:15:41.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:15:41.979+0000 7fbb2cc00700 1 -- 172.21.15.33:0/3867975128 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 3237+0+0 (secure 0 0 0) 0x7fbb200e1c30 con 0x7fbb301034f0 2024-05-03T18:15:41.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:15:41.980+0000 7fbb2f400700 1 --2- 172.21.15.33:0/3867975128 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fbb140b2900 0x7fbb140b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:15:41.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:15:41.980+0000 7fbb2f400700 1 --2- 172.21.15.33:0/3867975128 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fbb140b2900 0x7fbb140b4dc0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c009c60 tx=0x7fbb1c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:15:41.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:15:41.983+0000 7fbb2cc00700 1 -- 172.21.15.33:0/3867975128 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fbb20013070 con 0x7fbb301034f0 2024-05-03T18:15:42.044 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:41 smithi099 ceph-mon[31658]: Detected new or changed devices on smithi150 2024-05-03T18:15:42.044 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:41 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:15:42.044 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:41 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:15:42.044 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:41 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:15:42.044 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:41 smithi099 ceph-mon[31658]: Adjusting osd_memory_target on smithi150 to 17089M 2024-05-03T18:15:42.044 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:41 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:15:42.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:41 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:15:42.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:41 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:15:42.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:41 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:15:42.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:15:42.231+0000 7fbb37000700 1 -- 172.21.15.33:0/3867975128 --> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fbb300638e0 con 0x7fbb140b2900 2024-05-03T18:15:42.881 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:42 smithi150 ceph-mon[32492]: pgmap v112: 1 pgs: 1 creating+peering; 0 B data, 89 MiB used, 268 GiB / 268 GiB avail 2024-05-03T18:15:42.881 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:42 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T18:15:42.881 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:42 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T18:15:42.881 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:42 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:15:42.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:42 smithi033 ceph-mon[28330]: pgmap v112: 1 pgs: 1 creating+peering; 0 B data, 89 MiB used, 268 GiB / 268 GiB avail 2024-05-03T18:15:42.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:42 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T18:15:42.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:42 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T18:15:42.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:42 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:15:43.044 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:42 smithi099 ceph-mon[31658]: pgmap v112: 1 pgs: 1 creating+peering; 0 B data, 89 MiB used, 268 GiB / 268 GiB avail 2024-05-03T18:15:43.044 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:42 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T18:15:43.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:42 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T18:15:43.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:42 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:15:43.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:43 smithi033 ceph-mon[28330]: from='client.14301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:15:44.044 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:43 smithi099 ceph-mon[31658]: from='client.14301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:15:44.131 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:43 smithi150 ceph-mon[32492]: from='client.14301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:15:44.891 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:44 smithi033 ceph-mon[28330]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-03T18:15:45.044 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:44 smithi099 ceph-mon[31658]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-03T18:15:45.131 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:44 smithi150 ceph-mon[32492]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-03T18:15:46.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:46 smithi033 ceph-mon[28330]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-03T18:15:46.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:46 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/905174786' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d88ed022-4dcb-45fc-9fc6-41ef7c4bf202"}]: dispatch 2024-05-03T18:15:46.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:46 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/905174786' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d88ed022-4dcb-45fc-9fc6-41ef7c4bf202"}]': finished 2024-05-03T18:15:46.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:46 smithi033 ceph-mon[28330]: osdmap e22: 4 total, 3 up, 4 in 2024-05-03T18:15:46.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:46 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-03T18:15:47.044 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:46 smithi099 ceph-mon[31658]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-03T18:15:47.044 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:46 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/905174786' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d88ed022-4dcb-45fc-9fc6-41ef7c4bf202"}]: dispatch 2024-05-03T18:15:47.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:46 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/905174786' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d88ed022-4dcb-45fc-9fc6-41ef7c4bf202"}]': finished 2024-05-03T18:15:47.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:46 smithi099 ceph-mon[31658]: osdmap e22: 4 total, 3 up, 4 in 2024-05-03T18:15:47.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:46 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-03T18:15:47.131 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:46 smithi150 ceph-mon[32492]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-03T18:15:47.131 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:46 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/905174786' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d88ed022-4dcb-45fc-9fc6-41ef7c4bf202"}]: dispatch 2024-05-03T18:15:47.131 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:46 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/905174786' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d88ed022-4dcb-45fc-9fc6-41ef7c4bf202"}]': finished 2024-05-03T18:15:47.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:46 smithi150 ceph-mon[32492]: osdmap e22: 4 total, 3 up, 4 in 2024-05-03T18:15:47.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:46 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-03T18:15:47.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:47 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/3210380761' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T18:15:48.044 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:47 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/3210380761' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T18:15:48.131 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:47 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/3210380761' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T18:15:48.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:48 smithi033 ceph-mon[28330]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-03T18:15:49.044 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:48 smithi099 ceph-mon[31658]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-03T18:15:49.131 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:48 smithi150 ceph-mon[32492]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-03T18:15:50.960 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:50 smithi033 ceph-mon[28330]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-03T18:15:51.044 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:50 smithi099 ceph-mon[31658]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-03T18:15:51.131 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:50 smithi150 ceph-mon[32492]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-03T18:15:52.800 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:52 smithi033 ceph-mon[28330]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-03T18:15:53.044 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:52 smithi099 ceph-mon[31658]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-03T18:15:53.131 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:52 smithi150 ceph-mon[32492]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-03T18:15:53.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:53 smithi033 ceph-mon[28330]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-03T18:15:54.044 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:53 smithi099 ceph-mon[31658]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-03T18:15:54.131 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:53 smithi150 ceph-mon[32492]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-03T18:15:56.544 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:56 smithi099 ceph-mon[31658]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-03T18:15:56.544 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:56 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-03T18:15:56.544 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:56 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:15:56.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:56 smithi150 ceph-mon[32492]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-03T18:15:56.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:56 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-03T18:15:56.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:56 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:15:56.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:56 smithi033 ceph-mon[28330]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-03T18:15:56.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:56 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-03T18:15:56.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:56 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:15:57.544 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:57 smithi099 ceph-mon[31658]: Deploying daemon osd.3 on smithi033 2024-05-03T18:15:57.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:57 smithi150 ceph-mon[32492]: Deploying daemon osd.3 on smithi033 2024-05-03T18:15:57.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:57 smithi033 ceph-mon[28330]: Deploying daemon osd.3 on smithi033 2024-05-03T18:15:58.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:15:58 smithi033 ceph-mon[28330]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-03T18:15:58.544 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:15:58 smithi099 ceph-mon[31658]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-03T18:15:58.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:15:58 smithi150 ceph-mon[32492]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-03T18:16:00.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:00 smithi033 ceph-mon[28330]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-03T18:16:00.544 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:00 smithi099 ceph-mon[31658]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-03T18:16:00.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:00 smithi150 ceph-mon[32492]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-03T18:16:02.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:02 smithi033 ceph-mon[28330]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-03T18:16:02.544 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:02 smithi099 ceph-mon[31658]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-03T18:16:02.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:02 smithi150 ceph-mon[32492]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-03T18:16:04.544 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:04 smithi099 ceph-mon[31658]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-03T18:16:04.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:04 smithi150 ceph-mon[32492]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-03T18:16:04.637 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:04 smithi033 ceph-mon[28330]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-03T18:16:05.544 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:05 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:16:05.544 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:05 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:16:05.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:05 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:16:05.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:05 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:16:05.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:05 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:16:05.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:05 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:16:05.732 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:05 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:16:05.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:05 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:16:05.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:05 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:16:06.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:06 smithi033 ceph-mon[28330]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-03T18:16:06.544 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:06 smithi099 ceph-mon[31658]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-03T18:16:06.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:06 smithi150 ceph-mon[32492]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-03T18:16:08.416 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:08 smithi033 ceph-mon[28330]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-03T18:16:08.544 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:08 smithi099 ceph-mon[31658]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-03T18:16:08.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:08 smithi150 ceph-mon[32492]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-03T18:16:09.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:09 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:16:09.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:09 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:16:09.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:09 smithi033 ceph-mon[28330]: from='osd.3 [v2:172.21.15.33:6810/3814942724,v1:172.21.15.33:6811/3814942724]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-03T18:16:09.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:09 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:16:09.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:09 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:16:09.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:09 smithi150 ceph-mon[32492]: from='osd.3 [v2:172.21.15.33:6810/3814942724,v1:172.21.15.33:6811/3814942724]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-03T18:16:09.794 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:09 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:16:09.794 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:09 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:16:09.794 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:09 smithi099 ceph-mon[31658]: from='osd.3 [v2:172.21.15.33:6810/3814942724,v1:172.21.15.33:6811/3814942724]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-03T18:16:10.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:16:10.130+0000 7fbb2cc00700 1 -- 172.21.15.33:0/3867975128 <== mgr.14152 v2:172.21.15.33:6800/3642895501 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fbb300638e0 con 0x7fbb140b2900 2024-05-03T18:16:10.132 INFO:teuthology.orchestra.run.smithi033.stdout:Created osd(s) 3 on host 'smithi033' 2024-05-03T18:16:10.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:16:10.134+0000 7fbb37000700 1 -- 172.21.15.33:0/3867975128 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fbb140b2900 msgr2=0x7fbb140b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:16:10.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:16:10.134+0000 7fbb37000700 1 --2- 172.21.15.33:0/3867975128 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fbb140b2900 0x7fbb140b4dc0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c009c60 tx=0x7fbb1c009260 comp rx=0 tx=0).stop 2024-05-03T18:16:10.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:16:10.134+0000 7fbb37000700 1 -- 172.21.15.33:0/3867975128 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb301034f0 msgr2=0x7fbb30077240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:16:10.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:16:10.134+0000 7fbb37000700 1 --2- 172.21.15.33:0/3867975128 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb301034f0 0x7fbb30077240 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fbb200036f0 tx=0x7fbb2000f710 comp rx=0 tx=0).stop 2024-05-03T18:16:10.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:16:10.134+0000 7fbb37000700 1 -- 172.21.15.33:0/3867975128 shutdown_connections 2024-05-03T18:16:10.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:16:10.134+0000 7fbb37000700 1 --2- 172.21.15.33:0/3867975128 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fbb140b2900 0x7fbb140b4dc0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:16:10.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:16:10.134+0000 7fbb37000700 1 --2- 172.21.15.33:0/3867975128 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb30105090 0x7fbb30075dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:16:10.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:16:10.134+0000 7fbb37000700 1 --2- 172.21.15.33:0/3867975128 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb301034f0 0x7fbb30077240 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:16:10.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:16:10.134+0000 7fbb37000700 1 --2- 172.21.15.33:0/3867975128 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbb301046f0 0x7fbb30075890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:16:10.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:16:10.134+0000 7fbb37000700 1 -- 172.21.15.33:0/3867975128 >> 172.21.15.33:0/3867975128 conn(0x7fbb300fea90 msgr2=0x7fbb30107b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:16:10.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:16:10.135+0000 7fbb37000700 1 -- 172.21.15.33:0/3867975128 shutdown_connections 2024-05-03T18:16:10.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:16:10.135+0000 7fbb37000700 1 -- 172.21.15.33:0/3867975128 wait complete. 2024-05-03T18:16:10.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:10 smithi150 ceph-mon[32492]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-03T18:16:10.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:10 smithi150 ceph-mon[32492]: from='osd.3 [v2:172.21.15.33:6810/3814942724,v1:172.21.15.33:6811/3814942724]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-03T18:16:10.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:10 smithi150 ceph-mon[32492]: osdmap e23: 4 total, 3 up, 4 in 2024-05-03T18:16:10.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:10 smithi150 ceph-mon[32492]: from='osd.3 [v2:172.21.15.33:6810/3814942724,v1:172.21.15.33:6811/3814942724]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-05-03T18:16:10.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:10 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-03T18:16:10.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:10 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:16:10.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:10 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:16:10.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:10 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:16:10.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:10 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:16:10.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:10 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:16:10.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:10 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:16:10.696 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:10 smithi099 ceph-mon[31658]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-03T18:16:10.696 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:10 smithi099 ceph-mon[31658]: from='osd.3 [v2:172.21.15.33:6810/3814942724,v1:172.21.15.33:6811/3814942724]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-03T18:16:10.696 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:10 smithi099 ceph-mon[31658]: osdmap e23: 4 total, 3 up, 4 in 2024-05-03T18:16:10.697 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:10 smithi099 ceph-mon[31658]: from='osd.3 [v2:172.21.15.33:6810/3814942724,v1:172.21.15.33:6811/3814942724]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-05-03T18:16:10.697 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:10 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-03T18:16:10.697 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:10 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:16:10.697 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:10 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:16:10.697 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:10 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:16:10.697 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:10 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:16:10.697 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:10 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:16:10.697 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:10 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:16:10.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:10 smithi033 ceph-mon[28330]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-03T18:16:10.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:10 smithi033 ceph-mon[28330]: from='osd.3 [v2:172.21.15.33:6810/3814942724,v1:172.21.15.33:6811/3814942724]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-03T18:16:10.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:10 smithi033 ceph-mon[28330]: osdmap e23: 4 total, 3 up, 4 in 2024-05-03T18:16:10.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:10 smithi033 ceph-mon[28330]: from='osd.3 [v2:172.21.15.33:6810/3814942724,v1:172.21.15.33:6811/3814942724]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-05-03T18:16:10.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:10 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-03T18:16:10.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:10 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:16:10.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:10 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:16:10.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:10 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:16:10.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:10 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:16:10.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:10 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:16:10.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:10 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:16:10.927 DEBUG:teuthology.orchestra.run.smithi033:osd.3> sudo journalctl -f -n 0 -u ceph-5703bba4-0978-11ef-bc97-c7b262605968@osd.3.service 2024-05-03T18:16:10.931 INFO:tasks.cephadm:Deploying osd.4 on smithi099 with /dev/vg_nvme/lv_3... 2024-05-03T18:16:10.932 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-03T18:16:11.379 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.b/config 2024-05-03T18:16:11.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:11 smithi033 ceph-mon[28330]: from='osd.3 [v2:172.21.15.33:6810/3814942724,v1:172.21.15.33:6811/3814942724]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-05-03T18:16:11.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:11 smithi033 ceph-mon[28330]: osdmap e24: 4 total, 3 up, 4 in 2024-05-03T18:16:11.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:11 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-03T18:16:11.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:11 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-03T18:16:11.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:11 smithi150 ceph-mon[32492]: from='osd.3 [v2:172.21.15.33:6810/3814942724,v1:172.21.15.33:6811/3814942724]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-05-03T18:16:11.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:11 smithi150 ceph-mon[32492]: osdmap e24: 4 total, 3 up, 4 in 2024-05-03T18:16:11.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:11 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-03T18:16:11.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:11 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-03T18:16:11.698 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:11 smithi099 ceph-mon[31658]: from='osd.3 [v2:172.21.15.33:6810/3814942724,v1:172.21.15.33:6811/3814942724]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-05-03T18:16:11.698 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:11 smithi099 ceph-mon[31658]: osdmap e24: 4 total, 3 up, 4 in 2024-05-03T18:16:11.698 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:11 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-03T18:16:11.698 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:11 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-03T18:16:12.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:12 smithi150 ceph-mon[32492]: purged_snaps scrub starts 2024-05-03T18:16:12.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:12 smithi150 ceph-mon[32492]: purged_snaps scrub ok 2024-05-03T18:16:12.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:12 smithi150 ceph-mon[32492]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-03T18:16:12.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:12 smithi150 ceph-mon[32492]: osd.3 [v2:172.21.15.33:6810/3814942724,v1:172.21.15.33:6811/3814942724] boot 2024-05-03T18:16:12.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:12 smithi150 ceph-mon[32492]: osdmap e25: 4 total, 4 up, 4 in 2024-05-03T18:16:12.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:12 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-03T18:16:12.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:12 smithi150 ceph-mon[32492]: osdmap e26: 4 total, 4 up, 4 in 2024-05-03T18:16:12.696 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:12 smithi033 ceph-mon[28330]: purged_snaps scrub starts 2024-05-03T18:16:12.697 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:12 smithi033 ceph-mon[28330]: purged_snaps scrub ok 2024-05-03T18:16:12.697 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:12 smithi033 ceph-mon[28330]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-03T18:16:12.697 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:12 smithi033 ceph-mon[28330]: osd.3 [v2:172.21.15.33:6810/3814942724,v1:172.21.15.33:6811/3814942724] boot 2024-05-03T18:16:12.697 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:12 smithi033 ceph-mon[28330]: osdmap e25: 4 total, 4 up, 4 in 2024-05-03T18:16:12.697 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:12 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-03T18:16:12.697 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:12 smithi033 ceph-mon[28330]: osdmap e26: 4 total, 4 up, 4 in 2024-05-03T18:16:12.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:12 smithi099 ceph-mon[31658]: purged_snaps scrub starts 2024-05-03T18:16:12.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:12 smithi099 ceph-mon[31658]: purged_snaps scrub ok 2024-05-03T18:16:12.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:12 smithi099 ceph-mon[31658]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-03T18:16:12.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:12 smithi099 ceph-mon[31658]: osd.3 [v2:172.21.15.33:6810/3814942724,v1:172.21.15.33:6811/3814942724] boot 2024-05-03T18:16:12.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:12 smithi099 ceph-mon[31658]: osdmap e25: 4 total, 4 up, 4 in 2024-05-03T18:16:12.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:12 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-03T18:16:12.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:12 smithi099 ceph-mon[31658]: osdmap e26: 4 total, 4 up, 4 in 2024-05-03T18:16:13.881 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:13 smithi150 ceph-mon[32492]: osdmap e27: 4 total, 4 up, 4 in 2024-05-03T18:16:13.881 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:13 smithi150 ceph-mon[32492]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 150 KiB/s, 0 objects/s recovering 2024-05-03T18:16:13.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:13 smithi033 ceph-mon[28330]: osdmap e27: 4 total, 4 up, 4 in 2024-05-03T18:16:13.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:13 smithi033 ceph-mon[28330]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 150 KiB/s, 0 objects/s recovering 2024-05-03T18:16:14.044 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:13 smithi099 ceph-mon[31658]: osdmap e27: 4 total, 4 up, 4 in 2024-05-03T18:16:14.044 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:13 smithi099 ceph-mon[31658]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 150 KiB/s, 0 objects/s recovering 2024-05-03T18:16:14.397 INFO:teuthology.orchestra.run.smithi099.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-03T18:16:14.397 INFO:teuthology.orchestra.run.smithi099.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-03T18:16:14.398 INFO:teuthology.orchestra.run.smithi099.stderr: stderr: 10+0 records in 2024-05-03T18:16:14.398 INFO:teuthology.orchestra.run.smithi099.stderr:10+0 records out 2024-05-03T18:16:14.398 INFO:teuthology.orchestra.run.smithi099.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0549106 s, 191 MB/s 2024-05-03T18:16:14.398 INFO:teuthology.orchestra.run.smithi099.stderr:--> Zapping successful for: 2024-05-03T18:16:15.247 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi099:vg_nvme/lv_3 2024-05-03T18:16:15.675 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.b/config 2024-05-03T18:16:16.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:16 smithi099 ceph-mon[31658]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 119 KiB/s, 0 objects/s recovering 2024-05-03T18:16:16.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:16 smithi099 ceph-mon[31658]: Detected new or changed devices on smithi033 2024-05-03T18:16:16.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:16 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:16:16.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:16 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:16:16.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:16 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:16:16.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:16 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:16:16.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:16 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:16:16.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:16 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:16:16.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:16 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:16:16.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:16 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:16:16.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:16 smithi150 ceph-mon[32492]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 119 KiB/s, 0 objects/s recovering 2024-05-03T18:16:16.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:16 smithi150 ceph-mon[32492]: Detected new or changed devices on smithi033 2024-05-03T18:16:16.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:16 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:16:16.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:16 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:16:16.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:16 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:16:16.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:16 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:16:16.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:16 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:16:16.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:16 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:16:16.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:16 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:16:16.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:16 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:16:16.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:16 smithi033 ceph-mon[28330]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 119 KiB/s, 0 objects/s recovering 2024-05-03T18:16:16.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:16 smithi033 ceph-mon[28330]: Detected new or changed devices on smithi033 2024-05-03T18:16:16.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:16 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:16:16.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:16 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:16:16.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:16 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:16:16.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:16 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:16:16.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:16 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:16:16.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:16 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:16:16.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:16 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:16:16.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:16 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:16:17.453 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:17 smithi099 ceph-mon[31658]: Adjusting osd_memory_target on smithi033 to 8547M 2024-05-03T18:16:17.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:17 smithi150 ceph-mon[32492]: Adjusting osd_memory_target on smithi033 to 8547M 2024-05-03T18:16:17.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:17 smithi033 ceph-mon[28330]: Adjusting osd_memory_target on smithi033 to 8547M 2024-05-03T18:16:17.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:16:17.966+0000 7f252f600700 1 -- 172.21.15.99:0/735695415 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f252810b3e0 msgr2=0x7f252810d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:16:17.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:16:17.966+0000 7f252f600700 1 --2- 172.21.15.99:0/735695415 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f252810b3e0 0x7f252810d870 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f252000b0a0 tx=0x7f252000b3b0 comp rx=0 tx=0).stop 2024-05-03T18:16:17.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:16:17.966+0000 7f252f600700 1 -- 172.21.15.99:0/735695415 shutdown_connections 2024-05-03T18:16:17.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:16:17.966+0000 7f252f600700 1 --2- 172.21.15.99:0/735695415 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f252810b3e0 0x7f252810d870 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:16:17.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:16:17.966+0000 7f252f600700 1 --2- 172.21.15.99:0/735695415 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25280774d0 0x7f2528075930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:16:17.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:16:17.966+0000 7f252f600700 1 --2- 172.21.15.99:0/735695415 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2528075f00 0x7f2528076380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:16:17.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:16:17.966+0000 7f252f600700 1 -- 172.21.15.99:0/735695415 >> 172.21.15.99:0/735695415 conn(0x7f25280fea70 msgr2=0x7f2528100ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:16:17.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:16:17.966+0000 7f252f600700 1 -- 172.21.15.99:0/735695415 shutdown_connections 2024-05-03T18:16:17.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:16:17.966+0000 7f252f600700 1 -- 172.21.15.99:0/735695415 wait complete. 2024-05-03T18:16:17.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:16:17.968+0000 7f252f600700 1 Processor -- start 2024-05-03T18:16:17.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:16:17.968+0000 7f252f600700 1 -- start start 2024-05-03T18:16:17.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:16:17.968+0000 7f252f600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2528075f00 0x7f252819b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:16:17.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:16:17.968+0000 7f252f600700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25280774d0 0x7f252819bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:16:17.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:16:17.968+0000 7f252f600700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f252810b3e0 0x7f25281a2c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:16:17.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:16:17.968+0000 7f252f600700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2528111070 con 0x7f252810b3e0 2024-05-03T18:16:17.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:16:17.968+0000 7f252f600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2528110ef0 con 0x7f2528075f00 2024-05-03T18:16:17.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:16:17.968+0000 7f252f600700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f25281111f0 con 0x7f25280774d0 2024-05-03T18:16:17.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:16:17.969+0000 7f252cc00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2528075f00 0x7f252819b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:16:17.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:16:17.969+0000 7f252cc00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2528075f00 0x7f252819b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50110/0 (socket says 172.21.15.99:50110) 2024-05-03T18:16:17.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:16:17.969+0000 7f252cc00700 1 -- 172.21.15.99:0/1191535068 learned_addr learned my addr 172.21.15.99:0/1191535068 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-03T18:16:17.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:16:17.969+0000 7f252d600700 1 --2- 172.21.15.99:0/1191535068 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f252810b3e0 0x7f25281a2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:16:17.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:16:17.969+0000 7f2527e00700 1 --2- 172.21.15.99:0/1191535068 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25280774d0 0x7f252819bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:16:17.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:16:17.969+0000 7f252cc00700 1 -- 172.21.15.99:0/1191535068 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25280774d0 msgr2=0x7f252819bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:16:17.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:16:17.969+0000 7f252cc00700 1 --2- 172.21.15.99:0/1191535068 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25280774d0 0x7f252819bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:16:17.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:16:17.969+0000 7f252cc00700 1 -- 172.21.15.99:0/1191535068 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f252810b3e0 msgr2=0x7f25281a2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:16:17.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:16:17.969+0000 7f252cc00700 1 --2- 172.21.15.99:0/1191535068 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f252810b3e0 0x7f25281a2c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:16:17.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:16:17.969+0000 7f252cc00700 1 -- 172.21.15.99:0/1191535068 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25281a32d0 con 0x7f2528075f00 2024-05-03T18:16:17.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:16:17.969+0000 7f252d600700 1 --2- 172.21.15.99:0/1191535068 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f252810b3e0 0x7f25281a2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:16:17.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:16:17.969+0000 7f252cc00700 1 --2- 172.21.15.99:0/1191535068 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2528075f00 0x7f252819b670 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f251c00d950 tx=0x7f251c00dd10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:16:17.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:16:17.970+0000 7f2525600700 1 -- 172.21.15.99:0/1191535068 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f251c017070 con 0x7f2528075f00 2024-05-03T18:16:17.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:16:17.970+0000 7f2525600700 1 -- 172.21.15.99:0/1191535068 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f251c00de30 con 0x7f2528075f00 2024-05-03T18:16:17.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:16:17.970+0000 7f252f600700 1 -- 172.21.15.99:0/1191535068 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25281a35c0 con 0x7f2528075f00 2024-05-03T18:16:17.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:16:17.970+0000 7f2525600700 1 -- 172.21.15.99:0/1191535068 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f251c0053a0 con 0x7f2528075f00 2024-05-03T18:16:17.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:16:17.970+0000 7f252f600700 1 -- 172.21.15.99:0/1191535068 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25281a3ab0 con 0x7f2528075f00 2024-05-03T18:16:17.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:16:17.971+0000 7f252f600700 1 -- 172.21.15.99:0/1191535068 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f252806a190 con 0x7f2528075f00 2024-05-03T18:16:17.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:16:17.975+0000 7f2525600700 1 -- 172.21.15.99:0/1191535068 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f251c005540 con 0x7f2528075f00 2024-05-03T18:16:17.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:16:17.976+0000 7f2525600700 1 --2- 172.21.15.99:0/1191535068 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f25100b2900 0x7f25100b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:16:17.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:16:17.977+0000 7f2525600700 1 -- 172.21.15.99:0/1191535068 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 3669+0+0 (secure 0 0 0) 0x7f251c0e0d90 con 0x7f2528075f00 2024-05-03T18:16:17.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:16:17.977+0000 7f2527e00700 1 --2- 172.21.15.99:0/1191535068 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f25100b2900 0x7f25100b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:16:17.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:16:17.978+0000 7f2527e00700 1 --2- 172.21.15.99:0/1191535068 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f25100b2900 0x7f25100b4dc0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2528077200 tx=0x7f2514005d70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:16:17.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:16:17.979+0000 7f2525600700 1 -- 172.21.15.99:0/1191535068 <== mon.2 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f251c0a9c30 con 0x7f2528075f00 2024-05-03T18:16:18.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:16:18.221+0000 7f252f600700 1 -- 172.21.15.99:0/1191535068 --> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f25280638e0 con 0x7f25100b2900 2024-05-03T18:16:18.544 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:18 smithi099 ceph-mon[31658]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-03T18:16:18.544 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:18 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T18:16:18.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:18 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T18:16:18.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:18 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:16:18.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:18 smithi150 ceph-mon[32492]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-03T18:16:18.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:18 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T18:16:18.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:18 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T18:16:18.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:18 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:16:18.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:18 smithi033 ceph-mon[28330]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-03T18:16:18.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:18 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T18:16:18.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:18 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T18:16:18.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:18 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:16:19.472 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:19 smithi099 ceph-mon[31658]: from='client.34214 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:16:19.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:19 smithi150 ceph-mon[32492]: from='client.34214 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:16:19.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:19 smithi033 ceph-mon[28330]: from='client.34214 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:16:20.544 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:20 smithi099 ceph-mon[31658]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-05-03T18:16:20.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:20 smithi150 ceph-mon[32492]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-05-03T18:16:20.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:20 smithi033 ceph-mon[28330]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-05-03T18:16:22.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:22 smithi099 ceph-mon[31658]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-05-03T18:16:22.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:22 smithi099 ceph-mon[31658]: from='client.? 172.21.15.99:0/2972000772' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1235e1a6-ff6e-4668-ac1a-073c2a3b7c98"}]: dispatch 2024-05-03T18:16:22.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:22 smithi099 ceph-mon[31658]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1235e1a6-ff6e-4668-ac1a-073c2a3b7c98"}]: dispatch 2024-05-03T18:16:22.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:22 smithi099 ceph-mon[31658]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1235e1a6-ff6e-4668-ac1a-073c2a3b7c98"}]': finished 2024-05-03T18:16:22.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:22 smithi099 ceph-mon[31658]: osdmap e28: 5 total, 4 up, 5 in 2024-05-03T18:16:22.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:22 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-03T18:16:22.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:22 smithi150 ceph-mon[32492]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-05-03T18:16:22.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:22 smithi150 ceph-mon[32492]: from='client.? 172.21.15.99:0/2972000772' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1235e1a6-ff6e-4668-ac1a-073c2a3b7c98"}]: dispatch 2024-05-03T18:16:22.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:22 smithi150 ceph-mon[32492]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1235e1a6-ff6e-4668-ac1a-073c2a3b7c98"}]: dispatch 2024-05-03T18:16:22.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:22 smithi150 ceph-mon[32492]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1235e1a6-ff6e-4668-ac1a-073c2a3b7c98"}]': finished 2024-05-03T18:16:22.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:22 smithi150 ceph-mon[32492]: osdmap e28: 5 total, 4 up, 5 in 2024-05-03T18:16:22.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:22 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-03T18:16:22.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:22 smithi033 ceph-mon[28330]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-05-03T18:16:22.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:22 smithi033 ceph-mon[28330]: from='client.? 172.21.15.99:0/2972000772' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1235e1a6-ff6e-4668-ac1a-073c2a3b7c98"}]: dispatch 2024-05-03T18:16:22.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:22 smithi033 ceph-mon[28330]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1235e1a6-ff6e-4668-ac1a-073c2a3b7c98"}]: dispatch 2024-05-03T18:16:22.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:22 smithi033 ceph-mon[28330]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1235e1a6-ff6e-4668-ac1a-073c2a3b7c98"}]': finished 2024-05-03T18:16:22.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:22 smithi033 ceph-mon[28330]: osdmap e28: 5 total, 4 up, 5 in 2024-05-03T18:16:22.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:22 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-03T18:16:23.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:23 smithi150 ceph-mon[32492]: from='client.? 172.21.15.99:0/3460161317' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T18:16:23.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:23 smithi033 ceph-mon[28330]: from='client.? 172.21.15.99:0/3460161317' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T18:16:23.794 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:23 smithi099 ceph-mon[31658]: from='client.? 172.21.15.99:0/3460161317' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T18:16:24.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:24 smithi150 ceph-mon[32492]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-03T18:16:24.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:24 smithi033 ceph-mon[28330]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-03T18:16:24.794 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:24 smithi099 ceph-mon[31658]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-03T18:16:26.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:26 smithi150 ceph-mon[32492]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-03T18:16:26.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:26 smithi033 ceph-mon[28330]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-03T18:16:26.773 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:26 smithi099 ceph-mon[31658]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-03T18:16:28.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:28 smithi099 ceph-mon[31658]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-03T18:16:28.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:28 smithi150 ceph-mon[32492]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-03T18:16:28.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:28 smithi033 ceph-mon[28330]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-03T18:16:30.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:30 smithi150 ceph-mon[32492]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-03T18:16:30.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:30 smithi033 ceph-mon[28330]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-03T18:16:30.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:30 smithi099 ceph-mon[31658]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-03T18:16:32.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:32 smithi150 ceph-mon[32492]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-03T18:16:32.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:32 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-03T18:16:32.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:32 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:16:32.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:32 smithi150 ceph-mon[32492]: Deploying daemon osd.4 on smithi099 2024-05-03T18:16:32.703 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:32 smithi099 ceph-mon[31658]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-03T18:16:32.703 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:32 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-03T18:16:32.704 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:32 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:16:32.704 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:32 smithi099 ceph-mon[31658]: Deploying daemon osd.4 on smithi099 2024-05-03T18:16:32.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:32 smithi033 ceph-mon[28330]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-03T18:16:32.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:32 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-03T18:16:32.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:32 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:16:32.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:32 smithi033 ceph-mon[28330]: Deploying daemon osd.4 on smithi099 2024-05-03T18:16:34.544 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:34 smithi099 ceph-mon[31658]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-03T18:16:34.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:34 smithi150 ceph-mon[32492]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-03T18:16:34.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:34 smithi033 ceph-mon[28330]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-03T18:16:36.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:36 smithi099 ceph-mon[31658]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-03T18:16:36.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:36 smithi150 ceph-mon[32492]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-03T18:16:36.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:36 smithi033 ceph-mon[28330]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-03T18:16:38.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:38 smithi099 ceph-mon[31658]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-03T18:16:38.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:38 smithi150 ceph-mon[32492]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-03T18:16:38.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:38 smithi033 ceph-mon[28330]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-03T18:16:40.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:40 smithi150 ceph-mon[32492]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-03T18:16:40.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:40 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:16:40.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:40 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:16:40.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:40 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:16:40.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:40 smithi033 ceph-mon[28330]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-03T18:16:40.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:40 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:16:40.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:40 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:16:40.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:40 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:16:40.739 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:40 smithi099 ceph-mon[31658]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-03T18:16:40.739 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:40 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:16:40.739 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:40 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:16:40.739 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:40 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:16:42.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:42 smithi150 ceph-mon[32492]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-03T18:16:42.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:42 smithi033 ceph-mon[28330]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-03T18:16:42.761 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:42 smithi099 ceph-mon[31658]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-03T18:16:44.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:44 smithi099 ceph-mon[31658]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-03T18:16:44.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:44 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:16:44.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:44 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:16:44.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:44 smithi099 ceph-mon[31658]: from='osd.4 [v2:172.21.15.99:6808/29081029,v1:172.21.15.99:6809/29081029]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-03T18:16:44.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:44 smithi099 ceph-mon[31658]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-03T18:16:44.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:44 smithi150 ceph-mon[32492]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-03T18:16:44.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:44 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:16:44.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:44 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:16:44.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:44 smithi150 ceph-mon[32492]: from='osd.4 [v2:172.21.15.99:6808/29081029,v1:172.21.15.99:6809/29081029]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-03T18:16:44.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:44 smithi150 ceph-mon[32492]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-03T18:16:44.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:44 smithi033 ceph-mon[28330]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-03T18:16:44.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:44 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:16:44.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:44 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:16:44.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:44 smithi033 ceph-mon[28330]: from='osd.4 [v2:172.21.15.99:6808/29081029,v1:172.21.15.99:6809/29081029]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-03T18:16:44.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:44 smithi033 ceph-mon[28330]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-03T18:16:44.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:16:44.959+0000 7f2525600700 1 -- 172.21.15.99:0/1191535068 <== mgr.14152 v2:172.21.15.33:6800/3642895501 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f25280638e0 con 0x7f25100b2900 2024-05-03T18:16:44.959 INFO:teuthology.orchestra.run.smithi099.stdout:Created osd(s) 4 on host 'smithi099' 2024-05-03T18:16:44.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:16:44.963+0000 7f252f600700 1 -- 172.21.15.99:0/1191535068 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f25100b2900 msgr2=0x7f25100b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:16:44.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:16:44.963+0000 7f252f600700 1 --2- 172.21.15.99:0/1191535068 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f25100b2900 0x7f25100b4dc0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2528077200 tx=0x7f2514005d70 comp rx=0 tx=0).stop 2024-05-03T18:16:44.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:16:44.963+0000 7f252f600700 1 -- 172.21.15.99:0/1191535068 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2528075f00 msgr2=0x7f252819b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:16:44.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:16:44.963+0000 7f252f600700 1 --2- 172.21.15.99:0/1191535068 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2528075f00 0x7f252819b670 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f251c00d950 tx=0x7f251c00dd10 comp rx=0 tx=0).stop 2024-05-03T18:16:44.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:16:44.963+0000 7f252f600700 1 -- 172.21.15.99:0/1191535068 shutdown_connections 2024-05-03T18:16:44.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:16:44.963+0000 7f252f600700 1 --2- 172.21.15.99:0/1191535068 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f252810b3e0 0x7f25281a2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:16:44.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:16:44.963+0000 7f252f600700 1 --2- 172.21.15.99:0/1191535068 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f25100b2900 0x7f25100b4dc0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:16:44.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:16:44.963+0000 7f252f600700 1 --2- 172.21.15.99:0/1191535068 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2528075f00 0x7f252819b670 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:16:44.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:16:44.963+0000 7f252f600700 1 --2- 172.21.15.99:0/1191535068 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25280774d0 0x7f252819bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:16:44.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:16:44.963+0000 7f252f600700 1 -- 172.21.15.99:0/1191535068 >> 172.21.15.99:0/1191535068 conn(0x7f25280fea70 msgr2=0x7f25281072e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:16:44.964 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:16:44.964+0000 7f252f600700 1 -- 172.21.15.99:0/1191535068 shutdown_connections 2024-05-03T18:16:44.964 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:16:44.964+0000 7f252f600700 1 -- 172.21.15.99:0/1191535068 wait complete. 2024-05-03T18:16:45.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:45 smithi150 ceph-mon[32492]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-03T18:16:45.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:45 smithi150 ceph-mon[32492]: osdmap e29: 5 total, 4 up, 5 in 2024-05-03T18:16:45.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:45 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-03T18:16:45.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:45 smithi150 ceph-mon[32492]: from='osd.4 [v2:172.21.15.99:6808/29081029,v1:172.21.15.99:6809/29081029]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-05-03T18:16:45.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:45 smithi150 ceph-mon[32492]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-05-03T18:16:45.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:45 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:16:45.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:45 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:16:45.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:45 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:16:45.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:45 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:16:45.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:45 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:16:45.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:45 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:16:45.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:45 smithi033 ceph-mon[28330]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-03T18:16:45.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:45 smithi033 ceph-mon[28330]: osdmap e29: 5 total, 4 up, 5 in 2024-05-03T18:16:45.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:45 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-03T18:16:45.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:45 smithi033 ceph-mon[28330]: from='osd.4 [v2:172.21.15.99:6808/29081029,v1:172.21.15.99:6809/29081029]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-05-03T18:16:45.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:45 smithi033 ceph-mon[28330]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-05-03T18:16:45.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:45 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:16:45.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:45 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:16:45.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:45 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:16:45.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:45 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:16:45.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:45 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:16:45.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:45 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:16:45.739 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:45 smithi099 ceph-mon[31658]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-03T18:16:45.739 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:45 smithi099 ceph-mon[31658]: osdmap e29: 5 total, 4 up, 5 in 2024-05-03T18:16:45.739 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:45 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-03T18:16:45.739 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:45 smithi099 ceph-mon[31658]: from='osd.4 [v2:172.21.15.99:6808/29081029,v1:172.21.15.99:6809/29081029]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-05-03T18:16:45.739 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:45 smithi099 ceph-mon[31658]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-05-03T18:16:45.739 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:45 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:16:45.740 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:45 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:16:45.740 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:45 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:16:45.740 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:45 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:16:45.740 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:45 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:16:45.740 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:45 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:16:45.798 DEBUG:teuthology.orchestra.run.smithi099:osd.4> sudo journalctl -f -n 0 -u ceph-5703bba4-0978-11ef-bc97-c7b262605968@osd.4.service 2024-05-03T18:16:45.802 INFO:tasks.cephadm:Deploying osd.5 on smithi150 with /dev/vg_nvme/lv_3... 2024-05-03T18:16:45.803 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-03T18:16:46.252 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.c/config 2024-05-03T18:16:46.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:46 smithi099 ceph-mon[31658]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-03T18:16:46.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:46 smithi099 ceph-mon[31658]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-05-03T18:16:46.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:46 smithi099 ceph-mon[31658]: osdmap e30: 5 total, 4 up, 5 in 2024-05-03T18:16:46.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:46 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-03T18:16:46.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:46 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-03T18:16:46.719 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:46 smithi150 ceph-mon[32492]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-03T18:16:46.720 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:46 smithi150 ceph-mon[32492]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-05-03T18:16:46.720 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:46 smithi150 ceph-mon[32492]: osdmap e30: 5 total, 4 up, 5 in 2024-05-03T18:16:46.720 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:46 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-03T18:16:46.720 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:46 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-03T18:16:46.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:46 smithi033 ceph-mon[28330]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-03T18:16:46.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:46 smithi033 ceph-mon[28330]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-05-03T18:16:46.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:46 smithi033 ceph-mon[28330]: osdmap e30: 5 total, 4 up, 5 in 2024-05-03T18:16:46.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:46 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-03T18:16:46.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:46 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-03T18:16:47.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:47 smithi150 ceph-mon[32492]: purged_snaps scrub starts 2024-05-03T18:16:47.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:47 smithi150 ceph-mon[32492]: purged_snaps scrub ok 2024-05-03T18:16:47.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:47 smithi150 ceph-mon[32492]: osd.4 [v2:172.21.15.99:6808/29081029,v1:172.21.15.99:6809/29081029] boot 2024-05-03T18:16:47.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:47 smithi150 ceph-mon[32492]: osdmap e31: 5 total, 5 up, 5 in 2024-05-03T18:16:47.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:47 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-03T18:16:47.692 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:47 smithi099 ceph-mon[31658]: purged_snaps scrub starts 2024-05-03T18:16:47.693 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:47 smithi099 ceph-mon[31658]: purged_snaps scrub ok 2024-05-03T18:16:47.693 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:47 smithi099 ceph-mon[31658]: osd.4 [v2:172.21.15.99:6808/29081029,v1:172.21.15.99:6809/29081029] boot 2024-05-03T18:16:47.693 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:47 smithi099 ceph-mon[31658]: osdmap e31: 5 total, 5 up, 5 in 2024-05-03T18:16:47.693 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:47 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-03T18:16:47.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:47 smithi033 ceph-mon[28330]: purged_snaps scrub starts 2024-05-03T18:16:47.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:47 smithi033 ceph-mon[28330]: purged_snaps scrub ok 2024-05-03T18:16:47.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:47 smithi033 ceph-mon[28330]: osd.4 [v2:172.21.15.99:6808/29081029,v1:172.21.15.99:6809/29081029] boot 2024-05-03T18:16:47.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:47 smithi033 ceph-mon[28330]: osdmap e31: 5 total, 5 up, 5 in 2024-05-03T18:16:47.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:47 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-03T18:16:48.582 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:48 smithi150 ceph-mon[32492]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-03T18:16:48.582 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:48 smithi150 ceph-mon[32492]: osdmap e32: 5 total, 5 up, 5 in 2024-05-03T18:16:48.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:48 smithi033 ceph-mon[28330]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-03T18:16:48.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:48 smithi033 ceph-mon[28330]: osdmap e32: 5 total, 5 up, 5 in 2024-05-03T18:16:48.794 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:48 smithi099 ceph-mon[31658]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-03T18:16:48.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:48 smithi099 ceph-mon[31658]: osdmap e32: 5 total, 5 up, 5 in 2024-05-03T18:16:49.176 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-03T18:16:49.177 INFO:teuthology.orchestra.run.smithi150.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-03T18:16:49.177 INFO:teuthology.orchestra.run.smithi150.stderr: stderr: 10+0 records in 2024-05-03T18:16:49.177 INFO:teuthology.orchestra.run.smithi150.stderr:10+0 records out 2024-05-03T18:16:49.177 INFO:teuthology.orchestra.run.smithi150.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0316956 s, 331 MB/s 2024-05-03T18:16:49.177 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping successful for: 2024-05-03T18:16:50.052 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi150:vg_nvme/lv_3 2024-05-03T18:16:50.483 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.c/config 2024-05-03T18:16:50.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:50 smithi033 ceph-mon[28330]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-03T18:16:50.786 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:50 smithi150 ceph-mon[32492]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-03T18:16:50.794 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:50 smithi099 ceph-mon[31658]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-03T18:16:51.768 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:51 smithi150 ceph-mon[32492]: Detected new or changed devices on smithi099 2024-05-03T18:16:51.768 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:51 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:16:51.768 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:51 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:16:51.769 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:51 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:16:51.769 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:51 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:16:51.769 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:51 smithi150 ceph-mon[32492]: Adjusting osd_memory_target on smithi099 to 8544M 2024-05-03T18:16:51.769 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:51 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:16:51.769 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:51 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:16:51.769 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:51 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:16:51.769 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:51 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:16:51.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:51 smithi033 ceph-mon[28330]: Detected new or changed devices on smithi099 2024-05-03T18:16:51.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:51 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:16:51.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:51 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:16:51.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:51 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:16:51.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:51 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:16:51.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:51 smithi033 ceph-mon[28330]: Adjusting osd_memory_target on smithi099 to 8544M 2024-05-03T18:16:51.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:51 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:16:51.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:51 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:16:51.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:51 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:16:51.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:51 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:16:52.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:51 smithi099 ceph-mon[31658]: Detected new or changed devices on smithi099 2024-05-03T18:16:52.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:51 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:16:52.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:51 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:16:52.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:51 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:16:52.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:51 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:16:52.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:51 smithi099 ceph-mon[31658]: Adjusting osd_memory_target on smithi099 to 8544M 2024-05-03T18:16:52.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:51 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:16:52.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:51 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:16:52.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:51 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:16:52.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:51 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:16:52.610 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:16:52.610+0000 7fb5a3e00700 1 -- 172.21.15.150:0/3857570746 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb5a410b5d0 msgr2=0x7fb5a4111e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:16:52.610 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:16:52.610+0000 7fb5a3e00700 1 --2- 172.21.15.150:0/3857570746 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb5a410b5d0 0x7fb5a4111e80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb59c009990 tx=0x7fb59c009ca0 comp rx=0 tx=0).stop 2024-05-03T18:16:52.610 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:16:52.610+0000 7fb5a3e00700 1 -- 172.21.15.150:0/3857570746 shutdown_connections 2024-05-03T18:16:52.610 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:16:52.610+0000 7fb5a3e00700 1 --2- 172.21.15.150:0/3857570746 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb5a410b5d0 0x7fb5a4111e80 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:16:52.610 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:16:52.610+0000 7fb5a3e00700 1 --2- 172.21.15.150:0/3857570746 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5a4109a30 0x7fb5a4109e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:16:52.610 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:16:52.610+0000 7fb5a3e00700 1 --2- 172.21.15.150:0/3857570746 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5a410ac30 0x7fb5a410b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:16:52.610 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:16:52.610+0000 7fb5a3e00700 1 -- 172.21.15.150:0/3857570746 >> 172.21.15.150:0/3857570746 conn(0x7fb5a4078a00 msgr2=0x7fb5a407ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:16:52.611 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:16:52.610+0000 7fb5a3e00700 1 -- 172.21.15.150:0/3857570746 shutdown_connections 2024-05-03T18:16:52.611 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:16:52.610+0000 7fb5a3e00700 1 -- 172.21.15.150:0/3857570746 wait complete. 2024-05-03T18:16:52.612 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:16:52.611+0000 7fb5a3e00700 1 Processor -- start 2024-05-03T18:16:52.612 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:16:52.611+0000 7fb5a3e00700 1 -- start start 2024-05-03T18:16:52.612 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:16:52.612+0000 7fb5a3e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5a4109a30 0x7fb5a419fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:16:52.612 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:16:52.612+0000 7fb5a3e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5a410ac30 0x7fb5a419ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:16:52.613 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:16:52.612+0000 7fb5a3e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb5a410b5d0 0x7fb5a41a6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:16:52.613 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:16:52.612+0000 7fb5a3e00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb5a41153b0 con 0x7fb5a410ac30 2024-05-03T18:16:52.613 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:16:52.612+0000 7fb5a3e00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb5a4115230 con 0x7fb5a4109a30 2024-05-03T18:16:52.613 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:16:52.612+0000 7fb5a3e00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb5a4115530 con 0x7fb5a410b5d0 2024-05-03T18:16:52.613 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:16:52.612+0000 7fb5a3400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb5a410b5d0 0x7fb5a41a6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:16:52.613 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:16:52.612+0000 7fb5a3400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb5a410b5d0 0x7fb5a41a6fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:33840/0 (socket says 172.21.15.150:33840) 2024-05-03T18:16:52.613 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:16:52.612+0000 7fb5a3400700 1 -- 172.21.15.150:0/4008711839 learned_addr learned my addr 172.21.15.150:0/4008711839 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-03T18:16:52.613 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:16:52.612+0000 7fb59a000700 1 --2- 172.21.15.150:0/4008711839 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5a410ac30 0x7fb5a419ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:16:52.613 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:16:52.612+0000 7fb5a2a00700 1 --2- 172.21.15.150:0/4008711839 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5a4109a30 0x7fb5a419fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:16:52.613 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:16:52.612+0000 7fb5a3400700 1 -- 172.21.15.150:0/4008711839 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5a4109a30 msgr2=0x7fb5a419fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:16:52.613 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:16:52.612+0000 7fb5a3400700 1 --2- 172.21.15.150:0/4008711839 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5a4109a30 0x7fb5a419fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:16:52.613 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:16:52.612+0000 7fb5a3400700 1 -- 172.21.15.150:0/4008711839 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5a410ac30 msgr2=0x7fb5a419ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:16:52.613 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:16:52.612+0000 7fb5a3400700 1 --2- 172.21.15.150:0/4008711839 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5a410ac30 0x7fb5a419ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:16:52.614 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:16:52.612+0000 7fb5a3400700 1 -- 172.21.15.150:0/4008711839 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5a41a7680 con 0x7fb5a410b5d0 2024-05-03T18:16:52.614 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:16:52.612+0000 7fb59a000700 1 --2- 172.21.15.150:0/4008711839 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5a410ac30 0x7fb5a419ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:16:52.614 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:16:52.613+0000 7fb5a3400700 1 --2- 172.21.15.150:0/4008711839 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb5a410b5d0 0x7fb5a41a6fe0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb59c005910 tx=0x7fb59c009ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:16:52.615 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:16:52.613+0000 7fb5a8a00700 1 -- 172.21.15.150:0/4008711839 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb59c027070 con 0x7fb5a410b5d0 2024-05-03T18:16:52.615 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:16:52.613+0000 7fb5a8a00700 1 -- 172.21.15.150:0/4008711839 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb59c004090 con 0x7fb5a410b5d0 2024-05-03T18:16:52.615 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:16:52.613+0000 7fb5a3e00700 1 -- 172.21.15.150:0/4008711839 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5a41a7910 con 0x7fb5a410b5d0 2024-05-03T18:16:52.615 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:16:52.613+0000 7fb5a8a00700 1 -- 172.21.15.150:0/4008711839 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb59c022e40 con 0x7fb5a410b5d0 2024-05-03T18:16:52.615 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:16:52.613+0000 7fb5a3e00700 1 -- 172.21.15.150:0/4008711839 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5a41a7dc0 con 0x7fb5a410b5d0 2024-05-03T18:16:52.615 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:16:52.614+0000 7fb5a3e00700 1 -- 172.21.15.150:0/4008711839 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5a406a190 con 0x7fb5a410b5d0 2024-05-03T18:16:52.616 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:16:52.617+0000 7fb5a8a00700 1 -- 172.21.15.150:0/4008711839 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fb59c02a070 con 0x7fb5a410b5d0 2024-05-03T18:16:52.618 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:16:52.618+0000 7fb5a8a00700 1 --2- 172.21.15.150:0/4008711839 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fb5880b2900 0x7fb5880b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:16:52.618 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:16:52.618+0000 7fb5a8a00700 1 -- 172.21.15.150:0/4008711839 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4101+0+0 (secure 0 0 0) 0x7fb59c0e2640 con 0x7fb5a410b5d0 2024-05-03T18:16:52.618 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:16:52.619+0000 7fb5a2a00700 1 --2- 172.21.15.150:0/4008711839 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fb5880b2900 0x7fb5880b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:16:52.619 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:16:52.620+0000 7fb5a2a00700 1 --2- 172.21.15.150:0/4008711839 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fb5880b2900 0x7fb5880b4dc0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb5a410aa90 tx=0x7fb594005cf0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:16:52.621 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:16:52.622+0000 7fb5a8a00700 1 -- 172.21.15.150:0/4008711839 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fb59c013070 con 0x7fb5a410b5d0 2024-05-03T18:16:52.868 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:16:52.868+0000 7fb5a3e00700 1 -- 172.21.15.150:0/4008711839 --> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fb5a40638e0 con 0x7fb5880b2900 2024-05-03T18:16:52.869 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:52 smithi150 ceph-mon[32492]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-03T18:16:52.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:52 smithi033 ceph-mon[28330]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-03T18:16:53.044 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:52 smithi099 ceph-mon[31658]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-03T18:16:53.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:53 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T18:16:53.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:53 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T18:16:53.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:53 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:16:54.007 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:53 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T18:16:54.007 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:53 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T18:16:54.007 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:53 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:16:54.044 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:53 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T18:16:54.044 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:53 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T18:16:54.044 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:53 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:16:54.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:54 smithi033 ceph-mon[28330]: from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:16:54.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:54 smithi033 ceph-mon[28330]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-03T18:16:55.036 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:54 smithi150 ceph-mon[32492]: from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:16:55.036 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:54 smithi150 ceph-mon[32492]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-03T18:16:55.044 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:54 smithi099 ceph-mon[31658]: from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:16:55.044 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:54 smithi099 ceph-mon[31658]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-03T18:16:55.864 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:55 smithi150 ceph-mon[32492]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-03T18:16:55.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:55 smithi033 ceph-mon[28330]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-03T18:16:56.044 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:55 smithi099 ceph-mon[31658]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-03T18:16:57.233 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:56 smithi033 ceph-mon[28330]: from='client.? 172.21.15.150:0/1718186579' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4b84bff8-d6d7-4258-a0ee-53bc19802585"}]: dispatch 2024-05-03T18:16:57.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:56 smithi033 ceph-mon[28330]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4b84bff8-d6d7-4258-a0ee-53bc19802585"}]: dispatch 2024-05-03T18:16:57.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:56 smithi033 ceph-mon[28330]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4b84bff8-d6d7-4258-a0ee-53bc19802585"}]': finished 2024-05-03T18:16:57.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:56 smithi033 ceph-mon[28330]: osdmap e33: 6 total, 5 up, 6 in 2024-05-03T18:16:57.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:56 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-03T18:16:57.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:56 smithi099 ceph-mon[31658]: from='client.? 172.21.15.150:0/1718186579' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4b84bff8-d6d7-4258-a0ee-53bc19802585"}]: dispatch 2024-05-03T18:16:57.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:56 smithi099 ceph-mon[31658]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4b84bff8-d6d7-4258-a0ee-53bc19802585"}]: dispatch 2024-05-03T18:16:57.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:56 smithi099 ceph-mon[31658]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4b84bff8-d6d7-4258-a0ee-53bc19802585"}]': finished 2024-05-03T18:16:57.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:56 smithi099 ceph-mon[31658]: osdmap e33: 6 total, 5 up, 6 in 2024-05-03T18:16:57.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:56 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-03T18:16:57.381 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:56 smithi150 ceph-mon[32492]: from='client.? 172.21.15.150:0/1718186579' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4b84bff8-d6d7-4258-a0ee-53bc19802585"}]: dispatch 2024-05-03T18:16:57.381 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:56 smithi150 ceph-mon[32492]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4b84bff8-d6d7-4258-a0ee-53bc19802585"}]: dispatch 2024-05-03T18:16:57.381 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:56 smithi150 ceph-mon[32492]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4b84bff8-d6d7-4258-a0ee-53bc19802585"}]': finished 2024-05-03T18:16:57.381 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:56 smithi150 ceph-mon[32492]: osdmap e33: 6 total, 5 up, 6 in 2024-05-03T18:16:57.381 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:56 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-03T18:16:58.294 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:57 smithi099 ceph-mon[31658]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-03T18:16:58.294 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:16:57 smithi099 ceph-mon[31658]: from='client.? 172.21.15.150:0/180143421' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T18:16:58.381 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:57 smithi150 ceph-mon[32492]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-03T18:16:58.381 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:16:57 smithi150 ceph-mon[32492]: from='client.? 172.21.15.150:0/180143421' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T18:16:58.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:57 smithi033 ceph-mon[28330]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-03T18:16:58.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:16:57 smithi033 ceph-mon[28330]: from='client.? 172.21.15.150:0/180143421' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T18:17:00.544 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:00 smithi099 ceph-mon[31658]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-03T18:17:00.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:00 smithi150 ceph-mon[32492]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-03T18:17:00.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:00 smithi033 ceph-mon[28330]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-03T18:17:02.381 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:02 smithi150 ceph-mon[32492]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-03T18:17:02.544 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:02 smithi099 ceph-mon[31658]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-03T18:17:02.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:02 smithi033 ceph-mon[28330]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-03T18:17:04.544 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:04 smithi099 ceph-mon[31658]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-03T18:17:04.627 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:04 smithi150 ceph-mon[32492]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-03T18:17:04.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:04 smithi033 ceph-mon[28330]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-03T18:17:06.544 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:06 smithi099 ceph-mon[31658]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-03T18:17:06.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:06 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-03T18:17:06.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:06 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:17:06.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:06 smithi150 ceph-mon[32492]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-03T18:17:06.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:06 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-03T18:17:06.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:06 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:17:06.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:06 smithi033 ceph-mon[28330]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-03T18:17:06.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:06 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-03T18:17:06.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:06 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:17:07.580 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:07 smithi150 ceph-mon[32492]: Deploying daemon osd.5 on smithi150 2024-05-03T18:17:07.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:07 smithi033 ceph-mon[28330]: Deploying daemon osd.5 on smithi150 2024-05-03T18:17:07.794 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:07 smithi099 ceph-mon[31658]: Deploying daemon osd.5 on smithi150 2024-05-03T18:17:08.381 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:08 smithi150 ceph-mon[32492]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-03T18:17:08.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:08 smithi033 ceph-mon[28330]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-03T18:17:08.794 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:08 smithi099 ceph-mon[31658]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-03T18:17:10.597 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:10 smithi099 ceph-mon[31658]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-03T18:17:10.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:10 smithi150 ceph-mon[32492]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-03T18:17:10.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:10 smithi033 ceph-mon[28330]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-03T18:17:12.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:12 smithi099 ceph-mon[31658]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-03T18:17:12.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:12 smithi150 ceph-mon[32492]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-03T18:17:12.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:12 smithi033 ceph-mon[28330]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-03T18:17:14.414 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:14 smithi150 ceph-mon[32492]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-03T18:17:14.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:14 smithi033 ceph-mon[28330]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-03T18:17:14.794 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:14 smithi099 ceph-mon[31658]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-03T18:17:15.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:15 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:17:15.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:15 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:17:15.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:15 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:17:15.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:15 smithi033 ceph-mon[28330]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-03T18:17:16.021 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:15 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:17:16.021 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:15 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:17:16.022 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:15 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:17:16.022 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:15 smithi150 ceph-mon[32492]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-03T18:17:16.044 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:15 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:17:16.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:15 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:17:16.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:15 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:17:16.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:15 smithi099 ceph-mon[31658]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-03T18:17:18.544 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:18 smithi099 ceph-mon[31658]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-03T18:17:18.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:18 smithi150 ceph-mon[32492]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-03T18:17:18.687 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:18 smithi033 ceph-mon[28330]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-03T18:17:19.725 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:19 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:17:19.726 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:19 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:17:19.726 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:19 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:17:19.726 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:19 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:17:19.726 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:19 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:17:19.726 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:19 smithi033 ceph-mon[28330]: from='osd.5 [v2:172.21.15.150:6808/3172189533,v1:172.21.15.150:6809/3172189533]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-03T18:17:19.726 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:19 smithi033 ceph-mon[28330]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-03T18:17:19.741 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:17:19.741+0000 7fb5a8a00700 1 -- 172.21.15.150:0/4008711839 <== mgr.14152 v2:172.21.15.33:6800/3642895501 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb5a40638e0 con 0x7fb5880b2900 2024-05-03T18:17:19.741 INFO:teuthology.orchestra.run.smithi150.stdout:Created osd(s) 5 on host 'smithi150' 2024-05-03T18:17:19.742 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:19 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:17:19.742 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:19 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:17:19.742 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:19 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:17:19.742 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:19 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:17:19.742 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:19 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:17:19.742 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:19 smithi150 ceph-mon[32492]: from='osd.5 [v2:172.21.15.150:6808/3172189533,v1:172.21.15.150:6809/3172189533]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-03T18:17:19.742 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:19 smithi150 ceph-mon[32492]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-03T18:17:19.745 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:17:19.745+0000 7fb5a3e00700 1 -- 172.21.15.150:0/4008711839 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fb5880b2900 msgr2=0x7fb5880b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:17:19.745 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:17:19.745+0000 7fb5a3e00700 1 --2- 172.21.15.150:0/4008711839 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fb5880b2900 0x7fb5880b4dc0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb5a410aa90 tx=0x7fb594005cf0 comp rx=0 tx=0).stop 2024-05-03T18:17:19.745 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:17:19.745+0000 7fb5a3e00700 1 -- 172.21.15.150:0/4008711839 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb5a410b5d0 msgr2=0x7fb5a41a6fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:17:19.745 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:17:19.745+0000 7fb5a3e00700 1 --2- 172.21.15.150:0/4008711839 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb5a410b5d0 0x7fb5a41a6fe0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb59c005910 tx=0x7fb59c009ee0 comp rx=0 tx=0).stop 2024-05-03T18:17:19.746 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:17:19.745+0000 7fb5a3e00700 1 -- 172.21.15.150:0/4008711839 shutdown_connections 2024-05-03T18:17:19.746 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:17:19.745+0000 7fb5a3e00700 1 --2- 172.21.15.150:0/4008711839 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb5a410b5d0 0x7fb5a41a6fe0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:17:19.746 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:17:19.745+0000 7fb5a3e00700 1 --2- 172.21.15.150:0/4008711839 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fb5880b2900 0x7fb5880b4dc0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:17:19.746 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:17:19.745+0000 7fb5a3e00700 1 --2- 172.21.15.150:0/4008711839 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5a4109a30 0x7fb5a419fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:17:19.746 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:17:19.745+0000 7fb5a3e00700 1 --2- 172.21.15.150:0/4008711839 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5a410ac30 0x7fb5a419ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:17:19.747 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:17:19.745+0000 7fb5a3e00700 1 -- 172.21.15.150:0/4008711839 >> 172.21.15.150:0/4008711839 conn(0x7fb5a4078a00 msgr2=0x7fb5a410df50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:17:19.747 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:17:19.746+0000 7fb5a3e00700 1 -- 172.21.15.150:0/4008711839 shutdown_connections 2024-05-03T18:17:19.747 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:17:19.746+0000 7fb5a3e00700 1 -- 172.21.15.150:0/4008711839 wait complete. 2024-05-03T18:17:19.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:19 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:17:19.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:19 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:17:19.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:19 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:17:19.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:19 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:17:19.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:19 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:17:19.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:19 smithi099 ceph-mon[31658]: from='osd.5 [v2:172.21.15.150:6808/3172189533,v1:172.21.15.150:6809/3172189533]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-03T18:17:19.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:19 smithi099 ceph-mon[31658]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-03T18:17:20.562 DEBUG:teuthology.orchestra.run.smithi150:osd.5> sudo journalctl -f -n 0 -u ceph-5703bba4-0978-11ef-bc97-c7b262605968@osd.5.service 2024-05-03T18:17:20.567 INFO:tasks.cephadm:Deploying osd.6 on smithi033 with /dev/vg_nvme/lv_2... 2024-05-03T18:17:20.567 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-03T18:17:20.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:20 smithi033 ceph-mon[28330]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-03T18:17:20.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:20 smithi033 ceph-mon[28330]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-03T18:17:20.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:20 smithi033 ceph-mon[28330]: from='osd.5 [v2:172.21.15.150:6808/3172189533,v1:172.21.15.150:6809/3172189533]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-03T18:17:20.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:20 smithi033 ceph-mon[28330]: osdmap e34: 6 total, 5 up, 6 in 2024-05-03T18:17:20.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:20 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-03T18:17:20.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:20 smithi033 ceph-mon[28330]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-03T18:17:20.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:20 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:17:20.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:20 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:17:20.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:20 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:17:20.766 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:20 smithi150 ceph-mon[32492]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-03T18:17:20.766 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:20 smithi150 ceph-mon[32492]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-03T18:17:20.766 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:20 smithi150 ceph-mon[32492]: from='osd.5 [v2:172.21.15.150:6808/3172189533,v1:172.21.15.150:6809/3172189533]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-03T18:17:20.766 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:20 smithi150 ceph-mon[32492]: osdmap e34: 6 total, 5 up, 6 in 2024-05-03T18:17:20.766 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:20 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-03T18:17:20.766 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:20 smithi150 ceph-mon[32492]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-03T18:17:20.766 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:20 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:17:20.766 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:20 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:17:20.766 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:20 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:17:20.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:20 smithi099 ceph-mon[31658]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-03T18:17:20.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:20 smithi099 ceph-mon[31658]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-03T18:17:20.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:20 smithi099 ceph-mon[31658]: from='osd.5 [v2:172.21.15.150:6808/3172189533,v1:172.21.15.150:6809/3172189533]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-03T18:17:20.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:20 smithi099 ceph-mon[31658]: osdmap e34: 6 total, 5 up, 6 in 2024-05-03T18:17:20.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:20 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-03T18:17:20.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:20 smithi099 ceph-mon[31658]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-03T18:17:20.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:20 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:17:20.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:20 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:17:20.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:20 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:17:20.998 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:17:21.602 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:21 smithi150 ceph-mon[32492]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-05-03T18:17:21.602 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:21 smithi150 ceph-mon[32492]: osdmap e35: 6 total, 5 up, 6 in 2024-05-03T18:17:21.602 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:21 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-03T18:17:21.602 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:21 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-03T18:17:21.602 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:21 smithi150 ceph-mon[32492]: from='osd.5 ' entity='osd.5' 2024-05-03T18:17:21.603 INFO:journalctl@ceph.osd.5.smithi150.stdout:May 03 18:17:21 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-5[44561]: 2024-05-03T18:17:21.320+0000 7fdfafe00700 -1 osd.5 0 waiting for initial osdmap 2024-05-03T18:17:21.603 INFO:journalctl@ceph.osd.5.smithi150.stdout:May 03 18:17:21 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-5[44561]: 2024-05-03T18:17:21.354+0000 7fdfa5200700 -1 osd.5 35 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-03T18:17:21.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:21 smithi033 ceph-mon[28330]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-05-03T18:17:21.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:21 smithi033 ceph-mon[28330]: osdmap e35: 6 total, 5 up, 6 in 2024-05-03T18:17:21.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:21 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-03T18:17:21.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:21 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-03T18:17:21.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:21 smithi033 ceph-mon[28330]: from='osd.5 ' entity='osd.5' 2024-05-03T18:17:21.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:21 smithi099 ceph-mon[31658]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-05-03T18:17:21.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:21 smithi099 ceph-mon[31658]: osdmap e35: 6 total, 5 up, 6 in 2024-05-03T18:17:21.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:21 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-03T18:17:21.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:21 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-03T18:17:21.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:21 smithi099 ceph-mon[31658]: from='osd.5 ' entity='osd.5' 2024-05-03T18:17:22.538 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:22 smithi150 ceph-mon[32492]: purged_snaps scrub starts 2024-05-03T18:17:22.538 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:22 smithi150 ceph-mon[32492]: purged_snaps scrub ok 2024-05-03T18:17:22.538 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:22 smithi150 ceph-mon[32492]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-03T18:17:22.538 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:22 smithi150 ceph-mon[32492]: osd.5 [v2:172.21.15.150:6808/3172189533,v1:172.21.15.150:6809/3172189533] boot 2024-05-03T18:17:22.538 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:22 smithi150 ceph-mon[32492]: osdmap e36: 6 total, 6 up, 6 in 2024-05-03T18:17:22.538 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:22 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-03T18:17:22.538 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:22 smithi150 ceph-mon[32492]: osdmap e37: 6 total, 6 up, 6 in 2024-05-03T18:17:22.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:22 smithi033 ceph-mon[28330]: purged_snaps scrub starts 2024-05-03T18:17:22.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:22 smithi033 ceph-mon[28330]: purged_snaps scrub ok 2024-05-03T18:17:22.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:22 smithi033 ceph-mon[28330]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-03T18:17:22.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:22 smithi033 ceph-mon[28330]: osd.5 [v2:172.21.15.150:6808/3172189533,v1:172.21.15.150:6809/3172189533] boot 2024-05-03T18:17:22.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:22 smithi033 ceph-mon[28330]: osdmap e36: 6 total, 6 up, 6 in 2024-05-03T18:17:22.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:22 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-03T18:17:22.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:22 smithi033 ceph-mon[28330]: osdmap e37: 6 total, 6 up, 6 in 2024-05-03T18:17:22.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:22 smithi099 ceph-mon[31658]: purged_snaps scrub starts 2024-05-03T18:17:22.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:22 smithi099 ceph-mon[31658]: purged_snaps scrub ok 2024-05-03T18:17:22.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:22 smithi099 ceph-mon[31658]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-03T18:17:22.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:22 smithi099 ceph-mon[31658]: osd.5 [v2:172.21.15.150:6808/3172189533,v1:172.21.15.150:6809/3172189533] boot 2024-05-03T18:17:22.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:22 smithi099 ceph-mon[31658]: osdmap e36: 6 total, 6 up, 6 in 2024-05-03T18:17:22.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:22 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-03T18:17:22.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:22 smithi099 ceph-mon[31658]: osdmap e37: 6 total, 6 up, 6 in 2024-05-03T18:17:24.008 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:23 smithi033 ceph-mon[28330]: osdmap e38: 6 total, 6 up, 6 in 2024-05-03T18:17:24.008 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:23 smithi033 ceph-mon[28330]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 422 MiB used, 536 GiB / 536 GiB avail; 153 KiB/s, 0 objects/s recovering 2024-05-03T18:17:24.009 INFO:teuthology.orchestra.run.smithi033.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-03T18:17:24.009 INFO:teuthology.orchestra.run.smithi033.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-03T18:17:24.009 INFO:teuthology.orchestra.run.smithi033.stderr: stderr: 10+0 records in 2024-05-03T18:17:24.009 INFO:teuthology.orchestra.run.smithi033.stderr:10+0 records out 2024-05-03T18:17:24.009 INFO:teuthology.orchestra.run.smithi033.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0327052 s, 321 MB/s 2024-05-03T18:17:24.009 INFO:teuthology.orchestra.run.smithi033.stderr:--> Zapping successful for: 2024-05-03T18:17:24.044 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:23 smithi099 ceph-mon[31658]: osdmap e38: 6 total, 6 up, 6 in 2024-05-03T18:17:24.044 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:23 smithi099 ceph-mon[31658]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 422 MiB used, 536 GiB / 536 GiB avail; 153 KiB/s, 0 objects/s recovering 2024-05-03T18:17:24.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:23 smithi150 ceph-mon[32492]: osdmap e38: 6 total, 6 up, 6 in 2024-05-03T18:17:24.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:23 smithi150 ceph-mon[32492]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 422 MiB used, 536 GiB / 536 GiB avail; 153 KiB/s, 0 objects/s recovering 2024-05-03T18:17:24.964 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi033:vg_nvme/lv_2 2024-05-03T18:17:25.422 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:17:26.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:26 smithi099 ceph-mon[31658]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 422 MiB used, 536 GiB / 536 GiB avail; 121 KiB/s, 0 objects/s recovering 2024-05-03T18:17:26.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:26 smithi099 ceph-mon[31658]: Detected new or changed devices on smithi150 2024-05-03T18:17:26.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:26 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:17:26.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:26 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:17:26.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:26 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:17:26.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:26 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:17:26.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:26 smithi099 ceph-mon[31658]: Adjusting osd_memory_target on smithi150 to 8544M 2024-05-03T18:17:26.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:26 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:17:26.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:26 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:17:26.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:26 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:17:26.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:26 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:17:26.616 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:26 smithi033 ceph-mon[28330]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 422 MiB used, 536 GiB / 536 GiB avail; 121 KiB/s, 0 objects/s recovering 2024-05-03T18:17:26.616 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:26 smithi033 ceph-mon[28330]: Detected new or changed devices on smithi150 2024-05-03T18:17:26.616 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:26 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:17:26.616 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:26 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:17:26.616 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:26 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:17:26.616 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:26 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:17:26.617 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:26 smithi033 ceph-mon[28330]: Adjusting osd_memory_target on smithi150 to 8544M 2024-05-03T18:17:26.617 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:26 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:17:26.617 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:26 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:17:26.617 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:26 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:17:26.617 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:26 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:17:26.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:26 smithi150 ceph-mon[32492]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 422 MiB used, 536 GiB / 536 GiB avail; 121 KiB/s, 0 objects/s recovering 2024-05-03T18:17:26.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:26 smithi150 ceph-mon[32492]: Detected new or changed devices on smithi150 2024-05-03T18:17:26.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:26 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:17:26.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:26 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:17:26.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:26 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:17:26.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:26 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:17:26.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:26 smithi150 ceph-mon[32492]: Adjusting osd_memory_target on smithi150 to 8544M 2024-05-03T18:17:26.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:26 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:17:26.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:26 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:17:26.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:26 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:17:26.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:26 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:17:27.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:17:27.694+0000 7f467d200700 1 -- 172.21.15.33:0/700176312 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46781068c0 msgr2=0x7f4678108cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:17:27.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:17:27.694+0000 7f467d200700 1 --2- 172.21.15.33:0/700176312 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46781068c0 0x7f4678108cb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f4660009990 tx=0x7f4660009ca0 comp rx=0 tx=0).stop 2024-05-03T18:17:27.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:17:27.694+0000 7f467d200700 1 -- 172.21.15.33:0/700176312 shutdown_connections 2024-05-03T18:17:27.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:17:27.694+0000 7f467d200700 1 --2- 172.21.15.33:0/700176312 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46781091f0 0x7f467810b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:17:27.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:17:27.694+0000 7f467d200700 1 --2- 172.21.15.33:0/700176312 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4678103f90 0x7f4678106380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:17:27.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:17:27.694+0000 7f467d200700 1 --2- 172.21.15.33:0/700176312 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46781068c0 0x7f4678108cb0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:17:27.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:17:27.694+0000 7f467d200700 1 -- 172.21.15.33:0/700176312 >> 172.21.15.33:0/700176312 conn(0x7f46780fd9a0 msgr2=0x7f46780ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:17:27.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:17:27.694+0000 7f467d200700 1 -- 172.21.15.33:0/700176312 shutdown_connections 2024-05-03T18:17:27.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:17:27.694+0000 7f467d200700 1 -- 172.21.15.33:0/700176312 wait complete. 2024-05-03T18:17:27.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:17:27.695+0000 7f467d200700 1 Processor -- start 2024-05-03T18:17:27.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:17:27.696+0000 7f467d200700 1 -- start start 2024-05-03T18:17:27.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:17:27.696+0000 7f467d200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4678103f90 0x7f467819b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:17:27.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:17:27.696+0000 7f467d200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46781068c0 0x7f467819bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:17:27.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:17:27.696+0000 7f467d200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46781091f0 0x7f46781a2d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:17:27.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:17:27.696+0000 7f467d200700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f467810eec0 con 0x7f46781091f0 2024-05-03T18:17:27.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:17:27.696+0000 7f467d200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f467810ed40 con 0x7f46781068c0 2024-05-03T18:17:27.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:17:27.696+0000 7f467d200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f467810f040 con 0x7f4678103f90 2024-05-03T18:17:27.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:17:27.697+0000 7f4676800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46781091f0 0x7f46781a2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:17:27.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:17:27.697+0000 7f4676800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46781091f0 0x7f46781a2d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45246/0 (socket says 172.21.15.33:45246) 2024-05-03T18:17:27.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:17:27.697+0000 7f4676800700 1 -- 172.21.15.33:0/2290258656 learned_addr learned my addr 172.21.15.33:0/2290258656 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:17:27.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:17:27.697+0000 7f4675e00700 1 --2- 172.21.15.33:0/2290258656 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4678103f90 0x7f467819b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:17:27.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:17:27.697+0000 7f466fe00700 1 --2- 172.21.15.33:0/2290258656 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46781068c0 0x7f467819bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:17:27.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:17:27.697+0000 7f4676800700 1 -- 172.21.15.33:0/2290258656 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4678103f90 msgr2=0x7f467819b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:17:27.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:17:27.697+0000 7f4676800700 1 --2- 172.21.15.33:0/2290258656 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4678103f90 0x7f467819b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:17:27.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:17:27.697+0000 7f4676800700 1 -- 172.21.15.33:0/2290258656 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46781068c0 msgr2=0x7f467819bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:17:27.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:17:27.697+0000 7f4676800700 1 --2- 172.21.15.33:0/2290258656 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46781068c0 0x7f467819bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:17:27.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:17:27.697+0000 7f4676800700 1 -- 172.21.15.33:0/2290258656 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46781a3420 con 0x7f46781091f0 2024-05-03T18:17:27.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:17:27.698+0000 7f4676800700 1 --2- 172.21.15.33:0/2290258656 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46781091f0 0x7f46781a2d80 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f4670007d70 tx=0x7f467000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:17:27.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:17:27.698+0000 7f466ea00700 1 -- 172.21.15.33:0/2290258656 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4670016020 con 0x7f46781091f0 2024-05-03T18:17:27.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:17:27.698+0000 7f466ea00700 1 -- 172.21.15.33:0/2290258656 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f467000ad00 con 0x7f46781091f0 2024-05-03T18:17:27.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:17:27.698+0000 7f467d200700 1 -- 172.21.15.33:0/2290258656 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46781a3710 con 0x7f46781091f0 2024-05-03T18:17:27.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:17:27.698+0000 7f466ea00700 1 -- 172.21.15.33:0/2290258656 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46700115e0 con 0x7f46781091f0 2024-05-03T18:17:27.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:17:27.698+0000 7f467d200700 1 -- 172.21.15.33:0/2290258656 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46781a3c50 con 0x7f46781091f0 2024-05-03T18:17:27.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:17:27.699+0000 7f467d200700 1 -- 172.21.15.33:0/2290258656 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f467806a190 con 0x7f46781091f0 2024-05-03T18:17:27.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:17:27.702+0000 7f466ea00700 1 -- 172.21.15.33:0/2290258656 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f4670019020 con 0x7f46781091f0 2024-05-03T18:17:27.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:17:27.704+0000 7f466ea00700 1 --2- 172.21.15.33:0/2290258656 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f465c0b2900 0x7f465c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:17:27.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:17:27.704+0000 7f466ea00700 1 -- 172.21.15.33:0/2290258656 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4533+0+0 (secure 0 0 0) 0x7f46700e16d0 con 0x7f46781091f0 2024-05-03T18:17:27.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:17:27.704+0000 7f4675e00700 1 --2- 172.21.15.33:0/2290258656 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f465c0b2900 0x7f465c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:17:27.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:17:27.705+0000 7f4675e00700 1 --2- 172.21.15.33:0/2290258656 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f465c0b2900 0x7f465c0b4dc0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4668005950 tx=0x7f466800a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:17:27.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:17:27.707+0000 7f466ea00700 1 -- 172.21.15.33:0/2290258656 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f4670020020 con 0x7f46781091f0 2024-05-03T18:17:27.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:17:27.954+0000 7f467d200700 1 -- 172.21.15.33:0/2290258656 --> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f46780638e0 con 0x7f465c0b2900 2024-05-03T18:17:28.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:28 smithi099 ceph-mon[31658]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-03T18:17:28.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:28 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T18:17:28.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:28 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T18:17:28.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:28 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:17:28.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:28 smithi150 ceph-mon[32492]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-03T18:17:28.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:28 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T18:17:28.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:28 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T18:17:28.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:28 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:17:28.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:28 smithi033 ceph-mon[28330]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-03T18:17:28.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:28 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T18:17:28.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:28 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T18:17:28.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:28 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:17:29.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:29 smithi033 ceph-mon[28330]: from='client.14352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:17:29.544 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:29 smithi099 ceph-mon[31658]: from='client.14352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:17:29.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:29 smithi150 ceph-mon[32492]: from='client.14352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:17:30.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:30 smithi033 ceph-mon[28330]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-05-03T18:17:30.544 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:30 smithi099 ceph-mon[31658]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-05-03T18:17:30.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:30 smithi150 ceph-mon[32492]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-05-03T18:17:32.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:32 smithi099 ceph-mon[31658]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-05-03T18:17:32.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:32 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/421561299' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "51f5fbdd-3794-4c48-9f6c-478f0fa35ac5"}]: dispatch 2024-05-03T18:17:32.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:32 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/421561299' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "51f5fbdd-3794-4c48-9f6c-478f0fa35ac5"}]': finished 2024-05-03T18:17:32.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:32 smithi099 ceph-mon[31658]: osdmap e39: 7 total, 6 up, 7 in 2024-05-03T18:17:32.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:32 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-03T18:17:32.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:32 smithi150 ceph-mon[32492]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-05-03T18:17:32.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:32 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/421561299' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "51f5fbdd-3794-4c48-9f6c-478f0fa35ac5"}]: dispatch 2024-05-03T18:17:32.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:32 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/421561299' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "51f5fbdd-3794-4c48-9f6c-478f0fa35ac5"}]': finished 2024-05-03T18:17:32.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:32 smithi150 ceph-mon[32492]: osdmap e39: 7 total, 6 up, 7 in 2024-05-03T18:17:32.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:32 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-03T18:17:32.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:32 smithi033 ceph-mon[28330]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-05-03T18:17:32.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:32 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/421561299' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "51f5fbdd-3794-4c48-9f6c-478f0fa35ac5"}]: dispatch 2024-05-03T18:17:32.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:32 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/421561299' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "51f5fbdd-3794-4c48-9f6c-478f0fa35ac5"}]': finished 2024-05-03T18:17:32.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:32 smithi033 ceph-mon[28330]: osdmap e39: 7 total, 6 up, 7 in 2024-05-03T18:17:32.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:32 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-03T18:17:33.381 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:33 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/2326921945' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T18:17:33.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:33 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/2326921945' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T18:17:33.794 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:33 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/2326921945' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T18:17:34.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:34 smithi150 ceph-mon[32492]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-03T18:17:34.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:34 smithi033 ceph-mon[28330]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-03T18:17:34.794 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:34 smithi099 ceph-mon[31658]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-03T18:17:36.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:36 smithi150 ceph-mon[32492]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-03T18:17:36.668 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:36 smithi033 ceph-mon[28330]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-03T18:17:36.794 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:36 smithi099 ceph-mon[31658]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-03T18:17:38.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:38 smithi150 ceph-mon[32492]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-03T18:17:38.674 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:38 smithi033 ceph-mon[28330]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-03T18:17:38.794 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:38 smithi099 ceph-mon[31658]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-03T18:17:40.597 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:40 smithi099 ceph-mon[31658]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-03T18:17:40.605 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:40 smithi033 ceph-mon[28330]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-03T18:17:40.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:40 smithi150 ceph-mon[32492]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-03T18:17:42.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:42 smithi150 ceph-mon[32492]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-03T18:17:42.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:42 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-03T18:17:42.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:42 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:17:42.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:42 smithi033 ceph-mon[28330]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-03T18:17:42.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:42 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-03T18:17:42.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:42 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:17:42.794 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:42 smithi099 ceph-mon[31658]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-03T18:17:42.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:42 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-03T18:17:42.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:42 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:17:43.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:43 smithi033 ceph-mon[28330]: Deploying daemon osd.6 on smithi033 2024-05-03T18:17:43.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:43 smithi150 ceph-mon[32492]: Deploying daemon osd.6 on smithi033 2024-05-03T18:17:43.794 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:43 smithi099 ceph-mon[31658]: Deploying daemon osd.6 on smithi033 2024-05-03T18:17:44.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:44 smithi150 ceph-mon[32492]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-03T18:17:44.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:44 smithi033 ceph-mon[28330]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-03T18:17:44.794 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:44 smithi099 ceph-mon[31658]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-03T18:17:46.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:46 smithi150 ceph-mon[32492]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-03T18:17:46.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:46 smithi033 ceph-mon[28330]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-03T18:17:46.794 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:46 smithi099 ceph-mon[31658]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-03T18:17:48.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:48 smithi150 ceph-mon[32492]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-03T18:17:48.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:48 smithi033 ceph-mon[28330]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-03T18:17:48.794 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:48 smithi099 ceph-mon[31658]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-03T18:17:50.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:50 smithi150 ceph-mon[32492]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-03T18:17:50.640 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:50 smithi033 ceph-mon[28330]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-03T18:17:50.794 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:50 smithi099 ceph-mon[31658]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-03T18:17:52.160 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:51 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:17:52.160 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:51 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:17:52.160 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:51 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:17:52.160 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:51 smithi033 ceph-mon[28330]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-03T18:17:52.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:51 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:17:52.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:51 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:17:52.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:51 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:17:52.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:51 smithi099 ceph-mon[31658]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-03T18:17:52.381 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:51 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:17:52.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:51 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:17:52.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:51 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:17:52.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:51 smithi150 ceph-mon[32492]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-03T18:17:54.541 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:54 smithi033 ceph-mon[28330]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-03T18:17:54.544 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:54 smithi099 ceph-mon[31658]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-03T18:17:54.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:54 smithi150 ceph-mon[32492]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-03T18:17:55.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:55 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:17:55.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:55 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:17:55.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:55 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:17:55.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:55 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:17:55.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:55 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:17:55.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:55 smithi033 ceph-mon[28330]: from='osd.6 [v2:172.21.15.33:6818/4234541418,v1:172.21.15.33:6819/4234541418]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-03T18:17:55.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:55 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:17:55.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:55 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:17:55.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:55 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:17:55.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:55 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:17:55.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:55 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:17:55.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:55 smithi150 ceph-mon[32492]: from='osd.6 [v2:172.21.15.33:6818/4234541418,v1:172.21.15.33:6819/4234541418]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-03T18:17:56.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:55 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:17:56.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:55 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:17:56.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:55 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:17:56.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:55 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:17:56.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:55 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:17:56.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:55 smithi099 ceph-mon[31658]: from='osd.6 [v2:172.21.15.33:6818/4234541418,v1:172.21.15.33:6819/4234541418]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-03T18:17:56.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:17:56.513+0000 7f466ea00700 1 -- 172.21.15.33:0/2290258656 <== mgr.14152 v2:172.21.15.33:6800/3642895501 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f46780638e0 con 0x7f465c0b2900 2024-05-03T18:17:56.515 INFO:teuthology.orchestra.run.smithi033.stdout:Created osd(s) 6 on host 'smithi033' 2024-05-03T18:17:56.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:17:56.517+0000 7f467d200700 1 -- 172.21.15.33:0/2290258656 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f465c0b2900 msgr2=0x7f465c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:17:56.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:17:56.517+0000 7f467d200700 1 --2- 172.21.15.33:0/2290258656 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f465c0b2900 0x7f465c0b4dc0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4668005950 tx=0x7f466800a300 comp rx=0 tx=0).stop 2024-05-03T18:17:56.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:17:56.517+0000 7f467d200700 1 -- 172.21.15.33:0/2290258656 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46781091f0 msgr2=0x7f46781a2d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:17:56.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:17:56.517+0000 7f467d200700 1 --2- 172.21.15.33:0/2290258656 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46781091f0 0x7f46781a2d80 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f4670007d70 tx=0x7f467000a340 comp rx=0 tx=0).stop 2024-05-03T18:17:56.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:17:56.518+0000 7f467d200700 1 -- 172.21.15.33:0/2290258656 shutdown_connections 2024-05-03T18:17:56.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:17:56.518+0000 7f467d200700 1 --2- 172.21.15.33:0/2290258656 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46781091f0 0x7f46781a2d80 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:17:56.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:17:56.518+0000 7f467d200700 1 --2- 172.21.15.33:0/2290258656 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4678103f90 0x7f467819b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:17:56.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:17:56.518+0000 7f467d200700 1 --2- 172.21.15.33:0/2290258656 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f465c0b2900 0x7f465c0b4dc0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:17:56.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:17:56.518+0000 7f467d200700 1 --2- 172.21.15.33:0/2290258656 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46781068c0 0x7f467819bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:17:56.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:17:56.518+0000 7f467d200700 1 -- 172.21.15.33:0/2290258656 >> 172.21.15.33:0/2290258656 conn(0x7f46780fd9a0 msgr2=0x7f4678103710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:17:56.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:17:56.518+0000 7f467d200700 1 -- 172.21.15.33:0/2290258656 shutdown_connections 2024-05-03T18:17:56.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:17:56.518+0000 7f467d200700 1 -- 172.21.15.33:0/2290258656 wait complete. 2024-05-03T18:17:56.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:56 smithi033 ceph-mon[28330]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-03T18:17:56.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:56 smithi033 ceph-mon[28330]: from='osd.6 [v2:172.21.15.33:6818/4234541418,v1:172.21.15.33:6819/4234541418]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-03T18:17:56.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:56 smithi033 ceph-mon[28330]: osdmap e40: 7 total, 6 up, 7 in 2024-05-03T18:17:56.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:56 smithi033 ceph-mon[28330]: from='osd.6 [v2:172.21.15.33:6818/4234541418,v1:172.21.15.33:6819/4234541418]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-05-03T18:17:56.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:56 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-03T18:17:56.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:56 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:17:56.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:56 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:17:56.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:56 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:17:56.881 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:56 smithi150 ceph-mon[32492]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-03T18:17:56.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:56 smithi150 ceph-mon[32492]: from='osd.6 [v2:172.21.15.33:6818/4234541418,v1:172.21.15.33:6819/4234541418]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-03T18:17:56.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:56 smithi150 ceph-mon[32492]: osdmap e40: 7 total, 6 up, 7 in 2024-05-03T18:17:56.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:56 smithi150 ceph-mon[32492]: from='osd.6 [v2:172.21.15.33:6818/4234541418,v1:172.21.15.33:6819/4234541418]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-05-03T18:17:56.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:56 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-03T18:17:56.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:56 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:17:56.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:56 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:17:56.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:56 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:17:57.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:56 smithi099 ceph-mon[31658]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-03T18:17:57.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:56 smithi099 ceph-mon[31658]: from='osd.6 [v2:172.21.15.33:6818/4234541418,v1:172.21.15.33:6819/4234541418]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-03T18:17:57.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:56 smithi099 ceph-mon[31658]: osdmap e40: 7 total, 6 up, 7 in 2024-05-03T18:17:57.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:56 smithi099 ceph-mon[31658]: from='osd.6 [v2:172.21.15.33:6818/4234541418,v1:172.21.15.33:6819/4234541418]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-05-03T18:17:57.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:56 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-03T18:17:57.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:56 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:17:57.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:56 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:17:57.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:56 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:17:57.309 DEBUG:teuthology.orchestra.run.smithi033:osd.6> sudo journalctl -f -n 0 -u ceph-5703bba4-0978-11ef-bc97-c7b262605968@osd.6.service 2024-05-03T18:17:57.313 INFO:tasks.cephadm:Deploying osd.7 on smithi099 with /dev/vg_nvme/lv_2... 2024-05-03T18:17:57.313 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-03T18:17:57.754 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.b/config 2024-05-03T18:17:57.794 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:57 smithi099 ceph-mon[31658]: from='osd.6 [v2:172.21.15.33:6818/4234541418,v1:172.21.15.33:6819/4234541418]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-05-03T18:17:57.794 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:57 smithi099 ceph-mon[31658]: osdmap e41: 7 total, 6 up, 7 in 2024-05-03T18:17:57.794 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:57 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-03T18:17:57.794 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:57 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-03T18:17:57.875 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:57 smithi033 ceph-mon[28330]: from='osd.6 [v2:172.21.15.33:6818/4234541418,v1:172.21.15.33:6819/4234541418]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-05-03T18:17:57.876 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:57 smithi033 ceph-mon[28330]: osdmap e41: 7 total, 6 up, 7 in 2024-05-03T18:17:57.876 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:57 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-03T18:17:57.876 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:57 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-03T18:17:57.881 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:57 smithi150 ceph-mon[32492]: from='osd.6 [v2:172.21.15.33:6818/4234541418,v1:172.21.15.33:6819/4234541418]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-05-03T18:17:57.881 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:57 smithi150 ceph-mon[32492]: osdmap e41: 7 total, 6 up, 7 in 2024-05-03T18:17:57.881 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:57 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-03T18:17:57.881 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:57 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-03T18:17:58.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:58 smithi033 ceph-mon[28330]: purged_snaps scrub starts 2024-05-03T18:17:58.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:58 smithi033 ceph-mon[28330]: purged_snaps scrub ok 2024-05-03T18:17:58.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:58 smithi033 ceph-mon[28330]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-03T18:17:58.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:58 smithi033 ceph-mon[28330]: osd.6 [v2:172.21.15.33:6818/4234541418,v1:172.21.15.33:6819/4234541418] boot 2024-05-03T18:17:58.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:58 smithi033 ceph-mon[28330]: osdmap e42: 7 total, 7 up, 7 in 2024-05-03T18:17:58.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:58 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-03T18:17:58.881 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:58 smithi150 ceph-mon[32492]: purged_snaps scrub starts 2024-05-03T18:17:58.881 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:58 smithi150 ceph-mon[32492]: purged_snaps scrub ok 2024-05-03T18:17:58.881 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:58 smithi150 ceph-mon[32492]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-03T18:17:58.881 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:58 smithi150 ceph-mon[32492]: osd.6 [v2:172.21.15.33:6818/4234541418,v1:172.21.15.33:6819/4234541418] boot 2024-05-03T18:17:58.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:58 smithi150 ceph-mon[32492]: osdmap e42: 7 total, 7 up, 7 in 2024-05-03T18:17:58.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:58 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-03T18:17:58.927 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:58 smithi099 ceph-mon[31658]: purged_snaps scrub starts 2024-05-03T18:17:58.927 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:58 smithi099 ceph-mon[31658]: purged_snaps scrub ok 2024-05-03T18:17:58.927 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:58 smithi099 ceph-mon[31658]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-03T18:17:58.927 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:58 smithi099 ceph-mon[31658]: osd.6 [v2:172.21.15.33:6818/4234541418,v1:172.21.15.33:6819/4234541418] boot 2024-05-03T18:17:58.927 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:58 smithi099 ceph-mon[31658]: osdmap e42: 7 total, 7 up, 7 in 2024-05-03T18:17:58.927 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:58 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-03T18:17:59.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:17:59 smithi033 ceph-mon[28330]: osdmap e43: 7 total, 7 up, 7 in 2024-05-03T18:17:59.919 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:17:59 smithi099 ceph-mon[31658]: osdmap e43: 7 total, 7 up, 7 in 2024-05-03T18:18:00.131 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:17:59 smithi150 ceph-mon[32492]: osdmap e43: 7 total, 7 up, 7 in 2024-05-03T18:18:00.863 INFO:teuthology.orchestra.run.smithi099.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-03T18:18:00.863 INFO:teuthology.orchestra.run.smithi099.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-03T18:18:00.863 INFO:teuthology.orchestra.run.smithi099.stderr: stderr: 10+0 records in 2024-05-03T18:18:00.863 INFO:teuthology.orchestra.run.smithi099.stderr:10+0 records out 2024-05-03T18:18:00.863 INFO:teuthology.orchestra.run.smithi099.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0584607 s, 179 MB/s 2024-05-03T18:18:00.863 INFO:teuthology.orchestra.run.smithi099.stderr:--> Zapping successful for: 2024-05-03T18:18:00.902 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:00 smithi099 ceph-mon[31658]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-03T18:18:00.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:00 smithi033 ceph-mon[28330]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-03T18:18:01.131 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:00 smithi150 ceph-mon[32492]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-03T18:18:01.659 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi099:vg_nvme/lv_2 2024-05-03T18:18:02.090 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.b/config 2024-05-03T18:18:02.934 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:02 smithi150 ceph-mon[32492]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-03T18:18:02.935 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:02 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:18:02.935 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:02 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:18:02.935 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:02 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:18:02.935 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:02 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:18:02.935 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:02 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:18:02.935 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:02 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:18:02.935 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:02 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:18:02.935 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:02 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:18:02.935 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:02 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:18:02.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:02 smithi033 ceph-mon[28330]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-03T18:18:02.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:02 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:18:02.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:02 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:18:02.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:02 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:18:02.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:02 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:18:02.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:02 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:18:02.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:02 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:18:02.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:02 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:18:02.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:02 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:18:02.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:02 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:18:03.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:02 smithi099 ceph-mon[31658]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-03T18:18:03.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:02 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:18:03.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:02 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:18:03.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:02 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:18:03.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:02 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:18:03.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:02 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:18:03.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:02 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:18:03.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:02 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:18:03.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:02 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:18:03.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:02 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:18:03.854 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:03 smithi099 ceph-mon[31658]: Detected new or changed devices on smithi033 2024-05-03T18:18:03.854 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:03 smithi099 ceph-mon[31658]: Adjusting osd_memory_target on smithi033 to 5698M 2024-05-03T18:18:03.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:03 smithi033 ceph-mon[28330]: Detected new or changed devices on smithi033 2024-05-03T18:18:03.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:03 smithi033 ceph-mon[28330]: Adjusting osd_memory_target on smithi033 to 5698M 2024-05-03T18:18:04.131 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:03 smithi150 ceph-mon[32492]: Detected new or changed devices on smithi033 2024-05-03T18:18:04.131 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:03 smithi150 ceph-mon[32492]: Adjusting osd_memory_target on smithi033 to 5698M 2024-05-03T18:18:04.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:18:04.398+0000 7f3910a00700 1 -- 172.21.15.99:0/1007785047 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f390c1068c0 msgr2=0x7f390c108cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:18:04.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:18:04.398+0000 7f3910a00700 1 --2- 172.21.15.99:0/1007785047 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f390c1068c0 0x7f390c108cb0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f38f8009990 tx=0x7f38f8009ca0 comp rx=0 tx=0).stop 2024-05-03T18:18:04.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:18:04.398+0000 7f3910a00700 1 -- 172.21.15.99:0/1007785047 shutdown_connections 2024-05-03T18:18:04.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:18:04.398+0000 7f3910a00700 1 --2- 172.21.15.99:0/1007785047 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f390c1091f0 0x7f390c10b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:18:04.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:18:04.398+0000 7f3910a00700 1 --2- 172.21.15.99:0/1007785047 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f390c103f90 0x7f390c106380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:18:04.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:18:04.398+0000 7f3910a00700 1 --2- 172.21.15.99:0/1007785047 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f390c1068c0 0x7f390c108cb0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:18:04.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:18:04.398+0000 7f3910a00700 1 -- 172.21.15.99:0/1007785047 >> 172.21.15.99:0/1007785047 conn(0x7f390c0fd9c0 msgr2=0x7f390c0ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:18:04.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:18:04.398+0000 7f3910a00700 1 -- 172.21.15.99:0/1007785047 shutdown_connections 2024-05-03T18:18:04.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:18:04.398+0000 7f3910a00700 1 -- 172.21.15.99:0/1007785047 wait complete. 2024-05-03T18:18:04.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:18:04.399+0000 7f3910a00700 1 Processor -- start 2024-05-03T18:18:04.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:18:04.400+0000 7f3910a00700 1 -- start start 2024-05-03T18:18:04.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:18:04.400+0000 7f3910a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f390c103f90 0x7f390c19b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:18:04.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:18:04.400+0000 7f3910a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f390c1068c0 0x7f390c19bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:18:04.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:18:04.400+0000 7f3910a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f390c1091f0 0x7f390c1a2d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:18:04.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:18:04.400+0000 7f3910a00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f390c10ee80 con 0x7f390c1091f0 2024-05-03T18:18:04.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:18:04.400+0000 7f3910a00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f390c10ed00 con 0x7f390c103f90 2024-05-03T18:18:04.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:18:04.400+0000 7f3910a00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f390c10f000 con 0x7f390c1068c0 2024-05-03T18:18:04.403 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:18:04.400+0000 7f3909e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f390c103f90 0x7f390c19b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:18:04.403 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:18:04.400+0000 7f3909e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f390c103f90 0x7f390c19b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57228/0 (socket says 172.21.15.99:57228) 2024-05-03T18:18:04.403 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:18:04.400+0000 7f3909e00700 1 -- 172.21.15.99:0/781136653 learned_addr learned my addr 172.21.15.99:0/781136653 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-03T18:18:04.403 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:18:04.400+0000 7f3909400700 1 --2- 172.21.15.99:0/781136653 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f390c1068c0 0x7f390c19bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:18:04.403 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:18:04.401+0000 7f390a800700 1 --2- 172.21.15.99:0/781136653 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f390c1091f0 0x7f390c1a2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:18:04.403 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:18:04.401+0000 7f3909e00700 1 -- 172.21.15.99:0/781136653 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f390c1068c0 msgr2=0x7f390c19bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:18:04.403 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:18:04.401+0000 7f3909e00700 1 --2- 172.21.15.99:0/781136653 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f390c1068c0 0x7f390c19bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:18:04.403 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:18:04.401+0000 7f3909e00700 1 -- 172.21.15.99:0/781136653 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f390c1091f0 msgr2=0x7f390c1a2d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:18:04.403 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:18:04.401+0000 7f3909e00700 1 --2- 172.21.15.99:0/781136653 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f390c1091f0 0x7f390c1a2d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:18:04.403 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:18:04.401+0000 7f3909e00700 1 -- 172.21.15.99:0/781136653 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f390c1a33a0 con 0x7f390c103f90 2024-05-03T18:18:04.403 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:18:04.401+0000 7f3909400700 1 --2- 172.21.15.99:0/781136653 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f390c1068c0 0x7f390c19bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:18:04.403 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:18:04.401+0000 7f3909e00700 1 --2- 172.21.15.99:0/781136653 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f390c103f90 0x7f390c19b740 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f38fc00c920 tx=0x7f38fc00cce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:18:04.403 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:18:04.401+0000 7f3902a00700 1 -- 172.21.15.99:0/781136653 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38fc007b00 con 0x7f390c103f90 2024-05-03T18:18:04.403 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:18:04.401+0000 7f3902a00700 1 -- 172.21.15.99:0/781136653 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f38fc004cc0 con 0x7f390c103f90 2024-05-03T18:18:04.403 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:18:04.401+0000 7f3902a00700 1 -- 172.21.15.99:0/781136653 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38fc0054e0 con 0x7f390c103f90 2024-05-03T18:18:04.404 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:18:04.402+0000 7f3910a00700 1 -- 172.21.15.99:0/781136653 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f390c1a3690 con 0x7f390c103f90 2024-05-03T18:18:04.404 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:18:04.402+0000 7f3910a00700 1 -- 172.21.15.99:0/781136653 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f390c1a3c20 con 0x7f390c103f90 2024-05-03T18:18:04.404 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:18:04.403+0000 7f3910a00700 1 -- 172.21.15.99:0/781136653 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f390c06a190 con 0x7f390c103f90 2024-05-03T18:18:04.405 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:18:04.406+0000 7f3902a00700 1 -- 172.21.15.99:0/781136653 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f38fc004750 con 0x7f390c103f90 2024-05-03T18:18:04.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:18:04.407+0000 7f3902a00700 1 --2- 172.21.15.99:0/781136653 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f38f00b2980 0x7f38f00b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:18:04.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:18:04.407+0000 7f3902a00700 1 -- 172.21.15.99:0/781136653 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 4965+0+0 (secure 0 0 0) 0x7f38fc0e1530 con 0x7f390c103f90 2024-05-03T18:18:04.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:18:04.408+0000 7f3909400700 1 --2- 172.21.15.99:0/781136653 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f38f00b2980 0x7f38f00b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:18:04.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:18:04.409+0000 7f3909400700 1 --2- 172.21.15.99:0/781136653 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f38f00b2980 0x7f38f00b4e40 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f38f8003820 tx=0x7f38f8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:18:04.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:18:04.410+0000 7f3902a00700 1 -- 172.21.15.99:0/781136653 <== mon.2 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f38fc014030 con 0x7f390c103f90 2024-05-03T18:18:04.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:18:04.651+0000 7f3910a00700 1 -- 172.21.15.99:0/781136653 --> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f390c0638e0 con 0x7f38f00b2980 2024-05-03T18:18:04.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:04 smithi033 ceph-mon[28330]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-03T18:18:04.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:04 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T18:18:04.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:04 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T18:18:04.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:04 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:18:05.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:04 smithi099 ceph-mon[31658]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-03T18:18:05.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:04 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T18:18:05.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:04 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T18:18:05.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:04 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:18:05.131 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:04 smithi150 ceph-mon[32492]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-03T18:18:05.131 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:04 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T18:18:05.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:04 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T18:18:05.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:04 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:18:05.860 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:05 smithi099 ceph-mon[31658]: from='client.34277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:18:05.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:05 smithi033 ceph-mon[28330]: from='client.34277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:18:06.131 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:05 smithi150 ceph-mon[32492]: from='client.34277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:18:06.794 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:06 smithi099 ceph-mon[31658]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-03T18:18:06.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:06 smithi033 ceph-mon[28330]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-03T18:18:07.131 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:06 smithi150 ceph-mon[32492]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-03T18:18:08.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:08 smithi033 ceph-mon[28330]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-03T18:18:09.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:08 smithi099 ceph-mon[31658]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-03T18:18:09.131 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:08 smithi150 ceph-mon[32492]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-03T18:18:09.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:09 smithi033 ceph-mon[28330]: from='client.? 172.21.15.99:0/1100180987' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "982cc5b3-3163-4505-9bde-a27a8c289457"}]: dispatch 2024-05-03T18:18:09.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:09 smithi033 ceph-mon[28330]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "982cc5b3-3163-4505-9bde-a27a8c289457"}]: dispatch 2024-05-03T18:18:09.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:09 smithi033 ceph-mon[28330]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "982cc5b3-3163-4505-9bde-a27a8c289457"}]': finished 2024-05-03T18:18:09.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:09 smithi033 ceph-mon[28330]: osdmap e44: 8 total, 7 up, 8 in 2024-05-03T18:18:09.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:09 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-03T18:18:09.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:09 smithi033 ceph-mon[28330]: from='client.? 172.21.15.99:0/1891618892' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T18:18:10.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:09 smithi099 ceph-mon[31658]: from='client.? 172.21.15.99:0/1100180987' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "982cc5b3-3163-4505-9bde-a27a8c289457"}]: dispatch 2024-05-03T18:18:10.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:09 smithi099 ceph-mon[31658]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "982cc5b3-3163-4505-9bde-a27a8c289457"}]: dispatch 2024-05-03T18:18:10.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:09 smithi099 ceph-mon[31658]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "982cc5b3-3163-4505-9bde-a27a8c289457"}]': finished 2024-05-03T18:18:10.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:09 smithi099 ceph-mon[31658]: osdmap e44: 8 total, 7 up, 8 in 2024-05-03T18:18:10.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:09 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-03T18:18:10.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:09 smithi099 ceph-mon[31658]: from='client.? 172.21.15.99:0/1891618892' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T18:18:10.131 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:09 smithi150 ceph-mon[32492]: from='client.? 172.21.15.99:0/1100180987' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "982cc5b3-3163-4505-9bde-a27a8c289457"}]: dispatch 2024-05-03T18:18:10.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:09 smithi150 ceph-mon[32492]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "982cc5b3-3163-4505-9bde-a27a8c289457"}]: dispatch 2024-05-03T18:18:10.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:09 smithi150 ceph-mon[32492]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "982cc5b3-3163-4505-9bde-a27a8c289457"}]': finished 2024-05-03T18:18:10.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:09 smithi150 ceph-mon[32492]: osdmap e44: 8 total, 7 up, 8 in 2024-05-03T18:18:10.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:09 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-03T18:18:10.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:09 smithi150 ceph-mon[32492]: from='client.? 172.21.15.99:0/1891618892' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T18:18:10.949 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:10 smithi099 ceph-mon[31658]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-03T18:18:10.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:10 smithi033 ceph-mon[28330]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-03T18:18:11.131 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:10 smithi150 ceph-mon[32492]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-03T18:18:12.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:12 smithi033 ceph-mon[28330]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-03T18:18:13.005 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:12 smithi099 ceph-mon[31658]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-03T18:18:13.131 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:12 smithi150 ceph-mon[32492]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-03T18:18:13.794 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:13 smithi099 ceph-mon[31658]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-03T18:18:13.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:13 smithi033 ceph-mon[28330]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-03T18:18:14.131 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:13 smithi150 ceph-mon[32492]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-03T18:18:16.409 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:16 smithi099 ceph-mon[31658]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-03T18:18:16.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:16 smithi150 ceph-mon[32492]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-03T18:18:16.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:16 smithi033 ceph-mon[28330]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-03T18:18:18.544 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:18 smithi099 ceph-mon[31658]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-03T18:18:18.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:18 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-03T18:18:18.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:18 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:18:18.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:18 smithi033 ceph-mon[28330]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-03T18:18:18.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:18 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-03T18:18:18.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:18 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:18:18.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:18 smithi150 ceph-mon[32492]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-03T18:18:18.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:18 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-03T18:18:18.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:18 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:18:19.528 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:19 smithi099 ceph-mon[31658]: Deploying daemon osd.7 on smithi099 2024-05-03T18:18:19.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:19 smithi150 ceph-mon[32492]: Deploying daemon osd.7 on smithi099 2024-05-03T18:18:19.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:19 smithi033 ceph-mon[28330]: Deploying daemon osd.7 on smithi099 2024-05-03T18:18:20.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:20 smithi033 ceph-mon[28330]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-03T18:18:20.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:20 smithi150 ceph-mon[32492]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-03T18:18:20.794 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:20 smithi099 ceph-mon[31658]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-03T18:18:22.544 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:22 smithi099 ceph-mon[31658]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-03T18:18:22.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:22 smithi150 ceph-mon[32492]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-03T18:18:22.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:22 smithi033 ceph-mon[28330]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-03T18:18:24.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:24 smithi099 ceph-mon[31658]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-03T18:18:24.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:24 smithi150 ceph-mon[32492]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-03T18:18:24.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:24 smithi033 ceph-mon[28330]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-03T18:18:26.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:26 smithi099 ceph-mon[31658]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-03T18:18:26.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:26 smithi150 ceph-mon[32492]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-03T18:18:26.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:26 smithi033 ceph-mon[28330]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-03T18:18:28.029 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:27 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:18:28.029 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:27 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:18:28.029 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:27 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:18:28.029 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:27 smithi099 ceph-mon[31658]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-03T18:18:28.131 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:27 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:18:28.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:27 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:18:28.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:27 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:18:28.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:27 smithi150 ceph-mon[32492]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-03T18:18:28.233 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:27 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:18:28.233 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:27 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:18:28.233 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:27 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:18:28.233 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:27 smithi033 ceph-mon[28330]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-03T18:18:30.544 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:30 smithi099 ceph-mon[31658]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-03T18:18:30.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:30 smithi150 ceph-mon[32492]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-03T18:18:30.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:30 smithi033 ceph-mon[28330]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-03T18:18:31.924 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:18:31.922+0000 7f3902a00700 1 -- 172.21.15.99:0/781136653 <== mgr.14152 v2:172.21.15.33:6800/3642895501 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f390c0638e0 con 0x7f38f00b2980 2024-05-03T18:18:31.924 INFO:teuthology.orchestra.run.smithi099.stdout:Created osd(s) 7 on host 'smithi099' 2024-05-03T18:18:31.924 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:31 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:18:31.924 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:31 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:18:31.924 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:31 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:18:31.924 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:31 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:18:31.925 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:31 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:18:31.925 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:31 smithi099 ceph-mon[31658]: from='osd.7 [v2:172.21.15.99:6816/908673090,v1:172.21.15.99:6817/908673090]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-03T18:18:31.925 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:31 smithi099 ceph-mon[31658]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-03T18:18:31.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:18:31.927+0000 7f3910a00700 1 -- 172.21.15.99:0/781136653 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f38f00b2980 msgr2=0x7f38f00b4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:18:31.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:18:31.927+0000 7f3910a00700 1 --2- 172.21.15.99:0/781136653 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f38f00b2980 0x7f38f00b4e40 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f38f8003820 tx=0x7f38f8005fd0 comp rx=0 tx=0).stop 2024-05-03T18:18:31.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:18:31.927+0000 7f3910a00700 1 -- 172.21.15.99:0/781136653 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f390c103f90 msgr2=0x7f390c19b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:18:31.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:18:31.927+0000 7f3910a00700 1 --2- 172.21.15.99:0/781136653 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f390c103f90 0x7f390c19b740 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f38fc00c920 tx=0x7f38fc00cce0 comp rx=0 tx=0).stop 2024-05-03T18:18:31.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:18:31.927+0000 7f3910a00700 1 -- 172.21.15.99:0/781136653 shutdown_connections 2024-05-03T18:18:31.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:18:31.927+0000 7f3910a00700 1 --2- 172.21.15.99:0/781136653 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f390c1091f0 0x7f390c1a2d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:18:31.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:18:31.927+0000 7f3910a00700 1 --2- 172.21.15.99:0/781136653 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f38f00b2980 0x7f38f00b4e40 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:18:31.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:18:31.927+0000 7f3910a00700 1 --2- 172.21.15.99:0/781136653 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f390c103f90 0x7f390c19b740 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:18:31.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:18:31.927+0000 7f3910a00700 1 --2- 172.21.15.99:0/781136653 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f390c1068c0 0x7f390c19bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:18:31.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:18:31.927+0000 7f3910a00700 1 -- 172.21.15.99:0/781136653 >> 172.21.15.99:0/781136653 conn(0x7f390c0fd9c0 msgr2=0x7f390c1036e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:18:31.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:18:31.927+0000 7f3910a00700 1 -- 172.21.15.99:0/781136653 shutdown_connections 2024-05-03T18:18:31.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:18:31.927+0000 7f3910a00700 1 -- 172.21.15.99:0/781136653 wait complete. 2024-05-03T18:18:31.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:31 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:18:31.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:31 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:18:31.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:31 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:18:31.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:31 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:18:31.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:31 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:18:31.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:31 smithi033 ceph-mon[28330]: from='osd.7 [v2:172.21.15.99:6816/908673090,v1:172.21.15.99:6817/908673090]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-03T18:18:31.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:31 smithi033 ceph-mon[28330]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-03T18:18:32.131 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:31 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:18:32.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:31 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:18:32.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:31 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:18:32.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:31 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:18:32.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:31 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:18:32.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:31 smithi150 ceph-mon[32492]: from='osd.7 [v2:172.21.15.99:6816/908673090,v1:172.21.15.99:6817/908673090]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-03T18:18:32.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:31 smithi150 ceph-mon[32492]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-03T18:18:32.763 DEBUG:teuthology.orchestra.run.smithi099:osd.7> sudo journalctl -f -n 0 -u ceph-5703bba4-0978-11ef-bc97-c7b262605968@osd.7.service 2024-05-03T18:18:32.767 INFO:tasks.cephadm:Deploying osd.8 on smithi150 with /dev/vg_nvme/lv_2... 2024-05-03T18:18:32.767 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-03T18:18:32.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:32 smithi033 ceph-mon[28330]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-03T18:18:32.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:32 smithi033 ceph-mon[28330]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-03T18:18:32.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:32 smithi033 ceph-mon[28330]: osdmap e45: 8 total, 7 up, 8 in 2024-05-03T18:18:32.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:32 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-03T18:18:32.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:32 smithi033 ceph-mon[28330]: from='osd.7 [v2:172.21.15.99:6816/908673090,v1:172.21.15.99:6817/908673090]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-05-03T18:18:32.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:32 smithi033 ceph-mon[28330]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-05-03T18:18:32.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:32 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:18:32.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:32 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:18:32.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:32 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:18:33.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:32 smithi099 ceph-mon[31658]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-03T18:18:33.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:32 smithi099 ceph-mon[31658]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-03T18:18:33.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:32 smithi099 ceph-mon[31658]: osdmap e45: 8 total, 7 up, 8 in 2024-05-03T18:18:33.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:32 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-03T18:18:33.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:32 smithi099 ceph-mon[31658]: from='osd.7 [v2:172.21.15.99:6816/908673090,v1:172.21.15.99:6817/908673090]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-05-03T18:18:33.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:32 smithi099 ceph-mon[31658]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-05-03T18:18:33.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:32 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:18:33.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:32 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:18:33.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:32 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:18:33.131 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:32 smithi150 ceph-mon[32492]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-03T18:18:33.131 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:32 smithi150 ceph-mon[32492]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-03T18:18:33.131 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:32 smithi150 ceph-mon[32492]: osdmap e45: 8 total, 7 up, 8 in 2024-05-03T18:18:33.131 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:32 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-03T18:18:33.131 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:32 smithi150 ceph-mon[32492]: from='osd.7 [v2:172.21.15.99:6816/908673090,v1:172.21.15.99:6817/908673090]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-05-03T18:18:33.131 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:32 smithi150 ceph-mon[32492]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-05-03T18:18:33.131 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:32 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:18:33.131 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:32 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:18:33.131 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:32 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:18:33.205 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.c/config 2024-05-03T18:18:33.721 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:18:33 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7[49675]: 2024-05-03T18:18:33.547+0000 7ff8b5200700 -1 osd.7 0 waiting for initial osdmap 2024-05-03T18:18:33.721 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:18:33 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7[49675]: 2024-05-03T18:18:33.578+0000 7ff8ab000700 -1 osd.7 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-03T18:18:33.970 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:33 smithi099 ceph-mon[31658]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-05-03T18:18:33.970 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:33 smithi099 ceph-mon[31658]: osdmap e46: 8 total, 7 up, 8 in 2024-05-03T18:18:33.970 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:33 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-03T18:18:33.970 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:33 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-03T18:18:33.970 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:33 smithi099 ceph-mon[31658]: from='osd.7 ' entity='osd.7' 2024-05-03T18:18:33.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:33 smithi033 ceph-mon[28330]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-05-03T18:18:33.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:33 smithi033 ceph-mon[28330]: osdmap e46: 8 total, 7 up, 8 in 2024-05-03T18:18:33.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:33 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-03T18:18:33.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:33 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-03T18:18:33.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:33 smithi033 ceph-mon[28330]: from='osd.7 ' entity='osd.7' 2024-05-03T18:18:34.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:33 smithi150 ceph-mon[32492]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-05-03T18:18:34.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:33 smithi150 ceph-mon[32492]: osdmap e46: 8 total, 7 up, 8 in 2024-05-03T18:18:34.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:33 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-03T18:18:34.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:33 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-03T18:18:34.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:33 smithi150 ceph-mon[32492]: from='osd.7 ' entity='osd.7' 2024-05-03T18:18:34.909 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:34 smithi099 ceph-mon[31658]: purged_snaps scrub starts 2024-05-03T18:18:34.909 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:34 smithi099 ceph-mon[31658]: purged_snaps scrub ok 2024-05-03T18:18:34.909 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:34 smithi099 ceph-mon[31658]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-03T18:18:34.909 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:34 smithi099 ceph-mon[31658]: osd.7 [v2:172.21.15.99:6816/908673090,v1:172.21.15.99:6817/908673090] boot 2024-05-03T18:18:34.909 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:34 smithi099 ceph-mon[31658]: osdmap e47: 8 total, 8 up, 8 in 2024-05-03T18:18:34.909 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:34 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-03T18:18:34.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:34 smithi033 ceph-mon[28330]: purged_snaps scrub starts 2024-05-03T18:18:34.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:34 smithi033 ceph-mon[28330]: purged_snaps scrub ok 2024-05-03T18:18:34.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:34 smithi033 ceph-mon[28330]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-03T18:18:34.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:34 smithi033 ceph-mon[28330]: osd.7 [v2:172.21.15.99:6816/908673090,v1:172.21.15.99:6817/908673090] boot 2024-05-03T18:18:34.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:34 smithi033 ceph-mon[28330]: osdmap e47: 8 total, 8 up, 8 in 2024-05-03T18:18:34.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:34 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-03T18:18:35.025 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:34 smithi150 ceph-mon[32492]: purged_snaps scrub starts 2024-05-03T18:18:35.025 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:34 smithi150 ceph-mon[32492]: purged_snaps scrub ok 2024-05-03T18:18:35.026 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:34 smithi150 ceph-mon[32492]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-03T18:18:35.026 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:34 smithi150 ceph-mon[32492]: osd.7 [v2:172.21.15.99:6816/908673090,v1:172.21.15.99:6817/908673090] boot 2024-05-03T18:18:35.026 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:34 smithi150 ceph-mon[32492]: osdmap e47: 8 total, 8 up, 8 in 2024-05-03T18:18:35.026 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:34 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-03T18:18:35.932 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:35 smithi150 ceph-mon[32492]: osdmap e48: 8 total, 8 up, 8 in 2024-05-03T18:18:35.932 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:35 smithi150 ceph-mon[32492]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-05-03T18:18:36.044 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:35 smithi099 ceph-mon[31658]: osdmap e48: 8 total, 8 up, 8 in 2024-05-03T18:18:36.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:35 smithi099 ceph-mon[31658]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-05-03T18:18:36.233 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:35 smithi033 ceph-mon[28330]: osdmap e48: 8 total, 8 up, 8 in 2024-05-03T18:18:36.233 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:35 smithi033 ceph-mon[28330]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-05-03T18:18:36.299 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-03T18:18:36.299 INFO:teuthology.orchestra.run.smithi150.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-03T18:18:36.299 INFO:teuthology.orchestra.run.smithi150.stderr: stderr: 10+0 records in 2024-05-03T18:18:36.299 INFO:teuthology.orchestra.run.smithi150.stderr:10+0 records out 2024-05-03T18:18:36.299 INFO:teuthology.orchestra.run.smithi150.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0304065 s, 345 MB/s 2024-05-03T18:18:36.299 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping successful for: 2024-05-03T18:18:37.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:36 smithi099 ceph-mon[31658]: osdmap e49: 8 total, 8 up, 8 in 2024-05-03T18:18:37.131 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:36 smithi150 ceph-mon[32492]: osdmap e49: 8 total, 8 up, 8 in 2024-05-03T18:18:37.233 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:36 smithi033 ceph-mon[28330]: osdmap e49: 8 total, 8 up, 8 in 2024-05-03T18:18:37.328 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi150:vg_nvme/lv_2 2024-05-03T18:18:37.758 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.c/config 2024-05-03T18:18:38.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:38 smithi033 ceph-mon[28330]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-05-03T18:18:38.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:38 smithi033 ceph-mon[28330]: Detected new or changed devices on smithi099 2024-05-03T18:18:38.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:38 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:18:38.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:38 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:18:38.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:38 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:18:38.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:38 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:18:38.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:38 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:18:38.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:38 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:18:38.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:38 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:18:38.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:38 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:18:38.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:38 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:18:39.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:38 smithi099 ceph-mon[31658]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-05-03T18:18:39.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:38 smithi099 ceph-mon[31658]: Detected new or changed devices on smithi099 2024-05-03T18:18:39.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:38 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:18:39.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:38 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:18:39.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:38 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:18:39.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:38 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:18:39.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:38 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:18:39.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:38 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:18:39.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:38 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:18:39.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:38 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:18:39.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:38 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:18:39.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:38 smithi150 ceph-mon[32492]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-05-03T18:18:39.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:38 smithi150 ceph-mon[32492]: Detected new or changed devices on smithi099 2024-05-03T18:18:39.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:38 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:18:39.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:38 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:18:39.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:38 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:18:39.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:38 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:18:39.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:38 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:18:39.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:38 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:18:39.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:38 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:18:39.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:38 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:18:39.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:38 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:18:39.766 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:39 smithi150 ceph-mon[32492]: Adjusting osd_memory_target on smithi099 to 5696M 2024-05-03T18:18:39.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:39 smithi033 ceph-mon[28330]: Adjusting osd_memory_target on smithi099 to 5696M 2024-05-03T18:18:40.044 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:39 smithi099 ceph-mon[31658]: Adjusting osd_memory_target on smithi099 to 5696M 2024-05-03T18:18:40.078 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:18:40.078+0000 7fe6d1c00700 1 -- 172.21.15.150:0/1407509490 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe6cc105130 msgr2=0x7fe6cc10b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:18:40.078 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:18:40.078+0000 7fe6d1c00700 1 --2- 172.21.15.150:0/1407509490 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe6cc105130 0x7fe6cc10b9e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe6c400b0a0 tx=0x7fe6c400b3b0 comp rx=0 tx=0).stop 2024-05-03T18:18:40.078 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:18:40.078+0000 7fe6d1c00700 1 -- 172.21.15.150:0/1407509490 shutdown_connections 2024-05-03T18:18:40.078 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:18:40.078+0000 7fe6d1c00700 1 --2- 172.21.15.150:0/1407509490 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe6cc105130 0x7fe6cc10b9e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:18:40.079 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:18:40.078+0000 7fe6d1c00700 1 --2- 172.21.15.150:0/1407509490 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe6cc1017b0 0x7fe6cc101bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:18:40.079 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:18:40.078+0000 7fe6d1c00700 1 --2- 172.21.15.150:0/1407509490 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6cc104790 0x7fe6cc104bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:18:40.079 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:18:40.078+0000 7fe6d1c00700 1 -- 172.21.15.150:0/1407509490 >> 172.21.15.150:0/1407509490 conn(0x7fe6cc0fd130 msgr2=0x7fe6cc0ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:18:40.079 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:18:40.078+0000 7fe6d1c00700 1 -- 172.21.15.150:0/1407509490 shutdown_connections 2024-05-03T18:18:40.079 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:18:40.078+0000 7fe6d1c00700 1 -- 172.21.15.150:0/1407509490 wait complete. 2024-05-03T18:18:40.081 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:18:40.079+0000 7fe6d1c00700 1 Processor -- start 2024-05-03T18:18:40.081 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:18:40.080+0000 7fe6d1c00700 1 -- start start 2024-05-03T18:18:40.081 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:18:40.080+0000 7fe6d1c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe6cc1017b0 0x7fe6cc1040a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:18:40.081 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:18:40.080+0000 7fe6d1c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe6cc104790 0x7fe6cc102740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:18:40.081 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:18:40.080+0000 7fe6d1c00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6cc105130 0x7fe6cc102c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:18:40.081 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:18:40.080+0000 7fe6d1c00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe6cc10ef00 con 0x7fe6cc105130 2024-05-03T18:18:40.081 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:18:40.080+0000 7fe6d1c00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe6cc10ed80 con 0x7fe6cc1017b0 2024-05-03T18:18:40.081 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:18:40.080+0000 7fe6d1c00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe6cc10f080 con 0x7fe6cc104790 2024-05-03T18:18:40.081 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:18:40.080+0000 7fe6ca000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe6cc104790 0x7fe6cc102740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:18:40.081 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:18:40.080+0000 7fe6ca000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe6cc104790 0x7fe6cc102740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:39242/0 (socket says 172.21.15.150:39242) 2024-05-03T18:18:40.081 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:18:40.080+0000 7fe6ca000700 1 -- 172.21.15.150:0/2422283552 learned_addr learned my addr 172.21.15.150:0/2422283552 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-03T18:18:40.082 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:18:40.081+0000 7fe6cb400700 1 --2- 172.21.15.150:0/2422283552 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6cc105130 0x7fe6cc102c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:18:40.082 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:18:40.081+0000 7fe6caa00700 1 --2- 172.21.15.150:0/2422283552 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe6cc1017b0 0x7fe6cc1040a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:18:40.082 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:18:40.081+0000 7fe6ca000700 1 -- 172.21.15.150:0/2422283552 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe6cc1017b0 msgr2=0x7fe6cc1040a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:18:40.082 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:18:40.081+0000 7fe6ca000700 1 --2- 172.21.15.150:0/2422283552 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe6cc1017b0 0x7fe6cc1040a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:18:40.082 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:18:40.081+0000 7fe6ca000700 1 -- 172.21.15.150:0/2422283552 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6cc105130 msgr2=0x7fe6cc102c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:18:40.082 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:18:40.081+0000 7fe6ca000700 1 --2- 172.21.15.150:0/2422283552 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6cc105130 0x7fe6cc102c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:18:40.082 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:18:40.081+0000 7fe6ca000700 1 -- 172.21.15.150:0/2422283552 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6cc103550 con 0x7fe6cc104790 2024-05-03T18:18:40.083 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:18:40.082+0000 7fe6ca000700 1 --2- 172.21.15.150:0/2422283552 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe6cc104790 0x7fe6cc102740 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe6b800c920 tx=0x7fe6b800cc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:18:40.083 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:18:40.082+0000 7fe6c3400700 1 -- 172.21.15.150:0/2422283552 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6b8004a30 con 0x7fe6cc104790 2024-05-03T18:18:40.083 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:18:40.082+0000 7fe6c3400700 1 -- 172.21.15.150:0/2422283552 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe6b80054b0 con 0x7fe6cc104790 2024-05-03T18:18:40.083 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:18:40.082+0000 7fe6c3400700 1 -- 172.21.15.150:0/2422283552 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6b8007b70 con 0x7fe6cc104790 2024-05-03T18:18:40.083 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:18:40.082+0000 7fe6d1c00700 1 -- 172.21.15.150:0/2422283552 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6cc1a3700 con 0x7fe6cc104790 2024-05-03T18:18:40.083 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:18:40.082+0000 7fe6d1c00700 1 -- 172.21.15.150:0/2422283552 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6cc1a3ce0 con 0x7fe6cc104790 2024-05-03T18:18:40.083 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:18:40.083+0000 7fe6d1c00700 1 -- 172.21.15.150:0/2422283552 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6cc06a190 con 0x7fe6cc104790 2024-05-03T18:18:40.086 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:18:40.087+0000 7fe6c3400700 1 -- 172.21.15.150:0/2422283552 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fe6b8004bd0 con 0x7fe6cc104790 2024-05-03T18:18:40.088 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:18:40.088+0000 7fe6c3400700 1 --2- 172.21.15.150:0/2422283552 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fe6b00b2900 0x7fe6b00b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:18:40.088 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:18:40.088+0000 7fe6c3400700 1 -- 172.21.15.150:0/2422283552 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5397+0+0 (secure 0 0 0) 0x7fe6b80e1640 con 0x7fe6cc104790 2024-05-03T18:18:40.088 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:18:40.089+0000 7fe6caa00700 1 --2- 172.21.15.150:0/2422283552 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fe6b00b2900 0x7fe6b00b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:18:40.089 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:18:40.090+0000 7fe6caa00700 1 --2- 172.21.15.150:0/2422283552 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fe6b00b2900 0x7fe6b00b4dc0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc06af80 tx=0x7fe6bc009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:18:40.090 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:18:40.091+0000 7fe6c3400700 1 -- 172.21.15.150:0/2422283552 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fe6b8014030 con 0x7fe6cc104790 2024-05-03T18:18:40.331 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:18:40.330+0000 7fe6d1c00700 1 -- 172.21.15.150:0/2422283552 --> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fe6cc0639a0 con 0x7fe6b00b2900 2024-05-03T18:18:40.964 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:40 smithi099 ceph-mon[31658]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-05-03T18:18:40.964 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:40 smithi099 ceph-mon[31658]: from='client.24304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:18:40.964 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:40 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T18:18:40.964 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:40 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T18:18:40.964 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:40 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:18:40.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:40 smithi033 ceph-mon[28330]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-05-03T18:18:40.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:40 smithi033 ceph-mon[28330]: from='client.24304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:18:40.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:40 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T18:18:40.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:40 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T18:18:40.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:40 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:18:41.131 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:40 smithi150 ceph-mon[32492]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-05-03T18:18:41.131 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:40 smithi150 ceph-mon[32492]: from='client.24304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:18:41.131 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:40 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T18:18:41.131 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:40 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T18:18:41.131 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:40 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:18:41.881 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:41 smithi150 ceph-mon[32492]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-05-03T18:18:41.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:41 smithi033 ceph-mon[28330]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-05-03T18:18:42.044 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:41 smithi099 ceph-mon[31658]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-05-03T18:18:44.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:44 smithi150 ceph-mon[32492]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-05-03T18:18:44.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:44 smithi033 ceph-mon[28330]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-05-03T18:18:44.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:44 smithi099 ceph-mon[31658]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-05-03T18:18:45.381 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:45 smithi150 ceph-mon[32492]: from='client.? 172.21.15.150:0/1581758544' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "84c9d037-eb84-4eb4-8aa2-e784b703d36c"}]: dispatch 2024-05-03T18:18:45.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:45 smithi150 ceph-mon[32492]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "84c9d037-eb84-4eb4-8aa2-e784b703d36c"}]: dispatch 2024-05-03T18:18:45.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:45 smithi150 ceph-mon[32492]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "84c9d037-eb84-4eb4-8aa2-e784b703d36c"}]': finished 2024-05-03T18:18:45.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:45 smithi150 ceph-mon[32492]: osdmap e50: 9 total, 8 up, 9 in 2024-05-03T18:18:45.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:45 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-03T18:18:45.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:45 smithi033 ceph-mon[28330]: from='client.? 172.21.15.150:0/1581758544' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "84c9d037-eb84-4eb4-8aa2-e784b703d36c"}]: dispatch 2024-05-03T18:18:45.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:45 smithi033 ceph-mon[28330]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "84c9d037-eb84-4eb4-8aa2-e784b703d36c"}]: dispatch 2024-05-03T18:18:45.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:45 smithi033 ceph-mon[28330]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "84c9d037-eb84-4eb4-8aa2-e784b703d36c"}]': finished 2024-05-03T18:18:45.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:45 smithi033 ceph-mon[28330]: osdmap e50: 9 total, 8 up, 9 in 2024-05-03T18:18:45.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:45 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-03T18:18:45.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:45 smithi099 ceph-mon[31658]: from='client.? 172.21.15.150:0/1581758544' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "84c9d037-eb84-4eb4-8aa2-e784b703d36c"}]: dispatch 2024-05-03T18:18:45.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:45 smithi099 ceph-mon[31658]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "84c9d037-eb84-4eb4-8aa2-e784b703d36c"}]: dispatch 2024-05-03T18:18:45.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:45 smithi099 ceph-mon[31658]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "84c9d037-eb84-4eb4-8aa2-e784b703d36c"}]': finished 2024-05-03T18:18:45.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:45 smithi099 ceph-mon[31658]: osdmap e50: 9 total, 8 up, 9 in 2024-05-03T18:18:45.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:45 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-03T18:18:46.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:46 smithi150 ceph-mon[32492]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-05-03T18:18:46.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:46 smithi150 ceph-mon[32492]: from='client.? 172.21.15.150:0/4257446560' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T18:18:46.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:46 smithi033 ceph-mon[28330]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-05-03T18:18:46.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:46 smithi033 ceph-mon[28330]: from='client.? 172.21.15.150:0/4257446560' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T18:18:46.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:46 smithi099 ceph-mon[31658]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-05-03T18:18:46.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:46 smithi099 ceph-mon[31658]: from='client.? 172.21.15.150:0/4257446560' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T18:18:48.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:48 smithi150 ceph-mon[32492]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-03T18:18:48.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:48 smithi033 ceph-mon[28330]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-03T18:18:48.794 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:48 smithi099 ceph-mon[31658]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-03T18:18:50.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:50 smithi150 ceph-mon[32492]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-03T18:18:50.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:50 smithi033 ceph-mon[28330]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-03T18:18:50.794 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:50 smithi099 ceph-mon[31658]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-03T18:18:52.608 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:52 smithi150 ceph-mon[32492]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-03T18:18:52.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:52 smithi033 ceph-mon[28330]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-03T18:18:52.794 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:52 smithi099 ceph-mon[31658]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-03T18:18:54.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:54 smithi150 ceph-mon[32492]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-03T18:18:54.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:54 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-03T18:18:54.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:54 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:18:54.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:54 smithi033 ceph-mon[28330]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-03T18:18:54.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:54 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-03T18:18:54.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:54 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:18:54.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:54 smithi099 ceph-mon[31658]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-03T18:18:54.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:54 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-03T18:18:54.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:54 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:18:55.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:55 smithi150 ceph-mon[32492]: Deploying daemon osd.8 on smithi150 2024-05-03T18:18:55.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:55 smithi033 ceph-mon[28330]: Deploying daemon osd.8 on smithi150 2024-05-03T18:18:55.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:55 smithi099 ceph-mon[31658]: Deploying daemon osd.8 on smithi150 2024-05-03T18:18:56.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:56 smithi150 ceph-mon[32492]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-03T18:18:56.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:56 smithi033 ceph-mon[28330]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-03T18:18:56.794 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:56 smithi099 ceph-mon[31658]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-03T18:18:58.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:18:58 smithi150 ceph-mon[32492]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-03T18:18:58.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:18:58 smithi033 ceph-mon[28330]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-03T18:18:58.794 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:18:58 smithi099 ceph-mon[31658]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-03T18:19:00.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:00 smithi150 ceph-mon[32492]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-03T18:19:00.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:00 smithi033 ceph-mon[28330]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-03T18:19:00.794 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:00 smithi099 ceph-mon[31658]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-03T18:19:02.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:02 smithi150 ceph-mon[32492]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-03T18:19:02.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:02 smithi033 ceph-mon[28330]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-03T18:19:02.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:02 smithi099 ceph-mon[31658]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-03T18:19:04.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:04 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:19:04.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:04 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:19:04.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:04 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:19:04.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:04 smithi099 ceph-mon[31658]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-03T18:19:04.298 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:04 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:19:04.298 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:04 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:19:04.298 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:04 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:19:04.298 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:04 smithi150 ceph-mon[32492]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-03T18:19:04.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:04 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:19:04.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:04 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:19:04.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:04 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:19:04.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:04 smithi033 ceph-mon[28330]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-03T18:19:06.564 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:06 smithi150 ceph-mon[32492]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-03T18:19:06.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:06 smithi033 ceph-mon[28330]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-03T18:19:06.794 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:06 smithi099 ceph-mon[31658]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-03T18:19:08.099 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:19:08.099+0000 7fe6c3400700 1 -- 172.21.15.150:0/2422283552 <== mgr.14152 v2:172.21.15.33:6800/3642895501 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe6cc0639a0 con 0x7fe6b00b2900 2024-05-03T18:19:08.100 INFO:teuthology.orchestra.run.smithi150.stdout:Created osd(s) 8 on host 'smithi150' 2024-05-03T18:19:08.102 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:19:08.102+0000 7fe6d1c00700 1 -- 172.21.15.150:0/2422283552 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fe6b00b2900 msgr2=0x7fe6b00b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:19:08.102 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:19:08.102+0000 7fe6d1c00700 1 --2- 172.21.15.150:0/2422283552 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fe6b00b2900 0x7fe6b00b4dc0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc06af80 tx=0x7fe6bc009190 comp rx=0 tx=0).stop 2024-05-03T18:19:08.103 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:19:08.102+0000 7fe6d1c00700 1 -- 172.21.15.150:0/2422283552 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe6cc104790 msgr2=0x7fe6cc102740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:19:08.103 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:19:08.102+0000 7fe6d1c00700 1 --2- 172.21.15.150:0/2422283552 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe6cc104790 0x7fe6cc102740 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe6b800c920 tx=0x7fe6b800cc30 comp rx=0 tx=0).stop 2024-05-03T18:19:08.103 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:19:08.102+0000 7fe6d1c00700 1 -- 172.21.15.150:0/2422283552 shutdown_connections 2024-05-03T18:19:08.103 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:19:08.102+0000 7fe6d1c00700 1 --2- 172.21.15.150:0/2422283552 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6cc105130 0x7fe6cc102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:19:08.103 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:19:08.102+0000 7fe6d1c00700 1 --2- 172.21.15.150:0/2422283552 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fe6b00b2900 0x7fe6b00b4dc0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:19:08.103 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:19:08.102+0000 7fe6d1c00700 1 --2- 172.21.15.150:0/2422283552 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe6cc1017b0 0x7fe6cc1040a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:19:08.103 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:19:08.102+0000 7fe6d1c00700 1 --2- 172.21.15.150:0/2422283552 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe6cc104790 0x7fe6cc102740 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:19:08.103 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:19:08.102+0000 7fe6d1c00700 1 -- 172.21.15.150:0/2422283552 >> 172.21.15.150:0/2422283552 conn(0x7fe6cc0fd130 msgr2=0x7fe6cc107c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:19:08.103 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:19:08.102+0000 7fe6d1c00700 1 -- 172.21.15.150:0/2422283552 shutdown_connections 2024-05-03T18:19:08.103 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:19:08.103+0000 7fe6d1c00700 1 -- 172.21.15.150:0/2422283552 wait complete. 2024-05-03T18:19:08.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:08 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:19:08.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:08 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:19:08.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:08 smithi099 ceph-mon[31658]: from='osd.8 [v2:172.21.15.150:6816/1308083602,v1:172.21.15.150:6817/1308083602]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-03T18:19:08.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:08 smithi099 ceph-mon[31658]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-03T18:19:08.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:08 smithi099 ceph-mon[31658]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-03T18:19:08.381 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:08 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:19:08.381 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:08 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:19:08.381 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:08 smithi150 ceph-mon[32492]: from='osd.8 [v2:172.21.15.150:6816/1308083602,v1:172.21.15.150:6817/1308083602]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-03T18:19:08.381 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:08 smithi150 ceph-mon[32492]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-03T18:19:08.381 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:08 smithi150 ceph-mon[32492]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-03T18:19:08.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:08 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:19:08.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:08 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:19:08.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:08 smithi033 ceph-mon[28330]: from='osd.8 [v2:172.21.15.150:6816/1308083602,v1:172.21.15.150:6817/1308083602]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-03T18:19:08.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:08 smithi033 ceph-mon[28330]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-03T18:19:08.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:08 smithi033 ceph-mon[28330]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-03T18:19:08.979 DEBUG:teuthology.orchestra.run.smithi150:osd.8> sudo journalctl -f -n 0 -u ceph-5703bba4-0978-11ef-bc97-c7b262605968@osd.8.service 2024-05-03T18:19:08.985 INFO:tasks.cephadm:Deploying osd.9 on smithi033 with /dev/vg_nvme/lv_1... 2024-05-03T18:19:08.986 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-03T18:19:09.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:09 smithi099 ceph-mon[31658]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-03T18:19:09.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:09 smithi099 ceph-mon[31658]: from='osd.8 [v2:172.21.15.150:6816/1308083602,v1:172.21.15.150:6817/1308083602]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-03T18:19:09.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:09 smithi099 ceph-mon[31658]: osdmap e51: 9 total, 8 up, 9 in 2024-05-03T18:19:09.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:09 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-03T18:19:09.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:09 smithi099 ceph-mon[31658]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-03T18:19:09.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:09 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:19:09.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:09 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:19:09.381 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:09 smithi150 ceph-mon[32492]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-03T18:19:09.381 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:09 smithi150 ceph-mon[32492]: from='osd.8 [v2:172.21.15.150:6816/1308083602,v1:172.21.15.150:6817/1308083602]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-03T18:19:09.381 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:09 smithi150 ceph-mon[32492]: osdmap e51: 9 total, 8 up, 9 in 2024-05-03T18:19:09.381 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:09 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-03T18:19:09.381 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:09 smithi150 ceph-mon[32492]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-03T18:19:09.381 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:09 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:19:09.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:09 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:19:09.382 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:19:09 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:19:09.077+0000 7f0d89600700 -1 osd.8 52 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-03T18:19:09.443 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:19:09.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:09 smithi033 ceph-mon[28330]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-03T18:19:09.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:09 smithi033 ceph-mon[28330]: from='osd.8 [v2:172.21.15.150:6816/1308083602,v1:172.21.15.150:6817/1308083602]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-03T18:19:09.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:09 smithi033 ceph-mon[28330]: osdmap e51: 9 total, 8 up, 9 in 2024-05-03T18:19:09.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:09 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-03T18:19:09.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:09 smithi033 ceph-mon[28330]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-03T18:19:09.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:09 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:19:09.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:09 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:19:10.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:10 smithi150 ceph-mon[32492]: purged_snaps scrub starts 2024-05-03T18:19:10.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:10 smithi150 ceph-mon[32492]: purged_snaps scrub ok 2024-05-03T18:19:10.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:10 smithi150 ceph-mon[32492]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-05-03T18:19:10.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:10 smithi150 ceph-mon[32492]: osdmap e52: 9 total, 8 up, 9 in 2024-05-03T18:19:10.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:10 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-03T18:19:10.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:10 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-03T18:19:10.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:10 smithi150 ceph-mon[32492]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-03T18:19:10.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:10 smithi033 ceph-mon[28330]: purged_snaps scrub starts 2024-05-03T18:19:10.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:10 smithi033 ceph-mon[28330]: purged_snaps scrub ok 2024-05-03T18:19:10.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:10 smithi033 ceph-mon[28330]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-05-03T18:19:10.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:10 smithi033 ceph-mon[28330]: osdmap e52: 9 total, 8 up, 9 in 2024-05-03T18:19:10.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:10 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-03T18:19:10.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:10 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-03T18:19:10.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:10 smithi033 ceph-mon[28330]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-03T18:19:10.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:10 smithi099 ceph-mon[31658]: purged_snaps scrub starts 2024-05-03T18:19:10.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:10 smithi099 ceph-mon[31658]: purged_snaps scrub ok 2024-05-03T18:19:10.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:10 smithi099 ceph-mon[31658]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-05-03T18:19:10.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:10 smithi099 ceph-mon[31658]: osdmap e52: 9 total, 8 up, 9 in 2024-05-03T18:19:10.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:10 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-03T18:19:10.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:10 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-03T18:19:10.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:10 smithi099 ceph-mon[31658]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-03T18:19:11.330 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:11 smithi033 ceph-mon[28330]: osd.8 [v2:172.21.15.150:6816/1308083602,v1:172.21.15.150:6817/1308083602] boot 2024-05-03T18:19:11.330 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:11 smithi033 ceph-mon[28330]: osdmap e53: 9 total, 9 up, 9 in 2024-05-03T18:19:11.331 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:11 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-03T18:19:11.331 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:11 smithi033 ceph-mon[28330]: osdmap e54: 9 total, 9 up, 9 in 2024-05-03T18:19:11.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:11 smithi150 ceph-mon[32492]: osd.8 [v2:172.21.15.150:6816/1308083602,v1:172.21.15.150:6817/1308083602] boot 2024-05-03T18:19:11.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:11 smithi150 ceph-mon[32492]: osdmap e53: 9 total, 9 up, 9 in 2024-05-03T18:19:11.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:11 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-03T18:19:11.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:11 smithi150 ceph-mon[32492]: osdmap e54: 9 total, 9 up, 9 in 2024-05-03T18:19:11.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:11 smithi099 ceph-mon[31658]: osd.8 [v2:172.21.15.150:6816/1308083602,v1:172.21.15.150:6817/1308083602] boot 2024-05-03T18:19:11.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:11 smithi099 ceph-mon[31658]: osdmap e53: 9 total, 9 up, 9 in 2024-05-03T18:19:11.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:11 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-03T18:19:11.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:11 smithi099 ceph-mon[31658]: osdmap e54: 9 total, 9 up, 9 in 2024-05-03T18:19:12.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:12 smithi033 ceph-mon[28330]: pgmap v250: 1 pgs: 1 peering; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-03T18:19:12.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:12 smithi033 ceph-mon[28330]: osdmap e55: 9 total, 9 up, 9 in 2024-05-03T18:19:12.294 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:12 smithi099 ceph-mon[31658]: pgmap v250: 1 pgs: 1 peering; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-03T18:19:12.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:12 smithi099 ceph-mon[31658]: osdmap e55: 9 total, 9 up, 9 in 2024-05-03T18:19:12.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:12 smithi150 ceph-mon[32492]: pgmap v250: 1 pgs: 1 peering; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-03T18:19:12.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:12 smithi150 ceph-mon[32492]: osdmap e55: 9 total, 9 up, 9 in 2024-05-03T18:19:12.799 INFO:teuthology.orchestra.run.smithi033.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-03T18:19:12.800 INFO:teuthology.orchestra.run.smithi033.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-03T18:19:12.800 INFO:teuthology.orchestra.run.smithi033.stderr: stderr: 10+0 records in 2024-05-03T18:19:12.800 INFO:teuthology.orchestra.run.smithi033.stderr:10+0 records out 2024-05-03T18:19:12.800 INFO:teuthology.orchestra.run.smithi033.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0319508 s, 328 MB/s 2024-05-03T18:19:12.800 INFO:teuthology.orchestra.run.smithi033.stderr:--> Zapping successful for: 2024-05-03T18:19:13.749 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi033:vg_nvme/lv_1 2024-05-03T18:19:14.205 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:19:14.580 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:14 smithi033 ceph-mon[28330]: pgmap v252: 1 pgs: 1 peering; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-03T18:19:14.580 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:14 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:19:14.580 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:14 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:19:14.580 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:14 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:19:14.580 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:14 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:19:14.580 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:14 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:19:14.580 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:14 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:19:14.580 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:14 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:19:14.580 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:14 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:19:14.581 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:14 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:19:14.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:14 smithi150 ceph-mon[32492]: pgmap v252: 1 pgs: 1 peering; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-03T18:19:14.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:14 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:19:14.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:14 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:19:14.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:14 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:19:14.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:14 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:19:14.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:14 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:19:14.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:14 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:19:14.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:14 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:19:14.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:14 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:19:14.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:14 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:19:14.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:14 smithi099 ceph-mon[31658]: pgmap v252: 1 pgs: 1 peering; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-03T18:19:14.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:14 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:19:14.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:14 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:19:14.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:14 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:19:14.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:14 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:19:14.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:14 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:19:14.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:14 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:19:14.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:14 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:19:14.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:14 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:19:14.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:14 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:19:15.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:15 smithi150 ceph-mon[32492]: Detected new or changed devices on smithi150 2024-05-03T18:19:15.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:15 smithi150 ceph-mon[32492]: Adjusting osd_memory_target on smithi150 to 5696M 2024-05-03T18:19:15.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:15 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:19:15.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:15 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:19:15.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:15 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:19:15.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:15 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:19:15.693 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:15 smithi033 ceph-mon[28330]: Detected new or changed devices on smithi150 2024-05-03T18:19:15.693 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:15 smithi033 ceph-mon[28330]: Adjusting osd_memory_target on smithi150 to 5696M 2024-05-03T18:19:15.694 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:15 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:19:15.694 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:15 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:19:15.694 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:15 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:19:15.694 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:15 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:19:15.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:15 smithi099 ceph-mon[31658]: Detected new or changed devices on smithi150 2024-05-03T18:19:15.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:15 smithi099 ceph-mon[31658]: Adjusting osd_memory_target on smithi150 to 5696M 2024-05-03T18:19:15.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:15 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:19:15.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:15 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:19:15.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:15 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:19:15.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:15 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:19:16.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:19:16.499+0000 7eff6be00700 1 -- 172.21.15.33:0/2248038135 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff6c0774d0 msgr2=0x7eff6c075930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:19:16.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:19:16.499+0000 7eff6be00700 1 --2- 172.21.15.33:0/2248038135 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff6c0774d0 0x7eff6c075930 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7eff5c009990 tx=0x7eff5c009ca0 comp rx=0 tx=0).stop 2024-05-03T18:19:16.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:19:16.499+0000 7eff6be00700 1 -- 172.21.15.33:0/2248038135 shutdown_connections 2024-05-03T18:19:16.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:19:16.499+0000 7eff6be00700 1 --2- 172.21.15.33:0/2248038135 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff6c10fa10 0x7eff6c111e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:19:16.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:19:16.499+0000 7eff6be00700 1 --2- 172.21.15.33:0/2248038135 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff6c0774d0 0x7eff6c075930 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:19:16.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:19:16.499+0000 7eff6be00700 1 --2- 172.21.15.33:0/2248038135 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7eff6c075f00 0x7eff6c076380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:19:16.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:19:16.499+0000 7eff6be00700 1 -- 172.21.15.33:0/2248038135 >> 172.21.15.33:0/2248038135 conn(0x7eff6c0fe9d0 msgr2=0x7eff6c100e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:19:16.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:19:16.499+0000 7eff6be00700 1 -- 172.21.15.33:0/2248038135 shutdown_connections 2024-05-03T18:19:16.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:19:16.499+0000 7eff6be00700 1 -- 172.21.15.33:0/2248038135 wait complete. 2024-05-03T18:19:16.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:19:16.500+0000 7eff6be00700 1 Processor -- start 2024-05-03T18:19:16.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:19:16.500+0000 7eff6be00700 1 -- start start 2024-05-03T18:19:16.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:19:16.501+0000 7eff6be00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7eff6c075f00 0x7eff6c19fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:19:16.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:19:16.501+0000 7eff6be00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff6c0774d0 0x7eff6c1a0020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:19:16.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:19:16.501+0000 7eff6be00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff6c10fa10 0x7eff6c1a70a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:19:16.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:19:16.501+0000 7eff6be00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7eff6c1153c0 con 0x7eff6c0774d0 2024-05-03T18:19:16.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:19:16.501+0000 7eff6be00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7eff6c115240 con 0x7eff6c10fa10 2024-05-03T18:19:16.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:19:16.501+0000 7eff6be00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7eff6c115540 con 0x7eff6c075f00 2024-05-03T18:19:16.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:19:16.501+0000 7eff62000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff6c0774d0 0x7eff6c1a0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:19:16.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:19:16.501+0000 7eff62000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff6c0774d0 0x7eff6c1a0020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41642/0 (socket says 172.21.15.33:41642) 2024-05-03T18:19:16.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:19:16.501+0000 7eff62000700 1 -- 172.21.15.33:0/4277224913 learned_addr learned my addr 172.21.15.33:0/4277224913 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:19:16.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:19:16.501+0000 7eff6b400700 1 --2- 172.21.15.33:0/4277224913 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff6c10fa10 0x7eff6c1a70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:19:16.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:19:16.501+0000 7eff6aa00700 1 --2- 172.21.15.33:0/4277224913 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7eff6c075f00 0x7eff6c19fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:19:16.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:19:16.502+0000 7eff62000700 1 -- 172.21.15.33:0/4277224913 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7eff6c075f00 msgr2=0x7eff6c19fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:19:16.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:19:16.502+0000 7eff62000700 1 --2- 172.21.15.33:0/4277224913 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7eff6c075f00 0x7eff6c19fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:19:16.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:19:16.502+0000 7eff62000700 1 -- 172.21.15.33:0/4277224913 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff6c10fa10 msgr2=0x7eff6c1a70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:19:16.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:19:16.502+0000 7eff62000700 1 --2- 172.21.15.33:0/4277224913 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff6c10fa10 0x7eff6c1a70a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:19:16.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:19:16.502+0000 7eff62000700 1 -- 172.21.15.33:0/4277224913 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff6c1a7740 con 0x7eff6c0774d0 2024-05-03T18:19:16.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:19:16.502+0000 7eff62000700 1 --2- 172.21.15.33:0/4277224913 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff6c0774d0 0x7eff6c1a0020 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7eff5400d9b0 tx=0x7eff5400dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:19:16.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:19:16.502+0000 7eff63e00700 1 -- 172.21.15.33:0/4277224913 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff54017070 con 0x7eff6c0774d0 2024-05-03T18:19:16.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:19:16.502+0000 7eff63e00700 1 -- 172.21.15.33:0/4277224913 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7eff54010460 con 0x7eff6c0774d0 2024-05-03T18:19:16.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:19:16.502+0000 7eff6be00700 1 -- 172.21.15.33:0/4277224913 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff6c1a7a30 con 0x7eff6c0774d0 2024-05-03T18:19:16.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:19:16.502+0000 7eff63e00700 1 -- 172.21.15.33:0/4277224913 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff5400f620 con 0x7eff6c0774d0 2024-05-03T18:19:16.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:19:16.502+0000 7eff6be00700 1 -- 172.21.15.33:0/4277224913 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff6c1a7f20 con 0x7eff6c0774d0 2024-05-03T18:19:16.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:19:16.503+0000 7eff6be00700 1 -- 172.21.15.33:0/4277224913 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff6c06a190 con 0x7eff6c0774d0 2024-05-03T18:19:16.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:19:16.506+0000 7eff63e00700 1 -- 172.21.15.33:0/4277224913 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7eff54010610 con 0x7eff6c0774d0 2024-05-03T18:19:16.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:19:16.508+0000 7eff63e00700 1 --2- 172.21.15.33:0/4277224913 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7eff4c0b2880 0x7eff4c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:19:16.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:19:16.508+0000 7eff63e00700 1 -- 172.21.15.33:0/4277224913 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5829+0+0 (secure 0 0 0) 0x7eff540e1430 con 0x7eff6c0774d0 2024-05-03T18:19:16.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:19:16.508+0000 7eff6aa00700 1 --2- 172.21.15.33:0/4277224913 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7eff4c0b2880 0x7eff4c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:19:16.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:19:16.509+0000 7eff6aa00700 1 --2- 172.21.15.33:0/4277224913 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7eff4c0b2880 0x7eff4c0b4d40 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7eff5c00b5c0 tx=0x7eff5c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:19:16.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:19:16.511+0000 7eff63e00700 1 -- 172.21.15.33:0/4277224913 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7eff540a9a60 con 0x7eff6c0774d0 2024-05-03T18:19:16.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:19:16.758+0000 7eff6be00700 1 -- 172.21.15.33:0/4277224913 --> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7eff6c0638e0 con 0x7eff4c0b2880 2024-05-03T18:19:16.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:16 smithi033 ceph-mon[28330]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-03T18:19:17.044 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:16 smithi099 ceph-mon[31658]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-03T18:19:17.131 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:16 smithi150 ceph-mon[32492]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-03T18:19:17.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:17 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T18:19:17.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:17 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T18:19:17.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:17 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:19:18.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:17 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T18:19:18.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:17 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T18:19:18.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:17 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:19:18.131 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:17 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T18:19:18.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:17 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T18:19:18.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:17 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:19:18.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:18 smithi033 ceph-mon[28330]: from='client.14403 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:19:18.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:18 smithi033 ceph-mon[28330]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-05-03T18:19:19.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:18 smithi099 ceph-mon[31658]: from='client.14403 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:19:19.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:18 smithi099 ceph-mon[31658]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-05-03T18:19:19.131 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:18 smithi150 ceph-mon[32492]: from='client.14403 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:19:19.131 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:18 smithi150 ceph-mon[32492]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-05-03T18:19:20.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:20 smithi033 ceph-mon[28330]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-05-03T18:19:21.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:20 smithi099 ceph-mon[31658]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-05-03T18:19:21.131 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:20 smithi150 ceph-mon[32492]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-05-03T18:19:21.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:21 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/174928937' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a02e965f-0d8f-45e2-9490-337f7f6e3881"}]: dispatch 2024-05-03T18:19:21.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:21 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/174928937' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a02e965f-0d8f-45e2-9490-337f7f6e3881"}]': finished 2024-05-03T18:19:21.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:21 smithi033 ceph-mon[28330]: osdmap e56: 10 total, 9 up, 10 in 2024-05-03T18:19:21.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:21 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-03T18:19:22.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:21 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/174928937' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a02e965f-0d8f-45e2-9490-337f7f6e3881"}]: dispatch 2024-05-03T18:19:22.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:21 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/174928937' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a02e965f-0d8f-45e2-9490-337f7f6e3881"}]': finished 2024-05-03T18:19:22.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:21 smithi099 ceph-mon[31658]: osdmap e56: 10 total, 9 up, 10 in 2024-05-03T18:19:22.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:21 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-03T18:19:22.131 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:21 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/174928937' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a02e965f-0d8f-45e2-9490-337f7f6e3881"}]: dispatch 2024-05-03T18:19:22.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:21 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/174928937' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a02e965f-0d8f-45e2-9490-337f7f6e3881"}]': finished 2024-05-03T18:19:22.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:21 smithi150 ceph-mon[32492]: osdmap e56: 10 total, 9 up, 10 in 2024-05-03T18:19:22.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:21 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-03T18:19:22.922 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:22 smithi099 ceph-mon[31658]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-05-03T18:19:22.922 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:22 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/2947009521' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T18:19:22.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:22 smithi033 ceph-mon[28330]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-05-03T18:19:22.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:22 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/2947009521' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T18:19:23.131 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:22 smithi150 ceph-mon[32492]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-05-03T18:19:23.131 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:22 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/2947009521' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T18:19:24.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:24 smithi033 ceph-mon[28330]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-03T18:19:25.044 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:24 smithi099 ceph-mon[31658]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-03T18:19:25.131 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:24 smithi150 ceph-mon[32492]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-03T18:19:26.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:26 smithi033 ceph-mon[28330]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-03T18:19:27.044 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:26 smithi099 ceph-mon[31658]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-03T18:19:27.131 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:26 smithi150 ceph-mon[32492]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-03T18:19:28.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:28 smithi033 ceph-mon[28330]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-03T18:19:29.044 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:28 smithi099 ceph-mon[31658]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-03T18:19:29.131 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:28 smithi150 ceph-mon[32492]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-03T18:19:30.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:30 smithi033 ceph-mon[28330]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-03T18:19:31.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:30 smithi099 ceph-mon[31658]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-03T18:19:31.131 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:30 smithi150 ceph-mon[32492]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-03T18:19:31.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:31 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-03T18:19:31.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:31 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:19:32.033 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:31 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-03T18:19:32.033 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:31 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:19:32.044 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:31 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-03T18:19:32.044 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:31 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:19:32.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:32 smithi033 ceph-mon[28330]: Deploying daemon osd.9 on smithi033 2024-05-03T18:19:32.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:32 smithi033 ceph-mon[28330]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-03T18:19:33.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:32 smithi099 ceph-mon[31658]: Deploying daemon osd.9 on smithi033 2024-05-03T18:19:33.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:32 smithi099 ceph-mon[31658]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-03T18:19:33.131 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:32 smithi150 ceph-mon[32492]: Deploying daemon osd.9 on smithi033 2024-05-03T18:19:33.131 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:32 smithi150 ceph-mon[32492]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-03T18:19:34.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:34 smithi033 ceph-mon[28330]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-03T18:19:35.044 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:34 smithi099 ceph-mon[31658]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-03T18:19:35.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:34 smithi150 ceph-mon[32492]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-03T18:19:36.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:36 smithi033 ceph-mon[28330]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-03T18:19:37.044 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:36 smithi099 ceph-mon[31658]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-03T18:19:37.131 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:36 smithi150 ceph-mon[32492]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-03T18:19:38.973 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:38 smithi033 ceph-mon[28330]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-03T18:19:39.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:38 smithi099 ceph-mon[31658]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-03T18:19:39.131 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:38 smithi150 ceph-mon[32492]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-03T18:19:39.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:39 smithi033 ceph-mon[28330]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-03T18:19:40.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:39 smithi099 ceph-mon[31658]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-03T18:19:40.131 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:39 smithi150 ceph-mon[32492]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-03T18:19:41.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:41 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:19:41.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:41 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:19:41.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:41 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:19:41.597 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:41 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:19:41.597 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:41 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:19:41.597 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:41 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:19:41.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:41 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:19:41.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:41 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:19:41.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:41 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:19:42.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:42 smithi033 ceph-mon[28330]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-03T18:19:42.544 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:42 smithi099 ceph-mon[31658]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-03T18:19:42.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:42 smithi150 ceph-mon[32492]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-03T18:19:44.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:44 smithi099 ceph-mon[31658]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-03T18:19:44.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:44 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:19:44.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:44 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:19:44.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:44 smithi099 ceph-mon[31658]: from='osd.9 [v2:172.21.15.33:6826/4133096905,v1:172.21.15.33:6827/4133096905]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-03T18:19:44.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:44 smithi150 ceph-mon[32492]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-03T18:19:44.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:44 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:19:44.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:44 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:19:44.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:44 smithi150 ceph-mon[32492]: from='osd.9 [v2:172.21.15.33:6826/4133096905,v1:172.21.15.33:6827/4133096905]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-03T18:19:44.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:44 smithi033 ceph-mon[28330]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-03T18:19:44.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:44 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:19:44.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:44 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:19:44.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:44 smithi033 ceph-mon[28330]: from='osd.9 [v2:172.21.15.33:6826/4133096905,v1:172.21.15.33:6827/4133096905]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-03T18:19:45.259 INFO:teuthology.orchestra.run.smithi033.stdout:Created osd(s) 9 on host 'smithi033' 2024-05-03T18:19:45.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:19:45.258+0000 7eff63e00700 1 -- 172.21.15.33:0/4277224913 <== mgr.14152 v2:172.21.15.33:6800/3642895501 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7eff6c0638e0 con 0x7eff4c0b2880 2024-05-03T18:19:45.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:19:45.261+0000 7eff6be00700 1 -- 172.21.15.33:0/4277224913 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7eff4c0b2880 msgr2=0x7eff4c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:19:45.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:19:45.261+0000 7eff6be00700 1 --2- 172.21.15.33:0/4277224913 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7eff4c0b2880 0x7eff4c0b4d40 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7eff5c00b5c0 tx=0x7eff5c005fd0 comp rx=0 tx=0).stop 2024-05-03T18:19:45.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:19:45.261+0000 7eff6be00700 1 -- 172.21.15.33:0/4277224913 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff6c0774d0 msgr2=0x7eff6c1a0020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:19:45.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:19:45.261+0000 7eff6be00700 1 --2- 172.21.15.33:0/4277224913 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff6c0774d0 0x7eff6c1a0020 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7eff5400d9b0 tx=0x7eff5400dd70 comp rx=0 tx=0).stop 2024-05-03T18:19:45.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:19:45.261+0000 7eff6be00700 1 -- 172.21.15.33:0/4277224913 shutdown_connections 2024-05-03T18:19:45.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:19:45.261+0000 7eff6be00700 1 --2- 172.21.15.33:0/4277224913 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7eff4c0b2880 0x7eff4c0b4d40 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:19:45.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:19:45.261+0000 7eff6be00700 1 --2- 172.21.15.33:0/4277224913 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff6c10fa10 0x7eff6c1a70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:19:45.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:19:45.261+0000 7eff6be00700 1 --2- 172.21.15.33:0/4277224913 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7eff6c075f00 0x7eff6c19fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:19:45.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:19:45.261+0000 7eff6be00700 1 --2- 172.21.15.33:0/4277224913 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff6c0774d0 0x7eff6c1a0020 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:19:45.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:19:45.261+0000 7eff6be00700 1 -- 172.21.15.33:0/4277224913 >> 172.21.15.33:0/4277224913 conn(0x7eff6c0fe9d0 msgr2=0x7eff6c102db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:19:45.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:19:45.262+0000 7eff6be00700 1 -- 172.21.15.33:0/4277224913 shutdown_connections 2024-05-03T18:19:45.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:19:45.262+0000 7eff6be00700 1 -- 172.21.15.33:0/4277224913 wait complete. 2024-05-03T18:19:46.210 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:45 smithi033 ceph-mon[28330]: from='osd.9 [v2:172.21.15.33:6826/4133096905,v1:172.21.15.33:6827/4133096905]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-03T18:19:46.211 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:45 smithi033 ceph-mon[28330]: osdmap e57: 10 total, 9 up, 10 in 2024-05-03T18:19:46.211 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:45 smithi033 ceph-mon[28330]: from='osd.9 [v2:172.21.15.33:6826/4133096905,v1:172.21.15.33:6827/4133096905]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-05-03T18:19:46.211 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:45 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-03T18:19:46.211 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:45 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:19:46.211 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:45 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:19:46.211 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:45 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:19:46.211 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:45 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:19:46.211 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:45 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:19:46.211 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:45 smithi033 ceph-mon[28330]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-03T18:19:46.211 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:45 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:19:46.268 DEBUG:teuthology.orchestra.run.smithi033:osd.9> sudo journalctl -f -n 0 -u ceph-5703bba4-0978-11ef-bc97-c7b262605968@osd.9.service 2024-05-03T18:19:46.273 INFO:tasks.cephadm:Deploying osd.10 on smithi099 with /dev/vg_nvme/lv_1... 2024-05-03T18:19:46.273 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-03T18:19:46.294 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:45 smithi099 ceph-mon[31658]: from='osd.9 [v2:172.21.15.33:6826/4133096905,v1:172.21.15.33:6827/4133096905]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-03T18:19:46.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:45 smithi099 ceph-mon[31658]: osdmap e57: 10 total, 9 up, 10 in 2024-05-03T18:19:46.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:45 smithi099 ceph-mon[31658]: from='osd.9 [v2:172.21.15.33:6826/4133096905,v1:172.21.15.33:6827/4133096905]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-05-03T18:19:46.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:45 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-03T18:19:46.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:45 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:19:46.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:45 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:19:46.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:45 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:19:46.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:45 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:19:46.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:45 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:19:46.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:45 smithi099 ceph-mon[31658]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-03T18:19:46.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:45 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:19:46.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:45 smithi150 ceph-mon[32492]: from='osd.9 [v2:172.21.15.33:6826/4133096905,v1:172.21.15.33:6827/4133096905]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-03T18:19:46.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:45 smithi150 ceph-mon[32492]: osdmap e57: 10 total, 9 up, 10 in 2024-05-03T18:19:46.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:45 smithi150 ceph-mon[32492]: from='osd.9 [v2:172.21.15.33:6826/4133096905,v1:172.21.15.33:6827/4133096905]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-05-03T18:19:46.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:45 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-03T18:19:46.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:45 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:19:46.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:45 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:19:46.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:45 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:19:46.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:45 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:19:46.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:45 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:19:46.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:45 smithi150 ceph-mon[32492]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-03T18:19:46.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:45 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:19:46.703 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.b/config 2024-05-03T18:19:47.100 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:46 smithi099 ceph-mon[31658]: from='osd.9 [v2:172.21.15.33:6826/4133096905,v1:172.21.15.33:6827/4133096905]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-05-03T18:19:47.100 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:46 smithi099 ceph-mon[31658]: osdmap e58: 10 total, 9 up, 10 in 2024-05-03T18:19:47.100 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:46 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-03T18:19:47.100 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:46 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-03T18:19:47.100 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:46 smithi099 ceph-mon[31658]: from='osd.9 [v2:172.21.15.33:6826/4133096905,v1:172.21.15.33:6827/4133096905]' entity='osd.9' 2024-05-03T18:19:47.224 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:46 smithi033 ceph-mon[28330]: from='osd.9 [v2:172.21.15.33:6826/4133096905,v1:172.21.15.33:6827/4133096905]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-05-03T18:19:47.224 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:46 smithi033 ceph-mon[28330]: osdmap e58: 10 total, 9 up, 10 in 2024-05-03T18:19:47.224 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:46 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-03T18:19:47.224 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:46 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-03T18:19:47.224 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:46 smithi033 ceph-mon[28330]: from='osd.9 [v2:172.21.15.33:6826/4133096905,v1:172.21.15.33:6827/4133096905]' entity='osd.9' 2024-05-03T18:19:47.225 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:19:46 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:19:46.904+0000 7f0fd8600700 -1 osd.9 0 waiting for initial osdmap 2024-05-03T18:19:47.225 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:19:46 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:19:46.946+0000 7f0fce400700 -1 osd.9 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-03T18:19:47.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:46 smithi150 ceph-mon[32492]: from='osd.9 [v2:172.21.15.33:6826/4133096905,v1:172.21.15.33:6827/4133096905]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-05-03T18:19:47.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:46 smithi150 ceph-mon[32492]: osdmap e58: 10 total, 9 up, 10 in 2024-05-03T18:19:47.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:46 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-03T18:19:47.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:46 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-03T18:19:47.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:46 smithi150 ceph-mon[32492]: from='osd.9 [v2:172.21.15.33:6826/4133096905,v1:172.21.15.33:6827/4133096905]' entity='osd.9' 2024-05-03T18:19:48.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:47 smithi099 ceph-mon[31658]: purged_snaps scrub starts 2024-05-03T18:19:48.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:47 smithi099 ceph-mon[31658]: purged_snaps scrub ok 2024-05-03T18:19:48.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:47 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-03T18:19:48.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:47 smithi099 ceph-mon[31658]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-03T18:19:48.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:47 smithi099 ceph-mon[31658]: osd.9 [v2:172.21.15.33:6826/4133096905,v1:172.21.15.33:6827/4133096905] boot 2024-05-03T18:19:48.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:47 smithi099 ceph-mon[31658]: osdmap e59: 10 total, 10 up, 10 in 2024-05-03T18:19:48.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:47 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-03T18:19:48.103 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:47 smithi033 ceph-mon[28330]: purged_snaps scrub starts 2024-05-03T18:19:48.103 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:47 smithi033 ceph-mon[28330]: purged_snaps scrub ok 2024-05-03T18:19:48.104 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:47 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-03T18:19:48.104 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:47 smithi033 ceph-mon[28330]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-03T18:19:48.104 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:47 smithi033 ceph-mon[28330]: osd.9 [v2:172.21.15.33:6826/4133096905,v1:172.21.15.33:6827/4133096905] boot 2024-05-03T18:19:48.104 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:47 smithi033 ceph-mon[28330]: osdmap e59: 10 total, 10 up, 10 in 2024-05-03T18:19:48.104 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:47 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-03T18:19:48.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:47 smithi150 ceph-mon[32492]: purged_snaps scrub starts 2024-05-03T18:19:48.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:47 smithi150 ceph-mon[32492]: purged_snaps scrub ok 2024-05-03T18:19:48.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:47 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-03T18:19:48.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:47 smithi150 ceph-mon[32492]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-03T18:19:48.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:47 smithi150 ceph-mon[32492]: osd.9 [v2:172.21.15.33:6826/4133096905,v1:172.21.15.33:6827/4133096905] boot 2024-05-03T18:19:48.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:47 smithi150 ceph-mon[32492]: osdmap e59: 10 total, 10 up, 10 in 2024-05-03T18:19:48.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:47 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-03T18:19:49.777 INFO:teuthology.orchestra.run.smithi099.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-03T18:19:49.778 INFO:teuthology.orchestra.run.smithi099.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-03T18:19:49.778 INFO:teuthology.orchestra.run.smithi099.stderr: stderr: 10+0 records in 2024-05-03T18:19:49.778 INFO:teuthology.orchestra.run.smithi099.stderr:10+0 records out 2024-05-03T18:19:49.778 INFO:teuthology.orchestra.run.smithi099.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0325792 s, 322 MB/s 2024-05-03T18:19:49.778 INFO:teuthology.orchestra.run.smithi099.stderr:--> Zapping successful for: 2024-05-03T18:19:50.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:49 smithi099 ceph-mon[31658]: osdmap e60: 10 total, 10 up, 10 in 2024-05-03T18:19:50.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:49 smithi099 ceph-mon[31658]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-03T18:19:50.233 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:49 smithi033 ceph-mon[28330]: osdmap e60: 10 total, 10 up, 10 in 2024-05-03T18:19:50.233 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:49 smithi033 ceph-mon[28330]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-03T18:19:50.381 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:49 smithi150 ceph-mon[32492]: osdmap e60: 10 total, 10 up, 10 in 2024-05-03T18:19:50.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:49 smithi150 ceph-mon[32492]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-03T18:19:50.758 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi099:vg_nvme/lv_1 2024-05-03T18:19:51.231 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.b/config 2024-05-03T18:19:52.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:52 smithi099 ceph-mon[31658]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-03T18:19:52.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:52 smithi099 ceph-mon[31658]: Detected new or changed devices on smithi033 2024-05-03T18:19:52.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:52 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:19:52.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:52 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:19:52.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:52 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:19:52.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:52 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:19:52.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:52 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:19:52.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:52 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:19:52.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:52 smithi099 ceph-mon[31658]: Adjusting osd_memory_target on smithi033 to 4273M 2024-05-03T18:19:52.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:52 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:19:52.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:52 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:19:52.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:52 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:19:52.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:52 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:19:52.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:52 smithi150 ceph-mon[32492]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-03T18:19:52.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:52 smithi150 ceph-mon[32492]: Detected new or changed devices on smithi033 2024-05-03T18:19:52.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:52 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:19:52.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:52 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:19:52.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:52 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:19:52.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:52 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:19:52.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:52 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:19:52.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:52 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:19:52.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:52 smithi150 ceph-mon[32492]: Adjusting osd_memory_target on smithi033 to 4273M 2024-05-03T18:19:52.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:52 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:19:52.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:52 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:19:52.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:52 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:19:52.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:52 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:19:52.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:52 smithi033 ceph-mon[28330]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-03T18:19:52.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:52 smithi033 ceph-mon[28330]: Detected new or changed devices on smithi033 2024-05-03T18:19:52.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:52 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:19:52.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:52 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:19:52.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:52 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:19:52.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:52 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:19:52.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:52 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:19:52.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:52 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:19:52.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:52 smithi033 ceph-mon[28330]: Adjusting osd_memory_target on smithi033 to 4273M 2024-05-03T18:19:52.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:52 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:19:52.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:52 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:19:52.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:52 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:19:52.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:52 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:19:53.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:19:53.532+0000 7f99c8000700 1 -- 172.21.15.99:0/4193343192 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f99c0079a60 msgr2=0x7f99c007a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:19:53.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:19:53.532+0000 7f99c8000700 1 --2- 172.21.15.99:0/4193343192 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f99c0079a60 0x7f99c007a2e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f99b800b0a0 tx=0x7f99b800b3b0 comp rx=0 tx=0).stop 2024-05-03T18:19:53.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:19:53.532+0000 7f99c8000700 1 -- 172.21.15.99:0/4193343192 shutdown_connections 2024-05-03T18:19:53.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:19:53.532+0000 7f99c8000700 1 --2- 172.21.15.99:0/4193343192 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f99c0079a60 0x7f99c007a2e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:19:53.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:19:53.532+0000 7f99c8000700 1 --2- 172.21.15.99:0/4193343192 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f99c0077e50 0x7f99c0078270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:19:53.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:19:53.532+0000 7f99c8000700 1 --2- 172.21.15.99:0/4193343192 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99c00790a0 0x7f99c0079520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:19:53.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:19:53.532+0000 7f99c8000700 1 -- 172.21.15.99:0/4193343192 >> 172.21.15.99:0/4193343192 conn(0x7f99c0100cf0 msgr2=0x7f99c0103110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:19:53.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:19:53.532+0000 7f99c8000700 1 -- 172.21.15.99:0/4193343192 shutdown_connections 2024-05-03T18:19:53.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:19:53.532+0000 7f99c8000700 1 -- 172.21.15.99:0/4193343192 wait complete. 2024-05-03T18:19:53.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:19:53.533+0000 7f99c8000700 1 Processor -- start 2024-05-03T18:19:53.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:19:53.533+0000 7f99c8000700 1 -- start start 2024-05-03T18:19:53.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:19:53.534+0000 7f99c8000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99c0077e50 0x7f99c019fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:19:53.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:19:53.534+0000 7f99c8000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f99c00790a0 0x7f99c019ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:19:53.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:19:53.534+0000 7f99c8000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f99c0079a60 0x7f99c01a7030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:19:53.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:19:53.534+0000 7f99c8000700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f99c0115510 con 0x7f99c0077e50 2024-05-03T18:19:53.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:19:53.534+0000 7f99c8000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f99c0115390 con 0x7f99c0079a60 2024-05-03T18:19:53.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:19:53.534+0000 7f99c8000700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f99c0115690 con 0x7f99c00790a0 2024-05-03T18:19:53.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:19:53.534+0000 7f99c6000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f99c0079a60 0x7f99c01a7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:19:53.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:19:53.534+0000 7f99c6000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f99c0079a60 0x7f99c01a7030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52620/0 (socket says 172.21.15.99:52620) 2024-05-03T18:19:53.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:19:53.534+0000 7f99c6000700 1 -- 172.21.15.99:0/3480354813 learned_addr learned my addr 172.21.15.99:0/3480354813 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-05-03T18:19:53.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:19:53.534+0000 7f99c4c00700 1 --2- 172.21.15.99:0/3480354813 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f99c00790a0 0x7f99c019ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:19:53.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:19:53.534+0000 7f99c5600700 1 --2- 172.21.15.99:0/3480354813 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99c0077e50 0x7f99c019fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:19:53.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:19:53.535+0000 7f99c6000700 1 -- 172.21.15.99:0/3480354813 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f99c00790a0 msgr2=0x7f99c019ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:19:53.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:19:53.535+0000 7f99c6000700 1 --2- 172.21.15.99:0/3480354813 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f99c00790a0 0x7f99c019ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:19:53.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:19:53.535+0000 7f99c6000700 1 -- 172.21.15.99:0/3480354813 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99c0077e50 msgr2=0x7f99c019fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:19:53.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:19:53.535+0000 7f99c6000700 1 --2- 172.21.15.99:0/3480354813 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99c0077e50 0x7f99c019fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:19:53.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:19:53.535+0000 7f99c6000700 1 -- 172.21.15.99:0/3480354813 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99c01a76d0 con 0x7f99c0079a60 2024-05-03T18:19:53.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:19:53.535+0000 7f99c6000700 1 --2- 172.21.15.99:0/3480354813 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f99c0079a60 0x7f99c01a7030 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f99b8007ed0 tx=0x7f99b800be20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:19:53.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:19:53.535+0000 7f99be000700 1 -- 172.21.15.99:0/3480354813 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b8021070 con 0x7f99c0079a60 2024-05-03T18:19:53.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:19:53.535+0000 7f99be000700 1 -- 172.21.15.99:0/3480354813 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f99b80054d0 con 0x7f99c0079a60 2024-05-03T18:19:53.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:19:53.535+0000 7f99c8000700 1 -- 172.21.15.99:0/3480354813 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99c01a7960 con 0x7f99c0079a60 2024-05-03T18:19:53.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:19:53.535+0000 7f99be000700 1 -- 172.21.15.99:0/3480354813 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b801b8d0 con 0x7f99c0079a60 2024-05-03T18:19:53.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:19:53.536+0000 7f99c8000700 1 -- 172.21.15.99:0/3480354813 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99c01a7db0 con 0x7f99c0079a60 2024-05-03T18:19:53.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:19:53.537+0000 7f99c8000700 1 -- 172.21.15.99:0/3480354813 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99c002e510 con 0x7f99c0079a60 2024-05-03T18:19:53.539 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:19:53.539+0000 7f99be000700 1 -- 172.21.15.99:0/3480354813 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f99b8024070 con 0x7f99c0079a60 2024-05-03T18:19:53.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:19:53.540+0000 7f99be000700 1 --2- 172.21.15.99:0/3480354813 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f99a80b2880 0x7f99a80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:19:53.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:19:53.541+0000 7f99be000700 1 -- 172.21.15.99:0/3480354813 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6261+0+0 (secure 0 0 0) 0x7f99b800f020 con 0x7f99c0079a60 2024-05-03T18:19:53.542 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:19:53.541+0000 7f99c5600700 1 --2- 172.21.15.99:0/3480354813 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f99a80b2880 0x7f99a80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:19:53.543 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:19:53.542+0000 7f99c5600700 1 --2- 172.21.15.99:0/3480354813 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f99a80b2880 0x7f99a80b4d40 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f99b4005fd0 tx=0x7f99b400a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:19:53.546 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:19:53.546+0000 7f99be000700 1 -- 172.21.15.99:0/3480354813 <== mon.2 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f99b802b020 con 0x7f99c0079a60 2024-05-03T18:19:53.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:19:53.788+0000 7f99c8000700 1 -- 172.21.15.99:0/3480354813 --> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f99c00638e0 con 0x7f99a80b2880 2024-05-03T18:19:54.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:54 smithi099 ceph-mon[31658]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-03T18:19:54.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:54 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T18:19:54.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:54 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T18:19:54.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:54 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:19:54.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:54 smithi150 ceph-mon[32492]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-03T18:19:54.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:54 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T18:19:54.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:54 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T18:19:54.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:54 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:19:54.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:54 smithi033 ceph-mon[28330]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-03T18:19:54.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:54 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T18:19:54.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:54 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T18:19:54.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:54 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:19:55.794 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:55 smithi099 ceph-mon[31658]: from='client.34340 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:19:55.881 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:55 smithi150 ceph-mon[32492]: from='client.34340 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:19:55.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:55 smithi033 ceph-mon[28330]: from='client.34340 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:19:56.782 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:56 smithi099 ceph-mon[31658]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-03T18:19:56.881 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:56 smithi150 ceph-mon[32492]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-03T18:19:56.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:56 smithi033 ceph-mon[28330]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-03T18:19:58.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:58 smithi150 ceph-mon[32492]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-03T18:19:58.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:58 smithi150 ceph-mon[32492]: from='client.? 172.21.15.99:0/386322810' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8b29bb96-76e8-45a0-ab19-8d082d755fe9"}]: dispatch 2024-05-03T18:19:58.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:58 smithi150 ceph-mon[32492]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8b29bb96-76e8-45a0-ab19-8d082d755fe9"}]: dispatch 2024-05-03T18:19:58.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:58 smithi150 ceph-mon[32492]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8b29bb96-76e8-45a0-ab19-8d082d755fe9"}]': finished 2024-05-03T18:19:58.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:58 smithi150 ceph-mon[32492]: osdmap e61: 11 total, 10 up, 11 in 2024-05-03T18:19:58.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:58 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-03T18:19:58.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:58 smithi033 ceph-mon[28330]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-03T18:19:58.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:58 smithi033 ceph-mon[28330]: from='client.? 172.21.15.99:0/386322810' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8b29bb96-76e8-45a0-ab19-8d082d755fe9"}]: dispatch 2024-05-03T18:19:58.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:58 smithi033 ceph-mon[28330]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8b29bb96-76e8-45a0-ab19-8d082d755fe9"}]: dispatch 2024-05-03T18:19:58.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:58 smithi033 ceph-mon[28330]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8b29bb96-76e8-45a0-ab19-8d082d755fe9"}]': finished 2024-05-03T18:19:58.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:58 smithi033 ceph-mon[28330]: osdmap e61: 11 total, 10 up, 11 in 2024-05-03T18:19:58.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:58 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-03T18:19:59.044 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:58 smithi099 ceph-mon[31658]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-03T18:19:59.044 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:58 smithi099 ceph-mon[31658]: from='client.? 172.21.15.99:0/386322810' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8b29bb96-76e8-45a0-ab19-8d082d755fe9"}]: dispatch 2024-05-03T18:19:59.044 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:58 smithi099 ceph-mon[31658]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8b29bb96-76e8-45a0-ab19-8d082d755fe9"}]: dispatch 2024-05-03T18:19:59.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:58 smithi099 ceph-mon[31658]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8b29bb96-76e8-45a0-ab19-8d082d755fe9"}]': finished 2024-05-03T18:19:59.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:58 smithi099 ceph-mon[31658]: osdmap e61: 11 total, 10 up, 11 in 2024-05-03T18:19:59.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:58 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-03T18:19:59.881 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:19:59 smithi150 ceph-mon[32492]: from='client.? 172.21.15.99:0/898457522' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T18:19:59.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:19:59 smithi033 ceph-mon[28330]: from='client.? 172.21.15.99:0/898457522' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T18:20:00.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:19:59 smithi099 ceph-mon[31658]: from='client.? 172.21.15.99:0/898457522' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T18:20:00.881 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:00 smithi150 ceph-mon[32492]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-03T18:20:00.881 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:00 smithi150 ceph-mon[32492]: overall HEALTH_OK 2024-05-03T18:20:00.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:00 smithi033 ceph-mon[28330]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-03T18:20:00.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:00 smithi033 ceph-mon[28330]: overall HEALTH_OK 2024-05-03T18:20:01.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:00 smithi099 ceph-mon[31658]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-03T18:20:01.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:00 smithi099 ceph-mon[31658]: overall HEALTH_OK 2024-05-03T18:20:02.881 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:02 smithi150 ceph-mon[32492]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-03T18:20:02.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:02 smithi033 ceph-mon[28330]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-03T18:20:03.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:02 smithi099 ceph-mon[31658]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-03T18:20:04.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:04 smithi099 ceph-mon[31658]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-03T18:20:04.881 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:04 smithi150 ceph-mon[32492]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-03T18:20:04.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:04 smithi033 ceph-mon[28330]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-03T18:20:06.881 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:06 smithi150 ceph-mon[32492]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-03T18:20:06.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:06 smithi033 ceph-mon[28330]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-03T18:20:07.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:06 smithi099 ceph-mon[31658]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-03T18:20:07.794 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:07 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-03T18:20:07.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:07 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:20:07.881 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:07 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-03T18:20:07.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:07 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:20:07.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:07 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-03T18:20:07.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:07 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:20:08.881 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:08 smithi150 ceph-mon[32492]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-03T18:20:08.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:08 smithi150 ceph-mon[32492]: Deploying daemon osd.10 on smithi099 2024-05-03T18:20:08.903 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:08 smithi099 ceph-mon[31658]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-03T18:20:08.904 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:08 smithi099 ceph-mon[31658]: Deploying daemon osd.10 on smithi099 2024-05-03T18:20:08.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:08 smithi033 ceph-mon[28330]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-03T18:20:08.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:08 smithi033 ceph-mon[28330]: Deploying daemon osd.10 on smithi099 2024-05-03T18:20:10.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:10 smithi099 ceph-mon[31658]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-03T18:20:10.881 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:10 smithi150 ceph-mon[32492]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-03T18:20:10.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:10 smithi033 ceph-mon[28330]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-03T18:20:12.765 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:12 smithi099 ceph-mon[31658]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-03T18:20:12.881 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:12 smithi150 ceph-mon[32492]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-03T18:20:12.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:12 smithi033 ceph-mon[28330]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-03T18:20:14.780 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:14 smithi099 ceph-mon[31658]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-03T18:20:14.881 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:14 smithi150 ceph-mon[32492]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-03T18:20:14.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:14 smithi033 ceph-mon[28330]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-03T18:20:16.794 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:16 smithi099 ceph-mon[31658]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-03T18:20:16.794 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:16 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:20:16.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:16 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:20:16.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:16 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:20:16.881 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:16 smithi150 ceph-mon[32492]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-03T18:20:16.881 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:16 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:20:16.881 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:16 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:20:16.881 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:16 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:20:16.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:16 smithi033 ceph-mon[28330]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-03T18:20:16.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:16 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:20:16.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:16 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:20:16.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:16 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:20:18.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:18 smithi150 ceph-mon[32492]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-03T18:20:18.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:18 smithi033 ceph-mon[28330]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-03T18:20:19.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:18 smithi099 ceph-mon[31658]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-03T18:20:20.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:20 smithi033 ceph-mon[28330]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-03T18:20:20.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:20 smithi033 ceph-mon[28330]: from='osd.10 [v2:172.21.15.99:6824/935746136,v1:172.21.15.99:6825/935746136]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-03T18:20:20.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:20 smithi033 ceph-mon[28330]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-03T18:20:20.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:20 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:20:20.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:20 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:20:21.048 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:20 smithi099 ceph-mon[31658]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-03T18:20:21.048 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:20 smithi099 ceph-mon[31658]: from='osd.10 [v2:172.21.15.99:6824/935746136,v1:172.21.15.99:6825/935746136]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-03T18:20:21.048 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:20 smithi099 ceph-mon[31658]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-03T18:20:21.048 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:20 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:20:21.049 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:20 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:20:21.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:20 smithi150 ceph-mon[32492]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-03T18:20:21.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:20 smithi150 ceph-mon[32492]: from='osd.10 [v2:172.21.15.99:6824/935746136,v1:172.21.15.99:6825/935746136]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-03T18:20:21.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:20 smithi150 ceph-mon[32492]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-03T18:20:21.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:20 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:20:21.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:20 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:20:21.512 INFO:teuthology.orchestra.run.smithi099.stdout:Created osd(s) 10 on host 'smithi099' 2024-05-03T18:20:21.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:20:21.511+0000 7f99be000700 1 -- 172.21.15.99:0/3480354813 <== mgr.14152 v2:172.21.15.33:6800/3642895501 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f99c00638e0 con 0x7f99a80b2880 2024-05-03T18:20:21.516 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:20:21.516+0000 7f99c8000700 1 -- 172.21.15.99:0/3480354813 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f99a80b2880 msgr2=0x7f99a80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:20:21.516 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:20:21.516+0000 7f99c8000700 1 --2- 172.21.15.99:0/3480354813 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f99a80b2880 0x7f99a80b4d40 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f99b4005fd0 tx=0x7f99b400a560 comp rx=0 tx=0).stop 2024-05-03T18:20:21.516 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:20:21.516+0000 7f99c8000700 1 -- 172.21.15.99:0/3480354813 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f99c0079a60 msgr2=0x7f99c01a7030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:20:21.516 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:20:21.516+0000 7f99c8000700 1 --2- 172.21.15.99:0/3480354813 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f99c0079a60 0x7f99c01a7030 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f99b8007ed0 tx=0x7f99b800be20 comp rx=0 tx=0).stop 2024-05-03T18:20:21.516 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:20:21.516+0000 7f99c8000700 1 -- 172.21.15.99:0/3480354813 shutdown_connections 2024-05-03T18:20:21.517 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:20:21.516+0000 7f99c8000700 1 --2- 172.21.15.99:0/3480354813 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f99a80b2880 0x7f99a80b4d40 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:20:21.517 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:20:21.516+0000 7f99c8000700 1 --2- 172.21.15.99:0/3480354813 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f99c0079a60 0x7f99c01a7030 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:20:21.517 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:20:21.516+0000 7f99c8000700 1 --2- 172.21.15.99:0/3480354813 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99c0077e50 0x7f99c019fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:20:21.517 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:20:21.516+0000 7f99c8000700 1 --2- 172.21.15.99:0/3480354813 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f99c00790a0 0x7f99c019ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:20:21.517 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:20:21.516+0000 7f99c8000700 1 -- 172.21.15.99:0/3480354813 >> 172.21.15.99:0/3480354813 conn(0x7f99c0100cf0 msgr2=0x7f99c01027f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:20:21.517 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:20:21.516+0000 7f99c8000700 1 -- 172.21.15.99:0/3480354813 shutdown_connections 2024-05-03T18:20:21.517 INFO:teuthology.orchestra.run.smithi099.stderr:2024-05-03T18:20:21.517+0000 7f99c8000700 1 -- 172.21.15.99:0/3480354813 wait complete. 2024-05-03T18:20:21.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:21 smithi033 ceph-mon[28330]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-03T18:20:21.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:21 smithi033 ceph-mon[28330]: osdmap e62: 11 total, 10 up, 11 in 2024-05-03T18:20:21.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:21 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-03T18:20:21.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:21 smithi033 ceph-mon[28330]: from='osd.10 [v2:172.21.15.99:6824/935746136,v1:172.21.15.99:6825/935746136]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-05-03T18:20:21.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:21 smithi033 ceph-mon[28330]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-05-03T18:20:21.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:21 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:20:21.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:21 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:20:21.953 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:21 smithi099 ceph-mon[31658]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-03T18:20:21.953 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:21 smithi099 ceph-mon[31658]: osdmap e62: 11 total, 10 up, 11 in 2024-05-03T18:20:21.953 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:21 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-03T18:20:21.953 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:21 smithi099 ceph-mon[31658]: from='osd.10 [v2:172.21.15.99:6824/935746136,v1:172.21.15.99:6825/935746136]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-05-03T18:20:21.953 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:21 smithi099 ceph-mon[31658]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-05-03T18:20:21.953 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:21 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:20:21.953 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:21 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:20:22.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:21 smithi150 ceph-mon[32492]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-03T18:20:22.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:21 smithi150 ceph-mon[32492]: osdmap e62: 11 total, 10 up, 11 in 2024-05-03T18:20:22.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:21 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-03T18:20:22.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:21 smithi150 ceph-mon[32492]: from='osd.10 [v2:172.21.15.99:6824/935746136,v1:172.21.15.99:6825/935746136]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-05-03T18:20:22.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:21 smithi150 ceph-mon[32492]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-05-03T18:20:22.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:21 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:20:22.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:21 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:20:22.384 DEBUG:teuthology.orchestra.run.smithi099:osd.10> sudo journalctl -f -n 0 -u ceph-5703bba4-0978-11ef-bc97-c7b262605968@osd.10.service 2024-05-03T18:20:22.389 INFO:tasks.cephadm:Deploying osd.11 on smithi150 with /dev/vg_nvme/lv_1... 2024-05-03T18:20:22.389 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-03T18:20:22.839 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.c/config 2024-05-03T18:20:22.880 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:22 smithi099 ceph-mon[31658]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-03T18:20:22.880 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:22 smithi099 ceph-mon[31658]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-05-03T18:20:22.880 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:22 smithi099 ceph-mon[31658]: osdmap e63: 11 total, 10 up, 11 in 2024-05-03T18:20:22.880 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:22 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-03T18:20:22.880 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:22 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-03T18:20:22.881 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:22 smithi150 ceph-mon[32492]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-03T18:20:22.881 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:22 smithi150 ceph-mon[32492]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-05-03T18:20:22.881 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:22 smithi150 ceph-mon[32492]: osdmap e63: 11 total, 10 up, 11 in 2024-05-03T18:20:22.881 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:22 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-03T18:20:22.881 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:22 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-03T18:20:22.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:22 smithi033 ceph-mon[28330]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-03T18:20:22.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:22 smithi033 ceph-mon[28330]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-05-03T18:20:22.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:22 smithi033 ceph-mon[28330]: osdmap e63: 11 total, 10 up, 11 in 2024-05-03T18:20:22.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:22 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-03T18:20:22.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:22 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-03T18:20:23.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:23 smithi099 ceph-mon[31658]: purged_snaps scrub starts 2024-05-03T18:20:23.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:23 smithi099 ceph-mon[31658]: purged_snaps scrub ok 2024-05-03T18:20:23.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:23 smithi099 ceph-mon[31658]: osd.10 [v2:172.21.15.99:6824/935746136,v1:172.21.15.99:6825/935746136] boot 2024-05-03T18:20:23.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:23 smithi099 ceph-mon[31658]: osdmap e64: 11 total, 11 up, 11 in 2024-05-03T18:20:23.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:23 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-03T18:20:23.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:23 smithi033 ceph-mon[28330]: purged_snaps scrub starts 2024-05-03T18:20:23.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:23 smithi033 ceph-mon[28330]: purged_snaps scrub ok 2024-05-03T18:20:23.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:23 smithi033 ceph-mon[28330]: osd.10 [v2:172.21.15.99:6824/935746136,v1:172.21.15.99:6825/935746136] boot 2024-05-03T18:20:23.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:23 smithi033 ceph-mon[28330]: osdmap e64: 11 total, 11 up, 11 in 2024-05-03T18:20:23.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:23 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-03T18:20:24.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:23 smithi150 ceph-mon[32492]: purged_snaps scrub starts 2024-05-03T18:20:24.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:23 smithi150 ceph-mon[32492]: purged_snaps scrub ok 2024-05-03T18:20:24.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:23 smithi150 ceph-mon[32492]: osd.10 [v2:172.21.15.99:6824/935746136,v1:172.21.15.99:6825/935746136] boot 2024-05-03T18:20:24.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:23 smithi150 ceph-mon[32492]: osdmap e64: 11 total, 11 up, 11 in 2024-05-03T18:20:24.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:23 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-03T18:20:24.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:24 smithi033 ceph-mon[28330]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-05-03T18:20:24.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:24 smithi033 ceph-mon[28330]: osdmap e65: 11 total, 11 up, 11 in 2024-05-03T18:20:25.035 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:24 smithi150 ceph-mon[32492]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-05-03T18:20:25.035 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:24 smithi150 ceph-mon[32492]: osdmap e65: 11 total, 11 up, 11 in 2024-05-03T18:20:25.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:24 smithi099 ceph-mon[31658]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-05-03T18:20:25.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:24 smithi099 ceph-mon[31658]: osdmap e65: 11 total, 11 up, 11 in 2024-05-03T18:20:25.918 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-03T18:20:25.919 INFO:teuthology.orchestra.run.smithi150.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-03T18:20:25.919 INFO:teuthology.orchestra.run.smithi150.stderr: stderr: 10+0 records in 2024-05-03T18:20:25.919 INFO:teuthology.orchestra.run.smithi150.stderr:10+0 records out 2024-05-03T18:20:25.919 INFO:teuthology.orchestra.run.smithi150.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.033052 s, 317 MB/s 2024-05-03T18:20:25.919 INFO:teuthology.orchestra.run.smithi150.stderr: stderr: 2024-05-03T18:20:25.919 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping successful for: 2024-05-03T18:20:25.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:25 smithi033 ceph-mon[28330]: osdmap e66: 11 total, 11 up, 11 in 2024-05-03T18:20:26.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:25 smithi099 ceph-mon[31658]: osdmap e66: 11 total, 11 up, 11 in 2024-05-03T18:20:26.131 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:25 smithi150 ceph-mon[32492]: osdmap e66: 11 total, 11 up, 11 in 2024-05-03T18:20:26.737 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi150:vg_nvme/lv_1 2024-05-03T18:20:26.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:26 smithi150 ceph-mon[32492]: pgmap v299: 1 pgs: 1 activating+degraded; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2024-05-03T18:20:26.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:26 smithi150 ceph-mon[32492]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-05-03T18:20:26.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:26 smithi033 ceph-mon[28330]: pgmap v299: 1 pgs: 1 activating+degraded; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2024-05-03T18:20:26.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:26 smithi033 ceph-mon[28330]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-05-03T18:20:27.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:26 smithi099 ceph-mon[31658]: pgmap v299: 1 pgs: 1 activating+degraded; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2024-05-03T18:20:27.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:26 smithi099 ceph-mon[31658]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-05-03T18:20:27.143 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.c/config 2024-05-03T18:20:28.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:28 smithi150 ceph-mon[32492]: Detected new or changed devices on smithi099 2024-05-03T18:20:28.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:28 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:20:28.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:28 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:20:28.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:28 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:20:28.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:28 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:20:28.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:28 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:20:28.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:28 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:20:28.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:28 smithi150 ceph-mon[32492]: Adjusting osd_memory_target on smithi099 to 4272M 2024-05-03T18:20:28.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:28 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:20:28.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:28 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:20:28.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:28 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:20:28.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:28 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:20:28.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:28 smithi150 ceph-mon[32492]: pgmap v300: 1 pgs: 1 activating+degraded; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2024-05-03T18:20:28.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:28 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:20:28.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:28 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:20:28.384 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:28 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:20:28.384 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:28 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:20:28.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:28 smithi033 ceph-mon[28330]: Detected new or changed devices on smithi099 2024-05-03T18:20:28.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:28 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:20:28.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:28 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:20:28.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:28 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:20:28.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:28 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:20:28.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:28 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:20:28.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:28 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:20:28.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:28 smithi033 ceph-mon[28330]: Adjusting osd_memory_target on smithi099 to 4272M 2024-05-03T18:20:28.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:28 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:20:28.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:28 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:20:28.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:28 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:20:28.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:28 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:20:28.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:28 smithi033 ceph-mon[28330]: pgmap v300: 1 pgs: 1 activating+degraded; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2024-05-03T18:20:28.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:28 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:20:28.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:28 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:20:28.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:28 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:20:28.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:28 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:20:28.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:28 smithi099 ceph-mon[31658]: Detected new or changed devices on smithi099 2024-05-03T18:20:28.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:28 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:20:28.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:28 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:20:28.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:28 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:20:28.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:28 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:20:28.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:28 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:20:28.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:28 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:20:28.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:28 smithi099 ceph-mon[31658]: Adjusting osd_memory_target on smithi099 to 4272M 2024-05-03T18:20:28.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:28 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:20:28.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:28 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:20:28.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:28 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:20:28.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:28 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:20:28.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:28 smithi099 ceph-mon[31658]: pgmap v300: 1 pgs: 1 activating+degraded; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2024-05-03T18:20:28.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:28 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:20:28.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:28 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:20:28.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:28 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:20:28.547 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:28 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:20:29.500 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:20:29.499+0000 7fd644600700 1 -- 172.21.15.150:0/628322557 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd63c103600 msgr2=0x7fd63c103a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:20:29.500 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:20:29.499+0000 7fd644600700 1 --2- 172.21.15.150:0/628322557 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd63c103600 0x7fd63c103a60 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd628009990 tx=0x7fd628009ca0 comp rx=0 tx=0).stop 2024-05-03T18:20:29.500 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:20:29.499+0000 7fd644600700 1 -- 172.21.15.150:0/628322557 shutdown_connections 2024-05-03T18:20:29.500 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:20:29.499+0000 7fd644600700 1 --2- 172.21.15.150:0/628322557 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd63c103fa0 0x7fd63c10a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:20:29.500 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:20:29.499+0000 7fd644600700 1 --2- 172.21.15.150:0/628322557 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd63c102400 0x7fd63c102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:20:29.500 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:20:29.499+0000 7fd644600700 1 --2- 172.21.15.150:0/628322557 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd63c103600 0x7fd63c103a60 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:20:29.501 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:20:29.499+0000 7fd644600700 1 -- 172.21.15.150:0/628322557 >> 172.21.15.150:0/628322557 conn(0x7fd63c0fd9a0 msgr2=0x7fd63c0ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:20:29.501 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:20:29.499+0000 7fd644600700 1 -- 172.21.15.150:0/628322557 shutdown_connections 2024-05-03T18:20:29.501 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:20:29.499+0000 7fd644600700 1 -- 172.21.15.150:0/628322557 wait complete. 2024-05-03T18:20:29.503 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:20:29.500+0000 7fd644600700 1 Processor -- start 2024-05-03T18:20:29.503 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:20:29.500+0000 7fd644600700 1 -- start start 2024-05-03T18:20:29.503 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:20:29.501+0000 7fd644600700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd63c102400 0x7fd63c19b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:20:29.503 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:20:29.501+0000 7fd644600700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd63c103600 0x7fd63c19bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:20:29.503 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:20:29.501+0000 7fd644600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd63c103fa0 0x7fd63c1a2c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:20:29.503 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:20:29.501+0000 7fd644600700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd63c10ee70 con 0x7fd63c102400 2024-05-03T18:20:29.503 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:20:29.501+0000 7fd644600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd63c10ecf0 con 0x7fd63c103fa0 2024-05-03T18:20:29.504 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:20:29.501+0000 7fd644600700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd63c10eff0 con 0x7fd63c103600 2024-05-03T18:20:29.504 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:20:29.501+0000 7fd641200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd63c103600 0x7fd63c19bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:20:29.504 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:20:29.501+0000 7fd641200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd63c103600 0x7fd63c19bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59806/0 (socket says 172.21.15.150:59806) 2024-05-03T18:20:29.504 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:20:29.501+0000 7fd641200700 1 -- 172.21.15.150:0/2891297942 learned_addr learned my addr 172.21.15.150:0/2891297942 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-03T18:20:29.504 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:20:29.501+0000 7fd641c00700 1 --2- 172.21.15.150:0/2891297942 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd63c102400 0x7fd63c19b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:20:29.504 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:20:29.502+0000 7fd642600700 1 --2- 172.21.15.150:0/2891297942 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd63c103fa0 0x7fd63c1a2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:20:29.504 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:20:29.502+0000 7fd641200700 1 -- 172.21.15.150:0/2891297942 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd63c103fa0 msgr2=0x7fd63c1a2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:20:29.505 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:20:29.502+0000 7fd641200700 1 --2- 172.21.15.150:0/2891297942 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd63c103fa0 0x7fd63c1a2c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:20:29.505 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:20:29.502+0000 7fd641200700 1 -- 172.21.15.150:0/2891297942 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd63c102400 msgr2=0x7fd63c19b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:20:29.505 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:20:29.502+0000 7fd641200700 1 --2- 172.21.15.150:0/2891297942 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd63c102400 0x7fd63c19b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:20:29.505 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:20:29.502+0000 7fd641200700 1 -- 172.21.15.150:0/2891297942 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd63c1a32e0 con 0x7fd63c103600 2024-05-03T18:20:29.505 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:20:29.502+0000 7fd642600700 1 --2- 172.21.15.150:0/2891297942 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd63c103fa0 0x7fd63c1a2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:20:29.505 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:20:29.502+0000 7fd641200700 1 --2- 172.21.15.150:0/2891297942 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd63c103600 0x7fd63c19bbc0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd6280098e0 tx=0x7fd628009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:20:29.505 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:20:29.502+0000 7fd63a000700 1 -- 172.21.15.150:0/2891297942 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd628027070 con 0x7fd63c103600 2024-05-03T18:20:29.505 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:20:29.502+0000 7fd63a000700 1 -- 172.21.15.150:0/2891297942 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd6280041d0 con 0x7fd63c103600 2024-05-03T18:20:29.505 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:20:29.502+0000 7fd644600700 1 -- 172.21.15.150:0/2891297942 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd63c1a3570 con 0x7fd63c103600 2024-05-03T18:20:29.505 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:20:29.502+0000 7fd63a000700 1 -- 172.21.15.150:0/2891297942 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd62800f460 con 0x7fd63c103600 2024-05-03T18:20:29.505 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:20:29.503+0000 7fd644600700 1 -- 172.21.15.150:0/2891297942 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd63c1a3a20 con 0x7fd63c103600 2024-05-03T18:20:29.506 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:20:29.504+0000 7fd644600700 1 -- 172.21.15.150:0/2891297942 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd63c06a190 con 0x7fd63c103600 2024-05-03T18:20:29.506 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:20:29.506+0000 7fd63a000700 1 -- 172.21.15.150:0/2891297942 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fd62802a070 con 0x7fd63c103600 2024-05-03T18:20:29.507 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:20:29.507+0000 7fd63a000700 1 --2- 172.21.15.150:0/2891297942 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fd6240b2900 0x7fd6240b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:20:29.507 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:20:29.508+0000 7fd63a000700 1 -- 172.21.15.150:0/2891297942 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6694+0+0 (secure 0 0 0) 0x7fd6280e34e0 con 0x7fd63c103600 2024-05-03T18:20:29.508 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:20:29.508+0000 7fd641c00700 1 --2- 172.21.15.150:0/2891297942 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fd6240b2900 0x7fd6240b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:20:29.509 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:20:29.509+0000 7fd641c00700 1 --2- 172.21.15.150:0/2891297942 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fd6240b2900 0x7fd6240b4dc0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd630005fd0 tx=0x7fd630005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:20:29.510 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:20:29.510+0000 7fd63a000700 1 -- 172.21.15.150:0/2891297942 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fd628031070 con 0x7fd63c103600 2024-05-03T18:20:29.754 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:20:29.754+0000 7fd644600700 1 -- 172.21.15.150:0/2891297942 --> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fd63c10dff0 con 0x7fd6240b2900 2024-05-03T18:20:30.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:30 smithi150 ceph-mon[32492]: pgmap v301: 1 pgs: 1 activating+degraded; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2024-05-03T18:20:30.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:30 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T18:20:30.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:30 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T18:20:30.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:30 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:20:30.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:30 smithi033 ceph-mon[28330]: pgmap v301: 1 pgs: 1 activating+degraded; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2024-05-03T18:20:30.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:30 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T18:20:30.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:30 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T18:20:30.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:30 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:20:30.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:30 smithi099 ceph-mon[31658]: pgmap v301: 1 pgs: 1 activating+degraded; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2024-05-03T18:20:30.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:30 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T18:20:30.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:30 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T18:20:30.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:30 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:20:31.616 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:31 smithi033 ceph-mon[28330]: from='client.24367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:20:31.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:31 smithi150 ceph-mon[32492]: from='client.24367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:20:31.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:31 smithi099 ceph-mon[31658]: from='client.24367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:20:32.457 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:32 smithi150 ceph-mon[32492]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-03T18:20:32.457 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:32 smithi150 ceph-mon[32492]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-05-03T18:20:32.457 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:32 smithi150 ceph-mon[32492]: Cluster is now healthy 2024-05-03T18:20:32.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:32 smithi033 ceph-mon[28330]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-03T18:20:32.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:32 smithi033 ceph-mon[28330]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-05-03T18:20:32.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:32 smithi033 ceph-mon[28330]: Cluster is now healthy 2024-05-03T18:20:32.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:32 smithi099 ceph-mon[31658]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-03T18:20:32.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:32 smithi099 ceph-mon[31658]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-05-03T18:20:32.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:32 smithi099 ceph-mon[31658]: Cluster is now healthy 2024-05-03T18:20:34.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:34 smithi150 ceph-mon[32492]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-05-03T18:20:34.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:34 smithi150 ceph-mon[32492]: from='client.? 172.21.15.150:0/2760566937' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "07a6a825-c902-4bfd-97cc-cc9074532af5"}]: dispatch 2024-05-03T18:20:34.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:34 smithi150 ceph-mon[32492]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "07a6a825-c902-4bfd-97cc-cc9074532af5"}]: dispatch 2024-05-03T18:20:34.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:34 smithi150 ceph-mon[32492]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "07a6a825-c902-4bfd-97cc-cc9074532af5"}]': finished 2024-05-03T18:20:34.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:34 smithi150 ceph-mon[32492]: osdmap e67: 12 total, 11 up, 12 in 2024-05-03T18:20:34.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:34 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-03T18:20:34.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:34 smithi033 ceph-mon[28330]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-05-03T18:20:34.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:34 smithi033 ceph-mon[28330]: from='client.? 172.21.15.150:0/2760566937' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "07a6a825-c902-4bfd-97cc-cc9074532af5"}]: dispatch 2024-05-03T18:20:34.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:34 smithi033 ceph-mon[28330]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "07a6a825-c902-4bfd-97cc-cc9074532af5"}]: dispatch 2024-05-03T18:20:34.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:34 smithi033 ceph-mon[28330]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "07a6a825-c902-4bfd-97cc-cc9074532af5"}]': finished 2024-05-03T18:20:34.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:34 smithi033 ceph-mon[28330]: osdmap e67: 12 total, 11 up, 12 in 2024-05-03T18:20:34.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:34 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-03T18:20:34.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:34 smithi099 ceph-mon[31658]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-05-03T18:20:34.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:34 smithi099 ceph-mon[31658]: from='client.? 172.21.15.150:0/2760566937' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "07a6a825-c902-4bfd-97cc-cc9074532af5"}]: dispatch 2024-05-03T18:20:34.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:34 smithi099 ceph-mon[31658]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "07a6a825-c902-4bfd-97cc-cc9074532af5"}]: dispatch 2024-05-03T18:20:34.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:34 smithi099 ceph-mon[31658]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "07a6a825-c902-4bfd-97cc-cc9074532af5"}]': finished 2024-05-03T18:20:34.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:34 smithi099 ceph-mon[31658]: osdmap e67: 12 total, 11 up, 12 in 2024-05-03T18:20:34.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:34 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-03T18:20:35.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:35 smithi150 ceph-mon[32492]: from='client.? 172.21.15.150:0/1125065843' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T18:20:35.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:35 smithi033 ceph-mon[28330]: from='client.? 172.21.15.150:0/1125065843' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T18:20:35.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:35 smithi099 ceph-mon[31658]: from='client.? 172.21.15.150:0/1125065843' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T18:20:36.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:36 smithi150 ceph-mon[32492]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-03T18:20:36.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:36 smithi033 ceph-mon[28330]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-03T18:20:36.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:36 smithi099 ceph-mon[31658]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-03T18:20:38.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:38 smithi150 ceph-mon[32492]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-03T18:20:38.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:38 smithi033 ceph-mon[28330]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-03T18:20:38.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:38 smithi099 ceph-mon[31658]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-03T18:20:40.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:40 smithi150 ceph-mon[32492]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-03T18:20:40.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:40 smithi033 ceph-mon[28330]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-03T18:20:40.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:40 smithi099 ceph-mon[31658]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-03T18:20:42.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:42 smithi150 ceph-mon[32492]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-03T18:20:42.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:42 smithi033 ceph-mon[28330]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-03T18:20:42.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:42 smithi099 ceph-mon[31658]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-03T18:20:44.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:44 smithi099 ceph-mon[31658]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-03T18:20:44.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:44 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-03T18:20:44.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:44 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:20:44.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:44 smithi099 ceph-mon[31658]: Deploying daemon osd.11 on smithi150 2024-05-03T18:20:44.654 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:44 smithi150 ceph-mon[32492]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-03T18:20:44.654 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:44 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-03T18:20:44.654 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:44 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:20:44.654 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:44 smithi150 ceph-mon[32492]: Deploying daemon osd.11 on smithi150 2024-05-03T18:20:44.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:44 smithi033 ceph-mon[28330]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-03T18:20:44.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:44 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-03T18:20:44.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:44 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:20:44.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:44 smithi033 ceph-mon[28330]: Deploying daemon osd.11 on smithi150 2024-05-03T18:20:46.520 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:46 smithi150 ceph-mon[32492]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-03T18:20:46.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:46 smithi033 ceph-mon[28330]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-03T18:20:46.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:46 smithi099 ceph-mon[31658]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-03T18:20:48.611 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:48 smithi150 ceph-mon[32492]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-03T18:20:48.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:48 smithi033 ceph-mon[28330]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-03T18:20:48.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:48 smithi099 ceph-mon[31658]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-03T18:20:50.717 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:50 smithi150 ceph-mon[32492]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-03T18:20:50.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:50 smithi033 ceph-mon[28330]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-03T18:20:50.794 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:50 smithi099 ceph-mon[31658]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-03T18:20:52.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:52 smithi150 ceph-mon[32492]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-03T18:20:52.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:52 smithi033 ceph-mon[28330]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-03T18:20:52.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:52 smithi099 ceph-mon[31658]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-03T18:20:53.717 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:53 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:20:53.717 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:53 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:20:53.717 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:53 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:20:53.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:53 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:20:53.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:53 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:20:53.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:53 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:20:53.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:53 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:20:53.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:53 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:20:53.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:53 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:20:54.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:54 smithi099 ceph-mon[31658]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-03T18:20:54.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:54 smithi150 ceph-mon[32492]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-03T18:20:54.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:54 smithi033 ceph-mon[28330]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-03T18:20:56.688 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:56 smithi150 ceph-mon[32492]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-03T18:20:56.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:56 smithi099 ceph-mon[31658]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-03T18:20:56.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:56 smithi033 ceph-mon[28330]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-03T18:20:57.578 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:20:57.577+0000 7fd63a000700 1 -- 172.21.15.150:0/2891297942 <== mgr.14152 v2:172.21.15.33:6800/3642895501 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fd63c10dff0 con 0x7fd6240b2900 2024-05-03T18:20:57.578 INFO:teuthology.orchestra.run.smithi150.stdout:Created osd(s) 11 on host 'smithi150' 2024-05-03T18:20:57.582 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:20:57.581+0000 7fd644600700 1 -- 172.21.15.150:0/2891297942 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fd6240b2900 msgr2=0x7fd6240b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:20:57.582 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:20:57.581+0000 7fd644600700 1 --2- 172.21.15.150:0/2891297942 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fd6240b2900 0x7fd6240b4dc0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd630005fd0 tx=0x7fd630005e50 comp rx=0 tx=0).stop 2024-05-03T18:20:57.582 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:20:57.581+0000 7fd644600700 1 -- 172.21.15.150:0/2891297942 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd63c103600 msgr2=0x7fd63c19bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:20:57.582 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:20:57.581+0000 7fd644600700 1 --2- 172.21.15.150:0/2891297942 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd63c103600 0x7fd63c19bbc0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd6280098e0 tx=0x7fd628009f90 comp rx=0 tx=0).stop 2024-05-03T18:20:57.582 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:20:57.581+0000 7fd644600700 1 -- 172.21.15.150:0/2891297942 shutdown_connections 2024-05-03T18:20:57.582 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:20:57.581+0000 7fd644600700 1 --2- 172.21.15.150:0/2891297942 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fd6240b2900 0x7fd6240b4dc0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:20:57.582 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:20:57.581+0000 7fd644600700 1 --2- 172.21.15.150:0/2891297942 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd63c103fa0 0x7fd63c1a2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:20:57.582 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:20:57.581+0000 7fd644600700 1 --2- 172.21.15.150:0/2891297942 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd63c102400 0x7fd63c19b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:20:57.582 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:20:57.581+0000 7fd644600700 1 --2- 172.21.15.150:0/2891297942 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd63c103600 0x7fd63c19bbc0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:20:57.583 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:20:57.582+0000 7fd644600700 1 -- 172.21.15.150:0/2891297942 >> 172.21.15.150:0/2891297942 conn(0x7fd63c0fd9a0 msgr2=0x7fd63c10c8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:20:57.583 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:20:57.582+0000 7fd644600700 1 -- 172.21.15.150:0/2891297942 shutdown_connections 2024-05-03T18:20:57.583 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:20:57.582+0000 7fd644600700 1 -- 172.21.15.150:0/2891297942 wait complete. 2024-05-03T18:20:57.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:57 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:20:57.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:57 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:20:57.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:57 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:20:57.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:57 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:20:57.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:57 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:20:57.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:57 smithi150 ceph-mon[32492]: from='osd.11 [v2:172.21.15.150:6824/2069349808,v1:172.21.15.150:6825/2069349808]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-03T18:20:57.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:57 smithi150 ceph-mon[32492]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-03T18:20:57.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:57 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:20:57.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:57 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:20:57.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:57 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:20:57.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:57 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:20:57.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:57 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:20:57.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:57 smithi099 ceph-mon[31658]: from='osd.11 [v2:172.21.15.150:6824/2069349808,v1:172.21.15.150:6825/2069349808]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-03T18:20:57.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:57 smithi099 ceph-mon[31658]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-03T18:20:57.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:57 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:20:57.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:57 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:20:57.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:57 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:20:57.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:57 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:20:57.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:57 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:20:57.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:57 smithi033 ceph-mon[28330]: from='osd.11 [v2:172.21.15.150:6824/2069349808,v1:172.21.15.150:6825/2069349808]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-03T18:20:57.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:57 smithi033 ceph-mon[28330]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-03T18:20:58.397 DEBUG:teuthology.orchestra.run.smithi150:osd.11> sudo journalctl -f -n 0 -u ceph-5703bba4-0978-11ef-bc97-c7b262605968@osd.11.service 2024-05-03T18:20:58.402 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-05-03T18:20:58.402 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph osd stat -f json 2024-05-03T18:20:58.591 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:58 smithi150 ceph-mon[32492]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-03T18:20:58.591 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:58 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:20:58.591 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:58 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:20:58.591 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:58 smithi150 ceph-mon[32492]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-03T18:20:58.592 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:58 smithi150 ceph-mon[32492]: osdmap e68: 12 total, 11 up, 12 in 2024-05-03T18:20:58.592 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:58 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-03T18:20:58.592 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:58 smithi150 ceph-mon[32492]: from='osd.11 [v2:172.21.15.150:6824/2069349808,v1:172.21.15.150:6825/2069349808]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-03T18:20:58.592 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:58 smithi150 ceph-mon[32492]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-03T18:20:58.592 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:58 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:20:58.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:58 smithi033 ceph-mon[28330]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-03T18:20:58.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:58 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:20:58.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:58 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:20:58.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:58 smithi033 ceph-mon[28330]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-03T18:20:58.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:58 smithi033 ceph-mon[28330]: osdmap e68: 12 total, 11 up, 12 in 2024-05-03T18:20:58.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:58 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-03T18:20:58.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:58 smithi033 ceph-mon[28330]: from='osd.11 [v2:172.21.15.150:6824/2069349808,v1:172.21.15.150:6825/2069349808]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-03T18:20:58.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:58 smithi033 ceph-mon[28330]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-03T18:20:58.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:58 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:20:58.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:58 smithi099 ceph-mon[31658]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-03T18:20:58.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:58 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:20:58.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:58 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:20:58.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:58 smithi099 ceph-mon[31658]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-03T18:20:58.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:58 smithi099 ceph-mon[31658]: osdmap e68: 12 total, 11 up, 12 in 2024-05-03T18:20:58.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:58 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-03T18:20:58.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:58 smithi099 ceph-mon[31658]: from='osd.11 [v2:172.21.15.150:6824/2069349808,v1:172.21.15.150:6825/2069349808]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-03T18:20:58.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:58 smithi099 ceph-mon[31658]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-05-03T18:20:58.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:58 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:20:58.877 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:20:59.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:59 smithi033 ceph-mon[28330]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-05-03T18:20:59.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:59 smithi033 ceph-mon[28330]: osdmap e69: 12 total, 11 up, 12 in 2024-05-03T18:20:59.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:59 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-03T18:20:59.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:59 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-03T18:20:59.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:20:59 smithi033 ceph-mon[28330]: from='osd.11 ' entity='osd.11' 2024-05-03T18:20:59.841 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:59 smithi150 ceph-mon[32492]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-05-03T18:20:59.841 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:59 smithi150 ceph-mon[32492]: osdmap e69: 12 total, 11 up, 12 in 2024-05-03T18:20:59.841 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:59 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-03T18:20:59.842 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:59 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-03T18:20:59.842 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:20:59 smithi150 ceph-mon[32492]: from='osd.11 ' entity='osd.11' 2024-05-03T18:20:59.842 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:20:59 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:20:59.385+0000 7f4591000700 -1 osd.11 0 waiting for initial osdmap 2024-05-03T18:20:59.842 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:20:59 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:20:59.413+0000 7f4586400700 -1 osd.11 69 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-03T18:21:00.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:59 smithi099 ceph-mon[31658]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-05-03T18:21:00.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:59 smithi099 ceph-mon[31658]: osdmap e69: 12 total, 11 up, 12 in 2024-05-03T18:21:00.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:59 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-03T18:21:00.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:59 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-03T18:21:00.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:20:59 smithi099 ceph-mon[31658]: from='osd.11 ' entity='osd.11' 2024-05-03T18:21:00.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:00 smithi150 ceph-mon[32492]: purged_snaps scrub starts 2024-05-03T18:21:00.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:00 smithi150 ceph-mon[32492]: purged_snaps scrub ok 2024-05-03T18:21:00.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:00 smithi150 ceph-mon[32492]: pgmap v319: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-03T18:21:00.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:00 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-03T18:21:00.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:00 smithi150 ceph-mon[32492]: osd.11 [v2:172.21.15.150:6824/2069349808,v1:172.21.15.150:6825/2069349808] boot 2024-05-03T18:21:00.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:00 smithi150 ceph-mon[32492]: osdmap e70: 12 total, 12 up, 12 in 2024-05-03T18:21:00.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:00 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-03T18:21:00.918 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:00 smithi033 ceph-mon[28330]: purged_snaps scrub starts 2024-05-03T18:21:00.918 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:00 smithi033 ceph-mon[28330]: purged_snaps scrub ok 2024-05-03T18:21:00.918 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:00 smithi033 ceph-mon[28330]: pgmap v319: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-03T18:21:00.918 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:00 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-03T18:21:00.918 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:00 smithi033 ceph-mon[28330]: osd.11 [v2:172.21.15.150:6824/2069349808,v1:172.21.15.150:6825/2069349808] boot 2024-05-03T18:21:00.918 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:00 smithi033 ceph-mon[28330]: osdmap e70: 12 total, 12 up, 12 in 2024-05-03T18:21:00.918 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:00 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-03T18:21:01.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:00 smithi099 ceph-mon[31658]: purged_snaps scrub starts 2024-05-03T18:21:01.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:00 smithi099 ceph-mon[31658]: purged_snaps scrub ok 2024-05-03T18:21:01.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:00 smithi099 ceph-mon[31658]: pgmap v319: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-03T18:21:01.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:00 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-03T18:21:01.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:00 smithi099 ceph-mon[31658]: osd.11 [v2:172.21.15.150:6824/2069349808,v1:172.21.15.150:6825/2069349808] boot 2024-05-03T18:21:01.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:00 smithi099 ceph-mon[31658]: osdmap e70: 12 total, 12 up, 12 in 2024-05-03T18:21:01.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:00 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-03T18:21:01.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:01.392+0000 7f4eb7e00700 1 -- 172.21.15.33:0/1376605592 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4eb81071e0 msgr2=0x7f4eb810da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:21:01.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:01.392+0000 7f4eb7e00700 1 --2- 172.21.15.33:0/1376605592 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4eb81071e0 0x7f4eb810da90 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4eb000b0d0 tx=0x7f4eb000b3e0 comp rx=0 tx=0).stop 2024-05-03T18:21:01.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:01.392+0000 7f4eb7e00700 1 -- 172.21.15.33:0/1376605592 shutdown_connections 2024-05-03T18:21:01.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:01.392+0000 7f4eb7e00700 1 --2- 172.21.15.33:0/1376605592 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4eb81071e0 0x7f4eb810da90 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:21:01.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:01.392+0000 7f4eb7e00700 1 --2- 172.21.15.33:0/1376605592 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4eb8105640 0x7f4eb8105a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:21:01.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:01.392+0000 7f4eb7e00700 1 --2- 172.21.15.33:0/1376605592 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4eb8106840 0x7f4eb8106ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:21:01.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:01.392+0000 7f4eb7e00700 1 -- 172.21.15.33:0/1376605592 >> 172.21.15.33:0/1376605592 conn(0x7f4eb8100bc0 msgr2=0x7f4eb8103020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:21:01.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:01.393+0000 7f4eb7e00700 1 -- 172.21.15.33:0/1376605592 shutdown_connections 2024-05-03T18:21:01.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:01.393+0000 7f4eb7e00700 1 -- 172.21.15.33:0/1376605592 wait complete. 2024-05-03T18:21:01.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:01.394+0000 7f4eb7e00700 1 Processor -- start 2024-05-03T18:21:01.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:01.394+0000 7f4eb7e00700 1 -- start start 2024-05-03T18:21:01.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:01.394+0000 7f4eb7e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4eb8105640 0x7f4eb819b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:21:01.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:01.394+0000 7f4eb7e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4eb8106840 0x7f4eb819ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:21:01.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:01.394+0000 7f4eb7e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4eb81071e0 0x7f4eb81a2b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:21:01.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:01.394+0000 7f4eb7e00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb8110f60 con 0x7f4eb81071e0 2024-05-03T18:21:01.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:01.394+0000 7f4eb7e00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb8110de0 con 0x7f4eb8106840 2024-05-03T18:21:01.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:01.394+0000 7f4eb7e00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb81110e0 con 0x7f4eb8105640 2024-05-03T18:21:01.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:01.394+0000 7f4eb7400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4eb81071e0 0x7f4eb81a2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:21:01.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:01.395+0000 7f4eb7400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4eb81071e0 0x7f4eb81a2b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34172/0 (socket says 172.21.15.33:34172) 2024-05-03T18:21:01.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:01.395+0000 7f4eb7400700 1 -- 172.21.15.33:0/264224536 learned_addr learned my addr 172.21.15.33:0/264224536 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:21:01.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:01.395+0000 7f4eb6a00700 1 --2- 172.21.15.33:0/264224536 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4eb8105640 0x7f4eb819b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:21:01.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:01.395+0000 7f4eb6000700 1 --2- 172.21.15.33:0/264224536 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4eb8106840 0x7f4eb819ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:21:01.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:01.395+0000 7f4eb7400700 1 -- 172.21.15.33:0/264224536 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4eb8105640 msgr2=0x7f4eb819b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:21:01.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:01.395+0000 7f4eb7400700 1 --2- 172.21.15.33:0/264224536 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4eb8105640 0x7f4eb819b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:21:01.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:01.395+0000 7f4eb7400700 1 -- 172.21.15.33:0/264224536 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4eb8106840 msgr2=0x7f4eb819ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:21:01.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:01.395+0000 7f4eb7400700 1 --2- 172.21.15.33:0/264224536 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4eb8106840 0x7f4eb819ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:21:01.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:01.395+0000 7f4eb7400700 1 -- 172.21.15.33:0/264224536 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4eb81a31b0 con 0x7f4eb81071e0 2024-05-03T18:21:01.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:01.395+0000 7f4eb7400700 1 --2- 172.21.15.33:0/264224536 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4eb81071e0 0x7f4eb81a2b10 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4eb000d000 tx=0x7f4eb0009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:21:01.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:01.396+0000 7f4ebca00700 1 -- 172.21.15.33:0/264224536 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb001f030 con 0x7f4eb81071e0 2024-05-03T18:21:01.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:01.396+0000 7f4ebca00700 1 -- 172.21.15.33:0/264224536 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4eb0007a80 con 0x7f4eb81071e0 2024-05-03T18:21:01.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:01.396+0000 7f4eb7e00700 1 -- 172.21.15.33:0/264224536 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4eb81a3440 con 0x7f4eb81071e0 2024-05-03T18:21:01.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:01.396+0000 7f4ebca00700 1 -- 172.21.15.33:0/264224536 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb001b690 con 0x7f4eb81071e0 2024-05-03T18:21:01.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:01.396+0000 7f4eb7e00700 1 -- 172.21.15.33:0/264224536 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4eb81a3890 con 0x7f4eb81071e0 2024-05-03T18:21:01.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:01.397+0000 7f4eb7e00700 1 -- 172.21.15.33:0/264224536 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4eb806a190 con 0x7f4eb81071e0 2024-05-03T18:21:01.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:01.400+0000 7f4ebca00700 1 -- 172.21.15.33:0/264224536 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f4eb0022030 con 0x7f4eb81071e0 2024-05-03T18:21:01.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:01.401+0000 7f4ebca00700 1 --2- 172.21.15.33:0/264224536 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f4e9c0b2880 0x7f4e9c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:21:01.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:01.402+0000 7f4ebca00700 1 -- 172.21.15.33:0/264224536 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7f4eb000f020 con 0x7f4eb81071e0 2024-05-03T18:21:01.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:01.402+0000 7f4eb6a00700 1 --2- 172.21.15.33:0/264224536 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f4e9c0b2880 0x7f4e9c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:21:01.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:01.402+0000 7f4eb6a00700 1 --2- 172.21.15.33:0/264224536 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f4e9c0b2880 0x7f4e9c0b4d40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4ea8005fd0 tx=0x7f4ea8005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:21:01.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:01.404+0000 7f4ebca00700 1 -- 172.21.15.33:0/264224536 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f4eb0029020 con 0x7f4eb81071e0 2024-05-03T18:21:01.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:01.648+0000 7f4eb7e00700 1 -- 172.21.15.33:0/264224536 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f4eb802e510 con 0x7f4eb81071e0 2024-05-03T18:21:01.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:01.648+0000 7f4ebca00700 1 -- 172.21.15.33:0/264224536 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v71) ==== 74+0+133 (secure 0 0 0) 0x7f4eb00ab450 con 0x7f4eb81071e0 2024-05-03T18:21:01.649 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:21:01.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:01.651+0000 7f4eb7e00700 1 -- 172.21.15.33:0/264224536 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f4e9c0b2880 msgr2=0x7f4e9c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:21:01.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:01.651+0000 7f4eb7e00700 1 --2- 172.21.15.33:0/264224536 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f4e9c0b2880 0x7f4e9c0b4d40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4ea8005fd0 tx=0x7f4ea8005e50 comp rx=0 tx=0).stop 2024-05-03T18:21:01.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:01.651+0000 7f4eb7e00700 1 -- 172.21.15.33:0/264224536 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4eb81071e0 msgr2=0x7f4eb81a2b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:21:01.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:01.651+0000 7f4eb7e00700 1 --2- 172.21.15.33:0/264224536 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4eb81071e0 0x7f4eb81a2b10 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4eb000d000 tx=0x7f4eb0009c30 comp rx=0 tx=0).stop 2024-05-03T18:21:01.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:01.652+0000 7f4eb7e00700 1 -- 172.21.15.33:0/264224536 shutdown_connections 2024-05-03T18:21:01.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:01.652+0000 7f4eb7e00700 1 --2- 172.21.15.33:0/264224536 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4eb81071e0 0x7f4eb81a2b10 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:21:01.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:01.652+0000 7f4eb7e00700 1 --2- 172.21.15.33:0/264224536 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4eb8105640 0x7f4eb819b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:21:01.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:01.652+0000 7f4eb7e00700 1 --2- 172.21.15.33:0/264224536 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f4e9c0b2880 0x7f4e9c0b4d40 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:21:01.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:01.652+0000 7f4eb7e00700 1 --2- 172.21.15.33:0/264224536 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4eb8106840 0x7f4eb819ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:21:01.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:01.652+0000 7f4eb7e00700 1 -- 172.21.15.33:0/264224536 >> 172.21.15.33:0/264224536 conn(0x7f4eb8100bc0 msgr2=0x7f4eb8109ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:21:01.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:01.652+0000 7f4eb7e00700 1 -- 172.21.15.33:0/264224536 shutdown_connections 2024-05-03T18:21:01.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:01.652+0000 7f4eb7e00700 1 -- 172.21.15.33:0/264224536 wait complete. 2024-05-03T18:21:02.597 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":71,"num_osds":12,"num_up_osds":12,"osd_up_since":1714760460,"num_in_osds":12,"osd_in_since":1714760434,"num_remapped_pgs":0} 2024-05-03T18:21:02.598 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-03T18:21:02.672 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:02 smithi033 ceph-mon[28330]: pgmap v321: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:21:02.672 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:02 smithi033 ceph-mon[28330]: osdmap e71: 12 total, 12 up, 12 in 2024-05-03T18:21:02.672 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:02 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/264224536' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-03T18:21:02.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:02 smithi150 ceph-mon[32492]: pgmap v321: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:21:02.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:02 smithi150 ceph-mon[32492]: osdmap e71: 12 total, 12 up, 12 in 2024-05-03T18:21:02.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:02 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/264224536' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-03T18:21:03.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:02 smithi099 ceph-mon[31658]: pgmap v321: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:21:03.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:02 smithi099 ceph-mon[31658]: osdmap e71: 12 total, 12 up, 12 in 2024-05-03T18:21:03.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:02 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/264224536' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-03T18:21:03.053 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:21:04.386 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:04 smithi033 ceph-mon[28330]: Detected new or changed devices on smithi150 2024-05-03T18:21:04.386 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:04 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:04.386 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:04 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:04.387 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:04 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:21:04.387 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:04 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:21:04.387 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:04 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:21:04.387 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:04 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:21:04.387 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:04 smithi033 ceph-mon[28330]: Adjusting osd_memory_target on smithi150 to 4272M 2024-05-03T18:21:04.387 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:04 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:04.387 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:04 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:21:04.387 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:04 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:21:04.387 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:04 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:04.387 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:04 smithi033 ceph-mon[28330]: pgmap v323: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:21:04.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:04 smithi099 ceph-mon[31658]: Detected new or changed devices on smithi150 2024-05-03T18:21:04.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:04 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:04.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:04 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:04.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:04 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:21:04.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:04 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:21:04.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:04 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:21:04.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:04 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:21:04.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:04 smithi099 ceph-mon[31658]: Adjusting osd_memory_target on smithi150 to 4272M 2024-05-03T18:21:04.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:04 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:04.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:04 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:21:04.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:04 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:21:04.547 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:04 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:04.547 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:04 smithi099 ceph-mon[31658]: pgmap v323: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:21:04.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:04 smithi150 ceph-mon[32492]: Detected new or changed devices on smithi150 2024-05-03T18:21:04.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:04 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:04.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:04 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:04.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:04 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:21:04.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:04 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:21:04.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:04 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:21:04.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:04 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:21:04.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:04 smithi150 ceph-mon[32492]: Adjusting osd_memory_target on smithi150 to 4272M 2024-05-03T18:21:04.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:04 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:04.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:04 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:21:04.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:04 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:21:04.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:04 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:04.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:04 smithi150 ceph-mon[32492]: pgmap v323: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:21:05.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:05.224+0000 7ff2f1200700 1 -- 172.21.15.33:0/3179801415 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2ec10b660 msgr2=0x7ff2ec10ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:21:05.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:05.224+0000 7ff2f1200700 1 --2- 172.21.15.33:0/3179801415 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2ec10b660 0x7ff2ec10ba40 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc009990 tx=0x7ff2dc009ca0 comp rx=0 tx=0).stop 2024-05-03T18:21:05.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:05.224+0000 7ff2f1200700 1 -- 172.21.15.33:0/3179801415 shutdown_connections 2024-05-03T18:21:05.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:05.224+0000 7ff2f1200700 1 --2- 172.21.15.33:0/3179801415 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2ec105f90 0x7ff2ec1124a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:21:05.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:05.224+0000 7ff2f1200700 1 --2- 172.21.15.33:0/3179801415 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2ec10b660 0x7ff2ec10ba40 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:21:05.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:05.224+0000 7ff2f1200700 1 --2- 172.21.15.33:0/3179801415 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2ec1055d0 0x7ff2ec105a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:21:05.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:05.224+0000 7ff2f1200700 1 -- 172.21.15.33:0/3179801415 >> 172.21.15.33:0/3179801415 conn(0x7ff2ec1010c0 msgr2=0x7ff2ec1034e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:21:05.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:05.224+0000 7ff2f1200700 1 -- 172.21.15.33:0/3179801415 shutdown_connections 2024-05-03T18:21:05.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:05.224+0000 7ff2f1200700 1 -- 172.21.15.33:0/3179801415 wait complete. 2024-05-03T18:21:05.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:05.226+0000 7ff2f1200700 1 Processor -- start 2024-05-03T18:21:05.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:05.226+0000 7ff2f1200700 1 -- start start 2024-05-03T18:21:05.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:05.226+0000 7ff2f1200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2ec1055d0 0x7ff2ec0779b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:21:05.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:05.227+0000 7ff2f1200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2ec105f90 0x7ff2ec077ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:21:05.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:05.227+0000 7ff2f1200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2ec10b660 0x7ff2ec078430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:21:05.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:05.227+0000 7ff2f1200700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff2ec115660 con 0x7ff2ec1055d0 2024-05-03T18:21:05.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:05.227+0000 7ff2eb400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2ec1055d0 0x7ff2ec0779b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:21:05.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:05.227+0000 7ff2eb400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2ec1055d0 0x7ff2ec0779b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34182/0 (socket says 172.21.15.33:34182) 2024-05-03T18:21:05.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:05.227+0000 7ff2eb400700 1 -- 172.21.15.33:0/399618622 learned_addr learned my addr 172.21.15.33:0/399618622 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:21:05.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:05.227+0000 7ff2f1200700 1 -- 172.21.15.33:0/399618622 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff2ec1154e0 con 0x7ff2ec10b660 2024-05-03T18:21:05.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:05.227+0000 7ff2f1200700 1 -- 172.21.15.33:0/399618622 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff2ec1157e0 con 0x7ff2ec105f90 2024-05-03T18:21:05.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:05.227+0000 7ff2ebe00700 1 --2- 172.21.15.33:0/399618622 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2ec10b660 0x7ff2ec078430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:21:05.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:05.227+0000 7ff2eaa00700 1 --2- 172.21.15.33:0/399618622 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2ec105f90 0x7ff2ec077ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:21:05.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:05.227+0000 7ff2eb400700 1 -- 172.21.15.33:0/399618622 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2ec105f90 msgr2=0x7ff2ec077ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:21:05.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:05.227+0000 7ff2eb400700 1 --2- 172.21.15.33:0/399618622 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2ec105f90 0x7ff2ec077ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:21:05.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:05.227+0000 7ff2eb400700 1 -- 172.21.15.33:0/399618622 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2ec10b660 msgr2=0x7ff2ec078430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:21:05.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:05.227+0000 7ff2eb400700 1 --2- 172.21.15.33:0/399618622 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2ec10b660 0x7ff2ec078430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:21:05.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:05.227+0000 7ff2eb400700 1 -- 172.21.15.33:0/399618622 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2ec1ac1a0 con 0x7ff2ec1055d0 2024-05-03T18:21:05.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:05.228+0000 7ff2eb400700 1 --2- 172.21.15.33:0/399618622 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2ec1055d0 0x7ff2ec0779b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc000c00 tx=0x7ff2dc005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:21:05.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:05.228+0000 7ff2e3e00700 1 -- 172.21.15.33:0/399618622 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2dc027070 con 0x7ff2ec1055d0 2024-05-03T18:21:05.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:05.228+0000 7ff2e3e00700 1 -- 172.21.15.33:0/399618622 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff2dc00fd40 con 0x7ff2ec1055d0 2024-05-03T18:21:05.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:05.228+0000 7ff2e3e00700 1 -- 172.21.15.33:0/399618622 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2dc022cd0 con 0x7ff2ec1055d0 2024-05-03T18:21:05.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:05.228+0000 7ff2f1200700 1 -- 172.21.15.33:0/399618622 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2ec1ac430 con 0x7ff2ec1055d0 2024-05-03T18:21:05.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:05.228+0000 7ff2f1200700 1 -- 172.21.15.33:0/399618622 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2ec1ac960 con 0x7ff2ec1055d0 2024-05-03T18:21:05.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:05.230+0000 7ff2f1200700 1 -- 172.21.15.33:0/399618622 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2ec10db60 con 0x7ff2ec1055d0 2024-05-03T18:21:05.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:05.232+0000 7ff2e3e00700 1 -- 172.21.15.33:0/399618622 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7ff2dc02a070 con 0x7ff2ec1055d0 2024-05-03T18:21:05.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:05.234+0000 7ff2e3e00700 1 --2- 172.21.15.33:0/399618622 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7ff2d00b2900 0x7ff2d00b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:21:05.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:05.234+0000 7ff2e3e00700 1 -- 172.21.15.33:0/399618622 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7ff2dc0e2dd0 con 0x7ff2ec1055d0 2024-05-03T18:21:05.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:05.234+0000 7ff2eaa00700 1 --2- 172.21.15.33:0/399618622 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7ff2d00b2900 0x7ff2d00b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:21:05.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:05.235+0000 7ff2eaa00700 1 --2- 172.21.15.33:0/399618622 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7ff2d00b2900 0x7ff2d00b4dc0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff2d8005fd0 tx=0x7ff2d800a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:21:05.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:05.236+0000 7ff2e3e00700 1 -- 172.21.15.33:0/399618622 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7ff2dc031080 con 0x7ff2ec1055d0 2024-05-03T18:21:05.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:05.472+0000 7ff2f1200700 1 -- 172.21.15.33:0/399618622 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff2ec078ee0 con 0x7ff2ec1055d0 2024-05-03T18:21:05.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:05.476+0000 7ff2e3e00700 1 -- 172.21.15.33:0/399618622 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) ==== 74+0+18385 (secure 0 0 0) 0x7ff2dc013070 con 0x7ff2ec1055d0 2024-05-03T18:21:05.476 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:21:05.476 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":71,"fsid":"5703bba4-0978-11ef-bc97-c7b262605968","created":"2024-05-03T18:11:00.899879+0000","modified":"2024-05-03T18:21:01.555753+0000","last_up_change":"2024-05-03T18:21:00.390173+0000","last_in_change":"2024-05-03T18:20:34.103727+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-03T18:15:37.297738+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fdcc4e66-4ba0-4378-a9a4-7d907cd00b4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":637686669},{"type":"v1","addr":"172.21.15.33:6803","nonce":637686669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":637686669},{"type":"v1","addr":"172.21.15.33:6805","nonce":637686669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":637686669},{"type":"v1","addr":"172.21.15.33:6809","nonce":637686669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":637686669},{"type":"v1","addr":"172.21.15.33:6807","nonce":637686669}]},"public_addr":"172.21.15.33:6803/637686669","cluster_addr":"172.21.15.33:6805/637686669","heartbeat_back_addr":"172.21.15.33:6809/637686669","heartbeat_front_addr":"172.21.15.33:6807/637686669","state":["exists","up"]},{"osd":1,"uuid":"cf78cad3-fc24-43f4-8784-21cabea7277c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":1536122841},{"type":"v1","addr":"172.21.15.99:6801","nonce":1536122841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":1536122841},{"type":"v1","addr":"172.21.15.99:6803","nonce":1536122841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":1536122841},{"type":"v1","addr":"172.21.15.99:6807","nonce":1536122841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":1536122841},{"type":"v1","addr":"172.21.15.99:6805","nonce":1536122841}]},"public_addr":"172.21.15.99:6801/1536122841","cluster_addr":"172.21.15.99:6803/1536122841","heartbeat_back_addr":"172.21.15.99:6807/1536122841","heartbeat_front_addr":"172.21.15.99:6805/1536122841","state":["exists","up"]},{"osd":2,"uuid":"59f066d7-626b-410a-8d67-daca6fbbd3ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":1995659622},{"type":"v1","addr":"172.21.15.150:6801","nonce":1995659622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":1995659622},{"type":"v1","addr":"172.21.15.150:6803","nonce":1995659622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":1995659622},{"type":"v1","addr":"172.21.15.150:6807","nonce":1995659622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":1995659622},{"type":"v1","addr":"172.21.15.150:6805","nonce":1995659622}]},"public_addr":"172.21.15.150:6801/1995659622","cluster_addr":"172.21.15.150:6803/1995659622","heartbeat_back_addr":"172.21.15.150:6807/1995659622","heartbeat_front_addr":"172.21.15.150:6805/1995659622","state":["exists","up"]},{"osd":3,"uuid":"d88ed022-4dcb-45fc-9fc6-41ef7c4bf202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":3814942724},{"type":"v1","addr":"172.21.15.33:6811","nonce":3814942724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":3814942724},{"type":"v1","addr":"172.21.15.33:6813","nonce":3814942724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":3814942724},{"type":"v1","addr":"172.21.15.33:6817","nonce":3814942724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":3814942724},{"type":"v1","addr":"172.21.15.33:6815","nonce":3814942724}]},"public_addr":"172.21.15.33:6811/3814942724","cluster_addr":"172.21.15.33:6813/3814942724","heartbeat_back_addr":"172.21.15.33:6817/3814942724","heartbeat_front_addr":"172.21.15.33:6815/3814942724","state":["exists","up"]},{"osd":4,"uuid":"1235e1a6-ff6e-4668-ac1a-073c2a3b7c98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":29081029},{"type":"v1","addr":"172.21.15.99:6809","nonce":29081029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":29081029},{"type":"v1","addr":"172.21.15.99:6811","nonce":29081029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":29081029},{"type":"v1","addr":"172.21.15.99:6815","nonce":29081029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":29081029},{"type":"v1","addr":"172.21.15.99:6813","nonce":29081029}]},"public_addr":"172.21.15.99:6809/29081029","cluster_addr":"172.21.15.99:6811/29081029","heartbeat_back_addr":"172.21.15.99:6815/29081029","heartbeat_front_addr":"172.21.15.99:6813/29081029","state":["exists","up"]},{"osd":5,"uuid":"4b84bff8-d6d7-4258-a0ee-53bc19802585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":3172189533},{"type":"v1","addr":"172.21.15.150:6809","nonce":3172189533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3172189533},{"type":"v1","addr":"172.21.15.150:6811","nonce":3172189533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3172189533},{"type":"v1","addr":"172.21.15.150:6815","nonce":3172189533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3172189533},{"type":"v1","addr":"172.21.15.150:6813","nonce":3172189533}]},"public_addr":"172.21.15.150:6809/3172189533","cluster_addr":"172.21.15.150:6811/3172189533","heartbeat_back_addr":"172.21.15.150:6815/3172189533","heartbeat_front_addr":"172.21.15.150:6813/3172189533","state":["exists","up"]},{"osd":6,"uuid":"51f5fbdd-3794-4c48-9f6c-478f0fa35ac5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":4234541418},{"type":"v1","addr":"172.21.15.33:6819","nonce":4234541418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":4234541418},{"type":"v1","addr":"172.21.15.33:6821","nonce":4234541418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":4234541418},{"type":"v1","addr":"172.21.15.33:6825","nonce":4234541418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":4234541418},{"type":"v1","addr":"172.21.15.33:6823","nonce":4234541418}]},"public_addr":"172.21.15.33:6819/4234541418","cluster_addr":"172.21.15.33:6821/4234541418","heartbeat_back_addr":"172.21.15.33:6825/4234541418","heartbeat_front_addr":"172.21.15.33:6823/4234541418","state":["exists","up"]},{"osd":7,"uuid":"982cc5b3-3163-4505-9bde-a27a8c289457","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":908673090},{"type":"v1","addr":"172.21.15.99:6817","nonce":908673090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":908673090},{"type":"v1","addr":"172.21.15.99:6819","nonce":908673090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":908673090},{"type":"v1","addr":"172.21.15.99:6823","nonce":908673090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":908673090},{"type":"v1","addr":"172.21.15.99:6821","nonce":908673090}]},"public_addr":"172.21.15.99:6817/908673090","cluster_addr":"172.21.15.99:6819/908673090","heartbeat_back_addr":"172.21.15.99:6823/908673090","heartbeat_front_addr":"172.21.15.99:6821/908673090","state":["exists","up"]},{"osd":8,"uuid":"84c9d037-eb84-4eb4-8aa2-e784b703d36c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":1308083602},{"type":"v1","addr":"172.21.15.150:6817","nonce":1308083602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":1308083602},{"type":"v1","addr":"172.21.15.150:6819","nonce":1308083602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":1308083602},{"type":"v1","addr":"172.21.15.150:6823","nonce":1308083602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":1308083602},{"type":"v1","addr":"172.21.15.150:6821","nonce":1308083602}]},"public_addr":"172.21.15.150:6817/1308083602","cluster_addr":"172.21.15.150:6819/1308083602","heartbeat_back_addr":"172.21.15.150:6823/1308083602","heartbeat_front_addr":"172.21.15.150:6821/1308083602","state":["exists","up"]},{"osd":9,"uuid":"a02e965f-0d8f-45e2-9490-337f7f6e3881","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":4133096905},{"type":"v1","addr":"172.21.15.33:6827","nonce":4133096905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":4133096905},{"type":"v1","addr":"172.21.15.33:6829","nonce":4133096905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4133096905},{"type":"v1","addr":"172.21.15.33:6833","nonce":4133096905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":4133096905},{"type":"v1","addr":"172.21.15.33:6831","nonce":4133096905}]},"public_addr":"172.21.15.33:6827/4133096905","cluster_addr":"172.21.15.33:6829/4133096905","heartbeat_back_addr":"172.21.15.33:6833/4133096905","heartbeat_front_addr":"172.21.15.33:6831/4133096905","state":["exists","up"]},{"osd":10,"uuid":"8b29bb96-76e8-45a0-ab19-8d082d755fe9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":935746136},{"type":"v1","addr":"172.21.15.99:6825","nonce":935746136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":935746136},{"type":"v1","addr":"172.21.15.99:6827","nonce":935746136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":935746136},{"type":"v1","addr":"172.21.15.99:6831","nonce":935746136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":935746136},{"type":"v1","addr":"172.21.15.99:6829","nonce":935746136}]},"public_addr":"172.21.15.99:6825/935746136","cluster_addr":"172.21.15.99:6827/935746136","heartbeat_back_addr":"172.21.15.99:6831/935746136","heartbeat_front_addr":"172.21.15.99:6829/935746136","state":["exists","up"]},{"osd":11,"uuid":"07a6a825-c902-4bfd-97cc-cc9074532af5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2069349808},{"type":"v1","addr":"172.21.15.150:6825","nonce":2069349808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2069349808},{"type":"v1","addr":"172.21.15.150:6827","nonce":2069349808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2069349808},{"type":"v1","addr":"172.21.15.150:6831","nonce":2069349808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2069349808},{"type":"v1","addr":"172.21.15.150:6829","nonce":2069349808}]},"public_addr":"172.21.15.150:6825/2069349808","cluster_addr":"172.21.15.150:6827/2069349808","heartbeat_back_addr":"172.21.15.150:6831/2069349808","heartbeat_front_addr":"172.21.15.150:6829/2069349808","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:14:27.454536+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:15:00.538570+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:15:34.125289+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:16:09.853870+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:16:44.928107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:17:19.644037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:17:56.108443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:18:31.731499+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:19:08.062358+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:19:45.204430+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:20:21.013710+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:20:57.653698+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/3991947997":"2024-05-04T18:12:19.127652+0000","172.21.15.33:0/965859483":"2024-05-04T18:12:19.127652+0000","172.21.15.33:0/2014902144":"2024-05-04T18:11:44.239835+0000","172.21.15.33:0/1314704208":"2024-05-04T18:11:44.239835+0000","172.21.15.33:0/1614640996":"2024-05-04T18:11:44.239835+0000","172.21.15.33:6800/4007322071":"2024-05-04T18:11:44.239835+0000","172.21.15.33:6801/2830135294":"2024-05-04T18:12:19.127652+0000","172.21.15.33:6800/2830135294":"2024-05-04T18:12:19.127652+0000","172.21.15.33:0/3767665033":"2024-05-04T18:12:19.127652+0000","172.21.15.33:6801/4007322071":"2024-05-04T18:11:44.239835+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-03T18:21:05.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:05.478+0000 7ff2f1200700 1 -- 172.21.15.33:0/399618622 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7ff2d00b2900 msgr2=0x7ff2d00b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:21:05.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:05.478+0000 7ff2f1200700 1 --2- 172.21.15.33:0/399618622 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7ff2d00b2900 0x7ff2d00b4dc0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff2d8005fd0 tx=0x7ff2d800a560 comp rx=0 tx=0).stop 2024-05-03T18:21:05.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:05.479+0000 7ff2f1200700 1 -- 172.21.15.33:0/399618622 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2ec1055d0 msgr2=0x7ff2ec0779b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:21:05.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:05.479+0000 7ff2f1200700 1 --2- 172.21.15.33:0/399618622 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2ec1055d0 0x7ff2ec0779b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc000c00 tx=0x7ff2dc005ef0 comp rx=0 tx=0).stop 2024-05-03T18:21:05.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:05.479+0000 7ff2f1200700 1 -- 172.21.15.33:0/399618622 shutdown_connections 2024-05-03T18:21:05.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:05.479+0000 7ff2f1200700 1 --2- 172.21.15.33:0/399618622 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7ff2d00b2900 0x7ff2d00b4dc0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:21:05.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:05.479+0000 7ff2f1200700 1 --2- 172.21.15.33:0/399618622 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2ec10b660 0x7ff2ec078430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:21:05.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:05.479+0000 7ff2f1200700 1 --2- 172.21.15.33:0/399618622 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2ec1055d0 0x7ff2ec0779b0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:21:05.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:05.479+0000 7ff2f1200700 1 --2- 172.21.15.33:0/399618622 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2ec105f90 0x7ff2ec077ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:21:05.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:05.479+0000 7ff2f1200700 1 -- 172.21.15.33:0/399618622 >> 172.21.15.33:0/399618622 conn(0x7ff2ec1010c0 msgr2=0x7ff2ec1029c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:21:05.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:05.479+0000 7ff2f1200700 1 -- 172.21.15.33:0/399618622 shutdown_connections 2024-05-03T18:21:05.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:05.479+0000 7ff2f1200700 1 -- 172.21.15.33:0/399618622 wait complete. 2024-05-03T18:21:06.264 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-03T18:15:37.297738+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-03T18:21:06.264 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-05-03T18:21:06.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:06 smithi033 ceph-mon[28330]: pgmap v324: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:21:06.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:06 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/399618622' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T18:21:06.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:06 smithi150 ceph-mon[32492]: pgmap v324: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:21:06.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:06 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/399618622' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T18:21:06.739 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:21:06.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:06 smithi099 ceph-mon[31658]: pgmap v324: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:21:06.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:06 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/399618622' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T18:21:08.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:08 smithi033 ceph-mon[28330]: pgmap v325: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:21:08.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:08 smithi150 ceph-mon[32492]: pgmap v325: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:21:08.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:08 smithi099 ceph-mon[31658]: pgmap v325: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:21:09.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:09.040+0000 7fc050000700 1 -- 172.21.15.33:0/1367797138 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc048106080 msgr2=0x7fc048112590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:21:09.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:09.040+0000 7fc050000700 1 --2- 172.21.15.33:0/1367797138 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc048106080 0x7fc048112590 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc04000b0a0 tx=0x7fc04000b3b0 comp rx=0 tx=0).stop 2024-05-03T18:21:09.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:09.041+0000 7fc050000700 1 -- 172.21.15.33:0/1367797138 shutdown_connections 2024-05-03T18:21:09.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:09.041+0000 7fc050000700 1 --2- 172.21.15.33:0/1367797138 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc048106080 0x7fc048112590 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:21:09.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:09.041+0000 7fc050000700 1 --2- 172.21.15.33:0/1367797138 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc04810b6c0 0x7fc04810baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:21:09.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:09.041+0000 7fc050000700 1 --2- 172.21.15.33:0/1367797138 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0481056c0 0x7fc048105b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:21:09.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:09.041+0000 7fc050000700 1 -- 172.21.15.33:0/1367797138 >> 172.21.15.33:0/1367797138 conn(0x7fc0481011b0 msgr2=0x7fc0481035d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:21:09.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:09.041+0000 7fc050000700 1 -- 172.21.15.33:0/1367797138 shutdown_connections 2024-05-03T18:21:09.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:09.041+0000 7fc050000700 1 -- 172.21.15.33:0/1367797138 wait complete. 2024-05-03T18:21:09.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:09.042+0000 7fc050000700 1 Processor -- start 2024-05-03T18:21:09.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:09.042+0000 7fc050000700 1 -- start start 2024-05-03T18:21:09.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:09.042+0000 7fc050000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0481056c0 0x7fc0481a18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:21:09.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:09.042+0000 7fc050000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc048106080 0x7fc0481a1e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:21:09.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:09.042+0000 7fc050000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc04810b6c0 0x7fc04819bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:21:09.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:09.042+0000 7fc050000700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc048115800 con 0x7fc0481056c0 2024-05-03T18:21:09.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:09.042+0000 7fc050000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc048115680 con 0x7fc048106080 2024-05-03T18:21:09.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:09.042+0000 7fc050000700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc048115980 con 0x7fc04810b6c0 2024-05-03T18:21:09.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:09.043+0000 7fc04d600700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0481056c0 0x7fc0481a18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:21:09.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:09.043+0000 7fc04d600700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0481056c0 0x7fc0481a18d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55346/0 (socket says 172.21.15.33:55346) 2024-05-03T18:21:09.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:09.043+0000 7fc04d600700 1 -- 172.21.15.33:0/1240085512 learned_addr learned my addr 172.21.15.33:0/1240085512 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:21:09.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:09.043+0000 7fc04cc00700 1 --2- 172.21.15.33:0/1240085512 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc048106080 0x7fc0481a1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:21:09.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:09.043+0000 7fc04e000700 1 --2- 172.21.15.33:0/1240085512 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc04810b6c0 0x7fc04819bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:21:09.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:09.043+0000 7fc04d600700 1 -- 172.21.15.33:0/1240085512 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc04810b6c0 msgr2=0x7fc04819bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:21:09.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:09.043+0000 7fc04d600700 1 --2- 172.21.15.33:0/1240085512 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc04810b6c0 0x7fc04819bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:21:09.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:09.043+0000 7fc04d600700 1 -- 172.21.15.33:0/1240085512 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc048106080 msgr2=0x7fc0481a1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:21:09.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:09.043+0000 7fc04d600700 1 --2- 172.21.15.33:0/1240085512 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc048106080 0x7fc0481a1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:21:09.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:09.043+0000 7fc04d600700 1 -- 172.21.15.33:0/1240085512 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc04819c240 con 0x7fc0481056c0 2024-05-03T18:21:09.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:09.043+0000 7fc04cc00700 1 --2- 172.21.15.33:0/1240085512 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc048106080 0x7fc0481a1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:21:09.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:09.044+0000 7fc04d600700 1 --2- 172.21.15.33:0/1240085512 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0481056c0 0x7fc0481a18d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc03c00d980 tx=0x7fc03c00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:21:09.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:09.044+0000 7fc046000700 1 -- 172.21.15.33:0/1240085512 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc03c017070 con 0x7fc0481056c0 2024-05-03T18:21:09.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:09.044+0000 7fc046000700 1 -- 172.21.15.33:0/1240085512 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc03c004830 con 0x7fc0481056c0 2024-05-03T18:21:09.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:09.044+0000 7fc046000700 1 -- 172.21.15.33:0/1240085512 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc03c0053a0 con 0x7fc0481056c0 2024-05-03T18:21:09.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:09.044+0000 7fc050000700 1 -- 172.21.15.33:0/1240085512 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc04819c530 con 0x7fc0481056c0 2024-05-03T18:21:09.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:09.044+0000 7fc050000700 1 -- 172.21.15.33:0/1240085512 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0481a87a0 con 0x7fc0481056c0 2024-05-03T18:21:09.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:09.046+0000 7fc050000700 1 -- 172.21.15.33:0/1240085512 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc04806a190 con 0x7fc0481056c0 2024-05-03T18:21:09.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:09.048+0000 7fc046000700 1 -- 172.21.15.33:0/1240085512 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fc03c0049e0 con 0x7fc0481056c0 2024-05-03T18:21:09.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:09.050+0000 7fc046000700 1 --2- 172.21.15.33:0/1240085512 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fc0300b2880 0x7fc0300b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:21:09.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:09.050+0000 7fc046000700 1 -- 172.21.15.33:0/1240085512 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7fc03c0e1d50 con 0x7fc0481056c0 2024-05-03T18:21:09.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:09.050+0000 7fc04cc00700 1 --2- 172.21.15.33:0/1240085512 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fc0300b2880 0x7fc0300b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:21:09.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:09.051+0000 7fc04cc00700 1 --2- 172.21.15.33:0/1240085512 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fc0300b2880 0x7fc0300b4d40 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc034005950 tx=0x7fc0340058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:21:09.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:09.055+0000 7fc046000700 1 -- 172.21.15.33:0/1240085512 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fc03c020020 con 0x7fc0481056c0 2024-05-03T18:21:09.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:09.290+0000 7fc050000700 1 -- 172.21.15.33:0/1240085512 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc04802e510 con 0x7fc0481056c0 2024-05-03T18:21:09.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:09.290+0000 7fc046000700 1 -- 172.21.15.33:0/1240085512 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v71) ==== 93+0+10 (secure 0 0 0) 0x7fc03c0a9e70 con 0x7fc0481056c0 2024-05-03T18:21:09.290 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-05-03T18:21:09.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:09.293+0000 7fc050000700 1 -- 172.21.15.33:0/1240085512 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fc0300b2880 msgr2=0x7fc0300b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:21:09.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:09.293+0000 7fc050000700 1 --2- 172.21.15.33:0/1240085512 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fc0300b2880 0x7fc0300b4d40 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc034005950 tx=0x7fc0340058e0 comp rx=0 tx=0).stop 2024-05-03T18:21:09.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:09.293+0000 7fc050000700 1 -- 172.21.15.33:0/1240085512 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0481056c0 msgr2=0x7fc0481a18d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:21:09.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:09.293+0000 7fc050000700 1 --2- 172.21.15.33:0/1240085512 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0481056c0 0x7fc0481a18d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc03c00d980 tx=0x7fc03c00dd40 comp rx=0 tx=0).stop 2024-05-03T18:21:09.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:09.294+0000 7fc050000700 1 -- 172.21.15.33:0/1240085512 shutdown_connections 2024-05-03T18:21:09.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:09.294+0000 7fc050000700 1 --2- 172.21.15.33:0/1240085512 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc04810b6c0 0x7fc04819bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:21:09.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:09.294+0000 7fc050000700 1 --2- 172.21.15.33:0/1240085512 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0481056c0 0x7fc0481a18d0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:21:09.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:09.294+0000 7fc050000700 1 --2- 172.21.15.33:0/1240085512 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fc0300b2880 0x7fc0300b4d40 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:21:09.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:09.294+0000 7fc050000700 1 --2- 172.21.15.33:0/1240085512 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc048106080 0x7fc0481a1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:21:09.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:09.294+0000 7fc050000700 1 -- 172.21.15.33:0/1240085512 >> 172.21.15.33:0/1240085512 conn(0x7fc0481011b0 msgr2=0x7fc0481019c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:21:09.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:09.294+0000 7fc050000700 1 -- 172.21.15.33:0/1240085512 shutdown_connections 2024-05-03T18:21:09.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:09.294+0000 7fc050000700 1 -- 172.21.15.33:0/1240085512 wait complete. 2024-05-03T18:21:09.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:09 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/1240085512' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-03T18:21:09.664 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:09 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/1240085512' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-03T18:21:09.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:09 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/1240085512' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-03T18:21:10.206 INFO:tasks.cephadm:Adding mds.a on smithi033 2024-05-03T18:21:10.206 INFO:tasks.cephadm:Adding mds.d on smithi033 2024-05-03T18:21:10.206 INFO:tasks.cephadm:Adding mds.g on smithi033 2024-05-03T18:21:10.206 INFO:tasks.cephadm:Adding mds.j on smithi033 2024-05-03T18:21:10.207 INFO:tasks.cephadm:Adding mds.b on smithi099 2024-05-03T18:21:10.207 INFO:tasks.cephadm:Adding mds.e on smithi099 2024-05-03T18:21:10.207 INFO:tasks.cephadm:Adding mds.h on smithi099 2024-05-03T18:21:10.207 INFO:tasks.cephadm:Adding mds.k on smithi099 2024-05-03T18:21:10.207 INFO:tasks.cephadm:Adding mds.c on smithi150 2024-05-03T18:21:10.207 INFO:tasks.cephadm:Adding mds.f on smithi150 2024-05-03T18:21:10.207 INFO:tasks.cephadm:Adding mds.i on smithi150 2024-05-03T18:21:10.207 INFO:tasks.cephadm:Adding mds.l on smithi150 2024-05-03T18:21:10.207 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph orch apply mds all '12;smithi033=a;smithi033=d;smithi033=g;smithi033=j;smithi099=b;smithi099=e;smithi099=h;smithi099=k;smithi150=c;smithi150=f;smithi150=i;smithi150=l' 2024-05-03T18:21:10.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:10 smithi033 ceph-mon[28330]: pgmap v326: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:21:10.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:10 smithi150 ceph-mon[32492]: pgmap v326: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:21:10.654 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.c/config 2024-05-03T18:21:10.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:10 smithi099 ceph-mon[31658]: pgmap v326: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:21:12.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:12 smithi150 ceph-mon[32492]: pgmap v327: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:21:12.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:12 smithi033 ceph-mon[28330]: pgmap v327: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:21:12.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:12 smithi099 ceph-mon[31658]: pgmap v327: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:21:12.926 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:21:12.925+0000 7f9f53200700 1 -- 172.21.15.150:0/3446590501 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f54106820 msgr2=0x7f9f54106c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:21:12.927 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:21:12.925+0000 7f9f53200700 1 --2- 172.21.15.150:0/3446590501 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f54106820 0x7f9f54106c80 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c009990 tx=0x7f9f3c009ca0 comp rx=0 tx=0).stop 2024-05-03T18:21:12.927 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:21:12.925+0000 7f9f53200700 1 -- 172.21.15.150:0/3446590501 shutdown_connections 2024-05-03T18:21:12.927 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:21:12.925+0000 7f9f53200700 1 --2- 172.21.15.150:0/3446590501 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f541071c0 0x7f9f5410da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:21:12.927 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:21:12.925+0000 7f9f53200700 1 --2- 172.21.15.150:0/3446590501 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f54105620 0x7f9f54105a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:21:12.927 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:21:12.925+0000 7f9f53200700 1 --2- 172.21.15.150:0/3446590501 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f54106820 0x7f9f54106c80 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:21:12.927 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:21:12.925+0000 7f9f53200700 1 -- 172.21.15.150:0/3446590501 >> 172.21.15.150:0/3446590501 conn(0x7f9f54100bc0 msgr2=0x7f9f54103000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:21:12.927 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:21:12.925+0000 7f9f53200700 1 -- 172.21.15.150:0/3446590501 shutdown_connections 2024-05-03T18:21:12.927 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:21:12.925+0000 7f9f53200700 1 -- 172.21.15.150:0/3446590501 wait complete. 2024-05-03T18:21:12.927 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:21:12.926+0000 7f9f53200700 1 Processor -- start 2024-05-03T18:21:12.927 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:21:12.927+0000 7f9f53200700 1 -- start start 2024-05-03T18:21:12.929 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:21:12.927+0000 7f9f53200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f54105620 0x7f9f5419b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:21:12.929 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:21:12.927+0000 7f9f53200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f54106820 0x7f9f5419bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:21:12.929 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:21:12.927+0000 7f9f53200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f541071c0 0x7f9f541a2c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:21:12.930 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:21:12.927+0000 7f9f53200700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9f54110fa0 con 0x7f9f54105620 2024-05-03T18:21:12.930 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:21:12.927+0000 7f9f53200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9f54110e20 con 0x7f9f541071c0 2024-05-03T18:21:12.930 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:21:12.927+0000 7f9f53200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9f54111120 con 0x7f9f54106820 2024-05-03T18:21:12.930 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:21:12.927+0000 7f9f4be00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f54106820 0x7f9f5419bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:21:12.930 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:21:12.927+0000 7f9f4be00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f54106820 0x7f9f5419bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:40676/0 (socket says 172.21.15.150:40676) 2024-05-03T18:21:12.930 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:21:12.927+0000 7f9f4be00700 1 -- 172.21.15.150:0/1392355734 learned_addr learned my addr 172.21.15.150:0/1392355734 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-05-03T18:21:12.930 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:21:12.927+0000 7f9f51e00700 1 --2- 172.21.15.150:0/1392355734 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f54105620 0x7f9f5419b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:21:12.930 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:21:12.928+0000 7f9f52800700 1 --2- 172.21.15.150:0/1392355734 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f541071c0 0x7f9f541a2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:21:12.930 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:21:12.928+0000 7f9f4be00700 1 -- 172.21.15.150:0/1392355734 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f541071c0 msgr2=0x7f9f541a2c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:21:12.930 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:21:12.928+0000 7f9f4be00700 1 --2- 172.21.15.150:0/1392355734 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f541071c0 0x7f9f541a2c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:21:12.930 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:21:12.928+0000 7f9f4be00700 1 -- 172.21.15.150:0/1392355734 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f54105620 msgr2=0x7f9f5419b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:21:12.931 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:21:12.928+0000 7f9f4be00700 1 --2- 172.21.15.150:0/1392355734 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f54105620 0x7f9f5419b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:21:12.931 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:21:12.928+0000 7f9f4be00700 1 -- 172.21.15.150:0/1392355734 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f541a3300 con 0x7f9f54106820 2024-05-03T18:21:12.931 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:21:12.928+0000 7f9f4be00700 1 --2- 172.21.15.150:0/1392355734 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f54106820 0x7f9f5419bbe0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c005910 tx=0x7f9f3c0047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:21:12.931 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:21:12.928+0000 7f9f4aa00700 1 -- 172.21.15.150:0/1392355734 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f3c027070 con 0x7f9f54106820 2024-05-03T18:21:12.931 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:21:12.929+0000 7f9f4aa00700 1 -- 172.21.15.150:0/1392355734 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9f3c0040e0 con 0x7f9f54106820 2024-05-03T18:21:12.931 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:21:12.929+0000 7f9f53200700 1 -- 172.21.15.150:0/1392355734 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f541a3590 con 0x7f9f54106820 2024-05-03T18:21:12.931 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:21:12.929+0000 7f9f4aa00700 1 -- 172.21.15.150:0/1392355734 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f3c022e10 con 0x7f9f54106820 2024-05-03T18:21:12.931 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:21:12.929+0000 7f9f53200700 1 -- 172.21.15.150:0/1392355734 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f541a39e0 con 0x7f9f54106820 2024-05-03T18:21:12.932 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:21:12.930+0000 7f9f53200700 1 -- 172.21.15.150:0/1392355734 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f5406a190 con 0x7f9f54106820 2024-05-03T18:21:12.932 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:21:12.932+0000 7f9f4aa00700 1 -- 172.21.15.150:0/1392355734 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f9f3c02a070 con 0x7f9f54106820 2024-05-03T18:21:12.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:21:12.933+0000 7f9f4aa00700 1 --2- 172.21.15.150:0/1392355734 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f9f380b2880 0x7f9f380b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:21:12.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:21:12.934+0000 7f9f4aa00700 1 -- 172.21.15.150:0/1392355734 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f9f3c0e2d30 con 0x7f9f54106820 2024-05-03T18:21:12.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:21:12.934+0000 7f9f51e00700 1 --2- 172.21.15.150:0/1392355734 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f9f380b2880 0x7f9f380b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:21:12.935 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:21:12.935+0000 7f9f51e00700 1 --2- 172.21.15.150:0/1392355734 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f9f380b2880 0x7f9f380b4d40 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9f44005950 tx=0x7f9f440058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:21:12.939 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:21:12.938+0000 7f9f4aa00700 1 -- 172.21.15.150:0/1392355734 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f9f3c013070 con 0x7f9f54106820 2024-05-03T18:21:13.179 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:21:13.179+0000 7f9f53200700 1 -- 172.21.15.150:0/1392355734 --> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi033=a;smithi033=d;smithi033=g;smithi033=j;smithi099=b;smithi099=e;smithi099=h;smithi099=k;smithi150=c;smithi150=f;smithi150=i;smithi150=l", "target": ["mon-mgr", ""]}) -- 0x7f9f54063ba0 con 0x7f9f380b2880 2024-05-03T18:21:13.197 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:21:13.197+0000 7f9f4aa00700 1 -- 172.21.15.150:0/1392355734 <== mgr.14152 v2:172.21.15.33:6800/3642895501 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f9f54063ba0 con 0x7f9f380b2880 2024-05-03T18:21:13.198 INFO:teuthology.orchestra.run.smithi150.stdout:Scheduled mds.all update... 2024-05-03T18:21:13.200 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:21:13.200+0000 7f9f53200700 1 -- 172.21.15.150:0/1392355734 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f9f380b2880 msgr2=0x7f9f380b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:21:13.201 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:21:13.200+0000 7f9f53200700 1 --2- 172.21.15.150:0/1392355734 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f9f380b2880 0x7f9f380b4d40 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9f44005950 tx=0x7f9f440058e0 comp rx=0 tx=0).stop 2024-05-03T18:21:13.201 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:21:13.200+0000 7f9f53200700 1 -- 172.21.15.150:0/1392355734 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f54106820 msgr2=0x7f9f5419bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:21:13.201 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:21:13.200+0000 7f9f53200700 1 --2- 172.21.15.150:0/1392355734 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f54106820 0x7f9f5419bbe0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c005910 tx=0x7f9f3c0047b0 comp rx=0 tx=0).stop 2024-05-03T18:21:13.201 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:21:13.200+0000 7f9f53200700 1 -- 172.21.15.150:0/1392355734 shutdown_connections 2024-05-03T18:21:13.201 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:21:13.200+0000 7f9f53200700 1 --2- 172.21.15.150:0/1392355734 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f9f380b2880 0x7f9f380b4d40 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:21:13.201 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:21:13.200+0000 7f9f53200700 1 --2- 172.21.15.150:0/1392355734 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f541071c0 0x7f9f541a2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:21:13.201 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:21:13.200+0000 7f9f53200700 1 --2- 172.21.15.150:0/1392355734 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f54105620 0x7f9f5419b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:21:13.201 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:21:13.200+0000 7f9f53200700 1 --2- 172.21.15.150:0/1392355734 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f54106820 0x7f9f5419bbe0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:21:13.201 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:21:13.200+0000 7f9f53200700 1 -- 172.21.15.150:0/1392355734 >> 172.21.15.150:0/1392355734 conn(0x7f9f54100bc0 msgr2=0x7f9f54109be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:21:13.201 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:21:13.200+0000 7f9f53200700 1 -- 172.21.15.150:0/1392355734 shutdown_connections 2024-05-03T18:21:13.201 INFO:teuthology.orchestra.run.smithi150.stderr:2024-05-03T18:21:13.201+0000 7f9f53200700 1 -- 172.21.15.150:0/1392355734 wait complete. 2024-05-03T18:21:14.031 DEBUG:teuthology.orchestra.run.smithi033:mds.a> sudo journalctl -f -n 0 -u ceph-5703bba4-0978-11ef-bc97-c7b262605968@mds.a.service 2024-05-03T18:21:14.040 DEBUG:teuthology.orchestra.run.smithi033:mds.d> sudo journalctl -f -n 0 -u ceph-5703bba4-0978-11ef-bc97-c7b262605968@mds.d.service 2024-05-03T18:21:14.088 DEBUG:teuthology.orchestra.run.smithi033:mds.g> sudo journalctl -f -n 0 -u ceph-5703bba4-0978-11ef-bc97-c7b262605968@mds.g.service 2024-05-03T18:21:14.135 DEBUG:teuthology.orchestra.run.smithi033:mds.j> sudo journalctl -f -n 0 -u ceph-5703bba4-0978-11ef-bc97-c7b262605968@mds.j.service 2024-05-03T18:21:14.179 DEBUG:teuthology.orchestra.run.smithi099:mds.b> sudo journalctl -f -n 0 -u ceph-5703bba4-0978-11ef-bc97-c7b262605968@mds.b.service 2024-05-03T18:21:14.187 DEBUG:teuthology.orchestra.run.smithi099:mds.e> sudo journalctl -f -n 0 -u ceph-5703bba4-0978-11ef-bc97-c7b262605968@mds.e.service 2024-05-03T18:21:14.233 DEBUG:teuthology.orchestra.run.smithi099:mds.h> sudo journalctl -f -n 0 -u ceph-5703bba4-0978-11ef-bc97-c7b262605968@mds.h.service 2024-05-03T18:21:14.279 DEBUG:teuthology.orchestra.run.smithi099:mds.k> sudo journalctl -f -n 0 -u ceph-5703bba4-0978-11ef-bc97-c7b262605968@mds.k.service 2024-05-03T18:21:14.297 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:14 smithi150 ceph-mon[32492]: from='client.24406 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi033=a;smithi033=d;smithi033=g;smithi033=j;smithi099=b;smithi099=e;smithi099=h;smithi099=k;smithi150=c;smithi150=f;smithi150=i;smithi150=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:21:14.298 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:14 smithi150 ceph-mon[32492]: Saving service mds.all spec with placement smithi033=a;smithi033=d;smithi033=g;smithi033=j;smithi099=b;smithi099=e;smithi099=h;smithi099=k;smithi150=c;smithi150=f;smithi150=i;smithi150=l;count:12 2024-05-03T18:21:14.298 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:14 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:14.298 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:14 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:21:14.298 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:14 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:21:14.298 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:14 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:21:14.298 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:14 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:14.298 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:14 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T18:21:14.298 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:14 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T18:21:14.298 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:14 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:21:14.298 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:14 smithi150 ceph-mon[32492]: Deploying daemon mds.l on smithi150 2024-05-03T18:21:14.298 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:14 smithi150 ceph-mon[32492]: pgmap v328: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:21:14.323 DEBUG:teuthology.orchestra.run.smithi150:mds.c> sudo journalctl -f -n 0 -u ceph-5703bba4-0978-11ef-bc97-c7b262605968@mds.c.service 2024-05-03T18:21:14.332 DEBUG:teuthology.orchestra.run.smithi150:mds.f> sudo journalctl -f -n 0 -u ceph-5703bba4-0978-11ef-bc97-c7b262605968@mds.f.service 2024-05-03T18:21:14.377 DEBUG:teuthology.orchestra.run.smithi150:mds.i> sudo journalctl -f -n 0 -u ceph-5703bba4-0978-11ef-bc97-c7b262605968@mds.i.service 2024-05-03T18:21:14.421 DEBUG:teuthology.orchestra.run.smithi150:mds.l> sudo journalctl -f -n 0 -u ceph-5703bba4-0978-11ef-bc97-c7b262605968@mds.l.service 2024-05-03T18:21:14.468 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-05-03T18:21:14.468 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T18:21:14.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:14 smithi033 ceph-mon[28330]: from='client.24406 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi033=a;smithi033=d;smithi033=g;smithi033=j;smithi099=b;smithi099=e;smithi099=h;smithi099=k;smithi150=c;smithi150=f;smithi150=i;smithi150=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:21:14.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:14 smithi033 ceph-mon[28330]: Saving service mds.all spec with placement smithi033=a;smithi033=d;smithi033=g;smithi033=j;smithi099=b;smithi099=e;smithi099=h;smithi099=k;smithi150=c;smithi150=f;smithi150=i;smithi150=l;count:12 2024-05-03T18:21:14.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:14 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:14.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:14 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:21:14.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:14 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:21:14.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:14 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:21:14.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:14 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:14.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:14 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T18:21:14.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:14 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T18:21:14.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:14 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:21:14.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:14 smithi033 ceph-mon[28330]: Deploying daemon mds.l on smithi150 2024-05-03T18:21:14.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:14 smithi033 ceph-mon[28330]: pgmap v328: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:21:14.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:14 smithi099 ceph-mon[31658]: from='client.24406 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi033=a;smithi033=d;smithi033=g;smithi033=j;smithi099=b;smithi099=e;smithi099=h;smithi099=k;smithi150=c;smithi150=f;smithi150=i;smithi150=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:21:14.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:14 smithi099 ceph-mon[31658]: Saving service mds.all spec with placement smithi033=a;smithi033=d;smithi033=g;smithi033=j;smithi099=b;smithi099=e;smithi099=h;smithi099=k;smithi150=c;smithi150=f;smithi150=i;smithi150=l;count:12 2024-05-03T18:21:14.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:14 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:14.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:14 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:21:14.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:14 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:21:14.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:14 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:21:14.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:14 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:14.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:14 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T18:21:14.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:14 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T18:21:14.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:14 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:21:14.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:14 smithi099 ceph-mon[31658]: Deploying daemon mds.l on smithi150 2024-05-03T18:21:14.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:14 smithi099 ceph-mon[31658]: pgmap v328: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:21:14.927 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:21:16.621 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:16 smithi033 ceph-mon[28330]: pgmap v329: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:21:16.631 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:16 smithi150 ceph-mon[32492]: pgmap v329: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:21:16.632 INFO:journalctl@ceph.mds.l.smithi150.stdout:May 03 18:21:16 smithi150 systemd[1]: Starting Ceph mds.l for 5703bba4-0978-11ef-bc97-c7b262605968... 2024-05-03T18:21:16.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:16 smithi099 ceph-mon[31658]: pgmap v329: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:21:17.017 INFO:journalctl@ceph.mds.l.smithi150.stdout:May 03 18:21:16 smithi150 podman[62571]: 2024-05-03 18:21:16.609315575 +0000 UTC m=+0.032330157 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T18:21:17.018 INFO:journalctl@ceph.mds.l.smithi150.stdout:May 03 18:21:16 smithi150 podman[62571]: 2024-05-03 18:21:16.727671546 +0000 UTC m=+0.150686172 container create 14f4381f26949366e4605c59850eb394b5053417ed59d9ec0dee92482c658472 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-l, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.license=GPLv2) 2024-05-03T18:21:17.018 INFO:journalctl@ceph.mds.l.smithi150.stdout:May 03 18:21:17 smithi150 podman[62571]: 2024-05-03 18:21:17.015896866 +0000 UTC m=+0.438911479 container init 14f4381f26949366e4605c59850eb394b5053417ed59d9ec0dee92482c658472 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-l, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-03T18:21:17.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:17.108+0000 7faacec00700 1 -- 172.21.15.33:0/2414447969 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faac806bb40 msgr2=0x7faac8110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:21:17.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:17.108+0000 7faacec00700 1 --2- 172.21.15.33:0/2414447969 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faac806bb40 0x7faac8110330 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7faac0009990 tx=0x7faac0009ca0 comp rx=0 tx=0).stop 2024-05-03T18:21:17.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:17.108+0000 7faacec00700 1 -- 172.21.15.33:0/2414447969 shutdown_connections 2024-05-03T18:21:17.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:17.108+0000 7faacec00700 1 --2- 172.21.15.33:0/2414447969 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faac806bb40 0x7faac8110330 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:21:17.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:17.108+0000 7faacec00700 1 --2- 172.21.15.33:0/2414447969 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faac8109510 0x7faac81098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:21:17.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:17.108+0000 7faacec00700 1 --2- 172.21.15.33:0/2414447969 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faac806b180 0x7faac806b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:21:17.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:17.108+0000 7faacec00700 1 -- 172.21.15.33:0/2414447969 >> 172.21.15.33:0/2414447969 conn(0x7faac8079350 msgr2=0x7faac8079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:21:17.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:17.108+0000 7faacec00700 1 -- 172.21.15.33:0/2414447969 shutdown_connections 2024-05-03T18:21:17.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:17.108+0000 7faacec00700 1 -- 172.21.15.33:0/2414447969 wait complete. 2024-05-03T18:21:17.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:17.109+0000 7faacec00700 1 Processor -- start 2024-05-03T18:21:17.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:17.109+0000 7faacec00700 1 -- start start 2024-05-03T18:21:17.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:17.110+0000 7faacec00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faac806b180 0x7faac81a1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:21:17.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:17.110+0000 7faacec00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faac806bb40 0x7faac81a1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:21:17.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:17.110+0000 7faacec00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faac8109510 0x7faac819bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:21:17.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:17.110+0000 7faacec00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7faac8113520 con 0x7faac8109510 2024-05-03T18:21:17.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:17.110+0000 7faacec00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7faac81133a0 con 0x7faac806b180 2024-05-03T18:21:17.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:17.110+0000 7faacec00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7faac81136a0 con 0x7faac806bb40 2024-05-03T18:21:17.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:17.110+0000 7faaccc00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faac8109510 0x7faac819bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:21:17.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:17.110+0000 7faaccc00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faac8109510 0x7faac819bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52550/0 (socket says 172.21.15.33:52550) 2024-05-03T18:21:17.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:17.110+0000 7faaccc00700 1 -- 172.21.15.33:0/12362352 learned_addr learned my addr 172.21.15.33:0/12362352 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:21:17.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:17.110+0000 7faac7e00700 1 --2- 172.21.15.33:0/12362352 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faac806b180 0x7faac81a1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:21:17.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:17.110+0000 7faabf400700 1 --2- 172.21.15.33:0/12362352 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faac806bb40 0x7faac81a1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:21:17.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:17.110+0000 7faaccc00700 1 -- 172.21.15.33:0/12362352 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faac806bb40 msgr2=0x7faac81a1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:21:17.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:17.111+0000 7faaccc00700 1 --2- 172.21.15.33:0/12362352 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faac806bb40 0x7faac81a1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:21:17.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:17.111+0000 7faaccc00700 1 -- 172.21.15.33:0/12362352 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faac806b180 msgr2=0x7faac81a1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:21:17.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:17.111+0000 7faaccc00700 1 --2- 172.21.15.33:0/12362352 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faac806b180 0x7faac81a1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:21:17.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:17.111+0000 7faaccc00700 1 -- 172.21.15.33:0/12362352 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faac819c260 con 0x7faac8109510 2024-05-03T18:21:17.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:17.111+0000 7faabf400700 1 --2- 172.21.15.33:0/12362352 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faac806bb40 0x7faac81a1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:21:17.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:17.111+0000 7faaccc00700 1 --2- 172.21.15.33:0/12362352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faac8109510 0x7faac819bb00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7faac0005910 tx=0x7faac00057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:21:17.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:17.111+0000 7faac5600700 1 -- 172.21.15.33:0/12362352 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac0027070 con 0x7faac8109510 2024-05-03T18:21:17.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:17.111+0000 7faac5600700 1 -- 172.21.15.33:0/12362352 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faac0022470 con 0x7faac8109510 2024-05-03T18:21:17.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:17.111+0000 7faacec00700 1 -- 172.21.15.33:0/12362352 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faac819c4f0 con 0x7faac8109510 2024-05-03T18:21:17.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:17.111+0000 7faac5600700 1 -- 172.21.15.33:0/12362352 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac0017650 con 0x7faac8109510 2024-05-03T18:21:17.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:17.111+0000 7faacec00700 1 -- 172.21.15.33:0/12362352 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faac81a8810 con 0x7faac8109510 2024-05-03T18:21:17.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:17.113+0000 7faacec00700 1 -- 172.21.15.33:0/12362352 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faac806a190 con 0x7faac8109510 2024-05-03T18:21:17.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:17.115+0000 7faac5600700 1 -- 172.21.15.33:0/12362352 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7faac002a070 con 0x7faac8109510 2024-05-03T18:21:17.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:17.116+0000 7faac5600700 1 --2- 172.21.15.33:0/12362352 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7faaac0b2880 0x7faaac0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:21:17.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:17.117+0000 7faac5600700 1 -- 172.21.15.33:0/12362352 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7faac00e2f70 con 0x7faac8109510 2024-05-03T18:21:17.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:17.117+0000 7faac7e00700 1 --2- 172.21.15.33:0/12362352 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7faaac0b2880 0x7faaac0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:21:17.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:17.118+0000 7faac7e00700 1 --2- 172.21.15.33:0/12362352 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7faaac0b2880 0x7faaac0b4d40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7faab8009550 tx=0x7faab80091b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:21:17.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:17.119+0000 7faac5600700 1 -- 172.21.15.33:0/12362352 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7faac0031080 con 0x7faac8109510 2024-05-03T18:21:17.383 INFO:journalctl@ceph.mds.l.smithi150.stdout:May 03 18:21:17 smithi150 podman[62571]: 2024-05-03 18:21:17.026013543 +0000 UTC m=+0.449028164 container start 14f4381f26949366e4605c59850eb394b5053417ed59d9ec0dee92482c658472 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-l, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240430, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-05-03T18:21:17.383 INFO:journalctl@ceph.mds.l.smithi150.stdout:May 03 18:21:17 smithi150 ceph-mds[62589]: -- 172.21.15.150:0/1521907641 <== mon.1 v2:172.21.15.150:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55afb7d8da40 con 0x55afb7d61000 2024-05-03T18:21:17.383 INFO:journalctl@ceph.mds.l.smithi150.stdout:May 03 18:21:17 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-l[62585]: starting mds.l at 2024-05-03T18:21:17.383 INFO:journalctl@ceph.mds.l.smithi150.stdout:May 03 18:21:17 smithi150 bash[62571]: 14f4381f26949366e4605c59850eb394b5053417ed59d9ec0dee92482c658472 2024-05-03T18:21:17.383 INFO:journalctl@ceph.mds.l.smithi150.stdout:May 03 18:21:17 smithi150 systemd[1]: Started Ceph mds.l for 5703bba4-0978-11ef-bc97-c7b262605968. 2024-05-03T18:21:17.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:17.411+0000 7faacec00700 1 -- 172.21.15.33:0/12362352 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faac81a89e0 con 0x7faac8109510 2024-05-03T18:21:17.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:17.412+0000 7faac5600700 1 -- 172.21.15.33:0/12362352 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1393 (secure 0 0 0) 0x7faac0013070 con 0x7faac8109510 2024-05-03T18:21:17.412 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:21:17.412 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/203598909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":203598909},{"type":"v1","addr":"172.21.15.150:6833","nonce":203598909}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-03T18:21:17.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:17.415+0000 7faacec00700 1 -- 172.21.15.33:0/12362352 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7faaac0b2880 msgr2=0x7faaac0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:21:17.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:17.415+0000 7faacec00700 1 --2- 172.21.15.33:0/12362352 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7faaac0b2880 0x7faaac0b4d40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7faab8009550 tx=0x7faab80091b0 comp rx=0 tx=0).stop 2024-05-03T18:21:17.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:17.415+0000 7faacec00700 1 -- 172.21.15.33:0/12362352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faac8109510 msgr2=0x7faac819bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:21:17.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:17.415+0000 7faacec00700 1 --2- 172.21.15.33:0/12362352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faac8109510 0x7faac819bb00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7faac0005910 tx=0x7faac00057c0 comp rx=0 tx=0).stop 2024-05-03T18:21:17.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:17.415+0000 7faacec00700 1 -- 172.21.15.33:0/12362352 shutdown_connections 2024-05-03T18:21:17.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:17.415+0000 7faacec00700 1 --2- 172.21.15.33:0/12362352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faac8109510 0x7faac819bb00 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:21:17.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:17.415+0000 7faacec00700 1 --2- 172.21.15.33:0/12362352 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7faaac0b2880 0x7faaac0b4d40 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:21:17.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:17.415+0000 7faacec00700 1 --2- 172.21.15.33:0/12362352 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faac806b180 0x7faac81a1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:21:17.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:17.415+0000 7faacec00700 1 --2- 172.21.15.33:0/12362352 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faac806bb40 0x7faac81a1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:21:17.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:17.416+0000 7faacec00700 1 -- 172.21.15.33:0/12362352 >> 172.21.15.33:0/12362352 conn(0x7faac8079350 msgr2=0x7faac810e660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:21:17.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:17.416+0000 7faacec00700 1 -- 172.21.15.33:0/12362352 shutdown_connections 2024-05-03T18:21:17.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:17.416+0000 7faacec00700 1 -- 172.21.15.33:0/12362352 wait complete. 2024-05-03T18:21:17.417 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 2 2024-05-03T18:21:18.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:18 smithi150 ceph-mon[32492]: mds.? [v2:172.21.15.150:6832/203598909,v1:172.21.15.150:6833/203598909] up:boot 2024-05-03T18:21:18.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:18 smithi150 ceph-mon[32492]: fsmap 1 up:standby 2024-05-03T18:21:18.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:18 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-03T18:21:18.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:18 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:18.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:18 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:18.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:18 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:18.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:18 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T18:21:18.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:18 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T18:21:18.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:18 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:21:18.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:18 smithi150 ceph-mon[32492]: Deploying daemon mds.k on smithi099 2024-05-03T18:21:18.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:18 smithi150 ceph-mon[32492]: pgmap v330: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:21:18.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:18 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/12362352' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:21:18.479 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:18 smithi099 ceph-mon[31658]: mds.? [v2:172.21.15.150:6832/203598909,v1:172.21.15.150:6833/203598909] up:boot 2024-05-03T18:21:18.480 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:18 smithi099 ceph-mon[31658]: fsmap 1 up:standby 2024-05-03T18:21:18.480 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:18 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-03T18:21:18.480 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:18 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:18.480 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:18 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:18.480 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:18 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:18.480 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:18 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T18:21:18.480 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:18 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T18:21:18.480 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:18 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:21:18.480 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:18 smithi099 ceph-mon[31658]: Deploying daemon mds.k on smithi099 2024-05-03T18:21:18.480 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:18 smithi099 ceph-mon[31658]: pgmap v330: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:21:18.480 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:18 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/12362352' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:21:18.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:18 smithi033 ceph-mon[28330]: mds.? [v2:172.21.15.150:6832/203598909,v1:172.21.15.150:6833/203598909] up:boot 2024-05-03T18:21:18.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:18 smithi033 ceph-mon[28330]: fsmap 1 up:standby 2024-05-03T18:21:18.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:18 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-03T18:21:18.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:18 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:18.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:18 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:18.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:18 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:18.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:18 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T18:21:18.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:18 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T18:21:18.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:18 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:21:18.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:18 smithi033 ceph-mon[28330]: Deploying daemon mds.k on smithi099 2024-05-03T18:21:18.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:18 smithi033 ceph-mon[28330]: pgmap v330: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:21:18.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:18 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/12362352' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:21:20.225 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T18:21:20.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:20 smithi150 ceph-mon[32492]: pgmap v331: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:21:20.709 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:21:20.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:20 smithi033 ceph-mon[28330]: pgmap v331: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:21:20.745 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:20 smithi099 ceph-mon[31658]: pgmap v331: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:21:21.000 INFO:journalctl@ceph.mds.k.smithi099.stdout:May 03 18:21:20 smithi099 podman[61051]: 2024-05-03 18:21:20.815485626 +0000 UTC m=+0.154137786 container create d1fe17b0ec713c4b2e822c2725a8cbf48f010058e8ee785d5be0decd7430fb84 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-k, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-05-03T18:21:21.545 INFO:journalctl@ceph.mds.k.smithi099.stdout:May 03 18:21:21 smithi099 podman[61051]: 2024-05-03 18:21:21.103643354 +0000 UTC m=+0.442295555 container init d1fe17b0ec713c4b2e822c2725a8cbf48f010058e8ee785d5be0decd7430fb84 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-k, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-05-03T18:21:21.545 INFO:journalctl@ceph.mds.k.smithi099.stdout:May 03 18:21:21 smithi099 podman[61051]: 2024-05-03 18:21:21.114164811 +0000 UTC m=+0.452816982 container start d1fe17b0ec713c4b2e822c2725a8cbf48f010058e8ee785d5be0decd7430fb84 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-k, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.schema-version=1.0) 2024-05-03T18:21:21.545 INFO:journalctl@ceph.mds.k.smithi099.stdout:May 03 18:21:21 smithi099 ceph-mds[61077]: -- 172.21.15.99:0/3832652292 <== mon.2 v2:172.21.15.99:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564efdbaba40 con 0x564efdb7e800 2024-05-03T18:21:21.545 INFO:journalctl@ceph.mds.k.smithi099.stdout:May 03 18:21:21 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-k[61071]: starting mds.k at 2024-05-03T18:21:21.546 INFO:journalctl@ceph.mds.k.smithi099.stdout:May 03 18:21:21 smithi099 bash[61051]: d1fe17b0ec713c4b2e822c2725a8cbf48f010058e8ee785d5be0decd7430fb84 2024-05-03T18:21:21.546 INFO:journalctl@ceph.mds.k.smithi099.stdout:May 03 18:21:21 smithi099 systemd[1]: Started Ceph mds.k for 5703bba4-0978-11ef-bc97-c7b262605968. 2024-05-03T18:21:22.402 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:22 smithi033 ceph-mon[28330]: pgmap v332: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:21:22.402 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:22 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:22.402 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:22 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:22.402 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:22 smithi033 ceph-mon[28330]: mds.? [v2:172.21.15.99:6832/1865105479,v1:172.21.15.99:6833/1865105479] up:boot 2024-05-03T18:21:22.402 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:22 smithi033 ceph-mon[28330]: fsmap 2 up:standby 2024-05-03T18:21:22.402 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:22 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-03T18:21:22.403 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:22 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:22.403 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:22 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T18:21:22.403 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:22 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T18:21:22.403 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:22 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:21:22.403 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:22 smithi033 ceph-mon[28330]: Deploying daemon mds.j on smithi033 2024-05-03T18:21:22.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:22 smithi150 ceph-mon[32492]: pgmap v332: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:21:22.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:22 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:22.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:22 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:22.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:22 smithi150 ceph-mon[32492]: mds.? [v2:172.21.15.99:6832/1865105479,v1:172.21.15.99:6833/1865105479] up:boot 2024-05-03T18:21:22.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:22 smithi150 ceph-mon[32492]: fsmap 2 up:standby 2024-05-03T18:21:22.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:22 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-03T18:21:22.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:22 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:22.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:22 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T18:21:22.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:22 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T18:21:22.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:22 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:21:22.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:22 smithi150 ceph-mon[32492]: Deploying daemon mds.j on smithi033 2024-05-03T18:21:22.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:22 smithi099 ceph-mon[31658]: pgmap v332: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:21:22.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:22 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:22.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:22 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:22.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:22 smithi099 ceph-mon[31658]: mds.? [v2:172.21.15.99:6832/1865105479,v1:172.21.15.99:6833/1865105479] up:boot 2024-05-03T18:21:22.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:22 smithi099 ceph-mon[31658]: fsmap 2 up:standby 2024-05-03T18:21:22.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:22 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-03T18:21:22.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:22 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:22.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:22 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T18:21:22.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:22 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T18:21:22.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:22 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:21:22.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:22 smithi099 ceph-mon[31658]: Deploying daemon mds.j on smithi033 2024-05-03T18:21:22.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:22.942+0000 7f3baf200700 1 -- 172.21.15.33:0/4257553333 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3bb00761f0 msgr2=0x7f3bb010f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:21:22.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:22.942+0000 7f3baf200700 1 --2- 172.21.15.33:0/4257553333 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3bb00761f0 0x7f3bb010f6b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c009990 tx=0x7f3b9c009ca0 comp rx=0 tx=0).stop 2024-05-03T18:21:22.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:22.942+0000 7f3baf200700 1 -- 172.21.15.33:0/4257553333 shutdown_connections 2024-05-03T18:21:22.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:22.942+0000 7f3baf200700 1 --2- 172.21.15.33:0/4257553333 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3bb010fbf0 0x7f3bb0112060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:21:22.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:22.942+0000 7f3baf200700 1 --2- 172.21.15.33:0/4257553333 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3bb00758d0 0x7f3bb0075cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:21:22.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:22.942+0000 7f3baf200700 1 --2- 172.21.15.33:0/4257553333 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3bb00761f0 0x7f3bb010f6b0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:21:22.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:22.942+0000 7f3baf200700 1 -- 172.21.15.33:0/4257553333 >> 172.21.15.33:0/4257553333 conn(0x7f3bb00feec0 msgr2=0x7f3bb01012e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:21:22.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:22.942+0000 7f3baf200700 1 -- 172.21.15.33:0/4257553333 shutdown_connections 2024-05-03T18:21:22.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:22.942+0000 7f3baf200700 1 -- 172.21.15.33:0/4257553333 wait complete. 2024-05-03T18:21:22.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:22.943+0000 7f3baf200700 1 Processor -- start 2024-05-03T18:21:22.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:22.944+0000 7f3baf200700 1 -- start start 2024-05-03T18:21:22.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:22.944+0000 7f3baf200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3bb00758d0 0x7f3bb01a1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:21:22.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:22.944+0000 7f3baf200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3bb00761f0 0x7f3bb01a1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:21:22.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:22.944+0000 7f3baf200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3bb010fbf0 0x7f3bb019ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:21:22.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:22.944+0000 7f3baf200700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb0115670 con 0x7f3bb00758d0 2024-05-03T18:21:22.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:22.944+0000 7f3baf200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb01154f0 con 0x7f3bb010fbf0 2024-05-03T18:21:22.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:22.944+0000 7f3baf200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb01157f0 con 0x7f3bb00761f0 2024-05-03T18:21:22.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:22.944+0000 7f3bade00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3bb00758d0 0x7f3bb01a1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:21:22.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:22.944+0000 7f3bade00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3bb00758d0 0x7f3bb01a1850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52570/0 (socket says 172.21.15.33:52570) 2024-05-03T18:21:22.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:22.944+0000 7f3bade00700 1 -- 172.21.15.33:0/3601730867 learned_addr learned my addr 172.21.15.33:0/3601730867 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:21:22.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:22.945+0000 7f3bad400700 1 --2- 172.21.15.33:0/3601730867 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3bb00761f0 0x7f3bb01a1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:21:22.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:22.945+0000 7f3bae800700 1 --2- 172.21.15.33:0/3601730867 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3bb010fbf0 0x7f3bb019ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:21:22.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:22.945+0000 7f3bade00700 1 -- 172.21.15.33:0/3601730867 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3bb00761f0 msgr2=0x7f3bb01a1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:21:22.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:22.945+0000 7f3bade00700 1 --2- 172.21.15.33:0/3601730867 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3bb00761f0 0x7f3bb01a1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:21:22.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:22.945+0000 7f3bade00700 1 -- 172.21.15.33:0/3601730867 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3bb010fbf0 msgr2=0x7f3bb019ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:21:22.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:22.945+0000 7f3bade00700 1 --2- 172.21.15.33:0/3601730867 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3bb010fbf0 0x7f3bb019ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:21:22.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:22.945+0000 7f3bade00700 1 -- 172.21.15.33:0/3601730867 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bb019c170 con 0x7f3bb00758d0 2024-05-03T18:21:22.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:22.945+0000 7f3bade00700 1 --2- 172.21.15.33:0/3601730867 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3bb00758d0 0x7f3bb01a1850 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3ba000c920 tx=0x7f3ba000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:21:22.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:22.946+0000 7f3ba6a00700 1 -- 172.21.15.33:0/3601730867 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba0007b00 con 0x7f3bb00758d0 2024-05-03T18:21:22.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:22.946+0000 7f3ba6a00700 1 -- 172.21.15.33:0/3601730867 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3ba0004cc0 con 0x7f3bb00758d0 2024-05-03T18:21:22.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:22.946+0000 7f3ba6a00700 1 -- 172.21.15.33:0/3601730867 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba00054f0 con 0x7f3bb00758d0 2024-05-03T18:21:22.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:22.946+0000 7f3baf200700 1 -- 172.21.15.33:0/3601730867 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bb019c460 con 0x7f3bb00758d0 2024-05-03T18:21:22.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:22.946+0000 7f3baf200700 1 -- 172.21.15.33:0/3601730867 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bb01a8720 con 0x7f3bb00758d0 2024-05-03T18:21:22.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:22.947+0000 7f3baf200700 1 -- 172.21.15.33:0/3601730867 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bb006a190 con 0x7f3bb00758d0 2024-05-03T18:21:22.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:22.950+0000 7f3ba6a00700 1 -- 172.21.15.33:0/3601730867 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f3ba0007ca0 con 0x7f3bb00758d0 2024-05-03T18:21:22.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:22.951+0000 7f3ba6a00700 1 --2- 172.21.15.33:0/3601730867 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f3b940b2900 0x7f3b940b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:21:22.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:22.951+0000 7f3ba6a00700 1 -- 172.21.15.33:0/3601730867 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f3ba00e1db0 con 0x7f3bb00758d0 2024-05-03T18:21:22.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:22.952+0000 7f3bad400700 1 --2- 172.21.15.33:0/3601730867 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f3b940b2900 0x7f3b940b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:21:22.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:22.952+0000 7f3bad400700 1 --2- 172.21.15.33:0/3601730867 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f3b940b2900 0x7f3b940b4dc0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c00b5c0 tx=0x7f3b9c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:21:22.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:22.954+0000 7f3ba6a00700 1 -- 172.21.15.33:0/3601730867 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f3ba0014030 con 0x7f3bb00758d0 2024-05-03T18:21:23.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:23.248+0000 7f3baf200700 1 -- 172.21.15.33:0/3601730867 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3bb019cf40 con 0x7f3bb00758d0 2024-05-03T18:21:23.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:23.249+0000 7f3ba6a00700 1 -- 172.21.15.33:0/3601730867 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2218 (secure 0 0 0) 0x7f3ba00a9f50 con 0x7f3bb00758d0 2024-05-03T18:21:23.248 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:21:23.249 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/203598909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":203598909},{"type":"v1","addr":"172.21.15.150:6833","nonce":203598909}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":34418,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1865105479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1865105479},{"type":"v1","addr":"172.21.15.99:6833","nonce":1865105479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-05-03T18:21:23.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:23.253+0000 7f3baf200700 1 -- 172.21.15.33:0/3601730867 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f3b940b2900 msgr2=0x7f3b940b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:21:23.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:23.253+0000 7f3baf200700 1 --2- 172.21.15.33:0/3601730867 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f3b940b2900 0x7f3b940b4dc0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c00b5c0 tx=0x7f3b9c01a040 comp rx=0 tx=0).stop 2024-05-03T18:21:23.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:23.253+0000 7f3baf200700 1 -- 172.21.15.33:0/3601730867 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3bb00758d0 msgr2=0x7f3bb01a1850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:21:23.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:23.253+0000 7f3baf200700 1 --2- 172.21.15.33:0/3601730867 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3bb00758d0 0x7f3bb01a1850 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3ba000c920 tx=0x7f3ba000cce0 comp rx=0 tx=0).stop 2024-05-03T18:21:23.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:23.253+0000 7f3baf200700 1 -- 172.21.15.33:0/3601730867 shutdown_connections 2024-05-03T18:21:23.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:23.253+0000 7f3baf200700 1 --2- 172.21.15.33:0/3601730867 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f3b940b2900 0x7f3b940b4dc0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:21:23.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:23.253+0000 7f3baf200700 1 --2- 172.21.15.33:0/3601730867 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3bb010fbf0 0x7f3bb019ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:21:23.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:23.253+0000 7f3baf200700 1 --2- 172.21.15.33:0/3601730867 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3bb00758d0 0x7f3bb01a1850 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:21:23.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:23.253+0000 7f3baf200700 1 --2- 172.21.15.33:0/3601730867 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3bb00761f0 0x7f3bb01a1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:21:23.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:23.253+0000 7f3baf200700 1 -- 172.21.15.33:0/3601730867 >> 172.21.15.33:0/3601730867 conn(0x7f3bb00feec0 msgr2=0x7f3bb00ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:21:23.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:23.253+0000 7f3baf200700 1 -- 172.21.15.33:0/3601730867 shutdown_connections 2024-05-03T18:21:23.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:23.253+0000 7f3baf200700 1 -- 172.21.15.33:0/3601730867 wait complete. 2024-05-03T18:21:23.255 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 3 2024-05-03T18:21:23.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:23 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/3601730867' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:21:23.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:23 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/3601730867' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:21:23.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:23 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/3601730867' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:21:24.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:24 smithi150 ceph-mon[32492]: pgmap v333: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:21:24.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:24 smithi033 ceph-mon[28330]: pgmap v333: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:21:24.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:24 smithi099 ceph-mon[31658]: pgmap v333: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:21:25.985 INFO:journalctl@ceph.mds.j.smithi033.stdout:May 03 18:21:25 smithi033 systemd[1]: Starting Ceph mds.j for 5703bba4-0978-11ef-bc97-c7b262605968... 2024-05-03T18:21:26.333 INFO:journalctl@ceph.mds.j.smithi033.stdout:May 03 18:21:25 smithi033 podman[68887]: 2024-05-03 18:21:25.892681218 +0000 UTC m=+0.034604053 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T18:21:26.333 INFO:journalctl@ceph.mds.j.smithi033.stdout:May 03 18:21:26 smithi033 podman[68887]: 2024-05-03 18:21:26.026385969 +0000 UTC m=+0.168308842 container create 85f42a639f4287f6a7d09f2ffac894fd02536cdf06f89e7cefaca1c3f22ef141 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-j, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS) 2024-05-03T18:21:26.336 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T18:21:26.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:26 smithi150 ceph-mon[32492]: pgmap v334: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:21:26.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:26 smithi033 ceph-mon[28330]: pgmap v334: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:21:26.734 INFO:journalctl@ceph.mds.j.smithi033.stdout:May 03 18:21:26 smithi033 podman[68887]: 2024-05-03 18:21:26.356120494 +0000 UTC m=+0.498043371 container init 85f42a639f4287f6a7d09f2ffac894fd02536cdf06f89e7cefaca1c3f22ef141 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-j, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-03T18:21:26.734 INFO:journalctl@ceph.mds.j.smithi033.stdout:May 03 18:21:26 smithi033 podman[68887]: 2024-05-03 18:21:26.365714528 +0000 UTC m=+0.507637364 container start 85f42a639f4287f6a7d09f2ffac894fd02536cdf06f89e7cefaca1c3f22ef141 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-05-03T18:21:26.734 INFO:journalctl@ceph.mds.j.smithi033.stdout:May 03 18:21:26 smithi033 ceph-mds[68912]: -- 172.21.15.33:0/2054987997 <== mon.0 v2:172.21.15.33:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5601fc5a1a40 con 0x5601fc575000 2024-05-03T18:21:26.734 INFO:journalctl@ceph.mds.j.smithi033.stdout:May 03 18:21:26 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-j[68901]: starting mds.j at 2024-05-03T18:21:26.734 INFO:journalctl@ceph.mds.j.smithi033.stdout:May 03 18:21:26 smithi033 bash[68887]: 85f42a639f4287f6a7d09f2ffac894fd02536cdf06f89e7cefaca1c3f22ef141 2024-05-03T18:21:26.734 INFO:journalctl@ceph.mds.j.smithi033.stdout:May 03 18:21:26 smithi033 systemd[1]: Started Ceph mds.j for 5703bba4-0978-11ef-bc97-c7b262605968. 2024-05-03T18:21:26.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:26 smithi099 ceph-mon[31658]: pgmap v334: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:21:26.803 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:21:27.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:27 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:27.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:27 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:27.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:27 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:27.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:27 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T18:21:27.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:27 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T18:21:27.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:27 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:21:27.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:27 smithi150 ceph-mon[32492]: Deploying daemon mds.d on smithi033 2024-05-03T18:21:27.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:27 smithi150 ceph-mon[32492]: mds.? [v2:172.21.15.33:6834/1327393503,v1:172.21.15.33:6835/1327393503] up:boot 2024-05-03T18:21:27.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:27 smithi150 ceph-mon[32492]: fsmap 3 up:standby 2024-05-03T18:21:27.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:27 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-03T18:21:27.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:27 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:27.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:27 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:27.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:27 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:27.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:27 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T18:21:27.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:27 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T18:21:27.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:27 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:21:27.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:27 smithi033 ceph-mon[28330]: Deploying daemon mds.d on smithi033 2024-05-03T18:21:27.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:27 smithi033 ceph-mon[28330]: mds.? [v2:172.21.15.33:6834/1327393503,v1:172.21.15.33:6835/1327393503] up:boot 2024-05-03T18:21:27.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:27 smithi033 ceph-mon[28330]: fsmap 3 up:standby 2024-05-03T18:21:27.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:27 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-03T18:21:28.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:27 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:28.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:27 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:28.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:27 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:28.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:27 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T18:21:28.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:27 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T18:21:28.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:27 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:21:28.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:27 smithi099 ceph-mon[31658]: Deploying daemon mds.d on smithi033 2024-05-03T18:21:28.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:27 smithi099 ceph-mon[31658]: mds.? [v2:172.21.15.33:6834/1327393503,v1:172.21.15.33:6835/1327393503] up:boot 2024-05-03T18:21:28.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:27 smithi099 ceph-mon[31658]: fsmap 3 up:standby 2024-05-03T18:21:28.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:27 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-03T18:21:28.657 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:28 smithi033 ceph-mon[28330]: pgmap v335: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:21:28.881 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:28 smithi150 ceph-mon[32492]: pgmap v335: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:21:29.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:28 smithi099 ceph-mon[31658]: pgmap v335: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:21:29.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:29.159+0000 7fab5d800700 1 -- 172.21.15.33:0/2515731399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab5810fb60 msgr2=0x7fab5810ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:21:29.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:29.159+0000 7fab5d800700 1 --2- 172.21.15.33:0/2515731399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab5810fb60 0x7fab5810ff40 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fab48009990 tx=0x7fab48009ca0 comp rx=0 tx=0).stop 2024-05-03T18:21:29.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:29.159+0000 7fab5d800700 1 -- 172.21.15.33:0/2515731399 shutdown_connections 2024-05-03T18:21:29.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:29.159+0000 7fab5d800700 1 --2- 172.21.15.33:0/2515731399 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab5810a490 0x7fab581169a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:21:29.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:29.159+0000 7fab5d800700 1 --2- 172.21.15.33:0/2515731399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab5810fb60 0x7fab5810ff40 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:21:29.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:29.159+0000 7fab5d800700 1 --2- 172.21.15.33:0/2515731399 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab58109ad0 0x7fab58109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:21:29.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:29.159+0000 7fab5d800700 1 -- 172.21.15.33:0/2515731399 >> 172.21.15.33:0/2515731399 conn(0x7fab58076c70 msgr2=0x7fab581082c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:21:29.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:29.160+0000 7fab5d800700 1 -- 172.21.15.33:0/2515731399 shutdown_connections 2024-05-03T18:21:29.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:29.160+0000 7fab5d800700 1 -- 172.21.15.33:0/2515731399 wait complete. 2024-05-03T18:21:29.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:29.161+0000 7fab5d800700 1 Processor -- start 2024-05-03T18:21:29.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:29.161+0000 7fab5d800700 1 -- start start 2024-05-03T18:21:29.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:29.161+0000 7fab5d800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab58109ad0 0x7fab581a0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:21:29.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:29.161+0000 7fab5d800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab5810a490 0x7fab581a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:21:29.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:29.161+0000 7fab5d800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab5810fb60 0x7fab581a4980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:21:29.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:29.161+0000 7fab5d800700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fab58119bb0 con 0x7fab5810fb60 2024-05-03T18:21:29.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:29.161+0000 7fab5d800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fab58119a30 con 0x7fab5810a490 2024-05-03T18:21:29.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:29.161+0000 7fab5d800700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fab58119d30 con 0x7fab58109ad0 2024-05-03T18:21:29.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:29.161+0000 7fab57400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab5810fb60 0x7fab581a4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:21:29.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:29.162+0000 7fab57400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab5810fb60 0x7fab581a4980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56874/0 (socket says 172.21.15.33:56874) 2024-05-03T18:21:29.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:29.162+0000 7fab57400700 1 -- 172.21.15.33:0/2165349514 learned_addr learned my addr 172.21.15.33:0/2165349514 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:21:29.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:29.162+0000 7fab56a00700 1 --2- 172.21.15.33:0/2165349514 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab58109ad0 0x7fab581a0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:21:29.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:29.162+0000 7fab4e000700 1 --2- 172.21.15.33:0/2165349514 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab5810a490 0x7fab581a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:21:29.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:29.162+0000 7fab57400700 1 -- 172.21.15.33:0/2165349514 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab58109ad0 msgr2=0x7fab581a0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:21:29.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:29.162+0000 7fab57400700 1 --2- 172.21.15.33:0/2165349514 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab58109ad0 0x7fab581a0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:21:29.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:29.162+0000 7fab57400700 1 -- 172.21.15.33:0/2165349514 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab5810a490 msgr2=0x7fab581a0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:21:29.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:29.162+0000 7fab57400700 1 --2- 172.21.15.33:0/2165349514 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab5810a490 0x7fab581a0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:21:29.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:29.162+0000 7fab57400700 1 -- 172.21.15.33:0/2165349514 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab581a50c0 con 0x7fab5810fb60 2024-05-03T18:21:29.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:29.162+0000 7fab57400700 1 --2- 172.21.15.33:0/2165349514 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab5810fb60 0x7fab581a4980 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fab50007ee0 tx=0x7fab5000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:21:29.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:29.163+0000 7fab4fe00700 1 -- 172.21.15.33:0/2165349514 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab5000dbd0 con 0x7fab5810fb60 2024-05-03T18:21:29.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:29.163+0000 7fab5d800700 1 -- 172.21.15.33:0/2165349514 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab581a53b0 con 0x7fab5810fb60 2024-05-03T18:21:29.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:29.163+0000 7fab4fe00700 1 -- 172.21.15.33:0/2165349514 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fab5000dd70 con 0x7fab5810fb60 2024-05-03T18:21:29.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:29.163+0000 7fab4fe00700 1 -- 172.21.15.33:0/2165349514 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab500058d0 con 0x7fab5810fb60 2024-05-03T18:21:29.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:29.163+0000 7fab5d800700 1 -- 172.21.15.33:0/2165349514 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab581acc30 con 0x7fab5810fb60 2024-05-03T18:21:29.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:29.164+0000 7fab5d800700 1 -- 172.21.15.33:0/2165349514 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab5806a190 con 0x7fab5810fb60 2024-05-03T18:21:29.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:29.166+0000 7fab4fe00700 1 -- 172.21.15.33:0/2165349514 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fab5001a020 con 0x7fab5810fb60 2024-05-03T18:21:29.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:29.168+0000 7fab4fe00700 1 --2- 172.21.15.33:0/2165349514 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fab3c0b2880 0x7fab3c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:21:29.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:29.168+0000 7fab4fe00700 1 -- 172.21.15.33:0/2165349514 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7fab500e1cd0 con 0x7fab5810fb60 2024-05-03T18:21:29.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:29.169+0000 7fab56a00700 1 --2- 172.21.15.33:0/2165349514 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fab3c0b2880 0x7fab3c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:21:29.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:29.169+0000 7fab56a00700 1 --2- 172.21.15.33:0/2165349514 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fab3c0b2880 0x7fab3c0b4d40 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fab4800b5c0 tx=0x7fab4801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:21:29.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:29.170+0000 7fab4fe00700 1 -- 172.21.15.33:0/2165349514 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fab50010070 con 0x7fab5810fb60 2024-05-03T18:21:29.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:29.465+0000 7fab5d800700 1 -- 172.21.15.33:0/2165349514 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fab5802e510 con 0x7fab5810fb60 2024-05-03T18:21:29.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:29.466+0000 7fab4fe00700 1 -- 172.21.15.33:0/2165349514 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3043 (secure 0 0 0) 0x7fab500b0e20 con 0x7fab5810fb60 2024-05-03T18:21:29.466 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:21:29.467 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1327393503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1327393503},{"type":"v1","addr":"172.21.15.33:6835","nonce":1327393503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/203598909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":203598909},{"type":"v1","addr":"172.21.15.150:6833","nonce":203598909}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":34418,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1865105479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1865105479},{"type":"v1","addr":"172.21.15.99:6833","nonce":1865105479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-05-03T18:21:29.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:29.469+0000 7fab5d800700 1 -- 172.21.15.33:0/2165349514 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fab3c0b2880 msgr2=0x7fab3c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:21:29.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:29.469+0000 7fab5d800700 1 --2- 172.21.15.33:0/2165349514 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fab3c0b2880 0x7fab3c0b4d40 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fab4800b5c0 tx=0x7fab4801a040 comp rx=0 tx=0).stop 2024-05-03T18:21:29.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:29.469+0000 7fab5d800700 1 -- 172.21.15.33:0/2165349514 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab5810fb60 msgr2=0x7fab581a4980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:21:29.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:29.469+0000 7fab5d800700 1 --2- 172.21.15.33:0/2165349514 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab5810fb60 0x7fab581a4980 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fab50007ee0 tx=0x7fab5000d3b0 comp rx=0 tx=0).stop 2024-05-03T18:21:29.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:29.470+0000 7fab5d800700 1 -- 172.21.15.33:0/2165349514 shutdown_connections 2024-05-03T18:21:29.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:29.470+0000 7fab5d800700 1 --2- 172.21.15.33:0/2165349514 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab5810fb60 0x7fab581a4980 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:21:29.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:29.470+0000 7fab5d800700 1 --2- 172.21.15.33:0/2165349514 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab58109ad0 0x7fab581a0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:21:29.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:29.470+0000 7fab5d800700 1 --2- 172.21.15.33:0/2165349514 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fab3c0b2880 0x7fab3c0b4d40 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:21:29.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:29.470+0000 7fab5d800700 1 --2- 172.21.15.33:0/2165349514 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab5810a490 0x7fab581a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:21:29.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:29.470+0000 7fab5d800700 1 -- 172.21.15.33:0/2165349514 >> 172.21.15.33:0/2165349514 conn(0x7fab58076c70 msgr2=0x7fab580772d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:21:29.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:29.470+0000 7fab5d800700 1 -- 172.21.15.33:0/2165349514 shutdown_connections 2024-05-03T18:21:29.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:29.470+0000 7fab5d800700 1 -- 172.21.15.33:0/2165349514 wait complete. 2024-05-03T18:21:29.472 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 4 2024-05-03T18:21:29.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:29 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/2165349514' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:21:29.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:29 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/2165349514' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:21:30.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:29 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/2165349514' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:21:30.744 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:30 smithi033 ceph-mon[28330]: pgmap v336: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:21:30.881 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:30 smithi150 ceph-mon[32492]: pgmap v336: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:21:31.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:30 smithi099 ceph-mon[31658]: pgmap v336: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:21:31.735 INFO:journalctl@ceph.mds.d.smithi033.stdout:May 03 18:21:31 smithi033 podman[69347]: 2024-05-03 18:21:31.499285783 +0000 UTC m=+0.181948760 container create 8821c4cab9ce53c050e666561c3fa9eace584d67264b0cfea3aa99a8310f73f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-d, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240501.193033-9a047fe) 2024-05-03T18:21:32.234 INFO:journalctl@ceph.mds.d.smithi033.stdout:May 03 18:21:31 smithi033 podman[69347]: 2024-05-03 18:21:31.778477287 +0000 UTC m=+0.461140251 container init 8821c4cab9ce53c050e666561c3fa9eace584d67264b0cfea3aa99a8310f73f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-d, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, io.buildah.version=1.33.5) 2024-05-03T18:21:32.234 INFO:journalctl@ceph.mds.d.smithi033.stdout:May 03 18:21:31 smithi033 podman[69347]: 2024-05-03 18:21:31.790551706 +0000 UTC m=+0.473214630 container start 8821c4cab9ce53c050e666561c3fa9eace584d67264b0cfea3aa99a8310f73f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-d, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-03T18:21:32.234 INFO:journalctl@ceph.mds.d.smithi033.stdout:May 03 18:21:31 smithi033 ceph-mds[69365]: -- 172.21.15.33:0/3695476718 <== mon.0 v2:172.21.15.33:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56226993ba40 con 0x56226990f000 2024-05-03T18:21:32.235 INFO:journalctl@ceph.mds.d.smithi033.stdout:May 03 18:21:31 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-d[69361]: starting mds.d at 2024-05-03T18:21:32.235 INFO:journalctl@ceph.mds.d.smithi033.stdout:May 03 18:21:31 smithi033 bash[69347]: 8821c4cab9ce53c050e666561c3fa9eace584d67264b0cfea3aa99a8310f73f1 2024-05-03T18:21:32.235 INFO:journalctl@ceph.mds.d.smithi033.stdout:May 03 18:21:31 smithi033 systemd[1]: Started Ceph mds.d for 5703bba4-0978-11ef-bc97-c7b262605968. 2024-05-03T18:21:32.825 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T18:21:32.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:32 smithi150 ceph-mon[32492]: pgmap v337: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:21:32.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:32 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:32.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:32 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:32.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:32 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:32.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:32 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T18:21:32.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:32 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T18:21:32.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:32 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:21:32.913 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:32 smithi033 ceph-mon[28330]: pgmap v337: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:21:32.914 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:32 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:32.914 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:32 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:32.914 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:32 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:32.914 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:32 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T18:21:32.914 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:32 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T18:21:32.914 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:32 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:21:33.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:32 smithi099 ceph-mon[31658]: pgmap v337: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:21:33.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:32 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:33.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:32 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:33.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:32 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:33.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:32 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T18:21:33.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:32 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T18:21:33.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:32 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:21:33.312 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:21:33.716 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:33 smithi033 ceph-mon[28330]: Deploying daemon mds.e on smithi099 2024-05-03T18:21:33.716 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:33 smithi033 ceph-mon[28330]: mds.? [v2:172.21.15.33:6836/2461472948,v1:172.21.15.33:6837/2461472948] up:boot 2024-05-03T18:21:33.716 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:33 smithi033 ceph-mon[28330]: fsmap 4 up:standby 2024-05-03T18:21:33.716 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:33 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-03T18:21:33.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:33 smithi099 ceph-mon[31658]: Deploying daemon mds.e on smithi099 2024-05-03T18:21:33.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:33 smithi099 ceph-mon[31658]: mds.? [v2:172.21.15.33:6836/2461472948,v1:172.21.15.33:6837/2461472948] up:boot 2024-05-03T18:21:33.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:33 smithi099 ceph-mon[31658]: fsmap 4 up:standby 2024-05-03T18:21:33.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:33 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-03T18:21:34.079 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:33 smithi150 ceph-mon[32492]: Deploying daemon mds.e on smithi099 2024-05-03T18:21:34.079 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:33 smithi150 ceph-mon[32492]: mds.? [v2:172.21.15.33:6836/2461472948,v1:172.21.15.33:6837/2461472948] up:boot 2024-05-03T18:21:34.079 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:33 smithi150 ceph-mon[32492]: fsmap 4 up:standby 2024-05-03T18:21:34.079 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:33 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-03T18:21:34.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:34 smithi150 ceph-mon[32492]: pgmap v338: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:21:34.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:34 smithi033 ceph-mon[28330]: pgmap v338: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:21:35.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:34 smithi099 ceph-mon[31658]: pgmap v338: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:21:35.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:35.500+0000 7fee57a00700 1 -- 172.21.15.33:0/497620744 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee5010fb60 msgr2=0x7fee5010ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:21:35.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:35.500+0000 7fee57a00700 1 --2- 172.21.15.33:0/497620744 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee5010fb60 0x7fee5010ff40 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fee44009990 tx=0x7fee44009ca0 comp rx=0 tx=0).stop 2024-05-03T18:21:35.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:35.500+0000 7fee57a00700 1 -- 172.21.15.33:0/497620744 shutdown_connections 2024-05-03T18:21:35.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:35.500+0000 7fee57a00700 1 --2- 172.21.15.33:0/497620744 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee5010a490 0x7fee501169a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:21:35.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:35.500+0000 7fee57a00700 1 --2- 172.21.15.33:0/497620744 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee5010fb60 0x7fee5010ff40 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:21:35.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:35.501+0000 7fee57a00700 1 --2- 172.21.15.33:0/497620744 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fee50109ad0 0x7fee50109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:21:35.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:35.501+0000 7fee57a00700 1 -- 172.21.15.33:0/497620744 >> 172.21.15.33:0/497620744 conn(0x7fee50076c70 msgr2=0x7fee501082c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:21:35.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:35.501+0000 7fee57a00700 1 -- 172.21.15.33:0/497620744 shutdown_connections 2024-05-03T18:21:35.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:35.501+0000 7fee57a00700 1 -- 172.21.15.33:0/497620744 wait complete. 2024-05-03T18:21:35.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:35.502+0000 7fee57a00700 1 Processor -- start 2024-05-03T18:21:35.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:35.502+0000 7fee57a00700 1 -- start start 2024-05-03T18:21:35.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:35.502+0000 7fee57a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee50109ad0 0x7fee501a0120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:21:35.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:35.502+0000 7fee57a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fee5010a490 0x7fee501a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:21:35.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:35.502+0000 7fee57a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee5010fb60 0x7fee501a49f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:21:35.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:35.502+0000 7fee57a00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fee50119bb0 con 0x7fee5010fb60 2024-05-03T18:21:35.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:35.502+0000 7fee57a00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fee50119a30 con 0x7fee50109ad0 2024-05-03T18:21:35.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:35.502+0000 7fee57a00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fee50119d30 con 0x7fee5010a490 2024-05-03T18:21:35.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:35.503+0000 7fee55a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee5010fb60 0x7fee501a49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:21:35.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:35.503+0000 7fee55a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee5010fb60 0x7fee501a49f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56910/0 (socket says 172.21.15.33:56910) 2024-05-03T18:21:35.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:35.503+0000 7fee55a00700 1 -- 172.21.15.33:0/1671135193 learned_addr learned my addr 172.21.15.33:0/1671135193 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:21:35.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:35.503+0000 7fee4fe00700 1 --2- 172.21.15.33:0/1671135193 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fee5010a490 0x7fee501a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:21:35.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:35.503+0000 7fee55000700 1 --2- 172.21.15.33:0/1671135193 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee50109ad0 0x7fee501a0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:21:35.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:35.503+0000 7fee55a00700 1 -- 172.21.15.33:0/1671135193 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fee5010a490 msgr2=0x7fee501a0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:21:35.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:35.503+0000 7fee55a00700 1 --2- 172.21.15.33:0/1671135193 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fee5010a490 0x7fee501a0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:21:35.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:35.503+0000 7fee55a00700 1 -- 172.21.15.33:0/1671135193 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee50109ad0 msgr2=0x7fee501a0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:21:35.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:35.503+0000 7fee55a00700 1 --2- 172.21.15.33:0/1671135193 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee50109ad0 0x7fee501a0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:21:35.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:35.503+0000 7fee55a00700 1 -- 172.21.15.33:0/1671135193 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee501a5130 con 0x7fee5010fb60 2024-05-03T18:21:35.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:35.504+0000 7fee55a00700 1 --2- 172.21.15.33:0/1671135193 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee5010fb60 0x7fee501a49f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fee4800c370 tx=0x7fee4800c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:21:35.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:35.504+0000 7fee4d600700 1 -- 172.21.15.33:0/1671135193 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee480090d0 con 0x7fee5010fb60 2024-05-03T18:21:35.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:35.504+0000 7fee4d600700 1 -- 172.21.15.33:0/1671135193 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fee48009270 con 0x7fee5010fb60 2024-05-03T18:21:35.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:35.504+0000 7fee57a00700 1 -- 172.21.15.33:0/1671135193 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee501a53c0 con 0x7fee5010fb60 2024-05-03T18:21:35.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:35.504+0000 7fee4d600700 1 -- 172.21.15.33:0/1671135193 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee48004790 con 0x7fee5010fb60 2024-05-03T18:21:35.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:35.504+0000 7fee57a00700 1 -- 172.21.15.33:0/1671135193 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee501acca0 con 0x7fee5010fb60 2024-05-03T18:21:35.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:35.505+0000 7fee57a00700 1 -- 172.21.15.33:0/1671135193 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee5006a190 con 0x7fee5010fb60 2024-05-03T18:21:35.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:35.508+0000 7fee4d600700 1 -- 172.21.15.33:0/1671135193 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fee4801a020 con 0x7fee5010fb60 2024-05-03T18:21:35.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:35.510+0000 7fee4d600700 1 --2- 172.21.15.33:0/1671135193 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fee380b2900 0x7fee380b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:21:35.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:35.510+0000 7fee4d600700 1 -- 172.21.15.33:0/1671135193 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7fee480e1520 con 0x7fee5010fb60 2024-05-03T18:21:35.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:35.510+0000 7fee55000700 1 --2- 172.21.15.33:0/1671135193 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fee380b2900 0x7fee380b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:21:35.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:35.511+0000 7fee55000700 1 --2- 172.21.15.33:0/1671135193 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fee380b2900 0x7fee380b4dc0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fee4400b2e0 tx=0x7fee4401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:21:35.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:35.512+0000 7fee4d600700 1 -- 172.21.15.33:0/1671135193 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fee48021060 con 0x7fee5010fb60 2024-05-03T18:21:35.517 INFO:journalctl@ceph.mds.e.smithi099.stdout:May 03 18:21:35 smithi099 systemd[1]: Starting Ceph mds.e for 5703bba4-0978-11ef-bc97-c7b262605968... 2024-05-03T18:21:35.793 INFO:journalctl@ceph.mds.e.smithi099.stdout:May 03 18:21:35 smithi099 podman[61363]: 2024-05-03 18:21:35.422563992 +0000 UTC m=+0.036734248 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T18:21:35.793 INFO:journalctl@ceph.mds.e.smithi099.stdout:May 03 18:21:35 smithi099 podman[61363]: 2024-05-03 18:21:35.537605953 +0000 UTC m=+0.151776232 container create 827d8341c3f940ba3d37850c1c5f770aec9dad2bba5082ec65679cac9d706b67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-e, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5) 2024-05-03T18:21:35.793 INFO:journalctl@ceph.mds.e.smithi099.stdout:May 03 18:21:35 smithi099 podman[61363]: 2024-05-03 18:21:35.791651057 +0000 UTC m=+0.405821349 container init 827d8341c3f940ba3d37850c1c5f770aec9dad2bba5082ec65679cac9d706b67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-e, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240430, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-05-03T18:21:35.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:35.802+0000 7fee57a00700 1 -- 172.21.15.33:0/1671135193 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fee5002e510 con 0x7fee5010fb60 2024-05-03T18:21:35.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:35.803+0000 7fee4d600700 1 -- 172.21.15.33:0/1671135193 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3868 (secure 0 0 0) 0x7fee48010040 con 0x7fee5010fb60 2024-05-03T18:21:35.803 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:21:35.803 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1327393503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1327393503},{"type":"v1","addr":"172.21.15.33:6835","nonce":1327393503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/2461472948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":2461472948},{"type":"v1","addr":"172.21.15.33:6837","nonce":2461472948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/203598909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":203598909},{"type":"v1","addr":"172.21.15.150:6833","nonce":203598909}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":34418,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1865105479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1865105479},{"type":"v1","addr":"172.21.15.99:6833","nonce":1865105479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-05-03T18:21:35.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:35.806+0000 7fee57a00700 1 -- 172.21.15.33:0/1671135193 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fee380b2900 msgr2=0x7fee380b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:21:35.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:35.806+0000 7fee57a00700 1 --2- 172.21.15.33:0/1671135193 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fee380b2900 0x7fee380b4dc0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fee4400b2e0 tx=0x7fee4401a040 comp rx=0 tx=0).stop 2024-05-03T18:21:35.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:35.806+0000 7fee57a00700 1 -- 172.21.15.33:0/1671135193 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee5010fb60 msgr2=0x7fee501a49f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:21:35.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:35.806+0000 7fee57a00700 1 --2- 172.21.15.33:0/1671135193 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee5010fb60 0x7fee501a49f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fee4800c370 tx=0x7fee4800c730 comp rx=0 tx=0).stop 2024-05-03T18:21:35.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:35.806+0000 7fee57a00700 1 -- 172.21.15.33:0/1671135193 shutdown_connections 2024-05-03T18:21:35.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:35.806+0000 7fee57a00700 1 --2- 172.21.15.33:0/1671135193 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee5010fb60 0x7fee501a49f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:21:35.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:35.806+0000 7fee57a00700 1 --2- 172.21.15.33:0/1671135193 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fee380b2900 0x7fee380b4dc0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:21:35.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:35.806+0000 7fee57a00700 1 --2- 172.21.15.33:0/1671135193 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee50109ad0 0x7fee501a0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:21:35.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:35.806+0000 7fee57a00700 1 --2- 172.21.15.33:0/1671135193 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fee5010a490 0x7fee501a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:21:35.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:35.806+0000 7fee57a00700 1 -- 172.21.15.33:0/1671135193 >> 172.21.15.33:0/1671135193 conn(0x7fee50076c70 msgr2=0x7fee500772d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:21:35.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:35.807+0000 7fee57a00700 1 -- 172.21.15.33:0/1671135193 shutdown_connections 2024-05-03T18:21:35.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:35.807+0000 7fee57a00700 1 -- 172.21.15.33:0/1671135193 wait complete. 2024-05-03T18:21:35.807 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 5 2024-05-03T18:21:36.045 INFO:journalctl@ceph.mds.e.smithi099.stdout:May 03 18:21:35 smithi099 podman[61363]: 2024-05-03 18:21:35.802952722 +0000 UTC m=+0.417123050 container start 827d8341c3f940ba3d37850c1c5f770aec9dad2bba5082ec65679cac9d706b67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-e, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-05-03T18:21:36.046 INFO:journalctl@ceph.mds.e.smithi099.stdout:May 03 18:21:35 smithi099 ceph-mds[61382]: -- 172.21.15.99:0/1767933679 <== mon.0 v2:172.21.15.33:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cc05f5ba40 con 0x55cc05f2ec00 2024-05-03T18:21:36.046 INFO:journalctl@ceph.mds.e.smithi099.stdout:May 03 18:21:35 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-e[61378]: starting mds.e at 2024-05-03T18:21:36.046 INFO:journalctl@ceph.mds.e.smithi099.stdout:May 03 18:21:35 smithi099 bash[61363]: 827d8341c3f940ba3d37850c1c5f770aec9dad2bba5082ec65679cac9d706b67 2024-05-03T18:21:36.046 INFO:journalctl@ceph.mds.e.smithi099.stdout:May 03 18:21:35 smithi099 systemd[1]: Started Ceph mds.e for 5703bba4-0978-11ef-bc97-c7b262605968. 2024-05-03T18:21:36.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:36 smithi033 ceph-mon[28330]: pgmap v339: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:21:36.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:36 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/1671135193' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:21:36.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:36 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:36.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:36 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:36.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:36 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:36.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:36 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T18:21:36.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:36 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T18:21:36.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:36 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:21:37.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:36 smithi099 ceph-mon[31658]: pgmap v339: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:21:37.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:36 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/1671135193' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:21:37.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:36 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:37.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:36 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:37.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:36 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:37.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:36 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T18:21:37.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:36 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T18:21:37.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:36 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:21:37.072 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:36 smithi150 ceph-mon[32492]: pgmap v339: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:21:37.072 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:36 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/1671135193' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:21:37.072 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:36 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:37.072 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:36 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:37.072 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:36 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:37.072 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:36 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T18:21:37.072 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:36 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T18:21:37.072 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:36 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:21:37.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:37 smithi033 ceph-mon[28330]: Deploying daemon mds.f on smithi150 2024-05-03T18:21:37.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:37 smithi033 ceph-mon[28330]: mds.? [v2:172.21.15.99:6834/1206393914,v1:172.21.15.99:6835/1206393914] up:boot 2024-05-03T18:21:37.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:37 smithi033 ceph-mon[28330]: fsmap 5 up:standby 2024-05-03T18:21:37.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:37 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-03T18:21:38.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:37 smithi099 ceph-mon[31658]: Deploying daemon mds.f on smithi150 2024-05-03T18:21:38.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:37 smithi099 ceph-mon[31658]: mds.? [v2:172.21.15.99:6834/1206393914,v1:172.21.15.99:6835/1206393914] up:boot 2024-05-03T18:21:38.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:37 smithi099 ceph-mon[31658]: fsmap 5 up:standby 2024-05-03T18:21:38.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:37 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-03T18:21:38.071 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:37 smithi150 ceph-mon[32492]: Deploying daemon mds.f on smithi150 2024-05-03T18:21:38.071 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:37 smithi150 ceph-mon[32492]: mds.? [v2:172.21.15.99:6834/1206393914,v1:172.21.15.99:6835/1206393914] up:boot 2024-05-03T18:21:38.071 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:37 smithi150 ceph-mon[32492]: fsmap 5 up:standby 2024-05-03T18:21:38.071 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:37 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-03T18:21:38.561 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T18:21:38.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:38 smithi150 ceph-mon[32492]: pgmap v340: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:21:38.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:38 smithi033 ceph-mon[28330]: pgmap v340: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:21:39.033 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:21:39.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:38 smithi099 ceph-mon[31658]: pgmap v340: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:21:39.625 INFO:journalctl@ceph.mds.f.smithi150.stdout:May 03 18:21:39 smithi150 systemd[1]: Starting Ceph mds.f for 5703bba4-0978-11ef-bc97-c7b262605968... 2024-05-03T18:21:39.882 INFO:journalctl@ceph.mds.f.smithi150.stdout:May 03 18:21:39 smithi150 podman[62887]: 2024-05-03 18:21:39.533702859 +0000 UTC m=+0.039324647 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T18:21:39.882 INFO:journalctl@ceph.mds.f.smithi150.stdout:May 03 18:21:39 smithi150 podman[62887]: 2024-05-03 18:21:39.698164472 +0000 UTC m=+0.203786274 container create 2c9f0e84d96e01405d8920e5af7ad3d1a70ae61c9c78d1ad9eb312ee74701218 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-f, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.license=GPLv2, org.label-schema.build-date=20240430, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-03T18:21:40.382 INFO:journalctl@ceph.mds.f.smithi150.stdout:May 03 18:21:39 smithi150 podman[62887]: 2024-05-03 18:21:39.971021702 +0000 UTC m=+0.476643510 container init 2c9f0e84d96e01405d8920e5af7ad3d1a70ae61c9c78d1ad9eb312ee74701218 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-f, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-05-03T18:21:40.382 INFO:journalctl@ceph.mds.f.smithi150.stdout:May 03 18:21:39 smithi150 podman[62887]: 2024-05-03 18:21:39.980112224 +0000 UTC m=+0.485734029 container start 2c9f0e84d96e01405d8920e5af7ad3d1a70ae61c9c78d1ad9eb312ee74701218 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-f, GIT_CLEAN=True, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-05-03T18:21:40.382 INFO:journalctl@ceph.mds.f.smithi150.stdout:May 03 18:21:40 smithi150 ceph-mds[62906]: -- 172.21.15.150:0/518609744 <== mon.1 v2:172.21.15.150:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55abb1a25a40 con 0x55abb19f8800 2024-05-03T18:21:40.382 INFO:journalctl@ceph.mds.f.smithi150.stdout:May 03 18:21:40 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-f[62902]: starting mds.f at 2024-05-03T18:21:40.382 INFO:journalctl@ceph.mds.f.smithi150.stdout:May 03 18:21:40 smithi150 bash[62887]: 2c9f0e84d96e01405d8920e5af7ad3d1a70ae61c9c78d1ad9eb312ee74701218 2024-05-03T18:21:40.382 INFO:journalctl@ceph.mds.f.smithi150.stdout:May 03 18:21:40 smithi150 systemd[1]: Started Ceph mds.f for 5703bba4-0978-11ef-bc97-c7b262605968. 2024-05-03T18:21:40.898 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:40 smithi033 ceph-mon[28330]: pgmap v341: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:21:40.898 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:40 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:40.899 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:40 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:40.899 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:40 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:40.899 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:40 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T18:21:40.899 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:40 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T18:21:40.899 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:40 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:21:41.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:40 smithi099 ceph-mon[31658]: pgmap v341: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:21:41.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:40 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:41.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:40 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:41.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:40 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:41.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:40 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T18:21:41.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:40 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T18:21:41.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:40 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:21:41.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:41.067+0000 7fded1400700 1 -- 172.21.15.33:0/710078334 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdecc103fd0 msgr2=0x7fdecc108020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:21:41.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:41.067+0000 7fded1400700 1 --2- 172.21.15.33:0/710078334 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdecc103fd0 0x7fdecc108020 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fdeb40099c0 tx=0x7fdeb4009cd0 comp rx=0 tx=0).stop 2024-05-03T18:21:41.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:41.067+0000 7fded1400700 1 -- 172.21.15.33:0/710078334 shutdown_connections 2024-05-03T18:21:41.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:41.067+0000 7fded1400700 1 --2- 172.21.15.33:0/710078334 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdecc108700 0x7fdecc1102d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:21:41.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:41.067+0000 7fded1400700 1 --2- 172.21.15.33:0/710078334 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdecc103620 0x7fdecc103a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:21:41.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:41.067+0000 7fded1400700 1 --2- 172.21.15.33:0/710078334 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdecc103fd0 0x7fdecc108020 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:21:41.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:41.067+0000 7fded1400700 1 -- 172.21.15.33:0/710078334 >> 172.21.15.33:0/710078334 conn(0x7fdecc0feef0 msgr2=0x7fdecc101310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:21:41.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:41.067+0000 7fded1400700 1 -- 172.21.15.33:0/710078334 shutdown_connections 2024-05-03T18:21:41.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:41.067+0000 7fded1400700 1 -- 172.21.15.33:0/710078334 wait complete. 2024-05-03T18:21:41.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:41.069+0000 7fded1400700 1 Processor -- start 2024-05-03T18:21:41.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:41.070+0000 7fded1400700 1 -- start start 2024-05-03T18:21:41.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:41.070+0000 7fded1400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdecc103620 0x7fdecc1a18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:21:41.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:41.070+0000 7fded1400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdecc103fd0 0x7fdecc1a1df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:21:41.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:41.070+0000 7fded1400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdecc108700 0x7fdecc19ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:21:41.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:41.070+0000 7fded1400700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fdecc113490 con 0x7fdecc108700 2024-05-03T18:21:41.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:41.070+0000 7fded1400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fdecc113310 con 0x7fdecc103fd0 2024-05-03T18:21:41.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:41.070+0000 7fded1400700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fdecc113610 con 0x7fdecc103620 2024-05-03T18:21:41.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:41.071+0000 7fdecbe00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdecc103620 0x7fdecc1a18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:21:41.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:41.071+0000 7fdecbe00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdecc103620 0x7fdecc1a18b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:33764/0 (socket says 172.21.15.33:33764) 2024-05-03T18:21:41.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:41.071+0000 7fdecbe00700 1 -- 172.21.15.33:0/3414879318 learned_addr learned my addr 172.21.15.33:0/3414879318 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:21:41.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:41.071+0000 7fded0a00700 1 --2- 172.21.15.33:0/3414879318 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdecc108700 0x7fdecc19ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:21:41.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:41.071+0000 7fdec3400700 1 --2- 172.21.15.33:0/3414879318 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdecc103fd0 0x7fdecc1a1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:21:41.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:41.071+0000 7fdecbe00700 1 -- 172.21.15.33:0/3414879318 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdecc103fd0 msgr2=0x7fdecc1a1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:21:41.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:41.071+0000 7fdecbe00700 1 --2- 172.21.15.33:0/3414879318 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdecc103fd0 0x7fdecc1a1df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:21:41.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:41.071+0000 7fdecbe00700 1 -- 172.21.15.33:0/3414879318 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdecc108700 msgr2=0x7fdecc19ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:21:41.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:41.071+0000 7fdecbe00700 1 --2- 172.21.15.33:0/3414879318 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdecc108700 0x7fdecc19ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:21:41.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:41.071+0000 7fdecbe00700 1 -- 172.21.15.33:0/3414879318 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdecc19c1d0 con 0x7fdecc103620 2024-05-03T18:21:41.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:41.071+0000 7fded0a00700 1 --2- 172.21.15.33:0/3414879318 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdecc108700 0x7fdecc19ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:21:41.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:41.071+0000 7fdec3400700 1 --2- 172.21.15.33:0/3414879318 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdecc103fd0 0x7fdecc1a1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:21:41.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:41.072+0000 7fdecbe00700 1 --2- 172.21.15.33:0/3414879318 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdecc103620 0x7fdecc1a18b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fdebc007f10 tx=0x7fdebc00d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:21:41.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:41.072+0000 7fdec9600700 1 -- 172.21.15.33:0/3414879318 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdebc00dbd0 con 0x7fdecc103620 2024-05-03T18:21:41.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:41.072+0000 7fdec9600700 1 -- 172.21.15.33:0/3414879318 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdebc00dd70 con 0x7fdecc103620 2024-05-03T18:21:41.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:41.072+0000 7fded1400700 1 -- 172.21.15.33:0/3414879318 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdecc19c4c0 con 0x7fdecc103620 2024-05-03T18:21:41.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:41.072+0000 7fded1400700 1 -- 172.21.15.33:0/3414879318 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdecc1a8780 con 0x7fdecc103620 2024-05-03T18:21:41.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:41.072+0000 7fdec9600700 1 -- 172.21.15.33:0/3414879318 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdebc005950 con 0x7fdecc103620 2024-05-03T18:21:41.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:41.073+0000 7fded1400700 1 -- 172.21.15.33:0/3414879318 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdea8005330 con 0x7fdecc103620 2024-05-03T18:21:41.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:41.076+0000 7fdec9600700 1 -- 172.21.15.33:0/3414879318 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fdebc01a020 con 0x7fdecc103620 2024-05-03T18:21:41.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:41.077+0000 7fdec9600700 1 --2- 172.21.15.33:0/3414879318 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fdeb00b2900 0x7fdeb00b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:21:41.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:41.077+0000 7fdec9600700 1 -- 172.21.15.33:0/3414879318 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7fdebc0e1430 con 0x7fdecc103620 2024-05-03T18:21:41.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:41.077+0000 7fdec3400700 1 --2- 172.21.15.33:0/3414879318 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fdeb00b2900 0x7fdeb00b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:21:41.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:41.078+0000 7fdec3400700 1 --2- 172.21.15.33:0/3414879318 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fdeb00b2900 0x7fdeb00b4dc0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fdeb4000c00 tx=0x7fdeb4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:21:41.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:41.080+0000 7fdec9600700 1 -- 172.21.15.33:0/3414879318 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fdebc010070 con 0x7fdecc103620 2024-05-03T18:21:41.131 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:40 smithi150 ceph-mon[32492]: pgmap v341: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:21:41.131 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:40 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:41.131 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:40 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:41.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:40 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:41.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:40 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T18:21:41.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:40 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T18:21:41.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:40 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:21:41.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:41.366+0000 7fded1400700 1 -- 172.21.15.33:0/3414879318 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdea8005620 con 0x7fdecc103620 2024-05-03T18:21:41.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:41.368+0000 7fdec9600700 1 -- 172.21.15.33:0/3414879318 <== mon.1 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5521 (secure 0 0 0) 0x7fdebc0a9550 con 0x7fdecc103620 2024-05-03T18:21:41.369 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:21:41.369 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1327393503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1327393503},{"type":"v1","addr":"172.21.15.33:6835","nonce":1327393503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/2461472948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":2461472948},{"type":"v1","addr":"172.21.15.33:6837","nonce":2461472948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/203598909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":203598909},{"type":"v1","addr":"172.21.15.150:6833","nonce":203598909}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/4029842493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":4029842493},{"type":"v1","addr":"172.21.15.150:6835","nonce":4029842493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":34418,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1865105479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1865105479},{"type":"v1","addr":"172.21.15.99:6833","nonce":1865105479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34445,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/1206393914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1206393914},{"type":"v1","addr":"172.21.15.99:6835","nonce":1206393914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-05-03T18:21:41.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:41.371+0000 7fded1400700 1 -- 172.21.15.33:0/3414879318 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fdeb00b2900 msgr2=0x7fdeb00b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:21:41.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:41.371+0000 7fded1400700 1 --2- 172.21.15.33:0/3414879318 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fdeb00b2900 0x7fdeb00b4dc0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fdeb4000c00 tx=0x7fdeb4005fb0 comp rx=0 tx=0).stop 2024-05-03T18:21:41.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:41.372+0000 7fded1400700 1 -- 172.21.15.33:0/3414879318 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdecc103620 msgr2=0x7fdecc1a18b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:21:41.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:41.372+0000 7fded1400700 1 --2- 172.21.15.33:0/3414879318 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdecc103620 0x7fdecc1a18b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fdebc007f10 tx=0x7fdebc00d3b0 comp rx=0 tx=0).stop 2024-05-03T18:21:41.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:41.372+0000 7fded1400700 1 -- 172.21.15.33:0/3414879318 shutdown_connections 2024-05-03T18:21:41.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:41.372+0000 7fded1400700 1 --2- 172.21.15.33:0/3414879318 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdecc108700 0x7fdecc19ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:21:41.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:41.372+0000 7fded1400700 1 --2- 172.21.15.33:0/3414879318 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdecc103620 0x7fdecc1a18b0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:21:41.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:41.372+0000 7fded1400700 1 --2- 172.21.15.33:0/3414879318 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fdeb00b2900 0x7fdeb00b4dc0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:21:41.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:41.372+0000 7fded1400700 1 --2- 172.21.15.33:0/3414879318 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdecc103fd0 0x7fdecc1a1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:21:41.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:41.372+0000 7fded1400700 1 -- 172.21.15.33:0/3414879318 >> 172.21.15.33:0/3414879318 conn(0x7fdecc0feef0 msgr2=0x7fdecc101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:21:41.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:41.372+0000 7fded1400700 1 -- 172.21.15.33:0/3414879318 shutdown_connections 2024-05-03T18:21:41.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:41.372+0000 7fded1400700 1 -- 172.21.15.33:0/3414879318 wait complete. 2024-05-03T18:21:41.374 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 7 2024-05-03T18:21:41.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:41 smithi150 ceph-mon[32492]: Deploying daemon mds.c on smithi150 2024-05-03T18:21:41.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:41 smithi150 ceph-mon[32492]: mds.? [v2:172.21.15.150:6834/4029842493,v1:172.21.15.150:6835/4029842493] up:boot 2024-05-03T18:21:41.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:41 smithi150 ceph-mon[32492]: fsmap 6 up:standby 2024-05-03T18:21:41.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:41 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-03T18:21:41.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:41 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/3414879318' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:21:41.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:41 smithi033 ceph-mon[28330]: Deploying daemon mds.c on smithi150 2024-05-03T18:21:41.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:41 smithi033 ceph-mon[28330]: mds.? [v2:172.21.15.150:6834/4029842493,v1:172.21.15.150:6835/4029842493] up:boot 2024-05-03T18:21:41.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:41 smithi033 ceph-mon[28330]: fsmap 6 up:standby 2024-05-03T18:21:41.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:41 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-03T18:21:41.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:41 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/3414879318' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:21:42.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:41 smithi099 ceph-mon[31658]: Deploying daemon mds.c on smithi150 2024-05-03T18:21:42.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:41 smithi099 ceph-mon[31658]: mds.? [v2:172.21.15.150:6834/4029842493,v1:172.21.15.150:6835/4029842493] up:boot 2024-05-03T18:21:42.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:41 smithi099 ceph-mon[31658]: fsmap 6 up:standby 2024-05-03T18:21:42.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:41 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-03T18:21:42.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:41 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/3414879318' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:21:42.776 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:42 smithi150 ceph-mon[32492]: pgmap v342: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:21:42.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:42 smithi033 ceph-mon[28330]: pgmap v342: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:21:43.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:42 smithi099 ceph-mon[31658]: pgmap v342: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:21:43.817 INFO:journalctl@ceph.mds.c.smithi150.stdout:May 03 18:21:43 smithi150 systemd[1]: Starting Ceph mds.c for 5703bba4-0978-11ef-bc97-c7b262605968... 2024-05-03T18:21:44.123 INFO:journalctl@ceph.mds.c.smithi150.stdout:May 03 18:21:43 smithi150 podman[63191]: 2024-05-03 18:21:43.725719658 +0000 UTC m=+0.036941893 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T18:21:44.123 INFO:journalctl@ceph.mds.c.smithi150.stdout:May 03 18:21:43 smithi150 podman[63191]: 2024-05-03 18:21:43.839433934 +0000 UTC m=+0.150656196 container create 78f83371523d8fb123ac087ca9e9098de90b62ea27db5232d7ace81f9ef1cd26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-03T18:21:44.123 INFO:journalctl@ceph.mds.c.smithi150.stdout:May 03 18:21:44 smithi150 podman[63191]: 2024-05-03 18:21:44.122127678 +0000 UTC m=+0.433349943 container init 78f83371523d8fb123ac087ca9e9098de90b62ea27db5232d7ace81f9ef1cd26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-c, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.build-date=20240430, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-05-03T18:21:44.149 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T18:21:44.382 INFO:journalctl@ceph.mds.c.smithi150.stdout:May 03 18:21:44 smithi150 podman[63191]: 2024-05-03 18:21:44.131913352 +0000 UTC m=+0.443135583 container start 78f83371523d8fb123ac087ca9e9098de90b62ea27db5232d7ace81f9ef1cd26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-c, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-05-03T18:21:44.382 INFO:journalctl@ceph.mds.c.smithi150.stdout:May 03 18:21:44 smithi150 ceph-mds[63209]: -- 172.21.15.150:0/4065038884 <== mon.1 v2:172.21.15.150:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cb9753fa40 con 0x55cb97513000 2024-05-03T18:21:44.382 INFO:journalctl@ceph.mds.c.smithi150.stdout:May 03 18:21:44 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-c[63205]: starting mds.c at 2024-05-03T18:21:44.383 INFO:journalctl@ceph.mds.c.smithi150.stdout:May 03 18:21:44 smithi150 bash[63191]: 78f83371523d8fb123ac087ca9e9098de90b62ea27db5232d7ace81f9ef1cd26 2024-05-03T18:21:44.383 INFO:journalctl@ceph.mds.c.smithi150.stdout:May 03 18:21:44 smithi150 systemd[1]: Started Ceph mds.c for 5703bba4-0978-11ef-bc97-c7b262605968. 2024-05-03T18:21:44.614 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:21:44.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:44 smithi033 ceph-mon[28330]: pgmap v343: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:21:44.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:44 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:44.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:44 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:44.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:44 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:44.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:44 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T18:21:44.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:44 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T18:21:44.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:44 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:21:45.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:44 smithi099 ceph-mon[31658]: pgmap v343: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:21:45.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:44 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:45.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:44 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:45.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:44 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:45.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:44 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T18:21:45.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:44 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T18:21:45.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:44 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:21:45.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:44 smithi150 ceph-mon[32492]: pgmap v343: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:21:45.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:44 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:45.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:44 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:45.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:44 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:45.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:44 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T18:21:45.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:44 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T18:21:45.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:44 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:21:45.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:45 smithi033 ceph-mon[28330]: Deploying daemon mds.h on smithi099 2024-05-03T18:21:45.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:45 smithi033 ceph-mon[28330]: mds.? [v2:172.21.15.150:6836/3463664423,v1:172.21.15.150:6837/3463664423] up:boot 2024-05-03T18:21:45.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:45 smithi033 ceph-mon[28330]: fsmap 7 up:standby 2024-05-03T18:21:45.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:45 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-03T18:21:46.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:45 smithi099 ceph-mon[31658]: Deploying daemon mds.h on smithi099 2024-05-03T18:21:46.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:45 smithi099 ceph-mon[31658]: mds.? [v2:172.21.15.150:6836/3463664423,v1:172.21.15.150:6837/3463664423] up:boot 2024-05-03T18:21:46.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:45 smithi099 ceph-mon[31658]: fsmap 7 up:standby 2024-05-03T18:21:46.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:45 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-03T18:21:46.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:45 smithi150 ceph-mon[32492]: Deploying daemon mds.h on smithi099 2024-05-03T18:21:46.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:45 smithi150 ceph-mon[32492]: mds.? [v2:172.21.15.150:6836/3463664423,v1:172.21.15.150:6837/3463664423] up:boot 2024-05-03T18:21:46.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:45 smithi150 ceph-mon[32492]: fsmap 7 up:standby 2024-05-03T18:21:46.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:45 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-03T18:21:46.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:46.939+0000 7f395d000700 1 -- 172.21.15.33:0/3666324544 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f39580779d0 msgr2=0x7f3958077db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:21:46.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:46.939+0000 7f395d000700 1 --2- 172.21.15.33:0/3666324544 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f39580779d0 0x7f3958077db0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3948009990 tx=0x7f3948009ca0 comp rx=0 tx=0).stop 2024-05-03T18:21:46.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:46.940+0000 7f395d000700 1 -- 172.21.15.33:0/3666324544 shutdown_connections 2024-05-03T18:21:46.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:46.940+0000 7f395d000700 1 --2- 172.21.15.33:0/3666324544 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3958114350 0x7f3958116740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:21:46.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:46.940+0000 7f395d000700 1 --2- 172.21.15.33:0/3666324544 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f39580779d0 0x7f3958077db0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:21:46.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:46.940+0000 7f395d000700 1 --2- 172.21.15.33:0/3666324544 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39580782f0 0x7f3958113e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:21:46.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:46.940+0000 7f395d000700 1 -- 172.21.15.33:0/3666324544 >> 172.21.15.33:0/3666324544 conn(0x7f39581011a0 msgr2=0x7f39581035c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:21:46.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:46.940+0000 7f395d000700 1 -- 172.21.15.33:0/3666324544 shutdown_connections 2024-05-03T18:21:46.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:46.940+0000 7f395d000700 1 -- 172.21.15.33:0/3666324544 wait complete. 2024-05-03T18:21:46.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:46.941+0000 7f395d000700 1 Processor -- start 2024-05-03T18:21:46.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:46.941+0000 7f395d000700 1 -- start start 2024-05-03T18:21:46.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:46.941+0000 7f395d000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f39580779d0 0x7f39581a00c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:21:46.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:46.941+0000 7f395d000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39580782f0 0x7f39581a0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:21:46.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:46.941+0000 7f395d000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3958114350 0x7f39581a4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:21:46.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:46.941+0000 7f395d000700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3958119b70 con 0x7f39580782f0 2024-05-03T18:21:46.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:46.941+0000 7f395d000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f39581199f0 con 0x7f39580779d0 2024-05-03T18:21:46.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:46.941+0000 7f395d000700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3958119cf0 con 0x7f3958114350 2024-05-03T18:21:46.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:46.942+0000 7f3955400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39580782f0 0x7f39581a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:21:46.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:46.942+0000 7f3955e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f39580779d0 0x7f39581a00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:21:46.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:46.942+0000 7f3955e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f39580779d0 0x7f39581a00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:48648/0 (socket says 172.21.15.33:48648) 2024-05-03T18:21:46.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:46.942+0000 7f3955e00700 1 -- 172.21.15.33:0/2782785973 learned_addr learned my addr 172.21.15.33:0/2782785973 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:21:46.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:46.942+0000 7f3956800700 1 --2- 172.21.15.33:0/2782785973 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3958114350 0x7f39581a4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:21:46.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:46.942+0000 7f3955e00700 1 -- 172.21.15.33:0/2782785973 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3958114350 msgr2=0x7f39581a4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:21:46.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:46.942+0000 7f3955e00700 1 --2- 172.21.15.33:0/2782785973 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3958114350 0x7f39581a4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:21:46.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:46.942+0000 7f3955e00700 1 -- 172.21.15.33:0/2782785973 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39580782f0 msgr2=0x7f39581a0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:21:46.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:46.942+0000 7f3955e00700 1 --2- 172.21.15.33:0/2782785973 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39580782f0 0x7f39581a0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:21:46.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:46.942+0000 7f3955e00700 1 -- 172.21.15.33:0/2782785973 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39581a5080 con 0x7f39580779d0 2024-05-03T18:21:46.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:46.943+0000 7f3955e00700 1 --2- 172.21.15.33:0/2782785973 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f39580779d0 0x7f39581a00c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3948006010 tx=0x7f39480057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:21:46.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:46.943+0000 7f394ea00700 1 -- 172.21.15.33:0/2782785973 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3948027070 con 0x7f39580779d0 2024-05-03T18:21:46.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:46.943+0000 7f395d000700 1 -- 172.21.15.33:0/2782785973 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39581a5310 con 0x7f39580779d0 2024-05-03T18:21:46.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:46.943+0000 7f394ea00700 1 -- 172.21.15.33:0/2782785973 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f394800fb90 con 0x7f39580779d0 2024-05-03T18:21:46.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:46.943+0000 7f394ea00700 1 -- 172.21.15.33:0/2782785973 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39480173f0 con 0x7f39580779d0 2024-05-03T18:21:46.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:46.943+0000 7f395d000700 1 -- 172.21.15.33:0/2782785973 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39581acbf0 con 0x7f39580779d0 2024-05-03T18:21:46.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:46.944+0000 7f395d000700 1 -- 172.21.15.33:0/2782785973 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f395806a190 con 0x7f39580779d0 2024-05-03T18:21:46.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:46.949+0000 7f394ea00700 1 -- 172.21.15.33:0/2782785973 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f394802a070 con 0x7f39580779d0 2024-05-03T18:21:46.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:46.951+0000 7f394ea00700 1 --2- 172.21.15.33:0/2782785973 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f393c0b2880 0x7f393c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:21:46.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:46.951+0000 7f394ea00700 1 -- 172.21.15.33:0/2782785973 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f39480e30a0 con 0x7f39580779d0 2024-05-03T18:21:46.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:46.951+0000 7f3955400700 1 --2- 172.21.15.33:0/2782785973 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f393c0b2880 0x7f393c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:21:46.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:46.952+0000 7f3955400700 1 --2- 172.21.15.33:0/2782785973 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f393c0b2880 0x7f393c0b4d40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3944009680 tx=0x7f3944009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:21:46.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:46.963+0000 7f394ea00700 1 -- 172.21.15.33:0/2782785973 <== mon.2 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f3948013070 con 0x7f39580779d0 2024-05-03T18:21:46.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:46 smithi033 ceph-mon[28330]: pgmap v344: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:21:47.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:46 smithi099 ceph-mon[31658]: pgmap v344: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:21:47.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:46 smithi150 ceph-mon[32492]: pgmap v344: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:21:47.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:47.250+0000 7f395d000700 1 -- 172.21.15.33:0/2782785973 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f395802e510 con 0x7f39580779d0 2024-05-03T18:21:47.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:47.252+0000 7f394ea00700 1 -- 172.21.15.33:0/2782785973 <== mon.2 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6349 (secure 0 0 0) 0x7f39480ab250 con 0x7f39580779d0 2024-05-03T18:21:47.253 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:21:47.253 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1327393503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1327393503},{"type":"v1","addr":"172.21.15.33:6835","nonce":1327393503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/2461472948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":2461472948},{"type":"v1","addr":"172.21.15.33:6837","nonce":2461472948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/203598909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":203598909},{"type":"v1","addr":"172.21.15.150:6833","nonce":203598909}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/4029842493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":4029842493},{"type":"v1","addr":"172.21.15.150:6835","nonce":4029842493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3463664423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3463664423},{"type":"v1","addr":"172.21.15.150:6837","nonce":3463664423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":34418,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1865105479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1865105479},{"type":"v1","addr":"172.21.15.99:6833","nonce":1865105479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34445,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/1206393914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1206393914},{"type":"v1","addr":"172.21.15.99:6835","nonce":1206393914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-05-03T18:21:47.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:47.255+0000 7f395d000700 1 -- 172.21.15.33:0/2782785973 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f393c0b2880 msgr2=0x7f393c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:21:47.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:47.255+0000 7f395d000700 1 --2- 172.21.15.33:0/2782785973 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f393c0b2880 0x7f393c0b4d40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3944009680 tx=0x7f3944009260 comp rx=0 tx=0).stop 2024-05-03T18:21:47.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:47.255+0000 7f395d000700 1 -- 172.21.15.33:0/2782785973 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f39580779d0 msgr2=0x7f39581a00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:21:47.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:47.255+0000 7f395d000700 1 --2- 172.21.15.33:0/2782785973 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f39580779d0 0x7f39581a00c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3948006010 tx=0x7f39480057c0 comp rx=0 tx=0).stop 2024-05-03T18:21:47.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:47.255+0000 7f395d000700 1 -- 172.21.15.33:0/2782785973 shutdown_connections 2024-05-03T18:21:47.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:47.255+0000 7f395d000700 1 --2- 172.21.15.33:0/2782785973 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3958114350 0x7f39581a4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:21:47.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:47.255+0000 7f395d000700 1 --2- 172.21.15.33:0/2782785973 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f393c0b2880 0x7f393c0b4d40 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:21:47.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:47.255+0000 7f395d000700 1 --2- 172.21.15.33:0/2782785973 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f39580779d0 0x7f39581a00c0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:21:47.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:47.255+0000 7f395d000700 1 --2- 172.21.15.33:0/2782785973 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39580782f0 0x7f39581a0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:21:47.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:47.255+0000 7f395d000700 1 -- 172.21.15.33:0/2782785973 >> 172.21.15.33:0/2782785973 conn(0x7f39581011a0 msgr2=0x7f3958103590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:21:47.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:47.255+0000 7f395d000700 1 -- 172.21.15.33:0/2782785973 shutdown_connections 2024-05-03T18:21:47.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:47.255+0000 7f395d000700 1 -- 172.21.15.33:0/2782785973 wait complete. 2024-05-03T18:21:47.257 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 8 2024-05-03T18:21:47.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:47 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/2782785973' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:21:47.952 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:47 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/2782785973' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:21:48.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:47 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/2782785973' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:21:48.225 INFO:journalctl@ceph.mds.h.smithi099.stdout:May 03 18:21:47 smithi099 podman[61681]: 2024-05-03 18:21:47.902532243 +0000 UTC m=+0.157526390 container create 08471d7f10c42253437c204aa85aeedec31e2160dc9a58d43e8bd4efdd3dd604 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-h, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430) 2024-05-03T18:21:48.225 INFO:journalctl@ceph.mds.h.smithi099.stdout:May 03 18:21:48 smithi099 podman[61681]: 2024-05-03 18:21:48.224958149 +0000 UTC m=+0.479952326 container init 08471d7f10c42253437c204aa85aeedec31e2160dc9a58d43e8bd4efdd3dd604 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-h, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD) 2024-05-03T18:21:48.545 INFO:journalctl@ceph.mds.h.smithi099.stdout:May 03 18:21:48 smithi099 podman[61681]: 2024-05-03 18:21:48.235780567 +0000 UTC m=+0.490774744 container start 08471d7f10c42253437c204aa85aeedec31e2160dc9a58d43e8bd4efdd3dd604 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-h, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-05-03T18:21:48.546 INFO:journalctl@ceph.mds.h.smithi099.stdout:May 03 18:21:48 smithi099 ceph-mds[61700]: -- 172.21.15.99:0/1136256040 <== mon.2 v2:172.21.15.99:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5561fcf6ba40 con 0x5561fcf3f000 2024-05-03T18:21:48.546 INFO:journalctl@ceph.mds.h.smithi099.stdout:May 03 18:21:48 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-h[61696]: starting mds.h at 2024-05-03T18:21:48.546 INFO:journalctl@ceph.mds.h.smithi099.stdout:May 03 18:21:48 smithi099 bash[61681]: 08471d7f10c42253437c204aa85aeedec31e2160dc9a58d43e8bd4efdd3dd604 2024-05-03T18:21:48.546 INFO:journalctl@ceph.mds.h.smithi099.stdout:May 03 18:21:48 smithi099 systemd[1]: Started Ceph mds.h for 5703bba4-0978-11ef-bc97-c7b262605968. 2024-05-03T18:21:49.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:48 smithi099 ceph-mon[31658]: pgmap v345: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:21:49.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:48 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:49.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:48 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:49.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:48 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:49.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:48 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T18:21:49.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:48 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T18:21:49.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:48 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:21:49.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:48 smithi150 ceph-mon[32492]: pgmap v345: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:21:49.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:48 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:49.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:48 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:49.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:48 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:49.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:48 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T18:21:49.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:48 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T18:21:49.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:48 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:21:49.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:48 smithi033 ceph-mon[28330]: pgmap v345: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:21:49.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:48 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:49.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:48 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:49.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:48 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:49.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:48 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T18:21:49.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:48 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T18:21:49.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:48 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:21:49.848 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:49 smithi150 ceph-mon[32492]: Deploying daemon mds.i on smithi150 2024-05-03T18:21:49.848 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:49 smithi150 ceph-mon[32492]: mds.? [v2:172.21.15.99:6836/3360970342,v1:172.21.15.99:6837/3360970342] up:boot 2024-05-03T18:21:49.848 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:49 smithi150 ceph-mon[32492]: fsmap 8 up:standby 2024-05-03T18:21:49.848 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:49 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-03T18:21:49.848 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:49 smithi150 ceph-mon[32492]: pgmap v346: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:21:49.960 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:49 smithi033 ceph-mon[28330]: Deploying daemon mds.i on smithi150 2024-05-03T18:21:49.960 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:49 smithi033 ceph-mon[28330]: mds.? [v2:172.21.15.99:6836/3360970342,v1:172.21.15.99:6837/3360970342] up:boot 2024-05-03T18:21:49.961 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:49 smithi033 ceph-mon[28330]: fsmap 8 up:standby 2024-05-03T18:21:49.961 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:49 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-03T18:21:49.961 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:49 smithi033 ceph-mon[28330]: pgmap v346: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:21:50.027 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T18:21:50.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:49 smithi099 ceph-mon[31658]: Deploying daemon mds.i on smithi150 2024-05-03T18:21:50.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:49 smithi099 ceph-mon[31658]: mds.? [v2:172.21.15.99:6836/3360970342,v1:172.21.15.99:6837/3360970342] up:boot 2024-05-03T18:21:50.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:49 smithi099 ceph-mon[31658]: fsmap 8 up:standby 2024-05-03T18:21:50.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:49 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-03T18:21:50.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:49 smithi099 ceph-mon[31658]: pgmap v346: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:21:50.486 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:21:52.042 INFO:journalctl@ceph.mds.i.smithi150.stdout:May 03 18:21:51 smithi150 systemd[1]: Starting Ceph mds.i for 5703bba4-0978-11ef-bc97-c7b262605968... 2024-05-03T18:21:52.334 INFO:journalctl@ceph.mds.i.smithi150.stdout:May 03 18:21:52 smithi150 podman[63493]: 2024-05-03 18:21:51.947573343 +0000 UTC m=+0.039229481 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T18:21:52.334 INFO:journalctl@ceph.mds.i.smithi150.stdout:May 03 18:21:52 smithi150 podman[63493]: 2024-05-03 18:21:52.071223424 +0000 UTC m=+0.162879589 container create ed43a8474889db2d7cff52192388905ab8c90a19c7549b8e315b37ece40cecd1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-i, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-05-03T18:21:52.334 INFO:journalctl@ceph.mds.i.smithi150.stdout:May 03 18:21:52 smithi150 podman[63493]: 2024-05-03 18:21:52.333470605 +0000 UTC m=+0.425126777 container init ed43a8474889db2d7cff52192388905ab8c90a19c7549b8e315b37ece40cecd1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-i, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, io.buildah.version=1.33.5) 2024-05-03T18:21:52.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:52 smithi033 ceph-mon[28330]: pgmap v347: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:21:52.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:52 smithi150 ceph-mon[32492]: pgmap v347: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:21:52.632 INFO:journalctl@ceph.mds.i.smithi150.stdout:May 03 18:21:52 smithi150 podman[63493]: 2024-05-03 18:21:52.343260852 +0000 UTC m=+0.434916988 container start ed43a8474889db2d7cff52192388905ab8c90a19c7549b8e315b37ece40cecd1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-i, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-03T18:21:52.632 INFO:journalctl@ceph.mds.i.smithi150.stdout:May 03 18:21:52 smithi150 ceph-mds[63512]: -- 172.21.15.150:0/4184687670 <== mon.1 v2:172.21.15.150:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558ccb535a40 con 0x558ccb508c00 2024-05-03T18:21:52.632 INFO:journalctl@ceph.mds.i.smithi150.stdout:May 03 18:21:52 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-i[63508]: starting mds.i at 2024-05-03T18:21:52.632 INFO:journalctl@ceph.mds.i.smithi150.stdout:May 03 18:21:52 smithi150 bash[63493]: ed43a8474889db2d7cff52192388905ab8c90a19c7549b8e315b37ece40cecd1 2024-05-03T18:21:52.632 INFO:journalctl@ceph.mds.i.smithi150.stdout:May 03 18:21:52 smithi150 systemd[1]: Started Ceph mds.i for 5703bba4-0978-11ef-bc97-c7b262605968. 2024-05-03T18:21:52.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:52.659+0000 7f44af200700 1 -- 172.21.15.33:0/3972993219 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44b010fd30 msgr2=0x7f44b0112120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:21:52.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:52.659+0000 7f44af200700 1 --2- 172.21.15.33:0/3972993219 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44b010fd30 0x7f44b0112120 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f44a800b0a0 tx=0x7f44a800b3b0 comp rx=0 tx=0).stop 2024-05-03T18:21:52.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:52.659+0000 7f44af200700 1 -- 172.21.15.33:0/3972993219 shutdown_connections 2024-05-03T18:21:52.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:52.659+0000 7f44af200700 1 --2- 172.21.15.33:0/3972993219 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44b010fd30 0x7f44b0112120 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:21:52.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:52.659+0000 7f44af200700 1 --2- 172.21.15.33:0/3972993219 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44b0102180 0x7f44b0102560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:21:52.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:52.659+0000 7f44af200700 1 --2- 172.21.15.33:0/3972993219 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44b0102b30 0x7f44b010f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:21:52.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:52.659+0000 7f44af200700 1 -- 172.21.15.33:0/3972993219 >> 172.21.15.33:0/3972993219 conn(0x7f44b00fddc0 msgr2=0x7f44b01001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:21:52.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:52.659+0000 7f44af200700 1 -- 172.21.15.33:0/3972993219 shutdown_connections 2024-05-03T18:21:52.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:52.659+0000 7f44af200700 1 -- 172.21.15.33:0/3972993219 wait complete. 2024-05-03T18:21:52.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:52.660+0000 7f44af200700 1 Processor -- start 2024-05-03T18:21:52.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:52.660+0000 7f44af200700 1 -- start start 2024-05-03T18:21:52.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:52.661+0000 7f44af200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44b0102180 0x7f44b01a1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:21:52.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:52.661+0000 7f44af200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44b0102b30 0x7f44b01a1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:21:52.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:52.661+0000 7f44af200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44b010fd30 0x7f44b019ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:21:52.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:52.661+0000 7f44af200700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f44b01156c0 con 0x7f44b0102b30 2024-05-03T18:21:52.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:52.661+0000 7f44af200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f44b0115540 con 0x7f44b0102180 2024-05-03T18:21:52.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:52.661+0000 7f44af200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f44b0115840 con 0x7f44b010fd30 2024-05-03T18:21:52.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:52.661+0000 7f44ad400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44b0102b30 0x7f44b01a1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:21:52.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:52.661+0000 7f44ad400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44b0102b30 0x7f44b01a1d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54788/0 (socket says 172.21.15.33:54788) 2024-05-03T18:21:52.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:52.661+0000 7f44ad400700 1 -- 172.21.15.33:0/68590538 learned_addr learned my addr 172.21.15.33:0/68590538 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:21:52.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:52.661+0000 7f44ade00700 1 --2- 172.21.15.33:0/68590538 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44b0102180 0x7f44b01a1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:21:52.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:52.662+0000 7f44ae800700 1 --2- 172.21.15.33:0/68590538 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44b010fd30 0x7f44b019ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:21:52.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:52.662+0000 7f44ad400700 1 -- 172.21.15.33:0/68590538 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44b010fd30 msgr2=0x7f44b019ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:21:52.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:52.662+0000 7f44ad400700 1 --2- 172.21.15.33:0/68590538 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44b010fd30 0x7f44b019ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:21:52.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:52.662+0000 7f44ad400700 1 -- 172.21.15.33:0/68590538 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44b0102180 msgr2=0x7f44b01a1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:21:52.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:52.662+0000 7f44ad400700 1 --2- 172.21.15.33:0/68590538 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44b0102180 0x7f44b01a1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:21:52.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:52.662+0000 7f44ad400700 1 -- 172.21.15.33:0/68590538 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44b019c1c0 con 0x7f44b0102b30 2024-05-03T18:21:52.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:52.662+0000 7f44ae800700 1 --2- 172.21.15.33:0/68590538 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44b010fd30 0x7f44b019ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:21:52.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:52.662+0000 7f44ad400700 1 --2- 172.21.15.33:0/68590538 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44b0102b30 0x7f44b01a1d90 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f449c00c970 tx=0x7f449c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:21:52.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:52.663+0000 7f44a6a00700 1 -- 172.21.15.33:0/68590538 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f449c007aa0 con 0x7f44b0102b30 2024-05-03T18:21:52.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:52.663+0000 7f44a6a00700 1 -- 172.21.15.33:0/68590538 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f449c004cc0 con 0x7f44b0102b30 2024-05-03T18:21:52.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:52.663+0000 7f44a6a00700 1 -- 172.21.15.33:0/68590538 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f449c005450 con 0x7f44b0102b30 2024-05-03T18:21:52.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:52.663+0000 7f44af200700 1 -- 172.21.15.33:0/68590538 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44b019c4b0 con 0x7f44b0102b30 2024-05-03T18:21:52.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:52.663+0000 7f44af200700 1 -- 172.21.15.33:0/68590538 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44b01a8720 con 0x7f44b0102b30 2024-05-03T18:21:52.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:52.664+0000 7f44af200700 1 -- 172.21.15.33:0/68590538 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44b006a190 con 0x7f44b0102b30 2024-05-03T18:21:52.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:52.667+0000 7f44a6a00700 1 -- 172.21.15.33:0/68590538 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f449c007c40 con 0x7f44b0102b30 2024-05-03T18:21:52.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:52.669+0000 7f44a6a00700 1 --2- 172.21.15.33:0/68590538 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f44940b2900 0x7f44940b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:21:52.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:52.669+0000 7f44a6a00700 1 -- 172.21.15.33:0/68590538 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f449c0e1cf0 con 0x7f44b0102b30 2024-05-03T18:21:52.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:52.669+0000 7f44ade00700 1 --2- 172.21.15.33:0/68590538 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f44940b2900 0x7f44940b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:21:52.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:52.670+0000 7f44ade00700 1 --2- 172.21.15.33:0/68590538 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f44940b2900 0x7f44940b4dc0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f44b01104d0 tx=0x7f44a000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:21:52.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:52.671+0000 7f44a6a00700 1 -- 172.21.15.33:0/68590538 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f449c0a9fd0 con 0x7f44b0102b30 2024-05-03T18:21:52.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:52 smithi099 ceph-mon[31658]: pgmap v347: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:21:52.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:52.968+0000 7f44af200700 1 -- 172.21.15.33:0/68590538 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f44b019cf40 con 0x7f44b0102b30 2024-05-03T18:21:52.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:52.969+0000 7f44a6a00700 1 -- 172.21.15.33:0/68590538 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7174 (secure 0 0 0) 0x7f449c0a98d0 con 0x7f44b0102b30 2024-05-03T18:21:52.970 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:21:52.970 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1327393503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1327393503},{"type":"v1","addr":"172.21.15.33:6835","nonce":1327393503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/2461472948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":2461472948},{"type":"v1","addr":"172.21.15.33:6837","nonce":2461472948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/203598909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":203598909},{"type":"v1","addr":"172.21.15.150:6833","nonce":203598909}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/4029842493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":4029842493},{"type":"v1","addr":"172.21.15.150:6835","nonce":4029842493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3463664423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3463664423},{"type":"v1","addr":"172.21.15.150:6837","nonce":3463664423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":34418,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1865105479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1865105479},{"type":"v1","addr":"172.21.15.99:6833","nonce":1865105479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34445,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/1206393914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1206393914},{"type":"v1","addr":"172.21.15.99:6835","nonce":1206393914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34472,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/3360970342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3360970342},{"type":"v1","addr":"172.21.15.99:6837","nonce":3360970342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-05-03T18:21:52.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:52.972+0000 7f44af200700 1 -- 172.21.15.33:0/68590538 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f44940b2900 msgr2=0x7f44940b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:21:52.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:52.972+0000 7f44af200700 1 --2- 172.21.15.33:0/68590538 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f44940b2900 0x7f44940b4dc0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f44b01104d0 tx=0x7f44a000b410 comp rx=0 tx=0).stop 2024-05-03T18:21:52.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:52.972+0000 7f44af200700 1 -- 172.21.15.33:0/68590538 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44b0102b30 msgr2=0x7f44b01a1d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:21:52.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:52.972+0000 7f44af200700 1 --2- 172.21.15.33:0/68590538 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44b0102b30 0x7f44b01a1d90 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f449c00c970 tx=0x7f449c00cc80 comp rx=0 tx=0).stop 2024-05-03T18:21:52.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:52.972+0000 7f44af200700 1 -- 172.21.15.33:0/68590538 shutdown_connections 2024-05-03T18:21:52.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:52.972+0000 7f44af200700 1 --2- 172.21.15.33:0/68590538 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44b010fd30 0x7f44b019ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:21:52.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:52.972+0000 7f44af200700 1 --2- 172.21.15.33:0/68590538 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f44940b2900 0x7f44940b4dc0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:21:52.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:52.972+0000 7f44af200700 1 --2- 172.21.15.33:0/68590538 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44b0102180 0x7f44b01a1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:21:52.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:52.972+0000 7f44af200700 1 --2- 172.21.15.33:0/68590538 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44b0102b30 0x7f44b01a1d90 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:21:52.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:52.972+0000 7f44af200700 1 -- 172.21.15.33:0/68590538 >> 172.21.15.33:0/68590538 conn(0x7f44b00fddc0 msgr2=0x7f44b01001b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:21:52.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:52.972+0000 7f44af200700 1 -- 172.21.15.33:0/68590538 shutdown_connections 2024-05-03T18:21:52.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:52.972+0000 7f44af200700 1 -- 172.21.15.33:0/68590538 wait complete. 2024-05-03T18:21:52.975 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 9 2024-05-03T18:21:53.711 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:53 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:53.711 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:53 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:53.711 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:53 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:53.711 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:53 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T18:21:53.711 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:53 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T18:21:53.711 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:53 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:21:53.712 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:53 smithi033 ceph-mon[28330]: Deploying daemon mds.g on smithi033 2024-05-03T18:21:53.712 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:53 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/68590538' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:21:53.712 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:53 smithi033 ceph-mon[28330]: mds.? [v2:172.21.15.150:6838/4051648018,v1:172.21.15.150:6839/4051648018] up:boot 2024-05-03T18:21:53.712 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:53 smithi033 ceph-mon[28330]: fsmap 9 up:standby 2024-05-03T18:21:53.712 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:53 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-03T18:21:53.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:53 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:53.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:53 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:53.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:53 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:53.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:53 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T18:21:53.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:53 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T18:21:53.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:53 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:21:53.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:53 smithi150 ceph-mon[32492]: Deploying daemon mds.g on smithi033 2024-05-03T18:21:53.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:53 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/68590538' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:21:53.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:53 smithi150 ceph-mon[32492]: mds.? [v2:172.21.15.150:6838/4051648018,v1:172.21.15.150:6839/4051648018] up:boot 2024-05-03T18:21:53.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:53 smithi150 ceph-mon[32492]: fsmap 9 up:standby 2024-05-03T18:21:53.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:53 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-03T18:21:54.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:53 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:54.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:53 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:54.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:53 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:54.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:53 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T18:21:54.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:53 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T18:21:54.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:53 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:21:54.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:53 smithi099 ceph-mon[31658]: Deploying daemon mds.g on smithi033 2024-05-03T18:21:54.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:53 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/68590538' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:21:54.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:53 smithi099 ceph-mon[31658]: mds.? [v2:172.21.15.150:6838/4051648018,v1:172.21.15.150:6839/4051648018] up:boot 2024-05-03T18:21:54.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:53 smithi099 ceph-mon[31658]: fsmap 9 up:standby 2024-05-03T18:21:54.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:53 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-03T18:21:54.982 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:54 smithi099 ceph-mon[31658]: pgmap v348: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:21:54.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:54 smithi033 ceph-mon[28330]: pgmap v348: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:21:55.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:54 smithi150 ceph-mon[32492]: pgmap v348: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:21:55.777 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T18:21:56.234 INFO:journalctl@ceph.mds.g.smithi033.stdout:May 03 18:21:56 smithi033 systemd[1]: Starting Ceph mds.g for 5703bba4-0978-11ef-bc97-c7b262605968... 2024-05-03T18:21:56.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:56 smithi033 ceph-mon[28330]: pgmap v349: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:21:56.679 INFO:journalctl@ceph.mds.g.smithi033.stdout:May 03 18:21:56 smithi033 podman[70320]: 2024-05-03 18:21:56.296944108 +0000 UTC m=+0.036480631 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T18:21:56.679 INFO:journalctl@ceph.mds.g.smithi033.stdout:May 03 18:21:56 smithi033 podman[70320]: 2024-05-03 18:21:56.4042219 +0000 UTC m=+0.143758460 container create 518816162c381fd21b9339ea93ca2e88f33cdab2567beaf91724e80cd5042207 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-g, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe) 2024-05-03T18:21:56.679 INFO:journalctl@ceph.mds.g.smithi033.stdout:May 03 18:21:56 smithi033 podman[70320]: 2024-05-03 18:21:56.678310969 +0000 UTC m=+0.417847536 container init 518816162c381fd21b9339ea93ca2e88f33cdab2567beaf91724e80cd5042207 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-g, org.label-schema.build-date=20240430, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-05-03T18:21:56.825 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:21:56.980 INFO:journalctl@ceph.mds.g.smithi033.stdout:May 03 18:21:56 smithi033 podman[70320]: 2024-05-03 18:21:56.689141757 +0000 UTC m=+0.428678327 container start 518816162c381fd21b9339ea93ca2e88f33cdab2567beaf91724e80cd5042207 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-g, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-03T18:21:56.980 INFO:journalctl@ceph.mds.g.smithi033.stdout:May 03 18:21:56 smithi033 ceph-mds[70359]: -- 172.21.15.33:0/2229463274 <== mon.0 v2:172.21.15.33:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e7ff6eda40 con 0x55e7ff6c1000 2024-05-03T18:21:56.980 INFO:journalctl@ceph.mds.g.smithi033.stdout:May 03 18:21:56 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-g[70355]: starting mds.g at 2024-05-03T18:21:56.980 INFO:journalctl@ceph.mds.g.smithi033.stdout:May 03 18:21:56 smithi033 bash[70320]: 518816162c381fd21b9339ea93ca2e88f33cdab2567beaf91724e80cd5042207 2024-05-03T18:21:56.980 INFO:journalctl@ceph.mds.g.smithi033.stdout:May 03 18:21:56 smithi033 systemd[1]: Started Ceph mds.g for 5703bba4-0978-11ef-bc97-c7b262605968. 2024-05-03T18:21:57.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:56 smithi099 ceph-mon[31658]: pgmap v349: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:21:57.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:56 smithi150 ceph-mon[32492]: pgmap v349: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:21:58.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:57 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:58.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:57 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:58.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:57 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:58.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:57 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T18:21:58.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:57 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T18:21:58.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:57 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:21:58.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:57 smithi033 ceph-mon[28330]: Deploying daemon mds.a on smithi033 2024-05-03T18:21:58.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:57 smithi033 ceph-mon[28330]: pgmap v350: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:21:58.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:57 smithi033 ceph-mon[28330]: mds.? [v2:172.21.15.33:6838/442269281,v1:172.21.15.33:6839/442269281] up:boot 2024-05-03T18:21:58.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:57 smithi033 ceph-mon[28330]: fsmap 10 up:standby 2024-05-03T18:21:58.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:21:57 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-03T18:21:58.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:57 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:58.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:57 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:58.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:57 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:58.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:57 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T18:21:58.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:57 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T18:21:58.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:57 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:21:58.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:57 smithi099 ceph-mon[31658]: Deploying daemon mds.a on smithi033 2024-05-03T18:21:58.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:57 smithi099 ceph-mon[31658]: pgmap v350: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:21:58.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:57 smithi099 ceph-mon[31658]: mds.? [v2:172.21.15.33:6838/442269281,v1:172.21.15.33:6839/442269281] up:boot 2024-05-03T18:21:58.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:57 smithi099 ceph-mon[31658]: fsmap 10 up:standby 2024-05-03T18:21:58.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:21:57 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-03T18:21:58.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:57 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:58.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:57 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:58.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:57 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:21:58.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:57 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T18:21:58.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:57 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T18:21:58.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:57 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:21:58.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:57 smithi150 ceph-mon[32492]: Deploying daemon mds.a on smithi033 2024-05-03T18:21:58.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:57 smithi150 ceph-mon[32492]: pgmap v350: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:21:58.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:57 smithi150 ceph-mon[32492]: mds.? [v2:172.21.15.33:6838/442269281,v1:172.21.15.33:6839/442269281] up:boot 2024-05-03T18:21:58.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:57 smithi150 ceph-mon[32492]: fsmap 10 up:standby 2024-05-03T18:21:58.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:21:57 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-03T18:22:00.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:59.852+0000 7f6597000700 1 -- 172.21.15.33:0/1574245886 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6590109b10 msgr2=0x7f6590109f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:22:00.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:59.852+0000 7f6597000700 1 --2- 172.21.15.33:0/1574245886 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6590109b10 0x7f6590109f90 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f657c009990 tx=0x7f657c009ca0 comp rx=0 tx=0).stop 2024-05-03T18:22:00.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:59.853+0000 7f6597000700 1 -- 172.21.15.33:0/1574245886 shutdown_connections 2024-05-03T18:22:00.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:59.853+0000 7f6597000700 1 --2- 172.21.15.33:0/1574245886 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f659010a4d0 0x7f6590116990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:00.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:59.853+0000 7f6597000700 1 --2- 172.21.15.33:0/1574245886 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f659010fac0 0x7f659010fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:00.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:59.853+0000 7f6597000700 1 --2- 172.21.15.33:0/1574245886 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6590109b10 0x7f6590109f90 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:00.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:59.853+0000 7f6597000700 1 -- 172.21.15.33:0/1574245886 >> 172.21.15.33:0/1574245886 conn(0x7f6590076c70 msgr2=0x7f6590079090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:22:00.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:59.853+0000 7f6597000700 1 -- 172.21.15.33:0/1574245886 shutdown_connections 2024-05-03T18:22:00.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:59.853+0000 7f6597000700 1 -- 172.21.15.33:0/1574245886 wait complete. 2024-05-03T18:22:00.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:59.854+0000 7f6597000700 1 Processor -- start 2024-05-03T18:22:00.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:59.854+0000 7f6597000700 1 -- start start 2024-05-03T18:22:00.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:59.855+0000 7f6597000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6590109b10 0x7f65901a0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:22:00.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:59.855+0000 7f6597000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f659010a4d0 0x7f65901a05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:22:00.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:59.855+0000 7f6597000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f659010fac0 0x7f65901a4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:22:00.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:59.855+0000 7f6597000700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6590119c00 con 0x7f6590109b10 2024-05-03T18:22:00.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:59.855+0000 7f6597000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6590119a80 con 0x7f659010fac0 2024-05-03T18:22:00.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:59.855+0000 7f6597000700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6590119d80 con 0x7f659010a4d0 2024-05-03T18:22:00.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:59.855+0000 7f658fe00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6590109b10 0x7f65901a0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:22:00.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:59.855+0000 7f658fe00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6590109b10 0x7f65901a0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53174/0 (socket says 172.21.15.33:53174) 2024-05-03T18:22:00.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:59.855+0000 7f658fe00700 1 -- 172.21.15.33:0/1370995625 learned_addr learned my addr 172.21.15.33:0/1370995625 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:22:00.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:59.855+0000 7f6595000700 1 --2- 172.21.15.33:0/1370995625 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f659010fac0 0x7f65901a4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:22:00.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:59.855+0000 7f658f400700 1 --2- 172.21.15.33:0/1370995625 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f659010a4d0 0x7f65901a05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:22:00.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:59.856+0000 7f658fe00700 1 -- 172.21.15.33:0/1370995625 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f659010a4d0 msgr2=0x7f65901a05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:22:00.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:59.856+0000 7f658fe00700 1 --2- 172.21.15.33:0/1370995625 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f659010a4d0 0x7f65901a05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:00.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:59.856+0000 7f658fe00700 1 -- 172.21.15.33:0/1370995625 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f659010fac0 msgr2=0x7f65901a4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:22:00.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:59.856+0000 7f658fe00700 1 --2- 172.21.15.33:0/1370995625 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f659010fac0 0x7f65901a4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:00.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:59.856+0000 7f658fe00700 1 -- 172.21.15.33:0/1370995625 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65901a50a0 con 0x7f6590109b10 2024-05-03T18:22:00.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:59.856+0000 7f658f400700 1 --2- 172.21.15.33:0/1370995625 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f659010a4d0 0x7f65901a05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:22:00.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:59.856+0000 7f658fe00700 1 --2- 172.21.15.33:0/1370995625 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6590109b10 0x7f65901a0090 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f658000c920 tx=0x7f658000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:22:00.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:59.856+0000 7f658cc00700 1 -- 172.21.15.33:0/1370995625 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6580007b00 con 0x7f6590109b10 2024-05-03T18:22:00.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:59.856+0000 7f658cc00700 1 -- 172.21.15.33:0/1370995625 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6580004cc0 con 0x7f6590109b10 2024-05-03T18:22:00.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:59.856+0000 7f6597000700 1 -- 172.21.15.33:0/1370995625 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65901a5390 con 0x7f6590109b10 2024-05-03T18:22:00.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:59.856+0000 7f658cc00700 1 -- 172.21.15.33:0/1370995625 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65800054f0 con 0x7f6590109b10 2024-05-03T18:22:00.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:59.857+0000 7f6597000700 1 -- 172.21.15.33:0/1370995625 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65901acc10 con 0x7f6590109b10 2024-05-03T18:22:00.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:59.858+0000 7f6597000700 1 -- 172.21.15.33:0/1370995625 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f659006a190 con 0x7f6590109b10 2024-05-03T18:22:00.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:59.860+0000 7f658cc00700 1 -- 172.21.15.33:0/1370995625 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f6580007ca0 con 0x7f6590109b10 2024-05-03T18:22:00.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:59.862+0000 7f658cc00700 1 --2- 172.21.15.33:0/1370995625 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f65740b2900 0x7f65740b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:22:00.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:59.862+0000 7f658cc00700 1 -- 172.21.15.33:0/1370995625 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f65800e1970 con 0x7f6590109b10 2024-05-03T18:22:00.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:59.862+0000 7f658f400700 1 --2- 172.21.15.33:0/1370995625 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f65740b2900 0x7f65740b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:22:00.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:59.863+0000 7f658f400700 1 --2- 172.21.15.33:0/1370995625 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f65740b2900 0x7f65740b4dc0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f657c00b5c0 tx=0x7f657c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:22:00.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:21:59.866+0000 7f658cc00700 1 -- 172.21.15.33:0/1370995625 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f6580014030 con 0x7f6590109b10 2024-05-03T18:22:00.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:00.160+0000 7f6597000700 1 -- 172.21.15.33:0/1370995625 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f65901a1420 con 0x7f6590109b10 2024-05-03T18:22:00.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:00.161+0000 7f658cc00700 1 -- 172.21.15.33:0/1370995625 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8827 (secure 0 0 0) 0x7f65901a1420 con 0x7f6590109b10 2024-05-03T18:22:00.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:00.164+0000 7f6597000700 1 -- 172.21.15.33:0/1370995625 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f65740b2900 msgr2=0x7f65740b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:22:00.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:00.164+0000 7f6597000700 1 --2- 172.21.15.33:0/1370995625 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f65740b2900 0x7f65740b4dc0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f657c00b5c0 tx=0x7f657c005fb0 comp rx=0 tx=0).stop 2024-05-03T18:22:00.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:00.164+0000 7f6597000700 1 -- 172.21.15.33:0/1370995625 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6590109b10 msgr2=0x7f65901a0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:22:00.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:00.164+0000 7f6597000700 1 --2- 172.21.15.33:0/1370995625 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6590109b10 0x7f65901a0090 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f658000c920 tx=0x7f658000cce0 comp rx=0 tx=0).stop 2024-05-03T18:22:00.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:00.164+0000 7f6597000700 1 -- 172.21.15.33:0/1370995625 shutdown_connections 2024-05-03T18:22:00.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:00.164+0000 7f6597000700 1 --2- 172.21.15.33:0/1370995625 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f65740b2900 0x7f65740b4dc0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:00.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:00.164+0000 7f6597000700 1 --2- 172.21.15.33:0/1370995625 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f659010fac0 0x7f65901a4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:00.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:00.164+0000 7f6597000700 1 --2- 172.21.15.33:0/1370995625 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6590109b10 0x7f65901a0090 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:00.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:00.164+0000 7f6597000700 1 --2- 172.21.15.33:0/1370995625 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f659010a4d0 0x7f65901a05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:00.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:00.164+0000 7f6597000700 1 -- 172.21.15.33:0/1370995625 >> 172.21.15.33:0/1370995625 conn(0x7f6590076c70 msgr2=0x7f6590077560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:22:00.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:00.164+0000 7f6597000700 1 -- 172.21.15.33:0/1370995625 shutdown_connections 2024-05-03T18:22:00.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:00.165+0000 7f6597000700 1 -- 172.21.15.33:0/1370995625 wait complete. 2024-05-03T18:22:00.247 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 11 2024-05-03T18:22:00.248 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:22:00.248 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1327393503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1327393503},{"type":"v1","addr":"172.21.15.33:6835","nonce":1327393503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/2461472948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":2461472948},{"type":"v1","addr":"172.21.15.33:6837","nonce":2461472948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14559,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/442269281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":442269281},{"type":"v1","addr":"172.21.15.33:6839","nonce":442269281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/203598909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":203598909},{"type":"v1","addr":"172.21.15.150:6833","nonce":203598909}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/4029842493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":4029842493},{"type":"v1","addr":"172.21.15.150:6835","nonce":4029842493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3463664423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3463664423},{"type":"v1","addr":"172.21.15.150:6837","nonce":3463664423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24481,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6839/4051648018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":4051648018},{"type":"v1","addr":"172.21.15.150:6839","nonce":4051648018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34418,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1865105479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1865105479},{"type":"v1","addr":"172.21.15.99:6833","nonce":1865105479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34445,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/1206393914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1206393914},{"type":"v1","addr":"172.21.15.99:6835","nonce":1206393914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34472,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/3360970342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3360970342},{"type":"v1","addr":"172.21.15.99:6837","nonce":3360970342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-05-03T18:22:00.466 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:00 smithi033 ceph-mon[28330]: pgmap v351: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:22:00.466 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:00 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/1370995625' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:22:00.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:00 smithi150 ceph-mon[32492]: pgmap v351: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:22:00.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:00 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/1370995625' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:22:00.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:00 smithi099 ceph-mon[31658]: pgmap v351: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:22:00.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:00 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/1370995625' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:22:01.825 INFO:journalctl@ceph.mds.a.smithi033.stdout:May 03 18:22:01 smithi033 podman[70738]: 2024-05-03 18:22:01.524797413 +0000 UTC m=+0.172497353 container create dfbdd4791c709740fc8ffca5a634f24c2d64210b6b485d97633e117649e025cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430) 2024-05-03T18:22:01.825 INFO:journalctl@ceph.mds.a.smithi033.stdout:May 03 18:22:01 smithi033 podman[70738]: 2024-05-03 18:22:01.824216241 +0000 UTC m=+0.471916188 container init dfbdd4791c709740fc8ffca5a634f24c2d64210b6b485d97633e117649e025cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-a, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-05-03T18:22:02.234 INFO:journalctl@ceph.mds.a.smithi033.stdout:May 03 18:22:01 smithi033 podman[70738]: 2024-05-03 18:22:01.83403546 +0000 UTC m=+0.481735370 container start dfbdd4791c709740fc8ffca5a634f24c2d64210b6b485d97633e117649e025cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-a, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux ) 2024-05-03T18:22:02.234 INFO:journalctl@ceph.mds.a.smithi033.stdout:May 03 18:22:01 smithi033 ceph-mds[70757]: -- 172.21.15.33:0/1871897741 <== mon.0 v2:172.21.15.33:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560c708dfa40 con 0x560c708b2800 2024-05-03T18:22:02.234 INFO:journalctl@ceph.mds.a.smithi033.stdout:May 03 18:22:01 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-a[70753]: starting mds.a at 2024-05-03T18:22:02.234 INFO:journalctl@ceph.mds.a.smithi033.stdout:May 03 18:22:01 smithi033 bash[70738]: dfbdd4791c709740fc8ffca5a634f24c2d64210b6b485d97633e117649e025cb 2024-05-03T18:22:02.234 INFO:journalctl@ceph.mds.a.smithi033.stdout:May 03 18:22:01 smithi033 systemd[1]: Started Ceph mds.a for 5703bba4-0978-11ef-bc97-c7b262605968. 2024-05-03T18:22:02.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:02 smithi150 ceph-mon[32492]: pgmap v352: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:22:02.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:02 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:22:02.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:02 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:22:02.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:02 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:22:02.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:02 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T18:22:02.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:02 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T18:22:02.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:02 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:22:02.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:02 smithi033 ceph-mon[28330]: pgmap v352: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:22:02.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:02 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:22:02.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:02 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:22:02.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:02 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:22:02.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:02 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T18:22:02.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:02 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T18:22:02.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:02 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:22:02.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:02 smithi099 ceph-mon[31658]: pgmap v352: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:22:02.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:02 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:22:02.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:02 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:22:02.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:02 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:22:02.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:02 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T18:22:02.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:02 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T18:22:02.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:02 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:22:03.210 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T18:22:03.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:03 smithi150 ceph-mon[32492]: Deploying daemon mds.b on smithi099 2024-05-03T18:22:03.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:03 smithi150 ceph-mon[32492]: mds.? [v2:172.21.15.33:6840/645571756,v1:172.21.15.33:6841/645571756] up:boot 2024-05-03T18:22:03.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:03 smithi150 ceph-mon[32492]: fsmap 11 up:standby 2024-05-03T18:22:03.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:03 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-03T18:22:03.648 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:22:03.722 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:03 smithi033 ceph-mon[28330]: Deploying daemon mds.b on smithi099 2024-05-03T18:22:03.722 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:03 smithi033 ceph-mon[28330]: mds.? [v2:172.21.15.33:6840/645571756,v1:172.21.15.33:6841/645571756] up:boot 2024-05-03T18:22:03.723 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:03 smithi033 ceph-mon[28330]: fsmap 11 up:standby 2024-05-03T18:22:03.723 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:03 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-03T18:22:03.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:03 smithi099 ceph-mon[31658]: Deploying daemon mds.b on smithi099 2024-05-03T18:22:03.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:03 smithi099 ceph-mon[31658]: mds.? [v2:172.21.15.33:6840/645571756,v1:172.21.15.33:6841/645571756] up:boot 2024-05-03T18:22:03.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:03 smithi099 ceph-mon[31658]: fsmap 11 up:standby 2024-05-03T18:22:03.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:03 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-03T18:22:04.652 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:04 smithi099 ceph-mon[31658]: pgmap v353: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:22:04.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:04 smithi033 ceph-mon[28330]: pgmap v353: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:22:04.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:04 smithi150 ceph-mon[32492]: pgmap v353: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:22:05.545 INFO:journalctl@ceph.mds.b.smithi099.stdout:May 03 18:22:05 smithi099 systemd[1]: Starting Ceph mds.b for 5703bba4-0978-11ef-bc97-c7b262605968... 2024-05-03T18:22:05.907 INFO:journalctl@ceph.mds.b.smithi099.stdout:May 03 18:22:05 smithi099 podman[61983]: 2024-05-03 18:22:05.598115104 +0000 UTC m=+0.045544511 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T18:22:05.908 INFO:journalctl@ceph.mds.b.smithi099.stdout:May 03 18:22:05 smithi099 podman[61983]: 2024-05-03 18:22:05.734316806 +0000 UTC m=+0.181746188 container create 0372eed14b6b57538b9264c724ef39d4aa2e0051106cf2c623813ba5c9275660 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-b, org.label-schema.build-date=20240430, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-03T18:22:05.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:05.969+0000 7faf1a200700 1 -- 172.21.15.33:0/868711902 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf14103440 msgr2=0x7faf14103820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:22:05.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:05.969+0000 7faf1a200700 1 --2- 172.21.15.33:0/868711902 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf14103440 0x7faf14103820 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7faf08009990 tx=0x7faf08009ca0 comp rx=0 tx=0).stop 2024-05-03T18:22:05.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:05.969+0000 7faf1a200700 1 -- 172.21.15.33:0/868711902 shutdown_connections 2024-05-03T18:22:05.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:05.969+0000 7faf1a200700 1 --2- 172.21.15.33:0/868711902 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf1410c8c0 0x7faf1410ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:05.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:05.969+0000 7faf1a200700 1 --2- 172.21.15.33:0/868711902 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf14103440 0x7faf14103820 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:05.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:05.969+0000 7faf1a200700 1 --2- 172.21.15.33:0/868711902 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf14103df0 0x7faf1410c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:05.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:05.969+0000 7faf1a200700 1 -- 172.21.15.33:0/868711902 >> 172.21.15.33:0/868711902 conn(0x7faf14077140 msgr2=0x7faf14077550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:22:05.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:05.970+0000 7faf1a200700 1 -- 172.21.15.33:0/868711902 shutdown_connections 2024-05-03T18:22:05.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:05.970+0000 7faf1a200700 1 -- 172.21.15.33:0/868711902 wait complete. 2024-05-03T18:22:05.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:05.971+0000 7faf1a200700 1 Processor -- start 2024-05-03T18:22:05.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:05.971+0000 7faf1a200700 1 -- start start 2024-05-03T18:22:05.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:05.971+0000 7faf1a200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf14103440 0x7faf141042c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:22:05.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:05.971+0000 7faf1a200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf14103df0 0x7faf14104800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:22:05.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:05.971+0000 7faf1a200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf1410c8c0 0x7faf14104d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:22:05.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:05.971+0000 7faf1a200700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7faf14112380 con 0x7faf1410c8c0 2024-05-03T18:22:05.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:05.971+0000 7faf1a200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7faf14112200 con 0x7faf14103440 2024-05-03T18:22:05.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:05.971+0000 7faf1a200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7faf14112500 con 0x7faf14103df0 2024-05-03T18:22:05.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:05.972+0000 7faf19800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf1410c8c0 0x7faf14104d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:22:05.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:05.972+0000 7faf19800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf1410c8c0 0x7faf14104d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59936/0 (socket says 172.21.15.33:59936) 2024-05-03T18:22:05.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:05.972+0000 7faf19800700 1 -- 172.21.15.33:0/2253390914 learned_addr learned my addr 172.21.15.33:0/2253390914 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:22:05.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:05.972+0000 7faf18e00700 1 --2- 172.21.15.33:0/2253390914 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf14103440 0x7faf141042c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:22:05.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:05.972+0000 7faf13e00700 1 --2- 172.21.15.33:0/2253390914 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf14103df0 0x7faf14104800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:22:05.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:05.972+0000 7faf19800700 1 -- 172.21.15.33:0/2253390914 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf14103df0 msgr2=0x7faf14104800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:22:05.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:05.972+0000 7faf19800700 1 --2- 172.21.15.33:0/2253390914 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf14103df0 0x7faf14104800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:05.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:05.972+0000 7faf19800700 1 -- 172.21.15.33:0/2253390914 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf14103440 msgr2=0x7faf141042c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:22:05.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:05.972+0000 7faf19800700 1 --2- 172.21.15.33:0/2253390914 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf14103440 0x7faf141042c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:05.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:05.972+0000 7faf19800700 1 -- 172.21.15.33:0/2253390914 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf141a7eb0 con 0x7faf1410c8c0 2024-05-03T18:22:05.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:05.973+0000 7faf19800700 1 --2- 172.21.15.33:0/2253390914 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf1410c8c0 0x7faf14104d40 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7faf0c007ee0 tx=0x7faf0c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:22:05.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:05.973+0000 7faf11600700 1 -- 172.21.15.33:0/2253390914 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0c00dbd0 con 0x7faf1410c8c0 2024-05-03T18:22:05.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:05.973+0000 7faf11600700 1 -- 172.21.15.33:0/2253390914 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faf0c00dd70 con 0x7faf1410c8c0 2024-05-03T18:22:05.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:05.973+0000 7faf1a200700 1 -- 172.21.15.33:0/2253390914 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf141a81a0 con 0x7faf1410c8c0 2024-05-03T18:22:05.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:05.973+0000 7faf11600700 1 -- 172.21.15.33:0/2253390914 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0c0058d0 con 0x7faf1410c8c0 2024-05-03T18:22:05.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:05.973+0000 7faf1a200700 1 -- 172.21.15.33:0/2253390914 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf141a8730 con 0x7faf1410c8c0 2024-05-03T18:22:05.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:05.974+0000 7faf1a200700 1 -- 172.21.15.33:0/2253390914 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf1406a190 con 0x7faf1410c8c0 2024-05-03T18:22:05.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:05.977+0000 7faf11600700 1 -- 172.21.15.33:0/2253390914 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7faf0c01a020 con 0x7faf1410c8c0 2024-05-03T18:22:05.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:05.978+0000 7faf11600700 1 --2- 172.21.15.33:0/2253390914 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7faefc0b2900 0x7faefc0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:22:05.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:05.978+0000 7faf11600700 1 -- 172.21.15.33:0/2253390914 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7faf0c0e1490 con 0x7faf1410c8c0 2024-05-03T18:22:05.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:05.978+0000 7faf18e00700 1 --2- 172.21.15.33:0/2253390914 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7faefc0b2900 0x7faefc0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:22:05.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:05.979+0000 7faf18e00700 1 --2- 172.21.15.33:0/2253390914 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7faefc0b2900 0x7faefc0b4dc0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7faf0800b2e0 tx=0x7faf08005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:22:05.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:05.983+0000 7faf11600700 1 -- 172.21.15.33:0/2253390914 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7faf0c010070 con 0x7faf1410c8c0 2024-05-03T18:22:06.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:06.271+0000 7faf1a200700 1 -- 172.21.15.33:0/2253390914 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faf141a8990 con 0x7faf1410c8c0 2024-05-03T18:22:06.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:06.272+0000 7faf11600700 1 -- 172.21.15.33:0/2253390914 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9650 (secure 0 0 0) 0x7faf0c0a95b0 con 0x7faf1410c8c0 2024-05-03T18:22:06.273 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:22:06.273 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1327393503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1327393503},{"type":"v1","addr":"172.21.15.33:6835","nonce":1327393503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/2461472948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":2461472948},{"type":"v1","addr":"172.21.15.33:6837","nonce":2461472948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14559,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/442269281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":442269281},{"type":"v1","addr":"172.21.15.33:6839","nonce":442269281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14571,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/645571756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":645571756},{"type":"v1","addr":"172.21.15.33:6841","nonce":645571756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/203598909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":203598909},{"type":"v1","addr":"172.21.15.150:6833","nonce":203598909}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/4029842493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":4029842493},{"type":"v1","addr":"172.21.15.150:6835","nonce":4029842493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3463664423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3463664423},{"type":"v1","addr":"172.21.15.150:6837","nonce":3463664423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24481,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6839/4051648018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":4051648018},{"type":"v1","addr":"172.21.15.150:6839","nonce":4051648018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34418,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1865105479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1865105479},{"type":"v1","addr":"172.21.15.99:6833","nonce":1865105479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34445,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/1206393914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1206393914},{"type":"v1","addr":"172.21.15.99:6835","nonce":1206393914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34472,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/3360970342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3360970342},{"type":"v1","addr":"172.21.15.99:6837","nonce":3360970342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-05-03T18:22:06.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:06.275+0000 7faf1a200700 1 -- 172.21.15.33:0/2253390914 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7faefc0b2900 msgr2=0x7faefc0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:22:06.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:06.275+0000 7faf1a200700 1 --2- 172.21.15.33:0/2253390914 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7faefc0b2900 0x7faefc0b4dc0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7faf0800b2e0 tx=0x7faf08005fb0 comp rx=0 tx=0).stop 2024-05-03T18:22:06.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:06.275+0000 7faf1a200700 1 -- 172.21.15.33:0/2253390914 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf1410c8c0 msgr2=0x7faf14104d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:22:06.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:06.275+0000 7faf1a200700 1 --2- 172.21.15.33:0/2253390914 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf1410c8c0 0x7faf14104d40 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7faf0c007ee0 tx=0x7faf0c00d3b0 comp rx=0 tx=0).stop 2024-05-03T18:22:06.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:06.276+0000 7faf1a200700 1 -- 172.21.15.33:0/2253390914 shutdown_connections 2024-05-03T18:22:06.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:06.276+0000 7faf1a200700 1 --2- 172.21.15.33:0/2253390914 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf1410c8c0 0x7faf14104d40 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:06.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:06.276+0000 7faf1a200700 1 --2- 172.21.15.33:0/2253390914 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7faefc0b2900 0x7faefc0b4dc0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:06.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:06.276+0000 7faf1a200700 1 --2- 172.21.15.33:0/2253390914 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf14103440 0x7faf141042c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:06.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:06.276+0000 7faf1a200700 1 --2- 172.21.15.33:0/2253390914 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf14103df0 0x7faf14104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:06.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:06.276+0000 7faf1a200700 1 -- 172.21.15.33:0/2253390914 >> 172.21.15.33:0/2253390914 conn(0x7faf14077140 msgr2=0x7faf14102210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:22:06.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:06.276+0000 7faf1a200700 1 -- 172.21.15.33:0/2253390914 shutdown_connections 2024-05-03T18:22:06.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:06.276+0000 7faf1a200700 1 -- 172.21.15.33:0/2253390914 wait complete. 2024-05-03T18:22:06.277 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 12 2024-05-03T18:22:06.295 INFO:journalctl@ceph.mds.b.smithi099.stdout:May 03 18:22:06 smithi099 podman[61983]: 2024-05-03 18:22:06.031959192 +0000 UTC m=+0.479388581 container init 0372eed14b6b57538b9264c724ef39d4aa2e0051106cf2c623813ba5c9275660 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS) 2024-05-03T18:22:06.295 INFO:journalctl@ceph.mds.b.smithi099.stdout:May 03 18:22:06 smithi099 podman[61983]: 2024-05-03 18:22:06.040528394 +0000 UTC m=+0.487957809 container start 0372eed14b6b57538b9264c724ef39d4aa2e0051106cf2c623813ba5c9275660 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-b, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-05-03T18:22:06.295 INFO:journalctl@ceph.mds.b.smithi099.stdout:May 03 18:22:06 smithi099 ceph-mds[62010]: -- 172.21.15.99:0/705332070 <== mon.2 v2:172.21.15.99:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56331897da40 con 0x563318950800 2024-05-03T18:22:06.295 INFO:journalctl@ceph.mds.b.smithi099.stdout:May 03 18:22:06 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-b[62001]: starting mds.b at 2024-05-03T18:22:06.295 INFO:journalctl@ceph.mds.b.smithi099.stdout:May 03 18:22:06 smithi099 bash[61983]: 0372eed14b6b57538b9264c724ef39d4aa2e0051106cf2c623813ba5c9275660 2024-05-03T18:22:06.295 INFO:journalctl@ceph.mds.b.smithi099.stdout:May 03 18:22:06 smithi099 systemd[1]: Started Ceph mds.b for 5703bba4-0978-11ef-bc97-c7b262605968. 2024-05-03T18:22:06.703 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:06 smithi033 ceph-mon[28330]: pgmap v354: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:22:06.703 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:06 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:22:06.703 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:06 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:22:06.703 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:06 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/2253390914' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:22:06.703 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:06 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:22:06.703 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:06 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:22:06.703 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:06 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:22:06.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:06 smithi099 ceph-mon[31658]: pgmap v354: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:22:06.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:06 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:22:06.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:06 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:22:06.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:06 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/2253390914' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:22:06.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:06 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:22:06.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:06 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:22:06.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:06 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:22:06.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:06 smithi150 ceph-mon[32492]: pgmap v354: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:22:06.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:06 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:22:06.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:06 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:22:06.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:06 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/2253390914' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:22:06.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:06 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:22:06.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:06 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:22:06.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:06 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:22:07.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:07 smithi033 ceph-mon[28330]: mds.? [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] up:boot 2024-05-03T18:22:07.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:07 smithi033 ceph-mon[28330]: fsmap 12 up:standby 2024-05-03T18:22:07.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:07 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-03T18:22:07.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:07 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:22:07.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:07 smithi099 ceph-mon[31658]: mds.? [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] up:boot 2024-05-03T18:22:07.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:07 smithi099 ceph-mon[31658]: fsmap 12 up:standby 2024-05-03T18:22:07.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:07 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-03T18:22:07.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:07 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:22:07.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:07 smithi150 ceph-mon[32492]: mds.? [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] up:boot 2024-05-03T18:22:07.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:07 smithi150 ceph-mon[32492]: fsmap 12 up:standby 2024-05-03T18:22:07.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:07 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-03T18:22:07.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:07 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:22:08.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:08 smithi033 ceph-mon[28330]: pgmap v355: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:22:08.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:08 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:22:08.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:08 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:22:09.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:08 smithi099 ceph-mon[31658]: pgmap v355: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:22:09.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:08 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:22:09.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:08 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:22:09.117 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T18:22:09.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:08 smithi150 ceph-mon[32492]: pgmap v355: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:22:09.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:08 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:22:09.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:08 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:22:09.586 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:22:10.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:10 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:22:10.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:10 smithi099 ceph-mon[31658]: pgmap v356: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:22:10.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:10 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:22:10.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:10 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:22:10.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:10 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:22:10.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:10 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:22:10.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:10 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:22:10.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:10 smithi150 ceph-mon[32492]: pgmap v356: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:22:10.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:10 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:22:10.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:10 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:22:10.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:10 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:22:10.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:10 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:22:10.640 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:10 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:22:10.641 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:10 smithi033 ceph-mon[28330]: pgmap v356: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:22:10.641 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:10 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:22:10.641 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:10 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:22:10.641 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:10 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:22:10.641 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:10 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:22:11.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:11.790+0000 7fd603e00700 1 -- 172.21.15.33:0/2634975459 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd604102180 msgr2=0x7fd604102560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:22:11.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:11.790+0000 7fd603e00700 1 --2- 172.21.15.33:0/2634975459 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd604102180 0x7fd604102560 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd5f4009990 tx=0x7fd5f4009ca0 comp rx=0 tx=0).stop 2024-05-03T18:22:11.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:11.790+0000 7fd603e00700 1 -- 172.21.15.33:0/2634975459 shutdown_connections 2024-05-03T18:22:11.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:11.790+0000 7fd603e00700 1 --2- 172.21.15.33:0/2634975459 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd60410fd30 0x7fd604112120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:11.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:11.790+0000 7fd603e00700 1 --2- 172.21.15.33:0/2634975459 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd604102180 0x7fd604102560 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:11.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:11.790+0000 7fd603e00700 1 --2- 172.21.15.33:0/2634975459 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd604102b30 0x7fd60410f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:11.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:11.790+0000 7fd603e00700 1 -- 172.21.15.33:0/2634975459 >> 172.21.15.33:0/2634975459 conn(0x7fd6040fddc0 msgr2=0x7fd6041001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:22:11.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:11.790+0000 7fd603e00700 1 -- 172.21.15.33:0/2634975459 shutdown_connections 2024-05-03T18:22:11.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:11.790+0000 7fd603e00700 1 -- 172.21.15.33:0/2634975459 wait complete. 2024-05-03T18:22:11.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:11.791+0000 7fd603e00700 1 Processor -- start 2024-05-03T18:22:11.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:11.791+0000 7fd603e00700 1 -- start start 2024-05-03T18:22:11.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:11.792+0000 7fd603e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd604102180 0x7fd60419bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:22:11.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:11.792+0000 7fd603e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd604102b30 0x7fd60419c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:22:11.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:11.792+0000 7fd603e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd60410fd30 0x7fd6041a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:22:11.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:11.792+0000 7fd603e00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd6041157b0 con 0x7fd60410fd30 2024-05-03T18:22:11.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:11.792+0000 7fd603e00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd604115630 con 0x7fd604102180 2024-05-03T18:22:11.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:11.792+0000 7fd603e00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd604115930 con 0x7fd604102b30 2024-05-03T18:22:11.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:11.792+0000 7fd603400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd60410fd30 0x7fd6041a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:22:11.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:11.792+0000 7fd603400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd60410fd30 0x7fd6041a0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59952/0 (socket says 172.21.15.33:59952) 2024-05-03T18:22:11.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:11.792+0000 7fd603400700 1 -- 172.21.15.33:0/729329972 learned_addr learned my addr 172.21.15.33:0/729329972 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:22:11.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:11.792+0000 7fd602a00700 1 --2- 172.21.15.33:0/729329972 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd604102180 0x7fd60419bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:22:11.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:11.792+0000 7fd5fbe00700 1 --2- 172.21.15.33:0/729329972 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd604102b30 0x7fd60419c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:22:11.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:11.793+0000 7fd603400700 1 -- 172.21.15.33:0/729329972 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd604102b30 msgr2=0x7fd60419c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:22:11.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:11.793+0000 7fd603400700 1 --2- 172.21.15.33:0/729329972 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd604102b30 0x7fd60419c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:11.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:11.793+0000 7fd603400700 1 -- 172.21.15.33:0/729329972 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd604102180 msgr2=0x7fd60419bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:22:11.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:11.793+0000 7fd603400700 1 --2- 172.21.15.33:0/729329972 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd604102180 0x7fd60419bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:11.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:11.793+0000 7fd603400700 1 -- 172.21.15.33:0/729329972 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6041a0d60 con 0x7fd60410fd30 2024-05-03T18:22:11.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:11.793+0000 7fd603400700 1 --2- 172.21.15.33:0/729329972 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd60410fd30 0x7fd6041a0620 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc00b9e0 tx=0x7fd5fc00bcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:22:11.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:11.794+0000 7fd5fb400700 1 -- 172.21.15.33:0/729329972 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5fc00d8e0 con 0x7fd60410fd30 2024-05-03T18:22:11.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:11.794+0000 7fd5fb400700 1 -- 172.21.15.33:0/729329972 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd5fc015420 con 0x7fd60410fd30 2024-05-03T18:22:11.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:11.794+0000 7fd5fb400700 1 -- 172.21.15.33:0/729329972 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5fc014630 con 0x7fd60410fd30 2024-05-03T18:22:11.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:11.794+0000 7fd603e00700 1 -- 172.21.15.33:0/729329972 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6041a1050 con 0x7fd60410fd30 2024-05-03T18:22:11.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:11.794+0000 7fd603e00700 1 -- 172.21.15.33:0/729329972 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6041a88d0 con 0x7fd60410fd30 2024-05-03T18:22:11.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:11.795+0000 7fd603e00700 1 -- 172.21.15.33:0/729329972 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd60406a190 con 0x7fd60410fd30 2024-05-03T18:22:11.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:11.798+0000 7fd5fb400700 1 -- 172.21.15.33:0/729329972 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fd5fc0155d0 con 0x7fd60410fd30 2024-05-03T18:22:11.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:11.801+0000 7fd5fb400700 1 --2- 172.21.15.33:0/729329972 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fd5e80b2900 0x7fd5e80b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:22:11.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:11.801+0000 7fd5fb400700 1 -- 172.21.15.33:0/729329972 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7fd5fc0e19b0 con 0x7fd60410fd30 2024-05-03T18:22:11.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:11.801+0000 7fd602a00700 1 --2- 172.21.15.33:0/729329972 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fd5e80b2900 0x7fd5e80b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:22:11.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:11.802+0000 7fd602a00700 1 --2- 172.21.15.33:0/729329972 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fd5e80b2900 0x7fd5e80b4dc0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd5f4003760 tx=0x7fd5f400b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:22:11.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:11.804+0000 7fd5fb400700 1 -- 172.21.15.33:0/729329972 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fd5fc0a9ad0 con 0x7fd60410fd30 2024-05-03T18:22:12.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:12.113+0000 7fd603e00700 1 -- 172.21.15.33:0/729329972 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd60402e510 con 0x7fd60410fd30 2024-05-03T18:22:12.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:12.114+0000 7fd5fb400700 1 -- 172.21.15.33:0/729329972 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10473 (secure 0 0 0) 0x7fd5fc0a93d0 con 0x7fd60410fd30 2024-05-03T18:22:12.115 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:22:12.115 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1327393503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1327393503},{"type":"v1","addr":"172.21.15.33:6835","nonce":1327393503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/2461472948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":2461472948},{"type":"v1","addr":"172.21.15.33:6837","nonce":2461472948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14559,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/442269281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":442269281},{"type":"v1","addr":"172.21.15.33:6839","nonce":442269281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14571,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/645571756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":645571756},{"type":"v1","addr":"172.21.15.33:6841","nonce":645571756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/203598909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":203598909},{"type":"v1","addr":"172.21.15.150:6833","nonce":203598909}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/4029842493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":4029842493},{"type":"v1","addr":"172.21.15.150:6835","nonce":4029842493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3463664423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3463664423},{"type":"v1","addr":"172.21.15.150:6837","nonce":3463664423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24481,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6839/4051648018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":4051648018},{"type":"v1","addr":"172.21.15.150:6839","nonce":4051648018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34418,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1865105479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1865105479},{"type":"v1","addr":"172.21.15.99:6833","nonce":1865105479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34445,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/1206393914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1206393914},{"type":"v1","addr":"172.21.15.99:6835","nonce":1206393914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34472,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/3360970342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3360970342},{"type":"v1","addr":"172.21.15.99:6837","nonce":3360970342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":34508,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6839/919907308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":919907308},{"type":"v1","addr":"172.21.15.99:6839","nonce":919907308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-05-03T18:22:12.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:12.116+0000 7fd603e00700 1 -- 172.21.15.33:0/729329972 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fd5e80b2900 msgr2=0x7fd5e80b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:22:12.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:12.116+0000 7fd603e00700 1 --2- 172.21.15.33:0/729329972 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fd5e80b2900 0x7fd5e80b4dc0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd5f4003760 tx=0x7fd5f400b540 comp rx=0 tx=0).stop 2024-05-03T18:22:12.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:12.116+0000 7fd603e00700 1 -- 172.21.15.33:0/729329972 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd60410fd30 msgr2=0x7fd6041a0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:22:12.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:12.116+0000 7fd603e00700 1 --2- 172.21.15.33:0/729329972 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd60410fd30 0x7fd6041a0620 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc00b9e0 tx=0x7fd5fc00bcf0 comp rx=0 tx=0).stop 2024-05-03T18:22:12.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:12.117+0000 7fd603e00700 1 -- 172.21.15.33:0/729329972 shutdown_connections 2024-05-03T18:22:12.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:12.117+0000 7fd603e00700 1 --2- 172.21.15.33:0/729329972 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd60410fd30 0x7fd6041a0620 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:12.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:12.117+0000 7fd603e00700 1 --2- 172.21.15.33:0/729329972 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fd5e80b2900 0x7fd5e80b4dc0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:12.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:12.117+0000 7fd603e00700 1 --2- 172.21.15.33:0/729329972 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd604102180 0x7fd60419bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:12.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:12.117+0000 7fd603e00700 1 --2- 172.21.15.33:0/729329972 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd604102b30 0x7fd60419c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:12.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:12.117+0000 7fd603e00700 1 -- 172.21.15.33:0/729329972 >> 172.21.15.33:0/729329972 conn(0x7fd6040fddc0 msgr2=0x7fd6041001b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:22:12.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:12.117+0000 7fd603e00700 1 -- 172.21.15.33:0/729329972 shutdown_connections 2024-05-03T18:22:12.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:12.117+0000 7fd603e00700 1 -- 172.21.15.33:0/729329972 wait complete. 2024-05-03T18:22:12.118 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 13 2024-05-03T18:22:12.365 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:12 smithi033 ceph-mon[28330]: pgmap v357: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:22:12.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:12 smithi150 ceph-mon[32492]: pgmap v357: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:22:12.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:12 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:22:12.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:12 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:22:12.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:12 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/729329972' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:22:12.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:12 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:22:12.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:12 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:22:12.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:12 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/729329972' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:22:12.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:12 smithi099 ceph-mon[31658]: pgmap v357: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:22:12.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:12 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:22:12.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:12 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:22:12.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:12 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/729329972' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:22:12.860 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-05-03T18:22:13.317 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:22:13.465 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:13 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:22:13.465 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:13 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:22:13.465 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:13 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:22:13.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:13 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:22:13.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:13 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:22:13.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:13 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:22:13.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:13 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:22:13.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:13 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:22:13.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:13 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:22:14.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:14 smithi150 ceph-mon[32492]: pgmap v358: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:22:14.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:14 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:22:14.679 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:14 smithi033 ceph-mon[28330]: pgmap v358: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:22:14.679 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:14 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:22:14.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:14 smithi099 ceph-mon[31658]: pgmap v358: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:22:14.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:14 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:22:15.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:15.539+0000 7f96a5a00700 1 -- 172.21.15.33:0/2504515033 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96a010d9c0 msgr2=0x7f96a010fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:22:15.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:15.539+0000 7f96a5a00700 1 --2- 172.21.15.33:0/2504515033 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96a010d9c0 0x7f96a010fe80 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f969800b0a0 tx=0x7f969800b3b0 comp rx=0 tx=0).stop 2024-05-03T18:22:15.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:15.539+0000 7f96a5a00700 1 -- 172.21.15.33:0/2504515033 shutdown_connections 2024-05-03T18:22:15.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:15.539+0000 7f96a5a00700 1 --2- 172.21.15.33:0/2504515033 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96a010d9c0 0x7f96a010fe80 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:15.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:15.539+0000 7f96a5a00700 1 --2- 172.21.15.33:0/2504515033 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96a0104590 0x7f96a0104970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:15.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:15.539+0000 7f96a5a00700 1 --2- 172.21.15.33:0/2504515033 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f96a0104eb0 0x7f96a010d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:15.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:15.539+0000 7f96a5a00700 1 -- 172.21.15.33:0/2504515033 >> 172.21.15.33:0/2504515033 conn(0x7f96a00fddc0 msgr2=0x7f96a01001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:22:15.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:15.539+0000 7f96a5a00700 1 -- 172.21.15.33:0/2504515033 shutdown_connections 2024-05-03T18:22:15.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:15.540+0000 7f96a5a00700 1 -- 172.21.15.33:0/2504515033 wait complete. 2024-05-03T18:22:15.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:15.542+0000 7f96a5a00700 1 Processor -- start 2024-05-03T18:22:15.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:15.542+0000 7f96a5a00700 1 -- start start 2024-05-03T18:22:15.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:15.542+0000 7f96a5a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96a0104590 0x7f96a01021f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:22:15.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:15.542+0000 7f96a5a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f96a0104eb0 0x7f96a0102730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:22:15.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:15.542+0000 7f96a5a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96a010d9c0 0x7f96a0102c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:22:15.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:15.542+0000 7f96a5a00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f96a01134e0 con 0x7f96a010d9c0 2024-05-03T18:22:15.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:15.542+0000 7f96a5a00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f96a0113360 con 0x7f96a0104590 2024-05-03T18:22:15.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:15.542+0000 7f96a5a00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f96a0113660 con 0x7f96a0104eb0 2024-05-03T18:22:15.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:15.543+0000 7f969fe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96a0104590 0x7f96a01021f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:22:15.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:15.543+0000 7f969fe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96a0104590 0x7f96a01021f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:37906/0 (socket says 172.21.15.33:37906) 2024-05-03T18:22:15.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:15.543+0000 7f969fe00700 1 -- 172.21.15.33:0/4023943618 learned_addr learned my addr 172.21.15.33:0/4023943618 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:22:15.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:15.543+0000 7f96a5000700 1 --2- 172.21.15.33:0/4023943618 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96a010d9c0 0x7f96a0102c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:22:15.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:15.543+0000 7f969f400700 1 --2- 172.21.15.33:0/4023943618 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f96a0104eb0 0x7f96a0102730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:22:15.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:15.543+0000 7f969fe00700 1 -- 172.21.15.33:0/4023943618 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f96a0104eb0 msgr2=0x7f96a0102730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:22:15.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:15.543+0000 7f969fe00700 1 --2- 172.21.15.33:0/4023943618 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f96a0104eb0 0x7f96a0102730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:15.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:15.543+0000 7f969fe00700 1 -- 172.21.15.33:0/4023943618 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96a010d9c0 msgr2=0x7f96a0102c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:22:15.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:15.543+0000 7f969fe00700 1 --2- 172.21.15.33:0/4023943618 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96a010d9c0 0x7f96a0102c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:15.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:15.543+0000 7f969fe00700 1 -- 172.21.15.33:0/4023943618 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96a01a7f10 con 0x7f96a0104590 2024-05-03T18:22:15.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:15.544+0000 7f969fe00700 1 --2- 172.21.15.33:0/4023943618 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96a0104590 0x7f96a01021f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f969000e9a0 tx=0x7f969000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:22:15.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:15.544+0000 7f969cc00700 1 -- 172.21.15.33:0/4023943618 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f969000cbd0 con 0x7f96a0104590 2024-05-03T18:22:15.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:15.544+0000 7f96a5a00700 1 -- 172.21.15.33:0/4023943618 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96a01a81a0 con 0x7f96a0104590 2024-05-03T18:22:15.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:15.544+0000 7f969cc00700 1 -- 172.21.15.33:0/4023943618 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f969000cd70 con 0x7f96a0104590 2024-05-03T18:22:15.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:15.544+0000 7f969cc00700 1 -- 172.21.15.33:0/4023943618 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9690018680 con 0x7f96a0104590 2024-05-03T18:22:15.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:15.544+0000 7f96a5a00700 1 -- 172.21.15.33:0/4023943618 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96a01a8730 con 0x7f96a0104590 2024-05-03T18:22:15.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:15.545+0000 7f96a5a00700 1 -- 172.21.15.33:0/4023943618 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96a006a190 con 0x7f96a0104590 2024-05-03T18:22:15.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:15.548+0000 7f969cc00700 1 -- 172.21.15.33:0/4023943618 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f9690010430 con 0x7f96a0104590 2024-05-03T18:22:15.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:15.550+0000 7f969cc00700 1 --2- 172.21.15.33:0/4023943618 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f96840b2880 0x7f96840b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:22:15.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:15.550+0000 7f969cc00700 1 -- 172.21.15.33:0/4023943618 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f96900e23d0 con 0x7f96a0104590 2024-05-03T18:22:15.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:15.550+0000 7f969f400700 1 --2- 172.21.15.33:0/4023943618 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f96840b2880 0x7f96840b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:22:15.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:15.551+0000 7f969f400700 1 --2- 172.21.15.33:0/4023943618 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f96840b2880 0x7f96840b4d40 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f96a010ed00 tx=0x7f968c00b5d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:22:15.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:15.556+0000 7f969cc00700 1 -- 172.21.15.33:0/4023943618 <== mon.2 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f96900aa4f0 con 0x7f96a0104590 2024-05-03T18:22:15.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:15.799+0000 7f96a5a00700 1 -- 172.21.15.33:0/4023943618 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f96a002e510 con 0x7f96a0104590 2024-05-03T18:22:15.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:15.800+0000 7f969cc00700 1 -- 172.21.15.33:0/4023943618 <== mon.2 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v71) ==== 84+0+65 (secure 0 0 0) 0x7f96900a9df0 con 0x7f96a0104590 2024-05-03T18:22:15.800 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:22:15.800 INFO:teuthology.orchestra.run.smithi033.stdout:[ 2024-05-03T18:22:15.800 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:22:15.800 INFO:teuthology.orchestra.run.smithi033.stdout: "poolnum": 1, 2024-05-03T18:22:15.800 INFO:teuthology.orchestra.run.smithi033.stdout: "poolname": ".mgr" 2024-05-03T18:22:15.800 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:22:15.801 INFO:teuthology.orchestra.run.smithi033.stdout:] 2024-05-03T18:22:15.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:15.803+0000 7f96a5a00700 1 -- 172.21.15.33:0/4023943618 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f96840b2880 msgr2=0x7f96840b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:22:15.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:15.803+0000 7f96a5a00700 1 --2- 172.21.15.33:0/4023943618 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f96840b2880 0x7f96840b4d40 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f96a010ed00 tx=0x7f968c00b5d0 comp rx=0 tx=0).stop 2024-05-03T18:22:15.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:15.803+0000 7f96a5a00700 1 -- 172.21.15.33:0/4023943618 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96a0104590 msgr2=0x7f96a01021f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:22:15.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:15.803+0000 7f96a5a00700 1 --2- 172.21.15.33:0/4023943618 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96a0104590 0x7f96a01021f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f969000e9a0 tx=0x7f969000ecb0 comp rx=0 tx=0).stop 2024-05-03T18:22:15.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:15.803+0000 7f96a5a00700 1 -- 172.21.15.33:0/4023943618 shutdown_connections 2024-05-03T18:22:15.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:15.803+0000 7f96a5a00700 1 --2- 172.21.15.33:0/4023943618 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96a010d9c0 0x7f96a0102c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:15.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:15.803+0000 7f96a5a00700 1 --2- 172.21.15.33:0/4023943618 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f96840b2880 0x7f96840b4d40 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:15.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:15.803+0000 7f96a5a00700 1 --2- 172.21.15.33:0/4023943618 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96a0104590 0x7f96a01021f0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:15.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:15.803+0000 7f96a5a00700 1 --2- 172.21.15.33:0/4023943618 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f96a0104eb0 0x7f96a0102730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:15.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:15.803+0000 7f96a5a00700 1 -- 172.21.15.33:0/4023943618 >> 172.21.15.33:0/4023943618 conn(0x7f96a00fddc0 msgr2=0x7f96a00fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:22:15.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:15.804+0000 7f96a5a00700 1 -- 172.21.15.33:0/4023943618 shutdown_connections 2024-05-03T18:22:15.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:15.804+0000 7f96a5a00700 1 -- 172.21.15.33:0/4023943618 wait complete. 2024-05-03T18:22:16.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:16 smithi150 ceph-mon[32492]: pgmap v359: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:22:16.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:16 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/4023943618' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-03T18:22:16.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:16 smithi033 ceph-mon[28330]: pgmap v359: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:22:16.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:16 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/4023943618' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-03T18:22:16.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:16 smithi099 ceph-mon[31658]: pgmap v359: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:22:16.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:16 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/4023943618' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-03T18:22:16.961 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-03T18:22:16.961 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-03T18:22:17.424 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:22:17.746 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:17 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:22:17.747 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:17 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:22:17.747 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:17 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:22:17.747 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:17 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:22:17.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:17 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:22:17.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:17 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:22:17.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:17 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:22:17.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:17 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:22:17.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:17 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:22:17.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:17 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:22:17.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:17 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:22:17.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:17 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:22:18.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:18 smithi033 ceph-mon[28330]: pgmap v360: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:22:18.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:18 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:22:18.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:18 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:22:18.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:18 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:22:18.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:18 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:22:18.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:18 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:22:18.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:18 smithi099 ceph-mon[31658]: pgmap v360: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:22:18.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:18 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:22:18.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:18 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:22:18.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:18 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:22:18.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:18 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:22:18.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:18 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:22:18.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:18 smithi150 ceph-mon[32492]: pgmap v360: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:22:18.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:18 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:22:18.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:18 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:22:18.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:18 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:22:18.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:18 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:22:18.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:18 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:22:19.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:19.602+0000 7f0f11800700 1 -- 172.21.15.33:0/4154245457 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f0c10b6e0 msgr2=0x7f0f0c10bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:22:19.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:19.602+0000 7f0f11800700 1 --2- 172.21.15.33:0/4154245457 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f0c10b6e0 0x7f0f0c10bac0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f0efc009990 tx=0x7f0efc009ca0 comp rx=0 tx=0).stop 2024-05-03T18:22:19.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:19.602+0000 7f0f11800700 1 -- 172.21.15.33:0/4154245457 shutdown_connections 2024-05-03T18:22:19.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:19.602+0000 7f0f11800700 1 --2- 172.21.15.33:0/4154245457 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0f0c1060a0 0x7f0f0c1125b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:19.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:19.602+0000 7f0f11800700 1 --2- 172.21.15.33:0/4154245457 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f0c10b6e0 0x7f0f0c10bac0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:19.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:19.602+0000 7f0f11800700 1 --2- 172.21.15.33:0/4154245457 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f0c1056e0 0x7f0f0c105b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:19.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:19.602+0000 7f0f11800700 1 -- 172.21.15.33:0/4154245457 >> 172.21.15.33:0/4154245457 conn(0x7f0f0c1011d0 msgr2=0x7f0f0c1035f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:22:19.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:19.602+0000 7f0f11800700 1 -- 172.21.15.33:0/4154245457 shutdown_connections 2024-05-03T18:22:19.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:19.602+0000 7f0f11800700 1 -- 172.21.15.33:0/4154245457 wait complete. 2024-05-03T18:22:19.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:19.603+0000 7f0f11800700 1 Processor -- start 2024-05-03T18:22:19.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:19.604+0000 7f0f11800700 1 -- start start 2024-05-03T18:22:19.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:19.604+0000 7f0f11800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0f0c1056e0 0x7f0f0c1a1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:22:19.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:19.604+0000 7f0f11800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f0c1060a0 0x7f0f0c1a1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:22:19.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:19.604+0000 7f0f11800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f0c10b6e0 0x7f0f0c19baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:22:19.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:19.604+0000 7f0f11800700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0f0c115790 con 0x7f0f0c1060a0 2024-05-03T18:22:19.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:19.604+0000 7f0f11800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0f0c115610 con 0x7f0f0c10b6e0 2024-05-03T18:22:19.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:19.604+0000 7f0f11800700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f0f0c115910 con 0x7f0f0c1056e0 2024-05-03T18:22:19.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:19.604+0000 7f0f0a000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f0c1060a0 0x7f0f0c1a1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:22:19.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:19.604+0000 7f0f0a000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f0c1060a0 0x7f0f0c1a1e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43498/0 (socket says 172.21.15.33:43498) 2024-05-03T18:22:19.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:19.604+0000 7f0f0a000700 1 -- 172.21.15.33:0/3607038565 learned_addr learned my addr 172.21.15.33:0/3607038565 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:22:19.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:19.605+0000 7f0f0aa00700 1 --2- 172.21.15.33:0/3607038565 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0f0c1056e0 0x7f0f0c1a1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:22:19.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:19.605+0000 7f0f0a000700 1 -- 172.21.15.33:0/3607038565 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0f0c1056e0 msgr2=0x7f0f0c1a1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:22:19.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:19.605+0000 7f0f0a000700 1 --2- 172.21.15.33:0/3607038565 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0f0c1056e0 0x7f0f0c1a1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:19.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:19.605+0000 7f0f0a000700 1 -- 172.21.15.33:0/3607038565 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f0c10b6e0 msgr2=0x7f0f0c19baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:22:19.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:19.605+0000 7f0f0b400700 1 --2- 172.21.15.33:0/3607038565 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f0c10b6e0 0x7f0f0c19baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:22:19.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:19.605+0000 7f0f0a000700 1 --2- 172.21.15.33:0/3607038565 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f0c10b6e0 0x7f0f0c19baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:19.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:19.605+0000 7f0f0a000700 1 -- 172.21.15.33:0/3607038565 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f0c19c250 con 0x7f0f0c1060a0 2024-05-03T18:22:19.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:19.605+0000 7f0f0a000700 1 --2- 172.21.15.33:0/3607038565 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f0c1060a0 0x7f0f0c1a1e70 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0ef800d950 tx=0x7f0ef800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:22:19.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:19.605+0000 7f0f03400700 1 -- 172.21.15.33:0/3607038565 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef8016070 con 0x7f0f0c1060a0 2024-05-03T18:22:19.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:19.605+0000 7f0f03400700 1 -- 172.21.15.33:0/3607038565 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0ef8009dd0 con 0x7f0f0c1060a0 2024-05-03T18:22:19.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:19.605+0000 7f0f11800700 1 -- 172.21.15.33:0/3607038565 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f0c19c540 con 0x7f0f0c1060a0 2024-05-03T18:22:19.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:19.606+0000 7f0f03400700 1 -- 172.21.15.33:0/3607038565 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef800f3c0 con 0x7f0f0c1060a0 2024-05-03T18:22:19.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:19.606+0000 7f0f11800700 1 -- 172.21.15.33:0/3607038565 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f0c1a8800 con 0x7f0f0c1060a0 2024-05-03T18:22:19.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:19.607+0000 7f0f11800700 1 -- 172.21.15.33:0/3607038565 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f0c06a190 con 0x7f0f0c1060a0 2024-05-03T18:22:19.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:19.610+0000 7f0f03400700 1 -- 172.21.15.33:0/3607038565 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f0ef8019070 con 0x7f0f0c1060a0 2024-05-03T18:22:19.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:19.612+0000 7f0f03400700 1 --2- 172.21.15.33:0/3607038565 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f0ef00b2880 0x7f0ef00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:22:19.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:19.612+0000 7f0f03400700 1 -- 172.21.15.33:0/3607038565 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f0ef80e1a80 con 0x7f0f0c1060a0 2024-05-03T18:22:19.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:19.612+0000 7f0f0aa00700 1 --2- 172.21.15.33:0/3607038565 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f0ef00b2880 0x7f0ef00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:22:19.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:19.613+0000 7f0f0aa00700 1 --2- 172.21.15.33:0/3607038565 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f0ef00b2880 0x7f0ef00b4d40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0efc00b5c0 tx=0x7f0efc005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:22:19.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:19.616+0000 7f0f03400700 1 -- 172.21.15.33:0/3607038565 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f0ef80e3e20 con 0x7f0f0c1060a0 2024-05-03T18:22:19.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:19.861+0000 7f0f11800700 1 -- 172.21.15.33:0/3607038565 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f0f0c02e510 con 0x7f0f0c1060a0 2024-05-03T18:22:20.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:20.512+0000 7f0f03400700 1 -- 172.21.15.33:0/3607038565 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v72) ==== 138+0+0 (secure 0 0 0) 0x7f0ef80a9ba0 con 0x7f0f0c1060a0 2024-05-03T18:22:20.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:20.517+0000 7f0f11800700 1 -- 172.21.15.33:0/3607038565 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f0ef00b2880 msgr2=0x7f0ef00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:22:20.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:20.517+0000 7f0f11800700 1 --2- 172.21.15.33:0/3607038565 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f0ef00b2880 0x7f0ef00b4d40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0efc00b5c0 tx=0x7f0efc005960 comp rx=0 tx=0).stop 2024-05-03T18:22:20.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:20.517+0000 7f0f11800700 1 -- 172.21.15.33:0/3607038565 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f0c1060a0 msgr2=0x7f0f0c1a1e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:22:20.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:20.517+0000 7f0f11800700 1 --2- 172.21.15.33:0/3607038565 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f0c1060a0 0x7f0f0c1a1e70 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0ef800d950 tx=0x7f0ef800dc60 comp rx=0 tx=0).stop 2024-05-03T18:22:20.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:20.517+0000 7f0f11800700 1 -- 172.21.15.33:0/3607038565 shutdown_connections 2024-05-03T18:22:20.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:20.517+0000 7f0f11800700 1 --2- 172.21.15.33:0/3607038565 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f0ef00b2880 0x7f0ef00b4d40 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:20.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:20.517+0000 7f0f11800700 1 --2- 172.21.15.33:0/3607038565 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f0c10b6e0 0x7f0f0c19baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:20.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:20.517+0000 7f0f11800700 1 --2- 172.21.15.33:0/3607038565 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0f0c1056e0 0x7f0f0c1a1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:20.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:20.517+0000 7f0f11800700 1 --2- 172.21.15.33:0/3607038565 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f0c1060a0 0x7f0f0c1a1e70 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:20.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:20.517+0000 7f0f11800700 1 -- 172.21.15.33:0/3607038565 >> 172.21.15.33:0/3607038565 conn(0x7f0f0c1011d0 msgr2=0x7f0f0c1019c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:22:20.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:20.518+0000 7f0f11800700 1 -- 172.21.15.33:0/3607038565 shutdown_connections 2024-05-03T18:22:20.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:20.518+0000 7f0f11800700 1 -- 172.21.15.33:0/3607038565 wait complete. 2024-05-03T18:22:20.521 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_metadata' created 2024-05-03T18:22:20.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:20 smithi099 ceph-mon[31658]: pgmap v361: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:22:20.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:20 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/3607038565' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-03T18:22:20.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:20 smithi150 ceph-mon[32492]: pgmap v361: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:22:20.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:20 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/3607038565' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-03T18:22:20.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:20 smithi033 ceph-mon[28330]: pgmap v361: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:22:20.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:20 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/3607038565' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-03T18:22:21.511 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-05-03T18:22:21.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:21 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/3607038565' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-03T18:22:21.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:21 smithi033 ceph-mon[28330]: osdmap e72: 12 total, 12 up, 12 in 2024-05-03T18:22:21.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:21 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/3607038565' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-03T18:22:21.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:21 smithi099 ceph-mon[31658]: osdmap e72: 12 total, 12 up, 12 in 2024-05-03T18:22:21.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:21 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/3607038565' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-03T18:22:21.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:21 smithi150 ceph-mon[32492]: osdmap e72: 12 total, 12 up, 12 in 2024-05-03T18:22:21.973 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:22:22.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:22 smithi099 ceph-mon[31658]: pgmap v363: 33 pgs: 6 creating+peering, 1 active+clean, 26 unknown; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:22:22.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:22 smithi099 ceph-mon[31658]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-03T18:22:22.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:22 smithi099 ceph-mon[31658]: osdmap e73: 12 total, 12 up, 12 in 2024-05-03T18:22:22.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:22 smithi150 ceph-mon[32492]: pgmap v363: 33 pgs: 6 creating+peering, 1 active+clean, 26 unknown; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:22:22.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:22 smithi150 ceph-mon[32492]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-03T18:22:22.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:22 smithi150 ceph-mon[32492]: osdmap e73: 12 total, 12 up, 12 in 2024-05-03T18:22:22.936 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:22 smithi033 ceph-mon[28330]: pgmap v363: 33 pgs: 6 creating+peering, 1 active+clean, 26 unknown; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:22:22.936 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:22 smithi033 ceph-mon[28330]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-03T18:22:22.936 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:22 smithi033 ceph-mon[28330]: osdmap e73: 12 total, 12 up, 12 in 2024-05-03T18:22:23.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:23 smithi099 ceph-mon[31658]: osdmap e74: 12 total, 12 up, 12 in 2024-05-03T18:22:23.811 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:23 smithi033 ceph-mon[28330]: osdmap e74: 12 total, 12 up, 12 in 2024-05-03T18:22:23.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:23 smithi150 ceph-mon[32492]: osdmap e74: 12 total, 12 up, 12 in 2024-05-03T18:22:24.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:24.162+0000 7fe934a00700 1 -- 172.21.15.33:0/2265297318 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe93010fc60 msgr2=0x7fe930112070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:22:24.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:24.162+0000 7fe934a00700 1 --2- 172.21.15.33:0/2265297318 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe93010fc60 0x7fe930112070 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe92800b0a0 tx=0x7fe92800b3b0 comp rx=0 tx=0).stop 2024-05-03T18:22:24.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:24.162+0000 7fe934a00700 1 -- 172.21.15.33:0/2265297318 shutdown_connections 2024-05-03T18:22:24.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:24.162+0000 7fe934a00700 1 --2- 172.21.15.33:0/2265297318 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe93010fc60 0x7fe930112070 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:24.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:24.162+0000 7fe934a00700 1 --2- 172.21.15.33:0/2265297318 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe930075870 0x7fe930075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:24.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:24.162+0000 7fe934a00700 1 --2- 172.21.15.33:0/2265297318 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe930076190 0x7fe93010f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:24.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:24.162+0000 7fe934a00700 1 -- 172.21.15.33:0/2265297318 >> 172.21.15.33:0/2265297318 conn(0x7fe9300feef0 msgr2=0x7fe930101310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:22:24.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:24.162+0000 7fe934a00700 1 -- 172.21.15.33:0/2265297318 shutdown_connections 2024-05-03T18:22:24.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:24.162+0000 7fe934a00700 1 -- 172.21.15.33:0/2265297318 wait complete. 2024-05-03T18:22:24.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:24.164+0000 7fe934a00700 1 Processor -- start 2024-05-03T18:22:24.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:24.164+0000 7fe934a00700 1 -- start start 2024-05-03T18:22:24.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:24.164+0000 7fe934a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe930075870 0x7fe9301a1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:22:24.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:24.164+0000 7fe934a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe930076190 0x7fe9301a1da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:22:24.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:24.164+0000 7fe934a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe93010fc60 0x7fe93019ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:22:24.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:24.164+0000 7fe934a00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe9301156a0 con 0x7fe93010fc60 2024-05-03T18:22:24.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:24.164+0000 7fe934a00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe930115520 con 0x7fe930075870 2024-05-03T18:22:24.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:24.164+0000 7fe934a00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe930115820 con 0x7fe930076190 2024-05-03T18:22:24.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:24.165+0000 7fe92f200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe93010fc60 0x7fe93019ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:22:24.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:24.165+0000 7fe92f200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe93010fc60 0x7fe93019ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43506/0 (socket says 172.21.15.33:43506) 2024-05-03T18:22:24.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:24.165+0000 7fe92f200700 1 -- 172.21.15.33:0/1123726873 learned_addr learned my addr 172.21.15.33:0/1123726873 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:22:24.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:24.165+0000 7fe92e800700 1 --2- 172.21.15.33:0/1123726873 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe930075870 0x7fe9301a1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:22:24.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:24.165+0000 7fe925e00700 1 --2- 172.21.15.33:0/1123726873 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe930076190 0x7fe9301a1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:22:24.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:24.165+0000 7fe92f200700 1 -- 172.21.15.33:0/1123726873 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe930076190 msgr2=0x7fe9301a1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:22:24.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:24.165+0000 7fe92f200700 1 --2- 172.21.15.33:0/1123726873 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe930076190 0x7fe9301a1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:24.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:24.165+0000 7fe92f200700 1 -- 172.21.15.33:0/1123726873 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe930075870 msgr2=0x7fe9301a1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:22:24.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:24.165+0000 7fe92f200700 1 --2- 172.21.15.33:0/1123726873 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe930075870 0x7fe9301a1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:24.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:24.165+0000 7fe92f200700 1 -- 172.21.15.33:0/1123726873 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe93019c180 con 0x7fe93010fc60 2024-05-03T18:22:24.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:24.165+0000 7fe925e00700 1 --2- 172.21.15.33:0/1123726873 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe930076190 0x7fe9301a1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:22:24.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:24.166+0000 7fe92f200700 1 --2- 172.21.15.33:0/1123726873 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe93010fc60 0x7fe93019ba20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe928006070 tx=0x7fe928004e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:22:24.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:24.166+0000 7fe927e00700 1 -- 172.21.15.33:0/1123726873 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe928021070 con 0x7fe93010fc60 2024-05-03T18:22:24.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:24.166+0000 7fe927e00700 1 -- 172.21.15.33:0/1123726873 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe928007c00 con 0x7fe93010fc60 2024-05-03T18:22:24.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:24.166+0000 7fe927e00700 1 -- 172.21.15.33:0/1123726873 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9280047f0 con 0x7fe93010fc60 2024-05-03T18:22:24.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:24.166+0000 7fe934a00700 1 -- 172.21.15.33:0/1123726873 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe93019c410 con 0x7fe93010fc60 2024-05-03T18:22:24.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:24.166+0000 7fe934a00700 1 -- 172.21.15.33:0/1123726873 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9301a8730 con 0x7fe93010fc60 2024-05-03T18:22:24.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:24.167+0000 7fe934a00700 1 -- 172.21.15.33:0/1123726873 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe93006a190 con 0x7fe93010fc60 2024-05-03T18:22:24.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:24.170+0000 7fe927e00700 1 -- 172.21.15.33:0/1123726873 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fe928003f20 con 0x7fe93010fc60 2024-05-03T18:22:24.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:24.172+0000 7fe927e00700 1 --2- 172.21.15.33:0/1123726873 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fe9140b2900 0x7fe9140b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:22:24.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:24.172+0000 7fe927e00700 1 -- 172.21.15.33:0/1123726873 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7497+0+0 (secure 0 0 0) 0x7fe92800f070 con 0x7fe93010fc60 2024-05-03T18:22:24.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:24.172+0000 7fe92e800700 1 --2- 172.21.15.33:0/1123726873 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fe9140b2900 0x7fe9140b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:22:24.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:24.173+0000 7fe92e800700 1 --2- 172.21.15.33:0/1123726873 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fe9140b2900 0x7fe9140b4dc0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe920005950 tx=0x7fe9200058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:22:24.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:24.174+0000 7fe927e00700 1 -- 172.21.15.33:0/1123726873 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fe92802a020 con 0x7fe93010fc60 2024-05-03T18:22:24.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:24.414+0000 7fe934a00700 1 -- 172.21.15.33:0/1123726873 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7fe93002e510 con 0x7fe93010fc60 2024-05-03T18:22:24.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:24.550+0000 7fe927e00700 1 -- 172.21.15.33:0/1123726873 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v75) ==== 171+0+0 (secure 0 0 0) 0x7fe9280ab540 con 0x7fe93010fc60 2024-05-03T18:22:24.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:24.554+0000 7fe934a00700 1 -- 172.21.15.33:0/1123726873 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fe9140b2900 msgr2=0x7fe9140b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:22:24.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:24.554+0000 7fe934a00700 1 --2- 172.21.15.33:0/1123726873 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fe9140b2900 0x7fe9140b4dc0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe920005950 tx=0x7fe9200058e0 comp rx=0 tx=0).stop 2024-05-03T18:22:24.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:24.554+0000 7fe934a00700 1 -- 172.21.15.33:0/1123726873 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe93010fc60 msgr2=0x7fe93019ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:22:24.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:24.554+0000 7fe934a00700 1 --2- 172.21.15.33:0/1123726873 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe93010fc60 0x7fe93019ba20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe928006070 tx=0x7fe928004e00 comp rx=0 tx=0).stop 2024-05-03T18:22:24.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:24.555+0000 7fe934a00700 1 -- 172.21.15.33:0/1123726873 shutdown_connections 2024-05-03T18:22:24.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:24.555+0000 7fe934a00700 1 --2- 172.21.15.33:0/1123726873 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe93010fc60 0x7fe93019ba20 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:24.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:24.555+0000 7fe934a00700 1 --2- 172.21.15.33:0/1123726873 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fe9140b2900 0x7fe9140b4dc0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:24.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:24.555+0000 7fe934a00700 1 --2- 172.21.15.33:0/1123726873 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe930075870 0x7fe9301a1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:24.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:24.555+0000 7fe934a00700 1 --2- 172.21.15.33:0/1123726873 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe930076190 0x7fe9301a1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:24.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:24.555+0000 7fe934a00700 1 -- 172.21.15.33:0/1123726873 >> 172.21.15.33:0/1123726873 conn(0x7fe9300feef0 msgr2=0x7fe9300ffcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:22:24.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:24.555+0000 7fe934a00700 1 -- 172.21.15.33:0/1123726873 shutdown_connections 2024-05-03T18:22:24.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:24.555+0000 7fe934a00700 1 -- 172.21.15.33:0/1123726873 wait complete. 2024-05-03T18:22:24.556 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data' created 2024-05-03T18:22:24.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:24 smithi033 ceph-mon[28330]: pgmap v366: 33 pgs: 14 creating+peering, 13 active+clean, 6 unknown; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:22:24.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:24 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/1123726873' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-05-03T18:22:24.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:24 smithi150 ceph-mon[32492]: pgmap v366: 33 pgs: 14 creating+peering, 13 active+clean, 6 unknown; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:22:24.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:24 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/1123726873' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-05-03T18:22:25.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:24 smithi099 ceph-mon[31658]: pgmap v366: 33 pgs: 14 creating+peering, 13 active+clean, 6 unknown; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:22:25.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:24 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/1123726873' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-05-03T18:22:25.493 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-03T18:22:25.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:25 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/1123726873' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-05-03T18:22:25.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:25 smithi033 ceph-mon[28330]: osdmap e75: 12 total, 12 up, 12 in 2024-05-03T18:22:25.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:25 smithi033 ceph-mon[28330]: osdmap e76: 12 total, 12 up, 12 in 2024-05-03T18:22:25.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:25 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/1123726873' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-05-03T18:22:25.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:25 smithi150 ceph-mon[32492]: osdmap e75: 12 total, 12 up, 12 in 2024-05-03T18:22:25.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:25 smithi150 ceph-mon[32492]: osdmap e76: 12 total, 12 up, 12 in 2024-05-03T18:22:25.985 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:22:26.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:25 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/1123726873' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-05-03T18:22:26.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:25 smithi099 ceph-mon[31658]: osdmap e75: 12 total, 12 up, 12 in 2024-05-03T18:22:26.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:25 smithi099 ceph-mon[31658]: osdmap e76: 12 total, 12 up, 12 in 2024-05-03T18:22:26.646 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:26 smithi033 ceph-mon[28330]: pgmap v368: 97 pgs: 30 creating+peering, 19 active+clean, 48 unknown; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:22:26.646 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:26 smithi033 ceph-mon[28330]: osdmap e77: 12 total, 12 up, 12 in 2024-05-03T18:22:26.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:26 smithi150 ceph-mon[32492]: pgmap v368: 97 pgs: 30 creating+peering, 19 active+clean, 48 unknown; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:22:26.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:26 smithi150 ceph-mon[32492]: osdmap e77: 12 total, 12 up, 12 in 2024-05-03T18:22:27.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:26 smithi099 ceph-mon[31658]: pgmap v368: 97 pgs: 30 creating+peering, 19 active+clean, 48 unknown; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:22:27.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:26 smithi099 ceph-mon[31658]: osdmap e77: 12 total, 12 up, 12 in 2024-05-03T18:22:27.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:27 smithi150 ceph-mon[32492]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-03T18:22:27.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:27 smithi033 ceph-mon[28330]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-03T18:22:28.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:27 smithi099 ceph-mon[31658]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-03T18:22:28.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:28.531+0000 7f270a400700 1 -- 172.21.15.33:0/3181680367 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2704075ff0 msgr2=0x7f270410f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:22:28.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:28.531+0000 7f270a400700 1 --2- 172.21.15.33:0/3181680367 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2704075ff0 0x7f270410f960 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f26f0009990 tx=0x7f26f0009ca0 comp rx=0 tx=0).stop 2024-05-03T18:22:28.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:28.531+0000 7f270a400700 1 -- 172.21.15.33:0/3181680367 shutdown_connections 2024-05-03T18:22:28.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:28.531+0000 7f270a400700 1 --2- 172.21.15.33:0/3181680367 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f270410fea0 0x7f2704112290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:28.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:28.531+0000 7f270a400700 1 --2- 172.21.15.33:0/3181680367 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f27040756d0 0x7f2704075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:28.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:28.531+0000 7f270a400700 1 --2- 172.21.15.33:0/3181680367 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2704075ff0 0x7f270410f960 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:28.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:28.531+0000 7f270a400700 1 -- 172.21.15.33:0/3181680367 >> 172.21.15.33:0/3181680367 conn(0x7f270407a600 msgr2=0x7f270407aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:22:28.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:28.532+0000 7f270a400700 1 -- 172.21.15.33:0/3181680367 shutdown_connections 2024-05-03T18:22:28.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:28.532+0000 7f270a400700 1 -- 172.21.15.33:0/3181680367 wait complete. 2024-05-03T18:22:28.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:28.533+0000 7f270a400700 1 Processor -- start 2024-05-03T18:22:28.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:28.534+0000 7f270a400700 1 -- start start 2024-05-03T18:22:28.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:28.534+0000 7f270a400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f27040756d0 0x7f27041a18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:22:28.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:28.535+0000 7f270a400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2704075ff0 0x7f27041a1e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:22:28.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:28.535+0000 7f270a400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f270410fea0 0x7f270419baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:22:28.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:28.535+0000 7f270a400700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2704115720 con 0x7f270410fea0 2024-05-03T18:22:28.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:28.535+0000 7f270a400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f27041155a0 con 0x7f2704075ff0 2024-05-03T18:22:28.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:28.535+0000 7f270a400700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f27041158a0 con 0x7f27040756d0 2024-05-03T18:22:28.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:28.535+0000 7f2703e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f270410fea0 0x7f270419baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:22:28.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:28.535+0000 7f2703e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f270410fea0 0x7f270419baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37934/0 (socket says 172.21.15.33:37934) 2024-05-03T18:22:28.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:28.535+0000 7f2703e00700 1 -- 172.21.15.33:0/3257706979 learned_addr learned my addr 172.21.15.33:0/3257706979 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:22:28.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:28.535+0000 7f2702a00700 1 --2- 172.21.15.33:0/3257706979 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2704075ff0 0x7f27041a1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:22:28.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:28.535+0000 7f2703400700 1 --2- 172.21.15.33:0/3257706979 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f27040756d0 0x7f27041a18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:22:28.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:28.536+0000 7f2703e00700 1 -- 172.21.15.33:0/3257706979 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f27040756d0 msgr2=0x7f27041a18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:22:28.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:28.536+0000 7f2703e00700 1 --2- 172.21.15.33:0/3257706979 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f27040756d0 0x7f27041a18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:28.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:28.536+0000 7f2703e00700 1 -- 172.21.15.33:0/3257706979 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2704075ff0 msgr2=0x7f27041a1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:22:28.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:28.536+0000 7f2703e00700 1 --2- 172.21.15.33:0/3257706979 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2704075ff0 0x7f27041a1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:28.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:28.536+0000 7f2703e00700 1 -- 172.21.15.33:0/3257706979 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f270419c200 con 0x7f270410fea0 2024-05-03T18:22:28.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:28.536+0000 7f2703e00700 1 --2- 172.21.15.33:0/3257706979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f270410fea0 0x7f270419baa0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f26fc007ec0 tx=0x7f26fc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:22:28.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:28.536+0000 7f26fbe00700 1 -- 172.21.15.33:0/3257706979 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26fc00dbd0 con 0x7f270410fea0 2024-05-03T18:22:28.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:28.536+0000 7f270a400700 1 -- 172.21.15.33:0/3257706979 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f270419c4f0 con 0x7f270410fea0 2024-05-03T18:22:28.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:28.536+0000 7f26fbe00700 1 -- 172.21.15.33:0/3257706979 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f26fc00dd70 con 0x7f270410fea0 2024-05-03T18:22:28.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:28.536+0000 7f26fbe00700 1 -- 172.21.15.33:0/3257706979 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26fc004400 con 0x7f270410fea0 2024-05-03T18:22:28.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:28.536+0000 7f270a400700 1 -- 172.21.15.33:0/3257706979 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27041a87b0 con 0x7f270410fea0 2024-05-03T18:22:28.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:28.537+0000 7f270a400700 1 -- 172.21.15.33:0/3257706979 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f270406a190 con 0x7f270410fea0 2024-05-03T18:22:28.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:28.540+0000 7f26fbe00700 1 -- 172.21.15.33:0/3257706979 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f26fc01a030 con 0x7f270410fea0 2024-05-03T18:22:28.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:28.543+0000 7f26fbe00700 1 --2- 172.21.15.33:0/3257706979 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f26e80b2880 0x7f26e80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:22:28.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:28.543+0000 7f26fbe00700 1 -- 172.21.15.33:0/3257706979 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7879+0+0 (secure 0 0 0) 0x7f26fc0e1d20 con 0x7f270410fea0 2024-05-03T18:22:28.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:28.543+0000 7f2703400700 1 --2- 172.21.15.33:0/3257706979 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f26e80b2880 0x7f26e80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:22:28.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:28.544+0000 7f2703400700 1 --2- 172.21.15.33:0/3257706979 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f26e80b2880 0x7f26e80b4d40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f26f4005950 tx=0x7f26f40058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:22:28.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:28.546+0000 7f26fbe00700 1 -- 172.21.15.33:0/3257706979 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f26fc021020 con 0x7f270410fea0 2024-05-03T18:22:28.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:28.835+0000 7f270a400700 1 -- 172.21.15.33:0/3257706979 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f270402e510 con 0x7f270410fea0 2024-05-03T18:22:28.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:28 smithi150 ceph-mon[32492]: pgmap v371: 97 pgs: 7 creating+activating, 16 creating+peering, 46 active+clean, 28 unknown; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:22:28.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:28 smithi033 ceph-mon[28330]: pgmap v371: 97 pgs: 7 creating+activating, 16 creating+peering, 46 active+clean, 28 unknown; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:22:29.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:28 smithi099 ceph-mon[31658]: pgmap v371: 97 pgs: 7 creating+activating, 16 creating+peering, 46 active+clean, 28 unknown; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:22:29.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:29.640+0000 7f26fbe00700 1 -- 172.21.15.33:0/3257706979 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f26fc010070 con 0x7f270410fea0 2024-05-03T18:22:29.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:29.644+0000 7f270a400700 1 -- 172.21.15.33:0/3257706979 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f26e80b2880 msgr2=0x7f26e80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:22:29.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:29.644+0000 7f270a400700 1 --2- 172.21.15.33:0/3257706979 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f26e80b2880 0x7f26e80b4d40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f26f4005950 tx=0x7f26f40058e0 comp rx=0 tx=0).stop 2024-05-03T18:22:29.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:29.644+0000 7f270a400700 1 -- 172.21.15.33:0/3257706979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f270410fea0 msgr2=0x7f270419baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:22:29.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:29.644+0000 7f270a400700 1 --2- 172.21.15.33:0/3257706979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f270410fea0 0x7f270419baa0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f26fc007ec0 tx=0x7f26fc00d3b0 comp rx=0 tx=0).stop 2024-05-03T18:22:29.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:29.644+0000 7f270a400700 1 -- 172.21.15.33:0/3257706979 shutdown_connections 2024-05-03T18:22:29.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:29.644+0000 7f270a400700 1 --2- 172.21.15.33:0/3257706979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f270410fea0 0x7f270419baa0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:29.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:29.644+0000 7f270a400700 1 --2- 172.21.15.33:0/3257706979 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f27040756d0 0x7f27041a18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:29.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:29.644+0000 7f270a400700 1 --2- 172.21.15.33:0/3257706979 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f26e80b2880 0x7f26e80b4d40 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:29.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:29.645+0000 7f270a400700 1 --2- 172.21.15.33:0/3257706979 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2704075ff0 0x7f27041a1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:29.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:29.645+0000 7f270a400700 1 -- 172.21.15.33:0/3257706979 >> 172.21.15.33:0/3257706979 conn(0x7f270407a600 msgr2=0x7f27041007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:22:29.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:29.645+0000 7f270a400700 1 -- 172.21.15.33:0/3257706979 shutdown_connections 2024-05-03T18:22:29.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:29.645+0000 7f270a400700 1 -- 172.21.15.33:0/3257706979 wait complete. 2024-05-03T18:22:29.646 INFO:teuthology.orchestra.run.smithi033.stderr:new fs with metadata pool 2 and data pool 3 2024-05-03T18:22:29.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:29 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/3257706979' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-03T18:22:29.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:29 smithi150 ceph-mon[32492]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-03T18:22:29.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:29 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/3257706979' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-03T18:22:29.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:29 smithi033 ceph-mon[28330]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-03T18:22:30.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:29 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/3257706979' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-03T18:22:30.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:29 smithi099 ceph-mon[31658]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-03T18:22:30.601 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-05-03T18:22:30.602 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-05-03T18:22:30.684 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:30 smithi033 ceph-mon[28330]: pgmap v372: 97 pgs: 7 creating+activating, 16 creating+peering, 74 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:22:30.684 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:30 smithi033 ceph-mon[28330]: osdmap e78: 12 total, 12 up, 12 in 2024-05-03T18:22:30.684 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:30 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/3257706979' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-03T18:22:30.684 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:30 smithi033 ceph-mon[28330]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-03T18:22:30.684 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:30 smithi033 ceph-mon[28330]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-03T18:22:30.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:30 smithi150 ceph-mon[32492]: pgmap v372: 97 pgs: 7 creating+activating, 16 creating+peering, 74 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:22:30.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:30 smithi150 ceph-mon[32492]: osdmap e78: 12 total, 12 up, 12 in 2024-05-03T18:22:30.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:30 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/3257706979' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-03T18:22:30.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:30 smithi150 ceph-mon[32492]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-03T18:22:30.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:30 smithi150 ceph-mon[32492]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-03T18:22:31.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:30 smithi099 ceph-mon[31658]: pgmap v372: 97 pgs: 7 creating+activating, 16 creating+peering, 74 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:22:31.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:30 smithi099 ceph-mon[31658]: osdmap e78: 12 total, 12 up, 12 in 2024-05-03T18:22:31.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:30 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/3257706979' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-03T18:22:31.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:30 smithi099 ceph-mon[31658]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-03T18:22:31.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:30 smithi099 ceph-mon[31658]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-03T18:22:31.096 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:22:31.795 INFO:journalctl@ceph.mds.b.smithi099.stdout:May 03 18:22:31 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-b[62001]: 2024-05-03T18:22:31.419+0000 7fc61ee00700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-03T18:22:31.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:31 smithi099 ceph-mon[31658]: mds.? [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] up:active 2024-05-03T18:22:31.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:31 smithi099 ceph-mon[31658]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-03T18:22:31.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:31 smithi099 ceph-mon[31658]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-03T18:22:31.848 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:31 smithi033 ceph-mon[28330]: mds.? [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] up:active 2024-05-03T18:22:31.848 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:31 smithi033 ceph-mon[28330]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-03T18:22:31.848 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:31 smithi033 ceph-mon[28330]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-03T18:22:32.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:31 smithi150 ceph-mon[32492]: mds.? [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] up:active 2024-05-03T18:22:32.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:31 smithi150 ceph-mon[32492]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-03T18:22:32.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:31 smithi150 ceph-mon[32492]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-03T18:22:32.937 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:32 smithi033 ceph-mon[28330]: pgmap v374: 97 pgs: 97 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 3 op/s 2024-05-03T18:22:33.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:32 smithi033 ceph-mon[28330]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-03T18:22:33.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:32 smithi033 ceph-mon[28330]: Cluster is now healthy 2024-05-03T18:22:33.034 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:32 smithi150 ceph-mon[32492]: pgmap v374: 97 pgs: 97 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 3 op/s 2024-05-03T18:22:33.034 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:32 smithi150 ceph-mon[32492]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-03T18:22:33.034 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:32 smithi150 ceph-mon[32492]: Cluster is now healthy 2024-05-03T18:22:33.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:32 smithi099 ceph-mon[31658]: pgmap v374: 97 pgs: 97 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 3 op/s 2024-05-03T18:22:33.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:32 smithi099 ceph-mon[31658]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-03T18:22:33.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:32 smithi099 ceph-mon[31658]: Cluster is now healthy 2024-05-03T18:22:33.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:33.499+0000 7fe3cdc00700 1 -- 172.21.15.33:0/4253470134 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3c81033d0 msgr2=0x7fe3c8103850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:22:33.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:33.499+0000 7fe3cdc00700 1 --2- 172.21.15.33:0/4253470134 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3c81033d0 0x7fe3c8103850 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe3b4009990 tx=0x7fe3b4009ca0 comp rx=0 tx=0).stop 2024-05-03T18:22:33.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:33.500+0000 7fe3cdc00700 1 -- 172.21.15.33:0/4253470134 shutdown_connections 2024-05-03T18:22:33.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:33.500+0000 7fe3cdc00700 1 --2- 172.21.15.33:0/4253470134 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3c8103d90 0x7fe3c81102a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:33.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:33.500+0000 7fe3cdc00700 1 --2- 172.21.15.33:0/4253470134 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe3c8109460 0x7fe3c8109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:33.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:33.500+0000 7fe3cdc00700 1 --2- 172.21.15.33:0/4253470134 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3c81033d0 0x7fe3c8103850 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:33.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:33.500+0000 7fe3cdc00700 1 -- 172.21.15.33:0/4253470134 >> 172.21.15.33:0/4253470134 conn(0x7fe3c80fef20 msgr2=0x7fe3c8101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:22:33.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:33.500+0000 7fe3cdc00700 1 -- 172.21.15.33:0/4253470134 shutdown_connections 2024-05-03T18:22:33.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:33.500+0000 7fe3cdc00700 1 -- 172.21.15.33:0/4253470134 wait complete. 2024-05-03T18:22:33.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:33.501+0000 7fe3cdc00700 1 Processor -- start 2024-05-03T18:22:33.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:33.501+0000 7fe3cdc00700 1 -- start start 2024-05-03T18:22:33.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:33.502+0000 7fe3cdc00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3c81033d0 0x7fe3c819f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:22:33.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:33.502+0000 7fe3cdc00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3c8103d90 0x7fe3c819fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:22:33.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:33.502+0000 7fe3cdc00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe3c8109460 0x7fe3c81997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:22:33.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:33.502+0000 7fe3cdc00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c8113440 con 0x7fe3c81033d0 2024-05-03T18:22:33.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:33.502+0000 7fe3cdc00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c81132c0 con 0x7fe3c8103d90 2024-05-03T18:22:33.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:33.502+0000 7fe3cdc00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c81135c0 con 0x7fe3c8109460 2024-05-03T18:22:33.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:33.502+0000 7fe3c7e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3c81033d0 0x7fe3c819f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:22:33.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:33.502+0000 7fe3c7e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3c81033d0 0x7fe3c819f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37958/0 (socket says 172.21.15.33:37958) 2024-05-03T18:22:33.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:33.502+0000 7fe3c7e00700 1 -- 172.21.15.33:0/3728810319 learned_addr learned my addr 172.21.15.33:0/3728810319 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:22:33.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:33.503+0000 7fe3c7400700 1 --2- 172.21.15.33:0/3728810319 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3c8103d90 0x7fe3c819fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:22:33.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:33.503+0000 7fe3cd200700 1 --2- 172.21.15.33:0/3728810319 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe3c8109460 0x7fe3c81997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:22:33.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:33.503+0000 7fe3c7e00700 1 -- 172.21.15.33:0/3728810319 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe3c8109460 msgr2=0x7fe3c81997c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:22:33.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:33.503+0000 7fe3c7e00700 1 --2- 172.21.15.33:0/3728810319 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe3c8109460 0x7fe3c81997c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:33.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:33.503+0000 7fe3c7e00700 1 -- 172.21.15.33:0/3728810319 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3c8103d90 msgr2=0x7fe3c819fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:22:33.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:33.503+0000 7fe3c7e00700 1 --2- 172.21.15.33:0/3728810319 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3c8103d90 0x7fe3c819fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:33.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:33.503+0000 7fe3c7e00700 1 -- 172.21.15.33:0/3728810319 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3c8199fa0 con 0x7fe3c81033d0 2024-05-03T18:22:33.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:33.503+0000 7fe3c7400700 1 --2- 172.21.15.33:0/3728810319 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3c8103d90 0x7fe3c819fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:22:33.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:33.503+0000 7fe3c7e00700 1 --2- 172.21.15.33:0/3728810319 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3c81033d0 0x7fe3c819f640 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe3b800d980 tx=0x7fe3b800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:22:33.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:33.503+0000 7fe3c4c00700 1 -- 172.21.15.33:0/3728810319 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b8017070 con 0x7fe3c81033d0 2024-05-03T18:22:33.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:33.504+0000 7fe3c4c00700 1 -- 172.21.15.33:0/3728810319 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe3b8004830 con 0x7fe3c81033d0 2024-05-03T18:22:33.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:33.504+0000 7fe3c4c00700 1 -- 172.21.15.33:0/3728810319 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b80053a0 con 0x7fe3c81033d0 2024-05-03T18:22:33.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:33.504+0000 7fe3cdc00700 1 -- 172.21.15.33:0/3728810319 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3c819a290 con 0x7fe3c81033d0 2024-05-03T18:22:33.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:33.504+0000 7fe3cdc00700 1 -- 172.21.15.33:0/3728810319 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3c81a64a0 con 0x7fe3c81033d0 2024-05-03T18:22:33.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:33.505+0000 7fe3cdc00700 1 -- 172.21.15.33:0/3728810319 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3c8104b10 con 0x7fe3c81033d0 2024-05-03T18:22:33.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:33.509+0000 7fe3c4c00700 1 -- 172.21.15.33:0/3728810319 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fe3b80049e0 con 0x7fe3c81033d0 2024-05-03T18:22:33.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:33.510+0000 7fe3c4c00700 1 --2- 172.21.15.33:0/3728810319 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fe3ac0b2900 0x7fe3ac0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:22:33.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:33.511+0000 7fe3c4c00700 1 -- 172.21.15.33:0/3728810319 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7fe3b80e2430 con 0x7fe3c81033d0 2024-05-03T18:22:33.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:33.511+0000 7fe3c7400700 1 --2- 172.21.15.33:0/3728810319 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fe3ac0b2900 0x7fe3ac0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:22:33.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:33.512+0000 7fe3c7400700 1 --2- 172.21.15.33:0/3728810319 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fe3ac0b2900 0x7fe3ac0b4dc0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe3b4006010 tx=0x7fe3b401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:22:33.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:33.514+0000 7fe3c4c00700 1 -- 172.21.15.33:0/3728810319 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fe3b8020020 con 0x7fe3c81033d0 2024-05-03T18:22:33.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:33.776+0000 7fe3cdc00700 1 -- 172.21.15.33:0/3728810319 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7fe3c819acf0 con 0x7fe3c81033d0 2024-05-03T18:22:34.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:34.665+0000 7fe3c4c00700 1 -- 172.21.15.33:0/3728810319 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v79) ==== 157+0+0 (secure 0 0 0) 0x7fe3b80aa280 con 0x7fe3c81033d0 2024-05-03T18:22:34.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:34.668+0000 7fe3cdc00700 1 -- 172.21.15.33:0/3728810319 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fe3ac0b2900 msgr2=0x7fe3ac0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:22:34.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:34.668+0000 7fe3cdc00700 1 --2- 172.21.15.33:0/3728810319 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fe3ac0b2900 0x7fe3ac0b4dc0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe3b4006010 tx=0x7fe3b401a040 comp rx=0 tx=0).stop 2024-05-03T18:22:34.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:34.668+0000 7fe3cdc00700 1 -- 172.21.15.33:0/3728810319 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3c81033d0 msgr2=0x7fe3c819f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:22:34.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:34.668+0000 7fe3cdc00700 1 --2- 172.21.15.33:0/3728810319 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3c81033d0 0x7fe3c819f640 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe3b800d980 tx=0x7fe3b800dd40 comp rx=0 tx=0).stop 2024-05-03T18:22:34.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:34.669+0000 7fe3cdc00700 1 -- 172.21.15.33:0/3728810319 shutdown_connections 2024-05-03T18:22:34.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:34.669+0000 7fe3cdc00700 1 --2- 172.21.15.33:0/3728810319 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe3c8109460 0x7fe3c81997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:34.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:34.669+0000 7fe3cdc00700 1 --2- 172.21.15.33:0/3728810319 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3c81033d0 0x7fe3c819f640 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:34.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:34.669+0000 7fe3cdc00700 1 --2- 172.21.15.33:0/3728810319 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fe3ac0b2900 0x7fe3ac0b4dc0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:34.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:34.669+0000 7fe3cdc00700 1 --2- 172.21.15.33:0/3728810319 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3c8103d90 0x7fe3c819fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:34.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:34.669+0000 7fe3cdc00700 1 -- 172.21.15.33:0/3728810319 >> 172.21.15.33:0/3728810319 conn(0x7fe3c80fef20 msgr2=0x7fe3c80ff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:22:34.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:34.670+0000 7fe3cdc00700 1 -- 172.21.15.33:0/3728810319 shutdown_connections 2024-05-03T18:22:34.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:34.670+0000 7fe3cdc00700 1 -- 172.21.15.33:0/3728810319 wait complete. 2024-05-03T18:22:34.933 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:34 smithi033 ceph-mon[28330]: pgmap v375: 97 pgs: 97 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-05-03T18:22:34.933 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:34 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/3728810319' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-05-03T18:22:35.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:34 smithi099 ceph-mon[31658]: pgmap v375: 97 pgs: 97 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-05-03T18:22:35.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:34 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/3728810319' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-05-03T18:22:35.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:34 smithi150 ceph-mon[32492]: pgmap v375: 97 pgs: 97 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-05-03T18:22:35.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:34 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/3728810319' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-05-03T18:22:35.463 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-05-03T18:22:35.935 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:22:35.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:35 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/3728810319' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-05-03T18:22:35.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:35 smithi033 ceph-mon[28330]: osdmap e79: 12 total, 12 up, 12 in 2024-05-03T18:22:36.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:35 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/3728810319' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-05-03T18:22:36.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:35 smithi099 ceph-mon[31658]: osdmap e79: 12 total, 12 up, 12 in 2024-05-03T18:22:36.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:35 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/3728810319' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-05-03T18:22:36.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:35 smithi150 ceph-mon[32492]: osdmap e79: 12 total, 12 up, 12 in 2024-05-03T18:22:36.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:36 smithi033 ceph-mon[28330]: pgmap v377: 97 pgs: 97 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-03T18:22:37.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:36 smithi099 ceph-mon[31658]: pgmap v377: 97 pgs: 97 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-03T18:22:37.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:36 smithi150 ceph-mon[32492]: pgmap v377: 97 pgs: 97 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-03T18:22:38.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:38.314+0000 7f7968a00700 1 -- 172.21.15.33:0/1254992084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7964108930 msgr2=0x7f796406c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:22:38.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:38.314+0000 7f7968a00700 1 --2- 172.21.15.33:0/1254992084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7964108930 0x7f796406c0b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f7950009990 tx=0x7f7950009ca0 comp rx=0 tx=0).stop 2024-05-03T18:22:38.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:38.314+0000 7f7968a00700 1 -- 172.21.15.33:0/1254992084 shutdown_connections 2024-05-03T18:22:38.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:38.314+0000 7f7968a00700 1 --2- 172.21.15.33:0/1254992084 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7964105f30 0x7f79641063e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:38.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:38.314+0000 7f7968a00700 1 --2- 172.21.15.33:0/1254992084 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79641084c0 0x7f796406bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:38.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:38.314+0000 7f7968a00700 1 --2- 172.21.15.33:0/1254992084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7964108930 0x7f796406c0b0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:38.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:38.314+0000 7f7968a00700 1 -- 172.21.15.33:0/1254992084 >> 172.21.15.33:0/1254992084 conn(0x7f7964079350 msgr2=0x7f7964079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:22:38.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:38.314+0000 7f7968a00700 1 -- 172.21.15.33:0/1254992084 shutdown_connections 2024-05-03T18:22:38.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:38.314+0000 7f7968a00700 1 -- 172.21.15.33:0/1254992084 wait complete. 2024-05-03T18:22:38.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:38.316+0000 7f7968a00700 1 Processor -- start 2024-05-03T18:22:38.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:38.316+0000 7f7968a00700 1 -- start start 2024-05-03T18:22:38.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:38.316+0000 7f7968a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7964105f30 0x7f796419ff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:22:38.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:38.316+0000 7f7968a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79641084c0 0x7f79641a0470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:22:38.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:38.316+0000 7f7968a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7964108930 0x7f79641a47b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:22:38.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:38.316+0000 7f7968a00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7964119aa0 con 0x7f7964105f30 2024-05-03T18:22:38.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:38.316+0000 7f7968a00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7964119920 con 0x7f79641084c0 2024-05-03T18:22:38.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:38.316+0000 7f7968a00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f7964119c20 con 0x7f7964108930 2024-05-03T18:22:38.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:38.317+0000 7f7961e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7964105f30 0x7f796419ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:22:38.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:38.317+0000 7f7961e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7964105f30 0x7f796419ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43580/0 (socket says 172.21.15.33:43580) 2024-05-03T18:22:38.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:38.317+0000 7f7961e00700 1 -- 172.21.15.33:0/440254763 learned_addr learned my addr 172.21.15.33:0/440254763 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:22:38.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:38.317+0000 7f7962800700 1 --2- 172.21.15.33:0/440254763 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7964108930 0x7f79641a47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:22:38.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:38.317+0000 7f7961400700 1 --2- 172.21.15.33:0/440254763 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79641084c0 0x7f79641a0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:22:38.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:38.317+0000 7f7961e00700 1 -- 172.21.15.33:0/440254763 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7964108930 msgr2=0x7f79641a47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:22:38.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:38.317+0000 7f7961e00700 1 --2- 172.21.15.33:0/440254763 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7964108930 0x7f79641a47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:38.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:38.317+0000 7f7961e00700 1 -- 172.21.15.33:0/440254763 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79641084c0 msgr2=0x7f79641a0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:22:38.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:38.317+0000 7f7961e00700 1 --2- 172.21.15.33:0/440254763 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79641084c0 0x7f79641a0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:38.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:38.317+0000 7f7961e00700 1 -- 172.21.15.33:0/440254763 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79641a4ef0 con 0x7f7964105f30 2024-05-03T18:22:38.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:38.318+0000 7f7961e00700 1 --2- 172.21.15.33:0/440254763 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7964105f30 0x7f796419ff30 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f795400e9a0 tx=0x7f795400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:22:38.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:38.318+0000 7f795aa00700 1 -- 172.21.15.33:0/440254763 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f795400cbd0 con 0x7f7964105f30 2024-05-03T18:22:38.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:38.318+0000 7f795aa00700 1 -- 172.21.15.33:0/440254763 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f795400cd70 con 0x7f7964105f30 2024-05-03T18:22:38.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:38.318+0000 7f7968a00700 1 -- 172.21.15.33:0/440254763 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79641a51e0 con 0x7f7964105f30 2024-05-03T18:22:38.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:38.318+0000 7f795aa00700 1 -- 172.21.15.33:0/440254763 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7954018680 con 0x7f7964105f30 2024-05-03T18:22:38.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:38.318+0000 7f7968a00700 1 -- 172.21.15.33:0/440254763 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79641aca60 con 0x7f7964105f30 2024-05-03T18:22:38.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:38.319+0000 7f7968a00700 1 -- 172.21.15.33:0/440254763 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7964102a10 con 0x7f7964105f30 2024-05-03T18:22:38.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:38.322+0000 7f795aa00700 1 -- 172.21.15.33:0/440254763 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f7954010950 con 0x7f7964105f30 2024-05-03T18:22:38.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:38.323+0000 7f795aa00700 1 --2- 172.21.15.33:0/440254763 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f79480b2930 0x7f79480b4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:22:38.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:38.324+0000 7f795aa00700 1 -- 172.21.15.33:0/440254763 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8277+0+0 (secure 0 0 0) 0x7f79540e27e0 con 0x7f7964105f30 2024-05-03T18:22:38.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:38.324+0000 7f7961400700 1 --2- 172.21.15.33:0/440254763 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f79480b2930 0x7f79480b4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:22:38.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:38.325+0000 7f7961400700 1 --2- 172.21.15.33:0/440254763 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f79480b2930 0x7f79480b4df0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f795000b650 tx=0x7f795001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:22:38.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:38.326+0000 7f795aa00700 1 -- 172.21.15.33:0/440254763 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f79540aa470 con 0x7f7964105f30 2024-05-03T18:22:38.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:38.568+0000 7f7968a00700 1 -- 172.21.15.33:0/440254763 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f79641acc30 con 0x7f7964105f30 2024-05-03T18:22:38.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:38 smithi033 ceph-mon[28330]: pgmap v378: 97 pgs: 97 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-03T18:22:38.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:38 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/440254763' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-03T18:22:39.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:38 smithi099 ceph-mon[31658]: pgmap v378: 97 pgs: 97 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-03T18:22:39.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:38 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/440254763' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-03T18:22:39.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:38 smithi150 ceph-mon[32492]: pgmap v378: 97 pgs: 97 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-03T18:22:39.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:38 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/440254763' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-03T18:22:39.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:39.712+0000 7f795aa00700 1 -- 172.21.15.33:0/440254763 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v81) ==== 228+0+0 (secure 0 0 0) 0x7f79540a9d70 con 0x7f7964105f30 2024-05-03T18:22:39.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:39.717+0000 7f7968a00700 1 -- 172.21.15.33:0/440254763 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f79480b2930 msgr2=0x7f79480b4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:22:39.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:39.717+0000 7f7968a00700 1 --2- 172.21.15.33:0/440254763 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f79480b2930 0x7f79480b4df0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f795000b650 tx=0x7f795001a040 comp rx=0 tx=0).stop 2024-05-03T18:22:39.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:39.717+0000 7f7968a00700 1 -- 172.21.15.33:0/440254763 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7964105f30 msgr2=0x7f796419ff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:22:39.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:39.717+0000 7f7968a00700 1 --2- 172.21.15.33:0/440254763 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7964105f30 0x7f796419ff30 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f795400e9a0 tx=0x7f795400ed60 comp rx=0 tx=0).stop 2024-05-03T18:22:39.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:39.717+0000 7f7968a00700 1 -- 172.21.15.33:0/440254763 shutdown_connections 2024-05-03T18:22:39.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:39.717+0000 7f7968a00700 1 --2- 172.21.15.33:0/440254763 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7964108930 0x7f79641a47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:39.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:39.717+0000 7f7968a00700 1 --2- 172.21.15.33:0/440254763 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7964105f30 0x7f796419ff30 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:39.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:39.717+0000 7f7968a00700 1 --2- 172.21.15.33:0/440254763 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f79480b2930 0x7f79480b4df0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:39.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:39.717+0000 7f7968a00700 1 --2- 172.21.15.33:0/440254763 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79641084c0 0x7f79641a0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:39.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:39.718+0000 7f7968a00700 1 -- 172.21.15.33:0/440254763 >> 172.21.15.33:0/440254763 conn(0x7f7964079350 msgr2=0x7f7964104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:22:39.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:39.718+0000 7f7968a00700 1 -- 172.21.15.33:0/440254763 shutdown_connections 2024-05-03T18:22:39.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:39.718+0000 7f7968a00700 1 -- 172.21.15.33:0/440254763 wait complete. 2024-05-03T18:22:39.721 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data_ec' created 2024-05-03T18:22:39.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:39 smithi033 ceph-mon[28330]: osdmap e80: 12 total, 12 up, 12 in 2024-05-03T18:22:40.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:39 smithi099 ceph-mon[31658]: osdmap e80: 12 total, 12 up, 12 in 2024-05-03T18:22:40.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:39 smithi150 ceph-mon[32492]: osdmap e80: 12 total, 12 up, 12 in 2024-05-03T18:22:40.667 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-05-03T18:22:40.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:40 smithi033 ceph-mon[28330]: pgmap v380: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 3 op/s 2024-05-03T18:22:40.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:40 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/440254763' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-03T18:22:40.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:40 smithi033 ceph-mon[28330]: osdmap e81: 12 total, 12 up, 12 in 2024-05-03T18:22:41.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:40 smithi099 ceph-mon[31658]: pgmap v380: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 3 op/s 2024-05-03T18:22:41.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:40 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/440254763' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-03T18:22:41.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:40 smithi099 ceph-mon[31658]: osdmap e81: 12 total, 12 up, 12 in 2024-05-03T18:22:41.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:40 smithi150 ceph-mon[32492]: pgmap v380: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 3 op/s 2024-05-03T18:22:41.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:40 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/440254763' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-03T18:22:41.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:40 smithi150 ceph-mon[32492]: osdmap e81: 12 total, 12 up, 12 in 2024-05-03T18:22:41.151 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:22:41.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:41 smithi033 ceph-mon[28330]: osdmap e82: 12 total, 12 up, 12 in 2024-05-03T18:22:42.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:41 smithi099 ceph-mon[31658]: osdmap e82: 12 total, 12 up, 12 in 2024-05-03T18:22:42.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:41 smithi150 ceph-mon[32492]: osdmap e82: 12 total, 12 up, 12 in 2024-05-03T18:22:42.968 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:42 smithi033 ceph-mon[28330]: pgmap v383: 129 pgs: 6 creating+peering, 25 unknown, 98 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:22:42.968 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:42 smithi033 ceph-mon[28330]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-03T18:22:42.968 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:42 smithi033 ceph-mon[28330]: osdmap e83: 12 total, 12 up, 12 in 2024-05-03T18:22:43.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:42 smithi099 ceph-mon[31658]: pgmap v383: 129 pgs: 6 creating+peering, 25 unknown, 98 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:22:43.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:42 smithi099 ceph-mon[31658]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-03T18:22:43.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:42 smithi099 ceph-mon[31658]: osdmap e83: 12 total, 12 up, 12 in 2024-05-03T18:22:43.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:42 smithi150 ceph-mon[32492]: pgmap v383: 129 pgs: 6 creating+peering, 25 unknown, 98 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:22:43.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:42 smithi150 ceph-mon[32492]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-03T18:22:43.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:42 smithi150 ceph-mon[32492]: osdmap e83: 12 total, 12 up, 12 in 2024-05-03T18:22:43.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:43.500+0000 7f0920c00700 1 -- 172.21.15.33:0/1936570943 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f091c1056c0 msgr2=0x7f091c105b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:22:43.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:43.500+0000 7f0920c00700 1 --2- 172.21.15.33:0/1936570943 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f091c1056c0 0x7f091c105b40 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0908009990 tx=0x7f0908009ca0 comp rx=0 tx=0).stop 2024-05-03T18:22:43.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:43.500+0000 7f0920c00700 1 -- 172.21.15.33:0/1936570943 shutdown_connections 2024-05-03T18:22:43.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:43.500+0000 7f0920c00700 1 --2- 172.21.15.33:0/1936570943 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f091c106080 0x7f091c112590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:43.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:43.500+0000 7f0920c00700 1 --2- 172.21.15.33:0/1936570943 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f091c10b6c0 0x7f091c10baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:43.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:43.500+0000 7f0920c00700 1 --2- 172.21.15.33:0/1936570943 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f091c1056c0 0x7f091c105b40 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:43.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:43.500+0000 7f0920c00700 1 -- 172.21.15.33:0/1936570943 >> 172.21.15.33:0/1936570943 conn(0x7f091c1011b0 msgr2=0x7f091c1035d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:22:43.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:43.500+0000 7f0920c00700 1 -- 172.21.15.33:0/1936570943 shutdown_connections 2024-05-03T18:22:43.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:43.500+0000 7f0920c00700 1 -- 172.21.15.33:0/1936570943 wait complete. 2024-05-03T18:22:43.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:43.501+0000 7f0920c00700 1 Processor -- start 2024-05-03T18:22:43.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:43.502+0000 7f0920c00700 1 -- start start 2024-05-03T18:22:43.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:43.502+0000 7f0920c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f091c1056c0 0x7f091c1a18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:22:43.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:43.502+0000 7f0920c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f091c106080 0x7f091c1a1e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:22:43.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:43.502+0000 7f0920c00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f091c10b6c0 0x7f091c19ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:22:43.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:43.502+0000 7f0920c00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f091c1157b0 con 0x7f091c10b6c0 2024-05-03T18:22:43.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:43.502+0000 7f0920c00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f091c115630 con 0x7f091c106080 2024-05-03T18:22:43.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:43.502+0000 7f0920c00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f091c115930 con 0x7f091c1056c0 2024-05-03T18:22:43.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:43.502+0000 7f091a800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f091c10b6c0 0x7f091c19ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:22:43.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:43.502+0000 7f091a800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f091c10b6c0 0x7f091c19ba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43610/0 (socket says 172.21.15.33:43610) 2024-05-03T18:22:43.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:43.502+0000 7f091a800700 1 -- 172.21.15.33:0/10682646 learned_addr learned my addr 172.21.15.33:0/10682646 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:22:43.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:43.503+0000 7f0919400700 1 --2- 172.21.15.33:0/10682646 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f091c106080 0x7f091c1a1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:22:43.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:43.503+0000 7f0919e00700 1 --2- 172.21.15.33:0/10682646 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f091c1056c0 0x7f091c1a18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:22:43.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:43.503+0000 7f091a800700 1 -- 172.21.15.33:0/10682646 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f091c1056c0 msgr2=0x7f091c1a18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:22:43.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:43.503+0000 7f091a800700 1 --2- 172.21.15.33:0/10682646 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f091c1056c0 0x7f091c1a18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:43.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:43.503+0000 7f091a800700 1 -- 172.21.15.33:0/10682646 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f091c106080 msgr2=0x7f091c1a1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:22:43.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:43.503+0000 7f091a800700 1 --2- 172.21.15.33:0/10682646 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f091c106080 0x7f091c1a1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:43.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:43.503+0000 7f091a800700 1 -- 172.21.15.33:0/10682646 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f091c19c1f0 con 0x7f091c10b6c0 2024-05-03T18:22:43.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:43.503+0000 7f091a800700 1 --2- 172.21.15.33:0/10682646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f091c10b6c0 0x7f091c19ba90 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0914007ee0 tx=0x7f091400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:22:43.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:43.504+0000 7f0912a00700 1 -- 172.21.15.33:0/10682646 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f091400dbd0 con 0x7f091c10b6c0 2024-05-03T18:22:43.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:43.504+0000 7f0912a00700 1 -- 172.21.15.33:0/10682646 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f091400dd70 con 0x7f091c10b6c0 2024-05-03T18:22:43.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:43.504+0000 7f0920c00700 1 -- 172.21.15.33:0/10682646 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f091c19c4e0 con 0x7f091c10b6c0 2024-05-03T18:22:43.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:43.504+0000 7f0912a00700 1 -- 172.21.15.33:0/10682646 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09140058d0 con 0x7f091c10b6c0 2024-05-03T18:22:43.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:43.504+0000 7f0920c00700 1 -- 172.21.15.33:0/10682646 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f091c1a87a0 con 0x7f091c10b6c0 2024-05-03T18:22:43.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:43.505+0000 7f0920c00700 1 -- 172.21.15.33:0/10682646 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f091c06a190 con 0x7f091c10b6c0 2024-05-03T18:22:43.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:43.507+0000 7f0912a00700 1 -- 172.21.15.33:0/10682646 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f091401a020 con 0x7f091c10b6c0 2024-05-03T18:22:43.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:43.509+0000 7f0912a00700 1 --2- 172.21.15.33:0/10682646 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f09000b2900 0x7f09000b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:22:43.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:43.509+0000 7f0912a00700 1 -- 172.21.15.33:0/10682646 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8770+0+0 (secure 0 0 0) 0x7f09140e2280 con 0x7f091c10b6c0 2024-05-03T18:22:43.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:43.509+0000 7f0919e00700 1 --2- 172.21.15.33:0/10682646 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f09000b2900 0x7f09000b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:22:43.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:43.510+0000 7f0919e00700 1 --2- 172.21.15.33:0/10682646 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f09000b2900 0x7f09000b4dc0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f091c19cbd0 tx=0x7f090c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:22:43.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:43.512+0000 7f0912a00700 1 -- 172.21.15.33:0/10682646 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f0914010070 con 0x7f091c10b6c0 2024-05-03T18:22:43.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:43.746+0000 7f0920c00700 1 -- 172.21.15.33:0/10682646 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f091c02e510 con 0x7f091c10b6c0 2024-05-03T18:22:44.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:44.754+0000 7f0912a00700 1 -- 172.21.15.33:0/10682646 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v84) ==== 169+0+0 (secure 0 0 0) 0x7f09140a9d20 con 0x7f091c10b6c0 2024-05-03T18:22:44.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:44.760+0000 7f0920c00700 1 -- 172.21.15.33:0/10682646 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f09000b2900 msgr2=0x7f09000b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:22:44.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:44.760+0000 7f0920c00700 1 --2- 172.21.15.33:0/10682646 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f09000b2900 0x7f09000b4dc0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f091c19cbd0 tx=0x7f090c009260 comp rx=0 tx=0).stop 2024-05-03T18:22:44.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:44.760+0000 7f0920c00700 1 -- 172.21.15.33:0/10682646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f091c10b6c0 msgr2=0x7f091c19ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:22:44.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:44.760+0000 7f0920c00700 1 --2- 172.21.15.33:0/10682646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f091c10b6c0 0x7f091c19ba90 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0914007ee0 tx=0x7f091400d3b0 comp rx=0 tx=0).stop 2024-05-03T18:22:44.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:44.761+0000 7f0920c00700 1 -- 172.21.15.33:0/10682646 shutdown_connections 2024-05-03T18:22:44.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:44.761+0000 7f0920c00700 1 --2- 172.21.15.33:0/10682646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f091c10b6c0 0x7f091c19ba90 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:44.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:44.761+0000 7f0920c00700 1 --2- 172.21.15.33:0/10682646 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f091c1056c0 0x7f091c1a18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:44.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:44.761+0000 7f0920c00700 1 --2- 172.21.15.33:0/10682646 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f09000b2900 0x7f09000b4dc0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:44.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:44.761+0000 7f0920c00700 1 --2- 172.21.15.33:0/10682646 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f091c106080 0x7f091c1a1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:44.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:44.761+0000 7f0920c00700 1 -- 172.21.15.33:0/10682646 >> 172.21.15.33:0/10682646 conn(0x7f091c1011b0 msgr2=0x7f091c101a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:22:44.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:44.761+0000 7f0920c00700 1 -- 172.21.15.33:0/10682646 shutdown_connections 2024-05-03T18:22:44.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:44.761+0000 7f0920c00700 1 -- 172.21.15.33:0/10682646 wait complete. 2024-05-03T18:22:44.764 INFO:teuthology.orchestra.run.smithi033.stderr:set pool 4 allow_ec_overwrites to true 2024-05-03T18:22:44.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:44 smithi099 ceph-mon[31658]: pgmap v385: 129 pgs: 6 creating+peering, 3 unknown, 120 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:22:44.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:44 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/10682646' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-05-03T18:22:45.030 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:44 smithi033 ceph-mon[28330]: pgmap v385: 129 pgs: 6 creating+peering, 3 unknown, 120 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:22:45.030 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:44 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/10682646' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-05-03T18:22:45.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:44 smithi150 ceph-mon[32492]: pgmap v385: 129 pgs: 6 creating+peering, 3 unknown, 120 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:22:45.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:44 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/10682646' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-05-03T18:22:45.694 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph fs add_data_pool cephfs cephfs_data_ec 2024-05-03T18:22:45.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:45 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/10682646' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-05-03T18:22:45.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:45 smithi033 ceph-mon[28330]: osdmap e84: 12 total, 12 up, 12 in 2024-05-03T18:22:46.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:45 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/10682646' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-05-03T18:22:46.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:45 smithi099 ceph-mon[31658]: osdmap e84: 12 total, 12 up, 12 in 2024-05-03T18:22:46.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:45 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/10682646' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-05-03T18:22:46.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:45 smithi150 ceph-mon[32492]: osdmap e84: 12 total, 12 up, 12 in 2024-05-03T18:22:46.172 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:22:46.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:46 smithi033 ceph-mon[28330]: pgmap v387: 129 pgs: 3 creating+peering, 126 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:22:47.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:46 smithi099 ceph-mon[31658]: pgmap v387: 129 pgs: 3 creating+peering, 126 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:22:47.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:46 smithi150 ceph-mon[32492]: pgmap v387: 129 pgs: 3 creating+peering, 126 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:22:47.916 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:47 smithi033 ceph-mon[28330]: pgmap v388: 129 pgs: 129 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:22:48.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:47 smithi099 ceph-mon[31658]: pgmap v388: 129 pgs: 129 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:22:48.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:47 smithi150 ceph-mon[32492]: pgmap v388: 129 pgs: 129 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:22:48.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:48.483+0000 7fabc4e00700 1 -- 172.21.15.33:0/3926891454 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabc010ff50 msgr2=0x7fabc0112340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:22:48.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:48.483+0000 7fabc4e00700 1 --2- 172.21.15.33:0/3926891454 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabc010ff50 0x7fabc0112340 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fabb800b0a0 tx=0x7fabb800b3b0 comp rx=0 tx=0).stop 2024-05-03T18:22:48.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:48.484+0000 7fabc4e00700 1 -- 172.21.15.33:0/3926891454 shutdown_connections 2024-05-03T18:22:48.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:48.484+0000 7fabc4e00700 1 --2- 172.21.15.33:0/3926891454 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabc010ff50 0x7fabc0112340 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:48.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:48.484+0000 7fabc4e00700 1 --2- 172.21.15.33:0/3926891454 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fabc006b1b0 0x7fabc006b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:48.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:48.484+0000 7fabc4e00700 1 --2- 172.21.15.33:0/3926891454 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fabc006bad0 0x7fabc010fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:48.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:48.484+0000 7fabc4e00700 1 -- 172.21.15.33:0/3926891454 >> 172.21.15.33:0/3926891454 conn(0x7fabc0079350 msgr2=0x7fabc0079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:22:48.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:48.484+0000 7fabc4e00700 1 -- 172.21.15.33:0/3926891454 shutdown_connections 2024-05-03T18:22:48.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:48.484+0000 7fabc4e00700 1 -- 172.21.15.33:0/3926891454 wait complete. 2024-05-03T18:22:48.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:48.485+0000 7fabc4e00700 1 Processor -- start 2024-05-03T18:22:48.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:48.486+0000 7fabc4e00700 1 -- start start 2024-05-03T18:22:48.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:48.486+0000 7fabc4e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fabc006b1b0 0x7fabc01a1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:22:48.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:48.486+0000 7fabc4e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabc006bad0 0x7fabc01a1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:22:48.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:48.486+0000 7fabc4e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fabc010ff50 0x7fabc019bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:22:48.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:48.486+0000 7fabc4e00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fabc0115770 con 0x7fabc006bad0 2024-05-03T18:22:48.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:48.486+0000 7fabc4e00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fabc01155f0 con 0x7fabc006b1b0 2024-05-03T18:22:48.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:48.486+0000 7fabc4e00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fabc01158f0 con 0x7fabc010ff50 2024-05-03T18:22:48.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:48.487+0000 7fabbd400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabc006bad0 0x7fabc01a1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:22:48.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:48.487+0000 7fabbd400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabc006bad0 0x7fabc01a1f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43254/0 (socket says 172.21.15.33:43254) 2024-05-03T18:22:48.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:48.487+0000 7fabbd400700 1 -- 172.21.15.33:0/648907750 learned_addr learned my addr 172.21.15.33:0/648907750 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:22:48.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:48.487+0000 7fabbde00700 1 --2- 172.21.15.33:0/648907750 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fabc006b1b0 0x7fabc01a1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:22:48.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:48.487+0000 7fabbe800700 1 --2- 172.21.15.33:0/648907750 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fabc010ff50 0x7fabc019bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:22:48.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:48.487+0000 7fabbd400700 1 -- 172.21.15.33:0/648907750 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fabc010ff50 msgr2=0x7fabc019bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:22:48.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:48.487+0000 7fabbd400700 1 --2- 172.21.15.33:0/648907750 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fabc010ff50 0x7fabc019bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:48.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:48.487+0000 7fabbd400700 1 -- 172.21.15.33:0/648907750 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fabc006b1b0 msgr2=0x7fabc01a1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:22:48.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:48.487+0000 7fabbd400700 1 --2- 172.21.15.33:0/648907750 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fabc006b1b0 0x7fabc01a1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:48.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:48.487+0000 7fabbd400700 1 -- 172.21.15.33:0/648907750 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabc019c3f0 con 0x7fabc006bad0 2024-05-03T18:22:48.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:48.488+0000 7fabbd400700 1 --2- 172.21.15.33:0/648907750 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabc006bad0 0x7fabc01a1f80 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fabac00d980 tx=0x7fabac00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:22:48.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:48.488+0000 7fabb6a00700 1 -- 172.21.15.33:0/648907750 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabac017070 con 0x7fabc006bad0 2024-05-03T18:22:48.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:48.488+0000 7fabb6a00700 1 -- 172.21.15.33:0/648907750 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fabac004830 con 0x7fabc006bad0 2024-05-03T18:22:48.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:48.488+0000 7fabc4e00700 1 -- 172.21.15.33:0/648907750 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabc019c6e0 con 0x7fabc006bad0 2024-05-03T18:22:48.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:48.488+0000 7fabb6a00700 1 -- 172.21.15.33:0/648907750 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabac0053a0 con 0x7fabc006bad0 2024-05-03T18:22:48.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:48.488+0000 7fabc4e00700 1 -- 172.21.15.33:0/648907750 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabc01a89c0 con 0x7fabc006bad0 2024-05-03T18:22:48.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:48.490+0000 7fabc4e00700 1 -- 172.21.15.33:0/648907750 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabc01029c0 con 0x7fabc006bad0 2024-05-03T18:22:48.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:48.495+0000 7fabb6a00700 1 -- 172.21.15.33:0/648907750 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fabac0049e0 con 0x7fabc006bad0 2024-05-03T18:22:48.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:48.496+0000 7fabb6a00700 1 --2- 172.21.15.33:0/648907750 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7faba40b2900 0x7faba40b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:22:48.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:48.497+0000 7fabb6a00700 1 -- 172.21.15.33:0/648907750 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8770+0+0 (secure 0 0 0) 0x7fabac0e2350 con 0x7fabc006bad0 2024-05-03T18:22:48.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:48.497+0000 7fabbde00700 1 --2- 172.21.15.33:0/648907750 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7faba40b2900 0x7faba40b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:22:48.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:48.498+0000 7fabbde00700 1 --2- 172.21.15.33:0/648907750 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7faba40b2900 0x7faba40b4dc0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fabb0005950 tx=0x7fabb00058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:22:48.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:48.502+0000 7fabb6a00700 1 -- 172.21.15.33:0/648907750 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fabac020020 con 0x7fabc006bad0 2024-05-03T18:22:48.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:48.800+0000 7fabc4e00700 1 -- 172.21.15.33:0/648907750 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7fabc019cfd0 con 0x7fabc006bad0 2024-05-03T18:22:48.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:48.855+0000 7fabb6a00700 1 -- 172.21.15.33:0/648907750 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) ==== 137+0+0 (secure 0 0 0) 0x7fabac0b0e20 con 0x7fabc006bad0 2024-05-03T18:22:48.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:48.859+0000 7fabc4e00700 1 -- 172.21.15.33:0/648907750 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7faba40b2900 msgr2=0x7faba40b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:22:48.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:48.859+0000 7fabc4e00700 1 --2- 172.21.15.33:0/648907750 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7faba40b2900 0x7faba40b4dc0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fabb0005950 tx=0x7fabb00058e0 comp rx=0 tx=0).stop 2024-05-03T18:22:48.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:48.859+0000 7fabc4e00700 1 -- 172.21.15.33:0/648907750 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabc006bad0 msgr2=0x7fabc01a1f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:22:48.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:48.859+0000 7fabc4e00700 1 --2- 172.21.15.33:0/648907750 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabc006bad0 0x7fabc01a1f80 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fabac00d980 tx=0x7fabac00dd40 comp rx=0 tx=0).stop 2024-05-03T18:22:48.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:48.859+0000 7fabc4e00700 1 -- 172.21.15.33:0/648907750 shutdown_connections 2024-05-03T18:22:48.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:48.859+0000 7fabc4e00700 1 --2- 172.21.15.33:0/648907750 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fabc010ff50 0x7fabc019bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:48.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:48.859+0000 7fabc4e00700 1 --2- 172.21.15.33:0/648907750 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7faba40b2900 0x7faba40b4dc0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:48.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:48.859+0000 7fabc4e00700 1 --2- 172.21.15.33:0/648907750 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fabc006b1b0 0x7fabc01a1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:48.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:48.859+0000 7fabc4e00700 1 --2- 172.21.15.33:0/648907750 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabc006bad0 0x7fabc01a1f80 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:48.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:48.859+0000 7fabc4e00700 1 -- 172.21.15.33:0/648907750 >> 172.21.15.33:0/648907750 conn(0x7fabc0079350 msgr2=0x7fabc01114a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:22:48.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:48.859+0000 7fabc4e00700 1 -- 172.21.15.33:0/648907750 shutdown_connections 2024-05-03T18:22:48.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:48.860+0000 7fabc4e00700 1 -- 172.21.15.33:0/648907750 wait complete. 2024-05-03T18:22:48.862 INFO:teuthology.orchestra.run.smithi033.stderr:added data pool 4 to fsmap 2024-05-03T18:22:49.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:48 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/648907750' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-05-03T18:22:49.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:48 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/648907750' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-05-03T18:22:49.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:48 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/648907750' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-05-03T18:22:49.782 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T18:22:49.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:49 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/648907750' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-05-03T18:22:49.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:49 smithi033 ceph-mon[28330]: osdmap e85: 12 total, 12 up, 12 in 2024-05-03T18:22:49.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:49 smithi033 ceph-mon[28330]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-03T18:22:49.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:49 smithi033 ceph-mon[28330]: pgmap v390: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:22:50.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:49 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/648907750' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-05-03T18:22:50.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:49 smithi150 ceph-mon[32492]: osdmap e85: 12 total, 12 up, 12 in 2024-05-03T18:22:50.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:49 smithi150 ceph-mon[32492]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-03T18:22:50.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:49 smithi150 ceph-mon[32492]: pgmap v390: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:22:50.246 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:22:50.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:49 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/648907750' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-05-03T18:22:50.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:49 smithi099 ceph-mon[31658]: osdmap e85: 12 total, 12 up, 12 in 2024-05-03T18:22:50.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:49 smithi099 ceph-mon[31658]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-03T18:22:50.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:49 smithi099 ceph-mon[31658]: pgmap v390: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:22:50.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:50 smithi033 ceph-mon[28330]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-03T18:22:50.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:50 smithi033 ceph-mon[28330]: Cluster is now healthy 2024-05-03T18:22:51.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:50 smithi150 ceph-mon[32492]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-03T18:22:51.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:50 smithi150 ceph-mon[32492]: Cluster is now healthy 2024-05-03T18:22:51.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:50 smithi099 ceph-mon[31658]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-03T18:22:51.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:50 smithi099 ceph-mon[31658]: Cluster is now healthy 2024-05-03T18:22:52.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:51 smithi033 ceph-mon[28330]: pgmap v391: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:22:52.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:51 smithi099 ceph-mon[31658]: pgmap v391: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:22:52.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:51 smithi150 ceph-mon[32492]: pgmap v391: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:22:52.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:52.440+0000 7fb80e400700 1 -- 172.21.15.33:0/1952518971 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb80806bb40 msgr2=0x7fb808110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:22:52.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:52.440+0000 7fb80e400700 1 --2- 172.21.15.33:0/1952518971 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb80806bb40 0x7fb808110330 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb80000b0a0 tx=0x7fb80000b3b0 comp rx=0 tx=0).stop 2024-05-03T18:22:52.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:52.440+0000 7fb80e400700 1 -- 172.21.15.33:0/1952518971 shutdown_connections 2024-05-03T18:22:52.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:52.440+0000 7fb80e400700 1 --2- 172.21.15.33:0/1952518971 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb80806bb40 0x7fb808110330 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:52.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:52.440+0000 7fb80e400700 1 --2- 172.21.15.33:0/1952518971 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb808109510 0x7fb8081098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:52.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:52.440+0000 7fb80e400700 1 --2- 172.21.15.33:0/1952518971 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb80806b180 0x7fb80806b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:52.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:52.440+0000 7fb80e400700 1 -- 172.21.15.33:0/1952518971 >> 172.21.15.33:0/1952518971 conn(0x7fb808079350 msgr2=0x7fb808079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:22:52.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:52.440+0000 7fb80e400700 1 -- 172.21.15.33:0/1952518971 shutdown_connections 2024-05-03T18:22:52.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:52.440+0000 7fb80e400700 1 -- 172.21.15.33:0/1952518971 wait complete. 2024-05-03T18:22:52.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:52.441+0000 7fb80e400700 1 Processor -- start 2024-05-03T18:22:52.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:52.441+0000 7fb80e400700 1 -- start start 2024-05-03T18:22:52.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:52.442+0000 7fb80e400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb80806b180 0x7fb808102470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:22:52.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:52.442+0000 7fb80e400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb80806bb40 0x7fb8081029b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:22:52.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:52.442+0000 7fb80e400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb808109510 0x7fb808102ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:22:52.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:52.442+0000 7fb80e400700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb808113520 con 0x7fb80806bb40 2024-05-03T18:22:52.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:52.442+0000 7fb80e400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb8081133a0 con 0x7fb80806b180 2024-05-03T18:22:52.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:52.442+0000 7fb80e400700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb8081136a0 con 0x7fb808109510 2024-05-03T18:22:52.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:52.442+0000 7fb806a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb80806bb40 0x7fb8081029b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:22:52.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:52.442+0000 7fb806a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb80806bb40 0x7fb8081029b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43268/0 (socket says 172.21.15.33:43268) 2024-05-03T18:22:52.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:52.442+0000 7fb806a00700 1 -- 172.21.15.33:0/3475308265 learned_addr learned my addr 172.21.15.33:0/3475308265 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:22:52.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:52.442+0000 7fb807400700 1 --2- 172.21.15.33:0/3475308265 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb80806b180 0x7fb808102470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:22:52.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:52.442+0000 7fb807e00700 1 --2- 172.21.15.33:0/3475308265 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb808109510 0x7fb808102ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:22:52.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:52.443+0000 7fb806a00700 1 -- 172.21.15.33:0/3475308265 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb808109510 msgr2=0x7fb808102ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:22:52.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:52.443+0000 7fb806a00700 1 --2- 172.21.15.33:0/3475308265 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb808109510 0x7fb808102ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:52.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:52.443+0000 7fb806a00700 1 -- 172.21.15.33:0/3475308265 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb80806b180 msgr2=0x7fb808102470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:22:52.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:52.443+0000 7fb806a00700 1 --2- 172.21.15.33:0/3475308265 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb80806b180 0x7fb808102470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:52.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:52.443+0000 7fb806a00700 1 -- 172.21.15.33:0/3475308265 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8081a7ea0 con 0x7fb80806bb40 2024-05-03T18:22:52.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:52.443+0000 7fb806a00700 1 --2- 172.21.15.33:0/3475308265 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb80806bb40 0x7fb8081029b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb7f400c9b0 tx=0x7fb7f400cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:22:52.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:52.443+0000 7fb7ffe00700 1 -- 172.21.15.33:0/3475308265 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f4007b00 con 0x7fb80806bb40 2024-05-03T18:22:52.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:52.443+0000 7fb80e400700 1 -- 172.21.15.33:0/3475308265 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8081a8190 con 0x7fb80806bb40 2024-05-03T18:22:52.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:52.443+0000 7fb7ffe00700 1 -- 172.21.15.33:0/3475308265 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb7f4007ca0 con 0x7fb80806bb40 2024-05-03T18:22:52.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:52.443+0000 7fb7ffe00700 1 -- 172.21.15.33:0/3475308265 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f4018660 con 0x7fb80806bb40 2024-05-03T18:22:52.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:52.443+0000 7fb80e400700 1 -- 172.21.15.33:0/3475308265 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8081a86d0 con 0x7fb80806bb40 2024-05-03T18:22:52.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:52.445+0000 7fb80e400700 1 -- 172.21.15.33:0/3475308265 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb80806a190 con 0x7fb80806bb40 2024-05-03T18:22:52.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:52.448+0000 7fb7ffe00700 1 -- 172.21.15.33:0/3475308265 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fb7f400f450 con 0x7fb80806bb40 2024-05-03T18:22:52.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:52.449+0000 7fb7ffe00700 1 --2- 172.21.15.33:0/3475308265 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fb7ec0b2880 0x7fb7ec0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:22:52.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:52.450+0000 7fb7ffe00700 1 -- 172.21.15.33:0/3475308265 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7fb7f40e2580 con 0x7fb80806bb40 2024-05-03T18:22:52.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:52.450+0000 7fb807400700 1 --2- 172.21.15.33:0/3475308265 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fb7ec0b2880 0x7fb7ec0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:22:52.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:52.450+0000 7fb807400700 1 --2- 172.21.15.33:0/3475308265 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fb7ec0b2880 0x7fb7ec0b4d40 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb7f8005fd0 tx=0x7fb7f800af20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:22:52.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:52.454+0000 7fb7ffe00700 1 -- 172.21.15.33:0/3475308265 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fb7f4014030 con 0x7fb80806bb40 2024-05-03T18:22:52.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:52.743+0000 7fb80e400700 1 -- 172.21.15.33:0/3475308265 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb80802e510 con 0x7fb80806bb40 2024-05-03T18:22:52.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:52.744+0000 7fb7ffe00700 1 -- 172.21.15.33:0/3475308265 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11791 (secure 0 0 0) 0x7fb7f40aa000 con 0x7fb80806bb40 2024-05-03T18:22:52.746 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:22:52.746 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1327393503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1327393503},{"type":"v1","addr":"172.21.15.33:6835","nonce":1327393503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/2461472948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":2461472948},{"type":"v1","addr":"172.21.15.33:6837","nonce":2461472948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14559,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/442269281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":442269281},{"type":"v1","addr":"172.21.15.33:6839","nonce":442269281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14571,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/645571756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":645571756},{"type":"v1","addr":"172.21.15.33:6841","nonce":645571756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/203598909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":203598909},{"type":"v1","addr":"172.21.15.150:6833","nonce":203598909}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/4029842493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":4029842493},{"type":"v1","addr":"172.21.15.150:6835","nonce":4029842493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3463664423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3463664423},{"type":"v1","addr":"172.21.15.150:6837","nonce":3463664423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24481,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6839/4051648018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":4051648018},{"type":"v1","addr":"172.21.15.150:6839","nonce":4051648018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34418,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1865105479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1865105479},{"type":"v1","addr":"172.21.15.99:6833","nonce":1865105479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34445,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/1206393914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1206393914},{"type":"v1","addr":"172.21.15.99:6835","nonce":1206393914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34472,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/3360970342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3360970342},{"type":"v1","addr":"172.21.15.99:6837","nonce":3360970342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-03T18:22:28.837091+0000","modified":"2024-05-03T18:22:48.803163+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34508},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34508":{"gid":34508,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/919907308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":919907308},{"type":"v1","addr":"172.21.15.99:6839","nonce":919907308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34508,"qdb_cluster":[34508]},"id":1}]} 2024-05-03T18:22:52.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:52.747+0000 7fb80e400700 1 -- 172.21.15.33:0/3475308265 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fb7ec0b2880 msgr2=0x7fb7ec0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:22:52.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:52.747+0000 7fb80e400700 1 --2- 172.21.15.33:0/3475308265 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fb7ec0b2880 0x7fb7ec0b4d40 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb7f8005fd0 tx=0x7fb7f800af20 comp rx=0 tx=0).stop 2024-05-03T18:22:52.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:52.747+0000 7fb80e400700 1 -- 172.21.15.33:0/3475308265 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb80806bb40 msgr2=0x7fb8081029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:22:52.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:52.747+0000 7fb80e400700 1 --2- 172.21.15.33:0/3475308265 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb80806bb40 0x7fb8081029b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb7f400c9b0 tx=0x7fb7f400cd70 comp rx=0 tx=0).stop 2024-05-03T18:22:52.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:52.747+0000 7fb80e400700 1 -- 172.21.15.33:0/3475308265 shutdown_connections 2024-05-03T18:22:52.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:52.747+0000 7fb80e400700 1 --2- 172.21.15.33:0/3475308265 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb808109510 0x7fb808102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:52.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:52.747+0000 7fb80e400700 1 --2- 172.21.15.33:0/3475308265 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fb7ec0b2880 0x7fb7ec0b4d40 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:52.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:52.747+0000 7fb80e400700 1 --2- 172.21.15.33:0/3475308265 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb80806b180 0x7fb808102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:52.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:52.747+0000 7fb80e400700 1 --2- 172.21.15.33:0/3475308265 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb80806bb40 0x7fb8081029b0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:52.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:52.747+0000 7fb80e400700 1 -- 172.21.15.33:0/3475308265 >> 172.21.15.33:0/3475308265 conn(0x7fb808079350 msgr2=0x7fb808100ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:22:52.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:52.747+0000 7fb80e400700 1 -- 172.21.15.33:0/3475308265 shutdown_connections 2024-05-03T18:22:52.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:52.747+0000 7fb80e400700 1 -- 172.21.15.33:0/3475308265 wait complete. 2024-05-03T18:22:52.750 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 17 2024-05-03T18:22:53.146 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:52 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/3475308265' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:22:53.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:52 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/3475308265' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:22:53.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:52 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/3475308265' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:22:53.710 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-03T18:22:53.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:53 smithi033 ceph-mon[28330]: pgmap v392: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:22:54.184 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:22:54.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:53 smithi099 ceph-mon[31658]: pgmap v392: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:22:54.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:53 smithi150 ceph-mon[32492]: pgmap v392: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:22:56.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:56.576+0000 7f84ab000700 1 -- 172.21.15.33:0/2062374021 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84a4109510 msgr2=0x7f84a41098f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:22:56.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:56.576+0000 7f84ab000700 1 --2- 172.21.15.33:0/2062374021 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84a4109510 0x7f84a41098f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8498009990 tx=0x7f8498009ca0 comp rx=0 tx=0).stop 2024-05-03T18:22:56.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:56.577+0000 7f84ab000700 1 -- 172.21.15.33:0/2062374021 shutdown_connections 2024-05-03T18:22:56.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:56.577+0000 7f84ab000700 1 --2- 172.21.15.33:0/2062374021 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84a406c3d0 0x7f84a41102a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:56.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:56.577+0000 7f84ab000700 1 --2- 172.21.15.33:0/2062374021 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84a4109510 0x7f84a41098f0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:56.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:56.577+0000 7f84ab000700 1 --2- 172.21.15.33:0/2062374021 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84a406ba10 0x7f84a406be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:56.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:56.577+0000 7f84ab000700 1 -- 172.21.15.33:0/2062374021 >> 172.21.15.33:0/2062374021 conn(0x7f84a40791e0 msgr2=0x7f84a40795f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:22:56.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:56.577+0000 7f84ab000700 1 -- 172.21.15.33:0/2062374021 shutdown_connections 2024-05-03T18:22:56.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:56.577+0000 7f84ab000700 1 -- 172.21.15.33:0/2062374021 wait complete. 2024-05-03T18:22:56.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:56.578+0000 7f84ab000700 1 Processor -- start 2024-05-03T18:22:56.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:56.578+0000 7f84ab000700 1 -- start start 2024-05-03T18:22:56.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:56.579+0000 7f84ab000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84a406ba10 0x7f84a419f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:22:56.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:56.579+0000 7f84ab000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84a406c3d0 0x7f84a419fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:22:56.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:56.579+0000 7f84ab000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84a4109510 0x7f84a4199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:22:56.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:56.579+0000 7f84ab000700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f84a4113430 con 0x7f84a406c3d0 2024-05-03T18:22:56.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:56.579+0000 7f84ab000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f84a41132b0 con 0x7f84a4109510 2024-05-03T18:22:56.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:56.579+0000 7f84ab000700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f84a41135b0 con 0x7f84a406ba10 2024-05-03T18:22:56.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:56.579+0000 7f84a9200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84a406c3d0 0x7f84a419fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:22:56.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:56.579+0000 7f84a9200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84a406c3d0 0x7f84a419fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55862/0 (socket says 172.21.15.33:55862) 2024-05-03T18:22:56.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:56.579+0000 7f84a9200700 1 -- 172.21.15.33:0/1800075316 learned_addr learned my addr 172.21.15.33:0/1800075316 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:22:56.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:56.579+0000 7f84a9c00700 1 --2- 172.21.15.33:0/1800075316 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84a406ba10 0x7f84a419f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:22:56.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:56.579+0000 7f84aa600700 1 --2- 172.21.15.33:0/1800075316 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84a4109510 0x7f84a4199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:22:56.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:56.579+0000 7f84a9200700 1 -- 172.21.15.33:0/1800075316 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84a406ba10 msgr2=0x7f84a419f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:22:56.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:56.579+0000 7f84a9200700 1 --2- 172.21.15.33:0/1800075316 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84a406ba10 0x7f84a419f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:56.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:56.579+0000 7f84a9200700 1 -- 172.21.15.33:0/1800075316 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84a4109510 msgr2=0x7f84a4199710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:22:56.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:56.579+0000 7f84a9200700 1 --2- 172.21.15.33:0/1800075316 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84a4109510 0x7f84a4199710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:56.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:56.579+0000 7f84a9200700 1 -- 172.21.15.33:0/1800075316 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84a4199ff0 con 0x7f84a406c3d0 2024-05-03T18:22:56.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:56.580+0000 7f84a9200700 1 --2- 172.21.15.33:0/1800075316 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84a406c3d0 0x7f84a419fb80 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f849000c9b0 tx=0x7f849000cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:22:56.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:56.580+0000 7f84a2000700 1 -- 172.21.15.33:0/1800075316 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8490007b00 con 0x7f84a406c3d0 2024-05-03T18:22:56.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:56.580+0000 7f84a2000700 1 -- 172.21.15.33:0/1800075316 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8490007ca0 con 0x7f84a406c3d0 2024-05-03T18:22:56.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:56.580+0000 7f84ab000700 1 -- 172.21.15.33:0/1800075316 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84a419a2e0 con 0x7f84a406c3d0 2024-05-03T18:22:56.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:56.580+0000 7f84a2000700 1 -- 172.21.15.33:0/1800075316 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8490018660 con 0x7f84a406c3d0 2024-05-03T18:22:56.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:56.580+0000 7f84ab000700 1 -- 172.21.15.33:0/1800075316 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84a41a65c0 con 0x7f84a406c3d0 2024-05-03T18:22:56.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:56.582+0000 7f84ab000700 1 -- 172.21.15.33:0/1800075316 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84a4104b10 con 0x7f84a406c3d0 2024-05-03T18:22:56.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:56.584+0000 7f84a2000700 1 -- 172.21.15.33:0/1800075316 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f849000f450 con 0x7f84a406c3d0 2024-05-03T18:22:56.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:56.585+0000 7f84a2000700 1 --2- 172.21.15.33:0/1800075316 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f848c0b29d0 0x7f848c0b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:22:56.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:56.586+0000 7f84a2000700 1 -- 172.21.15.33:0/1800075316 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7f84900e2460 con 0x7f84a406c3d0 2024-05-03T18:22:56.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:56.586+0000 7f84a9c00700 1 --2- 172.21.15.33:0/1800075316 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f848c0b29d0 0x7f848c0b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:22:56.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:56.586+0000 7f84a9c00700 1 --2- 172.21.15.33:0/1800075316 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f848c0b29d0 0x7f848c0b4e90 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f849800b5c0 tx=0x7f849801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:22:56.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:56.590+0000 7f84a2000700 1 -- 172.21.15.33:0/1800075316 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f8490014030 con 0x7f84a406c3d0 2024-05-03T18:22:56.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:56 smithi033 ceph-mon[28330]: pgmap v393: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:22:56.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:56 smithi099 ceph-mon[31658]: pgmap v393: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:22:56.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:56.838+0000 7f84ab000700 1 -- 172.21.15.33:0/1800075316 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f84a419ac40 con 0x7f84a406c3d0 2024-05-03T18:22:56.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:56.840+0000 7f84a2000700 1 -- 172.21.15.33:0/1800075316 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+24057 (secure 0 0 0) 0x7f84900a9ee0 con 0x7f84a406c3d0 2024-05-03T18:22:56.842 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:22:56.842 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":85,"fsid":"5703bba4-0978-11ef-bc97-c7b262605968","created":"2024-05-03T18:11:00.899879+0000","modified":"2024-05-03T18:22:48.802907+0000","last_up_change":"2024-05-03T18:21:00.390173+0000","last_in_change":"2024-05-03T18:20:34.103727+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-03T18:15:37.297738+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-03T18:22:19.875368+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-03T18:22:24.432692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-03T18:22:38.722443+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"fdcc4e66-4ba0-4378-a9a4-7d907cd00b4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":637686669},{"type":"v1","addr":"172.21.15.33:6803","nonce":637686669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":637686669},{"type":"v1","addr":"172.21.15.33:6805","nonce":637686669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":637686669},{"type":"v1","addr":"172.21.15.33:6809","nonce":637686669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":637686669},{"type":"v1","addr":"172.21.15.33:6807","nonce":637686669}]},"public_addr":"172.21.15.33:6803/637686669","cluster_addr":"172.21.15.33:6805/637686669","heartbeat_back_addr":"172.21.15.33:6809/637686669","heartbeat_front_addr":"172.21.15.33:6807/637686669","state":["exists","up"]},{"osd":1,"uuid":"cf78cad3-fc24-43f4-8784-21cabea7277c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":1536122841},{"type":"v1","addr":"172.21.15.99:6801","nonce":1536122841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":1536122841},{"type":"v1","addr":"172.21.15.99:6803","nonce":1536122841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":1536122841},{"type":"v1","addr":"172.21.15.99:6807","nonce":1536122841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":1536122841},{"type":"v1","addr":"172.21.15.99:6805","nonce":1536122841}]},"public_addr":"172.21.15.99:6801/1536122841","cluster_addr":"172.21.15.99:6803/1536122841","heartbeat_back_addr":"172.21.15.99:6807/1536122841","heartbeat_front_addr":"172.21.15.99:6805/1536122841","state":["exists","up"]},{"osd":2,"uuid":"59f066d7-626b-410a-8d67-daca6fbbd3ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":1995659622},{"type":"v1","addr":"172.21.15.150:6801","nonce":1995659622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":1995659622},{"type":"v1","addr":"172.21.15.150:6803","nonce":1995659622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":1995659622},{"type":"v1","addr":"172.21.15.150:6807","nonce":1995659622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":1995659622},{"type":"v1","addr":"172.21.15.150:6805","nonce":1995659622}]},"public_addr":"172.21.15.150:6801/1995659622","cluster_addr":"172.21.15.150:6803/1995659622","heartbeat_back_addr":"172.21.15.150:6807/1995659622","heartbeat_front_addr":"172.21.15.150:6805/1995659622","state":["exists","up"]},{"osd":3,"uuid":"d88ed022-4dcb-45fc-9fc6-41ef7c4bf202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":3814942724},{"type":"v1","addr":"172.21.15.33:6811","nonce":3814942724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":3814942724},{"type":"v1","addr":"172.21.15.33:6813","nonce":3814942724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":3814942724},{"type":"v1","addr":"172.21.15.33:6817","nonce":3814942724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":3814942724},{"type":"v1","addr":"172.21.15.33:6815","nonce":3814942724}]},"public_addr":"172.21.15.33:6811/3814942724","cluster_addr":"172.21.15.33:6813/3814942724","heartbeat_back_addr":"172.21.15.33:6817/3814942724","heartbeat_front_addr":"172.21.15.33:6815/3814942724","state":["exists","up"]},{"osd":4,"uuid":"1235e1a6-ff6e-4668-ac1a-073c2a3b7c98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":29081029},{"type":"v1","addr":"172.21.15.99:6809","nonce":29081029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":29081029},{"type":"v1","addr":"172.21.15.99:6811","nonce":29081029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":29081029},{"type":"v1","addr":"172.21.15.99:6815","nonce":29081029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":29081029},{"type":"v1","addr":"172.21.15.99:6813","nonce":29081029}]},"public_addr":"172.21.15.99:6809/29081029","cluster_addr":"172.21.15.99:6811/29081029","heartbeat_back_addr":"172.21.15.99:6815/29081029","heartbeat_front_addr":"172.21.15.99:6813/29081029","state":["exists","up"]},{"osd":5,"uuid":"4b84bff8-d6d7-4258-a0ee-53bc19802585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":3172189533},{"type":"v1","addr":"172.21.15.150:6809","nonce":3172189533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3172189533},{"type":"v1","addr":"172.21.15.150:6811","nonce":3172189533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3172189533},{"type":"v1","addr":"172.21.15.150:6815","nonce":3172189533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3172189533},{"type":"v1","addr":"172.21.15.150:6813","nonce":3172189533}]},"public_addr":"172.21.15.150:6809/3172189533","cluster_addr":"172.21.15.150:6811/3172189533","heartbeat_back_addr":"172.21.15.150:6815/3172189533","heartbeat_front_addr":"172.21.15.150:6813/3172189533","state":["exists","up"]},{"osd":6,"uuid":"51f5fbdd-3794-4c48-9f6c-478f0fa35ac5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":4234541418},{"type":"v1","addr":"172.21.15.33:6819","nonce":4234541418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":4234541418},{"type":"v1","addr":"172.21.15.33:6821","nonce":4234541418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":4234541418},{"type":"v1","addr":"172.21.15.33:6825","nonce":4234541418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":4234541418},{"type":"v1","addr":"172.21.15.33:6823","nonce":4234541418}]},"public_addr":"172.21.15.33:6819/4234541418","cluster_addr":"172.21.15.33:6821/4234541418","heartbeat_back_addr":"172.21.15.33:6825/4234541418","heartbeat_front_addr":"172.21.15.33:6823/4234541418","state":["exists","up"]},{"osd":7,"uuid":"982cc5b3-3163-4505-9bde-a27a8c289457","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":908673090},{"type":"v1","addr":"172.21.15.99:6817","nonce":908673090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":908673090},{"type":"v1","addr":"172.21.15.99:6819","nonce":908673090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":908673090},{"type":"v1","addr":"172.21.15.99:6823","nonce":908673090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":908673090},{"type":"v1","addr":"172.21.15.99:6821","nonce":908673090}]},"public_addr":"172.21.15.99:6817/908673090","cluster_addr":"172.21.15.99:6819/908673090","heartbeat_back_addr":"172.21.15.99:6823/908673090","heartbeat_front_addr":"172.21.15.99:6821/908673090","state":["exists","up"]},{"osd":8,"uuid":"84c9d037-eb84-4eb4-8aa2-e784b703d36c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":1308083602},{"type":"v1","addr":"172.21.15.150:6817","nonce":1308083602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":1308083602},{"type":"v1","addr":"172.21.15.150:6819","nonce":1308083602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":1308083602},{"type":"v1","addr":"172.21.15.150:6823","nonce":1308083602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":1308083602},{"type":"v1","addr":"172.21.15.150:6821","nonce":1308083602}]},"public_addr":"172.21.15.150:6817/1308083602","cluster_addr":"172.21.15.150:6819/1308083602","heartbeat_back_addr":"172.21.15.150:6823/1308083602","heartbeat_front_addr":"172.21.15.150:6821/1308083602","state":["exists","up"]},{"osd":9,"uuid":"a02e965f-0d8f-45e2-9490-337f7f6e3881","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":4133096905},{"type":"v1","addr":"172.21.15.33:6827","nonce":4133096905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":4133096905},{"type":"v1","addr":"172.21.15.33:6829","nonce":4133096905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4133096905},{"type":"v1","addr":"172.21.15.33:6833","nonce":4133096905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":4133096905},{"type":"v1","addr":"172.21.15.33:6831","nonce":4133096905}]},"public_addr":"172.21.15.33:6827/4133096905","cluster_addr":"172.21.15.33:6829/4133096905","heartbeat_back_addr":"172.21.15.33:6833/4133096905","heartbeat_front_addr":"172.21.15.33:6831/4133096905","state":["exists","up"]},{"osd":10,"uuid":"8b29bb96-76e8-45a0-ab19-8d082d755fe9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":935746136},{"type":"v1","addr":"172.21.15.99:6825","nonce":935746136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":935746136},{"type":"v1","addr":"172.21.15.99:6827","nonce":935746136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":935746136},{"type":"v1","addr":"172.21.15.99:6831","nonce":935746136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":935746136},{"type":"v1","addr":"172.21.15.99:6829","nonce":935746136}]},"public_addr":"172.21.15.99:6825/935746136","cluster_addr":"172.21.15.99:6827/935746136","heartbeat_back_addr":"172.21.15.99:6831/935746136","heartbeat_front_addr":"172.21.15.99:6829/935746136","state":["exists","up"]},{"osd":11,"uuid":"07a6a825-c902-4bfd-97cc-cc9074532af5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2069349808},{"type":"v1","addr":"172.21.15.150:6825","nonce":2069349808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2069349808},{"type":"v1","addr":"172.21.15.150:6827","nonce":2069349808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2069349808},{"type":"v1","addr":"172.21.15.150:6831","nonce":2069349808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2069349808},{"type":"v1","addr":"172.21.15.150:6829","nonce":2069349808}]},"public_addr":"172.21.15.150:6825/2069349808","cluster_addr":"172.21.15.150:6827/2069349808","heartbeat_back_addr":"172.21.15.150:6831/2069349808","heartbeat_front_addr":"172.21.15.150:6829/2069349808","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:14:27.454536+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:15:00.538570+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:15:34.125289+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:16:09.853870+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:16:44.928107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:17:19.644037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:17:56.108443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:18:31.731499+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:19:08.062358+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:19:45.204430+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:20:21.013710+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:20:57.653698+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/3991947997":"2024-05-04T18:12:19.127652+0000","172.21.15.33:0/965859483":"2024-05-04T18:12:19.127652+0000","172.21.15.33:0/2014902144":"2024-05-04T18:11:44.239835+0000","172.21.15.33:0/1314704208":"2024-05-04T18:11:44.239835+0000","172.21.15.33:0/1614640996":"2024-05-04T18:11:44.239835+0000","172.21.15.33:6800/4007322071":"2024-05-04T18:11:44.239835+0000","172.21.15.33:6801/2830135294":"2024-05-04T18:12:19.127652+0000","172.21.15.33:6800/2830135294":"2024-05-04T18:12:19.127652+0000","172.21.15.33:0/3767665033":"2024-05-04T18:12:19.127652+0000","172.21.15.33:6801/4007322071":"2024-05-04T18:11:44.239835+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-03T18:22:56.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:56.843+0000 7f84ab000700 1 -- 172.21.15.33:0/1800075316 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f848c0b29d0 msgr2=0x7f848c0b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:22:56.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:56.843+0000 7f84ab000700 1 --2- 172.21.15.33:0/1800075316 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f848c0b29d0 0x7f848c0b4e90 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f849800b5c0 tx=0x7f849801a040 comp rx=0 tx=0).stop 2024-05-03T18:22:56.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:56.843+0000 7f84ab000700 1 -- 172.21.15.33:0/1800075316 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84a406c3d0 msgr2=0x7f84a419fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:22:56.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:56.843+0000 7f84ab000700 1 --2- 172.21.15.33:0/1800075316 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84a406c3d0 0x7f84a419fb80 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f849000c9b0 tx=0x7f849000cd70 comp rx=0 tx=0).stop 2024-05-03T18:22:56.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:56.843+0000 7f84ab000700 1 -- 172.21.15.33:0/1800075316 shutdown_connections 2024-05-03T18:22:56.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:56.843+0000 7f84ab000700 1 --2- 172.21.15.33:0/1800075316 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f848c0b29d0 0x7f848c0b4e90 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:56.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:56.843+0000 7f84ab000700 1 --2- 172.21.15.33:0/1800075316 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84a4109510 0x7f84a4199710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:56.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:56.843+0000 7f84ab000700 1 --2- 172.21.15.33:0/1800075316 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84a406ba10 0x7f84a419f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:56.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:56.843+0000 7f84ab000700 1 --2- 172.21.15.33:0/1800075316 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84a406c3d0 0x7f84a419fb80 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:22:56.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:56.843+0000 7f84ab000700 1 -- 172.21.15.33:0/1800075316 >> 172.21.15.33:0/1800075316 conn(0x7f84a40791e0 msgr2=0x7f84a41005d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:22:56.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:56.844+0000 7f84ab000700 1 -- 172.21.15.33:0/1800075316 shutdown_connections 2024-05-03T18:22:56.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:22:56.844+0000 7f84ab000700 1 -- 172.21.15.33:0/1800075316 wait complete. 2024-05-03T18:22:56.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:56 smithi150 ceph-mon[32492]: pgmap v393: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:22:57.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:57 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/1800075316' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T18:22:57.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:57 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/1800075316' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T18:22:57.859 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-03T18:22:57.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:57 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/1800075316' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T18:22:58.339 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:22:58.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:22:58 smithi033 ceph-mon[28330]: pgmap v394: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:22:58.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:22:58 smithi099 ceph-mon[31658]: pgmap v394: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:22:58.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:22:58 smithi150 ceph-mon[32492]: pgmap v394: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:23:00.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:23:00 smithi033 ceph-mon[28330]: pgmap v395: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:23:00.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:00.724+0000 7f2a1cc00700 1 -- 172.21.15.33:0/3185535020 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a180756d0 msgr2=0x7f2a18075ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:23:00.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:00.724+0000 7f2a1cc00700 1 --2- 172.21.15.33:0/3185535020 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a180756d0 0x7f2a18075ab0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2a08009990 tx=0x7f2a08009ca0 comp rx=0 tx=0).stop 2024-05-03T18:23:00.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:00.724+0000 7f2a1cc00700 1 -- 172.21.15.33:0/3185535020 shutdown_connections 2024-05-03T18:23:00.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:00.724+0000 7f2a1cc00700 1 --2- 172.21.15.33:0/3185535020 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2a1810fea0 0x7f2a18112290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:00.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:00.724+0000 7f2a1cc00700 1 --2- 172.21.15.33:0/3185535020 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a180756d0 0x7f2a18075ab0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:00.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:00.724+0000 7f2a1cc00700 1 --2- 172.21.15.33:0/3185535020 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a18075ff0 0x7f2a1810f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:00.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:00.724+0000 7f2a1cc00700 1 -- 172.21.15.33:0/3185535020 >> 172.21.15.33:0/3185535020 conn(0x7f2a1807a600 msgr2=0x7f2a1807aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:23:00.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:00.724+0000 7f2a1cc00700 1 -- 172.21.15.33:0/3185535020 shutdown_connections 2024-05-03T18:23:00.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:00.724+0000 7f2a1cc00700 1 -- 172.21.15.33:0/3185535020 wait complete. 2024-05-03T18:23:00.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:00.725+0000 7f2a1cc00700 1 Processor -- start 2024-05-03T18:23:00.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:00.725+0000 7f2a1cc00700 1 -- start start 2024-05-03T18:23:00.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:00.726+0000 7f2a1cc00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a180756d0 0x7f2a181a18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:23:00.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:00.726+0000 7f2a1cc00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a18075ff0 0x7f2a181a1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:23:00.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:00.726+0000 7f2a1cc00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2a1810fea0 0x7f2a1819bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:23:00.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:00.726+0000 7f2a1cc00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2a18115770 con 0x7f2a180756d0 2024-05-03T18:23:00.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:00.726+0000 7f2a1cc00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2a181155f0 con 0x7f2a18075ff0 2024-05-03T18:23:00.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:00.726+0000 7f2a1cc00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2a181158f0 con 0x7f2a1810fea0 2024-05-03T18:23:00.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:00.726+0000 7f2a15e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a180756d0 0x7f2a181a18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:23:00.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:00.726+0000 7f2a15e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a180756d0 0x7f2a181a18f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55886/0 (socket says 172.21.15.33:55886) 2024-05-03T18:23:00.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:00.726+0000 7f2a15e00700 1 -- 172.21.15.33:0/330820460 learned_addr learned my addr 172.21.15.33:0/330820460 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:23:00.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:00.726+0000 7f2a0d400700 1 --2- 172.21.15.33:0/330820460 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a18075ff0 0x7f2a181a1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:23:00.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:00.726+0000 7f2a16800700 1 --2- 172.21.15.33:0/330820460 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2a1810fea0 0x7f2a1819bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:23:00.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:00.727+0000 7f2a15e00700 1 -- 172.21.15.33:0/330820460 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2a1810fea0 msgr2=0x7f2a1819bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:23:00.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:00.727+0000 7f2a15e00700 1 --2- 172.21.15.33:0/330820460 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2a1810fea0 0x7f2a1819bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:00.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:00.727+0000 7f2a15e00700 1 -- 172.21.15.33:0/330820460 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a18075ff0 msgr2=0x7f2a181a1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:23:00.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:00.727+0000 7f2a15e00700 1 --2- 172.21.15.33:0/330820460 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a18075ff0 0x7f2a181a1e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:00.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:00.727+0000 7f2a15e00700 1 -- 172.21.15.33:0/330820460 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a1819c210 con 0x7f2a180756d0 2024-05-03T18:23:00.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:00.727+0000 7f2a0d400700 1 --2- 172.21.15.33:0/330820460 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a18075ff0 0x7f2a181a1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:23:00.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:00.727+0000 7f2a15e00700 1 --2- 172.21.15.33:0/330820460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a180756d0 0x7f2a181a18f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2a08000c00 tx=0x7f2a08005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:23:00.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:00.727+0000 7f2a0f400700 1 -- 172.21.15.33:0/330820460 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a08027070 con 0x7f2a180756d0 2024-05-03T18:23:00.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:00.727+0000 7f2a0f400700 1 -- 172.21.15.33:0/330820460 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2a0800fb80 con 0x7f2a180756d0 2024-05-03T18:23:00.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:00.727+0000 7f2a0f400700 1 -- 172.21.15.33:0/330820460 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a08022c30 con 0x7f2a180756d0 2024-05-03T18:23:00.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:00.727+0000 7f2a1cc00700 1 -- 172.21.15.33:0/330820460 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a1819c4a0 con 0x7f2a180756d0 2024-05-03T18:23:00.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:00.728+0000 7f2a1cc00700 1 -- 172.21.15.33:0/330820460 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a181a87c0 con 0x7f2a180756d0 2024-05-03T18:23:00.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:00.729+0000 7f2a1cc00700 1 -- 172.21.15.33:0/330820460 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a1806a190 con 0x7f2a180756d0 2024-05-03T18:23:00.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:00.731+0000 7f2a0f400700 1 -- 172.21.15.33:0/330820460 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f2a0802a070 con 0x7f2a180756d0 2024-05-03T18:23:00.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:00.733+0000 7f2a0f400700 1 --2- 172.21.15.33:0/330820460 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f29fc0b2900 0x7f29fc0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:23:00.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:00.733+0000 7f2a0f400700 1 -- 172.21.15.33:0/330820460 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7f2a080e3560 con 0x7f2a180756d0 2024-05-03T18:23:00.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:00.733+0000 7f2a0d400700 1 --2- 172.21.15.33:0/330820460 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f29fc0b2900 0x7f29fc0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:23:00.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:00.734+0000 7f2a0d400700 1 --2- 172.21.15.33:0/330820460 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f29fc0b2900 0x7f29fc0b4dc0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2a00005950 tx=0x7f2a000058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:23:00.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:00.735+0000 7f2a0f400700 1 -- 172.21.15.33:0/330820460 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f2a08031080 con 0x7f2a180756d0 2024-05-03T18:23:00.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:23:00 smithi099 ceph-mon[31658]: pgmap v395: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:23:00.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:23:00 smithi150 ceph-mon[32492]: pgmap v395: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:23:00.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:00.972+0000 7f2a1cc00700 1 -- 172.21.15.33:0/330820460 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2a1802e510 con 0x7f2a180756d0 2024-05-03T18:23:00.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:00.975+0000 7f2a0f400700 1 -- 172.21.15.33:0/330820460 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+24057 (secure 0 0 0) 0x7f2a08013070 con 0x7f2a180756d0 2024-05-03T18:23:00.975 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:23:00.975 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":85,"fsid":"5703bba4-0978-11ef-bc97-c7b262605968","created":"2024-05-03T18:11:00.899879+0000","modified":"2024-05-03T18:22:48.802907+0000","last_up_change":"2024-05-03T18:21:00.390173+0000","last_in_change":"2024-05-03T18:20:34.103727+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-03T18:15:37.297738+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-03T18:22:19.875368+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-03T18:22:24.432692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-03T18:22:38.722443+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"fdcc4e66-4ba0-4378-a9a4-7d907cd00b4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":637686669},{"type":"v1","addr":"172.21.15.33:6803","nonce":637686669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":637686669},{"type":"v1","addr":"172.21.15.33:6805","nonce":637686669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":637686669},{"type":"v1","addr":"172.21.15.33:6809","nonce":637686669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":637686669},{"type":"v1","addr":"172.21.15.33:6807","nonce":637686669}]},"public_addr":"172.21.15.33:6803/637686669","cluster_addr":"172.21.15.33:6805/637686669","heartbeat_back_addr":"172.21.15.33:6809/637686669","heartbeat_front_addr":"172.21.15.33:6807/637686669","state":["exists","up"]},{"osd":1,"uuid":"cf78cad3-fc24-43f4-8784-21cabea7277c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":1536122841},{"type":"v1","addr":"172.21.15.99:6801","nonce":1536122841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":1536122841},{"type":"v1","addr":"172.21.15.99:6803","nonce":1536122841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":1536122841},{"type":"v1","addr":"172.21.15.99:6807","nonce":1536122841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":1536122841},{"type":"v1","addr":"172.21.15.99:6805","nonce":1536122841}]},"public_addr":"172.21.15.99:6801/1536122841","cluster_addr":"172.21.15.99:6803/1536122841","heartbeat_back_addr":"172.21.15.99:6807/1536122841","heartbeat_front_addr":"172.21.15.99:6805/1536122841","state":["exists","up"]},{"osd":2,"uuid":"59f066d7-626b-410a-8d67-daca6fbbd3ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":1995659622},{"type":"v1","addr":"172.21.15.150:6801","nonce":1995659622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":1995659622},{"type":"v1","addr":"172.21.15.150:6803","nonce":1995659622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":1995659622},{"type":"v1","addr":"172.21.15.150:6807","nonce":1995659622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":1995659622},{"type":"v1","addr":"172.21.15.150:6805","nonce":1995659622}]},"public_addr":"172.21.15.150:6801/1995659622","cluster_addr":"172.21.15.150:6803/1995659622","heartbeat_back_addr":"172.21.15.150:6807/1995659622","heartbeat_front_addr":"172.21.15.150:6805/1995659622","state":["exists","up"]},{"osd":3,"uuid":"d88ed022-4dcb-45fc-9fc6-41ef7c4bf202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":3814942724},{"type":"v1","addr":"172.21.15.33:6811","nonce":3814942724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":3814942724},{"type":"v1","addr":"172.21.15.33:6813","nonce":3814942724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":3814942724},{"type":"v1","addr":"172.21.15.33:6817","nonce":3814942724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":3814942724},{"type":"v1","addr":"172.21.15.33:6815","nonce":3814942724}]},"public_addr":"172.21.15.33:6811/3814942724","cluster_addr":"172.21.15.33:6813/3814942724","heartbeat_back_addr":"172.21.15.33:6817/3814942724","heartbeat_front_addr":"172.21.15.33:6815/3814942724","state":["exists","up"]},{"osd":4,"uuid":"1235e1a6-ff6e-4668-ac1a-073c2a3b7c98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":29081029},{"type":"v1","addr":"172.21.15.99:6809","nonce":29081029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":29081029},{"type":"v1","addr":"172.21.15.99:6811","nonce":29081029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":29081029},{"type":"v1","addr":"172.21.15.99:6815","nonce":29081029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":29081029},{"type":"v1","addr":"172.21.15.99:6813","nonce":29081029}]},"public_addr":"172.21.15.99:6809/29081029","cluster_addr":"172.21.15.99:6811/29081029","heartbeat_back_addr":"172.21.15.99:6815/29081029","heartbeat_front_addr":"172.21.15.99:6813/29081029","state":["exists","up"]},{"osd":5,"uuid":"4b84bff8-d6d7-4258-a0ee-53bc19802585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":3172189533},{"type":"v1","addr":"172.21.15.150:6809","nonce":3172189533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3172189533},{"type":"v1","addr":"172.21.15.150:6811","nonce":3172189533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3172189533},{"type":"v1","addr":"172.21.15.150:6815","nonce":3172189533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3172189533},{"type":"v1","addr":"172.21.15.150:6813","nonce":3172189533}]},"public_addr":"172.21.15.150:6809/3172189533","cluster_addr":"172.21.15.150:6811/3172189533","heartbeat_back_addr":"172.21.15.150:6815/3172189533","heartbeat_front_addr":"172.21.15.150:6813/3172189533","state":["exists","up"]},{"osd":6,"uuid":"51f5fbdd-3794-4c48-9f6c-478f0fa35ac5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":4234541418},{"type":"v1","addr":"172.21.15.33:6819","nonce":4234541418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":4234541418},{"type":"v1","addr":"172.21.15.33:6821","nonce":4234541418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":4234541418},{"type":"v1","addr":"172.21.15.33:6825","nonce":4234541418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":4234541418},{"type":"v1","addr":"172.21.15.33:6823","nonce":4234541418}]},"public_addr":"172.21.15.33:6819/4234541418","cluster_addr":"172.21.15.33:6821/4234541418","heartbeat_back_addr":"172.21.15.33:6825/4234541418","heartbeat_front_addr":"172.21.15.33:6823/4234541418","state":["exists","up"]},{"osd":7,"uuid":"982cc5b3-3163-4505-9bde-a27a8c289457","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":908673090},{"type":"v1","addr":"172.21.15.99:6817","nonce":908673090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":908673090},{"type":"v1","addr":"172.21.15.99:6819","nonce":908673090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":908673090},{"type":"v1","addr":"172.21.15.99:6823","nonce":908673090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":908673090},{"type":"v1","addr":"172.21.15.99:6821","nonce":908673090}]},"public_addr":"172.21.15.99:6817/908673090","cluster_addr":"172.21.15.99:6819/908673090","heartbeat_back_addr":"172.21.15.99:6823/908673090","heartbeat_front_addr":"172.21.15.99:6821/908673090","state":["exists","up"]},{"osd":8,"uuid":"84c9d037-eb84-4eb4-8aa2-e784b703d36c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":1308083602},{"type":"v1","addr":"172.21.15.150:6817","nonce":1308083602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":1308083602},{"type":"v1","addr":"172.21.15.150:6819","nonce":1308083602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":1308083602},{"type":"v1","addr":"172.21.15.150:6823","nonce":1308083602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":1308083602},{"type":"v1","addr":"172.21.15.150:6821","nonce":1308083602}]},"public_addr":"172.21.15.150:6817/1308083602","cluster_addr":"172.21.15.150:6819/1308083602","heartbeat_back_addr":"172.21.15.150:6823/1308083602","heartbeat_front_addr":"172.21.15.150:6821/1308083602","state":["exists","up"]},{"osd":9,"uuid":"a02e965f-0d8f-45e2-9490-337f7f6e3881","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":4133096905},{"type":"v1","addr":"172.21.15.33:6827","nonce":4133096905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":4133096905},{"type":"v1","addr":"172.21.15.33:6829","nonce":4133096905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4133096905},{"type":"v1","addr":"172.21.15.33:6833","nonce":4133096905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":4133096905},{"type":"v1","addr":"172.21.15.33:6831","nonce":4133096905}]},"public_addr":"172.21.15.33:6827/4133096905","cluster_addr":"172.21.15.33:6829/4133096905","heartbeat_back_addr":"172.21.15.33:6833/4133096905","heartbeat_front_addr":"172.21.15.33:6831/4133096905","state":["exists","up"]},{"osd":10,"uuid":"8b29bb96-76e8-45a0-ab19-8d082d755fe9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":935746136},{"type":"v1","addr":"172.21.15.99:6825","nonce":935746136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":935746136},{"type":"v1","addr":"172.21.15.99:6827","nonce":935746136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":935746136},{"type":"v1","addr":"172.21.15.99:6831","nonce":935746136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":935746136},{"type":"v1","addr":"172.21.15.99:6829","nonce":935746136}]},"public_addr":"172.21.15.99:6825/935746136","cluster_addr":"172.21.15.99:6827/935746136","heartbeat_back_addr":"172.21.15.99:6831/935746136","heartbeat_front_addr":"172.21.15.99:6829/935746136","state":["exists","up"]},{"osd":11,"uuid":"07a6a825-c902-4bfd-97cc-cc9074532af5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2069349808},{"type":"v1","addr":"172.21.15.150:6825","nonce":2069349808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2069349808},{"type":"v1","addr":"172.21.15.150:6827","nonce":2069349808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2069349808},{"type":"v1","addr":"172.21.15.150:6831","nonce":2069349808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2069349808},{"type":"v1","addr":"172.21.15.150:6829","nonce":2069349808}]},"public_addr":"172.21.15.150:6825/2069349808","cluster_addr":"172.21.15.150:6827/2069349808","heartbeat_back_addr":"172.21.15.150:6831/2069349808","heartbeat_front_addr":"172.21.15.150:6829/2069349808","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:14:27.454536+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:15:00.538570+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:15:34.125289+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:16:09.853870+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:16:44.928107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:17:19.644037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:17:56.108443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:18:31.731499+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:19:08.062358+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:19:45.204430+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:20:21.013710+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:20:57.653698+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/3991947997":"2024-05-04T18:12:19.127652+0000","172.21.15.33:0/965859483":"2024-05-04T18:12:19.127652+0000","172.21.15.33:0/2014902144":"2024-05-04T18:11:44.239835+0000","172.21.15.33:0/1314704208":"2024-05-04T18:11:44.239835+0000","172.21.15.33:0/1614640996":"2024-05-04T18:11:44.239835+0000","172.21.15.33:6800/4007322071":"2024-05-04T18:11:44.239835+0000","172.21.15.33:6801/2830135294":"2024-05-04T18:12:19.127652+0000","172.21.15.33:6800/2830135294":"2024-05-04T18:12:19.127652+0000","172.21.15.33:0/3767665033":"2024-05-04T18:12:19.127652+0000","172.21.15.33:6801/4007322071":"2024-05-04T18:11:44.239835+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-03T18:23:00.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:00.979+0000 7f2a1cc00700 1 -- 172.21.15.33:0/330820460 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f29fc0b2900 msgr2=0x7f29fc0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:23:00.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:00.979+0000 7f2a1cc00700 1 --2- 172.21.15.33:0/330820460 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f29fc0b2900 0x7f29fc0b4dc0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2a00005950 tx=0x7f2a000058e0 comp rx=0 tx=0).stop 2024-05-03T18:23:00.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:00.979+0000 7f2a1cc00700 1 -- 172.21.15.33:0/330820460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a180756d0 msgr2=0x7f2a181a18f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:23:00.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:00.979+0000 7f2a1cc00700 1 --2- 172.21.15.33:0/330820460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a180756d0 0x7f2a181a18f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2a08000c00 tx=0x7f2a08005ef0 comp rx=0 tx=0).stop 2024-05-03T18:23:00.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:00.979+0000 7f2a1cc00700 1 -- 172.21.15.33:0/330820460 shutdown_connections 2024-05-03T18:23:00.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:00.979+0000 7f2a1cc00700 1 --2- 172.21.15.33:0/330820460 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2a1810fea0 0x7f2a1819bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:00.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:00.979+0000 7f2a1cc00700 1 --2- 172.21.15.33:0/330820460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a180756d0 0x7f2a181a18f0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:00.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:00.979+0000 7f2a1cc00700 1 --2- 172.21.15.33:0/330820460 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f29fc0b2900 0x7f29fc0b4dc0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:00.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:00.979+0000 7f2a1cc00700 1 --2- 172.21.15.33:0/330820460 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a18075ff0 0x7f2a181a1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:00.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:00.979+0000 7f2a1cc00700 1 -- 172.21.15.33:0/330820460 >> 172.21.15.33:0/330820460 conn(0x7f2a1807a600 msgr2=0x7f2a181006b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:23:00.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:00.979+0000 7f2a1cc00700 1 -- 172.21.15.33:0/330820460 shutdown_connections 2024-05-03T18:23:00.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:00.979+0000 7f2a1cc00700 1 -- 172.21.15.33:0/330820460 wait complete. 2024-05-03T18:23:01.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:23:01 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/330820460' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T18:23:01.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:23:01 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/330820460' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T18:23:01.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:23:01 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/330820460' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T18:23:01.941 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T18:23:02.422 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:23:02.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:23:02 smithi033 ceph-mon[28330]: pgmap v396: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:23:02.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:23:02 smithi099 ceph-mon[31658]: pgmap v396: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:23:02.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:23:02 smithi150 ceph-mon[32492]: pgmap v396: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:23:04.685 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:23:04 smithi150 ceph-mon[32492]: pgmap v397: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:23:04.694 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:23:04 smithi033 ceph-mon[28330]: pgmap v397: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:23:04.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:04.759+0000 7fb410a00700 1 -- 172.21.15.33:0/4109648804 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb40c10ff00 msgr2=0x7fb40c1122f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:23:04.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:04.759+0000 7fb410a00700 1 --2- 172.21.15.33:0/4109648804 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb40c10ff00 0x7fb40c1122f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc00b0a0 tx=0x7fb3fc00b3b0 comp rx=0 tx=0).stop 2024-05-03T18:23:04.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:04.760+0000 7fb410a00700 1 -- 172.21.15.33:0/4109648804 shutdown_connections 2024-05-03T18:23:04.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:04.760+0000 7fb410a00700 1 --2- 172.21.15.33:0/4109648804 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb40c10ff00 0x7fb40c1122f0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:04.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:04.760+0000 7fb410a00700 1 --2- 172.21.15.33:0/4109648804 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb40c0756d0 0x7fb40c075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:04.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:04.760+0000 7fb410a00700 1 --2- 172.21.15.33:0/4109648804 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb40c075ff0 0x7fb40c10f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:04.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:04.760+0000 7fb410a00700 1 -- 172.21.15.33:0/4109648804 >> 172.21.15.33:0/4109648804 conn(0x7fb40c07a600 msgr2=0x7fb40c07aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:23:04.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:04.760+0000 7fb410a00700 1 -- 172.21.15.33:0/4109648804 shutdown_connections 2024-05-03T18:23:04.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:04.760+0000 7fb410a00700 1 -- 172.21.15.33:0/4109648804 wait complete. 2024-05-03T18:23:04.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:04.761+0000 7fb410a00700 1 Processor -- start 2024-05-03T18:23:04.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:04.761+0000 7fb410a00700 1 -- start start 2024-05-03T18:23:04.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:04.762+0000 7fb410a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb40c0756d0 0x7fb40c1a17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:23:04.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:04.762+0000 7fb410a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb40c075ff0 0x7fb40c1a1d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:23:04.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:04.762+0000 7fb410a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb40c10ff00 0x7fb40c19b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:23:04.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:04.762+0000 7fb410a00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb40c115790 con 0x7fb40c0756d0 2024-05-03T18:23:04.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:04.762+0000 7fb410a00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb40c115610 con 0x7fb40c075ff0 2024-05-03T18:23:04.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:04.762+0000 7fb410a00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb40c115910 con 0x7fb40c10ff00 2024-05-03T18:23:04.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:04.762+0000 7fb409e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb40c0756d0 0x7fb40c1a17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:23:04.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:04.762+0000 7fb409e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb40c0756d0 0x7fb40c1a17e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55906/0 (socket says 172.21.15.33:55906) 2024-05-03T18:23:04.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:04.762+0000 7fb409e00700 1 -- 172.21.15.33:0/3098826605 learned_addr learned my addr 172.21.15.33:0/3098826605 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:23:04.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:04.762+0000 7fb409400700 1 --2- 172.21.15.33:0/3098826605 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb40c075ff0 0x7fb40c1a1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:23:04.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:04.762+0000 7fb40a800700 1 --2- 172.21.15.33:0/3098826605 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb40c10ff00 0x7fb40c19b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:23:04.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:04.763+0000 7fb409e00700 1 -- 172.21.15.33:0/3098826605 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb40c10ff00 msgr2=0x7fb40c19b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:23:04.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:04.763+0000 7fb409e00700 1 --2- 172.21.15.33:0/3098826605 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb40c10ff00 0x7fb40c19b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:04.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:04.763+0000 7fb409e00700 1 -- 172.21.15.33:0/3098826605 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb40c075ff0 msgr2=0x7fb40c1a1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:23:04.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:04.763+0000 7fb409e00700 1 --2- 172.21.15.33:0/3098826605 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb40c075ff0 0x7fb40c1a1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:04.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:04.763+0000 7fb409e00700 1 -- 172.21.15.33:0/3098826605 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb40c19c150 con 0x7fb40c0756d0 2024-05-03T18:23:04.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:04.763+0000 7fb409e00700 1 --2- 172.21.15.33:0/3098826605 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb40c0756d0 0x7fb40c1a17e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb3f800c9b0 tx=0x7fb3f800cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:23:04.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:04.763+0000 7fb402a00700 1 -- 172.21.15.33:0/3098826605 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f8007b00 con 0x7fb40c0756d0 2024-05-03T18:23:04.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:04.764+0000 7fb402a00700 1 -- 172.21.15.33:0/3098826605 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb3f8007ca0 con 0x7fb40c0756d0 2024-05-03T18:23:04.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:04.764+0000 7fb402a00700 1 -- 172.21.15.33:0/3098826605 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f8018660 con 0x7fb40c0756d0 2024-05-03T18:23:04.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:04.764+0000 7fb410a00700 1 -- 172.21.15.33:0/3098826605 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb40c19c440 con 0x7fb40c0756d0 2024-05-03T18:23:04.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:04.764+0000 7fb410a00700 1 -- 172.21.15.33:0/3098826605 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb40c1a86b0 con 0x7fb40c0756d0 2024-05-03T18:23:04.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:04.765+0000 7fb410a00700 1 -- 172.21.15.33:0/3098826605 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb40c06a190 con 0x7fb40c0756d0 2024-05-03T18:23:04.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:04.768+0000 7fb402a00700 1 -- 172.21.15.33:0/3098826605 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fb3f800f450 con 0x7fb40c0756d0 2024-05-03T18:23:04.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:04.770+0000 7fb402a00700 1 --2- 172.21.15.33:0/3098826605 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fb3f00b2900 0x7fb3f00b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:23:04.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:04.770+0000 7fb402a00700 1 -- 172.21.15.33:0/3098826605 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7fb3f80e2780 con 0x7fb40c0756d0 2024-05-03T18:23:04.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:04.770+0000 7fb409400700 1 --2- 172.21.15.33:0/3098826605 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fb3f00b2900 0x7fb3f00b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:23:04.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:04.771+0000 7fb409400700 1 --2- 172.21.15.33:0/3098826605 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fb3f00b2900 0x7fb3f00b4dc0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb404005950 tx=0x7fb4040058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:23:04.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:04.772+0000 7fb402a00700 1 -- 172.21.15.33:0/3098826605 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fb3f8014030 con 0x7fb40c0756d0 2024-05-03T18:23:04.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:23:04 smithi099 ceph-mon[31658]: pgmap v397: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:23:05.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:05.062+0000 7fb410a00700 1 -- 172.21.15.33:0/3098826605 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb40c19ced0 con 0x7fb40c0756d0 2024-05-03T18:23:05.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:05.063+0000 7fb402a00700 1 -- 172.21.15.33:0/3098826605 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11791 (secure 0 0 0) 0x7fb3f80aa310 con 0x7fb40c0756d0 2024-05-03T18:23:05.063 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:23:05.064 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1327393503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1327393503},{"type":"v1","addr":"172.21.15.33:6835","nonce":1327393503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/2461472948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":2461472948},{"type":"v1","addr":"172.21.15.33:6837","nonce":2461472948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14559,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/442269281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":442269281},{"type":"v1","addr":"172.21.15.33:6839","nonce":442269281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14571,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/645571756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":645571756},{"type":"v1","addr":"172.21.15.33:6841","nonce":645571756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/203598909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":203598909},{"type":"v1","addr":"172.21.15.150:6833","nonce":203598909}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/4029842493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":4029842493},{"type":"v1","addr":"172.21.15.150:6835","nonce":4029842493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3463664423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3463664423},{"type":"v1","addr":"172.21.15.150:6837","nonce":3463664423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24481,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6839/4051648018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":4051648018},{"type":"v1","addr":"172.21.15.150:6839","nonce":4051648018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34418,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1865105479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1865105479},{"type":"v1","addr":"172.21.15.99:6833","nonce":1865105479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34445,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/1206393914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1206393914},{"type":"v1","addr":"172.21.15.99:6835","nonce":1206393914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34472,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/3360970342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3360970342},{"type":"v1","addr":"172.21.15.99:6837","nonce":3360970342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-03T18:22:28.837091+0000","modified":"2024-05-03T18:22:48.803163+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34508},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34508":{"gid":34508,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/919907308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":919907308},{"type":"v1","addr":"172.21.15.99:6839","nonce":919907308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34508,"qdb_cluster":[34508]},"id":1}]} 2024-05-03T18:23:05.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:05.066+0000 7fb410a00700 1 -- 172.21.15.33:0/3098826605 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fb3f00b2900 msgr2=0x7fb3f00b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:23:05.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:05.066+0000 7fb410a00700 1 --2- 172.21.15.33:0/3098826605 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fb3f00b2900 0x7fb3f00b4dc0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb404005950 tx=0x7fb4040058e0 comp rx=0 tx=0).stop 2024-05-03T18:23:05.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:05.066+0000 7fb410a00700 1 -- 172.21.15.33:0/3098826605 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb40c0756d0 msgr2=0x7fb40c1a17e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:23:05.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:05.066+0000 7fb410a00700 1 --2- 172.21.15.33:0/3098826605 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb40c0756d0 0x7fb40c1a17e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb3f800c9b0 tx=0x7fb3f800cd70 comp rx=0 tx=0).stop 2024-05-03T18:23:05.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:05.066+0000 7fb410a00700 1 -- 172.21.15.33:0/3098826605 shutdown_connections 2024-05-03T18:23:05.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:05.066+0000 7fb410a00700 1 --2- 172.21.15.33:0/3098826605 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb40c10ff00 0x7fb40c19b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:05.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:05.066+0000 7fb410a00700 1 --2- 172.21.15.33:0/3098826605 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb40c0756d0 0x7fb40c1a17e0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:05.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:05.066+0000 7fb410a00700 1 --2- 172.21.15.33:0/3098826605 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fb3f00b2900 0x7fb3f00b4dc0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:05.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:05.066+0000 7fb410a00700 1 --2- 172.21.15.33:0/3098826605 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb40c075ff0 0x7fb40c1a1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:05.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:05.066+0000 7fb410a00700 1 -- 172.21.15.33:0/3098826605 >> 172.21.15.33:0/3098826605 conn(0x7fb40c07a600 msgr2=0x7fb40c100670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:23:05.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:05.067+0000 7fb410a00700 1 -- 172.21.15.33:0/3098826605 shutdown_connections 2024-05-03T18:23:05.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:05.067+0000 7fb410a00700 1 -- 172.21.15.33:0/3098826605 wait complete. 2024-05-03T18:23:05.068 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 17 2024-05-03T18:23:05.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:23:05 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/3098826605' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:23:05.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:23:05 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/3098826605' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:23:05.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:23:05 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/3098826605' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:23:05.911 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-03T18:23:06.363 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:23:06.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:23:06 smithi033 ceph-mon[28330]: pgmap v398: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:23:06.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:23:06 smithi099 ceph-mon[31658]: pgmap v398: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:23:06.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:23:06 smithi150 ceph-mon[32492]: pgmap v398: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:23:08.725 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:23:08 smithi033 ceph-mon[28330]: pgmap v399: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:23:08.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:08.724+0000 7f3f02200700 1 -- 172.21.15.33:0/3600274542 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3efc105f90 msgr2=0x7f3efc1124a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:23:08.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:08.724+0000 7f3f02200700 1 --2- 172.21.15.33:0/3600274542 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3efc105f90 0x7f3efc1124a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3ef400b0a0 tx=0x7f3ef400b3b0 comp rx=0 tx=0).stop 2024-05-03T18:23:08.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:08.724+0000 7f3f02200700 1 -- 172.21.15.33:0/3600274542 shutdown_connections 2024-05-03T18:23:08.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:08.724+0000 7f3f02200700 1 --2- 172.21.15.33:0/3600274542 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3efc105f90 0x7f3efc1124a0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:08.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:08.724+0000 7f3f02200700 1 --2- 172.21.15.33:0/3600274542 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3efc10b660 0x7f3efc10ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:08.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:08.724+0000 7f3f02200700 1 --2- 172.21.15.33:0/3600274542 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3efc1055d0 0x7f3efc105a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:08.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:08.724+0000 7f3f02200700 1 -- 172.21.15.33:0/3600274542 >> 172.21.15.33:0/3600274542 conn(0x7f3efc1010c0 msgr2=0x7f3efc1034e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:23:08.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:08.724+0000 7f3f02200700 1 -- 172.21.15.33:0/3600274542 shutdown_connections 2024-05-03T18:23:08.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:08.724+0000 7f3f02200700 1 -- 172.21.15.33:0/3600274542 wait complete. 2024-05-03T18:23:08.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:08.725+0000 7f3f02200700 1 Processor -- start 2024-05-03T18:23:08.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:08.725+0000 7f3f02200700 1 -- start start 2024-05-03T18:23:08.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:08.726+0000 7f3f02200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3efc1055d0 0x7f3efc1a1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:23:08.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:08.726+0000 7f3f02200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3efc105f90 0x7f3efc1a1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:23:08.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:08.726+0000 7f3f02200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3efc10b660 0x7f3efc19b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:23:08.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:08.726+0000 7f3f02200700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3efc115710 con 0x7f3efc1055d0 2024-05-03T18:23:08.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:08.726+0000 7f3f02200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3efc115590 con 0x7f3efc10b660 2024-05-03T18:23:08.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:08.726+0000 7f3f02200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3efc115890 con 0x7f3efc105f90 2024-05-03T18:23:08.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:08.726+0000 7f3f00e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3efc1055d0 0x7f3efc1a1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:23:08.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:08.726+0000 7f3f00e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3efc1055d0 0x7f3efc1a1760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46090/0 (socket says 172.21.15.33:46090) 2024-05-03T18:23:08.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:08.726+0000 7f3f00e00700 1 -- 172.21.15.33:0/2071045807 learned_addr learned my addr 172.21.15.33:0/2071045807 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:23:08.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:08.726+0000 7f3f01800700 1 --2- 172.21.15.33:0/2071045807 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3efc10b660 0x7f3efc19b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:23:08.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:08.727+0000 7f3efbe00700 1 --2- 172.21.15.33:0/2071045807 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3efc105f90 0x7f3efc1a1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:23:08.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:08.727+0000 7f3f00e00700 1 -- 172.21.15.33:0/2071045807 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3efc105f90 msgr2=0x7f3efc1a1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:23:08.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:08.727+0000 7f3f00e00700 1 --2- 172.21.15.33:0/2071045807 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3efc105f90 0x7f3efc1a1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:08.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:08.727+0000 7f3f00e00700 1 -- 172.21.15.33:0/2071045807 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3efc10b660 msgr2=0x7f3efc19b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:23:08.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:08.727+0000 7f3f00e00700 1 --2- 172.21.15.33:0/2071045807 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3efc10b660 0x7f3efc19b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:08.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:08.727+0000 7f3f00e00700 1 -- 172.21.15.33:0/2071045807 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3efc19c0d0 con 0x7f3efc1055d0 2024-05-03T18:23:08.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:08.727+0000 7f3efbe00700 1 --2- 172.21.15.33:0/2071045807 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3efc105f90 0x7f3efc1a1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:23:08.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:08.727+0000 7f3f01800700 1 --2- 172.21.15.33:0/2071045807 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3efc10b660 0x7f3efc19b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:23:08.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:08.727+0000 7f3f00e00700 1 --2- 172.21.15.33:0/2071045807 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3efc1055d0 0x7f3efc1a1760 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3ef000b500 tx=0x7f3ef000b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:23:08.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:08.727+0000 7f3ef9600700 1 -- 172.21.15.33:0/2071045807 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef0015020 con 0x7f3efc1055d0 2024-05-03T18:23:08.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:08.727+0000 7f3ef9600700 1 -- 172.21.15.33:0/2071045807 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3ef0011cc0 con 0x7f3efc1055d0 2024-05-03T18:23:08.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:08.727+0000 7f3ef9600700 1 -- 172.21.15.33:0/2071045807 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef0010ca0 con 0x7f3efc1055d0 2024-05-03T18:23:08.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:08.727+0000 7f3f02200700 1 -- 172.21.15.33:0/2071045807 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3efc19c360 con 0x7f3efc1055d0 2024-05-03T18:23:08.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:08.728+0000 7f3f02200700 1 -- 172.21.15.33:0/2071045807 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3efc1a8630 con 0x7f3efc1055d0 2024-05-03T18:23:08.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:08.729+0000 7f3f02200700 1 -- 172.21.15.33:0/2071045807 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3efc06a190 con 0x7f3efc1055d0 2024-05-03T18:23:08.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:08.731+0000 7f3ef9600700 1 -- 172.21.15.33:0/2071045807 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f3ef0018020 con 0x7f3efc1055d0 2024-05-03T18:23:08.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:08.732+0000 7f3ef9600700 1 --2- 172.21.15.33:0/2071045807 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f3ee40b2900 0x7f3ee40b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:23:08.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:08.733+0000 7f3ef9600700 1 -- 172.21.15.33:0/2071045807 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7f3ef00e2250 con 0x7f3efc1055d0 2024-05-03T18:23:08.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:08.733+0000 7f3efbe00700 1 --2- 172.21.15.33:0/2071045807 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f3ee40b2900 0x7f3ee40b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:23:08.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:08.734+0000 7f3efbe00700 1 --2- 172.21.15.33:0/2071045807 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f3ee40b2900 0x7f3ee40b4dc0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3ee8005fd0 tx=0x7f3ee8005e20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:23:08.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:08.736+0000 7f3ef9600700 1 -- 172.21.15.33:0/2071045807 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f3ef001f020 con 0x7f3efc1055d0 2024-05-03T18:23:08.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:23:08 smithi099 ceph-mon[31658]: pgmap v399: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:23:08.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:23:08 smithi150 ceph-mon[32492]: pgmap v399: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:23:08.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:08.975+0000 7f3f02200700 1 -- 172.21.15.33:0/2071045807 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3efc19ce50 con 0x7f3efc1055d0 2024-05-03T18:23:08.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:08.979+0000 7f3ef9600700 1 -- 172.21.15.33:0/2071045807 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+24057 (secure 0 0 0) 0x7f3ef00a9cd0 con 0x7f3efc1055d0 2024-05-03T18:23:08.979 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:23:08.980 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":85,"fsid":"5703bba4-0978-11ef-bc97-c7b262605968","created":"2024-05-03T18:11:00.899879+0000","modified":"2024-05-03T18:22:48.802907+0000","last_up_change":"2024-05-03T18:21:00.390173+0000","last_in_change":"2024-05-03T18:20:34.103727+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-03T18:15:37.297738+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-03T18:22:19.875368+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-03T18:22:24.432692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-03T18:22:38.722443+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"fdcc4e66-4ba0-4378-a9a4-7d907cd00b4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":637686669},{"type":"v1","addr":"172.21.15.33:6803","nonce":637686669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":637686669},{"type":"v1","addr":"172.21.15.33:6805","nonce":637686669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":637686669},{"type":"v1","addr":"172.21.15.33:6809","nonce":637686669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":637686669},{"type":"v1","addr":"172.21.15.33:6807","nonce":637686669}]},"public_addr":"172.21.15.33:6803/637686669","cluster_addr":"172.21.15.33:6805/637686669","heartbeat_back_addr":"172.21.15.33:6809/637686669","heartbeat_front_addr":"172.21.15.33:6807/637686669","state":["exists","up"]},{"osd":1,"uuid":"cf78cad3-fc24-43f4-8784-21cabea7277c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":1536122841},{"type":"v1","addr":"172.21.15.99:6801","nonce":1536122841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":1536122841},{"type":"v1","addr":"172.21.15.99:6803","nonce":1536122841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":1536122841},{"type":"v1","addr":"172.21.15.99:6807","nonce":1536122841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":1536122841},{"type":"v1","addr":"172.21.15.99:6805","nonce":1536122841}]},"public_addr":"172.21.15.99:6801/1536122841","cluster_addr":"172.21.15.99:6803/1536122841","heartbeat_back_addr":"172.21.15.99:6807/1536122841","heartbeat_front_addr":"172.21.15.99:6805/1536122841","state":["exists","up"]},{"osd":2,"uuid":"59f066d7-626b-410a-8d67-daca6fbbd3ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":1995659622},{"type":"v1","addr":"172.21.15.150:6801","nonce":1995659622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":1995659622},{"type":"v1","addr":"172.21.15.150:6803","nonce":1995659622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":1995659622},{"type":"v1","addr":"172.21.15.150:6807","nonce":1995659622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":1995659622},{"type":"v1","addr":"172.21.15.150:6805","nonce":1995659622}]},"public_addr":"172.21.15.150:6801/1995659622","cluster_addr":"172.21.15.150:6803/1995659622","heartbeat_back_addr":"172.21.15.150:6807/1995659622","heartbeat_front_addr":"172.21.15.150:6805/1995659622","state":["exists","up"]},{"osd":3,"uuid":"d88ed022-4dcb-45fc-9fc6-41ef7c4bf202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":3814942724},{"type":"v1","addr":"172.21.15.33:6811","nonce":3814942724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":3814942724},{"type":"v1","addr":"172.21.15.33:6813","nonce":3814942724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":3814942724},{"type":"v1","addr":"172.21.15.33:6817","nonce":3814942724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":3814942724},{"type":"v1","addr":"172.21.15.33:6815","nonce":3814942724}]},"public_addr":"172.21.15.33:6811/3814942724","cluster_addr":"172.21.15.33:6813/3814942724","heartbeat_back_addr":"172.21.15.33:6817/3814942724","heartbeat_front_addr":"172.21.15.33:6815/3814942724","state":["exists","up"]},{"osd":4,"uuid":"1235e1a6-ff6e-4668-ac1a-073c2a3b7c98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":29081029},{"type":"v1","addr":"172.21.15.99:6809","nonce":29081029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":29081029},{"type":"v1","addr":"172.21.15.99:6811","nonce":29081029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":29081029},{"type":"v1","addr":"172.21.15.99:6815","nonce":29081029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":29081029},{"type":"v1","addr":"172.21.15.99:6813","nonce":29081029}]},"public_addr":"172.21.15.99:6809/29081029","cluster_addr":"172.21.15.99:6811/29081029","heartbeat_back_addr":"172.21.15.99:6815/29081029","heartbeat_front_addr":"172.21.15.99:6813/29081029","state":["exists","up"]},{"osd":5,"uuid":"4b84bff8-d6d7-4258-a0ee-53bc19802585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":3172189533},{"type":"v1","addr":"172.21.15.150:6809","nonce":3172189533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3172189533},{"type":"v1","addr":"172.21.15.150:6811","nonce":3172189533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3172189533},{"type":"v1","addr":"172.21.15.150:6815","nonce":3172189533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3172189533},{"type":"v1","addr":"172.21.15.150:6813","nonce":3172189533}]},"public_addr":"172.21.15.150:6809/3172189533","cluster_addr":"172.21.15.150:6811/3172189533","heartbeat_back_addr":"172.21.15.150:6815/3172189533","heartbeat_front_addr":"172.21.15.150:6813/3172189533","state":["exists","up"]},{"osd":6,"uuid":"51f5fbdd-3794-4c48-9f6c-478f0fa35ac5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":4234541418},{"type":"v1","addr":"172.21.15.33:6819","nonce":4234541418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":4234541418},{"type":"v1","addr":"172.21.15.33:6821","nonce":4234541418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":4234541418},{"type":"v1","addr":"172.21.15.33:6825","nonce":4234541418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":4234541418},{"type":"v1","addr":"172.21.15.33:6823","nonce":4234541418}]},"public_addr":"172.21.15.33:6819/4234541418","cluster_addr":"172.21.15.33:6821/4234541418","heartbeat_back_addr":"172.21.15.33:6825/4234541418","heartbeat_front_addr":"172.21.15.33:6823/4234541418","state":["exists","up"]},{"osd":7,"uuid":"982cc5b3-3163-4505-9bde-a27a8c289457","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":908673090},{"type":"v1","addr":"172.21.15.99:6817","nonce":908673090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":908673090},{"type":"v1","addr":"172.21.15.99:6819","nonce":908673090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":908673090},{"type":"v1","addr":"172.21.15.99:6823","nonce":908673090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":908673090},{"type":"v1","addr":"172.21.15.99:6821","nonce":908673090}]},"public_addr":"172.21.15.99:6817/908673090","cluster_addr":"172.21.15.99:6819/908673090","heartbeat_back_addr":"172.21.15.99:6823/908673090","heartbeat_front_addr":"172.21.15.99:6821/908673090","state":["exists","up"]},{"osd":8,"uuid":"84c9d037-eb84-4eb4-8aa2-e784b703d36c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":1308083602},{"type":"v1","addr":"172.21.15.150:6817","nonce":1308083602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":1308083602},{"type":"v1","addr":"172.21.15.150:6819","nonce":1308083602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":1308083602},{"type":"v1","addr":"172.21.15.150:6823","nonce":1308083602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":1308083602},{"type":"v1","addr":"172.21.15.150:6821","nonce":1308083602}]},"public_addr":"172.21.15.150:6817/1308083602","cluster_addr":"172.21.15.150:6819/1308083602","heartbeat_back_addr":"172.21.15.150:6823/1308083602","heartbeat_front_addr":"172.21.15.150:6821/1308083602","state":["exists","up"]},{"osd":9,"uuid":"a02e965f-0d8f-45e2-9490-337f7f6e3881","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":4133096905},{"type":"v1","addr":"172.21.15.33:6827","nonce":4133096905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":4133096905},{"type":"v1","addr":"172.21.15.33:6829","nonce":4133096905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4133096905},{"type":"v1","addr":"172.21.15.33:6833","nonce":4133096905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":4133096905},{"type":"v1","addr":"172.21.15.33:6831","nonce":4133096905}]},"public_addr":"172.21.15.33:6827/4133096905","cluster_addr":"172.21.15.33:6829/4133096905","heartbeat_back_addr":"172.21.15.33:6833/4133096905","heartbeat_front_addr":"172.21.15.33:6831/4133096905","state":["exists","up"]},{"osd":10,"uuid":"8b29bb96-76e8-45a0-ab19-8d082d755fe9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":935746136},{"type":"v1","addr":"172.21.15.99:6825","nonce":935746136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":935746136},{"type":"v1","addr":"172.21.15.99:6827","nonce":935746136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":935746136},{"type":"v1","addr":"172.21.15.99:6831","nonce":935746136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":935746136},{"type":"v1","addr":"172.21.15.99:6829","nonce":935746136}]},"public_addr":"172.21.15.99:6825/935746136","cluster_addr":"172.21.15.99:6827/935746136","heartbeat_back_addr":"172.21.15.99:6831/935746136","heartbeat_front_addr":"172.21.15.99:6829/935746136","state":["exists","up"]},{"osd":11,"uuid":"07a6a825-c902-4bfd-97cc-cc9074532af5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2069349808},{"type":"v1","addr":"172.21.15.150:6825","nonce":2069349808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2069349808},{"type":"v1","addr":"172.21.15.150:6827","nonce":2069349808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2069349808},{"type":"v1","addr":"172.21.15.150:6831","nonce":2069349808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2069349808},{"type":"v1","addr":"172.21.15.150:6829","nonce":2069349808}]},"public_addr":"172.21.15.150:6825/2069349808","cluster_addr":"172.21.15.150:6827/2069349808","heartbeat_back_addr":"172.21.15.150:6831/2069349808","heartbeat_front_addr":"172.21.15.150:6829/2069349808","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:14:27.454536+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:15:00.538570+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:15:34.125289+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:16:09.853870+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:16:44.928107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:17:19.644037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:17:56.108443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:18:31.731499+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:19:08.062358+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:19:45.204430+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:20:21.013710+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:20:57.653698+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/3991947997":"2024-05-04T18:12:19.127652+0000","172.21.15.33:0/965859483":"2024-05-04T18:12:19.127652+0000","172.21.15.33:0/2014902144":"2024-05-04T18:11:44.239835+0000","172.21.15.33:0/1314704208":"2024-05-04T18:11:44.239835+0000","172.21.15.33:0/1614640996":"2024-05-04T18:11:44.239835+0000","172.21.15.33:6800/4007322071":"2024-05-04T18:11:44.239835+0000","172.21.15.33:6801/2830135294":"2024-05-04T18:12:19.127652+0000","172.21.15.33:6800/2830135294":"2024-05-04T18:12:19.127652+0000","172.21.15.33:0/3767665033":"2024-05-04T18:12:19.127652+0000","172.21.15.33:6801/4007322071":"2024-05-04T18:11:44.239835+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-03T18:23:08.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:08.982+0000 7f3f02200700 1 -- 172.21.15.33:0/2071045807 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f3ee40b2900 msgr2=0x7f3ee40b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:23:08.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:08.982+0000 7f3f02200700 1 --2- 172.21.15.33:0/2071045807 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f3ee40b2900 0x7f3ee40b4dc0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3ee8005fd0 tx=0x7f3ee8005e20 comp rx=0 tx=0).stop 2024-05-03T18:23:08.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:08.982+0000 7f3f02200700 1 -- 172.21.15.33:0/2071045807 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3efc1055d0 msgr2=0x7f3efc1a1760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:23:08.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:08.982+0000 7f3f02200700 1 --2- 172.21.15.33:0/2071045807 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3efc1055d0 0x7f3efc1a1760 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3ef000b500 tx=0x7f3ef000b8c0 comp rx=0 tx=0).stop 2024-05-03T18:23:08.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:08.983+0000 7f3f02200700 1 -- 172.21.15.33:0/2071045807 shutdown_connections 2024-05-03T18:23:08.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:08.983+0000 7f3f02200700 1 --2- 172.21.15.33:0/2071045807 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f3ee40b2900 0x7f3ee40b4dc0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:08.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:08.983+0000 7f3f02200700 1 --2- 172.21.15.33:0/2071045807 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3efc10b660 0x7f3efc19b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:08.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:08.983+0000 7f3f02200700 1 --2- 172.21.15.33:0/2071045807 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3efc1055d0 0x7f3efc1a1760 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:08.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:08.983+0000 7f3f02200700 1 --2- 172.21.15.33:0/2071045807 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3efc105f90 0x7f3efc1a1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:08.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:08.983+0000 7f3f02200700 1 -- 172.21.15.33:0/2071045807 >> 172.21.15.33:0/2071045807 conn(0x7f3efc1010c0 msgr2=0x7f3efc102680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:23:08.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:08.983+0000 7f3f02200700 1 -- 172.21.15.33:0/2071045807 shutdown_connections 2024-05-03T18:23:08.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:08.983+0000 7f3f02200700 1 -- 172.21.15.33:0/2071045807 wait complete. 2024-05-03T18:23:09.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:23:09 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/2071045807' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T18:23:09.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:23:09 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/2071045807' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T18:23:09.876 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-03T18:22:28.837091+0000', 'modified': '2024-05-03T18:22:48.803163+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 34508}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_34508': {'gid': 34508, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.99:6839/919907308', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6838', 'nonce': 919907308}, {'type': 'v1', 'addr': '172.21.15.99:6839', 'nonce': 919907308}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 34508, 'qdb_cluster': [34508]} 2024-05-03T18:23:09.876 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-03T18:23:09.876 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.b status 2024-05-03T18:23:09.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:23:09 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/2071045807' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T18:23:10.354 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:23:10.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:23:10 smithi033 ceph-mon[28330]: pgmap v400: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:23:10.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:23:10 smithi099 ceph-mon[31658]: pgmap v400: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:23:10.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:23:10 smithi150 ceph-mon[32492]: pgmap v400: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:23:12.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.535+0000 7f7363400700 1 -- 172.21.15.33:0/1034152357 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f735c06b180 msgr2=0x7f735c06b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:23:12.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.535+0000 7f7363400700 1 --2- 172.21.15.33:0/1034152357 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f735c06b180 0x7f735c06b600 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f734c009990 tx=0x7f734c009ca0 comp rx=0 tx=0).stop 2024-05-03T18:23:12.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.535+0000 7f7363400700 1 -- 172.21.15.33:0/1034152357 shutdown_connections 2024-05-03T18:23:12.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.535+0000 7f7363400700 1 --2- 172.21.15.33:0/1034152357 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f735c06bb40 0x7f735c110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:12.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.535+0000 7f7363400700 1 --2- 172.21.15.33:0/1034152357 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f735c109510 0x7f735c1098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:12.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.535+0000 7f7363400700 1 --2- 172.21.15.33:0/1034152357 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f735c06b180 0x7f735c06b600 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:12.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.535+0000 7f7363400700 1 -- 172.21.15.33:0/1034152357 >> 172.21.15.33:0/1034152357 conn(0x7f735c079350 msgr2=0x7f735c079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:23:12.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.535+0000 7f7363400700 1 -- 172.21.15.33:0/1034152357 shutdown_connections 2024-05-03T18:23:12.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.535+0000 7f7363400700 1 -- 172.21.15.33:0/1034152357 wait complete. 2024-05-03T18:23:12.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.536+0000 7f7363400700 1 Processor -- start 2024-05-03T18:23:12.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.537+0000 7f7363400700 1 -- start start 2024-05-03T18:23:12.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.537+0000 7f7363400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f735c06b180 0x7f735c1a1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:23:12.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.537+0000 7f7363400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f735c06bb40 0x7f735c1a1f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:23:12.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.537+0000 7f7363400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f735c109510 0x7f735c19bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:23:12.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.537+0000 7f7363400700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f735c113520 con 0x7f735c06b180 2024-05-03T18:23:12.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.537+0000 7f7363400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f735c1133a0 con 0x7f735c109510 2024-05-03T18:23:12.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.537+0000 7f7363400700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f735c1136a0 con 0x7f735c06bb40 2024-05-03T18:23:12.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.537+0000 7f7360a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f735c06b180 0x7f735c1a1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:23:12.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.537+0000 7f7360a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f735c06b180 0x7f735c1a1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46094/0 (socket says 172.21.15.33:46094) 2024-05-03T18:23:12.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.537+0000 7f7360a00700 1 -- 172.21.15.33:0/3528148258 learned_addr learned my addr 172.21.15.33:0/3528148258 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:23:12.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.538+0000 7f7353e00700 1 --2- 172.21.15.33:0/3528148258 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f735c06bb40 0x7f735c1a1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:23:12.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.538+0000 7f7361400700 1 --2- 172.21.15.33:0/3528148258 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f735c109510 0x7f735c19bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:23:12.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.538+0000 7f7360a00700 1 -- 172.21.15.33:0/3528148258 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f735c06bb40 msgr2=0x7f735c1a1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:23:12.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.538+0000 7f7360a00700 1 --2- 172.21.15.33:0/3528148258 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f735c06bb40 0x7f735c1a1f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:12.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.538+0000 7f7360a00700 1 -- 172.21.15.33:0/3528148258 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f735c109510 msgr2=0x7f735c19bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:23:12.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.538+0000 7f7360a00700 1 --2- 172.21.15.33:0/3528148258 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f735c109510 0x7f735c19bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:12.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.538+0000 7f7360a00700 1 -- 172.21.15.33:0/3528148258 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f735c19c400 con 0x7f735c06b180 2024-05-03T18:23:12.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.538+0000 7f7361400700 1 --2- 172.21.15.33:0/3528148258 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f735c109510 0x7f735c19bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:23:12.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.538+0000 7f7360a00700 1 --2- 172.21.15.33:0/3528148258 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f735c06b180 0x7f735c1a1a50 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f734400d950 tx=0x7f734400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:23:12.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.538+0000 7f735a000700 1 -- 172.21.15.33:0/3528148258 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7344016070 con 0x7f735c06b180 2024-05-03T18:23:12.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.538+0000 7f735a000700 1 -- 172.21.15.33:0/3528148258 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7344010460 con 0x7f735c06b180 2024-05-03T18:23:12.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.538+0000 7f735a000700 1 -- 172.21.15.33:0/3528148258 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f734400f620 con 0x7f735c06b180 2024-05-03T18:23:12.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.538+0000 7f7363400700 1 -- 172.21.15.33:0/3528148258 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f735c19c6f0 con 0x7f735c06b180 2024-05-03T18:23:12.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.539+0000 7f7363400700 1 -- 172.21.15.33:0/3528148258 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f735c1a89d0 con 0x7f735c06b180 2024-05-03T18:23:12.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.542+0000 7f735a000700 1 -- 172.21.15.33:0/3528148258 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f7344019070 con 0x7f735c06b180 2024-05-03T18:23:12.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.544+0000 7f736de10b80 1 Processor -- start 2024-05-03T18:23:12.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.544+0000 7f736de10b80 1 -- start start 2024-05-03T18:23:12.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.544+0000 7f735a000700 1 --2- 172.21.15.33:0/3528148258 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f733c0b2900 0x7f733c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:23:12.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.544+0000 7f736de10b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556a05bf38e0 0x556a05bf3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:23:12.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.544+0000 7f736de10b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a05bf4200 0x556a05bedee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:23:12.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.544+0000 7f736de10b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556a05bee500 0x556a05bee980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:23:12.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.544+0000 7f735a000700 1 -- 172.21.15.33:0/3528148258 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7f73440e1fc0 con 0x7f735c06b180 2024-05-03T18:23:12.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.544+0000 7f736de10b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556a05b409d0 con 0x556a05bf4200 2024-05-03T18:23:12.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.544+0000 7f736de10b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x556a05ade710 con 0x556a05bf38e0 2024-05-03T18:23:12.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.544+0000 7f736de10b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x556a05b3c990 con 0x556a05bee500 2024-05-03T18:23:12.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.544+0000 7f7361400700 1 --2- 172.21.15.33:0/3528148258 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f733c0b2900 0x7f733c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:23:12.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.545+0000 7f7353e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a05bf4200 0x556a05bedee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:23:12.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.545+0000 7f7353e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a05bf4200 0x556a05bedee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46110/0 (socket says 172.21.15.33:46110) 2024-05-03T18:23:12.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.545+0000 7f7353e00700 1 -- 172.21.15.33:0/3629263332 learned_addr learned my addr 172.21.15.33:0/3629263332 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:23:12.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.545+0000 7f7353e00700 1 -- 172.21.15.33:0/3629263332 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556a05bee500 msgr2=0x556a05bee980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:23:12.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.545+0000 7f7353e00700 1 --2- 172.21.15.33:0/3629263332 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556a05bee500 0x556a05bee980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:12.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.545+0000 7f7353e00700 1 -- 172.21.15.33:0/3629263332 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556a05bf38e0 msgr2=0x556a05bf3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:23:12.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.545+0000 7f7360a00700 1 --2- 172.21.15.33:0/3629263332 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556a05bf38e0 0x556a05bf3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:23:12.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.545+0000 7f7353e00700 1 --2- 172.21.15.33:0/3629263332 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556a05bf38e0 0x556a05bf3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:12.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.545+0000 7f7353e00700 1 -- 172.21.15.33:0/3629263332 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a05a8d880 con 0x556a05bf4200 2024-05-03T18:23:12.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.545+0000 7f7360a00700 1 --2- 172.21.15.33:0/3629263332 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556a05bf38e0 0x556a05bf3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:23:12.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.546+0000 7f7361400700 1 --2- 172.21.15.33:0/3528148258 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f733c0b2900 0x7f733c0b4dc0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f735400d260 tx=0x7f735400d620 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:23:12.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.546+0000 7f7353e00700 1 --2- 172.21.15.33:0/3629263332 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a05bf4200 0x556a05bedee0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f734c000c00 tx=0x7f734c0096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edf526babbb267a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:23:12.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.546+0000 7f7352000700 1 -- 172.21.15.33:0/3629263332 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f734c026070 con 0x556a05bf4200 2024-05-03T18:23:12.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.546+0000 7f7352000700 1 -- 172.21.15.33:0/3629263332 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f734c0054c0 con 0x556a05bf4200 2024-05-03T18:23:12.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.546+0000 7f7352000700 1 -- 172.21.15.33:0/3629263332 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f734c00b7d0 con 0x556a05bf4200 2024-05-03T18:23:12.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.547+0000 7f736de10b80 1 -- 172.21.15.33:0/3629263332 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a05bf4200 msgr2=0x556a05bedee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:23:12.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.547+0000 7f736de10b80 1 --2- 172.21.15.33:0/3629263332 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a05bf4200 0x556a05bedee0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f734c000c00 tx=0x7f734c0096c0 comp rx=0 tx=0).stop 2024-05-03T18:23:12.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.547+0000 7f736de10b80 1 -- 172.21.15.33:0/3629263332 shutdown_connections 2024-05-03T18:23:12.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.547+0000 7f736de10b80 1 --2- 172.21.15.33:0/3629263332 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556a05bee500 0x556a05bee980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:12.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.547+0000 7f736de10b80 1 --2- 172.21.15.33:0/3629263332 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556a05bf38e0 0x556a05bf3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:12.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.547+0000 7f736de10b80 1 --2- 172.21.15.33:0/3629263332 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a05bf4200 0x556a05bedee0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:12.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.547+0000 7f736de10b80 1 -- 172.21.15.33:0/3629263332 >> 172.21.15.33:0/3629263332 conn(0x556a05b66680 msgr2=0x556a05be88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:23:12.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.547+0000 7f736de10b80 1 -- 172.21.15.33:0/3629263332 shutdown_connections 2024-05-03T18:23:12.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.547+0000 7f736de10b80 1 -- 172.21.15.33:0/3629263332 wait complete. 2024-05-03T18:23:12.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.548+0000 7f736de10b80 1 -- start start 2024-05-03T18:23:12.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.548+0000 7f736de10b80 1 Processor -- start 2024-05-03T18:23:12.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.549+0000 7f736de10b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a05bee500 0x556a05bf9e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:23:12.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.549+0000 7f736de10b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556a05bf38e0 0x556a05bf7220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:23:12.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.549+0000 7f736de10b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556a05bf4200 0x556a05bf7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:23:12.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.549+0000 7f736de10b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556a05bfac50 con 0x556a05bee500 2024-05-03T18:23:12.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.549+0000 7f736de10b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x556a05aa6ad0 con 0x556a05bf4200 2024-05-03T18:23:12.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.549+0000 7f736de10b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x556a05bfadd0 con 0x556a05bf38e0 2024-05-03T18:23:12.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.549+0000 7f736de10b80 10 client.0 _ll_register_callbacks cb 0x556a05ae7c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:23:12.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.549+0000 7f7360a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a05bee500 0x556a05bf9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:23:12.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.549+0000 7f7360a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a05bee500 0x556a05bf9e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46118/0 (socket says 172.21.15.33:46118) 2024-05-03T18:23:12.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.549+0000 7f7360a00700 1 -- 172.21.15.33:0/2661196053 learned_addr learned my addr 172.21.15.33:0/2661196053 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:23:12.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.549+0000 7f7353e00700 1 --2- 172.21.15.33:0/2661196053 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556a05bf38e0 0x556a05bf7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:23:12.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.549+0000 7f7361400700 1 --2- 172.21.15.33:0/2661196053 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556a05bf4200 0x556a05bf7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:23:12.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.549+0000 7f7360a00700 1 -- 172.21.15.33:0/2661196053 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556a05bf38e0 msgr2=0x556a05bf7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:23:12.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.549+0000 7f7360a00700 1 --2- 172.21.15.33:0/2661196053 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556a05bf38e0 0x556a05bf7220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:12.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.549+0000 7f7360a00700 1 -- 172.21.15.33:0/2661196053 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556a05bf4200 msgr2=0x556a05bf7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:23:12.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.549+0000 7f7360a00700 1 --2- 172.21.15.33:0/2661196053 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556a05bf4200 0x556a05bf7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:12.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.549+0000 7f7360a00700 1 -- 172.21.15.33:0/2661196053 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a05b063d0 con 0x556a05bee500 2024-05-03T18:23:12.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.550+0000 7f7360a00700 1 --2- 172.21.15.33:0/2661196053 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a05bee500 0x556a05bf9e30 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f73440df790 tx=0x7f73440a9af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25a13ded634cd767 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:23:12.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.550+0000 7f734aa00700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:23:12.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.550+0000 7f734aa00700 1 -- 172.21.15.33:0/2661196053 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7344020040 con 0x556a05bee500 2024-05-03T18:23:12.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.550+0000 7f734aa00700 1 -- 172.21.15.33:0/2661196053 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7f73440a95a0 con 0x556a05bee500 2024-05-03T18:23:12.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.550+0000 7f736de10b80 1 -- 172.21.15.33:0/2661196053 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f734c0054c0 con 0x556a05bee500 2024-05-03T18:23:12.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.550+0000 7f734aa00700 1 -- 172.21.15.33:0/2661196053 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f73440b21a0 con 0x556a05bee500 2024-05-03T18:23:12.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.550+0000 7f736de10b80 10 client.14676 fetch_fsmap learned FSMap version 17 2024-05-03T18:23:12.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.550+0000 7f736de10b80 1 -- 172.21.15.33:0/2661196053 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x556a05c05360 con 0x556a05bee500 2024-05-03T18:23:12.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.551+0000 7f734aa00700 1 -- 172.21.15.33:0/2661196053 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f734c0054c0 con 0x556a05bee500 2024-05-03T18:23:12.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.551+0000 7f734aa00700 1 -- 172.21.15.33:0/2661196053 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7f73440dfce0 con 0x556a05bee500 2024-05-03T18:23:12.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.551+0000 7f736de10b80 10 client.14676 fetch_fsmap finished waiting for FSMap version 17 2024-05-03T18:23:12.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.551+0000 7f736de10b80 10 client.14676 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-03T18:23:12.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.551+0000 7f736de10b80 20 client.14676 populate_metadata read hostname 'smithi033' 2024-05-03T18:23:12.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.551+0000 7f736de10b80 1 --2- 172.21.15.33:0/2661196053 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x556a05c079a0 0x556a05c08e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:23:12.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.551+0000 7f736de10b80 4 client.14676 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:23:12.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.551+0000 7f736de10b80 1 -- 172.21.15.33:0/2661196053 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556a05bbafa0 con 0x556a05c079a0 2024-05-03T18:23:12.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.552+0000 7f7353e00700 1 --2- 172.21.15.33:0/2661196053 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x556a05c079a0 0x556a05c08e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:23:12.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.553+0000 7f7353e00700 1 --2- 172.21.15.33:0/2661196053 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x556a05c079a0 0x556a05c08e10 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51e42ba0ee495a23 server_cookie=8bb27b9236d5c90a in_seq=0 out_seq=0 2024-05-03T18:23:12.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.553+0000 7f734aa00700 10 client.14676 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:23:12.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.556+0000 7f734aa00700 1 -- 172.21.15.33:0/2661196053 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556a05bbafa0 con 0x556a05c079a0 2024-05-03T18:23:12.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.556+0000 7f734aa00700 10 client.14676 handle_command_reply: tid=0 2024-05-03T18:23:12.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.556+0000 7f736de10b80 1 client.14676 shutdown 2024-05-03T18:23:12.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.556+0000 7f736de10b80 1 -- 172.21.15.33:0/2661196053 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a05bee500 msgr2=0x556a05bf9e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:23:12.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.556+0000 7f736de10b80 1 --2- 172.21.15.33:0/2661196053 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a05bee500 0x556a05bf9e30 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f73440df790 tx=0x7f73440a9af0 comp rx=0 tx=0).stop 2024-05-03T18:23:12.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.556+0000 7f736de10b80 1 -- 172.21.15.33:0/2661196053 shutdown_connections 2024-05-03T18:23:12.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.556+0000 7f736de10b80 1 --2- 172.21.15.33:0/2661196053 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x556a05c079a0 0x556a05c08e10 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:12.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.556+0000 7f736de10b80 1 --2- 172.21.15.33:0/2661196053 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556a05bf4200 0x556a05bf7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:12.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.556+0000 7f736de10b80 1 --2- 172.21.15.33:0/2661196053 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a05bee500 0x556a05bf9e30 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:12.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.556+0000 7f736de10b80 1 --2- 172.21.15.33:0/2661196053 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556a05bf38e0 0x556a05bf7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:12.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.556+0000 7f734aa00700 0 client.14676 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:23:12.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.556+0000 7f736de10b80 1 -- 172.21.15.33:0/2661196053 >> 172.21.15.33:0/2661196053 conn(0x556a05b66680 msgr2=0x556a05be3360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:23:12.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.556+0000 7f736de10b80 1 -- 172.21.15.33:0/2661196053 shutdown_connections 2024-05-03T18:23:12.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.556+0000 7f736de10b80 1 -- 172.21.15.33:0/2661196053 wait complete. 2024-05-03T18:23:12.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.557+0000 7f736de10b80 20 client.14676 trim_cache size 0 max 16384 2024-05-03T18:23:12.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.580+0000 7f736de10b80 1 Processor -- start 2024-05-03T18:23:12.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.580+0000 7f736de10b80 1 -- start start 2024-05-03T18:23:12.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.580+0000 7f736de10b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a05bee500 0x556a05bedee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:23:12.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.580+0000 7f736de10b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556a05bf38e0 0x556a05c12270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:23:12.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.580+0000 7f736de10b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556a05bf4200 0x556a05c127b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:23:12.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.580+0000 7f736de10b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556a05c07150 con 0x556a05bee500 2024-05-03T18:23:12.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.580+0000 7f736de10b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x556a05c07820 con 0x556a05bf4200 2024-05-03T18:23:12.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.580+0000 7f736de10b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x556a05c07ef0 con 0x556a05bf38e0 2024-05-03T18:23:12.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.581+0000 7f7360a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a05bee500 0x556a05bedee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:23:12.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.581+0000 7f7360a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a05bee500 0x556a05bedee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46122/0 (socket says 172.21.15.33:46122) 2024-05-03T18:23:12.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.581+0000 7f7360a00700 1 -- 172.21.15.33:0/1111502730 learned_addr learned my addr 172.21.15.33:0/1111502730 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:23:12.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.581+0000 7f7353e00700 1 --2- 172.21.15.33:0/1111502730 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556a05bf38e0 0x556a05c12270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:23:12.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.581+0000 7f7361400700 1 --2- 172.21.15.33:0/1111502730 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556a05bf4200 0x556a05c127b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:23:12.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.581+0000 7f7360a00700 1 -- 172.21.15.33:0/1111502730 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556a05bf38e0 msgr2=0x556a05c12270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:23:12.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.581+0000 7f7360a00700 1 --2- 172.21.15.33:0/1111502730 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556a05bf38e0 0x556a05c12270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:12.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.581+0000 7f7360a00700 1 -- 172.21.15.33:0/1111502730 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556a05bf4200 msgr2=0x556a05c127b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:23:12.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.581+0000 7f7360a00700 1 --2- 172.21.15.33:0/1111502730 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556a05bf4200 0x556a05c127b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:12.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.581+0000 7f7360a00700 1 -- 172.21.15.33:0/1111502730 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a05bf9a70 con 0x556a05bee500 2024-05-03T18:23:12.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.581+0000 7f7360a00700 1 --2- 172.21.15.33:0/1111502730 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a05bee500 0x556a05bedee0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7344005950 tx=0x7f7344000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3886575b15898e95 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:23:12.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.582+0000 7f7351600700 1 -- 172.21.15.33:0/1111502730 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7344020030 con 0x556a05bee500 2024-05-03T18:23:12.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.582+0000 7f7351600700 1 -- 172.21.15.33:0/1111502730 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f73440a8640 con 0x556a05bee500 2024-05-03T18:23:12.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.582+0000 7f7351600700 1 -- 172.21.15.33:0/1111502730 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73440a9800 con 0x556a05bee500 2024-05-03T18:23:12.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.582+0000 7f736de10b80 1 -- 172.21.15.33:0/1111502730 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a05bee500 msgr2=0x556a05bedee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:23:12.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.582+0000 7f736de10b80 1 --2- 172.21.15.33:0/1111502730 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a05bee500 0x556a05bedee0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7344005950 tx=0x7f7344000bc0 comp rx=0 tx=0).stop 2024-05-03T18:23:12.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.583+0000 7f736de10b80 1 -- 172.21.15.33:0/1111502730 shutdown_connections 2024-05-03T18:23:12.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.583+0000 7f736de10b80 1 --2- 172.21.15.33:0/1111502730 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556a05bf4200 0x556a05c127b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:12.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.583+0000 7f736de10b80 1 --2- 172.21.15.33:0/1111502730 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a05bee500 0x556a05bedee0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:12.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.583+0000 7f736de10b80 1 --2- 172.21.15.33:0/1111502730 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556a05bf38e0 0x556a05c12270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:12.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.583+0000 7f736de10b80 1 -- 172.21.15.33:0/1111502730 >> 172.21.15.33:0/1111502730 conn(0x556a05b66680 msgr2=0x556a05bf5b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:23:12.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.583+0000 7f736de10b80 1 -- 172.21.15.33:0/1111502730 shutdown_connections 2024-05-03T18:23:12.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.583+0000 7f736de10b80 1 -- 172.21.15.33:0/1111502730 wait complete. 2024-05-03T18:23:12.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.583+0000 7f736de10b80 1 -- start start 2024-05-03T18:23:12.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.584+0000 7f736de10b80 1 Processor -- start 2024-05-03T18:23:12.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.584+0000 7f736de10b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a05bee500 0x556a05bed490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:23:12.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.584+0000 7f736de10b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556a05bf38e0 0x556a05bed9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:23:12.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.584+0000 7f736de10b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556a05bf4200 0x556a05c10890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:23:12.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.584+0000 7f736de10b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556a05c08c90 con 0x556a05bee500 2024-05-03T18:23:12.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.584+0000 7f736de10b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x556a05c085c0 con 0x556a05bf4200 2024-05-03T18:23:12.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.584+0000 7f736de10b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x556a05c09360 con 0x556a05bf38e0 2024-05-03T18:23:12.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.584+0000 7f736de10b80 10 client.0 _ll_register_callbacks cb 0x556a05ae7c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:23:12.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.585+0000 7f7360a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a05bee500 0x556a05bed490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:23:12.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.585+0000 7f7360a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a05bee500 0x556a05bed490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46138/0 (socket says 172.21.15.33:46138) 2024-05-03T18:23:12.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.585+0000 7f7360a00700 1 -- 172.21.15.33:0/2099690580 learned_addr learned my addr 172.21.15.33:0/2099690580 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:23:12.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.585+0000 7f7353e00700 1 --2- 172.21.15.33:0/2099690580 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556a05bf38e0 0x556a05bed9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:23:12.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.585+0000 7f7361400700 1 --2- 172.21.15.33:0/2099690580 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556a05bf4200 0x556a05c10890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:23:12.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.585+0000 7f7360a00700 1 -- 172.21.15.33:0/2099690580 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556a05bf38e0 msgr2=0x556a05bed9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:23:12.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.585+0000 7f7360a00700 1 --2- 172.21.15.33:0/2099690580 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556a05bf38e0 0x556a05bed9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:12.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.585+0000 7f7360a00700 1 -- 172.21.15.33:0/2099690580 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556a05bf4200 msgr2=0x556a05c10890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:23:12.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.585+0000 7f7360a00700 1 --2- 172.21.15.33:0/2099690580 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556a05bf4200 0x556a05c10890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:12.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.585+0000 7f7360a00700 1 -- 172.21.15.33:0/2099690580 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a05c10dd0 con 0x556a05bee500 2024-05-03T18:23:12.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.585+0000 7f7353e00700 1 --2- 172.21.15.33:0/2099690580 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556a05bf38e0 0x556a05bed9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:23:12.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.585+0000 7f7360a00700 1 --2- 172.21.15.33:0/2099690580 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a05bee500 0x556a05bed490 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f734400bfd0 tx=0x7f73440a9880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=420850b9d7b78c73 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:23:12.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.585+0000 7f734aa00700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:23:12.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.586+0000 7f734aa00700 1 -- 172.21.15.33:0/2099690580 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7344020070 con 0x556a05bee500 2024-05-03T18:23:12.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.586+0000 7f734aa00700 1 -- 172.21.15.33:0/2099690580 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7f7344009d50 con 0x556a05bee500 2024-05-03T18:23:12.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.586+0000 7f736de10b80 1 -- 172.21.15.33:0/2099690580 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73440a8640 con 0x556a05bee500 2024-05-03T18:23:12.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.586+0000 7f734aa00700 1 -- 172.21.15.33:0/2099690580 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f73440a9340 con 0x556a05bee500 2024-05-03T18:23:12.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.586+0000 7f736de10b80 10 client.14682 fetch_fsmap learned FSMap version 17 2024-05-03T18:23:12.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.586+0000 7f736de10b80 1 -- 172.21.15.33:0/2099690580 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x556a05c735b0 con 0x556a05bee500 2024-05-03T18:23:12.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.586+0000 7f734aa00700 1 -- 172.21.15.33:0/2099690580 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f73440a8640 con 0x556a05bee500 2024-05-03T18:23:12.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.586+0000 7f734aa00700 1 -- 172.21.15.33:0/2099690580 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7f73440dfce0 con 0x556a05bee500 2024-05-03T18:23:12.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.586+0000 7f736de10b80 10 client.14682 fetch_fsmap finished waiting for FSMap version 17 2024-05-03T18:23:12.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.587+0000 7f736de10b80 10 client.14682 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-03T18:23:12.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.587+0000 7f736de10b80 20 client.14682 populate_metadata read hostname 'smithi033' 2024-05-03T18:23:12.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.587+0000 7f736de10b80 1 --2- 172.21.15.33:0/2099690580 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x556a05c77d50 0x556a05c76a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:23:12.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.587+0000 7f736de10b80 4 client.14682 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-03T18:23:12.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.587+0000 7f736de10b80 1 -- 172.21.15.33:0/2099690580 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "status"}) -- 0x556a05b4e3b0 con 0x556a05c77d50 2024-05-03T18:23:12.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.587+0000 7f7353e00700 1 --2- 172.21.15.33:0/2099690580 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x556a05c77d50 0x556a05c76a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:23:12.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.588+0000 7f7353e00700 1 --2- 172.21.15.33:0/2099690580 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x556a05c77d50 0x556a05c76a60 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42e3b3b2dac7944e server_cookie=e6c0063476b26781 in_seq=0 out_seq=0 2024-05-03T18:23:12.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.588+0000 7f734aa00700 10 client.14682 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:23:12.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.589+0000 7f734aa00700 1 -- 172.21.15.33:0/2099690580 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x556a05b4e3b0 con 0x556a05c77d50 2024-05-03T18:23:12.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.589+0000 7f734aa00700 10 client.14682 handle_command_reply: tid=0 2024-05-03T18:23:12.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.589+0000 7f736de10b80 1 client.14682 shutdown 2024-05-03T18:23:12.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.589+0000 7f736de10b80 1 -- 172.21.15.33:0/2099690580 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a05bee500 msgr2=0x556a05bed490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:23:12.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.589+0000 7f736de10b80 1 --2- 172.21.15.33:0/2099690580 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a05bee500 0x556a05bed490 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f734400bfd0 tx=0x7f73440a9880 comp rx=0 tx=0).stop 2024-05-03T18:23:12.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.589+0000 7f736de10b80 1 -- 172.21.15.33:0/2099690580 shutdown_connections 2024-05-03T18:23:12.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.589+0000 7f736de10b80 1 --2- 172.21.15.33:0/2099690580 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x556a05c77d50 0x556a05c76a60 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:12.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.589+0000 7f736de10b80 1 --2- 172.21.15.33:0/2099690580 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556a05bf4200 0x556a05c10890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:12.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.589+0000 7f736de10b80 1 --2- 172.21.15.33:0/2099690580 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a05bee500 0x556a05bed490 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:12.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.589+0000 7f736de10b80 1 --2- 172.21.15.33:0/2099690580 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556a05bf38e0 0x556a05bed9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:12.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.589+0000 7f736de10b80 1 -- 172.21.15.33:0/2099690580 >> 172.21.15.33:0/2099690580 conn(0x556a05b66680 msgr2=0x556a05be2ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:23:12.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.589+0000 7f734aa00700 0 client.14682 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:23:12.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.589+0000 7f736de10b80 1 -- 172.21.15.33:0/2099690580 shutdown_connections 2024-05-03T18:23:12.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.589+0000 7f736de10b80 1 -- 172.21.15.33:0/2099690580 wait complete. 2024-05-03T18:23:12.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.589+0000 7f736de10b80 20 client.14682 trim_cache size 0 max 16384 2024-05-03T18:23:12.592 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-03T18:23:12.593 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "5703bba4-0978-11ef-bc97-c7b262605968", 2024-05-03T18:23:12.593 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 0, 2024-05-03T18:23:12.593 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 34508, 2024-05-03T18:23:12.593 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2024-05-03T18:23:12.593 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2024-05-03T18:23:12.593 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2024-05-03T18:23:12.593 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 42.945346884000003, 2024-05-03T18:23:12.593 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 17, 2024-05-03T18:23:12.593 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 85, 2024-05-03T18:23:12.593 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 84, 2024-05-03T18:23:12.593 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 66.502404033999994 2024-05-03T18:23:12.593 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-03T18:23:12.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.590+0000 7f7363400700 1 -- 172.21.15.33:0/3528148258 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f733c0b2900 msgr2=0x7f733c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:23:12.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.590+0000 7f7363400700 1 --2- 172.21.15.33:0/3528148258 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f733c0b2900 0x7f733c0b4dc0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f735400d260 tx=0x7f735400d620 comp rx=0 tx=0).stop 2024-05-03T18:23:12.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.590+0000 7f7363400700 1 -- 172.21.15.33:0/3528148258 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f735c06b180 msgr2=0x7f735c1a1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:23:12.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.590+0000 7f7363400700 1 --2- 172.21.15.33:0/3528148258 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f735c06b180 0x7f735c1a1a50 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f734400d950 tx=0x7f734400dd10 comp rx=0 tx=0).stop 2024-05-03T18:23:12.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.590+0000 7f7363400700 1 -- 172.21.15.33:0/3528148258 shutdown_connections 2024-05-03T18:23:12.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.590+0000 7f7363400700 1 --2- 172.21.15.33:0/3528148258 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f733c0b2900 0x7f733c0b4dc0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:12.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.590+0000 7f7363400700 1 --2- 172.21.15.33:0/3528148258 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f735c109510 0x7f735c19bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:12.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.590+0000 7f7363400700 1 --2- 172.21.15.33:0/3528148258 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f735c06b180 0x7f735c1a1a50 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:12.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.590+0000 7f7363400700 1 --2- 172.21.15.33:0/3528148258 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f735c06bb40 0x7f735c1a1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:12.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.590+0000 7f7363400700 1 -- 172.21.15.33:0/3528148258 >> 172.21.15.33:0/3528148258 conn(0x7f735c079350 msgr2=0x7f735c10e6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:23:12.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.590+0000 7f7363400700 1 -- 172.21.15.33:0/3528148258 shutdown_connections 2024-05-03T18:23:12.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:12.590+0000 7f7363400700 1 -- 172.21.15.33:0/3528148258 wait complete. 2024-05-03T18:23:12.609 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:23:12 smithi033 ceph-mon[28330]: pgmap v401: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:23:12.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:23:12 smithi099 ceph-mon[31658]: pgmap v401: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:23:12.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:23:12 smithi150 ceph-mon[32492]: pgmap v401: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:23:13.575 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-03T18:23:13.576 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-05-03T18:23:13.576 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph fs ls 2024-05-03T18:23:14.041 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:23:14.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:23:14 smithi033 ceph-mon[28330]: pgmap v402: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:23:14.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:23:14 smithi099 ceph-mon[31658]: pgmap v402: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:23:14.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:23:14 smithi150 ceph-mon[32492]: pgmap v402: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:23:16.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:16.215+0000 7ff974000700 1 -- 172.21.15.33:0/2639205858 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff96c103ee0 msgr2=0x7ff96c110370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:23:16.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:16.215+0000 7ff974000700 1 --2- 172.21.15.33:0/2639205858 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff96c103ee0 0x7ff96c110370 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff96400b0a0 tx=0x7ff96400b3b0 comp rx=0 tx=0).stop 2024-05-03T18:23:16.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:16.215+0000 7ff974000700 1 -- 172.21.15.33:0/2639205858 shutdown_connections 2024-05-03T18:23:16.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:16.215+0000 7ff974000700 1 --2- 172.21.15.33:0/2639205858 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff96c103ee0 0x7ff96c110370 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:16.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:16.215+0000 7ff974000700 1 --2- 172.21.15.33:0/2639205858 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff96c109530 0x7ff96c109910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:16.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:16.215+0000 7ff974000700 1 --2- 172.21.15.33:0/2639205858 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff96c103520 0x7ff96c1039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:16.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:16.215+0000 7ff974000700 1 -- 172.21.15.33:0/2639205858 >> 172.21.15.33:0/2639205858 conn(0x7ff96c07a570 msgr2=0x7ff96c07a980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:23:16.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:16.216+0000 7ff974000700 1 -- 172.21.15.33:0/2639205858 shutdown_connections 2024-05-03T18:23:16.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:16.216+0000 7ff974000700 1 -- 172.21.15.33:0/2639205858 wait complete. 2024-05-03T18:23:16.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:16.217+0000 7ff974000700 1 Processor -- start 2024-05-03T18:23:16.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:16.217+0000 7ff974000700 1 -- start start 2024-05-03T18:23:16.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:16.217+0000 7ff974000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff96c103520 0x7ff96c1a1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:23:16.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:16.217+0000 7ff974000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff96c103ee0 0x7ff96c1a1d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:23:16.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:16.217+0000 7ff974000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff96c109530 0x7ff96c19b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:23:16.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:16.217+0000 7ff974000700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff96c113560 con 0x7ff96c103ee0 2024-05-03T18:23:16.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:16.217+0000 7ff974000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff96c1133e0 con 0x7ff96c109530 2024-05-03T18:23:16.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:16.217+0000 7ff974000700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff96c1136e0 con 0x7ff96c103520 2024-05-03T18:23:16.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:16.218+0000 7ff970c00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff96c103ee0 0x7ff96c1a1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:23:16.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:16.218+0000 7ff970c00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff96c103ee0 0x7ff96c1a1d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47338/0 (socket says 172.21.15.33:47338) 2024-05-03T18:23:16.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:16.218+0000 7ff970c00700 1 -- 172.21.15.33:0/251375619 learned_addr learned my addr 172.21.15.33:0/251375619 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:23:16.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:16.218+0000 7ff972000700 1 --2- 172.21.15.33:0/251375619 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff96c109530 0x7ff96c19b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:23:16.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:16.218+0000 7ff971600700 1 --2- 172.21.15.33:0/251375619 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff96c103520 0x7ff96c1a1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:23:16.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:16.218+0000 7ff970c00700 1 -- 172.21.15.33:0/251375619 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff96c103520 msgr2=0x7ff96c1a1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:23:16.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:16.218+0000 7ff970c00700 1 --2- 172.21.15.33:0/251375619 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff96c103520 0x7ff96c1a1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:16.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:16.218+0000 7ff970c00700 1 -- 172.21.15.33:0/251375619 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff96c109530 msgr2=0x7ff96c19b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:23:16.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:16.218+0000 7ff970c00700 1 --2- 172.21.15.33:0/251375619 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff96c109530 0x7ff96c19b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:16.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:16.218+0000 7ff970c00700 1 -- 172.21.15.33:0/251375619 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff96c19c150 con 0x7ff96c103ee0 2024-05-03T18:23:16.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:16.219+0000 7ff970c00700 1 --2- 172.21.15.33:0/251375619 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff96c103ee0 0x7ff96c1a1d70 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff95800c920 tx=0x7ff95800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:23:16.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:16.219+0000 7ff96a000700 1 -- 172.21.15.33:0/251375619 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff958007b00 con 0x7ff96c103ee0 2024-05-03T18:23:16.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:16.219+0000 7ff96a000700 1 -- 172.21.15.33:0/251375619 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff958004cc0 con 0x7ff96c103ee0 2024-05-03T18:23:16.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:16.219+0000 7ff974000700 1 -- 172.21.15.33:0/251375619 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff96c19c3e0 con 0x7ff96c103ee0 2024-05-03T18:23:16.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:16.219+0000 7ff96a000700 1 -- 172.21.15.33:0/251375619 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff958005440 con 0x7ff96c103ee0 2024-05-03T18:23:16.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:16.219+0000 7ff974000700 1 -- 172.21.15.33:0/251375619 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff96c1a8700 con 0x7ff96c103ee0 2024-05-03T18:23:16.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:16.221+0000 7ff974000700 1 -- 172.21.15.33:0/251375619 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff96c06a190 con 0x7ff96c103ee0 2024-05-03T18:23:16.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:16.223+0000 7ff96a000700 1 -- 172.21.15.33:0/251375619 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7ff958004750 con 0x7ff96c103ee0 2024-05-03T18:23:16.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:16.224+0000 7ff96a000700 1 --2- 172.21.15.33:0/251375619 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7ff9540b2880 0x7ff9540b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:23:16.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:16.225+0000 7ff96a000700 1 -- 172.21.15.33:0/251375619 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7ff9580e2430 con 0x7ff96c103ee0 2024-05-03T18:23:16.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:16.225+0000 7ff971600700 1 --2- 172.21.15.33:0/251375619 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7ff9540b2880 0x7ff9540b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:23:16.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:16.226+0000 7ff971600700 1 --2- 172.21.15.33:0/251375619 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7ff9540b2880 0x7ff9540b4d40 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff960005950 tx=0x7ff9600058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:23:16.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:16.228+0000 7ff96a000700 1 -- 172.21.15.33:0/251375619 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7ff958014030 con 0x7ff96c103ee0 2024-05-03T18:23:16.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:16.519+0000 7ff974000700 1 -- 172.21.15.33:0/251375619 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ff96c1a8930 con 0x7ff96c103ee0 2024-05-03T18:23:16.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:16.520+0000 7ff96a000700 1 -- 172.21.15.33:0/251375619 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+88 (secure 0 0 0) 0x7ff9580a9eb0 con 0x7ff96c103ee0 2024-05-03T18:23:16.520 INFO:teuthology.orchestra.run.smithi033.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-03T18:23:16.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:16.523+0000 7ff974000700 1 -- 172.21.15.33:0/251375619 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7ff9540b2880 msgr2=0x7ff9540b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:23:16.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:16.523+0000 7ff974000700 1 --2- 172.21.15.33:0/251375619 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7ff9540b2880 0x7ff9540b4d40 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff960005950 tx=0x7ff9600058e0 comp rx=0 tx=0).stop 2024-05-03T18:23:16.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:16.523+0000 7ff974000700 1 -- 172.21.15.33:0/251375619 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff96c103ee0 msgr2=0x7ff96c1a1d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:23:16.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:16.523+0000 7ff974000700 1 --2- 172.21.15.33:0/251375619 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff96c103ee0 0x7ff96c1a1d70 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff95800c920 tx=0x7ff95800cc30 comp rx=0 tx=0).stop 2024-05-03T18:23:16.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:16.523+0000 7ff974000700 1 -- 172.21.15.33:0/251375619 shutdown_connections 2024-05-03T18:23:16.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:16.524+0000 7ff974000700 1 --2- 172.21.15.33:0/251375619 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7ff9540b2880 0x7ff9540b4d40 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:16.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:16.524+0000 7ff974000700 1 --2- 172.21.15.33:0/251375619 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff96c109530 0x7ff96c19b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:16.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:16.524+0000 7ff974000700 1 --2- 172.21.15.33:0/251375619 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff96c103520 0x7ff96c1a1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:16.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:16.524+0000 7ff974000700 1 --2- 172.21.15.33:0/251375619 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff96c103ee0 0x7ff96c1a1d70 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:16.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:16.524+0000 7ff974000700 1 -- 172.21.15.33:0/251375619 >> 172.21.15.33:0/251375619 conn(0x7ff96c07a570 msgr2=0x7ff96c075960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:23:16.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:16.524+0000 7ff974000700 1 -- 172.21.15.33:0/251375619 shutdown_connections 2024-05-03T18:23:16.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:16.524+0000 7ff974000700 1 -- 172.21.15.33:0/251375619 wait complete. 2024-05-03T18:23:16.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:23:16 smithi099 ceph-mon[31658]: pgmap v403: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:23:16.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:23:16 smithi150 ceph-mon[32492]: pgmap v403: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:23:16.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:23:16 smithi033 ceph-mon[28330]: pgmap v403: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:23:17.512 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-03T18:23:17.513 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-03T18:23:17.513 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi033.front.sepia.ceph.com 2024-05-03T18:23:17.513 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-05-03T18:23:17.513 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-05-03T18:23:17.513 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-03T18:23:17.513 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-03T18:23:17.513 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-03T18:23:17.513 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-05-03T18:23:17.513 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T18:23:17.514 DEBUG:teuthology.orchestra.run.smithi033:> ip addr 2024-05-03T18:23:17.571 INFO:teuthology.orchestra.run.smithi033.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-03T18:23:17.572 INFO:teuthology.orchestra.run.smithi033.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-03T18:23:17.572 INFO:teuthology.orchestra.run.smithi033.stdout: inet 127.0.0.1/8 scope host lo 2024-05-03T18:23:17.572 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2024-05-03T18:23:17.572 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 ::1/128 scope host 2024-05-03T18:23:17.572 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2024-05-03T18:23:17.572 INFO:teuthology.orchestra.run.smithi033.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-03T18:23:17.572 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:14 brd ff:ff:ff:ff:ff:ff 2024-05-03T18:23:17.572 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f0 2024-05-03T18:23:17.572 INFO:teuthology.orchestra.run.smithi033.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-03T18:23:17.572 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:15 brd ff:ff:ff:ff:ff:ff 2024-05-03T18:23:17.572 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f1 2024-05-03T18:23:17.573 INFO:teuthology.orchestra.run.smithi033.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-03T18:23:17.573 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:94 brd ff:ff:ff:ff:ff:ff 2024-05-03T18:23:17.573 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f0 2024-05-03T18:23:17.573 INFO:teuthology.orchestra.run.smithi033.stdout: inet 172.21.15.33/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-05-03T18:23:17.573 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft 41888sec preferred_lft 41888sec 2024-05-03T18:23:17.573 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::ae1f:6bff:fea5:d294/64 scope link noprefixroute 2024-05-03T18:23:17.573 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2024-05-03T18:23:17.573 INFO:teuthology.orchestra.run.smithi033.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-03T18:23:17.573 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:95 brd ff:ff:ff:ff:ff:ff 2024-05-03T18:23:17.573 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f1 2024-05-03T18:23:17.574 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-03T18:23:17.575 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-03T18:23:17.575 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2024-05-03T18:23:17.575 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link add name ceph-brx type bridge 2024-05-03T18:23:17.575 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip addr flush dev ceph-brx 2024-05-03T18:23:17.575 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set ceph-brx up 2024-05-03T18:23:17.575 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-03T18:23:17.575 DEBUG:teuthology.orchestra.run.smithi033:> ') 2024-05-03T18:23:17.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-03T18:23:17.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:23:17 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/251375619' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-03T18:23:17.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:23:17 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/251375619' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-03T18:23:17.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:23:17 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/251375619' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-03T18:23:18.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-03T18:23:18.079 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T18:23:18.079 DEBUG:teuthology.orchestra.run.smithi033:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-03T18:23:18.146 INFO:teuthology.orchestra.run.smithi033.stdout:1 2024-05-03T18:23:18.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T18:23:18.149 DEBUG:teuthology.orchestra.run.smithi033:> route 2024-05-03T18:23:18.208 INFO:teuthology.orchestra.run.smithi033.stdout:Kernel IP routing table 2024-05-03T18:23:18.208 INFO:teuthology.orchestra.run.smithi033.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-03T18:23:18.208 INFO:teuthology.orchestra.run.smithi033.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-03T18:23:18.208 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-03T18:23:18.208 INFO:teuthology.orchestra.run.smithi033.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-03T18:23:18.209 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-03T18:23:18.209 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2024-05-03T18:23:18.209 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-03T18:23:18.209 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-03T18:23:18.209 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-03T18:23:18.209 DEBUG:teuthology.orchestra.run.smithi033:> ') 2024-05-03T18:23:18.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-03T18:23:18.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:23:18 smithi099 ceph-mon[31658]: pgmap v404: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:23:18.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:23:18 smithi150 ceph-mon[32492]: pgmap v404: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:23:18.939 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:23:18 smithi033 ceph-mon[28330]: pgmap v404: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:23:18.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-03T18:23:18.989 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T18:23:18.989 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2024-05-03T18:23:19.030 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T18:23:19.030 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list-id 2024-05-03T18:23:19.110 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-03T18:23:19.110 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2024-05-03T18:23:19.110 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-03T18:23:19.110 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-05-03T18:23:19.111 DEBUG:teuthology.orchestra.run.smithi033:> ') 2024-05-03T18:23:19.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-03T18:23:19.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-03T18:23:19.337 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-05-03T18:23:19.337 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-03T18:23:19.337 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2024-05-03T18:23:19.338 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-05-03T18:23:19.338 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-03T18:23:19.338 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-05-03T18:23:19.338 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-05-03T18:23:19.338 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-05-03T18:23:19.338 DEBUG:teuthology.orchestra.run.smithi033:> ') 2024-05-03T18:23:19.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-03T18:23:19.549 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:23:19 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:23:19.550 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:23:19 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]: dispatch 2024-05-03T18:23:19.550 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:23:19 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]: dispatch 2024-05-03T18:23:19.550 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:23:19 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]}]: dispatch 2024-05-03T18:23:19.550 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:23:19 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-05-03T18:23:19.550 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:23:19 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]: dispatch 2024-05-03T18:23:19.550 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:23:19 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-05-03T18:23:19.550 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:23:19 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11, 5, 8]}]: dispatch 2024-05-03T18:23:19.550 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:23:19 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]: dispatch 2024-05-03T18:23:19.550 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:23:19 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]: dispatch 2024-05-03T18:23:19.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-03T18:23:19.634 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-03T18:23:19.634 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2024-05-03T18:23:19.634 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set brx.0 up 2024-05-03T18:23:19.634 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set dev brx.0 master ceph-brx 2024-05-03T18:23:19.634 DEBUG:teuthology.orchestra.run.smithi033:> ') 2024-05-03T18:23:19.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-03T18:23:19.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:23:19 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:23:19.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:23:19 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]: dispatch 2024-05-03T18:23:19.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:23:19 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]: dispatch 2024-05-03T18:23:19.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:23:19 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]}]: dispatch 2024-05-03T18:23:19.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:23:19 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-05-03T18:23:19.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:23:19 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]: dispatch 2024-05-03T18:23:19.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:23:19 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-05-03T18:23:19.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:23:19 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11, 5, 8]}]: dispatch 2024-05-03T18:23:19.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:23:19 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]: dispatch 2024-05-03T18:23:19.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:23:19 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]: dispatch 2024-05-03T18:23:19.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-03T18:23:19.822 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-05-03T18:23:19.822 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-03T18:23:19.822 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-05-03T18:23:19.860 INFO:teuthology.orchestra.run.smithi033.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-05-03T18:23:19.861 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-03T18:23:19.861 DEBUG:teuthology.orchestra.run.smithi033:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-05-03T18:23:19.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:23:19 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:23:19.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:23:19 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]: dispatch 2024-05-03T18:23:19.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:23:19 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]: dispatch 2024-05-03T18:23:19.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:23:19 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]}]: dispatch 2024-05-03T18:23:19.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:23:19 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-05-03T18:23:19.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:23:19 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]: dispatch 2024-05-03T18:23:19.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:23:19 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-05-03T18:23:19.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:23:19 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11, 5, 8]}]: dispatch 2024-05-03T18:23:19.884 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:23:19 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]: dispatch 2024-05-03T18:23:19.884 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:23:19 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]: dispatch 2024-05-03T18:23:19.943 DEBUG:teuthology.orchestra.run.smithi033:> sudo modprobe fuse 2024-05-03T18:23:20.056 DEBUG:teuthology.orchestra.run.smithi033:> cat /proc/self/mounts | awk '{print $2}' 2024-05-03T18:23:20.106 INFO:teuthology.orchestra.run.smithi033.stdout:/proc 2024-05-03T18:23:20.106 INFO:teuthology.orchestra.run.smithi033.stdout:/sys 2024-05-03T18:23:20.106 INFO:teuthology.orchestra.run.smithi033.stdout:/dev 2024-05-03T18:23:20.106 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/security 2024-05-03T18:23:20.107 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/shm 2024-05-03T18:23:20.107 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/pts 2024-05-03T18:23:20.107 INFO:teuthology.orchestra.run.smithi033.stdout:/run 2024-05-03T18:23:20.107 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/cgroup 2024-05-03T18:23:20.107 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/pstore 2024-05-03T18:23:20.107 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/bpf 2024-05-03T18:23:20.107 INFO:teuthology.orchestra.run.smithi033.stdout:/ 2024-05-03T18:23:20.107 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/selinux 2024-05-03T18:23:20.107 INFO:teuthology.orchestra.run.smithi033.stdout:/proc/sys/fs/binfmt_misc 2024-05-03T18:23:20.107 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/hugepages 2024-05-03T18:23:20.107 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/mqueue 2024-05-03T18:23:20.107 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/debug 2024-05-03T18:23:20.107 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/tracing 2024-05-03T18:23:20.107 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/fuse/connections 2024-05-03T18:23:20.107 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/config 2024-05-03T18:23:20.107 INFO:teuthology.orchestra.run.smithi033.stdout:/run/credentials/systemd-sysctl.service 2024-05-03T18:23:20.108 INFO:teuthology.orchestra.run.smithi033.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-03T18:23:20.108 INFO:teuthology.orchestra.run.smithi033.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-03T18:23:20.108 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/nfs/rpc_pipefs 2024-05-03T18:23:20.108 INFO:teuthology.orchestra.run.smithi033.stdout:/run/user/1000 2024-05-03T18:23:20.108 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/ceph 2024-05-03T18:23:20.108 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay 2024-05-03T18:23:20.108 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/a997dc00d3ffddd941056f480754621e1da9a7c0780c198dce5b7fdfdb5c6518/merged 2024-05-03T18:23:20.108 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/c6be5e3078e9176f50155e446bc1b3af81d7b408fd959a1268e9eb05ab65d47a/merged 2024-05-03T18:23:20.108 INFO:teuthology.orchestra.run.smithi033.stdout:/run/user/0 2024-05-03T18:23:20.108 INFO:teuthology.orchestra.run.smithi033.stdout:/proc/sys/fs/binfmt_misc 2024-05-03T18:23:20.108 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/8f77dd5f9057bf2e6e1a041a8e431aeba47a3737a23202ff7576105d576b222e/merged 2024-05-03T18:23:20.108 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/cb734dec4e7f6b54f637623026860263b8cbfd9afb3f74e8a1581ae03d9a4214/merged 2024-05-03T18:23:20.108 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/4ee9bd3566b1d56373b8c3b03d01a2c876e264155b41da297b0bc3bbcf486161/merged 2024-05-03T18:23:20.108 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/b09ff39c1b8ffc6e847ee4a5a642e5e1e59e10af91359fe6562b05d8728bab04/merged 2024-05-03T18:23:20.108 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/add9b5ad17c6d4c36447f671d7e66e634e26235b5bc07a4d2e48f73dae31d4cc/merged 2024-05-03T18:23:20.109 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/8e1b9f689981c0d088659be1650fd55ef94d3066f00b7066218f53549e16c7a5/merged 2024-05-03T18:23:20.109 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/566ad7a71ceef06ae29a24854e0d1a988dbced8d06582cadd1572fae7758e921/merged 2024-05-03T18:23:20.109 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/52f47aa2e49f2bbd5b8cdff45563f7aff478c10e86a30fe10a1ee15611d1e9de/merged 2024-05-03T18:23:20.109 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns 2024-05-03T18:23:20.109 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-03T18:23:20.109 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-03T18:23:20.110 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T18:23:20.110 DEBUG:teuthology.orchestra.run.smithi033:> ls /sys/fs/fuse/connections 2024-05-03T18:23:20.156 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-03T18:23:20.156 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-05-03T18:23:20.201 DEBUG:teuthology.orchestra.run.smithi033:> sudo modprobe fuse 2024-05-03T18:23:20.268 DEBUG:teuthology.orchestra.run.smithi033:> cat /proc/self/mounts | awk '{print $2}' 2024-05-03T18:23:20.307 INFO:teuthology.orchestra.run.smithi033.stdout:/proc 2024-05-03T18:23:20.307 INFO:teuthology.orchestra.run.smithi033.stdout:/sys 2024-05-03T18:23:20.307 INFO:teuthology.orchestra.run.smithi033.stdout:/dev 2024-05-03T18:23:20.307 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/security 2024-05-03T18:23:20.308 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/shm 2024-05-03T18:23:20.308 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/pts 2024-05-03T18:23:20.308 INFO:teuthology.orchestra.run.smithi033.stdout:/run 2024-05-03T18:23:20.308 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/cgroup 2024-05-03T18:23:20.308 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/pstore 2024-05-03T18:23:20.308 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/bpf 2024-05-03T18:23:20.308 INFO:teuthology.orchestra.run.smithi033.stdout:/ 2024-05-03T18:23:20.308 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/selinux 2024-05-03T18:23:20.308 INFO:teuthology.orchestra.run.smithi033.stdout:/proc/sys/fs/binfmt_misc 2024-05-03T18:23:20.308 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/hugepages 2024-05-03T18:23:20.308 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/mqueue 2024-05-03T18:23:20.308 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/debug 2024-05-03T18:23:20.308 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/tracing 2024-05-03T18:23:20.308 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/fuse/connections 2024-05-03T18:23:20.308 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/config 2024-05-03T18:23:20.308 INFO:teuthology.orchestra.run.smithi033.stdout:/run/credentials/systemd-sysctl.service 2024-05-03T18:23:20.309 INFO:teuthology.orchestra.run.smithi033.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-03T18:23:20.309 INFO:teuthology.orchestra.run.smithi033.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-03T18:23:20.309 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/nfs/rpc_pipefs 2024-05-03T18:23:20.309 INFO:teuthology.orchestra.run.smithi033.stdout:/run/user/1000 2024-05-03T18:23:20.309 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/ceph 2024-05-03T18:23:20.309 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay 2024-05-03T18:23:20.309 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/a997dc00d3ffddd941056f480754621e1da9a7c0780c198dce5b7fdfdb5c6518/merged 2024-05-03T18:23:20.309 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/c6be5e3078e9176f50155e446bc1b3af81d7b408fd959a1268e9eb05ab65d47a/merged 2024-05-03T18:23:20.309 INFO:teuthology.orchestra.run.smithi033.stdout:/run/user/0 2024-05-03T18:23:20.309 INFO:teuthology.orchestra.run.smithi033.stdout:/proc/sys/fs/binfmt_misc 2024-05-03T18:23:20.309 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/8f77dd5f9057bf2e6e1a041a8e431aeba47a3737a23202ff7576105d576b222e/merged 2024-05-03T18:23:20.309 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/cb734dec4e7f6b54f637623026860263b8cbfd9afb3f74e8a1581ae03d9a4214/merged 2024-05-03T18:23:20.309 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/4ee9bd3566b1d56373b8c3b03d01a2c876e264155b41da297b0bc3bbcf486161/merged 2024-05-03T18:23:20.309 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/b09ff39c1b8ffc6e847ee4a5a642e5e1e59e10af91359fe6562b05d8728bab04/merged 2024-05-03T18:23:20.309 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/add9b5ad17c6d4c36447f671d7e66e634e26235b5bc07a4d2e48f73dae31d4cc/merged 2024-05-03T18:23:20.309 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/8e1b9f689981c0d088659be1650fd55ef94d3066f00b7066218f53549e16c7a5/merged 2024-05-03T18:23:20.310 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/566ad7a71ceef06ae29a24854e0d1a988dbced8d06582cadd1572fae7758e921/merged 2024-05-03T18:23:20.310 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/52f47aa2e49f2bbd5b8cdff45563f7aff478c10e86a30fe10a1ee15611d1e9de/merged 2024-05-03T18:23:20.310 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns 2024-05-03T18:23:20.310 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-03T18:23:20.310 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-03T18:23:20.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T18:23:20.311 DEBUG:teuthology.orchestra.run.smithi033:> ls /sys/fs/fuse/connections 2024-05-03T18:23:20.459 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi033.stderr:2024-05-03T18:23:20.460+0000 7f5f08302480 -1 init, newargv = 0x55b318e96350 newargc=15 2024-05-03T18:23:20.460 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi033.stderr:ceph-fuse[75044]: starting ceph client 2024-05-03T18:23:20.471 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi033.stderr:ceph-fuse[75044]: starting fuse 2024-05-03T18:23:20.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:23:20 smithi099 ceph-mon[31658]: pgmap v405: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:23:20.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:23:20 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]': finished 2024-05-03T18:23:20.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:23:20 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]': finished 2024-05-03T18:23:20.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:23:20 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]}]': finished 2024-05-03T18:23:20.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:23:20 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-05-03T18:23:20.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:23:20 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]': finished 2024-05-03T18:23:20.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:23:20 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-05-03T18:23:20.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:23:20 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11, 5, 8]}]': finished 2024-05-03T18:23:20.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:23:20 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]': finished 2024-05-03T18:23:20.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:23:20 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]': finished 2024-05-03T18:23:20.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:23:20 smithi099 ceph-mon[31658]: osdmap e86: 12 total, 12 up, 12 in 2024-05-03T18:23:20.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:23:20 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:23:20.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:23:20 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:23:20.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:23:20 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:23:20.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:23:20 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:23:20.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:23:20 smithi150 ceph-mon[32492]: pgmap v405: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:23:20.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:23:20 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]': finished 2024-05-03T18:23:20.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:23:20 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]': finished 2024-05-03T18:23:20.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:23:20 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]}]': finished 2024-05-03T18:23:20.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:23:20 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-05-03T18:23:20.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:23:20 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]': finished 2024-05-03T18:23:20.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:23:20 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-05-03T18:23:20.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:23:20 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11, 5, 8]}]': finished 2024-05-03T18:23:20.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:23:20 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]': finished 2024-05-03T18:23:20.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:23:20 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]': finished 2024-05-03T18:23:20.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:23:20 smithi150 ceph-mon[32492]: osdmap e86: 12 total, 12 up, 12 in 2024-05-03T18:23:20.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:23:20 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:23:20.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:23:20 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:23:20.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:23:20 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:23:20.884 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:23:20 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:23:20.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:23:20 smithi033 ceph-mon[28330]: pgmap v405: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:23:20.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:23:20 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]': finished 2024-05-03T18:23:20.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:23:20 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]': finished 2024-05-03T18:23:20.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:23:20 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]}]': finished 2024-05-03T18:23:20.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:23:20 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-05-03T18:23:20.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:23:20 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]': finished 2024-05-03T18:23:20.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:23:20 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-05-03T18:23:20.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:23:20 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11, 5, 8]}]': finished 2024-05-03T18:23:20.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:23:20 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]': finished 2024-05-03T18:23:20.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:23:20 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]': finished 2024-05-03T18:23:20.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:23:20 smithi033 ceph-mon[28330]: osdmap e86: 12 total, 12 up, 12 in 2024-05-03T18:23:20.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:23:20 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:23:20.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:23:20 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:23:20.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:23:20 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:23:20.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:23:20 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:23:21.507 DEBUG:teuthology.orchestra.run.smithi033:> sudo modprobe fuse 2024-05-03T18:23:21.590 DEBUG:teuthology.orchestra.run.smithi033:> cat /proc/self/mounts | awk '{print $2}' 2024-05-03T18:23:21.633 INFO:teuthology.orchestra.run.smithi033.stdout:/proc 2024-05-03T18:23:21.633 INFO:teuthology.orchestra.run.smithi033.stdout:/sys 2024-05-03T18:23:21.633 INFO:teuthology.orchestra.run.smithi033.stdout:/dev 2024-05-03T18:23:21.633 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/security 2024-05-03T18:23:21.633 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/shm 2024-05-03T18:23:21.633 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/pts 2024-05-03T18:23:21.634 INFO:teuthology.orchestra.run.smithi033.stdout:/run 2024-05-03T18:23:21.634 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/cgroup 2024-05-03T18:23:21.634 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/pstore 2024-05-03T18:23:21.634 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/bpf 2024-05-03T18:23:21.634 INFO:teuthology.orchestra.run.smithi033.stdout:/ 2024-05-03T18:23:21.634 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/selinux 2024-05-03T18:23:21.634 INFO:teuthology.orchestra.run.smithi033.stdout:/proc/sys/fs/binfmt_misc 2024-05-03T18:23:21.634 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/hugepages 2024-05-03T18:23:21.634 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/mqueue 2024-05-03T18:23:21.634 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/debug 2024-05-03T18:23:21.634 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/tracing 2024-05-03T18:23:21.634 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/fuse/connections 2024-05-03T18:23:21.634 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/config 2024-05-03T18:23:21.634 INFO:teuthology.orchestra.run.smithi033.stdout:/run/credentials/systemd-sysctl.service 2024-05-03T18:23:21.634 INFO:teuthology.orchestra.run.smithi033.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-03T18:23:21.635 INFO:teuthology.orchestra.run.smithi033.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-03T18:23:21.635 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/nfs/rpc_pipefs 2024-05-03T18:23:21.635 INFO:teuthology.orchestra.run.smithi033.stdout:/run/user/1000 2024-05-03T18:23:21.635 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/ceph 2024-05-03T18:23:21.635 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay 2024-05-03T18:23:21.635 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/a997dc00d3ffddd941056f480754621e1da9a7c0780c198dce5b7fdfdb5c6518/merged 2024-05-03T18:23:21.635 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/c6be5e3078e9176f50155e446bc1b3af81d7b408fd959a1268e9eb05ab65d47a/merged 2024-05-03T18:23:21.635 INFO:teuthology.orchestra.run.smithi033.stdout:/run/user/0 2024-05-03T18:23:21.635 INFO:teuthology.orchestra.run.smithi033.stdout:/proc/sys/fs/binfmt_misc 2024-05-03T18:23:21.635 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/8f77dd5f9057bf2e6e1a041a8e431aeba47a3737a23202ff7576105d576b222e/merged 2024-05-03T18:23:21.635 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/cb734dec4e7f6b54f637623026860263b8cbfd9afb3f74e8a1581ae03d9a4214/merged 2024-05-03T18:23:21.635 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/4ee9bd3566b1d56373b8c3b03d01a2c876e264155b41da297b0bc3bbcf486161/merged 2024-05-03T18:23:21.635 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/b09ff39c1b8ffc6e847ee4a5a642e5e1e59e10af91359fe6562b05d8728bab04/merged 2024-05-03T18:23:21.635 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/add9b5ad17c6d4c36447f671d7e66e634e26235b5bc07a4d2e48f73dae31d4cc/merged 2024-05-03T18:23:21.636 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/8e1b9f689981c0d088659be1650fd55ef94d3066f00b7066218f53549e16c7a5/merged 2024-05-03T18:23:21.636 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/566ad7a71ceef06ae29a24854e0d1a988dbced8d06582cadd1572fae7758e921/merged 2024-05-03T18:23:21.636 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/52f47aa2e49f2bbd5b8cdff45563f7aff478c10e86a30fe10a1ee15611d1e9de/merged 2024-05-03T18:23:21.636 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns 2024-05-03T18:23:21.636 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-03T18:23:21.636 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-03T18:23:21.636 INFO:teuthology.orchestra.run.smithi033.stdout:/home/ubuntu/cephtest/mnt.admin 2024-05-03T18:23:21.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T18:23:21.637 DEBUG:teuthology.orchestra.run.smithi033:> ls /sys/fs/fuse/connections 2024-05-03T18:23:21.724 INFO:teuthology.orchestra.run.smithi033.stdout:114 2024-05-03T18:23:21.724 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [114] 2024-05-03T18:23:21.724 DEBUG:teuthology.orchestra.run.smithi033:> sudo stdin-killer -- python3 -c ' 2024-05-03T18:23:21.724 DEBUG:teuthology.orchestra.run.smithi033:> import glob 2024-05-03T18:23:21.724 DEBUG:teuthology.orchestra.run.smithi033:> import re 2024-05-03T18:23:21.724 DEBUG:teuthology.orchestra.run.smithi033:> import os 2024-05-03T18:23:21.724 DEBUG:teuthology.orchestra.run.smithi033:> import subprocess 2024-05-03T18:23:21.724 DEBUG:teuthology.orchestra.run.smithi033:> 2024-05-03T18:23:21.725 DEBUG:teuthology.orchestra.run.smithi033:> def _find_admin_socket(client_name): 2024-05-03T18:23:21.725 DEBUG:teuthology.orchestra.run.smithi033:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-05-03T18:23:21.725 DEBUG:teuthology.orchestra.run.smithi033:> files = glob.glob(asok_path) 2024-05-03T18:23:21.725 DEBUG:teuthology.orchestra.run.smithi033:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-05-03T18:23:21.725 DEBUG:teuthology.orchestra.run.smithi033:> 2024-05-03T18:23:21.725 DEBUG:teuthology.orchestra.run.smithi033:> # Given a non-glob path, it better be there 2024-05-03T18:23:21.725 DEBUG:teuthology.orchestra.run.smithi033:> if "*" not in asok_path: 2024-05-03T18:23:21.725 DEBUG:teuthology.orchestra.run.smithi033:> assert(len(files) == 1) 2024-05-03T18:23:21.725 DEBUG:teuthology.orchestra.run.smithi033:> return files[0] 2024-05-03T18:23:21.725 DEBUG:teuthology.orchestra.run.smithi033:> 2024-05-03T18:23:21.725 DEBUG:teuthology.orchestra.run.smithi033:> for f in files: 2024-05-03T18:23:21.725 DEBUG:teuthology.orchestra.run.smithi033:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-03T18:23:21.725 DEBUG:teuthology.orchestra.run.smithi033:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-03T18:23:21.725 DEBUG:teuthology.orchestra.run.smithi033:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-03T18:23:21.725 DEBUG:teuthology.orchestra.run.smithi033:> contents = proc_f.read() 2024-05-03T18:23:21.726 DEBUG:teuthology.orchestra.run.smithi033:> if mountpoint in contents: 2024-05-03T18:23:21.726 DEBUG:teuthology.orchestra.run.smithi033:> return f 2024-05-03T18:23:21.726 DEBUG:teuthology.orchestra.run.smithi033:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-03T18:23:21.726 DEBUG:teuthology.orchestra.run.smithi033:> 2024-05-03T18:23:21.726 DEBUG:teuthology.orchestra.run.smithi033:> print(_find_admin_socket("client.admin")) 2024-05-03T18:23:21.726 DEBUG:teuthology.orchestra.run.smithi033:> ' 2024-05-03T18:23:21.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:23:21 smithi099 ceph-mon[31658]: osdmap e87: 12 total, 12 up, 12 in 2024-05-03T18:23:21.826 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:23:21 smithi033 ceph-mon[28330]: osdmap e87: 12 total, 12 up, 12 in 2024-05-03T18:23:21.881 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:23:21 smithi150 ceph-mon[32492]: osdmap e87: 12 total, 12 up, 12 in 2024-05-03T18:23:21.924 INFO:teuthology.orchestra.run.smithi033.stdout:/var/run/ceph/ceph-client.admin.75044.asok 2024-05-03T18:23:21.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-03T18:23:21.943 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.75044.asok 2024-05-03T18:23:21.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T18:23:21.943 DEBUG:teuthology.orchestra.run.smithi033:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.75044.asok status 2024-05-03T18:23:22.164 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-03T18:23:22.164 INFO:teuthology.orchestra.run.smithi033.stdout: "metadata": { 2024-05-03T18:23:22.164 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_sha1": "9a047fe7599a549042d06a3fb5b6813a203eabea", 2024-05-03T18:23:22.164 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_version": "ceph version 19.0.0-3450-g9a047fe7 (9a047fe7599a549042d06a3fb5b6813a203eabea) squid (dev)", 2024-05-03T18:23:22.164 INFO:teuthology.orchestra.run.smithi033.stdout: "entity_id": "admin", 2024-05-03T18:23:22.164 INFO:teuthology.orchestra.run.smithi033.stdout: "hostname": "smithi033", 2024-05-03T18:23:22.164 INFO:teuthology.orchestra.run.smithi033.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-05-03T18:23:22.164 INFO:teuthology.orchestra.run.smithi033.stdout: "pid": "75044", 2024-05-03T18:23:22.164 INFO:teuthology.orchestra.run.smithi033.stdout: "root": "/" 2024-05-03T18:23:22.165 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:23:22.165 INFO:teuthology.orchestra.run.smithi033.stdout: "dentry_count": 0, 2024-05-03T18:23:22.165 INFO:teuthology.orchestra.run.smithi033.stdout: "dentry_pinned_count": 0, 2024-05-03T18:23:22.165 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 14694, 2024-05-03T18:23:22.165 INFO:teuthology.orchestra.run.smithi033.stdout: "inst": { 2024-05-03T18:23:22.165 INFO:teuthology.orchestra.run.smithi033.stdout: "name": { 2024-05-03T18:23:22.165 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "client", 2024-05-03T18:23:22.165 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 14694 2024-05-03T18:23:22.165 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:23:22.165 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": { 2024-05-03T18:23:22.165 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-03T18:23:22.165 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "192.168.0.1:0", 2024-05-03T18:23:22.165 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 2768837271 2024-05-03T18:23:22.165 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:23:22.166 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:23:22.166 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": { 2024-05-03T18:23:22.166 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-03T18:23:22.166 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "192.168.0.1:0", 2024-05-03T18:23:22.166 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 2768837271 2024-05-03T18:23:22.166 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:23:22.166 INFO:teuthology.orchestra.run.smithi033.stdout: "inst_str": "client.14694 192.168.0.1:0/2768837271", 2024-05-03T18:23:22.166 INFO:teuthology.orchestra.run.smithi033.stdout: "addr_str": "192.168.0.1:0/2768837271", 2024-05-03T18:23:22.166 INFO:teuthology.orchestra.run.smithi033.stdout: "inode_count": 1, 2024-05-03T18:23:22.166 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_epoch": 17, 2024-05-03T18:23:22.166 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_epoch": 86, 2024-05-03T18:23:22.166 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_epoch_barrier": 0, 2024-05-03T18:23:22.166 INFO:teuthology.orchestra.run.smithi033.stdout: "blocklisted": false, 2024-05-03T18:23:22.166 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs" 2024-05-03T18:23:22.167 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-03T18:23:22.186 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T18:23:22.187 DEBUG:teuthology.orchestra.run.smithi033:> stat --file-system '--printf=%T 2024-05-03T18:23:22.187 DEBUG:teuthology.orchestra.run.smithi033:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-05-03T18:23:22.240 INFO:teuthology.orchestra.run.smithi033.stdout:fuseblk 2024-05-03T18:23:22.241 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-05-03T18:23:22.241 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T18:23:22.241 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-05-03T18:23:22.318 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-05-03T18:23:22.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:22 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-03T18:23:22.462 INFO:teuthology.orchestra.run.smithi033.stdout:# file: . 2024-05-03T18:23:22.462 INFO:teuthology.orchestra.run.smithi033.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-05-03T18:23:22.462 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:23:22.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-03T18:23:22.473 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-03T18:23:22.473 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-03T18:23:22.525 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph osd blocklist ls 2024-05-03T18:23:22.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:23:22 smithi099 ceph-mon[31658]: pgmap v408: 129 pgs: 3 peering, 126 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:23:22.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:23:22 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:23:22.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:23:22 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:23:22.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:23:22 smithi150 ceph-mon[32492]: pgmap v408: 129 pgs: 3 peering, 126 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:23:22.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:23:22 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:23:22.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:23:22 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:23:22.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:23:22 smithi033 ceph-mon[28330]: pgmap v408: 129 pgs: 3 peering, 126 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:23:22.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:23:22 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:23:22.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:23:22 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:23:23.011 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:23:23.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:23:23 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:23:23.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:23:23 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:23:23.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:23:23 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:23:23.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:23:23 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:23:23.987 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:23:23 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:23:23.988 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:23:23 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:23:23.988 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:23:23 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:23:23.988 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:23:23 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:23:24.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:23:23 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:23:24.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:23:23 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:23:24.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:23:23 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:23:24.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:23:23 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:23:24.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:23:24 smithi033 ceph-mon[28330]: pgmap v409: 129 pgs: 3 peering, 126 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-05-03T18:23:24.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:23:24 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:23:24.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:23:24 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:23:24.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:23:24 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:23:25.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:23:24 smithi099 ceph-mon[31658]: pgmap v409: 129 pgs: 3 peering, 126 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-05-03T18:23:25.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:23:24 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:23:25.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:23:24 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:23:25.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:23:24 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:23:25.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:23:24 smithi150 ceph-mon[32492]: pgmap v409: 129 pgs: 3 peering, 126 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-05-03T18:23:25.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:23:24 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:23:25.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:23:24 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:23:25.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:23:24 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:23:25.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:25.292+0000 7fe187600700 1 -- 172.21.15.33:0/429324635 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe180075ff0 msgr2=0x7fe18010f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:23:25.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:25.292+0000 7fe187600700 1 --2- 172.21.15.33:0/429324635 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe180075ff0 0x7fe18010f960 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe168009990 tx=0x7fe168009ca0 comp rx=0 tx=0).stop 2024-05-03T18:23:25.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:25.292+0000 7fe187600700 1 -- 172.21.15.33:0/429324635 shutdown_connections 2024-05-03T18:23:25.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:25.292+0000 7fe187600700 1 --2- 172.21.15.33:0/429324635 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe18010fea0 0x7fe180112290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:25.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:25.292+0000 7fe187600700 1 --2- 172.21.15.33:0/429324635 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe1800756d0 0x7fe180075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:25.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:25.292+0000 7fe187600700 1 --2- 172.21.15.33:0/429324635 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe180075ff0 0x7fe18010f960 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:25.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:25.292+0000 7fe187600700 1 -- 172.21.15.33:0/429324635 >> 172.21.15.33:0/429324635 conn(0x7fe18007a600 msgr2=0x7fe18007aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:23:25.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:25.292+0000 7fe187600700 1 -- 172.21.15.33:0/429324635 shutdown_connections 2024-05-03T18:23:25.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:25.292+0000 7fe187600700 1 -- 172.21.15.33:0/429324635 wait complete. 2024-05-03T18:23:25.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:25.293+0000 7fe187600700 1 Processor -- start 2024-05-03T18:23:25.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:25.294+0000 7fe187600700 1 -- start start 2024-05-03T18:23:25.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:25.294+0000 7fe187600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe1800756d0 0x7fe1801a1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:23:25.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:25.294+0000 7fe187600700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe180075ff0 0x7fe1801a1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:23:25.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:25.294+0000 7fe187600700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe18010fea0 0x7fe18019bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:23:25.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:25.294+0000 7fe187600700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe180115780 con 0x7fe180075ff0 2024-05-03T18:23:25.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:25.294+0000 7fe187600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe180115600 con 0x7fe1800756d0 2024-05-03T18:23:25.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:25.294+0000 7fe187600700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe180115900 con 0x7fe18010fea0 2024-05-03T18:23:25.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:25.295+0000 7fe177e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe180075ff0 0x7fe1801a1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:23:25.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:25.295+0000 7fe177e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe180075ff0 0x7fe1801a1e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47366/0 (socket says 172.21.15.33:47366) 2024-05-03T18:23:25.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:25.295+0000 7fe177e00700 1 -- 172.21.15.33:0/3434657039 learned_addr learned my addr 172.21.15.33:0/3434657039 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:23:25.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:25.295+0000 7fe185600700 1 --2- 172.21.15.33:0/3434657039 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe18010fea0 0x7fe18019bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:23:25.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:25.295+0000 7fe184c00700 1 --2- 172.21.15.33:0/3434657039 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe1800756d0 0x7fe1801a1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:23:25.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:25.295+0000 7fe177e00700 1 -- 172.21.15.33:0/3434657039 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe18010fea0 msgr2=0x7fe18019bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:23:25.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:25.295+0000 7fe177e00700 1 --2- 172.21.15.33:0/3434657039 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe18010fea0 0x7fe18019bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:25.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:25.295+0000 7fe177e00700 1 -- 172.21.15.33:0/3434657039 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe1800756d0 msgr2=0x7fe1801a1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:23:25.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:25.295+0000 7fe177e00700 1 --2- 172.21.15.33:0/3434657039 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe1800756d0 0x7fe1801a1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:25.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:25.295+0000 7fe177e00700 1 -- 172.21.15.33:0/3434657039 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe18019c2a0 con 0x7fe180075ff0 2024-05-03T18:23:25.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:25.296+0000 7fe177e00700 1 --2- 172.21.15.33:0/3434657039 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe180075ff0 0x7fe1801a1e70 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe168000c00 tx=0x7fe1680057a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:23:25.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:25.296+0000 7fe17e000700 1 -- 172.21.15.33:0/3434657039 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe168027070 con 0x7fe180075ff0 2024-05-03T18:23:25.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:25.296+0000 7fe17e000700 1 -- 172.21.15.33:0/3434657039 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe16800fb00 con 0x7fe180075ff0 2024-05-03T18:23:25.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:25.296+0000 7fe187600700 1 -- 172.21.15.33:0/3434657039 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe18019c530 con 0x7fe180075ff0 2024-05-03T18:23:25.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:25.296+0000 7fe17e000700 1 -- 172.21.15.33:0/3434657039 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe168022a00 con 0x7fe180075ff0 2024-05-03T18:23:25.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:25.296+0000 7fe187600700 1 -- 172.21.15.33:0/3434657039 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1801a8800 con 0x7fe180075ff0 2024-05-03T18:23:25.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:25.298+0000 7fe187600700 1 -- 172.21.15.33:0/3434657039 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe18006a190 con 0x7fe180075ff0 2024-05-03T18:23:25.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:25.300+0000 7fe17e000700 1 -- 172.21.15.33:0/3434657039 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fe16802a070 con 0x7fe180075ff0 2024-05-03T18:23:25.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:25.303+0000 7fe17e000700 1 --2- 172.21.15.33:0/3434657039 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fe1640b2880 0x7fe1640b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:23:25.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:25.303+0000 7fe17e000700 1 -- 172.21.15.33:0/3434657039 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9071+0+0 (secure 0 0 0) 0x7fe1680e3100 con 0x7fe180075ff0 2024-05-03T18:23:25.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:25.303+0000 7fe184c00700 1 --2- 172.21.15.33:0/3434657039 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fe1640b2880 0x7fe1640b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:23:25.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:25.304+0000 7fe184c00700 1 --2- 172.21.15.33:0/3434657039 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fe1640b2880 0x7fe1640b4d40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe170005950 tx=0x7fe17000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:23:25.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:25.307+0000 7fe17e000700 1 -- 172.21.15.33:0/3434657039 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fe16805fd70 con 0x7fe180075ff0 2024-05-03T18:23:25.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:25.565+0000 7fe187600700 1 -- 172.21.15.33:0/3434657039 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe18002e510 con 0x7fe180075ff0 2024-05-03T18:23:25.567 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/3991947997 2024-05-04T18:12:19.127652+0000 2024-05-03T18:23:25.567 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/965859483 2024-05-04T18:12:19.127652+0000 2024-05-03T18:23:25.567 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/2014902144 2024-05-04T18:11:44.239835+0000 2024-05-03T18:23:25.567 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/1314704208 2024-05-04T18:11:44.239835+0000 2024-05-03T18:23:25.567 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/1614640996 2024-05-04T18:11:44.239835+0000 2024-05-03T18:23:25.567 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6800/4007322071 2024-05-04T18:11:44.239835+0000 2024-05-03T18:23:25.567 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6801/2830135294 2024-05-04T18:12:19.127652+0000 2024-05-03T18:23:25.567 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6800/2830135294 2024-05-04T18:12:19.127652+0000 2024-05-03T18:23:25.567 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/3767665033 2024-05-04T18:12:19.127652+0000 2024-05-03T18:23:25.568 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6801/4007322071 2024-05-04T18:11:44.239835+0000 2024-05-03T18:23:25.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:25.566+0000 7fe17e000700 1 -- 172.21.15.33:0/3434657039 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v87) ==== 81+0+591 (secure 0 0 0) 0x7fe168013070 con 0x7fe180075ff0 2024-05-03T18:23:25.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:25.570+0000 7fe187600700 1 -- 172.21.15.33:0/3434657039 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fe1640b2880 msgr2=0x7fe1640b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:23:25.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:25.570+0000 7fe187600700 1 --2- 172.21.15.33:0/3434657039 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fe1640b2880 0x7fe1640b4d40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe170005950 tx=0x7fe17000b410 comp rx=0 tx=0).stop 2024-05-03T18:23:25.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:25.570+0000 7fe187600700 1 -- 172.21.15.33:0/3434657039 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe180075ff0 msgr2=0x7fe1801a1e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:23:25.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:25.570+0000 7fe187600700 1 --2- 172.21.15.33:0/3434657039 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe180075ff0 0x7fe1801a1e70 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe168000c00 tx=0x7fe1680057a0 comp rx=0 tx=0).stop 2024-05-03T18:23:25.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:25.571+0000 7fe187600700 1 -- 172.21.15.33:0/3434657039 shutdown_connections 2024-05-03T18:23:25.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:25.571+0000 7fe187600700 1 --2- 172.21.15.33:0/3434657039 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe18010fea0 0x7fe18019bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:25.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:25.571+0000 7fe187600700 1 --2- 172.21.15.33:0/3434657039 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fe1640b2880 0x7fe1640b4d40 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:25.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:25.571+0000 7fe187600700 1 --2- 172.21.15.33:0/3434657039 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe1800756d0 0x7fe1801a1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:25.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:25.571+0000 7fe187600700 1 --2- 172.21.15.33:0/3434657039 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe180075ff0 0x7fe1801a1e70 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:25.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:25.571+0000 7fe187600700 1 -- 172.21.15.33:0/3434657039 >> 172.21.15.33:0/3434657039 conn(0x7fe18007a600 msgr2=0x7fe180100750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:23:25.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:25.571+0000 7fe187600700 1 -- 172.21.15.33:0/3434657039 shutdown_connections 2024-05-03T18:23:25.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:25.571+0000 7fe187600700 1 -- 172.21.15.33:0/3434657039 wait complete. 2024-05-03T18:23:25.572 INFO:teuthology.orchestra.run.smithi033.stderr:listed 10 entries 2024-05-03T18:23:25.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:23:25 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/3434657039' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-03T18:23:26.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:23:25 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/3434657039' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-03T18:23:26.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:23:25 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/3434657039' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-03T18:23:26.390 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-03T18:23:26.390 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-03T18:23:26.447 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph osd blocklist ls 2024-05-03T18:23:26.908 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:23:26.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:23:26 smithi033 ceph-mon[28330]: pgmap v410: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-05-03T18:23:27.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:23:26 smithi099 ceph-mon[31658]: pgmap v410: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-05-03T18:23:27.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:23:26 smithi150 ceph-mon[32492]: pgmap v410: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-05-03T18:23:28.948 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:23:28 smithi033 ceph-mon[28330]: pgmap v411: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-05-03T18:23:29.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:23:28 smithi099 ceph-mon[31658]: pgmap v411: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-05-03T18:23:29.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:23:28 smithi150 ceph-mon[32492]: pgmap v411: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-05-03T18:23:29.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:29.303+0000 7fa07ca00700 1 -- 172.21.15.33:0/4099778984 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa078102ee0 msgr2=0x7fa078106f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:23:29.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:29.303+0000 7fa07ca00700 1 --2- 172.21.15.33:0/4099778984 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa078102ee0 0x7fa078106f30 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa0640099c0 tx=0x7fa064009cd0 comp rx=0 tx=0).stop 2024-05-03T18:23:29.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:29.303+0000 7fa07ca00700 1 -- 172.21.15.33:0/4099778984 shutdown_connections 2024-05-03T18:23:29.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:29.303+0000 7fa07ca00700 1 --2- 172.21.15.33:0/4099778984 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa078107610 0x7fa07810f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:29.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:29.303+0000 7fa07ca00700 1 --2- 172.21.15.33:0/4099778984 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa078102530 0x7fa078102910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:29.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:29.303+0000 7fa07ca00700 1 --2- 172.21.15.33:0/4099778984 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa078102ee0 0x7fa078106f30 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:29.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:29.303+0000 7fa07ca00700 1 -- 172.21.15.33:0/4099778984 >> 172.21.15.33:0/4099778984 conn(0x7fa0780fddc0 msgr2=0x7fa0781001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:23:29.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:29.303+0000 7fa07ca00700 1 -- 172.21.15.33:0/4099778984 shutdown_connections 2024-05-03T18:23:29.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:29.303+0000 7fa07ca00700 1 -- 172.21.15.33:0/4099778984 wait complete. 2024-05-03T18:23:29.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:29.304+0000 7fa07ca00700 1 Processor -- start 2024-05-03T18:23:29.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:29.304+0000 7fa07ca00700 1 -- start start 2024-05-03T18:23:29.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:29.305+0000 7fa07ca00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa078102530 0x7fa078103fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:23:29.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:29.305+0000 7fa07ca00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa078102ee0 0x7fa078104510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:23:29.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:29.305+0000 7fa07ca00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa078107610 0x7fa078104a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:23:29.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:29.305+0000 7fa07ca00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa078112380 con 0x7fa078107610 2024-05-03T18:23:29.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:29.305+0000 7fa07ca00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa078112200 con 0x7fa078102ee0 2024-05-03T18:23:29.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:29.305+0000 7fa07ca00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa078112500 con 0x7fa078102530 2024-05-03T18:23:29.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:29.305+0000 7fa077e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa078107610 0x7fa078104a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:23:29.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:29.305+0000 7fa077e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa078107610 0x7fa078104a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57564/0 (socket says 172.21.15.33:57564) 2024-05-03T18:23:29.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:29.305+0000 7fa077e00700 1 -- 172.21.15.33:0/2386551602 learned_addr learned my addr 172.21.15.33:0/2386551602 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:23:29.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:29.305+0000 7fa076a00700 1 --2- 172.21.15.33:0/2386551602 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa078102ee0 0x7fa078104510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:23:29.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:29.305+0000 7fa077400700 1 --2- 172.21.15.33:0/2386551602 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa078102530 0x7fa078103fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:23:29.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:29.306+0000 7fa077e00700 1 -- 172.21.15.33:0/2386551602 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa078102530 msgr2=0x7fa078103fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:23:29.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:29.306+0000 7fa077e00700 1 --2- 172.21.15.33:0/2386551602 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa078102530 0x7fa078103fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:29.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:29.306+0000 7fa077e00700 1 -- 172.21.15.33:0/2386551602 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa078102ee0 msgr2=0x7fa078104510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:23:29.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:29.306+0000 7fa077e00700 1 --2- 172.21.15.33:0/2386551602 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa078102ee0 0x7fa078104510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:29.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:29.306+0000 7fa077e00700 1 -- 172.21.15.33:0/2386551602 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0781a7f80 con 0x7fa078107610 2024-05-03T18:23:29.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:29.306+0000 7fa077e00700 1 --2- 172.21.15.33:0/2386551602 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa078107610 0x7fa078104a50 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa070007ee0 tx=0x7fa07000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:23:29.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:29.306+0000 7fa06fe00700 1 -- 172.21.15.33:0/2386551602 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa070004bb0 con 0x7fa078107610 2024-05-03T18:23:29.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:29.307+0000 7fa07ca00700 1 -- 172.21.15.33:0/2386551602 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0781a8270 con 0x7fa078107610 2024-05-03T18:23:29.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:29.307+0000 7fa06fe00700 1 -- 172.21.15.33:0/2386551602 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa070004d50 con 0x7fa078107610 2024-05-03T18:23:29.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:29.307+0000 7fa06fe00700 1 -- 172.21.15.33:0/2386551602 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0700058b0 con 0x7fa078107610 2024-05-03T18:23:29.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:29.307+0000 7fa07ca00700 1 -- 172.21.15.33:0/2386551602 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0781a87d0 con 0x7fa078107610 2024-05-03T18:23:29.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:29.308+0000 7fa07ca00700 1 -- 172.21.15.33:0/2386551602 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa07806a190 con 0x7fa078107610 2024-05-03T18:23:29.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:29.310+0000 7fa06fe00700 1 -- 172.21.15.33:0/2386551602 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fa07001a020 con 0x7fa078107610 2024-05-03T18:23:29.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:29.312+0000 7fa06fe00700 1 --2- 172.21.15.33:0/2386551602 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fa05c0b2900 0x7fa05c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:23:29.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:29.312+0000 7fa06fe00700 1 -- 172.21.15.33:0/2386551602 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9071+0+0 (secure 0 0 0) 0x7fa0700e1b30 con 0x7fa078107610 2024-05-03T18:23:29.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:29.312+0000 7fa077400700 1 --2- 172.21.15.33:0/2386551602 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fa05c0b2900 0x7fa05c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:23:29.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:29.313+0000 7fa077400700 1 --2- 172.21.15.33:0/2386551602 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fa05c0b2900 0x7fa05c0b4dc0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa068005950 tx=0x7fa06800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:23:29.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:29.317+0000 7fa06fe00700 1 -- 172.21.15.33:0/2386551602 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fa070010070 con 0x7fa078107610 2024-05-03T18:23:29.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:29.575+0000 7fa07ca00700 1 -- 172.21.15.33:0/2386551602 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa07802e510 con 0x7fa078107610 2024-05-03T18:23:29.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:29.576+0000 7fa06fe00700 1 -- 172.21.15.33:0/2386551602 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v87) ==== 81+0+591 (secure 0 0 0) 0x7fa0700a94b0 con 0x7fa078107610 2024-05-03T18:23:29.576 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/3991947997 2024-05-04T18:12:19.127652+0000 2024-05-03T18:23:29.576 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/965859483 2024-05-04T18:12:19.127652+0000 2024-05-03T18:23:29.576 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/2014902144 2024-05-04T18:11:44.239835+0000 2024-05-03T18:23:29.576 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/1314704208 2024-05-04T18:11:44.239835+0000 2024-05-03T18:23:29.576 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/1614640996 2024-05-04T18:11:44.239835+0000 2024-05-03T18:23:29.576 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6800/4007322071 2024-05-04T18:11:44.239835+0000 2024-05-03T18:23:29.576 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6801/2830135294 2024-05-04T18:12:19.127652+0000 2024-05-03T18:23:29.576 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6800/2830135294 2024-05-04T18:12:19.127652+0000 2024-05-03T18:23:29.576 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/3767665033 2024-05-04T18:12:19.127652+0000 2024-05-03T18:23:29.577 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6801/4007322071 2024-05-04T18:11:44.239835+0000 2024-05-03T18:23:29.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:29.579+0000 7fa07ca00700 1 -- 172.21.15.33:0/2386551602 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fa05c0b2900 msgr2=0x7fa05c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:23:29.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:29.579+0000 7fa07ca00700 1 --2- 172.21.15.33:0/2386551602 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fa05c0b2900 0x7fa05c0b4dc0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa068005950 tx=0x7fa06800b410 comp rx=0 tx=0).stop 2024-05-03T18:23:29.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:29.579+0000 7fa07ca00700 1 -- 172.21.15.33:0/2386551602 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa078107610 msgr2=0x7fa078104a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:23:29.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:29.579+0000 7fa07ca00700 1 --2- 172.21.15.33:0/2386551602 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa078107610 0x7fa078104a50 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa070007ee0 tx=0x7fa07000d3b0 comp rx=0 tx=0).stop 2024-05-03T18:23:29.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:29.580+0000 7fa07ca00700 1 -- 172.21.15.33:0/2386551602 shutdown_connections 2024-05-03T18:23:29.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:29.580+0000 7fa07ca00700 1 --2- 172.21.15.33:0/2386551602 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa078107610 0x7fa078104a50 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:29.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:29.580+0000 7fa07ca00700 1 --2- 172.21.15.33:0/2386551602 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa078102530 0x7fa078103fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:29.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:29.580+0000 7fa07ca00700 1 --2- 172.21.15.33:0/2386551602 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fa05c0b2900 0x7fa05c0b4dc0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:29.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:29.580+0000 7fa07ca00700 1 --2- 172.21.15.33:0/2386551602 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa078102ee0 0x7fa078104510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:29.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:29.580+0000 7fa07ca00700 1 -- 172.21.15.33:0/2386551602 >> 172.21.15.33:0/2386551602 conn(0x7fa0780fddc0 msgr2=0x7fa07806b560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:23:29.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:29.580+0000 7fa07ca00700 1 -- 172.21.15.33:0/2386551602 shutdown_connections 2024-05-03T18:23:29.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:29.580+0000 7fa07ca00700 1 -- 172.21.15.33:0/2386551602 wait complete. 2024-05-03T18:23:29.582 INFO:teuthology.orchestra.run.smithi033.stderr:listed 10 entries 2024-05-03T18:23:29.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:23:29 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/2386551602' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-03T18:23:30.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:23:29 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/2386551602' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-03T18:23:30.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:23:29 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/2386551602' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-03T18:23:30.449 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi033.front.sepia.ceph.com... 2024-05-03T18:23:30.450 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T18:23:30.450 DEBUG:teuthology.orchestra.run.smithi033:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-05-03T18:23:30.550 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi033.stderr:ceph-fuse[75044]: fuse finished with error 0 and tester_r 0 2024-05-03T18:23:30.557 INFO:teuthology.orchestra.run:waiting for 300 2024-05-03T18:23:30.633 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi033.front.sepia.ceph.com 2024-05-03T18:23:30.634 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T18:23:30.634 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-05-03T18:23:30.702 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-03T18:23:30.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:23:30 smithi033 ceph-mon[28330]: pgmap v412: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 416 B/s wr, 0 op/s 2024-05-03T18:23:31.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:23:30 smithi099 ceph-mon[31658]: pgmap v412: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 416 B/s wr, 0 op/s 2024-05-03T18:23:31.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:23:30 smithi150 ceph-mon[32492]: pgmap v412: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 416 B/s wr, 0 op/s 2024-05-03T18:23:31.166 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:23:32.920 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:23:32 smithi033 ceph-mon[28330]: pgmap v413: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 378 B/s wr, 0 op/s 2024-05-03T18:23:33.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:23:32 smithi099 ceph-mon[31658]: pgmap v413: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 378 B/s wr, 0 op/s 2024-05-03T18:23:33.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:23:32 smithi150 ceph-mon[32492]: pgmap v413: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 378 B/s wr, 0 op/s 2024-05-03T18:23:33.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:33.446+0000 7fbf33200700 1 -- 172.21.15.33:0/1225081705 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf2c06bb40 msgr2=0x7fbf2c110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:23:33.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:33.446+0000 7fbf33200700 1 --2- 172.21.15.33:0/1225081705 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf2c06bb40 0x7fbf2c110330 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fbf2400b0a0 tx=0x7fbf2400b3b0 comp rx=0 tx=0).stop 2024-05-03T18:23:33.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:33.446+0000 7fbf33200700 1 -- 172.21.15.33:0/1225081705 shutdown_connections 2024-05-03T18:23:33.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:33.446+0000 7fbf33200700 1 --2- 172.21.15.33:0/1225081705 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf2c06bb40 0x7fbf2c110330 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:33.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:33.446+0000 7fbf33200700 1 --2- 172.21.15.33:0/1225081705 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbf2c109510 0x7fbf2c1098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:33.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:33.446+0000 7fbf33200700 1 --2- 172.21.15.33:0/1225081705 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf2c06b180 0x7fbf2c06b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:33.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:33.446+0000 7fbf33200700 1 -- 172.21.15.33:0/1225081705 >> 172.21.15.33:0/1225081705 conn(0x7fbf2c079350 msgr2=0x7fbf2c079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:23:33.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:33.447+0000 7fbf33200700 1 -- 172.21.15.33:0/1225081705 shutdown_connections 2024-05-03T18:23:33.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:33.447+0000 7fbf33200700 1 -- 172.21.15.33:0/1225081705 wait complete. 2024-05-03T18:23:33.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:33.449+0000 7fbf33200700 1 Processor -- start 2024-05-03T18:23:33.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:33.449+0000 7fbf33200700 1 -- start start 2024-05-03T18:23:33.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:33.449+0000 7fbf33200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbf2c06b180 0x7fbf2c1a1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:23:33.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:33.449+0000 7fbf33200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf2c06bb40 0x7fbf2c1a1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:23:33.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:33.449+0000 7fbf33200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf2c109510 0x7fbf2c19bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:23:33.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:33.449+0000 7fbf33200700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbf2c113520 con 0x7fbf2c06bb40 2024-05-03T18:23:33.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:33.449+0000 7fbf33200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbf2c1133a0 con 0x7fbf2c06b180 2024-05-03T18:23:33.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:33.449+0000 7fbf33200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fbf2c1136a0 con 0x7fbf2c109510 2024-05-03T18:23:33.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:33.450+0000 7fbf23400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf2c06bb40 0x7fbf2c1a1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:23:33.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:33.450+0000 7fbf23400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf2c06bb40 0x7fbf2c1a1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57586/0 (socket says 172.21.15.33:57586) 2024-05-03T18:23:33.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:33.450+0000 7fbf23400700 1 -- 172.21.15.33:0/3958437578 learned_addr learned my addr 172.21.15.33:0/3958437578 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:23:33.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:33.450+0000 7fbf2be00700 1 --2- 172.21.15.33:0/3958437578 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbf2c06b180 0x7fbf2c1a1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:23:33.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:33.450+0000 7fbf31200700 1 --2- 172.21.15.33:0/3958437578 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf2c109510 0x7fbf2c19bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:23:33.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:33.450+0000 7fbf23400700 1 -- 172.21.15.33:0/3958437578 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf2c109510 msgr2=0x7fbf2c19bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:23:33.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:33.450+0000 7fbf23400700 1 --2- 172.21.15.33:0/3958437578 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf2c109510 0x7fbf2c19bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:33.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:33.450+0000 7fbf23400700 1 -- 172.21.15.33:0/3958437578 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbf2c06b180 msgr2=0x7fbf2c1a1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:23:33.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:33.450+0000 7fbf23400700 1 --2- 172.21.15.33:0/3958437578 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbf2c06b180 0x7fbf2c1a1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:33.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:33.450+0000 7fbf23400700 1 -- 172.21.15.33:0/3958437578 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf2c19c260 con 0x7fbf2c06bb40 2024-05-03T18:23:33.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:33.451+0000 7fbf23400700 1 --2- 172.21.15.33:0/3958437578 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf2c06bb40 0x7fbf2c1a1e80 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fbf1400d950 tx=0x7fbf1400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:23:33.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:33.451+0000 7fbf29600700 1 -- 172.21.15.33:0/3958437578 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf14016070 con 0x7fbf2c06bb40 2024-05-03T18:23:33.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:33.451+0000 7fbf29600700 1 -- 172.21.15.33:0/3958437578 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbf14010460 con 0x7fbf2c06bb40 2024-05-03T18:23:33.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:33.451+0000 7fbf33200700 1 -- 172.21.15.33:0/3958437578 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf2c19c4f0 con 0x7fbf2c06bb40 2024-05-03T18:23:33.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:33.451+0000 7fbf29600700 1 -- 172.21.15.33:0/3958437578 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf1400f620 con 0x7fbf2c06bb40 2024-05-03T18:23:33.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:33.451+0000 7fbf33200700 1 -- 172.21.15.33:0/3958437578 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf2c1a8810 con 0x7fbf2c06bb40 2024-05-03T18:23:33.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:33.452+0000 7fbf33200700 1 -- 172.21.15.33:0/3958437578 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf2c06a190 con 0x7fbf2c06bb40 2024-05-03T18:23:33.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:33.456+0000 7fbf29600700 1 -- 172.21.15.33:0/3958437578 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fbf14019070 con 0x7fbf2c06bb40 2024-05-03T18:23:33.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:33.458+0000 7fbf29600700 1 --2- 172.21.15.33:0/3958437578 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fbf100b2900 0x7fbf100b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:23:33.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:33.458+0000 7fbf29600700 1 -- 172.21.15.33:0/3958437578 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9071+0+0 (secure 0 0 0) 0x7fbf140e2130 con 0x7fbf2c06bb40 2024-05-03T18:23:33.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:33.458+0000 7fbf2be00700 1 --2- 172.21.15.33:0/3958437578 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fbf100b2900 0x7fbf100b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:23:33.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:33.459+0000 7fbf2be00700 1 --2- 172.21.15.33:0/3958437578 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fbf100b2900 0x7fbf100b4dc0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c009bd0 tx=0x7fbf1c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:23:33.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:33.461+0000 7fbf29600700 1 -- 172.21.15.33:0/3958437578 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fbf140a9ab0 con 0x7fbf2c06bb40 2024-05-03T18:23:33.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:33.701+0000 7fbf33200700 1 -- 172.21.15.33:0/3958437578 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbf2c19cff0 con 0x7fbf2c06bb40 2024-05-03T18:23:33.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:33.703+0000 7fbf29600700 1 -- 172.21.15.33:0/3958437578 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+24499 (secure 0 0 0) 0x7fbf2c19cff0 con 0x7fbf2c06bb40 2024-05-03T18:23:33.705 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:23:33.705 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":87,"fsid":"5703bba4-0978-11ef-bc97-c7b262605968","created":"2024-05-03T18:11:00.899879+0000","modified":"2024-05-03T18:23:20.514086+0000","last_up_change":"2024-05-03T18:21:00.390173+0000","last_in_change":"2024-05-03T18:20:34.103727+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-03T18:15:37.297738+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-03T18:22:19.875368+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-03T18:22:24.432692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-03T18:22:38.722443+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"fdcc4e66-4ba0-4378-a9a4-7d907cd00b4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":637686669},{"type":"v1","addr":"172.21.15.33:6803","nonce":637686669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":637686669},{"type":"v1","addr":"172.21.15.33:6805","nonce":637686669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":637686669},{"type":"v1","addr":"172.21.15.33:6809","nonce":637686669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":637686669},{"type":"v1","addr":"172.21.15.33:6807","nonce":637686669}]},"public_addr":"172.21.15.33:6803/637686669","cluster_addr":"172.21.15.33:6805/637686669","heartbeat_back_addr":"172.21.15.33:6809/637686669","heartbeat_front_addr":"172.21.15.33:6807/637686669","state":["exists","up"]},{"osd":1,"uuid":"cf78cad3-fc24-43f4-8784-21cabea7277c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":1536122841},{"type":"v1","addr":"172.21.15.99:6801","nonce":1536122841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":1536122841},{"type":"v1","addr":"172.21.15.99:6803","nonce":1536122841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":1536122841},{"type":"v1","addr":"172.21.15.99:6807","nonce":1536122841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":1536122841},{"type":"v1","addr":"172.21.15.99:6805","nonce":1536122841}]},"public_addr":"172.21.15.99:6801/1536122841","cluster_addr":"172.21.15.99:6803/1536122841","heartbeat_back_addr":"172.21.15.99:6807/1536122841","heartbeat_front_addr":"172.21.15.99:6805/1536122841","state":["exists","up"]},{"osd":2,"uuid":"59f066d7-626b-410a-8d67-daca6fbbd3ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":1995659622},{"type":"v1","addr":"172.21.15.150:6801","nonce":1995659622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":1995659622},{"type":"v1","addr":"172.21.15.150:6803","nonce":1995659622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":1995659622},{"type":"v1","addr":"172.21.15.150:6807","nonce":1995659622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":1995659622},{"type":"v1","addr":"172.21.15.150:6805","nonce":1995659622}]},"public_addr":"172.21.15.150:6801/1995659622","cluster_addr":"172.21.15.150:6803/1995659622","heartbeat_back_addr":"172.21.15.150:6807/1995659622","heartbeat_front_addr":"172.21.15.150:6805/1995659622","state":["exists","up"]},{"osd":3,"uuid":"d88ed022-4dcb-45fc-9fc6-41ef7c4bf202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":3814942724},{"type":"v1","addr":"172.21.15.33:6811","nonce":3814942724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":3814942724},{"type":"v1","addr":"172.21.15.33:6813","nonce":3814942724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":3814942724},{"type":"v1","addr":"172.21.15.33:6817","nonce":3814942724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":3814942724},{"type":"v1","addr":"172.21.15.33:6815","nonce":3814942724}]},"public_addr":"172.21.15.33:6811/3814942724","cluster_addr":"172.21.15.33:6813/3814942724","heartbeat_back_addr":"172.21.15.33:6817/3814942724","heartbeat_front_addr":"172.21.15.33:6815/3814942724","state":["exists","up"]},{"osd":4,"uuid":"1235e1a6-ff6e-4668-ac1a-073c2a3b7c98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":29081029},{"type":"v1","addr":"172.21.15.99:6809","nonce":29081029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":29081029},{"type":"v1","addr":"172.21.15.99:6811","nonce":29081029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":29081029},{"type":"v1","addr":"172.21.15.99:6815","nonce":29081029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":29081029},{"type":"v1","addr":"172.21.15.99:6813","nonce":29081029}]},"public_addr":"172.21.15.99:6809/29081029","cluster_addr":"172.21.15.99:6811/29081029","heartbeat_back_addr":"172.21.15.99:6815/29081029","heartbeat_front_addr":"172.21.15.99:6813/29081029","state":["exists","up"]},{"osd":5,"uuid":"4b84bff8-d6d7-4258-a0ee-53bc19802585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":3172189533},{"type":"v1","addr":"172.21.15.150:6809","nonce":3172189533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3172189533},{"type":"v1","addr":"172.21.15.150:6811","nonce":3172189533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3172189533},{"type":"v1","addr":"172.21.15.150:6815","nonce":3172189533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3172189533},{"type":"v1","addr":"172.21.15.150:6813","nonce":3172189533}]},"public_addr":"172.21.15.150:6809/3172189533","cluster_addr":"172.21.15.150:6811/3172189533","heartbeat_back_addr":"172.21.15.150:6815/3172189533","heartbeat_front_addr":"172.21.15.150:6813/3172189533","state":["exists","up"]},{"osd":6,"uuid":"51f5fbdd-3794-4c48-9f6c-478f0fa35ac5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":4234541418},{"type":"v1","addr":"172.21.15.33:6819","nonce":4234541418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":4234541418},{"type":"v1","addr":"172.21.15.33:6821","nonce":4234541418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":4234541418},{"type":"v1","addr":"172.21.15.33:6825","nonce":4234541418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":4234541418},{"type":"v1","addr":"172.21.15.33:6823","nonce":4234541418}]},"public_addr":"172.21.15.33:6819/4234541418","cluster_addr":"172.21.15.33:6821/4234541418","heartbeat_back_addr":"172.21.15.33:6825/4234541418","heartbeat_front_addr":"172.21.15.33:6823/4234541418","state":["exists","up"]},{"osd":7,"uuid":"982cc5b3-3163-4505-9bde-a27a8c289457","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":908673090},{"type":"v1","addr":"172.21.15.99:6817","nonce":908673090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":908673090},{"type":"v1","addr":"172.21.15.99:6819","nonce":908673090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":908673090},{"type":"v1","addr":"172.21.15.99:6823","nonce":908673090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":908673090},{"type":"v1","addr":"172.21.15.99:6821","nonce":908673090}]},"public_addr":"172.21.15.99:6817/908673090","cluster_addr":"172.21.15.99:6819/908673090","heartbeat_back_addr":"172.21.15.99:6823/908673090","heartbeat_front_addr":"172.21.15.99:6821/908673090","state":["exists","up"]},{"osd":8,"uuid":"84c9d037-eb84-4eb4-8aa2-e784b703d36c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":1308083602},{"type":"v1","addr":"172.21.15.150:6817","nonce":1308083602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":1308083602},{"type":"v1","addr":"172.21.15.150:6819","nonce":1308083602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":1308083602},{"type":"v1","addr":"172.21.15.150:6823","nonce":1308083602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":1308083602},{"type":"v1","addr":"172.21.15.150:6821","nonce":1308083602}]},"public_addr":"172.21.15.150:6817/1308083602","cluster_addr":"172.21.15.150:6819/1308083602","heartbeat_back_addr":"172.21.15.150:6823/1308083602","heartbeat_front_addr":"172.21.15.150:6821/1308083602","state":["exists","up"]},{"osd":9,"uuid":"a02e965f-0d8f-45e2-9490-337f7f6e3881","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":4133096905},{"type":"v1","addr":"172.21.15.33:6827","nonce":4133096905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":4133096905},{"type":"v1","addr":"172.21.15.33:6829","nonce":4133096905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4133096905},{"type":"v1","addr":"172.21.15.33:6833","nonce":4133096905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":4133096905},{"type":"v1","addr":"172.21.15.33:6831","nonce":4133096905}]},"public_addr":"172.21.15.33:6827/4133096905","cluster_addr":"172.21.15.33:6829/4133096905","heartbeat_back_addr":"172.21.15.33:6833/4133096905","heartbeat_front_addr":"172.21.15.33:6831/4133096905","state":["exists","up"]},{"osd":10,"uuid":"8b29bb96-76e8-45a0-ab19-8d082d755fe9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":935746136},{"type":"v1","addr":"172.21.15.99:6825","nonce":935746136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":935746136},{"type":"v1","addr":"172.21.15.99:6827","nonce":935746136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":935746136},{"type":"v1","addr":"172.21.15.99:6831","nonce":935746136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":935746136},{"type":"v1","addr":"172.21.15.99:6829","nonce":935746136}]},"public_addr":"172.21.15.99:6825/935746136","cluster_addr":"172.21.15.99:6827/935746136","heartbeat_back_addr":"172.21.15.99:6831/935746136","heartbeat_front_addr":"172.21.15.99:6829/935746136","state":["exists","up"]},{"osd":11,"uuid":"07a6a825-c902-4bfd-97cc-cc9074532af5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2069349808},{"type":"v1","addr":"172.21.15.150:6825","nonce":2069349808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2069349808},{"type":"v1","addr":"172.21.15.150:6827","nonce":2069349808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2069349808},{"type":"v1","addr":"172.21.15.150:6831","nonce":2069349808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2069349808},{"type":"v1","addr":"172.21.15.150:6829","nonce":2069349808}]},"public_addr":"172.21.15.150:6825/2069349808","cluster_addr":"172.21.15.150:6827/2069349808","heartbeat_back_addr":"172.21.15.150:6831/2069349808","heartbeat_front_addr":"172.21.15.150:6829/2069349808","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:14:27.454536+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:15:00.538570+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:15:34.125289+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:16:09.853870+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:16:44.928107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:17:19.644037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:17:56.108443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:18:31.731499+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:19:08.062358+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:19:45.204430+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:20:21.013710+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:20:57.653698+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11},{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/3991947997":"2024-05-04T18:12:19.127652+0000","172.21.15.33:0/965859483":"2024-05-04T18:12:19.127652+0000","172.21.15.33:0/2014902144":"2024-05-04T18:11:44.239835+0000","172.21.15.33:0/1314704208":"2024-05-04T18:11:44.239835+0000","172.21.15.33:0/1614640996":"2024-05-04T18:11:44.239835+0000","172.21.15.33:6800/4007322071":"2024-05-04T18:11:44.239835+0000","172.21.15.33:6801/2830135294":"2024-05-04T18:12:19.127652+0000","172.21.15.33:6800/2830135294":"2024-05-04T18:12:19.127652+0000","172.21.15.33:0/3767665033":"2024-05-04T18:12:19.127652+0000","172.21.15.33:6801/4007322071":"2024-05-04T18:11:44.239835+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-03T18:23:33.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:33.706+0000 7fbf33200700 1 -- 172.21.15.33:0/3958437578 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fbf100b2900 msgr2=0x7fbf100b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:23:33.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:33.706+0000 7fbf33200700 1 --2- 172.21.15.33:0/3958437578 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fbf100b2900 0x7fbf100b4dc0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c009bd0 tx=0x7fbf1c009260 comp rx=0 tx=0).stop 2024-05-03T18:23:33.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:33.706+0000 7fbf33200700 1 -- 172.21.15.33:0/3958437578 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf2c06bb40 msgr2=0x7fbf2c1a1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:23:33.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:33.706+0000 7fbf33200700 1 --2- 172.21.15.33:0/3958437578 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf2c06bb40 0x7fbf2c1a1e80 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fbf1400d950 tx=0x7fbf1400dd10 comp rx=0 tx=0).stop 2024-05-03T18:23:33.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:33.706+0000 7fbf33200700 1 -- 172.21.15.33:0/3958437578 shutdown_connections 2024-05-03T18:23:33.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:33.706+0000 7fbf33200700 1 --2- 172.21.15.33:0/3958437578 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf2c109510 0x7fbf2c19bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:33.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:33.706+0000 7fbf33200700 1 --2- 172.21.15.33:0/3958437578 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fbf100b2900 0x7fbf100b4dc0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:33.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:33.706+0000 7fbf33200700 1 --2- 172.21.15.33:0/3958437578 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbf2c06b180 0x7fbf2c1a1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:33.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:33.706+0000 7fbf33200700 1 --2- 172.21.15.33:0/3958437578 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf2c06bb40 0x7fbf2c1a1e80 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:33.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:33.706+0000 7fbf33200700 1 -- 172.21.15.33:0/3958437578 >> 172.21.15.33:0/3958437578 conn(0x7fbf2c079350 msgr2=0x7fbf2c10e660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:23:33.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:33.707+0000 7fbf33200700 1 -- 172.21.15.33:0/3958437578 shutdown_connections 2024-05-03T18:23:33.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:33.707+0000 7fbf33200700 1 -- 172.21.15.33:0/3958437578 wait complete. 2024-05-03T18:23:34.528 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-03T18:23:34.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:23:34 smithi033 ceph-mon[28330]: pgmap v414: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-03T18:23:34.735 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:23:34 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/3958437578' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T18:23:34.957 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:23:34 smithi150 ceph-mon[32492]: pgmap v414: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-03T18:23:34.957 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:23:34 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/3958437578' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T18:23:35.014 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:23:35.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:23:34 smithi099 ceph-mon[31658]: pgmap v414: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-03T18:23:35.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:23:34 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/3958437578' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T18:23:36.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:23:36 smithi150 ceph-mon[32492]: pgmap v415: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-03T18:23:36.970 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:23:36 smithi033 ceph-mon[28330]: pgmap v415: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-03T18:23:37.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:23:36 smithi099 ceph-mon[31658]: pgmap v415: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-03T18:23:37.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:37.255+0000 7f09f1a00700 1 -- 172.21.15.33:0/563823323 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09ec1056c0 msgr2=0x7f09ec105b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:23:37.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:37.255+0000 7f09f1a00700 1 --2- 172.21.15.33:0/563823323 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09ec1056c0 0x7f09ec105b40 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f09d8009990 tx=0x7f09d8009ca0 comp rx=0 tx=0).stop 2024-05-03T18:23:37.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:37.255+0000 7f09f1a00700 1 -- 172.21.15.33:0/563823323 shutdown_connections 2024-05-03T18:23:37.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:37.255+0000 7f09f1a00700 1 --2- 172.21.15.33:0/563823323 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09ec106080 0x7f09ec112590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:37.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:37.255+0000 7f09f1a00700 1 --2- 172.21.15.33:0/563823323 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f09ec10b6c0 0x7f09ec10baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:37.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:37.255+0000 7f09f1a00700 1 --2- 172.21.15.33:0/563823323 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09ec1056c0 0x7f09ec105b40 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:37.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:37.255+0000 7f09f1a00700 1 -- 172.21.15.33:0/563823323 >> 172.21.15.33:0/563823323 conn(0x7f09ec1011b0 msgr2=0x7f09ec1035d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:23:37.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:37.256+0000 7f09f1a00700 1 -- 172.21.15.33:0/563823323 shutdown_connections 2024-05-03T18:23:37.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:37.256+0000 7f09f1a00700 1 -- 172.21.15.33:0/563823323 wait complete. 2024-05-03T18:23:37.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:37.257+0000 7f09f1a00700 1 Processor -- start 2024-05-03T18:23:37.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:37.257+0000 7f09f1a00700 1 -- start start 2024-05-03T18:23:37.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:37.257+0000 7f09f1a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09ec1056c0 0x7f09ec1a1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:23:37.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:37.257+0000 7f09f1a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f09ec106080 0x7f09ec1a1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:23:37.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:37.257+0000 7f09f1a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09ec10b6c0 0x7f09ec19baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:23:37.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:37.257+0000 7f09f1a00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f09ec1157c0 con 0x7f09ec10b6c0 2024-05-03T18:23:37.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:37.257+0000 7f09f1a00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f09ec115640 con 0x7f09ec106080 2024-05-03T18:23:37.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:37.257+0000 7f09f1a00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f09ec115940 con 0x7f09ec1056c0 2024-05-03T18:23:37.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:37.257+0000 7f09eb400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09ec10b6c0 0x7f09ec19baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:23:37.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:37.258+0000 7f09eb400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09ec10b6c0 0x7f09ec19baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38700/0 (socket says 172.21.15.33:38700) 2024-05-03T18:23:37.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:37.258+0000 7f09eb400700 1 -- 172.21.15.33:0/2416517492 learned_addr learned my addr 172.21.15.33:0/2416517492 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:23:37.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:37.258+0000 7f09eaa00700 1 --2- 172.21.15.33:0/2416517492 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09ec1056c0 0x7f09ec1a1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:23:37.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:37.258+0000 7f09ea000700 1 --2- 172.21.15.33:0/2416517492 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f09ec106080 0x7f09ec1a1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:23:37.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:37.258+0000 7f09eb400700 1 -- 172.21.15.33:0/2416517492 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09ec1056c0 msgr2=0x7f09ec1a1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:23:37.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:37.258+0000 7f09eb400700 1 --2- 172.21.15.33:0/2416517492 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09ec1056c0 0x7f09ec1a1890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:37.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:37.258+0000 7f09eb400700 1 -- 172.21.15.33:0/2416517492 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f09ec106080 msgr2=0x7f09ec1a1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:23:37.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:37.258+0000 7f09eb400700 1 --2- 172.21.15.33:0/2416517492 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f09ec106080 0x7f09ec1a1dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:37.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:37.258+0000 7f09eb400700 1 -- 172.21.15.33:0/2416517492 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09ec19c200 con 0x7f09ec10b6c0 2024-05-03T18:23:37.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:37.258+0000 7f09ea000700 1 --2- 172.21.15.33:0/2416517492 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f09ec106080 0x7f09ec1a1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:23:37.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:37.258+0000 7f09eb400700 1 --2- 172.21.15.33:0/2416517492 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09ec10b6c0 0x7f09ec19baa0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f09e4007ee0 tx=0x7f09e400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:23:37.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:37.259+0000 7f09e3400700 1 -- 172.21.15.33:0/2416517492 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09e400dbd0 con 0x7f09ec10b6c0 2024-05-03T18:23:37.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:37.259+0000 7f09e3400700 1 -- 172.21.15.33:0/2416517492 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f09e400dd70 con 0x7f09ec10b6c0 2024-05-03T18:23:37.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:37.259+0000 7f09f1a00700 1 -- 172.21.15.33:0/2416517492 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09ec19c4f0 con 0x7f09ec10b6c0 2024-05-03T18:23:37.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:37.259+0000 7f09e3400700 1 -- 172.21.15.33:0/2416517492 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09e40058d0 con 0x7f09ec10b6c0 2024-05-03T18:23:37.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:37.259+0000 7f09f1a00700 1 -- 172.21.15.33:0/2416517492 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09ec1a8760 con 0x7f09ec10b6c0 2024-05-03T18:23:37.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:37.260+0000 7f09f1a00700 1 -- 172.21.15.33:0/2416517492 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09ec06a190 con 0x7f09ec10b6c0 2024-05-03T18:23:37.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:37.263+0000 7f09e3400700 1 -- 172.21.15.33:0/2416517492 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f09e401a020 con 0x7f09ec10b6c0 2024-05-03T18:23:37.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:37.264+0000 7f09e3400700 1 --2- 172.21.15.33:0/2416517492 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f09d00b2900 0x7f09d00b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:23:37.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:37.264+0000 7f09e3400700 1 -- 172.21.15.33:0/2416517492 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9071+0+0 (secure 0 0 0) 0x7f09e40e2340 con 0x7f09ec10b6c0 2024-05-03T18:23:37.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:37.264+0000 7f09eaa00700 1 --2- 172.21.15.33:0/2416517492 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f09d00b2900 0x7f09d00b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:23:37.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:37.265+0000 7f09eaa00700 1 --2- 172.21.15.33:0/2416517492 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f09d00b2900 0x7f09d00b4dc0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f09ec103220 tx=0x7f09dc00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:23:37.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:37.267+0000 7f09e3400700 1 -- 172.21.15.33:0/2416517492 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f09e4010070 con 0x7f09ec10b6c0 2024-05-03T18:23:37.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:37.504+0000 7f09f1a00700 1 -- 172.21.15.33:0/2416517492 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f09ec02e510 con 0x7f09ec10b6c0 2024-05-03T18:23:37.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:37.507+0000 7f09e3400700 1 -- 172.21.15.33:0/2416517492 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+24499 (secure 0 0 0) 0x7f09e40a9cc0 con 0x7f09ec10b6c0 2024-05-03T18:23:37.509 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:23:37.509 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":87,"fsid":"5703bba4-0978-11ef-bc97-c7b262605968","created":"2024-05-03T18:11:00.899879+0000","modified":"2024-05-03T18:23:20.514086+0000","last_up_change":"2024-05-03T18:21:00.390173+0000","last_in_change":"2024-05-03T18:20:34.103727+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-03T18:15:37.297738+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-03T18:22:19.875368+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-03T18:22:24.432692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-03T18:22:38.722443+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"fdcc4e66-4ba0-4378-a9a4-7d907cd00b4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":637686669},{"type":"v1","addr":"172.21.15.33:6803","nonce":637686669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":637686669},{"type":"v1","addr":"172.21.15.33:6805","nonce":637686669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":637686669},{"type":"v1","addr":"172.21.15.33:6809","nonce":637686669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":637686669},{"type":"v1","addr":"172.21.15.33:6807","nonce":637686669}]},"public_addr":"172.21.15.33:6803/637686669","cluster_addr":"172.21.15.33:6805/637686669","heartbeat_back_addr":"172.21.15.33:6809/637686669","heartbeat_front_addr":"172.21.15.33:6807/637686669","state":["exists","up"]},{"osd":1,"uuid":"cf78cad3-fc24-43f4-8784-21cabea7277c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":1536122841},{"type":"v1","addr":"172.21.15.99:6801","nonce":1536122841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":1536122841},{"type":"v1","addr":"172.21.15.99:6803","nonce":1536122841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":1536122841},{"type":"v1","addr":"172.21.15.99:6807","nonce":1536122841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":1536122841},{"type":"v1","addr":"172.21.15.99:6805","nonce":1536122841}]},"public_addr":"172.21.15.99:6801/1536122841","cluster_addr":"172.21.15.99:6803/1536122841","heartbeat_back_addr":"172.21.15.99:6807/1536122841","heartbeat_front_addr":"172.21.15.99:6805/1536122841","state":["exists","up"]},{"osd":2,"uuid":"59f066d7-626b-410a-8d67-daca6fbbd3ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":1995659622},{"type":"v1","addr":"172.21.15.150:6801","nonce":1995659622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":1995659622},{"type":"v1","addr":"172.21.15.150:6803","nonce":1995659622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":1995659622},{"type":"v1","addr":"172.21.15.150:6807","nonce":1995659622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":1995659622},{"type":"v1","addr":"172.21.15.150:6805","nonce":1995659622}]},"public_addr":"172.21.15.150:6801/1995659622","cluster_addr":"172.21.15.150:6803/1995659622","heartbeat_back_addr":"172.21.15.150:6807/1995659622","heartbeat_front_addr":"172.21.15.150:6805/1995659622","state":["exists","up"]},{"osd":3,"uuid":"d88ed022-4dcb-45fc-9fc6-41ef7c4bf202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":3814942724},{"type":"v1","addr":"172.21.15.33:6811","nonce":3814942724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":3814942724},{"type":"v1","addr":"172.21.15.33:6813","nonce":3814942724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":3814942724},{"type":"v1","addr":"172.21.15.33:6817","nonce":3814942724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":3814942724},{"type":"v1","addr":"172.21.15.33:6815","nonce":3814942724}]},"public_addr":"172.21.15.33:6811/3814942724","cluster_addr":"172.21.15.33:6813/3814942724","heartbeat_back_addr":"172.21.15.33:6817/3814942724","heartbeat_front_addr":"172.21.15.33:6815/3814942724","state":["exists","up"]},{"osd":4,"uuid":"1235e1a6-ff6e-4668-ac1a-073c2a3b7c98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":29081029},{"type":"v1","addr":"172.21.15.99:6809","nonce":29081029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":29081029},{"type":"v1","addr":"172.21.15.99:6811","nonce":29081029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":29081029},{"type":"v1","addr":"172.21.15.99:6815","nonce":29081029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":29081029},{"type":"v1","addr":"172.21.15.99:6813","nonce":29081029}]},"public_addr":"172.21.15.99:6809/29081029","cluster_addr":"172.21.15.99:6811/29081029","heartbeat_back_addr":"172.21.15.99:6815/29081029","heartbeat_front_addr":"172.21.15.99:6813/29081029","state":["exists","up"]},{"osd":5,"uuid":"4b84bff8-d6d7-4258-a0ee-53bc19802585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":3172189533},{"type":"v1","addr":"172.21.15.150:6809","nonce":3172189533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3172189533},{"type":"v1","addr":"172.21.15.150:6811","nonce":3172189533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3172189533},{"type":"v1","addr":"172.21.15.150:6815","nonce":3172189533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3172189533},{"type":"v1","addr":"172.21.15.150:6813","nonce":3172189533}]},"public_addr":"172.21.15.150:6809/3172189533","cluster_addr":"172.21.15.150:6811/3172189533","heartbeat_back_addr":"172.21.15.150:6815/3172189533","heartbeat_front_addr":"172.21.15.150:6813/3172189533","state":["exists","up"]},{"osd":6,"uuid":"51f5fbdd-3794-4c48-9f6c-478f0fa35ac5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":4234541418},{"type":"v1","addr":"172.21.15.33:6819","nonce":4234541418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":4234541418},{"type":"v1","addr":"172.21.15.33:6821","nonce":4234541418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":4234541418},{"type":"v1","addr":"172.21.15.33:6825","nonce":4234541418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":4234541418},{"type":"v1","addr":"172.21.15.33:6823","nonce":4234541418}]},"public_addr":"172.21.15.33:6819/4234541418","cluster_addr":"172.21.15.33:6821/4234541418","heartbeat_back_addr":"172.21.15.33:6825/4234541418","heartbeat_front_addr":"172.21.15.33:6823/4234541418","state":["exists","up"]},{"osd":7,"uuid":"982cc5b3-3163-4505-9bde-a27a8c289457","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":908673090},{"type":"v1","addr":"172.21.15.99:6817","nonce":908673090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":908673090},{"type":"v1","addr":"172.21.15.99:6819","nonce":908673090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":908673090},{"type":"v1","addr":"172.21.15.99:6823","nonce":908673090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":908673090},{"type":"v1","addr":"172.21.15.99:6821","nonce":908673090}]},"public_addr":"172.21.15.99:6817/908673090","cluster_addr":"172.21.15.99:6819/908673090","heartbeat_back_addr":"172.21.15.99:6823/908673090","heartbeat_front_addr":"172.21.15.99:6821/908673090","state":["exists","up"]},{"osd":8,"uuid":"84c9d037-eb84-4eb4-8aa2-e784b703d36c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":1308083602},{"type":"v1","addr":"172.21.15.150:6817","nonce":1308083602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":1308083602},{"type":"v1","addr":"172.21.15.150:6819","nonce":1308083602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":1308083602},{"type":"v1","addr":"172.21.15.150:6823","nonce":1308083602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":1308083602},{"type":"v1","addr":"172.21.15.150:6821","nonce":1308083602}]},"public_addr":"172.21.15.150:6817/1308083602","cluster_addr":"172.21.15.150:6819/1308083602","heartbeat_back_addr":"172.21.15.150:6823/1308083602","heartbeat_front_addr":"172.21.15.150:6821/1308083602","state":["exists","up"]},{"osd":9,"uuid":"a02e965f-0d8f-45e2-9490-337f7f6e3881","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":4133096905},{"type":"v1","addr":"172.21.15.33:6827","nonce":4133096905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":4133096905},{"type":"v1","addr":"172.21.15.33:6829","nonce":4133096905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4133096905},{"type":"v1","addr":"172.21.15.33:6833","nonce":4133096905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":4133096905},{"type":"v1","addr":"172.21.15.33:6831","nonce":4133096905}]},"public_addr":"172.21.15.33:6827/4133096905","cluster_addr":"172.21.15.33:6829/4133096905","heartbeat_back_addr":"172.21.15.33:6833/4133096905","heartbeat_front_addr":"172.21.15.33:6831/4133096905","state":["exists","up"]},{"osd":10,"uuid":"8b29bb96-76e8-45a0-ab19-8d082d755fe9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":935746136},{"type":"v1","addr":"172.21.15.99:6825","nonce":935746136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":935746136},{"type":"v1","addr":"172.21.15.99:6827","nonce":935746136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":935746136},{"type":"v1","addr":"172.21.15.99:6831","nonce":935746136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":935746136},{"type":"v1","addr":"172.21.15.99:6829","nonce":935746136}]},"public_addr":"172.21.15.99:6825/935746136","cluster_addr":"172.21.15.99:6827/935746136","heartbeat_back_addr":"172.21.15.99:6831/935746136","heartbeat_front_addr":"172.21.15.99:6829/935746136","state":["exists","up"]},{"osd":11,"uuid":"07a6a825-c902-4bfd-97cc-cc9074532af5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2069349808},{"type":"v1","addr":"172.21.15.150:6825","nonce":2069349808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2069349808},{"type":"v1","addr":"172.21.15.150:6827","nonce":2069349808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2069349808},{"type":"v1","addr":"172.21.15.150:6831","nonce":2069349808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2069349808},{"type":"v1","addr":"172.21.15.150:6829","nonce":2069349808}]},"public_addr":"172.21.15.150:6825/2069349808","cluster_addr":"172.21.15.150:6827/2069349808","heartbeat_back_addr":"172.21.15.150:6831/2069349808","heartbeat_front_addr":"172.21.15.150:6829/2069349808","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:14:27.454536+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:15:00.538570+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:15:34.125289+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:16:09.853870+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:16:44.928107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:17:19.644037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:17:56.108443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:18:31.731499+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:19:08.062358+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:19:45.204430+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:20:21.013710+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:20:57.653698+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11},{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/3991947997":"2024-05-04T18:12:19.127652+0000","172.21.15.33:0/965859483":"2024-05-04T18:12:19.127652+0000","172.21.15.33:0/2014902144":"2024-05-04T18:11:44.239835+0000","172.21.15.33:0/1314704208":"2024-05-04T18:11:44.239835+0000","172.21.15.33:0/1614640996":"2024-05-04T18:11:44.239835+0000","172.21.15.33:6800/4007322071":"2024-05-04T18:11:44.239835+0000","172.21.15.33:6801/2830135294":"2024-05-04T18:12:19.127652+0000","172.21.15.33:6800/2830135294":"2024-05-04T18:12:19.127652+0000","172.21.15.33:0/3767665033":"2024-05-04T18:12:19.127652+0000","172.21.15.33:6801/4007322071":"2024-05-04T18:11:44.239835+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-03T18:23:37.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:37.510+0000 7f09f1a00700 1 -- 172.21.15.33:0/2416517492 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f09d00b2900 msgr2=0x7f09d00b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:23:37.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:37.510+0000 7f09f1a00700 1 --2- 172.21.15.33:0/2416517492 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f09d00b2900 0x7f09d00b4dc0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f09ec103220 tx=0x7f09dc00a380 comp rx=0 tx=0).stop 2024-05-03T18:23:37.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:37.510+0000 7f09f1a00700 1 -- 172.21.15.33:0/2416517492 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09ec10b6c0 msgr2=0x7f09ec19baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:23:37.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:37.510+0000 7f09f1a00700 1 --2- 172.21.15.33:0/2416517492 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09ec10b6c0 0x7f09ec19baa0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f09e4007ee0 tx=0x7f09e400d3b0 comp rx=0 tx=0).stop 2024-05-03T18:23:37.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:37.511+0000 7f09f1a00700 1 -- 172.21.15.33:0/2416517492 shutdown_connections 2024-05-03T18:23:37.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:37.511+0000 7f09f1a00700 1 --2- 172.21.15.33:0/2416517492 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09ec10b6c0 0x7f09ec19baa0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:37.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:37.511+0000 7f09f1a00700 1 --2- 172.21.15.33:0/2416517492 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09ec1056c0 0x7f09ec1a1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:37.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:37.511+0000 7f09f1a00700 1 --2- 172.21.15.33:0/2416517492 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f09d00b2900 0x7f09d00b4dc0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:37.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:37.511+0000 7f09f1a00700 1 --2- 172.21.15.33:0/2416517492 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f09ec106080 0x7f09ec1a1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:37.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:37.511+0000 7f09f1a00700 1 -- 172.21.15.33:0/2416517492 >> 172.21.15.33:0/2416517492 conn(0x7f09ec1011b0 msgr2=0x7f09ec101840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:23:37.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:37.511+0000 7f09f1a00700 1 -- 172.21.15.33:0/2416517492 shutdown_connections 2024-05-03T18:23:37.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:37.511+0000 7f09f1a00700 1 -- 172.21.15.33:0/2416517492 wait complete. 2024-05-03T18:23:37.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:23:37 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/2416517492' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T18:23:38.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:23:37 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/2416517492' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T18:23:38.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:23:37 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/2416517492' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T18:23:38.348 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-05-03T18:23:38.815 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:23:38.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:23:38 smithi033 ceph-mon[28330]: pgmap v416: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-03T18:23:39.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:23:38 smithi099 ceph-mon[31658]: pgmap v416: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-03T18:23:39.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:23:38 smithi150 ceph-mon[32492]: pgmap v416: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-03T18:23:40.915 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:23:40 smithi033 ceph-mon[28330]: pgmap v417: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-03T18:23:41.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:23:40 smithi099 ceph-mon[31658]: pgmap v417: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-03T18:23:41.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:41.096+0000 7f9725200700 1 -- 172.21.15.33:0/100226932 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97200779d0 msgr2=0x7f9720077db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:23:41.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:41.096+0000 7f9725200700 1 --2- 172.21.15.33:0/100226932 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97200779d0 0x7f9720077db0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9710009990 tx=0x7f9710009ca0 comp rx=0 tx=0).stop 2024-05-03T18:23:41.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:41.096+0000 7f9725200700 1 -- 172.21.15.33:0/100226932 shutdown_connections 2024-05-03T18:23:41.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:41.096+0000 7f9725200700 1 --2- 172.21.15.33:0/100226932 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9720114380 0x7f9720116770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:41.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:41.096+0000 7f9725200700 1 --2- 172.21.15.33:0/100226932 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97200779d0 0x7f9720077db0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:41.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:41.096+0000 7f9725200700 1 --2- 172.21.15.33:0/100226932 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97200782f0 0x7f9720113e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:41.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:41.096+0000 7f9725200700 1 -- 172.21.15.33:0/100226932 >> 172.21.15.33:0/100226932 conn(0x7f97201011d0 msgr2=0x7f97201035f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:23:41.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:41.096+0000 7f9725200700 1 -- 172.21.15.33:0/100226932 shutdown_connections 2024-05-03T18:23:41.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:41.097+0000 7f9725200700 1 -- 172.21.15.33:0/100226932 wait complete. 2024-05-03T18:23:41.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:41.098+0000 7f9725200700 1 Processor -- start 2024-05-03T18:23:41.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:41.098+0000 7f9725200700 1 -- start start 2024-05-03T18:23:41.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:41.099+0000 7f9725200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97200779d0 0x7f97201a01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:23:41.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:41.099+0000 7f9725200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97200782f0 0x7f97201a06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:23:41.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:41.099+0000 7f9725200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9720114380 0x7f97201a4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:23:41.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:41.099+0000 7f9725200700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9720119c10 con 0x7f97200782f0 2024-05-03T18:23:41.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:41.099+0000 7f9725200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9720119a90 con 0x7f97200779d0 2024-05-03T18:23:41.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:41.099+0000 7f9725200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9720119d90 con 0x7f9720114380 2024-05-03T18:23:41.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:41.099+0000 7f9717e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97200782f0 0x7f97201a06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:23:41.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:41.099+0000 7f9717e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97200782f0 0x7f97201a06e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38708/0 (socket says 172.21.15.33:38708) 2024-05-03T18:23:41.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:41.099+0000 7f9717e00700 1 -- 172.21.15.33:0/3057180274 learned_addr learned my addr 172.21.15.33:0/3057180274 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:23:41.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:41.099+0000 7f971de00700 1 --2- 172.21.15.33:0/3057180274 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97200779d0 0x7f97201a01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:23:41.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:41.099+0000 7f971e800700 1 --2- 172.21.15.33:0/3057180274 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9720114380 0x7f97201a4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:23:41.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:41.100+0000 7f9717e00700 1 -- 172.21.15.33:0/3057180274 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9720114380 msgr2=0x7f97201a4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:23:41.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:41.100+0000 7f9717e00700 1 --2- 172.21.15.33:0/3057180274 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9720114380 0x7f97201a4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:41.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:41.100+0000 7f9717e00700 1 -- 172.21.15.33:0/3057180274 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97200779d0 msgr2=0x7f97201a01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:23:41.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:41.100+0000 7f9717e00700 1 --2- 172.21.15.33:0/3057180274 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97200779d0 0x7f97201a01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:41.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:41.100+0000 7f9717e00700 1 -- 172.21.15.33:0/3057180274 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97201a5160 con 0x7f97200782f0 2024-05-03T18:23:41.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:41.100+0000 7f9717e00700 1 --2- 172.21.15.33:0/3057180274 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97200782f0 0x7f97201a06e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f970800d980 tx=0x7f970800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:23:41.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:41.100+0000 7f9716a00700 1 -- 172.21.15.33:0/3057180274 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9708017070 con 0x7f97200782f0 2024-05-03T18:23:41.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:41.100+0000 7f9716a00700 1 -- 172.21.15.33:0/3057180274 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9708004830 con 0x7f97200782f0 2024-05-03T18:23:41.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:41.100+0000 7f9725200700 1 -- 172.21.15.33:0/3057180274 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97201a5450 con 0x7f97200782f0 2024-05-03T18:23:41.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:41.100+0000 7f9716a00700 1 -- 172.21.15.33:0/3057180274 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97080053a0 con 0x7f97200782f0 2024-05-03T18:23:41.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:41.101+0000 7f9725200700 1 -- 172.21.15.33:0/3057180274 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97201accd0 con 0x7f97200782f0 2024-05-03T18:23:41.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:41.102+0000 7f9725200700 1 -- 172.21.15.33:0/3057180274 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f972006a190 con 0x7f97200782f0 2024-05-03T18:23:41.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:41.105+0000 7f9716a00700 1 -- 172.21.15.33:0/3057180274 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f97080049e0 con 0x7f97200782f0 2024-05-03T18:23:41.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:41.108+0000 7f9716a00700 1 --2- 172.21.15.33:0/3057180274 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f97040b2900 0x7f97040b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:23:41.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:41.108+0000 7f9716a00700 1 -- 172.21.15.33:0/3057180274 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9071+0+0 (secure 0 0 0) 0x7f97080e28d0 con 0x7f97200782f0 2024-05-03T18:23:41.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:41.108+0000 7f971de00700 1 --2- 172.21.15.33:0/3057180274 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f97040b2900 0x7f97040b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:23:41.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:41.109+0000 7f971de00700 1 --2- 172.21.15.33:0/3057180274 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f97040b2900 0x7f97040b4dc0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f971000b5c0 tx=0x7f971001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:23:41.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:41.109+0000 7f9716a00700 1 -- 172.21.15.33:0/3057180274 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f9708020020 con 0x7f97200782f0 2024-05-03T18:23:41.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:23:40 smithi150 ceph-mon[32492]: pgmap v417: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-03T18:23:41.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:41.408+0000 7f9725200700 1 -- 172.21.15.33:0/3057180274 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f972002e510 con 0x7f97200782f0 2024-05-03T18:23:41.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:41.729+0000 7f9716a00700 1 -- 172.21.15.33:0/3057180274 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) ==== 118+0+0 (secure 0 0 0) 0x7f97080aa2d0 con 0x7f97200782f0 2024-05-03T18:23:41.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:41.733+0000 7f9725200700 1 -- 172.21.15.33:0/3057180274 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f97040b2900 msgr2=0x7f97040b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:23:41.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:41.733+0000 7f9725200700 1 --2- 172.21.15.33:0/3057180274 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f97040b2900 0x7f97040b4dc0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f971000b5c0 tx=0x7f971001a040 comp rx=0 tx=0).stop 2024-05-03T18:23:41.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:41.733+0000 7f9725200700 1 -- 172.21.15.33:0/3057180274 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97200782f0 msgr2=0x7f97201a06e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:23:41.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:41.733+0000 7f9725200700 1 --2- 172.21.15.33:0/3057180274 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97200782f0 0x7f97201a06e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f970800d980 tx=0x7f970800dd40 comp rx=0 tx=0).stop 2024-05-03T18:23:41.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:41.734+0000 7f9725200700 1 -- 172.21.15.33:0/3057180274 shutdown_connections 2024-05-03T18:23:41.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:41.734+0000 7f9725200700 1 --2- 172.21.15.33:0/3057180274 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9720114380 0x7f97201a4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:41.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:41.734+0000 7f9725200700 1 --2- 172.21.15.33:0/3057180274 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f97040b2900 0x7f97040b4dc0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:41.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:41.734+0000 7f9725200700 1 --2- 172.21.15.33:0/3057180274 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97200779d0 0x7f97201a01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:41.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:41.734+0000 7f9725200700 1 --2- 172.21.15.33:0/3057180274 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97200782f0 0x7f97201a06e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:41.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:41.734+0000 7f9725200700 1 -- 172.21.15.33:0/3057180274 >> 172.21.15.33:0/3057180274 conn(0x7f97201011d0 msgr2=0x7f97201035c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:23:41.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:41.734+0000 7f9725200700 1 -- 172.21.15.33:0/3057180274 shutdown_connections 2024-05-03T18:23:41.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:41.734+0000 7f9725200700 1 -- 172.21.15.33:0/3057180274 wait complete. 2024-05-03T18:23:41.987 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:23:41 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/3057180274' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-03T18:23:42.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:23:41 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/3057180274' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-03T18:23:42.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:23:41 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/3057180274' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-03T18:23:42.584 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 1, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-05-03T18:23:42.585 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-05-03T18:23:42.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:23:42 smithi033 ceph-mon[28330]: pgmap v418: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-03T18:23:42.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:23:42 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/3057180274' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-03T18:23:42.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:23:42 smithi033 ceph-mon[28330]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-03T18:23:43.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:23:42 smithi099 ceph-mon[31658]: pgmap v418: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-03T18:23:43.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:23:42 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/3057180274' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-03T18:23:43.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:23:42 smithi099 ceph-mon[31658]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-03T18:23:43.064 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:23:43.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:23:42 smithi150 ceph-mon[32492]: pgmap v418: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-03T18:23:43.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:23:42 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/3057180274' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-03T18:23:43.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:23:42 smithi150 ceph-mon[32492]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-03T18:23:44.866 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:23:44 smithi033 ceph-mon[28330]: pgmap v419: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-03T18:23:45.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:23:44 smithi099 ceph-mon[31658]: pgmap v419: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-03T18:23:45.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:23:44 smithi150 ceph-mon[32492]: pgmap v419: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-03T18:23:45.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:45.392+0000 7fa35ca00700 1 -- 172.21.15.33:0/911463076 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa35810b6b0 msgr2=0x7fa35810ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:23:45.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:45.392+0000 7fa35ca00700 1 --2- 172.21.15.33:0/911463076 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa35810b6b0 0x7fa35810ba90 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa348009990 tx=0x7fa348009ca0 comp rx=0 tx=0).stop 2024-05-03T18:23:45.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:45.392+0000 7fa35ca00700 1 -- 172.21.15.33:0/911463076 shutdown_connections 2024-05-03T18:23:45.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:45.392+0000 7fa35ca00700 1 --2- 172.21.15.33:0/911463076 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa358106070 0x7fa358112580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:45.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:45.392+0000 7fa35ca00700 1 --2- 172.21.15.33:0/911463076 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa35810b6b0 0x7fa35810ba90 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:45.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:45.392+0000 7fa35ca00700 1 --2- 172.21.15.33:0/911463076 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3581056b0 0x7fa358105b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:45.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:45.392+0000 7fa35ca00700 1 -- 172.21.15.33:0/911463076 >> 172.21.15.33:0/911463076 conn(0x7fa3581011a0 msgr2=0x7fa3581035c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:23:45.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:45.392+0000 7fa35ca00700 1 -- 172.21.15.33:0/911463076 shutdown_connections 2024-05-03T18:23:45.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:45.392+0000 7fa35ca00700 1 -- 172.21.15.33:0/911463076 wait complete. 2024-05-03T18:23:45.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:45.394+0000 7fa35ca00700 1 Processor -- start 2024-05-03T18:23:45.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:45.394+0000 7fa35ca00700 1 -- start start 2024-05-03T18:23:45.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:45.394+0000 7fa35ca00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3581056b0 0x7fa3581a1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:23:45.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:45.394+0000 7fa35ca00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa358106070 0x7fa3581a1e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:23:45.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:45.394+0000 7fa35ca00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa35810b6b0 0x7fa35819bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:23:45.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:45.394+0000 7fa35ca00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa3581157a0 con 0x7fa35810b6b0 2024-05-03T18:23:45.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:45.394+0000 7fa35ca00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa358115620 con 0x7fa358106070 2024-05-03T18:23:45.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:45.394+0000 7fa35ca00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa358115920 con 0x7fa3581056b0 2024-05-03T18:23:45.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:45.395+0000 7fa356800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa35810b6b0 0x7fa35819bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:23:45.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:45.395+0000 7fa356800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa35810b6b0 0x7fa35819bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38734/0 (socket says 172.21.15.33:38734) 2024-05-03T18:23:45.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:45.395+0000 7fa356800700 1 -- 172.21.15.33:0/3599663903 learned_addr learned my addr 172.21.15.33:0/3599663903 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:23:45.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:45.395+0000 7fa355e00700 1 --2- 172.21.15.33:0/3599663903 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3581056b0 0x7fa3581a1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:23:45.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:45.395+0000 7fa355400700 1 --2- 172.21.15.33:0/3599663903 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa358106070 0x7fa3581a1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:23:45.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:45.395+0000 7fa356800700 1 -- 172.21.15.33:0/3599663903 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3581056b0 msgr2=0x7fa3581a1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:23:45.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:45.395+0000 7fa356800700 1 --2- 172.21.15.33:0/3599663903 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3581056b0 0x7fa3581a1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:45.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:45.395+0000 7fa356800700 1 -- 172.21.15.33:0/3599663903 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa358106070 msgr2=0x7fa3581a1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:23:45.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:45.395+0000 7fa356800700 1 --2- 172.21.15.33:0/3599663903 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa358106070 0x7fa3581a1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:45.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:45.395+0000 7fa356800700 1 -- 172.21.15.33:0/3599663903 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa35819c240 con 0x7fa35810b6b0 2024-05-03T18:23:45.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:45.396+0000 7fa356800700 1 --2- 172.21.15.33:0/3599663903 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa35810b6b0 0x7fa35819bae0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa350007ee0 tx=0x7fa35000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:23:45.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:45.396+0000 7fa34ea00700 1 -- 172.21.15.33:0/3599663903 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa35000dbd0 con 0x7fa35810b6b0 2024-05-03T18:23:45.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:45.396+0000 7fa34ea00700 1 -- 172.21.15.33:0/3599663903 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa35000dd70 con 0x7fa35810b6b0 2024-05-03T18:23:45.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:45.396+0000 7fa34ea00700 1 -- 172.21.15.33:0/3599663903 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3500058d0 con 0x7fa35810b6b0 2024-05-03T18:23:45.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:45.396+0000 7fa35ca00700 1 -- 172.21.15.33:0/3599663903 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa35819c530 con 0x7fa35810b6b0 2024-05-03T18:23:45.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:45.396+0000 7fa35ca00700 1 -- 172.21.15.33:0/3599663903 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3581a87f0 con 0x7fa35810b6b0 2024-05-03T18:23:45.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:45.397+0000 7fa35ca00700 1 -- 172.21.15.33:0/3599663903 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa35806a190 con 0x7fa35810b6b0 2024-05-03T18:23:45.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:45.400+0000 7fa34ea00700 1 -- 172.21.15.33:0/3599663903 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fa35001a020 con 0x7fa35810b6b0 2024-05-03T18:23:45.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:45.401+0000 7fa34ea00700 1 --2- 172.21.15.33:0/3599663903 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fa33c0b2900 0x7fa33c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:23:45.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:45.402+0000 7fa34ea00700 1 -- 172.21.15.33:0/3599663903 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9071+0+0 (secure 0 0 0) 0x7fa3500e1d40 con 0x7fa35810b6b0 2024-05-03T18:23:45.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:45.402+0000 7fa355e00700 1 --2- 172.21.15.33:0/3599663903 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fa33c0b2900 0x7fa33c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:23:45.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:45.402+0000 7fa355e00700 1 --2- 172.21.15.33:0/3599663903 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fa33c0b2900 0x7fa33c0b4dc0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa34800b5c0 tx=0x7fa34801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:23:45.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:45.406+0000 7fa34ea00700 1 -- 172.21.15.33:0/3599663903 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fa350010070 con 0x7fa35810b6b0 2024-05-03T18:23:45.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:45.705+0000 7fa35ca00700 1 -- 172.21.15.33:0/3599663903 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7fa35802e510 con 0x7fa35810b6b0 2024-05-03T18:23:45.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:45.750+0000 7fa34ea00700 1 -- 172.21.15.33:0/3599663903 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v19) ==== 121+0+0 (secure 0 0 0) 0x7fa3500a96c0 con 0x7fa35810b6b0 2024-05-03T18:23:45.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:45.754+0000 7fa35ca00700 1 -- 172.21.15.33:0/3599663903 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fa33c0b2900 msgr2=0x7fa33c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:23:45.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:45.754+0000 7fa35ca00700 1 --2- 172.21.15.33:0/3599663903 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fa33c0b2900 0x7fa33c0b4dc0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa34800b5c0 tx=0x7fa34801a040 comp rx=0 tx=0).stop 2024-05-03T18:23:45.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:45.754+0000 7fa35ca00700 1 -- 172.21.15.33:0/3599663903 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa35810b6b0 msgr2=0x7fa35819bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:23:45.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:45.754+0000 7fa35ca00700 1 --2- 172.21.15.33:0/3599663903 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa35810b6b0 0x7fa35819bae0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa350007ee0 tx=0x7fa35000d3b0 comp rx=0 tx=0).stop 2024-05-03T18:23:45.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:45.754+0000 7fa35ca00700 1 -- 172.21.15.33:0/3599663903 shutdown_connections 2024-05-03T18:23:45.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:45.754+0000 7fa35ca00700 1 --2- 172.21.15.33:0/3599663903 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa35810b6b0 0x7fa35819bae0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:45.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:45.754+0000 7fa35ca00700 1 --2- 172.21.15.33:0/3599663903 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3581056b0 0x7fa3581a1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:45.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:45.754+0000 7fa35ca00700 1 --2- 172.21.15.33:0/3599663903 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fa33c0b2900 0x7fa33c0b4dc0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:45.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:45.754+0000 7fa35ca00700 1 --2- 172.21.15.33:0/3599663903 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa358106070 0x7fa3581a1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:45.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:45.754+0000 7fa35ca00700 1 -- 172.21.15.33:0/3599663903 >> 172.21.15.33:0/3599663903 conn(0x7fa3581011a0 msgr2=0x7fa358101a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:23:45.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:45.755+0000 7fa35ca00700 1 -- 172.21.15.33:0/3599663903 shutdown_connections 2024-05-03T18:23:45.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:45.755+0000 7fa35ca00700 1 -- 172.21.15.33:0/3599663903 wait complete. 2024-05-03T18:23:45.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:23:45 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/3599663903' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-03T18:23:46.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:23:45 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/3599663903' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-03T18:23:46.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:23:45 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/3599663903' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-03T18:23:46.568 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-05-03T18:23:47.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:23:46 smithi099 ceph-mon[31658]: pgmap v420: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:23:47.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:23:46 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/3599663903' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-03T18:23:47.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:23:46 smithi099 ceph-mon[31658]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-03T18:23:47.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:23:46 smithi099 ceph-mon[31658]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-05-03T18:23:47.071 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:23:47.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:23:46 smithi150 ceph-mon[32492]: pgmap v420: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:23:47.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:23:46 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/3599663903' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-03T18:23:47.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:23:46 smithi150 ceph-mon[32492]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-03T18:23:47.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:23:46 smithi150 ceph-mon[32492]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-05-03T18:23:47.211 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:23:46 smithi033 ceph-mon[28330]: pgmap v420: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:23:47.212 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:23:46 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/3599663903' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-03T18:23:47.212 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:23:46 smithi033 ceph-mon[28330]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-03T18:23:47.212 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:23:46 smithi033 ceph-mon[28330]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-05-03T18:23:49.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:23:48 smithi099 ceph-mon[31658]: pgmap v421: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s rd, 0 op/s 2024-05-03T18:23:49.095 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:23:48 smithi033 ceph-mon[28330]: pgmap v421: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s rd, 0 op/s 2024-05-03T18:23:49.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:23:48 smithi150 ceph-mon[32492]: pgmap v421: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s rd, 0 op/s 2024-05-03T18:23:49.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:49.435+0000 7f962d200700 1 -- 172.21.15.33:0/3955122760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9628109510 msgr2=0x7f96281098f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:23:49.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:49.435+0000 7f962d200700 1 --2- 172.21.15.33:0/3955122760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9628109510 0x7f96281098f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f9618009990 tx=0x7f9618009ca0 comp rx=0 tx=0).stop 2024-05-03T18:23:49.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:49.435+0000 7f962d200700 1 -- 172.21.15.33:0/3955122760 shutdown_connections 2024-05-03T18:23:49.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:49.435+0000 7f962d200700 1 --2- 172.21.15.33:0/3955122760 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f962806bb40 0x7f9628110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:49.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:49.435+0000 7f962d200700 1 --2- 172.21.15.33:0/3955122760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9628109510 0x7f96281098f0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:49.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:49.435+0000 7f962d200700 1 --2- 172.21.15.33:0/3955122760 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f962806b180 0x7f962806b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:49.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:49.435+0000 7f962d200700 1 -- 172.21.15.33:0/3955122760 >> 172.21.15.33:0/3955122760 conn(0x7f9628079350 msgr2=0x7f9628079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:23:49.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:49.435+0000 7f962d200700 1 -- 172.21.15.33:0/3955122760 shutdown_connections 2024-05-03T18:23:49.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:49.435+0000 7f962d200700 1 -- 172.21.15.33:0/3955122760 wait complete. 2024-05-03T18:23:49.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:49.437+0000 7f962d200700 1 Processor -- start 2024-05-03T18:23:49.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:49.437+0000 7f962d200700 1 -- start start 2024-05-03T18:23:49.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:49.437+0000 7f962d200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f962806b180 0x7f9628102470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:23:49.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:49.438+0000 7f962d200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f962806bb40 0x7f96281029b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:23:49.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:49.438+0000 7f962d200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9628109510 0x7f9628102ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:23:49.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:49.438+0000 7f962d200700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9628113520 con 0x7f962806b180 2024-05-03T18:23:49.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:49.438+0000 7f962d200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f96281133a0 con 0x7f9628109510 2024-05-03T18:23:49.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:49.438+0000 7f962d200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f96281136a0 con 0x7f962806bb40 2024-05-03T18:23:49.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:49.438+0000 7f9625e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f962806b180 0x7f9628102470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:23:49.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:49.438+0000 7f9625e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f962806b180 0x7f9628102470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60094/0 (socket says 172.21.15.33:60094) 2024-05-03T18:23:49.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:49.438+0000 7f9625e00700 1 -- 172.21.15.33:0/730412958 learned_addr learned my addr 172.21.15.33:0/730412958 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:23:49.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:49.438+0000 7f9625400700 1 --2- 172.21.15.33:0/730412958 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f962806bb40 0x7f96281029b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:23:49.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:49.438+0000 7f9626800700 1 --2- 172.21.15.33:0/730412958 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9628109510 0x7f9628102ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:23:49.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:49.439+0000 7f9625e00700 1 -- 172.21.15.33:0/730412958 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f962806bb40 msgr2=0x7f96281029b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:23:49.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:49.439+0000 7f9625e00700 1 --2- 172.21.15.33:0/730412958 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f962806bb40 0x7f96281029b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:49.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:49.439+0000 7f9625e00700 1 -- 172.21.15.33:0/730412958 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9628109510 msgr2=0x7f9628102ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:23:49.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:49.439+0000 7f9625e00700 1 --2- 172.21.15.33:0/730412958 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9628109510 0x7f9628102ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:49.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:49.439+0000 7f9625e00700 1 -- 172.21.15.33:0/730412958 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96281a8000 con 0x7f962806b180 2024-05-03T18:23:49.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:49.439+0000 7f9625e00700 1 --2- 172.21.15.33:0/730412958 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f962806b180 0x7f9628102470 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9618000c00 tx=0x7f96180036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:23:49.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:49.439+0000 7f961ea00700 1 -- 172.21.15.33:0/730412958 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9618027070 con 0x7f962806b180 2024-05-03T18:23:49.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:49.439+0000 7f961ea00700 1 -- 172.21.15.33:0/730412958 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f961800fc80 con 0x7f962806b180 2024-05-03T18:23:49.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:49.439+0000 7f962d200700 1 -- 172.21.15.33:0/730412958 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96281a8290 con 0x7f962806b180 2024-05-03T18:23:49.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:49.439+0000 7f961ea00700 1 -- 172.21.15.33:0/730412958 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9618022d60 con 0x7f962806b180 2024-05-03T18:23:49.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:49.440+0000 7f962d200700 1 -- 172.21.15.33:0/730412958 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96281a86e0 con 0x7f962806b180 2024-05-03T18:23:49.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:49.441+0000 7f962d200700 1 -- 172.21.15.33:0/730412958 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f962810b9f0 con 0x7f962806b180 2024-05-03T18:23:49.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:49.443+0000 7f961ea00700 1 -- 172.21.15.33:0/730412958 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f961802a070 con 0x7f962806b180 2024-05-03T18:23:49.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:49.444+0000 7f961ea00700 1 --2- 172.21.15.33:0/730412958 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f960c0b2880 0x7f960c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:23:49.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:49.445+0000 7f961ea00700 1 -- 172.21.15.33:0/730412958 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9071+0+0 (secure 0 0 0) 0x7f96180e34c0 con 0x7f962806b180 2024-05-03T18:23:49.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:49.445+0000 7f9625400700 1 --2- 172.21.15.33:0/730412958 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f960c0b2880 0x7f960c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:23:49.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:49.446+0000 7f9625400700 1 --2- 172.21.15.33:0/730412958 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f960c0b2880 0x7f960c0b4d40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9614005fd0 tx=0x7f961400a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:23:49.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:49.450+0000 7f961ea00700 1 -- 172.21.15.33:0/730412958 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f9618013070 con 0x7f962806b180 2024-05-03T18:23:49.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:49.756+0000 7f962d200700 1 -- 172.21.15.33:0/730412958 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f962806a190 con 0x7f962806b180 2024-05-03T18:23:49.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:49.771+0000 7f961ea00700 1 -- 172.21.15.33:0/730412958 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v21) ==== 115+0+0 (secure 0 0 0) 0x7f96180aaed0 con 0x7f962806b180 2024-05-03T18:23:49.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:49.774+0000 7f962d200700 1 -- 172.21.15.33:0/730412958 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f960c0b2880 msgr2=0x7f960c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:23:49.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:49.774+0000 7f962d200700 1 --2- 172.21.15.33:0/730412958 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f960c0b2880 0x7f960c0b4d40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9614005fd0 tx=0x7f961400a560 comp rx=0 tx=0).stop 2024-05-03T18:23:49.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:49.774+0000 7f962d200700 1 -- 172.21.15.33:0/730412958 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f962806b180 msgr2=0x7f9628102470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:23:49.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:49.774+0000 7f962d200700 1 --2- 172.21.15.33:0/730412958 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f962806b180 0x7f9628102470 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9618000c00 tx=0x7f96180036f0 comp rx=0 tx=0).stop 2024-05-03T18:23:49.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:49.774+0000 7f962d200700 1 -- 172.21.15.33:0/730412958 shutdown_connections 2024-05-03T18:23:49.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:49.774+0000 7f962d200700 1 --2- 172.21.15.33:0/730412958 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f960c0b2880 0x7f960c0b4d40 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:49.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:49.774+0000 7f962d200700 1 --2- 172.21.15.33:0/730412958 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9628109510 0x7f9628102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:49.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:49.774+0000 7f962d200700 1 --2- 172.21.15.33:0/730412958 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f962806b180 0x7f9628102470 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:49.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:49.774+0000 7f962d200700 1 --2- 172.21.15.33:0/730412958 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f962806bb40 0x7f96281029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:49.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:49.774+0000 7f962d200700 1 -- 172.21.15.33:0/730412958 >> 172.21.15.33:0/730412958 conn(0x7f9628079350 msgr2=0x7f9628101130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:23:49.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:49.774+0000 7f962d200700 1 -- 172.21.15.33:0/730412958 shutdown_connections 2024-05-03T18:23:49.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:49.774+0000 7f962d200700 1 -- 172.21.15.33:0/730412958 wait complete. 2024-05-03T18:23:50.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:23:49 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/730412958' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-03T18:23:50.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:23:49 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/730412958' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-03T18:23:50.207 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:23:49 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/730412958' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-03T18:23:50.623 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--size 25000000000'} subvols for filesystem 'cephfs' 2024-05-03T18:23:50.623 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-05-03T18:23:50.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:23:50 smithi033 ceph-mon[28330]: pgmap v422: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 0 op/s 2024-05-03T18:23:50.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:23:50 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/730412958' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-03T18:23:50.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:23:50 smithi033 ceph-mon[28330]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-05-03T18:23:51.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:23:50 smithi099 ceph-mon[31658]: pgmap v422: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 0 op/s 2024-05-03T18:23:51.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:23:50 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/730412958' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-03T18:23:51.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:23:50 smithi099 ceph-mon[31658]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-05-03T18:23:51.064 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:23:51.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:23:50 smithi150 ceph-mon[32492]: pgmap v422: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 0 op/s 2024-05-03T18:23:51.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:23:50 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/730412958' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-03T18:23:51.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:23:50 smithi150 ceph-mon[32492]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-05-03T18:23:53.042 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:23:52 smithi033 ceph-mon[28330]: pgmap v423: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 0 op/s 2024-05-03T18:23:53.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:23:52 smithi099 ceph-mon[31658]: pgmap v423: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 0 op/s 2024-05-03T18:23:53.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:23:52 smithi150 ceph-mon[32492]: pgmap v423: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 0 op/s 2024-05-03T18:23:53.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:53.397+0000 7f1703e00700 1 -- 172.21.15.33:0/3139295532 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17041055d0 msgr2=0x7f1704105a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:23:53.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:53.397+0000 7f1703e00700 1 --2- 172.21.15.33:0/3139295532 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17041055d0 0x7f1704105a50 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f16ec009990 tx=0x7f16ec009ca0 comp rx=0 tx=0).stop 2024-05-03T18:23:53.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:53.398+0000 7f1703e00700 1 -- 172.21.15.33:0/3139295532 shutdown_connections 2024-05-03T18:23:53.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:53.398+0000 7f1703e00700 1 --2- 172.21.15.33:0/3139295532 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1704105f90 0x7f17041124a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:53.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:53.398+0000 7f1703e00700 1 --2- 172.21.15.33:0/3139295532 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f170410b660 0x7f170410ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:53.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:53.398+0000 7f1703e00700 1 --2- 172.21.15.33:0/3139295532 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17041055d0 0x7f1704105a50 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:53.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:53.398+0000 7f1703e00700 1 -- 172.21.15.33:0/3139295532 >> 172.21.15.33:0/3139295532 conn(0x7f17041010c0 msgr2=0x7f17041034e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:23:53.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:53.398+0000 7f1703e00700 1 -- 172.21.15.33:0/3139295532 shutdown_connections 2024-05-03T18:23:53.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:53.398+0000 7f1703e00700 1 -- 172.21.15.33:0/3139295532 wait complete. 2024-05-03T18:23:53.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:53.399+0000 7f1703e00700 1 Processor -- start 2024-05-03T18:23:53.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:53.399+0000 7f1703e00700 1 -- start start 2024-05-03T18:23:53.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:53.399+0000 7f1703e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f17041055d0 0x7f17041a1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:23:53.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:53.399+0000 7f1703e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1704105f90 0x7f17041a1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:23:53.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:53.399+0000 7f1703e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f170410b660 0x7f170419b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:23:53.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:53.400+0000 7f1703e00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f17041156d0 con 0x7f1704105f90 2024-05-03T18:23:53.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:53.400+0000 7f1703e00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1704115550 con 0x7f17041055d0 2024-05-03T18:23:53.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:53.400+0000 7f1703e00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f1704115850 con 0x7f170410b660 2024-05-03T18:23:53.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:53.400+0000 7f16fbe00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1704105f90 0x7f17041a1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:23:53.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:53.400+0000 7f16fbe00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1704105f90 0x7f17041a1c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60116/0 (socket says 172.21.15.33:60116) 2024-05-03T18:23:53.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:53.400+0000 7f16fbe00700 1 -- 172.21.15.33:0/3555099657 learned_addr learned my addr 172.21.15.33:0/3555099657 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:23:53.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:53.400+0000 7f1702a00700 1 --2- 172.21.15.33:0/3555099657 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f17041055d0 0x7f17041a1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:23:53.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:53.400+0000 7f1703400700 1 --2- 172.21.15.33:0/3555099657 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f170410b660 0x7f170419b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:23:53.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:53.400+0000 7f16fbe00700 1 -- 172.21.15.33:0/3555099657 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f170410b660 msgr2=0x7f170419b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:23:53.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:53.400+0000 7f16fbe00700 1 --2- 172.21.15.33:0/3555099657 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f170410b660 0x7f170419b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:53.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:53.400+0000 7f16fbe00700 1 -- 172.21.15.33:0/3555099657 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f17041055d0 msgr2=0x7f17041a1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:23:53.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:53.400+0000 7f16fbe00700 1 --2- 172.21.15.33:0/3555099657 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f17041055d0 0x7f17041a1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:53.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:53.400+0000 7f16fbe00700 1 -- 172.21.15.33:0/3555099657 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f170419c070 con 0x7f1704105f90 2024-05-03T18:23:53.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:53.400+0000 7f1703400700 1 --2- 172.21.15.33:0/3555099657 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f170410b660 0x7f170419b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:23:53.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:53.401+0000 7f16fbe00700 1 --2- 172.21.15.33:0/3555099657 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1704105f90 0x7f17041a1c90 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f16ec000c00 tx=0x7f16ec0036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:23:53.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:53.401+0000 7f16fb400700 1 -- 172.21.15.33:0/3555099657 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16ec027070 con 0x7f1704105f90 2024-05-03T18:23:53.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:53.401+0000 7f1703e00700 1 -- 172.21.15.33:0/3555099657 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f170419c300 con 0x7f1704105f90 2024-05-03T18:23:53.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:53.401+0000 7f16fb400700 1 -- 172.21.15.33:0/3555099657 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f16ec00fc80 con 0x7f1704105f90 2024-05-03T18:23:53.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:53.401+0000 7f16fb400700 1 -- 172.21.15.33:0/3555099657 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16ec022d60 con 0x7f1704105f90 2024-05-03T18:23:53.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:53.401+0000 7f1703e00700 1 -- 172.21.15.33:0/3555099657 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17041a8620 con 0x7f1704105f90 2024-05-03T18:23:53.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:53.402+0000 7f1703e00700 1 -- 172.21.15.33:0/3555099657 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f170406a190 con 0x7f1704105f90 2024-05-03T18:23:53.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:53.405+0000 7f16fb400700 1 -- 172.21.15.33:0/3555099657 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f16ec02a070 con 0x7f1704105f90 2024-05-03T18:23:53.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:53.406+0000 7f16fb400700 1 --2- 172.21.15.33:0/3555099657 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f16e80b2880 0x7f16e80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:23:53.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:53.407+0000 7f16fb400700 1 -- 172.21.15.33:0/3555099657 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9071+0+0 (secure 0 0 0) 0x7f16ec0e36b0 con 0x7f1704105f90 2024-05-03T18:23:53.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:53.407+0000 7f1702a00700 1 --2- 172.21.15.33:0/3555099657 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f16e80b2880 0x7f16e80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:23:53.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:53.408+0000 7f1702a00700 1 --2- 172.21.15.33:0/3555099657 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f16e80b2880 0x7f16e80b4d40 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f170419ca50 tx=0x7f16f400a5e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:23:53.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:53.409+0000 7f16fb400700 1 -- 172.21.15.33:0/3555099657 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f16ec031080 con 0x7f1704105f90 2024-05-03T18:23:53.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:53.687+0000 7f1703e00700 1 -- 172.21.15.33:0/3555099657 --> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f170419ce40 con 0x7f16e80b2880 2024-05-03T18:23:53.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:53.739+0000 7f16fb400700 1 -- 172.21.15.33:0/3555099657 <== mgr.14152 v2:172.21.15.33:6800/3642895501 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f170419ce40 con 0x7f16e80b2880 2024-05-03T18:23:53.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:53.742+0000 7f1703e00700 1 -- 172.21.15.33:0/3555099657 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f16e80b2880 msgr2=0x7f16e80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:23:53.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:53.742+0000 7f1703e00700 1 --2- 172.21.15.33:0/3555099657 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f16e80b2880 0x7f16e80b4d40 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f170419ca50 tx=0x7f16f400a5e0 comp rx=0 tx=0).stop 2024-05-03T18:23:53.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:53.742+0000 7f1703e00700 1 -- 172.21.15.33:0/3555099657 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1704105f90 msgr2=0x7f17041a1c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:23:53.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:53.742+0000 7f1703e00700 1 --2- 172.21.15.33:0/3555099657 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1704105f90 0x7f17041a1c90 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f16ec000c00 tx=0x7f16ec0036f0 comp rx=0 tx=0).stop 2024-05-03T18:23:53.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:53.742+0000 7f1703e00700 1 -- 172.21.15.33:0/3555099657 shutdown_connections 2024-05-03T18:23:53.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:53.742+0000 7f1703e00700 1 --2- 172.21.15.33:0/3555099657 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f170410b660 0x7f170419b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:53.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:53.742+0000 7f1703e00700 1 --2- 172.21.15.33:0/3555099657 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f16e80b2880 0x7f16e80b4d40 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:53.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:53.742+0000 7f1703e00700 1 --2- 172.21.15.33:0/3555099657 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f17041055d0 0x7f17041a1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:53.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:53.742+0000 7f1703e00700 1 --2- 172.21.15.33:0/3555099657 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1704105f90 0x7f17041a1c90 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:53.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:53.742+0000 7f1703e00700 1 -- 172.21.15.33:0/3555099657 >> 172.21.15.33:0/3555099657 conn(0x7f17041010c0 msgr2=0x7f1704102810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:23:53.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:53.743+0000 7f1703e00700 1 -- 172.21.15.33:0/3555099657 shutdown_connections 2024-05-03T18:23:53.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:53.743+0000 7f1703e00700 1 -- 172.21.15.33:0/3555099657 wait complete. 2024-05-03T18:23:53.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:23:53 smithi033 ceph-mon[28330]: pgmap v424: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 1 op/s 2024-05-03T18:23:53.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:23:53 smithi033 ceph-mon[28330]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:23:53.985 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:23:53 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:23:53.699+0000 7f2bc4c00700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-03T18:23:53.985 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:23:53 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:23:53.699+0000 7f2bc4c00700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-03T18:23:53.985 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:23:53 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:23:53.699+0000 7f2bc4c00700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-03T18:23:53.985 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:23:53 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:23:53.699+0000 7f2bc4c00700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-03T18:23:53.985 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:23:53 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:23:53.699+0000 7f2bc4c00700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-03T18:23:54.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:23:53 smithi150 ceph-mon[32492]: pgmap v424: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 1 op/s 2024-05-03T18:23:54.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:23:53 smithi150 ceph-mon[32492]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:23:54.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:23:53 smithi099 ceph-mon[31658]: pgmap v424: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 1 op/s 2024-05-03T18:23:54.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:23:53 smithi099 ceph-mon[31658]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:23:54.553 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-05-03T18:23:55.018 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:23:56.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:23:56 smithi033 ceph-mon[28330]: pgmap v425: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 1 op/s 2024-05-03T18:23:56.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:23:56 smithi099 ceph-mon[31658]: pgmap v425: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 1 op/s 2024-05-03T18:23:56.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:23:56 smithi150 ceph-mon[32492]: pgmap v425: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 1 op/s 2024-05-03T18:23:57.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:57.306+0000 7fb5e3400700 1 -- 172.21.15.33:0/490424236 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5dc109500 msgr2=0x7fb5dc1098e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:23:57.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:57.306+0000 7fb5e3400700 1 --2- 172.21.15.33:0/490424236 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5dc109500 0x7fb5dc1098e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb5d0009990 tx=0x7fb5d0009ca0 comp rx=0 tx=0).stop 2024-05-03T18:23:57.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:57.306+0000 7fb5e3400700 1 -- 172.21.15.33:0/490424236 shutdown_connections 2024-05-03T18:23:57.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:57.306+0000 7fb5e3400700 1 --2- 172.21.15.33:0/490424236 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb5dc06c3d0 0x7fb5dc110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:57.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:57.306+0000 7fb5e3400700 1 --2- 172.21.15.33:0/490424236 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5dc109500 0x7fb5dc1098e0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:57.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:57.306+0000 7fb5e3400700 1 --2- 172.21.15.33:0/490424236 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5dc06ba10 0x7fb5dc06be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:57.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:57.306+0000 7fb5e3400700 1 -- 172.21.15.33:0/490424236 >> 172.21.15.33:0/490424236 conn(0x7fb5dc079310 msgr2=0x7fb5dc079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:23:57.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:57.306+0000 7fb5e3400700 1 -- 172.21.15.33:0/490424236 shutdown_connections 2024-05-03T18:23:57.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:57.306+0000 7fb5e3400700 1 -- 172.21.15.33:0/490424236 wait complete. 2024-05-03T18:23:57.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:57.308+0000 7fb5e3400700 1 Processor -- start 2024-05-03T18:23:57.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:57.308+0000 7fb5e3400700 1 -- start start 2024-05-03T18:23:57.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:57.309+0000 7fb5e3400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5dc06ba10 0x7fb5dc1a18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:23:57.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:57.309+0000 7fb5e3400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb5dc06c3d0 0x7fb5dc1a1e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:23:57.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:57.309+0000 7fb5e3400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5dc109500 0x7fb5dc19ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:23:57.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:57.309+0000 7fb5e3400700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb5dc1134e0 con 0x7fb5dc06ba10 2024-05-03T18:23:57.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:57.309+0000 7fb5e3400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb5dc113360 con 0x7fb5dc109500 2024-05-03T18:23:57.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:57.309+0000 7fb5e3400700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb5dc113660 con 0x7fb5dc06c3d0 2024-05-03T18:23:57.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:57.309+0000 7fb5e0a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5dc06ba10 0x7fb5dc1a18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:23:57.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:57.309+0000 7fb5e0a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5dc06ba10 0x7fb5dc1a18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54930/0 (socket says 172.21.15.33:54930) 2024-05-03T18:23:57.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:57.309+0000 7fb5e0a00700 1 -- 172.21.15.33:0/2741388519 learned_addr learned my addr 172.21.15.33:0/2741388519 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:23:57.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:57.309+0000 7fb5dbe00700 1 --2- 172.21.15.33:0/2741388519 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb5dc06c3d0 0x7fb5dc1a1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:23:57.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:57.309+0000 7fb5e1400700 1 --2- 172.21.15.33:0/2741388519 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5dc109500 0x7fb5dc19ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:23:57.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:57.309+0000 7fb5e0a00700 1 -- 172.21.15.33:0/2741388519 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb5dc06c3d0 msgr2=0x7fb5dc1a1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:23:57.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:57.309+0000 7fb5e0a00700 1 --2- 172.21.15.33:0/2741388519 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb5dc06c3d0 0x7fb5dc1a1e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:57.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:57.309+0000 7fb5e0a00700 1 -- 172.21.15.33:0/2741388519 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5dc109500 msgr2=0x7fb5dc19ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:23:57.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:57.309+0000 7fb5e0a00700 1 --2- 172.21.15.33:0/2741388519 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5dc109500 0x7fb5dc19ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:57.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:57.309+0000 7fb5e0a00700 1 -- 172.21.15.33:0/2741388519 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5dc19c1e0 con 0x7fb5dc06ba10 2024-05-03T18:23:57.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:57.309+0000 7fb5dbe00700 1 --2- 172.21.15.33:0/2741388519 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb5dc06c3d0 0x7fb5dc1a1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:23:57.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:57.309+0000 7fb5e1400700 1 --2- 172.21.15.33:0/2741388519 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5dc109500 0x7fb5dc19ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:23:57.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:57.310+0000 7fb5e0a00700 1 --2- 172.21.15.33:0/2741388519 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5dc06ba10 0x7fb5dc1a18c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb5d00057c0 tx=0x7fb5d0009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:23:57.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:57.310+0000 7fb5d9600700 1 -- 172.21.15.33:0/2741388519 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5d0027070 con 0x7fb5dc06ba10 2024-05-03T18:23:57.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:57.310+0000 7fb5d9600700 1 -- 172.21.15.33:0/2741388519 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb5d000fda0 con 0x7fb5dc06ba10 2024-05-03T18:23:57.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:57.310+0000 7fb5d9600700 1 -- 172.21.15.33:0/2741388519 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5d0022ca0 con 0x7fb5dc06ba10 2024-05-03T18:23:57.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:57.310+0000 7fb5e3400700 1 -- 172.21.15.33:0/2741388519 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5dc19c470 con 0x7fb5dc06ba10 2024-05-03T18:23:57.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:57.310+0000 7fb5e3400700 1 -- 172.21.15.33:0/2741388519 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5dc1a8790 con 0x7fb5dc06ba10 2024-05-03T18:23:57.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:57.312+0000 7fb5e3400700 1 -- 172.21.15.33:0/2741388519 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5dc06a190 con 0x7fb5dc06ba10 2024-05-03T18:23:57.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:57.314+0000 7fb5d9600700 1 -- 172.21.15.33:0/2741388519 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fb5d002a070 con 0x7fb5dc06ba10 2024-05-03T18:23:57.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:57.315+0000 7fb5d9600700 1 --2- 172.21.15.33:0/2741388519 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fb5c40b2910 0x7fb5c40b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:23:57.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:57.316+0000 7fb5d9600700 1 -- 172.21.15.33:0/2741388519 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9071+0+0 (secure 0 0 0) 0x7fb5d00e3760 con 0x7fb5dc06ba10 2024-05-03T18:23:57.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:57.316+0000 7fb5dbe00700 1 --2- 172.21.15.33:0/2741388519 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fb5c40b2910 0x7fb5c40b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:23:57.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:57.317+0000 7fb5dbe00700 1 --2- 172.21.15.33:0/2741388519 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fb5c40b2910 0x7fb5c40b4dd0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb5c8005fd0 tx=0x7fb5c8005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:23:57.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:57.319+0000 7fb5d9600700 1 -- 172.21.15.33:0/2741388519 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fb5d0031080 con 0x7fb5dc06ba10 2024-05-03T18:23:57.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:57.599+0000 7fb5e3400700 1 -- 172.21.15.33:0/2741388519 --> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fb5dc10ed80 con 0x7fb5c40b2910 2024-05-03T18:23:57.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:57.664+0000 7fb5d9600700 1 -- 172.21.15.33:0/2741388519 <== mgr.14152 v2:172.21.15.33:6800/3642895501 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb5dc10ed80 con 0x7fb5c40b2910 2024-05-03T18:23:57.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:57.667+0000 7fb5e3400700 1 -- 172.21.15.33:0/2741388519 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fb5c40b2910 msgr2=0x7fb5c40b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:23:57.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:57.667+0000 7fb5e3400700 1 --2- 172.21.15.33:0/2741388519 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fb5c40b2910 0x7fb5c40b4dd0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb5c8005fd0 tx=0x7fb5c8005ee0 comp rx=0 tx=0).stop 2024-05-03T18:23:57.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:57.667+0000 7fb5e3400700 1 -- 172.21.15.33:0/2741388519 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5dc06ba10 msgr2=0x7fb5dc1a18c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:23:57.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:57.667+0000 7fb5e3400700 1 --2- 172.21.15.33:0/2741388519 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5dc06ba10 0x7fb5dc1a18c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb5d00057c0 tx=0x7fb5d0009f90 comp rx=0 tx=0).stop 2024-05-03T18:23:57.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:57.667+0000 7fb5e3400700 1 -- 172.21.15.33:0/2741388519 shutdown_connections 2024-05-03T18:23:57.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:57.667+0000 7fb5e3400700 1 --2- 172.21.15.33:0/2741388519 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fb5c40b2910 0x7fb5c40b4dd0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:57.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:57.667+0000 7fb5e3400700 1 --2- 172.21.15.33:0/2741388519 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5dc109500 0x7fb5dc19ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:57.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:57.667+0000 7fb5e3400700 1 --2- 172.21.15.33:0/2741388519 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5dc06ba10 0x7fb5dc1a18c0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:57.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:57.667+0000 7fb5e3400700 1 --2- 172.21.15.33:0/2741388519 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb5dc06c3d0 0x7fb5dc1a1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:23:57.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:57.667+0000 7fb5e3400700 1 -- 172.21.15.33:0/2741388519 >> 172.21.15.33:0/2741388519 conn(0x7fb5dc079310 msgr2=0x7fb5dc10e670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:23:57.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:57.667+0000 7fb5e3400700 1 -- 172.21.15.33:0/2741388519 shutdown_connections 2024-05-03T18:23:57.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:23:57.667+0000 7fb5e3400700 1 -- 172.21.15.33:0/2741388519 wait complete. 2024-05-03T18:23:57.687 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:23:57 smithi033 ceph-mon[28330]: mgrmap e17: x(active, since 11m), standbys: z, y 2024-05-03T18:23:57.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:23:57 smithi099 ceph-mon[31658]: mgrmap e17: x(active, since 11m), standbys: z, y 2024-05-03T18:23:57.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:23:57 smithi150 ceph-mon[32492]: mgrmap e17: x(active, since 11m), standbys: z, y 2024-05-03T18:23:58.531 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-05-03T18:23:58.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:23:58 smithi033 ceph-mon[28330]: pgmap v426: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 1 op/s 2024-05-03T18:23:58.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:23:58 smithi033 ceph-mon[28330]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:23:58.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:23:58 smithi099 ceph-mon[31658]: pgmap v426: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 1 op/s 2024-05-03T18:23:58.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:23:58 smithi099 ceph-mon[31658]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:23:58.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:23:58 smithi150 ceph-mon[32492]: pgmap v426: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 1 op/s 2024-05-03T18:23:58.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:23:58 smithi150 ceph-mon[32492]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:23:59.004 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:24:00.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:24:00 smithi033 ceph-mon[28330]: pgmap v427: 129 pgs: 129 active+clean; 601 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 2.2 KiB/s wr, 3 op/s 2024-05-03T18:24:00.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:24:00 smithi099 ceph-mon[31658]: pgmap v427: 129 pgs: 129 active+clean; 601 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 2.2 KiB/s wr, 3 op/s 2024-05-03T18:24:00.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:24:00 smithi150 ceph-mon[32492]: pgmap v427: 129 pgs: 129 active+clean; 601 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 2.2 KiB/s wr, 3 op/s 2024-05-03T18:24:01.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:01.325+0000 7faeeca00700 1 -- 172.21.15.33:0/118267505 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faee80756d0 msgr2=0x7faee8075ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:24:01.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:01.325+0000 7faeeca00700 1 --2- 172.21.15.33:0/118267505 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faee80756d0 0x7faee8075ab0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7faed8009990 tx=0x7faed8009ca0 comp rx=0 tx=0).stop 2024-05-03T18:24:01.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:01.325+0000 7faeeca00700 1 -- 172.21.15.33:0/118267505 shutdown_connections 2024-05-03T18:24:01.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:01.325+0000 7faeeca00700 1 --2- 172.21.15.33:0/118267505 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faee8114670 0x7faee8116a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:01.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:01.325+0000 7faeeca00700 1 --2- 172.21.15.33:0/118267505 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faee80756d0 0x7faee8075ab0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:01.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:01.325+0000 7faeeca00700 1 --2- 172.21.15.33:0/118267505 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faee8075ff0 0x7faee8114130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:01.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:01.325+0000 7faeeca00700 1 -- 172.21.15.33:0/118267505 >> 172.21.15.33:0/118267505 conn(0x7faee807a600 msgr2=0x7faee807aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:24:01.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:01.325+0000 7faeeca00700 1 -- 172.21.15.33:0/118267505 shutdown_connections 2024-05-03T18:24:01.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:01.325+0000 7faeeca00700 1 -- 172.21.15.33:0/118267505 wait complete. 2024-05-03T18:24:01.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:01.327+0000 7faeeca00700 1 Processor -- start 2024-05-03T18:24:01.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:01.328+0000 7faeeca00700 1 -- start start 2024-05-03T18:24:01.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:01.328+0000 7faeeca00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faee8075ff0 0x7faee81a0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:24:01.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:01.328+0000 7faeeca00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faee8114670 0x7faee81a0870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:24:01.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:01.328+0000 7faeeca00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faee81a4c00 0x7faee81a5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:24:01.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:01.328+0000 7faeeca00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7faee8119b50 con 0x7faee81a4c00 2024-05-03T18:24:01.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:01.328+0000 7faeeca00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7faee81199d0 con 0x7faee8075ff0 2024-05-03T18:24:01.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:01.328+0000 7faeeca00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7faee8119cd0 con 0x7faee8114670 2024-05-03T18:24:01.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:01.329+0000 7faee6800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faee81a4c00 0x7faee81a5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:24:01.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:01.329+0000 7faee6800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faee81a4c00 0x7faee81a5080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54958/0 (socket says 172.21.15.33:54958) 2024-05-03T18:24:01.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:01.329+0000 7faee6800700 1 -- 172.21.15.33:0/3001878316 learned_addr learned my addr 172.21.15.33:0/3001878316 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:24:01.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:01.329+0000 7faee5e00700 1 --2- 172.21.15.33:0/3001878316 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faee8075ff0 0x7faee81a0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:24:01.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:01.329+0000 7faedd400700 1 --2- 172.21.15.33:0/3001878316 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faee8114670 0x7faee81a0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:24:01.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:01.329+0000 7faee6800700 1 -- 172.21.15.33:0/3001878316 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faee8114670 msgr2=0x7faee81a0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:24:01.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:01.329+0000 7faee6800700 1 --2- 172.21.15.33:0/3001878316 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faee8114670 0x7faee81a0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:01.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:01.329+0000 7faee6800700 1 -- 172.21.15.33:0/3001878316 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faee8075ff0 msgr2=0x7faee81a0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:24:01.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:01.329+0000 7faee6800700 1 --2- 172.21.15.33:0/3001878316 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faee8075ff0 0x7faee81a0330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:01.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:01.329+0000 7faee6800700 1 -- 172.21.15.33:0/3001878316 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faee81a5720 con 0x7faee81a4c00 2024-05-03T18:24:01.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:01.329+0000 7faee5e00700 1 --2- 172.21.15.33:0/3001878316 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faee8075ff0 0x7faee81a0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:24:01.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:01.330+0000 7faee6800700 1 --2- 172.21.15.33:0/3001878316 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faee81a4c00 0x7faee81a5080 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7faee0007d40 tx=0x7faee000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:24:01.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:01.330+0000 7faedf400700 1 -- 172.21.15.33:0/3001878316 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faee0016020 con 0x7faee81a4c00 2024-05-03T18:24:01.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:01.330+0000 7faedf400700 1 -- 172.21.15.33:0/3001878316 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faee000ad40 con 0x7faee81a4c00 2024-05-03T18:24:01.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:01.330+0000 7faeeca00700 1 -- 172.21.15.33:0/3001878316 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faee81a5a10 con 0x7faee81a4c00 2024-05-03T18:24:01.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:01.330+0000 7faedf400700 1 -- 172.21.15.33:0/3001878316 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faee00115e0 con 0x7faee81a4c00 2024-05-03T18:24:01.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:01.330+0000 7faeeca00700 1 -- 172.21.15.33:0/3001878316 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faee81ad1a0 con 0x7faee81a4c00 2024-05-03T18:24:01.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:01.331+0000 7faeeca00700 1 -- 172.21.15.33:0/3001878316 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faee806a190 con 0x7faee81a4c00 2024-05-03T18:24:01.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:01.336+0000 7faedf400700 1 -- 172.21.15.33:0/3001878316 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7faee0019020 con 0x7faee81a4c00 2024-05-03T18:24:01.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:01.337+0000 7faedf400700 1 --2- 172.21.15.33:0/3001878316 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7faecc0b2910 0x7faecc0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:24:01.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:01.337+0000 7faedf400700 1 -- 172.21.15.33:0/3001878316 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9071+0+0 (secure 0 0 0) 0x7faee00e29d0 con 0x7faee81a4c00 2024-05-03T18:24:01.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:01.338+0000 7faee5e00700 1 --2- 172.21.15.33:0/3001878316 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7faecc0b2910 0x7faecc0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:24:01.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:01.338+0000 7faee5e00700 1 --2- 172.21.15.33:0/3001878316 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7faecc0b2910 0x7faecc0b4dd0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7faed8000c00 tx=0x7faed801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:24:01.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:01.339+0000 7faedf400700 1 -- 172.21.15.33:0/3001878316 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7faee0020020 con 0x7faee81a4c00 2024-05-03T18:24:01.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:01.625+0000 7faeeca00700 1 -- 172.21.15.33:0/3001878316 --> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7faee8113d10 con 0x7faecc0b2910 2024-05-03T18:24:01.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:01.643+0000 7faedf400700 1 -- 172.21.15.33:0/3001878316 <== mgr.14152 v2:172.21.15.33:6800/3642895501 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7faee8113d10 con 0x7faecc0b2910 2024-05-03T18:24:01.644 INFO:teuthology.orchestra.run.smithi033.stdout:/volumes/qa/sv_0/70093236-67a4-4327-bd9b-9b6228a777c0 2024-05-03T18:24:01.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:01.647+0000 7faeeca00700 1 -- 172.21.15.33:0/3001878316 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7faecc0b2910 msgr2=0x7faecc0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:24:01.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:01.647+0000 7faeeca00700 1 --2- 172.21.15.33:0/3001878316 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7faecc0b2910 0x7faecc0b4dd0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7faed8000c00 tx=0x7faed801a040 comp rx=0 tx=0).stop 2024-05-03T18:24:01.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:01.647+0000 7faeeca00700 1 -- 172.21.15.33:0/3001878316 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faee81a4c00 msgr2=0x7faee81a5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:24:01.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:01.647+0000 7faeeca00700 1 --2- 172.21.15.33:0/3001878316 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faee81a4c00 0x7faee81a5080 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7faee0007d40 tx=0x7faee000a340 comp rx=0 tx=0).stop 2024-05-03T18:24:01.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:01.647+0000 7faeeca00700 1 -- 172.21.15.33:0/3001878316 shutdown_connections 2024-05-03T18:24:01.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:01.647+0000 7faeeca00700 1 --2- 172.21.15.33:0/3001878316 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faee81a4c00 0x7faee81a5080 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:01.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:01.647+0000 7faeeca00700 1 --2- 172.21.15.33:0/3001878316 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7faecc0b2910 0x7faecc0b4dd0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:01.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:01.647+0000 7faeeca00700 1 --2- 172.21.15.33:0/3001878316 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faee8075ff0 0x7faee81a0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:01.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:01.647+0000 7faeeca00700 1 --2- 172.21.15.33:0/3001878316 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faee8114670 0x7faee81a0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:01.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:01.647+0000 7faeeca00700 1 -- 172.21.15.33:0/3001878316 >> 172.21.15.33:0/3001878316 conn(0x7faee807a600 msgr2=0x7faee8113600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:24:01.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:01.647+0000 7faeeca00700 1 -- 172.21.15.33:0/3001878316 shutdown_connections 2024-05-03T18:24:01.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:01.648+0000 7faeeca00700 1 -- 172.21.15.33:0/3001878316 wait complete. 2024-05-03T18:24:02.510 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-05-03T18:24:02.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:24:02 smithi033 ceph-mon[28330]: pgmap v428: 129 pgs: 129 active+clean; 601 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2.2 KiB/s wr, 2 op/s 2024-05-03T18:24:02.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:24:02 smithi033 ceph-mon[28330]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:24:02.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:24:02 smithi099 ceph-mon[31658]: pgmap v428: 129 pgs: 129 active+clean; 601 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2.2 KiB/s wr, 2 op/s 2024-05-03T18:24:02.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:24:02 smithi099 ceph-mon[31658]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:24:02.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:24:02 smithi150 ceph-mon[32492]: pgmap v428: 129 pgs: 129 active+clean; 601 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2.2 KiB/s wr, 2 op/s 2024-05-03T18:24:02.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:24:02 smithi150 ceph-mon[32492]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:24:02.989 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:24:04.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:24:04 smithi033 ceph-mon[28330]: pgmap v429: 129 pgs: 129 active+clean; 607 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 2.9 KiB/s wr, 4 op/s 2024-05-03T18:24:04.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:24:04 smithi099 ceph-mon[31658]: pgmap v429: 129 pgs: 129 active+clean; 607 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 2.9 KiB/s wr, 4 op/s 2024-05-03T18:24:04.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:24:04 smithi150 ceph-mon[32492]: pgmap v429: 129 pgs: 129 active+clean; 607 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 2.9 KiB/s wr, 4 op/s 2024-05-03T18:24:05.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:05.402+0000 7f44a8800700 1 -- 172.21.15.33:0/3561447711 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44a00756d0 msgr2=0x7f44a0075ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:24:05.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:05.402+0000 7f44a8800700 1 --2- 172.21.15.33:0/3561447711 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44a00756d0 0x7f44a0075ab0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f4494009990 tx=0x7f4494009ca0 comp rx=0 tx=0).stop 2024-05-03T18:24:05.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:05.402+0000 7f44a8800700 1 -- 172.21.15.33:0/3561447711 shutdown_connections 2024-05-03T18:24:05.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:05.402+0000 7f44a8800700 1 --2- 172.21.15.33:0/3561447711 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44a010fea0 0x7f44a0112290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:05.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:05.402+0000 7f44a8800700 1 --2- 172.21.15.33:0/3561447711 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44a00756d0 0x7f44a0075ab0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:05.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:05.402+0000 7f44a8800700 1 --2- 172.21.15.33:0/3561447711 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44a0075ff0 0x7f44a010f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:05.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:05.402+0000 7f44a8800700 1 -- 172.21.15.33:0/3561447711 >> 172.21.15.33:0/3561447711 conn(0x7f44a007a600 msgr2=0x7f44a007aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:24:05.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:05.402+0000 7f44a8800700 1 -- 172.21.15.33:0/3561447711 shutdown_connections 2024-05-03T18:24:05.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:05.402+0000 7f44a8800700 1 -- 172.21.15.33:0/3561447711 wait complete. 2024-05-03T18:24:05.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:05.403+0000 7f44a8800700 1 Processor -- start 2024-05-03T18:24:05.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:05.404+0000 7f44a8800700 1 -- start start 2024-05-03T18:24:05.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:05.404+0000 7f44a8800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44a00756d0 0x7f44a01a18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:24:05.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:05.404+0000 7f44a8800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44a0075ff0 0x7f44a01a1df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:24:05.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:05.404+0000 7f44a8800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44a010fea0 0x7f44a019ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:24:05.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:05.404+0000 7f44a8800700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f44a0115730 con 0x7f44a0075ff0 2024-05-03T18:24:05.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:05.404+0000 7f44a8800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f44a01155b0 con 0x7f44a010fea0 2024-05-03T18:24:05.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:05.404+0000 7f44a8800700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f44a01158b0 con 0x7f44a00756d0 2024-05-03T18:24:05.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:05.405+0000 7f44a5400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44a0075ff0 0x7f44a01a1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:24:05.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:05.405+0000 7f44a5400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44a0075ff0 0x7f44a01a1df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54976/0 (socket says 172.21.15.33:54976) 2024-05-03T18:24:05.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:05.405+0000 7f44a5400700 1 -- 172.21.15.33:0/962057592 learned_addr learned my addr 172.21.15.33:0/962057592 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:24:05.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:05.405+0000 7f44a5e00700 1 --2- 172.21.15.33:0/962057592 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44a00756d0 0x7f44a01a18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:24:05.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:05.405+0000 7f44a6800700 1 --2- 172.21.15.33:0/962057592 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44a010fea0 0x7f44a019ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:24:05.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:05.405+0000 7f44a5400700 1 -- 172.21.15.33:0/962057592 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44a00756d0 msgr2=0x7f44a01a18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:24:05.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:05.405+0000 7f44a5400700 1 --2- 172.21.15.33:0/962057592 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44a00756d0 0x7f44a01a18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:05.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:05.405+0000 7f44a5400700 1 -- 172.21.15.33:0/962057592 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44a010fea0 msgr2=0x7f44a019ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:24:05.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:05.405+0000 7f44a5400700 1 --2- 172.21.15.33:0/962057592 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44a010fea0 0x7f44a019ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:05.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:05.405+0000 7f44a5400700 1 -- 172.21.15.33:0/962057592 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44a019c1d0 con 0x7f44a0075ff0 2024-05-03T18:24:05.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:05.405+0000 7f44a6800700 1 --2- 172.21.15.33:0/962057592 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44a010fea0 0x7f44a019ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:24:05.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:05.406+0000 7f44a5400700 1 --2- 172.21.15.33:0/962057592 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44a0075ff0 0x7f44a01a1df0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f449000c920 tx=0x7f449000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:24:05.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:05.406+0000 7f449aa00700 1 -- 172.21.15.33:0/962057592 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4490007b00 con 0x7f44a0075ff0 2024-05-03T18:24:05.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:05.406+0000 7f449aa00700 1 -- 172.21.15.33:0/962057592 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4490004cc0 con 0x7f44a0075ff0 2024-05-03T18:24:05.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:05.406+0000 7f44a8800700 1 -- 172.21.15.33:0/962057592 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44a019c4c0 con 0x7f44a0075ff0 2024-05-03T18:24:05.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:05.406+0000 7f449aa00700 1 -- 172.21.15.33:0/962057592 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44900054f0 con 0x7f44a0075ff0 2024-05-03T18:24:05.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:05.406+0000 7f44a8800700 1 -- 172.21.15.33:0/962057592 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44a01a8780 con 0x7f44a0075ff0 2024-05-03T18:24:05.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:05.408+0000 7f44a8800700 1 -- 172.21.15.33:0/962057592 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44a006a190 con 0x7f44a0075ff0 2024-05-03T18:24:05.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:05.410+0000 7f449aa00700 1 -- 172.21.15.33:0/962057592 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f4490007ca0 con 0x7f44a0075ff0 2024-05-03T18:24:05.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:05.412+0000 7f449aa00700 1 --2- 172.21.15.33:0/962057592 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f44880b2910 0x7f44880b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:24:05.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:05.412+0000 7f449aa00700 1 -- 172.21.15.33:0/962057592 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9071+0+0 (secure 0 0 0) 0x7f44900e2380 con 0x7f44a0075ff0 2024-05-03T18:24:05.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:05.412+0000 7f44a5e00700 1 --2- 172.21.15.33:0/962057592 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f44880b2910 0x7f44880b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:24:05.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:05.413+0000 7f44a5e00700 1 --2- 172.21.15.33:0/962057592 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f44880b2910 0x7f44880b4dd0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4494005850 tx=0x7f449401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:24:05.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:05.417+0000 7f449aa00700 1 -- 172.21.15.33:0/962057592 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f4490014030 con 0x7f44a0075ff0 2024-05-03T18:24:05.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:05.699+0000 7f44a8800700 1 -- 172.21.15.33:0/962057592 --> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f44a00638e0 con 0x7f44880b2910 2024-05-03T18:24:05.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:05.763+0000 7f449aa00700 1 -- 172.21.15.33:0/962057592 <== mgr.14152 v2:172.21.15.33:6800/3642895501 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f44a00638e0 con 0x7f44880b2910 2024-05-03T18:24:05.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:05.766+0000 7f44a8800700 1 -- 172.21.15.33:0/962057592 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f44880b2910 msgr2=0x7f44880b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:24:05.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:05.766+0000 7f44a8800700 1 --2- 172.21.15.33:0/962057592 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f44880b2910 0x7f44880b4dd0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4494005850 tx=0x7f449401a040 comp rx=0 tx=0).stop 2024-05-03T18:24:05.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:05.766+0000 7f44a8800700 1 -- 172.21.15.33:0/962057592 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44a0075ff0 msgr2=0x7f44a01a1df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:24:05.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:05.766+0000 7f44a8800700 1 --2- 172.21.15.33:0/962057592 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44a0075ff0 0x7f44a01a1df0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f449000c920 tx=0x7f449000cce0 comp rx=0 tx=0).stop 2024-05-03T18:24:05.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:05.766+0000 7f44a8800700 1 -- 172.21.15.33:0/962057592 shutdown_connections 2024-05-03T18:24:05.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:05.766+0000 7f44a8800700 1 --2- 172.21.15.33:0/962057592 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f44880b2910 0x7f44880b4dd0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:05.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:05.766+0000 7f44a8800700 1 --2- 172.21.15.33:0/962057592 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44a010fea0 0x7f44a019ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:05.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:05.766+0000 7f44a8800700 1 --2- 172.21.15.33:0/962057592 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44a00756d0 0x7f44a01a18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:05.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:05.767+0000 7f44a8800700 1 --2- 172.21.15.33:0/962057592 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44a0075ff0 0x7f44a01a1df0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:05.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:05.767+0000 7f44a8800700 1 -- 172.21.15.33:0/962057592 >> 172.21.15.33:0/962057592 conn(0x7f44a007a600 msgr2=0x7f44a01007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:24:05.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:05.767+0000 7f44a8800700 1 -- 172.21.15.33:0/962057592 shutdown_connections 2024-05-03T18:24:05.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:05.767+0000 7f44a8800700 1 -- 172.21.15.33:0/962057592 wait complete. 2024-05-03T18:24:06.645 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-05-03T18:24:06.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:24:06 smithi033 ceph-mon[28330]: pgmap v430: 129 pgs: 129 active+clean; 607 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.9 KiB/s wr, 3 op/s 2024-05-03T18:24:06.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:24:06 smithi033 ceph-mon[28330]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:24:06.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:24:06 smithi099 ceph-mon[31658]: pgmap v430: 129 pgs: 129 active+clean; 607 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.9 KiB/s wr, 3 op/s 2024-05-03T18:24:06.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:24:06 smithi099 ceph-mon[31658]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:24:06.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:24:06 smithi150 ceph-mon[32492]: pgmap v430: 129 pgs: 129 active+clean; 607 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.9 KiB/s wr, 3 op/s 2024-05-03T18:24:06.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:24:06 smithi150 ceph-mon[32492]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:24:07.107 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:24:08.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:24:08 smithi033 ceph-mon[28330]: pgmap v431: 129 pgs: 129 active+clean; 607 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.9 KiB/s wr, 3 op/s 2024-05-03T18:24:08.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:24:08 smithi099 ceph-mon[31658]: pgmap v431: 129 pgs: 129 active+clean; 607 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.9 KiB/s wr, 3 op/s 2024-05-03T18:24:08.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:24:08 smithi150 ceph-mon[32492]: pgmap v431: 129 pgs: 129 active+clean; 607 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.9 KiB/s wr, 3 op/s 2024-05-03T18:24:09.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:09.367+0000 7f4e3f400700 1 -- 172.21.15.33:0/744035501 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e380756d0 msgr2=0x7f4e38075ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:24:09.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:09.367+0000 7f4e3f400700 1 --2- 172.21.15.33:0/744035501 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e380756d0 0x7f4e38075ab0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c009990 tx=0x7f4e2c009ca0 comp rx=0 tx=0).stop 2024-05-03T18:24:09.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:09.368+0000 7f4e3f400700 1 -- 172.21.15.33:0/744035501 shutdown_connections 2024-05-03T18:24:09.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:09.368+0000 7f4e3f400700 1 --2- 172.21.15.33:0/744035501 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4e38114670 0x7f4e38116a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:09.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:09.368+0000 7f4e3f400700 1 --2- 172.21.15.33:0/744035501 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e380756d0 0x7f4e38075ab0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:09.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:09.368+0000 7f4e3f400700 1 --2- 172.21.15.33:0/744035501 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e38075ff0 0x7f4e38114130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:09.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:09.368+0000 7f4e3f400700 1 -- 172.21.15.33:0/744035501 >> 172.21.15.33:0/744035501 conn(0x7f4e3807a600 msgr2=0x7f4e3807aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:24:09.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:09.368+0000 7f4e3f400700 1 -- 172.21.15.33:0/744035501 shutdown_connections 2024-05-03T18:24:09.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:09.368+0000 7f4e3f400700 1 -- 172.21.15.33:0/744035501 wait complete. 2024-05-03T18:24:09.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:09.370+0000 7f4e3f400700 1 Processor -- start 2024-05-03T18:24:09.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:09.370+0000 7f4e3f400700 1 -- start start 2024-05-03T18:24:09.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:09.370+0000 7f4e3f400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e380756d0 0x7f4e381a0080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:24:09.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:09.370+0000 7f4e3f400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4e38075ff0 0x7f4e381a05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:24:09.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:09.370+0000 7f4e3f400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e38114670 0x7f4e381a4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:24:09.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:09.371+0000 7f4e3f400700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4e38119b30 con 0x7f4e380756d0 2024-05-03T18:24:09.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:09.371+0000 7f4e3f400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4e381199b0 con 0x7f4e38114670 2024-05-03T18:24:09.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:09.371+0000 7f4e3f400700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f4e38119cb0 con 0x7f4e38075ff0 2024-05-03T18:24:09.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:09.371+0000 7f4e3ca00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e380756d0 0x7f4e381a0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:24:09.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:09.371+0000 7f4e3ca00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e380756d0 0x7f4e381a0080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45942/0 (socket says 172.21.15.33:45942) 2024-05-03T18:24:09.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:09.371+0000 7f4e3ca00700 1 -- 172.21.15.33:0/361659263 learned_addr learned my addr 172.21.15.33:0/361659263 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:24:09.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:09.371+0000 7f4e3d400700 1 --2- 172.21.15.33:0/361659263 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e38114670 0x7f4e381a4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:24:09.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:09.371+0000 7f4e37e00700 1 --2- 172.21.15.33:0/361659263 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4e38075ff0 0x7f4e381a05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:24:09.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:09.371+0000 7f4e3ca00700 1 -- 172.21.15.33:0/361659263 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4e38075ff0 msgr2=0x7f4e381a05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:24:09.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:09.371+0000 7f4e3ca00700 1 --2- 172.21.15.33:0/361659263 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4e38075ff0 0x7f4e381a05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:09.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:09.371+0000 7f4e3ca00700 1 -- 172.21.15.33:0/361659263 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e38114670 msgr2=0x7f4e381a4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:24:09.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:09.371+0000 7f4e3ca00700 1 --2- 172.21.15.33:0/361659263 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e38114670 0x7f4e381a4950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:09.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:09.371+0000 7f4e3ca00700 1 -- 172.21.15.33:0/361659263 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e381a5090 con 0x7f4e380756d0 2024-05-03T18:24:09.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:09.371+0000 7f4e3d400700 1 --2- 172.21.15.33:0/361659263 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e38114670 0x7f4e381a4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:24:09.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:09.372+0000 7f4e3ca00700 1 --2- 172.21.15.33:0/361659263 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e380756d0 0x7f4e381a0080 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c0057c0 tx=0x7f4e2c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:24:09.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:09.372+0000 7f4e35600700 1 -- 172.21.15.33:0/361659263 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e2c027070 con 0x7f4e380756d0 2024-05-03T18:24:09.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:09.372+0000 7f4e3f400700 1 -- 172.21.15.33:0/361659263 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e381a5320 con 0x7f4e380756d0 2024-05-03T18:24:09.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:09.372+0000 7f4e35600700 1 -- 172.21.15.33:0/361659263 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4e2c00fda0 con 0x7f4e380756d0 2024-05-03T18:24:09.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:09.372+0000 7f4e35600700 1 -- 172.21.15.33:0/361659263 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e2c022ca0 con 0x7f4e380756d0 2024-05-03T18:24:09.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:09.372+0000 7f4e3f400700 1 -- 172.21.15.33:0/361659263 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e381acc00 con 0x7f4e380756d0 2024-05-03T18:24:09.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:09.373+0000 7f4e3f400700 1 -- 172.21.15.33:0/361659263 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e3806a190 con 0x7f4e380756d0 2024-05-03T18:24:09.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:09.376+0000 7f4e35600700 1 -- 172.21.15.33:0/361659263 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f4e2c02a070 con 0x7f4e380756d0 2024-05-03T18:24:09.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:09.378+0000 7f4e35600700 1 --2- 172.21.15.33:0/361659263 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f4e200b2990 0x7f4e200b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:24:09.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:09.379+0000 7f4e35600700 1 -- 172.21.15.33:0/361659263 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9071+0+0 (secure 0 0 0) 0x7f4e2c0e3740 con 0x7f4e380756d0 2024-05-03T18:24:09.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:09.379+0000 7f4e37e00700 1 --2- 172.21.15.33:0/361659263 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f4e200b2990 0x7f4e200b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:24:09.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:09.380+0000 7f4e37e00700 1 --2- 172.21.15.33:0/361659263 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f4e200b2990 0x7f4e200b4e50 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4e24005950 tx=0x7f4e240058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:24:09.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:09.381+0000 7f4e35600700 1 -- 172.21.15.33:0/361659263 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f4e2c031080 con 0x7f4e380756d0 2024-05-03T18:24:09.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:09.665+0000 7f4e3f400700 1 -- 172.21.15.33:0/361659263 --> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f4e380638e0 con 0x7f4e200b2990 2024-05-03T18:24:09.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:09.678+0000 7f4e35600700 1 -- 172.21.15.33:0/361659263 <== mgr.14152 v2:172.21.15.33:6800/3642895501 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f4e380638e0 con 0x7f4e200b2990 2024-05-03T18:24:09.678 INFO:teuthology.orchestra.run.smithi033.stdout:/volumes/qa/sv_1/87cdf5cb-b1da-41d4-9bbd-f5ad5a404bb9 2024-05-03T18:24:09.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:09.681+0000 7f4e3f400700 1 -- 172.21.15.33:0/361659263 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f4e200b2990 msgr2=0x7f4e200b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:24:09.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:09.681+0000 7f4e3f400700 1 --2- 172.21.15.33:0/361659263 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f4e200b2990 0x7f4e200b4e50 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4e24005950 tx=0x7f4e240058e0 comp rx=0 tx=0).stop 2024-05-03T18:24:09.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:09.681+0000 7f4e3f400700 1 -- 172.21.15.33:0/361659263 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e380756d0 msgr2=0x7f4e381a0080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:24:09.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:09.681+0000 7f4e3f400700 1 --2- 172.21.15.33:0/361659263 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e380756d0 0x7f4e381a0080 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c0057c0 tx=0x7f4e2c009f90 comp rx=0 tx=0).stop 2024-05-03T18:24:09.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:09.681+0000 7f4e3f400700 1 -- 172.21.15.33:0/361659263 shutdown_connections 2024-05-03T18:24:09.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:09.681+0000 7f4e3f400700 1 --2- 172.21.15.33:0/361659263 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f4e200b2990 0x7f4e200b4e50 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:09.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:09.681+0000 7f4e3f400700 1 --2- 172.21.15.33:0/361659263 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e38114670 0x7f4e381a4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:09.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:09.681+0000 7f4e3f400700 1 --2- 172.21.15.33:0/361659263 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e380756d0 0x7f4e381a0080 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:09.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:09.681+0000 7f4e3f400700 1 --2- 172.21.15.33:0/361659263 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4e38075ff0 0x7f4e381a05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:09.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:09.681+0000 7f4e3f400700 1 -- 172.21.15.33:0/361659263 >> 172.21.15.33:0/361659263 conn(0x7f4e3807a600 msgr2=0x7f4e38111570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:24:09.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:09.681+0000 7f4e3f400700 1 -- 172.21.15.33:0/361659263 shutdown_connections 2024-05-03T18:24:09.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:09.682+0000 7f4e3f400700 1 -- 172.21.15.33:0/361659263 wait complete. 2024-05-03T18:24:10.656 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T18:24:10.742 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:24:10 smithi033 ceph-mon[28330]: pgmap v432: 129 pgs: 129 active+clean; 631 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 5.3 KiB/s wr, 5 op/s 2024-05-03T18:24:10.742 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:24:10 smithi033 ceph-mon[28330]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:24:10.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:24:10 smithi099 ceph-mon[31658]: pgmap v432: 129 pgs: 129 active+clean; 631 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 5.3 KiB/s wr, 5 op/s 2024-05-03T18:24:10.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:24:10 smithi099 ceph-mon[31658]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:24:10.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:24:10 smithi150 ceph-mon[32492]: pgmap v432: 129 pgs: 129 active+clean; 631 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 5.3 KiB/s wr, 5 op/s 2024-05-03T18:24:10.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:24:10 smithi150 ceph-mon[32492]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:24:11.147 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:24:12.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:24:12 smithi099 ceph-mon[31658]: pgmap v433: 129 pgs: 129 active+clean; 631 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 3.2 KiB/s wr, 3 op/s 2024-05-03T18:24:12.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:24:12 smithi150 ceph-mon[32492]: pgmap v433: 129 pgs: 129 active+clean; 631 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 3.2 KiB/s wr, 3 op/s 2024-05-03T18:24:12.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:24:12 smithi033 ceph-mon[28330]: pgmap v433: 129 pgs: 129 active+clean; 631 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 3.2 KiB/s wr, 3 op/s 2024-05-03T18:24:13.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:13.540+0000 7fae37c00700 1 -- 172.21.15.33:0/495645682 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae3006bb40 msgr2=0x7fae30110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:24:13.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:13.540+0000 7fae37c00700 1 --2- 172.21.15.33:0/495645682 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae3006bb40 0x7fae30110330 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fae2800b0a0 tx=0x7fae2800b3b0 comp rx=0 tx=0).stop 2024-05-03T18:24:13.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:13.540+0000 7fae37c00700 1 -- 172.21.15.33:0/495645682 shutdown_connections 2024-05-03T18:24:13.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:13.540+0000 7fae37c00700 1 --2- 172.21.15.33:0/495645682 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae3006bb40 0x7fae30110330 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:13.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:13.540+0000 7fae37c00700 1 --2- 172.21.15.33:0/495645682 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fae30109510 0x7fae301098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:13.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:13.540+0000 7fae37c00700 1 --2- 172.21.15.33:0/495645682 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fae3006b180 0x7fae3006b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:13.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:13.540+0000 7fae37c00700 1 -- 172.21.15.33:0/495645682 >> 172.21.15.33:0/495645682 conn(0x7fae30079350 msgr2=0x7fae30079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:24:13.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:13.540+0000 7fae37c00700 1 -- 172.21.15.33:0/495645682 shutdown_connections 2024-05-03T18:24:13.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:13.540+0000 7fae37c00700 1 -- 172.21.15.33:0/495645682 wait complete. 2024-05-03T18:24:13.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:13.541+0000 7fae37c00700 1 Processor -- start 2024-05-03T18:24:13.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:13.542+0000 7fae37c00700 1 -- start start 2024-05-03T18:24:13.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:13.542+0000 7fae37c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fae3006b180 0x7fae3019f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:24:13.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:13.542+0000 7fae37c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fae3006bb40 0x7fae3019fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:24:13.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:13.542+0000 7fae37c00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae30109510 0x7fae301998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:24:13.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:13.542+0000 7fae37c00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fae30113540 con 0x7fae30109510 2024-05-03T18:24:13.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:13.542+0000 7fae37c00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fae301133c0 con 0x7fae3006b180 2024-05-03T18:24:13.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:13.542+0000 7fae37c00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fae301136c0 con 0x7fae3006bb40 2024-05-03T18:24:13.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:13.542+0000 7fae35c00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae30109510 0x7fae301998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:24:13.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:13.543+0000 7fae35c00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae30109510 0x7fae301998a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45946/0 (socket says 172.21.15.33:45946) 2024-05-03T18:24:13.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:13.543+0000 7fae35c00700 1 -- 172.21.15.33:0/1521610380 learned_addr learned my addr 172.21.15.33:0/1521610380 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:24:13.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:13.543+0000 7fae27e00700 1 --2- 172.21.15.33:0/1521610380 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fae3006bb40 0x7fae3019fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:24:13.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:13.543+0000 7fae35200700 1 --2- 172.21.15.33:0/1521610380 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fae3006b180 0x7fae3019f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:24:13.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:13.543+0000 7fae35c00700 1 -- 172.21.15.33:0/1521610380 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fae3006bb40 msgr2=0x7fae3019fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:24:13.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:13.543+0000 7fae35c00700 1 --2- 172.21.15.33:0/1521610380 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fae3006bb40 0x7fae3019fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:13.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:13.543+0000 7fae35c00700 1 -- 172.21.15.33:0/1521610380 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fae3006b180 msgr2=0x7fae3019f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:24:13.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:13.543+0000 7fae35c00700 1 --2- 172.21.15.33:0/1521610380 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fae3006b180 0x7fae3019f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:13.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:13.543+0000 7fae35c00700 1 -- 172.21.15.33:0/1521610380 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae3019a000 con 0x7fae30109510 2024-05-03T18:24:13.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:13.543+0000 7fae35c00700 1 --2- 172.21.15.33:0/1521610380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae30109510 0x7fae301998a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fae2800b070 tx=0x7fae28004e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:24:13.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:13.544+0000 7fae2e000700 1 -- 172.21.15.33:0/1521610380 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae28021070 con 0x7fae30109510 2024-05-03T18:24:13.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:13.544+0000 7fae2e000700 1 -- 172.21.15.33:0/1521610380 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fae28007c00 con 0x7fae30109510 2024-05-03T18:24:13.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:13.544+0000 7fae2e000700 1 -- 172.21.15.33:0/1521610380 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae28004780 con 0x7fae30109510 2024-05-03T18:24:13.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:13.544+0000 7fae37c00700 1 -- 172.21.15.33:0/1521610380 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae3019a290 con 0x7fae30109510 2024-05-03T18:24:13.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:13.544+0000 7fae37c00700 1 -- 172.21.15.33:0/1521610380 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae301a65b0 con 0x7fae30109510 2024-05-03T18:24:13.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:13.545+0000 7fae37c00700 1 -- 172.21.15.33:0/1521610380 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae30104bd0 con 0x7fae30109510 2024-05-03T18:24:13.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:13.547+0000 7fae2e000700 1 -- 172.21.15.33:0/1521610380 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fae28003f20 con 0x7fae30109510 2024-05-03T18:24:13.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:13.550+0000 7fae2e000700 1 --2- 172.21.15.33:0/1521610380 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fae140b2990 0x7fae140b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:24:13.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:13.550+0000 7fae2e000700 1 -- 172.21.15.33:0/1521610380 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9071+0+0 (secure 0 0 0) 0x7fae2800f020 con 0x7fae30109510 2024-05-03T18:24:13.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:13.550+0000 7fae35200700 1 --2- 172.21.15.33:0/1521610380 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fae140b2990 0x7fae140b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:24:13.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:13.551+0000 7fae35200700 1 --2- 172.21.15.33:0/1521610380 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fae140b2990 0x7fae140b4e50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fae3019a9e0 tx=0x7fae2000b5b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:24:13.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:13.553+0000 7fae2e000700 1 -- 172.21.15.33:0/1521610380 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fae2802a020 con 0x7fae30109510 2024-05-03T18:24:13.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:13.847+0000 7fae37c00700 1 -- 172.21.15.33:0/1521610380 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fae3002e510 con 0x7fae30109510 2024-05-03T18:24:13.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:13.849+0000 7fae2e000700 1 -- 172.21.15.33:0/1521610380 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11801 (secure 0 0 0) 0x7fae280ab8e0 con 0x7fae30109510 2024-05-03T18:24:13.849 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:24:13.849 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1327393503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1327393503},{"type":"v1","addr":"172.21.15.33:6835","nonce":1327393503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/2461472948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":2461472948},{"type":"v1","addr":"172.21.15.33:6837","nonce":2461472948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14559,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/442269281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":442269281},{"type":"v1","addr":"172.21.15.33:6839","nonce":442269281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14571,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/645571756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":645571756},{"type":"v1","addr":"172.21.15.33:6841","nonce":645571756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/203598909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":203598909},{"type":"v1","addr":"172.21.15.150:6833","nonce":203598909}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/4029842493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":4029842493},{"type":"v1","addr":"172.21.15.150:6835","nonce":4029842493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3463664423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3463664423},{"type":"v1","addr":"172.21.15.150:6837","nonce":3463664423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24481,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6839/4051648018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":4051648018},{"type":"v1","addr":"172.21.15.150:6839","nonce":4051648018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34418,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1865105479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1865105479},{"type":"v1","addr":"172.21.15.99:6833","nonce":1865105479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34445,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/1206393914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1206393914},{"type":"v1","addr":"172.21.15.99:6835","nonce":1206393914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-03T18:22:28.837091+0000","modified":"2024-05-03T18:23:49.758398+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34508},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34472":{"gid":34472,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6837/3360970342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3360970342},{"type":"v1","addr":"172.21.15.99:6837","nonce":3360970342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34508":{"gid":34508,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/919907308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":919907308},{"type":"v1","addr":"172.21.15.99:6839","nonce":919907308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34508,"qdb_cluster":[34508]},"id":1}]} 2024-05-03T18:24:13.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:13.852+0000 7fae37c00700 1 -- 172.21.15.33:0/1521610380 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fae140b2990 msgr2=0x7fae140b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:24:13.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:13.852+0000 7fae37c00700 1 --2- 172.21.15.33:0/1521610380 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fae140b2990 0x7fae140b4e50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fae3019a9e0 tx=0x7fae2000b5b0 comp rx=0 tx=0).stop 2024-05-03T18:24:13.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:13.852+0000 7fae37c00700 1 -- 172.21.15.33:0/1521610380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae30109510 msgr2=0x7fae301998a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:24:13.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:13.852+0000 7fae37c00700 1 --2- 172.21.15.33:0/1521610380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae30109510 0x7fae301998a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fae2800b070 tx=0x7fae28004e00 comp rx=0 tx=0).stop 2024-05-03T18:24:13.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:13.852+0000 7fae37c00700 1 -- 172.21.15.33:0/1521610380 shutdown_connections 2024-05-03T18:24:13.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:13.852+0000 7fae37c00700 1 --2- 172.21.15.33:0/1521610380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae30109510 0x7fae301998a0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:13.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:13.852+0000 7fae37c00700 1 --2- 172.21.15.33:0/1521610380 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fae140b2990 0x7fae140b4e50 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:13.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:13.852+0000 7fae37c00700 1 --2- 172.21.15.33:0/1521610380 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fae3006b180 0x7fae3019f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:13.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:13.852+0000 7fae37c00700 1 --2- 172.21.15.33:0/1521610380 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fae3006bb40 0x7fae3019fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:13.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:13.852+0000 7fae37c00700 1 -- 172.21.15.33:0/1521610380 >> 172.21.15.33:0/1521610380 conn(0x7fae30079350 msgr2=0x7fae30101180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:24:13.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:13.853+0000 7fae37c00700 1 -- 172.21.15.33:0/1521610380 shutdown_connections 2024-05-03T18:24:13.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:13.853+0000 7fae37c00700 1 -- 172.21.15.33:0/1521610380 wait complete. 2024-05-03T18:24:13.854 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 21 2024-05-03T18:24:14.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:24:14 smithi099 ceph-mon[31658]: pgmap v434: 129 pgs: 129 active+clean; 632 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.3 KiB/s wr, 4 op/s 2024-05-03T18:24:14.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:24:14 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/1521610380' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:24:14.805 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:24:14 smithi033 ceph-mon[28330]: pgmap v434: 129 pgs: 129 active+clean; 632 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.3 KiB/s wr, 4 op/s 2024-05-03T18:24:14.805 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:24:14 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/1521610380' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:24:14.810 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-03T18:24:14.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:24:14 smithi150 ceph-mon[32492]: pgmap v434: 129 pgs: 129 active+clean; 632 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.3 KiB/s wr, 4 op/s 2024-05-03T18:24:14.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:24:14 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/1521610380' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:24:15.303 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:24:16.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:24:16 smithi099 ceph-mon[31658]: pgmap v435: 129 pgs: 129 active+clean; 632 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.7 KiB/s wr, 3 op/s 2024-05-03T18:24:16.862 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:24:16 smithi033 ceph-mon[28330]: pgmap v435: 129 pgs: 129 active+clean; 632 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.7 KiB/s wr, 3 op/s 2024-05-03T18:24:16.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:24:16 smithi150 ceph-mon[32492]: pgmap v435: 129 pgs: 129 active+clean; 632 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.7 KiB/s wr, 3 op/s 2024-05-03T18:24:17.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:17.589+0000 7f059ea00700 1 -- 172.21.15.33:0/2666508274 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05981095d0 msgr2=0x7f05981099b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:24:17.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:17.589+0000 7f059ea00700 1 --2- 172.21.15.33:0/2666508274 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05981095d0 0x7f05981099b0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f05880099c0 tx=0x7f0588009cd0 comp rx=0 tx=0).stop 2024-05-03T18:24:17.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:17.589+0000 7f059ea00700 1 -- 172.21.15.33:0/2666508274 shutdown_connections 2024-05-03T18:24:17.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:17.589+0000 7f059ea00700 1 --2- 172.21.15.33:0/2666508274 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0598103ef0 0x7f05981103a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:17.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:17.589+0000 7f059ea00700 1 --2- 172.21.15.33:0/2666508274 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05981095d0 0x7f05981099b0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:17.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:17.589+0000 7f059ea00700 1 --2- 172.21.15.33:0/2666508274 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0598103530 0x7f05981039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:17.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:17.589+0000 7f059ea00700 1 -- 172.21.15.33:0/2666508274 >> 172.21.15.33:0/2666508274 conn(0x7f059807a690 msgr2=0x7f059807aaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:24:17.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:17.590+0000 7f059ea00700 1 -- 172.21.15.33:0/2666508274 shutdown_connections 2024-05-03T18:24:17.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:17.590+0000 7f059ea00700 1 -- 172.21.15.33:0/2666508274 wait complete. 2024-05-03T18:24:17.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:17.591+0000 7f059ea00700 1 Processor -- start 2024-05-03T18:24:17.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:17.591+0000 7f059ea00700 1 -- start start 2024-05-03T18:24:17.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:17.592+0000 7f059ea00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0598103530 0x7f05980757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:24:17.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:17.592+0000 7f059ea00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0598103ef0 0x7f0598075ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:24:17.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:17.592+0000 7f059ea00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05981095d0 0x7f0598076220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:24:17.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:17.592+0000 7f059ea00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0598113580 con 0x7f05981095d0 2024-05-03T18:24:17.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:17.592+0000 7f059ea00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0598113400 con 0x7f0598103530 2024-05-03T18:24:17.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:17.592+0000 7f059ea00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f0598113700 con 0x7f0598103ef0 2024-05-03T18:24:17.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:17.592+0000 7f059ca00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05981095d0 0x7f0598076220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:24:17.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:17.592+0000 7f059ca00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05981095d0 0x7f0598076220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60534/0 (socket says 172.21.15.33:60534) 2024-05-03T18:24:17.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:17.592+0000 7f059ca00700 1 -- 172.21.15.33:0/1469228895 learned_addr learned my addr 172.21.15.33:0/1469228895 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:24:17.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:17.592+0000 7f0597e00700 1 --2- 172.21.15.33:0/1469228895 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0598103530 0x7f05980757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:24:17.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:17.593+0000 7f0597400700 1 --2- 172.21.15.33:0/1469228895 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0598103ef0 0x7f0598075ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:24:17.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:17.593+0000 7f059ca00700 1 -- 172.21.15.33:0/1469228895 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0598103ef0 msgr2=0x7f0598075ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:24:17.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:17.593+0000 7f059ca00700 1 --2- 172.21.15.33:0/1469228895 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0598103ef0 0x7f0598075ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:17.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:17.593+0000 7f059ca00700 1 -- 172.21.15.33:0/1469228895 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0598103530 msgr2=0x7f05980757a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:24:17.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:17.593+0000 7f059ca00700 1 --2- 172.21.15.33:0/1469228895 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0598103530 0x7f05980757a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:17.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:17.593+0000 7f059ca00700 1 -- 172.21.15.33:0/1469228895 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05981a8080 con 0x7f05981095d0 2024-05-03T18:24:17.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:17.593+0000 7f059ca00700 1 --2- 172.21.15.33:0/1469228895 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05981095d0 0x7f0598076220 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f0590007ee0 tx=0x7f059000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:24:17.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:17.594+0000 7f0594c00700 1 -- 172.21.15.33:0/1469228895 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f059000dbd0 con 0x7f05981095d0 2024-05-03T18:24:17.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:17.594+0000 7f059ea00700 1 -- 172.21.15.33:0/1469228895 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05981a8370 con 0x7f05981095d0 2024-05-03T18:24:17.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:17.594+0000 7f0594c00700 1 -- 172.21.15.33:0/1469228895 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f059000dd70 con 0x7f05981095d0 2024-05-03T18:24:17.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:17.594+0000 7f0594c00700 1 -- 172.21.15.33:0/1469228895 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05900058d0 con 0x7f05981095d0 2024-05-03T18:24:17.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:17.594+0000 7f059ea00700 1 -- 172.21.15.33:0/1469228895 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05981a8900 con 0x7f05981095d0 2024-05-03T18:24:17.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:17.596+0000 7f059ea00700 1 -- 172.21.15.33:0/1469228895 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f059806a190 con 0x7f05981095d0 2024-05-03T18:24:17.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:17.599+0000 7f0594c00700 1 -- 172.21.15.33:0/1469228895 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f059001a020 con 0x7f05981095d0 2024-05-03T18:24:17.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:17.601+0000 7f0594c00700 1 --2- 172.21.15.33:0/1469228895 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f057c0b2990 0x7f057c0b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:24:17.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:17.601+0000 7f0594c00700 1 -- 172.21.15.33:0/1469228895 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9071+0+0 (secure 0 0 0) 0x7f05900e1c30 con 0x7f05981095d0 2024-05-03T18:24:17.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:17.601+0000 7f0597e00700 1 --2- 172.21.15.33:0/1469228895 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f057c0b2990 0x7f057c0b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:24:17.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:17.602+0000 7f0597e00700 1 --2- 172.21.15.33:0/1469228895 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f057c0b2990 0x7f057c0b4e50 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0588000c00 tx=0x7f058801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:24:17.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:17.606+0000 7f0594c00700 1 -- 172.21.15.33:0/1469228895 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f0590010070 con 0x7f05981095d0 2024-05-03T18:24:17.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:17.843+0000 7f059ea00700 1 -- 172.21.15.33:0/1469228895 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f059802e510 con 0x7f05981095d0 2024-05-03T18:24:17.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:17.845+0000 7f0594c00700 1 -- 172.21.15.33:0/1469228895 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+24499 (secure 0 0 0) 0x7f05900a9450 con 0x7f05981095d0 2024-05-03T18:24:17.846 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:24:17.846 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":87,"fsid":"5703bba4-0978-11ef-bc97-c7b262605968","created":"2024-05-03T18:11:00.899879+0000","modified":"2024-05-03T18:23:20.514086+0000","last_up_change":"2024-05-03T18:21:00.390173+0000","last_in_change":"2024-05-03T18:20:34.103727+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-03T18:15:37.297738+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-03T18:22:19.875368+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-03T18:22:24.432692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-03T18:22:38.722443+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"fdcc4e66-4ba0-4378-a9a4-7d907cd00b4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":637686669},{"type":"v1","addr":"172.21.15.33:6803","nonce":637686669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":637686669},{"type":"v1","addr":"172.21.15.33:6805","nonce":637686669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":637686669},{"type":"v1","addr":"172.21.15.33:6809","nonce":637686669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":637686669},{"type":"v1","addr":"172.21.15.33:6807","nonce":637686669}]},"public_addr":"172.21.15.33:6803/637686669","cluster_addr":"172.21.15.33:6805/637686669","heartbeat_back_addr":"172.21.15.33:6809/637686669","heartbeat_front_addr":"172.21.15.33:6807/637686669","state":["exists","up"]},{"osd":1,"uuid":"cf78cad3-fc24-43f4-8784-21cabea7277c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":1536122841},{"type":"v1","addr":"172.21.15.99:6801","nonce":1536122841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":1536122841},{"type":"v1","addr":"172.21.15.99:6803","nonce":1536122841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":1536122841},{"type":"v1","addr":"172.21.15.99:6807","nonce":1536122841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":1536122841},{"type":"v1","addr":"172.21.15.99:6805","nonce":1536122841}]},"public_addr":"172.21.15.99:6801/1536122841","cluster_addr":"172.21.15.99:6803/1536122841","heartbeat_back_addr":"172.21.15.99:6807/1536122841","heartbeat_front_addr":"172.21.15.99:6805/1536122841","state":["exists","up"]},{"osd":2,"uuid":"59f066d7-626b-410a-8d67-daca6fbbd3ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":1995659622},{"type":"v1","addr":"172.21.15.150:6801","nonce":1995659622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":1995659622},{"type":"v1","addr":"172.21.15.150:6803","nonce":1995659622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":1995659622},{"type":"v1","addr":"172.21.15.150:6807","nonce":1995659622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":1995659622},{"type":"v1","addr":"172.21.15.150:6805","nonce":1995659622}]},"public_addr":"172.21.15.150:6801/1995659622","cluster_addr":"172.21.15.150:6803/1995659622","heartbeat_back_addr":"172.21.15.150:6807/1995659622","heartbeat_front_addr":"172.21.15.150:6805/1995659622","state":["exists","up"]},{"osd":3,"uuid":"d88ed022-4dcb-45fc-9fc6-41ef7c4bf202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":3814942724},{"type":"v1","addr":"172.21.15.33:6811","nonce":3814942724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":3814942724},{"type":"v1","addr":"172.21.15.33:6813","nonce":3814942724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":3814942724},{"type":"v1","addr":"172.21.15.33:6817","nonce":3814942724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":3814942724},{"type":"v1","addr":"172.21.15.33:6815","nonce":3814942724}]},"public_addr":"172.21.15.33:6811/3814942724","cluster_addr":"172.21.15.33:6813/3814942724","heartbeat_back_addr":"172.21.15.33:6817/3814942724","heartbeat_front_addr":"172.21.15.33:6815/3814942724","state":["exists","up"]},{"osd":4,"uuid":"1235e1a6-ff6e-4668-ac1a-073c2a3b7c98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":29081029},{"type":"v1","addr":"172.21.15.99:6809","nonce":29081029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":29081029},{"type":"v1","addr":"172.21.15.99:6811","nonce":29081029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":29081029},{"type":"v1","addr":"172.21.15.99:6815","nonce":29081029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":29081029},{"type":"v1","addr":"172.21.15.99:6813","nonce":29081029}]},"public_addr":"172.21.15.99:6809/29081029","cluster_addr":"172.21.15.99:6811/29081029","heartbeat_back_addr":"172.21.15.99:6815/29081029","heartbeat_front_addr":"172.21.15.99:6813/29081029","state":["exists","up"]},{"osd":5,"uuid":"4b84bff8-d6d7-4258-a0ee-53bc19802585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":3172189533},{"type":"v1","addr":"172.21.15.150:6809","nonce":3172189533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3172189533},{"type":"v1","addr":"172.21.15.150:6811","nonce":3172189533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3172189533},{"type":"v1","addr":"172.21.15.150:6815","nonce":3172189533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3172189533},{"type":"v1","addr":"172.21.15.150:6813","nonce":3172189533}]},"public_addr":"172.21.15.150:6809/3172189533","cluster_addr":"172.21.15.150:6811/3172189533","heartbeat_back_addr":"172.21.15.150:6815/3172189533","heartbeat_front_addr":"172.21.15.150:6813/3172189533","state":["exists","up"]},{"osd":6,"uuid":"51f5fbdd-3794-4c48-9f6c-478f0fa35ac5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":4234541418},{"type":"v1","addr":"172.21.15.33:6819","nonce":4234541418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":4234541418},{"type":"v1","addr":"172.21.15.33:6821","nonce":4234541418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":4234541418},{"type":"v1","addr":"172.21.15.33:6825","nonce":4234541418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":4234541418},{"type":"v1","addr":"172.21.15.33:6823","nonce":4234541418}]},"public_addr":"172.21.15.33:6819/4234541418","cluster_addr":"172.21.15.33:6821/4234541418","heartbeat_back_addr":"172.21.15.33:6825/4234541418","heartbeat_front_addr":"172.21.15.33:6823/4234541418","state":["exists","up"]},{"osd":7,"uuid":"982cc5b3-3163-4505-9bde-a27a8c289457","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":908673090},{"type":"v1","addr":"172.21.15.99:6817","nonce":908673090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":908673090},{"type":"v1","addr":"172.21.15.99:6819","nonce":908673090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":908673090},{"type":"v1","addr":"172.21.15.99:6823","nonce":908673090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":908673090},{"type":"v1","addr":"172.21.15.99:6821","nonce":908673090}]},"public_addr":"172.21.15.99:6817/908673090","cluster_addr":"172.21.15.99:6819/908673090","heartbeat_back_addr":"172.21.15.99:6823/908673090","heartbeat_front_addr":"172.21.15.99:6821/908673090","state":["exists","up"]},{"osd":8,"uuid":"84c9d037-eb84-4eb4-8aa2-e784b703d36c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":1308083602},{"type":"v1","addr":"172.21.15.150:6817","nonce":1308083602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":1308083602},{"type":"v1","addr":"172.21.15.150:6819","nonce":1308083602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":1308083602},{"type":"v1","addr":"172.21.15.150:6823","nonce":1308083602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":1308083602},{"type":"v1","addr":"172.21.15.150:6821","nonce":1308083602}]},"public_addr":"172.21.15.150:6817/1308083602","cluster_addr":"172.21.15.150:6819/1308083602","heartbeat_back_addr":"172.21.15.150:6823/1308083602","heartbeat_front_addr":"172.21.15.150:6821/1308083602","state":["exists","up"]},{"osd":9,"uuid":"a02e965f-0d8f-45e2-9490-337f7f6e3881","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":4133096905},{"type":"v1","addr":"172.21.15.33:6827","nonce":4133096905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":4133096905},{"type":"v1","addr":"172.21.15.33:6829","nonce":4133096905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4133096905},{"type":"v1","addr":"172.21.15.33:6833","nonce":4133096905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":4133096905},{"type":"v1","addr":"172.21.15.33:6831","nonce":4133096905}]},"public_addr":"172.21.15.33:6827/4133096905","cluster_addr":"172.21.15.33:6829/4133096905","heartbeat_back_addr":"172.21.15.33:6833/4133096905","heartbeat_front_addr":"172.21.15.33:6831/4133096905","state":["exists","up"]},{"osd":10,"uuid":"8b29bb96-76e8-45a0-ab19-8d082d755fe9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":935746136},{"type":"v1","addr":"172.21.15.99:6825","nonce":935746136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":935746136},{"type":"v1","addr":"172.21.15.99:6827","nonce":935746136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":935746136},{"type":"v1","addr":"172.21.15.99:6831","nonce":935746136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":935746136},{"type":"v1","addr":"172.21.15.99:6829","nonce":935746136}]},"public_addr":"172.21.15.99:6825/935746136","cluster_addr":"172.21.15.99:6827/935746136","heartbeat_back_addr":"172.21.15.99:6831/935746136","heartbeat_front_addr":"172.21.15.99:6829/935746136","state":["exists","up"]},{"osd":11,"uuid":"07a6a825-c902-4bfd-97cc-cc9074532af5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2069349808},{"type":"v1","addr":"172.21.15.150:6825","nonce":2069349808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2069349808},{"type":"v1","addr":"172.21.15.150:6827","nonce":2069349808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2069349808},{"type":"v1","addr":"172.21.15.150:6831","nonce":2069349808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2069349808},{"type":"v1","addr":"172.21.15.150:6829","nonce":2069349808}]},"public_addr":"172.21.15.150:6825/2069349808","cluster_addr":"172.21.15.150:6827/2069349808","heartbeat_back_addr":"172.21.15.150:6831/2069349808","heartbeat_front_addr":"172.21.15.150:6829/2069349808","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:14:27.454536+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:15:00.538570+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:15:34.125289+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:16:09.853870+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:16:44.928107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:17:19.644037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:17:56.108443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:18:31.731499+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:19:08.062358+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:19:45.204430+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:20:21.013710+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:20:57.653698+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11},{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/3991947997":"2024-05-04T18:12:19.127652+0000","172.21.15.33:0/965859483":"2024-05-04T18:12:19.127652+0000","172.21.15.33:0/2014902144":"2024-05-04T18:11:44.239835+0000","172.21.15.33:0/1314704208":"2024-05-04T18:11:44.239835+0000","172.21.15.33:0/1614640996":"2024-05-04T18:11:44.239835+0000","172.21.15.33:6800/4007322071":"2024-05-04T18:11:44.239835+0000","172.21.15.33:6801/2830135294":"2024-05-04T18:12:19.127652+0000","172.21.15.33:6800/2830135294":"2024-05-04T18:12:19.127652+0000","172.21.15.33:0/3767665033":"2024-05-04T18:12:19.127652+0000","172.21.15.33:6801/4007322071":"2024-05-04T18:11:44.239835+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-03T18:24:17.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:17.848+0000 7f059ea00700 1 -- 172.21.15.33:0/1469228895 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f057c0b2990 msgr2=0x7f057c0b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:24:17.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:17.848+0000 7f059ea00700 1 --2- 172.21.15.33:0/1469228895 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f057c0b2990 0x7f057c0b4e50 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0588000c00 tx=0x7f058801a040 comp rx=0 tx=0).stop 2024-05-03T18:24:17.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:17.848+0000 7f059ea00700 1 -- 172.21.15.33:0/1469228895 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05981095d0 msgr2=0x7f0598076220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:24:17.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:17.848+0000 7f059ea00700 1 --2- 172.21.15.33:0/1469228895 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05981095d0 0x7f0598076220 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f0590007ee0 tx=0x7f059000d3b0 comp rx=0 tx=0).stop 2024-05-03T18:24:17.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:17.849+0000 7f059ea00700 1 -- 172.21.15.33:0/1469228895 shutdown_connections 2024-05-03T18:24:17.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:17.849+0000 7f059ea00700 1 --2- 172.21.15.33:0/1469228895 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05981095d0 0x7f0598076220 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:17.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:17.849+0000 7f059ea00700 1 --2- 172.21.15.33:0/1469228895 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f057c0b2990 0x7f057c0b4e50 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:17.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:17.849+0000 7f059ea00700 1 --2- 172.21.15.33:0/1469228895 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0598103530 0x7f05980757a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:17.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:17.849+0000 7f059ea00700 1 --2- 172.21.15.33:0/1469228895 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0598103ef0 0x7f0598075ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:17.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:17.849+0000 7f059ea00700 1 -- 172.21.15.33:0/1469228895 >> 172.21.15.33:0/1469228895 conn(0x7f059807a690 msgr2=0x7f05981023c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:24:17.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:17.849+0000 7f059ea00700 1 -- 172.21.15.33:0/1469228895 shutdown_connections 2024-05-03T18:24:17.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:17.849+0000 7f059ea00700 1 -- 172.21.15.33:0/1469228895 wait complete. 2024-05-03T18:24:18.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:24:18 smithi099 ceph-mon[31658]: pgmap v436: 129 pgs: 129 active+clean; 632 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.7 KiB/s wr, 3 op/s 2024-05-03T18:24:18.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:24:18 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/1469228895' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T18:24:18.819 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:24:18 smithi033 ceph-mon[28330]: pgmap v436: 129 pgs: 129 active+clean; 632 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.7 KiB/s wr, 3 op/s 2024-05-03T18:24:18.819 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:24:18 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/1469228895' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T18:24:18.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:24:18 smithi150 ceph-mon[32492]: pgmap v436: 129 pgs: 129 active+clean; 632 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.7 KiB/s wr, 3 op/s 2024-05-03T18:24:18.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:24:18 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/1469228895' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T18:24:18.899 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-03T18:24:18.900 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-03T18:24:19.363 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:24:20.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:24:20 smithi033 ceph-mon[28330]: pgmap v437: 129 pgs: 129 active+clean; 632 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 2.7 KiB/s wr, 4 op/s 2024-05-03T18:24:20.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:24:20 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-05-03T18:24:20.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:24:20 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-05-03T18:24:20.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:24:20 smithi099 ceph-mon[31658]: pgmap v437: 129 pgs: 129 active+clean; 632 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 2.7 KiB/s wr, 4 op/s 2024-05-03T18:24:20.812 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:24:20 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-05-03T18:24:20.812 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:24:20 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-05-03T18:24:20.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:24:20 smithi150 ceph-mon[32492]: pgmap v437: 129 pgs: 129 active+clean; 632 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 2.7 KiB/s wr, 4 op/s 2024-05-03T18:24:20.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:24:20 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-05-03T18:24:20.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:24:20 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-05-03T18:24:21.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:21.570+0000 7f2853e00700 1 -- 172.21.15.33:0/3852092280 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f285410a3a0 msgr2=0x7f28541168a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:24:21.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:21.570+0000 7f2853e00700 1 --2- 172.21.15.33:0/3852092280 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f285410a3a0 0x7f28541168a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f284c00b0a0 tx=0x7f284c00b3b0 comp rx=0 tx=0).stop 2024-05-03T18:24:21.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:21.570+0000 7f2853e00700 1 -- 172.21.15.33:0/3852092280 shutdown_connections 2024-05-03T18:24:21.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:21.570+0000 7f2853e00700 1 --2- 172.21.15.33:0/3852092280 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f285410a3a0 0x7f28541168a0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:21.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:21.570+0000 7f2853e00700 1 --2- 172.21.15.33:0/3852092280 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f285410f9d0 0x7f285410fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:21.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:21.570+0000 7f2853e00700 1 --2- 172.21.15.33:0/3852092280 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f28541099e0 0x7f2854109e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:21.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:21.570+0000 7f2853e00700 1 -- 172.21.15.33:0/3852092280 >> 172.21.15.33:0/3852092280 conn(0x7f2854079200 msgr2=0x7f28541082c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:24:21.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:21.570+0000 7f2853e00700 1 -- 172.21.15.33:0/3852092280 shutdown_connections 2024-05-03T18:24:21.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:21.571+0000 7f2853e00700 1 -- 172.21.15.33:0/3852092280 wait complete. 2024-05-03T18:24:21.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:21.572+0000 7f2853e00700 1 Processor -- start 2024-05-03T18:24:21.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:21.572+0000 7f2853e00700 1 -- start start 2024-05-03T18:24:21.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:21.573+0000 7f2853e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f28541099e0 0x7f285419ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:24:21.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:21.573+0000 7f2853e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f285410a3a0 0x7f28541a0460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:24:21.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:21.573+0000 7f2853e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f285410f9d0 0x7f28541a47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:24:21.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:21.573+0000 7f2853e00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2854119b10 con 0x7f285410f9d0 2024-05-03T18:24:21.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:21.573+0000 7f2853e00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2854119990 con 0x7f28541099e0 2024-05-03T18:24:21.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:21.573+0000 7f2853e00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2854119c90 con 0x7f285410a3a0 2024-05-03T18:24:21.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:21.573+0000 7f2853400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f285410f9d0 0x7f28541a47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:24:21.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:21.573+0000 7f2853400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f285410f9d0 0x7f28541a47f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60554/0 (socket says 172.21.15.33:60554) 2024-05-03T18:24:21.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:21.573+0000 7f2853400700 1 -- 172.21.15.33:0/139936257 learned_addr learned my addr 172.21.15.33:0/139936257 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:24:21.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:21.573+0000 7f2852a00700 1 --2- 172.21.15.33:0/139936257 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f28541099e0 0x7f285419ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:24:21.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:21.574+0000 7f2853400700 1 -- 172.21.15.33:0/139936257 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f285410a3a0 msgr2=0x7f28541a0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:24:21.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:21.574+0000 7f2853400700 1 --2- 172.21.15.33:0/139936257 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f285410a3a0 0x7f28541a0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:21.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:21.574+0000 7f2853400700 1 -- 172.21.15.33:0/139936257 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f28541099e0 msgr2=0x7f285419ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:24:21.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:21.574+0000 7f2853400700 1 --2- 172.21.15.33:0/139936257 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f28541099e0 0x7f285419ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:21.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:21.574+0000 7f2853400700 1 -- 172.21.15.33:0/139936257 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28541a4f30 con 0x7f285410f9d0 2024-05-03T18:24:21.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:21.574+0000 7f2853400700 1 --2- 172.21.15.33:0/139936257 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f285410f9d0 0x7f28541a47f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f284c009610 tx=0x7f284c004e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:24:21.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:21.574+0000 7f284b400700 1 -- 172.21.15.33:0/139936257 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f284c021070 con 0x7f285410f9d0 2024-05-03T18:24:21.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:21.574+0000 7f284b400700 1 -- 172.21.15.33:0/139936257 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f284c007c00 con 0x7f285410f9d0 2024-05-03T18:24:21.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:21.574+0000 7f2853e00700 1 -- 172.21.15.33:0/139936257 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28541a51c0 con 0x7f285410f9d0 2024-05-03T18:24:21.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:21.574+0000 7f284b400700 1 -- 172.21.15.33:0/139936257 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f284c0052a0 con 0x7f285410f9d0 2024-05-03T18:24:21.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:21.574+0000 7f2853e00700 1 -- 172.21.15.33:0/139936257 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28541acaa0 con 0x7f285410f9d0 2024-05-03T18:24:21.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:21.576+0000 7f2853e00700 1 -- 172.21.15.33:0/139936257 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f285406a190 con 0x7f285410f9d0 2024-05-03T18:24:21.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:21.580+0000 7f284b400700 1 -- 172.21.15.33:0/139936257 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f284c024070 con 0x7f285410f9d0 2024-05-03T18:24:21.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:21.581+0000 7f284b400700 1 --2- 172.21.15.33:0/139936257 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f28380b2910 0x7f28380b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:24:21.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:21.581+0000 7f284b400700 1 -- 172.21.15.33:0/139936257 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f284c00f020 con 0x7f285410f9d0 2024-05-03T18:24:21.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:21.581+0000 7f2852a00700 1 --2- 172.21.15.33:0/139936257 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f28380b2910 0x7f28380b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:24:21.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:21.582+0000 7f2852a00700 1 --2- 172.21.15.33:0/139936257 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f28380b2910 0x7f28380b4dd0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2844005ea0 tx=0x7f2844005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:24:21.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:21.584+0000 7f284b400700 1 -- 172.21.15.33:0/139936257 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f284c02b020 con 0x7f285410f9d0 2024-05-03T18:24:21.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:24:21 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-05-03T18:24:21.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:24:21 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-05-03T18:24:21.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:24:21 smithi033 ceph-mon[28330]: osdmap e88: 12 total, 12 up, 12 in 2024-05-03T18:24:21.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:21.816+0000 7f2853e00700 1 -- 172.21.15.33:0/139936257 --> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f28541a1280 con 0x7f28380b2910 2024-05-03T18:24:21.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:21.851+0000 7f284b400700 1 -- 172.21.15.33:0/139936257 <== mgr.14152 v2:172.21.15.33:6800/3642895501 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340316 (secure 0 0 0) 0x7f28541a1280 con 0x7f28380b2910 2024-05-03T18:24:21.857 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:24:21.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:21.856+0000 7f2853e00700 1 -- 172.21.15.33:0/139936257 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f28380b2910 msgr2=0x7f28380b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:24:21.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:21.856+0000 7f2853e00700 1 --2- 172.21.15.33:0/139936257 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f28380b2910 0x7f28380b4dd0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2844005ea0 tx=0x7f2844005e30 comp rx=0 tx=0).stop 2024-05-03T18:24:21.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:21.856+0000 7f2853e00700 1 -- 172.21.15.33:0/139936257 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f285410f9d0 msgr2=0x7f28541a47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:24:21.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:21.856+0000 7f2853e00700 1 --2- 172.21.15.33:0/139936257 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f285410f9d0 0x7f28541a47f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f284c009610 tx=0x7f284c004e70 comp rx=0 tx=0).stop 2024-05-03T18:24:21.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:21.857+0000 7f2853e00700 1 -- 172.21.15.33:0/139936257 shutdown_connections 2024-05-03T18:24:21.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:21.857+0000 7f2853e00700 1 --2- 172.21.15.33:0/139936257 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f285410f9d0 0x7f28541a47f0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:21.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:21.857+0000 7f2853e00700 1 --2- 172.21.15.33:0/139936257 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f28380b2910 0x7f28380b4dd0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:21.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:21.857+0000 7f2853e00700 1 --2- 172.21.15.33:0/139936257 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f28541099e0 0x7f285419ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:21.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:21.857+0000 7f2853e00700 1 --2- 172.21.15.33:0/139936257 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f285410a3a0 0x7f28541a0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:21.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:21.857+0000 7f2853e00700 1 -- 172.21.15.33:0/139936257 >> 172.21.15.33:0/139936257 conn(0x7f2854079200 msgr2=0x7f2854079780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:24:21.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:21.857+0000 7f2853e00700 1 -- 172.21.15.33:0/139936257 shutdown_connections 2024-05-03T18:24:21.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:21.857+0000 7f2853e00700 1 -- 172.21.15.33:0/139936257 wait complete. 2024-05-03T18:24:21.861 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-05-03T18:24:21.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:24:21 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-05-03T18:24:21.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:24:21 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-05-03T18:24:21.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:24:21 smithi150 ceph-mon[32492]: osdmap e88: 12 total, 12 up, 12 in 2024-05-03T18:24:22.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:24:21 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-05-03T18:24:22.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:24:21 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-05-03T18:24:22.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:24:21 smithi099 ceph-mon[31658]: osdmap e88: 12 total, 12 up, 12 in 2024-05-03T18:24:22.760 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":440,"stamp":"2024-05-03T18:24:21.546419+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":647515,"num_objects":26,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":159,"num_read_kb":166,"num_write":206,"num_write_kb":2236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":163,"ondisk_log_size":163,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":366924,"kb_used_data":13592,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124574900,"statfs":{"total":1151940427776,"available":1151564697600,"internally_reserved":0,"allocated":13918208,"data_stored":7900856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":361674042},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":5,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.184071"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-03T18:24:20.539684+0000","last_change":"2024-05-03T18:22:25.595640+0000","last_active":"2024-05-03T18:24:20.539684+0000","last_peered":"2024-05-03T18:24:20.539684+0000","last_clean":"2024-05-03T18:24:20.539684+0000","last_became_active":"2024-05-03T18:22:25.595284+0000","last_became_peered":"2024-05-03T18:22:25.595284+0000","last_unstale":"2024-05-03T18:24:20.539684+0000","last_undegraded":"2024-05-03T18:24:20.539684+0000","last_fullsized":"2024-05-03T18:24:20.539684+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:50:08.867670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-03T18:23:20.524525+0000","last_change":"2024-05-03T18:22:25.584950+0000","last_active":"2024-05-03T18:23:20.524525+0000","last_peered":"2024-05-03T18:23:20.524525+0000","last_clean":"2024-05-03T18:23:20.524525+0000","last_became_active":"2024-05-03T18:22:25.584613+0000","last_became_peered":"2024-05-03T18:22:25.584613+0000","last_unstale":"2024-05-03T18:23:20.524525+0000","last_undegraded":"2024-05-03T18:23:20.524525+0000","last_fullsized":"2024-05-03T18:23:20.524525+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:00:24.959429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-03T18:23:20.523798+0000","last_change":"2024-05-03T18:22:25.596799+0000","last_active":"2024-05-03T18:23:20.523798+0000","last_peered":"2024-05-03T18:23:20.523798+0000","last_clean":"2024-05-03T18:23:20.523798+0000","last_became_active":"2024-05-03T18:22:25.596386+0000","last_became_peered":"2024-05-03T18:22:25.596386+0000","last_unstale":"2024-05-03T18:23:20.523798+0000","last_undegraded":"2024-05-03T18:23:20.523798+0000","last_fullsized":"2024-05-03T18:23:20.523798+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:20:07.818282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-03T18:23:20.535534+0000","last_change":"2024-05-03T18:23:20.534672+0000","last_active":"2024-05-03T18:23:20.535534+0000","last_peered":"2024-05-03T18:23:20.535534+0000","last_clean":"2024-05-03T18:23:20.535534+0000","last_became_active":"2024-05-03T18:23:20.534073+0000","last_became_peered":"2024-05-03T18:23:20.534073+0000","last_unstale":"2024-05-03T18:23:20.535534+0000","last_undegraded":"2024-05-03T18:23:20.535534+0000","last_fullsized":"2024-05-03T18:23:20.535534+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:07:40.204299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-03T18:24:20.917187+0000","last_change":"2024-05-03T18:22:25.589810+0000","last_active":"2024-05-03T18:24:20.917187+0000","last_peered":"2024-05-03T18:24:20.917187+0000","last_clean":"2024-05-03T18:24:20.917187+0000","last_became_active":"2024-05-03T18:22:25.588220+0000","last_became_peered":"2024-05-03T18:22:25.588220+0000","last_unstale":"2024-05-03T18:24:20.917187+0000","last_undegraded":"2024-05-03T18:24:20.917187+0000","last_fullsized":"2024-05-03T18:24:20.917187+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:43:20.904737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-03T18:23:20.572630+0000","last_change":"2024-05-03T18:22:25.603148+0000","last_active":"2024-05-03T18:23:20.572630+0000","last_peered":"2024-05-03T18:23:20.572630+0000","last_clean":"2024-05-03T18:23:20.572630+0000","last_became_active":"2024-05-03T18:22:25.602743+0000","last_became_peered":"2024-05-03T18:22:25.602743+0000","last_unstale":"2024-05-03T18:23:20.572630+0000","last_undegraded":"2024-05-03T18:23:20.572630+0000","last_fullsized":"2024-05-03T18:23:20.572630+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:49:27.387863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-03T18:23:20.532077+0000","last_change":"2024-05-03T18:23:20.532077+0000","last_active":"2024-05-03T18:23:20.532077+0000","last_peered":"2024-05-03T18:23:20.532077+0000","last_clean":"2024-05-03T18:23:20.532077+0000","last_became_active":"2024-05-03T18:23:20.531528+0000","last_became_peered":"2024-05-03T18:23:20.531528+0000","last_unstale":"2024-05-03T18:23:20.532077+0000","last_undegraded":"2024-05-03T18:23:20.532077+0000","last_fullsized":"2024-05-03T18:23:20.532077+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:10:37.037114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-03T18:23:20.574237+0000","last_change":"2024-05-03T18:22:25.600192+0000","last_active":"2024-05-03T18:23:20.574237+0000","last_peered":"2024-05-03T18:23:20.574237+0000","last_clean":"2024-05-03T18:23:20.574237+0000","last_became_active":"2024-05-03T18:22:25.599663+0000","last_became_peered":"2024-05-03T18:22:25.599663+0000","last_unstale":"2024-05-03T18:23:20.574237+0000","last_undegraded":"2024-05-03T18:23:20.574237+0000","last_fullsized":"2024-05-03T18:23:20.574237+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:09:51.614784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-03T18:23:20.573360+0000","last_change":"2024-05-03T18:22:25.600302+0000","last_active":"2024-05-03T18:23:20.573360+0000","last_peered":"2024-05-03T18:23:20.573360+0000","last_clean":"2024-05-03T18:23:20.573360+0000","last_became_active":"2024-05-03T18:22:25.599832+0000","last_became_peered":"2024-05-03T18:22:25.599832+0000","last_unstale":"2024-05-03T18:23:20.573360+0000","last_undegraded":"2024-05-03T18:23:20.573360+0000","last_fullsized":"2024-05-03T18:23:20.573360+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:52:12.326156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-03T18:23:20.573749+0000","last_change":"2024-05-03T18:22:25.590892+0000","last_active":"2024-05-03T18:23:20.573749+0000","last_peered":"2024-05-03T18:23:20.573749+0000","last_clean":"2024-05-03T18:23:20.573749+0000","last_became_active":"2024-05-03T18:22:25.590516+0000","last_became_peered":"2024-05-03T18:22:25.590516+0000","last_unstale":"2024-05-03T18:23:20.573749+0000","last_undegraded":"2024-05-03T18:23:20.573749+0000","last_fullsized":"2024-05-03T18:23:20.573749+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:57:57.999234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-03T18:23:20.527496+0000","last_change":"2024-05-03T18:22:25.597307+0000","last_active":"2024-05-03T18:23:20.527496+0000","last_peered":"2024-05-03T18:23:20.527496+0000","last_clean":"2024-05-03T18:23:20.527496+0000","last_became_active":"2024-05-03T18:22:25.596989+0000","last_became_peered":"2024-05-03T18:22:25.596989+0000","last_unstale":"2024-05-03T18:23:20.527496+0000","last_undegraded":"2024-05-03T18:23:20.527496+0000","last_fullsized":"2024-05-03T18:23:20.527496+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:53:09.003353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-03T18:23:20.520386+0000","last_change":"2024-05-03T18:22:25.601723+0000","last_active":"2024-05-03T18:23:20.520386+0000","last_peered":"2024-05-03T18:23:20.520386+0000","last_clean":"2024-05-03T18:23:20.520386+0000","last_became_active":"2024-05-03T18:22:25.599428+0000","last_became_peered":"2024-05-03T18:22:25.599428+0000","last_unstale":"2024-05-03T18:23:20.520386+0000","last_undegraded":"2024-05-03T18:23:20.520386+0000","last_fullsized":"2024-05-03T18:23:20.520386+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:41:18.584497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-03T18:23:20.549484+0000","last_change":"2024-05-03T18:22:25.594885+0000","last_active":"2024-05-03T18:23:20.549484+0000","last_peered":"2024-05-03T18:23:20.549484+0000","last_clean":"2024-05-03T18:23:20.549484+0000","last_became_active":"2024-05-03T18:22:25.593939+0000","last_became_peered":"2024-05-03T18:22:25.593939+0000","last_unstale":"2024-05-03T18:23:20.549484+0000","last_undegraded":"2024-05-03T18:23:20.549484+0000","last_fullsized":"2024-05-03T18:23:20.549484+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:03:30.052886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-03T18:23:20.523059+0000","last_change":"2024-05-03T18:22:25.595228+0000","last_active":"2024-05-03T18:23:20.523059+0000","last_peered":"2024-05-03T18:23:20.523059+0000","last_clean":"2024-05-03T18:23:20.523059+0000","last_became_active":"2024-05-03T18:22:25.589825+0000","last_became_peered":"2024-05-03T18:22:25.589825+0000","last_unstale":"2024-05-03T18:23:20.523059+0000","last_undegraded":"2024-05-03T18:23:20.523059+0000","last_fullsized":"2024-05-03T18:23:20.523059+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:44:05.638939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-03T18:23:20.523344+0000","last_change":"2024-05-03T18:22:25.595630+0000","last_active":"2024-05-03T18:23:20.523344+0000","last_peered":"2024-05-03T18:23:20.523344+0000","last_clean":"2024-05-03T18:23:20.523344+0000","last_became_active":"2024-05-03T18:22:25.591869+0000","last_became_peered":"2024-05-03T18:22:25.591869+0000","last_unstale":"2024-05-03T18:23:20.523344+0000","last_undegraded":"2024-05-03T18:23:20.523344+0000","last_fullsized":"2024-05-03T18:23:20.523344+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:27:43.579329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-03T18:23:20.572080+0000","last_change":"2024-05-03T18:22:25.589482+0000","last_active":"2024-05-03T18:23:20.572080+0000","last_peered":"2024-05-03T18:23:20.572080+0000","last_clean":"2024-05-03T18:23:20.572080+0000","last_became_active":"2024-05-03T18:22:25.589142+0000","last_became_peered":"2024-05-03T18:22:25.589142+0000","last_unstale":"2024-05-03T18:23:20.572080+0000","last_undegraded":"2024-05-03T18:23:20.572080+0000","last_fullsized":"2024-05-03T18:23:20.572080+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:37:55.472823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-03T18:24:20.539368+0000","last_change":"2024-05-03T18:22:25.594569+0000","last_active":"2024-05-03T18:24:20.539368+0000","last_peered":"2024-05-03T18:24:20.539368+0000","last_clean":"2024-05-03T18:24:20.539368+0000","last_became_active":"2024-05-03T18:22:25.594218+0000","last_became_peered":"2024-05-03T18:22:25.594218+0000","last_unstale":"2024-05-03T18:24:20.539368+0000","last_undegraded":"2024-05-03T18:24:20.539368+0000","last_fullsized":"2024-05-03T18:24:20.539368+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:30:42.775673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-03T18:23:20.528263+0000","last_change":"2024-05-03T18:22:25.597426+0000","last_active":"2024-05-03T18:23:20.528263+0000","last_peered":"2024-05-03T18:23:20.528263+0000","last_clean":"2024-05-03T18:23:20.528263+0000","last_became_active":"2024-05-03T18:22:25.594524+0000","last_became_peered":"2024-05-03T18:22:25.594524+0000","last_unstale":"2024-05-03T18:23:20.528263+0000","last_undegraded":"2024-05-03T18:23:20.528263+0000","last_fullsized":"2024-05-03T18:23:20.528263+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:41:36.984253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-03T18:24:20.923388+0000","last_change":"2024-05-03T18:22:25.588002+0000","last_active":"2024-05-03T18:24:20.923388+0000","last_peered":"2024-05-03T18:24:20.923388+0000","last_clean":"2024-05-03T18:24:20.923388+0000","last_became_active":"2024-05-03T18:22:25.587547+0000","last_became_peered":"2024-05-03T18:22:25.587547+0000","last_unstale":"2024-05-03T18:24:20.923388+0000","last_undegraded":"2024-05-03T18:24:20.923388+0000","last_fullsized":"2024-05-03T18:24:20.923388+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:47:14.115517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-03T18:24:20.921989+0000","last_change":"2024-05-03T18:22:40.754602+0000","last_active":"2024-05-03T18:24:20.921989+0000","last_peered":"2024-05-03T18:24:20.921989+0000","last_clean":"2024-05-03T18:24:20.921989+0000","last_became_active":"2024-05-03T18:22:40.752917+0000","last_became_peered":"2024-05-03T18:22:40.752917+0000","last_unstale":"2024-05-03T18:24:20.921989+0000","last_undegraded":"2024-05-03T18:24:20.921989+0000","last_fullsized":"2024-05-03T18:24:20.921989+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:51:44.711811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"78'4","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-03T18:23:20.527885+0000","last_change":"2024-05-03T18:22:21.558560+0000","last_active":"2024-05-03T18:23:20.527885+0000","last_peered":"2024-05-03T18:23:20.527885+0000","last_clean":"2024-05-03T18:23:20.527885+0000","last_became_active":"2024-05-03T18:22:21.558185+0000","last_became_peered":"2024-05-03T18:22:21.558185+0000","last_unstale":"2024-05-03T18:23:20.527885+0000","last_undegraded":"2024-05-03T18:23:20.527885+0000","last_fullsized":"2024-05-03T18:23:20.527885+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:18:19.666724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-03T18:23:20.574799+0000","last_change":"2024-05-03T18:22:25.603934+0000","last_active":"2024-05-03T18:23:20.574799+0000","last_peered":"2024-05-03T18:23:20.574799+0000","last_clean":"2024-05-03T18:23:20.574799+0000","last_became_active":"2024-05-03T18:22:25.603529+0000","last_became_peered":"2024-05-03T18:22:25.603529+0000","last_unstale":"2024-05-03T18:23:20.574799+0000","last_undegraded":"2024-05-03T18:23:20.574799+0000","last_fullsized":"2024-05-03T18:23:20.574799+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:08:50.288871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-03T18:23:20.576254+0000","last_change":"2024-05-03T18:23:20.576254+0000","last_active":"2024-05-03T18:23:20.576254+0000","last_peered":"2024-05-03T18:23:20.576254+0000","last_clean":"2024-05-03T18:23:20.576254+0000","last_became_active":"2024-05-03T18:23:20.575774+0000","last_became_peered":"2024-05-03T18:23:20.575774+0000","last_unstale":"2024-05-03T18:23:20.576254+0000","last_undegraded":"2024-05-03T18:23:20.576254+0000","last_fullsized":"2024-05-03T18:23:20.576254+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:22:39.090938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"78'1","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-03T18:23:20.575211+0000","last_change":"2024-05-03T18:22:21.734171+0000","last_active":"2024-05-03T18:23:20.575211+0000","last_peered":"2024-05-03T18:23:20.575211+0000","last_clean":"2024-05-03T18:23:20.575211+0000","last_became_active":"2024-05-03T18:22:21.733301+0000","last_became_peered":"2024-05-03T18:22:21.733301+0000","last_unstale":"2024-05-03T18:23:20.575211+0000","last_undegraded":"2024-05-03T18:23:20.575211+0000","last_fullsized":"2024-05-03T18:23:20.575211+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:09:48.861247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-03T18:23:20.525669+0000","last_change":"2024-05-03T18:22:25.598016+0000","last_active":"2024-05-03T18:23:20.525669+0000","last_peered":"2024-05-03T18:23:20.525669+0000","last_clean":"2024-05-03T18:23:20.525669+0000","last_became_active":"2024-05-03T18:22:25.594888+0000","last_became_peered":"2024-05-03T18:22:25.594888+0000","last_unstale":"2024-05-03T18:23:20.525669+0000","last_undegraded":"2024-05-03T18:23:20.525669+0000","last_fullsized":"2024-05-03T18:23:20.525669+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:47:46.400247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-03T18:23:20.519990+0000","last_change":"2024-05-03T18:22:40.755148+0000","last_active":"2024-05-03T18:23:20.519990+0000","last_peered":"2024-05-03T18:23:20.519990+0000","last_clean":"2024-05-03T18:23:20.519990+0000","last_became_active":"2024-05-03T18:22:40.754656+0000","last_became_peered":"2024-05-03T18:22:40.754656+0000","last_unstale":"2024-05-03T18:23:20.519990+0000","last_undegraded":"2024-05-03T18:23:20.519990+0000","last_fullsized":"2024-05-03T18:23:20.519990+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:48:23.124936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"78'2","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-03T18:24:20.917791+0000","last_change":"2024-05-03T18:22:21.568060+0000","last_active":"2024-05-03T18:24:20.917791+0000","last_peered":"2024-05-03T18:24:20.917791+0000","last_clean":"2024-05-03T18:24:20.917791+0000","last_became_active":"2024-05-03T18:22:21.567434+0000","last_became_peered":"2024-05-03T18:22:21.567434+0000","last_unstale":"2024-05-03T18:24:20.917791+0000","last_undegraded":"2024-05-03T18:24:20.917791+0000","last_fullsized":"2024-05-03T18:24:20.917791+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:06:05.802621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-03T18:23:20.573299+0000","last_change":"2024-05-03T18:22:25.597777+0000","last_active":"2024-05-03T18:23:20.573299+0000","last_peered":"2024-05-03T18:23:20.573299+0000","last_clean":"2024-05-03T18:23:20.573299+0000","last_became_active":"2024-05-03T18:22:25.597391+0000","last_became_peered":"2024-05-03T18:22:25.597391+0000","last_unstale":"2024-05-03T18:23:20.573299+0000","last_undegraded":"2024-05-03T18:23:20.573299+0000","last_fullsized":"2024-05-03T18:23:20.573299+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:02:24.708193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-03T18:23:20.520629+0000","last_change":"2024-05-03T18:22:40.752318+0000","last_active":"2024-05-03T18:23:20.520629+0000","last_peered":"2024-05-03T18:23:20.520629+0000","last_clean":"2024-05-03T18:23:20.520629+0000","last_became_active":"2024-05-03T18:22:40.751433+0000","last_became_peered":"2024-05-03T18:22:40.751433+0000","last_unstale":"2024-05-03T18:23:20.520629+0000","last_undegraded":"2024-05-03T18:23:20.520629+0000","last_fullsized":"2024-05-03T18:23:20.520629+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:21:55.746143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-03T18:23:20.520232+0000","last_change":"2024-05-03T18:22:21.560646+0000","last_active":"2024-05-03T18:23:20.520232+0000","last_peered":"2024-05-03T18:23:20.520232+0000","last_clean":"2024-05-03T18:23:20.520232+0000","last_became_active":"2024-05-03T18:22:21.560138+0000","last_became_peered":"2024-05-03T18:22:21.560138+0000","last_unstale":"2024-05-03T18:23:20.520232+0000","last_undegraded":"2024-05-03T18:23:20.520232+0000","last_fullsized":"2024-05-03T18:23:20.520232+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:10:43.967781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-03T18:24:20.908752+0000","last_change":"2024-05-03T18:23:20.575102+0000","last_active":"2024-05-03T18:24:20.908752+0000","last_peered":"2024-05-03T18:24:20.908752+0000","last_clean":"2024-05-03T18:24:20.908752+0000","last_became_active":"2024-05-03T18:23:20.574657+0000","last_became_peered":"2024-05-03T18:23:20.574657+0000","last_unstale":"2024-05-03T18:24:20.908752+0000","last_undegraded":"2024-05-03T18:24:20.908752+0000","last_fullsized":"2024-05-03T18:24:20.908752+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:08:26.280866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-03T18:24:20.922780+0000","last_change":"2024-05-03T18:22:40.761052+0000","last_active":"2024-05-03T18:24:20.922780+0000","last_peered":"2024-05-03T18:24:20.922780+0000","last_clean":"2024-05-03T18:24:20.922780+0000","last_became_active":"2024-05-03T18:22:40.760179+0000","last_became_peered":"2024-05-03T18:22:40.760179+0000","last_unstale":"2024-05-03T18:24:20.922780+0000","last_undegraded":"2024-05-03T18:24:20.922780+0000","last_fullsized":"2024-05-03T18:24:20.922780+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:44:48.610680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"78'1","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-03T18:24:20.909244+0000","last_change":"2024-05-03T18:22:21.563421+0000","last_active":"2024-05-03T18:24:20.909244+0000","last_peered":"2024-05-03T18:24:20.909244+0000","last_clean":"2024-05-03T18:24:20.909244+0000","last_became_active":"2024-05-03T18:22:21.560991+0000","last_became_peered":"2024-05-03T18:22:21.560991+0000","last_unstale":"2024-05-03T18:24:20.909244+0000","last_undegraded":"2024-05-03T18:24:20.909244+0000","last_fullsized":"2024-05-03T18:24:20.909244+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:50:32.640699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-03T18:24:20.916187+0000","last_change":"2024-05-03T18:22:25.599049+0000","last_active":"2024-05-03T18:24:20.916187+0000","last_peered":"2024-05-03T18:24:20.916187+0000","last_clean":"2024-05-03T18:24:20.916187+0000","last_became_active":"2024-05-03T18:22:25.597879+0000","last_became_peered":"2024-05-03T18:22:25.597879+0000","last_unstale":"2024-05-03T18:24:20.916187+0000","last_undegraded":"2024-05-03T18:24:20.916187+0000","last_fullsized":"2024-05-03T18:24:20.916187+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:51:08.487499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-03T18:24:20.922822+0000","last_change":"2024-05-03T18:22:40.760908+0000","last_active":"2024-05-03T18:24:20.922822+0000","last_peered":"2024-05-03T18:24:20.922822+0000","last_clean":"2024-05-03T18:24:20.922822+0000","last_became_active":"2024-05-03T18:22:40.760331+0000","last_became_peered":"2024-05-03T18:22:40.760331+0000","last_unstale":"2024-05-03T18:24:20.922822+0000","last_undegraded":"2024-05-03T18:24:20.922822+0000","last_fullsized":"2024-05-03T18:24:20.922822+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:27:08.216015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"78'1","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-03T18:23:20.572489+0000","last_change":"2024-05-03T18:22:21.565328+0000","last_active":"2024-05-03T18:23:20.572489+0000","last_peered":"2024-05-03T18:23:20.572489+0000","last_clean":"2024-05-03T18:23:20.572489+0000","last_became_active":"2024-05-03T18:22:21.564583+0000","last_became_peered":"2024-05-03T18:22:21.564583+0000","last_unstale":"2024-05-03T18:23:20.572489+0000","last_undegraded":"2024-05-03T18:23:20.572489+0000","last_fullsized":"2024-05-03T18:23:20.572489+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:58:32.370458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-03T18:24:20.909560+0000","last_change":"2024-05-03T18:22:25.607585+0000","last_active":"2024-05-03T18:24:20.909560+0000","last_peered":"2024-05-03T18:24:20.909560+0000","last_clean":"2024-05-03T18:24:20.909560+0000","last_became_active":"2024-05-03T18:22:25.607168+0000","last_became_peered":"2024-05-03T18:22:25.607168+0000","last_unstale":"2024-05-03T18:24:20.909560+0000","last_undegraded":"2024-05-03T18:24:20.909560+0000","last_fullsized":"2024-05-03T18:24:20.909560+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:04:39.750823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-03T18:24:20.539356+0000","last_change":"2024-05-03T18:22:40.754744+0000","last_active":"2024-05-03T18:24:20.539356+0000","last_peered":"2024-05-03T18:24:20.539356+0000","last_clean":"2024-05-03T18:24:20.539356+0000","last_became_active":"2024-05-03T18:22:40.754262+0000","last_became_peered":"2024-05-03T18:22:40.754262+0000","last_unstale":"2024-05-03T18:24:20.539356+0000","last_undegraded":"2024-05-03T18:24:20.539356+0000","last_fullsized":"2024-05-03T18:24:20.539356+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:11:47.979546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-03T18:23:20.548506+0000","last_change":"2024-05-03T18:22:21.550048+0000","last_active":"2024-05-03T18:23:20.548506+0000","last_peered":"2024-05-03T18:23:20.548506+0000","last_clean":"2024-05-03T18:23:20.548506+0000","last_became_active":"2024-05-03T18:22:21.549334+0000","last_became_peered":"2024-05-03T18:22:21.549334+0000","last_unstale":"2024-05-03T18:23:20.548506+0000","last_undegraded":"2024-05-03T18:23:20.548506+0000","last_fullsized":"2024-05-03T18:23:20.548506+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:52:54.401185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-03T18:23:20.574215+0000","last_change":"2024-05-03T18:22:25.583074+0000","last_active":"2024-05-03T18:23:20.574215+0000","last_peered":"2024-05-03T18:23:20.574215+0000","last_clean":"2024-05-03T18:23:20.574215+0000","last_became_active":"2024-05-03T18:22:25.582637+0000","last_became_peered":"2024-05-03T18:22:25.582637+0000","last_unstale":"2024-05-03T18:23:20.574215+0000","last_undegraded":"2024-05-03T18:23:20.574215+0000","last_fullsized":"2024-05-03T18:23:20.574215+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:20:09.021402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-03T18:23:20.524326+0000","last_change":"2024-05-03T18:22:40.748509+0000","last_active":"2024-05-03T18:23:20.524326+0000","last_peered":"2024-05-03T18:23:20.524326+0000","last_clean":"2024-05-03T18:23:20.524326+0000","last_became_active":"2024-05-03T18:22:40.748051+0000","last_became_peered":"2024-05-03T18:22:40.748051+0000","last_unstale":"2024-05-03T18:23:20.524326+0000","last_undegraded":"2024-05-03T18:23:20.524326+0000","last_fullsized":"2024-05-03T18:23:20.524326+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:08:44.389460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-03T18:24:20.908131+0000","last_change":"2024-05-03T18:22:21.562185+0000","last_active":"2024-05-03T18:24:20.908131+0000","last_peered":"2024-05-03T18:24:20.908131+0000","last_clean":"2024-05-03T18:24:20.908131+0000","last_became_active":"2024-05-03T18:22:21.560523+0000","last_became_peered":"2024-05-03T18:22:21.560523+0000","last_unstale":"2024-05-03T18:24:20.908131+0000","last_undegraded":"2024-05-03T18:24:20.908131+0000","last_fullsized":"2024-05-03T18:24:20.908131+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:05:41.788286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-03T18:23:20.525662+0000","last_change":"2024-05-03T18:22:25.583895+0000","last_active":"2024-05-03T18:23:20.525662+0000","last_peered":"2024-05-03T18:23:20.525662+0000","last_clean":"2024-05-03T18:23:20.525662+0000","last_became_active":"2024-05-03T18:22:25.583503+0000","last_became_peered":"2024-05-03T18:22:25.583503+0000","last_unstale":"2024-05-03T18:23:20.525662+0000","last_undegraded":"2024-05-03T18:23:20.525662+0000","last_fullsized":"2024-05-03T18:23:20.525662+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:45:32.404394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-03T18:23:20.573928+0000","last_change":"2024-05-03T18:22:40.752814+0000","last_active":"2024-05-03T18:23:20.573928+0000","last_peered":"2024-05-03T18:23:20.573928+0000","last_clean":"2024-05-03T18:23:20.573928+0000","last_became_active":"2024-05-03T18:22:40.752328+0000","last_became_peered":"2024-05-03T18:22:40.752328+0000","last_unstale":"2024-05-03T18:23:20.573928+0000","last_undegraded":"2024-05-03T18:23:20.573928+0000","last_fullsized":"2024-05-03T18:23:20.573928+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:16:39.488122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-03T18:23:20.572852+0000","last_change":"2024-05-03T18:22:21.566266+0000","last_active":"2024-05-03T18:23:20.572852+0000","last_peered":"2024-05-03T18:23:20.572852+0000","last_clean":"2024-05-03T18:23:20.572852+0000","last_became_active":"2024-05-03T18:22:21.565612+0000","last_became_peered":"2024-05-03T18:22:21.565612+0000","last_unstale":"2024-05-03T18:23:20.572852+0000","last_undegraded":"2024-05-03T18:23:20.572852+0000","last_fullsized":"2024-05-03T18:23:20.572852+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:22:57.216912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-03T18:23:20.573919+0000","last_change":"2024-05-03T18:22:25.596182+0000","last_active":"2024-05-03T18:23:20.573919+0000","last_peered":"2024-05-03T18:23:20.573919+0000","last_clean":"2024-05-03T18:23:20.573919+0000","last_became_active":"2024-05-03T18:22:25.595472+0000","last_became_peered":"2024-05-03T18:22:25.595472+0000","last_unstale":"2024-05-03T18:23:20.573919+0000","last_undegraded":"2024-05-03T18:23:20.573919+0000","last_fullsized":"2024-05-03T18:23:20.573919+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:48:49.583879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-03T18:23:20.524208+0000","last_change":"2024-05-03T18:22:40.756378+0000","last_active":"2024-05-03T18:23:20.524208+0000","last_peered":"2024-05-03T18:23:20.524208+0000","last_clean":"2024-05-03T18:23:20.524208+0000","last_became_active":"2024-05-03T18:22:40.755692+0000","last_became_peered":"2024-05-03T18:22:40.755692+0000","last_unstale":"2024-05-03T18:23:20.524208+0000","last_undegraded":"2024-05-03T18:23:20.524208+0000","last_fullsized":"2024-05-03T18:23:20.524208+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:22:50.269817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-03T18:23:20.527358+0000","last_change":"2024-05-03T18:22:21.561381+0000","last_active":"2024-05-03T18:23:20.527358+0000","last_peered":"2024-05-03T18:23:20.527358+0000","last_clean":"2024-05-03T18:23:20.527358+0000","last_became_active":"2024-05-03T18:22:21.559411+0000","last_became_peered":"2024-05-03T18:22:21.559411+0000","last_unstale":"2024-05-03T18:23:20.527358+0000","last_undegraded":"2024-05-03T18:23:20.527358+0000","last_fullsized":"2024-05-03T18:23:20.527358+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:04:08.283538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-03T18:23:20.551361+0000","last_change":"2024-05-03T18:23:20.551361+0000","last_active":"2024-05-03T18:23:20.551361+0000","last_peered":"2024-05-03T18:23:20.551361+0000","last_clean":"2024-05-03T18:23:20.551361+0000","last_became_active":"2024-05-03T18:23:20.550689+0000","last_became_peered":"2024-05-03T18:23:20.550689+0000","last_unstale":"2024-05-03T18:23:20.551361+0000","last_undegraded":"2024-05-03T18:23:20.551361+0000","last_fullsized":"2024-05-03T18:23:20.551361+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:36:52.767110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-03T18:23:20.520189+0000","last_change":"2024-05-03T18:22:40.758006+0000","last_active":"2024-05-03T18:23:20.520189+0000","last_peered":"2024-05-03T18:23:20.520189+0000","last_clean":"2024-05-03T18:23:20.520189+0000","last_became_active":"2024-05-03T18:22:40.757075+0000","last_became_peered":"2024-05-03T18:22:40.757075+0000","last_unstale":"2024-05-03T18:23:20.520189+0000","last_undegraded":"2024-05-03T18:23:20.520189+0000","last_fullsized":"2024-05-03T18:23:20.520189+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:06:02.390184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"78'3","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-03T18:24:20.908745+0000","last_change":"2024-05-03T18:22:21.740041+0000","last_active":"2024-05-03T18:24:20.908745+0000","last_peered":"2024-05-03T18:24:20.908745+0000","last_clean":"2024-05-03T18:24:20.908745+0000","last_became_active":"2024-05-03T18:22:21.738777+0000","last_became_peered":"2024-05-03T18:22:21.738777+0000","last_unstale":"2024-05-03T18:24:20.908745+0000","last_undegraded":"2024-05-03T18:24:20.908745+0000","last_fullsized":"2024-05-03T18:24:20.908745+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:35:30.098325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-03T18:23:20.572938+0000","last_change":"2024-05-03T18:22:25.606854+0000","last_active":"2024-05-03T18:23:20.572938+0000","last_peered":"2024-05-03T18:23:20.572938+0000","last_clean":"2024-05-03T18:23:20.572938+0000","last_became_active":"2024-05-03T18:22:25.606376+0000","last_became_peered":"2024-05-03T18:22:25.606376+0000","last_unstale":"2024-05-03T18:23:20.572938+0000","last_undegraded":"2024-05-03T18:23:20.572938+0000","last_fullsized":"2024-05-03T18:23:20.572938+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:29:59.461181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-03T18:23:20.526456+0000","last_change":"2024-05-03T18:22:40.752863+0000","last_active":"2024-05-03T18:23:20.526456+0000","last_peered":"2024-05-03T18:23:20.526456+0000","last_clean":"2024-05-03T18:23:20.526456+0000","last_became_active":"2024-05-03T18:22:40.752309+0000","last_became_peered":"2024-05-03T18:22:40.752309+0000","last_unstale":"2024-05-03T18:23:20.526456+0000","last_undegraded":"2024-05-03T18:23:20.526456+0000","last_fullsized":"2024-05-03T18:23:20.526456+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:01:06.110570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"87'30","reported_seq":142,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-03T18:24:16.886801+0000","last_change":"2024-05-03T18:22:21.566162+0000","last_active":"2024-05-03T18:24:16.886801+0000","last_peered":"2024-05-03T18:24:16.886801+0000","last_clean":"2024-05-03T18:24:16.886801+0000","last_became_active":"2024-05-03T18:22:21.565641+0000","last_became_peered":"2024-05-03T18:22:21.565641+0000","last_unstale":"2024-05-03T18:24:16.886801+0000","last_undegraded":"2024-05-03T18:24:16.886801+0000","last_fullsized":"2024-05-03T18:24:16.886801+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:23:45.283135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56115,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":72,"num_read_kb":91,"num_write":30,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-03T18:24:20.918228+0000","last_change":"2024-05-03T18:22:25.586687+0000","last_active":"2024-05-03T18:24:20.918228+0000","last_peered":"2024-05-03T18:24:20.918228+0000","last_clean":"2024-05-03T18:24:20.918228+0000","last_became_active":"2024-05-03T18:22:25.586275+0000","last_became_peered":"2024-05-03T18:22:25.586275+0000","last_unstale":"2024-05-03T18:24:20.918228+0000","last_undegraded":"2024-05-03T18:24:20.918228+0000","last_fullsized":"2024-05-03T18:24:20.918228+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:10:50.339865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-03T18:24:20.909336+0000","last_change":"2024-05-03T18:22:40.758003+0000","last_active":"2024-05-03T18:24:20.909336+0000","last_peered":"2024-05-03T18:24:20.909336+0000","last_clean":"2024-05-03T18:24:20.909336+0000","last_became_active":"2024-05-03T18:22:40.757559+0000","last_became_peered":"2024-05-03T18:22:40.757559+0000","last_unstale":"2024-05-03T18:24:20.909336+0000","last_undegraded":"2024-05-03T18:24:20.909336+0000","last_fullsized":"2024-05-03T18:24:20.909336+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:54:23.377748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"78'2","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-03T18:24:20.922567+0000","last_change":"2024-05-03T18:22:21.564546+0000","last_active":"2024-05-03T18:24:20.922567+0000","last_peered":"2024-05-03T18:24:20.922567+0000","last_clean":"2024-05-03T18:24:20.922567+0000","last_became_active":"2024-05-03T18:22:21.564105+0000","last_became_peered":"2024-05-03T18:22:21.564105+0000","last_unstale":"2024-05-03T18:24:20.922567+0000","last_undegraded":"2024-05-03T18:24:20.922567+0000","last_fullsized":"2024-05-03T18:24:20.922567+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:36:07.790643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-03T18:23:20.548886+0000","last_change":"2024-05-03T18:22:25.584059+0000","last_active":"2024-05-03T18:23:20.548886+0000","last_peered":"2024-05-03T18:23:20.548886+0000","last_clean":"2024-05-03T18:23:20.548886+0000","last_became_active":"2024-05-03T18:22:25.583742+0000","last_became_peered":"2024-05-03T18:22:25.583742+0000","last_unstale":"2024-05-03T18:23:20.548886+0000","last_undegraded":"2024-05-03T18:23:20.548886+0000","last_fullsized":"2024-05-03T18:23:20.548886+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:15:48.288422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-03T18:24:20.538368+0000","last_change":"2024-05-03T18:22:40.751362+0000","last_active":"2024-05-03T18:24:20.538368+0000","last_peered":"2024-05-03T18:24:20.538368+0000","last_clean":"2024-05-03T18:24:20.538368+0000","last_became_active":"2024-05-03T18:22:40.750365+0000","last_became_peered":"2024-05-03T18:22:40.750365+0000","last_unstale":"2024-05-03T18:24:20.538368+0000","last_undegraded":"2024-05-03T18:24:20.538368+0000","last_fullsized":"2024-05-03T18:24:20.538368+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:35:50.629332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-03T18:24:20.908346+0000","last_change":"2024-05-03T18:22:21.556123+0000","last_active":"2024-05-03T18:24:20.908346+0000","last_peered":"2024-05-03T18:24:20.908346+0000","last_clean":"2024-05-03T18:24:20.908346+0000","last_became_active":"2024-05-03T18:22:21.554918+0000","last_became_peered":"2024-05-03T18:22:21.554918+0000","last_unstale":"2024-05-03T18:24:20.908346+0000","last_undegraded":"2024-05-03T18:24:20.908346+0000","last_fullsized":"2024-05-03T18:24:20.908346+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:09:12.383601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-03T18:23:20.527234+0000","last_change":"2024-05-03T18:22:25.601849+0000","last_active":"2024-05-03T18:23:20.527234+0000","last_peered":"2024-05-03T18:23:20.527234+0000","last_clean":"2024-05-03T18:23:20.527234+0000","last_became_active":"2024-05-03T18:22:25.601434+0000","last_became_peered":"2024-05-03T18:22:25.601434+0000","last_unstale":"2024-05-03T18:23:20.527234+0000","last_undegraded":"2024-05-03T18:23:20.527234+0000","last_fullsized":"2024-05-03T18:23:20.527234+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:43:18.273389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-03T18:23:20.527383+0000","last_change":"2024-05-03T18:22:40.754448+0000","last_active":"2024-05-03T18:23:20.527383+0000","last_peered":"2024-05-03T18:23:20.527383+0000","last_clean":"2024-05-03T18:23:20.527383+0000","last_became_active":"2024-05-03T18:22:40.753122+0000","last_became_peered":"2024-05-03T18:22:40.753122+0000","last_unstale":"2024-05-03T18:23:20.527383+0000","last_undegraded":"2024-05-03T18:23:20.527383+0000","last_fullsized":"2024-05-03T18:23:20.527383+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:43:15.216222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-03T18:23:20.519827+0000","last_change":"2024-05-03T18:22:21.559292+0000","last_active":"2024-05-03T18:23:20.519827+0000","last_peered":"2024-05-03T18:23:20.519827+0000","last_clean":"2024-05-03T18:23:20.519827+0000","last_became_active":"2024-05-03T18:22:21.557657+0000","last_became_peered":"2024-05-03T18:22:21.557657+0000","last_unstale":"2024-05-03T18:23:20.519827+0000","last_undegraded":"2024-05-03T18:23:20.519827+0000","last_fullsized":"2024-05-03T18:23:20.519827+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:51:57.709387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-03T18:23:20.526726+0000","last_change":"2024-05-03T18:22:25.604340+0000","last_active":"2024-05-03T18:23:20.526726+0000","last_peered":"2024-05-03T18:23:20.526726+0000","last_clean":"2024-05-03T18:23:20.526726+0000","last_became_active":"2024-05-03T18:22:25.603651+0000","last_became_peered":"2024-05-03T18:22:25.603651+0000","last_unstale":"2024-05-03T18:23:20.526726+0000","last_undegraded":"2024-05-03T18:23:20.526726+0000","last_fullsized":"2024-05-03T18:23:20.526726+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:49:36.379556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-03T18:23:20.549953+0000","last_change":"2024-05-03T18:22:40.738065+0000","last_active":"2024-05-03T18:23:20.549953+0000","last_peered":"2024-05-03T18:23:20.549953+0000","last_clean":"2024-05-03T18:23:20.549953+0000","last_became_active":"2024-05-03T18:22:40.737546+0000","last_became_peered":"2024-05-03T18:22:40.737546+0000","last_unstale":"2024-05-03T18:23:20.549953+0000","last_undegraded":"2024-05-03T18:23:20.549953+0000","last_fullsized":"2024-05-03T18:23:20.549953+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:45:31.528581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"78'8","reported_seq":50,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-03T18:24:20.915893+0000","last_change":"2024-05-03T18:22:21.739131+0000","last_active":"2024-05-03T18:24:20.915893+0000","last_peered":"2024-05-03T18:24:20.915893+0000","last_clean":"2024-05-03T18:24:20.915893+0000","last_became_active":"2024-05-03T18:22:21.737243+0000","last_became_peered":"2024-05-03T18:22:21.737243+0000","last_unstale":"2024-05-03T18:24:20.915893+0000","last_undegraded":"2024-05-03T18:24:20.915893+0000","last_fullsized":"2024-05-03T18:24:20.915893+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:46:24.011140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-03T18:24:20.908222+0000","last_change":"2024-05-03T18:22:25.602011+0000","last_active":"2024-05-03T18:24:20.908222+0000","last_peered":"2024-05-03T18:24:20.908222+0000","last_clean":"2024-05-03T18:24:20.908222+0000","last_became_active":"2024-05-03T18:22:25.601604+0000","last_became_peered":"2024-05-03T18:22:25.601604+0000","last_unstale":"2024-05-03T18:24:20.908222+0000","last_undegraded":"2024-05-03T18:24:20.908222+0000","last_fullsized":"2024-05-03T18:24:20.908222+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:25:57.743203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-03T18:24:20.908583+0000","last_change":"2024-05-03T18:22:40.757885+0000","last_active":"2024-05-03T18:24:20.908583+0000","last_peered":"2024-05-03T18:24:20.908583+0000","last_clean":"2024-05-03T18:24:20.908583+0000","last_became_active":"2024-05-03T18:22:40.757208+0000","last_became_peered":"2024-05-03T18:22:40.757208+0000","last_unstale":"2024-05-03T18:24:20.908583+0000","last_undegraded":"2024-05-03T18:24:20.908583+0000","last_fullsized":"2024-05-03T18:24:20.908583+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:37:00.792138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"78'1","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-03T18:24:20.918111+0000","last_change":"2024-05-03T18:22:21.554932+0000","last_active":"2024-05-03T18:24:20.918111+0000","last_peered":"2024-05-03T18:24:20.918111+0000","last_clean":"2024-05-03T18:24:20.918111+0000","last_became_active":"2024-05-03T18:22:21.554123+0000","last_became_peered":"2024-05-03T18:22:21.554123+0000","last_unstale":"2024-05-03T18:24:20.918111+0000","last_undegraded":"2024-05-03T18:24:20.918111+0000","last_fullsized":"2024-05-03T18:24:20.918111+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:38:59.859138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-03T18:24:20.538972+0000","last_change":"2024-05-03T18:22:25.603400+0000","last_active":"2024-05-03T18:24:20.538972+0000","last_peered":"2024-05-03T18:24:20.538972+0000","last_clean":"2024-05-03T18:24:20.538972+0000","last_became_active":"2024-05-03T18:22:25.603045+0000","last_became_peered":"2024-05-03T18:22:25.603045+0000","last_unstale":"2024-05-03T18:24:20.538972+0000","last_undegraded":"2024-05-03T18:24:20.538972+0000","last_fullsized":"2024-05-03T18:24:20.538972+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:51:33.502972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-03T18:24:20.916027+0000","last_change":"2024-05-03T18:22:40.750857+0000","last_active":"2024-05-03T18:24:20.916027+0000","last_peered":"2024-05-03T18:24:20.916027+0000","last_clean":"2024-05-03T18:24:20.916027+0000","last_became_active":"2024-05-03T18:22:40.750381+0000","last_became_peered":"2024-05-03T18:22:40.750381+0000","last_unstale":"2024-05-03T18:24:20.916027+0000","last_undegraded":"2024-05-03T18:24:20.916027+0000","last_fullsized":"2024-05-03T18:24:20.916027+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:15:28.542612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-03T18:23:20.527368+0000","last_change":"2024-05-03T18:22:21.739345+0000","last_active":"2024-05-03T18:23:20.527368+0000","last_peered":"2024-05-03T18:23:20.527368+0000","last_clean":"2024-05-03T18:23:20.527368+0000","last_became_active":"2024-05-03T18:22:21.738874+0000","last_became_peered":"2024-05-03T18:22:21.738874+0000","last_unstale":"2024-05-03T18:23:20.527368+0000","last_undegraded":"2024-05-03T18:23:20.527368+0000","last_fullsized":"2024-05-03T18:23:20.527368+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:57:59.706356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-03T18:23:20.572963+0000","last_change":"2024-05-03T18:22:25.586543+0000","last_active":"2024-05-03T18:23:20.572963+0000","last_peered":"2024-05-03T18:23:20.572963+0000","last_clean":"2024-05-03T18:23:20.572963+0000","last_became_active":"2024-05-03T18:22:25.585124+0000","last_became_peered":"2024-05-03T18:22:25.585124+0000","last_unstale":"2024-05-03T18:23:20.572963+0000","last_undegraded":"2024-05-03T18:23:20.572963+0000","last_fullsized":"2024-05-03T18:23:20.572963+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:41:32.633444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-03T18:23:20.549407+0000","last_change":"2024-05-03T18:22:40.749665+0000","last_active":"2024-05-03T18:23:20.549407+0000","last_peered":"2024-05-03T18:23:20.549407+0000","last_clean":"2024-05-03T18:23:20.549407+0000","last_became_active":"2024-05-03T18:22:40.749062+0000","last_became_peered":"2024-05-03T18:22:40.749062+0000","last_unstale":"2024-05-03T18:23:20.549407+0000","last_undegraded":"2024-05-03T18:23:20.549407+0000","last_fullsized":"2024-05-03T18:23:20.549407+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:09:17.194507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-03T18:23:20.527049+0000","last_change":"2024-05-03T18:22:21.560944+0000","last_active":"2024-05-03T18:23:20.527049+0000","last_peered":"2024-05-03T18:23:20.527049+0000","last_clean":"2024-05-03T18:23:20.527049+0000","last_became_active":"2024-05-03T18:22:21.560412+0000","last_became_peered":"2024-05-03T18:22:21.560412+0000","last_unstale":"2024-05-03T18:23:20.527049+0000","last_undegraded":"2024-05-03T18:23:20.527049+0000","last_fullsized":"2024-05-03T18:23:20.527049+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:12:04.069564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-03T18:23:20.572558+0000","last_change":"2024-05-03T18:22:25.598235+0000","last_active":"2024-05-03T18:23:20.572558+0000","last_peered":"2024-05-03T18:23:20.572558+0000","last_clean":"2024-05-03T18:23:20.572558+0000","last_became_active":"2024-05-03T18:22:25.595993+0000","last_became_peered":"2024-05-03T18:22:25.595993+0000","last_unstale":"2024-05-03T18:23:20.572558+0000","last_undegraded":"2024-05-03T18:23:20.572558+0000","last_fullsized":"2024-05-03T18:23:20.572558+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:53:16.069119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-03T18:23:20.549908+0000","last_change":"2024-05-03T18:22:40.745005+0000","last_active":"2024-05-03T18:23:20.549908+0000","last_peered":"2024-05-03T18:23:20.549908+0000","last_clean":"2024-05-03T18:23:20.549908+0000","last_became_active":"2024-05-03T18:22:40.742134+0000","last_became_peered":"2024-05-03T18:22:40.742134+0000","last_unstale":"2024-05-03T18:23:20.549908+0000","last_undegraded":"2024-05-03T18:23:20.549908+0000","last_fullsized":"2024-05-03T18:23:20.549908+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:47:12.217012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-03T18:24:20.909493+0000","last_change":"2024-05-03T18:22:25.604640+0000","last_active":"2024-05-03T18:24:20.909493+0000","last_peered":"2024-05-03T18:24:20.909493+0000","last_clean":"2024-05-03T18:24:20.909493+0000","last_became_active":"2024-05-03T18:22:25.603759+0000","last_became_peered":"2024-05-03T18:22:25.603759+0000","last_unstale":"2024-05-03T18:24:20.909493+0000","last_undegraded":"2024-05-03T18:24:20.909493+0000","last_fullsized":"2024-05-03T18:24:20.909493+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:56:55.744321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-03T18:23:20.548398+0000","last_change":"2024-05-03T18:22:21.741006+0000","last_active":"2024-05-03T18:23:20.548398+0000","last_peered":"2024-05-03T18:23:20.548398+0000","last_clean":"2024-05-03T18:23:20.548398+0000","last_became_active":"2024-05-03T18:22:21.737999+0000","last_became_peered":"2024-05-03T18:22:21.737999+0000","last_unstale":"2024-05-03T18:23:20.548398+0000","last_undegraded":"2024-05-03T18:23:20.548398+0000","last_fullsized":"2024-05-03T18:23:20.548398+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:29:54.931940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-03T18:23:20.527480+0000","last_change":"2024-05-03T18:22:40.761044+0000","last_active":"2024-05-03T18:23:20.527480+0000","last_peered":"2024-05-03T18:23:20.527480+0000","last_clean":"2024-05-03T18:23:20.527480+0000","last_became_active":"2024-05-03T18:22:40.760245+0000","last_became_peered":"2024-05-03T18:22:40.760245+0000","last_unstale":"2024-05-03T18:23:20.527480+0000","last_undegraded":"2024-05-03T18:23:20.527480+0000","last_fullsized":"2024-05-03T18:23:20.527480+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:31:12.729078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-03T18:23:20.574654+0000","last_change":"2024-05-03T18:22:25.595759+0000","last_active":"2024-05-03T18:23:20.574654+0000","last_peered":"2024-05-03T18:23:20.574654+0000","last_clean":"2024-05-03T18:23:20.574654+0000","last_became_active":"2024-05-03T18:22:25.594368+0000","last_became_peered":"2024-05-03T18:22:25.594368+0000","last_unstale":"2024-05-03T18:23:20.574654+0000","last_undegraded":"2024-05-03T18:23:20.574654+0000","last_fullsized":"2024-05-03T18:23:20.574654+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:26:19.196877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-03T18:23:20.520138+0000","last_change":"2024-05-03T18:22:21.557123+0000","last_active":"2024-05-03T18:23:20.520138+0000","last_peered":"2024-05-03T18:23:20.520138+0000","last_clean":"2024-05-03T18:23:20.520138+0000","last_became_active":"2024-05-03T18:22:21.556432+0000","last_became_peered":"2024-05-03T18:22:21.556432+0000","last_unstale":"2024-05-03T18:23:20.520138+0000","last_undegraded":"2024-05-03T18:23:20.520138+0000","last_fullsized":"2024-05-03T18:23:20.520138+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:50:03.179320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":115,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-03T18:23:20.526590+0000","last_change":"2024-05-03T18:20:24.937044+0000","last_active":"2024-05-03T18:23:20.526590+0000","last_peered":"2024-05-03T18:23:20.526590+0000","last_clean":"2024-05-03T18:23:20.526590+0000","last_became_active":"2024-05-03T18:20:24.922095+0000","last_became_peered":"2024-05-03T18:20:24.922095+0000","last_unstale":"2024-05-03T18:23:20.526590+0000","last_undegraded":"2024-05-03T18:23:20.526590+0000","last_fullsized":"2024-05-03T18:23:20.526590+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:15:37.353911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:15:37.353911+0000","last_clean_scrub_stamp":"2024-05-03T18:15:37.353911+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:40:07.341441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-03T18:24:20.917001+0000","last_change":"2024-05-03T18:23:20.575064+0000","last_active":"2024-05-03T18:24:20.917001+0000","last_peered":"2024-05-03T18:24:20.917001+0000","last_clean":"2024-05-03T18:24:20.917001+0000","last_became_active":"2024-05-03T18:23:20.574557+0000","last_became_peered":"2024-05-03T18:23:20.574557+0000","last_unstale":"2024-05-03T18:24:20.917001+0000","last_undegraded":"2024-05-03T18:24:20.917001+0000","last_fullsized":"2024-05-03T18:24:20.917001+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:34:08.520396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,6,10],"acting":[8,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-03T18:23:20.548366+0000","last_change":"2024-05-03T18:22:25.591269+0000","last_active":"2024-05-03T18:23:20.548366+0000","last_peered":"2024-05-03T18:23:20.548366+0000","last_clean":"2024-05-03T18:23:20.548366+0000","last_became_active":"2024-05-03T18:22:25.590409+0000","last_became_peered":"2024-05-03T18:22:25.590409+0000","last_unstale":"2024-05-03T18:23:20.548366+0000","last_undegraded":"2024-05-03T18:23:20.548366+0000","last_fullsized":"2024-05-03T18:23:20.548366+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:03:57.432331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-03T18:23:20.526711+0000","last_change":"2024-05-03T18:22:21.560137+0000","last_active":"2024-05-03T18:23:20.526711+0000","last_peered":"2024-05-03T18:23:20.526711+0000","last_clean":"2024-05-03T18:23:20.526711+0000","last_became_active":"2024-05-03T18:22:21.559672+0000","last_became_peered":"2024-05-03T18:22:21.559672+0000","last_unstale":"2024-05-03T18:23:20.526711+0000","last_undegraded":"2024-05-03T18:23:20.526711+0000","last_fullsized":"2024-05-03T18:23:20.526711+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:20:02.070390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-03T18:24:20.538212+0000","last_change":"2024-05-03T18:22:40.753160+0000","last_active":"2024-05-03T18:24:20.538212+0000","last_peered":"2024-05-03T18:24:20.538212+0000","last_clean":"2024-05-03T18:24:20.538212+0000","last_became_active":"2024-05-03T18:22:40.750800+0000","last_became_peered":"2024-05-03T18:22:40.750800+0000","last_unstale":"2024-05-03T18:24:20.538212+0000","last_undegraded":"2024-05-03T18:24:20.538212+0000","last_fullsized":"2024-05-03T18:24:20.538212+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:27:48.756140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-03T18:24:20.917748+0000","last_change":"2024-05-03T18:22:21.739490+0000","last_active":"2024-05-03T18:24:20.917748+0000","last_peered":"2024-05-03T18:24:20.917748+0000","last_clean":"2024-05-03T18:24:20.917748+0000","last_became_active":"2024-05-03T18:22:21.738512+0000","last_became_peered":"2024-05-03T18:22:21.738512+0000","last_unstale":"2024-05-03T18:24:20.917748+0000","last_undegraded":"2024-05-03T18:24:20.917748+0000","last_fullsized":"2024-05-03T18:24:20.917748+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:26:45.323519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-03T18:23:20.549639+0000","last_change":"2024-05-03T18:22:25.591509+0000","last_active":"2024-05-03T18:23:20.549639+0000","last_peered":"2024-05-03T18:23:20.549639+0000","last_clean":"2024-05-03T18:23:20.549639+0000","last_became_active":"2024-05-03T18:22:25.590188+0000","last_became_peered":"2024-05-03T18:22:25.590188+0000","last_unstale":"2024-05-03T18:23:20.549639+0000","last_undegraded":"2024-05-03T18:23:20.549639+0000","last_fullsized":"2024-05-03T18:23:20.549639+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:33:23.919101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-03T18:23:20.573929+0000","last_change":"2024-05-03T18:22:40.747176+0000","last_active":"2024-05-03T18:23:20.573929+0000","last_peered":"2024-05-03T18:23:20.573929+0000","last_clean":"2024-05-03T18:23:20.573929+0000","last_became_active":"2024-05-03T18:22:40.746526+0000","last_became_peered":"2024-05-03T18:22:40.746526+0000","last_unstale":"2024-05-03T18:23:20.573929+0000","last_undegraded":"2024-05-03T18:23:20.573929+0000","last_fullsized":"2024-05-03T18:23:20.573929+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:59:25.838382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"87'7","reported_seq":47,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-03T18:24:06.672208+0000","last_change":"2024-05-03T18:22:21.555978+0000","last_active":"2024-05-03T18:24:06.672208+0000","last_peered":"2024-05-03T18:24:06.672208+0000","last_clean":"2024-05-03T18:24:06.672208+0000","last_became_active":"2024-05-03T18:22:21.555395+0000","last_became_peered":"2024-05-03T18:22:21.555395+0000","last_unstale":"2024-05-03T18:24:06.672208+0000","last_undegraded":"2024-05-03T18:24:06.672208+0000","last_fullsized":"2024-05-03T18:24:06.672208+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:05:44.795978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-03T18:23:20.576174+0000","last_change":"2024-05-03T18:22:25.589071+0000","last_active":"2024-05-03T18:23:20.576174+0000","last_peered":"2024-05-03T18:23:20.576174+0000","last_clean":"2024-05-03T18:23:20.576174+0000","last_became_active":"2024-05-03T18:22:25.588679+0000","last_became_peered":"2024-05-03T18:22:25.588679+0000","last_unstale":"2024-05-03T18:23:20.576174+0000","last_undegraded":"2024-05-03T18:23:20.576174+0000","last_fullsized":"2024-05-03T18:23:20.576174+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:20:53.983166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-03T18:24:20.921781+0000","last_change":"2024-05-03T18:22:40.755154+0000","last_active":"2024-05-03T18:24:20.921781+0000","last_peered":"2024-05-03T18:24:20.921781+0000","last_clean":"2024-05-03T18:24:20.921781+0000","last_became_active":"2024-05-03T18:22:40.754635+0000","last_became_peered":"2024-05-03T18:22:40.754635+0000","last_unstale":"2024-05-03T18:24:20.921781+0000","last_undegraded":"2024-05-03T18:24:20.921781+0000","last_fullsized":"2024-05-03T18:24:20.921781+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:25:56.787406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"78'2","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-03T18:23:20.521258+0000","last_change":"2024-05-03T18:22:21.854322+0000","last_active":"2024-05-03T18:23:20.521258+0000","last_peered":"2024-05-03T18:23:20.521258+0000","last_clean":"2024-05-03T18:23:20.521258+0000","last_became_active":"2024-05-03T18:22:21.853359+0000","last_became_peered":"2024-05-03T18:22:21.853359+0000","last_unstale":"2024-05-03T18:23:20.521258+0000","last_undegraded":"2024-05-03T18:23:20.521258+0000","last_fullsized":"2024-05-03T18:23:20.521258+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:06:57.061775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-03T18:23:20.524723+0000","last_change":"2024-05-03T18:22:25.605614+0000","last_active":"2024-05-03T18:23:20.524723+0000","last_peered":"2024-05-03T18:23:20.524723+0000","last_clean":"2024-05-03T18:23:20.524723+0000","last_became_active":"2024-05-03T18:22:25.604277+0000","last_became_peered":"2024-05-03T18:22:25.604277+0000","last_unstale":"2024-05-03T18:23:20.524723+0000","last_undegraded":"2024-05-03T18:23:20.524723+0000","last_fullsized":"2024-05-03T18:23:20.524723+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:17:09.812304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-03T18:23:20.520063+0000","last_change":"2024-05-03T18:22:40.763353+0000","last_active":"2024-05-03T18:23:20.520063+0000","last_peered":"2024-05-03T18:23:20.520063+0000","last_clean":"2024-05-03T18:23:20.520063+0000","last_became_active":"2024-05-03T18:22:40.762774+0000","last_became_peered":"2024-05-03T18:22:40.762774+0000","last_unstale":"2024-05-03T18:23:20.520063+0000","last_undegraded":"2024-05-03T18:23:20.520063+0000","last_fullsized":"2024-05-03T18:23:20.520063+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:38:37.247050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-03T18:23:20.572201+0000","last_change":"2024-05-03T18:22:21.740308+0000","last_active":"2024-05-03T18:23:20.572201+0000","last_peered":"2024-05-03T18:23:20.572201+0000","last_clean":"2024-05-03T18:23:20.572201+0000","last_became_active":"2024-05-03T18:22:21.739475+0000","last_became_peered":"2024-05-03T18:22:21.739475+0000","last_unstale":"2024-05-03T18:23:20.572201+0000","last_undegraded":"2024-05-03T18:23:20.572201+0000","last_fullsized":"2024-05-03T18:23:20.572201+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:58:03.479971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-03T18:24:20.915938+0000","last_change":"2024-05-03T18:22:25.588011+0000","last_active":"2024-05-03T18:24:20.915938+0000","last_peered":"2024-05-03T18:24:20.915938+0000","last_clean":"2024-05-03T18:24:20.915938+0000","last_became_active":"2024-05-03T18:22:25.587602+0000","last_became_peered":"2024-05-03T18:22:25.587602+0000","last_unstale":"2024-05-03T18:24:20.915938+0000","last_undegraded":"2024-05-03T18:24:20.915938+0000","last_fullsized":"2024-05-03T18:24:20.915938+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:18:13.186088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-03T18:23:20.527613+0000","last_change":"2024-05-03T18:22:40.755731+0000","last_active":"2024-05-03T18:23:20.527613+0000","last_peered":"2024-05-03T18:23:20.527613+0000","last_clean":"2024-05-03T18:23:20.527613+0000","last_became_active":"2024-05-03T18:22:40.755173+0000","last_became_peered":"2024-05-03T18:22:40.755173+0000","last_unstale":"2024-05-03T18:23:20.527613+0000","last_undegraded":"2024-05-03T18:23:20.527613+0000","last_fullsized":"2024-05-03T18:23:20.527613+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:36:36.548095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-03T18:23:20.573860+0000","last_change":"2024-05-03T18:22:21.561481+0000","last_active":"2024-05-03T18:23:20.573860+0000","last_peered":"2024-05-03T18:23:20.573860+0000","last_clean":"2024-05-03T18:23:20.573860+0000","last_became_active":"2024-05-03T18:22:21.560088+0000","last_became_peered":"2024-05-03T18:22:21.560088+0000","last_unstale":"2024-05-03T18:23:20.573860+0000","last_undegraded":"2024-05-03T18:23:20.573860+0000","last_fullsized":"2024-05-03T18:23:20.573860+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:45:40.184813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-03T18:24:20.538152+0000","last_change":"2024-05-03T18:22:25.584556+0000","last_active":"2024-05-03T18:24:20.538152+0000","last_peered":"2024-05-03T18:24:20.538152+0000","last_clean":"2024-05-03T18:24:20.538152+0000","last_became_active":"2024-05-03T18:22:25.583873+0000","last_became_peered":"2024-05-03T18:22:25.583873+0000","last_unstale":"2024-05-03T18:24:20.538152+0000","last_undegraded":"2024-05-03T18:24:20.538152+0000","last_fullsized":"2024-05-03T18:24:20.538152+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:28:39.028662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-03T18:23:20.520115+0000","last_change":"2024-05-03T18:22:40.763479+0000","last_active":"2024-05-03T18:23:20.520115+0000","last_peered":"2024-05-03T18:23:20.520115+0000","last_clean":"2024-05-03T18:23:20.520115+0000","last_became_active":"2024-05-03T18:22:40.762897+0000","last_became_peered":"2024-05-03T18:22:40.762897+0000","last_unstale":"2024-05-03T18:23:20.520115+0000","last_undegraded":"2024-05-03T18:23:20.520115+0000","last_fullsized":"2024-05-03T18:23:20.520115+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:00:21.090730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-03T18:24:20.921952+0000","last_change":"2024-05-03T18:22:21.562929+0000","last_active":"2024-05-03T18:24:20.921952+0000","last_peered":"2024-05-03T18:24:20.921952+0000","last_clean":"2024-05-03T18:24:20.921952+0000","last_became_active":"2024-05-03T18:22:21.560712+0000","last_became_peered":"2024-05-03T18:22:21.560712+0000","last_unstale":"2024-05-03T18:24:20.921952+0000","last_undegraded":"2024-05-03T18:24:20.921952+0000","last_fullsized":"2024-05-03T18:24:20.921952+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:31:34.722437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-03T18:23:20.575956+0000","last_change":"2024-05-03T18:22:25.581971+0000","last_active":"2024-05-03T18:23:20.575956+0000","last_peered":"2024-05-03T18:23:20.575956+0000","last_clean":"2024-05-03T18:23:20.575956+0000","last_became_active":"2024-05-03T18:22:25.581565+0000","last_became_peered":"2024-05-03T18:22:25.581565+0000","last_unstale":"2024-05-03T18:23:20.575956+0000","last_undegraded":"2024-05-03T18:23:20.575956+0000","last_fullsized":"2024-05-03T18:23:20.575956+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:10:20.607999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-03T18:23:20.520753+0000","last_change":"2024-05-03T18:22:40.761458+0000","last_active":"2024-05-03T18:23:20.520753+0000","last_peered":"2024-05-03T18:23:20.520753+0000","last_clean":"2024-05-03T18:23:20.520753+0000","last_became_active":"2024-05-03T18:22:40.760969+0000","last_became_peered":"2024-05-03T18:22:40.760969+0000","last_unstale":"2024-05-03T18:23:20.520753+0000","last_undegraded":"2024-05-03T18:23:20.520753+0000","last_fullsized":"2024-05-03T18:23:20.520753+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:19:16.804720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"78'2","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-03T18:24:20.916772+0000","last_change":"2024-05-03T18:22:21.565869+0000","last_active":"2024-05-03T18:24:20.916772+0000","last_peered":"2024-05-03T18:24:20.916772+0000","last_clean":"2024-05-03T18:24:20.916772+0000","last_became_active":"2024-05-03T18:22:21.564119+0000","last_became_peered":"2024-05-03T18:22:21.564119+0000","last_unstale":"2024-05-03T18:24:20.916772+0000","last_undegraded":"2024-05-03T18:24:20.916772+0000","last_fullsized":"2024-05-03T18:24:20.916772+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:50:29.894070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-03T18:23:20.527082+0000","last_change":"2024-05-03T18:22:25.596199+0000","last_active":"2024-05-03T18:23:20.527082+0000","last_peered":"2024-05-03T18:23:20.527082+0000","last_clean":"2024-05-03T18:23:20.527082+0000","last_became_active":"2024-05-03T18:22:25.595167+0000","last_became_peered":"2024-05-03T18:22:25.595167+0000","last_unstale":"2024-05-03T18:23:20.527082+0000","last_undegraded":"2024-05-03T18:23:20.527082+0000","last_fullsized":"2024-05-03T18:23:20.527082+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:28:56.599064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-03T18:23:20.573454+0000","last_change":"2024-05-03T18:22:40.753020+0000","last_active":"2024-05-03T18:23:20.573454+0000","last_peered":"2024-05-03T18:23:20.573454+0000","last_clean":"2024-05-03T18:23:20.573454+0000","last_became_active":"2024-05-03T18:22:40.752502+0000","last_became_peered":"2024-05-03T18:22:40.752502+0000","last_unstale":"2024-05-03T18:23:20.573454+0000","last_undegraded":"2024-05-03T18:23:20.573454+0000","last_fullsized":"2024-05-03T18:23:20.573454+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:33:51.277879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-03T18:24:20.922987+0000","last_change":"2024-05-03T18:22:21.854481+0000","last_active":"2024-05-03T18:24:20.922987+0000","last_peered":"2024-05-03T18:24:20.922987+0000","last_clean":"2024-05-03T18:24:20.922987+0000","last_became_active":"2024-05-03T18:22:21.853172+0000","last_became_peered":"2024-05-03T18:22:21.853172+0000","last_unstale":"2024-05-03T18:24:20.922987+0000","last_undegraded":"2024-05-03T18:24:20.922987+0000","last_fullsized":"2024-05-03T18:24:20.922987+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:11:47.820105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-03T18:23:20.527619+0000","last_change":"2024-05-03T18:22:25.600856+0000","last_active":"2024-05-03T18:23:20.527619+0000","last_peered":"2024-05-03T18:23:20.527619+0000","last_clean":"2024-05-03T18:23:20.527619+0000","last_became_active":"2024-05-03T18:22:25.595351+0000","last_became_peered":"2024-05-03T18:22:25.595351+0000","last_unstale":"2024-05-03T18:23:20.527619+0000","last_undegraded":"2024-05-03T18:23:20.527619+0000","last_fullsized":"2024-05-03T18:23:20.527619+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:29:24.232506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-03T18:23:20.578331+0000","last_change":"2024-05-03T18:23:20.578331+0000","last_active":"2024-05-03T18:23:20.578331+0000","last_peered":"2024-05-03T18:23:20.578331+0000","last_clean":"2024-05-03T18:23:20.578331+0000","last_became_active":"2024-05-03T18:23:20.576974+0000","last_became_peered":"2024-05-03T18:23:20.576974+0000","last_unstale":"2024-05-03T18:23:20.578331+0000","last_undegraded":"2024-05-03T18:23:20.578331+0000","last_fullsized":"2024-05-03T18:23:20.578331+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:57:32.922175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-03T18:24:20.909004+0000","last_change":"2024-05-03T18:22:21.552331+0000","last_active":"2024-05-03T18:24:20.909004+0000","last_peered":"2024-05-03T18:24:20.909004+0000","last_clean":"2024-05-03T18:24:20.909004+0000","last_became_active":"2024-05-03T18:22:21.549786+0000","last_became_peered":"2024-05-03T18:22:21.549786+0000","last_unstale":"2024-05-03T18:24:20.909004+0000","last_undegraded":"2024-05-03T18:24:20.909004+0000","last_fullsized":"2024-05-03T18:24:20.909004+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:08:18.513405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-03T18:23:20.527305+0000","last_change":"2024-05-03T18:22:25.596437+0000","last_active":"2024-05-03T18:23:20.527305+0000","last_peered":"2024-05-03T18:23:20.527305+0000","last_clean":"2024-05-03T18:23:20.527305+0000","last_became_active":"2024-05-03T18:22:25.594040+0000","last_became_peered":"2024-05-03T18:22:25.594040+0000","last_unstale":"2024-05-03T18:23:20.527305+0000","last_undegraded":"2024-05-03T18:23:20.527305+0000","last_fullsized":"2024-05-03T18:23:20.527305+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:09:36.473565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-03T18:24:20.907890+0000","last_change":"2024-05-03T18:22:40.754641+0000","last_active":"2024-05-03T18:24:20.907890+0000","last_peered":"2024-05-03T18:24:20.907890+0000","last_clean":"2024-05-03T18:24:20.907890+0000","last_became_active":"2024-05-03T18:22:40.754166+0000","last_became_peered":"2024-05-03T18:22:40.754166+0000","last_unstale":"2024-05-03T18:24:20.907890+0000","last_undegraded":"2024-05-03T18:24:20.907890+0000","last_fullsized":"2024-05-03T18:24:20.907890+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:29:50.021671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-03T18:23:20.549912+0000","last_change":"2024-05-03T18:22:21.559354+0000","last_active":"2024-05-03T18:23:20.549912+0000","last_peered":"2024-05-03T18:23:20.549912+0000","last_clean":"2024-05-03T18:23:20.549912+0000","last_became_active":"2024-05-03T18:22:21.558680+0000","last_became_peered":"2024-05-03T18:22:21.558680+0000","last_unstale":"2024-05-03T18:23:20.549912+0000","last_undegraded":"2024-05-03T18:23:20.549912+0000","last_fullsized":"2024-05-03T18:23:20.549912+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:13:04.001847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-03T18:23:20.527647+0000","last_change":"2024-05-03T18:22:25.590746+0000","last_active":"2024-05-03T18:23:20.527647+0000","last_peered":"2024-05-03T18:23:20.527647+0000","last_clean":"2024-05-03T18:23:20.527647+0000","last_became_active":"2024-05-03T18:22:25.587906+0000","last_became_peered":"2024-05-03T18:22:25.587906+0000","last_unstale":"2024-05-03T18:23:20.527647+0000","last_undegraded":"2024-05-03T18:23:20.527647+0000","last_fullsized":"2024-05-03T18:23:20.527647+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:19:43.410259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-03T18:24:20.538494+0000","last_change":"2024-05-03T18:22:25.597628+0000","last_active":"2024-05-03T18:24:20.538494+0000","last_peered":"2024-05-03T18:24:20.538494+0000","last_clean":"2024-05-03T18:24:20.538494+0000","last_became_active":"2024-05-03T18:22:25.596535+0000","last_became_peered":"2024-05-03T18:22:25.596535+0000","last_unstale":"2024-05-03T18:24:20.538494+0000","last_undegraded":"2024-05-03T18:24:20.538494+0000","last_fullsized":"2024-05-03T18:24:20.538494+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:54:52.632258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-03T18:24:20.922413+0000","last_change":"2024-05-03T18:22:25.601474+0000","last_active":"2024-05-03T18:24:20.922413+0000","last_peered":"2024-05-03T18:24:20.922413+0000","last_clean":"2024-05-03T18:24:20.922413+0000","last_became_active":"2024-05-03T18:22:25.601097+0000","last_became_peered":"2024-05-03T18:22:25.601097+0000","last_unstale":"2024-05-03T18:24:20.922413+0000","last_undegraded":"2024-05-03T18:24:20.922413+0000","last_fullsized":"2024-05-03T18:24:20.922413+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:14:46.001500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-03T18:24:20.921940+0000","last_change":"2024-05-03T18:23:20.535409+0000","last_active":"2024-05-03T18:24:20.921940+0000","last_peered":"2024-05-03T18:24:20.921940+0000","last_clean":"2024-05-03T18:24:20.921940+0000","last_became_active":"2024-05-03T18:23:20.534883+0000","last_became_peered":"2024-05-03T18:23:20.534883+0000","last_unstale":"2024-05-03T18:24:20.921940+0000","last_undegraded":"2024-05-03T18:24:20.921940+0000","last_fullsized":"2024-05-03T18:24:20.921940+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:26:37.252898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-03T18:23:20.524567+0000","last_change":"2024-05-03T18:22:25.607115+0000","last_active":"2024-05-03T18:23:20.524567+0000","last_peered":"2024-05-03T18:23:20.524567+0000","last_clean":"2024-05-03T18:23:20.524567+0000","last_became_active":"2024-05-03T18:22:25.606764+0000","last_became_peered":"2024-05-03T18:22:25.606764+0000","last_unstale":"2024-05-03T18:23:20.524567+0000","last_undegraded":"2024-05-03T18:23:20.524567+0000","last_fullsized":"2024-05-03T18:23:20.524567+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:44:37.165032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-03T18:23:20.549890+0000","last_change":"2024-05-03T18:22:25.601886+0000","last_active":"2024-05-03T18:23:20.549890+0000","last_peered":"2024-05-03T18:23:20.549890+0000","last_clean":"2024-05-03T18:23:20.549890+0000","last_became_active":"2024-05-03T18:22:25.601494+0000","last_became_peered":"2024-05-03T18:22:25.601494+0000","last_unstale":"2024-05-03T18:23:20.549890+0000","last_undegraded":"2024-05-03T18:23:20.549890+0000","last_fullsized":"2024-05-03T18:23:20.549890+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:35:30.080124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-03T18:24:20.539233+0000","last_change":"2024-05-03T18:23:20.529091+0000","last_active":"2024-05-03T18:24:20.539233+0000","last_peered":"2024-05-03T18:24:20.539233+0000","last_clean":"2024-05-03T18:24:20.539233+0000","last_became_active":"2024-05-03T18:23:20.528288+0000","last_became_peered":"2024-05-03T18:23:20.528288+0000","last_unstale":"2024-05-03T18:24:20.539233+0000","last_undegraded":"2024-05-03T18:24:20.539233+0000","last_fullsized":"2024-05-03T18:24:20.539233+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:32:25.749461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-03T18:24:20.923462+0000","last_change":"2024-05-03T18:22:25.595367+0000","last_active":"2024-05-03T18:24:20.923462+0000","last_peered":"2024-05-03T18:24:20.923462+0000","last_clean":"2024-05-03T18:24:20.923462+0000","last_became_active":"2024-05-03T18:22:25.594987+0000","last_became_peered":"2024-05-03T18:22:25.594987+0000","last_unstale":"2024-05-03T18:24:20.923462+0000","last_undegraded":"2024-05-03T18:24:20.923462+0000","last_fullsized":"2024-05-03T18:24:20.923462+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:40:55.385595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-03T18:23:20.576517+0000","last_change":"2024-05-03T18:22:25.588320+0000","last_active":"2024-05-03T18:23:20.576517+0000","last_peered":"2024-05-03T18:23:20.576517+0000","last_clean":"2024-05-03T18:23:20.576517+0000","last_became_active":"2024-05-03T18:22:25.587861+0000","last_became_peered":"2024-05-03T18:22:25.587861+0000","last_unstale":"2024-05-03T18:23:20.576517+0000","last_undegraded":"2024-05-03T18:23:20.576517+0000","last_fullsized":"2024-05-03T18:23:20.576517+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:17:02.538798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-03T18:24:20.908263+0000","last_change":"2024-05-03T18:22:25.597874+0000","last_active":"2024-05-03T18:24:20.908263+0000","last_peered":"2024-05-03T18:24:20.908263+0000","last_clean":"2024-05-03T18:24:20.908263+0000","last_became_active":"2024-05-03T18:22:25.597448+0000","last_became_peered":"2024-05-03T18:22:25.597448+0000","last_unstale":"2024-05-03T18:24:20.908263+0000","last_undegraded":"2024-05-03T18:24:20.908263+0000","last_fullsized":"2024-05-03T18:24:20.908263+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:55:52.894491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-03T18:23:20.574148+0000","last_change":"2024-05-03T18:22:25.593509+0000","last_active":"2024-05-03T18:23:20.574148+0000","last_peered":"2024-05-03T18:23:20.574148+0000","last_clean":"2024-05-03T18:23:20.574148+0000","last_became_active":"2024-05-03T18:22:25.590736+0000","last_became_peered":"2024-05-03T18:22:25.590736+0000","last_unstale":"2024-05-03T18:23:20.574148+0000","last_undegraded":"2024-05-03T18:23:20.574148+0000","last_fullsized":"2024-05-03T18:23:20.574148+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:54:45.431776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-03T18:23:20.573668+0000","last_change":"2024-05-03T18:22:25.590022+0000","last_active":"2024-05-03T18:23:20.573668+0000","last_peered":"2024-05-03T18:23:20.573668+0000","last_clean":"2024-05-03T18:23:20.573668+0000","last_became_active":"2024-05-03T18:22:25.589432+0000","last_became_peered":"2024-05-03T18:22:25.589432+0000","last_unstale":"2024-05-03T18:23:20.573668+0000","last_undegraded":"2024-05-03T18:23:20.573668+0000","last_fullsized":"2024-05-03T18:23:20.573668+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:03:02.091228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-03T18:23:20.549196+0000","last_change":"2024-05-03T18:22:25.581452+0000","last_active":"2024-05-03T18:23:20.549196+0000","last_peered":"2024-05-03T18:23:20.549196+0000","last_clean":"2024-05-03T18:23:20.549196+0000","last_became_active":"2024-05-03T18:22:25.581098+0000","last_became_peered":"2024-05-03T18:22:25.581098+0000","last_unstale":"2024-05-03T18:23:20.549196+0000","last_undegraded":"2024-05-03T18:23:20.549196+0000","last_fullsized":"2024-05-03T18:23:20.549196+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:04:46.551234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-03T18:24:20.538650+0000","last_change":"2024-05-03T18:22:25.589105+0000","last_active":"2024-05-03T18:24:20.538650+0000","last_peered":"2024-05-03T18:24:20.538650+0000","last_clean":"2024-05-03T18:24:20.538650+0000","last_became_active":"2024-05-03T18:22:25.588697+0000","last_became_peered":"2024-05-03T18:22:25.588697+0000","last_unstale":"2024-05-03T18:24:20.538650+0000","last_undegraded":"2024-05-03T18:24:20.538650+0000","last_fullsized":"2024-05-03T18:24:20.538650+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:41:23.563296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":56913,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":78,"num_read_kb":97,"num_write":81,"num_write_kb":84,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":258048,"data_stored":170739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":69,"ondisk_log_size":69,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710763,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32160,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712992,"statfs":{"total":95995035648,"available":95962103808,"internally_reserved":0,"allocated":1003520,"data_stored":504990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906993,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28632,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716520,"statfs":{"total":95995035648,"available":95965716480,"internally_reserved":0,"allocated":1654784,"data_stored":1138184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070520,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":995328,"data_stored":500281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266751,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":995328,"data_stored":496207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462984,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32728,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712424,"statfs":{"total":95995035648,"available":95961522176,"internally_reserved":0,"allocated":1589248,"data_stored":1090649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626511,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":983040,"data_stored":491507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574962,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1048576,"data_stored":547622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738489,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":987136,"data_stored":491597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411436,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32224,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712928,"statfs":{"total":95995035648,"available":95962038272,"internally_reserved":0,"allocated":1069056,"data_stored":557043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182498,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":991232,"data_stored":491611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986268,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":991232,"data_stored":495625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822741,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32752,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712400,"statfs":{"total":95995035648,"available":95961497600,"internally_reserved":0,"allocated":1609728,"data_stored":1095540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":56115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":56149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":56309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-03T18:24:22.766 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-03T18:24:22.848 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:24:22 smithi033 ceph-mon[28330]: pgmap v439: 129 pgs: 129 active+clean; 632 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 307 B/s wr, 2 op/s 2024-05-03T18:24:22.848 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:24:22 smithi033 ceph-mon[28330]: osdmap e89: 12 total, 12 up, 12 in 2024-05-03T18:24:22.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:24:22 smithi150 ceph-mon[32492]: pgmap v439: 129 pgs: 129 active+clean; 632 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 307 B/s wr, 2 op/s 2024-05-03T18:24:22.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:24:22 smithi150 ceph-mon[32492]: osdmap e89: 12 total, 12 up, 12 in 2024-05-03T18:24:23.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:24:22 smithi099 ceph-mon[31658]: pgmap v439: 129 pgs: 129 active+clean; 632 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 307 B/s wr, 2 op/s 2024-05-03T18:24:23.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:24:22 smithi099 ceph-mon[31658]: osdmap e89: 12 total, 12 up, 12 in 2024-05-03T18:24:23.227 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:24:23.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:24:23 smithi033 ceph-mon[28330]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-03T18:24:23.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:24:23 smithi150 ceph-mon[32492]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-03T18:24:24.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:24:23 smithi099 ceph-mon[31658]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-03T18:24:24.646 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:24:24 smithi033 ceph-mon[28330]: pgmap v441: 129 pgs: 129 active+clean; 632 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:24:24.646 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:24:24 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:24:24.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:24:24 smithi150 ceph-mon[32492]: pgmap v441: 129 pgs: 129 active+clean; 632 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:24:24.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:24:24 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:24:25.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:24:24 smithi099 ceph-mon[31658]: pgmap v441: 129 pgs: 129 active+clean; 632 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:24:25.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:24:24 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:24:25.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:25.442+0000 7f9a9d800700 1 -- 172.21.15.33:0/3774318198 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a98075ff0 msgr2=0x7f9a9810f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:24:25.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:25.442+0000 7f9a9d800700 1 --2- 172.21.15.33:0/3774318198 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a98075ff0 0x7f9a9810f9c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f9a84009990 tx=0x7f9a84009ca0 comp rx=0 tx=0).stop 2024-05-03T18:24:25.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:25.442+0000 7f9a9d800700 1 -- 172.21.15.33:0/3774318198 shutdown_connections 2024-05-03T18:24:25.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:25.442+0000 7f9a9d800700 1 --2- 172.21.15.33:0/3774318198 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a9810ff00 0x7f9a981122f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:25.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:25.442+0000 7f9a9d800700 1 --2- 172.21.15.33:0/3774318198 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a980756d0 0x7f9a98075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:25.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:25.442+0000 7f9a9d800700 1 --2- 172.21.15.33:0/3774318198 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a98075ff0 0x7f9a9810f9c0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:25.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:25.442+0000 7f9a9d800700 1 -- 172.21.15.33:0/3774318198 >> 172.21.15.33:0/3774318198 conn(0x7f9a9807a600 msgr2=0x7f9a9807aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:24:25.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:25.442+0000 7f9a9d800700 1 -- 172.21.15.33:0/3774318198 shutdown_connections 2024-05-03T18:24:25.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:25.442+0000 7f9a9d800700 1 -- 172.21.15.33:0/3774318198 wait complete. 2024-05-03T18:24:25.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:25.444+0000 7f9a9d800700 1 Processor -- start 2024-05-03T18:24:25.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:25.444+0000 7f9a9d800700 1 -- start start 2024-05-03T18:24:25.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:25.444+0000 7f9a9d800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a980756d0 0x7f9a981a1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:24:25.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:25.444+0000 7f9a9d800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a98075ff0 0x7f9a981a1e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:24:25.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:25.444+0000 7f9a9d800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a9810ff00 0x7f9a9819bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:24:25.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:25.444+0000 7f9a9d800700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9a981157a0 con 0x7f9a9810ff00 2024-05-03T18:24:25.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:25.444+0000 7f9a9d800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9a98115620 con 0x7f9a98075ff0 2024-05-03T18:24:25.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:25.444+0000 7f9a9d800700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9a98115920 con 0x7f9a980756d0 2024-05-03T18:24:25.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:25.445+0000 7f9a96a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a980756d0 0x7f9a981a1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:24:25.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:25.445+0000 7f9a96a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a980756d0 0x7f9a981a1900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:58924/0 (socket says 172.21.15.33:58924) 2024-05-03T18:24:25.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:25.445+0000 7f9a96a00700 1 -- 172.21.15.33:0/46223811 learned_addr learned my addr 172.21.15.33:0/46223811 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:24:25.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:25.445+0000 7f9a97400700 1 --2- 172.21.15.33:0/46223811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a9810ff00 0x7f9a9819bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:24:25.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:25.445+0000 7f9a96000700 1 --2- 172.21.15.33:0/46223811 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a98075ff0 0x7f9a981a1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:24:25.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:25.446+0000 7f9a97400700 1 -- 172.21.15.33:0/46223811 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a980756d0 msgr2=0x7f9a981a1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:24:25.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:25.446+0000 7f9a97400700 1 --2- 172.21.15.33:0/46223811 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a980756d0 0x7f9a981a1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:25.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:25.446+0000 7f9a97400700 1 -- 172.21.15.33:0/46223811 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a98075ff0 msgr2=0x7f9a981a1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:24:25.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:25.446+0000 7f9a97400700 1 --2- 172.21.15.33:0/46223811 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a98075ff0 0x7f9a981a1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:25.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:25.446+0000 7f9a97400700 1 -- 172.21.15.33:0/46223811 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a9819c220 con 0x7f9a9810ff00 2024-05-03T18:24:25.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:25.446+0000 7f9a96a00700 1 --2- 172.21.15.33:0/46223811 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a980756d0 0x7f9a981a1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:24:25.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:25.446+0000 7f9a97400700 1 --2- 172.21.15.33:0/46223811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a9810ff00 0x7f9a9819bac0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f9a9000ef90 tx=0x7f9a9000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:24:25.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:25.446+0000 7f9a8f400700 1 -- 172.21.15.33:0/46223811 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a900092e0 con 0x7f9a9810ff00 2024-05-03T18:24:25.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:25.446+0000 7f9a9d800700 1 -- 172.21.15.33:0/46223811 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a9819c4b0 con 0x7f9a9810ff00 2024-05-03T18:24:25.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:25.447+0000 7f9a9d800700 1 -- 172.21.15.33:0/46223811 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a981a87d0 con 0x7f9a9810ff00 2024-05-03T18:24:25.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:25.447+0000 7f9a8f400700 1 -- 172.21.15.33:0/46223811 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9a90004bb0 con 0x7f9a9810ff00 2024-05-03T18:24:25.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:25.447+0000 7f9a8f400700 1 -- 172.21.15.33:0/46223811 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a9000ce60 con 0x7f9a9810ff00 2024-05-03T18:24:25.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:25.448+0000 7f9a9d800700 1 -- 172.21.15.33:0/46223811 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a78005330 con 0x7f9a9810ff00 2024-05-03T18:24:25.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:25.450+0000 7f9a8f400700 1 -- 172.21.15.33:0/46223811 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f9a9001b030 con 0x7f9a9810ff00 2024-05-03T18:24:25.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:25.452+0000 7f9a8f400700 1 --2- 172.21.15.33:0/46223811 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f9a7c0b2910 0x7f9a7c0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:24:25.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:25.452+0000 7f9a8f400700 1 -- 172.21.15.33:0/46223811 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f9a900e2b10 con 0x7f9a9810ff00 2024-05-03T18:24:25.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:25.453+0000 7f9a96a00700 1 --2- 172.21.15.33:0/46223811 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f9a7c0b2910 0x7f9a7c0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:24:25.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:25.453+0000 7f9a96a00700 1 --2- 172.21.15.33:0/46223811 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f9a7c0b2910 0x7f9a7c0b4dd0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9a9819cc00 tx=0x7f9a8800b500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:24:25.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:25.454+0000 7f9a8f400700 1 -- 172.21.15.33:0/46223811 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f9a90022020 con 0x7f9a9810ff00 2024-05-03T18:24:25.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:25.735+0000 7f9a9d800700 1 -- 172.21.15.33:0/46223811 --> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9a78000bc0 con 0x7f9a7c0b2910 2024-05-03T18:24:25.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:25.746+0000 7f9a8f400700 1 -- 172.21.15.33:0/46223811 <== mgr.14152 v2:172.21.15.33:6800/3642895501 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340426 (secure 0 0 0) 0x7f9a78000bc0 con 0x7f9a7c0b2910 2024-05-03T18:24:25.747 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:24:25.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:25.753+0000 7f9a9d800700 1 -- 172.21.15.33:0/46223811 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f9a7c0b2910 msgr2=0x7f9a7c0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:24:25.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:25.753+0000 7f9a9d800700 1 --2- 172.21.15.33:0/46223811 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f9a7c0b2910 0x7f9a7c0b4dd0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9a9819cc00 tx=0x7f9a8800b500 comp rx=0 tx=0).stop 2024-05-03T18:24:25.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:25.753+0000 7f9a9d800700 1 -- 172.21.15.33:0/46223811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a9810ff00 msgr2=0x7f9a9819bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:24:25.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:25.753+0000 7f9a9d800700 1 --2- 172.21.15.33:0/46223811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a9810ff00 0x7f9a9819bac0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f9a9000ef90 tx=0x7f9a9000c3a0 comp rx=0 tx=0).stop 2024-05-03T18:24:25.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:25.753+0000 7f9a9d800700 1 -- 172.21.15.33:0/46223811 shutdown_connections 2024-05-03T18:24:25.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:25.753+0000 7f9a9d800700 1 --2- 172.21.15.33:0/46223811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a9810ff00 0x7f9a9819bac0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:25.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:25.753+0000 7f9a9d800700 1 --2- 172.21.15.33:0/46223811 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a980756d0 0x7f9a981a1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:25.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:25.753+0000 7f9a9d800700 1 --2- 172.21.15.33:0/46223811 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f9a7c0b2910 0x7f9a7c0b4dd0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:25.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:25.753+0000 7f9a9d800700 1 --2- 172.21.15.33:0/46223811 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a98075ff0 0x7f9a981a1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:25.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:25.753+0000 7f9a9d800700 1 -- 172.21.15.33:0/46223811 >> 172.21.15.33:0/46223811 conn(0x7f9a9807a600 msgr2=0x7f9a98100730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:24:25.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:25.753+0000 7f9a9d800700 1 -- 172.21.15.33:0/46223811 shutdown_connections 2024-05-03T18:24:25.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:25.753+0000 7f9a9d800700 1 -- 172.21.15.33:0/46223811 wait complete. 2024-05-03T18:24:25.755 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-05-03T18:24:26.807 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:24:26 smithi033 ceph-mon[28330]: pgmap v442: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:24:26.807 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:24:26 smithi033 ceph-mon[28330]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-03T18:24:26.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:24:26 smithi150 ceph-mon[32492]: pgmap v442: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:24:26.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:24:26 smithi150 ceph-mon[32492]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-03T18:24:27.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:24:26 smithi099 ceph-mon[31658]: pgmap v442: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:24:27.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:24:26 smithi099 ceph-mon[31658]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-03T18:24:27.358 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":442,"stamp":"2024-05-03T18:24:25.368223+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":647515,"num_objects":26,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":169,"num_read_kb":171,"num_write":206,"num_write_kb":2236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":163,"ondisk_log_size":163,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":367196,"kb_used_data":13816,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124574628,"statfs":{"total":1151940427776,"available":1151564419072,"internally_reserved":0,"allocated":14147584,"data_stored":8032022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":361674042},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003679"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-03T18:24:20.539684+0000","last_change":"2024-05-03T18:22:25.595640+0000","last_active":"2024-05-03T18:24:20.539684+0000","last_peered":"2024-05-03T18:24:20.539684+0000","last_clean":"2024-05-03T18:24:20.539684+0000","last_became_active":"2024-05-03T18:22:25.595284+0000","last_became_peered":"2024-05-03T18:22:25.595284+0000","last_unstale":"2024-05-03T18:24:20.539684+0000","last_undegraded":"2024-05-03T18:24:20.539684+0000","last_fullsized":"2024-05-03T18:24:20.539684+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:50:08.867670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.802159+0000","last_change":"2024-05-03T18:22:25.584950+0000","last_active":"2024-05-03T18:24:21.802159+0000","last_peered":"2024-05-03T18:24:21.802159+0000","last_clean":"2024-05-03T18:24:21.802159+0000","last_became_active":"2024-05-03T18:22:25.584613+0000","last_became_peered":"2024-05-03T18:22:25.584613+0000","last_unstale":"2024-05-03T18:24:21.802159+0000","last_undegraded":"2024-05-03T18:24:21.802159+0000","last_fullsized":"2024-05-03T18:24:21.802159+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:00:24.959429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.801872+0000","last_change":"2024-05-03T18:22:25.596799+0000","last_active":"2024-05-03T18:24:21.801872+0000","last_peered":"2024-05-03T18:24:21.801872+0000","last_clean":"2024-05-03T18:24:21.801872+0000","last_became_active":"2024-05-03T18:22:25.596386+0000","last_became_peered":"2024-05-03T18:22:25.596386+0000","last_unstale":"2024-05-03T18:24:21.801872+0000","last_undegraded":"2024-05-03T18:24:21.801872+0000","last_fullsized":"2024-05-03T18:24:21.801872+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:20:07.818282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.549312+0000","last_change":"2024-05-03T18:23:20.534672+0000","last_active":"2024-05-03T18:24:21.549312+0000","last_peered":"2024-05-03T18:24:21.549312+0000","last_clean":"2024-05-03T18:24:21.549312+0000","last_became_active":"2024-05-03T18:23:20.534073+0000","last_became_peered":"2024-05-03T18:23:20.534073+0000","last_unstale":"2024-05-03T18:24:21.549312+0000","last_undegraded":"2024-05-03T18:24:21.549312+0000","last_fullsized":"2024-05-03T18:24:21.549312+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:07:40.204299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-03T18:24:20.917187+0000","last_change":"2024-05-03T18:22:25.589810+0000","last_active":"2024-05-03T18:24:20.917187+0000","last_peered":"2024-05-03T18:24:20.917187+0000","last_clean":"2024-05-03T18:24:20.917187+0000","last_became_active":"2024-05-03T18:22:25.588220+0000","last_became_peered":"2024-05-03T18:22:25.588220+0000","last_unstale":"2024-05-03T18:24:20.917187+0000","last_undegraded":"2024-05-03T18:24:20.917187+0000","last_fullsized":"2024-05-03T18:24:20.917187+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:43:20.904737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.550368+0000","last_change":"2024-05-03T18:22:25.603148+0000","last_active":"2024-05-03T18:24:21.550368+0000","last_peered":"2024-05-03T18:24:21.550368+0000","last_clean":"2024-05-03T18:24:21.550368+0000","last_became_active":"2024-05-03T18:22:25.602743+0000","last_became_peered":"2024-05-03T18:22:25.602743+0000","last_unstale":"2024-05-03T18:24:21.550368+0000","last_undegraded":"2024-05-03T18:24:21.550368+0000","last_fullsized":"2024-05-03T18:24:21.550368+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:49:27.387863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.576639+0000","last_change":"2024-05-03T18:23:20.532077+0000","last_active":"2024-05-03T18:24:21.576639+0000","last_peered":"2024-05-03T18:24:21.576639+0000","last_clean":"2024-05-03T18:24:21.576639+0000","last_became_active":"2024-05-03T18:23:20.531528+0000","last_became_peered":"2024-05-03T18:23:20.531528+0000","last_unstale":"2024-05-03T18:24:21.576639+0000","last_undegraded":"2024-05-03T18:24:21.576639+0000","last_fullsized":"2024-05-03T18:24:21.576639+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:10:37.037114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.627720+0000","last_change":"2024-05-03T18:22:25.600192+0000","last_active":"2024-05-03T18:24:21.627720+0000","last_peered":"2024-05-03T18:24:21.627720+0000","last_clean":"2024-05-03T18:24:21.627720+0000","last_became_active":"2024-05-03T18:22:25.599663+0000","last_became_peered":"2024-05-03T18:22:25.599663+0000","last_unstale":"2024-05-03T18:24:21.627720+0000","last_undegraded":"2024-05-03T18:24:21.627720+0000","last_fullsized":"2024-05-03T18:24:21.627720+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:09:51.614784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.556257+0000","last_change":"2024-05-03T18:22:25.600302+0000","last_active":"2024-05-03T18:24:21.556257+0000","last_peered":"2024-05-03T18:24:21.556257+0000","last_clean":"2024-05-03T18:24:21.556257+0000","last_became_active":"2024-05-03T18:22:25.599832+0000","last_became_peered":"2024-05-03T18:22:25.599832+0000","last_unstale":"2024-05-03T18:24:21.556257+0000","last_undegraded":"2024-05-03T18:24:21.556257+0000","last_fullsized":"2024-05-03T18:24:21.556257+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:52:12.326156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.551618+0000","last_change":"2024-05-03T18:22:25.590892+0000","last_active":"2024-05-03T18:24:21.551618+0000","last_peered":"2024-05-03T18:24:21.551618+0000","last_clean":"2024-05-03T18:24:21.551618+0000","last_became_active":"2024-05-03T18:22:25.590516+0000","last_became_peered":"2024-05-03T18:22:25.590516+0000","last_unstale":"2024-05-03T18:24:21.551618+0000","last_undegraded":"2024-05-03T18:24:21.551618+0000","last_fullsized":"2024-05-03T18:24:21.551618+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:57:57.999234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.578073+0000","last_change":"2024-05-03T18:22:25.597307+0000","last_active":"2024-05-03T18:24:21.578073+0000","last_peered":"2024-05-03T18:24:21.578073+0000","last_clean":"2024-05-03T18:24:21.578073+0000","last_became_active":"2024-05-03T18:22:25.596989+0000","last_became_peered":"2024-05-03T18:22:25.596989+0000","last_unstale":"2024-05-03T18:24:21.578073+0000","last_undegraded":"2024-05-03T18:24:21.578073+0000","last_fullsized":"2024-05-03T18:24:21.578073+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:53:09.003353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.926313+0000","last_change":"2024-05-03T18:22:25.601723+0000","last_active":"2024-05-03T18:24:21.926313+0000","last_peered":"2024-05-03T18:24:21.926313+0000","last_clean":"2024-05-03T18:24:21.926313+0000","last_became_active":"2024-05-03T18:22:25.599428+0000","last_became_peered":"2024-05-03T18:22:25.599428+0000","last_unstale":"2024-05-03T18:24:21.926313+0000","last_undegraded":"2024-05-03T18:24:21.926313+0000","last_fullsized":"2024-05-03T18:24:21.926313+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:41:18.584497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.923630+0000","last_change":"2024-05-03T18:22:25.594885+0000","last_active":"2024-05-03T18:24:21.923630+0000","last_peered":"2024-05-03T18:24:21.923630+0000","last_clean":"2024-05-03T18:24:21.923630+0000","last_became_active":"2024-05-03T18:22:25.593939+0000","last_became_peered":"2024-05-03T18:22:25.593939+0000","last_unstale":"2024-05-03T18:24:21.923630+0000","last_undegraded":"2024-05-03T18:24:21.923630+0000","last_fullsized":"2024-05-03T18:24:21.923630+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:03:30.052886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.801444+0000","last_change":"2024-05-03T18:22:25.595228+0000","last_active":"2024-05-03T18:24:21.801444+0000","last_peered":"2024-05-03T18:24:21.801444+0000","last_clean":"2024-05-03T18:24:21.801444+0000","last_became_active":"2024-05-03T18:22:25.589825+0000","last_became_peered":"2024-05-03T18:22:25.589825+0000","last_unstale":"2024-05-03T18:24:21.801444+0000","last_undegraded":"2024-05-03T18:24:21.801444+0000","last_fullsized":"2024-05-03T18:24:21.801444+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:44:05.638939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.801259+0000","last_change":"2024-05-03T18:22:25.595630+0000","last_active":"2024-05-03T18:24:21.801259+0000","last_peered":"2024-05-03T18:24:21.801259+0000","last_clean":"2024-05-03T18:24:21.801259+0000","last_became_active":"2024-05-03T18:22:25.591869+0000","last_became_peered":"2024-05-03T18:22:25.591869+0000","last_unstale":"2024-05-03T18:24:21.801259+0000","last_undegraded":"2024-05-03T18:24:21.801259+0000","last_fullsized":"2024-05-03T18:24:21.801259+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:27:43.579329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.555626+0000","last_change":"2024-05-03T18:22:25.589482+0000","last_active":"2024-05-03T18:24:21.555626+0000","last_peered":"2024-05-03T18:24:21.555626+0000","last_clean":"2024-05-03T18:24:21.555626+0000","last_became_active":"2024-05-03T18:22:25.589142+0000","last_became_peered":"2024-05-03T18:22:25.589142+0000","last_unstale":"2024-05-03T18:24:21.555626+0000","last_undegraded":"2024-05-03T18:24:21.555626+0000","last_fullsized":"2024-05-03T18:24:21.555626+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:37:55.472823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-03T18:24:20.539368+0000","last_change":"2024-05-03T18:22:25.594569+0000","last_active":"2024-05-03T18:24:20.539368+0000","last_peered":"2024-05-03T18:24:20.539368+0000","last_clean":"2024-05-03T18:24:20.539368+0000","last_became_active":"2024-05-03T18:22:25.594218+0000","last_became_peered":"2024-05-03T18:22:25.594218+0000","last_unstale":"2024-05-03T18:24:20.539368+0000","last_undegraded":"2024-05-03T18:24:20.539368+0000","last_fullsized":"2024-05-03T18:24:20.539368+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:30:42.775673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.549668+0000","last_change":"2024-05-03T18:22:25.597426+0000","last_active":"2024-05-03T18:24:21.549668+0000","last_peered":"2024-05-03T18:24:21.549668+0000","last_clean":"2024-05-03T18:24:21.549668+0000","last_became_active":"2024-05-03T18:22:25.594524+0000","last_became_peered":"2024-05-03T18:22:25.594524+0000","last_unstale":"2024-05-03T18:24:21.549668+0000","last_undegraded":"2024-05-03T18:24:21.549668+0000","last_fullsized":"2024-05-03T18:24:21.549668+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:41:36.984253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-03T18:24:20.923388+0000","last_change":"2024-05-03T18:22:25.588002+0000","last_active":"2024-05-03T18:24:20.923388+0000","last_peered":"2024-05-03T18:24:20.923388+0000","last_clean":"2024-05-03T18:24:20.923388+0000","last_became_active":"2024-05-03T18:22:25.587547+0000","last_became_peered":"2024-05-03T18:22:25.587547+0000","last_unstale":"2024-05-03T18:24:20.923388+0000","last_undegraded":"2024-05-03T18:24:20.923388+0000","last_fullsized":"2024-05-03T18:24:20.923388+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:47:14.115517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-03T18:24:20.921989+0000","last_change":"2024-05-03T18:22:40.754602+0000","last_active":"2024-05-03T18:24:20.921989+0000","last_peered":"2024-05-03T18:24:20.921989+0000","last_clean":"2024-05-03T18:24:20.921989+0000","last_became_active":"2024-05-03T18:22:40.752917+0000","last_became_peered":"2024-05-03T18:22:40.752917+0000","last_unstale":"2024-05-03T18:24:20.921989+0000","last_undegraded":"2024-05-03T18:24:20.921989+0000","last_fullsized":"2024-05-03T18:24:20.921989+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:51:44.711811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"78'4","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.549993+0000","last_change":"2024-05-03T18:22:21.558560+0000","last_active":"2024-05-03T18:24:21.549993+0000","last_peered":"2024-05-03T18:24:21.549993+0000","last_clean":"2024-05-03T18:24:21.549993+0000","last_became_active":"2024-05-03T18:22:21.558185+0000","last_became_peered":"2024-05-03T18:22:21.558185+0000","last_unstale":"2024-05-03T18:24:21.549993+0000","last_undegraded":"2024-05-03T18:24:21.549993+0000","last_fullsized":"2024-05-03T18:24:21.549993+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:18:19.666724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.628738+0000","last_change":"2024-05-03T18:22:25.603934+0000","last_active":"2024-05-03T18:24:21.628738+0000","last_peered":"2024-05-03T18:24:21.628738+0000","last_clean":"2024-05-03T18:24:21.628738+0000","last_became_active":"2024-05-03T18:22:25.603529+0000","last_became_peered":"2024-05-03T18:22:25.603529+0000","last_unstale":"2024-05-03T18:24:21.628738+0000","last_undegraded":"2024-05-03T18:24:21.628738+0000","last_fullsized":"2024-05-03T18:24:21.628738+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:08:50.288871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.924688+0000","last_change":"2024-05-03T18:23:20.576254+0000","last_active":"2024-05-03T18:24:21.924688+0000","last_peered":"2024-05-03T18:24:21.924688+0000","last_clean":"2024-05-03T18:24:21.924688+0000","last_became_active":"2024-05-03T18:23:20.575774+0000","last_became_peered":"2024-05-03T18:23:20.575774+0000","last_unstale":"2024-05-03T18:24:21.924688+0000","last_undegraded":"2024-05-03T18:24:21.924688+0000","last_fullsized":"2024-05-03T18:24:21.924688+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:22:39.090938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"78'1","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.628643+0000","last_change":"2024-05-03T18:22:21.734171+0000","last_active":"2024-05-03T18:24:21.628643+0000","last_peered":"2024-05-03T18:24:21.628643+0000","last_clean":"2024-05-03T18:24:21.628643+0000","last_became_active":"2024-05-03T18:22:21.733301+0000","last_became_peered":"2024-05-03T18:22:21.733301+0000","last_unstale":"2024-05-03T18:24:21.628643+0000","last_undegraded":"2024-05-03T18:24:21.628643+0000","last_fullsized":"2024-05-03T18:24:21.628643+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:09:48.861247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.802308+0000","last_change":"2024-05-03T18:22:25.598016+0000","last_active":"2024-05-03T18:24:21.802308+0000","last_peered":"2024-05-03T18:24:21.802308+0000","last_clean":"2024-05-03T18:24:21.802308+0000","last_became_active":"2024-05-03T18:22:25.594888+0000","last_became_peered":"2024-05-03T18:22:25.594888+0000","last_unstale":"2024-05-03T18:24:21.802308+0000","last_undegraded":"2024-05-03T18:24:21.802308+0000","last_fullsized":"2024-05-03T18:24:21.802308+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:47:46.400247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.924979+0000","last_change":"2024-05-03T18:22:40.755148+0000","last_active":"2024-05-03T18:24:21.924979+0000","last_peered":"2024-05-03T18:24:21.924979+0000","last_clean":"2024-05-03T18:24:21.924979+0000","last_became_active":"2024-05-03T18:22:40.754656+0000","last_became_peered":"2024-05-03T18:22:40.754656+0000","last_unstale":"2024-05-03T18:24:21.924979+0000","last_undegraded":"2024-05-03T18:24:21.924979+0000","last_fullsized":"2024-05-03T18:24:21.924979+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:48:23.124936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"78'2","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-03T18:24:20.917791+0000","last_change":"2024-05-03T18:22:21.568060+0000","last_active":"2024-05-03T18:24:20.917791+0000","last_peered":"2024-05-03T18:24:20.917791+0000","last_clean":"2024-05-03T18:24:20.917791+0000","last_became_active":"2024-05-03T18:22:21.567434+0000","last_became_peered":"2024-05-03T18:22:21.567434+0000","last_unstale":"2024-05-03T18:24:20.917791+0000","last_undegraded":"2024-05-03T18:24:20.917791+0000","last_fullsized":"2024-05-03T18:24:20.917791+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:06:05.802621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.555936+0000","last_change":"2024-05-03T18:22:25.597777+0000","last_active":"2024-05-03T18:24:21.555936+0000","last_peered":"2024-05-03T18:24:21.555936+0000","last_clean":"2024-05-03T18:24:21.555936+0000","last_became_active":"2024-05-03T18:22:25.597391+0000","last_became_peered":"2024-05-03T18:22:25.597391+0000","last_unstale":"2024-05-03T18:24:21.555936+0000","last_undegraded":"2024-05-03T18:24:21.555936+0000","last_fullsized":"2024-05-03T18:24:21.555936+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:02:24.708193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.924850+0000","last_change":"2024-05-03T18:22:40.752318+0000","last_active":"2024-05-03T18:24:21.924850+0000","last_peered":"2024-05-03T18:24:21.924850+0000","last_clean":"2024-05-03T18:24:21.924850+0000","last_became_active":"2024-05-03T18:22:40.751433+0000","last_became_peered":"2024-05-03T18:22:40.751433+0000","last_unstale":"2024-05-03T18:24:21.924850+0000","last_undegraded":"2024-05-03T18:24:21.924850+0000","last_fullsized":"2024-05-03T18:24:21.924850+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:21:55.746143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-03T18:23:20.520232+0000","last_change":"2024-05-03T18:22:21.560646+0000","last_active":"2024-05-03T18:23:20.520232+0000","last_peered":"2024-05-03T18:23:20.520232+0000","last_clean":"2024-05-03T18:23:20.520232+0000","last_became_active":"2024-05-03T18:22:21.560138+0000","last_became_peered":"2024-05-03T18:22:21.560138+0000","last_unstale":"2024-05-03T18:23:20.520232+0000","last_undegraded":"2024-05-03T18:23:20.520232+0000","last_fullsized":"2024-05-03T18:23:20.520232+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:10:43.967781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-03T18:24:20.908752+0000","last_change":"2024-05-03T18:23:20.575102+0000","last_active":"2024-05-03T18:24:20.908752+0000","last_peered":"2024-05-03T18:24:20.908752+0000","last_clean":"2024-05-03T18:24:20.908752+0000","last_became_active":"2024-05-03T18:23:20.574657+0000","last_became_peered":"2024-05-03T18:23:20.574657+0000","last_unstale":"2024-05-03T18:24:20.908752+0000","last_undegraded":"2024-05-03T18:24:20.908752+0000","last_fullsized":"2024-05-03T18:24:20.908752+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:08:26.280866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-03T18:24:20.922780+0000","last_change":"2024-05-03T18:22:40.761052+0000","last_active":"2024-05-03T18:24:20.922780+0000","last_peered":"2024-05-03T18:24:20.922780+0000","last_clean":"2024-05-03T18:24:20.922780+0000","last_became_active":"2024-05-03T18:22:40.760179+0000","last_became_peered":"2024-05-03T18:22:40.760179+0000","last_unstale":"2024-05-03T18:24:20.922780+0000","last_undegraded":"2024-05-03T18:24:20.922780+0000","last_fullsized":"2024-05-03T18:24:20.922780+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:44:48.610680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"78'1","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-03T18:24:20.909244+0000","last_change":"2024-05-03T18:22:21.563421+0000","last_active":"2024-05-03T18:24:20.909244+0000","last_peered":"2024-05-03T18:24:20.909244+0000","last_clean":"2024-05-03T18:24:20.909244+0000","last_became_active":"2024-05-03T18:22:21.560991+0000","last_became_peered":"2024-05-03T18:22:21.560991+0000","last_unstale":"2024-05-03T18:24:20.909244+0000","last_undegraded":"2024-05-03T18:24:20.909244+0000","last_fullsized":"2024-05-03T18:24:20.909244+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:50:32.640699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-03T18:24:20.916187+0000","last_change":"2024-05-03T18:22:25.599049+0000","last_active":"2024-05-03T18:24:20.916187+0000","last_peered":"2024-05-03T18:24:20.916187+0000","last_clean":"2024-05-03T18:24:20.916187+0000","last_became_active":"2024-05-03T18:22:25.597879+0000","last_became_peered":"2024-05-03T18:22:25.597879+0000","last_unstale":"2024-05-03T18:24:20.916187+0000","last_undegraded":"2024-05-03T18:24:20.916187+0000","last_fullsized":"2024-05-03T18:24:20.916187+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:51:08.487499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-03T18:24:20.922822+0000","last_change":"2024-05-03T18:22:40.760908+0000","last_active":"2024-05-03T18:24:20.922822+0000","last_peered":"2024-05-03T18:24:20.922822+0000","last_clean":"2024-05-03T18:24:20.922822+0000","last_became_active":"2024-05-03T18:22:40.760331+0000","last_became_peered":"2024-05-03T18:22:40.760331+0000","last_unstale":"2024-05-03T18:24:20.922822+0000","last_undegraded":"2024-05-03T18:24:20.922822+0000","last_fullsized":"2024-05-03T18:24:20.922822+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:27:08.216015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"78'1","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.550282+0000","last_change":"2024-05-03T18:22:21.565328+0000","last_active":"2024-05-03T18:24:21.550282+0000","last_peered":"2024-05-03T18:24:21.550282+0000","last_clean":"2024-05-03T18:24:21.550282+0000","last_became_active":"2024-05-03T18:22:21.564583+0000","last_became_peered":"2024-05-03T18:22:21.564583+0000","last_unstale":"2024-05-03T18:24:21.550282+0000","last_undegraded":"2024-05-03T18:24:21.550282+0000","last_fullsized":"2024-05-03T18:24:21.550282+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:58:32.370458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-03T18:24:20.909560+0000","last_change":"2024-05-03T18:22:25.607585+0000","last_active":"2024-05-03T18:24:20.909560+0000","last_peered":"2024-05-03T18:24:20.909560+0000","last_clean":"2024-05-03T18:24:20.909560+0000","last_became_active":"2024-05-03T18:22:25.607168+0000","last_became_peered":"2024-05-03T18:22:25.607168+0000","last_unstale":"2024-05-03T18:24:20.909560+0000","last_undegraded":"2024-05-03T18:24:20.909560+0000","last_fullsized":"2024-05-03T18:24:20.909560+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:04:39.750823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-03T18:24:20.539356+0000","last_change":"2024-05-03T18:22:40.754744+0000","last_active":"2024-05-03T18:24:20.539356+0000","last_peered":"2024-05-03T18:24:20.539356+0000","last_clean":"2024-05-03T18:24:20.539356+0000","last_became_active":"2024-05-03T18:22:40.754262+0000","last_became_peered":"2024-05-03T18:22:40.754262+0000","last_unstale":"2024-05-03T18:24:20.539356+0000","last_undegraded":"2024-05-03T18:24:20.539356+0000","last_fullsized":"2024-05-03T18:24:20.539356+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:11:47.979546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.922915+0000","last_change":"2024-05-03T18:22:21.550048+0000","last_active":"2024-05-03T18:24:21.922915+0000","last_peered":"2024-05-03T18:24:21.922915+0000","last_clean":"2024-05-03T18:24:21.922915+0000","last_became_active":"2024-05-03T18:22:21.549334+0000","last_became_peered":"2024-05-03T18:22:21.549334+0000","last_unstale":"2024-05-03T18:24:21.922915+0000","last_undegraded":"2024-05-03T18:24:21.922915+0000","last_fullsized":"2024-05-03T18:24:21.922915+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:52:54.401185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.627606+0000","last_change":"2024-05-03T18:22:25.583074+0000","last_active":"2024-05-03T18:24:21.627606+0000","last_peered":"2024-05-03T18:24:21.627606+0000","last_clean":"2024-05-03T18:24:21.627606+0000","last_became_active":"2024-05-03T18:22:25.582637+0000","last_became_peered":"2024-05-03T18:22:25.582637+0000","last_unstale":"2024-05-03T18:24:21.627606+0000","last_undegraded":"2024-05-03T18:24:21.627606+0000","last_fullsized":"2024-05-03T18:24:21.627606+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:20:09.021402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.802054+0000","last_change":"2024-05-03T18:22:40.748509+0000","last_active":"2024-05-03T18:24:21.802054+0000","last_peered":"2024-05-03T18:24:21.802054+0000","last_clean":"2024-05-03T18:24:21.802054+0000","last_became_active":"2024-05-03T18:22:40.748051+0000","last_became_peered":"2024-05-03T18:22:40.748051+0000","last_unstale":"2024-05-03T18:24:21.802054+0000","last_undegraded":"2024-05-03T18:24:21.802054+0000","last_fullsized":"2024-05-03T18:24:21.802054+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:08:44.389460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-03T18:24:20.908131+0000","last_change":"2024-05-03T18:22:21.562185+0000","last_active":"2024-05-03T18:24:20.908131+0000","last_peered":"2024-05-03T18:24:20.908131+0000","last_clean":"2024-05-03T18:24:20.908131+0000","last_became_active":"2024-05-03T18:22:21.560523+0000","last_became_peered":"2024-05-03T18:22:21.560523+0000","last_unstale":"2024-05-03T18:24:20.908131+0000","last_undegraded":"2024-05-03T18:24:20.908131+0000","last_fullsized":"2024-05-03T18:24:20.908131+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:05:41.788286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.548630+0000","last_change":"2024-05-03T18:22:25.583895+0000","last_active":"2024-05-03T18:24:21.548630+0000","last_peered":"2024-05-03T18:24:21.548630+0000","last_clean":"2024-05-03T18:24:21.548630+0000","last_became_active":"2024-05-03T18:22:25.583503+0000","last_became_peered":"2024-05-03T18:22:25.583503+0000","last_unstale":"2024-05-03T18:24:21.548630+0000","last_undegraded":"2024-05-03T18:24:21.548630+0000","last_fullsized":"2024-05-03T18:24:21.548630+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:45:32.404394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.627927+0000","last_change":"2024-05-03T18:22:40.752814+0000","last_active":"2024-05-03T18:24:21.627927+0000","last_peered":"2024-05-03T18:24:21.627927+0000","last_clean":"2024-05-03T18:24:21.627927+0000","last_became_active":"2024-05-03T18:22:40.752328+0000","last_became_peered":"2024-05-03T18:22:40.752328+0000","last_unstale":"2024-05-03T18:24:21.627927+0000","last_undegraded":"2024-05-03T18:24:21.627927+0000","last_fullsized":"2024-05-03T18:24:21.627927+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:16:39.488122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.551417+0000","last_change":"2024-05-03T18:22:21.566266+0000","last_active":"2024-05-03T18:24:21.551417+0000","last_peered":"2024-05-03T18:24:21.551417+0000","last_clean":"2024-05-03T18:24:21.551417+0000","last_became_active":"2024-05-03T18:22:21.565612+0000","last_became_peered":"2024-05-03T18:22:21.565612+0000","last_unstale":"2024-05-03T18:24:21.551417+0000","last_undegraded":"2024-05-03T18:24:21.551417+0000","last_fullsized":"2024-05-03T18:24:21.551417+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:22:57.216912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.557406+0000","last_change":"2024-05-03T18:22:25.596182+0000","last_active":"2024-05-03T18:24:21.557406+0000","last_peered":"2024-05-03T18:24:21.557406+0000","last_clean":"2024-05-03T18:24:21.557406+0000","last_became_active":"2024-05-03T18:22:25.595472+0000","last_became_peered":"2024-05-03T18:22:25.595472+0000","last_unstale":"2024-05-03T18:24:21.557406+0000","last_undegraded":"2024-05-03T18:24:21.557406+0000","last_fullsized":"2024-05-03T18:24:21.557406+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:48:49.583879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.548471+0000","last_change":"2024-05-03T18:22:40.756378+0000","last_active":"2024-05-03T18:24:21.548471+0000","last_peered":"2024-05-03T18:24:21.548471+0000","last_clean":"2024-05-03T18:24:21.548471+0000","last_became_active":"2024-05-03T18:22:40.755692+0000","last_became_peered":"2024-05-03T18:22:40.755692+0000","last_unstale":"2024-05-03T18:24:21.548471+0000","last_undegraded":"2024-05-03T18:24:21.548471+0000","last_fullsized":"2024-05-03T18:24:21.548471+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:22:50.269817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.576278+0000","last_change":"2024-05-03T18:22:21.561381+0000","last_active":"2024-05-03T18:24:21.576278+0000","last_peered":"2024-05-03T18:24:21.576278+0000","last_clean":"2024-05-03T18:24:21.576278+0000","last_became_active":"2024-05-03T18:22:21.559411+0000","last_became_peered":"2024-05-03T18:22:21.559411+0000","last_unstale":"2024-05-03T18:24:21.576278+0000","last_undegraded":"2024-05-03T18:24:21.576278+0000","last_fullsized":"2024-05-03T18:24:21.576278+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:04:08.283538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.802312+0000","last_change":"2024-05-03T18:23:20.551361+0000","last_active":"2024-05-03T18:24:21.802312+0000","last_peered":"2024-05-03T18:24:21.802312+0000","last_clean":"2024-05-03T18:24:21.802312+0000","last_became_active":"2024-05-03T18:23:20.550689+0000","last_became_peered":"2024-05-03T18:23:20.550689+0000","last_unstale":"2024-05-03T18:24:21.802312+0000","last_undegraded":"2024-05-03T18:24:21.802312+0000","last_fullsized":"2024-05-03T18:24:21.802312+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:36:52.767110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.925637+0000","last_change":"2024-05-03T18:22:40.758006+0000","last_active":"2024-05-03T18:24:21.925637+0000","last_peered":"2024-05-03T18:24:21.925637+0000","last_clean":"2024-05-03T18:24:21.925637+0000","last_became_active":"2024-05-03T18:22:40.757075+0000","last_became_peered":"2024-05-03T18:22:40.757075+0000","last_unstale":"2024-05-03T18:24:21.925637+0000","last_undegraded":"2024-05-03T18:24:21.925637+0000","last_fullsized":"2024-05-03T18:24:21.925637+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:06:02.390184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"78'3","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-03T18:24:20.908745+0000","last_change":"2024-05-03T18:22:21.740041+0000","last_active":"2024-05-03T18:24:20.908745+0000","last_peered":"2024-05-03T18:24:20.908745+0000","last_clean":"2024-05-03T18:24:20.908745+0000","last_became_active":"2024-05-03T18:22:21.738777+0000","last_became_peered":"2024-05-03T18:22:21.738777+0000","last_unstale":"2024-05-03T18:24:20.908745+0000","last_undegraded":"2024-05-03T18:24:20.908745+0000","last_fullsized":"2024-05-03T18:24:20.908745+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:35:30.098325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.556062+0000","last_change":"2024-05-03T18:22:25.606854+0000","last_active":"2024-05-03T18:24:21.556062+0000","last_peered":"2024-05-03T18:24:21.556062+0000","last_clean":"2024-05-03T18:24:21.556062+0000","last_became_active":"2024-05-03T18:22:25.606376+0000","last_became_peered":"2024-05-03T18:22:25.606376+0000","last_unstale":"2024-05-03T18:24:21.556062+0000","last_undegraded":"2024-05-03T18:24:21.556062+0000","last_fullsized":"2024-05-03T18:24:21.556062+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:29:59.461181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.548796+0000","last_change":"2024-05-03T18:22:40.752863+0000","last_active":"2024-05-03T18:24:21.548796+0000","last_peered":"2024-05-03T18:24:21.548796+0000","last_clean":"2024-05-03T18:24:21.548796+0000","last_became_active":"2024-05-03T18:22:40.752309+0000","last_became_peered":"2024-05-03T18:22:40.752309+0000","last_unstale":"2024-05-03T18:24:21.548796+0000","last_undegraded":"2024-05-03T18:24:21.548796+0000","last_fullsized":"2024-05-03T18:24:21.548796+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:01:06.110570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"87'30","reported_seq":156,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.911733+0000","last_change":"2024-05-03T18:22:21.566162+0000","last_active":"2024-05-03T18:24:21.911733+0000","last_peered":"2024-05-03T18:24:21.911733+0000","last_clean":"2024-05-03T18:24:21.911733+0000","last_became_active":"2024-05-03T18:22:21.565641+0000","last_became_peered":"2024-05-03T18:22:21.565641+0000","last_unstale":"2024-05-03T18:24:21.911733+0000","last_undegraded":"2024-05-03T18:24:21.911733+0000","last_fullsized":"2024-05-03T18:24:21.911733+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:23:45.283135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56115,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":82,"num_read_kb":96,"num_write":30,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-03T18:24:20.918228+0000","last_change":"2024-05-03T18:22:25.586687+0000","last_active":"2024-05-03T18:24:20.918228+0000","last_peered":"2024-05-03T18:24:20.918228+0000","last_clean":"2024-05-03T18:24:20.918228+0000","last_became_active":"2024-05-03T18:22:25.586275+0000","last_became_peered":"2024-05-03T18:22:25.586275+0000","last_unstale":"2024-05-03T18:24:20.918228+0000","last_undegraded":"2024-05-03T18:24:20.918228+0000","last_fullsized":"2024-05-03T18:24:20.918228+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:10:50.339865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-03T18:24:20.909336+0000","last_change":"2024-05-03T18:22:40.758003+0000","last_active":"2024-05-03T18:24:20.909336+0000","last_peered":"2024-05-03T18:24:20.909336+0000","last_clean":"2024-05-03T18:24:20.909336+0000","last_became_active":"2024-05-03T18:22:40.757559+0000","last_became_peered":"2024-05-03T18:22:40.757559+0000","last_unstale":"2024-05-03T18:24:20.909336+0000","last_undegraded":"2024-05-03T18:24:20.909336+0000","last_fullsized":"2024-05-03T18:24:20.909336+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:54:23.377748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"78'2","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-03T18:24:20.922567+0000","last_change":"2024-05-03T18:22:21.564546+0000","last_active":"2024-05-03T18:24:20.922567+0000","last_peered":"2024-05-03T18:24:20.922567+0000","last_clean":"2024-05-03T18:24:20.922567+0000","last_became_active":"2024-05-03T18:22:21.564105+0000","last_became_peered":"2024-05-03T18:22:21.564105+0000","last_unstale":"2024-05-03T18:24:20.922567+0000","last_undegraded":"2024-05-03T18:24:20.922567+0000","last_fullsized":"2024-05-03T18:24:20.922567+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:36:07.790643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.923345+0000","last_change":"2024-05-03T18:22:25.584059+0000","last_active":"2024-05-03T18:24:21.923345+0000","last_peered":"2024-05-03T18:24:21.923345+0000","last_clean":"2024-05-03T18:24:21.923345+0000","last_became_active":"2024-05-03T18:22:25.583742+0000","last_became_peered":"2024-05-03T18:22:25.583742+0000","last_unstale":"2024-05-03T18:24:21.923345+0000","last_undegraded":"2024-05-03T18:24:21.923345+0000","last_fullsized":"2024-05-03T18:24:21.923345+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:15:48.288422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-03T18:24:20.538368+0000","last_change":"2024-05-03T18:22:40.751362+0000","last_active":"2024-05-03T18:24:20.538368+0000","last_peered":"2024-05-03T18:24:20.538368+0000","last_clean":"2024-05-03T18:24:20.538368+0000","last_became_active":"2024-05-03T18:22:40.750365+0000","last_became_peered":"2024-05-03T18:22:40.750365+0000","last_unstale":"2024-05-03T18:24:20.538368+0000","last_undegraded":"2024-05-03T18:24:20.538368+0000","last_fullsized":"2024-05-03T18:24:20.538368+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:35:50.629332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-03T18:24:20.908346+0000","last_change":"2024-05-03T18:22:21.556123+0000","last_active":"2024-05-03T18:24:20.908346+0000","last_peered":"2024-05-03T18:24:20.908346+0000","last_clean":"2024-05-03T18:24:20.908346+0000","last_became_active":"2024-05-03T18:22:21.554918+0000","last_became_peered":"2024-05-03T18:22:21.554918+0000","last_unstale":"2024-05-03T18:24:20.908346+0000","last_undegraded":"2024-05-03T18:24:20.908346+0000","last_fullsized":"2024-05-03T18:24:20.908346+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:09:12.383601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.577647+0000","last_change":"2024-05-03T18:22:25.601849+0000","last_active":"2024-05-03T18:24:21.577647+0000","last_peered":"2024-05-03T18:24:21.577647+0000","last_clean":"2024-05-03T18:24:21.577647+0000","last_became_active":"2024-05-03T18:22:25.601434+0000","last_became_peered":"2024-05-03T18:22:25.601434+0000","last_unstale":"2024-05-03T18:24:21.577647+0000","last_undegraded":"2024-05-03T18:24:21.577647+0000","last_fullsized":"2024-05-03T18:24:21.577647+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:43:18.273389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.549739+0000","last_change":"2024-05-03T18:22:40.754448+0000","last_active":"2024-05-03T18:24:21.549739+0000","last_peered":"2024-05-03T18:24:21.549739+0000","last_clean":"2024-05-03T18:24:21.549739+0000","last_became_active":"2024-05-03T18:22:40.753122+0000","last_became_peered":"2024-05-03T18:22:40.753122+0000","last_unstale":"2024-05-03T18:24:21.549739+0000","last_undegraded":"2024-05-03T18:24:21.549739+0000","last_fullsized":"2024-05-03T18:24:21.549739+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:43:15.216222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.924236+0000","last_change":"2024-05-03T18:22:21.559292+0000","last_active":"2024-05-03T18:24:21.924236+0000","last_peered":"2024-05-03T18:24:21.924236+0000","last_clean":"2024-05-03T18:24:21.924236+0000","last_became_active":"2024-05-03T18:22:21.557657+0000","last_became_peered":"2024-05-03T18:22:21.557657+0000","last_unstale":"2024-05-03T18:24:21.924236+0000","last_undegraded":"2024-05-03T18:24:21.924236+0000","last_fullsized":"2024-05-03T18:24:21.924236+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:51:57.709387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.576800+0000","last_change":"2024-05-03T18:22:25.604340+0000","last_active":"2024-05-03T18:24:21.576800+0000","last_peered":"2024-05-03T18:24:21.576800+0000","last_clean":"2024-05-03T18:24:21.576800+0000","last_became_active":"2024-05-03T18:22:25.603651+0000","last_became_peered":"2024-05-03T18:22:25.603651+0000","last_unstale":"2024-05-03T18:24:21.576800+0000","last_undegraded":"2024-05-03T18:24:21.576800+0000","last_fullsized":"2024-05-03T18:24:21.576800+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:49:36.379556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.923902+0000","last_change":"2024-05-03T18:22:40.738065+0000","last_active":"2024-05-03T18:24:21.923902+0000","last_peered":"2024-05-03T18:24:21.923902+0000","last_clean":"2024-05-03T18:24:21.923902+0000","last_became_active":"2024-05-03T18:22:40.737546+0000","last_became_peered":"2024-05-03T18:22:40.737546+0000","last_unstale":"2024-05-03T18:24:21.923902+0000","last_undegraded":"2024-05-03T18:24:21.923902+0000","last_fullsized":"2024-05-03T18:24:21.923902+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:45:31.528581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"78'8","reported_seq":50,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-03T18:24:20.915893+0000","last_change":"2024-05-03T18:22:21.739131+0000","last_active":"2024-05-03T18:24:20.915893+0000","last_peered":"2024-05-03T18:24:20.915893+0000","last_clean":"2024-05-03T18:24:20.915893+0000","last_became_active":"2024-05-03T18:22:21.737243+0000","last_became_peered":"2024-05-03T18:22:21.737243+0000","last_unstale":"2024-05-03T18:24:20.915893+0000","last_undegraded":"2024-05-03T18:24:20.915893+0000","last_fullsized":"2024-05-03T18:24:20.915893+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:46:24.011140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-03T18:24:20.908222+0000","last_change":"2024-05-03T18:22:25.602011+0000","last_active":"2024-05-03T18:24:20.908222+0000","last_peered":"2024-05-03T18:24:20.908222+0000","last_clean":"2024-05-03T18:24:20.908222+0000","last_became_active":"2024-05-03T18:22:25.601604+0000","last_became_peered":"2024-05-03T18:22:25.601604+0000","last_unstale":"2024-05-03T18:24:20.908222+0000","last_undegraded":"2024-05-03T18:24:20.908222+0000","last_fullsized":"2024-05-03T18:24:20.908222+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:25:57.743203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-03T18:24:20.908583+0000","last_change":"2024-05-03T18:22:40.757885+0000","last_active":"2024-05-03T18:24:20.908583+0000","last_peered":"2024-05-03T18:24:20.908583+0000","last_clean":"2024-05-03T18:24:20.908583+0000","last_became_active":"2024-05-03T18:22:40.757208+0000","last_became_peered":"2024-05-03T18:22:40.757208+0000","last_unstale":"2024-05-03T18:24:20.908583+0000","last_undegraded":"2024-05-03T18:24:20.908583+0000","last_fullsized":"2024-05-03T18:24:20.908583+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:37:00.792138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"78'1","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-03T18:24:20.918111+0000","last_change":"2024-05-03T18:22:21.554932+0000","last_active":"2024-05-03T18:24:20.918111+0000","last_peered":"2024-05-03T18:24:20.918111+0000","last_clean":"2024-05-03T18:24:20.918111+0000","last_became_active":"2024-05-03T18:22:21.554123+0000","last_became_peered":"2024-05-03T18:22:21.554123+0000","last_unstale":"2024-05-03T18:24:20.918111+0000","last_undegraded":"2024-05-03T18:24:20.918111+0000","last_fullsized":"2024-05-03T18:24:20.918111+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:38:59.859138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-03T18:24:20.538972+0000","last_change":"2024-05-03T18:22:25.603400+0000","last_active":"2024-05-03T18:24:20.538972+0000","last_peered":"2024-05-03T18:24:20.538972+0000","last_clean":"2024-05-03T18:24:20.538972+0000","last_became_active":"2024-05-03T18:22:25.603045+0000","last_became_peered":"2024-05-03T18:22:25.603045+0000","last_unstale":"2024-05-03T18:24:20.538972+0000","last_undegraded":"2024-05-03T18:24:20.538972+0000","last_fullsized":"2024-05-03T18:24:20.538972+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:51:33.502972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-03T18:24:20.916027+0000","last_change":"2024-05-03T18:22:40.750857+0000","last_active":"2024-05-03T18:24:20.916027+0000","last_peered":"2024-05-03T18:24:20.916027+0000","last_clean":"2024-05-03T18:24:20.916027+0000","last_became_active":"2024-05-03T18:22:40.750381+0000","last_became_peered":"2024-05-03T18:22:40.750381+0000","last_unstale":"2024-05-03T18:24:20.916027+0000","last_undegraded":"2024-05-03T18:24:20.916027+0000","last_fullsized":"2024-05-03T18:24:20.916027+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:15:28.542612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.549132+0000","last_change":"2024-05-03T18:22:21.739345+0000","last_active":"2024-05-03T18:24:21.549132+0000","last_peered":"2024-05-03T18:24:21.549132+0000","last_clean":"2024-05-03T18:24:21.549132+0000","last_became_active":"2024-05-03T18:22:21.738874+0000","last_became_peered":"2024-05-03T18:22:21.738874+0000","last_unstale":"2024-05-03T18:24:21.549132+0000","last_undegraded":"2024-05-03T18:24:21.549132+0000","last_fullsized":"2024-05-03T18:24:21.549132+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:57:59.706356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.551088+0000","last_change":"2024-05-03T18:22:25.586543+0000","last_active":"2024-05-03T18:24:21.551088+0000","last_peered":"2024-05-03T18:24:21.551088+0000","last_clean":"2024-05-03T18:24:21.551088+0000","last_became_active":"2024-05-03T18:22:25.585124+0000","last_became_peered":"2024-05-03T18:22:25.585124+0000","last_unstale":"2024-05-03T18:24:21.551088+0000","last_undegraded":"2024-05-03T18:24:21.551088+0000","last_fullsized":"2024-05-03T18:24:21.551088+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:41:32.633444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.923432+0000","last_change":"2024-05-03T18:22:40.749665+0000","last_active":"2024-05-03T18:24:21.923432+0000","last_peered":"2024-05-03T18:24:21.923432+0000","last_clean":"2024-05-03T18:24:21.923432+0000","last_became_active":"2024-05-03T18:22:40.749062+0000","last_became_peered":"2024-05-03T18:22:40.749062+0000","last_unstale":"2024-05-03T18:24:21.923432+0000","last_undegraded":"2024-05-03T18:24:21.923432+0000","last_fullsized":"2024-05-03T18:24:21.923432+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:09:17.194507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.575569+0000","last_change":"2024-05-03T18:22:21.560944+0000","last_active":"2024-05-03T18:24:21.575569+0000","last_peered":"2024-05-03T18:24:21.575569+0000","last_clean":"2024-05-03T18:24:21.575569+0000","last_became_active":"2024-05-03T18:22:21.560412+0000","last_became_peered":"2024-05-03T18:22:21.560412+0000","last_unstale":"2024-05-03T18:24:21.575569+0000","last_undegraded":"2024-05-03T18:24:21.575569+0000","last_fullsized":"2024-05-03T18:24:21.575569+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:12:04.069564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.550559+0000","last_change":"2024-05-03T18:22:25.598235+0000","last_active":"2024-05-03T18:24:21.550559+0000","last_peered":"2024-05-03T18:24:21.550559+0000","last_clean":"2024-05-03T18:24:21.550559+0000","last_became_active":"2024-05-03T18:22:25.595993+0000","last_became_peered":"2024-05-03T18:22:25.595993+0000","last_unstale":"2024-05-03T18:24:21.550559+0000","last_undegraded":"2024-05-03T18:24:21.550559+0000","last_fullsized":"2024-05-03T18:24:21.550559+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:53:16.069119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.924105+0000","last_change":"2024-05-03T18:22:40.745005+0000","last_active":"2024-05-03T18:24:21.924105+0000","last_peered":"2024-05-03T18:24:21.924105+0000","last_clean":"2024-05-03T18:24:21.924105+0000","last_became_active":"2024-05-03T18:22:40.742134+0000","last_became_peered":"2024-05-03T18:22:40.742134+0000","last_unstale":"2024-05-03T18:24:21.924105+0000","last_undegraded":"2024-05-03T18:24:21.924105+0000","last_fullsized":"2024-05-03T18:24:21.924105+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:47:12.217012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-03T18:24:20.909493+0000","last_change":"2024-05-03T18:22:25.604640+0000","last_active":"2024-05-03T18:24:20.909493+0000","last_peered":"2024-05-03T18:24:20.909493+0000","last_clean":"2024-05-03T18:24:20.909493+0000","last_became_active":"2024-05-03T18:22:25.603759+0000","last_became_peered":"2024-05-03T18:22:25.603759+0000","last_unstale":"2024-05-03T18:24:20.909493+0000","last_undegraded":"2024-05-03T18:24:20.909493+0000","last_fullsized":"2024-05-03T18:24:20.909493+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:56:55.744321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.922932+0000","last_change":"2024-05-03T18:22:21.741006+0000","last_active":"2024-05-03T18:24:21.922932+0000","last_peered":"2024-05-03T18:24:21.922932+0000","last_clean":"2024-05-03T18:24:21.922932+0000","last_became_active":"2024-05-03T18:22:21.737999+0000","last_became_peered":"2024-05-03T18:22:21.737999+0000","last_unstale":"2024-05-03T18:24:21.922932+0000","last_undegraded":"2024-05-03T18:24:21.922932+0000","last_fullsized":"2024-05-03T18:24:21.922932+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:29:54.931940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.577867+0000","last_change":"2024-05-03T18:22:40.761044+0000","last_active":"2024-05-03T18:24:21.577867+0000","last_peered":"2024-05-03T18:24:21.577867+0000","last_clean":"2024-05-03T18:24:21.577867+0000","last_became_active":"2024-05-03T18:22:40.760245+0000","last_became_peered":"2024-05-03T18:22:40.760245+0000","last_unstale":"2024-05-03T18:24:21.577867+0000","last_undegraded":"2024-05-03T18:24:21.577867+0000","last_fullsized":"2024-05-03T18:24:21.577867+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:31:12.729078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.629111+0000","last_change":"2024-05-03T18:22:25.595759+0000","last_active":"2024-05-03T18:24:21.629111+0000","last_peered":"2024-05-03T18:24:21.629111+0000","last_clean":"2024-05-03T18:24:21.629111+0000","last_became_active":"2024-05-03T18:22:25.594368+0000","last_became_peered":"2024-05-03T18:22:25.594368+0000","last_unstale":"2024-05-03T18:24:21.629111+0000","last_undegraded":"2024-05-03T18:24:21.629111+0000","last_fullsized":"2024-05-03T18:24:21.629111+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:26:19.196877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.924331+0000","last_change":"2024-05-03T18:22:21.557123+0000","last_active":"2024-05-03T18:24:21.924331+0000","last_peered":"2024-05-03T18:24:21.924331+0000","last_clean":"2024-05-03T18:24:21.924331+0000","last_became_active":"2024-05-03T18:22:21.556432+0000","last_became_peered":"2024-05-03T18:22:21.556432+0000","last_unstale":"2024-05-03T18:24:21.924331+0000","last_undegraded":"2024-05-03T18:24:21.924331+0000","last_fullsized":"2024-05-03T18:24:21.924331+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:50:03.179320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":119,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.576282+0000","last_change":"2024-05-03T18:20:24.937044+0000","last_active":"2024-05-03T18:24:21.576282+0000","last_peered":"2024-05-03T18:24:21.576282+0000","last_clean":"2024-05-03T18:24:21.576282+0000","last_became_active":"2024-05-03T18:20:24.922095+0000","last_became_peered":"2024-05-03T18:20:24.922095+0000","last_unstale":"2024-05-03T18:24:21.576282+0000","last_undegraded":"2024-05-03T18:24:21.576282+0000","last_fullsized":"2024-05-03T18:24:21.576282+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:15:37.353911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:15:37.353911+0000","last_clean_scrub_stamp":"2024-05-03T18:15:37.353911+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:40:07.341441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-03T18:24:20.917001+0000","last_change":"2024-05-03T18:23:20.575064+0000","last_active":"2024-05-03T18:24:20.917001+0000","last_peered":"2024-05-03T18:24:20.917001+0000","last_clean":"2024-05-03T18:24:20.917001+0000","last_became_active":"2024-05-03T18:23:20.574557+0000","last_became_peered":"2024-05-03T18:23:20.574557+0000","last_unstale":"2024-05-03T18:24:20.917001+0000","last_undegraded":"2024-05-03T18:24:20.917001+0000","last_fullsized":"2024-05-03T18:24:20.917001+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:34:08.520396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,6,10],"acting":[8,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.923143+0000","last_change":"2024-05-03T18:22:25.591269+0000","last_active":"2024-05-03T18:24:21.923143+0000","last_peered":"2024-05-03T18:24:21.923143+0000","last_clean":"2024-05-03T18:24:21.923143+0000","last_became_active":"2024-05-03T18:22:25.590409+0000","last_became_peered":"2024-05-03T18:22:25.590409+0000","last_unstale":"2024-05-03T18:24:21.923143+0000","last_undegraded":"2024-05-03T18:24:21.923143+0000","last_fullsized":"2024-05-03T18:24:21.923143+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:03:57.432331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.576656+0000","last_change":"2024-05-03T18:22:21.560137+0000","last_active":"2024-05-03T18:24:21.576656+0000","last_peered":"2024-05-03T18:24:21.576656+0000","last_clean":"2024-05-03T18:24:21.576656+0000","last_became_active":"2024-05-03T18:22:21.559672+0000","last_became_peered":"2024-05-03T18:22:21.559672+0000","last_unstale":"2024-05-03T18:24:21.576656+0000","last_undegraded":"2024-05-03T18:24:21.576656+0000","last_fullsized":"2024-05-03T18:24:21.576656+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:20:02.070390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-03T18:24:20.538212+0000","last_change":"2024-05-03T18:22:40.753160+0000","last_active":"2024-05-03T18:24:20.538212+0000","last_peered":"2024-05-03T18:24:20.538212+0000","last_clean":"2024-05-03T18:24:20.538212+0000","last_became_active":"2024-05-03T18:22:40.750800+0000","last_became_peered":"2024-05-03T18:22:40.750800+0000","last_unstale":"2024-05-03T18:24:20.538212+0000","last_undegraded":"2024-05-03T18:24:20.538212+0000","last_fullsized":"2024-05-03T18:24:20.538212+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:27:48.756140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-03T18:24:20.917748+0000","last_change":"2024-05-03T18:22:21.739490+0000","last_active":"2024-05-03T18:24:20.917748+0000","last_peered":"2024-05-03T18:24:20.917748+0000","last_clean":"2024-05-03T18:24:20.917748+0000","last_became_active":"2024-05-03T18:22:21.738512+0000","last_became_peered":"2024-05-03T18:22:21.738512+0000","last_unstale":"2024-05-03T18:24:20.917748+0000","last_undegraded":"2024-05-03T18:24:20.917748+0000","last_fullsized":"2024-05-03T18:24:20.917748+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:26:45.323519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.923544+0000","last_change":"2024-05-03T18:22:25.591509+0000","last_active":"2024-05-03T18:24:21.923544+0000","last_peered":"2024-05-03T18:24:21.923544+0000","last_clean":"2024-05-03T18:24:21.923544+0000","last_became_active":"2024-05-03T18:22:25.590188+0000","last_became_peered":"2024-05-03T18:22:25.590188+0000","last_unstale":"2024-05-03T18:24:21.923544+0000","last_undegraded":"2024-05-03T18:24:21.923544+0000","last_fullsized":"2024-05-03T18:24:21.923544+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:33:23.919101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.627483+0000","last_change":"2024-05-03T18:22:40.747176+0000","last_active":"2024-05-03T18:24:21.627483+0000","last_peered":"2024-05-03T18:24:21.627483+0000","last_clean":"2024-05-03T18:24:21.627483+0000","last_became_active":"2024-05-03T18:22:40.746526+0000","last_became_peered":"2024-05-03T18:22:40.746526+0000","last_unstale":"2024-05-03T18:24:21.627483+0000","last_undegraded":"2024-05-03T18:24:21.627483+0000","last_fullsized":"2024-05-03T18:24:21.627483+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:59:25.838382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"87'7","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.549662+0000","last_change":"2024-05-03T18:22:21.555978+0000","last_active":"2024-05-03T18:24:21.549662+0000","last_peered":"2024-05-03T18:24:21.549662+0000","last_clean":"2024-05-03T18:24:21.549662+0000","last_became_active":"2024-05-03T18:22:21.555395+0000","last_became_peered":"2024-05-03T18:22:21.555395+0000","last_unstale":"2024-05-03T18:24:21.549662+0000","last_undegraded":"2024-05-03T18:24:21.549662+0000","last_fullsized":"2024-05-03T18:24:21.549662+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:05:44.795978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.628608+0000","last_change":"2024-05-03T18:22:25.589071+0000","last_active":"2024-05-03T18:24:21.628608+0000","last_peered":"2024-05-03T18:24:21.628608+0000","last_clean":"2024-05-03T18:24:21.628608+0000","last_became_active":"2024-05-03T18:22:25.588679+0000","last_became_peered":"2024-05-03T18:22:25.588679+0000","last_unstale":"2024-05-03T18:24:21.628608+0000","last_undegraded":"2024-05-03T18:24:21.628608+0000","last_fullsized":"2024-05-03T18:24:21.628608+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:20:53.983166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-03T18:24:20.921781+0000","last_change":"2024-05-03T18:22:40.755154+0000","last_active":"2024-05-03T18:24:20.921781+0000","last_peered":"2024-05-03T18:24:20.921781+0000","last_clean":"2024-05-03T18:24:20.921781+0000","last_became_active":"2024-05-03T18:22:40.754635+0000","last_became_peered":"2024-05-03T18:22:40.754635+0000","last_unstale":"2024-05-03T18:24:20.921781+0000","last_undegraded":"2024-05-03T18:24:20.921781+0000","last_fullsized":"2024-05-03T18:24:20.921781+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:25:56.787406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"78'2","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.924388+0000","last_change":"2024-05-03T18:22:21.854322+0000","last_active":"2024-05-03T18:24:21.924388+0000","last_peered":"2024-05-03T18:24:21.924388+0000","last_clean":"2024-05-03T18:24:21.924388+0000","last_became_active":"2024-05-03T18:22:21.853359+0000","last_became_peered":"2024-05-03T18:22:21.853359+0000","last_unstale":"2024-05-03T18:24:21.924388+0000","last_undegraded":"2024-05-03T18:24:21.924388+0000","last_fullsized":"2024-05-03T18:24:21.924388+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:06:57.061775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.802187+0000","last_change":"2024-05-03T18:22:25.605614+0000","last_active":"2024-05-03T18:24:21.802187+0000","last_peered":"2024-05-03T18:24:21.802187+0000","last_clean":"2024-05-03T18:24:21.802187+0000","last_became_active":"2024-05-03T18:22:25.604277+0000","last_became_peered":"2024-05-03T18:22:25.604277+0000","last_unstale":"2024-05-03T18:24:21.802187+0000","last_undegraded":"2024-05-03T18:24:21.802187+0000","last_fullsized":"2024-05-03T18:24:21.802187+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:17:09.812304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.926174+0000","last_change":"2024-05-03T18:22:40.763353+0000","last_active":"2024-05-03T18:24:21.926174+0000","last_peered":"2024-05-03T18:24:21.926174+0000","last_clean":"2024-05-03T18:24:21.926174+0000","last_became_active":"2024-05-03T18:22:40.762774+0000","last_became_peered":"2024-05-03T18:22:40.762774+0000","last_unstale":"2024-05-03T18:24:21.926174+0000","last_undegraded":"2024-05-03T18:24:21.926174+0000","last_fullsized":"2024-05-03T18:24:21.926174+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:38:37.247050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.803989+0000","last_change":"2024-05-03T18:24:21.803989+0000","last_active":"2024-05-03T18:24:21.803989+0000","last_peered":"2024-05-03T18:24:21.803989+0000","last_clean":"2024-05-03T18:24:21.803989+0000","last_became_active":"2024-05-03T18:24:21.803072+0000","last_became_peered":"2024-05-03T18:24:21.803072+0000","last_unstale":"2024-05-03T18:24:21.803989+0000","last_undegraded":"2024-05-03T18:24:21.803989+0000","last_fullsized":"2024-05-03T18:24:21.803989+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:32:30.562148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-03T18:24:20.915938+0000","last_change":"2024-05-03T18:22:25.588011+0000","last_active":"2024-05-03T18:24:20.915938+0000","last_peered":"2024-05-03T18:24:20.915938+0000","last_clean":"2024-05-03T18:24:20.915938+0000","last_became_active":"2024-05-03T18:22:25.587602+0000","last_became_peered":"2024-05-03T18:22:25.587602+0000","last_unstale":"2024-05-03T18:24:20.915938+0000","last_undegraded":"2024-05-03T18:24:20.915938+0000","last_fullsized":"2024-05-03T18:24:20.915938+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:18:13.186088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.549997+0000","last_change":"2024-05-03T18:22:40.755731+0000","last_active":"2024-05-03T18:24:21.549997+0000","last_peered":"2024-05-03T18:24:21.549997+0000","last_clean":"2024-05-03T18:24:21.549997+0000","last_became_active":"2024-05-03T18:22:40.755173+0000","last_became_peered":"2024-05-03T18:22:40.755173+0000","last_unstale":"2024-05-03T18:24:21.549997+0000","last_undegraded":"2024-05-03T18:24:21.549997+0000","last_fullsized":"2024-05-03T18:24:21.549997+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:36:36.548095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.627138+0000","last_change":"2024-05-03T18:22:21.561481+0000","last_active":"2024-05-03T18:24:21.627138+0000","last_peered":"2024-05-03T18:24:21.627138+0000","last_clean":"2024-05-03T18:24:21.627138+0000","last_became_active":"2024-05-03T18:22:21.560088+0000","last_became_peered":"2024-05-03T18:22:21.560088+0000","last_unstale":"2024-05-03T18:24:21.627138+0000","last_undegraded":"2024-05-03T18:24:21.627138+0000","last_fullsized":"2024-05-03T18:24:21.627138+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:45:40.184813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-03T18:24:20.538152+0000","last_change":"2024-05-03T18:22:25.584556+0000","last_active":"2024-05-03T18:24:20.538152+0000","last_peered":"2024-05-03T18:24:20.538152+0000","last_clean":"2024-05-03T18:24:20.538152+0000","last_became_active":"2024-05-03T18:22:25.583873+0000","last_became_peered":"2024-05-03T18:22:25.583873+0000","last_unstale":"2024-05-03T18:24:20.538152+0000","last_undegraded":"2024-05-03T18:24:20.538152+0000","last_fullsized":"2024-05-03T18:24:20.538152+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:28:39.028662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.925915+0000","last_change":"2024-05-03T18:22:40.763479+0000","last_active":"2024-05-03T18:24:21.925915+0000","last_peered":"2024-05-03T18:24:21.925915+0000","last_clean":"2024-05-03T18:24:21.925915+0000","last_became_active":"2024-05-03T18:22:40.762897+0000","last_became_peered":"2024-05-03T18:22:40.762897+0000","last_unstale":"2024-05-03T18:24:21.925915+0000","last_undegraded":"2024-05-03T18:24:21.925915+0000","last_fullsized":"2024-05-03T18:24:21.925915+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:00:21.090730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-03T18:24:20.921952+0000","last_change":"2024-05-03T18:22:21.562929+0000","last_active":"2024-05-03T18:24:20.921952+0000","last_peered":"2024-05-03T18:24:20.921952+0000","last_clean":"2024-05-03T18:24:20.921952+0000","last_became_active":"2024-05-03T18:22:21.560712+0000","last_became_peered":"2024-05-03T18:22:21.560712+0000","last_unstale":"2024-05-03T18:24:20.921952+0000","last_undegraded":"2024-05-03T18:24:20.921952+0000","last_fullsized":"2024-05-03T18:24:20.921952+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:31:34.722437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.628357+0000","last_change":"2024-05-03T18:22:25.581971+0000","last_active":"2024-05-03T18:24:21.628357+0000","last_peered":"2024-05-03T18:24:21.628357+0000","last_clean":"2024-05-03T18:24:21.628357+0000","last_became_active":"2024-05-03T18:22:25.581565+0000","last_became_peered":"2024-05-03T18:22:25.581565+0000","last_unstale":"2024-05-03T18:24:21.628357+0000","last_undegraded":"2024-05-03T18:24:21.628357+0000","last_fullsized":"2024-05-03T18:24:21.628357+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:10:20.607999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.925930+0000","last_change":"2024-05-03T18:22:40.761458+0000","last_active":"2024-05-03T18:24:21.925930+0000","last_peered":"2024-05-03T18:24:21.925930+0000","last_clean":"2024-05-03T18:24:21.925930+0000","last_became_active":"2024-05-03T18:22:40.760969+0000","last_became_peered":"2024-05-03T18:22:40.760969+0000","last_unstale":"2024-05-03T18:24:21.925930+0000","last_undegraded":"2024-05-03T18:24:21.925930+0000","last_fullsized":"2024-05-03T18:24:21.925930+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:19:16.804720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"78'2","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-03T18:24:20.916772+0000","last_change":"2024-05-03T18:22:21.565869+0000","last_active":"2024-05-03T18:24:20.916772+0000","last_peered":"2024-05-03T18:24:20.916772+0000","last_clean":"2024-05-03T18:24:20.916772+0000","last_became_active":"2024-05-03T18:22:21.564119+0000","last_became_peered":"2024-05-03T18:22:21.564119+0000","last_unstale":"2024-05-03T18:24:20.916772+0000","last_undegraded":"2024-05-03T18:24:20.916772+0000","last_fullsized":"2024-05-03T18:24:20.916772+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:50:29.894070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.577267+0000","last_change":"2024-05-03T18:22:25.596199+0000","last_active":"2024-05-03T18:24:21.577267+0000","last_peered":"2024-05-03T18:24:21.577267+0000","last_clean":"2024-05-03T18:24:21.577267+0000","last_became_active":"2024-05-03T18:22:25.595167+0000","last_became_peered":"2024-05-03T18:22:25.595167+0000","last_unstale":"2024-05-03T18:24:21.577267+0000","last_undegraded":"2024-05-03T18:24:21.577267+0000","last_fullsized":"2024-05-03T18:24:21.577267+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:28:56.599064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.556598+0000","last_change":"2024-05-03T18:22:40.753020+0000","last_active":"2024-05-03T18:24:21.556598+0000","last_peered":"2024-05-03T18:24:21.556598+0000","last_clean":"2024-05-03T18:24:21.556598+0000","last_became_active":"2024-05-03T18:22:40.752502+0000","last_became_peered":"2024-05-03T18:22:40.752502+0000","last_unstale":"2024-05-03T18:24:21.556598+0000","last_undegraded":"2024-05-03T18:24:21.556598+0000","last_fullsized":"2024-05-03T18:24:21.556598+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:33:51.277879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-03T18:24:20.922987+0000","last_change":"2024-05-03T18:22:21.854481+0000","last_active":"2024-05-03T18:24:20.922987+0000","last_peered":"2024-05-03T18:24:20.922987+0000","last_clean":"2024-05-03T18:24:20.922987+0000","last_became_active":"2024-05-03T18:22:21.853172+0000","last_became_peered":"2024-05-03T18:22:21.853172+0000","last_unstale":"2024-05-03T18:24:20.922987+0000","last_undegraded":"2024-05-03T18:24:20.922987+0000","last_fullsized":"2024-05-03T18:24:20.922987+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:11:47.820105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.578275+0000","last_change":"2024-05-03T18:22:25.600856+0000","last_active":"2024-05-03T18:24:21.578275+0000","last_peered":"2024-05-03T18:24:21.578275+0000","last_clean":"2024-05-03T18:24:21.578275+0000","last_became_active":"2024-05-03T18:22:25.595351+0000","last_became_peered":"2024-05-03T18:22:25.595351+0000","last_unstale":"2024-05-03T18:24:21.578275+0000","last_undegraded":"2024-05-03T18:24:21.578275+0000","last_fullsized":"2024-05-03T18:24:21.578275+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:29:24.232506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.801462+0000","last_change":"2024-05-03T18:23:20.578331+0000","last_active":"2024-05-03T18:24:21.801462+0000","last_peered":"2024-05-03T18:24:21.801462+0000","last_clean":"2024-05-03T18:24:21.801462+0000","last_became_active":"2024-05-03T18:23:20.576974+0000","last_became_peered":"2024-05-03T18:23:20.576974+0000","last_unstale":"2024-05-03T18:24:21.801462+0000","last_undegraded":"2024-05-03T18:24:21.801462+0000","last_fullsized":"2024-05-03T18:24:21.801462+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:57:32.922175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-03T18:24:20.909004+0000","last_change":"2024-05-03T18:22:21.552331+0000","last_active":"2024-05-03T18:24:20.909004+0000","last_peered":"2024-05-03T18:24:20.909004+0000","last_clean":"2024-05-03T18:24:20.909004+0000","last_became_active":"2024-05-03T18:22:21.549786+0000","last_became_peered":"2024-05-03T18:22:21.549786+0000","last_unstale":"2024-05-03T18:24:20.909004+0000","last_undegraded":"2024-05-03T18:24:20.909004+0000","last_fullsized":"2024-05-03T18:24:20.909004+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:08:18.513405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.577652+0000","last_change":"2024-05-03T18:22:25.596437+0000","last_active":"2024-05-03T18:24:21.577652+0000","last_peered":"2024-05-03T18:24:21.577652+0000","last_clean":"2024-05-03T18:24:21.577652+0000","last_became_active":"2024-05-03T18:22:25.594040+0000","last_became_peered":"2024-05-03T18:22:25.594040+0000","last_unstale":"2024-05-03T18:24:21.577652+0000","last_undegraded":"2024-05-03T18:24:21.577652+0000","last_fullsized":"2024-05-03T18:24:21.577652+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:09:36.473565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-03T18:24:20.907890+0000","last_change":"2024-05-03T18:22:40.754641+0000","last_active":"2024-05-03T18:24:20.907890+0000","last_peered":"2024-05-03T18:24:20.907890+0000","last_clean":"2024-05-03T18:24:20.907890+0000","last_became_active":"2024-05-03T18:22:40.754166+0000","last_became_peered":"2024-05-03T18:22:40.754166+0000","last_unstale":"2024-05-03T18:24:20.907890+0000","last_undegraded":"2024-05-03T18:24:20.907890+0000","last_fullsized":"2024-05-03T18:24:20.907890+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:29:50.021671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.922955+0000","last_change":"2024-05-03T18:22:21.559354+0000","last_active":"2024-05-03T18:24:21.922955+0000","last_peered":"2024-05-03T18:24:21.922955+0000","last_clean":"2024-05-03T18:24:21.922955+0000","last_became_active":"2024-05-03T18:22:21.558680+0000","last_became_peered":"2024-05-03T18:22:21.558680+0000","last_unstale":"2024-05-03T18:24:21.922955+0000","last_undegraded":"2024-05-03T18:24:21.922955+0000","last_fullsized":"2024-05-03T18:24:21.922955+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:13:04.001847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.578440+0000","last_change":"2024-05-03T18:22:25.590746+0000","last_active":"2024-05-03T18:24:21.578440+0000","last_peered":"2024-05-03T18:24:21.578440+0000","last_clean":"2024-05-03T18:24:21.578440+0000","last_became_active":"2024-05-03T18:22:25.587906+0000","last_became_peered":"2024-05-03T18:22:25.587906+0000","last_unstale":"2024-05-03T18:24:21.578440+0000","last_undegraded":"2024-05-03T18:24:21.578440+0000","last_fullsized":"2024-05-03T18:24:21.578440+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:19:43.410259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-03T18:24:20.538494+0000","last_change":"2024-05-03T18:22:25.597628+0000","last_active":"2024-05-03T18:24:20.538494+0000","last_peered":"2024-05-03T18:24:20.538494+0000","last_clean":"2024-05-03T18:24:20.538494+0000","last_became_active":"2024-05-03T18:22:25.596535+0000","last_became_peered":"2024-05-03T18:22:25.596535+0000","last_unstale":"2024-05-03T18:24:20.538494+0000","last_undegraded":"2024-05-03T18:24:20.538494+0000","last_fullsized":"2024-05-03T18:24:20.538494+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:54:52.632258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-03T18:24:20.922413+0000","last_change":"2024-05-03T18:22:25.601474+0000","last_active":"2024-05-03T18:24:20.922413+0000","last_peered":"2024-05-03T18:24:20.922413+0000","last_clean":"2024-05-03T18:24:20.922413+0000","last_became_active":"2024-05-03T18:22:25.601097+0000","last_became_peered":"2024-05-03T18:22:25.601097+0000","last_unstale":"2024-05-03T18:24:20.922413+0000","last_undegraded":"2024-05-03T18:24:20.922413+0000","last_fullsized":"2024-05-03T18:24:20.922413+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:14:46.001500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-03T18:24:20.921940+0000","last_change":"2024-05-03T18:23:20.535409+0000","last_active":"2024-05-03T18:24:20.921940+0000","last_peered":"2024-05-03T18:24:20.921940+0000","last_clean":"2024-05-03T18:24:20.921940+0000","last_became_active":"2024-05-03T18:23:20.534883+0000","last_became_peered":"2024-05-03T18:23:20.534883+0000","last_unstale":"2024-05-03T18:24:20.921940+0000","last_undegraded":"2024-05-03T18:24:20.921940+0000","last_fullsized":"2024-05-03T18:24:20.921940+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:26:37.252898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.801597+0000","last_change":"2024-05-03T18:22:25.607115+0000","last_active":"2024-05-03T18:24:21.801597+0000","last_peered":"2024-05-03T18:24:21.801597+0000","last_clean":"2024-05-03T18:24:21.801597+0000","last_became_active":"2024-05-03T18:22:25.606764+0000","last_became_peered":"2024-05-03T18:22:25.606764+0000","last_unstale":"2024-05-03T18:24:21.801597+0000","last_undegraded":"2024-05-03T18:24:21.801597+0000","last_fullsized":"2024-05-03T18:24:21.801597+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:44:37.165032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.923726+0000","last_change":"2024-05-03T18:22:25.601886+0000","last_active":"2024-05-03T18:24:21.923726+0000","last_peered":"2024-05-03T18:24:21.923726+0000","last_clean":"2024-05-03T18:24:21.923726+0000","last_became_active":"2024-05-03T18:22:25.601494+0000","last_became_peered":"2024-05-03T18:22:25.601494+0000","last_unstale":"2024-05-03T18:24:21.923726+0000","last_undegraded":"2024-05-03T18:24:21.923726+0000","last_fullsized":"2024-05-03T18:24:21.923726+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:35:30.080124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-03T18:24:20.539233+0000","last_change":"2024-05-03T18:23:20.529091+0000","last_active":"2024-05-03T18:24:20.539233+0000","last_peered":"2024-05-03T18:24:20.539233+0000","last_clean":"2024-05-03T18:24:20.539233+0000","last_became_active":"2024-05-03T18:23:20.528288+0000","last_became_peered":"2024-05-03T18:23:20.528288+0000","last_unstale":"2024-05-03T18:24:20.539233+0000","last_undegraded":"2024-05-03T18:24:20.539233+0000","last_fullsized":"2024-05-03T18:24:20.539233+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:32:25.749461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-03T18:24:20.923462+0000","last_change":"2024-05-03T18:22:25.595367+0000","last_active":"2024-05-03T18:24:20.923462+0000","last_peered":"2024-05-03T18:24:20.923462+0000","last_clean":"2024-05-03T18:24:20.923462+0000","last_became_active":"2024-05-03T18:22:25.594987+0000","last_became_peered":"2024-05-03T18:22:25.594987+0000","last_unstale":"2024-05-03T18:24:20.923462+0000","last_undegraded":"2024-05-03T18:24:20.923462+0000","last_fullsized":"2024-05-03T18:24:20.923462+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:40:55.385595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.628825+0000","last_change":"2024-05-03T18:22:25.588320+0000","last_active":"2024-05-03T18:24:21.628825+0000","last_peered":"2024-05-03T18:24:21.628825+0000","last_clean":"2024-05-03T18:24:21.628825+0000","last_became_active":"2024-05-03T18:22:25.587861+0000","last_became_peered":"2024-05-03T18:22:25.587861+0000","last_unstale":"2024-05-03T18:24:21.628825+0000","last_undegraded":"2024-05-03T18:24:21.628825+0000","last_fullsized":"2024-05-03T18:24:21.628825+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:17:02.538798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-03T18:24:20.908263+0000","last_change":"2024-05-03T18:22:25.597874+0000","last_active":"2024-05-03T18:24:20.908263+0000","last_peered":"2024-05-03T18:24:20.908263+0000","last_clean":"2024-05-03T18:24:20.908263+0000","last_became_active":"2024-05-03T18:22:25.597448+0000","last_became_peered":"2024-05-03T18:22:25.597448+0000","last_unstale":"2024-05-03T18:24:20.908263+0000","last_undegraded":"2024-05-03T18:24:20.908263+0000","last_fullsized":"2024-05-03T18:24:20.908263+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:55:52.894491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.627908+0000","last_change":"2024-05-03T18:22:25.593509+0000","last_active":"2024-05-03T18:24:21.627908+0000","last_peered":"2024-05-03T18:24:21.627908+0000","last_clean":"2024-05-03T18:24:21.627908+0000","last_became_active":"2024-05-03T18:22:25.590736+0000","last_became_peered":"2024-05-03T18:22:25.590736+0000","last_unstale":"2024-05-03T18:24:21.627908+0000","last_undegraded":"2024-05-03T18:24:21.627908+0000","last_fullsized":"2024-05-03T18:24:21.627908+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:54:45.431776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.556246+0000","last_change":"2024-05-03T18:22:25.590022+0000","last_active":"2024-05-03T18:24:21.556246+0000","last_peered":"2024-05-03T18:24:21.556246+0000","last_clean":"2024-05-03T18:24:21.556246+0000","last_became_active":"2024-05-03T18:22:25.589432+0000","last_became_peered":"2024-05-03T18:22:25.589432+0000","last_unstale":"2024-05-03T18:24:21.556246+0000","last_undegraded":"2024-05-03T18:24:21.556246+0000","last_fullsized":"2024-05-03T18:24:21.556246+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:03:02.091228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.923486+0000","last_change":"2024-05-03T18:22:25.581452+0000","last_active":"2024-05-03T18:24:21.923486+0000","last_peered":"2024-05-03T18:24:21.923486+0000","last_clean":"2024-05-03T18:24:21.923486+0000","last_became_active":"2024-05-03T18:22:25.581098+0000","last_became_peered":"2024-05-03T18:22:25.581098+0000","last_unstale":"2024-05-03T18:24:21.923486+0000","last_undegraded":"2024-05-03T18:24:21.923486+0000","last_fullsized":"2024-05-03T18:24:21.923486+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:04:46.551234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-03T18:24:20.538650+0000","last_change":"2024-05-03T18:22:25.589105+0000","last_active":"2024-05-03T18:24:20.538650+0000","last_peered":"2024-05-03T18:24:20.538650+0000","last_clean":"2024-05-03T18:24:20.538650+0000","last_became_active":"2024-05-03T18:22:25.588697+0000","last_became_peered":"2024-05-03T18:22:25.588697+0000","last_unstale":"2024-05-03T18:24:20.538650+0000","last_undegraded":"2024-05-03T18:24:20.538650+0000","last_fullsized":"2024-05-03T18:24:20.538650+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:41:23.563296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":56913,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":88,"num_read_kb":102,"num_write":81,"num_write_kb":84,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":258048,"data_stored":170739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":69,"ondisk_log_size":69,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710763,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32160,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712992,"statfs":{"total":95995035648,"available":95962103808,"internally_reserved":0,"allocated":1003520,"data_stored":504990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906994,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28672,"kb_used_data":1648,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716480,"statfs":{"total":95995035648,"available":95965675520,"internally_reserved":0,"allocated":1687552,"data_stored":1156916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070521,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1028096,"data_stored":519013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266751,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":995328,"data_stored":496207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462985,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32768,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712384,"statfs":{"total":95995035648,"available":95961481216,"internally_reserved":0,"allocated":1622016,"data_stored":1109381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626512,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1015808,"data_stored":510239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574963,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32240,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712912,"statfs":{"total":95995035648,"available":95962021888,"internally_reserved":0,"allocated":1081344,"data_stored":566354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738490,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1019904,"data_stored":510329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411436,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32224,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712928,"statfs":{"total":95995035648,"available":95962038272,"internally_reserved":0,"allocated":1069056,"data_stored":557043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182499,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1007616,"data_stored":500998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986269,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1007616,"data_stored":505012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822741,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32752,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712400,"statfs":{"total":95995035648,"available":95961497600,"internally_reserved":0,"allocated":1609728,"data_stored":1095540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":56115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":56149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":56309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-03T18:24:27.363 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-03T18:24:27.364 INFO:tasks.cephadm:Setting up client nodes... 2024-05-03T18:24:27.364 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-05-03T18:24:27.871 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:24:28.277 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:24:28 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:24:28.277 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:24:28 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:24:28.277 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:24:28 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:24:28.277 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:24:28 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:24:28.277 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:24:28 smithi033 ceph-mon[28330]: pgmap v443: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-05-03T18:24:28.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:24:28 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:24:28.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:24:28 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:24:28.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:24:28 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:24:28.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:24:28 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:24:28.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:24:28 smithi099 ceph-mon[31658]: pgmap v443: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-05-03T18:24:28.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:24:28 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:24:28.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:24:28 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:24:28.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:24:28 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:24:28.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:24:28 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:24:28.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:24:28 smithi150 ceph-mon[32492]: pgmap v443: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-05-03T18:24:30.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:30.274+0000 7f0d18200700 1 -- 172.21.15.33:0/2356880497 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d100790a0 msgr2=0x7f0d10079520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:24:30.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:30.274+0000 7f0d18200700 1 --2- 172.21.15.33:0/2356880497 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d100790a0 0x7f0d10079520 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc009990 tx=0x7f0cfc009ca0 comp rx=0 tx=0).stop 2024-05-03T18:24:30.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:30.274+0000 7f0d18200700 1 -- 172.21.15.33:0/2356880497 shutdown_connections 2024-05-03T18:24:30.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:30.274+0000 7f0d18200700 1 --2- 172.21.15.33:0/2356880497 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d10079a60 0x7f0d1007a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:30.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:30.274+0000 7f0d18200700 1 --2- 172.21.15.33:0/2356880497 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0d10077e50 0x7f0d10078270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:30.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:30.274+0000 7f0d18200700 1 --2- 172.21.15.33:0/2356880497 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d100790a0 0x7f0d10079520 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:30.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:30.274+0000 7f0d18200700 1 -- 172.21.15.33:0/2356880497 >> 172.21.15.33:0/2356880497 conn(0x7f0d10100cf0 msgr2=0x7f0d10103110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:24:30.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:30.274+0000 7f0d18200700 1 -- 172.21.15.33:0/2356880497 shutdown_connections 2024-05-03T18:24:30.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:30.274+0000 7f0d18200700 1 -- 172.21.15.33:0/2356880497 wait complete. 2024-05-03T18:24:30.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:30.276+0000 7f0d18200700 1 Processor -- start 2024-05-03T18:24:30.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:30.276+0000 7f0d18200700 1 -- start start 2024-05-03T18:24:30.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:30.276+0000 7f0d18200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d10077e50 0x7f0d1019fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:24:30.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:30.276+0000 7f0d18200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d100790a0 0x7f0d1019ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:24:30.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:30.276+0000 7f0d18200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0d10079a60 0x7f0d101a7020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:24:30.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:30.276+0000 7f0d18200700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0d10115510 con 0x7f0d100790a0 2024-05-03T18:24:30.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:30.276+0000 7f0d18200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0d10115390 con 0x7f0d10077e50 2024-05-03T18:24:30.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:30.276+0000 7f0d18200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f0d10115690 con 0x7f0d10079a60 2024-05-03T18:24:30.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:30.277+0000 7f0d14e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d100790a0 0x7f0d1019ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:24:30.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:30.277+0000 7f0d14e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d100790a0 0x7f0d1019ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52352/0 (socket says 172.21.15.33:52352) 2024-05-03T18:24:30.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:30.277+0000 7f0d14e00700 1 -- 172.21.15.33:0/4245135404 learned_addr learned my addr 172.21.15.33:0/4245135404 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:24:30.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:30.277+0000 7f0d16200700 1 --2- 172.21.15.33:0/4245135404 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0d10079a60 0x7f0d101a7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:24:30.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:30.277+0000 7f0d15800700 1 --2- 172.21.15.33:0/4245135404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d10077e50 0x7f0d1019fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:24:30.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:30.277+0000 7f0d14e00700 1 -- 172.21.15.33:0/4245135404 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0d10079a60 msgr2=0x7f0d101a7020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:24:30.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:30.277+0000 7f0d14e00700 1 --2- 172.21.15.33:0/4245135404 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0d10079a60 0x7f0d101a7020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:30.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:30.277+0000 7f0d14e00700 1 -- 172.21.15.33:0/4245135404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d10077e50 msgr2=0x7f0d1019fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:24:30.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:30.277+0000 7f0d14e00700 1 --2- 172.21.15.33:0/4245135404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d10077e50 0x7f0d1019fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:30.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:30.277+0000 7f0d14e00700 1 -- 172.21.15.33:0/4245135404 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d101a76c0 con 0x7f0d100790a0 2024-05-03T18:24:30.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:30.277+0000 7f0d16200700 1 --2- 172.21.15.33:0/4245135404 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0d10079a60 0x7f0d101a7020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:24:30.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:30.277+0000 7f0d15800700 1 --2- 172.21.15.33:0/4245135404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d10077e50 0x7f0d1019fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:24:30.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:30.278+0000 7f0d14e00700 1 --2- 172.21.15.33:0/4245135404 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d100790a0 0x7f0d1019ffa0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc0057c0 tx=0x7f0cfc005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:24:30.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:30.278+0000 7f0d0e000700 1 -- 172.21.15.33:0/4245135404 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc027070 con 0x7f0d100790a0 2024-05-03T18:24:30.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:30.278+0000 7f0d0e000700 1 -- 172.21.15.33:0/4245135404 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0cfc0041c0 con 0x7f0d100790a0 2024-05-03T18:24:30.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:30.278+0000 7f0d18200700 1 -- 172.21.15.33:0/4245135404 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d101a7950 con 0x7f0d100790a0 2024-05-03T18:24:30.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:30.278+0000 7f0d0e000700 1 -- 172.21.15.33:0/4245135404 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc00f460 con 0x7f0d100790a0 2024-05-03T18:24:30.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:30.278+0000 7f0d18200700 1 -- 172.21.15.33:0/4245135404 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d101a7da0 con 0x7f0d100790a0 2024-05-03T18:24:30.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:30.280+0000 7f0d18200700 1 -- 172.21.15.33:0/4245135404 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d1002e510 con 0x7f0d100790a0 2024-05-03T18:24:30.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:30.282+0000 7f0d0e000700 1 -- 172.21.15.33:0/4245135404 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f0cfc02a070 con 0x7f0d100790a0 2024-05-03T18:24:30.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:30.284+0000 7f0d0e000700 1 --2- 172.21.15.33:0/4245135404 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f0cf80b29b0 0x7f0cf80b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:24:30.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:30.284+0000 7f0d0e000700 1 -- 172.21.15.33:0/4245135404 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f0cfc0e4100 con 0x7f0d100790a0 2024-05-03T18:24:30.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:30.284+0000 7f0d15800700 1 --2- 172.21.15.33:0/4245135404 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f0cf80b29b0 0x7f0cf80b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:24:30.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:30.285+0000 7f0d15800700 1 --2- 172.21.15.33:0/4245135404 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f0cf80b29b0 0x7f0cf80b4e70 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0d04005950 tx=0x7f0d04005f30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:24:30.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:30.287+0000 7f0d0e000700 1 -- 172.21.15.33:0/4245135404 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f0cfc031020 con 0x7f0d100790a0 2024-05-03T18:24:30.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:24:30 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:24:30.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:24:30 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:24:30.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:24:30 smithi150 ceph-mon[32492]: pgmap v444: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s; 0 B/s, 0 objects/s recovering 2024-05-03T18:24:30.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:24:30 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:24:30.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:24:30 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:24:30.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:24:30 smithi033 ceph-mon[28330]: pgmap v444: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s; 0 B/s, 0 objects/s recovering 2024-05-03T18:24:30.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:24:30 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:24:30.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:24:30 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:24:30.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:24:30 smithi099 ceph-mon[31658]: pgmap v444: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s; 0 B/s, 0 objects/s recovering 2024-05-03T18:24:30.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:30.628+0000 7f0d18200700 1 -- 172.21.15.33:0/4245135404 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f0d100640b0 con 0x7f0d100790a0 2024-05-03T18:24:30.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:30.632+0000 7f0d0e000700 1 -- 172.21.15.33:0/4245135404 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v34) ==== 170+0+59 (secure 0 0 0) 0x7f0cfc013070 con 0x7f0d100790a0 2024-05-03T18:24:30.632 INFO:teuthology.orchestra.run.smithi033.stdout:[client.0] 2024-05-03T18:24:30.632 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQDeKzVmwHiIJRAAPT9TV0GLPrQHmc3tQ6J5Fw== 2024-05-03T18:24:30.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:30.634+0000 7f0d18200700 1 -- 172.21.15.33:0/4245135404 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f0cf80b29b0 msgr2=0x7f0cf80b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:24:30.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:30.634+0000 7f0d18200700 1 --2- 172.21.15.33:0/4245135404 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f0cf80b29b0 0x7f0cf80b4e70 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0d04005950 tx=0x7f0d04005f30 comp rx=0 tx=0).stop 2024-05-03T18:24:30.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:30.635+0000 7f0d18200700 1 -- 172.21.15.33:0/4245135404 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d100790a0 msgr2=0x7f0d1019ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:24:30.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:30.635+0000 7f0d18200700 1 --2- 172.21.15.33:0/4245135404 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d100790a0 0x7f0d1019ffa0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc0057c0 tx=0x7f0cfc005f10 comp rx=0 tx=0).stop 2024-05-03T18:24:30.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:30.635+0000 7f0d18200700 1 -- 172.21.15.33:0/4245135404 shutdown_connections 2024-05-03T18:24:30.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:30.635+0000 7f0d18200700 1 --2- 172.21.15.33:0/4245135404 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0d10079a60 0x7f0d101a7020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:30.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:30.635+0000 7f0d18200700 1 --2- 172.21.15.33:0/4245135404 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f0cf80b29b0 0x7f0cf80b4e70 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:30.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:30.635+0000 7f0d18200700 1 --2- 172.21.15.33:0/4245135404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d10077e50 0x7f0d1019fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:30.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:30.635+0000 7f0d18200700 1 --2- 172.21.15.33:0/4245135404 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d100790a0 0x7f0d1019ffa0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:30.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:30.635+0000 7f0d18200700 1 -- 172.21.15.33:0/4245135404 >> 172.21.15.33:0/4245135404 conn(0x7f0d10100cf0 msgr2=0x7f0d101027f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:24:30.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:30.635+0000 7f0d18200700 1 -- 172.21.15.33:0/4245135404 shutdown_connections 2024-05-03T18:24:30.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:30.635+0000 7f0d18200700 1 -- 172.21.15.33:0/4245135404 wait complete. 2024-05-03T18:24:31.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:24:31 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:24:31.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:24:31 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:24:31.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:24:31 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:24:31.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:24:31 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/4245135404' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-03T18:24:31.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:24:31 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/4245135404' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-03T18:24:31.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:24:31 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:24:31.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:24:31 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:24:31.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:24:31 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:24:31.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:24:31 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/4245135404' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-03T18:24:31.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:24:31 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/4245135404' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-03T18:24:31.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:24:31 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:24:31.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:24:31 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:24:31.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:24:31 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:24:31.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:24:31 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/4245135404' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-03T18:24:31.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:24:31 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/4245135404' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-03T18:24:31.591 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-03T18:24:31.592 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-05-03T18:24:31.592 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-03T18:24:31.735 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-03T18:24:31.735 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-05-03T18:24:31.735 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph mgr dump --format=json 2024-05-03T18:24:32.252 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:24:32.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:24:32 smithi150 ceph-mon[32492]: pgmap v445: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-05-03T18:24:32.401 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:24:32 smithi033 ceph-mon[28330]: pgmap v445: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-05-03T18:24:32.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:24:32 smithi099 ceph-mon[31658]: pgmap v445: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-05-03T18:24:34.591 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:24:34 smithi033 ceph-mon[28330]: pgmap v446: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s; 0 B/s, 0 objects/s recovering 2024-05-03T18:24:34.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:34.640+0000 7fe25d800700 1 -- 172.21.15.33:0/2924847710 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe258106080 msgr2=0x7fe258112590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:24:34.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:34.640+0000 7fe25d800700 1 --2- 172.21.15.33:0/2924847710 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe258106080 0x7fe258112590 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fe25000b0a0 tx=0x7fe25000b3b0 comp rx=0 tx=0).stop 2024-05-03T18:24:34.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:34.640+0000 7fe25d800700 1 -- 172.21.15.33:0/2924847710 shutdown_connections 2024-05-03T18:24:34.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:34.640+0000 7fe25d800700 1 --2- 172.21.15.33:0/2924847710 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe258106080 0x7fe258112590 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:34.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:34.640+0000 7fe25d800700 1 --2- 172.21.15.33:0/2924847710 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe25810b6c0 0x7fe25810baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:34.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:34.640+0000 7fe25d800700 1 --2- 172.21.15.33:0/2924847710 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe2581056c0 0x7fe258105b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:34.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:34.640+0000 7fe25d800700 1 -- 172.21.15.33:0/2924847710 >> 172.21.15.33:0/2924847710 conn(0x7fe2581011b0 msgr2=0x7fe2581035d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:24:34.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:34.640+0000 7fe25d800700 1 -- 172.21.15.33:0/2924847710 shutdown_connections 2024-05-03T18:24:34.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:34.640+0000 7fe25d800700 1 -- 172.21.15.33:0/2924847710 wait complete. 2024-05-03T18:24:34.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:34.642+0000 7fe25d800700 1 Processor -- start 2024-05-03T18:24:34.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:34.642+0000 7fe25d800700 1 -- start start 2024-05-03T18:24:34.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:34.642+0000 7fe25d800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2581056c0 0x7fe2581a1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:24:34.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:34.642+0000 7fe25d800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe258106080 0x7fe2581a1e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:24:34.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:34.642+0000 7fe25d800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe25810b6c0 0x7fe25819bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:24:34.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:34.642+0000 7fe25d800700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe2581157c0 con 0x7fe25810b6c0 2024-05-03T18:24:34.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:34.642+0000 7fe25d800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe258115640 con 0x7fe2581056c0 2024-05-03T18:24:34.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:34.642+0000 7fe25d800700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe258115940 con 0x7fe258106080 2024-05-03T18:24:34.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:34.643+0000 7fe257400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe25810b6c0 0x7fe25819bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:24:34.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:34.643+0000 7fe257400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe25810b6c0 0x7fe25819bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52370/0 (socket says 172.21.15.33:52370) 2024-05-03T18:24:34.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:34.643+0000 7fe257400700 1 -- 172.21.15.33:0/1663751075 learned_addr learned my addr 172.21.15.33:0/1663751075 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:24:34.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:34.643+0000 7fe256000700 1 --2- 172.21.15.33:0/1663751075 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe258106080 0x7fe2581a1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:24:34.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:34.643+0000 7fe256a00700 1 --2- 172.21.15.33:0/1663751075 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2581056c0 0x7fe2581a1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:24:34.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:34.643+0000 7fe257400700 1 -- 172.21.15.33:0/1663751075 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe258106080 msgr2=0x7fe2581a1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:24:34.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:34.643+0000 7fe257400700 1 --2- 172.21.15.33:0/1663751075 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe258106080 0x7fe2581a1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:34.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:34.643+0000 7fe257400700 1 -- 172.21.15.33:0/1663751075 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2581056c0 msgr2=0x7fe2581a1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:24:34.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:34.643+0000 7fe257400700 1 --2- 172.21.15.33:0/1663751075 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2581056c0 0x7fe2581a1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:34.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:34.643+0000 7fe257400700 1 -- 172.21.15.33:0/1663751075 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe25819c220 con 0x7fe25810b6c0 2024-05-03T18:24:34.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:34.644+0000 7fe257400700 1 --2- 172.21.15.33:0/1663751075 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe25810b6c0 0x7fe25819bac0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe25000b070 tx=0x7fe250004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:24:34.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:34.644+0000 7fe24f400700 1 -- 172.21.15.33:0/1663751075 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe250021070 con 0x7fe25810b6c0 2024-05-03T18:24:34.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:34.644+0000 7fe24f400700 1 -- 172.21.15.33:0/1663751075 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe250007b80 con 0x7fe25810b6c0 2024-05-03T18:24:34.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:34.644+0000 7fe25d800700 1 -- 172.21.15.33:0/1663751075 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe25819c4b0 con 0x7fe25810b6c0 2024-05-03T18:24:34.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:34.644+0000 7fe24f400700 1 -- 172.21.15.33:0/1663751075 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe250004650 con 0x7fe25810b6c0 2024-05-03T18:24:34.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:34.644+0000 7fe25d800700 1 -- 172.21.15.33:0/1663751075 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2581a87d0 con 0x7fe25810b6c0 2024-05-03T18:24:34.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:34.646+0000 7fe25d800700 1 -- 172.21.15.33:0/1663751075 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe25806a190 con 0x7fe25810b6c0 2024-05-03T18:24:34.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:34.648+0000 7fe24f400700 1 -- 172.21.15.33:0/1663751075 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fe250003a00 con 0x7fe25810b6c0 2024-05-03T18:24:34.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:34.649+0000 7fe24f400700 1 --2- 172.21.15.33:0/1663751075 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fe23c0b2910 0x7fe23c0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:24:34.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:34.650+0000 7fe24f400700 1 -- 172.21.15.33:0/1663751075 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7fe25000f020 con 0x7fe25810b6c0 2024-05-03T18:24:34.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:34.650+0000 7fe256a00700 1 --2- 172.21.15.33:0/1663751075 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fe23c0b2910 0x7fe23c0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:24:34.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:34.651+0000 7fe256a00700 1 --2- 172.21.15.33:0/1663751075 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fe23c0b2910 0x7fe23c0b4dd0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe25819cc00 tx=0x7fe24800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:24:34.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:34.654+0000 7fe24f400700 1 -- 172.21.15.33:0/1663751075 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fe25002a020 con 0x7fe25810b6c0 2024-05-03T18:24:34.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:24:34 smithi099 ceph-mon[31658]: pgmap v446: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s; 0 B/s, 0 objects/s recovering 2024-05-03T18:24:34.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:24:34 smithi150 ceph-mon[32492]: pgmap v446: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s; 0 B/s, 0 objects/s recovering 2024-05-03T18:24:34.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:34.957+0000 7fe25d800700 1 -- 172.21.15.33:0/1663751075 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fe25802e510 con 0x7fe25810b6c0 2024-05-03T18:24:34.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:34.968+0000 7fe24f400700 1 -- 172.21.15.33:0/1663751075 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+288644 (secure 0 0 0) 0x7fe2500047f0 con 0x7fe25810b6c0 2024-05-03T18:24:34.972 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:24:34.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:34.975+0000 7fe25d800700 1 -- 172.21.15.33:0/1663751075 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fe23c0b2910 msgr2=0x7fe23c0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:24:34.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:34.975+0000 7fe25d800700 1 --2- 172.21.15.33:0/1663751075 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fe23c0b2910 0x7fe23c0b4dd0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe25819cc00 tx=0x7fe24800b410 comp rx=0 tx=0).stop 2024-05-03T18:24:34.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:34.975+0000 7fe25d800700 1 -- 172.21.15.33:0/1663751075 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe25810b6c0 msgr2=0x7fe25819bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:24:34.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:34.975+0000 7fe25d800700 1 --2- 172.21.15.33:0/1663751075 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe25810b6c0 0x7fe25819bac0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe25000b070 tx=0x7fe250004c80 comp rx=0 tx=0).stop 2024-05-03T18:24:34.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:34.975+0000 7fe25d800700 1 -- 172.21.15.33:0/1663751075 shutdown_connections 2024-05-03T18:24:34.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:34.975+0000 7fe25d800700 1 --2- 172.21.15.33:0/1663751075 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe25810b6c0 0x7fe25819bac0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:34.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:34.975+0000 7fe25d800700 1 --2- 172.21.15.33:0/1663751075 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fe23c0b2910 0x7fe23c0b4dd0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:34.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:34.975+0000 7fe25d800700 1 --2- 172.21.15.33:0/1663751075 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2581056c0 0x7fe2581a1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:34.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:34.975+0000 7fe25d800700 1 --2- 172.21.15.33:0/1663751075 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe258106080 0x7fe2581a1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:34.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:34.976+0000 7fe25d800700 1 -- 172.21.15.33:0/1663751075 >> 172.21.15.33:0/1663751075 conn(0x7fe2581011b0 msgr2=0x7fe2581018d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:24:34.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:34.976+0000 7fe25d800700 1 -- 172.21.15.33:0/1663751075 shutdown_connections 2024-05-03T18:24:34.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:34.976+0000 7fe25d800700 1 -- 172.21.15.33:0/1663751075 wait complete. 2024-05-03T18:24:35.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:24:35 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/1663751075' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-03T18:24:35.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:24:35 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/1663751075' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-03T18:24:35.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:24:35 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/1663751075' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-03T18:24:35.980 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":3642895501},{"type":"v1","addr":"172.21.15.33:6801","nonce":3642895501}]},"active_addr":"172.21.15.33:6801/3642895501","active_change":"2024-05-03T18:12:19.127978+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24112,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":34112,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.33:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.33:0","nonce":1300142939}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.33:0","nonce":791712778}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.33:0","nonce":757614565}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.33:0","nonce":806149166}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.33:0","nonce":4003417322}]}]} 2024-05-03T18:24:35.986 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-05-03T18:24:35.986 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-05-03T18:24:35.987 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-03T18:24:36.491 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:24:36.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:24:36 smithi033 ceph-mon[28330]: pgmap v447: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-05-03T18:24:36.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:24:36 smithi099 ceph-mon[31658]: pgmap v447: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-05-03T18:24:36.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:24:36 smithi150 ceph-mon[32492]: pgmap v447: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-05-03T18:24:38.620 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:24:38 smithi033 ceph-mon[28330]: pgmap v448: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-05-03T18:24:38.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:38.782+0000 7f067f400700 1 -- 172.21.15.33:0/2798995195 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f067810b6c0 msgr2=0x7f067810baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:24:38.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:38.782+0000 7f067f400700 1 --2- 172.21.15.33:0/2798995195 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f067810b6c0 0x7f067810baa0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f066c009990 tx=0x7f066c009ca0 comp rx=0 tx=0).stop 2024-05-03T18:24:38.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:38.782+0000 7f067f400700 1 -- 172.21.15.33:0/2798995195 shutdown_connections 2024-05-03T18:24:38.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:38.782+0000 7f067f400700 1 --2- 172.21.15.33:0/2798995195 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0678106080 0x7f0678112590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:38.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:38.782+0000 7f067f400700 1 --2- 172.21.15.33:0/2798995195 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f067810b6c0 0x7f067810baa0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:38.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:38.782+0000 7f067f400700 1 --2- 172.21.15.33:0/2798995195 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f06781056c0 0x7f0678105b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:38.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:38.782+0000 7f067f400700 1 -- 172.21.15.33:0/2798995195 >> 172.21.15.33:0/2798995195 conn(0x7f06781011b0 msgr2=0x7f06781035d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:24:38.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:38.782+0000 7f067f400700 1 -- 172.21.15.33:0/2798995195 shutdown_connections 2024-05-03T18:24:38.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:38.782+0000 7f067f400700 1 -- 172.21.15.33:0/2798995195 wait complete. 2024-05-03T18:24:38.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:38.783+0000 7f067f400700 1 Processor -- start 2024-05-03T18:24:38.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:38.783+0000 7f067f400700 1 -- start start 2024-05-03T18:24:38.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:38.784+0000 7f067f400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f06781056c0 0x7f06781a18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:24:38.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:38.784+0000 7f067f400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0678106080 0x7f06781a1e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:24:38.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:38.784+0000 7f067f400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f067810b6c0 0x7f067819ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:24:38.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:38.784+0000 7f067f400700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0678115750 con 0x7f0678106080 2024-05-03T18:24:38.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:38.784+0000 7f067f400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f06781155d0 con 0x7f067810b6c0 2024-05-03T18:24:38.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:38.784+0000 7f067f400700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f06781158d0 con 0x7f06781056c0 2024-05-03T18:24:38.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:38.784+0000 7f0677e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0678106080 0x7f06781a1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:24:38.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:38.784+0000 7f0677e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0678106080 0x7f06781a1e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42298/0 (socket says 172.21.15.33:42298) 2024-05-03T18:24:38.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:38.784+0000 7f0677e00700 1 -- 172.21.15.33:0/253195992 learned_addr learned my addr 172.21.15.33:0/253195992 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:24:38.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:38.784+0000 7f067d400700 1 --2- 172.21.15.33:0/253195992 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f067810b6c0 0x7f067819ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:24:38.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:38.784+0000 7f067ca00700 1 --2- 172.21.15.33:0/253195992 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f06781056c0 0x7f06781a18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:24:38.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:38.784+0000 7f0677e00700 1 -- 172.21.15.33:0/253195992 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f06781056c0 msgr2=0x7f06781a18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:24:38.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:38.784+0000 7f0677e00700 1 --2- 172.21.15.33:0/253195992 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f06781056c0 0x7f06781a18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:38.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:38.784+0000 7f0677e00700 1 -- 172.21.15.33:0/253195992 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f067810b6c0 msgr2=0x7f067819ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:24:38.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:38.784+0000 7f0677e00700 1 --2- 172.21.15.33:0/253195992 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f067810b6c0 0x7f067819ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:38.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:38.784+0000 7f0677e00700 1 -- 172.21.15.33:0/253195992 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f067819c1f0 con 0x7f0678106080 2024-05-03T18:24:38.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:38.785+0000 7f0677e00700 1 --2- 172.21.15.33:0/253195992 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0678106080 0x7f06781a1e10 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f066400c920 tx=0x7f066400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:24:38.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:38.785+0000 7f0675600700 1 -- 172.21.15.33:0/253195992 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0664007b00 con 0x7f0678106080 2024-05-03T18:24:38.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:38.785+0000 7f0675600700 1 -- 172.21.15.33:0/253195992 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0664004cc0 con 0x7f0678106080 2024-05-03T18:24:38.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:38.785+0000 7f0675600700 1 -- 172.21.15.33:0/253195992 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06640054f0 con 0x7f0678106080 2024-05-03T18:24:38.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:38.785+0000 7f067f400700 1 -- 172.21.15.33:0/253195992 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f067819c4e0 con 0x7f0678106080 2024-05-03T18:24:38.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:38.785+0000 7f067f400700 1 -- 172.21.15.33:0/253195992 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06781a87a0 con 0x7f0678106080 2024-05-03T18:24:38.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:38.787+0000 7f067f400700 1 -- 172.21.15.33:0/253195992 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f067806a190 con 0x7f0678106080 2024-05-03T18:24:38.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:38.789+0000 7f0675600700 1 -- 172.21.15.33:0/253195992 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f0664007ca0 con 0x7f0678106080 2024-05-03T18:24:38.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:38.790+0000 7f0675600700 1 --2- 172.21.15.33:0/253195992 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f06600b2990 0x7f06600b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:24:38.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:38.790+0000 7f0675600700 1 -- 172.21.15.33:0/253195992 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f06640e2350 con 0x7f0678106080 2024-05-03T18:24:38.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:38.791+0000 7f067ca00700 1 --2- 172.21.15.33:0/253195992 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f06600b2990 0x7f06600b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:24:38.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:38.791+0000 7f067ca00700 1 --2- 172.21.15.33:0/253195992 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f06600b2990 0x7f06600b4e50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f066c005850 tx=0x7f066c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:24:38.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:24:38 smithi099 ceph-mon[31658]: pgmap v448: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-05-03T18:24:38.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:38.795+0000 7f0675600700 1 -- 172.21.15.33:0/253195992 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f0664014030 con 0x7f0678106080 2024-05-03T18:24:38.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:24:38 smithi150 ceph-mon[32492]: pgmap v448: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-05-03T18:24:39.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:39.024+0000 7f067f400700 1 -- 172.21.15.33:0/253195992 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f067802e510 con 0x7f0678106080 2024-05-03T18:24:39.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:39.027+0000 7f0675600700 1 -- 172.21.15.33:0/253195992 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24650 (secure 0 0 0) 0x7f06640a9b30 con 0x7f0678106080 2024-05-03T18:24:39.028 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:24:39.029 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":89,"fsid":"5703bba4-0978-11ef-bc97-c7b262605968","created":"2024-05-03T18:11:00.899879+0000","modified":"2024-05-03T18:24:21.532082+0000","last_up_change":"2024-05-03T18:21:00.390173+0000","last_in_change":"2024-05-03T18:20:34.103727+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-03T18:15:37.297738+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-03T18:22:19.875368+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-03T18:22:24.432692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-03T18:22:38.722443+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"fdcc4e66-4ba0-4378-a9a4-7d907cd00b4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":637686669},{"type":"v1","addr":"172.21.15.33:6803","nonce":637686669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":637686669},{"type":"v1","addr":"172.21.15.33:6805","nonce":637686669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":637686669},{"type":"v1","addr":"172.21.15.33:6809","nonce":637686669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":637686669},{"type":"v1","addr":"172.21.15.33:6807","nonce":637686669}]},"public_addr":"172.21.15.33:6803/637686669","cluster_addr":"172.21.15.33:6805/637686669","heartbeat_back_addr":"172.21.15.33:6809/637686669","heartbeat_front_addr":"172.21.15.33:6807/637686669","state":["exists","up"]},{"osd":1,"uuid":"cf78cad3-fc24-43f4-8784-21cabea7277c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":1536122841},{"type":"v1","addr":"172.21.15.99:6801","nonce":1536122841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":1536122841},{"type":"v1","addr":"172.21.15.99:6803","nonce":1536122841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":1536122841},{"type":"v1","addr":"172.21.15.99:6807","nonce":1536122841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":1536122841},{"type":"v1","addr":"172.21.15.99:6805","nonce":1536122841}]},"public_addr":"172.21.15.99:6801/1536122841","cluster_addr":"172.21.15.99:6803/1536122841","heartbeat_back_addr":"172.21.15.99:6807/1536122841","heartbeat_front_addr":"172.21.15.99:6805/1536122841","state":["exists","up"]},{"osd":2,"uuid":"59f066d7-626b-410a-8d67-daca6fbbd3ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":1995659622},{"type":"v1","addr":"172.21.15.150:6801","nonce":1995659622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":1995659622},{"type":"v1","addr":"172.21.15.150:6803","nonce":1995659622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":1995659622},{"type":"v1","addr":"172.21.15.150:6807","nonce":1995659622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":1995659622},{"type":"v1","addr":"172.21.15.150:6805","nonce":1995659622}]},"public_addr":"172.21.15.150:6801/1995659622","cluster_addr":"172.21.15.150:6803/1995659622","heartbeat_back_addr":"172.21.15.150:6807/1995659622","heartbeat_front_addr":"172.21.15.150:6805/1995659622","state":["exists","up"]},{"osd":3,"uuid":"d88ed022-4dcb-45fc-9fc6-41ef7c4bf202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":3814942724},{"type":"v1","addr":"172.21.15.33:6811","nonce":3814942724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":3814942724},{"type":"v1","addr":"172.21.15.33:6813","nonce":3814942724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":3814942724},{"type":"v1","addr":"172.21.15.33:6817","nonce":3814942724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":3814942724},{"type":"v1","addr":"172.21.15.33:6815","nonce":3814942724}]},"public_addr":"172.21.15.33:6811/3814942724","cluster_addr":"172.21.15.33:6813/3814942724","heartbeat_back_addr":"172.21.15.33:6817/3814942724","heartbeat_front_addr":"172.21.15.33:6815/3814942724","state":["exists","up"]},{"osd":4,"uuid":"1235e1a6-ff6e-4668-ac1a-073c2a3b7c98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":29081029},{"type":"v1","addr":"172.21.15.99:6809","nonce":29081029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":29081029},{"type":"v1","addr":"172.21.15.99:6811","nonce":29081029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":29081029},{"type":"v1","addr":"172.21.15.99:6815","nonce":29081029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":29081029},{"type":"v1","addr":"172.21.15.99:6813","nonce":29081029}]},"public_addr":"172.21.15.99:6809/29081029","cluster_addr":"172.21.15.99:6811/29081029","heartbeat_back_addr":"172.21.15.99:6815/29081029","heartbeat_front_addr":"172.21.15.99:6813/29081029","state":["exists","up"]},{"osd":5,"uuid":"4b84bff8-d6d7-4258-a0ee-53bc19802585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":3172189533},{"type":"v1","addr":"172.21.15.150:6809","nonce":3172189533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3172189533},{"type":"v1","addr":"172.21.15.150:6811","nonce":3172189533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3172189533},{"type":"v1","addr":"172.21.15.150:6815","nonce":3172189533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3172189533},{"type":"v1","addr":"172.21.15.150:6813","nonce":3172189533}]},"public_addr":"172.21.15.150:6809/3172189533","cluster_addr":"172.21.15.150:6811/3172189533","heartbeat_back_addr":"172.21.15.150:6815/3172189533","heartbeat_front_addr":"172.21.15.150:6813/3172189533","state":["exists","up"]},{"osd":6,"uuid":"51f5fbdd-3794-4c48-9f6c-478f0fa35ac5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":4234541418},{"type":"v1","addr":"172.21.15.33:6819","nonce":4234541418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":4234541418},{"type":"v1","addr":"172.21.15.33:6821","nonce":4234541418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":4234541418},{"type":"v1","addr":"172.21.15.33:6825","nonce":4234541418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":4234541418},{"type":"v1","addr":"172.21.15.33:6823","nonce":4234541418}]},"public_addr":"172.21.15.33:6819/4234541418","cluster_addr":"172.21.15.33:6821/4234541418","heartbeat_back_addr":"172.21.15.33:6825/4234541418","heartbeat_front_addr":"172.21.15.33:6823/4234541418","state":["exists","up"]},{"osd":7,"uuid":"982cc5b3-3163-4505-9bde-a27a8c289457","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":908673090},{"type":"v1","addr":"172.21.15.99:6817","nonce":908673090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":908673090},{"type":"v1","addr":"172.21.15.99:6819","nonce":908673090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":908673090},{"type":"v1","addr":"172.21.15.99:6823","nonce":908673090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":908673090},{"type":"v1","addr":"172.21.15.99:6821","nonce":908673090}]},"public_addr":"172.21.15.99:6817/908673090","cluster_addr":"172.21.15.99:6819/908673090","heartbeat_back_addr":"172.21.15.99:6823/908673090","heartbeat_front_addr":"172.21.15.99:6821/908673090","state":["exists","up"]},{"osd":8,"uuid":"84c9d037-eb84-4eb4-8aa2-e784b703d36c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":1308083602},{"type":"v1","addr":"172.21.15.150:6817","nonce":1308083602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":1308083602},{"type":"v1","addr":"172.21.15.150:6819","nonce":1308083602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":1308083602},{"type":"v1","addr":"172.21.15.150:6823","nonce":1308083602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":1308083602},{"type":"v1","addr":"172.21.15.150:6821","nonce":1308083602}]},"public_addr":"172.21.15.150:6817/1308083602","cluster_addr":"172.21.15.150:6819/1308083602","heartbeat_back_addr":"172.21.15.150:6823/1308083602","heartbeat_front_addr":"172.21.15.150:6821/1308083602","state":["exists","up"]},{"osd":9,"uuid":"a02e965f-0d8f-45e2-9490-337f7f6e3881","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":4133096905},{"type":"v1","addr":"172.21.15.33:6827","nonce":4133096905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":4133096905},{"type":"v1","addr":"172.21.15.33:6829","nonce":4133096905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4133096905},{"type":"v1","addr":"172.21.15.33:6833","nonce":4133096905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":4133096905},{"type":"v1","addr":"172.21.15.33:6831","nonce":4133096905}]},"public_addr":"172.21.15.33:6827/4133096905","cluster_addr":"172.21.15.33:6829/4133096905","heartbeat_back_addr":"172.21.15.33:6833/4133096905","heartbeat_front_addr":"172.21.15.33:6831/4133096905","state":["exists","up"]},{"osd":10,"uuid":"8b29bb96-76e8-45a0-ab19-8d082d755fe9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":935746136},{"type":"v1","addr":"172.21.15.99:6825","nonce":935746136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":935746136},{"type":"v1","addr":"172.21.15.99:6827","nonce":935746136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":935746136},{"type":"v1","addr":"172.21.15.99:6831","nonce":935746136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":935746136},{"type":"v1","addr":"172.21.15.99:6829","nonce":935746136}]},"public_addr":"172.21.15.99:6825/935746136","cluster_addr":"172.21.15.99:6827/935746136","heartbeat_back_addr":"172.21.15.99:6831/935746136","heartbeat_front_addr":"172.21.15.99:6829/935746136","state":["exists","up"]},{"osd":11,"uuid":"07a6a825-c902-4bfd-97cc-cc9074532af5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2069349808},{"type":"v1","addr":"172.21.15.150:6825","nonce":2069349808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2069349808},{"type":"v1","addr":"172.21.15.150:6827","nonce":2069349808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2069349808},{"type":"v1","addr":"172.21.15.150:6831","nonce":2069349808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2069349808},{"type":"v1","addr":"172.21.15.150:6829","nonce":2069349808}]},"public_addr":"172.21.15.150:6825/2069349808","cluster_addr":"172.21.15.150:6827/2069349808","heartbeat_back_addr":"172.21.15.150:6831/2069349808","heartbeat_front_addr":"172.21.15.150:6829/2069349808","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:14:27.454536+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:15:00.538570+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:15:34.125289+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:16:09.853870+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:16:44.928107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:17:19.644037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:17:56.108443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:18:31.731499+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:19:08.062358+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:19:45.204430+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:20:21.013710+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:20:57.653698+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11},{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/3991947997":"2024-05-04T18:12:19.127652+0000","172.21.15.33:0/965859483":"2024-05-04T18:12:19.127652+0000","172.21.15.33:0/2014902144":"2024-05-04T18:11:44.239835+0000","172.21.15.33:0/1314704208":"2024-05-04T18:11:44.239835+0000","172.21.15.33:0/1614640996":"2024-05-04T18:11:44.239835+0000","172.21.15.33:6800/4007322071":"2024-05-04T18:11:44.239835+0000","172.21.15.33:6801/2830135294":"2024-05-04T18:12:19.127652+0000","172.21.15.33:6800/2830135294":"2024-05-04T18:12:19.127652+0000","172.21.15.33:0/3767665033":"2024-05-04T18:12:19.127652+0000","172.21.15.33:6801/4007322071":"2024-05-04T18:11:44.239835+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-03T18:24:39.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:39.030+0000 7f067f400700 1 -- 172.21.15.33:0/253195992 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f06600b2990 msgr2=0x7f06600b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:24:39.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:39.030+0000 7f067f400700 1 --2- 172.21.15.33:0/253195992 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f06600b2990 0x7f06600b4e50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f066c005850 tx=0x7f066c01a040 comp rx=0 tx=0).stop 2024-05-03T18:24:39.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:39.030+0000 7f067f400700 1 -- 172.21.15.33:0/253195992 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0678106080 msgr2=0x7f06781a1e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:24:39.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:39.030+0000 7f067f400700 1 --2- 172.21.15.33:0/253195992 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0678106080 0x7f06781a1e10 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f066400c920 tx=0x7f066400cce0 comp rx=0 tx=0).stop 2024-05-03T18:24:39.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:39.030+0000 7f067f400700 1 -- 172.21.15.33:0/253195992 shutdown_connections 2024-05-03T18:24:39.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:39.030+0000 7f067f400700 1 --2- 172.21.15.33:0/253195992 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f06600b2990 0x7f06600b4e50 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:39.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:39.030+0000 7f067f400700 1 --2- 172.21.15.33:0/253195992 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f067810b6c0 0x7f067819ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:39.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:39.030+0000 7f067f400700 1 --2- 172.21.15.33:0/253195992 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f06781056c0 0x7f06781a18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:39.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:39.030+0000 7f067f400700 1 --2- 172.21.15.33:0/253195992 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0678106080 0x7f06781a1e10 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:39.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:39.030+0000 7f067f400700 1 -- 172.21.15.33:0/253195992 >> 172.21.15.33:0/253195992 conn(0x7f06781011b0 msgr2=0x7f0678101940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:24:39.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:39.030+0000 7f067f400700 1 -- 172.21.15.33:0/253195992 shutdown_connections 2024-05-03T18:24:39.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:39.030+0000 7f067f400700 1 -- 172.21.15.33:0/253195992 wait complete. 2024-05-03T18:24:39.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:24:39 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/253195992' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T18:24:39.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:24:39 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/253195992' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T18:24:39.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:24:39 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/253195992' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T18:24:39.943 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-05-03T18:24:39.944 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-03T18:24:40.398 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:24:40.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:24:40 smithi033 ceph-mon[28330]: pgmap v449: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:24:40.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:24:40 smithi099 ceph-mon[31658]: pgmap v449: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:24:40.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:24:40 smithi150 ceph-mon[32492]: pgmap v449: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:24:42.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:24:42 smithi033 ceph-mon[28330]: pgmap v450: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:24:42.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:42.738+0000 7f988fe00700 1 -- 172.21.15.33:0/3931761839 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9890103620 msgr2=0x7f9890103a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:24:42.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:42.738+0000 7f988fe00700 1 --2- 172.21.15.33:0/3931761839 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9890103620 0x7f9890103a00 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f98800099c0 tx=0x7f9880009cd0 comp rx=0 tx=0).stop 2024-05-03T18:24:42.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:42.738+0000 7f988fe00700 1 -- 172.21.15.33:0/3931761839 shutdown_connections 2024-05-03T18:24:42.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:42.738+0000 7f988fe00700 1 --2- 172.21.15.33:0/3931761839 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9890108700 0x7f98901102d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:42.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:42.738+0000 7f988fe00700 1 --2- 172.21.15.33:0/3931761839 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9890103620 0x7f9890103a00 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:42.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:42.738+0000 7f988fe00700 1 --2- 172.21.15.33:0/3931761839 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9890103fd0 0x7f9890108020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:42.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:42.738+0000 7f988fe00700 1 -- 172.21.15.33:0/3931761839 >> 172.21.15.33:0/3931761839 conn(0x7f98900feef0 msgr2=0x7f9890101310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:24:42.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:42.738+0000 7f988fe00700 1 -- 172.21.15.33:0/3931761839 shutdown_connections 2024-05-03T18:24:42.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:42.739+0000 7f988fe00700 1 -- 172.21.15.33:0/3931761839 wait complete. 2024-05-03T18:24:42.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:42.739+0000 7f988fe00700 1 Processor -- start 2024-05-03T18:24:42.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:42.740+0000 7f988fe00700 1 -- start start 2024-05-03T18:24:42.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:42.740+0000 7f988fe00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9890103620 0x7f98900757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:24:42.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:42.740+0000 7f988fe00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9890103fd0 0x7f9890075ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:24:42.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:42.740+0000 7f988fe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9890108700 0x7f9890076220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:24:42.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:42.740+0000 7f988fe00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9890113490 con 0x7f9890103fd0 2024-05-03T18:24:42.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:42.740+0000 7f988fe00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9890113310 con 0x7f9890108700 2024-05-03T18:24:42.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:42.740+0000 7f988fe00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9890113610 con 0x7f9890103620 2024-05-03T18:24:42.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:42.740+0000 7f988e000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9890103fd0 0x7f9890075ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:24:42.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:42.740+0000 7f988e000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9890103fd0 0x7f9890075ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42310/0 (socket says 172.21.15.33:42310) 2024-05-03T18:24:42.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:42.740+0000 7f988e000700 1 -- 172.21.15.33:0/2280726864 learned_addr learned my addr 172.21.15.33:0/2280726864 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:24:42.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:42.741+0000 7f988ea00700 1 --2- 172.21.15.33:0/2280726864 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9890103620 0x7f98900757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:24:42.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:42.741+0000 7f988f400700 1 --2- 172.21.15.33:0/2280726864 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9890108700 0x7f9890076220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:24:42.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:42.741+0000 7f988e000700 1 -- 172.21.15.33:0/2280726864 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9890103620 msgr2=0x7f98900757a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:24:42.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:42.741+0000 7f988e000700 1 --2- 172.21.15.33:0/2280726864 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9890103620 0x7f98900757a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:42.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:42.741+0000 7f988e000700 1 -- 172.21.15.33:0/2280726864 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9890108700 msgr2=0x7f9890076220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:24:42.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:42.741+0000 7f988e000700 1 --2- 172.21.15.33:0/2280726864 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9890108700 0x7f9890076220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:42.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:42.741+0000 7f988e000700 1 -- 172.21.15.33:0/2280726864 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98901a7f10 con 0x7f9890103fd0 2024-05-03T18:24:42.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:42.741+0000 7f988f400700 1 --2- 172.21.15.33:0/2280726864 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9890108700 0x7f9890076220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:24:42.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:42.741+0000 7f988e000700 1 --2- 172.21.15.33:0/2280726864 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9890103fd0 0x7f9890075ce0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f987c00d980 tx=0x7f987c00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:24:42.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:42.742+0000 7f9887400700 1 -- 172.21.15.33:0/2280726864 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f987c017070 con 0x7f9890103fd0 2024-05-03T18:24:42.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:42.742+0000 7f988fe00700 1 -- 172.21.15.33:0/2280726864 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98901a8200 con 0x7f9890103fd0 2024-05-03T18:24:42.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:42.742+0000 7f9887400700 1 -- 172.21.15.33:0/2280726864 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f987c010460 con 0x7f9890103fd0 2024-05-03T18:24:42.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:42.742+0000 7f9887400700 1 -- 172.21.15.33:0/2280726864 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f987c00de60 con 0x7f9890103fd0 2024-05-03T18:24:42.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:42.742+0000 7f988fe00700 1 -- 172.21.15.33:0/2280726864 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98901a8790 con 0x7f9890103fd0 2024-05-03T18:24:42.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:42.743+0000 7f988fe00700 1 -- 172.21.15.33:0/2280726864 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f989006a190 con 0x7f9890103fd0 2024-05-03T18:24:42.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:42.749+0000 7f9887400700 1 -- 172.21.15.33:0/2280726864 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f987c010610 con 0x7f9890103fd0 2024-05-03T18:24:42.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:42.750+0000 7f9887400700 1 --2- 172.21.15.33:0/2280726864 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f98740b2990 0x7f98740b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:24:42.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:42.750+0000 7f9887400700 1 -- 172.21.15.33:0/2280726864 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f987c0e2210 con 0x7f9890103fd0 2024-05-03T18:24:42.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:42.751+0000 7f988ea00700 1 --2- 172.21.15.33:0/2280726864 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f98740b2990 0x7f98740b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:24:42.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:42.751+0000 7f988ea00700 1 --2- 172.21.15.33:0/2280726864 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f98740b2990 0x7f98740b4e50 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9880000c00 tx=0x7f988001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:24:42.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:42.752+0000 7f9887400700 1 -- 172.21.15.33:0/2280726864 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f987c0a9a70 con 0x7f9890103fd0 2024-05-03T18:24:42.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:24:42 smithi099 ceph-mon[31658]: pgmap v450: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:24:42.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:24:42 smithi150 ceph-mon[32492]: pgmap v450: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:24:42.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:42.990+0000 7f988fe00700 1 -- 172.21.15.33:0/2280726864 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f989002e510 con 0x7f9890103fd0 2024-05-03T18:24:42.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:42.995+0000 7f9887400700 1 -- 172.21.15.33:0/2280726864 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24650 (secure 0 0 0) 0x7f987c0a9370 con 0x7f9890103fd0 2024-05-03T18:24:42.996 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:24:42.996 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":89,"fsid":"5703bba4-0978-11ef-bc97-c7b262605968","created":"2024-05-03T18:11:00.899879+0000","modified":"2024-05-03T18:24:21.532082+0000","last_up_change":"2024-05-03T18:21:00.390173+0000","last_in_change":"2024-05-03T18:20:34.103727+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-03T18:15:37.297738+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-03T18:22:19.875368+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-03T18:22:24.432692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-03T18:22:38.722443+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"fdcc4e66-4ba0-4378-a9a4-7d907cd00b4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":637686669},{"type":"v1","addr":"172.21.15.33:6803","nonce":637686669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":637686669},{"type":"v1","addr":"172.21.15.33:6805","nonce":637686669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":637686669},{"type":"v1","addr":"172.21.15.33:6809","nonce":637686669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":637686669},{"type":"v1","addr":"172.21.15.33:6807","nonce":637686669}]},"public_addr":"172.21.15.33:6803/637686669","cluster_addr":"172.21.15.33:6805/637686669","heartbeat_back_addr":"172.21.15.33:6809/637686669","heartbeat_front_addr":"172.21.15.33:6807/637686669","state":["exists","up"]},{"osd":1,"uuid":"cf78cad3-fc24-43f4-8784-21cabea7277c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":1536122841},{"type":"v1","addr":"172.21.15.99:6801","nonce":1536122841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":1536122841},{"type":"v1","addr":"172.21.15.99:6803","nonce":1536122841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":1536122841},{"type":"v1","addr":"172.21.15.99:6807","nonce":1536122841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":1536122841},{"type":"v1","addr":"172.21.15.99:6805","nonce":1536122841}]},"public_addr":"172.21.15.99:6801/1536122841","cluster_addr":"172.21.15.99:6803/1536122841","heartbeat_back_addr":"172.21.15.99:6807/1536122841","heartbeat_front_addr":"172.21.15.99:6805/1536122841","state":["exists","up"]},{"osd":2,"uuid":"59f066d7-626b-410a-8d67-daca6fbbd3ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":1995659622},{"type":"v1","addr":"172.21.15.150:6801","nonce":1995659622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":1995659622},{"type":"v1","addr":"172.21.15.150:6803","nonce":1995659622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":1995659622},{"type":"v1","addr":"172.21.15.150:6807","nonce":1995659622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":1995659622},{"type":"v1","addr":"172.21.15.150:6805","nonce":1995659622}]},"public_addr":"172.21.15.150:6801/1995659622","cluster_addr":"172.21.15.150:6803/1995659622","heartbeat_back_addr":"172.21.15.150:6807/1995659622","heartbeat_front_addr":"172.21.15.150:6805/1995659622","state":["exists","up"]},{"osd":3,"uuid":"d88ed022-4dcb-45fc-9fc6-41ef7c4bf202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":3814942724},{"type":"v1","addr":"172.21.15.33:6811","nonce":3814942724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":3814942724},{"type":"v1","addr":"172.21.15.33:6813","nonce":3814942724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":3814942724},{"type":"v1","addr":"172.21.15.33:6817","nonce":3814942724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":3814942724},{"type":"v1","addr":"172.21.15.33:6815","nonce":3814942724}]},"public_addr":"172.21.15.33:6811/3814942724","cluster_addr":"172.21.15.33:6813/3814942724","heartbeat_back_addr":"172.21.15.33:6817/3814942724","heartbeat_front_addr":"172.21.15.33:6815/3814942724","state":["exists","up"]},{"osd":4,"uuid":"1235e1a6-ff6e-4668-ac1a-073c2a3b7c98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":29081029},{"type":"v1","addr":"172.21.15.99:6809","nonce":29081029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":29081029},{"type":"v1","addr":"172.21.15.99:6811","nonce":29081029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":29081029},{"type":"v1","addr":"172.21.15.99:6815","nonce":29081029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":29081029},{"type":"v1","addr":"172.21.15.99:6813","nonce":29081029}]},"public_addr":"172.21.15.99:6809/29081029","cluster_addr":"172.21.15.99:6811/29081029","heartbeat_back_addr":"172.21.15.99:6815/29081029","heartbeat_front_addr":"172.21.15.99:6813/29081029","state":["exists","up"]},{"osd":5,"uuid":"4b84bff8-d6d7-4258-a0ee-53bc19802585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":3172189533},{"type":"v1","addr":"172.21.15.150:6809","nonce":3172189533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3172189533},{"type":"v1","addr":"172.21.15.150:6811","nonce":3172189533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3172189533},{"type":"v1","addr":"172.21.15.150:6815","nonce":3172189533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3172189533},{"type":"v1","addr":"172.21.15.150:6813","nonce":3172189533}]},"public_addr":"172.21.15.150:6809/3172189533","cluster_addr":"172.21.15.150:6811/3172189533","heartbeat_back_addr":"172.21.15.150:6815/3172189533","heartbeat_front_addr":"172.21.15.150:6813/3172189533","state":["exists","up"]},{"osd":6,"uuid":"51f5fbdd-3794-4c48-9f6c-478f0fa35ac5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":4234541418},{"type":"v1","addr":"172.21.15.33:6819","nonce":4234541418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":4234541418},{"type":"v1","addr":"172.21.15.33:6821","nonce":4234541418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":4234541418},{"type":"v1","addr":"172.21.15.33:6825","nonce":4234541418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":4234541418},{"type":"v1","addr":"172.21.15.33:6823","nonce":4234541418}]},"public_addr":"172.21.15.33:6819/4234541418","cluster_addr":"172.21.15.33:6821/4234541418","heartbeat_back_addr":"172.21.15.33:6825/4234541418","heartbeat_front_addr":"172.21.15.33:6823/4234541418","state":["exists","up"]},{"osd":7,"uuid":"982cc5b3-3163-4505-9bde-a27a8c289457","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":908673090},{"type":"v1","addr":"172.21.15.99:6817","nonce":908673090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":908673090},{"type":"v1","addr":"172.21.15.99:6819","nonce":908673090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":908673090},{"type":"v1","addr":"172.21.15.99:6823","nonce":908673090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":908673090},{"type":"v1","addr":"172.21.15.99:6821","nonce":908673090}]},"public_addr":"172.21.15.99:6817/908673090","cluster_addr":"172.21.15.99:6819/908673090","heartbeat_back_addr":"172.21.15.99:6823/908673090","heartbeat_front_addr":"172.21.15.99:6821/908673090","state":["exists","up"]},{"osd":8,"uuid":"84c9d037-eb84-4eb4-8aa2-e784b703d36c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":1308083602},{"type":"v1","addr":"172.21.15.150:6817","nonce":1308083602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":1308083602},{"type":"v1","addr":"172.21.15.150:6819","nonce":1308083602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":1308083602},{"type":"v1","addr":"172.21.15.150:6823","nonce":1308083602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":1308083602},{"type":"v1","addr":"172.21.15.150:6821","nonce":1308083602}]},"public_addr":"172.21.15.150:6817/1308083602","cluster_addr":"172.21.15.150:6819/1308083602","heartbeat_back_addr":"172.21.15.150:6823/1308083602","heartbeat_front_addr":"172.21.15.150:6821/1308083602","state":["exists","up"]},{"osd":9,"uuid":"a02e965f-0d8f-45e2-9490-337f7f6e3881","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":4133096905},{"type":"v1","addr":"172.21.15.33:6827","nonce":4133096905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":4133096905},{"type":"v1","addr":"172.21.15.33:6829","nonce":4133096905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4133096905},{"type":"v1","addr":"172.21.15.33:6833","nonce":4133096905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":4133096905},{"type":"v1","addr":"172.21.15.33:6831","nonce":4133096905}]},"public_addr":"172.21.15.33:6827/4133096905","cluster_addr":"172.21.15.33:6829/4133096905","heartbeat_back_addr":"172.21.15.33:6833/4133096905","heartbeat_front_addr":"172.21.15.33:6831/4133096905","state":["exists","up"]},{"osd":10,"uuid":"8b29bb96-76e8-45a0-ab19-8d082d755fe9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":935746136},{"type":"v1","addr":"172.21.15.99:6825","nonce":935746136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":935746136},{"type":"v1","addr":"172.21.15.99:6827","nonce":935746136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":935746136},{"type":"v1","addr":"172.21.15.99:6831","nonce":935746136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":935746136},{"type":"v1","addr":"172.21.15.99:6829","nonce":935746136}]},"public_addr":"172.21.15.99:6825/935746136","cluster_addr":"172.21.15.99:6827/935746136","heartbeat_back_addr":"172.21.15.99:6831/935746136","heartbeat_front_addr":"172.21.15.99:6829/935746136","state":["exists","up"]},{"osd":11,"uuid":"07a6a825-c902-4bfd-97cc-cc9074532af5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2069349808},{"type":"v1","addr":"172.21.15.150:6825","nonce":2069349808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2069349808},{"type":"v1","addr":"172.21.15.150:6827","nonce":2069349808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2069349808},{"type":"v1","addr":"172.21.15.150:6831","nonce":2069349808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2069349808},{"type":"v1","addr":"172.21.15.150:6829","nonce":2069349808}]},"public_addr":"172.21.15.150:6825/2069349808","cluster_addr":"172.21.15.150:6827/2069349808","heartbeat_back_addr":"172.21.15.150:6831/2069349808","heartbeat_front_addr":"172.21.15.150:6829/2069349808","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:14:27.454536+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:15:00.538570+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:15:34.125289+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:16:09.853870+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:16:44.928107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:17:19.644037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:17:56.108443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:18:31.731499+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:19:08.062358+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:19:45.204430+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:20:21.013710+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:20:57.653698+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11},{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/3991947997":"2024-05-04T18:12:19.127652+0000","172.21.15.33:0/965859483":"2024-05-04T18:12:19.127652+0000","172.21.15.33:0/2014902144":"2024-05-04T18:11:44.239835+0000","172.21.15.33:0/1314704208":"2024-05-04T18:11:44.239835+0000","172.21.15.33:0/1614640996":"2024-05-04T18:11:44.239835+0000","172.21.15.33:6800/4007322071":"2024-05-04T18:11:44.239835+0000","172.21.15.33:6801/2830135294":"2024-05-04T18:12:19.127652+0000","172.21.15.33:6800/2830135294":"2024-05-04T18:12:19.127652+0000","172.21.15.33:0/3767665033":"2024-05-04T18:12:19.127652+0000","172.21.15.33:6801/4007322071":"2024-05-04T18:11:44.239835+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-03T18:24:43.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:42.998+0000 7f988fe00700 1 -- 172.21.15.33:0/2280726864 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f98740b2990 msgr2=0x7f98740b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:24:43.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:42.998+0000 7f988fe00700 1 --2- 172.21.15.33:0/2280726864 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f98740b2990 0x7f98740b4e50 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9880000c00 tx=0x7f988001a040 comp rx=0 tx=0).stop 2024-05-03T18:24:43.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:42.998+0000 7f988fe00700 1 -- 172.21.15.33:0/2280726864 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9890103fd0 msgr2=0x7f9890075ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:24:43.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:42.998+0000 7f988fe00700 1 --2- 172.21.15.33:0/2280726864 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9890103fd0 0x7f9890075ce0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f987c00d980 tx=0x7f987c00dd40 comp rx=0 tx=0).stop 2024-05-03T18:24:43.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:42.998+0000 7f988fe00700 1 -- 172.21.15.33:0/2280726864 shutdown_connections 2024-05-03T18:24:43.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:42.998+0000 7f988fe00700 1 --2- 172.21.15.33:0/2280726864 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f98740b2990 0x7f98740b4e50 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:43.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:42.998+0000 7f988fe00700 1 --2- 172.21.15.33:0/2280726864 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9890108700 0x7f9890076220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:43.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:42.998+0000 7f988fe00700 1 --2- 172.21.15.33:0/2280726864 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9890103620 0x7f98900757a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:43.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:42.998+0000 7f988fe00700 1 --2- 172.21.15.33:0/2280726864 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9890103fd0 0x7f9890075ce0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:43.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:42.998+0000 7f988fe00700 1 -- 172.21.15.33:0/2280726864 >> 172.21.15.33:0/2280726864 conn(0x7f98900feef0 msgr2=0x7f98901002b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:24:43.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:42.999+0000 7f988fe00700 1 -- 172.21.15.33:0/2280726864 shutdown_connections 2024-05-03T18:24:43.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:42.999+0000 7f988fe00700 1 -- 172.21.15.33:0/2280726864 wait complete. 2024-05-03T18:24:43.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:24:43 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/2280726864' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T18:24:43.786 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-05-03T18:24:43.786 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-05-03T18:24:43.787 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-05-03T18:24:43.787 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-05-03T18:24:43.787 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-05-03T18:24:43.788 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-05-03T18:24:43.788 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-05-03T18:24:43.788 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-05-03T18:24:43.789 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-05-03T18:24:43.789 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-05-03T18:24:43.790 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-05-03T18:24:43.791 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-05-03T18:24:43.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:24:43 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/2280726864' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T18:24:43.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:24:43 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/2280726864' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T18:24:44.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:24:44 smithi099 ceph-mon[31658]: pgmap v451: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:24:44.820 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:24:44.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:24:44 smithi150 ceph-mon[32492]: pgmap v451: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:24:44.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:24:44 smithi033 ceph-mon[28330]: pgmap v451: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:24:46.441 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:24:46.441 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:24:46.441 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:24:46.452 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:24:46.460 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:24:46.465 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:24:46.476 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:24:46.476 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:24:46.479 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:24:46.482 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:24:46.484 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:24:46.646 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:24:46 smithi033 ceph-mon[28330]: pgmap v452: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:24:46.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:24:46 smithi099 ceph-mon[31658]: pgmap v452: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:24:46.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:24:46 smithi150 ceph-mon[32492]: pgmap v452: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:24:48.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:24:48 smithi033 ceph-mon[28330]: pgmap v453: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:24:48.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:24:48 smithi099 ceph-mon[31658]: pgmap v453: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:24:48.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:24:48 smithi150 ceph-mon[32492]: pgmap v453: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:24:50.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:24:50 smithi099 ceph-mon[31658]: pgmap v454: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:24:50.860 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:24:50 smithi033 ceph-mon[28330]: pgmap v454: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:24:50.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:24:50 smithi150 ceph-mon[32492]: pgmap v454: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:24:52.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:24:52 smithi033 ceph-mon[28330]: pgmap v455: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:24:52.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:24:52 smithi099 ceph-mon[31658]: pgmap v455: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:24:52.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:24:52 smithi150 ceph-mon[32492]: pgmap v455: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:24:54.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:54.236+0000 7fdba0a00700 1 -- 172.21.15.33:0/496588724 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb9c104eb0 msgr2=0x7fdb9c10d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:24:54.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:54.236+0000 7fdba0a00700 1 --2- 172.21.15.33:0/496588724 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb9c104eb0 0x7fdb9c10d3a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fdb88009990 tx=0x7fdb88009ca0 comp rx=0 tx=0).stop 2024-05-03T18:24:54.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:54.236+0000 7fdba0a00700 1 -- 172.21.15.33:0/496588724 shutdown_connections 2024-05-03T18:24:54.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:54.236+0000 7fdba0a00700 1 --2- 172.21.15.33:0/496588724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb9c10d9c0 0x7fdb9c10fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:54.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:54.236+0000 7fdba0a00700 1 --2- 172.21.15.33:0/496588724 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb9c104590 0x7fdb9c104970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:54.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:54.236+0000 7fdba0a00700 1 --2- 172.21.15.33:0/496588724 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb9c104eb0 0x7fdb9c10d3a0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:54.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:54.236+0000 7fdba0a00700 1 -- 172.21.15.33:0/496588724 >> 172.21.15.33:0/496588724 conn(0x7fdb9c0fddc0 msgr2=0x7fdb9c1001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:24:54.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:54.236+0000 7fdba0a00700 1 -- 172.21.15.33:0/496588724 shutdown_connections 2024-05-03T18:24:54.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:54.236+0000 7fdba0a00700 1 -- 172.21.15.33:0/496588724 wait complete. 2024-05-03T18:24:54.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:54.237+0000 7fdba0a00700 1 Processor -- start 2024-05-03T18:24:54.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:54.238+0000 7fdba0a00700 1 -- start start 2024-05-03T18:24:54.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:54.238+0000 7fdba0a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb9c104590 0x7fdb9c19bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:24:54.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:54.238+0000 7fdba0a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb9c104eb0 0x7fdb9c19c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:24:54.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:54.238+0000 7fdba0a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb9c10d9c0 0x7fdb9c19f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:24:54.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:54.238+0000 7fdba0a00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fdb9c113490 con 0x7fdb9c104590 2024-05-03T18:24:54.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:54.238+0000 7fdba0a00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fdb9c113310 con 0x7fdb9c104eb0 2024-05-03T18:24:54.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:54.238+0000 7fdba0a00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fdb9c113610 con 0x7fdb9c10d9c0 2024-05-03T18:24:54.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:54.238+0000 7fdb9b400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb9c104590 0x7fdb9c19bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:24:54.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:54.238+0000 7fdb9b400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb9c104590 0x7fdb9c19bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55484/0 (socket says 172.21.15.33:55484) 2024-05-03T18:24:54.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:54.238+0000 7fdb9b400700 1 -- 172.21.15.33:0/516227598 learned_addr learned my addr 172.21.15.33:0/516227598 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:24:54.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:54.239+0000 7fdb9aa00700 1 --2- 172.21.15.33:0/516227598 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb9c104eb0 0x7fdb9c19c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:24:54.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:54.239+0000 7fdb9be00700 1 --2- 172.21.15.33:0/516227598 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb9c10d9c0 0x7fdb9c19f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:24:54.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:54.239+0000 7fdb9b400700 1 -- 172.21.15.33:0/516227598 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb9c10d9c0 msgr2=0x7fdb9c19f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:24:54.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:54.239+0000 7fdb9b400700 1 --2- 172.21.15.33:0/516227598 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb9c10d9c0 0x7fdb9c19f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:54.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:54.239+0000 7fdb9b400700 1 -- 172.21.15.33:0/516227598 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb9c104eb0 msgr2=0x7fdb9c19c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:24:54.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:54.239+0000 7fdb9b400700 1 --2- 172.21.15.33:0/516227598 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb9c104eb0 0x7fdb9c19c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:54.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:54.239+0000 7fdb9b400700 1 -- 172.21.15.33:0/516227598 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb9c19fc10 con 0x7fdb9c104590 2024-05-03T18:24:54.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:54.239+0000 7fdb9aa00700 1 --2- 172.21.15.33:0/516227598 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb9c104eb0 0x7fdb9c19c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:24:54.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:54.239+0000 7fdb9be00700 1 --2- 172.21.15.33:0/516227598 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb9c10d9c0 0x7fdb9c19f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:24:54.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:54.239+0000 7fdb9b400700 1 --2- 172.21.15.33:0/516227598 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb9c104590 0x7fdb9c19bcc0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c00d950 tx=0x7fdb8c00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:24:54.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:54.240+0000 7fdb93e00700 1 -- 172.21.15.33:0/516227598 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb8c016070 con 0x7fdb9c104590 2024-05-03T18:24:54.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:54.240+0000 7fdb93e00700 1 -- 172.21.15.33:0/516227598 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdb8c010460 con 0x7fdb9c104590 2024-05-03T18:24:54.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:54.240+0000 7fdb93e00700 1 -- 172.21.15.33:0/516227598 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb8c00f620 con 0x7fdb9c104590 2024-05-03T18:24:54.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:54.240+0000 7fdba0a00700 1 -- 172.21.15.33:0/516227598 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb9c19ff00 con 0x7fdb9c104590 2024-05-03T18:24:54.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:54.240+0000 7fdba0a00700 1 -- 172.21.15.33:0/516227598 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb9c1a8870 con 0x7fdb9c104590 2024-05-03T18:24:54.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:54.241+0000 7fdba0a00700 1 -- 172.21.15.33:0/516227598 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdb9c103c70 con 0x7fdb9c104590 2024-05-03T18:24:54.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:54.244+0000 7fdb93e00700 1 -- 172.21.15.33:0/516227598 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fdb8c019070 con 0x7fdb9c104590 2024-05-03T18:24:54.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:54.245+0000 7fdb93e00700 1 --2- 172.21.15.33:0/516227598 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fdb800b2990 0x7fdb800b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:24:54.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:54.245+0000 7fdb93e00700 1 -- 172.21.15.33:0/516227598 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7fdb8c0e2840 con 0x7fdb9c104590 2024-05-03T18:24:54.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:54.245+0000 7fdb9aa00700 1 --2- 172.21.15.33:0/516227598 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fdb800b2990 0x7fdb800b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:24:54.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:54.246+0000 7fdb93e00700 1 --2- 172.21.15.33:0/516227598 >> [v2:172.21.15.33:6802/637686669,v1:172.21.15.33:6803/637686669] conn(0x7fdb800bdb70 0x7fdb800c0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:24:54.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:54.246+0000 7fdb93e00700 1 -- 172.21.15.33:0/516227598 --> [v2:172.21.15.33:6802/637686669,v1:172.21.15.33:6803/637686669] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdb8c013ec0 con 0x7fdb800bdb70 2024-05-03T18:24:54.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:54.246+0000 7fdb93e00700 1 -- 172.21.15.33:0/516227598 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7fdb8c0e2bf0 con 0x7fdb9c104590 2024-05-03T18:24:54.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:54.246+0000 7fdb9be00700 1 --2- 172.21.15.33:0/516227598 >> [v2:172.21.15.33:6802/637686669,v1:172.21.15.33:6803/637686669] conn(0x7fdb800bdb70 0x7fdb800c0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:24:54.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:54.246+0000 7fdb9aa00700 1 --2- 172.21.15.33:0/516227598 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fdb800b2990 0x7fdb800b4e50 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c110240 tx=0x7fdb8801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:24:54.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:54.247+0000 7fdb9be00700 1 --2- 172.21.15.33:0/516227598 >> [v2:172.21.15.33:6802/637686669,v1:172.21.15.33:6803/637686669] conn(0x7fdb800bdb70 0x7fdb800c0010 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:24:54.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:54.248+0000 7fdb93e00700 1 -- 172.21.15.33:0/516227598 <== osd.0 v2:172.21.15.33:6802/637686669 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdb8c013ec0 con 0x7fdb800bdb70 2024-05-03T18:24:54.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:54.267+0000 7fdba0a00700 1 -- 172.21.15.33:0/516227598 --> [v2:172.21.15.33:6802/637686669,v1:172.21.15.33:6803/637686669] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fdb9c06ba60 con 0x7fdb800bdb70 2024-05-03T18:24:54.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:54.267+0000 7fdb93e00700 1 -- 172.21.15.33:0/516227598 <== osd.0 v2:172.21.15.33:6802/637686669 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fdb9c06ba60 con 0x7fdb800bdb70 2024-05-03T18:24:54.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:54.268+0000 7fdba0a00700 1 -- 172.21.15.33:0/516227598 >> [v2:172.21.15.33:6802/637686669,v1:172.21.15.33:6803/637686669] conn(0x7fdb800bdb70 msgr2=0x7fdb800c0010 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:24:54.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:54.268+0000 7fdba0a00700 1 --2- 172.21.15.33:0/516227598 >> [v2:172.21.15.33:6802/637686669,v1:172.21.15.33:6803/637686669] conn(0x7fdb800bdb70 0x7fdb800c0010 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:54.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:54.268+0000 7fdba0a00700 1 -- 172.21.15.33:0/516227598 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fdb800b2990 msgr2=0x7fdb800b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:24:54.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:54.268+0000 7fdba0a00700 1 --2- 172.21.15.33:0/516227598 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fdb800b2990 0x7fdb800b4e50 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c110240 tx=0x7fdb8801a040 comp rx=0 tx=0).stop 2024-05-03T18:24:54.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:54.268+0000 7fdba0a00700 1 -- 172.21.15.33:0/516227598 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb9c104590 msgr2=0x7fdb9c19bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:24:54.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:54.268+0000 7fdba0a00700 1 --2- 172.21.15.33:0/516227598 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb9c104590 0x7fdb9c19bcc0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c00d950 tx=0x7fdb8c00dd10 comp rx=0 tx=0).stop 2024-05-03T18:24:54.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:54.268+0000 7fdb9be00700 1 -- 172.21.15.33:0/516227598 reap_dead start 2024-05-03T18:24:54.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:54.269+0000 7fdba0a00700 1 -- 172.21.15.33:0/516227598 shutdown_connections 2024-05-03T18:24:54.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:54.269+0000 7fdba0a00700 1 -- 172.21.15.33:0/516227598 >> 172.21.15.33:0/516227598 conn(0x7fdb9c0fddc0 msgr2=0x7fdb9c10ed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:24:54.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:54.269+0000 7fdba0a00700 1 -- 172.21.15.33:0/516227598 shutdown_connections 2024-05-03T18:24:54.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:54.269+0000 7fdba0a00700 1 -- 172.21.15.33:0/516227598 wait complete. 2024-05-03T18:24:54.634 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:24:54 smithi033 ceph-mon[28330]: pgmap v456: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:24:54.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:24:54 smithi099 ceph-mon[31658]: pgmap v456: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:24:54.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:24:54 smithi150 ceph-mon[32492]: pgmap v456: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:24:56.689 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:24:56 smithi033 ceph-mon[28330]: pgmap v457: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:24:56.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:24:56 smithi099 ceph-mon[31658]: pgmap v457: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:24:56.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:24:56 smithi150 ceph-mon[32492]: pgmap v457: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:24:57.421 INFO:teuthology.orchestra.run.smithi033.stdout:34359738496 2024-05-03T18:24:57.421 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-05-03T18:24:58.648 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:24:58 smithi033 ceph-mon[28330]: pgmap v458: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:24:58.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:24:58 smithi099 ceph-mon[31658]: pgmap v458: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:24:58.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:24:58 smithi150 ceph-mon[32492]: pgmap v458: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:24:59.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:58.857+0000 7f86e2400700 1 -- 172.21.15.33:0/491290062 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86dc075840 msgr2=0x7f86dc075c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:24:59.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:58.857+0000 7f86e2400700 1 --2- 172.21.15.33:0/491290062 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86dc075840 0x7f86dc075c20 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f86d0009990 tx=0x7f86d0009ca0 comp rx=0 tx=0).stop 2024-05-03T18:24:59.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:58.857+0000 7f86e2400700 1 -- 172.21.15.33:0/491290062 shutdown_connections 2024-05-03T18:24:59.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:58.857+0000 7f86e2400700 1 --2- 172.21.15.33:0/491290062 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86dc1143e0 0x7f86dc1167d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:59.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:58.857+0000 7f86e2400700 1 --2- 172.21.15.33:0/491290062 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86dc075840 0x7f86dc075c20 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:59.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:58.857+0000 7f86e2400700 1 --2- 172.21.15.33:0/491290062 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86dc076160 0x7f86dc113ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:59.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:58.857+0000 7f86e2400700 1 -- 172.21.15.33:0/491290062 >> 172.21.15.33:0/491290062 conn(0x7f86dc0fef20 msgr2=0x7f86dc101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:24:59.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:58.857+0000 7f86e2400700 1 -- 172.21.15.33:0/491290062 shutdown_connections 2024-05-03T18:24:59.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:58.857+0000 7f86e2400700 1 -- 172.21.15.33:0/491290062 wait complete. 2024-05-03T18:24:59.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:58.859+0000 7f86e2400700 1 Processor -- start 2024-05-03T18:24:59.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:58.859+0000 7f86e2400700 1 -- start start 2024-05-03T18:24:59.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:58.859+0000 7f86e2400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86dc075840 0x7f86dc19ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:24:59.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:58.859+0000 7f86e2400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86dc076160 0x7f86dc1a0510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:24:59.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:58.859+0000 7f86e2400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86dc1143e0 0x7f86dc1a4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:24:59.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:58.859+0000 7f86e2400700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f86dc119ac0 con 0x7f86dc076160 2024-05-03T18:24:59.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:58.859+0000 7f86e2400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f86dc119940 con 0x7f86dc1143e0 2024-05-03T18:24:59.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:58.859+0000 7f86e2400700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f86dc119c40 con 0x7f86dc075840 2024-05-03T18:24:59.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:58.860+0000 7f86dbe00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86dc076160 0x7f86dc1a0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:24:59.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:58.860+0000 7f86dbe00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86dc076160 0x7f86dc1a0510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54016/0 (socket says 172.21.15.33:54016) 2024-05-03T18:24:59.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:58.860+0000 7f86dbe00700 1 -- 172.21.15.33:0/118317582 learned_addr learned my addr 172.21.15.33:0/118317582 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:24:59.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:58.860+0000 7f86e1000700 1 --2- 172.21.15.33:0/118317582 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86dc075840 0x7f86dc19ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:24:59.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:58.860+0000 7f86e1a00700 1 --2- 172.21.15.33:0/118317582 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86dc1143e0 0x7f86dc1a4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:24:59.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:58.860+0000 7f86dbe00700 1 -- 172.21.15.33:0/118317582 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86dc075840 msgr2=0x7f86dc19ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:24:59.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:58.860+0000 7f86dbe00700 1 --2- 172.21.15.33:0/118317582 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86dc075840 0x7f86dc19ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:59.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:58.860+0000 7f86dbe00700 1 -- 172.21.15.33:0/118317582 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86dc1143e0 msgr2=0x7f86dc1a4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:24:59.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:58.860+0000 7f86dbe00700 1 --2- 172.21.15.33:0/118317582 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86dc1143e0 0x7f86dc1a4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:59.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:58.860+0000 7f86dbe00700 1 -- 172.21.15.33:0/118317582 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86dc1a4f90 con 0x7f86dc076160 2024-05-03T18:24:59.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:58.860+0000 7f86e1000700 1 --2- 172.21.15.33:0/118317582 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86dc075840 0x7f86dc19ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:24:59.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:58.860+0000 7f86e1a00700 1 --2- 172.21.15.33:0/118317582 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86dc1143e0 0x7f86dc1a4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:24:59.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:58.861+0000 7f86dbe00700 1 --2- 172.21.15.33:0/118317582 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86dc076160 0x7f86dc1a0510 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f86c800d950 tx=0x7f86c800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:24:59.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:58.861+0000 7f86d9600700 1 -- 172.21.15.33:0/118317582 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c8016070 con 0x7f86dc076160 2024-05-03T18:24:59.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:58.861+0000 7f86e2400700 1 -- 172.21.15.33:0/118317582 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86dc1a5280 con 0x7f86dc076160 2024-05-03T18:24:59.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:58.861+0000 7f86d9600700 1 -- 172.21.15.33:0/118317582 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f86c8010460 con 0x7f86dc076160 2024-05-03T18:24:59.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:58.861+0000 7f86d9600700 1 -- 172.21.15.33:0/118317582 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c800f620 con 0x7f86dc076160 2024-05-03T18:24:59.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:58.861+0000 7f86e2400700 1 -- 172.21.15.33:0/118317582 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86dc1acb00 con 0x7f86dc076160 2024-05-03T18:24:59.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:58.862+0000 7f86e2400700 1 -- 172.21.15.33:0/118317582 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f86bc001030 con 0x7f86dc076160 2024-05-03T18:24:59.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:58.865+0000 7f86d9600700 1 -- 172.21.15.33:0/118317582 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f86c8019070 con 0x7f86dc076160 2024-05-03T18:24:59.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:58.866+0000 7f86d9600700 1 --2- 172.21.15.33:0/118317582 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f86c40b2990 0x7f86c40b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:24:59.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:58.866+0000 7f86d9600700 1 -- 172.21.15.33:0/118317582 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f86c80e27f0 con 0x7f86dc076160 2024-05-03T18:24:59.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:58.867+0000 7f86e1000700 1 --2- 172.21.15.33:0/118317582 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f86c40b2990 0x7f86c40b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:24:59.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:58.867+0000 7f86d9600700 1 --2- 172.21.15.33:0/118317582 >> [v2:172.21.15.99:6808/29081029,v1:172.21.15.99:6809/29081029] conn(0x7f86c40bdb70 0x7f86c40c0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:24:59.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:58.867+0000 7f86d9600700 1 -- 172.21.15.33:0/118317582 --> [v2:172.21.15.99:6808/29081029,v1:172.21.15.99:6809/29081029] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f86c40c06e0 con 0x7f86c40bdb70 2024-05-03T18:24:59.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:58.867+0000 7f86d9600700 1 -- 172.21.15.33:0/118317582 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f86c80e2ba0 con 0x7f86dc076160 2024-05-03T18:24:59.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:58.867+0000 7f86e1000700 1 --2- 172.21.15.33:0/118317582 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f86c40b2990 0x7f86c40b4e50 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f86d000b5c0 tx=0x7f86d001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:24:59.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:58.867+0000 7f86e1a00700 1 --2- 172.21.15.33:0/118317582 >> [v2:172.21.15.99:6808/29081029,v1:172.21.15.99:6809/29081029] conn(0x7f86c40bdb70 0x7f86c40c0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:24:59.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:58.868+0000 7f86e1a00700 1 --2- 172.21.15.33:0/118317582 >> [v2:172.21.15.99:6808/29081029,v1:172.21.15.99:6809/29081029] conn(0x7f86c40bdb70 0x7f86c40c0010 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:24:59.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:58.870+0000 7f86d9600700 1 -- 172.21.15.33:0/118317582 <== osd.4 v2:172.21.15.99:6808/29081029 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f86c40c06e0 con 0x7f86c40bdb70 2024-05-03T18:24:59.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:58.889+0000 7f86e2400700 1 -- 172.21.15.33:0/118317582 --> [v2:172.21.15.99:6808/29081029,v1:172.21.15.99:6809/29081029] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f86bc002e60 con 0x7f86c40bdb70 2024-05-03T18:24:59.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:58.890+0000 7f86d9600700 1 -- 172.21.15.33:0/118317582 <== osd.4 v2:172.21.15.99:6808/29081029 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f86bc002e60 con 0x7f86c40bdb70 2024-05-03T18:24:59.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:58.890+0000 7f86e2400700 1 -- 172.21.15.33:0/118317582 >> [v2:172.21.15.99:6808/29081029,v1:172.21.15.99:6809/29081029] conn(0x7f86c40bdb70 msgr2=0x7f86c40c0010 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:24:59.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:58.890+0000 7f86e2400700 1 --2- 172.21.15.33:0/118317582 >> [v2:172.21.15.99:6808/29081029,v1:172.21.15.99:6809/29081029] conn(0x7f86c40bdb70 0x7f86c40c0010 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:24:59.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:58.891+0000 7f86e2400700 1 -- 172.21.15.33:0/118317582 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f86c40b2990 msgr2=0x7f86c40b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:24:59.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:58.891+0000 7f86e2400700 1 --2- 172.21.15.33:0/118317582 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f86c40b2990 0x7f86c40b4e50 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f86d000b5c0 tx=0x7f86d001a040 comp rx=0 tx=0).stop 2024-05-03T18:24:59.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:58.891+0000 7f86e2400700 1 -- 172.21.15.33:0/118317582 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86dc076160 msgr2=0x7f86dc1a0510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:24:59.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:58.891+0000 7f86e2400700 1 --2- 172.21.15.33:0/118317582 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86dc076160 0x7f86dc1a0510 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f86c800d950 tx=0x7f86c800dd10 comp rx=0 tx=0).stop 2024-05-03T18:24:59.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:58.891+0000 7f86e1a00700 1 -- 172.21.15.33:0/118317582 reap_dead start 2024-05-03T18:24:59.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:58.891+0000 7f86e2400700 1 -- 172.21.15.33:0/118317582 shutdown_connections 2024-05-03T18:24:59.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:58.891+0000 7f86e2400700 1 -- 172.21.15.33:0/118317582 >> 172.21.15.33:0/118317582 conn(0x7f86dc0fef20 msgr2=0x7f86dc0ffbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:24:59.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:58.891+0000 7f86e2400700 1 -- 172.21.15.33:0/118317582 shutdown_connections 2024-05-03T18:24:59.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:24:58.891+0000 7f86e2400700 1 -- 172.21.15.33:0/118317582 wait complete. 2024-05-03T18:25:00.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:00.429+0000 7f42fb200700 1 -- 172.21.15.33:0/3641617818 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42fc1045d0 msgr2=0x7f42fc1049b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:00.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:00.429+0000 7f42fb200700 1 --2- 172.21.15.33:0/3641617818 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42fc1045d0 0x7f42fc1049b0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f42ec009990 tx=0x7f42ec009ca0 comp rx=0 tx=0).stop 2024-05-03T18:25:00.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:00.430+0000 7f42fb200700 1 -- 172.21.15.33:0/3641617818 shutdown_connections 2024-05-03T18:25:00.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:00.430+0000 7f42fb200700 1 --2- 172.21.15.33:0/3641617818 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42fc10da00 0x7f42fc10fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:00.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:00.430+0000 7f42fb200700 1 --2- 172.21.15.33:0/3641617818 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42fc1045d0 0x7f42fc1049b0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:00.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:00.430+0000 7f42fb200700 1 --2- 172.21.15.33:0/3641617818 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f42fc104ef0 0x7f42fc10d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:00.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:00.430+0000 7f42fb200700 1 -- 172.21.15.33:0/3641617818 >> 172.21.15.33:0/3641617818 conn(0x7f42fc0fddc0 msgr2=0x7f42fc1001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:25:00.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:00.430+0000 7f42fb200700 1 -- 172.21.15.33:0/3641617818 shutdown_connections 2024-05-03T18:25:00.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:00.430+0000 7f42fb200700 1 -- 172.21.15.33:0/3641617818 wait complete. 2024-05-03T18:25:00.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:00.431+0000 7f42fb200700 1 Processor -- start 2024-05-03T18:25:00.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:00.431+0000 7f42fb200700 1 -- start start 2024-05-03T18:25:00.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:00.432+0000 7f42fb200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42fc1045d0 0x7f42fc074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:00.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:00.432+0000 7f42fb200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f42fc104ef0 0x7f42fc06dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:00.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:00.432+0000 7f42fb200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42fc10da00 0x7f42fc06e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:00.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:00.432+0000 7f42fb200700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f42fc1134a0 con 0x7f42fc10da00 2024-05-03T18:25:00.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:00.432+0000 7f42fb200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f42fc113320 con 0x7f42fc1045d0 2024-05-03T18:25:00.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:00.432+0000 7f42fb200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f42fc113620 con 0x7f42fc104ef0 2024-05-03T18:25:00.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:00.432+0000 7f42fa800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42fc10da00 0x7f42fc06e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:00.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:00.432+0000 7f42fa800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42fc10da00 0x7f42fc06e2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54028/0 (socket says 172.21.15.33:54028) 2024-05-03T18:25:00.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:00.432+0000 7f42fa800700 1 -- 172.21.15.33:0/3037074260 learned_addr learned my addr 172.21.15.33:0/3037074260 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:25:00.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:00.432+0000 7f42f9e00700 1 --2- 172.21.15.33:0/3037074260 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42fc1045d0 0x7f42fc074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:00.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:00.433+0000 7f42f9400700 1 --2- 172.21.15.33:0/3037074260 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f42fc104ef0 0x7f42fc06dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:00.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:00.433+0000 7f42fa800700 1 -- 172.21.15.33:0/3037074260 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f42fc104ef0 msgr2=0x7f42fc06dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:00.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:00.433+0000 7f42fa800700 1 --2- 172.21.15.33:0/3037074260 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f42fc104ef0 0x7f42fc06dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:00.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:00.433+0000 7f42fa800700 1 -- 172.21.15.33:0/3037074260 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42fc1045d0 msgr2=0x7f42fc074d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:00.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:00.433+0000 7f42fa800700 1 --2- 172.21.15.33:0/3037074260 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42fc1045d0 0x7f42fc074d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:00.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:00.433+0000 7f42fa800700 1 -- 172.21.15.33:0/3037074260 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42fc06e940 con 0x7f42fc10da00 2024-05-03T18:25:00.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:00.433+0000 7f42fa800700 1 --2- 172.21.15.33:0/3037074260 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42fc10da00 0x7f42fc06e2d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f42f4007ee0 tx=0x7f42f400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:25:00.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:00.434+0000 7f42f2a00700 1 -- 172.21.15.33:0/3037074260 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f4004bb0 con 0x7f42fc10da00 2024-05-03T18:25:00.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:00.434+0000 7f42f2a00700 1 -- 172.21.15.33:0/3037074260 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f42f4004d50 con 0x7f42fc10da00 2024-05-03T18:25:00.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:00.434+0000 7f42f2a00700 1 -- 172.21.15.33:0/3037074260 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f40058b0 con 0x7f42fc10da00 2024-05-03T18:25:00.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:00.434+0000 7f42fb200700 1 -- 172.21.15.33:0/3037074260 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42fc072830 con 0x7f42fc10da00 2024-05-03T18:25:00.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:00.434+0000 7f42fb200700 1 -- 172.21.15.33:0/3037074260 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42fc072d20 con 0x7f42fc10da00 2024-05-03T18:25:00.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:00.435+0000 7f42fb200700 1 -- 172.21.15.33:0/3037074260 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f42fc073250 con 0x7f42fc10da00 2024-05-03T18:25:00.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:00.438+0000 7f42f2a00700 1 -- 172.21.15.33:0/3037074260 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f42f401a020 con 0x7f42fc10da00 2024-05-03T18:25:00.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:00.440+0000 7f42f2a00700 1 --2- 172.21.15.33:0/3037074260 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f42e00b2990 0x7f42e00b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:00.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:00.440+0000 7f42f2a00700 1 -- 172.21.15.33:0/3037074260 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f42f40e2400 con 0x7f42fc10da00 2024-05-03T18:25:00.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:00.440+0000 7f42f9e00700 1 --2- 172.21.15.33:0/3037074260 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f42e00b2990 0x7f42e00b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:00.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:00.441+0000 7f42f2a00700 1 --2- 172.21.15.33:0/3037074260 >> [v2:172.21.15.99:6800/1536122841,v1:172.21.15.99:6801/1536122841] conn(0x7f42e00bdb70 0x7f42e00c0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:00.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:00.441+0000 7f42f2a00700 1 -- 172.21.15.33:0/3037074260 --> [v2:172.21.15.99:6800/1536122841,v1:172.21.15.99:6801/1536122841] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f42f4012ec0 con 0x7f42e00bdb70 2024-05-03T18:25:00.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:00.441+0000 7f42f2a00700 1 -- 172.21.15.33:0/3037074260 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f42f40e27b0 con 0x7f42fc10da00 2024-05-03T18:25:00.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:00.441+0000 7f42f9e00700 1 --2- 172.21.15.33:0/3037074260 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f42e00b2990 0x7f42e00b4e50 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f42ec006010 tx=0x7f42ec00b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:25:00.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:00.441+0000 7f42f9400700 1 --2- 172.21.15.33:0/3037074260 >> [v2:172.21.15.99:6800/1536122841,v1:172.21.15.99:6801/1536122841] conn(0x7f42e00bdb70 0x7f42e00c0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:00.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:00.442+0000 7f42f9400700 1 --2- 172.21.15.33:0/3037074260 >> [v2:172.21.15.99:6800/1536122841,v1:172.21.15.99:6801/1536122841] conn(0x7f42e00bdb70 0x7f42e00c0010 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:25:00.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:00.445+0000 7f42f2a00700 1 -- 172.21.15.33:0/3037074260 <== osd.1 v2:172.21.15.99:6800/1536122841 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f42f4012ec0 con 0x7f42e00bdb70 2024-05-03T18:25:00.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:00.463+0000 7f42fb200700 1 -- 172.21.15.33:0/3037074260 --> [v2:172.21.15.99:6800/1536122841,v1:172.21.15.99:6801/1536122841] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f42fc06ba20 con 0x7f42e00bdb70 2024-05-03T18:25:00.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:00.464+0000 7f42f2a00700 1 -- 172.21.15.33:0/3037074260 <== osd.1 v2:172.21.15.99:6800/1536122841 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f42fc06ba20 con 0x7f42e00bdb70 2024-05-03T18:25:00.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:00.464+0000 7f42fb200700 1 -- 172.21.15.33:0/3037074260 >> [v2:172.21.15.99:6800/1536122841,v1:172.21.15.99:6801/1536122841] conn(0x7f42e00bdb70 msgr2=0x7f42e00c0010 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:00.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:00.464+0000 7f42fb200700 1 --2- 172.21.15.33:0/3037074260 >> [v2:172.21.15.99:6800/1536122841,v1:172.21.15.99:6801/1536122841] conn(0x7f42e00bdb70 0x7f42e00c0010 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:00.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:00.465+0000 7f42fb200700 1 -- 172.21.15.33:0/3037074260 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f42e00b2990 msgr2=0x7f42e00b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:00.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:00.465+0000 7f42fb200700 1 --2- 172.21.15.33:0/3037074260 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f42e00b2990 0x7f42e00b4e50 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f42ec006010 tx=0x7f42ec00b540 comp rx=0 tx=0).stop 2024-05-03T18:25:00.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:00.465+0000 7f42fb200700 1 -- 172.21.15.33:0/3037074260 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42fc10da00 msgr2=0x7f42fc06e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:00.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:00.465+0000 7f42fb200700 1 --2- 172.21.15.33:0/3037074260 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42fc10da00 0x7f42fc06e2d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f42f4007ee0 tx=0x7f42f400d3b0 comp rx=0 tx=0).stop 2024-05-03T18:25:00.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:00.465+0000 7f42fa800700 1 -- 172.21.15.33:0/3037074260 reap_dead start 2024-05-03T18:25:00.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:00.465+0000 7f42fb200700 1 -- 172.21.15.33:0/3037074260 shutdown_connections 2024-05-03T18:25:00.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:00.465+0000 7f42fb200700 1 -- 172.21.15.33:0/3037074260 >> 172.21.15.33:0/3037074260 conn(0x7f42fc0fddc0 msgr2=0x7f42fc0feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:25:00.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:00.465+0000 7f42fb200700 1 -- 172.21.15.33:0/3037074260 shutdown_connections 2024-05-03T18:25:00.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:00.465+0000 7f42fb200700 1 -- 172.21.15.33:0/3037074260 wait complete. 2024-05-03T18:25:00.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:25:00 smithi033 ceph-mon[28330]: pgmap v459: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:25:00.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:25:00 smithi099 ceph-mon[31658]: pgmap v459: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:25:00.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:25:00 smithi150 ceph-mon[32492]: pgmap v459: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:25:02.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.266+0000 7ff0c1e00700 1 -- 172.21.15.33:0/3407532521 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0bc104100 msgr2=0x7ff0bc108120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:02.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.266+0000 7ff0c1e00700 1 --2- 172.21.15.33:0/3407532521 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0bc104100 0x7ff0bc108120 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff0a80099c0 tx=0x7ff0a8009cd0 comp rx=0 tx=0).stop 2024-05-03T18:25:02.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.267+0000 7ff0c1e00700 1 -- 172.21.15.33:0/3407532521 shutdown_connections 2024-05-03T18:25:02.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.267+0000 7ff0c1e00700 1 --2- 172.21.15.33:0/3407532521 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0bc108850 0x7ff0bc1103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:02.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.267+0000 7ff0c1e00700 1 --2- 172.21.15.33:0/3407532521 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0bc103750 0x7ff0bc103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:02.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.267+0000 7ff0c1e00700 1 --2- 172.21.15.33:0/3407532521 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0bc104100 0x7ff0bc108120 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:02.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.267+0000 7ff0c1e00700 1 -- 172.21.15.33:0/3407532521 >> 172.21.15.33:0/3407532521 conn(0x7ff0bc07a600 msgr2=0x7ff0bc07aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:25:02.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.268+0000 7ff0c1e00700 1 -- 172.21.15.33:0/3407532521 shutdown_connections 2024-05-03T18:25:02.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.268+0000 7ff0c1e00700 1 -- 172.21.15.33:0/3407532521 wait complete. 2024-05-03T18:25:02.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.269+0000 7ff0c1e00700 1 Processor -- start 2024-05-03T18:25:02.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.269+0000 7ff0c1e00700 1 -- start start 2024-05-03T18:25:02.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.269+0000 7ff0c1e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0bc103750 0x7ff0bc1a18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:02.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.269+0000 7ff0c1e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0bc104100 0x7ff0bc1a1df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:02.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.269+0000 7ff0c1e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0bc108850 0x7ff0bc19ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:02.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.269+0000 7ff0c1e00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff0bc113510 con 0x7ff0bc108850 2024-05-03T18:25:02.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.269+0000 7ff0c1e00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff0bc113390 con 0x7ff0bc104100 2024-05-03T18:25:02.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.269+0000 7ff0c1e00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff0bc113690 con 0x7ff0bc103750 2024-05-03T18:25:02.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.270+0000 7ff0ba000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0bc104100 0x7ff0bc1a1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:02.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.270+0000 7ff0bb400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0bc108850 0x7ff0bc19ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:02.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.270+0000 7ff0ba000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0bc104100 0x7ff0bc1a1df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:47488/0 (socket says 172.21.15.33:47488) 2024-05-03T18:25:02.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.270+0000 7ff0ba000700 1 -- 172.21.15.33:0/3001833256 learned_addr learned my addr 172.21.15.33:0/3001833256 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:25:02.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.270+0000 7ff0baa00700 1 --2- 172.21.15.33:0/3001833256 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0bc103750 0x7ff0bc1a18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:02.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.271+0000 7ff0bb400700 1 -- 172.21.15.33:0/3001833256 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0bc103750 msgr2=0x7ff0bc1a18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:02.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.271+0000 7ff0bb400700 1 --2- 172.21.15.33:0/3001833256 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0bc103750 0x7ff0bc1a18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:02.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.271+0000 7ff0bb400700 1 -- 172.21.15.33:0/3001833256 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0bc104100 msgr2=0x7ff0bc1a1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:02.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.271+0000 7ff0bb400700 1 --2- 172.21.15.33:0/3001833256 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0bc104100 0x7ff0bc1a1df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:02.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.271+0000 7ff0bb400700 1 -- 172.21.15.33:0/3001833256 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0bc19c1d0 con 0x7ff0bc108850 2024-05-03T18:25:02.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.271+0000 7ff0baa00700 1 --2- 172.21.15.33:0/3001833256 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0bc103750 0x7ff0bc1a18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:25:02.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.271+0000 7ff0bb400700 1 --2- 172.21.15.33:0/3001833256 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0bc108850 0x7ff0bc19ba70 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff0b4007f10 tx=0x7ff0b400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:25:02.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.271+0000 7ff0c0a00700 1 -- 172.21.15.33:0/3001833256 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0b400ddd0 con 0x7ff0bc108850 2024-05-03T18:25:02.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.271+0000 7ff0c0a00700 1 -- 172.21.15.33:0/3001833256 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff0b4009e20 con 0x7ff0bc108850 2024-05-03T18:25:02.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.271+0000 7ff0c1e00700 1 -- 172.21.15.33:0/3001833256 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0bc19c4c0 con 0x7ff0bc108850 2024-05-03T18:25:02.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.271+0000 7ff0c0a00700 1 -- 172.21.15.33:0/3001833256 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0b4015400 con 0x7ff0bc108850 2024-05-03T18:25:02.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.272+0000 7ff0c1e00700 1 -- 172.21.15.33:0/3001833256 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0bc1a8780 con 0x7ff0bc108850 2024-05-03T18:25:02.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.273+0000 7ff0c1e00700 1 -- 172.21.15.33:0/3001833256 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff0bc102090 con 0x7ff0bc108850 2024-05-03T18:25:02.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.275+0000 7ff0c0a00700 1 -- 172.21.15.33:0/3001833256 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7ff0b4005000 con 0x7ff0bc108850 2024-05-03T18:25:02.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.278+0000 7ff0c0a00700 1 --2- 172.21.15.33:0/3001833256 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7ff0a00b2910 0x7ff0a00b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:02.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.278+0000 7ff0c0a00700 1 -- 172.21.15.33:0/3001833256 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7ff0b40e2c60 con 0x7ff0bc108850 2024-05-03T18:25:02.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.278+0000 7ff0baa00700 1 --2- 172.21.15.33:0/3001833256 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7ff0a00b2910 0x7ff0a00b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:02.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.278+0000 7ff0c0a00700 1 --2- 172.21.15.33:0/3001833256 >> [v2:172.21.15.33:6810/3814942724,v1:172.21.15.33:6811/3814942724] conn(0x7ff0a00bdaf0 0x7ff0a00bff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:02.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.278+0000 7ff0c0a00700 1 -- 172.21.15.33:0/3001833256 --> [v2:172.21.15.33:6810/3814942724,v1:172.21.15.33:6811/3814942724] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff0b400be50 con 0x7ff0a00bdaf0 2024-05-03T18:25:02.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.278+0000 7ff0c0a00700 1 -- 172.21.15.33:0/3001833256 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7ff0b40e30b0 con 0x7ff0bc108850 2024-05-03T18:25:02.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.279+0000 7ff0ba000700 1 --2- 172.21.15.33:0/3001833256 >> [v2:172.21.15.33:6810/3814942724,v1:172.21.15.33:6811/3814942724] conn(0x7ff0a00bdaf0 0x7ff0a00bff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:02.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.279+0000 7ff0baa00700 1 --2- 172.21.15.33:0/3001833256 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7ff0a00b2910 0x7ff0a00b4dd0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac005950 tx=0x7ff0ac0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:25:02.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.279+0000 7ff0ba000700 1 --2- 172.21.15.33:0/3001833256 >> [v2:172.21.15.33:6810/3814942724,v1:172.21.15.33:6811/3814942724] conn(0x7ff0a00bdaf0 0x7ff0a00bff90 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:25:02.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.281+0000 7ff0c0a00700 1 -- 172.21.15.33:0/3001833256 <== osd.3 v2:172.21.15.33:6810/3814942724 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff0b400be50 con 0x7ff0a00bdaf0 2024-05-03T18:25:02.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.303+0000 7ff0c1e00700 1 -- 172.21.15.33:0/3001833256 --> [v2:172.21.15.33:6810/3814942724,v1:172.21.15.33:6811/3814942724] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff0bc104be0 con 0x7ff0a00bdaf0 2024-05-03T18:25:02.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.304+0000 7ff0c0a00700 1 -- 172.21.15.33:0/3001833256 <== osd.3 v2:172.21.15.33:6810/3814942724 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ff0bc104be0 con 0x7ff0a00bdaf0 2024-05-03T18:25:02.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.305+0000 7ff0c1e00700 1 -- 172.21.15.33:0/3001833256 >> [v2:172.21.15.33:6810/3814942724,v1:172.21.15.33:6811/3814942724] conn(0x7ff0a00bdaf0 msgr2=0x7ff0a00bff90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:02.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.305+0000 7ff0c1e00700 1 --2- 172.21.15.33:0/3001833256 >> [v2:172.21.15.33:6810/3814942724,v1:172.21.15.33:6811/3814942724] conn(0x7ff0a00bdaf0 0x7ff0a00bff90 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:02.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.305+0000 7ff0c1e00700 1 -- 172.21.15.33:0/3001833256 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7ff0a00b2910 msgr2=0x7ff0a00b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:02.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.305+0000 7ff0c1e00700 1 --2- 172.21.15.33:0/3001833256 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7ff0a00b2910 0x7ff0a00b4dd0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac005950 tx=0x7ff0ac0058e0 comp rx=0 tx=0).stop 2024-05-03T18:25:02.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.305+0000 7ff0c1e00700 1 -- 172.21.15.33:0/3001833256 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0bc108850 msgr2=0x7ff0bc19ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:02.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.305+0000 7ff0c1e00700 1 --2- 172.21.15.33:0/3001833256 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0bc108850 0x7ff0bc19ba70 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff0b4007f10 tx=0x7ff0b400d3b0 comp rx=0 tx=0).stop 2024-05-03T18:25:02.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.305+0000 7ff0bb400700 1 -- 172.21.15.33:0/3001833256 reap_dead start 2024-05-03T18:25:02.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.305+0000 7ff0c1e00700 1 -- 172.21.15.33:0/3001833256 shutdown_connections 2024-05-03T18:25:02.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.305+0000 7ff0c1e00700 1 -- 172.21.15.33:0/3001833256 >> 172.21.15.33:0/3001833256 conn(0x7ff0bc07a600 msgr2=0x7ff0bc101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:25:02.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.305+0000 7ff0c1e00700 1 -- 172.21.15.33:0/3001833256 shutdown_connections 2024-05-03T18:25:02.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.305+0000 7ff0c1e00700 1 -- 172.21.15.33:0/3001833256 wait complete. 2024-05-03T18:25:02.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.578+0000 7f0b7b200700 1 -- 172.21.15.33:0/4158001544 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b7c1033d0 msgr2=0x7f0b7c103850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:02.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.578+0000 7f0b7b200700 1 --2- 172.21.15.33:0/4158001544 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b7c1033d0 0x7f0b7c103850 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f0b68009990 tx=0x7f0b68009ca0 comp rx=0 tx=0).stop 2024-05-03T18:25:02.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.578+0000 7f0b7b200700 1 -- 172.21.15.33:0/4158001544 shutdown_connections 2024-05-03T18:25:02.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.578+0000 7f0b7b200700 1 --2- 172.21.15.33:0/4158001544 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b7c103d90 0x7f0b7c1102a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:02.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.578+0000 7f0b7b200700 1 --2- 172.21.15.33:0/4158001544 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0b7c109460 0x7f0b7c109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:02.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.578+0000 7f0b7b200700 1 --2- 172.21.15.33:0/4158001544 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b7c1033d0 0x7f0b7c103850 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:02.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.578+0000 7f0b7b200700 1 -- 172.21.15.33:0/4158001544 >> 172.21.15.33:0/4158001544 conn(0x7f0b7c0fef20 msgr2=0x7f0b7c101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:25:02.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.578+0000 7f0b7b200700 1 -- 172.21.15.33:0/4158001544 shutdown_connections 2024-05-03T18:25:02.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.579+0000 7f0b7b200700 1 -- 172.21.15.33:0/4158001544 wait complete. 2024-05-03T18:25:02.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.579+0000 7f0b7b200700 1 Processor -- start 2024-05-03T18:25:02.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.580+0000 7f0b7b200700 1 -- start start 2024-05-03T18:25:02.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.580+0000 7f0b7b200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b7c1033d0 0x7f0b7c1a1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:02.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.580+0000 7f0b7b200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b7c103d90 0x7f0b7c1a1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:02.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.580+0000 7f0b7b200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0b7c109460 0x7f0b7c19b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:02.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.580+0000 7f0b7b200700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0b7c113420 con 0x7f0b7c1033d0 2024-05-03T18:25:02.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.580+0000 7f0b7b200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0b7c1132a0 con 0x7f0b7c103d90 2024-05-03T18:25:02.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.580+0000 7f0b7b200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f0b7c1135a0 con 0x7f0b7c109460 2024-05-03T18:25:02.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.580+0000 7f0b79e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b7c1033d0 0x7f0b7c1a1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:02.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.580+0000 7f0b79e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b7c1033d0 0x7f0b7c1a1800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54056/0 (socket says 172.21.15.33:54056) 2024-05-03T18:25:02.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.580+0000 7f0b79e00700 1 -- 172.21.15.33:0/637207912 learned_addr learned my addr 172.21.15.33:0/637207912 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:25:02.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.581+0000 7f0b79400700 1 --2- 172.21.15.33:0/637207912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b7c103d90 0x7f0b7c1a1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:02.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.581+0000 7f0b7a800700 1 --2- 172.21.15.33:0/637207912 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0b7c109460 0x7f0b7c19b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:02.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.581+0000 7f0b79e00700 1 -- 172.21.15.33:0/637207912 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0b7c109460 msgr2=0x7f0b7c19b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:02.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.581+0000 7f0b79e00700 1 --2- 172.21.15.33:0/637207912 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0b7c109460 0x7f0b7c19b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:02.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.581+0000 7f0b79e00700 1 -- 172.21.15.33:0/637207912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b7c103d90 msgr2=0x7f0b7c1a1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:02.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.581+0000 7f0b79e00700 1 --2- 172.21.15.33:0/637207912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b7c103d90 0x7f0b7c1a1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:02.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.581+0000 7f0b79e00700 1 -- 172.21.15.33:0/637207912 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b7c19c120 con 0x7f0b7c1033d0 2024-05-03T18:25:02.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.581+0000 7f0b79e00700 1 --2- 172.21.15.33:0/637207912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b7c1033d0 0x7f0b7c1a1800 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c00c920 tx=0x7f0b6c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:25:02.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.582+0000 7f0b72a00700 1 -- 172.21.15.33:0/637207912 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b6c007b00 con 0x7f0b7c1033d0 2024-05-03T18:25:02.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.582+0000 7f0b72a00700 1 -- 172.21.15.33:0/637207912 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0b6c004cc0 con 0x7f0b7c1033d0 2024-05-03T18:25:02.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.582+0000 7f0b7b200700 1 -- 172.21.15.33:0/637207912 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b7c19c410 con 0x7f0b7c1033d0 2024-05-03T18:25:02.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.582+0000 7f0b72a00700 1 -- 172.21.15.33:0/637207912 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b6c0054f0 con 0x7f0b7c1033d0 2024-05-03T18:25:02.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.582+0000 7f0b7b200700 1 -- 172.21.15.33:0/637207912 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b7c1a86d0 con 0x7f0b7c1033d0 2024-05-03T18:25:02.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.583+0000 7f0b7b200700 1 -- 172.21.15.33:0/637207912 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0b7c1007c0 con 0x7f0b7c1033d0 2024-05-03T18:25:02.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.585+0000 7f0b72a00700 1 -- 172.21.15.33:0/637207912 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f0b6c007ca0 con 0x7f0b7c1033d0 2024-05-03T18:25:02.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.587+0000 7f0b72a00700 1 --2- 172.21.15.33:0/637207912 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f0b600b2910 0x7f0b600b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:02.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.587+0000 7f0b72a00700 1 -- 172.21.15.33:0/637207912 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f0b6c0e2770 con 0x7f0b7c1033d0 2024-05-03T18:25:02.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.587+0000 7f0b79400700 1 --2- 172.21.15.33:0/637207912 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f0b600b2910 0x7f0b600b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:02.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.587+0000 7f0b72a00700 1 --2- 172.21.15.33:0/637207912 >> [v2:172.21.15.99:6824/935746136,v1:172.21.15.99:6825/935746136] conn(0x7f0b600bdaf0 0x7f0b600bff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:02.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.587+0000 7f0b72a00700 1 -- 172.21.15.33:0/637207912 --> [v2:172.21.15.99:6824/935746136,v1:172.21.15.99:6825/935746136] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0b600c0660 con 0x7f0b600bdaf0 2024-05-03T18:25:02.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.587+0000 7f0b72a00700 1 -- 172.21.15.33:0/637207912 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f0b6c0e2b20 con 0x7f0b7c1033d0 2024-05-03T18:25:02.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.588+0000 7f0b79400700 1 --2- 172.21.15.33:0/637207912 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f0b600b2910 0x7f0b600b4dd0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0b68005850 tx=0x7f0b6801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:25:02.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.588+0000 7f0b7a800700 1 --2- 172.21.15.33:0/637207912 >> [v2:172.21.15.99:6824/935746136,v1:172.21.15.99:6825/935746136] conn(0x7f0b600bdaf0 0x7f0b600bff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:02.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.589+0000 7f0b7a800700 1 --2- 172.21.15.33:0/637207912 >> [v2:172.21.15.99:6824/935746136,v1:172.21.15.99:6825/935746136] conn(0x7f0b600bdaf0 0x7f0b600bff90 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:25:02.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.592+0000 7f0b72a00700 1 -- 172.21.15.33:0/637207912 <== osd.10 v2:172.21.15.99:6824/935746136 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0b600c0660 con 0x7f0b600bdaf0 2024-05-03T18:25:02.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.611+0000 7f0b7b200700 1 -- 172.21.15.33:0/637207912 --> [v2:172.21.15.99:6824/935746136,v1:172.21.15.99:6825/935746136] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0b7c104b10 con 0x7f0b600bdaf0 2024-05-03T18:25:02.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.612+0000 7f0b72a00700 1 -- 172.21.15.33:0/637207912 <== osd.10 v2:172.21.15.99:6824/935746136 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0b7c104b10 con 0x7f0b600bdaf0 2024-05-03T18:25:02.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.612+0000 7f0b7b200700 1 -- 172.21.15.33:0/637207912 >> [v2:172.21.15.99:6824/935746136,v1:172.21.15.99:6825/935746136] conn(0x7f0b600bdaf0 msgr2=0x7f0b600bff90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:02.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.612+0000 7f0b7b200700 1 --2- 172.21.15.33:0/637207912 >> [v2:172.21.15.99:6824/935746136,v1:172.21.15.99:6825/935746136] conn(0x7f0b600bdaf0 0x7f0b600bff90 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:02.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.612+0000 7f0b7b200700 1 -- 172.21.15.33:0/637207912 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f0b600b2910 msgr2=0x7f0b600b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:02.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.612+0000 7f0b7b200700 1 --2- 172.21.15.33:0/637207912 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f0b600b2910 0x7f0b600b4dd0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0b68005850 tx=0x7f0b6801a040 comp rx=0 tx=0).stop 2024-05-03T18:25:02.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.612+0000 7f0b7b200700 1 -- 172.21.15.33:0/637207912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b7c1033d0 msgr2=0x7f0b7c1a1800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:02.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.612+0000 7f0b7b200700 1 --2- 172.21.15.33:0/637207912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b7c1033d0 0x7f0b7c1a1800 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c00c920 tx=0x7f0b6c00cce0 comp rx=0 tx=0).stop 2024-05-03T18:25:02.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.612+0000 7f0b7a800700 1 -- 172.21.15.33:0/637207912 reap_dead start 2024-05-03T18:25:02.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.613+0000 7f0b7b200700 1 -- 172.21.15.33:0/637207912 shutdown_connections 2024-05-03T18:25:02.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.613+0000 7f0b7b200700 1 -- 172.21.15.33:0/637207912 >> 172.21.15.33:0/637207912 conn(0x7f0b7c0fef20 msgr2=0x7f0b7c10eba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:25:02.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.613+0000 7f0b7b200700 1 -- 172.21.15.33:0/637207912 shutdown_connections 2024-05-03T18:25:02.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:01.613+0000 7f0b7b200700 1 -- 172.21.15.33:0/637207912 wait complete. 2024-05-03T18:25:02.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:25:02 smithi150 ceph-mon[32492]: pgmap v460: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:25:02.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:25:02 smithi033 ceph-mon[28330]: pgmap v460: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:25:03.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:25:02 smithi099 ceph-mon[31658]: pgmap v460: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:25:03.269 INFO:teuthology.orchestra.run.smithi033.stdout:55834574971 2024-05-03T18:25:03.270 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-05-03T18:25:04.659 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:25:04 smithi033 ceph-mon[28330]: pgmap v461: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:25:04.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:04.546+0000 7eff8be00700 1 -- 172.21.15.33:0/1858333972 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff8c104610 msgr2=0x7eff8c1049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:04.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:04.546+0000 7eff8be00700 1 --2- 172.21.15.33:0/1858333972 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff8c104610 0x7eff8c1049f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7eff7c009990 tx=0x7eff7c009ca0 comp rx=0 tx=0).stop 2024-05-03T18:25:04.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:04.546+0000 7eff8be00700 1 -- 172.21.15.33:0/1858333972 shutdown_connections 2024-05-03T18:25:04.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:04.546+0000 7eff8be00700 1 --2- 172.21.15.33:0/1858333972 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7eff8c10d9f0 0x7eff8c10fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:04.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:04.546+0000 7eff8be00700 1 --2- 172.21.15.33:0/1858333972 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff8c104610 0x7eff8c1049f0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:04.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:04.546+0000 7eff8be00700 1 --2- 172.21.15.33:0/1858333972 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff8c104f30 0x7eff8c10d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:04.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:04.546+0000 7eff8be00700 1 -- 172.21.15.33:0/1858333972 >> 172.21.15.33:0/1858333972 conn(0x7eff8c0fddc0 msgr2=0x7eff8c1001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:25:04.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:04.546+0000 7eff8be00700 1 -- 172.21.15.33:0/1858333972 shutdown_connections 2024-05-03T18:25:04.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:04.547+0000 7eff8be00700 1 -- 172.21.15.33:0/1858333972 wait complete. 2024-05-03T18:25:04.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:04.548+0000 7eff8be00700 1 Processor -- start 2024-05-03T18:25:04.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:04.548+0000 7eff8be00700 1 -- start start 2024-05-03T18:25:04.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:04.548+0000 7eff8be00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7eff8c104f30 0x7eff8c19f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:04.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:04.548+0000 7eff8be00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff8c10d9f0 0x7eff8c19fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:04.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:04.548+0000 7eff8be00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff8c199ad0 0x7eff8c199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:04.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:04.548+0000 7eff8be00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7eff8c1134d0 con 0x7eff8c199ad0 2024-05-03T18:25:04.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:04.548+0000 7eff8be00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7eff8c113350 con 0x7eff8c10d9f0 2024-05-03T18:25:04.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:04.548+0000 7eff8be00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7eff8c113650 con 0x7eff8c104f30 2024-05-03T18:25:04.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:04.549+0000 7eff8b400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff8c199ad0 0x7eff8c199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:04.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:04.549+0000 7eff8b400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff8c199ad0 0x7eff8c199f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54078/0 (socket says 172.21.15.33:54078) 2024-05-03T18:25:04.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:04.549+0000 7eff8b400700 1 -- 172.21.15.33:0/745987102 learned_addr learned my addr 172.21.15.33:0/745987102 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:25:04.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:04.549+0000 7eff8aa00700 1 --2- 172.21.15.33:0/745987102 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7eff8c104f30 0x7eff8c19f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:04.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:04.549+0000 7eff8a000700 1 --2- 172.21.15.33:0/745987102 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff8c10d9f0 0x7eff8c19fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:04.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:04.549+0000 7eff8b400700 1 -- 172.21.15.33:0/745987102 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7eff8c104f30 msgr2=0x7eff8c19f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:04.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:04.549+0000 7eff8b400700 1 --2- 172.21.15.33:0/745987102 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7eff8c104f30 0x7eff8c19f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:04.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:04.549+0000 7eff8b400700 1 -- 172.21.15.33:0/745987102 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff8c10d9f0 msgr2=0x7eff8c19fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:04.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:04.549+0000 7eff8b400700 1 --2- 172.21.15.33:0/745987102 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff8c10d9f0 0x7eff8c19fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:04.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:04.549+0000 7eff8b400700 1 -- 172.21.15.33:0/745987102 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff8c19a650 con 0x7eff8c199ad0 2024-05-03T18:25:04.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:04.550+0000 7eff8b400700 1 --2- 172.21.15.33:0/745987102 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff8c199ad0 0x7eff8c199f50 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7eff84007ee0 tx=0x7eff8400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:25:04.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:04.550+0000 7eff83400700 1 -- 172.21.15.33:0/745987102 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff8400dbd0 con 0x7eff8c199ad0 2024-05-03T18:25:04.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:04.550+0000 7eff8be00700 1 -- 172.21.15.33:0/745987102 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff8c1a67e0 con 0x7eff8c199ad0 2024-05-03T18:25:04.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:04.550+0000 7eff83400700 1 -- 172.21.15.33:0/745987102 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7eff8400dd70 con 0x7eff8c199ad0 2024-05-03T18:25:04.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:04.550+0000 7eff83400700 1 -- 172.21.15.33:0/745987102 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff840058d0 con 0x7eff8c199ad0 2024-05-03T18:25:04.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:04.551+0000 7eff8be00700 1 -- 172.21.15.33:0/745987102 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff8c102370 con 0x7eff8c199ad0 2024-05-03T18:25:04.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:04.552+0000 7eff8be00700 1 -- 172.21.15.33:0/745987102 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7eff8c063bf0 con 0x7eff8c199ad0 2024-05-03T18:25:04.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:04.556+0000 7eff83400700 1 -- 172.21.15.33:0/745987102 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7eff8401a020 con 0x7eff8c199ad0 2024-05-03T18:25:04.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:04.558+0000 7eff83400700 1 --2- 172.21.15.33:0/745987102 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7eff700b2910 0x7eff700b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:04.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:04.558+0000 7eff83400700 1 -- 172.21.15.33:0/745987102 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7eff840e1e80 con 0x7eff8c199ad0 2024-05-03T18:25:04.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:04.558+0000 7eff8aa00700 1 --2- 172.21.15.33:0/745987102 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7eff700b2910 0x7eff700b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:04.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:04.559+0000 7eff83400700 1 --2- 172.21.15.33:0/745987102 >> [v2:172.21.15.150:6800/1995659622,v1:172.21.15.150:6801/1995659622] conn(0x7eff700bda70 0x7eff700bff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:04.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:04.559+0000 7eff83400700 1 -- 172.21.15.33:0/745987102 --> [v2:172.21.15.150:6800/1995659622,v1:172.21.15.150:6801/1995659622] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7eff700c05e0 con 0x7eff700bda70 2024-05-03T18:25:04.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:04.559+0000 7eff8aa00700 1 --2- 172.21.15.33:0/745987102 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7eff700b2910 0x7eff700b4dd0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7eff7c000c00 tx=0x7eff7c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:25:04.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:04.559+0000 7eff8a000700 1 --2- 172.21.15.33:0/745987102 >> [v2:172.21.15.150:6800/1995659622,v1:172.21.15.150:6801/1995659622] conn(0x7eff700bda70 0x7eff700bff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:04.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:04.560+0000 7eff83400700 1 -- 172.21.15.33:0/745987102 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7eff840a4e20 con 0x7eff8c199ad0 2024-05-03T18:25:04.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:04.561+0000 7eff8a000700 1 --2- 172.21.15.33:0/745987102 >> [v2:172.21.15.150:6800/1995659622,v1:172.21.15.150:6801/1995659622] conn(0x7eff700bda70 0x7eff700bff10 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:25:04.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:04.563+0000 7eff83400700 1 -- 172.21.15.33:0/745987102 <== osd.2 v2:172.21.15.150:6800/1995659622 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7eff700c05e0 con 0x7eff700bda70 2024-05-03T18:25:04.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:04.583+0000 7eff8be00700 1 -- 172.21.15.33:0/745987102 --> [v2:172.21.15.150:6800/1995659622,v1:172.21.15.150:6801/1995659622] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7eff8c064070 con 0x7eff700bda70 2024-05-03T18:25:04.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:04.584+0000 7eff83400700 1 -- 172.21.15.33:0/745987102 <== osd.2 v2:172.21.15.150:6800/1995659622 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7eff8c064070 con 0x7eff700bda70 2024-05-03T18:25:04.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:04.584+0000 7eff8be00700 1 -- 172.21.15.33:0/745987102 >> [v2:172.21.15.150:6800/1995659622,v1:172.21.15.150:6801/1995659622] conn(0x7eff700bda70 msgr2=0x7eff700bff10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:04.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:04.584+0000 7eff8be00700 1 --2- 172.21.15.33:0/745987102 >> [v2:172.21.15.150:6800/1995659622,v1:172.21.15.150:6801/1995659622] conn(0x7eff700bda70 0x7eff700bff10 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:04.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:04.584+0000 7eff8be00700 1 -- 172.21.15.33:0/745987102 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7eff700b2910 msgr2=0x7eff700b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:04.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:04.584+0000 7eff8be00700 1 --2- 172.21.15.33:0/745987102 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7eff700b2910 0x7eff700b4dd0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7eff7c000c00 tx=0x7eff7c005fd0 comp rx=0 tx=0).stop 2024-05-03T18:25:04.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:04.584+0000 7eff8be00700 1 -- 172.21.15.33:0/745987102 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff8c199ad0 msgr2=0x7eff8c199f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:04.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:04.584+0000 7eff8be00700 1 --2- 172.21.15.33:0/745987102 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff8c199ad0 0x7eff8c199f50 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7eff84007ee0 tx=0x7eff8400d3b0 comp rx=0 tx=0).stop 2024-05-03T18:25:04.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:04.584+0000 7eff8b400700 1 -- 172.21.15.33:0/745987102 reap_dead start 2024-05-03T18:25:04.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:04.585+0000 7eff8be00700 1 -- 172.21.15.33:0/745987102 shutdown_connections 2024-05-03T18:25:04.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:04.585+0000 7eff8be00700 1 -- 172.21.15.33:0/745987102 >> 172.21.15.33:0/745987102 conn(0x7eff8c0fddc0 msgr2=0x7eff8c0fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:25:04.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:04.585+0000 7eff8be00700 1 -- 172.21.15.33:0/745987102 shutdown_connections 2024-05-03T18:25:04.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:04.585+0000 7eff8be00700 1 -- 172.21.15.33:0/745987102 wait complete. 2024-05-03T18:25:04.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:25:04 smithi150 ceph-mon[32492]: pgmap v461: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:25:05.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:25:04 smithi099 ceph-mon[31658]: pgmap v461: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:25:05.614 INFO:teuthology.orchestra.run.smithi033.stdout:274877907002 2024-05-03T18:25:05.614 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-05-03T18:25:05.948 INFO:teuthology.orchestra.run.smithi033.stdout:77309411445 2024-05-03T18:25:05.949 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-05-03T18:25:06.802 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:25:06 smithi033 ceph-mon[28330]: pgmap v462: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:25:06.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:25:06 smithi150 ceph-mon[32492]: pgmap v462: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:25:07.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:25:06 smithi099 ceph-mon[31658]: pgmap v462: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:25:07.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.034+0000 7f5271800700 1 -- 172.21.15.33:0/1082780484 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f526c10ff40 msgr2=0x7f526c112330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:07.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.034+0000 7f5271800700 1 --2- 172.21.15.33:0/1082780484 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f526c10ff40 0x7f526c112330 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f526400b0a0 tx=0x7f526400b3b0 comp rx=0 tx=0).stop 2024-05-03T18:25:07.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.034+0000 7f5271800700 1 -- 172.21.15.33:0/1082780484 shutdown_connections 2024-05-03T18:25:07.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.034+0000 7f5271800700 1 --2- 172.21.15.33:0/1082780484 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f526c10ff40 0x7f526c112330 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:07.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.034+0000 7f5271800700 1 --2- 172.21.15.33:0/1082780484 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f526c06ba40 0x7f526c06be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:07.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.034+0000 7f5271800700 1 --2- 172.21.15.33:0/1082780484 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f526c06c360 0x7f526c10fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:07.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.034+0000 7f5271800700 1 -- 172.21.15.33:0/1082780484 >> 172.21.15.33:0/1082780484 conn(0x7f526c079310 msgr2=0x7f526c079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:25:07.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.034+0000 7f5271800700 1 -- 172.21.15.33:0/1082780484 shutdown_connections 2024-05-03T18:25:07.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.034+0000 7f5271800700 1 -- 172.21.15.33:0/1082780484 wait complete. 2024-05-03T18:25:07.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.035+0000 7f5271800700 1 Processor -- start 2024-05-03T18:25:07.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.036+0000 7f5271800700 1 -- start start 2024-05-03T18:25:07.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.036+0000 7f5271800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f526c06ba40 0x7f526c1a18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:07.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.036+0000 7f5271800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f526c06c360 0x7f526c1a1e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:07.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.036+0000 7f5271800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f526c10ff40 0x7f526c19ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:07.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.036+0000 7f5271800700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f526c115700 con 0x7f526c06ba40 2024-05-03T18:25:07.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.036+0000 7f5271800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f526c115580 con 0x7f526c06c360 2024-05-03T18:25:07.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.036+0000 7f5271800700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f526c115880 con 0x7f526c10ff40 2024-05-03T18:25:07.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.037+0000 7f526aa00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f526c06ba40 0x7f526c1a18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:07.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.037+0000 7f526aa00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f526c06ba40 0x7f526c1a18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58806/0 (socket says 172.21.15.33:58806) 2024-05-03T18:25:07.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.037+0000 7f526aa00700 1 -- 172.21.15.33:0/3605337555 learned_addr learned my addr 172.21.15.33:0/3605337555 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:25:07.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.037+0000 7f526a000700 1 --2- 172.21.15.33:0/3605337555 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f526c06c360 0x7f526c1a1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:07.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.037+0000 7f526b400700 1 --2- 172.21.15.33:0/3605337555 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f526c10ff40 0x7f526c19ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:07.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.037+0000 7f526aa00700 1 -- 172.21.15.33:0/3605337555 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f526c10ff40 msgr2=0x7f526c19ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:07.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.037+0000 7f526aa00700 1 --2- 172.21.15.33:0/3605337555 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f526c10ff40 0x7f526c19ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:07.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.037+0000 7f526aa00700 1 -- 172.21.15.33:0/3605337555 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f526c06c360 msgr2=0x7f526c1a1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:07.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.037+0000 7f526aa00700 1 --2- 172.21.15.33:0/3605337555 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f526c06c360 0x7f526c1a1e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:07.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.037+0000 7f526aa00700 1 -- 172.21.15.33:0/3605337555 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f526c19c1e0 con 0x7f526c06ba40 2024-05-03T18:25:07.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.038+0000 7f526aa00700 1 --2- 172.21.15.33:0/3605337555 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f526c06ba40 0x7f526c1a18c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f525c00e9a0 tx=0x7f525c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:25:07.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.038+0000 7f5263400700 1 -- 172.21.15.33:0/3605337555 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f525c00cbd0 con 0x7f526c06ba40 2024-05-03T18:25:07.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.038+0000 7f5263400700 1 -- 172.21.15.33:0/3605337555 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f525c00cd70 con 0x7f526c06ba40 2024-05-03T18:25:07.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.038+0000 7f5271800700 1 -- 172.21.15.33:0/3605337555 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f526c19c4d0 con 0x7f526c06ba40 2024-05-03T18:25:07.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.038+0000 7f5263400700 1 -- 172.21.15.33:0/3605337555 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f525c018680 con 0x7f526c06ba40 2024-05-03T18:25:07.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.038+0000 7f5271800700 1 -- 172.21.15.33:0/3605337555 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f526c1a8790 con 0x7f526c06ba40 2024-05-03T18:25:07.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.040+0000 7f5271800700 1 -- 172.21.15.33:0/3605337555 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f526c063c30 con 0x7f526c06ba40 2024-05-03T18:25:07.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.042+0000 7f5263400700 1 -- 172.21.15.33:0/3605337555 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f525c010950 con 0x7f526c06ba40 2024-05-03T18:25:07.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.044+0000 7f5263400700 1 --2- 172.21.15.33:0/3605337555 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f52500b2910 0x7f52500b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:07.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.044+0000 7f5263400700 1 -- 172.21.15.33:0/3605337555 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f525c0e2e90 con 0x7f526c06ba40 2024-05-03T18:25:07.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.044+0000 7f526a000700 1 --2- 172.21.15.33:0/3605337555 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f52500b2910 0x7f52500b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:07.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.044+0000 7f5263400700 1 --2- 172.21.15.33:0/3605337555 >> [v2:172.21.15.150:6816/1308083602,v1:172.21.15.150:6817/1308083602] conn(0x7f52500bda70 0x7f52500bff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:07.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.044+0000 7f5263400700 1 -- 172.21.15.33:0/3605337555 --> [v2:172.21.15.150:6816/1308083602,v1:172.21.15.150:6817/1308083602] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f525c024e70 con 0x7f52500bda70 2024-05-03T18:25:07.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.044+0000 7f5263400700 1 -- 172.21.15.33:0/3605337555 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f525c0e32e0 con 0x7f526c06ba40 2024-05-03T18:25:07.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.045+0000 7f526b400700 1 --2- 172.21.15.33:0/3605337555 >> [v2:172.21.15.150:6816/1308083602,v1:172.21.15.150:6817/1308083602] conn(0x7f52500bda70 0x7f52500bff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:07.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.045+0000 7f526a000700 1 --2- 172.21.15.33:0/3605337555 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f52500b2910 0x7f52500b4dd0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f526c101f50 tx=0x7f525800b590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:25:07.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.046+0000 7f526b400700 1 --2- 172.21.15.33:0/3605337555 >> [v2:172.21.15.150:6816/1308083602,v1:172.21.15.150:6817/1308083602] conn(0x7f52500bda70 0x7f52500bff10 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:25:07.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.048+0000 7f5263400700 1 -- 172.21.15.33:0/3605337555 <== osd.8 v2:172.21.15.150:6816/1308083602 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f525c024e70 con 0x7f52500bda70 2024-05-03T18:25:07.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.082+0000 7f5271800700 1 -- 172.21.15.33:0/3605337555 --> [v2:172.21.15.150:6816/1308083602,v1:172.21.15.150:6817/1308083602] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f526c0640b0 con 0x7f52500bda70 2024-05-03T18:25:07.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.083+0000 7f5263400700 1 -- 172.21.15.33:0/3605337555 <== osd.8 v2:172.21.15.150:6816/1308083602 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f526c0640b0 con 0x7f52500bda70 2024-05-03T18:25:07.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.083+0000 7f5271800700 1 -- 172.21.15.33:0/3605337555 >> [v2:172.21.15.150:6816/1308083602,v1:172.21.15.150:6817/1308083602] conn(0x7f52500bda70 msgr2=0x7f52500bff10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:07.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.083+0000 7f5271800700 1 --2- 172.21.15.33:0/3605337555 >> [v2:172.21.15.150:6816/1308083602,v1:172.21.15.150:6817/1308083602] conn(0x7f52500bda70 0x7f52500bff10 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:07.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.084+0000 7f5271800700 1 -- 172.21.15.33:0/3605337555 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f52500b2910 msgr2=0x7f52500b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:07.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.084+0000 7f5271800700 1 --2- 172.21.15.33:0/3605337555 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f52500b2910 0x7f52500b4dd0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f526c101f50 tx=0x7f525800b590 comp rx=0 tx=0).stop 2024-05-03T18:25:07.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.084+0000 7f5271800700 1 -- 172.21.15.33:0/3605337555 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f526c06ba40 msgr2=0x7f526c1a18c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:07.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.084+0000 7f5271800700 1 --2- 172.21.15.33:0/3605337555 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f526c06ba40 0x7f526c1a18c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f525c00e9a0 tx=0x7f525c00ed60 comp rx=0 tx=0).stop 2024-05-03T18:25:07.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.084+0000 7f526b400700 1 -- 172.21.15.33:0/3605337555 reap_dead start 2024-05-03T18:25:07.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.084+0000 7f5271800700 1 -- 172.21.15.33:0/3605337555 shutdown_connections 2024-05-03T18:25:07.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.084+0000 7f5271800700 1 -- 172.21.15.33:0/3605337555 >> 172.21.15.33:0/3605337555 conn(0x7f526c079310 msgr2=0x7f526c111510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:25:07.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.084+0000 7f5271800700 1 -- 172.21.15.33:0/3605337555 shutdown_connections 2024-05-03T18:25:07.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.084+0000 7f5271800700 1 -- 172.21.15.33:0/3605337555 wait complete. 2024-05-03T18:25:07.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.159+0000 7fab60c00700 1 -- 172.21.15.33:0/4061509141 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab5c104100 msgr2=0x7fab5c108120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:07.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.159+0000 7fab60c00700 1 --2- 172.21.15.33:0/4061509141 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab5c104100 0x7fab5c108120 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fab480099c0 tx=0x7fab48009cd0 comp rx=0 tx=0).stop 2024-05-03T18:25:07.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.159+0000 7fab60c00700 1 -- 172.21.15.33:0/4061509141 shutdown_connections 2024-05-03T18:25:07.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.159+0000 7fab60c00700 1 --2- 172.21.15.33:0/4061509141 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab5c108850 0x7fab5c1103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:07.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.159+0000 7fab60c00700 1 --2- 172.21.15.33:0/4061509141 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab5c103750 0x7fab5c103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:07.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.159+0000 7fab60c00700 1 --2- 172.21.15.33:0/4061509141 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab5c104100 0x7fab5c108120 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:07.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.159+0000 7fab60c00700 1 -- 172.21.15.33:0/4061509141 >> 172.21.15.33:0/4061509141 conn(0x7fab5c07a600 msgr2=0x7fab5c07aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:25:07.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.159+0000 7fab60c00700 1 -- 172.21.15.33:0/4061509141 shutdown_connections 2024-05-03T18:25:07.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.159+0000 7fab60c00700 1 -- 172.21.15.33:0/4061509141 wait complete. 2024-05-03T18:25:07.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.160+0000 7fab60c00700 1 Processor -- start 2024-05-03T18:25:07.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.161+0000 7fab60c00700 1 -- start start 2024-05-03T18:25:07.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.161+0000 7fab60c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab5c103750 0x7fab5c1a18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:07.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.161+0000 7fab60c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab5c104100 0x7fab5c1a1df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:07.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.161+0000 7fab60c00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab5c108850 0x7fab5c19ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:07.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.161+0000 7fab60c00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fab5c113530 con 0x7fab5c108850 2024-05-03T18:25:07.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.161+0000 7fab60c00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fab5c1133b0 con 0x7fab5c103750 2024-05-03T18:25:07.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.161+0000 7fab60c00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fab5c1136b0 con 0x7fab5c104100 2024-05-03T18:25:07.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.161+0000 7fab5a800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab5c108850 0x7fab5c19ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:07.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.161+0000 7fab5a800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab5c108850 0x7fab5c19ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58820/0 (socket says 172.21.15.33:58820) 2024-05-03T18:25:07.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.161+0000 7fab5a800700 1 -- 172.21.15.33:0/2278386418 learned_addr learned my addr 172.21.15.33:0/2278386418 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:25:07.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.162+0000 7fab59e00700 1 --2- 172.21.15.33:0/2278386418 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab5c103750 0x7fab5c1a18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:07.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.162+0000 7fab59400700 1 --2- 172.21.15.33:0/2278386418 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab5c104100 0x7fab5c1a1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:07.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.162+0000 7fab5a800700 1 -- 172.21.15.33:0/2278386418 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab5c104100 msgr2=0x7fab5c1a1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:07.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.162+0000 7fab5a800700 1 --2- 172.21.15.33:0/2278386418 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab5c104100 0x7fab5c1a1df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:07.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.162+0000 7fab5a800700 1 -- 172.21.15.33:0/2278386418 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab5c103750 msgr2=0x7fab5c1a18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:07.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.162+0000 7fab5a800700 1 --2- 172.21.15.33:0/2278386418 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab5c103750 0x7fab5c1a18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:07.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.162+0000 7fab5a800700 1 -- 172.21.15.33:0/2278386418 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab5c19c1d0 con 0x7fab5c108850 2024-05-03T18:25:07.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.162+0000 7fab59e00700 1 --2- 172.21.15.33:0/2278386418 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab5c103750 0x7fab5c1a18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:25:07.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.162+0000 7fab59400700 1 --2- 172.21.15.33:0/2278386418 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab5c104100 0x7fab5c1a1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:25:07.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.162+0000 7fab5a800700 1 --2- 172.21.15.33:0/2278386418 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab5c108850 0x7fab5c19ba70 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fab54007ee0 tx=0x7fab5400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:25:07.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.163+0000 7fab52a00700 1 -- 172.21.15.33:0/2278386418 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab5400dbd0 con 0x7fab5c108850 2024-05-03T18:25:07.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.163+0000 7fab52a00700 1 -- 172.21.15.33:0/2278386418 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fab5400dd70 con 0x7fab5c108850 2024-05-03T18:25:07.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.163+0000 7fab60c00700 1 -- 172.21.15.33:0/2278386418 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab5c19c4c0 con 0x7fab5c108850 2024-05-03T18:25:07.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.163+0000 7fab52a00700 1 -- 172.21.15.33:0/2278386418 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab540058d0 con 0x7fab5c108850 2024-05-03T18:25:07.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.163+0000 7fab60c00700 1 -- 172.21.15.33:0/2278386418 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab5c1a8780 con 0x7fab5c108850 2024-05-03T18:25:07.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.164+0000 7fab60c00700 1 -- 172.21.15.33:0/2278386418 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fab5c10fa80 con 0x7fab5c108850 2024-05-03T18:25:07.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.166+0000 7fab52a00700 1 -- 172.21.15.33:0/2278386418 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fab5401a020 con 0x7fab5c108850 2024-05-03T18:25:07.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.169+0000 7fab52a00700 1 --2- 172.21.15.33:0/2278386418 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fab400b2990 0x7fab400b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:07.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.169+0000 7fab52a00700 1 -- 172.21.15.33:0/2278386418 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7fab540e2620 con 0x7fab5c108850 2024-05-03T18:25:07.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.169+0000 7fab59e00700 1 --2- 172.21.15.33:0/2278386418 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fab400b2990 0x7fab400b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:07.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.170+0000 7fab52a00700 1 --2- 172.21.15.33:0/2278386418 >> [v2:172.21.15.99:6816/908673090,v1:172.21.15.99:6817/908673090] conn(0x7fab400bdb70 0x7fab400c0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:07.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.170+0000 7fab52a00700 1 -- 172.21.15.33:0/2278386418 --> [v2:172.21.15.99:6816/908673090,v1:172.21.15.99:6817/908673090] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fab54012ec0 con 0x7fab400bdb70 2024-05-03T18:25:07.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.170+0000 7fab52a00700 1 -- 172.21.15.33:0/2278386418 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7fab540e29d0 con 0x7fab5c108850 2024-05-03T18:25:07.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.170+0000 7fab59e00700 1 --2- 172.21.15.33:0/2278386418 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fab400b2990 0x7fab400b4e50 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fab4c005950 tx=0x7fab4c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:25:07.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.170+0000 7fab59400700 1 --2- 172.21.15.33:0/2278386418 >> [v2:172.21.15.99:6816/908673090,v1:172.21.15.99:6817/908673090] conn(0x7fab400bdb70 0x7fab400c0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:07.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.171+0000 7fab59400700 1 --2- 172.21.15.33:0/2278386418 >> [v2:172.21.15.99:6816/908673090,v1:172.21.15.99:6817/908673090] conn(0x7fab400bdb70 0x7fab400c0010 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:25:07.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.173+0000 7fab52a00700 1 -- 172.21.15.33:0/2278386418 <== osd.7 v2:172.21.15.99:6816/908673090 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fab54012ec0 con 0x7fab400bdb70 2024-05-03T18:25:07.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.192+0000 7fab60c00700 1 -- 172.21.15.33:0/2278386418 --> [v2:172.21.15.99:6816/908673090,v1:172.21.15.99:6817/908673090] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fab5c104be0 con 0x7fab400bdb70 2024-05-03T18:25:07.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.193+0000 7fab52a00700 1 -- 172.21.15.33:0/2278386418 <== osd.7 v2:172.21.15.99:6816/908673090 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fab5c104be0 con 0x7fab400bdb70 2024-05-03T18:25:07.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.193+0000 7fab60c00700 1 -- 172.21.15.33:0/2278386418 >> [v2:172.21.15.99:6816/908673090,v1:172.21.15.99:6817/908673090] conn(0x7fab400bdb70 msgr2=0x7fab400c0010 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:07.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.193+0000 7fab60c00700 1 --2- 172.21.15.33:0/2278386418 >> [v2:172.21.15.99:6816/908673090,v1:172.21.15.99:6817/908673090] conn(0x7fab400bdb70 0x7fab400c0010 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:07.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.193+0000 7fab60c00700 1 -- 172.21.15.33:0/2278386418 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fab400b2990 msgr2=0x7fab400b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:07.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.193+0000 7fab60c00700 1 --2- 172.21.15.33:0/2278386418 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fab400b2990 0x7fab400b4e50 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fab4c005950 tx=0x7fab4c0058e0 comp rx=0 tx=0).stop 2024-05-03T18:25:07.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.193+0000 7fab60c00700 1 -- 172.21.15.33:0/2278386418 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab5c108850 msgr2=0x7fab5c19ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:07.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.193+0000 7fab60c00700 1 --2- 172.21.15.33:0/2278386418 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab5c108850 0x7fab5c19ba70 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fab54007ee0 tx=0x7fab5400d3b0 comp rx=0 tx=0).stop 2024-05-03T18:25:07.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.193+0000 7fab5a800700 1 -- 172.21.15.33:0/2278386418 reap_dead start 2024-05-03T18:25:07.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.194+0000 7fab60c00700 1 -- 172.21.15.33:0/2278386418 shutdown_connections 2024-05-03T18:25:07.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.194+0000 7fab60c00700 1 -- 172.21.15.33:0/2278386418 >> 172.21.15.33:0/2278386418 conn(0x7fab5c07a600 msgr2=0x7fab5c101940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:25:07.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.194+0000 7fab60c00700 1 -- 172.21.15.33:0/2278386418 shutdown_connections 2024-05-03T18:25:07.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.194+0000 7fab60c00700 1 -- 172.21.15.33:0/2278386418 wait complete. 2024-05-03T18:25:07.845 INFO:teuthology.orchestra.run.smithi033.stdout:107374182507 2024-05-03T18:25:07.846 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-05-03T18:25:08.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:25:08 smithi150 ceph-mon[32492]: pgmap v463: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:25:08.966 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:25:08 smithi033 ceph-mon[28330]: pgmap v463: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:25:09.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:25:08 smithi099 ceph-mon[31658]: pgmap v463: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:25:09.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:09.140+0000 7fd54be00700 1 -- 172.21.15.33:0/1875023407 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd54410b6c0 msgr2=0x7fd54410baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:09.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:09.140+0000 7fd54be00700 1 --2- 172.21.15.33:0/1875023407 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd54410b6c0 0x7fd54410baa0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd53800b0a0 tx=0x7fd53800b3b0 comp rx=0 tx=0).stop 2024-05-03T18:25:09.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:09.140+0000 7fd54be00700 1 -- 172.21.15.33:0/1875023407 shutdown_connections 2024-05-03T18:25:09.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:09.140+0000 7fd54be00700 1 --2- 172.21.15.33:0/1875023407 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd544106080 0x7fd544112590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:09.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:09.140+0000 7fd54be00700 1 --2- 172.21.15.33:0/1875023407 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd54410b6c0 0x7fd54410baa0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:09.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:09.140+0000 7fd54be00700 1 --2- 172.21.15.33:0/1875023407 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5441056c0 0x7fd544105b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:09.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:09.140+0000 7fd54be00700 1 -- 172.21.15.33:0/1875023407 >> 172.21.15.33:0/1875023407 conn(0x7fd5441011b0 msgr2=0x7fd5441035d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:25:09.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:09.140+0000 7fd54be00700 1 -- 172.21.15.33:0/1875023407 shutdown_connections 2024-05-03T18:25:09.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:09.140+0000 7fd54be00700 1 -- 172.21.15.33:0/1875023407 wait complete. 2024-05-03T18:25:09.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:09.142+0000 7fd54be00700 1 Processor -- start 2024-05-03T18:25:09.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:09.142+0000 7fd54be00700 1 -- start start 2024-05-03T18:25:09.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:09.142+0000 7fd54be00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5441056c0 0x7fd54419bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:09.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:09.142+0000 7fd54be00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd544106080 0x7fd54419c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:09.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:09.142+0000 7fd54be00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd54410b6c0 0x7fd5441a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:09.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:09.142+0000 7fd54be00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd544115800 con 0x7fd5441056c0 2024-05-03T18:25:09.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:09.142+0000 7fd54be00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd544115680 con 0x7fd544106080 2024-05-03T18:25:09.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:09.142+0000 7fd54be00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd544115980 con 0x7fd54410b6c0 2024-05-03T18:25:09.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:09.143+0000 7fd549400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5441056c0 0x7fd54419bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:09.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:09.143+0000 7fd549400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5441056c0 0x7fd54419bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58854/0 (socket says 172.21.15.33:58854) 2024-05-03T18:25:09.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:09.143+0000 7fd549400700 1 -- 172.21.15.33:0/2516425797 learned_addr learned my addr 172.21.15.33:0/2516425797 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:25:09.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:09.143+0000 7fd549e00700 1 --2- 172.21.15.33:0/2516425797 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd54410b6c0 0x7fd5441a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:09.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:09.143+0000 7fd548a00700 1 --2- 172.21.15.33:0/2516425797 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd544106080 0x7fd54419c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:09.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:09.143+0000 7fd549400700 1 -- 172.21.15.33:0/2516425797 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd54410b6c0 msgr2=0x7fd5441a05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:09.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:09.143+0000 7fd549400700 1 --2- 172.21.15.33:0/2516425797 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd54410b6c0 0x7fd5441a05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:09.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:09.143+0000 7fd549400700 1 -- 172.21.15.33:0/2516425797 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd544106080 msgr2=0x7fd54419c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:09.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:09.143+0000 7fd549400700 1 --2- 172.21.15.33:0/2516425797 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd544106080 0x7fd54419c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:09.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:09.143+0000 7fd549400700 1 -- 172.21.15.33:0/2516425797 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5441a0d20 con 0x7fd5441056c0 2024-05-03T18:25:09.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:09.143+0000 7fd549e00700 1 --2- 172.21.15.33:0/2516425797 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd54410b6c0 0x7fd5441a05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:25:09.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:09.144+0000 7fd549400700 1 --2- 172.21.15.33:0/2516425797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5441056c0 0x7fd54419bd10 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fd53800b070 tx=0x7fd538004ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:25:09.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:09.144+0000 7fd542000700 1 -- 172.21.15.33:0/2516425797 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd538021070 con 0x7fd5441056c0 2024-05-03T18:25:09.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:09.144+0000 7fd542000700 1 -- 172.21.15.33:0/2516425797 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd538007d00 con 0x7fd5441056c0 2024-05-03T18:25:09.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:09.144+0000 7fd54be00700 1 -- 172.21.15.33:0/2516425797 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5441a0fb0 con 0x7fd5441056c0 2024-05-03T18:25:09.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:09.144+0000 7fd542000700 1 -- 172.21.15.33:0/2516425797 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd53801d450 con 0x7fd5441056c0 2024-05-03T18:25:09.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:09.144+0000 7fd54be00700 1 -- 172.21.15.33:0/2516425797 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5441a8890 con 0x7fd5441056c0 2024-05-03T18:25:09.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:09.145+0000 7fd54be00700 1 -- 172.21.15.33:0/2516425797 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd544063c30 con 0x7fd5441056c0 2024-05-03T18:25:09.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:09.147+0000 7fd542000700 1 -- 172.21.15.33:0/2516425797 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fd538004140 con 0x7fd5441056c0 2024-05-03T18:25:09.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:09.149+0000 7fd542000700 1 --2- 172.21.15.33:0/2516425797 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fd52c0b2910 0x7fd52c0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:09.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:09.149+0000 7fd542000700 1 -- 172.21.15.33:0/2516425797 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7fd53800f020 con 0x7fd5441056c0 2024-05-03T18:25:09.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:09.149+0000 7fd548a00700 1 --2- 172.21.15.33:0/2516425797 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fd52c0b2910 0x7fd52c0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:09.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:09.149+0000 7fd542000700 1 --2- 172.21.15.33:0/2516425797 >> [v2:172.21.15.33:6818/4234541418,v1:172.21.15.33:6819/4234541418] conn(0x7fd52c0bdaf0 0x7fd52c0bff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:09.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:09.149+0000 7fd542000700 1 -- 172.21.15.33:0/2516425797 --> [v2:172.21.15.33:6818/4234541418,v1:172.21.15.33:6819/4234541418] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd52c0c0660 con 0x7fd52c0bdaf0 2024-05-03T18:25:09.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:09.149+0000 7fd542000700 1 -- 172.21.15.33:0/2516425797 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7fd5380e44e0 con 0x7fd5441056c0 2024-05-03T18:25:09.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:09.150+0000 7fd549e00700 1 --2- 172.21.15.33:0/2516425797 >> [v2:172.21.15.33:6818/4234541418,v1:172.21.15.33:6819/4234541418] conn(0x7fd52c0bdaf0 0x7fd52c0bff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:09.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:09.150+0000 7fd548a00700 1 --2- 172.21.15.33:0/2516425797 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fd52c0b2910 0x7fd52c0b4dd0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd544103830 tx=0x7fd530005eb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:25:09.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:09.150+0000 7fd549e00700 1 --2- 172.21.15.33:0/2516425797 >> [v2:172.21.15.33:6818/4234541418,v1:172.21.15.33:6819/4234541418] conn(0x7fd52c0bdaf0 0x7fd52c0bff90 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:25:09.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:09.151+0000 7fd542000700 1 -- 172.21.15.33:0/2516425797 <== osd.6 v2:172.21.15.33:6818/4234541418 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd52c0c0660 con 0x7fd52c0bdaf0 2024-05-03T18:25:09.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:09.170+0000 7fd54be00700 1 -- 172.21.15.33:0/2516425797 --> [v2:172.21.15.33:6818/4234541418,v1:172.21.15.33:6819/4234541418] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd5440640b0 con 0x7fd52c0bdaf0 2024-05-03T18:25:09.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:09.170+0000 7fd542000700 1 -- 172.21.15.33:0/2516425797 <== osd.6 v2:172.21.15.33:6818/4234541418 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fd5440640b0 con 0x7fd52c0bdaf0 2024-05-03T18:25:09.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:09.170+0000 7fd54be00700 1 -- 172.21.15.33:0/2516425797 >> [v2:172.21.15.33:6818/4234541418,v1:172.21.15.33:6819/4234541418] conn(0x7fd52c0bdaf0 msgr2=0x7fd52c0bff90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:09.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:09.170+0000 7fd54be00700 1 --2- 172.21.15.33:0/2516425797 >> [v2:172.21.15.33:6818/4234541418,v1:172.21.15.33:6819/4234541418] conn(0x7fd52c0bdaf0 0x7fd52c0bff90 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:09.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:09.171+0000 7fd54be00700 1 -- 172.21.15.33:0/2516425797 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fd52c0b2910 msgr2=0x7fd52c0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:09.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:09.171+0000 7fd54be00700 1 --2- 172.21.15.33:0/2516425797 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fd52c0b2910 0x7fd52c0b4dd0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd544103830 tx=0x7fd530005eb0 comp rx=0 tx=0).stop 2024-05-03T18:25:09.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:09.171+0000 7fd54be00700 1 -- 172.21.15.33:0/2516425797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5441056c0 msgr2=0x7fd54419bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:09.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:09.171+0000 7fd54be00700 1 --2- 172.21.15.33:0/2516425797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5441056c0 0x7fd54419bd10 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fd53800b070 tx=0x7fd538004ed0 comp rx=0 tx=0).stop 2024-05-03T18:25:09.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:09.171+0000 7fd549e00700 1 -- 172.21.15.33:0/2516425797 reap_dead start 2024-05-03T18:25:09.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:09.171+0000 7fd54be00700 1 -- 172.21.15.33:0/2516425797 shutdown_connections 2024-05-03T18:25:09.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:09.171+0000 7fd54be00700 1 -- 172.21.15.33:0/2516425797 >> 172.21.15.33:0/2516425797 conn(0x7fd5441011b0 msgr2=0x7fd544101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:25:09.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:09.171+0000 7fd54be00700 1 -- 172.21.15.33:0/2516425797 shutdown_connections 2024-05-03T18:25:09.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:09.171+0000 7fd54be00700 1 -- 172.21.15.33:0/2516425797 wait complete. 2024-05-03T18:25:09.260 INFO:teuthology.orchestra.run.smithi033.stdout:133143986277 2024-05-03T18:25:09.260 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-05-03T18:25:09.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.582+0000 7f445cc00700 1 -- 172.21.15.33:0/2407636073 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44581045e0 msgr2=0x7f44581049c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:09.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.582+0000 7f445cc00700 1 --2- 172.21.15.33:0/2407636073 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44581045e0 0x7f44581049c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f4448009990 tx=0x7f4448009ca0 comp rx=0 tx=0).stop 2024-05-03T18:25:09.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.582+0000 7f445cc00700 1 -- 172.21.15.33:0/2407636073 shutdown_connections 2024-05-03T18:25:09.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.582+0000 7f445cc00700 1 --2- 172.21.15.33:0/2407636073 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f445810da10 0x7f445810fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:09.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.582+0000 7f445cc00700 1 --2- 172.21.15.33:0/2407636073 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44581045e0 0x7f44581049c0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:09.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.582+0000 7f445cc00700 1 --2- 172.21.15.33:0/2407636073 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4458104f00 0x7f445810d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:09.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.582+0000 7f445cc00700 1 -- 172.21.15.33:0/2407636073 >> 172.21.15.33:0/2407636073 conn(0x7f44580fde30 msgr2=0x7f4458100250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:25:09.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.582+0000 7f445cc00700 1 -- 172.21.15.33:0/2407636073 shutdown_connections 2024-05-03T18:25:09.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.582+0000 7f445cc00700 1 -- 172.21.15.33:0/2407636073 wait complete. 2024-05-03T18:25:09.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.584+0000 7f445cc00700 1 Processor -- start 2024-05-03T18:25:09.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.584+0000 7f445cc00700 1 -- start start 2024-05-03T18:25:09.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.585+0000 7f445cc00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44581045e0 0x7f44581a1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:09.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.585+0000 7f445cc00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4458104f00 0x7f44581a1e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:09.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.585+0000 7f445cc00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f445810da10 0x7f445819bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:09.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.585+0000 7f445cc00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f44581134e0 con 0x7f44581045e0 2024-05-03T18:25:09.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.585+0000 7f445cc00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4458113360 con 0x7f445810da10 2024-05-03T18:25:09.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.585+0000 7f445cc00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f4458113660 con 0x7f4458104f00 2024-05-03T18:25:09.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.585+0000 7f4457400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44581045e0 0x7f44581a1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:09.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.585+0000 7f4457400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44581045e0 0x7f44581a1900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58844/0 (socket says 172.21.15.33:58844) 2024-05-03T18:25:09.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.585+0000 7f4457400700 1 -- 172.21.15.33:0/860568801 learned_addr learned my addr 172.21.15.33:0/860568801 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:25:09.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.585+0000 7f4456a00700 1 --2- 172.21.15.33:0/860568801 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4458104f00 0x7f44581a1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:09.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.585+0000 7f4457e00700 1 --2- 172.21.15.33:0/860568801 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f445810da10 0x7f445819bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:09.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.585+0000 7f4457400700 1 -- 172.21.15.33:0/860568801 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4458104f00 msgr2=0x7f44581a1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:09.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.585+0000 7f4457400700 1 --2- 172.21.15.33:0/860568801 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4458104f00 0x7f44581a1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:09.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.585+0000 7f4457400700 1 -- 172.21.15.33:0/860568801 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f445810da10 msgr2=0x7f445819bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:09.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.586+0000 7f4457400700 1 --2- 172.21.15.33:0/860568801 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f445810da10 0x7f445819bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:09.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.586+0000 7f4457400700 1 -- 172.21.15.33:0/860568801 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f445819c220 con 0x7f44581045e0 2024-05-03T18:25:09.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.586+0000 7f4456a00700 1 --2- 172.21.15.33:0/860568801 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4458104f00 0x7f44581a1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:25:09.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.586+0000 7f4457e00700 1 --2- 172.21.15.33:0/860568801 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f445810da10 0x7f445819bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:25:09.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.586+0000 7f4457400700 1 --2- 172.21.15.33:0/860568801 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44581045e0 0x7f44581a1900 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f4448000c00 tx=0x7f44480047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:25:09.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.586+0000 7f444fe00700 1 -- 172.21.15.33:0/860568801 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4448027070 con 0x7f44581045e0 2024-05-03T18:25:09.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.586+0000 7f444fe00700 1 -- 172.21.15.33:0/860568801 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f444800fdb0 con 0x7f44581045e0 2024-05-03T18:25:09.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.586+0000 7f444fe00700 1 -- 172.21.15.33:0/860568801 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44480173f0 con 0x7f44581045e0 2024-05-03T18:25:09.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.586+0000 7f445cc00700 1 -- 172.21.15.33:0/860568801 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f445819c4b0 con 0x7f44581045e0 2024-05-03T18:25:09.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.587+0000 7f445cc00700 1 -- 172.21.15.33:0/860568801 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44581a87d0 con 0x7f44581045e0 2024-05-03T18:25:09.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.587+0000 7f445cc00700 1 -- 172.21.15.33:0/860568801 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4458063b70 con 0x7f44581045e0 2024-05-03T18:25:09.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.590+0000 7f444fe00700 1 -- 172.21.15.33:0/860568801 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f444802a070 con 0x7f44581045e0 2024-05-03T18:25:09.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.592+0000 7f444fe00700 1 --2- 172.21.15.33:0/860568801 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f443c0b2990 0x7f443c0b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:09.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.592+0000 7f444fe00700 1 -- 172.21.15.33:0/860568801 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f44480e3f40 con 0x7f44581045e0 2024-05-03T18:25:09.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.592+0000 7f4456a00700 1 --2- 172.21.15.33:0/860568801 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f443c0b2990 0x7f443c0b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:09.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.592+0000 7f444fe00700 1 --2- 172.21.15.33:0/860568801 >> [v2:172.21.15.150:6824/2069349808,v1:172.21.15.150:6825/2069349808] conn(0x7f443c0bda40 0x7f443c0bfee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:09.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.592+0000 7f444fe00700 1 -- 172.21.15.33:0/860568801 --> [v2:172.21.15.150:6824/2069349808,v1:172.21.15.150:6825/2069349808] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f443c0c05b0 con 0x7f443c0bda40 2024-05-03T18:25:09.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.592+0000 7f444fe00700 1 -- 172.21.15.33:0/860568801 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f44480e42f0 con 0x7f44581045e0 2024-05-03T18:25:09.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.593+0000 7f4456a00700 1 --2- 172.21.15.33:0/860568801 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f443c0b2990 0x7f443c0b4e50 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4444005fd0 tx=0x7f4444005e60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:25:09.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.593+0000 7f4457e00700 1 --2- 172.21.15.33:0/860568801 >> [v2:172.21.15.150:6824/2069349808,v1:172.21.15.150:6825/2069349808] conn(0x7f443c0bda40 0x7f443c0bfee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:09.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.594+0000 7f4457e00700 1 --2- 172.21.15.33:0/860568801 >> [v2:172.21.15.150:6824/2069349808,v1:172.21.15.150:6825/2069349808] conn(0x7f443c0bda40 0x7f443c0bfee0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:25:09.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.596+0000 7f444fe00700 1 -- 172.21.15.33:0/860568801 <== osd.11 v2:172.21.15.150:6824/2069349808 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f443c0c05b0 con 0x7f443c0bda40 2024-05-03T18:25:09.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.615+0000 7f445cc00700 1 -- 172.21.15.33:0/860568801 --> [v2:172.21.15.150:6824/2069349808,v1:172.21.15.150:6825/2069349808] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4458063fc0 con 0x7f443c0bda40 2024-05-03T18:25:09.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.616+0000 7f444fe00700 1 -- 172.21.15.33:0/860568801 <== osd.11 v2:172.21.15.150:6824/2069349808 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f4458063fc0 con 0x7f443c0bda40 2024-05-03T18:25:09.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.616+0000 7f445cc00700 1 -- 172.21.15.33:0/860568801 >> [v2:172.21.15.150:6824/2069349808,v1:172.21.15.150:6825/2069349808] conn(0x7f443c0bda40 msgr2=0x7f443c0bfee0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:09.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.616+0000 7f445cc00700 1 --2- 172.21.15.33:0/860568801 >> [v2:172.21.15.150:6824/2069349808,v1:172.21.15.150:6825/2069349808] conn(0x7f443c0bda40 0x7f443c0bfee0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:09.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.617+0000 7f445cc00700 1 -- 172.21.15.33:0/860568801 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f443c0b2990 msgr2=0x7f443c0b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:09.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.617+0000 7f445cc00700 1 --2- 172.21.15.33:0/860568801 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f443c0b2990 0x7f443c0b4e50 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4444005fd0 tx=0x7f4444005e60 comp rx=0 tx=0).stop 2024-05-03T18:25:09.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.617+0000 7f445cc00700 1 -- 172.21.15.33:0/860568801 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44581045e0 msgr2=0x7f44581a1900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:09.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.617+0000 7f445cc00700 1 --2- 172.21.15.33:0/860568801 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44581045e0 0x7f44581a1900 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f4448000c00 tx=0x7f44480047b0 comp rx=0 tx=0).stop 2024-05-03T18:25:09.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.617+0000 7f4457e00700 1 -- 172.21.15.33:0/860568801 reap_dead start 2024-05-03T18:25:09.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.617+0000 7f445cc00700 1 -- 172.21.15.33:0/860568801 shutdown_connections 2024-05-03T18:25:09.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.617+0000 7f445cc00700 1 -- 172.21.15.33:0/860568801 >> 172.21.15.33:0/860568801 conn(0x7f44580fde30 msgr2=0x7f4458100220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:25:09.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.617+0000 7f445cc00700 1 -- 172.21.15.33:0/860568801 shutdown_connections 2024-05-03T18:25:09.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:07.617+0000 7f445cc00700 1 -- 172.21.15.33:0/860568801 wait complete. 2024-05-03T18:25:10.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:10.454+0000 7f1cd8400700 1 -- 172.21.15.33:0/2610734797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1cd0104100 msgr2=0x7f1cd0108120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:10.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:10.454+0000 7f1cd8400700 1 --2- 172.21.15.33:0/2610734797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1cd0104100 0x7f1cd0108120 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc0099c0 tx=0x7f1cbc009cd0 comp rx=0 tx=0).stop 2024-05-03T18:25:10.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:10.454+0000 7f1cd8400700 1 -- 172.21.15.33:0/2610734797 shutdown_connections 2024-05-03T18:25:10.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:10.454+0000 7f1cd8400700 1 --2- 172.21.15.33:0/2610734797 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1cd0108850 0x7f1cd01103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:10.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:10.454+0000 7f1cd8400700 1 --2- 172.21.15.33:0/2610734797 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1cd0103750 0x7f1cd0103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:10.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:10.454+0000 7f1cd8400700 1 --2- 172.21.15.33:0/2610734797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1cd0104100 0x7f1cd0108120 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:10.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:10.454+0000 7f1cd8400700 1 -- 172.21.15.33:0/2610734797 >> 172.21.15.33:0/2610734797 conn(0x7f1cd007a600 msgr2=0x7f1cd007aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:25:10.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:10.455+0000 7f1cd8400700 1 -- 172.21.15.33:0/2610734797 shutdown_connections 2024-05-03T18:25:10.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:10.455+0000 7f1cd8400700 1 -- 172.21.15.33:0/2610734797 wait complete. 2024-05-03T18:25:10.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:10.456+0000 7f1cd8400700 1 Processor -- start 2024-05-03T18:25:10.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:10.456+0000 7f1cd8400700 1 -- start start 2024-05-03T18:25:10.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:10.456+0000 7f1cd8400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1cd0103750 0x7f1cd00757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:10.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:10.456+0000 7f1cd8400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1cd0104100 0x7f1cd0075ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:10.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:10.456+0000 7f1cd8400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1cd0108850 0x7f1cd0076220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:10.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:10.456+0000 7f1cd8400700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1cd0113510 con 0x7f1cd0104100 2024-05-03T18:25:10.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:10.456+0000 7f1cd8400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1cd0113390 con 0x7f1cd0108850 2024-05-03T18:25:10.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:10.456+0000 7f1cd8400700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f1cd0113690 con 0x7f1cd0103750 2024-05-03T18:25:10.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:10.457+0000 7f1cd5000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1cd0104100 0x7f1cd0075ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:10.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:10.457+0000 7f1cd5000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1cd0104100 0x7f1cd0075ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58868/0 (socket says 172.21.15.33:58868) 2024-05-03T18:25:10.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:10.457+0000 7f1cd5000700 1 -- 172.21.15.33:0/351612686 learned_addr learned my addr 172.21.15.33:0/351612686 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:25:10.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:10.457+0000 7f1cd5a00700 1 --2- 172.21.15.33:0/351612686 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1cd0103750 0x7f1cd00757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:10.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:10.457+0000 7f1cd6400700 1 --2- 172.21.15.33:0/351612686 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1cd0108850 0x7f1cd0076220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:10.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:10.457+0000 7f1cd5000700 1 -- 172.21.15.33:0/351612686 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1cd0103750 msgr2=0x7f1cd00757a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:10.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:10.457+0000 7f1cd5000700 1 --2- 172.21.15.33:0/351612686 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1cd0103750 0x7f1cd00757a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:10.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:10.457+0000 7f1cd5000700 1 -- 172.21.15.33:0/351612686 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1cd0108850 msgr2=0x7f1cd0076220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:10.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:10.457+0000 7f1cd5000700 1 --2- 172.21.15.33:0/351612686 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1cd0108850 0x7f1cd0076220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:10.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:10.457+0000 7f1cd5000700 1 -- 172.21.15.33:0/351612686 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cd01a7f20 con 0x7f1cd0104100 2024-05-03T18:25:10.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:10.457+0000 7f1cd5000700 1 --2- 172.21.15.33:0/351612686 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1cd0104100 0x7f1cd0075ce0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc00b580 tx=0x7f1cbc00f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:25:10.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:10.458+0000 7f1cce000700 1 -- 172.21.15.33:0/351612686 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cbc026070 con 0x7f1cd0104100 2024-05-03T18:25:10.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:10.458+0000 7f1cd8400700 1 -- 172.21.15.33:0/351612686 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cd01a81b0 con 0x7f1cd0104100 2024-05-03T18:25:10.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:10.458+0000 7f1cce000700 1 -- 172.21.15.33:0/351612686 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1cbc0054c0 con 0x7f1cd0104100 2024-05-03T18:25:10.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:10.458+0000 7f1cce000700 1 -- 172.21.15.33:0/351612686 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cbc004c00 con 0x7f1cd0104100 2024-05-03T18:25:10.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:10.458+0000 7f1cd8400700 1 -- 172.21.15.33:0/351612686 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cd01a8600 con 0x7f1cd0104100 2024-05-03T18:25:10.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:10.459+0000 7f1cd8400700 1 -- 172.21.15.33:0/351612686 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1cd0063c30 con 0x7f1cd0104100 2024-05-03T18:25:10.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:10.462+0000 7f1cce000700 1 -- 172.21.15.33:0/351612686 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f1cbc029070 con 0x7f1cd0104100 2024-05-03T18:25:10.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:10.463+0000 7f1cce000700 1 --2- 172.21.15.33:0/351612686 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f1cb80b2910 0x7f1cb80b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:10.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:10.463+0000 7f1cce000700 1 -- 172.21.15.33:0/351612686 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f1cbc0e3a60 con 0x7f1cd0104100 2024-05-03T18:25:10.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:10.463+0000 7f1cd5a00700 1 --2- 172.21.15.33:0/351612686 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f1cb80b2910 0x7f1cb80b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:10.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:10.464+0000 7f1cce000700 1 --2- 172.21.15.33:0/351612686 >> [v2:172.21.15.33:6826/4133096905,v1:172.21.15.33:6827/4133096905] conn(0x7f1cb80bdaf0 0x7f1cb80bff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:10.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:10.464+0000 7f1cce000700 1 -- 172.21.15.33:0/351612686 --> [v2:172.21.15.33:6826/4133096905,v1:172.21.15.33:6827/4133096905] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1cbc00b0b0 con 0x7f1cb80bdaf0 2024-05-03T18:25:10.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:10.464+0000 7f1cce000700 1 -- 172.21.15.33:0/351612686 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f1cbc0e3e10 con 0x7f1cd0104100 2024-05-03T18:25:10.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:10.464+0000 7f1cd6400700 1 --2- 172.21.15.33:0/351612686 >> [v2:172.21.15.33:6826/4133096905,v1:172.21.15.33:6827/4133096905] conn(0x7f1cb80bdaf0 0x7f1cb80bff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:10.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:10.464+0000 7f1cd5a00700 1 --2- 172.21.15.33:0/351612686 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f1cb80b2910 0x7f1cb80b4dd0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1cc4005fd0 tx=0x7f1cc400a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:25:10.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:10.465+0000 7f1cd6400700 1 --2- 172.21.15.33:0/351612686 >> [v2:172.21.15.33:6826/4133096905,v1:172.21.15.33:6827/4133096905] conn(0x7f1cb80bdaf0 0x7f1cb80bff90 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:25:10.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:10.466+0000 7f1cce000700 1 -- 172.21.15.33:0/351612686 <== osd.9 v2:172.21.15.33:6826/4133096905 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1cbc00b0b0 con 0x7f1cb80bdaf0 2024-05-03T18:25:10.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:10.484+0000 7f1cd8400700 1 -- 172.21.15.33:0/351612686 --> [v2:172.21.15.33:6826/4133096905,v1:172.21.15.33:6827/4133096905] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1cd00640b0 con 0x7f1cb80bdaf0 2024-05-03T18:25:10.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:10.485+0000 7f1cce000700 1 -- 172.21.15.33:0/351612686 <== osd.9 v2:172.21.15.33:6826/4133096905 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f1cd00640b0 con 0x7f1cb80bdaf0 2024-05-03T18:25:10.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:10.485+0000 7f1cd8400700 1 -- 172.21.15.33:0/351612686 >> [v2:172.21.15.33:6826/4133096905,v1:172.21.15.33:6827/4133096905] conn(0x7f1cb80bdaf0 msgr2=0x7f1cb80bff90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:10.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:10.485+0000 7f1cd8400700 1 --2- 172.21.15.33:0/351612686 >> [v2:172.21.15.33:6826/4133096905,v1:172.21.15.33:6827/4133096905] conn(0x7f1cb80bdaf0 0x7f1cb80bff90 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:10.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:10.485+0000 7f1cd8400700 1 -- 172.21.15.33:0/351612686 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f1cb80b2910 msgr2=0x7f1cb80b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:10.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:10.485+0000 7f1cd8400700 1 --2- 172.21.15.33:0/351612686 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f1cb80b2910 0x7f1cb80b4dd0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1cc4005fd0 tx=0x7f1cc400a560 comp rx=0 tx=0).stop 2024-05-03T18:25:10.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:10.485+0000 7f1cd8400700 1 -- 172.21.15.33:0/351612686 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1cd0104100 msgr2=0x7f1cd0075ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:10.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:10.485+0000 7f1cd8400700 1 --2- 172.21.15.33:0/351612686 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1cd0104100 0x7f1cd0075ce0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc00b580 tx=0x7f1cbc00f740 comp rx=0 tx=0).stop 2024-05-03T18:25:10.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:10.485+0000 7f1cd6400700 1 -- 172.21.15.33:0/351612686 reap_dead start 2024-05-03T18:25:10.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:10.486+0000 7f1cd8400700 1 -- 172.21.15.33:0/351612686 shutdown_connections 2024-05-03T18:25:10.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:10.486+0000 7f1cd8400700 1 -- 172.21.15.33:0/351612686 >> 172.21.15.33:0/351612686 conn(0x7f1cd007a600 msgr2=0x7f1cd0101840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:25:10.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:10.486+0000 7f1cd8400700 1 -- 172.21.15.33:0/351612686 shutdown_connections 2024-05-03T18:25:10.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:10.486+0000 7f1cd8400700 1 -- 172.21.15.33:0/351612686 wait complete. 2024-05-03T18:25:10.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:25:10 smithi150 ceph-mon[32492]: pgmap v464: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:25:10.918 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:25:10 smithi033 ceph-mon[28330]: pgmap v464: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:25:11.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:25:10 smithi099 ceph-mon[31658]: pgmap v464: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:25:11.670 INFO:teuthology.orchestra.run.smithi033.stdout:227633266761 2024-05-03T18:25:11.670 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-05-03T18:25:11.796 INFO:teuthology.orchestra.run.smithi033.stdout:300647710773 2024-05-03T18:25:11.796 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-05-03T18:25:12.299 INFO:teuthology.orchestra.run.smithi033.stdout:180388626522 2024-05-03T18:25:12.300 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-05-03T18:25:12.848 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:25:12 smithi099 ceph-mon[31658]: pgmap v465: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:25:12.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:25:12 smithi150 ceph-mon[32492]: pgmap v465: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:25:12.935 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:25:12 smithi033 ceph-mon[28330]: pgmap v465: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:25:12.987 INFO:teuthology.orchestra.run.smithi033.stdout:253403070531 2024-05-03T18:25:12.987 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-05-03T18:25:13.414 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:25:13.435 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:25:13.456 INFO:teuthology.orchestra.run.smithi033.stdout:201863462994 2024-05-03T18:25:13.456 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-05-03T18:25:13.516 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:25:14.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:14.478+0000 7f9610400700 1 -- 172.21.15.33:0/1853937836 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f960806bb40 msgr2=0x7f9608110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:14.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:14.478+0000 7f9610400700 1 --2- 172.21.15.33:0/1853937836 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f960806bb40 0x7f9608110330 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f960000b0a0 tx=0x7f960000b3b0 comp rx=0 tx=0).stop 2024-05-03T18:25:14.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:14.478+0000 7f9610400700 1 -- 172.21.15.33:0/1853937836 shutdown_connections 2024-05-03T18:25:14.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:14.478+0000 7f9610400700 1 --2- 172.21.15.33:0/1853937836 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f960806bb40 0x7f9608110330 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:14.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:14.478+0000 7f9610400700 1 --2- 172.21.15.33:0/1853937836 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9608109510 0x7f96081098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:14.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:14.478+0000 7f9610400700 1 --2- 172.21.15.33:0/1853937836 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f960806b180 0x7f960806b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:14.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:14.478+0000 7f9610400700 1 -- 172.21.15.33:0/1853937836 >> 172.21.15.33:0/1853937836 conn(0x7f9608079350 msgr2=0x7f9608079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:25:14.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:14.479+0000 7f9610400700 1 -- 172.21.15.33:0/1853937836 shutdown_connections 2024-05-03T18:25:14.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:14.479+0000 7f9610400700 1 -- 172.21.15.33:0/1853937836 wait complete. 2024-05-03T18:25:14.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:14.479+0000 7f9610400700 1 Processor -- start 2024-05-03T18:25:14.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:14.480+0000 7f9610400700 1 -- start start 2024-05-03T18:25:14.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:14.480+0000 7f9610400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f960806b180 0x7f96081a18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:14.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:14.480+0000 7f9610400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f960806bb40 0x7f96081a1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:14.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:14.480+0000 7f9610400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9608109510 0x7f960819bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:14.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:14.480+0000 7f9610400700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f96081134d0 con 0x7f960806b180 2024-05-03T18:25:14.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:14.480+0000 7f9610400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9608113350 con 0x7f9608109510 2024-05-03T18:25:14.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:14.480+0000 7f9610400700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9608113650 con 0x7f960806bb40 2024-05-03T18:25:14.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:14.480+0000 7f960da00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f960806b180 0x7f96081a18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:14.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:14.480+0000 7f960da00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f960806b180 0x7f96081a18f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58874/0 (socket says 172.21.15.33:58874) 2024-05-03T18:25:14.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:14.480+0000 7f960da00700 1 -- 172.21.15.33:0/3363523848 learned_addr learned my addr 172.21.15.33:0/3363523848 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:25:14.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:14.481+0000 7f960d000700 1 --2- 172.21.15.33:0/3363523848 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f960806bb40 0x7f96081a1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:14.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:14.481+0000 7f960e400700 1 --2- 172.21.15.33:0/3363523848 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9608109510 0x7f960819bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:14.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:14.481+0000 7f960da00700 1 -- 172.21.15.33:0/3363523848 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f960806bb40 msgr2=0x7f96081a1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:14.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:14.481+0000 7f960da00700 1 --2- 172.21.15.33:0/3363523848 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f960806bb40 0x7f96081a1e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:14.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:14.481+0000 7f960da00700 1 -- 172.21.15.33:0/3363523848 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9608109510 msgr2=0x7f960819bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:14.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:14.481+0000 7f960da00700 1 --2- 172.21.15.33:0/3363523848 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9608109510 0x7f960819bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:14.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:14.481+0000 7f960da00700 1 -- 172.21.15.33:0/3363523848 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f960819c210 con 0x7f960806b180 2024-05-03T18:25:14.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:14.481+0000 7f960da00700 1 --2- 172.21.15.33:0/3363523848 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f960806b180 0x7f96081a18f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f95fc00c920 tx=0x7f95fc00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:25:14.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:14.481+0000 7f9606000700 1 -- 172.21.15.33:0/3363523848 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95fc007b00 con 0x7f960806b180 2024-05-03T18:25:14.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:14.482+0000 7f9606000700 1 -- 172.21.15.33:0/3363523848 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f95fc004cc0 con 0x7f960806b180 2024-05-03T18:25:14.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:14.482+0000 7f9610400700 1 -- 172.21.15.33:0/3363523848 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f960819c4a0 con 0x7f960806b180 2024-05-03T18:25:14.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:14.482+0000 7f9606000700 1 -- 172.21.15.33:0/3363523848 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95fc0054f0 con 0x7f960806b180 2024-05-03T18:25:14.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:14.482+0000 7f9610400700 1 -- 172.21.15.33:0/3363523848 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96081a87c0 con 0x7f960806b180 2024-05-03T18:25:14.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:14.482+0000 7f9610400700 1 -- 172.21.15.33:0/3363523848 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f960810f040 con 0x7f960806b180 2024-05-03T18:25:14.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:14.485+0000 7f9606000700 1 -- 172.21.15.33:0/3363523848 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f95fc007ca0 con 0x7f960806b180 2024-05-03T18:25:14.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:14.487+0000 7f9606000700 1 --2- 172.21.15.33:0/3363523848 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f95f00b2990 0x7f95f00b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:14.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:14.487+0000 7f9606000700 1 -- 172.21.15.33:0/3363523848 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f95fc0e26a0 con 0x7f960806b180 2024-05-03T18:25:14.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:14.487+0000 7f960d000700 1 --2- 172.21.15.33:0/3363523848 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f95f00b2990 0x7f95f00b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:14.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:14.487+0000 7f9606000700 1 --2- 172.21.15.33:0/3363523848 >> [v2:172.21.15.150:6808/3172189533,v1:172.21.15.150:6809/3172189533] conn(0x7f95f00bdb70 0x7f95f00c0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:14.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:14.487+0000 7f9606000700 1 -- 172.21.15.33:0/3363523848 --> [v2:172.21.15.150:6808/3172189533,v1:172.21.15.150:6809/3172189533] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f95f00c06e0 con 0x7f95f00bdb70 2024-05-03T18:25:14.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:14.487+0000 7f9606000700 1 -- 172.21.15.33:0/3363523848 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f95fc0e2a50 con 0x7f960806b180 2024-05-03T18:25:14.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:14.488+0000 7f960d000700 1 --2- 172.21.15.33:0/3363523848 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f95f00b2990 0x7f95f00b4e50 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f95f4005fd0 tx=0x7f95f400a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:25:14.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:14.488+0000 7f960e400700 1 --2- 172.21.15.33:0/3363523848 >> [v2:172.21.15.150:6808/3172189533,v1:172.21.15.150:6809/3172189533] conn(0x7f95f00bdb70 0x7f95f00c0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:14.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:14.489+0000 7f960e400700 1 --2- 172.21.15.33:0/3363523848 >> [v2:172.21.15.150:6808/3172189533,v1:172.21.15.150:6809/3172189533] conn(0x7f95f00bdb70 0x7f95f00c0010 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:25:14.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:14.491+0000 7f9606000700 1 -- 172.21.15.33:0/3363523848 <== osd.5 v2:172.21.15.150:6808/3172189533 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f95f00c06e0 con 0x7f95f00bdb70 2024-05-03T18:25:14.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:14.511+0000 7f9610400700 1 -- 172.21.15.33:0/3363523848 --> [v2:172.21.15.150:6808/3172189533,v1:172.21.15.150:6809/3172189533] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9608104bd0 con 0x7f95f00bdb70 2024-05-03T18:25:14.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:14.512+0000 7f9606000700 1 -- 172.21.15.33:0/3363523848 <== osd.5 v2:172.21.15.150:6808/3172189533 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9608104bd0 con 0x7f95f00bdb70 2024-05-03T18:25:14.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:14.512+0000 7f9610400700 1 -- 172.21.15.33:0/3363523848 >> [v2:172.21.15.150:6808/3172189533,v1:172.21.15.150:6809/3172189533] conn(0x7f95f00bdb70 msgr2=0x7f95f00c0010 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:14.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:14.512+0000 7f9610400700 1 --2- 172.21.15.33:0/3363523848 >> [v2:172.21.15.150:6808/3172189533,v1:172.21.15.150:6809/3172189533] conn(0x7f95f00bdb70 0x7f95f00c0010 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:14.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:14.513+0000 7f9610400700 1 -- 172.21.15.33:0/3363523848 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f95f00b2990 msgr2=0x7f95f00b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:14.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:14.513+0000 7f9610400700 1 --2- 172.21.15.33:0/3363523848 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f95f00b2990 0x7f95f00b4e50 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f95f4005fd0 tx=0x7f95f400a560 comp rx=0 tx=0).stop 2024-05-03T18:25:14.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:14.513+0000 7f9610400700 1 -- 172.21.15.33:0/3363523848 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f960806b180 msgr2=0x7f96081a18f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:14.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:14.513+0000 7f9610400700 1 --2- 172.21.15.33:0/3363523848 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f960806b180 0x7f96081a18f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f95fc00c920 tx=0x7f95fc00cce0 comp rx=0 tx=0).stop 2024-05-03T18:25:14.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:14.513+0000 7f960e400700 1 -- 172.21.15.33:0/3363523848 reap_dead start 2024-05-03T18:25:14.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:14.513+0000 7f9610400700 1 -- 172.21.15.33:0/3363523848 shutdown_connections 2024-05-03T18:25:14.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:14.513+0000 7f9610400700 1 -- 172.21.15.33:0/3363523848 >> 172.21.15.33:0/3363523848 conn(0x7f9608079350 msgr2=0x7f960810e660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:25:14.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:14.513+0000 7f9610400700 1 -- 172.21.15.33:0/3363523848 shutdown_connections 2024-05-03T18:25:14.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:14.514+0000 7f9610400700 1 -- 172.21.15.33:0/3363523848 wait complete. 2024-05-03T18:25:14.860 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:25:14 smithi033 ceph-mon[28330]: pgmap v466: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:25:14.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:25:14 smithi150 ceph-mon[32492]: pgmap v466: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:25:15.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:25:14 smithi099 ceph-mon[31658]: pgmap v466: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:25:15.801 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:25:16.261 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:25:16.272 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:25:16.275 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:25:16.276 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:25:16.311 INFO:teuthology.orchestra.run.smithi033.stdout:154618822752 2024-05-03T18:25:16.311 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-05-03T18:25:16.762 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:25:16 smithi033 ceph-mon[28330]: pgmap v467: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:25:16.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:25:16 smithi150 ceph-mon[32492]: pgmap v467: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:25:17.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:25:16 smithi099 ceph-mon[31658]: pgmap v467: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:25:18.735 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:25:18 smithi033 ceph-mon[28330]: pgmap v468: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:25:18.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:25:18 smithi150 ceph-mon[32492]: pgmap v468: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:25:19.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:25:18 smithi099 ceph-mon[31658]: pgmap v468: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:25:19.988 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:25:19.995 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:25:20.714 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:25:20 smithi033 ceph-mon[28330]: pgmap v469: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:25:20.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:25:20 smithi150 ceph-mon[32492]: pgmap v469: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:25:21.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:25:20 smithi099 ceph-mon[31658]: pgmap v469: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:25:22.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:25:22 smithi150 ceph-mon[32492]: pgmap v470: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:25:22.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:25:22 smithi033 ceph-mon[28330]: pgmap v470: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:25:23.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:25:22 smithi099 ceph-mon[31658]: pgmap v470: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:25:24.693 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:25:24 smithi033 ceph-mon[28330]: pgmap v471: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 85 B/s wr, 2 op/s 2024-05-03T18:25:24.693 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:25:24 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/52891993' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-03T18:25:24.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:25:24 smithi150 ceph-mon[32492]: pgmap v471: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 85 B/s wr, 2 op/s 2024-05-03T18:25:24.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:25:24 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/52891993' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-03T18:25:25.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:25.023+0000 7f94d6000700 1 -- 172.21.15.33:0/3684680825 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94d00761f0 msgr2=0x7f94d010f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:25.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:25.023+0000 7f94d6000700 1 --2- 172.21.15.33:0/3684680825 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94d00761f0 0x7f94d010f6d0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f94bc009990 tx=0x7f94bc009ca0 comp rx=0 tx=0).stop 2024-05-03T18:25:25.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:25.023+0000 7f94d6000700 1 -- 172.21.15.33:0/3684680825 shutdown_connections 2024-05-03T18:25:25.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:25.023+0000 7f94d6000700 1 --2- 172.21.15.33:0/3684680825 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94d010fc10 0x7f94d0112080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:25.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:25.023+0000 7f94d6000700 1 --2- 172.21.15.33:0/3684680825 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f94d00758d0 0x7f94d0075cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:25.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:25.023+0000 7f94d6000700 1 --2- 172.21.15.33:0/3684680825 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94d00761f0 0x7f94d010f6d0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:25.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:25.023+0000 7f94d6000700 1 -- 172.21.15.33:0/3684680825 >> 172.21.15.33:0/3684680825 conn(0x7f94d00fef20 msgr2=0x7f94d0101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:25:25.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:25.023+0000 7f94d6000700 1 -- 172.21.15.33:0/3684680825 shutdown_connections 2024-05-03T18:25:25.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:25.023+0000 7f94d6000700 1 -- 172.21.15.33:0/3684680825 wait complete. 2024-05-03T18:25:25.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:25.024+0000 7f94d6000700 1 Processor -- start 2024-05-03T18:25:25.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:25.024+0000 7f94d6000700 1 -- start start 2024-05-03T18:25:25.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:25.025+0000 7f94d6000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94d00758d0 0x7f94d01a1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:25.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:25.025+0000 7f94d6000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f94d00761f0 0x7f94d01a1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:25.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:25.025+0000 7f94d6000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94d010fc10 0x7f94d019ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:25.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:25.025+0000 7f94d6000700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f94d01156a0 con 0x7f94d010fc10 2024-05-03T18:25:25.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:25.025+0000 7f94d6000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f94d0115520 con 0x7f94d00758d0 2024-05-03T18:25:25.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:25.025+0000 7f94d6000700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f94d0115820 con 0x7f94d00761f0 2024-05-03T18:25:25.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:25.025+0000 7f94d5600700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94d010fc10 0x7f94d019ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:25.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:25.025+0000 7f94d5600700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94d010fc10 0x7f94d019ba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53894/0 (socket says 172.21.15.33:53894) 2024-05-03T18:25:25.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:25.025+0000 7f94d5600700 1 -- 172.21.15.33:0/2966773004 learned_addr learned my addr 172.21.15.33:0/2966773004 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:25:25.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:25.025+0000 7f94d4c00700 1 --2- 172.21.15.33:0/2966773004 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94d00758d0 0x7f94d01a1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:25.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:25.025+0000 7f94cfe00700 1 --2- 172.21.15.33:0/2966773004 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f94d00761f0 0x7f94d01a1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:25.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:25.026+0000 7f94d5600700 1 -- 172.21.15.33:0/2966773004 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f94d00761f0 msgr2=0x7f94d01a1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:25.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:25.026+0000 7f94d5600700 1 --2- 172.21.15.33:0/2966773004 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f94d00761f0 0x7f94d01a1dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:25.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:25.026+0000 7f94d5600700 1 -- 172.21.15.33:0/2966773004 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94d00758d0 msgr2=0x7f94d01a1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:25.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:25.026+0000 7f94d5600700 1 --2- 172.21.15.33:0/2966773004 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94d00758d0 0x7f94d01a1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:25.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:25.026+0000 7f94d5600700 1 -- 172.21.15.33:0/2966773004 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94d019c1f0 con 0x7f94d010fc10 2024-05-03T18:25:25.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:25.026+0000 7f94d5600700 1 --2- 172.21.15.33:0/2966773004 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94d010fc10 0x7f94d019ba90 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f94c800ada0 tx=0x7f94c800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:25:25.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:25.027+0000 7f94cd600700 1 -- 172.21.15.33:0/2966773004 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c8017020 con 0x7f94d010fc10 2024-05-03T18:25:25.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:25.027+0000 7f94cd600700 1 -- 172.21.15.33:0/2966773004 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f94c800cd60 con 0x7f94d010fc10 2024-05-03T18:25:25.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:25.027+0000 7f94d6000700 1 -- 172.21.15.33:0/2966773004 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94d019c4e0 con 0x7f94d010fc10 2024-05-03T18:25:25.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:25.027+0000 7f94cd600700 1 -- 172.21.15.33:0/2966773004 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c8005280 con 0x7f94d010fc10 2024-05-03T18:25:25.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:25.027+0000 7f94d6000700 1 -- 172.21.15.33:0/2966773004 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94d01a8750 con 0x7f94d010fc10 2024-05-03T18:25:25.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:25.028+0000 7f94d6000700 1 -- 172.21.15.33:0/2966773004 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94b0005330 con 0x7f94d010fc10 2024-05-03T18:25:25.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:25.030+0000 7f94cd600700 1 -- 172.21.15.33:0/2966773004 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f94c800a090 con 0x7f94d010fc10 2024-05-03T18:25:25.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:25.033+0000 7f94cd600700 1 --2- 172.21.15.33:0/2966773004 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f94b80b2990 0x7f94b80b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:25.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:25.033+0000 7f94cd600700 1 -- 172.21.15.33:0/2966773004 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f94c80e2270 con 0x7f94d010fc10 2024-05-03T18:25:25.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:25.033+0000 7f94d4c00700 1 --2- 172.21.15.33:0/2966773004 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f94b80b2990 0x7f94b80b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:25.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:25.034+0000 7f94d4c00700 1 --2- 172.21.15.33:0/2966773004 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f94b80b2990 0x7f94b80b4e50 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f94c4005950 tx=0x7f94c400a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:25:25.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:25.036+0000 7f94cd600700 1 -- 172.21.15.33:0/2966773004 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f94c8021020 con 0x7f94d010fc10 2024-05-03T18:25:25.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:25:24 smithi099 ceph-mon[31658]: pgmap v471: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 85 B/s wr, 2 op/s 2024-05-03T18:25:25.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:25:24 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/52891993' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-03T18:25:25.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:24.118+0000 7f7536e00700 1 -- 172.21.15.33:0/1177977922 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f753006c3d0 msgr2=0x7f7530110320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:25.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:24.118+0000 7f7536e00700 1 --2- 172.21.15.33:0/1177977922 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f753006c3d0 0x7f7530110320 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f752800b0a0 tx=0x7f752800b3b0 comp rx=0 tx=0).stop 2024-05-03T18:25:25.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:24.119+0000 7f7536e00700 1 -- 172.21.15.33:0/1177977922 shutdown_connections 2024-05-03T18:25:25.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:24.119+0000 7f7536e00700 1 --2- 172.21.15.33:0/1177977922 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f753006c3d0 0x7f7530110320 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:25.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:24.119+0000 7f7536e00700 1 --2- 172.21.15.33:0/1177977922 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7530109500 0x7f75301098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:25.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:24.119+0000 7f7536e00700 1 --2- 172.21.15.33:0/1177977922 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f753006ba10 0x7f753006be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:25.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:24.119+0000 7f7536e00700 1 -- 172.21.15.33:0/1177977922 >> 172.21.15.33:0/1177977922 conn(0x7f7530079310 msgr2=0x7f7530079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:25:25.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:24.119+0000 7f7536e00700 1 -- 172.21.15.33:0/1177977922 shutdown_connections 2024-05-03T18:25:25.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:24.119+0000 7f7536e00700 1 -- 172.21.15.33:0/1177977922 wait complete. 2024-05-03T18:25:25.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:24.120+0000 7f7536e00700 1 Processor -- start 2024-05-03T18:25:25.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:24.121+0000 7f7536e00700 1 -- start start 2024-05-03T18:25:25.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:24.121+0000 7f7536e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f753006ba10 0x7f753019f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:25.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:24.121+0000 7f7536e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f753006c3d0 0x7f753019fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:25.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:24.121+0000 7f7536e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7530109500 0x7f75301999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:25.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:24.121+0000 7f7536e00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7530113590 con 0x7f753006c3d0 2024-05-03T18:25:25.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:24.121+0000 7f7536e00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7530113410 con 0x7f7530109500 2024-05-03T18:25:25.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:24.121+0000 7f7536e00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f7530113710 con 0x7f753006ba10 2024-05-03T18:25:25.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:24.122+0000 7f7527e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f753006c3d0 0x7f753019fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:25.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:24.122+0000 7f7527e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f753006c3d0 0x7f753019fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53874/0 (socket says 172.21.15.33:53874) 2024-05-03T18:25:25.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:24.122+0000 7f7527e00700 1 -- 172.21.15.33:0/52891993 learned_addr learned my addr 172.21.15.33:0/52891993 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:25:25.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:24.122+0000 7f752fe00700 1 --2- 172.21.15.33:0/52891993 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f753006ba10 0x7f753019f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:25.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:24.122+0000 7f7534e00700 1 --2- 172.21.15.33:0/52891993 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7530109500 0x7f75301999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:25.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:24.122+0000 7f7527e00700 1 -- 172.21.15.33:0/52891993 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f753006ba10 msgr2=0x7f753019f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:25.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:24.122+0000 7f7527e00700 1 --2- 172.21.15.33:0/52891993 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f753006ba10 0x7f753019f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:25.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:24.122+0000 7f7527e00700 1 -- 172.21.15.33:0/52891993 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7530109500 msgr2=0x7f75301999b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:25.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:24.122+0000 7f7527e00700 1 --2- 172.21.15.33:0/52891993 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7530109500 0x7f75301999b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:25.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:24.122+0000 7f7527e00700 1 -- 172.21.15.33:0/52891993 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f753019a110 con 0x7f753006c3d0 2024-05-03T18:25:25.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:24.122+0000 7f7534e00700 1 --2- 172.21.15.33:0/52891993 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7530109500 0x7f75301999b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:25:25.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:24.123+0000 7f7527e00700 1 --2- 172.21.15.33:0/52891993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f753006c3d0 0x7f753019fd30 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f751800d950 tx=0x7f751800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:25:25.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:24.123+0000 7f752d600700 1 -- 172.21.15.33:0/52891993 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7518016070 con 0x7f753006c3d0 2024-05-03T18:25:25.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:24.123+0000 7f7536e00700 1 -- 172.21.15.33:0/52891993 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f753019a3a0 con 0x7f753006c3d0 2024-05-03T18:25:25.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:24.123+0000 7f752d600700 1 -- 172.21.15.33:0/52891993 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7518010460 con 0x7f753006c3d0 2024-05-03T18:25:25.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:24.123+0000 7f752d600700 1 -- 172.21.15.33:0/52891993 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f751800f620 con 0x7f753006c3d0 2024-05-03T18:25:25.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:24.123+0000 7f7536e00700 1 -- 172.21.15.33:0/52891993 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75301a66c0 con 0x7f753006c3d0 2024-05-03T18:25:25.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:24.125+0000 7f7536e00700 1 -- 172.21.15.33:0/52891993 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f753010b9e0 con 0x7f753006c3d0 2024-05-03T18:25:25.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:24.127+0000 7f752d600700 1 -- 172.21.15.33:0/52891993 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f7518019070 con 0x7f753006c3d0 2024-05-03T18:25:25.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:24.128+0000 7f752d600700 1 --2- 172.21.15.33:0/52891993 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f75140b2990 0x7f75140b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:25.212 INFO:teuthology.orchestra.run.smithi033.stdout:55834574976 2024-05-03T18:25:25.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:24.128+0000 7f752d600700 1 -- 172.21.15.33:0/52891993 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f75180e2380 con 0x7f753006c3d0 2024-05-03T18:25:25.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:24.129+0000 7f752fe00700 1 --2- 172.21.15.33:0/52891993 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f75140b2990 0x7f75140b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:25.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:24.129+0000 7f752fe00700 1 --2- 172.21.15.33:0/52891993 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f75140b2990 0x7f75140b4e50 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7520006fd0 tx=0x7f7520008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:25:25.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:24.133+0000 7f752d600700 1 -- 172.21.15.33:0/52891993 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f75180a9b60 con 0x7f753006c3d0 2024-05-03T18:25:25.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:24.382+0000 7f7536e00700 1 -- 172.21.15.33:0/52891993 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f753019aee0 con 0x7f753006c3d0 2024-05-03T18:25:25.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:24.383+0000 7f752d600700 1 -- 172.21.15.33:0/52891993 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f75180a9460 con 0x7f753006c3d0 2024-05-03T18:25:25.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:24.386+0000 7f7536e00700 1 -- 172.21.15.33:0/52891993 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f75140b2990 msgr2=0x7f75140b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:25.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:24.387+0000 7f7536e00700 1 --2- 172.21.15.33:0/52891993 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f75140b2990 0x7f75140b4e50 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7520006fd0 tx=0x7f7520008040 comp rx=0 tx=0).stop 2024-05-03T18:25:25.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:24.387+0000 7f7536e00700 1 -- 172.21.15.33:0/52891993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f753006c3d0 msgr2=0x7f753019fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:25.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:24.387+0000 7f7536e00700 1 --2- 172.21.15.33:0/52891993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f753006c3d0 0x7f753019fd30 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f751800d950 tx=0x7f751800dd10 comp rx=0 tx=0).stop 2024-05-03T18:25:25.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:24.387+0000 7f7536e00700 1 -- 172.21.15.33:0/52891993 shutdown_connections 2024-05-03T18:25:25.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:24.387+0000 7f7536e00700 1 --2- 172.21.15.33:0/52891993 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f75140b2990 0x7f75140b4e50 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:25.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:24.387+0000 7f7536e00700 1 --2- 172.21.15.33:0/52891993 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7530109500 0x7f75301999b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:25.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:24.387+0000 7f7536e00700 1 --2- 172.21.15.33:0/52891993 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f753006ba10 0x7f753019f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:25.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:24.387+0000 7f7536e00700 1 --2- 172.21.15.33:0/52891993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f753006c3d0 0x7f753019fd30 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:25.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:24.387+0000 7f7536e00700 1 -- 172.21.15.33:0/52891993 >> 172.21.15.33:0/52891993 conn(0x7f7530079310 msgr2=0x7f75301011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:25:25.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:24.387+0000 7f7536e00700 1 -- 172.21.15.33:0/52891993 shutdown_connections 2024-05-03T18:25:25.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:24.387+0000 7f7536e00700 1 -- 172.21.15.33:0/52891993 wait complete. 2024-05-03T18:25:25.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:25.287+0000 7f94d6000700 1 -- 172.21.15.33:0/2966773004 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f94b00058f0 con 0x7f94d010fc10 2024-05-03T18:25:25.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:25.287+0000 7f94cd600700 1 -- 172.21.15.33:0/2966773004 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f94c8004ad0 con 0x7f94d010fc10 2024-05-03T18:25:25.287 INFO:teuthology.orchestra.run.smithi033.stdout:133143986282 2024-05-03T18:25:25.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:25.290+0000 7f94d6000700 1 -- 172.21.15.33:0/2966773004 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f94b80b2990 msgr2=0x7f94b80b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:25.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:25.290+0000 7f94d6000700 1 --2- 172.21.15.33:0/2966773004 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f94b80b2990 0x7f94b80b4e50 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f94c4005950 tx=0x7f94c400a300 comp rx=0 tx=0).stop 2024-05-03T18:25:25.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:25.290+0000 7f94d6000700 1 -- 172.21.15.33:0/2966773004 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94d010fc10 msgr2=0x7f94d019ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:25.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:25.290+0000 7f94d6000700 1 --2- 172.21.15.33:0/2966773004 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94d010fc10 0x7f94d019ba90 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f94c800ada0 tx=0x7f94c800c330 comp rx=0 tx=0).stop 2024-05-03T18:25:25.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:25.290+0000 7f94d6000700 1 -- 172.21.15.33:0/2966773004 shutdown_connections 2024-05-03T18:25:25.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:25.290+0000 7f94d6000700 1 --2- 172.21.15.33:0/2966773004 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94d010fc10 0x7f94d019ba90 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:25.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:25.290+0000 7f94d6000700 1 --2- 172.21.15.33:0/2966773004 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f94b80b2990 0x7f94b80b4e50 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:25.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:25.290+0000 7f94d6000700 1 --2- 172.21.15.33:0/2966773004 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94d00758d0 0x7f94d01a1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:25.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:25.290+0000 7f94d6000700 1 --2- 172.21.15.33:0/2966773004 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f94d00761f0 0x7f94d01a1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:25.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:25.290+0000 7f94d6000700 1 -- 172.21.15.33:0/2966773004 >> 172.21.15.33:0/2966773004 conn(0x7f94d00fef20 msgr2=0x7f94d00ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:25:25.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:25.291+0000 7f94d6000700 1 -- 172.21.15.33:0/2966773004 shutdown_connections 2024-05-03T18:25:25.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:25.291+0000 7f94d6000700 1 -- 172.21.15.33:0/2966773004 wait complete. 2024-05-03T18:25:25.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:25:25 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/2966773004' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-03T18:25:25.906 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:25:25 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/2966773004' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-03T18:25:26.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:25:25 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/2966773004' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-03T18:25:26.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:26.211+0000 7fe465400700 1 -- 172.21.15.33:0/2579723787 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe46006bb40 msgr2=0x7fe460110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:26.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:26.211+0000 7fe465400700 1 --2- 172.21.15.33:0/2579723787 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe46006bb40 0x7fe460110330 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe45800b0a0 tx=0x7fe45800b3b0 comp rx=0 tx=0).stop 2024-05-03T18:25:26.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:26.211+0000 7fe465400700 1 -- 172.21.15.33:0/2579723787 shutdown_connections 2024-05-03T18:25:26.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:26.211+0000 7fe465400700 1 --2- 172.21.15.33:0/2579723787 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe46006bb40 0x7fe460110330 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:26.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:26.211+0000 7fe465400700 1 --2- 172.21.15.33:0/2579723787 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe460109510 0x7fe4601098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:26.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:26.211+0000 7fe465400700 1 --2- 172.21.15.33:0/2579723787 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe46006b180 0x7fe46006b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:26.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:26.211+0000 7fe465400700 1 -- 172.21.15.33:0/2579723787 >> 172.21.15.33:0/2579723787 conn(0x7fe460079350 msgr2=0x7fe460079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:25:26.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:26.211+0000 7fe465400700 1 -- 172.21.15.33:0/2579723787 shutdown_connections 2024-05-03T18:25:26.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:26.211+0000 7fe465400700 1 -- 172.21.15.33:0/2579723787 wait complete. 2024-05-03T18:25:26.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:26.212+0000 7fe465400700 1 Processor -- start 2024-05-03T18:25:26.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:26.213+0000 7fe465400700 1 -- start start 2024-05-03T18:25:26.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:26.213+0000 7fe465400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe46006b180 0x7fe4601a18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:26.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:26.213+0000 7fe465400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe46006bb40 0x7fe4601a1e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:26.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:26.213+0000 7fe465400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe460109510 0x7fe46019baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:26.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:26.213+0000 7fe465400700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe4601134f0 con 0x7fe460109510 2024-05-03T18:25:26.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:26.213+0000 7fe465400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe460113370 con 0x7fe46006b180 2024-05-03T18:25:26.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:26.213+0000 7fe465400700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe460113670 con 0x7fe46006bb40 2024-05-03T18:25:26.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:26.213+0000 7fe45f200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe460109510 0x7fe46019baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:26.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:26.213+0000 7fe45f200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe460109510 0x7fe46019baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60566/0 (socket says 172.21.15.33:60566) 2024-05-03T18:25:26.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:26.214+0000 7fe45f200700 1 -- 172.21.15.33:0/4259555943 learned_addr learned my addr 172.21.15.33:0/4259555943 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:25:26.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:26.214+0000 7fe45e800700 1 --2- 172.21.15.33:0/4259555943 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe46006b180 0x7fe4601a18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:26.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:26.214+0000 7fe457e00700 1 --2- 172.21.15.33:0/4259555943 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe46006bb40 0x7fe4601a1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:26.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:26.214+0000 7fe45f200700 1 -- 172.21.15.33:0/4259555943 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe46006bb40 msgr2=0x7fe4601a1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:26.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:26.214+0000 7fe45f200700 1 --2- 172.21.15.33:0/4259555943 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe46006bb40 0x7fe4601a1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:26.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:26.214+0000 7fe45f200700 1 -- 172.21.15.33:0/4259555943 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe46006b180 msgr2=0x7fe4601a18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:26.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:26.214+0000 7fe45f200700 1 --2- 172.21.15.33:0/4259555943 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe46006b180 0x7fe4601a18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:26.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:26.214+0000 7fe45f200700 1 -- 172.21.15.33:0/4259555943 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe46019c250 con 0x7fe460109510 2024-05-03T18:25:26.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:26.214+0000 7fe45f200700 1 --2- 172.21.15.33:0/4259555943 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe460109510 0x7fe46019baf0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fe45800b830 tx=0x7fe458004db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:25:26.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:26.215+0000 7fe457400700 1 -- 172.21.15.33:0/4259555943 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe458021070 con 0x7fe460109510 2024-05-03T18:25:26.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:26.215+0000 7fe457400700 1 -- 172.21.15.33:0/4259555943 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe458007c80 con 0x7fe460109510 2024-05-03T18:25:26.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:26.215+0000 7fe457400700 1 -- 172.21.15.33:0/4259555943 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe458005260 con 0x7fe460109510 2024-05-03T18:25:26.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:26.215+0000 7fe465400700 1 -- 172.21.15.33:0/4259555943 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe46019c4e0 con 0x7fe460109510 2024-05-03T18:25:26.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:26.215+0000 7fe465400700 1 -- 172.21.15.33:0/4259555943 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4601a87b0 con 0x7fe460109510 2024-05-03T18:25:26.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:26.216+0000 7fe465400700 1 -- 172.21.15.33:0/4259555943 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe460104bd0 con 0x7fe460109510 2024-05-03T18:25:26.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:26.218+0000 7fe457400700 1 -- 172.21.15.33:0/4259555943 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fe458024070 con 0x7fe460109510 2024-05-03T18:25:26.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:26.221+0000 7fe457400700 1 --2- 172.21.15.33:0/4259555943 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fe4440b2990 0x7fe4440b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:26.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:26.221+0000 7fe457400700 1 -- 172.21.15.33:0/4259555943 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7fe45800f020 con 0x7fe460109510 2024-05-03T18:25:26.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:26.221+0000 7fe45e800700 1 --2- 172.21.15.33:0/4259555943 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fe4440b2990 0x7fe4440b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:26.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:26.222+0000 7fe45e800700 1 --2- 172.21.15.33:0/4259555943 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fe4440b2990 0x7fe4440b4e50 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe450005e10 tx=0x7fe450005d40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:25:26.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:26.224+0000 7fe457400700 1 -- 172.21.15.33:0/4259555943 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fe45802b020 con 0x7fe460109510 2024-05-03T18:25:26.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:26.466+0000 7fe465400700 1 -- 172.21.15.33:0/4259555943 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fe4601a8980 con 0x7fe460109510 2024-05-03T18:25:26.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:26.467+0000 7fe457400700 1 -- 172.21.15.33:0/4259555943 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fe4580aba40 con 0x7fe460109510 2024-05-03T18:25:26.467 INFO:teuthology.orchestra.run.smithi033.stdout:107374182512 2024-05-03T18:25:26.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:26.470+0000 7fe465400700 1 -- 172.21.15.33:0/4259555943 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fe4440b2990 msgr2=0x7fe4440b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:26.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:26.470+0000 7fe465400700 1 --2- 172.21.15.33:0/4259555943 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fe4440b2990 0x7fe4440b4e50 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe450005e10 tx=0x7fe450005d40 comp rx=0 tx=0).stop 2024-05-03T18:25:26.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:26.470+0000 7fe465400700 1 -- 172.21.15.33:0/4259555943 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe460109510 msgr2=0x7fe46019baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:26.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:26.470+0000 7fe465400700 1 --2- 172.21.15.33:0/4259555943 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe460109510 0x7fe46019baf0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fe45800b830 tx=0x7fe458004db0 comp rx=0 tx=0).stop 2024-05-03T18:25:26.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:26.470+0000 7fe465400700 1 -- 172.21.15.33:0/4259555943 shutdown_connections 2024-05-03T18:25:26.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:26.470+0000 7fe465400700 1 --2- 172.21.15.33:0/4259555943 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe460109510 0x7fe46019baf0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:26.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:26.471+0000 7fe465400700 1 --2- 172.21.15.33:0/4259555943 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fe4440b2990 0x7fe4440b4e50 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:26.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:26.471+0000 7fe465400700 1 --2- 172.21.15.33:0/4259555943 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe46006b180 0x7fe4601a18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:26.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:26.471+0000 7fe465400700 1 --2- 172.21.15.33:0/4259555943 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe46006bb40 0x7fe4601a1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:26.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:26.471+0000 7fe465400700 1 -- 172.21.15.33:0/4259555943 >> 172.21.15.33:0/4259555943 conn(0x7fe460079350 msgr2=0x7fe460100e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:25:26.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:26.471+0000 7fe465400700 1 -- 172.21.15.33:0/4259555943 shutdown_connections 2024-05-03T18:25:26.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:26.471+0000 7fe465400700 1 -- 172.21.15.33:0/4259555943 wait complete. 2024-05-03T18:25:26.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:25:26 smithi033 ceph-mon[28330]: pgmap v472: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 85 B/s wr, 1 op/s 2024-05-03T18:25:26.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:25:26 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/4259555943' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-03T18:25:27.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:25:26 smithi099 ceph-mon[31658]: pgmap v472: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 85 B/s wr, 1 op/s 2024-05-03T18:25:27.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:25:26 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/4259555943' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-03T18:25:27.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:25:26 smithi150 ceph-mon[32492]: pgmap v472: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 85 B/s wr, 1 op/s 2024-05-03T18:25:27.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:25:26 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/4259555943' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-03T18:25:28.138 INFO:tasks.cephadm.ceph_manager.ceph:need seq 133143986277 got 133143986282 for osd.4 2024-05-03T18:25:28.139 DEBUG:teuthology.parallel:result is None 2024-05-03T18:25:28.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:28.587+0000 7f80d1600700 1 -- 172.21.15.33:0/3960455121 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80cc108460 msgr2=0x7f80cc108840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:28.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:28.587+0000 7f80d1600700 1 --2- 172.21.15.33:0/3960455121 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80cc108460 0x7f80cc108840 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f80bc009990 tx=0x7f80bc009ca0 comp rx=0 tx=0).stop 2024-05-03T18:25:28.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:28.587+0000 7f80d1600700 1 -- 172.21.15.33:0/3960455121 shutdown_connections 2024-05-03T18:25:28.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:28.587+0000 7f80d1600700 1 --2- 172.21.15.33:0/3960455121 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f80cc103ef0 0x7f80cc1103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:28.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:28.587+0000 7f80d1600700 1 --2- 172.21.15.33:0/3960455121 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80cc108460 0x7f80cc108840 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:28.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:28.587+0000 7f80d1600700 1 --2- 172.21.15.33:0/3960455121 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80cc103550 0x7f80cc1039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:28.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:28.587+0000 7f80d1600700 1 -- 172.21.15.33:0/3960455121 >> 172.21.15.33:0/3960455121 conn(0x7f80cc07a600 msgr2=0x7f80cc07aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:25:28.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:28.587+0000 7f80d1600700 1 -- 172.21.15.33:0/3960455121 shutdown_connections 2024-05-03T18:25:28.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:28.587+0000 7f80d1600700 1 -- 172.21.15.33:0/3960455121 wait complete. 2024-05-03T18:25:28.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:28.588+0000 7f80d1600700 1 Processor -- start 2024-05-03T18:25:28.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:28.588+0000 7f80d1600700 1 -- start start 2024-05-03T18:25:28.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:28.589+0000 7f80d1600700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f80cc103550 0x7f80cc19bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:28.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:28.589+0000 7f80d1600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80cc103ef0 0x7f80cc19c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:28.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:28.589+0000 7f80d1600700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80cc108460 0x7f80cc1a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:28.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:28.589+0000 7f80d1600700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f80cc113590 con 0x7f80cc108460 2024-05-03T18:25:28.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:28.589+0000 7f80d1600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f80cc113410 con 0x7f80cc103ef0 2024-05-03T18:25:28.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:28.589+0000 7f80d1600700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f80cc113710 con 0x7f80cc103550 2024-05-03T18:25:28.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:28.589+0000 7f80cb400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80cc108460 0x7f80cc1a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:28.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:28.589+0000 7f80cb400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80cc108460 0x7f80cc1a0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60574/0 (socket says 172.21.15.33:60574) 2024-05-03T18:25:28.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:28.589+0000 7f80cb400700 1 -- 172.21.15.33:0/398943580 learned_addr learned my addr 172.21.15.33:0/398943580 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:25:28.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:28.589+0000 7f80caa00700 1 --2- 172.21.15.33:0/398943580 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f80cc103550 0x7f80cc19bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:28.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:28.589+0000 7f80ca000700 1 --2- 172.21.15.33:0/398943580 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80cc103ef0 0x7f80cc19c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:28.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:28.590+0000 7f80cb400700 1 -- 172.21.15.33:0/398943580 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f80cc103550 msgr2=0x7f80cc19bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:28.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:28.590+0000 7f80cb400700 1 --2- 172.21.15.33:0/398943580 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f80cc103550 0x7f80cc19bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:28.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:28.590+0000 7f80cb400700 1 -- 172.21.15.33:0/398943580 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80cc103ef0 msgr2=0x7f80cc19c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:28.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:28.590+0000 7f80cb400700 1 --2- 172.21.15.33:0/398943580 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80cc103ef0 0x7f80cc19c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:28.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:28.590+0000 7f80cb400700 1 -- 172.21.15.33:0/398943580 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80cc1a0d70 con 0x7f80cc108460 2024-05-03T18:25:28.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:28.590+0000 7f80ca000700 1 --2- 172.21.15.33:0/398943580 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80cc103ef0 0x7f80cc19c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:25:28.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:28.590+0000 7f80caa00700 1 --2- 172.21.15.33:0/398943580 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f80cc103550 0x7f80cc19bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:25:28.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:28.590+0000 7f80cb400700 1 --2- 172.21.15.33:0/398943580 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80cc108460 0x7f80cc1a0630 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f80c400ab50 tx=0x7f80c400af10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:25:28.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:28.590+0000 7f80c3400700 1 -- 172.21.15.33:0/398943580 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c4017020 con 0x7f80cc108460 2024-05-03T18:25:28.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:28.590+0000 7f80c3400700 1 -- 172.21.15.33:0/398943580 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f80c4004bb0 con 0x7f80cc108460 2024-05-03T18:25:28.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:28.590+0000 7f80d1600700 1 -- 172.21.15.33:0/398943580 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80cc1a1060 con 0x7f80cc108460 2024-05-03T18:25:28.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:28.590+0000 7f80c3400700 1 -- 172.21.15.33:0/398943580 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c4005280 con 0x7f80cc108460 2024-05-03T18:25:28.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:28.591+0000 7f80d1600700 1 -- 172.21.15.33:0/398943580 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80cc1a88e0 con 0x7f80cc108460 2024-05-03T18:25:28.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:28.592+0000 7f80d1600700 1 -- 172.21.15.33:0/398943580 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80cc06a190 con 0x7f80cc108460 2024-05-03T18:25:28.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:28.594+0000 7f80c3400700 1 -- 172.21.15.33:0/398943580 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f80c400c6d0 con 0x7f80cc108460 2024-05-03T18:25:28.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:28.597+0000 7f80c3400700 1 --2- 172.21.15.33:0/398943580 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f80b00b2910 0x7f80b00b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:28.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:28.597+0000 7f80c3400700 1 -- 172.21.15.33:0/398943580 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f80c40e2350 con 0x7f80cc108460 2024-05-03T18:25:28.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:28.597+0000 7f80caa00700 1 --2- 172.21.15.33:0/398943580 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f80b00b2910 0x7f80b00b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:28.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:28.598+0000 7f80caa00700 1 --2- 172.21.15.33:0/398943580 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f80b00b2910 0x7f80b00b4dd0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f80bc006010 tx=0x7f80bc01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:25:28.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:28.599+0000 7f80c3400700 1 -- 172.21.15.33:0/398943580 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f80c4021020 con 0x7f80cc108460 2024-05-03T18:25:28.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:25:28 smithi033 ceph-mon[28330]: pgmap v473: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 85 B/s wr, 1 op/s 2024-05-03T18:25:28.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:28.854+0000 7f80d1600700 1 -- 172.21.15.33:0/398943580 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f80cc1a8ab0 con 0x7f80cc108460 2024-05-03T18:25:28.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:28.854+0000 7f80c3400700 1 -- 172.21.15.33:0/398943580 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f80c40a9bc0 con 0x7f80cc108460 2024-05-03T18:25:28.855 INFO:teuthology.orchestra.run.smithi033.stdout:34359738503 2024-05-03T18:25:28.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:28.858+0000 7f80d1600700 1 -- 172.21.15.33:0/398943580 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f80b00b2910 msgr2=0x7f80b00b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:28.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:28.858+0000 7f80d1600700 1 --2- 172.21.15.33:0/398943580 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f80b00b2910 0x7f80b00b4dd0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f80bc006010 tx=0x7f80bc01a040 comp rx=0 tx=0).stop 2024-05-03T18:25:28.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:28.858+0000 7f80d1600700 1 -- 172.21.15.33:0/398943580 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80cc108460 msgr2=0x7f80cc1a0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:28.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:28.858+0000 7f80d1600700 1 --2- 172.21.15.33:0/398943580 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80cc108460 0x7f80cc1a0630 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f80c400ab50 tx=0x7f80c400af10 comp rx=0 tx=0).stop 2024-05-03T18:25:28.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:28.858+0000 7f80d1600700 1 -- 172.21.15.33:0/398943580 shutdown_connections 2024-05-03T18:25:28.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:28.858+0000 7f80d1600700 1 --2- 172.21.15.33:0/398943580 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80cc108460 0x7f80cc1a0630 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:28.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:28.858+0000 7f80d1600700 1 --2- 172.21.15.33:0/398943580 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f80cc103550 0x7f80cc19bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:28.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:28.858+0000 7f80d1600700 1 --2- 172.21.15.33:0/398943580 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f80b00b2910 0x7f80b00b4dd0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:28.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:28.858+0000 7f80d1600700 1 --2- 172.21.15.33:0/398943580 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80cc103ef0 0x7f80cc19c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:28.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:28.858+0000 7f80d1600700 1 -- 172.21.15.33:0/398943580 >> 172.21.15.33:0/398943580 conn(0x7f80cc07a600 msgr2=0x7f80cc1013c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:25:28.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:28.858+0000 7f80d1600700 1 -- 172.21.15.33:0/398943580 shutdown_connections 2024-05-03T18:25:28.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:28.859+0000 7f80d1600700 1 -- 172.21.15.33:0/398943580 wait complete. 2024-05-03T18:25:29.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:25:28 smithi099 ceph-mon[31658]: pgmap v473: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 85 B/s wr, 1 op/s 2024-05-03T18:25:29.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:25:28 smithi150 ceph-mon[32492]: pgmap v473: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 85 B/s wr, 1 op/s 2024-05-03T18:25:29.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:25:29 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/398943580' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-03T18:25:30.011 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574971 got 55834574976 for osd.1 2024-05-03T18:25:30.011 DEBUG:teuthology.parallel:result is None 2024-05-03T18:25:30.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:25:29 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/398943580' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-03T18:25:30.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:25:29 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/398943580' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-03T18:25:30.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:25:30 smithi033 ceph-mon[28330]: pgmap v474: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 85 B/s wr, 2 op/s 2024-05-03T18:25:31.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:25:30 smithi099 ceph-mon[31658]: pgmap v474: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 85 B/s wr, 2 op/s 2024-05-03T18:25:31.131 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:25:30 smithi150 ceph-mon[32492]: pgmap v474: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 85 B/s wr, 2 op/s 2024-05-03T18:25:31.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:31.308+0000 7f9d31200700 1 -- 172.21.15.33:0/2027903485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d2c103750 msgr2=0x7f9d2c103b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:31.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:31.308+0000 7f9d31200700 1 --2- 172.21.15.33:0/2027903485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d2c103750 0x7f9d2c103b30 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c0099c0 tx=0x7f9d1c009cd0 comp rx=0 tx=0).stop 2024-05-03T18:25:31.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:31.308+0000 7f9d31200700 1 -- 172.21.15.33:0/2027903485 shutdown_connections 2024-05-03T18:25:31.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:31.308+0000 7f9d31200700 1 --2- 172.21.15.33:0/2027903485 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d2c108850 0x7f9d2c1103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:31.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:31.308+0000 7f9d31200700 1 --2- 172.21.15.33:0/2027903485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d2c103750 0x7f9d2c103b30 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:31.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:31.308+0000 7f9d31200700 1 --2- 172.21.15.33:0/2027903485 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d2c104100 0x7f9d2c108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:31.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:31.308+0000 7f9d31200700 1 -- 172.21.15.33:0/2027903485 >> 172.21.15.33:0/2027903485 conn(0x7f9d2c07a600 msgr2=0x7f9d2c07aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:25:31.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:31.308+0000 7f9d31200700 1 -- 172.21.15.33:0/2027903485 shutdown_connections 2024-05-03T18:25:31.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:31.308+0000 7f9d31200700 1 -- 172.21.15.33:0/2027903485 wait complete. 2024-05-03T18:25:31.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:31.309+0000 7f9d31200700 1 Processor -- start 2024-05-03T18:25:31.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:31.309+0000 7f9d31200700 1 -- start start 2024-05-03T18:25:31.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:31.310+0000 7f9d31200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d2c103750 0x7f9d2c1a1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:31.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:31.310+0000 7f9d31200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d2c104100 0x7f9d2c1a1d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:31.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:31.310+0000 7f9d31200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d2c108850 0x7f9d2c19ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:31.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:31.310+0000 7f9d31200700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9d2c113540 con 0x7f9d2c103750 2024-05-03T18:25:31.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:31.310+0000 7f9d31200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9d2c1133c0 con 0x7f9d2c108850 2024-05-03T18:25:31.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:31.310+0000 7f9d31200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9d2c1136c0 con 0x7f9d2c104100 2024-05-03T18:25:31.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:31.310+0000 7f9d29e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d2c103750 0x7f9d2c1a1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:31.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:31.310+0000 7f9d29e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d2c103750 0x7f9d2c1a1840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60588/0 (socket says 172.21.15.33:60588) 2024-05-03T18:25:31.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:31.310+0000 7f9d29e00700 1 -- 172.21.15.33:0/1257096600 learned_addr learned my addr 172.21.15.33:0/1257096600 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:25:31.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:31.310+0000 7f9d2a800700 1 --2- 172.21.15.33:0/1257096600 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d2c108850 0x7f9d2c19ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:31.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:31.310+0000 7f9d23e00700 1 --2- 172.21.15.33:0/1257096600 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d2c104100 0x7f9d2c1a1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:31.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:31.311+0000 7f9d29e00700 1 -- 172.21.15.33:0/1257096600 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d2c104100 msgr2=0x7f9d2c1a1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:31.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:31.311+0000 7f9d29e00700 1 --2- 172.21.15.33:0/1257096600 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d2c104100 0x7f9d2c1a1d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:31.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:31.311+0000 7f9d29e00700 1 -- 172.21.15.33:0/1257096600 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d2c108850 msgr2=0x7f9d2c19ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:31.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:31.311+0000 7f9d29e00700 1 --2- 172.21.15.33:0/1257096600 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d2c108850 0x7f9d2c19ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:31.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:31.311+0000 7f9d29e00700 1 -- 172.21.15.33:0/1257096600 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d2c19c160 con 0x7f9d2c103750 2024-05-03T18:25:31.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:31.311+0000 7f9d29e00700 1 --2- 172.21.15.33:0/1257096600 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d2c103750 0x7f9d2c1a1840 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c000c00 tx=0x7f9d1c00f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:25:31.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:31.311+0000 7f9d22a00700 1 -- 172.21.15.33:0/1257096600 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d1c027070 con 0x7f9d2c103750 2024-05-03T18:25:31.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:31.311+0000 7f9d22a00700 1 -- 172.21.15.33:0/1257096600 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9d1c0056d0 con 0x7f9d2c103750 2024-05-03T18:25:31.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:31.311+0000 7f9d22a00700 1 -- 172.21.15.33:0/1257096600 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d1c0173f0 con 0x7f9d2c103750 2024-05-03T18:25:31.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:31.311+0000 7f9d31200700 1 -- 172.21.15.33:0/1257096600 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d2c19c3f0 con 0x7f9d2c103750 2024-05-03T18:25:31.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:31.312+0000 7f9d31200700 1 -- 172.21.15.33:0/1257096600 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d2c1a8710 con 0x7f9d2c103750 2024-05-03T18:25:31.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:31.313+0000 7f9d31200700 1 -- 172.21.15.33:0/1257096600 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d2c104be0 con 0x7f9d2c103750 2024-05-03T18:25:31.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:31.315+0000 7f9d22a00700 1 -- 172.21.15.33:0/1257096600 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f9d1c02a070 con 0x7f9d2c103750 2024-05-03T18:25:31.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:31.317+0000 7f9d22a00700 1 --2- 172.21.15.33:0/1257096600 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f9d100b2990 0x7f9d100b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:31.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:31.317+0000 7f9d22a00700 1 -- 172.21.15.33:0/1257096600 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f9d1c0e33d0 con 0x7f9d2c103750 2024-05-03T18:25:31.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:31.317+0000 7f9d23e00700 1 --2- 172.21.15.33:0/1257096600 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f9d100b2990 0x7f9d100b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:31.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:31.318+0000 7f9d23e00700 1 --2- 172.21.15.33:0/1257096600 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f9d100b2990 0x7f9d100b4e50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c103260 tx=0x7f9d14005eb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:25:31.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:31.321+0000 7f9d22a00700 1 -- 172.21.15.33:0/1257096600 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f9d1c013070 con 0x7f9d2c103750 2024-05-03T18:25:31.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:31.567+0000 7f9d31200700 1 -- 172.21.15.33:0/1257096600 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f9d2c1a88e0 con 0x7f9d2c103750 2024-05-03T18:25:31.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:31.568+0000 7f9d22a00700 1 -- 172.21.15.33:0/1257096600 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f9d1c0aac40 con 0x7f9d2c103750 2024-05-03T18:25:31.569 INFO:teuthology.orchestra.run.smithi033.stdout:180388626526 2024-05-03T18:25:31.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:31.571+0000 7f9d31200700 1 -- 172.21.15.33:0/1257096600 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f9d100b2990 msgr2=0x7f9d100b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:31.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:31.571+0000 7f9d31200700 1 --2- 172.21.15.33:0/1257096600 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f9d100b2990 0x7f9d100b4e50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c103260 tx=0x7f9d14005eb0 comp rx=0 tx=0).stop 2024-05-03T18:25:31.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:31.571+0000 7f9d31200700 1 -- 172.21.15.33:0/1257096600 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d2c103750 msgr2=0x7f9d2c1a1840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:31.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:31.571+0000 7f9d31200700 1 --2- 172.21.15.33:0/1257096600 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d2c103750 0x7f9d2c1a1840 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c000c00 tx=0x7f9d1c00f740 comp rx=0 tx=0).stop 2024-05-03T18:25:31.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:31.571+0000 7f9d31200700 1 -- 172.21.15.33:0/1257096600 shutdown_connections 2024-05-03T18:25:31.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:31.571+0000 7f9d31200700 1 --2- 172.21.15.33:0/1257096600 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f9d100b2990 0x7f9d100b4e50 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:31.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:31.571+0000 7f9d31200700 1 --2- 172.21.15.33:0/1257096600 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d2c108850 0x7f9d2c19ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:31.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:31.571+0000 7f9d31200700 1 --2- 172.21.15.33:0/1257096600 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d2c103750 0x7f9d2c1a1840 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:31.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:31.571+0000 7f9d31200700 1 --2- 172.21.15.33:0/1257096600 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d2c104100 0x7f9d2c1a1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:31.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:31.571+0000 7f9d31200700 1 -- 172.21.15.33:0/1257096600 >> 172.21.15.33:0/1257096600 conn(0x7f9d2c07a600 msgr2=0x7f9d2c1018e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:25:31.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:31.571+0000 7f9d31200700 1 -- 172.21.15.33:0/1257096600 shutdown_connections 2024-05-03T18:25:31.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:31.572+0000 7f9d31200700 1 -- 172.21.15.33:0/1257096600 wait complete. 2024-05-03T18:25:31.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:25:31 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:25:31.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:25:31 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/1257096600' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-03T18:25:32.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:25:31 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:25:32.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:25:31 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/1257096600' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-03T18:25:32.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:25:31 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:25:32.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:25:31 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/1257096600' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-03T18:25:32.155 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182507 got 107374182512 for osd.3 2024-05-03T18:25:32.156 DEBUG:teuthology.parallel:result is None 2024-05-03T18:25:32.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:31.765+0000 7f0682a00700 1 -- 172.21.15.33:0/3040724102 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f067c103750 msgr2=0x7f067c103b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:32.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:31.765+0000 7f0682a00700 1 --2- 172.21.15.33:0/3040724102 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f067c103750 0x7f067c103b30 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f066c0099c0 tx=0x7f066c009cd0 comp rx=0 tx=0).stop 2024-05-03T18:25:32.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:31.765+0000 7f0682a00700 1 -- 172.21.15.33:0/3040724102 shutdown_connections 2024-05-03T18:25:32.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:31.765+0000 7f0682a00700 1 --2- 172.21.15.33:0/3040724102 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f067c108850 0x7f067c1103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:32.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:31.765+0000 7f0682a00700 1 --2- 172.21.15.33:0/3040724102 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f067c103750 0x7f067c103b30 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:32.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:31.765+0000 7f0682a00700 1 --2- 172.21.15.33:0/3040724102 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f067c104100 0x7f067c108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:32.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:31.765+0000 7f0682a00700 1 -- 172.21.15.33:0/3040724102 >> 172.21.15.33:0/3040724102 conn(0x7f067c07a600 msgr2=0x7f067c07aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:25:32.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:31.765+0000 7f0682a00700 1 -- 172.21.15.33:0/3040724102 shutdown_connections 2024-05-03T18:25:32.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:31.765+0000 7f0682a00700 1 -- 172.21.15.33:0/3040724102 wait complete. 2024-05-03T18:25:32.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:31.767+0000 7f0682a00700 1 Processor -- start 2024-05-03T18:25:32.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:31.767+0000 7f0682a00700 1 -- start start 2024-05-03T18:25:32.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:31.767+0000 7f0682a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f067c103750 0x7f067c19f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:32.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:31.767+0000 7f0682a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f067c104100 0x7f067c19fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:32.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:31.767+0000 7f0682a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f067c108850 0x7f067c1998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:32.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:31.767+0000 7f0682a00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f067c113540 con 0x7f067c103750 2024-05-03T18:25:32.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:31.767+0000 7f0682a00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f067c1133c0 con 0x7f067c108850 2024-05-03T18:25:32.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:31.767+0000 7f0682a00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f067c1136c0 con 0x7f067c104100 2024-05-03T18:25:32.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:31.768+0000 7f067be00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f067c103750 0x7f067c19f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:32.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:31.768+0000 7f067be00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f067c103750 0x7f067c19f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60608/0 (socket says 172.21.15.33:60608) 2024-05-03T18:25:32.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:31.768+0000 7f067be00700 1 -- 172.21.15.33:0/567539444 learned_addr learned my addr 172.21.15.33:0/567539444 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:25:32.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:31.768+0000 7f0680a00700 1 --2- 172.21.15.33:0/567539444 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f067c108850 0x7f067c1998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:32.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:31.768+0000 7f0673e00700 1 --2- 172.21.15.33:0/567539444 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f067c104100 0x7f067c19fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:32.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:31.768+0000 7f067be00700 1 -- 172.21.15.33:0/567539444 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f067c104100 msgr2=0x7f067c19fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:32.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:31.768+0000 7f067be00700 1 --2- 172.21.15.33:0/567539444 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f067c104100 0x7f067c19fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:32.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:31.768+0000 7f067be00700 1 -- 172.21.15.33:0/567539444 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f067c108850 msgr2=0x7f067c1998e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:32.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:31.768+0000 7f067be00700 1 --2- 172.21.15.33:0/567539444 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f067c108850 0x7f067c1998e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:32.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:31.768+0000 7f067be00700 1 -- 172.21.15.33:0/567539444 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f067c19a040 con 0x7f067c103750 2024-05-03T18:25:32.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:31.769+0000 7f067be00700 1 --2- 172.21.15.33:0/567539444 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f067c103750 0x7f067c19f720 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f066c005910 tx=0x7f066c0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:25:32.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:31.769+0000 7f0679600700 1 -- 172.21.15.33:0/567539444 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f066c026070 con 0x7f067c103750 2024-05-03T18:25:32.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:31.769+0000 7f0682a00700 1 -- 172.21.15.33:0/567539444 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f067c19a2d0 con 0x7f067c103750 2024-05-03T18:25:32.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:31.769+0000 7f0682a00700 1 -- 172.21.15.33:0/567539444 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f067c1a65f0 con 0x7f067c103750 2024-05-03T18:25:32.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:31.769+0000 7f0679600700 1 -- 172.21.15.33:0/567539444 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f066c00b790 con 0x7f067c103750 2024-05-03T18:25:32.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:31.769+0000 7f0679600700 1 -- 172.21.15.33:0/567539444 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f066c00f6c0 con 0x7f067c103750 2024-05-03T18:25:32.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:31.770+0000 7f0682a00700 1 -- 172.21.15.33:0/567539444 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0658005330 con 0x7f067c103750 2024-05-03T18:25:32.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:31.773+0000 7f0679600700 1 -- 172.21.15.33:0/567539444 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f066c029070 con 0x7f067c103750 2024-05-03T18:25:32.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:31.775+0000 7f0679600700 1 --2- 172.21.15.33:0/567539444 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f06600b2990 0x7f06600b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:32.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:31.775+0000 7f0679600700 1 -- 172.21.15.33:0/567539444 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f066c0e32c0 con 0x7f067c103750 2024-05-03T18:25:32.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:31.775+0000 7f0673e00700 1 --2- 172.21.15.33:0/567539444 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f06600b2990 0x7f06600b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:32.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:31.776+0000 7f0673e00700 1 --2- 172.21.15.33:0/567539444 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f06600b2990 0x7f06600b4e50 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f067c103130 tx=0x7f066400b5b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:25:32.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:31.777+0000 7f0679600700 1 -- 172.21.15.33:0/567539444 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f066c030080 con 0x7f067c103750 2024-05-03T18:25:32.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:32.016+0000 7f0682a00700 1 -- 172.21.15.33:0/567539444 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f06580058f0 con 0x7f067c103750 2024-05-03T18:25:32.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:32.017+0000 7f0679600700 1 -- 172.21.15.33:0/567539444 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f066c013070 con 0x7f067c103750 2024-05-03T18:25:32.268 INFO:teuthology.orchestra.run.smithi033.stdout:300647710778 2024-05-03T18:25:32.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:32.020+0000 7f0682a00700 1 -- 172.21.15.33:0/567539444 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f06600b2990 msgr2=0x7f06600b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:32.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:32.020+0000 7f0682a00700 1 --2- 172.21.15.33:0/567539444 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f06600b2990 0x7f06600b4e50 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f067c103130 tx=0x7f066400b5b0 comp rx=0 tx=0).stop 2024-05-03T18:25:32.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:32.020+0000 7f0682a00700 1 -- 172.21.15.33:0/567539444 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f067c103750 msgr2=0x7f067c19f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:32.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:32.020+0000 7f0682a00700 1 --2- 172.21.15.33:0/567539444 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f067c103750 0x7f067c19f720 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f066c005910 tx=0x7f066c0045e0 comp rx=0 tx=0).stop 2024-05-03T18:25:32.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:32.020+0000 7f0682a00700 1 -- 172.21.15.33:0/567539444 shutdown_connections 2024-05-03T18:25:32.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:32.020+0000 7f0682a00700 1 --2- 172.21.15.33:0/567539444 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f06600b2990 0x7f06600b4e50 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:32.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:32.020+0000 7f0682a00700 1 --2- 172.21.15.33:0/567539444 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f067c108850 0x7f067c1998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:32.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:32.020+0000 7f0682a00700 1 --2- 172.21.15.33:0/567539444 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f067c103750 0x7f067c19f720 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:32.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:32.020+0000 7f0682a00700 1 --2- 172.21.15.33:0/567539444 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f067c104100 0x7f067c19fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:32.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:32.020+0000 7f0682a00700 1 -- 172.21.15.33:0/567539444 >> 172.21.15.33:0/567539444 conn(0x7f067c07a600 msgr2=0x7f067c1018b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:25:32.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:32.020+0000 7f0682a00700 1 -- 172.21.15.33:0/567539444 shutdown_connections 2024-05-03T18:25:32.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:32.020+0000 7f0682a00700 1 -- 172.21.15.33:0/567539444 wait complete. 2024-05-03T18:25:32.853 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:25:32 smithi033 ceph-mon[28330]: pgmap v475: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 85 B/s wr, 1 op/s 2024-05-03T18:25:32.854 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:25:32 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/567539444' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-03T18:25:33.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:25:32 smithi099 ceph-mon[31658]: pgmap v475: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 85 B/s wr, 1 op/s 2024-05-03T18:25:33.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:25:32 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/567539444' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-03T18:25:33.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:25:32 smithi150 ceph-mon[32492]: pgmap v475: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 85 B/s wr, 1 op/s 2024-05-03T18:25:33.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:25:32 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/567539444' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-03T18:25:33.674 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:25:33 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:25:33.675 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:25:33 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:25:33.675 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:25:33 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:25:33.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:33.750+0000 7ff837200700 1 -- 172.21.15.33:0/2315152823 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff838108700 msgr2=0x7ff8381102d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:33.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:33.750+0000 7ff837200700 1 --2- 172.21.15.33:0/2315152823 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff838108700 0x7ff8381102d0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ff83000b0d0 tx=0x7ff83000b3e0 comp rx=0 tx=0).stop 2024-05-03T18:25:33.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:33.750+0000 7ff837200700 1 -- 172.21.15.33:0/2315152823 shutdown_connections 2024-05-03T18:25:33.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:33.750+0000 7ff837200700 1 --2- 172.21.15.33:0/2315152823 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff838108700 0x7ff8381102d0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:33.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:33.750+0000 7ff837200700 1 --2- 172.21.15.33:0/2315152823 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff838103620 0x7ff838103a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:33.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:33.750+0000 7ff837200700 1 --2- 172.21.15.33:0/2315152823 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff838103fd0 0x7ff838108020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:33.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:33.750+0000 7ff837200700 1 -- 172.21.15.33:0/2315152823 >> 172.21.15.33:0/2315152823 conn(0x7ff8380feef0 msgr2=0x7ff838101310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:25:33.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:33.750+0000 7ff837200700 1 -- 172.21.15.33:0/2315152823 shutdown_connections 2024-05-03T18:25:33.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:33.750+0000 7ff837200700 1 -- 172.21.15.33:0/2315152823 wait complete. 2024-05-03T18:25:33.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:33.751+0000 7ff837200700 1 Processor -- start 2024-05-03T18:25:33.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:33.752+0000 7ff837200700 1 -- start start 2024-05-03T18:25:33.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:33.752+0000 7ff837200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff838103620 0x7ff83819f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:33.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:33.752+0000 7ff837200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff838103fd0 0x7ff83819fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:33.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:33.752+0000 7ff837200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff838108700 0x7ff838199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:33.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:33.752+0000 7ff837200700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff838113510 con 0x7ff838103620 2024-05-03T18:25:33.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:33.752+0000 7ff837200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff838113390 con 0x7ff838103fd0 2024-05-03T18:25:33.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:33.752+0000 7ff837200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff838113690 con 0x7ff838108700 2024-05-03T18:25:33.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:33.752+0000 7ff835e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff838103620 0x7ff83819f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:33.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:33.752+0000 7ff835e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff838103620 0x7ff83819f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60654/0 (socket says 172.21.15.33:60654) 2024-05-03T18:25:33.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:33.752+0000 7ff835e00700 1 -- 172.21.15.33:0/2668130524 learned_addr learned my addr 172.21.15.33:0/2668130524 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:25:33.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:33.753+0000 7ff835400700 1 --2- 172.21.15.33:0/2668130524 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff838103fd0 0x7ff83819fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:33.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:33.753+0000 7ff836800700 1 --2- 172.21.15.33:0/2668130524 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff838108700 0x7ff838199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:33.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:33.753+0000 7ff835e00700 1 -- 172.21.15.33:0/2668130524 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff838108700 msgr2=0x7ff838199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:33.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:33.753+0000 7ff835e00700 1 --2- 172.21.15.33:0/2668130524 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff838108700 0x7ff838199830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:33.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:33.753+0000 7ff835e00700 1 -- 172.21.15.33:0/2668130524 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff838103fd0 msgr2=0x7ff83819fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:33.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:33.753+0000 7ff835e00700 1 --2- 172.21.15.33:0/2668130524 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff838103fd0 0x7ff83819fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:33.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:33.753+0000 7ff835e00700 1 -- 172.21.15.33:0/2668130524 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff83819a010 con 0x7ff838103620 2024-05-03T18:25:33.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:33.753+0000 7ff835400700 1 --2- 172.21.15.33:0/2668130524 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff838103fd0 0x7ff83819fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:25:33.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:33.754+0000 7ff835e00700 1 --2- 172.21.15.33:0/2668130524 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff838103620 0x7ff83819f6b0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff82800b530 tx=0x7ff82800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:25:33.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:33.754+0000 7ff82ea00700 1 -- 172.21.15.33:0/2668130524 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff828015020 con 0x7ff838103620 2024-05-03T18:25:33.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:33.754+0000 7ff82ea00700 1 -- 172.21.15.33:0/2668130524 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff828011cc0 con 0x7ff838103620 2024-05-03T18:25:33.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:33.754+0000 7ff837200700 1 -- 172.21.15.33:0/2668130524 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff83819a300 con 0x7ff838103620 2024-05-03T18:25:33.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:33.754+0000 7ff82ea00700 1 -- 172.21.15.33:0/2668130524 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff828010d30 con 0x7ff838103620 2024-05-03T18:25:33.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:33.754+0000 7ff837200700 1 -- 172.21.15.33:0/2668130524 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8380756d0 con 0x7ff838103620 2024-05-03T18:25:33.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:33.755+0000 7ff837200700 1 -- 172.21.15.33:0/2668130524 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff838104b00 con 0x7ff838103620 2024-05-03T18:25:33.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:33.758+0000 7ff82ea00700 1 -- 172.21.15.33:0/2668130524 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7ff828018020 con 0x7ff838103620 2024-05-03T18:25:33.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:33.759+0000 7ff82ea00700 1 --2- 172.21.15.33:0/2668130524 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7ff81c0b2990 0x7ff81c0b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:33.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:33.759+0000 7ff82ea00700 1 -- 172.21.15.33:0/2668130524 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7ff8280e2400 con 0x7ff838103620 2024-05-03T18:25:33.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:33.760+0000 7ff835400700 1 --2- 172.21.15.33:0/2668130524 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7ff81c0b2990 0x7ff81c0b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:33.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:33.761+0000 7ff835400700 1 --2- 172.21.15.33:0/2668130524 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7ff81c0b2990 0x7ff81c0b4e50 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff8380ff410 tx=0x7ff82400b5f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:25:33.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:33.762+0000 7ff82ea00700 1 -- 172.21.15.33:0/2668130524 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7ff82801f020 con 0x7ff838103620 2024-05-03T18:25:34.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:34.006+0000 7ff837200700 1 -- 172.21.15.33:0/2668130524 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7ff83819ad60 con 0x7ff838103620 2024-05-03T18:25:34.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:34.007+0000 7ff82ea00700 1 -- 172.21.15.33:0/2668130524 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7ff8280a9cf0 con 0x7ff838103620 2024-05-03T18:25:34.008 INFO:teuthology.orchestra.run.smithi033.stdout:274877907008 2024-05-03T18:25:34.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:34.010+0000 7ff837200700 1 -- 172.21.15.33:0/2668130524 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7ff81c0b2990 msgr2=0x7ff81c0b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:34.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:34.010+0000 7ff837200700 1 --2- 172.21.15.33:0/2668130524 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7ff81c0b2990 0x7ff81c0b4e50 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff8380ff410 tx=0x7ff82400b5f0 comp rx=0 tx=0).stop 2024-05-03T18:25:34.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:34.010+0000 7ff837200700 1 -- 172.21.15.33:0/2668130524 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff838103620 msgr2=0x7ff83819f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:34.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:34.010+0000 7ff837200700 1 --2- 172.21.15.33:0/2668130524 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff838103620 0x7ff83819f6b0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff82800b530 tx=0x7ff82800b8f0 comp rx=0 tx=0).stop 2024-05-03T18:25:34.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:34.010+0000 7ff837200700 1 -- 172.21.15.33:0/2668130524 shutdown_connections 2024-05-03T18:25:34.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:34.010+0000 7ff837200700 1 --2- 172.21.15.33:0/2668130524 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff838108700 0x7ff838199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:34.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:34.010+0000 7ff837200700 1 --2- 172.21.15.33:0/2668130524 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff838103620 0x7ff83819f6b0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:34.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:34.010+0000 7ff837200700 1 --2- 172.21.15.33:0/2668130524 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7ff81c0b2990 0x7ff81c0b4e50 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:34.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:34.010+0000 7ff837200700 1 --2- 172.21.15.33:0/2668130524 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff838103fd0 0x7ff83819fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:34.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:34.010+0000 7ff837200700 1 -- 172.21.15.33:0/2668130524 >> 172.21.15.33:0/2668130524 conn(0x7ff8380feef0 msgr2=0x7ff838100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:25:34.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:34.010+0000 7ff837200700 1 -- 172.21.15.33:0/2668130524 shutdown_connections 2024-05-03T18:25:34.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:34.010+0000 7ff837200700 1 -- 172.21.15.33:0/2668130524 wait complete. 2024-05-03T18:25:34.034 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:25:33 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:25:34.034 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:25:33 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:25:34.034 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:25:33 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:25:34.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:33.663+0000 7ff83d600700 1 -- 172.21.15.33:0/2985861868 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff838104100 msgr2=0x7ff838108120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:34.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:33.663+0000 7ff83d600700 1 --2- 172.21.15.33:0/2985861868 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff838104100 0x7ff838108120 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ff8240099c0 tx=0x7ff824009cd0 comp rx=0 tx=0).stop 2024-05-03T18:25:34.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:33.663+0000 7ff83d600700 1 -- 172.21.15.33:0/2985861868 shutdown_connections 2024-05-03T18:25:34.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:33.663+0000 7ff83d600700 1 --2- 172.21.15.33:0/2985861868 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff838108850 0x7ff8381103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:34.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:33.663+0000 7ff83d600700 1 --2- 172.21.15.33:0/2985861868 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff838103750 0x7ff838103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:34.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:33.663+0000 7ff83d600700 1 --2- 172.21.15.33:0/2985861868 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff838104100 0x7ff838108120 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:34.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:33.663+0000 7ff83d600700 1 -- 172.21.15.33:0/2985861868 >> 172.21.15.33:0/2985861868 conn(0x7ff83807a600 msgr2=0x7ff83807aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:25:34.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:33.663+0000 7ff83d600700 1 -- 172.21.15.33:0/2985861868 shutdown_connections 2024-05-03T18:25:34.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:33.663+0000 7ff83d600700 1 -- 172.21.15.33:0/2985861868 wait complete. 2024-05-03T18:25:34.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:33.665+0000 7ff83d600700 1 Processor -- start 2024-05-03T18:25:34.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:33.665+0000 7ff83d600700 1 -- start start 2024-05-03T18:25:34.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:33.665+0000 7ff83d600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff838103750 0x7ff8380757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:34.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:33.665+0000 7ff83d600700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff838104100 0x7ff838075ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:34.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:33.665+0000 7ff83d600700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff838108850 0x7ff838076220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:34.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:33.665+0000 7ff83d600700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff8381134e0 con 0x7ff838108850 2024-05-03T18:25:34.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:33.665+0000 7ff83d600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff838113360 con 0x7ff838103750 2024-05-03T18:25:34.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:33.665+0000 7ff83d600700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff838113660 con 0x7ff838104100 2024-05-03T18:25:34.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:33.666+0000 7ff837400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff838108850 0x7ff838076220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:34.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:33.666+0000 7ff837400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff838108850 0x7ff838076220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60632/0 (socket says 172.21.15.33:60632) 2024-05-03T18:25:34.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:33.666+0000 7ff837400700 1 -- 172.21.15.33:0/1096862745 learned_addr learned my addr 172.21.15.33:0/1096862745 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:25:34.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:33.666+0000 7ff836a00700 1 --2- 172.21.15.33:0/1096862745 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff838103750 0x7ff8380757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:34.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:33.666+0000 7ff836000700 1 --2- 172.21.15.33:0/1096862745 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff838104100 0x7ff838075ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:34.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:33.666+0000 7ff837400700 1 -- 172.21.15.33:0/1096862745 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff838104100 msgr2=0x7ff838075ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:34.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:33.666+0000 7ff837400700 1 --2- 172.21.15.33:0/1096862745 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff838104100 0x7ff838075ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:34.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:33.666+0000 7ff837400700 1 -- 172.21.15.33:0/1096862745 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff838103750 msgr2=0x7ff8380757a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:34.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:33.666+0000 7ff837400700 1 --2- 172.21.15.33:0/1096862745 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff838103750 0x7ff8380757a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:34.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:33.666+0000 7ff837400700 1 -- 172.21.15.33:0/1096862745 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8381a7f00 con 0x7ff838108850 2024-05-03T18:25:34.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:33.666+0000 7ff836a00700 1 --2- 172.21.15.33:0/1096862745 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff838103750 0x7ff8380757a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:25:34.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:33.667+0000 7ff837400700 1 --2- 172.21.15.33:0/1096862745 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff838108850 0x7ff838076220 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff830007d70 tx=0x7ff83000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:25:34.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:33.668+0000 7ff82f400700 1 -- 172.21.15.33:0/1096862745 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff830016020 con 0x7ff838108850 2024-05-03T18:25:34.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:33.668+0000 7ff82f400700 1 -- 172.21.15.33:0/1096862745 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff83000ad40 con 0x7ff838108850 2024-05-03T18:25:34.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:33.668+0000 7ff83d600700 1 -- 172.21.15.33:0/1096862745 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8381a81f0 con 0x7ff838108850 2024-05-03T18:25:34.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:33.668+0000 7ff82f400700 1 -- 172.21.15.33:0/1096862745 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8300115e0 con 0x7ff838108850 2024-05-03T18:25:34.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:33.669+0000 7ff83d600700 1 -- 172.21.15.33:0/1096862745 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8381a86b0 con 0x7ff838108850 2024-05-03T18:25:34.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:33.670+0000 7ff83d600700 1 -- 172.21.15.33:0/1096862745 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff83810ba20 con 0x7ff838108850 2024-05-03T18:25:34.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:33.673+0000 7ff82f400700 1 -- 172.21.15.33:0/1096862745 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7ff830019020 con 0x7ff838108850 2024-05-03T18:25:34.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:33.676+0000 7ff82f400700 1 --2- 172.21.15.33:0/1096862745 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7ff81c0b2910 0x7ff81c0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:34.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:33.676+0000 7ff82f400700 1 -- 172.21.15.33:0/1096862745 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7ff8300e2a70 con 0x7ff838108850 2024-05-03T18:25:34.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:33.676+0000 7ff836a00700 1 --2- 172.21.15.33:0/1096862745 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7ff81c0b2910 0x7ff81c0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:34.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:33.677+0000 7ff836a00700 1 --2- 172.21.15.33:0/1096862745 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7ff81c0b2910 0x7ff81c0b4dd0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff82800bcf0 tx=0x7ff82800b2c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:25:34.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:33.678+0000 7ff82f400700 1 -- 172.21.15.33:0/1096862745 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7ff830020020 con 0x7ff838108850 2024-05-03T18:25:34.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:33.928+0000 7ff83d600700 1 -- 172.21.15.33:0/1096862745 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7ff83810b400 con 0x7ff838108850 2024-05-03T18:25:34.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:33.929+0000 7ff82f400700 1 -- 172.21.15.33:0/1096862745 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7ff830055e20 con 0x7ff838108850 2024-05-03T18:25:34.049 INFO:teuthology.orchestra.run.smithi033.stdout:227633266766 2024-05-03T18:25:34.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:33.932+0000 7ff83d600700 1 -- 172.21.15.33:0/1096862745 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7ff81c0b2910 msgr2=0x7ff81c0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:34.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:33.932+0000 7ff83d600700 1 --2- 172.21.15.33:0/1096862745 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7ff81c0b2910 0x7ff81c0b4dd0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff82800bcf0 tx=0x7ff82800b2c0 comp rx=0 tx=0).stop 2024-05-03T18:25:34.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:33.932+0000 7ff83d600700 1 -- 172.21.15.33:0/1096862745 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff838108850 msgr2=0x7ff838076220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:34.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:33.932+0000 7ff83d600700 1 --2- 172.21.15.33:0/1096862745 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff838108850 0x7ff838076220 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff830007d70 tx=0x7ff83000a340 comp rx=0 tx=0).stop 2024-05-03T18:25:34.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:33.933+0000 7ff83d600700 1 -- 172.21.15.33:0/1096862745 shutdown_connections 2024-05-03T18:25:34.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:33.933+0000 7ff83d600700 1 --2- 172.21.15.33:0/1096862745 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff838108850 0x7ff838076220 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:34.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:33.933+0000 7ff83d600700 1 --2- 172.21.15.33:0/1096862745 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7ff81c0b2910 0x7ff81c0b4dd0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:34.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:33.933+0000 7ff83d600700 1 --2- 172.21.15.33:0/1096862745 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff838103750 0x7ff8380757a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:34.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:33.933+0000 7ff83d600700 1 --2- 172.21.15.33:0/1096862745 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff838104100 0x7ff838075ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:34.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:33.933+0000 7ff83d600700 1 -- 172.21.15.33:0/1096862745 >> 172.21.15.33:0/1096862745 conn(0x7ff83807a600 msgr2=0x7ff8381018d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:25:34.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:33.933+0000 7ff83d600700 1 -- 172.21.15.33:0/1096862745 shutdown_connections 2024-05-03T18:25:34.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:33.933+0000 7ff83d600700 1 -- 172.21.15.33:0/1096862745 wait complete. 2024-05-03T18:25:34.051 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:25:33 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:25:34.051 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:25:33 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:25:34.051 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:25:33 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:25:34.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:25:34 smithi033 ceph-mon[28330]: pgmap v476: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 85 B/s wr, 2 op/s 2024-05-03T18:25:34.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:25:34 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/1096862745' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-03T18:25:34.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:25:34 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/2668130524' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-03T18:25:35.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:25:34 smithi099 ceph-mon[31658]: pgmap v476: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 85 B/s wr, 2 op/s 2024-05-03T18:25:35.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:25:34 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/1096862745' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-03T18:25:35.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:25:34 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/2668130524' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-03T18:25:35.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:25:34 smithi150 ceph-mon[32492]: pgmap v476: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 85 B/s wr, 2 op/s 2024-05-03T18:25:35.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:25:34 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/1096862745' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-03T18:25:35.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:25:34 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/2668130524' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-03T18:25:35.172 INFO:tasks.cephadm.ceph_manager.ceph:need seq 274877907002 got 274877907008 for osd.10 2024-05-03T18:25:35.172 DEBUG:teuthology.parallel:result is None 2024-05-03T18:25:35.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:34.769+0000 7ff15d400700 1 -- 172.21.15.33:0/4033737849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff15806bb40 msgr2=0x7ff158110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:35.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:34.769+0000 7ff15d400700 1 --2- 172.21.15.33:0/4033737849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff15806bb40 0x7ff158110330 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ff15000b0a0 tx=0x7ff15000b3b0 comp rx=0 tx=0).stop 2024-05-03T18:25:35.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:34.769+0000 7ff15d400700 1 -- 172.21.15.33:0/4033737849 shutdown_connections 2024-05-03T18:25:35.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:34.769+0000 7ff15d400700 1 --2- 172.21.15.33:0/4033737849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff15806bb40 0x7ff158110330 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:35.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:34.769+0000 7ff15d400700 1 --2- 172.21.15.33:0/4033737849 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff158109510 0x7ff1581098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:35.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:34.769+0000 7ff15d400700 1 --2- 172.21.15.33:0/4033737849 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff15806b180 0x7ff15806b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:35.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:34.769+0000 7ff15d400700 1 -- 172.21.15.33:0/4033737849 >> 172.21.15.33:0/4033737849 conn(0x7ff158079350 msgr2=0x7ff158079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:25:35.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:34.769+0000 7ff15d400700 1 -- 172.21.15.33:0/4033737849 shutdown_connections 2024-05-03T18:25:35.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:34.769+0000 7ff15d400700 1 -- 172.21.15.33:0/4033737849 wait complete. 2024-05-03T18:25:35.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:34.770+0000 7ff15d400700 1 Processor -- start 2024-05-03T18:25:35.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:34.770+0000 7ff15d400700 1 -- start start 2024-05-03T18:25:35.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:34.770+0000 7ff15d400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff15806b180 0x7ff1581a1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:35.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:34.770+0000 7ff15d400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff15806bb40 0x7ff1581a1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:35.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:34.771+0000 7ff15d400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff158109510 0x7ff15819ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:35.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:34.771+0000 7ff15d400700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff1581134f0 con 0x7ff15806bb40 2024-05-03T18:25:35.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:34.771+0000 7ff15d400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff158113370 con 0x7ff158109510 2024-05-03T18:25:35.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:34.771+0000 7ff15d400700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff158113670 con 0x7ff15806b180 2024-05-03T18:25:35.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:34.771+0000 7ff14de00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff15806bb40 0x7ff1581a1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:35.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:34.771+0000 7ff14de00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff15806bb40 0x7ff1581a1dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60664/0 (socket says 172.21.15.33:60664) 2024-05-03T18:25:35.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:34.771+0000 7ff14de00700 1 -- 172.21.15.33:0/3671908111 learned_addr learned my addr 172.21.15.33:0/3671908111 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:25:35.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:34.771+0000 7ff157200700 1 --2- 172.21.15.33:0/3671908111 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff158109510 0x7ff15819ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:35.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:34.771+0000 7ff156800700 1 --2- 172.21.15.33:0/3671908111 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff15806b180 0x7ff1581a1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:35.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:34.771+0000 7ff14de00700 1 -- 172.21.15.33:0/3671908111 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff15806b180 msgr2=0x7ff1581a1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:35.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:34.771+0000 7ff14de00700 1 --2- 172.21.15.33:0/3671908111 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff15806b180 0x7ff1581a1890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:35.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:34.771+0000 7ff14de00700 1 -- 172.21.15.33:0/3671908111 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff158109510 msgr2=0x7ff15819ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:35.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:34.771+0000 7ff14de00700 1 --2- 172.21.15.33:0/3671908111 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff158109510 0x7ff15819ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:35.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:34.771+0000 7ff14de00700 1 -- 172.21.15.33:0/3671908111 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff15819c1b0 con 0x7ff15806bb40 2024-05-03T18:25:35.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:34.771+0000 7ff157200700 1 --2- 172.21.15.33:0/3671908111 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff158109510 0x7ff15819ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:25:35.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:34.771+0000 7ff156800700 1 --2- 172.21.15.33:0/3671908111 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff15806b180 0x7ff1581a1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:25:35.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:34.772+0000 7ff14de00700 1 --2- 172.21.15.33:0/3671908111 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff15806bb40 0x7ff1581a1dd0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ff14000d980 tx=0x7ff14000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:25:35.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:34.772+0000 7ff14fe00700 1 -- 172.21.15.33:0/3671908111 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff140017070 con 0x7ff15806bb40 2024-05-03T18:25:35.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:34.772+0000 7ff14fe00700 1 -- 172.21.15.33:0/3671908111 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff140004830 con 0x7ff15806bb40 2024-05-03T18:25:35.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:34.772+0000 7ff15d400700 1 -- 172.21.15.33:0/3671908111 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff15819c440 con 0x7ff15806bb40 2024-05-03T18:25:35.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:34.772+0000 7ff14fe00700 1 -- 172.21.15.33:0/3671908111 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1400053a0 con 0x7ff15806bb40 2024-05-03T18:25:35.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:34.772+0000 7ff15d400700 1 -- 172.21.15.33:0/3671908111 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1581a8760 con 0x7ff15806bb40 2024-05-03T18:25:35.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:34.773+0000 7ff15d400700 1 -- 172.21.15.33:0/3671908111 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff15806a190 con 0x7ff15806bb40 2024-05-03T18:25:35.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:34.776+0000 7ff14fe00700 1 -- 172.21.15.33:0/3671908111 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7ff1400049e0 con 0x7ff15806bb40 2024-05-03T18:25:35.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:34.777+0000 7ff14fe00700 1 --2- 172.21.15.33:0/3671908111 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7ff13c0b2910 0x7ff13c0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:35.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:34.777+0000 7ff14fe00700 1 -- 172.21.15.33:0/3671908111 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7ff1400e2b10 con 0x7ff15806bb40 2024-05-03T18:25:35.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:34.778+0000 7ff156800700 1 --2- 172.21.15.33:0/3671908111 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7ff13c0b2910 0x7ff13c0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:35.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:34.778+0000 7ff156800700 1 --2- 172.21.15.33:0/3671908111 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7ff13c0b2910 0x7ff13c0b4dd0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff148005e10 tx=0x7ff148005d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:25:35.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:34.781+0000 7ff14fe00700 1 -- 172.21.15.33:0/3671908111 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7ff140020020 con 0x7ff15806bb40 2024-05-03T18:25:35.350 INFO:teuthology.orchestra.run.smithi033.stdout:77309411451 2024-05-03T18:25:35.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:35.033+0000 7ff15d400700 1 -- 172.21.15.33:0/3671908111 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7ff1581a8960 con 0x7ff15806bb40 2024-05-03T18:25:35.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:35.034+0000 7ff14fe00700 1 -- 172.21.15.33:0/3671908111 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff1400aa2f0 con 0x7ff15806bb40 2024-05-03T18:25:35.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:35.037+0000 7ff15d400700 1 -- 172.21.15.33:0/3671908111 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7ff13c0b2910 msgr2=0x7ff13c0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:35.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:35.037+0000 7ff15d400700 1 --2- 172.21.15.33:0/3671908111 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7ff13c0b2910 0x7ff13c0b4dd0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff148005e10 tx=0x7ff148005d20 comp rx=0 tx=0).stop 2024-05-03T18:25:35.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:35.037+0000 7ff15d400700 1 -- 172.21.15.33:0/3671908111 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff15806bb40 msgr2=0x7ff1581a1dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:35.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:35.037+0000 7ff15d400700 1 --2- 172.21.15.33:0/3671908111 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff15806bb40 0x7ff1581a1dd0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ff14000d980 tx=0x7ff14000dd40 comp rx=0 tx=0).stop 2024-05-03T18:25:35.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:35.038+0000 7ff15d400700 1 -- 172.21.15.33:0/3671908111 shutdown_connections 2024-05-03T18:25:35.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:35.038+0000 7ff15d400700 1 --2- 172.21.15.33:0/3671908111 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7ff13c0b2910 0x7ff13c0b4dd0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:35.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:35.038+0000 7ff15d400700 1 --2- 172.21.15.33:0/3671908111 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff158109510 0x7ff15819ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:35.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:35.038+0000 7ff15d400700 1 --2- 172.21.15.33:0/3671908111 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff15806b180 0x7ff1581a1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:35.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:35.038+0000 7ff15d400700 1 --2- 172.21.15.33:0/3671908111 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff15806bb40 0x7ff1581a1dd0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:35.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:35.038+0000 7ff15d400700 1 -- 172.21.15.33:0/3671908111 >> 172.21.15.33:0/3671908111 conn(0x7ff158079350 msgr2=0x7ff158100f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:25:35.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:35.038+0000 7ff15d400700 1 -- 172.21.15.33:0/3671908111 shutdown_connections 2024-05-03T18:25:35.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:35.038+0000 7ff15d400700 1 -- 172.21.15.33:0/3671908111 wait complete. 2024-05-03T18:25:35.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:25:35 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/3671908111' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-03T18:25:36.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:25:35 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/3671908111' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-03T18:25:36.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:25:35 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/3671908111' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-03T18:25:36.287 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738496 got 34359738503 for osd.0 2024-05-03T18:25:36.287 DEBUG:teuthology.parallel:result is None 2024-05-03T18:25:36.523 INFO:tasks.cephadm.ceph_manager.ceph:need seq 300647710773 got 300647710778 for osd.11 2024-05-03T18:25:36.523 DEBUG:teuthology.parallel:result is None 2024-05-03T18:25:36.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:25:36 smithi033 ceph-mon[28330]: pgmap v477: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:25:37.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:25:36 smithi099 ceph-mon[31658]: pgmap v477: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:25:37.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:25:36 smithi150 ceph-mon[32492]: pgmap v477: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:25:37.324 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411445 got 77309411451 for osd.2 2024-05-03T18:25:37.325 DEBUG:teuthology.parallel:result is None 2024-05-03T18:25:37.818 INFO:tasks.cephadm.ceph_manager.ceph:need seq 180388626522 got 180388626526 for osd.6 2024-05-03T18:25:37.818 DEBUG:teuthology.parallel:result is None 2024-05-03T18:25:38.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:38.135+0000 7f2bbd600700 1 -- 172.21.15.33:0/1522777589 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2bb806c3d0 msgr2=0x7f2bb8110320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:38.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:38.135+0000 7f2bbd600700 1 --2- 172.21.15.33:0/1522777589 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2bb806c3d0 0x7f2bb8110320 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f2bb000b0a0 tx=0x7f2bb000b3b0 comp rx=0 tx=0).stop 2024-05-03T18:25:38.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:38.135+0000 7f2bbd600700 1 -- 172.21.15.33:0/1522777589 shutdown_connections 2024-05-03T18:25:38.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:38.135+0000 7f2bbd600700 1 --2- 172.21.15.33:0/1522777589 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2bb806c3d0 0x7f2bb8110320 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:38.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:38.135+0000 7f2bbd600700 1 --2- 172.21.15.33:0/1522777589 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2bb8109500 0x7f2bb81098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:38.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:38.135+0000 7f2bbd600700 1 --2- 172.21.15.33:0/1522777589 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2bb806ba10 0x7f2bb806be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:38.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:38.135+0000 7f2bbd600700 1 -- 172.21.15.33:0/1522777589 >> 172.21.15.33:0/1522777589 conn(0x7f2bb8079310 msgr2=0x7f2bb8079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:25:38.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:38.135+0000 7f2bbd600700 1 -- 172.21.15.33:0/1522777589 shutdown_connections 2024-05-03T18:25:38.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:38.135+0000 7f2bbd600700 1 -- 172.21.15.33:0/1522777589 wait complete. 2024-05-03T18:25:38.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:38.137+0000 7f2bbd600700 1 Processor -- start 2024-05-03T18:25:38.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:38.137+0000 7f2bbd600700 1 -- start start 2024-05-03T18:25:38.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:38.137+0000 7f2bbd600700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2bb806ba10 0x7f2bb81a18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:38.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:38.137+0000 7f2bbd600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2bb806c3d0 0x7f2bb81a1e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:38.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:38.137+0000 7f2bbd600700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2bb8109500 0x7f2bb819ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:38.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:38.137+0000 7f2bbd600700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb8113510 con 0x7f2bb8109500 2024-05-03T18:25:38.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:38.137+0000 7f2bbd600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb8113390 con 0x7f2bb806c3d0 2024-05-03T18:25:38.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:38.137+0000 7f2bbd600700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb8113690 con 0x7f2bb806ba10 2024-05-03T18:25:38.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:38.138+0000 7f2bb7400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2bb8109500 0x7f2bb819ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:38.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:38.138+0000 7f2bb7400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2bb8109500 0x7f2bb819ba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36186/0 (socket says 172.21.15.33:36186) 2024-05-03T18:25:38.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:38.138+0000 7f2bb7400700 1 -- 172.21.15.33:0/26017984 learned_addr learned my addr 172.21.15.33:0/26017984 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:25:38.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:38.138+0000 7f2bb6a00700 1 --2- 172.21.15.33:0/26017984 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2bb806ba10 0x7f2bb81a18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:38.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:38.138+0000 7f2bb6000700 1 --2- 172.21.15.33:0/26017984 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2bb806c3d0 0x7f2bb81a1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:38.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:38.138+0000 7f2bb7400700 1 -- 172.21.15.33:0/26017984 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2bb806ba10 msgr2=0x7f2bb81a18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:38.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:38.138+0000 7f2bb7400700 1 --2- 172.21.15.33:0/26017984 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2bb806ba10 0x7f2bb81a18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:38.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:38.138+0000 7f2bb7400700 1 -- 172.21.15.33:0/26017984 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2bb806c3d0 msgr2=0x7f2bb81a1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:38.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:38.138+0000 7f2bb7400700 1 --2- 172.21.15.33:0/26017984 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2bb806c3d0 0x7f2bb81a1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:38.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:38.138+0000 7f2bb7400700 1 -- 172.21.15.33:0/26017984 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bb819c1f0 con 0x7f2bb8109500 2024-05-03T18:25:38.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:38.138+0000 7f2bb6000700 1 --2- 172.21.15.33:0/26017984 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2bb806c3d0 0x7f2bb81a1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:25:38.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:38.139+0000 7f2bb7400700 1 --2- 172.21.15.33:0/26017984 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2bb8109500 0x7f2bb819ba90 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f2bb0004dc0 tx=0x7f2bb0004df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:25:38.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:38.139+0000 7f2baf400700 1 -- 172.21.15.33:0/26017984 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb0021070 con 0x7f2bb8109500 2024-05-03T18:25:38.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:38.139+0000 7f2baf400700 1 -- 172.21.15.33:0/26017984 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2bb0007c00 con 0x7f2bb8109500 2024-05-03T18:25:38.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:38.139+0000 7f2baf400700 1 -- 172.21.15.33:0/26017984 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb00052a0 con 0x7f2bb8109500 2024-05-03T18:25:38.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:38.139+0000 7f2bbd600700 1 -- 172.21.15.33:0/26017984 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bb819c480 con 0x7f2bb8109500 2024-05-03T18:25:38.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:38.140+0000 7f2bbd600700 1 -- 172.21.15.33:0/26017984 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bb81a87a0 con 0x7f2bb8109500 2024-05-03T18:25:38.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:38.141+0000 7f2bbd600700 1 -- 172.21.15.33:0/26017984 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bb8104bc0 con 0x7f2bb8109500 2024-05-03T18:25:38.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:38.145+0000 7f2baf400700 1 -- 172.21.15.33:0/26017984 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f2bb0024070 con 0x7f2bb8109500 2024-05-03T18:25:38.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:38.147+0000 7f2baf400700 1 --2- 172.21.15.33:0/26017984 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f2b9c0b2990 0x7f2b9c0b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:38.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:38.147+0000 7f2baf400700 1 -- 172.21.15.33:0/26017984 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f2bb000f020 con 0x7f2bb8109500 2024-05-03T18:25:38.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:38.148+0000 7f2bb6a00700 1 --2- 172.21.15.33:0/26017984 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f2b9c0b2990 0x7f2b9c0b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:38.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:38.148+0000 7f2bb6a00700 1 --2- 172.21.15.33:0/26017984 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f2b9c0b2990 0x7f2b9c0b4e50 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2bb819cbd0 tx=0x7f2ba800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:25:38.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:38.153+0000 7f2baf400700 1 -- 172.21.15.33:0/26017984 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f2bb002b020 con 0x7f2bb8109500 2024-05-03T18:25:38.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:38.398+0000 7f2bbd600700 1 -- 172.21.15.33:0/26017984 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f2bb81a8970 con 0x7f2bb8109500 2024-05-03T18:25:38.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:38.399+0000 7f2baf400700 1 -- 172.21.15.33:0/26017984 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f2bb00cd020 con 0x7f2bb8109500 2024-05-03T18:25:38.400 INFO:teuthology.orchestra.run.smithi033.stdout:253403070536 2024-05-03T18:25:38.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:38.403+0000 7f2bbd600700 1 -- 172.21.15.33:0/26017984 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f2b9c0b2990 msgr2=0x7f2b9c0b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:38.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:38.403+0000 7f2bbd600700 1 --2- 172.21.15.33:0/26017984 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f2b9c0b2990 0x7f2b9c0b4e50 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2bb819cbd0 tx=0x7f2ba800a380 comp rx=0 tx=0).stop 2024-05-03T18:25:38.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:38.403+0000 7f2bbd600700 1 -- 172.21.15.33:0/26017984 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2bb8109500 msgr2=0x7f2bb819ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:38.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:38.403+0000 7f2bbd600700 1 --2- 172.21.15.33:0/26017984 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2bb8109500 0x7f2bb819ba90 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f2bb0004dc0 tx=0x7f2bb0004df0 comp rx=0 tx=0).stop 2024-05-03T18:25:38.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:38.403+0000 7f2bbd600700 1 -- 172.21.15.33:0/26017984 shutdown_connections 2024-05-03T18:25:38.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:38.403+0000 7f2bbd600700 1 --2- 172.21.15.33:0/26017984 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2bb8109500 0x7f2bb819ba90 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:38.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:38.403+0000 7f2bbd600700 1 --2- 172.21.15.33:0/26017984 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2bb806ba10 0x7f2bb81a18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:38.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:38.403+0000 7f2bbd600700 1 --2- 172.21.15.33:0/26017984 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f2b9c0b2990 0x7f2b9c0b4e50 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:38.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:38.403+0000 7f2bbd600700 1 --2- 172.21.15.33:0/26017984 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2bb806c3d0 0x7f2bb81a1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:38.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:38.403+0000 7f2bbd600700 1 -- 172.21.15.33:0/26017984 >> 172.21.15.33:0/26017984 conn(0x7f2bb8079310 msgr2=0x7f2bb810e650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:25:38.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:38.404+0000 7f2bbd600700 1 -- 172.21.15.33:0/26017984 shutdown_connections 2024-05-03T18:25:38.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:38.404+0000 7f2bbd600700 1 -- 172.21.15.33:0/26017984 wait complete. 2024-05-03T18:25:38.478 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:25:38.512 INFO:tasks.cephadm.ceph_manager.ceph:need seq 227633266761 got 227633266766 for osd.8 2024-05-03T18:25:38.513 DEBUG:teuthology.parallel:result is None 2024-05-03T18:25:38.743 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:25:38.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:25:38 smithi033 ceph-mon[28330]: pgmap v478: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:25:38.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:25:38 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/26017984' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-03T18:25:39.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:25:38 smithi099 ceph-mon[31658]: pgmap v478: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:25:39.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:25:38 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/26017984' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-03T18:25:39.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:25:38 smithi150 ceph-mon[32492]: pgmap v478: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:25:39.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:25:38 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/26017984' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-03T18:25:40.714 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070531 got 253403070536 for osd.9 2024-05-03T18:25:40.714 DEBUG:teuthology.parallel:result is None 2024-05-03T18:25:40.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:25:40 smithi033 ceph-mon[28330]: pgmap v479: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:25:41.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:25:40 smithi099 ceph-mon[31658]: pgmap v479: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:25:41.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:25:40 smithi150 ceph-mon[32492]: pgmap v479: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:25:42.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.499+0000 7f6d68400700 1 -- 172.21.15.33:0/1985242990 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d601084d0 msgr2=0x7f6d601088b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:42.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.499+0000 7f6d68400700 1 --2- 172.21.15.33:0/1985242990 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d601084d0 0x7f6d601088b0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6d540099c0 tx=0x7f6d54009cd0 comp rx=0 tx=0).stop 2024-05-03T18:25:42.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.499+0000 7f6d68400700 1 -- 172.21.15.33:0/1985242990 shutdown_connections 2024-05-03T18:25:42.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.499+0000 7f6d68400700 1 --2- 172.21.15.33:0/1985242990 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d60103060 0x7f6d6010f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:42.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.499+0000 7f6d68400700 1 --2- 172.21.15.33:0/1985242990 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d601084d0 0x7f6d601088b0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:42.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.499+0000 7f6d68400700 1 --2- 172.21.15.33:0/1985242990 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d601026c0 0x7f6d60102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:42.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.499+0000 7f6d68400700 1 -- 172.21.15.33:0/1985242990 >> 172.21.15.33:0/1985242990 conn(0x7f6d60079170 msgr2=0x7f6d60079580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:25:42.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.499+0000 7f6d68400700 1 -- 172.21.15.33:0/1985242990 shutdown_connections 2024-05-03T18:25:42.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.499+0000 7f6d68400700 1 -- 172.21.15.33:0/1985242990 wait complete. 2024-05-03T18:25:42.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.500+0000 7f6d68400700 1 Processor -- start 2024-05-03T18:25:42.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.500+0000 7f6d68400700 1 -- start start 2024-05-03T18:25:42.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.501+0000 7f6d68400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d601026c0 0x7f6d601a18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:42.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.501+0000 7f6d68400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d60103060 0x7f6d601a1e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:42.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.501+0000 7f6d68400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d601084d0 0x7f6d6019ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:42.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.501+0000 7f6d68400700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6d60112470 con 0x7f6d601084d0 2024-05-03T18:25:42.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.501+0000 7f6d68400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6d601122f0 con 0x7f6d601026c0 2024-05-03T18:25:42.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.501+0000 7f6d68400700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6d601125f0 con 0x7f6d60103060 2024-05-03T18:25:42.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.501+0000 7f6d66400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d601084d0 0x7f6d6019ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:42.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.501+0000 7f6d66400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d601084d0 0x7f6d6019ba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36194/0 (socket says 172.21.15.33:36194) 2024-05-03T18:25:42.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.501+0000 7f6d66400700 1 -- 172.21.15.33:0/1632734607 learned_addr learned my addr 172.21.15.33:0/1632734607 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:25:42.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.501+0000 7f6d65a00700 1 --2- 172.21.15.33:0/1632734607 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d601026c0 0x7f6d601a18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:42.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.501+0000 7f6d65000700 1 --2- 172.21.15.33:0/1632734607 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d60103060 0x7f6d601a1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:42.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.502+0000 7f6d66400700 1 -- 172.21.15.33:0/1632734607 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d60103060 msgr2=0x7f6d601a1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:42.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.502+0000 7f6d66400700 1 --2- 172.21.15.33:0/1632734607 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d60103060 0x7f6d601a1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:42.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.502+0000 7f6d66400700 1 -- 172.21.15.33:0/1632734607 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d601026c0 msgr2=0x7f6d601a18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:42.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.502+0000 7f6d66400700 1 --2- 172.21.15.33:0/1632734607 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d601026c0 0x7f6d601a18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:42.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.502+0000 7f6d66400700 1 -- 172.21.15.33:0/1632734607 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d6019c1f0 con 0x7f6d601084d0 2024-05-03T18:25:42.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.502+0000 7f6d65a00700 1 --2- 172.21.15.33:0/1632734607 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d601026c0 0x7f6d601a18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:25:42.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.502+0000 7f6d66400700 1 --2- 172.21.15.33:0/1632734607 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d601084d0 0x7f6d6019ba90 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f6d58007ee0 tx=0x7f6d5800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:25:42.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.502+0000 7f6d5e000700 1 -- 172.21.15.33:0/1632734607 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d5800dbd0 con 0x7f6d601084d0 2024-05-03T18:25:42.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.502+0000 7f6d5e000700 1 -- 172.21.15.33:0/1632734607 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6d5800dd70 con 0x7f6d601084d0 2024-05-03T18:25:42.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.502+0000 7f6d68400700 1 -- 172.21.15.33:0/1632734607 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d6019c4e0 con 0x7f6d601084d0 2024-05-03T18:25:42.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.502+0000 7f6d5e000700 1 -- 172.21.15.33:0/1632734607 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d580058d0 con 0x7f6d601084d0 2024-05-03T18:25:42.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.503+0000 7f6d68400700 1 -- 172.21.15.33:0/1632734607 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d601a87a0 con 0x7f6d601084d0 2024-05-03T18:25:42.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.504+0000 7f6d68400700 1 -- 172.21.15.33:0/1632734607 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d6006a190 con 0x7f6d601084d0 2024-05-03T18:25:42.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.506+0000 7f6d5e000700 1 -- 172.21.15.33:0/1632734607 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f6d5801a020 con 0x7f6d601084d0 2024-05-03T18:25:42.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.507+0000 7f6d5e000700 1 --2- 172.21.15.33:0/1632734607 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f6d480b2990 0x7f6d480b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:42.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.508+0000 7f6d5e000700 1 -- 172.21.15.33:0/1632734607 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f6d580e1ed0 con 0x7f6d601084d0 2024-05-03T18:25:42.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.508+0000 7f6d65a00700 1 --2- 172.21.15.33:0/1632734607 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f6d480b2990 0x7f6d480b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:42.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.509+0000 7f6d65a00700 1 --2- 172.21.15.33:0/1632734607 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f6d480b2990 0x7f6d480b4e50 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6d54009990 tx=0x7f6d5401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:25:42.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.512+0000 7f6d5e000700 1 -- 172.21.15.33:0/1632734607 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f6d58010070 con 0x7f6d601084d0 2024-05-03T18:25:42.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.724+0000 7f0e55000700 1 -- 172.21.15.33:0/3489165977 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e500756d0 msgr2=0x7f0e50075ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:42.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.724+0000 7f0e55000700 1 --2- 172.21.15.33:0/3489165977 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e500756d0 0x7f0e50075ab0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f0e40009990 tx=0x7f0e40009ca0 comp rx=0 tx=0).stop 2024-05-03T18:25:42.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.724+0000 7f0e55000700 1 -- 172.21.15.33:0/3489165977 shutdown_connections 2024-05-03T18:25:42.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.724+0000 7f0e55000700 1 --2- 172.21.15.33:0/3489165977 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0e5010fea0 0x7f0e50112290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:42.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.724+0000 7f0e55000700 1 --2- 172.21.15.33:0/3489165977 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e500756d0 0x7f0e50075ab0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:42.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.724+0000 7f0e55000700 1 --2- 172.21.15.33:0/3489165977 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e50075ff0 0x7f0e5010f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:42.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.724+0000 7f0e55000700 1 -- 172.21.15.33:0/3489165977 >> 172.21.15.33:0/3489165977 conn(0x7f0e5007a600 msgr2=0x7f0e5007aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:25:42.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.724+0000 7f0e55000700 1 -- 172.21.15.33:0/3489165977 shutdown_connections 2024-05-03T18:25:42.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.724+0000 7f0e55000700 1 -- 172.21.15.33:0/3489165977 wait complete. 2024-05-03T18:25:42.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.725+0000 7f0e55000700 1 Processor -- start 2024-05-03T18:25:42.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.725+0000 7f0e55000700 1 -- start start 2024-05-03T18:25:42.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.726+0000 7f0e55000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0e500756d0 0x7f0e5019bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:42.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.726+0000 7f0e55000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e50075ff0 0x7f0e5019c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:42.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.726+0000 7f0e55000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e5010fea0 0x7f0e501a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:42.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.726+0000 7f0e55000700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0e50115770 con 0x7f0e50075ff0 2024-05-03T18:25:42.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.726+0000 7f0e55000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0e501155f0 con 0x7f0e5010fea0 2024-05-03T18:25:42.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.726+0000 7f0e55000700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f0e501158f0 con 0x7f0e500756d0 2024-05-03T18:25:42.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.726+0000 7f0e4d400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e50075ff0 0x7f0e5019c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:42.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.726+0000 7f0e4d400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e50075ff0 0x7f0e5019c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36214/0 (socket says 172.21.15.33:36214) 2024-05-03T18:25:42.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.726+0000 7f0e4d400700 1 -- 172.21.15.33:0/473759830 learned_addr learned my addr 172.21.15.33:0/473759830 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:25:42.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.727+0000 7f0e4de00700 1 --2- 172.21.15.33:0/473759830 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0e500756d0 0x7f0e5019bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:42.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.727+0000 7f0e4e800700 1 --2- 172.21.15.33:0/473759830 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e5010fea0 0x7f0e501a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:42.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.727+0000 7f0e4d400700 1 -- 172.21.15.33:0/473759830 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0e500756d0 msgr2=0x7f0e5019bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:42.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.727+0000 7f0e4d400700 1 --2- 172.21.15.33:0/473759830 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0e500756d0 0x7f0e5019bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:42.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.727+0000 7f0e4d400700 1 -- 172.21.15.33:0/473759830 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e5010fea0 msgr2=0x7f0e501a05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:42.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.727+0000 7f0e4d400700 1 --2- 172.21.15.33:0/473759830 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e5010fea0 0x7f0e501a05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:42.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.727+0000 7f0e4d400700 1 -- 172.21.15.33:0/473759830 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e501a0d30 con 0x7f0e50075ff0 2024-05-03T18:25:42.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.727+0000 7f0e4d400700 1 --2- 172.21.15.33:0/473759830 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e50075ff0 0x7f0e5019c260 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c00d980 tx=0x7f0e3c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:25:42.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.728+0000 7f0e46a00700 1 -- 172.21.15.33:0/473759830 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e3c017030 con 0x7f0e50075ff0 2024-05-03T18:25:42.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.728+0000 7f0e46a00700 1 -- 172.21.15.33:0/473759830 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0e3c0044b0 con 0x7f0e50075ff0 2024-05-03T18:25:42.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.728+0000 7f0e55000700 1 -- 172.21.15.33:0/473759830 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e501a1020 con 0x7f0e50075ff0 2024-05-03T18:25:42.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.728+0000 7f0e46a00700 1 -- 172.21.15.33:0/473759830 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e3c0038e0 con 0x7f0e50075ff0 2024-05-03T18:25:42.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.728+0000 7f0e55000700 1 -- 172.21.15.33:0/473759830 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e501a88a0 con 0x7f0e50075ff0 2024-05-03T18:25:42.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.729+0000 7f0e3be00700 1 -- 172.21.15.33:0/473759830 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e5006a190 con 0x7f0e50075ff0 2024-05-03T18:25:42.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.732+0000 7f0e46a00700 1 -- 172.21.15.33:0/473759830 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f0e3c01a070 con 0x7f0e50075ff0 2024-05-03T18:25:42.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.733+0000 7f0e46a00700 1 --2- 172.21.15.33:0/473759830 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f0e340b2910 0x7f0e340b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:42.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.733+0000 7f0e46a00700 1 -- 172.21.15.33:0/473759830 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f0e3c0e1bb0 con 0x7f0e50075ff0 2024-05-03T18:25:42.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.734+0000 7f0e4de00700 1 --2- 172.21.15.33:0/473759830 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f0e340b2910 0x7f0e340b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:42.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.734+0000 7f0e4de00700 1 --2- 172.21.15.33:0/473759830 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f0e340b2910 0x7f0e340b4dd0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0e40003760 tx=0x7f0e40005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:25:42.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.736+0000 7f0e46a00700 1 -- 172.21.15.33:0/473759830 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f0e3c009730 con 0x7f0e50075ff0 2024-05-03T18:25:42.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.758+0000 7f6d68400700 1 -- 172.21.15.33:0/1632734607 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f6d601a89d0 con 0x7f6d601084d0 2024-05-03T18:25:42.759 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:25:42 smithi033 ceph-mon[28330]: pgmap v480: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:25:42.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.759+0000 7f6d5e000700 1 -- 172.21.15.33:0/1632734607 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f6d580a96b0 con 0x7f6d601084d0 2024-05-03T18:25:42.760 INFO:teuthology.orchestra.run.smithi033.stdout:201863463001 2024-05-03T18:25:42.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.762+0000 7f6d68400700 1 -- 172.21.15.33:0/1632734607 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f6d480b2990 msgr2=0x7f6d480b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:42.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.762+0000 7f6d68400700 1 --2- 172.21.15.33:0/1632734607 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f6d480b2990 0x7f6d480b4e50 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6d54009990 tx=0x7f6d5401a040 comp rx=0 tx=0).stop 2024-05-03T18:25:42.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.762+0000 7f6d68400700 1 -- 172.21.15.33:0/1632734607 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d601084d0 msgr2=0x7f6d6019ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:42.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.762+0000 7f6d68400700 1 --2- 172.21.15.33:0/1632734607 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d601084d0 0x7f6d6019ba90 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f6d58007ee0 tx=0x7f6d5800d3b0 comp rx=0 tx=0).stop 2024-05-03T18:25:42.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.762+0000 7f6d68400700 1 -- 172.21.15.33:0/1632734607 shutdown_connections 2024-05-03T18:25:42.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.762+0000 7f6d68400700 1 --2- 172.21.15.33:0/1632734607 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d601084d0 0x7f6d6019ba90 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:42.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.762+0000 7f6d68400700 1 --2- 172.21.15.33:0/1632734607 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f6d480b2990 0x7f6d480b4e50 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:42.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.762+0000 7f6d68400700 1 --2- 172.21.15.33:0/1632734607 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d601026c0 0x7f6d601a18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:42.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.762+0000 7f6d68400700 1 --2- 172.21.15.33:0/1632734607 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d60103060 0x7f6d601a1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:42.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.762+0000 7f6d68400700 1 -- 172.21.15.33:0/1632734607 >> 172.21.15.33:0/1632734607 conn(0x7f6d60079170 msgr2=0x7f6d6006bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:25:42.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.762+0000 7f6d68400700 1 -- 172.21.15.33:0/1632734607 shutdown_connections 2024-05-03T18:25:42.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.763+0000 7f6d68400700 1 -- 172.21.15.33:0/1632734607 wait complete. 2024-05-03T18:25:42.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.980+0000 7f0e3be00700 1 -- 172.21.15.33:0/473759830 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f0e5019d080 con 0x7f0e50075ff0 2024-05-03T18:25:42.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.981+0000 7f0e46a00700 1 -- 172.21.15.33:0/473759830 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f0e3c0a9390 con 0x7f0e50075ff0 2024-05-03T18:25:42.982 INFO:teuthology.orchestra.run.smithi033.stdout:154618822757 2024-05-03T18:25:42.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.984+0000 7f0e3be00700 1 -- 172.21.15.33:0/473759830 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f0e340b2910 msgr2=0x7f0e340b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:42.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.984+0000 7f0e3be00700 1 --2- 172.21.15.33:0/473759830 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f0e340b2910 0x7f0e340b4dd0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0e40003760 tx=0x7f0e40005fd0 comp rx=0 tx=0).stop 2024-05-03T18:25:42.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.984+0000 7f0e3be00700 1 -- 172.21.15.33:0/473759830 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e50075ff0 msgr2=0x7f0e5019c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:42.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.984+0000 7f0e3be00700 1 --2- 172.21.15.33:0/473759830 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e50075ff0 0x7f0e5019c260 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c00d980 tx=0x7f0e3c00dc90 comp rx=0 tx=0).stop 2024-05-03T18:25:42.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.984+0000 7f0e3be00700 1 -- 172.21.15.33:0/473759830 shutdown_connections 2024-05-03T18:25:42.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.984+0000 7f0e3be00700 1 --2- 172.21.15.33:0/473759830 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f0e340b2910 0x7f0e340b4dd0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:42.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.984+0000 7f0e3be00700 1 --2- 172.21.15.33:0/473759830 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e5010fea0 0x7f0e501a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:42.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.984+0000 7f0e3be00700 1 --2- 172.21.15.33:0/473759830 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0e500756d0 0x7f0e5019bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:42.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.984+0000 7f0e3be00700 1 --2- 172.21.15.33:0/473759830 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e50075ff0 0x7f0e5019c260 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:42.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.984+0000 7f0e3be00700 1 -- 172.21.15.33:0/473759830 >> 172.21.15.33:0/473759830 conn(0x7f0e5007a600 msgr2=0x7f0e50101b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:25:42.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.985+0000 7f0e3be00700 1 -- 172.21.15.33:0/473759830 shutdown_connections 2024-05-03T18:25:42.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:42.985+0000 7f0e3be00700 1 -- 172.21.15.33:0/473759830 wait complete. 2024-05-03T18:25:43.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:25:42 smithi099 ceph-mon[31658]: pgmap v480: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:25:43.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:25:42 smithi150 ceph-mon[32492]: pgmap v480: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:25:43.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:25:43 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/1632734607' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-03T18:25:43.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:25:43 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/473759830' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-03T18:25:44.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:25:43 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/1632734607' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-03T18:25:44.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:25:43 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/473759830' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-03T18:25:44.104 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462994 got 201863463001 for osd.7 2024-05-03T18:25:44.104 DEBUG:teuthology.parallel:result is None 2024-05-03T18:25:44.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:25:43 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/1632734607' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-03T18:25:44.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:25:43 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/473759830' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-03T18:25:44.234 INFO:tasks.cephadm.ceph_manager.ceph:need seq 154618822752 got 154618822757 for osd.5 2024-05-03T18:25:44.234 DEBUG:teuthology.parallel:result is None 2024-05-03T18:25:44.234 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-03T18:25:44.234 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-03T18:25:44.698 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:25:44.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:25:44 smithi033 ceph-mon[28330]: pgmap v481: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:25:45.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:25:44 smithi099 ceph-mon[31658]: pgmap v481: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:25:45.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:25:44 smithi150 ceph-mon[32492]: pgmap v481: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:25:46.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:46.974+0000 7fcd6f000700 1 -- 172.21.15.33:0/984851747 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd68105f90 msgr2=0x7fcd681124a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:46.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:46.974+0000 7fcd6f000700 1 --2- 172.21.15.33:0/984851747 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd68105f90 0x7fcd681124a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fcd6000b0a0 tx=0x7fcd6000b3b0 comp rx=0 tx=0).stop 2024-05-03T18:25:46.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:46.974+0000 7fcd6f000700 1 -- 172.21.15.33:0/984851747 shutdown_connections 2024-05-03T18:25:46.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:46.974+0000 7fcd6f000700 1 --2- 172.21.15.33:0/984851747 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd68105f90 0x7fcd681124a0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:46.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:46.974+0000 7fcd6f000700 1 --2- 172.21.15.33:0/984851747 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcd6810b660 0x7fcd6810ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:46.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:46.974+0000 7fcd6f000700 1 --2- 172.21.15.33:0/984851747 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd681055d0 0x7fcd68105a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:46.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:46.974+0000 7fcd6f000700 1 -- 172.21.15.33:0/984851747 >> 172.21.15.33:0/984851747 conn(0x7fcd681010c0 msgr2=0x7fcd681034e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:25:46.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:46.974+0000 7fcd6f000700 1 -- 172.21.15.33:0/984851747 shutdown_connections 2024-05-03T18:25:46.977 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:25:46 smithi033 ceph-mon[28330]: pgmap v482: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:25:46.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:46.974+0000 7fcd6f000700 1 -- 172.21.15.33:0/984851747 wait complete. 2024-05-03T18:25:46.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:46.976+0000 7fcd6f000700 1 Processor -- start 2024-05-03T18:25:46.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:46.976+0000 7fcd6f000700 1 -- start start 2024-05-03T18:25:46.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:46.976+0000 7fcd6f000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd681055d0 0x7fcd681a1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:46.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:46.976+0000 7fcd6f000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd68105f90 0x7fcd681a1c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:46.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:46.976+0000 7fcd6f000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcd6810b660 0x7fcd6819b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:46.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:46.976+0000 7fcd6f000700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcd681156b0 con 0x7fcd68105f90 2024-05-03T18:25:46.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:46.976+0000 7fcd6f000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fcd68115530 con 0x7fcd681055d0 2024-05-03T18:25:46.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:46.976+0000 7fcd6f000700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fcd68115830 con 0x7fcd6810b660 2024-05-03T18:25:46.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:46.977+0000 7fcd6d200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd68105f90 0x7fcd681a1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:46.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:46.977+0000 7fcd6dc00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd681055d0 0x7fcd681a1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:46.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:46.977+0000 7fcd6d200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd68105f90 0x7fcd681a1c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57702/0 (socket says 172.21.15.33:57702) 2024-05-03T18:25:46.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:46.977+0000 7fcd6d200700 1 -- 172.21.15.33:0/107853349 learned_addr learned my addr 172.21.15.33:0/107853349 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:25:46.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:46.977+0000 7fcd6e600700 1 --2- 172.21.15.33:0/107853349 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcd6810b660 0x7fcd6819b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:46.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:46.978+0000 7fcd6d200700 1 -- 172.21.15.33:0/107853349 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcd6810b660 msgr2=0x7fcd6819b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:46.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:46.978+0000 7fcd6d200700 1 --2- 172.21.15.33:0/107853349 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcd6810b660 0x7fcd6819b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:46.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:46.978+0000 7fcd6d200700 1 -- 172.21.15.33:0/107853349 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd681055d0 msgr2=0x7fcd681a1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:46.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:46.978+0000 7fcd6d200700 1 --2- 172.21.15.33:0/107853349 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd681055d0 0x7fcd681a1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:46.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:46.978+0000 7fcd6d200700 1 -- 172.21.15.33:0/107853349 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd6819c050 con 0x7fcd68105f90 2024-05-03T18:25:46.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:46.978+0000 7fcd6d200700 1 --2- 172.21.15.33:0/107853349 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd68105f90 0x7fcd681a1c70 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fcd5400d980 tx=0x7fcd5400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:25:46.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:46.979+0000 7fcd66000700 1 -- 172.21.15.33:0/107853349 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd54016070 con 0x7fcd68105f90 2024-05-03T18:25:46.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:46.979+0000 7fcd66000700 1 -- 172.21.15.33:0/107853349 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcd54004830 con 0x7fcd68105f90 2024-05-03T18:25:46.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:46.979+0000 7fcd6f000700 1 -- 172.21.15.33:0/107853349 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd6819c2e0 con 0x7fcd68105f90 2024-05-03T18:25:46.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:46.979+0000 7fcd66000700 1 -- 172.21.15.33:0/107853349 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd540054c0 con 0x7fcd68105f90 2024-05-03T18:25:46.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:46.979+0000 7fcd6f000700 1 -- 172.21.15.33:0/107853349 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd681a8600 con 0x7fcd68105f90 2024-05-03T18:25:46.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:46.980+0000 7fcd6f000700 1 -- 172.21.15.33:0/107853349 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd6806a190 con 0x7fcd68105f90 2024-05-03T18:25:46.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:46.983+0000 7fcd66000700 1 -- 172.21.15.33:0/107853349 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fcd54004220 con 0x7fcd68105f90 2024-05-03T18:25:46.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:46.984+0000 7fcd66000700 1 --2- 172.21.15.33:0/107853349 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fcd500b2910 0x7fcd500b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:46.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:46.985+0000 7fcd66000700 1 -- 172.21.15.33:0/107853349 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7fcd540e2a20 con 0x7fcd68105f90 2024-05-03T18:25:46.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:46.985+0000 7fcd6dc00700 1 --2- 172.21.15.33:0/107853349 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fcd500b2910 0x7fcd500b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:46.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:46.985+0000 7fcd6dc00700 1 --2- 172.21.15.33:0/107853349 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fcd500b2910 0x7fcd500b4dd0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fcd6819ca30 tx=0x7fcd5c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:25:46.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:46.988+0000 7fcd66000700 1 -- 172.21.15.33:0/107853349 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fcd5401f020 con 0x7fcd68105f90 2024-05-03T18:25:47.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:25:46 smithi099 ceph-mon[31658]: pgmap v482: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:25:47.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:25:46 smithi150 ceph-mon[32492]: pgmap v482: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:25:47.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:47.218+0000 7fcd6f000700 1 -- 172.21.15.33:0/107853349 --> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fcd6819ce20 con 0x7fcd500b2910 2024-05-03T18:25:47.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:47.250+0000 7fcd66000700 1 -- 172.21.15.33:0/107853349 <== mgr.14152 v2:172.21.15.33:6800/3642895501 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340276 (secure 0 0 0) 0x7fcd6819ce20 con 0x7fcd500b2910 2024-05-03T18:25:47.254 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:25:47.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:47.258+0000 7fcd6f000700 1 -- 172.21.15.33:0/107853349 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fcd500b2910 msgr2=0x7fcd500b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:47.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:47.258+0000 7fcd6f000700 1 --2- 172.21.15.33:0/107853349 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fcd500b2910 0x7fcd500b4dd0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fcd6819ca30 tx=0x7fcd5c00b410 comp rx=0 tx=0).stop 2024-05-03T18:25:47.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:47.258+0000 7fcd6f000700 1 -- 172.21.15.33:0/107853349 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd68105f90 msgr2=0x7fcd681a1c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:47.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:47.258+0000 7fcd6f000700 1 --2- 172.21.15.33:0/107853349 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd68105f90 0x7fcd681a1c70 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fcd5400d980 tx=0x7fcd5400dc90 comp rx=0 tx=0).stop 2024-05-03T18:25:47.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:47.258+0000 7fcd6f000700 1 -- 172.21.15.33:0/107853349 shutdown_connections 2024-05-03T18:25:47.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:47.258+0000 7fcd6f000700 1 --2- 172.21.15.33:0/107853349 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcd6810b660 0x7fcd6819b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:47.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:47.258+0000 7fcd6f000700 1 --2- 172.21.15.33:0/107853349 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fcd500b2910 0x7fcd500b4dd0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:47.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:47.258+0000 7fcd6f000700 1 --2- 172.21.15.33:0/107853349 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd681055d0 0x7fcd681a1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:47.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:47.258+0000 7fcd6f000700 1 --2- 172.21.15.33:0/107853349 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd68105f90 0x7fcd681a1c70 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:47.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:47.258+0000 7fcd6f000700 1 -- 172.21.15.33:0/107853349 >> 172.21.15.33:0/107853349 conn(0x7fcd681010c0 msgr2=0x7fcd68102800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:25:47.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:47.258+0000 7fcd6f000700 1 -- 172.21.15.33:0/107853349 shutdown_connections 2024-05-03T18:25:47.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:47.258+0000 7fcd6f000700 1 -- 172.21.15.33:0/107853349 wait complete. 2024-05-03T18:25:47.262 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-05-03T18:25:48.110 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":482,"stamp":"2024-05-03T18:25:45.411879+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":647515,"num_objects":26,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":329,"num_read_kb":251,"num_write":209,"num_write_kb":2237,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":164,"ondisk_log_size":164,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":367332,"kb_used_data":13928,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124574492,"statfs":{"total":1151940427776,"available":1151564279808,"internally_reserved":0,"allocated":14262272,"data_stored":8097479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":361674042},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006366"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.563614+0000","last_change":"2024-05-03T18:22:25.595640+0000","last_active":"2024-05-03T18:24:21.563614+0000","last_peered":"2024-05-03T18:24:21.563614+0000","last_clean":"2024-05-03T18:24:21.563614+0000","last_became_active":"2024-05-03T18:22:25.595284+0000","last_became_peered":"2024-05-03T18:22:25.595284+0000","last_unstale":"2024-05-03T18:24:21.563614+0000","last_undegraded":"2024-05-03T18:24:21.563614+0000","last_fullsized":"2024-05-03T18:24:21.563614+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:50:08.867670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.802262+0000","last_change":"2024-05-03T18:22:25.584950+0000","last_active":"2024-05-03T18:24:21.802262+0000","last_peered":"2024-05-03T18:24:21.802262+0000","last_clean":"2024-05-03T18:24:21.802262+0000","last_became_active":"2024-05-03T18:22:25.584613+0000","last_became_peered":"2024-05-03T18:22:25.584613+0000","last_unstale":"2024-05-03T18:24:21.802262+0000","last_undegraded":"2024-05-03T18:24:21.802262+0000","last_fullsized":"2024-05-03T18:24:21.802262+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:00:24.959429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.801924+0000","last_change":"2024-05-03T18:22:25.596799+0000","last_active":"2024-05-03T18:24:21.801924+0000","last_peered":"2024-05-03T18:24:21.801924+0000","last_clean":"2024-05-03T18:24:21.801924+0000","last_became_active":"2024-05-03T18:22:25.596386+0000","last_became_peered":"2024-05-03T18:22:25.596386+0000","last_unstale":"2024-05-03T18:24:21.801924+0000","last_undegraded":"2024-05-03T18:24:21.801924+0000","last_fullsized":"2024-05-03T18:24:21.801924+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:20:07.818282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.549382+0000","last_change":"2024-05-03T18:23:20.534672+0000","last_active":"2024-05-03T18:24:21.549382+0000","last_peered":"2024-05-03T18:24:21.549382+0000","last_clean":"2024-05-03T18:24:21.549382+0000","last_became_active":"2024-05-03T18:23:20.534073+0000","last_became_peered":"2024-05-03T18:23:20.534073+0000","last_unstale":"2024-05-03T18:24:21.549382+0000","last_undegraded":"2024-05-03T18:24:21.549382+0000","last_fullsized":"2024-05-03T18:24:21.549382+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:07:40.204299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.572860+0000","last_change":"2024-05-03T18:22:25.589810+0000","last_active":"2024-05-03T18:24:21.572860+0000","last_peered":"2024-05-03T18:24:21.572860+0000","last_clean":"2024-05-03T18:24:21.572860+0000","last_became_active":"2024-05-03T18:22:25.588220+0000","last_became_peered":"2024-05-03T18:22:25.588220+0000","last_unstale":"2024-05-03T18:24:21.572860+0000","last_undegraded":"2024-05-03T18:24:21.572860+0000","last_fullsized":"2024-05-03T18:24:21.572860+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:43:20.904737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.550489+0000","last_change":"2024-05-03T18:22:25.603148+0000","last_active":"2024-05-03T18:24:21.550489+0000","last_peered":"2024-05-03T18:24:21.550489+0000","last_clean":"2024-05-03T18:24:21.550489+0000","last_became_active":"2024-05-03T18:22:25.602743+0000","last_became_peered":"2024-05-03T18:22:25.602743+0000","last_unstale":"2024-05-03T18:24:21.550489+0000","last_undegraded":"2024-05-03T18:24:21.550489+0000","last_fullsized":"2024-05-03T18:24:21.550489+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:49:27.387863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.576774+0000","last_change":"2024-05-03T18:23:20.532077+0000","last_active":"2024-05-03T18:24:21.576774+0000","last_peered":"2024-05-03T18:24:21.576774+0000","last_clean":"2024-05-03T18:24:21.576774+0000","last_became_active":"2024-05-03T18:23:20.531528+0000","last_became_peered":"2024-05-03T18:23:20.531528+0000","last_unstale":"2024-05-03T18:24:21.576774+0000","last_undegraded":"2024-05-03T18:24:21.576774+0000","last_fullsized":"2024-05-03T18:24:21.576774+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:10:37.037114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.627783+0000","last_change":"2024-05-03T18:22:25.600192+0000","last_active":"2024-05-03T18:24:21.627783+0000","last_peered":"2024-05-03T18:24:21.627783+0000","last_clean":"2024-05-03T18:24:21.627783+0000","last_became_active":"2024-05-03T18:22:25.599663+0000","last_became_peered":"2024-05-03T18:22:25.599663+0000","last_unstale":"2024-05-03T18:24:21.627783+0000","last_undegraded":"2024-05-03T18:24:21.627783+0000","last_fullsized":"2024-05-03T18:24:21.627783+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:09:51.614784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.556385+0000","last_change":"2024-05-03T18:22:25.600302+0000","last_active":"2024-05-03T18:24:21.556385+0000","last_peered":"2024-05-03T18:24:21.556385+0000","last_clean":"2024-05-03T18:24:21.556385+0000","last_became_active":"2024-05-03T18:22:25.599832+0000","last_became_peered":"2024-05-03T18:22:25.599832+0000","last_unstale":"2024-05-03T18:24:21.556385+0000","last_undegraded":"2024-05-03T18:24:21.556385+0000","last_fullsized":"2024-05-03T18:24:21.556385+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:52:12.326156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.551747+0000","last_change":"2024-05-03T18:22:25.590892+0000","last_active":"2024-05-03T18:24:21.551747+0000","last_peered":"2024-05-03T18:24:21.551747+0000","last_clean":"2024-05-03T18:24:21.551747+0000","last_became_active":"2024-05-03T18:22:25.590516+0000","last_became_peered":"2024-05-03T18:22:25.590516+0000","last_unstale":"2024-05-03T18:24:21.551747+0000","last_undegraded":"2024-05-03T18:24:21.551747+0000","last_fullsized":"2024-05-03T18:24:21.551747+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:57:57.999234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.578198+0000","last_change":"2024-05-03T18:22:25.597307+0000","last_active":"2024-05-03T18:24:21.578198+0000","last_peered":"2024-05-03T18:24:21.578198+0000","last_clean":"2024-05-03T18:24:21.578198+0000","last_became_active":"2024-05-03T18:22:25.596989+0000","last_became_peered":"2024-05-03T18:22:25.596989+0000","last_unstale":"2024-05-03T18:24:21.578198+0000","last_undegraded":"2024-05-03T18:24:21.578198+0000","last_fullsized":"2024-05-03T18:24:21.578198+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:53:09.003353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.926392+0000","last_change":"2024-05-03T18:22:25.601723+0000","last_active":"2024-05-03T18:24:21.926392+0000","last_peered":"2024-05-03T18:24:21.926392+0000","last_clean":"2024-05-03T18:24:21.926392+0000","last_became_active":"2024-05-03T18:22:25.599428+0000","last_became_peered":"2024-05-03T18:22:25.599428+0000","last_unstale":"2024-05-03T18:24:21.926392+0000","last_undegraded":"2024-05-03T18:24:21.926392+0000","last_fullsized":"2024-05-03T18:24:21.926392+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:41:18.584497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.923716+0000","last_change":"2024-05-03T18:22:25.594885+0000","last_active":"2024-05-03T18:24:21.923716+0000","last_peered":"2024-05-03T18:24:21.923716+0000","last_clean":"2024-05-03T18:24:21.923716+0000","last_became_active":"2024-05-03T18:22:25.593939+0000","last_became_peered":"2024-05-03T18:22:25.593939+0000","last_unstale":"2024-05-03T18:24:21.923716+0000","last_undegraded":"2024-05-03T18:24:21.923716+0000","last_fullsized":"2024-05-03T18:24:21.923716+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:03:30.052886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.801635+0000","last_change":"2024-05-03T18:22:25.595228+0000","last_active":"2024-05-03T18:24:21.801635+0000","last_peered":"2024-05-03T18:24:21.801635+0000","last_clean":"2024-05-03T18:24:21.801635+0000","last_became_active":"2024-05-03T18:22:25.589825+0000","last_became_peered":"2024-05-03T18:22:25.589825+0000","last_unstale":"2024-05-03T18:24:21.801635+0000","last_undegraded":"2024-05-03T18:24:21.801635+0000","last_fullsized":"2024-05-03T18:24:21.801635+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:44:05.638939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.801321+0000","last_change":"2024-05-03T18:22:25.595630+0000","last_active":"2024-05-03T18:24:21.801321+0000","last_peered":"2024-05-03T18:24:21.801321+0000","last_clean":"2024-05-03T18:24:21.801321+0000","last_became_active":"2024-05-03T18:22:25.591869+0000","last_became_peered":"2024-05-03T18:22:25.591869+0000","last_unstale":"2024-05-03T18:24:21.801321+0000","last_undegraded":"2024-05-03T18:24:21.801321+0000","last_fullsized":"2024-05-03T18:24:21.801321+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:27:43.579329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.555855+0000","last_change":"2024-05-03T18:22:25.589482+0000","last_active":"2024-05-03T18:24:21.555855+0000","last_peered":"2024-05-03T18:24:21.555855+0000","last_clean":"2024-05-03T18:24:21.555855+0000","last_became_active":"2024-05-03T18:22:25.589142+0000","last_became_peered":"2024-05-03T18:22:25.589142+0000","last_unstale":"2024-05-03T18:24:21.555855+0000","last_undegraded":"2024-05-03T18:24:21.555855+0000","last_fullsized":"2024-05-03T18:24:21.555855+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:37:55.472823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.563264+0000","last_change":"2024-05-03T18:22:25.594569+0000","last_active":"2024-05-03T18:24:21.563264+0000","last_peered":"2024-05-03T18:24:21.563264+0000","last_clean":"2024-05-03T18:24:21.563264+0000","last_became_active":"2024-05-03T18:22:25.594218+0000","last_became_peered":"2024-05-03T18:22:25.594218+0000","last_unstale":"2024-05-03T18:24:21.563264+0000","last_undegraded":"2024-05-03T18:24:21.563264+0000","last_fullsized":"2024-05-03T18:24:21.563264+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:30:42.775673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.549731+0000","last_change":"2024-05-03T18:22:25.597426+0000","last_active":"2024-05-03T18:24:21.549731+0000","last_peered":"2024-05-03T18:24:21.549731+0000","last_clean":"2024-05-03T18:24:21.549731+0000","last_became_active":"2024-05-03T18:22:25.594524+0000","last_became_peered":"2024-05-03T18:22:25.594524+0000","last_unstale":"2024-05-03T18:24:21.549731+0000","last_undegraded":"2024-05-03T18:24:21.549731+0000","last_fullsized":"2024-05-03T18:24:21.549731+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:41:36.984253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.553049+0000","last_change":"2024-05-03T18:22:25.588002+0000","last_active":"2024-05-03T18:24:21.553049+0000","last_peered":"2024-05-03T18:24:21.553049+0000","last_clean":"2024-05-03T18:24:21.553049+0000","last_became_active":"2024-05-03T18:22:25.587547+0000","last_became_peered":"2024-05-03T18:22:25.587547+0000","last_unstale":"2024-05-03T18:24:21.553049+0000","last_undegraded":"2024-05-03T18:24:21.553049+0000","last_fullsized":"2024-05-03T18:24:21.553049+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:47:14.115517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.551614+0000","last_change":"2024-05-03T18:22:40.754602+0000","last_active":"2024-05-03T18:24:21.551614+0000","last_peered":"2024-05-03T18:24:21.551614+0000","last_clean":"2024-05-03T18:24:21.551614+0000","last_became_active":"2024-05-03T18:22:40.752917+0000","last_became_peered":"2024-05-03T18:22:40.752917+0000","last_unstale":"2024-05-03T18:24:21.551614+0000","last_undegraded":"2024-05-03T18:24:21.551614+0000","last_fullsized":"2024-05-03T18:24:21.551614+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:51:44.711811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"78'4","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.550063+0000","last_change":"2024-05-03T18:22:21.558560+0000","last_active":"2024-05-03T18:24:21.550063+0000","last_peered":"2024-05-03T18:24:21.550063+0000","last_clean":"2024-05-03T18:24:21.550063+0000","last_became_active":"2024-05-03T18:22:21.558185+0000","last_became_peered":"2024-05-03T18:22:21.558185+0000","last_unstale":"2024-05-03T18:24:21.550063+0000","last_undegraded":"2024-05-03T18:24:21.550063+0000","last_fullsized":"2024-05-03T18:24:21.550063+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:18:19.666724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.628787+0000","last_change":"2024-05-03T18:22:25.603934+0000","last_active":"2024-05-03T18:24:21.628787+0000","last_peered":"2024-05-03T18:24:21.628787+0000","last_clean":"2024-05-03T18:24:21.628787+0000","last_became_active":"2024-05-03T18:22:25.603529+0000","last_became_peered":"2024-05-03T18:22:25.603529+0000","last_unstale":"2024-05-03T18:24:21.628787+0000","last_undegraded":"2024-05-03T18:24:21.628787+0000","last_fullsized":"2024-05-03T18:24:21.628787+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:08:50.288871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.924818+0000","last_change":"2024-05-03T18:23:20.576254+0000","last_active":"2024-05-03T18:24:21.924818+0000","last_peered":"2024-05-03T18:24:21.924818+0000","last_clean":"2024-05-03T18:24:21.924818+0000","last_became_active":"2024-05-03T18:23:20.575774+0000","last_became_peered":"2024-05-03T18:23:20.575774+0000","last_unstale":"2024-05-03T18:24:21.924818+0000","last_undegraded":"2024-05-03T18:24:21.924818+0000","last_fullsized":"2024-05-03T18:24:21.924818+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:22:39.090938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"78'1","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.628643+0000","last_change":"2024-05-03T18:22:21.734171+0000","last_active":"2024-05-03T18:24:21.628643+0000","last_peered":"2024-05-03T18:24:21.628643+0000","last_clean":"2024-05-03T18:24:21.628643+0000","last_became_active":"2024-05-03T18:22:21.733301+0000","last_became_peered":"2024-05-03T18:22:21.733301+0000","last_unstale":"2024-05-03T18:24:21.628643+0000","last_undegraded":"2024-05-03T18:24:21.628643+0000","last_fullsized":"2024-05-03T18:24:21.628643+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:09:48.861247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.802381+0000","last_change":"2024-05-03T18:22:25.598016+0000","last_active":"2024-05-03T18:24:21.802381+0000","last_peered":"2024-05-03T18:24:21.802381+0000","last_clean":"2024-05-03T18:24:21.802381+0000","last_became_active":"2024-05-03T18:22:25.594888+0000","last_became_peered":"2024-05-03T18:22:25.594888+0000","last_unstale":"2024-05-03T18:24:21.802381+0000","last_undegraded":"2024-05-03T18:24:21.802381+0000","last_fullsized":"2024-05-03T18:24:21.802381+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:47:46.400247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.925114+0000","last_change":"2024-05-03T18:22:40.755148+0000","last_active":"2024-05-03T18:24:21.925114+0000","last_peered":"2024-05-03T18:24:21.925114+0000","last_clean":"2024-05-03T18:24:21.925114+0000","last_became_active":"2024-05-03T18:22:40.754656+0000","last_became_peered":"2024-05-03T18:22:40.754656+0000","last_unstale":"2024-05-03T18:24:21.925114+0000","last_undegraded":"2024-05-03T18:24:21.925114+0000","last_fullsized":"2024-05-03T18:24:21.925114+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:48:23.124936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"78'2","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.573377+0000","last_change":"2024-05-03T18:22:21.568060+0000","last_active":"2024-05-03T18:24:21.573377+0000","last_peered":"2024-05-03T18:24:21.573377+0000","last_clean":"2024-05-03T18:24:21.573377+0000","last_became_active":"2024-05-03T18:22:21.567434+0000","last_became_peered":"2024-05-03T18:22:21.567434+0000","last_unstale":"2024-05-03T18:24:21.573377+0000","last_undegraded":"2024-05-03T18:24:21.573377+0000","last_fullsized":"2024-05-03T18:24:21.573377+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:06:05.802621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.556074+0000","last_change":"2024-05-03T18:22:25.597777+0000","last_active":"2024-05-03T18:24:21.556074+0000","last_peered":"2024-05-03T18:24:21.556074+0000","last_clean":"2024-05-03T18:24:21.556074+0000","last_became_active":"2024-05-03T18:22:25.597391+0000","last_became_peered":"2024-05-03T18:22:25.597391+0000","last_unstale":"2024-05-03T18:24:21.556074+0000","last_undegraded":"2024-05-03T18:24:21.556074+0000","last_fullsized":"2024-05-03T18:24:21.556074+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:02:24.708193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.925062+0000","last_change":"2024-05-03T18:22:40.752318+0000","last_active":"2024-05-03T18:24:21.925062+0000","last_peered":"2024-05-03T18:24:21.925062+0000","last_clean":"2024-05-03T18:24:21.925062+0000","last_became_active":"2024-05-03T18:22:40.751433+0000","last_became_peered":"2024-05-03T18:22:40.751433+0000","last_unstale":"2024-05-03T18:24:21.925062+0000","last_undegraded":"2024-05-03T18:24:21.925062+0000","last_fullsized":"2024-05-03T18:24:21.925062+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:21:55.746143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.583558+0000","last_change":"2024-05-03T18:24:21.582330+0000","last_active":"2024-05-03T18:24:21.583558+0000","last_peered":"2024-05-03T18:24:21.583558+0000","last_clean":"2024-05-03T18:24:21.583558+0000","last_became_active":"2024-05-03T18:24:21.572706+0000","last_became_peered":"2024-05-03T18:24:21.572706+0000","last_unstale":"2024-05-03T18:24:21.583558+0000","last_undegraded":"2024-05-03T18:24:21.583558+0000","last_fullsized":"2024-05-03T18:24:21.583558+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:55:40.233995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.572852+0000","last_change":"2024-05-03T18:23:20.575102+0000","last_active":"2024-05-03T18:24:21.572852+0000","last_peered":"2024-05-03T18:24:21.572852+0000","last_clean":"2024-05-03T18:24:21.572852+0000","last_became_active":"2024-05-03T18:23:20.574657+0000","last_became_peered":"2024-05-03T18:23:20.574657+0000","last_unstale":"2024-05-03T18:24:21.572852+0000","last_undegraded":"2024-05-03T18:24:21.572852+0000","last_fullsized":"2024-05-03T18:24:21.572852+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:08:26.280866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.552660+0000","last_change":"2024-05-03T18:22:40.761052+0000","last_active":"2024-05-03T18:24:21.552660+0000","last_peered":"2024-05-03T18:24:21.552660+0000","last_clean":"2024-05-03T18:24:21.552660+0000","last_became_active":"2024-05-03T18:22:40.760179+0000","last_became_peered":"2024-05-03T18:22:40.760179+0000","last_unstale":"2024-05-03T18:24:21.552660+0000","last_undegraded":"2024-05-03T18:24:21.552660+0000","last_fullsized":"2024-05-03T18:24:21.552660+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:44:48.610680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"78'1","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.568744+0000","last_change":"2024-05-03T18:22:21.563421+0000","last_active":"2024-05-03T18:24:21.568744+0000","last_peered":"2024-05-03T18:24:21.568744+0000","last_clean":"2024-05-03T18:24:21.568744+0000","last_became_active":"2024-05-03T18:22:21.560991+0000","last_became_peered":"2024-05-03T18:22:21.560991+0000","last_unstale":"2024-05-03T18:24:21.568744+0000","last_undegraded":"2024-05-03T18:24:21.568744+0000","last_fullsized":"2024-05-03T18:24:21.568744+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:50:32.640699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.570792+0000","last_change":"2024-05-03T18:22:25.599049+0000","last_active":"2024-05-03T18:24:21.570792+0000","last_peered":"2024-05-03T18:24:21.570792+0000","last_clean":"2024-05-03T18:24:21.570792+0000","last_became_active":"2024-05-03T18:22:25.597879+0000","last_became_peered":"2024-05-03T18:22:25.597879+0000","last_unstale":"2024-05-03T18:24:21.570792+0000","last_undegraded":"2024-05-03T18:24:21.570792+0000","last_fullsized":"2024-05-03T18:24:21.570792+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:51:08.487499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.552741+0000","last_change":"2024-05-03T18:22:40.760908+0000","last_active":"2024-05-03T18:24:21.552741+0000","last_peered":"2024-05-03T18:24:21.552741+0000","last_clean":"2024-05-03T18:24:21.552741+0000","last_became_active":"2024-05-03T18:22:40.760331+0000","last_became_peered":"2024-05-03T18:22:40.760331+0000","last_unstale":"2024-05-03T18:24:21.552741+0000","last_undegraded":"2024-05-03T18:24:21.552741+0000","last_fullsized":"2024-05-03T18:24:21.552741+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:27:08.216015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"78'1","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.550440+0000","last_change":"2024-05-03T18:22:21.565328+0000","last_active":"2024-05-03T18:24:21.550440+0000","last_peered":"2024-05-03T18:24:21.550440+0000","last_clean":"2024-05-03T18:24:21.550440+0000","last_became_active":"2024-05-03T18:22:21.564583+0000","last_became_peered":"2024-05-03T18:22:21.564583+0000","last_unstale":"2024-05-03T18:24:21.550440+0000","last_undegraded":"2024-05-03T18:24:21.550440+0000","last_fullsized":"2024-05-03T18:24:21.550440+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:58:32.370458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.569755+0000","last_change":"2024-05-03T18:22:25.607585+0000","last_active":"2024-05-03T18:24:21.569755+0000","last_peered":"2024-05-03T18:24:21.569755+0000","last_clean":"2024-05-03T18:24:21.569755+0000","last_became_active":"2024-05-03T18:22:25.607168+0000","last_became_peered":"2024-05-03T18:22:25.607168+0000","last_unstale":"2024-05-03T18:24:21.569755+0000","last_undegraded":"2024-05-03T18:24:21.569755+0000","last_fullsized":"2024-05-03T18:24:21.569755+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:04:39.750823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.562868+0000","last_change":"2024-05-03T18:22:40.754744+0000","last_active":"2024-05-03T18:24:21.562868+0000","last_peered":"2024-05-03T18:24:21.562868+0000","last_clean":"2024-05-03T18:24:21.562868+0000","last_became_active":"2024-05-03T18:22:40.754262+0000","last_became_peered":"2024-05-03T18:22:40.754262+0000","last_unstale":"2024-05-03T18:24:21.562868+0000","last_undegraded":"2024-05-03T18:24:21.562868+0000","last_fullsized":"2024-05-03T18:24:21.562868+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:11:47.979546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.923003+0000","last_change":"2024-05-03T18:22:21.550048+0000","last_active":"2024-05-03T18:24:21.923003+0000","last_peered":"2024-05-03T18:24:21.923003+0000","last_clean":"2024-05-03T18:24:21.923003+0000","last_became_active":"2024-05-03T18:22:21.549334+0000","last_became_peered":"2024-05-03T18:22:21.549334+0000","last_unstale":"2024-05-03T18:24:21.923003+0000","last_undegraded":"2024-05-03T18:24:21.923003+0000","last_fullsized":"2024-05-03T18:24:21.923003+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:52:54.401185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.627661+0000","last_change":"2024-05-03T18:22:25.583074+0000","last_active":"2024-05-03T18:24:21.627661+0000","last_peered":"2024-05-03T18:24:21.627661+0000","last_clean":"2024-05-03T18:24:21.627661+0000","last_became_active":"2024-05-03T18:22:25.582637+0000","last_became_peered":"2024-05-03T18:22:25.582637+0000","last_unstale":"2024-05-03T18:24:21.627661+0000","last_undegraded":"2024-05-03T18:24:21.627661+0000","last_fullsized":"2024-05-03T18:24:21.627661+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:20:09.021402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.802120+0000","last_change":"2024-05-03T18:22:40.748509+0000","last_active":"2024-05-03T18:24:21.802120+0000","last_peered":"2024-05-03T18:24:21.802120+0000","last_clean":"2024-05-03T18:24:21.802120+0000","last_became_active":"2024-05-03T18:22:40.748051+0000","last_became_peered":"2024-05-03T18:22:40.748051+0000","last_unstale":"2024-05-03T18:24:21.802120+0000","last_undegraded":"2024-05-03T18:24:21.802120+0000","last_fullsized":"2024-05-03T18:24:21.802120+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:08:44.389460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.567340+0000","last_change":"2024-05-03T18:22:21.562185+0000","last_active":"2024-05-03T18:24:21.567340+0000","last_peered":"2024-05-03T18:24:21.567340+0000","last_clean":"2024-05-03T18:24:21.567340+0000","last_became_active":"2024-05-03T18:22:21.560523+0000","last_became_peered":"2024-05-03T18:22:21.560523+0000","last_unstale":"2024-05-03T18:24:21.567340+0000","last_undegraded":"2024-05-03T18:24:21.567340+0000","last_fullsized":"2024-05-03T18:24:21.567340+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:05:41.788286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.548693+0000","last_change":"2024-05-03T18:22:25.583895+0000","last_active":"2024-05-03T18:24:21.548693+0000","last_peered":"2024-05-03T18:24:21.548693+0000","last_clean":"2024-05-03T18:24:21.548693+0000","last_became_active":"2024-05-03T18:22:25.583503+0000","last_became_peered":"2024-05-03T18:22:25.583503+0000","last_unstale":"2024-05-03T18:24:21.548693+0000","last_undegraded":"2024-05-03T18:24:21.548693+0000","last_fullsized":"2024-05-03T18:24:21.548693+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:45:32.404394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.628056+0000","last_change":"2024-05-03T18:22:40.752814+0000","last_active":"2024-05-03T18:24:21.628056+0000","last_peered":"2024-05-03T18:24:21.628056+0000","last_clean":"2024-05-03T18:24:21.628056+0000","last_became_active":"2024-05-03T18:22:40.752328+0000","last_became_peered":"2024-05-03T18:22:40.752328+0000","last_unstale":"2024-05-03T18:24:21.628056+0000","last_undegraded":"2024-05-03T18:24:21.628056+0000","last_fullsized":"2024-05-03T18:24:21.628056+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:16:39.488122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.551544+0000","last_change":"2024-05-03T18:22:21.566266+0000","last_active":"2024-05-03T18:24:21.551544+0000","last_peered":"2024-05-03T18:24:21.551544+0000","last_clean":"2024-05-03T18:24:21.551544+0000","last_became_active":"2024-05-03T18:22:21.565612+0000","last_became_peered":"2024-05-03T18:22:21.565612+0000","last_unstale":"2024-05-03T18:24:21.551544+0000","last_undegraded":"2024-05-03T18:24:21.551544+0000","last_fullsized":"2024-05-03T18:24:21.551544+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:22:57.216912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.557537+0000","last_change":"2024-05-03T18:22:25.596182+0000","last_active":"2024-05-03T18:24:21.557537+0000","last_peered":"2024-05-03T18:24:21.557537+0000","last_clean":"2024-05-03T18:24:21.557537+0000","last_became_active":"2024-05-03T18:22:25.595472+0000","last_became_peered":"2024-05-03T18:22:25.595472+0000","last_unstale":"2024-05-03T18:24:21.557537+0000","last_undegraded":"2024-05-03T18:24:21.557537+0000","last_fullsized":"2024-05-03T18:24:21.557537+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:48:49.583879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.548535+0000","last_change":"2024-05-03T18:22:40.756378+0000","last_active":"2024-05-03T18:24:21.548535+0000","last_peered":"2024-05-03T18:24:21.548535+0000","last_clean":"2024-05-03T18:24:21.548535+0000","last_became_active":"2024-05-03T18:22:40.755692+0000","last_became_peered":"2024-05-03T18:22:40.755692+0000","last_unstale":"2024-05-03T18:24:21.548535+0000","last_undegraded":"2024-05-03T18:24:21.548535+0000","last_fullsized":"2024-05-03T18:24:21.548535+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:22:50.269817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.576475+0000","last_change":"2024-05-03T18:22:21.561381+0000","last_active":"2024-05-03T18:24:21.576475+0000","last_peered":"2024-05-03T18:24:21.576475+0000","last_clean":"2024-05-03T18:24:21.576475+0000","last_became_active":"2024-05-03T18:22:21.559411+0000","last_became_peered":"2024-05-03T18:22:21.559411+0000","last_unstale":"2024-05-03T18:24:21.576475+0000","last_undegraded":"2024-05-03T18:24:21.576475+0000","last_fullsized":"2024-05-03T18:24:21.576475+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:04:08.283538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.802450+0000","last_change":"2024-05-03T18:23:20.551361+0000","last_active":"2024-05-03T18:24:21.802450+0000","last_peered":"2024-05-03T18:24:21.802450+0000","last_clean":"2024-05-03T18:24:21.802450+0000","last_became_active":"2024-05-03T18:23:20.550689+0000","last_became_peered":"2024-05-03T18:23:20.550689+0000","last_unstale":"2024-05-03T18:24:21.802450+0000","last_undegraded":"2024-05-03T18:24:21.802450+0000","last_fullsized":"2024-05-03T18:24:21.802450+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:36:52.767110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.925712+0000","last_change":"2024-05-03T18:22:40.758006+0000","last_active":"2024-05-03T18:24:21.925712+0000","last_peered":"2024-05-03T18:24:21.925712+0000","last_clean":"2024-05-03T18:24:21.925712+0000","last_became_active":"2024-05-03T18:22:40.757075+0000","last_became_peered":"2024-05-03T18:22:40.757075+0000","last_unstale":"2024-05-03T18:24:21.925712+0000","last_undegraded":"2024-05-03T18:24:21.925712+0000","last_fullsized":"2024-05-03T18:24:21.925712+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:06:02.390184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"78'3","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.572570+0000","last_change":"2024-05-03T18:22:21.740041+0000","last_active":"2024-05-03T18:24:21.572570+0000","last_peered":"2024-05-03T18:24:21.572570+0000","last_clean":"2024-05-03T18:24:21.572570+0000","last_became_active":"2024-05-03T18:22:21.738777+0000","last_became_peered":"2024-05-03T18:22:21.738777+0000","last_unstale":"2024-05-03T18:24:21.572570+0000","last_undegraded":"2024-05-03T18:24:21.572570+0000","last_fullsized":"2024-05-03T18:24:21.572570+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:35:30.098325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.556215+0000","last_change":"2024-05-03T18:22:25.606854+0000","last_active":"2024-05-03T18:24:21.556215+0000","last_peered":"2024-05-03T18:24:21.556215+0000","last_clean":"2024-05-03T18:24:21.556215+0000","last_became_active":"2024-05-03T18:22:25.606376+0000","last_became_peered":"2024-05-03T18:22:25.606376+0000","last_unstale":"2024-05-03T18:24:21.556215+0000","last_undegraded":"2024-05-03T18:24:21.556215+0000","last_fullsized":"2024-05-03T18:24:21.556215+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:29:59.461181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.548855+0000","last_change":"2024-05-03T18:22:40.752863+0000","last_active":"2024-05-03T18:24:21.548855+0000","last_peered":"2024-05-03T18:24:21.548855+0000","last_clean":"2024-05-03T18:24:21.548855+0000","last_became_active":"2024-05-03T18:22:40.752309+0000","last_became_peered":"2024-05-03T18:22:40.752309+0000","last_unstale":"2024-05-03T18:24:21.548855+0000","last_undegraded":"2024-05-03T18:24:21.548855+0000","last_fullsized":"2024-05-03T18:24:21.548855+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:01:06.110570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"87'30","reported_seq":317,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:25:42.138021+0000","last_change":"2024-05-03T18:22:21.566162+0000","last_active":"2024-05-03T18:25:42.138021+0000","last_peered":"2024-05-03T18:25:42.138021+0000","last_clean":"2024-05-03T18:25:42.138021+0000","last_became_active":"2024-05-03T18:22:21.565641+0000","last_became_peered":"2024-05-03T18:22:21.565641+0000","last_unstale":"2024-05-03T18:25:42.138021+0000","last_undegraded":"2024-05-03T18:25:42.138021+0000","last_fullsized":"2024-05-03T18:25:42.138021+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:23:45.283135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56115,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":242,"num_read_kb":176,"num_write":30,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.574003+0000","last_change":"2024-05-03T18:22:25.586687+0000","last_active":"2024-05-03T18:24:21.574003+0000","last_peered":"2024-05-03T18:24:21.574003+0000","last_clean":"2024-05-03T18:24:21.574003+0000","last_became_active":"2024-05-03T18:22:25.586275+0000","last_became_peered":"2024-05-03T18:22:25.586275+0000","last_unstale":"2024-05-03T18:24:21.574003+0000","last_undegraded":"2024-05-03T18:24:21.574003+0000","last_fullsized":"2024-05-03T18:24:21.574003+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:10:50.339865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.573415+0000","last_change":"2024-05-03T18:22:40.758003+0000","last_active":"2024-05-03T18:24:21.573415+0000","last_peered":"2024-05-03T18:24:21.573415+0000","last_clean":"2024-05-03T18:24:21.573415+0000","last_became_active":"2024-05-03T18:22:40.757559+0000","last_became_peered":"2024-05-03T18:22:40.757559+0000","last_unstale":"2024-05-03T18:24:21.573415+0000","last_undegraded":"2024-05-03T18:24:21.573415+0000","last_fullsized":"2024-05-03T18:24:21.573415+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:54:23.377748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"78'2","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.552571+0000","last_change":"2024-05-03T18:22:21.564546+0000","last_active":"2024-05-03T18:24:21.552571+0000","last_peered":"2024-05-03T18:24:21.552571+0000","last_clean":"2024-05-03T18:24:21.552571+0000","last_became_active":"2024-05-03T18:22:21.564105+0000","last_became_peered":"2024-05-03T18:22:21.564105+0000","last_unstale":"2024-05-03T18:24:21.552571+0000","last_undegraded":"2024-05-03T18:24:21.552571+0000","last_fullsized":"2024-05-03T18:24:21.552571+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:36:07.790643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.923406+0000","last_change":"2024-05-03T18:22:25.584059+0000","last_active":"2024-05-03T18:24:21.923406+0000","last_peered":"2024-05-03T18:24:21.923406+0000","last_clean":"2024-05-03T18:24:21.923406+0000","last_became_active":"2024-05-03T18:22:25.583742+0000","last_became_peered":"2024-05-03T18:22:25.583742+0000","last_unstale":"2024-05-03T18:24:21.923406+0000","last_undegraded":"2024-05-03T18:24:21.923406+0000","last_fullsized":"2024-05-03T18:24:21.923406+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:15:48.288422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.562883+0000","last_change":"2024-05-03T18:22:40.751362+0000","last_active":"2024-05-03T18:24:21.562883+0000","last_peered":"2024-05-03T18:24:21.562883+0000","last_clean":"2024-05-03T18:24:21.562883+0000","last_became_active":"2024-05-03T18:22:40.750365+0000","last_became_peered":"2024-05-03T18:22:40.750365+0000","last_unstale":"2024-05-03T18:24:21.562883+0000","last_undegraded":"2024-05-03T18:24:21.562883+0000","last_fullsized":"2024-05-03T18:24:21.562883+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:35:50.629332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.570011+0000","last_change":"2024-05-03T18:22:21.556123+0000","last_active":"2024-05-03T18:24:21.570011+0000","last_peered":"2024-05-03T18:24:21.570011+0000","last_clean":"2024-05-03T18:24:21.570011+0000","last_became_active":"2024-05-03T18:22:21.554918+0000","last_became_peered":"2024-05-03T18:22:21.554918+0000","last_unstale":"2024-05-03T18:24:21.570011+0000","last_undegraded":"2024-05-03T18:24:21.570011+0000","last_fullsized":"2024-05-03T18:24:21.570011+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:09:12.383601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.577809+0000","last_change":"2024-05-03T18:22:25.601849+0000","last_active":"2024-05-03T18:24:21.577809+0000","last_peered":"2024-05-03T18:24:21.577809+0000","last_clean":"2024-05-03T18:24:21.577809+0000","last_became_active":"2024-05-03T18:22:25.601434+0000","last_became_peered":"2024-05-03T18:22:25.601434+0000","last_unstale":"2024-05-03T18:24:21.577809+0000","last_undegraded":"2024-05-03T18:24:21.577809+0000","last_fullsized":"2024-05-03T18:24:21.577809+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:43:18.273389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.549827+0000","last_change":"2024-05-03T18:22:40.754448+0000","last_active":"2024-05-03T18:24:21.549827+0000","last_peered":"2024-05-03T18:24:21.549827+0000","last_clean":"2024-05-03T18:24:21.549827+0000","last_became_active":"2024-05-03T18:22:40.753122+0000","last_became_peered":"2024-05-03T18:22:40.753122+0000","last_unstale":"2024-05-03T18:24:21.549827+0000","last_undegraded":"2024-05-03T18:24:21.549827+0000","last_fullsized":"2024-05-03T18:24:21.549827+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:43:15.216222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.924400+0000","last_change":"2024-05-03T18:22:21.559292+0000","last_active":"2024-05-03T18:24:21.924400+0000","last_peered":"2024-05-03T18:24:21.924400+0000","last_clean":"2024-05-03T18:24:21.924400+0000","last_became_active":"2024-05-03T18:22:21.557657+0000","last_became_peered":"2024-05-03T18:22:21.557657+0000","last_unstale":"2024-05-03T18:24:21.924400+0000","last_undegraded":"2024-05-03T18:24:21.924400+0000","last_fullsized":"2024-05-03T18:24:21.924400+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:51:57.709387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.576970+0000","last_change":"2024-05-03T18:22:25.604340+0000","last_active":"2024-05-03T18:24:21.576970+0000","last_peered":"2024-05-03T18:24:21.576970+0000","last_clean":"2024-05-03T18:24:21.576970+0000","last_became_active":"2024-05-03T18:22:25.603651+0000","last_became_peered":"2024-05-03T18:22:25.603651+0000","last_unstale":"2024-05-03T18:24:21.576970+0000","last_undegraded":"2024-05-03T18:24:21.576970+0000","last_fullsized":"2024-05-03T18:24:21.576970+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:49:36.379556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.923959+0000","last_change":"2024-05-03T18:22:40.738065+0000","last_active":"2024-05-03T18:24:21.923959+0000","last_peered":"2024-05-03T18:24:21.923959+0000","last_clean":"2024-05-03T18:24:21.923959+0000","last_became_active":"2024-05-03T18:22:40.737546+0000","last_became_peered":"2024-05-03T18:22:40.737546+0000","last_unstale":"2024-05-03T18:24:21.923959+0000","last_undegraded":"2024-05-03T18:24:21.923959+0000","last_fullsized":"2024-05-03T18:24:21.923959+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:45:31.528581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"78'8","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.573336+0000","last_change":"2024-05-03T18:22:21.739131+0000","last_active":"2024-05-03T18:24:21.573336+0000","last_peered":"2024-05-03T18:24:21.573336+0000","last_clean":"2024-05-03T18:24:21.573336+0000","last_became_active":"2024-05-03T18:22:21.737243+0000","last_became_peered":"2024-05-03T18:22:21.737243+0000","last_unstale":"2024-05-03T18:24:21.573336+0000","last_undegraded":"2024-05-03T18:24:21.573336+0000","last_fullsized":"2024-05-03T18:24:21.573336+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:46:24.011140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.569261+0000","last_change":"2024-05-03T18:22:25.602011+0000","last_active":"2024-05-03T18:24:21.569261+0000","last_peered":"2024-05-03T18:24:21.569261+0000","last_clean":"2024-05-03T18:24:21.569261+0000","last_became_active":"2024-05-03T18:22:25.601604+0000","last_became_peered":"2024-05-03T18:22:25.601604+0000","last_unstale":"2024-05-03T18:24:21.569261+0000","last_undegraded":"2024-05-03T18:24:21.569261+0000","last_fullsized":"2024-05-03T18:24:21.569261+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:25:57.743203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.568962+0000","last_change":"2024-05-03T18:22:40.757885+0000","last_active":"2024-05-03T18:24:21.568962+0000","last_peered":"2024-05-03T18:24:21.568962+0000","last_clean":"2024-05-03T18:24:21.568962+0000","last_became_active":"2024-05-03T18:22:40.757208+0000","last_became_peered":"2024-05-03T18:22:40.757208+0000","last_unstale":"2024-05-03T18:24:21.568962+0000","last_undegraded":"2024-05-03T18:24:21.568962+0000","last_fullsized":"2024-05-03T18:24:21.568962+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:37:00.792138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"78'1","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.573666+0000","last_change":"2024-05-03T18:22:21.554932+0000","last_active":"2024-05-03T18:24:21.573666+0000","last_peered":"2024-05-03T18:24:21.573666+0000","last_clean":"2024-05-03T18:24:21.573666+0000","last_became_active":"2024-05-03T18:22:21.554123+0000","last_became_peered":"2024-05-03T18:22:21.554123+0000","last_unstale":"2024-05-03T18:24:21.573666+0000","last_undegraded":"2024-05-03T18:24:21.573666+0000","last_fullsized":"2024-05-03T18:24:21.573666+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:38:59.859138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.563219+0000","last_change":"2024-05-03T18:22:25.603400+0000","last_active":"2024-05-03T18:24:21.563219+0000","last_peered":"2024-05-03T18:24:21.563219+0000","last_clean":"2024-05-03T18:24:21.563219+0000","last_became_active":"2024-05-03T18:22:25.603045+0000","last_became_peered":"2024-05-03T18:22:25.603045+0000","last_unstale":"2024-05-03T18:24:21.563219+0000","last_undegraded":"2024-05-03T18:24:21.563219+0000","last_fullsized":"2024-05-03T18:24:21.563219+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:51:33.502972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.570489+0000","last_change":"2024-05-03T18:22:40.750857+0000","last_active":"2024-05-03T18:24:21.570489+0000","last_peered":"2024-05-03T18:24:21.570489+0000","last_clean":"2024-05-03T18:24:21.570489+0000","last_became_active":"2024-05-03T18:22:40.750381+0000","last_became_peered":"2024-05-03T18:22:40.750381+0000","last_unstale":"2024-05-03T18:24:21.570489+0000","last_undegraded":"2024-05-03T18:24:21.570489+0000","last_fullsized":"2024-05-03T18:24:21.570489+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:15:28.542612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.549200+0000","last_change":"2024-05-03T18:22:21.739345+0000","last_active":"2024-05-03T18:24:21.549200+0000","last_peered":"2024-05-03T18:24:21.549200+0000","last_clean":"2024-05-03T18:24:21.549200+0000","last_became_active":"2024-05-03T18:22:21.738874+0000","last_became_peered":"2024-05-03T18:22:21.738874+0000","last_unstale":"2024-05-03T18:24:21.549200+0000","last_undegraded":"2024-05-03T18:24:21.549200+0000","last_fullsized":"2024-05-03T18:24:21.549200+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:57:59.706356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.551257+0000","last_change":"2024-05-03T18:22:25.586543+0000","last_active":"2024-05-03T18:24:21.551257+0000","last_peered":"2024-05-03T18:24:21.551257+0000","last_clean":"2024-05-03T18:24:21.551257+0000","last_became_active":"2024-05-03T18:22:25.585124+0000","last_became_peered":"2024-05-03T18:22:25.585124+0000","last_unstale":"2024-05-03T18:24:21.551257+0000","last_undegraded":"2024-05-03T18:24:21.551257+0000","last_fullsized":"2024-05-03T18:24:21.551257+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:41:32.633444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.923713+0000","last_change":"2024-05-03T18:22:40.749665+0000","last_active":"2024-05-03T18:24:21.923713+0000","last_peered":"2024-05-03T18:24:21.923713+0000","last_clean":"2024-05-03T18:24:21.923713+0000","last_became_active":"2024-05-03T18:22:40.749062+0000","last_became_peered":"2024-05-03T18:22:40.749062+0000","last_unstale":"2024-05-03T18:24:21.923713+0000","last_undegraded":"2024-05-03T18:24:21.923713+0000","last_fullsized":"2024-05-03T18:24:21.923713+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:09:17.194507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.575719+0000","last_change":"2024-05-03T18:22:21.560944+0000","last_active":"2024-05-03T18:24:21.575719+0000","last_peered":"2024-05-03T18:24:21.575719+0000","last_clean":"2024-05-03T18:24:21.575719+0000","last_became_active":"2024-05-03T18:22:21.560412+0000","last_became_peered":"2024-05-03T18:22:21.560412+0000","last_unstale":"2024-05-03T18:24:21.575719+0000","last_undegraded":"2024-05-03T18:24:21.575719+0000","last_fullsized":"2024-05-03T18:24:21.575719+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:12:04.069564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.550710+0000","last_change":"2024-05-03T18:22:25.598235+0000","last_active":"2024-05-03T18:24:21.550710+0000","last_peered":"2024-05-03T18:24:21.550710+0000","last_clean":"2024-05-03T18:24:21.550710+0000","last_became_active":"2024-05-03T18:22:25.595993+0000","last_became_peered":"2024-05-03T18:22:25.595993+0000","last_unstale":"2024-05-03T18:24:21.550710+0000","last_undegraded":"2024-05-03T18:24:21.550710+0000","last_fullsized":"2024-05-03T18:24:21.550710+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:53:16.069119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.924180+0000","last_change":"2024-05-03T18:22:40.745005+0000","last_active":"2024-05-03T18:24:21.924180+0000","last_peered":"2024-05-03T18:24:21.924180+0000","last_clean":"2024-05-03T18:24:21.924180+0000","last_became_active":"2024-05-03T18:22:40.742134+0000","last_became_peered":"2024-05-03T18:22:40.742134+0000","last_unstale":"2024-05-03T18:24:21.924180+0000","last_undegraded":"2024-05-03T18:24:21.924180+0000","last_fullsized":"2024-05-03T18:24:21.924180+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:47:12.217012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.568357+0000","last_change":"2024-05-03T18:22:25.604640+0000","last_active":"2024-05-03T18:24:21.568357+0000","last_peered":"2024-05-03T18:24:21.568357+0000","last_clean":"2024-05-03T18:24:21.568357+0000","last_became_active":"2024-05-03T18:22:25.603759+0000","last_became_peered":"2024-05-03T18:22:25.603759+0000","last_unstale":"2024-05-03T18:24:21.568357+0000","last_undegraded":"2024-05-03T18:24:21.568357+0000","last_fullsized":"2024-05-03T18:24:21.568357+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:56:55.744321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.923006+0000","last_change":"2024-05-03T18:22:21.741006+0000","last_active":"2024-05-03T18:24:21.923006+0000","last_peered":"2024-05-03T18:24:21.923006+0000","last_clean":"2024-05-03T18:24:21.923006+0000","last_became_active":"2024-05-03T18:22:21.737999+0000","last_became_peered":"2024-05-03T18:22:21.737999+0000","last_unstale":"2024-05-03T18:24:21.923006+0000","last_undegraded":"2024-05-03T18:24:21.923006+0000","last_fullsized":"2024-05-03T18:24:21.923006+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:29:54.931940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.578080+0000","last_change":"2024-05-03T18:22:40.761044+0000","last_active":"2024-05-03T18:24:21.578080+0000","last_peered":"2024-05-03T18:24:21.578080+0000","last_clean":"2024-05-03T18:24:21.578080+0000","last_became_active":"2024-05-03T18:22:40.760245+0000","last_became_peered":"2024-05-03T18:22:40.760245+0000","last_unstale":"2024-05-03T18:24:21.578080+0000","last_undegraded":"2024-05-03T18:24:21.578080+0000","last_fullsized":"2024-05-03T18:24:21.578080+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:31:12.729078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.629236+0000","last_change":"2024-05-03T18:22:25.595759+0000","last_active":"2024-05-03T18:24:21.629236+0000","last_peered":"2024-05-03T18:24:21.629236+0000","last_clean":"2024-05-03T18:24:21.629236+0000","last_became_active":"2024-05-03T18:22:25.594368+0000","last_became_peered":"2024-05-03T18:22:25.594368+0000","last_unstale":"2024-05-03T18:24:21.629236+0000","last_undegraded":"2024-05-03T18:24:21.629236+0000","last_fullsized":"2024-05-03T18:24:21.629236+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:26:19.196877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.924468+0000","last_change":"2024-05-03T18:22:21.557123+0000","last_active":"2024-05-03T18:24:21.924468+0000","last_peered":"2024-05-03T18:24:21.924468+0000","last_clean":"2024-05-03T18:24:21.924468+0000","last_became_active":"2024-05-03T18:22:21.556432+0000","last_became_peered":"2024-05-03T18:22:21.556432+0000","last_unstale":"2024-05-03T18:24:21.924468+0000","last_undegraded":"2024-05-03T18:24:21.924468+0000","last_fullsized":"2024-05-03T18:24:21.924468+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:50:03.179320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":120,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.576449+0000","last_change":"2024-05-03T18:20:24.937044+0000","last_active":"2024-05-03T18:24:21.576449+0000","last_peered":"2024-05-03T18:24:21.576449+0000","last_clean":"2024-05-03T18:24:21.576449+0000","last_became_active":"2024-05-03T18:20:24.922095+0000","last_became_peered":"2024-05-03T18:20:24.922095+0000","last_unstale":"2024-05-03T18:24:21.576449+0000","last_undegraded":"2024-05-03T18:24:21.576449+0000","last_fullsized":"2024-05-03T18:24:21.576449+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:15:37.353911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:15:37.353911+0000","last_clean_scrub_stamp":"2024-05-03T18:15:37.353911+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:40:07.341441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.571355+0000","last_change":"2024-05-03T18:23:20.575064+0000","last_active":"2024-05-03T18:24:21.571355+0000","last_peered":"2024-05-03T18:24:21.571355+0000","last_clean":"2024-05-03T18:24:21.571355+0000","last_became_active":"2024-05-03T18:23:20.574557+0000","last_became_peered":"2024-05-03T18:23:20.574557+0000","last_unstale":"2024-05-03T18:24:21.571355+0000","last_undegraded":"2024-05-03T18:24:21.571355+0000","last_fullsized":"2024-05-03T18:24:21.571355+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:34:08.520396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,6,10],"acting":[8,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.923210+0000","last_change":"2024-05-03T18:22:25.591269+0000","last_active":"2024-05-03T18:24:21.923210+0000","last_peered":"2024-05-03T18:24:21.923210+0000","last_clean":"2024-05-03T18:24:21.923210+0000","last_became_active":"2024-05-03T18:22:25.590409+0000","last_became_peered":"2024-05-03T18:22:25.590409+0000","last_unstale":"2024-05-03T18:24:21.923210+0000","last_undegraded":"2024-05-03T18:24:21.923210+0000","last_fullsized":"2024-05-03T18:24:21.923210+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:03:57.432331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.576656+0000","last_change":"2024-05-03T18:22:21.560137+0000","last_active":"2024-05-03T18:24:21.576656+0000","last_peered":"2024-05-03T18:24:21.576656+0000","last_clean":"2024-05-03T18:24:21.576656+0000","last_became_active":"2024-05-03T18:22:21.559672+0000","last_became_peered":"2024-05-03T18:22:21.559672+0000","last_unstale":"2024-05-03T18:24:21.576656+0000","last_undegraded":"2024-05-03T18:24:21.576656+0000","last_fullsized":"2024-05-03T18:24:21.576656+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:20:02.070390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.561971+0000","last_change":"2024-05-03T18:22:40.753160+0000","last_active":"2024-05-03T18:24:21.561971+0000","last_peered":"2024-05-03T18:24:21.561971+0000","last_clean":"2024-05-03T18:24:21.561971+0000","last_became_active":"2024-05-03T18:22:40.750800+0000","last_became_peered":"2024-05-03T18:22:40.750800+0000","last_unstale":"2024-05-03T18:24:21.561971+0000","last_undegraded":"2024-05-03T18:24:21.561971+0000","last_fullsized":"2024-05-03T18:24:21.561971+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:27:48.756140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.573735+0000","last_change":"2024-05-03T18:22:21.739490+0000","last_active":"2024-05-03T18:24:21.573735+0000","last_peered":"2024-05-03T18:24:21.573735+0000","last_clean":"2024-05-03T18:24:21.573735+0000","last_became_active":"2024-05-03T18:22:21.738512+0000","last_became_peered":"2024-05-03T18:22:21.738512+0000","last_unstale":"2024-05-03T18:24:21.573735+0000","last_undegraded":"2024-05-03T18:24:21.573735+0000","last_fullsized":"2024-05-03T18:24:21.573735+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:26:45.323519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.923643+0000","last_change":"2024-05-03T18:22:25.591509+0000","last_active":"2024-05-03T18:24:21.923643+0000","last_peered":"2024-05-03T18:24:21.923643+0000","last_clean":"2024-05-03T18:24:21.923643+0000","last_became_active":"2024-05-03T18:22:25.590188+0000","last_became_peered":"2024-05-03T18:22:25.590188+0000","last_unstale":"2024-05-03T18:24:21.923643+0000","last_undegraded":"2024-05-03T18:24:21.923643+0000","last_fullsized":"2024-05-03T18:24:21.923643+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:33:23.919101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.627615+0000","last_change":"2024-05-03T18:22:40.747176+0000","last_active":"2024-05-03T18:24:21.627615+0000","last_peered":"2024-05-03T18:24:21.627615+0000","last_clean":"2024-05-03T18:24:21.627615+0000","last_became_active":"2024-05-03T18:22:40.746526+0000","last_became_peered":"2024-05-03T18:22:40.746526+0000","last_unstale":"2024-05-03T18:24:21.627615+0000","last_undegraded":"2024-05-03T18:24:21.627615+0000","last_fullsized":"2024-05-03T18:24:21.627615+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:59:25.838382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'8","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:33.012948+0000","last_change":"2024-05-03T18:22:21.555978+0000","last_active":"2024-05-03T18:24:33.012948+0000","last_peered":"2024-05-03T18:24:33.012948+0000","last_clean":"2024-05-03T18:24:33.012948+0000","last_became_active":"2024-05-03T18:22:21.555395+0000","last_became_peered":"2024-05-03T18:22:21.555395+0000","last_unstale":"2024-05-03T18:24:33.012948+0000","last_undegraded":"2024-05-03T18:24:33.012948+0000","last_fullsized":"2024-05-03T18:24:33.012948+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:05:44.795978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.628663+0000","last_change":"2024-05-03T18:22:25.589071+0000","last_active":"2024-05-03T18:24:21.628663+0000","last_peered":"2024-05-03T18:24:21.628663+0000","last_clean":"2024-05-03T18:24:21.628663+0000","last_became_active":"2024-05-03T18:22:25.588679+0000","last_became_peered":"2024-05-03T18:22:25.588679+0000","last_unstale":"2024-05-03T18:24:21.628663+0000","last_undegraded":"2024-05-03T18:24:21.628663+0000","last_fullsized":"2024-05-03T18:24:21.628663+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:20:53.983166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.551419+0000","last_change":"2024-05-03T18:22:40.755154+0000","last_active":"2024-05-03T18:24:21.551419+0000","last_peered":"2024-05-03T18:24:21.551419+0000","last_clean":"2024-05-03T18:24:21.551419+0000","last_became_active":"2024-05-03T18:22:40.754635+0000","last_became_peered":"2024-05-03T18:22:40.754635+0000","last_unstale":"2024-05-03T18:24:21.551419+0000","last_undegraded":"2024-05-03T18:24:21.551419+0000","last_fullsized":"2024-05-03T18:24:21.551419+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:25:56.787406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"78'2","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.924562+0000","last_change":"2024-05-03T18:22:21.854322+0000","last_active":"2024-05-03T18:24:21.924562+0000","last_peered":"2024-05-03T18:24:21.924562+0000","last_clean":"2024-05-03T18:24:21.924562+0000","last_became_active":"2024-05-03T18:22:21.853359+0000","last_became_peered":"2024-05-03T18:22:21.853359+0000","last_unstale":"2024-05-03T18:24:21.924562+0000","last_undegraded":"2024-05-03T18:24:21.924562+0000","last_fullsized":"2024-05-03T18:24:21.924562+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:06:57.061775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.802235+0000","last_change":"2024-05-03T18:22:25.605614+0000","last_active":"2024-05-03T18:24:21.802235+0000","last_peered":"2024-05-03T18:24:21.802235+0000","last_clean":"2024-05-03T18:24:21.802235+0000","last_became_active":"2024-05-03T18:22:25.604277+0000","last_became_peered":"2024-05-03T18:22:25.604277+0000","last_unstale":"2024-05-03T18:24:21.802235+0000","last_undegraded":"2024-05-03T18:24:21.802235+0000","last_fullsized":"2024-05-03T18:24:21.802235+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:17:09.812304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.926319+0000","last_change":"2024-05-03T18:22:40.763353+0000","last_active":"2024-05-03T18:24:21.926319+0000","last_peered":"2024-05-03T18:24:21.926319+0000","last_clean":"2024-05-03T18:24:21.926319+0000","last_became_active":"2024-05-03T18:22:40.762774+0000","last_became_peered":"2024-05-03T18:22:40.762774+0000","last_unstale":"2024-05-03T18:24:21.926319+0000","last_undegraded":"2024-05-03T18:24:21.926319+0000","last_fullsized":"2024-05-03T18:24:21.926319+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:38:37.247050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.805152+0000","last_change":"2024-05-03T18:24:21.803989+0000","last_active":"2024-05-03T18:24:21.805152+0000","last_peered":"2024-05-03T18:24:21.805152+0000","last_clean":"2024-05-03T18:24:21.805152+0000","last_became_active":"2024-05-03T18:24:21.803072+0000","last_became_peered":"2024-05-03T18:24:21.803072+0000","last_unstale":"2024-05-03T18:24:21.805152+0000","last_undegraded":"2024-05-03T18:24:21.805152+0000","last_fullsized":"2024-05-03T18:24:21.805152+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:32:30.562148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.572223+0000","last_change":"2024-05-03T18:22:25.588011+0000","last_active":"2024-05-03T18:24:21.572223+0000","last_peered":"2024-05-03T18:24:21.572223+0000","last_clean":"2024-05-03T18:24:21.572223+0000","last_became_active":"2024-05-03T18:22:25.587602+0000","last_became_peered":"2024-05-03T18:22:25.587602+0000","last_unstale":"2024-05-03T18:24:21.572223+0000","last_undegraded":"2024-05-03T18:24:21.572223+0000","last_fullsized":"2024-05-03T18:24:21.572223+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:18:13.186088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.550104+0000","last_change":"2024-05-03T18:22:40.755731+0000","last_active":"2024-05-03T18:24:21.550104+0000","last_peered":"2024-05-03T18:24:21.550104+0000","last_clean":"2024-05-03T18:24:21.550104+0000","last_became_active":"2024-05-03T18:22:40.755173+0000","last_became_peered":"2024-05-03T18:22:40.755173+0000","last_unstale":"2024-05-03T18:24:21.550104+0000","last_undegraded":"2024-05-03T18:24:21.550104+0000","last_fullsized":"2024-05-03T18:24:21.550104+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:36:36.548095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.627203+0000","last_change":"2024-05-03T18:22:21.561481+0000","last_active":"2024-05-03T18:24:21.627203+0000","last_peered":"2024-05-03T18:24:21.627203+0000","last_clean":"2024-05-03T18:24:21.627203+0000","last_became_active":"2024-05-03T18:22:21.560088+0000","last_became_peered":"2024-05-03T18:22:21.560088+0000","last_unstale":"2024-05-03T18:24:21.627203+0000","last_undegraded":"2024-05-03T18:24:21.627203+0000","last_fullsized":"2024-05-03T18:24:21.627203+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:45:40.184813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.562222+0000","last_change":"2024-05-03T18:22:25.584556+0000","last_active":"2024-05-03T18:24:21.562222+0000","last_peered":"2024-05-03T18:24:21.562222+0000","last_clean":"2024-05-03T18:24:21.562222+0000","last_became_active":"2024-05-03T18:22:25.583873+0000","last_became_peered":"2024-05-03T18:22:25.583873+0000","last_unstale":"2024-05-03T18:24:21.562222+0000","last_undegraded":"2024-05-03T18:24:21.562222+0000","last_fullsized":"2024-05-03T18:24:21.562222+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:28:39.028662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.926099+0000","last_change":"2024-05-03T18:22:40.763479+0000","last_active":"2024-05-03T18:24:21.926099+0000","last_peered":"2024-05-03T18:24:21.926099+0000","last_clean":"2024-05-03T18:24:21.926099+0000","last_became_active":"2024-05-03T18:22:40.762897+0000","last_became_peered":"2024-05-03T18:22:40.762897+0000","last_unstale":"2024-05-03T18:24:21.926099+0000","last_undegraded":"2024-05-03T18:24:21.926099+0000","last_fullsized":"2024-05-03T18:24:21.926099+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:00:21.090730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.552033+0000","last_change":"2024-05-03T18:22:21.562929+0000","last_active":"2024-05-03T18:24:21.552033+0000","last_peered":"2024-05-03T18:24:21.552033+0000","last_clean":"2024-05-03T18:24:21.552033+0000","last_became_active":"2024-05-03T18:22:21.560712+0000","last_became_peered":"2024-05-03T18:22:21.560712+0000","last_unstale":"2024-05-03T18:24:21.552033+0000","last_undegraded":"2024-05-03T18:24:21.552033+0000","last_fullsized":"2024-05-03T18:24:21.552033+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:31:34.722437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.628433+0000","last_change":"2024-05-03T18:22:25.581971+0000","last_active":"2024-05-03T18:24:21.628433+0000","last_peered":"2024-05-03T18:24:21.628433+0000","last_clean":"2024-05-03T18:24:21.628433+0000","last_became_active":"2024-05-03T18:22:25.581565+0000","last_became_peered":"2024-05-03T18:22:25.581565+0000","last_unstale":"2024-05-03T18:24:21.628433+0000","last_undegraded":"2024-05-03T18:24:21.628433+0000","last_fullsized":"2024-05-03T18:24:21.628433+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:10:20.607999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.926088+0000","last_change":"2024-05-03T18:22:40.761458+0000","last_active":"2024-05-03T18:24:21.926088+0000","last_peered":"2024-05-03T18:24:21.926088+0000","last_clean":"2024-05-03T18:24:21.926088+0000","last_became_active":"2024-05-03T18:22:40.760969+0000","last_became_peered":"2024-05-03T18:22:40.760969+0000","last_unstale":"2024-05-03T18:24:21.926088+0000","last_undegraded":"2024-05-03T18:24:21.926088+0000","last_fullsized":"2024-05-03T18:24:21.926088+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:19:16.804720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"78'2","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.572763+0000","last_change":"2024-05-03T18:22:21.565869+0000","last_active":"2024-05-03T18:24:21.572763+0000","last_peered":"2024-05-03T18:24:21.572763+0000","last_clean":"2024-05-03T18:24:21.572763+0000","last_became_active":"2024-05-03T18:22:21.564119+0000","last_became_peered":"2024-05-03T18:22:21.564119+0000","last_unstale":"2024-05-03T18:24:21.572763+0000","last_undegraded":"2024-05-03T18:24:21.572763+0000","last_fullsized":"2024-05-03T18:24:21.572763+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:50:29.894070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.577348+0000","last_change":"2024-05-03T18:22:25.596199+0000","last_active":"2024-05-03T18:24:21.577348+0000","last_peered":"2024-05-03T18:24:21.577348+0000","last_clean":"2024-05-03T18:24:21.577348+0000","last_became_active":"2024-05-03T18:22:25.595167+0000","last_became_peered":"2024-05-03T18:22:25.595167+0000","last_unstale":"2024-05-03T18:24:21.577348+0000","last_undegraded":"2024-05-03T18:24:21.577348+0000","last_fullsized":"2024-05-03T18:24:21.577348+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:28:56.599064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.556733+0000","last_change":"2024-05-03T18:22:40.753020+0000","last_active":"2024-05-03T18:24:21.556733+0000","last_peered":"2024-05-03T18:24:21.556733+0000","last_clean":"2024-05-03T18:24:21.556733+0000","last_became_active":"2024-05-03T18:22:40.752502+0000","last_became_peered":"2024-05-03T18:22:40.752502+0000","last_unstale":"2024-05-03T18:24:21.556733+0000","last_undegraded":"2024-05-03T18:24:21.556733+0000","last_fullsized":"2024-05-03T18:24:21.556733+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:33:51.277879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.553056+0000","last_change":"2024-05-03T18:22:21.854481+0000","last_active":"2024-05-03T18:24:21.553056+0000","last_peered":"2024-05-03T18:24:21.553056+0000","last_clean":"2024-05-03T18:24:21.553056+0000","last_became_active":"2024-05-03T18:22:21.853172+0000","last_became_peered":"2024-05-03T18:22:21.853172+0000","last_unstale":"2024-05-03T18:24:21.553056+0000","last_undegraded":"2024-05-03T18:24:21.553056+0000","last_fullsized":"2024-05-03T18:24:21.553056+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:11:47.820105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.578365+0000","last_change":"2024-05-03T18:22:25.600856+0000","last_active":"2024-05-03T18:24:21.578365+0000","last_peered":"2024-05-03T18:24:21.578365+0000","last_clean":"2024-05-03T18:24:21.578365+0000","last_became_active":"2024-05-03T18:22:25.595351+0000","last_became_peered":"2024-05-03T18:22:25.595351+0000","last_unstale":"2024-05-03T18:24:21.578365+0000","last_undegraded":"2024-05-03T18:24:21.578365+0000","last_fullsized":"2024-05-03T18:24:21.578365+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:29:24.232506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.801516+0000","last_change":"2024-05-03T18:23:20.578331+0000","last_active":"2024-05-03T18:24:21.801516+0000","last_peered":"2024-05-03T18:24:21.801516+0000","last_clean":"2024-05-03T18:24:21.801516+0000","last_became_active":"2024-05-03T18:23:20.576974+0000","last_became_peered":"2024-05-03T18:23:20.576974+0000","last_unstale":"2024-05-03T18:24:21.801516+0000","last_undegraded":"2024-05-03T18:24:21.801516+0000","last_fullsized":"2024-05-03T18:24:21.801516+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:57:32.922175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.567562+0000","last_change":"2024-05-03T18:22:21.552331+0000","last_active":"2024-05-03T18:24:21.567562+0000","last_peered":"2024-05-03T18:24:21.567562+0000","last_clean":"2024-05-03T18:24:21.567562+0000","last_became_active":"2024-05-03T18:22:21.549786+0000","last_became_peered":"2024-05-03T18:22:21.549786+0000","last_unstale":"2024-05-03T18:24:21.567562+0000","last_undegraded":"2024-05-03T18:24:21.567562+0000","last_fullsized":"2024-05-03T18:24:21.567562+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:08:18.513405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.577829+0000","last_change":"2024-05-03T18:22:25.596437+0000","last_active":"2024-05-03T18:24:21.577829+0000","last_peered":"2024-05-03T18:24:21.577829+0000","last_clean":"2024-05-03T18:24:21.577829+0000","last_became_active":"2024-05-03T18:22:25.594040+0000","last_became_peered":"2024-05-03T18:22:25.594040+0000","last_unstale":"2024-05-03T18:24:21.577829+0000","last_undegraded":"2024-05-03T18:24:21.577829+0000","last_fullsized":"2024-05-03T18:24:21.577829+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:09:36.473565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.567569+0000","last_change":"2024-05-03T18:22:40.754641+0000","last_active":"2024-05-03T18:24:21.567569+0000","last_peered":"2024-05-03T18:24:21.567569+0000","last_clean":"2024-05-03T18:24:21.567569+0000","last_became_active":"2024-05-03T18:22:40.754166+0000","last_became_peered":"2024-05-03T18:22:40.754166+0000","last_unstale":"2024-05-03T18:24:21.567569+0000","last_undegraded":"2024-05-03T18:24:21.567569+0000","last_fullsized":"2024-05-03T18:24:21.567569+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:29:50.021671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.923035+0000","last_change":"2024-05-03T18:22:21.559354+0000","last_active":"2024-05-03T18:24:21.923035+0000","last_peered":"2024-05-03T18:24:21.923035+0000","last_clean":"2024-05-03T18:24:21.923035+0000","last_became_active":"2024-05-03T18:22:21.558680+0000","last_became_peered":"2024-05-03T18:22:21.558680+0000","last_unstale":"2024-05-03T18:24:21.923035+0000","last_undegraded":"2024-05-03T18:24:21.923035+0000","last_fullsized":"2024-05-03T18:24:21.923035+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:13:04.001847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.578505+0000","last_change":"2024-05-03T18:22:25.590746+0000","last_active":"2024-05-03T18:24:21.578505+0000","last_peered":"2024-05-03T18:24:21.578505+0000","last_clean":"2024-05-03T18:24:21.578505+0000","last_became_active":"2024-05-03T18:22:25.587906+0000","last_became_peered":"2024-05-03T18:22:25.587906+0000","last_unstale":"2024-05-03T18:24:21.578505+0000","last_undegraded":"2024-05-03T18:24:21.578505+0000","last_fullsized":"2024-05-03T18:24:21.578505+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:19:43.410259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.562510+0000","last_change":"2024-05-03T18:22:25.597628+0000","last_active":"2024-05-03T18:24:21.562510+0000","last_peered":"2024-05-03T18:24:21.562510+0000","last_clean":"2024-05-03T18:24:21.562510+0000","last_became_active":"2024-05-03T18:22:25.596535+0000","last_became_peered":"2024-05-03T18:22:25.596535+0000","last_unstale":"2024-05-03T18:24:21.562510+0000","last_undegraded":"2024-05-03T18:24:21.562510+0000","last_fullsized":"2024-05-03T18:24:21.562510+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:54:52.632258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.551984+0000","last_change":"2024-05-03T18:22:25.601474+0000","last_active":"2024-05-03T18:24:21.551984+0000","last_peered":"2024-05-03T18:24:21.551984+0000","last_clean":"2024-05-03T18:24:21.551984+0000","last_became_active":"2024-05-03T18:22:25.601097+0000","last_became_peered":"2024-05-03T18:22:25.601097+0000","last_unstale":"2024-05-03T18:24:21.551984+0000","last_undegraded":"2024-05-03T18:24:21.551984+0000","last_fullsized":"2024-05-03T18:24:21.551984+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:14:46.001500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.551669+0000","last_change":"2024-05-03T18:23:20.535409+0000","last_active":"2024-05-03T18:24:21.551669+0000","last_peered":"2024-05-03T18:24:21.551669+0000","last_clean":"2024-05-03T18:24:21.551669+0000","last_became_active":"2024-05-03T18:23:20.534883+0000","last_became_peered":"2024-05-03T18:23:20.534883+0000","last_unstale":"2024-05-03T18:24:21.551669+0000","last_undegraded":"2024-05-03T18:24:21.551669+0000","last_fullsized":"2024-05-03T18:24:21.551669+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:26:37.252898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.801651+0000","last_change":"2024-05-03T18:22:25.607115+0000","last_active":"2024-05-03T18:24:21.801651+0000","last_peered":"2024-05-03T18:24:21.801651+0000","last_clean":"2024-05-03T18:24:21.801651+0000","last_became_active":"2024-05-03T18:22:25.606764+0000","last_became_peered":"2024-05-03T18:22:25.606764+0000","last_unstale":"2024-05-03T18:24:21.801651+0000","last_undegraded":"2024-05-03T18:24:21.801651+0000","last_fullsized":"2024-05-03T18:24:21.801651+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:44:37.165032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.923791+0000","last_change":"2024-05-03T18:22:25.601886+0000","last_active":"2024-05-03T18:24:21.923791+0000","last_peered":"2024-05-03T18:24:21.923791+0000","last_clean":"2024-05-03T18:24:21.923791+0000","last_became_active":"2024-05-03T18:22:25.601494+0000","last_became_peered":"2024-05-03T18:22:25.601494+0000","last_unstale":"2024-05-03T18:24:21.923791+0000","last_undegraded":"2024-05-03T18:24:21.923791+0000","last_fullsized":"2024-05-03T18:24:21.923791+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:35:30.080124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.564453+0000","last_change":"2024-05-03T18:23:20.529091+0000","last_active":"2024-05-03T18:24:21.564453+0000","last_peered":"2024-05-03T18:24:21.564453+0000","last_clean":"2024-05-03T18:24:21.564453+0000","last_became_active":"2024-05-03T18:23:20.528288+0000","last_became_peered":"2024-05-03T18:23:20.528288+0000","last_unstale":"2024-05-03T18:24:21.564453+0000","last_undegraded":"2024-05-03T18:24:21.564453+0000","last_fullsized":"2024-05-03T18:24:21.564453+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:32:25.749461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.553306+0000","last_change":"2024-05-03T18:22:25.595367+0000","last_active":"2024-05-03T18:24:21.553306+0000","last_peered":"2024-05-03T18:24:21.553306+0000","last_clean":"2024-05-03T18:24:21.553306+0000","last_became_active":"2024-05-03T18:22:25.594987+0000","last_became_peered":"2024-05-03T18:22:25.594987+0000","last_unstale":"2024-05-03T18:24:21.553306+0000","last_undegraded":"2024-05-03T18:24:21.553306+0000","last_fullsized":"2024-05-03T18:24:21.553306+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:40:55.385595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.628956+0000","last_change":"2024-05-03T18:22:25.588320+0000","last_active":"2024-05-03T18:24:21.628956+0000","last_peered":"2024-05-03T18:24:21.628956+0000","last_clean":"2024-05-03T18:24:21.628956+0000","last_became_active":"2024-05-03T18:22:25.587861+0000","last_became_peered":"2024-05-03T18:22:25.587861+0000","last_unstale":"2024-05-03T18:24:21.628956+0000","last_undegraded":"2024-05-03T18:24:21.628956+0000","last_fullsized":"2024-05-03T18:24:21.628956+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:17:02.538798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.567899+0000","last_change":"2024-05-03T18:22:25.597874+0000","last_active":"2024-05-03T18:24:21.567899+0000","last_peered":"2024-05-03T18:24:21.567899+0000","last_clean":"2024-05-03T18:24:21.567899+0000","last_became_active":"2024-05-03T18:22:25.597448+0000","last_became_peered":"2024-05-03T18:22:25.597448+0000","last_unstale":"2024-05-03T18:24:21.567899+0000","last_undegraded":"2024-05-03T18:24:21.567899+0000","last_fullsized":"2024-05-03T18:24:21.567899+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:55:52.894491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.627958+0000","last_change":"2024-05-03T18:22:25.593509+0000","last_active":"2024-05-03T18:24:21.627958+0000","last_peered":"2024-05-03T18:24:21.627958+0000","last_clean":"2024-05-03T18:24:21.627958+0000","last_became_active":"2024-05-03T18:22:25.590736+0000","last_became_peered":"2024-05-03T18:22:25.590736+0000","last_unstale":"2024-05-03T18:24:21.627958+0000","last_undegraded":"2024-05-03T18:24:21.627958+0000","last_fullsized":"2024-05-03T18:24:21.627958+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:54:45.431776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.556367+0000","last_change":"2024-05-03T18:22:25.590022+0000","last_active":"2024-05-03T18:24:21.556367+0000","last_peered":"2024-05-03T18:24:21.556367+0000","last_clean":"2024-05-03T18:24:21.556367+0000","last_became_active":"2024-05-03T18:22:25.589432+0000","last_became_peered":"2024-05-03T18:22:25.589432+0000","last_unstale":"2024-05-03T18:24:21.556367+0000","last_undegraded":"2024-05-03T18:24:21.556367+0000","last_fullsized":"2024-05-03T18:24:21.556367+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:03:02.091228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.923561+0000","last_change":"2024-05-03T18:22:25.581452+0000","last_active":"2024-05-03T18:24:21.923561+0000","last_peered":"2024-05-03T18:24:21.923561+0000","last_clean":"2024-05-03T18:24:21.923561+0000","last_became_active":"2024-05-03T18:22:25.581098+0000","last_became_peered":"2024-05-03T18:22:25.581098+0000","last_unstale":"2024-05-03T18:24:21.923561+0000","last_undegraded":"2024-05-03T18:24:21.923561+0000","last_fullsized":"2024-05-03T18:24:21.923561+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:04:46.551234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.563535+0000","last_change":"2024-05-03T18:22:25.589105+0000","last_active":"2024-05-03T18:24:21.563535+0000","last_peered":"2024-05-03T18:24:21.563535+0000","last_clean":"2024-05-03T18:24:21.563535+0000","last_became_active":"2024-05-03T18:22:25.588697+0000","last_became_peered":"2024-05-03T18:22:25.588697+0000","last_unstale":"2024-05-03T18:24:21.563535+0000","last_undegraded":"2024-05-03T18:24:21.563535+0000","last_fullsized":"2024-05-03T18:24:21.563535+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:41:23.563296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":56913,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":248,"num_read_kb":182,"num_write":84,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":258048,"data_stored":170739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":70,"ondisk_log_size":70,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710780,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1019904,"data_stored":514335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877907011,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28672,"kb_used_data":1648,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716480,"statfs":{"total":95995035648,"available":95965675520,"internally_reserved":0,"allocated":1687552,"data_stored":1156916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070538,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1028096,"data_stored":519013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266768,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1028096,"data_stored":514939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863463002,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32768,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712384,"statfs":{"total":95995035648,"available":95961481216,"internally_reserved":0,"allocated":1622016,"data_stored":1109381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626529,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1015808,"data_stored":510239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574980,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32240,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712912,"statfs":{"total":95995035648,"available":95962021888,"internally_reserved":0,"allocated":1081344,"data_stored":566354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738507,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1019904,"data_stored":510329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411453,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32244,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712908,"statfs":{"total":95995035648,"available":95962017792,"internally_reserved":0,"allocated":1085440,"data_stored":566388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182516,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":510343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986286,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1024000,"data_stored":514357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822758,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32772,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712380,"statfs":{"total":95995035648,"available":95961477120,"internally_reserved":0,"allocated":1626112,"data_stored":1104885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":56115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":56149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":56309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-03T18:25:48.120 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-03T18:25:48.619 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:25:48.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:25:48 smithi033 ceph-mon[28330]: from='client.14970 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-03T18:25:48.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:25:48 smithi033 ceph-mon[28330]: pgmap v483: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:25:49.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:25:48 smithi099 ceph-mon[31658]: from='client.14970 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-03T18:25:49.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:25:48 smithi099 ceph-mon[31658]: pgmap v483: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:25:49.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:25:48 smithi150 ceph-mon[32492]: from='client.14970 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-03T18:25:49.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:25:48 smithi150 ceph-mon[32492]: pgmap v483: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:25:50.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:50.865+0000 7f7cdf800700 1 -- 172.21.15.33:0/2495427369 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cd80756d0 msgr2=0x7f7cd8075ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:50.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:50.865+0000 7f7cdf800700 1 --2- 172.21.15.33:0/2495427369 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cd80756d0 0x7f7cd8075ab0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc009990 tx=0x7f7ccc009ca0 comp rx=0 tx=0).stop 2024-05-03T18:25:50.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:50.865+0000 7f7cdf800700 1 -- 172.21.15.33:0/2495427369 shutdown_connections 2024-05-03T18:25:50.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:50.865+0000 7f7cdf800700 1 --2- 172.21.15.33:0/2495427369 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7cd810fea0 0x7f7cd8112290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:50.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:50.865+0000 7f7cdf800700 1 --2- 172.21.15.33:0/2495427369 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cd80756d0 0x7f7cd8075ab0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:50.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:50.865+0000 7f7cdf800700 1 --2- 172.21.15.33:0/2495427369 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7cd8075ff0 0x7f7cd810f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:50.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:50.865+0000 7f7cdf800700 1 -- 172.21.15.33:0/2495427369 >> 172.21.15.33:0/2495427369 conn(0x7f7cd807a600 msgr2=0x7f7cd807aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:25:50.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:50.866+0000 7f7cdf800700 1 -- 172.21.15.33:0/2495427369 shutdown_connections 2024-05-03T18:25:50.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:50.866+0000 7f7cdf800700 1 -- 172.21.15.33:0/2495427369 wait complete. 2024-05-03T18:25:50.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:50.867+0000 7f7cdf800700 1 Processor -- start 2024-05-03T18:25:50.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:50.867+0000 7f7cdf800700 1 -- start start 2024-05-03T18:25:50.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:50.867+0000 7f7cdf800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7cd80756d0 0x7f7cd81a1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:50.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:50.867+0000 7f7cdf800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7cd8075ff0 0x7f7cd81a1e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:50.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:50.867+0000 7f7cdf800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cd810fea0 0x7f7cd819bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:50.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:50.868+0000 7f7cdf800700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7cd81156f0 con 0x7f7cd810fea0 2024-05-03T18:25:50.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:50.868+0000 7f7cdf800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7cd8115570 con 0x7f7cd80756d0 2024-05-03T18:25:50.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:50.868+0000 7f7cdf800700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f7cd8115870 con 0x7f7cd8075ff0 2024-05-03T18:25:50.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:50.868+0000 7f7cdd800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cd810fea0 0x7f7cd819bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:50.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:50.868+0000 7f7cdd800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cd810fea0 0x7f7cd819bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57722/0 (socket says 172.21.15.33:57722) 2024-05-03T18:25:50.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:50.868+0000 7f7cdd800700 1 -- 172.21.15.33:0/1886160267 learned_addr learned my addr 172.21.15.33:0/1886160267 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:25:50.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:50.868+0000 7f7cdce00700 1 --2- 172.21.15.33:0/1886160267 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7cd80756d0 0x7f7cd81a1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:50.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:50.868+0000 7f7cd7e00700 1 --2- 172.21.15.33:0/1886160267 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7cd8075ff0 0x7f7cd81a1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:50.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:50.868+0000 7f7cdd800700 1 -- 172.21.15.33:0/1886160267 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7cd8075ff0 msgr2=0x7f7cd81a1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:50.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:50.868+0000 7f7cdd800700 1 --2- 172.21.15.33:0/1886160267 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7cd8075ff0 0x7f7cd81a1e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:50.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:50.868+0000 7f7cdd800700 1 -- 172.21.15.33:0/1886160267 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7cd80756d0 msgr2=0x7f7cd81a1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:50.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:50.868+0000 7f7cdd800700 1 --2- 172.21.15.33:0/1886160267 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7cd80756d0 0x7f7cd81a1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:50.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:50.868+0000 7f7cdd800700 1 -- 172.21.15.33:0/1886160267 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cd819c220 con 0x7f7cd810fea0 2024-05-03T18:25:50.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:50.869+0000 7f7cdd800700 1 --2- 172.21.15.33:0/1886160267 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cd810fea0 0x7f7cd819bac0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f7cd000ada0 tx=0x7f7cd000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:25:50.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:50.869+0000 7f7cd5600700 1 -- 172.21.15.33:0/1886160267 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cd0017020 con 0x7f7cd810fea0 2024-05-03T18:25:50.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:50.869+0000 7f7cd5600700 1 -- 172.21.15.33:0/1886160267 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7cd000ce10 con 0x7f7cd810fea0 2024-05-03T18:25:50.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:50.869+0000 7f7cd5600700 1 -- 172.21.15.33:0/1886160267 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cd0005440 con 0x7f7cd810fea0 2024-05-03T18:25:50.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:50.869+0000 7f7cdf800700 1 -- 172.21.15.33:0/1886160267 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cd819c510 con 0x7f7cd810fea0 2024-05-03T18:25:50.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:50.869+0000 7f7cdf800700 1 -- 172.21.15.33:0/1886160267 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cd81a87d0 con 0x7f7cd810fea0 2024-05-03T18:25:50.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:50.870+0000 7f7cdf800700 1 -- 172.21.15.33:0/1886160267 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cd806a190 con 0x7f7cd810fea0 2024-05-03T18:25:50.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:50.873+0000 7f7cd5600700 1 -- 172.21.15.33:0/1886160267 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f7cd0004ad0 con 0x7f7cd810fea0 2024-05-03T18:25:50.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:50.874+0000 7f7cd5600700 1 --2- 172.21.15.33:0/1886160267 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f7cc00b2990 0x7f7cc00b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:50.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:50.875+0000 7f7cd5600700 1 -- 172.21.15.33:0/1886160267 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f7cd00e2430 con 0x7f7cd810fea0 2024-05-03T18:25:50.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:50.875+0000 7f7cdce00700 1 --2- 172.21.15.33:0/1886160267 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f7cc00b2990 0x7f7cc00b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:50.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:50.876+0000 7f7cdce00700 1 --2- 172.21.15.33:0/1886160267 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f7cc00b2990 0x7f7cc00b4e50 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc005850 tx=0x7f7ccc01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:25:50.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:50.877+0000 7f7cd5600700 1 -- 172.21.15.33:0/1886160267 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f7cd0021020 con 0x7f7cd810fea0 2024-05-03T18:25:50.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:25:50 smithi033 ceph-mon[28330]: pgmap v484: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:25:51.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:25:50 smithi099 ceph-mon[31658]: pgmap v484: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:25:51.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:51.113+0000 7f7cdf800700 1 -- 172.21.15.33:0/1886160267 --> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7cd80638e0 con 0x7f7cc00b2990 2024-05-03T18:25:51.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:25:50 smithi150 ceph-mon[32492]: pgmap v484: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:25:51.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:51.131+0000 7f7cd5600700 1 -- 172.21.15.33:0/1886160267 <== mgr.14152 v2:172.21.15.33:6800/3642895501 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340276 (secure 0 0 0) 0x7f7cd80638e0 con 0x7f7cc00b2990 2024-05-03T18:25:51.134 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:25:51.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:51.140+0000 7f7cdf800700 1 -- 172.21.15.33:0/1886160267 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f7cc00b2990 msgr2=0x7f7cc00b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:51.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:51.140+0000 7f7cdf800700 1 --2- 172.21.15.33:0/1886160267 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f7cc00b2990 0x7f7cc00b4e50 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc005850 tx=0x7f7ccc01a040 comp rx=0 tx=0).stop 2024-05-03T18:25:51.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:51.140+0000 7f7cdf800700 1 -- 172.21.15.33:0/1886160267 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cd810fea0 msgr2=0x7f7cd819bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:51.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:51.140+0000 7f7cdf800700 1 --2- 172.21.15.33:0/1886160267 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cd810fea0 0x7f7cd819bac0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f7cd000ada0 tx=0x7f7cd000c330 comp rx=0 tx=0).stop 2024-05-03T18:25:51.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:51.140+0000 7f7cdf800700 1 -- 172.21.15.33:0/1886160267 shutdown_connections 2024-05-03T18:25:51.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:51.140+0000 7f7cdf800700 1 --2- 172.21.15.33:0/1886160267 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cd810fea0 0x7f7cd819bac0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:51.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:51.140+0000 7f7cdf800700 1 --2- 172.21.15.33:0/1886160267 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f7cc00b2990 0x7f7cc00b4e50 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:51.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:51.140+0000 7f7cdf800700 1 --2- 172.21.15.33:0/1886160267 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7cd80756d0 0x7f7cd81a1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:51.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:51.140+0000 7f7cdf800700 1 --2- 172.21.15.33:0/1886160267 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7cd8075ff0 0x7f7cd81a1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:51.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:51.140+0000 7f7cdf800700 1 -- 172.21.15.33:0/1886160267 >> 172.21.15.33:0/1886160267 conn(0x7f7cd807a600 msgr2=0x7f7cd81007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:25:51.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:51.140+0000 7f7cdf800700 1 -- 172.21.15.33:0/1886160267 shutdown_connections 2024-05-03T18:25:51.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:51.140+0000 7f7cdf800700 1 -- 172.21.15.33:0/1886160267 wait complete. 2024-05-03T18:25:51.143 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-05-03T18:25:51.998 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":484,"stamp":"2024-05-03T18:25:49.414066+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":647515,"num_objects":26,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":339,"num_read_kb":256,"num_write":209,"num_write_kb":2237,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":164,"ondisk_log_size":164,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":367332,"kb_used_data":13928,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124574492,"statfs":{"total":1151940427776,"available":1151564279808,"internally_reserved":0,"allocated":14262272,"data_stored":8097479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":361674042},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":15,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006220"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.563614+0000","last_change":"2024-05-03T18:22:25.595640+0000","last_active":"2024-05-03T18:24:21.563614+0000","last_peered":"2024-05-03T18:24:21.563614+0000","last_clean":"2024-05-03T18:24:21.563614+0000","last_became_active":"2024-05-03T18:22:25.595284+0000","last_became_peered":"2024-05-03T18:22:25.595284+0000","last_unstale":"2024-05-03T18:24:21.563614+0000","last_undegraded":"2024-05-03T18:24:21.563614+0000","last_fullsized":"2024-05-03T18:24:21.563614+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:50:08.867670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.802262+0000","last_change":"2024-05-03T18:22:25.584950+0000","last_active":"2024-05-03T18:24:21.802262+0000","last_peered":"2024-05-03T18:24:21.802262+0000","last_clean":"2024-05-03T18:24:21.802262+0000","last_became_active":"2024-05-03T18:22:25.584613+0000","last_became_peered":"2024-05-03T18:22:25.584613+0000","last_unstale":"2024-05-03T18:24:21.802262+0000","last_undegraded":"2024-05-03T18:24:21.802262+0000","last_fullsized":"2024-05-03T18:24:21.802262+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:00:24.959429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.801924+0000","last_change":"2024-05-03T18:22:25.596799+0000","last_active":"2024-05-03T18:24:21.801924+0000","last_peered":"2024-05-03T18:24:21.801924+0000","last_clean":"2024-05-03T18:24:21.801924+0000","last_became_active":"2024-05-03T18:22:25.596386+0000","last_became_peered":"2024-05-03T18:22:25.596386+0000","last_unstale":"2024-05-03T18:24:21.801924+0000","last_undegraded":"2024-05-03T18:24:21.801924+0000","last_fullsized":"2024-05-03T18:24:21.801924+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:20:07.818282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.549382+0000","last_change":"2024-05-03T18:23:20.534672+0000","last_active":"2024-05-03T18:24:21.549382+0000","last_peered":"2024-05-03T18:24:21.549382+0000","last_clean":"2024-05-03T18:24:21.549382+0000","last_became_active":"2024-05-03T18:23:20.534073+0000","last_became_peered":"2024-05-03T18:23:20.534073+0000","last_unstale":"2024-05-03T18:24:21.549382+0000","last_undegraded":"2024-05-03T18:24:21.549382+0000","last_fullsized":"2024-05-03T18:24:21.549382+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:07:40.204299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.572860+0000","last_change":"2024-05-03T18:22:25.589810+0000","last_active":"2024-05-03T18:24:21.572860+0000","last_peered":"2024-05-03T18:24:21.572860+0000","last_clean":"2024-05-03T18:24:21.572860+0000","last_became_active":"2024-05-03T18:22:25.588220+0000","last_became_peered":"2024-05-03T18:22:25.588220+0000","last_unstale":"2024-05-03T18:24:21.572860+0000","last_undegraded":"2024-05-03T18:24:21.572860+0000","last_fullsized":"2024-05-03T18:24:21.572860+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:43:20.904737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.550489+0000","last_change":"2024-05-03T18:22:25.603148+0000","last_active":"2024-05-03T18:24:21.550489+0000","last_peered":"2024-05-03T18:24:21.550489+0000","last_clean":"2024-05-03T18:24:21.550489+0000","last_became_active":"2024-05-03T18:22:25.602743+0000","last_became_peered":"2024-05-03T18:22:25.602743+0000","last_unstale":"2024-05-03T18:24:21.550489+0000","last_undegraded":"2024-05-03T18:24:21.550489+0000","last_fullsized":"2024-05-03T18:24:21.550489+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:49:27.387863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.576774+0000","last_change":"2024-05-03T18:23:20.532077+0000","last_active":"2024-05-03T18:24:21.576774+0000","last_peered":"2024-05-03T18:24:21.576774+0000","last_clean":"2024-05-03T18:24:21.576774+0000","last_became_active":"2024-05-03T18:23:20.531528+0000","last_became_peered":"2024-05-03T18:23:20.531528+0000","last_unstale":"2024-05-03T18:24:21.576774+0000","last_undegraded":"2024-05-03T18:24:21.576774+0000","last_fullsized":"2024-05-03T18:24:21.576774+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:10:37.037114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.627783+0000","last_change":"2024-05-03T18:22:25.600192+0000","last_active":"2024-05-03T18:24:21.627783+0000","last_peered":"2024-05-03T18:24:21.627783+0000","last_clean":"2024-05-03T18:24:21.627783+0000","last_became_active":"2024-05-03T18:22:25.599663+0000","last_became_peered":"2024-05-03T18:22:25.599663+0000","last_unstale":"2024-05-03T18:24:21.627783+0000","last_undegraded":"2024-05-03T18:24:21.627783+0000","last_fullsized":"2024-05-03T18:24:21.627783+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:09:51.614784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.556385+0000","last_change":"2024-05-03T18:22:25.600302+0000","last_active":"2024-05-03T18:24:21.556385+0000","last_peered":"2024-05-03T18:24:21.556385+0000","last_clean":"2024-05-03T18:24:21.556385+0000","last_became_active":"2024-05-03T18:22:25.599832+0000","last_became_peered":"2024-05-03T18:22:25.599832+0000","last_unstale":"2024-05-03T18:24:21.556385+0000","last_undegraded":"2024-05-03T18:24:21.556385+0000","last_fullsized":"2024-05-03T18:24:21.556385+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:52:12.326156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.551747+0000","last_change":"2024-05-03T18:22:25.590892+0000","last_active":"2024-05-03T18:24:21.551747+0000","last_peered":"2024-05-03T18:24:21.551747+0000","last_clean":"2024-05-03T18:24:21.551747+0000","last_became_active":"2024-05-03T18:22:25.590516+0000","last_became_peered":"2024-05-03T18:22:25.590516+0000","last_unstale":"2024-05-03T18:24:21.551747+0000","last_undegraded":"2024-05-03T18:24:21.551747+0000","last_fullsized":"2024-05-03T18:24:21.551747+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:57:57.999234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.578198+0000","last_change":"2024-05-03T18:22:25.597307+0000","last_active":"2024-05-03T18:24:21.578198+0000","last_peered":"2024-05-03T18:24:21.578198+0000","last_clean":"2024-05-03T18:24:21.578198+0000","last_became_active":"2024-05-03T18:22:25.596989+0000","last_became_peered":"2024-05-03T18:22:25.596989+0000","last_unstale":"2024-05-03T18:24:21.578198+0000","last_undegraded":"2024-05-03T18:24:21.578198+0000","last_fullsized":"2024-05-03T18:24:21.578198+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:53:09.003353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.926392+0000","last_change":"2024-05-03T18:22:25.601723+0000","last_active":"2024-05-03T18:24:21.926392+0000","last_peered":"2024-05-03T18:24:21.926392+0000","last_clean":"2024-05-03T18:24:21.926392+0000","last_became_active":"2024-05-03T18:22:25.599428+0000","last_became_peered":"2024-05-03T18:22:25.599428+0000","last_unstale":"2024-05-03T18:24:21.926392+0000","last_undegraded":"2024-05-03T18:24:21.926392+0000","last_fullsized":"2024-05-03T18:24:21.926392+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:41:18.584497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.923716+0000","last_change":"2024-05-03T18:22:25.594885+0000","last_active":"2024-05-03T18:24:21.923716+0000","last_peered":"2024-05-03T18:24:21.923716+0000","last_clean":"2024-05-03T18:24:21.923716+0000","last_became_active":"2024-05-03T18:22:25.593939+0000","last_became_peered":"2024-05-03T18:22:25.593939+0000","last_unstale":"2024-05-03T18:24:21.923716+0000","last_undegraded":"2024-05-03T18:24:21.923716+0000","last_fullsized":"2024-05-03T18:24:21.923716+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:03:30.052886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.801635+0000","last_change":"2024-05-03T18:22:25.595228+0000","last_active":"2024-05-03T18:24:21.801635+0000","last_peered":"2024-05-03T18:24:21.801635+0000","last_clean":"2024-05-03T18:24:21.801635+0000","last_became_active":"2024-05-03T18:22:25.589825+0000","last_became_peered":"2024-05-03T18:22:25.589825+0000","last_unstale":"2024-05-03T18:24:21.801635+0000","last_undegraded":"2024-05-03T18:24:21.801635+0000","last_fullsized":"2024-05-03T18:24:21.801635+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:44:05.638939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.801321+0000","last_change":"2024-05-03T18:22:25.595630+0000","last_active":"2024-05-03T18:24:21.801321+0000","last_peered":"2024-05-03T18:24:21.801321+0000","last_clean":"2024-05-03T18:24:21.801321+0000","last_became_active":"2024-05-03T18:22:25.591869+0000","last_became_peered":"2024-05-03T18:22:25.591869+0000","last_unstale":"2024-05-03T18:24:21.801321+0000","last_undegraded":"2024-05-03T18:24:21.801321+0000","last_fullsized":"2024-05-03T18:24:21.801321+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:27:43.579329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.555855+0000","last_change":"2024-05-03T18:22:25.589482+0000","last_active":"2024-05-03T18:24:21.555855+0000","last_peered":"2024-05-03T18:24:21.555855+0000","last_clean":"2024-05-03T18:24:21.555855+0000","last_became_active":"2024-05-03T18:22:25.589142+0000","last_became_peered":"2024-05-03T18:22:25.589142+0000","last_unstale":"2024-05-03T18:24:21.555855+0000","last_undegraded":"2024-05-03T18:24:21.555855+0000","last_fullsized":"2024-05-03T18:24:21.555855+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:37:55.472823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.563264+0000","last_change":"2024-05-03T18:22:25.594569+0000","last_active":"2024-05-03T18:24:21.563264+0000","last_peered":"2024-05-03T18:24:21.563264+0000","last_clean":"2024-05-03T18:24:21.563264+0000","last_became_active":"2024-05-03T18:22:25.594218+0000","last_became_peered":"2024-05-03T18:22:25.594218+0000","last_unstale":"2024-05-03T18:24:21.563264+0000","last_undegraded":"2024-05-03T18:24:21.563264+0000","last_fullsized":"2024-05-03T18:24:21.563264+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:30:42.775673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.549731+0000","last_change":"2024-05-03T18:22:25.597426+0000","last_active":"2024-05-03T18:24:21.549731+0000","last_peered":"2024-05-03T18:24:21.549731+0000","last_clean":"2024-05-03T18:24:21.549731+0000","last_became_active":"2024-05-03T18:22:25.594524+0000","last_became_peered":"2024-05-03T18:22:25.594524+0000","last_unstale":"2024-05-03T18:24:21.549731+0000","last_undegraded":"2024-05-03T18:24:21.549731+0000","last_fullsized":"2024-05-03T18:24:21.549731+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:41:36.984253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.553049+0000","last_change":"2024-05-03T18:22:25.588002+0000","last_active":"2024-05-03T18:24:21.553049+0000","last_peered":"2024-05-03T18:24:21.553049+0000","last_clean":"2024-05-03T18:24:21.553049+0000","last_became_active":"2024-05-03T18:22:25.587547+0000","last_became_peered":"2024-05-03T18:22:25.587547+0000","last_unstale":"2024-05-03T18:24:21.553049+0000","last_undegraded":"2024-05-03T18:24:21.553049+0000","last_fullsized":"2024-05-03T18:24:21.553049+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:47:14.115517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.551614+0000","last_change":"2024-05-03T18:22:40.754602+0000","last_active":"2024-05-03T18:24:21.551614+0000","last_peered":"2024-05-03T18:24:21.551614+0000","last_clean":"2024-05-03T18:24:21.551614+0000","last_became_active":"2024-05-03T18:22:40.752917+0000","last_became_peered":"2024-05-03T18:22:40.752917+0000","last_unstale":"2024-05-03T18:24:21.551614+0000","last_undegraded":"2024-05-03T18:24:21.551614+0000","last_fullsized":"2024-05-03T18:24:21.551614+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:51:44.711811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"78'4","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.550063+0000","last_change":"2024-05-03T18:22:21.558560+0000","last_active":"2024-05-03T18:24:21.550063+0000","last_peered":"2024-05-03T18:24:21.550063+0000","last_clean":"2024-05-03T18:24:21.550063+0000","last_became_active":"2024-05-03T18:22:21.558185+0000","last_became_peered":"2024-05-03T18:22:21.558185+0000","last_unstale":"2024-05-03T18:24:21.550063+0000","last_undegraded":"2024-05-03T18:24:21.550063+0000","last_fullsized":"2024-05-03T18:24:21.550063+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:18:19.666724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.628787+0000","last_change":"2024-05-03T18:22:25.603934+0000","last_active":"2024-05-03T18:24:21.628787+0000","last_peered":"2024-05-03T18:24:21.628787+0000","last_clean":"2024-05-03T18:24:21.628787+0000","last_became_active":"2024-05-03T18:22:25.603529+0000","last_became_peered":"2024-05-03T18:22:25.603529+0000","last_unstale":"2024-05-03T18:24:21.628787+0000","last_undegraded":"2024-05-03T18:24:21.628787+0000","last_fullsized":"2024-05-03T18:24:21.628787+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:08:50.288871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.924818+0000","last_change":"2024-05-03T18:23:20.576254+0000","last_active":"2024-05-03T18:24:21.924818+0000","last_peered":"2024-05-03T18:24:21.924818+0000","last_clean":"2024-05-03T18:24:21.924818+0000","last_became_active":"2024-05-03T18:23:20.575774+0000","last_became_peered":"2024-05-03T18:23:20.575774+0000","last_unstale":"2024-05-03T18:24:21.924818+0000","last_undegraded":"2024-05-03T18:24:21.924818+0000","last_fullsized":"2024-05-03T18:24:21.924818+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:22:39.090938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"78'1","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.628643+0000","last_change":"2024-05-03T18:22:21.734171+0000","last_active":"2024-05-03T18:24:21.628643+0000","last_peered":"2024-05-03T18:24:21.628643+0000","last_clean":"2024-05-03T18:24:21.628643+0000","last_became_active":"2024-05-03T18:22:21.733301+0000","last_became_peered":"2024-05-03T18:22:21.733301+0000","last_unstale":"2024-05-03T18:24:21.628643+0000","last_undegraded":"2024-05-03T18:24:21.628643+0000","last_fullsized":"2024-05-03T18:24:21.628643+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:09:48.861247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.802381+0000","last_change":"2024-05-03T18:22:25.598016+0000","last_active":"2024-05-03T18:24:21.802381+0000","last_peered":"2024-05-03T18:24:21.802381+0000","last_clean":"2024-05-03T18:24:21.802381+0000","last_became_active":"2024-05-03T18:22:25.594888+0000","last_became_peered":"2024-05-03T18:22:25.594888+0000","last_unstale":"2024-05-03T18:24:21.802381+0000","last_undegraded":"2024-05-03T18:24:21.802381+0000","last_fullsized":"2024-05-03T18:24:21.802381+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:47:46.400247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.925114+0000","last_change":"2024-05-03T18:22:40.755148+0000","last_active":"2024-05-03T18:24:21.925114+0000","last_peered":"2024-05-03T18:24:21.925114+0000","last_clean":"2024-05-03T18:24:21.925114+0000","last_became_active":"2024-05-03T18:22:40.754656+0000","last_became_peered":"2024-05-03T18:22:40.754656+0000","last_unstale":"2024-05-03T18:24:21.925114+0000","last_undegraded":"2024-05-03T18:24:21.925114+0000","last_fullsized":"2024-05-03T18:24:21.925114+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:48:23.124936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"78'2","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.573377+0000","last_change":"2024-05-03T18:22:21.568060+0000","last_active":"2024-05-03T18:24:21.573377+0000","last_peered":"2024-05-03T18:24:21.573377+0000","last_clean":"2024-05-03T18:24:21.573377+0000","last_became_active":"2024-05-03T18:22:21.567434+0000","last_became_peered":"2024-05-03T18:22:21.567434+0000","last_unstale":"2024-05-03T18:24:21.573377+0000","last_undegraded":"2024-05-03T18:24:21.573377+0000","last_fullsized":"2024-05-03T18:24:21.573377+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:06:05.802621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.556074+0000","last_change":"2024-05-03T18:22:25.597777+0000","last_active":"2024-05-03T18:24:21.556074+0000","last_peered":"2024-05-03T18:24:21.556074+0000","last_clean":"2024-05-03T18:24:21.556074+0000","last_became_active":"2024-05-03T18:22:25.597391+0000","last_became_peered":"2024-05-03T18:22:25.597391+0000","last_unstale":"2024-05-03T18:24:21.556074+0000","last_undegraded":"2024-05-03T18:24:21.556074+0000","last_fullsized":"2024-05-03T18:24:21.556074+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:02:24.708193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.925062+0000","last_change":"2024-05-03T18:22:40.752318+0000","last_active":"2024-05-03T18:24:21.925062+0000","last_peered":"2024-05-03T18:24:21.925062+0000","last_clean":"2024-05-03T18:24:21.925062+0000","last_became_active":"2024-05-03T18:22:40.751433+0000","last_became_peered":"2024-05-03T18:22:40.751433+0000","last_unstale":"2024-05-03T18:24:21.925062+0000","last_undegraded":"2024-05-03T18:24:21.925062+0000","last_fullsized":"2024-05-03T18:24:21.925062+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:21:55.746143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.583558+0000","last_change":"2024-05-03T18:24:21.582330+0000","last_active":"2024-05-03T18:24:21.583558+0000","last_peered":"2024-05-03T18:24:21.583558+0000","last_clean":"2024-05-03T18:24:21.583558+0000","last_became_active":"2024-05-03T18:24:21.572706+0000","last_became_peered":"2024-05-03T18:24:21.572706+0000","last_unstale":"2024-05-03T18:24:21.583558+0000","last_undegraded":"2024-05-03T18:24:21.583558+0000","last_fullsized":"2024-05-03T18:24:21.583558+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:55:40.233995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.572852+0000","last_change":"2024-05-03T18:23:20.575102+0000","last_active":"2024-05-03T18:24:21.572852+0000","last_peered":"2024-05-03T18:24:21.572852+0000","last_clean":"2024-05-03T18:24:21.572852+0000","last_became_active":"2024-05-03T18:23:20.574657+0000","last_became_peered":"2024-05-03T18:23:20.574657+0000","last_unstale":"2024-05-03T18:24:21.572852+0000","last_undegraded":"2024-05-03T18:24:21.572852+0000","last_fullsized":"2024-05-03T18:24:21.572852+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:08:26.280866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.552660+0000","last_change":"2024-05-03T18:22:40.761052+0000","last_active":"2024-05-03T18:24:21.552660+0000","last_peered":"2024-05-03T18:24:21.552660+0000","last_clean":"2024-05-03T18:24:21.552660+0000","last_became_active":"2024-05-03T18:22:40.760179+0000","last_became_peered":"2024-05-03T18:22:40.760179+0000","last_unstale":"2024-05-03T18:24:21.552660+0000","last_undegraded":"2024-05-03T18:24:21.552660+0000","last_fullsized":"2024-05-03T18:24:21.552660+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:44:48.610680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"78'1","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.568744+0000","last_change":"2024-05-03T18:22:21.563421+0000","last_active":"2024-05-03T18:24:21.568744+0000","last_peered":"2024-05-03T18:24:21.568744+0000","last_clean":"2024-05-03T18:24:21.568744+0000","last_became_active":"2024-05-03T18:22:21.560991+0000","last_became_peered":"2024-05-03T18:22:21.560991+0000","last_unstale":"2024-05-03T18:24:21.568744+0000","last_undegraded":"2024-05-03T18:24:21.568744+0000","last_fullsized":"2024-05-03T18:24:21.568744+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:50:32.640699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.570792+0000","last_change":"2024-05-03T18:22:25.599049+0000","last_active":"2024-05-03T18:24:21.570792+0000","last_peered":"2024-05-03T18:24:21.570792+0000","last_clean":"2024-05-03T18:24:21.570792+0000","last_became_active":"2024-05-03T18:22:25.597879+0000","last_became_peered":"2024-05-03T18:22:25.597879+0000","last_unstale":"2024-05-03T18:24:21.570792+0000","last_undegraded":"2024-05-03T18:24:21.570792+0000","last_fullsized":"2024-05-03T18:24:21.570792+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:51:08.487499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.552741+0000","last_change":"2024-05-03T18:22:40.760908+0000","last_active":"2024-05-03T18:24:21.552741+0000","last_peered":"2024-05-03T18:24:21.552741+0000","last_clean":"2024-05-03T18:24:21.552741+0000","last_became_active":"2024-05-03T18:22:40.760331+0000","last_became_peered":"2024-05-03T18:22:40.760331+0000","last_unstale":"2024-05-03T18:24:21.552741+0000","last_undegraded":"2024-05-03T18:24:21.552741+0000","last_fullsized":"2024-05-03T18:24:21.552741+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:27:08.216015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"78'1","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.550440+0000","last_change":"2024-05-03T18:22:21.565328+0000","last_active":"2024-05-03T18:24:21.550440+0000","last_peered":"2024-05-03T18:24:21.550440+0000","last_clean":"2024-05-03T18:24:21.550440+0000","last_became_active":"2024-05-03T18:22:21.564583+0000","last_became_peered":"2024-05-03T18:22:21.564583+0000","last_unstale":"2024-05-03T18:24:21.550440+0000","last_undegraded":"2024-05-03T18:24:21.550440+0000","last_fullsized":"2024-05-03T18:24:21.550440+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:58:32.370458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.569755+0000","last_change":"2024-05-03T18:22:25.607585+0000","last_active":"2024-05-03T18:24:21.569755+0000","last_peered":"2024-05-03T18:24:21.569755+0000","last_clean":"2024-05-03T18:24:21.569755+0000","last_became_active":"2024-05-03T18:22:25.607168+0000","last_became_peered":"2024-05-03T18:22:25.607168+0000","last_unstale":"2024-05-03T18:24:21.569755+0000","last_undegraded":"2024-05-03T18:24:21.569755+0000","last_fullsized":"2024-05-03T18:24:21.569755+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:04:39.750823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.562868+0000","last_change":"2024-05-03T18:22:40.754744+0000","last_active":"2024-05-03T18:24:21.562868+0000","last_peered":"2024-05-03T18:24:21.562868+0000","last_clean":"2024-05-03T18:24:21.562868+0000","last_became_active":"2024-05-03T18:22:40.754262+0000","last_became_peered":"2024-05-03T18:22:40.754262+0000","last_unstale":"2024-05-03T18:24:21.562868+0000","last_undegraded":"2024-05-03T18:24:21.562868+0000","last_fullsized":"2024-05-03T18:24:21.562868+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:11:47.979546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.923003+0000","last_change":"2024-05-03T18:22:21.550048+0000","last_active":"2024-05-03T18:24:21.923003+0000","last_peered":"2024-05-03T18:24:21.923003+0000","last_clean":"2024-05-03T18:24:21.923003+0000","last_became_active":"2024-05-03T18:22:21.549334+0000","last_became_peered":"2024-05-03T18:22:21.549334+0000","last_unstale":"2024-05-03T18:24:21.923003+0000","last_undegraded":"2024-05-03T18:24:21.923003+0000","last_fullsized":"2024-05-03T18:24:21.923003+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:52:54.401185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.627661+0000","last_change":"2024-05-03T18:22:25.583074+0000","last_active":"2024-05-03T18:24:21.627661+0000","last_peered":"2024-05-03T18:24:21.627661+0000","last_clean":"2024-05-03T18:24:21.627661+0000","last_became_active":"2024-05-03T18:22:25.582637+0000","last_became_peered":"2024-05-03T18:22:25.582637+0000","last_unstale":"2024-05-03T18:24:21.627661+0000","last_undegraded":"2024-05-03T18:24:21.627661+0000","last_fullsized":"2024-05-03T18:24:21.627661+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:20:09.021402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.802120+0000","last_change":"2024-05-03T18:22:40.748509+0000","last_active":"2024-05-03T18:24:21.802120+0000","last_peered":"2024-05-03T18:24:21.802120+0000","last_clean":"2024-05-03T18:24:21.802120+0000","last_became_active":"2024-05-03T18:22:40.748051+0000","last_became_peered":"2024-05-03T18:22:40.748051+0000","last_unstale":"2024-05-03T18:24:21.802120+0000","last_undegraded":"2024-05-03T18:24:21.802120+0000","last_fullsized":"2024-05-03T18:24:21.802120+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:08:44.389460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.567340+0000","last_change":"2024-05-03T18:22:21.562185+0000","last_active":"2024-05-03T18:24:21.567340+0000","last_peered":"2024-05-03T18:24:21.567340+0000","last_clean":"2024-05-03T18:24:21.567340+0000","last_became_active":"2024-05-03T18:22:21.560523+0000","last_became_peered":"2024-05-03T18:22:21.560523+0000","last_unstale":"2024-05-03T18:24:21.567340+0000","last_undegraded":"2024-05-03T18:24:21.567340+0000","last_fullsized":"2024-05-03T18:24:21.567340+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:05:41.788286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.548693+0000","last_change":"2024-05-03T18:22:25.583895+0000","last_active":"2024-05-03T18:24:21.548693+0000","last_peered":"2024-05-03T18:24:21.548693+0000","last_clean":"2024-05-03T18:24:21.548693+0000","last_became_active":"2024-05-03T18:22:25.583503+0000","last_became_peered":"2024-05-03T18:22:25.583503+0000","last_unstale":"2024-05-03T18:24:21.548693+0000","last_undegraded":"2024-05-03T18:24:21.548693+0000","last_fullsized":"2024-05-03T18:24:21.548693+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:45:32.404394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.628056+0000","last_change":"2024-05-03T18:22:40.752814+0000","last_active":"2024-05-03T18:24:21.628056+0000","last_peered":"2024-05-03T18:24:21.628056+0000","last_clean":"2024-05-03T18:24:21.628056+0000","last_became_active":"2024-05-03T18:22:40.752328+0000","last_became_peered":"2024-05-03T18:22:40.752328+0000","last_unstale":"2024-05-03T18:24:21.628056+0000","last_undegraded":"2024-05-03T18:24:21.628056+0000","last_fullsized":"2024-05-03T18:24:21.628056+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:16:39.488122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.551544+0000","last_change":"2024-05-03T18:22:21.566266+0000","last_active":"2024-05-03T18:24:21.551544+0000","last_peered":"2024-05-03T18:24:21.551544+0000","last_clean":"2024-05-03T18:24:21.551544+0000","last_became_active":"2024-05-03T18:22:21.565612+0000","last_became_peered":"2024-05-03T18:22:21.565612+0000","last_unstale":"2024-05-03T18:24:21.551544+0000","last_undegraded":"2024-05-03T18:24:21.551544+0000","last_fullsized":"2024-05-03T18:24:21.551544+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:22:57.216912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.557537+0000","last_change":"2024-05-03T18:22:25.596182+0000","last_active":"2024-05-03T18:24:21.557537+0000","last_peered":"2024-05-03T18:24:21.557537+0000","last_clean":"2024-05-03T18:24:21.557537+0000","last_became_active":"2024-05-03T18:22:25.595472+0000","last_became_peered":"2024-05-03T18:22:25.595472+0000","last_unstale":"2024-05-03T18:24:21.557537+0000","last_undegraded":"2024-05-03T18:24:21.557537+0000","last_fullsized":"2024-05-03T18:24:21.557537+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:48:49.583879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.548535+0000","last_change":"2024-05-03T18:22:40.756378+0000","last_active":"2024-05-03T18:24:21.548535+0000","last_peered":"2024-05-03T18:24:21.548535+0000","last_clean":"2024-05-03T18:24:21.548535+0000","last_became_active":"2024-05-03T18:22:40.755692+0000","last_became_peered":"2024-05-03T18:22:40.755692+0000","last_unstale":"2024-05-03T18:24:21.548535+0000","last_undegraded":"2024-05-03T18:24:21.548535+0000","last_fullsized":"2024-05-03T18:24:21.548535+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:22:50.269817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.576475+0000","last_change":"2024-05-03T18:22:21.561381+0000","last_active":"2024-05-03T18:24:21.576475+0000","last_peered":"2024-05-03T18:24:21.576475+0000","last_clean":"2024-05-03T18:24:21.576475+0000","last_became_active":"2024-05-03T18:22:21.559411+0000","last_became_peered":"2024-05-03T18:22:21.559411+0000","last_unstale":"2024-05-03T18:24:21.576475+0000","last_undegraded":"2024-05-03T18:24:21.576475+0000","last_fullsized":"2024-05-03T18:24:21.576475+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:04:08.283538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.802450+0000","last_change":"2024-05-03T18:23:20.551361+0000","last_active":"2024-05-03T18:24:21.802450+0000","last_peered":"2024-05-03T18:24:21.802450+0000","last_clean":"2024-05-03T18:24:21.802450+0000","last_became_active":"2024-05-03T18:23:20.550689+0000","last_became_peered":"2024-05-03T18:23:20.550689+0000","last_unstale":"2024-05-03T18:24:21.802450+0000","last_undegraded":"2024-05-03T18:24:21.802450+0000","last_fullsized":"2024-05-03T18:24:21.802450+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:36:52.767110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.925712+0000","last_change":"2024-05-03T18:22:40.758006+0000","last_active":"2024-05-03T18:24:21.925712+0000","last_peered":"2024-05-03T18:24:21.925712+0000","last_clean":"2024-05-03T18:24:21.925712+0000","last_became_active":"2024-05-03T18:22:40.757075+0000","last_became_peered":"2024-05-03T18:22:40.757075+0000","last_unstale":"2024-05-03T18:24:21.925712+0000","last_undegraded":"2024-05-03T18:24:21.925712+0000","last_fullsized":"2024-05-03T18:24:21.925712+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:06:02.390184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"78'3","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.572570+0000","last_change":"2024-05-03T18:22:21.740041+0000","last_active":"2024-05-03T18:24:21.572570+0000","last_peered":"2024-05-03T18:24:21.572570+0000","last_clean":"2024-05-03T18:24:21.572570+0000","last_became_active":"2024-05-03T18:22:21.738777+0000","last_became_peered":"2024-05-03T18:22:21.738777+0000","last_unstale":"2024-05-03T18:24:21.572570+0000","last_undegraded":"2024-05-03T18:24:21.572570+0000","last_fullsized":"2024-05-03T18:24:21.572570+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:35:30.098325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.556215+0000","last_change":"2024-05-03T18:22:25.606854+0000","last_active":"2024-05-03T18:24:21.556215+0000","last_peered":"2024-05-03T18:24:21.556215+0000","last_clean":"2024-05-03T18:24:21.556215+0000","last_became_active":"2024-05-03T18:22:25.606376+0000","last_became_peered":"2024-05-03T18:22:25.606376+0000","last_unstale":"2024-05-03T18:24:21.556215+0000","last_undegraded":"2024-05-03T18:24:21.556215+0000","last_fullsized":"2024-05-03T18:24:21.556215+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:29:59.461181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.548855+0000","last_change":"2024-05-03T18:22:40.752863+0000","last_active":"2024-05-03T18:24:21.548855+0000","last_peered":"2024-05-03T18:24:21.548855+0000","last_clean":"2024-05-03T18:24:21.548855+0000","last_became_active":"2024-05-03T18:22:40.752309+0000","last_became_peered":"2024-05-03T18:22:40.752309+0000","last_unstale":"2024-05-03T18:24:21.548855+0000","last_undegraded":"2024-05-03T18:24:21.548855+0000","last_fullsized":"2024-05-03T18:24:21.548855+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:01:06.110570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"87'30","reported_seq":327,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:25:47.152922+0000","last_change":"2024-05-03T18:22:21.566162+0000","last_active":"2024-05-03T18:25:47.152922+0000","last_peered":"2024-05-03T18:25:47.152922+0000","last_clean":"2024-05-03T18:25:47.152922+0000","last_became_active":"2024-05-03T18:22:21.565641+0000","last_became_peered":"2024-05-03T18:22:21.565641+0000","last_unstale":"2024-05-03T18:25:47.152922+0000","last_undegraded":"2024-05-03T18:25:47.152922+0000","last_fullsized":"2024-05-03T18:25:47.152922+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:23:45.283135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56115,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":252,"num_read_kb":181,"num_write":30,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.574003+0000","last_change":"2024-05-03T18:22:25.586687+0000","last_active":"2024-05-03T18:24:21.574003+0000","last_peered":"2024-05-03T18:24:21.574003+0000","last_clean":"2024-05-03T18:24:21.574003+0000","last_became_active":"2024-05-03T18:22:25.586275+0000","last_became_peered":"2024-05-03T18:22:25.586275+0000","last_unstale":"2024-05-03T18:24:21.574003+0000","last_undegraded":"2024-05-03T18:24:21.574003+0000","last_fullsized":"2024-05-03T18:24:21.574003+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:10:50.339865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.573415+0000","last_change":"2024-05-03T18:22:40.758003+0000","last_active":"2024-05-03T18:24:21.573415+0000","last_peered":"2024-05-03T18:24:21.573415+0000","last_clean":"2024-05-03T18:24:21.573415+0000","last_became_active":"2024-05-03T18:22:40.757559+0000","last_became_peered":"2024-05-03T18:22:40.757559+0000","last_unstale":"2024-05-03T18:24:21.573415+0000","last_undegraded":"2024-05-03T18:24:21.573415+0000","last_fullsized":"2024-05-03T18:24:21.573415+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:54:23.377748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"78'2","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.552571+0000","last_change":"2024-05-03T18:22:21.564546+0000","last_active":"2024-05-03T18:24:21.552571+0000","last_peered":"2024-05-03T18:24:21.552571+0000","last_clean":"2024-05-03T18:24:21.552571+0000","last_became_active":"2024-05-03T18:22:21.564105+0000","last_became_peered":"2024-05-03T18:22:21.564105+0000","last_unstale":"2024-05-03T18:24:21.552571+0000","last_undegraded":"2024-05-03T18:24:21.552571+0000","last_fullsized":"2024-05-03T18:24:21.552571+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:36:07.790643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.923406+0000","last_change":"2024-05-03T18:22:25.584059+0000","last_active":"2024-05-03T18:24:21.923406+0000","last_peered":"2024-05-03T18:24:21.923406+0000","last_clean":"2024-05-03T18:24:21.923406+0000","last_became_active":"2024-05-03T18:22:25.583742+0000","last_became_peered":"2024-05-03T18:22:25.583742+0000","last_unstale":"2024-05-03T18:24:21.923406+0000","last_undegraded":"2024-05-03T18:24:21.923406+0000","last_fullsized":"2024-05-03T18:24:21.923406+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:15:48.288422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.562883+0000","last_change":"2024-05-03T18:22:40.751362+0000","last_active":"2024-05-03T18:24:21.562883+0000","last_peered":"2024-05-03T18:24:21.562883+0000","last_clean":"2024-05-03T18:24:21.562883+0000","last_became_active":"2024-05-03T18:22:40.750365+0000","last_became_peered":"2024-05-03T18:22:40.750365+0000","last_unstale":"2024-05-03T18:24:21.562883+0000","last_undegraded":"2024-05-03T18:24:21.562883+0000","last_fullsized":"2024-05-03T18:24:21.562883+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:35:50.629332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.570011+0000","last_change":"2024-05-03T18:22:21.556123+0000","last_active":"2024-05-03T18:24:21.570011+0000","last_peered":"2024-05-03T18:24:21.570011+0000","last_clean":"2024-05-03T18:24:21.570011+0000","last_became_active":"2024-05-03T18:22:21.554918+0000","last_became_peered":"2024-05-03T18:22:21.554918+0000","last_unstale":"2024-05-03T18:24:21.570011+0000","last_undegraded":"2024-05-03T18:24:21.570011+0000","last_fullsized":"2024-05-03T18:24:21.570011+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:09:12.383601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.577809+0000","last_change":"2024-05-03T18:22:25.601849+0000","last_active":"2024-05-03T18:24:21.577809+0000","last_peered":"2024-05-03T18:24:21.577809+0000","last_clean":"2024-05-03T18:24:21.577809+0000","last_became_active":"2024-05-03T18:22:25.601434+0000","last_became_peered":"2024-05-03T18:22:25.601434+0000","last_unstale":"2024-05-03T18:24:21.577809+0000","last_undegraded":"2024-05-03T18:24:21.577809+0000","last_fullsized":"2024-05-03T18:24:21.577809+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:43:18.273389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.549827+0000","last_change":"2024-05-03T18:22:40.754448+0000","last_active":"2024-05-03T18:24:21.549827+0000","last_peered":"2024-05-03T18:24:21.549827+0000","last_clean":"2024-05-03T18:24:21.549827+0000","last_became_active":"2024-05-03T18:22:40.753122+0000","last_became_peered":"2024-05-03T18:22:40.753122+0000","last_unstale":"2024-05-03T18:24:21.549827+0000","last_undegraded":"2024-05-03T18:24:21.549827+0000","last_fullsized":"2024-05-03T18:24:21.549827+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:43:15.216222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.924400+0000","last_change":"2024-05-03T18:22:21.559292+0000","last_active":"2024-05-03T18:24:21.924400+0000","last_peered":"2024-05-03T18:24:21.924400+0000","last_clean":"2024-05-03T18:24:21.924400+0000","last_became_active":"2024-05-03T18:22:21.557657+0000","last_became_peered":"2024-05-03T18:22:21.557657+0000","last_unstale":"2024-05-03T18:24:21.924400+0000","last_undegraded":"2024-05-03T18:24:21.924400+0000","last_fullsized":"2024-05-03T18:24:21.924400+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:51:57.709387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.576970+0000","last_change":"2024-05-03T18:22:25.604340+0000","last_active":"2024-05-03T18:24:21.576970+0000","last_peered":"2024-05-03T18:24:21.576970+0000","last_clean":"2024-05-03T18:24:21.576970+0000","last_became_active":"2024-05-03T18:22:25.603651+0000","last_became_peered":"2024-05-03T18:22:25.603651+0000","last_unstale":"2024-05-03T18:24:21.576970+0000","last_undegraded":"2024-05-03T18:24:21.576970+0000","last_fullsized":"2024-05-03T18:24:21.576970+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:49:36.379556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.923959+0000","last_change":"2024-05-03T18:22:40.738065+0000","last_active":"2024-05-03T18:24:21.923959+0000","last_peered":"2024-05-03T18:24:21.923959+0000","last_clean":"2024-05-03T18:24:21.923959+0000","last_became_active":"2024-05-03T18:22:40.737546+0000","last_became_peered":"2024-05-03T18:22:40.737546+0000","last_unstale":"2024-05-03T18:24:21.923959+0000","last_undegraded":"2024-05-03T18:24:21.923959+0000","last_fullsized":"2024-05-03T18:24:21.923959+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:45:31.528581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"78'8","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.573336+0000","last_change":"2024-05-03T18:22:21.739131+0000","last_active":"2024-05-03T18:24:21.573336+0000","last_peered":"2024-05-03T18:24:21.573336+0000","last_clean":"2024-05-03T18:24:21.573336+0000","last_became_active":"2024-05-03T18:22:21.737243+0000","last_became_peered":"2024-05-03T18:22:21.737243+0000","last_unstale":"2024-05-03T18:24:21.573336+0000","last_undegraded":"2024-05-03T18:24:21.573336+0000","last_fullsized":"2024-05-03T18:24:21.573336+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:46:24.011140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.569261+0000","last_change":"2024-05-03T18:22:25.602011+0000","last_active":"2024-05-03T18:24:21.569261+0000","last_peered":"2024-05-03T18:24:21.569261+0000","last_clean":"2024-05-03T18:24:21.569261+0000","last_became_active":"2024-05-03T18:22:25.601604+0000","last_became_peered":"2024-05-03T18:22:25.601604+0000","last_unstale":"2024-05-03T18:24:21.569261+0000","last_undegraded":"2024-05-03T18:24:21.569261+0000","last_fullsized":"2024-05-03T18:24:21.569261+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:25:57.743203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.568962+0000","last_change":"2024-05-03T18:22:40.757885+0000","last_active":"2024-05-03T18:24:21.568962+0000","last_peered":"2024-05-03T18:24:21.568962+0000","last_clean":"2024-05-03T18:24:21.568962+0000","last_became_active":"2024-05-03T18:22:40.757208+0000","last_became_peered":"2024-05-03T18:22:40.757208+0000","last_unstale":"2024-05-03T18:24:21.568962+0000","last_undegraded":"2024-05-03T18:24:21.568962+0000","last_fullsized":"2024-05-03T18:24:21.568962+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:37:00.792138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"78'1","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.573666+0000","last_change":"2024-05-03T18:22:21.554932+0000","last_active":"2024-05-03T18:24:21.573666+0000","last_peered":"2024-05-03T18:24:21.573666+0000","last_clean":"2024-05-03T18:24:21.573666+0000","last_became_active":"2024-05-03T18:22:21.554123+0000","last_became_peered":"2024-05-03T18:22:21.554123+0000","last_unstale":"2024-05-03T18:24:21.573666+0000","last_undegraded":"2024-05-03T18:24:21.573666+0000","last_fullsized":"2024-05-03T18:24:21.573666+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:38:59.859138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.563219+0000","last_change":"2024-05-03T18:22:25.603400+0000","last_active":"2024-05-03T18:24:21.563219+0000","last_peered":"2024-05-03T18:24:21.563219+0000","last_clean":"2024-05-03T18:24:21.563219+0000","last_became_active":"2024-05-03T18:22:25.603045+0000","last_became_peered":"2024-05-03T18:22:25.603045+0000","last_unstale":"2024-05-03T18:24:21.563219+0000","last_undegraded":"2024-05-03T18:24:21.563219+0000","last_fullsized":"2024-05-03T18:24:21.563219+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:51:33.502972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.570489+0000","last_change":"2024-05-03T18:22:40.750857+0000","last_active":"2024-05-03T18:24:21.570489+0000","last_peered":"2024-05-03T18:24:21.570489+0000","last_clean":"2024-05-03T18:24:21.570489+0000","last_became_active":"2024-05-03T18:22:40.750381+0000","last_became_peered":"2024-05-03T18:22:40.750381+0000","last_unstale":"2024-05-03T18:24:21.570489+0000","last_undegraded":"2024-05-03T18:24:21.570489+0000","last_fullsized":"2024-05-03T18:24:21.570489+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:15:28.542612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.549200+0000","last_change":"2024-05-03T18:22:21.739345+0000","last_active":"2024-05-03T18:24:21.549200+0000","last_peered":"2024-05-03T18:24:21.549200+0000","last_clean":"2024-05-03T18:24:21.549200+0000","last_became_active":"2024-05-03T18:22:21.738874+0000","last_became_peered":"2024-05-03T18:22:21.738874+0000","last_unstale":"2024-05-03T18:24:21.549200+0000","last_undegraded":"2024-05-03T18:24:21.549200+0000","last_fullsized":"2024-05-03T18:24:21.549200+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:57:59.706356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.551257+0000","last_change":"2024-05-03T18:22:25.586543+0000","last_active":"2024-05-03T18:24:21.551257+0000","last_peered":"2024-05-03T18:24:21.551257+0000","last_clean":"2024-05-03T18:24:21.551257+0000","last_became_active":"2024-05-03T18:22:25.585124+0000","last_became_peered":"2024-05-03T18:22:25.585124+0000","last_unstale":"2024-05-03T18:24:21.551257+0000","last_undegraded":"2024-05-03T18:24:21.551257+0000","last_fullsized":"2024-05-03T18:24:21.551257+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:41:32.633444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.923713+0000","last_change":"2024-05-03T18:22:40.749665+0000","last_active":"2024-05-03T18:24:21.923713+0000","last_peered":"2024-05-03T18:24:21.923713+0000","last_clean":"2024-05-03T18:24:21.923713+0000","last_became_active":"2024-05-03T18:22:40.749062+0000","last_became_peered":"2024-05-03T18:22:40.749062+0000","last_unstale":"2024-05-03T18:24:21.923713+0000","last_undegraded":"2024-05-03T18:24:21.923713+0000","last_fullsized":"2024-05-03T18:24:21.923713+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:09:17.194507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.575719+0000","last_change":"2024-05-03T18:22:21.560944+0000","last_active":"2024-05-03T18:24:21.575719+0000","last_peered":"2024-05-03T18:24:21.575719+0000","last_clean":"2024-05-03T18:24:21.575719+0000","last_became_active":"2024-05-03T18:22:21.560412+0000","last_became_peered":"2024-05-03T18:22:21.560412+0000","last_unstale":"2024-05-03T18:24:21.575719+0000","last_undegraded":"2024-05-03T18:24:21.575719+0000","last_fullsized":"2024-05-03T18:24:21.575719+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:12:04.069564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.550710+0000","last_change":"2024-05-03T18:22:25.598235+0000","last_active":"2024-05-03T18:24:21.550710+0000","last_peered":"2024-05-03T18:24:21.550710+0000","last_clean":"2024-05-03T18:24:21.550710+0000","last_became_active":"2024-05-03T18:22:25.595993+0000","last_became_peered":"2024-05-03T18:22:25.595993+0000","last_unstale":"2024-05-03T18:24:21.550710+0000","last_undegraded":"2024-05-03T18:24:21.550710+0000","last_fullsized":"2024-05-03T18:24:21.550710+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:53:16.069119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.924180+0000","last_change":"2024-05-03T18:22:40.745005+0000","last_active":"2024-05-03T18:24:21.924180+0000","last_peered":"2024-05-03T18:24:21.924180+0000","last_clean":"2024-05-03T18:24:21.924180+0000","last_became_active":"2024-05-03T18:22:40.742134+0000","last_became_peered":"2024-05-03T18:22:40.742134+0000","last_unstale":"2024-05-03T18:24:21.924180+0000","last_undegraded":"2024-05-03T18:24:21.924180+0000","last_fullsized":"2024-05-03T18:24:21.924180+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:47:12.217012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.568357+0000","last_change":"2024-05-03T18:22:25.604640+0000","last_active":"2024-05-03T18:24:21.568357+0000","last_peered":"2024-05-03T18:24:21.568357+0000","last_clean":"2024-05-03T18:24:21.568357+0000","last_became_active":"2024-05-03T18:22:25.603759+0000","last_became_peered":"2024-05-03T18:22:25.603759+0000","last_unstale":"2024-05-03T18:24:21.568357+0000","last_undegraded":"2024-05-03T18:24:21.568357+0000","last_fullsized":"2024-05-03T18:24:21.568357+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:56:55.744321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.923006+0000","last_change":"2024-05-03T18:22:21.741006+0000","last_active":"2024-05-03T18:24:21.923006+0000","last_peered":"2024-05-03T18:24:21.923006+0000","last_clean":"2024-05-03T18:24:21.923006+0000","last_became_active":"2024-05-03T18:22:21.737999+0000","last_became_peered":"2024-05-03T18:22:21.737999+0000","last_unstale":"2024-05-03T18:24:21.923006+0000","last_undegraded":"2024-05-03T18:24:21.923006+0000","last_fullsized":"2024-05-03T18:24:21.923006+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:29:54.931940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.578080+0000","last_change":"2024-05-03T18:22:40.761044+0000","last_active":"2024-05-03T18:24:21.578080+0000","last_peered":"2024-05-03T18:24:21.578080+0000","last_clean":"2024-05-03T18:24:21.578080+0000","last_became_active":"2024-05-03T18:22:40.760245+0000","last_became_peered":"2024-05-03T18:22:40.760245+0000","last_unstale":"2024-05-03T18:24:21.578080+0000","last_undegraded":"2024-05-03T18:24:21.578080+0000","last_fullsized":"2024-05-03T18:24:21.578080+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:31:12.729078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.629236+0000","last_change":"2024-05-03T18:22:25.595759+0000","last_active":"2024-05-03T18:24:21.629236+0000","last_peered":"2024-05-03T18:24:21.629236+0000","last_clean":"2024-05-03T18:24:21.629236+0000","last_became_active":"2024-05-03T18:22:25.594368+0000","last_became_peered":"2024-05-03T18:22:25.594368+0000","last_unstale":"2024-05-03T18:24:21.629236+0000","last_undegraded":"2024-05-03T18:24:21.629236+0000","last_fullsized":"2024-05-03T18:24:21.629236+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:26:19.196877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.924468+0000","last_change":"2024-05-03T18:22:21.557123+0000","last_active":"2024-05-03T18:24:21.924468+0000","last_peered":"2024-05-03T18:24:21.924468+0000","last_clean":"2024-05-03T18:24:21.924468+0000","last_became_active":"2024-05-03T18:22:21.556432+0000","last_became_peered":"2024-05-03T18:22:21.556432+0000","last_unstale":"2024-05-03T18:24:21.924468+0000","last_undegraded":"2024-05-03T18:24:21.924468+0000","last_fullsized":"2024-05-03T18:24:21.924468+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:50:03.179320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":120,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.576449+0000","last_change":"2024-05-03T18:20:24.937044+0000","last_active":"2024-05-03T18:24:21.576449+0000","last_peered":"2024-05-03T18:24:21.576449+0000","last_clean":"2024-05-03T18:24:21.576449+0000","last_became_active":"2024-05-03T18:20:24.922095+0000","last_became_peered":"2024-05-03T18:20:24.922095+0000","last_unstale":"2024-05-03T18:24:21.576449+0000","last_undegraded":"2024-05-03T18:24:21.576449+0000","last_fullsized":"2024-05-03T18:24:21.576449+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:15:37.353911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:15:37.353911+0000","last_clean_scrub_stamp":"2024-05-03T18:15:37.353911+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:40:07.341441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.571355+0000","last_change":"2024-05-03T18:23:20.575064+0000","last_active":"2024-05-03T18:24:21.571355+0000","last_peered":"2024-05-03T18:24:21.571355+0000","last_clean":"2024-05-03T18:24:21.571355+0000","last_became_active":"2024-05-03T18:23:20.574557+0000","last_became_peered":"2024-05-03T18:23:20.574557+0000","last_unstale":"2024-05-03T18:24:21.571355+0000","last_undegraded":"2024-05-03T18:24:21.571355+0000","last_fullsized":"2024-05-03T18:24:21.571355+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:34:08.520396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,6,10],"acting":[8,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.923210+0000","last_change":"2024-05-03T18:22:25.591269+0000","last_active":"2024-05-03T18:24:21.923210+0000","last_peered":"2024-05-03T18:24:21.923210+0000","last_clean":"2024-05-03T18:24:21.923210+0000","last_became_active":"2024-05-03T18:22:25.590409+0000","last_became_peered":"2024-05-03T18:22:25.590409+0000","last_unstale":"2024-05-03T18:24:21.923210+0000","last_undegraded":"2024-05-03T18:24:21.923210+0000","last_fullsized":"2024-05-03T18:24:21.923210+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:03:57.432331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.576656+0000","last_change":"2024-05-03T18:22:21.560137+0000","last_active":"2024-05-03T18:24:21.576656+0000","last_peered":"2024-05-03T18:24:21.576656+0000","last_clean":"2024-05-03T18:24:21.576656+0000","last_became_active":"2024-05-03T18:22:21.559672+0000","last_became_peered":"2024-05-03T18:22:21.559672+0000","last_unstale":"2024-05-03T18:24:21.576656+0000","last_undegraded":"2024-05-03T18:24:21.576656+0000","last_fullsized":"2024-05-03T18:24:21.576656+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:20:02.070390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.561971+0000","last_change":"2024-05-03T18:22:40.753160+0000","last_active":"2024-05-03T18:24:21.561971+0000","last_peered":"2024-05-03T18:24:21.561971+0000","last_clean":"2024-05-03T18:24:21.561971+0000","last_became_active":"2024-05-03T18:22:40.750800+0000","last_became_peered":"2024-05-03T18:22:40.750800+0000","last_unstale":"2024-05-03T18:24:21.561971+0000","last_undegraded":"2024-05-03T18:24:21.561971+0000","last_fullsized":"2024-05-03T18:24:21.561971+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:27:48.756140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.573735+0000","last_change":"2024-05-03T18:22:21.739490+0000","last_active":"2024-05-03T18:24:21.573735+0000","last_peered":"2024-05-03T18:24:21.573735+0000","last_clean":"2024-05-03T18:24:21.573735+0000","last_became_active":"2024-05-03T18:22:21.738512+0000","last_became_peered":"2024-05-03T18:22:21.738512+0000","last_unstale":"2024-05-03T18:24:21.573735+0000","last_undegraded":"2024-05-03T18:24:21.573735+0000","last_fullsized":"2024-05-03T18:24:21.573735+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:26:45.323519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.923643+0000","last_change":"2024-05-03T18:22:25.591509+0000","last_active":"2024-05-03T18:24:21.923643+0000","last_peered":"2024-05-03T18:24:21.923643+0000","last_clean":"2024-05-03T18:24:21.923643+0000","last_became_active":"2024-05-03T18:22:25.590188+0000","last_became_peered":"2024-05-03T18:22:25.590188+0000","last_unstale":"2024-05-03T18:24:21.923643+0000","last_undegraded":"2024-05-03T18:24:21.923643+0000","last_fullsized":"2024-05-03T18:24:21.923643+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:33:23.919101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.627615+0000","last_change":"2024-05-03T18:22:40.747176+0000","last_active":"2024-05-03T18:24:21.627615+0000","last_peered":"2024-05-03T18:24:21.627615+0000","last_clean":"2024-05-03T18:24:21.627615+0000","last_became_active":"2024-05-03T18:22:40.746526+0000","last_became_peered":"2024-05-03T18:22:40.746526+0000","last_unstale":"2024-05-03T18:24:21.627615+0000","last_undegraded":"2024-05-03T18:24:21.627615+0000","last_fullsized":"2024-05-03T18:24:21.627615+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:59:25.838382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'8","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:33.012948+0000","last_change":"2024-05-03T18:22:21.555978+0000","last_active":"2024-05-03T18:24:33.012948+0000","last_peered":"2024-05-03T18:24:33.012948+0000","last_clean":"2024-05-03T18:24:33.012948+0000","last_became_active":"2024-05-03T18:22:21.555395+0000","last_became_peered":"2024-05-03T18:22:21.555395+0000","last_unstale":"2024-05-03T18:24:33.012948+0000","last_undegraded":"2024-05-03T18:24:33.012948+0000","last_fullsized":"2024-05-03T18:24:33.012948+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:05:44.795978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.628663+0000","last_change":"2024-05-03T18:22:25.589071+0000","last_active":"2024-05-03T18:24:21.628663+0000","last_peered":"2024-05-03T18:24:21.628663+0000","last_clean":"2024-05-03T18:24:21.628663+0000","last_became_active":"2024-05-03T18:22:25.588679+0000","last_became_peered":"2024-05-03T18:22:25.588679+0000","last_unstale":"2024-05-03T18:24:21.628663+0000","last_undegraded":"2024-05-03T18:24:21.628663+0000","last_fullsized":"2024-05-03T18:24:21.628663+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:20:53.983166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.551419+0000","last_change":"2024-05-03T18:22:40.755154+0000","last_active":"2024-05-03T18:24:21.551419+0000","last_peered":"2024-05-03T18:24:21.551419+0000","last_clean":"2024-05-03T18:24:21.551419+0000","last_became_active":"2024-05-03T18:22:40.754635+0000","last_became_peered":"2024-05-03T18:22:40.754635+0000","last_unstale":"2024-05-03T18:24:21.551419+0000","last_undegraded":"2024-05-03T18:24:21.551419+0000","last_fullsized":"2024-05-03T18:24:21.551419+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:25:56.787406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"78'2","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.924562+0000","last_change":"2024-05-03T18:22:21.854322+0000","last_active":"2024-05-03T18:24:21.924562+0000","last_peered":"2024-05-03T18:24:21.924562+0000","last_clean":"2024-05-03T18:24:21.924562+0000","last_became_active":"2024-05-03T18:22:21.853359+0000","last_became_peered":"2024-05-03T18:22:21.853359+0000","last_unstale":"2024-05-03T18:24:21.924562+0000","last_undegraded":"2024-05-03T18:24:21.924562+0000","last_fullsized":"2024-05-03T18:24:21.924562+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:06:57.061775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.802235+0000","last_change":"2024-05-03T18:22:25.605614+0000","last_active":"2024-05-03T18:24:21.802235+0000","last_peered":"2024-05-03T18:24:21.802235+0000","last_clean":"2024-05-03T18:24:21.802235+0000","last_became_active":"2024-05-03T18:22:25.604277+0000","last_became_peered":"2024-05-03T18:22:25.604277+0000","last_unstale":"2024-05-03T18:24:21.802235+0000","last_undegraded":"2024-05-03T18:24:21.802235+0000","last_fullsized":"2024-05-03T18:24:21.802235+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:17:09.812304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.926319+0000","last_change":"2024-05-03T18:22:40.763353+0000","last_active":"2024-05-03T18:24:21.926319+0000","last_peered":"2024-05-03T18:24:21.926319+0000","last_clean":"2024-05-03T18:24:21.926319+0000","last_became_active":"2024-05-03T18:22:40.762774+0000","last_became_peered":"2024-05-03T18:22:40.762774+0000","last_unstale":"2024-05-03T18:24:21.926319+0000","last_undegraded":"2024-05-03T18:24:21.926319+0000","last_fullsized":"2024-05-03T18:24:21.926319+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:38:37.247050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.805152+0000","last_change":"2024-05-03T18:24:21.803989+0000","last_active":"2024-05-03T18:24:21.805152+0000","last_peered":"2024-05-03T18:24:21.805152+0000","last_clean":"2024-05-03T18:24:21.805152+0000","last_became_active":"2024-05-03T18:24:21.803072+0000","last_became_peered":"2024-05-03T18:24:21.803072+0000","last_unstale":"2024-05-03T18:24:21.805152+0000","last_undegraded":"2024-05-03T18:24:21.805152+0000","last_fullsized":"2024-05-03T18:24:21.805152+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:32:30.562148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.572223+0000","last_change":"2024-05-03T18:22:25.588011+0000","last_active":"2024-05-03T18:24:21.572223+0000","last_peered":"2024-05-03T18:24:21.572223+0000","last_clean":"2024-05-03T18:24:21.572223+0000","last_became_active":"2024-05-03T18:22:25.587602+0000","last_became_peered":"2024-05-03T18:22:25.587602+0000","last_unstale":"2024-05-03T18:24:21.572223+0000","last_undegraded":"2024-05-03T18:24:21.572223+0000","last_fullsized":"2024-05-03T18:24:21.572223+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:18:13.186088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.550104+0000","last_change":"2024-05-03T18:22:40.755731+0000","last_active":"2024-05-03T18:24:21.550104+0000","last_peered":"2024-05-03T18:24:21.550104+0000","last_clean":"2024-05-03T18:24:21.550104+0000","last_became_active":"2024-05-03T18:22:40.755173+0000","last_became_peered":"2024-05-03T18:22:40.755173+0000","last_unstale":"2024-05-03T18:24:21.550104+0000","last_undegraded":"2024-05-03T18:24:21.550104+0000","last_fullsized":"2024-05-03T18:24:21.550104+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:36:36.548095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.627203+0000","last_change":"2024-05-03T18:22:21.561481+0000","last_active":"2024-05-03T18:24:21.627203+0000","last_peered":"2024-05-03T18:24:21.627203+0000","last_clean":"2024-05-03T18:24:21.627203+0000","last_became_active":"2024-05-03T18:22:21.560088+0000","last_became_peered":"2024-05-03T18:22:21.560088+0000","last_unstale":"2024-05-03T18:24:21.627203+0000","last_undegraded":"2024-05-03T18:24:21.627203+0000","last_fullsized":"2024-05-03T18:24:21.627203+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:45:40.184813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.562222+0000","last_change":"2024-05-03T18:22:25.584556+0000","last_active":"2024-05-03T18:24:21.562222+0000","last_peered":"2024-05-03T18:24:21.562222+0000","last_clean":"2024-05-03T18:24:21.562222+0000","last_became_active":"2024-05-03T18:22:25.583873+0000","last_became_peered":"2024-05-03T18:22:25.583873+0000","last_unstale":"2024-05-03T18:24:21.562222+0000","last_undegraded":"2024-05-03T18:24:21.562222+0000","last_fullsized":"2024-05-03T18:24:21.562222+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:28:39.028662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.926099+0000","last_change":"2024-05-03T18:22:40.763479+0000","last_active":"2024-05-03T18:24:21.926099+0000","last_peered":"2024-05-03T18:24:21.926099+0000","last_clean":"2024-05-03T18:24:21.926099+0000","last_became_active":"2024-05-03T18:22:40.762897+0000","last_became_peered":"2024-05-03T18:22:40.762897+0000","last_unstale":"2024-05-03T18:24:21.926099+0000","last_undegraded":"2024-05-03T18:24:21.926099+0000","last_fullsized":"2024-05-03T18:24:21.926099+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:00:21.090730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.552033+0000","last_change":"2024-05-03T18:22:21.562929+0000","last_active":"2024-05-03T18:24:21.552033+0000","last_peered":"2024-05-03T18:24:21.552033+0000","last_clean":"2024-05-03T18:24:21.552033+0000","last_became_active":"2024-05-03T18:22:21.560712+0000","last_became_peered":"2024-05-03T18:22:21.560712+0000","last_unstale":"2024-05-03T18:24:21.552033+0000","last_undegraded":"2024-05-03T18:24:21.552033+0000","last_fullsized":"2024-05-03T18:24:21.552033+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:31:34.722437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.628433+0000","last_change":"2024-05-03T18:22:25.581971+0000","last_active":"2024-05-03T18:24:21.628433+0000","last_peered":"2024-05-03T18:24:21.628433+0000","last_clean":"2024-05-03T18:24:21.628433+0000","last_became_active":"2024-05-03T18:22:25.581565+0000","last_became_peered":"2024-05-03T18:22:25.581565+0000","last_unstale":"2024-05-03T18:24:21.628433+0000","last_undegraded":"2024-05-03T18:24:21.628433+0000","last_fullsized":"2024-05-03T18:24:21.628433+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:10:20.607999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.926088+0000","last_change":"2024-05-03T18:22:40.761458+0000","last_active":"2024-05-03T18:24:21.926088+0000","last_peered":"2024-05-03T18:24:21.926088+0000","last_clean":"2024-05-03T18:24:21.926088+0000","last_became_active":"2024-05-03T18:22:40.760969+0000","last_became_peered":"2024-05-03T18:22:40.760969+0000","last_unstale":"2024-05-03T18:24:21.926088+0000","last_undegraded":"2024-05-03T18:24:21.926088+0000","last_fullsized":"2024-05-03T18:24:21.926088+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:19:16.804720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"78'2","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.572763+0000","last_change":"2024-05-03T18:22:21.565869+0000","last_active":"2024-05-03T18:24:21.572763+0000","last_peered":"2024-05-03T18:24:21.572763+0000","last_clean":"2024-05-03T18:24:21.572763+0000","last_became_active":"2024-05-03T18:22:21.564119+0000","last_became_peered":"2024-05-03T18:22:21.564119+0000","last_unstale":"2024-05-03T18:24:21.572763+0000","last_undegraded":"2024-05-03T18:24:21.572763+0000","last_fullsized":"2024-05-03T18:24:21.572763+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:50:29.894070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.577348+0000","last_change":"2024-05-03T18:22:25.596199+0000","last_active":"2024-05-03T18:24:21.577348+0000","last_peered":"2024-05-03T18:24:21.577348+0000","last_clean":"2024-05-03T18:24:21.577348+0000","last_became_active":"2024-05-03T18:22:25.595167+0000","last_became_peered":"2024-05-03T18:22:25.595167+0000","last_unstale":"2024-05-03T18:24:21.577348+0000","last_undegraded":"2024-05-03T18:24:21.577348+0000","last_fullsized":"2024-05-03T18:24:21.577348+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:28:56.599064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.556733+0000","last_change":"2024-05-03T18:22:40.753020+0000","last_active":"2024-05-03T18:24:21.556733+0000","last_peered":"2024-05-03T18:24:21.556733+0000","last_clean":"2024-05-03T18:24:21.556733+0000","last_became_active":"2024-05-03T18:22:40.752502+0000","last_became_peered":"2024-05-03T18:22:40.752502+0000","last_unstale":"2024-05-03T18:24:21.556733+0000","last_undegraded":"2024-05-03T18:24:21.556733+0000","last_fullsized":"2024-05-03T18:24:21.556733+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:33:51.277879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.553056+0000","last_change":"2024-05-03T18:22:21.854481+0000","last_active":"2024-05-03T18:24:21.553056+0000","last_peered":"2024-05-03T18:24:21.553056+0000","last_clean":"2024-05-03T18:24:21.553056+0000","last_became_active":"2024-05-03T18:22:21.853172+0000","last_became_peered":"2024-05-03T18:22:21.853172+0000","last_unstale":"2024-05-03T18:24:21.553056+0000","last_undegraded":"2024-05-03T18:24:21.553056+0000","last_fullsized":"2024-05-03T18:24:21.553056+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:11:47.820105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.578365+0000","last_change":"2024-05-03T18:22:25.600856+0000","last_active":"2024-05-03T18:24:21.578365+0000","last_peered":"2024-05-03T18:24:21.578365+0000","last_clean":"2024-05-03T18:24:21.578365+0000","last_became_active":"2024-05-03T18:22:25.595351+0000","last_became_peered":"2024-05-03T18:22:25.595351+0000","last_unstale":"2024-05-03T18:24:21.578365+0000","last_undegraded":"2024-05-03T18:24:21.578365+0000","last_fullsized":"2024-05-03T18:24:21.578365+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:29:24.232506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.801516+0000","last_change":"2024-05-03T18:23:20.578331+0000","last_active":"2024-05-03T18:24:21.801516+0000","last_peered":"2024-05-03T18:24:21.801516+0000","last_clean":"2024-05-03T18:24:21.801516+0000","last_became_active":"2024-05-03T18:23:20.576974+0000","last_became_peered":"2024-05-03T18:23:20.576974+0000","last_unstale":"2024-05-03T18:24:21.801516+0000","last_undegraded":"2024-05-03T18:24:21.801516+0000","last_fullsized":"2024-05-03T18:24:21.801516+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:57:32.922175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.567562+0000","last_change":"2024-05-03T18:22:21.552331+0000","last_active":"2024-05-03T18:24:21.567562+0000","last_peered":"2024-05-03T18:24:21.567562+0000","last_clean":"2024-05-03T18:24:21.567562+0000","last_became_active":"2024-05-03T18:22:21.549786+0000","last_became_peered":"2024-05-03T18:22:21.549786+0000","last_unstale":"2024-05-03T18:24:21.567562+0000","last_undegraded":"2024-05-03T18:24:21.567562+0000","last_fullsized":"2024-05-03T18:24:21.567562+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:08:18.513405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.577829+0000","last_change":"2024-05-03T18:22:25.596437+0000","last_active":"2024-05-03T18:24:21.577829+0000","last_peered":"2024-05-03T18:24:21.577829+0000","last_clean":"2024-05-03T18:24:21.577829+0000","last_became_active":"2024-05-03T18:22:25.594040+0000","last_became_peered":"2024-05-03T18:22:25.594040+0000","last_unstale":"2024-05-03T18:24:21.577829+0000","last_undegraded":"2024-05-03T18:24:21.577829+0000","last_fullsized":"2024-05-03T18:24:21.577829+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:09:36.473565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.567569+0000","last_change":"2024-05-03T18:22:40.754641+0000","last_active":"2024-05-03T18:24:21.567569+0000","last_peered":"2024-05-03T18:24:21.567569+0000","last_clean":"2024-05-03T18:24:21.567569+0000","last_became_active":"2024-05-03T18:22:40.754166+0000","last_became_peered":"2024-05-03T18:22:40.754166+0000","last_unstale":"2024-05-03T18:24:21.567569+0000","last_undegraded":"2024-05-03T18:24:21.567569+0000","last_fullsized":"2024-05-03T18:24:21.567569+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:29:50.021671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.923035+0000","last_change":"2024-05-03T18:22:21.559354+0000","last_active":"2024-05-03T18:24:21.923035+0000","last_peered":"2024-05-03T18:24:21.923035+0000","last_clean":"2024-05-03T18:24:21.923035+0000","last_became_active":"2024-05-03T18:22:21.558680+0000","last_became_peered":"2024-05-03T18:22:21.558680+0000","last_unstale":"2024-05-03T18:24:21.923035+0000","last_undegraded":"2024-05-03T18:24:21.923035+0000","last_fullsized":"2024-05-03T18:24:21.923035+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:13:04.001847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.578505+0000","last_change":"2024-05-03T18:22:25.590746+0000","last_active":"2024-05-03T18:24:21.578505+0000","last_peered":"2024-05-03T18:24:21.578505+0000","last_clean":"2024-05-03T18:24:21.578505+0000","last_became_active":"2024-05-03T18:22:25.587906+0000","last_became_peered":"2024-05-03T18:22:25.587906+0000","last_unstale":"2024-05-03T18:24:21.578505+0000","last_undegraded":"2024-05-03T18:24:21.578505+0000","last_fullsized":"2024-05-03T18:24:21.578505+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:19:43.410259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.562510+0000","last_change":"2024-05-03T18:22:25.597628+0000","last_active":"2024-05-03T18:24:21.562510+0000","last_peered":"2024-05-03T18:24:21.562510+0000","last_clean":"2024-05-03T18:24:21.562510+0000","last_became_active":"2024-05-03T18:22:25.596535+0000","last_became_peered":"2024-05-03T18:22:25.596535+0000","last_unstale":"2024-05-03T18:24:21.562510+0000","last_undegraded":"2024-05-03T18:24:21.562510+0000","last_fullsized":"2024-05-03T18:24:21.562510+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:54:52.632258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.551984+0000","last_change":"2024-05-03T18:22:25.601474+0000","last_active":"2024-05-03T18:24:21.551984+0000","last_peered":"2024-05-03T18:24:21.551984+0000","last_clean":"2024-05-03T18:24:21.551984+0000","last_became_active":"2024-05-03T18:22:25.601097+0000","last_became_peered":"2024-05-03T18:22:25.601097+0000","last_unstale":"2024-05-03T18:24:21.551984+0000","last_undegraded":"2024-05-03T18:24:21.551984+0000","last_fullsized":"2024-05-03T18:24:21.551984+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:14:46.001500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.551669+0000","last_change":"2024-05-03T18:23:20.535409+0000","last_active":"2024-05-03T18:24:21.551669+0000","last_peered":"2024-05-03T18:24:21.551669+0000","last_clean":"2024-05-03T18:24:21.551669+0000","last_became_active":"2024-05-03T18:23:20.534883+0000","last_became_peered":"2024-05-03T18:23:20.534883+0000","last_unstale":"2024-05-03T18:24:21.551669+0000","last_undegraded":"2024-05-03T18:24:21.551669+0000","last_fullsized":"2024-05-03T18:24:21.551669+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:26:37.252898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.801651+0000","last_change":"2024-05-03T18:22:25.607115+0000","last_active":"2024-05-03T18:24:21.801651+0000","last_peered":"2024-05-03T18:24:21.801651+0000","last_clean":"2024-05-03T18:24:21.801651+0000","last_became_active":"2024-05-03T18:22:25.606764+0000","last_became_peered":"2024-05-03T18:22:25.606764+0000","last_unstale":"2024-05-03T18:24:21.801651+0000","last_undegraded":"2024-05-03T18:24:21.801651+0000","last_fullsized":"2024-05-03T18:24:21.801651+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:44:37.165032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.923791+0000","last_change":"2024-05-03T18:22:25.601886+0000","last_active":"2024-05-03T18:24:21.923791+0000","last_peered":"2024-05-03T18:24:21.923791+0000","last_clean":"2024-05-03T18:24:21.923791+0000","last_became_active":"2024-05-03T18:22:25.601494+0000","last_became_peered":"2024-05-03T18:22:25.601494+0000","last_unstale":"2024-05-03T18:24:21.923791+0000","last_undegraded":"2024-05-03T18:24:21.923791+0000","last_fullsized":"2024-05-03T18:24:21.923791+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:35:30.080124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.564453+0000","last_change":"2024-05-03T18:23:20.529091+0000","last_active":"2024-05-03T18:24:21.564453+0000","last_peered":"2024-05-03T18:24:21.564453+0000","last_clean":"2024-05-03T18:24:21.564453+0000","last_became_active":"2024-05-03T18:23:20.528288+0000","last_became_peered":"2024-05-03T18:23:20.528288+0000","last_unstale":"2024-05-03T18:24:21.564453+0000","last_undegraded":"2024-05-03T18:24:21.564453+0000","last_fullsized":"2024-05-03T18:24:21.564453+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:32:25.749461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.553306+0000","last_change":"2024-05-03T18:22:25.595367+0000","last_active":"2024-05-03T18:24:21.553306+0000","last_peered":"2024-05-03T18:24:21.553306+0000","last_clean":"2024-05-03T18:24:21.553306+0000","last_became_active":"2024-05-03T18:22:25.594987+0000","last_became_peered":"2024-05-03T18:22:25.594987+0000","last_unstale":"2024-05-03T18:24:21.553306+0000","last_undegraded":"2024-05-03T18:24:21.553306+0000","last_fullsized":"2024-05-03T18:24:21.553306+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:40:55.385595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.628956+0000","last_change":"2024-05-03T18:22:25.588320+0000","last_active":"2024-05-03T18:24:21.628956+0000","last_peered":"2024-05-03T18:24:21.628956+0000","last_clean":"2024-05-03T18:24:21.628956+0000","last_became_active":"2024-05-03T18:22:25.587861+0000","last_became_peered":"2024-05-03T18:22:25.587861+0000","last_unstale":"2024-05-03T18:24:21.628956+0000","last_undegraded":"2024-05-03T18:24:21.628956+0000","last_fullsized":"2024-05-03T18:24:21.628956+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:17:02.538798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.567899+0000","last_change":"2024-05-03T18:22:25.597874+0000","last_active":"2024-05-03T18:24:21.567899+0000","last_peered":"2024-05-03T18:24:21.567899+0000","last_clean":"2024-05-03T18:24:21.567899+0000","last_became_active":"2024-05-03T18:22:25.597448+0000","last_became_peered":"2024-05-03T18:22:25.597448+0000","last_unstale":"2024-05-03T18:24:21.567899+0000","last_undegraded":"2024-05-03T18:24:21.567899+0000","last_fullsized":"2024-05-03T18:24:21.567899+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:55:52.894491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.627958+0000","last_change":"2024-05-03T18:22:25.593509+0000","last_active":"2024-05-03T18:24:21.627958+0000","last_peered":"2024-05-03T18:24:21.627958+0000","last_clean":"2024-05-03T18:24:21.627958+0000","last_became_active":"2024-05-03T18:22:25.590736+0000","last_became_peered":"2024-05-03T18:22:25.590736+0000","last_unstale":"2024-05-03T18:24:21.627958+0000","last_undegraded":"2024-05-03T18:24:21.627958+0000","last_fullsized":"2024-05-03T18:24:21.627958+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:54:45.431776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.556367+0000","last_change":"2024-05-03T18:22:25.590022+0000","last_active":"2024-05-03T18:24:21.556367+0000","last_peered":"2024-05-03T18:24:21.556367+0000","last_clean":"2024-05-03T18:24:21.556367+0000","last_became_active":"2024-05-03T18:22:25.589432+0000","last_became_peered":"2024-05-03T18:22:25.589432+0000","last_unstale":"2024-05-03T18:24:21.556367+0000","last_undegraded":"2024-05-03T18:24:21.556367+0000","last_fullsized":"2024-05-03T18:24:21.556367+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:03:02.091228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.923561+0000","last_change":"2024-05-03T18:22:25.581452+0000","last_active":"2024-05-03T18:24:21.923561+0000","last_peered":"2024-05-03T18:24:21.923561+0000","last_clean":"2024-05-03T18:24:21.923561+0000","last_became_active":"2024-05-03T18:22:25.581098+0000","last_became_peered":"2024-05-03T18:22:25.581098+0000","last_unstale":"2024-05-03T18:24:21.923561+0000","last_undegraded":"2024-05-03T18:24:21.923561+0000","last_fullsized":"2024-05-03T18:24:21.923561+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:04:46.551234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-03T18:24:21.563535+0000","last_change":"2024-05-03T18:22:25.589105+0000","last_active":"2024-05-03T18:24:21.563535+0000","last_peered":"2024-05-03T18:24:21.563535+0000","last_clean":"2024-05-03T18:24:21.563535+0000","last_became_active":"2024-05-03T18:22:25.588697+0000","last_became_peered":"2024-05-03T18:22:25.588697+0000","last_unstale":"2024-05-03T18:24:21.563535+0000","last_undegraded":"2024-05-03T18:24:21.563535+0000","last_fullsized":"2024-05-03T18:24:21.563535+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:41:23.563296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":56913,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":258,"num_read_kb":187,"num_write":84,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":258048,"data_stored":170739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":70,"ondisk_log_size":70,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710781,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1019904,"data_stored":514335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877907012,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28672,"kb_used_data":1648,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716480,"statfs":{"total":95995035648,"available":95965675520,"internally_reserved":0,"allocated":1687552,"data_stored":1156916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070539,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1028096,"data_stored":519013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266769,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1028096,"data_stored":514939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863463002,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32768,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712384,"statfs":{"total":95995035648,"available":95961481216,"internally_reserved":0,"allocated":1622016,"data_stored":1109381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626530,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1015808,"data_stored":510239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574981,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32240,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712912,"statfs":{"total":95995035648,"available":95962021888,"internally_reserved":0,"allocated":1081344,"data_stored":566354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738507,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1019904,"data_stored":510329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411454,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32244,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712908,"statfs":{"total":95995035648,"available":95962017792,"internally_reserved":0,"allocated":1085440,"data_stored":566388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182517,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":510343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986287,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1024000,"data_stored":514357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822759,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32772,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712380,"statfs":{"total":95995035648,"available":95961477120,"internally_reserved":0,"allocated":1626112,"data_stored":1104885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":56115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":56149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":56309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-03T18:25:52.003 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-03T18:25:52.003 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-03T18:25:52.004 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-05-03T18:25:52.004 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph health --format=json 2024-05-03T18:25:52.511 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:25:52.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:25:52 smithi033 ceph-mon[28330]: from='client.14976 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-03T18:25:52.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:25:52 smithi033 ceph-mon[28330]: pgmap v485: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:25:53.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:25:52 smithi099 ceph-mon[31658]: from='client.14976 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-03T18:25:53.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:25:52 smithi099 ceph-mon[31658]: pgmap v485: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:25:53.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:25:52 smithi150 ceph-mon[32492]: from='client.14976 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-03T18:25:53.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:25:52 smithi150 ceph-mon[32492]: pgmap v485: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:25:54.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:54.817+0000 7fbcd8800700 1 -- 172.21.15.33:0/2008049394 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcd01056c0 msgr2=0x7fbcd0105b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:54.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:54.817+0000 7fbcd8800700 1 --2- 172.21.15.33:0/2008049394 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcd01056c0 0x7fbcd0105b40 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fbcc0009990 tx=0x7fbcc0009ca0 comp rx=0 tx=0).stop 2024-05-03T18:25:54.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:54.817+0000 7fbcd8800700 1 -- 172.21.15.33:0/2008049394 shutdown_connections 2024-05-03T18:25:54.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:54.817+0000 7fbcd8800700 1 --2- 172.21.15.33:0/2008049394 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbcd0106080 0x7fbcd0112590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:54.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:54.817+0000 7fbcd8800700 1 --2- 172.21.15.33:0/2008049394 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbcd010b6c0 0x7fbcd010baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:54.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:54.817+0000 7fbcd8800700 1 --2- 172.21.15.33:0/2008049394 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcd01056c0 0x7fbcd0105b40 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:54.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:54.817+0000 7fbcd8800700 1 -- 172.21.15.33:0/2008049394 >> 172.21.15.33:0/2008049394 conn(0x7fbcd01011b0 msgr2=0x7fbcd01035d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:25:54.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:54.817+0000 7fbcd8800700 1 -- 172.21.15.33:0/2008049394 shutdown_connections 2024-05-03T18:25:54.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:54.817+0000 7fbcd8800700 1 -- 172.21.15.33:0/2008049394 wait complete. 2024-05-03T18:25:54.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:54.818+0000 7fbcd8800700 1 Processor -- start 2024-05-03T18:25:54.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:54.818+0000 7fbcd8800700 1 -- start start 2024-05-03T18:25:54.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:54.820+0000 7fbcd8800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcd01056c0 0x7fbcd01a1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:54.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:54.820+0000 7fbcd8800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbcd0106080 0x7fbcd01a1e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:54.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:54.820+0000 7fbcd8800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbcd010b6c0 0x7fbcd019bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:54.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:54.820+0000 7fbcd8800700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd0115800 con 0x7fbcd01056c0 2024-05-03T18:25:54.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:54.820+0000 7fbcd8800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd0115680 con 0x7fbcd0106080 2024-05-03T18:25:54.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:54.820+0000 7fbcd8800700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd0115980 con 0x7fbcd010b6c0 2024-05-03T18:25:54.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:54.820+0000 7fbcd5e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcd01056c0 0x7fbcd01a1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:54.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:54.820+0000 7fbcd5e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcd01056c0 0x7fbcd01a1900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57732/0 (socket says 172.21.15.33:57732) 2024-05-03T18:25:54.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:54.820+0000 7fbcd5e00700 1 -- 172.21.15.33:0/1676879800 learned_addr learned my addr 172.21.15.33:0/1676879800 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:25:54.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:54.820+0000 7fbcd5400700 1 --2- 172.21.15.33:0/1676879800 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbcd0106080 0x7fbcd01a1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:54.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:54.820+0000 7fbcd6800700 1 --2- 172.21.15.33:0/1676879800 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbcd010b6c0 0x7fbcd019bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:54.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:54.820+0000 7fbcd5e00700 1 -- 172.21.15.33:0/1676879800 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbcd010b6c0 msgr2=0x7fbcd019bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:54.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:54.820+0000 7fbcd5e00700 1 --2- 172.21.15.33:0/1676879800 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbcd010b6c0 0x7fbcd019bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:54.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:54.820+0000 7fbcd5e00700 1 -- 172.21.15.33:0/1676879800 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbcd0106080 msgr2=0x7fbcd01a1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:54.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:54.820+0000 7fbcd5e00700 1 --2- 172.21.15.33:0/1676879800 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbcd0106080 0x7fbcd01a1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:54.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:54.820+0000 7fbcd5e00700 1 -- 172.21.15.33:0/1676879800 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcd019c220 con 0x7fbcd01056c0 2024-05-03T18:25:54.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:54.820+0000 7fbcd5e00700 1 --2- 172.21.15.33:0/1676879800 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcd01056c0 0x7fbcd01a1900 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fbcc400d980 tx=0x7fbcc400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:25:54.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:54.820+0000 7fbccaa00700 1 -- 172.21.15.33:0/1676879800 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc4017070 con 0x7fbcd01056c0 2024-05-03T18:25:54.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:54.820+0000 7fbccaa00700 1 -- 172.21.15.33:0/1676879800 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbcc4004830 con 0x7fbcd01056c0 2024-05-03T18:25:54.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:54.820+0000 7fbcd8800700 1 -- 172.21.15.33:0/1676879800 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcd019c510 con 0x7fbcd01056c0 2024-05-03T18:25:54.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:54.820+0000 7fbccaa00700 1 -- 172.21.15.33:0/1676879800 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc40053a0 con 0x7fbcd01056c0 2024-05-03T18:25:54.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:54.820+0000 7fbcd8800700 1 -- 172.21.15.33:0/1676879800 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcd01a87d0 con 0x7fbcd01056c0 2024-05-03T18:25:54.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:54.823+0000 7fbcd8800700 1 -- 172.21.15.33:0/1676879800 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcd006a190 con 0x7fbcd01056c0 2024-05-03T18:25:54.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:54.825+0000 7fbccaa00700 1 -- 172.21.15.33:0/1676879800 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fbcc40049e0 con 0x7fbcd01056c0 2024-05-03T18:25:54.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:54.826+0000 7fbccaa00700 1 --2- 172.21.15.33:0/1676879800 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fbcb80b2990 0x7fbcb80b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:54.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:54.826+0000 7fbccaa00700 1 -- 172.21.15.33:0/1676879800 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7fbcc40e2640 con 0x7fbcd01056c0 2024-05-03T18:25:54.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:54.826+0000 7fbcd5400700 1 --2- 172.21.15.33:0/1676879800 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fbcb80b2990 0x7fbcb80b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:54.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:54.827+0000 7fbcd5400700 1 --2- 172.21.15.33:0/1676879800 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fbcb80b2990 0x7fbcb80b4e50 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fbcc000b5c0 tx=0x7fbcc001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:25:54.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:54.831+0000 7fbccaa00700 1 -- 172.21.15.33:0/1676879800 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fbcc4020020 con 0x7fbcd01056c0 2024-05-03T18:25:54.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:25:54 smithi033 ceph-mon[28330]: pgmap v486: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:25:55.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:25:54 smithi099 ceph-mon[31658]: pgmap v486: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:25:55.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:55.122+0000 7fbcd8800700 1 -- 172.21.15.33:0/1676879800 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fbcd002e510 con 0x7fbcd01056c0 2024-05-03T18:25:55.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:55.123+0000 7fbccaa00700 1 -- 172.21.15.33:0/1676879800 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fbcc40a9e20 con 0x7fbcd01056c0 2024-05-03T18:25:55.123 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:25:55.123 INFO:teuthology.orchestra.run.smithi033.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-03T18:25:55.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:55.126+0000 7fbcd8800700 1 -- 172.21.15.33:0/1676879800 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fbcb80b2990 msgr2=0x7fbcb80b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:55.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:55.126+0000 7fbcd8800700 1 --2- 172.21.15.33:0/1676879800 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fbcb80b2990 0x7fbcb80b4e50 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fbcc000b5c0 tx=0x7fbcc001a040 comp rx=0 tx=0).stop 2024-05-03T18:25:55.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:55.126+0000 7fbcd8800700 1 -- 172.21.15.33:0/1676879800 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcd01056c0 msgr2=0x7fbcd01a1900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:55.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:55.126+0000 7fbcd8800700 1 --2- 172.21.15.33:0/1676879800 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcd01056c0 0x7fbcd01a1900 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fbcc400d980 tx=0x7fbcc400dd40 comp rx=0 tx=0).stop 2024-05-03T18:25:55.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:55.126+0000 7fbcd8800700 1 -- 172.21.15.33:0/1676879800 shutdown_connections 2024-05-03T18:25:55.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:55.126+0000 7fbcd8800700 1 --2- 172.21.15.33:0/1676879800 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbcd010b6c0 0x7fbcd019bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:55.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:55.126+0000 7fbcd8800700 1 --2- 172.21.15.33:0/1676879800 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcd01056c0 0x7fbcd01a1900 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:55.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:55.126+0000 7fbcd8800700 1 --2- 172.21.15.33:0/1676879800 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fbcb80b2990 0x7fbcb80b4e50 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:55.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:55.126+0000 7fbcd8800700 1 --2- 172.21.15.33:0/1676879800 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbcd0106080 0x7fbcd01a1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:55.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:55.126+0000 7fbcd8800700 1 -- 172.21.15.33:0/1676879800 >> 172.21.15.33:0/1676879800 conn(0x7fbcd01011b0 msgr2=0x7fbcd0101a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:25:55.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:55.126+0000 7fbcd8800700 1 -- 172.21.15.33:0/1676879800 shutdown_connections 2024-05-03T18:25:55.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:55.126+0000 7fbcd8800700 1 -- 172.21.15.33:0/1676879800 wait complete. 2024-05-03T18:25:55.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:25:54 smithi150 ceph-mon[32492]: pgmap v486: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:25:55.848 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:25:55 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/1676879800' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-03T18:25:55.926 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-05-03T18:25:55.926 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T18:25:56.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:25:55 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/1676879800' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-03T18:25:56.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:25:55 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/1676879800' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-03T18:25:56.396 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:25:56.902 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:25:56 smithi033 ceph-mon[28330]: pgmap v487: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:25:57.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:25:56 smithi099 ceph-mon[31658]: pgmap v487: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:25:57.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:25:56 smithi150 ceph-mon[32492]: pgmap v487: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:25:58.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:58.720+0000 7f6d59000700 1 -- 172.21.15.33:0/2285066362 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d54109500 msgr2=0x7f6d541098e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:58.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:58.720+0000 7f6d59000700 1 --2- 172.21.15.33:0/2285066362 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d54109500 0x7f6d541098e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f6d44009990 tx=0x7f6d44009ca0 comp rx=0 tx=0).stop 2024-05-03T18:25:58.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:58.720+0000 7f6d59000700 1 -- 172.21.15.33:0/2285066362 shutdown_connections 2024-05-03T18:25:58.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:58.720+0000 7f6d59000700 1 --2- 172.21.15.33:0/2285066362 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d5406c3d0 0x7f6d54110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:58.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:58.720+0000 7f6d59000700 1 --2- 172.21.15.33:0/2285066362 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d54109500 0x7f6d541098e0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:58.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:58.721+0000 7f6d59000700 1 --2- 172.21.15.33:0/2285066362 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d5406ba10 0x7f6d5406be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:58.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:58.721+0000 7f6d59000700 1 -- 172.21.15.33:0/2285066362 >> 172.21.15.33:0/2285066362 conn(0x7f6d54079310 msgr2=0x7f6d54079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:25:58.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:58.721+0000 7f6d59000700 1 -- 172.21.15.33:0/2285066362 shutdown_connections 2024-05-03T18:25:58.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:58.721+0000 7f6d59000700 1 -- 172.21.15.33:0/2285066362 wait complete. 2024-05-03T18:25:58.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:58.722+0000 7f6d59000700 1 Processor -- start 2024-05-03T18:25:58.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:58.722+0000 7f6d59000700 1 -- start start 2024-05-03T18:25:58.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:58.722+0000 7f6d59000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d5406ba10 0x7f6d5410a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:58.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:58.722+0000 7f6d59000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d5406c3d0 0x7f6d5410a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:58.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:58.722+0000 7f6d59000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d54109500 0x7f6d5410adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:58.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:58.723+0000 7f6d59000700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6d54113510 con 0x7f6d54109500 2024-05-03T18:25:58.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:58.723+0000 7f6d59000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6d54113390 con 0x7f6d5406c3d0 2024-05-03T18:25:58.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:58.723+0000 7f6d59000700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6d54113690 con 0x7f6d5406ba10 2024-05-03T18:25:58.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:58.723+0000 7f6d52800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d54109500 0x7f6d5410adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:58.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:58.723+0000 7f6d52800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d54109500 0x7f6d5410adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37712/0 (socket says 172.21.15.33:37712) 2024-05-03T18:25:58.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:58.723+0000 7f6d52800700 1 -- 172.21.15.33:0/825458113 learned_addr learned my addr 172.21.15.33:0/825458113 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:25:58.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:58.723+0000 7f6d4be00700 1 --2- 172.21.15.33:0/825458113 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d5406c3d0 0x7f6d5410a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:58.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:58.723+0000 7f6d51e00700 1 --2- 172.21.15.33:0/825458113 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d5406ba10 0x7f6d5410a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:58.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:58.724+0000 7f6d52800700 1 -- 172.21.15.33:0/825458113 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d5406ba10 msgr2=0x7f6d5410a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:58.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:58.724+0000 7f6d52800700 1 --2- 172.21.15.33:0/825458113 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d5406ba10 0x7f6d5410a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:58.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:58.724+0000 7f6d52800700 1 -- 172.21.15.33:0/825458113 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d5406c3d0 msgr2=0x7f6d5410a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:58.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:58.724+0000 7f6d52800700 1 --2- 172.21.15.33:0/825458113 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d5406c3d0 0x7f6d5410a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:58.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:58.724+0000 7f6d52800700 1 -- 172.21.15.33:0/825458113 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d541ac330 con 0x7f6d54109500 2024-05-03T18:25:58.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:58.724+0000 7f6d51e00700 1 --2- 172.21.15.33:0/825458113 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d5406ba10 0x7f6d5410a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:25:58.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:58.724+0000 7f6d4be00700 1 --2- 172.21.15.33:0/825458113 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d5406c3d0 0x7f6d5410a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:25:58.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:58.724+0000 7f6d52800700 1 --2- 172.21.15.33:0/825458113 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d54109500 0x7f6d5410adb0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c007d70 tx=0x7f6d4c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:25:58.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:58.724+0000 7f6d4aa00700 1 -- 172.21.15.33:0/825458113 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d4c016020 con 0x7f6d54109500 2024-05-03T18:25:58.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:58.724+0000 7f6d4aa00700 1 -- 172.21.15.33:0/825458113 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6d4c00ad40 con 0x7f6d54109500 2024-05-03T18:25:58.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:58.724+0000 7f6d59000700 1 -- 172.21.15.33:0/825458113 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d541ac620 con 0x7f6d54109500 2024-05-03T18:25:58.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:58.724+0000 7f6d4aa00700 1 -- 172.21.15.33:0/825458113 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d4c0115e0 con 0x7f6d54109500 2024-05-03T18:25:58.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:58.724+0000 7f6d59000700 1 -- 172.21.15.33:0/825458113 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d541acb60 con 0x7f6d54109500 2024-05-03T18:25:58.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:58.726+0000 7f6d59000700 1 -- 172.21.15.33:0/825458113 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d54104bc0 con 0x7f6d54109500 2024-05-03T18:25:58.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:58.728+0000 7f6d4aa00700 1 -- 172.21.15.33:0/825458113 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f6d4c019020 con 0x7f6d54109500 2024-05-03T18:25:58.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:58.730+0000 7f6d4aa00700 1 --2- 172.21.15.33:0/825458113 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f6d380b2990 0x7f6d380b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:25:58.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:58.730+0000 7f6d4aa00700 1 -- 172.21.15.33:0/825458113 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f6d4c0e2210 con 0x7f6d54109500 2024-05-03T18:25:58.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:58.730+0000 7f6d51e00700 1 --2- 172.21.15.33:0/825458113 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f6d380b2990 0x7f6d380b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:25:58.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:58.731+0000 7f6d51e00700 1 --2- 172.21.15.33:0/825458113 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f6d380b2990 0x7f6d380b4e50 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6d4400b5c0 tx=0x7f6d4401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:25:58.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:58.734+0000 7f6d4aa00700 1 -- 172.21.15.33:0/825458113 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f6d4c020020 con 0x7f6d54109500 2024-05-03T18:25:58.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:25:58 smithi033 ceph-mon[28330]: pgmap v488: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:25:59.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:59.029+0000 7f6d59000700 1 -- 172.21.15.33:0/825458113 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6d5406a190 con 0x7f6d54109500 2024-05-03T18:25:59.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:59.030+0000 7f6d4aa00700 1 -- 172.21.15.33:0/825458113 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11801 (secure 0 0 0) 0x7f6d4c0a9b00 con 0x7f6d54109500 2024-05-03T18:25:59.031 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:25:59.031 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1327393503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1327393503},{"type":"v1","addr":"172.21.15.33:6835","nonce":1327393503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/2461472948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":2461472948},{"type":"v1","addr":"172.21.15.33:6837","nonce":2461472948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14559,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/442269281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":442269281},{"type":"v1","addr":"172.21.15.33:6839","nonce":442269281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14571,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/645571756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":645571756},{"type":"v1","addr":"172.21.15.33:6841","nonce":645571756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/203598909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":203598909},{"type":"v1","addr":"172.21.15.150:6833","nonce":203598909}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/4029842493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":4029842493},{"type":"v1","addr":"172.21.15.150:6835","nonce":4029842493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3463664423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3463664423},{"type":"v1","addr":"172.21.15.150:6837","nonce":3463664423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24481,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6839/4051648018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":4051648018},{"type":"v1","addr":"172.21.15.150:6839","nonce":4051648018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34418,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1865105479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1865105479},{"type":"v1","addr":"172.21.15.99:6833","nonce":1865105479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34445,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/1206393914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1206393914},{"type":"v1","addr":"172.21.15.99:6835","nonce":1206393914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-03T18:22:28.837091+0000","modified":"2024-05-03T18:23:49.758398+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34508},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34472":{"gid":34472,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6837/3360970342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3360970342},{"type":"v1","addr":"172.21.15.99:6837","nonce":3360970342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34508":{"gid":34508,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/919907308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":919907308},{"type":"v1","addr":"172.21.15.99:6839","nonce":919907308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34508,"qdb_cluster":[34508]},"id":1}]} 2024-05-03T18:25:59.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:59.035+0000 7f6d59000700 1 -- 172.21.15.33:0/825458113 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f6d380b2990 msgr2=0x7f6d380b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:59.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:59.035+0000 7f6d59000700 1 --2- 172.21.15.33:0/825458113 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f6d380b2990 0x7f6d380b4e50 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6d4400b5c0 tx=0x7f6d4401a040 comp rx=0 tx=0).stop 2024-05-03T18:25:59.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:59.035+0000 7f6d59000700 1 -- 172.21.15.33:0/825458113 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d54109500 msgr2=0x7f6d5410adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:25:59.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:59.035+0000 7f6d59000700 1 --2- 172.21.15.33:0/825458113 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d54109500 0x7f6d5410adb0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c007d70 tx=0x7f6d4c00a340 comp rx=0 tx=0).stop 2024-05-03T18:25:59.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:59.035+0000 7f6d59000700 1 -- 172.21.15.33:0/825458113 shutdown_connections 2024-05-03T18:25:59.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:59.035+0000 7f6d59000700 1 --2- 172.21.15.33:0/825458113 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d54109500 0x7f6d5410adb0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:59.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:59.035+0000 7f6d59000700 1 --2- 172.21.15.33:0/825458113 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d5406ba10 0x7f6d5410a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:59.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:59.035+0000 7f6d59000700 1 --2- 172.21.15.33:0/825458113 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f6d380b2990 0x7f6d380b4e50 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:59.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:59.035+0000 7f6d59000700 1 --2- 172.21.15.33:0/825458113 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d5406c3d0 0x7f6d5410a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:25:59.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:59.035+0000 7f6d59000700 1 -- 172.21.15.33:0/825458113 >> 172.21.15.33:0/825458113 conn(0x7f6d54079310 msgr2=0x7f6d5410e670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:25:59.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:59.035+0000 7f6d59000700 1 -- 172.21.15.33:0/825458113 shutdown_connections 2024-05-03T18:25:59.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:25:59.035+0000 7f6d59000700 1 -- 172.21.15.33:0/825458113 wait complete. 2024-05-03T18:25:59.037 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 21 2024-05-03T18:25:59.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:25:58 smithi099 ceph-mon[31658]: pgmap v488: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:25:59.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:25:58 smithi150 ceph-mon[32492]: pgmap v488: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:25:59.940 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T18:26:00.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:25:59 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/825458113' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:26:00.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:25:59 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/825458113' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:26:00.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:25:59 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/825458113' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:26:00.399 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:26:00.805 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:26:00 smithi033 ceph-mon[28330]: pgmap v489: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:26:01.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:26:00 smithi099 ceph-mon[31658]: pgmap v489: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:26:01.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:26:00 smithi150 ceph-mon[32492]: pgmap v489: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:26:02.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:02.705+0000 7fe224e00700 1 -- 172.21.15.33:0/3885695995 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe220075ff0 msgr2=0x7fe22010f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:26:02.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:02.705+0000 7fe224e00700 1 --2- 172.21.15.33:0/3885695995 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe220075ff0 0x7fe22010f960 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fe208009990 tx=0x7fe208009ca0 comp rx=0 tx=0).stop 2024-05-03T18:26:02.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:02.705+0000 7fe224e00700 1 -- 172.21.15.33:0/3885695995 shutdown_connections 2024-05-03T18:26:02.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:02.705+0000 7fe224e00700 1 --2- 172.21.15.33:0/3885695995 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe22010fea0 0x7fe220112290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:02.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:02.705+0000 7fe224e00700 1 --2- 172.21.15.33:0/3885695995 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe2200756d0 0x7fe220075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:02.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:02.705+0000 7fe224e00700 1 --2- 172.21.15.33:0/3885695995 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe220075ff0 0x7fe22010f960 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:02.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:02.705+0000 7fe224e00700 1 -- 172.21.15.33:0/3885695995 >> 172.21.15.33:0/3885695995 conn(0x7fe22007a600 msgr2=0x7fe22007aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:26:02.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:02.705+0000 7fe224e00700 1 -- 172.21.15.33:0/3885695995 shutdown_connections 2024-05-03T18:26:02.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:02.705+0000 7fe224e00700 1 -- 172.21.15.33:0/3885695995 wait complete. 2024-05-03T18:26:02.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:02.706+0000 7fe224e00700 1 Processor -- start 2024-05-03T18:26:02.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:02.707+0000 7fe224e00700 1 -- start start 2024-05-03T18:26:02.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:02.707+0000 7fe224e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe2200756d0 0x7fe2201a18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:26:02.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:02.707+0000 7fe224e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe220075ff0 0x7fe2201a1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:26:02.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:02.707+0000 7fe224e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe22010fea0 0x7fe22019bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:26:02.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:02.707+0000 7fe224e00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe220115760 con 0x7fe220075ff0 2024-05-03T18:26:02.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:02.707+0000 7fe224e00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe2201155e0 con 0x7fe22010fea0 2024-05-03T18:26:02.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:02.707+0000 7fe224e00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe2201158e0 con 0x7fe2200756d0 2024-05-03T18:26:02.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:02.707+0000 7fe215400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe220075ff0 0x7fe2201a1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:26:02.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:02.707+0000 7fe215400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe220075ff0 0x7fe2201a1e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37722/0 (socket says 172.21.15.33:37722) 2024-05-03T18:26:02.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:02.707+0000 7fe215400700 1 -- 172.21.15.33:0/4188257651 learned_addr learned my addr 172.21.15.33:0/4188257651 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:26:02.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:02.708+0000 7fe21de00700 1 --2- 172.21.15.33:0/4188257651 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe2200756d0 0x7fe2201a18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:26:02.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:02.708+0000 7fe21e800700 1 --2- 172.21.15.33:0/4188257651 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe22010fea0 0x7fe22019bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:26:02.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:02.708+0000 7fe215400700 1 -- 172.21.15.33:0/4188257651 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe2200756d0 msgr2=0x7fe2201a18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:26:02.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:02.708+0000 7fe215400700 1 --2- 172.21.15.33:0/4188257651 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe2200756d0 0x7fe2201a18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:02.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:02.708+0000 7fe215400700 1 -- 172.21.15.33:0/4188257651 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe22010fea0 msgr2=0x7fe22019bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:26:02.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:02.708+0000 7fe215400700 1 --2- 172.21.15.33:0/4188257651 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe22010fea0 0x7fe22019bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:02.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:02.708+0000 7fe215400700 1 -- 172.21.15.33:0/4188257651 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe22019c260 con 0x7fe220075ff0 2024-05-03T18:26:02.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:02.708+0000 7fe21e800700 1 --2- 172.21.15.33:0/4188257651 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe22010fea0 0x7fe22019bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:26:02.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:02.709+0000 7fe215400700 1 --2- 172.21.15.33:0/4188257651 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe220075ff0 0x7fe2201a1e30 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fe208000c00 tx=0x7fe208003710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:26:02.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:02.709+0000 7fe217400700 1 -- 172.21.15.33:0/4188257651 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe208027070 con 0x7fe220075ff0 2024-05-03T18:26:02.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:02.709+0000 7fe224e00700 1 -- 172.21.15.33:0/4188257651 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe22019c4f0 con 0x7fe220075ff0 2024-05-03T18:26:02.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:02.709+0000 7fe217400700 1 -- 172.21.15.33:0/4188257651 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe20800fc00 con 0x7fe220075ff0 2024-05-03T18:26:02.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:02.709+0000 7fe217400700 1 -- 172.21.15.33:0/4188257651 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe208022cd0 con 0x7fe220075ff0 2024-05-03T18:26:02.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:02.709+0000 7fe224e00700 1 -- 172.21.15.33:0/4188257651 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2201a87c0 con 0x7fe220075ff0 2024-05-03T18:26:02.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:02.711+0000 7fe224e00700 1 -- 172.21.15.33:0/4188257651 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe22006a190 con 0x7fe220075ff0 2024-05-03T18:26:02.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:02.713+0000 7fe217400700 1 -- 172.21.15.33:0/4188257651 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fe20802a070 con 0x7fe220075ff0 2024-05-03T18:26:02.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:02.715+0000 7fe217400700 1 --2- 172.21.15.33:0/4188257651 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fe2040b2990 0x7fe2040b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:26:02.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:02.715+0000 7fe217400700 1 -- 172.21.15.33:0/4188257651 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7fe2080e3d90 con 0x7fe220075ff0 2024-05-03T18:26:02.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:02.715+0000 7fe21de00700 1 --2- 172.21.15.33:0/4188257651 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fe2040b2990 0x7fe2040b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:26:02.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:02.716+0000 7fe21de00700 1 --2- 172.21.15.33:0/4188257651 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fe2040b2990 0x7fe2040b4e50 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe22007ae30 tx=0x7fe21000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:26:02.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:02.722+0000 7fe217400700 1 -- 172.21.15.33:0/4188257651 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fe208031070 con 0x7fe220075ff0 2024-05-03T18:26:03.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:03.011+0000 7fe224e00700 1 -- 172.21.15.33:0/4188257651 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe22019cfe0 con 0x7fe220075ff0 2024-05-03T18:26:03.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:03.013+0000 7fe217400700 1 -- 172.21.15.33:0/4188257651 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11801 (secure 0 0 0) 0x7fe208013070 con 0x7fe220075ff0 2024-05-03T18:26:03.013 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:26:03.013 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1327393503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1327393503},{"type":"v1","addr":"172.21.15.33:6835","nonce":1327393503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/2461472948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":2461472948},{"type":"v1","addr":"172.21.15.33:6837","nonce":2461472948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14559,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/442269281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":442269281},{"type":"v1","addr":"172.21.15.33:6839","nonce":442269281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14571,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/645571756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":645571756},{"type":"v1","addr":"172.21.15.33:6841","nonce":645571756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/203598909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":203598909},{"type":"v1","addr":"172.21.15.150:6833","nonce":203598909}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/4029842493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":4029842493},{"type":"v1","addr":"172.21.15.150:6835","nonce":4029842493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3463664423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3463664423},{"type":"v1","addr":"172.21.15.150:6837","nonce":3463664423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24481,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6839/4051648018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":4051648018},{"type":"v1","addr":"172.21.15.150:6839","nonce":4051648018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34418,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1865105479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1865105479},{"type":"v1","addr":"172.21.15.99:6833","nonce":1865105479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34445,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/1206393914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1206393914},{"type":"v1","addr":"172.21.15.99:6835","nonce":1206393914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-03T18:22:28.837091+0000","modified":"2024-05-03T18:23:49.758398+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34508},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34472":{"gid":34472,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6837/3360970342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3360970342},{"type":"v1","addr":"172.21.15.99:6837","nonce":3360970342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34508":{"gid":34508,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/919907308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":919907308},{"type":"v1","addr":"172.21.15.99:6839","nonce":919907308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34508,"qdb_cluster":[34508]},"id":1}]} 2024-05-03T18:26:03.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:03.016+0000 7fe224e00700 1 -- 172.21.15.33:0/4188257651 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fe2040b2990 msgr2=0x7fe2040b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:26:03.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:03.016+0000 7fe224e00700 1 --2- 172.21.15.33:0/4188257651 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fe2040b2990 0x7fe2040b4e50 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe22007ae30 tx=0x7fe21000a380 comp rx=0 tx=0).stop 2024-05-03T18:26:03.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:03.016+0000 7fe224e00700 1 -- 172.21.15.33:0/4188257651 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe220075ff0 msgr2=0x7fe2201a1e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:26:03.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:03.016+0000 7fe224e00700 1 --2- 172.21.15.33:0/4188257651 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe220075ff0 0x7fe2201a1e30 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fe208000c00 tx=0x7fe208003710 comp rx=0 tx=0).stop 2024-05-03T18:26:03.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:03.016+0000 7fe224e00700 1 -- 172.21.15.33:0/4188257651 shutdown_connections 2024-05-03T18:26:03.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:03.016+0000 7fe224e00700 1 --2- 172.21.15.33:0/4188257651 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fe2040b2990 0x7fe2040b4e50 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:03.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:03.016+0000 7fe224e00700 1 --2- 172.21.15.33:0/4188257651 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe22010fea0 0x7fe22019bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:03.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:03.016+0000 7fe224e00700 1 --2- 172.21.15.33:0/4188257651 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe2200756d0 0x7fe2201a18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:03.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:03.016+0000 7fe224e00700 1 --2- 172.21.15.33:0/4188257651 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe220075ff0 0x7fe2201a1e30 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:03.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:03.016+0000 7fe224e00700 1 -- 172.21.15.33:0/4188257651 >> 172.21.15.33:0/4188257651 conn(0x7fe22007a600 msgr2=0x7fe220101990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:26:03.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:03.017+0000 7fe224e00700 1 -- 172.21.15.33:0/4188257651 shutdown_connections 2024-05-03T18:26:03.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:03.017+0000 7fe224e00700 1 -- 172.21.15.33:0/4188257651 wait complete. 2024-05-03T18:26:03.018 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 21 2024-05-03T18:26:03.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:26:02 smithi150 ceph-mon[32492]: pgmap v490: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:26:03.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:26:02 smithi033 ceph-mon[28330]: pgmap v490: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:26:03.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:26:02 smithi099 ceph-mon[31658]: pgmap v490: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:26:03.961 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-03T18:26:04.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:26:03 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/4188257651' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:26:04.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:26:03 smithi150 ceph-mon[32492]: pgmap v491: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:26:04.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:26:03 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/4188257651' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:26:04.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:26:03 smithi033 ceph-mon[28330]: pgmap v491: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:26:04.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:26:03 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/4188257651' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:26:04.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:26:03 smithi099 ceph-mon[31658]: pgmap v491: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:26:04.418 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:26:06.624 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:26:06 smithi033 ceph-mon[28330]: pgmap v492: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:26:06.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:06.793+0000 7f6400a00700 1 -- 172.21.15.33:0/2466528872 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63fc1055d0 msgr2=0x7f63fc105a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:26:06.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:06.793+0000 7f6400a00700 1 --2- 172.21.15.33:0/2466528872 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63fc1055d0 0x7f63fc105a50 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f63e8009990 tx=0x7f63e8009ca0 comp rx=0 tx=0).stop 2024-05-03T18:26:06.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:06.793+0000 7f6400a00700 1 -- 172.21.15.33:0/2466528872 shutdown_connections 2024-05-03T18:26:06.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:06.793+0000 7f6400a00700 1 --2- 172.21.15.33:0/2466528872 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f63fc105f90 0x7f63fc1124a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:06.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:06.793+0000 7f6400a00700 1 --2- 172.21.15.33:0/2466528872 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63fc10b660 0x7f63fc10ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:06.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:06.793+0000 7f6400a00700 1 --2- 172.21.15.33:0/2466528872 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63fc1055d0 0x7f63fc105a50 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:06.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:06.793+0000 7f6400a00700 1 -- 172.21.15.33:0/2466528872 >> 172.21.15.33:0/2466528872 conn(0x7f63fc1010c0 msgr2=0x7f63fc1034e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:26:06.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:06.793+0000 7f6400a00700 1 -- 172.21.15.33:0/2466528872 shutdown_connections 2024-05-03T18:26:06.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:06.794+0000 7f6400a00700 1 -- 172.21.15.33:0/2466528872 wait complete. 2024-05-03T18:26:06.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:06.795+0000 7f6400a00700 1 Processor -- start 2024-05-03T18:26:06.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:06.795+0000 7f6400a00700 1 -- start start 2024-05-03T18:26:06.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:06.795+0000 7f6400a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f63fc1055d0 0x7f63fc1a1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:26:06.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:06.795+0000 7f6400a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63fc105f90 0x7f63fc1a1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:26:06.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:06.795+0000 7f6400a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63fc10b660 0x7f63fc19b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:26:06.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:06.795+0000 7f6400a00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f63fc115680 con 0x7f63fc105f90 2024-05-03T18:26:06.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:06.795+0000 7f6400a00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f63fc115500 con 0x7f63fc10b660 2024-05-03T18:26:06.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:06.795+0000 7f6400a00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f63fc115800 con 0x7f63fc1055d0 2024-05-03T18:26:06.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:26:06 smithi099 ceph-mon[31658]: pgmap v492: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:26:06.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:06.796+0000 7f63f9e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63fc105f90 0x7f63fc1a1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:26:06.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:06.796+0000 7f63f9e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63fc105f90 0x7f63fc1a1cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46210/0 (socket says 172.21.15.33:46210) 2024-05-03T18:26:06.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:06.796+0000 7f63f9e00700 1 -- 172.21.15.33:0/2165717033 learned_addr learned my addr 172.21.15.33:0/2165717033 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:26:06.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:06.796+0000 7f63fa800700 1 --2- 172.21.15.33:0/2165717033 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f63fc1055d0 0x7f63fc1a1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:26:06.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:06.796+0000 7f63fb200700 1 --2- 172.21.15.33:0/2165717033 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63fc10b660 0x7f63fc19b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:26:06.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:06.796+0000 7f63f9e00700 1 -- 172.21.15.33:0/2165717033 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f63fc1055d0 msgr2=0x7f63fc1a1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:26:06.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:06.796+0000 7f63f9e00700 1 --2- 172.21.15.33:0/2165717033 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f63fc1055d0 0x7f63fc1a1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:06.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:06.796+0000 7f63f9e00700 1 -- 172.21.15.33:0/2165717033 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63fc10b660 msgr2=0x7f63fc19b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:26:06.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:06.796+0000 7f63f9e00700 1 --2- 172.21.15.33:0/2165717033 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63fc10b660 0x7f63fc19b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:06.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:06.796+0000 7f63f9e00700 1 -- 172.21.15.33:0/2165717033 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63fc19c0a0 con 0x7f63fc105f90 2024-05-03T18:26:06.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:06.796+0000 7f63fa800700 1 --2- 172.21.15.33:0/2165717033 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f63fc1055d0 0x7f63fc1a1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:26:06.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:06.797+0000 7f63f9e00700 1 --2- 172.21.15.33:0/2165717033 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63fc105f90 0x7f63fc1a1cc0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f63e80057c0 tx=0x7f63e8009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:26:06.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:06.797+0000 7f63f3400700 1 -- 172.21.15.33:0/2165717033 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63e8027070 con 0x7f63fc105f90 2024-05-03T18:26:06.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:06.797+0000 7f63f3400700 1 -- 172.21.15.33:0/2165717033 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f63e800fda0 con 0x7f63fc105f90 2024-05-03T18:26:06.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:06.797+0000 7f63f3400700 1 -- 172.21.15.33:0/2165717033 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63e8022ca0 con 0x7f63fc105f90 2024-05-03T18:26:06.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:06.797+0000 7f6400a00700 1 -- 172.21.15.33:0/2165717033 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63fc19c330 con 0x7f63fc105f90 2024-05-03T18:26:06.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:06.797+0000 7f6400a00700 1 -- 172.21.15.33:0/2165717033 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63fc1a8650 con 0x7f63fc105f90 2024-05-03T18:26:06.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:06.798+0000 7f6400a00700 1 -- 172.21.15.33:0/2165717033 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63fc06a190 con 0x7f63fc105f90 2024-05-03T18:26:06.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:06.801+0000 7f63f3400700 1 -- 172.21.15.33:0/2165717033 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f63e802a070 con 0x7f63fc105f90 2024-05-03T18:26:06.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:06.802+0000 7f63f3400700 1 --2- 172.21.15.33:0/2165717033 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f63e00b2990 0x7f63e00b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:26:06.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:06.802+0000 7f63f3400700 1 -- 172.21.15.33:0/2165717033 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f63e80e3750 con 0x7f63fc105f90 2024-05-03T18:26:06.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:06.803+0000 7f63fa800700 1 --2- 172.21.15.33:0/2165717033 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f63e00b2990 0x7f63e00b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:26:06.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:06.803+0000 7f63fa800700 1 --2- 172.21.15.33:0/2165717033 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f63e00b2990 0x7f63e00b4e50 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f63ec005950 tx=0x7f63ec0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:26:06.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:06.805+0000 7f63f3400700 1 -- 172.21.15.33:0/2165717033 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f63e8013070 con 0x7f63fc105f90 2024-05-03T18:26:06.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:26:06 smithi150 ceph-mon[32492]: pgmap v492: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:26:07.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:07.046+0000 7f6400a00700 1 -- 172.21.15.33:0/2165717033 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f63fc02e510 con 0x7f63fc105f90 2024-05-03T18:26:07.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:07.050+0000 7f63f3400700 1 -- 172.21.15.33:0/2165717033 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24650 (secure 0 0 0) 0x7f63e80095a0 con 0x7f63fc105f90 2024-05-03T18:26:07.051 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:26:07.051 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":89,"fsid":"5703bba4-0978-11ef-bc97-c7b262605968","created":"2024-05-03T18:11:00.899879+0000","modified":"2024-05-03T18:24:21.532082+0000","last_up_change":"2024-05-03T18:21:00.390173+0000","last_in_change":"2024-05-03T18:20:34.103727+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-03T18:15:37.297738+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-03T18:22:19.875368+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-03T18:22:24.432692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-03T18:22:38.722443+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"fdcc4e66-4ba0-4378-a9a4-7d907cd00b4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":637686669},{"type":"v1","addr":"172.21.15.33:6803","nonce":637686669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":637686669},{"type":"v1","addr":"172.21.15.33:6805","nonce":637686669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":637686669},{"type":"v1","addr":"172.21.15.33:6809","nonce":637686669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":637686669},{"type":"v1","addr":"172.21.15.33:6807","nonce":637686669}]},"public_addr":"172.21.15.33:6803/637686669","cluster_addr":"172.21.15.33:6805/637686669","heartbeat_back_addr":"172.21.15.33:6809/637686669","heartbeat_front_addr":"172.21.15.33:6807/637686669","state":["exists","up"]},{"osd":1,"uuid":"cf78cad3-fc24-43f4-8784-21cabea7277c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":1536122841},{"type":"v1","addr":"172.21.15.99:6801","nonce":1536122841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":1536122841},{"type":"v1","addr":"172.21.15.99:6803","nonce":1536122841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":1536122841},{"type":"v1","addr":"172.21.15.99:6807","nonce":1536122841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":1536122841},{"type":"v1","addr":"172.21.15.99:6805","nonce":1536122841}]},"public_addr":"172.21.15.99:6801/1536122841","cluster_addr":"172.21.15.99:6803/1536122841","heartbeat_back_addr":"172.21.15.99:6807/1536122841","heartbeat_front_addr":"172.21.15.99:6805/1536122841","state":["exists","up"]},{"osd":2,"uuid":"59f066d7-626b-410a-8d67-daca6fbbd3ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":1995659622},{"type":"v1","addr":"172.21.15.150:6801","nonce":1995659622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":1995659622},{"type":"v1","addr":"172.21.15.150:6803","nonce":1995659622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":1995659622},{"type":"v1","addr":"172.21.15.150:6807","nonce":1995659622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":1995659622},{"type":"v1","addr":"172.21.15.150:6805","nonce":1995659622}]},"public_addr":"172.21.15.150:6801/1995659622","cluster_addr":"172.21.15.150:6803/1995659622","heartbeat_back_addr":"172.21.15.150:6807/1995659622","heartbeat_front_addr":"172.21.15.150:6805/1995659622","state":["exists","up"]},{"osd":3,"uuid":"d88ed022-4dcb-45fc-9fc6-41ef7c4bf202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":3814942724},{"type":"v1","addr":"172.21.15.33:6811","nonce":3814942724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":3814942724},{"type":"v1","addr":"172.21.15.33:6813","nonce":3814942724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":3814942724},{"type":"v1","addr":"172.21.15.33:6817","nonce":3814942724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":3814942724},{"type":"v1","addr":"172.21.15.33:6815","nonce":3814942724}]},"public_addr":"172.21.15.33:6811/3814942724","cluster_addr":"172.21.15.33:6813/3814942724","heartbeat_back_addr":"172.21.15.33:6817/3814942724","heartbeat_front_addr":"172.21.15.33:6815/3814942724","state":["exists","up"]},{"osd":4,"uuid":"1235e1a6-ff6e-4668-ac1a-073c2a3b7c98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":29081029},{"type":"v1","addr":"172.21.15.99:6809","nonce":29081029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":29081029},{"type":"v1","addr":"172.21.15.99:6811","nonce":29081029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":29081029},{"type":"v1","addr":"172.21.15.99:6815","nonce":29081029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":29081029},{"type":"v1","addr":"172.21.15.99:6813","nonce":29081029}]},"public_addr":"172.21.15.99:6809/29081029","cluster_addr":"172.21.15.99:6811/29081029","heartbeat_back_addr":"172.21.15.99:6815/29081029","heartbeat_front_addr":"172.21.15.99:6813/29081029","state":["exists","up"]},{"osd":5,"uuid":"4b84bff8-d6d7-4258-a0ee-53bc19802585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":3172189533},{"type":"v1","addr":"172.21.15.150:6809","nonce":3172189533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3172189533},{"type":"v1","addr":"172.21.15.150:6811","nonce":3172189533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3172189533},{"type":"v1","addr":"172.21.15.150:6815","nonce":3172189533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3172189533},{"type":"v1","addr":"172.21.15.150:6813","nonce":3172189533}]},"public_addr":"172.21.15.150:6809/3172189533","cluster_addr":"172.21.15.150:6811/3172189533","heartbeat_back_addr":"172.21.15.150:6815/3172189533","heartbeat_front_addr":"172.21.15.150:6813/3172189533","state":["exists","up"]},{"osd":6,"uuid":"51f5fbdd-3794-4c48-9f6c-478f0fa35ac5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":4234541418},{"type":"v1","addr":"172.21.15.33:6819","nonce":4234541418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":4234541418},{"type":"v1","addr":"172.21.15.33:6821","nonce":4234541418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":4234541418},{"type":"v1","addr":"172.21.15.33:6825","nonce":4234541418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":4234541418},{"type":"v1","addr":"172.21.15.33:6823","nonce":4234541418}]},"public_addr":"172.21.15.33:6819/4234541418","cluster_addr":"172.21.15.33:6821/4234541418","heartbeat_back_addr":"172.21.15.33:6825/4234541418","heartbeat_front_addr":"172.21.15.33:6823/4234541418","state":["exists","up"]},{"osd":7,"uuid":"982cc5b3-3163-4505-9bde-a27a8c289457","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":908673090},{"type":"v1","addr":"172.21.15.99:6817","nonce":908673090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":908673090},{"type":"v1","addr":"172.21.15.99:6819","nonce":908673090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":908673090},{"type":"v1","addr":"172.21.15.99:6823","nonce":908673090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":908673090},{"type":"v1","addr":"172.21.15.99:6821","nonce":908673090}]},"public_addr":"172.21.15.99:6817/908673090","cluster_addr":"172.21.15.99:6819/908673090","heartbeat_back_addr":"172.21.15.99:6823/908673090","heartbeat_front_addr":"172.21.15.99:6821/908673090","state":["exists","up"]},{"osd":8,"uuid":"84c9d037-eb84-4eb4-8aa2-e784b703d36c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":1308083602},{"type":"v1","addr":"172.21.15.150:6817","nonce":1308083602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":1308083602},{"type":"v1","addr":"172.21.15.150:6819","nonce":1308083602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":1308083602},{"type":"v1","addr":"172.21.15.150:6823","nonce":1308083602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":1308083602},{"type":"v1","addr":"172.21.15.150:6821","nonce":1308083602}]},"public_addr":"172.21.15.150:6817/1308083602","cluster_addr":"172.21.15.150:6819/1308083602","heartbeat_back_addr":"172.21.15.150:6823/1308083602","heartbeat_front_addr":"172.21.15.150:6821/1308083602","state":["exists","up"]},{"osd":9,"uuid":"a02e965f-0d8f-45e2-9490-337f7f6e3881","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":4133096905},{"type":"v1","addr":"172.21.15.33:6827","nonce":4133096905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":4133096905},{"type":"v1","addr":"172.21.15.33:6829","nonce":4133096905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4133096905},{"type":"v1","addr":"172.21.15.33:6833","nonce":4133096905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":4133096905},{"type":"v1","addr":"172.21.15.33:6831","nonce":4133096905}]},"public_addr":"172.21.15.33:6827/4133096905","cluster_addr":"172.21.15.33:6829/4133096905","heartbeat_back_addr":"172.21.15.33:6833/4133096905","heartbeat_front_addr":"172.21.15.33:6831/4133096905","state":["exists","up"]},{"osd":10,"uuid":"8b29bb96-76e8-45a0-ab19-8d082d755fe9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":935746136},{"type":"v1","addr":"172.21.15.99:6825","nonce":935746136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":935746136},{"type":"v1","addr":"172.21.15.99:6827","nonce":935746136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":935746136},{"type":"v1","addr":"172.21.15.99:6831","nonce":935746136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":935746136},{"type":"v1","addr":"172.21.15.99:6829","nonce":935746136}]},"public_addr":"172.21.15.99:6825/935746136","cluster_addr":"172.21.15.99:6827/935746136","heartbeat_back_addr":"172.21.15.99:6831/935746136","heartbeat_front_addr":"172.21.15.99:6829/935746136","state":["exists","up"]},{"osd":11,"uuid":"07a6a825-c902-4bfd-97cc-cc9074532af5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2069349808},{"type":"v1","addr":"172.21.15.150:6825","nonce":2069349808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2069349808},{"type":"v1","addr":"172.21.15.150:6827","nonce":2069349808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2069349808},{"type":"v1","addr":"172.21.15.150:6831","nonce":2069349808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2069349808},{"type":"v1","addr":"172.21.15.150:6829","nonce":2069349808}]},"public_addr":"172.21.15.150:6825/2069349808","cluster_addr":"172.21.15.150:6827/2069349808","heartbeat_back_addr":"172.21.15.150:6831/2069349808","heartbeat_front_addr":"172.21.15.150:6829/2069349808","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:14:27.454536+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:15:00.538570+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:15:34.125289+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:16:09.853870+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:16:44.928107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:17:19.644037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:17:56.108443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:18:31.731499+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:19:08.062358+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:19:45.204430+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:20:21.013710+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:20:57.653698+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11},{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/3991947997":"2024-05-04T18:12:19.127652+0000","172.21.15.33:0/965859483":"2024-05-04T18:12:19.127652+0000","172.21.15.33:0/2014902144":"2024-05-04T18:11:44.239835+0000","172.21.15.33:0/1314704208":"2024-05-04T18:11:44.239835+0000","172.21.15.33:0/1614640996":"2024-05-04T18:11:44.239835+0000","172.21.15.33:6800/4007322071":"2024-05-04T18:11:44.239835+0000","172.21.15.33:6801/2830135294":"2024-05-04T18:12:19.127652+0000","172.21.15.33:6800/2830135294":"2024-05-04T18:12:19.127652+0000","172.21.15.33:0/3767665033":"2024-05-04T18:12:19.127652+0000","172.21.15.33:6801/4007322071":"2024-05-04T18:11:44.239835+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-03T18:26:07.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:07.053+0000 7f6400a00700 1 -- 172.21.15.33:0/2165717033 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f63e00b2990 msgr2=0x7f63e00b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:26:07.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:07.053+0000 7f6400a00700 1 --2- 172.21.15.33:0/2165717033 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f63e00b2990 0x7f63e00b4e50 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f63ec005950 tx=0x7f63ec0058e0 comp rx=0 tx=0).stop 2024-05-03T18:26:07.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:07.053+0000 7f6400a00700 1 -- 172.21.15.33:0/2165717033 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63fc105f90 msgr2=0x7f63fc1a1cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:26:07.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:07.053+0000 7f6400a00700 1 --2- 172.21.15.33:0/2165717033 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63fc105f90 0x7f63fc1a1cc0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f63e80057c0 tx=0x7f63e8009f90 comp rx=0 tx=0).stop 2024-05-03T18:26:07.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:07.054+0000 7f6400a00700 1 -- 172.21.15.33:0/2165717033 shutdown_connections 2024-05-03T18:26:07.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:07.054+0000 7f6400a00700 1 --2- 172.21.15.33:0/2165717033 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f63e00b2990 0x7f63e00b4e50 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:07.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:07.054+0000 7f6400a00700 1 --2- 172.21.15.33:0/2165717033 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63fc10b660 0x7f63fc19b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:07.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:07.054+0000 7f6400a00700 1 --2- 172.21.15.33:0/2165717033 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f63fc1055d0 0x7f63fc1a1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:07.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:07.054+0000 7f6400a00700 1 --2- 172.21.15.33:0/2165717033 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63fc105f90 0x7f63fc1a1cc0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:07.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:07.054+0000 7f6400a00700 1 -- 172.21.15.33:0/2165717033 >> 172.21.15.33:0/2165717033 conn(0x7f63fc1010c0 msgr2=0x7f63fc102940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:26:07.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:07.054+0000 7f6400a00700 1 -- 172.21.15.33:0/2165717033 shutdown_connections 2024-05-03T18:26:07.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:07.054+0000 7f6400a00700 1 -- 172.21.15.33:0/2165717033 wait complete. 2024-05-03T18:26:07.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:26:07 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/2165717033' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T18:26:07.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:26:07 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/2165717033' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T18:26:07.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:26:07 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/2165717033' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T18:26:08.049 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T18:26:08.533 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:26:08.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:26:08 smithi099 ceph-mon[31658]: pgmap v493: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:26:08.854 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:26:08 smithi033 ceph-mon[28330]: pgmap v493: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:26:08.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:26:08 smithi150 ceph-mon[32492]: pgmap v493: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:26:10.614 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:26:10 smithi033 ceph-mon[28330]: pgmap v494: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:26:10.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:26:10 smithi099 ceph-mon[31658]: pgmap v494: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:26:10.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:26:10 smithi150 ceph-mon[32492]: pgmap v494: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:26:10.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:10.901+0000 7f2d6a600700 1 -- 172.21.15.33:0/3513015890 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d64104eb0 msgr2=0x7f2d6410d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:26:10.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:10.901+0000 7f2d6a600700 1 --2- 172.21.15.33:0/3513015890 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d64104eb0 0x7f2d6410d3a0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c009990 tx=0x7f2d4c009ca0 comp rx=0 tx=0).stop 2024-05-03T18:26:10.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:10.901+0000 7f2d6a600700 1 -- 172.21.15.33:0/3513015890 shutdown_connections 2024-05-03T18:26:10.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:10.901+0000 7f2d6a600700 1 --2- 172.21.15.33:0/3513015890 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2d6410d9c0 0x7f2d6410fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:10.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:10.901+0000 7f2d6a600700 1 --2- 172.21.15.33:0/3513015890 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d64104590 0x7f2d64104970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:10.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:10.901+0000 7f2d6a600700 1 --2- 172.21.15.33:0/3513015890 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d64104eb0 0x7f2d6410d3a0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:10.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:10.901+0000 7f2d6a600700 1 -- 172.21.15.33:0/3513015890 >> 172.21.15.33:0/3513015890 conn(0x7f2d640fddc0 msgr2=0x7f2d641001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:26:10.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:10.901+0000 7f2d6a600700 1 -- 172.21.15.33:0/3513015890 shutdown_connections 2024-05-03T18:26:10.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:10.901+0000 7f2d6a600700 1 -- 172.21.15.33:0/3513015890 wait complete. 2024-05-03T18:26:10.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:10.902+0000 7f2d6a600700 1 Processor -- start 2024-05-03T18:26:10.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:10.903+0000 7f2d6a600700 1 -- start start 2024-05-03T18:26:10.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:10.903+0000 7f2d6a600700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2d64104590 0x7f2d6419bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:26:10.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:10.903+0000 7f2d6a600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d64104eb0 0x7f2d6419c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:26:10.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:10.903+0000 7f2d6a600700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d6410d9c0 0x7f2d641a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:26:10.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:10.903+0000 7f2d6a600700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2d64113590 con 0x7f2d6410d9c0 2024-05-03T18:26:10.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:10.903+0000 7f2d6a600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2d64113410 con 0x7f2d64104eb0 2024-05-03T18:26:10.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:10.903+0000 7f2d6a600700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2d64113710 con 0x7f2d64104590 2024-05-03T18:26:10.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:10.904+0000 7f2d69c00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d6410d9c0 0x7f2d641a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:26:10.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:10.904+0000 7f2d69c00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d6410d9c0 0x7f2d641a0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46224/0 (socket says 172.21.15.33:46224) 2024-05-03T18:26:10.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:10.904+0000 7f2d69c00700 1 -- 172.21.15.33:0/4232652246 learned_addr learned my addr 172.21.15.33:0/4232652246 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:26:10.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:10.904+0000 7f2d5be00700 1 --2- 172.21.15.33:0/4232652246 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d64104eb0 0x7f2d6419c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:26:10.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:10.904+0000 7f2d69200700 1 --2- 172.21.15.33:0/4232652246 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2d64104590 0x7f2d6419bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:26:10.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:10.904+0000 7f2d69c00700 1 -- 172.21.15.33:0/4232652246 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2d64104590 msgr2=0x7f2d6419bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:26:10.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:10.904+0000 7f2d69c00700 1 --2- 172.21.15.33:0/4232652246 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2d64104590 0x7f2d6419bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:10.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:10.904+0000 7f2d69c00700 1 -- 172.21.15.33:0/4232652246 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d64104eb0 msgr2=0x7f2d6419c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:26:10.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:10.904+0000 7f2d69c00700 1 --2- 172.21.15.33:0/4232652246 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d64104eb0 0x7f2d6419c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:10.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:10.904+0000 7f2d69c00700 1 -- 172.21.15.33:0/4232652246 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d641a0d90 con 0x7f2d6410d9c0 2024-05-03T18:26:10.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:10.905+0000 7f2d69c00700 1 --2- 172.21.15.33:0/4232652246 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d6410d9c0 0x7f2d641a0650 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c00b970 tx=0x7f2d5c00bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:26:10.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:10.905+0000 7f2d62000700 1 -- 172.21.15.33:0/4232652246 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d5c019070 con 0x7f2d6410d9c0 2024-05-03T18:26:10.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:10.905+0000 7f2d62000700 1 -- 172.21.15.33:0/4232652246 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2d5c00ce40 con 0x7f2d6410d9c0 2024-05-03T18:26:10.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:10.905+0000 7f2d62000700 1 -- 172.21.15.33:0/4232652246 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d5c007370 con 0x7f2d6410d9c0 2024-05-03T18:26:10.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:10.905+0000 7f2d6a600700 1 -- 172.21.15.33:0/4232652246 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d641a1080 con 0x7f2d6410d9c0 2024-05-03T18:26:10.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:10.905+0000 7f2d6a600700 1 -- 172.21.15.33:0/4232652246 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d641a8900 con 0x7f2d6410d9c0 2024-05-03T18:26:10.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:10.906+0000 7f2d6a600700 1 -- 172.21.15.33:0/4232652246 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d6406a190 con 0x7f2d6410d9c0 2024-05-03T18:26:10.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:10.911+0000 7f2d62000700 1 -- 172.21.15.33:0/4232652246 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f2d5c01c070 con 0x7f2d6410d9c0 2024-05-03T18:26:10.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:10.912+0000 7f2d62000700 1 --2- 172.21.15.33:0/4232652246 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f2d480b2910 0x7f2d480b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:26:10.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:10.912+0000 7f2d62000700 1 -- 172.21.15.33:0/4232652246 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f2d5c0e2360 con 0x7f2d6410d9c0 2024-05-03T18:26:10.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:10.912+0000 7f2d69200700 1 --2- 172.21.15.33:0/4232652246 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f2d480b2910 0x7f2d480b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:26:10.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:10.913+0000 7f2d69200700 1 --2- 172.21.15.33:0/4232652246 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f2d480b2910 0x7f2d480b4dd0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2d54006fd0 tx=0x7f2d54008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:26:10.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:10.915+0000 7f2d62000700 1 -- 172.21.15.33:0/4232652246 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f2d5c023070 con 0x7f2d6410d9c0 2024-05-03T18:26:11.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:11.201+0000 7f2d6a600700 1 -- 172.21.15.33:0/4232652246 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2d6402e510 con 0x7f2d6410d9c0 2024-05-03T18:26:11.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:11.202+0000 7f2d62000700 1 -- 172.21.15.33:0/4232652246 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11801 (secure 0 0 0) 0x7f2d5c0a9ce0 con 0x7f2d6410d9c0 2024-05-03T18:26:11.202 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:26:11.202 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1327393503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1327393503},{"type":"v1","addr":"172.21.15.33:6835","nonce":1327393503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/2461472948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":2461472948},{"type":"v1","addr":"172.21.15.33:6837","nonce":2461472948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14559,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/442269281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":442269281},{"type":"v1","addr":"172.21.15.33:6839","nonce":442269281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14571,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/645571756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":645571756},{"type":"v1","addr":"172.21.15.33:6841","nonce":645571756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/203598909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":203598909},{"type":"v1","addr":"172.21.15.150:6833","nonce":203598909}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/4029842493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":4029842493},{"type":"v1","addr":"172.21.15.150:6835","nonce":4029842493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3463664423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3463664423},{"type":"v1","addr":"172.21.15.150:6837","nonce":3463664423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24481,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6839/4051648018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":4051648018},{"type":"v1","addr":"172.21.15.150:6839","nonce":4051648018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34418,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1865105479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1865105479},{"type":"v1","addr":"172.21.15.99:6833","nonce":1865105479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34445,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/1206393914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1206393914},{"type":"v1","addr":"172.21.15.99:6835","nonce":1206393914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-03T18:22:28.837091+0000","modified":"2024-05-03T18:23:49.758398+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34508},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34472":{"gid":34472,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6837/3360970342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3360970342},{"type":"v1","addr":"172.21.15.99:6837","nonce":3360970342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34508":{"gid":34508,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/919907308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":919907308},{"type":"v1","addr":"172.21.15.99:6839","nonce":919907308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34508,"qdb_cluster":[34508]},"id":1}]} 2024-05-03T18:26:11.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:11.205+0000 7f2d6a600700 1 -- 172.21.15.33:0/4232652246 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f2d480b2910 msgr2=0x7f2d480b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:26:11.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:11.205+0000 7f2d6a600700 1 --2- 172.21.15.33:0/4232652246 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f2d480b2910 0x7f2d480b4dd0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2d54006fd0 tx=0x7f2d54008040 comp rx=0 tx=0).stop 2024-05-03T18:26:11.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:11.205+0000 7f2d6a600700 1 -- 172.21.15.33:0/4232652246 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d6410d9c0 msgr2=0x7f2d641a0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:26:11.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:11.205+0000 7f2d6a600700 1 --2- 172.21.15.33:0/4232652246 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d6410d9c0 0x7f2d641a0650 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c00b970 tx=0x7f2d5c00bd30 comp rx=0 tx=0).stop 2024-05-03T18:26:11.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:11.205+0000 7f2d6a600700 1 -- 172.21.15.33:0/4232652246 shutdown_connections 2024-05-03T18:26:11.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:11.205+0000 7f2d6a600700 1 --2- 172.21.15.33:0/4232652246 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d6410d9c0 0x7f2d641a0650 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:11.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:11.205+0000 7f2d6a600700 1 --2- 172.21.15.33:0/4232652246 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2d64104590 0x7f2d6419bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:11.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:11.205+0000 7f2d6a600700 1 --2- 172.21.15.33:0/4232652246 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f2d480b2910 0x7f2d480b4dd0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:11.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:11.205+0000 7f2d6a600700 1 --2- 172.21.15.33:0/4232652246 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d64104eb0 0x7f2d6419c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:11.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:11.205+0000 7f2d6a600700 1 -- 172.21.15.33:0/4232652246 >> 172.21.15.33:0/4232652246 conn(0x7f2d640fddc0 msgr2=0x7f2d640fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:26:11.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:11.206+0000 7f2d6a600700 1 -- 172.21.15.33:0/4232652246 shutdown_connections 2024-05-03T18:26:11.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:11.206+0000 7f2d6a600700 1 -- 172.21.15.33:0/4232652246 wait complete. 2024-05-03T18:26:11.207 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 21 2024-05-03T18:26:11.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:26:11 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/4232652246' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:26:11.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:26:11 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/4232652246' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:26:11.963 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:26:11 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/4232652246' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:26:12.024 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 21, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-03T18:22:28.837091+0000', 'modified': '2024-05-03T18:23:49.758398+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 34508}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_34472': {'gid': 34472, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.99:6837/3360970342', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6836', 'nonce': 3360970342}, {'type': 'v1', 'addr': '172.21.15.99:6837', 'nonce': 3360970342}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34508': {'gid': 34508, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.99:6839/919907308', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6838', 'nonce': 919907308}, {'type': 'v1', 'addr': '172.21.15.99:6839', 'nonce': 919907308}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 34508, 'qdb_cluster': [34508]} 2024-05-03T18:26:12.025 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-03T18:26:12.025 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.b status 2024-05-03T18:26:12.497 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:26:12.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:26:12 smithi099 ceph-mon[31658]: pgmap v495: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:26:12.845 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:26:12 smithi033 ceph-mon[28330]: pgmap v495: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:26:12.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:26:12 smithi150 ceph-mon[32492]: pgmap v495: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:26:14.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:26:14 smithi099 ceph-mon[31658]: pgmap v496: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:26:14.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.810+0000 7ff707200700 1 -- 172.21.15.33:0/1931765550 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff70010b6d0 msgr2=0x7ff70010bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:26:14.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.810+0000 7ff707200700 1 --2- 172.21.15.33:0/1931765550 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff70010b6d0 0x7ff70010bab0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ff6f0009990 tx=0x7ff6f0009ca0 comp rx=0 tx=0).stop 2024-05-03T18:26:14.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.811+0000 7ff707200700 1 -- 172.21.15.33:0/1931765550 shutdown_connections 2024-05-03T18:26:14.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.811+0000 7ff707200700 1 --2- 172.21.15.33:0/1931765550 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff700106090 0x7ff7001125a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:14.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.811+0000 7ff707200700 1 --2- 172.21.15.33:0/1931765550 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff70010b6d0 0x7ff70010bab0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:14.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.811+0000 7ff707200700 1 --2- 172.21.15.33:0/1931765550 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7001056d0 0x7ff700105b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:14.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.811+0000 7ff707200700 1 -- 172.21.15.33:0/1931765550 >> 172.21.15.33:0/1931765550 conn(0x7ff7001011a0 msgr2=0x7ff7001035c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:26:14.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.811+0000 7ff707200700 1 -- 172.21.15.33:0/1931765550 shutdown_connections 2024-05-03T18:26:14.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.811+0000 7ff707200700 1 -- 172.21.15.33:0/1931765550 wait complete. 2024-05-03T18:26:14.812 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:26:14 smithi033 ceph-mon[28330]: pgmap v496: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:26:14.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.812+0000 7ff707200700 1 Processor -- start 2024-05-03T18:26:14.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.812+0000 7ff707200700 1 -- start start 2024-05-03T18:26:14.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.813+0000 7ff707200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7001056d0 0x7ff7001a1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:26:14.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.813+0000 7ff707200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff700106090 0x7ff7001a1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:26:14.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.813+0000 7ff707200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff70010b6d0 0x7ff70019bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:26:14.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.813+0000 7ff707200700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff7001157d0 con 0x7ff700106090 2024-05-03T18:26:14.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.813+0000 7ff707200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff700115650 con 0x7ff70010b6d0 2024-05-03T18:26:14.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.813+0000 7ff707200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff700115950 con 0x7ff7001056d0 2024-05-03T18:26:14.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.813+0000 7ff6f7e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff700106090 0x7ff7001a1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:26:14.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.813+0000 7ff6f7e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff700106090 0x7ff7001a1f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46244/0 (socket says 172.21.15.33:46244) 2024-05-03T18:26:14.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.813+0000 7ff6f7e00700 1 -- 172.21.15.33:0/2306026523 learned_addr learned my addr 172.21.15.33:0/2306026523 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:26:14.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.813+0000 7ff705200700 1 --2- 172.21.15.33:0/2306026523 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff70010b6d0 0x7ff70019bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:26:14.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.813+0000 7ff6ffe00700 1 --2- 172.21.15.33:0/2306026523 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7001056d0 0x7ff7001a1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:26:14.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.814+0000 7ff6f7e00700 1 -- 172.21.15.33:0/2306026523 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7001056d0 msgr2=0x7ff7001a1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:26:14.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.814+0000 7ff6f7e00700 1 --2- 172.21.15.33:0/2306026523 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7001056d0 0x7ff7001a1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:14.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.814+0000 7ff6f7e00700 1 -- 172.21.15.33:0/2306026523 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff70010b6d0 msgr2=0x7ff70019bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:26:14.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.814+0000 7ff6f7e00700 1 --2- 172.21.15.33:0/2306026523 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff70010b6d0 0x7ff70019bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:14.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.814+0000 7ff6f7e00700 1 -- 172.21.15.33:0/2306026523 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff70019c3f0 con 0x7ff700106090 2024-05-03T18:26:14.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.814+0000 7ff705200700 1 --2- 172.21.15.33:0/2306026523 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff70010b6d0 0x7ff70019bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:26:14.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.815+0000 7ff6f7e00700 1 --2- 172.21.15.33:0/2306026523 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff700106090 0x7ff7001a1f80 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ff6e800e9c0 tx=0x7ff6e800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:26:14.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.815+0000 7ff6fd600700 1 -- 172.21.15.33:0/2306026523 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e800cbd0 con 0x7ff700106090 2024-05-03T18:26:14.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.815+0000 7ff6fd600700 1 -- 172.21.15.33:0/2306026523 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff6e800cd70 con 0x7ff700106090 2024-05-03T18:26:14.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.815+0000 7ff707200700 1 -- 172.21.15.33:0/2306026523 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff70019c6e0 con 0x7ff700106090 2024-05-03T18:26:14.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.815+0000 7ff6fd600700 1 -- 172.21.15.33:0/2306026523 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e8005540 con 0x7ff700106090 2024-05-03T18:26:14.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.815+0000 7ff707200700 1 -- 172.21.15.33:0/2306026523 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7001a89c0 con 0x7ff700106090 2024-05-03T18:26:14.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.819+0000 7ff6fd600700 1 -- 172.21.15.33:0/2306026523 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7ff6e8004750 con 0x7ff700106090 2024-05-03T18:26:14.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.820+0000 7ff711ad2b80 1 Processor -- start 2024-05-03T18:26:14.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.820+0000 7ff711ad2b80 1 -- start start 2024-05-03T18:26:14.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.820+0000 7ff711ad2b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e470a77760 0x55e470a77b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:26:14.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.820+0000 7ff711ad2b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e470a78080 0x55e470a72f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:26:14.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.820+0000 7ff711ad2b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e470a73580 0x55e470a739e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:26:14.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.820+0000 7ff711ad2b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e4709c59d0 con 0x55e470a77760 2024-05-03T18:26:14.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.820+0000 7ff711ad2b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55e470963710 con 0x55e470a78080 2024-05-03T18:26:14.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.820+0000 7ff711ad2b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55e4709c1990 con 0x55e470a73580 2024-05-03T18:26:14.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.821+0000 7ff705200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e470a77760 0x55e470a77b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:26:14.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.821+0000 7ff705200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e470a77760 0x55e470a77b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46250/0 (socket says 172.21.15.33:46250) 2024-05-03T18:26:14.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.821+0000 7ff705200700 1 -- 172.21.15.33:0/257702096 learned_addr learned my addr 172.21.15.33:0/257702096 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:26:14.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.821+0000 7ff6fd600700 1 --2- 172.21.15.33:0/2306026523 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7ff6e40b2910 0x7ff6e40b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:26:14.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.821+0000 7ff6ffe00700 1 --2- 172.21.15.33:0/257702096 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e470a78080 0x55e470a72f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:26:14.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.821+0000 7ff6fd600700 1 -- 172.21.15.33:0/2306026523 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7ff6e80e29d0 con 0x7ff700106090 2024-05-03T18:26:14.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.821+0000 7ff6f7e00700 1 --2- 172.21.15.33:0/257702096 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e470a73580 0x55e470a739e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:26:14.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.821+0000 7ff705200700 1 -- 172.21.15.33:0/257702096 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e470a73580 msgr2=0x55e470a739e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:26:14.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.821+0000 7ff705200700 1 --2- 172.21.15.33:0/257702096 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e470a73580 0x55e470a739e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:14.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.821+0000 7ff705200700 1 -- 172.21.15.33:0/257702096 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e470a78080 msgr2=0x55e470a72f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:26:14.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.822+0000 7ff705200700 1 --2- 172.21.15.33:0/257702096 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e470a78080 0x55e470a72f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:14.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.822+0000 7ff705200700 1 -- 172.21.15.33:0/257702096 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e470912880 con 0x55e470a77760 2024-05-03T18:26:14.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.822+0000 7ff6f7e00700 1 --2- 172.21.15.33:0/257702096 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e470a73580 0x55e470a739e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:26:14.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.822+0000 7ff705200700 1 --2- 172.21.15.33:0/2306026523 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7ff6e40b2910 0x7ff6e40b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:26:14.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.822+0000 7ff705200700 1 --2- 172.21.15.33:0/257702096 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e470a77760 0x55e470a77b40 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ff6f8007530 tx=0x7ff6f8007840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f580792f352a2431 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:26:14.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.823+0000 7ff6f5600700 1 -- 172.21.15.33:0/257702096 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f801c040 con 0x55e470a77760 2024-05-03T18:26:14.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.823+0000 7ff6f5600700 1 -- 172.21.15.33:0/257702096 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff6f801d020 con 0x55e470a77760 2024-05-03T18:26:14.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.823+0000 7ff6f5600700 1 -- 172.21.15.33:0/257702096 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f8022850 con 0x55e470a77760 2024-05-03T18:26:14.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.823+0000 7ff711ad2b80 1 -- 172.21.15.33:0/257702096 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e470a77760 msgr2=0x55e470a77b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:26:14.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.823+0000 7ff711ad2b80 1 --2- 172.21.15.33:0/257702096 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e470a77760 0x55e470a77b40 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ff6f8007530 tx=0x7ff6f8007840 comp rx=0 tx=0).stop 2024-05-03T18:26:14.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.823+0000 7ff705200700 1 --2- 172.21.15.33:0/2306026523 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7ff6e40b2910 0x7ff6e40b4dd0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff6f801e000 tx=0x7ff6f801f000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:26:14.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.824+0000 7ff711ad2b80 1 -- 172.21.15.33:0/257702096 shutdown_connections 2024-05-03T18:26:14.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.824+0000 7ff711ad2b80 1 --2- 172.21.15.33:0/257702096 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e470a73580 0x55e470a739e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:14.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.824+0000 7ff711ad2b80 1 --2- 172.21.15.33:0/257702096 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e470a77760 0x55e470a77b40 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:14.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.824+0000 7ff711ad2b80 1 --2- 172.21.15.33:0/257702096 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e470a78080 0x55e470a72f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:14.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.824+0000 7ff711ad2b80 1 -- 172.21.15.33:0/257702096 >> 172.21.15.33:0/257702096 conn(0x55e4709a1b10 msgr2=0x55e470a6d7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:26:14.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.824+0000 7ff711ad2b80 1 -- 172.21.15.33:0/257702096 shutdown_connections 2024-05-03T18:26:14.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.824+0000 7ff711ad2b80 1 -- 172.21.15.33:0/257702096 wait complete. 2024-05-03T18:26:14.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.824+0000 7ff711ad2b80 1 -- start start 2024-05-03T18:26:14.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.825+0000 7ff711ad2b80 1 Processor -- start 2024-05-03T18:26:14.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.825+0000 7ff711ad2b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e470a73580 0x55e470a7ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:26:14.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.825+0000 7ff711ad2b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e470a77760 0x55e470a7c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:26:14.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.825+0000 7ff711ad2b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e470a78080 0x55e470a7c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:26:14.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.825+0000 7ff711ad2b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e470a7fc50 con 0x55e470a77760 2024-05-03T18:26:14.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.825+0000 7ff711ad2b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55e47092bad0 con 0x55e470a78080 2024-05-03T18:26:14.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.825+0000 7ff711ad2b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55e470a7fdd0 con 0x55e470a73580 2024-05-03T18:26:14.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.825+0000 7ff711ad2b80 10 client.0 _ll_register_callbacks cb 0x55e47096cc20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:26:14.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.826+0000 7ff6f7e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e470a77760 0x55e470a7c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:26:14.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.826+0000 7ff6f7e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e470a77760 0x55e470a7c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46264/0 (socket says 172.21.15.33:46264) 2024-05-03T18:26:14.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.826+0000 7ff6f7e00700 1 -- 172.21.15.33:0/1551260944 learned_addr learned my addr 172.21.15.33:0/1551260944 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:26:14.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.826+0000 7ff6ffe00700 1 --2- 172.21.15.33:0/1551260944 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e470a73580 0x55e470a7ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:26:14.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.826+0000 7ff705200700 1 --2- 172.21.15.33:0/1551260944 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e470a78080 0x55e470a7c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:26:14.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.826+0000 7ff6f7e00700 1 -- 172.21.15.33:0/1551260944 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e470a73580 msgr2=0x55e470a7ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:26:14.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.826+0000 7ff6f7e00700 1 --2- 172.21.15.33:0/1551260944 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e470a73580 0x55e470a7ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:14.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.826+0000 7ff6f7e00700 1 -- 172.21.15.33:0/1551260944 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e470a78080 msgr2=0x55e470a7c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:26:14.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.826+0000 7ff6f7e00700 1 --2- 172.21.15.33:0/1551260944 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e470a78080 0x55e470a7c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:14.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.826+0000 7ff6f7e00700 1 -- 172.21.15.33:0/1551260944 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e47098b3d0 con 0x55e470a77760 2024-05-03T18:26:14.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.826+0000 7ff705200700 1 --2- 172.21.15.33:0/1551260944 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e470a78080 0x55e470a7c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:26:14.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.827+0000 7ff6f7e00700 1 --2- 172.21.15.33:0/1551260944 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e470a77760 0x55e470a7c120 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7ff6e80e0030 tx=0x7ff6e80aa0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93e1358a7d81bcc4 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:26:14.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.827+0000 7ff6ee000700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:26:14.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.827+0000 7ff6ee000700 1 -- 172.21.15.33:0/1551260944 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e80b22b0 con 0x55e470a77760 2024-05-03T18:26:14.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.827+0000 7ff6ee000700 1 -- 172.21.15.33:0/1551260944 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7ff6e80a8f80 con 0x55e470a77760 2024-05-03T18:26:14.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.827+0000 7ff711ad2b80 1 -- 172.21.15.33:0/1551260944 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6f801d020 con 0x55e470a77760 2024-05-03T18:26:14.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.827+0000 7ff6ee000700 1 -- 172.21.15.33:0/1551260944 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7ff6e80a9620 con 0x55e470a77760 2024-05-03T18:26:14.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.827+0000 7ff711ad2b80 10 client.15018 fetch_fsmap learned FSMap version 21 2024-05-03T18:26:14.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.827+0000 7ff711ad2b80 1 -- 172.21.15.33:0/1551260944 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55e470a8a2f0 con 0x55e470a77760 2024-05-03T18:26:14.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.828+0000 7ff6ee000700 1 -- 172.21.15.33:0/1551260944 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff6f801d020 con 0x55e470a77760 2024-05-03T18:26:14.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.828+0000 7ff6ee000700 1 -- 172.21.15.33:0/1551260944 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7ff6e80c4080 con 0x55e470a77760 2024-05-03T18:26:14.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.828+0000 7ff711ad2b80 10 client.15018 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:26:14.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.828+0000 7ff711ad2b80 10 client.15018 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-03T18:26:14.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.828+0000 7ff711ad2b80 20 client.15018 populate_metadata read hostname 'smithi033' 2024-05-03T18:26:14.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.828+0000 7ff711ad2b80 1 --2- 172.21.15.33:0/1551260944 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55e470a8c8b0 0x55e470a8dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:26:14.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.828+0000 7ff711ad2b80 4 client.15018 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:26:14.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.828+0000 7ff711ad2b80 1 -- 172.21.15.33:0/1551260944 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e470a40180 con 0x55e470a8c8b0 2024-05-03T18:26:14.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.829+0000 7ff6ffe00700 1 --2- 172.21.15.33:0/1551260944 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55e470a8c8b0 0x55e470a8dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:26:14.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.830+0000 7ff6ffe00700 1 --2- 172.21.15.33:0/1551260944 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55e470a8c8b0 0x55e470a8dd20 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fddf335019cadd51 server_cookie=9fb8f25a4f7ccd0a in_seq=0 out_seq=0 2024-05-03T18:26:14.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.830+0000 7ff6ee000700 10 client.15018 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:26:14.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.832+0000 7ff6ee000700 1 -- 172.21.15.33:0/1551260944 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e470a40180 con 0x55e470a8c8b0 2024-05-03T18:26:14.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.832+0000 7ff6ee000700 10 client.15018 handle_command_reply: tid=0 2024-05-03T18:26:14.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.832+0000 7ff711ad2b80 1 client.15018 shutdown 2024-05-03T18:26:14.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.832+0000 7ff711ad2b80 1 -- 172.21.15.33:0/1551260944 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e470a77760 msgr2=0x55e470a7c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:26:14.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.832+0000 7ff711ad2b80 1 --2- 172.21.15.33:0/1551260944 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e470a77760 0x55e470a7c120 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7ff6e80e0030 tx=0x7ff6e80aa0d0 comp rx=0 tx=0).stop 2024-05-03T18:26:14.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.832+0000 7ff711ad2b80 1 -- 172.21.15.33:0/1551260944 shutdown_connections 2024-05-03T18:26:14.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.832+0000 7ff711ad2b80 1 --2- 172.21.15.33:0/1551260944 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55e470a8c8b0 0x55e470a8dd20 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:14.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.832+0000 7ff711ad2b80 1 --2- 172.21.15.33:0/1551260944 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e470a78080 0x55e470a7c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:14.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.832+0000 7ff711ad2b80 1 --2- 172.21.15.33:0/1551260944 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e470a73580 0x55e470a7ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:14.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.832+0000 7ff711ad2b80 1 --2- 172.21.15.33:0/1551260944 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e470a77760 0x55e470a7c120 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:14.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.832+0000 7ff711ad2b80 1 -- 172.21.15.33:0/1551260944 >> 172.21.15.33:0/1551260944 conn(0x55e4709a1b10 msgr2=0x55e470a78d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:26:14.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.832+0000 7ff6ee000700 0 client.15018 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:26:14.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.832+0000 7ff711ad2b80 1 -- 172.21.15.33:0/1551260944 shutdown_connections 2024-05-03T18:26:14.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.832+0000 7ff711ad2b80 1 -- 172.21.15.33:0/1551260944 wait complete. 2024-05-03T18:26:14.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.832+0000 7ff711ad2b80 20 client.15018 trim_cache size 0 max 16384 2024-05-03T18:26:14.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.856+0000 7ff711ad2b80 1 Processor -- start 2024-05-03T18:26:14.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.856+0000 7ff711ad2b80 1 -- start start 2024-05-03T18:26:14.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.856+0000 7ff711ad2b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e470a73580 0x55e470a7b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:26:14.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.856+0000 7ff711ad2b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e470a77760 0x55e470a6d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:26:14.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.856+0000 7ff711ad2b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e470a78080 0x55e470a70f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:26:14.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.856+0000 7ff711ad2b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e470a8c060 con 0x55e470a78080 2024-05-03T18:26:14.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.856+0000 7ff711ad2b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55e470a8c730 con 0x55e470a77760 2024-05-03T18:26:14.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.856+0000 7ff711ad2b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55e470a8ce00 con 0x55e470a73580 2024-05-03T18:26:14.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.857+0000 7ff705200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e470a78080 0x55e470a70f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:26:14.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.857+0000 7ff705200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e470a78080 0x55e470a70f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46280/0 (socket says 172.21.15.33:46280) 2024-05-03T18:26:14.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.857+0000 7ff705200700 1 -- 172.21.15.33:0/3540696276 learned_addr learned my addr 172.21.15.33:0/3540696276 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:26:14.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.857+0000 7ff6ffe00700 1 --2- 172.21.15.33:0/3540696276 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e470a73580 0x55e470a7b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:26:14.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.857+0000 7ff6f7e00700 1 --2- 172.21.15.33:0/3540696276 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e470a77760 0x55e470a6d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:26:14.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.858+0000 7ff705200700 1 -- 172.21.15.33:0/3540696276 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e470a73580 msgr2=0x55e470a7b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:26:14.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.858+0000 7ff705200700 1 --2- 172.21.15.33:0/3540696276 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e470a73580 0x55e470a7b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:14.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.858+0000 7ff705200700 1 -- 172.21.15.33:0/3540696276 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e470a77760 msgr2=0x55e470a6d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:26:14.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.858+0000 7ff705200700 1 --2- 172.21.15.33:0/3540696276 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e470a77760 0x55e470a6d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:14.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.858+0000 7ff705200700 1 -- 172.21.15.33:0/3540696276 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e470a773b0 con 0x55e470a78080 2024-05-03T18:26:14.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.858+0000 7ff6f7e00700 1 --2- 172.21.15.33:0/3540696276 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e470a77760 0x55e470a6d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:26:14.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.858+0000 7ff705200700 1 --2- 172.21.15.33:0/3540696276 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e470a78080 0x55e470a70f50 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7ff6f80094f0 tx=0x7ff6f8013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7acdbc0646b17397 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:26:14.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.858+0000 7ff6f4c00700 1 -- 172.21.15.33:0/3540696276 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f802f070 con 0x55e470a78080 2024-05-03T18:26:14.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.858+0000 7ff6f4c00700 1 -- 172.21.15.33:0/3540696276 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff6f8004a90 con 0x55e470a78080 2024-05-03T18:26:14.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.858+0000 7ff6f4c00700 1 -- 172.21.15.33:0/3540696276 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f801c040 con 0x55e470a78080 2024-05-03T18:26:14.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.859+0000 7ff711ad2b80 1 -- 172.21.15.33:0/3540696276 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e470a78080 msgr2=0x55e470a70f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:26:14.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.859+0000 7ff711ad2b80 1 --2- 172.21.15.33:0/3540696276 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e470a78080 0x55e470a70f50 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7ff6f80094f0 tx=0x7ff6f8013000 comp rx=0 tx=0).stop 2024-05-03T18:26:14.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.859+0000 7ff711ad2b80 1 -- 172.21.15.33:0/3540696276 shutdown_connections 2024-05-03T18:26:14.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.859+0000 7ff711ad2b80 1 --2- 172.21.15.33:0/3540696276 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e470a78080 0x55e470a70f50 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:14.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.859+0000 7ff711ad2b80 1 --2- 172.21.15.33:0/3540696276 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e470a73580 0x55e470a7b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:14.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.859+0000 7ff711ad2b80 1 --2- 172.21.15.33:0/3540696276 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e470a77760 0x55e470a6d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:14.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.859+0000 7ff711ad2b80 1 -- 172.21.15.33:0/3540696276 >> 172.21.15.33:0/3540696276 conn(0x55e4709a1b10 msgr2=0x55e470a88920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:26:14.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.859+0000 7ff711ad2b80 1 -- 172.21.15.33:0/3540696276 shutdown_connections 2024-05-03T18:26:14.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.859+0000 7ff711ad2b80 1 -- 172.21.15.33:0/3540696276 wait complete. 2024-05-03T18:26:14.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.860+0000 7ff711ad2b80 1 -- start start 2024-05-03T18:26:14.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.860+0000 7ff711ad2b80 1 Processor -- start 2024-05-03T18:26:14.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.860+0000 7ff711ad2b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e470a73580 0x55e470a96190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:26:14.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.860+0000 7ff711ad2b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e470a77760 0x55e470a966d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:26:14.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.861+0000 7ff711ad2b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e470a78080 0x55e470a96c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:26:14.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.861+0000 7ff711ad2b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e470a8dba0 con 0x55e470a73580 2024-05-03T18:26:14.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.861+0000 7ff711ad2b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55e470a8d4d0 con 0x55e470a78080 2024-05-03T18:26:14.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.861+0000 7ff711ad2b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55e470a8e270 con 0x55e470a77760 2024-05-03T18:26:14.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.861+0000 7ff711ad2b80 10 client.0 _ll_register_callbacks cb 0x55e47096cc20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:26:14.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.861+0000 7ff6ffe00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e470a73580 0x55e470a96190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:26:14.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.861+0000 7ff6ffe00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e470a73580 0x55e470a96190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46286/0 (socket says 172.21.15.33:46286) 2024-05-03T18:26:14.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.861+0000 7ff6ffe00700 1 -- 172.21.15.33:0/761219452 learned_addr learned my addr 172.21.15.33:0/761219452 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:26:14.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.861+0000 7ff6f7e00700 1 --2- 172.21.15.33:0/761219452 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e470a77760 0x55e470a966d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:26:14.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.861+0000 7ff705200700 1 --2- 172.21.15.33:0/761219452 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e470a78080 0x55e470a96c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:26:14.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.861+0000 7ff6ffe00700 1 -- 172.21.15.33:0/761219452 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e470a77760 msgr2=0x55e470a966d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:26:14.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.861+0000 7ff6ffe00700 1 --2- 172.21.15.33:0/761219452 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e470a77760 0x55e470a966d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:14.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.861+0000 7ff6ffe00700 1 -- 172.21.15.33:0/761219452 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e470a78080 msgr2=0x55e470a96c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:26:14.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.861+0000 7ff6ffe00700 1 --2- 172.21.15.33:0/761219452 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e470a78080 0x55e470a96c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:14.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.861+0000 7ff6ffe00700 1 -- 172.21.15.33:0/761219452 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e470a97420 con 0x55e470a73580 2024-05-03T18:26:14.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.861+0000 7ff6f7e00700 1 --2- 172.21.15.33:0/761219452 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e470a77760 0x55e470a966d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:26:14.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.862+0000 7ff6ffe00700 1 --2- 172.21.15.33:0/761219452 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e470a73580 0x55e470a96190 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ff6f0009990 tx=0x7ff6f00052e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83c1cdb6c24c2cf2 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:26:14.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.862+0000 7ff6ee000700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:26:14.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.862+0000 7ff6ee000700 1 -- 172.21.15.33:0/761219452 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f0003880 con 0x55e470a73580 2024-05-03T18:26:14.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.862+0000 7ff711ad2b80 1 -- 172.21.15.33:0/761219452 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6f8004a90 con 0x55e470a73580 2024-05-03T18:26:14.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.862+0000 7ff6ee000700 1 -- 172.21.15.33:0/761219452 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7ff6f0003a20 con 0x55e470a73580 2024-05-03T18:26:14.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.862+0000 7ff6ee000700 1 -- 172.21.15.33:0/761219452 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7ff6f0036050 con 0x55e470a73580 2024-05-03T18:26:14.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.862+0000 7ff711ad2b80 10 client.15024 fetch_fsmap learned FSMap version 21 2024-05-03T18:26:14.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.862+0000 7ff711ad2b80 1 -- 172.21.15.33:0/761219452 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55e470a78dc0 con 0x55e470a73580 2024-05-03T18:26:14.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.863+0000 7ff6ee000700 1 -- 172.21.15.33:0/761219452 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff6f8004a90 con 0x55e470a73580 2024-05-03T18:26:14.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.863+0000 7ff6ee000700 1 -- 172.21.15.33:0/761219452 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7ff6f000f460 con 0x55e470a73580 2024-05-03T18:26:14.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.863+0000 7ff711ad2b80 10 client.15024 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:26:14.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.863+0000 7ff711ad2b80 10 client.15024 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-03T18:26:14.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.863+0000 7ff711ad2b80 20 client.15024 populate_metadata read hostname 'smithi033' 2024-05-03T18:26:14.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.863+0000 7ff711ad2b80 1 --2- 172.21.15.33:0/761219452 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55e470b05b50 0x55e470b04860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:26:14.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.863+0000 7ff711ad2b80 4 client.15024 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-03T18:26:14.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.863+0000 7ff711ad2b80 1 -- 172.21.15.33:0/761219452 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "status"}) -- 0x55e4709d33b0 con 0x55e470b05b50 2024-05-03T18:26:14.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.864+0000 7ff6f7e00700 1 --2- 172.21.15.33:0/761219452 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55e470b05b50 0x55e470b04860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:26:14.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.864+0000 7ff6f7e00700 1 --2- 172.21.15.33:0/761219452 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55e470b05b50 0x55e470b04860 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c9cc8c40010c0d8 server_cookie=607e818c20ddc1b1 in_seq=0 out_seq=0 2024-05-03T18:26:14.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.864+0000 7ff6ee000700 10 client.15024 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:26:14.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.865+0000 7ff6ee000700 1 -- 172.21.15.33:0/761219452 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55e4709d33b0 con 0x55e470b05b50 2024-05-03T18:26:14.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.865+0000 7ff6ee000700 10 client.15024 handle_command_reply: tid=0 2024-05-03T18:26:14.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.865+0000 7ff711ad2b80 1 client.15024 shutdown 2024-05-03T18:26:14.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.865+0000 7ff711ad2b80 1 -- 172.21.15.33:0/761219452 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e470a73580 msgr2=0x55e470a96190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:26:14.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.865+0000 7ff711ad2b80 1 --2- 172.21.15.33:0/761219452 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e470a73580 0x55e470a96190 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ff6f0009990 tx=0x7ff6f00052e0 comp rx=0 tx=0).stop 2024-05-03T18:26:14.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.865+0000 7ff711ad2b80 1 -- 172.21.15.33:0/761219452 shutdown_connections 2024-05-03T18:26:14.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.865+0000 7ff711ad2b80 1 --2- 172.21.15.33:0/761219452 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55e470b05b50 0x55e470b04860 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:14.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.865+0000 7ff711ad2b80 1 --2- 172.21.15.33:0/761219452 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e470a78080 0x55e470a96c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:14.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.865+0000 7ff711ad2b80 1 --2- 172.21.15.33:0/761219452 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e470a73580 0x55e470a96190 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:14.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.865+0000 7ff711ad2b80 1 --2- 172.21.15.33:0/761219452 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e470a77760 0x55e470a966d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:14.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.865+0000 7ff711ad2b80 1 -- 172.21.15.33:0/761219452 >> 172.21.15.33:0/761219452 conn(0x55e4709a1b10 msgr2=0x55e470a88090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:26:14.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.866+0000 7ff6ee000700 0 client.15024 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:26:14.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.866+0000 7ff711ad2b80 1 -- 172.21.15.33:0/761219452 shutdown_connections 2024-05-03T18:26:14.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.866+0000 7ff711ad2b80 1 -- 172.21.15.33:0/761219452 wait complete. 2024-05-03T18:26:14.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.866+0000 7ff711ad2b80 20 client.15024 trim_cache size 0 max 16384 2024-05-03T18:26:14.868 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-03T18:26:14.868 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "5703bba4-0978-11ef-bc97-c7b262605968", 2024-05-03T18:26:14.868 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 0, 2024-05-03T18:26:14.868 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 34508, 2024-05-03T18:26:14.868 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2024-05-03T18:26:14.868 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2024-05-03T18:26:14.868 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2024-05-03T18:26:14.868 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 225.221544368, 2024-05-03T18:26:14.868 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 21, 2024-05-03T18:26:14.869 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 89, 2024-05-03T18:26:14.869 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 87, 2024-05-03T18:26:14.869 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 248.77863654399999 2024-05-03T18:26:14.869 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-03T18:26:14.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.866+0000 7ff707200700 1 -- 172.21.15.33:0/2306026523 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7ff6e40b2910 msgr2=0x7ff6e40b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:26:14.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.866+0000 7ff707200700 1 --2- 172.21.15.33:0/2306026523 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7ff6e40b2910 0x7ff6e40b4dd0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff6f801e000 tx=0x7ff6f801f000 comp rx=0 tx=0).stop 2024-05-03T18:26:14.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.866+0000 7ff707200700 1 -- 172.21.15.33:0/2306026523 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff700106090 msgr2=0x7ff7001a1f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:26:14.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.866+0000 7ff707200700 1 --2- 172.21.15.33:0/2306026523 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff700106090 0x7ff7001a1f80 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ff6e800e9c0 tx=0x7ff6e800ecd0 comp rx=0 tx=0).stop 2024-05-03T18:26:14.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.867+0000 7ff707200700 1 -- 172.21.15.33:0/2306026523 shutdown_connections 2024-05-03T18:26:14.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.867+0000 7ff707200700 1 --2- 172.21.15.33:0/2306026523 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7ff6e40b2910 0x7ff6e40b4dd0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:14.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.867+0000 7ff707200700 1 --2- 172.21.15.33:0/2306026523 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff70010b6d0 0x7ff70019bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:14.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.867+0000 7ff707200700 1 --2- 172.21.15.33:0/2306026523 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7001056d0 0x7ff7001a1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:14.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.867+0000 7ff707200700 1 --2- 172.21.15.33:0/2306026523 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff700106090 0x7ff7001a1f80 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:14.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.867+0000 7ff707200700 1 -- 172.21.15.33:0/2306026523 >> 172.21.15.33:0/2306026523 conn(0x7ff7001011a0 msgr2=0x7ff700101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:26:14.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.867+0000 7ff707200700 1 -- 172.21.15.33:0/2306026523 shutdown_connections 2024-05-03T18:26:14.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:14.867+0000 7ff707200700 1 -- 172.21.15.33:0/2306026523 wait complete. 2024-05-03T18:26:14.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:26:14 smithi150 ceph-mon[32492]: pgmap v496: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:26:15.830 INFO:tasks.cephadm:Setup complete, yielding 2024-05-03T18:26:15.831 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-03T18:26:15.841 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi033.front.sepia.ceph.com 2024-05-03T18:26:15.841 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- bash -c 'ceph orch status' 2024-05-03T18:26:16.295 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:26:16.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:26:16 smithi033 ceph-mon[28330]: pgmap v497: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:26:16.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:26:16 smithi099 ceph-mon[31658]: pgmap v497: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:26:16.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:26:16 smithi150 ceph-mon[32492]: pgmap v497: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:26:18.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:18.708+0000 7f9a97e00700 1 -- 172.21.15.33:0/2171998639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a98104620 msgr2=0x7f9a98104a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:26:18.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:18.708+0000 7f9a97e00700 1 --2- 172.21.15.33:0/2171998639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a98104620 0x7f9a98104a80 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f9a840099c0 tx=0x7f9a84009cd0 comp rx=0 tx=0).stop 2024-05-03T18:26:18.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:18.708+0000 7f9a97e00700 1 -- 172.21.15.33:0/2171998639 shutdown_connections 2024-05-03T18:26:18.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:18.708+0000 7f9a97e00700 1 --2- 172.21.15.33:0/2171998639 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a98104fc0 0x7f9a9810b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:18.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:18.708+0000 7f9a97e00700 1 --2- 172.21.15.33:0/2171998639 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a98103420 0x7f9a98103840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:18.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:18.708+0000 7f9a97e00700 1 --2- 172.21.15.33:0/2171998639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a98104620 0x7f9a98104a80 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:18.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:18.708+0000 7f9a97e00700 1 -- 172.21.15.33:0/2171998639 >> 172.21.15.33:0/2171998639 conn(0x7f9a980fe9e0 msgr2=0x7f9a98100e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:26:18.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:18.708+0000 7f9a97e00700 1 -- 172.21.15.33:0/2171998639 shutdown_connections 2024-05-03T18:26:18.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:18.708+0000 7f9a97e00700 1 -- 172.21.15.33:0/2171998639 wait complete. 2024-05-03T18:26:18.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:18.709+0000 7f9a97e00700 1 Processor -- start 2024-05-03T18:26:18.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:18.709+0000 7f9a97e00700 1 -- start start 2024-05-03T18:26:18.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:18.710+0000 7f9a97e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a98103420 0x7f9a9819b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:26:18.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:18.710+0000 7f9a97e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a98104620 0x7f9a9819bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:26:18.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:18.710+0000 7f9a97e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a98104fc0 0x7f9a981a2c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:26:18.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:18.710+0000 7f9a97e00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9a9810edc0 con 0x7f9a98103420 2024-05-03T18:26:18.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:18.710+0000 7f9a97e00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9a9810ec40 con 0x7f9a98104620 2024-05-03T18:26:18.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:18.710+0000 7f9a97e00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9a9810ef40 con 0x7f9a98104fc0 2024-05-03T18:26:18.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:18.710+0000 7f9a96a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a98103420 0x7f9a9819b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:26:18.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:18.710+0000 7f9a96a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a98103420 0x7f9a9819b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55546/0 (socket says 172.21.15.33:55546) 2024-05-03T18:26:18.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:18.710+0000 7f9a96a00700 1 -- 172.21.15.33:0/1825992791 learned_addr learned my addr 172.21.15.33:0/1825992791 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:26:18.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:18.710+0000 7f9a96000700 1 --2- 172.21.15.33:0/1825992791 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a98104620 0x7f9a9819bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:26:18.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:18.711+0000 7f9a97400700 1 --2- 172.21.15.33:0/1825992791 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a98104fc0 0x7f9a981a2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:26:18.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:18.711+0000 7f9a96a00700 1 -- 172.21.15.33:0/1825992791 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a98104fc0 msgr2=0x7f9a981a2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:26:18.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:18.711+0000 7f9a96a00700 1 --2- 172.21.15.33:0/1825992791 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a98104fc0 0x7f9a981a2c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:18.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:18.711+0000 7f9a96a00700 1 -- 172.21.15.33:0/1825992791 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a98104620 msgr2=0x7f9a9819bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:26:18.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:18.711+0000 7f9a96a00700 1 --2- 172.21.15.33:0/1825992791 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a98104620 0x7f9a9819bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:18.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:18.711+0000 7f9a96a00700 1 -- 172.21.15.33:0/1825992791 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a981a32d0 con 0x7f9a98103420 2024-05-03T18:26:18.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:18.711+0000 7f9a96a00700 1 --2- 172.21.15.33:0/1825992791 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a98103420 0x7f9a9819b670 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f9a8800d9b0 tx=0x7f9a8800dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:26:18.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:18.711+0000 7f9a8f400700 1 -- 172.21.15.33:0/1825992791 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a88017030 con 0x7f9a98103420 2024-05-03T18:26:18.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:18.711+0000 7f9a8f400700 1 -- 172.21.15.33:0/1825992791 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9a880044b0 con 0x7f9a98103420 2024-05-03T18:26:18.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:18.711+0000 7f9a97e00700 1 -- 172.21.15.33:0/1825992791 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a981a35c0 con 0x7f9a98103420 2024-05-03T18:26:18.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:18.712+0000 7f9a8f400700 1 -- 172.21.15.33:0/1825992791 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a880038e0 con 0x7f9a98103420 2024-05-03T18:26:18.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:18.712+0000 7f9a97e00700 1 -- 172.21.15.33:0/1825992791 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a981a3b00 con 0x7f9a98103420 2024-05-03T18:26:18.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:18.713+0000 7f9a97e00700 1 -- 172.21.15.33:0/1825992791 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a9806a190 con 0x7f9a98103420 2024-05-03T18:26:18.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:18.715+0000 7f9a8f400700 1 -- 172.21.15.33:0/1825992791 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f9a8801a070 con 0x7f9a98103420 2024-05-03T18:26:18.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:18.717+0000 7f9a8f400700 1 --2- 172.21.15.33:0/1825992791 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f9a7c0b2910 0x7f9a7c0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:26:18.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:18.717+0000 7f9a8f400700 1 -- 172.21.15.33:0/1825992791 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f9a880e2190 con 0x7f9a98103420 2024-05-03T18:26:18.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:18.717+0000 7f9a96000700 1 --2- 172.21.15.33:0/1825992791 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f9a7c0b2910 0x7f9a7c0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:26:18.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:18.718+0000 7f9a96000700 1 --2- 172.21.15.33:0/1825992791 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f9a7c0b2910 0x7f9a7c0b4dd0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9a84000c00 tx=0x7f9a8401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:26:18.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:18.721+0000 7f9a8f400700 1 -- 172.21.15.33:0/1825992791 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f9a880a9970 con 0x7f9a98103420 2024-05-03T18:26:18.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:26:18 smithi099 ceph-mon[31658]: pgmap v498: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:26:18.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:26:18 smithi150 ceph-mon[32492]: pgmap v498: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:26:18.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:18.975+0000 7f9a97e00700 1 -- 172.21.15.33:0/1825992791 --> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f9a981a3dc0 con 0x7f9a7c0b2910 2024-05-03T18:26:18.975 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:26:18 smithi033 ceph-mon[28330]: pgmap v498: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:26:18.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:18.978+0000 7f9a8f400700 1 -- 172.21.15.33:0/1825992791 <== mgr.14152 v2:172.21.15.33:6800/3642895501 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f9a981a3dc0 con 0x7f9a7c0b2910 2024-05-03T18:26:18.978 INFO:teuthology.orchestra.run.smithi033.stdout:Backend: cephadm 2024-05-03T18:26:18.978 INFO:teuthology.orchestra.run.smithi033.stdout:Available: Yes 2024-05-03T18:26:18.978 INFO:teuthology.orchestra.run.smithi033.stdout:Paused: No 2024-05-03T18:26:18.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:18.981+0000 7f9a97e00700 1 -- 172.21.15.33:0/1825992791 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f9a7c0b2910 msgr2=0x7f9a7c0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:26:18.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:18.981+0000 7f9a97e00700 1 --2- 172.21.15.33:0/1825992791 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f9a7c0b2910 0x7f9a7c0b4dd0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9a84000c00 tx=0x7f9a8401a040 comp rx=0 tx=0).stop 2024-05-03T18:26:18.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:18.981+0000 7f9a97e00700 1 -- 172.21.15.33:0/1825992791 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a98103420 msgr2=0x7f9a9819b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:26:18.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:18.981+0000 7f9a97e00700 1 --2- 172.21.15.33:0/1825992791 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a98103420 0x7f9a9819b670 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f9a8800d9b0 tx=0x7f9a8800dd70 comp rx=0 tx=0).stop 2024-05-03T18:26:18.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:18.981+0000 7f9a97e00700 1 -- 172.21.15.33:0/1825992791 shutdown_connections 2024-05-03T18:26:18.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:18.981+0000 7f9a97e00700 1 --2- 172.21.15.33:0/1825992791 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a98104fc0 0x7f9a981a2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:18.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:18.981+0000 7f9a97e00700 1 --2- 172.21.15.33:0/1825992791 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a98103420 0x7f9a9819b670 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:18.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:18.981+0000 7f9a97e00700 1 --2- 172.21.15.33:0/1825992791 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f9a7c0b2910 0x7f9a7c0b4dd0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:18.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:18.981+0000 7f9a97e00700 1 --2- 172.21.15.33:0/1825992791 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a98104620 0x7f9a9819bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:18.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:18.981+0000 7f9a97e00700 1 -- 172.21.15.33:0/1825992791 >> 172.21.15.33:0/1825992791 conn(0x7f9a980fe9e0 msgr2=0x7f9a98109e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:26:18.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:18.982+0000 7f9a97e00700 1 -- 172.21.15.33:0/1825992791 shutdown_connections 2024-05-03T18:26:18.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:18.982+0000 7f9a97e00700 1 -- 172.21.15.33:0/1825992791 wait complete. 2024-05-03T18:26:19.829 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- bash -c 'ceph orch ps' 2024-05-03T18:26:20.305 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:26:20.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:26:20 smithi033 ceph-mon[28330]: from='client.15030 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:26:20.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:26:20 smithi033 ceph-mon[28330]: pgmap v499: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:26:20.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:26:20 smithi150 ceph-mon[32492]: from='client.15030 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:26:20.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:26:20 smithi150 ceph-mon[32492]: pgmap v499: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:26:21.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:26:20 smithi099 ceph-mon[31658]: from='client.15030 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:26:21.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:26:20 smithi099 ceph-mon[31658]: pgmap v499: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:26:22.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:26:22 smithi033 ceph-mon[28330]: pgmap v500: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:26:22.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:22.757+0000 7f777ee00700 1 -- 172.21.15.33:0/2466074508 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77781050d0 msgr2=0x7f777810b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:26:22.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:22.757+0000 7f777ee00700 1 --2- 172.21.15.33:0/2466074508 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77781050d0 0x7f777810b980 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f777000b0d0 tx=0x7f777000b3e0 comp rx=0 tx=0).stop 2024-05-03T18:26:22.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:22.757+0000 7f777ee00700 1 -- 172.21.15.33:0/2466074508 shutdown_connections 2024-05-03T18:26:22.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:22.757+0000 7f777ee00700 1 --2- 172.21.15.33:0/2466074508 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77781050d0 0x7f777810b980 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:22.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:22.757+0000 7f777ee00700 1 --2- 172.21.15.33:0/2466074508 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7778103530 0x7f7778103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:22.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:22.757+0000 7f777ee00700 1 --2- 172.21.15.33:0/2466074508 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7778104730 0x7f7778104b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:22.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:22.757+0000 7f777ee00700 1 -- 172.21.15.33:0/2466074508 >> 172.21.15.33:0/2466074508 conn(0x7f77780feaf0 msgr2=0x7f7778100f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:26:22.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:22.757+0000 7f777ee00700 1 -- 172.21.15.33:0/2466074508 shutdown_connections 2024-05-03T18:26:22.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:22.757+0000 7f777ee00700 1 -- 172.21.15.33:0/2466074508 wait complete. 2024-05-03T18:26:22.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:22.758+0000 7f777ee00700 1 Processor -- start 2024-05-03T18:26:22.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:22.758+0000 7f777ee00700 1 -- start start 2024-05-03T18:26:22.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:22.759+0000 7f777ee00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7778103530 0x7f777819b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:26:22.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:22.759+0000 7f777ee00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7778104730 0x7f777819bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:26:22.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:22.759+0000 7f777ee00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77781050d0 0x7f77781a2d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:26:22.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:22.759+0000 7f777ee00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f777810ee80 con 0x7f77781050d0 2024-05-03T18:26:22.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:22.759+0000 7f777ee00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f777810ed00 con 0x7f7778103530 2024-05-03T18:26:22.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:22.759+0000 7f777ee00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f777810f000 con 0x7f7778104730 2024-05-03T18:26:22.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:22.759+0000 7f777ce00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77781050d0 0x7f77781a2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:26:22.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:22.759+0000 7f777ce00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77781050d0 0x7f77781a2d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55572/0 (socket says 172.21.15.33:55572) 2024-05-03T18:26:22.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:22.759+0000 7f777ce00700 1 -- 172.21.15.33:0/1948802936 learned_addr learned my addr 172.21.15.33:0/1948802936 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:26:22.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:22.759+0000 7f7777e00700 1 --2- 172.21.15.33:0/1948802936 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7778103530 0x7f777819b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:26:22.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:22.759+0000 7f776f400700 1 --2- 172.21.15.33:0/1948802936 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7778104730 0x7f777819bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:26:22.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:22.760+0000 7f777ce00700 1 -- 172.21.15.33:0/1948802936 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7778104730 msgr2=0x7f777819bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:26:22.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:22.760+0000 7f777ce00700 1 --2- 172.21.15.33:0/1948802936 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7778104730 0x7f777819bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:22.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:22.760+0000 7f777ce00700 1 -- 172.21.15.33:0/1948802936 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7778103530 msgr2=0x7f777819b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:26:22.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:22.760+0000 7f777ce00700 1 --2- 172.21.15.33:0/1948802936 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7778103530 0x7f777819b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:22.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:22.760+0000 7f777ce00700 1 -- 172.21.15.33:0/1948802936 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77781a33d0 con 0x7f77781050d0 2024-05-03T18:26:22.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:22.760+0000 7f777ce00700 1 --2- 172.21.15.33:0/1948802936 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77781050d0 0x7f77781a2d30 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f777000d000 tx=0x7f7770009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:26:22.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:22.760+0000 7f7775600700 1 -- 172.21.15.33:0/1948802936 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f777001f030 con 0x7f77781050d0 2024-05-03T18:26:22.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:22.761+0000 7f7775600700 1 -- 172.21.15.33:0/1948802936 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7770005040 con 0x7f77781050d0 2024-05-03T18:26:22.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:22.761+0000 7f777ee00700 1 -- 172.21.15.33:0/1948802936 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77781a3660 con 0x7f77781050d0 2024-05-03T18:26:22.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:22.761+0000 7f7775600700 1 -- 172.21.15.33:0/1948802936 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f777001b690 con 0x7f77781050d0 2024-05-03T18:26:22.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:22.761+0000 7f777ee00700 1 -- 172.21.15.33:0/1948802936 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77781a3b10 con 0x7f77781050d0 2024-05-03T18:26:22.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:22.762+0000 7f777ee00700 1 -- 172.21.15.33:0/1948802936 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f777806a190 con 0x7f77781050d0 2024-05-03T18:26:22.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:22.766+0000 7f7775600700 1 -- 172.21.15.33:0/1948802936 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f7770022030 con 0x7f77781050d0 2024-05-03T18:26:22.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:22.767+0000 7f7775600700 1 --2- 172.21.15.33:0/1948802936 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f775c0b2990 0x7f775c0b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:26:22.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:22.768+0000 7f7775600700 1 -- 172.21.15.33:0/1948802936 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f777000f020 con 0x7f77781050d0 2024-05-03T18:26:22.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:22.768+0000 7f7777e00700 1 --2- 172.21.15.33:0/1948802936 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f775c0b2990 0x7f775c0b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:26:22.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:22.769+0000 7f7777e00700 1 --2- 172.21.15.33:0/1948802936 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f775c0b2990 0x7f775c0b4e50 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7778104590 tx=0x7f776800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:26:22.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:22.770+0000 7f7775600700 1 -- 172.21.15.33:0/1948802936 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f7770029020 con 0x7f77781050d0 2024-05-03T18:26:22.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:26:22 smithi150 ceph-mon[32492]: pgmap v500: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:26:23.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:23.022+0000 7f777ee00700 1 -- 172.21.15.33:0/1948802936 --> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f77781a3e30 con 0x7f775c0b2990 2024-05-03T18:26:23.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:26:22 smithi099 ceph-mon[31658]: pgmap v500: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:26:23.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:23.064+0000 7f7775600700 1 -- 172.21.15.33:0/1948802936 <== mgr.14152 v2:172.21.15.33:6800/3642895501 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7f77781a3e30 con 0x7f775c0b2990 2024-05-03T18:26:23.065 INFO:teuthology.orchestra.run.smithi033.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-03T18:26:23.065 INFO:teuthology.orchestra.run.smithi033.stdout:mds.a smithi033 running (4m) 113s ago 4m 12.9M - 19.0.0-3450-g9a047fe7 ee4d84620249 dfbdd4791c70 2024-05-03T18:26:23.065 INFO:teuthology.orchestra.run.smithi033.stdout:mds.b smithi099 running (4m) 115s ago 4m 24.6M - 19.0.0-3450-g9a047fe7 ee4d84620249 0372eed14b6b 2024-05-03T18:26:23.066 INFO:teuthology.orchestra.run.smithi033.stdout:mds.c smithi150 running (4m) 115s ago 4m 13.2M - 19.0.0-3450-g9a047fe7 ee4d84620249 78f83371523d 2024-05-03T18:26:23.066 INFO:teuthology.orchestra.run.smithi033.stdout:mds.d smithi033 running (4m) 113s ago 4m 13.2M - 19.0.0-3450-g9a047fe7 ee4d84620249 8821c4cab9ce 2024-05-03T18:26:23.066 INFO:teuthology.orchestra.run.smithi033.stdout:mds.e smithi099 running (4m) 115s ago 4m 13.1M - 19.0.0-3450-g9a047fe7 ee4d84620249 827d8341c3f9 2024-05-03T18:26:23.066 INFO:teuthology.orchestra.run.smithi033.stdout:mds.f smithi150 running (4m) 115s ago 4m 13.0M - 19.0.0-3450-g9a047fe7 ee4d84620249 2c9f0e84d96e 2024-05-03T18:26:23.066 INFO:teuthology.orchestra.run.smithi033.stdout:mds.g smithi033 running (4m) 113s ago 4m 15.1M - 19.0.0-3450-g9a047fe7 ee4d84620249 518816162c38 2024-05-03T18:26:23.066 INFO:teuthology.orchestra.run.smithi033.stdout:mds.h smithi099 running (4m) 115s ago 4m 16.5M - 19.0.0-3450-g9a047fe7 ee4d84620249 08471d7f10c4 2024-05-03T18:26:23.066 INFO:teuthology.orchestra.run.smithi033.stdout:mds.i smithi150 running (4m) 115s ago 4m 13.1M - 19.0.0-3450-g9a047fe7 ee4d84620249 ed43a8474889 2024-05-03T18:26:23.066 INFO:teuthology.orchestra.run.smithi033.stdout:mds.j smithi033 running (4m) 113s ago 4m 13.2M - 19.0.0-3450-g9a047fe7 ee4d84620249 85f42a639f42 2024-05-03T18:26:23.066 INFO:teuthology.orchestra.run.smithi033.stdout:mds.k smithi099 running (5m) 115s ago 5m 13.2M - 19.0.0-3450-g9a047fe7 ee4d84620249 d1fe17b0ec71 2024-05-03T18:26:23.066 INFO:teuthology.orchestra.run.smithi033.stdout:mds.l smithi150 running (5m) 115s ago 5m 13.3M - 19.0.0-3450-g9a047fe7 ee4d84620249 14f4381f2694 2024-05-03T18:26:23.066 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.x smithi033 *:9283,8765 running (15m) 113s ago 15m 531M - 19.0.0-3450-g9a047fe7 ee4d84620249 baf6dc704ff5 2024-05-03T18:26:23.066 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.y smithi099 *:8443,8765 running (12m) 115s ago 12m 466M - 19.0.0-3450-g9a047fe7 ee4d84620249 028f4cdf6284 2024-05-03T18:26:23.066 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.z smithi150 *:8443,8765 running (12m) 115s ago 12m 465M - 19.0.0-3450-g9a047fe7 ee4d84620249 aa8e47cd8dde 2024-05-03T18:26:23.067 INFO:teuthology.orchestra.run.smithi033.stdout:mon.a smithi033 running (15m) 113s ago 15m 50.7M 2048M 19.0.0-3450-g9a047fe7 ee4d84620249 b6180ce9a1ab 2024-05-03T18:26:23.067 INFO:teuthology.orchestra.run.smithi033.stdout:mon.b smithi099 running (12m) 115s ago 12m 39.8M 2048M 19.0.0-3450-g9a047fe7 ee4d84620249 7fd86560ad99 2024-05-03T18:26:23.067 INFO:teuthology.orchestra.run.smithi033.stdout:mon.c smithi150 running (12m) 115s ago 12m 40.3M 2048M 19.0.0-3450-g9a047fe7 ee4d84620249 a45cbbe8663d 2024-05-03T18:26:23.067 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0 smithi033 running (12m) 113s ago 12m 44.0M 4273M 19.0.0-3450-g9a047fe7 ee4d84620249 9fc394357a1c 2024-05-03T18:26:23.067 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1 smithi099 running (11m) 115s ago 11m 53.1M 4272M 19.0.0-3450-g9a047fe7 ee4d84620249 1ee7cabe9359 2024-05-03T18:26:23.067 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2 smithi150 running (10m) 115s ago 10m 52.8M 4272M 19.0.0-3450-g9a047fe7 ee4d84620249 c200ebbca765 2024-05-03T18:26:23.067 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3 smithi033 running (10m) 113s ago 10m 44.1M 4273M 19.0.0-3450-g9a047fe7 ee4d84620249 a18ce44dad59 2024-05-03T18:26:23.067 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4 smithi099 running (9m) 115s ago 9m 42.0M 4272M 19.0.0-3450-g9a047fe7 ee4d84620249 64303dee81d8 2024-05-03T18:26:23.067 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5 smithi150 running (9m) 115s ago 9m 51.5M 4272M 19.0.0-3450-g9a047fe7 ee4d84620249 87bc6b436276 2024-05-03T18:26:23.067 INFO:teuthology.orchestra.run.smithi033.stdout:osd.6 smithi033 running (8m) 113s ago 8m 41.1M 4273M 19.0.0-3450-g9a047fe7 ee4d84620249 01db5bde5c9b 2024-05-03T18:26:23.067 INFO:teuthology.orchestra.run.smithi033.stdout:osd.7 smithi099 running (7m) 115s ago 7m 52.2M 4272M 19.0.0-3450-g9a047fe7 ee4d84620249 90cd82296c0e 2024-05-03T18:26:23.067 INFO:teuthology.orchestra.run.smithi033.stdout:osd.8 smithi150 running (7m) 115s ago 7m 42.8M 4272M 19.0.0-3450-g9a047fe7 ee4d84620249 16783a1beef8 2024-05-03T18:26:23.067 INFO:teuthology.orchestra.run.smithi033.stdout:osd.9 smithi033 running (6m) 113s ago 6m 49.9M 4273M 19.0.0-3450-g9a047fe7 ee4d84620249 2b58646c996a 2024-05-03T18:26:23.067 INFO:teuthology.orchestra.run.smithi033.stdout:osd.10 smithi099 running (6m) 115s ago 6m 42.6M 4272M 19.0.0-3450-g9a047fe7 ee4d84620249 4a52b3693de2 2024-05-03T18:26:23.067 INFO:teuthology.orchestra.run.smithi033.stdout:osd.11 smithi150 running (5m) 115s ago 5m 48.3M 4272M 19.0.0-3450-g9a047fe7 ee4d84620249 d6f67eb3ce89 2024-05-03T18:26:23.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:23.068+0000 7f777ee00700 1 -- 172.21.15.33:0/1948802936 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f775c0b2990 msgr2=0x7f775c0b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:26:23.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:23.068+0000 7f777ee00700 1 --2- 172.21.15.33:0/1948802936 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f775c0b2990 0x7f775c0b4e50 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7778104590 tx=0x7f776800b410 comp rx=0 tx=0).stop 2024-05-03T18:26:23.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:23.068+0000 7f777ee00700 1 -- 172.21.15.33:0/1948802936 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77781050d0 msgr2=0x7f77781a2d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:26:23.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:23.068+0000 7f777ee00700 1 --2- 172.21.15.33:0/1948802936 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77781050d0 0x7f77781a2d30 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f777000d000 tx=0x7f7770009b80 comp rx=0 tx=0).stop 2024-05-03T18:26:23.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:23.068+0000 7f777ee00700 1 -- 172.21.15.33:0/1948802936 shutdown_connections 2024-05-03T18:26:23.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:23.069+0000 7f777ee00700 1 --2- 172.21.15.33:0/1948802936 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77781050d0 0x7f77781a2d30 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:23.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:23.069+0000 7f777ee00700 1 --2- 172.21.15.33:0/1948802936 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f775c0b2990 0x7f775c0b4e50 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:23.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:23.069+0000 7f777ee00700 1 --2- 172.21.15.33:0/1948802936 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7778103530 0x7f777819b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:23.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:23.069+0000 7f777ee00700 1 --2- 172.21.15.33:0/1948802936 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7778104730 0x7f777819bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:23.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:23.069+0000 7f777ee00700 1 -- 172.21.15.33:0/1948802936 >> 172.21.15.33:0/1948802936 conn(0x7f77780feaf0 msgr2=0x7f77780756d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:26:23.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:23.069+0000 7f777ee00700 1 -- 172.21.15.33:0/1948802936 shutdown_connections 2024-05-03T18:26:23.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:23.069+0000 7f777ee00700 1 -- 172.21.15.33:0/1948802936 wait complete. 2024-05-03T18:26:23.969 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- bash -c 'ceph orch ls' 2024-05-03T18:26:24.445 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:26:24.813 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:26:24 smithi033 ceph-mon[28330]: from='client.15036 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:26:24.813 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:26:24 smithi033 ceph-mon[28330]: pgmap v501: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:26:24.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:26:24 smithi150 ceph-mon[32492]: from='client.15036 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:26:24.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:26:24 smithi150 ceph-mon[32492]: pgmap v501: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:26:25.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:26:24 smithi099 ceph-mon[31658]: from='client.15036 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:26:25.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:26:24 smithi099 ceph-mon[31658]: pgmap v501: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:26:26.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:26.753+0000 7f5f93400700 1 -- 172.21.15.33:0/3543717557 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f8c104050 msgr2=0x7f5f8c106440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:26:26.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:26.753+0000 7f5f93400700 1 --2- 172.21.15.33:0/3543717557 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f8c104050 0x7f5f8c106440 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c009990 tx=0x7f5f7c009ca0 comp rx=0 tx=0).stop 2024-05-03T18:26:26.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:26.753+0000 7f5f93400700 1 -- 172.21.15.33:0/3543717557 shutdown_connections 2024-05-03T18:26:26.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:26.753+0000 7f5f93400700 1 --2- 172.21.15.33:0/3543717557 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f8c1092b0 0x7f5f8c10b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:26.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:26.753+0000 7f5f93400700 1 --2- 172.21.15.33:0/3543717557 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f8c104050 0x7f5f8c106440 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:26.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:26.753+0000 7f5f93400700 1 --2- 172.21.15.33:0/3543717557 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f8c106980 0x7f5f8c108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:26.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:26.753+0000 7f5f93400700 1 -- 172.21.15.33:0/3543717557 >> 172.21.15.33:0/3543717557 conn(0x7f5f8c0fd9a0 msgr2=0x7f5f8c0ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:26:26.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:26.753+0000 7f5f93400700 1 -- 172.21.15.33:0/3543717557 shutdown_connections 2024-05-03T18:26:26.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:26.753+0000 7f5f93400700 1 -- 172.21.15.33:0/3543717557 wait complete. 2024-05-03T18:26:26.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:26.754+0000 7f5f93400700 1 Processor -- start 2024-05-03T18:26:26.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:26.754+0000 7f5f93400700 1 -- start start 2024-05-03T18:26:26.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:26.755+0000 7f5f93400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f8c104050 0x7f5f8c19b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:26:26.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:26.755+0000 7f5f93400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f8c106980 0x7f5f8c19bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:26:26.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:26.755+0000 7f5f93400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f8c1092b0 0x7f5f8c1a2df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:26:26.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:26.755+0000 7f5f93400700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5f8c10ee50 con 0x7f5f8c104050 2024-05-03T18:26:26.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:26.755+0000 7f5f93400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5f8c10ecd0 con 0x7f5f8c1092b0 2024-05-03T18:26:26.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:26.755+0000 7f5f93400700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f5f8c10efd0 con 0x7f5f8c106980 2024-05-03T18:26:26.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:26.755+0000 7f5f90a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f8c104050 0x7f5f8c19b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:26:26.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:26.755+0000 7f5f90a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f8c104050 0x7f5f8c19b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52166/0 (socket says 172.21.15.33:52166) 2024-05-03T18:26:26.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:26.755+0000 7f5f90a00700 1 -- 172.21.15.33:0/2439420685 learned_addr learned my addr 172.21.15.33:0/2439420685 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:26:26.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:26.755+0000 7f5f91400700 1 --2- 172.21.15.33:0/2439420685 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f8c1092b0 0x7f5f8c1a2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:26:26.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:26.755+0000 7f5f83e00700 1 --2- 172.21.15.33:0/2439420685 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f8c106980 0x7f5f8c19bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:26:26.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:26.755+0000 7f5f90a00700 1 -- 172.21.15.33:0/2439420685 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f8c106980 msgr2=0x7f5f8c19bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:26:26.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:26.755+0000 7f5f90a00700 1 --2- 172.21.15.33:0/2439420685 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f8c106980 0x7f5f8c19bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:26.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:26.755+0000 7f5f90a00700 1 -- 172.21.15.33:0/2439420685 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f8c1092b0 msgr2=0x7f5f8c1a2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:26:26.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:26.755+0000 7f5f90a00700 1 --2- 172.21.15.33:0/2439420685 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f8c1092b0 0x7f5f8c1a2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:26.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:26.756+0000 7f5f90a00700 1 -- 172.21.15.33:0/2439420685 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f8c1a3490 con 0x7f5f8c104050 2024-05-03T18:26:26.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:26.756+0000 7f5f83e00700 1 --2- 172.21.15.33:0/2439420685 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f8c106980 0x7f5f8c19bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:26:26.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:26.756+0000 7f5f91400700 1 --2- 172.21.15.33:0/2439420685 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f8c1092b0 0x7f5f8c1a2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:26:26.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:26.756+0000 7f5f90a00700 1 --2- 172.21.15.33:0/2439420685 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f8c104050 0x7f5f8c19b830 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c0057c0 tx=0x7f5f7c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:26:26.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:26.756+0000 7f5f8a000700 1 -- 172.21.15.33:0/2439420685 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f7c027070 con 0x7f5f8c104050 2024-05-03T18:26:26.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:26.756+0000 7f5f8a000700 1 -- 172.21.15.33:0/2439420685 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5f7c022470 con 0x7f5f8c104050 2024-05-03T18:26:26.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:26.756+0000 7f5f93400700 1 -- 172.21.15.33:0/2439420685 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f8c1a3720 con 0x7f5f8c104050 2024-05-03T18:26:26.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:26.756+0000 7f5f8a000700 1 -- 172.21.15.33:0/2439420685 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f7c00f6c0 con 0x7f5f8c104050 2024-05-03T18:26:26.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:26.757+0000 7f5f93400700 1 -- 172.21.15.33:0/2439420685 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f8c1a3bd0 con 0x7f5f8c104050 2024-05-03T18:26:26.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:26.758+0000 7f5f93400700 1 -- 172.21.15.33:0/2439420685 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f8c06a190 con 0x7f5f8c104050 2024-05-03T18:26:26.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:26.760+0000 7f5f8a000700 1 -- 172.21.15.33:0/2439420685 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f5f7c02a070 con 0x7f5f8c104050 2024-05-03T18:26:26.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:26.762+0000 7f5f8a000700 1 --2- 172.21.15.33:0/2439420685 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f5f700b2990 0x7f5f700b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:26:26.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:26.762+0000 7f5f8a000700 1 -- 172.21.15.33:0/2439420685 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f5f7c0e3a90 con 0x7f5f8c104050 2024-05-03T18:26:26.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:26.762+0000 7f5f83e00700 1 --2- 172.21.15.33:0/2439420685 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f5f700b2990 0x7f5f700b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:26:26.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:26.763+0000 7f5f83e00700 1 --2- 172.21.15.33:0/2439420685 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f5f700b2990 0x7f5f700b4e50 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5f74005fd0 tx=0x7f5f74005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:26:26.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:26.764+0000 7f5f8a000700 1 -- 172.21.15.33:0/2439420685 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f5f7c031080 con 0x7f5f8c104050 2024-05-03T18:26:26.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:26:26 smithi150 ceph-mon[32492]: pgmap v502: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:26:26.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:26:26 smithi033 ceph-mon[28330]: pgmap v502: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:26:27.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:27.011+0000 7f5f93400700 1 -- 172.21.15.33:0/2439420685 --> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f5f8c102770 con 0x7f5f700b2990 2024-05-03T18:26:27.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:27.029+0000 7f5f8a000700 1 -- 172.21.15.33:0/2439420685 <== mgr.14152 v2:172.21.15.33:6800/3642895501 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f5f8c102770 con 0x7f5f700b2990 2024-05-03T18:26:27.029 INFO:teuthology.orchestra.run.smithi033.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-05-03T18:26:27.029 INFO:teuthology.orchestra.run.smithi033.stdout:mds.all 12/12 119s ago 5m smithi033=a;smithi033=d;smithi033=g;smithi033=j;smithi099=b;smithi099=e;smithi099=h;smithi099=k;smithi150=c;smithi150=f;smithi150=i;smithi150=l;count:12 2024-05-03T18:26:27.030 INFO:teuthology.orchestra.run.smithi033.stdout:mgr 3/3 119s ago 12m smithi033=x;smithi099=y;smithi150=z;count:3 2024-05-03T18:26:27.030 INFO:teuthology.orchestra.run.smithi033.stdout:mon 3/3 119s ago 13m smithi033:172.21.15.33=a;smithi099:172.21.15.99=b;smithi150:172.21.15.150=c;count:3 2024-05-03T18:26:27.030 INFO:teuthology.orchestra.run.smithi033.stdout:osd 12 119s ago - 2024-05-03T18:26:27.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:27.032+0000 7f5f93400700 1 -- 172.21.15.33:0/2439420685 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f5f700b2990 msgr2=0x7f5f700b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:26:27.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:27.032+0000 7f5f93400700 1 --2- 172.21.15.33:0/2439420685 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f5f700b2990 0x7f5f700b4e50 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5f74005fd0 tx=0x7f5f74005dc0 comp rx=0 tx=0).stop 2024-05-03T18:26:27.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:27.032+0000 7f5f93400700 1 -- 172.21.15.33:0/2439420685 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f8c104050 msgr2=0x7f5f8c19b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:26:27.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:27.032+0000 7f5f93400700 1 --2- 172.21.15.33:0/2439420685 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f8c104050 0x7f5f8c19b830 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c0057c0 tx=0x7f5f7c009f90 comp rx=0 tx=0).stop 2024-05-03T18:26:27.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:27.032+0000 7f5f93400700 1 -- 172.21.15.33:0/2439420685 shutdown_connections 2024-05-03T18:26:27.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:27.032+0000 7f5f93400700 1 --2- 172.21.15.33:0/2439420685 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f5f700b2990 0x7f5f700b4e50 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:27.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:27.032+0000 7f5f93400700 1 --2- 172.21.15.33:0/2439420685 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f8c1092b0 0x7f5f8c1a2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:27.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:27.032+0000 7f5f93400700 1 --2- 172.21.15.33:0/2439420685 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f8c104050 0x7f5f8c19b830 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:27.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:27.032+0000 7f5f93400700 1 --2- 172.21.15.33:0/2439420685 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f8c106980 0x7f5f8c19bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:27.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:27.032+0000 7f5f93400700 1 -- 172.21.15.33:0/2439420685 >> 172.21.15.33:0/2439420685 conn(0x7f5f8c0fd9a0 msgr2=0x7f5f8c102060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:26:27.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:27.033+0000 7f5f93400700 1 -- 172.21.15.33:0/2439420685 shutdown_connections 2024-05-03T18:26:27.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:27.033+0000 7f5f93400700 1 -- 172.21.15.33:0/2439420685 wait complete. 2024-05-03T18:26:27.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:26:26 smithi099 ceph-mon[31658]: pgmap v502: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:26:28.001 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- bash -c 'ceph orch host ls' 2024-05-03T18:26:28.471 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:26:28.856 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:26:28 smithi033 ceph-mon[28330]: from='client.15042 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:26:28.856 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:26:28 smithi033 ceph-mon[28330]: pgmap v503: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:26:28.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:26:28 smithi150 ceph-mon[32492]: from='client.15042 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:26:28.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:26:28 smithi150 ceph-mon[32492]: pgmap v503: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:26:29.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:26:28 smithi099 ceph-mon[31658]: from='client.15042 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:26:29.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:26:28 smithi099 ceph-mon[31658]: pgmap v503: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:26:30.613 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:26:30 smithi033 ceph-mon[28330]: pgmap v504: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:26:30.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:26:30 smithi150 ceph-mon[32492]: pgmap v504: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:26:30.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:30.895+0000 7f3366000700 1 -- 172.21.15.33:0/3386241145 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33601072f0 msgr2=0x7f336010dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:26:30.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:30.895+0000 7f3366000700 1 --2- 172.21.15.33:0/3386241145 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33601072f0 0x7f336010dba0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f335800b0d0 tx=0x7f335800b3e0 comp rx=0 tx=0).stop 2024-05-03T18:26:30.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:30.895+0000 7f3366000700 1 -- 172.21.15.33:0/3386241145 shutdown_connections 2024-05-03T18:26:30.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:30.895+0000 7f3366000700 1 --2- 172.21.15.33:0/3386241145 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33601072f0 0x7f336010dba0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:30.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:30.895+0000 7f3366000700 1 --2- 172.21.15.33:0/3386241145 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3360105750 0x7f3360105b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:30.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:30.895+0000 7f3366000700 1 --2- 172.21.15.33:0/3386241145 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3360106950 0x7f3360106db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:30.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:30.895+0000 7f3366000700 1 -- 172.21.15.33:0/3386241145 >> 172.21.15.33:0/3386241145 conn(0x7f3360100cf0 msgr2=0x7f3360103130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:26:30.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:30.895+0000 7f3366000700 1 -- 172.21.15.33:0/3386241145 shutdown_connections 2024-05-03T18:26:30.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:30.895+0000 7f3366000700 1 -- 172.21.15.33:0/3386241145 wait complete. 2024-05-03T18:26:30.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:30.896+0000 7f3366000700 1 Processor -- start 2024-05-03T18:26:30.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:30.897+0000 7f3366000700 1 -- start start 2024-05-03T18:26:30.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:30.897+0000 7f3366000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3360105750 0x7f336019b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:26:30.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:30.897+0000 7f3366000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3360106950 0x7f336019bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:26:30.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:30.897+0000 7f3366000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33601072f0 0x7f33601a2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:26:30.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:30.897+0000 7f3366000700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3360111080 con 0x7f33601072f0 2024-05-03T18:26:30.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:30.897+0000 7f3366000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3360110f00 con 0x7f3360106950 2024-05-03T18:26:30.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:30.897+0000 7f3366000700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3360111200 con 0x7f3360105750 2024-05-03T18:26:30.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:30.898+0000 7f335fe00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33601072f0 0x7f33601a2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:26:30.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:30.898+0000 7f335fe00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33601072f0 0x7f33601a2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52198/0 (socket says 172.21.15.33:52198) 2024-05-03T18:26:30.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:30.898+0000 7f335fe00700 1 -- 172.21.15.33:0/1807644380 learned_addr learned my addr 172.21.15.33:0/1807644380 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:26:30.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:30.898+0000 7f335f400700 1 --2- 172.21.15.33:0/1807644380 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3360105750 0x7f336019b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:26:30.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:30.898+0000 7f335fe00700 1 -- 172.21.15.33:0/1807644380 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3360105750 msgr2=0x7f336019b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:26:30.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:30.898+0000 7f335fe00700 1 --2- 172.21.15.33:0/1807644380 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3360105750 0x7f336019b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:30.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:30.898+0000 7f335fe00700 1 -- 172.21.15.33:0/1807644380 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3360106950 msgr2=0x7f336019bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:26:30.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:30.898+0000 7f335fe00700 1 --2- 172.21.15.33:0/1807644380 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3360106950 0x7f336019bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:30.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:30.898+0000 7f335fe00700 1 -- 172.21.15.33:0/1807644380 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33601a3350 con 0x7f33601072f0 2024-05-03T18:26:30.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:30.898+0000 7f335fe00700 1 --2- 172.21.15.33:0/1807644380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33601072f0 0x7f33601a2cb0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f3358009010 tx=0x7f3358009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:26:30.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:30.899+0000 7f3357e00700 1 -- 172.21.15.33:0/1807644380 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f335801f070 con 0x7f33601072f0 2024-05-03T18:26:30.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:30.899+0000 7f3357e00700 1 -- 172.21.15.33:0/1807644380 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f33580054d0 con 0x7f33601072f0 2024-05-03T18:26:30.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:30.899+0000 7f3366000700 1 -- 172.21.15.33:0/1807644380 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33601a35e0 con 0x7f33601072f0 2024-05-03T18:26:30.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:30.899+0000 7f3357e00700 1 -- 172.21.15.33:0/1807644380 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f335801b7f0 con 0x7f33601072f0 2024-05-03T18:26:30.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:30.899+0000 7f3366000700 1 -- 172.21.15.33:0/1807644380 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33601a3a30 con 0x7f33601072f0 2024-05-03T18:26:30.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:30.900+0000 7f3366000700 1 -- 172.21.15.33:0/1807644380 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f336006a190 con 0x7f33601072f0 2024-05-03T18:26:30.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:30.903+0000 7f3357e00700 1 -- 172.21.15.33:0/1807644380 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f3358022070 con 0x7f33601072f0 2024-05-03T18:26:30.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:30.904+0000 7f3357e00700 1 --2- 172.21.15.33:0/1807644380 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f33440b2990 0x7f33440b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:26:30.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:30.905+0000 7f3357e00700 1 -- 172.21.15.33:0/1807644380 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f33580e3a40 con 0x7f33601072f0 2024-05-03T18:26:30.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:30.905+0000 7f335f400700 1 --2- 172.21.15.33:0/1807644380 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f33440b2990 0x7f33440b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:26:30.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:30.906+0000 7f335f400700 1 --2- 172.21.15.33:0/1807644380 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f33440b2990 0x7f33440b4e50 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3350005950 tx=0x7f33500058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:26:30.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:30.908+0000 7f3357e00700 1 -- 172.21.15.33:0/1807644380 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f33580ab2b0 con 0x7f33601072f0 2024-05-03T18:26:31.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:26:30 smithi099 ceph-mon[31658]: pgmap v504: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:26:31.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:31.156+0000 7f3366000700 1 -- 172.21.15.33:0/1807644380 --> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f33600638e0 con 0x7f33440b2990 2024-05-03T18:26:31.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:31.159+0000 7f3357e00700 1 -- 172.21.15.33:0/1807644380 <== mgr.14152 v2:172.21.15.33:6800/3642895501 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f33600638e0 con 0x7f33440b2990 2024-05-03T18:26:31.160 INFO:teuthology.orchestra.run.smithi033.stdout:HOST ADDR LABELS STATUS 2024-05-03T18:26:31.161 INFO:teuthology.orchestra.run.smithi033.stdout:smithi033 172.21.15.33 2024-05-03T18:26:31.161 INFO:teuthology.orchestra.run.smithi033.stdout:smithi099 172.21.15.99 2024-05-03T18:26:31.161 INFO:teuthology.orchestra.run.smithi033.stdout:smithi150 172.21.15.150 2024-05-03T18:26:31.161 INFO:teuthology.orchestra.run.smithi033.stdout:3 hosts in cluster 2024-05-03T18:26:31.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:31.162+0000 7f3366000700 1 -- 172.21.15.33:0/1807644380 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f33440b2990 msgr2=0x7f33440b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:26:31.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:31.162+0000 7f3366000700 1 --2- 172.21.15.33:0/1807644380 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f33440b2990 0x7f33440b4e50 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3350005950 tx=0x7f33500058e0 comp rx=0 tx=0).stop 2024-05-03T18:26:31.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:31.162+0000 7f3366000700 1 -- 172.21.15.33:0/1807644380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33601072f0 msgr2=0x7f33601a2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:26:31.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:31.162+0000 7f3366000700 1 --2- 172.21.15.33:0/1807644380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33601072f0 0x7f33601a2cb0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f3358009010 tx=0x7f3358009c30 comp rx=0 tx=0).stop 2024-05-03T18:26:31.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:31.163+0000 7f3366000700 1 -- 172.21.15.33:0/1807644380 shutdown_connections 2024-05-03T18:26:31.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:31.163+0000 7f3366000700 1 --2- 172.21.15.33:0/1807644380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33601072f0 0x7f33601a2cb0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:31.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:31.163+0000 7f3366000700 1 --2- 172.21.15.33:0/1807644380 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3360105750 0x7f336019b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:31.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:31.163+0000 7f3366000700 1 --2- 172.21.15.33:0/1807644380 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f33440b2990 0x7f33440b4e50 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:31.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:31.163+0000 7f3366000700 1 --2- 172.21.15.33:0/1807644380 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3360106950 0x7f336019bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:31.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:31.163+0000 7f3366000700 1 -- 172.21.15.33:0/1807644380 >> 172.21.15.33:0/1807644380 conn(0x7f3360100cf0 msgr2=0x7f3360109ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:26:31.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:31.163+0000 7f3366000700 1 -- 172.21.15.33:0/1807644380 shutdown_connections 2024-05-03T18:26:31.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:31.163+0000 7f3366000700 1 -- 172.21.15.33:0/1807644380 wait complete. 2024-05-03T18:26:31.985 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- bash -c 'ceph orch device ls' 2024-05-03T18:26:32.458 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:26:32.840 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:26:32 smithi033 ceph-mon[28330]: from='client.15048 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:26:32.840 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:26:32 smithi033 ceph-mon[28330]: pgmap v505: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:26:32.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:26:32 smithi150 ceph-mon[32492]: from='client.15048 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:26:32.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:26:32 smithi150 ceph-mon[32492]: pgmap v505: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:26:33.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:26:32 smithi099 ceph-mon[31658]: from='client.15048 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:26:33.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:26:32 smithi099 ceph-mon[31658]: pgmap v505: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:26:33.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:26:33 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:26:33.967 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:26:33 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:26:34.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:26:33 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:26:34.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:34.867+0000 7fb5b5600700 1 -- 172.21.15.33:0/1583224452 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5b010faf0 msgr2=0x7fb5b0111ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:26:34.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:34.867+0000 7fb5b5600700 1 --2- 172.21.15.33:0/1583224452 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5b010faf0 0x7fb5b0111ee0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fb5a800b0a0 tx=0x7fb5a800b3b0 comp rx=0 tx=0).stop 2024-05-03T18:26:34.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:34.867+0000 7fb5b5600700 1 -- 172.21.15.33:0/1583224452 shutdown_connections 2024-05-03T18:26:34.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:34.867+0000 7fb5b5600700 1 --2- 172.21.15.33:0/1583224452 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5b010faf0 0x7fb5b0111ee0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:34.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:34.867+0000 7fb5b5600700 1 --2- 172.21.15.33:0/1583224452 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5b00774d0 0x7fb5b0075930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:34.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:34.867+0000 7fb5b5600700 1 --2- 172.21.15.33:0/1583224452 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb5b0075f00 0x7fb5b0076380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:34.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:34.867+0000 7fb5b5600700 1 -- 172.21.15.33:0/1583224452 >> 172.21.15.33:0/1583224452 conn(0x7fb5b00fea70 msgr2=0x7fb5b0100ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:26:34.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:34.868+0000 7fb5b5600700 1 -- 172.21.15.33:0/1583224452 shutdown_connections 2024-05-03T18:26:34.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:34.868+0000 7fb5b5600700 1 -- 172.21.15.33:0/1583224452 wait complete. 2024-05-03T18:26:34.870 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:26:34 smithi033 ceph-mon[28330]: pgmap v506: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:26:34.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:34.869+0000 7fb5b5600700 1 Processor -- start 2024-05-03T18:26:34.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:34.869+0000 7fb5b5600700 1 -- start start 2024-05-03T18:26:34.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:34.869+0000 7fb5b5600700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5b0075f00 0x7fb5b019f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:26:34.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:34.869+0000 7fb5b5600700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb5b00774d0 0x7fb5b019ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:26:34.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:34.869+0000 7fb5b5600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5b010faf0 0x7fb5b01a6f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:26:34.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:34.869+0000 7fb5b5600700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b0115450 con 0x7fb5b0075f00 2024-05-03T18:26:34.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:34.869+0000 7fb5b5600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b01152d0 con 0x7fb5b010faf0 2024-05-03T18:26:34.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:34.869+0000 7fb5b5600700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b01155d0 con 0x7fb5b00774d0 2024-05-03T18:26:34.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:34.870+0000 7fb5aea00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5b0075f00 0x7fb5b019f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:26:34.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:34.870+0000 7fb5aea00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5b0075f00 0x7fb5b019f9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52208/0 (socket says 172.21.15.33:52208) 2024-05-03T18:26:34.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:34.870+0000 7fb5aea00700 1 -- 172.21.15.33:0/1346220965 learned_addr learned my addr 172.21.15.33:0/1346220965 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:26:34.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:34.870+0000 7fb5ae000700 1 --2- 172.21.15.33:0/1346220965 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb5b00774d0 0x7fb5b019ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:26:34.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:34.870+0000 7fb5af400700 1 --2- 172.21.15.33:0/1346220965 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5b010faf0 0x7fb5b01a6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:26:34.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:34.870+0000 7fb5aea00700 1 -- 172.21.15.33:0/1346220965 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb5b00774d0 msgr2=0x7fb5b019ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:26:34.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:34.870+0000 7fb5aea00700 1 --2- 172.21.15.33:0/1346220965 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb5b00774d0 0x7fb5b019ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:34.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:34.870+0000 7fb5aea00700 1 -- 172.21.15.33:0/1346220965 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5b010faf0 msgr2=0x7fb5b01a6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:26:34.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:34.870+0000 7fb5aea00700 1 --2- 172.21.15.33:0/1346220965 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5b010faf0 0x7fb5b01a6f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:34.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:34.870+0000 7fb5aea00700 1 -- 172.21.15.33:0/1346220965 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5b01a7630 con 0x7fb5b0075f00 2024-05-03T18:26:34.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:34.871+0000 7fb5aea00700 1 --2- 172.21.15.33:0/1346220965 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5b0075f00 0x7fb5b019f9d0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fb5a000ca10 tx=0x7fb5a000cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:26:34.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:34.871+0000 7fb5a7400700 1 -- 172.21.15.33:0/1346220965 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a0007b20 con 0x7fb5b0075f00 2024-05-03T18:26:34.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:34.871+0000 7fb5b5600700 1 -- 172.21.15.33:0/1346220965 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5b01a7920 con 0x7fb5b0075f00 2024-05-03T18:26:34.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:34.871+0000 7fb5a7400700 1 -- 172.21.15.33:0/1346220965 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb5a0007cc0 con 0x7fb5b0075f00 2024-05-03T18:26:34.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:34.871+0000 7fb5a7400700 1 -- 172.21.15.33:0/1346220965 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a0018660 con 0x7fb5b0075f00 2024-05-03T18:26:34.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:34.871+0000 7fb5b5600700 1 -- 172.21.15.33:0/1346220965 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5b01a7eb0 con 0x7fb5b0075f00 2024-05-03T18:26:34.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:34.872+0000 7fb5b5600700 1 -- 172.21.15.33:0/1346220965 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5b006a190 con 0x7fb5b0075f00 2024-05-03T18:26:34.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:34.875+0000 7fb5a7400700 1 -- 172.21.15.33:0/1346220965 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fb5a000f450 con 0x7fb5b0075f00 2024-05-03T18:26:34.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:34.877+0000 7fb5a7400700 1 --2- 172.21.15.33:0/1346220965 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fb5940b2990 0x7fb5940b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:26:34.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:34.878+0000 7fb5a7400700 1 -- 172.21.15.33:0/1346220965 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7fb5a00e2780 con 0x7fb5b0075f00 2024-05-03T18:26:34.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:34.878+0000 7fb5ae000700 1 --2- 172.21.15.33:0/1346220965 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fb5940b2990 0x7fb5940b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:26:34.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:34.878+0000 7fb5ae000700 1 --2- 172.21.15.33:0/1346220965 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fb5940b2990 0x7fb5940b4e50 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb59c005fd0 tx=0x7fb59c005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:26:34.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:26:34 smithi150 ceph-mon[32492]: pgmap v506: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:26:34.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:34.882+0000 7fb5a7400700 1 -- 172.21.15.33:0/1346220965 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fb5a0014030 con 0x7fb5b0075f00 2024-05-03T18:26:35.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:26:34 smithi099 ceph-mon[31658]: pgmap v506: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:26:35.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:35.140+0000 7fb5b5600700 1 -- 172.21.15.33:0/1346220965 --> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7fb5b00638e0 con 0x7fb5940b2990 2024-05-03T18:26:35.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:35.143+0000 7fb5a7400700 1 -- 172.21.15.33:0/1346220965 <== mgr.14152 v2:172.21.15.33:6800/3642895501 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7fb5b00638e0 con 0x7fb5940b2990 2024-05-03T18:26:35.144 INFO:teuthology.orchestra.run.smithi033.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-05-03T18:26:35.144 INFO:teuthology.orchestra.run.smithi033.stdout:smithi033 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT5330008Q400BGN 372G No 6m ago Has a FileSystem, LVM detected 2024-05-03T18:26:35.145 INFO:teuthology.orchestra.run.smithi033.stdout:smithi099 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT62040101400BGN 372G No 6m ago Has a FileSystem, LVM detected 2024-05-03T18:26:35.145 INFO:teuthology.orchestra.run.smithi033.stdout:smithi150 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300GX400BGN 372G No 5m ago Has a FileSystem, LVM detected 2024-05-03T18:26:35.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:35.147+0000 7fb5b5600700 1 -- 172.21.15.33:0/1346220965 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fb5940b2990 msgr2=0x7fb5940b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:26:35.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:35.147+0000 7fb5b5600700 1 --2- 172.21.15.33:0/1346220965 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fb5940b2990 0x7fb5940b4e50 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb59c005fd0 tx=0x7fb59c005e50 comp rx=0 tx=0).stop 2024-05-03T18:26:35.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:35.147+0000 7fb5b5600700 1 -- 172.21.15.33:0/1346220965 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5b0075f00 msgr2=0x7fb5b019f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:26:35.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:35.147+0000 7fb5b5600700 1 --2- 172.21.15.33:0/1346220965 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5b0075f00 0x7fb5b019f9d0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fb5a000ca10 tx=0x7fb5a000cdd0 comp rx=0 tx=0).stop 2024-05-03T18:26:35.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:35.147+0000 7fb5b5600700 1 -- 172.21.15.33:0/1346220965 shutdown_connections 2024-05-03T18:26:35.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:35.147+0000 7fb5b5600700 1 --2- 172.21.15.33:0/1346220965 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fb5940b2990 0x7fb5940b4e50 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:35.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:35.147+0000 7fb5b5600700 1 --2- 172.21.15.33:0/1346220965 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5b010faf0 0x7fb5b01a6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:35.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:35.147+0000 7fb5b5600700 1 --2- 172.21.15.33:0/1346220965 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5b0075f00 0x7fb5b019f9d0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:35.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:35.147+0000 7fb5b5600700 1 --2- 172.21.15.33:0/1346220965 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb5b00774d0 0x7fb5b019ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:35.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:35.147+0000 7fb5b5600700 1 -- 172.21.15.33:0/1346220965 >> 172.21.15.33:0/1346220965 conn(0x7fb5b00fea70 msgr2=0x7fb5b0102e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:26:35.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:35.147+0000 7fb5b5600700 1 -- 172.21.15.33:0/1346220965 shutdown_connections 2024-05-03T18:26:35.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:35.147+0000 7fb5b5600700 1 -- 172.21.15.33:0/1346220965 wait complete. 2024-05-03T18:26:35.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:26:35 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:26:35.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:26:35 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:26:35.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:26:35 smithi150 ceph-mon[32492]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:26:35.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:26:35 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:26:35.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:26:35 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:26:35.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:26:35 smithi033 ceph-mon[28330]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:26:36.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:26:35 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:26:36.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:26:35 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:26:36.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:26:35 smithi099 ceph-mon[31658]: from='mgr.14152 172.21.15.33:0/502487534' entity='mgr.x' 2024-05-03T18:26:36.114 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-03T18:26:36.123 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi033.front.sepia.ceph.com 2024-05-03T18:26:36.123 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- bash -c 'ceph fs dump' 2024-05-03T18:26:36.614 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:26:36.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:26:36 smithi150 ceph-mon[32492]: from='client.15054 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:26:36.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:26:36 smithi150 ceph-mon[32492]: pgmap v507: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:26:36.964 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:26:36 smithi033 ceph-mon[28330]: from='client.15054 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:26:36.964 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:26:36 smithi033 ceph-mon[28330]: pgmap v507: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:26:37.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:26:36 smithi099 ceph-mon[31658]: from='client.15054 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:26:37.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:26:36 smithi099 ceph-mon[31658]: pgmap v507: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:26:38.680 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:26:38 smithi033 ceph-mon[28330]: pgmap v508: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:26:38.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:26:38 smithi150 ceph-mon[32492]: pgmap v508: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:26:38.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:38.966+0000 7fa338e00700 1 -- 172.21.15.33:0/992010620 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3341070c0 msgr2=0x7fa334107540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:26:38.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:38.966+0000 7fa338e00700 1 --2- 172.21.15.33:0/992010620 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3341070c0 0x7fa334107540 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fa32c00b0a0 tx=0x7fa32c00b3b0 comp rx=0 tx=0).stop 2024-05-03T18:26:38.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:38.966+0000 7fa338e00700 1 -- 172.21.15.33:0/992010620 shutdown_connections 2024-05-03T18:26:38.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:38.966+0000 7fa338e00700 1 --2- 172.21.15.33:0/992010620 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3341070c0 0x7fa334107540 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:38.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:38.966+0000 7fa338e00700 1 --2- 172.21.15.33:0/992010620 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa334101d80 0x7fa3341041b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:38.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:38.966+0000 7fa338e00700 1 --2- 172.21.15.33:0/992010620 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3341046f0 0x7fa334106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:38.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:38.966+0000 7fa338e00700 1 -- 172.21.15.33:0/992010620 >> 172.21.15.33:0/992010620 conn(0x7fa3340fd9c0 msgr2=0x7fa3340ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:26:38.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:38.966+0000 7fa338e00700 1 -- 172.21.15.33:0/992010620 shutdown_connections 2024-05-03T18:26:38.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:38.966+0000 7fa338e00700 1 -- 172.21.15.33:0/992010620 wait complete. 2024-05-03T18:26:38.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:38.967+0000 7fa338e00700 1 Processor -- start 2024-05-03T18:26:38.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:38.967+0000 7fa338e00700 1 -- start start 2024-05-03T18:26:38.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:38.968+0000 7fa338e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa334101d80 0x7fa33419f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:26:38.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:38.968+0000 7fa338e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3341046f0 0x7fa33419fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:26:38.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:38.968+0000 7fa338e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3341070c0 0x7fa3341a6f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:26:38.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:38.968+0000 7fa338e00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa3341153f0 con 0x7fa3341046f0 2024-05-03T18:26:38.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:38.968+0000 7fa338e00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa334115270 con 0x7fa3341070c0 2024-05-03T18:26:38.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:38.968+0000 7fa338e00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa334115570 con 0x7fa334101d80 2024-05-03T18:26:38.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:38.968+0000 7fa331400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3341046f0 0x7fa33419fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:26:38.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:38.968+0000 7fa331400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3341046f0 0x7fa33419fee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52020/0 (socket says 172.21.15.33:52020) 2024-05-03T18:26:38.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:38.968+0000 7fa331400700 1 -- 172.21.15.33:0/2889539702 learned_addr learned my addr 172.21.15.33:0/2889539702 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:26:38.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:38.969+0000 7fa331e00700 1 --2- 172.21.15.33:0/2889539702 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa334101d80 0x7fa33419f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:26:38.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:38.969+0000 7fa332800700 1 --2- 172.21.15.33:0/2889539702 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3341070c0 0x7fa3341a6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:26:38.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:38.969+0000 7fa331400700 1 -- 172.21.15.33:0/2889539702 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa334101d80 msgr2=0x7fa33419f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:26:38.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:38.969+0000 7fa331400700 1 --2- 172.21.15.33:0/2889539702 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa334101d80 0x7fa33419f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:38.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:38.969+0000 7fa331400700 1 -- 172.21.15.33:0/2889539702 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3341070c0 msgr2=0x7fa3341a6f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:26:38.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:38.969+0000 7fa331400700 1 --2- 172.21.15.33:0/2889539702 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3341070c0 0x7fa3341a6f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:38.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:38.969+0000 7fa331400700 1 -- 172.21.15.33:0/2889539702 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3341a7600 con 0x7fa3341046f0 2024-05-03T18:26:38.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:38.969+0000 7fa332800700 1 --2- 172.21.15.33:0/2889539702 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3341070c0 0x7fa3341a6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:26:38.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:38.969+0000 7fa331400700 1 --2- 172.21.15.33:0/2889539702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3341046f0 0x7fa33419fee0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fa32000c9e0 tx=0x7fa32000ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:26:38.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:38.970+0000 7fa32aa00700 1 -- 172.21.15.33:0/2889539702 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa320007ac0 con 0x7fa3341046f0 2024-05-03T18:26:38.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:38.970+0000 7fa32aa00700 1 -- 172.21.15.33:0/2889539702 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa320007c60 con 0x7fa3341046f0 2024-05-03T18:26:38.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:38.970+0000 7fa338e00700 1 -- 172.21.15.33:0/2889539702 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3341a78f0 con 0x7fa3341046f0 2024-05-03T18:26:38.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:38.970+0000 7fa32aa00700 1 -- 172.21.15.33:0/2889539702 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa320018660 con 0x7fa3341046f0 2024-05-03T18:26:38.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:38.970+0000 7fa338e00700 1 -- 172.21.15.33:0/2889539702 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3341a7e30 con 0x7fa3341046f0 2024-05-03T18:26:38.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:38.971+0000 7fa338e00700 1 -- 172.21.15.33:0/2889539702 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa33406a190 con 0x7fa3341046f0 2024-05-03T18:26:38.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:38.975+0000 7fa32aa00700 1 -- 172.21.15.33:0/2889539702 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fa32000f450 con 0x7fa3341046f0 2024-05-03T18:26:38.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:38.976+0000 7fa32aa00700 1 --2- 172.21.15.33:0/2889539702 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fa3180b2910 0x7fa3180b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:26:38.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:38.976+0000 7fa32aa00700 1 -- 172.21.15.33:0/2889539702 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7fa3200e2a20 con 0x7fa3341046f0 2024-05-03T18:26:38.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:38.977+0000 7fa331e00700 1 --2- 172.21.15.33:0/2889539702 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fa3180b2910 0x7fa3180b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:26:38.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:38.977+0000 7fa331e00700 1 --2- 172.21.15.33:0/2889539702 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fa3180b2910 0x7fa3180b4dd0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa32400ac50 tx=0x7fa32400a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:26:38.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:38.978+0000 7fa32aa00700 1 -- 172.21.15.33:0/2889539702 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fa320014030 con 0x7fa3341046f0 2024-05-03T18:26:39.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:26:38 smithi099 ceph-mon[31658]: pgmap v508: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:26:39.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:39.267+0000 7fa338e00700 1 -- 172.21.15.33:0/2889539702 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fa33402e590 con 0x7fa3341046f0 2024-05-03T18:26:39.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:39.268+0000 7fa32aa00700 1 -- 172.21.15.33:0/2889539702 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 21 v21) ==== 76+0+3004 (secure 0 0 0) 0x7fa3200aa290 con 0x7fa3341046f0 2024-05-03T18:26:39.271 INFO:teuthology.orchestra.run.smithi033.stdout:e21 2024-05-03T18:26:39.271 INFO:teuthology.orchestra.run.smithi033.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-03T18:26:39.271 INFO:teuthology.orchestra.run.smithi033.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-03T18:26:39.271 INFO:teuthology.orchestra.run.smithi033.stdout:legacy client fscid: 1 2024-05-03T18:26:39.271 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:26:39.271 INFO:teuthology.orchestra.run.smithi033.stdout:Filesystem 'cephfs' (1) 2024-05-03T18:26:39.271 INFO:teuthology.orchestra.run.smithi033.stdout:fs_name cephfs 2024-05-03T18:26:39.271 INFO:teuthology.orchestra.run.smithi033.stdout:epoch 21 2024-05-03T18:26:39.271 INFO:teuthology.orchestra.run.smithi033.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-03T18:26:39.271 INFO:teuthology.orchestra.run.smithi033.stdout:created 2024-05-03T18:22:28.837091+0000 2024-05-03T18:26:39.271 INFO:teuthology.orchestra.run.smithi033.stdout:modified 2024-05-03T18:23:49.758398+0000 2024-05-03T18:26:39.271 INFO:teuthology.orchestra.run.smithi033.stdout:tableserver 0 2024-05-03T18:26:39.272 INFO:teuthology.orchestra.run.smithi033.stdout:root 0 2024-05-03T18:26:39.272 INFO:teuthology.orchestra.run.smithi033.stdout:session_timeout 300 2024-05-03T18:26:39.272 INFO:teuthology.orchestra.run.smithi033.stdout:session_autoclose 300 2024-05-03T18:26:39.272 INFO:teuthology.orchestra.run.smithi033.stdout:max_file_size 1099511627776 2024-05-03T18:26:39.272 INFO:teuthology.orchestra.run.smithi033.stdout:max_xattr_size 65536 2024-05-03T18:26:39.272 INFO:teuthology.orchestra.run.smithi033.stdout:required_client_features {} 2024-05-03T18:26:39.272 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure 0 2024-05-03T18:26:39.272 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure_osd_epoch 0 2024-05-03T18:26:39.272 INFO:teuthology.orchestra.run.smithi033.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-03T18:26:39.272 INFO:teuthology.orchestra.run.smithi033.stdout:max_mds 1 2024-05-03T18:26:39.272 INFO:teuthology.orchestra.run.smithi033.stdout:in 0 2024-05-03T18:26:39.272 INFO:teuthology.orchestra.run.smithi033.stdout:up {0=34508} 2024-05-03T18:26:39.272 INFO:teuthology.orchestra.run.smithi033.stdout:failed 2024-05-03T18:26:39.272 INFO:teuthology.orchestra.run.smithi033.stdout:damaged 2024-05-03T18:26:39.272 INFO:teuthology.orchestra.run.smithi033.stdout:stopped 2024-05-03T18:26:39.273 INFO:teuthology.orchestra.run.smithi033.stdout:data_pools [3,4] 2024-05-03T18:26:39.273 INFO:teuthology.orchestra.run.smithi033.stdout:metadata_pool 2 2024-05-03T18:26:39.273 INFO:teuthology.orchestra.run.smithi033.stdout:inline_data disabled 2024-05-03T18:26:39.273 INFO:teuthology.orchestra.run.smithi033.stdout:balancer 2024-05-03T18:26:39.273 INFO:teuthology.orchestra.run.smithi033.stdout:bal_rank_mask -1 2024-05-03T18:26:39.273 INFO:teuthology.orchestra.run.smithi033.stdout:standby_count_wanted 0 2024-05-03T18:26:39.273 INFO:teuthology.orchestra.run.smithi033.stdout:qdb_cluster leader: 34508 members: 34508 2024-05-03T18:26:39.273 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.b{0:34508} state up:active seq 7 addr [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] compat {c=[1],r=[1],i=[1fff]}] 2024-05-03T18:26:39.273 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.h{0:34472} state up:standby-replay seq 1 addr [v2:172.21.15.99:6836/3360970342,v1:172.21.15.99:6837/3360970342] compat {c=[1],r=[1],i=[1fff]}] 2024-05-03T18:26:39.273 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:26:39.273 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:26:39.273 INFO:teuthology.orchestra.run.smithi033.stdout:Standby daemons: 2024-05-03T18:26:39.273 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:26:39.273 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.j{-1:14499} state up:standby seq 1 addr [v2:172.21.15.33:6834/1327393503,v1:172.21.15.33:6835/1327393503] compat {c=[1],r=[1],i=[1fff]}] 2024-05-03T18:26:39.273 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.d{-1:14511} state up:standby seq 1 addr [v2:172.21.15.33:6836/2461472948,v1:172.21.15.33:6837/2461472948] compat {c=[1],r=[1],i=[1fff]}] 2024-05-03T18:26:39.274 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.g{-1:14559} state up:standby seq 1 addr [v2:172.21.15.33:6838/442269281,v1:172.21.15.33:6839/442269281] compat {c=[1],r=[1],i=[1fff]}] 2024-05-03T18:26:39.274 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.a{-1:14571} state up:standby seq 1 addr [v2:172.21.15.33:6840/645571756,v1:172.21.15.33:6841/645571756] compat {c=[1],r=[1],i=[1fff]}] 2024-05-03T18:26:39.274 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.l{-1:24412} state up:standby seq 1 addr [v2:172.21.15.150:6832/203598909,v1:172.21.15.150:6833/203598909] compat {c=[1],r=[1],i=[1fff]}] 2024-05-03T18:26:39.274 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.f{-1:24457} state up:standby seq 1 addr [v2:172.21.15.150:6834/4029842493,v1:172.21.15.150:6835/4029842493] compat {c=[1],r=[1],i=[1fff]}] 2024-05-03T18:26:39.274 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.c{-1:24469} state up:standby seq 1 addr [v2:172.21.15.150:6836/3463664423,v1:172.21.15.150:6837/3463664423] compat {c=[1],r=[1],i=[1fff]}] 2024-05-03T18:26:39.274 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.i{-1:24481} state up:standby seq 1 addr [v2:172.21.15.150:6838/4051648018,v1:172.21.15.150:6839/4051648018] compat {c=[1],r=[1],i=[1fff]}] 2024-05-03T18:26:39.274 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.k{-1:34418} state up:standby seq 1 addr [v2:172.21.15.99:6832/1865105479,v1:172.21.15.99:6833/1865105479] compat {c=[1],r=[1],i=[1fff]}] 2024-05-03T18:26:39.274 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.e{-1:34445} state up:standby seq 1 addr [v2:172.21.15.99:6834/1206393914,v1:172.21.15.99:6835/1206393914] compat {c=[1],r=[1],i=[1fff]}] 2024-05-03T18:26:39.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:39.271+0000 7fa338e00700 1 -- 172.21.15.33:0/2889539702 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fa3180b2910 msgr2=0x7fa3180b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:26:39.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:39.271+0000 7fa338e00700 1 --2- 172.21.15.33:0/2889539702 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fa3180b2910 0x7fa3180b4dd0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa32400ac50 tx=0x7fa32400a380 comp rx=0 tx=0).stop 2024-05-03T18:26:39.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:39.271+0000 7fa338e00700 1 -- 172.21.15.33:0/2889539702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3341046f0 msgr2=0x7fa33419fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:26:39.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:39.271+0000 7fa338e00700 1 --2- 172.21.15.33:0/2889539702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3341046f0 0x7fa33419fee0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fa32000c9e0 tx=0x7fa32000ccf0 comp rx=0 tx=0).stop 2024-05-03T18:26:39.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:39.271+0000 7fa338e00700 1 -- 172.21.15.33:0/2889539702 shutdown_connections 2024-05-03T18:26:39.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:39.271+0000 7fa338e00700 1 --2- 172.21.15.33:0/2889539702 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fa3180b2910 0x7fa3180b4dd0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:39.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:39.271+0000 7fa338e00700 1 --2- 172.21.15.33:0/2889539702 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3341070c0 0x7fa3341a6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:39.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:39.272+0000 7fa338e00700 1 --2- 172.21.15.33:0/2889539702 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa334101d80 0x7fa33419f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:39.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:39.272+0000 7fa338e00700 1 --2- 172.21.15.33:0/2889539702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3341046f0 0x7fa33419fee0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:39.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:39.272+0000 7fa338e00700 1 -- 172.21.15.33:0/2889539702 >> 172.21.15.33:0/2889539702 conn(0x7fa3340fd9c0 msgr2=0x7fa334102a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:26:39.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:39.272+0000 7fa338e00700 1 -- 172.21.15.33:0/2889539702 shutdown_connections 2024-05-03T18:26:39.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:39.272+0000 7fa338e00700 1 -- 172.21.15.33:0/2889539702 wait complete. 2024-05-03T18:26:39.276 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 21 2024-05-03T18:26:39.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:26:39 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/2889539702' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-03T18:26:39.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:26:39 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/2889539702' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-03T18:26:40.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:26:39 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/2889539702' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-03T18:26:40.178 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- bash -c 'ceph osd dump' 2024-05-03T18:26:40.653 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:26:40.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:26:40 smithi150 ceph-mon[32492]: pgmap v509: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:26:40.976 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:26:40 smithi033 ceph-mon[28330]: pgmap v509: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:26:41.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:26:40 smithi099 ceph-mon[31658]: pgmap v509: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:26:42.852 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:26:42 smithi033 ceph-mon[28330]: pgmap v510: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:26:43.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:43.029+0000 7f8470600700 1 -- 172.21.15.33:0/1596466529 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8468106940 msgr2=0x7f8468106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:26:43.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:43.029+0000 7f8470600700 1 --2- 172.21.15.33:0/1596466529 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8468106940 0x7f8468106da0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f84540099c0 tx=0x7f8454009cd0 comp rx=0 tx=0).stop 2024-05-03T18:26:43.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:43.030+0000 7f8470600700 1 -- 172.21.15.33:0/1596466529 shutdown_connections 2024-05-03T18:26:43.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:43.030+0000 7f8470600700 1 --2- 172.21.15.33:0/1596466529 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84681072e0 0x7f846810db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:43.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:43.030+0000 7f8470600700 1 --2- 172.21.15.33:0/1596466529 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8468105740 0x7f8468105b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:43.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:43.030+0000 7f8470600700 1 --2- 172.21.15.33:0/1596466529 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8468106940 0x7f8468106da0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:43.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:43.030+0000 7f8470600700 1 -- 172.21.15.33:0/1596466529 >> 172.21.15.33:0/1596466529 conn(0x7f8468100d00 msgr2=0x7f8468103120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:26:43.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:43.030+0000 7f8470600700 1 -- 172.21.15.33:0/1596466529 shutdown_connections 2024-05-03T18:26:43.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:43.030+0000 7f8470600700 1 -- 172.21.15.33:0/1596466529 wait complete. 2024-05-03T18:26:43.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:43.031+0000 7f8470600700 1 Processor -- start 2024-05-03T18:26:43.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:43.031+0000 7f8470600700 1 -- start start 2024-05-03T18:26:43.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:43.032+0000 7f8470600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8468105740 0x7f846819b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:26:43.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:43.032+0000 7f8470600700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8468106940 0x7f846819bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:26:43.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:43.032+0000 7f8470600700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84681072e0 0x7f84681a2d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:26:43.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:43.032+0000 7f8470600700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f84681110e0 con 0x7f84681072e0 2024-05-03T18:26:43.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:43.032+0000 7f8470600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8468110f60 con 0x7f8468105740 2024-05-03T18:26:43.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:43.032+0000 7f8470600700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8468111260 con 0x7f8468106940 2024-05-03T18:26:43.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:43.032+0000 7f846e600700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84681072e0 0x7f84681a2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:26:43.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:43.032+0000 7f846e600700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84681072e0 0x7f84681a2d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52046/0 (socket says 172.21.15.33:52046) 2024-05-03T18:26:43.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:43.032+0000 7f846e600700 1 -- 172.21.15.33:0/1518212570 learned_addr learned my addr 172.21.15.33:0/1518212570 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:26:43.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:43.032+0000 7f846dc00700 1 --2- 172.21.15.33:0/1518212570 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8468105740 0x7f846819b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:26:43.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:43.032+0000 7f846d200700 1 --2- 172.21.15.33:0/1518212570 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8468106940 0x7f846819bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:26:43.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:43.033+0000 7f846e600700 1 -- 172.21.15.33:0/1518212570 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8468106940 msgr2=0x7f846819bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:26:43.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:43.033+0000 7f846e600700 1 --2- 172.21.15.33:0/1518212570 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8468106940 0x7f846819bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:43.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:43.033+0000 7f846e600700 1 -- 172.21.15.33:0/1518212570 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8468105740 msgr2=0x7f846819b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:26:43.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:43.033+0000 7f846e600700 1 --2- 172.21.15.33:0/1518212570 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8468105740 0x7f846819b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:43.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:43.033+0000 7f846e600700 1 -- 172.21.15.33:0/1518212570 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84681a33b0 con 0x7f84681072e0 2024-05-03T18:26:43.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:43.033+0000 7f846e600700 1 --2- 172.21.15.33:0/1518212570 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84681072e0 0x7f84681a2d10 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f846000ef60 tx=0x7f846000ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:26:43.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:43.033+0000 7f8466000700 1 -- 172.21.15.33:0/1518212570 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84600092e0 con 0x7f84681072e0 2024-05-03T18:26:43.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:43.034+0000 7f8470600700 1 -- 172.21.15.33:0/1518212570 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84681a3640 con 0x7f84681072e0 2024-05-03T18:26:43.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:43.034+0000 7f8466000700 1 -- 172.21.15.33:0/1518212570 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8460004bb0 con 0x7f84681072e0 2024-05-03T18:26:43.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:43.034+0000 7f8466000700 1 -- 172.21.15.33:0/1518212570 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8460004020 con 0x7f84681072e0 2024-05-03T18:26:43.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:43.034+0000 7f8470600700 1 -- 172.21.15.33:0/1518212570 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84681a3b80 con 0x7f84681072e0 2024-05-03T18:26:43.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:43.035+0000 7f8470600700 1 -- 172.21.15.33:0/1518212570 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f846806a190 con 0x7f84681072e0 2024-05-03T18:26:43.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:43.037+0000 7f8466000700 1 -- 172.21.15.33:0/1518212570 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f846001b030 con 0x7f84681072e0 2024-05-03T18:26:43.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:43.039+0000 7f8466000700 1 --2- 172.21.15.33:0/1518212570 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f84500b2910 0x7f84500b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:26:43.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:43.039+0000 7f8466000700 1 -- 172.21.15.33:0/1518212570 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f84600e22b0 con 0x7f84681072e0 2024-05-03T18:26:43.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:43.039+0000 7f846dc00700 1 --2- 172.21.15.33:0/1518212570 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f84500b2910 0x7f84500b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:26:43.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:43.040+0000 7f846dc00700 1 --2- 172.21.15.33:0/1518212570 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f84500b2910 0x7f84500b4dd0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f845c005950 tx=0x7f845c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:26:43.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:43.044+0000 7f8466000700 1 -- 172.21.15.33:0/1518212570 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f8460022020 con 0x7f84681072e0 2024-05-03T18:26:43.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:26:42 smithi099 ceph-mon[31658]: pgmap v510: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:26:43.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:26:42 smithi150 ceph-mon[32492]: pgmap v510: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:26:43.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:43.283+0000 7f8470600700 1 -- 172.21.15.33:0/1518212570 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f846802e510 con 0x7f84681072e0 2024-05-03T18:26:43.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:43.286+0000 7f8466000700 1 -- 172.21.15.33:0/1518212570 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v89) ==== 56+0+5610 (secure 0 0 0) 0x7f8460010040 con 0x7f84681072e0 2024-05-03T18:26:43.287 INFO:teuthology.orchestra.run.smithi033.stdout:epoch 89 2024-05-03T18:26:43.287 INFO:teuthology.orchestra.run.smithi033.stdout:fsid 5703bba4-0978-11ef-bc97-c7b262605968 2024-05-03T18:26:43.288 INFO:teuthology.orchestra.run.smithi033.stdout:created 2024-05-03T18:11:00.899879+0000 2024-05-03T18:26:43.288 INFO:teuthology.orchestra.run.smithi033.stdout:modified 2024-05-03T18:24:21.532082+0000 2024-05-03T18:26:43.288 INFO:teuthology.orchestra.run.smithi033.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-05-03T18:26:43.288 INFO:teuthology.orchestra.run.smithi033.stdout:crush_version 27 2024-05-03T18:26:43.288 INFO:teuthology.orchestra.run.smithi033.stdout:full_ratio 0.95 2024-05-03T18:26:43.288 INFO:teuthology.orchestra.run.smithi033.stdout:backfillfull_ratio 0.9 2024-05-03T18:26:43.288 INFO:teuthology.orchestra.run.smithi033.stdout:nearfull_ratio 0.85 2024-05-03T18:26:43.288 INFO:teuthology.orchestra.run.smithi033.stdout:require_min_compat_client luminous 2024-05-03T18:26:43.288 INFO:teuthology.orchestra.run.smithi033.stdout:min_compat_client luminous 2024-05-03T18:26:43.288 INFO:teuthology.orchestra.run.smithi033.stdout:require_osd_release squid 2024-05-03T18:26:43.288 INFO:teuthology.orchestra.run.smithi033.stdout:stretch_mode_enabled false 2024-05-03T18:26:43.288 INFO:teuthology.orchestra.run.smithi033.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-05-03T18:26:43.288 INFO:teuthology.orchestra.run.smithi033.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 78 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-05-03T18:26:43.288 INFO:teuthology.orchestra.run.smithi033.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 78 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-05-03T18:26:43.288 INFO:teuthology.orchestra.run.smithi033.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 85 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-05-03T18:26:43.289 INFO:teuthology.orchestra.run.smithi033.stdout:max_osd 12 2024-05-03T18:26:43.289 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0 up in weight 1 up_from 8 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.33:6802/637686669,v1:172.21.15.33:6803/637686669] [v2:172.21.15.33:6804/637686669,v1:172.21.15.33:6805/637686669] exists,up fdcc4e66-4ba0-4378-a9a4-7d907cd00b4e 2024-05-03T18:26:43.289 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1 up in weight 1 up_from 13 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.99:6800/1536122841,v1:172.21.15.99:6801/1536122841] [v2:172.21.15.99:6802/1536122841,v1:172.21.15.99:6803/1536122841] exists,up cf78cad3-fc24-43f4-8784-21cabea7277c 2024-05-03T18:26:43.289 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2 up in weight 1 up_from 18 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.150:6800/1995659622,v1:172.21.15.150:6801/1995659622] [v2:172.21.15.150:6802/1995659622,v1:172.21.15.150:6803/1995659622] exists,up 59f066d7-626b-410a-8d67-daca6fbbd3ce 2024-05-03T18:26:43.289 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3 up in weight 1 up_from 25 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.33:6810/3814942724,v1:172.21.15.33:6811/3814942724] [v2:172.21.15.33:6812/3814942724,v1:172.21.15.33:6813/3814942724] exists,up d88ed022-4dcb-45fc-9fc6-41ef7c4bf202 2024-05-03T18:26:43.289 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4 up in weight 1 up_from 31 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.99:6808/29081029,v1:172.21.15.99:6809/29081029] [v2:172.21.15.99:6810/29081029,v1:172.21.15.99:6811/29081029] exists,up 1235e1a6-ff6e-4668-ac1a-073c2a3b7c98 2024-05-03T18:26:43.289 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5 up in weight 1 up_from 36 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.150:6808/3172189533,v1:172.21.15.150:6809/3172189533] [v2:172.21.15.150:6810/3172189533,v1:172.21.15.150:6811/3172189533] exists,up 4b84bff8-d6d7-4258-a0ee-53bc19802585 2024-05-03T18:26:43.289 INFO:teuthology.orchestra.run.smithi033.stdout:osd.6 up in weight 1 up_from 42 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.33:6818/4234541418,v1:172.21.15.33:6819/4234541418] [v2:172.21.15.33:6820/4234541418,v1:172.21.15.33:6821/4234541418] exists,up 51f5fbdd-3794-4c48-9f6c-478f0fa35ac5 2024-05-03T18:26:43.289 INFO:teuthology.orchestra.run.smithi033.stdout:osd.7 up in weight 1 up_from 47 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.99:6816/908673090,v1:172.21.15.99:6817/908673090] [v2:172.21.15.99:6818/908673090,v1:172.21.15.99:6819/908673090] exists,up 982cc5b3-3163-4505-9bde-a27a8c289457 2024-05-03T18:26:43.289 INFO:teuthology.orchestra.run.smithi033.stdout:osd.8 up in weight 1 up_from 53 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.150:6816/1308083602,v1:172.21.15.150:6817/1308083602] [v2:172.21.15.150:6818/1308083602,v1:172.21.15.150:6819/1308083602] exists,up 84c9d037-eb84-4eb4-8aa2-e784b703d36c 2024-05-03T18:26:43.289 INFO:teuthology.orchestra.run.smithi033.stdout:osd.9 up in weight 1 up_from 59 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.33:6826/4133096905,v1:172.21.15.33:6827/4133096905] [v2:172.21.15.33:6828/4133096905,v1:172.21.15.33:6829/4133096905] exists,up a02e965f-0d8f-45e2-9490-337f7f6e3881 2024-05-03T18:26:43.289 INFO:teuthology.orchestra.run.smithi033.stdout:osd.10 up in weight 1 up_from 64 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.99:6824/935746136,v1:172.21.15.99:6825/935746136] [v2:172.21.15.99:6826/935746136,v1:172.21.15.99:6827/935746136] exists,up 8b29bb96-76e8-45a0-ab19-8d082d755fe9 2024-05-03T18:26:43.289 INFO:teuthology.orchestra.run.smithi033.stdout:osd.11 up in weight 1 up_from 70 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.150:6824/2069349808,v1:172.21.15.150:6825/2069349808] [v2:172.21.15.150:6826/2069349808,v1:172.21.15.150:6827/2069349808] exists,up 07a6a825-c902-4bfd-97cc-cc9074532af5 2024-05-03T18:26:43.289 INFO:teuthology.orchestra.run.smithi033.stdout:pg_upmap_items 2.8 [2,11] 2024-05-03T18:26:43.289 INFO:teuthology.orchestra.run.smithi033.stdout:pg_upmap_items 2.1c [8,11] 2024-05-03T18:26:43.290 INFO:teuthology.orchestra.run.smithi033.stdout:pg_upmap_items 3.17 [4,6] 2024-05-03T18:26:43.290 INFO:teuthology.orchestra.run.smithi033.stdout:pg_upmap_items 3.1d [4,8] 2024-05-03T18:26:43.290 INFO:teuthology.orchestra.run.smithi033.stdout:pg_upmap_items 3.22 [3,9] 2024-05-03T18:26:43.290 INFO:teuthology.orchestra.run.smithi033.stdout:pg_upmap_items 3.25 [5,9] 2024-05-03T18:26:43.290 INFO:teuthology.orchestra.run.smithi033.stdout:pg_upmap_items 3.39 [5,8] 2024-05-03T18:26:43.290 INFO:teuthology.orchestra.run.smithi033.stdout:pg_upmap_items 3.3c [4,9] 2024-05-03T18:26:43.290 INFO:teuthology.orchestra.run.smithi033.stdout:pg_upmap_items 4.6 [9,11,5,8] 2024-05-03T18:26:43.290 INFO:teuthology.orchestra.run.smithi033.stdout:pg_upmap_items 4.b [9,8] 2024-05-03T18:26:43.290 INFO:teuthology.orchestra.run.smithi033.stdout:pg_upmap_items 4.18 [9,8] 2024-05-03T18:26:43.290 INFO:teuthology.orchestra.run.smithi033.stdout:blocklist 172.21.15.33:0/3991947997 expires 2024-05-04T18:12:19.127652+0000 2024-05-03T18:26:43.290 INFO:teuthology.orchestra.run.smithi033.stdout:blocklist 172.21.15.33:0/965859483 expires 2024-05-04T18:12:19.127652+0000 2024-05-03T18:26:43.290 INFO:teuthology.orchestra.run.smithi033.stdout:blocklist 172.21.15.33:0/2014902144 expires 2024-05-04T18:11:44.239835+0000 2024-05-03T18:26:43.291 INFO:teuthology.orchestra.run.smithi033.stdout:blocklist 172.21.15.33:0/1314704208 expires 2024-05-04T18:11:44.239835+0000 2024-05-03T18:26:43.291 INFO:teuthology.orchestra.run.smithi033.stdout:blocklist 172.21.15.33:0/1614640996 expires 2024-05-04T18:11:44.239835+0000 2024-05-03T18:26:43.291 INFO:teuthology.orchestra.run.smithi033.stdout:blocklist 172.21.15.33:6800/4007322071 expires 2024-05-04T18:11:44.239835+0000 2024-05-03T18:26:43.291 INFO:teuthology.orchestra.run.smithi033.stdout:blocklist 172.21.15.33:6801/2830135294 expires 2024-05-04T18:12:19.127652+0000 2024-05-03T18:26:43.291 INFO:teuthology.orchestra.run.smithi033.stdout:blocklist 172.21.15.33:6800/2830135294 expires 2024-05-04T18:12:19.127652+0000 2024-05-03T18:26:43.291 INFO:teuthology.orchestra.run.smithi033.stdout:blocklist 172.21.15.33:0/3767665033 expires 2024-05-04T18:12:19.127652+0000 2024-05-03T18:26:43.291 INFO:teuthology.orchestra.run.smithi033.stdout:blocklist 172.21.15.33:6801/4007322071 expires 2024-05-04T18:11:44.239835+0000 2024-05-03T18:26:43.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:43.288+0000 7f8470600700 1 -- 172.21.15.33:0/1518212570 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f84500b2910 msgr2=0x7f84500b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:26:43.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:43.288+0000 7f8470600700 1 --2- 172.21.15.33:0/1518212570 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f84500b2910 0x7f84500b4dd0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f845c005950 tx=0x7f845c0058e0 comp rx=0 tx=0).stop 2024-05-03T18:26:43.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:43.288+0000 7f8470600700 1 -- 172.21.15.33:0/1518212570 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84681072e0 msgr2=0x7f84681a2d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:26:43.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:43.288+0000 7f8470600700 1 --2- 172.21.15.33:0/1518212570 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84681072e0 0x7f84681a2d10 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f846000ef60 tx=0x7f846000ef90 comp rx=0 tx=0).stop 2024-05-03T18:26:43.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:43.289+0000 7f8470600700 1 -- 172.21.15.33:0/1518212570 shutdown_connections 2024-05-03T18:26:43.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:43.289+0000 7f8470600700 1 --2- 172.21.15.33:0/1518212570 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84681072e0 0x7f84681a2d10 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:43.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:43.289+0000 7f8470600700 1 --2- 172.21.15.33:0/1518212570 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f84500b2910 0x7f84500b4dd0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:43.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:43.289+0000 7f8470600700 1 --2- 172.21.15.33:0/1518212570 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8468105740 0x7f846819b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:43.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:43.289+0000 7f8470600700 1 --2- 172.21.15.33:0/1518212570 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8468106940 0x7f846819bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:43.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:43.289+0000 7f8470600700 1 -- 172.21.15.33:0/1518212570 >> 172.21.15.33:0/1518212570 conn(0x7f8468100d00 msgr2=0x7f8468109cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:26:43.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:43.289+0000 7f8470600700 1 -- 172.21.15.33:0/1518212570 shutdown_connections 2024-05-03T18:26:43.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:43.289+0000 7f8470600700 1 -- 172.21.15.33:0/1518212570 wait complete. 2024-05-03T18:26:43.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:26:43 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/1518212570' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-03T18:26:44.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:26:43 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/1518212570' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-03T18:26:44.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:26:43 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/1518212570' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-03T18:26:44.173 INFO:teuthology.run_tasks:Running task fs.ready... 2024-05-03T18:26:44.186 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T18:26:44.670 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:26:44.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:26:44 smithi033 ceph-mon[28330]: pgmap v511: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:26:45.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:26:44 smithi099 ceph-mon[31658]: pgmap v511: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:26:45.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:26:44 smithi150 ceph-mon[32492]: pgmap v511: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:26:46.798 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:26:46 smithi033 ceph-mon[28330]: pgmap v512: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:26:47.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:26:46 smithi099 ceph-mon[31658]: pgmap v512: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:26:47.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:47.086+0000 7fa731a00700 1 -- 172.21.15.33:0/1742949665 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa72c10b670 msgr2=0x7fa72c10ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:26:47.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:47.086+0000 7fa731a00700 1 --2- 172.21.15.33:0/1742949665 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa72c10b670 0x7fa72c10ba50 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fa71c0099c0 tx=0x7fa71c009cd0 comp rx=0 tx=0).stop 2024-05-03T18:26:47.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:47.086+0000 7fa731a00700 1 -- 172.21.15.33:0/1742949665 shutdown_connections 2024-05-03T18:26:47.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:47.086+0000 7fa731a00700 1 --2- 172.21.15.33:0/1742949665 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa72c105f80 0x7fa72c112440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:47.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:47.086+0000 7fa731a00700 1 --2- 172.21.15.33:0/1742949665 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa72c10b670 0x7fa72c10ba50 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:47.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:47.086+0000 7fa731a00700 1 --2- 172.21.15.33:0/1742949665 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa72c1055c0 0x7fa72c105a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:47.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:47.086+0000 7fa731a00700 1 -- 172.21.15.33:0/1742949665 >> 172.21.15.33:0/1742949665 conn(0x7fa72c1010d0 msgr2=0x7fa72c1034f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:26:47.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:47.086+0000 7fa731a00700 1 -- 172.21.15.33:0/1742949665 shutdown_connections 2024-05-03T18:26:47.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:47.086+0000 7fa731a00700 1 -- 172.21.15.33:0/1742949665 wait complete. 2024-05-03T18:26:47.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:47.087+0000 7fa731a00700 1 Processor -- start 2024-05-03T18:26:47.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:47.088+0000 7fa731a00700 1 -- start start 2024-05-03T18:26:47.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:47.088+0000 7fa731a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa72c1055c0 0x7fa72c1a16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:26:47.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:47.088+0000 7fa731a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa72c105f80 0x7fa72c1a1c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:26:47.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:47.088+0000 7fa731a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa72c10b670 0x7fa72c19b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:26:47.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:47.088+0000 7fa731a00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa72c115640 con 0x7fa72c105f80 2024-05-03T18:26:47.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:47.088+0000 7fa731a00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa72c1154c0 con 0x7fa72c10b670 2024-05-03T18:26:47.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:47.088+0000 7fa731a00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa72c1157c0 con 0x7fa72c1055c0 2024-05-03T18:26:47.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:47.088+0000 7fa72b400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa72c105f80 0x7fa72c1a1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:26:47.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:47.088+0000 7fa72b400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa72c105f80 0x7fa72c1a1c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33192/0 (socket says 172.21.15.33:33192) 2024-05-03T18:26:47.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:47.088+0000 7fa72b400700 1 -- 172.21.15.33:0/4038110395 learned_addr learned my addr 172.21.15.33:0/4038110395 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:26:47.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:47.089+0000 7fa72be00700 1 --2- 172.21.15.33:0/4038110395 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa72c1055c0 0x7fa72c1a16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:26:47.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:47.089+0000 7fa731000700 1 --2- 172.21.15.33:0/4038110395 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa72c10b670 0x7fa72c19b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:26:47.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:47.089+0000 7fa72b400700 1 -- 172.21.15.33:0/4038110395 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa72c1055c0 msgr2=0x7fa72c1a16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:26:47.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:47.089+0000 7fa72b400700 1 --2- 172.21.15.33:0/4038110395 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa72c1055c0 0x7fa72c1a16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:47.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:47.089+0000 7fa72b400700 1 -- 172.21.15.33:0/4038110395 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa72c10b670 msgr2=0x7fa72c19b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:26:47.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:47.089+0000 7fa72b400700 1 --2- 172.21.15.33:0/4038110395 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa72c10b670 0x7fa72c19b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:47.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:47.089+0000 7fa72b400700 1 -- 172.21.15.33:0/4038110395 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa72c19c060 con 0x7fa72c105f80 2024-05-03T18:26:47.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:47.089+0000 7fa72b400700 1 --2- 172.21.15.33:0/4038110395 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa72c105f80 0x7fa72c1a1c30 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fa71800d950 tx=0x7fa71800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:26:47.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:47.090+0000 7fa728c00700 1 -- 172.21.15.33:0/4038110395 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa718016070 con 0x7fa72c105f80 2024-05-03T18:26:47.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:47.090+0000 7fa728c00700 1 -- 172.21.15.33:0/4038110395 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa718009dd0 con 0x7fa72c105f80 2024-05-03T18:26:47.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:47.090+0000 7fa728c00700 1 -- 172.21.15.33:0/4038110395 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa71800f3c0 con 0x7fa72c105f80 2024-05-03T18:26:47.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:47.090+0000 7fa731a00700 1 -- 172.21.15.33:0/4038110395 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa72c19c350 con 0x7fa72c105f80 2024-05-03T18:26:47.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:47.090+0000 7fa731a00700 1 -- 172.21.15.33:0/4038110395 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa72c1a85c0 con 0x7fa72c105f80 2024-05-03T18:26:47.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:47.091+0000 7fa731a00700 1 -- 172.21.15.33:0/4038110395 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa72c10db00 con 0x7fa72c105f80 2024-05-03T18:26:47.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:47.094+0000 7fa728c00700 1 -- 172.21.15.33:0/4038110395 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fa718019070 con 0x7fa72c105f80 2024-05-03T18:26:47.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:47.096+0000 7fa728c00700 1 --2- 172.21.15.33:0/4038110395 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fa7100b2990 0x7fa7100b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:26:47.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:47.096+0000 7fa728c00700 1 -- 172.21.15.33:0/4038110395 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7fa7180e2340 con 0x7fa72c105f80 2024-05-03T18:26:47.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:47.096+0000 7fa72be00700 1 --2- 172.21.15.33:0/4038110395 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fa7100b2990 0x7fa7100b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:26:47.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:47.097+0000 7fa72be00700 1 --2- 172.21.15.33:0/4038110395 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fa7100b2990 0x7fa7100b4e50 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa71c009990 tx=0x7fa71c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:26:47.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:47.098+0000 7fa728c00700 1 -- 172.21.15.33:0/4038110395 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fa7180a9b20 con 0x7fa72c105f80 2024-05-03T18:26:47.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:26:46 smithi150 ceph-mon[32492]: pgmap v512: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:26:47.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:47.390+0000 7fa731a00700 1 -- 172.21.15.33:0/4038110395 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa72c06a190 con 0x7fa72c105f80 2024-05-03T18:26:47.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:47.392+0000 7fa728c00700 1 -- 172.21.15.33:0/4038110395 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11801 (secure 0 0 0) 0x7fa7180a9420 con 0x7fa72c105f80 2024-05-03T18:26:47.394 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:26:47.394 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1327393503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1327393503},{"type":"v1","addr":"172.21.15.33:6835","nonce":1327393503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/2461472948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":2461472948},{"type":"v1","addr":"172.21.15.33:6837","nonce":2461472948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14559,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/442269281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":442269281},{"type":"v1","addr":"172.21.15.33:6839","nonce":442269281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14571,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/645571756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":645571756},{"type":"v1","addr":"172.21.15.33:6841","nonce":645571756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/203598909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":203598909},{"type":"v1","addr":"172.21.15.150:6833","nonce":203598909}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/4029842493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":4029842493},{"type":"v1","addr":"172.21.15.150:6835","nonce":4029842493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3463664423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3463664423},{"type":"v1","addr":"172.21.15.150:6837","nonce":3463664423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24481,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6839/4051648018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":4051648018},{"type":"v1","addr":"172.21.15.150:6839","nonce":4051648018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34418,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1865105479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1865105479},{"type":"v1","addr":"172.21.15.99:6833","nonce":1865105479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34445,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/1206393914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1206393914},{"type":"v1","addr":"172.21.15.99:6835","nonce":1206393914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-03T18:22:28.837091+0000","modified":"2024-05-03T18:23:49.758398+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34508},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34472":{"gid":34472,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6837/3360970342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3360970342},{"type":"v1","addr":"172.21.15.99:6837","nonce":3360970342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34508":{"gid":34508,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/919907308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":919907308},{"type":"v1","addr":"172.21.15.99:6839","nonce":919907308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34508,"qdb_cluster":[34508]},"id":1}]} 2024-05-03T18:26:47.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:47.396+0000 7fa731a00700 1 -- 172.21.15.33:0/4038110395 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fa7100b2990 msgr2=0x7fa7100b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:26:47.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:47.396+0000 7fa731a00700 1 --2- 172.21.15.33:0/4038110395 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fa7100b2990 0x7fa7100b4e50 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa71c009990 tx=0x7fa71c01a040 comp rx=0 tx=0).stop 2024-05-03T18:26:47.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:47.396+0000 7fa731a00700 1 -- 172.21.15.33:0/4038110395 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa72c105f80 msgr2=0x7fa72c1a1c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:26:47.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:47.396+0000 7fa731a00700 1 --2- 172.21.15.33:0/4038110395 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa72c105f80 0x7fa72c1a1c30 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fa71800d950 tx=0x7fa71800dc60 comp rx=0 tx=0).stop 2024-05-03T18:26:47.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:47.396+0000 7fa731a00700 1 -- 172.21.15.33:0/4038110395 shutdown_connections 2024-05-03T18:26:47.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:47.396+0000 7fa731a00700 1 --2- 172.21.15.33:0/4038110395 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fa7100b2990 0x7fa7100b4e50 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:47.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:47.396+0000 7fa731a00700 1 --2- 172.21.15.33:0/4038110395 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa72c10b670 0x7fa72c19b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:47.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:47.396+0000 7fa731a00700 1 --2- 172.21.15.33:0/4038110395 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa72c1055c0 0x7fa72c1a16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:47.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:47.396+0000 7fa731a00700 1 --2- 172.21.15.33:0/4038110395 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa72c105f80 0x7fa72c1a1c30 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:47.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:47.396+0000 7fa731a00700 1 -- 172.21.15.33:0/4038110395 >> 172.21.15.33:0/4038110395 conn(0x7fa72c1010d0 msgr2=0x7fa72c102910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:26:47.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:47.396+0000 7fa731a00700 1 -- 172.21.15.33:0/4038110395 shutdown_connections 2024-05-03T18:26:47.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:47.396+0000 7fa731a00700 1 -- 172.21.15.33:0/4038110395 wait complete. 2024-05-03T18:26:47.399 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 21 2024-05-03T18:26:47.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:26:47 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/4038110395' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:26:48.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:26:47 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/4038110395' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:26:48.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:26:47 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/4038110395' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:26:48.294 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T18:26:48.769 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:26:48.919 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:26:48 smithi033 ceph-mon[28330]: pgmap v513: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:26:49.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:26:48 smithi099 ceph-mon[31658]: pgmap v513: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:26:49.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:26:48 smithi150 ceph-mon[32492]: pgmap v513: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:26:50.767 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:26:50 smithi033 ceph-mon[28330]: pgmap v514: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:26:51.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:26:50 smithi099 ceph-mon[31658]: pgmap v514: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:26:51.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:51.073+0000 7f7d08c00700 1 -- 172.21.15.33:0/2863077785 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d0410b660 msgr2=0x7f7d0410ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:26:51.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:51.073+0000 7f7d08c00700 1 --2- 172.21.15.33:0/2863077785 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d0410b660 0x7f7d0410ba40 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f7cf4009990 tx=0x7f7cf4009ca0 comp rx=0 tx=0).stop 2024-05-03T18:26:51.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:51.073+0000 7f7d08c00700 1 -- 172.21.15.33:0/2863077785 shutdown_connections 2024-05-03T18:26:51.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:51.073+0000 7f7d08c00700 1 --2- 172.21.15.33:0/2863077785 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7d04105f90 0x7f7d041124a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:51.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:51.073+0000 7f7d08c00700 1 --2- 172.21.15.33:0/2863077785 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d0410b660 0x7f7d0410ba40 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:51.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:51.073+0000 7f7d08c00700 1 --2- 172.21.15.33:0/2863077785 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d041055d0 0x7f7d04105a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:51.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:51.073+0000 7f7d08c00700 1 -- 172.21.15.33:0/2863077785 >> 172.21.15.33:0/2863077785 conn(0x7f7d04101080 msgr2=0x7f7d041034a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:26:51.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:51.073+0000 7f7d08c00700 1 -- 172.21.15.33:0/2863077785 shutdown_connections 2024-05-03T18:26:51.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:51.074+0000 7f7d08c00700 1 -- 172.21.15.33:0/2863077785 wait complete. 2024-05-03T18:26:51.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:51.075+0000 7f7d08c00700 1 Processor -- start 2024-05-03T18:26:51.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:51.075+0000 7f7d08c00700 1 -- start start 2024-05-03T18:26:51.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:51.076+0000 7f7d08c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d041055d0 0x7f7d041a1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:26:51.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:51.076+0000 7f7d08c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7d04105f90 0x7f7d041a1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:26:51.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:51.076+0000 7f7d08c00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d0410b660 0x7f7d0419b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:26:51.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:51.076+0000 7f7d08c00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7d04115660 con 0x7f7d0410b660 2024-05-03T18:26:51.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:51.076+0000 7f7d08c00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7d041154e0 con 0x7f7d04105f90 2024-05-03T18:26:51.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:51.076+0000 7f7d08c00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f7d041157e0 con 0x7f7d041055d0 2024-05-03T18:26:51.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:51.076+0000 7f7d03e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d0410b660 0x7f7d0419b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:26:51.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:51.076+0000 7f7d03e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d0410b660 0x7f7d0419b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33216/0 (socket says 172.21.15.33:33216) 2024-05-03T18:26:51.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:51.076+0000 7f7d03e00700 1 -- 172.21.15.33:0/1488542095 learned_addr learned my addr 172.21.15.33:0/1488542095 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:26:51.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:51.077+0000 7f7d03400700 1 --2- 172.21.15.33:0/1488542095 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d041055d0 0x7f7d041a1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:26:51.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:51.077+0000 7f7d02a00700 1 --2- 172.21.15.33:0/1488542095 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7d04105f90 0x7f7d041a1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:26:51.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:51.077+0000 7f7d03e00700 1 -- 172.21.15.33:0/1488542095 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d041055d0 msgr2=0x7f7d041a1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:26:51.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:51.077+0000 7f7d03e00700 1 --2- 172.21.15.33:0/1488542095 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d041055d0 0x7f7d041a1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:51.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:51.077+0000 7f7d03e00700 1 -- 172.21.15.33:0/1488542095 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7d04105f90 msgr2=0x7f7d041a1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:26:51.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:51.077+0000 7f7d03e00700 1 --2- 172.21.15.33:0/1488542095 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7d04105f90 0x7f7d041a1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:51.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:51.077+0000 7f7d03e00700 1 -- 172.21.15.33:0/1488542095 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d0419c0a0 con 0x7f7d0410b660 2024-05-03T18:26:51.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:51.077+0000 7f7d02a00700 1 --2- 172.21.15.33:0/1488542095 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7d04105f90 0x7f7d041a1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:26:51.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:51.077+0000 7f7d03e00700 1 --2- 172.21.15.33:0/1488542095 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d0410b660 0x7f7d0419b940 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc00ef30 tx=0x7f7cfc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:26:51.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:51.078+0000 7f7cfbe00700 1 -- 172.21.15.33:0/1488542095 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cfc0092e0 con 0x7f7d0410b660 2024-05-03T18:26:51.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:51.078+0000 7f7cfbe00700 1 -- 172.21.15.33:0/1488542095 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7cfc004bb0 con 0x7f7d0410b660 2024-05-03T18:26:51.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:51.078+0000 7f7d08c00700 1 -- 172.21.15.33:0/1488542095 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d0419c330 con 0x7f7d0410b660 2024-05-03T18:26:51.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:51.078+0000 7f7cfbe00700 1 -- 172.21.15.33:0/1488542095 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cfc004020 con 0x7f7d0410b660 2024-05-03T18:26:51.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:51.078+0000 7f7d08c00700 1 -- 172.21.15.33:0/1488542095 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d041a8650 con 0x7f7d0410b660 2024-05-03T18:26:51.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:51.079+0000 7f7d08c00700 1 -- 172.21.15.33:0/1488542095 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d0410db60 con 0x7f7d0410b660 2024-05-03T18:26:51.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:51.083+0000 7f7cfbe00700 1 -- 172.21.15.33:0/1488542095 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f7cfc01b030 con 0x7f7d0410b660 2024-05-03T18:26:51.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:51.085+0000 7f7cfbe00700 1 --2- 172.21.15.33:0/1488542095 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f7ce80b2910 0x7f7ce80b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:26:51.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:51.085+0000 7f7cfbe00700 1 -- 172.21.15.33:0/1488542095 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f7cfc0e2cd0 con 0x7f7d0410b660 2024-05-03T18:26:51.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:51.085+0000 7f7d03400700 1 --2- 172.21.15.33:0/1488542095 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f7ce80b2910 0x7f7ce80b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:26:51.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:51.086+0000 7f7d03400700 1 --2- 172.21.15.33:0/1488542095 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f7ce80b2910 0x7f7ce80b4dd0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7cf400b650 tx=0x7f7cf401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:26:51.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:51.087+0000 7f7cfbe00700 1 -- 172.21.15.33:0/1488542095 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f7cfc022020 con 0x7f7d0410b660 2024-05-03T18:26:51.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:26:50 smithi150 ceph-mon[32492]: pgmap v514: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:26:51.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:51.368+0000 7f7d08c00700 1 -- 172.21.15.33:0/1488542095 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7d0406a190 con 0x7f7d0410b660 2024-05-03T18:26:51.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:51.369+0000 7f7cfbe00700 1 -- 172.21.15.33:0/1488542095 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11801 (secure 0 0 0) 0x7f7cfc010040 con 0x7f7d0410b660 2024-05-03T18:26:51.371 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:26:51.371 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1327393503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1327393503},{"type":"v1","addr":"172.21.15.33:6835","nonce":1327393503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/2461472948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":2461472948},{"type":"v1","addr":"172.21.15.33:6837","nonce":2461472948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14559,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/442269281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":442269281},{"type":"v1","addr":"172.21.15.33:6839","nonce":442269281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14571,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/645571756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":645571756},{"type":"v1","addr":"172.21.15.33:6841","nonce":645571756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/203598909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":203598909},{"type":"v1","addr":"172.21.15.150:6833","nonce":203598909}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/4029842493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":4029842493},{"type":"v1","addr":"172.21.15.150:6835","nonce":4029842493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3463664423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3463664423},{"type":"v1","addr":"172.21.15.150:6837","nonce":3463664423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24481,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6839/4051648018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":4051648018},{"type":"v1","addr":"172.21.15.150:6839","nonce":4051648018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34418,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1865105479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1865105479},{"type":"v1","addr":"172.21.15.99:6833","nonce":1865105479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34445,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/1206393914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1206393914},{"type":"v1","addr":"172.21.15.99:6835","nonce":1206393914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-03T18:22:28.837091+0000","modified":"2024-05-03T18:23:49.758398+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34508},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34472":{"gid":34472,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6837/3360970342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3360970342},{"type":"v1","addr":"172.21.15.99:6837","nonce":3360970342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34508":{"gid":34508,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/919907308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":919907308},{"type":"v1","addr":"172.21.15.99:6839","nonce":919907308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34508,"qdb_cluster":[34508]},"id":1}]} 2024-05-03T18:26:51.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:51.372+0000 7f7d08c00700 1 -- 172.21.15.33:0/1488542095 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f7ce80b2910 msgr2=0x7f7ce80b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:26:51.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:51.372+0000 7f7d08c00700 1 --2- 172.21.15.33:0/1488542095 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f7ce80b2910 0x7f7ce80b4dd0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7cf400b650 tx=0x7f7cf401a040 comp rx=0 tx=0).stop 2024-05-03T18:26:51.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:51.372+0000 7f7d08c00700 1 -- 172.21.15.33:0/1488542095 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d0410b660 msgr2=0x7f7d0419b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:26:51.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:51.372+0000 7f7d08c00700 1 --2- 172.21.15.33:0/1488542095 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d0410b660 0x7f7d0419b940 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc00ef30 tx=0x7f7cfc00c3a0 comp rx=0 tx=0).stop 2024-05-03T18:26:51.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:51.373+0000 7f7d08c00700 1 -- 172.21.15.33:0/1488542095 shutdown_connections 2024-05-03T18:26:51.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:51.373+0000 7f7d08c00700 1 --2- 172.21.15.33:0/1488542095 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d0410b660 0x7f7d0419b940 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:51.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:51.373+0000 7f7d08c00700 1 --2- 172.21.15.33:0/1488542095 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d041055d0 0x7f7d041a1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:51.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:51.373+0000 7f7d08c00700 1 --2- 172.21.15.33:0/1488542095 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f7ce80b2910 0x7f7ce80b4dd0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:51.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:51.373+0000 7f7d08c00700 1 --2- 172.21.15.33:0/1488542095 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7d04105f90 0x7f7d041a1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:51.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:51.373+0000 7f7d08c00700 1 -- 172.21.15.33:0/1488542095 >> 172.21.15.33:0/1488542095 conn(0x7f7d04101080 msgr2=0x7f7d041028f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:26:51.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:51.373+0000 7f7d08c00700 1 -- 172.21.15.33:0/1488542095 shutdown_connections 2024-05-03T18:26:51.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:51.373+0000 7f7d08c00700 1 -- 172.21.15.33:0/1488542095 wait complete. 2024-05-03T18:26:51.375 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 21 2024-05-03T18:26:51.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:26:51 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/1488542095' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:26:52.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:26:51 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/1488542095' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:26:52.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:26:51 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/1488542095' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:26:52.329 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-03T18:26:52.801 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:26:52.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:26:52 smithi033 ceph-mon[28330]: pgmap v515: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:26:53.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:26:52 smithi099 ceph-mon[31658]: pgmap v515: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:26:53.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:26:52 smithi150 ceph-mon[32492]: pgmap v515: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:26:54.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:26:54 smithi033 ceph-mon[28330]: pgmap v516: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:26:55.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:26:54 smithi099 ceph-mon[31658]: pgmap v516: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:26:55.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:26:54 smithi150 ceph-mon[32492]: pgmap v516: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:26:55.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:55.258+0000 7f0614a00700 1 -- 172.21.15.33:0/2080605419 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0610109540 msgr2=0x7f0610109920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:26:55.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:55.258+0000 7f0614a00700 1 --2- 172.21.15.33:0/2080605419 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0610109540 0x7f0610109920 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f0600009990 tx=0x7f0600009ca0 comp rx=0 tx=0).stop 2024-05-03T18:26:55.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:55.258+0000 7f0614a00700 1 -- 172.21.15.33:0/2080605419 shutdown_connections 2024-05-03T18:26:55.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:55.258+0000 7f0614a00700 1 --2- 172.21.15.33:0/2080605419 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f061006c530 0x7f0610110360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:55.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:55.258+0000 7f0614a00700 1 --2- 172.21.15.33:0/2080605419 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0610109540 0x7f0610109920 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:55.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:55.258+0000 7f0614a00700 1 --2- 172.21.15.33:0/2080605419 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f061006bb70 0x7f061006bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:55.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:55.258+0000 7f0614a00700 1 -- 172.21.15.33:0/2080605419 >> 172.21.15.33:0/2080605419 conn(0x7f06100793e0 msgr2=0x7f06100797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:26:55.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:55.258+0000 7f0614a00700 1 -- 172.21.15.33:0/2080605419 shutdown_connections 2024-05-03T18:26:55.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:55.258+0000 7f0614a00700 1 -- 172.21.15.33:0/2080605419 wait complete. 2024-05-03T18:26:55.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:55.260+0000 7f0614a00700 1 Processor -- start 2024-05-03T18:26:55.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:55.260+0000 7f0614a00700 1 -- start start 2024-05-03T18:26:55.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:55.261+0000 7f0614a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f061006bb70 0x7f06101024a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:26:55.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:55.261+0000 7f0614a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f061006c530 0x7f06101029e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:26:55.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:55.261+0000 7f0614a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0610109540 0x7f0610102f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:26:55.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:55.261+0000 7f0614a00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0610113570 con 0x7f061006c530 2024-05-03T18:26:55.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:55.261+0000 7f0614a00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f06101133f0 con 0x7f0610109540 2024-05-03T18:26:55.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:55.261+0000 7f0614a00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f06101136f0 con 0x7f061006bb70 2024-05-03T18:26:55.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:55.261+0000 7f0607e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f061006c530 0x7f06101029e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:26:55.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:55.261+0000 7f0607e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f061006c530 0x7f06101029e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33236/0 (socket says 172.21.15.33:33236) 2024-05-03T18:26:55.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:55.261+0000 7f0607e00700 1 -- 172.21.15.33:0/2931270529 learned_addr learned my addr 172.21.15.33:0/2931270529 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:26:55.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:55.261+0000 7f060de00700 1 --2- 172.21.15.33:0/2931270529 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f061006bb70 0x7f06101024a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:26:55.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:55.261+0000 7f060e800700 1 --2- 172.21.15.33:0/2931270529 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0610109540 0x7f0610102f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:26:55.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:55.262+0000 7f0607e00700 1 -- 172.21.15.33:0/2931270529 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f061006bb70 msgr2=0x7f06101024a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:26:55.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:55.262+0000 7f0607e00700 1 --2- 172.21.15.33:0/2931270529 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f061006bb70 0x7f06101024a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:55.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:55.262+0000 7f0607e00700 1 -- 172.21.15.33:0/2931270529 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0610109540 msgr2=0x7f0610102f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:26:55.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:55.262+0000 7f0607e00700 1 --2- 172.21.15.33:0/2931270529 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0610109540 0x7f0610102f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:55.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:55.262+0000 7f0607e00700 1 -- 172.21.15.33:0/2931270529 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06101a80f0 con 0x7f061006c530 2024-05-03T18:26:55.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:55.262+0000 7f0607e00700 1 --2- 172.21.15.33:0/2931270529 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f061006c530 0x7f06101029e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f05f800c9b0 tx=0x7f05f800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:26:55.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:55.263+0000 7f0606a00700 1 -- 172.21.15.33:0/2931270529 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f8007b00 con 0x7f061006c530 2024-05-03T18:26:55.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:55.263+0000 7f0606a00700 1 -- 172.21.15.33:0/2931270529 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f05f800ce50 con 0x7f061006c530 2024-05-03T18:26:55.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:55.263+0000 7f0614a00700 1 -- 172.21.15.33:0/2931270529 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06101a83e0 con 0x7f061006c530 2024-05-03T18:26:55.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:55.263+0000 7f0606a00700 1 -- 172.21.15.33:0/2931270529 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f8018660 con 0x7f061006c530 2024-05-03T18:26:55.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:55.263+0000 7f0614a00700 1 -- 172.21.15.33:0/2931270529 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06101a8970 con 0x7f061006c530 2024-05-03T18:26:55.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:55.264+0000 7f0614a00700 1 -- 172.21.15.33:0/2931270529 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f061010ba20 con 0x7f061006c530 2024-05-03T18:26:55.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:55.267+0000 7f0606a00700 1 -- 172.21.15.33:0/2931270529 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f05f800f450 con 0x7f061006c530 2024-05-03T18:26:55.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:55.269+0000 7f0606a00700 1 --2- 172.21.15.33:0/2931270529 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f05f40b2990 0x7f05f40b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:26:55.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:55.269+0000 7f0606a00700 1 -- 172.21.15.33:0/2931270529 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f05f80e28e0 con 0x7f061006c530 2024-05-03T18:26:55.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:55.269+0000 7f060de00700 1 --2- 172.21.15.33:0/2931270529 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f05f40b2990 0x7f05f40b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:26:55.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:55.270+0000 7f060de00700 1 --2- 172.21.15.33:0/2931270529 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f05f40b2990 0x7f05f40b4e50 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0600006010 tx=0x7f060000b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:26:55.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:55.271+0000 7f0606a00700 1 -- 172.21.15.33:0/2931270529 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f05f8014030 con 0x7f061006c530 2024-05-03T18:26:55.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:55.506+0000 7f0614a00700 1 -- 172.21.15.33:0/2931270529 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f06101039d0 con 0x7f061006c530 2024-05-03T18:26:55.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:55.510+0000 7f0606a00700 1 -- 172.21.15.33:0/2931270529 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24650 (secure 0 0 0) 0x7f05f80aa1d0 con 0x7f061006c530 2024-05-03T18:26:55.511 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:26:55.511 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":89,"fsid":"5703bba4-0978-11ef-bc97-c7b262605968","created":"2024-05-03T18:11:00.899879+0000","modified":"2024-05-03T18:24:21.532082+0000","last_up_change":"2024-05-03T18:21:00.390173+0000","last_in_change":"2024-05-03T18:20:34.103727+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-03T18:15:37.297738+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-03T18:22:19.875368+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-03T18:22:24.432692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-03T18:22:38.722443+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"fdcc4e66-4ba0-4378-a9a4-7d907cd00b4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":637686669},{"type":"v1","addr":"172.21.15.33:6803","nonce":637686669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":637686669},{"type":"v1","addr":"172.21.15.33:6805","nonce":637686669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":637686669},{"type":"v1","addr":"172.21.15.33:6809","nonce":637686669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":637686669},{"type":"v1","addr":"172.21.15.33:6807","nonce":637686669}]},"public_addr":"172.21.15.33:6803/637686669","cluster_addr":"172.21.15.33:6805/637686669","heartbeat_back_addr":"172.21.15.33:6809/637686669","heartbeat_front_addr":"172.21.15.33:6807/637686669","state":["exists","up"]},{"osd":1,"uuid":"cf78cad3-fc24-43f4-8784-21cabea7277c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":1536122841},{"type":"v1","addr":"172.21.15.99:6801","nonce":1536122841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":1536122841},{"type":"v1","addr":"172.21.15.99:6803","nonce":1536122841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":1536122841},{"type":"v1","addr":"172.21.15.99:6807","nonce":1536122841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":1536122841},{"type":"v1","addr":"172.21.15.99:6805","nonce":1536122841}]},"public_addr":"172.21.15.99:6801/1536122841","cluster_addr":"172.21.15.99:6803/1536122841","heartbeat_back_addr":"172.21.15.99:6807/1536122841","heartbeat_front_addr":"172.21.15.99:6805/1536122841","state":["exists","up"]},{"osd":2,"uuid":"59f066d7-626b-410a-8d67-daca6fbbd3ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":1995659622},{"type":"v1","addr":"172.21.15.150:6801","nonce":1995659622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":1995659622},{"type":"v1","addr":"172.21.15.150:6803","nonce":1995659622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":1995659622},{"type":"v1","addr":"172.21.15.150:6807","nonce":1995659622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":1995659622},{"type":"v1","addr":"172.21.15.150:6805","nonce":1995659622}]},"public_addr":"172.21.15.150:6801/1995659622","cluster_addr":"172.21.15.150:6803/1995659622","heartbeat_back_addr":"172.21.15.150:6807/1995659622","heartbeat_front_addr":"172.21.15.150:6805/1995659622","state":["exists","up"]},{"osd":3,"uuid":"d88ed022-4dcb-45fc-9fc6-41ef7c4bf202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":3814942724},{"type":"v1","addr":"172.21.15.33:6811","nonce":3814942724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":3814942724},{"type":"v1","addr":"172.21.15.33:6813","nonce":3814942724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":3814942724},{"type":"v1","addr":"172.21.15.33:6817","nonce":3814942724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":3814942724},{"type":"v1","addr":"172.21.15.33:6815","nonce":3814942724}]},"public_addr":"172.21.15.33:6811/3814942724","cluster_addr":"172.21.15.33:6813/3814942724","heartbeat_back_addr":"172.21.15.33:6817/3814942724","heartbeat_front_addr":"172.21.15.33:6815/3814942724","state":["exists","up"]},{"osd":4,"uuid":"1235e1a6-ff6e-4668-ac1a-073c2a3b7c98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":29081029},{"type":"v1","addr":"172.21.15.99:6809","nonce":29081029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":29081029},{"type":"v1","addr":"172.21.15.99:6811","nonce":29081029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":29081029},{"type":"v1","addr":"172.21.15.99:6815","nonce":29081029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":29081029},{"type":"v1","addr":"172.21.15.99:6813","nonce":29081029}]},"public_addr":"172.21.15.99:6809/29081029","cluster_addr":"172.21.15.99:6811/29081029","heartbeat_back_addr":"172.21.15.99:6815/29081029","heartbeat_front_addr":"172.21.15.99:6813/29081029","state":["exists","up"]},{"osd":5,"uuid":"4b84bff8-d6d7-4258-a0ee-53bc19802585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":3172189533},{"type":"v1","addr":"172.21.15.150:6809","nonce":3172189533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3172189533},{"type":"v1","addr":"172.21.15.150:6811","nonce":3172189533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3172189533},{"type":"v1","addr":"172.21.15.150:6815","nonce":3172189533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3172189533},{"type":"v1","addr":"172.21.15.150:6813","nonce":3172189533}]},"public_addr":"172.21.15.150:6809/3172189533","cluster_addr":"172.21.15.150:6811/3172189533","heartbeat_back_addr":"172.21.15.150:6815/3172189533","heartbeat_front_addr":"172.21.15.150:6813/3172189533","state":["exists","up"]},{"osd":6,"uuid":"51f5fbdd-3794-4c48-9f6c-478f0fa35ac5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":4234541418},{"type":"v1","addr":"172.21.15.33:6819","nonce":4234541418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":4234541418},{"type":"v1","addr":"172.21.15.33:6821","nonce":4234541418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":4234541418},{"type":"v1","addr":"172.21.15.33:6825","nonce":4234541418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":4234541418},{"type":"v1","addr":"172.21.15.33:6823","nonce":4234541418}]},"public_addr":"172.21.15.33:6819/4234541418","cluster_addr":"172.21.15.33:6821/4234541418","heartbeat_back_addr":"172.21.15.33:6825/4234541418","heartbeat_front_addr":"172.21.15.33:6823/4234541418","state":["exists","up"]},{"osd":7,"uuid":"982cc5b3-3163-4505-9bde-a27a8c289457","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":908673090},{"type":"v1","addr":"172.21.15.99:6817","nonce":908673090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":908673090},{"type":"v1","addr":"172.21.15.99:6819","nonce":908673090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":908673090},{"type":"v1","addr":"172.21.15.99:6823","nonce":908673090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":908673090},{"type":"v1","addr":"172.21.15.99:6821","nonce":908673090}]},"public_addr":"172.21.15.99:6817/908673090","cluster_addr":"172.21.15.99:6819/908673090","heartbeat_back_addr":"172.21.15.99:6823/908673090","heartbeat_front_addr":"172.21.15.99:6821/908673090","state":["exists","up"]},{"osd":8,"uuid":"84c9d037-eb84-4eb4-8aa2-e784b703d36c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":1308083602},{"type":"v1","addr":"172.21.15.150:6817","nonce":1308083602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":1308083602},{"type":"v1","addr":"172.21.15.150:6819","nonce":1308083602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":1308083602},{"type":"v1","addr":"172.21.15.150:6823","nonce":1308083602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":1308083602},{"type":"v1","addr":"172.21.15.150:6821","nonce":1308083602}]},"public_addr":"172.21.15.150:6817/1308083602","cluster_addr":"172.21.15.150:6819/1308083602","heartbeat_back_addr":"172.21.15.150:6823/1308083602","heartbeat_front_addr":"172.21.15.150:6821/1308083602","state":["exists","up"]},{"osd":9,"uuid":"a02e965f-0d8f-45e2-9490-337f7f6e3881","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":4133096905},{"type":"v1","addr":"172.21.15.33:6827","nonce":4133096905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":4133096905},{"type":"v1","addr":"172.21.15.33:6829","nonce":4133096905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4133096905},{"type":"v1","addr":"172.21.15.33:6833","nonce":4133096905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":4133096905},{"type":"v1","addr":"172.21.15.33:6831","nonce":4133096905}]},"public_addr":"172.21.15.33:6827/4133096905","cluster_addr":"172.21.15.33:6829/4133096905","heartbeat_back_addr":"172.21.15.33:6833/4133096905","heartbeat_front_addr":"172.21.15.33:6831/4133096905","state":["exists","up"]},{"osd":10,"uuid":"8b29bb96-76e8-45a0-ab19-8d082d755fe9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":935746136},{"type":"v1","addr":"172.21.15.99:6825","nonce":935746136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":935746136},{"type":"v1","addr":"172.21.15.99:6827","nonce":935746136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":935746136},{"type":"v1","addr":"172.21.15.99:6831","nonce":935746136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":935746136},{"type":"v1","addr":"172.21.15.99:6829","nonce":935746136}]},"public_addr":"172.21.15.99:6825/935746136","cluster_addr":"172.21.15.99:6827/935746136","heartbeat_back_addr":"172.21.15.99:6831/935746136","heartbeat_front_addr":"172.21.15.99:6829/935746136","state":["exists","up"]},{"osd":11,"uuid":"07a6a825-c902-4bfd-97cc-cc9074532af5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2069349808},{"type":"v1","addr":"172.21.15.150:6825","nonce":2069349808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2069349808},{"type":"v1","addr":"172.21.15.150:6827","nonce":2069349808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2069349808},{"type":"v1","addr":"172.21.15.150:6831","nonce":2069349808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2069349808},{"type":"v1","addr":"172.21.15.150:6829","nonce":2069349808}]},"public_addr":"172.21.15.150:6825/2069349808","cluster_addr":"172.21.15.150:6827/2069349808","heartbeat_back_addr":"172.21.15.150:6831/2069349808","heartbeat_front_addr":"172.21.15.150:6829/2069349808","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:14:27.454536+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:15:00.538570+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:15:34.125289+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:16:09.853870+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:16:44.928107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:17:19.644037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:17:56.108443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:18:31.731499+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:19:08.062358+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:19:45.204430+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:20:21.013710+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:20:57.653698+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11},{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/3991947997":"2024-05-04T18:12:19.127652+0000","172.21.15.33:0/965859483":"2024-05-04T18:12:19.127652+0000","172.21.15.33:0/2014902144":"2024-05-04T18:11:44.239835+0000","172.21.15.33:0/1314704208":"2024-05-04T18:11:44.239835+0000","172.21.15.33:0/1614640996":"2024-05-04T18:11:44.239835+0000","172.21.15.33:6800/4007322071":"2024-05-04T18:11:44.239835+0000","172.21.15.33:6801/2830135294":"2024-05-04T18:12:19.127652+0000","172.21.15.33:6800/2830135294":"2024-05-04T18:12:19.127652+0000","172.21.15.33:0/3767665033":"2024-05-04T18:12:19.127652+0000","172.21.15.33:6801/4007322071":"2024-05-04T18:11:44.239835+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-03T18:26:55.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:55.514+0000 7f0614a00700 1 -- 172.21.15.33:0/2931270529 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f05f40b2990 msgr2=0x7f05f40b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:26:55.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:55.514+0000 7f0614a00700 1 --2- 172.21.15.33:0/2931270529 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f05f40b2990 0x7f05f40b4e50 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0600006010 tx=0x7f060000b540 comp rx=0 tx=0).stop 2024-05-03T18:26:55.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:55.514+0000 7f0614a00700 1 -- 172.21.15.33:0/2931270529 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f061006c530 msgr2=0x7f06101029e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:26:55.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:55.514+0000 7f0614a00700 1 --2- 172.21.15.33:0/2931270529 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f061006c530 0x7f06101029e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f05f800c9b0 tx=0x7f05f800ccc0 comp rx=0 tx=0).stop 2024-05-03T18:26:55.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:55.514+0000 7f0614a00700 1 -- 172.21.15.33:0/2931270529 shutdown_connections 2024-05-03T18:26:55.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:55.514+0000 7f0614a00700 1 --2- 172.21.15.33:0/2931270529 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f05f40b2990 0x7f05f40b4e50 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:55.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:55.514+0000 7f0614a00700 1 --2- 172.21.15.33:0/2931270529 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0610109540 0x7f0610102f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:55.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:55.514+0000 7f0614a00700 1 --2- 172.21.15.33:0/2931270529 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f061006bb70 0x7f06101024a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:55.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:55.514+0000 7f0614a00700 1 --2- 172.21.15.33:0/2931270529 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f061006c530 0x7f06101029e0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:55.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:55.514+0000 7f0614a00700 1 -- 172.21.15.33:0/2931270529 >> 172.21.15.33:0/2931270529 conn(0x7f06100793e0 msgr2=0x7f0610101240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:26:55.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:55.515+0000 7f0614a00700 1 -- 172.21.15.33:0/2931270529 shutdown_connections 2024-05-03T18:26:55.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:55.515+0000 7f0614a00700 1 -- 172.21.15.33:0/2931270529 wait complete. 2024-05-03T18:26:55.907 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:26:55 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/2931270529' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T18:26:56.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:26:55 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/2931270529' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T18:26:56.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:26:55 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/2931270529' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T18:26:56.348 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 21, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-03T18:22:28.837091+0000', 'modified': '2024-05-03T18:23:49.758398+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 34508}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_34472': {'gid': 34472, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.99:6837/3360970342', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6836', 'nonce': 3360970342}, {'type': 'v1', 'addr': '172.21.15.99:6837', 'nonce': 3360970342}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34508': {'gid': 34508, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.99:6839/919907308', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6838', 'nonce': 919907308}, {'type': 'v1', 'addr': '172.21.15.99:6839', 'nonce': 919907308}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 34508, 'qdb_cluster': [34508]} 2024-05-03T18:26:56.349 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-03T18:26:56.349 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.b status 2024-05-03T18:26:56.826 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:26:56.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:26:56 smithi033 ceph-mon[28330]: pgmap v517: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:26:57.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:26:56 smithi099 ceph-mon[31658]: pgmap v517: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:26:57.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:26:56 smithi150 ceph-mon[32492]: pgmap v517: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:26:58.988 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:26:58 smithi033 ceph-mon[28330]: pgmap v518: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:26:59.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:26:58 smithi099 ceph-mon[31658]: pgmap v518: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:26:59.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:26:58 smithi150 ceph-mon[32492]: pgmap v518: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:26:59.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.280+0000 7f45eca00700 1 -- 172.21.15.33:0/3144408107 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e0003b40 con 0x7f45f010b6e0 2024-05-03T18:26:59.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.281+0000 7f45f5200700 1 -- 172.21.15.33:0/3144408107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45f010b6e0 msgr2=0x7f45f010bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:26:59.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.281+0000 7f45f5200700 1 --2- 172.21.15.33:0/3144408107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45f010b6e0 0x7f45f010bac0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f45e0009990 tx=0x7f45e0009ca0 comp rx=0 tx=0).stop 2024-05-03T18:26:59.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.281+0000 7f45f5200700 1 -- 172.21.15.33:0/3144408107 shutdown_connections 2024-05-03T18:26:59.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.281+0000 7f45f5200700 1 --2- 172.21.15.33:0/3144408107 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45f01060a0 0x7f45f01125b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:59.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.281+0000 7f45f5200700 1 --2- 172.21.15.33:0/3144408107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45f010b6e0 0x7f45f010bac0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:59.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.281+0000 7f45f5200700 1 --2- 172.21.15.33:0/3144408107 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f45f01056e0 0x7f45f0105b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:59.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.281+0000 7f45f5200700 1 -- 172.21.15.33:0/3144408107 >> 172.21.15.33:0/3144408107 conn(0x7f45f01011d0 msgr2=0x7f45f01035f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:26:59.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.281+0000 7f45f5200700 1 -- 172.21.15.33:0/3144408107 shutdown_connections 2024-05-03T18:26:59.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.282+0000 7f45f5200700 1 -- 172.21.15.33:0/3144408107 wait complete. 2024-05-03T18:26:59.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.283+0000 7f45f5200700 1 Processor -- start 2024-05-03T18:26:59.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.283+0000 7f45f5200700 1 -- start start 2024-05-03T18:26:59.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.283+0000 7f45f5200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45f01056e0 0x7f45f01a18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:26:59.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.284+0000 7f45f5200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45f01060a0 0x7f45f01a1e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:26:59.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.284+0000 7f45f5200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f45f010b6e0 0x7f45f019baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:26:59.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.284+0000 7f45f5200700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f45f0115790 con 0x7f45f01060a0 2024-05-03T18:26:59.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.284+0000 7f45f5200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f45f0115610 con 0x7f45f01056e0 2024-05-03T18:26:59.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.284+0000 7f45f5200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f45f0115910 con 0x7f45f010b6e0 2024-05-03T18:26:59.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.284+0000 7f45ed400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45f01060a0 0x7f45f01a1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:26:59.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.284+0000 7f45ed400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45f01060a0 0x7f45f01a1e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52592/0 (socket says 172.21.15.33:52592) 2024-05-03T18:26:59.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.284+0000 7f45ed400700 1 -- 172.21.15.33:0/1578713272 learned_addr learned my addr 172.21.15.33:0/1578713272 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:26:59.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.284+0000 7f45ede00700 1 --2- 172.21.15.33:0/1578713272 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45f01056e0 0x7f45f01a18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:26:59.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.284+0000 7f45ee800700 1 --2- 172.21.15.33:0/1578713272 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f45f010b6e0 0x7f45f019baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:26:59.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.285+0000 7f45ed400700 1 -- 172.21.15.33:0/1578713272 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f45f010b6e0 msgr2=0x7f45f019baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:26:59.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.285+0000 7f45ed400700 1 --2- 172.21.15.33:0/1578713272 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f45f010b6e0 0x7f45f019baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:59.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.285+0000 7f45ed400700 1 -- 172.21.15.33:0/1578713272 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45f01056e0 msgr2=0x7f45f01a18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:26:59.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.285+0000 7f45ed400700 1 --2- 172.21.15.33:0/1578713272 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45f01056e0 0x7f45f01a18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:59.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.285+0000 7f45ed400700 1 -- 172.21.15.33:0/1578713272 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45f019c250 con 0x7f45f01060a0 2024-05-03T18:26:59.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.285+0000 7f45ed400700 1 --2- 172.21.15.33:0/1578713272 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45f01060a0 0x7f45f01a1e20 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f45dc00d980 tx=0x7f45dc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:26:59.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.285+0000 7f45e6a00700 1 -- 172.21.15.33:0/1578713272 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45dc017070 con 0x7f45f01060a0 2024-05-03T18:26:59.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.285+0000 7f45e6a00700 1 -- 172.21.15.33:0/1578713272 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f45dc005020 con 0x7f45f01060a0 2024-05-03T18:26:59.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.285+0000 7f45e6a00700 1 -- 172.21.15.33:0/1578713272 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45dc00f980 con 0x7f45f01060a0 2024-05-03T18:26:59.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.285+0000 7f45f5200700 1 -- 172.21.15.33:0/1578713272 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45f019c540 con 0x7f45f01060a0 2024-05-03T18:26:59.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.286+0000 7f45f5200700 1 -- 172.21.15.33:0/1578713272 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45f01a87b0 con 0x7f45f01060a0 2024-05-03T18:26:59.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.291+0000 7f45ffaa4b80 1 Processor -- start 2024-05-03T18:26:59.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.291+0000 7f45ffaa4b80 1 -- start start 2024-05-03T18:26:59.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.291+0000 7f45ffaa4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb5657a760 0x55cb5657ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:26:59.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.291+0000 7f45ffaa4b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cb5657b080 0x55cb56575f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:26:59.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.291+0000 7f45ffaa4b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cb56576580 0x55cb565769e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:26:59.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.291+0000 7f45ffaa4b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55cb564c89d0 con 0x55cb56576580 2024-05-03T18:26:59.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.292+0000 7f45ffaa4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55cb56466710 con 0x55cb5657a760 2024-05-03T18:26:59.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.292+0000 7f45ffaa4b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55cb564c4990 con 0x55cb5657b080 2024-05-03T18:26:59.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.292+0000 7f45e6a00700 1 -- 172.21.15.33:0/1578713272 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f45dc004a30 con 0x7f45f01060a0 2024-05-03T18:26:59.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.292+0000 7f45ed400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cb56576580 0x55cb565769e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:26:59.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.292+0000 7f45ee800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb5657a760 0x55cb5657ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:26:59.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.292+0000 7f45ede00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cb5657b080 0x55cb56575f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:26:59.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.292+0000 7f45ee800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb5657a760 0x55cb5657ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:57734/0 (socket says 172.21.15.33:57734) 2024-05-03T18:26:59.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.292+0000 7f45ee800700 1 -- 172.21.15.33:0/3018212397 learned_addr learned my addr 172.21.15.33:0/3018212397 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:26:59.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.292+0000 7f45ed400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cb56576580 0x55cb565769e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52602/0 (socket says 172.21.15.33:52602) 2024-05-03T18:26:59.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.293+0000 7f45ede00700 1 -- 172.21.15.33:0/3018212397 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb5657a760 msgr2=0x55cb5657ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:26:59.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.293+0000 7f45ede00700 1 --2- 172.21.15.33:0/3018212397 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb5657a760 0x55cb5657ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:59.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.293+0000 7f45ede00700 1 -- 172.21.15.33:0/3018212397 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cb56576580 msgr2=0x55cb565769e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:26:59.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.293+0000 7f45ede00700 1 --2- 172.21.15.33:0/3018212397 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cb56576580 0x55cb565769e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:59.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.293+0000 7f45ede00700 1 -- 172.21.15.33:0/3018212397 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb56415880 con 0x55cb5657b080 2024-05-03T18:26:59.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.293+0000 7f45ed400700 1 --2- 172.21.15.33:0/3018212397 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cb56576580 0x55cb565769e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:26:59.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.294+0000 7f45ede00700 1 --2- 172.21.15.33:0/3018212397 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cb5657b080 0x55cb56575f60 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f45e0006010 tx=0x7f45e0009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31f13acde0d32013 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:26:59.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.294+0000 7f45db400700 1 -- 172.21.15.33:0/3018212397 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e000b9d0 con 0x55cb5657b080 2024-05-03T18:26:59.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.294+0000 7f45db400700 1 -- 172.21.15.33:0/3018212397 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f45e0027070 con 0x55cb5657b080 2024-05-03T18:26:59.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.294+0000 7f45db400700 1 -- 172.21.15.33:0/3018212397 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e000fd80 con 0x55cb5657b080 2024-05-03T18:26:59.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.294+0000 7f45e6a00700 1 --2- 172.21.15.33:0/1578713272 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f45d40b2990 0x7f45d40b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:26:59.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.294+0000 7f45e6a00700 1 -- 172.21.15.33:0/1578713272 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f45dc0e2580 con 0x7f45f01060a0 2024-05-03T18:26:59.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.295+0000 7f45ee800700 1 --2- 172.21.15.33:0/1578713272 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f45d40b2990 0x7f45d40b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:26:59.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.295+0000 7f45ffaa4b80 1 -- 172.21.15.33:0/3018212397 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cb5657b080 msgr2=0x55cb56575f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:26:59.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.295+0000 7f45ffaa4b80 1 --2- 172.21.15.33:0/3018212397 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cb5657b080 0x55cb56575f60 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f45e0006010 tx=0x7f45e0009f90 comp rx=0 tx=0).stop 2024-05-03T18:26:59.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.295+0000 7f45ffaa4b80 1 -- 172.21.15.33:0/3018212397 shutdown_connections 2024-05-03T18:26:59.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.295+0000 7f45ffaa4b80 1 --2- 172.21.15.33:0/3018212397 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cb56576580 0x55cb565769e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:59.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.295+0000 7f45ffaa4b80 1 --2- 172.21.15.33:0/3018212397 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb5657a760 0x55cb5657ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:59.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.295+0000 7f45ffaa4b80 1 --2- 172.21.15.33:0/3018212397 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cb5657b080 0x55cb56575f60 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:59.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.295+0000 7f45ffaa4b80 1 -- 172.21.15.33:0/3018212397 >> 172.21.15.33:0/3018212397 conn(0x55cb564a4b10 msgr2=0x55cb565707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:26:59.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.295+0000 7f45ee800700 1 --2- 172.21.15.33:0/1578713272 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f45d40b2990 0x7f45d40b4e50 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f45e8007fd0 tx=0x7f45e8007950 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:26:59.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.295+0000 7f45ffaa4b80 1 -- 172.21.15.33:0/3018212397 shutdown_connections 2024-05-03T18:26:59.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.295+0000 7f45ffaa4b80 1 -- 172.21.15.33:0/3018212397 wait complete. 2024-05-03T18:26:59.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.296+0000 7f45ffaa4b80 1 -- start start 2024-05-03T18:26:59.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.296+0000 7f45ffaa4b80 1 Processor -- start 2024-05-03T18:26:59.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.297+0000 7f45ffaa4b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cb56576580 0x55cb56581dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:26:59.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.297+0000 7f45ffaa4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb5657a760 0x55cb5657f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:26:59.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.297+0000 7f45ffaa4b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cb5657b080 0x55cb5657f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:26:59.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.297+0000 7f45ffaa4b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55cb56582bf0 con 0x55cb56576580 2024-05-03T18:26:59.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.297+0000 7f45ffaa4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55cb5642ead0 con 0x55cb5657a760 2024-05-03T18:26:59.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.297+0000 7f45ffaa4b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55cb56582d70 con 0x55cb5657b080 2024-05-03T18:26:59.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.297+0000 7f45ffaa4b80 10 client.0 _ll_register_callbacks cb 0x55cb5646fc20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:26:59.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.297+0000 7f45ede00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cb56576580 0x55cb56581dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:26:59.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.297+0000 7f45ede00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cb56576580 0x55cb56581dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52616/0 (socket says 172.21.15.33:52616) 2024-05-03T18:26:59.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.297+0000 7f45ede00700 1 -- 172.21.15.33:0/856827268 learned_addr learned my addr 172.21.15.33:0/856827268 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:26:59.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.297+0000 7f45ed400700 1 --2- 172.21.15.33:0/856827268 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb5657a760 0x55cb5657f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:26:59.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.297+0000 7f45ee800700 1 --2- 172.21.15.33:0/856827268 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cb5657b080 0x55cb5657f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:26:59.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.297+0000 7f45ede00700 1 -- 172.21.15.33:0/856827268 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cb5657b080 msgr2=0x55cb5657f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:26:59.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.297+0000 7f45ede00700 1 --2- 172.21.15.33:0/856827268 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cb5657b080 0x55cb5657f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:59.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.298+0000 7f45ede00700 1 -- 172.21.15.33:0/856827268 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb5657a760 msgr2=0x55cb5657f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:26:59.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.298+0000 7f45ede00700 1 --2- 172.21.15.33:0/856827268 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb5657a760 0x55cb5657f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:59.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.298+0000 7f45ede00700 1 -- 172.21.15.33:0/856827268 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb5648e3d0 con 0x55cb56576580 2024-05-03T18:26:59.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.298+0000 7f45ed400700 1 --2- 172.21.15.33:0/856827268 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb5657a760 0x55cb5657f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:26:59.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.298+0000 7f45ede00700 1 --2- 172.21.15.33:0/856827268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cb56576580 0x55cb56581dd0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f45e000bdf0 tx=0x7f45e000bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8053a93ce7cb3029 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:26:59.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.298+0000 7f45cfe00700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:26:59.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.298+0000 7f45cfe00700 1 -- 172.21.15.33:0/856827268 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e0031070 con 0x55cb56576580 2024-05-03T18:26:59.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.298+0000 7f45cfe00700 1 -- 172.21.15.33:0/856827268 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f45e000f9b0 con 0x55cb56576580 2024-05-03T18:26:59.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.298+0000 7f45ffaa4b80 1 -- 172.21.15.33:0/856827268 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45e0027070 con 0x55cb56576580 2024-05-03T18:26:59.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.299+0000 7f45cfe00700 1 -- 172.21.15.33:0/856827268 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f45e003d050 con 0x55cb56576580 2024-05-03T18:26:59.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.299+0000 7f45ffaa4b80 10 client.15096 fetch_fsmap learned FSMap version 21 2024-05-03T18:26:59.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.299+0000 7f45ffaa4b80 1 -- 172.21.15.33:0/856827268 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55cb5658d3e0 con 0x55cb56576580 2024-05-03T18:26:59.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.299+0000 7f45cfe00700 1 -- 172.21.15.33:0/856827268 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f45e0027070 con 0x55cb56576580 2024-05-03T18:26:59.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.299+0000 7f45cfe00700 1 -- 172.21.15.33:0/856827268 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f45e002bc20 con 0x55cb56576580 2024-05-03T18:26:59.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.299+0000 7f45ffaa4b80 10 client.15096 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:26:59.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.299+0000 7f45ffaa4b80 10 client.15096 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-03T18:26:59.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.299+0000 7f45ffaa4b80 20 client.15096 populate_metadata read hostname 'smithi033' 2024-05-03T18:26:59.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.299+0000 7f45ffaa4b80 1 --2- 172.21.15.33:0/856827268 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55cb5658e730 0x55cb565908d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:26:59.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.300+0000 7f45ffaa4b80 4 client.15096 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:26:59.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.300+0000 7f45ffaa4b80 1 -- 172.21.15.33:0/856827268 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cb56543180 con 0x55cb5658e730 2024-05-03T18:26:59.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.300+0000 7f45ed400700 1 --2- 172.21.15.33:0/856827268 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55cb5658e730 0x55cb565908d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:26:59.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.301+0000 7f45ed400700 1 --2- 172.21.15.33:0/856827268 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55cb5658e730 0x55cb565908d0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8858e2a4e5df450 server_cookie=200e681fa372dbc9 in_seq=0 out_seq=0 2024-05-03T18:26:59.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.301+0000 7f45cfe00700 10 client.15096 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:26:59.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.303+0000 7f45cfe00700 1 -- 172.21.15.33:0/856827268 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55cb56543180 con 0x55cb5658e730 2024-05-03T18:26:59.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.303+0000 7f45cfe00700 10 client.15096 handle_command_reply: tid=0 2024-05-03T18:26:59.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.303+0000 7f45ffaa4b80 1 client.15096 shutdown 2024-05-03T18:26:59.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.303+0000 7f45ffaa4b80 1 -- 172.21.15.33:0/856827268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cb56576580 msgr2=0x55cb56581dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:26:59.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.303+0000 7f45ffaa4b80 1 --2- 172.21.15.33:0/856827268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cb56576580 0x55cb56581dd0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f45e000bdf0 tx=0x7f45e000bed0 comp rx=0 tx=0).stop 2024-05-03T18:26:59.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.303+0000 7f45ffaa4b80 1 -- 172.21.15.33:0/856827268 shutdown_connections 2024-05-03T18:26:59.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.303+0000 7f45ffaa4b80 1 --2- 172.21.15.33:0/856827268 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55cb5658e730 0x55cb565908d0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:59.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.303+0000 7f45ffaa4b80 1 --2- 172.21.15.33:0/856827268 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cb5657b080 0x55cb5657f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:59.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.303+0000 7f45ffaa4b80 1 --2- 172.21.15.33:0/856827268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cb56576580 0x55cb56581dd0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:59.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.303+0000 7f45ffaa4b80 1 --2- 172.21.15.33:0/856827268 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb5657a760 0x55cb5657f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:59.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.303+0000 7f45ffaa4b80 1 -- 172.21.15.33:0/856827268 >> 172.21.15.33:0/856827268 conn(0x55cb564a4b10 msgr2=0x55cb5657bdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:26:59.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.303+0000 7f45cfe00700 0 client.15096 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:26:59.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.304+0000 7f45ffaa4b80 1 -- 172.21.15.33:0/856827268 shutdown_connections 2024-05-03T18:26:59.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.304+0000 7f45ffaa4b80 1 -- 172.21.15.33:0/856827268 wait complete. 2024-05-03T18:26:59.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.304+0000 7f45ffaa4b80 20 client.15096 trim_cache size 0 max 16384 2024-05-03T18:26:59.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.328+0000 7f45ffaa4b80 1 Processor -- start 2024-05-03T18:26:59.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.328+0000 7f45ffaa4b80 1 -- start start 2024-05-03T18:26:59.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.328+0000 7f45ffaa4b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cb56576580 0x55cb5658ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:26:59.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.328+0000 7f45ffaa4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb5657a760 0x55cb5656fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:26:59.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.328+0000 7f45ffaa4b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cb5657b080 0x55cb56573f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:26:59.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.328+0000 7f45ffaa4b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55cb5658ec10 con 0x55cb56576580 2024-05-03T18:26:59.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.328+0000 7f45ffaa4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55cb5658f2e0 con 0x55cb5657a760 2024-05-03T18:26:59.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.328+0000 7f45ffaa4b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55cb5658f9b0 con 0x55cb5657b080 2024-05-03T18:26:59.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.329+0000 7f45ede00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cb56576580 0x55cb5658ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:26:59.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.329+0000 7f45ede00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cb56576580 0x55cb5658ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52626/0 (socket says 172.21.15.33:52626) 2024-05-03T18:26:59.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.329+0000 7f45ede00700 1 -- 172.21.15.33:0/2318768878 learned_addr learned my addr 172.21.15.33:0/2318768878 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:26:59.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.329+0000 7f45ed400700 1 --2- 172.21.15.33:0/2318768878 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb5657a760 0x55cb5656fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:26:59.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.329+0000 7f45ee800700 1 --2- 172.21.15.33:0/2318768878 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cb5657b080 0x55cb56573f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:26:59.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.329+0000 7f45ede00700 1 -- 172.21.15.33:0/2318768878 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cb5657b080 msgr2=0x55cb56573f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:26:59.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.329+0000 7f45ede00700 1 --2- 172.21.15.33:0/2318768878 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cb5657b080 0x55cb56573f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:59.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.329+0000 7f45ede00700 1 -- 172.21.15.33:0/2318768878 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb5657a760 msgr2=0x55cb5656fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:26:59.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.329+0000 7f45ede00700 1 --2- 172.21.15.33:0/2318768878 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb5657a760 0x55cb5656fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:59.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.329+0000 7f45ede00700 1 -- 172.21.15.33:0/2318768878 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb5658c040 con 0x55cb56576580 2024-05-03T18:26:59.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.329+0000 7f45ede00700 1 --2- 172.21.15.33:0/2318768878 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cb56576580 0x55cb5658ed90 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f45e0009540 tx=0x7f45e0022840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9315e644392bffb8 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:26:59.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.330+0000 7f45daa00700 1 -- 172.21.15.33:0/2318768878 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e0031070 con 0x55cb56576580 2024-05-03T18:26:59.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.330+0000 7f45daa00700 1 -- 172.21.15.33:0/2318768878 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f45e0027040 con 0x55cb56576580 2024-05-03T18:26:59.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.330+0000 7f45daa00700 1 -- 172.21.15.33:0/2318768878 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e0022e20 con 0x55cb56576580 2024-05-03T18:26:59.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.330+0000 7f45ffaa4b80 1 -- 172.21.15.33:0/2318768878 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cb56576580 msgr2=0x55cb5658ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:26:59.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.330+0000 7f45ffaa4b80 1 --2- 172.21.15.33:0/2318768878 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cb56576580 0x55cb5658ed90 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f45e0009540 tx=0x7f45e0022840 comp rx=0 tx=0).stop 2024-05-03T18:26:59.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.331+0000 7f45ffaa4b80 1 -- 172.21.15.33:0/2318768878 shutdown_connections 2024-05-03T18:26:59.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.331+0000 7f45ffaa4b80 1 --2- 172.21.15.33:0/2318768878 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cb5657b080 0x55cb56573f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:59.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.331+0000 7f45ffaa4b80 1 --2- 172.21.15.33:0/2318768878 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cb56576580 0x55cb5658ed90 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:59.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.331+0000 7f45ffaa4b80 1 --2- 172.21.15.33:0/2318768878 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb5657a760 0x55cb5656fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:59.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.331+0000 7f45ffaa4b80 1 -- 172.21.15.33:0/2318768878 >> 172.21.15.33:0/2318768878 conn(0x55cb564a4b10 msgr2=0x55cb56570720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:26:59.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.331+0000 7f45ffaa4b80 1 -- 172.21.15.33:0/2318768878 shutdown_connections 2024-05-03T18:26:59.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.331+0000 7f45ffaa4b80 1 -- 172.21.15.33:0/2318768878 wait complete. 2024-05-03T18:26:59.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.331+0000 7f45ffaa4b80 1 -- start start 2024-05-03T18:26:59.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.332+0000 7f45ffaa4b80 1 Processor -- start 2024-05-03T18:26:59.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.332+0000 7f45ffaa4b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cb56576580 0x55cb56573bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:26:59.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.332+0000 7f45ffaa4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb5657a760 0x55cb56574100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:26:59.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.332+0000 7f45ffaa4b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cb5657b080 0x55cb56574640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:26:59.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.332+0000 7f45ffaa4b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55cb56590750 con 0x55cb56576580 2024-05-03T18:26:59.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.332+0000 7f45ffaa4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55cb56590080 con 0x55cb5657a760 2024-05-03T18:26:59.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.332+0000 7f45ffaa4b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55cb56590e20 con 0x55cb5657b080 2024-05-03T18:26:59.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.332+0000 7f45ffaa4b80 10 client.0 _ll_register_callbacks cb 0x55cb5646fc20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:26:59.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.332+0000 7f45ede00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cb56576580 0x55cb56573bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:26:59.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.332+0000 7f45ede00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cb56576580 0x55cb56573bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52628/0 (socket says 172.21.15.33:52628) 2024-05-03T18:26:59.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.332+0000 7f45ede00700 1 -- 172.21.15.33:0/1242386267 learned_addr learned my addr 172.21.15.33:0/1242386267 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:26:59.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.333+0000 7f45ed400700 1 --2- 172.21.15.33:0/1242386267 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb5657a760 0x55cb56574100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:26:59.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.333+0000 7f45ee800700 1 --2- 172.21.15.33:0/1242386267 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cb5657b080 0x55cb56574640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:26:59.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.333+0000 7f45ede00700 1 -- 172.21.15.33:0/1242386267 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cb5657b080 msgr2=0x55cb56574640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:26:59.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.333+0000 7f45ede00700 1 --2- 172.21.15.33:0/1242386267 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cb5657b080 0x55cb56574640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:59.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.333+0000 7f45ede00700 1 -- 172.21.15.33:0/1242386267 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb5657a760 msgr2=0x55cb56574100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:26:59.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.333+0000 7f45ede00700 1 --2- 172.21.15.33:0/1242386267 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb5657a760 0x55cb56574100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:59.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.333+0000 7f45ede00700 1 -- 172.21.15.33:0/1242386267 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb56596470 con 0x55cb56576580 2024-05-03T18:26:59.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.333+0000 7f45ede00700 1 --2- 172.21.15.33:0/1242386267 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cb56576580 0x55cb56573bc0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f45e0009350 tx=0x7f45e0022770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b519e30898abc0e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:26:59.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.333+0000 7f45cfe00700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:26:59.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.333+0000 7f45cfe00700 1 -- 172.21.15.33:0/1242386267 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e0031040 con 0x55cb56576580 2024-05-03T18:26:59.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.334+0000 7f45ffaa4b80 1 -- 172.21.15.33:0/1242386267 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45e0027040 con 0x55cb56576580 2024-05-03T18:26:59.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.334+0000 7f45cfe00700 1 -- 172.21.15.33:0/1242386267 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f45e00050b0 con 0x55cb56576580 2024-05-03T18:26:59.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.334+0000 7f45cfe00700 1 -- 172.21.15.33:0/1242386267 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f45e0022e10 con 0x55cb56576580 2024-05-03T18:26:59.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.334+0000 7f45ffaa4b80 10 client.15102 fetch_fsmap learned FSMap version 21 2024-05-03T18:26:59.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.334+0000 7f45ffaa4b80 1 -- 172.21.15.33:0/1242386267 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55cb566035a0 con 0x55cb56576580 2024-05-03T18:26:59.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.334+0000 7f45cfe00700 1 -- 172.21.15.33:0/1242386267 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f45e0027040 con 0x55cb56576580 2024-05-03T18:26:59.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.334+0000 7f45cfe00700 1 -- 172.21.15.33:0/1242386267 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f45e000f7b0 con 0x55cb56576580 2024-05-03T18:26:59.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.334+0000 7f45ffaa4b80 10 client.15102 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:26:59.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.334+0000 7f45ffaa4b80 10 client.15102 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-03T18:26:59.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.334+0000 7f45ffaa4b80 20 client.15102 populate_metadata read hostname 'smithi033' 2024-05-03T18:26:59.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.334+0000 7f45ffaa4b80 1 --2- 172.21.15.33:0/1242386267 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55cb56607cf0 0x55cb56606a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:26:59.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.334+0000 7f45ffaa4b80 4 client.15102 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-03T18:26:59.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.334+0000 7f45ffaa4b80 1 -- 172.21.15.33:0/1242386267 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "status"}) -- 0x55cb564d63b0 con 0x55cb56607cf0 2024-05-03T18:26:59.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.335+0000 7f45ed400700 1 --2- 172.21.15.33:0/1242386267 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55cb56607cf0 0x55cb56606a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:26:59.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.336+0000 7f45ed400700 1 --2- 172.21.15.33:0/1242386267 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55cb56607cf0 0x55cb56606a00 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c86d0ecbbe1bc766 server_cookie=11a067872d5855c1 in_seq=0 out_seq=0 2024-05-03T18:26:59.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.336+0000 7f45cfe00700 10 client.15102 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:26:59.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.337+0000 7f45cfe00700 1 -- 172.21.15.33:0/1242386267 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55cb564d63b0 con 0x55cb56607cf0 2024-05-03T18:26:59.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.337+0000 7f45cfe00700 10 client.15102 handle_command_reply: tid=0 2024-05-03T18:26:59.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.337+0000 7f45ffaa4b80 1 client.15102 shutdown 2024-05-03T18:26:59.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.337+0000 7f45ffaa4b80 1 -- 172.21.15.33:0/1242386267 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cb56576580 msgr2=0x55cb56573bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:26:59.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.337+0000 7f45ffaa4b80 1 --2- 172.21.15.33:0/1242386267 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cb56576580 0x55cb56573bc0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f45e0009350 tx=0x7f45e0022770 comp rx=0 tx=0).stop 2024-05-03T18:26:59.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.337+0000 7f45ffaa4b80 1 -- 172.21.15.33:0/1242386267 shutdown_connections 2024-05-03T18:26:59.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.337+0000 7f45ffaa4b80 1 --2- 172.21.15.33:0/1242386267 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55cb56607cf0 0x55cb56606a00 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:59.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.337+0000 7f45ffaa4b80 1 --2- 172.21.15.33:0/1242386267 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cb5657b080 0x55cb56574640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:59.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.337+0000 7f45ffaa4b80 1 --2- 172.21.15.33:0/1242386267 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cb56576580 0x55cb56573bc0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:59.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.337+0000 7f45ffaa4b80 1 --2- 172.21.15.33:0/1242386267 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb5657a760 0x55cb56574100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:59.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.337+0000 7f45cfe00700 0 client.15102 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:26:59.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.337+0000 7f45ffaa4b80 1 -- 172.21.15.33:0/1242386267 >> 172.21.15.33:0/1242386267 conn(0x55cb564a4b10 msgr2=0x55cb565752e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:26:59.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.337+0000 7f45ffaa4b80 1 -- 172.21.15.33:0/1242386267 shutdown_connections 2024-05-03T18:26:59.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.337+0000 7f45ffaa4b80 1 -- 172.21.15.33:0/1242386267 wait complete. 2024-05-03T18:26:59.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.337+0000 7f45ffaa4b80 20 client.15102 trim_cache size 0 max 16384 2024-05-03T18:26:59.340 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-03T18:26:59.340 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "5703bba4-0978-11ef-bc97-c7b262605968", 2024-05-03T18:26:59.340 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 0, 2024-05-03T18:26:59.340 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 34508, 2024-05-03T18:26:59.340 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2024-05-03T18:26:59.340 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2024-05-03T18:26:59.340 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2024-05-03T18:26:59.340 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 269.69324476999998, 2024-05-03T18:26:59.341 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 21, 2024-05-03T18:26:59.341 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 89, 2024-05-03T18:26:59.341 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 87, 2024-05-03T18:26:59.341 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 293.25031071299998 2024-05-03T18:26:59.341 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-03T18:26:59.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.338+0000 7f45f5200700 1 -- 172.21.15.33:0/1578713272 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f45d40b2990 msgr2=0x7f45d40b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:26:59.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.338+0000 7f45f5200700 1 --2- 172.21.15.33:0/1578713272 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f45d40b2990 0x7f45d40b4e50 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f45e8007fd0 tx=0x7f45e8007950 comp rx=0 tx=0).stop 2024-05-03T18:26:59.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.338+0000 7f45f5200700 1 -- 172.21.15.33:0/1578713272 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45f01060a0 msgr2=0x7f45f01a1e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:26:59.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.338+0000 7f45f5200700 1 --2- 172.21.15.33:0/1578713272 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45f01060a0 0x7f45f01a1e20 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f45dc00d980 tx=0x7f45dc00dc90 comp rx=0 tx=0).stop 2024-05-03T18:26:59.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.338+0000 7f45f5200700 1 -- 172.21.15.33:0/1578713272 shutdown_connections 2024-05-03T18:26:59.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.338+0000 7f45f5200700 1 --2- 172.21.15.33:0/1578713272 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f45f010b6e0 0x7f45f019baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:59.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.338+0000 7f45f5200700 1 --2- 172.21.15.33:0/1578713272 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f45d40b2990 0x7f45d40b4e50 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:59.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.338+0000 7f45f5200700 1 --2- 172.21.15.33:0/1578713272 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45f01056e0 0x7f45f01a18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:59.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.338+0000 7f45f5200700 1 --2- 172.21.15.33:0/1578713272 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45f01060a0 0x7f45f01a1e20 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:26:59.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.338+0000 7f45f5200700 1 -- 172.21.15.33:0/1578713272 >> 172.21.15.33:0/1578713272 conn(0x7f45f01011d0 msgr2=0x7f45f01018d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:26:59.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.339+0000 7f45f5200700 1 -- 172.21.15.33:0/1578713272 shutdown_connections 2024-05-03T18:26:59.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:26:59.339+0000 7f45f5200700 1 -- 172.21.15.33:0/1578713272 wait complete. 2024-05-03T18:27:00.266 INFO:teuthology.run_tasks:Running task sequential... 2024-05-03T18:27:00.276 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-05-03T18:27:00.276 INFO:teuthology.task.sequential:In sequential, running task print... 2024-05-03T18:27:00.276 INFO:teuthology.task.print:Enabling mgr modules 2024-05-03T18:27:00.276 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-05-03T18:27:00.277 INFO:teuthology.task.exec:Executing custom commands... 2024-05-03T18:27:00.277 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi033.front.sepia.ceph.com 2024-05-03T18:27:00.277 DEBUG:teuthology.orchestra.run.smithi033:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-05-03T18:27:00.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:00.496+0000 7f0e2d200640 1 -- 172.21.15.33:0/4273392981 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e281010a0 msgr2=0x7f0e281034d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:00.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:00.496+0000 7f0e2d200640 1 --2- 172.21.15.33:0/4273392981 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e281010a0 0x7f0e281034d0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f0e18009900 tx=0x7f0e1802a410 comp rx=0 tx=0).stop 2024-05-03T18:27:00.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:00.496+0000 7f0e2d200640 1 -- 172.21.15.33:0/4273392981 shutdown_connections 2024-05-03T18:27:00.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:00.496+0000 7f0e2d200640 1 --2- 172.21.15.33:0/4273392981 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0e281063c0 0x7f0e28106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:00.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:00.496+0000 7f0e2d200640 1 --2- 172.21.15.33:0/4273392981 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e28103a10 0x7f0e28105e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:00.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:00.496+0000 7f0e2d200640 1 --2- 172.21.15.33:0/4273392981 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e281010a0 0x7f0e281034d0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:00.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:00.496+0000 7f0e2d200640 1 -- 172.21.15.33:0/4273392981 >> 172.21.15.33:0/4273392981 conn(0x7f0e280fcca0 msgr2=0x7f0e280ff0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:27:00.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:00.496+0000 7f0e2d200640 1 -- 172.21.15.33:0/4273392981 shutdown_connections 2024-05-03T18:27:00.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:00.496+0000 7f0e2d200640 1 -- 172.21.15.33:0/4273392981 wait complete. 2024-05-03T18:27:00.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:00.497+0000 7f0e2d200640 1 Processor -- start 2024-05-03T18:27:00.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:00.497+0000 7f0e2d200640 1 -- start start 2024-05-03T18:27:00.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:00.497+0000 7f0e2d200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0e281010a0 0x7f0e2819ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:00.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:00.497+0000 7f0e2d200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e28103a10 0x7f0e2819f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:00.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:00.497+0000 7f0e2d200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e281063c0 0x7f0e281a6530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:00.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:00.497+0000 7f0e2d200640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0e28114800 con 0x7f0e28103a10 2024-05-03T18:27:00.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:00.497+0000 7f0e2d200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0e28114680 con 0x7f0e281063c0 2024-05-03T18:27:00.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:00.497+0000 7f0e2d200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f0e28114980 con 0x7f0e281010a0 2024-05-03T18:27:00.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:00.498+0000 7f0e26a00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e28103a10 0x7f0e2819f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:00.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:00.498+0000 7f0e26a00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e28103a10 0x7f0e2819f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52642/0 (socket says 172.21.15.33:52642) 2024-05-03T18:27:00.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:00.498+0000 7f0e26a00640 1 -- 172.21.15.33:0/288952291 learned_addr learned my addr 172.21.15.33:0/288952291 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:27:00.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:00.498+0000 7f0e27400640 1 --2- 172.21.15.33:0/288952291 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0e281010a0 0x7f0e2819ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:00.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:00.498+0000 7f0e27e00640 1 --2- 172.21.15.33:0/288952291 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e281063c0 0x7f0e281a6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:00.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:00.498+0000 7f0e26a00640 1 -- 172.21.15.33:0/288952291 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0e281010a0 msgr2=0x7f0e2819ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:00.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:00.498+0000 7f0e26a00640 1 --2- 172.21.15.33:0/288952291 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0e281010a0 0x7f0e2819ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:00.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:00.498+0000 7f0e26a00640 1 -- 172.21.15.33:0/288952291 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e281063c0 msgr2=0x7f0e281a6530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:00.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:00.498+0000 7f0e26a00640 1 --2- 172.21.15.33:0/288952291 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e281063c0 0x7f0e281a6530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:00.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:00.498+0000 7f0e26a00640 1 -- 172.21.15.33:0/288952291 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e281a6ba0 con 0x7f0e28103a10 2024-05-03T18:27:00.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:00.498+0000 7f0e27e00640 1 --2- 172.21.15.33:0/288952291 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e281063c0 0x7f0e281a6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:27:00.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:00.499+0000 7f0e26a00640 1 --2- 172.21.15.33:0/288952291 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e28103a10 0x7f0e2819f4c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f0e1400eab0 tx=0x7f0e1400ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:27:00.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:00.499+0000 7f0e1fe00640 1 -- 172.21.15.33:0/288952291 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e1400ce40 con 0x7f0e28103a10 2024-05-03T18:27:00.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:00.499+0000 7f0e1fe00640 1 -- 172.21.15.33:0/288952291 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0e14011900 con 0x7f0e28103a10 2024-05-03T18:27:00.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:00.499+0000 7f0e2d200640 1 -- 172.21.15.33:0/288952291 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e281a6e90 con 0x7f0e28103a10 2024-05-03T18:27:00.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:00.499+0000 7f0e1fe00640 1 -- 172.21.15.33:0/288952291 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e14010990 con 0x7f0e28103a10 2024-05-03T18:27:00.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:00.499+0000 7f0e2d200640 1 -- 172.21.15.33:0/288952291 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e281a7350 con 0x7f0e28103a10 2024-05-03T18:27:00.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:00.500+0000 7f0e2d200640 1 -- 172.21.15.33:0/288952291 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e280fe790 con 0x7f0e28103a10 2024-05-03T18:27:00.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:00.503+0000 7f0e1fe00640 1 -- 172.21.15.33:0/288952291 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f0e14011470 con 0x7f0e28103a10 2024-05-03T18:27:00.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:00.505+0000 7f0e1fe00640 1 --2- 172.21.15.33:0/288952291 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f0e0c0b2870 0x7f0e0c0b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:00.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:00.505+0000 7f0e1fe00640 1 -- 172.21.15.33:0/288952291 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f0e140e2e70 con 0x7f0e28103a10 2024-05-03T18:27:00.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:00.505+0000 7f0e27400640 1 --2- 172.21.15.33:0/288952291 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f0e0c0b2870 0x7f0e0c0b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:00.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:00.506+0000 7f0e27400640 1 --2- 172.21.15.33:0/288952291 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f0e0c0b2870 0x7f0e0c0b4d30 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0e18000c00 tx=0x7f0e18031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:27:00.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:00.510+0000 7f0e1fe00640 1 -- 172.21.15.33:0/288952291 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f0e14016030 con 0x7f0e28103a10 2024-05-03T18:27:00.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:00.783+0000 7f0e2d200640 1 -- 172.21.15.33:0/288952291 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f0e2802dd20 con 0x7f0e28103a10 2024-05-03T18:27:01.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:00 smithi099 ceph-mon[31658]: pgmap v519: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:27:01.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:00 smithi150 ceph-mon[32492]: pgmap v519: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:27:01.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:00 smithi033 ceph-mon[28330]: pgmap v519: 129 pgs: 129 active+clean; 632 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:27:01.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:01.764+0000 7f0e1fe00640 1 -- 172.21.15.33:0/288952291 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v18) ==== 92+0+0 (secure 0 0 0) 0x7f0e140aa650 con 0x7f0e28103a10 2024-05-03T18:27:01.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:01.768+0000 7f0e2d200640 1 -- 172.21.15.33:0/288952291 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f0e0c0b2870 msgr2=0x7f0e0c0b4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:01.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:01.768+0000 7f0e2d200640 1 --2- 172.21.15.33:0/288952291 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f0e0c0b2870 0x7f0e0c0b4d30 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0e18000c00 tx=0x7f0e18031400 comp rx=0 tx=0).stop 2024-05-03T18:27:01.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:01.768+0000 7f0e2d200640 1 -- 172.21.15.33:0/288952291 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e28103a10 msgr2=0x7f0e2819f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:01.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:01.768+0000 7f0e2d200640 1 --2- 172.21.15.33:0/288952291 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e28103a10 0x7f0e2819f4c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f0e1400eab0 tx=0x7f0e1400ef70 comp rx=0 tx=0).stop 2024-05-03T18:27:01.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:01.768+0000 7f0e2d200640 1 -- 172.21.15.33:0/288952291 shutdown_connections 2024-05-03T18:27:01.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:01.768+0000 7f0e2d200640 1 --2- 172.21.15.33:0/288952291 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f0e0c0b2870 0x7f0e0c0b4d30 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:01.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:01.768+0000 7f0e2d200640 1 --2- 172.21.15.33:0/288952291 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e281063c0 0x7f0e281a6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:01.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:01.768+0000 7f0e2d200640 1 --2- 172.21.15.33:0/288952291 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e28103a10 0x7f0e2819f4c0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:01.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:01.768+0000 7f0e2d200640 1 --2- 172.21.15.33:0/288952291 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0e281010a0 0x7f0e2819ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:01.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:01.768+0000 7f0e2d200640 1 -- 172.21.15.33:0/288952291 >> 172.21.15.33:0/288952291 conn(0x7f0e280fcca0 msgr2=0x7f0e281044c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:27:01.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:01.769+0000 7f0e2d200640 1 -- 172.21.15.33:0/288952291 shutdown_connections 2024-05-03T18:27:01.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:01.769+0000 7f0e2d200640 1 -- 172.21.15.33:0/288952291 wait complete. 2024-05-03T18:27:01.797 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-05-03T18:27:01.809 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-05-03T18:27:01.809 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2024-05-03T18:27:01.809 INFO:tasks.ceph_fuse:client.0 config is {'mntopts': ['context=system_u:object_r:postgresql_db_t:s0'], 'mount_subvol_num': 0, 'mountpoint': '/tmp/cephfs'} 2024-05-03T18:27:01.809 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-03T18:27:01.810 INFO:tasks.cephfs.mount:using client_config["hostfs_mntpt"] = /tmp/cephfs 2024-05-03T18:27:01.810 INFO:tasks.cephfs.mount:hostfs_mntpt = /tmp/cephfs 2024-05-03T18:27:01.810 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T18:27:01.810 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2024-05-03T18:27:01.857 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-05-03T18:27:01.858 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T18:27:01.858 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-03T18:27:01.972 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T18:27:01.972 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link delete ceph-brx 2024-05-03T18:27:02.034 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:01 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/288952291' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-05-03T18:27:02.035 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:27:01 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: ignoring --setuser ceph since I am not root 2024-05-03T18:27:02.035 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:27:01 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: ignoring --setgroup ceph since I am not root 2024-05-03T18:27:02.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:01 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/288952291' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-05-03T18:27:02.045 INFO:journalctl@ceph.mgr.y.smithi099.stdout:May 03 18:27:01 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-y[33047]: ignoring --setuser ceph since I am not root 2024-05-03T18:27:02.045 INFO:journalctl@ceph.mgr.y.smithi099.stdout:May 03 18:27:01 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-y[33047]: ignoring --setgroup ceph since I am not root 2024-05-03T18:27:02.046 INFO:journalctl@ceph.mgr.y.smithi099.stdout:May 03 18:27:01 smithi099 ceph-mgr[33051]: -- 172.21.15.99:0/124181124 <== mon.2 v2:172.21.15.99:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5588933365a0 con 0x55889330b000 2024-05-03T18:27:02.085 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:01 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/288952291' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-05-03T18:27:02.086 INFO:journalctl@ceph.mgr.z.smithi150.stdout:May 03 18:27:01 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-z[34362]: ignoring --setuser ceph since I am not root 2024-05-03T18:27:02.086 INFO:journalctl@ceph.mgr.z.smithi150.stdout:May 03 18:27:01 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-z[34362]: ignoring --setgroup ceph since I am not root 2024-05-03T18:27:02.086 INFO:journalctl@ceph.mgr.z.smithi150.stdout:May 03 18:27:01 smithi150 ceph-mgr[34366]: -- 172.21.15.150:0/1176340120 <== mon.1 v2:172.21.15.150:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560ce00065a0 con 0x560cdffdb000 2024-05-03T18:27:02.086 INFO:journalctl@ceph.mgr.z.smithi150.stdout:May 03 18:27:02 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-z[34362]: 2024-05-03T18:27:02.084+0000 7f948c8e8200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-03T18:27:02.097 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-05-03T18:27:02.097 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph fs ls 2024-05-03T18:27:02.359 INFO:journalctl@ceph.mgr.y.smithi099.stdout:May 03 18:27:02 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-y[33047]: 2024-05-03T18:27:02.102+0000 7f5edc2ee200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-03T18:27:02.382 INFO:journalctl@ceph.mgr.z.smithi150.stdout:May 03 18:27:02 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-z[34362]: 2024-05-03T18:27:02.333+0000 7f948c8e8200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-03T18:27:02.392 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:27:02 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:27:02.118+0000 7f529c48b200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-03T18:27:02.530 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:27:02.685 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:27:02 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:27:02.391+0000 7f529c48b200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-03T18:27:02.685 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:27:02 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:27:02.547+0000 7f529c48b200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-03T18:27:02.765 INFO:journalctl@ceph.mgr.z.smithi150.stdout:May 03 18:27:02 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-z[34362]: 2024-05-03T18:27:02.463+0000 7f948c8e8200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-03T18:27:02.765 INFO:journalctl@ceph.mgr.y.smithi099.stdout:May 03 18:27:02 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-y[33047]: 2024-05-03T18:27:02.357+0000 7f5edc2ee200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-03T18:27:02.766 INFO:journalctl@ceph.mgr.y.smithi099.stdout:May 03 18:27:02 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-y[33047]: 2024-05-03T18:27:02.489+0000 7f5edc2ee200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-03T18:27:02.952 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:02 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/288952291' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-05-03T18:27:02.952 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:02 smithi033 ceph-mon[28330]: mgrmap e18: x(active, since 14m), standbys: z, y 2024-05-03T18:27:03.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:02 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/288952291' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-05-03T18:27:03.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:02 smithi099 ceph-mon[31658]: mgrmap e18: x(active, since 14m), standbys: z, y 2024-05-03T18:27:03.046 INFO:journalctl@ceph.mgr.y.smithi099.stdout:May 03 18:27:02 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-y[33047]: 2024-05-03T18:27:02.884+0000 7f5edc2ee200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-03T18:27:03.046 INFO:journalctl@ceph.mgr.y.smithi099.stdout:May 03 18:27:02 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-y[33047]: 2024-05-03T18:27:02.995+0000 7f5edc2ee200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-03T18:27:03.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:02 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/288952291' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-05-03T18:27:03.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:02 smithi150 ceph-mon[32492]: mgrmap e18: x(active, since 14m), standbys: z, y 2024-05-03T18:27:03.132 INFO:journalctl@ceph.mgr.z.smithi150.stdout:May 03 18:27:02 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-z[34362]: 2024-05-03T18:27:02.857+0000 7f948c8e8200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-03T18:27:03.133 INFO:journalctl@ceph.mgr.z.smithi150.stdout:May 03 18:27:02 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-z[34362]: 2024-05-03T18:27:02.968+0000 7f948c8e8200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-03T18:27:03.234 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:27:02 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:27:02.969+0000 7f529c48b200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-03T18:27:03.234 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:27:03 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:27:03.088+0000 7f529c48b200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-03T18:27:04.795 INFO:journalctl@ceph.mgr.y.smithi099.stdout:May 03 18:27:04 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-y[33047]: 2024-05-03T18:27:04.526+0000 7f5edc2ee200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-03T18:27:04.795 INFO:journalctl@ceph.mgr.y.smithi099.stdout:May 03 18:27:04 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-y[33047]: 2024-05-03T18:27:04.755+0000 7f5edc2ee200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-03T18:27:04.882 INFO:journalctl@ceph.mgr.z.smithi150.stdout:May 03 18:27:04 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-z[34362]: 2024-05-03T18:27:04.501+0000 7f948c8e8200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-03T18:27:04.883 INFO:journalctl@ceph.mgr.z.smithi150.stdout:May 03 18:27:04 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-z[34362]: 2024-05-03T18:27:04.731+0000 7f948c8e8200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-03T18:27:04.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:04.934+0000 7fac8da00700 1 -- 172.21.15.33:0/3550597745 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac88075ff0 msgr2=0x7fac8810f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:04.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:04.934+0000 7fac8da00700 1 --2- 172.21.15.33:0/3550597745 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac88075ff0 0x7fac8810f960 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fac74009990 tx=0x7fac74009ca0 comp rx=0 tx=0).stop 2024-05-03T18:27:04.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:04.934+0000 7fac8da00700 1 -- 172.21.15.33:0/3550597745 shutdown_connections 2024-05-03T18:27:04.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:04.934+0000 7fac8da00700 1 --2- 172.21.15.33:0/3550597745 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac8810fea0 0x7fac88112290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:04.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:04.934+0000 7fac8da00700 1 --2- 172.21.15.33:0/3550597745 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac880756d0 0x7fac88075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:04.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:04.934+0000 7fac8da00700 1 --2- 172.21.15.33:0/3550597745 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac88075ff0 0x7fac8810f960 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:04.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:04.934+0000 7fac8da00700 1 -- 172.21.15.33:0/3550597745 >> 172.21.15.33:0/3550597745 conn(0x7fac8807a600 msgr2=0x7fac8807aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:27:04.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:04.934+0000 7fac8da00700 1 -- 172.21.15.33:0/3550597745 shutdown_connections 2024-05-03T18:27:04.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:04.934+0000 7fac8da00700 1 -- 172.21.15.33:0/3550597745 wait complete. 2024-05-03T18:27:04.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:04.935+0000 7fac8da00700 1 Processor -- start 2024-05-03T18:27:04.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:04.935+0000 7fac8da00700 1 -- start start 2024-05-03T18:27:04.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:04.936+0000 7fac8da00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac880756d0 0x7fac881a18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:04.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:04.936+0000 7fac8da00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac88075ff0 0x7fac881a1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:04.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:04.936+0000 7fac8da00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac8810fea0 0x7fac8819bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:04.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:04.936+0000 7fac8da00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fac88115780 con 0x7fac8810fea0 2024-05-03T18:27:04.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:04.936+0000 7fac8da00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fac88115600 con 0x7fac880756d0 2024-05-03T18:27:04.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:04.936+0000 7fac8da00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fac88115900 con 0x7fac88075ff0 2024-05-03T18:27:04.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:04.936+0000 7fac87400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac8810fea0 0x7fac8819bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:04.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:04.936+0000 7fac87400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac8810fea0 0x7fac8819bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52688/0 (socket says 172.21.15.33:52688) 2024-05-03T18:27:04.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:04.936+0000 7fac87400700 1 -- 172.21.15.33:0/109179902 learned_addr learned my addr 172.21.15.33:0/109179902 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:27:04.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:04.936+0000 7fac86a00700 1 --2- 172.21.15.33:0/109179902 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac880756d0 0x7fac881a18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:04.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:04.937+0000 7fac86000700 1 --2- 172.21.15.33:0/109179902 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac88075ff0 0x7fac881a1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:04.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:04.937+0000 7fac87400700 1 -- 172.21.15.33:0/109179902 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac88075ff0 msgr2=0x7fac881a1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:04.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:04.937+0000 7fac87400700 1 --2- 172.21.15.33:0/109179902 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac88075ff0 0x7fac881a1e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:04.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:04.937+0000 7fac87400700 1 -- 172.21.15.33:0/109179902 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac880756d0 msgr2=0x7fac881a18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:04.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:04.937+0000 7fac87400700 1 --2- 172.21.15.33:0/109179902 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac880756d0 0x7fac881a18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:04.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:04.937+0000 7fac87400700 1 -- 172.21.15.33:0/109179902 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac8819c260 con 0x7fac8810fea0 2024-05-03T18:27:04.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:04.937+0000 7fac87400700 1 --2- 172.21.15.33:0/109179902 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac8810fea0 0x7fac8819bb00 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fac800094f0 tx=0x7fac8000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:27:04.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:04.937+0000 7fac7f400700 1 -- 172.21.15.33:0/109179902 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac8000ddd0 con 0x7fac8810fea0 2024-05-03T18:27:04.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:04.937+0000 7fac8da00700 1 -- 172.21.15.33:0/109179902 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac8819c550 con 0x7fac8810fea0 2024-05-03T18:27:04.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:04.937+0000 7fac7f400700 1 -- 172.21.15.33:0/109179902 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fac80009e20 con 0x7fac8810fea0 2024-05-03T18:27:04.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:04.937+0000 7fac7f400700 1 -- 172.21.15.33:0/109179902 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac80015400 con 0x7fac8810fea0 2024-05-03T18:27:04.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:04.938+0000 7fac8da00700 1 -- 172.21.15.33:0/109179902 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac881a87c0 con 0x7fac8810fea0 2024-05-03T18:27:04.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:04.939+0000 7fac8da00700 1 -- 172.21.15.33:0/109179902 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac8806a190 con 0x7fac8810fea0 2024-05-03T18:27:04.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:04.941+0000 7fac7f400700 1 -- 172.21.15.33:0/109179902 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 149720+0+0 (secure 0 0 0) 0x7fac80005020 con 0x7fac8810fea0 2024-05-03T18:27:04.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:04.943+0000 7fac7f400700 1 --2- 172.21.15.33:0/109179902 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fac6c0b2960 0x7fac6c0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:04.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:04.943+0000 7fac86a00700 1 -- 172.21.15.33:0/109179902 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fac6c0b2960 msgr2=0x7fac6c0b4e20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/3642895501 2024-05-03T18:27:04.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:04.943+0000 7fac7f400700 1 -- 172.21.15.33:0/109179902 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7fac800e29d0 con 0x7fac8810fea0 2024-05-03T18:27:04.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:04.943+0000 7fac86a00700 1 --2- 172.21.15.33:0/109179902 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fac6c0b2960 0x7fac6c0b4e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-03T18:27:04.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:04.950+0000 7fac7f400700 1 -- 172.21.15.33:0/109179902 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fac80010070 con 0x7fac8810fea0 2024-05-03T18:27:04.972 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:27:04 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:27:04.715+0000 7f529c48b200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-03T18:27:04.972 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:27:04 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:27:04.970+0000 7f529c48b200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-03T18:27:05.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:05.144+0000 7fac86a00700 1 -- 172.21.15.33:0/109179902 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fac6c0b2960 msgr2=0x7fac6c0b4e20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/3642895501 2024-05-03T18:27:05.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:05.144+0000 7fac86a00700 1 --2- 172.21.15.33:0/109179902 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fac6c0b2960 0x7fac6c0b4e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-03T18:27:05.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:05.226+0000 7fac8da00700 1 -- 172.21.15.33:0/109179902 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fac8819cfe0 con 0x7fac8810fea0 2024-05-03T18:27:05.227 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:27:05 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:27:05.198+0000 7f529c48b200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-03T18:27:05.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:05.227+0000 7fac7f400700 1 -- 172.21.15.33:0/109179902 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v21) ==== 53+0+88 (secure 0 0 0) 0x7fac800aa210 con 0x7fac8810fea0 2024-05-03T18:27:05.228 INFO:teuthology.orchestra.run.smithi033.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-03T18:27:05.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:05.230+0000 7fac8da00700 1 -- 172.21.15.33:0/109179902 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fac6c0b2960 msgr2=0x7fac6c0b4e20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-03T18:27:05.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:05.230+0000 7fac8da00700 1 --2- 172.21.15.33:0/109179902 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fac6c0b2960 0x7fac6c0b4e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:05.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:05.230+0000 7fac8da00700 1 -- 172.21.15.33:0/109179902 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac8810fea0 msgr2=0x7fac8819bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:05.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:05.230+0000 7fac8da00700 1 --2- 172.21.15.33:0/109179902 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac8810fea0 0x7fac8819bb00 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fac800094f0 tx=0x7fac8000d3b0 comp rx=0 tx=0).stop 2024-05-03T18:27:05.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:05.230+0000 7fac8da00700 1 -- 172.21.15.33:0/109179902 shutdown_connections 2024-05-03T18:27:05.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:05.230+0000 7fac8da00700 1 --2- 172.21.15.33:0/109179902 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac8810fea0 0x7fac8819bb00 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:05.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:05.230+0000 7fac8da00700 1 --2- 172.21.15.33:0/109179902 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fac6c0b2960 0x7fac6c0b4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:05.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:05.230+0000 7fac8da00700 1 --2- 172.21.15.33:0/109179902 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac880756d0 0x7fac881a18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:05.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:05.230+0000 7fac8da00700 1 --2- 172.21.15.33:0/109179902 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac88075ff0 0x7fac881a1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:05.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:05.231+0000 7fac8da00700 1 -- 172.21.15.33:0/109179902 >> 172.21.15.33:0/109179902 conn(0x7fac8807a600 msgr2=0x7fac88100750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:27:05.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:05.231+0000 7fac8da00700 1 -- 172.21.15.33:0/109179902 shutdown_connections 2024-05-03T18:27:05.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:05.231+0000 7fac8da00700 1 -- 172.21.15.33:0/109179902 wait complete. 2024-05-03T18:27:05.281 INFO:journalctl@ceph.mgr.y.smithi099.stdout:May 03 18:27:04 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-y[33047]: 2024-05-03T18:27:04.975+0000 7f5edc2ee200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-03T18:27:05.281 INFO:journalctl@ceph.mgr.z.smithi150.stdout:May 03 18:27:04 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-z[34362]: 2024-05-03T18:27:04.953+0000 7f948c8e8200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-03T18:27:05.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:05 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/109179902' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-03T18:27:05.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:05 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/109179902' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-03T18:27:05.545 INFO:journalctl@ceph.mgr.y.smithi099.stdout:May 03 18:27:05 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-y[33047]: 2024-05-03T18:27:05.359+0000 7f5edc2ee200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-03T18:27:05.546 INFO:journalctl@ceph.mgr.y.smithi099.stdout:May 03 18:27:05 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-y[33047]: 2024-05-03T18:27:05.483+0000 7f5edc2ee200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-03T18:27:05.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:05 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/109179902' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-03T18:27:05.632 INFO:journalctl@ceph.mgr.z.smithi150.stdout:May 03 18:27:05 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-z[34362]: 2024-05-03T18:27:05.336+0000 7f948c8e8200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-03T18:27:05.632 INFO:journalctl@ceph.mgr.z.smithi150.stdout:May 03 18:27:05 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-z[34362]: 2024-05-03T18:27:05.459+0000 7f948c8e8200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-03T18:27:05.978 INFO:journalctl@ceph.mgr.z.smithi150.stdout:May 03 18:27:05 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-z[34362]: 2024-05-03T18:27:05.716+0000 7f948c8e8200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-03T18:27:05.978 INFO:journalctl@ceph.mgr.z.smithi150.stdout:May 03 18:27:05 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-z[34362]: 2024-05-03T18:27:05.976+0000 7f948c8e8200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-03T18:27:05.984 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:27:05 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:27:05.616+0000 7f529c48b200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-03T18:27:05.984 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:27:05 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:27:05.745+0000 7f529c48b200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-03T18:27:05.990 INFO:journalctl@ceph.mgr.y.smithi099.stdout:May 03 18:27:05 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-y[33047]: 2024-05-03T18:27:05.738+0000 7f5edc2ee200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-03T18:27:06.171 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-03T18:27:06.171 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-03T18:27:06.171 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi033.front.sepia.ceph.com 2024-05-03T18:27:06.171 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-03T18:27:06.171 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /tmp/cephfs 2024-05-03T18:27:06.171 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-03T18:27:06.172 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-03T18:27:06.172 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-03T18:27:06.172 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/tmp/cephfs' 2024-05-03T18:27:06.172 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T18:27:06.172 DEBUG:teuthology.orchestra.run.smithi033:> ip addr 2024-05-03T18:27:06.226 INFO:teuthology.orchestra.run.smithi033.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-03T18:27:06.226 INFO:teuthology.orchestra.run.smithi033.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-03T18:27:06.227 INFO:teuthology.orchestra.run.smithi033.stdout: inet 127.0.0.1/8 scope host lo 2024-05-03T18:27:06.227 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2024-05-03T18:27:06.227 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 ::1/128 scope host 2024-05-03T18:27:06.227 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2024-05-03T18:27:06.227 INFO:teuthology.orchestra.run.smithi033.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-03T18:27:06.227 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:14 brd ff:ff:ff:ff:ff:ff 2024-05-03T18:27:06.227 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f0 2024-05-03T18:27:06.227 INFO:teuthology.orchestra.run.smithi033.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-03T18:27:06.227 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:15 brd ff:ff:ff:ff:ff:ff 2024-05-03T18:27:06.227 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f1 2024-05-03T18:27:06.227 INFO:teuthology.orchestra.run.smithi033.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-03T18:27:06.227 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:94 brd ff:ff:ff:ff:ff:ff 2024-05-03T18:27:06.227 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f0 2024-05-03T18:27:06.227 INFO:teuthology.orchestra.run.smithi033.stdout: inet 172.21.15.33/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-05-03T18:27:06.227 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft 41659sec preferred_lft 41659sec 2024-05-03T18:27:06.228 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::ae1f:6bff:fea5:d294/64 scope link noprefixroute 2024-05-03T18:27:06.228 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2024-05-03T18:27:06.228 INFO:teuthology.orchestra.run.smithi033.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-03T18:27:06.228 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:95 brd ff:ff:ff:ff:ff:ff 2024-05-03T18:27:06.228 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f1 2024-05-03T18:27:06.228 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-03T18:27:06.228 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-03T18:27:06.228 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2024-05-03T18:27:06.228 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link add name ceph-brx type bridge 2024-05-03T18:27:06.228 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip addr flush dev ceph-brx 2024-05-03T18:27:06.229 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set ceph-brx up 2024-05-03T18:27:06.229 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-03T18:27:06.229 DEBUG:teuthology.orchestra.run.smithi033:> ') 2024-05-03T18:27:06.286 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:27:06 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:27:06.012+0000 7f529c48b200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-03T18:27:06.286 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:27:06 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:27:06.285+0000 7f529c48b200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-03T18:27:06.295 INFO:journalctl@ceph.mgr.y.smithi099.stdout:May 03 18:27:05 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-y[33047]: 2024-05-03T18:27:05.989+0000 7f5edc2ee200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-03T18:27:06.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-03T18:27:06.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-03T18:27:06.475 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T18:27:06.476 DEBUG:teuthology.orchestra.run.smithi033:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-03T18:27:06.537 INFO:teuthology.orchestra.run.smithi033.stdout:1 2024-05-03T18:27:06.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T18:27:06.542 DEBUG:teuthology.orchestra.run.smithi033:> route 2024-05-03T18:27:06.585 INFO:teuthology.orchestra.run.smithi033.stdout:Kernel IP routing table 2024-05-03T18:27:06.586 INFO:teuthology.orchestra.run.smithi033.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-03T18:27:06.586 INFO:teuthology.orchestra.run.smithi033.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-03T18:27:06.586 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-03T18:27:06.586 INFO:teuthology.orchestra.run.smithi033.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-03T18:27:06.586 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-03T18:27:06.586 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2024-05-03T18:27:06.586 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-03T18:27:06.586 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-03T18:27:06.587 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-03T18:27:06.587 DEBUG:teuthology.orchestra.run.smithi033:> ') 2024-05-03T18:27:06.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-03T18:27:06.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-03T18:27:06.837 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T18:27:06.837 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2024-05-03T18:27:06.883 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T18:27:06.883 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list-id 2024-05-03T18:27:06.929 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-03T18:27:06.929 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2024-05-03T18:27:06.929 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns add ceph-ns--tmp-cephfs 2024-05-03T18:27:06.929 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns set ceph-ns--tmp-cephfs 0 2024-05-03T18:27:06.929 DEBUG:teuthology.orchestra.run.smithi033:> ') 2024-05-03T18:27:07.045 INFO:journalctl@ceph.mgr.y.smithi099.stdout:May 03 18:27:06 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-y[33047]: 2024-05-03T18:27:06.777+0000 7f5edc2ee200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-03T18:27:07.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-03T18:27:07.095 INFO:journalctl@ceph.mgr.z.smithi150.stdout:May 03 18:27:06 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-z[34362]: 2024-05-03T18:27:06.799+0000 7f948c8e8200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-03T18:27:07.095 INFO:journalctl@ceph.mgr.z.smithi150.stdout:May 03 18:27:07 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-z[34362]: 2024-05-03T18:27:07.094+0000 7f948c8e8200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-03T18:27:07.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-03T18:27:07.150 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--tmp-cephfs' with 192.168.0.1/16 2024-05-03T18:27:07.151 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-03T18:27:07.151 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2024-05-03T18:27:07.151 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link add veth0 netns ceph-ns--tmp-cephfs type veth peer name brx.0 2024-05-03T18:27:07.151 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--tmp-cephfs ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-03T18:27:07.151 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set veth0 up 2024-05-03T18:27:07.151 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set lo up 2024-05-03T18:27:07.151 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--tmp-cephfs ip route add default via 192.168.255.254 2024-05-03T18:27:07.151 DEBUG:teuthology.orchestra.run.smithi033:> ') 2024-05-03T18:27:07.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-03T18:27:07.354 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:27:07 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:27:07.143+0000 7f529c48b200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-03T18:27:07.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-03T18:27:07.402 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-03T18:27:07.402 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2024-05-03T18:27:07.402 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set brx.0 up 2024-05-03T18:27:07.402 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set dev brx.0 master ceph-brx 2024-05-03T18:27:07.402 DEBUG:teuthology.orchestra.run.smithi033:> ') 2024-05-03T18:27:07.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-03T18:27:07.545 INFO:journalctl@ceph.mgr.y.smithi099.stdout:May 03 18:27:07 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-y[33047]: 2024-05-03T18:27:07.065+0000 7f5edc2ee200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-03T18:27:07.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-03T18:27:07.603 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {'mntopts': ['context=system_u:object_r:postgresql_db_t:s0'], 'mount_subvol_num': 0, 'mountpoint': '/tmp/cephfs'} 2024-05-03T18:27:07.603 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-03T18:27:07.603 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p -v /tmp/cephfs 2024-05-03T18:27:07.643 INFO:teuthology.orchestra.run.smithi033.stdout:mkdir: created directory '/tmp/cephfs' 2024-05-03T18:27:07.643 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-03T18:27:07.644 DEBUG:teuthology.orchestra.run.smithi033:> chmod 0000 /tmp/cephfs 2024-05-03T18:27:07.732 DEBUG:teuthology.orchestra.run.smithi033:> sudo modprobe fuse 2024-05-03T18:27:07.733 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:27:07 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:27:07.458+0000 7f529c48b200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-03T18:27:07.794 INFO:journalctl@ceph.mgr.z.smithi150.stdout:May 03 18:27:07 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-z[34362]: 2024-05-03T18:27:07.664+0000 7f948c8e8200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-03T18:27:07.794 INFO:journalctl@ceph.mgr.z.smithi150.stdout:May 03 18:27:07 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-z[34362]: 2024-05-03T18:27:07.792+0000 7f948c8e8200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-03T18:27:07.801 DEBUG:teuthology.orchestra.run.smithi033:> cat /proc/self/mounts | awk '{print $2}' 2024-05-03T18:27:07.844 INFO:teuthology.orchestra.run.smithi033.stdout:/proc 2024-05-03T18:27:07.844 INFO:teuthology.orchestra.run.smithi033.stdout:/sys 2024-05-03T18:27:07.844 INFO:teuthology.orchestra.run.smithi033.stdout:/dev 2024-05-03T18:27:07.844 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/security 2024-05-03T18:27:07.844 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/shm 2024-05-03T18:27:07.844 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/pts 2024-05-03T18:27:07.844 INFO:teuthology.orchestra.run.smithi033.stdout:/run 2024-05-03T18:27:07.845 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/cgroup 2024-05-03T18:27:07.845 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/pstore 2024-05-03T18:27:07.845 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/bpf 2024-05-03T18:27:07.845 INFO:teuthology.orchestra.run.smithi033.stdout:/ 2024-05-03T18:27:07.845 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/selinux 2024-05-03T18:27:07.845 INFO:teuthology.orchestra.run.smithi033.stdout:/proc/sys/fs/binfmt_misc 2024-05-03T18:27:07.845 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/hugepages 2024-05-03T18:27:07.845 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/mqueue 2024-05-03T18:27:07.845 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/debug 2024-05-03T18:27:07.845 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/tracing 2024-05-03T18:27:07.845 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/fuse/connections 2024-05-03T18:27:07.845 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/config 2024-05-03T18:27:07.846 INFO:teuthology.orchestra.run.smithi033.stdout:/run/credentials/systemd-sysctl.service 2024-05-03T18:27:07.846 INFO:teuthology.orchestra.run.smithi033.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-03T18:27:07.846 INFO:teuthology.orchestra.run.smithi033.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-03T18:27:07.846 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/nfs/rpc_pipefs 2024-05-03T18:27:07.846 INFO:teuthology.orchestra.run.smithi033.stdout:/run/user/1000 2024-05-03T18:27:07.846 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/ceph 2024-05-03T18:27:07.846 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay 2024-05-03T18:27:07.846 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/a997dc00d3ffddd941056f480754621e1da9a7c0780c198dce5b7fdfdb5c6518/merged 2024-05-03T18:27:07.846 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/c6be5e3078e9176f50155e446bc1b3af81d7b408fd959a1268e9eb05ab65d47a/merged 2024-05-03T18:27:07.846 INFO:teuthology.orchestra.run.smithi033.stdout:/run/user/0 2024-05-03T18:27:07.847 INFO:teuthology.orchestra.run.smithi033.stdout:/proc/sys/fs/binfmt_misc 2024-05-03T18:27:07.847 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/8f77dd5f9057bf2e6e1a041a8e431aeba47a3737a23202ff7576105d576b222e/merged 2024-05-03T18:27:07.847 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/cb734dec4e7f6b54f637623026860263b8cbfd9afb3f74e8a1581ae03d9a4214/merged 2024-05-03T18:27:07.847 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/4ee9bd3566b1d56373b8c3b03d01a2c876e264155b41da297b0bc3bbcf486161/merged 2024-05-03T18:27:07.847 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/b09ff39c1b8ffc6e847ee4a5a642e5e1e59e10af91359fe6562b05d8728bab04/merged 2024-05-03T18:27:07.847 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/add9b5ad17c6d4c36447f671d7e66e634e26235b5bc07a4d2e48f73dae31d4cc/merged 2024-05-03T18:27:07.847 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/8e1b9f689981c0d088659be1650fd55ef94d3066f00b7066218f53549e16c7a5/merged 2024-05-03T18:27:07.847 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/566ad7a71ceef06ae29a24854e0d1a988dbced8d06582cadd1572fae7758e921/merged 2024-05-03T18:27:07.848 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/52f47aa2e49f2bbd5b8cdff45563f7aff478c10e86a30fe10a1ee15611d1e9de/merged 2024-05-03T18:27:07.848 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns 2024-05-03T18:27:07.848 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns/ceph-ns--tmp-cephfs 2024-05-03T18:27:07.848 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns/ceph-ns--tmp-cephfs 2024-05-03T18:27:07.849 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T18:27:07.849 DEBUG:teuthology.orchestra.run.smithi033:> ls /sys/fs/fuse/connections 2024-05-03T18:27:07.937 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-03T18:27:07.938 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--tmp-cephfs sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /tmp/cephfs --id 0 --client_mountpoint=/volumes/qa/sv_0/70093236-67a4-4327-bd9b-9b6228a777c0 --client_fs=cephfs -o context=system_u:object_r:postgresql_db_t:s0) 2024-05-03T18:27:07.982 DEBUG:teuthology.orchestra.run.smithi033:> sudo modprobe fuse 2024-05-03T18:27:08.045 INFO:journalctl@ceph.mgr.y.smithi099.stdout:May 03 18:27:07 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-y[33047]: 2024-05-03T18:27:07.617+0000 7f5edc2ee200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-03T18:27:08.045 INFO:journalctl@ceph.mgr.y.smithi099.stdout:May 03 18:27:07 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-y[33047]: 2024-05-03T18:27:07.745+0000 7f5edc2ee200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-03T18:27:08.050 DEBUG:teuthology.orchestra.run.smithi033:> cat /proc/self/mounts | awk '{print $2}' 2024-05-03T18:27:08.100 INFO:teuthology.orchestra.run.smithi033.stdout:/proc 2024-05-03T18:27:08.100 INFO:teuthology.orchestra.run.smithi033.stdout:/sys 2024-05-03T18:27:08.100 INFO:teuthology.orchestra.run.smithi033.stdout:/dev 2024-05-03T18:27:08.100 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/security 2024-05-03T18:27:08.100 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/shm 2024-05-03T18:27:08.100 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/pts 2024-05-03T18:27:08.101 INFO:teuthology.orchestra.run.smithi033.stdout:/run 2024-05-03T18:27:08.101 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/cgroup 2024-05-03T18:27:08.101 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/pstore 2024-05-03T18:27:08.101 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/bpf 2024-05-03T18:27:08.101 INFO:teuthology.orchestra.run.smithi033.stdout:/ 2024-05-03T18:27:08.101 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/selinux 2024-05-03T18:27:08.101 INFO:teuthology.orchestra.run.smithi033.stdout:/proc/sys/fs/binfmt_misc 2024-05-03T18:27:08.101 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/hugepages 2024-05-03T18:27:08.101 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/mqueue 2024-05-03T18:27:08.101 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/debug 2024-05-03T18:27:08.101 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/tracing 2024-05-03T18:27:08.101 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/fuse/connections 2024-05-03T18:27:08.102 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/config 2024-05-03T18:27:08.102 INFO:teuthology.orchestra.run.smithi033.stdout:/run/credentials/systemd-sysctl.service 2024-05-03T18:27:08.102 INFO:teuthology.orchestra.run.smithi033.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-03T18:27:08.102 INFO:teuthology.orchestra.run.smithi033.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-03T18:27:08.102 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/nfs/rpc_pipefs 2024-05-03T18:27:08.102 INFO:teuthology.orchestra.run.smithi033.stdout:/run/user/1000 2024-05-03T18:27:08.102 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/ceph 2024-05-03T18:27:08.102 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay 2024-05-03T18:27:08.102 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/a997dc00d3ffddd941056f480754621e1da9a7c0780c198dce5b7fdfdb5c6518/merged 2024-05-03T18:27:08.102 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/c6be5e3078e9176f50155e446bc1b3af81d7b408fd959a1268e9eb05ab65d47a/merged 2024-05-03T18:27:08.102 INFO:teuthology.orchestra.run.smithi033.stdout:/run/user/0 2024-05-03T18:27:08.102 INFO:teuthology.orchestra.run.smithi033.stdout:/proc/sys/fs/binfmt_misc 2024-05-03T18:27:08.103 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/8f77dd5f9057bf2e6e1a041a8e431aeba47a3737a23202ff7576105d576b222e/merged 2024-05-03T18:27:08.103 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/cb734dec4e7f6b54f637623026860263b8cbfd9afb3f74e8a1581ae03d9a4214/merged 2024-05-03T18:27:08.103 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/4ee9bd3566b1d56373b8c3b03d01a2c876e264155b41da297b0bc3bbcf486161/merged 2024-05-03T18:27:08.103 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/b09ff39c1b8ffc6e847ee4a5a642e5e1e59e10af91359fe6562b05d8728bab04/merged 2024-05-03T18:27:08.103 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/add9b5ad17c6d4c36447f671d7e66e634e26235b5bc07a4d2e48f73dae31d4cc/merged 2024-05-03T18:27:08.103 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/8e1b9f689981c0d088659be1650fd55ef94d3066f00b7066218f53549e16c7a5/merged 2024-05-03T18:27:08.103 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/566ad7a71ceef06ae29a24854e0d1a988dbced8d06582cadd1572fae7758e921/merged 2024-05-03T18:27:08.103 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/52f47aa2e49f2bbd5b8cdff45563f7aff478c10e86a30fe10a1ee15611d1e9de/merged 2024-05-03T18:27:08.103 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns 2024-05-03T18:27:08.103 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns/ceph-ns--tmp-cephfs 2024-05-03T18:27:08.103 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns/ceph-ns--tmp-cephfs 2024-05-03T18:27:08.104 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T18:27:08.105 DEBUG:teuthology.orchestra.run.smithi033:> ls /sys/fs/fuse/connections 2024-05-03T18:27:08.154 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi033.stderr:2024-05-03T18:27:08.154+0000 7f1c36127480 -1 init, newargv = 0x55e5988f13c0 newargc=17 2024-05-03T18:27:08.154 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi033.stderr:ceph-fuse[86974]: starting ceph client 2024-05-03T18:27:08.170 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi033.stderr:ceph-fuse[86974]: starting fuse 2024-05-03T18:27:08.401 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:27:08 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:27:08.078+0000 7f529c48b200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-03T18:27:08.402 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:27:08 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:27:08.226+0000 7f529c48b200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-03T18:27:08.545 INFO:journalctl@ceph.mgr.y.smithi099.stdout:May 03 18:27:08 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-y[33047]: 2024-05-03T18:27:08.212+0000 7f5edc2ee200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-03T18:27:08.632 INFO:journalctl@ceph.mgr.z.smithi150.stdout:May 03 18:27:08 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-z[34362]: 2024-05-03T18:27:08.262+0000 7f948c8e8200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-03T18:27:08.984 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:27:08 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:27:08.720+0000 7f529c48b200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-03T18:27:09.045 INFO:journalctl@ceph.mgr.y.smithi099.stdout:May 03 18:27:08 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-y[33047]: 2024-05-03T18:27:08.663+0000 7f5edc2ee200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-03T18:27:09.045 INFO:journalctl@ceph.mgr.y.smithi099.stdout:May 03 18:27:08 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-y[33047]: 2024-05-03T18:27:08.786+0000 7f5edc2ee200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-03T18:27:09.132 INFO:journalctl@ceph.mgr.z.smithi150.stdout:May 03 18:27:08 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-z[34362]: 2024-05-03T18:27:08.715+0000 7f948c8e8200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-03T18:27:09.133 INFO:journalctl@ceph.mgr.z.smithi150.stdout:May 03 18:27:08 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-z[34362]: 2024-05-03T18:27:08.840+0000 7f948c8e8200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-03T18:27:09.246 DEBUG:teuthology.orchestra.run.smithi033:> sudo modprobe fuse 2024-05-03T18:27:09.325 DEBUG:teuthology.orchestra.run.smithi033:> cat /proc/self/mounts | awk '{print $2}' 2024-05-03T18:27:09.368 INFO:teuthology.orchestra.run.smithi033.stdout:/proc 2024-05-03T18:27:09.368 INFO:teuthology.orchestra.run.smithi033.stdout:/sys 2024-05-03T18:27:09.368 INFO:teuthology.orchestra.run.smithi033.stdout:/dev 2024-05-03T18:27:09.368 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/security 2024-05-03T18:27:09.369 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/shm 2024-05-03T18:27:09.369 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/pts 2024-05-03T18:27:09.369 INFO:teuthology.orchestra.run.smithi033.stdout:/run 2024-05-03T18:27:09.369 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/cgroup 2024-05-03T18:27:09.369 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/pstore 2024-05-03T18:27:09.369 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/bpf 2024-05-03T18:27:09.369 INFO:teuthology.orchestra.run.smithi033.stdout:/ 2024-05-03T18:27:09.369 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/selinux 2024-05-03T18:27:09.369 INFO:teuthology.orchestra.run.smithi033.stdout:/proc/sys/fs/binfmt_misc 2024-05-03T18:27:09.369 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/hugepages 2024-05-03T18:27:09.369 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/mqueue 2024-05-03T18:27:09.369 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/debug 2024-05-03T18:27:09.369 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/tracing 2024-05-03T18:27:09.369 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/fuse/connections 2024-05-03T18:27:09.370 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/config 2024-05-03T18:27:09.370 INFO:teuthology.orchestra.run.smithi033.stdout:/run/credentials/systemd-sysctl.service 2024-05-03T18:27:09.370 INFO:teuthology.orchestra.run.smithi033.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-03T18:27:09.370 INFO:teuthology.orchestra.run.smithi033.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-03T18:27:09.370 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/nfs/rpc_pipefs 2024-05-03T18:27:09.370 INFO:teuthology.orchestra.run.smithi033.stdout:/run/user/1000 2024-05-03T18:27:09.370 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/ceph 2024-05-03T18:27:09.370 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay 2024-05-03T18:27:09.370 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/a997dc00d3ffddd941056f480754621e1da9a7c0780c198dce5b7fdfdb5c6518/merged 2024-05-03T18:27:09.370 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/c6be5e3078e9176f50155e446bc1b3af81d7b408fd959a1268e9eb05ab65d47a/merged 2024-05-03T18:27:09.370 INFO:teuthology.orchestra.run.smithi033.stdout:/run/user/0 2024-05-03T18:27:09.370 INFO:teuthology.orchestra.run.smithi033.stdout:/proc/sys/fs/binfmt_misc 2024-05-03T18:27:09.370 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/8f77dd5f9057bf2e6e1a041a8e431aeba47a3737a23202ff7576105d576b222e/merged 2024-05-03T18:27:09.371 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/cb734dec4e7f6b54f637623026860263b8cbfd9afb3f74e8a1581ae03d9a4214/merged 2024-05-03T18:27:09.371 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/4ee9bd3566b1d56373b8c3b03d01a2c876e264155b41da297b0bc3bbcf486161/merged 2024-05-03T18:27:09.371 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/b09ff39c1b8ffc6e847ee4a5a642e5e1e59e10af91359fe6562b05d8728bab04/merged 2024-05-03T18:27:09.371 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/add9b5ad17c6d4c36447f671d7e66e634e26235b5bc07a4d2e48f73dae31d4cc/merged 2024-05-03T18:27:09.371 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/8e1b9f689981c0d088659be1650fd55ef94d3066f00b7066218f53549e16c7a5/merged 2024-05-03T18:27:09.371 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/566ad7a71ceef06ae29a24854e0d1a988dbced8d06582cadd1572fae7758e921/merged 2024-05-03T18:27:09.371 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/52f47aa2e49f2bbd5b8cdff45563f7aff478c10e86a30fe10a1ee15611d1e9de/merged 2024-05-03T18:27:09.371 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns 2024-05-03T18:27:09.371 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns/ceph-ns--tmp-cephfs 2024-05-03T18:27:09.371 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns/ceph-ns--tmp-cephfs 2024-05-03T18:27:09.371 INFO:teuthology.orchestra.run.smithi033.stdout:/tmp/cephfs 2024-05-03T18:27:09.372 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T18:27:09.372 DEBUG:teuthology.orchestra.run.smithi033:> ls /sys/fs/fuse/connections 2024-05-03T18:27:09.460 INFO:teuthology.orchestra.run.smithi033.stdout:114 2024-05-03T18:27:09.460 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [114] 2024-05-03T18:27:09.460 DEBUG:teuthology.orchestra.run.smithi033:> sudo stdin-killer -- python3 -c ' 2024-05-03T18:27:09.461 DEBUG:teuthology.orchestra.run.smithi033:> import glob 2024-05-03T18:27:09.461 DEBUG:teuthology.orchestra.run.smithi033:> import re 2024-05-03T18:27:09.461 DEBUG:teuthology.orchestra.run.smithi033:> import os 2024-05-03T18:27:09.461 DEBUG:teuthology.orchestra.run.smithi033:> import subprocess 2024-05-03T18:27:09.461 DEBUG:teuthology.orchestra.run.smithi033:> 2024-05-03T18:27:09.461 DEBUG:teuthology.orchestra.run.smithi033:> def _find_admin_socket(client_name): 2024-05-03T18:27:09.461 DEBUG:teuthology.orchestra.run.smithi033:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-03T18:27:09.461 DEBUG:teuthology.orchestra.run.smithi033:> files = glob.glob(asok_path) 2024-05-03T18:27:09.461 DEBUG:teuthology.orchestra.run.smithi033:> mountpoint = "/tmp/cephfs" 2024-05-03T18:27:09.461 DEBUG:teuthology.orchestra.run.smithi033:> 2024-05-03T18:27:09.461 DEBUG:teuthology.orchestra.run.smithi033:> # Given a non-glob path, it better be there 2024-05-03T18:27:09.461 DEBUG:teuthology.orchestra.run.smithi033:> if "*" not in asok_path: 2024-05-03T18:27:09.461 DEBUG:teuthology.orchestra.run.smithi033:> assert(len(files) == 1) 2024-05-03T18:27:09.461 DEBUG:teuthology.orchestra.run.smithi033:> return files[0] 2024-05-03T18:27:09.461 DEBUG:teuthology.orchestra.run.smithi033:> 2024-05-03T18:27:09.462 DEBUG:teuthology.orchestra.run.smithi033:> for f in files: 2024-05-03T18:27:09.462 DEBUG:teuthology.orchestra.run.smithi033:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-03T18:27:09.462 DEBUG:teuthology.orchestra.run.smithi033:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-03T18:27:09.462 DEBUG:teuthology.orchestra.run.smithi033:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-03T18:27:09.462 DEBUG:teuthology.orchestra.run.smithi033:> contents = proc_f.read() 2024-05-03T18:27:09.462 DEBUG:teuthology.orchestra.run.smithi033:> if mountpoint in contents: 2024-05-03T18:27:09.462 DEBUG:teuthology.orchestra.run.smithi033:> return f 2024-05-03T18:27:09.462 DEBUG:teuthology.orchestra.run.smithi033:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-03T18:27:09.462 DEBUG:teuthology.orchestra.run.smithi033:> 2024-05-03T18:27:09.462 DEBUG:teuthology.orchestra.run.smithi033:> print(_find_admin_socket("client.0")) 2024-05-03T18:27:09.462 DEBUG:teuthology.orchestra.run.smithi033:> ' 2024-05-03T18:27:09.483 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:27:09 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:27:09.208+0000 7f529c48b200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-03T18:27:09.484 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:27:09 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:27:09.345+0000 7f529c48b200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-03T18:27:09.545 INFO:journalctl@ceph.mgr.y.smithi099.stdout:May 03 18:27:09 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-y[33047]: 2024-05-03T18:27:09.105+0000 7f5edc2ee200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-03T18:27:09.545 INFO:journalctl@ceph.mgr.y.smithi099.stdout:May 03 18:27:09 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-y[33047]: 2024-05-03T18:27:09.285+0000 7f5edc2ee200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-03T18:27:09.632 INFO:journalctl@ceph.mgr.z.smithi150.stdout:May 03 18:27:09 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-z[34362]: 2024-05-03T18:27:09.162+0000 7f948c8e8200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-03T18:27:09.632 INFO:journalctl@ceph.mgr.z.smithi150.stdout:May 03 18:27:09 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-z[34362]: 2024-05-03T18:27:09.339+0000 7f948c8e8200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-03T18:27:09.641 INFO:teuthology.orchestra.run.smithi033.stdout:/var/run/ceph/ceph-client.0.86974.asok 2024-05-03T18:27:09.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-03T18:27:09.657 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.86974.asok 2024-05-03T18:27:09.657 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T18:27:09.657 DEBUG:teuthology.orchestra.run.smithi033:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.86974.asok status 2024-05-03T18:27:09.876 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-03T18:27:09.876 INFO:teuthology.orchestra.run.smithi033.stdout: "metadata": { 2024-05-03T18:27:09.876 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_sha1": "9a047fe7599a549042d06a3fb5b6813a203eabea", 2024-05-03T18:27:09.876 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_version": "ceph version 19.0.0-3450-g9a047fe7 (9a047fe7599a549042d06a3fb5b6813a203eabea) squid (dev)", 2024-05-03T18:27:09.876 INFO:teuthology.orchestra.run.smithi033.stdout: "entity_id": "0", 2024-05-03T18:27:09.876 INFO:teuthology.orchestra.run.smithi033.stdout: "hostname": "smithi033", 2024-05-03T18:27:09.876 INFO:teuthology.orchestra.run.smithi033.stdout: "mount_point": "/tmp/cephfs", 2024-05-03T18:27:09.876 INFO:teuthology.orchestra.run.smithi033.stdout: "pid": "86974", 2024-05-03T18:27:09.876 INFO:teuthology.orchestra.run.smithi033.stdout: "root": "/volumes/qa/sv_0/70093236-67a4-4327-bd9b-9b6228a777c0" 2024-05-03T18:27:09.876 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:27:09.876 INFO:teuthology.orchestra.run.smithi033.stdout: "dentry_count": 0, 2024-05-03T18:27:09.876 INFO:teuthology.orchestra.run.smithi033.stdout: "dentry_pinned_count": 0, 2024-05-03T18:27:09.877 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 15138, 2024-05-03T18:27:09.877 INFO:teuthology.orchestra.run.smithi033.stdout: "inst": { 2024-05-03T18:27:09.877 INFO:teuthology.orchestra.run.smithi033.stdout: "name": { 2024-05-03T18:27:09.877 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "client", 2024-05-03T18:27:09.877 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 15138 2024-05-03T18:27:09.877 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:27:09.877 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": { 2024-05-03T18:27:09.877 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-03T18:27:09.877 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "192.168.0.1:0", 2024-05-03T18:27:09.877 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 3382425018 2024-05-03T18:27:09.877 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:27:09.877 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:27:09.877 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": { 2024-05-03T18:27:09.877 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-03T18:27:09.877 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "192.168.0.1:0", 2024-05-03T18:27:09.878 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 3382425018 2024-05-03T18:27:09.878 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:27:09.878 INFO:teuthology.orchestra.run.smithi033.stdout: "inst_str": "client.15138 192.168.0.1:0/3382425018", 2024-05-03T18:27:09.878 INFO:teuthology.orchestra.run.smithi033.stdout: "addr_str": "192.168.0.1:0/3382425018", 2024-05-03T18:27:09.878 INFO:teuthology.orchestra.run.smithi033.stdout: "inode_count": 5, 2024-05-03T18:27:09.878 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_epoch": 21, 2024-05-03T18:27:09.878 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_epoch": 89, 2024-05-03T18:27:09.878 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_epoch_barrier": 0, 2024-05-03T18:27:09.878 INFO:teuthology.orchestra.run.smithi033.stdout: "blocklisted": false, 2024-05-03T18:27:09.878 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs" 2024-05-03T18:27:09.878 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-03T18:27:09.887 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:27:09 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:27:09.696+0000 7f529c48b200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-03T18:27:09.890 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T18:27:09.890 DEBUG:teuthology.orchestra.run.smithi033:> stat --file-system '--printf=%T 2024-05-03T18:27:09.890 DEBUG:teuthology.orchestra.run.smithi033:> ' -- /tmp/cephfs 2024-05-03T18:27:09.937 INFO:journalctl@ceph.mgr.y.smithi099.stdout:May 03 18:27:09 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-y[33047]: 2024-05-03T18:27:09.659+0000 7f5edc2ee200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-03T18:27:09.937 INFO:journalctl@ceph.mgr.y.smithi099.stdout:May 03 18:27:09 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-y[33047]: 2024-05-03T18:27:09.811+0000 7f5edc2ee200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-03T18:27:09.937 INFO:journalctl@ceph.mgr.y.smithi099.stdout:May 03 18:27:09 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-y[33047]: 2024-05-03T18:27:09.936+0000 7f5edc2ee200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-03T18:27:09.938 INFO:teuthology.orchestra.run.smithi033.stdout:fuseblk 2024-05-03T18:27:09.938 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /tmp/cephfs 2024-05-03T18:27:09.938 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T18:27:09.939 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 1777 /tmp/cephfs 2024-05-03T18:27:09.995 INFO:journalctl@ceph.mgr.z.smithi150.stdout:May 03 18:27:09 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-z[34362]: 2024-05-03T18:27:09.719+0000 7f948c8e8200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-03T18:27:09.995 INFO:journalctl@ceph.mgr.z.smithi150.stdout:May 03 18:27:09 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-z[34362]: 2024-05-03T18:27:09.871+0000 7f948c8e8200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-03T18:27:10.013 INFO:teuthology.run_tasks:Running task check-counter... 2024-05-03T18:27:10.025 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-05-03T18:27:10.060 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-03T18:27:10.137 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:27:09 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:27:09.891+0000 7f529c48b200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-03T18:27:10.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.266+0000 7f5e66a00640 1 -- 172.21.15.33:0/1136541282 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e60101730 msgr2=0x7f5e60101b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:10.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.266+0000 7f5e66a00640 1 --2- 172.21.15.33:0/1136541282 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e60101730 0x7f5e60101b50 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f5e54009900 tx=0x7f5e5402a410 comp rx=0 tx=0).stop 2024-05-03T18:27:10.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.266+0000 7f5e66a00640 1 -- 172.21.15.33:0/1136541282 shutdown_connections 2024-05-03T18:27:10.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.266+0000 7f5e66a00640 1 --2- 172.21.15.33:0/1136541282 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e601032e0 0x7f5e60109b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:10.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.266+0000 7f5e66a00640 1 --2- 172.21.15.33:0/1136541282 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5e60102920 0x7f5e60102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:10.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.266+0000 7f5e66a00640 1 --2- 172.21.15.33:0/1136541282 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e60101730 0x7f5e60101b50 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:10.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.266+0000 7f5e66a00640 1 -- 172.21.15.33:0/1136541282 >> 172.21.15.33:0/1136541282 conn(0x7f5e600fcc80 msgr2=0x7f5e600ff0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:27:10.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.266+0000 7f5e66a00640 1 -- 172.21.15.33:0/1136541282 shutdown_connections 2024-05-03T18:27:10.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.266+0000 7f5e66a00640 1 -- 172.21.15.33:0/1136541282 wait complete. 2024-05-03T18:27:10.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.267+0000 7f5e66a00640 1 Processor -- start 2024-05-03T18:27:10.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.267+0000 7f5e66a00640 1 -- start start 2024-05-03T18:27:10.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.268+0000 7f5e66a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e60101730 0x7f5e6019ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:10.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.268+0000 7f5e66a00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e60102920 0x7f5e6019b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:10.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.268+0000 7f5e66a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5e601032e0 0x7f5e601a21b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:10.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.268+0000 7f5e66a00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5e6010d130 con 0x7f5e60102920 2024-05-03T18:27:10.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.268+0000 7f5e66a00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5e6010cfb0 con 0x7f5e60101730 2024-05-03T18:27:10.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.268+0000 7f5e66a00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f5e6010d2b0 con 0x7f5e601032e0 2024-05-03T18:27:10.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.268+0000 7f5e64c00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e60102920 0x7f5e6019b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:10.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.268+0000 7f5e64c00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e60102920 0x7f5e6019b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47434/0 (socket says 172.21.15.33:47434) 2024-05-03T18:27:10.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.268+0000 7f5e64c00640 1 -- 172.21.15.33:0/1612109386 learned_addr learned my addr 172.21.15.33:0/1612109386 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:27:10.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.268+0000 7f5e65600640 1 --2- 172.21.15.33:0/1612109386 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e60101730 0x7f5e6019ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:10.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.268+0000 7f5e66000640 1 --2- 172.21.15.33:0/1612109386 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5e601032e0 0x7f5e601a21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:10.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.269+0000 7f5e64c00640 1 -- 172.21.15.33:0/1612109386 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5e601032e0 msgr2=0x7f5e601a21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:10.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.269+0000 7f5e64c00640 1 --2- 172.21.15.33:0/1612109386 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5e601032e0 0x7f5e601a21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:10.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.269+0000 7f5e64c00640 1 -- 172.21.15.33:0/1612109386 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e60101730 msgr2=0x7f5e6019ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:10.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.269+0000 7f5e64c00640 1 --2- 172.21.15.33:0/1612109386 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e60101730 0x7f5e6019ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:10.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.269+0000 7f5e64c00640 1 -- 172.21.15.33:0/1612109386 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e601a2850 con 0x7f5e60102920 2024-05-03T18:27:10.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.269+0000 7f5e64c00640 1 --2- 172.21.15.33:0/1612109386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e60102920 0x7f5e6019b180 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c00d950 tx=0x7f5e4c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:27:10.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.269+0000 7f5e5e000640 1 -- 172.21.15.33:0/1612109386 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e4c017070 con 0x7f5e60102920 2024-05-03T18:27:10.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.269+0000 7f5e5e000640 1 -- 172.21.15.33:0/1612109386 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5e4c009c30 con 0x7f5e60102920 2024-05-03T18:27:10.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.269+0000 7f5e66a00640 1 -- 172.21.15.33:0/1612109386 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e601a2b40 con 0x7f5e60102920 2024-05-03T18:27:10.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.269+0000 7f5e5e000640 1 -- 172.21.15.33:0/1612109386 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e4c00f620 con 0x7f5e60102920 2024-05-03T18:27:10.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.269+0000 7f5e66a00640 1 -- 172.21.15.33:0/1612109386 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e601a3060 con 0x7f5e60102920 2024-05-03T18:27:10.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.271+0000 7f5e66a00640 1 -- 172.21.15.33:0/1612109386 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e601a3230 con 0x7f5e60102920 2024-05-03T18:27:10.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.275+0000 7f5e5e000640 1 -- 172.21.15.33:0/1612109386 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 149720+0+0 (secure 0 0 0) 0x7f5e4c010460 con 0x7f5e60102920 2024-05-03T18:27:10.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.277+0000 7f5e5e000640 1 --2- 172.21.15.33:0/1612109386 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f5e480b28d0 0x7f5e480b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:10.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.277+0000 7f5e65600640 1 -- 172.21.15.33:0/1612109386 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f5e480b28d0 msgr2=0x7f5e480b4d90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/3642895501 2024-05-03T18:27:10.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.277+0000 7f5e5e000640 1 -- 172.21.15.33:0/1612109386 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f5e4c0e2550 con 0x7f5e60102920 2024-05-03T18:27:10.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.277+0000 7f5e65600640 1 --2- 172.21.15.33:0/1612109386 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f5e480b28d0 0x7f5e480b4d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-03T18:27:10.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.282+0000 7f5e5e000640 1 -- 172.21.15.33:0/1612109386 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f5e4c0a9d30 con 0x7f5e60102920 2024-05-03T18:27:10.382 INFO:journalctl@ceph.mgr.z.smithi150.stdout:May 03 18:27:09 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-z[34362]: 2024-05-03T18:27:09.994+0000 7f948c8e8200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-03T18:27:10.446 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:27:10 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:27:10.284+0000 7f529c48b200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-03T18:27:10.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.478+0000 7f5e65600640 1 -- 172.21.15.33:0/1612109386 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f5e480b28d0 msgr2=0x7f5e480b4d90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/3642895501 2024-05-03T18:27:10.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.478+0000 7f5e65600640 1 --2- 172.21.15.33:0/1612109386 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f5e480b28d0 0x7f5e480b4d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-03T18:27:10.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.491+0000 7f5e66a00640 1 -- 172.21.15.33:0/1612109386 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5e601a35c0 con 0x7f5e60102920 2024-05-03T18:27:10.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.494+0000 7f5e5e000640 1 -- 172.21.15.33:0/1612109386 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24650 (secure 0 0 0) 0x7f5e4c009de0 con 0x7f5e60102920 2024-05-03T18:27:10.494 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:27:10.494 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":89,"fsid":"5703bba4-0978-11ef-bc97-c7b262605968","created":"2024-05-03T18:11:00.899879+0000","modified":"2024-05-03T18:24:21.532082+0000","last_up_change":"2024-05-03T18:21:00.390173+0000","last_in_change":"2024-05-03T18:20:34.103727+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-03T18:15:37.297738+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-03T18:22:19.875368+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-03T18:22:24.432692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-03T18:22:38.722443+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"fdcc4e66-4ba0-4378-a9a4-7d907cd00b4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":637686669},{"type":"v1","addr":"172.21.15.33:6803","nonce":637686669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":637686669},{"type":"v1","addr":"172.21.15.33:6805","nonce":637686669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":637686669},{"type":"v1","addr":"172.21.15.33:6809","nonce":637686669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":637686669},{"type":"v1","addr":"172.21.15.33:6807","nonce":637686669}]},"public_addr":"172.21.15.33:6803/637686669","cluster_addr":"172.21.15.33:6805/637686669","heartbeat_back_addr":"172.21.15.33:6809/637686669","heartbeat_front_addr":"172.21.15.33:6807/637686669","state":["exists","up"]},{"osd":1,"uuid":"cf78cad3-fc24-43f4-8784-21cabea7277c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":1536122841},{"type":"v1","addr":"172.21.15.99:6801","nonce":1536122841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":1536122841},{"type":"v1","addr":"172.21.15.99:6803","nonce":1536122841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":1536122841},{"type":"v1","addr":"172.21.15.99:6807","nonce":1536122841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":1536122841},{"type":"v1","addr":"172.21.15.99:6805","nonce":1536122841}]},"public_addr":"172.21.15.99:6801/1536122841","cluster_addr":"172.21.15.99:6803/1536122841","heartbeat_back_addr":"172.21.15.99:6807/1536122841","heartbeat_front_addr":"172.21.15.99:6805/1536122841","state":["exists","up"]},{"osd":2,"uuid":"59f066d7-626b-410a-8d67-daca6fbbd3ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":1995659622},{"type":"v1","addr":"172.21.15.150:6801","nonce":1995659622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":1995659622},{"type":"v1","addr":"172.21.15.150:6803","nonce":1995659622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":1995659622},{"type":"v1","addr":"172.21.15.150:6807","nonce":1995659622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":1995659622},{"type":"v1","addr":"172.21.15.150:6805","nonce":1995659622}]},"public_addr":"172.21.15.150:6801/1995659622","cluster_addr":"172.21.15.150:6803/1995659622","heartbeat_back_addr":"172.21.15.150:6807/1995659622","heartbeat_front_addr":"172.21.15.150:6805/1995659622","state":["exists","up"]},{"osd":3,"uuid":"d88ed022-4dcb-45fc-9fc6-41ef7c4bf202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":3814942724},{"type":"v1","addr":"172.21.15.33:6811","nonce":3814942724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":3814942724},{"type":"v1","addr":"172.21.15.33:6813","nonce":3814942724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":3814942724},{"type":"v1","addr":"172.21.15.33:6817","nonce":3814942724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":3814942724},{"type":"v1","addr":"172.21.15.33:6815","nonce":3814942724}]},"public_addr":"172.21.15.33:6811/3814942724","cluster_addr":"172.21.15.33:6813/3814942724","heartbeat_back_addr":"172.21.15.33:6817/3814942724","heartbeat_front_addr":"172.21.15.33:6815/3814942724","state":["exists","up"]},{"osd":4,"uuid":"1235e1a6-ff6e-4668-ac1a-073c2a3b7c98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":29081029},{"type":"v1","addr":"172.21.15.99:6809","nonce":29081029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":29081029},{"type":"v1","addr":"172.21.15.99:6811","nonce":29081029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":29081029},{"type":"v1","addr":"172.21.15.99:6815","nonce":29081029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":29081029},{"type":"v1","addr":"172.21.15.99:6813","nonce":29081029}]},"public_addr":"172.21.15.99:6809/29081029","cluster_addr":"172.21.15.99:6811/29081029","heartbeat_back_addr":"172.21.15.99:6815/29081029","heartbeat_front_addr":"172.21.15.99:6813/29081029","state":["exists","up"]},{"osd":5,"uuid":"4b84bff8-d6d7-4258-a0ee-53bc19802585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":3172189533},{"type":"v1","addr":"172.21.15.150:6809","nonce":3172189533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3172189533},{"type":"v1","addr":"172.21.15.150:6811","nonce":3172189533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3172189533},{"type":"v1","addr":"172.21.15.150:6815","nonce":3172189533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3172189533},{"type":"v1","addr":"172.21.15.150:6813","nonce":3172189533}]},"public_addr":"172.21.15.150:6809/3172189533","cluster_addr":"172.21.15.150:6811/3172189533","heartbeat_back_addr":"172.21.15.150:6815/3172189533","heartbeat_front_addr":"172.21.15.150:6813/3172189533","state":["exists","up"]},{"osd":6,"uuid":"51f5fbdd-3794-4c48-9f6c-478f0fa35ac5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":4234541418},{"type":"v1","addr":"172.21.15.33:6819","nonce":4234541418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":4234541418},{"type":"v1","addr":"172.21.15.33:6821","nonce":4234541418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":4234541418},{"type":"v1","addr":"172.21.15.33:6825","nonce":4234541418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":4234541418},{"type":"v1","addr":"172.21.15.33:6823","nonce":4234541418}]},"public_addr":"172.21.15.33:6819/4234541418","cluster_addr":"172.21.15.33:6821/4234541418","heartbeat_back_addr":"172.21.15.33:6825/4234541418","heartbeat_front_addr":"172.21.15.33:6823/4234541418","state":["exists","up"]},{"osd":7,"uuid":"982cc5b3-3163-4505-9bde-a27a8c289457","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":908673090},{"type":"v1","addr":"172.21.15.99:6817","nonce":908673090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":908673090},{"type":"v1","addr":"172.21.15.99:6819","nonce":908673090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":908673090},{"type":"v1","addr":"172.21.15.99:6823","nonce":908673090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":908673090},{"type":"v1","addr":"172.21.15.99:6821","nonce":908673090}]},"public_addr":"172.21.15.99:6817/908673090","cluster_addr":"172.21.15.99:6819/908673090","heartbeat_back_addr":"172.21.15.99:6823/908673090","heartbeat_front_addr":"172.21.15.99:6821/908673090","state":["exists","up"]},{"osd":8,"uuid":"84c9d037-eb84-4eb4-8aa2-e784b703d36c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":1308083602},{"type":"v1","addr":"172.21.15.150:6817","nonce":1308083602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":1308083602},{"type":"v1","addr":"172.21.15.150:6819","nonce":1308083602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":1308083602},{"type":"v1","addr":"172.21.15.150:6823","nonce":1308083602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":1308083602},{"type":"v1","addr":"172.21.15.150:6821","nonce":1308083602}]},"public_addr":"172.21.15.150:6817/1308083602","cluster_addr":"172.21.15.150:6819/1308083602","heartbeat_back_addr":"172.21.15.150:6823/1308083602","heartbeat_front_addr":"172.21.15.150:6821/1308083602","state":["exists","up"]},{"osd":9,"uuid":"a02e965f-0d8f-45e2-9490-337f7f6e3881","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":4133096905},{"type":"v1","addr":"172.21.15.33:6827","nonce":4133096905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":4133096905},{"type":"v1","addr":"172.21.15.33:6829","nonce":4133096905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4133096905},{"type":"v1","addr":"172.21.15.33:6833","nonce":4133096905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":4133096905},{"type":"v1","addr":"172.21.15.33:6831","nonce":4133096905}]},"public_addr":"172.21.15.33:6827/4133096905","cluster_addr":"172.21.15.33:6829/4133096905","heartbeat_back_addr":"172.21.15.33:6833/4133096905","heartbeat_front_addr":"172.21.15.33:6831/4133096905","state":["exists","up"]},{"osd":10,"uuid":"8b29bb96-76e8-45a0-ab19-8d082d755fe9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":935746136},{"type":"v1","addr":"172.21.15.99:6825","nonce":935746136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":935746136},{"type":"v1","addr":"172.21.15.99:6827","nonce":935746136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":935746136},{"type":"v1","addr":"172.21.15.99:6831","nonce":935746136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":935746136},{"type":"v1","addr":"172.21.15.99:6829","nonce":935746136}]},"public_addr":"172.21.15.99:6825/935746136","cluster_addr":"172.21.15.99:6827/935746136","heartbeat_back_addr":"172.21.15.99:6831/935746136","heartbeat_front_addr":"172.21.15.99:6829/935746136","state":["exists","up"]},{"osd":11,"uuid":"07a6a825-c902-4bfd-97cc-cc9074532af5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2069349808},{"type":"v1","addr":"172.21.15.150:6825","nonce":2069349808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2069349808},{"type":"v1","addr":"172.21.15.150:6827","nonce":2069349808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2069349808},{"type":"v1","addr":"172.21.15.150:6831","nonce":2069349808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2069349808},{"type":"v1","addr":"172.21.15.150:6829","nonce":2069349808}]},"public_addr":"172.21.15.150:6825/2069349808","cluster_addr":"172.21.15.150:6827/2069349808","heartbeat_back_addr":"172.21.15.150:6831/2069349808","heartbeat_front_addr":"172.21.15.150:6829/2069349808","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:14:27.454536+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:15:00.538570+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:15:34.125289+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:16:09.853870+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:16:44.928107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:17:19.644037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:17:56.108443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:18:31.731499+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:19:08.062358+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:19:45.204430+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:20:21.013710+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:20:57.653698+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11},{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/3991947997":"2024-05-04T18:12:19.127652+0000","172.21.15.33:0/965859483":"2024-05-04T18:12:19.127652+0000","172.21.15.33:0/2014902144":"2024-05-04T18:11:44.239835+0000","172.21.15.33:0/1314704208":"2024-05-04T18:11:44.239835+0000","172.21.15.33:0/1614640996":"2024-05-04T18:11:44.239835+0000","172.21.15.33:6800/4007322071":"2024-05-04T18:11:44.239835+0000","172.21.15.33:6801/2830135294":"2024-05-04T18:12:19.127652+0000","172.21.15.33:6800/2830135294":"2024-05-04T18:12:19.127652+0000","172.21.15.33:0/3767665033":"2024-05-04T18:12:19.127652+0000","172.21.15.33:6801/4007322071":"2024-05-04T18:11:44.239835+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-03T18:27:10.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.496+0000 7f5e66a00640 1 -- 172.21.15.33:0/1612109386 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f5e480b28d0 msgr2=0x7f5e480b4d90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-03T18:27:10.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.496+0000 7f5e66a00640 1 --2- 172.21.15.33:0/1612109386 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f5e480b28d0 0x7f5e480b4d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:10.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.496+0000 7f5e66a00640 1 -- 172.21.15.33:0/1612109386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e60102920 msgr2=0x7f5e6019b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:10.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.496+0000 7f5e66a00640 1 --2- 172.21.15.33:0/1612109386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e60102920 0x7f5e6019b180 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c00d950 tx=0x7f5e4c00de10 comp rx=0 tx=0).stop 2024-05-03T18:27:10.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.496+0000 7f5e66a00640 1 -- 172.21.15.33:0/1612109386 shutdown_connections 2024-05-03T18:27:10.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.496+0000 7f5e66a00640 1 --2- 172.21.15.33:0/1612109386 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f5e480b28d0 0x7f5e480b4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:10.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.496+0000 7f5e66a00640 1 --2- 172.21.15.33:0/1612109386 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5e601032e0 0x7f5e601a21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:10.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.496+0000 7f5e66a00640 1 --2- 172.21.15.33:0/1612109386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e60102920 0x7f5e6019b180 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:10.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.496+0000 7f5e66a00640 1 --2- 172.21.15.33:0/1612109386 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e60101730 0x7f5e6019ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:10.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.496+0000 7f5e66a00640 1 -- 172.21.15.33:0/1612109386 >> 172.21.15.33:0/1612109386 conn(0x7f5e600fcc80 msgr2=0x7f5e60069bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:27:10.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.497+0000 7f5e66a00640 1 -- 172.21.15.33:0/1612109386 shutdown_connections 2024-05-03T18:27:10.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.497+0000 7f5e66a00640 1 -- 172.21.15.33:0/1612109386 wait complete. 2024-05-03T18:27:10.516 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-03T18:15:37.297738+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-03T18:22:19.875368+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-03T18:22:24.432692+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-03T18:22:38.722443+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-03T18:27:10.517 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-03T18:27:10.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.711+0000 7f0055000640 1 -- 172.21.15.33:0/521647928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0050108600 msgr2=0x7f005010a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:10.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.711+0000 7f0055000640 1 --2- 172.21.15.33:0/521647928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0050108600 0x7f005010a9f0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f0048009ca0 tx=0x7f004802a8c0 comp rx=0 tx=0).stop 2024-05-03T18:27:10.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.711+0000 7f0055000640 1 -- 172.21.15.33:0/521647928 shutdown_connections 2024-05-03T18:27:10.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.711+0000 7f0055000640 1 --2- 172.21.15.33:0/521647928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0050108600 0x7f005010a9f0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:10.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.711+0000 7f0055000640 1 --2- 172.21.15.33:0/521647928 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0050105cd0 0x7f00501080c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:10.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.711+0000 7f0055000640 1 --2- 172.21.15.33:0/521647928 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f00501033a0 0x7f0050105790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:10.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.711+0000 7f0055000640 1 -- 172.21.15.33:0/521647928 >> 172.21.15.33:0/521647928 conn(0x7f00500fcd50 msgr2=0x7f00500ff170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:27:10.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.711+0000 7f0055000640 1 -- 172.21.15.33:0/521647928 shutdown_connections 2024-05-03T18:27:10.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.711+0000 7f0055000640 1 -- 172.21.15.33:0/521647928 wait complete. 2024-05-03T18:27:10.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.712+0000 7f0055000640 1 Processor -- start 2024-05-03T18:27:10.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.712+0000 7f0055000640 1 -- start start 2024-05-03T18:27:10.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.712+0000 7f0055000640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00501033a0 0x7f00501989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:10.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.712+0000 7f0055000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0050105cd0 0x7f0050198ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:10.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.712+0000 7f0055000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0050108600 0x7f005019ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:10.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.712+0000 7f0055000640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f005010e310 con 0x7f00501033a0 2024-05-03T18:27:10.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.712+0000 7f0055000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f005010e190 con 0x7f0050108600 2024-05-03T18:27:10.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.712+0000 7f0055000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f005010e490 con 0x7f0050105cd0 2024-05-03T18:27:10.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.712+0000 7f004f400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00501033a0 0x7f00501989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:10.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.713+0000 7f004f400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00501033a0 0x7f00501989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47452/0 (socket says 172.21.15.33:47452) 2024-05-03T18:27:10.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.713+0000 7f004f400640 1 -- 172.21.15.33:0/2046994678 learned_addr learned my addr 172.21.15.33:0/2046994678 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:27:10.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.713+0000 7f0047e00640 1 --2- 172.21.15.33:0/2046994678 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0050105cd0 0x7f0050198ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:10.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.713+0000 7f004fe00640 1 --2- 172.21.15.33:0/2046994678 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0050108600 0x7f005019ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:10.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.713+0000 7f004f400640 1 -- 172.21.15.33:0/2046994678 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0050105cd0 msgr2=0x7f0050198ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:10.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.713+0000 7f004f400640 1 --2- 172.21.15.33:0/2046994678 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0050105cd0 0x7f0050198ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:10.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.713+0000 7f004f400640 1 -- 172.21.15.33:0/2046994678 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0050108600 msgr2=0x7f005019ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:10.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.713+0000 7f004f400640 1 --2- 172.21.15.33:0/2046994678 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0050108600 0x7f005019ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:10.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.713+0000 7f004f400640 1 -- 172.21.15.33:0/2046994678 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00501a05d0 con 0x7f00501033a0 2024-05-03T18:27:10.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.713+0000 7f004fe00640 1 --2- 172.21.15.33:0/2046994678 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0050108600 0x7f005019ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:27:10.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.713+0000 7f004f400640 1 --2- 172.21.15.33:0/2046994678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00501033a0 0x7f00501989b0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f004000e9f0 tx=0x7f004000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:27:10.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.713+0000 7f004cc00640 1 -- 172.21.15.33:0/2046994678 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f004000cd80 con 0x7f00501033a0 2024-05-03T18:27:10.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.714+0000 7f004cc00640 1 -- 172.21.15.33:0/2046994678 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f00400054b0 con 0x7f00501033a0 2024-05-03T18:27:10.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.714+0000 7f004cc00640 1 -- 172.21.15.33:0/2046994678 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0040010430 con 0x7f00501033a0 2024-05-03T18:27:10.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.714+0000 7f0055000640 1 -- 172.21.15.33:0/2046994678 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00501a08c0 con 0x7f00501033a0 2024-05-03T18:27:10.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.714+0000 7f0055000640 1 -- 172.21.15.33:0/2046994678 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00501a0dc0 con 0x7f00501033a0 2024-05-03T18:27:10.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.715+0000 7f0055000640 1 -- 172.21.15.33:0/2046994678 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00500fe840 con 0x7f00501033a0 2024-05-03T18:27:10.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.717+0000 7f004cc00640 1 -- 172.21.15.33:0/2046994678 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 149720+0+0 (secure 0 0 0) 0x7f00400046c0 con 0x7f00501033a0 2024-05-03T18:27:10.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.719+0000 7f004cc00640 1 --2- 172.21.15.33:0/2046994678 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f00340b28d0 0x7f00340b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:10.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.719+0000 7f0047e00640 1 -- 172.21.15.33:0/2046994678 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f00340b28d0 msgr2=0x7f00340b4d90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/3642895501 2024-05-03T18:27:10.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.719+0000 7f004cc00640 1 -- 172.21.15.33:0/2046994678 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7f00400e2cb0 con 0x7f00501033a0 2024-05-03T18:27:10.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.719+0000 7f0047e00640 1 --2- 172.21.15.33:0/2046994678 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f00340b28d0 0x7f00340b4d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-03T18:27:10.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.723+0000 7f004cc00640 1 -- 172.21.15.33:0/2046994678 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f00400aa490 con 0x7f00501033a0 2024-05-03T18:27:10.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:10 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/1612109386' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T18:27:10.734 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:27:10 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:27:10.445+0000 7f529c48b200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-03T18:27:10.734 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:27:10 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:27:10.576+0000 7f529c48b200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-03T18:27:10.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:10 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/1612109386' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T18:27:10.882 INFO:journalctl@ceph.mgr.z.smithi150.stdout:May 03 18:27:10 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-z[34362]: 2024-05-03T18:27:10.755+0000 7f948c8e8200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-03T18:27:10.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.920+0000 7f0047e00640 1 -- 172.21.15.33:0/2046994678 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f00340b28d0 msgr2=0x7f00340b4d90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/3642895501 2024-05-03T18:27:10.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.920+0000 7f0047e00640 1 --2- 172.21.15.33:0/2046994678 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f00340b28d0 0x7f00340b4d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-03T18:27:10.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.933+0000 7f0055000640 1 -- 172.21.15.33:0/2046994678 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f00500635b0 con 0x7f00501033a0 2024-05-03T18:27:10.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.934+0000 7f004cc00640 1 -- 172.21.15.33:0/2046994678 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7f00400a9d90 con 0x7f00501033a0 2024-05-03T18:27:10.934 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-05-03T18:27:10.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.937+0000 7f0055000640 1 -- 172.21.15.33:0/2046994678 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f00340b28d0 msgr2=0x7f00340b4d90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-03T18:27:10.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.937+0000 7f0055000640 1 --2- 172.21.15.33:0/2046994678 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f00340b28d0 0x7f00340b4d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:10.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.937+0000 7f0055000640 1 -- 172.21.15.33:0/2046994678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00501033a0 msgr2=0x7f00501989b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:10.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.937+0000 7f0055000640 1 --2- 172.21.15.33:0/2046994678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00501033a0 0x7f00501989b0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f004000e9f0 tx=0x7f004000edf0 comp rx=0 tx=0).stop 2024-05-03T18:27:10.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.937+0000 7f0055000640 1 -- 172.21.15.33:0/2046994678 shutdown_connections 2024-05-03T18:27:10.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.937+0000 7f0055000640 1 --2- 172.21.15.33:0/2046994678 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7f00340b28d0 0x7f00340b4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:10.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.937+0000 7f0055000640 1 --2- 172.21.15.33:0/2046994678 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0050108600 0x7f005019ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:10.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.937+0000 7f0055000640 1 --2- 172.21.15.33:0/2046994678 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0050105cd0 0x7f0050198ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:10.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.937+0000 7f0055000640 1 --2- 172.21.15.33:0/2046994678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00501033a0 0x7f00501989b0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:10.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.937+0000 7f0055000640 1 -- 172.21.15.33:0/2046994678 >> 172.21.15.33:0/2046994678 conn(0x7f00500fcd50 msgr2=0x7f0050103ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:27:10.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.937+0000 7f0055000640 1 -- 172.21.15.33:0/2046994678 shutdown_connections 2024-05-03T18:27:10.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:10.937+0000 7f0055000640 1 -- 172.21.15.33:0/2046994678 wait complete. 2024-05-03T18:27:10.955 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-03T18:27:10.985 INFO:journalctl@ceph.mgr.y.smithi099.stdout:May 03 18:27:10 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-y[33047]: 2024-05-03T18:27:10.695+0000 7f5edc2ee200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-03T18:27:10.985 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:10 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/1612109386' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T18:27:11.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.163+0000 7fe76fe00640 1 -- 172.21.15.33:0/4112852151 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe770104490 msgr2=0x7fe77010ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:11.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.163+0000 7fe76fe00640 1 --2- 172.21.15.33:0/4112852151 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe770104490 0x7fe77010ad20 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fe768009ca0 tx=0x7fe76802a8c0 comp rx=0 tx=0).stop 2024-05-03T18:27:11.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.163+0000 7fe76fe00640 1 -- 172.21.15.33:0/4112852151 shutdown_connections 2024-05-03T18:27:11.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.163+0000 7fe76fe00640 1 --2- 172.21.15.33:0/4112852151 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe770104490 0x7fe77010ad20 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:11.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.163+0000 7fe76fe00640 1 --2- 172.21.15.33:0/4112852151 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe770103ad0 0x7fe770103f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:11.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.163+0000 7fe76fe00640 1 --2- 172.21.15.33:0/4112852151 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe7701028e0 0x7fe770102d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:11.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.163+0000 7fe76fe00640 1 -- 172.21.15.33:0/4112852151 >> 172.21.15.33:0/4112852151 conn(0x7fe7700fde50 msgr2=0x7fe770100290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:27:11.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.163+0000 7fe76fe00640 1 -- 172.21.15.33:0/4112852151 shutdown_connections 2024-05-03T18:27:11.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.164+0000 7fe76fe00640 1 -- 172.21.15.33:0/4112852151 wait complete. 2024-05-03T18:27:11.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.164+0000 7fe76fe00640 1 Processor -- start 2024-05-03T18:27:11.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.165+0000 7fe76fe00640 1 -- start start 2024-05-03T18:27:11.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.165+0000 7fe76fe00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe7701028e0 0x7fe77019abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:11.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.165+0000 7fe76fe00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe770103ad0 0x7fe77019b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:11.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.165+0000 7fe76fe00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe770104490 0x7fe7701a2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:11.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.165+0000 7fe76fe00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe77010e2f0 con 0x7fe770103ad0 2024-05-03T18:27:11.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.165+0000 7fe76fe00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe77010e170 con 0x7fe770104490 2024-05-03T18:27:11.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.165+0000 7fe76fe00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe77010e470 con 0x7fe7701028e0 2024-05-03T18:27:11.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.165+0000 7fe76e000640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe770103ad0 0x7fe77019b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:11.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.165+0000 7fe76e000640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe770103ad0 0x7fe77019b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47476/0 (socket says 172.21.15.33:47476) 2024-05-03T18:27:11.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.165+0000 7fe76e000640 1 -- 172.21.15.33:0/875428575 learned_addr learned my addr 172.21.15.33:0/875428575 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:27:11.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.165+0000 7fe76ea00640 1 --2- 172.21.15.33:0/875428575 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe7701028e0 0x7fe77019abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:11.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.165+0000 7fe76f400640 1 --2- 172.21.15.33:0/875428575 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe770104490 0x7fe7701a2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:11.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.166+0000 7fe76e000640 1 -- 172.21.15.33:0/875428575 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe7701028e0 msgr2=0x7fe77019abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:11.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.166+0000 7fe76e000640 1 --2- 172.21.15.33:0/875428575 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe7701028e0 0x7fe77019abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:11.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.166+0000 7fe76e000640 1 -- 172.21.15.33:0/875428575 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe770104490 msgr2=0x7fe7701a2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:11.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.166+0000 7fe76e000640 1 --2- 172.21.15.33:0/875428575 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe770104490 0x7fe7701a2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:11.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.166+0000 7fe76e000640 1 -- 172.21.15.33:0/875428575 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7701a27d0 con 0x7fe770103ad0 2024-05-03T18:27:11.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.166+0000 7fe76e000640 1 --2- 172.21.15.33:0/875428575 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe770103ad0 0x7fe77019b0f0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fe75c00d950 tx=0x7fe75c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:27:11.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.166+0000 7fe767400640 1 -- 172.21.15.33:0/875428575 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe75c017070 con 0x7fe770103ad0 2024-05-03T18:27:11.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.166+0000 7fe767400640 1 -- 172.21.15.33:0/875428575 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe75c009c30 con 0x7fe770103ad0 2024-05-03T18:27:11.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.166+0000 7fe767400640 1 -- 172.21.15.33:0/875428575 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe75c00f620 con 0x7fe770103ad0 2024-05-03T18:27:11.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.166+0000 7fe76fe00640 1 -- 172.21.15.33:0/875428575 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7701a2ac0 con 0x7fe770103ad0 2024-05-03T18:27:11.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.167+0000 7fe76fe00640 1 -- 172.21.15.33:0/875428575 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7701a3060 con 0x7fe770103ad0 2024-05-03T18:27:11.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.168+0000 7fe76fe00640 1 -- 172.21.15.33:0/875428575 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7701a3230 con 0x7fe770103ad0 2024-05-03T18:27:11.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.173+0000 7fe767400640 1 -- 172.21.15.33:0/875428575 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 149720+0+0 (secure 0 0 0) 0x7fe75c010460 con 0x7fe770103ad0 2024-05-03T18:27:11.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.175+0000 7fe767400640 1 --2- 172.21.15.33:0/875428575 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fe7540b2950 0x7fe7540b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:11.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.175+0000 7fe76ea00640 1 -- 172.21.15.33:0/875428575 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fe7540b2950 msgr2=0x7fe7540b4e10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/3642895501 2024-05-03T18:27:11.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.175+0000 7fe767400640 1 -- 172.21.15.33:0/875428575 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7fe75c0e2370 con 0x7fe770103ad0 2024-05-03T18:27:11.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.175+0000 7fe76ea00640 1 --2- 172.21.15.33:0/875428575 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fe7540b2950 0x7fe7540b4e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-03T18:27:11.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.179+0000 7fe767400640 1 -- 172.21.15.33:0/875428575 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fe75c0a9b50 con 0x7fe770103ad0 2024-05-03T18:27:11.295 INFO:journalctl@ceph.mgr.y.smithi099.stdout:May 03 18:27:10 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-y[33047]: 2024-05-03T18:27:10.984+0000 7f5edc2ee200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-03T18:27:11.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.376+0000 7fe76ea00640 1 -- 172.21.15.33:0/875428575 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fe7540b2950 msgr2=0x7fe7540b4e10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/3642895501 2024-05-03T18:27:11.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.376+0000 7fe76ea00640 1 --2- 172.21.15.33:0/875428575 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fe7540b2950 0x7fe7540b4e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-03T18:27:11.382 INFO:journalctl@ceph.mgr.z.smithi150.stdout:May 03 18:27:11 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-z[34362]: 2024-05-03T18:27:11.043+0000 7f948c8e8200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-03T18:27:11.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.385+0000 7fe76fe00640 1 -- 172.21.15.33:0/875428575 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe7701a34a0 con 0x7fe770103ad0 2024-05-03T18:27:11.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.386+0000 7fe767400640 1 -- 172.21.15.33:0/875428575 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7fe75c009de0 con 0x7fe770103ad0 2024-05-03T18:27:11.387 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-05-03T18:27:11.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.389+0000 7fe76fe00640 1 -- 172.21.15.33:0/875428575 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fe7540b2950 msgr2=0x7fe7540b4e10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-03T18:27:11.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.389+0000 7fe76fe00640 1 --2- 172.21.15.33:0/875428575 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fe7540b2950 0x7fe7540b4e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:11.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.389+0000 7fe76fe00640 1 -- 172.21.15.33:0/875428575 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe770103ad0 msgr2=0x7fe77019b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:11.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.389+0000 7fe76fe00640 1 --2- 172.21.15.33:0/875428575 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe770103ad0 0x7fe77019b0f0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fe75c00d950 tx=0x7fe75c00de10 comp rx=0 tx=0).stop 2024-05-03T18:27:11.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.389+0000 7fe76fe00640 1 -- 172.21.15.33:0/875428575 shutdown_connections 2024-05-03T18:27:11.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.389+0000 7fe76fe00640 1 --2- 172.21.15.33:0/875428575 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fe7540b2950 0x7fe7540b4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:11.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.389+0000 7fe76fe00640 1 --2- 172.21.15.33:0/875428575 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe770104490 0x7fe7701a2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:11.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.389+0000 7fe76fe00640 1 --2- 172.21.15.33:0/875428575 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe770103ad0 0x7fe77019b0f0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:11.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.389+0000 7fe76fe00640 1 --2- 172.21.15.33:0/875428575 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe7701028e0 0x7fe77019abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:11.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.389+0000 7fe76fe00640 1 -- 172.21.15.33:0/875428575 >> 172.21.15.33:0/875428575 conn(0x7fe7700fde50 msgr2=0x7fe770074be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:27:11.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.390+0000 7fe76fe00640 1 -- 172.21.15.33:0/875428575 shutdown_connections 2024-05-03T18:27:11.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.390+0000 7fe76fe00640 1 -- 172.21.15.33:0/875428575 wait complete. 2024-05-03T18:27:11.407 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-03T18:27:11.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.608+0000 7fc102000640 1 -- 172.21.15.33:0/2355421699 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0fc1044f0 msgr2=0x7fc0fc10ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:11.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.608+0000 7fc102000640 1 --2- 172.21.15.33:0/2355421699 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0fc1044f0 0x7fc0fc10ad80 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fc0f4009ca0 tx=0x7fc0f402a8c0 comp rx=0 tx=0).stop 2024-05-03T18:27:11.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.608+0000 7fc102000640 1 -- 172.21.15.33:0/2355421699 shutdown_connections 2024-05-03T18:27:11.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.608+0000 7fc102000640 1 --2- 172.21.15.33:0/2355421699 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0fc1044f0 0x7fc0fc10ad80 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:11.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.608+0000 7fc102000640 1 --2- 172.21.15.33:0/2355421699 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0fc103b30 0x7fc0fc103fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:11.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.608+0000 7fc102000640 1 --2- 172.21.15.33:0/2355421699 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc0fc102940 0x7fc0fc102d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:11.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.608+0000 7fc102000640 1 -- 172.21.15.33:0/2355421699 >> 172.21.15.33:0/2355421699 conn(0x7fc0fc0fdeb0 msgr2=0x7fc0fc1002f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:27:11.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.609+0000 7fc102000640 1 -- 172.21.15.33:0/2355421699 shutdown_connections 2024-05-03T18:27:11.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.609+0000 7fc102000640 1 -- 172.21.15.33:0/2355421699 wait complete. 2024-05-03T18:27:11.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.610+0000 7fc102000640 1 Processor -- start 2024-05-03T18:27:11.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.610+0000 7fc102000640 1 -- start start 2024-05-03T18:27:11.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.610+0000 7fc102000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0fc102940 0x7fc0fc19abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:11.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.610+0000 7fc102000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc0fc103b30 0x7fc0fc19b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:11.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.610+0000 7fc102000640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0fc1044f0 0x7fc0fc1a2190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:11.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.610+0000 7fc102000640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc0fc10e300 con 0x7fc0fc1044f0 2024-05-03T18:27:11.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.610+0000 7fc102000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc0fc10e180 con 0x7fc0fc102940 2024-05-03T18:27:11.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.610+0000 7fc102000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc0fc10e480 con 0x7fc0fc103b30 2024-05-03T18:27:11.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.611+0000 7fc101600640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0fc1044f0 0x7fc0fc1a2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:11.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.611+0000 7fc101600640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0fc1044f0 0x7fc0fc1a2190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47496/0 (socket says 172.21.15.33:47496) 2024-05-03T18:27:11.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.611+0000 7fc101600640 1 -- 172.21.15.33:0/3245388124 learned_addr learned my addr 172.21.15.33:0/3245388124 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:27:11.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.611+0000 7fc0fbe00640 1 --2- 172.21.15.33:0/3245388124 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc0fc103b30 0x7fc0fc19b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:11.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.611+0000 7fc100c00640 1 --2- 172.21.15.33:0/3245388124 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0fc102940 0x7fc0fc19abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:11.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.611+0000 7fc101600640 1 -- 172.21.15.33:0/3245388124 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc0fc103b30 msgr2=0x7fc0fc19b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:11.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.611+0000 7fc101600640 1 --2- 172.21.15.33:0/3245388124 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc0fc103b30 0x7fc0fc19b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:11.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.611+0000 7fc101600640 1 -- 172.21.15.33:0/3245388124 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0fc102940 msgr2=0x7fc0fc19abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:11.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.611+0000 7fc101600640 1 --2- 172.21.15.33:0/3245388124 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0fc102940 0x7fc0fc19abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:11.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.611+0000 7fc101600640 1 -- 172.21.15.33:0/3245388124 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0fc1a2800 con 0x7fc0fc1044f0 2024-05-03T18:27:11.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.612+0000 7fc101600640 1 --2- 172.21.15.33:0/3245388124 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0fc1044f0 0x7fc0fc1a2190 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fc0f4031040 tx=0x7fc0f4003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:27:11.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.612+0000 7fc0f9600640 1 -- 172.21.15.33:0/3245388124 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f4004040 con 0x7fc0fc1044f0 2024-05-03T18:27:11.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.612+0000 7fc0f9600640 1 -- 172.21.15.33:0/3245388124 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc0f4004670 con 0x7fc0fc1044f0 2024-05-03T18:27:11.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.612+0000 7fc102000640 1 -- 172.21.15.33:0/3245388124 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0fc1a2a90 con 0x7fc0fc1044f0 2024-05-03T18:27:11.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.612+0000 7fc0f9600640 1 -- 172.21.15.33:0/3245388124 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f4030d20 con 0x7fc0fc1044f0 2024-05-03T18:27:11.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.612+0000 7fc102000640 1 -- 172.21.15.33:0/3245388124 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0fc1a3030 con 0x7fc0fc1044f0 2024-05-03T18:27:11.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.613+0000 7fc102000640 1 -- 172.21.15.33:0/3245388124 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0fc1a3200 con 0x7fc0fc1044f0 2024-05-03T18:27:11.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.615+0000 7fc0f9600640 1 -- 172.21.15.33:0/3245388124 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) ==== 149720+0+0 (secure 0 0 0) 0x7fc0f40305c0 con 0x7fc0fc1044f0 2024-05-03T18:27:11.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.617+0000 7fc0f9600640 1 --2- 172.21.15.33:0/3245388124 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fc0e40b2950 0x7fc0e40b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:11.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.617+0000 7fc100c00640 1 -- 172.21.15.33:0/3245388124 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fc0e40b2950 msgr2=0x7fc0e40b4e10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/3642895501 2024-05-03T18:27:11.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.617+0000 7fc0f9600640 1 -- 172.21.15.33:0/3245388124 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9129+0+0 (secure 0 0 0) 0x7fc0f4103f90 con 0x7fc0fc1044f0 2024-05-03T18:27:11.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.617+0000 7fc100c00640 1 --2- 172.21.15.33:0/3245388124 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fc0e40b2950 0x7fc0e40b4e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-03T18:27:11.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.619+0000 7fc0f9600640 1 -- 172.21.15.33:0/3245388124 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fc0f40cb880 con 0x7fc0fc1044f0 2024-05-03T18:27:11.718 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:27:11 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:27:11.403+0000 7f529c48b200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-03T18:27:11.719 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:11 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/2046994678' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-03T18:27:11.719 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:11 smithi033 ceph-mon[28330]: Standby manager daemon y restarted 2024-05-03T18:27:11.719 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:11 smithi033 ceph-mon[28330]: Standby manager daemon y started 2024-05-03T18:27:11.719 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:11 smithi033 ceph-mon[28330]: from='mgr.? 172.21.15.99:0/3076344983' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-03T18:27:11.720 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:11 smithi033 ceph-mon[28330]: from='mgr.? 172.21.15.99:0/3076344983' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-03T18:27:11.720 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:11 smithi033 ceph-mon[28330]: from='mgr.? 172.21.15.99:0/3076344983' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-03T18:27:11.720 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:11 smithi033 ceph-mon[28330]: from='mgr.? 172.21.15.99:0/3076344983' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-03T18:27:11.720 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:11 smithi033 ceph-mon[28330]: Standby manager daemon z restarted 2024-05-03T18:27:11.720 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:11 smithi033 ceph-mon[28330]: Standby manager daemon z started 2024-05-03T18:27:11.720 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:11 smithi033 ceph-mon[28330]: from='mgr.? 172.21.15.150:0/4020805386' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-03T18:27:11.720 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:11 smithi033 ceph-mon[28330]: from='mgr.? 172.21.15.150:0/4020805386' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-03T18:27:11.720 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:11 smithi033 ceph-mon[28330]: from='mgr.? 172.21.15.150:0/4020805386' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-03T18:27:11.720 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:11 smithi033 ceph-mon[28330]: from='mgr.? 172.21.15.150:0/4020805386' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-03T18:27:11.720 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:11 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/875428575' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-03T18:27:11.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.752+0000 7fc0f9600640 1 -- 172.21.15.33:0/3245388124 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 20) ==== 149337+0+0 (secure 0 0 0) 0x7fc0f40c7410 con 0x7fc0fc1044f0 2024-05-03T18:27:11.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.753+0000 7fc0f9600640 1 -- 172.21.15.33:0/3245388124 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fc0e40b2950 msgr2=0x7fc0e40b4e10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-03T18:27:11.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.753+0000 7fc0f9600640 1 --2- 172.21.15.33:0/3245388124 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fc0e40b2950 0x7fc0e40b4e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:11.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.881+0000 7fc102000640 1 -- 172.21.15.33:0/3245388124 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc0fc1a33d0 con 0x7fc0fc1044f0 2024-05-03T18:27:11.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.881+0000 7fc0f9600640 1 -- 172.21.15.33:0/3245388124 <== mon.0 v2:172.21.15.33:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7fc0f40f9db0 con 0x7fc0fc1044f0 2024-05-03T18:27:11.881 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2024-05-03T18:27:11.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:11 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/2046994678' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-03T18:27:11.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:11 smithi150 ceph-mon[32492]: Standby manager daemon y restarted 2024-05-03T18:27:11.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:11 smithi150 ceph-mon[32492]: Standby manager daemon y started 2024-05-03T18:27:11.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:11 smithi150 ceph-mon[32492]: from='mgr.? 172.21.15.99:0/3076344983' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-03T18:27:11.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:11 smithi150 ceph-mon[32492]: from='mgr.? 172.21.15.99:0/3076344983' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-03T18:27:11.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:11 smithi150 ceph-mon[32492]: from='mgr.? 172.21.15.99:0/3076344983' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-03T18:27:11.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:11 smithi150 ceph-mon[32492]: from='mgr.? 172.21.15.99:0/3076344983' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-03T18:27:11.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:11 smithi150 ceph-mon[32492]: Standby manager daemon z restarted 2024-05-03T18:27:11.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:11 smithi150 ceph-mon[32492]: Standby manager daemon z started 2024-05-03T18:27:11.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:11 smithi150 ceph-mon[32492]: from='mgr.? 172.21.15.150:0/4020805386' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-03T18:27:11.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:11 smithi150 ceph-mon[32492]: from='mgr.? 172.21.15.150:0/4020805386' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-03T18:27:11.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:11 smithi150 ceph-mon[32492]: from='mgr.? 172.21.15.150:0/4020805386' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-03T18:27:11.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:11 smithi150 ceph-mon[32492]: from='mgr.? 172.21.15.150:0/4020805386' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-03T18:27:11.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:11 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/875428575' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-03T18:27:11.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.884+0000 7fc102000640 1 -- 172.21.15.33:0/3245388124 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0fc1044f0 msgr2=0x7fc0fc1a2190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:11.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.884+0000 7fc102000640 1 --2- 172.21.15.33:0/3245388124 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0fc1044f0 0x7fc0fc1a2190 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fc0f4031040 tx=0x7fc0f4003ba0 comp rx=0 tx=0).stop 2024-05-03T18:27:11.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.884+0000 7fc102000640 1 -- 172.21.15.33:0/3245388124 shutdown_connections 2024-05-03T18:27:11.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.884+0000 7fc102000640 1 --2- 172.21.15.33:0/3245388124 >> [v2:172.21.15.33:6800/3642895501,v1:172.21.15.33:6801/3642895501] conn(0x7fc0e40b2950 0x7fc0e40b4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:11.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.884+0000 7fc102000640 1 --2- 172.21.15.33:0/3245388124 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0fc1044f0 0x7fc0fc1a2190 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:11.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.884+0000 7fc102000640 1 --2- 172.21.15.33:0/3245388124 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc0fc103b30 0x7fc0fc19b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:11.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.884+0000 7fc102000640 1 --2- 172.21.15.33:0/3245388124 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0fc102940 0x7fc0fc19abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:11.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.884+0000 7fc102000640 1 -- 172.21.15.33:0/3245388124 >> 172.21.15.33:0/3245388124 conn(0x7fc0fc0fdeb0 msgr2=0x7fc0fc074be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:27:11.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.885+0000 7fc102000640 1 -- 172.21.15.33:0/3245388124 shutdown_connections 2024-05-03T18:27:11.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:11.885+0000 7fc102000640 1 -- 172.21.15.33:0/3245388124 wait complete. 2024-05-03T18:27:11.904 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-03T18:27:11.971 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:27:11 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:27:11.717+0000 7f529c48b200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-03T18:27:11.996 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:11 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/2046994678' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-03T18:27:11.996 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:11 smithi099 ceph-mon[31658]: Standby manager daemon y restarted 2024-05-03T18:27:11.996 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:11 smithi099 ceph-mon[31658]: Standby manager daemon y started 2024-05-03T18:27:11.996 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:11 smithi099 ceph-mon[31658]: from='mgr.? 172.21.15.99:0/3076344983' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-03T18:27:11.996 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:11 smithi099 ceph-mon[31658]: from='mgr.? 172.21.15.99:0/3076344983' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-03T18:27:11.996 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:11 smithi099 ceph-mon[31658]: from='mgr.? 172.21.15.99:0/3076344983' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-03T18:27:11.996 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:11 smithi099 ceph-mon[31658]: from='mgr.? 172.21.15.99:0/3076344983' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-03T18:27:11.996 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:11 smithi099 ceph-mon[31658]: Standby manager daemon z restarted 2024-05-03T18:27:11.996 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:11 smithi099 ceph-mon[31658]: Standby manager daemon z started 2024-05-03T18:27:11.996 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:11 smithi099 ceph-mon[31658]: from='mgr.? 172.21.15.150:0/4020805386' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-03T18:27:11.996 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:11 smithi099 ceph-mon[31658]: from='mgr.? 172.21.15.150:0/4020805386' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-03T18:27:11.997 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:11 smithi099 ceph-mon[31658]: from='mgr.? 172.21.15.150:0/4020805386' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-03T18:27:11.997 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:11 smithi099 ceph-mon[31658]: from='mgr.? 172.21.15.150:0/4020805386' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-03T18:27:11.997 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:11 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/875428575' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-03T18:27:12.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:12.127+0000 7f60f5000640 1 -- 172.21.15.33:0/387774268 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60f0104af0 msgr2=0x7f60f0104f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:12.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:12.127+0000 7f60f5000640 1 --2- 172.21.15.33:0/387774268 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60f0104af0 0x7f60f0104f10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f60e0009860 tx=0x7f60e002a340 comp rx=0 tx=0).stop 2024-05-03T18:27:12.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:12.127+0000 7f60f5000640 1 -- 172.21.15.33:0/387774268 shutdown_connections 2024-05-03T18:27:12.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:12.127+0000 7f60f5000640 1 --2- 172.21.15.33:0/387774268 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60f01066a0 0x7f60f010cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:12.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:12.127+0000 7f60f5000640 1 --2- 172.21.15.33:0/387774268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60f0105ce0 0x7f60f0106160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:12.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:12.127+0000 7f60f5000640 1 --2- 172.21.15.33:0/387774268 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60f0104af0 0x7f60f0104f10 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:12.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:12.127+0000 7f60f5000640 1 -- 172.21.15.33:0/387774268 >> 172.21.15.33:0/387774268 conn(0x7f60f0100060 msgr2=0x7f60f01024a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:27:12.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:12.129+0000 7f60f5000640 1 -- 172.21.15.33:0/387774268 shutdown_connections 2024-05-03T18:27:12.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:12.129+0000 7f60f5000640 1 -- 172.21.15.33:0/387774268 wait complete. 2024-05-03T18:27:12.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:12.130+0000 7f60f5000640 1 Processor -- start 2024-05-03T18:27:12.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:12.130+0000 7f60f5000640 1 -- start start 2024-05-03T18:27:12.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:12.131+0000 7f60f5000640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60f0104af0 0x7f60f019ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:12.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:12.131+0000 7f60f5000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60f0105ce0 0x7f60f019b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:12.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:12.131+0000 7f60f5000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60f01066a0 0x7f60f01a21c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:12.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:12.131+0000 7f60f5000640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f60f01104d0 con 0x7f60f0104af0 2024-05-03T18:27:12.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:12.131+0000 7f60f5000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f60f0110350 con 0x7f60f0105ce0 2024-05-03T18:27:12.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:12.131+0000 7f60f5000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f60f0110650 con 0x7f60f01066a0 2024-05-03T18:27:12.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:12.131+0000 7f60ef400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60f0104af0 0x7f60f019ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:12.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:12.131+0000 7f60ef400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60f0104af0 0x7f60f019ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47576/0 (socket says 172.21.15.33:47576) 2024-05-03T18:27:12.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:12.131+0000 7f60ef400640 1 -- 172.21.15.33:0/429222979 learned_addr learned my addr 172.21.15.33:0/429222979 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:27:12.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:12.131+0000 7f60efe00640 1 --2- 172.21.15.33:0/429222979 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60f01066a0 0x7f60f01a21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:12.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:12.132+0000 7f60eea00640 1 --2- 172.21.15.33:0/429222979 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60f0105ce0 0x7f60f019b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:12.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:12.132+0000 7f60ef400640 1 -- 172.21.15.33:0/429222979 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60f01066a0 msgr2=0x7f60f01a21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:12.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:12.132+0000 7f60ef400640 1 --2- 172.21.15.33:0/429222979 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60f01066a0 0x7f60f01a21c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:12.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:12.132+0000 7f60ef400640 1 -- 172.21.15.33:0/429222979 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60f0105ce0 msgr2=0x7f60f019b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:12.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:12.132+0000 7f60ef400640 1 --2- 172.21.15.33:0/429222979 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60f0105ce0 0x7f60f019b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:12.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:12.132+0000 7f60ef400640 1 -- 172.21.15.33:0/429222979 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60f01a2860 con 0x7f60f0104af0 2024-05-03T18:27:12.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:12.133+0000 7f60ef400640 1 --2- 172.21.15.33:0/429222979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60f0104af0 0x7f60f019ac50 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f60e002a930 tx=0x7f60e0030760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:27:12.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:12.133+0000 7f60e7e00640 1 -- 172.21.15.33:0/429222979 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e0037040 con 0x7f60f0104af0 2024-05-03T18:27:12.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:12.133+0000 7f60f5000640 1 -- 172.21.15.33:0/429222979 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60f01a2af0 con 0x7f60f0104af0 2024-05-03T18:27:12.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:12.133+0000 7f60f5000640 1 -- 172.21.15.33:0/429222979 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60f01a2fd0 con 0x7f60f0104af0 2024-05-03T18:27:12.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:12.134+0000 7f60e7e00640 1 -- 172.21.15.33:0/429222979 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f60e0032dd0 con 0x7f60f0104af0 2024-05-03T18:27:12.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:12.134+0000 7f60e7e00640 1 -- 172.21.15.33:0/429222979 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e0005050 con 0x7f60f0104af0 2024-05-03T18:27:12.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:12.136+0000 7f60f5000640 1 -- 172.21.15.33:0/429222979 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60d00052c0 con 0x7f60f0104af0 2024-05-03T18:27:12.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:12.140+0000 7f60e7e00640 1 -- 172.21.15.33:0/429222979 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) ==== 149337+0+0 (secure 0 0 0) 0x7f60e0043020 con 0x7f60f0104af0 2024-05-03T18:27:12.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:12.142+0000 7f60e7e00640 1 -- 172.21.15.33:0/429222979 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f60e0101a00 con 0x7f60f0104af0 2024-05-03T18:27:12.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:12.153+0000 7f60e7e00640 1 -- 172.21.15.33:0/429222979 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f60e0103e20 con 0x7f60f0104af0 2024-05-03T18:27:12.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:12.435+0000 7f60f5000640 1 -- 172.21.15.33:0/429222979 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f60d0005dc0 con 0x7f60f0104af0 2024-05-03T18:27:12.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:12.436+0000 7f60e7e00640 1 -- 172.21.15.33:0/429222979 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v90) ==== 103+0+11 (secure 0 0 0) 0x7f60e004a020 con 0x7f60f0104af0 2024-05-03T18:27:12.436 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-05-03T18:27:12.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:12.439+0000 7f60f5000640 1 -- 172.21.15.33:0/429222979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60f0104af0 msgr2=0x7f60f019ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:12.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:12.439+0000 7f60f5000640 1 --2- 172.21.15.33:0/429222979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60f0104af0 0x7f60f019ac50 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f60e002a930 tx=0x7f60e0030760 comp rx=0 tx=0).stop 2024-05-03T18:27:12.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:12.439+0000 7f60f5000640 1 -- 172.21.15.33:0/429222979 shutdown_connections 2024-05-03T18:27:12.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:12.439+0000 7f60f5000640 1 --2- 172.21.15.33:0/429222979 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60f01066a0 0x7f60f01a21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:12.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:12.439+0000 7f60f5000640 1 --2- 172.21.15.33:0/429222979 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60f0105ce0 0x7f60f019b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:12.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:12.439+0000 7f60f5000640 1 --2- 172.21.15.33:0/429222979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60f0104af0 0x7f60f019ac50 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:12.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:12.439+0000 7f60f5000640 1 -- 172.21.15.33:0/429222979 >> 172.21.15.33:0/429222979 conn(0x7f60f0100060 msgr2=0x7f60f0109170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:27:12.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:12.439+0000 7f60f5000640 1 -- 172.21.15.33:0/429222979 shutdown_connections 2024-05-03T18:27:12.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:12.439+0000 7f60f5000640 1 -- 172.21.15.33:0/429222979 wait complete. 2024-05-03T18:27:12.456 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-05-03T18:27:12.456 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T18:27:12.734 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:27:12 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:27:12.344+0000 7f51eda00700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-03T18:27:12.734 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:27:12 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:27:12.344+0000 7f51eda00700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-03T18:27:12.734 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:27:12 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:27:12.344+0000 7f51eda00700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-03T18:27:12.735 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:27:12 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:27:12.344+0000 7f51eda00700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-03T18:27:12.735 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:27:12 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:27:12.344+0000 7f51eda00700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-03T18:27:12.735 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:27:12 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:27:12.356+0000 7f51f1600700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-03T18:27:12.735 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:27:12 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:27:12.356+0000 7f51f1600700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-03T18:27:12.735 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:27:12 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:27:12.356+0000 7f51f1600700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-03T18:27:12.735 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:27:12 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:27:12.356+0000 7f51f1600700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-03T18:27:12.735 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:27:12 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:27:12.356+0000 7f51f1600700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-03T18:27:12.735 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:12 smithi033 ceph-mon[28330]: mgrmap e19: x(active, since 14m), standbys: z, y 2024-05-03T18:27:12.736 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:12 smithi033 ceph-mon[28330]: Active manager daemon x restarted 2024-05-03T18:27:12.736 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:12 smithi033 ceph-mon[28330]: Activating manager daemon x 2024-05-03T18:27:12.736 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:12 smithi033 ceph-mon[28330]: osdmap e90: 12 total, 12 up, 12 in 2024-05-03T18:27:12.736 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:12 smithi033 ceph-mon[28330]: mgrmap e20: x(active, starting, since 0.0246879s), standbys: z, y 2024-05-03T18:27:12.736 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:12 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-03T18:27:12.736 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:12 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T18:27:12.736 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:12 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T18:27:12.736 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:12 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-03T18:27:12.736 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:12 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-03T18:27:12.736 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:12 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-03T18:27:12.736 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:12 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-03T18:27:12.736 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:12 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-03T18:27:12.736 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:12 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-03T18:27:12.736 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:12 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-03T18:27:12.736 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:12 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-03T18:27:12.737 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:12 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-03T18:27:12.737 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:12 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-03T18:27:12.737 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:12 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-03T18:27:12.737 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:12 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-03T18:27:12.737 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:12 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-03T18:27:12.737 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:12 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-03T18:27:12.737 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:12 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-03T18:27:12.737 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:12 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-03T18:27:12.737 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:12 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-03T18:27:12.737 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:12 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-03T18:27:12.737 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:12 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-03T18:27:12.737 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:12 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-03T18:27:12.737 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:12 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-03T18:27:12.737 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:12 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-03T18:27:12.738 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:12 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-03T18:27:12.738 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:12 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-03T18:27:12.738 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:12 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-03T18:27:12.738 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:12 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-03T18:27:12.738 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:12 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-03T18:27:12.738 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:12 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-03T18:27:12.738 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:12 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-03T18:27:12.738 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:12 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-03T18:27:12.738 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:12 smithi033 ceph-mon[28330]: Manager daemon x is now available 2024-05-03T18:27:12.738 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:12 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:27:12.738 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:12 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/3245388124' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-03T18:27:12.738 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:12 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-03T18:27:12.738 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:12 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-03T18:27:12.738 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:12 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/429222979' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-03T18:27:12.884 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:12 smithi150 ceph-mon[32492]: mgrmap e19: x(active, since 14m), standbys: z, y 2024-05-03T18:27:12.884 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:12 smithi150 ceph-mon[32492]: Active manager daemon x restarted 2024-05-03T18:27:12.884 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:12 smithi150 ceph-mon[32492]: Activating manager daemon x 2024-05-03T18:27:12.884 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:12 smithi150 ceph-mon[32492]: osdmap e90: 12 total, 12 up, 12 in 2024-05-03T18:27:12.885 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:12 smithi150 ceph-mon[32492]: mgrmap e20: x(active, starting, since 0.0246879s), standbys: z, y 2024-05-03T18:27:12.885 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:12 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-03T18:27:12.885 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:12 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T18:27:12.885 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:12 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T18:27:12.885 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:12 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-03T18:27:12.885 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:12 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-03T18:27:12.885 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:12 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-03T18:27:12.885 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:12 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-03T18:27:12.885 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:12 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-03T18:27:12.885 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:12 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-03T18:27:12.885 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:12 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-03T18:27:12.885 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:12 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-03T18:27:12.885 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:12 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-03T18:27:12.886 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:12 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-03T18:27:12.886 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:12 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-03T18:27:12.886 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:12 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-03T18:27:12.886 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:12 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-03T18:27:12.886 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:12 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-03T18:27:12.886 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:12 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-03T18:27:12.886 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:12 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-03T18:27:12.886 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:12 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-03T18:27:12.886 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:12 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-03T18:27:12.886 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:12 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-03T18:27:12.886 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:12 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-03T18:27:12.887 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:12 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-03T18:27:12.887 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:12 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-03T18:27:12.887 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:12 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-03T18:27:12.887 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:12 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-03T18:27:12.887 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:12 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-03T18:27:12.887 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:12 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-03T18:27:12.887 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:12 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-03T18:27:12.887 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:12 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-03T18:27:12.887 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:12 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-03T18:27:12.887 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:12 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-03T18:27:12.887 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:12 smithi150 ceph-mon[32492]: Manager daemon x is now available 2024-05-03T18:27:12.887 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:12 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:27:12.888 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:12 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/3245388124' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-03T18:27:12.888 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:12 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-03T18:27:12.888 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:12 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-03T18:27:12.888 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:12 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/429222979' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-03T18:27:12.892 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:27:12.987 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:12 smithi099 ceph-mon[31658]: mgrmap e19: x(active, since 14m), standbys: z, y 2024-05-03T18:27:12.987 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:12 smithi099 ceph-mon[31658]: Active manager daemon x restarted 2024-05-03T18:27:12.987 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:12 smithi099 ceph-mon[31658]: Activating manager daemon x 2024-05-03T18:27:12.987 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:12 smithi099 ceph-mon[31658]: osdmap e90: 12 total, 12 up, 12 in 2024-05-03T18:27:12.987 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:12 smithi099 ceph-mon[31658]: mgrmap e20: x(active, starting, since 0.0246879s), standbys: z, y 2024-05-03T18:27:12.987 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:12 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-03T18:27:12.987 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:12 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T18:27:12.987 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:12 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T18:27:12.987 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:12 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-03T18:27:12.988 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:12 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-03T18:27:12.988 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:12 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-03T18:27:12.988 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:12 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-03T18:27:12.988 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:12 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-03T18:27:12.988 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:12 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-03T18:27:12.988 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:12 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-03T18:27:12.988 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:12 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-03T18:27:12.988 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:12 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-03T18:27:12.988 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:12 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-03T18:27:12.988 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:12 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-03T18:27:12.988 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:12 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-03T18:27:12.988 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:12 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-03T18:27:12.988 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:12 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-03T18:27:12.988 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:12 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-03T18:27:12.988 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:12 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-03T18:27:12.989 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:12 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-03T18:27:12.989 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:12 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-03T18:27:12.989 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:12 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-03T18:27:12.989 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:12 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-03T18:27:12.989 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:12 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-03T18:27:12.989 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:12 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-03T18:27:12.989 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:12 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-03T18:27:12.989 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:12 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-03T18:27:12.989 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:12 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-03T18:27:12.990 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:12 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-03T18:27:12.990 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:12 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-03T18:27:12.990 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:12 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-03T18:27:12.990 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:12 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-03T18:27:12.990 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:12 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-03T18:27:12.990 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:12 smithi099 ceph-mon[31658]: Manager daemon x is now available 2024-05-03T18:27:12.990 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:12 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:27:12.990 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:12 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/3245388124' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-03T18:27:12.990 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:12 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-03T18:27:12.990 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:12 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-03T18:27:12.990 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:12 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/429222979' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-03T18:27:14.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:13 smithi099 ceph-mon[31658]: [03/May/2024:18:27:12] ENGINE Bus STARTING 2024-05-03T18:27:14.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:13 smithi099 ceph-mon[31658]: mgrmap e21: x(active, since 1.03441s), standbys: z, y 2024-05-03T18:27:14.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:13 smithi099 ceph-mon[31658]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:27:14.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:13 smithi099 ceph-mon[31658]: [03/May/2024:18:27:12] ENGINE Serving on https://172.21.15.33:7150 2024-05-03T18:27:14.073 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:13 smithi033 ceph-mon[28330]: [03/May/2024:18:27:12] ENGINE Bus STARTING 2024-05-03T18:27:14.073 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:13 smithi033 ceph-mon[28330]: mgrmap e21: x(active, since 1.03441s), standbys: z, y 2024-05-03T18:27:14.073 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:13 smithi033 ceph-mon[28330]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:27:14.074 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:13 smithi033 ceph-mon[28330]: [03/May/2024:18:27:12] ENGINE Serving on https://172.21.15.33:7150 2024-05-03T18:27:14.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:13 smithi150 ceph-mon[32492]: [03/May/2024:18:27:12] ENGINE Bus STARTING 2024-05-03T18:27:14.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:13 smithi150 ceph-mon[32492]: mgrmap e21: x(active, since 1.03441s), standbys: z, y 2024-05-03T18:27:14.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:13 smithi150 ceph-mon[32492]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:27:14.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:13 smithi150 ceph-mon[32492]: [03/May/2024:18:27:12] ENGINE Serving on https://172.21.15.33:7150 2024-05-03T18:27:14.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:14 smithi033 ceph-mon[28330]: [03/May/2024:18:27:12] ENGINE Serving on http://172.21.15.33:8765 2024-05-03T18:27:14.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:14 smithi033 ceph-mon[28330]: [03/May/2024:18:27:12] ENGINE Bus STARTED 2024-05-03T18:27:14.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:14 smithi033 ceph-mon[28330]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:27:14.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:14 smithi033 ceph-mon[28330]: mgrmap e22: x(active, since 2s), standbys: z, y 2024-05-03T18:27:15.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:14 smithi099 ceph-mon[31658]: [03/May/2024:18:27:12] ENGINE Serving on http://172.21.15.33:8765 2024-05-03T18:27:15.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:14 smithi099 ceph-mon[31658]: [03/May/2024:18:27:12] ENGINE Bus STARTED 2024-05-03T18:27:15.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:14 smithi099 ceph-mon[31658]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:27:15.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:14 smithi099 ceph-mon[31658]: mgrmap e22: x(active, since 2s), standbys: z, y 2024-05-03T18:27:15.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:14 smithi150 ceph-mon[32492]: [03/May/2024:18:27:12] ENGINE Serving on http://172.21.15.33:8765 2024-05-03T18:27:15.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:14 smithi150 ceph-mon[32492]: [03/May/2024:18:27:12] ENGINE Bus STARTED 2024-05-03T18:27:15.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:14 smithi150 ceph-mon[32492]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:27:15.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:14 smithi150 ceph-mon[32492]: mgrmap e22: x(active, since 2s), standbys: z, y 2024-05-03T18:27:15.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:15.313+0000 7f9b1fe00700 1 -- 172.21.15.33:0/2721947281 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b18108850 msgr2=0x7f9b181103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:15.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:15.313+0000 7f9b1fe00700 1 --2- 172.21.15.33:0/2721947281 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b18108850 0x7f9b181103d0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f9b1000b0d0 tx=0x7f9b1000b3e0 comp rx=0 tx=0).stop 2024-05-03T18:27:15.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:15.314+0000 7f9b1fe00700 1 -- 172.21.15.33:0/2721947281 shutdown_connections 2024-05-03T18:27:15.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:15.314+0000 7f9b1fe00700 1 --2- 172.21.15.33:0/2721947281 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b18108850 0x7f9b181103d0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:15.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:15.314+0000 7f9b1fe00700 1 --2- 172.21.15.33:0/2721947281 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b18103750 0x7f9b18103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:15.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:15.314+0000 7f9b1fe00700 1 --2- 172.21.15.33:0/2721947281 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b18104100 0x7f9b18108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:15.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:15.314+0000 7f9b1fe00700 1 -- 172.21.15.33:0/2721947281 >> 172.21.15.33:0/2721947281 conn(0x7f9b1807a600 msgr2=0x7f9b1807aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:27:15.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:15.314+0000 7f9b1fe00700 1 -- 172.21.15.33:0/2721947281 shutdown_connections 2024-05-03T18:27:15.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:15.314+0000 7f9b1fe00700 1 -- 172.21.15.33:0/2721947281 wait complete. 2024-05-03T18:27:15.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:15.315+0000 7f9b1fe00700 1 Processor -- start 2024-05-03T18:27:15.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:15.315+0000 7f9b1fe00700 1 -- start start 2024-05-03T18:27:15.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:15.316+0000 7f9b1fe00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b18103750 0x7f9b1819f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:15.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:15.316+0000 7f9b1fe00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b18104100 0x7f9b1819fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:15.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:15.316+0000 7f9b1fe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b18108850 0x7f9b18199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:15.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:15.316+0000 7f9b1fe00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9b18113540 con 0x7f9b18104100 2024-05-03T18:27:15.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:15.316+0000 7f9b1fe00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9b181133c0 con 0x7f9b18108850 2024-05-03T18:27:15.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:15.316+0000 7f9b1fe00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9b181136c0 con 0x7f9b18103750 2024-05-03T18:27:15.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:15.316+0000 7f9b1ca00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b18104100 0x7f9b1819fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:15.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:15.316+0000 7f9b1ca00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b18104100 0x7f9b1819fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47646/0 (socket says 172.21.15.33:47646) 2024-05-03T18:27:15.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:15.316+0000 7f9b1ca00700 1 -- 172.21.15.33:0/1603502424 learned_addr learned my addr 172.21.15.33:0/1603502424 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:27:15.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:15.316+0000 7f9b1d400700 1 --2- 172.21.15.33:0/1603502424 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b18103750 0x7f9b1819f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:15.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:15.316+0000 7f9b1de00700 1 --2- 172.21.15.33:0/1603502424 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b18108850 0x7f9b18199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:15.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:15.316+0000 7f9b1ca00700 1 -- 172.21.15.33:0/1603502424 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b18103750 msgr2=0x7f9b1819f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:15.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:15.317+0000 7f9b1ca00700 1 --2- 172.21.15.33:0/1603502424 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b18103750 0x7f9b1819f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:15.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:15.317+0000 7f9b1ca00700 1 -- 172.21.15.33:0/1603502424 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b18108850 msgr2=0x7f9b18199820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:15.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:15.317+0000 7f9b1ca00700 1 --2- 172.21.15.33:0/1603502424 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b18108850 0x7f9b18199820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:15.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:15.317+0000 7f9b1ca00700 1 -- 172.21.15.33:0/1603502424 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b1819a000 con 0x7f9b18104100 2024-05-03T18:27:15.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:15.317+0000 7f9b1de00700 1 --2- 172.21.15.33:0/1603502424 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b18108850 0x7f9b18199820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:27:15.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:15.317+0000 7f9b1d400700 1 --2- 172.21.15.33:0/1603502424 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b18103750 0x7f9b1819f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:27:15.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:15.317+0000 7f9b1ca00700 1 --2- 172.21.15.33:0/1603502424 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b18104100 0x7f9b1819fbe0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f9b0400e9a0 tx=0x7f9b0400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:27:15.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:15.317+0000 7f9b16000700 1 -- 172.21.15.33:0/1603502424 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b0400cbd0 con 0x7f9b18104100 2024-05-03T18:27:15.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:15.317+0000 7f9b16000700 1 -- 172.21.15.33:0/1603502424 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9b0400cd70 con 0x7f9b18104100 2024-05-03T18:27:15.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:15.317+0000 7f9b16000700 1 -- 172.21.15.33:0/1603502424 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b04018680 con 0x7f9b18104100 2024-05-03T18:27:15.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:15.317+0000 7f9b1fe00700 1 -- 172.21.15.33:0/1603502424 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b1819a2f0 con 0x7f9b18104100 2024-05-03T18:27:15.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:15.317+0000 7f9b1fe00700 1 -- 172.21.15.33:0/1603502424 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b181a6500 con 0x7f9b18104100 2024-05-03T18:27:15.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:15.319+0000 7f9b1fe00700 1 -- 172.21.15.33:0/1603502424 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b18104be0 con 0x7f9b18104100 2024-05-03T18:27:15.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:15.321+0000 7f9b16000700 1 -- 172.21.15.33:0/1603502424 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 22) ==== 149772+0+0 (secure 0 0 0) 0x7f9b04010950 con 0x7f9b18104100 2024-05-03T18:27:15.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:15.323+0000 7f9b16000700 1 --2- 172.21.15.33:0/1603502424 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f9b000b2a70 0x7f9b000b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:15.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:15.323+0000 7f9b16000700 1 -- 172.21.15.33:0/1603502424 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f9b040e2cb0 con 0x7f9b18104100 2024-05-03T18:27:15.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:15.323+0000 7f9b1d400700 1 --2- 172.21.15.33:0/1603502424 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f9b000b2a70 0x7f9b000b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:15.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:15.324+0000 7f9b1d400700 1 --2- 172.21.15.33:0/1603502424 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f9b000b2a70 0x7f9b000b4f30 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c00bcf0 tx=0x7f9b0c00b2c0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:27:15.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:15.327+0000 7f9b16000700 1 -- 172.21.15.33:0/1603502424 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f9b040aa2a0 con 0x7f9b18104100 2024-05-03T18:27:15.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:15.620+0000 7f9b1fe00700 1 -- 172.21.15.33:0/1603502424 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9b1802e510 con 0x7f9b18104100 2024-05-03T18:27:15.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:15.621+0000 7f9b16000700 1 -- 172.21.15.33:0/1603502424 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11801 (secure 0 0 0) 0x7f9b040b2320 con 0x7f9b18104100 2024-05-03T18:27:15.621 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:27:15.621 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1327393503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1327393503},{"type":"v1","addr":"172.21.15.33:6835","nonce":1327393503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/2461472948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":2461472948},{"type":"v1","addr":"172.21.15.33:6837","nonce":2461472948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14559,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/442269281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":442269281},{"type":"v1","addr":"172.21.15.33:6839","nonce":442269281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14571,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/645571756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":645571756},{"type":"v1","addr":"172.21.15.33:6841","nonce":645571756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/203598909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":203598909},{"type":"v1","addr":"172.21.15.150:6833","nonce":203598909}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/4029842493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":4029842493},{"type":"v1","addr":"172.21.15.150:6835","nonce":4029842493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3463664423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3463664423},{"type":"v1","addr":"172.21.15.150:6837","nonce":3463664423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24481,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6839/4051648018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":4051648018},{"type":"v1","addr":"172.21.15.150:6839","nonce":4051648018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34418,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1865105479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1865105479},{"type":"v1","addr":"172.21.15.99:6833","nonce":1865105479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34445,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/1206393914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1206393914},{"type":"v1","addr":"172.21.15.99:6835","nonce":1206393914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-03T18:22:28.837091+0000","modified":"2024-05-03T18:23:49.758398+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34508},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34472":{"gid":34472,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6837/3360970342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3360970342},{"type":"v1","addr":"172.21.15.99:6837","nonce":3360970342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34508":{"gid":34508,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/919907308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":919907308},{"type":"v1","addr":"172.21.15.99:6839","nonce":919907308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34508,"qdb_cluster":[34508]},"id":1}]} 2024-05-03T18:27:15.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:15.624+0000 7f9b1fe00700 1 -- 172.21.15.33:0/1603502424 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f9b000b2a70 msgr2=0x7f9b000b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:15.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:15.624+0000 7f9b1fe00700 1 --2- 172.21.15.33:0/1603502424 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f9b000b2a70 0x7f9b000b4f30 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c00bcf0 tx=0x7f9b0c00b2c0 comp rx=0 tx=0).stop 2024-05-03T18:27:15.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:15.624+0000 7f9b1fe00700 1 -- 172.21.15.33:0/1603502424 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b18104100 msgr2=0x7f9b1819fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:15.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:15.624+0000 7f9b1fe00700 1 --2- 172.21.15.33:0/1603502424 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b18104100 0x7f9b1819fbe0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f9b0400e9a0 tx=0x7f9b0400ed60 comp rx=0 tx=0).stop 2024-05-03T18:27:15.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:15.624+0000 7f9b1fe00700 1 -- 172.21.15.33:0/1603502424 shutdown_connections 2024-05-03T18:27:15.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:15.624+0000 7f9b1fe00700 1 --2- 172.21.15.33:0/1603502424 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f9b000b2a70 0x7f9b000b4f30 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:15.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:15.624+0000 7f9b1fe00700 1 --2- 172.21.15.33:0/1603502424 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b18108850 0x7f9b18199820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:15.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:15.624+0000 7f9b1fe00700 1 --2- 172.21.15.33:0/1603502424 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b18103750 0x7f9b1819f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:15.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:15.624+0000 7f9b1fe00700 1 --2- 172.21.15.33:0/1603502424 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b18104100 0x7f9b1819fbe0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:15.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:15.624+0000 7f9b1fe00700 1 -- 172.21.15.33:0/1603502424 >> 172.21.15.33:0/1603502424 conn(0x7f9b1807a600 msgr2=0x7f9b181017e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:27:15.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:15.625+0000 7f9b1fe00700 1 -- 172.21.15.33:0/1603502424 shutdown_connections 2024-05-03T18:27:15.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:15.625+0000 7f9b1fe00700 1 -- 172.21.15.33:0/1603502424 wait complete. 2024-05-03T18:27:15.626 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 21 2024-05-03T18:27:16.513 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:16 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:27:16.513 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:16 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:27:16.513 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:16 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:27:16.514 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:16 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:27:16.514 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:16 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/1603502424' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:27:16.514 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:16 smithi033 ceph-mon[28330]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:27:16.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:16 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:27:16.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:16 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:27:16.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:16 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:27:16.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:16 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:27:16.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:16 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/1603502424' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:27:16.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:16 smithi099 ceph-mon[31658]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:27:16.568 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-05-03T18:27:16.569 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-05-03T18:27:16.569 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-03T18:27:16.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:16 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:27:16.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:16 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:27:16.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:16 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:27:16.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:16 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:27:16.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:16 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/1603502424' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:27:16.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:16 smithi150 ceph-mon[32492]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:27:16.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:16.774+0000 7f364bc00640 1 -- 172.21.15.33:0/4133276603 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36440785b0 msgr2=0x7f3644078a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:16.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:16.774+0000 7f364bc00640 1 --2- 172.21.15.33:0/4133276603 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36440785b0 0x7f3644078a10 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f3630009900 tx=0x7f363002a410 comp rx=0 tx=0).stop 2024-05-03T18:27:16.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:16.774+0000 7f364bc00640 1 -- 172.21.15.33:0/4133276603 shutdown_connections 2024-05-03T18:27:16.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:16.774+0000 7f364bc00640 1 --2- 172.21.15.33:0/4133276603 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3644078f50 0x7f36440797f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:16.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:16.774+0000 7f364bc00640 1 --2- 172.21.15.33:0/4133276603 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36440785b0 0x7f3644078a10 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:16.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:16.774+0000 7f364bc00640 1 --2- 172.21.15.33:0/4133276603 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3644077370 0x7f3644077790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:16.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:16.774+0000 7f364bc00640 1 -- 172.21.15.33:0/4133276603 >> 172.21.15.33:0/4133276603 conn(0x7f36441001a0 msgr2=0x7f36441025e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:27:16.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:16.774+0000 7f364bc00640 1 -- 172.21.15.33:0/4133276603 shutdown_connections 2024-05-03T18:27:16.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:16.774+0000 7f364bc00640 1 -- 172.21.15.33:0/4133276603 wait complete. 2024-05-03T18:27:16.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:16.775+0000 7f364bc00640 1 Processor -- start 2024-05-03T18:27:16.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:16.776+0000 7f364bc00640 1 -- start start 2024-05-03T18:27:16.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:16.776+0000 7f364bc00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3644077370 0x7f364419f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:16.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:16.776+0000 7f364bc00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36440785b0 0x7f364419f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:16.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:16.776+0000 7f364bc00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3644078f50 0x7f36441a6720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:16.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:16.776+0000 7f364bc00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3644114a60 con 0x7f36440785b0 2024-05-03T18:27:16.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:16.776+0000 7f364bc00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f36441148e0 con 0x7f3644078f50 2024-05-03T18:27:16.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:16.776+0000 7f364bc00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3644114be0 con 0x7f3644077370 2024-05-03T18:27:16.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:16.776+0000 7f3643e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36440785b0 0x7f364419f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:16.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:16.776+0000 7f3643e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36440785b0 0x7f364419f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50750/0 (socket says 172.21.15.33:50750) 2024-05-03T18:27:16.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:16.776+0000 7f3643e00640 1 -- 172.21.15.33:0/2713868663 learned_addr learned my addr 172.21.15.33:0/2713868663 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:27:16.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:16.776+0000 7f3649200640 1 --2- 172.21.15.33:0/2713868663 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3644077370 0x7f364419f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:16.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:16.776+0000 7f3649c00640 1 --2- 172.21.15.33:0/2713868663 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3644078f50 0x7f36441a6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:16.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:16.777+0000 7f3643e00640 1 -- 172.21.15.33:0/2713868663 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3644077370 msgr2=0x7f364419f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:16.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:16.777+0000 7f3643e00640 1 --2- 172.21.15.33:0/2713868663 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3644077370 0x7f364419f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:16.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:16.777+0000 7f3643e00640 1 -- 172.21.15.33:0/2713868663 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3644078f50 msgr2=0x7f36441a6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:16.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:16.777+0000 7f3643e00640 1 --2- 172.21.15.33:0/2713868663 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3644078f50 0x7f36441a6720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:16.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:16.777+0000 7f3643e00640 1 -- 172.21.15.33:0/2713868663 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36441a6d90 con 0x7f36440785b0 2024-05-03T18:27:16.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:16.777+0000 7f3649200640 1 --2- 172.21.15.33:0/2713868663 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3644077370 0x7f364419f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:27:16.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:16.777+0000 7f3643e00640 1 --2- 172.21.15.33:0/2713868663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36440785b0 0x7f364419f700 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f3630031460 tx=0x7f3630003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:27:16.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:16.778+0000 7f3641600640 1 -- 172.21.15.33:0/2713868663 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f363003f070 con 0x7f36440785b0 2024-05-03T18:27:16.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:16.778+0000 7f3641600640 1 -- 172.21.15.33:0/2713868663 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3630031ba0 con 0x7f36440785b0 2024-05-03T18:27:16.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:16.778+0000 7f3641600640 1 -- 172.21.15.33:0/2713868663 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3630035030 con 0x7f36440785b0 2024-05-03T18:27:16.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:16.778+0000 7f364bc00640 1 -- 172.21.15.33:0/2713868663 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36441a7020 con 0x7f36440785b0 2024-05-03T18:27:16.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:16.778+0000 7f364bc00640 1 -- 172.21.15.33:0/2713868663 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36441a74c0 con 0x7f36440785b0 2024-05-03T18:27:16.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:16.779+0000 7f364bc00640 1 -- 172.21.15.33:0/2713868663 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36441117c0 con 0x7f36440785b0 2024-05-03T18:27:16.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:16.782+0000 7f3641600640 1 -- 172.21.15.33:0/2713868663 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 23) ==== 149828+0+0 (secure 0 0 0) 0x7f3630042070 con 0x7f36440785b0 2024-05-03T18:27:16.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:16.784+0000 7f3641600640 1 --2- 172.21.15.33:0/2713868663 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f362c0b2a70 0x7f362c0b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:16.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:16.785+0000 7f3641600640 1 -- 172.21.15.33:0/2713868663 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f3630103270 con 0x7f36440785b0 2024-05-03T18:27:16.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:16.785+0000 7f3649200640 1 --2- 172.21.15.33:0/2713868663 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f362c0b2a70 0x7f362c0b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:16.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:16.785+0000 7f3649200640 1 --2- 172.21.15.33:0/2713868663 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f362c0b2a70 0x7f362c0b4f30 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3638009fc0 tx=0x7f3638005f90 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:27:16.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:16.788+0000 7f3641600640 1 -- 172.21.15.33:0/2713868663 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f36300ca100 con 0x7f36440785b0 2024-05-03T18:27:17.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.013+0000 7f364bc00640 1 -- 172.21.15.33:0/2713868663 --> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3644062df0 con 0x7f362c0b2a70 2024-05-03T18:27:17.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.025+0000 7f3641600640 1 -- 172.21.15.33:0/2713868663 <== mgr.15120 v2:172.21.15.33:6800/3302011241 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339452 (secure 0 0 0) 0x7f3644062df0 con 0x7f362c0b2a70 2024-05-03T18:27:17.030 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:27:17.032 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-05-03T18:27:17.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.031+0000 7f364bc00640 1 -- 172.21.15.33:0/2713868663 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f362c0b2a70 msgr2=0x7f362c0b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:17.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.031+0000 7f364bc00640 1 --2- 172.21.15.33:0/2713868663 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f362c0b2a70 0x7f362c0b4f30 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3638009fc0 tx=0x7f3638005f90 comp rx=0 tx=0).stop 2024-05-03T18:27:17.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.031+0000 7f364bc00640 1 -- 172.21.15.33:0/2713868663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36440785b0 msgr2=0x7f364419f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:17.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.031+0000 7f364bc00640 1 --2- 172.21.15.33:0/2713868663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36440785b0 0x7f364419f700 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f3630031460 tx=0x7f3630003c10 comp rx=0 tx=0).stop 2024-05-03T18:27:17.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.031+0000 7f364bc00640 1 -- 172.21.15.33:0/2713868663 shutdown_connections 2024-05-03T18:27:17.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.031+0000 7f364bc00640 1 --2- 172.21.15.33:0/2713868663 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f362c0b2a70 0x7f362c0b4f30 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:17.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.031+0000 7f364bc00640 1 --2- 172.21.15.33:0/2713868663 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3644078f50 0x7f36441a6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:17.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.031+0000 7f364bc00640 1 --2- 172.21.15.33:0/2713868663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36440785b0 0x7f364419f700 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:17.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.031+0000 7f364bc00640 1 --2- 172.21.15.33:0/2713868663 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3644077370 0x7f364419f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:17.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.031+0000 7f364bc00640 1 -- 172.21.15.33:0/2713868663 >> 172.21.15.33:0/2713868663 conn(0x7f36441001a0 msgr2=0x7f3644101960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:27:17.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.032+0000 7f364bc00640 1 -- 172.21.15.33:0/2713868663 shutdown_connections 2024-05-03T18:27:17.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.032+0000 7f364bc00640 1 -- 172.21.15.33:0/2713868663 wait complete. 2024-05-03T18:27:17.050 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-05-03T18:27:15.766443+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1185455,"num_objects":28,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":606,"num_read_kb":435,"num_write":328,"num_write_kb":3562,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":745,"ondisk_log_size":745,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":368728,"kb_used_data":15320,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124573096,"statfs":{"total":1151940427776,"available":1151562850304,"internally_reserved":0,"allocated":15687680,"data_stored":9435691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":361674042},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":57,"apply_latency_ms":57,"commit_latency_ns":57000000,"apply_latency_ns":57000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.980393"},"pg_stats":[{"pgid":"3.3f","version":"90'8","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.845032+0000","last_change":"2024-05-03T18:22:25.595640+0000","last_active":"2024-05-03T18:27:11.845032+0000","last_peered":"2024-05-03T18:27:11.845032+0000","last_clean":"2024-05-03T18:27:11.845032+0000","last_became_active":"2024-05-03T18:22:25.595284+0000","last_became_peered":"2024-05-03T18:22:25.595284+0000","last_unstale":"2024-05-03T18:27:11.845032+0000","last_undegraded":"2024-05-03T18:27:11.845032+0000","last_fullsized":"2024-05-03T18:27:11.845032+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:50:08.867670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.835095+0000","last_change":"2024-05-03T18:22:25.584950+0000","last_active":"2024-05-03T18:27:11.835095+0000","last_peered":"2024-05-03T18:27:11.835095+0000","last_clean":"2024-05-03T18:27:11.835095+0000","last_became_active":"2024-05-03T18:22:25.584613+0000","last_became_peered":"2024-05-03T18:22:25.584613+0000","last_unstale":"2024-05-03T18:27:11.835095+0000","last_undegraded":"2024-05-03T18:27:11.835095+0000","last_fullsized":"2024-05-03T18:27:11.835095+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:00:24.959429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.834333+0000","last_change":"2024-05-03T18:22:25.596799+0000","last_active":"2024-05-03T18:27:11.834333+0000","last_peered":"2024-05-03T18:27:11.834333+0000","last_clean":"2024-05-03T18:27:11.834333+0000","last_became_active":"2024-05-03T18:22:25.596386+0000","last_became_peered":"2024-05-03T18:22:25.596386+0000","last_unstale":"2024-05-03T18:27:11.834333+0000","last_undegraded":"2024-05-03T18:27:11.834333+0000","last_fullsized":"2024-05-03T18:27:11.834333+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:20:07.818282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.830961+0000","last_change":"2024-05-03T18:23:20.534672+0000","last_active":"2024-05-03T18:27:11.830961+0000","last_peered":"2024-05-03T18:27:11.830961+0000","last_clean":"2024-05-03T18:27:11.830961+0000","last_became_active":"2024-05-03T18:23:20.534073+0000","last_became_peered":"2024-05-03T18:23:20.534073+0000","last_unstale":"2024-05-03T18:27:11.830961+0000","last_undegraded":"2024-05-03T18:27:11.830961+0000","last_fullsized":"2024-05-03T18:27:11.830961+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:07:40.204299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.827187+0000","last_change":"2024-05-03T18:22:25.589810+0000","last_active":"2024-05-03T18:27:11.827187+0000","last_peered":"2024-05-03T18:27:11.827187+0000","last_clean":"2024-05-03T18:27:11.827187+0000","last_became_active":"2024-05-03T18:22:25.588220+0000","last_became_peered":"2024-05-03T18:22:25.588220+0000","last_unstale":"2024-05-03T18:27:11.827187+0000","last_undegraded":"2024-05-03T18:27:11.827187+0000","last_fullsized":"2024-05-03T18:27:11.827187+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:43:20.904737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.834092+0000","last_change":"2024-05-03T18:22:25.603148+0000","last_active":"2024-05-03T18:27:11.834092+0000","last_peered":"2024-05-03T18:27:11.834092+0000","last_clean":"2024-05-03T18:27:11.834092+0000","last_became_active":"2024-05-03T18:22:25.602743+0000","last_became_peered":"2024-05-03T18:22:25.602743+0000","last_unstale":"2024-05-03T18:27:11.834092+0000","last_undegraded":"2024-05-03T18:27:11.834092+0000","last_fullsized":"2024-05-03T18:27:11.834092+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:49:27.387863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.837215+0000","last_change":"2024-05-03T18:23:20.532077+0000","last_active":"2024-05-03T18:27:11.837215+0000","last_peered":"2024-05-03T18:27:11.837215+0000","last_clean":"2024-05-03T18:27:11.837215+0000","last_became_active":"2024-05-03T18:23:20.531528+0000","last_became_peered":"2024-05-03T18:23:20.531528+0000","last_unstale":"2024-05-03T18:27:11.837215+0000","last_undegraded":"2024-05-03T18:27:11.837215+0000","last_fullsized":"2024-05-03T18:27:11.837215+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:10:37.037114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.835717+0000","last_change":"2024-05-03T18:22:25.600192+0000","last_active":"2024-05-03T18:27:11.835717+0000","last_peered":"2024-05-03T18:27:11.835717+0000","last_clean":"2024-05-03T18:27:11.835717+0000","last_became_active":"2024-05-03T18:22:25.599663+0000","last_became_peered":"2024-05-03T18:22:25.599663+0000","last_unstale":"2024-05-03T18:27:11.835717+0000","last_undegraded":"2024-05-03T18:27:11.835717+0000","last_fullsized":"2024-05-03T18:27:11.835717+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:09:51.614784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.833605+0000","last_change":"2024-05-03T18:22:25.600302+0000","last_active":"2024-05-03T18:27:11.833605+0000","last_peered":"2024-05-03T18:27:11.833605+0000","last_clean":"2024-05-03T18:27:11.833605+0000","last_became_active":"2024-05-03T18:22:25.599832+0000","last_became_peered":"2024-05-03T18:22:25.599832+0000","last_unstale":"2024-05-03T18:27:11.833605+0000","last_undegraded":"2024-05-03T18:27:11.833605+0000","last_fullsized":"2024-05-03T18:27:11.833605+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:52:12.326156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":57,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.830746+0000","last_change":"2024-05-03T18:22:25.590892+0000","last_active":"2024-05-03T18:27:11.830746+0000","last_peered":"2024-05-03T18:27:11.830746+0000","last_clean":"2024-05-03T18:27:11.830746+0000","last_became_active":"2024-05-03T18:22:25.590516+0000","last_became_peered":"2024-05-03T18:22:25.590516+0000","last_unstale":"2024-05-03T18:27:11.830746+0000","last_undegraded":"2024-05-03T18:27:11.830746+0000","last_fullsized":"2024-05-03T18:27:11.830746+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:57:57.999234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.837902+0000","last_change":"2024-05-03T18:22:25.597307+0000","last_active":"2024-05-03T18:27:11.837902+0000","last_peered":"2024-05-03T18:27:11.837902+0000","last_clean":"2024-05-03T18:27:11.837902+0000","last_became_active":"2024-05-03T18:22:25.596989+0000","last_became_peered":"2024-05-03T18:22:25.596989+0000","last_unstale":"2024-05-03T18:27:11.837902+0000","last_undegraded":"2024-05-03T18:27:11.837902+0000","last_fullsized":"2024-05-03T18:27:11.837902+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:53:09.003353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.831580+0000","last_change":"2024-05-03T18:22:25.601723+0000","last_active":"2024-05-03T18:27:11.831580+0000","last_peered":"2024-05-03T18:27:11.831580+0000","last_clean":"2024-05-03T18:27:11.831580+0000","last_became_active":"2024-05-03T18:22:25.599428+0000","last_became_peered":"2024-05-03T18:22:25.599428+0000","last_unstale":"2024-05-03T18:27:11.831580+0000","last_undegraded":"2024-05-03T18:27:11.831580+0000","last_fullsized":"2024-05-03T18:27:11.831580+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:41:18.584497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.838263+0000","last_change":"2024-05-03T18:22:25.594885+0000","last_active":"2024-05-03T18:27:11.838263+0000","last_peered":"2024-05-03T18:27:11.838263+0000","last_clean":"2024-05-03T18:27:11.838263+0000","last_became_active":"2024-05-03T18:22:25.593939+0000","last_became_peered":"2024-05-03T18:22:25.593939+0000","last_unstale":"2024-05-03T18:27:11.838263+0000","last_undegraded":"2024-05-03T18:27:11.838263+0000","last_fullsized":"2024-05-03T18:27:11.838263+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:03:30.052886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.837979+0000","last_change":"2024-05-03T18:22:25.595228+0000","last_active":"2024-05-03T18:27:11.837979+0000","last_peered":"2024-05-03T18:27:11.837979+0000","last_clean":"2024-05-03T18:27:11.837979+0000","last_became_active":"2024-05-03T18:22:25.589825+0000","last_became_peered":"2024-05-03T18:22:25.589825+0000","last_unstale":"2024-05-03T18:27:11.837979+0000","last_undegraded":"2024-05-03T18:27:11.837979+0000","last_fullsized":"2024-05-03T18:27:11.837979+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:44:05.638939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.835425+0000","last_change":"2024-05-03T18:22:25.595630+0000","last_active":"2024-05-03T18:27:11.835425+0000","last_peered":"2024-05-03T18:27:11.835425+0000","last_clean":"2024-05-03T18:27:11.835425+0000","last_became_active":"2024-05-03T18:22:25.591869+0000","last_became_peered":"2024-05-03T18:22:25.591869+0000","last_unstale":"2024-05-03T18:27:11.835425+0000","last_undegraded":"2024-05-03T18:27:11.835425+0000","last_fullsized":"2024-05-03T18:27:11.835425+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:27:43.579329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.841007+0000","last_change":"2024-05-03T18:22:25.589482+0000","last_active":"2024-05-03T18:27:11.841007+0000","last_peered":"2024-05-03T18:27:11.841007+0000","last_clean":"2024-05-03T18:27:11.841007+0000","last_became_active":"2024-05-03T18:22:25.589142+0000","last_became_peered":"2024-05-03T18:22:25.589142+0000","last_unstale":"2024-05-03T18:27:11.841007+0000","last_undegraded":"2024-05-03T18:27:11.841007+0000","last_fullsized":"2024-05-03T18:27:11.841007+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:37:55.472823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.845642+0000","last_change":"2024-05-03T18:22:25.594569+0000","last_active":"2024-05-03T18:27:11.845642+0000","last_peered":"2024-05-03T18:27:11.845642+0000","last_clean":"2024-05-03T18:27:11.845642+0000","last_became_active":"2024-05-03T18:22:25.594218+0000","last_became_peered":"2024-05-03T18:22:25.594218+0000","last_unstale":"2024-05-03T18:27:11.845642+0000","last_undegraded":"2024-05-03T18:27:11.845642+0000","last_fullsized":"2024-05-03T18:27:11.845642+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:30:42.775673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.835918+0000","last_change":"2024-05-03T18:22:25.597426+0000","last_active":"2024-05-03T18:27:11.835918+0000","last_peered":"2024-05-03T18:27:11.835918+0000","last_clean":"2024-05-03T18:27:11.835918+0000","last_became_active":"2024-05-03T18:22:25.594524+0000","last_became_peered":"2024-05-03T18:22:25.594524+0000","last_unstale":"2024-05-03T18:27:11.835918+0000","last_undegraded":"2024-05-03T18:27:11.835918+0000","last_fullsized":"2024-05-03T18:27:11.835918+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:41:36.984253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.826557+0000","last_change":"2024-05-03T18:22:25.588002+0000","last_active":"2024-05-03T18:27:11.826557+0000","last_peered":"2024-05-03T18:27:11.826557+0000","last_clean":"2024-05-03T18:27:11.826557+0000","last_became_active":"2024-05-03T18:22:25.587547+0000","last_became_peered":"2024-05-03T18:22:25.587547+0000","last_unstale":"2024-05-03T18:27:11.826557+0000","last_undegraded":"2024-05-03T18:27:11.826557+0000","last_fullsized":"2024-05-03T18:27:11.826557+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:47:14.115517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.767050+0000","last_change":"2024-05-03T18:22:40.754602+0000","last_active":"2024-05-03T18:27:11.767050+0000","last_peered":"2024-05-03T18:27:11.767050+0000","last_clean":"2024-05-03T18:27:11.767050+0000","last_became_active":"2024-05-03T18:22:40.752917+0000","last_became_peered":"2024-05-03T18:22:40.752917+0000","last_unstale":"2024-05-03T18:27:11.767050+0000","last_undegraded":"2024-05-03T18:27:11.767050+0000","last_fullsized":"2024-05-03T18:27:11.767050+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:51:44.711811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"78'4","reported_seq":53,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.781114+0000","last_change":"2024-05-03T18:22:21.558560+0000","last_active":"2024-05-03T18:27:11.781114+0000","last_peered":"2024-05-03T18:27:11.781114+0000","last_clean":"2024-05-03T18:27:11.781114+0000","last_became_active":"2024-05-03T18:22:21.558185+0000","last_became_peered":"2024-05-03T18:22:21.558185+0000","last_unstale":"2024-05-03T18:27:11.781114+0000","last_undegraded":"2024-05-03T18:27:11.781114+0000","last_fullsized":"2024-05-03T18:27:11.781114+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:18:19.666724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.830855+0000","last_change":"2024-05-03T18:22:25.603934+0000","last_active":"2024-05-03T18:27:11.830855+0000","last_peered":"2024-05-03T18:27:11.830855+0000","last_clean":"2024-05-03T18:27:11.830855+0000","last_became_active":"2024-05-03T18:22:25.603529+0000","last_became_peered":"2024-05-03T18:22:25.603529+0000","last_unstale":"2024-05-03T18:27:11.830855+0000","last_undegraded":"2024-05-03T18:27:11.830855+0000","last_fullsized":"2024-05-03T18:27:11.830855+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:08:50.288871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.781004+0000","last_change":"2024-05-03T18:23:20.576254+0000","last_active":"2024-05-03T18:27:11.781004+0000","last_peered":"2024-05-03T18:27:11.781004+0000","last_clean":"2024-05-03T18:27:11.781004+0000","last_became_active":"2024-05-03T18:23:20.575774+0000","last_became_peered":"2024-05-03T18:23:20.575774+0000","last_unstale":"2024-05-03T18:27:11.781004+0000","last_undegraded":"2024-05-03T18:27:11.781004+0000","last_fullsized":"2024-05-03T18:27:11.781004+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:22:39.090938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"78'1","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.780167+0000","last_change":"2024-05-03T18:22:21.734171+0000","last_active":"2024-05-03T18:27:11.780167+0000","last_peered":"2024-05-03T18:27:11.780167+0000","last_clean":"2024-05-03T18:27:11.780167+0000","last_became_active":"2024-05-03T18:22:21.733301+0000","last_became_peered":"2024-05-03T18:22:21.733301+0000","last_unstale":"2024-05-03T18:27:11.780167+0000","last_undegraded":"2024-05-03T18:27:11.780167+0000","last_fullsized":"2024-05-03T18:27:11.780167+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:09:48.861247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.845701+0000","last_change":"2024-05-03T18:22:25.598016+0000","last_active":"2024-05-03T18:27:11.845701+0000","last_peered":"2024-05-03T18:27:11.845701+0000","last_clean":"2024-05-03T18:27:11.845701+0000","last_became_active":"2024-05-03T18:22:25.594888+0000","last_became_peered":"2024-05-03T18:22:25.594888+0000","last_unstale":"2024-05-03T18:27:11.845701+0000","last_undegraded":"2024-05-03T18:27:11.845701+0000","last_fullsized":"2024-05-03T18:27:11.845701+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:47:46.400247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.785843+0000","last_change":"2024-05-03T18:22:40.755148+0000","last_active":"2024-05-03T18:27:11.785843+0000","last_peered":"2024-05-03T18:27:11.785843+0000","last_clean":"2024-05-03T18:27:11.785843+0000","last_became_active":"2024-05-03T18:22:40.754656+0000","last_became_peered":"2024-05-03T18:22:40.754656+0000","last_unstale":"2024-05-03T18:27:11.785843+0000","last_undegraded":"2024-05-03T18:27:11.785843+0000","last_fullsized":"2024-05-03T18:27:11.785843+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:48:23.124936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"78'2","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.784439+0000","last_change":"2024-05-03T18:22:21.568060+0000","last_active":"2024-05-03T18:27:11.784439+0000","last_peered":"2024-05-03T18:27:11.784439+0000","last_clean":"2024-05-03T18:27:11.784439+0000","last_became_active":"2024-05-03T18:22:21.567434+0000","last_became_peered":"2024-05-03T18:22:21.567434+0000","last_unstale":"2024-05-03T18:27:11.784439+0000","last_undegraded":"2024-05-03T18:27:11.784439+0000","last_fullsized":"2024-05-03T18:27:11.784439+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:06:05.802621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.829909+0000","last_change":"2024-05-03T18:22:25.597777+0000","last_active":"2024-05-03T18:27:11.829909+0000","last_peered":"2024-05-03T18:27:11.829909+0000","last_clean":"2024-05-03T18:27:11.829909+0000","last_became_active":"2024-05-03T18:22:25.597391+0000","last_became_peered":"2024-05-03T18:22:25.597391+0000","last_unstale":"2024-05-03T18:27:11.829909+0000","last_undegraded":"2024-05-03T18:27:11.829909+0000","last_fullsized":"2024-05-03T18:27:11.829909+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:02:24.708193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.783801+0000","last_change":"2024-05-03T18:22:40.752318+0000","last_active":"2024-05-03T18:27:11.783801+0000","last_peered":"2024-05-03T18:27:11.783801+0000","last_clean":"2024-05-03T18:27:11.783801+0000","last_became_active":"2024-05-03T18:22:40.751433+0000","last_became_peered":"2024-05-03T18:22:40.751433+0000","last_unstale":"2024-05-03T18:27:11.783801+0000","last_undegraded":"2024-05-03T18:27:11.783801+0000","last_fullsized":"2024-05-03T18:27:11.783801+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:21:55.746143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.791173+0000","last_change":"2024-05-03T18:24:21.582330+0000","last_active":"2024-05-03T18:27:11.791173+0000","last_peered":"2024-05-03T18:27:11.791173+0000","last_clean":"2024-05-03T18:27:11.791173+0000","last_became_active":"2024-05-03T18:24:21.572706+0000","last_became_peered":"2024-05-03T18:24:21.572706+0000","last_unstale":"2024-05-03T18:27:11.791173+0000","last_undegraded":"2024-05-03T18:27:11.791173+0000","last_fullsized":"2024-05-03T18:27:11.791173+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:55:40.233995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":62,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.838824+0000","last_change":"2024-05-03T18:23:20.575102+0000","last_active":"2024-05-03T18:27:11.838824+0000","last_peered":"2024-05-03T18:27:11.838824+0000","last_clean":"2024-05-03T18:27:11.838824+0000","last_became_active":"2024-05-03T18:23:20.574657+0000","last_became_peered":"2024-05-03T18:23:20.574657+0000","last_unstale":"2024-05-03T18:27:11.838824+0000","last_undegraded":"2024-05-03T18:27:11.838824+0000","last_fullsized":"2024-05-03T18:27:11.838824+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:08:26.280866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.767873+0000","last_change":"2024-05-03T18:22:40.761052+0000","last_active":"2024-05-03T18:27:11.767873+0000","last_peered":"2024-05-03T18:27:11.767873+0000","last_clean":"2024-05-03T18:27:11.767873+0000","last_became_active":"2024-05-03T18:22:40.760179+0000","last_became_peered":"2024-05-03T18:22:40.760179+0000","last_unstale":"2024-05-03T18:27:11.767873+0000","last_undegraded":"2024-05-03T18:27:11.767873+0000","last_fullsized":"2024-05-03T18:27:11.767873+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:44:48.610680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"78'1","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.745780+0000","last_change":"2024-05-03T18:22:21.563421+0000","last_active":"2024-05-03T18:27:11.745780+0000","last_peered":"2024-05-03T18:27:11.745780+0000","last_clean":"2024-05-03T18:27:11.745780+0000","last_became_active":"2024-05-03T18:22:21.560991+0000","last_became_peered":"2024-05-03T18:22:21.560991+0000","last_unstale":"2024-05-03T18:27:11.745780+0000","last_undegraded":"2024-05-03T18:27:11.745780+0000","last_fullsized":"2024-05-03T18:27:11.745780+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:50:32.640699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.823442+0000","last_change":"2024-05-03T18:22:25.599049+0000","last_active":"2024-05-03T18:27:11.823442+0000","last_peered":"2024-05-03T18:27:11.823442+0000","last_clean":"2024-05-03T18:27:11.823442+0000","last_became_active":"2024-05-03T18:22:25.597879+0000","last_became_peered":"2024-05-03T18:22:25.597879+0000","last_unstale":"2024-05-03T18:27:11.823442+0000","last_undegraded":"2024-05-03T18:27:11.823442+0000","last_fullsized":"2024-05-03T18:27:11.823442+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:51:08.487499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.767790+0000","last_change":"2024-05-03T18:22:40.760908+0000","last_active":"2024-05-03T18:27:11.767790+0000","last_peered":"2024-05-03T18:27:11.767790+0000","last_clean":"2024-05-03T18:27:11.767790+0000","last_became_active":"2024-05-03T18:22:40.760331+0000","last_became_peered":"2024-05-03T18:22:40.760331+0000","last_unstale":"2024-05-03T18:27:11.767790+0000","last_undegraded":"2024-05-03T18:27:11.767790+0000","last_fullsized":"2024-05-03T18:27:11.767790+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:27:08.216015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"78'1","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.761175+0000","last_change":"2024-05-03T18:22:21.565328+0000","last_active":"2024-05-03T18:27:11.761175+0000","last_peered":"2024-05-03T18:27:11.761175+0000","last_clean":"2024-05-03T18:27:11.761175+0000","last_became_active":"2024-05-03T18:22:21.564583+0000","last_became_peered":"2024-05-03T18:22:21.564583+0000","last_unstale":"2024-05-03T18:27:11.761175+0000","last_undegraded":"2024-05-03T18:27:11.761175+0000","last_fullsized":"2024-05-03T18:27:11.761175+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:58:32.370458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.828897+0000","last_change":"2024-05-03T18:22:25.607585+0000","last_active":"2024-05-03T18:27:11.828897+0000","last_peered":"2024-05-03T18:27:11.828897+0000","last_clean":"2024-05-03T18:27:11.828897+0000","last_became_active":"2024-05-03T18:22:25.607168+0000","last_became_peered":"2024-05-03T18:22:25.607168+0000","last_unstale":"2024-05-03T18:27:11.828897+0000","last_undegraded":"2024-05-03T18:27:11.828897+0000","last_fullsized":"2024-05-03T18:27:11.828897+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:04:39.750823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.803875+0000","last_change":"2024-05-03T18:22:40.754744+0000","last_active":"2024-05-03T18:27:11.803875+0000","last_peered":"2024-05-03T18:27:11.803875+0000","last_clean":"2024-05-03T18:27:11.803875+0000","last_became_active":"2024-05-03T18:22:40.754262+0000","last_became_peered":"2024-05-03T18:22:40.754262+0000","last_unstale":"2024-05-03T18:27:11.803875+0000","last_undegraded":"2024-05-03T18:27:11.803875+0000","last_fullsized":"2024-05-03T18:27:11.803875+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:11:47.979546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.738652+0000","last_change":"2024-05-03T18:22:21.550048+0000","last_active":"2024-05-03T18:27:11.738652+0000","last_peered":"2024-05-03T18:27:11.738652+0000","last_clean":"2024-05-03T18:27:11.738652+0000","last_became_active":"2024-05-03T18:22:21.549334+0000","last_became_peered":"2024-05-03T18:22:21.549334+0000","last_unstale":"2024-05-03T18:27:11.738652+0000","last_undegraded":"2024-05-03T18:27:11.738652+0000","last_fullsized":"2024-05-03T18:27:11.738652+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:52:54.401185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.834700+0000","last_change":"2024-05-03T18:22:25.583074+0000","last_active":"2024-05-03T18:27:11.834700+0000","last_peered":"2024-05-03T18:27:11.834700+0000","last_clean":"2024-05-03T18:27:11.834700+0000","last_became_active":"2024-05-03T18:22:25.582637+0000","last_became_peered":"2024-05-03T18:22:25.582637+0000","last_unstale":"2024-05-03T18:27:11.834700+0000","last_undegraded":"2024-05-03T18:27:11.834700+0000","last_fullsized":"2024-05-03T18:27:11.834700+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:20:09.021402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.778047+0000","last_change":"2024-05-03T18:22:40.748509+0000","last_active":"2024-05-03T18:27:11.778047+0000","last_peered":"2024-05-03T18:27:11.778047+0000","last_clean":"2024-05-03T18:27:11.778047+0000","last_became_active":"2024-05-03T18:22:40.748051+0000","last_became_peered":"2024-05-03T18:22:40.748051+0000","last_unstale":"2024-05-03T18:27:11.778047+0000","last_undegraded":"2024-05-03T18:27:11.778047+0000","last_fullsized":"2024-05-03T18:27:11.778047+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:08:44.389460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.742290+0000","last_change":"2024-05-03T18:22:21.562185+0000","last_active":"2024-05-03T18:27:11.742290+0000","last_peered":"2024-05-03T18:27:11.742290+0000","last_clean":"2024-05-03T18:27:11.742290+0000","last_became_active":"2024-05-03T18:22:21.560523+0000","last_became_peered":"2024-05-03T18:22:21.560523+0000","last_unstale":"2024-05-03T18:27:11.742290+0000","last_undegraded":"2024-05-03T18:27:11.742290+0000","last_fullsized":"2024-05-03T18:27:11.742290+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:05:41.788286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.831975+0000","last_change":"2024-05-03T18:22:25.583895+0000","last_active":"2024-05-03T18:27:11.831975+0000","last_peered":"2024-05-03T18:27:11.831975+0000","last_clean":"2024-05-03T18:27:11.831975+0000","last_became_active":"2024-05-03T18:22:25.583503+0000","last_became_peered":"2024-05-03T18:22:25.583503+0000","last_unstale":"2024-05-03T18:27:11.831975+0000","last_undegraded":"2024-05-03T18:27:11.831975+0000","last_fullsized":"2024-05-03T18:27:11.831975+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:45:32.404394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.774438+0000","last_change":"2024-05-03T18:22:40.752814+0000","last_active":"2024-05-03T18:27:11.774438+0000","last_peered":"2024-05-03T18:27:11.774438+0000","last_clean":"2024-05-03T18:27:11.774438+0000","last_became_active":"2024-05-03T18:22:40.752328+0000","last_became_peered":"2024-05-03T18:22:40.752328+0000","last_unstale":"2024-05-03T18:27:11.774438+0000","last_undegraded":"2024-05-03T18:27:11.774438+0000","last_fullsized":"2024-05-03T18:27:11.774438+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:16:39.488122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.762244+0000","last_change":"2024-05-03T18:22:21.566266+0000","last_active":"2024-05-03T18:27:11.762244+0000","last_peered":"2024-05-03T18:27:11.762244+0000","last_clean":"2024-05-03T18:27:11.762244+0000","last_became_active":"2024-05-03T18:22:21.565612+0000","last_became_peered":"2024-05-03T18:22:21.565612+0000","last_unstale":"2024-05-03T18:27:11.762244+0000","last_undegraded":"2024-05-03T18:27:11.762244+0000","last_fullsized":"2024-05-03T18:27:11.762244+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:22:57.216912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.836780+0000","last_change":"2024-05-03T18:22:25.596182+0000","last_active":"2024-05-03T18:27:11.836780+0000","last_peered":"2024-05-03T18:27:11.836780+0000","last_clean":"2024-05-03T18:27:11.836780+0000","last_became_active":"2024-05-03T18:22:25.595472+0000","last_became_peered":"2024-05-03T18:22:25.595472+0000","last_unstale":"2024-05-03T18:27:11.836780+0000","last_undegraded":"2024-05-03T18:27:11.836780+0000","last_fullsized":"2024-05-03T18:27:11.836780+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:48:49.583879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.783452+0000","last_change":"2024-05-03T18:22:40.756378+0000","last_active":"2024-05-03T18:27:11.783452+0000","last_peered":"2024-05-03T18:27:11.783452+0000","last_clean":"2024-05-03T18:27:11.783452+0000","last_became_active":"2024-05-03T18:22:40.755692+0000","last_became_peered":"2024-05-03T18:22:40.755692+0000","last_unstale":"2024-05-03T18:27:11.783452+0000","last_undegraded":"2024-05-03T18:27:11.783452+0000","last_fullsized":"2024-05-03T18:27:11.783452+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:22:50.269817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.744083+0000","last_change":"2024-05-03T18:22:21.561381+0000","last_active":"2024-05-03T18:27:11.744083+0000","last_peered":"2024-05-03T18:27:11.744083+0000","last_clean":"2024-05-03T18:27:11.744083+0000","last_became_active":"2024-05-03T18:22:21.559411+0000","last_became_peered":"2024-05-03T18:22:21.559411+0000","last_unstale":"2024-05-03T18:27:11.744083+0000","last_undegraded":"2024-05-03T18:27:11.744083+0000","last_fullsized":"2024-05-03T18:27:11.744083+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:04:08.283538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":56,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.839913+0000","last_change":"2024-05-03T18:23:20.551361+0000","last_active":"2024-05-03T18:27:11.839913+0000","last_peered":"2024-05-03T18:27:11.839913+0000","last_clean":"2024-05-03T18:27:11.839913+0000","last_became_active":"2024-05-03T18:23:20.550689+0000","last_became_peered":"2024-05-03T18:23:20.550689+0000","last_unstale":"2024-05-03T18:27:11.839913+0000","last_undegraded":"2024-05-03T18:27:11.839913+0000","last_fullsized":"2024-05-03T18:27:11.839913+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:36:52.767110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.787425+0000","last_change":"2024-05-03T18:22:40.758006+0000","last_active":"2024-05-03T18:27:11.787425+0000","last_peered":"2024-05-03T18:27:11.787425+0000","last_clean":"2024-05-03T18:27:11.787425+0000","last_became_active":"2024-05-03T18:22:40.757075+0000","last_became_peered":"2024-05-03T18:22:40.757075+0000","last_unstale":"2024-05-03T18:27:11.787425+0000","last_undegraded":"2024-05-03T18:27:11.787425+0000","last_fullsized":"2024-05-03T18:27:11.787425+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:06:02.390184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"78'3","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.744164+0000","last_change":"2024-05-03T18:22:21.740041+0000","last_active":"2024-05-03T18:27:11.744164+0000","last_peered":"2024-05-03T18:27:11.744164+0000","last_clean":"2024-05-03T18:27:11.744164+0000","last_became_active":"2024-05-03T18:22:21.738777+0000","last_became_peered":"2024-05-03T18:22:21.738777+0000","last_unstale":"2024-05-03T18:27:11.744164+0000","last_undegraded":"2024-05-03T18:27:11.744164+0000","last_fullsized":"2024-05-03T18:27:11.744164+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:35:30.098325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":53,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.825545+0000","last_change":"2024-05-03T18:22:25.606854+0000","last_active":"2024-05-03T18:27:11.825545+0000","last_peered":"2024-05-03T18:27:11.825545+0000","last_clean":"2024-05-03T18:27:11.825545+0000","last_became_active":"2024-05-03T18:22:25.606376+0000","last_became_peered":"2024-05-03T18:22:25.606376+0000","last_unstale":"2024-05-03T18:27:11.825545+0000","last_undegraded":"2024-05-03T18:27:11.825545+0000","last_fullsized":"2024-05-03T18:27:11.825545+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:29:59.461181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.784158+0000","last_change":"2024-05-03T18:22:40.752863+0000","last_active":"2024-05-03T18:27:11.784158+0000","last_peered":"2024-05-03T18:27:11.784158+0000","last_clean":"2024-05-03T18:27:11.784158+0000","last_became_active":"2024-05-03T18:22:40.752309+0000","last_became_peered":"2024-05-03T18:22:40.752309+0000","last_unstale":"2024-05-03T18:27:11.784158+0000","last_undegraded":"2024-05-03T18:27:11.784158+0000","last_fullsized":"2024-05-03T18:27:11.784158+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:01:06.110570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'37","reported_seq":510,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:12.403811+0000","last_change":"2024-05-03T18:22:21.566162+0000","last_active":"2024-05-03T18:27:12.403811+0000","last_peered":"2024-05-03T18:27:12.403811+0000","last_clean":"2024-05-03T18:27:12.403811+0000","last_became_active":"2024-05-03T18:22:21.565641+0000","last_became_peered":"2024-05-03T18:22:21.565641+0000","last_unstale":"2024-05-03T18:27:12.403811+0000","last_undegraded":"2024-05-03T18:27:12.403811+0000","last_fullsized":"2024-05-03T18:27:12.403811+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:23:45.283135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69239,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":425,"num_read_kb":281,"num_write":37,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.833787+0000","last_change":"2024-05-03T18:22:25.586687+0000","last_active":"2024-05-03T18:27:11.833787+0000","last_peered":"2024-05-03T18:27:11.833787+0000","last_clean":"2024-05-03T18:27:11.833787+0000","last_became_active":"2024-05-03T18:22:25.586275+0000","last_became_peered":"2024-05-03T18:22:25.586275+0000","last_unstale":"2024-05-03T18:27:11.833787+0000","last_undegraded":"2024-05-03T18:27:11.833787+0000","last_fullsized":"2024-05-03T18:27:11.833787+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:10:50.339865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.744602+0000","last_change":"2024-05-03T18:22:40.758003+0000","last_active":"2024-05-03T18:27:11.744602+0000","last_peered":"2024-05-03T18:27:11.744602+0000","last_clean":"2024-05-03T18:27:11.744602+0000","last_became_active":"2024-05-03T18:22:40.757559+0000","last_became_peered":"2024-05-03T18:22:40.757559+0000","last_unstale":"2024-05-03T18:27:11.744602+0000","last_undegraded":"2024-05-03T18:27:11.744602+0000","last_fullsized":"2024-05-03T18:27:11.744602+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:54:23.377748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"78'2","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.767568+0000","last_change":"2024-05-03T18:22:21.564546+0000","last_active":"2024-05-03T18:27:11.767568+0000","last_peered":"2024-05-03T18:27:11.767568+0000","last_clean":"2024-05-03T18:27:11.767568+0000","last_became_active":"2024-05-03T18:22:21.564105+0000","last_became_peered":"2024-05-03T18:22:21.564105+0000","last_unstale":"2024-05-03T18:27:11.767568+0000","last_undegraded":"2024-05-03T18:27:11.767568+0000","last_fullsized":"2024-05-03T18:27:11.767568+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:36:07.790643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.838916+0000","last_change":"2024-05-03T18:22:25.584059+0000","last_active":"2024-05-03T18:27:11.838916+0000","last_peered":"2024-05-03T18:27:11.838916+0000","last_clean":"2024-05-03T18:27:11.838916+0000","last_became_active":"2024-05-03T18:22:25.583742+0000","last_became_peered":"2024-05-03T18:22:25.583742+0000","last_unstale":"2024-05-03T18:27:11.838916+0000","last_undegraded":"2024-05-03T18:27:11.838916+0000","last_fullsized":"2024-05-03T18:27:11.838916+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:15:48.288422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.791333+0000","last_change":"2024-05-03T18:22:40.751362+0000","last_active":"2024-05-03T18:27:11.791333+0000","last_peered":"2024-05-03T18:27:11.791333+0000","last_clean":"2024-05-03T18:27:11.791333+0000","last_became_active":"2024-05-03T18:22:40.750365+0000","last_became_peered":"2024-05-03T18:22:40.750365+0000","last_unstale":"2024-05-03T18:27:11.791333+0000","last_undegraded":"2024-05-03T18:27:11.791333+0000","last_fullsized":"2024-05-03T18:27:11.791333+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:35:50.629332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.742961+0000","last_change":"2024-05-03T18:22:21.556123+0000","last_active":"2024-05-03T18:27:11.742961+0000","last_peered":"2024-05-03T18:27:11.742961+0000","last_clean":"2024-05-03T18:27:11.742961+0000","last_became_active":"2024-05-03T18:22:21.554918+0000","last_became_peered":"2024-05-03T18:22:21.554918+0000","last_unstale":"2024-05-03T18:27:11.742961+0000","last_undegraded":"2024-05-03T18:27:11.742961+0000","last_fullsized":"2024-05-03T18:27:11.742961+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:09:12.383601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":57,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.840168+0000","last_change":"2024-05-03T18:22:25.601849+0000","last_active":"2024-05-03T18:27:11.840168+0000","last_peered":"2024-05-03T18:27:11.840168+0000","last_clean":"2024-05-03T18:27:11.840168+0000","last_became_active":"2024-05-03T18:22:25.601434+0000","last_became_peered":"2024-05-03T18:22:25.601434+0000","last_unstale":"2024-05-03T18:27:11.840168+0000","last_undegraded":"2024-05-03T18:27:11.840168+0000","last_fullsized":"2024-05-03T18:27:11.840168+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:43:18.273389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.780768+0000","last_change":"2024-05-03T18:22:40.754448+0000","last_active":"2024-05-03T18:27:11.780768+0000","last_peered":"2024-05-03T18:27:11.780768+0000","last_clean":"2024-05-03T18:27:11.780768+0000","last_became_active":"2024-05-03T18:22:40.753122+0000","last_became_peered":"2024-05-03T18:22:40.753122+0000","last_unstale":"2024-05-03T18:27:11.780768+0000","last_undegraded":"2024-05-03T18:27:11.780768+0000","last_fullsized":"2024-05-03T18:27:11.780768+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:43:15.216222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.778755+0000","last_change":"2024-05-03T18:22:21.559292+0000","last_active":"2024-05-03T18:27:11.778755+0000","last_peered":"2024-05-03T18:27:11.778755+0000","last_clean":"2024-05-03T18:27:11.778755+0000","last_became_active":"2024-05-03T18:22:21.557657+0000","last_became_peered":"2024-05-03T18:22:21.557657+0000","last_unstale":"2024-05-03T18:27:11.778755+0000","last_undegraded":"2024-05-03T18:27:11.778755+0000","last_fullsized":"2024-05-03T18:27:11.778755+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:51:57.709387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.837734+0000","last_change":"2024-05-03T18:22:25.604340+0000","last_active":"2024-05-03T18:27:11.837734+0000","last_peered":"2024-05-03T18:27:11.837734+0000","last_clean":"2024-05-03T18:27:11.837734+0000","last_became_active":"2024-05-03T18:22:25.603651+0000","last_became_peered":"2024-05-03T18:22:25.603651+0000","last_unstale":"2024-05-03T18:27:11.837734+0000","last_undegraded":"2024-05-03T18:27:11.837734+0000","last_fullsized":"2024-05-03T18:27:11.837734+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:49:36.379556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.740450+0000","last_change":"2024-05-03T18:22:40.738065+0000","last_active":"2024-05-03T18:27:11.740450+0000","last_peered":"2024-05-03T18:27:11.740450+0000","last_clean":"2024-05-03T18:27:11.740450+0000","last_became_active":"2024-05-03T18:22:40.737546+0000","last_became_peered":"2024-05-03T18:22:40.737546+0000","last_unstale":"2024-05-03T18:27:11.740450+0000","last_undegraded":"2024-05-03T18:27:11.740450+0000","last_fullsized":"2024-05-03T18:27:11.740450+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:45:31.528581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"78'8","reported_seq":56,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.778653+0000","last_change":"2024-05-03T18:22:21.739131+0000","last_active":"2024-05-03T18:27:11.778653+0000","last_peered":"2024-05-03T18:27:11.778653+0000","last_clean":"2024-05-03T18:27:11.778653+0000","last_became_active":"2024-05-03T18:22:21.737243+0000","last_became_peered":"2024-05-03T18:22:21.737243+0000","last_unstale":"2024-05-03T18:27:11.778653+0000","last_undegraded":"2024-05-03T18:27:11.778653+0000","last_fullsized":"2024-05-03T18:27:11.778653+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:46:24.011140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.830488+0000","last_change":"2024-05-03T18:22:25.602011+0000","last_active":"2024-05-03T18:27:11.830488+0000","last_peered":"2024-05-03T18:27:11.830488+0000","last_clean":"2024-05-03T18:27:11.830488+0000","last_became_active":"2024-05-03T18:22:25.601604+0000","last_became_peered":"2024-05-03T18:22:25.601604+0000","last_unstale":"2024-05-03T18:27:11.830488+0000","last_undegraded":"2024-05-03T18:27:11.830488+0000","last_fullsized":"2024-05-03T18:27:11.830488+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:25:57.743203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.742365+0000","last_change":"2024-05-03T18:22:40.757885+0000","last_active":"2024-05-03T18:27:11.742365+0000","last_peered":"2024-05-03T18:27:11.742365+0000","last_clean":"2024-05-03T18:27:11.742365+0000","last_became_active":"2024-05-03T18:22:40.757208+0000","last_became_peered":"2024-05-03T18:22:40.757208+0000","last_unstale":"2024-05-03T18:27:11.742365+0000","last_undegraded":"2024-05-03T18:27:11.742365+0000","last_fullsized":"2024-05-03T18:27:11.742365+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:37:00.792138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"78'1","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.791831+0000","last_change":"2024-05-03T18:22:21.554932+0000","last_active":"2024-05-03T18:27:11.791831+0000","last_peered":"2024-05-03T18:27:11.791831+0000","last_clean":"2024-05-03T18:27:11.791831+0000","last_became_active":"2024-05-03T18:22:21.554123+0000","last_became_peered":"2024-05-03T18:22:21.554123+0000","last_unstale":"2024-05-03T18:27:11.791831+0000","last_undegraded":"2024-05-03T18:27:11.791831+0000","last_fullsized":"2024-05-03T18:27:11.791831+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:38:59.859138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.831370+0000","last_change":"2024-05-03T18:22:25.603400+0000","last_active":"2024-05-03T18:27:11.831370+0000","last_peered":"2024-05-03T18:27:11.831370+0000","last_clean":"2024-05-03T18:27:11.831370+0000","last_became_active":"2024-05-03T18:22:25.603045+0000","last_became_peered":"2024-05-03T18:22:25.603045+0000","last_unstale":"2024-05-03T18:27:11.831370+0000","last_undegraded":"2024-05-03T18:27:11.831370+0000","last_fullsized":"2024-05-03T18:27:11.831370+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:51:33.502972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.778451+0000","last_change":"2024-05-03T18:22:40.750857+0000","last_active":"2024-05-03T18:27:11.778451+0000","last_peered":"2024-05-03T18:27:11.778451+0000","last_clean":"2024-05-03T18:27:11.778451+0000","last_became_active":"2024-05-03T18:22:40.750381+0000","last_became_peered":"2024-05-03T18:22:40.750381+0000","last_unstale":"2024-05-03T18:27:11.778451+0000","last_undegraded":"2024-05-03T18:27:11.778451+0000","last_fullsized":"2024-05-03T18:27:11.778451+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:15:28.542612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.781805+0000","last_change":"2024-05-03T18:22:21.739345+0000","last_active":"2024-05-03T18:27:11.781805+0000","last_peered":"2024-05-03T18:27:11.781805+0000","last_clean":"2024-05-03T18:27:11.781805+0000","last_became_active":"2024-05-03T18:22:21.738874+0000","last_became_peered":"2024-05-03T18:22:21.738874+0000","last_unstale":"2024-05-03T18:27:11.781805+0000","last_undegraded":"2024-05-03T18:27:11.781805+0000","last_fullsized":"2024-05-03T18:27:11.781805+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:57:59.706356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"90'4","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.834563+0000","last_change":"2024-05-03T18:22:25.586543+0000","last_active":"2024-05-03T18:27:11.834563+0000","last_peered":"2024-05-03T18:27:11.834563+0000","last_clean":"2024-05-03T18:27:11.834563+0000","last_became_active":"2024-05-03T18:22:25.585124+0000","last_became_peered":"2024-05-03T18:22:25.585124+0000","last_unstale":"2024-05-03T18:27:11.834563+0000","last_undegraded":"2024-05-03T18:27:11.834563+0000","last_fullsized":"2024-05-03T18:27:11.834563+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:41:32.633444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.739379+0000","last_change":"2024-05-03T18:22:40.749665+0000","last_active":"2024-05-03T18:27:11.739379+0000","last_peered":"2024-05-03T18:27:11.739379+0000","last_clean":"2024-05-03T18:27:11.739379+0000","last_became_active":"2024-05-03T18:22:40.749062+0000","last_became_peered":"2024-05-03T18:22:40.749062+0000","last_unstale":"2024-05-03T18:27:11.739379+0000","last_undegraded":"2024-05-03T18:27:11.739379+0000","last_fullsized":"2024-05-03T18:27:11.739379+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:09:17.194507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.742374+0000","last_change":"2024-05-03T18:22:21.560944+0000","last_active":"2024-05-03T18:27:11.742374+0000","last_peered":"2024-05-03T18:27:11.742374+0000","last_clean":"2024-05-03T18:27:11.742374+0000","last_became_active":"2024-05-03T18:22:21.560412+0000","last_became_peered":"2024-05-03T18:22:21.560412+0000","last_unstale":"2024-05-03T18:27:11.742374+0000","last_undegraded":"2024-05-03T18:27:11.742374+0000","last_fullsized":"2024-05-03T18:27:11.742374+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:12:04.069564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.841297+0000","last_change":"2024-05-03T18:22:25.598235+0000","last_active":"2024-05-03T18:27:11.841297+0000","last_peered":"2024-05-03T18:27:11.841297+0000","last_clean":"2024-05-03T18:27:11.841297+0000","last_became_active":"2024-05-03T18:22:25.595993+0000","last_became_peered":"2024-05-03T18:22:25.595993+0000","last_unstale":"2024-05-03T18:27:11.841297+0000","last_undegraded":"2024-05-03T18:27:11.841297+0000","last_fullsized":"2024-05-03T18:27:11.841297+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:53:16.069119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.741050+0000","last_change":"2024-05-03T18:22:40.745005+0000","last_active":"2024-05-03T18:27:11.741050+0000","last_peered":"2024-05-03T18:27:11.741050+0000","last_clean":"2024-05-03T18:27:11.741050+0000","last_became_active":"2024-05-03T18:22:40.742134+0000","last_became_peered":"2024-05-03T18:22:40.742134+0000","last_unstale":"2024-05-03T18:27:11.741050+0000","last_undegraded":"2024-05-03T18:27:11.741050+0000","last_fullsized":"2024-05-03T18:27:11.741050+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:47:12.217012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.829885+0000","last_change":"2024-05-03T18:22:25.604640+0000","last_active":"2024-05-03T18:27:11.829885+0000","last_peered":"2024-05-03T18:27:11.829885+0000","last_clean":"2024-05-03T18:27:11.829885+0000","last_became_active":"2024-05-03T18:22:25.603759+0000","last_became_peered":"2024-05-03T18:22:25.603759+0000","last_unstale":"2024-05-03T18:27:11.829885+0000","last_undegraded":"2024-05-03T18:27:11.829885+0000","last_fullsized":"2024-05-03T18:27:11.829885+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:56:55.744321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.738330+0000","last_change":"2024-05-03T18:22:21.741006+0000","last_active":"2024-05-03T18:27:11.738330+0000","last_peered":"2024-05-03T18:27:11.738330+0000","last_clean":"2024-05-03T18:27:11.738330+0000","last_became_active":"2024-05-03T18:22:21.737999+0000","last_became_peered":"2024-05-03T18:22:21.737999+0000","last_unstale":"2024-05-03T18:27:11.738330+0000","last_undegraded":"2024-05-03T18:27:11.738330+0000","last_fullsized":"2024-05-03T18:27:11.738330+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:29:54.931940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.743698+0000","last_change":"2024-05-03T18:22:40.761044+0000","last_active":"2024-05-03T18:27:11.743698+0000","last_peered":"2024-05-03T18:27:11.743698+0000","last_clean":"2024-05-03T18:27:11.743698+0000","last_became_active":"2024-05-03T18:22:40.760245+0000","last_became_peered":"2024-05-03T18:22:40.760245+0000","last_unstale":"2024-05-03T18:27:11.743698+0000","last_undegraded":"2024-05-03T18:27:11.743698+0000","last_fullsized":"2024-05-03T18:27:11.743698+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:31:12.729078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.832106+0000","last_change":"2024-05-03T18:22:25.595759+0000","last_active":"2024-05-03T18:27:11.832106+0000","last_peered":"2024-05-03T18:27:11.832106+0000","last_clean":"2024-05-03T18:27:11.832106+0000","last_became_active":"2024-05-03T18:22:25.594368+0000","last_became_peered":"2024-05-03T18:22:25.594368+0000","last_unstale":"2024-05-03T18:27:11.832106+0000","last_undegraded":"2024-05-03T18:27:11.832106+0000","last_fullsized":"2024-05-03T18:27:11.832106+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:26:19.196877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.787190+0000","last_change":"2024-05-03T18:22:21.557123+0000","last_active":"2024-05-03T18:27:11.787190+0000","last_peered":"2024-05-03T18:27:11.787190+0000","last_clean":"2024-05-03T18:27:11.787190+0000","last_became_active":"2024-05-03T18:22:21.556432+0000","last_became_peered":"2024-05-03T18:22:21.556432+0000","last_unstale":"2024-05-03T18:27:11.787190+0000","last_undegraded":"2024-05-03T18:27:11.787190+0000","last_fullsized":"2024-05-03T18:27:11.787190+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:50:03.179320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":123,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.741954+0000","last_change":"2024-05-03T18:20:24.937044+0000","last_active":"2024-05-03T18:27:11.741954+0000","last_peered":"2024-05-03T18:27:11.741954+0000","last_clean":"2024-05-03T18:27:11.741954+0000","last_became_active":"2024-05-03T18:20:24.922095+0000","last_became_peered":"2024-05-03T18:20:24.922095+0000","last_unstale":"2024-05-03T18:27:11.741954+0000","last_undegraded":"2024-05-03T18:27:11.741954+0000","last_fullsized":"2024-05-03T18:27:11.741954+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:15:37.353911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:15:37.353911+0000","last_clean_scrub_stamp":"2024-05-03T18:15:37.353911+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:40:07.341441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.780628+0000","last_change":"2024-05-03T18:23:20.575064+0000","last_active":"2024-05-03T18:27:11.780628+0000","last_peered":"2024-05-03T18:27:11.780628+0000","last_clean":"2024-05-03T18:27:11.780628+0000","last_became_active":"2024-05-03T18:23:20.574557+0000","last_became_peered":"2024-05-03T18:23:20.574557+0000","last_unstale":"2024-05-03T18:27:11.780628+0000","last_undegraded":"2024-05-03T18:27:11.780628+0000","last_fullsized":"2024-05-03T18:27:11.780628+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:34:08.520396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,6,10],"acting":[8,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.830147+0000","last_change":"2024-05-03T18:22:25.591269+0000","last_active":"2024-05-03T18:27:11.830147+0000","last_peered":"2024-05-03T18:27:11.830147+0000","last_clean":"2024-05-03T18:27:11.830147+0000","last_became_active":"2024-05-03T18:22:25.590409+0000","last_became_peered":"2024-05-03T18:22:25.590409+0000","last_unstale":"2024-05-03T18:27:11.830147+0000","last_undegraded":"2024-05-03T18:27:11.830147+0000","last_fullsized":"2024-05-03T18:27:11.830147+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:03:57.432331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.742525+0000","last_change":"2024-05-03T18:22:21.560137+0000","last_active":"2024-05-03T18:27:11.742525+0000","last_peered":"2024-05-03T18:27:11.742525+0000","last_clean":"2024-05-03T18:27:11.742525+0000","last_became_active":"2024-05-03T18:22:21.559672+0000","last_became_peered":"2024-05-03T18:22:21.559672+0000","last_unstale":"2024-05-03T18:27:11.742525+0000","last_undegraded":"2024-05-03T18:27:11.742525+0000","last_fullsized":"2024-05-03T18:27:11.742525+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:20:02.070390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.791935+0000","last_change":"2024-05-03T18:22:40.753160+0000","last_active":"2024-05-03T18:27:11.791935+0000","last_peered":"2024-05-03T18:27:11.791935+0000","last_clean":"2024-05-03T18:27:11.791935+0000","last_became_active":"2024-05-03T18:22:40.750800+0000","last_became_peered":"2024-05-03T18:22:40.750800+0000","last_unstale":"2024-05-03T18:27:11.791935+0000","last_undegraded":"2024-05-03T18:27:11.791935+0000","last_fullsized":"2024-05-03T18:27:11.791935+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:27:48.756140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.784686+0000","last_change":"2024-05-03T18:22:21.739490+0000","last_active":"2024-05-03T18:27:11.784686+0000","last_peered":"2024-05-03T18:27:11.784686+0000","last_clean":"2024-05-03T18:27:11.784686+0000","last_became_active":"2024-05-03T18:22:21.738512+0000","last_became_peered":"2024-05-03T18:22:21.738512+0000","last_unstale":"2024-05-03T18:27:11.784686+0000","last_undegraded":"2024-05-03T18:27:11.784686+0000","last_fullsized":"2024-05-03T18:27:11.784686+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:26:45.323519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.837222+0000","last_change":"2024-05-03T18:22:25.591509+0000","last_active":"2024-05-03T18:27:11.837222+0000","last_peered":"2024-05-03T18:27:11.837222+0000","last_clean":"2024-05-03T18:27:11.837222+0000","last_became_active":"2024-05-03T18:22:25.590188+0000","last_became_peered":"2024-05-03T18:22:25.590188+0000","last_unstale":"2024-05-03T18:27:11.837222+0000","last_undegraded":"2024-05-03T18:27:11.837222+0000","last_fullsized":"2024-05-03T18:27:11.837222+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:33:23.919101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.773559+0000","last_change":"2024-05-03T18:22:40.747176+0000","last_active":"2024-05-03T18:27:11.773559+0000","last_peered":"2024-05-03T18:27:11.773559+0000","last_clean":"2024-05-03T18:27:11.773559+0000","last_became_active":"2024-05-03T18:22:40.746526+0000","last_became_peered":"2024-05-03T18:22:40.746526+0000","last_unstale":"2024-05-03T18:27:11.773559+0000","last_undegraded":"2024-05-03T18:27:11.773559+0000","last_fullsized":"2024-05-03T18:27:11.773559+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:59:25.838382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'44","reported_seq":115,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:12.148271+0000","last_change":"2024-05-03T18:22:21.555978+0000","last_active":"2024-05-03T18:27:12.148271+0000","last_peered":"2024-05-03T18:27:12.148271+0000","last_clean":"2024-05-03T18:27:12.148271+0000","last_became_active":"2024-05-03T18:22:21.555395+0000","last_became_peered":"2024-05-03T18:22:21.555395+0000","last_unstale":"2024-05-03T18:27:12.148271+0000","last_undegraded":"2024-05-03T18:27:12.148271+0000","last_fullsized":"2024-05-03T18:27:12.148271+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:05:44.795978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":80,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.838632+0000","last_change":"2024-05-03T18:22:25.589071+0000","last_active":"2024-05-03T18:27:11.838632+0000","last_peered":"2024-05-03T18:27:11.838632+0000","last_clean":"2024-05-03T18:27:11.838632+0000","last_became_active":"2024-05-03T18:22:25.588679+0000","last_became_peered":"2024-05-03T18:22:25.588679+0000","last_unstale":"2024-05-03T18:27:11.838632+0000","last_undegraded":"2024-05-03T18:27:11.838632+0000","last_fullsized":"2024-05-03T18:27:11.838632+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:20:53.983166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.766671+0000","last_change":"2024-05-03T18:22:40.755154+0000","last_active":"2024-05-03T18:27:11.766671+0000","last_peered":"2024-05-03T18:27:11.766671+0000","last_clean":"2024-05-03T18:27:11.766671+0000","last_became_active":"2024-05-03T18:22:40.754635+0000","last_became_peered":"2024-05-03T18:22:40.754635+0000","last_unstale":"2024-05-03T18:27:11.766671+0000","last_undegraded":"2024-05-03T18:27:11.766671+0000","last_fullsized":"2024-05-03T18:27:11.766671+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:25:56.787406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"78'2","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.787950+0000","last_change":"2024-05-03T18:22:21.854322+0000","last_active":"2024-05-03T18:27:11.787950+0000","last_peered":"2024-05-03T18:27:11.787950+0000","last_clean":"2024-05-03T18:27:11.787950+0000","last_became_active":"2024-05-03T18:22:21.853359+0000","last_became_peered":"2024-05-03T18:22:21.853359+0000","last_unstale":"2024-05-03T18:27:11.787950+0000","last_undegraded":"2024-05-03T18:27:11.787950+0000","last_fullsized":"2024-05-03T18:27:11.787950+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:06:57.061775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.835206+0000","last_change":"2024-05-03T18:22:25.605614+0000","last_active":"2024-05-03T18:27:11.835206+0000","last_peered":"2024-05-03T18:27:11.835206+0000","last_clean":"2024-05-03T18:27:11.835206+0000","last_became_active":"2024-05-03T18:22:25.604277+0000","last_became_peered":"2024-05-03T18:22:25.604277+0000","last_unstale":"2024-05-03T18:27:11.835206+0000","last_undegraded":"2024-05-03T18:27:11.835206+0000","last_fullsized":"2024-05-03T18:27:11.835206+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:17:09.812304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.790222+0000","last_change":"2024-05-03T18:22:40.763353+0000","last_active":"2024-05-03T18:27:11.790222+0000","last_peered":"2024-05-03T18:27:11.790222+0000","last_clean":"2024-05-03T18:27:11.790222+0000","last_became_active":"2024-05-03T18:22:40.762774+0000","last_became_peered":"2024-05-03T18:22:40.762774+0000","last_unstale":"2024-05-03T18:27:11.790222+0000","last_undegraded":"2024-05-03T18:27:11.790222+0000","last_fullsized":"2024-05-03T18:27:11.790222+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:38:37.247050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.766499+0000","last_change":"2024-05-03T18:24:21.803989+0000","last_active":"2024-05-03T18:27:11.766499+0000","last_peered":"2024-05-03T18:27:11.766499+0000","last_clean":"2024-05-03T18:27:11.766499+0000","last_became_active":"2024-05-03T18:24:21.803072+0000","last_became_peered":"2024-05-03T18:24:21.803072+0000","last_unstale":"2024-05-03T18:27:11.766499+0000","last_undegraded":"2024-05-03T18:27:11.766499+0000","last_fullsized":"2024-05-03T18:27:11.766499+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:32:30.562148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.815005+0000","last_change":"2024-05-03T18:22:25.588011+0000","last_active":"2024-05-03T18:27:11.815005+0000","last_peered":"2024-05-03T18:27:11.815005+0000","last_clean":"2024-05-03T18:27:11.815005+0000","last_became_active":"2024-05-03T18:22:25.587602+0000","last_became_peered":"2024-05-03T18:22:25.587602+0000","last_unstale":"2024-05-03T18:27:11.815005+0000","last_undegraded":"2024-05-03T18:27:11.815005+0000","last_fullsized":"2024-05-03T18:27:11.815005+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:18:13.186088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.780962+0000","last_change":"2024-05-03T18:22:40.755731+0000","last_active":"2024-05-03T18:27:11.780962+0000","last_peered":"2024-05-03T18:27:11.780962+0000","last_clean":"2024-05-03T18:27:11.780962+0000","last_became_active":"2024-05-03T18:22:40.755173+0000","last_became_peered":"2024-05-03T18:22:40.755173+0000","last_unstale":"2024-05-03T18:27:11.780962+0000","last_undegraded":"2024-05-03T18:27:11.780962+0000","last_fullsized":"2024-05-03T18:27:11.780962+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:36:36.548095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.774547+0000","last_change":"2024-05-03T18:22:21.561481+0000","last_active":"2024-05-03T18:27:11.774547+0000","last_peered":"2024-05-03T18:27:11.774547+0000","last_clean":"2024-05-03T18:27:11.774547+0000","last_became_active":"2024-05-03T18:22:21.560088+0000","last_became_peered":"2024-05-03T18:22:21.560088+0000","last_unstale":"2024-05-03T18:27:11.774547+0000","last_undegraded":"2024-05-03T18:27:11.774547+0000","last_fullsized":"2024-05-03T18:27:11.774547+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:45:40.184813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.839958+0000","last_change":"2024-05-03T18:22:25.584556+0000","last_active":"2024-05-03T18:27:11.839958+0000","last_peered":"2024-05-03T18:27:11.839958+0000","last_clean":"2024-05-03T18:27:11.839958+0000","last_became_active":"2024-05-03T18:22:25.583873+0000","last_became_peered":"2024-05-03T18:22:25.583873+0000","last_unstale":"2024-05-03T18:27:11.839958+0000","last_undegraded":"2024-05-03T18:27:11.839958+0000","last_fullsized":"2024-05-03T18:27:11.839958+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:28:39.028662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.803445+0000","last_change":"2024-05-03T18:22:40.763479+0000","last_active":"2024-05-03T18:27:11.803445+0000","last_peered":"2024-05-03T18:27:11.803445+0000","last_clean":"2024-05-03T18:27:11.803445+0000","last_became_active":"2024-05-03T18:22:40.762897+0000","last_became_peered":"2024-05-03T18:22:40.762897+0000","last_unstale":"2024-05-03T18:27:11.803445+0000","last_undegraded":"2024-05-03T18:27:11.803445+0000","last_fullsized":"2024-05-03T18:27:11.803445+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:00:21.090730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.767323+0000","last_change":"2024-05-03T18:22:21.562929+0000","last_active":"2024-05-03T18:27:11.767323+0000","last_peered":"2024-05-03T18:27:11.767323+0000","last_clean":"2024-05-03T18:27:11.767323+0000","last_became_active":"2024-05-03T18:22:21.560712+0000","last_became_peered":"2024-05-03T18:22:21.560712+0000","last_unstale":"2024-05-03T18:27:11.767323+0000","last_undegraded":"2024-05-03T18:27:11.767323+0000","last_fullsized":"2024-05-03T18:27:11.767323+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:31:34.722437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.840750+0000","last_change":"2024-05-03T18:22:25.581971+0000","last_active":"2024-05-03T18:27:11.840750+0000","last_peered":"2024-05-03T18:27:11.840750+0000","last_clean":"2024-05-03T18:27:11.840750+0000","last_became_active":"2024-05-03T18:22:25.581565+0000","last_became_peered":"2024-05-03T18:22:25.581565+0000","last_unstale":"2024-05-03T18:27:11.840750+0000","last_undegraded":"2024-05-03T18:27:11.840750+0000","last_fullsized":"2024-05-03T18:27:11.840750+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:10:20.607999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.787659+0000","last_change":"2024-05-03T18:22:40.761458+0000","last_active":"2024-05-03T18:27:11.787659+0000","last_peered":"2024-05-03T18:27:11.787659+0000","last_clean":"2024-05-03T18:27:11.787659+0000","last_became_active":"2024-05-03T18:22:40.760969+0000","last_became_peered":"2024-05-03T18:22:40.760969+0000","last_unstale":"2024-05-03T18:27:11.787659+0000","last_undegraded":"2024-05-03T18:27:11.787659+0000","last_fullsized":"2024-05-03T18:27:11.787659+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:19:16.804720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"78'2","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.780499+0000","last_change":"2024-05-03T18:22:21.565869+0000","last_active":"2024-05-03T18:27:11.780499+0000","last_peered":"2024-05-03T18:27:11.780499+0000","last_clean":"2024-05-03T18:27:11.780499+0000","last_became_active":"2024-05-03T18:22:21.564119+0000","last_became_peered":"2024-05-03T18:22:21.564119+0000","last_unstale":"2024-05-03T18:27:11.780499+0000","last_undegraded":"2024-05-03T18:27:11.780499+0000","last_fullsized":"2024-05-03T18:27:11.780499+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:50:29.894070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.839023+0000","last_change":"2024-05-03T18:22:25.596199+0000","last_active":"2024-05-03T18:27:11.839023+0000","last_peered":"2024-05-03T18:27:11.839023+0000","last_clean":"2024-05-03T18:27:11.839023+0000","last_became_active":"2024-05-03T18:22:25.595167+0000","last_became_peered":"2024-05-03T18:22:25.595167+0000","last_unstale":"2024-05-03T18:27:11.839023+0000","last_undegraded":"2024-05-03T18:27:11.839023+0000","last_fullsized":"2024-05-03T18:27:11.839023+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:28:56.599064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.766667+0000","last_change":"2024-05-03T18:22:40.753020+0000","last_active":"2024-05-03T18:27:11.766667+0000","last_peered":"2024-05-03T18:27:11.766667+0000","last_clean":"2024-05-03T18:27:11.766667+0000","last_became_active":"2024-05-03T18:22:40.752502+0000","last_became_peered":"2024-05-03T18:22:40.752502+0000","last_unstale":"2024-05-03T18:27:11.766667+0000","last_undegraded":"2024-05-03T18:27:11.766667+0000","last_fullsized":"2024-05-03T18:27:11.766667+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:33:51.277879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.768258+0000","last_change":"2024-05-03T18:22:21.854481+0000","last_active":"2024-05-03T18:27:11.768258+0000","last_peered":"2024-05-03T18:27:11.768258+0000","last_clean":"2024-05-03T18:27:11.768258+0000","last_became_active":"2024-05-03T18:22:21.853172+0000","last_became_peered":"2024-05-03T18:22:21.853172+0000","last_unstale":"2024-05-03T18:27:11.768258+0000","last_undegraded":"2024-05-03T18:27:11.768258+0000","last_fullsized":"2024-05-03T18:27:11.768258+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:11:47.820105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.837912+0000","last_change":"2024-05-03T18:22:25.600856+0000","last_active":"2024-05-03T18:27:11.837912+0000","last_peered":"2024-05-03T18:27:11.837912+0000","last_clean":"2024-05-03T18:27:11.837912+0000","last_became_active":"2024-05-03T18:22:25.595351+0000","last_became_peered":"2024-05-03T18:22:25.595351+0000","last_unstale":"2024-05-03T18:27:11.837912+0000","last_undegraded":"2024-05-03T18:27:11.837912+0000","last_fullsized":"2024-05-03T18:27:11.837912+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:29:24.232506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.773733+0000","last_change":"2024-05-03T18:23:20.578331+0000","last_active":"2024-05-03T18:27:11.773733+0000","last_peered":"2024-05-03T18:27:11.773733+0000","last_clean":"2024-05-03T18:27:11.773733+0000","last_became_active":"2024-05-03T18:23:20.576974+0000","last_became_peered":"2024-05-03T18:23:20.576974+0000","last_unstale":"2024-05-03T18:27:11.773733+0000","last_undegraded":"2024-05-03T18:27:11.773733+0000","last_fullsized":"2024-05-03T18:27:11.773733+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:57:32.922175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"90'39","reported_seq":95,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:12.135624+0000","last_change":"2024-05-03T18:22:21.552331+0000","last_active":"2024-05-03T18:27:12.135624+0000","last_peered":"2024-05-03T18:27:12.135624+0000","last_clean":"2024-05-03T18:27:12.135624+0000","last_became_active":"2024-05-03T18:22:21.549786+0000","last_became_peered":"2024-05-03T18:22:21.549786+0000","last_unstale":"2024-05-03T18:27:12.135624+0000","last_undegraded":"2024-05-03T18:27:12.135624+0000","last_fullsized":"2024-05-03T18:27:12.135624+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:08:18.513405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"90'6","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.837679+0000","last_change":"2024-05-03T18:22:25.596437+0000","last_active":"2024-05-03T18:27:11.837679+0000","last_peered":"2024-05-03T18:27:11.837679+0000","last_clean":"2024-05-03T18:27:11.837679+0000","last_became_active":"2024-05-03T18:22:25.594040+0000","last_became_peered":"2024-05-03T18:22:25.594040+0000","last_unstale":"2024-05-03T18:27:11.837679+0000","last_undegraded":"2024-05-03T18:27:11.837679+0000","last_fullsized":"2024-05-03T18:27:11.837679+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:09:36.473565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.741898+0000","last_change":"2024-05-03T18:22:40.754641+0000","last_active":"2024-05-03T18:27:11.741898+0000","last_peered":"2024-05-03T18:27:11.741898+0000","last_clean":"2024-05-03T18:27:11.741898+0000","last_became_active":"2024-05-03T18:22:40.754166+0000","last_became_peered":"2024-05-03T18:22:40.754166+0000","last_unstale":"2024-05-03T18:27:11.741898+0000","last_undegraded":"2024-05-03T18:27:11.741898+0000","last_fullsized":"2024-05-03T18:27:11.741898+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:29:50.021671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.741202+0000","last_change":"2024-05-03T18:22:21.559354+0000","last_active":"2024-05-03T18:27:11.741202+0000","last_peered":"2024-05-03T18:27:11.741202+0000","last_clean":"2024-05-03T18:27:11.741202+0000","last_became_active":"2024-05-03T18:22:21.558680+0000","last_became_peered":"2024-05-03T18:22:21.558680+0000","last_unstale":"2024-05-03T18:27:11.741202+0000","last_undegraded":"2024-05-03T18:27:11.741202+0000","last_fullsized":"2024-05-03T18:27:11.741202+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:13:04.001847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.837002+0000","last_change":"2024-05-03T18:22:25.590746+0000","last_active":"2024-05-03T18:27:11.837002+0000","last_peered":"2024-05-03T18:27:11.837002+0000","last_clean":"2024-05-03T18:27:11.837002+0000","last_became_active":"2024-05-03T18:22:25.587906+0000","last_became_peered":"2024-05-03T18:22:25.587906+0000","last_unstale":"2024-05-03T18:27:11.837002+0000","last_undegraded":"2024-05-03T18:27:11.837002+0000","last_fullsized":"2024-05-03T18:27:11.837002+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:19:43.410259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.833036+0000","last_change":"2024-05-03T18:22:25.597628+0000","last_active":"2024-05-03T18:27:11.833036+0000","last_peered":"2024-05-03T18:27:11.833036+0000","last_clean":"2024-05-03T18:27:11.833036+0000","last_became_active":"2024-05-03T18:22:25.596535+0000","last_became_peered":"2024-05-03T18:22:25.596535+0000","last_unstale":"2024-05-03T18:27:11.833036+0000","last_undegraded":"2024-05-03T18:27:11.833036+0000","last_fullsized":"2024-05-03T18:27:11.833036+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:54:52.632258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.829560+0000","last_change":"2024-05-03T18:22:25.601474+0000","last_active":"2024-05-03T18:27:11.829560+0000","last_peered":"2024-05-03T18:27:11.829560+0000","last_clean":"2024-05-03T18:27:11.829560+0000","last_became_active":"2024-05-03T18:22:25.601097+0000","last_became_peered":"2024-05-03T18:22:25.601097+0000","last_unstale":"2024-05-03T18:27:11.829560+0000","last_undegraded":"2024-05-03T18:27:11.829560+0000","last_fullsized":"2024-05-03T18:27:11.829560+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:14:46.001500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":56,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.820221+0000","last_change":"2024-05-03T18:23:20.535409+0000","last_active":"2024-05-03T18:27:11.820221+0000","last_peered":"2024-05-03T18:27:11.820221+0000","last_clean":"2024-05-03T18:27:11.820221+0000","last_became_active":"2024-05-03T18:23:20.534883+0000","last_became_peered":"2024-05-03T18:23:20.534883+0000","last_unstale":"2024-05-03T18:27:11.820221+0000","last_undegraded":"2024-05-03T18:27:11.820221+0000","last_fullsized":"2024-05-03T18:27:11.820221+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:26:37.252898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.835637+0000","last_change":"2024-05-03T18:22:25.607115+0000","last_active":"2024-05-03T18:27:11.835637+0000","last_peered":"2024-05-03T18:27:11.835637+0000","last_clean":"2024-05-03T18:27:11.835637+0000","last_became_active":"2024-05-03T18:22:25.606764+0000","last_became_peered":"2024-05-03T18:22:25.606764+0000","last_unstale":"2024-05-03T18:27:11.835637+0000","last_undegraded":"2024-05-03T18:27:11.835637+0000","last_fullsized":"2024-05-03T18:27:11.835637+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:44:37.165032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.842362+0000","last_change":"2024-05-03T18:22:25.601886+0000","last_active":"2024-05-03T18:27:11.842362+0000","last_peered":"2024-05-03T18:27:11.842362+0000","last_clean":"2024-05-03T18:27:11.842362+0000","last_became_active":"2024-05-03T18:22:25.601494+0000","last_became_peered":"2024-05-03T18:22:25.601494+0000","last_unstale":"2024-05-03T18:27:11.842362+0000","last_undegraded":"2024-05-03T18:27:11.842362+0000","last_fullsized":"2024-05-03T18:27:11.842362+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:35:30.080124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.832632+0000","last_change":"2024-05-03T18:23:20.529091+0000","last_active":"2024-05-03T18:27:11.832632+0000","last_peered":"2024-05-03T18:27:11.832632+0000","last_clean":"2024-05-03T18:27:11.832632+0000","last_became_active":"2024-05-03T18:23:20.528288+0000","last_became_peered":"2024-05-03T18:23:20.528288+0000","last_unstale":"2024-05-03T18:27:11.832632+0000","last_undegraded":"2024-05-03T18:27:11.832632+0000","last_fullsized":"2024-05-03T18:27:11.832632+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:32:25.749461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":57,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.823878+0000","last_change":"2024-05-03T18:22:25.595367+0000","last_active":"2024-05-03T18:27:11.823878+0000","last_peered":"2024-05-03T18:27:11.823878+0000","last_clean":"2024-05-03T18:27:11.823878+0000","last_became_active":"2024-05-03T18:22:25.594987+0000","last_became_peered":"2024-05-03T18:22:25.594987+0000","last_unstale":"2024-05-03T18:27:11.823878+0000","last_undegraded":"2024-05-03T18:27:11.823878+0000","last_fullsized":"2024-05-03T18:27:11.823878+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:40:55.385595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.823841+0000","last_change":"2024-05-03T18:22:25.588320+0000","last_active":"2024-05-03T18:27:11.823841+0000","last_peered":"2024-05-03T18:27:11.823841+0000","last_clean":"2024-05-03T18:27:11.823841+0000","last_became_active":"2024-05-03T18:22:25.587861+0000","last_became_peered":"2024-05-03T18:22:25.587861+0000","last_unstale":"2024-05-03T18:27:11.823841+0000","last_undegraded":"2024-05-03T18:27:11.823841+0000","last_fullsized":"2024-05-03T18:27:11.823841+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:17:02.538798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.831427+0000","last_change":"2024-05-03T18:22:25.597874+0000","last_active":"2024-05-03T18:27:11.831427+0000","last_peered":"2024-05-03T18:27:11.831427+0000","last_clean":"2024-05-03T18:27:11.831427+0000","last_became_active":"2024-05-03T18:22:25.597448+0000","last_became_peered":"2024-05-03T18:22:25.597448+0000","last_unstale":"2024-05-03T18:27:11.831427+0000","last_undegraded":"2024-05-03T18:27:11.831427+0000","last_fullsized":"2024-05-03T18:27:11.831427+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:55:52.894491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.836134+0000","last_change":"2024-05-03T18:22:25.593509+0000","last_active":"2024-05-03T18:27:11.836134+0000","last_peered":"2024-05-03T18:27:11.836134+0000","last_clean":"2024-05-03T18:27:11.836134+0000","last_became_active":"2024-05-03T18:22:25.590736+0000","last_became_peered":"2024-05-03T18:22:25.590736+0000","last_unstale":"2024-05-03T18:27:11.836134+0000","last_undegraded":"2024-05-03T18:27:11.836134+0000","last_fullsized":"2024-05-03T18:27:11.836134+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:54:45.431776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.837344+0000","last_change":"2024-05-03T18:22:25.590022+0000","last_active":"2024-05-03T18:27:11.837344+0000","last_peered":"2024-05-03T18:27:11.837344+0000","last_clean":"2024-05-03T18:27:11.837344+0000","last_became_active":"2024-05-03T18:22:25.589432+0000","last_became_peered":"2024-05-03T18:22:25.589432+0000","last_unstale":"2024-05-03T18:27:11.837344+0000","last_undegraded":"2024-05-03T18:27:11.837344+0000","last_fullsized":"2024-05-03T18:27:11.837344+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:03:02.091228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":53,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.841863+0000","last_change":"2024-05-03T18:22:25.581452+0000","last_active":"2024-05-03T18:27:11.841863+0000","last_peered":"2024-05-03T18:27:11.841863+0000","last_clean":"2024-05-03T18:27:11.841863+0000","last_became_active":"2024-05-03T18:22:25.581098+0000","last_became_peered":"2024-05-03T18:22:25.581098+0000","last_unstale":"2024-05-03T18:27:11.841863+0000","last_undegraded":"2024-05-03T18:27:11.841863+0000","last_fullsized":"2024-05-03T18:27:11.841863+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:04:46.551234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.823559+0000","last_change":"2024-05-03T18:22:25.589105+0000","last_active":"2024-05-03T18:27:11.823559+0000","last_peered":"2024-05-03T18:27:11.823559+0000","last_clean":"2024-05-03T18:27:11.823559+0000","last_became_active":"2024-05-03T18:22:25.588697+0000","last_became_peered":"2024-05-03T18:22:25.588697+0000","last_unstale":"2024-05-03T18:27:11.823559+0000","last_undegraded":"2024-05-03T18:27:11.823559+0000","last_fullsized":"2024-05-03T18:27:11.823559+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:41:23.563296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":594853,"num_objects":24,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":523,"num_read_kb":366,"num_write":203,"num_write_kb":1410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1880064,"data_stored":1784559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":151,"ondisk_log_size":151,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710797,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1036288,"data_stored":524266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":10,"up_from":64,"seq":274877907027,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28884,"kb_used_data":1808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716268,"statfs":{"total":95995035648,"available":95965458432,"internally_reserved":0,"allocated":1851392,"data_stored":1311571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070554,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36744,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708408,"statfs":{"total":95995035648,"available":95957409792,"internally_reserved":0,"allocated":1437696,"data_stored":922160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266785,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1044480,"data_stored":524870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":47,"seq":201863463018,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32780,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712372,"statfs":{"total":95995035648,"available":95961468928,"internally_reserved":0,"allocated":1638400,"data_stored":1119312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626545,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1032192,"data_stored":520170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574996,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32264,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712888,"statfs":{"total":95995035648,"available":95961997312,"internally_reserved":0,"allocated":1110016,"data_stored":588627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738523,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1036288,"data_stored":520260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411470,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32268,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712884,"statfs":{"total":95995035648,"available":95961993216,"internally_reserved":0,"allocated":1114112,"data_stored":588661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182532,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28548,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716604,"statfs":{"total":95995035648,"available":95965802496,"internally_reserved":0,"allocated":1568768,"data_stored":1045090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986302,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1040384,"data_stored":524288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822775,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32912,"kb_used_data":1736,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712240,"statfs":{"total":95995035648,"available":95961333760,"internally_reserved":0,"allocated":1777664,"data_stored":1246416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":69239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":69273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":536576,"data_stored":524920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":131782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":393820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":397312,"data_stored":393798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":225280,"data_stored":201033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-03T18:27:17.056 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-03T18:27:17.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.283+0000 7fbe86800640 1 -- 172.21.15.33:0/1165943311 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe80101870 msgr2=0x7fbe80101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:17.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.283+0000 7fbe86800640 1 --2- 172.21.15.33:0/1165943311 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe80101870 0x7fbe80101cf0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fbe6c009900 tx=0x7fbe6c02a3f0 comp rx=0 tx=0).stop 2024-05-03T18:27:17.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.283+0000 7fbe86800640 1 -- 172.21.15.33:0/1165943311 shutdown_connections 2024-05-03T18:27:17.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.283+0000 7fbe86800640 1 --2- 172.21.15.33:0/1165943311 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe80102230 0x7fbe80102ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:17.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.283+0000 7fbe86800640 1 --2- 172.21.15.33:0/1165943311 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe80101870 0x7fbe80101cf0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:17.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.283+0000 7fbe86800640 1 --2- 172.21.15.33:0/1165943311 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe80100b10 0x7fbe80100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:17.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.283+0000 7fbe86800640 1 -- 172.21.15.33:0/1165943311 >> 172.21.15.33:0/1165943311 conn(0x7fbe800fc460 msgr2=0x7fbe800fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:27:17.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.283+0000 7fbe86800640 1 -- 172.21.15.33:0/1165943311 shutdown_connections 2024-05-03T18:27:17.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.284+0000 7fbe86800640 1 -- 172.21.15.33:0/1165943311 wait complete. 2024-05-03T18:27:17.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.285+0000 7fbe86800640 1 Processor -- start 2024-05-03T18:27:17.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.285+0000 7fbe86800640 1 -- start start 2024-05-03T18:27:17.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.285+0000 7fbe86800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe80100b10 0x7fbe8019ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:17.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.285+0000 7fbe86800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe80101870 0x7fbe8019b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:17.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.285+0000 7fbe86800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe80102230 0x7fbe801a21b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:17.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.285+0000 7fbe86800640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbe80110430 con 0x7fbe80102230 2024-05-03T18:27:17.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.285+0000 7fbe86800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbe801102b0 con 0x7fbe80100b10 2024-05-03T18:27:17.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.285+0000 7fbe86800640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fbe801105b0 con 0x7fbe80101870 2024-05-03T18:27:17.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.285+0000 7fbe85e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe80102230 0x7fbe801a21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:17.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.285+0000 7fbe85e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe80102230 0x7fbe801a21b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50762/0 (socket says 172.21.15.33:50762) 2024-05-03T18:27:17.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.285+0000 7fbe85e00640 1 -- 172.21.15.33:0/3989252137 learned_addr learned my addr 172.21.15.33:0/3989252137 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:27:17.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.286+0000 7fbe85400640 1 --2- 172.21.15.33:0/3989252137 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe80100b10 0x7fbe8019ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:17.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.286+0000 7fbe84a00640 1 --2- 172.21.15.33:0/3989252137 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe80101870 0x7fbe8019b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:17.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.286+0000 7fbe85e00640 1 -- 172.21.15.33:0/3989252137 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe80101870 msgr2=0x7fbe8019b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:17.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.286+0000 7fbe85e00640 1 --2- 172.21.15.33:0/3989252137 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe80101870 0x7fbe8019b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:17.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.286+0000 7fbe85e00640 1 -- 172.21.15.33:0/3989252137 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe80100b10 msgr2=0x7fbe8019ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:17.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.286+0000 7fbe85e00640 1 --2- 172.21.15.33:0/3989252137 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe80100b10 0x7fbe8019ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:17.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.286+0000 7fbe85e00640 1 -- 172.21.15.33:0/3989252137 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe801a2850 con 0x7fbe80102230 2024-05-03T18:27:17.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.286+0000 7fbe85e00640 1 --2- 172.21.15.33:0/3989252137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe80102230 0x7fbe801a21b0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fbe7800eb10 tx=0x7fbe7800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:27:17.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.286+0000 7fbe7e000640 1 -- 172.21.15.33:0/3989252137 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe78004b20 con 0x7fbe80102230 2024-05-03T18:27:17.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.287+0000 7fbe7e000640 1 -- 172.21.15.33:0/3989252137 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbe78004cc0 con 0x7fbe80102230 2024-05-03T18:27:17.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.287+0000 7fbe86800640 1 -- 172.21.15.33:0/3989252137 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe801a2b40 con 0x7fbe80102230 2024-05-03T18:27:17.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.287+0000 7fbe7e000640 1 -- 172.21.15.33:0/3989252137 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe78010690 con 0x7fbe80102230 2024-05-03T18:27:17.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.287+0000 7fbe86800640 1 -- 172.21.15.33:0/3989252137 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe801a3040 con 0x7fbe80102230 2024-05-03T18:27:17.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.288+0000 7fbe86800640 1 -- 172.21.15.33:0/3989252137 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe8010c940 con 0x7fbe80102230 2024-05-03T18:27:17.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.291+0000 7fbe7e000640 1 -- 172.21.15.33:0/3989252137 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 23) ==== 149828+0+0 (secure 0 0 0) 0x7fbe78005000 con 0x7fbe80102230 2024-05-03T18:27:17.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.292+0000 7fbe7e000640 1 --2- 172.21.15.33:0/3989252137 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fbe680b29f0 0x7fbe680b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:17.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.293+0000 7fbe7e000640 1 -- 172.21.15.33:0/3989252137 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7fbe780e3430 con 0x7fbe80102230 2024-05-03T18:27:17.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.293+0000 7fbe85400640 1 --2- 172.21.15.33:0/3989252137 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fbe680b29f0 0x7fbe680b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:17.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.293+0000 7fbe85400640 1 --2- 172.21.15.33:0/3989252137 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fbe680b29f0 0x7fbe680b4eb0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fbe8019c110 tx=0x7fbe74005f90 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:27:17.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.298+0000 7fbe7e000640 1 -- 172.21.15.33:0/3989252137 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fbe78054d70 con 0x7fbe80102230 2024-05-03T18:27:17.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.528+0000 7fbe86800640 1 -- 172.21.15.33:0/3989252137 --> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbe80062df0 con 0x7fbe680b29f0 2024-05-03T18:27:17.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.539+0000 7fbe7e000640 1 -- 172.21.15.33:0/3989252137 <== mgr.15120 v2:172.21.15.33:6800/3302011241 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339452 (secure 0 0 0) 0x7fbe80062df0 con 0x7fbe680b29f0 2024-05-03T18:27:17.543 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:27:17.545 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-05-03T18:27:17.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.545+0000 7fbe86800640 1 -- 172.21.15.33:0/3989252137 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fbe680b29f0 msgr2=0x7fbe680b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:17.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.546+0000 7fbe86800640 1 --2- 172.21.15.33:0/3989252137 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fbe680b29f0 0x7fbe680b4eb0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fbe8019c110 tx=0x7fbe74005f90 comp rx=0 tx=0).stop 2024-05-03T18:27:17.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.546+0000 7fbe86800640 1 -- 172.21.15.33:0/3989252137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe80102230 msgr2=0x7fbe801a21b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:17.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.546+0000 7fbe86800640 1 --2- 172.21.15.33:0/3989252137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe80102230 0x7fbe801a21b0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fbe7800eb10 tx=0x7fbe7800c2d0 comp rx=0 tx=0).stop 2024-05-03T18:27:17.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.546+0000 7fbe86800640 1 -- 172.21.15.33:0/3989252137 shutdown_connections 2024-05-03T18:27:17.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.546+0000 7fbe86800640 1 --2- 172.21.15.33:0/3989252137 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fbe680b29f0 0x7fbe680b4eb0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:17.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.546+0000 7fbe86800640 1 --2- 172.21.15.33:0/3989252137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe80102230 0x7fbe801a21b0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:17.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.546+0000 7fbe86800640 1 --2- 172.21.15.33:0/3989252137 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe80101870 0x7fbe8019b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:17.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.546+0000 7fbe86800640 1 --2- 172.21.15.33:0/3989252137 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe80100b10 0x7fbe8019ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:17.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.546+0000 7fbe86800640 1 -- 172.21.15.33:0/3989252137 >> 172.21.15.33:0/3989252137 conn(0x7fbe800fc460 msgr2=0x7fbe800fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:27:17.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.546+0000 7fbe86800640 1 -- 172.21.15.33:0/3989252137 shutdown_connections 2024-05-03T18:27:17.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.546+0000 7fbe86800640 1 -- 172.21.15.33:0/3989252137 wait complete. 2024-05-03T18:27:17.562 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:17 smithi033 ceph-mon[28330]: mgrmap e23: x(active, since 4s), standbys: z, y 2024-05-03T18:27:17.564 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:17 smithi150 ceph-mon[32492]: mgrmap e23: x(active, since 4s), standbys: z, y 2024-05-03T18:27:17.565 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-05-03T18:27:15.766443+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1185455,"num_objects":28,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":606,"num_read_kb":435,"num_write":328,"num_write_kb":3562,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":745,"ondisk_log_size":745,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":368728,"kb_used_data":15320,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124573096,"statfs":{"total":1151940427776,"available":1151562850304,"internally_reserved":0,"allocated":15687680,"data_stored":9435691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":361674042},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":57,"apply_latency_ms":57,"commit_latency_ns":57000000,"apply_latency_ns":57000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.980393"},"pg_stats":[{"pgid":"3.3f","version":"90'8","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.845032+0000","last_change":"2024-05-03T18:22:25.595640+0000","last_active":"2024-05-03T18:27:11.845032+0000","last_peered":"2024-05-03T18:27:11.845032+0000","last_clean":"2024-05-03T18:27:11.845032+0000","last_became_active":"2024-05-03T18:22:25.595284+0000","last_became_peered":"2024-05-03T18:22:25.595284+0000","last_unstale":"2024-05-03T18:27:11.845032+0000","last_undegraded":"2024-05-03T18:27:11.845032+0000","last_fullsized":"2024-05-03T18:27:11.845032+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:50:08.867670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.835095+0000","last_change":"2024-05-03T18:22:25.584950+0000","last_active":"2024-05-03T18:27:11.835095+0000","last_peered":"2024-05-03T18:27:11.835095+0000","last_clean":"2024-05-03T18:27:11.835095+0000","last_became_active":"2024-05-03T18:22:25.584613+0000","last_became_peered":"2024-05-03T18:22:25.584613+0000","last_unstale":"2024-05-03T18:27:11.835095+0000","last_undegraded":"2024-05-03T18:27:11.835095+0000","last_fullsized":"2024-05-03T18:27:11.835095+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:00:24.959429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.834333+0000","last_change":"2024-05-03T18:22:25.596799+0000","last_active":"2024-05-03T18:27:11.834333+0000","last_peered":"2024-05-03T18:27:11.834333+0000","last_clean":"2024-05-03T18:27:11.834333+0000","last_became_active":"2024-05-03T18:22:25.596386+0000","last_became_peered":"2024-05-03T18:22:25.596386+0000","last_unstale":"2024-05-03T18:27:11.834333+0000","last_undegraded":"2024-05-03T18:27:11.834333+0000","last_fullsized":"2024-05-03T18:27:11.834333+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:20:07.818282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.830961+0000","last_change":"2024-05-03T18:23:20.534672+0000","last_active":"2024-05-03T18:27:11.830961+0000","last_peered":"2024-05-03T18:27:11.830961+0000","last_clean":"2024-05-03T18:27:11.830961+0000","last_became_active":"2024-05-03T18:23:20.534073+0000","last_became_peered":"2024-05-03T18:23:20.534073+0000","last_unstale":"2024-05-03T18:27:11.830961+0000","last_undegraded":"2024-05-03T18:27:11.830961+0000","last_fullsized":"2024-05-03T18:27:11.830961+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:07:40.204299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.827187+0000","last_change":"2024-05-03T18:22:25.589810+0000","last_active":"2024-05-03T18:27:11.827187+0000","last_peered":"2024-05-03T18:27:11.827187+0000","last_clean":"2024-05-03T18:27:11.827187+0000","last_became_active":"2024-05-03T18:22:25.588220+0000","last_became_peered":"2024-05-03T18:22:25.588220+0000","last_unstale":"2024-05-03T18:27:11.827187+0000","last_undegraded":"2024-05-03T18:27:11.827187+0000","last_fullsized":"2024-05-03T18:27:11.827187+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:43:20.904737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.834092+0000","last_change":"2024-05-03T18:22:25.603148+0000","last_active":"2024-05-03T18:27:11.834092+0000","last_peered":"2024-05-03T18:27:11.834092+0000","last_clean":"2024-05-03T18:27:11.834092+0000","last_became_active":"2024-05-03T18:22:25.602743+0000","last_became_peered":"2024-05-03T18:22:25.602743+0000","last_unstale":"2024-05-03T18:27:11.834092+0000","last_undegraded":"2024-05-03T18:27:11.834092+0000","last_fullsized":"2024-05-03T18:27:11.834092+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:49:27.387863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.837215+0000","last_change":"2024-05-03T18:23:20.532077+0000","last_active":"2024-05-03T18:27:11.837215+0000","last_peered":"2024-05-03T18:27:11.837215+0000","last_clean":"2024-05-03T18:27:11.837215+0000","last_became_active":"2024-05-03T18:23:20.531528+0000","last_became_peered":"2024-05-03T18:23:20.531528+0000","last_unstale":"2024-05-03T18:27:11.837215+0000","last_undegraded":"2024-05-03T18:27:11.837215+0000","last_fullsized":"2024-05-03T18:27:11.837215+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:10:37.037114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.835717+0000","last_change":"2024-05-03T18:22:25.600192+0000","last_active":"2024-05-03T18:27:11.835717+0000","last_peered":"2024-05-03T18:27:11.835717+0000","last_clean":"2024-05-03T18:27:11.835717+0000","last_became_active":"2024-05-03T18:22:25.599663+0000","last_became_peered":"2024-05-03T18:22:25.599663+0000","last_unstale":"2024-05-03T18:27:11.835717+0000","last_undegraded":"2024-05-03T18:27:11.835717+0000","last_fullsized":"2024-05-03T18:27:11.835717+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:09:51.614784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.833605+0000","last_change":"2024-05-03T18:22:25.600302+0000","last_active":"2024-05-03T18:27:11.833605+0000","last_peered":"2024-05-03T18:27:11.833605+0000","last_clean":"2024-05-03T18:27:11.833605+0000","last_became_active":"2024-05-03T18:22:25.599832+0000","last_became_peered":"2024-05-03T18:22:25.599832+0000","last_unstale":"2024-05-03T18:27:11.833605+0000","last_undegraded":"2024-05-03T18:27:11.833605+0000","last_fullsized":"2024-05-03T18:27:11.833605+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:52:12.326156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":57,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.830746+0000","last_change":"2024-05-03T18:22:25.590892+0000","last_active":"2024-05-03T18:27:11.830746+0000","last_peered":"2024-05-03T18:27:11.830746+0000","last_clean":"2024-05-03T18:27:11.830746+0000","last_became_active":"2024-05-03T18:22:25.590516+0000","last_became_peered":"2024-05-03T18:22:25.590516+0000","last_unstale":"2024-05-03T18:27:11.830746+0000","last_undegraded":"2024-05-03T18:27:11.830746+0000","last_fullsized":"2024-05-03T18:27:11.830746+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:57:57.999234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.837902+0000","last_change":"2024-05-03T18:22:25.597307+0000","last_active":"2024-05-03T18:27:11.837902+0000","last_peered":"2024-05-03T18:27:11.837902+0000","last_clean":"2024-05-03T18:27:11.837902+0000","last_became_active":"2024-05-03T18:22:25.596989+0000","last_became_peered":"2024-05-03T18:22:25.596989+0000","last_unstale":"2024-05-03T18:27:11.837902+0000","last_undegraded":"2024-05-03T18:27:11.837902+0000","last_fullsized":"2024-05-03T18:27:11.837902+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:53:09.003353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.831580+0000","last_change":"2024-05-03T18:22:25.601723+0000","last_active":"2024-05-03T18:27:11.831580+0000","last_peered":"2024-05-03T18:27:11.831580+0000","last_clean":"2024-05-03T18:27:11.831580+0000","last_became_active":"2024-05-03T18:22:25.599428+0000","last_became_peered":"2024-05-03T18:22:25.599428+0000","last_unstale":"2024-05-03T18:27:11.831580+0000","last_undegraded":"2024-05-03T18:27:11.831580+0000","last_fullsized":"2024-05-03T18:27:11.831580+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:41:18.584497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.838263+0000","last_change":"2024-05-03T18:22:25.594885+0000","last_active":"2024-05-03T18:27:11.838263+0000","last_peered":"2024-05-03T18:27:11.838263+0000","last_clean":"2024-05-03T18:27:11.838263+0000","last_became_active":"2024-05-03T18:22:25.593939+0000","last_became_peered":"2024-05-03T18:22:25.593939+0000","last_unstale":"2024-05-03T18:27:11.838263+0000","last_undegraded":"2024-05-03T18:27:11.838263+0000","last_fullsized":"2024-05-03T18:27:11.838263+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:03:30.052886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.837979+0000","last_change":"2024-05-03T18:22:25.595228+0000","last_active":"2024-05-03T18:27:11.837979+0000","last_peered":"2024-05-03T18:27:11.837979+0000","last_clean":"2024-05-03T18:27:11.837979+0000","last_became_active":"2024-05-03T18:22:25.589825+0000","last_became_peered":"2024-05-03T18:22:25.589825+0000","last_unstale":"2024-05-03T18:27:11.837979+0000","last_undegraded":"2024-05-03T18:27:11.837979+0000","last_fullsized":"2024-05-03T18:27:11.837979+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:44:05.638939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.835425+0000","last_change":"2024-05-03T18:22:25.595630+0000","last_active":"2024-05-03T18:27:11.835425+0000","last_peered":"2024-05-03T18:27:11.835425+0000","last_clean":"2024-05-03T18:27:11.835425+0000","last_became_active":"2024-05-03T18:22:25.591869+0000","last_became_peered":"2024-05-03T18:22:25.591869+0000","last_unstale":"2024-05-03T18:27:11.835425+0000","last_undegraded":"2024-05-03T18:27:11.835425+0000","last_fullsized":"2024-05-03T18:27:11.835425+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:27:43.579329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.841007+0000","last_change":"2024-05-03T18:22:25.589482+0000","last_active":"2024-05-03T18:27:11.841007+0000","last_peered":"2024-05-03T18:27:11.841007+0000","last_clean":"2024-05-03T18:27:11.841007+0000","last_became_active":"2024-05-03T18:22:25.589142+0000","last_became_peered":"2024-05-03T18:22:25.589142+0000","last_unstale":"2024-05-03T18:27:11.841007+0000","last_undegraded":"2024-05-03T18:27:11.841007+0000","last_fullsized":"2024-05-03T18:27:11.841007+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:37:55.472823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.845642+0000","last_change":"2024-05-03T18:22:25.594569+0000","last_active":"2024-05-03T18:27:11.845642+0000","last_peered":"2024-05-03T18:27:11.845642+0000","last_clean":"2024-05-03T18:27:11.845642+0000","last_became_active":"2024-05-03T18:22:25.594218+0000","last_became_peered":"2024-05-03T18:22:25.594218+0000","last_unstale":"2024-05-03T18:27:11.845642+0000","last_undegraded":"2024-05-03T18:27:11.845642+0000","last_fullsized":"2024-05-03T18:27:11.845642+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:30:42.775673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.835918+0000","last_change":"2024-05-03T18:22:25.597426+0000","last_active":"2024-05-03T18:27:11.835918+0000","last_peered":"2024-05-03T18:27:11.835918+0000","last_clean":"2024-05-03T18:27:11.835918+0000","last_became_active":"2024-05-03T18:22:25.594524+0000","last_became_peered":"2024-05-03T18:22:25.594524+0000","last_unstale":"2024-05-03T18:27:11.835918+0000","last_undegraded":"2024-05-03T18:27:11.835918+0000","last_fullsized":"2024-05-03T18:27:11.835918+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:41:36.984253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.826557+0000","last_change":"2024-05-03T18:22:25.588002+0000","last_active":"2024-05-03T18:27:11.826557+0000","last_peered":"2024-05-03T18:27:11.826557+0000","last_clean":"2024-05-03T18:27:11.826557+0000","last_became_active":"2024-05-03T18:22:25.587547+0000","last_became_peered":"2024-05-03T18:22:25.587547+0000","last_unstale":"2024-05-03T18:27:11.826557+0000","last_undegraded":"2024-05-03T18:27:11.826557+0000","last_fullsized":"2024-05-03T18:27:11.826557+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:47:14.115517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.767050+0000","last_change":"2024-05-03T18:22:40.754602+0000","last_active":"2024-05-03T18:27:11.767050+0000","last_peered":"2024-05-03T18:27:11.767050+0000","last_clean":"2024-05-03T18:27:11.767050+0000","last_became_active":"2024-05-03T18:22:40.752917+0000","last_became_peered":"2024-05-03T18:22:40.752917+0000","last_unstale":"2024-05-03T18:27:11.767050+0000","last_undegraded":"2024-05-03T18:27:11.767050+0000","last_fullsized":"2024-05-03T18:27:11.767050+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:51:44.711811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"78'4","reported_seq":53,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.781114+0000","last_change":"2024-05-03T18:22:21.558560+0000","last_active":"2024-05-03T18:27:11.781114+0000","last_peered":"2024-05-03T18:27:11.781114+0000","last_clean":"2024-05-03T18:27:11.781114+0000","last_became_active":"2024-05-03T18:22:21.558185+0000","last_became_peered":"2024-05-03T18:22:21.558185+0000","last_unstale":"2024-05-03T18:27:11.781114+0000","last_undegraded":"2024-05-03T18:27:11.781114+0000","last_fullsized":"2024-05-03T18:27:11.781114+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:18:19.666724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.830855+0000","last_change":"2024-05-03T18:22:25.603934+0000","last_active":"2024-05-03T18:27:11.830855+0000","last_peered":"2024-05-03T18:27:11.830855+0000","last_clean":"2024-05-03T18:27:11.830855+0000","last_became_active":"2024-05-03T18:22:25.603529+0000","last_became_peered":"2024-05-03T18:22:25.603529+0000","last_unstale":"2024-05-03T18:27:11.830855+0000","last_undegraded":"2024-05-03T18:27:11.830855+0000","last_fullsized":"2024-05-03T18:27:11.830855+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:08:50.288871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.781004+0000","last_change":"2024-05-03T18:23:20.576254+0000","last_active":"2024-05-03T18:27:11.781004+0000","last_peered":"2024-05-03T18:27:11.781004+0000","last_clean":"2024-05-03T18:27:11.781004+0000","last_became_active":"2024-05-03T18:23:20.575774+0000","last_became_peered":"2024-05-03T18:23:20.575774+0000","last_unstale":"2024-05-03T18:27:11.781004+0000","last_undegraded":"2024-05-03T18:27:11.781004+0000","last_fullsized":"2024-05-03T18:27:11.781004+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:22:39.090938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"78'1","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.780167+0000","last_change":"2024-05-03T18:22:21.734171+0000","last_active":"2024-05-03T18:27:11.780167+0000","last_peered":"2024-05-03T18:27:11.780167+0000","last_clean":"2024-05-03T18:27:11.780167+0000","last_became_active":"2024-05-03T18:22:21.733301+0000","last_became_peered":"2024-05-03T18:22:21.733301+0000","last_unstale":"2024-05-03T18:27:11.780167+0000","last_undegraded":"2024-05-03T18:27:11.780167+0000","last_fullsized":"2024-05-03T18:27:11.780167+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:09:48.861247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.845701+0000","last_change":"2024-05-03T18:22:25.598016+0000","last_active":"2024-05-03T18:27:11.845701+0000","last_peered":"2024-05-03T18:27:11.845701+0000","last_clean":"2024-05-03T18:27:11.845701+0000","last_became_active":"2024-05-03T18:22:25.594888+0000","last_became_peered":"2024-05-03T18:22:25.594888+0000","last_unstale":"2024-05-03T18:27:11.845701+0000","last_undegraded":"2024-05-03T18:27:11.845701+0000","last_fullsized":"2024-05-03T18:27:11.845701+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:47:46.400247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.785843+0000","last_change":"2024-05-03T18:22:40.755148+0000","last_active":"2024-05-03T18:27:11.785843+0000","last_peered":"2024-05-03T18:27:11.785843+0000","last_clean":"2024-05-03T18:27:11.785843+0000","last_became_active":"2024-05-03T18:22:40.754656+0000","last_became_peered":"2024-05-03T18:22:40.754656+0000","last_unstale":"2024-05-03T18:27:11.785843+0000","last_undegraded":"2024-05-03T18:27:11.785843+0000","last_fullsized":"2024-05-03T18:27:11.785843+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:48:23.124936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"78'2","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.784439+0000","last_change":"2024-05-03T18:22:21.568060+0000","last_active":"2024-05-03T18:27:11.784439+0000","last_peered":"2024-05-03T18:27:11.784439+0000","last_clean":"2024-05-03T18:27:11.784439+0000","last_became_active":"2024-05-03T18:22:21.567434+0000","last_became_peered":"2024-05-03T18:22:21.567434+0000","last_unstale":"2024-05-03T18:27:11.784439+0000","last_undegraded":"2024-05-03T18:27:11.784439+0000","last_fullsized":"2024-05-03T18:27:11.784439+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:06:05.802621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.829909+0000","last_change":"2024-05-03T18:22:25.597777+0000","last_active":"2024-05-03T18:27:11.829909+0000","last_peered":"2024-05-03T18:27:11.829909+0000","last_clean":"2024-05-03T18:27:11.829909+0000","last_became_active":"2024-05-03T18:22:25.597391+0000","last_became_peered":"2024-05-03T18:22:25.597391+0000","last_unstale":"2024-05-03T18:27:11.829909+0000","last_undegraded":"2024-05-03T18:27:11.829909+0000","last_fullsized":"2024-05-03T18:27:11.829909+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:02:24.708193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.783801+0000","last_change":"2024-05-03T18:22:40.752318+0000","last_active":"2024-05-03T18:27:11.783801+0000","last_peered":"2024-05-03T18:27:11.783801+0000","last_clean":"2024-05-03T18:27:11.783801+0000","last_became_active":"2024-05-03T18:22:40.751433+0000","last_became_peered":"2024-05-03T18:22:40.751433+0000","last_unstale":"2024-05-03T18:27:11.783801+0000","last_undegraded":"2024-05-03T18:27:11.783801+0000","last_fullsized":"2024-05-03T18:27:11.783801+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:21:55.746143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.791173+0000","last_change":"2024-05-03T18:24:21.582330+0000","last_active":"2024-05-03T18:27:11.791173+0000","last_peered":"2024-05-03T18:27:11.791173+0000","last_clean":"2024-05-03T18:27:11.791173+0000","last_became_active":"2024-05-03T18:24:21.572706+0000","last_became_peered":"2024-05-03T18:24:21.572706+0000","last_unstale":"2024-05-03T18:27:11.791173+0000","last_undegraded":"2024-05-03T18:27:11.791173+0000","last_fullsized":"2024-05-03T18:27:11.791173+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:55:40.233995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":62,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.838824+0000","last_change":"2024-05-03T18:23:20.575102+0000","last_active":"2024-05-03T18:27:11.838824+0000","last_peered":"2024-05-03T18:27:11.838824+0000","last_clean":"2024-05-03T18:27:11.838824+0000","last_became_active":"2024-05-03T18:23:20.574657+0000","last_became_peered":"2024-05-03T18:23:20.574657+0000","last_unstale":"2024-05-03T18:27:11.838824+0000","last_undegraded":"2024-05-03T18:27:11.838824+0000","last_fullsized":"2024-05-03T18:27:11.838824+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:08:26.280866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.767873+0000","last_change":"2024-05-03T18:22:40.761052+0000","last_active":"2024-05-03T18:27:11.767873+0000","last_peered":"2024-05-03T18:27:11.767873+0000","last_clean":"2024-05-03T18:27:11.767873+0000","last_became_active":"2024-05-03T18:22:40.760179+0000","last_became_peered":"2024-05-03T18:22:40.760179+0000","last_unstale":"2024-05-03T18:27:11.767873+0000","last_undegraded":"2024-05-03T18:27:11.767873+0000","last_fullsized":"2024-05-03T18:27:11.767873+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:44:48.610680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"78'1","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.745780+0000","last_change":"2024-05-03T18:22:21.563421+0000","last_active":"2024-05-03T18:27:11.745780+0000","last_peered":"2024-05-03T18:27:11.745780+0000","last_clean":"2024-05-03T18:27:11.745780+0000","last_became_active":"2024-05-03T18:22:21.560991+0000","last_became_peered":"2024-05-03T18:22:21.560991+0000","last_unstale":"2024-05-03T18:27:11.745780+0000","last_undegraded":"2024-05-03T18:27:11.745780+0000","last_fullsized":"2024-05-03T18:27:11.745780+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:50:32.640699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.823442+0000","last_change":"2024-05-03T18:22:25.599049+0000","last_active":"2024-05-03T18:27:11.823442+0000","last_peered":"2024-05-03T18:27:11.823442+0000","last_clean":"2024-05-03T18:27:11.823442+0000","last_became_active":"2024-05-03T18:22:25.597879+0000","last_became_peered":"2024-05-03T18:22:25.597879+0000","last_unstale":"2024-05-03T18:27:11.823442+0000","last_undegraded":"2024-05-03T18:27:11.823442+0000","last_fullsized":"2024-05-03T18:27:11.823442+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:51:08.487499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.767790+0000","last_change":"2024-05-03T18:22:40.760908+0000","last_active":"2024-05-03T18:27:11.767790+0000","last_peered":"2024-05-03T18:27:11.767790+0000","last_clean":"2024-05-03T18:27:11.767790+0000","last_became_active":"2024-05-03T18:22:40.760331+0000","last_became_peered":"2024-05-03T18:22:40.760331+0000","last_unstale":"2024-05-03T18:27:11.767790+0000","last_undegraded":"2024-05-03T18:27:11.767790+0000","last_fullsized":"2024-05-03T18:27:11.767790+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:27:08.216015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"78'1","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.761175+0000","last_change":"2024-05-03T18:22:21.565328+0000","last_active":"2024-05-03T18:27:11.761175+0000","last_peered":"2024-05-03T18:27:11.761175+0000","last_clean":"2024-05-03T18:27:11.761175+0000","last_became_active":"2024-05-03T18:22:21.564583+0000","last_became_peered":"2024-05-03T18:22:21.564583+0000","last_unstale":"2024-05-03T18:27:11.761175+0000","last_undegraded":"2024-05-03T18:27:11.761175+0000","last_fullsized":"2024-05-03T18:27:11.761175+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:58:32.370458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.828897+0000","last_change":"2024-05-03T18:22:25.607585+0000","last_active":"2024-05-03T18:27:11.828897+0000","last_peered":"2024-05-03T18:27:11.828897+0000","last_clean":"2024-05-03T18:27:11.828897+0000","last_became_active":"2024-05-03T18:22:25.607168+0000","last_became_peered":"2024-05-03T18:22:25.607168+0000","last_unstale":"2024-05-03T18:27:11.828897+0000","last_undegraded":"2024-05-03T18:27:11.828897+0000","last_fullsized":"2024-05-03T18:27:11.828897+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:04:39.750823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.803875+0000","last_change":"2024-05-03T18:22:40.754744+0000","last_active":"2024-05-03T18:27:11.803875+0000","last_peered":"2024-05-03T18:27:11.803875+0000","last_clean":"2024-05-03T18:27:11.803875+0000","last_became_active":"2024-05-03T18:22:40.754262+0000","last_became_peered":"2024-05-03T18:22:40.754262+0000","last_unstale":"2024-05-03T18:27:11.803875+0000","last_undegraded":"2024-05-03T18:27:11.803875+0000","last_fullsized":"2024-05-03T18:27:11.803875+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:11:47.979546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.738652+0000","last_change":"2024-05-03T18:22:21.550048+0000","last_active":"2024-05-03T18:27:11.738652+0000","last_peered":"2024-05-03T18:27:11.738652+0000","last_clean":"2024-05-03T18:27:11.738652+0000","last_became_active":"2024-05-03T18:22:21.549334+0000","last_became_peered":"2024-05-03T18:22:21.549334+0000","last_unstale":"2024-05-03T18:27:11.738652+0000","last_undegraded":"2024-05-03T18:27:11.738652+0000","last_fullsized":"2024-05-03T18:27:11.738652+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:52:54.401185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.834700+0000","last_change":"2024-05-03T18:22:25.583074+0000","last_active":"2024-05-03T18:27:11.834700+0000","last_peered":"2024-05-03T18:27:11.834700+0000","last_clean":"2024-05-03T18:27:11.834700+0000","last_became_active":"2024-05-03T18:22:25.582637+0000","last_became_peered":"2024-05-03T18:22:25.582637+0000","last_unstale":"2024-05-03T18:27:11.834700+0000","last_undegraded":"2024-05-03T18:27:11.834700+0000","last_fullsized":"2024-05-03T18:27:11.834700+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:20:09.021402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.778047+0000","last_change":"2024-05-03T18:22:40.748509+0000","last_active":"2024-05-03T18:27:11.778047+0000","last_peered":"2024-05-03T18:27:11.778047+0000","last_clean":"2024-05-03T18:27:11.778047+0000","last_became_active":"2024-05-03T18:22:40.748051+0000","last_became_peered":"2024-05-03T18:22:40.748051+0000","last_unstale":"2024-05-03T18:27:11.778047+0000","last_undegraded":"2024-05-03T18:27:11.778047+0000","last_fullsized":"2024-05-03T18:27:11.778047+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:08:44.389460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.742290+0000","last_change":"2024-05-03T18:22:21.562185+0000","last_active":"2024-05-03T18:27:11.742290+0000","last_peered":"2024-05-03T18:27:11.742290+0000","last_clean":"2024-05-03T18:27:11.742290+0000","last_became_active":"2024-05-03T18:22:21.560523+0000","last_became_peered":"2024-05-03T18:22:21.560523+0000","last_unstale":"2024-05-03T18:27:11.742290+0000","last_undegraded":"2024-05-03T18:27:11.742290+0000","last_fullsized":"2024-05-03T18:27:11.742290+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:05:41.788286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.831975+0000","last_change":"2024-05-03T18:22:25.583895+0000","last_active":"2024-05-03T18:27:11.831975+0000","last_peered":"2024-05-03T18:27:11.831975+0000","last_clean":"2024-05-03T18:27:11.831975+0000","last_became_active":"2024-05-03T18:22:25.583503+0000","last_became_peered":"2024-05-03T18:22:25.583503+0000","last_unstale":"2024-05-03T18:27:11.831975+0000","last_undegraded":"2024-05-03T18:27:11.831975+0000","last_fullsized":"2024-05-03T18:27:11.831975+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:45:32.404394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.774438+0000","last_change":"2024-05-03T18:22:40.752814+0000","last_active":"2024-05-03T18:27:11.774438+0000","last_peered":"2024-05-03T18:27:11.774438+0000","last_clean":"2024-05-03T18:27:11.774438+0000","last_became_active":"2024-05-03T18:22:40.752328+0000","last_became_peered":"2024-05-03T18:22:40.752328+0000","last_unstale":"2024-05-03T18:27:11.774438+0000","last_undegraded":"2024-05-03T18:27:11.774438+0000","last_fullsized":"2024-05-03T18:27:11.774438+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:16:39.488122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.762244+0000","last_change":"2024-05-03T18:22:21.566266+0000","last_active":"2024-05-03T18:27:11.762244+0000","last_peered":"2024-05-03T18:27:11.762244+0000","last_clean":"2024-05-03T18:27:11.762244+0000","last_became_active":"2024-05-03T18:22:21.565612+0000","last_became_peered":"2024-05-03T18:22:21.565612+0000","last_unstale":"2024-05-03T18:27:11.762244+0000","last_undegraded":"2024-05-03T18:27:11.762244+0000","last_fullsized":"2024-05-03T18:27:11.762244+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:22:57.216912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.836780+0000","last_change":"2024-05-03T18:22:25.596182+0000","last_active":"2024-05-03T18:27:11.836780+0000","last_peered":"2024-05-03T18:27:11.836780+0000","last_clean":"2024-05-03T18:27:11.836780+0000","last_became_active":"2024-05-03T18:22:25.595472+0000","last_became_peered":"2024-05-03T18:22:25.595472+0000","last_unstale":"2024-05-03T18:27:11.836780+0000","last_undegraded":"2024-05-03T18:27:11.836780+0000","last_fullsized":"2024-05-03T18:27:11.836780+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:48:49.583879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.783452+0000","last_change":"2024-05-03T18:22:40.756378+0000","last_active":"2024-05-03T18:27:11.783452+0000","last_peered":"2024-05-03T18:27:11.783452+0000","last_clean":"2024-05-03T18:27:11.783452+0000","last_became_active":"2024-05-03T18:22:40.755692+0000","last_became_peered":"2024-05-03T18:22:40.755692+0000","last_unstale":"2024-05-03T18:27:11.783452+0000","last_undegraded":"2024-05-03T18:27:11.783452+0000","last_fullsized":"2024-05-03T18:27:11.783452+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:22:50.269817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.744083+0000","last_change":"2024-05-03T18:22:21.561381+0000","last_active":"2024-05-03T18:27:11.744083+0000","last_peered":"2024-05-03T18:27:11.744083+0000","last_clean":"2024-05-03T18:27:11.744083+0000","last_became_active":"2024-05-03T18:22:21.559411+0000","last_became_peered":"2024-05-03T18:22:21.559411+0000","last_unstale":"2024-05-03T18:27:11.744083+0000","last_undegraded":"2024-05-03T18:27:11.744083+0000","last_fullsized":"2024-05-03T18:27:11.744083+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:04:08.283538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":56,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.839913+0000","last_change":"2024-05-03T18:23:20.551361+0000","last_active":"2024-05-03T18:27:11.839913+0000","last_peered":"2024-05-03T18:27:11.839913+0000","last_clean":"2024-05-03T18:27:11.839913+0000","last_became_active":"2024-05-03T18:23:20.550689+0000","last_became_peered":"2024-05-03T18:23:20.550689+0000","last_unstale":"2024-05-03T18:27:11.839913+0000","last_undegraded":"2024-05-03T18:27:11.839913+0000","last_fullsized":"2024-05-03T18:27:11.839913+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:36:52.767110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.787425+0000","last_change":"2024-05-03T18:22:40.758006+0000","last_active":"2024-05-03T18:27:11.787425+0000","last_peered":"2024-05-03T18:27:11.787425+0000","last_clean":"2024-05-03T18:27:11.787425+0000","last_became_active":"2024-05-03T18:22:40.757075+0000","last_became_peered":"2024-05-03T18:22:40.757075+0000","last_unstale":"2024-05-03T18:27:11.787425+0000","last_undegraded":"2024-05-03T18:27:11.787425+0000","last_fullsized":"2024-05-03T18:27:11.787425+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:06:02.390184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"78'3","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.744164+0000","last_change":"2024-05-03T18:22:21.740041+0000","last_active":"2024-05-03T18:27:11.744164+0000","last_peered":"2024-05-03T18:27:11.744164+0000","last_clean":"2024-05-03T18:27:11.744164+0000","last_became_active":"2024-05-03T18:22:21.738777+0000","last_became_peered":"2024-05-03T18:22:21.738777+0000","last_unstale":"2024-05-03T18:27:11.744164+0000","last_undegraded":"2024-05-03T18:27:11.744164+0000","last_fullsized":"2024-05-03T18:27:11.744164+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:35:30.098325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":53,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.825545+0000","last_change":"2024-05-03T18:22:25.606854+0000","last_active":"2024-05-03T18:27:11.825545+0000","last_peered":"2024-05-03T18:27:11.825545+0000","last_clean":"2024-05-03T18:27:11.825545+0000","last_became_active":"2024-05-03T18:22:25.606376+0000","last_became_peered":"2024-05-03T18:22:25.606376+0000","last_unstale":"2024-05-03T18:27:11.825545+0000","last_undegraded":"2024-05-03T18:27:11.825545+0000","last_fullsized":"2024-05-03T18:27:11.825545+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:29:59.461181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.784158+0000","last_change":"2024-05-03T18:22:40.752863+0000","last_active":"2024-05-03T18:27:11.784158+0000","last_peered":"2024-05-03T18:27:11.784158+0000","last_clean":"2024-05-03T18:27:11.784158+0000","last_became_active":"2024-05-03T18:22:40.752309+0000","last_became_peered":"2024-05-03T18:22:40.752309+0000","last_unstale":"2024-05-03T18:27:11.784158+0000","last_undegraded":"2024-05-03T18:27:11.784158+0000","last_fullsized":"2024-05-03T18:27:11.784158+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:01:06.110570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'37","reported_seq":510,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:12.403811+0000","last_change":"2024-05-03T18:22:21.566162+0000","last_active":"2024-05-03T18:27:12.403811+0000","last_peered":"2024-05-03T18:27:12.403811+0000","last_clean":"2024-05-03T18:27:12.403811+0000","last_became_active":"2024-05-03T18:22:21.565641+0000","last_became_peered":"2024-05-03T18:22:21.565641+0000","last_unstale":"2024-05-03T18:27:12.403811+0000","last_undegraded":"2024-05-03T18:27:12.403811+0000","last_fullsized":"2024-05-03T18:27:12.403811+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:23:45.283135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69239,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":425,"num_read_kb":281,"num_write":37,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.833787+0000","last_change":"2024-05-03T18:22:25.586687+0000","last_active":"2024-05-03T18:27:11.833787+0000","last_peered":"2024-05-03T18:27:11.833787+0000","last_clean":"2024-05-03T18:27:11.833787+0000","last_became_active":"2024-05-03T18:22:25.586275+0000","last_became_peered":"2024-05-03T18:22:25.586275+0000","last_unstale":"2024-05-03T18:27:11.833787+0000","last_undegraded":"2024-05-03T18:27:11.833787+0000","last_fullsized":"2024-05-03T18:27:11.833787+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:10:50.339865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.744602+0000","last_change":"2024-05-03T18:22:40.758003+0000","last_active":"2024-05-03T18:27:11.744602+0000","last_peered":"2024-05-03T18:27:11.744602+0000","last_clean":"2024-05-03T18:27:11.744602+0000","last_became_active":"2024-05-03T18:22:40.757559+0000","last_became_peered":"2024-05-03T18:22:40.757559+0000","last_unstale":"2024-05-03T18:27:11.744602+0000","last_undegraded":"2024-05-03T18:27:11.744602+0000","last_fullsized":"2024-05-03T18:27:11.744602+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:54:23.377748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"78'2","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.767568+0000","last_change":"2024-05-03T18:22:21.564546+0000","last_active":"2024-05-03T18:27:11.767568+0000","last_peered":"2024-05-03T18:27:11.767568+0000","last_clean":"2024-05-03T18:27:11.767568+0000","last_became_active":"2024-05-03T18:22:21.564105+0000","last_became_peered":"2024-05-03T18:22:21.564105+0000","last_unstale":"2024-05-03T18:27:11.767568+0000","last_undegraded":"2024-05-03T18:27:11.767568+0000","last_fullsized":"2024-05-03T18:27:11.767568+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:36:07.790643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.838916+0000","last_change":"2024-05-03T18:22:25.584059+0000","last_active":"2024-05-03T18:27:11.838916+0000","last_peered":"2024-05-03T18:27:11.838916+0000","last_clean":"2024-05-03T18:27:11.838916+0000","last_became_active":"2024-05-03T18:22:25.583742+0000","last_became_peered":"2024-05-03T18:22:25.583742+0000","last_unstale":"2024-05-03T18:27:11.838916+0000","last_undegraded":"2024-05-03T18:27:11.838916+0000","last_fullsized":"2024-05-03T18:27:11.838916+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:15:48.288422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.791333+0000","last_change":"2024-05-03T18:22:40.751362+0000","last_active":"2024-05-03T18:27:11.791333+0000","last_peered":"2024-05-03T18:27:11.791333+0000","last_clean":"2024-05-03T18:27:11.791333+0000","last_became_active":"2024-05-03T18:22:40.750365+0000","last_became_peered":"2024-05-03T18:22:40.750365+0000","last_unstale":"2024-05-03T18:27:11.791333+0000","last_undegraded":"2024-05-03T18:27:11.791333+0000","last_fullsized":"2024-05-03T18:27:11.791333+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:35:50.629332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.742961+0000","last_change":"2024-05-03T18:22:21.556123+0000","last_active":"2024-05-03T18:27:11.742961+0000","last_peered":"2024-05-03T18:27:11.742961+0000","last_clean":"2024-05-03T18:27:11.742961+0000","last_became_active":"2024-05-03T18:22:21.554918+0000","last_became_peered":"2024-05-03T18:22:21.554918+0000","last_unstale":"2024-05-03T18:27:11.742961+0000","last_undegraded":"2024-05-03T18:27:11.742961+0000","last_fullsized":"2024-05-03T18:27:11.742961+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:09:12.383601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":57,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.840168+0000","last_change":"2024-05-03T18:22:25.601849+0000","last_active":"2024-05-03T18:27:11.840168+0000","last_peered":"2024-05-03T18:27:11.840168+0000","last_clean":"2024-05-03T18:27:11.840168+0000","last_became_active":"2024-05-03T18:22:25.601434+0000","last_became_peered":"2024-05-03T18:22:25.601434+0000","last_unstale":"2024-05-03T18:27:11.840168+0000","last_undegraded":"2024-05-03T18:27:11.840168+0000","last_fullsized":"2024-05-03T18:27:11.840168+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:43:18.273389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.780768+0000","last_change":"2024-05-03T18:22:40.754448+0000","last_active":"2024-05-03T18:27:11.780768+0000","last_peered":"2024-05-03T18:27:11.780768+0000","last_clean":"2024-05-03T18:27:11.780768+0000","last_became_active":"2024-05-03T18:22:40.753122+0000","last_became_peered":"2024-05-03T18:22:40.753122+0000","last_unstale":"2024-05-03T18:27:11.780768+0000","last_undegraded":"2024-05-03T18:27:11.780768+0000","last_fullsized":"2024-05-03T18:27:11.780768+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:43:15.216222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.778755+0000","last_change":"2024-05-03T18:22:21.559292+0000","last_active":"2024-05-03T18:27:11.778755+0000","last_peered":"2024-05-03T18:27:11.778755+0000","last_clean":"2024-05-03T18:27:11.778755+0000","last_became_active":"2024-05-03T18:22:21.557657+0000","last_became_peered":"2024-05-03T18:22:21.557657+0000","last_unstale":"2024-05-03T18:27:11.778755+0000","last_undegraded":"2024-05-03T18:27:11.778755+0000","last_fullsized":"2024-05-03T18:27:11.778755+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:51:57.709387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.837734+0000","last_change":"2024-05-03T18:22:25.604340+0000","last_active":"2024-05-03T18:27:11.837734+0000","last_peered":"2024-05-03T18:27:11.837734+0000","last_clean":"2024-05-03T18:27:11.837734+0000","last_became_active":"2024-05-03T18:22:25.603651+0000","last_became_peered":"2024-05-03T18:22:25.603651+0000","last_unstale":"2024-05-03T18:27:11.837734+0000","last_undegraded":"2024-05-03T18:27:11.837734+0000","last_fullsized":"2024-05-03T18:27:11.837734+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:49:36.379556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.740450+0000","last_change":"2024-05-03T18:22:40.738065+0000","last_active":"2024-05-03T18:27:11.740450+0000","last_peered":"2024-05-03T18:27:11.740450+0000","last_clean":"2024-05-03T18:27:11.740450+0000","last_became_active":"2024-05-03T18:22:40.737546+0000","last_became_peered":"2024-05-03T18:22:40.737546+0000","last_unstale":"2024-05-03T18:27:11.740450+0000","last_undegraded":"2024-05-03T18:27:11.740450+0000","last_fullsized":"2024-05-03T18:27:11.740450+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:45:31.528581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"78'8","reported_seq":56,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.778653+0000","last_change":"2024-05-03T18:22:21.739131+0000","last_active":"2024-05-03T18:27:11.778653+0000","last_peered":"2024-05-03T18:27:11.778653+0000","last_clean":"2024-05-03T18:27:11.778653+0000","last_became_active":"2024-05-03T18:22:21.737243+0000","last_became_peered":"2024-05-03T18:22:21.737243+0000","last_unstale":"2024-05-03T18:27:11.778653+0000","last_undegraded":"2024-05-03T18:27:11.778653+0000","last_fullsized":"2024-05-03T18:27:11.778653+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:46:24.011140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.830488+0000","last_change":"2024-05-03T18:22:25.602011+0000","last_active":"2024-05-03T18:27:11.830488+0000","last_peered":"2024-05-03T18:27:11.830488+0000","last_clean":"2024-05-03T18:27:11.830488+0000","last_became_active":"2024-05-03T18:22:25.601604+0000","last_became_peered":"2024-05-03T18:22:25.601604+0000","last_unstale":"2024-05-03T18:27:11.830488+0000","last_undegraded":"2024-05-03T18:27:11.830488+0000","last_fullsized":"2024-05-03T18:27:11.830488+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:25:57.743203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.742365+0000","last_change":"2024-05-03T18:22:40.757885+0000","last_active":"2024-05-03T18:27:11.742365+0000","last_peered":"2024-05-03T18:27:11.742365+0000","last_clean":"2024-05-03T18:27:11.742365+0000","last_became_active":"2024-05-03T18:22:40.757208+0000","last_became_peered":"2024-05-03T18:22:40.757208+0000","last_unstale":"2024-05-03T18:27:11.742365+0000","last_undegraded":"2024-05-03T18:27:11.742365+0000","last_fullsized":"2024-05-03T18:27:11.742365+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:37:00.792138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"78'1","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.791831+0000","last_change":"2024-05-03T18:22:21.554932+0000","last_active":"2024-05-03T18:27:11.791831+0000","last_peered":"2024-05-03T18:27:11.791831+0000","last_clean":"2024-05-03T18:27:11.791831+0000","last_became_active":"2024-05-03T18:22:21.554123+0000","last_became_peered":"2024-05-03T18:22:21.554123+0000","last_unstale":"2024-05-03T18:27:11.791831+0000","last_undegraded":"2024-05-03T18:27:11.791831+0000","last_fullsized":"2024-05-03T18:27:11.791831+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:38:59.859138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.831370+0000","last_change":"2024-05-03T18:22:25.603400+0000","last_active":"2024-05-03T18:27:11.831370+0000","last_peered":"2024-05-03T18:27:11.831370+0000","last_clean":"2024-05-03T18:27:11.831370+0000","last_became_active":"2024-05-03T18:22:25.603045+0000","last_became_peered":"2024-05-03T18:22:25.603045+0000","last_unstale":"2024-05-03T18:27:11.831370+0000","last_undegraded":"2024-05-03T18:27:11.831370+0000","last_fullsized":"2024-05-03T18:27:11.831370+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:51:33.502972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.778451+0000","last_change":"2024-05-03T18:22:40.750857+0000","last_active":"2024-05-03T18:27:11.778451+0000","last_peered":"2024-05-03T18:27:11.778451+0000","last_clean":"2024-05-03T18:27:11.778451+0000","last_became_active":"2024-05-03T18:22:40.750381+0000","last_became_peered":"2024-05-03T18:22:40.750381+0000","last_unstale":"2024-05-03T18:27:11.778451+0000","last_undegraded":"2024-05-03T18:27:11.778451+0000","last_fullsized":"2024-05-03T18:27:11.778451+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:15:28.542612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.781805+0000","last_change":"2024-05-03T18:22:21.739345+0000","last_active":"2024-05-03T18:27:11.781805+0000","last_peered":"2024-05-03T18:27:11.781805+0000","last_clean":"2024-05-03T18:27:11.781805+0000","last_became_active":"2024-05-03T18:22:21.738874+0000","last_became_peered":"2024-05-03T18:22:21.738874+0000","last_unstale":"2024-05-03T18:27:11.781805+0000","last_undegraded":"2024-05-03T18:27:11.781805+0000","last_fullsized":"2024-05-03T18:27:11.781805+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:57:59.706356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"90'4","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.834563+0000","last_change":"2024-05-03T18:22:25.586543+0000","last_active":"2024-05-03T18:27:11.834563+0000","last_peered":"2024-05-03T18:27:11.834563+0000","last_clean":"2024-05-03T18:27:11.834563+0000","last_became_active":"2024-05-03T18:22:25.585124+0000","last_became_peered":"2024-05-03T18:22:25.585124+0000","last_unstale":"2024-05-03T18:27:11.834563+0000","last_undegraded":"2024-05-03T18:27:11.834563+0000","last_fullsized":"2024-05-03T18:27:11.834563+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:41:32.633444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.739379+0000","last_change":"2024-05-03T18:22:40.749665+0000","last_active":"2024-05-03T18:27:11.739379+0000","last_peered":"2024-05-03T18:27:11.739379+0000","last_clean":"2024-05-03T18:27:11.739379+0000","last_became_active":"2024-05-03T18:22:40.749062+0000","last_became_peered":"2024-05-03T18:22:40.749062+0000","last_unstale":"2024-05-03T18:27:11.739379+0000","last_undegraded":"2024-05-03T18:27:11.739379+0000","last_fullsized":"2024-05-03T18:27:11.739379+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:09:17.194507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.742374+0000","last_change":"2024-05-03T18:22:21.560944+0000","last_active":"2024-05-03T18:27:11.742374+0000","last_peered":"2024-05-03T18:27:11.742374+0000","last_clean":"2024-05-03T18:27:11.742374+0000","last_became_active":"2024-05-03T18:22:21.560412+0000","last_became_peered":"2024-05-03T18:22:21.560412+0000","last_unstale":"2024-05-03T18:27:11.742374+0000","last_undegraded":"2024-05-03T18:27:11.742374+0000","last_fullsized":"2024-05-03T18:27:11.742374+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:12:04.069564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.841297+0000","last_change":"2024-05-03T18:22:25.598235+0000","last_active":"2024-05-03T18:27:11.841297+0000","last_peered":"2024-05-03T18:27:11.841297+0000","last_clean":"2024-05-03T18:27:11.841297+0000","last_became_active":"2024-05-03T18:22:25.595993+0000","last_became_peered":"2024-05-03T18:22:25.595993+0000","last_unstale":"2024-05-03T18:27:11.841297+0000","last_undegraded":"2024-05-03T18:27:11.841297+0000","last_fullsized":"2024-05-03T18:27:11.841297+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:53:16.069119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.741050+0000","last_change":"2024-05-03T18:22:40.745005+0000","last_active":"2024-05-03T18:27:11.741050+0000","last_peered":"2024-05-03T18:27:11.741050+0000","last_clean":"2024-05-03T18:27:11.741050+0000","last_became_active":"2024-05-03T18:22:40.742134+0000","last_became_peered":"2024-05-03T18:22:40.742134+0000","last_unstale":"2024-05-03T18:27:11.741050+0000","last_undegraded":"2024-05-03T18:27:11.741050+0000","last_fullsized":"2024-05-03T18:27:11.741050+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:47:12.217012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.829885+0000","last_change":"2024-05-03T18:22:25.604640+0000","last_active":"2024-05-03T18:27:11.829885+0000","last_peered":"2024-05-03T18:27:11.829885+0000","last_clean":"2024-05-03T18:27:11.829885+0000","last_became_active":"2024-05-03T18:22:25.603759+0000","last_became_peered":"2024-05-03T18:22:25.603759+0000","last_unstale":"2024-05-03T18:27:11.829885+0000","last_undegraded":"2024-05-03T18:27:11.829885+0000","last_fullsized":"2024-05-03T18:27:11.829885+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:56:55.744321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.738330+0000","last_change":"2024-05-03T18:22:21.741006+0000","last_active":"2024-05-03T18:27:11.738330+0000","last_peered":"2024-05-03T18:27:11.738330+0000","last_clean":"2024-05-03T18:27:11.738330+0000","last_became_active":"2024-05-03T18:22:21.737999+0000","last_became_peered":"2024-05-03T18:22:21.737999+0000","last_unstale":"2024-05-03T18:27:11.738330+0000","last_undegraded":"2024-05-03T18:27:11.738330+0000","last_fullsized":"2024-05-03T18:27:11.738330+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:29:54.931940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.743698+0000","last_change":"2024-05-03T18:22:40.761044+0000","last_active":"2024-05-03T18:27:11.743698+0000","last_peered":"2024-05-03T18:27:11.743698+0000","last_clean":"2024-05-03T18:27:11.743698+0000","last_became_active":"2024-05-03T18:22:40.760245+0000","last_became_peered":"2024-05-03T18:22:40.760245+0000","last_unstale":"2024-05-03T18:27:11.743698+0000","last_undegraded":"2024-05-03T18:27:11.743698+0000","last_fullsized":"2024-05-03T18:27:11.743698+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:31:12.729078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.832106+0000","last_change":"2024-05-03T18:22:25.595759+0000","last_active":"2024-05-03T18:27:11.832106+0000","last_peered":"2024-05-03T18:27:11.832106+0000","last_clean":"2024-05-03T18:27:11.832106+0000","last_became_active":"2024-05-03T18:22:25.594368+0000","last_became_peered":"2024-05-03T18:22:25.594368+0000","last_unstale":"2024-05-03T18:27:11.832106+0000","last_undegraded":"2024-05-03T18:27:11.832106+0000","last_fullsized":"2024-05-03T18:27:11.832106+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:26:19.196877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.787190+0000","last_change":"2024-05-03T18:22:21.557123+0000","last_active":"2024-05-03T18:27:11.787190+0000","last_peered":"2024-05-03T18:27:11.787190+0000","last_clean":"2024-05-03T18:27:11.787190+0000","last_became_active":"2024-05-03T18:22:21.556432+0000","last_became_peered":"2024-05-03T18:22:21.556432+0000","last_unstale":"2024-05-03T18:27:11.787190+0000","last_undegraded":"2024-05-03T18:27:11.787190+0000","last_fullsized":"2024-05-03T18:27:11.787190+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:50:03.179320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":123,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.741954+0000","last_change":"2024-05-03T18:20:24.937044+0000","last_active":"2024-05-03T18:27:11.741954+0000","last_peered":"2024-05-03T18:27:11.741954+0000","last_clean":"2024-05-03T18:27:11.741954+0000","last_became_active":"2024-05-03T18:20:24.922095+0000","last_became_peered":"2024-05-03T18:20:24.922095+0000","last_unstale":"2024-05-03T18:27:11.741954+0000","last_undegraded":"2024-05-03T18:27:11.741954+0000","last_fullsized":"2024-05-03T18:27:11.741954+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:15:37.353911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:15:37.353911+0000","last_clean_scrub_stamp":"2024-05-03T18:15:37.353911+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:40:07.341441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.780628+0000","last_change":"2024-05-03T18:23:20.575064+0000","last_active":"2024-05-03T18:27:11.780628+0000","last_peered":"2024-05-03T18:27:11.780628+0000","last_clean":"2024-05-03T18:27:11.780628+0000","last_became_active":"2024-05-03T18:23:20.574557+0000","last_became_peered":"2024-05-03T18:23:20.574557+0000","last_unstale":"2024-05-03T18:27:11.780628+0000","last_undegraded":"2024-05-03T18:27:11.780628+0000","last_fullsized":"2024-05-03T18:27:11.780628+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:34:08.520396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,6,10],"acting":[8,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.830147+0000","last_change":"2024-05-03T18:22:25.591269+0000","last_active":"2024-05-03T18:27:11.830147+0000","last_peered":"2024-05-03T18:27:11.830147+0000","last_clean":"2024-05-03T18:27:11.830147+0000","last_became_active":"2024-05-03T18:22:25.590409+0000","last_became_peered":"2024-05-03T18:22:25.590409+0000","last_unstale":"2024-05-03T18:27:11.830147+0000","last_undegraded":"2024-05-03T18:27:11.830147+0000","last_fullsized":"2024-05-03T18:27:11.830147+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:03:57.432331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.742525+0000","last_change":"2024-05-03T18:22:21.560137+0000","last_active":"2024-05-03T18:27:11.742525+0000","last_peered":"2024-05-03T18:27:11.742525+0000","last_clean":"2024-05-03T18:27:11.742525+0000","last_became_active":"2024-05-03T18:22:21.559672+0000","last_became_peered":"2024-05-03T18:22:21.559672+0000","last_unstale":"2024-05-03T18:27:11.742525+0000","last_undegraded":"2024-05-03T18:27:11.742525+0000","last_fullsized":"2024-05-03T18:27:11.742525+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:20:02.070390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.791935+0000","last_change":"2024-05-03T18:22:40.753160+0000","last_active":"2024-05-03T18:27:11.791935+0000","last_peered":"2024-05-03T18:27:11.791935+0000","last_clean":"2024-05-03T18:27:11.791935+0000","last_became_active":"2024-05-03T18:22:40.750800+0000","last_became_peered":"2024-05-03T18:22:40.750800+0000","last_unstale":"2024-05-03T18:27:11.791935+0000","last_undegraded":"2024-05-03T18:27:11.791935+0000","last_fullsized":"2024-05-03T18:27:11.791935+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:27:48.756140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.784686+0000","last_change":"2024-05-03T18:22:21.739490+0000","last_active":"2024-05-03T18:27:11.784686+0000","last_peered":"2024-05-03T18:27:11.784686+0000","last_clean":"2024-05-03T18:27:11.784686+0000","last_became_active":"2024-05-03T18:22:21.738512+0000","last_became_peered":"2024-05-03T18:22:21.738512+0000","last_unstale":"2024-05-03T18:27:11.784686+0000","last_undegraded":"2024-05-03T18:27:11.784686+0000","last_fullsized":"2024-05-03T18:27:11.784686+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:26:45.323519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.837222+0000","last_change":"2024-05-03T18:22:25.591509+0000","last_active":"2024-05-03T18:27:11.837222+0000","last_peered":"2024-05-03T18:27:11.837222+0000","last_clean":"2024-05-03T18:27:11.837222+0000","last_became_active":"2024-05-03T18:22:25.590188+0000","last_became_peered":"2024-05-03T18:22:25.590188+0000","last_unstale":"2024-05-03T18:27:11.837222+0000","last_undegraded":"2024-05-03T18:27:11.837222+0000","last_fullsized":"2024-05-03T18:27:11.837222+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:33:23.919101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.773559+0000","last_change":"2024-05-03T18:22:40.747176+0000","last_active":"2024-05-03T18:27:11.773559+0000","last_peered":"2024-05-03T18:27:11.773559+0000","last_clean":"2024-05-03T18:27:11.773559+0000","last_became_active":"2024-05-03T18:22:40.746526+0000","last_became_peered":"2024-05-03T18:22:40.746526+0000","last_unstale":"2024-05-03T18:27:11.773559+0000","last_undegraded":"2024-05-03T18:27:11.773559+0000","last_fullsized":"2024-05-03T18:27:11.773559+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:59:25.838382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'44","reported_seq":115,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:12.148271+0000","last_change":"2024-05-03T18:22:21.555978+0000","last_active":"2024-05-03T18:27:12.148271+0000","last_peered":"2024-05-03T18:27:12.148271+0000","last_clean":"2024-05-03T18:27:12.148271+0000","last_became_active":"2024-05-03T18:22:21.555395+0000","last_became_peered":"2024-05-03T18:22:21.555395+0000","last_unstale":"2024-05-03T18:27:12.148271+0000","last_undegraded":"2024-05-03T18:27:12.148271+0000","last_fullsized":"2024-05-03T18:27:12.148271+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:05:44.795978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":80,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.838632+0000","last_change":"2024-05-03T18:22:25.589071+0000","last_active":"2024-05-03T18:27:11.838632+0000","last_peered":"2024-05-03T18:27:11.838632+0000","last_clean":"2024-05-03T18:27:11.838632+0000","last_became_active":"2024-05-03T18:22:25.588679+0000","last_became_peered":"2024-05-03T18:22:25.588679+0000","last_unstale":"2024-05-03T18:27:11.838632+0000","last_undegraded":"2024-05-03T18:27:11.838632+0000","last_fullsized":"2024-05-03T18:27:11.838632+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:20:53.983166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.766671+0000","last_change":"2024-05-03T18:22:40.755154+0000","last_active":"2024-05-03T18:27:11.766671+0000","last_peered":"2024-05-03T18:27:11.766671+0000","last_clean":"2024-05-03T18:27:11.766671+0000","last_became_active":"2024-05-03T18:22:40.754635+0000","last_became_peered":"2024-05-03T18:22:40.754635+0000","last_unstale":"2024-05-03T18:27:11.766671+0000","last_undegraded":"2024-05-03T18:27:11.766671+0000","last_fullsized":"2024-05-03T18:27:11.766671+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:25:56.787406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"78'2","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.787950+0000","last_change":"2024-05-03T18:22:21.854322+0000","last_active":"2024-05-03T18:27:11.787950+0000","last_peered":"2024-05-03T18:27:11.787950+0000","last_clean":"2024-05-03T18:27:11.787950+0000","last_became_active":"2024-05-03T18:22:21.853359+0000","last_became_peered":"2024-05-03T18:22:21.853359+0000","last_unstale":"2024-05-03T18:27:11.787950+0000","last_undegraded":"2024-05-03T18:27:11.787950+0000","last_fullsized":"2024-05-03T18:27:11.787950+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:06:57.061775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.835206+0000","last_change":"2024-05-03T18:22:25.605614+0000","last_active":"2024-05-03T18:27:11.835206+0000","last_peered":"2024-05-03T18:27:11.835206+0000","last_clean":"2024-05-03T18:27:11.835206+0000","last_became_active":"2024-05-03T18:22:25.604277+0000","last_became_peered":"2024-05-03T18:22:25.604277+0000","last_unstale":"2024-05-03T18:27:11.835206+0000","last_undegraded":"2024-05-03T18:27:11.835206+0000","last_fullsized":"2024-05-03T18:27:11.835206+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:17:09.812304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.790222+0000","last_change":"2024-05-03T18:22:40.763353+0000","last_active":"2024-05-03T18:27:11.790222+0000","last_peered":"2024-05-03T18:27:11.790222+0000","last_clean":"2024-05-03T18:27:11.790222+0000","last_became_active":"2024-05-03T18:22:40.762774+0000","last_became_peered":"2024-05-03T18:22:40.762774+0000","last_unstale":"2024-05-03T18:27:11.790222+0000","last_undegraded":"2024-05-03T18:27:11.790222+0000","last_fullsized":"2024-05-03T18:27:11.790222+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:38:37.247050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.766499+0000","last_change":"2024-05-03T18:24:21.803989+0000","last_active":"2024-05-03T18:27:11.766499+0000","last_peered":"2024-05-03T18:27:11.766499+0000","last_clean":"2024-05-03T18:27:11.766499+0000","last_became_active":"2024-05-03T18:24:21.803072+0000","last_became_peered":"2024-05-03T18:24:21.803072+0000","last_unstale":"2024-05-03T18:27:11.766499+0000","last_undegraded":"2024-05-03T18:27:11.766499+0000","last_fullsized":"2024-05-03T18:27:11.766499+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:32:30.562148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.815005+0000","last_change":"2024-05-03T18:22:25.588011+0000","last_active":"2024-05-03T18:27:11.815005+0000","last_peered":"2024-05-03T18:27:11.815005+0000","last_clean":"2024-05-03T18:27:11.815005+0000","last_became_active":"2024-05-03T18:22:25.587602+0000","last_became_peered":"2024-05-03T18:22:25.587602+0000","last_unstale":"2024-05-03T18:27:11.815005+0000","last_undegraded":"2024-05-03T18:27:11.815005+0000","last_fullsized":"2024-05-03T18:27:11.815005+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:18:13.186088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.780962+0000","last_change":"2024-05-03T18:22:40.755731+0000","last_active":"2024-05-03T18:27:11.780962+0000","last_peered":"2024-05-03T18:27:11.780962+0000","last_clean":"2024-05-03T18:27:11.780962+0000","last_became_active":"2024-05-03T18:22:40.755173+0000","last_became_peered":"2024-05-03T18:22:40.755173+0000","last_unstale":"2024-05-03T18:27:11.780962+0000","last_undegraded":"2024-05-03T18:27:11.780962+0000","last_fullsized":"2024-05-03T18:27:11.780962+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:36:36.548095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.774547+0000","last_change":"2024-05-03T18:22:21.561481+0000","last_active":"2024-05-03T18:27:11.774547+0000","last_peered":"2024-05-03T18:27:11.774547+0000","last_clean":"2024-05-03T18:27:11.774547+0000","last_became_active":"2024-05-03T18:22:21.560088+0000","last_became_peered":"2024-05-03T18:22:21.560088+0000","last_unstale":"2024-05-03T18:27:11.774547+0000","last_undegraded":"2024-05-03T18:27:11.774547+0000","last_fullsized":"2024-05-03T18:27:11.774547+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:45:40.184813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.839958+0000","last_change":"2024-05-03T18:22:25.584556+0000","last_active":"2024-05-03T18:27:11.839958+0000","last_peered":"2024-05-03T18:27:11.839958+0000","last_clean":"2024-05-03T18:27:11.839958+0000","last_became_active":"2024-05-03T18:22:25.583873+0000","last_became_peered":"2024-05-03T18:22:25.583873+0000","last_unstale":"2024-05-03T18:27:11.839958+0000","last_undegraded":"2024-05-03T18:27:11.839958+0000","last_fullsized":"2024-05-03T18:27:11.839958+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:28:39.028662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.803445+0000","last_change":"2024-05-03T18:22:40.763479+0000","last_active":"2024-05-03T18:27:11.803445+0000","last_peered":"2024-05-03T18:27:11.803445+0000","last_clean":"2024-05-03T18:27:11.803445+0000","last_became_active":"2024-05-03T18:22:40.762897+0000","last_became_peered":"2024-05-03T18:22:40.762897+0000","last_unstale":"2024-05-03T18:27:11.803445+0000","last_undegraded":"2024-05-03T18:27:11.803445+0000","last_fullsized":"2024-05-03T18:27:11.803445+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:00:21.090730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.767323+0000","last_change":"2024-05-03T18:22:21.562929+0000","last_active":"2024-05-03T18:27:11.767323+0000","last_peered":"2024-05-03T18:27:11.767323+0000","last_clean":"2024-05-03T18:27:11.767323+0000","last_became_active":"2024-05-03T18:22:21.560712+0000","last_became_peered":"2024-05-03T18:22:21.560712+0000","last_unstale":"2024-05-03T18:27:11.767323+0000","last_undegraded":"2024-05-03T18:27:11.767323+0000","last_fullsized":"2024-05-03T18:27:11.767323+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:31:34.722437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.840750+0000","last_change":"2024-05-03T18:22:25.581971+0000","last_active":"2024-05-03T18:27:11.840750+0000","last_peered":"2024-05-03T18:27:11.840750+0000","last_clean":"2024-05-03T18:27:11.840750+0000","last_became_active":"2024-05-03T18:22:25.581565+0000","last_became_peered":"2024-05-03T18:22:25.581565+0000","last_unstale":"2024-05-03T18:27:11.840750+0000","last_undegraded":"2024-05-03T18:27:11.840750+0000","last_fullsized":"2024-05-03T18:27:11.840750+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:10:20.607999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.787659+0000","last_change":"2024-05-03T18:22:40.761458+0000","last_active":"2024-05-03T18:27:11.787659+0000","last_peered":"2024-05-03T18:27:11.787659+0000","last_clean":"2024-05-03T18:27:11.787659+0000","last_became_active":"2024-05-03T18:22:40.760969+0000","last_became_peered":"2024-05-03T18:22:40.760969+0000","last_unstale":"2024-05-03T18:27:11.787659+0000","last_undegraded":"2024-05-03T18:27:11.787659+0000","last_fullsized":"2024-05-03T18:27:11.787659+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:19:16.804720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"78'2","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.780499+0000","last_change":"2024-05-03T18:22:21.565869+0000","last_active":"2024-05-03T18:27:11.780499+0000","last_peered":"2024-05-03T18:27:11.780499+0000","last_clean":"2024-05-03T18:27:11.780499+0000","last_became_active":"2024-05-03T18:22:21.564119+0000","last_became_peered":"2024-05-03T18:22:21.564119+0000","last_unstale":"2024-05-03T18:27:11.780499+0000","last_undegraded":"2024-05-03T18:27:11.780499+0000","last_fullsized":"2024-05-03T18:27:11.780499+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:50:29.894070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.839023+0000","last_change":"2024-05-03T18:22:25.596199+0000","last_active":"2024-05-03T18:27:11.839023+0000","last_peered":"2024-05-03T18:27:11.839023+0000","last_clean":"2024-05-03T18:27:11.839023+0000","last_became_active":"2024-05-03T18:22:25.595167+0000","last_became_peered":"2024-05-03T18:22:25.595167+0000","last_unstale":"2024-05-03T18:27:11.839023+0000","last_undegraded":"2024-05-03T18:27:11.839023+0000","last_fullsized":"2024-05-03T18:27:11.839023+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:28:56.599064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.766667+0000","last_change":"2024-05-03T18:22:40.753020+0000","last_active":"2024-05-03T18:27:11.766667+0000","last_peered":"2024-05-03T18:27:11.766667+0000","last_clean":"2024-05-03T18:27:11.766667+0000","last_became_active":"2024-05-03T18:22:40.752502+0000","last_became_peered":"2024-05-03T18:22:40.752502+0000","last_unstale":"2024-05-03T18:27:11.766667+0000","last_undegraded":"2024-05-03T18:27:11.766667+0000","last_fullsized":"2024-05-03T18:27:11.766667+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:33:51.277879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.768258+0000","last_change":"2024-05-03T18:22:21.854481+0000","last_active":"2024-05-03T18:27:11.768258+0000","last_peered":"2024-05-03T18:27:11.768258+0000","last_clean":"2024-05-03T18:27:11.768258+0000","last_became_active":"2024-05-03T18:22:21.853172+0000","last_became_peered":"2024-05-03T18:22:21.853172+0000","last_unstale":"2024-05-03T18:27:11.768258+0000","last_undegraded":"2024-05-03T18:27:11.768258+0000","last_fullsized":"2024-05-03T18:27:11.768258+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:11:47.820105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.837912+0000","last_change":"2024-05-03T18:22:25.600856+0000","last_active":"2024-05-03T18:27:11.837912+0000","last_peered":"2024-05-03T18:27:11.837912+0000","last_clean":"2024-05-03T18:27:11.837912+0000","last_became_active":"2024-05-03T18:22:25.595351+0000","last_became_peered":"2024-05-03T18:22:25.595351+0000","last_unstale":"2024-05-03T18:27:11.837912+0000","last_undegraded":"2024-05-03T18:27:11.837912+0000","last_fullsized":"2024-05-03T18:27:11.837912+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:29:24.232506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.773733+0000","last_change":"2024-05-03T18:23:20.578331+0000","last_active":"2024-05-03T18:27:11.773733+0000","last_peered":"2024-05-03T18:27:11.773733+0000","last_clean":"2024-05-03T18:27:11.773733+0000","last_became_active":"2024-05-03T18:23:20.576974+0000","last_became_peered":"2024-05-03T18:23:20.576974+0000","last_unstale":"2024-05-03T18:27:11.773733+0000","last_undegraded":"2024-05-03T18:27:11.773733+0000","last_fullsized":"2024-05-03T18:27:11.773733+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:57:32.922175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"90'39","reported_seq":95,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:12.135624+0000","last_change":"2024-05-03T18:22:21.552331+0000","last_active":"2024-05-03T18:27:12.135624+0000","last_peered":"2024-05-03T18:27:12.135624+0000","last_clean":"2024-05-03T18:27:12.135624+0000","last_became_active":"2024-05-03T18:22:21.549786+0000","last_became_peered":"2024-05-03T18:22:21.549786+0000","last_unstale":"2024-05-03T18:27:12.135624+0000","last_undegraded":"2024-05-03T18:27:12.135624+0000","last_fullsized":"2024-05-03T18:27:12.135624+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:08:18.513405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"90'6","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.837679+0000","last_change":"2024-05-03T18:22:25.596437+0000","last_active":"2024-05-03T18:27:11.837679+0000","last_peered":"2024-05-03T18:27:11.837679+0000","last_clean":"2024-05-03T18:27:11.837679+0000","last_became_active":"2024-05-03T18:22:25.594040+0000","last_became_peered":"2024-05-03T18:22:25.594040+0000","last_unstale":"2024-05-03T18:27:11.837679+0000","last_undegraded":"2024-05-03T18:27:11.837679+0000","last_fullsized":"2024-05-03T18:27:11.837679+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:09:36.473565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.741898+0000","last_change":"2024-05-03T18:22:40.754641+0000","last_active":"2024-05-03T18:27:11.741898+0000","last_peered":"2024-05-03T18:27:11.741898+0000","last_clean":"2024-05-03T18:27:11.741898+0000","last_became_active":"2024-05-03T18:22:40.754166+0000","last_became_peered":"2024-05-03T18:22:40.754166+0000","last_unstale":"2024-05-03T18:27:11.741898+0000","last_undegraded":"2024-05-03T18:27:11.741898+0000","last_fullsized":"2024-05-03T18:27:11.741898+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:29:50.021671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.741202+0000","last_change":"2024-05-03T18:22:21.559354+0000","last_active":"2024-05-03T18:27:11.741202+0000","last_peered":"2024-05-03T18:27:11.741202+0000","last_clean":"2024-05-03T18:27:11.741202+0000","last_became_active":"2024-05-03T18:22:21.558680+0000","last_became_peered":"2024-05-03T18:22:21.558680+0000","last_unstale":"2024-05-03T18:27:11.741202+0000","last_undegraded":"2024-05-03T18:27:11.741202+0000","last_fullsized":"2024-05-03T18:27:11.741202+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:13:04.001847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.837002+0000","last_change":"2024-05-03T18:22:25.590746+0000","last_active":"2024-05-03T18:27:11.837002+0000","last_peered":"2024-05-03T18:27:11.837002+0000","last_clean":"2024-05-03T18:27:11.837002+0000","last_became_active":"2024-05-03T18:22:25.587906+0000","last_became_peered":"2024-05-03T18:22:25.587906+0000","last_unstale":"2024-05-03T18:27:11.837002+0000","last_undegraded":"2024-05-03T18:27:11.837002+0000","last_fullsized":"2024-05-03T18:27:11.837002+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:19:43.410259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.833036+0000","last_change":"2024-05-03T18:22:25.597628+0000","last_active":"2024-05-03T18:27:11.833036+0000","last_peered":"2024-05-03T18:27:11.833036+0000","last_clean":"2024-05-03T18:27:11.833036+0000","last_became_active":"2024-05-03T18:22:25.596535+0000","last_became_peered":"2024-05-03T18:22:25.596535+0000","last_unstale":"2024-05-03T18:27:11.833036+0000","last_undegraded":"2024-05-03T18:27:11.833036+0000","last_fullsized":"2024-05-03T18:27:11.833036+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:54:52.632258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.829560+0000","last_change":"2024-05-03T18:22:25.601474+0000","last_active":"2024-05-03T18:27:11.829560+0000","last_peered":"2024-05-03T18:27:11.829560+0000","last_clean":"2024-05-03T18:27:11.829560+0000","last_became_active":"2024-05-03T18:22:25.601097+0000","last_became_peered":"2024-05-03T18:22:25.601097+0000","last_unstale":"2024-05-03T18:27:11.829560+0000","last_undegraded":"2024-05-03T18:27:11.829560+0000","last_fullsized":"2024-05-03T18:27:11.829560+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:14:46.001500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":56,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.820221+0000","last_change":"2024-05-03T18:23:20.535409+0000","last_active":"2024-05-03T18:27:11.820221+0000","last_peered":"2024-05-03T18:27:11.820221+0000","last_clean":"2024-05-03T18:27:11.820221+0000","last_became_active":"2024-05-03T18:23:20.534883+0000","last_became_peered":"2024-05-03T18:23:20.534883+0000","last_unstale":"2024-05-03T18:27:11.820221+0000","last_undegraded":"2024-05-03T18:27:11.820221+0000","last_fullsized":"2024-05-03T18:27:11.820221+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:26:37.252898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.835637+0000","last_change":"2024-05-03T18:22:25.607115+0000","last_active":"2024-05-03T18:27:11.835637+0000","last_peered":"2024-05-03T18:27:11.835637+0000","last_clean":"2024-05-03T18:27:11.835637+0000","last_became_active":"2024-05-03T18:22:25.606764+0000","last_became_peered":"2024-05-03T18:22:25.606764+0000","last_unstale":"2024-05-03T18:27:11.835637+0000","last_undegraded":"2024-05-03T18:27:11.835637+0000","last_fullsized":"2024-05-03T18:27:11.835637+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:44:37.165032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.842362+0000","last_change":"2024-05-03T18:22:25.601886+0000","last_active":"2024-05-03T18:27:11.842362+0000","last_peered":"2024-05-03T18:27:11.842362+0000","last_clean":"2024-05-03T18:27:11.842362+0000","last_became_active":"2024-05-03T18:22:25.601494+0000","last_became_peered":"2024-05-03T18:22:25.601494+0000","last_unstale":"2024-05-03T18:27:11.842362+0000","last_undegraded":"2024-05-03T18:27:11.842362+0000","last_fullsized":"2024-05-03T18:27:11.842362+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:35:30.080124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.832632+0000","last_change":"2024-05-03T18:23:20.529091+0000","last_active":"2024-05-03T18:27:11.832632+0000","last_peered":"2024-05-03T18:27:11.832632+0000","last_clean":"2024-05-03T18:27:11.832632+0000","last_became_active":"2024-05-03T18:23:20.528288+0000","last_became_peered":"2024-05-03T18:23:20.528288+0000","last_unstale":"2024-05-03T18:27:11.832632+0000","last_undegraded":"2024-05-03T18:27:11.832632+0000","last_fullsized":"2024-05-03T18:27:11.832632+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:32:25.749461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":57,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.823878+0000","last_change":"2024-05-03T18:22:25.595367+0000","last_active":"2024-05-03T18:27:11.823878+0000","last_peered":"2024-05-03T18:27:11.823878+0000","last_clean":"2024-05-03T18:27:11.823878+0000","last_became_active":"2024-05-03T18:22:25.594987+0000","last_became_peered":"2024-05-03T18:22:25.594987+0000","last_unstale":"2024-05-03T18:27:11.823878+0000","last_undegraded":"2024-05-03T18:27:11.823878+0000","last_fullsized":"2024-05-03T18:27:11.823878+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:40:55.385595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.823841+0000","last_change":"2024-05-03T18:22:25.588320+0000","last_active":"2024-05-03T18:27:11.823841+0000","last_peered":"2024-05-03T18:27:11.823841+0000","last_clean":"2024-05-03T18:27:11.823841+0000","last_became_active":"2024-05-03T18:22:25.587861+0000","last_became_peered":"2024-05-03T18:22:25.587861+0000","last_unstale":"2024-05-03T18:27:11.823841+0000","last_undegraded":"2024-05-03T18:27:11.823841+0000","last_fullsized":"2024-05-03T18:27:11.823841+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:17:02.538798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.831427+0000","last_change":"2024-05-03T18:22:25.597874+0000","last_active":"2024-05-03T18:27:11.831427+0000","last_peered":"2024-05-03T18:27:11.831427+0000","last_clean":"2024-05-03T18:27:11.831427+0000","last_became_active":"2024-05-03T18:22:25.597448+0000","last_became_peered":"2024-05-03T18:22:25.597448+0000","last_unstale":"2024-05-03T18:27:11.831427+0000","last_undegraded":"2024-05-03T18:27:11.831427+0000","last_fullsized":"2024-05-03T18:27:11.831427+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:55:52.894491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.836134+0000","last_change":"2024-05-03T18:22:25.593509+0000","last_active":"2024-05-03T18:27:11.836134+0000","last_peered":"2024-05-03T18:27:11.836134+0000","last_clean":"2024-05-03T18:27:11.836134+0000","last_became_active":"2024-05-03T18:22:25.590736+0000","last_became_peered":"2024-05-03T18:22:25.590736+0000","last_unstale":"2024-05-03T18:27:11.836134+0000","last_undegraded":"2024-05-03T18:27:11.836134+0000","last_fullsized":"2024-05-03T18:27:11.836134+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:54:45.431776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.837344+0000","last_change":"2024-05-03T18:22:25.590022+0000","last_active":"2024-05-03T18:27:11.837344+0000","last_peered":"2024-05-03T18:27:11.837344+0000","last_clean":"2024-05-03T18:27:11.837344+0000","last_became_active":"2024-05-03T18:22:25.589432+0000","last_became_peered":"2024-05-03T18:22:25.589432+0000","last_unstale":"2024-05-03T18:27:11.837344+0000","last_undegraded":"2024-05-03T18:27:11.837344+0000","last_fullsized":"2024-05-03T18:27:11.837344+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:03:02.091228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":53,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.841863+0000","last_change":"2024-05-03T18:22:25.581452+0000","last_active":"2024-05-03T18:27:11.841863+0000","last_peered":"2024-05-03T18:27:11.841863+0000","last_clean":"2024-05-03T18:27:11.841863+0000","last_became_active":"2024-05-03T18:22:25.581098+0000","last_became_peered":"2024-05-03T18:22:25.581098+0000","last_unstale":"2024-05-03T18:27:11.841863+0000","last_undegraded":"2024-05-03T18:27:11.841863+0000","last_fullsized":"2024-05-03T18:27:11.841863+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:04:46.551234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.823559+0000","last_change":"2024-05-03T18:22:25.589105+0000","last_active":"2024-05-03T18:27:11.823559+0000","last_peered":"2024-05-03T18:27:11.823559+0000","last_clean":"2024-05-03T18:27:11.823559+0000","last_became_active":"2024-05-03T18:22:25.588697+0000","last_became_peered":"2024-05-03T18:22:25.588697+0000","last_unstale":"2024-05-03T18:27:11.823559+0000","last_undegraded":"2024-05-03T18:27:11.823559+0000","last_fullsized":"2024-05-03T18:27:11.823559+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:41:23.563296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":594853,"num_objects":24,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":523,"num_read_kb":366,"num_write":203,"num_write_kb":1410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1880064,"data_stored":1784559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":151,"ondisk_log_size":151,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710797,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1036288,"data_stored":524266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":10,"up_from":64,"seq":274877907027,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28884,"kb_used_data":1808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716268,"statfs":{"total":95995035648,"available":95965458432,"internally_reserved":0,"allocated":1851392,"data_stored":1311571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070554,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36744,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708408,"statfs":{"total":95995035648,"available":95957409792,"internally_reserved":0,"allocated":1437696,"data_stored":922160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266785,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1044480,"data_stored":524870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":47,"seq":201863463018,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32780,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712372,"statfs":{"total":95995035648,"available":95961468928,"internally_reserved":0,"allocated":1638400,"data_stored":1119312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626545,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1032192,"data_stored":520170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574996,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32264,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712888,"statfs":{"total":95995035648,"available":95961997312,"internally_reserved":0,"allocated":1110016,"data_stored":588627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738523,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1036288,"data_stored":520260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411470,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32268,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712884,"statfs":{"total":95995035648,"available":95961993216,"internally_reserved":0,"allocated":1114112,"data_stored":588661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182532,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28548,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716604,"statfs":{"total":95995035648,"available":95965802496,"internally_reserved":0,"allocated":1568768,"data_stored":1045090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986302,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1040384,"data_stored":524288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822775,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32912,"kb_used_data":1736,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712240,"statfs":{"total":95995035648,"available":95961333760,"internally_reserved":0,"allocated":1777664,"data_stored":1246416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":69239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":69273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":536576,"data_stored":524920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":131782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":393820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":397312,"data_stored":393798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":225280,"data_stored":201033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-03T18:27:17.570 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-05-03T18:27:17.570 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-03T18:27:17.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:17 smithi099 ceph-mon[31658]: mgrmap e23: x(active, since 4s), standbys: z, y 2024-05-03T18:27:17.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.811+0000 7fcda5200640 1 -- 172.21.15.33:0/444628914 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcda00769f0 msgr2=0x7fcda0074e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:17.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.811+0000 7fcda5200640 1 --2- 172.21.15.33:0/444628914 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcda00769f0 0x7fcda0074e40 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fcd90009900 tx=0x7fcd9002a410 comp rx=0 tx=0).stop 2024-05-03T18:27:17.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.811+0000 7fcda5200640 1 -- 172.21.15.33:0/444628914 shutdown_connections 2024-05-03T18:27:17.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.811+0000 7fcda5200640 1 --2- 172.21.15.33:0/444628914 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcda010a710 0x7fcda010cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:17.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.811+0000 7fcda5200640 1 --2- 172.21.15.33:0/444628914 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcda0075410 0x7fcda0075890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:17.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.811+0000 7fcda5200640 1 --2- 172.21.15.33:0/444628914 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcda00769f0 0x7fcda0074e40 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:17.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.811+0000 7fcda5200640 1 -- 172.21.15.33:0/444628914 >> 172.21.15.33:0/444628914 conn(0x7fcda00fddc0 msgr2=0x7fcda0100200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:27:17.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.811+0000 7fcda5200640 1 -- 172.21.15.33:0/444628914 shutdown_connections 2024-05-03T18:27:17.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.811+0000 7fcda5200640 1 -- 172.21.15.33:0/444628914 wait complete. 2024-05-03T18:27:17.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.812+0000 7fcda5200640 1 Processor -- start 2024-05-03T18:27:17.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.813+0000 7fcda5200640 1 -- start start 2024-05-03T18:27:17.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.813+0000 7fcda5200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcda0075410 0x7fcda019abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:17.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.813+0000 7fcda5200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcda00769f0 0x7fcda019b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:17.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.813+0000 7fcda5200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcda010a710 0x7fcda01a2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:17.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.813+0000 7fcda5200640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcda0110460 con 0x7fcda010a710 2024-05-03T18:27:17.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.813+0000 7fcda5200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fcda01102e0 con 0x7fcda00769f0 2024-05-03T18:27:17.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.813+0000 7fcda5200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fcda01105e0 con 0x7fcda0075410 2024-05-03T18:27:17.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.813+0000 7fcd9fe00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcda010a710 0x7fcda01a2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:17.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.813+0000 7fcd9fe00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcda010a710 0x7fcda01a2160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50768/0 (socket says 172.21.15.33:50768) 2024-05-03T18:27:17.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.813+0000 7fcd9fe00640 1 -- 172.21.15.33:0/138711123 learned_addr learned my addr 172.21.15.33:0/138711123 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:27:17.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.814+0000 7fcd9f400640 1 --2- 172.21.15.33:0/138711123 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcda0075410 0x7fcda019abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:17.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.814+0000 7fcd9ea00640 1 --2- 172.21.15.33:0/138711123 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcda00769f0 0x7fcda019b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:17.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.814+0000 7fcd9fe00640 1 -- 172.21.15.33:0/138711123 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcda0075410 msgr2=0x7fcda019abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:17.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.814+0000 7fcd9fe00640 1 --2- 172.21.15.33:0/138711123 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcda0075410 0x7fcda019abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:17.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.814+0000 7fcd9fe00640 1 -- 172.21.15.33:0/138711123 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcda00769f0 msgr2=0x7fcda019b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:17.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.814+0000 7fcd9fe00640 1 --2- 172.21.15.33:0/138711123 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcda00769f0 0x7fcda019b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:17.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.814+0000 7fcd9fe00640 1 -- 172.21.15.33:0/138711123 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcda01a27d0 con 0x7fcda010a710 2024-05-03T18:27:17.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.814+0000 7fcd9ea00640 1 --2- 172.21.15.33:0/138711123 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcda00769f0 0x7fcda019b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:27:17.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.814+0000 7fcd9fe00640 1 --2- 172.21.15.33:0/138711123 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcda010a710 0x7fcda01a2160 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fcd9800eaa0 tx=0x7fcd9800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:27:17.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.815+0000 7fcd97e00640 1 -- 172.21.15.33:0/138711123 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd9800cdb0 con 0x7fcda010a710 2024-05-03T18:27:17.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.815+0000 7fcd97e00640 1 -- 172.21.15.33:0/138711123 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcd98005020 con 0x7fcda010a710 2024-05-03T18:27:17.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.815+0000 7fcda5200640 1 -- 172.21.15.33:0/138711123 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcda01a2ac0 con 0x7fcda010a710 2024-05-03T18:27:17.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.815+0000 7fcd97e00640 1 -- 172.21.15.33:0/138711123 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd980116d0 con 0x7fcda010a710 2024-05-03T18:27:17.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.815+0000 7fcda5200640 1 -- 172.21.15.33:0/138711123 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcda01a3060 con 0x7fcda010a710 2024-05-03T18:27:17.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.816+0000 7fcda5200640 1 -- 172.21.15.33:0/138711123 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcda0109440 con 0x7fcda010a710 2024-05-03T18:27:17.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.818+0000 7fcd97e00640 1 -- 172.21.15.33:0/138711123 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 23) ==== 149828+0+0 (secure 0 0 0) 0x7fcd980051d0 con 0x7fcda010a710 2024-05-03T18:27:17.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.820+0000 7fcd97e00640 1 --2- 172.21.15.33:0/138711123 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fcd840b2a70 0x7fcd840b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:17.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.820+0000 7fcd97e00640 1 -- 172.21.15.33:0/138711123 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7fcd980e2dc0 con 0x7fcda010a710 2024-05-03T18:27:17.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.820+0000 7fcd9f400640 1 --2- 172.21.15.33:0/138711123 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fcd840b2a70 0x7fcd840b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:17.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.821+0000 7fcd9f400640 1 --2- 172.21.15.33:0/138711123 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fcd840b2a70 0x7fcd840b4f30 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fcd90006010 tx=0x7fcd90031400 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:27:17.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:17.825+0000 7fcd97e00640 1 -- 172.21.15.33:0/138711123 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fcd98016030 con 0x7fcda010a710 2024-05-03T18:27:18.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:18.033+0000 7fcda5200640 1 -- 172.21.15.33:0/138711123 --> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fcda0062df0 con 0x7fcd840b2a70 2024-05-03T18:27:18.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:18.046+0000 7fcd97e00640 1 -- 172.21.15.33:0/138711123 <== mgr.15120 v2:172.21.15.33:6800/3302011241 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339440 (secure 0 0 0) 0x7fcda0062df0 con 0x7fcd840b2a70 2024-05-03T18:27:18.051 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:27:18.052 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-05-03T18:27:18.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:18.052+0000 7fcda5200640 1 -- 172.21.15.33:0/138711123 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fcd840b2a70 msgr2=0x7fcd840b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:18.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:18.052+0000 7fcda5200640 1 --2- 172.21.15.33:0/138711123 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fcd840b2a70 0x7fcd840b4f30 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fcd90006010 tx=0x7fcd90031400 comp rx=0 tx=0).stop 2024-05-03T18:27:18.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:18.052+0000 7fcda5200640 1 -- 172.21.15.33:0/138711123 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcda010a710 msgr2=0x7fcda01a2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:18.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:18.052+0000 7fcda5200640 1 --2- 172.21.15.33:0/138711123 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcda010a710 0x7fcda01a2160 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fcd9800eaa0 tx=0x7fcd9800ef60 comp rx=0 tx=0).stop 2024-05-03T18:27:18.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:18.053+0000 7fcda5200640 1 -- 172.21.15.33:0/138711123 shutdown_connections 2024-05-03T18:27:18.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:18.053+0000 7fcda5200640 1 --2- 172.21.15.33:0/138711123 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fcd840b2a70 0x7fcd840b4f30 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:18.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:18.053+0000 7fcda5200640 1 --2- 172.21.15.33:0/138711123 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcda010a710 0x7fcda01a2160 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:18.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:18.053+0000 7fcda5200640 1 --2- 172.21.15.33:0/138711123 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcda00769f0 0x7fcda019b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:18.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:18.053+0000 7fcda5200640 1 --2- 172.21.15.33:0/138711123 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcda0075410 0x7fcda019abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:18.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:18.053+0000 7fcda5200640 1 -- 172.21.15.33:0/138711123 >> 172.21.15.33:0/138711123 conn(0x7fcda00fddc0 msgr2=0x7fcda0106610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:27:18.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:18.053+0000 7fcda5200640 1 -- 172.21.15.33:0/138711123 shutdown_connections 2024-05-03T18:27:18.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:18.053+0000 7fcda5200640 1 -- 172.21.15.33:0/138711123 wait complete. 2024-05-03T18:27:18.073 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":6,"stamp":"2024-05-03T18:27:17.767116+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1185455,"num_objects":28,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":606,"num_read_kb":435,"num_write":328,"num_write_kb":3562,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":745,"ondisk_log_size":745,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":368728,"kb_used_data":15320,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124573096,"statfs":{"total":1151940427776,"available":1151562850304,"internally_reserved":0,"allocated":15687680,"data_stored":9435691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":361674042},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":54,"apply_latency_ms":54,"commit_latency_ns":54000000,"apply_latency_ns":54000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.981066"},"pg_stats":[{"pgid":"3.3f","version":"90'8","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.845032+0000","last_change":"2024-05-03T18:22:25.595640+0000","last_active":"2024-05-03T18:27:11.845032+0000","last_peered":"2024-05-03T18:27:11.845032+0000","last_clean":"2024-05-03T18:27:11.845032+0000","last_became_active":"2024-05-03T18:22:25.595284+0000","last_became_peered":"2024-05-03T18:22:25.595284+0000","last_unstale":"2024-05-03T18:27:11.845032+0000","last_undegraded":"2024-05-03T18:27:11.845032+0000","last_fullsized":"2024-05-03T18:27:11.845032+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:50:08.867670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.835095+0000","last_change":"2024-05-03T18:22:25.584950+0000","last_active":"2024-05-03T18:27:11.835095+0000","last_peered":"2024-05-03T18:27:11.835095+0000","last_clean":"2024-05-03T18:27:11.835095+0000","last_became_active":"2024-05-03T18:22:25.584613+0000","last_became_peered":"2024-05-03T18:22:25.584613+0000","last_unstale":"2024-05-03T18:27:11.835095+0000","last_undegraded":"2024-05-03T18:27:11.835095+0000","last_fullsized":"2024-05-03T18:27:11.835095+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:00:24.959429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.834333+0000","last_change":"2024-05-03T18:22:25.596799+0000","last_active":"2024-05-03T18:27:11.834333+0000","last_peered":"2024-05-03T18:27:11.834333+0000","last_clean":"2024-05-03T18:27:11.834333+0000","last_became_active":"2024-05-03T18:22:25.596386+0000","last_became_peered":"2024-05-03T18:22:25.596386+0000","last_unstale":"2024-05-03T18:27:11.834333+0000","last_undegraded":"2024-05-03T18:27:11.834333+0000","last_fullsized":"2024-05-03T18:27:11.834333+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:20:07.818282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.830961+0000","last_change":"2024-05-03T18:23:20.534672+0000","last_active":"2024-05-03T18:27:11.830961+0000","last_peered":"2024-05-03T18:27:11.830961+0000","last_clean":"2024-05-03T18:27:11.830961+0000","last_became_active":"2024-05-03T18:23:20.534073+0000","last_became_peered":"2024-05-03T18:23:20.534073+0000","last_unstale":"2024-05-03T18:27:11.830961+0000","last_undegraded":"2024-05-03T18:27:11.830961+0000","last_fullsized":"2024-05-03T18:27:11.830961+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:07:40.204299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.827187+0000","last_change":"2024-05-03T18:22:25.589810+0000","last_active":"2024-05-03T18:27:11.827187+0000","last_peered":"2024-05-03T18:27:11.827187+0000","last_clean":"2024-05-03T18:27:11.827187+0000","last_became_active":"2024-05-03T18:22:25.588220+0000","last_became_peered":"2024-05-03T18:22:25.588220+0000","last_unstale":"2024-05-03T18:27:11.827187+0000","last_undegraded":"2024-05-03T18:27:11.827187+0000","last_fullsized":"2024-05-03T18:27:11.827187+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:43:20.904737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.834092+0000","last_change":"2024-05-03T18:22:25.603148+0000","last_active":"2024-05-03T18:27:11.834092+0000","last_peered":"2024-05-03T18:27:11.834092+0000","last_clean":"2024-05-03T18:27:11.834092+0000","last_became_active":"2024-05-03T18:22:25.602743+0000","last_became_peered":"2024-05-03T18:22:25.602743+0000","last_unstale":"2024-05-03T18:27:11.834092+0000","last_undegraded":"2024-05-03T18:27:11.834092+0000","last_fullsized":"2024-05-03T18:27:11.834092+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:49:27.387863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.837215+0000","last_change":"2024-05-03T18:23:20.532077+0000","last_active":"2024-05-03T18:27:11.837215+0000","last_peered":"2024-05-03T18:27:11.837215+0000","last_clean":"2024-05-03T18:27:11.837215+0000","last_became_active":"2024-05-03T18:23:20.531528+0000","last_became_peered":"2024-05-03T18:23:20.531528+0000","last_unstale":"2024-05-03T18:27:11.837215+0000","last_undegraded":"2024-05-03T18:27:11.837215+0000","last_fullsized":"2024-05-03T18:27:11.837215+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:10:37.037114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.835717+0000","last_change":"2024-05-03T18:22:25.600192+0000","last_active":"2024-05-03T18:27:11.835717+0000","last_peered":"2024-05-03T18:27:11.835717+0000","last_clean":"2024-05-03T18:27:11.835717+0000","last_became_active":"2024-05-03T18:22:25.599663+0000","last_became_peered":"2024-05-03T18:22:25.599663+0000","last_unstale":"2024-05-03T18:27:11.835717+0000","last_undegraded":"2024-05-03T18:27:11.835717+0000","last_fullsized":"2024-05-03T18:27:11.835717+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:09:51.614784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.833605+0000","last_change":"2024-05-03T18:22:25.600302+0000","last_active":"2024-05-03T18:27:11.833605+0000","last_peered":"2024-05-03T18:27:11.833605+0000","last_clean":"2024-05-03T18:27:11.833605+0000","last_became_active":"2024-05-03T18:22:25.599832+0000","last_became_peered":"2024-05-03T18:22:25.599832+0000","last_unstale":"2024-05-03T18:27:11.833605+0000","last_undegraded":"2024-05-03T18:27:11.833605+0000","last_fullsized":"2024-05-03T18:27:11.833605+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:52:12.326156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":57,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.830746+0000","last_change":"2024-05-03T18:22:25.590892+0000","last_active":"2024-05-03T18:27:11.830746+0000","last_peered":"2024-05-03T18:27:11.830746+0000","last_clean":"2024-05-03T18:27:11.830746+0000","last_became_active":"2024-05-03T18:22:25.590516+0000","last_became_peered":"2024-05-03T18:22:25.590516+0000","last_unstale":"2024-05-03T18:27:11.830746+0000","last_undegraded":"2024-05-03T18:27:11.830746+0000","last_fullsized":"2024-05-03T18:27:11.830746+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:57:57.999234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.837902+0000","last_change":"2024-05-03T18:22:25.597307+0000","last_active":"2024-05-03T18:27:11.837902+0000","last_peered":"2024-05-03T18:27:11.837902+0000","last_clean":"2024-05-03T18:27:11.837902+0000","last_became_active":"2024-05-03T18:22:25.596989+0000","last_became_peered":"2024-05-03T18:22:25.596989+0000","last_unstale":"2024-05-03T18:27:11.837902+0000","last_undegraded":"2024-05-03T18:27:11.837902+0000","last_fullsized":"2024-05-03T18:27:11.837902+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:53:09.003353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.831580+0000","last_change":"2024-05-03T18:22:25.601723+0000","last_active":"2024-05-03T18:27:11.831580+0000","last_peered":"2024-05-03T18:27:11.831580+0000","last_clean":"2024-05-03T18:27:11.831580+0000","last_became_active":"2024-05-03T18:22:25.599428+0000","last_became_peered":"2024-05-03T18:22:25.599428+0000","last_unstale":"2024-05-03T18:27:11.831580+0000","last_undegraded":"2024-05-03T18:27:11.831580+0000","last_fullsized":"2024-05-03T18:27:11.831580+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:41:18.584497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.838263+0000","last_change":"2024-05-03T18:22:25.594885+0000","last_active":"2024-05-03T18:27:11.838263+0000","last_peered":"2024-05-03T18:27:11.838263+0000","last_clean":"2024-05-03T18:27:11.838263+0000","last_became_active":"2024-05-03T18:22:25.593939+0000","last_became_peered":"2024-05-03T18:22:25.593939+0000","last_unstale":"2024-05-03T18:27:11.838263+0000","last_undegraded":"2024-05-03T18:27:11.838263+0000","last_fullsized":"2024-05-03T18:27:11.838263+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:03:30.052886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.837979+0000","last_change":"2024-05-03T18:22:25.595228+0000","last_active":"2024-05-03T18:27:11.837979+0000","last_peered":"2024-05-03T18:27:11.837979+0000","last_clean":"2024-05-03T18:27:11.837979+0000","last_became_active":"2024-05-03T18:22:25.589825+0000","last_became_peered":"2024-05-03T18:22:25.589825+0000","last_unstale":"2024-05-03T18:27:11.837979+0000","last_undegraded":"2024-05-03T18:27:11.837979+0000","last_fullsized":"2024-05-03T18:27:11.837979+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:44:05.638939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.835425+0000","last_change":"2024-05-03T18:22:25.595630+0000","last_active":"2024-05-03T18:27:11.835425+0000","last_peered":"2024-05-03T18:27:11.835425+0000","last_clean":"2024-05-03T18:27:11.835425+0000","last_became_active":"2024-05-03T18:22:25.591869+0000","last_became_peered":"2024-05-03T18:22:25.591869+0000","last_unstale":"2024-05-03T18:27:11.835425+0000","last_undegraded":"2024-05-03T18:27:11.835425+0000","last_fullsized":"2024-05-03T18:27:11.835425+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:27:43.579329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.841007+0000","last_change":"2024-05-03T18:22:25.589482+0000","last_active":"2024-05-03T18:27:11.841007+0000","last_peered":"2024-05-03T18:27:11.841007+0000","last_clean":"2024-05-03T18:27:11.841007+0000","last_became_active":"2024-05-03T18:22:25.589142+0000","last_became_peered":"2024-05-03T18:22:25.589142+0000","last_unstale":"2024-05-03T18:27:11.841007+0000","last_undegraded":"2024-05-03T18:27:11.841007+0000","last_fullsized":"2024-05-03T18:27:11.841007+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:37:55.472823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.845642+0000","last_change":"2024-05-03T18:22:25.594569+0000","last_active":"2024-05-03T18:27:11.845642+0000","last_peered":"2024-05-03T18:27:11.845642+0000","last_clean":"2024-05-03T18:27:11.845642+0000","last_became_active":"2024-05-03T18:22:25.594218+0000","last_became_peered":"2024-05-03T18:22:25.594218+0000","last_unstale":"2024-05-03T18:27:11.845642+0000","last_undegraded":"2024-05-03T18:27:11.845642+0000","last_fullsized":"2024-05-03T18:27:11.845642+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:30:42.775673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.835918+0000","last_change":"2024-05-03T18:22:25.597426+0000","last_active":"2024-05-03T18:27:11.835918+0000","last_peered":"2024-05-03T18:27:11.835918+0000","last_clean":"2024-05-03T18:27:11.835918+0000","last_became_active":"2024-05-03T18:22:25.594524+0000","last_became_peered":"2024-05-03T18:22:25.594524+0000","last_unstale":"2024-05-03T18:27:11.835918+0000","last_undegraded":"2024-05-03T18:27:11.835918+0000","last_fullsized":"2024-05-03T18:27:11.835918+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:41:36.984253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.826557+0000","last_change":"2024-05-03T18:22:25.588002+0000","last_active":"2024-05-03T18:27:11.826557+0000","last_peered":"2024-05-03T18:27:11.826557+0000","last_clean":"2024-05-03T18:27:11.826557+0000","last_became_active":"2024-05-03T18:22:25.587547+0000","last_became_peered":"2024-05-03T18:22:25.587547+0000","last_unstale":"2024-05-03T18:27:11.826557+0000","last_undegraded":"2024-05-03T18:27:11.826557+0000","last_fullsized":"2024-05-03T18:27:11.826557+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:47:14.115517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.767050+0000","last_change":"2024-05-03T18:22:40.754602+0000","last_active":"2024-05-03T18:27:11.767050+0000","last_peered":"2024-05-03T18:27:11.767050+0000","last_clean":"2024-05-03T18:27:11.767050+0000","last_became_active":"2024-05-03T18:22:40.752917+0000","last_became_peered":"2024-05-03T18:22:40.752917+0000","last_unstale":"2024-05-03T18:27:11.767050+0000","last_undegraded":"2024-05-03T18:27:11.767050+0000","last_fullsized":"2024-05-03T18:27:11.767050+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:51:44.711811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"78'4","reported_seq":53,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.781114+0000","last_change":"2024-05-03T18:22:21.558560+0000","last_active":"2024-05-03T18:27:11.781114+0000","last_peered":"2024-05-03T18:27:11.781114+0000","last_clean":"2024-05-03T18:27:11.781114+0000","last_became_active":"2024-05-03T18:22:21.558185+0000","last_became_peered":"2024-05-03T18:22:21.558185+0000","last_unstale":"2024-05-03T18:27:11.781114+0000","last_undegraded":"2024-05-03T18:27:11.781114+0000","last_fullsized":"2024-05-03T18:27:11.781114+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:18:19.666724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.830855+0000","last_change":"2024-05-03T18:22:25.603934+0000","last_active":"2024-05-03T18:27:11.830855+0000","last_peered":"2024-05-03T18:27:11.830855+0000","last_clean":"2024-05-03T18:27:11.830855+0000","last_became_active":"2024-05-03T18:22:25.603529+0000","last_became_peered":"2024-05-03T18:22:25.603529+0000","last_unstale":"2024-05-03T18:27:11.830855+0000","last_undegraded":"2024-05-03T18:27:11.830855+0000","last_fullsized":"2024-05-03T18:27:11.830855+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:08:50.288871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.781004+0000","last_change":"2024-05-03T18:23:20.576254+0000","last_active":"2024-05-03T18:27:11.781004+0000","last_peered":"2024-05-03T18:27:11.781004+0000","last_clean":"2024-05-03T18:27:11.781004+0000","last_became_active":"2024-05-03T18:23:20.575774+0000","last_became_peered":"2024-05-03T18:23:20.575774+0000","last_unstale":"2024-05-03T18:27:11.781004+0000","last_undegraded":"2024-05-03T18:27:11.781004+0000","last_fullsized":"2024-05-03T18:27:11.781004+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:22:39.090938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"78'1","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.780167+0000","last_change":"2024-05-03T18:22:21.734171+0000","last_active":"2024-05-03T18:27:11.780167+0000","last_peered":"2024-05-03T18:27:11.780167+0000","last_clean":"2024-05-03T18:27:11.780167+0000","last_became_active":"2024-05-03T18:22:21.733301+0000","last_became_peered":"2024-05-03T18:22:21.733301+0000","last_unstale":"2024-05-03T18:27:11.780167+0000","last_undegraded":"2024-05-03T18:27:11.780167+0000","last_fullsized":"2024-05-03T18:27:11.780167+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:09:48.861247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.845701+0000","last_change":"2024-05-03T18:22:25.598016+0000","last_active":"2024-05-03T18:27:11.845701+0000","last_peered":"2024-05-03T18:27:11.845701+0000","last_clean":"2024-05-03T18:27:11.845701+0000","last_became_active":"2024-05-03T18:22:25.594888+0000","last_became_peered":"2024-05-03T18:22:25.594888+0000","last_unstale":"2024-05-03T18:27:11.845701+0000","last_undegraded":"2024-05-03T18:27:11.845701+0000","last_fullsized":"2024-05-03T18:27:11.845701+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:47:46.400247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.785843+0000","last_change":"2024-05-03T18:22:40.755148+0000","last_active":"2024-05-03T18:27:11.785843+0000","last_peered":"2024-05-03T18:27:11.785843+0000","last_clean":"2024-05-03T18:27:11.785843+0000","last_became_active":"2024-05-03T18:22:40.754656+0000","last_became_peered":"2024-05-03T18:22:40.754656+0000","last_unstale":"2024-05-03T18:27:11.785843+0000","last_undegraded":"2024-05-03T18:27:11.785843+0000","last_fullsized":"2024-05-03T18:27:11.785843+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:48:23.124936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"78'2","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.784439+0000","last_change":"2024-05-03T18:22:21.568060+0000","last_active":"2024-05-03T18:27:11.784439+0000","last_peered":"2024-05-03T18:27:11.784439+0000","last_clean":"2024-05-03T18:27:11.784439+0000","last_became_active":"2024-05-03T18:22:21.567434+0000","last_became_peered":"2024-05-03T18:22:21.567434+0000","last_unstale":"2024-05-03T18:27:11.784439+0000","last_undegraded":"2024-05-03T18:27:11.784439+0000","last_fullsized":"2024-05-03T18:27:11.784439+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:06:05.802621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.829909+0000","last_change":"2024-05-03T18:22:25.597777+0000","last_active":"2024-05-03T18:27:11.829909+0000","last_peered":"2024-05-03T18:27:11.829909+0000","last_clean":"2024-05-03T18:27:11.829909+0000","last_became_active":"2024-05-03T18:22:25.597391+0000","last_became_peered":"2024-05-03T18:22:25.597391+0000","last_unstale":"2024-05-03T18:27:11.829909+0000","last_undegraded":"2024-05-03T18:27:11.829909+0000","last_fullsized":"2024-05-03T18:27:11.829909+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:02:24.708193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.783801+0000","last_change":"2024-05-03T18:22:40.752318+0000","last_active":"2024-05-03T18:27:11.783801+0000","last_peered":"2024-05-03T18:27:11.783801+0000","last_clean":"2024-05-03T18:27:11.783801+0000","last_became_active":"2024-05-03T18:22:40.751433+0000","last_became_peered":"2024-05-03T18:22:40.751433+0000","last_unstale":"2024-05-03T18:27:11.783801+0000","last_undegraded":"2024-05-03T18:27:11.783801+0000","last_fullsized":"2024-05-03T18:27:11.783801+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:21:55.746143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.791173+0000","last_change":"2024-05-03T18:24:21.582330+0000","last_active":"2024-05-03T18:27:11.791173+0000","last_peered":"2024-05-03T18:27:11.791173+0000","last_clean":"2024-05-03T18:27:11.791173+0000","last_became_active":"2024-05-03T18:24:21.572706+0000","last_became_peered":"2024-05-03T18:24:21.572706+0000","last_unstale":"2024-05-03T18:27:11.791173+0000","last_undegraded":"2024-05-03T18:27:11.791173+0000","last_fullsized":"2024-05-03T18:27:11.791173+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:55:40.233995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":62,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.838824+0000","last_change":"2024-05-03T18:23:20.575102+0000","last_active":"2024-05-03T18:27:11.838824+0000","last_peered":"2024-05-03T18:27:11.838824+0000","last_clean":"2024-05-03T18:27:11.838824+0000","last_became_active":"2024-05-03T18:23:20.574657+0000","last_became_peered":"2024-05-03T18:23:20.574657+0000","last_unstale":"2024-05-03T18:27:11.838824+0000","last_undegraded":"2024-05-03T18:27:11.838824+0000","last_fullsized":"2024-05-03T18:27:11.838824+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:08:26.280866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.767873+0000","last_change":"2024-05-03T18:22:40.761052+0000","last_active":"2024-05-03T18:27:11.767873+0000","last_peered":"2024-05-03T18:27:11.767873+0000","last_clean":"2024-05-03T18:27:11.767873+0000","last_became_active":"2024-05-03T18:22:40.760179+0000","last_became_peered":"2024-05-03T18:22:40.760179+0000","last_unstale":"2024-05-03T18:27:11.767873+0000","last_undegraded":"2024-05-03T18:27:11.767873+0000","last_fullsized":"2024-05-03T18:27:11.767873+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:44:48.610680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"78'1","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.745780+0000","last_change":"2024-05-03T18:22:21.563421+0000","last_active":"2024-05-03T18:27:11.745780+0000","last_peered":"2024-05-03T18:27:11.745780+0000","last_clean":"2024-05-03T18:27:11.745780+0000","last_became_active":"2024-05-03T18:22:21.560991+0000","last_became_peered":"2024-05-03T18:22:21.560991+0000","last_unstale":"2024-05-03T18:27:11.745780+0000","last_undegraded":"2024-05-03T18:27:11.745780+0000","last_fullsized":"2024-05-03T18:27:11.745780+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:50:32.640699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.823442+0000","last_change":"2024-05-03T18:22:25.599049+0000","last_active":"2024-05-03T18:27:11.823442+0000","last_peered":"2024-05-03T18:27:11.823442+0000","last_clean":"2024-05-03T18:27:11.823442+0000","last_became_active":"2024-05-03T18:22:25.597879+0000","last_became_peered":"2024-05-03T18:22:25.597879+0000","last_unstale":"2024-05-03T18:27:11.823442+0000","last_undegraded":"2024-05-03T18:27:11.823442+0000","last_fullsized":"2024-05-03T18:27:11.823442+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:51:08.487499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.767790+0000","last_change":"2024-05-03T18:22:40.760908+0000","last_active":"2024-05-03T18:27:11.767790+0000","last_peered":"2024-05-03T18:27:11.767790+0000","last_clean":"2024-05-03T18:27:11.767790+0000","last_became_active":"2024-05-03T18:22:40.760331+0000","last_became_peered":"2024-05-03T18:22:40.760331+0000","last_unstale":"2024-05-03T18:27:11.767790+0000","last_undegraded":"2024-05-03T18:27:11.767790+0000","last_fullsized":"2024-05-03T18:27:11.767790+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:27:08.216015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"78'1","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.761175+0000","last_change":"2024-05-03T18:22:21.565328+0000","last_active":"2024-05-03T18:27:11.761175+0000","last_peered":"2024-05-03T18:27:11.761175+0000","last_clean":"2024-05-03T18:27:11.761175+0000","last_became_active":"2024-05-03T18:22:21.564583+0000","last_became_peered":"2024-05-03T18:22:21.564583+0000","last_unstale":"2024-05-03T18:27:11.761175+0000","last_undegraded":"2024-05-03T18:27:11.761175+0000","last_fullsized":"2024-05-03T18:27:11.761175+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:58:32.370458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.828897+0000","last_change":"2024-05-03T18:22:25.607585+0000","last_active":"2024-05-03T18:27:11.828897+0000","last_peered":"2024-05-03T18:27:11.828897+0000","last_clean":"2024-05-03T18:27:11.828897+0000","last_became_active":"2024-05-03T18:22:25.607168+0000","last_became_peered":"2024-05-03T18:22:25.607168+0000","last_unstale":"2024-05-03T18:27:11.828897+0000","last_undegraded":"2024-05-03T18:27:11.828897+0000","last_fullsized":"2024-05-03T18:27:11.828897+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:04:39.750823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.803875+0000","last_change":"2024-05-03T18:22:40.754744+0000","last_active":"2024-05-03T18:27:11.803875+0000","last_peered":"2024-05-03T18:27:11.803875+0000","last_clean":"2024-05-03T18:27:11.803875+0000","last_became_active":"2024-05-03T18:22:40.754262+0000","last_became_peered":"2024-05-03T18:22:40.754262+0000","last_unstale":"2024-05-03T18:27:11.803875+0000","last_undegraded":"2024-05-03T18:27:11.803875+0000","last_fullsized":"2024-05-03T18:27:11.803875+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:11:47.979546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.738652+0000","last_change":"2024-05-03T18:22:21.550048+0000","last_active":"2024-05-03T18:27:11.738652+0000","last_peered":"2024-05-03T18:27:11.738652+0000","last_clean":"2024-05-03T18:27:11.738652+0000","last_became_active":"2024-05-03T18:22:21.549334+0000","last_became_peered":"2024-05-03T18:22:21.549334+0000","last_unstale":"2024-05-03T18:27:11.738652+0000","last_undegraded":"2024-05-03T18:27:11.738652+0000","last_fullsized":"2024-05-03T18:27:11.738652+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:52:54.401185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.834700+0000","last_change":"2024-05-03T18:22:25.583074+0000","last_active":"2024-05-03T18:27:11.834700+0000","last_peered":"2024-05-03T18:27:11.834700+0000","last_clean":"2024-05-03T18:27:11.834700+0000","last_became_active":"2024-05-03T18:22:25.582637+0000","last_became_peered":"2024-05-03T18:22:25.582637+0000","last_unstale":"2024-05-03T18:27:11.834700+0000","last_undegraded":"2024-05-03T18:27:11.834700+0000","last_fullsized":"2024-05-03T18:27:11.834700+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:20:09.021402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.778047+0000","last_change":"2024-05-03T18:22:40.748509+0000","last_active":"2024-05-03T18:27:11.778047+0000","last_peered":"2024-05-03T18:27:11.778047+0000","last_clean":"2024-05-03T18:27:11.778047+0000","last_became_active":"2024-05-03T18:22:40.748051+0000","last_became_peered":"2024-05-03T18:22:40.748051+0000","last_unstale":"2024-05-03T18:27:11.778047+0000","last_undegraded":"2024-05-03T18:27:11.778047+0000","last_fullsized":"2024-05-03T18:27:11.778047+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:08:44.389460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.742290+0000","last_change":"2024-05-03T18:22:21.562185+0000","last_active":"2024-05-03T18:27:11.742290+0000","last_peered":"2024-05-03T18:27:11.742290+0000","last_clean":"2024-05-03T18:27:11.742290+0000","last_became_active":"2024-05-03T18:22:21.560523+0000","last_became_peered":"2024-05-03T18:22:21.560523+0000","last_unstale":"2024-05-03T18:27:11.742290+0000","last_undegraded":"2024-05-03T18:27:11.742290+0000","last_fullsized":"2024-05-03T18:27:11.742290+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:05:41.788286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.831975+0000","last_change":"2024-05-03T18:22:25.583895+0000","last_active":"2024-05-03T18:27:11.831975+0000","last_peered":"2024-05-03T18:27:11.831975+0000","last_clean":"2024-05-03T18:27:11.831975+0000","last_became_active":"2024-05-03T18:22:25.583503+0000","last_became_peered":"2024-05-03T18:22:25.583503+0000","last_unstale":"2024-05-03T18:27:11.831975+0000","last_undegraded":"2024-05-03T18:27:11.831975+0000","last_fullsized":"2024-05-03T18:27:11.831975+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:45:32.404394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.774438+0000","last_change":"2024-05-03T18:22:40.752814+0000","last_active":"2024-05-03T18:27:11.774438+0000","last_peered":"2024-05-03T18:27:11.774438+0000","last_clean":"2024-05-03T18:27:11.774438+0000","last_became_active":"2024-05-03T18:22:40.752328+0000","last_became_peered":"2024-05-03T18:22:40.752328+0000","last_unstale":"2024-05-03T18:27:11.774438+0000","last_undegraded":"2024-05-03T18:27:11.774438+0000","last_fullsized":"2024-05-03T18:27:11.774438+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:16:39.488122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.762244+0000","last_change":"2024-05-03T18:22:21.566266+0000","last_active":"2024-05-03T18:27:11.762244+0000","last_peered":"2024-05-03T18:27:11.762244+0000","last_clean":"2024-05-03T18:27:11.762244+0000","last_became_active":"2024-05-03T18:22:21.565612+0000","last_became_peered":"2024-05-03T18:22:21.565612+0000","last_unstale":"2024-05-03T18:27:11.762244+0000","last_undegraded":"2024-05-03T18:27:11.762244+0000","last_fullsized":"2024-05-03T18:27:11.762244+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:22:57.216912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.836780+0000","last_change":"2024-05-03T18:22:25.596182+0000","last_active":"2024-05-03T18:27:11.836780+0000","last_peered":"2024-05-03T18:27:11.836780+0000","last_clean":"2024-05-03T18:27:11.836780+0000","last_became_active":"2024-05-03T18:22:25.595472+0000","last_became_peered":"2024-05-03T18:22:25.595472+0000","last_unstale":"2024-05-03T18:27:11.836780+0000","last_undegraded":"2024-05-03T18:27:11.836780+0000","last_fullsized":"2024-05-03T18:27:11.836780+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:48:49.583879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.783452+0000","last_change":"2024-05-03T18:22:40.756378+0000","last_active":"2024-05-03T18:27:11.783452+0000","last_peered":"2024-05-03T18:27:11.783452+0000","last_clean":"2024-05-03T18:27:11.783452+0000","last_became_active":"2024-05-03T18:22:40.755692+0000","last_became_peered":"2024-05-03T18:22:40.755692+0000","last_unstale":"2024-05-03T18:27:11.783452+0000","last_undegraded":"2024-05-03T18:27:11.783452+0000","last_fullsized":"2024-05-03T18:27:11.783452+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:22:50.269817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.744083+0000","last_change":"2024-05-03T18:22:21.561381+0000","last_active":"2024-05-03T18:27:11.744083+0000","last_peered":"2024-05-03T18:27:11.744083+0000","last_clean":"2024-05-03T18:27:11.744083+0000","last_became_active":"2024-05-03T18:22:21.559411+0000","last_became_peered":"2024-05-03T18:22:21.559411+0000","last_unstale":"2024-05-03T18:27:11.744083+0000","last_undegraded":"2024-05-03T18:27:11.744083+0000","last_fullsized":"2024-05-03T18:27:11.744083+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:04:08.283538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":56,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.839913+0000","last_change":"2024-05-03T18:23:20.551361+0000","last_active":"2024-05-03T18:27:11.839913+0000","last_peered":"2024-05-03T18:27:11.839913+0000","last_clean":"2024-05-03T18:27:11.839913+0000","last_became_active":"2024-05-03T18:23:20.550689+0000","last_became_peered":"2024-05-03T18:23:20.550689+0000","last_unstale":"2024-05-03T18:27:11.839913+0000","last_undegraded":"2024-05-03T18:27:11.839913+0000","last_fullsized":"2024-05-03T18:27:11.839913+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:36:52.767110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.787425+0000","last_change":"2024-05-03T18:22:40.758006+0000","last_active":"2024-05-03T18:27:11.787425+0000","last_peered":"2024-05-03T18:27:11.787425+0000","last_clean":"2024-05-03T18:27:11.787425+0000","last_became_active":"2024-05-03T18:22:40.757075+0000","last_became_peered":"2024-05-03T18:22:40.757075+0000","last_unstale":"2024-05-03T18:27:11.787425+0000","last_undegraded":"2024-05-03T18:27:11.787425+0000","last_fullsized":"2024-05-03T18:27:11.787425+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:06:02.390184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"78'3","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.744164+0000","last_change":"2024-05-03T18:22:21.740041+0000","last_active":"2024-05-03T18:27:11.744164+0000","last_peered":"2024-05-03T18:27:11.744164+0000","last_clean":"2024-05-03T18:27:11.744164+0000","last_became_active":"2024-05-03T18:22:21.738777+0000","last_became_peered":"2024-05-03T18:22:21.738777+0000","last_unstale":"2024-05-03T18:27:11.744164+0000","last_undegraded":"2024-05-03T18:27:11.744164+0000","last_fullsized":"2024-05-03T18:27:11.744164+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:35:30.098325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":53,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.825545+0000","last_change":"2024-05-03T18:22:25.606854+0000","last_active":"2024-05-03T18:27:11.825545+0000","last_peered":"2024-05-03T18:27:11.825545+0000","last_clean":"2024-05-03T18:27:11.825545+0000","last_became_active":"2024-05-03T18:22:25.606376+0000","last_became_peered":"2024-05-03T18:22:25.606376+0000","last_unstale":"2024-05-03T18:27:11.825545+0000","last_undegraded":"2024-05-03T18:27:11.825545+0000","last_fullsized":"2024-05-03T18:27:11.825545+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:29:59.461181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.784158+0000","last_change":"2024-05-03T18:22:40.752863+0000","last_active":"2024-05-03T18:27:11.784158+0000","last_peered":"2024-05-03T18:27:11.784158+0000","last_clean":"2024-05-03T18:27:11.784158+0000","last_became_active":"2024-05-03T18:22:40.752309+0000","last_became_peered":"2024-05-03T18:22:40.752309+0000","last_unstale":"2024-05-03T18:27:11.784158+0000","last_undegraded":"2024-05-03T18:27:11.784158+0000","last_fullsized":"2024-05-03T18:27:11.784158+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:01:06.110570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'37","reported_seq":510,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:12.403811+0000","last_change":"2024-05-03T18:22:21.566162+0000","last_active":"2024-05-03T18:27:12.403811+0000","last_peered":"2024-05-03T18:27:12.403811+0000","last_clean":"2024-05-03T18:27:12.403811+0000","last_became_active":"2024-05-03T18:22:21.565641+0000","last_became_peered":"2024-05-03T18:22:21.565641+0000","last_unstale":"2024-05-03T18:27:12.403811+0000","last_undegraded":"2024-05-03T18:27:12.403811+0000","last_fullsized":"2024-05-03T18:27:12.403811+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:23:45.283135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69239,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":425,"num_read_kb":281,"num_write":37,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.833787+0000","last_change":"2024-05-03T18:22:25.586687+0000","last_active":"2024-05-03T18:27:11.833787+0000","last_peered":"2024-05-03T18:27:11.833787+0000","last_clean":"2024-05-03T18:27:11.833787+0000","last_became_active":"2024-05-03T18:22:25.586275+0000","last_became_peered":"2024-05-03T18:22:25.586275+0000","last_unstale":"2024-05-03T18:27:11.833787+0000","last_undegraded":"2024-05-03T18:27:11.833787+0000","last_fullsized":"2024-05-03T18:27:11.833787+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:10:50.339865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.744602+0000","last_change":"2024-05-03T18:22:40.758003+0000","last_active":"2024-05-03T18:27:11.744602+0000","last_peered":"2024-05-03T18:27:11.744602+0000","last_clean":"2024-05-03T18:27:11.744602+0000","last_became_active":"2024-05-03T18:22:40.757559+0000","last_became_peered":"2024-05-03T18:22:40.757559+0000","last_unstale":"2024-05-03T18:27:11.744602+0000","last_undegraded":"2024-05-03T18:27:11.744602+0000","last_fullsized":"2024-05-03T18:27:11.744602+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:54:23.377748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"78'2","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.767568+0000","last_change":"2024-05-03T18:22:21.564546+0000","last_active":"2024-05-03T18:27:11.767568+0000","last_peered":"2024-05-03T18:27:11.767568+0000","last_clean":"2024-05-03T18:27:11.767568+0000","last_became_active":"2024-05-03T18:22:21.564105+0000","last_became_peered":"2024-05-03T18:22:21.564105+0000","last_unstale":"2024-05-03T18:27:11.767568+0000","last_undegraded":"2024-05-03T18:27:11.767568+0000","last_fullsized":"2024-05-03T18:27:11.767568+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:36:07.790643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.838916+0000","last_change":"2024-05-03T18:22:25.584059+0000","last_active":"2024-05-03T18:27:11.838916+0000","last_peered":"2024-05-03T18:27:11.838916+0000","last_clean":"2024-05-03T18:27:11.838916+0000","last_became_active":"2024-05-03T18:22:25.583742+0000","last_became_peered":"2024-05-03T18:22:25.583742+0000","last_unstale":"2024-05-03T18:27:11.838916+0000","last_undegraded":"2024-05-03T18:27:11.838916+0000","last_fullsized":"2024-05-03T18:27:11.838916+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:15:48.288422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.791333+0000","last_change":"2024-05-03T18:22:40.751362+0000","last_active":"2024-05-03T18:27:11.791333+0000","last_peered":"2024-05-03T18:27:11.791333+0000","last_clean":"2024-05-03T18:27:11.791333+0000","last_became_active":"2024-05-03T18:22:40.750365+0000","last_became_peered":"2024-05-03T18:22:40.750365+0000","last_unstale":"2024-05-03T18:27:11.791333+0000","last_undegraded":"2024-05-03T18:27:11.791333+0000","last_fullsized":"2024-05-03T18:27:11.791333+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:35:50.629332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.742961+0000","last_change":"2024-05-03T18:22:21.556123+0000","last_active":"2024-05-03T18:27:11.742961+0000","last_peered":"2024-05-03T18:27:11.742961+0000","last_clean":"2024-05-03T18:27:11.742961+0000","last_became_active":"2024-05-03T18:22:21.554918+0000","last_became_peered":"2024-05-03T18:22:21.554918+0000","last_unstale":"2024-05-03T18:27:11.742961+0000","last_undegraded":"2024-05-03T18:27:11.742961+0000","last_fullsized":"2024-05-03T18:27:11.742961+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:09:12.383601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":57,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.840168+0000","last_change":"2024-05-03T18:22:25.601849+0000","last_active":"2024-05-03T18:27:11.840168+0000","last_peered":"2024-05-03T18:27:11.840168+0000","last_clean":"2024-05-03T18:27:11.840168+0000","last_became_active":"2024-05-03T18:22:25.601434+0000","last_became_peered":"2024-05-03T18:22:25.601434+0000","last_unstale":"2024-05-03T18:27:11.840168+0000","last_undegraded":"2024-05-03T18:27:11.840168+0000","last_fullsized":"2024-05-03T18:27:11.840168+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:43:18.273389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.780768+0000","last_change":"2024-05-03T18:22:40.754448+0000","last_active":"2024-05-03T18:27:11.780768+0000","last_peered":"2024-05-03T18:27:11.780768+0000","last_clean":"2024-05-03T18:27:11.780768+0000","last_became_active":"2024-05-03T18:22:40.753122+0000","last_became_peered":"2024-05-03T18:22:40.753122+0000","last_unstale":"2024-05-03T18:27:11.780768+0000","last_undegraded":"2024-05-03T18:27:11.780768+0000","last_fullsized":"2024-05-03T18:27:11.780768+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:43:15.216222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.778755+0000","last_change":"2024-05-03T18:22:21.559292+0000","last_active":"2024-05-03T18:27:11.778755+0000","last_peered":"2024-05-03T18:27:11.778755+0000","last_clean":"2024-05-03T18:27:11.778755+0000","last_became_active":"2024-05-03T18:22:21.557657+0000","last_became_peered":"2024-05-03T18:22:21.557657+0000","last_unstale":"2024-05-03T18:27:11.778755+0000","last_undegraded":"2024-05-03T18:27:11.778755+0000","last_fullsized":"2024-05-03T18:27:11.778755+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:51:57.709387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.837734+0000","last_change":"2024-05-03T18:22:25.604340+0000","last_active":"2024-05-03T18:27:11.837734+0000","last_peered":"2024-05-03T18:27:11.837734+0000","last_clean":"2024-05-03T18:27:11.837734+0000","last_became_active":"2024-05-03T18:22:25.603651+0000","last_became_peered":"2024-05-03T18:22:25.603651+0000","last_unstale":"2024-05-03T18:27:11.837734+0000","last_undegraded":"2024-05-03T18:27:11.837734+0000","last_fullsized":"2024-05-03T18:27:11.837734+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:49:36.379556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.740450+0000","last_change":"2024-05-03T18:22:40.738065+0000","last_active":"2024-05-03T18:27:11.740450+0000","last_peered":"2024-05-03T18:27:11.740450+0000","last_clean":"2024-05-03T18:27:11.740450+0000","last_became_active":"2024-05-03T18:22:40.737546+0000","last_became_peered":"2024-05-03T18:22:40.737546+0000","last_unstale":"2024-05-03T18:27:11.740450+0000","last_undegraded":"2024-05-03T18:27:11.740450+0000","last_fullsized":"2024-05-03T18:27:11.740450+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:45:31.528581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"78'8","reported_seq":56,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.778653+0000","last_change":"2024-05-03T18:22:21.739131+0000","last_active":"2024-05-03T18:27:11.778653+0000","last_peered":"2024-05-03T18:27:11.778653+0000","last_clean":"2024-05-03T18:27:11.778653+0000","last_became_active":"2024-05-03T18:22:21.737243+0000","last_became_peered":"2024-05-03T18:22:21.737243+0000","last_unstale":"2024-05-03T18:27:11.778653+0000","last_undegraded":"2024-05-03T18:27:11.778653+0000","last_fullsized":"2024-05-03T18:27:11.778653+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:46:24.011140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.830488+0000","last_change":"2024-05-03T18:22:25.602011+0000","last_active":"2024-05-03T18:27:11.830488+0000","last_peered":"2024-05-03T18:27:11.830488+0000","last_clean":"2024-05-03T18:27:11.830488+0000","last_became_active":"2024-05-03T18:22:25.601604+0000","last_became_peered":"2024-05-03T18:22:25.601604+0000","last_unstale":"2024-05-03T18:27:11.830488+0000","last_undegraded":"2024-05-03T18:27:11.830488+0000","last_fullsized":"2024-05-03T18:27:11.830488+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:25:57.743203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.742365+0000","last_change":"2024-05-03T18:22:40.757885+0000","last_active":"2024-05-03T18:27:11.742365+0000","last_peered":"2024-05-03T18:27:11.742365+0000","last_clean":"2024-05-03T18:27:11.742365+0000","last_became_active":"2024-05-03T18:22:40.757208+0000","last_became_peered":"2024-05-03T18:22:40.757208+0000","last_unstale":"2024-05-03T18:27:11.742365+0000","last_undegraded":"2024-05-03T18:27:11.742365+0000","last_fullsized":"2024-05-03T18:27:11.742365+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:37:00.792138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"78'1","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.791831+0000","last_change":"2024-05-03T18:22:21.554932+0000","last_active":"2024-05-03T18:27:11.791831+0000","last_peered":"2024-05-03T18:27:11.791831+0000","last_clean":"2024-05-03T18:27:11.791831+0000","last_became_active":"2024-05-03T18:22:21.554123+0000","last_became_peered":"2024-05-03T18:22:21.554123+0000","last_unstale":"2024-05-03T18:27:11.791831+0000","last_undegraded":"2024-05-03T18:27:11.791831+0000","last_fullsized":"2024-05-03T18:27:11.791831+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:38:59.859138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.831370+0000","last_change":"2024-05-03T18:22:25.603400+0000","last_active":"2024-05-03T18:27:11.831370+0000","last_peered":"2024-05-03T18:27:11.831370+0000","last_clean":"2024-05-03T18:27:11.831370+0000","last_became_active":"2024-05-03T18:22:25.603045+0000","last_became_peered":"2024-05-03T18:22:25.603045+0000","last_unstale":"2024-05-03T18:27:11.831370+0000","last_undegraded":"2024-05-03T18:27:11.831370+0000","last_fullsized":"2024-05-03T18:27:11.831370+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:51:33.502972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.778451+0000","last_change":"2024-05-03T18:22:40.750857+0000","last_active":"2024-05-03T18:27:11.778451+0000","last_peered":"2024-05-03T18:27:11.778451+0000","last_clean":"2024-05-03T18:27:11.778451+0000","last_became_active":"2024-05-03T18:22:40.750381+0000","last_became_peered":"2024-05-03T18:22:40.750381+0000","last_unstale":"2024-05-03T18:27:11.778451+0000","last_undegraded":"2024-05-03T18:27:11.778451+0000","last_fullsized":"2024-05-03T18:27:11.778451+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:15:28.542612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.781805+0000","last_change":"2024-05-03T18:22:21.739345+0000","last_active":"2024-05-03T18:27:11.781805+0000","last_peered":"2024-05-03T18:27:11.781805+0000","last_clean":"2024-05-03T18:27:11.781805+0000","last_became_active":"2024-05-03T18:22:21.738874+0000","last_became_peered":"2024-05-03T18:22:21.738874+0000","last_unstale":"2024-05-03T18:27:11.781805+0000","last_undegraded":"2024-05-03T18:27:11.781805+0000","last_fullsized":"2024-05-03T18:27:11.781805+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:57:59.706356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"90'4","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.834563+0000","last_change":"2024-05-03T18:22:25.586543+0000","last_active":"2024-05-03T18:27:11.834563+0000","last_peered":"2024-05-03T18:27:11.834563+0000","last_clean":"2024-05-03T18:27:11.834563+0000","last_became_active":"2024-05-03T18:22:25.585124+0000","last_became_peered":"2024-05-03T18:22:25.585124+0000","last_unstale":"2024-05-03T18:27:11.834563+0000","last_undegraded":"2024-05-03T18:27:11.834563+0000","last_fullsized":"2024-05-03T18:27:11.834563+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:41:32.633444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.739379+0000","last_change":"2024-05-03T18:22:40.749665+0000","last_active":"2024-05-03T18:27:11.739379+0000","last_peered":"2024-05-03T18:27:11.739379+0000","last_clean":"2024-05-03T18:27:11.739379+0000","last_became_active":"2024-05-03T18:22:40.749062+0000","last_became_peered":"2024-05-03T18:22:40.749062+0000","last_unstale":"2024-05-03T18:27:11.739379+0000","last_undegraded":"2024-05-03T18:27:11.739379+0000","last_fullsized":"2024-05-03T18:27:11.739379+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:09:17.194507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.742374+0000","last_change":"2024-05-03T18:22:21.560944+0000","last_active":"2024-05-03T18:27:11.742374+0000","last_peered":"2024-05-03T18:27:11.742374+0000","last_clean":"2024-05-03T18:27:11.742374+0000","last_became_active":"2024-05-03T18:22:21.560412+0000","last_became_peered":"2024-05-03T18:22:21.560412+0000","last_unstale":"2024-05-03T18:27:11.742374+0000","last_undegraded":"2024-05-03T18:27:11.742374+0000","last_fullsized":"2024-05-03T18:27:11.742374+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:12:04.069564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.841297+0000","last_change":"2024-05-03T18:22:25.598235+0000","last_active":"2024-05-03T18:27:11.841297+0000","last_peered":"2024-05-03T18:27:11.841297+0000","last_clean":"2024-05-03T18:27:11.841297+0000","last_became_active":"2024-05-03T18:22:25.595993+0000","last_became_peered":"2024-05-03T18:22:25.595993+0000","last_unstale":"2024-05-03T18:27:11.841297+0000","last_undegraded":"2024-05-03T18:27:11.841297+0000","last_fullsized":"2024-05-03T18:27:11.841297+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:53:16.069119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.741050+0000","last_change":"2024-05-03T18:22:40.745005+0000","last_active":"2024-05-03T18:27:11.741050+0000","last_peered":"2024-05-03T18:27:11.741050+0000","last_clean":"2024-05-03T18:27:11.741050+0000","last_became_active":"2024-05-03T18:22:40.742134+0000","last_became_peered":"2024-05-03T18:22:40.742134+0000","last_unstale":"2024-05-03T18:27:11.741050+0000","last_undegraded":"2024-05-03T18:27:11.741050+0000","last_fullsized":"2024-05-03T18:27:11.741050+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:47:12.217012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.829885+0000","last_change":"2024-05-03T18:22:25.604640+0000","last_active":"2024-05-03T18:27:11.829885+0000","last_peered":"2024-05-03T18:27:11.829885+0000","last_clean":"2024-05-03T18:27:11.829885+0000","last_became_active":"2024-05-03T18:22:25.603759+0000","last_became_peered":"2024-05-03T18:22:25.603759+0000","last_unstale":"2024-05-03T18:27:11.829885+0000","last_undegraded":"2024-05-03T18:27:11.829885+0000","last_fullsized":"2024-05-03T18:27:11.829885+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:56:55.744321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.738330+0000","last_change":"2024-05-03T18:22:21.741006+0000","last_active":"2024-05-03T18:27:11.738330+0000","last_peered":"2024-05-03T18:27:11.738330+0000","last_clean":"2024-05-03T18:27:11.738330+0000","last_became_active":"2024-05-03T18:22:21.737999+0000","last_became_peered":"2024-05-03T18:22:21.737999+0000","last_unstale":"2024-05-03T18:27:11.738330+0000","last_undegraded":"2024-05-03T18:27:11.738330+0000","last_fullsized":"2024-05-03T18:27:11.738330+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:29:54.931940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.743698+0000","last_change":"2024-05-03T18:22:40.761044+0000","last_active":"2024-05-03T18:27:11.743698+0000","last_peered":"2024-05-03T18:27:11.743698+0000","last_clean":"2024-05-03T18:27:11.743698+0000","last_became_active":"2024-05-03T18:22:40.760245+0000","last_became_peered":"2024-05-03T18:22:40.760245+0000","last_unstale":"2024-05-03T18:27:11.743698+0000","last_undegraded":"2024-05-03T18:27:11.743698+0000","last_fullsized":"2024-05-03T18:27:11.743698+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:31:12.729078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.832106+0000","last_change":"2024-05-03T18:22:25.595759+0000","last_active":"2024-05-03T18:27:11.832106+0000","last_peered":"2024-05-03T18:27:11.832106+0000","last_clean":"2024-05-03T18:27:11.832106+0000","last_became_active":"2024-05-03T18:22:25.594368+0000","last_became_peered":"2024-05-03T18:22:25.594368+0000","last_unstale":"2024-05-03T18:27:11.832106+0000","last_undegraded":"2024-05-03T18:27:11.832106+0000","last_fullsized":"2024-05-03T18:27:11.832106+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:26:19.196877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.787190+0000","last_change":"2024-05-03T18:22:21.557123+0000","last_active":"2024-05-03T18:27:11.787190+0000","last_peered":"2024-05-03T18:27:11.787190+0000","last_clean":"2024-05-03T18:27:11.787190+0000","last_became_active":"2024-05-03T18:22:21.556432+0000","last_became_peered":"2024-05-03T18:22:21.556432+0000","last_unstale":"2024-05-03T18:27:11.787190+0000","last_undegraded":"2024-05-03T18:27:11.787190+0000","last_fullsized":"2024-05-03T18:27:11.787190+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:50:03.179320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":123,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.741954+0000","last_change":"2024-05-03T18:20:24.937044+0000","last_active":"2024-05-03T18:27:11.741954+0000","last_peered":"2024-05-03T18:27:11.741954+0000","last_clean":"2024-05-03T18:27:11.741954+0000","last_became_active":"2024-05-03T18:20:24.922095+0000","last_became_peered":"2024-05-03T18:20:24.922095+0000","last_unstale":"2024-05-03T18:27:11.741954+0000","last_undegraded":"2024-05-03T18:27:11.741954+0000","last_fullsized":"2024-05-03T18:27:11.741954+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:15:37.353911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:15:37.353911+0000","last_clean_scrub_stamp":"2024-05-03T18:15:37.353911+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:40:07.341441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.780628+0000","last_change":"2024-05-03T18:23:20.575064+0000","last_active":"2024-05-03T18:27:11.780628+0000","last_peered":"2024-05-03T18:27:11.780628+0000","last_clean":"2024-05-03T18:27:11.780628+0000","last_became_active":"2024-05-03T18:23:20.574557+0000","last_became_peered":"2024-05-03T18:23:20.574557+0000","last_unstale":"2024-05-03T18:27:11.780628+0000","last_undegraded":"2024-05-03T18:27:11.780628+0000","last_fullsized":"2024-05-03T18:27:11.780628+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:34:08.520396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,6,10],"acting":[8,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.830147+0000","last_change":"2024-05-03T18:22:25.591269+0000","last_active":"2024-05-03T18:27:11.830147+0000","last_peered":"2024-05-03T18:27:11.830147+0000","last_clean":"2024-05-03T18:27:11.830147+0000","last_became_active":"2024-05-03T18:22:25.590409+0000","last_became_peered":"2024-05-03T18:22:25.590409+0000","last_unstale":"2024-05-03T18:27:11.830147+0000","last_undegraded":"2024-05-03T18:27:11.830147+0000","last_fullsized":"2024-05-03T18:27:11.830147+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:03:57.432331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.742525+0000","last_change":"2024-05-03T18:22:21.560137+0000","last_active":"2024-05-03T18:27:11.742525+0000","last_peered":"2024-05-03T18:27:11.742525+0000","last_clean":"2024-05-03T18:27:11.742525+0000","last_became_active":"2024-05-03T18:22:21.559672+0000","last_became_peered":"2024-05-03T18:22:21.559672+0000","last_unstale":"2024-05-03T18:27:11.742525+0000","last_undegraded":"2024-05-03T18:27:11.742525+0000","last_fullsized":"2024-05-03T18:27:11.742525+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:20:02.070390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.791935+0000","last_change":"2024-05-03T18:22:40.753160+0000","last_active":"2024-05-03T18:27:11.791935+0000","last_peered":"2024-05-03T18:27:11.791935+0000","last_clean":"2024-05-03T18:27:11.791935+0000","last_became_active":"2024-05-03T18:22:40.750800+0000","last_became_peered":"2024-05-03T18:22:40.750800+0000","last_unstale":"2024-05-03T18:27:11.791935+0000","last_undegraded":"2024-05-03T18:27:11.791935+0000","last_fullsized":"2024-05-03T18:27:11.791935+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:27:48.756140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.784686+0000","last_change":"2024-05-03T18:22:21.739490+0000","last_active":"2024-05-03T18:27:11.784686+0000","last_peered":"2024-05-03T18:27:11.784686+0000","last_clean":"2024-05-03T18:27:11.784686+0000","last_became_active":"2024-05-03T18:22:21.738512+0000","last_became_peered":"2024-05-03T18:22:21.738512+0000","last_unstale":"2024-05-03T18:27:11.784686+0000","last_undegraded":"2024-05-03T18:27:11.784686+0000","last_fullsized":"2024-05-03T18:27:11.784686+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:26:45.323519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.837222+0000","last_change":"2024-05-03T18:22:25.591509+0000","last_active":"2024-05-03T18:27:11.837222+0000","last_peered":"2024-05-03T18:27:11.837222+0000","last_clean":"2024-05-03T18:27:11.837222+0000","last_became_active":"2024-05-03T18:22:25.590188+0000","last_became_peered":"2024-05-03T18:22:25.590188+0000","last_unstale":"2024-05-03T18:27:11.837222+0000","last_undegraded":"2024-05-03T18:27:11.837222+0000","last_fullsized":"2024-05-03T18:27:11.837222+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:33:23.919101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.773559+0000","last_change":"2024-05-03T18:22:40.747176+0000","last_active":"2024-05-03T18:27:11.773559+0000","last_peered":"2024-05-03T18:27:11.773559+0000","last_clean":"2024-05-03T18:27:11.773559+0000","last_became_active":"2024-05-03T18:22:40.746526+0000","last_became_peered":"2024-05-03T18:22:40.746526+0000","last_unstale":"2024-05-03T18:27:11.773559+0000","last_undegraded":"2024-05-03T18:27:11.773559+0000","last_fullsized":"2024-05-03T18:27:11.773559+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:59:25.838382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'44","reported_seq":115,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:12.148271+0000","last_change":"2024-05-03T18:22:21.555978+0000","last_active":"2024-05-03T18:27:12.148271+0000","last_peered":"2024-05-03T18:27:12.148271+0000","last_clean":"2024-05-03T18:27:12.148271+0000","last_became_active":"2024-05-03T18:22:21.555395+0000","last_became_peered":"2024-05-03T18:22:21.555395+0000","last_unstale":"2024-05-03T18:27:12.148271+0000","last_undegraded":"2024-05-03T18:27:12.148271+0000","last_fullsized":"2024-05-03T18:27:12.148271+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:05:44.795978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":80,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.838632+0000","last_change":"2024-05-03T18:22:25.589071+0000","last_active":"2024-05-03T18:27:11.838632+0000","last_peered":"2024-05-03T18:27:11.838632+0000","last_clean":"2024-05-03T18:27:11.838632+0000","last_became_active":"2024-05-03T18:22:25.588679+0000","last_became_peered":"2024-05-03T18:22:25.588679+0000","last_unstale":"2024-05-03T18:27:11.838632+0000","last_undegraded":"2024-05-03T18:27:11.838632+0000","last_fullsized":"2024-05-03T18:27:11.838632+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:20:53.983166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.766671+0000","last_change":"2024-05-03T18:22:40.755154+0000","last_active":"2024-05-03T18:27:11.766671+0000","last_peered":"2024-05-03T18:27:11.766671+0000","last_clean":"2024-05-03T18:27:11.766671+0000","last_became_active":"2024-05-03T18:22:40.754635+0000","last_became_peered":"2024-05-03T18:22:40.754635+0000","last_unstale":"2024-05-03T18:27:11.766671+0000","last_undegraded":"2024-05-03T18:27:11.766671+0000","last_fullsized":"2024-05-03T18:27:11.766671+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:25:56.787406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"78'2","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.787950+0000","last_change":"2024-05-03T18:22:21.854322+0000","last_active":"2024-05-03T18:27:11.787950+0000","last_peered":"2024-05-03T18:27:11.787950+0000","last_clean":"2024-05-03T18:27:11.787950+0000","last_became_active":"2024-05-03T18:22:21.853359+0000","last_became_peered":"2024-05-03T18:22:21.853359+0000","last_unstale":"2024-05-03T18:27:11.787950+0000","last_undegraded":"2024-05-03T18:27:11.787950+0000","last_fullsized":"2024-05-03T18:27:11.787950+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:06:57.061775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.835206+0000","last_change":"2024-05-03T18:22:25.605614+0000","last_active":"2024-05-03T18:27:11.835206+0000","last_peered":"2024-05-03T18:27:11.835206+0000","last_clean":"2024-05-03T18:27:11.835206+0000","last_became_active":"2024-05-03T18:22:25.604277+0000","last_became_peered":"2024-05-03T18:22:25.604277+0000","last_unstale":"2024-05-03T18:27:11.835206+0000","last_undegraded":"2024-05-03T18:27:11.835206+0000","last_fullsized":"2024-05-03T18:27:11.835206+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:17:09.812304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.790222+0000","last_change":"2024-05-03T18:22:40.763353+0000","last_active":"2024-05-03T18:27:11.790222+0000","last_peered":"2024-05-03T18:27:11.790222+0000","last_clean":"2024-05-03T18:27:11.790222+0000","last_became_active":"2024-05-03T18:22:40.762774+0000","last_became_peered":"2024-05-03T18:22:40.762774+0000","last_unstale":"2024-05-03T18:27:11.790222+0000","last_undegraded":"2024-05-03T18:27:11.790222+0000","last_fullsized":"2024-05-03T18:27:11.790222+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:38:37.247050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.766499+0000","last_change":"2024-05-03T18:24:21.803989+0000","last_active":"2024-05-03T18:27:11.766499+0000","last_peered":"2024-05-03T18:27:11.766499+0000","last_clean":"2024-05-03T18:27:11.766499+0000","last_became_active":"2024-05-03T18:24:21.803072+0000","last_became_peered":"2024-05-03T18:24:21.803072+0000","last_unstale":"2024-05-03T18:27:11.766499+0000","last_undegraded":"2024-05-03T18:27:11.766499+0000","last_fullsized":"2024-05-03T18:27:11.766499+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:32:30.562148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.815005+0000","last_change":"2024-05-03T18:22:25.588011+0000","last_active":"2024-05-03T18:27:11.815005+0000","last_peered":"2024-05-03T18:27:11.815005+0000","last_clean":"2024-05-03T18:27:11.815005+0000","last_became_active":"2024-05-03T18:22:25.587602+0000","last_became_peered":"2024-05-03T18:22:25.587602+0000","last_unstale":"2024-05-03T18:27:11.815005+0000","last_undegraded":"2024-05-03T18:27:11.815005+0000","last_fullsized":"2024-05-03T18:27:11.815005+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:18:13.186088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.781014+0000","last_change":"2024-05-03T18:22:40.755731+0000","last_active":"2024-05-03T18:27:11.781014+0000","last_peered":"2024-05-03T18:27:11.781014+0000","last_clean":"2024-05-03T18:27:11.781014+0000","last_became_active":"2024-05-03T18:22:40.755173+0000","last_became_peered":"2024-05-03T18:22:40.755173+0000","last_unstale":"2024-05-03T18:27:11.781014+0000","last_undegraded":"2024-05-03T18:27:11.781014+0000","last_fullsized":"2024-05-03T18:27:11.781014+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:36:36.548095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.774547+0000","last_change":"2024-05-03T18:22:21.561481+0000","last_active":"2024-05-03T18:27:11.774547+0000","last_peered":"2024-05-03T18:27:11.774547+0000","last_clean":"2024-05-03T18:27:11.774547+0000","last_became_active":"2024-05-03T18:22:21.560088+0000","last_became_peered":"2024-05-03T18:22:21.560088+0000","last_unstale":"2024-05-03T18:27:11.774547+0000","last_undegraded":"2024-05-03T18:27:11.774547+0000","last_fullsized":"2024-05-03T18:27:11.774547+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:45:40.184813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.839958+0000","last_change":"2024-05-03T18:22:25.584556+0000","last_active":"2024-05-03T18:27:11.839958+0000","last_peered":"2024-05-03T18:27:11.839958+0000","last_clean":"2024-05-03T18:27:11.839958+0000","last_became_active":"2024-05-03T18:22:25.583873+0000","last_became_peered":"2024-05-03T18:22:25.583873+0000","last_unstale":"2024-05-03T18:27:11.839958+0000","last_undegraded":"2024-05-03T18:27:11.839958+0000","last_fullsized":"2024-05-03T18:27:11.839958+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:28:39.028662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.803445+0000","last_change":"2024-05-03T18:22:40.763479+0000","last_active":"2024-05-03T18:27:11.803445+0000","last_peered":"2024-05-03T18:27:11.803445+0000","last_clean":"2024-05-03T18:27:11.803445+0000","last_became_active":"2024-05-03T18:22:40.762897+0000","last_became_peered":"2024-05-03T18:22:40.762897+0000","last_unstale":"2024-05-03T18:27:11.803445+0000","last_undegraded":"2024-05-03T18:27:11.803445+0000","last_fullsized":"2024-05-03T18:27:11.803445+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:00:21.090730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.767323+0000","last_change":"2024-05-03T18:22:21.562929+0000","last_active":"2024-05-03T18:27:11.767323+0000","last_peered":"2024-05-03T18:27:11.767323+0000","last_clean":"2024-05-03T18:27:11.767323+0000","last_became_active":"2024-05-03T18:22:21.560712+0000","last_became_peered":"2024-05-03T18:22:21.560712+0000","last_unstale":"2024-05-03T18:27:11.767323+0000","last_undegraded":"2024-05-03T18:27:11.767323+0000","last_fullsized":"2024-05-03T18:27:11.767323+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:31:34.722437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.840750+0000","last_change":"2024-05-03T18:22:25.581971+0000","last_active":"2024-05-03T18:27:11.840750+0000","last_peered":"2024-05-03T18:27:11.840750+0000","last_clean":"2024-05-03T18:27:11.840750+0000","last_became_active":"2024-05-03T18:22:25.581565+0000","last_became_peered":"2024-05-03T18:22:25.581565+0000","last_unstale":"2024-05-03T18:27:11.840750+0000","last_undegraded":"2024-05-03T18:27:11.840750+0000","last_fullsized":"2024-05-03T18:27:11.840750+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:10:20.607999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.787659+0000","last_change":"2024-05-03T18:22:40.761458+0000","last_active":"2024-05-03T18:27:11.787659+0000","last_peered":"2024-05-03T18:27:11.787659+0000","last_clean":"2024-05-03T18:27:11.787659+0000","last_became_active":"2024-05-03T18:22:40.760969+0000","last_became_peered":"2024-05-03T18:22:40.760969+0000","last_unstale":"2024-05-03T18:27:11.787659+0000","last_undegraded":"2024-05-03T18:27:11.787659+0000","last_fullsized":"2024-05-03T18:27:11.787659+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:19:16.804720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"78'2","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.780499+0000","last_change":"2024-05-03T18:22:21.565869+0000","last_active":"2024-05-03T18:27:11.780499+0000","last_peered":"2024-05-03T18:27:11.780499+0000","last_clean":"2024-05-03T18:27:11.780499+0000","last_became_active":"2024-05-03T18:22:21.564119+0000","last_became_peered":"2024-05-03T18:22:21.564119+0000","last_unstale":"2024-05-03T18:27:11.780499+0000","last_undegraded":"2024-05-03T18:27:11.780499+0000","last_fullsized":"2024-05-03T18:27:11.780499+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:50:29.894070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.839023+0000","last_change":"2024-05-03T18:22:25.596199+0000","last_active":"2024-05-03T18:27:11.839023+0000","last_peered":"2024-05-03T18:27:11.839023+0000","last_clean":"2024-05-03T18:27:11.839023+0000","last_became_active":"2024-05-03T18:22:25.595167+0000","last_became_peered":"2024-05-03T18:22:25.595167+0000","last_unstale":"2024-05-03T18:27:11.839023+0000","last_undegraded":"2024-05-03T18:27:11.839023+0000","last_fullsized":"2024-05-03T18:27:11.839023+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:28:56.599064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.766667+0000","last_change":"2024-05-03T18:22:40.753020+0000","last_active":"2024-05-03T18:27:11.766667+0000","last_peered":"2024-05-03T18:27:11.766667+0000","last_clean":"2024-05-03T18:27:11.766667+0000","last_became_active":"2024-05-03T18:22:40.752502+0000","last_became_peered":"2024-05-03T18:22:40.752502+0000","last_unstale":"2024-05-03T18:27:11.766667+0000","last_undegraded":"2024-05-03T18:27:11.766667+0000","last_fullsized":"2024-05-03T18:27:11.766667+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:33:51.277879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.768258+0000","last_change":"2024-05-03T18:22:21.854481+0000","last_active":"2024-05-03T18:27:11.768258+0000","last_peered":"2024-05-03T18:27:11.768258+0000","last_clean":"2024-05-03T18:27:11.768258+0000","last_became_active":"2024-05-03T18:22:21.853172+0000","last_became_peered":"2024-05-03T18:22:21.853172+0000","last_unstale":"2024-05-03T18:27:11.768258+0000","last_undegraded":"2024-05-03T18:27:11.768258+0000","last_fullsized":"2024-05-03T18:27:11.768258+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:11:47.820105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.837912+0000","last_change":"2024-05-03T18:22:25.600856+0000","last_active":"2024-05-03T18:27:11.837912+0000","last_peered":"2024-05-03T18:27:11.837912+0000","last_clean":"2024-05-03T18:27:11.837912+0000","last_became_active":"2024-05-03T18:22:25.595351+0000","last_became_peered":"2024-05-03T18:22:25.595351+0000","last_unstale":"2024-05-03T18:27:11.837912+0000","last_undegraded":"2024-05-03T18:27:11.837912+0000","last_fullsized":"2024-05-03T18:27:11.837912+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:29:24.232506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.773733+0000","last_change":"2024-05-03T18:23:20.578331+0000","last_active":"2024-05-03T18:27:11.773733+0000","last_peered":"2024-05-03T18:27:11.773733+0000","last_clean":"2024-05-03T18:27:11.773733+0000","last_became_active":"2024-05-03T18:23:20.576974+0000","last_became_peered":"2024-05-03T18:23:20.576974+0000","last_unstale":"2024-05-03T18:27:11.773733+0000","last_undegraded":"2024-05-03T18:27:11.773733+0000","last_fullsized":"2024-05-03T18:27:11.773733+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:57:32.922175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"90'39","reported_seq":95,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:12.135624+0000","last_change":"2024-05-03T18:22:21.552331+0000","last_active":"2024-05-03T18:27:12.135624+0000","last_peered":"2024-05-03T18:27:12.135624+0000","last_clean":"2024-05-03T18:27:12.135624+0000","last_became_active":"2024-05-03T18:22:21.549786+0000","last_became_peered":"2024-05-03T18:22:21.549786+0000","last_unstale":"2024-05-03T18:27:12.135624+0000","last_undegraded":"2024-05-03T18:27:12.135624+0000","last_fullsized":"2024-05-03T18:27:12.135624+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:08:18.513405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"90'6","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.837679+0000","last_change":"2024-05-03T18:22:25.596437+0000","last_active":"2024-05-03T18:27:11.837679+0000","last_peered":"2024-05-03T18:27:11.837679+0000","last_clean":"2024-05-03T18:27:11.837679+0000","last_became_active":"2024-05-03T18:22:25.594040+0000","last_became_peered":"2024-05-03T18:22:25.594040+0000","last_unstale":"2024-05-03T18:27:11.837679+0000","last_undegraded":"2024-05-03T18:27:11.837679+0000","last_fullsized":"2024-05-03T18:27:11.837679+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:09:36.473565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.741898+0000","last_change":"2024-05-03T18:22:40.754641+0000","last_active":"2024-05-03T18:27:11.741898+0000","last_peered":"2024-05-03T18:27:11.741898+0000","last_clean":"2024-05-03T18:27:11.741898+0000","last_became_active":"2024-05-03T18:22:40.754166+0000","last_became_peered":"2024-05-03T18:22:40.754166+0000","last_unstale":"2024-05-03T18:27:11.741898+0000","last_undegraded":"2024-05-03T18:27:11.741898+0000","last_fullsized":"2024-05-03T18:27:11.741898+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:39.698433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:39.698433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:29:50.021671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.741202+0000","last_change":"2024-05-03T18:22:21.559354+0000","last_active":"2024-05-03T18:27:11.741202+0000","last_peered":"2024-05-03T18:27:11.741202+0000","last_clean":"2024-05-03T18:27:11.741202+0000","last_became_active":"2024-05-03T18:22:21.558680+0000","last_became_peered":"2024-05-03T18:22:21.558680+0000","last_unstale":"2024-05-03T18:27:11.741202+0000","last_undegraded":"2024-05-03T18:27:11.741202+0000","last_fullsized":"2024-05-03T18:27:11.741202+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:20.493263+0000","last_clean_scrub_stamp":"2024-05-03T18:22:20.493263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:13:04.001847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.837002+0000","last_change":"2024-05-03T18:22:25.590746+0000","last_active":"2024-05-03T18:27:11.837002+0000","last_peered":"2024-05-03T18:27:11.837002+0000","last_clean":"2024-05-03T18:27:11.837002+0000","last_became_active":"2024-05-03T18:22:25.587906+0000","last_became_peered":"2024-05-03T18:22:25.587906+0000","last_unstale":"2024-05-03T18:27:11.837002+0000","last_undegraded":"2024-05-03T18:27:11.837002+0000","last_fullsized":"2024-05-03T18:27:11.837002+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:19:43.410259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.833036+0000","last_change":"2024-05-03T18:22:25.597628+0000","last_active":"2024-05-03T18:27:11.833036+0000","last_peered":"2024-05-03T18:27:11.833036+0000","last_clean":"2024-05-03T18:27:11.833036+0000","last_became_active":"2024-05-03T18:22:25.596535+0000","last_became_peered":"2024-05-03T18:22:25.596535+0000","last_unstale":"2024-05-03T18:27:11.833036+0000","last_undegraded":"2024-05-03T18:27:11.833036+0000","last_fullsized":"2024-05-03T18:27:11.833036+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:54:52.632258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.829560+0000","last_change":"2024-05-03T18:22:25.601474+0000","last_active":"2024-05-03T18:27:11.829560+0000","last_peered":"2024-05-03T18:27:11.829560+0000","last_clean":"2024-05-03T18:27:11.829560+0000","last_became_active":"2024-05-03T18:22:25.601097+0000","last_became_peered":"2024-05-03T18:22:25.601097+0000","last_unstale":"2024-05-03T18:27:11.829560+0000","last_undegraded":"2024-05-03T18:27:11.829560+0000","last_fullsized":"2024-05-03T18:27:11.829560+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:14:46.001500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":56,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.820221+0000","last_change":"2024-05-03T18:23:20.535409+0000","last_active":"2024-05-03T18:27:11.820221+0000","last_peered":"2024-05-03T18:27:11.820221+0000","last_clean":"2024-05-03T18:27:11.820221+0000","last_became_active":"2024-05-03T18:23:20.534883+0000","last_became_peered":"2024-05-03T18:23:20.534883+0000","last_unstale":"2024-05-03T18:27:11.820221+0000","last_undegraded":"2024-05-03T18:27:11.820221+0000","last_fullsized":"2024-05-03T18:27:11.820221+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:26:37.252898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.835637+0000","last_change":"2024-05-03T18:22:25.607115+0000","last_active":"2024-05-03T18:27:11.835637+0000","last_peered":"2024-05-03T18:27:11.835637+0000","last_clean":"2024-05-03T18:27:11.835637+0000","last_became_active":"2024-05-03T18:22:25.606764+0000","last_became_peered":"2024-05-03T18:22:25.606764+0000","last_unstale":"2024-05-03T18:27:11.835637+0000","last_undegraded":"2024-05-03T18:27:11.835637+0000","last_fullsized":"2024-05-03T18:27:11.835637+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:44:37.165032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.842362+0000","last_change":"2024-05-03T18:22:25.601886+0000","last_active":"2024-05-03T18:27:11.842362+0000","last_peered":"2024-05-03T18:27:11.842362+0000","last_clean":"2024-05-03T18:27:11.842362+0000","last_became_active":"2024-05-03T18:22:25.601494+0000","last_became_peered":"2024-05-03T18:22:25.601494+0000","last_unstale":"2024-05-03T18:27:11.842362+0000","last_undegraded":"2024-05-03T18:27:11.842362+0000","last_fullsized":"2024-05-03T18:27:11.842362+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:35:30.080124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.832632+0000","last_change":"2024-05-03T18:23:20.529091+0000","last_active":"2024-05-03T18:27:11.832632+0000","last_peered":"2024-05-03T18:27:11.832632+0000","last_clean":"2024-05-03T18:27:11.832632+0000","last_became_active":"2024-05-03T18:23:20.528288+0000","last_became_peered":"2024-05-03T18:23:20.528288+0000","last_unstale":"2024-05-03T18:27:11.832632+0000","last_undegraded":"2024-05-03T18:27:11.832632+0000","last_fullsized":"2024-05-03T18:27:11.832632+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:32:25.749461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":57,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.823878+0000","last_change":"2024-05-03T18:22:25.595367+0000","last_active":"2024-05-03T18:27:11.823878+0000","last_peered":"2024-05-03T18:27:11.823878+0000","last_clean":"2024-05-03T18:27:11.823878+0000","last_became_active":"2024-05-03T18:22:25.594987+0000","last_became_peered":"2024-05-03T18:22:25.594987+0000","last_unstale":"2024-05-03T18:27:11.823878+0000","last_undegraded":"2024-05-03T18:27:11.823878+0000","last_fullsized":"2024-05-03T18:27:11.823878+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:40:55.385595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.823841+0000","last_change":"2024-05-03T18:22:25.588320+0000","last_active":"2024-05-03T18:27:11.823841+0000","last_peered":"2024-05-03T18:27:11.823841+0000","last_clean":"2024-05-03T18:27:11.823841+0000","last_became_active":"2024-05-03T18:22:25.587861+0000","last_became_peered":"2024-05-03T18:22:25.587861+0000","last_unstale":"2024-05-03T18:27:11.823841+0000","last_undegraded":"2024-05-03T18:27:11.823841+0000","last_fullsized":"2024-05-03T18:27:11.823841+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:17:02.538798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.831427+0000","last_change":"2024-05-03T18:22:25.597874+0000","last_active":"2024-05-03T18:27:11.831427+0000","last_peered":"2024-05-03T18:27:11.831427+0000","last_clean":"2024-05-03T18:27:11.831427+0000","last_became_active":"2024-05-03T18:22:25.597448+0000","last_became_peered":"2024-05-03T18:22:25.597448+0000","last_unstale":"2024-05-03T18:27:11.831427+0000","last_undegraded":"2024-05-03T18:27:11.831427+0000","last_fullsized":"2024-05-03T18:27:11.831427+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:55:52.894491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.836134+0000","last_change":"2024-05-03T18:22:25.593509+0000","last_active":"2024-05-03T18:27:11.836134+0000","last_peered":"2024-05-03T18:27:11.836134+0000","last_clean":"2024-05-03T18:27:11.836134+0000","last_became_active":"2024-05-03T18:22:25.590736+0000","last_became_peered":"2024-05-03T18:22:25.590736+0000","last_unstale":"2024-05-03T18:27:11.836134+0000","last_undegraded":"2024-05-03T18:27:11.836134+0000","last_fullsized":"2024-05-03T18:27:11.836134+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:54:45.431776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.837344+0000","last_change":"2024-05-03T18:22:25.590022+0000","last_active":"2024-05-03T18:27:11.837344+0000","last_peered":"2024-05-03T18:27:11.837344+0000","last_clean":"2024-05-03T18:27:11.837344+0000","last_became_active":"2024-05-03T18:22:25.589432+0000","last_became_peered":"2024-05-03T18:22:25.589432+0000","last_unstale":"2024-05-03T18:27:11.837344+0000","last_undegraded":"2024-05-03T18:27:11.837344+0000","last_fullsized":"2024-05-03T18:27:11.837344+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:03:02.091228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":53,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.841863+0000","last_change":"2024-05-03T18:22:25.581452+0000","last_active":"2024-05-03T18:27:11.841863+0000","last_peered":"2024-05-03T18:27:11.841863+0000","last_clean":"2024-05-03T18:27:11.841863+0000","last_became_active":"2024-05-03T18:22:25.581098+0000","last_became_peered":"2024-05-03T18:22:25.581098+0000","last_unstale":"2024-05-03T18:27:11.841863+0000","last_undegraded":"2024-05-03T18:27:11.841863+0000","last_fullsized":"2024-05-03T18:27:11.841863+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:04:46.551234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-03T18:27:11.823559+0000","last_change":"2024-05-03T18:22:25.589105+0000","last_active":"2024-05-03T18:27:11.823559+0000","last_peered":"2024-05-03T18:27:11.823559+0000","last_clean":"2024-05-03T18:27:11.823559+0000","last_became_active":"2024-05-03T18:22:25.588697+0000","last_became_peered":"2024-05-03T18:22:25.588697+0000","last_unstale":"2024-05-03T18:27:11.823559+0000","last_undegraded":"2024-05-03T18:27:11.823559+0000","last_fullsized":"2024-05-03T18:27:11.823559+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:22:24.542433+0000","last_clean_scrub_stamp":"2024-05-03T18:22:24.542433+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:41:23.563296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":594853,"num_objects":24,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":523,"num_read_kb":366,"num_write":203,"num_write_kb":1410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1880064,"data_stored":1784559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":151,"ondisk_log_size":151,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710797,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1036288,"data_stored":524266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":10,"up_from":64,"seq":274877907027,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28884,"kb_used_data":1808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716268,"statfs":{"total":95995035648,"available":95965458432,"internally_reserved":0,"allocated":1851392,"data_stored":1311571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070555,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36744,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708408,"statfs":{"total":95995035648,"available":95957409792,"internally_reserved":0,"allocated":1437696,"data_stored":922160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266785,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1044480,"data_stored":524870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":47,"seq":201863463018,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32780,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712372,"statfs":{"total":95995035648,"available":95961468928,"internally_reserved":0,"allocated":1638400,"data_stored":1119312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626545,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1032192,"data_stored":520170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574996,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32264,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712888,"statfs":{"total":95995035648,"available":95961997312,"internally_reserved":0,"allocated":1110016,"data_stored":588627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738523,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1036288,"data_stored":520260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411470,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32268,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712884,"statfs":{"total":95995035648,"available":95961993216,"internally_reserved":0,"allocated":1114112,"data_stored":588661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182532,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28548,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716604,"statfs":{"total":95995035648,"available":95965802496,"internally_reserved":0,"allocated":1568768,"data_stored":1045090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986302,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1040384,"data_stored":524288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822775,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32912,"kb_used_data":1736,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712240,"statfs":{"total":95995035648,"available":95961333760,"internally_reserved":0,"allocated":1777664,"data_stored":1246416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":69239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":69273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":536576,"data_stored":524920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":131782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":393820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":397312,"data_stored":393798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":225280,"data_stored":201033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-03T18:27:18.078 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T18:27:18.566 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:27:18.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:18 smithi150 ceph-mon[32492]: from='client.15222 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-03T18:27:18.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:18 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:27:18.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:18 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:27:18.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:18 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:27:18.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:18 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:27:18.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:18 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:27:18.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:18 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:27:18.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:18 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:27:18.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:18 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:27:18.634 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:18 smithi150 ceph-mon[32492]: Adjusting osd_memory_target on smithi099 to 176.4M 2024-05-03T18:27:18.634 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:18 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:27:18.634 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:18 smithi150 ceph-mon[32492]: Unable to set osd_memory_target on smithi099 to 185040486: error parsing value: Value '185040486' is below minimum 939524096 2024-05-03T18:27:18.634 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:18 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:27:18.634 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:18 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:27:18.634 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:18 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:27:18.634 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:18 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:27:18.634 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:18 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:27:18.634 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:18 smithi150 ceph-mon[32492]: Adjusting osd_memory_target on smithi150 to 176.4M 2024-05-03T18:27:18.634 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:18 smithi150 ceph-mon[32492]: Unable to set osd_memory_target on smithi150 to 185064857: error parsing value: Value '185064857' is below minimum 939524096 2024-05-03T18:27:18.634 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:18 smithi150 ceph-mon[32492]: from='client.15228 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-03T18:27:18.634 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:18 smithi150 ceph-mon[32492]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:27:18.653 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:18 smithi033 ceph-mon[28330]: from='client.15222 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-03T18:27:18.653 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:18 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:27:18.653 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:18 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:27:18.653 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:18 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:27:18.653 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:18 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:27:18.653 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:18 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:27:18.654 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:18 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:27:18.654 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:18 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:27:18.654 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:18 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:27:18.654 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:18 smithi033 ceph-mon[28330]: Adjusting osd_memory_target on smithi099 to 176.4M 2024-05-03T18:27:18.654 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:18 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:27:18.654 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:18 smithi033 ceph-mon[28330]: Unable to set osd_memory_target on smithi099 to 185040486: error parsing value: Value '185040486' is below minimum 939524096 2024-05-03T18:27:18.654 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:18 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:27:18.654 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:18 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:27:18.654 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:18 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:27:18.654 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:18 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:27:18.654 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:18 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:27:18.654 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:18 smithi033 ceph-mon[28330]: Adjusting osd_memory_target on smithi150 to 176.4M 2024-05-03T18:27:18.654 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:18 smithi033 ceph-mon[28330]: Unable to set osd_memory_target on smithi150 to 185064857: error parsing value: Value '185064857' is below minimum 939524096 2024-05-03T18:27:18.654 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:18 smithi033 ceph-mon[28330]: from='client.15228 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-03T18:27:18.654 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:18 smithi033 ceph-mon[28330]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:27:18.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:18 smithi099 ceph-mon[31658]: from='client.15222 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-03T18:27:18.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:18 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:27:18.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:18 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:27:18.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:18 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:27:18.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:18 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:27:18.797 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:18 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:27:18.797 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:18 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:27:18.797 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:18 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:27:18.797 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:18 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:27:18.797 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:18 smithi099 ceph-mon[31658]: Adjusting osd_memory_target on smithi099 to 176.4M 2024-05-03T18:27:18.797 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:18 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:27:18.797 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:18 smithi099 ceph-mon[31658]: Unable to set osd_memory_target on smithi099 to 185040486: error parsing value: Value '185040486' is below minimum 939524096 2024-05-03T18:27:18.797 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:18 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:27:18.797 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:18 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:27:18.797 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:18 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:27:18.797 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:18 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:27:18.797 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:18 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:27:18.797 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:18 smithi099 ceph-mon[31658]: Adjusting osd_memory_target on smithi150 to 176.4M 2024-05-03T18:27:18.797 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:18 smithi099 ceph-mon[31658]: Unable to set osd_memory_target on smithi150 to 185064857: error parsing value: Value '185064857' is below minimum 939524096 2024-05-03T18:27:18.798 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:18 smithi099 ceph-mon[31658]: from='client.15228 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-03T18:27:18.798 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:18 smithi099 ceph-mon[31658]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:27:19.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:19 smithi033 ceph-mon[28330]: from='client.15234 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-03T18:27:19.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:19 smithi099 ceph-mon[31658]: from='client.15234 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-03T18:27:19.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:19 smithi150 ceph-mon[32492]: from='client.15234 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-03T18:27:20.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:20 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:27:20.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:20 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:27:20.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:20 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:27:20.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:20 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:27:20.735 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:20 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:27:20.735 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:20 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:27:20.735 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:20 smithi033 ceph-mon[28330]: Adjusting osd_memory_target on smithi033 to 177.5M 2024-05-03T18:27:20.735 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:20 smithi033 ceph-mon[28330]: Unable to set osd_memory_target on smithi033 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-05-03T18:27:20.735 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:20 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:27:20.735 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:20 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:27:20.735 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:20 smithi033 ceph-mon[28330]: Updating smithi033:/etc/ceph/ceph.conf 2024-05-03T18:27:20.735 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:20 smithi033 ceph-mon[28330]: Updating smithi099:/etc/ceph/ceph.conf 2024-05-03T18:27:20.735 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:20 smithi033 ceph-mon[28330]: Updating smithi150:/etc/ceph/ceph.conf 2024-05-03T18:27:20.735 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:20 smithi033 ceph-mon[28330]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 0 B/s wr, 12 op/s 2024-05-03T18:27:20.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:20 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:27:20.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:20 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:27:20.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:20 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:27:20.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:20 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:27:20.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:20 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:27:20.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:20 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:27:20.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:20 smithi099 ceph-mon[31658]: Adjusting osd_memory_target on smithi033 to 177.5M 2024-05-03T18:27:20.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:20 smithi099 ceph-mon[31658]: Unable to set osd_memory_target on smithi033 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-05-03T18:27:20.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:20 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:27:20.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:20 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:27:20.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:20 smithi099 ceph-mon[31658]: Updating smithi033:/etc/ceph/ceph.conf 2024-05-03T18:27:20.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:20 smithi099 ceph-mon[31658]: Updating smithi099:/etc/ceph/ceph.conf 2024-05-03T18:27:20.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:20 smithi099 ceph-mon[31658]: Updating smithi150:/etc/ceph/ceph.conf 2024-05-03T18:27:20.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:20 smithi099 ceph-mon[31658]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 0 B/s wr, 12 op/s 2024-05-03T18:27:20.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:20 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:27:20.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:20 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:27:20.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:20 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:27:20.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:20 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:27:20.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:20 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:27:20.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:20 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:27:20.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:20 smithi150 ceph-mon[32492]: Adjusting osd_memory_target on smithi033 to 177.5M 2024-05-03T18:27:20.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:20 smithi150 ceph-mon[32492]: Unable to set osd_memory_target on smithi033 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-05-03T18:27:20.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:20 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:27:20.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:20 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:27:20.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:20 smithi150 ceph-mon[32492]: Updating smithi033:/etc/ceph/ceph.conf 2024-05-03T18:27:20.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:20 smithi150 ceph-mon[32492]: Updating smithi099:/etc/ceph/ceph.conf 2024-05-03T18:27:20.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:20 smithi150 ceph-mon[32492]: Updating smithi150:/etc/ceph/ceph.conf 2024-05-03T18:27:20.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:20 smithi150 ceph-mon[32492]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 0 B/s wr, 12 op/s 2024-05-03T18:27:20.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:20.929+0000 7efe92600700 1 -- 172.21.15.33:0/514986388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe8c075ff0 msgr2=0x7efe8c10f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:20.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:20.929+0000 7efe92600700 1 --2- 172.21.15.33:0/514986388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe8c075ff0 0x7efe8c10f960 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7efe78009990 tx=0x7efe78009ca0 comp rx=0 tx=0).stop 2024-05-03T18:27:20.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:20.929+0000 7efe92600700 1 -- 172.21.15.33:0/514986388 shutdown_connections 2024-05-03T18:27:20.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:20.929+0000 7efe92600700 1 --2- 172.21.15.33:0/514986388 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe8c10fea0 0x7efe8c112290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:20.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:20.929+0000 7efe92600700 1 --2- 172.21.15.33:0/514986388 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe8c0756d0 0x7efe8c075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:20.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:20.929+0000 7efe92600700 1 --2- 172.21.15.33:0/514986388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe8c075ff0 0x7efe8c10f960 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:20.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:20.929+0000 7efe92600700 1 -- 172.21.15.33:0/514986388 >> 172.21.15.33:0/514986388 conn(0x7efe8c07a600 msgr2=0x7efe8c07aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:27:20.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:20.929+0000 7efe92600700 1 -- 172.21.15.33:0/514986388 shutdown_connections 2024-05-03T18:27:20.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:20.929+0000 7efe92600700 1 -- 172.21.15.33:0/514986388 wait complete. 2024-05-03T18:27:20.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:20.930+0000 7efe92600700 1 Processor -- start 2024-05-03T18:27:20.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:20.930+0000 7efe92600700 1 -- start start 2024-05-03T18:27:20.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:20.931+0000 7efe92600700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe8c0756d0 0x7efe8c1a1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:20.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:20.931+0000 7efe92600700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe8c075ff0 0x7efe8c1a1e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:20.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:20.931+0000 7efe92600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe8c10fea0 0x7efe8c19bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:20.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:20.931+0000 7efe92600700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7efe8c115780 con 0x7efe8c075ff0 2024-05-03T18:27:20.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:20.931+0000 7efe92600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7efe8c115600 con 0x7efe8c10fea0 2024-05-03T18:27:20.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:20.931+0000 7efe92600700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7efe8c115900 con 0x7efe8c0756d0 2024-05-03T18:27:20.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:20.931+0000 7efe8aa00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe8c075ff0 0x7efe8c1a1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:20.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:20.931+0000 7efe8b400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe8c0756d0 0x7efe8c1a1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:20.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:20.931+0000 7efe8aa00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe8c075ff0 0x7efe8c1a1e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50776/0 (socket says 172.21.15.33:50776) 2024-05-03T18:27:20.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:20.931+0000 7efe8aa00700 1 -- 172.21.15.33:0/256808453 learned_addr learned my addr 172.21.15.33:0/256808453 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:27:20.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:20.931+0000 7efe8be00700 1 --2- 172.21.15.33:0/256808453 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe8c10fea0 0x7efe8c19bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:20.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:20.932+0000 7efe8b400700 1 -- 172.21.15.33:0/256808453 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe8c10fea0 msgr2=0x7efe8c19bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:20.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:20.932+0000 7efe8b400700 1 --2- 172.21.15.33:0/256808453 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe8c10fea0 0x7efe8c19bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:20.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:20.932+0000 7efe8b400700 1 -- 172.21.15.33:0/256808453 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe8c075ff0 msgr2=0x7efe8c1a1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:20.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:20.932+0000 7efe8b400700 1 --2- 172.21.15.33:0/256808453 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe8c075ff0 0x7efe8c1a1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:20.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:20.932+0000 7efe8b400700 1 -- 172.21.15.33:0/256808453 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe8c19c240 con 0x7efe8c0756d0 2024-05-03T18:27:20.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:20.932+0000 7efe8aa00700 1 --2- 172.21.15.33:0/256808453 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe8c075ff0 0x7efe8c1a1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:27:20.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:20.932+0000 7efe8b400700 1 --2- 172.21.15.33:0/256808453 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe8c0756d0 0x7efe8c1a1920 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7efe7c00d750 tx=0x7efe7c00da60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:27:20.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:20.932+0000 7efe83e00700 1 -- 172.21.15.33:0/256808453 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe7c014030 con 0x7efe8c0756d0 2024-05-03T18:27:20.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:20.932+0000 7efe92600700 1 -- 172.21.15.33:0/256808453 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe8c19c530 con 0x7efe8c0756d0 2024-05-03T18:27:20.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:20.932+0000 7efe83e00700 1 -- 172.21.15.33:0/256808453 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efe7c0044b0 con 0x7efe8c0756d0 2024-05-03T18:27:20.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:20.932+0000 7efe83e00700 1 -- 172.21.15.33:0/256808453 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe7c005280 con 0x7efe8c0756d0 2024-05-03T18:27:20.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:20.933+0000 7efe92600700 1 -- 172.21.15.33:0/256808453 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe8c1a87f0 con 0x7efe8c0756d0 2024-05-03T18:27:20.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:20.934+0000 7efe92600700 1 -- 172.21.15.33:0/256808453 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe8c06a190 con 0x7efe8c0756d0 2024-05-03T18:27:20.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:20.937+0000 7efe83e00700 1 -- 172.21.15.33:0/256808453 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 23) ==== 149828+0+0 (secure 0 0 0) 0x7efe7c0049e0 con 0x7efe8c0756d0 2024-05-03T18:27:20.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:20.938+0000 7efe83e00700 1 --2- 172.21.15.33:0/256808453 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7efe700b2b00 0x7efe700b4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:20.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:20.939+0000 7efe83e00700 1 -- 172.21.15.33:0/256808453 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7efe7c0e2780 con 0x7efe8c0756d0 2024-05-03T18:27:20.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:20.939+0000 7efe8aa00700 1 --2- 172.21.15.33:0/256808453 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7efe700b2b00 0x7efe700b4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:20.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:20.939+0000 7efe8aa00700 1 --2- 172.21.15.33:0/256808453 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7efe700b2b00 0x7efe700b4fc0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7efe78003760 tx=0x7efe7801a040 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:27:20.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:20.944+0000 7efe83e00700 1 -- 172.21.15.33:0/256808453 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7efe7c0a9c00 con 0x7efe8c0756d0 2024-05-03T18:27:21.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:21.272+0000 7efe92600700 1 -- 172.21.15.33:0/256808453 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efe8c19cfd0 con 0x7efe8c0756d0 2024-05-03T18:27:21.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:21.274+0000 7efe83e00700 1 -- 172.21.15.33:0/256808453 <== mon.1 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11801 (secure 0 0 0) 0x7efe7c0a9500 con 0x7efe8c0756d0 2024-05-03T18:27:21.275 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:27:21.275 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1327393503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1327393503},{"type":"v1","addr":"172.21.15.33:6835","nonce":1327393503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/2461472948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":2461472948},{"type":"v1","addr":"172.21.15.33:6837","nonce":2461472948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14559,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/442269281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":442269281},{"type":"v1","addr":"172.21.15.33:6839","nonce":442269281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14571,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/645571756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":645571756},{"type":"v1","addr":"172.21.15.33:6841","nonce":645571756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/203598909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":203598909},{"type":"v1","addr":"172.21.15.150:6833","nonce":203598909}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/4029842493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":4029842493},{"type":"v1","addr":"172.21.15.150:6835","nonce":4029842493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3463664423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3463664423},{"type":"v1","addr":"172.21.15.150:6837","nonce":3463664423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24481,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6839/4051648018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":4051648018},{"type":"v1","addr":"172.21.15.150:6839","nonce":4051648018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34418,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1865105479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1865105479},{"type":"v1","addr":"172.21.15.99:6833","nonce":1865105479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34445,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/1206393914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1206393914},{"type":"v1","addr":"172.21.15.99:6835","nonce":1206393914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-03T18:22:28.837091+0000","modified":"2024-05-03T18:23:49.758398+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34508},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34472":{"gid":34472,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6837/3360970342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3360970342},{"type":"v1","addr":"172.21.15.99:6837","nonce":3360970342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34508":{"gid":34508,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/919907308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":919907308},{"type":"v1","addr":"172.21.15.99:6839","nonce":919907308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34508,"qdb_cluster":[34508]},"id":1}]} 2024-05-03T18:27:21.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:21.277+0000 7efe92600700 1 -- 172.21.15.33:0/256808453 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7efe700b2b00 msgr2=0x7efe700b4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:21.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:21.277+0000 7efe92600700 1 --2- 172.21.15.33:0/256808453 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7efe700b2b00 0x7efe700b4fc0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7efe78003760 tx=0x7efe7801a040 comp rx=0 tx=0).stop 2024-05-03T18:27:21.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:21.277+0000 7efe92600700 1 -- 172.21.15.33:0/256808453 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe8c0756d0 msgr2=0x7efe8c1a1920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:21.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:21.277+0000 7efe92600700 1 --2- 172.21.15.33:0/256808453 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe8c0756d0 0x7efe8c1a1920 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7efe7c00d750 tx=0x7efe7c00da60 comp rx=0 tx=0).stop 2024-05-03T18:27:21.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:21.277+0000 7efe92600700 1 -- 172.21.15.33:0/256808453 shutdown_connections 2024-05-03T18:27:21.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:21.277+0000 7efe92600700 1 --2- 172.21.15.33:0/256808453 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7efe700b2b00 0x7efe700b4fc0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:21.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:21.277+0000 7efe92600700 1 --2- 172.21.15.33:0/256808453 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe8c10fea0 0x7efe8c19bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:21.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:21.277+0000 7efe92600700 1 --2- 172.21.15.33:0/256808453 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe8c0756d0 0x7efe8c1a1920 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:21.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:21.277+0000 7efe92600700 1 --2- 172.21.15.33:0/256808453 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe8c075ff0 0x7efe8c1a1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:21.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:21.277+0000 7efe92600700 1 -- 172.21.15.33:0/256808453 >> 172.21.15.33:0/256808453 conn(0x7efe8c07a600 msgr2=0x7efe8c100810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:27:21.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:21.278+0000 7efe92600700 1 -- 172.21.15.33:0/256808453 shutdown_connections 2024-05-03T18:27:21.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:21.278+0000 7efe92600700 1 -- 172.21.15.33:0/256808453 wait complete. 2024-05-03T18:27:21.279 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 21 2024-05-03T18:27:21.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:21 smithi099 ceph-mon[31658]: Updating smithi099:/var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-03T18:27:21.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:21 smithi099 ceph-mon[31658]: Updating smithi150:/var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-03T18:27:21.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:21 smithi099 ceph-mon[31658]: Updating smithi033:/var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-03T18:27:21.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:21 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/256808453' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:27:21.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:21 smithi150 ceph-mon[32492]: Updating smithi099:/var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-03T18:27:21.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:21 smithi150 ceph-mon[32492]: Updating smithi150:/var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-03T18:27:21.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:21 smithi150 ceph-mon[32492]: Updating smithi033:/var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-03T18:27:21.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:21 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/256808453' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:27:21.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:21 smithi033 ceph-mon[28330]: Updating smithi099:/var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-03T18:27:21.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:21 smithi033 ceph-mon[28330]: Updating smithi150:/var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-03T18:27:21.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:21 smithi033 ceph-mon[28330]: Updating smithi033:/var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-03T18:27:21.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:21 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/256808453' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:27:22.234 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-03T18:27:22.693 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:27:22.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:22 smithi099 ceph-mon[31658]: Updating smithi033:/etc/ceph/ceph.client.admin.keyring 2024-05-03T18:27:22.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:22 smithi099 ceph-mon[31658]: Updating smithi099:/etc/ceph/ceph.client.admin.keyring 2024-05-03T18:27:22.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:22 smithi099 ceph-mon[31658]: Updating smithi150:/etc/ceph/ceph.client.admin.keyring 2024-05-03T18:27:22.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:22 smithi099 ceph-mon[31658]: Updating smithi033:/var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-03T18:27:22.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:22 smithi099 ceph-mon[31658]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 0 B/s wr, 9 op/s 2024-05-03T18:27:22.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:22 smithi099 ceph-mon[31658]: Updating smithi150:/var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-03T18:27:22.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:22 smithi099 ceph-mon[31658]: Updating smithi099:/var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-03T18:27:22.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:22 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:27:22.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:22 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:27:22.844 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:22 smithi033 ceph-mon[28330]: Updating smithi033:/etc/ceph/ceph.client.admin.keyring 2024-05-03T18:27:22.844 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:22 smithi033 ceph-mon[28330]: Updating smithi099:/etc/ceph/ceph.client.admin.keyring 2024-05-03T18:27:22.844 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:22 smithi033 ceph-mon[28330]: Updating smithi150:/etc/ceph/ceph.client.admin.keyring 2024-05-03T18:27:22.844 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:22 smithi033 ceph-mon[28330]: Updating smithi033:/var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-03T18:27:22.844 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:22 smithi033 ceph-mon[28330]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 0 B/s wr, 9 op/s 2024-05-03T18:27:22.844 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:22 smithi033 ceph-mon[28330]: Updating smithi150:/var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-03T18:27:22.844 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:22 smithi033 ceph-mon[28330]: Updating smithi099:/var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-03T18:27:22.844 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:22 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:27:22.844 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:22 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:27:22.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:22 smithi150 ceph-mon[32492]: Updating smithi033:/etc/ceph/ceph.client.admin.keyring 2024-05-03T18:27:22.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:22 smithi150 ceph-mon[32492]: Updating smithi099:/etc/ceph/ceph.client.admin.keyring 2024-05-03T18:27:22.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:22 smithi150 ceph-mon[32492]: Updating smithi150:/etc/ceph/ceph.client.admin.keyring 2024-05-03T18:27:22.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:22 smithi150 ceph-mon[32492]: Updating smithi033:/var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-03T18:27:22.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:22 smithi150 ceph-mon[32492]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 0 B/s wr, 9 op/s 2024-05-03T18:27:22.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:22 smithi150 ceph-mon[32492]: Updating smithi150:/var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-03T18:27:22.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:22 smithi150 ceph-mon[32492]: Updating smithi099:/var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-03T18:27:22.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:22 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:27:22.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:22 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:27:23.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:23 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:27:23.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:23 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:27:23.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:23 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:27:23.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:23 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:27:23.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:23 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:27:23.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:23 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:27:23.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:23 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:27:23.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:23 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:27:23.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:23 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:27:23.876 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:23 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:27:23.876 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:23 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:27:23.876 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:23 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:27:23.876 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:23 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:27:23.876 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:23 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:27:23.877 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:23 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:27:23.877 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:23 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:27:23.877 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:23 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:27:23.877 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:23 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:27:23.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:23 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:27:23.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:23 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:27:23.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:23 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:27:23.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:23 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:27:23.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:23 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:27:23.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:23 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:27:23.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:23 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:27:23.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:23 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:27:23.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:23 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:27:24.746 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:24 smithi033 ceph-mon[28330]: pgmap v9: 129 pgs: 129 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 279 B/s wr, 8 op/s 2024-05-03T18:27:24.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:24 smithi150 ceph-mon[32492]: pgmap v9: 129 pgs: 129 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 279 B/s wr, 8 op/s 2024-05-03T18:27:25.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:25.033+0000 7facace00700 1 -- 172.21.15.33:0/2445424598 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faca810b6c0 msgr2=0x7faca810baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:25.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:25.033+0000 7facace00700 1 --2- 172.21.15.33:0/2445424598 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faca810b6c0 0x7faca810baa0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fac9800b0a0 tx=0x7fac9800b3b0 comp rx=0 tx=0).stop 2024-05-03T18:27:25.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:25.033+0000 7facace00700 1 -- 172.21.15.33:0/2445424598 shutdown_connections 2024-05-03T18:27:25.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:25.033+0000 7facace00700 1 --2- 172.21.15.33:0/2445424598 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faca8106080 0x7faca8112590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:25.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:25.033+0000 7facace00700 1 --2- 172.21.15.33:0/2445424598 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faca810b6c0 0x7faca810baa0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:25.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:25.033+0000 7facace00700 1 --2- 172.21.15.33:0/2445424598 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faca81056c0 0x7faca8105b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:25.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:25.033+0000 7facace00700 1 -- 172.21.15.33:0/2445424598 >> 172.21.15.33:0/2445424598 conn(0x7faca81011b0 msgr2=0x7faca81035d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:27:25.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:25.034+0000 7facace00700 1 -- 172.21.15.33:0/2445424598 shutdown_connections 2024-05-03T18:27:25.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:25.034+0000 7facace00700 1 -- 172.21.15.33:0/2445424598 wait complete. 2024-05-03T18:27:25.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:25.035+0000 7facace00700 1 Processor -- start 2024-05-03T18:27:25.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:25.035+0000 7facace00700 1 -- start start 2024-05-03T18:27:25.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:25.035+0000 7facace00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faca81056c0 0x7faca819bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:25.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:25.035+0000 7facace00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faca8106080 0x7faca819c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:25.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:25.035+0000 7facace00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faca810b6c0 0x7faca81a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:25.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:25.035+0000 7facace00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7faca81157b0 con 0x7faca8106080 2024-05-03T18:27:25.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:25.035+0000 7facace00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7faca8115630 con 0x7faca810b6c0 2024-05-03T18:27:25.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:25.035+0000 7facace00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7faca8115930 con 0x7faca81056c0 2024-05-03T18:27:25.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:25.036+0000 7fac9d400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faca8106080 0x7faca819c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:25.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:25.036+0000 7fac9d400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faca8106080 0x7faca819c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50802/0 (socket says 172.21.15.33:50802) 2024-05-03T18:27:25.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:25.036+0000 7fac9d400700 1 -- 172.21.15.33:0/1641190690 learned_addr learned my addr 172.21.15.33:0/1641190690 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:27:25.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:25.036+0000 7faca5e00700 1 --2- 172.21.15.33:0/1641190690 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faca81056c0 0x7faca819bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:25.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:25.036+0000 7faca6800700 1 --2- 172.21.15.33:0/1641190690 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faca810b6c0 0x7faca81a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:25.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:25.036+0000 7fac9d400700 1 -- 172.21.15.33:0/1641190690 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faca81056c0 msgr2=0x7faca819bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:25.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:25.036+0000 7fac9d400700 1 --2- 172.21.15.33:0/1641190690 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faca81056c0 0x7faca819bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:25.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:25.036+0000 7fac9d400700 1 -- 172.21.15.33:0/1641190690 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faca810b6c0 msgr2=0x7faca81a0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:25.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:25.036+0000 7fac9d400700 1 --2- 172.21.15.33:0/1641190690 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faca810b6c0 0x7faca81a0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:25.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:25.036+0000 7fac9d400700 1 -- 172.21.15.33:0/1641190690 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faca81a0d50 con 0x7faca8106080 2024-05-03T18:27:25.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:25.037+0000 7fac9d400700 1 --2- 172.21.15.33:0/1641190690 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faca8106080 0x7faca819c280 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fac9000c9b0 tx=0x7fac9000cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:27:25.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:25.037+0000 7fac9f400700 1 -- 172.21.15.33:0/1641190690 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac90007b00 con 0x7faca8106080 2024-05-03T18:27:25.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:25.037+0000 7fac9f400700 1 -- 172.21.15.33:0/1641190690 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fac90007ca0 con 0x7faca8106080 2024-05-03T18:27:25.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:25.037+0000 7fac9f400700 1 -- 172.21.15.33:0/1641190690 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac90018660 con 0x7faca8106080 2024-05-03T18:27:25.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:25.037+0000 7facace00700 1 -- 172.21.15.33:0/1641190690 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faca81a1040 con 0x7faca8106080 2024-05-03T18:27:25.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:25.037+0000 7facace00700 1 -- 172.21.15.33:0/1641190690 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faca81a88c0 con 0x7faca8106080 2024-05-03T18:27:25.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:25.038+0000 7facace00700 1 -- 172.21.15.33:0/1641190690 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faca806a190 con 0x7faca8106080 2024-05-03T18:27:25.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:25.041+0000 7fac9f400700 1 -- 172.21.15.33:0/1641190690 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 23) ==== 149828+0+0 (secure 0 0 0) 0x7fac9000f450 con 0x7faca8106080 2024-05-03T18:27:25.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:25.042+0000 7fac9f400700 1 --2- 172.21.15.33:0/1641190690 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fac8c0b2b00 0x7fac8c0b4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:25.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:25.042+0000 7fac9f400700 1 -- 172.21.15.33:0/1641190690 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7fac900e2e80 con 0x7faca8106080 2024-05-03T18:27:25.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:25.043+0000 7faca5e00700 1 --2- 172.21.15.33:0/1641190690 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fac8c0b2b00 0x7fac8c0b4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:25.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:25.043+0000 7faca5e00700 1 --2- 172.21.15.33:0/1641190690 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fac8c0b2b00 0x7fac8c0b4fc0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fac98009fd0 tx=0x7fac9800b7b0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:27:25.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:25.045+0000 7fac9f400700 1 -- 172.21.15.33:0/1641190690 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fac90014030 con 0x7faca8106080 2024-05-03T18:27:25.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:24 smithi099 ceph-mon[31658]: pgmap v9: 129 pgs: 129 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 279 B/s wr, 8 op/s 2024-05-03T18:27:25.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:25.283+0000 7facace00700 1 -- 172.21.15.33:0/1641190690 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7faca802e510 con 0x7faca8106080 2024-05-03T18:27:25.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:25.287+0000 7fac9f400700 1 -- 172.21.15.33:0/1641190690 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+25087 (secure 0 0 0) 0x7fac900aa410 con 0x7faca8106080 2024-05-03T18:27:25.288 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:27:25.289 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":90,"fsid":"5703bba4-0978-11ef-bc97-c7b262605968","created":"2024-05-03T18:11:00.899879+0000","modified":"2024-05-03T18:27:11.722658+0000","last_up_change":"2024-05-03T18:21:00.390173+0000","last_in_change":"2024-05-03T18:20:34.103727+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-03T18:15:37.297738+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-03T18:22:19.875368+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-03T18:22:24.432692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-03T18:22:38.722443+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"fdcc4e66-4ba0-4378-a9a4-7d907cd00b4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":637686669},{"type":"v1","addr":"172.21.15.33:6803","nonce":637686669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":637686669},{"type":"v1","addr":"172.21.15.33:6805","nonce":637686669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":637686669},{"type":"v1","addr":"172.21.15.33:6809","nonce":637686669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":637686669},{"type":"v1","addr":"172.21.15.33:6807","nonce":637686669}]},"public_addr":"172.21.15.33:6803/637686669","cluster_addr":"172.21.15.33:6805/637686669","heartbeat_back_addr":"172.21.15.33:6809/637686669","heartbeat_front_addr":"172.21.15.33:6807/637686669","state":["exists","up"]},{"osd":1,"uuid":"cf78cad3-fc24-43f4-8784-21cabea7277c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":1536122841},{"type":"v1","addr":"172.21.15.99:6801","nonce":1536122841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":1536122841},{"type":"v1","addr":"172.21.15.99:6803","nonce":1536122841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":1536122841},{"type":"v1","addr":"172.21.15.99:6807","nonce":1536122841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":1536122841},{"type":"v1","addr":"172.21.15.99:6805","nonce":1536122841}]},"public_addr":"172.21.15.99:6801/1536122841","cluster_addr":"172.21.15.99:6803/1536122841","heartbeat_back_addr":"172.21.15.99:6807/1536122841","heartbeat_front_addr":"172.21.15.99:6805/1536122841","state":["exists","up"]},{"osd":2,"uuid":"59f066d7-626b-410a-8d67-daca6fbbd3ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":1995659622},{"type":"v1","addr":"172.21.15.150:6801","nonce":1995659622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":1995659622},{"type":"v1","addr":"172.21.15.150:6803","nonce":1995659622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":1995659622},{"type":"v1","addr":"172.21.15.150:6807","nonce":1995659622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":1995659622},{"type":"v1","addr":"172.21.15.150:6805","nonce":1995659622}]},"public_addr":"172.21.15.150:6801/1995659622","cluster_addr":"172.21.15.150:6803/1995659622","heartbeat_back_addr":"172.21.15.150:6807/1995659622","heartbeat_front_addr":"172.21.15.150:6805/1995659622","state":["exists","up"]},{"osd":3,"uuid":"d88ed022-4dcb-45fc-9fc6-41ef7c4bf202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":3814942724},{"type":"v1","addr":"172.21.15.33:6811","nonce":3814942724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":3814942724},{"type":"v1","addr":"172.21.15.33:6813","nonce":3814942724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":3814942724},{"type":"v1","addr":"172.21.15.33:6817","nonce":3814942724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":3814942724},{"type":"v1","addr":"172.21.15.33:6815","nonce":3814942724}]},"public_addr":"172.21.15.33:6811/3814942724","cluster_addr":"172.21.15.33:6813/3814942724","heartbeat_back_addr":"172.21.15.33:6817/3814942724","heartbeat_front_addr":"172.21.15.33:6815/3814942724","state":["exists","up"]},{"osd":4,"uuid":"1235e1a6-ff6e-4668-ac1a-073c2a3b7c98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":29081029},{"type":"v1","addr":"172.21.15.99:6809","nonce":29081029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":29081029},{"type":"v1","addr":"172.21.15.99:6811","nonce":29081029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":29081029},{"type":"v1","addr":"172.21.15.99:6815","nonce":29081029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":29081029},{"type":"v1","addr":"172.21.15.99:6813","nonce":29081029}]},"public_addr":"172.21.15.99:6809/29081029","cluster_addr":"172.21.15.99:6811/29081029","heartbeat_back_addr":"172.21.15.99:6815/29081029","heartbeat_front_addr":"172.21.15.99:6813/29081029","state":["exists","up"]},{"osd":5,"uuid":"4b84bff8-d6d7-4258-a0ee-53bc19802585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":3172189533},{"type":"v1","addr":"172.21.15.150:6809","nonce":3172189533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3172189533},{"type":"v1","addr":"172.21.15.150:6811","nonce":3172189533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3172189533},{"type":"v1","addr":"172.21.15.150:6815","nonce":3172189533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3172189533},{"type":"v1","addr":"172.21.15.150:6813","nonce":3172189533}]},"public_addr":"172.21.15.150:6809/3172189533","cluster_addr":"172.21.15.150:6811/3172189533","heartbeat_back_addr":"172.21.15.150:6815/3172189533","heartbeat_front_addr":"172.21.15.150:6813/3172189533","state":["exists","up"]},{"osd":6,"uuid":"51f5fbdd-3794-4c48-9f6c-478f0fa35ac5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":4234541418},{"type":"v1","addr":"172.21.15.33:6819","nonce":4234541418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":4234541418},{"type":"v1","addr":"172.21.15.33:6821","nonce":4234541418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":4234541418},{"type":"v1","addr":"172.21.15.33:6825","nonce":4234541418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":4234541418},{"type":"v1","addr":"172.21.15.33:6823","nonce":4234541418}]},"public_addr":"172.21.15.33:6819/4234541418","cluster_addr":"172.21.15.33:6821/4234541418","heartbeat_back_addr":"172.21.15.33:6825/4234541418","heartbeat_front_addr":"172.21.15.33:6823/4234541418","state":["exists","up"]},{"osd":7,"uuid":"982cc5b3-3163-4505-9bde-a27a8c289457","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":908673090},{"type":"v1","addr":"172.21.15.99:6817","nonce":908673090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":908673090},{"type":"v1","addr":"172.21.15.99:6819","nonce":908673090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":908673090},{"type":"v1","addr":"172.21.15.99:6823","nonce":908673090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":908673090},{"type":"v1","addr":"172.21.15.99:6821","nonce":908673090}]},"public_addr":"172.21.15.99:6817/908673090","cluster_addr":"172.21.15.99:6819/908673090","heartbeat_back_addr":"172.21.15.99:6823/908673090","heartbeat_front_addr":"172.21.15.99:6821/908673090","state":["exists","up"]},{"osd":8,"uuid":"84c9d037-eb84-4eb4-8aa2-e784b703d36c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":1308083602},{"type":"v1","addr":"172.21.15.150:6817","nonce":1308083602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":1308083602},{"type":"v1","addr":"172.21.15.150:6819","nonce":1308083602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":1308083602},{"type":"v1","addr":"172.21.15.150:6823","nonce":1308083602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":1308083602},{"type":"v1","addr":"172.21.15.150:6821","nonce":1308083602}]},"public_addr":"172.21.15.150:6817/1308083602","cluster_addr":"172.21.15.150:6819/1308083602","heartbeat_back_addr":"172.21.15.150:6823/1308083602","heartbeat_front_addr":"172.21.15.150:6821/1308083602","state":["exists","up"]},{"osd":9,"uuid":"a02e965f-0d8f-45e2-9490-337f7f6e3881","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":4133096905},{"type":"v1","addr":"172.21.15.33:6827","nonce":4133096905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":4133096905},{"type":"v1","addr":"172.21.15.33:6829","nonce":4133096905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4133096905},{"type":"v1","addr":"172.21.15.33:6833","nonce":4133096905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":4133096905},{"type":"v1","addr":"172.21.15.33:6831","nonce":4133096905}]},"public_addr":"172.21.15.33:6827/4133096905","cluster_addr":"172.21.15.33:6829/4133096905","heartbeat_back_addr":"172.21.15.33:6833/4133096905","heartbeat_front_addr":"172.21.15.33:6831/4133096905","state":["exists","up"]},{"osd":10,"uuid":"8b29bb96-76e8-45a0-ab19-8d082d755fe9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":935746136},{"type":"v1","addr":"172.21.15.99:6825","nonce":935746136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":935746136},{"type":"v1","addr":"172.21.15.99:6827","nonce":935746136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":935746136},{"type":"v1","addr":"172.21.15.99:6831","nonce":935746136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":935746136},{"type":"v1","addr":"172.21.15.99:6829","nonce":935746136}]},"public_addr":"172.21.15.99:6825/935746136","cluster_addr":"172.21.15.99:6827/935746136","heartbeat_back_addr":"172.21.15.99:6831/935746136","heartbeat_front_addr":"172.21.15.99:6829/935746136","state":["exists","up"]},{"osd":11,"uuid":"07a6a825-c902-4bfd-97cc-cc9074532af5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2069349808},{"type":"v1","addr":"172.21.15.150:6825","nonce":2069349808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2069349808},{"type":"v1","addr":"172.21.15.150:6827","nonce":2069349808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2069349808},{"type":"v1","addr":"172.21.15.150:6831","nonce":2069349808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2069349808},{"type":"v1","addr":"172.21.15.150:6829","nonce":2069349808}]},"public_addr":"172.21.15.150:6825/2069349808","cluster_addr":"172.21.15.150:6827/2069349808","heartbeat_back_addr":"172.21.15.150:6831/2069349808","heartbeat_front_addr":"172.21.15.150:6829/2069349808","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:14:27.454536+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:15:00.538570+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:15:34.125289+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:16:09.853870+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:16:44.928107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:17:19.644037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:17:56.108443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:18:31.731499+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:19:08.062358+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:19:45.204430+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:20:21.013710+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:20:57.653698+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11},{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/4003417322":"2024-05-04T18:27:11.722511+0000","172.21.15.33:6801/4007322071":"2024-05-04T18:11:44.239835+0000","172.21.15.33:6801/2830135294":"2024-05-04T18:12:19.127652+0000","172.21.15.33:0/3767665033":"2024-05-04T18:12:19.127652+0000","172.21.15.33:6800/4007322071":"2024-05-04T18:11:44.239835+0000","172.21.15.33:0/1614640996":"2024-05-04T18:11:44.239835+0000","172.21.15.33:0/1314704208":"2024-05-04T18:11:44.239835+0000","172.21.15.33:0/2014902144":"2024-05-04T18:11:44.239835+0000","172.21.15.33:0/965859483":"2024-05-04T18:12:19.127652+0000","172.21.15.33:6801/3642895501":"2024-05-04T18:27:11.722511+0000","172.21.15.33:6800/2830135294":"2024-05-04T18:12:19.127652+0000","172.21.15.33:0/791712778":"2024-05-04T18:27:11.722511+0000","172.21.15.33:0/3991947997":"2024-05-04T18:12:19.127652+0000","172.21.15.33:0/806149166":"2024-05-04T18:27:11.722511+0000","172.21.15.33:0/1300142939":"2024-05-04T18:27:11.722511+0000","172.21.15.33:6800/3642895501":"2024-05-04T18:27:11.722511+0000","172.21.15.33:0/757614565":"2024-05-04T18:27:11.722511+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-03T18:27:25.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:25.291+0000 7facace00700 1 -- 172.21.15.33:0/1641190690 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fac8c0b2b00 msgr2=0x7fac8c0b4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:25.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:25.291+0000 7facace00700 1 --2- 172.21.15.33:0/1641190690 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fac8c0b2b00 0x7fac8c0b4fc0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fac98009fd0 tx=0x7fac9800b7b0 comp rx=0 tx=0).stop 2024-05-03T18:27:25.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:25.291+0000 7facace00700 1 -- 172.21.15.33:0/1641190690 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faca8106080 msgr2=0x7faca819c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:25.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:25.291+0000 7facace00700 1 --2- 172.21.15.33:0/1641190690 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faca8106080 0x7faca819c280 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fac9000c9b0 tx=0x7fac9000cd70 comp rx=0 tx=0).stop 2024-05-03T18:27:25.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:25.291+0000 7facace00700 1 -- 172.21.15.33:0/1641190690 shutdown_connections 2024-05-03T18:27:25.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:25.291+0000 7facace00700 1 --2- 172.21.15.33:0/1641190690 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fac8c0b2b00 0x7fac8c0b4fc0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:25.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:25.291+0000 7facace00700 1 --2- 172.21.15.33:0/1641190690 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faca810b6c0 0x7faca81a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:25.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:25.291+0000 7facace00700 1 --2- 172.21.15.33:0/1641190690 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faca81056c0 0x7faca819bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:25.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:25.291+0000 7facace00700 1 --2- 172.21.15.33:0/1641190690 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faca8106080 0x7faca819c280 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:25.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:25.291+0000 7facace00700 1 -- 172.21.15.33:0/1641190690 >> 172.21.15.33:0/1641190690 conn(0x7faca81011b0 msgr2=0x7faca8101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:27:25.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:25.291+0000 7facace00700 1 -- 172.21.15.33:0/1641190690 shutdown_connections 2024-05-03T18:27:25.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:25.291+0000 7facace00700 1 -- 172.21.15.33:0/1641190690 wait complete. 2024-05-03T18:27:25.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:25 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/1641190690' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T18:27:25.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:25 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/1641190690' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T18:27:26.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:25 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/1641190690' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T18:27:26.253 DEBUG:tasks.fwd_scrub:Yielding 2024-05-03T18:27:26.253 INFO:teuthology.run_tasks:Running task exec... 2024-05-03T18:27:26.263 INFO:teuthology.task.exec:Executing custom commands... 2024-05-03T18:27:26.263 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi033.front.sepia.ceph.com 2024-05-03T18:27:26.263 DEBUG:teuthology.orchestra.run.smithi033:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/log_level debug' 2024-05-03T18:27:26.264 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-05-03T18:27:26.264 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-03T18:27:26.264 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-03T18:27:26.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:26.464+0000 7fa5df200640 1 -- 172.21.15.33:0/2951122317 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5e0074d60 msgr2=0x7fa5e0075140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:26.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:26.464+0000 7fa5df200640 1 --2- 172.21.15.33:0/2951122317 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5e0074d60 0x7fa5e0075140 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fa5d0009900 tx=0x7fa5d002a480 comp rx=0 tx=0).stop 2024-05-03T18:27:26.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:26.464+0000 7fa5df200640 1 -- 172.21.15.33:0/2951122317 shutdown_connections 2024-05-03T18:27:26.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:26.464+0000 7fa5df200640 1 --2- 172.21.15.33:0/2951122317 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5e010f080 0x7fa5e0111470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:26.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:26.464+0000 7fa5df200640 1 --2- 172.21.15.33:0/2951122317 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa5e0075680 0x7fa5e010eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:26.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:26.464+0000 7fa5df200640 1 --2- 172.21.15.33:0/2951122317 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5e0074d60 0x7fa5e0075140 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:26.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:26.464+0000 7fa5df200640 1 -- 172.21.15.33:0/2951122317 >> 172.21.15.33:0/2951122317 conn(0x7fa5e00fe2f0 msgr2=0x7fa5e0100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:27:26.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:26.464+0000 7fa5df200640 1 -- 172.21.15.33:0/2951122317 shutdown_connections 2024-05-03T18:27:26.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:26.464+0000 7fa5df200640 1 -- 172.21.15.33:0/2951122317 wait complete. 2024-05-03T18:27:26.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:26.465+0000 7fa5df200640 1 Processor -- start 2024-05-03T18:27:26.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:26.465+0000 7fa5df200640 1 -- start start 2024-05-03T18:27:26.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:26.466+0000 7fa5df200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa5e0074d60 0x7fa5e019b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:26.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:26.466+0000 7fa5df200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5e0075680 0x7fa5e019b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:26.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:26.466+0000 7fa5df200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5e010f080 0x7fa5e019fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:26.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:26.466+0000 7fa5df200640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa5e0114b50 con 0x7fa5e0075680 2024-05-03T18:27:26.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:26.466+0000 7fa5df200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa5e01149d0 con 0x7fa5e010f080 2024-05-03T18:27:26.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:26.466+0000 7fa5df200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa5e0114cd0 con 0x7fa5e0074d60 2024-05-03T18:27:26.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:26.466+0000 7fa5dd400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5e0075680 0x7fa5e019b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:26.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:26.466+0000 7fa5dd400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5e0075680 0x7fa5e019b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40822/0 (socket says 172.21.15.33:40822) 2024-05-03T18:27:26.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:26.466+0000 7fa5dd400640 1 -- 172.21.15.33:0/180502075 learned_addr learned my addr 172.21.15.33:0/180502075 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:27:26.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:26.467+0000 7fa5dde00640 1 --2- 172.21.15.33:0/180502075 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa5e0074d60 0x7fa5e019b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:26.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:26.467+0000 7fa5de800640 1 --2- 172.21.15.33:0/180502075 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5e010f080 0x7fa5e019fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:26.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:26.467+0000 7fa5dd400640 1 -- 172.21.15.33:0/180502075 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa5e0074d60 msgr2=0x7fa5e019b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:26.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:26.467+0000 7fa5dd400640 1 --2- 172.21.15.33:0/180502075 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa5e0074d60 0x7fa5e019b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:26.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:26.467+0000 7fa5dd400640 1 -- 172.21.15.33:0/180502075 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5e010f080 msgr2=0x7fa5e019fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:26.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:26.467+0000 7fa5dd400640 1 --2- 172.21.15.33:0/180502075 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5e010f080 0x7fa5e019fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:26.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:26.467+0000 7fa5dd400640 1 -- 172.21.15.33:0/180502075 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5e01a0130 con 0x7fa5e0075680 2024-05-03T18:27:26.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:26.467+0000 7fa5dd400640 1 --2- 172.21.15.33:0/180502075 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5e0075680 0x7fa5e019b680 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc00c890 tx=0x7fa5cc00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:27:26.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:26.467+0000 7fa5d6a00640 1 -- 172.21.15.33:0/180502075 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5cc004220 con 0x7fa5e0075680 2024-05-03T18:27:26.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:26.467+0000 7fa5d6a00640 1 -- 172.21.15.33:0/180502075 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa5cc007e40 con 0x7fa5e0075680 2024-05-03T18:27:26.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:26.467+0000 7fa5df200640 1 -- 172.21.15.33:0/180502075 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5e01a0420 con 0x7fa5e0075680 2024-05-03T18:27:26.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:26.467+0000 7fa5d6a00640 1 -- 172.21.15.33:0/180502075 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5cc004a00 con 0x7fa5e0075680 2024-05-03T18:27:26.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:26.468+0000 7fa5df200640 1 -- 172.21.15.33:0/180502075 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5e01a7cb0 con 0x7fa5e0075680 2024-05-03T18:27:26.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:26.469+0000 7fa5df200640 1 -- 172.21.15.33:0/180502075 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5e0050f90 con 0x7fa5e0075680 2024-05-03T18:27:26.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:26.471+0000 7fa5d6a00640 1 -- 172.21.15.33:0/180502075 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 23) ==== 149828+0+0 (secure 0 0 0) 0x7fa5cc0079b0 con 0x7fa5e0075680 2024-05-03T18:27:26.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:26.473+0000 7fa5d6a00640 1 --2- 172.21.15.33:0/180502075 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fa5c40b29e0 0x7fa5c40b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:26.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:26.474+0000 7fa5d6a00640 1 -- 172.21.15.33:0/180502075 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7fa5cc0e2d20 con 0x7fa5e0075680 2024-05-03T18:27:26.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:26.474+0000 7fa5dde00640 1 --2- 172.21.15.33:0/180502075 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fa5c40b29e0 0x7fa5c40b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:26.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:26.475+0000 7fa5dde00640 1 --2- 172.21.15.33:0/180502075 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fa5c40b29e0 0x7fa5c40b4ea0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa5d0006010 tx=0x7fa5d0005a20 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:27:26.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:26.479+0000 7fa5d6a00640 1 -- 172.21.15.33:0/180502075 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fa5cc014030 con 0x7fa5e0075680 2024-05-03T18:27:26.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:26.720+0000 7fa5df200640 1 -- 172.21.15.33:0/180502075 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/log_level}] v 0) -- 0x7fa5e019c490 con 0x7fa5e0075680 2024-05-03T18:27:26.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:26.726+0000 7fa5d6a00640 1 -- 172.21.15.33:0/180502075 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/log_level}]=0 v29)=0 v29) ==== 129+0+0 (secure 0 0 0) 0x7fa5e019c490 con 0x7fa5e0075680 2024-05-03T18:27:26.729 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:27:26.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:26.730+0000 7fa5df200640 1 -- 172.21.15.33:0/180502075 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fa5c40b29e0 msgr2=0x7fa5c40b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:26.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:26.730+0000 7fa5df200640 1 --2- 172.21.15.33:0/180502075 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fa5c40b29e0 0x7fa5c40b4ea0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa5d0006010 tx=0x7fa5d0005a20 comp rx=0 tx=0).stop 2024-05-03T18:27:26.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:26.730+0000 7fa5df200640 1 -- 172.21.15.33:0/180502075 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5e0075680 msgr2=0x7fa5e019b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:26.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:26.730+0000 7fa5df200640 1 --2- 172.21.15.33:0/180502075 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5e0075680 0x7fa5e019b680 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc00c890 tx=0x7fa5cc00cd50 comp rx=0 tx=0).stop 2024-05-03T18:27:26.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:26.730+0000 7fa5df200640 1 -- 172.21.15.33:0/180502075 shutdown_connections 2024-05-03T18:27:26.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:26.730+0000 7fa5df200640 1 --2- 172.21.15.33:0/180502075 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fa5c40b29e0 0x7fa5c40b4ea0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:26.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:26.730+0000 7fa5df200640 1 --2- 172.21.15.33:0/180502075 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5e010f080 0x7fa5e019fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:26.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:26.730+0000 7fa5df200640 1 --2- 172.21.15.33:0/180502075 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5e0075680 0x7fa5e019b680 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:26.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:26.730+0000 7fa5df200640 1 --2- 172.21.15.33:0/180502075 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa5e0074d60 0x7fa5e019b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:26.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:26.730+0000 7fa5df200640 1 -- 172.21.15.33:0/180502075 >> 172.21.15.33:0/180502075 conn(0x7fa5e00fe2f0 msgr2=0x7fa5e00ffd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:27:26.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:26.730+0000 7fa5df200640 1 -- 172.21.15.33:0/180502075 shutdown_connections 2024-05-03T18:27:26.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:26.730+0000 7fa5df200640 1 -- 172.21.15.33:0/180502075 wait complete. 2024-05-03T18:27:26.752 DEBUG:teuthology.orchestra.run.smithi033:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-05-03T18:27:26.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:26 smithi150 ceph-mon[32492]: pgmap v10: 129 pgs: 129 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-03T18:27:26.882 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:26 smithi033 ceph-mon[28330]: pgmap v10: 129 pgs: 129 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-03T18:27:26.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:26.940+0000 7f76fa000640 1 -- 172.21.15.33:0/4070837032 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76f4075700 msgr2=0x7f76f410ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:26.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:26.940+0000 7f76fa000640 1 --2- 172.21.15.33:0/4070837032 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76f4075700 0x7f76f410ec90 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f76dc009900 tx=0x7f76dc02a480 comp rx=0 tx=0).stop 2024-05-03T18:27:26.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:26.941+0000 7f76fa000640 1 -- 172.21.15.33:0/4070837032 shutdown_connections 2024-05-03T18:27:26.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:26.941+0000 7f76fa000640 1 --2- 172.21.15.33:0/4070837032 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76f410f1d0 0x7f76f4111620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:26.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:26.941+0000 7f76fa000640 1 --2- 172.21.15.33:0/4070837032 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76f4075700 0x7f76f410ec90 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:26.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:26.941+0000 7f76fa000640 1 --2- 172.21.15.33:0/4070837032 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76f4074de0 0x7f76f40751c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:26.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:26.941+0000 7f76fa000640 1 -- 172.21.15.33:0/4070837032 >> 172.21.15.33:0/4070837032 conn(0x7f76f40fe460 msgr2=0x7f76f4100880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:27:26.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:26.941+0000 7f76fa000640 1 -- 172.21.15.33:0/4070837032 shutdown_connections 2024-05-03T18:27:26.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:26.941+0000 7f76fa000640 1 -- 172.21.15.33:0/4070837032 wait complete. 2024-05-03T18:27:26.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:26.942+0000 7f76fa000640 1 Processor -- start 2024-05-03T18:27:26.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:26.942+0000 7f76fa000640 1 -- start start 2024-05-03T18:27:26.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:26.942+0000 7f76fa000640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76f4074de0 0x7f76f419b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:26.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:26.942+0000 7f76fa000640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76f4075700 0x7f76f419b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:26.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:26.942+0000 7f76fa000640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76f410f1d0 0x7f76f419fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:26.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:26.942+0000 7f76fa000640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f76f4114cd0 con 0x7f76f410f1d0 2024-05-03T18:27:26.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:26.942+0000 7f76fa000640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f76f4114b50 con 0x7f76f4075700 2024-05-03T18:27:26.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:26.942+0000 7f76fa000640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f76f4114e50 con 0x7f76f4074de0 2024-05-03T18:27:26.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:26.943+0000 7f76f3e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76f410f1d0 0x7f76f419fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:26.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:26.943+0000 7f76f3e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76f410f1d0 0x7f76f419fab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40832/0 (socket says 172.21.15.33:40832) 2024-05-03T18:27:26.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:26.943+0000 7f76f3e00640 1 -- 172.21.15.33:0/2315304424 learned_addr learned my addr 172.21.15.33:0/2315304424 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:27:26.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:26.943+0000 7f76f2a00640 1 --2- 172.21.15.33:0/2315304424 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76f4075700 0x7f76f419b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:26.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:26.943+0000 7f76f3400640 1 --2- 172.21.15.33:0/2315304424 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76f4074de0 0x7f76f419b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:26.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:26.943+0000 7f76f3e00640 1 -- 172.21.15.33:0/2315304424 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76f4074de0 msgr2=0x7f76f419b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:26.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:26.943+0000 7f76f3e00640 1 --2- 172.21.15.33:0/2315304424 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76f4074de0 0x7f76f419b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:26.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:26.943+0000 7f76f3e00640 1 -- 172.21.15.33:0/2315304424 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76f4075700 msgr2=0x7f76f419b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:26.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:26.943+0000 7f76f3e00640 1 --2- 172.21.15.33:0/2315304424 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76f4075700 0x7f76f419b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:26.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:26.943+0000 7f76f3e00640 1 -- 172.21.15.33:0/2315304424 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76f41a01e0 con 0x7f76f410f1d0 2024-05-03T18:27:26.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:26.944+0000 7f76f3e00640 1 --2- 172.21.15.33:0/2315304424 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76f410f1d0 0x7f76f419fab0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f76ec00cce0 tx=0x7f76ec00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:27:26.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:26.944+0000 7f76ebe00640 1 -- 172.21.15.33:0/2315304424 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76ec00f450 con 0x7f76f410f1d0 2024-05-03T18:27:26.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:26.944+0000 7f76ebe00640 1 -- 172.21.15.33:0/2315304424 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f76ec00fa80 con 0x7f76f410f1d0 2024-05-03T18:27:26.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:26.944+0000 7f76fa000640 1 -- 172.21.15.33:0/2315304424 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76f41a04d0 con 0x7f76f410f1d0 2024-05-03T18:27:26.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:26.944+0000 7f76ebe00640 1 -- 172.21.15.33:0/2315304424 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76ec018ad0 con 0x7f76f410f1d0 2024-05-03T18:27:26.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:26.944+0000 7f76fa000640 1 -- 172.21.15.33:0/2315304424 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76f41a7d60 con 0x7f76f410f1d0 2024-05-03T18:27:26.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:26.945+0000 7f76fa000640 1 -- 172.21.15.33:0/2315304424 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76f4050740 con 0x7f76f410f1d0 2024-05-03T18:27:26.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:26.948+0000 7f76ebe00640 1 -- 172.21.15.33:0/2315304424 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 23) ==== 149828+0+0 (secure 0 0 0) 0x7f76ec00f5f0 con 0x7f76f410f1d0 2024-05-03T18:27:26.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:26.949+0000 7f76ebe00640 1 --2- 172.21.15.33:0/2315304424 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f76d80b2a90 0x7f76d80b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:26.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:26.950+0000 7f76ebe00640 1 -- 172.21.15.33:0/2315304424 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f76ec0e32c0 con 0x7f76f410f1d0 2024-05-03T18:27:26.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:26.950+0000 7f76f3400640 1 --2- 172.21.15.33:0/2315304424 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f76d80b2a90 0x7f76d80b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:26.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:26.951+0000 7f76f3400640 1 --2- 172.21.15.33:0/2315304424 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f76d80b2a90 0x7f76d80b4f50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f76e4005fd0 tx=0x7f76e4005850 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:27:26.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:26.954+0000 7f76ebe00640 1 -- 172.21.15.33:0/2315304424 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f76ec014030 con 0x7f76f410f1d0 2024-05-03T18:27:27.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:26 smithi099 ceph-mon[31658]: pgmap v10: 129 pgs: 129 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-03T18:27:27.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.178+0000 7f76fa000640 1 -- 172.21.15.33:0/2315304424 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f76f402db50 con 0x7f76f410f1d0 2024-05-03T18:27:27.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.184+0000 7f76ebe00640 1 -- 172.21.15.33:0/2315304424 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v30)=0 v30) ==== 138+0+0 (secure 0 0 0) 0x7f76ec0aa740 con 0x7f76f410f1d0 2024-05-03T18:27:27.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.188+0000 7f76fa000640 1 -- 172.21.15.33:0/2315304424 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f76d80b2a90 msgr2=0x7f76d80b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:27.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.188+0000 7f76fa000640 1 --2- 172.21.15.33:0/2315304424 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f76d80b2a90 0x7f76d80b4f50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f76e4005fd0 tx=0x7f76e4005850 comp rx=0 tx=0).stop 2024-05-03T18:27:27.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.188+0000 7f76fa000640 1 -- 172.21.15.33:0/2315304424 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76f410f1d0 msgr2=0x7f76f419fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:27.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.188+0000 7f76fa000640 1 --2- 172.21.15.33:0/2315304424 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76f410f1d0 0x7f76f419fab0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f76ec00cce0 tx=0x7f76ec00a620 comp rx=0 tx=0).stop 2024-05-03T18:27:27.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.188+0000 7f76fa000640 1 -- 172.21.15.33:0/2315304424 shutdown_connections 2024-05-03T18:27:27.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.188+0000 7f76fa000640 1 --2- 172.21.15.33:0/2315304424 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f76d80b2a90 0x7f76d80b4f50 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:27.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.188+0000 7f76fa000640 1 --2- 172.21.15.33:0/2315304424 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76f410f1d0 0x7f76f419fab0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:27.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.188+0000 7f76fa000640 1 --2- 172.21.15.33:0/2315304424 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76f4075700 0x7f76f419b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:27.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.188+0000 7f76fa000640 1 --2- 172.21.15.33:0/2315304424 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76f4074de0 0x7f76f419b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:27.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.188+0000 7f76fa000640 1 -- 172.21.15.33:0/2315304424 >> 172.21.15.33:0/2315304424 conn(0x7f76f40fe460 msgr2=0x7f76f40ff170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:27:27.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.188+0000 7f76fa000640 1 -- 172.21.15.33:0/2315304424 shutdown_connections 2024-05-03T18:27:27.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.188+0000 7f76fa000640 1 -- 172.21.15.33:0/2315304424 wait complete. 2024-05-03T18:27:27.212 DEBUG:teuthology.orchestra.run.smithi033:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-05-03T18:27:27.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.400+0000 7f99e3200640 1 -- 172.21.15.33:0/2182359782 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f99e4074de0 msgr2=0x7f99e40751c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:27.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.400+0000 7f99e3200640 1 --2- 172.21.15.33:0/2182359782 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f99e4074de0 0x7f99e40751c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f99d4009ca0 tx=0x7f99d402a950 comp rx=0 tx=0).stop 2024-05-03T18:27:27.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.400+0000 7f99e3200640 1 -- 172.21.15.33:0/2182359782 shutdown_connections 2024-05-03T18:27:27.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.400+0000 7f99e3200640 1 --2- 172.21.15.33:0/2182359782 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99e410eff0 0x7f99e4111440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:27.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.400+0000 7f99e3200640 1 --2- 172.21.15.33:0/2182359782 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f99e4075700 0x7f99e410eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:27.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.400+0000 7f99e3200640 1 --2- 172.21.15.33:0/2182359782 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f99e4074de0 0x7f99e40751c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:27.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.400+0000 7f99e3200640 1 -- 172.21.15.33:0/2182359782 >> 172.21.15.33:0/2182359782 conn(0x7f99e40fe300 msgr2=0x7f99e4100720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:27:27.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.401+0000 7f99e3200640 1 -- 172.21.15.33:0/2182359782 shutdown_connections 2024-05-03T18:27:27.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.401+0000 7f99e3200640 1 -- 172.21.15.33:0/2182359782 wait complete. 2024-05-03T18:27:27.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.402+0000 7f99e3200640 1 Processor -- start 2024-05-03T18:27:27.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.402+0000 7f99e3200640 1 -- start start 2024-05-03T18:27:27.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.402+0000 7f99e3200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f99e4074de0 0x7f99e419b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:27.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.402+0000 7f99e3200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f99e4075700 0x7f99e419b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:27.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.403+0000 7f99e3200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99e410eff0 0x7f99e419fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:27.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.403+0000 7f99e3200640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f99e4114b40 con 0x7f99e410eff0 2024-05-03T18:27:27.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.403+0000 7f99e3200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f99e41149c0 con 0x7f99e4074de0 2024-05-03T18:27:27.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.403+0000 7f99e3200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f99e4114cc0 con 0x7f99e4075700 2024-05-03T18:27:27.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.403+0000 7f99e2800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99e410eff0 0x7f99e419fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:27.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.403+0000 7f99e2800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99e410eff0 0x7f99e419fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40844/0 (socket says 172.21.15.33:40844) 2024-05-03T18:27:27.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.403+0000 7f99e2800640 1 -- 172.21.15.33:0/1466112879 learned_addr learned my addr 172.21.15.33:0/1466112879 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:27:27.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.403+0000 7f99e1e00640 1 --2- 172.21.15.33:0/1466112879 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f99e4074de0 0x7f99e419b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:27.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.403+0000 7f99e1400640 1 --2- 172.21.15.33:0/1466112879 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f99e4075700 0x7f99e419b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:27.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.403+0000 7f99e2800640 1 -- 172.21.15.33:0/1466112879 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f99e4075700 msgr2=0x7f99e419b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:27.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.403+0000 7f99e2800640 1 --2- 172.21.15.33:0/1466112879 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f99e4075700 0x7f99e419b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:27.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.403+0000 7f99e2800640 1 -- 172.21.15.33:0/1466112879 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f99e4074de0 msgr2=0x7f99e419b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:27.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.403+0000 7f99e2800640 1 --2- 172.21.15.33:0/1466112879 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f99e4074de0 0x7f99e419b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:27.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.403+0000 7f99e2800640 1 -- 172.21.15.33:0/1466112879 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99e41a0130 con 0x7f99e410eff0 2024-05-03T18:27:27.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.404+0000 7f99e2800640 1 --2- 172.21.15.33:0/1466112879 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99e410eff0 0x7f99e419fa00 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f99dc00e9b0 tx=0x7f99dc00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:27:27.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.404+0000 7f99daa00640 1 -- 172.21.15.33:0/1466112879 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99dc00cd80 con 0x7f99e410eff0 2024-05-03T18:27:27.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.404+0000 7f99e3200640 1 -- 172.21.15.33:0/1466112879 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99e41a0420 con 0x7f99e410eff0 2024-05-03T18:27:27.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.404+0000 7f99daa00640 1 -- 172.21.15.33:0/1466112879 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f99dc0035f0 con 0x7f99e410eff0 2024-05-03T18:27:27.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.404+0000 7f99daa00640 1 -- 172.21.15.33:0/1466112879 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99dc010690 con 0x7f99e410eff0 2024-05-03T18:27:27.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.404+0000 7f99e3200640 1 -- 172.21.15.33:0/1466112879 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99e41a7cb0 con 0x7f99e410eff0 2024-05-03T18:27:27.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.405+0000 7f99e3200640 1 -- 172.21.15.33:0/1466112879 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99e4050f90 con 0x7f99e410eff0 2024-05-03T18:27:27.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.408+0000 7f99daa00640 1 -- 172.21.15.33:0/1466112879 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 23) ==== 149828+0+0 (secure 0 0 0) 0x7f99dc004000 con 0x7f99e410eff0 2024-05-03T18:27:27.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.409+0000 7f99daa00640 1 --2- 172.21.15.33:0/1466112879 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f99c80b29e0 0x7f99c80b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:27.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.409+0000 7f99daa00640 1 -- 172.21.15.33:0/1466112879 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f99dc0e2f60 con 0x7f99e410eff0 2024-05-03T18:27:27.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.410+0000 7f99e1e00640 1 --2- 172.21.15.33:0/1466112879 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f99c80b29e0 0x7f99c80b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:27.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.411+0000 7f99e1e00640 1 --2- 172.21.15.33:0/1466112879 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f99c80b29e0 0x7f99c80b4ea0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f99d402af40 tx=0x7f99d4005d80 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:27:27.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.412+0000 7f99daa00640 1 -- 172.21.15.33:0/1466112879 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f99dc0aa4d0 con 0x7f99e410eff0 2024-05-03T18:27:27.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.634+0000 7f99e3200640 1 -- 172.21.15.33:0/1466112879 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f99e4109be0 con 0x7f99e410eff0 2024-05-03T18:27:27.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.640+0000 7f99daa00640 1 -- 172.21.15.33:0/1466112879 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v31)=0 v31) ==== 133+0+0 (secure 0 0 0) 0x7f99dc0a9dd0 con 0x7f99e410eff0 2024-05-03T18:27:27.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.643+0000 7f99e3200640 1 -- 172.21.15.33:0/1466112879 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f99c80b29e0 msgr2=0x7f99c80b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:27.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.643+0000 7f99e3200640 1 --2- 172.21.15.33:0/1466112879 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f99c80b29e0 0x7f99c80b4ea0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f99d402af40 tx=0x7f99d4005d80 comp rx=0 tx=0).stop 2024-05-03T18:27:27.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.643+0000 7f99e3200640 1 -- 172.21.15.33:0/1466112879 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99e410eff0 msgr2=0x7f99e419fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:27.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.643+0000 7f99e3200640 1 --2- 172.21.15.33:0/1466112879 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99e410eff0 0x7f99e419fa00 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f99dc00e9b0 tx=0x7f99dc00ee70 comp rx=0 tx=0).stop 2024-05-03T18:27:27.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.643+0000 7f99e3200640 1 -- 172.21.15.33:0/1466112879 shutdown_connections 2024-05-03T18:27:27.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.643+0000 7f99e3200640 1 --2- 172.21.15.33:0/1466112879 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f99c80b29e0 0x7f99c80b4ea0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:27.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.643+0000 7f99e3200640 1 --2- 172.21.15.33:0/1466112879 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99e410eff0 0x7f99e419fa00 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:27.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.643+0000 7f99e3200640 1 --2- 172.21.15.33:0/1466112879 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f99e4075700 0x7f99e419b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:27.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.643+0000 7f99e3200640 1 --2- 172.21.15.33:0/1466112879 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f99e4074de0 0x7f99e419b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:27.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.643+0000 7f99e3200640 1 -- 172.21.15.33:0/1466112879 >> 172.21.15.33:0/1466112879 conn(0x7f99e40fe300 msgr2=0x7f99e40ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:27:27.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.643+0000 7f99e3200640 1 -- 172.21.15.33:0/1466112879 shutdown_connections 2024-05-03T18:27:27.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.643+0000 7f99e3200640 1 -- 172.21.15.33:0/1466112879 wait complete. 2024-05-03T18:27:27.664 DEBUG:teuthology.orchestra.run.smithi033:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-05-03T18:27:27.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.861+0000 7f7f38400640 1 -- 172.21.15.33:0/3845523653 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f30103370 msgr2=0x7f7f3010f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:27.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.861+0000 7f7f38400640 1 --2- 172.21.15.33:0/3845523653 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f30103370 0x7f7f3010f880 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f7f28009cd0 tx=0x7f7f2802a8f0 comp rx=0 tx=0).stop 2024-05-03T18:27:27.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.861+0000 7f7f38400640 1 -- 172.21.15.33:0/3845523653 shutdown_connections 2024-05-03T18:27:27.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.861+0000 7f7f38400640 1 --2- 172.21.15.33:0/3845523653 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f30103370 0x7f7f3010f880 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:27.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.861+0000 7f7f38400640 1 --2- 172.21.15.33:0/3845523653 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f301029b0 0x7f7f30102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:27.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.861+0000 7f7f38400640 1 --2- 172.21.15.33:0/3845523653 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f301089c0 0x7f7f30108da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:27.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.861+0000 7f7f38400640 1 -- 172.21.15.33:0/3845523653 >> 172.21.15.33:0/3845523653 conn(0x7f7f300fe440 msgr2=0x7f7f30100860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:27:27.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.861+0000 7f7f38400640 1 -- 172.21.15.33:0/3845523653 shutdown_connections 2024-05-03T18:27:27.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.861+0000 7f7f38400640 1 -- 172.21.15.33:0/3845523653 wait complete. 2024-05-03T18:27:27.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.862+0000 7f7f38400640 1 Processor -- start 2024-05-03T18:27:27.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.863+0000 7f7f38400640 1 -- start start 2024-05-03T18:27:27.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.863+0000 7f7f38400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f301029b0 0x7f7f30074cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:27.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.863+0000 7f7f38400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f30103370 0x7f7f300751f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:27.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.863+0000 7f7f38400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f301089c0 0x7f7f30075730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:27.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.863+0000 7f7f38400640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7f30112b80 con 0x7f7f30103370 2024-05-03T18:27:27.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.863+0000 7f7f38400640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7f30112a00 con 0x7f7f301029b0 2024-05-03T18:27:27.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.863+0000 7f7f38400640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f7f30112d00 con 0x7f7f301089c0 2024-05-03T18:27:27.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.863+0000 7f7f35000640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f30103370 0x7f7f300751f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:27.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.863+0000 7f7f35000640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f30103370 0x7f7f300751f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40856/0 (socket says 172.21.15.33:40856) 2024-05-03T18:27:27.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.863+0000 7f7f35000640 1 -- 172.21.15.33:0/3220380055 learned_addr learned my addr 172.21.15.33:0/3220380055 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:27:27.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.863+0000 7f7f35a00640 1 --2- 172.21.15.33:0/3220380055 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f301029b0 0x7f7f30074cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:27.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.864+0000 7f7f36400640 1 --2- 172.21.15.33:0/3220380055 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f301089c0 0x7f7f30075730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:27.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.864+0000 7f7f35000640 1 -- 172.21.15.33:0/3220380055 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f301089c0 msgr2=0x7f7f30075730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:27.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.864+0000 7f7f35000640 1 --2- 172.21.15.33:0/3220380055 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f301089c0 0x7f7f30075730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:27.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.864+0000 7f7f35000640 1 -- 172.21.15.33:0/3220380055 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f301029b0 msgr2=0x7f7f30074cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:27.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.864+0000 7f7f35000640 1 --2- 172.21.15.33:0/3220380055 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f301029b0 0x7f7f30074cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:27.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.864+0000 7f7f35000640 1 -- 172.21.15.33:0/3220380055 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f301a74e0 con 0x7f7f30103370 2024-05-03T18:27:27.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.864+0000 7f7f35000640 1 --2- 172.21.15.33:0/3220380055 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f30103370 0x7f7f300751f0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c00d950 tx=0x7f7f1c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:27:27.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.864+0000 7f7f2e000640 1 -- 172.21.15.33:0/3220380055 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f1c017070 con 0x7f7f30103370 2024-05-03T18:27:27.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.864+0000 7f7f2e000640 1 -- 172.21.15.33:0/3220380055 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7f1c009c30 con 0x7f7f30103370 2024-05-03T18:27:27.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.864+0000 7f7f38400640 1 -- 172.21.15.33:0/3220380055 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f301a77d0 con 0x7f7f30103370 2024-05-03T18:27:27.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.865+0000 7f7f2e000640 1 -- 172.21.15.33:0/3220380055 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f1c00f620 con 0x7f7f30103370 2024-05-03T18:27:27.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.865+0000 7f7f38400640 1 -- 172.21.15.33:0/3220380055 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f301a7d70 con 0x7f7f30103370 2024-05-03T18:27:27.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.866+0000 7f7f38400640 1 -- 172.21.15.33:0/3220380055 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f30050740 con 0x7f7f30103370 2024-05-03T18:27:27.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.870+0000 7f7f2e000640 1 -- 172.21.15.33:0/3220380055 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 23) ==== 149828+0+0 (secure 0 0 0) 0x7f7f1c005030 con 0x7f7f30103370 2024-05-03T18:27:27.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.871+0000 7f7f2e000640 1 --2- 172.21.15.33:0/3220380055 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f7f180b2b40 0x7f7f180b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:27.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.872+0000 7f7f2e000640 1 -- 172.21.15.33:0/3220380055 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f7f1c0e2710 con 0x7f7f30103370 2024-05-03T18:27:27.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.872+0000 7f7f35a00640 1 --2- 172.21.15.33:0/3220380055 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f7f180b2b40 0x7f7f180b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:27.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.872+0000 7f7f35a00640 1 --2- 172.21.15.33:0/3220380055 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f7f180b2b40 0x7f7f180b5000 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7f24005fd0 tx=0x7f7f2400b390 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:27:27.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:27.873+0000 7f7f2e000640 1 -- 172.21.15.33:0/3220380055 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f7f1c0a9c10 con 0x7f7f30103370 2024-05-03T18:27:27.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:27 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/180502075' entity='client.admin' 2024-05-03T18:27:27.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:27 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:27:27.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:27 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:27:27.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:27 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:27:27.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:27 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:27:27.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:27 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/2315304424' entity='client.admin' 2024-05-03T18:27:27.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:27 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:27:27.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:27 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:27:27.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:27 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:27:27.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:27 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:27:27.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:27 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/1466112879' entity='client.admin' 2024-05-03T18:27:27.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:27 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:27:27.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:27 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:27:27.986 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:27 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:27:27.986 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:27 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:27:28.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:27 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/180502075' entity='client.admin' 2024-05-03T18:27:28.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:27 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:27:28.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:27 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:27:28.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:27 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:27:28.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:27 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:27:28.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:27 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/2315304424' entity='client.admin' 2024-05-03T18:27:28.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:27 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:27:28.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:27 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:27:28.047 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:27 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:27:28.047 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:27 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:27:28.047 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:27 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/1466112879' entity='client.admin' 2024-05-03T18:27:28.047 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:27 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:27:28.047 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:27 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:27:28.047 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:27 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:27:28.047 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:27 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:27:28.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:28.124+0000 7f7f38400640 1 -- 172.21.15.33:0/3220380055 --> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f7f3010af90 con 0x7f7f180b2b40 2024-05-03T18:27:28.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:27 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/180502075' entity='client.admin' 2024-05-03T18:27:28.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:27 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:27:28.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:27 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:27:28.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:27 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:27:28.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:27 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:27:28.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:27 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/2315304424' entity='client.admin' 2024-05-03T18:27:28.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:27 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:27:28.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:27 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:27:28.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:27 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:27:28.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:27 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:27:28.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:27 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/1466112879' entity='client.admin' 2024-05-03T18:27:28.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:27 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:27:28.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:27 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:27:28.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:27 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:27:28.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:27 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:27:28.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:28.289+0000 7f7f2e000640 1 -- 172.21.15.33:0/3220380055 <== mgr.15120 v2:172.21.15.33:6800/3302011241 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f7f24004220 con 0x7f7f180b2b40 2024-05-03T18:27:28.291 INFO:teuthology.orchestra.run.smithi033.stdout:Schedule set for path / 2024-05-03T18:27:28.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:28.292+0000 7f7f38400640 1 -- 172.21.15.33:0/3220380055 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f7f180b2b40 msgr2=0x7f7f180b5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:28.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:28.292+0000 7f7f38400640 1 --2- 172.21.15.33:0/3220380055 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f7f180b2b40 0x7f7f180b5000 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7f24005fd0 tx=0x7f7f2400b390 comp rx=0 tx=0).stop 2024-05-03T18:27:28.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:28.292+0000 7f7f38400640 1 -- 172.21.15.33:0/3220380055 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f30103370 msgr2=0x7f7f300751f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:28.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:28.292+0000 7f7f38400640 1 --2- 172.21.15.33:0/3220380055 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f30103370 0x7f7f300751f0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c00d950 tx=0x7f7f1c00de10 comp rx=0 tx=0).stop 2024-05-03T18:27:28.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:28.292+0000 7f7f38400640 1 -- 172.21.15.33:0/3220380055 shutdown_connections 2024-05-03T18:27:28.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:28.292+0000 7f7f38400640 1 --2- 172.21.15.33:0/3220380055 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f7f180b2b40 0x7f7f180b5000 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:28.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:28.292+0000 7f7f38400640 1 --2- 172.21.15.33:0/3220380055 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f301089c0 0x7f7f30075730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:28.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:28.292+0000 7f7f38400640 1 --2- 172.21.15.33:0/3220380055 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f30103370 0x7f7f300751f0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:28.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:28.292+0000 7f7f38400640 1 --2- 172.21.15.33:0/3220380055 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f301029b0 0x7f7f30074cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:28.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:28.292+0000 7f7f38400640 1 -- 172.21.15.33:0/3220380055 >> 172.21.15.33:0/3220380055 conn(0x7f7f300fe440 msgr2=0x7f7f300fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:27:28.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:28.292+0000 7f7f38400640 1 -- 172.21.15.33:0/3220380055 shutdown_connections 2024-05-03T18:27:28.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:28.292+0000 7f7f38400640 1 -- 172.21.15.33:0/3220380055 wait complete. 2024-05-03T18:27:28.319 DEBUG:teuthology.orchestra.run.smithi033:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-05-03T18:27:28.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:28.515+0000 7f9df1400640 1 -- 172.21.15.33:0/2458864316 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9dec109870 msgr2=0x7f9dec115cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:28.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:28.515+0000 7f9df1400640 1 --2- 172.21.15.33:0/2458864316 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9dec109870 0x7f9dec115cb0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f9de4009cd0 tx=0x7f9de402a8e0 comp rx=0 tx=0).stop 2024-05-03T18:27:28.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:28.515+0000 7f9df1400640 1 -- 172.21.15.33:0/2458864316 shutdown_connections 2024-05-03T18:27:28.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:28.515+0000 7f9df1400640 1 --2- 172.21.15.33:0/2458864316 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9dec109870 0x7f9dec115cb0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:28.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:28.515+0000 7f9df1400640 1 --2- 172.21.15.33:0/2458864316 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9dec108ef0 0x7f9dec109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:28.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:28.515+0000 7f9df1400640 1 --2- 172.21.15.33:0/2458864316 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9dec10ee80 0x7f9dec10f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:28.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:28.515+0000 7f9df1400640 1 -- 172.21.15.33:0/2458864316 >> 172.21.15.33:0/2458864316 conn(0x7f9dec078790 msgr2=0x7f9dec1077d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:27:28.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:28.515+0000 7f9df1400640 1 -- 172.21.15.33:0/2458864316 shutdown_connections 2024-05-03T18:27:28.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:28.515+0000 7f9df1400640 1 -- 172.21.15.33:0/2458864316 wait complete. 2024-05-03T18:27:28.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:28.516+0000 7f9df1400640 1 Processor -- start 2024-05-03T18:27:28.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:28.516+0000 7f9df1400640 1 -- start start 2024-05-03T18:27:28.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:28.517+0000 7f9df1400640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9dec108ef0 0x7f9dec19f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:28.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:28.517+0000 7f9df1400640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9dec109870 0x7f9dec19fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:28.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:28.517+0000 7f9df1400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9dec10ee80 0x7f9dec1a3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:28.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:28.517+0000 7f9df1400640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9dec118f40 con 0x7f9dec10ee80 2024-05-03T18:27:28.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:28.517+0000 7f9df1400640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9dec118dc0 con 0x7f9dec109870 2024-05-03T18:27:28.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:28.517+0000 7f9df1400640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9dec1190c0 con 0x7f9dec108ef0 2024-05-03T18:27:28.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:28.517+0000 7f9df0a00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9dec10ee80 0x7f9dec1a3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:28.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:28.517+0000 7f9df0a00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9dec10ee80 0x7f9dec1a3dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40870/0 (socket says 172.21.15.33:40870) 2024-05-03T18:27:28.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:28.517+0000 7f9df0a00640 1 -- 172.21.15.33:0/3656367310 learned_addr learned my addr 172.21.15.33:0/3656367310 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:27:28.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:28.517+0000 7f9deb400640 1 --2- 172.21.15.33:0/3656367310 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9dec109870 0x7f9dec19fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:28.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:28.517+0000 7f9debe00640 1 --2- 172.21.15.33:0/3656367310 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9dec108ef0 0x7f9dec19f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:28.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:28.517+0000 7f9df0a00640 1 -- 172.21.15.33:0/3656367310 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9dec108ef0 msgr2=0x7f9dec19f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:28.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:28.517+0000 7f9df0a00640 1 --2- 172.21.15.33:0/3656367310 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9dec108ef0 0x7f9dec19f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:28.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:28.517+0000 7f9df0a00640 1 -- 172.21.15.33:0/3656367310 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9dec109870 msgr2=0x7f9dec19fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:28.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:28.517+0000 7f9df0a00640 1 --2- 172.21.15.33:0/3656367310 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9dec109870 0x7f9dec19fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:28.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:28.517+0000 7f9df0a00640 1 -- 172.21.15.33:0/3656367310 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dec1a44f0 con 0x7f9dec10ee80 2024-05-03T18:27:28.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:28.517+0000 7f9debe00640 1 --2- 172.21.15.33:0/3656367310 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9dec108ef0 0x7f9dec19f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:27:28.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:28.518+0000 7f9df0a00640 1 --2- 172.21.15.33:0/3656367310 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9dec10ee80 0x7f9dec1a3dc0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f9de4005e30 tx=0x7f9de4003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:27:28.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:28.518+0000 7f9de8c00640 1 -- 172.21.15.33:0/3656367310 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9de4004040 con 0x7f9dec10ee80 2024-05-03T18:27:28.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:28.518+0000 7f9de8c00640 1 -- 172.21.15.33:0/3656367310 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9de4004670 con 0x7f9dec10ee80 2024-05-03T18:27:28.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:28.518+0000 7f9de8c00640 1 -- 172.21.15.33:0/3656367310 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9de4030d70 con 0x7f9dec10ee80 2024-05-03T18:27:28.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:28.518+0000 7f9df1400640 1 -- 172.21.15.33:0/3656367310 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dec1a4780 con 0x7f9dec10ee80 2024-05-03T18:27:28.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:28.518+0000 7f9df1400640 1 -- 172.21.15.33:0/3656367310 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dec1ac070 con 0x7f9dec10ee80 2024-05-03T18:27:28.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:28.519+0000 7f9df1400640 1 -- 172.21.15.33:0/3656367310 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9dec050f90 con 0x7f9dec10ee80 2024-05-03T18:27:28.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:28.522+0000 7f9de8c00640 1 -- 172.21.15.33:0/3656367310 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 23) ==== 149828+0+0 (secure 0 0 0) 0x7f9de40305a0 con 0x7f9dec10ee80 2024-05-03T18:27:28.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:28.524+0000 7f9de8c00640 1 --2- 172.21.15.33:0/3656367310 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f9dd00b2a60 0x7f9dd00b4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:28.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:28.524+0000 7f9de8c00640 1 -- 172.21.15.33:0/3656367310 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f9de402e030 con 0x7f9dec10ee80 2024-05-03T18:27:28.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:28.524+0000 7f9debe00640 1 --2- 172.21.15.33:0/3656367310 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f9dd00b2a60 0x7f9dd00b4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:28.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:28.525+0000 7f9debe00640 1 --2- 172.21.15.33:0/3656367310 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f9dd00b2a60 0x7f9dd00b4f20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc009ad0 tx=0x7f9ddc005f90 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:27:28.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:28.526+0000 7f9de8c00640 1 -- 172.21.15.33:0/3656367310 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f9de40cb3e0 con 0x7f9dec10ee80 2024-05-03T18:27:28.795 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:28 smithi033 ceph-mon[28330]: pgmap v11: 129 pgs: 129 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-03T18:27:28.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:28.819+0000 7f9df1400640 1 -- 172.21.15.33:0/3656367310 --> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7f9dec062df0 con 0x7f9dd00b2a60 2024-05-03T18:27:28.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:28.986+0000 7f9de8c00640 1 -- 172.21.15.33:0/3656367310 <== mgr.15120 v2:172.21.15.33:6800/3302011241 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7f9dec062df0 con 0x7f9dd00b2a60 2024-05-03T18:27:28.988 INFO:teuthology.orchestra.run.smithi033.stdout:Retention added to path / 2024-05-03T18:27:28.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:28.989+0000 7f9df1400640 1 -- 172.21.15.33:0/3656367310 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f9dd00b2a60 msgr2=0x7f9dd00b4f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:28.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:28.989+0000 7f9df1400640 1 --2- 172.21.15.33:0/3656367310 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f9dd00b2a60 0x7f9dd00b4f20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc009ad0 tx=0x7f9ddc005f90 comp rx=0 tx=0).stop 2024-05-03T18:27:28.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:28.990+0000 7f9df1400640 1 -- 172.21.15.33:0/3656367310 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9dec10ee80 msgr2=0x7f9dec1a3dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:28.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:28.990+0000 7f9df1400640 1 --2- 172.21.15.33:0/3656367310 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9dec10ee80 0x7f9dec1a3dc0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f9de4005e30 tx=0x7f9de4003ba0 comp rx=0 tx=0).stop 2024-05-03T18:27:28.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:28.990+0000 7f9df1400640 1 -- 172.21.15.33:0/3656367310 shutdown_connections 2024-05-03T18:27:28.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:28.990+0000 7f9df1400640 1 --2- 172.21.15.33:0/3656367310 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f9dd00b2a60 0x7f9dd00b4f20 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:28.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:28.990+0000 7f9df1400640 1 --2- 172.21.15.33:0/3656367310 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9dec10ee80 0x7f9dec1a3dc0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:28.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:28.990+0000 7f9df1400640 1 --2- 172.21.15.33:0/3656367310 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9dec109870 0x7f9dec19fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:28.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:28.990+0000 7f9df1400640 1 --2- 172.21.15.33:0/3656367310 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9dec108ef0 0x7f9dec19f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:28.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:28.990+0000 7f9df1400640 1 -- 172.21.15.33:0/3656367310 >> 172.21.15.33:0/3656367310 conn(0x7f9dec078790 msgr2=0x7f9dec0799f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:27:28.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:28.990+0000 7f9df1400640 1 -- 172.21.15.33:0/3656367310 shutdown_connections 2024-05-03T18:27:28.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:28.990+0000 7f9df1400640 1 -- 172.21.15.33:0/3656367310 wait complete. 2024-05-03T18:27:29.009 DEBUG:teuthology.orchestra.run.smithi033:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-05-03T18:27:29.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.021+0000 7f262de00700 1 -- 172.21.15.33:0/413825234 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f262810fea0 msgr2=0x7f2628112290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:29.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.021+0000 7f262de00700 1 --2- 172.21.15.33:0/413825234 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f262810fea0 0x7f2628112290 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f262000b0a0 tx=0x7f262000b3b0 comp rx=0 tx=0).stop 2024-05-03T18:27:29.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.021+0000 7f262de00700 1 -- 172.21.15.33:0/413825234 shutdown_connections 2024-05-03T18:27:29.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.022+0000 7f262de00700 1 --2- 172.21.15.33:0/413825234 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f262810fea0 0x7f2628112290 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:29.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.022+0000 7f262de00700 1 --2- 172.21.15.33:0/413825234 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26280756d0 0x7f2628075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:29.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.022+0000 7f262de00700 1 --2- 172.21.15.33:0/413825234 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2628075ff0 0x7f262810f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:29.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.022+0000 7f262de00700 1 -- 172.21.15.33:0/413825234 >> 172.21.15.33:0/413825234 conn(0x7f262807a650 msgr2=0x7f262807aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:27:29.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.022+0000 7f262de00700 1 -- 172.21.15.33:0/413825234 shutdown_connections 2024-05-03T18:27:29.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.022+0000 7f262de00700 1 -- 172.21.15.33:0/413825234 wait complete. 2024-05-03T18:27:29.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.023+0000 7f262de00700 1 Processor -- start 2024-05-03T18:27:29.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.023+0000 7f262de00700 1 -- start start 2024-05-03T18:27:29.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.023+0000 7f262de00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26280756d0 0x7f26281a18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:29.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.023+0000 7f262de00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2628075ff0 0x7f26281a1e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:29.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.023+0000 7f262de00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f262810fea0 0x7f262819ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:29.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.023+0000 7f262de00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2628115730 con 0x7f262810fea0 2024-05-03T18:27:29.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.023+0000 7f262de00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f26281155b0 con 0x7f26280756d0 2024-05-03T18:27:29.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.023+0000 7f262de00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f26281158b0 con 0x7f2628075ff0 2024-05-03T18:27:29.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.024+0000 7f2627400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f262810fea0 0x7f262819ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:29.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.024+0000 7f2627400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f262810fea0 0x7f262819ba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40896/0 (socket says 172.21.15.33:40896) 2024-05-03T18:27:29.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.024+0000 7f2627400700 1 -- 172.21.15.33:0/3957713055 learned_addr learned my addr 172.21.15.33:0/3957713055 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:27:29.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.024+0000 7f2626a00700 1 --2- 172.21.15.33:0/3957713055 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26280756d0 0x7f26281a18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:29.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.024+0000 7f2626000700 1 --2- 172.21.15.33:0/3957713055 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2628075ff0 0x7f26281a1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:29.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.024+0000 7f2627400700 1 -- 172.21.15.33:0/3957713055 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2628075ff0 msgr2=0x7f26281a1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:29.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.024+0000 7f2627400700 1 --2- 172.21.15.33:0/3957713055 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2628075ff0 0x7f26281a1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:29.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.024+0000 7f2627400700 1 -- 172.21.15.33:0/3957713055 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26280756d0 msgr2=0x7f26281a18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:29.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.024+0000 7f2627400700 1 --2- 172.21.15.33:0/3957713055 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26280756d0 0x7f26281a18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:29.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.024+0000 7f2627400700 1 -- 172.21.15.33:0/3957713055 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f262819c1f0 con 0x7f262810fea0 2024-05-03T18:27:29.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.024+0000 7f2626000700 1 --2- 172.21.15.33:0/3957713055 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2628075ff0 0x7f26281a1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:27:29.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.025+0000 7f2627400700 1 --2- 172.21.15.33:0/3957713055 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f262810fea0 0x7f262819ba90 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f2620009cd0 tx=0x7f2620004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:27:29.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.025+0000 7f262ca00700 1 -- 172.21.15.33:0/3957713055 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2620021070 con 0x7f262810fea0 2024-05-03T18:27:29.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.025+0000 7f262ca00700 1 -- 172.21.15.33:0/3957713055 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2620007b80 con 0x7f262810fea0 2024-05-03T18:27:29.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.025+0000 7f262ca00700 1 -- 172.21.15.33:0/3957713055 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2620005040 con 0x7f262810fea0 2024-05-03T18:27:29.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.025+0000 7f262de00700 1 -- 172.21.15.33:0/3957713055 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f262819c480 con 0x7f262810fea0 2024-05-03T18:27:29.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.025+0000 7f262de00700 1 -- 172.21.15.33:0/3957713055 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26281a87a0 con 0x7f262810fea0 2024-05-03T18:27:29.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.029+0000 7f262ca00700 1 -- 172.21.15.33:0/3957713055 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 23) ==== 149828+0+0 (secure 0 0 0) 0x7f2620024070 con 0x7f262810fea0 2024-05-03T18:27:29.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.030+0000 7f26386fcb80 1 Processor -- start 2024-05-03T18:27:29.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.030+0000 7f26386fcb80 1 -- start start 2024-05-03T18:27:29.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.031+0000 7f26386fcb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ed96ba9b0 0x559ed96bad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:29.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.031+0000 7f26386fcb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ed96bb2d0 0x559ed96b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:29.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.031+0000 7f26386fcb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559ed96b55d0 0x559ed96b5a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:29.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.031+0000 7f26386fcb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x559ed9607aa0 con 0x559ed96bb2d0 2024-05-03T18:27:29.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.031+0000 7f26386fcb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x559ed95a57e0 con 0x559ed96ba9b0 2024-05-03T18:27:29.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.031+0000 7f26386fcb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x559ed9603a60 con 0x559ed96b55d0 2024-05-03T18:27:29.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.031+0000 7f262ca00700 1 --2- 172.21.15.33:0/3957713055 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f260c0b2b00 0x7f260c0b4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:29.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.031+0000 7f262ca00700 1 -- 172.21.15.33:0/3957713055 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f262000f020 con 0x7f262810fea0 2024-05-03T18:27:29.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.031+0000 7f2626000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ed96ba9b0 0x559ed96bad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:29.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.031+0000 7f2626000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559ed96b55d0 0x559ed96b5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:29.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.031+0000 7f2626a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ed96bb2d0 0x559ed96b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:29.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.031+0000 7f2626a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ed96bb2d0 0x559ed96b4fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40906/0 (socket says 172.21.15.33:40906) 2024-05-03T18:27:29.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.031+0000 7f2626a00700 1 -- 172.21.15.33:0/679016844 learned_addr learned my addr 172.21.15.33:0/679016844 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:27:29.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.032+0000 7f2627400700 1 --2- 172.21.15.33:0/3957713055 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f260c0b2b00 0x7f260c0b4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:29.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.032+0000 7f2626a00700 1 -- 172.21.15.33:0/679016844 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559ed96b55d0 msgr2=0x559ed96b5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:27:29.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.032+0000 7f2626a00700 1 --2- 172.21.15.33:0/679016844 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559ed96b55d0 0x559ed96b5a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:29.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.032+0000 7f2626a00700 1 -- 172.21.15.33:0/679016844 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ed96ba9b0 msgr2=0x559ed96bad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:27:29.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.032+0000 7f2626a00700 1 --2- 172.21.15.33:0/679016844 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ed96ba9b0 0x559ed96bad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:29.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.032+0000 7f2626a00700 1 -- 172.21.15.33:0/679016844 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ed9554950 con 0x559ed96bb2d0 2024-05-03T18:27:29.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.033+0000 7f2626a00700 1 --2- 172.21.15.33:0/679016844 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ed96bb2d0 0x559ed96b4fb0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f261800ac70 tx=0x7f261800af80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9b51c831bf4f851 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:27:29.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.033+0000 7f2627400700 1 --2- 172.21.15.33:0/3957713055 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f260c0b2b00 0x7f260c0b4fc0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f26200e1420 tx=0x7f2620009040 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:27:29.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.033+0000 7f261cc00700 1 -- 172.21.15.33:0/679016844 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f261800fde0 con 0x559ed96bb2d0 2024-05-03T18:27:29.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.033+0000 7f261cc00700 1 -- 172.21.15.33:0/679016844 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2618004cc0 con 0x559ed96bb2d0 2024-05-03T18:27:29.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.033+0000 7f261cc00700 1 -- 172.21.15.33:0/679016844 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2618018440 con 0x559ed96bb2d0 2024-05-03T18:27:29.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.034+0000 7f26386fcb80 1 -- 172.21.15.33:0/679016844 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ed96bb2d0 msgr2=0x559ed96b4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:29.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.034+0000 7f26386fcb80 1 --2- 172.21.15.33:0/679016844 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ed96bb2d0 0x559ed96b4fb0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f261800ac70 tx=0x7f261800af80 comp rx=0 tx=0).stop 2024-05-03T18:27:29.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.035+0000 7f26386fcb80 1 -- 172.21.15.33:0/679016844 shutdown_connections 2024-05-03T18:27:29.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.035+0000 7f26386fcb80 1 --2- 172.21.15.33:0/679016844 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559ed96b55d0 0x559ed96b5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:29.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.035+0000 7f26386fcb80 1 --2- 172.21.15.33:0/679016844 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ed96ba9b0 0x559ed96bad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:29.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.035+0000 7f26386fcb80 1 --2- 172.21.15.33:0/679016844 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ed96bb2d0 0x559ed96b4fb0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:29.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.035+0000 7f26386fcb80 1 -- 172.21.15.33:0/679016844 >> 172.21.15.33:0/679016844 conn(0x559ed962d750 msgr2=0x559ed96af980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:27:29.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.035+0000 7f26386fcb80 1 -- 172.21.15.33:0/679016844 shutdown_connections 2024-05-03T18:27:29.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.035+0000 7f26386fcb80 1 -- 172.21.15.33:0/679016844 wait complete. 2024-05-03T18:27:29.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.035+0000 7f26386fcb80 1 -- start start 2024-05-03T18:27:29.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.036+0000 7f26386fcb80 1 Processor -- start 2024-05-03T18:27:29.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.036+0000 7f26386fcb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559ed96b55d0 0x559ed96c0e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:29.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.036+0000 7f26386fcb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ed96ba9b0 0x559ed96be220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:29.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.036+0000 7f26386fcb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ed96bb2d0 0x559ed96be760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:29.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.036+0000 7f26386fcb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x559ed96c1d80 con 0x559ed96ba9b0 2024-05-03T18:27:29.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.036+0000 7f26386fcb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x559ed956dba0 con 0x559ed96bb2d0 2024-05-03T18:27:29.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.036+0000 7f26386fcb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x559ed96c1f00 con 0x559ed96b55d0 2024-05-03T18:27:29.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.036+0000 7f26386fcb80 10 client.0 _ll_register_callbacks cb 0x559ed95aecf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:27:29.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.037+0000 7f2626a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ed96ba9b0 0x559ed96be220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:29.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.037+0000 7f2626a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ed96ba9b0 0x559ed96be220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40914/0 (socket says 172.21.15.33:40914) 2024-05-03T18:27:29.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.037+0000 7f2626a00700 1 -- 172.21.15.33:0/1401968996 learned_addr learned my addr 172.21.15.33:0/1401968996 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:27:29.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.037+0000 7f2626000700 1 --2- 172.21.15.33:0/1401968996 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559ed96b55d0 0x559ed96c0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:29.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.037+0000 7f2626000700 1 --2- 172.21.15.33:0/1401968996 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ed96bb2d0 0x559ed96be760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:29.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.037+0000 7f2626a00700 1 -- 172.21.15.33:0/1401968996 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559ed96b55d0 msgr2=0x559ed96c0e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:27:29.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.037+0000 7f2626a00700 1 --2- 172.21.15.33:0/1401968996 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559ed96b55d0 0x559ed96c0e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:29.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.037+0000 7f2626a00700 1 -- 172.21.15.33:0/1401968996 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ed96bb2d0 msgr2=0x559ed96be760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:27:29.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.037+0000 7f2626a00700 1 --2- 172.21.15.33:0/1401968996 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ed96bb2d0 0x559ed96be760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:29.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.037+0000 7f2626a00700 1 -- 172.21.15.33:0/1401968996 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ed95cd4a0 con 0x559ed96ba9b0 2024-05-03T18:27:29.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.037+0000 7f2626000700 1 --2- 172.21.15.33:0/1401968996 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559ed96b55d0 0x559ed96c0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:27:29.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.038+0000 7f2626a00700 1 --2- 172.21.15.33:0/1401968996 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ed96ba9b0 0x559ed96be220 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f2618005700 tx=0x7f26180057e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d498da192bcd40c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:27:29.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.038+0000 7f2611600700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:27:29.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.038+0000 7f2611600700 1 -- 172.21.15.33:0/1401968996 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2618026070 con 0x559ed96ba9b0 2024-05-03T18:27:29.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.038+0000 7f2611600700 1 -- 172.21.15.33:0/1401968996 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f2618004750 con 0x559ed96ba9b0 2024-05-03T18:27:29.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.038+0000 7f26386fcb80 1 -- 172.21.15.33:0/1401968996 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2618004cc0 con 0x559ed96ba9b0 2024-05-03T18:27:29.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.039+0000 7f2611600700 1 -- 172.21.15.33:0/1401968996 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f2618030050 con 0x559ed96ba9b0 2024-05-03T18:27:29.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.039+0000 7f26386fcb80 10 client.15288 fetch_fsmap learned FSMap version 21 2024-05-03T18:27:29.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.039+0000 7f26386fcb80 1 -- 172.21.15.33:0/1401968996 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x559ed96d4ca0 con 0x559ed96ba9b0 2024-05-03T18:27:29.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.039+0000 7f2611600700 1 -- 172.21.15.33:0/1401968996 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2618004cc0 con 0x559ed96ba9b0 2024-05-03T18:27:29.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.039+0000 7f2611600700 1 -- 172.21.15.33:0/1401968996 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f2618018440 con 0x559ed96ba9b0 2024-05-03T18:27:29.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.040+0000 7f26386fcb80 10 client.15288 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:27:29.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.040+0000 7f26386fcb80 10 client.15288 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:27:29.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.040+0000 7f26386fcb80 20 client.15288 populate_metadata read hostname 'smithi033' 2024-05-03T18:27:29.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.040+0000 7f26386fcb80 1 --2- 172.21.15.33:0/1401968996 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x559ed96d7280 0x559ed96d9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:29.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.040+0000 7f26386fcb80 4 client.15288 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:27:29.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.040+0000 7f26386fcb80 1 -- 172.21.15.33:0/1401968996 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559ed9682070 con 0x559ed96d7280 2024-05-03T18:27:29.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.040+0000 7f2626000700 1 --2- 172.21.15.33:0/1401968996 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x559ed96d7280 0x559ed96d9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:29.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.041+0000 7f2626000700 1 --2- 172.21.15.33:0/1401968996 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x559ed96d7280 0x559ed96d9670 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c33bc08582cd4fd7 server_cookie=df74bd9154c47b4 in_seq=0 out_seq=0 2024-05-03T18:27:29.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.041+0000 7f2611600700 10 client.15288 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:27:29.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.043+0000 7f2611600700 1 -- 172.21.15.33:0/1401968996 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559ed9682070 con 0x559ed96d7280 2024-05-03T18:27:29.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.043+0000 7f2611600700 10 client.15288 handle_command_reply: tid=0 2024-05-03T18:27:29.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.043+0000 7f26386fcb80 1 client.15288 shutdown 2024-05-03T18:27:29.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.043+0000 7f26386fcb80 1 -- 172.21.15.33:0/1401968996 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ed96ba9b0 msgr2=0x559ed96be220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:29.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.043+0000 7f26386fcb80 1 --2- 172.21.15.33:0/1401968996 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ed96ba9b0 0x559ed96be220 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f2618005700 tx=0x7f26180057e0 comp rx=0 tx=0).stop 2024-05-03T18:27:29.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.044+0000 7f26386fcb80 1 -- 172.21.15.33:0/1401968996 shutdown_connections 2024-05-03T18:27:29.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.044+0000 7f26386fcb80 1 --2- 172.21.15.33:0/1401968996 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x559ed96d7280 0x559ed96d9670 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:29.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.044+0000 7f26386fcb80 1 --2- 172.21.15.33:0/1401968996 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ed96bb2d0 0x559ed96be760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:29.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.044+0000 7f26386fcb80 1 --2- 172.21.15.33:0/1401968996 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559ed96b55d0 0x559ed96c0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:29.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.044+0000 7f26386fcb80 1 --2- 172.21.15.33:0/1401968996 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ed96ba9b0 0x559ed96be220 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:29.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.044+0000 7f26386fcb80 1 -- 172.21.15.33:0/1401968996 >> 172.21.15.33:0/1401968996 conn(0x559ed962d750 msgr2=0x559ed96aa430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:27:29.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.044+0000 7f2611600700 0 client.15288 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:27:29.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.044+0000 7f26386fcb80 1 -- 172.21.15.33:0/1401968996 shutdown_connections 2024-05-03T18:27:29.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.044+0000 7f26386fcb80 1 -- 172.21.15.33:0/1401968996 wait complete. 2024-05-03T18:27:29.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.044+0000 7f26386fcb80 20 client.15288 trim_cache size 0 max 16384 2024-05-03T18:27:29.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:28 smithi099 ceph-mon[31658]: pgmap v11: 129 pgs: 129 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-03T18:27:29.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.066+0000 7f26386fcb80 1 Processor -- start 2024-05-03T18:27:29.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.066+0000 7f26386fcb80 1 -- start start 2024-05-03T18:27:29.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.066+0000 7f26386fcb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ed96b55d0 0x559ed96b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:29.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.066+0000 7f26386fcb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559ed96bb2d0 0x559ed96bbde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:29.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.067+0000 7f26386fcb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ed96bca20 0x559ed96bce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:29.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.067+0000 7f26386fcb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f26281ab1a0 con 0x559ed96bca20 2024-05-03T18:27:29.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.067+0000 7f26386fcb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f26281ada20 con 0x559ed96b55d0 2024-05-03T18:27:29.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.067+0000 7f26386fcb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f26281ab810 con 0x559ed96bb2d0 2024-05-03T18:27:29.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.067+0000 7f2626000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ed96bca20 0x559ed96bce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:29.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.067+0000 7f2626000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ed96bca20 0x559ed96bce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40928/0 (socket says 172.21.15.33:40928) 2024-05-03T18:27:29.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.067+0000 7f2626000700 1 -- 172.21.15.33:0/754953759 learned_addr learned my addr 172.21.15.33:0/754953759 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:27:29.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.067+0000 7f2626000700 1 --2- 172.21.15.33:0/754953759 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ed96b55d0 0x559ed96b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:29.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.067+0000 7f2626a00700 1 --2- 172.21.15.33:0/754953759 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559ed96bb2d0 0x559ed96bbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:29.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.067+0000 7f2626000700 1 -- 172.21.15.33:0/754953759 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559ed96bb2d0 msgr2=0x559ed96bbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:27:29.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.067+0000 7f2626000700 1 --2- 172.21.15.33:0/754953759 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559ed96bb2d0 0x559ed96bbde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:29.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.067+0000 7f2626000700 1 -- 172.21.15.33:0/754953759 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ed96b55d0 msgr2=0x559ed96b4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:27:29.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.067+0000 7f2626000700 1 --2- 172.21.15.33:0/754953759 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ed96b55d0 0x559ed96b4fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:29.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.067+0000 7f2626000700 1 -- 172.21.15.33:0/754953759 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ed96c0ac0 con 0x559ed96bca20 2024-05-03T18:27:29.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.068+0000 7f2626a00700 1 --2- 172.21.15.33:0/754953759 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559ed96bb2d0 0x559ed96bbde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:27:29.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.068+0000 7f2626000700 1 --2- 172.21.15.33:0/754953759 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ed96bca20 0x559ed96bce60 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f2614013880 tx=0x7f26140047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fca2e3cae1aa8ef5 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:27:29.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.068+0000 7f2613e00700 1 -- 172.21.15.33:0/754953759 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2614004020 con 0x559ed96bca20 2024-05-03T18:27:29.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.068+0000 7f2613e00700 1 -- 172.21.15.33:0/754953759 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2614004830 con 0x559ed96bca20 2024-05-03T18:27:29.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.068+0000 7f2613e00700 1 -- 172.21.15.33:0/754953759 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f261400fa20 con 0x559ed96bca20 2024-05-03T18:27:29.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.069+0000 7f26386fcb80 1 -- 172.21.15.33:0/754953759 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ed96bca20 msgr2=0x559ed96bce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:29.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.069+0000 7f26386fcb80 1 --2- 172.21.15.33:0/754953759 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ed96bca20 0x559ed96bce60 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f2614013880 tx=0x7f26140047c0 comp rx=0 tx=0).stop 2024-05-03T18:27:29.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.069+0000 7f26386fcb80 1 -- 172.21.15.33:0/754953759 shutdown_connections 2024-05-03T18:27:29.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.069+0000 7f26386fcb80 1 --2- 172.21.15.33:0/754953759 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ed96bca20 0x559ed96bce60 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:29.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.069+0000 7f26386fcb80 1 --2- 172.21.15.33:0/754953759 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ed96b55d0 0x559ed96b4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:29.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.069+0000 7f26386fcb80 1 --2- 172.21.15.33:0/754953759 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559ed96bb2d0 0x559ed96bbde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:29.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.069+0000 7f26386fcb80 1 -- 172.21.15.33:0/754953759 >> 172.21.15.33:0/754953759 conn(0x559ed962d750 msgr2=0x559ed96aa430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:27:29.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.069+0000 7f26386fcb80 1 -- 172.21.15.33:0/754953759 shutdown_connections 2024-05-03T18:27:29.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.070+0000 7f26386fcb80 1 -- 172.21.15.33:0/754953759 wait complete. 2024-05-03T18:27:29.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.070+0000 7f26386fcb80 1 -- start start 2024-05-03T18:27:29.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.070+0000 7f26386fcb80 1 Processor -- start 2024-05-03T18:27:29.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.071+0000 7f26386fcb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ed96b55d0 0x559ed96dc720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:29.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.071+0000 7f26386fcb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559ed96bb2d0 0x559ed96daad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:29.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.071+0000 7f26386fcb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ed96bca20 0x559ed96db010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:29.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.071+0000 7f26386fcb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f26281abee0 con 0x559ed96bca20 2024-05-03T18:27:29.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.071+0000 7f26386fcb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f26281ad350 con 0x559ed96b55d0 2024-05-03T18:27:29.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.071+0000 7f26386fcb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f26281ac5b0 con 0x559ed96bb2d0 2024-05-03T18:27:29.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.071+0000 7f26386fcb80 10 client.0 _ll_register_callbacks cb 0x559ed95aecf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:27:29.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.071+0000 7f2626000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ed96bca20 0x559ed96db010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:29.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.071+0000 7f2626000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ed96bca20 0x559ed96db010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40944/0 (socket says 172.21.15.33:40944) 2024-05-03T18:27:29.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.071+0000 7f2626000700 1 -- 172.21.15.33:0/3341358168 learned_addr learned my addr 172.21.15.33:0/3341358168 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:27:29.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.071+0000 7f2626000700 1 --2- 172.21.15.33:0/3341358168 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ed96b55d0 0x559ed96dc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:29.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.072+0000 7f2626a00700 1 --2- 172.21.15.33:0/3341358168 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559ed96bb2d0 0x559ed96daad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:29.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.072+0000 7f2626000700 1 -- 172.21.15.33:0/3341358168 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559ed96bb2d0 msgr2=0x559ed96daad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:27:29.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.072+0000 7f2626000700 1 --2- 172.21.15.33:0/3341358168 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559ed96bb2d0 0x559ed96daad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:29.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.072+0000 7f2626000700 1 -- 172.21.15.33:0/3341358168 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ed96b55d0 msgr2=0x559ed96dc720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:27:29.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.072+0000 7f2626000700 1 --2- 172.21.15.33:0/3341358168 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ed96b55d0 0x559ed96dc720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:29.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.072+0000 7f2626000700 1 -- 172.21.15.33:0/3341358168 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ed96db710 con 0x559ed96bca20 2024-05-03T18:27:29.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.072+0000 7f2626a00700 1 --2- 172.21.15.33:0/3341358168 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559ed96bb2d0 0x559ed96daad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:27:29.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.072+0000 7f2626000700 1 --2- 172.21.15.33:0/3341358168 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ed96bca20 0x559ed96db010 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f2614005f70 tx=0x7f2614004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5549042c843f5871 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:27:29.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.072+0000 7f2611600700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:27:29.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.072+0000 7f2611600700 1 -- 172.21.15.33:0/3341358168 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2614013930 con 0x559ed96bca20 2024-05-03T18:27:29.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.072+0000 7f2611600700 1 -- 172.21.15.33:0/3341358168 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f2614013ad0 con 0x559ed96bca20 2024-05-03T18:27:29.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.072+0000 7f26386fcb80 1 -- 172.21.15.33:0/3341358168 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2614004830 con 0x559ed96bca20 2024-05-03T18:27:29.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.073+0000 7f2611600700 1 -- 172.21.15.33:0/3341358168 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f261400f800 con 0x559ed96bca20 2024-05-03T18:27:29.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.073+0000 7f26386fcb80 10 client.15294 fetch_fsmap learned FSMap version 21 2024-05-03T18:27:29.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.073+0000 7f26386fcb80 1 -- 172.21.15.33:0/3341358168 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x559ed9742690 con 0x559ed96bca20 2024-05-03T18:27:29.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.073+0000 7f2611600700 1 -- 172.21.15.33:0/3341358168 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2614004830 con 0x559ed96bca20 2024-05-03T18:27:29.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.073+0000 7f2611600700 1 -- 172.21.15.33:0/3341358168 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f26140208d0 con 0x559ed96bca20 2024-05-03T18:27:29.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.073+0000 7f26386fcb80 10 client.15294 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:27:29.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.073+0000 7f26386fcb80 10 client.15294 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:27:29.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.073+0000 7f26386fcb80 20 client.15294 populate_metadata read hostname 'smithi033' 2024-05-03T18:27:29.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.073+0000 7f26386fcb80 1 --2- 172.21.15.33:0/3341358168 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x559ed9742cc0 0x559ed97450e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:29.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.073+0000 7f26386fcb80 4 client.15294 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-03T18:27:29.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.073+0000 7f26386fcb80 1 -- 172.21.15.33:0/3341358168 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559ed9615480 con 0x559ed9742cc0 2024-05-03T18:27:29.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.074+0000 7f2626a00700 1 --2- 172.21.15.33:0/3341358168 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x559ed9742cc0 0x559ed97450e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:29.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.075+0000 7f2626a00700 1 --2- 172.21.15.33:0/3341358168 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x559ed9742cc0 0x559ed97450e0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7432c0a00f3f4daf server_cookie=d09d70a45e2a2387 in_seq=0 out_seq=0 2024-05-03T18:27:29.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.075+0000 7f2611600700 10 client.15294 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:27:29.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.078+0000 7f2611600700 1 -- 172.21.15.33:0/3341358168 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559ed9615480 con 0x559ed9742cc0 2024-05-03T18:27:29.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.078+0000 7f2611600700 10 client.15294 handle_command_reply: tid=0 2024-05-03T18:27:29.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.078+0000 7f26386fcb80 1 client.15294 shutdown 2024-05-03T18:27:29.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.078+0000 7f26386fcb80 1 -- 172.21.15.33:0/3341358168 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ed96bca20 msgr2=0x559ed96db010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:29.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.078+0000 7f26386fcb80 1 --2- 172.21.15.33:0/3341358168 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ed96bca20 0x559ed96db010 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f2614005f70 tx=0x7f2614004750 comp rx=0 tx=0).stop 2024-05-03T18:27:29.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.078+0000 7f26386fcb80 1 -- 172.21.15.33:0/3341358168 shutdown_connections 2024-05-03T18:27:29.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.078+0000 7f26386fcb80 1 --2- 172.21.15.33:0/3341358168 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x559ed9742cc0 0x559ed97450e0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:29.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.078+0000 7f26386fcb80 1 --2- 172.21.15.33:0/3341358168 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ed96bca20 0x559ed96db010 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:29.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.078+0000 7f26386fcb80 1 --2- 172.21.15.33:0/3341358168 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ed96b55d0 0x559ed96dc720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:29.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.078+0000 7f26386fcb80 1 --2- 172.21.15.33:0/3341358168 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559ed96bb2d0 0x559ed96daad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:29.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.078+0000 7f2611600700 0 client.15294 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:27:29.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.078+0000 7f26386fcb80 1 -- 172.21.15.33:0/3341358168 >> 172.21.15.33:0/3341358168 conn(0x559ed962d750 msgr2=0x559ed96aa430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:27:29.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.079+0000 7f26386fcb80 1 -- 172.21.15.33:0/3341358168 shutdown_connections 2024-05-03T18:27:29.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.079+0000 7f26386fcb80 1 -- 172.21.15.33:0/3341358168 wait complete. 2024-05-03T18:27:29.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.079+0000 7f26386fcb80 20 client.15294 trim_cache size 0 max 16384 2024-05-03T18:27:29.081 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-03T18:27:29.081 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-05-03T18:27:29.081 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "3c637612-1754-4b99-bd79-a359d97aa674", 2024-05-03T18:27:29.081 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-05-03T18:27:29.081 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-03T18:27:29.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.080+0000 7f262de00700 1 -- 172.21.15.33:0/3957713055 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f260c0b2b00 msgr2=0x7f260c0b4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:29.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.080+0000 7f262de00700 1 --2- 172.21.15.33:0/3957713055 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f260c0b2b00 0x7f260c0b4fc0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f26200e1420 tx=0x7f2620009040 comp rx=0 tx=0).stop 2024-05-03T18:27:29.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.080+0000 7f262de00700 1 -- 172.21.15.33:0/3957713055 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f262810fea0 msgr2=0x7f262819ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:29.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.080+0000 7f262de00700 1 --2- 172.21.15.33:0/3957713055 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f262810fea0 0x7f262819ba90 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f2620009cd0 tx=0x7f2620004ce0 comp rx=0 tx=0).stop 2024-05-03T18:27:29.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.080+0000 7f262de00700 1 -- 172.21.15.33:0/3957713055 shutdown_connections 2024-05-03T18:27:29.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.080+0000 7f262de00700 1 --2- 172.21.15.33:0/3957713055 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f262810fea0 0x7f262819ba90 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:29.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.080+0000 7f262de00700 1 --2- 172.21.15.33:0/3957713055 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f260c0b2b00 0x7f260c0b4fc0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:29.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.080+0000 7f262de00700 1 --2- 172.21.15.33:0/3957713055 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26280756d0 0x7f26281a18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:29.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.080+0000 7f262de00700 1 --2- 172.21.15.33:0/3957713055 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2628075ff0 0x7f26281a1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:29.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.080+0000 7f262de00700 1 -- 172.21.15.33:0/3957713055 >> 172.21.15.33:0/3957713055 conn(0x7f262807a650 msgr2=0x7f2628100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:27:29.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.080+0000 7f262de00700 1 -- 172.21.15.33:0/3957713055 shutdown_connections 2024-05-03T18:27:29.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.080+0000 7f262de00700 1 -- 172.21.15.33:0/3957713055 wait complete. 2024-05-03T18:27:29.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:28 smithi150 ceph-mon[32492]: pgmap v11: 129 pgs: 129 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-03T18:27:29.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.192+0000 7f744fe00640 1 -- 172.21.15.33:0/254812232 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7450101f30 msgr2=0x7f745010eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:29.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.192+0000 7f744fe00640 1 --2- 172.21.15.33:0/254812232 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7450101f30 0x7f745010eb60 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f7438009900 tx=0x7f743802a3f0 comp rx=0 tx=0).stop 2024-05-03T18:27:29.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.192+0000 7f744fe00640 1 -- 172.21.15.33:0/254812232 shutdown_connections 2024-05-03T18:27:29.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.192+0000 7f744fe00640 1 --2- 172.21.15.33:0/254812232 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f745010f130 0x7f7450111520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:29.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.192+0000 7f744fe00640 1 --2- 172.21.15.33:0/254812232 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7450101f30 0x7f745010eb60 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:29.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.192+0000 7f744fe00640 1 --2- 172.21.15.33:0/254812232 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7450101580 0x7f7450101960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:29.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.192+0000 7f744fe00640 1 -- 172.21.15.33:0/254812232 >> 172.21.15.33:0/254812232 conn(0x7f74500fd1d0 msgr2=0x7f74500ff5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:27:29.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.192+0000 7f744fe00640 1 -- 172.21.15.33:0/254812232 shutdown_connections 2024-05-03T18:27:29.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.192+0000 7f744fe00640 1 -- 172.21.15.33:0/254812232 wait complete. 2024-05-03T18:27:29.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.193+0000 7f744fe00640 1 Processor -- start 2024-05-03T18:27:29.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.193+0000 7f744fe00640 1 -- start start 2024-05-03T18:27:29.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.193+0000 7f744fe00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7450101580 0x7f745019b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:29.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.193+0000 7f744fe00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7450101f30 0x7f745019b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:29.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.193+0000 7f744fe00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f745010f130 0x7f745019fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:29.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.193+0000 7f744fe00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7450114b50 con 0x7f7450101f30 2024-05-03T18:27:29.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.193+0000 7f744fe00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f74501149d0 con 0x7f7450101580 2024-05-03T18:27:29.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.193+0000 7f744fe00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f7450114cd0 con 0x7f745010f130 2024-05-03T18:27:29.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.194+0000 7f744e000640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7450101f30 0x7f745019b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:29.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.194+0000 7f744e000640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7450101f30 0x7f745019b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40956/0 (socket says 172.21.15.33:40956) 2024-05-03T18:27:29.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.194+0000 7f744e000640 1 -- 172.21.15.33:0/1796856205 learned_addr learned my addr 172.21.15.33:0/1796856205 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:27:29.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.194+0000 7f744f400640 1 --2- 172.21.15.33:0/1796856205 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f745010f130 0x7f745019fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:29.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.194+0000 7f744ea00640 1 --2- 172.21.15.33:0/1796856205 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7450101580 0x7f745019b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:29.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.194+0000 7f744e000640 1 -- 172.21.15.33:0/1796856205 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f745010f130 msgr2=0x7f745019fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:29.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.194+0000 7f744e000640 1 --2- 172.21.15.33:0/1796856205 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f745010f130 0x7f745019fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:29.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.194+0000 7f744e000640 1 -- 172.21.15.33:0/1796856205 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7450101580 msgr2=0x7f745019b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:29.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.194+0000 7f744e000640 1 --2- 172.21.15.33:0/1796856205 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7450101580 0x7f745019b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:29.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.194+0000 7f744e000640 1 -- 172.21.15.33:0/1796856205 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74501a0170 con 0x7f7450101f30 2024-05-03T18:27:29.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.194+0000 7f744ea00640 1 --2- 172.21.15.33:0/1796856205 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7450101580 0x7f745019b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:27:29.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.195+0000 7f744e000640 1 --2- 172.21.15.33:0/1796856205 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7450101f30 0x7f745019b710 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f743802a910 tx=0x7f7438003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:27:29.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.195+0000 7f7447400640 1 -- 172.21.15.33:0/1796856205 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f743803f070 con 0x7f7450101f30 2024-05-03T18:27:29.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.195+0000 7f7447400640 1 -- 172.21.15.33:0/1796856205 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7438003e40 con 0x7f7450101f30 2024-05-03T18:27:29.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.195+0000 7f744fe00640 1 -- 172.21.15.33:0/1796856205 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74501a0400 con 0x7f7450101f30 2024-05-03T18:27:29.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.195+0000 7f7447400640 1 -- 172.21.15.33:0/1796856205 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7438035030 con 0x7f7450101f30 2024-05-03T18:27:29.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.195+0000 7f744fe00640 1 -- 172.21.15.33:0/1796856205 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74501a7cf0 con 0x7f7450101f30 2024-05-03T18:27:29.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.196+0000 7f744fe00640 1 -- 172.21.15.33:0/1796856205 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7450050f90 con 0x7f7450101f30 2024-05-03T18:27:29.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.199+0000 7f7447400640 1 -- 172.21.15.33:0/1796856205 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 23) ==== 149828+0+0 (secure 0 0 0) 0x7f7438042070 con 0x7f7450101f30 2024-05-03T18:27:29.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.201+0000 7f7447400640 1 --2- 172.21.15.33:0/1796856205 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f74340b29e0 0x7f74340b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:29.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.201+0000 7f7447400640 1 -- 172.21.15.33:0/1796856205 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f7438103870 con 0x7f7450101f30 2024-05-03T18:27:29.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.202+0000 7f744ea00640 1 --2- 172.21.15.33:0/1796856205 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f74340b29e0 0x7f74340b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:29.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.202+0000 7f744ea00640 1 --2- 172.21.15.33:0/1796856205 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f74340b29e0 0x7f74340b4ea0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7440005fd0 tx=0x7f74400093e0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:27:29.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.204+0000 7f7447400640 1 -- 172.21.15.33:0/1796856205 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f74380cacf0 con 0x7f7450101f30 2024-05-03T18:27:29.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.436+0000 7f744fe00640 1 -- 172.21.15.33:0/1796856205 --> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f745010a260 con 0x7f74340b29e0 2024-05-03T18:27:29.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.458+0000 7f7447400640 1 -- 172.21.15.33:0/1796856205 <== mgr.15120 v2:172.21.15.33:6800/3302011241 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f745010a260 con 0x7f74340b29e0 2024-05-03T18:27:29.459 INFO:teuthology.orchestra.run.smithi033.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-05-03T00:00:00", "created": "2024-05-03T18:27:28", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-05-03T18:27:29.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.460+0000 7f744fe00640 1 -- 172.21.15.33:0/1796856205 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f74340b29e0 msgr2=0x7f74340b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:29.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.460+0000 7f744fe00640 1 --2- 172.21.15.33:0/1796856205 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f74340b29e0 0x7f74340b4ea0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7440005fd0 tx=0x7f74400093e0 comp rx=0 tx=0).stop 2024-05-03T18:27:29.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.460+0000 7f744fe00640 1 -- 172.21.15.33:0/1796856205 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7450101f30 msgr2=0x7f745019b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:29.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.460+0000 7f744fe00640 1 --2- 172.21.15.33:0/1796856205 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7450101f30 0x7f745019b710 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f743802a910 tx=0x7f7438003c20 comp rx=0 tx=0).stop 2024-05-03T18:27:29.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.461+0000 7f744fe00640 1 -- 172.21.15.33:0/1796856205 shutdown_connections 2024-05-03T18:27:29.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.461+0000 7f744fe00640 1 --2- 172.21.15.33:0/1796856205 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f74340b29e0 0x7f74340b4ea0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:29.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.461+0000 7f744fe00640 1 --2- 172.21.15.33:0/1796856205 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f745010f130 0x7f745019fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:29.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.461+0000 7f744fe00640 1 --2- 172.21.15.33:0/1796856205 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7450101f30 0x7f745019b710 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:29.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.461+0000 7f744fe00640 1 --2- 172.21.15.33:0/1796856205 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7450101580 0x7f745019b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:29.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.461+0000 7f744fe00640 1 -- 172.21.15.33:0/1796856205 >> 172.21.15.33:0/1796856205 conn(0x7f74500fd1d0 msgr2=0x7f74500ff5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:27:29.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.461+0000 7f744fe00640 1 -- 172.21.15.33:0/1796856205 shutdown_connections 2024-05-03T18:27:29.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.461+0000 7f744fe00640 1 -- 172.21.15.33:0/1796856205 wait complete. 2024-05-03T18:27:29.479 DEBUG:teuthology.orchestra.run.smithi033:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-05-03T18:27:29.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.730+0000 7f8d86600640 1 -- 172.21.15.33:0/3974177244 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8d80109870 msgr2=0x7f8d80115cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:29.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.730+0000 7f8d86600640 1 --2- 172.21.15.33:0/3974177244 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8d80109870 0x7f8d80115cb0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f8d78009930 tx=0x7f8d7802a420 comp rx=0 tx=0).stop 2024-05-03T18:27:29.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.730+0000 7f8d86600640 1 -- 172.21.15.33:0/3974177244 shutdown_connections 2024-05-03T18:27:29.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.730+0000 7f8d86600640 1 --2- 172.21.15.33:0/3974177244 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8d80109870 0x7f8d80115cb0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:29.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.730+0000 7f8d86600640 1 --2- 172.21.15.33:0/3974177244 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d80108ef0 0x7f8d80109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:29.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.730+0000 7f8d86600640 1 --2- 172.21.15.33:0/3974177244 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d8010ee80 0x7f8d8010f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:29.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.730+0000 7f8d86600640 1 -- 172.21.15.33:0/3974177244 >> 172.21.15.33:0/3974177244 conn(0x7f8d80078790 msgr2=0x7f8d801077d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:27:29.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.730+0000 7f8d86600640 1 -- 172.21.15.33:0/3974177244 shutdown_connections 2024-05-03T18:27:29.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.730+0000 7f8d86600640 1 -- 172.21.15.33:0/3974177244 wait complete. 2024-05-03T18:27:29.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.731+0000 7f8d86600640 1 Processor -- start 2024-05-03T18:27:29.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.731+0000 7f8d86600640 1 -- start start 2024-05-03T18:27:29.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.731+0000 7f8d86600640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d80108ef0 0x7f8d8019f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:29.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.732+0000 7f8d86600640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8d80109870 0x7f8d8019fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:29.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.732+0000 7f8d86600640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d8010ee80 0x7f8d801a3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:29.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.732+0000 7f8d86600640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8d80118f10 con 0x7f8d80109870 2024-05-03T18:27:29.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.732+0000 7f8d86600640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8d80118d90 con 0x7f8d8010ee80 2024-05-03T18:27:29.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.732+0000 7f8d86600640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8d80119090 con 0x7f8d80108ef0 2024-05-03T18:27:29.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.732+0000 7f8d7fe00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8d80109870 0x7f8d8019fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:29.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.732+0000 7f8d7fe00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8d80109870 0x7f8d8019fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40974/0 (socket says 172.21.15.33:40974) 2024-05-03T18:27:29.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.732+0000 7f8d7fe00640 1 -- 172.21.15.33:0/1387216926 learned_addr learned my addr 172.21.15.33:0/1387216926 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:27:29.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.732+0000 7f8d85200640 1 --2- 172.21.15.33:0/1387216926 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d80108ef0 0x7f8d8019f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:29.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.732+0000 7f8d85c00640 1 --2- 172.21.15.33:0/1387216926 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d8010ee80 0x7f8d801a3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:29.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.733+0000 7f8d7fe00640 1 -- 172.21.15.33:0/1387216926 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d80108ef0 msgr2=0x7f8d8019f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:29.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.733+0000 7f8d7fe00640 1 --2- 172.21.15.33:0/1387216926 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d80108ef0 0x7f8d8019f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:29.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.733+0000 7f8d7fe00640 1 -- 172.21.15.33:0/1387216926 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d8010ee80 msgr2=0x7f8d801a3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:29.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.733+0000 7f8d7fe00640 1 --2- 172.21.15.33:0/1387216926 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d8010ee80 0x7f8d801a3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:29.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.733+0000 7f8d7fe00640 1 -- 172.21.15.33:0/1387216926 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d801a44c0 con 0x7f8d80109870 2024-05-03T18:27:29.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.733+0000 7f8d7fe00640 1 --2- 172.21.15.33:0/1387216926 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8d80109870 0x7f8d8019fa10 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f8d7000eab0 tx=0x7f8d7000ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:27:29.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.733+0000 7f8d7d600640 1 -- 172.21.15.33:0/1387216926 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d7000ce40 con 0x7f8d80109870 2024-05-03T18:27:29.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.733+0000 7f8d7d600640 1 -- 172.21.15.33:0/1387216926 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8d70011900 con 0x7f8d80109870 2024-05-03T18:27:29.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.733+0000 7f8d7d600640 1 -- 172.21.15.33:0/1387216926 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d70010990 con 0x7f8d80109870 2024-05-03T18:27:29.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.733+0000 7f8d86600640 1 -- 172.21.15.33:0/1387216926 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d801a47b0 con 0x7f8d80109870 2024-05-03T18:27:29.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.734+0000 7f8d86600640 1 -- 172.21.15.33:0/1387216926 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d801ac040 con 0x7f8d80109870 2024-05-03T18:27:29.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.735+0000 7f8d86600640 1 -- 172.21.15.33:0/1387216926 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d80050f90 con 0x7f8d80109870 2024-05-03T18:27:29.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.737+0000 7f8d7d600640 1 -- 172.21.15.33:0/1387216926 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 23) ==== 149828+0+0 (secure 0 0 0) 0x7f8d70011470 con 0x7f8d80109870 2024-05-03T18:27:29.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.739+0000 7f8d7d600640 1 --2- 172.21.15.33:0/1387216926 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f8d640b29e0 0x7f8d640b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:29.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.739+0000 7f8d7d600640 1 -- 172.21.15.33:0/1387216926 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f8d700e3140 con 0x7f8d80109870 2024-05-03T18:27:29.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.739+0000 7f8d85200640 1 --2- 172.21.15.33:0/1387216926 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f8d640b29e0 0x7f8d640b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:29.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.740+0000 7f8d85200640 1 --2- 172.21.15.33:0/1387216926 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f8d640b29e0 0x7f8d640b4ea0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8d68005fd0 tx=0x7f8d6800b390 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:27:29.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.743+0000 7f8d7d600640 1 -- 172.21.15.33:0/1387216926 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f8d70016030 con 0x7f8d80109870 2024-05-03T18:27:29.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:29.987+0000 7f8d86600640 1 -- 172.21.15.33:0/1387216926 --> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f8d80062df0 con 0x7f8d640b29e0 2024-05-03T18:27:29.994 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-03T18:27:30.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:30.001+0000 7f8d7d600640 1 -- 172.21.15.33:0/1387216926 <== mgr.15120 v2:172.21.15.33:6800/3302011241 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f8d80062df0 con 0x7f8d640b29e0 2024-05-03T18:27:30.002 INFO:teuthology.orchestra.run.smithi033.stdout:/ 1m 6m3h 2024-05-03T18:27:30.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:30.003+0000 7f8d86600640 1 -- 172.21.15.33:0/1387216926 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f8d640b29e0 msgr2=0x7f8d640b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:30.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:30.003+0000 7f8d86600640 1 --2- 172.21.15.33:0/1387216926 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f8d640b29e0 0x7f8d640b4ea0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8d68005fd0 tx=0x7f8d6800b390 comp rx=0 tx=0).stop 2024-05-03T18:27:30.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:30.003+0000 7f8d86600640 1 -- 172.21.15.33:0/1387216926 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8d80109870 msgr2=0x7f8d8019fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:30.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:30.003+0000 7f8d86600640 1 --2- 172.21.15.33:0/1387216926 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8d80109870 0x7f8d8019fa10 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f8d7000eab0 tx=0x7f8d7000ef70 comp rx=0 tx=0).stop 2024-05-03T18:27:30.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:30.004+0000 7f8d86600640 1 -- 172.21.15.33:0/1387216926 shutdown_connections 2024-05-03T18:27:30.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:30.004+0000 7f8d86600640 1 --2- 172.21.15.33:0/1387216926 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f8d640b29e0 0x7f8d640b4ea0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:30.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:30.004+0000 7f8d86600640 1 --2- 172.21.15.33:0/1387216926 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d8010ee80 0x7f8d801a3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:30.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:30.004+0000 7f8d86600640 1 --2- 172.21.15.33:0/1387216926 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8d80109870 0x7f8d8019fa10 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:30.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:30.004+0000 7f8d86600640 1 --2- 172.21.15.33:0/1387216926 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d80108ef0 0x7f8d8019f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:30.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:30.004+0000 7f8d86600640 1 -- 172.21.15.33:0/1387216926 >> 172.21.15.33:0/1387216926 conn(0x7f8d80078790 msgr2=0x7f8d80079a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:27:30.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:30.004+0000 7f8d86600640 1 -- 172.21.15.33:0/1387216926 shutdown_connections 2024-05-03T18:27:30.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:30.004+0000 7f8d86600640 1 -- 172.21.15.33:0/1387216926 wait complete. 2024-05-03T18:27:30.022 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:29 smithi033 ceph-mon[28330]: from='client.15270 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:27:30.022 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:29 smithi033 ceph-mon[28330]: from='client.15276 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:27:30.022 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:29 smithi033 ceph-mon[28330]: scrub queued for path: / 2024-05-03T18:27:30.022 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:29 smithi033 ceph-mon[28330]: scrub summary: idle+waiting paths [/] 2024-05-03T18:27:30.022 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:29 smithi033 ceph-mon[28330]: scrub summary: active paths [/] 2024-05-03T18:27:30.023 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:29 smithi033 ceph-mon[28330]: scrub summary: idle+waiting paths [/] 2024-05-03T18:27:30.024 DEBUG:teuthology.orchestra.run.smithi033:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-05-03T18:27:30.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:29 smithi099 ceph-mon[31658]: from='client.15270 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:27:30.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:29 smithi099 ceph-mon[31658]: from='client.15276 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:27:30.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:29 smithi099 ceph-mon[31658]: scrub queued for path: / 2024-05-03T18:27:30.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:29 smithi099 ceph-mon[31658]: scrub summary: idle+waiting paths [/] 2024-05-03T18:27:30.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:29 smithi099 ceph-mon[31658]: scrub summary: active paths [/] 2024-05-03T18:27:30.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:29 smithi099 ceph-mon[31658]: scrub summary: idle+waiting paths [/] 2024-05-03T18:27:30.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:29 smithi150 ceph-mon[32492]: from='client.15270 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:27:30.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:29 smithi150 ceph-mon[32492]: from='client.15276 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:27:30.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:29 smithi150 ceph-mon[32492]: scrub queued for path: / 2024-05-03T18:27:30.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:29 smithi150 ceph-mon[32492]: scrub summary: idle+waiting paths [/] 2024-05-03T18:27:30.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:29 smithi150 ceph-mon[32492]: scrub summary: active paths [/] 2024-05-03T18:27:30.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:29 smithi150 ceph-mon[32492]: scrub summary: idle+waiting paths [/] 2024-05-03T18:27:30.144 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-05-03T18:27:30.158 INFO:teuthology.run_tasks:Running task exec... 2024-05-03T18:27:30.205 INFO:teuthology.task.exec:Executing custom commands... 2024-05-03T18:27:30.206 INFO:teuthology.task.exec:Running commands on role client.0 host ubuntu@smithi033.front.sepia.ceph.com 2024-05-03T18:27:30.206 DEBUG:teuthology.orchestra.run.smithi033:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/' 2024-05-03T18:27:30.323 INFO:teuthology.orchestra.run.smithi033.stdout:total 0 2024-05-03T18:27:30.351 INFO:teuthology.orchestra.run.smithi033.stdout:Filesystem Size Used Avail Use% Mounted on 2024-05-03T18:27:30.351 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-fuse 1.1T 364M 1.1T 1% /tmp/cephfs 2024-05-03T18:27:30.357 DEBUG:teuthology.orchestra.run.smithi033:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres' 2024-05-03T18:27:30.440 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:27:30.504 DEBUG:teuthology.orchestra.run.smithi033:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data' 2024-05-03T18:27:30.625 DEBUG:teuthology.orchestra.run.smithi033:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?"' 2024-05-03T18:27:30.835 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:30 smithi033 ceph-mon[28330]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:27:30.835 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:30 smithi033 ceph-mon[28330]: pgmap v12: 129 pgs: 129 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-03T18:27:31.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:30 smithi099 ceph-mon[31658]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:27:31.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:30 smithi099 ceph-mon[31658]: pgmap v12: 129 pgs: 129 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-03T18:27:31.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:30 smithi150 ceph-mon[32492]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:27:31.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:30 smithi150 ceph-mon[32492]: pgmap v12: 129 pgs: 129 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-03T18:27:31.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:31 smithi033 ceph-mon[28330]: from='client.15306 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:27:31.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:31 smithi033 ceph-mon[28330]: scrub summary: idle 2024-05-03T18:27:32.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:31 smithi099 ceph-mon[31658]: from='client.15306 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:27:32.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:31 smithi099 ceph-mon[31658]: scrub summary: idle 2024-05-03T18:27:32.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:31 smithi150 ceph-mon[32492]: from='client.15306 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:27:32.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:31 smithi150 ceph-mon[32492]: scrub summary: idle 2024-05-03T18:27:33.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:32 smithi099 ceph-mon[31658]: pgmap v13: 129 pgs: 129 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 255 B/s wr, 1 op/s 2024-05-03T18:27:33.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:32 smithi150 ceph-mon[32492]: pgmap v13: 129 pgs: 129 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 255 B/s wr, 1 op/s 2024-05-03T18:27:33.140 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:32 smithi033 ceph-mon[28330]: pgmap v13: 129 pgs: 129 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 255 B/s wr, 1 op/s 2024-05-03T18:27:33.228 DEBUG:teuthology.orchestra.run.smithi033:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local' 2024-05-03T18:27:33.363 INFO:teuthology.orchestra.run.smithi033.stdout:/tmp/cephfs/postgres(/.*)? system_u:object_r:postgresql_db_t:s0 2024-05-03T18:27:33.370 DEBUG:teuthology.orchestra.run.smithi033:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo restorecon -R -v /tmp/cephfs/postgres' 2024-05-03T18:27:33.488 DEBUG:teuthology.orchestra.run.smithi033:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-05-03T18:27:33.624 INFO:teuthology.orchestra.run.smithi033.stdout:/tmp/cephfs/postgres/: 2024-05-03T18:27:33.624 INFO:teuthology.orchestra.run.smithi033.stdout:total 2 2024-05-03T18:27:33.624 INFO:teuthology.orchestra.run.smithi033.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 . 2024-05-03T18:27:33.625 INFO:teuthology.orchestra.run.smithi033.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 .. 2024-05-03T18:27:33.625 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 data 2024-05-03T18:27:33.625 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:27:33.625 INFO:teuthology.orchestra.run.smithi033.stdout:/tmp/cephfs/postgres/data: 2024-05-03T18:27:33.625 INFO:teuthology.orchestra.run.smithi033.stdout:total 1 2024-05-03T18:27:33.625 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 . 2024-05-03T18:27:33.625 INFO:teuthology.orchestra.run.smithi033.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 .. 2024-05-03T18:27:33.631 DEBUG:teuthology.orchestra.run.smithi033:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '"'"'[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n'"'"' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf' 2024-05-03T18:27:33.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.726+0000 7f8fb0800700 1 -- 172.21.15.33:0/3483627021 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8fa806bb40 msgr2=0x7f8fa8110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:33.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.726+0000 7f8fb0800700 1 --2- 172.21.15.33:0/3483627021 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8fa806bb40 0x7f8fa8110330 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f8fa400b0a0 tx=0x7f8fa400b3b0 comp rx=0 tx=0).stop 2024-05-03T18:27:33.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.726+0000 7f8fb0800700 1 -- 172.21.15.33:0/3483627021 shutdown_connections 2024-05-03T18:27:33.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.726+0000 7f8fb0800700 1 --2- 172.21.15.33:0/3483627021 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8fa806bb40 0x7f8fa8110330 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:33.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.726+0000 7f8fb0800700 1 --2- 172.21.15.33:0/3483627021 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8fa8109510 0x7f8fa81098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:33.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.726+0000 7f8fb0800700 1 --2- 172.21.15.33:0/3483627021 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8fa806b180 0x7f8fa806b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:33.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.726+0000 7f8fb0800700 1 -- 172.21.15.33:0/3483627021 >> 172.21.15.33:0/3483627021 conn(0x7f8fa8079350 msgr2=0x7f8fa8079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:27:33.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.726+0000 7f8fb0800700 1 -- 172.21.15.33:0/3483627021 shutdown_connections 2024-05-03T18:27:33.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.726+0000 7f8fb0800700 1 -- 172.21.15.33:0/3483627021 wait complete. 2024-05-03T18:27:33.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.728+0000 7f8fb0800700 1 Processor -- start 2024-05-03T18:27:33.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.728+0000 7f8fb0800700 1 -- start start 2024-05-03T18:27:33.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.728+0000 7f8fb0800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8fa806b180 0x7f8fa819f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:33.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.728+0000 7f8fb0800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8fa806bb40 0x7f8fa819fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:33.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.728+0000 7f8fb0800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8fa8109510 0x7f8fa8199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:33.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.728+0000 7f8fb0800700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8fa81134f0 con 0x7f8fa8109510 2024-05-03T18:27:33.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.729+0000 7f8fb0800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8fa8113370 con 0x7f8fa806b180 2024-05-03T18:27:33.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.729+0000 7f8fb0800700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8fa8113670 con 0x7f8fa806bb40 2024-05-03T18:27:33.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.729+0000 7f8fae800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8fa8109510 0x7f8fa8199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:33.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.729+0000 7f8fae800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8fa8109510 0x7f8fa8199830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40984/0 (socket says 172.21.15.33:40984) 2024-05-03T18:27:33.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.729+0000 7f8fae800700 1 -- 172.21.15.33:0/345844194 learned_addr learned my addr 172.21.15.33:0/345844194 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:27:33.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.729+0000 7f8fade00700 1 --2- 172.21.15.33:0/345844194 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8fa806b180 0x7f8fa819f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:33.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.729+0000 7f8fad400700 1 --2- 172.21.15.33:0/345844194 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8fa806bb40 0x7f8fa819fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:33.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.729+0000 7f8fae800700 1 -- 172.21.15.33:0/345844194 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8fa806bb40 msgr2=0x7f8fa819fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:33.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.729+0000 7f8fae800700 1 --2- 172.21.15.33:0/345844194 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8fa806bb40 0x7f8fa819fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:33.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.729+0000 7f8fae800700 1 -- 172.21.15.33:0/345844194 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8fa806b180 msgr2=0x7f8fa819f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:33.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.729+0000 7f8fae800700 1 --2- 172.21.15.33:0/345844194 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8fa806b180 0x7f8fa819f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:33.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.729+0000 7f8fae800700 1 -- 172.21.15.33:0/345844194 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fa819a010 con 0x7f8fa8109510 2024-05-03T18:27:33.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.730+0000 7f8fae800700 1 --2- 172.21.15.33:0/345844194 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8fa8109510 0x7f8fa8199830 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f8fa400b830 tx=0x7f8fa4004db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:27:33.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.730+0000 7f8fa2a00700 1 -- 172.21.15.33:0/345844194 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa4022070 con 0x7f8fa8109510 2024-05-03T18:27:33.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.730+0000 7f8fa2a00700 1 -- 172.21.15.33:0/345844194 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8fa4007c80 con 0x7f8fa8109510 2024-05-03T18:27:33.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.730+0000 7f8fb0800700 1 -- 172.21.15.33:0/345844194 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fa819a2a0 con 0x7f8fa8109510 2024-05-03T18:27:33.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.730+0000 7f8fa2a00700 1 -- 172.21.15.33:0/345844194 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa4005540 con 0x7f8fa8109510 2024-05-03T18:27:33.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.731+0000 7f8fb0800700 1 -- 172.21.15.33:0/345844194 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fa81023a0 con 0x7f8fa8109510 2024-05-03T18:27:33.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.736+0000 7f8fa2a00700 1 -- 172.21.15.33:0/345844194 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 23) ==== 149828+0+0 (secure 0 0 0) 0x7f8fa402d020 con 0x7f8fa8109510 2024-05-03T18:27:33.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.737+0000 7f8fbb211b80 1 Processor -- start 2024-05-03T18:27:33.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.737+0000 7f8fbb211b80 1 -- start start 2024-05-03T18:27:33.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.738+0000 7f8fbb211b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e785da7b0 0x557e785dab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:33.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.738+0000 7f8fbb211b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e785db0d0 0x557e785d5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:33.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.738+0000 7f8fbb211b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e785d65d0 0x557e785d6a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:33.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.738+0000 7f8fbb211b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x557e78528a20 con 0x557e785da7b0 2024-05-03T18:27:33.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.738+0000 7f8fbb211b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x557e784c6760 con 0x557e785d65d0 2024-05-03T18:27:33.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.738+0000 7f8fbb211b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x557e785249e0 con 0x557e785db0d0 2024-05-03T18:27:33.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.738+0000 7f8fa2a00700 1 --2- 172.21.15.33:0/345844194 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f8f900b2a80 0x7f8f900b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:33.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.738+0000 7f8fad400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e785da7b0 0x557e785dab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:33.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.738+0000 7f8fad400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e785da7b0 0x557e785dab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40986/0 (socket says 172.21.15.33:40986) 2024-05-03T18:27:33.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.738+0000 7f8fad400700 1 -- 172.21.15.33:0/2464317519 learned_addr learned my addr 172.21.15.33:0/2464317519 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:27:33.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.738+0000 7f8fade00700 1 --2- 172.21.15.33:0/2464317519 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e785db0d0 0x557e785d5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:33.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.738+0000 7f8fae800700 1 --2- 172.21.15.33:0/345844194 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f8f900b2a80 0x7f8f900b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:33.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.738+0000 7f8fad400700 1 --2- 172.21.15.33:0/2464317519 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e785d65d0 0x557e785d6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:33.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.738+0000 7f8fa2a00700 1 -- 172.21.15.33:0/345844194 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f8fa400f020 con 0x7f8fa8109510 2024-05-03T18:27:33.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.739+0000 7f8fad400700 1 -- 172.21.15.33:0/2464317519 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e785db0d0 msgr2=0x557e785d5fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:27:33.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.739+0000 7f8fad400700 1 --2- 172.21.15.33:0/2464317519 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e785db0d0 0x557e785d5fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:33.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.739+0000 7f8fad400700 1 -- 172.21.15.33:0/2464317519 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e785d65d0 msgr2=0x557e785d6a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:27:33.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.739+0000 7f8fad400700 1 --2- 172.21.15.33:0/2464317519 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e785d65d0 0x557e785d6a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:33.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.739+0000 7f8fad400700 1 -- 172.21.15.33:0/2464317519 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e784758d0 con 0x557e785da7b0 2024-05-03T18:27:33.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.739+0000 7f8fade00700 1 --2- 172.21.15.33:0/2464317519 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e785db0d0 0x557e785d5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:27:33.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.739+0000 7f8fad400700 1 --2- 172.21.15.33:0/2464317519 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e785da7b0 0x557e785dab90 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f8f98008fa0 tx=0x7f8f98008620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a465bf639672c5d server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:27:33.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.739+0000 7f8fae800700 1 --2- 172.21.15.33:0/345844194 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f8f900b2a80 0x7f8f900b4f40 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8fa40e4bd0 tx=0x7f8fa4004f90 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:27:33.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.740+0000 7f8f97400700 1 -- 172.21.15.33:0/2464317519 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f98018030 con 0x557e785da7b0 2024-05-03T18:27:33.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.740+0000 7f8f97400700 1 -- 172.21.15.33:0/2464317519 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8f98004cc0 con 0x557e785da7b0 2024-05-03T18:27:33.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.740+0000 7f8f97400700 1 -- 172.21.15.33:0/2464317519 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f9800f450 con 0x557e785da7b0 2024-05-03T18:27:33.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.740+0000 7f8fbb211b80 1 -- 172.21.15.33:0/2464317519 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e785da7b0 msgr2=0x557e785dab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:33.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.740+0000 7f8fbb211b80 1 --2- 172.21.15.33:0/2464317519 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e785da7b0 0x557e785dab90 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f8f98008fa0 tx=0x7f8f98008620 comp rx=0 tx=0).stop 2024-05-03T18:27:33.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.741+0000 7f8fbb211b80 1 -- 172.21.15.33:0/2464317519 shutdown_connections 2024-05-03T18:27:33.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.741+0000 7f8fbb211b80 1 --2- 172.21.15.33:0/2464317519 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e785d65d0 0x557e785d6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:33.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.741+0000 7f8fbb211b80 1 --2- 172.21.15.33:0/2464317519 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e785da7b0 0x557e785dab90 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:33.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.741+0000 7f8fbb211b80 1 --2- 172.21.15.33:0/2464317519 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e785db0d0 0x557e785d5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:33.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.741+0000 7f8fbb211b80 1 -- 172.21.15.33:0/2464317519 >> 172.21.15.33:0/2464317519 conn(0x557e78504b60 msgr2=0x557e785d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:27:33.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.741+0000 7f8fbb211b80 1 -- 172.21.15.33:0/2464317519 shutdown_connections 2024-05-03T18:27:33.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.741+0000 7f8fbb211b80 1 -- 172.21.15.33:0/2464317519 wait complete. 2024-05-03T18:27:33.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.742+0000 7f8fbb211b80 1 -- start start 2024-05-03T18:27:33.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.742+0000 7f8fbb211b80 1 Processor -- start 2024-05-03T18:27:33.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.742+0000 7f8fbb211b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e785d65d0 0x557e785e1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:33.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.742+0000 7f8fbb211b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e785da7b0 0x557e785df3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:33.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.742+0000 7f8fbb211b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e785db0d0 0x557e785df900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:33.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.742+0000 7f8fbb211b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x557e785e2ca0 con 0x557e785da7b0 2024-05-03T18:27:33.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.742+0000 7f8fbb211b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x557e7848eb20 con 0x557e785d65d0 2024-05-03T18:27:33.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.742+0000 7f8fbb211b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x557e785e2e20 con 0x557e785db0d0 2024-05-03T18:27:33.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.743+0000 7f8fbb211b80 10 client.0 _ll_register_callbacks cb 0x557e784cfc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:27:33.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.743+0000 7f8fad400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e785d65d0 0x557e785e1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:33.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.743+0000 7f8fade00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e785da7b0 0x557e785df3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:33.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.743+0000 7f8fade00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e785da7b0 0x557e785df3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40990/0 (socket says 172.21.15.33:40990) 2024-05-03T18:27:33.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.743+0000 7f8fade00700 1 -- 172.21.15.33:0/2960964845 learned_addr learned my addr 172.21.15.33:0/2960964845 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:27:33.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.743+0000 7f8fad400700 1 --2- 172.21.15.33:0/2960964845 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e785db0d0 0x557e785df900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:33.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.744+0000 7f8fade00700 1 -- 172.21.15.33:0/2960964845 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e785db0d0 msgr2=0x557e785df900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:27:33.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.744+0000 7f8fade00700 1 --2- 172.21.15.33:0/2960964845 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e785db0d0 0x557e785df900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:33.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.744+0000 7f8fade00700 1 -- 172.21.15.33:0/2960964845 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e785d65d0 msgr2=0x557e785e1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:27:33.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.744+0000 7f8fade00700 1 --2- 172.21.15.33:0/2960964845 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e785d65d0 0x557e785e1fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:33.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.744+0000 7f8fade00700 1 -- 172.21.15.33:0/2960964845 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e784ee420 con 0x557e785da7b0 2024-05-03T18:27:33.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.744+0000 7f8fade00700 1 --2- 172.21.15.33:0/2960964845 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e785da7b0 0x557e785df3c0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c010670 tx=0x7f8f9c010980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53bd1300f5625dc8 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:27:33.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.744+0000 7f8f8be00700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:27:33.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.745+0000 7f8f8be00700 1 -- 172.21.15.33:0/2960964845 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f9c012860 con 0x557e785da7b0 2024-05-03T18:27:33.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.745+0000 7f8f8be00700 1 -- 172.21.15.33:0/2960964845 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f8f9c012a00 con 0x557e785da7b0 2024-05-03T18:27:33.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.745+0000 7f8fbb211b80 1 -- 172.21.15.33:0/2960964845 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f98004cc0 con 0x557e785da7b0 2024-05-03T18:27:33.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.745+0000 7f8f8be00700 1 -- 172.21.15.33:0/2960964845 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f8f9c012c00 con 0x557e785da7b0 2024-05-03T18:27:33.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.745+0000 7f8fbb211b80 10 client.15318 fetch_fsmap learned FSMap version 21 2024-05-03T18:27:33.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.745+0000 7f8fbb211b80 1 -- 172.21.15.33:0/2960964845 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x557e785ed570 con 0x557e785da7b0 2024-05-03T18:27:33.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.745+0000 7f8f8be00700 1 -- 172.21.15.33:0/2960964845 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8f98004cc0 con 0x557e785da7b0 2024-05-03T18:27:33.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.746+0000 7f8f8be00700 1 -- 172.21.15.33:0/2960964845 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f8f9c00f040 con 0x557e785da7b0 2024-05-03T18:27:33.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.746+0000 7f8fbb211b80 10 client.15318 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:27:33.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.746+0000 7f8fbb211b80 10 client.15318 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:27:33.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.746+0000 7f8fbb211b80 20 client.15318 populate_metadata read hostname 'smithi033' 2024-05-03T18:27:33.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.746+0000 7f8fbb211b80 1 --2- 172.21.15.33:0/2960964845 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x557e785efbb0 0x557e785f1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:33.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.746+0000 7f8fbb211b80 4 client.15318 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:27:33.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.746+0000 7f8fbb211b80 1 -- 172.21.15.33:0/2960964845 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557e785a31d0 con 0x557e785efbb0 2024-05-03T18:27:33.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.746+0000 7f8fad400700 1 --2- 172.21.15.33:0/2960964845 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x557e785efbb0 0x557e785f1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:33.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.747+0000 7f8fad400700 1 --2- 172.21.15.33:0/2960964845 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x557e785efbb0 0x557e785f1fa0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80fed986b2fd043d server_cookie=d3f250f3437b6ac8 in_seq=0 out_seq=0 2024-05-03T18:27:33.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.747+0000 7f8f8be00700 10 client.15318 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:27:33.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.749+0000 7f8f8be00700 1 -- 172.21.15.33:0/2960964845 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557e785a31d0 con 0x557e785efbb0 2024-05-03T18:27:33.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.749+0000 7f8f8be00700 10 client.15318 handle_command_reply: tid=0 2024-05-03T18:27:33.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.749+0000 7f8fbb211b80 1 client.15318 shutdown 2024-05-03T18:27:33.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.750+0000 7f8fbb211b80 1 -- 172.21.15.33:0/2960964845 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e785da7b0 msgr2=0x557e785df3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:33.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.750+0000 7f8fbb211b80 1 --2- 172.21.15.33:0/2960964845 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e785da7b0 0x557e785df3c0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c010670 tx=0x7f8f9c010980 comp rx=0 tx=0).stop 2024-05-03T18:27:33.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.750+0000 7f8fbb211b80 1 -- 172.21.15.33:0/2960964845 shutdown_connections 2024-05-03T18:27:33.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.750+0000 7f8fbb211b80 1 --2- 172.21.15.33:0/2960964845 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x557e785efbb0 0x557e785f1fa0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:33.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.750+0000 7f8fbb211b80 1 --2- 172.21.15.33:0/2960964845 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e785db0d0 0x557e785df900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:33.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.750+0000 7f8fbb211b80 1 --2- 172.21.15.33:0/2960964845 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e785d65d0 0x557e785e1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:33.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.750+0000 7f8fbb211b80 1 --2- 172.21.15.33:0/2960964845 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e785da7b0 0x557e785df3c0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:33.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.750+0000 7f8f8be00700 0 client.15318 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:27:33.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.750+0000 7f8fbb211b80 1 -- 172.21.15.33:0/2960964845 >> 172.21.15.33:0/2960964845 conn(0x557e78504b60 msgr2=0x557e785cf650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:27:33.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.750+0000 7f8fbb211b80 1 -- 172.21.15.33:0/2960964845 shutdown_connections 2024-05-03T18:27:33.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.750+0000 7f8fbb211b80 1 -- 172.21.15.33:0/2960964845 wait complete. 2024-05-03T18:27:33.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.750+0000 7f8fbb211b80 20 client.15318 trim_cache size 0 max 16384 2024-05-03T18:27:33.768 INFO:teuthology.orchestra.run.smithi033.stdout:[Service] 2024-05-03T18:27:33.768 INFO:teuthology.orchestra.run.smithi033.stdout:Environment=PGDATA=/tmp/cephfs/postgres/data 2024-05-03T18:27:33.768 INFO:teuthology.orchestra.run.smithi033.stdout:Environment=PGLOG=/tmp/cephfs/postgres/pgstartup.log 2024-05-03T18:27:33.774 DEBUG:teuthology.orchestra.run.smithi033:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- postgresql-setup --initdb' 2024-05-03T18:27:33.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.775+0000 7f8fbb211b80 1 Processor -- start 2024-05-03T18:27:33.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.775+0000 7f8fbb211b80 1 -- start start 2024-05-03T18:27:33.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.775+0000 7f8fbb211b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e785d65d0 0x557e785dfa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:33.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.775+0000 7f8fbb211b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e785da7b0 0x557e785ebd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:33.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.776+0000 7f8fbb211b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e785db0d0 0x557e785d0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:33.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.776+0000 7f8fbb211b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8fa81aaff0 con 0x557e785db0d0 2024-05-03T18:27:33.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.776+0000 7f8fbb211b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8fa81ad870 con 0x557e785d65d0 2024-05-03T18:27:33.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.776+0000 7f8fbb211b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8fa81ab660 con 0x557e785da7b0 2024-05-03T18:27:33.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.776+0000 7f8fad400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e785db0d0 0x557e785d0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:33.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.776+0000 7f8fad400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e785db0d0 0x557e785d0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41002/0 (socket says 172.21.15.33:41002) 2024-05-03T18:27:33.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.776+0000 7f8fad400700 1 -- 172.21.15.33:0/1341546930 learned_addr learned my addr 172.21.15.33:0/1341546930 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:27:33.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.776+0000 7f8fade00700 1 --2- 172.21.15.33:0/1341546930 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e785da7b0 0x557e785ebd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:33.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.776+0000 7f8fad400700 1 --2- 172.21.15.33:0/1341546930 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e785d65d0 0x557e785dfa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:33.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.776+0000 7f8fad400700 1 -- 172.21.15.33:0/1341546930 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e785da7b0 msgr2=0x557e785ebd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:27:33.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.776+0000 7f8fad400700 1 --2- 172.21.15.33:0/1341546930 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e785da7b0 0x557e785ebd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:33.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.776+0000 7f8fad400700 1 -- 172.21.15.33:0/1341546930 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e785d65d0 msgr2=0x557e785dfa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:27:33.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.776+0000 7f8fad400700 1 --2- 172.21.15.33:0/1341546930 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e785d65d0 0x557e785dfa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:33.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.776+0000 7f8fad400700 1 -- 172.21.15.33:0/1341546930 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e785e1c10 con 0x557e785db0d0 2024-05-03T18:27:33.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.777+0000 7f8fad400700 1 --2- 172.21.15.33:0/1341546930 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e785db0d0 0x557e785d0660 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f8f98008fa0 tx=0x7f8f98008ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57c2c53d093c9b40 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:27:33.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.777+0000 7f8f96a00700 1 -- 172.21.15.33:0/1341546930 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f98004ad0 con 0x557e785db0d0 2024-05-03T18:27:33.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.777+0000 7f8f96a00700 1 -- 172.21.15.33:0/1341546930 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8f98004c70 con 0x557e785db0d0 2024-05-03T18:27:33.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.777+0000 7f8f96a00700 1 -- 172.21.15.33:0/1341546930 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f9802d850 con 0x557e785db0d0 2024-05-03T18:27:33.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.778+0000 7f8fbb211b80 1 -- 172.21.15.33:0/1341546930 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e785db0d0 msgr2=0x557e785d0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:33.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.778+0000 7f8fbb211b80 1 --2- 172.21.15.33:0/1341546930 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e785db0d0 0x557e785d0660 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f8f98008fa0 tx=0x7f8f98008ee0 comp rx=0 tx=0).stop 2024-05-03T18:27:33.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.778+0000 7f8fbb211b80 1 -- 172.21.15.33:0/1341546930 shutdown_connections 2024-05-03T18:27:33.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.778+0000 7f8fbb211b80 1 --2- 172.21.15.33:0/1341546930 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e785db0d0 0x557e785d0660 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:33.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.778+0000 7f8fbb211b80 1 --2- 172.21.15.33:0/1341546930 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e785d65d0 0x557e785dfa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:33.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.778+0000 7f8fbb211b80 1 --2- 172.21.15.33:0/1341546930 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e785da7b0 0x557e785ebd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:33.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.778+0000 7f8fbb211b80 1 -- 172.21.15.33:0/1341546930 >> 172.21.15.33:0/1341546930 conn(0x557e78504b60 msgr2=0x557e785df380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:27:33.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.778+0000 7f8fbb211b80 1 -- 172.21.15.33:0/1341546930 shutdown_connections 2024-05-03T18:27:33.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.778+0000 7f8fbb211b80 1 -- 172.21.15.33:0/1341546930 wait complete. 2024-05-03T18:27:33.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.779+0000 7f8fbb211b80 1 -- start start 2024-05-03T18:27:33.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.779+0000 7f8fbb211b80 1 Processor -- start 2024-05-03T18:27:33.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.780+0000 7f8fbb211b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e785d65d0 0x557e785e8550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:33.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.780+0000 7f8fbb211b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e785da7b0 0x557e785e38d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:33.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.780+0000 7f8fbb211b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e785db0d0 0x557e785e3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:33.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.780+0000 7f8fbb211b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8fa81abd30 con 0x557e785db0d0 2024-05-03T18:27:33.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.780+0000 7f8fbb211b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8fa81ad1a0 con 0x557e785da7b0 2024-05-03T18:27:33.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.780+0000 7f8fbb211b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8fa81ac400 con 0x557e785d65d0 2024-05-03T18:27:33.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.780+0000 7f8fbb211b80 10 client.0 _ll_register_callbacks cb 0x557e784cfc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:27:33.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.780+0000 7f8fad400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e785db0d0 0x557e785e3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:33.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.780+0000 7f8fad400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e785db0d0 0x557e785e3e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41016/0 (socket says 172.21.15.33:41016) 2024-05-03T18:27:33.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.780+0000 7f8fad400700 1 -- 172.21.15.33:0/3659168634 learned_addr learned my addr 172.21.15.33:0/3659168634 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:27:33.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.780+0000 7f8fade00700 1 --2- 172.21.15.33:0/3659168634 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e785da7b0 0x557e785e38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:33.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.780+0000 7f8fad400700 1 --2- 172.21.15.33:0/3659168634 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e785d65d0 0x557e785e8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:33.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.781+0000 7f8fad400700 1 -- 172.21.15.33:0/3659168634 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e785d65d0 msgr2=0x557e785e8550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:27:33.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.781+0000 7f8fad400700 1 --2- 172.21.15.33:0/3659168634 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e785d65d0 0x557e785e8550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:33.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.781+0000 7f8fad400700 1 -- 172.21.15.33:0/3659168634 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e785da7b0 msgr2=0x557e785e38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:27:33.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.781+0000 7f8fad400700 1 --2- 172.21.15.33:0/3659168634 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e785da7b0 0x557e785e38d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:33.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.781+0000 7f8fad400700 1 -- 172.21.15.33:0/3659168634 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e785e4650 con 0x557e785db0d0 2024-05-03T18:27:33.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.781+0000 7f8fad400700 1 --2- 172.21.15.33:0/3659168634 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e785db0d0 0x557e785e3e10 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f8f98008f70 tx=0x7f8f98014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c31dd15128be4d0b server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:27:33.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.781+0000 7f8f8be00700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:27:33.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.781+0000 7f8f8be00700 1 -- 172.21.15.33:0/3659168634 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f98033040 con 0x557e785db0d0 2024-05-03T18:27:33.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.781+0000 7f8fbb211b80 1 -- 172.21.15.33:0/3659168634 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f98004c70 con 0x557e785db0d0 2024-05-03T18:27:33.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.781+0000 7f8f8be00700 1 -- 172.21.15.33:0/3659168634 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f8f98005470 con 0x557e785db0d0 2024-05-03T18:27:33.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.782+0000 7f8f8be00700 1 -- 172.21.15.33:0/3659168634 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f8f98005670 con 0x557e785db0d0 2024-05-03T18:27:33.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.782+0000 7f8fbb211b80 10 client.15324 fetch_fsmap learned FSMap version 21 2024-05-03T18:27:33.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.782+0000 7f8fbb211b80 1 -- 172.21.15.33:0/3659168634 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x557e7865a3c0 con 0x557e785db0d0 2024-05-03T18:27:33.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.782+0000 7f8f8be00700 1 -- 172.21.15.33:0/3659168634 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8f98004c70 con 0x557e785db0d0 2024-05-03T18:27:33.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.782+0000 7f8f8be00700 1 -- 172.21.15.33:0/3659168634 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f8f98039060 con 0x557e785db0d0 2024-05-03T18:27:33.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.782+0000 7f8fbb211b80 10 client.15324 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:27:33.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.782+0000 7f8fbb211b80 10 client.15324 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:27:33.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.783+0000 7f8fbb211b80 20 client.15324 populate_metadata read hostname 'smithi033' 2024-05-03T18:27:33.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.783+0000 7f8fbb211b80 1 --2- 172.21.15.33:0/3659168634 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x557e7865a800 0x557e7865cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:33.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.783+0000 7f8fbb211b80 4 client.15324 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-03T18:27:33.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.783+0000 7f8fbb211b80 1 -- 172.21.15.33:0/3659168634 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557e78536400 con 0x557e7865a800 2024-05-03T18:27:33.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.783+0000 7f8fade00700 1 --2- 172.21.15.33:0/3659168634 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x557e7865a800 0x557e7865cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:33.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.784+0000 7f8fade00700 1 --2- 172.21.15.33:0/3659168634 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x557e7865a800 0x557e7865cc20 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64f407e60e9668b server_cookie=2896f8037c8e2872 in_seq=0 out_seq=0 2024-05-03T18:27:33.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.784+0000 7f8f8be00700 10 client.15324 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:27:33.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.785+0000 7f8f8be00700 1 -- 172.21.15.33:0/3659168634 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x557e78536400 con 0x557e7865a800 2024-05-03T18:27:33.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.785+0000 7f8f8be00700 10 client.15324 handle_command_reply: tid=0 2024-05-03T18:27:33.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.785+0000 7f8fbb211b80 1 client.15324 shutdown 2024-05-03T18:27:33.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.785+0000 7f8fbb211b80 1 -- 172.21.15.33:0/3659168634 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e785db0d0 msgr2=0x557e785e3e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:33.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.785+0000 7f8fbb211b80 1 --2- 172.21.15.33:0/3659168634 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e785db0d0 0x557e785e3e10 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f8f98008f70 tx=0x7f8f98014040 comp rx=0 tx=0).stop 2024-05-03T18:27:33.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.785+0000 7f8fbb211b80 1 -- 172.21.15.33:0/3659168634 shutdown_connections 2024-05-03T18:27:33.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.785+0000 7f8fbb211b80 1 --2- 172.21.15.33:0/3659168634 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x557e7865a800 0x557e7865cc20 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:33.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.785+0000 7f8fbb211b80 1 --2- 172.21.15.33:0/3659168634 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e785db0d0 0x557e785e3e10 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:33.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.785+0000 7f8fbb211b80 1 --2- 172.21.15.33:0/3659168634 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e785d65d0 0x557e785e8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:33.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.785+0000 7f8fbb211b80 1 --2- 172.21.15.33:0/3659168634 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e785da7b0 0x557e785e38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:33.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.785+0000 7f8f8be00700 0 client.15324 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:27:33.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.785+0000 7f8fbb211b80 1 -- 172.21.15.33:0/3659168634 >> 172.21.15.33:0/3659168634 conn(0x557e78504b60 msgr2=0x557e785e0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:27:33.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.785+0000 7f8fbb211b80 1 -- 172.21.15.33:0/3659168634 shutdown_connections 2024-05-03T18:27:33.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.785+0000 7f8fbb211b80 1 -- 172.21.15.33:0/3659168634 wait complete. 2024-05-03T18:27:33.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.786+0000 7f8fbb211b80 20 client.15324 trim_cache size 0 max 16384 2024-05-03T18:27:33.789 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-03T18:27:33.789 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-05-03T18:27:33.789 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-05-03T18:27:33.789 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-03T18:27:33.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.786+0000 7f8fb0800700 1 -- 172.21.15.33:0/345844194 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f8f900b2a80 msgr2=0x7f8f900b4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:33.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.786+0000 7f8fb0800700 1 --2- 172.21.15.33:0/345844194 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f8f900b2a80 0x7f8f900b4f40 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8fa40e4bd0 tx=0x7f8fa4004f90 comp rx=0 tx=0).stop 2024-05-03T18:27:33.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.786+0000 7f8fb0800700 1 -- 172.21.15.33:0/345844194 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8fa8109510 msgr2=0x7f8fa8199830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:33.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.786+0000 7f8fb0800700 1 --2- 172.21.15.33:0/345844194 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8fa8109510 0x7f8fa8199830 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f8fa400b830 tx=0x7f8fa4004db0 comp rx=0 tx=0).stop 2024-05-03T18:27:33.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.786+0000 7f8fb0800700 1 -- 172.21.15.33:0/345844194 shutdown_connections 2024-05-03T18:27:33.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.786+0000 7f8fb0800700 1 --2- 172.21.15.33:0/345844194 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8fa8109510 0x7f8fa8199830 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:33.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.786+0000 7f8fb0800700 1 --2- 172.21.15.33:0/345844194 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f8f900b2a80 0x7f8f900b4f40 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:33.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.786+0000 7f8fb0800700 1 --2- 172.21.15.33:0/345844194 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8fa806b180 0x7f8fa819f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:33.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.786+0000 7f8fb0800700 1 --2- 172.21.15.33:0/345844194 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8fa806bb40 0x7f8fa819fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:33.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.786+0000 7f8fb0800700 1 -- 172.21.15.33:0/345844194 >> 172.21.15.33:0/345844194 conn(0x7f8fa8079350 msgr2=0x7f8fa81010e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:27:33.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.787+0000 7f8fb0800700 1 -- 172.21.15.33:0/345844194 shutdown_connections 2024-05-03T18:27:33.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:33.787+0000 7f8fb0800700 1 -- 172.21.15.33:0/345844194 wait complete. 2024-05-03T18:27:34.005 INFO:teuthology.orchestra.run.smithi033.stderr: * Initializing database in '/tmp/cephfs/postgres/data' 2024-05-03T18:27:34.794 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-03T18:27:34.794 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T18:27:34.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:34 smithi033 ceph-mon[28330]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 78 KiB/s wr, 35 op/s 2024-05-03T18:27:35.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:34 smithi150 ceph-mon[32492]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 78 KiB/s wr, 35 op/s 2024-05-03T18:27:35.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:34 smithi099 ceph-mon[31658]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 78 KiB/s wr, 35 op/s 2024-05-03T18:27:35.313 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:27:37.088 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:36 smithi033 ceph-mon[28330]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 77 KiB/s wr, 34 op/s 2024-05-03T18:27:37.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:36 smithi150 ceph-mon[32492]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 77 KiB/s wr, 34 op/s 2024-05-03T18:27:37.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:36 smithi099 ceph-mon[31658]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 77 KiB/s wr, 34 op/s 2024-05-03T18:27:37.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:37.682+0000 7f2409200700 1 -- 172.21.15.33:0/3178980462 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2404108850 msgr2=0x7f24041103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:37.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:37.682+0000 7f2409200700 1 --2- 172.21.15.33:0/3178980462 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2404108850 0x7f24041103d0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f23fc00b0d0 tx=0x7f23fc00b3e0 comp rx=0 tx=0).stop 2024-05-03T18:27:37.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:37.682+0000 7f2409200700 1 -- 172.21.15.33:0/3178980462 shutdown_connections 2024-05-03T18:27:37.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:37.682+0000 7f2409200700 1 --2- 172.21.15.33:0/3178980462 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2404108850 0x7f24041103d0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:37.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:37.682+0000 7f2409200700 1 --2- 172.21.15.33:0/3178980462 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2404103750 0x7f2404103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:37.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:37.682+0000 7f2409200700 1 --2- 172.21.15.33:0/3178980462 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2404104100 0x7f2404108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:37.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:37.682+0000 7f2409200700 1 -- 172.21.15.33:0/3178980462 >> 172.21.15.33:0/3178980462 conn(0x7f240407a600 msgr2=0x7f240407aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:27:37.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:37.683+0000 7f2409200700 1 -- 172.21.15.33:0/3178980462 shutdown_connections 2024-05-03T18:27:37.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:37.683+0000 7f2409200700 1 -- 172.21.15.33:0/3178980462 wait complete. 2024-05-03T18:27:37.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:37.684+0000 7f2409200700 1 Processor -- start 2024-05-03T18:27:37.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:37.684+0000 7f2409200700 1 -- start start 2024-05-03T18:27:37.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:37.684+0000 7f2409200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2404103750 0x7f24041a17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:37.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:37.685+0000 7f2409200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2404104100 0x7f24041a1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:37.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:37.685+0000 7f2409200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2404108850 0x7f240419b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:37.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:37.685+0000 7f2409200700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f24041134d0 con 0x7f2404108850 2024-05-03T18:27:37.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:37.685+0000 7f2409200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2404113350 con 0x7f2404103750 2024-05-03T18:27:37.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:37.685+0000 7f2409200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2404113650 con 0x7f2404104100 2024-05-03T18:27:37.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:37.685+0000 7f2401e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2404103750 0x7f24041a17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:37.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:37.685+0000 7f2402800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2404108850 0x7f240419b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:37.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:37.685+0000 7f2401e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2404103750 0x7f24041a17b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:49250/0 (socket says 172.21.15.33:49250) 2024-05-03T18:27:37.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:37.685+0000 7f2401e00700 1 -- 172.21.15.33:0/146587389 learned_addr learned my addr 172.21.15.33:0/146587389 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:27:37.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:37.685+0000 7f2401400700 1 --2- 172.21.15.33:0/146587389 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2404104100 0x7f24041a1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:37.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:37.686+0000 7f2402800700 1 -- 172.21.15.33:0/146587389 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2404104100 msgr2=0x7f24041a1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:37.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:37.686+0000 7f2402800700 1 --2- 172.21.15.33:0/146587389 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2404104100 0x7f24041a1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:37.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:37.686+0000 7f2402800700 1 -- 172.21.15.33:0/146587389 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2404103750 msgr2=0x7f24041a17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:37.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:37.686+0000 7f2402800700 1 --2- 172.21.15.33:0/146587389 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2404103750 0x7f24041a17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:37.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:37.686+0000 7f2402800700 1 -- 172.21.15.33:0/146587389 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f240419c0d0 con 0x7f2404108850 2024-05-03T18:27:37.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:37.686+0000 7f2402800700 1 --2- 172.21.15.33:0/146587389 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2404108850 0x7f240419b970 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f23fc00b0d0 tx=0x7f23fc007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:27:37.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:37.686+0000 7f23faa00700 1 -- 172.21.15.33:0/146587389 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23fc01f070 con 0x7f2404108850 2024-05-03T18:27:37.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:37.686+0000 7f23faa00700 1 -- 172.21.15.33:0/146587389 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f23fc005500 con 0x7f2404108850 2024-05-03T18:27:37.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:37.686+0000 7f2409200700 1 -- 172.21.15.33:0/146587389 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f240419c360 con 0x7f2404108850 2024-05-03T18:27:37.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:37.687+0000 7f23faa00700 1 -- 172.21.15.33:0/146587389 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23fc01b690 con 0x7f2404108850 2024-05-03T18:27:37.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:37.687+0000 7f2409200700 1 -- 172.21.15.33:0/146587389 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24041a8680 con 0x7f2404108850 2024-05-03T18:27:37.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:37.688+0000 7f2409200700 1 -- 172.21.15.33:0/146587389 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f240406a190 con 0x7f2404108850 2024-05-03T18:27:37.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:37.692+0000 7f23faa00700 1 -- 172.21.15.33:0/146587389 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 23) ==== 149828+0+0 (secure 0 0 0) 0x7f23fc022070 con 0x7f2404108850 2024-05-03T18:27:37.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:37.693+0000 7f23faa00700 1 --2- 172.21.15.33:0/146587389 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f23e80b2b00 0x7f23e80b4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:37.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:37.693+0000 7f23faa00700 1 -- 172.21.15.33:0/146587389 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f23fc00f020 con 0x7f2404108850 2024-05-03T18:27:37.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:37.694+0000 7f2401e00700 1 --2- 172.21.15.33:0/146587389 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f23e80b2b00 0x7f23e80b4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:37.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:37.694+0000 7f2401e00700 1 --2- 172.21.15.33:0/146587389 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f23e80b2b00 0x7f23e80b4fc0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f240419cab0 tx=0x7f23f400b500 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:27:37.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:37.696+0000 7f23faa00700 1 -- 172.21.15.33:0/146587389 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f23fc029020 con 0x7f2404108850 2024-05-03T18:27:38.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:38.026+0000 7f2409200700 1 -- 172.21.15.33:0/146587389 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f24041a8850 con 0x7f2404108850 2024-05-03T18:27:38.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:38.028+0000 7f23faa00700 1 -- 172.21.15.33:0/146587389 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11801 (secure 0 0 0) 0x7f23fc0ab830 con 0x7f2404108850 2024-05-03T18:27:38.029 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:27:38.030 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1327393503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1327393503},{"type":"v1","addr":"172.21.15.33:6835","nonce":1327393503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/2461472948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":2461472948},{"type":"v1","addr":"172.21.15.33:6837","nonce":2461472948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14559,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/442269281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":442269281},{"type":"v1","addr":"172.21.15.33:6839","nonce":442269281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14571,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/645571756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":645571756},{"type":"v1","addr":"172.21.15.33:6841","nonce":645571756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/203598909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":203598909},{"type":"v1","addr":"172.21.15.150:6833","nonce":203598909}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/4029842493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":4029842493},{"type":"v1","addr":"172.21.15.150:6835","nonce":4029842493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3463664423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3463664423},{"type":"v1","addr":"172.21.15.150:6837","nonce":3463664423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24481,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6839/4051648018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":4051648018},{"type":"v1","addr":"172.21.15.150:6839","nonce":4051648018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34418,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1865105479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1865105479},{"type":"v1","addr":"172.21.15.99:6833","nonce":1865105479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34445,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/1206393914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1206393914},{"type":"v1","addr":"172.21.15.99:6835","nonce":1206393914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-03T18:22:28.837091+0000","modified":"2024-05-03T18:23:49.758398+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34508},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34472":{"gid":34472,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6837/3360970342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3360970342},{"type":"v1","addr":"172.21.15.99:6837","nonce":3360970342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34508":{"gid":34508,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/919907308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":919907308},{"type":"v1","addr":"172.21.15.99:6839","nonce":919907308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34508,"qdb_cluster":[34508]},"id":1}]} 2024-05-03T18:27:38.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:38.031+0000 7f2409200700 1 -- 172.21.15.33:0/146587389 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f23e80b2b00 msgr2=0x7f23e80b4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:38.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:38.031+0000 7f2409200700 1 --2- 172.21.15.33:0/146587389 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f23e80b2b00 0x7f23e80b4fc0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f240419cab0 tx=0x7f23f400b500 comp rx=0 tx=0).stop 2024-05-03T18:27:38.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:38.031+0000 7f2409200700 1 -- 172.21.15.33:0/146587389 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2404108850 msgr2=0x7f240419b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:38.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:38.031+0000 7f2409200700 1 --2- 172.21.15.33:0/146587389 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2404108850 0x7f240419b970 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f23fc00b0d0 tx=0x7f23fc007d90 comp rx=0 tx=0).stop 2024-05-03T18:27:38.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:38.031+0000 7f2409200700 1 -- 172.21.15.33:0/146587389 shutdown_connections 2024-05-03T18:27:38.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:38.031+0000 7f2409200700 1 --2- 172.21.15.33:0/146587389 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2404108850 0x7f240419b970 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:38.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:38.031+0000 7f2409200700 1 --2- 172.21.15.33:0/146587389 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f23e80b2b00 0x7f23e80b4fc0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:38.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:38.031+0000 7f2409200700 1 --2- 172.21.15.33:0/146587389 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2404103750 0x7f24041a17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:38.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:38.031+0000 7f2409200700 1 --2- 172.21.15.33:0/146587389 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2404104100 0x7f24041a1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:38.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:38.031+0000 7f2409200700 1 -- 172.21.15.33:0/146587389 >> 172.21.15.33:0/146587389 conn(0x7f240407a600 msgr2=0x7f2404101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:27:38.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:38.031+0000 7f2409200700 1 -- 172.21.15.33:0/146587389 shutdown_connections 2024-05-03T18:27:38.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:38.032+0000 7f2409200700 1 -- 172.21.15.33:0/146587389 wait complete. 2024-05-03T18:27:38.035 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 21 2024-05-03T18:27:38.863 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-03T18:27:39.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:38 smithi150 ceph-mon[32492]: pgmap v16: 129 pgs: 129 active+clean; 6.0 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 473 KiB/s wr, 35 op/s 2024-05-03T18:27:39.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:38 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/146587389' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:27:39.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:38 smithi033 ceph-mon[28330]: pgmap v16: 129 pgs: 129 active+clean; 6.0 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 473 KiB/s wr, 35 op/s 2024-05-03T18:27:39.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:38 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/146587389' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:27:39.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:38 smithi099 ceph-mon[31658]: pgmap v16: 129 pgs: 129 active+clean; 6.0 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 473 KiB/s wr, 35 op/s 2024-05-03T18:27:39.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:38 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/146587389' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:27:39.362 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:27:41.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:40 smithi150 ceph-mon[32492]: pgmap v17: 129 pgs: 129 active+clean; 19 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 1.5 MiB/s wr, 40 op/s 2024-05-03T18:27:41.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:40 smithi033 ceph-mon[28330]: pgmap v17: 129 pgs: 129 active+clean; 19 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 1.5 MiB/s wr, 40 op/s 2024-05-03T18:27:41.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:40 smithi099 ceph-mon[31658]: pgmap v17: 129 pgs: 129 active+clean; 19 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 1.5 MiB/s wr, 40 op/s 2024-05-03T18:27:41.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.889+0000 7f3c17200700 1 -- 172.21.15.33:0/1244844641 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c1006b1b0 msgr2=0x7f3c1006b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:41.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.889+0000 7f3c17200700 1 --2- 172.21.15.33:0/1244844641 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c1006b1b0 0x7f3c1006b590 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f3c00009990 tx=0x7f3c00009ca0 comp rx=0 tx=0).stop 2024-05-03T18:27:41.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.889+0000 7f3c17200700 1 -- 172.21.15.33:0/1244844641 shutdown_connections 2024-05-03T18:27:41.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.889+0000 7f3c17200700 1 --2- 172.21.15.33:0/1244844641 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3c1010ff50 0x7f3c10112340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:41.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.889+0000 7f3c17200700 1 --2- 172.21.15.33:0/1244844641 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c1006b1b0 0x7f3c1006b590 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:41.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.889+0000 7f3c17200700 1 --2- 172.21.15.33:0/1244844641 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3c1006bad0 0x7f3c1010fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:41.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.890+0000 7f3c17200700 1 -- 172.21.15.33:0/1244844641 >> 172.21.15.33:0/1244844641 conn(0x7f3c10079350 msgr2=0x7f3c10079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:27:41.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.890+0000 7f3c17200700 1 -- 172.21.15.33:0/1244844641 shutdown_connections 2024-05-03T18:27:41.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.890+0000 7f3c17200700 1 -- 172.21.15.33:0/1244844641 wait complete. 2024-05-03T18:27:41.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.891+0000 7f3c17200700 1 Processor -- start 2024-05-03T18:27:41.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.891+0000 7f3c17200700 1 -- start start 2024-05-03T18:27:41.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.892+0000 7f3c17200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c1006b1b0 0x7f3c101a1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:41.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.892+0000 7f3c17200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3c1006bad0 0x7f3c101a1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:41.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.892+0000 7f3c17200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3c1010ff50 0x7f3c1019bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:41.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.892+0000 7f3c17200700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3c10115770 con 0x7f3c1006b1b0 2024-05-03T18:27:41.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.892+0000 7f3c17200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3c101155f0 con 0x7f3c1010ff50 2024-05-03T18:27:41.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.892+0000 7f3c17200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3c101158f0 con 0x7f3c1006bad0 2024-05-03T18:27:41.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.893+0000 7f3c15200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3c1010ff50 0x7f3c1019bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:41.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.893+0000 7f3c15200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3c1010ff50 0x7f3c1019bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:49270/0 (socket says 172.21.15.33:49270) 2024-05-03T18:27:41.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.893+0000 7f3c15200700 1 -- 172.21.15.33:0/114229919 learned_addr learned my addr 172.21.15.33:0/114229919 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:27:41.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.893+0000 7f3c0fe00700 1 --2- 172.21.15.33:0/114229919 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c1006b1b0 0x7f3c101a1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:41.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.893+0000 7f3c0f400700 1 --2- 172.21.15.33:0/114229919 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3c1006bad0 0x7f3c101a1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:41.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.894+0000 7f3c15200700 1 -- 172.21.15.33:0/114229919 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3c1006bad0 msgr2=0x7f3c101a1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:41.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.894+0000 7f3c15200700 1 --2- 172.21.15.33:0/114229919 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3c1006bad0 0x7f3c101a1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:41.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.894+0000 7f3c15200700 1 -- 172.21.15.33:0/114229919 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c1006b1b0 msgr2=0x7f3c101a1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:41.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.894+0000 7f3c15200700 1 --2- 172.21.15.33:0/114229919 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c1006b1b0 0x7f3c101a1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:41.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.894+0000 7f3c15200700 1 -- 172.21.15.33:0/114229919 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c1019c410 con 0x7f3c1010ff50 2024-05-03T18:27:41.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.895+0000 7f3c15200700 1 --2- 172.21.15.33:0/114229919 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3c1010ff50 0x7f3c1019bb30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3c080094f0 tx=0x7f3c0800d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:27:41.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.895+0000 7f3c0cc00700 1 -- 172.21.15.33:0/114229919 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c08004db0 con 0x7f3c1010ff50 2024-05-03T18:27:41.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.895+0000 7f3c0cc00700 1 -- 172.21.15.33:0/114229919 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3c08009e20 con 0x7f3c1010ff50 2024-05-03T18:27:41.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.895+0000 7f3c0cc00700 1 -- 172.21.15.33:0/114229919 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c08015400 con 0x7f3c1010ff50 2024-05-03T18:27:41.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.897+0000 7f3c17200700 1 -- 172.21.15.33:0/114229919 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c1019c700 con 0x7f3c1010ff50 2024-05-03T18:27:41.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.897+0000 7f3c17200700 1 -- 172.21.15.33:0/114229919 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c101a89e0 con 0x7f3c1010ff50 2024-05-03T18:27:41.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.904+0000 7f3c0cc00700 1 -- 172.21.15.33:0/114229919 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 23) ==== 149828+0+0 (secure 0 0 0) 0x7f3c08005220 con 0x7f3c1010ff50 2024-05-03T18:27:41.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.904+0000 7f3c21b1eb80 1 Processor -- start 2024-05-03T18:27:41.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.905+0000 7f3c21b1eb80 1 -- start start 2024-05-03T18:27:41.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.905+0000 7f3c21b1eb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dbdba08140 0x55dbdba08520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:41.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.906+0000 7f3c21b1eb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dbdba08a60 0x55dbdba02740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:41.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.906+0000 7f3c21b1eb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dbdba02d60 0x55dbdba031e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:41.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.906+0000 7f3c21b1eb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55dbdb94ca20 con 0x55dbdba08140 2024-05-03T18:27:41.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.906+0000 7f3c21b1eb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55dbdb8ea760 con 0x55dbdba02d60 2024-05-03T18:27:41.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.906+0000 7f3c21b1eb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55dbdb9489e0 con 0x55dbdba08a60 2024-05-03T18:27:41.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.906+0000 7f3c0cc00700 1 --2- 172.21.15.33:0/114229919 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f3bf40b2b00 0x7f3bf40b4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:41.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.906+0000 7f3c0f400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dbdba08140 0x55dbdba08520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:41.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.906+0000 7f3c0f400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dbdba08140 0x55dbdba08520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41302/0 (socket says 172.21.15.33:41302) 2024-05-03T18:27:41.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.906+0000 7f3c0f400700 1 -- 172.21.15.33:0/825243993 learned_addr learned my addr 172.21.15.33:0/825243993 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:27:41.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.906+0000 7f3c0cc00700 1 -- 172.21.15.33:0/114229919 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f3c080e2490 con 0x7f3c1010ff50 2024-05-03T18:27:41.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.906+0000 7f3c15200700 1 --2- 172.21.15.33:0/114229919 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f3bf40b2b00 0x7f3bf40b4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:41.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.907+0000 7f3c0fe00700 1 --2- 172.21.15.33:0/825243993 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dbdba08a60 0x55dbdba02740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:41.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.908+0000 7f3c0fe00700 1 -- 172.21.15.33:0/825243993 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dbdba02d60 msgr2=0x55dbdba031e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-03T18:27:41.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.908+0000 7f3c0fe00700 1 --2- 172.21.15.33:0/825243993 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dbdba02d60 0x55dbdba031e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:41.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.908+0000 7f3c0fe00700 1 -- 172.21.15.33:0/825243993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dbdba08140 msgr2=0x55dbdba08520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:27:41.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.908+0000 7f3c0fe00700 1 --2- 172.21.15.33:0/825243993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dbdba08140 0x55dbdba08520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:41.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.908+0000 7f3c0fe00700 1 -- 172.21.15.33:0/825243993 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbdb8998d0 con 0x55dbdba08a60 2024-05-03T18:27:41.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.908+0000 7f3c15200700 1 --2- 172.21.15.33:0/114229919 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f3bf40b2b00 0x7f3bf40b4fc0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3c080df790 tx=0x7f3c080096d0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:27:41.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.908+0000 7f3c0fe00700 1 --2- 172.21.15.33:0/825243993 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dbdba08a60 0x55dbdba02740 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3c000098e0 tx=0x7f3c000096e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78f28fca95b3f457 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:27:41.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.909+0000 7f3c05600700 1 -- 172.21.15.33:0/825243993 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c0000b9d0 con 0x55dbdba08a60 2024-05-03T18:27:41.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.909+0000 7f3c05600700 1 -- 172.21.15.33:0/825243993 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3c00027070 con 0x55dbdba08a60 2024-05-03T18:27:41.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.909+0000 7f3c05600700 1 -- 172.21.15.33:0/825243993 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c0000fcb0 con 0x55dbdba08a60 2024-05-03T18:27:41.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.910+0000 7f3c21b1eb80 1 -- 172.21.15.33:0/825243993 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dbdba08a60 msgr2=0x55dbdba02740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:41.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.910+0000 7f3c21b1eb80 1 --2- 172.21.15.33:0/825243993 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dbdba08a60 0x55dbdba02740 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3c000098e0 tx=0x7f3c000096e0 comp rx=0 tx=0).stop 2024-05-03T18:27:41.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.911+0000 7f3c21b1eb80 1 -- 172.21.15.33:0/825243993 shutdown_connections 2024-05-03T18:27:41.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.911+0000 7f3c21b1eb80 1 --2- 172.21.15.33:0/825243993 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dbdba02d60 0x55dbdba031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:41.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.911+0000 7f3c21b1eb80 1 --2- 172.21.15.33:0/825243993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dbdba08140 0x55dbdba08520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:41.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.911+0000 7f3c21b1eb80 1 --2- 172.21.15.33:0/825243993 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dbdba08a60 0x55dbdba02740 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:41.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.911+0000 7f3c21b1eb80 1 -- 172.21.15.33:0/825243993 >> 172.21.15.33:0/825243993 conn(0x55dbdb9726d0 msgr2=0x55dbdb9f4900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:27:41.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.911+0000 7f3c21b1eb80 1 -- 172.21.15.33:0/825243993 shutdown_connections 2024-05-03T18:27:41.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.912+0000 7f3c21b1eb80 1 -- 172.21.15.33:0/825243993 wait complete. 2024-05-03T18:27:41.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.913+0000 7f3c21b1eb80 1 -- start start 2024-05-03T18:27:41.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.913+0000 7f3c21b1eb80 1 Processor -- start 2024-05-03T18:27:41.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.914+0000 7f3c21b1eb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dbdba02d60 0x55dbdba0e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:41.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.914+0000 7f3c21b1eb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dbdba08140 0x55dbdba0ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:41.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.914+0000 7f3c21b1eb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dbdba08a60 0x55dbdba0bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:41.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.914+0000 7f3c21b1eb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55dbdba0f4b0 con 0x55dbdba02d60 2024-05-03T18:27:41.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.914+0000 7f3c21b1eb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55dbdb8b2b20 con 0x55dbdba08140 2024-05-03T18:27:41.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.914+0000 7f3c21b1eb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55dbdba0f630 con 0x55dbdba08a60 2024-05-03T18:27:41.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.914+0000 7f3c21b1eb80 10 client.0 _ll_register_callbacks cb 0x55dbdb8f3c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:27:41.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.915+0000 7f3c0f400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dbdba02d60 0x55dbdba0e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:41.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.915+0000 7f3c0fe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dbdba08140 0x55dbdba0ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:41.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.915+0000 7f3c0f400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dbdba02d60 0x55dbdba0e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41312/0 (socket says 172.21.15.33:41312) 2024-05-03T18:27:41.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.915+0000 7f3c0fe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dbdba08140 0x55dbdba0ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:49276/0 (socket says 172.21.15.33:49276) 2024-05-03T18:27:41.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.915+0000 7f3c0fe00700 1 -- 172.21.15.33:0/3573901398 learned_addr learned my addr 172.21.15.33:0/3573901398 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:27:41.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.915+0000 7f3c0f400700 1 --2- 172.21.15.33:0/3573901398 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dbdba08a60 0x55dbdba0bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:41.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.916+0000 7f3c0fe00700 1 -- 172.21.15.33:0/3573901398 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dbdba08a60 msgr2=0x55dbdba0bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:27:41.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.916+0000 7f3c0fe00700 1 --2- 172.21.15.33:0/3573901398 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dbdba08a60 0x55dbdba0bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:41.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.916+0000 7f3c0fe00700 1 -- 172.21.15.33:0/3573901398 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dbdba02d60 msgr2=0x55dbdba0e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:27:41.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.916+0000 7f3c0fe00700 1 --2- 172.21.15.33:0/3573901398 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dbdba02d60 0x55dbdba0e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:41.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.916+0000 7f3c0fe00700 1 -- 172.21.15.33:0/3573901398 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbdb912420 con 0x55dbdba08140 2024-05-03T18:27:41.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.916+0000 7f3c0fe00700 1 --2- 172.21.15.33:0/3573901398 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dbdba08140 0x55dbdba0ba40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3c0000bf00 tx=0x7f3c0000bf30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=313e73b4b143857c server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:27:41.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.917+0000 7f3bfa000700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-03T18:27:41.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.917+0000 7f3bfa000700 1 -- 172.21.15.33:0/3573901398 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c00031020 con 0x55dbdba08140 2024-05-03T18:27:41.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.917+0000 7f3bfa000700 1 -- 172.21.15.33:0/3573901398 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f3c0000f930 con 0x55dbdba08140 2024-05-03T18:27:41.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.917+0000 7f3c21b1eb80 1 -- 172.21.15.33:0/3573901398 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c00027070 con 0x55dbdba08140 2024-05-03T18:27:41.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.918+0000 7f3bfa000700 1 -- 172.21.15.33:0/3573901398 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f3c0000fb30 con 0x55dbdba08140 2024-05-03T18:27:41.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.918+0000 7f3c21b1eb80 10 client.35072 fetch_fsmap learned FSMap version 21 2024-05-03T18:27:41.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.918+0000 7f3c21b1eb80 1 -- 172.21.15.33:0/3573901398 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55dbdba2ac50 con 0x55dbdba08140 2024-05-03T18:27:41.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.919+0000 7f3bfa000700 1 -- 172.21.15.33:0/3573901398 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3c00027070 con 0x55dbdba08140 2024-05-03T18:27:41.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.919+0000 7f3bfa000700 1 -- 172.21.15.33:0/3573901398 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f3c0002bc50 con 0x55dbdba08140 2024-05-03T18:27:41.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.919+0000 7f3c21b1eb80 10 client.35072 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:27:41.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.919+0000 7f3c21b1eb80 10 client.35072 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:27:41.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.920+0000 7f3c21b1eb80 20 client.35072 populate_metadata read hostname 'smithi033' 2024-05-03T18:27:41.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.920+0000 7f3c21b1eb80 1 --2- 172.21.15.33:0/3573901398 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55dbdba2d1c0 0x55dbdba2f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:41.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.920+0000 7f3c21b1eb80 4 client.35072 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:27:41.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.920+0000 7f3c21b1eb80 1 -- 172.21.15.33:0/3573901398 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dbdb9c6ff0 con 0x55dbdba2d1c0 2024-05-03T18:27:41.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.920+0000 7f3c0f400700 1 --2- 172.21.15.33:0/3573901398 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55dbdba2d1c0 0x55dbdba2f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:41.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.921+0000 7f3c0f400700 1 --2- 172.21.15.33:0/3573901398 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55dbdba2d1c0 0x55dbdba2f620 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43db09a43341019b server_cookie=307e3a791c698704 in_seq=0 out_seq=0 2024-05-03T18:27:41.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.921+0000 7f3bfa000700 10 client.35072 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:27:41.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.923+0000 7f3bfa000700 1 -- 172.21.15.33:0/3573901398 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55dbdb9c6ff0 con 0x55dbdba2d1c0 2024-05-03T18:27:41.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.923+0000 7f3bfa000700 10 client.35072 handle_command_reply: tid=0 2024-05-03T18:27:41.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.923+0000 7f3c21b1eb80 1 client.35072 shutdown 2024-05-03T18:27:41.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.923+0000 7f3c21b1eb80 1 -- 172.21.15.33:0/3573901398 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dbdba08140 msgr2=0x55dbdba0ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:41.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.923+0000 7f3c21b1eb80 1 --2- 172.21.15.33:0/3573901398 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dbdba08140 0x55dbdba0ba40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3c0000bf00 tx=0x7f3c0000bf30 comp rx=0 tx=0).stop 2024-05-03T18:27:41.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.924+0000 7f3c21b1eb80 1 -- 172.21.15.33:0/3573901398 shutdown_connections 2024-05-03T18:27:41.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.924+0000 7f3c21b1eb80 1 --2- 172.21.15.33:0/3573901398 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55dbdba2d1c0 0x55dbdba2f620 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:41.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.924+0000 7f3c21b1eb80 1 --2- 172.21.15.33:0/3573901398 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dbdba08a60 0x55dbdba0bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:41.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.924+0000 7f3c21b1eb80 1 --2- 172.21.15.33:0/3573901398 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dbdba02d60 0x55dbdba0e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:41.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.924+0000 7f3c21b1eb80 1 --2- 172.21.15.33:0/3573901398 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dbdba08140 0x55dbdba0ba40 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:41.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.924+0000 7f3c21b1eb80 1 -- 172.21.15.33:0/3573901398 >> 172.21.15.33:0/3573901398 conn(0x55dbdb9726d0 msgr2=0x55dbdb9ef3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:27:41.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.924+0000 7f3bfa000700 0 client.35072 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:27:41.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.924+0000 7f3c21b1eb80 1 -- 172.21.15.33:0/3573901398 shutdown_connections 2024-05-03T18:27:41.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.924+0000 7f3c21b1eb80 1 -- 172.21.15.33:0/3573901398 wait complete. 2024-05-03T18:27:41.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.924+0000 7f3c21b1eb80 20 client.35072 trim_cache size 0 max 16384 2024-05-03T18:27:41.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.946+0000 7f3c21b1eb80 1 Processor -- start 2024-05-03T18:27:41.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.946+0000 7f3c21b1eb80 1 -- start start 2024-05-03T18:27:41.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.946+0000 7f3c21b1eb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dbdba02d60 0x55dbdba0a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:41.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.947+0000 7f3c21b1eb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dbdba08140 0x55dbdba0a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:41.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.947+0000 7f3c21b1eb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dbdba08a60 0x55dbdba34ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:41.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.947+0000 7f3c21b1eb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3c1006f510 con 0x55dbdba02d60 2024-05-03T18:27:41.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.947+0000 7f3c21b1eb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3c10071db0 con 0x55dbdba08a60 2024-05-03T18:27:41.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.947+0000 7f3c21b1eb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3c1006fba0 con 0x55dbdba08140 2024-05-03T18:27:41.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.947+0000 7f3c0f400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dbdba02d60 0x55dbdba0a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:41.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.947+0000 7f3c0f400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dbdba02d60 0x55dbdba0a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41320/0 (socket says 172.21.15.33:41320) 2024-05-03T18:27:41.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.947+0000 7f3c0f400700 1 -- 172.21.15.33:0/2233023136 learned_addr learned my addr 172.21.15.33:0/2233023136 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:27:41.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.947+0000 7f3c0f400700 1 -- 172.21.15.33:0/2233023136 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dbdba08140 msgr2=0x55dbdba0a890 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-03T18:27:41.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.948+0000 7f3c0f400700 1 --2- 172.21.15.33:0/2233023136 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dbdba08140 0x55dbdba0a890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:41.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.948+0000 7f3c0f400700 1 -- 172.21.15.33:0/2233023136 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dbdba08a60 msgr2=0x55dbdba34ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:27:41.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.948+0000 7f3c0f400700 1 --2- 172.21.15.33:0/2233023136 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dbdba08a60 0x55dbdba34ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:41.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.948+0000 7f3c0f400700 1 -- 172.21.15.33:0/2233023136 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbdb9ece60 con 0x55dbdba02d60 2024-05-03T18:27:41.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.948+0000 7f3c0f400700 1 --2- 172.21.15.33:0/2233023136 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dbdba02d60 0x55dbdba0a350 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc00b4c0 tx=0x7f3bfc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39a3e94389bfb66f server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:27:41.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.948+0000 7f3c04c00700 1 -- 172.21.15.33:0/2233023136 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bfc003c10 con 0x55dbdba02d60 2024-05-03T18:27:41.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.948+0000 7f3c04c00700 1 -- 172.21.15.33:0/2233023136 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3bfc004cc0 con 0x55dbdba02d60 2024-05-03T18:27:41.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.948+0000 7f3c04c00700 1 -- 172.21.15.33:0/2233023136 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bfc005370 con 0x55dbdba02d60 2024-05-03T18:27:41.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.950+0000 7f3c21b1eb80 1 -- 172.21.15.33:0/2233023136 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dbdba02d60 msgr2=0x55dbdba0a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:41.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.950+0000 7f3c21b1eb80 1 --2- 172.21.15.33:0/2233023136 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dbdba02d60 0x55dbdba0a350 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc00b4c0 tx=0x7f3bfc004750 comp rx=0 tx=0).stop 2024-05-03T18:27:41.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.950+0000 7f3c21b1eb80 1 -- 172.21.15.33:0/2233023136 shutdown_connections 2024-05-03T18:27:41.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.950+0000 7f3c21b1eb80 1 --2- 172.21.15.33:0/2233023136 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dbdba08a60 0x55dbdba34ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:41.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.950+0000 7f3c21b1eb80 1 --2- 172.21.15.33:0/2233023136 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dbdba02d60 0x55dbdba0a350 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:41.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.950+0000 7f3c21b1eb80 1 --2- 172.21.15.33:0/2233023136 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dbdba08140 0x55dbdba0a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:41.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.950+0000 7f3c21b1eb80 1 -- 172.21.15.33:0/2233023136 >> 172.21.15.33:0/2233023136 conn(0x55dbdb9726d0 msgr2=0x55dbdba0ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:27:41.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.950+0000 7f3c21b1eb80 1 -- 172.21.15.33:0/2233023136 shutdown_connections 2024-05-03T18:27:41.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.950+0000 7f3c21b1eb80 1 -- 172.21.15.33:0/2233023136 wait complete. 2024-05-03T18:27:41.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.951+0000 7f3c21b1eb80 1 -- start start 2024-05-03T18:27:41.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.951+0000 7f3c21b1eb80 1 Processor -- start 2024-05-03T18:27:41.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.952+0000 7f3c21b1eb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dbdba02d60 0x55dbdba14ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:41.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.952+0000 7f3c21b1eb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dbdba08140 0x55dbdba15010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:41.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.952+0000 7f3c21b1eb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dbdba08a60 0x55dbdba33f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:41.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.952+0000 7f3c21b1eb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3c10070270 con 0x55dbdba08a60 2024-05-03T18:27:41.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.952+0000 7f3c21b1eb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3c100716e0 con 0x55dbdba02d60 2024-05-03T18:27:41.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.952+0000 7f3c21b1eb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3c10070940 con 0x55dbdba08140 2024-05-03T18:27:41.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.952+0000 7f3c21b1eb80 10 client.0 _ll_register_callbacks cb 0x55dbdb8f3c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:27:41.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.953+0000 7f3c0fe00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dbdba08140 0x55dbdba15010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:41.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.953+0000 7f3c0fe00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dbdba08140 0x55dbdba15010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:35118/0 (socket says 172.21.15.33:35118) 2024-05-03T18:27:41.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.953+0000 7f3c0fe00700 1 -- 172.21.15.33:0/3414907626 learned_addr learned my addr 172.21.15.33:0/3414907626 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:27:41.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.953+0000 7f3c0f400700 1 --2- 172.21.15.33:0/3414907626 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dbdba08a60 0x55dbdba33f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:41.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.953+0000 7f3c0f400700 1 --2- 172.21.15.33:0/3414907626 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dbdba02d60 0x55dbdba14ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:41.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.954+0000 7f3c0fe00700 1 -- 172.21.15.33:0/3414907626 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dbdba02d60 msgr2=0x55dbdba14ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:27:41.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.954+0000 7f3c0fe00700 1 --2- 172.21.15.33:0/3414907626 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dbdba02d60 0x55dbdba14ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:41.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.954+0000 7f3c0fe00700 1 -- 172.21.15.33:0/3414907626 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dbdba08a60 msgr2=0x55dbdba33f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:27:41.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.954+0000 7f3c0fe00700 1 --2- 172.21.15.33:0/3414907626 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dbdba08a60 0x55dbdba33f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:41.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.954+0000 7f3c0fe00700 1 -- 172.21.15.33:0/3414907626 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbdba34720 con 0x55dbdba08140 2024-05-03T18:27:41.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.954+0000 7f3c0fe00700 1 --2- 172.21.15.33:0/3414907626 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dbdba08140 0x55dbdba15010 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3c00029f80 tx=0x7f3c00009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=825c41306408b922 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:27:41.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.954+0000 7f3bfa000700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-03T18:27:41.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.955+0000 7f3bfa000700 1 -- 172.21.15.33:0/3414907626 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c0000fb30 con 0x55dbdba08140 2024-05-03T18:27:41.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.955+0000 7f3bfa000700 1 -- 172.21.15.33:0/3414907626 <== mon.1 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f3c0000f930 con 0x55dbdba08140 2024-05-03T18:27:41.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.955+0000 7f3c21b1eb80 1 -- 172.21.15.33:0/3414907626 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3bfc004cc0 con 0x55dbdba08140 2024-05-03T18:27:41.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.956+0000 7f3bfa000700 1 -- 172.21.15.33:0/3414907626 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f3c0002ab30 con 0x55dbdba08140 2024-05-03T18:27:41.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.956+0000 7f3c21b1eb80 10 client.25087 fetch_fsmap learned FSMap version 21 2024-05-03T18:27:41.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.956+0000 7f3c21b1eb80 1 -- 172.21.15.33:0/3414907626 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55dbdba975a0 con 0x55dbdba08140 2024-05-03T18:27:41.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.956+0000 7f3bfa000700 1 -- 172.21.15.33:0/3414907626 <== mon.1 v2:172.21.15.150:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3bfc004cc0 con 0x55dbdba08140 2024-05-03T18:27:41.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.958+0000 7f3bfa000700 1 -- 172.21.15.33:0/3414907626 <== mon.1 v2:172.21.15.150:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f3c0002bc50 con 0x55dbdba08140 2024-05-03T18:27:41.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.958+0000 7f3c21b1eb80 10 client.25087 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:27:41.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.958+0000 7f3c21b1eb80 10 client.25087 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:27:41.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.958+0000 7f3c21b1eb80 20 client.25087 populate_metadata read hostname 'smithi033' 2024-05-03T18:27:41.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.958+0000 7f3c21b1eb80 1 --2- 172.21.15.33:0/3414907626 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55dbdba97ac0 0x55dbdba99ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:41.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.958+0000 7f3c21b1eb80 4 client.25087 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-03T18:27:41.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.958+0000 7f3c21b1eb80 1 -- 172.21.15.33:0/3414907626 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dbdb95a400 con 0x55dbdba97ac0 2024-05-03T18:27:41.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.959+0000 7f3c0f400700 1 --2- 172.21.15.33:0/3414907626 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55dbdba97ac0 0x55dbdba99ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:41.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.961+0000 7f3c0f400700 1 --2- 172.21.15.33:0/3414907626 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55dbdba97ac0 0x55dbdba99ee0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56d28911d50134dc server_cookie=1c23cb3873570659 in_seq=0 out_seq=0 2024-05-03T18:27:41.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.961+0000 7f3bfa000700 10 client.25087 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:27:41.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.961+0000 7f3bfa000700 1 -- 172.21.15.33:0/3414907626 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dbdb95a400 con 0x55dbdba97ac0 2024-05-03T18:27:41.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.961+0000 7f3bfa000700 10 client.25087 handle_command_reply: tid=0 2024-05-03T18:27:41.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.961+0000 7f3c21b1eb80 1 client.25087 shutdown 2024-05-03T18:27:41.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.962+0000 7f3c21b1eb80 1 -- 172.21.15.33:0/3414907626 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dbdba08140 msgr2=0x55dbdba15010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:41.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.962+0000 7f3c21b1eb80 1 --2- 172.21.15.33:0/3414907626 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dbdba08140 0x55dbdba15010 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3c00029f80 tx=0x7f3c00009ca0 comp rx=0 tx=0).stop 2024-05-03T18:27:41.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.962+0000 7f3c21b1eb80 1 -- 172.21.15.33:0/3414907626 shutdown_connections 2024-05-03T18:27:41.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.962+0000 7f3c21b1eb80 1 --2- 172.21.15.33:0/3414907626 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55dbdba97ac0 0x55dbdba99ee0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:41.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.962+0000 7f3c21b1eb80 1 --2- 172.21.15.33:0/3414907626 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dbdba08a60 0x55dbdba33f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:41.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.962+0000 7f3c21b1eb80 1 --2- 172.21.15.33:0/3414907626 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dbdba02d60 0x55dbdba14ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:41.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.962+0000 7f3c21b1eb80 1 --2- 172.21.15.33:0/3414907626 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dbdba08140 0x55dbdba15010 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:41.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.962+0000 7f3bfa000700 0 client.25087 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:27:41.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.962+0000 7f3c21b1eb80 1 -- 172.21.15.33:0/3414907626 >> 172.21.15.33:0/3414907626 conn(0x55dbdb9726d0 msgr2=0x55dbdba0a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:27:41.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.962+0000 7f3c21b1eb80 1 -- 172.21.15.33:0/3414907626 shutdown_connections 2024-05-03T18:27:41.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.963+0000 7f3c21b1eb80 1 -- 172.21.15.33:0/3414907626 wait complete. 2024-05-03T18:27:41.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.963+0000 7f3c21b1eb80 20 client.25087 trim_cache size 0 max 16384 2024-05-03T18:27:41.964 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-03T18:27:41.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.964+0000 7f3c17200700 1 -- 172.21.15.33:0/114229919 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f3bf40b2b00 msgr2=0x7f3bf40b4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:41.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.964+0000 7f3c17200700 1 --2- 172.21.15.33:0/114229919 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f3bf40b2b00 0x7f3bf40b4fc0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3c080df790 tx=0x7f3c080096d0 comp rx=0 tx=0).stop 2024-05-03T18:27:41.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.964+0000 7f3c17200700 1 -- 172.21.15.33:0/114229919 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3c1010ff50 msgr2=0x7f3c1019bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:41.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.964+0000 7f3c17200700 1 --2- 172.21.15.33:0/114229919 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3c1010ff50 0x7f3c1019bb30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3c080094f0 tx=0x7f3c0800d3b0 comp rx=0 tx=0).stop 2024-05-03T18:27:41.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.964+0000 7f3c17200700 1 -- 172.21.15.33:0/114229919 shutdown_connections 2024-05-03T18:27:41.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.964+0000 7f3c17200700 1 --2- 172.21.15.33:0/114229919 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f3bf40b2b00 0x7f3bf40b4fc0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:41.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.964+0000 7f3c17200700 1 --2- 172.21.15.33:0/114229919 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3c1010ff50 0x7f3c1019bb30 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:41.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.964+0000 7f3c17200700 1 --2- 172.21.15.33:0/114229919 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c1006b1b0 0x7f3c101a1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:41.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.964+0000 7f3c17200700 1 --2- 172.21.15.33:0/114229919 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3c1006bad0 0x7f3c101a1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:41.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.964+0000 7f3c17200700 1 -- 172.21.15.33:0/114229919 >> 172.21.15.33:0/114229919 conn(0x7f3c10079350 msgr2=0x7f3c10111520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:27:41.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.965+0000 7f3c17200700 1 -- 172.21.15.33:0/114229919 shutdown_connections 2024-05-03T18:27:41.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:41.965+0000 7f3c17200700 1 -- 172.21.15.33:0/114229919 wait complete. 2024-05-03T18:27:43.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:42 smithi150 ceph-mon[32492]: pgmap v18: 129 pgs: 129 active+clean; 19 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 1.5 MiB/s wr, 39 op/s 2024-05-03T18:27:43.137 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:42 smithi099 ceph-mon[31658]: pgmap v18: 129 pgs: 129 active+clean; 19 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 1.5 MiB/s wr, 39 op/s 2024-05-03T18:27:43.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:42 smithi033 ceph-mon[28330]: pgmap v18: 129 pgs: 129 active+clean; 19 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 1.5 MiB/s wr, 39 op/s 2024-05-03T18:27:43.900 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-03T18:27:43.901 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-03T18:27:44.409 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:27:45.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:44 smithi099 ceph-mon[31658]: pgmap v19: 129 pgs: 129 active+clean; 22 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 1.8 MiB/s wr, 57 op/s 2024-05-03T18:27:45.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:44 smithi150 ceph-mon[32492]: pgmap v19: 129 pgs: 129 active+clean; 22 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 1.8 MiB/s wr, 57 op/s 2024-05-03T18:27:45.155 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:44 smithi033 ceph-mon[28330]: pgmap v19: 129 pgs: 129 active+clean; 22 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 1.8 MiB/s wr, 57 op/s 2024-05-03T18:27:47.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.018+0000 7f8009800700 1 -- 172.21.15.33:0/1101185444 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f800410f9d0 msgr2=0x7f800410fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:47.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.018+0000 7f8009800700 1 --2- 172.21.15.33:0/1101185444 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f800410f9d0 0x7f800410fdb0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f7ff4009990 tx=0x7f7ff4009ca0 comp rx=0 tx=0).stop 2024-05-03T18:27:47.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.020+0000 7f8009800700 1 -- 172.21.15.33:0/1101185444 shutdown_connections 2024-05-03T18:27:47.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.020+0000 7f8009800700 1 --2- 172.21.15.33:0/1101185444 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f800410a3a0 0x7f80041168a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:47.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.020+0000 7f8009800700 1 --2- 172.21.15.33:0/1101185444 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f800410f9d0 0x7f800410fdb0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:47.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.020+0000 7f8009800700 1 --2- 172.21.15.33:0/1101185444 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80041099e0 0x7f8004109e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:47.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.020+0000 7f8009800700 1 -- 172.21.15.33:0/1101185444 >> 172.21.15.33:0/1101185444 conn(0x7f8004079200 msgr2=0x7f80041082c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:27:47.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.020+0000 7f8009800700 1 -- 172.21.15.33:0/1101185444 shutdown_connections 2024-05-03T18:27:47.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.020+0000 7f8009800700 1 -- 172.21.15.33:0/1101185444 wait complete. 2024-05-03T18:27:47.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.021+0000 7f8009800700 1 Processor -- start 2024-05-03T18:27:47.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.021+0000 7f8009800700 1 -- start start 2024-05-03T18:27:47.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.022+0000 7f8009800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f80041099e0 0x7f800419ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:47.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.022+0000 7f8009800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f800410a3a0 0x7f80041a0500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:47.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.022+0000 7f8009800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f800410f9d0 0x7f80041a4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:47.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.022+0000 7f8009800700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8004119ad0 con 0x7f800410a3a0 2024-05-03T18:27:47.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.022+0000 7f8009800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8004119950 con 0x7f800410f9d0 2024-05-03T18:27:47.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.022+0000 7f8009800700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8004119c50 con 0x7f80041099e0 2024-05-03T18:27:47.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.022+0000 7f8003400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f800410a3a0 0x7f80041a0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:47.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.023+0000 7f8003400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f800410a3a0 0x7f80041a0500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39842/0 (socket says 172.21.15.33:39842) 2024-05-03T18:27:47.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.023+0000 7f8003400700 1 -- 172.21.15.33:0/3600189309 learned_addr learned my addr 172.21.15.33:0/3600189309 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:27:47.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.023+0000 7f8008e00700 1 --2- 172.21.15.33:0/3600189309 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f800410f9d0 0x7f80041a4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:47.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.023+0000 7f8003e00700 1 --2- 172.21.15.33:0/3600189309 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f80041099e0 0x7f800419ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:47.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.023+0000 7f8003400700 1 -- 172.21.15.33:0/3600189309 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f80041099e0 msgr2=0x7f800419ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:47.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.023+0000 7f8003400700 1 --2- 172.21.15.33:0/3600189309 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f80041099e0 0x7f800419ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:47.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.023+0000 7f8003400700 1 -- 172.21.15.33:0/3600189309 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f800410f9d0 msgr2=0x7f80041a4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:47.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.023+0000 7f8003400700 1 --2- 172.21.15.33:0/3600189309 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f800410f9d0 0x7f80041a4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:47.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.023+0000 7f8003400700 1 -- 172.21.15.33:0/3600189309 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80041a4fd0 con 0x7f800410a3a0 2024-05-03T18:27:47.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.024+0000 7f8003400700 1 --2- 172.21.15.33:0/3600189309 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f800410a3a0 0x7f80041a0500 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f7ff000d950 tx=0x7f7ff000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:27:47.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.024+0000 7f8000c00700 1 -- 172.21.15.33:0/3600189309 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff0016070 con 0x7f800410a3a0 2024-05-03T18:27:47.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.024+0000 7f8000c00700 1 -- 172.21.15.33:0/3600189309 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7ff0010460 con 0x7f800410a3a0 2024-05-03T18:27:47.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.024+0000 7f8000c00700 1 -- 172.21.15.33:0/3600189309 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff000f620 con 0x7f800410a3a0 2024-05-03T18:27:47.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.024+0000 7f8009800700 1 -- 172.21.15.33:0/3600189309 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80041a52c0 con 0x7f800410a3a0 2024-05-03T18:27:47.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.024+0000 7f8009800700 1 -- 172.21.15.33:0/3600189309 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f800406cec0 con 0x7f800410a3a0 2024-05-03T18:27:47.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.031+0000 7f8000c00700 1 -- 172.21.15.33:0/3600189309 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 23) ==== 149828+0+0 (secure 0 0 0) 0x7f7ff0019070 con 0x7f800410a3a0 2024-05-03T18:27:47.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.031+0000 7f8015826b80 1 Processor -- start 2024-05-03T18:27:47.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.032+0000 7f8015826b80 1 -- start start 2024-05-03T18:27:47.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.032+0000 7f8015826b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600bdb3cac0 0x5600bdb3cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:47.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.032+0000 7f8015826b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5600bdb3d3e0 0x5600bdb37030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:47.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.032+0000 7f8015826b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600bdb37650 0x5600bdb37ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:47.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.032+0000 7f8015826b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5600bda89aa0 con 0x5600bdb37650 2024-05-03T18:27:47.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.032+0000 7f8015826b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5600bda277e0 con 0x5600bdb3d3e0 2024-05-03T18:27:47.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.032+0000 7f8015826b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5600bda85a60 con 0x5600bdb3cac0 2024-05-03T18:27:47.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.033+0000 7f8003e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5600bdb3d3e0 0x5600bdb37030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:47.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.033+0000 7f8008e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600bdb3cac0 0x5600bdb3cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:47.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.033+0000 7f8003e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5600bdb3d3e0 0x5600bdb37030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:49690/0 (socket says 172.21.15.33:49690) 2024-05-03T18:27:47.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.033+0000 7f8008e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600bdb3cac0 0x5600bdb3cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:41958/0 (socket says 172.21.15.33:41958) 2024-05-03T18:27:47.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.033+0000 7f8003e00700 1 -- 172.21.15.33:0/358490207 learned_addr learned my addr 172.21.15.33:0/358490207 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:27:47.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.034+0000 7f8000c00700 1 --2- 172.21.15.33:0/3600189309 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f7fe80b2b00 0x7f7fe80b4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:47.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.034+0000 7f8003e00700 1 -- 172.21.15.33:0/358490207 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600bdb3cac0 msgr2=0x5600bdb3cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:27:47.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.034+0000 7f8003e00700 1 --2- 172.21.15.33:0/358490207 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600bdb3cac0 0x5600bdb3cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:47.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.034+0000 7f8003e00700 1 -- 172.21.15.33:0/358490207 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600bdb37650 msgr2=0x5600bdb37ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:27:47.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.034+0000 7f8000c00700 1 -- 172.21.15.33:0/3600189309 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f7ff00e26e0 con 0x7f800410a3a0 2024-05-03T18:27:47.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.034+0000 7f8003400700 1 --2- 172.21.15.33:0/358490207 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600bdb37650 0x5600bdb37ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:47.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.035+0000 7f8003e00700 1 --2- 172.21.15.33:0/358490207 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600bdb37650 0x5600bdb37ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:47.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.035+0000 7f8003e00700 1 -- 172.21.15.33:0/358490207 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600bd9d6950 con 0x5600bdb3d3e0 2024-05-03T18:27:47.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.035+0000 7f8008e00700 1 --2- 172.21.15.33:0/358490207 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600bdb3cac0 0x5600bdb3cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:27:47.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.035+0000 7f8003400700 1 --2- 172.21.15.33:0/358490207 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600bdb37650 0x5600bdb37ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:27:47.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.036+0000 7f8003e00700 1 --2- 172.21.15.33:0/358490207 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5600bdb3d3e0 0x5600bdb37030 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7ff4015000 tx=0x7f7ff4005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57ff61179e827827 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:27:47.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.036+0000 7f8008e00700 1 --2- 172.21.15.33:0/3600189309 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f7fe80b2b00 0x7f7fe80b4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:47.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.036+0000 7f7ff9600700 1 -- 172.21.15.33:0/358490207 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff4026070 con 0x5600bdb3d3e0 2024-05-03T18:27:47.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.037+0000 7f7ff9600700 1 -- 172.21.15.33:0/358490207 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7ff4005070 con 0x5600bdb3d3e0 2024-05-03T18:27:47.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.037+0000 7f7ff9600700 1 -- 172.21.15.33:0/358490207 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff4003bf0 con 0x5600bdb3d3e0 2024-05-03T18:27:47.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.038+0000 7f8015826b80 1 -- 172.21.15.33:0/358490207 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5600bdb3d3e0 msgr2=0x5600bdb37030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:47.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.038+0000 7f8015826b80 1 --2- 172.21.15.33:0/358490207 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5600bdb3d3e0 0x5600bdb37030 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7ff4015000 tx=0x7f7ff4005c00 comp rx=0 tx=0).stop 2024-05-03T18:27:47.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.038+0000 7f8008e00700 1 --2- 172.21.15.33:0/3600189309 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f7fe80b2b00 0x7f7fe80b4fc0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc009550 tx=0x7f7ffc004ad0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:27:47.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.038+0000 7f8015826b80 1 -- 172.21.15.33:0/358490207 shutdown_connections 2024-05-03T18:27:47.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.038+0000 7f8015826b80 1 --2- 172.21.15.33:0/358490207 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600bdb37650 0x5600bdb37ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:47.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.038+0000 7f8015826b80 1 --2- 172.21.15.33:0/358490207 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600bdb3cac0 0x5600bdb3cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:47.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.038+0000 7f8015826b80 1 --2- 172.21.15.33:0/358490207 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5600bdb3d3e0 0x5600bdb37030 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:47.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.038+0000 7f8015826b80 1 -- 172.21.15.33:0/358490207 >> 172.21.15.33:0/358490207 conn(0x5600bda65be0 msgr2=0x5600bdb31a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:27:47.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.039+0000 7f8015826b80 1 -- 172.21.15.33:0/358490207 shutdown_connections 2024-05-03T18:27:47.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.040+0000 7f8015826b80 1 -- 172.21.15.33:0/358490207 wait complete. 2024-05-03T18:27:47.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.040+0000 7f8015826b80 1 -- start start 2024-05-03T18:27:47.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.041+0000 7f8015826b80 1 Processor -- start 2024-05-03T18:27:47.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.042+0000 7f8015826b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600bdb37650 0x5600bdb43310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:47.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.042+0000 7f8015826b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5600bdb3cac0 0x5600bdb40610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:47.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.042+0000 7f8015826b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600bdb40b50 0x5600bdb40fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:47.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.042+0000 7f8015826b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5600bdb43df0 con 0x5600bdb40b50 2024-05-03T18:27:47.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.042+0000 7f8015826b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5600bd9efba0 con 0x5600bdb3cac0 2024-05-03T18:27:47.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.042+0000 7f8015826b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5600bdb43f70 con 0x5600bdb37650 2024-05-03T18:27:47.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.042+0000 7f8015826b80 10 client.0 _ll_register_callbacks cb 0x5600bda30cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:27:47.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.042+0000 7f8003400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5600bdb3cac0 0x5600bdb40610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:47.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.042+0000 7f8008e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600bdb40b50 0x5600bdb40fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:47.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.043+0000 7f8003400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5600bdb3cac0 0x5600bdb40610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:49704/0 (socket says 172.21.15.33:49704) 2024-05-03T18:27:47.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.043+0000 7f8003400700 1 -- 172.21.15.33:0/1735050695 learned_addr learned my addr 172.21.15.33:0/1735050695 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:27:47.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.043+0000 7f8003e00700 1 --2- 172.21.15.33:0/1735050695 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600bdb37650 0x5600bdb43310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:47.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.044+0000 7f8003400700 1 -- 172.21.15.33:0/1735050695 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600bdb37650 msgr2=0x5600bdb43310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:27:47.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.044+0000 7f8003400700 1 --2- 172.21.15.33:0/1735050695 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600bdb37650 0x5600bdb43310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:47.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.044+0000 7f8003400700 1 -- 172.21.15.33:0/1735050695 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600bdb40b50 msgr2=0x5600bdb40fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:27:47.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.044+0000 7f8003400700 1 --2- 172.21.15.33:0/1735050695 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600bdb40b50 0x5600bdb40fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:47.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.044+0000 7f8003400700 1 -- 172.21.15.33:0/1735050695 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600bda4f4a0 con 0x5600bdb3cac0 2024-05-03T18:27:47.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.044+0000 7f8003400700 1 --2- 172.21.15.33:0/1735050695 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5600bdb3cac0 0x5600bdb40610 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7ff000b2f0 tx=0x7f7ff00a8ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ada6f80ea36e09 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:27:47.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.044+0000 7f7fee000700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-03T18:27:47.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.045+0000 7f7fee000700 1 -- 172.21.15.33:0/1735050695 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff0020040 con 0x5600bdb3cac0 2024-05-03T18:27:47.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.045+0000 7f7fee000700 1 -- 172.21.15.33:0/1735050695 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f7ff00e22f0 con 0x5600bdb3cac0 2024-05-03T18:27:47.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.045+0000 7f8015826b80 1 -- 172.21.15.33:0/1735050695 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ff4005070 con 0x5600bdb3cac0 2024-05-03T18:27:47.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.045+0000 7f7fee000700 1 -- 172.21.15.33:0/1735050695 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f7ff00dfea0 con 0x5600bdb3cac0 2024-05-03T18:27:47.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.045+0000 7f8015826b80 10 client.35087 fetch_fsmap learned FSMap version 21 2024-05-03T18:27:47.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.045+0000 7f8015826b80 1 -- 172.21.15.33:0/1735050695 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5600bdb4ec50 con 0x5600bdb3cac0 2024-05-03T18:27:47.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.046+0000 7f7fee000700 1 -- 172.21.15.33:0/1735050695 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7ff4005070 con 0x5600bdb3cac0 2024-05-03T18:27:47.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.047+0000 7f7fee000700 1 -- 172.21.15.33:0/1735050695 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f7ff00a3be0 con 0x5600bdb3cac0 2024-05-03T18:27:47.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.047+0000 7f8015826b80 10 client.35087 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:27:47.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.047+0000 7f8015826b80 10 client.35087 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:27:47.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.047+0000 7f8015826b80 20 client.35087 populate_metadata read hostname 'smithi033' 2024-05-03T18:27:47.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.047+0000 7f8015826b80 1 --2- 172.21.15.33:0/1735050695 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5600bdb51290 0x5600bdb536c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:47.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.047+0000 7f8015826b80 4 client.35087 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:27:47.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.047+0000 7f8015826b80 1 -- 172.21.15.33:0/1735050695 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5600bdb04250 con 0x5600bdb51290 2024-05-03T18:27:47.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.047+0000 7f8003e00700 1 --2- 172.21.15.33:0/1735050695 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5600bdb51290 0x5600bdb536c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:47.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.049+0000 7f8003e00700 1 --2- 172.21.15.33:0/1735050695 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5600bdb51290 0x5600bdb536c0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c112689e3ae2055 server_cookie=61e3c8e55ba6edd9 in_seq=0 out_seq=0 2024-05-03T18:27:47.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.049+0000 7f7fee000700 10 client.35087 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:27:47.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.050+0000 7f7fee000700 1 -- 172.21.15.33:0/1735050695 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5600bdb04250 con 0x5600bdb51290 2024-05-03T18:27:47.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.050+0000 7f7fee000700 10 client.35087 handle_command_reply: tid=0 2024-05-03T18:27:47.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.051+0000 7f8015826b80 1 client.35087 shutdown 2024-05-03T18:27:47.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.051+0000 7f8015826b80 1 -- 172.21.15.33:0/1735050695 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5600bdb3cac0 msgr2=0x5600bdb40610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:47.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.051+0000 7f8015826b80 1 --2- 172.21.15.33:0/1735050695 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5600bdb3cac0 0x5600bdb40610 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7ff000b2f0 tx=0x7f7ff00a8ca0 comp rx=0 tx=0).stop 2024-05-03T18:27:47.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.051+0000 7f8015826b80 1 -- 172.21.15.33:0/1735050695 shutdown_connections 2024-05-03T18:27:47.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.051+0000 7f8015826b80 1 --2- 172.21.15.33:0/1735050695 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5600bdb51290 0x5600bdb536c0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:47.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.051+0000 7f8015826b80 1 --2- 172.21.15.33:0/1735050695 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600bdb40b50 0x5600bdb40fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:47.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.051+0000 7f8015826b80 1 --2- 172.21.15.33:0/1735050695 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600bdb37650 0x5600bdb43310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:47.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.051+0000 7f8015826b80 1 --2- 172.21.15.33:0/1735050695 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5600bdb3cac0 0x5600bdb40610 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:47.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.051+0000 7f7fee000700 0 client.35087 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:27:47.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.051+0000 7f8015826b80 1 -- 172.21.15.33:0/1735050695 >> 172.21.15.33:0/1735050695 conn(0x5600bda65be0 msgr2=0x5600bdb31100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:27:47.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.051+0000 7f8015826b80 1 -- 172.21.15.33:0/1735050695 shutdown_connections 2024-05-03T18:27:47.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.051+0000 7f8015826b80 1 -- 172.21.15.33:0/1735050695 wait complete. 2024-05-03T18:27:47.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.052+0000 7f8015826b80 20 client.35087 trim_cache size 0 max 16384 2024-05-03T18:27:47.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.079+0000 7f8015826b80 1 Processor -- start 2024-05-03T18:27:47.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.079+0000 7f8015826b80 1 -- start start 2024-05-03T18:27:47.080 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:46 smithi033 ceph-mon[28330]: pgmap v20: 129 pgs: 129 active+clean; 22 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 1.7 MiB/s wr, 23 op/s 2024-05-03T18:27:47.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.079+0000 7f8015826b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600bdb37650 0x5600bdb3adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:47.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.079+0000 7f8015826b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5600bdb3cac0 0x5600bdb3bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:47.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.079+0000 7f8015826b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600bdb40b50 0x5600bdb58f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:47.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.079+0000 7f8015826b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8004073ae0 con 0x5600bdb40b50 2024-05-03T18:27:47.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.079+0000 7f8015826b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f80041be7e0 con 0x5600bdb3cac0 2024-05-03T18:27:47.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.079+0000 7f8015826b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8004074150 con 0x5600bdb37650 2024-05-03T18:27:47.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.080+0000 7f8008e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600bdb40b50 0x5600bdb58f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:47.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.080+0000 7f8008e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600bdb40b50 0x5600bdb58f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39868/0 (socket says 172.21.15.33:39868) 2024-05-03T18:27:47.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.080+0000 7f8008e00700 1 -- 172.21.15.33:0/3955056752 learned_addr learned my addr 172.21.15.33:0/3955056752 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:27:47.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.080+0000 7f8008e00700 1 -- 172.21.15.33:0/3955056752 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600bdb37650 msgr2=0x5600bdb3adf0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-03T18:27:47.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.080+0000 7f8008e00700 1 --2- 172.21.15.33:0/3955056752 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600bdb37650 0x5600bdb3adf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:47.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.080+0000 7f8008e00700 1 -- 172.21.15.33:0/3955056752 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5600bdb3cac0 msgr2=0x5600bdb3bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:27:47.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.080+0000 7f8008e00700 1 --2- 172.21.15.33:0/3955056752 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5600bdb3cac0 0x5600bdb3bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:47.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.080+0000 7f8008e00700 1 -- 172.21.15.33:0/3955056752 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600bdb30cd0 con 0x5600bdb40b50 2024-05-03T18:27:47.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.081+0000 7f8008e00700 1 --2- 172.21.15.33:0/3955056752 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600bdb40b50 0x5600bdb58f90 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc017550 tx=0x7f7ffc016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de28db0a1fe924e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:27:47.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.081+0000 7f7ff8c00700 1 -- 172.21.15.33:0/3955056752 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ffc027070 con 0x5600bdb40b50 2024-05-03T18:27:47.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.081+0000 7f7ff8c00700 1 -- 172.21.15.33:0/3955056752 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7ffc01b030 con 0x5600bdb40b50 2024-05-03T18:27:47.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.081+0000 7f7ff8c00700 1 -- 172.21.15.33:0/3955056752 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ffc0166c0 con 0x5600bdb40b50 2024-05-03T18:27:47.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.082+0000 7f8015826b80 1 -- 172.21.15.33:0/3955056752 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600bdb40b50 msgr2=0x5600bdb58f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:47.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.082+0000 7f8015826b80 1 --2- 172.21.15.33:0/3955056752 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600bdb40b50 0x5600bdb58f90 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc017550 tx=0x7f7ffc016000 comp rx=0 tx=0).stop 2024-05-03T18:27:47.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.082+0000 7f8015826b80 1 -- 172.21.15.33:0/3955056752 shutdown_connections 2024-05-03T18:27:47.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.082+0000 7f8015826b80 1 --2- 172.21.15.33:0/3955056752 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600bdb40b50 0x5600bdb58f90 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:47.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.082+0000 7f8015826b80 1 --2- 172.21.15.33:0/3955056752 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600bdb37650 0x5600bdb3adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:47.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.082+0000 7f8015826b80 1 --2- 172.21.15.33:0/3955056752 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5600bdb3cac0 0x5600bdb3bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:47.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.082+0000 7f8015826b80 1 -- 172.21.15.33:0/3955056752 >> 172.21.15.33:0/3955056752 conn(0x5600bda65be0 msgr2=0x5600bdb3b7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:27:47.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.082+0000 7f8015826b80 1 -- 172.21.15.33:0/3955056752 shutdown_connections 2024-05-03T18:27:47.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.083+0000 7f8015826b80 1 -- 172.21.15.33:0/3955056752 wait complete. 2024-05-03T18:27:47.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.083+0000 7f8015826b80 1 -- start start 2024-05-03T18:27:47.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.083+0000 7f8015826b80 1 Processor -- start 2024-05-03T18:27:47.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.084+0000 7f8015826b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600bdb37650 0x5600bdb461d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:47.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.084+0000 7f8015826b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5600bdb3cac0 0x5600bdb46710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:47.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.084+0000 7f8015826b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600bdb40b50 0x5600bdb46c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:47.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.084+0000 7f8015826b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8004074820 con 0x5600bdb37650 2024-05-03T18:27:47.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.084+0000 7f8015826b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f80041be110 con 0x5600bdb3cac0 2024-05-03T18:27:47.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.084+0000 7f8015826b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8004074ef0 con 0x5600bdb40b50 2024-05-03T18:27:47.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.084+0000 7f8015826b80 10 client.0 _ll_register_callbacks cb 0x5600bda30cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:27:47.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.085+0000 7f8003e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600bdb37650 0x5600bdb461d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:47.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.085+0000 7f8003400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5600bdb3cac0 0x5600bdb46710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:47.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.085+0000 7f8003400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5600bdb3cac0 0x5600bdb46710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:49714/0 (socket says 172.21.15.33:49714) 2024-05-03T18:27:47.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.085+0000 7f8008e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600bdb40b50 0x5600bdb46c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:47.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.085+0000 7f8003400700 1 -- 172.21.15.33:0/2811590788 learned_addr learned my addr 172.21.15.33:0/2811590788 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:27:47.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.085+0000 7f8003e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600bdb37650 0x5600bdb461d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39878/0 (socket says 172.21.15.33:39878) 2024-05-03T18:27:47.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.086+0000 7f8003e00700 1 -- 172.21.15.33:0/2811590788 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600bdb40b50 msgr2=0x5600bdb46c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:27:47.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.086+0000 7f8003e00700 1 --2- 172.21.15.33:0/2811590788 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600bdb40b50 0x5600bdb46c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:47.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.086+0000 7f8003e00700 1 -- 172.21.15.33:0/2811590788 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5600bdb3cac0 msgr2=0x5600bdb46710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:27:47.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.086+0000 7f8003e00700 1 --2- 172.21.15.33:0/2811590788 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5600bdb3cac0 0x5600bdb46710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:47.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.086+0000 7f8003e00700 1 -- 172.21.15.33:0/2811590788 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600bdbbbb80 con 0x5600bdb37650 2024-05-03T18:27:47.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.086+0000 7f8003400700 1 --2- 172.21.15.33:0/2811590788 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5600bdb3cac0 0x5600bdb46710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:27:47.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.087+0000 7f8003e00700 1 --2- 172.21.15.33:0/2811590788 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600bdb37650 0x5600bdb461d0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f7ff4004000 tx=0x7f7ff400bcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7515aa9beec4e6f server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:27:47.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.087+0000 7f7fee000700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:27:47.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.087+0000 7f7fee000700 1 -- 172.21.15.33:0/2811590788 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff4026040 con 0x5600bdb37650 2024-05-03T18:27:47.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.087+0000 7f7fee000700 1 -- 172.21.15.33:0/2811590788 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f7ff40050a0 con 0x5600bdb37650 2024-05-03T18:27:47.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.087+0000 7f8015826b80 1 -- 172.21.15.33:0/2811590788 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ffc01b030 con 0x5600bdb37650 2024-05-03T18:27:47.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.088+0000 7f7fee000700 1 -- 172.21.15.33:0/2811590788 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f7ff40095b0 con 0x5600bdb37650 2024-05-03T18:27:47.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.088+0000 7f8015826b80 10 client.15363 fetch_fsmap learned FSMap version 21 2024-05-03T18:27:47.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.088+0000 7f8015826b80 1 -- 172.21.15.33:0/2811590788 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5600bdbbce80 con 0x5600bdb37650 2024-05-03T18:27:47.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.088+0000 7f7fee000700 1 -- 172.21.15.33:0/2811590788 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7ffc01b030 con 0x5600bdb37650 2024-05-03T18:27:47.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.088+0000 7f7fee000700 1 -- 172.21.15.33:0/2811590788 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f7ff402c040 con 0x5600bdb37650 2024-05-03T18:27:47.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.088+0000 7f8015826b80 10 client.15363 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:27:47.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.088+0000 7f8015826b80 10 client.15363 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:27:47.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.089+0000 7f8015826b80 20 client.15363 populate_metadata read hostname 'smithi033' 2024-05-03T18:27:47.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.089+0000 7f8015826b80 1 --2- 172.21.15.33:0/2811590788 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5600bdbbd0b0 0x5600bdbbf4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:47.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.089+0000 7f8015826b80 4 client.15363 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-03T18:27:47.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.089+0000 7f8015826b80 1 -- 172.21.15.33:0/2811590788 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5600bda97480 con 0x5600bdbbd0b0 2024-05-03T18:27:47.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.089+0000 7f8003400700 1 --2- 172.21.15.33:0/2811590788 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5600bdbbd0b0 0x5600bdbbf4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:47.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.090+0000 7f8003400700 1 --2- 172.21.15.33:0/2811590788 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5600bdbbd0b0 0x5600bdbbf4a0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffd314ef76be7447 server_cookie=f0c00a4cdccc2a58 in_seq=0 out_seq=0 2024-05-03T18:27:47.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.090+0000 7f7fee000700 10 client.15363 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:27:47.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.093+0000 7f7fee000700 1 -- 172.21.15.33:0/2811590788 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5600bda97480 con 0x5600bdbbd0b0 2024-05-03T18:27:47.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.093+0000 7f7fee000700 10 client.15363 handle_command_reply: tid=0 2024-05-03T18:27:47.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.093+0000 7f8015826b80 1 client.15363 shutdown 2024-05-03T18:27:47.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.093+0000 7f8015826b80 1 -- 172.21.15.33:0/2811590788 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600bdb37650 msgr2=0x5600bdb461d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:47.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.093+0000 7f8015826b80 1 --2- 172.21.15.33:0/2811590788 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600bdb37650 0x5600bdb461d0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f7ff4004000 tx=0x7f7ff400bcf0 comp rx=0 tx=0).stop 2024-05-03T18:27:47.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.093+0000 7f8015826b80 1 -- 172.21.15.33:0/2811590788 shutdown_connections 2024-05-03T18:27:47.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.093+0000 7f8015826b80 1 --2- 172.21.15.33:0/2811590788 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5600bdbbd0b0 0x5600bdbbf4a0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:47.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.093+0000 7f8015826b80 1 --2- 172.21.15.33:0/2811590788 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5600bdb40b50 0x5600bdb46c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:47.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.093+0000 7f8015826b80 1 --2- 172.21.15.33:0/2811590788 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600bdb37650 0x5600bdb461d0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:47.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.093+0000 7f8015826b80 1 --2- 172.21.15.33:0/2811590788 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5600bdb3cac0 0x5600bdb46710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:47.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.093+0000 7f8015826b80 1 -- 172.21.15.33:0/2811590788 >> 172.21.15.33:0/2811590788 conn(0x5600bda65be0 msgr2=0x5600bdb3d7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:27:47.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.093+0000 7f7fee000700 0 client.15363 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:27:47.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.093+0000 7f8015826b80 1 -- 172.21.15.33:0/2811590788 shutdown_connections 2024-05-03T18:27:47.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.094+0000 7f8015826b80 1 -- 172.21.15.33:0/2811590788 wait complete. 2024-05-03T18:27:47.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.094+0000 7f8015826b80 20 client.15363 trim_cache size 0 max 16384 2024-05-03T18:27:47.096 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-03T18:27:47.096 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-05-03T18:27:47.096 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "86228b22-942c-4282-a2a1-1c2d6696bb7f", 2024-05-03T18:27:47.096 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-05-03T18:27:47.096 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-03T18:27:47.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.095+0000 7f8009800700 1 -- 172.21.15.33:0/3600189309 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f7fe80b2b00 msgr2=0x7f7fe80b4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:47.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.095+0000 7f8009800700 1 --2- 172.21.15.33:0/3600189309 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f7fe80b2b00 0x7f7fe80b4fc0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc009550 tx=0x7f7ffc004ad0 comp rx=0 tx=0).stop 2024-05-03T18:27:47.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.095+0000 7f8009800700 1 -- 172.21.15.33:0/3600189309 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f800410a3a0 msgr2=0x7f80041a0500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:47.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.095+0000 7f8009800700 1 --2- 172.21.15.33:0/3600189309 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f800410a3a0 0x7f80041a0500 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f7ff000d950 tx=0x7f7ff000dd10 comp rx=0 tx=0).stop 2024-05-03T18:27:47.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.095+0000 7f8009800700 1 -- 172.21.15.33:0/3600189309 shutdown_connections 2024-05-03T18:27:47.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.095+0000 7f8009800700 1 --2- 172.21.15.33:0/3600189309 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f7fe80b2b00 0x7f7fe80b4fc0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:47.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.095+0000 7f8009800700 1 --2- 172.21.15.33:0/3600189309 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f800410f9d0 0x7f80041a4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:47.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.095+0000 7f8009800700 1 --2- 172.21.15.33:0/3600189309 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f80041099e0 0x7f800419ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:47.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.095+0000 7f8009800700 1 --2- 172.21.15.33:0/3600189309 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f800410a3a0 0x7f80041a0500 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:47.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.095+0000 7f8009800700 1 -- 172.21.15.33:0/3600189309 >> 172.21.15.33:0/3600189309 conn(0x7f8004079200 msgr2=0x7f8004079820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:27:47.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.095+0000 7f8009800700 1 -- 172.21.15.33:0/3600189309 shutdown_connections 2024-05-03T18:27:47.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:47.096+0000 7f8009800700 1 -- 172.21.15.33:0/3600189309 wait complete. 2024-05-03T18:27:47.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:46 smithi150 ceph-mon[32492]: pgmap v20: 129 pgs: 129 active+clean; 22 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 1.7 MiB/s wr, 23 op/s 2024-05-03T18:27:47.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:46 smithi099 ceph-mon[31658]: pgmap v20: 129 pgs: 129 active+clean; 22 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 1.7 MiB/s wr, 23 op/s 2024-05-03T18:27:47.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:47 smithi033 ceph-mon[28330]: scrub queued for path: / 2024-05-03T18:27:47.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:47 smithi033 ceph-mon[28330]: scrub summary: idle+waiting paths [/] 2024-05-03T18:27:47.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:47 smithi033 ceph-mon[28330]: scrub summary: active paths [/] 2024-05-03T18:27:48.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:47 smithi150 ceph-mon[32492]: scrub queued for path: / 2024-05-03T18:27:48.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:47 smithi150 ceph-mon[32492]: scrub summary: idle+waiting paths [/] 2024-05-03T18:27:48.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:47 smithi150 ceph-mon[32492]: scrub summary: active paths [/] 2024-05-03T18:27:48.262 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-03T18:27:48.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:47 smithi099 ceph-mon[31658]: scrub queued for path: / 2024-05-03T18:27:48.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:47 smithi099 ceph-mon[31658]: scrub summary: idle+waiting paths [/] 2024-05-03T18:27:48.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:47 smithi099 ceph-mon[31658]: scrub summary: active paths [/] 2024-05-03T18:27:48.773 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:27:49.163 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:48 smithi033 ceph-mon[28330]: scrub summary: idle+waiting paths [/] 2024-05-03T18:27:49.163 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:48 smithi033 ceph-mon[28330]: pgmap v21: 129 pgs: 129 active+clean; 23 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 2.0 MiB/s wr, 29 op/s 2024-05-03T18:27:49.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:48 smithi099 ceph-mon[31658]: scrub summary: idle+waiting paths [/] 2024-05-03T18:27:49.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:48 smithi099 ceph-mon[31658]: pgmap v21: 129 pgs: 129 active+clean; 23 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 2.0 MiB/s wr, 29 op/s 2024-05-03T18:27:49.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:48 smithi150 ceph-mon[32492]: scrub summary: idle+waiting paths [/] 2024-05-03T18:27:49.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:48 smithi150 ceph-mon[32492]: pgmap v21: 129 pgs: 129 active+clean; 23 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 2.0 MiB/s wr, 29 op/s 2024-05-03T18:27:50.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:50 smithi033 ceph-mon[28330]: pgmap v22: 129 pgs: 129 active+clean; 26 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 1.9 MiB/s wr, 43 op/s 2024-05-03T18:27:51.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.262+0000 7fc73a000700 1 -- 172.21.15.33:0/1310370757 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc734109510 msgr2=0x7fc7341098f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:51.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.262+0000 7fc73a000700 1 --2- 172.21.15.33:0/1310370757 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc734109510 0x7fc7341098f0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fc724009990 tx=0x7fc724009ca0 comp rx=0 tx=0).stop 2024-05-03T18:27:51.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.262+0000 7fc73a000700 1 -- 172.21.15.33:0/1310370757 shutdown_connections 2024-05-03T18:27:51.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.262+0000 7fc73a000700 1 --2- 172.21.15.33:0/1310370757 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc73406bb40 0x7fc734110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:51.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.262+0000 7fc73a000700 1 --2- 172.21.15.33:0/1310370757 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc734109510 0x7fc7341098f0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:51.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.262+0000 7fc73a000700 1 --2- 172.21.15.33:0/1310370757 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc73406b180 0x7fc73406b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:51.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.262+0000 7fc73a000700 1 -- 172.21.15.33:0/1310370757 >> 172.21.15.33:0/1310370757 conn(0x7fc734079350 msgr2=0x7fc734079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:27:51.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.262+0000 7fc73a000700 1 -- 172.21.15.33:0/1310370757 shutdown_connections 2024-05-03T18:27:51.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.262+0000 7fc73a000700 1 -- 172.21.15.33:0/1310370757 wait complete. 2024-05-03T18:27:51.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.264+0000 7fc73a000700 1 Processor -- start 2024-05-03T18:27:51.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.264+0000 7fc73a000700 1 -- start start 2024-05-03T18:27:51.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.265+0000 7fc73a000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc73406b180 0x7fc73419f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:51.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.265+0000 7fc73a000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc73406bb40 0x7fc73419fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:51.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.265+0000 7fc73a000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc734109510 0x7fc734199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:51.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.265+0000 7fc73a000700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc7341134f0 con 0x7fc73406bb40 2024-05-03T18:27:51.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.265+0000 7fc73a000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc734113370 con 0x7fc734109510 2024-05-03T18:27:51.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.265+0000 7fc73a000700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc734113670 con 0x7fc73406b180 2024-05-03T18:27:51.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.265+0000 7fc72aa00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc73406bb40 0x7fc73419fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:51.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.265+0000 7fc72aa00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc73406bb40 0x7fc73419fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39900/0 (socket says 172.21.15.33:39900) 2024-05-03T18:27:51.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.265+0000 7fc72aa00700 1 -- 172.21.15.33:0/1082981719 learned_addr learned my addr 172.21.15.33:0/1082981719 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:27:51.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.266+0000 7fc733400700 1 --2- 172.21.15.33:0/1082981719 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc73406b180 0x7fc73419f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:51.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.266+0000 7fc733e00700 1 --2- 172.21.15.33:0/1082981719 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc734109510 0x7fc734199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:51.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.266+0000 7fc72aa00700 1 -- 172.21.15.33:0/1082981719 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc73406b180 msgr2=0x7fc73419f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:51.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.266+0000 7fc72aa00700 1 --2- 172.21.15.33:0/1082981719 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc73406b180 0x7fc73419f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:51.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.266+0000 7fc72aa00700 1 -- 172.21.15.33:0/1082981719 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc734109510 msgr2=0x7fc734199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:51.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.266+0000 7fc72aa00700 1 --2- 172.21.15.33:0/1082981719 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc734109510 0x7fc734199830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:51.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.266+0000 7fc72aa00700 1 -- 172.21.15.33:0/1082981719 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc73419a010 con 0x7fc73406bb40 2024-05-03T18:27:51.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.267+0000 7fc72aa00700 1 --2- 172.21.15.33:0/1082981719 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc73406bb40 0x7fc73419fbf0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fc71c00d980 tx=0x7fc71c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:27:51.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.267+0000 7fc730c00700 1 -- 172.21.15.33:0/1082981719 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc71c017070 con 0x7fc73406bb40 2024-05-03T18:27:51.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.267+0000 7fc730c00700 1 -- 172.21.15.33:0/1082981719 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc71c005020 con 0x7fc73406bb40 2024-05-03T18:27:51.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.267+0000 7fc730c00700 1 -- 172.21.15.33:0/1082981719 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc71c009ab0 con 0x7fc73406bb40 2024-05-03T18:27:51.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.267+0000 7fc73a000700 1 -- 172.21.15.33:0/1082981719 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc73419a300 con 0x7fc73406bb40 2024-05-03T18:27:51.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.267+0000 7fc73a000700 1 -- 172.21.15.33:0/1082981719 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7341a6510 con 0x7fc73406bb40 2024-05-03T18:27:51.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.273+0000 7fc730c00700 1 -- 172.21.15.33:0/1082981719 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 23) ==== 149828+0+0 (secure 0 0 0) 0x7fc71c0051d0 con 0x7fc73406bb40 2024-05-03T18:27:51.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.273+0000 7fc7448cbb80 1 Processor -- start 2024-05-03T18:27:51.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.274+0000 7fc7448cbb80 1 -- start start 2024-05-03T18:27:51.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.274+0000 7fc7448cbb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599e097ad40 0x5599e097b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:51.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.274+0000 7fc7448cbb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5599e097b660 0x5599e0975340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:51.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.274+0000 7fc7448cbb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599e0975960 0x5599e0975de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:51.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.274+0000 7fc7448cbb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5599e08c3a20 con 0x5599e0975960 2024-05-03T18:27:51.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.274+0000 7fc7448cbb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5599e0861760 con 0x5599e097ad40 2024-05-03T18:27:51.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.274+0000 7fc7448cbb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5599e08bf9e0 con 0x5599e097b660 2024-05-03T18:27:51.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.275+0000 7fc72aa00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599e0975960 0x5599e0975de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:51.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.275+0000 7fc733e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599e097ad40 0x5599e097b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:51.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.275+0000 7fc733400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5599e097b660 0x5599e0975340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:51.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.275+0000 7fc72aa00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599e0975960 0x5599e0975de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39910/0 (socket says 172.21.15.33:39910) 2024-05-03T18:27:51.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.275+0000 7fc72aa00700 1 -- 172.21.15.33:0/1596826876 learned_addr learned my addr 172.21.15.33:0/1596826876 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:27:51.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.275+0000 7fc72aa00700 1 -- 172.21.15.33:0/1596826876 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5599e097b660 msgr2=0x5599e0975340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:27:51.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.275+0000 7fc72aa00700 1 --2- 172.21.15.33:0/1596826876 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5599e097b660 0x5599e0975340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:51.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.275+0000 7fc72aa00700 1 -- 172.21.15.33:0/1596826876 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599e097ad40 msgr2=0x5599e097b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:27:51.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.275+0000 7fc72aa00700 1 --2- 172.21.15.33:0/1596826876 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599e097ad40 0x5599e097b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:51.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.275+0000 7fc72aa00700 1 -- 172.21.15.33:0/1596826876 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599e08108d0 con 0x5599e0975960 2024-05-03T18:27:51.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.275+0000 7fc730c00700 1 --2- 172.21.15.33:0/1082981719 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fc7180b2b00 0x7fc7180b4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:51.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.276+0000 7fc730c00700 1 -- 172.21.15.33:0/1082981719 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7fc71c0e29d0 con 0x7fc73406bb40 2024-05-03T18:27:51.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.276+0000 7fc72aa00700 1 --2- 172.21.15.33:0/1596826876 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599e0975960 0x5599e0975de0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fc71c0dfc70 tx=0x7fc71c0e6db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d10914ca924708f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:27:51.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.276+0000 7fc733e00700 1 --2- 172.21.15.33:0/1082981719 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fc7180b2b00 0x7fc7180b4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:51.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.276+0000 7fc728c00700 1 -- 172.21.15.33:0/1596826876 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc71c0a5670 con 0x5599e0975960 2024-05-03T18:27:51.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.276+0000 7fc728c00700 1 -- 172.21.15.33:0/1596826876 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc71c0e2e50 con 0x5599e0975960 2024-05-03T18:27:51.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.276+0000 7fc728c00700 1 -- 172.21.15.33:0/1596826876 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc71c0a8f20 con 0x5599e0975960 2024-05-03T18:27:51.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.277+0000 7fc7448cbb80 1 -- 172.21.15.33:0/1596826876 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599e0975960 msgr2=0x5599e0975de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:51.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.277+0000 7fc733e00700 1 --2- 172.21.15.33:0/1082981719 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fc7180b2b00 0x7fc7180b4fc0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc72c011440 tx=0x7fc72c009040 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:27:51.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.277+0000 7fc7448cbb80 1 --2- 172.21.15.33:0/1596826876 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599e0975960 0x5599e0975de0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fc71c0dfc70 tx=0x7fc71c0e6db0 comp rx=0 tx=0).stop 2024-05-03T18:27:51.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.277+0000 7fc7448cbb80 1 -- 172.21.15.33:0/1596826876 shutdown_connections 2024-05-03T18:27:51.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.277+0000 7fc7448cbb80 1 --2- 172.21.15.33:0/1596826876 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599e0975960 0x5599e0975de0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:51.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.277+0000 7fc7448cbb80 1 --2- 172.21.15.33:0/1596826876 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599e097ad40 0x5599e097b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:51.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.277+0000 7fc7448cbb80 1 --2- 172.21.15.33:0/1596826876 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5599e097b660 0x5599e0975340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:51.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.277+0000 7fc7448cbb80 1 -- 172.21.15.33:0/1596826876 >> 172.21.15.33:0/1596826876 conn(0x5599e08e96d0 msgr2=0x5599e096b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:27:51.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.278+0000 7fc7448cbb80 1 -- 172.21.15.33:0/1596826876 shutdown_connections 2024-05-03T18:27:51.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.278+0000 7fc7448cbb80 1 -- 172.21.15.33:0/1596826876 wait complete. 2024-05-03T18:27:51.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.278+0000 7fc7448cbb80 1 -- start start 2024-05-03T18:27:51.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.278+0000 7fc7448cbb80 1 Processor -- start 2024-05-03T18:27:51.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.279+0000 7fc7448cbb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5599e0975960 0x5599e09812e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:51.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.279+0000 7fc7448cbb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599e097ad40 0x5599e097e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:51.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.279+0000 7fc7448cbb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599e097b660 0x5599e097ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:51.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.279+0000 7fc7448cbb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5599e0982110 con 0x5599e097b660 2024-05-03T18:27:51.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.279+0000 7fc7448cbb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5599e0829b20 con 0x5599e097ad40 2024-05-03T18:27:51.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.279+0000 7fc7448cbb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5599e0982290 con 0x5599e0975960 2024-05-03T18:27:51.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.279+0000 7fc7448cbb80 10 client.0 _ll_register_callbacks cb 0x5599e086ac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:27:51.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.279+0000 7fc733e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599e097b660 0x5599e097ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:51.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.280+0000 7fc733400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5599e0975960 0x5599e09812e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:51.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.280+0000 7fc733e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599e097b660 0x5599e097ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39918/0 (socket says 172.21.15.33:39918) 2024-05-03T18:27:51.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.280+0000 7fc733e00700 1 -- 172.21.15.33:0/3427462161 learned_addr learned my addr 172.21.15.33:0/3427462161 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:27:51.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.280+0000 7fc72aa00700 1 --2- 172.21.15.33:0/3427462161 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599e097ad40 0x5599e097e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:51.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.280+0000 7fc733e00700 1 -- 172.21.15.33:0/3427462161 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5599e0975960 msgr2=0x5599e09812e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:27:51.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.280+0000 7fc733e00700 1 --2- 172.21.15.33:0/3427462161 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5599e0975960 0x5599e09812e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:51.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.280+0000 7fc733e00700 1 -- 172.21.15.33:0/3427462161 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599e097ad40 msgr2=0x5599e097e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:27:51.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.280+0000 7fc733e00700 1 --2- 172.21.15.33:0/3427462161 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599e097ad40 0x5599e097e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:51.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.280+0000 7fc733e00700 1 -- 172.21.15.33:0/3427462161 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599e0889420 con 0x5599e097b660 2024-05-03T18:27:51.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.282+0000 7fc733e00700 1 --2- 172.21.15.33:0/3427462161 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599e097b660 0x5599e097ebc0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fc72c017e50 tx=0x7fc72c004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=419f22f2e6e37b81 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:27:51.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.282+0000 7fc721600700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:27:51.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.282+0000 7fc721600700 1 -- 172.21.15.33:0/3427462161 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc72c01d050 con 0x5599e097b660 2024-05-03T18:27:51.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.282+0000 7fc721600700 1 -- 172.21.15.33:0/3427462161 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7fc72c015c50 con 0x5599e097b660 2024-05-03T18:27:51.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.282+0000 7fc7448cbb80 1 -- 172.21.15.33:0/3427462161 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc71c0e2e50 con 0x5599e097b660 2024-05-03T18:27:51.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.283+0000 7fc721600700 1 -- 172.21.15.33:0/3427462161 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fc72c030050 con 0x5599e097b660 2024-05-03T18:27:51.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.283+0000 7fc7448cbb80 10 client.15375 fetch_fsmap learned FSMap version 21 2024-05-03T18:27:51.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.283+0000 7fc7448cbb80 1 -- 172.21.15.33:0/3427462161 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5599e098c8b0 con 0x5599e097b660 2024-05-03T18:27:51.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.283+0000 7fc721600700 1 -- 172.21.15.33:0/3427462161 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc71c0e2e50 con 0x5599e097b660 2024-05-03T18:27:51.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.283+0000 7fc721600700 1 -- 172.21.15.33:0/3427462161 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fc72c0143d0 con 0x5599e097b660 2024-05-03T18:27:51.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.284+0000 7fc7448cbb80 10 client.15375 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:27:51.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.284+0000 7fc7448cbb80 10 client.15375 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:27:51.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.284+0000 7fc7448cbb80 20 client.15375 populate_metadata read hostname 'smithi033' 2024-05-03T18:27:51.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.284+0000 7fc7448cbb80 1 --2- 172.21.15.33:0/3427462161 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5599e098eef0 0x5599e0991330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:51.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.284+0000 7fc7448cbb80 4 client.15375 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:27:51.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.284+0000 7fc7448cbb80 1 -- 172.21.15.33:0/3427462161 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5599e093dff0 con 0x5599e098eef0 2024-05-03T18:27:51.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.285+0000 7fc733400700 1 --2- 172.21.15.33:0/3427462161 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5599e098eef0 0x5599e0991330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:51.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.286+0000 7fc733400700 1 --2- 172.21.15.33:0/3427462161 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5599e098eef0 0x5599e0991330 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb6410055f234617 server_cookie=609b2d53d50e7ff9 in_seq=0 out_seq=0 2024-05-03T18:27:51.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.286+0000 7fc721600700 10 client.15375 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:27:51.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:50 smithi099 ceph-mon[31658]: pgmap v22: 129 pgs: 129 active+clean; 26 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 1.9 MiB/s wr, 43 op/s 2024-05-03T18:27:51.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.300+0000 7fc721600700 1 -- 172.21.15.33:0/3427462161 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5599e093dff0 con 0x5599e098eef0 2024-05-03T18:27:51.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.300+0000 7fc721600700 10 client.15375 handle_command_reply: tid=0 2024-05-03T18:27:51.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.300+0000 7fc7448cbb80 1 client.15375 shutdown 2024-05-03T18:27:51.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.300+0000 7fc7448cbb80 1 -- 172.21.15.33:0/3427462161 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599e097b660 msgr2=0x5599e097ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:51.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.300+0000 7fc7448cbb80 1 --2- 172.21.15.33:0/3427462161 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599e097b660 0x5599e097ebc0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fc72c017e50 tx=0x7fc72c004b40 comp rx=0 tx=0).stop 2024-05-03T18:27:51.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.300+0000 7fc7448cbb80 1 -- 172.21.15.33:0/3427462161 shutdown_connections 2024-05-03T18:27:51.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.300+0000 7fc7448cbb80 1 --2- 172.21.15.33:0/3427462161 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5599e098eef0 0x5599e0991330 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:51.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.300+0000 7fc7448cbb80 1 --2- 172.21.15.33:0/3427462161 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599e097b660 0x5599e097ebc0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:51.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.300+0000 7fc7448cbb80 1 --2- 172.21.15.33:0/3427462161 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5599e0975960 0x5599e09812e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:51.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.300+0000 7fc7448cbb80 1 --2- 172.21.15.33:0/3427462161 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599e097ad40 0x5599e097e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:51.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.300+0000 7fc721600700 0 client.15375 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:27:51.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.300+0000 7fc7448cbb80 1 -- 172.21.15.33:0/3427462161 >> 172.21.15.33:0/3427462161 conn(0x5599e08e96d0 msgr2=0x5599e09663b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:27:51.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.300+0000 7fc7448cbb80 1 -- 172.21.15.33:0/3427462161 shutdown_connections 2024-05-03T18:27:51.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.300+0000 7fc7448cbb80 1 -- 172.21.15.33:0/3427462161 wait complete. 2024-05-03T18:27:51.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.300+0000 7fc7448cbb80 20 client.15375 trim_cache size 0 max 16384 2024-05-03T18:27:51.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.328+0000 7fc7448cbb80 1 Processor -- start 2024-05-03T18:27:51.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.328+0000 7fc7448cbb80 1 -- start start 2024-05-03T18:27:51.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.329+0000 7fc7448cbb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5599e0975960 0x5599e098b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:51.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.329+0000 7fc7448cbb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599e097ad40 0x5599e0975340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:51.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.329+0000 7fc7448cbb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599e097b660 0x5599e0996c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:51.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.329+0000 7fc7448cbb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc7341ab220 con 0x5599e097b660 2024-05-03T18:27:51.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.329+0000 7fc7448cbb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc7341ada80 con 0x5599e097ad40 2024-05-03T18:27:51.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.329+0000 7fc7448cbb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc7341ab870 con 0x5599e0975960 2024-05-03T18:27:51.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.329+0000 7fc733e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599e097b660 0x5599e0996c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:51.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.329+0000 7fc733e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599e097b660 0x5599e0996c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39924/0 (socket says 172.21.15.33:39924) 2024-05-03T18:27:51.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.329+0000 7fc733e00700 1 -- 172.21.15.33:0/347959585 learned_addr learned my addr 172.21.15.33:0/347959585 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:27:51.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.329+0000 7fc733400700 1 --2- 172.21.15.33:0/347959585 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5599e0975960 0x5599e098b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:51.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.329+0000 7fc72aa00700 1 --2- 172.21.15.33:0/347959585 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599e097ad40 0x5599e0975340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:51.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.330+0000 7fc733e00700 1 -- 172.21.15.33:0/347959585 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5599e0975960 msgr2=0x5599e098b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:27:51.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.330+0000 7fc733e00700 1 --2- 172.21.15.33:0/347959585 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5599e0975960 0x5599e098b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:51.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.330+0000 7fc733e00700 1 -- 172.21.15.33:0/347959585 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599e097ad40 msgr2=0x5599e0975340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:27:51.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.330+0000 7fc733e00700 1 --2- 172.21.15.33:0/347959585 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599e097ad40 0x5599e0975340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:51.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.330+0000 7fc733e00700 1 -- 172.21.15.33:0/347959585 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599e0963e60 con 0x5599e097b660 2024-05-03T18:27:51.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.330+0000 7fc733e00700 1 --2- 172.21.15.33:0/347959585 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599e097b660 0x5599e0996c00 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fc72c012010 tx=0x7fc72c007f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a630d74388487bac server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:27:51.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.331+0000 7fc723e00700 1 -- 172.21.15.33:0/347959585 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc72c01d040 con 0x5599e097b660 2024-05-03T18:27:51.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.331+0000 7fc723e00700 1 -- 172.21.15.33:0/347959585 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc72c027070 con 0x5599e097b660 2024-05-03T18:27:51.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.331+0000 7fc723e00700 1 -- 172.21.15.33:0/347959585 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc72c015e60 con 0x5599e097b660 2024-05-03T18:27:51.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.332+0000 7fc7448cbb80 1 -- 172.21.15.33:0/347959585 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599e097b660 msgr2=0x5599e0996c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:51.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.332+0000 7fc7448cbb80 1 --2- 172.21.15.33:0/347959585 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599e097b660 0x5599e0996c00 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fc72c012010 tx=0x7fc72c007f10 comp rx=0 tx=0).stop 2024-05-03T18:27:51.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.332+0000 7fc7448cbb80 1 -- 172.21.15.33:0/347959585 shutdown_connections 2024-05-03T18:27:51.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.332+0000 7fc7448cbb80 1 --2- 172.21.15.33:0/347959585 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599e097b660 0x5599e0996c00 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:51.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.332+0000 7fc7448cbb80 1 --2- 172.21.15.33:0/347959585 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5599e0975960 0x5599e098b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:51.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.332+0000 7fc7448cbb80 1 --2- 172.21.15.33:0/347959585 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599e097ad40 0x5599e0975340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:51.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.332+0000 7fc7448cbb80 1 -- 172.21.15.33:0/347959585 >> 172.21.15.33:0/347959585 conn(0x5599e08e96d0 msgr2=0x5599e097cf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:27:51.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.333+0000 7fc7448cbb80 1 -- 172.21.15.33:0/347959585 shutdown_connections 2024-05-03T18:27:51.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.333+0000 7fc7448cbb80 1 -- 172.21.15.33:0/347959585 wait complete. 2024-05-03T18:27:51.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.333+0000 7fc7448cbb80 1 -- start start 2024-05-03T18:27:51.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.334+0000 7fc7448cbb80 1 Processor -- start 2024-05-03T18:27:51.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.334+0000 7fc7448cbb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5599e0975960 0x5599e0987520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:51.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.334+0000 7fc7448cbb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599e097ad40 0x5599e0987a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:51.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.334+0000 7fc7448cbb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599e097b660 0x5599e096ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:51.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.334+0000 7fc7448cbb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc7341abf40 con 0x5599e097b660 2024-05-03T18:27:51.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.334+0000 7fc7448cbb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc7341ad3b0 con 0x5599e097ad40 2024-05-03T18:27:51.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.334+0000 7fc7448cbb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc7341ac610 con 0x5599e0975960 2024-05-03T18:27:51.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.335+0000 7fc7448cbb80 10 client.0 _ll_register_callbacks cb 0x5599e086ac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:27:51.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.335+0000 7fc733e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599e097b660 0x5599e096ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:51.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.335+0000 7fc733400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5599e0975960 0x5599e0987520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:51.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.335+0000 7fc733e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599e097b660 0x5599e096ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39936/0 (socket says 172.21.15.33:39936) 2024-05-03T18:27:51.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.335+0000 7fc733e00700 1 -- 172.21.15.33:0/3294942473 learned_addr learned my addr 172.21.15.33:0/3294942473 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:27:51.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.335+0000 7fc72aa00700 1 --2- 172.21.15.33:0/3294942473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599e097ad40 0x5599e0987a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:51.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.335+0000 7fc733e00700 1 -- 172.21.15.33:0/3294942473 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5599e0975960 msgr2=0x5599e0987520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:27:51.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.335+0000 7fc733e00700 1 --2- 172.21.15.33:0/3294942473 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5599e0975960 0x5599e0987520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:51.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.335+0000 7fc733e00700 1 -- 172.21.15.33:0/3294942473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599e097ad40 msgr2=0x5599e0987a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:27:51.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.335+0000 7fc733e00700 1 --2- 172.21.15.33:0/3294942473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599e097ad40 0x5599e0987a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:51.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.336+0000 7fc733e00700 1 -- 172.21.15.33:0/3294942473 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599e0970460 con 0x5599e097b660 2024-05-03T18:27:51.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.336+0000 7fc733e00700 1 --2- 172.21.15.33:0/3294942473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599e097b660 0x5599e096ff20 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fc72c019010 tx=0x7fc72c0101d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af8ba847b57dfc6c server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:27:51.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.336+0000 7fc721600700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:27:51.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.336+0000 7fc721600700 1 -- 172.21.15.33:0/3294942473 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc72c01d070 con 0x5599e097b660 2024-05-03T18:27:51.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.336+0000 7fc721600700 1 -- 172.21.15.33:0/3294942473 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7fc72c005e00 con 0x5599e097b660 2024-05-03T18:27:51.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.336+0000 7fc7448cbb80 1 -- 172.21.15.33:0/3294942473 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc72c027070 con 0x5599e097b660 2024-05-03T18:27:51.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.337+0000 7fc721600700 1 -- 172.21.15.33:0/3294942473 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fc72c004ac0 con 0x5599e097b660 2024-05-03T18:27:51.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.337+0000 7fc7448cbb80 10 client.15381 fetch_fsmap learned FSMap version 21 2024-05-03T18:27:51.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.337+0000 7fc7448cbb80 1 -- 172.21.15.33:0/3294942473 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5599e09fe690 con 0x5599e097b660 2024-05-03T18:27:51.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.337+0000 7fc721600700 1 -- 172.21.15.33:0/3294942473 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc72c027070 con 0x5599e097b660 2024-05-03T18:27:51.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.337+0000 7fc721600700 1 -- 172.21.15.33:0/3294942473 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fc72c005240 con 0x5599e097b660 2024-05-03T18:27:51.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.338+0000 7fc7448cbb80 10 client.15381 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:27:51.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.338+0000 7fc7448cbb80 10 client.15381 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:27:51.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.338+0000 7fc7448cbb80 20 client.15381 populate_metadata read hostname 'smithi033' 2024-05-03T18:27:51.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.338+0000 7fc7448cbb80 1 --2- 172.21.15.33:0/3294942473 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5599e09fec00 0x5599e0a01020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:51.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.338+0000 7fc7448cbb80 4 client.15381 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-03T18:27:51.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.338+0000 7fc7448cbb80 1 -- 172.21.15.33:0/3294942473 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5599e08d1400 con 0x5599e09fec00 2024-05-03T18:27:51.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.338+0000 7fc733400700 1 --2- 172.21.15.33:0/3294942473 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5599e09fec00 0x5599e0a01020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:51.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.339+0000 7fc733400700 1 --2- 172.21.15.33:0/3294942473 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5599e09fec00 0x5599e0a01020 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36be346da7d02d40 server_cookie=d778382f4d83f9bd in_seq=0 out_seq=0 2024-05-03T18:27:51.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.339+0000 7fc721600700 10 client.15381 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:27:51.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.342+0000 7fc721600700 1 -- 172.21.15.33:0/3294942473 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5599e08d1400 con 0x5599e09fec00 2024-05-03T18:27:51.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.342+0000 7fc721600700 10 client.15381 handle_command_reply: tid=0 2024-05-03T18:27:51.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.342+0000 7fc7448cbb80 1 client.15381 shutdown 2024-05-03T18:27:51.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.342+0000 7fc7448cbb80 1 -- 172.21.15.33:0/3294942473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599e097b660 msgr2=0x5599e096ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:51.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.342+0000 7fc7448cbb80 1 --2- 172.21.15.33:0/3294942473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599e097b660 0x5599e096ff20 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fc72c019010 tx=0x7fc72c0101d0 comp rx=0 tx=0).stop 2024-05-03T18:27:51.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.343+0000 7fc7448cbb80 1 -- 172.21.15.33:0/3294942473 shutdown_connections 2024-05-03T18:27:51.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.343+0000 7fc7448cbb80 1 --2- 172.21.15.33:0/3294942473 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5599e09fec00 0x5599e0a01020 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:51.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.343+0000 7fc7448cbb80 1 --2- 172.21.15.33:0/3294942473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599e097b660 0x5599e096ff20 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:51.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.343+0000 7fc7448cbb80 1 --2- 172.21.15.33:0/3294942473 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5599e0975960 0x5599e0987520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:51.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.343+0000 7fc7448cbb80 1 --2- 172.21.15.33:0/3294942473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599e097ad40 0x5599e0987a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:51.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.343+0000 7fc721600700 0 client.15381 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:27:51.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.343+0000 7fc7448cbb80 1 -- 172.21.15.33:0/3294942473 >> 172.21.15.33:0/3294942473 conn(0x5599e08e96d0 msgr2=0x5599e097ba40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:27:51.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.343+0000 7fc7448cbb80 1 -- 172.21.15.33:0/3294942473 shutdown_connections 2024-05-03T18:27:51.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.343+0000 7fc7448cbb80 1 -- 172.21.15.33:0/3294942473 wait complete. 2024-05-03T18:27:51.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.343+0000 7fc7448cbb80 20 client.15381 trim_cache size 0 max 16384 2024-05-03T18:27:51.345 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-03T18:27:51.345 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-05-03T18:27:51.345 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-05-03T18:27:51.345 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-03T18:27:51.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.344+0000 7fc73a000700 1 -- 172.21.15.33:0/1082981719 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fc7180b2b00 msgr2=0x7fc7180b4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:51.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.344+0000 7fc73a000700 1 --2- 172.21.15.33:0/1082981719 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fc7180b2b00 0x7fc7180b4fc0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc72c011440 tx=0x7fc72c009040 comp rx=0 tx=0).stop 2024-05-03T18:27:51.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.344+0000 7fc73a000700 1 -- 172.21.15.33:0/1082981719 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc73406bb40 msgr2=0x7fc73419fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:51.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.344+0000 7fc73a000700 1 --2- 172.21.15.33:0/1082981719 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc73406bb40 0x7fc73419fbf0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fc71c00d980 tx=0x7fc71c00dc90 comp rx=0 tx=0).stop 2024-05-03T18:27:51.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.344+0000 7fc73a000700 1 -- 172.21.15.33:0/1082981719 shutdown_connections 2024-05-03T18:27:51.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.344+0000 7fc73a000700 1 --2- 172.21.15.33:0/1082981719 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fc7180b2b00 0x7fc7180b4fc0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:51.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.344+0000 7fc73a000700 1 --2- 172.21.15.33:0/1082981719 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc734109510 0x7fc734199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:51.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.344+0000 7fc73a000700 1 --2- 172.21.15.33:0/1082981719 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc73406b180 0x7fc73419f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:51.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.344+0000 7fc73a000700 1 --2- 172.21.15.33:0/1082981719 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc73406bb40 0x7fc73419fbf0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:51.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.344+0000 7fc73a000700 1 -- 172.21.15.33:0/1082981719 >> 172.21.15.33:0/1082981719 conn(0x7fc734079350 msgr2=0x7fc734101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:27:51.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.344+0000 7fc73a000700 1 -- 172.21.15.33:0/1082981719 shutdown_connections 2024-05-03T18:27:51.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:51.344+0000 7fc73a000700 1 -- 172.21.15.33:0/1082981719 wait complete. 2024-05-03T18:27:51.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:50 smithi150 ceph-mon[32492]: pgmap v22: 129 pgs: 129 active+clean; 26 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 1.9 MiB/s wr, 43 op/s 2024-05-03T18:27:52.228 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:51 smithi033 ceph-mon[28330]: scrub summary: idle 2024-05-03T18:27:52.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:51 smithi099 ceph-mon[31658]: scrub summary: idle 2024-05-03T18:27:52.313 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-03T18:27:52.313 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T18:27:52.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:51 smithi150 ceph-mon[32492]: scrub summary: idle 2024-05-03T18:27:52.826 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:27:53.195 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:52 smithi033 ceph-mon[28330]: pgmap v23: 129 pgs: 129 active+clean; 26 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 881 KiB/s wr, 37 op/s 2024-05-03T18:27:53.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:52 smithi099 ceph-mon[31658]: pgmap v23: 129 pgs: 129 active+clean; 26 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 881 KiB/s wr, 37 op/s 2024-05-03T18:27:53.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:52 smithi150 ceph-mon[32492]: pgmap v23: 129 pgs: 129 active+clean; 26 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 881 KiB/s wr, 37 op/s 2024-05-03T18:27:55.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:55.232+0000 7f6c22c00700 1 -- 172.21.15.33:0/1673301781 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c1c103d90 msgr2=0x7f6c1c1102a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:55.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:55.232+0000 7f6c22c00700 1 --2- 172.21.15.33:0/1673301781 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c1c103d90 0x7f6c1c1102a0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f6c1400b0a0 tx=0x7f6c1400b3b0 comp rx=0 tx=0).stop 2024-05-03T18:27:55.233 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:54 smithi033 ceph-mon[28330]: pgmap v24: 129 pgs: 129 active+clean; 34 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 2.3 MiB/s wr, 82 op/s 2024-05-03T18:27:55.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:55.232+0000 7f6c22c00700 1 -- 172.21.15.33:0/1673301781 shutdown_connections 2024-05-03T18:27:55.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:55.232+0000 7f6c22c00700 1 --2- 172.21.15.33:0/1673301781 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c1c103d90 0x7f6c1c1102a0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:55.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:55.232+0000 7f6c22c00700 1 --2- 172.21.15.33:0/1673301781 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c1c109460 0x7f6c1c109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:55.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:55.232+0000 7f6c22c00700 1 --2- 172.21.15.33:0/1673301781 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6c1c1033d0 0x7f6c1c103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:55.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:55.232+0000 7f6c22c00700 1 -- 172.21.15.33:0/1673301781 >> 172.21.15.33:0/1673301781 conn(0x7f6c1c0fef20 msgr2=0x7f6c1c101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:27:55.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:55.233+0000 7f6c22c00700 1 -- 172.21.15.33:0/1673301781 shutdown_connections 2024-05-03T18:27:55.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:55.233+0000 7f6c22c00700 1 -- 172.21.15.33:0/1673301781 wait complete. 2024-05-03T18:27:55.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:55.235+0000 7f6c22c00700 1 Processor -- start 2024-05-03T18:27:55.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:55.235+0000 7f6c22c00700 1 -- start start 2024-05-03T18:27:55.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:55.235+0000 7f6c22c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c1c1033d0 0x7f6c1c1a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:55.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:55.235+0000 7f6c22c00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c1c103d90 0x7f6c1c1a1c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:55.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:55.235+0000 7f6c22c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6c1c109460 0x7f6c1c19b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:55.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:55.235+0000 7f6c22c00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6c1c113450 con 0x7f6c1c103d90 2024-05-03T18:27:55.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:55.235+0000 7f6c22c00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6c1c1132d0 con 0x7f6c1c1033d0 2024-05-03T18:27:55.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:55.235+0000 7f6c22c00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6c1c1135d0 con 0x7f6c1c109460 2024-05-03T18:27:55.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:55.236+0000 7f6c20e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c1c103d90 0x7f6c1c1a1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:55.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:55.236+0000 7f6c20e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c1c103d90 0x7f6c1c1a1c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39940/0 (socket says 172.21.15.33:39940) 2024-05-03T18:27:55.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:55.236+0000 7f6c20e00700 1 -- 172.21.15.33:0/491181110 learned_addr learned my addr 172.21.15.33:0/491181110 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:27:55.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:55.236+0000 7f6c21800700 1 --2- 172.21.15.33:0/491181110 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c1c1033d0 0x7f6c1c1a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:55.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:55.236+0000 7f6c22200700 1 --2- 172.21.15.33:0/491181110 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6c1c109460 0x7f6c1c19b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:55.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:55.236+0000 7f6c20e00700 1 -- 172.21.15.33:0/491181110 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6c1c109460 msgr2=0x7f6c1c19b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:55.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:55.236+0000 7f6c20e00700 1 --2- 172.21.15.33:0/491181110 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6c1c109460 0x7f6c1c19b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:55.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:55.236+0000 7f6c20e00700 1 -- 172.21.15.33:0/491181110 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c1c1033d0 msgr2=0x7f6c1c1a1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:55.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:55.236+0000 7f6c20e00700 1 --2- 172.21.15.33:0/491181110 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c1c1033d0 0x7f6c1c1a1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:55.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:55.237+0000 7f6c20e00700 1 -- 172.21.15.33:0/491181110 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c1c19c030 con 0x7f6c1c103d90 2024-05-03T18:27:55.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:55.237+0000 7f6c20e00700 1 --2- 172.21.15.33:0/491181110 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c1c103d90 0x7f6c1c1a1c50 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c00be80 tx=0x7f6c0c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:27:55.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:55.238+0000 7f6c1a000700 1 -- 172.21.15.33:0/491181110 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c0c01c070 con 0x7f6c1c103d90 2024-05-03T18:27:55.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:55.238+0000 7f6c1a000700 1 -- 172.21.15.33:0/491181110 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6c0c00de40 con 0x7f6c1c103d90 2024-05-03T18:27:55.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:55.238+0000 7f6c22c00700 1 -- 172.21.15.33:0/491181110 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c1c19c320 con 0x7f6c1c103d90 2024-05-03T18:27:55.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:55.238+0000 7f6c1a000700 1 -- 172.21.15.33:0/491181110 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c0c009730 con 0x7f6c1c103d90 2024-05-03T18:27:55.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:55.238+0000 7f6c22c00700 1 -- 172.21.15.33:0/491181110 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c1c10a250 con 0x7f6c1c103d90 2024-05-03T18:27:55.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:55.239+0000 7f6c22c00700 1 -- 172.21.15.33:0/491181110 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c1c06a190 con 0x7f6c1c103d90 2024-05-03T18:27:55.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:55.242+0000 7f6c1a000700 1 -- 172.21.15.33:0/491181110 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 23) ==== 149828+0+0 (secure 0 0 0) 0x7f6c0c0098d0 con 0x7f6c1c103d90 2024-05-03T18:27:55.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:55.244+0000 7f6c1a000700 1 --2- 172.21.15.33:0/491181110 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f6c040b2a80 0x7f6c040b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:55.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:55.245+0000 7f6c1a000700 1 -- 172.21.15.33:0/491181110 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f6c0c0ea9c0 con 0x7f6c1c103d90 2024-05-03T18:27:55.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:55.245+0000 7f6c21800700 1 --2- 172.21.15.33:0/491181110 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f6c040b2a80 0x7f6c040b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:55.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:55.245+0000 7f6c21800700 1 --2- 172.21.15.33:0/491181110 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f6c040b2a80 0x7f6c040b4f40 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6c10005fd0 tx=0x7f6c1000af00 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:27:55.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:55.249+0000 7f6c1a000700 1 -- 172.21.15.33:0/491181110 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f6c0c0b1f50 con 0x7f6c1c103d90 2024-05-03T18:27:55.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:54 smithi099 ceph-mon[31658]: pgmap v24: 129 pgs: 129 active+clean; 34 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 2.3 MiB/s wr, 82 op/s 2024-05-03T18:27:55.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:54 smithi150 ceph-mon[32492]: pgmap v24: 129 pgs: 129 active+clean; 34 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 2.3 MiB/s wr, 82 op/s 2024-05-03T18:27:55.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:55.571+0000 7f6c22c00700 1 -- 172.21.15.33:0/491181110 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6c1c19ce00 con 0x7f6c1c103d90 2024-05-03T18:27:55.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:55.573+0000 7f6c1a000700 1 -- 172.21.15.33:0/491181110 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11801 (secure 0 0 0) 0x7f6c0c005000 con 0x7f6c1c103d90 2024-05-03T18:27:55.574 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:27:55.574 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1327393503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1327393503},{"type":"v1","addr":"172.21.15.33:6835","nonce":1327393503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/2461472948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":2461472948},{"type":"v1","addr":"172.21.15.33:6837","nonce":2461472948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14559,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/442269281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":442269281},{"type":"v1","addr":"172.21.15.33:6839","nonce":442269281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14571,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/645571756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":645571756},{"type":"v1","addr":"172.21.15.33:6841","nonce":645571756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/203598909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":203598909},{"type":"v1","addr":"172.21.15.150:6833","nonce":203598909}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/4029842493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":4029842493},{"type":"v1","addr":"172.21.15.150:6835","nonce":4029842493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3463664423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3463664423},{"type":"v1","addr":"172.21.15.150:6837","nonce":3463664423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24481,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6839/4051648018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":4051648018},{"type":"v1","addr":"172.21.15.150:6839","nonce":4051648018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34418,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1865105479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1865105479},{"type":"v1","addr":"172.21.15.99:6833","nonce":1865105479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34445,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/1206393914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1206393914},{"type":"v1","addr":"172.21.15.99:6835","nonce":1206393914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-03T18:22:28.837091+0000","modified":"2024-05-03T18:23:49.758398+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34508},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34472":{"gid":34472,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6837/3360970342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3360970342},{"type":"v1","addr":"172.21.15.99:6837","nonce":3360970342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34508":{"gid":34508,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/919907308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":919907308},{"type":"v1","addr":"172.21.15.99:6839","nonce":919907308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34508,"qdb_cluster":[34508]},"id":1}]} 2024-05-03T18:27:55.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:55.577+0000 7f6c22c00700 1 -- 172.21.15.33:0/491181110 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f6c040b2a80 msgr2=0x7f6c040b4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:55.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:55.577+0000 7f6c22c00700 1 --2- 172.21.15.33:0/491181110 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f6c040b2a80 0x7f6c040b4f40 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6c10005fd0 tx=0x7f6c1000af00 comp rx=0 tx=0).stop 2024-05-03T18:27:55.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:55.577+0000 7f6c22c00700 1 -- 172.21.15.33:0/491181110 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c1c103d90 msgr2=0x7f6c1c1a1c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:55.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:55.577+0000 7f6c22c00700 1 --2- 172.21.15.33:0/491181110 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c1c103d90 0x7f6c1c1a1c50 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c00be80 tx=0x7f6c0c00dc60 comp rx=0 tx=0).stop 2024-05-03T18:27:55.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:55.577+0000 7f6c22c00700 1 -- 172.21.15.33:0/491181110 shutdown_connections 2024-05-03T18:27:55.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:55.577+0000 7f6c22c00700 1 --2- 172.21.15.33:0/491181110 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6c1c109460 0x7f6c1c19b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:55.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:55.577+0000 7f6c22c00700 1 --2- 172.21.15.33:0/491181110 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f6c040b2a80 0x7f6c040b4f40 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:55.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:55.577+0000 7f6c22c00700 1 --2- 172.21.15.33:0/491181110 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c1c1033d0 0x7f6c1c1a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:55.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:55.577+0000 7f6c22c00700 1 --2- 172.21.15.33:0/491181110 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c1c103d90 0x7f6c1c1a1c50 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:55.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:55.577+0000 7f6c22c00700 1 -- 172.21.15.33:0/491181110 >> 172.21.15.33:0/491181110 conn(0x7f6c1c0fef20 msgr2=0x7f6c1c0ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:27:55.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:55.577+0000 7f6c22c00700 1 -- 172.21.15.33:0/491181110 shutdown_connections 2024-05-03T18:27:55.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:55.578+0000 7f6c22c00700 1 -- 172.21.15.33:0/491181110 wait complete. 2024-05-03T18:27:55.579 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 21 2024-05-03T18:27:56.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:55 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/491181110' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:27:56.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:55 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/491181110' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:27:56.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:55 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/491181110' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:27:56.520 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-03T18:27:57.023 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:27:57.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:56 smithi033 ceph-mon[28330]: pgmap v25: 129 pgs: 129 active+clean; 34 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 2.0 MiB/s wr, 64 op/s 2024-05-03T18:27:57.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:56 smithi099 ceph-mon[31658]: pgmap v25: 129 pgs: 129 active+clean; 34 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 2.0 MiB/s wr, 64 op/s 2024-05-03T18:27:57.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:56 smithi150 ceph-mon[32492]: pgmap v25: 129 pgs: 129 active+clean; 34 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 2.0 MiB/s wr, 64 op/s 2024-05-03T18:27:59.174 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:58 smithi033 ceph-mon[28330]: pgmap v26: 129 pgs: 129 active+clean; 37 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 2.3 MiB/s wr, 75 op/s 2024-05-03T18:27:59.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:58 smithi099 ceph-mon[31658]: pgmap v26: 129 pgs: 129 active+clean; 37 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 2.3 MiB/s wr, 75 op/s 2024-05-03T18:27:59.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:58 smithi150 ceph-mon[32492]: pgmap v26: 129 pgs: 129 active+clean; 37 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 2.3 MiB/s wr, 75 op/s 2024-05-03T18:27:59.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.468+0000 7f51afe00700 1 -- 172.21.15.33:0/3737928217 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51b0109460 msgr2=0x7f51b0109840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:59.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.468+0000 7f51afe00700 1 --2- 172.21.15.33:0/3737928217 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51b0109460 0x7f51b0109840 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f51a0009990 tx=0x7f51a0009ca0 comp rx=0 tx=0).stop 2024-05-03T18:27:59.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.468+0000 7f51afe00700 1 -- 172.21.15.33:0/3737928217 shutdown_connections 2024-05-03T18:27:59.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.468+0000 7f51afe00700 1 --2- 172.21.15.33:0/3737928217 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f51b0103d90 0x7f51b01102a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:59.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.468+0000 7f51afe00700 1 --2- 172.21.15.33:0/3737928217 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51b0109460 0x7f51b0109840 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:59.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.468+0000 7f51afe00700 1 --2- 172.21.15.33:0/3737928217 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51b01033d0 0x7f51b0103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:59.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.468+0000 7f51afe00700 1 -- 172.21.15.33:0/3737928217 >> 172.21.15.33:0/3737928217 conn(0x7f51b00fef20 msgr2=0x7f51b0101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:27:59.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.468+0000 7f51afe00700 1 -- 172.21.15.33:0/3737928217 shutdown_connections 2024-05-03T18:27:59.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.469+0000 7f51afe00700 1 -- 172.21.15.33:0/3737928217 wait complete. 2024-05-03T18:27:59.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.470+0000 7f51afe00700 1 Processor -- start 2024-05-03T18:27:59.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.471+0000 7f51afe00700 1 -- start start 2024-05-03T18:27:59.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.471+0000 7f51afe00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51b01033d0 0x7f51b01a1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:59.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.471+0000 7f51afe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f51b0103d90 0x7f51b01a1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:59.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.471+0000 7f51afe00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51b0109460 0x7f51b019b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:59.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.471+0000 7f51afe00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f51b0113420 con 0x7f51b01033d0 2024-05-03T18:27:59.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.471+0000 7f51afe00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f51b01132a0 con 0x7f51b0103d90 2024-05-03T18:27:59.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.471+0000 7f51afe00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f51b01135a0 con 0x7f51b0109460 2024-05-03T18:27:59.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.472+0000 7f51aea00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51b01033d0 0x7f51b01a1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:59.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.472+0000 7f51aea00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51b01033d0 0x7f51b01a1800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48170/0 (socket says 172.21.15.33:48170) 2024-05-03T18:27:59.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.472+0000 7f51aea00700 1 -- 172.21.15.33:0/1414130710 learned_addr learned my addr 172.21.15.33:0/1414130710 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:27:59.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.472+0000 7f51ae000700 1 --2- 172.21.15.33:0/1414130710 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f51b0103d90 0x7f51b01a1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:59.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.472+0000 7f51aea00700 1 -- 172.21.15.33:0/1414130710 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51b0109460 msgr2=0x7f51b019b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:59.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.472+0000 7f51af400700 1 --2- 172.21.15.33:0/1414130710 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51b0109460 0x7f51b019b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:59.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.472+0000 7f51aea00700 1 --2- 172.21.15.33:0/1414130710 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51b0109460 0x7f51b019b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:59.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.472+0000 7f51aea00700 1 -- 172.21.15.33:0/1414130710 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f51b0103d90 msgr2=0x7f51b01a1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:59.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.472+0000 7f51aea00700 1 --2- 172.21.15.33:0/1414130710 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f51b0103d90 0x7f51b01a1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:59.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.472+0000 7f51aea00700 1 -- 172.21.15.33:0/1414130710 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51b019c120 con 0x7f51b01033d0 2024-05-03T18:27:59.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.472+0000 7f51ae000700 1 --2- 172.21.15.33:0/1414130710 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f51b0103d90 0x7f51b01a1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:27:59.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.472+0000 7f51af400700 1 --2- 172.21.15.33:0/1414130710 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51b0109460 0x7f51b019b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:27:59.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.473+0000 7f51aea00700 1 --2- 172.21.15.33:0/1414130710 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51b01033d0 0x7f51b01a1800 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f51a00057c0 tx=0x7f51a0009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:27:59.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.473+0000 7f51a7400700 1 -- 172.21.15.33:0/1414130710 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a0027070 con 0x7f51b01033d0 2024-05-03T18:27:59.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.473+0000 7f51a7400700 1 -- 172.21.15.33:0/1414130710 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f51a000fda0 con 0x7f51b01033d0 2024-05-03T18:27:59.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.473+0000 7f51afe00700 1 -- 172.21.15.33:0/1414130710 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51b019c3b0 con 0x7f51b01033d0 2024-05-03T18:27:59.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.473+0000 7f51afe00700 1 -- 172.21.15.33:0/1414130710 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51b01a86d0 con 0x7f51b01033d0 2024-05-03T18:27:59.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.473+0000 7f51a7400700 1 -- 172.21.15.33:0/1414130710 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a0022ca0 con 0x7f51b01033d0 2024-05-03T18:27:59.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.478+0000 7f51a7400700 1 -- 172.21.15.33:0/1414130710 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 23) ==== 149828+0+0 (secure 0 0 0) 0x7f51a002a070 con 0x7f51b01033d0 2024-05-03T18:27:59.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.479+0000 7f51c02f8b80 1 Processor -- start 2024-05-03T18:27:59.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.479+0000 7f51c02f8b80 1 -- start start 2024-05-03T18:27:59.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.479+0000 7f51c02f8b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a31da99bd0 0x55a31da99fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:59.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.479+0000 7f51c02f8b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a31da93f30 0x55a31da943b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:59.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.480+0000 7f51c02f8b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a31da948f0 0x55a31daa0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:59.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.480+0000 7f51c02f8b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a31d9e6a20 con 0x55a31da93f30 2024-05-03T18:27:59.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.480+0000 7f51c02f8b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55a31d984760 con 0x55a31da948f0 2024-05-03T18:27:59.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.480+0000 7f51c02f8b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55a31d9e29e0 con 0x55a31da99bd0 2024-05-03T18:27:59.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.480+0000 7f51aea00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a31da93f30 0x55a31da943b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:59.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.480+0000 7f51aea00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a31da93f30 0x55a31da943b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48186/0 (socket says 172.21.15.33:48186) 2024-05-03T18:27:59.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.480+0000 7f51aea00700 1 -- 172.21.15.33:0/117237596 learned_addr learned my addr 172.21.15.33:0/117237596 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:27:59.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.480+0000 7f51a7400700 1 --2- 172.21.15.33:0/1414130710 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f51940b2a80 0x7f51940b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:59.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.480+0000 7f51af400700 1 --2- 172.21.15.33:0/117237596 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a31da99bd0 0x55a31da99fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:59.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.480+0000 7f51ae000700 1 --2- 172.21.15.33:0/117237596 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a31da948f0 0x55a31daa0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:59.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.480+0000 7f51a7400700 1 -- 172.21.15.33:0/1414130710 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f51a00e3990 con 0x7f51b01033d0 2024-05-03T18:27:59.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.481+0000 7f51aea00700 1 -- 172.21.15.33:0/117237596 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a31da99bd0 msgr2=0x55a31da99fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:27:59.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.481+0000 7f51aea00700 1 --2- 172.21.15.33:0/117237596 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a31da99bd0 0x55a31da99fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:59.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.481+0000 7f51aea00700 1 -- 172.21.15.33:0/117237596 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a31da948f0 msgr2=0x55a31daa0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:27:59.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.481+0000 7f51aea00700 1 --2- 172.21.15.33:0/117237596 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a31da948f0 0x55a31daa0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:59.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.481+0000 7f51aea00700 1 -- 172.21.15.33:0/117237596 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a31d9338d0 con 0x55a31da93f30 2024-05-03T18:27:59.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.481+0000 7f51af400700 1 --2- 172.21.15.33:0/1414130710 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f51940b2a80 0x7f51940b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:59.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.481+0000 7f51aea00700 1 --2- 172.21.15.33:0/117237596 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a31da93f30 0x55a31da943b0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f51a00aa730 tx=0x7f51a0005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c07e0acfa3faa46b server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:27:59.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.481+0000 7f519be00700 1 -- 172.21.15.33:0/117237596 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a00aa570 con 0x55a31da93f30 2024-05-03T18:27:59.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.482+0000 7f519be00700 1 -- 172.21.15.33:0/117237596 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f51a00e10e0 con 0x55a31da93f30 2024-05-03T18:27:59.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.482+0000 7f519be00700 1 -- 172.21.15.33:0/117237596 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a00a9f10 con 0x55a31da93f30 2024-05-03T18:27:59.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.482+0000 7f51c02f8b80 1 -- 172.21.15.33:0/117237596 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a31da93f30 msgr2=0x55a31da943b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:59.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.483+0000 7f51c02f8b80 1 --2- 172.21.15.33:0/117237596 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a31da93f30 0x55a31da943b0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f51a00aa730 tx=0x7f51a0005fd0 comp rx=0 tx=0).stop 2024-05-03T18:27:59.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.482+0000 7f51af400700 1 --2- 172.21.15.33:0/1414130710 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f51940b2a80 0x7f51940b4f40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f51a8008a90 tx=0x7f51a8004b40 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:27:59.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.483+0000 7f51c02f8b80 1 -- 172.21.15.33:0/117237596 shutdown_connections 2024-05-03T18:27:59.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.483+0000 7f51c02f8b80 1 --2- 172.21.15.33:0/117237596 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a31da948f0 0x55a31daa0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:59.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.483+0000 7f51c02f8b80 1 --2- 172.21.15.33:0/117237596 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a31da99bd0 0x55a31da99fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:59.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.483+0000 7f51c02f8b80 1 --2- 172.21.15.33:0/117237596 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a31da93f30 0x55a31da943b0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:59.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.483+0000 7f51c02f8b80 1 -- 172.21.15.33:0/117237596 >> 172.21.15.33:0/117237596 conn(0x55a31da0c6d0 msgr2=0x55a31da8ebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:27:59.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.483+0000 7f51c02f8b80 1 -- 172.21.15.33:0/117237596 shutdown_connections 2024-05-03T18:27:59.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.483+0000 7f51c02f8b80 1 -- 172.21.15.33:0/117237596 wait complete. 2024-05-03T18:27:59.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.484+0000 7f51c02f8b80 1 -- start start 2024-05-03T18:27:59.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.484+0000 7f51c02f8b80 1 Processor -- start 2024-05-03T18:27:59.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.484+0000 7f51c02f8b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a31da93f30 0x55a31daa0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:59.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.484+0000 7f51c02f8b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a31da948f0 0x55a31da9d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:59.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.484+0000 7f51c02f8b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a31da99bd0 0x55a31da9de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:59.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.485+0000 7f51c02f8b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a31d9e6a20 con 0x55a31da93f30 2024-05-03T18:27:59.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.485+0000 7f51c02f8b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55a31d94cb20 con 0x55a31da99bd0 2024-05-03T18:27:59.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.485+0000 7f51c02f8b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55a31da9a6a0 con 0x55a31da948f0 2024-05-03T18:27:59.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.485+0000 7f51c02f8b80 10 client.0 _ll_register_callbacks cb 0x55a31d98dc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:27:59.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.485+0000 7f51aea00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a31da93f30 0x55a31daa0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:59.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.485+0000 7f51aea00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a31da93f30 0x55a31daa0530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48198/0 (socket says 172.21.15.33:48198) 2024-05-03T18:27:59.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.485+0000 7f51aea00700 1 -- 172.21.15.33:0/2257229402 learned_addr learned my addr 172.21.15.33:0/2257229402 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:27:59.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.485+0000 7f51ae000700 1 --2- 172.21.15.33:0/2257229402 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a31da948f0 0x55a31da9d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:59.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.485+0000 7f51aea00700 1 -- 172.21.15.33:0/2257229402 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a31da948f0 msgr2=0x55a31da9d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:27:59.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.485+0000 7f51aea00700 1 --2- 172.21.15.33:0/2257229402 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a31da948f0 0x55a31da9d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:59.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.485+0000 7f51aea00700 1 -- 172.21.15.33:0/2257229402 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a31da99bd0 msgr2=0x55a31da9de10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-03T18:27:59.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.485+0000 7f51aea00700 1 --2- 172.21.15.33:0/2257229402 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a31da99bd0 0x55a31da9de10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:59.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.485+0000 7f51aea00700 1 -- 172.21.15.33:0/2257229402 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a31d9ac420 con 0x55a31da93f30 2024-05-03T18:27:59.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.486+0000 7f51aea00700 1 --2- 172.21.15.33:0/2257229402 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a31da93f30 0x55a31daa0530 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f51a00a6820 tx=0x7f51a00b3440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4133d722e01420f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:27:59.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.486+0000 7f5198c00700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:27:59.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.486+0000 7f5198c00700 1 -- 172.21.15.33:0/2257229402 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a00aa0f0 con 0x55a31da93f30 2024-05-03T18:27:59.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.486+0000 7f5198c00700 1 -- 172.21.15.33:0/2257229402 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f51a000fb20 con 0x55a31da93f30 2024-05-03T18:27:59.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.486+0000 7f51c02f8b80 1 -- 172.21.15.33:0/2257229402 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a31da60ff0 con 0x55a31da93f30 2024-05-03T18:27:59.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.487+0000 7f5198c00700 1 -- 172.21.15.33:0/2257229402 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f51a00a9f10 con 0x55a31da93f30 2024-05-03T18:27:59.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.487+0000 7f51c02f8b80 10 client.15399 fetch_fsmap learned FSMap version 21 2024-05-03T18:27:59.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.487+0000 7f51c02f8b80 1 -- 172.21.15.33:0/2257229402 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55a31daab330 con 0x55a31da93f30 2024-05-03T18:27:59.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.487+0000 7f5198c00700 1 -- 172.21.15.33:0/2257229402 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55a31da60ff0 con 0x55a31da93f30 2024-05-03T18:27:59.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.487+0000 7f5198c00700 1 -- 172.21.15.33:0/2257229402 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f51a00afce0 con 0x55a31da93f30 2024-05-03T18:27:59.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.487+0000 7f51c02f8b80 10 client.15399 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:27:59.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.487+0000 7f51c02f8b80 10 client.15399 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:27:59.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.487+0000 7f51c02f8b80 20 client.15399 populate_metadata read hostname 'smithi033' 2024-05-03T18:27:59.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.487+0000 7f51c02f8b80 1 --2- 172.21.15.33:0/2257229402 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55a31daad800 0x55a31daafbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:59.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.487+0000 7f51c02f8b80 4 client.15399 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:27:59.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.487+0000 7f51c02f8b80 1 -- 172.21.15.33:0/2257229402 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a31d9f4400 con 0x55a31daad800 2024-05-03T18:27:59.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.488+0000 7f51ae000700 1 --2- 172.21.15.33:0/2257229402 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55a31daad800 0x55a31daafbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:59.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.489+0000 7f51ae000700 1 --2- 172.21.15.33:0/2257229402 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55a31daad800 0x55a31daafbf0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf53ffe0ca3aeb7c server_cookie=21220f3e01a64a9 in_seq=0 out_seq=0 2024-05-03T18:27:59.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.489+0000 7f5198c00700 10 client.15399 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:27:59.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.491+0000 7f5198c00700 1 -- 172.21.15.33:0/2257229402 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a31d9f4400 con 0x55a31daad800 2024-05-03T18:27:59.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.491+0000 7f5198c00700 10 client.15399 handle_command_reply: tid=0 2024-05-03T18:27:59.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.491+0000 7f51c02f8b80 1 client.15399 shutdown 2024-05-03T18:27:59.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.491+0000 7f51c02f8b80 1 -- 172.21.15.33:0/2257229402 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a31da93f30 msgr2=0x55a31daa0530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:59.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.491+0000 7f51c02f8b80 1 --2- 172.21.15.33:0/2257229402 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a31da93f30 0x55a31daa0530 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f51a00a6820 tx=0x7f51a00b3440 comp rx=0 tx=0).stop 2024-05-03T18:27:59.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.492+0000 7f51c02f8b80 1 -- 172.21.15.33:0/2257229402 shutdown_connections 2024-05-03T18:27:59.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.492+0000 7f51c02f8b80 1 --2- 172.21.15.33:0/2257229402 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55a31daad800 0x55a31daafbf0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:59.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.492+0000 7f51c02f8b80 1 --2- 172.21.15.33:0/2257229402 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a31da99bd0 0x55a31da9de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:59.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.492+0000 7f51c02f8b80 1 --2- 172.21.15.33:0/2257229402 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a31da93f30 0x55a31daa0530 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:59.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.492+0000 7f51c02f8b80 1 --2- 172.21.15.33:0/2257229402 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a31da948f0 0x55a31da9d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:59.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.492+0000 7f51c02f8b80 1 -- 172.21.15.33:0/2257229402 >> 172.21.15.33:0/2257229402 conn(0x55a31da0c6d0 msgr2=0x55a31da92f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:27:59.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.492+0000 7f5198c00700 0 client.15399 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:27:59.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.492+0000 7f51c02f8b80 1 -- 172.21.15.33:0/2257229402 shutdown_connections 2024-05-03T18:27:59.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.492+0000 7f51c02f8b80 1 -- 172.21.15.33:0/2257229402 wait complete. 2024-05-03T18:27:59.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.492+0000 7f51c02f8b80 20 client.15399 trim_cache size 0 max 16384 2024-05-03T18:27:59.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.514+0000 7f51c02f8b80 1 Processor -- start 2024-05-03T18:27:59.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.514+0000 7f51c02f8b80 1 -- start start 2024-05-03T18:27:59.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.515+0000 7f51c02f8b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a31da93f30 0x55a31daa9c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:59.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.515+0000 7f51c02f8b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a31da948f0 0x55a31da99170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:59.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.515+0000 7f51c02f8b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a31da99bd0 0x55a31da9d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:59.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.515+0000 7f51c02f8b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f519000ccc0 con 0x55a31da948f0 2024-05-03T18:27:59.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.515+0000 7f51c02f8b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f519000f540 con 0x55a31da99bd0 2024-05-03T18:27:59.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.515+0000 7f51c02f8b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f519000d330 con 0x55a31da93f30 2024-05-03T18:27:59.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.515+0000 7f51ae000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a31da948f0 0x55a31da99170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:59.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.515+0000 7f51ae000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a31da948f0 0x55a31da99170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48214/0 (socket says 172.21.15.33:48214) 2024-05-03T18:27:59.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.515+0000 7f51ae000700 1 -- 172.21.15.33:0/3247981699 learned_addr learned my addr 172.21.15.33:0/3247981699 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:27:59.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.515+0000 7f51aea00700 1 --2- 172.21.15.33:0/3247981699 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a31da93f30 0x55a31daa9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:59.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.516+0000 7f51ae000700 1 -- 172.21.15.33:0/3247981699 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a31da93f30 msgr2=0x55a31daa9c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:27:59.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.516+0000 7f51ae000700 1 --2- 172.21.15.33:0/3247981699 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a31da93f30 0x55a31daa9c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:59.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.516+0000 7f51af400700 1 --2- 172.21.15.33:0/3247981699 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a31da99bd0 0x55a31da9d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:59.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.516+0000 7f51ae000700 1 -- 172.21.15.33:0/3247981699 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a31da99bd0 msgr2=0x55a31da9d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:27:59.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.516+0000 7f51ae000700 1 --2- 172.21.15.33:0/3247981699 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a31da99bd0 0x55a31da9d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:59.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.516+0000 7f51ae000700 1 -- 172.21.15.33:0/3247981699 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a31da60fd0 con 0x55a31da948f0 2024-05-03T18:27:59.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.516+0000 7f51aea00700 1 --2- 172.21.15.33:0/3247981699 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a31da93f30 0x55a31daa9c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:27:59.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.516+0000 7f51af400700 1 --2- 172.21.15.33:0/3247981699 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a31da99bd0 0x55a31da9d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:27:59.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.516+0000 7f51ae000700 1 --2- 172.21.15.33:0/3247981699 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a31da948f0 0x55a31da99170 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x55a31da8fa90 tx=0x7f519c0096d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=572aa687858cd9ad server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:27:59.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.517+0000 7f519b400700 1 -- 172.21.15.33:0/3247981699 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f519c013d70 con 0x55a31da948f0 2024-05-03T18:27:59.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.517+0000 7f519b400700 1 -- 172.21.15.33:0/3247981699 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f519c01e030 con 0x55a31da948f0 2024-05-03T18:27:59.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.517+0000 7f519b400700 1 -- 172.21.15.33:0/3247981699 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f519c0118e0 con 0x55a31da948f0 2024-05-03T18:27:59.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.517+0000 7f51c02f8b80 1 -- 172.21.15.33:0/3247981699 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a31da948f0 msgr2=0x55a31da99170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:59.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.517+0000 7f51c02f8b80 1 --2- 172.21.15.33:0/3247981699 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a31da948f0 0x55a31da99170 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x55a31da8fa90 tx=0x7f519c0096d0 comp rx=0 tx=0).stop 2024-05-03T18:27:59.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.518+0000 7f51c02f8b80 1 -- 172.21.15.33:0/3247981699 shutdown_connections 2024-05-03T18:27:59.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.518+0000 7f51c02f8b80 1 --2- 172.21.15.33:0/3247981699 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a31da99bd0 0x55a31da9d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:59.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.518+0000 7f51c02f8b80 1 --2- 172.21.15.33:0/3247981699 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a31da93f30 0x55a31daa9c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:59.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.518+0000 7f51c02f8b80 1 --2- 172.21.15.33:0/3247981699 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a31da948f0 0x55a31da99170 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:59.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.518+0000 7f51c02f8b80 1 -- 172.21.15.33:0/3247981699 >> 172.21.15.33:0/3247981699 conn(0x55a31da0c6d0 msgr2=0x55a31da98a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:27:59.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.518+0000 7f51c02f8b80 1 -- 172.21.15.33:0/3247981699 shutdown_connections 2024-05-03T18:27:59.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.518+0000 7f51c02f8b80 1 -- 172.21.15.33:0/3247981699 wait complete. 2024-05-03T18:27:59.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.519+0000 7f51c02f8b80 1 -- start start 2024-05-03T18:27:59.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.519+0000 7f51c02f8b80 1 Processor -- start 2024-05-03T18:27:59.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.519+0000 7f51c02f8b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a31da93f30 0x55a31daa63e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:59.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.519+0000 7f51c02f8b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a31da948f0 0x55a31dab4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:59.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.519+0000 7f51c02f8b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a31da99bd0 0x55a31dab4f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:59.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.519+0000 7f51c02f8b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f519000da00 con 0x55a31da948f0 2024-05-03T18:27:59.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.519+0000 7f51c02f8b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f519000ee70 con 0x55a31da93f30 2024-05-03T18:27:59.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.519+0000 7f51c02f8b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f519000e0d0 con 0x55a31da99bd0 2024-05-03T18:27:59.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.520+0000 7f51c02f8b80 10 client.0 _ll_register_callbacks cb 0x55a31d98dc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:27:59.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.520+0000 7f51ae000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a31da948f0 0x55a31dab4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:59.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.520+0000 7f51ae000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a31da948f0 0x55a31dab4a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48226/0 (socket says 172.21.15.33:48226) 2024-05-03T18:27:59.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.520+0000 7f51ae000700 1 -- 172.21.15.33:0/3913545421 learned_addr learned my addr 172.21.15.33:0/3913545421 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:27:59.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.520+0000 7f51aea00700 1 --2- 172.21.15.33:0/3913545421 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a31da93f30 0x55a31daa63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:59.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.520+0000 7f51af400700 1 --2- 172.21.15.33:0/3913545421 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a31da99bd0 0x55a31dab4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:59.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.520+0000 7f51ae000700 1 -- 172.21.15.33:0/3913545421 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a31da99bd0 msgr2=0x55a31dab4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:27:59.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.520+0000 7f51ae000700 1 --2- 172.21.15.33:0/3913545421 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a31da99bd0 0x55a31dab4f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:59.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.520+0000 7f51ae000700 1 -- 172.21.15.33:0/3913545421 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a31da93f30 msgr2=0x55a31daa63e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:27:59.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.520+0000 7f51ae000700 1 --2- 172.21.15.33:0/3913545421 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a31da93f30 0x55a31daa63e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:59.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.520+0000 7f51ae000700 1 -- 172.21.15.33:0/3913545421 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a31dab56b0 con 0x55a31da948f0 2024-05-03T18:27:59.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.521+0000 7f51ae000700 1 --2- 172.21.15.33:0/3913545421 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a31da948f0 0x55a31dab4a30 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f519c021000 tx=0x7f519c009990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f7231d3a6835c32 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:27:59.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.521+0000 7f5198c00700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:27:59.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.521+0000 7f5198c00700 1 -- 172.21.15.33:0/3913545421 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f519c013c30 con 0x55a31da948f0 2024-05-03T18:27:59.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.521+0000 7f51c02f8b80 1 -- 172.21.15.33:0/3913545421 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f519c01e030 con 0x55a31da948f0 2024-05-03T18:27:59.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.521+0000 7f5198c00700 1 -- 172.21.15.33:0/3913545421 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f519c013dd0 con 0x55a31da948f0 2024-05-03T18:27:59.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.521+0000 7f5198c00700 1 -- 172.21.15.33:0/3913545421 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f519c037050 con 0x55a31da948f0 2024-05-03T18:27:59.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.521+0000 7f51c02f8b80 10 client.15405 fetch_fsmap learned FSMap version 21 2024-05-03T18:27:59.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.521+0000 7f51c02f8b80 1 -- 172.21.15.33:0/3913545421 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55a31db17ea0 con 0x55a31da948f0 2024-05-03T18:27:59.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.522+0000 7f5198c00700 1 -- 172.21.15.33:0/3913545421 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f519c01e030 con 0x55a31da948f0 2024-05-03T18:27:59.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.522+0000 7f5198c00700 1 -- 172.21.15.33:0/3913545421 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f519c00f040 con 0x55a31da948f0 2024-05-03T18:27:59.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.522+0000 7f51c02f8b80 10 client.15405 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:27:59.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.522+0000 7f51c02f8b80 10 client.15405 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:27:59.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.522+0000 7f51c02f8b80 20 client.15405 populate_metadata read hostname 'smithi033' 2024-05-03T18:27:59.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.522+0000 7f51c02f8b80 1 --2- 172.21.15.33:0/3913545421 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55a31db185b0 0x55a31db1a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:27:59.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.522+0000 7f51c02f8b80 4 client.15405 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-03T18:27:59.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.522+0000 7f51c02f8b80 1 -- 172.21.15.33:0/3913545421 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a31d9ee4a0 con 0x55a31db185b0 2024-05-03T18:27:59.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.523+0000 7f51aea00700 1 --2- 172.21.15.33:0/3913545421 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55a31db185b0 0x55a31db1a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:27:59.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.524+0000 7f51aea00700 1 --2- 172.21.15.33:0/3913545421 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55a31db185b0 0x55a31db1a9d0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d9d575a9331c1e server_cookie=bf7e703c0dfe389b in_seq=0 out_seq=0 2024-05-03T18:27:59.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.524+0000 7f5198c00700 10 client.15405 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:27:59.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.524+0000 7f5198c00700 1 -- 172.21.15.33:0/3913545421 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a31d9ee4a0 con 0x55a31db185b0 2024-05-03T18:27:59.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.524+0000 7f5198c00700 10 client.15405 handle_command_reply: tid=0 2024-05-03T18:27:59.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.524+0000 7f51c02f8b80 1 client.15405 shutdown 2024-05-03T18:27:59.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.524+0000 7f51c02f8b80 1 -- 172.21.15.33:0/3913545421 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a31da948f0 msgr2=0x55a31dab4a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:59.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.524+0000 7f51c02f8b80 1 --2- 172.21.15.33:0/3913545421 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a31da948f0 0x55a31dab4a30 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f519c021000 tx=0x7f519c009990 comp rx=0 tx=0).stop 2024-05-03T18:27:59.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.524+0000 7f51c02f8b80 1 -- 172.21.15.33:0/3913545421 shutdown_connections 2024-05-03T18:27:59.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.524+0000 7f51c02f8b80 1 --2- 172.21.15.33:0/3913545421 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55a31db185b0 0x55a31db1a9d0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:59.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.524+0000 7f51c02f8b80 1 --2- 172.21.15.33:0/3913545421 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a31da99bd0 0x55a31dab4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:59.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.524+0000 7f51c02f8b80 1 --2- 172.21.15.33:0/3913545421 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a31da93f30 0x55a31daa63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:59.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.524+0000 7f51c02f8b80 1 --2- 172.21.15.33:0/3913545421 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a31da948f0 0x55a31dab4a30 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:59.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.524+0000 7f5198c00700 0 client.15405 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:27:59.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.524+0000 7f51c02f8b80 1 -- 172.21.15.33:0/3913545421 >> 172.21.15.33:0/3913545421 conn(0x55a31da0c6d0 msgr2=0x55a31da92f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:27:59.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.525+0000 7f51c02f8b80 1 -- 172.21.15.33:0/3913545421 shutdown_connections 2024-05-03T18:27:59.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.525+0000 7f51c02f8b80 1 -- 172.21.15.33:0/3913545421 wait complete. 2024-05-03T18:27:59.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.525+0000 7f51c02f8b80 20 client.15405 trim_cache size 0 max 16384 2024-05-03T18:27:59.527 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-03T18:27:59.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.526+0000 7f51afe00700 1 -- 172.21.15.33:0/1414130710 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f51940b2a80 msgr2=0x7f51940b4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:59.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.526+0000 7f51afe00700 1 --2- 172.21.15.33:0/1414130710 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f51940b2a80 0x7f51940b4f40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f51a8008a90 tx=0x7f51a8004b40 comp rx=0 tx=0).stop 2024-05-03T18:27:59.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.526+0000 7f51afe00700 1 -- 172.21.15.33:0/1414130710 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51b01033d0 msgr2=0x7f51b01a1800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:27:59.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.526+0000 7f51afe00700 1 --2- 172.21.15.33:0/1414130710 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51b01033d0 0x7f51b01a1800 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f51a00057c0 tx=0x7f51a0009f90 comp rx=0 tx=0).stop 2024-05-03T18:27:59.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.526+0000 7f51afe00700 1 -- 172.21.15.33:0/1414130710 shutdown_connections 2024-05-03T18:27:59.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.526+0000 7f51afe00700 1 --2- 172.21.15.33:0/1414130710 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51b0109460 0x7f51b019b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:59.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.526+0000 7f51afe00700 1 --2- 172.21.15.33:0/1414130710 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51b01033d0 0x7f51b01a1800 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:59.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.526+0000 7f51afe00700 1 --2- 172.21.15.33:0/1414130710 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f51940b2a80 0x7f51940b4f40 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:59.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.526+0000 7f51afe00700 1 --2- 172.21.15.33:0/1414130710 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f51b0103d90 0x7f51b01a1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:27:59.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.526+0000 7f51afe00700 1 -- 172.21.15.33:0/1414130710 >> 172.21.15.33:0/1414130710 conn(0x7f51b00fef20 msgr2=0x7f51b010eb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:27:59.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.526+0000 7f51afe00700 1 -- 172.21.15.33:0/1414130710 shutdown_connections 2024-05-03T18:27:59.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:27:59.526+0000 7f51afe00700 1 -- 172.21.15.33:0/1414130710 wait complete. 2024-05-03T18:28:00.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:27:59 smithi033 ceph-mon[28330]: pgmap v27: 129 pgs: 129 active+clean; 44 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 2.7 MiB/s wr, 86 op/s 2024-05-03T18:28:00.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:27:59 smithi099 ceph-mon[31658]: pgmap v27: 129 pgs: 129 active+clean; 44 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 2.7 MiB/s wr, 86 op/s 2024-05-03T18:28:00.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:27:59 smithi150 ceph-mon[32492]: pgmap v27: 129 pgs: 129 active+clean; 44 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 2.7 MiB/s wr, 86 op/s 2024-05-03T18:28:01.484 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-03T18:28:01.485 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-03T18:28:01.485 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:28:01 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:28:01.018+0000 7f51c1e00700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-03T18:28:01.485 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:28:01 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:28:01.018+0000 7f51c1e00700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-03T18:28:01.486 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:28:01 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:28:01.018+0000 7f51c1e00700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-03T18:28:01.486 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:28:01 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:28:01.018+0000 7f51c1e00700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-03T18:28:01.486 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:28:01 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:28:01.018+0000 7f51c1e00700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-03T18:28:02.019 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:28:02.823 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:28:02 smithi033 ceph-mon[28330]: pgmap v28: 129 pgs: 129 active+clean; 44 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 2.3 MiB/s wr, 71 op/s 2024-05-03T18:28:02.824 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:28:02 smithi033 ceph-mon[28330]: mgrmap e24: x(active, since 50s), standbys: z, y 2024-05-03T18:28:03.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:28:02 smithi150 ceph-mon[32492]: pgmap v28: 129 pgs: 129 active+clean; 44 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 2.3 MiB/s wr, 71 op/s 2024-05-03T18:28:03.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:28:02 smithi150 ceph-mon[32492]: mgrmap e24: x(active, since 50s), standbys: z, y 2024-05-03T18:28:03.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:28:02 smithi099 ceph-mon[31658]: pgmap v28: 129 pgs: 129 active+clean; 44 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 2.3 MiB/s wr, 71 op/s 2024-05-03T18:28:03.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:28:02 smithi099 ceph-mon[31658]: mgrmap e24: x(active, since 50s), standbys: z, y 2024-05-03T18:28:04.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:28:04 smithi033 ceph-mon[28330]: pgmap v29: 129 pgs: 129 active+clean; 46 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 2.7 MiB/s wr, 104 op/s 2024-05-03T18:28:05.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:28:04 smithi150 ceph-mon[32492]: pgmap v29: 129 pgs: 129 active+clean; 46 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 2.7 MiB/s wr, 104 op/s 2024-05-03T18:28:05.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:28:04 smithi099 ceph-mon[31658]: pgmap v29: 129 pgs: 129 active+clean; 46 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 2.7 MiB/s wr, 104 op/s 2024-05-03T18:28:06.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.040+0000 7f6bffe00700 1 -- 172.21.15.33:0/1956378341 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6bf810fed0 msgr2=0x7f6bf81122c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:28:06.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.040+0000 7f6bffe00700 1 --2- 172.21.15.33:0/1956378341 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6bf810fed0 0x7f6bf81122c0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f6bf0009990 tx=0x7f6bf0009ca0 comp rx=0 tx=0).stop 2024-05-03T18:28:06.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.040+0000 7f6bffe00700 1 -- 172.21.15.33:0/1956378341 shutdown_connections 2024-05-03T18:28:06.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.040+0000 7f6bffe00700 1 --2- 172.21.15.33:0/1956378341 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6bf810fed0 0x7f6bf81122c0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:06.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.040+0000 7f6bffe00700 1 --2- 172.21.15.33:0/1956378341 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6bf80756d0 0x7f6bf8075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:06.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.040+0000 7f6bffe00700 1 --2- 172.21.15.33:0/1956378341 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6bf8075ff0 0x7f6bf810f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:06.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.040+0000 7f6bffe00700 1 -- 172.21.15.33:0/1956378341 >> 172.21.15.33:0/1956378341 conn(0x7f6bf807a650 msgr2=0x7f6bf807aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:28:06.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.040+0000 7f6bffe00700 1 -- 172.21.15.33:0/1956378341 shutdown_connections 2024-05-03T18:28:06.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.040+0000 7f6bffe00700 1 -- 172.21.15.33:0/1956378341 wait complete. 2024-05-03T18:28:06.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.041+0000 7f6bffe00700 1 Processor -- start 2024-05-03T18:28:06.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.041+0000 7f6bffe00700 1 -- start start 2024-05-03T18:28:06.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.042+0000 7f6bffe00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6bf80756d0 0x7f6bf81a1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:28:06.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.042+0000 7f6bffe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6bf8075ff0 0x7f6bf81a1e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:28:06.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.042+0000 7f6bffe00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6bf810fed0 0x7f6bf819bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:28:06.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.042+0000 7f6bffe00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6bf81157c0 con 0x7f6bf810fed0 2024-05-03T18:28:06.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.042+0000 7f6bffe00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6bf8115640 con 0x7f6bf8075ff0 2024-05-03T18:28:06.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.042+0000 7f6bffe00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6bf8115940 con 0x7f6bf80756d0 2024-05-03T18:28:06.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.043+0000 7f6bfde00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6bf810fed0 0x7f6bf819bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:28:06.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.043+0000 7f6bfde00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6bf810fed0 0x7f6bf819bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47212/0 (socket says 172.21.15.33:47212) 2024-05-03T18:28:06.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.043+0000 7f6bfde00700 1 -- 172.21.15.33:0/376838154 learned_addr learned my addr 172.21.15.33:0/376838154 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:28:06.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.043+0000 7f6bfca00700 1 --2- 172.21.15.33:0/376838154 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6bf8075ff0 0x7f6bf81a1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:28:06.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.043+0000 7f6bfd400700 1 --2- 172.21.15.33:0/376838154 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6bf80756d0 0x7f6bf81a1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:28:06.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.043+0000 7f6bfde00700 1 -- 172.21.15.33:0/376838154 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6bf80756d0 msgr2=0x7f6bf81a1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:28:06.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.043+0000 7f6bfde00700 1 --2- 172.21.15.33:0/376838154 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6bf80756d0 0x7f6bf81a1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:06.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.043+0000 7f6bfde00700 1 -- 172.21.15.33:0/376838154 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6bf8075ff0 msgr2=0x7f6bf81a1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:28:06.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.043+0000 7f6bfde00700 1 --2- 172.21.15.33:0/376838154 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6bf8075ff0 0x7f6bf81a1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:06.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.043+0000 7f6bfde00700 1 -- 172.21.15.33:0/376838154 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bf819c220 con 0x7f6bf810fed0 2024-05-03T18:28:06.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.044+0000 7f6bfde00700 1 --2- 172.21.15.33:0/376838154 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6bf810fed0 0x7f6bf819bac0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f6bf00057f0 tx=0x7f6bf0005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:28:06.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.044+0000 7f6bf6000700 1 -- 172.21.15.33:0/376838154 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf0027070 con 0x7f6bf810fed0 2024-05-03T18:28:06.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.044+0000 7f6bf6000700 1 -- 172.21.15.33:0/376838154 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6bf000fb00 con 0x7f6bf810fed0 2024-05-03T18:28:06.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.044+0000 7f6bf6000700 1 -- 172.21.15.33:0/376838154 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf0022b20 con 0x7f6bf810fed0 2024-05-03T18:28:06.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.044+0000 7f6bffe00700 1 -- 172.21.15.33:0/376838154 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bf819c4b0 con 0x7f6bf810fed0 2024-05-03T18:28:06.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.044+0000 7f6bffe00700 1 -- 172.21.15.33:0/376838154 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bf81a87d0 con 0x7f6bf810fed0 2024-05-03T18:28:06.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.048+0000 7f6bf6000700 1 -- 172.21.15.33:0/376838154 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 24) ==== 149885+0+0 (secure 0 0 0) 0x7f6bf002a070 con 0x7f6bf810fed0 2024-05-03T18:28:06.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.049+0000 7f6c0a877b80 1 Processor -- start 2024-05-03T18:28:06.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.050+0000 7f6bf6000700 1 --2- 172.21.15.33:0/376838154 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f6be00b2b90 0x7f6be00b5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:28:06.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.050+0000 7f6c0a877b80 1 -- start start 2024-05-03T18:28:06.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.050+0000 7f6bf6000700 1 -- 172.21.15.33:0/376838154 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f6bf00e3b00 con 0x7f6bf810fed0 2024-05-03T18:28:06.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.050+0000 7f6bfca00700 1 --2- 172.21.15.33:0/376838154 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f6be00b2b90 0x7f6be00b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:28:06.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.050+0000 7f6c0a877b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cb474001b0 0x55cb47400590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:28:06.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.050+0000 7f6c0a877b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cb47400ad0 0x55cb473fa840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:28:06.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.050+0000 7f6c0a877b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb473fae60 0x55cb473fb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:28:06.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.051+0000 7f6c0a877b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55cb47344aa0 con 0x55cb47400ad0 2024-05-03T18:28:06.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.051+0000 7f6c0a877b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55cb472e27e0 con 0x55cb473fae60 2024-05-03T18:28:06.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.051+0000 7f6c0a877b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55cb47340a60 con 0x55cb474001b0 2024-05-03T18:28:06.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.051+0000 7f6bfca00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cb47400ad0 0x55cb473fa840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:28:06.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.051+0000 7f6bfd400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cb474001b0 0x55cb47400590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:28:06.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.051+0000 7f6bfca00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cb47400ad0 0x55cb473fa840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47228/0 (socket says 172.21.15.33:47228) 2024-05-03T18:28:06.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.051+0000 7f6bfca00700 1 -- 172.21.15.33:0/189929993 learned_addr learned my addr 172.21.15.33:0/189929993 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:28:06.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.052+0000 7f6bfca00700 1 --2- 172.21.15.33:0/376838154 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f6be00b2b90 0x7f6be00b5050 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6be4013570 tx=0x7f6be4013930 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:28:06.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.052+0000 7f6bfd400700 1 -- 172.21.15.33:0/189929993 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb473fae60 msgr2=0x55cb473fb2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:28:06.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.052+0000 7f6bfd400700 1 --2- 172.21.15.33:0/189929993 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb473fae60 0x55cb473fb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:06.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.052+0000 7f6bfd400700 1 -- 172.21.15.33:0/189929993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cb47400ad0 msgr2=0x55cb473fa840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:28:06.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.052+0000 7f6bfd400700 1 --2- 172.21.15.33:0/189929993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cb47400ad0 0x55cb473fa840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:06.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.052+0000 7f6bfd400700 1 -- 172.21.15.33:0/189929993 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb47291950 con 0x55cb474001b0 2024-05-03T18:28:06.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.052+0000 7f6bfca00700 1 --2- 172.21.15.33:0/189929993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cb47400ad0 0x55cb473fa840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:28:06.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.052+0000 7f6bfd400700 1 --2- 172.21.15.33:0/189929993 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cb474001b0 0x55cb47400590 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6bec003f70 tx=0x7f6bec009ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16566b782dfa12e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:28:06.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.053+0000 7f6beaa00700 1 -- 172.21.15.33:0/189929993 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bec004610 con 0x55cb474001b0 2024-05-03T18:28:06.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.053+0000 7f6beaa00700 1 -- 172.21.15.33:0/189929993 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6bec01a070 con 0x55cb474001b0 2024-05-03T18:28:06.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.053+0000 7f6beaa00700 1 -- 172.21.15.33:0/189929993 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bec0156e0 con 0x55cb474001b0 2024-05-03T18:28:06.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.054+0000 7f6c0a877b80 1 -- 172.21.15.33:0/189929993 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cb474001b0 msgr2=0x55cb47400590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:28:06.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.054+0000 7f6c0a877b80 1 --2- 172.21.15.33:0/189929993 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cb474001b0 0x55cb47400590 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6bec003f70 tx=0x7f6bec009ba0 comp rx=0 tx=0).stop 2024-05-03T18:28:06.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.054+0000 7f6c0a877b80 1 -- 172.21.15.33:0/189929993 shutdown_connections 2024-05-03T18:28:06.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.054+0000 7f6c0a877b80 1 --2- 172.21.15.33:0/189929993 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb473fae60 0x55cb473fb2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:06.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.054+0000 7f6c0a877b80 1 --2- 172.21.15.33:0/189929993 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cb474001b0 0x55cb47400590 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:06.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.054+0000 7f6c0a877b80 1 --2- 172.21.15.33:0/189929993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cb47400ad0 0x55cb473fa840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:06.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.054+0000 7f6c0a877b80 1 -- 172.21.15.33:0/189929993 >> 172.21.15.33:0/189929993 conn(0x55cb47320be0 msgr2=0x55cb473ec870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:28:06.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.054+0000 7f6c0a877b80 1 -- 172.21.15.33:0/189929993 shutdown_connections 2024-05-03T18:28:06.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.054+0000 7f6c0a877b80 1 -- 172.21.15.33:0/189929993 wait complete. 2024-05-03T18:28:06.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.055+0000 7f6c0a877b80 1 -- start start 2024-05-03T18:28:06.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.055+0000 7f6c0a877b80 1 Processor -- start 2024-05-03T18:28:06.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.056+0000 7f6c0a877b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cb473fae60 0x55cb47404140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:28:06.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.056+0000 7f6c0a877b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cb474001b0 0x55cb47404680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:28:06.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.056+0000 7f6c0a877b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb47400ad0 0x55cb47404bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:28:06.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.056+0000 7f6c0a877b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55cb473f2460 con 0x55cb474001b0 2024-05-03T18:28:06.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.056+0000 7f6c0a877b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55cb472aaba0 con 0x55cb47400ad0 2024-05-03T18:28:06.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.056+0000 7f6c0a877b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55cb473f25e0 con 0x55cb473fae60 2024-05-03T18:28:06.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.056+0000 7f6c0a877b80 10 client.0 _ll_register_callbacks cb 0x55cb472ebcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:28:06.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.056+0000 7f6bfd400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cb473fae60 0x55cb47404140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:28:06.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.056+0000 7f6bfd400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cb473fae60 0x55cb47404140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:60490/0 (socket says 172.21.15.33:60490) 2024-05-03T18:28:06.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.056+0000 7f6bfd400700 1 -- 172.21.15.33:0/2145791838 learned_addr learned my addr 172.21.15.33:0/2145791838 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:28:06.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.056+0000 7f6bfde00700 1 --2- 172.21.15.33:0/2145791838 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb47400ad0 0x55cb47404bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:28:06.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.057+0000 7f6bfca00700 1 --2- 172.21.15.33:0/2145791838 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cb474001b0 0x55cb47404680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:28:06.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.057+0000 7f6bfd400700 1 -- 172.21.15.33:0/2145791838 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb47400ad0 msgr2=0x55cb47404bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:28:06.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.057+0000 7f6bfd400700 1 --2- 172.21.15.33:0/2145791838 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb47400ad0 0x55cb47404bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:06.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.057+0000 7f6bfd400700 1 -- 172.21.15.33:0/2145791838 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cb474001b0 msgr2=0x55cb47404680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:28:06.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.057+0000 7f6bfd400700 1 --2- 172.21.15.33:0/2145791838 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cb474001b0 0x55cb47404680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:06.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.057+0000 7f6bfd400700 1 -- 172.21.15.33:0/2145791838 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb4730a4a0 con 0x55cb473fae60 2024-05-03T18:28:06.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.057+0000 7f6bfde00700 1 --2- 172.21.15.33:0/2145791838 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb47400ad0 0x55cb47404bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:28:06.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.057+0000 7f6bfd400700 1 --2- 172.21.15.33:0/2145791838 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cb473fae60 0x55cb47404140 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6bec000f80 tx=0x7f6bec0048f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90c31a412ed739d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:28:06.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.057+0000 7f6bdf400700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-03T18:28:06.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.058+0000 7f6bdf400700 1 -- 172.21.15.33:0/2145791838 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bec007a10 con 0x55cb473fae60 2024-05-03T18:28:06.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.058+0000 7f6c0a877b80 1 -- 172.21.15.33:0/2145791838 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6bec01a070 con 0x55cb473fae60 2024-05-03T18:28:06.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.058+0000 7f6bdf400700 1 -- 172.21.15.33:0/2145791838 <== mon.1 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f6bec015480 con 0x55cb473fae60 2024-05-03T18:28:06.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.058+0000 7f6bdf400700 1 -- 172.21.15.33:0/2145791838 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f6bec015680 con 0x55cb473fae60 2024-05-03T18:28:06.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.058+0000 7f6c0a877b80 10 client.25156 fetch_fsmap learned FSMap version 21 2024-05-03T18:28:06.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.058+0000 7f6c0a877b80 1 -- 172.21.15.33:0/2145791838 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55cb4741a680 con 0x55cb473fae60 2024-05-03T18:28:06.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.059+0000 7f6bdf400700 1 -- 172.21.15.33:0/2145791838 <== mon.1 v2:172.21.15.150:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6bec01a070 con 0x55cb473fae60 2024-05-03T18:28:06.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.059+0000 7f6bdf400700 1 -- 172.21.15.33:0/2145791838 <== mon.1 v2:172.21.15.150:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f6bec0276e0 con 0x55cb473fae60 2024-05-03T18:28:06.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.059+0000 7f6c0a877b80 10 client.25156 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:28:06.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.059+0000 7f6c0a877b80 10 client.25156 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:28:06.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.059+0000 7f6c0a877b80 20 client.25156 populate_metadata read hostname 'smithi033' 2024-05-03T18:28:06.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.059+0000 7f6c0a877b80 1 --2- 172.21.15.33:0/2145791838 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55cb4741cbb0 0x55cb4741efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:28:06.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.059+0000 7f6c0a877b80 4 client.25156 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:28:06.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.059+0000 7f6c0a877b80 1 -- 172.21.15.33:0/2145791838 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cb473bf250 con 0x55cb4741cbb0 2024-05-03T18:28:06.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.060+0000 7f6bfca00700 1 --2- 172.21.15.33:0/2145791838 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55cb4741cbb0 0x55cb4741efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:28:06.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.061+0000 7f6bfca00700 1 --2- 172.21.15.33:0/2145791838 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55cb4741cbb0 0x55cb4741efa0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ad6fded061773d server_cookie=43d33c11b2cd9dba in_seq=0 out_seq=0 2024-05-03T18:28:06.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.061+0000 7f6bdf400700 10 client.25156 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:28:06.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.064+0000 7f6bdf400700 1 -- 172.21.15.33:0/2145791838 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55cb473bf250 con 0x55cb4741cbb0 2024-05-03T18:28:06.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.064+0000 7f6bdf400700 10 client.25156 handle_command_reply: tid=0 2024-05-03T18:28:06.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.064+0000 7f6c0a877b80 1 client.25156 shutdown 2024-05-03T18:28:06.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.064+0000 7f6c0a877b80 1 -- 172.21.15.33:0/2145791838 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cb473fae60 msgr2=0x55cb47404140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:28:06.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.064+0000 7f6c0a877b80 1 --2- 172.21.15.33:0/2145791838 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cb473fae60 0x55cb47404140 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6bec000f80 tx=0x7f6bec0048f0 comp rx=0 tx=0).stop 2024-05-03T18:28:06.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.065+0000 7f6c0a877b80 1 -- 172.21.15.33:0/2145791838 shutdown_connections 2024-05-03T18:28:06.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.065+0000 7f6c0a877b80 1 --2- 172.21.15.33:0/2145791838 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55cb4741cbb0 0x55cb4741efa0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:06.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.065+0000 7f6c0a877b80 1 --2- 172.21.15.33:0/2145791838 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb47400ad0 0x55cb47404bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:06.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.065+0000 7f6c0a877b80 1 --2- 172.21.15.33:0/2145791838 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cb473fae60 0x55cb47404140 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:06.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.065+0000 7f6c0a877b80 1 --2- 172.21.15.33:0/2145791838 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cb474001b0 0x55cb47404680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:06.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.065+0000 7f6bdf400700 0 client.25156 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:28:06.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.065+0000 7f6c0a877b80 1 -- 172.21.15.33:0/2145791838 >> 172.21.15.33:0/2145791838 conn(0x55cb47320be0 msgr2=0x55cb473ff5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:28:06.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.065+0000 7f6c0a877b80 1 -- 172.21.15.33:0/2145791838 shutdown_connections 2024-05-03T18:28:06.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.065+0000 7f6c0a877b80 1 -- 172.21.15.33:0/2145791838 wait complete. 2024-05-03T18:28:06.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.065+0000 7f6c0a877b80 20 client.25156 trim_cache size 0 max 16384 2024-05-03T18:28:06.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.089+0000 7f6c0a877b80 1 Processor -- start 2024-05-03T18:28:06.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.089+0000 7f6c0a877b80 1 -- start start 2024-05-03T18:28:06.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.090+0000 7f6c0a877b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb473fae60 0x55cb474046b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:28:06.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.090+0000 7f6c0a877b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cb474001b0 0x55cb47404bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:28:06.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.090+0000 7f6c0a877b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cb47400ad0 0x55cb473f8870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:28:06.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.090+0000 7f6c0a877b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6bf81ab080 con 0x55cb474001b0 2024-05-03T18:28:06.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.090+0000 7f6c0a877b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6bf81ad900 con 0x55cb473fae60 2024-05-03T18:28:06.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.090+0000 7f6c0a877b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6bf81ab6f0 con 0x55cb47400ad0 2024-05-03T18:28:06.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.090+0000 7f6bfca00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cb474001b0 0x55cb47404bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:28:06.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.090+0000 7f6bfd400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb473fae60 0x55cb474046b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:28:06.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.090+0000 7f6bfd400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb473fae60 0x55cb474046b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:46676/0 (socket says 172.21.15.33:46676) 2024-05-03T18:28:06.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.090+0000 7f6bfd400700 1 -- 172.21.15.33:0/3300656676 learned_addr learned my addr 172.21.15.33:0/3300656676 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:28:06.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.091+0000 7f6bfde00700 1 --2- 172.21.15.33:0/3300656676 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cb47400ad0 0x55cb473f8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:28:06.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.091+0000 7f6bfca00700 1 -- 172.21.15.33:0/3300656676 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cb47400ad0 msgr2=0x55cb473f8870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:28:06.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.091+0000 7f6bfca00700 1 --2- 172.21.15.33:0/3300656676 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cb47400ad0 0x55cb473f8870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:06.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.091+0000 7f6bfca00700 1 -- 172.21.15.33:0/3300656676 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb473fae60 msgr2=0x55cb474046b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:28:06.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.091+0000 7f6bfca00700 1 --2- 172.21.15.33:0/3300656676 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb473fae60 0x55cb474046b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:06.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.091+0000 7f6bfca00700 1 -- 172.21.15.33:0/3300656676 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb473e7820 con 0x55cb474001b0 2024-05-03T18:28:06.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.091+0000 7f6bfca00700 1 --2- 172.21.15.33:0/3300656676 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cb474001b0 0x55cb47404bf0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f6be400ae70 tx=0x7f6be401b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e83616644d1b653b server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:28:06.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.092+0000 7f6bea000700 1 -- 172.21.15.33:0/3300656676 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6be4003c60 con 0x55cb474001b0 2024-05-03T18:28:06.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.092+0000 7f6bea000700 1 -- 172.21.15.33:0/3300656676 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6be402f070 con 0x55cb474001b0 2024-05-03T18:28:06.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.092+0000 7f6bea000700 1 -- 172.21.15.33:0/3300656676 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6be40207f0 con 0x55cb474001b0 2024-05-03T18:28:06.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.093+0000 7f6c0a877b80 1 -- 172.21.15.33:0/3300656676 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cb474001b0 msgr2=0x55cb47404bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:28:06.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.093+0000 7f6c0a877b80 1 --2- 172.21.15.33:0/3300656676 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cb474001b0 0x55cb47404bf0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f6be400ae70 tx=0x7f6be401b000 comp rx=0 tx=0).stop 2024-05-03T18:28:06.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.093+0000 7f6c0a877b80 1 -- 172.21.15.33:0/3300656676 shutdown_connections 2024-05-03T18:28:06.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.093+0000 7f6c0a877b80 1 --2- 172.21.15.33:0/3300656676 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cb47400ad0 0x55cb473f8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:06.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.093+0000 7f6c0a877b80 1 --2- 172.21.15.33:0/3300656676 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb473fae60 0x55cb474046b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:06.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.093+0000 7f6c0a877b80 1 --2- 172.21.15.33:0/3300656676 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cb474001b0 0x55cb47404bf0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:06.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.093+0000 7f6c0a877b80 1 -- 172.21.15.33:0/3300656676 >> 172.21.15.33:0/3300656676 conn(0x55cb47320be0 msgr2=0x55cb473ebf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:28:06.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.093+0000 7f6c0a877b80 1 -- 172.21.15.33:0/3300656676 shutdown_connections 2024-05-03T18:28:06.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.093+0000 7f6c0a877b80 1 -- 172.21.15.33:0/3300656676 wait complete. 2024-05-03T18:28:06.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.093+0000 7f6c0a877b80 1 -- start start 2024-05-03T18:28:06.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.094+0000 7f6c0a877b80 1 Processor -- start 2024-05-03T18:28:06.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.094+0000 7f6c0a877b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb473fae60 0x55cb473f9080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:28:06.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.094+0000 7f6c0a877b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cb474001b0 0x55cb473f7430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:28:06.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.094+0000 7f6c0a877b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cb47400ad0 0x55cb473f7970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:28:06.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.094+0000 7f6c0a877b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6bf81abdc0 con 0x55cb474001b0 2024-05-03T18:28:06.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.094+0000 7f6c0a877b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6bf81ad230 con 0x55cb473fae60 2024-05-03T18:28:06.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.094+0000 7f6c0a877b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6bf81ac490 con 0x55cb47400ad0 2024-05-03T18:28:06.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.094+0000 7f6c0a877b80 10 client.0 _ll_register_callbacks cb 0x55cb472ebcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:28:06.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.095+0000 7f6bfca00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cb474001b0 0x55cb473f7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:28:06.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.095+0000 7f6bfca00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cb474001b0 0x55cb473f7430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47244/0 (socket says 172.21.15.33:47244) 2024-05-03T18:28:06.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.095+0000 7f6bfca00700 1 -- 172.21.15.33:0/1744849477 learned_addr learned my addr 172.21.15.33:0/1744849477 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:28:06.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.095+0000 7f6bfd400700 1 --2- 172.21.15.33:0/1744849477 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb473fae60 0x55cb473f9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:28:06.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.095+0000 7f6bfde00700 1 --2- 172.21.15.33:0/1744849477 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cb47400ad0 0x55cb473f7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:28:06.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.095+0000 7f6bfca00700 1 -- 172.21.15.33:0/1744849477 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cb47400ad0 msgr2=0x55cb473f7970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:28:06.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.095+0000 7f6bfca00700 1 --2- 172.21.15.33:0/1744849477 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cb47400ad0 0x55cb473f7970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:06.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.095+0000 7f6bfca00700 1 -- 172.21.15.33:0/1744849477 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb473fae60 msgr2=0x55cb473f9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:28:06.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.095+0000 7f6bfca00700 1 --2- 172.21.15.33:0/1744849477 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb473fae60 0x55cb473f9080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:06.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.095+0000 7f6bfca00700 1 -- 172.21.15.33:0/1744849477 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb473ee4f0 con 0x55cb474001b0 2024-05-03T18:28:06.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.096+0000 7f6bfca00700 1 --2- 172.21.15.33:0/1744849477 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cb474001b0 0x55cb473f7430 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f6be4005fa0 tx=0x7f6be4003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bca89dbdd178a1a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:28:06.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.096+0000 7f6bdf400700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:28:06.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.096+0000 7f6bdf400700 1 -- 172.21.15.33:0/1744849477 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6be4005510 con 0x55cb474001b0 2024-05-03T18:28:06.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.096+0000 7f6c0a877b80 1 -- 172.21.15.33:0/1744849477 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6be402f070 con 0x55cb474001b0 2024-05-03T18:28:06.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.096+0000 7f6bdf400700 1 -- 172.21.15.33:0/1744849477 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f6be402b430 con 0x55cb474001b0 2024-05-03T18:28:06.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.096+0000 7f6bdf400700 1 -- 172.21.15.33:0/1744849477 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f6be4045050 con 0x55cb474001b0 2024-05-03T18:28:06.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.096+0000 7f6c0a877b80 10 client.15429 fetch_fsmap learned FSMap version 21 2024-05-03T18:28:06.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.096+0000 7f6c0a877b80 1 -- 172.21.15.33:0/1744849477 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55cb47487d60 con 0x55cb474001b0 2024-05-03T18:28:06.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.097+0000 7f6bdf400700 1 -- 172.21.15.33:0/1744849477 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6be402f070 con 0x55cb474001b0 2024-05-03T18:28:06.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.097+0000 7f6bdf400700 1 -- 172.21.15.33:0/1744849477 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f6be4004a00 con 0x55cb474001b0 2024-05-03T18:28:06.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.097+0000 7f6c0a877b80 10 client.15429 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:28:06.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.097+0000 7f6c0a877b80 10 client.15429 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:28:06.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.097+0000 7f6c0a877b80 20 client.15429 populate_metadata read hostname 'smithi033' 2024-05-03T18:28:06.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.097+0000 7f6c0a877b80 1 --2- 172.21.15.33:0/1744849477 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55cb47488490 0x55cb4748a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:28:06.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.097+0000 7f6c0a877b80 4 client.15429 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-03T18:28:06.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.097+0000 7f6c0a877b80 1 -- 172.21.15.33:0/1744849477 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55cb47352480 con 0x55cb47488490 2024-05-03T18:28:06.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.098+0000 7f6bfd400700 1 --2- 172.21.15.33:0/1744849477 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55cb47488490 0x55cb4748a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:28:06.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.099+0000 7f6bfd400700 1 --2- 172.21.15.33:0/1744849477 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55cb47488490 0x55cb4748a8b0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=381848d5244b0b0e server_cookie=df9b4726184eba7c in_seq=0 out_seq=0 2024-05-03T18:28:06.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.099+0000 7f6bdf400700 10 client.15429 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:28:06.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.108+0000 7f6bdf400700 1 -- 172.21.15.33:0/1744849477 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55cb47352480 con 0x55cb47488490 2024-05-03T18:28:06.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.108+0000 7f6bdf400700 10 client.15429 handle_command_reply: tid=0 2024-05-03T18:28:06.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.108+0000 7f6c0a877b80 1 client.15429 shutdown 2024-05-03T18:28:06.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.109+0000 7f6c0a877b80 1 -- 172.21.15.33:0/1744849477 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cb474001b0 msgr2=0x55cb473f7430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:28:06.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.109+0000 7f6c0a877b80 1 --2- 172.21.15.33:0/1744849477 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cb474001b0 0x55cb473f7430 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f6be4005fa0 tx=0x7f6be4003b90 comp rx=0 tx=0).stop 2024-05-03T18:28:06.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.109+0000 7f6c0a877b80 1 -- 172.21.15.33:0/1744849477 shutdown_connections 2024-05-03T18:28:06.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.109+0000 7f6c0a877b80 1 --2- 172.21.15.33:0/1744849477 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55cb47488490 0x55cb4748a8b0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:06.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.109+0000 7f6c0a877b80 1 --2- 172.21.15.33:0/1744849477 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cb47400ad0 0x55cb473f7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:06.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.109+0000 7f6c0a877b80 1 --2- 172.21.15.33:0/1744849477 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cb473fae60 0x55cb473f9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:06.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.109+0000 7f6c0a877b80 1 --2- 172.21.15.33:0/1744849477 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cb474001b0 0x55cb473f7430 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:06.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.109+0000 7f6c0a877b80 1 -- 172.21.15.33:0/1744849477 >> 172.21.15.33:0/1744849477 conn(0x55cb47320be0 msgr2=0x55cb473fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:28:06.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.109+0000 7f6bdf400700 0 client.15429 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:28:06.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.109+0000 7f6c0a877b80 1 -- 172.21.15.33:0/1744849477 shutdown_connections 2024-05-03T18:28:06.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.109+0000 7f6c0a877b80 1 -- 172.21.15.33:0/1744849477 wait complete. 2024-05-03T18:28:06.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.110+0000 7f6c0a877b80 20 client.15429 trim_cache size 0 max 16384 2024-05-03T18:28:06.111 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-03T18:28:06.112 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-05-03T18:28:06.112 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "0495ecba-a7a5-45a5-a00c-035f6ae59f83", 2024-05-03T18:28:06.112 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-05-03T18:28:06.112 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-03T18:28:06.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.111+0000 7f6bffe00700 1 -- 172.21.15.33:0/376838154 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f6be00b2b90 msgr2=0x7f6be00b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:28:06.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.111+0000 7f6bffe00700 1 --2- 172.21.15.33:0/376838154 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f6be00b2b90 0x7f6be00b5050 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6be4013570 tx=0x7f6be4013930 comp rx=0 tx=0).stop 2024-05-03T18:28:06.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.111+0000 7f6bffe00700 1 -- 172.21.15.33:0/376838154 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6bf810fed0 msgr2=0x7f6bf819bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:28:06.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.111+0000 7f6bffe00700 1 --2- 172.21.15.33:0/376838154 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6bf810fed0 0x7f6bf819bac0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f6bf00057f0 tx=0x7f6bf0005ef0 comp rx=0 tx=0).stop 2024-05-03T18:28:06.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.111+0000 7f6bffe00700 1 -- 172.21.15.33:0/376838154 shutdown_connections 2024-05-03T18:28:06.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.111+0000 7f6bffe00700 1 --2- 172.21.15.33:0/376838154 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6bf810fed0 0x7f6bf819bac0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:06.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.111+0000 7f6bffe00700 1 --2- 172.21.15.33:0/376838154 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6bf80756d0 0x7f6bf81a1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:06.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.111+0000 7f6bffe00700 1 --2- 172.21.15.33:0/376838154 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f6be00b2b90 0x7f6be00b5050 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:06.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.111+0000 7f6bffe00700 1 --2- 172.21.15.33:0/376838154 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6bf8075ff0 0x7f6bf81a1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:06.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.111+0000 7f6bffe00700 1 -- 172.21.15.33:0/376838154 >> 172.21.15.33:0/376838154 conn(0x7f6bf807a650 msgr2=0x7f6bf8100740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:28:06.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.111+0000 7f6bffe00700 1 -- 172.21.15.33:0/376838154 shutdown_connections 2024-05-03T18:28:06.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:06.112+0000 7f6bffe00700 1 -- 172.21.15.33:0/376838154 wait complete. 2024-05-03T18:28:07.049 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-03T18:28:07.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:28:06 smithi150 ceph-mon[32492]: pgmap v30: 129 pgs: 129 active+clean; 46 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 1.3 MiB/s wr, 59 op/s 2024-05-03T18:28:07.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:28:06 smithi150 ceph-mon[32492]: scrub queued for path: / 2024-05-03T18:28:07.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:28:06 smithi150 ceph-mon[32492]: scrub summary: idle+waiting paths [/] 2024-05-03T18:28:07.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:28:06 smithi150 ceph-mon[32492]: scrub summary: active paths [/] 2024-05-03T18:28:07.136 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:28:06 smithi033 ceph-mon[28330]: pgmap v30: 129 pgs: 129 active+clean; 46 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 1.3 MiB/s wr, 59 op/s 2024-05-03T18:28:07.136 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:28:06 smithi033 ceph-mon[28330]: scrub queued for path: / 2024-05-03T18:28:07.136 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:28:06 smithi033 ceph-mon[28330]: scrub summary: idle+waiting paths [/] 2024-05-03T18:28:07.136 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:28:06 smithi033 ceph-mon[28330]: scrub summary: active paths [/] 2024-05-03T18:28:07.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:28:06 smithi099 ceph-mon[31658]: pgmap v30: 129 pgs: 129 active+clean; 46 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 1.3 MiB/s wr, 59 op/s 2024-05-03T18:28:07.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:28:06 smithi099 ceph-mon[31658]: scrub queued for path: / 2024-05-03T18:28:07.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:28:06 smithi099 ceph-mon[31658]: scrub summary: idle+waiting paths [/] 2024-05-03T18:28:07.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:28:06 smithi099 ceph-mon[31658]: scrub summary: active paths [/] 2024-05-03T18:28:07.556 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:28:07.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:28:07 smithi033 ceph-mon[28330]: scrub summary: idle+waiting paths [/] 2024-05-03T18:28:08.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:28:07 smithi150 ceph-mon[32492]: scrub summary: idle+waiting paths [/] 2024-05-03T18:28:08.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:28:07 smithi099 ceph-mon[31658]: scrub summary: idle+waiting paths [/] 2024-05-03T18:28:08.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:28:08 smithi033 ceph-mon[28330]: pgmap v31: 129 pgs: 129 active+clean; 46 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 1.3 MiB/s wr, 61 op/s 2024-05-03T18:28:09.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:28:08 smithi150 ceph-mon[32492]: pgmap v31: 129 pgs: 129 active+clean; 46 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 1.3 MiB/s wr, 61 op/s 2024-05-03T18:28:09.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:28:08 smithi099 ceph-mon[31658]: pgmap v31: 129 pgs: 129 active+clean; 46 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 1.3 MiB/s wr, 61 op/s 2024-05-03T18:28:09.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:09.999+0000 7f9689a00700 1 -- 172.21.15.33:0/1216673476 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f968410b6c0 msgr2=0x7f968410baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:28:09.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:09.999+0000 7f9689a00700 1 --2- 172.21.15.33:0/1216673476 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f968410b6c0 0x7f968410baa0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f9674009990 tx=0x7f9674009ca0 comp rx=0 tx=0).stop 2024-05-03T18:28:09.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:09.999+0000 7f9689a00700 1 -- 172.21.15.33:0/1216673476 shutdown_connections 2024-05-03T18:28:09.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:09.999+0000 7f9689a00700 1 --2- 172.21.15.33:0/1216673476 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9684106080 0x7f9684112590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:09.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:09.999+0000 7f9689a00700 1 --2- 172.21.15.33:0/1216673476 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f968410b6c0 0x7f968410baa0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:10.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:09.999+0000 7f9689a00700 1 --2- 172.21.15.33:0/1216673476 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96841056c0 0x7f9684105b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:10.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:09.999+0000 7f9689a00700 1 -- 172.21.15.33:0/1216673476 >> 172.21.15.33:0/1216673476 conn(0x7f96841011b0 msgr2=0x7f96841035d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:28:10.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:09.999+0000 7f9689a00700 1 -- 172.21.15.33:0/1216673476 shutdown_connections 2024-05-03T18:28:10.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:09.999+0000 7f9689a00700 1 -- 172.21.15.33:0/1216673476 wait complete. 2024-05-03T18:28:10.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.000+0000 7f9689a00700 1 Processor -- start 2024-05-03T18:28:10.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.000+0000 7f9689a00700 1 -- start start 2024-05-03T18:28:10.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.001+0000 7f9689a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96841056c0 0x7f96841a18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:28:10.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.001+0000 7f9689a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9684106080 0x7f96841a1df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:28:10.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.001+0000 7f9689a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f968410b6c0 0x7f968419ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:28:10.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.001+0000 7f9689a00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f96841157b0 con 0x7f968410b6c0 2024-05-03T18:28:10.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.001+0000 7f9689a00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9684115630 con 0x7f96841056c0 2024-05-03T18:28:10.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.001+0000 7f9689a00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9684115930 con 0x7f9684106080 2024-05-03T18:28:10.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.001+0000 7f9683400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f968410b6c0 0x7f968419ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:28:10.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.001+0000 7f9683400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f968410b6c0 0x7f968419ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47270/0 (socket says 172.21.15.33:47270) 2024-05-03T18:28:10.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.001+0000 7f9683400700 1 -- 172.21.15.33:0/2182405688 learned_addr learned my addr 172.21.15.33:0/2182405688 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:28:10.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.001+0000 7f9682a00700 1 --2- 172.21.15.33:0/2182405688 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96841056c0 0x7f96841a18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:28:10.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.001+0000 7f9682000700 1 --2- 172.21.15.33:0/2182405688 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9684106080 0x7f96841a1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:28:10.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.002+0000 7f9683400700 1 -- 172.21.15.33:0/2182405688 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9684106080 msgr2=0x7f96841a1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:28:10.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.002+0000 7f9683400700 1 --2- 172.21.15.33:0/2182405688 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9684106080 0x7f96841a1df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:10.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.002+0000 7f9683400700 1 -- 172.21.15.33:0/2182405688 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96841056c0 msgr2=0x7f96841a18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:28:10.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.002+0000 7f9683400700 1 --2- 172.21.15.33:0/2182405688 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96841056c0 0x7f96841a18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:10.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.002+0000 7f9683400700 1 -- 172.21.15.33:0/2182405688 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f968419c1d0 con 0x7f968410b6c0 2024-05-03T18:28:10.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.002+0000 7f9683400700 1 --2- 172.21.15.33:0/2182405688 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f968410b6c0 0x7f968419ba70 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f967c00ada0 tx=0x7f967c00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:28:10.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.002+0000 7f967b400700 1 -- 172.21.15.33:0/2182405688 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f967c017020 con 0x7f968410b6c0 2024-05-03T18:28:10.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.003+0000 7f967b400700 1 -- 172.21.15.33:0/2182405688 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f967c00cd60 con 0x7f968410b6c0 2024-05-03T18:28:10.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.003+0000 7f967b400700 1 -- 172.21.15.33:0/2182405688 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f967c005280 con 0x7f968410b6c0 2024-05-03T18:28:10.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.003+0000 7f9689a00700 1 -- 172.21.15.33:0/2182405688 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f968419c4c0 con 0x7f968410b6c0 2024-05-03T18:28:10.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.003+0000 7f9689a00700 1 -- 172.21.15.33:0/2182405688 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96841a8780 con 0x7f968410b6c0 2024-05-03T18:28:10.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.007+0000 7f967b400700 1 -- 172.21.15.33:0/2182405688 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 24) ==== 149885+0+0 (secure 0 0 0) 0x7f967c00c8d0 con 0x7f968410b6c0 2024-05-03T18:28:10.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.008+0000 7f96942f0b80 1 Processor -- start 2024-05-03T18:28:10.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.008+0000 7f96942f0b80 1 -- start start 2024-05-03T18:28:10.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.008+0000 7f96942f0b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c60aa3930 0x563c60aa3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:28:10.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.008+0000 7f96942f0b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c60aa4250 0x563c60a9df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:28:10.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.008+0000 7f96942f0b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563c60a9e550 0x563c60a9e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:28:10.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.008+0000 7f96942f0b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563c609f0a20 con 0x563c60aa4250 2024-05-03T18:28:10.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.008+0000 7f96942f0b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x563c6098e760 con 0x563c60aa3930 2024-05-03T18:28:10.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.008+0000 7f96942f0b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x563c609ec9e0 con 0x563c60a9e550 2024-05-03T18:28:10.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.009+0000 7f9682a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c60aa4250 0x563c60a9df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:28:10.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.009+0000 7f967b400700 1 --2- 172.21.15.33:0/2182405688 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f96680b2b90 0x7f96680b5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:28:10.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.009+0000 7f9682a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c60aa4250 0x563c60a9df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47276/0 (socket says 172.21.15.33:47276) 2024-05-03T18:28:10.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.009+0000 7f9682a00700 1 -- 172.21.15.33:0/3306539071 learned_addr learned my addr 172.21.15.33:0/3306539071 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:28:10.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.009+0000 7f9682000700 1 --2- 172.21.15.33:0/3306539071 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c60aa3930 0x563c60aa3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:28:10.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.009+0000 7f967b400700 1 -- 172.21.15.33:0/2182405688 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f967c0e2310 con 0x7f968410b6c0 2024-05-03T18:28:10.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.009+0000 7f9682000700 1 --2- 172.21.15.33:0/3306539071 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563c60a9e550 0x563c60a9e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:28:10.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.009+0000 7f9683400700 1 --2- 172.21.15.33:0/2182405688 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f96680b2b90 0x7f96680b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:28:10.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.010+0000 7f9682a00700 1 -- 172.21.15.33:0/3306539071 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563c60a9e550 msgr2=0x563c60a9e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:28:10.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.010+0000 7f9682a00700 1 --2- 172.21.15.33:0/3306539071 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563c60a9e550 0x563c60a9e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:10.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.010+0000 7f9682a00700 1 -- 172.21.15.33:0/3306539071 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c60aa3930 msgr2=0x563c60aa3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:28:10.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.010+0000 7f9682a00700 1 --2- 172.21.15.33:0/3306539071 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c60aa3930 0x563c60aa3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:10.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.010+0000 7f9682a00700 1 -- 172.21.15.33:0/3306539071 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c6093d8d0 con 0x563c60aa4250 2024-05-03T18:28:10.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.010+0000 7f9682a00700 1 --2- 172.21.15.33:0/3306539071 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c60aa4250 0x563c60a9df30 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f9674009990 tx=0x7f9674005bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d55639de4a8d0123 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:28:10.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.010+0000 7f9683400700 1 --2- 172.21.15.33:0/2182405688 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f96680b2b90 0x7f96680b5050 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f967c0df550 tx=0x7f967c004f20 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:28:10.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.010+0000 7f966fe00700 1 -- 172.21.15.33:0/3306539071 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f967400b9d0 con 0x563c60aa4250 2024-05-03T18:28:10.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.011+0000 7f966fe00700 1 -- 172.21.15.33:0/3306539071 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9674027070 con 0x563c60aa4250 2024-05-03T18:28:10.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.011+0000 7f966fe00700 1 -- 172.21.15.33:0/3306539071 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f967400fd90 con 0x563c60aa4250 2024-05-03T18:28:10.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.011+0000 7f96942f0b80 1 -- 172.21.15.33:0/3306539071 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c60aa4250 msgr2=0x563c60a9df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:28:10.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.011+0000 7f96942f0b80 1 --2- 172.21.15.33:0/3306539071 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c60aa4250 0x563c60a9df30 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f9674009990 tx=0x7f9674005bc0 comp rx=0 tx=0).stop 2024-05-03T18:28:10.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.012+0000 7f96942f0b80 1 -- 172.21.15.33:0/3306539071 shutdown_connections 2024-05-03T18:28:10.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.012+0000 7f96942f0b80 1 --2- 172.21.15.33:0/3306539071 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563c60a9e550 0x563c60a9e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:10.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.012+0000 7f96942f0b80 1 --2- 172.21.15.33:0/3306539071 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c60aa3930 0x563c60aa3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:10.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.012+0000 7f96942f0b80 1 --2- 172.21.15.33:0/3306539071 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c60aa4250 0x563c60a9df30 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:10.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.012+0000 7f96942f0b80 1 -- 172.21.15.33:0/3306539071 >> 172.21.15.33:0/3306539071 conn(0x563c60a166d0 msgr2=0x563c60a98900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:28:10.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.012+0000 7f96942f0b80 1 -- 172.21.15.33:0/3306539071 shutdown_connections 2024-05-03T18:28:10.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.012+0000 7f96942f0b80 1 -- 172.21.15.33:0/3306539071 wait complete. 2024-05-03T18:28:10.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.012+0000 7f96942f0b80 1 -- start start 2024-05-03T18:28:10.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.013+0000 7f96942f0b80 1 Processor -- start 2024-05-03T18:28:10.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.013+0000 7f96942f0b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c60a9e550 0x563c60aa9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:28:10.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.013+0000 7f96942f0b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563c60aa3930 0x563c60aa7250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:28:10.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.013+0000 7f96942f0b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c60aa4250 0x563c60aa7790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:28:10.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.013+0000 7f96942f0b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563c60aaaca0 con 0x563c60a9e550 2024-05-03T18:28:10.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.013+0000 7f96942f0b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x563c60956b20 con 0x563c60aa4250 2024-05-03T18:28:10.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.013+0000 7f96942f0b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x563c60aaae20 con 0x563c60aa3930 2024-05-03T18:28:10.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.013+0000 7f96942f0b80 10 client.0 _ll_register_callbacks cb 0x563c60997c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:28:10.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.014+0000 7f9682000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c60a9e550 0x563c60aa9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:28:10.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.014+0000 7f9682000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c60a9e550 0x563c60aa9eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47278/0 (socket says 172.21.15.33:47278) 2024-05-03T18:28:10.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.014+0000 7f9682000700 1 -- 172.21.15.33:0/1726971125 learned_addr learned my addr 172.21.15.33:0/1726971125 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:28:10.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.014+0000 7f9682a00700 1 --2- 172.21.15.33:0/1726971125 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563c60aa3930 0x563c60aa7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:28:10.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.014+0000 7f9682000700 1 -- 172.21.15.33:0/1726971125 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563c60aa3930 msgr2=0x563c60aa7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:28:10.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.014+0000 7f9682000700 1 --2- 172.21.15.33:0/1726971125 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563c60aa3930 0x563c60aa7250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:10.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.014+0000 7f9682000700 1 -- 172.21.15.33:0/1726971125 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c60aa4250 msgr2=0x563c60aa7790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:28:10.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.014+0000 7f9682000700 1 --2- 172.21.15.33:0/1726971125 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c60aa4250 0x563c60aa7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:10.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.014+0000 7f9682000700 1 -- 172.21.15.33:0/1726971125 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c609b6420 con 0x563c60a9e550 2024-05-03T18:28:10.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.015+0000 7f9682000700 1 --2- 172.21.15.33:0/1726971125 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c60a9e550 0x563c60aa9eb0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f967000ae10 tx=0x7f9670003c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff22fd4aa7be7086 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:28:10.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.015+0000 7f966cc00700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:28:10.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.015+0000 7f966cc00700 1 -- 172.21.15.33:0/1726971125 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9670011b70 con 0x563c60a9e550 2024-05-03T18:28:10.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.015+0000 7f96942f0b80 1 -- 172.21.15.33:0/1726971125 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9674027070 con 0x563c60a9e550 2024-05-03T18:28:10.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.015+0000 7f966cc00700 1 -- 172.21.15.33:0/1726971125 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f9670011d10 con 0x563c60a9e550 2024-05-03T18:28:10.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.015+0000 7f966cc00700 1 -- 172.21.15.33:0/1726971125 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f9670027050 con 0x563c60a9e550 2024-05-03T18:28:10.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.015+0000 7f96942f0b80 10 client.15441 fetch_fsmap learned FSMap version 21 2024-05-03T18:28:10.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.015+0000 7f96942f0b80 1 -- 172.21.15.33:0/1726971125 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x563c60ab54d0 con 0x563c60a9e550 2024-05-03T18:28:10.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.016+0000 7f966cc00700 1 -- 172.21.15.33:0/1726971125 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9674027070 con 0x563c60a9e550 2024-05-03T18:28:10.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.016+0000 7f966cc00700 1 -- 172.21.15.33:0/1726971125 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f967000f430 con 0x563c60a9e550 2024-05-03T18:28:10.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.016+0000 7f96942f0b80 10 client.15441 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:28:10.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.016+0000 7f96942f0b80 10 client.15441 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:28:10.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.016+0000 7f96942f0b80 20 client.15441 populate_metadata read hostname 'smithi033' 2024-05-03T18:28:10.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.016+0000 7f96942f0b80 1 --2- 172.21.15.33:0/1726971125 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x563c60ab7b10 0x563c60ab9f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:28:10.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.016+0000 7f96942f0b80 4 client.15441 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:28:10.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.016+0000 7f96942f0b80 1 -- 172.21.15.33:0/1726971125 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563c60a6aff0 con 0x563c60ab7b10 2024-05-03T18:28:10.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.017+0000 7f9682a00700 1 --2- 172.21.15.33:0/1726971125 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x563c60ab7b10 0x563c60ab9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:28:10.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.019+0000 7f9682a00700 1 --2- 172.21.15.33:0/1726971125 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x563c60ab7b10 0x563c60ab9f00 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57cf389cebe0150d server_cookie=377e613336462f9c in_seq=0 out_seq=0 2024-05-03T18:28:10.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.019+0000 7f966cc00700 10 client.15441 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:28:10.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.021+0000 7f966cc00700 1 -- 172.21.15.33:0/1726971125 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563c60a6aff0 con 0x563c60ab7b10 2024-05-03T18:28:10.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.021+0000 7f966cc00700 10 client.15441 handle_command_reply: tid=0 2024-05-03T18:28:10.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.021+0000 7f96942f0b80 1 client.15441 shutdown 2024-05-03T18:28:10.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.022+0000 7f96942f0b80 1 -- 172.21.15.33:0/1726971125 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c60a9e550 msgr2=0x563c60aa9eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:28:10.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.022+0000 7f96942f0b80 1 --2- 172.21.15.33:0/1726971125 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c60a9e550 0x563c60aa9eb0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f967000ae10 tx=0x7f9670003c90 comp rx=0 tx=0).stop 2024-05-03T18:28:10.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.022+0000 7f96942f0b80 1 -- 172.21.15.33:0/1726971125 shutdown_connections 2024-05-03T18:28:10.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.022+0000 7f96942f0b80 1 --2- 172.21.15.33:0/1726971125 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x563c60ab7b10 0x563c60ab9f00 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:10.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.022+0000 7f96942f0b80 1 --2- 172.21.15.33:0/1726971125 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c60aa4250 0x563c60aa7790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:10.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.022+0000 7f96942f0b80 1 --2- 172.21.15.33:0/1726971125 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c60a9e550 0x563c60aa9eb0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:10.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.022+0000 7f96942f0b80 1 --2- 172.21.15.33:0/1726971125 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563c60aa3930 0x563c60aa7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:10.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.022+0000 7f96942f0b80 1 -- 172.21.15.33:0/1726971125 >> 172.21.15.33:0/1726971125 conn(0x563c60a166d0 msgr2=0x563c60a933b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:28:10.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.022+0000 7f966cc00700 0 client.15441 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:28:10.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.022+0000 7f96942f0b80 1 -- 172.21.15.33:0/1726971125 shutdown_connections 2024-05-03T18:28:10.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.022+0000 7f96942f0b80 1 -- 172.21.15.33:0/1726971125 wait complete. 2024-05-03T18:28:10.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.023+0000 7f96942f0b80 20 client.15441 trim_cache size 0 max 16384 2024-05-03T18:28:10.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.047+0000 7f96942f0b80 1 Processor -- start 2024-05-03T18:28:10.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.047+0000 7f96942f0b80 1 -- start start 2024-05-03T18:28:10.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.047+0000 7f96942f0b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c60a9e550 0x563c60aa5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:28:10.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.047+0000 7f96942f0b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c60aa3930 0x563c60aa6080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:28:10.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.047+0000 7f96942f0b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563c60aa4250 0x563c60abf750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:28:10.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.047+0000 7f96942f0b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f96841af5c0 con 0x563c60aa3930 2024-05-03T18:28:10.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.047+0000 7f96942f0b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f96841b1e40 con 0x563c60a9e550 2024-05-03T18:28:10.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.047+0000 7f96942f0b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f96841afc30 con 0x563c60aa4250 2024-05-03T18:28:10.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.047+0000 7f9682a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c60aa3930 0x563c60aa6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:28:10.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.047+0000 7f9682a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c60aa3930 0x563c60aa6080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47288/0 (socket says 172.21.15.33:47288) 2024-05-03T18:28:10.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.047+0000 7f9682a00700 1 -- 172.21.15.33:0/33254772 learned_addr learned my addr 172.21.15.33:0/33254772 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:28:10.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.048+0000 7f9682000700 1 --2- 172.21.15.33:0/33254772 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c60a9e550 0x563c60aa5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:28:10.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.048+0000 7f9682000700 1 --2- 172.21.15.33:0/33254772 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563c60aa4250 0x563c60abf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:28:10.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.048+0000 7f9682a00700 1 -- 172.21.15.33:0/33254772 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563c60aa4250 msgr2=0x563c60abf750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:28:10.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.048+0000 7f9682a00700 1 --2- 172.21.15.33:0/33254772 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563c60aa4250 0x563c60abf750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:10.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.048+0000 7f9682a00700 1 -- 172.21.15.33:0/33254772 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c60a9e550 msgr2=0x563c60aa5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:28:10.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.048+0000 7f9682a00700 1 --2- 172.21.15.33:0/33254772 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c60a9e550 0x563c60aa5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:10.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.048+0000 7f9682a00700 1 -- 172.21.15.33:0/33254772 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c60aa1cd0 con 0x563c60aa3930 2024-05-03T18:28:10.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.048+0000 7f9682000700 1 --2- 172.21.15.33:0/33254772 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c60a9e550 0x563c60aa5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:28:10.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.048+0000 7f9682a00700 1 --2- 172.21.15.33:0/33254772 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c60aa3930 0x563c60aa6080 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f9674009ce0 tx=0x7f967400b540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b46f1e5e04aba771 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:28:10.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.049+0000 7f966f400700 1 -- 172.21.15.33:0/33254772 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f967400b9d0 con 0x563c60aa3930 2024-05-03T18:28:10.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.049+0000 7f966f400700 1 -- 172.21.15.33:0/33254772 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9674038070 con 0x563c60aa3930 2024-05-03T18:28:10.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.049+0000 7f966f400700 1 -- 172.21.15.33:0/33254772 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96740321f0 con 0x563c60aa3930 2024-05-03T18:28:10.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.050+0000 7f96942f0b80 1 -- 172.21.15.33:0/33254772 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c60aa3930 msgr2=0x563c60aa6080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:28:10.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.050+0000 7f96942f0b80 1 --2- 172.21.15.33:0/33254772 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c60aa3930 0x563c60aa6080 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f9674009ce0 tx=0x7f967400b540 comp rx=0 tx=0).stop 2024-05-03T18:28:10.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.050+0000 7f96942f0b80 1 -- 172.21.15.33:0/33254772 shutdown_connections 2024-05-03T18:28:10.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.050+0000 7f96942f0b80 1 --2- 172.21.15.33:0/33254772 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563c60aa4250 0x563c60abf750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:10.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.050+0000 7f96942f0b80 1 --2- 172.21.15.33:0/33254772 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c60a9e550 0x563c60aa5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:10.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.050+0000 7f96942f0b80 1 --2- 172.21.15.33:0/33254772 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c60aa3930 0x563c60aa6080 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:10.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.050+0000 7f96942f0b80 1 -- 172.21.15.33:0/33254772 >> 172.21.15.33:0/33254772 conn(0x563c60a166d0 msgr2=0x563c60ab9b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:28:10.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.050+0000 7f96942f0b80 1 -- 172.21.15.33:0/33254772 shutdown_connections 2024-05-03T18:28:10.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.050+0000 7f96942f0b80 1 -- 172.21.15.33:0/33254772 wait complete. 2024-05-03T18:28:10.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.051+0000 7f96942f0b80 1 -- start start 2024-05-03T18:28:10.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.051+0000 7f96942f0b80 1 Processor -- start 2024-05-03T18:28:10.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.051+0000 7f96942f0b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c60a9e550 0x563c60abf1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:28:10.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.051+0000 7f96942f0b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563c60aa3930 0x563c60abf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:28:10.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.051+0000 7f96942f0b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c60aa4250 0x563c60abfc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:28:10.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.052+0000 7f96942f0b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f96841b0300 con 0x563c60a9e550 2024-05-03T18:28:10.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.052+0000 7f96942f0b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f96841b1770 con 0x563c60aa4250 2024-05-03T18:28:10.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.052+0000 7f96942f0b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f96841b09d0 con 0x563c60aa3930 2024-05-03T18:28:10.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.052+0000 7f96942f0b80 10 client.0 _ll_register_callbacks cb 0x563c60997c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:28:10.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.052+0000 7f9682000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c60a9e550 0x563c60abf1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:28:10.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.052+0000 7f9682000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c60a9e550 0x563c60abf1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47292/0 (socket says 172.21.15.33:47292) 2024-05-03T18:28:10.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.052+0000 7f9682000700 1 -- 172.21.15.33:0/4152279384 learned_addr learned my addr 172.21.15.33:0/4152279384 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:28:10.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.052+0000 7f9682a00700 1 --2- 172.21.15.33:0/4152279384 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563c60aa3930 0x563c60abf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:28:10.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.052+0000 7f9682000700 1 -- 172.21.15.33:0/4152279384 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563c60aa3930 msgr2=0x563c60abf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:28:10.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.052+0000 7f9682000700 1 --2- 172.21.15.33:0/4152279384 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563c60aa3930 0x563c60abf720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:10.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.052+0000 7f9682000700 1 -- 172.21.15.33:0/4152279384 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c60aa4250 msgr2=0x563c60abfc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:28:10.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.053+0000 7f9682000700 1 --2- 172.21.15.33:0/4152279384 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c60aa4250 0x563c60abfc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:10.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.053+0000 7f9682000700 1 -- 172.21.15.33:0/4152279384 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c60b21560 con 0x563c60a9e550 2024-05-03T18:28:10.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.053+0000 7f9682a00700 1 --2- 172.21.15.33:0/4152279384 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563c60aa3930 0x563c60abf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:28:10.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.053+0000 7f9682000700 1 --2- 172.21.15.33:0/4152279384 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c60a9e550 0x563c60abf1e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f967000ae10 tx=0x7f967000a9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70d5b512f970cc7a server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:28:10.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.053+0000 7f966cc00700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:28:10.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.053+0000 7f966cc00700 1 -- 172.21.15.33:0/4152279384 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f967000f640 con 0x563c60a9e550 2024-05-03T18:28:10.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.053+0000 7f966cc00700 1 -- 172.21.15.33:0/4152279384 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f967001a040 con 0x563c60a9e550 2024-05-03T18:28:10.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.053+0000 7f96942f0b80 1 -- 172.21.15.33:0/4152279384 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9674038070 con 0x563c60a9e550 2024-05-03T18:28:10.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.054+0000 7f966cc00700 1 -- 172.21.15.33:0/4152279384 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f967003c050 con 0x563c60a9e550 2024-05-03T18:28:10.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.054+0000 7f96942f0b80 10 client.15447 fetch_fsmap learned FSMap version 21 2024-05-03T18:28:10.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.054+0000 7f96942f0b80 1 -- 172.21.15.33:0/4152279384 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x563c60b227c0 con 0x563c60a9e550 2024-05-03T18:28:10.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.054+0000 7f966cc00700 1 -- 172.21.15.33:0/4152279384 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9674038070 con 0x563c60a9e550 2024-05-03T18:28:10.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.054+0000 7f966cc00700 1 -- 172.21.15.33:0/4152279384 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f96700111c0 con 0x563c60a9e550 2024-05-03T18:28:10.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.054+0000 7f96942f0b80 10 client.15447 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:28:10.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.054+0000 7f96942f0b80 10 client.15447 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:28:10.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.054+0000 7f96942f0b80 20 client.15447 populate_metadata read hostname 'smithi033' 2024-05-03T18:28:10.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.054+0000 7f96942f0b80 1 --2- 172.21.15.33:0/4152279384 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x563c60b22f00 0x563c60b25320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:28:10.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.054+0000 7f96942f0b80 4 client.15447 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-03T18:28:10.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.054+0000 7f96942f0b80 1 -- 172.21.15.33:0/4152279384 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563c609fe400 con 0x563c60b22f00 2024-05-03T18:28:10.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.055+0000 7f9682a00700 1 --2- 172.21.15.33:0/4152279384 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x563c60b22f00 0x563c60b25320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:28:10.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.056+0000 7f9682a00700 1 --2- 172.21.15.33:0/4152279384 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x563c60b22f00 0x563c60b25320 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dafeda5a376adec9 server_cookie=c9e59600d822fd3a in_seq=0 out_seq=0 2024-05-03T18:28:10.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.056+0000 7f966cc00700 10 client.15447 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:28:10.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.064+0000 7f966cc00700 1 -- 172.21.15.33:0/4152279384 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x563c609fe400 con 0x563c60b22f00 2024-05-03T18:28:10.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.064+0000 7f966cc00700 10 client.15447 handle_command_reply: tid=0 2024-05-03T18:28:10.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.064+0000 7f96942f0b80 1 client.15447 shutdown 2024-05-03T18:28:10.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.064+0000 7f96942f0b80 1 -- 172.21.15.33:0/4152279384 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c60a9e550 msgr2=0x563c60abf1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:28:10.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.064+0000 7f96942f0b80 1 --2- 172.21.15.33:0/4152279384 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c60a9e550 0x563c60abf1e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f967000ae10 tx=0x7f967000a9e0 comp rx=0 tx=0).stop 2024-05-03T18:28:10.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.065+0000 7f96942f0b80 1 -- 172.21.15.33:0/4152279384 shutdown_connections 2024-05-03T18:28:10.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.065+0000 7f96942f0b80 1 --2- 172.21.15.33:0/4152279384 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x563c60b22f00 0x563c60b25320 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:10.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.065+0000 7f96942f0b80 1 --2- 172.21.15.33:0/4152279384 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c60aa4250 0x563c60abfc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:10.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.065+0000 7f96942f0b80 1 --2- 172.21.15.33:0/4152279384 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c60a9e550 0x563c60abf1e0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:10.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.065+0000 7f96942f0b80 1 --2- 172.21.15.33:0/4152279384 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563c60aa3930 0x563c60abf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:10.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.065+0000 7f96942f0b80 1 -- 172.21.15.33:0/4152279384 >> 172.21.15.33:0/4152279384 conn(0x563c60a166d0 msgr2=0x563c60ab9b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:28:10.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.065+0000 7f966cc00700 0 client.15447 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:28:10.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.065+0000 7f96942f0b80 1 -- 172.21.15.33:0/4152279384 shutdown_connections 2024-05-03T18:28:10.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.065+0000 7f96942f0b80 1 -- 172.21.15.33:0/4152279384 wait complete. 2024-05-03T18:28:10.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.065+0000 7f96942f0b80 20 client.15447 trim_cache size 0 max 16384 2024-05-03T18:28:10.066 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-03T18:28:10.067 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-03T18:28:10.067 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-05-03T18:28:10.067 INFO:teuthology.orchestra.run.smithi033.stdout: "0495ecba-a7a5-45a5-a00c-035f6ae59f83": { 2024-05-03T18:28:10.067 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-05-03T18:28:10.067 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "0495ecba-a7a5-45a5-a00c-035f6ae59f83", 2024-05-03T18:28:10.067 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-05-03T18:28:10.067 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:28:10.067 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:28:10.067 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-03T18:28:10.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.066+0000 7f9689a00700 1 -- 172.21.15.33:0/2182405688 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f96680b2b90 msgr2=0x7f96680b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:28:10.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.066+0000 7f9689a00700 1 --2- 172.21.15.33:0/2182405688 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f96680b2b90 0x7f96680b5050 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f967c0df550 tx=0x7f967c004f20 comp rx=0 tx=0).stop 2024-05-03T18:28:10.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.066+0000 7f9689a00700 1 -- 172.21.15.33:0/2182405688 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f968410b6c0 msgr2=0x7f968419ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:28:10.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.066+0000 7f9689a00700 1 --2- 172.21.15.33:0/2182405688 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f968410b6c0 0x7f968419ba70 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f967c00ada0 tx=0x7f967c00c330 comp rx=0 tx=0).stop 2024-05-03T18:28:10.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.067+0000 7f9689a00700 1 -- 172.21.15.33:0/2182405688 shutdown_connections 2024-05-03T18:28:10.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.067+0000 7f9689a00700 1 --2- 172.21.15.33:0/2182405688 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f968410b6c0 0x7f968419ba70 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:10.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.067+0000 7f9689a00700 1 --2- 172.21.15.33:0/2182405688 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f96680b2b90 0x7f96680b5050 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:10.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.067+0000 7f9689a00700 1 --2- 172.21.15.33:0/2182405688 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96841056c0 0x7f96841a18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:10.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.067+0000 7f9689a00700 1 --2- 172.21.15.33:0/2182405688 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9684106080 0x7f96841a1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:10.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.067+0000 7f9689a00700 1 -- 172.21.15.33:0/2182405688 >> 172.21.15.33:0/2182405688 conn(0x7f96841011b0 msgr2=0x7f96841019b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:28:10.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.067+0000 7f9689a00700 1 -- 172.21.15.33:0/2182405688 shutdown_connections 2024-05-03T18:28:10.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:10.067+0000 7f9689a00700 1 -- 172.21.15.33:0/2182405688 wait complete. 2024-05-03T18:28:11.072 INFO:tasks.cephfs.filesystem:scrub status for tag:0495ecba-a7a5-45a5-a00c-035f6ae59f83 - {'path': '/', 'tag': '0495ecba-a7a5-45a5-a00c-035f6ae59f83', 'options': 'recursive,force'} 2024-05-03T18:28:11.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:28:10 smithi150 ceph-mon[32492]: pgmap v32: 129 pgs: 129 active+clean; 49 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 1.2 MiB/s wr, 69 op/s 2024-05-03T18:28:11.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:28:10 smithi033 ceph-mon[28330]: pgmap v32: 129 pgs: 129 active+clean; 49 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 1.2 MiB/s wr, 69 op/s 2024-05-03T18:28:11.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:28:10 smithi099 ceph-mon[31658]: pgmap v32: 129 pgs: 129 active+clean; 49 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 1.2 MiB/s wr, 69 op/s 2024-05-03T18:28:12.134 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:28:11 smithi150 ceph-mon[32492]: scrub summary: idle 2024-05-03T18:28:12.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:28:11 smithi033 ceph-mon[28330]: scrub summary: idle 2024-05-03T18:28:12.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:28:11 smithi099 ceph-mon[31658]: scrub summary: idle 2024-05-03T18:28:13.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:28:12 smithi150 ceph-mon[32492]: pgmap v33: 129 pgs: 129 active+clean; 49 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 620 KiB/s wr, 52 op/s 2024-05-03T18:28:13.197 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:28:12 smithi099 ceph-mon[31658]: pgmap v33: 129 pgs: 129 active+clean; 49 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 620 KiB/s wr, 52 op/s 2024-05-03T18:28:13.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:28:12 smithi033 ceph-mon[28330]: pgmap v33: 129 pgs: 129 active+clean; 49 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 620 KiB/s wr, 52 op/s 2024-05-03T18:28:15.176 INFO:teuthology.orchestra.run.smithi033.stderr: * Initialized, logs are in /var/lib/pgsql/initdb_postgresql.log 2024-05-03T18:28:15.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:28:14 smithi033 ceph-mon[28330]: pgmap v34: 129 pgs: 129 active+clean; 52 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 891 KiB/s wr, 99 op/s 2024-05-03T18:28:15.185 DEBUG:teuthology.orchestra.run.smithi033:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-05-03T18:28:15.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:28:14 smithi099 ceph-mon[31658]: pgmap v34: 129 pgs: 129 active+clean; 52 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 891 KiB/s wr, 99 op/s 2024-05-03T18:28:15.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:28:14 smithi150 ceph-mon[32492]: pgmap v34: 129 pgs: 129 active+clean; 52 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 891 KiB/s wr, 99 op/s 2024-05-03T18:28:17.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:28:16 smithi033 ceph-mon[28330]: pgmap v35: 129 pgs: 129 active+clean; 52 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 475 KiB/s wr, 66 op/s 2024-05-03T18:28:17.262 INFO:teuthology.orchestra.run.smithi033.stdout:/tmp/cephfs/postgres/: 2024-05-03T18:28:17.263 INFO:teuthology.orchestra.run.smithi033.stdout:total 2 2024-05-03T18:28:17.263 INFO:teuthology.orchestra.run.smithi033.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 3 18:27 . 2024-05-03T18:28:17.263 INFO:teuthology.orchestra.run.smithi033.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 41283110 May 3 18:27 .. 2024-05-03T18:28:17.263 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 3 18:28 data 2024-05-03T18:28:17.263 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:28:17.263 INFO:teuthology.orchestra.run.smithi033.stdout:/tmp/cephfs/postgres/data: 2024-05-03T18:28:17.263 INFO:teuthology.orchestra.run.smithi033.stdout:total 45 2024-05-03T18:28:17.263 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 3 18:28 . 2024-05-03T18:28:17.263 INFO:teuthology.orchestra.run.smithi033.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 3 18:27 .. 2024-05-03T18:28:17.263 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 23889417 May 3 18:27 base 2024-05-03T18:28:17.263 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 549376 May 3 18:27 global 2024-05-03T18:28:17.263 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:28 log 2024-05-03T18:28:17.263 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 pg_commit_ts 2024-05-03T18:28:17.264 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 pg_dynshmem 2024-05-03T18:28:17.264 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4516 May 3 18:27 pg_hba.conf 2024-05-03T18:28:17.264 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 May 3 18:27 pg_ident.conf 2024-05-03T18:28:17.264 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 May 3 18:27 pg_logical 2024-05-03T18:28:17.264 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 pg_multixact 2024-05-03T18:28:17.264 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 pg_notify 2024-05-03T18:28:17.264 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 pg_replslot 2024-05-03T18:28:17.264 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 pg_serial 2024-05-03T18:28:17.264 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 pg_snapshots 2024-05-03T18:28:17.264 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 pg_stat 2024-05-03T18:28:17.264 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 pg_stat_tmp 2024-05-03T18:28:17.264 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 pg_subtrans 2024-05-03T18:28:17.264 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 pg_tblspc 2024-05-03T18:28:17.265 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 pg_twophase 2024-05-03T18:28:17.265 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 3 18:27 PG_VERSION 2024-05-03T18:28:17.265 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 3 18:27 pg_wal 2024-05-03T18:28:17.265 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 pg_xact 2024-05-03T18:28:17.265 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 May 3 18:27 postgresql.auto.conf 2024-05-03T18:28:17.265 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 28082 May 3 18:27 postgresql.conf 2024-05-03T18:28:17.265 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:28:17.265 INFO:teuthology.orchestra.run.smithi033.stdout:/tmp/cephfs/postgres/data/base: 2024-05-03T18:28:17.265 INFO:teuthology.orchestra.run.smithi033.stdout:total 3 2024-05-03T18:28:17.265 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 23889417 May 3 18:27 . 2024-05-03T18:28:17.265 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 3 18:28 .. 2024-05-03T18:28:17.265 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7963139 May 3 18:27 1 2024-05-03T18:28:17.265 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7963139 May 3 18:27 13493 2024-05-03T18:28:17.265 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7963139 May 3 18:27 13494 2024-05-03T18:28:17.266 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:28:17.266 INFO:teuthology.orchestra.run.smithi033.stdout:/tmp/cephfs/postgres/data/base/1: 2024-05-03T18:28:17.266 INFO:teuthology.orchestra.run.smithi033.stdout:total 7778 2024-05-03T18:28:17.266 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7963139 May 3 18:27 . 2024-05-03T18:28:17.266 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 23889417 May 3 18:27 .. 2024-05-03T18:28:17.266 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 112 2024-05-03T18:28:17.266 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 113 2024-05-03T18:28:17.266 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 3 18:27 1247 2024-05-03T18:28:17.266 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 1247_fsm 2024-05-03T18:28:17.266 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 1247_vm 2024-05-03T18:28:17.266 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 May 3 18:27 1249 2024-05-03T18:28:17.266 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 1249_fsm 2024-05-03T18:28:17.266 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 1249_vm 2024-05-03T18:28:17.266 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 May 3 18:27 1255 2024-05-03T18:28:17.267 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 1255_fsm 2024-05-03T18:28:17.267 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 1255_vm 2024-05-03T18:28:17.267 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 May 3 18:27 1259 2024-05-03T18:28:17.267 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 1259_fsm 2024-05-03T18:28:17.267 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 1259_vm 2024-05-03T18:28:17.267 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 May 3 18:27 13344 2024-05-03T18:28:17.267 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 13344_fsm 2024-05-03T18:28:17.267 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 13344_vm 2024-05-03T18:28:17.267 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 13346 2024-05-03T18:28:17.267 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 13348 2024-05-03T18:28:17.267 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 13349 2024-05-03T18:28:17.267 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 13349_fsm 2024-05-03T18:28:17.268 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 13349_vm 2024-05-03T18:28:17.268 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 13351 2024-05-03T18:28:17.268 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 13353 2024-05-03T18:28:17.268 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 13354 2024-05-03T18:28:17.268 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 13354_fsm 2024-05-03T18:28:17.268 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 13354_vm 2024-05-03T18:28:17.268 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 13356 2024-05-03T18:28:17.268 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 13358 2024-05-03T18:28:17.268 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 13359 2024-05-03T18:28:17.268 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 13359_fsm 2024-05-03T18:28:17.268 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 13359_vm 2024-05-03T18:28:17.268 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 13361 2024-05-03T18:28:17.268 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 13363 2024-05-03T18:28:17.268 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 1417 2024-05-03T18:28:17.268 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 1418 2024-05-03T18:28:17.269 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 174 2024-05-03T18:28:17.269 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 175 2024-05-03T18:28:17.269 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2187 2024-05-03T18:28:17.269 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 2224 2024-05-03T18:28:17.269 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 2328 2024-05-03T18:28:17.269 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 2336 2024-05-03T18:28:17.269 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2337 2024-05-03T18:28:17.269 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2579 2024-05-03T18:28:17.269 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2600 2024-05-03T18:28:17.269 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2600_fsm 2024-05-03T18:28:17.269 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2600_vm 2024-05-03T18:28:17.269 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2601 2024-05-03T18:28:17.269 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2601_fsm 2024-05-03T18:28:17.269 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2601_vm 2024-05-03T18:28:17.270 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 3 18:27 2602 2024-05-03T18:28:17.270 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2602_fsm 2024-05-03T18:28:17.270 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2602_vm 2024-05-03T18:28:17.270 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 3 18:27 2603 2024-05-03T18:28:17.270 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2603_fsm 2024-05-03T18:28:17.270 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2603_vm 2024-05-03T18:28:17.270 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 2604 2024-05-03T18:28:17.270 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2605 2024-05-03T18:28:17.270 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2605_fsm 2024-05-03T18:28:17.270 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2605_vm 2024-05-03T18:28:17.270 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2606 2024-05-03T18:28:17.270 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2606_fsm 2024-05-03T18:28:17.270 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2606_vm 2024-05-03T18:28:17.270 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2607 2024-05-03T18:28:17.271 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2607_fsm 2024-05-03T18:28:17.271 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2607_vm 2024-05-03T18:28:17.271 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 May 3 18:27 2608 2024-05-03T18:28:17.271 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2608_fsm 2024-05-03T18:28:17.271 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2608_vm 2024-05-03T18:28:17.271 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 May 3 18:27 2609 2024-05-03T18:28:17.271 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2609_fsm 2024-05-03T18:28:17.271 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2609_vm 2024-05-03T18:28:17.271 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 3 18:27 2610 2024-05-03T18:28:17.271 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2610_fsm 2024-05-03T18:28:17.271 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2610_vm 2024-05-03T18:28:17.271 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 2611 2024-05-03T18:28:17.271 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2612 2024-05-03T18:28:17.271 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2612_fsm 2024-05-03T18:28:17.272 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2612_vm 2024-05-03T18:28:17.272 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 2613 2024-05-03T18:28:17.272 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2615 2024-05-03T18:28:17.272 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2615_fsm 2024-05-03T18:28:17.272 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2615_vm 2024-05-03T18:28:17.272 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2616 2024-05-03T18:28:17.272 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2616_fsm 2024-05-03T18:28:17.272 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2616_vm 2024-05-03T18:28:17.272 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 May 3 18:27 2617 2024-05-03T18:28:17.272 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2617_fsm 2024-05-03T18:28:17.273 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2617_vm 2024-05-03T18:28:17.273 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 May 3 18:27 2618 2024-05-03T18:28:17.273 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2618_fsm 2024-05-03T18:28:17.273 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2618_vm 2024-05-03T18:28:17.273 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 May 3 18:27 2619 2024-05-03T18:28:17.273 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2619_fsm 2024-05-03T18:28:17.273 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2619_vm 2024-05-03T18:28:17.273 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 2620 2024-05-03T18:28:17.273 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2650 2024-05-03T18:28:17.274 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2651 2024-05-03T18:28:17.274 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2652 2024-05-03T18:28:17.274 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 3 18:27 2653 2024-05-03T18:28:17.274 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 3 18:27 2654 2024-05-03T18:28:17.274 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 3 18:27 2655 2024-05-03T18:28:17.274 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2656 2024-05-03T18:28:17.274 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2657 2024-05-03T18:28:17.274 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 May 3 18:27 2658 2024-05-03T18:28:17.274 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 3 18:27 2659 2024-05-03T18:28:17.274 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2660 2024-05-03T18:28:17.274 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2661 2024-05-03T18:28:17.274 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 3 18:27 2662 2024-05-03T18:28:17.274 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 3 18:27 2663 2024-05-03T18:28:17.275 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2664 2024-05-03T18:28:17.275 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2665 2024-05-03T18:28:17.275 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2666 2024-05-03T18:28:17.275 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2667 2024-05-03T18:28:17.275 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2668 2024-05-03T18:28:17.275 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2669 2024-05-03T18:28:17.275 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2670 2024-05-03T18:28:17.275 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 May 3 18:27 2673 2024-05-03T18:28:17.275 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 May 3 18:27 2674 2024-05-03T18:28:17.275 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 May 3 18:27 2675 2024-05-03T18:28:17.275 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2678 2024-05-03T18:28:17.275 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2679 2024-05-03T18:28:17.275 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2680 2024-05-03T18:28:17.276 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2681 2024-05-03T18:28:17.276 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2682 2024-05-03T18:28:17.276 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2683 2024-05-03T18:28:17.276 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2684 2024-05-03T18:28:17.276 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2685 2024-05-03T18:28:17.276 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2686 2024-05-03T18:28:17.276 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2687 2024-05-03T18:28:17.276 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 3 18:27 2688 2024-05-03T18:28:17.276 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 3 18:27 2689 2024-05-03T18:28:17.276 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 May 3 18:27 2690 2024-05-03T18:28:17.276 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 May 3 18:27 2691 2024-05-03T18:28:17.276 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2692 2024-05-03T18:28:17.276 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2693 2024-05-03T18:28:17.276 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2696 2024-05-03T18:28:17.276 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2699 2024-05-03T18:28:17.277 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2701 2024-05-03T18:28:17.277 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2702 2024-05-03T18:28:17.277 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 3 18:27 2703 2024-05-03T18:28:17.277 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 3 18:27 2704 2024-05-03T18:28:17.277 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2753 2024-05-03T18:28:17.277 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2753_fsm 2024-05-03T18:28:17.277 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2753_vm 2024-05-03T18:28:17.277 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2754 2024-05-03T18:28:17.277 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2755 2024-05-03T18:28:17.277 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 3 18:27 2756 2024-05-03T18:28:17.277 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 3 18:27 2757 2024-05-03T18:28:17.277 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 2830 2024-05-03T18:28:17.277 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2831 2024-05-03T18:28:17.277 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 2832 2024-05-03T18:28:17.277 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2833 2024-05-03T18:28:17.277 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 2834 2024-05-03T18:28:17.278 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2835 2024-05-03T18:28:17.278 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 2836 2024-05-03T18:28:17.278 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2837 2024-05-03T18:28:17.278 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 May 3 18:27 2838 2024-05-03T18:28:17.278 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2838_fsm 2024-05-03T18:28:17.278 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2838_vm 2024-05-03T18:28:17.278 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2839 2024-05-03T18:28:17.278 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2840 2024-05-03T18:28:17.278 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2840_fsm 2024-05-03T18:28:17.278 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2840_vm 2024-05-03T18:28:17.278 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2841 2024-05-03T18:28:17.278 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 2995 2024-05-03T18:28:17.278 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2996 2024-05-03T18:28:17.278 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3079 2024-05-03T18:28:17.278 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 3079_fsm 2024-05-03T18:28:17.278 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3079_vm 2024-05-03T18:28:17.279 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 3080 2024-05-03T18:28:17.279 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 3081 2024-05-03T18:28:17.279 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 3 18:27 3085 2024-05-03T18:28:17.279 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 3118 2024-05-03T18:28:17.279 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3119 2024-05-03T18:28:17.279 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 May 3 18:27 3164 2024-05-03T18:28:17.279 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 3256 2024-05-03T18:28:17.279 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3257 2024-05-03T18:28:17.279 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3258 2024-05-03T18:28:17.279 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 3350 2024-05-03T18:28:17.279 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3351 2024-05-03T18:28:17.279 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3379 2024-05-03T18:28:17.279 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3380 2024-05-03T18:28:17.279 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 3381 2024-05-03T18:28:17.279 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 3394 2024-05-03T18:28:17.280 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 3394_fsm 2024-05-03T18:28:17.280 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3394_vm 2024-05-03T18:28:17.280 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 3395 2024-05-03T18:28:17.280 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 3429 2024-05-03T18:28:17.280 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 3430 2024-05-03T18:28:17.280 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3431 2024-05-03T18:28:17.280 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3433 2024-05-03T18:28:17.280 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 3439 2024-05-03T18:28:17.280 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3440 2024-05-03T18:28:17.280 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 3 18:27 3455 2024-05-03T18:28:17.280 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 May 3 18:27 3456 2024-05-03T18:28:17.280 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 3456_fsm 2024-05-03T18:28:17.280 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3456_vm 2024-05-03T18:28:17.280 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 3466 2024-05-03T18:28:17.281 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3467 2024-05-03T18:28:17.281 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3468 2024-05-03T18:28:17.281 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 3501 2024-05-03T18:28:17.281 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3502 2024-05-03T18:28:17.281 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3503 2024-05-03T18:28:17.281 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3534 2024-05-03T18:28:17.281 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3541 2024-05-03T18:28:17.281 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 3541_fsm 2024-05-03T18:28:17.281 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3541_vm 2024-05-03T18:28:17.281 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 3542 2024-05-03T18:28:17.281 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3574 2024-05-03T18:28:17.281 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3575 2024-05-03T18:28:17.281 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 3576 2024-05-03T18:28:17.282 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 3596 2024-05-03T18:28:17.282 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3597 2024-05-03T18:28:17.282 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 3598 2024-05-03T18:28:17.282 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3599 2024-05-03T18:28:17.282 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3600 2024-05-03T18:28:17.282 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 3600_fsm 2024-05-03T18:28:17.282 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3600_vm 2024-05-03T18:28:17.282 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3601 2024-05-03T18:28:17.282 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 3601_fsm 2024-05-03T18:28:17.282 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3601_vm 2024-05-03T18:28:17.282 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3602 2024-05-03T18:28:17.282 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 3602_fsm 2024-05-03T18:28:17.282 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3602_vm 2024-05-03T18:28:17.282 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 3603 2024-05-03T18:28:17.283 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 3603_fsm 2024-05-03T18:28:17.283 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3603_vm 2024-05-03T18:28:17.283 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 3604 2024-05-03T18:28:17.283 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 3605 2024-05-03T18:28:17.283 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 3606 2024-05-03T18:28:17.283 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 3607 2024-05-03T18:28:17.283 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 3608 2024-05-03T18:28:17.283 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 3 18:27 3609 2024-05-03T18:28:17.283 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 3712 2024-05-03T18:28:17.283 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3764 2024-05-03T18:28:17.283 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 3764_fsm 2024-05-03T18:28:17.283 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3764_vm 2024-05-03T18:28:17.284 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 3766 2024-05-03T18:28:17.284 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 3767 2024-05-03T18:28:17.284 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3997 2024-05-03T18:28:17.284 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4143 2024-05-03T18:28:17.284 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4144 2024-05-03T18:28:17.284 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4145 2024-05-03T18:28:17.284 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4146 2024-05-03T18:28:17.284 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4147 2024-05-03T18:28:17.284 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4148 2024-05-03T18:28:17.284 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4149 2024-05-03T18:28:17.284 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4150 2024-05-03T18:28:17.284 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4151 2024-05-03T18:28:17.284 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4152 2024-05-03T18:28:17.284 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4153 2024-05-03T18:28:17.284 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4154 2024-05-03T18:28:17.285 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4155 2024-05-03T18:28:17.285 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4156 2024-05-03T18:28:17.285 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4157 2024-05-03T18:28:17.285 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4158 2024-05-03T18:28:17.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:28:16 smithi099 ceph-mon[31658]: pgmap v35: 129 pgs: 129 active+clean; 52 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 475 KiB/s wr, 66 op/s 2024-05-03T18:28:17.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:28:16 smithi150 ceph-mon[32492]: pgmap v35: 129 pgs: 129 active+clean; 52 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 475 KiB/s wr, 66 op/s 2024-05-03T18:28:19.074 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4159 2024-05-03T18:28:19.074 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4160 2024-05-03T18:28:19.075 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4161 2024-05-03T18:28:19.075 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4162 2024-05-03T18:28:19.075 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4163 2024-05-03T18:28:19.075 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4164 2024-05-03T18:28:19.075 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4165 2024-05-03T18:28:19.075 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4166 2024-05-03T18:28:19.075 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4167 2024-05-03T18:28:19.075 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4168 2024-05-03T18:28:19.075 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4169 2024-05-03T18:28:19.075 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4170 2024-05-03T18:28:19.075 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4171 2024-05-03T18:28:19.076 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4172 2024-05-03T18:28:19.076 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4173 2024-05-03T18:28:19.076 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4174 2024-05-03T18:28:19.076 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 5002 2024-05-03T18:28:19.076 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 548 2024-05-03T18:28:19.076 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 549 2024-05-03T18:28:19.076 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 6102 2024-05-03T18:28:19.076 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 6104 2024-05-03T18:28:19.076 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 6106 2024-05-03T18:28:19.076 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 6110 2024-05-03T18:28:19.076 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 6111 2024-05-03T18:28:19.076 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 6112 2024-05-03T18:28:19.076 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 6113 2024-05-03T18:28:19.077 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 6117 2024-05-03T18:28:19.077 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 826 2024-05-03T18:28:19.077 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 827 2024-05-03T18:28:19.077 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 828 2024-05-03T18:28:19.077 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 May 3 18:27 pg_filenode.map 2024-05-03T18:28:19.077 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 3 18:27 PG_VERSION 2024-05-03T18:28:19.077 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:28:19.077 INFO:teuthology.orchestra.run.smithi033.stdout:/tmp/cephfs/postgres/data/base/13493: 2024-05-03T18:28:19.077 INFO:teuthology.orchestra.run.smithi033.stdout:total 7778 2024-05-03T18:28:19.077 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7963139 May 3 18:27 . 2024-05-03T18:28:19.077 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 23889417 May 3 18:27 .. 2024-05-03T18:28:19.077 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 112 2024-05-03T18:28:19.077 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 113 2024-05-03T18:28:19.077 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 3 18:27 1247 2024-05-03T18:28:19.077 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 1247_fsm 2024-05-03T18:28:19.077 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 1247_vm 2024-05-03T18:28:19.078 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 May 3 18:27 1249 2024-05-03T18:28:19.078 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 1249_fsm 2024-05-03T18:28:19.078 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 1249_vm 2024-05-03T18:28:19.078 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 May 3 18:27 1255 2024-05-03T18:28:19.078 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 1255_fsm 2024-05-03T18:28:19.078 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 1255_vm 2024-05-03T18:28:19.078 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 May 3 18:27 1259 2024-05-03T18:28:19.078 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 1259_fsm 2024-05-03T18:28:19.078 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 1259_vm 2024-05-03T18:28:19.078 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 May 3 18:27 13344 2024-05-03T18:28:19.078 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 13344_fsm 2024-05-03T18:28:19.078 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 13344_vm 2024-05-03T18:28:19.078 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 13346 2024-05-03T18:28:19.079 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 13348 2024-05-03T18:28:19.079 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 13349 2024-05-03T18:28:19.079 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 13349_fsm 2024-05-03T18:28:19.079 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 13349_vm 2024-05-03T18:28:19.079 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 13351 2024-05-03T18:28:19.079 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 13353 2024-05-03T18:28:19.079 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 13354 2024-05-03T18:28:19.079 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 13354_fsm 2024-05-03T18:28:19.079 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 13354_vm 2024-05-03T18:28:19.079 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 13356 2024-05-03T18:28:19.079 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 13358 2024-05-03T18:28:19.079 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 13359 2024-05-03T18:28:19.079 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 13359_fsm 2024-05-03T18:28:19.080 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 13359_vm 2024-05-03T18:28:19.080 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 13361 2024-05-03T18:28:19.080 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 13363 2024-05-03T18:28:19.080 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 1417 2024-05-03T18:28:19.080 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 1418 2024-05-03T18:28:19.080 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 174 2024-05-03T18:28:19.080 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 175 2024-05-03T18:28:19.080 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2187 2024-05-03T18:28:19.080 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 2224 2024-05-03T18:28:19.080 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 2328 2024-05-03T18:28:19.080 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 2336 2024-05-03T18:28:19.080 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2337 2024-05-03T18:28:19.080 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2579 2024-05-03T18:28:19.080 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2600 2024-05-03T18:28:19.081 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2600_fsm 2024-05-03T18:28:19.081 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2600_vm 2024-05-03T18:28:19.081 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2601 2024-05-03T18:28:19.081 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2601_fsm 2024-05-03T18:28:19.081 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2601_vm 2024-05-03T18:28:19.081 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 3 18:27 2602 2024-05-03T18:28:19.081 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2602_fsm 2024-05-03T18:28:19.081 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2602_vm 2024-05-03T18:28:19.081 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 3 18:27 2603 2024-05-03T18:28:19.081 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2603_fsm 2024-05-03T18:28:19.081 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2603_vm 2024-05-03T18:28:19.081 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 2604 2024-05-03T18:28:19.081 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2605 2024-05-03T18:28:19.081 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2605_fsm 2024-05-03T18:28:19.082 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2605_vm 2024-05-03T18:28:19.082 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2606 2024-05-03T18:28:19.082 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2606_fsm 2024-05-03T18:28:19.082 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2606_vm 2024-05-03T18:28:19.082 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2607 2024-05-03T18:28:19.082 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2607_fsm 2024-05-03T18:28:19.082 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2607_vm 2024-05-03T18:28:19.082 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 May 3 18:27 2608 2024-05-03T18:28:19.082 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2608_fsm 2024-05-03T18:28:19.082 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2608_vm 2024-05-03T18:28:19.082 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 May 3 18:27 2609 2024-05-03T18:28:19.082 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2609_fsm 2024-05-03T18:28:19.083 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2609_vm 2024-05-03T18:28:19.083 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 3 18:27 2610 2024-05-03T18:28:19.083 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2610_fsm 2024-05-03T18:28:19.083 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2610_vm 2024-05-03T18:28:19.083 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 2611 2024-05-03T18:28:19.083 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2612 2024-05-03T18:28:19.083 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2612_fsm 2024-05-03T18:28:19.083 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2612_vm 2024-05-03T18:28:19.083 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 2613 2024-05-03T18:28:19.083 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2615 2024-05-03T18:28:19.083 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2615_fsm 2024-05-03T18:28:19.083 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2615_vm 2024-05-03T18:28:19.084 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2616 2024-05-03T18:28:19.084 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2616_fsm 2024-05-03T18:28:19.084 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2616_vm 2024-05-03T18:28:19.084 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 May 3 18:27 2617 2024-05-03T18:28:19.084 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2617_fsm 2024-05-03T18:28:19.084 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2617_vm 2024-05-03T18:28:19.084 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 May 3 18:27 2618 2024-05-03T18:28:19.084 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2618_fsm 2024-05-03T18:28:19.084 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2618_vm 2024-05-03T18:28:19.084 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 May 3 18:27 2619 2024-05-03T18:28:19.084 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2619_fsm 2024-05-03T18:28:19.084 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2619_vm 2024-05-03T18:28:19.084 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 2620 2024-05-03T18:28:19.084 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2650 2024-05-03T18:28:19.085 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2651 2024-05-03T18:28:19.085 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2652 2024-05-03T18:28:19.085 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 3 18:27 2653 2024-05-03T18:28:19.085 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 3 18:27 2654 2024-05-03T18:28:19.085 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 3 18:27 2655 2024-05-03T18:28:19.085 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2656 2024-05-03T18:28:19.085 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2657 2024-05-03T18:28:19.085 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 May 3 18:27 2658 2024-05-03T18:28:19.085 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 3 18:27 2659 2024-05-03T18:28:19.085 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2660 2024-05-03T18:28:19.085 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2661 2024-05-03T18:28:19.085 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 3 18:27 2662 2024-05-03T18:28:19.085 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 3 18:27 2663 2024-05-03T18:28:19.086 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2664 2024-05-03T18:28:19.086 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2665 2024-05-03T18:28:19.086 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2666 2024-05-03T18:28:19.086 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2667 2024-05-03T18:28:19.086 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2668 2024-05-03T18:28:19.086 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2669 2024-05-03T18:28:19.086 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2670 2024-05-03T18:28:19.086 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 May 3 18:27 2673 2024-05-03T18:28:19.086 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 May 3 18:27 2674 2024-05-03T18:28:19.086 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 May 3 18:27 2675 2024-05-03T18:28:19.086 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2678 2024-05-03T18:28:19.086 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2679 2024-05-03T18:28:19.086 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2680 2024-05-03T18:28:19.087 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2681 2024-05-03T18:28:19.087 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2682 2024-05-03T18:28:19.087 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2683 2024-05-03T18:28:19.087 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2684 2024-05-03T18:28:19.087 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2685 2024-05-03T18:28:19.087 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2686 2024-05-03T18:28:19.087 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2687 2024-05-03T18:28:19.087 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 3 18:27 2688 2024-05-03T18:28:19.087 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 3 18:27 2689 2024-05-03T18:28:19.087 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 May 3 18:27 2690 2024-05-03T18:28:19.087 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 May 3 18:27 2691 2024-05-03T18:28:19.087 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2692 2024-05-03T18:28:19.087 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2693 2024-05-03T18:28:19.088 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2696 2024-05-03T18:28:19.088 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2699 2024-05-03T18:28:19.088 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2701 2024-05-03T18:28:19.088 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2702 2024-05-03T18:28:19.088 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 3 18:27 2703 2024-05-03T18:28:19.088 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 3 18:27 2704 2024-05-03T18:28:19.088 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2753 2024-05-03T18:28:19.088 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2753_fsm 2024-05-03T18:28:19.088 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2753_vm 2024-05-03T18:28:19.088 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2754 2024-05-03T18:28:19.088 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2755 2024-05-03T18:28:19.088 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 3 18:27 2756 2024-05-03T18:28:19.089 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 3 18:27 2757 2024-05-03T18:28:19.089 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 2830 2024-05-03T18:28:19.089 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2831 2024-05-03T18:28:19.089 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 2832 2024-05-03T18:28:19.089 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2833 2024-05-03T18:28:19.089 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 2834 2024-05-03T18:28:19.089 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2835 2024-05-03T18:28:19.089 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 2836 2024-05-03T18:28:19.089 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2837 2024-05-03T18:28:19.089 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 May 3 18:27 2838 2024-05-03T18:28:19.089 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2838_fsm 2024-05-03T18:28:19.089 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2838_vm 2024-05-03T18:28:19.089 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2839 2024-05-03T18:28:19.090 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2840 2024-05-03T18:28:19.090 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2840_fsm 2024-05-03T18:28:19.090 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2840_vm 2024-05-03T18:28:19.090 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2841 2024-05-03T18:28:19.090 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 2995 2024-05-03T18:28:19.090 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2996 2024-05-03T18:28:19.090 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3079 2024-05-03T18:28:19.090 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 3079_fsm 2024-05-03T18:28:19.090 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3079_vm 2024-05-03T18:28:19.090 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 3080 2024-05-03T18:28:19.090 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 3081 2024-05-03T18:28:19.090 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 3 18:27 3085 2024-05-03T18:28:19.091 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 3118 2024-05-03T18:28:19.091 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3119 2024-05-03T18:28:19.091 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 May 3 18:27 3164 2024-05-03T18:28:19.091 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 3256 2024-05-03T18:28:19.091 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3257 2024-05-03T18:28:19.091 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3258 2024-05-03T18:28:19.091 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 3350 2024-05-03T18:28:19.091 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3351 2024-05-03T18:28:19.091 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3379 2024-05-03T18:28:19.091 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3380 2024-05-03T18:28:19.091 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 3381 2024-05-03T18:28:19.091 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 3394 2024-05-03T18:28:19.091 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 3394_fsm 2024-05-03T18:28:19.091 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3394_vm 2024-05-03T18:28:19.092 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 3395 2024-05-03T18:28:19.092 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 3429 2024-05-03T18:28:19.092 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 3430 2024-05-03T18:28:19.092 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3431 2024-05-03T18:28:19.092 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3433 2024-05-03T18:28:19.092 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 3439 2024-05-03T18:28:19.092 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3440 2024-05-03T18:28:19.092 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 3 18:27 3455 2024-05-03T18:28:19.092 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 May 3 18:27 3456 2024-05-03T18:28:19.092 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 3456_fsm 2024-05-03T18:28:19.092 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3456_vm 2024-05-03T18:28:19.092 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 3466 2024-05-03T18:28:19.092 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3467 2024-05-03T18:28:19.092 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3468 2024-05-03T18:28:19.093 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 3501 2024-05-03T18:28:19.093 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3502 2024-05-03T18:28:19.093 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3503 2024-05-03T18:28:19.093 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3534 2024-05-03T18:28:19.093 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3541 2024-05-03T18:28:19.093 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 3541_fsm 2024-05-03T18:28:19.093 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3541_vm 2024-05-03T18:28:19.093 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 3542 2024-05-03T18:28:19.093 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3574 2024-05-03T18:28:19.093 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3575 2024-05-03T18:28:19.093 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 3576 2024-05-03T18:28:19.093 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 3596 2024-05-03T18:28:19.093 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3597 2024-05-03T18:28:19.094 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 3598 2024-05-03T18:28:19.094 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3599 2024-05-03T18:28:19.094 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3600 2024-05-03T18:28:19.094 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 3600_fsm 2024-05-03T18:28:19.094 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3600_vm 2024-05-03T18:28:19.094 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3601 2024-05-03T18:28:19.094 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 3601_fsm 2024-05-03T18:28:19.094 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3601_vm 2024-05-03T18:28:19.094 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3602 2024-05-03T18:28:19.094 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 3602_fsm 2024-05-03T18:28:19.094 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3602_vm 2024-05-03T18:28:19.094 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 3603 2024-05-03T18:28:19.094 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 3603_fsm 2024-05-03T18:28:19.095 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3603_vm 2024-05-03T18:28:19.095 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 3604 2024-05-03T18:28:19.095 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 3605 2024-05-03T18:28:19.095 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 3606 2024-05-03T18:28:19.095 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 3607 2024-05-03T18:28:19.095 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 3608 2024-05-03T18:28:19.095 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 3 18:27 3609 2024-05-03T18:28:19.095 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 3712 2024-05-03T18:28:19.095 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3764 2024-05-03T18:28:19.095 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 3764_fsm 2024-05-03T18:28:19.095 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3764_vm 2024-05-03T18:28:19.095 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 3766 2024-05-03T18:28:19.095 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 3767 2024-05-03T18:28:19.096 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3997 2024-05-03T18:28:19.096 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4143 2024-05-03T18:28:19.096 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4144 2024-05-03T18:28:19.096 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4145 2024-05-03T18:28:19.096 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4146 2024-05-03T18:28:19.096 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4147 2024-05-03T18:28:19.096 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4148 2024-05-03T18:28:19.096 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4149 2024-05-03T18:28:19.096 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4150 2024-05-03T18:28:19.096 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4151 2024-05-03T18:28:19.096 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4152 2024-05-03T18:28:19.096 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4153 2024-05-03T18:28:19.097 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4154 2024-05-03T18:28:19.097 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4155 2024-05-03T18:28:19.097 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4156 2024-05-03T18:28:19.097 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4157 2024-05-03T18:28:19.097 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4158 2024-05-03T18:28:19.097 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4159 2024-05-03T18:28:19.097 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4160 2024-05-03T18:28:19.097 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4161 2024-05-03T18:28:19.097 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4162 2024-05-03T18:28:19.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:28:18 smithi033 ceph-mon[28330]: pgmap v36: 129 pgs: 129 active+clean; 53 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 551 KiB/s rd, 579 KiB/s wr, 86 op/s 2024-05-03T18:28:19.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:28:18 smithi099 ceph-mon[31658]: pgmap v36: 129 pgs: 129 active+clean; 53 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 551 KiB/s rd, 579 KiB/s wr, 86 op/s 2024-05-03T18:28:19.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:28:18 smithi150 ceph-mon[32492]: pgmap v36: 129 pgs: 129 active+clean; 53 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 551 KiB/s rd, 579 KiB/s wr, 86 op/s 2024-05-03T18:28:20.898 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4163 2024-05-03T18:28:20.899 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4164 2024-05-03T18:28:20.899 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4165 2024-05-03T18:28:20.899 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4166 2024-05-03T18:28:20.899 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4167 2024-05-03T18:28:20.899 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4168 2024-05-03T18:28:20.899 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4169 2024-05-03T18:28:20.899 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4170 2024-05-03T18:28:20.899 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4171 2024-05-03T18:28:20.899 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4172 2024-05-03T18:28:20.899 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4173 2024-05-03T18:28:20.899 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4174 2024-05-03T18:28:20.900 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 5002 2024-05-03T18:28:20.900 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 548 2024-05-03T18:28:20.900 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 549 2024-05-03T18:28:20.900 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 6102 2024-05-03T18:28:20.900 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 6104 2024-05-03T18:28:20.900 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 6106 2024-05-03T18:28:20.900 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 6110 2024-05-03T18:28:20.900 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 6111 2024-05-03T18:28:20.900 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 6112 2024-05-03T18:28:20.900 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 6113 2024-05-03T18:28:20.900 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 6117 2024-05-03T18:28:20.901 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 826 2024-05-03T18:28:20.901 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 827 2024-05-03T18:28:20.901 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 828 2024-05-03T18:28:20.901 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 May 3 18:27 pg_filenode.map 2024-05-03T18:28:20.901 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 3 18:27 PG_VERSION 2024-05-03T18:28:20.901 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:28:20.902 INFO:teuthology.orchestra.run.smithi033.stdout:/tmp/cephfs/postgres/data/base/13494: 2024-05-03T18:28:20.902 INFO:teuthology.orchestra.run.smithi033.stdout:total 7778 2024-05-03T18:28:20.902 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7963139 May 3 18:27 . 2024-05-03T18:28:20.902 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 23889417 May 3 18:27 .. 2024-05-03T18:28:20.902 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 112 2024-05-03T18:28:20.902 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 113 2024-05-03T18:28:20.903 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 3 18:27 1247 2024-05-03T18:28:20.903 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 1247_fsm 2024-05-03T18:28:20.903 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 1247_vm 2024-05-03T18:28:20.903 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 May 3 18:27 1249 2024-05-03T18:28:20.903 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 1249_fsm 2024-05-03T18:28:20.903 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 1249_vm 2024-05-03T18:28:20.904 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 May 3 18:27 1255 2024-05-03T18:28:20.904 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 1255_fsm 2024-05-03T18:28:20.904 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 1255_vm 2024-05-03T18:28:20.904 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 May 3 18:27 1259 2024-05-03T18:28:20.904 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 1259_fsm 2024-05-03T18:28:20.904 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 1259_vm 2024-05-03T18:28:20.905 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 May 3 18:27 13344 2024-05-03T18:28:20.905 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 13344_fsm 2024-05-03T18:28:20.905 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 13344_vm 2024-05-03T18:28:20.905 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 13346 2024-05-03T18:28:20.905 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 13348 2024-05-03T18:28:20.905 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 13349 2024-05-03T18:28:20.905 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 13349_fsm 2024-05-03T18:28:20.906 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 13349_vm 2024-05-03T18:28:20.906 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 13351 2024-05-03T18:28:20.906 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 13353 2024-05-03T18:28:20.906 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 13354 2024-05-03T18:28:20.906 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 13354_fsm 2024-05-03T18:28:20.907 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 13354_vm 2024-05-03T18:28:20.907 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 13356 2024-05-03T18:28:20.907 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 13358 2024-05-03T18:28:20.907 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 13359 2024-05-03T18:28:20.907 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 13359_fsm 2024-05-03T18:28:20.907 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 13359_vm 2024-05-03T18:28:20.907 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 13361 2024-05-03T18:28:20.908 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 13363 2024-05-03T18:28:20.908 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 1417 2024-05-03T18:28:20.908 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 1418 2024-05-03T18:28:20.908 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 174 2024-05-03T18:28:20.908 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 175 2024-05-03T18:28:20.908 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2187 2024-05-03T18:28:20.909 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 2224 2024-05-03T18:28:20.909 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 2328 2024-05-03T18:28:20.909 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 2336 2024-05-03T18:28:20.909 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2337 2024-05-03T18:28:20.909 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2579 2024-05-03T18:28:20.909 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2600 2024-05-03T18:28:20.909 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2600_fsm 2024-05-03T18:28:20.910 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2600_vm 2024-05-03T18:28:20.910 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2601 2024-05-03T18:28:20.910 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2601_fsm 2024-05-03T18:28:20.910 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2601_vm 2024-05-03T18:28:20.910 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 3 18:27 2602 2024-05-03T18:28:20.910 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2602_fsm 2024-05-03T18:28:20.911 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2602_vm 2024-05-03T18:28:20.911 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 3 18:27 2603 2024-05-03T18:28:20.911 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2603_fsm 2024-05-03T18:28:20.911 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2603_vm 2024-05-03T18:28:20.911 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 2604 2024-05-03T18:28:20.912 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2605 2024-05-03T18:28:20.912 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2605_fsm 2024-05-03T18:28:20.912 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2605_vm 2024-05-03T18:28:20.912 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2606 2024-05-03T18:28:20.912 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2606_fsm 2024-05-03T18:28:20.912 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2606_vm 2024-05-03T18:28:20.912 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2607 2024-05-03T18:28:20.913 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2607_fsm 2024-05-03T18:28:20.913 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2607_vm 2024-05-03T18:28:20.913 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 May 3 18:27 2608 2024-05-03T18:28:20.913 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2608_fsm 2024-05-03T18:28:20.913 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2608_vm 2024-05-03T18:28:20.913 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 May 3 18:27 2609 2024-05-03T18:28:20.914 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2609_fsm 2024-05-03T18:28:20.914 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2609_vm 2024-05-03T18:28:20.914 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 3 18:27 2610 2024-05-03T18:28:20.914 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2610_fsm 2024-05-03T18:28:20.914 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2610_vm 2024-05-03T18:28:20.914 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 2611 2024-05-03T18:28:20.915 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2612 2024-05-03T18:28:20.915 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2612_fsm 2024-05-03T18:28:20.915 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2612_vm 2024-05-03T18:28:20.915 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 2613 2024-05-03T18:28:20.915 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2615 2024-05-03T18:28:20.915 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2615_fsm 2024-05-03T18:28:20.916 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2615_vm 2024-05-03T18:28:20.916 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2616 2024-05-03T18:28:20.916 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2616_fsm 2024-05-03T18:28:20.916 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2616_vm 2024-05-03T18:28:20.916 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 May 3 18:27 2617 2024-05-03T18:28:20.916 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2617_fsm 2024-05-03T18:28:20.917 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2617_vm 2024-05-03T18:28:20.917 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 May 3 18:27 2618 2024-05-03T18:28:20.917 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2618_fsm 2024-05-03T18:28:20.917 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2618_vm 2024-05-03T18:28:20.917 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 May 3 18:27 2619 2024-05-03T18:28:20.917 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2619_fsm 2024-05-03T18:28:20.918 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2619_vm 2024-05-03T18:28:20.918 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 2620 2024-05-03T18:28:20.918 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2650 2024-05-03T18:28:20.918 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2651 2024-05-03T18:28:20.918 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2652 2024-05-03T18:28:20.918 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 3 18:27 2653 2024-05-03T18:28:20.919 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 3 18:27 2654 2024-05-03T18:28:20.919 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 3 18:27 2655 2024-05-03T18:28:20.919 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2656 2024-05-03T18:28:20.919 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2657 2024-05-03T18:28:20.920 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 May 3 18:27 2658 2024-05-03T18:28:20.920 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 3 18:27 2659 2024-05-03T18:28:20.920 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2660 2024-05-03T18:28:20.920 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2661 2024-05-03T18:28:20.920 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 3 18:27 2662 2024-05-03T18:28:20.921 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 3 18:27 2663 2024-05-03T18:28:20.921 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2664 2024-05-03T18:28:20.921 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2665 2024-05-03T18:28:20.921 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2666 2024-05-03T18:28:20.921 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2667 2024-05-03T18:28:20.921 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2668 2024-05-03T18:28:20.922 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2669 2024-05-03T18:28:20.922 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2670 2024-05-03T18:28:20.922 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 May 3 18:27 2673 2024-05-03T18:28:20.922 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 May 3 18:27 2674 2024-05-03T18:28:20.922 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 May 3 18:27 2675 2024-05-03T18:28:20.922 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2678 2024-05-03T18:28:20.922 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2679 2024-05-03T18:28:20.922 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2680 2024-05-03T18:28:20.923 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2681 2024-05-03T18:28:20.923 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2682 2024-05-03T18:28:20.923 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2683 2024-05-03T18:28:20.923 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2684 2024-05-03T18:28:20.923 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2685 2024-05-03T18:28:20.923 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2686 2024-05-03T18:28:20.923 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2687 2024-05-03T18:28:20.923 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 3 18:27 2688 2024-05-03T18:28:20.923 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 3 18:27 2689 2024-05-03T18:28:20.923 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 May 3 18:27 2690 2024-05-03T18:28:20.923 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 May 3 18:27 2691 2024-05-03T18:28:20.923 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2692 2024-05-03T18:28:20.923 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2693 2024-05-03T18:28:20.923 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2696 2024-05-03T18:28:20.923 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2699 2024-05-03T18:28:20.924 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2701 2024-05-03T18:28:20.924 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2702 2024-05-03T18:28:20.924 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 3 18:27 2703 2024-05-03T18:28:20.924 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 3 18:27 2704 2024-05-03T18:28:20.924 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2753 2024-05-03T18:28:20.924 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2753_fsm 2024-05-03T18:28:20.924 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2753_vm 2024-05-03T18:28:20.924 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2754 2024-05-03T18:28:20.924 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2755 2024-05-03T18:28:20.924 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 3 18:27 2756 2024-05-03T18:28:20.925 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 3 18:27 2757 2024-05-03T18:28:20.925 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 2830 2024-05-03T18:28:20.925 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2831 2024-05-03T18:28:20.925 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 2832 2024-05-03T18:28:20.925 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2833 2024-05-03T18:28:20.925 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 2834 2024-05-03T18:28:20.925 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2835 2024-05-03T18:28:20.925 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 2836 2024-05-03T18:28:20.925 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2837 2024-05-03T18:28:20.925 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 May 3 18:27 2838 2024-05-03T18:28:20.925 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2838_fsm 2024-05-03T18:28:20.925 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2838_vm 2024-05-03T18:28:20.925 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2839 2024-05-03T18:28:20.925 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2840 2024-05-03T18:28:20.926 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2840_fsm 2024-05-03T18:28:20.926 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2840_vm 2024-05-03T18:28:20.926 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2841 2024-05-03T18:28:20.926 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 2995 2024-05-03T18:28:20.926 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2996 2024-05-03T18:28:20.926 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3079 2024-05-03T18:28:20.926 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 3079_fsm 2024-05-03T18:28:20.926 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3079_vm 2024-05-03T18:28:20.926 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 3080 2024-05-03T18:28:20.926 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 3081 2024-05-03T18:28:20.926 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 3 18:27 3085 2024-05-03T18:28:20.926 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 3118 2024-05-03T18:28:20.926 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3119 2024-05-03T18:28:20.926 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 May 3 18:27 3164 2024-05-03T18:28:20.927 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 3256 2024-05-03T18:28:20.927 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3257 2024-05-03T18:28:20.927 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3258 2024-05-03T18:28:20.927 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 3350 2024-05-03T18:28:20.927 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3351 2024-05-03T18:28:20.927 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3379 2024-05-03T18:28:20.927 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3380 2024-05-03T18:28:20.927 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 3381 2024-05-03T18:28:20.927 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 3394 2024-05-03T18:28:20.927 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 3394_fsm 2024-05-03T18:28:20.927 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3394_vm 2024-05-03T18:28:20.927 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 3395 2024-05-03T18:28:20.927 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 3429 2024-05-03T18:28:20.927 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 3430 2024-05-03T18:28:20.927 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3431 2024-05-03T18:28:20.928 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3433 2024-05-03T18:28:20.928 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 3439 2024-05-03T18:28:20.928 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3440 2024-05-03T18:28:20.928 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 3 18:27 3455 2024-05-03T18:28:20.928 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 May 3 18:27 3456 2024-05-03T18:28:20.928 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 3456_fsm 2024-05-03T18:28:20.928 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3456_vm 2024-05-03T18:28:20.928 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 3466 2024-05-03T18:28:20.928 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3467 2024-05-03T18:28:20.928 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3468 2024-05-03T18:28:20.928 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 3501 2024-05-03T18:28:20.928 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3502 2024-05-03T18:28:20.928 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3503 2024-05-03T18:28:20.928 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3534 2024-05-03T18:28:20.929 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3541 2024-05-03T18:28:20.929 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 3541_fsm 2024-05-03T18:28:20.929 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3541_vm 2024-05-03T18:28:20.929 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 3542 2024-05-03T18:28:20.929 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3574 2024-05-03T18:28:20.929 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3575 2024-05-03T18:28:20.929 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 3576 2024-05-03T18:28:20.929 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 3596 2024-05-03T18:28:20.929 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3597 2024-05-03T18:28:20.929 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 3598 2024-05-03T18:28:20.929 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3599 2024-05-03T18:28:20.929 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3600 2024-05-03T18:28:20.929 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 3600_fsm 2024-05-03T18:28:20.929 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3600_vm 2024-05-03T18:28:20.930 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3601 2024-05-03T18:28:20.930 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 3601_fsm 2024-05-03T18:28:20.930 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3601_vm 2024-05-03T18:28:20.930 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3602 2024-05-03T18:28:20.930 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 3602_fsm 2024-05-03T18:28:20.930 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3602_vm 2024-05-03T18:28:20.930 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 3603 2024-05-03T18:28:20.930 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 3603_fsm 2024-05-03T18:28:20.930 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3603_vm 2024-05-03T18:28:20.930 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 3604 2024-05-03T18:28:20.930 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 3605 2024-05-03T18:28:20.930 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 3606 2024-05-03T18:28:20.930 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 3607 2024-05-03T18:28:20.931 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 3608 2024-05-03T18:28:20.931 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 3 18:27 3609 2024-05-03T18:28:20.931 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 3712 2024-05-03T18:28:20.931 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3764 2024-05-03T18:28:20.931 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 3764_fsm 2024-05-03T18:28:20.931 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3764_vm 2024-05-03T18:28:20.931 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 3766 2024-05-03T18:28:20.931 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 3767 2024-05-03T18:28:20.931 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3997 2024-05-03T18:28:20.931 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4143 2024-05-03T18:28:20.931 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4144 2024-05-03T18:28:20.931 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4145 2024-05-03T18:28:20.931 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4146 2024-05-03T18:28:20.931 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4147 2024-05-03T18:28:20.932 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4148 2024-05-03T18:28:20.932 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4149 2024-05-03T18:28:20.932 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4150 2024-05-03T18:28:20.932 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4151 2024-05-03T18:28:20.932 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4152 2024-05-03T18:28:20.932 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4153 2024-05-03T18:28:20.932 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4154 2024-05-03T18:28:20.932 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4155 2024-05-03T18:28:20.932 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4156 2024-05-03T18:28:20.932 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4157 2024-05-03T18:28:20.932 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4158 2024-05-03T18:28:20.932 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4159 2024-05-03T18:28:20.932 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4160 2024-05-03T18:28:20.932 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4161 2024-05-03T18:28:20.933 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4162 2024-05-03T18:28:20.933 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4163 2024-05-03T18:28:20.933 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4164 2024-05-03T18:28:20.933 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4165 2024-05-03T18:28:20.933 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4166 2024-05-03T18:28:20.933 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4167 2024-05-03T18:28:21.208 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4168 2024-05-03T18:28:21.208 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4169 2024-05-03T18:28:21.208 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4170 2024-05-03T18:28:21.208 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4171 2024-05-03T18:28:21.208 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4172 2024-05-03T18:28:21.208 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4173 2024-05-03T18:28:21.208 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4174 2024-05-03T18:28:21.208 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 5002 2024-05-03T18:28:21.208 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 548 2024-05-03T18:28:21.208 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 549 2024-05-03T18:28:21.208 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 6102 2024-05-03T18:28:21.208 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 6104 2024-05-03T18:28:21.209 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 6106 2024-05-03T18:28:21.209 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 6110 2024-05-03T18:28:21.209 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 6111 2024-05-03T18:28:21.209 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 6112 2024-05-03T18:28:21.209 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 6113 2024-05-03T18:28:21.209 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 6117 2024-05-03T18:28:21.209 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 826 2024-05-03T18:28:21.209 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 827 2024-05-03T18:28:21.209 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 828 2024-05-03T18:28:21.209 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 May 3 18:27 pg_filenode.map 2024-05-03T18:28:21.209 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 3 18:27 PG_VERSION 2024-05-03T18:28:21.209 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:28:21.209 INFO:teuthology.orchestra.run.smithi033.stdout:/tmp/cephfs/postgres/data/global: 2024-05-03T18:28:21.209 INFO:teuthology.orchestra.run.smithi033.stdout:total 538 2024-05-03T18:28:21.210 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 549376 May 3 18:27 . 2024-05-03T18:28:21.210 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 3 18:28 .. 2024-05-03T18:28:21.210 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 1213 2024-05-03T18:28:21.210 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 1213_fsm 2024-05-03T18:28:21.210 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 1213_vm 2024-05-03T18:28:21.210 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 1214 2024-05-03T18:28:21.210 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 1214_fsm 2024-05-03T18:28:21.210 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 1214_vm 2024-05-03T18:28:21.210 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 1232 2024-05-03T18:28:21.210 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 1233 2024-05-03T18:28:21.210 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 1260 2024-05-03T18:28:21.210 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 1260_fsm 2024-05-03T18:28:21.210 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 1260_vm 2024-05-03T18:28:21.210 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 1261 2024-05-03T18:28:21.211 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 1261_fsm 2024-05-03T18:28:21.211 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 1261_vm 2024-05-03T18:28:21.211 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 1262 2024-05-03T18:28:21.211 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 1262_fsm 2024-05-03T18:28:21.223 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 1262_vm 2024-05-03T18:28:21.223 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2396 2024-05-03T18:28:21.223 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2396_fsm 2024-05-03T18:28:21.223 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2396_vm 2024-05-03T18:28:21.223 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2397 2024-05-03T18:28:21.223 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2671 2024-05-03T18:28:21.223 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2672 2024-05-03T18:28:21.224 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2676 2024-05-03T18:28:21.224 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2677 2024-05-03T18:28:21.224 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2694 2024-05-03T18:28:21.224 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2695 2024-05-03T18:28:21.224 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2697 2024-05-03T18:28:21.224 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2698 2024-05-03T18:28:21.224 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 2846 2024-05-03T18:28:21.224 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2847 2024-05-03T18:28:21.224 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 2964 2024-05-03T18:28:21.224 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2965 2024-05-03T18:28:21.224 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 2966 2024-05-03T18:28:21.224 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2967 2024-05-03T18:28:21.224 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 3592 2024-05-03T18:28:21.224 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3593 2024-05-03T18:28:21.225 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4060 2024-05-03T18:28:21.225 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4061 2024-05-03T18:28:21.225 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4175 2024-05-03T18:28:21.225 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4176 2024-05-03T18:28:21.225 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4177 2024-05-03T18:28:21.225 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4178 2024-05-03T18:28:21.225 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4181 2024-05-03T18:28:21.225 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4182 2024-05-03T18:28:21.225 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4183 2024-05-03T18:28:21.225 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4184 2024-05-03T18:28:21.225 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4185 2024-05-03T18:28:21.225 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4186 2024-05-03T18:28:21.225 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 6000 2024-05-03T18:28:21.225 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 6001 2024-05-03T18:28:21.226 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 6002 2024-05-03T18:28:21.226 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 6100 2024-05-03T18:28:21.226 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 6114 2024-05-03T18:28:21.226 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 6115 2024-05-03T18:28:21.226 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 pg_control 2024-05-03T18:28:21.226 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 May 3 18:27 pg_filenode.map 2024-05-03T18:28:21.226 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:28:21.226 INFO:teuthology.orchestra.run.smithi033.stdout:/tmp/cephfs/postgres/data/log: 2024-05-03T18:28:21.226 INFO:teuthology.orchestra.run.smithi033.stdout:total 1 2024-05-03T18:28:21.226 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:28 . 2024-05-03T18:28:21.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:28:20 smithi033 ceph-mon[28330]: pgmap v37: 129 pgs: 129 active+clean; 53 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 551 KiB/s rd, 578 KiB/s wr, 85 op/s 2024-05-03T18:28:21.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:28:20 smithi099 ceph-mon[31658]: pgmap v37: 129 pgs: 129 active+clean; 53 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 551 KiB/s rd, 578 KiB/s wr, 85 op/s 2024-05-03T18:28:21.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:28:20 smithi150 ceph-mon[32492]: pgmap v37: 129 pgs: 129 active+clean; 53 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 551 KiB/s rd, 578 KiB/s wr, 85 op/s 2024-05-03T18:28:21.514 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 3 18:28 .. 2024-05-03T18:28:21.514 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:28:21.515 INFO:teuthology.orchestra.run.smithi033.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2024-05-03T18:28:21.515 INFO:teuthology.orchestra.run.smithi033.stdout:total 1 2024-05-03T18:28:21.515 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 . 2024-05-03T18:28:21.515 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 3 18:28 .. 2024-05-03T18:28:21.515 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:28:21.515 INFO:teuthology.orchestra.run.smithi033.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2024-05-03T18:28:21.515 INFO:teuthology.orchestra.run.smithi033.stdout:total 1 2024-05-03T18:28:21.515 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 . 2024-05-03T18:28:21.515 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 3 18:28 .. 2024-05-03T18:28:21.515 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:28:21.515 INFO:teuthology.orchestra.run.smithi033.stdout:/tmp/cephfs/postgres/data/pg_logical: 2024-05-03T18:28:21.515 INFO:teuthology.orchestra.run.smithi033.stdout:total 3 2024-05-03T18:28:21.515 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 May 3 18:27 . 2024-05-03T18:28:21.515 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 3 18:28 .. 2024-05-03T18:28:21.516 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 mappings 2024-05-03T18:28:21.516 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 May 3 18:27 replorigin_checkpoint 2024-05-03T18:28:21.516 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 snapshots 2024-05-03T18:28:21.516 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:28:21.516 INFO:teuthology.orchestra.run.smithi033.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2024-05-03T18:28:21.516 INFO:teuthology.orchestra.run.smithi033.stdout:total 1 2024-05-03T18:28:21.516 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 . 2024-05-03T18:28:21.516 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 May 3 18:27 .. 2024-05-03T18:28:21.516 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:28:21.516 INFO:teuthology.orchestra.run.smithi033.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2024-05-03T18:28:21.516 INFO:teuthology.orchestra.run.smithi033.stdout:total 1 2024-05-03T18:28:21.516 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 . 2024-05-03T18:28:21.516 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 May 3 18:27 .. 2024-05-03T18:28:21.516 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:28:21.516 INFO:teuthology.orchestra.run.smithi033.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2024-05-03T18:28:21.517 INFO:teuthology.orchestra.run.smithi033.stdout:total 2 2024-05-03T18:28:21.517 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 . 2024-05-03T18:28:21.517 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 3 18:28 .. 2024-05-03T18:28:21.517 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 members 2024-05-03T18:28:21.517 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 offsets 2024-05-03T18:28:21.517 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:28:21.517 INFO:teuthology.orchestra.run.smithi033.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2024-05-03T18:28:21.517 INFO:teuthology.orchestra.run.smithi033.stdout:total 9 2024-05-03T18:28:21.517 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 . 2024-05-03T18:28:21.517 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 .. 2024-05-03T18:28:21.517 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 0000 2024-05-03T18:28:21.518 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:28:21.518 INFO:teuthology.orchestra.run.smithi033.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2024-05-03T18:28:21.518 INFO:teuthology.orchestra.run.smithi033.stdout:total 9 2024-05-03T18:28:21.518 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 . 2024-05-03T18:28:21.518 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 .. 2024-05-03T18:28:21.518 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 0000 2024-05-03T18:28:21.518 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:28:21.518 INFO:teuthology.orchestra.run.smithi033.stdout:/tmp/cephfs/postgres/data/pg_notify: 2024-05-03T18:28:21.518 INFO:teuthology.orchestra.run.smithi033.stdout:total 1 2024-05-03T18:28:21.518 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 . 2024-05-03T18:28:21.518 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 3 18:28 .. 2024-05-03T18:28:21.519 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:28:21.519 INFO:teuthology.orchestra.run.smithi033.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2024-05-03T18:28:21.519 INFO:teuthology.orchestra.run.smithi033.stdout:total 1 2024-05-03T18:28:21.519 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 . 2024-05-03T18:28:21.519 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 3 18:28 .. 2024-05-03T18:28:21.519 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:28:21.519 INFO:teuthology.orchestra.run.smithi033.stdout:/tmp/cephfs/postgres/data/pg_serial: 2024-05-03T18:28:21.519 INFO:teuthology.orchestra.run.smithi033.stdout:total 1 2024-05-03T18:28:21.519 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 . 2024-05-03T18:28:21.519 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 3 18:28 .. 2024-05-03T18:28:21.519 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:28:21.519 INFO:teuthology.orchestra.run.smithi033.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2024-05-03T18:28:21.519 INFO:teuthology.orchestra.run.smithi033.stdout:total 1 2024-05-03T18:28:21.519 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 . 2024-05-03T18:28:21.519 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 3 18:28 .. 2024-05-03T18:28:21.520 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:28:21.520 INFO:teuthology.orchestra.run.smithi033.stdout:/tmp/cephfs/postgres/data/pg_stat: 2024-05-03T18:28:21.520 INFO:teuthology.orchestra.run.smithi033.stdout:total 1 2024-05-03T18:28:21.520 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 . 2024-05-03T18:28:21.520 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 3 18:28 .. 2024-05-03T18:28:21.520 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:28:21.520 INFO:teuthology.orchestra.run.smithi033.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2024-05-03T18:28:21.520 INFO:teuthology.orchestra.run.smithi033.stdout:total 1 2024-05-03T18:28:21.520 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 . 2024-05-03T18:28:21.520 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 3 18:28 .. 2024-05-03T18:28:21.520 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:28:21.521 INFO:teuthology.orchestra.run.smithi033.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2024-05-03T18:28:21.615 INFO:teuthology.orchestra.run.smithi033.stdout:total 9 2024-05-03T18:28:21.615 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 . 2024-05-03T18:28:21.616 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 3 18:28 .. 2024-05-03T18:28:21.616 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 0000 2024-05-03T18:28:21.616 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:28:21.616 INFO:teuthology.orchestra.run.smithi033.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2024-05-03T18:28:21.616 INFO:teuthology.orchestra.run.smithi033.stdout:total 1 2024-05-03T18:28:21.616 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 . 2024-05-03T18:28:21.616 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 3 18:28 .. 2024-05-03T18:28:21.616 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:28:21.616 INFO:teuthology.orchestra.run.smithi033.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2024-05-03T18:28:21.616 INFO:teuthology.orchestra.run.smithi033.stdout:total 1 2024-05-03T18:28:21.616 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 . 2024-05-03T18:28:21.616 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 3 18:28 .. 2024-05-03T18:28:21.616 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:28:21.617 INFO:teuthology.orchestra.run.smithi033.stdout:/tmp/cephfs/postgres/data/pg_wal: 2024-05-03T18:28:21.617 INFO:teuthology.orchestra.run.smithi033.stdout:total 16386 2024-05-03T18:28:21.617 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 3 18:27 . 2024-05-03T18:28:21.617 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 3 18:28 .. 2024-05-03T18:28:21.617 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 3 18:27 000000010000000000000001 2024-05-03T18:28:21.617 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 archive_status 2024-05-03T18:28:21.617 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:28:21.617 INFO:teuthology.orchestra.run.smithi033.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2024-05-03T18:28:21.617 INFO:teuthology.orchestra.run.smithi033.stdout:total 1 2024-05-03T18:28:21.617 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 . 2024-05-03T18:28:21.617 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 3 18:27 .. 2024-05-03T18:28:21.617 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:28:21.617 INFO:teuthology.orchestra.run.smithi033.stdout:/tmp/cephfs/postgres/data/pg_xact: 2024-05-03T18:28:21.617 INFO:teuthology.orchestra.run.smithi033.stdout:total 9 2024-05-03T18:28:21.618 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 . 2024-05-03T18:28:21.618 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 3 18:28 .. 2024-05-03T18:28:21.618 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 0000 2024-05-03T18:28:21.626 DEBUG:teuthology.orchestra.run.smithi033:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl start postgresql' 2024-05-03T18:28:22.260 DEBUG:teuthology.orchestra.run.smithi033:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -s 32 -i' 2024-05-03T18:28:22.526 INFO:teuthology.orchestra.run.smithi033.stderr:dropping old tables... 2024-05-03T18:28:22.551 INFO:teuthology.orchestra.run.smithi033.stderr:NOTICE: table "pgbench_accounts" does not exist, skipping 2024-05-03T18:28:22.551 INFO:teuthology.orchestra.run.smithi033.stderr:NOTICE: table "pgbench_branches" does not exist, skipping 2024-05-03T18:28:22.551 INFO:teuthology.orchestra.run.smithi033.stderr:NOTICE: table "pgbench_history" does not exist, skipping 2024-05-03T18:28:22.551 INFO:teuthology.orchestra.run.smithi033.stderr:NOTICE: table "pgbench_tellers" does not exist, skipping 2024-05-03T18:28:22.551 INFO:teuthology.orchestra.run.smithi033.stderr:creating tables... 2024-05-03T18:28:22.761 INFO:teuthology.orchestra.run.smithi033.stderr:generating data (client-side)... 2024-05-03T18:28:23.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:28:22 smithi033 ceph-mon[28330]: pgmap v38: 129 pgs: 129 active+clean; 53 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 374 KiB/s wr, 65 op/s 2024-05-03T18:28:23.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:28:22 smithi099 ceph-mon[31658]: pgmap v38: 129 pgs: 129 active+clean; 53 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 374 KiB/s wr, 65 op/s 2024-05-03T18:28:23.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:28:22 smithi150 ceph-mon[32492]: pgmap v38: 129 pgs: 129 active+clean; 53 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 374 KiB/s wr, 65 op/s 2024-05-03T18:28:25.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:28:24 smithi033 ceph-mon[28330]: pgmap v39: 129 pgs: 129 active+clean; 53 MiB data, 538 MiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 433 KiB/s wr, 75 op/s 2024-05-03T18:28:25.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:28:24 smithi099 ceph-mon[31658]: pgmap v39: 129 pgs: 129 active+clean; 53 MiB data, 538 MiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 433 KiB/s wr, 75 op/s 2024-05-03T18:28:25.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:28:24 smithi150 ceph-mon[32492]: pgmap v39: 129 pgs: 129 active+clean; 53 MiB data, 538 MiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 433 KiB/s wr, 75 op/s 2024-05-03T18:28:25.439 INFO:teuthology.orchestra.run.smithi033.stderr:100000 of 3200000 tuples (3%) done (elapsed 2.55 s, remaining 78.91 s) 2024-05-03T18:28:27.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:28:26 smithi033 ceph-mon[28330]: pgmap v40: 129 pgs: 129 active+clean; 53 MiB data, 538 MiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 162 KiB/s wr, 28 op/s 2024-05-03T18:28:27.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:28:26 smithi099 ceph-mon[31658]: pgmap v40: 129 pgs: 129 active+clean; 53 MiB data, 538 MiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 162 KiB/s wr, 28 op/s 2024-05-03T18:28:27.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:28:26 smithi150 ceph-mon[32492]: pgmap v40: 129 pgs: 129 active+clean; 53 MiB data, 538 MiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 162 KiB/s wr, 28 op/s 2024-05-03T18:28:29.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:28:28 smithi033 ceph-mon[28330]: pgmap v41: 129 pgs: 129 active+clean; 89 MiB data, 600 MiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 3.3 MiB/s wr, 33 op/s 2024-05-03T18:28:29.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:28:28 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:28:29.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:28:28 smithi099 ceph-mon[31658]: pgmap v41: 129 pgs: 129 active+clean; 89 MiB data, 600 MiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 3.3 MiB/s wr, 33 op/s 2024-05-03T18:28:29.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:28:28 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:28:29.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:28:28 smithi150 ceph-mon[32492]: pgmap v41: 129 pgs: 129 active+clean; 89 MiB data, 600 MiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 3.3 MiB/s wr, 33 op/s 2024-05-03T18:28:29.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:28:28 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:28:29.748 INFO:teuthology.orchestra.run.smithi033.stderr:200000 of 3200000 tuples (6%) done (elapsed 6.85 s, remaining 102.82 s) 2024-05-03T18:28:30.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:28:29 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:28:30.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:28:29 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:28:30.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:28:29 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:28:30.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:28:29 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:28:30.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:28:29 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:28:30.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:28:29 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:28:30.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:28:29 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:28:30.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:28:29 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:28:30.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:28:29 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:28:31.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:28:30 smithi033 ceph-mon[28330]: pgmap v42: 129 pgs: 129 active+clean; 89 MiB data, 600 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 3.2 MiB/s wr, 13 op/s 2024-05-03T18:28:31.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:28:30 smithi099 ceph-mon[31658]: pgmap v42: 129 pgs: 129 active+clean; 89 MiB data, 600 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 3.2 MiB/s wr, 13 op/s 2024-05-03T18:28:31.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:28:30 smithi150 ceph-mon[32492]: pgmap v42: 129 pgs: 129 active+clean; 89 MiB data, 600 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 3.2 MiB/s wr, 13 op/s 2024-05-03T18:28:33.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:28:32 smithi033 ceph-mon[28330]: pgmap v43: 129 pgs: 129 active+clean; 89 MiB data, 600 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 3.2 MiB/s wr, 13 op/s 2024-05-03T18:28:33.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:28:32 smithi099 ceph-mon[31658]: pgmap v43: 129 pgs: 129 active+clean; 89 MiB data, 600 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 3.2 MiB/s wr, 13 op/s 2024-05-03T18:28:33.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:28:32 smithi150 ceph-mon[32492]: pgmap v43: 129 pgs: 129 active+clean; 89 MiB data, 600 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 3.2 MiB/s wr, 13 op/s 2024-05-03T18:28:34.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:28:33 smithi033 ceph-mon[28330]: pgmap v44: 129 pgs: 129 active+clean; 112 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 7.6 MiB/s wr, 19 op/s 2024-05-03T18:28:34.284 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:28:33 smithi150 ceph-mon[32492]: pgmap v44: 129 pgs: 129 active+clean; 112 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 7.6 MiB/s wr, 19 op/s 2024-05-03T18:28:34.297 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:28:33 smithi099 ceph-mon[31658]: pgmap v44: 129 pgs: 129 active+clean; 112 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 7.6 MiB/s wr, 19 op/s 2024-05-03T18:28:35.032 INFO:teuthology.orchestra.run.smithi033.stderr:300000 of 3200000 tuples (9%) done (elapsed 12.14 s, remaining 117.33 s) 2024-05-03T18:28:37.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:28:36 smithi150 ceph-mon[32492]: pgmap v45: 129 pgs: 129 active+clean; 112 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 7.5 MiB/s wr, 9 op/s 2024-05-03T18:28:37.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:28:36 smithi033 ceph-mon[28330]: pgmap v45: 129 pgs: 129 active+clean; 112 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 7.5 MiB/s wr, 9 op/s 2024-05-03T18:28:37.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:28:36 smithi099 ceph-mon[31658]: pgmap v45: 129 pgs: 129 active+clean; 112 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 7.5 MiB/s wr, 9 op/s 2024-05-03T18:28:38.972 INFO:teuthology.orchestra.run.smithi033.stderr:400000 of 3200000 tuples (12%) done (elapsed 16.08 s, remaining 112.55 s) 2024-05-03T18:28:39.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:28:38 smithi150 ceph-mon[32492]: pgmap v46: 129 pgs: 129 active+clean; 140 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 12 MiB/s wr, 14 op/s 2024-05-03T18:28:39.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:28:38 smithi033 ceph-mon[28330]: pgmap v46: 129 pgs: 129 active+clean; 140 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 12 MiB/s wr, 14 op/s 2024-05-03T18:28:39.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:28:38 smithi099 ceph-mon[31658]: pgmap v46: 129 pgs: 129 active+clean; 140 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 12 MiB/s wr, 14 op/s 2024-05-03T18:28:41.074 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-03T18:28:41.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:28:40 smithi150 ceph-mon[32492]: pgmap v47: 129 pgs: 129 active+clean; 140 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.6 MiB/s wr, 9 op/s 2024-05-03T18:28:41.164 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:28:40 smithi033 ceph-mon[28330]: pgmap v47: 129 pgs: 129 active+clean; 140 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.6 MiB/s wr, 9 op/s 2024-05-03T18:28:41.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:28:40 smithi099 ceph-mon[31658]: pgmap v47: 129 pgs: 129 active+clean; 140 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.6 MiB/s wr, 9 op/s 2024-05-03T18:28:41.597 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:28:42.987 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:28:42 smithi033 ceph-mon[28330]: pgmap v48: 129 pgs: 129 active+clean; 140 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.6 MiB/s wr, 9 op/s 2024-05-03T18:28:43.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:28:42 smithi099 ceph-mon[31658]: pgmap v48: 129 pgs: 129 active+clean; 140 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.6 MiB/s wr, 9 op/s 2024-05-03T18:28:43.336 INFO:teuthology.orchestra.run.smithi033.stderr:500000 of 3200000 tuples (15%) done (elapsed 20.44 s, remaining 110.39 s) 2024-05-03T18:28:43.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:28:42 smithi150 ceph-mon[32492]: pgmap v48: 129 pgs: 129 active+clean; 140 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.6 MiB/s wr, 9 op/s 2024-05-03T18:28:44.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.065+0000 7fa10f000700 1 -- 172.21.15.33:0/1436405088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa10806b180 msgr2=0x7fa10806b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:28:44.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.065+0000 7fa10f000700 1 --2- 172.21.15.33:0/1436405088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa10806b180 0x7fa10806b600 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fa0f4009990 tx=0x7fa0f4009ca0 comp rx=0 tx=0).stop 2024-05-03T18:28:44.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.065+0000 7fa10f000700 1 -- 172.21.15.33:0/1436405088 shutdown_connections 2024-05-03T18:28:44.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.065+0000 7fa10f000700 1 --2- 172.21.15.33:0/1436405088 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa10806bb40 0x7fa108110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:44.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.065+0000 7fa10f000700 1 --2- 172.21.15.33:0/1436405088 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa108109510 0x7fa1081098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:44.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.065+0000 7fa10f000700 1 --2- 172.21.15.33:0/1436405088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa10806b180 0x7fa10806b600 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:44.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.065+0000 7fa10f000700 1 -- 172.21.15.33:0/1436405088 >> 172.21.15.33:0/1436405088 conn(0x7fa108079350 msgr2=0x7fa108079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:28:44.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.065+0000 7fa10f000700 1 -- 172.21.15.33:0/1436405088 shutdown_connections 2024-05-03T18:28:44.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.065+0000 7fa10f000700 1 -- 172.21.15.33:0/1436405088 wait complete. 2024-05-03T18:28:44.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.067+0000 7fa10f000700 1 Processor -- start 2024-05-03T18:28:44.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.067+0000 7fa10f000700 1 -- start start 2024-05-03T18:28:44.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.067+0000 7fa10f000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa10806b180 0x7fa10819bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:28:44.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.067+0000 7fa10f000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa10806bb40 0x7fa10819c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:28:44.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.067+0000 7fa10f000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa108109510 0x7fa1081a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:28:44.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.067+0000 7fa10f000700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa108113540 con 0x7fa108109510 2024-05-03T18:28:44.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.067+0000 7fa10f000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa1081133c0 con 0x7fa10806b180 2024-05-03T18:28:44.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.067+0000 7fa10f000700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa1081136c0 con 0x7fa10806bb40 2024-05-03T18:28:44.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.068+0000 7fa10d000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa108109510 0x7fa1081a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:28:44.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.068+0000 7fa10d000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa108109510 0x7fa1081a05e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38318/0 (socket says 172.21.15.33:38318) 2024-05-03T18:28:44.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.068+0000 7fa10d000700 1 -- 172.21.15.33:0/19684947 learned_addr learned my addr 172.21.15.33:0/19684947 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:28:44.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.068+0000 7fa107e00700 1 --2- 172.21.15.33:0/19684947 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa10806b180 0x7fa10819bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:28:44.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.068+0000 7fa10d000700 1 -- 172.21.15.33:0/19684947 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa10806bb40 msgr2=0x7fa10819c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:28:44.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.068+0000 7fa10d000700 1 --2- 172.21.15.33:0/19684947 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa10806bb40 0x7fa10819c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:44.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.068+0000 7fa10d000700 1 -- 172.21.15.33:0/19684947 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa10806b180 msgr2=0x7fa10819bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:28:44.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.068+0000 7fa10d000700 1 --2- 172.21.15.33:0/19684947 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa10806b180 0x7fa10819bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:44.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.069+0000 7fa10d000700 1 -- 172.21.15.33:0/19684947 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1081a0d20 con 0x7fa108109510 2024-05-03T18:28:44.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.069+0000 7fa10d000700 1 --2- 172.21.15.33:0/19684947 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa108109510 0x7fa1081a05e0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fa10000d980 tx=0x7fa10000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:28:44.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.069+0000 7fa104c00700 1 -- 172.21.15.33:0/19684947 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa100017070 con 0x7fa108109510 2024-05-03T18:28:44.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.069+0000 7fa10f000700 1 -- 172.21.15.33:0/19684947 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1081a1010 con 0x7fa108109510 2024-05-03T18:28:44.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.070+0000 7fa10f000700 1 -- 172.21.15.33:0/19684947 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1081a8890 con 0x7fa108109510 2024-05-03T18:28:44.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.070+0000 7fa104c00700 1 -- 172.21.15.33:0/19684947 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa100009dd0 con 0x7fa108109510 2024-05-03T18:28:44.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.070+0000 7fa104c00700 1 -- 172.21.15.33:0/19684947 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa100005280 con 0x7fa108109510 2024-05-03T18:28:44.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.076+0000 7fa104c00700 1 -- 172.21.15.33:0/19684947 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fa100009940 con 0x7fa108109510 2024-05-03T18:28:44.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.077+0000 7fa119911b80 1 Processor -- start 2024-05-03T18:28:44.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.077+0000 7fa119911b80 1 -- start start 2024-05-03T18:28:44.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.078+0000 7fa119911b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5601e928c940 0x5601e928cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:28:44.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.078+0000 7fa119911b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601e928d260 0x5601e9286f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:28:44.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.078+0000 7fa119911b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5601e9287560 0x5601e92879c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:28:44.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.078+0000 7fa119911b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5601e91c8a20 con 0x5601e928d260 2024-05-03T18:28:44.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.078+0000 7fa119911b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5601e9166760 con 0x5601e928c940 2024-05-03T18:28:44.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.078+0000 7fa119911b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5601e91c49e0 con 0x5601e9287560 2024-05-03T18:28:44.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.079+0000 7fa107400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5601e928c940 0x5601e928cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:28:44.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.079+0000 7fa104c00700 1 --2- 172.21.15.33:0/19684947 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fa0ec0b2a70 0x7fa0ec0b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:28:44.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.079+0000 7fa107400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5601e928c940 0x5601e928cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:34322/0 (socket says 172.21.15.33:34322) 2024-05-03T18:28:44.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.079+0000 7fa107400700 1 -- 172.21.15.33:0/3256017836 learned_addr learned my addr 172.21.15.33:0/3256017836 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:28:44.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.079+0000 7fa107400700 1 --2- 172.21.15.33:0/3256017836 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5601e9287560 0x5601e92879c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:28:44.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.079+0000 7fa104c00700 1 -- 172.21.15.33:0/19684947 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7fa1000e2870 con 0x7fa108109510 2024-05-03T18:28:44.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.080+0000 7fa10d000700 1 --2- 172.21.15.33:0/19684947 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fa0ec0b2a70 0x7fa0ec0b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:28:44.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.080+0000 7fa107400700 1 -- 172.21.15.33:0/3256017836 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5601e9287560 msgr2=0x5601e92879c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:28:44.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.080+0000 7fa107400700 1 --2- 172.21.15.33:0/3256017836 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5601e9287560 0x5601e92879c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:44.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.080+0000 7fa107400700 1 -- 172.21.15.33:0/3256017836 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601e928d260 msgr2=0x5601e9286f40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-03T18:28:44.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.080+0000 7fa107400700 1 --2- 172.21.15.33:0/3256017836 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601e928d260 0x5601e9286f40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:44.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.080+0000 7fa107400700 1 -- 172.21.15.33:0/3256017836 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601e91158d0 con 0x5601e928c940 2024-05-03T18:28:44.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.081+0000 7fa107400700 1 --2- 172.21.15.33:0/3256017836 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5601e928c940 0x5601e928cd20 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa0f4006010 tx=0x7fa0f4005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69f506570327cf server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:28:44.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.082+0000 7fa0fd600700 1 -- 172.21.15.33:0/3256017836 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f4020040 con 0x5601e928c940 2024-05-03T18:28:44.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.082+0000 7fa0fd600700 1 -- 172.21.15.33:0/3256017836 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa0f4005500 con 0x5601e928c940 2024-05-03T18:28:44.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.082+0000 7fa0fd600700 1 -- 172.21.15.33:0/3256017836 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f4003ab0 con 0x5601e928c940 2024-05-03T18:28:44.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.083+0000 7fa119911b80 1 -- 172.21.15.33:0/3256017836 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5601e928c940 msgr2=0x5601e928cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:28:44.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.083+0000 7fa119911b80 1 --2- 172.21.15.33:0/3256017836 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5601e928c940 0x5601e928cd20 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa0f4006010 tx=0x7fa0f4005c00 comp rx=0 tx=0).stop 2024-05-03T18:28:44.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.083+0000 7fa10d000700 1 --2- 172.21.15.33:0/19684947 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fa0ec0b2a70 0x7fa0ec0b4f30 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa100004e30 tx=0x7fa1000a56f0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:28:44.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.083+0000 7fa119911b80 1 -- 172.21.15.33:0/3256017836 shutdown_connections 2024-05-03T18:28:44.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.083+0000 7fa119911b80 1 --2- 172.21.15.33:0/3256017836 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5601e9287560 0x5601e92879c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:44.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.083+0000 7fa119911b80 1 --2- 172.21.15.33:0/3256017836 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5601e928c940 0x5601e928cd20 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:44.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.083+0000 7fa119911b80 1 --2- 172.21.15.33:0/3256017836 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601e928d260 0x5601e9286f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:44.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.083+0000 7fa119911b80 1 -- 172.21.15.33:0/3256017836 >> 172.21.15.33:0/3256017836 conn(0x5601e91ee6d0 msgr2=0x5601e9270900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:28:44.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.083+0000 7fa119911b80 1 -- 172.21.15.33:0/3256017836 shutdown_connections 2024-05-03T18:28:44.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.083+0000 7fa119911b80 1 -- 172.21.15.33:0/3256017836 wait complete. 2024-05-03T18:28:44.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.084+0000 7fa119911b80 1 -- start start 2024-05-03T18:28:44.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.084+0000 7fa119911b80 1 Processor -- start 2024-05-03T18:28:44.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.085+0000 7fa119911b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601e9287560 0x5601e9292b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:28:44.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.085+0000 7fa119911b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5601e928c940 0x5601e9293060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:28:44.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.085+0000 7fa119911b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5601e928d260 0x5601e9290080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:28:44.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.085+0000 7fa119911b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5601e9293d70 con 0x5601e9287560 2024-05-03T18:28:44.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.085+0000 7fa119911b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5601e912eb20 con 0x5601e928d260 2024-05-03T18:28:44.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.085+0000 7fa119911b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5601e9293ef0 con 0x5601e928c940 2024-05-03T18:28:44.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.085+0000 7fa119911b80 10 client.0 _ll_register_callbacks cb 0x5601e916fc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:28:44.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.086+0000 7fa107400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601e9287560 0x5601e9292b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:28:44.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.086+0000 7fa107400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601e9287560 0x5601e9292b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38322/0 (socket says 172.21.15.33:38322) 2024-05-03T18:28:44.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.086+0000 7fa107400700 1 -- 172.21.15.33:0/1619884719 learned_addr learned my addr 172.21.15.33:0/1619884719 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:28:44.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.086+0000 7fa107e00700 1 --2- 172.21.15.33:0/1619884719 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5601e928c940 0x5601e9293060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:28:44.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.086+0000 7fa107400700 1 -- 172.21.15.33:0/1619884719 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5601e928c940 msgr2=0x5601e9293060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:28:44.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.086+0000 7fa107400700 1 --2- 172.21.15.33:0/1619884719 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5601e928c940 0x5601e9293060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:44.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.086+0000 7fa107400700 1 -- 172.21.15.33:0/1619884719 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5601e928d260 msgr2=0x5601e9290080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:28:44.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.086+0000 7fa107400700 1 --2- 172.21.15.33:0/1619884719 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5601e928d260 0x5601e9290080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:44.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.086+0000 7fa107400700 1 -- 172.21.15.33:0/1619884719 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601e918e420 con 0x5601e9287560 2024-05-03T18:28:44.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.087+0000 7fa107400700 1 --2- 172.21.15.33:0/1619884719 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601e9287560 0x5601e9292b20 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fa0f4005d10 tx=0x7fa0f4005a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41ad2957983703c2 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:28:44.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.087+0000 7fa0f2000700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:28:44.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.087+0000 7fa0f2000700 1 -- 172.21.15.33:0/1619884719 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f4003ab0 con 0x5601e9287560 2024-05-03T18:28:44.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.087+0000 7fa0f2000700 1 -- 172.21.15.33:0/1619884719 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7fa0f4005070 con 0x5601e9287560 2024-05-03T18:28:44.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.087+0000 7fa119911b80 1 -- 172.21.15.33:0/1619884719 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0f4005500 con 0x5601e9287560 2024-05-03T18:28:44.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.087+0000 7fa0f2000700 1 -- 172.21.15.33:0/1619884719 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fa0f403a050 con 0x5601e9287560 2024-05-03T18:28:44.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.088+0000 7fa119911b80 10 client.15456 fetch_fsmap learned FSMap version 21 2024-05-03T18:28:44.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.088+0000 7fa119911b80 1 -- 172.21.15.33:0/1619884719 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5601e9293d70 con 0x5601e9287560 2024-05-03T18:28:44.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.088+0000 7fa0f2000700 1 -- 172.21.15.33:0/1619884719 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa0f4005500 con 0x5601e9287560 2024-05-03T18:28:44.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.088+0000 7fa0f2000700 1 -- 172.21.15.33:0/1619884719 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fa0f4027430 con 0x5601e9287560 2024-05-03T18:28:44.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.088+0000 7fa119911b80 10 client.15456 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:28:44.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.088+0000 7fa119911b80 10 client.15456 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:28:44.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.088+0000 7fa119911b80 20 client.15456 populate_metadata read hostname 'smithi033' 2024-05-03T18:28:44.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.088+0000 7fa119911b80 1 --2- 172.21.15.33:0/1619884719 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5601e92a8e80 0x5601e92ab270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:28:44.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.088+0000 7fa119911b80 4 client.15456 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:28:44.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.088+0000 7fa119911b80 1 -- 172.21.15.33:0/1619884719 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5601e9242ff0 con 0x5601e92a8e80 2024-05-03T18:28:44.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.089+0000 7fa107e00700 1 --2- 172.21.15.33:0/1619884719 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5601e92a8e80 0x5601e92ab270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:28:44.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.090+0000 7fa107e00700 1 --2- 172.21.15.33:0/1619884719 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5601e92a8e80 0x5601e92ab270 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=978bdd5260b19857 server_cookie=cae4440774cee146 in_seq=0 out_seq=0 2024-05-03T18:28:44.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.090+0000 7fa0f2000700 10 client.15456 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:28:44.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.092+0000 7fa0f2000700 1 -- 172.21.15.33:0/1619884719 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5601e9242ff0 con 0x5601e92a8e80 2024-05-03T18:28:44.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.092+0000 7fa0f2000700 10 client.15456 handle_command_reply: tid=0 2024-05-03T18:28:44.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.092+0000 7fa119911b80 1 client.15456 shutdown 2024-05-03T18:28:44.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.093+0000 7fa119911b80 1 -- 172.21.15.33:0/1619884719 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601e9287560 msgr2=0x5601e9292b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:28:44.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.093+0000 7fa119911b80 1 --2- 172.21.15.33:0/1619884719 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601e9287560 0x5601e9292b20 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fa0f4005d10 tx=0x7fa0f4005a90 comp rx=0 tx=0).stop 2024-05-03T18:28:44.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.093+0000 7fa119911b80 1 -- 172.21.15.33:0/1619884719 shutdown_connections 2024-05-03T18:28:44.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.093+0000 7fa119911b80 1 --2- 172.21.15.33:0/1619884719 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5601e92a8e80 0x5601e92ab270 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:44.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.093+0000 7fa119911b80 1 --2- 172.21.15.33:0/1619884719 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5601e928d260 0x5601e9290080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:44.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.093+0000 7fa119911b80 1 --2- 172.21.15.33:0/1619884719 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601e9287560 0x5601e9292b20 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:44.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.093+0000 7fa119911b80 1 --2- 172.21.15.33:0/1619884719 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5601e928c940 0x5601e9293060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:44.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.093+0000 7fa119911b80 1 -- 172.21.15.33:0/1619884719 >> 172.21.15.33:0/1619884719 conn(0x5601e91ee6d0 msgr2=0x5601e926b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:28:44.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.093+0000 7fa0f2000700 0 client.15456 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:28:44.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.093+0000 7fa119911b80 1 -- 172.21.15.33:0/1619884719 shutdown_connections 2024-05-03T18:28:44.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.093+0000 7fa119911b80 1 -- 172.21.15.33:0/1619884719 wait complete. 2024-05-03T18:28:44.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.093+0000 7fa119911b80 20 client.15456 trim_cache size 0 max 16384 2024-05-03T18:28:44.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.119+0000 7fa119911b80 1 Processor -- start 2024-05-03T18:28:44.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.119+0000 7fa119911b80 1 -- start start 2024-05-03T18:28:44.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.120+0000 7fa119911b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5601e9287560 0x5601e928d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:28:44.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.120+0000 7fa119911b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5601e928c940 0x5601e928db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:28:44.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.120+0000 7fa119911b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601e928d260 0x5601e928e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:28:44.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.120+0000 7fa119911b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e800cd90 con 0x5601e928d260 2024-05-03T18:28:44.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.120+0000 7fa119911b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e800f610 con 0x5601e928c940 2024-05-03T18:28:44.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.120+0000 7fa119911b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e800d400 con 0x5601e9287560 2024-05-03T18:28:44.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.120+0000 7fa107400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601e928d260 0x5601e928e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:28:44.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.120+0000 7fa107400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601e928d260 0x5601e928e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38330/0 (socket says 172.21.15.33:38330) 2024-05-03T18:28:44.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.120+0000 7fa107400700 1 -- 172.21.15.33:0/3237536079 learned_addr learned my addr 172.21.15.33:0/3237536079 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:28:44.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.120+0000 7fa107e00700 1 --2- 172.21.15.33:0/3237536079 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5601e928c940 0x5601e928db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:28:44.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.121+0000 7fa107400700 1 --2- 172.21.15.33:0/3237536079 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5601e9287560 0x5601e928d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:28:44.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.121+0000 7fa107400700 1 -- 172.21.15.33:0/3237536079 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5601e9287560 msgr2=0x5601e928d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:28:44.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.121+0000 7fa107400700 1 --2- 172.21.15.33:0/3237536079 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5601e9287560 0x5601e928d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:44.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.121+0000 7fa107400700 1 -- 172.21.15.33:0/3237536079 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5601e928c940 msgr2=0x5601e928db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:28:44.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.121+0000 7fa107400700 1 --2- 172.21.15.33:0/3237536079 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5601e928c940 0x5601e928db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:44.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.121+0000 7fa107400700 1 -- 172.21.15.33:0/3237536079 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601e928e5f0 con 0x5601e928d260 2024-05-03T18:28:44.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.121+0000 7fa107400700 1 --2- 172.21.15.33:0/3237536079 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601e928d260 0x5601e928e970 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fa0f4005a60 tx=0x7fa0f402c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bae888fc85539b66 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:28:44.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.122+0000 7fa0fcc00700 1 -- 172.21.15.33:0/3237536079 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f4027e50 con 0x5601e928d260 2024-05-03T18:28:44.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.122+0000 7fa0fcc00700 1 -- 172.21.15.33:0/3237536079 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa0f400b5c0 con 0x5601e928d260 2024-05-03T18:28:44.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.122+0000 7fa0fcc00700 1 -- 172.21.15.33:0/3237536079 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f400f760 con 0x5601e928d260 2024-05-03T18:28:44.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.123+0000 7fa119911b80 1 -- 172.21.15.33:0/3237536079 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601e928d260 msgr2=0x5601e928e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:28:44.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.123+0000 7fa119911b80 1 --2- 172.21.15.33:0/3237536079 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601e928d260 0x5601e928e970 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fa0f4005a60 tx=0x7fa0f402c040 comp rx=0 tx=0).stop 2024-05-03T18:28:44.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.123+0000 7fa119911b80 1 -- 172.21.15.33:0/3237536079 shutdown_connections 2024-05-03T18:28:44.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.123+0000 7fa119911b80 1 --2- 172.21.15.33:0/3237536079 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601e928d260 0x5601e928e970 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:44.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.123+0000 7fa119911b80 1 --2- 172.21.15.33:0/3237536079 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5601e9287560 0x5601e928d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:44.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.123+0000 7fa119911b80 1 --2- 172.21.15.33:0/3237536079 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5601e928c940 0x5601e928db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:44.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.123+0000 7fa119911b80 1 -- 172.21.15.33:0/3237536079 >> 172.21.15.33:0/3237536079 conn(0x5601e91ee6d0 msgr2=0x5601e9292b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:28:44.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.123+0000 7fa119911b80 1 -- 172.21.15.33:0/3237536079 shutdown_connections 2024-05-03T18:28:44.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.123+0000 7fa119911b80 1 -- 172.21.15.33:0/3237536079 wait complete. 2024-05-03T18:28:44.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.124+0000 7fa119911b80 1 -- start start 2024-05-03T18:28:44.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.124+0000 7fa119911b80 1 Processor -- start 2024-05-03T18:28:44.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.125+0000 7fa119911b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601e9287560 0x5601e92a1370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:28:44.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.125+0000 7fa119911b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5601e928c940 0x5601e92a18b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:28:44.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.125+0000 7fa119911b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5601e928d260 0x5601e9312510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:28:44.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.125+0000 7fa119911b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e800dad0 con 0x5601e9287560 2024-05-03T18:28:44.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.125+0000 7fa119911b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e800ef40 con 0x5601e928d260 2024-05-03T18:28:44.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.125+0000 7fa119911b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e800e1a0 con 0x5601e928c940 2024-05-03T18:28:44.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.125+0000 7fa119911b80 10 client.0 _ll_register_callbacks cb 0x5601e916fc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:28:44.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.125+0000 7fa107400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601e9287560 0x5601e92a1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:28:44.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.125+0000 7fa107400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601e9287560 0x5601e92a1370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38340/0 (socket says 172.21.15.33:38340) 2024-05-03T18:28:44.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.125+0000 7fa107400700 1 -- 172.21.15.33:0/1244743597 learned_addr learned my addr 172.21.15.33:0/1244743597 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:28:44.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.125+0000 7fa107e00700 1 --2- 172.21.15.33:0/1244743597 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5601e928c940 0x5601e92a18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:28:44.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.125+0000 7fa107400700 1 --2- 172.21.15.33:0/1244743597 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5601e928d260 0x5601e9312510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:28:44.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.126+0000 7fa107400700 1 -- 172.21.15.33:0/1244743597 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5601e928c940 msgr2=0x5601e92a18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:28:44.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.126+0000 7fa107400700 1 --2- 172.21.15.33:0/1244743597 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5601e928c940 0x5601e92a18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:44.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.126+0000 7fa107400700 1 -- 172.21.15.33:0/1244743597 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5601e928d260 msgr2=0x5601e9312510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:28:44.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.126+0000 7fa107400700 1 --2- 172.21.15.33:0/1244743597 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5601e928d260 0x5601e9312510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:44.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.126+0000 7fa107400700 1 -- 172.21.15.33:0/1244743597 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601e9312a50 con 0x5601e9287560 2024-05-03T18:28:44.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.126+0000 7fa107400700 1 --2- 172.21.15.33:0/1244743597 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601e9287560 0x5601e92a1370 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fa0f400bfa0 tx=0x7fa0f4009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b74555b574e97025 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:28:44.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.126+0000 7fa0f2000700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:28:44.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.126+0000 7fa0f2000700 1 -- 172.21.15.33:0/1244743597 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f4045020 con 0x5601e9287560 2024-05-03T18:28:44.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.127+0000 7fa0f2000700 1 -- 172.21.15.33:0/1244743597 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7fa0f40050d0 con 0x5601e9287560 2024-05-03T18:28:44.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.127+0000 7fa119911b80 1 -- 172.21.15.33:0/1244743597 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0f400b5c0 con 0x5601e9287560 2024-05-03T18:28:44.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.127+0000 7fa0f2000700 1 -- 172.21.15.33:0/1244743597 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fa0f4027630 con 0x5601e9287560 2024-05-03T18:28:44.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.127+0000 7fa119911b80 10 client.15462 fetch_fsmap learned FSMap version 21 2024-05-03T18:28:44.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.127+0000 7fa119911b80 1 -- 172.21.15.33:0/1244743597 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5601e9313bc0 con 0x5601e9287560 2024-05-03T18:28:44.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.128+0000 7fa0f2000700 1 -- 172.21.15.33:0/1244743597 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa0f400b5c0 con 0x5601e9287560 2024-05-03T18:28:44.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.128+0000 7fa0f2000700 1 -- 172.21.15.33:0/1244743597 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fa0f400fb90 con 0x5601e9287560 2024-05-03T18:28:44.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.128+0000 7fa119911b80 10 client.15462 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:28:44.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.128+0000 7fa119911b80 10 client.15462 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:28:44.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.128+0000 7fa119911b80 20 client.15462 populate_metadata read hostname 'smithi033' 2024-05-03T18:28:44.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.128+0000 7fa119911b80 1 --2- 172.21.15.33:0/1244743597 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5601e9313f60 0x5601e9316380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:28:44.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.128+0000 7fa119911b80 4 client.15462 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-03T18:28:44.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.128+0000 7fa119911b80 1 -- 172.21.15.33:0/1244743597 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5601e91d6400 con 0x5601e9313f60 2024-05-03T18:28:44.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.129+0000 7fa107e00700 1 --2- 172.21.15.33:0/1244743597 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5601e9313f60 0x5601e9316380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:28:44.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.130+0000 7fa107e00700 1 --2- 172.21.15.33:0/1244743597 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5601e9313f60 0x5601e9316380 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=505cc616b4301f08 server_cookie=390c104129cef540 in_seq=0 out_seq=0 2024-05-03T18:28:44.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.130+0000 7fa0f2000700 10 client.15462 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:28:44.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.130+0000 7fa0f2000700 1 -- 172.21.15.33:0/1244743597 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5601e91d6400 con 0x5601e9313f60 2024-05-03T18:28:44.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.130+0000 7fa0f2000700 10 client.15462 handle_command_reply: tid=0 2024-05-03T18:28:44.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.130+0000 7fa119911b80 1 client.15462 shutdown 2024-05-03T18:28:44.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.131+0000 7fa119911b80 1 -- 172.21.15.33:0/1244743597 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601e9287560 msgr2=0x5601e92a1370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:28:44.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.131+0000 7fa119911b80 1 --2- 172.21.15.33:0/1244743597 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601e9287560 0x5601e92a1370 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fa0f400bfa0 tx=0x7fa0f4009f80 comp rx=0 tx=0).stop 2024-05-03T18:28:44.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.131+0000 7fa119911b80 1 -- 172.21.15.33:0/1244743597 shutdown_connections 2024-05-03T18:28:44.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.131+0000 7fa119911b80 1 --2- 172.21.15.33:0/1244743597 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5601e9313f60 0x5601e9316380 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:44.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.131+0000 7fa119911b80 1 --2- 172.21.15.33:0/1244743597 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5601e928d260 0x5601e9312510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:44.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.131+0000 7fa119911b80 1 --2- 172.21.15.33:0/1244743597 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5601e9287560 0x5601e92a1370 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:44.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.131+0000 7fa119911b80 1 --2- 172.21.15.33:0/1244743597 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5601e928c940 0x5601e92a18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:44.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.131+0000 7fa119911b80 1 -- 172.21.15.33:0/1244743597 >> 172.21.15.33:0/1244743597 conn(0x5601e91ee6d0 msgr2=0x5601e928d640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:28:44.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.131+0000 7fa0f2000700 0 client.15462 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:28:44.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.132+0000 7fa119911b80 1 -- 172.21.15.33:0/1244743597 shutdown_connections 2024-05-03T18:28:44.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.132+0000 7fa119911b80 1 -- 172.21.15.33:0/1244743597 wait complete. 2024-05-03T18:28:44.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.132+0000 7fa119911b80 20 client.15462 trim_cache size 0 max 16384 2024-05-03T18:28:44.134 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-03T18:28:44.134 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-05-03T18:28:44.134 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-05-03T18:28:44.134 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-03T18:28:44.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.133+0000 7fa10f000700 1 -- 172.21.15.33:0/19684947 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fa0ec0b2a70 msgr2=0x7fa0ec0b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:28:44.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.133+0000 7fa10f000700 1 --2- 172.21.15.33:0/19684947 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fa0ec0b2a70 0x7fa0ec0b4f30 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa100004e30 tx=0x7fa1000a56f0 comp rx=0 tx=0).stop 2024-05-03T18:28:44.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.133+0000 7fa10f000700 1 -- 172.21.15.33:0/19684947 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa108109510 msgr2=0x7fa1081a05e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:28:44.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.133+0000 7fa10f000700 1 --2- 172.21.15.33:0/19684947 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa108109510 0x7fa1081a05e0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fa10000d980 tx=0x7fa10000dd40 comp rx=0 tx=0).stop 2024-05-03T18:28:44.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.133+0000 7fa10f000700 1 -- 172.21.15.33:0/19684947 shutdown_connections 2024-05-03T18:28:44.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.133+0000 7fa10f000700 1 --2- 172.21.15.33:0/19684947 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa108109510 0x7fa1081a05e0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:44.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.133+0000 7fa10f000700 1 --2- 172.21.15.33:0/19684947 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fa0ec0b2a70 0x7fa0ec0b4f30 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:44.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.133+0000 7fa10f000700 1 --2- 172.21.15.33:0/19684947 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa10806b180 0x7fa10819bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:44.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.133+0000 7fa10f000700 1 --2- 172.21.15.33:0/19684947 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa10806bb40 0x7fa10819c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:44.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.133+0000 7fa10f000700 1 -- 172.21.15.33:0/19684947 >> 172.21.15.33:0/19684947 conn(0x7fa108079350 msgr2=0x7fa10810e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:28:44.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.134+0000 7fa10f000700 1 -- 172.21.15.33:0/19684947 shutdown_connections 2024-05-03T18:28:44.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:44.134+0000 7fa10f000700 1 -- 172.21.15.33:0/19684947 wait complete. 2024-05-03T18:28:45.021 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-03T18:28:45.021 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T18:28:45.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:28:44 smithi033 ceph-mon[28330]: pgmap v49: 129 pgs: 129 active+clean; 169 MiB data, 775 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 13 MiB/s wr, 16 op/s 2024-05-03T18:28:45.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:28:44 smithi033 ceph-mon[28330]: mgrmap e25: x(active, since 92s), standbys: z, y 2024-05-03T18:28:45.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:28:44 smithi099 ceph-mon[31658]: pgmap v49: 129 pgs: 129 active+clean; 169 MiB data, 775 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 13 MiB/s wr, 16 op/s 2024-05-03T18:28:45.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:28:44 smithi099 ceph-mon[31658]: mgrmap e25: x(active, since 92s), standbys: z, y 2024-05-03T18:28:45.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:28:44 smithi150 ceph-mon[32492]: pgmap v49: 129 pgs: 129 active+clean; 169 MiB data, 775 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 13 MiB/s wr, 16 op/s 2024-05-03T18:28:45.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:28:44 smithi150 ceph-mon[32492]: mgrmap e25: x(active, since 92s), standbys: z, y 2024-05-03T18:28:45.519 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:28:45.705 INFO:teuthology.orchestra.run.smithi033.stderr:600000 of 3200000 tuples (18%) done (elapsed 22.81 s, remaining 98.85 s) 2024-05-03T18:28:46.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:28:46 smithi033 ceph-mon[28330]: pgmap v50: 129 pgs: 129 active+clean; 169 MiB data, 775 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.8 MiB/s wr, 11 op/s 2024-05-03T18:28:47.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:28:46 smithi099 ceph-mon[31658]: pgmap v50: 129 pgs: 129 active+clean; 169 MiB data, 775 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.8 MiB/s wr, 11 op/s 2024-05-03T18:28:47.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:28:46 smithi150 ceph-mon[32492]: pgmap v50: 129 pgs: 129 active+clean; 169 MiB data, 775 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.8 MiB/s wr, 11 op/s 2024-05-03T18:28:48.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:48.004+0000 7f7e4e200700 1 -- 172.21.15.33:0/3911991230 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e480756d0 msgr2=0x7f7e48075ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:28:48.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:48.004+0000 7f7e4e200700 1 --2- 172.21.15.33:0/3911991230 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e480756d0 0x7f7e48075ab0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f7e38009990 tx=0x7f7e38009ca0 comp rx=0 tx=0).stop 2024-05-03T18:28:48.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:48.004+0000 7f7e4e200700 1 -- 172.21.15.33:0/3911991230 shutdown_connections 2024-05-03T18:28:48.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:48.004+0000 7f7e4e200700 1 --2- 172.21.15.33:0/3911991230 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e4810fea0 0x7f7e48112290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:48.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:48.004+0000 7f7e4e200700 1 --2- 172.21.15.33:0/3911991230 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e480756d0 0x7f7e48075ab0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:48.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:48.004+0000 7f7e4e200700 1 --2- 172.21.15.33:0/3911991230 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7e48075ff0 0x7f7e4810f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:48.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:48.004+0000 7f7e4e200700 1 -- 172.21.15.33:0/3911991230 >> 172.21.15.33:0/3911991230 conn(0x7f7e4807a650 msgr2=0x7f7e4807aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:28:48.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:48.005+0000 7f7e4e200700 1 -- 172.21.15.33:0/3911991230 shutdown_connections 2024-05-03T18:28:48.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:48.005+0000 7f7e4e200700 1 -- 172.21.15.33:0/3911991230 wait complete. 2024-05-03T18:28:48.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:48.006+0000 7f7e4e200700 1 Processor -- start 2024-05-03T18:28:48.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:48.006+0000 7f7e4e200700 1 -- start start 2024-05-03T18:28:48.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:48.006+0000 7f7e4e200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e480756d0 0x7f7e4819bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:28:48.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:48.006+0000 7f7e4e200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7e48075ff0 0x7f7e4819c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:28:48.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:48.007+0000 7f7e4e200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e4810fea0 0x7f7e481a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:28:48.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:48.007+0000 7f7e4e200700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7e48115820 con 0x7f7e4810fea0 2024-05-03T18:28:48.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:48.007+0000 7f7e4e200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7e481156a0 con 0x7f7e480756d0 2024-05-03T18:28:48.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:48.007+0000 7f7e4e200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f7e481159a0 con 0x7f7e48075ff0 2024-05-03T18:28:48.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:48.007+0000 7f7e47e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e4810fea0 0x7f7e481a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:28:48.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:48.007+0000 7f7e47e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e4810fea0 0x7f7e481a0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44928/0 (socket says 172.21.15.33:44928) 2024-05-03T18:28:48.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:48.007+0000 7f7e47e00700 1 -- 172.21.15.33:0/1505451468 learned_addr learned my addr 172.21.15.33:0/1505451468 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:28:48.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:48.007+0000 7f7e47400700 1 --2- 172.21.15.33:0/1505451468 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e480756d0 0x7f7e4819bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:28:48.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:48.007+0000 7f7e46a00700 1 --2- 172.21.15.33:0/1505451468 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7e48075ff0 0x7f7e4819c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:28:48.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:48.008+0000 7f7e47e00700 1 -- 172.21.15.33:0/1505451468 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7e48075ff0 msgr2=0x7f7e4819c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:28:48.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:48.008+0000 7f7e47e00700 1 --2- 172.21.15.33:0/1505451468 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7e48075ff0 0x7f7e4819c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:48.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:48.008+0000 7f7e47e00700 1 -- 172.21.15.33:0/1505451468 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e480756d0 msgr2=0x7f7e4819bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:28:48.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:48.008+0000 7f7e47e00700 1 --2- 172.21.15.33:0/1505451468 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e480756d0 0x7f7e4819bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:48.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:48.008+0000 7f7e47e00700 1 -- 172.21.15.33:0/1505451468 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e481a0dc0 con 0x7f7e4810fea0 2024-05-03T18:28:48.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:48.008+0000 7f7e46a00700 1 --2- 172.21.15.33:0/1505451468 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7e48075ff0 0x7f7e4819c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:28:48.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:48.008+0000 7f7e47400700 1 --2- 172.21.15.33:0/1505451468 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e480756d0 0x7f7e4819bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:28:48.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:48.008+0000 7f7e47e00700 1 --2- 172.21.15.33:0/1505451468 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e4810fea0 0x7f7e481a0680 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f7e40007f10 tx=0x7f7e4000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:28:48.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:48.008+0000 7f7e3fe00700 1 -- 172.21.15.33:0/1505451468 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e4000ddd0 con 0x7f7e4810fea0 2024-05-03T18:28:48.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:48.008+0000 7f7e4e200700 1 -- 172.21.15.33:0/1505451468 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e481a10b0 con 0x7f7e4810fea0 2024-05-03T18:28:48.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:48.009+0000 7f7e3fe00700 1 -- 172.21.15.33:0/1505451468 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7e40009e20 con 0x7f7e4810fea0 2024-05-03T18:28:48.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:48.009+0000 7f7e4e200700 1 -- 172.21.15.33:0/1505451468 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e481a8930 con 0x7f7e4810fea0 2024-05-03T18:28:48.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:48.009+0000 7f7e3fe00700 1 -- 172.21.15.33:0/1505451468 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e40005a40 con 0x7f7e4810fea0 2024-05-03T18:28:48.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:48.011+0000 7f7e4e200700 1 -- 172.21.15.33:0/1505451468 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e4806a190 con 0x7f7e4810fea0 2024-05-03T18:28:48.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:48.013+0000 7f7e3fe00700 1 -- 172.21.15.33:0/1505451468 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f7e40005000 con 0x7f7e4810fea0 2024-05-03T18:28:48.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:48.014+0000 7f7e3fe00700 1 --2- 172.21.15.33:0/1505451468 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f7e2c0b29f0 0x7f7e2c0b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:28:48.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:48.015+0000 7f7e3fe00700 1 -- 172.21.15.33:0/1505451468 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f7e400e24d0 con 0x7f7e4810fea0 2024-05-03T18:28:48.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:48.015+0000 7f7e47400700 1 --2- 172.21.15.33:0/1505451468 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f7e2c0b29f0 0x7f7e2c0b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:28:48.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:48.016+0000 7f7e47400700 1 --2- 172.21.15.33:0/1505451468 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f7e2c0b29f0 0x7f7e2c0b4eb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7e3800b5c0 tx=0x7f7e3801a040 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:28:48.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:48.020+0000 7f7e3fe00700 1 -- 172.21.15.33:0/1505451468 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f7e40010070 con 0x7f7e4810fea0 2024-05-03T18:28:48.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:48.360+0000 7f7e4e200700 1 -- 172.21.15.33:0/1505451468 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7e4802e510 con 0x7f7e4810fea0 2024-05-03T18:28:48.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:48.362+0000 7f7e3fe00700 1 -- 172.21.15.33:0/1505451468 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11801 (secure 0 0 0) 0x7f7e400a9a20 con 0x7f7e4810fea0 2024-05-03T18:28:48.363 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:28:48.363 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1327393503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1327393503},{"type":"v1","addr":"172.21.15.33:6835","nonce":1327393503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/2461472948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":2461472948},{"type":"v1","addr":"172.21.15.33:6837","nonce":2461472948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14559,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/442269281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":442269281},{"type":"v1","addr":"172.21.15.33:6839","nonce":442269281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14571,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/645571756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":645571756},{"type":"v1","addr":"172.21.15.33:6841","nonce":645571756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/203598909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":203598909},{"type":"v1","addr":"172.21.15.150:6833","nonce":203598909}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/4029842493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":4029842493},{"type":"v1","addr":"172.21.15.150:6835","nonce":4029842493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3463664423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3463664423},{"type":"v1","addr":"172.21.15.150:6837","nonce":3463664423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24481,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6839/4051648018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":4051648018},{"type":"v1","addr":"172.21.15.150:6839","nonce":4051648018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34418,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1865105479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1865105479},{"type":"v1","addr":"172.21.15.99:6833","nonce":1865105479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34445,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/1206393914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1206393914},{"type":"v1","addr":"172.21.15.99:6835","nonce":1206393914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-03T18:22:28.837091+0000","modified":"2024-05-03T18:23:49.758398+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34508},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34472":{"gid":34472,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6837/3360970342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3360970342},{"type":"v1","addr":"172.21.15.99:6837","nonce":3360970342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34508":{"gid":34508,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/919907308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":919907308},{"type":"v1","addr":"172.21.15.99:6839","nonce":919907308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34508,"qdb_cluster":[34508]},"id":1}]} 2024-05-03T18:28:48.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:48.365+0000 7f7e4e200700 1 -- 172.21.15.33:0/1505451468 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f7e2c0b29f0 msgr2=0x7f7e2c0b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:28:48.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:48.365+0000 7f7e4e200700 1 --2- 172.21.15.33:0/1505451468 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f7e2c0b29f0 0x7f7e2c0b4eb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7e3800b5c0 tx=0x7f7e3801a040 comp rx=0 tx=0).stop 2024-05-03T18:28:48.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:48.365+0000 7f7e4e200700 1 -- 172.21.15.33:0/1505451468 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e4810fea0 msgr2=0x7f7e481a0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:28:48.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:48.365+0000 7f7e4e200700 1 --2- 172.21.15.33:0/1505451468 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e4810fea0 0x7f7e481a0680 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f7e40007f10 tx=0x7f7e4000d3b0 comp rx=0 tx=0).stop 2024-05-03T18:28:48.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:48.366+0000 7f7e4e200700 1 -- 172.21.15.33:0/1505451468 shutdown_connections 2024-05-03T18:28:48.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:48.366+0000 7f7e4e200700 1 --2- 172.21.15.33:0/1505451468 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e4810fea0 0x7f7e481a0680 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:48.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:48.366+0000 7f7e4e200700 1 --2- 172.21.15.33:0/1505451468 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f7e2c0b29f0 0x7f7e2c0b4eb0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:48.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:48.366+0000 7f7e4e200700 1 --2- 172.21.15.33:0/1505451468 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e480756d0 0x7f7e4819bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:48.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:48.366+0000 7f7e4e200700 1 --2- 172.21.15.33:0/1505451468 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7e48075ff0 0x7f7e4819c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:48.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:48.366+0000 7f7e4e200700 1 -- 172.21.15.33:0/1505451468 >> 172.21.15.33:0/1505451468 conn(0x7f7e4807a650 msgr2=0x7f7e48100750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:28:48.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:48.366+0000 7f7e4e200700 1 -- 172.21.15.33:0/1505451468 shutdown_connections 2024-05-03T18:28:48.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:48.366+0000 7f7e4e200700 1 -- 172.21.15.33:0/1505451468 wait complete. 2024-05-03T18:28:48.368 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 21 2024-05-03T18:28:49.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:28:48 smithi033 ceph-mon[28330]: pgmap v51: 129 pgs: 129 active+clean; 206 MiB data, 847 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 15 MiB/s wr, 16 op/s 2024-05-03T18:28:49.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:28:48 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/1505451468' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:28:49.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:28:48 smithi099 ceph-mon[31658]: pgmap v51: 129 pgs: 129 active+clean; 206 MiB data, 847 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 15 MiB/s wr, 16 op/s 2024-05-03T18:28:49.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:28:48 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/1505451468' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:28:49.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:28:48 smithi150 ceph-mon[32492]: pgmap v51: 129 pgs: 129 active+clean; 206 MiB data, 847 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 15 MiB/s wr, 16 op/s 2024-05-03T18:28:49.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:28:48 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/1505451468' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:28:49.471 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-03T18:28:49.654 INFO:teuthology.orchestra.run.smithi033.stderr:700000 of 3200000 tuples (21%) done (elapsed 26.76 s, remaining 95.57 s) 2024-05-03T18:28:49.987 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:28:51.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:28:50 smithi033 ceph-mon[28330]: pgmap v52: 129 pgs: 129 active+clean; 206 MiB data, 847 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 11 op/s 2024-05-03T18:28:51.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:28:50 smithi099 ceph-mon[31658]: pgmap v52: 129 pgs: 129 active+clean; 206 MiB data, 847 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 11 op/s 2024-05-03T18:28:51.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:28:50 smithi150 ceph-mon[32492]: pgmap v52: 129 pgs: 129 active+clean; 206 MiB data, 847 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 11 op/s 2024-05-03T18:28:52.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.424+0000 7f7726400700 1 -- 172.21.15.33:0/819019322 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f772006b180 msgr2=0x7f772006b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:28:52.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.424+0000 7f7726400700 1 --2- 172.21.15.33:0/819019322 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f772006b180 0x7f772006b600 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f770c009990 tx=0x7f770c009ca0 comp rx=0 tx=0).stop 2024-05-03T18:28:52.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.424+0000 7f7726400700 1 -- 172.21.15.33:0/819019322 shutdown_connections 2024-05-03T18:28:52.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.424+0000 7f7726400700 1 --2- 172.21.15.33:0/819019322 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f772006bb40 0x7f7720110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:52.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.424+0000 7f7726400700 1 --2- 172.21.15.33:0/819019322 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7720109510 0x7f77201098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:52.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.424+0000 7f7726400700 1 --2- 172.21.15.33:0/819019322 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f772006b180 0x7f772006b600 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:52.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.424+0000 7f7726400700 1 -- 172.21.15.33:0/819019322 >> 172.21.15.33:0/819019322 conn(0x7f7720079350 msgr2=0x7f7720079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:28:52.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.424+0000 7f7726400700 1 -- 172.21.15.33:0/819019322 shutdown_connections 2024-05-03T18:28:52.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.424+0000 7f7726400700 1 -- 172.21.15.33:0/819019322 wait complete. 2024-05-03T18:28:52.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.426+0000 7f7726400700 1 Processor -- start 2024-05-03T18:28:52.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.426+0000 7f7726400700 1 -- start start 2024-05-03T18:28:52.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.426+0000 7f7726400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f772006b180 0x7f772019bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:28:52.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.426+0000 7f7726400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f772006bb40 0x7f772019c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:28:52.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.426+0000 7f7726400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7720109510 0x7f77201a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:28:52.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.426+0000 7f7726400700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f77201134f0 con 0x7f7720109510 2024-05-03T18:28:52.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.426+0000 7f7726400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7720113370 con 0x7f772006bb40 2024-05-03T18:28:52.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.426+0000 7f7726400700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f7720113670 con 0x7f772006b180 2024-05-03T18:28:52.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.427+0000 7f771fe00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7720109510 0x7f77201a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:28:52.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.427+0000 7f771f400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f772006b180 0x7f772019bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:28:52.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.427+0000 7f771fe00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7720109510 0x7f77201a0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44942/0 (socket says 172.21.15.33:44942) 2024-05-03T18:28:52.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.427+0000 7f771fe00700 1 -- 172.21.15.33:0/4092547757 learned_addr learned my addr 172.21.15.33:0/4092547757 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:28:52.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.427+0000 7f771f400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f772006b180 0x7f772019bd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:58086/0 (socket says 172.21.15.33:58086) 2024-05-03T18:28:52.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.427+0000 7f771ea00700 1 --2- 172.21.15.33:0/4092547757 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f772006bb40 0x7f772019c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:28:52.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.428+0000 7f771fe00700 1 -- 172.21.15.33:0/4092547757 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f772006b180 msgr2=0x7f772019bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:28:52.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.428+0000 7f771fe00700 1 --2- 172.21.15.33:0/4092547757 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f772006b180 0x7f772019bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:52.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.428+0000 7f771fe00700 1 -- 172.21.15.33:0/4092547757 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f772006bb40 msgr2=0x7f772019c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:28:52.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.428+0000 7f771fe00700 1 --2- 172.21.15.33:0/4092547757 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f772006bb40 0x7f772019c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:52.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.428+0000 7f771fe00700 1 -- 172.21.15.33:0/4092547757 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77201a0d80 con 0x7f7720109510 2024-05-03T18:28:52.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.428+0000 7f771fe00700 1 --2- 172.21.15.33:0/4092547757 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7720109510 0x7f77201a0640 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f7718007ee0 tx=0x7f771800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:28:52.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.429+0000 7f7717e00700 1 -- 172.21.15.33:0/4092547757 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f771800dd20 con 0x7f7720109510 2024-05-03T18:28:52.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.429+0000 7f7726400700 1 -- 172.21.15.33:0/4092547757 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77201a1070 con 0x7f7720109510 2024-05-03T18:28:52.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.429+0000 7f7726400700 1 -- 172.21.15.33:0/4092547757 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77201a88f0 con 0x7f7720109510 2024-05-03T18:28:52.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.430+0000 7f7717e00700 1 -- 172.21.15.33:0/4092547757 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7718004020 con 0x7f7720109510 2024-05-03T18:28:52.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.430+0000 7f7717e00700 1 -- 172.21.15.33:0/4092547757 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7718015640 con 0x7f7720109510 2024-05-03T18:28:52.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.434+0000 7f7717e00700 1 -- 172.21.15.33:0/4092547757 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f77180157e0 con 0x7f7720109510 2024-05-03T18:28:52.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.435+0000 7f7730d2cb80 1 Processor -- start 2024-05-03T18:28:52.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.435+0000 7f7730d2cb80 1 -- start start 2024-05-03T18:28:52.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.436+0000 7f7730d2cb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5576135927b0 0x557613592b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:28:52.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.436+0000 7f7730d2cb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5576135930d0 0x55761358dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:28:52.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.436+0000 7f7730d2cb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55761358e5d0 0x55761358ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:28:52.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.436+0000 7f7730d2cb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5576134e0a20 con 0x5576135927b0 2024-05-03T18:28:52.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.436+0000 7f7730d2cb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55761347e760 con 0x5576135930d0 2024-05-03T18:28:52.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.436+0000 7f7730d2cb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5576134dc9e0 con 0x55761358e5d0 2024-05-03T18:28:52.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.436+0000 7f771ea00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5576135927b0 0x557613592b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:28:52.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.436+0000 7f771ea00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5576135927b0 0x557613592b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44946/0 (socket says 172.21.15.33:44946) 2024-05-03T18:28:52.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.436+0000 7f771ea00700 1 -- 172.21.15.33:0/1152448487 learned_addr learned my addr 172.21.15.33:0/1152448487 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:28:52.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.437+0000 7f771f400700 1 --2- 172.21.15.33:0/1152448487 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5576135930d0 0x55761358dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:28:52.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.437+0000 7f771ea00700 1 --2- 172.21.15.33:0/1152448487 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55761358e5d0 0x55761358ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:28:52.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.437+0000 7f7717e00700 1 --2- 172.21.15.33:0/4092547757 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f77040b2a70 0x7f77040b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:28:52.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.437+0000 7f771ea00700 1 -- 172.21.15.33:0/1152448487 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55761358e5d0 msgr2=0x55761358ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:28:52.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.437+0000 7f771ea00700 1 --2- 172.21.15.33:0/1152448487 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55761358e5d0 0x55761358ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:52.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.437+0000 7f771ea00700 1 -- 172.21.15.33:0/1152448487 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5576135930d0 msgr2=0x55761358dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:28:52.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.437+0000 7f771ea00700 1 --2- 172.21.15.33:0/1152448487 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5576135930d0 0x55761358dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:52.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.437+0000 7f771ea00700 1 -- 172.21.15.33:0/1152448487 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55761342d8d0 con 0x5576135927b0 2024-05-03T18:28:52.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.437+0000 7f7717e00700 1 -- 172.21.15.33:0/4092547757 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f77180e2400 con 0x7f7720109510 2024-05-03T18:28:52.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.437+0000 7f771fe00700 1 --2- 172.21.15.33:0/4092547757 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f77040b2a70 0x7f77040b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:28:52.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.437+0000 7f771ea00700 1 --2- 172.21.15.33:0/1152448487 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5576135927b0 0x557613592b90 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f770c000c00 tx=0x7f770c005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34b7c3ae46419551 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:28:52.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.438+0000 7f7714c00700 1 -- 172.21.15.33:0/1152448487 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f770c021040 con 0x5576135927b0 2024-05-03T18:28:52.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.438+0000 7f7714c00700 1 -- 172.21.15.33:0/1152448487 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f770c003d10 con 0x5576135927b0 2024-05-03T18:28:52.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.438+0000 7f7714c00700 1 -- 172.21.15.33:0/1152448487 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f770c00f460 con 0x5576135927b0 2024-05-03T18:28:52.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.439+0000 7f7730d2cb80 1 -- 172.21.15.33:0/1152448487 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5576135927b0 msgr2=0x557613592b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:28:52.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.439+0000 7f771fe00700 1 --2- 172.21.15.33:0/4092547757 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f77040b2a70 0x7f77040b4f30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f77180df820 tx=0x7f771800dec0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:28:52.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.439+0000 7f7730d2cb80 1 --2- 172.21.15.33:0/1152448487 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5576135927b0 0x557613592b90 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f770c000c00 tx=0x7f770c005e80 comp rx=0 tx=0).stop 2024-05-03T18:28:52.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.439+0000 7f7730d2cb80 1 -- 172.21.15.33:0/1152448487 shutdown_connections 2024-05-03T18:28:52.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.439+0000 7f7730d2cb80 1 --2- 172.21.15.33:0/1152448487 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55761358e5d0 0x55761358ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:52.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.439+0000 7f7730d2cb80 1 --2- 172.21.15.33:0/1152448487 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5576135927b0 0x557613592b90 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:52.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.439+0000 7f7730d2cb80 1 --2- 172.21.15.33:0/1152448487 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5576135930d0 0x55761358dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:52.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.439+0000 7f7730d2cb80 1 -- 172.21.15.33:0/1152448487 >> 172.21.15.33:0/1152448487 conn(0x5576134bcb60 msgr2=0x5576135887f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:28:52.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.439+0000 7f7730d2cb80 1 -- 172.21.15.33:0/1152448487 shutdown_connections 2024-05-03T18:28:52.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.439+0000 7f7730d2cb80 1 -- 172.21.15.33:0/1152448487 wait complete. 2024-05-03T18:28:52.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.440+0000 7f7730d2cb80 1 -- start start 2024-05-03T18:28:52.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.440+0000 7f7730d2cb80 1 Processor -- start 2024-05-03T18:28:52.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.441+0000 7f7730d2cb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55761358e5d0 0x557613599dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:28:52.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.441+0000 7f7730d2cb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5576135927b0 0x557613597160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:28:52.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.441+0000 7f7730d2cb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5576135930d0 0x5576135976a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:28:52.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.441+0000 7f7730d2cb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55761359ac90 con 0x55761358e5d0 2024-05-03T18:28:52.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.441+0000 7f7730d2cb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x557613446b20 con 0x5576135930d0 2024-05-03T18:28:52.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.441+0000 7f7730d2cb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55761359ae10 con 0x5576135927b0 2024-05-03T18:28:52.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.441+0000 7f7730d2cb80 10 client.0 _ll_register_callbacks cb 0x557613487c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:28:52.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.441+0000 7f771ea00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55761358e5d0 0x557613599dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:28:52.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.441+0000 7f771ea00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55761358e5d0 0x557613599dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44948/0 (socket says 172.21.15.33:44948) 2024-05-03T18:28:52.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.441+0000 7f771ea00700 1 -- 172.21.15.33:0/3387028906 learned_addr learned my addr 172.21.15.33:0/3387028906 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:28:52.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.442+0000 7f771f400700 1 --2- 172.21.15.33:0/3387028906 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5576135927b0 0x557613597160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:28:52.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.442+0000 7f771ea00700 1 -- 172.21.15.33:0/3387028906 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5576135927b0 msgr2=0x557613597160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:28:52.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.442+0000 7f771ea00700 1 --2- 172.21.15.33:0/3387028906 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5576135927b0 0x557613597160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:52.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.442+0000 7f771ea00700 1 -- 172.21.15.33:0/3387028906 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5576135930d0 msgr2=0x5576135976a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:28:52.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.442+0000 7f771ea00700 1 --2- 172.21.15.33:0/3387028906 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5576135930d0 0x5576135976a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:52.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.442+0000 7f771ea00700 1 -- 172.21.15.33:0/3387028906 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576134a6420 con 0x55761358e5d0 2024-05-03T18:28:52.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.442+0000 7f771f400700 1 --2- 172.21.15.33:0/3387028906 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5576135927b0 0x557613597160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:28:52.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.446+0000 7f771ea00700 1 --2- 172.21.15.33:0/3387028906 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55761358e5d0 0x557613599dc0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f770c009350 tx=0x7f770c004f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fdc599f37905201 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:28:52.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.446+0000 7f7709600700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:28:52.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.446+0000 7f7709600700 1 -- 172.21.15.33:0/3387028906 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f770c003880 con 0x55761358e5d0 2024-05-03T18:28:52.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.446+0000 7f7709600700 1 -- 172.21.15.33:0/3387028906 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f770c003a20 con 0x55761358e5d0 2024-05-03T18:28:52.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.446+0000 7f7730d2cb80 1 -- 172.21.15.33:0/3387028906 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f770c003d10 con 0x55761358e5d0 2024-05-03T18:28:52.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.447+0000 7f7709600700 1 -- 172.21.15.33:0/3387028906 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f770c033470 con 0x55761358e5d0 2024-05-03T18:28:52.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.447+0000 7f7730d2cb80 10 client.15480 fetch_fsmap learned FSMap version 21 2024-05-03T18:28:52.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.447+0000 7f7730d2cb80 1 -- 172.21.15.33:0/3387028906 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5576135a5420 con 0x55761358e5d0 2024-05-03T18:28:52.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.447+0000 7f7709600700 1 -- 172.21.15.33:0/3387028906 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f770c003d10 con 0x55761358e5d0 2024-05-03T18:28:52.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.447+0000 7f7709600700 1 -- 172.21.15.33:0/3387028906 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f770c028580 con 0x55761358e5d0 2024-05-03T18:28:52.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.448+0000 7f7730d2cb80 10 client.15480 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:28:52.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.448+0000 7f7730d2cb80 10 client.15480 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:28:52.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.448+0000 7f7730d2cb80 20 client.15480 populate_metadata read hostname 'smithi033' 2024-05-03T18:28:52.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.448+0000 7f7730d2cb80 1 --2- 172.21.15.33:0/3387028906 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5576135a78f0 0x5576135a9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:28:52.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.448+0000 7f7730d2cb80 4 client.15480 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:28:52.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.448+0000 7f7730d2cb80 1 -- 172.21.15.33:0/3387028906 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55761355b1d0 con 0x5576135a78f0 2024-05-03T18:28:52.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.449+0000 7f771f400700 1 --2- 172.21.15.33:0/3387028906 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5576135a78f0 0x5576135a9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:28:52.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.450+0000 7f771f400700 1 --2- 172.21.15.33:0/3387028906 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5576135a78f0 0x5576135a9ce0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e7fa4e502483050 server_cookie=7cdedc4c23289bf2 in_seq=0 out_seq=0 2024-05-03T18:28:52.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.450+0000 7f7709600700 10 client.15480 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:28:52.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.452+0000 7f7709600700 1 -- 172.21.15.33:0/3387028906 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55761355b1d0 con 0x5576135a78f0 2024-05-03T18:28:52.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.452+0000 7f7709600700 10 client.15480 handle_command_reply: tid=0 2024-05-03T18:28:52.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.452+0000 7f7730d2cb80 1 client.15480 shutdown 2024-05-03T18:28:52.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.453+0000 7f7730d2cb80 1 -- 172.21.15.33:0/3387028906 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55761358e5d0 msgr2=0x557613599dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:28:52.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.453+0000 7f7730d2cb80 1 --2- 172.21.15.33:0/3387028906 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55761358e5d0 0x557613599dc0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f770c009350 tx=0x7f770c004f80 comp rx=0 tx=0).stop 2024-05-03T18:28:52.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.453+0000 7f7730d2cb80 1 -- 172.21.15.33:0/3387028906 shutdown_connections 2024-05-03T18:28:52.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.453+0000 7f7730d2cb80 1 --2- 172.21.15.33:0/3387028906 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5576135a78f0 0x5576135a9ce0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:52.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.453+0000 7f7730d2cb80 1 --2- 172.21.15.33:0/3387028906 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5576135930d0 0x5576135976a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:52.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.453+0000 7f7709600700 0 client.15480 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:28:52.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.453+0000 7f7730d2cb80 1 --2- 172.21.15.33:0/3387028906 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55761358e5d0 0x557613599dc0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:52.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.453+0000 7f7730d2cb80 1 --2- 172.21.15.33:0/3387028906 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5576135927b0 0x557613597160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:52.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.453+0000 7f7730d2cb80 1 -- 172.21.15.33:0/3387028906 >> 172.21.15.33:0/3387028906 conn(0x5576134bcb60 msgr2=0x557613587650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:28:52.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.453+0000 7f7730d2cb80 1 -- 172.21.15.33:0/3387028906 shutdown_connections 2024-05-03T18:28:52.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.453+0000 7f7730d2cb80 1 -- 172.21.15.33:0/3387028906 wait complete. 2024-05-03T18:28:52.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.453+0000 7f7730d2cb80 20 client.15480 trim_cache size 0 max 16384 2024-05-03T18:28:52.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.473+0000 7f7730d2cb80 1 Processor -- start 2024-05-03T18:28:52.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.473+0000 7f7730d2cb80 1 -- start start 2024-05-03T18:28:52.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.473+0000 7f7730d2cb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55761358e5d0 0x557613596a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:28:52.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.473+0000 7f7730d2cb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5576135927b0 0x557613587d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:28:52.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.473+0000 7f7730d2cb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5576135930d0 0x5576135acd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:28:52.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.473+0000 7f7730d2cb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f770000ccb0 con 0x55761358e5d0 2024-05-03T18:28:52.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.473+0000 7f7730d2cb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f770000f530 con 0x5576135927b0 2024-05-03T18:28:52.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.473+0000 7f7730d2cb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f770000d320 con 0x5576135930d0 2024-05-03T18:28:52.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.473+0000 7f771ea00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55761358e5d0 0x557613596a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:28:52.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.474+0000 7f771ea00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55761358e5d0 0x557613596a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44958/0 (socket says 172.21.15.33:44958) 2024-05-03T18:28:52.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.474+0000 7f771ea00700 1 -- 172.21.15.33:0/1674177440 learned_addr learned my addr 172.21.15.33:0/1674177440 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:28:52.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.474+0000 7f771f400700 1 --2- 172.21.15.33:0/1674177440 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5576135927b0 0x557613587d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:28:52.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.474+0000 7f771ea00700 1 -- 172.21.15.33:0/1674177440 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5576135930d0 msgr2=0x5576135acd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:28:52.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.474+0000 7f771ea00700 1 --2- 172.21.15.33:0/1674177440 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5576135930d0 0x5576135acd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:52.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.474+0000 7f771ea00700 1 -- 172.21.15.33:0/1674177440 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5576135927b0 msgr2=0x557613587d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:28:52.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.474+0000 7f771ea00700 1 --2- 172.21.15.33:0/1674177440 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5576135927b0 0x557613587d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:52.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.474+0000 7f771ea00700 1 -- 172.21.15.33:0/1674177440 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576135956d0 con 0x55761358e5d0 2024-05-03T18:28:52.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.474+0000 7f771ea00700 1 --2- 172.21.15.33:0/1674177440 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55761358e5d0 0x557613596a60 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f770c000c00 tx=0x7f770c0051d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6450705828ea03e server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:28:52.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.475+0000 7f770be00700 1 -- 172.21.15.33:0/1674177440 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f770c028b70 con 0x55761358e5d0 2024-05-03T18:28:52.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.475+0000 7f770be00700 1 -- 172.21.15.33:0/1674177440 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f770c02e070 con 0x55761358e5d0 2024-05-03T18:28:52.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.475+0000 7f770be00700 1 -- 172.21.15.33:0/1674177440 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f770c003e10 con 0x55761358e5d0 2024-05-03T18:28:52.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.475+0000 7f7730d2cb80 1 -- 172.21.15.33:0/1674177440 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55761358e5d0 msgr2=0x557613596a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:28:52.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.475+0000 7f7730d2cb80 1 --2- 172.21.15.33:0/1674177440 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55761358e5d0 0x557613596a60 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f770c000c00 tx=0x7f770c0051d0 comp rx=0 tx=0).stop 2024-05-03T18:28:52.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.475+0000 7f7730d2cb80 1 -- 172.21.15.33:0/1674177440 shutdown_connections 2024-05-03T18:28:52.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.475+0000 7f7730d2cb80 1 --2- 172.21.15.33:0/1674177440 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5576135930d0 0x5576135acd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:52.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.475+0000 7f7730d2cb80 1 --2- 172.21.15.33:0/1674177440 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55761358e5d0 0x557613596a60 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:52.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.475+0000 7f7730d2cb80 1 --2- 172.21.15.33:0/1674177440 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5576135927b0 0x557613587d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:52.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.475+0000 7f7730d2cb80 1 -- 172.21.15.33:0/1674177440 >> 172.21.15.33:0/1674177440 conn(0x5576134bcb60 msgr2=0x557613587650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:28:52.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.476+0000 7f7730d2cb80 1 -- 172.21.15.33:0/1674177440 shutdown_connections 2024-05-03T18:28:52.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.476+0000 7f7730d2cb80 1 -- 172.21.15.33:0/1674177440 wait complete. 2024-05-03T18:28:52.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.476+0000 7f7730d2cb80 1 -- start start 2024-05-03T18:28:52.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.477+0000 7f7730d2cb80 1 Processor -- start 2024-05-03T18:28:52.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.477+0000 7f7730d2cb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55761358e5d0 0x5576135aaa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:28:52.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.477+0000 7f7730d2cb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5576135927b0 0x5576135aaf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:28:52.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.477+0000 7f7730d2cb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5576135930d0 0x5576135ab480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:28:52.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.477+0000 7f7730d2cb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f770000d9f0 con 0x5576135927b0 2024-05-03T18:28:52.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.477+0000 7f7730d2cb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f770000ee60 con 0x55761358e5d0 2024-05-03T18:28:52.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.477+0000 7f7730d2cb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f770000e0c0 con 0x5576135930d0 2024-05-03T18:28:52.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.477+0000 7f7730d2cb80 10 client.0 _ll_register_callbacks cb 0x557613487c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:28:52.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.477+0000 7f771f400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5576135927b0 0x5576135aaf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:28:52.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.478+0000 7f771f400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5576135927b0 0x5576135aaf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44962/0 (socket says 172.21.15.33:44962) 2024-05-03T18:28:52.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.478+0000 7f771f400700 1 -- 172.21.15.33:0/969900834 learned_addr learned my addr 172.21.15.33:0/969900834 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:28:52.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.478+0000 7f771ea00700 1 --2- 172.21.15.33:0/969900834 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55761358e5d0 0x5576135aaa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:28:52.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.478+0000 7f771f400700 1 -- 172.21.15.33:0/969900834 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5576135930d0 msgr2=0x5576135ab480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:28:52.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.478+0000 7f771f400700 1 --2- 172.21.15.33:0/969900834 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5576135930d0 0x5576135ab480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:52.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.478+0000 7f771f400700 1 -- 172.21.15.33:0/969900834 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55761358e5d0 msgr2=0x5576135aaa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:28:52.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.478+0000 7f771f400700 1 --2- 172.21.15.33:0/969900834 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55761358e5d0 0x5576135aaa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:52.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.478+0000 7f771f400700 1 -- 172.21.15.33:0/969900834 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557613619790 con 0x5576135927b0 2024-05-03T18:28:52.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.479+0000 7f771f400700 1 --2- 172.21.15.33:0/969900834 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5576135927b0 0x5576135aaf40 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x55761359ae40 tx=0x7f7710009af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8bfaaaafbc0a7b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:28:52.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.479+0000 7f7709600700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:28:52.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.479+0000 7f7709600700 1 -- 172.21.15.33:0/969900834 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f771001d940 con 0x5576135927b0 2024-05-03T18:28:52.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.479+0000 7f7730d2cb80 1 -- 172.21.15.33:0/969900834 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f770c02e070 con 0x5576135927b0 2024-05-03T18:28:52.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.479+0000 7f7709600700 1 -- 172.21.15.33:0/969900834 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f7710004830 con 0x5576135927b0 2024-05-03T18:28:52.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.480+0000 7f7709600700 1 -- 172.21.15.33:0/969900834 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f771002a050 con 0x5576135927b0 2024-05-03T18:28:52.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.480+0000 7f7730d2cb80 10 client.15486 fetch_fsmap learned FSMap version 21 2024-05-03T18:28:52.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.480+0000 7f7730d2cb80 1 -- 172.21.15.33:0/969900834 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55761361ab30 con 0x5576135927b0 2024-05-03T18:28:52.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.480+0000 7f7709600700 1 -- 172.21.15.33:0/969900834 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f770c02e070 con 0x5576135927b0 2024-05-03T18:28:52.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.480+0000 7f7709600700 1 -- 172.21.15.33:0/969900834 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f77100106c0 con 0x5576135927b0 2024-05-03T18:28:52.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.480+0000 7f7730d2cb80 10 client.15486 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:28:52.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.481+0000 7f7730d2cb80 10 client.15486 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:28:52.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.481+0000 7f7730d2cb80 20 client.15486 populate_metadata read hostname 'smithi033' 2024-05-03T18:28:52.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.481+0000 7f7730d2cb80 1 --2- 172.21.15.33:0/969900834 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55761361b270 0x55761361d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:28:52.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.481+0000 7f7730d2cb80 4 client.15486 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-03T18:28:52.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.481+0000 7f7730d2cb80 1 -- 172.21.15.33:0/969900834 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5576134ee400 con 0x55761361b270 2024-05-03T18:28:52.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.481+0000 7f771ea00700 1 --2- 172.21.15.33:0/969900834 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55761361b270 0x55761361d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:28:52.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.482+0000 7f771ea00700 1 --2- 172.21.15.33:0/969900834 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55761361b270 0x55761361d690 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c12b2724ebb2c0 server_cookie=624e3e7b7498ada2 in_seq=0 out_seq=0 2024-05-03T18:28:52.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.482+0000 7f7709600700 10 client.15486 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:28:52.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.483+0000 7f7709600700 1 -- 172.21.15.33:0/969900834 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5576134ee400 con 0x55761361b270 2024-05-03T18:28:52.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.483+0000 7f7709600700 10 client.15486 handle_command_reply: tid=0 2024-05-03T18:28:52.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.483+0000 7f7730d2cb80 1 client.15486 shutdown 2024-05-03T18:28:52.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.483+0000 7f7730d2cb80 1 -- 172.21.15.33:0/969900834 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5576135927b0 msgr2=0x5576135aaf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:28:52.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.483+0000 7f7730d2cb80 1 --2- 172.21.15.33:0/969900834 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5576135927b0 0x5576135aaf40 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x55761359ae40 tx=0x7f7710009af0 comp rx=0 tx=0).stop 2024-05-03T18:28:52.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.483+0000 7f7730d2cb80 1 -- 172.21.15.33:0/969900834 shutdown_connections 2024-05-03T18:28:52.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.483+0000 7f7730d2cb80 1 --2- 172.21.15.33:0/969900834 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55761361b270 0x55761361d690 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:52.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.483+0000 7f7730d2cb80 1 --2- 172.21.15.33:0/969900834 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5576135930d0 0x5576135ab480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:52.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.483+0000 7f7730d2cb80 1 --2- 172.21.15.33:0/969900834 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55761358e5d0 0x5576135aaa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:52.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.483+0000 7f7730d2cb80 1 --2- 172.21.15.33:0/969900834 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5576135927b0 0x5576135aaf40 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:52.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.483+0000 7f7730d2cb80 1 -- 172.21.15.33:0/969900834 >> 172.21.15.33:0/969900834 conn(0x5576134bcb60 msgr2=0x5576135a39c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:28:52.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.484+0000 7f7709600700 0 client.15486 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:28:52.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.484+0000 7f7730d2cb80 1 -- 172.21.15.33:0/969900834 shutdown_connections 2024-05-03T18:28:52.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.484+0000 7f7730d2cb80 1 -- 172.21.15.33:0/969900834 wait complete. 2024-05-03T18:28:52.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.484+0000 7f7730d2cb80 20 client.15486 trim_cache size 0 max 16384 2024-05-03T18:28:52.485 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-03T18:28:52.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.485+0000 7f7726400700 1 -- 172.21.15.33:0/4092547757 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f77040b2a70 msgr2=0x7f77040b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:28:52.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.485+0000 7f7726400700 1 --2- 172.21.15.33:0/4092547757 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f77040b2a70 0x7f77040b4f30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f77180df820 tx=0x7f771800dec0 comp rx=0 tx=0).stop 2024-05-03T18:28:52.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.485+0000 7f7726400700 1 -- 172.21.15.33:0/4092547757 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7720109510 msgr2=0x7f77201a0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:28:52.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.485+0000 7f7726400700 1 --2- 172.21.15.33:0/4092547757 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7720109510 0x7f77201a0640 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f7718007ee0 tx=0x7f771800d3b0 comp rx=0 tx=0).stop 2024-05-03T18:28:52.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.485+0000 7f7726400700 1 -- 172.21.15.33:0/4092547757 shutdown_connections 2024-05-03T18:28:52.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.485+0000 7f7726400700 1 --2- 172.21.15.33:0/4092547757 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7720109510 0x7f77201a0640 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:52.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.485+0000 7f7726400700 1 --2- 172.21.15.33:0/4092547757 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f772006b180 0x7f772019bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:52.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.485+0000 7f7726400700 1 --2- 172.21.15.33:0/4092547757 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f77040b2a70 0x7f77040b4f30 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:52.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.485+0000 7f7726400700 1 --2- 172.21.15.33:0/4092547757 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f772006bb40 0x7f772019c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:52.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.485+0000 7f7726400700 1 -- 172.21.15.33:0/4092547757 >> 172.21.15.33:0/4092547757 conn(0x7f7720079350 msgr2=0x7f772010e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:28:52.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.487+0000 7f7726400700 1 -- 172.21.15.33:0/4092547757 shutdown_connections 2024-05-03T18:28:52.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:52.487+0000 7f7726400700 1 -- 172.21.15.33:0/4092547757 wait complete. 2024-05-03T18:28:53.190 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:28:52 smithi033 ceph-mon[28330]: pgmap v53: 129 pgs: 129 active+clean; 206 MiB data, 847 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 11 op/s 2024-05-03T18:28:53.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:28:52 smithi099 ceph-mon[31658]: pgmap v53: 129 pgs: 129 active+clean; 206 MiB data, 847 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 11 op/s 2024-05-03T18:28:53.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:28:52 smithi150 ceph-mon[32492]: pgmap v53: 129 pgs: 129 active+clean; 206 MiB data, 847 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 11 op/s 2024-05-03T18:28:53.499 INFO:teuthology.orchestra.run.smithi033.stderr:800000 of 3200000 tuples (25%) done (elapsed 30.61 s, remaining 91.82 s) 2024-05-03T18:28:54.274 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-03T18:28:54.274 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-03T18:28:54.786 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:28:55.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:28:54 smithi033 ceph-mon[28330]: pgmap v54: 129 pgs: 129 active+clean; 242 MiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 17 MiB/s wr, 17 op/s 2024-05-03T18:28:55.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:28:54 smithi099 ceph-mon[31658]: pgmap v54: 129 pgs: 129 active+clean; 242 MiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 17 MiB/s wr, 17 op/s 2024-05-03T18:28:55.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:28:54 smithi150 ceph-mon[32492]: pgmap v54: 129 pgs: 129 active+clean; 242 MiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 17 MiB/s wr, 17 op/s 2024-05-03T18:28:56.685 INFO:teuthology.orchestra.run.smithi033.stderr:900000 of 3200000 tuples (28%) done (elapsed 33.79 s, remaining 86.35 s) 2024-05-03T18:28:57.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.154+0000 7fa06ba00700 1 -- 172.21.15.33:0/2961640856 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa06410ff50 msgr2=0x7fa064112340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:28:57.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.154+0000 7fa06ba00700 1 --2- 172.21.15.33:0/2961640856 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa06410ff50 0x7fa064112340 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fa05c00b0a0 tx=0x7fa05c00b3b0 comp rx=0 tx=0).stop 2024-05-03T18:28:57.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.154+0000 7fa06ba00700 1 -- 172.21.15.33:0/2961640856 shutdown_connections 2024-05-03T18:28:57.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.154+0000 7fa06ba00700 1 --2- 172.21.15.33:0/2961640856 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa06410ff50 0x7fa064112340 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:57.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.154+0000 7fa06ba00700 1 --2- 172.21.15.33:0/2961640856 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa06406b1b0 0x7fa06406b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:57.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.154+0000 7fa06ba00700 1 --2- 172.21.15.33:0/2961640856 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa06406bad0 0x7fa06410fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:57.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.154+0000 7fa06ba00700 1 -- 172.21.15.33:0/2961640856 >> 172.21.15.33:0/2961640856 conn(0x7fa064079350 msgr2=0x7fa064079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:28:57.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.154+0000 7fa06ba00700 1 -- 172.21.15.33:0/2961640856 shutdown_connections 2024-05-03T18:28:57.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.154+0000 7fa06ba00700 1 -- 172.21.15.33:0/2961640856 wait complete. 2024-05-03T18:28:57.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.155+0000 7fa06ba00700 1 Processor -- start 2024-05-03T18:28:57.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.156+0000 7fa06ba00700 1 -- start start 2024-05-03T18:28:57.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.156+0000 7fa06ba00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa06406b1b0 0x7fa0641a18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:28:57.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.156+0000 7fa06ba00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa06406bad0 0x7fa0641a1e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:28:57.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.156+0000 7fa06ba00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa06410ff50 0x7fa06419ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:28:57.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.156+0000 7fa06ba00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa064115770 con 0x7fa06406bad0 2024-05-03T18:28:57.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.156+0000 7fa06ba00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa0641155f0 con 0x7fa06410ff50 2024-05-03T18:28:57.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.156+0000 7fa06ba00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa0641158f0 con 0x7fa06406b1b0 2024-05-03T18:28:57.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.157+0000 7fa063e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa06406bad0 0x7fa0641a1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:28:57.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.157+0000 7fa063e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa06406bad0 0x7fa0641a1e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53564/0 (socket says 172.21.15.33:53564) 2024-05-03T18:28:57.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.157+0000 7fa063e00700 1 -- 172.21.15.33:0/3629724275 learned_addr learned my addr 172.21.15.33:0/3629724275 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:28:57.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.157+0000 7fa069000700 1 --2- 172.21.15.33:0/3629724275 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa06406b1b0 0x7fa0641a18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:28:57.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.157+0000 7fa069a00700 1 --2- 172.21.15.33:0/3629724275 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa06410ff50 0x7fa06419ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:28:57.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.157+0000 7fa063e00700 1 -- 172.21.15.33:0/3629724275 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa06406b1b0 msgr2=0x7fa0641a18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:28:57.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.157+0000 7fa063e00700 1 --2- 172.21.15.33:0/3629724275 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa06406b1b0 0x7fa0641a18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:57.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.157+0000 7fa063e00700 1 -- 172.21.15.33:0/3629724275 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa06410ff50 msgr2=0x7fa06419ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:28:57.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.157+0000 7fa063e00700 1 --2- 172.21.15.33:0/3629724275 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa06410ff50 0x7fa06419ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:57.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.157+0000 7fa063e00700 1 -- 172.21.15.33:0/3629724275 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa06419c1f0 con 0x7fa06406bad0 2024-05-03T18:28:57.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.158+0000 7fa063e00700 1 --2- 172.21.15.33:0/3629724275 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa06406bad0 0x7fa0641a1e10 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fa05000d950 tx=0x7fa05000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:28:57.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.158+0000 7fa061600700 1 -- 172.21.15.33:0/3629724275 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa050016070 con 0x7fa06406bad0 2024-05-03T18:28:57.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.158+0000 7fa061600700 1 -- 172.21.15.33:0/3629724275 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa050009dd0 con 0x7fa06406bad0 2024-05-03T18:28:57.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.158+0000 7fa06ba00700 1 -- 172.21.15.33:0/3629724275 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa06419c480 con 0x7fa06406bad0 2024-05-03T18:28:57.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.158+0000 7fa061600700 1 -- 172.21.15.33:0/3629724275 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa05000f3c0 con 0x7fa06406bad0 2024-05-03T18:28:57.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.159+0000 7fa06ba00700 1 -- 172.21.15.33:0/3629724275 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0641a87a0 con 0x7fa06406bad0 2024-05-03T18:28:57.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.163+0000 7fa061600700 1 -- 172.21.15.33:0/3629724275 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fa050019070 con 0x7fa06406bad0 2024-05-03T18:28:57.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.164+0000 7fa076430b80 1 Processor -- start 2024-05-03T18:28:57.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.164+0000 7fa076430b80 1 -- start start 2024-05-03T18:28:57.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.164+0000 7fa076430b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5562b84449b0 0x5562b8444d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:28:57.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.164+0000 7fa076430b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5562b84452d0 0x5562b843efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:28:57.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.164+0000 7fa076430b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5562b843f5d0 0x5562b843fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:28:57.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.164+0000 7fa076430b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5562b8391aa0 con 0x5562b843f5d0 2024-05-03T18:28:57.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.164+0000 7fa076430b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5562b832f7e0 con 0x5562b84449b0 2024-05-03T18:28:57.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.164+0000 7fa076430b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5562b838da60 con 0x5562b84452d0 2024-05-03T18:28:57.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.165+0000 7fa069a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5562b84449b0 0x5562b8444d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:28:57.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.165+0000 7fa069a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5562b84449b0 0x5562b8444d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:50152/0 (socket says 172.21.15.33:50152) 2024-05-03T18:28:57.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.165+0000 7fa069a00700 1 -- 172.21.15.33:0/145354894 learned_addr learned my addr 172.21.15.33:0/145354894 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:28:57.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.165+0000 7fa061600700 1 --2- 172.21.15.33:0/3629724275 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fa04c0b2a70 0x7fa04c0b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:28:57.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.166+0000 7fa069000700 1 --2- 172.21.15.33:0/145354894 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5562b84452d0 0x5562b843efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:28:57.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.166+0000 7fa061600700 1 -- 172.21.15.33:0/3629724275 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7fa0500e2530 con 0x7fa06406bad0 2024-05-03T18:28:57.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.166+0000 7fa069a00700 1 --2- 172.21.15.33:0/3629724275 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fa04c0b2a70 0x7fa04c0b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:28:57.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.166+0000 7fa063e00700 1 --2- 172.21.15.33:0/145354894 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5562b843f5d0 0x5562b843fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:28:57.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.167+0000 7fa069000700 1 -- 172.21.15.33:0/145354894 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5562b84449b0 msgr2=0x5562b8444d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:28:57.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.167+0000 7fa069000700 1 --2- 172.21.15.33:0/145354894 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5562b84449b0 0x5562b8444d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:57.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.167+0000 7fa069000700 1 -- 172.21.15.33:0/145354894 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5562b843f5d0 msgr2=0x5562b843fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:28:57.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.167+0000 7fa069000700 1 --2- 172.21.15.33:0/145354894 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5562b843f5d0 0x5562b843fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:57.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.167+0000 7fa069000700 1 -- 172.21.15.33:0/145354894 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562b82de950 con 0x5562b84452d0 2024-05-03T18:28:57.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.167+0000 7fa069a00700 1 --2- 172.21.15.33:0/145354894 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5562b84449b0 0x5562b8444d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:28:57.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.167+0000 7fa063e00700 1 --2- 172.21.15.33:0/145354894 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5562b843f5d0 0x5562b843fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:28:57.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.168+0000 7fa069000700 1 --2- 172.21.15.33:0/145354894 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5562b84452d0 0x5562b843efb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa05800bc40 tx=0x7fa05800bf50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3eb5e5a53518d5f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:28:57.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.168+0000 7fa069a00700 1 --2- 172.21.15.33:0/3629724275 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fa04c0b2a70 0x7fa04c0b4f30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa05c009e60 tx=0x7fa05c00b3b0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:28:57.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.169+0000 7fa056000700 1 -- 172.21.15.33:0/145354894 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa058015070 con 0x5562b84452d0 2024-05-03T18:28:57.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.169+0000 7fa056000700 1 -- 172.21.15.33:0/145354894 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa058004830 con 0x5562b84452d0 2024-05-03T18:28:57.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.169+0000 7fa056000700 1 -- 172.21.15.33:0/145354894 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa058005420 con 0x5562b84452d0 2024-05-03T18:28:57.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.170+0000 7fa076430b80 1 -- 172.21.15.33:0/145354894 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5562b84452d0 msgr2=0x5562b843efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:28:57.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.170+0000 7fa076430b80 1 --2- 172.21.15.33:0/145354894 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5562b84452d0 0x5562b843efb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa05800bc40 tx=0x7fa05800bf50 comp rx=0 tx=0).stop 2024-05-03T18:28:57.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.170+0000 7fa076430b80 1 -- 172.21.15.33:0/145354894 shutdown_connections 2024-05-03T18:28:57.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.170+0000 7fa076430b80 1 --2- 172.21.15.33:0/145354894 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5562b843f5d0 0x5562b843fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:57.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.170+0000 7fa076430b80 1 --2- 172.21.15.33:0/145354894 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5562b84449b0 0x5562b8444d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:57.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.170+0000 7fa076430b80 1 --2- 172.21.15.33:0/145354894 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5562b84452d0 0x5562b843efb0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:57.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.170+0000 7fa076430b80 1 -- 172.21.15.33:0/145354894 >> 172.21.15.33:0/145354894 conn(0x5562b83b7750 msgr2=0x5562b8439980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:28:57.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.170+0000 7fa076430b80 1 -- 172.21.15.33:0/145354894 shutdown_connections 2024-05-03T18:28:57.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.171+0000 7fa076430b80 1 -- 172.21.15.33:0/145354894 wait complete. 2024-05-03T18:28:57.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.171+0000 7fa076430b80 1 -- start start 2024-05-03T18:28:57.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.172+0000 7fa076430b80 1 Processor -- start 2024-05-03T18:28:57.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.173+0000 7fa076430b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5562b843f5d0 0x5562b844af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:28:57.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.173+0000 7fa076430b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5562b84449b0 0x5562b84482b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:28:57.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.173+0000 7fa076430b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5562b84452d0 0x5562b84487f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:28:57.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.173+0000 7fa076430b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5562b844bd80 con 0x5562b84449b0 2024-05-03T18:28:57.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.173+0000 7fa076430b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5562b82f7ba0 con 0x5562b84452d0 2024-05-03T18:28:57.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.174+0000 7fa076430b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5562b844bf00 con 0x5562b843f5d0 2024-05-03T18:28:57.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.174+0000 7fa069000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5562b843f5d0 0x5562b844af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:28:57.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.174+0000 7fa076430b80 10 client.0 _ll_register_callbacks cb 0x5562b8338cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:28:57.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.174+0000 7fa063e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5562b84449b0 0x5562b84482b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:28:57.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.174+0000 7fa063e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5562b84449b0 0x5562b84482b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53578/0 (socket says 172.21.15.33:53578) 2024-05-03T18:28:57.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.174+0000 7fa063e00700 1 -- 172.21.15.33:0/2995191237 learned_addr learned my addr 172.21.15.33:0/2995191237 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:28:57.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.174+0000 7fa069a00700 1 --2- 172.21.15.33:0/2995191237 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5562b84452d0 0x5562b84487f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:28:57.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.175+0000 7fa063e00700 1 -- 172.21.15.33:0/2995191237 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5562b843f5d0 msgr2=0x5562b844af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:28:57.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.175+0000 7fa063e00700 1 --2- 172.21.15.33:0/2995191237 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5562b843f5d0 0x5562b844af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:57.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.175+0000 7fa063e00700 1 -- 172.21.15.33:0/2995191237 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5562b84452d0 msgr2=0x5562b84487f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:28:57.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.175+0000 7fa063e00700 1 --2- 172.21.15.33:0/2995191237 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5562b84452d0 0x5562b84487f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:57.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.175+0000 7fa063e00700 1 -- 172.21.15.33:0/2995191237 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562b83574a0 con 0x5562b84449b0 2024-05-03T18:28:57.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.175+0000 7fa063e00700 1 --2- 172.21.15.33:0/2995191237 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5562b84449b0 0x5562b84482b0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fa0500dfef0 tx=0x7fa0500108d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e13cf35d4c2e81d server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:28:57.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.175+0000 7fa04aa00700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:28:57.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.176+0000 7fa04aa00700 1 -- 172.21.15.33:0/2995191237 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa050020040 con 0x5562b84449b0 2024-05-03T18:28:57.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.176+0000 7fa04aa00700 1 -- 172.21.15.33:0/2995191237 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7fa0500e2330 con 0x5562b84449b0 2024-05-03T18:28:57.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.176+0000 7fa076430b80 1 -- 172.21.15.33:0/2995191237 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa058004830 con 0x5562b84449b0 2024-05-03T18:28:57.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.176+0000 7fa04aa00700 1 -- 172.21.15.33:0/2995191237 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fa0500a9480 con 0x5562b84449b0 2024-05-03T18:28:57.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.176+0000 7fa076430b80 10 client.15498 fetch_fsmap learned FSMap version 21 2024-05-03T18:28:57.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.176+0000 7fa076430b80 1 -- 172.21.15.33:0/2995191237 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5562b84564e0 con 0x5562b84449b0 2024-05-03T18:28:57.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.177+0000 7fa04aa00700 1 -- 172.21.15.33:0/2995191237 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa058004830 con 0x5562b84449b0 2024-05-03T18:28:57.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.177+0000 7fa04aa00700 1 -- 172.21.15.33:0/2995191237 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fa0500b1e50 con 0x5562b84449b0 2024-05-03T18:28:57.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.177+0000 7fa076430b80 10 client.15498 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:28:57.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.177+0000 7fa076430b80 10 client.15498 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:28:57.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.177+0000 7fa076430b80 20 client.15498 populate_metadata read hostname 'smithi033' 2024-05-03T18:28:57.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.177+0000 7fa076430b80 1 --2- 172.21.15.33:0/2995191237 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5562b8458a70 0x5562b845ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:28:57.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.177+0000 7fa076430b80 4 client.15498 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:28:57.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.177+0000 7fa076430b80 1 -- 172.21.15.33:0/2995191237 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5562b840c070 con 0x5562b8458a70 2024-05-03T18:28:57.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.178+0000 7fa069000700 1 --2- 172.21.15.33:0/2995191237 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5562b8458a70 0x5562b845ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:28:57.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.179+0000 7fa069000700 1 --2- 172.21.15.33:0/2995191237 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5562b8458a70 0x5562b845ae60 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c94d5900c4ea5a9e server_cookie=ec4d89d5fbf61e62 in_seq=0 out_seq=0 2024-05-03T18:28:57.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.179+0000 7fa04aa00700 10 client.15498 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:28:57.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.181+0000 7fa04aa00700 1 -- 172.21.15.33:0/2995191237 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5562b840c070 con 0x5562b8458a70 2024-05-03T18:28:57.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.181+0000 7fa04aa00700 10 client.15498 handle_command_reply: tid=0 2024-05-03T18:28:57.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.181+0000 7fa076430b80 1 client.15498 shutdown 2024-05-03T18:28:57.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.182+0000 7fa076430b80 1 -- 172.21.15.33:0/2995191237 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5562b84449b0 msgr2=0x5562b84482b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:28:57.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.182+0000 7fa076430b80 1 --2- 172.21.15.33:0/2995191237 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5562b84449b0 0x5562b84482b0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fa0500dfef0 tx=0x7fa0500108d0 comp rx=0 tx=0).stop 2024-05-03T18:28:57.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.182+0000 7fa076430b80 1 -- 172.21.15.33:0/2995191237 shutdown_connections 2024-05-03T18:28:57.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.182+0000 7fa076430b80 1 --2- 172.21.15.33:0/2995191237 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5562b8458a70 0x5562b845ae60 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:57.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.182+0000 7fa076430b80 1 --2- 172.21.15.33:0/2995191237 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5562b84452d0 0x5562b84487f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:57.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.182+0000 7fa076430b80 1 --2- 172.21.15.33:0/2995191237 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5562b843f5d0 0x5562b844af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:57.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.182+0000 7fa076430b80 1 --2- 172.21.15.33:0/2995191237 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5562b84449b0 0x5562b84482b0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:57.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.182+0000 7fa076430b80 1 -- 172.21.15.33:0/2995191237 >> 172.21.15.33:0/2995191237 conn(0x5562b83b7750 msgr2=0x5562b8434430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:28:57.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.182+0000 7fa04aa00700 0 client.15498 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:28:57.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.182+0000 7fa076430b80 1 -- 172.21.15.33:0/2995191237 shutdown_connections 2024-05-03T18:28:57.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.182+0000 7fa076430b80 1 -- 172.21.15.33:0/2995191237 wait complete. 2024-05-03T18:28:57.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.182+0000 7fa076430b80 20 client.15498 trim_cache size 0 max 16384 2024-05-03T18:28:57.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.217+0000 7fa076430b80 1 Processor -- start 2024-05-03T18:28:57.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.217+0000 7fa076430b80 1 -- start start 2024-05-03T18:28:57.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.218+0000 7fa076430b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5562b83b7750 0x5562b843efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:28:57.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.218+0000 7fa076430b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5562b843f5d0 0x5562b8445e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:28:57.218 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:28:56 smithi033 ceph-mon[28330]: pgmap v55: 129 pgs: 129 active+clean; 242 MiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 MiB/s wr, 11 op/s 2024-05-03T18:28:57.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.218+0000 7fa076430b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5562b84452d0 0x5562b8460710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:28:57.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.218+0000 7fa076430b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa0641ab280 con 0x5562b843f5d0 2024-05-03T18:28:57.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.218+0000 7fa076430b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa0641adb00 con 0x5562b84452d0 2024-05-03T18:28:57.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.218+0000 7fa076430b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa0641ab8f0 con 0x5562b83b7750 2024-05-03T18:28:57.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.218+0000 7fa063e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5562b843f5d0 0x5562b8445e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:28:57.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.218+0000 7fa063e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5562b843f5d0 0x5562b8445e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53592/0 (socket says 172.21.15.33:53592) 2024-05-03T18:28:57.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.218+0000 7fa063e00700 1 -- 172.21.15.33:0/1419539065 learned_addr learned my addr 172.21.15.33:0/1419539065 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:28:57.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.219+0000 7fa069000700 1 --2- 172.21.15.33:0/1419539065 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5562b83b7750 0x5562b843efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:28:57.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.219+0000 7fa063e00700 1 -- 172.21.15.33:0/1419539065 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5562b83b7750 msgr2=0x5562b843efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:28:57.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.219+0000 7fa063e00700 1 --2- 172.21.15.33:0/1419539065 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5562b83b7750 0x5562b843efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:57.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.219+0000 7fa063e00700 1 -- 172.21.15.33:0/1419539065 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5562b84452d0 msgr2=0x5562b8460710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:28:57.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.219+0000 7fa063e00700 1 --2- 172.21.15.33:0/1419539065 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5562b84452d0 0x5562b8460710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:57.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.219+0000 7fa063e00700 1 -- 172.21.15.33:0/1419539065 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562b8448080 con 0x5562b843f5d0 2024-05-03T18:28:57.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.219+0000 7fa069000700 1 --2- 172.21.15.33:0/1419539065 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5562b83b7750 0x5562b843efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:28:57.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.219+0000 7fa063e00700 1 --2- 172.21.15.33:0/1419539065 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5562b843f5d0 0x5562b8445e60 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fa05000b2f0 tx=0x7fa050000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5cb3d390d16a524 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:28:57.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.220+0000 7fa055600700 1 -- 172.21.15.33:0/1419539065 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0500c7020 con 0x5562b843f5d0 2024-05-03T18:28:57.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.220+0000 7fa055600700 1 -- 172.21.15.33:0/1419539065 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa0500ad830 con 0x5562b843f5d0 2024-05-03T18:28:57.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.220+0000 7fa055600700 1 -- 172.21.15.33:0/1419539065 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa050020030 con 0x5562b843f5d0 2024-05-03T18:28:57.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.221+0000 7fa076430b80 1 -- 172.21.15.33:0/1419539065 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5562b843f5d0 msgr2=0x5562b8445e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:28:57.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.221+0000 7fa076430b80 1 --2- 172.21.15.33:0/1419539065 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5562b843f5d0 0x5562b8445e60 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fa05000b2f0 tx=0x7fa050000bc0 comp rx=0 tx=0).stop 2024-05-03T18:28:57.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.222+0000 7fa076430b80 1 -- 172.21.15.33:0/1419539065 shutdown_connections 2024-05-03T18:28:57.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.222+0000 7fa076430b80 1 --2- 172.21.15.33:0/1419539065 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5562b84452d0 0x5562b8460710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:57.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.222+0000 7fa076430b80 1 --2- 172.21.15.33:0/1419539065 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5562b83b7750 0x5562b843efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:57.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.222+0000 7fa076430b80 1 --2- 172.21.15.33:0/1419539065 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5562b843f5d0 0x5562b8445e60 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:57.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.222+0000 7fa076430b80 1 -- 172.21.15.33:0/1419539065 >> 172.21.15.33:0/1419539065 conn(0x5562b843b880 msgr2=0x5562b8445750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:28:57.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.222+0000 7fa076430b80 1 -- 172.21.15.33:0/1419539065 shutdown_connections 2024-05-03T18:28:57.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.222+0000 7fa076430b80 1 -- 172.21.15.33:0/1419539065 wait complete. 2024-05-03T18:28:57.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.223+0000 7fa076430b80 1 -- start start 2024-05-03T18:28:57.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.223+0000 7fa076430b80 1 Processor -- start 2024-05-03T18:28:57.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.224+0000 7fa076430b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5562b83b7750 0x5562b844e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:28:57.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.224+0000 7fa076430b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5562b843f5d0 0x5562b844e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:28:57.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.224+0000 7fa076430b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5562b84452d0 0x5562b844ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:28:57.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.225+0000 7fa076430b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa0641abfc0 con 0x5562b84452d0 2024-05-03T18:28:57.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.225+0000 7fa076430b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa0641ad430 con 0x5562b83b7750 2024-05-03T18:28:57.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.225+0000 7fa076430b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa0641ac690 con 0x5562b843f5d0 2024-05-03T18:28:57.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.225+0000 7fa076430b80 10 client.0 _ll_register_callbacks cb 0x5562b8338cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:28:57.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.225+0000 7fa069a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5562b84452d0 0x5562b844ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:28:57.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.225+0000 7fa069a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5562b84452d0 0x5562b844ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53604/0 (socket says 172.21.15.33:53604) 2024-05-03T18:28:57.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.225+0000 7fa069a00700 1 -- 172.21.15.33:0/2145951862 learned_addr learned my addr 172.21.15.33:0/2145951862 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:28:57.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.225+0000 7fa069000700 1 --2- 172.21.15.33:0/2145951862 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5562b83b7750 0x5562b844e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:28:57.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.225+0000 7fa069a00700 1 -- 172.21.15.33:0/2145951862 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5562b843f5d0 msgr2=0x5562b844e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:28:57.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.225+0000 7fa069a00700 1 --2- 172.21.15.33:0/2145951862 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5562b843f5d0 0x5562b844e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:57.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.225+0000 7fa069a00700 1 -- 172.21.15.33:0/2145951862 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5562b83b7750 msgr2=0x5562b844e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:28:57.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.225+0000 7fa069a00700 1 --2- 172.21.15.33:0/2145951862 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5562b83b7750 0x5562b844e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:57.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.225+0000 7fa069a00700 1 -- 172.21.15.33:0/2145951862 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562b84c32d0 con 0x5562b84452d0 2024-05-03T18:28:57.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.226+0000 7fa069a00700 1 --2- 172.21.15.33:0/2145951862 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5562b84452d0 0x5562b844ebe0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fa05c015000 tx=0x7fa05c009dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e1882a03dee446b server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:28:57.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.226+0000 7fa04aa00700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:28:57.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.226+0000 7fa04aa00700 1 -- 172.21.15.33:0/2145951862 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa05c027040 con 0x5562b84452d0 2024-05-03T18:28:57.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.226+0000 7fa04aa00700 1 -- 172.21.15.33:0/2145951862 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7fa05c0056d0 con 0x5562b84452d0 2024-05-03T18:28:57.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.227+0000 7fa076430b80 1 -- 172.21.15.33:0/2145951862 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0500ad830 con 0x5562b84452d0 2024-05-03T18:28:57.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.227+0000 7fa04aa00700 1 -- 172.21.15.33:0/2145951862 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fa05c039050 con 0x5562b84452d0 2024-05-03T18:28:57.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.228+0000 7fa076430b80 10 client.15504 fetch_fsmap learned FSMap version 21 2024-05-03T18:28:57.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.228+0000 7fa076430b80 1 -- 172.21.15.33:0/2145951862 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5562b84c44d0 con 0x5562b84452d0 2024-05-03T18:28:57.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.228+0000 7fa04aa00700 1 -- 172.21.15.33:0/2145951862 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa0500ad830 con 0x5562b84452d0 2024-05-03T18:28:57.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.228+0000 7fa04aa00700 1 -- 172.21.15.33:0/2145951862 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fa05c02a070 con 0x5562b84452d0 2024-05-03T18:28:57.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.228+0000 7fa076430b80 10 client.15504 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:28:57.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.228+0000 7fa076430b80 10 client.15504 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:28:57.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.228+0000 7fa076430b80 20 client.15504 populate_metadata read hostname 'smithi033' 2024-05-03T18:28:57.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.228+0000 7fa076430b80 1 --2- 172.21.15.33:0/2145951862 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5562b84c4c60 0x5562b84c7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:28:57.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.228+0000 7fa076430b80 4 client.15504 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-03T18:28:57.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.228+0000 7fa076430b80 1 -- 172.21.15.33:0/2145951862 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5562b839f480 con 0x5562b84c4c60 2024-05-03T18:28:57.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.230+0000 7fa069000700 1 --2- 172.21.15.33:0/2145951862 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5562b84c4c60 0x5562b84c7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:28:57.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.231+0000 7fa04aa00700 10 client.15504 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:28:57.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.231+0000 7fa069000700 1 --2- 172.21.15.33:0/2145951862 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5562b84c4c60 0x5562b84c7080 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a56986d450822c24 server_cookie=b4cf1263ba474390 in_seq=0 out_seq=0 2024-05-03T18:28:57.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.234+0000 7fa04aa00700 1 -- 172.21.15.33:0/2145951862 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5562b839f480 con 0x5562b84c4c60 2024-05-03T18:28:57.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.234+0000 7fa04aa00700 10 client.15504 handle_command_reply: tid=0 2024-05-03T18:28:57.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.234+0000 7fa076430b80 1 client.15504 shutdown 2024-05-03T18:28:57.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.234+0000 7fa076430b80 1 -- 172.21.15.33:0/2145951862 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5562b84452d0 msgr2=0x5562b844ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:28:57.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.234+0000 7fa076430b80 1 --2- 172.21.15.33:0/2145951862 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5562b84452d0 0x5562b844ebe0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fa05c015000 tx=0x7fa05c009dd0 comp rx=0 tx=0).stop 2024-05-03T18:28:57.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.234+0000 7fa076430b80 1 -- 172.21.15.33:0/2145951862 shutdown_connections 2024-05-03T18:28:57.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.234+0000 7fa076430b80 1 --2- 172.21.15.33:0/2145951862 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5562b84c4c60 0x5562b84c7080 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:57.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.234+0000 7fa076430b80 1 --2- 172.21.15.33:0/2145951862 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5562b84452d0 0x5562b844ebe0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:57.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.234+0000 7fa076430b80 1 --2- 172.21.15.33:0/2145951862 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5562b83b7750 0x5562b844e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:57.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.234+0000 7fa076430b80 1 --2- 172.21.15.33:0/2145951862 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5562b843f5d0 0x5562b844e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:57.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.234+0000 7fa076430b80 1 -- 172.21.15.33:0/2145951862 >> 172.21.15.33:0/2145951862 conn(0x5562b843b880 msgr2=0x5562b8433ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:28:57.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.234+0000 7fa04aa00700 0 client.15504 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:28:57.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.235+0000 7fa076430b80 1 -- 172.21.15.33:0/2145951862 shutdown_connections 2024-05-03T18:28:57.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.235+0000 7fa076430b80 1 -- 172.21.15.33:0/2145951862 wait complete. 2024-05-03T18:28:57.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.235+0000 7fa076430b80 20 client.15504 trim_cache size 0 max 16384 2024-05-03T18:28:57.236 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-03T18:28:57.236 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-05-03T18:28:57.236 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "99ad8cc4-a299-4459-87da-e5ec78054801", 2024-05-03T18:28:57.237 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-05-03T18:28:57.237 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-03T18:28:57.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.236+0000 7fa06ba00700 1 -- 172.21.15.33:0/3629724275 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fa04c0b2a70 msgr2=0x7fa04c0b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:28:57.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.236+0000 7fa06ba00700 1 --2- 172.21.15.33:0/3629724275 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fa04c0b2a70 0x7fa04c0b4f30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa05c009e60 tx=0x7fa05c00b3b0 comp rx=0 tx=0).stop 2024-05-03T18:28:57.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.236+0000 7fa06ba00700 1 -- 172.21.15.33:0/3629724275 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa06406bad0 msgr2=0x7fa0641a1e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:28:57.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.236+0000 7fa06ba00700 1 --2- 172.21.15.33:0/3629724275 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa06406bad0 0x7fa0641a1e10 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fa05000d950 tx=0x7fa05000dc60 comp rx=0 tx=0).stop 2024-05-03T18:28:57.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.237+0000 7fa06ba00700 1 -- 172.21.15.33:0/3629724275 shutdown_connections 2024-05-03T18:28:57.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.237+0000 7fa06ba00700 1 --2- 172.21.15.33:0/3629724275 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fa04c0b2a70 0x7fa04c0b4f30 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:57.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.237+0000 7fa06ba00700 1 --2- 172.21.15.33:0/3629724275 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa06410ff50 0x7fa06419ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:57.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.237+0000 7fa06ba00700 1 --2- 172.21.15.33:0/3629724275 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa06406b1b0 0x7fa0641a18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:57.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.237+0000 7fa06ba00700 1 --2- 172.21.15.33:0/3629724275 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa06406bad0 0x7fa0641a1e10 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:28:57.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.237+0000 7fa06ba00700 1 -- 172.21.15.33:0/3629724275 >> 172.21.15.33:0/3629724275 conn(0x7fa064079350 msgr2=0x7fa064111460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:28:57.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.237+0000 7fa06ba00700 1 -- 172.21.15.33:0/3629724275 shutdown_connections 2024-05-03T18:28:57.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:28:57.237+0000 7fa06ba00700 1 -- 172.21.15.33:0/3629724275 wait complete. 2024-05-03T18:28:57.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:28:56 smithi099 ceph-mon[31658]: pgmap v55: 129 pgs: 129 active+clean; 242 MiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 MiB/s wr, 11 op/s 2024-05-03T18:28:57.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:28:56 smithi150 ceph-mon[32492]: pgmap v55: 129 pgs: 129 active+clean; 242 MiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 MiB/s wr, 11 op/s 2024-05-03T18:28:58.209 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-03T18:28:58.293 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:28:57 smithi033 ceph-mon[28330]: pgmap v56: 129 pgs: 129 active+clean; 278 MiB data, 969 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 MiB/s wr, 16 op/s 2024-05-03T18:28:58.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:28:57 smithi099 ceph-mon[31658]: pgmap v56: 129 pgs: 129 active+clean; 278 MiB data, 969 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 MiB/s wr, 16 op/s 2024-05-03T18:28:58.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:28:57 smithi150 ceph-mon[32492]: pgmap v56: 129 pgs: 129 active+clean; 278 MiB data, 969 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 MiB/s wr, 16 op/s 2024-05-03T18:28:58.737 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:28:59.141 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:28:59 smithi033 ceph-mon[28330]: scrub queued for path: / 2024-05-03T18:28:59.141 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:28:59 smithi033 ceph-mon[28330]: scrub summary: idle+waiting paths [/] 2024-05-03T18:28:59.141 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:28:59 smithi033 ceph-mon[28330]: scrub summary: active paths [/] 2024-05-03T18:28:59.141 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:28:59 smithi033 ceph-mon[28330]: scrub summary: idle+waiting paths [/] 2024-05-03T18:28:59.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:28:59 smithi099 ceph-mon[31658]: scrub queued for path: / 2024-05-03T18:28:59.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:28:59 smithi099 ceph-mon[31658]: scrub summary: idle+waiting paths [/] 2024-05-03T18:28:59.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:28:59 smithi099 ceph-mon[31658]: scrub summary: active paths [/] 2024-05-03T18:28:59.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:28:59 smithi099 ceph-mon[31658]: scrub summary: idle+waiting paths [/] 2024-05-03T18:28:59.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:28:59 smithi150 ceph-mon[32492]: scrub queued for path: / 2024-05-03T18:28:59.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:28:59 smithi150 ceph-mon[32492]: scrub summary: idle+waiting paths [/] 2024-05-03T18:28:59.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:28:59 smithi150 ceph-mon[32492]: scrub summary: active paths [/] 2024-05-03T18:28:59.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:28:59 smithi150 ceph-mon[32492]: scrub summary: idle+waiting paths [/] 2024-05-03T18:29:00.060 INFO:teuthology.orchestra.run.smithi033.stderr:1000000 of 3200000 tuples (31%) done (elapsed 37.17 s, remaining 81.77 s) 2024-05-03T18:29:00.194 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:29:00 smithi033 ceph-mon[28330]: pgmap v57: 129 pgs: 129 active+clean; 278 MiB data, 969 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 MiB/s wr, 11 op/s 2024-05-03T18:29:00.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:29:00 smithi099 ceph-mon[31658]: pgmap v57: 129 pgs: 129 active+clean; 278 MiB data, 969 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 MiB/s wr, 11 op/s 2024-05-03T18:29:00.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:29:00 smithi150 ceph-mon[32492]: pgmap v57: 129 pgs: 129 active+clean; 278 MiB data, 969 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 MiB/s wr, 11 op/s 2024-05-03T18:29:01.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.105+0000 7fe563e00700 1 -- 172.21.15.33:0/1703515341 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5641033d0 msgr2=0x7fe564103850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:01.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.105+0000 7fe563e00700 1 --2- 172.21.15.33:0/1703515341 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5641033d0 0x7fe564103850 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fe550009990 tx=0x7fe550009ca0 comp rx=0 tx=0).stop 2024-05-03T18:29:01.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.105+0000 7fe563e00700 1 -- 172.21.15.33:0/1703515341 shutdown_connections 2024-05-03T18:29:01.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.105+0000 7fe563e00700 1 --2- 172.21.15.33:0/1703515341 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe564103d90 0x7fe5641102a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:01.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.105+0000 7fe563e00700 1 --2- 172.21.15.33:0/1703515341 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe564109460 0x7fe564109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:01.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.105+0000 7fe563e00700 1 --2- 172.21.15.33:0/1703515341 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5641033d0 0x7fe564103850 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:01.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.105+0000 7fe563e00700 1 -- 172.21.15.33:0/1703515341 >> 172.21.15.33:0/1703515341 conn(0x7fe5640fef20 msgr2=0x7fe564101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:29:01.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.105+0000 7fe563e00700 1 -- 172.21.15.33:0/1703515341 shutdown_connections 2024-05-03T18:29:01.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.105+0000 7fe563e00700 1 -- 172.21.15.33:0/1703515341 wait complete. 2024-05-03T18:29:01.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.106+0000 7fe563e00700 1 Processor -- start 2024-05-03T18:29:01.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.107+0000 7fe563e00700 1 -- start start 2024-05-03T18:29:01.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.107+0000 7fe563e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5641033d0 0x7fe5640757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:01.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.107+0000 7fe563e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe564103d90 0x7fe564075ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:01.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.107+0000 7fe563e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe564109460 0x7fe564076220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:01.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.107+0000 7fe563e00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe5641134a0 con 0x7fe5641033d0 2024-05-03T18:29:01.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.107+0000 7fe563e00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe564113320 con 0x7fe564103d90 2024-05-03T18:29:01.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.107+0000 7fe563e00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe564113620 con 0x7fe564109460 2024-05-03T18:29:01.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.108+0000 7fe562a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5641033d0 0x7fe5640757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:01.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.108+0000 7fe563400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe564109460 0x7fe564076220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:01.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.108+0000 7fe562a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5641033d0 0x7fe5640757a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53622/0 (socket says 172.21.15.33:53622) 2024-05-03T18:29:01.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.108+0000 7fe562a00700 1 -- 172.21.15.33:0/3364795419 learned_addr learned my addr 172.21.15.33:0/3364795419 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:29:01.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.108+0000 7fe562000700 1 --2- 172.21.15.33:0/3364795419 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe564103d90 0x7fe564075ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:01.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.108+0000 7fe562a00700 1 -- 172.21.15.33:0/3364795419 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe564109460 msgr2=0x7fe564076220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:01.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.108+0000 7fe562a00700 1 --2- 172.21.15.33:0/3364795419 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe564109460 0x7fe564076220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:01.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.108+0000 7fe562a00700 1 -- 172.21.15.33:0/3364795419 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe564103d90 msgr2=0x7fe564075ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:01.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.108+0000 7fe562a00700 1 --2- 172.21.15.33:0/3364795419 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe564103d90 0x7fe564075ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:01.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.109+0000 7fe562a00700 1 -- 172.21.15.33:0/3364795419 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5641ac3a0 con 0x7fe5641033d0 2024-05-03T18:29:01.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.109+0000 7fe562a00700 1 --2- 172.21.15.33:0/3364795419 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5641033d0 0x7fe5640757a0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fe55400d950 tx=0x7fe55400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:29:01.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.109+0000 7fe568a00700 1 -- 172.21.15.33:0/3364795419 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe554016070 con 0x7fe5641033d0 2024-05-03T18:29:01.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.109+0000 7fe568a00700 1 -- 172.21.15.33:0/3364795419 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe554009dd0 con 0x7fe5641033d0 2024-05-03T18:29:01.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.109+0000 7fe568a00700 1 -- 172.21.15.33:0/3364795419 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe55400f3c0 con 0x7fe5641033d0 2024-05-03T18:29:01.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.109+0000 7fe563e00700 1 -- 172.21.15.33:0/3364795419 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5641ac630 con 0x7fe5641033d0 2024-05-03T18:29:01.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.110+0000 7fe563e00700 1 -- 172.21.15.33:0/3364795419 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5641acb20 con 0x7fe5641033d0 2024-05-03T18:29:01.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.115+0000 7fe568a00700 1 -- 172.21.15.33:0/3364795419 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fe554019070 con 0x7fe5641033d0 2024-05-03T18:29:01.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.115+0000 7fe5747c4b80 1 Processor -- start 2024-05-03T18:29:01.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.116+0000 7fe5747c4b80 1 -- start start 2024-05-03T18:29:01.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.116+0000 7fe5747c4b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5621014fbbd0 0x5621014fbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:01.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.116+0000 7fe5747c4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621014f5f30 0x5621014f63b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:01.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.116+0000 7fe5747c4b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621014f68f0 0x562101502990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:01.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.116+0000 7fe5747c4b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x562101448a20 con 0x5621014f68f0 2024-05-03T18:29:01.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.116+0000 7fe5747c4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5621013e6760 con 0x5621014f5f30 2024-05-03T18:29:01.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.116+0000 7fe5747c4b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5621014449e0 con 0x5621014fbbd0 2024-05-03T18:29:01.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.117+0000 7fe562000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621014f68f0 0x562101502990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:01.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.117+0000 7fe562000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621014f68f0 0x562101502990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53636/0 (socket says 172.21.15.33:53636) 2024-05-03T18:29:01.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.117+0000 7fe562000700 1 -- 172.21.15.33:0/2893409904 learned_addr learned my addr 172.21.15.33:0/2893409904 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:29:01.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.117+0000 7fe562a00700 1 --2- 172.21.15.33:0/2893409904 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621014f5f30 0x5621014f63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:01.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.117+0000 7fe563400700 1 --2- 172.21.15.33:0/2893409904 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5621014fbbd0 0x5621014fbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:01.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.117+0000 7fe568a00700 1 --2- 172.21.15.33:0/3364795419 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fe5480b2a70 0x7fe5480b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:01.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.118+0000 7fe568a00700 1 -- 172.21.15.33:0/3364795419 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7fe5540e2530 con 0x7fe5641033d0 2024-05-03T18:29:01.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.118+0000 7fe562a00700 1 -- 172.21.15.33:0/2893409904 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5621014fbbd0 msgr2=0x5621014fbfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:29:01.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.118+0000 7fe562a00700 1 --2- 172.21.15.33:0/2893409904 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5621014fbbd0 0x5621014fbfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:01.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.118+0000 7fe562a00700 1 -- 172.21.15.33:0/2893409904 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621014f68f0 msgr2=0x562101502990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:29:01.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.118+0000 7fe562a00700 1 --2- 172.21.15.33:0/2893409904 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621014f68f0 0x562101502990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:01.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.118+0000 7fe562a00700 1 -- 172.21.15.33:0/2893409904 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621013958d0 con 0x5621014f5f30 2024-05-03T18:29:01.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.118+0000 7fe563400700 1 --2- 172.21.15.33:0/3364795419 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fe5480b2a70 0x7fe5480b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:01.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.119+0000 7fe562a00700 1 --2- 172.21.15.33:0/2893409904 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621014f5f30 0x5621014f63b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe5540df8f0 tx=0x7fe554000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8af9e2f38b608efc server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:29:01.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.119+0000 7fe563400700 1 --2- 172.21.15.33:0/3364795419 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fe5480b2a70 0x7fe5480b4f30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe55c009690 tx=0x7fe55c004ad0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:29:01.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.120+0000 7fe558c00700 1 -- 172.21.15.33:0/2893409904 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe554020070 con 0x5621014f5f30 2024-05-03T18:29:01.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.120+0000 7fe558c00700 1 -- 172.21.15.33:0/2893409904 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe5540e5c90 con 0x5621014f5f30 2024-05-03T18:29:01.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.120+0000 7fe558c00700 1 -- 172.21.15.33:0/2893409904 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5540a9480 con 0x5621014f5f30 2024-05-03T18:29:01.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.121+0000 7fe5747c4b80 1 -- 172.21.15.33:0/2893409904 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621014f5f30 msgr2=0x5621014f63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:01.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.121+0000 7fe5747c4b80 1 --2- 172.21.15.33:0/2893409904 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621014f5f30 0x5621014f63b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe5540df8f0 tx=0x7fe554000bc0 comp rx=0 tx=0).stop 2024-05-03T18:29:01.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.121+0000 7fe5747c4b80 1 -- 172.21.15.33:0/2893409904 shutdown_connections 2024-05-03T18:29:01.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.121+0000 7fe5747c4b80 1 --2- 172.21.15.33:0/2893409904 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621014f68f0 0x562101502990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:01.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.121+0000 7fe5747c4b80 1 --2- 172.21.15.33:0/2893409904 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5621014fbbd0 0x5621014fbfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:01.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.121+0000 7fe5747c4b80 1 --2- 172.21.15.33:0/2893409904 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621014f5f30 0x5621014f63b0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:01.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.121+0000 7fe5747c4b80 1 -- 172.21.15.33:0/2893409904 >> 172.21.15.33:0/2893409904 conn(0x56210146e6d0 msgr2=0x5621014f0bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:29:01.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.121+0000 7fe5747c4b80 1 -- 172.21.15.33:0/2893409904 shutdown_connections 2024-05-03T18:29:01.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.121+0000 7fe5747c4b80 1 -- 172.21.15.33:0/2893409904 wait complete. 2024-05-03T18:29:01.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.122+0000 7fe5747c4b80 1 -- start start 2024-05-03T18:29:01.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.122+0000 7fe5747c4b80 1 Processor -- start 2024-05-03T18:29:01.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.123+0000 7fe5747c4b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5621014f5f30 0x562101502570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:01.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.123+0000 7fe5747c4b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621014f68f0 0x5621014ff910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:01.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.123+0000 7fe5747c4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621014fbbd0 0x5621014ffe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:01.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.123+0000 7fe5747c4b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x562101502ed0 con 0x5621014f68f0 2024-05-03T18:29:01.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.123+0000 7fe5747c4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5621013aeb20 con 0x5621014fbbd0 2024-05-03T18:29:01.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.123+0000 7fe5747c4b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x562101503050 con 0x5621014f5f30 2024-05-03T18:29:01.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.123+0000 7fe5747c4b80 10 client.0 _ll_register_callbacks cb 0x5621013efc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:29:01.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.124+0000 7fe562000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621014f68f0 0x5621014ff910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:01.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.124+0000 7fe563400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621014fbbd0 0x5621014ffe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:01.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.124+0000 7fe562a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5621014f5f30 0x562101502570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:01.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.124+0000 7fe562000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621014f68f0 0x5621014ff910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53652/0 (socket says 172.21.15.33:53652) 2024-05-03T18:29:01.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.124+0000 7fe563400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621014fbbd0 0x5621014ffe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:50222/0 (socket says 172.21.15.33:50222) 2024-05-03T18:29:01.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.124+0000 7fe562000700 1 -- 172.21.15.33:0/3049728142 learned_addr learned my addr 172.21.15.33:0/3049728142 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:29:01.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.124+0000 7fe562a00700 1 --2- 172.21.15.33:0/3049728142 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5621014f5f30 0x562101502570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:50952/0 (socket says 172.21.15.33:50952) 2024-05-03T18:29:01.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.124+0000 7fe562000700 1 -- 172.21.15.33:0/3049728142 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5621014f5f30 msgr2=0x562101502570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:29:01.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.124+0000 7fe562000700 1 --2- 172.21.15.33:0/3049728142 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5621014f5f30 0x562101502570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:01.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.124+0000 7fe562000700 1 -- 172.21.15.33:0/3049728142 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621014fbbd0 msgr2=0x5621014ffe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:29:01.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.124+0000 7fe562000700 1 --2- 172.21.15.33:0/3049728142 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621014fbbd0 0x5621014ffe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:01.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.124+0000 7fe562000700 1 -- 172.21.15.33:0/3049728142 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56210140e420 con 0x5621014f68f0 2024-05-03T18:29:01.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.125+0000 7fe562000700 1 --2- 172.21.15.33:0/3049728142 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621014f68f0 0x5621014ff910 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fe550009990 tx=0x7fe55000bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9e8658209229fa5 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:29:01.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.125+0000 7fe54d600700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:29:01.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.125+0000 7fe54d600700 1 -- 172.21.15.33:0/3049728142 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe550005270 con 0x5621014f68f0 2024-05-03T18:29:01.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.126+0000 7fe54d600700 1 -- 172.21.15.33:0/3049728142 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7fe550005410 con 0x5621014f68f0 2024-05-03T18:29:01.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.126+0000 7fe5747c4b80 1 -- 172.21.15.33:0/3049728142 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5621014c2ff0 con 0x5621014f68f0 2024-05-03T18:29:01.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.126+0000 7fe54d600700 1 -- 172.21.15.33:0/3049728142 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fe550031050 con 0x5621014f68f0 2024-05-03T18:29:01.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.126+0000 7fe5747c4b80 10 client.15516 fetch_fsmap learned FSMap version 21 2024-05-03T18:29:01.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.126+0000 7fe5747c4b80 1 -- 172.21.15.33:0/3049728142 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x56210151e640 con 0x5621014f68f0 2024-05-03T18:29:01.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.126+0000 7fe54d600700 1 -- 172.21.15.33:0/3049728142 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x5621014c2ff0 con 0x5621014f68f0 2024-05-03T18:29:01.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.127+0000 7fe54d600700 1 -- 172.21.15.33:0/3049728142 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fe550023470 con 0x5621014f68f0 2024-05-03T18:29:01.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.127+0000 7fe5747c4b80 10 client.15516 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:29:01.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.127+0000 7fe5747c4b80 10 client.15516 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:29:01.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.127+0000 7fe5747c4b80 20 client.15516 populate_metadata read hostname 'smithi033' 2024-05-03T18:29:01.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.127+0000 7fe5747c4b80 1 --2- 172.21.15.33:0/3049728142 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x562101520cd0 0x562101523100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:01.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.127+0000 7fe5747c4b80 4 client.15516 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:29:01.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.127+0000 7fe5747c4b80 1 -- 172.21.15.33:0/3049728142 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562101456400 con 0x562101520cd0 2024-05-03T18:29:01.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.127+0000 7fe562a00700 1 --2- 172.21.15.33:0/3049728142 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x562101520cd0 0x562101523100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:01.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.128+0000 7fe562a00700 1 --2- 172.21.15.33:0/3049728142 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x562101520cd0 0x562101523100 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12b7c1def1537c50 server_cookie=94db3c273729fe27 in_seq=0 out_seq=0 2024-05-03T18:29:01.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.128+0000 7fe54d600700 10 client.15516 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:29:01.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.130+0000 7fe54d600700 1 -- 172.21.15.33:0/3049728142 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562101456400 con 0x562101520cd0 2024-05-03T18:29:01.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.130+0000 7fe54d600700 10 client.15516 handle_command_reply: tid=0 2024-05-03T18:29:01.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.130+0000 7fe5747c4b80 1 client.15516 shutdown 2024-05-03T18:29:01.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.130+0000 7fe5747c4b80 1 -- 172.21.15.33:0/3049728142 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621014f68f0 msgr2=0x5621014ff910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:01.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.130+0000 7fe5747c4b80 1 --2- 172.21.15.33:0/3049728142 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621014f68f0 0x5621014ff910 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fe550009990 tx=0x7fe55000bf90 comp rx=0 tx=0).stop 2024-05-03T18:29:01.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.131+0000 7fe5747c4b80 1 -- 172.21.15.33:0/3049728142 shutdown_connections 2024-05-03T18:29:01.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.131+0000 7fe5747c4b80 1 --2- 172.21.15.33:0/3049728142 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x562101520cd0 0x562101523100 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:01.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.131+0000 7fe5747c4b80 1 --2- 172.21.15.33:0/3049728142 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621014fbbd0 0x5621014ffe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:01.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.131+0000 7fe5747c4b80 1 --2- 172.21.15.33:0/3049728142 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5621014f5f30 0x562101502570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:01.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.131+0000 7fe5747c4b80 1 --2- 172.21.15.33:0/3049728142 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621014f68f0 0x5621014ff910 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:01.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.131+0000 7fe5747c4b80 1 -- 172.21.15.33:0/3049728142 >> 172.21.15.33:0/3049728142 conn(0x56210146e6d0 msgr2=0x5621014f4f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:29:01.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.131+0000 7fe54d600700 0 client.15516 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:29:01.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.131+0000 7fe5747c4b80 1 -- 172.21.15.33:0/3049728142 shutdown_connections 2024-05-03T18:29:01.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.131+0000 7fe5747c4b80 1 -- 172.21.15.33:0/3049728142 wait complete. 2024-05-03T18:29:01.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.131+0000 7fe5747c4b80 20 client.15516 trim_cache size 0 max 16384 2024-05-03T18:29:01.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.163+0000 7fe5747c4b80 1 Processor -- start 2024-05-03T18:29:01.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.163+0000 7fe5747c4b80 1 -- start start 2024-05-03T18:29:01.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.163+0000 7fe5747c4b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56210146e6d0 0x5621014fe200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:01.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.163+0000 7fe5747c4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621014f5f30 0x5621014fe740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:01.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.163+0000 7fe5747c4b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621014f68f0 0x5621015289d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:01.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.163+0000 7fe5747c4b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe5641a46c0 con 0x5621014f68f0 2024-05-03T18:29:01.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.163+0000 7fe5747c4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe5641ad9b0 con 0x5621014f5f30 2024-05-03T18:29:01.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.163+0000 7fe5747c4b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe5641a4d30 con 0x56210146e6d0 2024-05-03T18:29:01.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.164+0000 7fe563400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621014f68f0 0x5621015289d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:01.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.164+0000 7fe563400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621014f68f0 0x5621015289d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53658/0 (socket says 172.21.15.33:53658) 2024-05-03T18:29:01.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.164+0000 7fe563400700 1 -- 172.21.15.33:0/19521097 learned_addr learned my addr 172.21.15.33:0/19521097 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:29:01.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.164+0000 7fe562a00700 1 --2- 172.21.15.33:0/19521097 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56210146e6d0 0x5621014fe200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:01.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.164+0000 7fe562000700 1 --2- 172.21.15.33:0/19521097 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621014f5f30 0x5621014fe740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:01.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.165+0000 7fe563400700 1 -- 172.21.15.33:0/19521097 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56210146e6d0 msgr2=0x5621014fe200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:29:01.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.165+0000 7fe563400700 1 --2- 172.21.15.33:0/19521097 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56210146e6d0 0x5621014fe200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:01.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.165+0000 7fe563400700 1 -- 172.21.15.33:0/19521097 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621014f5f30 msgr2=0x5621014fe740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:29:01.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.165+0000 7fe563400700 1 --2- 172.21.15.33:0/19521097 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621014f5f30 0x5621014fe740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:01.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.165+0000 7fe563400700 1 -- 172.21.15.33:0/19521097 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621014fde80 con 0x5621014f68f0 2024-05-03T18:29:01.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.165+0000 7fe563400700 1 --2- 172.21.15.33:0/19521097 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621014f68f0 0x5621015289d0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fe55c006070 tx=0x7fe55c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e140a393b0677c53 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:29:01.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.166+0000 7fe54fe00700 1 -- 172.21.15.33:0/19521097 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe55c0089a0 con 0x5621014f68f0 2024-05-03T18:29:01.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.166+0000 7fe54fe00700 1 -- 172.21.15.33:0/19521097 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe55c029070 con 0x5621014f68f0 2024-05-03T18:29:01.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.166+0000 7fe54fe00700 1 -- 172.21.15.33:0/19521097 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe55c017db0 con 0x5621014f68f0 2024-05-03T18:29:01.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.166+0000 7fe5747c4b80 1 -- 172.21.15.33:0/19521097 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621014f68f0 msgr2=0x5621015289d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:01.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.166+0000 7fe5747c4b80 1 --2- 172.21.15.33:0/19521097 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621014f68f0 0x5621015289d0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fe55c006070 tx=0x7fe55c016000 comp rx=0 tx=0).stop 2024-05-03T18:29:01.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.167+0000 7fe5747c4b80 1 -- 172.21.15.33:0/19521097 shutdown_connections 2024-05-03T18:29:01.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.167+0000 7fe5747c4b80 1 --2- 172.21.15.33:0/19521097 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621014f68f0 0x5621015289d0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:01.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.167+0000 7fe5747c4b80 1 --2- 172.21.15.33:0/19521097 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56210146e6d0 0x5621014fe200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:01.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.167+0000 7fe5747c4b80 1 --2- 172.21.15.33:0/19521097 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621014f5f30 0x5621014fe740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:01.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.167+0000 7fe5747c4b80 1 -- 172.21.15.33:0/19521097 >> 172.21.15.33:0/19521097 conn(0x5621014ea6c0 msgr2=0x5621014f06b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:29:01.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.167+0000 7fe5747c4b80 1 -- 172.21.15.33:0/19521097 shutdown_connections 2024-05-03T18:29:01.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.167+0000 7fe5747c4b80 1 -- 172.21.15.33:0/19521097 wait complete. 2024-05-03T18:29:01.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.167+0000 7fe5747c4b80 1 -- start start 2024-05-03T18:29:01.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.168+0000 7fe5747c4b80 1 Processor -- start 2024-05-03T18:29:01.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.168+0000 7fe5747c4b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56210146e6d0 0x562101528490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:01.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.168+0000 7fe5747c4b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5621014f5f30 0x5621015289d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:01.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.168+0000 7fe5747c4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621014f68f0 0x562101528f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:01.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.168+0000 7fe5747c4b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe5641a5400 con 0x56210146e6d0 2024-05-03T18:29:01.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.168+0000 7fe5747c4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe5641ad2e0 con 0x5621014f68f0 2024-05-03T18:29:01.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.168+0000 7fe5747c4b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe5641a5ad0 con 0x5621014f5f30 2024-05-03T18:29:01.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.168+0000 7fe5747c4b80 10 client.0 _ll_register_callbacks cb 0x5621013efc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:29:01.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.169+0000 7fe562a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56210146e6d0 0x562101528490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:01.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.169+0000 7fe562a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56210146e6d0 0x562101528490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53664/0 (socket says 172.21.15.33:53664) 2024-05-03T18:29:01.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.169+0000 7fe562a00700 1 -- 172.21.15.33:0/3137910969 learned_addr learned my addr 172.21.15.33:0/3137910969 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:29:01.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.169+0000 7fe562000700 1 --2- 172.21.15.33:0/3137910969 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5621014f5f30 0x5621015289d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:01.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.169+0000 7fe563400700 1 --2- 172.21.15.33:0/3137910969 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621014f68f0 0x562101528f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:01.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.169+0000 7fe562a00700 1 -- 172.21.15.33:0/3137910969 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5621014f5f30 msgr2=0x5621015289d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:29:01.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.169+0000 7fe562a00700 1 --2- 172.21.15.33:0/3137910969 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5621014f5f30 0x5621015289d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:01.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.169+0000 7fe562a00700 1 -- 172.21.15.33:0/3137910969 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621014f68f0 msgr2=0x562101528f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:29:01.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.169+0000 7fe562a00700 1 --2- 172.21.15.33:0/3137910969 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621014f68f0 0x562101528f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:01.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.169+0000 7fe562a00700 1 -- 172.21.15.33:0/3137910969 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56210158a860 con 0x56210146e6d0 2024-05-03T18:29:01.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.170+0000 7fe562a00700 1 --2- 172.21.15.33:0/3137910969 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56210146e6d0 0x562101528490 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fe5540a4d80 tx=0x7fe5540108d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b8d4c01a4be1107 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:29:01.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.170+0000 7fe54d600700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:29:01.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.170+0000 7fe54d600700 1 -- 172.21.15.33:0/3137910969 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe554020040 con 0x56210146e6d0 2024-05-03T18:29:01.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.170+0000 7fe54d600700 1 -- 172.21.15.33:0/3137910969 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7fe5540e2330 con 0x56210146e6d0 2024-05-03T18:29:01.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.170+0000 7fe5747c4b80 1 -- 172.21.15.33:0/3137910969 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe55c029070 con 0x56210146e6d0 2024-05-03T18:29:01.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.171+0000 7fe54d600700 1 -- 172.21.15.33:0/3137910969 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fe5540a9b80 con 0x56210146e6d0 2024-05-03T18:29:01.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.171+0000 7fe5747c4b80 10 client.15522 fetch_fsmap learned FSMap version 21 2024-05-03T18:29:01.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.171+0000 7fe5747c4b80 1 -- 172.21.15.33:0/3137910969 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x56210158bb50 con 0x56210146e6d0 2024-05-03T18:29:01.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.171+0000 7fe54d600700 1 -- 172.21.15.33:0/3137910969 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe55c029070 con 0x56210146e6d0 2024-05-03T18:29:01.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.171+0000 7fe54d600700 1 -- 172.21.15.33:0/3137910969 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fe5540ca2e0 con 0x56210146e6d0 2024-05-03T18:29:01.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.172+0000 7fe5747c4b80 10 client.15522 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:29:01.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.172+0000 7fe5747c4b80 10 client.15522 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:29:01.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.172+0000 7fe5747c4b80 20 client.15522 populate_metadata read hostname 'smithi033' 2024-05-03T18:29:01.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.172+0000 7fe5747c4b80 1 --2- 172.21.15.33:0/3137910969 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5621014fbbd0 0x56210158ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:01.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.172+0000 7fe5747c4b80 4 client.15522 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-03T18:29:01.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.172+0000 7fe5747c4b80 1 -- 172.21.15.33:0/3137910969 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5621014504a0 con 0x5621014fbbd0 2024-05-03T18:29:01.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.173+0000 7fe562000700 1 --2- 172.21.15.33:0/3137910969 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5621014fbbd0 0x56210158ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:01.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.174+0000 7fe54d600700 10 client.15522 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:29:01.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.174+0000 7fe562000700 1 --2- 172.21.15.33:0/3137910969 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5621014fbbd0 0x56210158ddf0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a6911c1c676795b server_cookie=8d79ec02c905dce2 in_seq=0 out_seq=0 2024-05-03T18:29:01.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.175+0000 7fe54d600700 1 -- 172.21.15.33:0/3137910969 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5621014504a0 con 0x5621014fbbd0 2024-05-03T18:29:01.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.175+0000 7fe54d600700 10 client.15522 handle_command_reply: tid=0 2024-05-03T18:29:01.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.175+0000 7fe5747c4b80 1 client.15522 shutdown 2024-05-03T18:29:01.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.175+0000 7fe5747c4b80 1 -- 172.21.15.33:0/3137910969 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56210146e6d0 msgr2=0x562101528490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:01.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.175+0000 7fe5747c4b80 1 --2- 172.21.15.33:0/3137910969 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56210146e6d0 0x562101528490 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fe5540a4d80 tx=0x7fe5540108d0 comp rx=0 tx=0).stop 2024-05-03T18:29:01.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.175+0000 7fe5747c4b80 1 -- 172.21.15.33:0/3137910969 shutdown_connections 2024-05-03T18:29:01.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.175+0000 7fe5747c4b80 1 --2- 172.21.15.33:0/3137910969 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5621014fbbd0 0x56210158ddf0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:01.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.175+0000 7fe5747c4b80 1 --2- 172.21.15.33:0/3137910969 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621014f68f0 0x562101528f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:01.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.175+0000 7fe5747c4b80 1 --2- 172.21.15.33:0/3137910969 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56210146e6d0 0x562101528490 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:01.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.175+0000 7fe5747c4b80 1 --2- 172.21.15.33:0/3137910969 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5621014f5f30 0x5621015289d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:01.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.175+0000 7fe54d600700 0 client.15522 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:29:01.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.175+0000 7fe5747c4b80 1 -- 172.21.15.33:0/3137910969 >> 172.21.15.33:0/3137910969 conn(0x5621014ea6c0 msgr2=0x5621014f06b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:29:01.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.176+0000 7fe5747c4b80 1 -- 172.21.15.33:0/3137910969 shutdown_connections 2024-05-03T18:29:01.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.176+0000 7fe5747c4b80 1 -- 172.21.15.33:0/3137910969 wait complete. 2024-05-03T18:29:01.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.176+0000 7fe5747c4b80 20 client.15522 trim_cache size 0 max 16384 2024-05-03T18:29:01.177 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-03T18:29:01.177 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-05-03T18:29:01.177 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-05-03T18:29:01.177 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-03T18:29:01.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.177+0000 7fe563e00700 1 -- 172.21.15.33:0/3364795419 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fe5480b2a70 msgr2=0x7fe5480b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:01.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.177+0000 7fe563e00700 1 --2- 172.21.15.33:0/3364795419 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fe5480b2a70 0x7fe5480b4f30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe55c009690 tx=0x7fe55c004ad0 comp rx=0 tx=0).stop 2024-05-03T18:29:01.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.177+0000 7fe563e00700 1 -- 172.21.15.33:0/3364795419 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5641033d0 msgr2=0x7fe5640757a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:01.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.177+0000 7fe563e00700 1 --2- 172.21.15.33:0/3364795419 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5641033d0 0x7fe5640757a0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fe55400d950 tx=0x7fe55400dc60 comp rx=0 tx=0).stop 2024-05-03T18:29:01.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.177+0000 7fe563e00700 1 -- 172.21.15.33:0/3364795419 shutdown_connections 2024-05-03T18:29:01.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.177+0000 7fe563e00700 1 --2- 172.21.15.33:0/3364795419 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe564109460 0x7fe564076220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:01.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.177+0000 7fe563e00700 1 --2- 172.21.15.33:0/3364795419 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5641033d0 0x7fe5640757a0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:01.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.177+0000 7fe563e00700 1 --2- 172.21.15.33:0/3364795419 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fe5480b2a70 0x7fe5480b4f30 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:01.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.177+0000 7fe563e00700 1 --2- 172.21.15.33:0/3364795419 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe564103d90 0x7fe564075ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:01.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.177+0000 7fe563e00700 1 -- 172.21.15.33:0/3364795419 >> 172.21.15.33:0/3364795419 conn(0x7fe5640fef20 msgr2=0x7fe5640ff690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:29:01.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.178+0000 7fe563e00700 1 -- 172.21.15.33:0/3364795419 shutdown_connections 2024-05-03T18:29:01.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:01.178+0000 7fe563e00700 1 -- 172.21.15.33:0/3364795419 wait complete. 2024-05-03T18:29:01.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:29:01 smithi033 ceph-mon[28330]: scrub summary: idle 2024-05-03T18:29:01.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:29:01 smithi099 ceph-mon[31658]: scrub summary: idle 2024-05-03T18:29:01.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:29:01 smithi150 ceph-mon[32492]: scrub summary: idle 2024-05-03T18:29:02.152 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-03T18:29:02.152 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T18:29:02.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:29:02 smithi033 ceph-mon[28330]: pgmap v58: 129 pgs: 129 active+clean; 278 MiB data, 969 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 MiB/s wr, 11 op/s 2024-05-03T18:29:02.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:29:02 smithi099 ceph-mon[31658]: pgmap v58: 129 pgs: 129 active+clean; 278 MiB data, 969 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 MiB/s wr, 11 op/s 2024-05-03T18:29:02.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:29:02 smithi150 ceph-mon[32492]: pgmap v58: 129 pgs: 129 active+clean; 278 MiB data, 969 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 MiB/s wr, 11 op/s 2024-05-03T18:29:02.664 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:29:03.478 INFO:teuthology.orchestra.run.smithi033.stderr:1100000 of 3200000 tuples (34%) done (elapsed 40.58 s, remaining 77.48 s) 2024-05-03T18:29:04.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:04.949+0000 7faf42000700 1 -- 172.21.15.33:0/2937962196 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf3c108850 msgr2=0x7faf3c1103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:04.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:04.949+0000 7faf42000700 1 --2- 172.21.15.33:0/2937962196 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf3c108850 0x7faf3c1103d0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7faf3400b0d0 tx=0x7faf3400b3e0 comp rx=0 tx=0).stop 2024-05-03T18:29:04.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:04.949+0000 7faf42000700 1 -- 172.21.15.33:0/2937962196 shutdown_connections 2024-05-03T18:29:04.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:04.949+0000 7faf42000700 1 --2- 172.21.15.33:0/2937962196 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf3c108850 0x7faf3c1103d0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:04.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:04.949+0000 7faf42000700 1 --2- 172.21.15.33:0/2937962196 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf3c103750 0x7faf3c103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:04.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:04.949+0000 7faf42000700 1 --2- 172.21.15.33:0/2937962196 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf3c104100 0x7faf3c108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:04.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:04.949+0000 7faf42000700 1 -- 172.21.15.33:0/2937962196 >> 172.21.15.33:0/2937962196 conn(0x7faf3c07a600 msgr2=0x7faf3c07aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:29:04.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:04.949+0000 7faf42000700 1 -- 172.21.15.33:0/2937962196 shutdown_connections 2024-05-03T18:29:04.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:04.949+0000 7faf42000700 1 -- 172.21.15.33:0/2937962196 wait complete. 2024-05-03T18:29:04.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:04.951+0000 7faf42000700 1 Processor -- start 2024-05-03T18:29:04.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:04.951+0000 7faf42000700 1 -- start start 2024-05-03T18:29:04.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:04.951+0000 7faf42000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf3c103750 0x7faf3c1a1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:04.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:04.951+0000 7faf42000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf3c104100 0x7faf3c1a1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:04.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:04.951+0000 7faf42000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf3c108850 0x7faf3c19ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:04.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:04.951+0000 7faf42000700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7faf3c113540 con 0x7faf3c104100 2024-05-03T18:29:04.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:04.951+0000 7faf42000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7faf3c1133c0 con 0x7faf3c103750 2024-05-03T18:29:04.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:04.951+0000 7faf42000700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7faf3c1136c0 con 0x7faf3c108850 2024-05-03T18:29:04.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:04.952+0000 7faf3aa00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf3c104100 0x7faf3c1a1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:04.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:04.952+0000 7faf3aa00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf3c104100 0x7faf3c1a1d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53676/0 (socket says 172.21.15.33:53676) 2024-05-03T18:29:04.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:04.952+0000 7faf3aa00700 1 -- 172.21.15.33:0/1322253381 learned_addr learned my addr 172.21.15.33:0/1322253381 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:29:04.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:04.952+0000 7faf3b400700 1 --2- 172.21.15.33:0/1322253381 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf3c103750 0x7faf3c1a1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:04.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:04.952+0000 7faf3be00700 1 --2- 172.21.15.33:0/1322253381 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf3c108850 0x7faf3c19ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:04.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:04.952+0000 7faf3aa00700 1 -- 172.21.15.33:0/1322253381 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf3c108850 msgr2=0x7faf3c19ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:04.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:04.952+0000 7faf3aa00700 1 --2- 172.21.15.33:0/1322253381 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf3c108850 0x7faf3c19ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:04.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:04.952+0000 7faf3aa00700 1 -- 172.21.15.33:0/1322253381 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf3c103750 msgr2=0x7faf3c1a1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:04.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:04.952+0000 7faf3aa00700 1 --2- 172.21.15.33:0/1322253381 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf3c103750 0x7faf3c1a1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:04.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:04.952+0000 7faf3aa00700 1 -- 172.21.15.33:0/1322253381 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf3c19c170 con 0x7faf3c104100 2024-05-03T18:29:04.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:04.952+0000 7faf3be00700 1 --2- 172.21.15.33:0/1322253381 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf3c108850 0x7faf3c19ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:29:04.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:04.953+0000 7faf3aa00700 1 --2- 172.21.15.33:0/1322253381 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf3c104100 0x7faf3c1a1d90 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7faf2800d980 tx=0x7faf2800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:29:04.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:04.953+0000 7faf33e00700 1 -- 172.21.15.33:0/1322253381 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf28017070 con 0x7faf3c104100 2024-05-03T18:29:04.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:04.953+0000 7faf33e00700 1 -- 172.21.15.33:0/1322253381 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faf28004830 con 0x7faf3c104100 2024-05-03T18:29:04.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:04.953+0000 7faf42000700 1 -- 172.21.15.33:0/1322253381 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf3c19c460 con 0x7faf3c104100 2024-05-03T18:29:04.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:04.953+0000 7faf42000700 1 -- 172.21.15.33:0/1322253381 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf3c1a8720 con 0x7faf3c104100 2024-05-03T18:29:04.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:04.953+0000 7faf33e00700 1 -- 172.21.15.33:0/1322253381 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf280053a0 con 0x7faf3c104100 2024-05-03T18:29:04.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:04.955+0000 7faf42000700 1 -- 172.21.15.33:0/1322253381 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf3c06a190 con 0x7faf3c104100 2024-05-03T18:29:04.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:04.959+0000 7faf33e00700 1 -- 172.21.15.33:0/1322253381 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7faf280049e0 con 0x7faf3c104100 2024-05-03T18:29:04.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:04.960+0000 7faf33e00700 1 --2- 172.21.15.33:0/1322253381 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7faf200b29f0 0x7faf200b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:04.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:04.960+0000 7faf33e00700 1 -- 172.21.15.33:0/1322253381 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7faf280e2da0 con 0x7faf3c104100 2024-05-03T18:29:04.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:04.961+0000 7faf3b400700 1 --2- 172.21.15.33:0/1322253381 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7faf200b29f0 0x7faf200b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:04.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:04.961+0000 7faf3b400700 1 --2- 172.21.15.33:0/1322253381 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7faf200b29f0 0x7faf200b4eb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7faf2c005950 tx=0x7faf2c0058e0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:29:04.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:04.962+0000 7faf33e00700 1 -- 172.21.15.33:0/1322253381 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7faf28020020 con 0x7faf3c104100 2024-05-03T18:29:05.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:29:04 smithi150 ceph-mon[32492]: pgmap v59: 129 pgs: 129 active+clean; 325 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 17 MiB/s wr, 32 op/s 2024-05-03T18:29:05.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:29:04 smithi033 ceph-mon[28330]: pgmap v59: 129 pgs: 129 active+clean; 325 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 17 MiB/s wr, 32 op/s 2024-05-03T18:29:05.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:05.287+0000 7faf42000700 1 -- 172.21.15.33:0/1322253381 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faf3c1a8950 con 0x7faf3c104100 2024-05-03T18:29:05.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:05.289+0000 7faf33e00700 1 -- 172.21.15.33:0/1322253381 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11801 (secure 0 0 0) 0x7faf280aa390 con 0x7faf3c104100 2024-05-03T18:29:05.290 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:29:05.290 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1327393503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1327393503},{"type":"v1","addr":"172.21.15.33:6835","nonce":1327393503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/2461472948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":2461472948},{"type":"v1","addr":"172.21.15.33:6837","nonce":2461472948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14559,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/442269281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":442269281},{"type":"v1","addr":"172.21.15.33:6839","nonce":442269281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14571,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/645571756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":645571756},{"type":"v1","addr":"172.21.15.33:6841","nonce":645571756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/203598909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":203598909},{"type":"v1","addr":"172.21.15.150:6833","nonce":203598909}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/4029842493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":4029842493},{"type":"v1","addr":"172.21.15.150:6835","nonce":4029842493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3463664423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3463664423},{"type":"v1","addr":"172.21.15.150:6837","nonce":3463664423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24481,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6839/4051648018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":4051648018},{"type":"v1","addr":"172.21.15.150:6839","nonce":4051648018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34418,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1865105479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1865105479},{"type":"v1","addr":"172.21.15.99:6833","nonce":1865105479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34445,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/1206393914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1206393914},{"type":"v1","addr":"172.21.15.99:6835","nonce":1206393914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-03T18:22:28.837091+0000","modified":"2024-05-03T18:23:49.758398+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34508},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34472":{"gid":34472,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6837/3360970342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3360970342},{"type":"v1","addr":"172.21.15.99:6837","nonce":3360970342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34508":{"gid":34508,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/919907308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":919907308},{"type":"v1","addr":"172.21.15.99:6839","nonce":919907308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34508,"qdb_cluster":[34508]},"id":1}]} 2024-05-03T18:29:05.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:05.293+0000 7faf42000700 1 -- 172.21.15.33:0/1322253381 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7faf200b29f0 msgr2=0x7faf200b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:05.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:05.293+0000 7faf42000700 1 --2- 172.21.15.33:0/1322253381 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7faf200b29f0 0x7faf200b4eb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7faf2c005950 tx=0x7faf2c0058e0 comp rx=0 tx=0).stop 2024-05-03T18:29:05.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:05.293+0000 7faf42000700 1 -- 172.21.15.33:0/1322253381 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf3c104100 msgr2=0x7faf3c1a1d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:05.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:05.293+0000 7faf42000700 1 --2- 172.21.15.33:0/1322253381 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf3c104100 0x7faf3c1a1d90 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7faf2800d980 tx=0x7faf2800dd40 comp rx=0 tx=0).stop 2024-05-03T18:29:05.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:05.293+0000 7faf42000700 1 -- 172.21.15.33:0/1322253381 shutdown_connections 2024-05-03T18:29:05.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:05.293+0000 7faf42000700 1 --2- 172.21.15.33:0/1322253381 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf3c108850 0x7faf3c19ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:05.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:05.293+0000 7faf42000700 1 --2- 172.21.15.33:0/1322253381 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7faf200b29f0 0x7faf200b4eb0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:05.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:05.293+0000 7faf42000700 1 --2- 172.21.15.33:0/1322253381 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf3c103750 0x7faf3c1a1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:05.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:05.293+0000 7faf42000700 1 --2- 172.21.15.33:0/1322253381 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf3c104100 0x7faf3c1a1d90 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:05.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:05.293+0000 7faf42000700 1 -- 172.21.15.33:0/1322253381 >> 172.21.15.33:0/1322253381 conn(0x7faf3c07a600 msgr2=0x7faf3c101500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:29:05.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:05.294+0000 7faf42000700 1 -- 172.21.15.33:0/1322253381 shutdown_connections 2024-05-03T18:29:05.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:05.294+0000 7faf42000700 1 -- 172.21.15.33:0/1322253381 wait complete. 2024-05-03T18:29:05.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:29:04 smithi099 ceph-mon[31658]: pgmap v59: 129 pgs: 129 active+clean; 325 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 17 MiB/s wr, 32 op/s 2024-05-03T18:29:05.295 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 21 2024-05-03T18:29:05.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:29:05 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/1322253381' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:29:06.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:29:05 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/1322253381' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:29:06.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:29:05 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/1322253381' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:29:07.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:29:06 smithi033 ceph-mon[28330]: pgmap v60: 129 pgs: 129 active+clean; 325 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 11 MiB/s wr, 26 op/s 2024-05-03T18:29:07.242 INFO:teuthology.orchestra.run.smithi033.stderr:1200000 of 3200000 tuples (37%) done (elapsed 44.35 s, remaining 73.92 s) 2024-05-03T18:29:07.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:29:06 smithi099 ceph-mon[31658]: pgmap v60: 129 pgs: 129 active+clean; 325 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 11 MiB/s wr, 26 op/s 2024-05-03T18:29:07.349 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-03T18:29:07.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:29:06 smithi150 ceph-mon[32492]: pgmap v60: 129 pgs: 129 active+clean; 325 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 11 MiB/s wr, 26 op/s 2024-05-03T18:29:07.905 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:29:09.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:29:08 smithi033 ceph-mon[28330]: pgmap v61: 129 pgs: 129 active+clean; 352 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 16 MiB/s wr, 30 op/s 2024-05-03T18:29:09.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:29:08 smithi099 ceph-mon[31658]: pgmap v61: 129 pgs: 129 active+clean; 352 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 16 MiB/s wr, 30 op/s 2024-05-03T18:29:09.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:29:08 smithi150 ceph-mon[32492]: pgmap v61: 129 pgs: 129 active+clean; 352 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 16 MiB/s wr, 30 op/s 2024-05-03T18:29:10.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.533+0000 7feb98c00700 1 -- 172.21.15.33:0/3299685772 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb94103eb0 msgr2=0x7feb94110390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:10.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.533+0000 7feb98c00700 1 --2- 172.21.15.33:0/3299685772 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb94103eb0 0x7feb94110390 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7feb8c00b0a0 tx=0x7feb8c00b3b0 comp rx=0 tx=0).stop 2024-05-03T18:29:10.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.533+0000 7feb98c00700 1 -- 172.21.15.33:0/3299685772 shutdown_connections 2024-05-03T18:29:10.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.533+0000 7feb98c00700 1 --2- 172.21.15.33:0/3299685772 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb94103eb0 0x7feb94110390 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:10.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.533+0000 7feb98c00700 1 --2- 172.21.15.33:0/3299685772 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb94109600 0x7feb941099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:10.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.533+0000 7feb98c00700 1 --2- 172.21.15.33:0/3299685772 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb941034f0 0x7feb94103970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:10.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.533+0000 7feb98c00700 1 -- 172.21.15.33:0/3299685772 >> 172.21.15.33:0/3299685772 conn(0x7feb9407a650 msgr2=0x7feb9407aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:29:10.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.533+0000 7feb98c00700 1 -- 172.21.15.33:0/3299685772 shutdown_connections 2024-05-03T18:29:10.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.533+0000 7feb98c00700 1 -- 172.21.15.33:0/3299685772 wait complete. 2024-05-03T18:29:10.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.534+0000 7feb98c00700 1 Processor -- start 2024-05-03T18:29:10.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.535+0000 7feb98c00700 1 -- start start 2024-05-03T18:29:10.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.535+0000 7feb98c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb941034f0 0x7feb941a1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:10.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.535+0000 7feb98c00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb94103eb0 0x7feb941a1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:10.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.535+0000 7feb98c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb94109600 0x7feb9419b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:10.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.535+0000 7feb98c00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7feb94113500 con 0x7feb94103eb0 2024-05-03T18:29:10.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.535+0000 7feb98c00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7feb94113380 con 0x7feb94109600 2024-05-03T18:29:10.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.535+0000 7feb98c00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7feb94113680 con 0x7feb941034f0 2024-05-03T18:29:10.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.536+0000 7feb91400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb94103eb0 0x7feb941a1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:10.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.536+0000 7feb91400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb94103eb0 0x7feb941a1cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54710/0 (socket says 172.21.15.33:54710) 2024-05-03T18:29:10.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.536+0000 7feb91400700 1 -- 172.21.15.33:0/2922285046 learned_addr learned my addr 172.21.15.33:0/2922285046 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:29:10.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.536+0000 7feb91e00700 1 --2- 172.21.15.33:0/2922285046 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb941034f0 0x7feb941a1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:10.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.536+0000 7feb91400700 1 -- 172.21.15.33:0/2922285046 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb941034f0 msgr2=0x7feb941a1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:10.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.536+0000 7feb91400700 1 --2- 172.21.15.33:0/2922285046 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb941034f0 0x7feb941a1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:10.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.536+0000 7feb91400700 1 -- 172.21.15.33:0/2922285046 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb94109600 msgr2=0x7feb9419b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:10.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.536+0000 7feb91400700 1 --2- 172.21.15.33:0/2922285046 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb94109600 0x7feb9419b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:10.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.536+0000 7feb91400700 1 -- 172.21.15.33:0/2922285046 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb9419c0a0 con 0x7feb94103eb0 2024-05-03T18:29:10.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.537+0000 7feb91400700 1 --2- 172.21.15.33:0/2922285046 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb94103eb0 0x7feb941a1cc0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7feb8000d950 tx=0x7feb8000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:29:10.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.537+0000 7feb8aa00700 1 -- 172.21.15.33:0/2922285046 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb80016070 con 0x7feb94103eb0 2024-05-03T18:29:10.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.537+0000 7feb98c00700 1 -- 172.21.15.33:0/2922285046 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb9419c390 con 0x7feb94103eb0 2024-05-03T18:29:10.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.537+0000 7feb8aa00700 1 -- 172.21.15.33:0/2922285046 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7feb80010460 con 0x7feb94103eb0 2024-05-03T18:29:10.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.537+0000 7feb8aa00700 1 -- 172.21.15.33:0/2922285046 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb8000f620 con 0x7feb94103eb0 2024-05-03T18:29:10.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.537+0000 7feb98c00700 1 -- 172.21.15.33:0/2922285046 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb941a8650 con 0x7feb94103eb0 2024-05-03T18:29:10.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.542+0000 7feb8aa00700 1 -- 172.21.15.33:0/2922285046 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7feb80019070 con 0x7feb94103eb0 2024-05-03T18:29:10.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.543+0000 7feba3606b80 1 Processor -- start 2024-05-03T18:29:10.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.543+0000 7feba3606b80 1 -- start start 2024-05-03T18:29:10.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.543+0000 7feba3606b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56188c53e930 0x56188c53ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:10.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.543+0000 7feba3606b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56188c53f250 0x56188c538f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:10.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.543+0000 7feba3606b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56188c539550 0x56188c5399d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:10.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.543+0000 7feba3606b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56188c48ba20 con 0x56188c539550 2024-05-03T18:29:10.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.543+0000 7feba3606b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56188c429760 con 0x56188c53e930 2024-05-03T18:29:10.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.543+0000 7feba3606b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56188c4879e0 con 0x56188c53f250 2024-05-03T18:29:10.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.544+0000 7feb91400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56188c539550 0x56188c5399d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:10.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.544+0000 7feb91400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56188c539550 0x56188c5399d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54726/0 (socket says 172.21.15.33:54726) 2024-05-03T18:29:10.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.544+0000 7feb92800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56188c53e930 0x56188c53ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:10.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.544+0000 7feb91e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56188c53f250 0x56188c538f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:10.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.544+0000 7feb91400700 1 -- 172.21.15.33:0/4084587104 learned_addr learned my addr 172.21.15.33:0/4084587104 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:29:10.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.545+0000 7feb8aa00700 1 --2- 172.21.15.33:0/2922285046 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7feb780b29f0 0x7feb780b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:10.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.545+0000 7feb91e00700 1 -- 172.21.15.33:0/4084587104 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56188c53e930 msgr2=0x56188c53ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:29:10.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.545+0000 7feb91e00700 1 --2- 172.21.15.33:0/4084587104 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56188c53e930 0x56188c53ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:10.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.545+0000 7feb91e00700 1 -- 172.21.15.33:0/4084587104 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56188c539550 msgr2=0x56188c5399d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:29:10.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.545+0000 7feb91e00700 1 --2- 172.21.15.33:0/4084587104 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56188c539550 0x56188c5399d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:10.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.545+0000 7feb91e00700 1 -- 172.21.15.33:0/4084587104 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56188c3d88d0 con 0x56188c53f250 2024-05-03T18:29:10.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.545+0000 7feb91400700 1 --2- 172.21.15.33:0/4084587104 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56188c539550 0x56188c5399d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:29:10.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.545+0000 7feb8aa00700 1 -- 172.21.15.33:0/2922285046 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7feb800e25f0 con 0x7feb94103eb0 2024-05-03T18:29:10.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.545+0000 7feb92800700 1 --2- 172.21.15.33:0/2922285046 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7feb780b29f0 0x7feb780b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:10.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.546+0000 7feb91e00700 1 --2- 172.21.15.33:0/4084587104 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56188c53f250 0x56188c538f30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7feb84008b40 tx=0x7feb84008e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=935af093a16767d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:29:10.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.546+0000 7feb92800700 1 --2- 172.21.15.33:0/2922285046 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7feb780b29f0 0x7feb780b4eb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7feb8c009fd0 tx=0x7feb8c0099e0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:29:10.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.546+0000 7feb7f400700 1 -- 172.21.15.33:0/4084587104 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb84014070 con 0x56188c53f250 2024-05-03T18:29:10.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.546+0000 7feb7f400700 1 -- 172.21.15.33:0/4084587104 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7feb840044c0 con 0x56188c53f250 2024-05-03T18:29:10.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.546+0000 7feb7f400700 1 -- 172.21.15.33:0/4084587104 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb84004e60 con 0x56188c53f250 2024-05-03T18:29:10.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.547+0000 7feba3606b80 1 -- 172.21.15.33:0/4084587104 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56188c53f250 msgr2=0x56188c538f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:10.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.547+0000 7feba3606b80 1 --2- 172.21.15.33:0/4084587104 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56188c53f250 0x56188c538f30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7feb84008b40 tx=0x7feb84008e50 comp rx=0 tx=0).stop 2024-05-03T18:29:10.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.547+0000 7feba3606b80 1 -- 172.21.15.33:0/4084587104 shutdown_connections 2024-05-03T18:29:10.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.547+0000 7feba3606b80 1 --2- 172.21.15.33:0/4084587104 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56188c539550 0x56188c5399d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:10.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.547+0000 7feba3606b80 1 --2- 172.21.15.33:0/4084587104 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56188c53e930 0x56188c53ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:10.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.547+0000 7feba3606b80 1 --2- 172.21.15.33:0/4084587104 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56188c53f250 0x56188c538f30 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:10.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.547+0000 7feba3606b80 1 -- 172.21.15.33:0/4084587104 >> 172.21.15.33:0/4084587104 conn(0x56188c4b16d0 msgr2=0x56188c533900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:29:10.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.548+0000 7feba3606b80 1 -- 172.21.15.33:0/4084587104 shutdown_connections 2024-05-03T18:29:10.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.548+0000 7feba3606b80 1 -- 172.21.15.33:0/4084587104 wait complete. 2024-05-03T18:29:10.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.548+0000 7feba3606b80 1 -- start start 2024-05-03T18:29:10.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.548+0000 7feba3606b80 1 Processor -- start 2024-05-03T18:29:10.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.549+0000 7feba3606b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56188c539550 0x56188c544e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:10.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.549+0000 7feba3606b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56188c53e930 0x56188c5421a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:10.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.549+0000 7feba3606b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56188c53f250 0x56188c5426e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:10.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.549+0000 7feba3606b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56188c545d00 con 0x56188c539550 2024-05-03T18:29:10.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.549+0000 7feba3606b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56188c3f1b20 con 0x56188c53f250 2024-05-03T18:29:10.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.549+0000 7feba3606b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56188c545e80 con 0x56188c53e930 2024-05-03T18:29:10.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.549+0000 7feba3606b80 10 client.0 _ll_register_callbacks cb 0x56188c432c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:29:10.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.550+0000 7feb91e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56188c539550 0x56188c544e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:10.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.550+0000 7feb91400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56188c53e930 0x56188c5421a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:10.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.550+0000 7feb92800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56188c53f250 0x56188c5426e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:10.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.550+0000 7feb91400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56188c53e930 0x56188c5421a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:54034/0 (socket says 172.21.15.33:54034) 2024-05-03T18:29:10.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.550+0000 7feb91400700 1 -- 172.21.15.33:0/3439229340 learned_addr learned my addr 172.21.15.33:0/3439229340 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:29:10.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.550+0000 7feb91400700 1 -- 172.21.15.33:0/3439229340 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56188c53f250 msgr2=0x56188c5426e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:29:10.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.550+0000 7feb91400700 1 --2- 172.21.15.33:0/3439229340 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56188c53f250 0x56188c5426e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:10.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.550+0000 7feb91400700 1 -- 172.21.15.33:0/3439229340 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56188c539550 msgr2=0x56188c544e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:29:10.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.550+0000 7feb91400700 1 --2- 172.21.15.33:0/3439229340 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56188c539550 0x56188c544e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:10.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.550+0000 7feb91400700 1 -- 172.21.15.33:0/3439229340 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56188c451420 con 0x56188c53e930 2024-05-03T18:29:10.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.551+0000 7feb91400700 1 --2- 172.21.15.33:0/3439229340 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56188c53e930 0x56188c5421a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7feb800a9240 tx=0x7feb80010890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7f8d477a2a9eea7 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:29:10.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.551+0000 7feb73e00700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-03T18:29:10.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.551+0000 7feb73e00700 1 -- 172.21.15.33:0/3439229340 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb80020030 con 0x56188c53e930 2024-05-03T18:29:10.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.551+0000 7feba3606b80 1 -- 172.21.15.33:0/3439229340 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb840044c0 con 0x56188c53e930 2024-05-03T18:29:10.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.551+0000 7feb73e00700 1 -- 172.21.15.33:0/3439229340 <== mon.1 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7feb800e23f0 con 0x56188c53e930 2024-05-03T18:29:10.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.552+0000 7feb73e00700 1 -- 172.21.15.33:0/3439229340 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7feb800a8e90 con 0x56188c53e930 2024-05-03T18:29:10.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.552+0000 7feba3606b80 10 client.25234 fetch_fsmap learned FSMap version 21 2024-05-03T18:29:10.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.552+0000 7feba3606b80 1 -- 172.21.15.33:0/3439229340 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x56188c5504f0 con 0x56188c53e930 2024-05-03T18:29:10.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.552+0000 7feb73e00700 1 -- 172.21.15.33:0/3439229340 <== mon.1 v2:172.21.15.150:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7feb840044c0 con 0x56188c53e930 2024-05-03T18:29:10.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.553+0000 7feb73e00700 1 -- 172.21.15.33:0/3439229340 <== mon.1 v2:172.21.15.150:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7feb800a97a0 con 0x56188c53e930 2024-05-03T18:29:10.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.553+0000 7feba3606b80 10 client.25234 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:29:10.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.553+0000 7feba3606b80 10 client.25234 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:29:10.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.553+0000 7feba3606b80 20 client.25234 populate_metadata read hostname 'smithi033' 2024-05-03T18:29:10.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.553+0000 7feba3606b80 1 --2- 172.21.15.33:0/3439229340 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x56188c552b80 0x56188c554fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:10.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.553+0000 7feba3606b80 4 client.25234 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:29:10.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.553+0000 7feba3606b80 1 -- 172.21.15.33:0/3439229340 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56188c505ff0 con 0x56188c552b80 2024-05-03T18:29:10.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.554+0000 7feb91e00700 1 --2- 172.21.15.33:0/3439229340 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x56188c552b80 0x56188c554fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:10.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.555+0000 7feb91e00700 1 --2- 172.21.15.33:0/3439229340 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x56188c552b80 0x56188c554fd0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3bcbcaa9919e53c7 server_cookie=7a1d8d72ebee81bf in_seq=0 out_seq=0 2024-05-03T18:29:10.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.555+0000 7feb73e00700 10 client.25234 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:29:10.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.557+0000 7feb73e00700 1 -- 172.21.15.33:0/3439229340 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56188c505ff0 con 0x56188c552b80 2024-05-03T18:29:10.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.557+0000 7feb73e00700 10 client.25234 handle_command_reply: tid=0 2024-05-03T18:29:10.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.557+0000 7feba3606b80 1 client.25234 shutdown 2024-05-03T18:29:10.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.557+0000 7feba3606b80 1 -- 172.21.15.33:0/3439229340 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56188c53e930 msgr2=0x56188c5421a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:10.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.557+0000 7feba3606b80 1 --2- 172.21.15.33:0/3439229340 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56188c53e930 0x56188c5421a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7feb800a9240 tx=0x7feb80010890 comp rx=0 tx=0).stop 2024-05-03T18:29:10.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.557+0000 7feba3606b80 1 -- 172.21.15.33:0/3439229340 shutdown_connections 2024-05-03T18:29:10.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.557+0000 7feba3606b80 1 --2- 172.21.15.33:0/3439229340 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x56188c552b80 0x56188c554fd0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:10.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.557+0000 7feba3606b80 1 --2- 172.21.15.33:0/3439229340 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56188c53f250 0x56188c5426e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:10.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.557+0000 7feba3606b80 1 --2- 172.21.15.33:0/3439229340 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56188c539550 0x56188c544e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:10.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.557+0000 7feb73e00700 0 client.25234 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:29:10.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.557+0000 7feba3606b80 1 --2- 172.21.15.33:0/3439229340 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56188c53e930 0x56188c5421a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:10.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.557+0000 7feba3606b80 1 -- 172.21.15.33:0/3439229340 >> 172.21.15.33:0/3439229340 conn(0x56188c4b16d0 msgr2=0x56188c52e3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:29:10.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.557+0000 7feba3606b80 1 -- 172.21.15.33:0/3439229340 shutdown_connections 2024-05-03T18:29:10.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.557+0000 7feba3606b80 1 -- 172.21.15.33:0/3439229340 wait complete. 2024-05-03T18:29:10.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.558+0000 7feba3606b80 20 client.25234 trim_cache size 0 max 16384 2024-05-03T18:29:10.574 INFO:teuthology.orchestra.run.smithi033.stderr:1300000 of 3200000 tuples (40%) done (elapsed 47.68 s, remaining 69.69 s) 2024-05-03T18:29:10.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.577+0000 7feba3606b80 1 Processor -- start 2024-05-03T18:29:10.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.577+0000 7feba3606b80 1 -- start start 2024-05-03T18:29:10.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.577+0000 7feba3606b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56188c539550 0x56188c54edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:10.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.577+0000 7feba3606b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56188c53e930 0x56188c55a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:10.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.577+0000 7feba3606b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56188c53f250 0x56188c55ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:10.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.577+0000 7feba3606b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7feb941af4e0 con 0x56188c539550 2024-05-03T18:29:10.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.577+0000 7feba3606b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7feb941b1d60 con 0x56188c53f250 2024-05-03T18:29:10.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.577+0000 7feba3606b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7feb941afb50 con 0x56188c53e930 2024-05-03T18:29:10.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.578+0000 7feb91e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56188c539550 0x56188c54edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:10.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.578+0000 7feb91e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56188c539550 0x56188c54edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54752/0 (socket says 172.21.15.33:54752) 2024-05-03T18:29:10.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.578+0000 7feb91e00700 1 -- 172.21.15.33:0/518945735 learned_addr learned my addr 172.21.15.33:0/518945735 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:29:10.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.578+0000 7feb91400700 1 --2- 172.21.15.33:0/518945735 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56188c53e930 0x56188c55a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:10.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.578+0000 7feb92800700 1 --2- 172.21.15.33:0/518945735 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56188c53f250 0x56188c55ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:10.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.578+0000 7feb91e00700 1 -- 172.21.15.33:0/518945735 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56188c53e930 msgr2=0x56188c55a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:29:10.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.578+0000 7feb91e00700 1 --2- 172.21.15.33:0/518945735 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56188c53e930 0x56188c55a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:10.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.578+0000 7feb91e00700 1 -- 172.21.15.33:0/518945735 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56188c53f250 msgr2=0x56188c55ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:29:10.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.578+0000 7feb91e00700 1 --2- 172.21.15.33:0/518945735 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56188c53f250 0x56188c55ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:10.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.578+0000 7feb91e00700 1 -- 172.21.15.33:0/518945735 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56188c555650 con 0x56188c539550 2024-05-03T18:29:10.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.579+0000 7feb91e00700 1 --2- 172.21.15.33:0/518945735 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56188c539550 0x56188c54edf0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7feb84005950 tx=0x7feb840058e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82e5bbab6573123d server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:29:10.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.579+0000 7feb7ea00700 1 -- 172.21.15.33:0/518945735 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb8402c070 con 0x56188c539550 2024-05-03T18:29:10.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.579+0000 7feb7ea00700 1 -- 172.21.15.33:0/518945735 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7feb84027560 con 0x56188c539550 2024-05-03T18:29:10.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.579+0000 7feb7ea00700 1 -- 172.21.15.33:0/518945735 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb84014040 con 0x56188c539550 2024-05-03T18:29:10.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.580+0000 7feba3606b80 1 -- 172.21.15.33:0/518945735 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56188c539550 msgr2=0x56188c54edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:10.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.580+0000 7feba3606b80 1 --2- 172.21.15.33:0/518945735 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56188c539550 0x56188c54edf0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7feb84005950 tx=0x7feb840058e0 comp rx=0 tx=0).stop 2024-05-03T18:29:10.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.580+0000 7feba3606b80 1 -- 172.21.15.33:0/518945735 shutdown_connections 2024-05-03T18:29:10.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.580+0000 7feba3606b80 1 --2- 172.21.15.33:0/518945735 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56188c53f250 0x56188c55ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:10.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.580+0000 7feba3606b80 1 --2- 172.21.15.33:0/518945735 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56188c539550 0x56188c54edf0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:10.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.580+0000 7feba3606b80 1 --2- 172.21.15.33:0/518945735 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56188c53e930 0x56188c55a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:10.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.580+0000 7feba3606b80 1 -- 172.21.15.33:0/518945735 >> 172.21.15.33:0/518945735 conn(0x56188c4b16d0 msgr2=0x56188c540a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:29:10.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.580+0000 7feba3606b80 1 -- 172.21.15.33:0/518945735 shutdown_connections 2024-05-03T18:29:10.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.580+0000 7feba3606b80 1 -- 172.21.15.33:0/518945735 wait complete. 2024-05-03T18:29:10.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.581+0000 7feba3606b80 1 -- start start 2024-05-03T18:29:10.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.581+0000 7feba3606b80 1 Processor -- start 2024-05-03T18:29:10.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.581+0000 7feba3606b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56188c539550 0x56188c559ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:10.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.581+0000 7feba3606b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56188c53e930 0x56188c55a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:10.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.581+0000 7feba3606b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56188c53f250 0x56188c55a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:10.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.582+0000 7feba3606b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7feb941b0220 con 0x56188c53f250 2024-05-03T18:29:10.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.582+0000 7feba3606b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7feb941b1690 con 0x56188c539550 2024-05-03T18:29:10.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.582+0000 7feba3606b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7feb941b08f0 con 0x56188c53e930 2024-05-03T18:29:10.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.582+0000 7feba3606b80 10 client.0 _ll_register_callbacks cb 0x56188c432c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:29:10.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.582+0000 7feb92800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56188c53f250 0x56188c55a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:10.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.582+0000 7feb92800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56188c53f250 0x56188c55a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54766/0 (socket says 172.21.15.33:54766) 2024-05-03T18:29:10.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.582+0000 7feb92800700 1 -- 172.21.15.33:0/3164411961 learned_addr learned my addr 172.21.15.33:0/3164411961 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:29:10.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.582+0000 7feb91e00700 1 --2- 172.21.15.33:0/3164411961 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56188c539550 0x56188c559ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:10.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.582+0000 7feb92800700 1 -- 172.21.15.33:0/3164411961 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56188c53e930 msgr2=0x56188c55a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:29:10.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.582+0000 7feb92800700 1 --2- 172.21.15.33:0/3164411961 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56188c53e930 0x56188c55a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:10.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.582+0000 7feb92800700 1 -- 172.21.15.33:0/3164411961 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56188c539550 msgr2=0x56188c559ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:29:10.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.583+0000 7feb92800700 1 --2- 172.21.15.33:0/3164411961 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56188c539550 0x56188c559ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:10.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.583+0000 7feb92800700 1 -- 172.21.15.33:0/3164411961 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56188c5bc490 con 0x56188c53f250 2024-05-03T18:29:10.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.583+0000 7feb92800700 1 --2- 172.21.15.33:0/3164411961 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56188c53f250 0x56188c55a950 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7feb8c007760 tx=0x7feb8c00b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4176833511675611 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:29:10.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.583+0000 7feb73e00700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:29:10.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.583+0000 7feb73e00700 1 -- 172.21.15.33:0/3164411961 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb8c005040 con 0x56188c53f250 2024-05-03T18:29:10.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.583+0000 7feba3606b80 1 -- 172.21.15.33:0/3164411961 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb84027560 con 0x56188c53f250 2024-05-03T18:29:10.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.583+0000 7feb73e00700 1 -- 172.21.15.33:0/3164411961 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7feb8c0051e0 con 0x56188c53f250 2024-05-03T18:29:10.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.584+0000 7feb73e00700 1 -- 172.21.15.33:0/3164411961 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7feb8c0053e0 con 0x56188c53f250 2024-05-03T18:29:10.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.584+0000 7feba3606b80 10 client.15546 fetch_fsmap learned FSMap version 21 2024-05-03T18:29:10.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.584+0000 7feba3606b80 1 -- 172.21.15.33:0/3164411961 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x56188c5bd790 con 0x56188c53f250 2024-05-03T18:29:10.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.584+0000 7feb73e00700 1 -- 172.21.15.33:0/3164411961 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7feb84027560 con 0x56188c53f250 2024-05-03T18:29:10.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.584+0000 7feb73e00700 1 -- 172.21.15.33:0/3164411961 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7feb8c024070 con 0x56188c53f250 2024-05-03T18:29:10.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.584+0000 7feba3606b80 10 client.15546 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:29:10.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.585+0000 7feba3606b80 10 client.15546 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:29:10.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.585+0000 7feba3606b80 20 client.15546 populate_metadata read hostname 'smithi033' 2024-05-03T18:29:10.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.585+0000 7feba3606b80 1 --2- 172.21.15.33:0/3164411961 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x56188c5bddf0 0x56188c5c0230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:10.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.585+0000 7feba3606b80 4 client.15546 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-03T18:29:10.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.585+0000 7feba3606b80 1 -- 172.21.15.33:0/3164411961 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56188c499400 con 0x56188c5bddf0 2024-05-03T18:29:10.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.585+0000 7feb91e00700 1 --2- 172.21.15.33:0/3164411961 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x56188c5bddf0 0x56188c5c0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:10.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.586+0000 7feb91e00700 1 --2- 172.21.15.33:0/3164411961 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x56188c5bddf0 0x56188c5c0230 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b7af6debde1926e server_cookie=695bbfd5e7b1aa2d in_seq=0 out_seq=0 2024-05-03T18:29:10.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.586+0000 7feb73e00700 10 client.15546 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:29:10.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.587+0000 7feb73e00700 1 -- 172.21.15.33:0/3164411961 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56188c499400 con 0x56188c5bddf0 2024-05-03T18:29:10.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.587+0000 7feb73e00700 10 client.15546 handle_command_reply: tid=0 2024-05-03T18:29:10.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.587+0000 7feba3606b80 1 client.15546 shutdown 2024-05-03T18:29:10.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.587+0000 7feba3606b80 1 -- 172.21.15.33:0/3164411961 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56188c53f250 msgr2=0x56188c55a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:10.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.587+0000 7feba3606b80 1 --2- 172.21.15.33:0/3164411961 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56188c53f250 0x56188c55a950 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7feb8c007760 tx=0x7feb8c00b7b0 comp rx=0 tx=0).stop 2024-05-03T18:29:10.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.587+0000 7feba3606b80 1 -- 172.21.15.33:0/3164411961 shutdown_connections 2024-05-03T18:29:10.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.587+0000 7feba3606b80 1 --2- 172.21.15.33:0/3164411961 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x56188c5bddf0 0x56188c5c0230 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:10.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.588+0000 7feba3606b80 1 --2- 172.21.15.33:0/3164411961 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56188c53f250 0x56188c55a950 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:10.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.588+0000 7feba3606b80 1 --2- 172.21.15.33:0/3164411961 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56188c539550 0x56188c559ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:10.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.588+0000 7feba3606b80 1 --2- 172.21.15.33:0/3164411961 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56188c53e930 0x56188c55a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:10.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.588+0000 7feba3606b80 1 -- 172.21.15.33:0/3164411961 >> 172.21.15.33:0/3164411961 conn(0x56188c4b16d0 msgr2=0x56188c53f630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:29:10.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.588+0000 7feb73e00700 0 client.15546 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:29:10.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.588+0000 7feba3606b80 1 -- 172.21.15.33:0/3164411961 shutdown_connections 2024-05-03T18:29:10.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.588+0000 7feba3606b80 1 -- 172.21.15.33:0/3164411961 wait complete. 2024-05-03T18:29:10.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.588+0000 7feba3606b80 20 client.15546 trim_cache size 0 max 16384 2024-05-03T18:29:10.589 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-03T18:29:10.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.589+0000 7feb98c00700 1 -- 172.21.15.33:0/2922285046 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7feb780b29f0 msgr2=0x7feb780b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:10.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.589+0000 7feb98c00700 1 --2- 172.21.15.33:0/2922285046 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7feb780b29f0 0x7feb780b4eb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7feb8c009fd0 tx=0x7feb8c0099e0 comp rx=0 tx=0).stop 2024-05-03T18:29:10.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.589+0000 7feb98c00700 1 -- 172.21.15.33:0/2922285046 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb94103eb0 msgr2=0x7feb941a1cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:10.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.589+0000 7feb98c00700 1 --2- 172.21.15.33:0/2922285046 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb94103eb0 0x7feb941a1cc0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7feb8000d950 tx=0x7feb8000dd10 comp rx=0 tx=0).stop 2024-05-03T18:29:10.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.590+0000 7feb98c00700 1 -- 172.21.15.33:0/2922285046 shutdown_connections 2024-05-03T18:29:10.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.590+0000 7feb98c00700 1 --2- 172.21.15.33:0/2922285046 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7feb780b29f0 0x7feb780b4eb0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:10.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.590+0000 7feb98c00700 1 --2- 172.21.15.33:0/2922285046 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb94109600 0x7feb9419b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:10.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.590+0000 7feb98c00700 1 --2- 172.21.15.33:0/2922285046 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb941034f0 0x7feb941a1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:10.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.590+0000 7feb98c00700 1 --2- 172.21.15.33:0/2922285046 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb94103eb0 0x7feb941a1cc0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:10.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.590+0000 7feb98c00700 1 -- 172.21.15.33:0/2922285046 >> 172.21.15.33:0/2922285046 conn(0x7feb9407a650 msgr2=0x7feb9410e910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:29:10.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.590+0000 7feb98c00700 1 -- 172.21.15.33:0/2922285046 shutdown_connections 2024-05-03T18:29:10.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:10.590+0000 7feb98c00700 1 -- 172.21.15.33:0/2922285046 wait complete. 2024-05-03T18:29:11.192 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:29:10 smithi033 ceph-mon[28330]: pgmap v62: 129 pgs: 129 active+clean; 352 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 10 MiB/s wr, 25 op/s 2024-05-03T18:29:11.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:29:10 smithi099 ceph-mon[31658]: pgmap v62: 129 pgs: 129 active+clean; 352 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 10 MiB/s wr, 25 op/s 2024-05-03T18:29:11.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:29:10 smithi150 ceph-mon[32492]: pgmap v62: 129 pgs: 129 active+clean; 352 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 10 MiB/s wr, 25 op/s 2024-05-03T18:29:12.528 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-03T18:29:12.528 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-03T18:29:12.915 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:29:12 smithi033 ceph-mon[28330]: pgmap v63: 129 pgs: 129 active+clean; 352 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 10 MiB/s wr, 25 op/s 2024-05-03T18:29:13.049 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:29:13.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:29:12 smithi099 ceph-mon[31658]: pgmap v63: 129 pgs: 129 active+clean; 352 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 10 MiB/s wr, 25 op/s 2024-05-03T18:29:13.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:29:12 smithi150 ceph-mon[32492]: pgmap v63: 129 pgs: 129 active+clean; 352 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 10 MiB/s wr, 25 op/s 2024-05-03T18:29:14.411 INFO:teuthology.orchestra.run.smithi033.stderr:1400000 of 3200000 tuples (43%) done (elapsed 51.52 s, remaining 66.24 s) 2024-05-03T18:29:15.229 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:29:14 smithi033 ceph-mon[28330]: pgmap v64: 129 pgs: 129 active+clean; 381 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 15 MiB/s wr, 30 op/s 2024-05-03T18:29:15.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:29:14 smithi099 ceph-mon[31658]: pgmap v64: 129 pgs: 129 active+clean; 381 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 15 MiB/s wr, 30 op/s 2024-05-03T18:29:15.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:29:14 smithi150 ceph-mon[32492]: pgmap v64: 129 pgs: 129 active+clean; 381 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 15 MiB/s wr, 30 op/s 2024-05-03T18:29:15.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.409+0000 7f98b9e00700 1 -- 172.21.15.33:0/538506398 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98b410b6c0 msgr2=0x7f98b410baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:15.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.409+0000 7f98b9e00700 1 --2- 172.21.15.33:0/538506398 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98b410b6c0 0x7f98b410baa0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f98a4009990 tx=0x7f98a4009ca0 comp rx=0 tx=0).stop 2024-05-03T18:29:15.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.409+0000 7f98b9e00700 1 -- 172.21.15.33:0/538506398 shutdown_connections 2024-05-03T18:29:15.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.409+0000 7f98b9e00700 1 --2- 172.21.15.33:0/538506398 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f98b4106080 0x7f98b4112590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:15.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.409+0000 7f98b9e00700 1 --2- 172.21.15.33:0/538506398 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98b410b6c0 0x7f98b410baa0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:15.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.409+0000 7f98b9e00700 1 --2- 172.21.15.33:0/538506398 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f98b41056c0 0x7f98b4105b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:15.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.409+0000 7f98b9e00700 1 -- 172.21.15.33:0/538506398 >> 172.21.15.33:0/538506398 conn(0x7f98b41011d0 msgr2=0x7f98b41035f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:29:15.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.409+0000 7f98b9e00700 1 -- 172.21.15.33:0/538506398 shutdown_connections 2024-05-03T18:29:15.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.409+0000 7f98b9e00700 1 -- 172.21.15.33:0/538506398 wait complete. 2024-05-03T18:29:15.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.411+0000 7f98b9e00700 1 Processor -- start 2024-05-03T18:29:15.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.412+0000 7f98b9e00700 1 -- start start 2024-05-03T18:29:15.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.412+0000 7f98b9e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98b41056c0 0x7f98b41a1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:15.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.412+0000 7f98b9e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f98b4106080 0x7f98b41a1e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:15.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.412+0000 7f98b9e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f98b410b6c0 0x7f98b419bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:15.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.412+0000 7f98b9e00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f98b41157b0 con 0x7f98b41056c0 2024-05-03T18:29:15.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.412+0000 7f98b9e00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f98b4115630 con 0x7f98b410b6c0 2024-05-03T18:29:15.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.412+0000 7f98b9e00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f98b4115930 con 0x7f98b4106080 2024-05-03T18:29:15.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.413+0000 7f98b2a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98b41056c0 0x7f98b41a1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:15.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.413+0000 7f98b2a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98b41056c0 0x7f98b41a1910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54792/0 (socket says 172.21.15.33:54792) 2024-05-03T18:29:15.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.413+0000 7f98b2a00700 1 -- 172.21.15.33:0/1736935407 learned_addr learned my addr 172.21.15.33:0/1736935407 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:29:15.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.413+0000 7f98b3400700 1 --2- 172.21.15.33:0/1736935407 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f98b410b6c0 0x7f98b419bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:15.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.413+0000 7f98b2000700 1 --2- 172.21.15.33:0/1736935407 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f98b4106080 0x7f98b41a1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:15.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.413+0000 7f98b2a00700 1 -- 172.21.15.33:0/1736935407 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f98b4106080 msgr2=0x7f98b41a1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:15.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.413+0000 7f98b2a00700 1 --2- 172.21.15.33:0/1736935407 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f98b4106080 0x7f98b41a1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:15.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.413+0000 7f98b2a00700 1 -- 172.21.15.33:0/1736935407 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f98b410b6c0 msgr2=0x7f98b419bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:15.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.413+0000 7f98b2a00700 1 --2- 172.21.15.33:0/1736935407 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f98b410b6c0 0x7f98b419bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:15.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.413+0000 7f98b2a00700 1 -- 172.21.15.33:0/1736935407 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98b419c230 con 0x7f98b41056c0 2024-05-03T18:29:15.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.414+0000 7f98b2a00700 1 --2- 172.21.15.33:0/1736935407 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98b41056c0 0x7f98b41a1910 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f98a400b5c0 tx=0x7f98a4005990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:29:15.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.415+0000 7f98b8a00700 1 -- 172.21.15.33:0/1736935407 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a4004830 con 0x7f98b41056c0 2024-05-03T18:29:15.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.415+0000 7f98b8a00700 1 -- 172.21.15.33:0/1736935407 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f98a4005020 con 0x7f98b41056c0 2024-05-03T18:29:15.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.415+0000 7f98b8a00700 1 -- 172.21.15.33:0/1736935407 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a4003cc0 con 0x7f98b41056c0 2024-05-03T18:29:15.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.415+0000 7f98b9e00700 1 -- 172.21.15.33:0/1736935407 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98b419c4c0 con 0x7f98b41056c0 2024-05-03T18:29:15.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.415+0000 7f98b9e00700 1 -- 172.21.15.33:0/1736935407 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98b419c8f0 con 0x7f98b41056c0 2024-05-03T18:29:15.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.420+0000 7f98b8a00700 1 -- 172.21.15.33:0/1736935407 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f98a40049d0 con 0x7f98b41056c0 2024-05-03T18:29:15.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.421+0000 7f98c47b4b80 1 Processor -- start 2024-05-03T18:29:15.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.421+0000 7f98c47b4b80 1 -- start start 2024-05-03T18:29:15.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.422+0000 7f98b8a00700 1 --2- 172.21.15.33:0/1736935407 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f98980b29f0 0x7f98980b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:15.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.422+0000 7f98c47b4b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e68c50f9b0 0x55e68c50fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:15.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.422+0000 7f98c47b4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e68c5102d0 0x55e68c509fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:15.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.422+0000 7f98b8a00700 1 -- 172.21.15.33:0/1736935407 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f98a40e3910 con 0x7f98b41056c0 2024-05-03T18:29:15.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.422+0000 7f98c47b4b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e68c50a5d0 0x55e68c50aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:15.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.422+0000 7f98c47b4b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e68c45caa0 con 0x55e68c50f9b0 2024-05-03T18:29:15.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.422+0000 7f98c47b4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55e68c3fa7e0 con 0x55e68c5102d0 2024-05-03T18:29:15.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.422+0000 7f98c47b4b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55e68c458a60 con 0x55e68c50a5d0 2024-05-03T18:29:15.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.422+0000 7f98b3400700 1 --2- 172.21.15.33:0/1736935407 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f98980b29f0 0x7f98980b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:15.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.422+0000 7f98b2a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e68c50f9b0 0x55e68c50fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:15.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.423+0000 7f98b2a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e68c50f9b0 0x55e68c50fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54796/0 (socket says 172.21.15.33:54796) 2024-05-03T18:29:15.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.423+0000 7f98b2a00700 1 -- 172.21.15.33:0/3258051503 learned_addr learned my addr 172.21.15.33:0/3258051503 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:29:15.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.423+0000 7f98b2000700 1 --2- 172.21.15.33:0/3258051503 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e68c5102d0 0x55e68c509fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:15.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.423+0000 7f98b3400700 1 --2- 172.21.15.33:0/3258051503 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e68c50a5d0 0x55e68c50aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:15.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.423+0000 7f98b2a00700 1 -- 172.21.15.33:0/3258051503 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e68c50a5d0 msgr2=0x55e68c50aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:29:15.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.424+0000 7f98b2a00700 1 --2- 172.21.15.33:0/3258051503 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e68c50a5d0 0x55e68c50aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:15.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.424+0000 7f98b2a00700 1 -- 172.21.15.33:0/3258051503 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e68c5102d0 msgr2=0x55e68c509fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:29:15.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.424+0000 7f98b2a00700 1 --2- 172.21.15.33:0/3258051503 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e68c5102d0 0x55e68c509fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:15.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.424+0000 7f98b2a00700 1 -- 172.21.15.33:0/3258051503 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e68c3a9950 con 0x55e68c50f9b0 2024-05-03T18:29:15.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.424+0000 7f98b3400700 1 --2- 172.21.15.33:0/3258051503 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e68c50a5d0 0x55e68c50aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:29:15.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.424+0000 7f98b3400700 1 --2- 172.21.15.33:0/1736935407 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f98980b29f0 0x7f98980b4eb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f98ac009cb0 tx=0x7f98ac0077e0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:29:15.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.424+0000 7f98b2a00700 1 --2- 172.21.15.33:0/3258051503 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e68c50f9b0 0x55e68c50fd90 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f98a40a6970 tx=0x7f98a4009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=354226cebefbdec4 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:29:15.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.425+0000 7f98a8c00700 1 -- 172.21.15.33:0/3258051503 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a40095c0 con 0x55e68c50f9b0 2024-05-03T18:29:15.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.425+0000 7f98a8c00700 1 -- 172.21.15.33:0/3258051503 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f98a40aa400 con 0x55e68c50f9b0 2024-05-03T18:29:15.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.425+0000 7f98a8c00700 1 -- 172.21.15.33:0/3258051503 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a40b10d0 con 0x55e68c50f9b0 2024-05-03T18:29:15.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.425+0000 7f98c47b4b80 1 -- 172.21.15.33:0/3258051503 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e68c50f9b0 msgr2=0x55e68c50fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:15.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.425+0000 7f98c47b4b80 1 --2- 172.21.15.33:0/3258051503 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e68c50f9b0 0x55e68c50fd90 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f98a40a6970 tx=0x7f98a4009ca0 comp rx=0 tx=0).stop 2024-05-03T18:29:15.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.426+0000 7f98c47b4b80 1 -- 172.21.15.33:0/3258051503 shutdown_connections 2024-05-03T18:29:15.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.426+0000 7f98c47b4b80 1 --2- 172.21.15.33:0/3258051503 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e68c50a5d0 0x55e68c50aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:15.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.426+0000 7f98c47b4b80 1 --2- 172.21.15.33:0/3258051503 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e68c50f9b0 0x55e68c50fd90 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:15.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.426+0000 7f98c47b4b80 1 --2- 172.21.15.33:0/3258051503 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e68c5102d0 0x55e68c509fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:15.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.426+0000 7f98c47b4b80 1 -- 172.21.15.33:0/3258051503 >> 172.21.15.33:0/3258051503 conn(0x55e68c482750 msgr2=0x55e68c504980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:29:15.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.426+0000 7f98c47b4b80 1 -- 172.21.15.33:0/3258051503 shutdown_connections 2024-05-03T18:29:15.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.426+0000 7f98c47b4b80 1 -- 172.21.15.33:0/3258051503 wait complete. 2024-05-03T18:29:15.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.426+0000 7f98c47b4b80 1 -- start start 2024-05-03T18:29:15.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.427+0000 7f98c47b4b80 1 Processor -- start 2024-05-03T18:29:15.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.427+0000 7f98c47b4b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e68c50a5d0 0x55e68c515ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:15.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.427+0000 7f98c47b4b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e68c50f9b0 0x55e68c513280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:15.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.427+0000 7f98c47b4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e68c5102d0 0x55e68c5137c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:15.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.427+0000 7f98c47b4b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e68c516b40 con 0x55e68c50f9b0 2024-05-03T18:29:15.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.427+0000 7f98c47b4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55e68c3c2ba0 con 0x55e68c5102d0 2024-05-03T18:29:15.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.427+0000 7f98c47b4b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55e68c516cc0 con 0x55e68c50a5d0 2024-05-03T18:29:15.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.428+0000 7f98c47b4b80 10 client.0 _ll_register_callbacks cb 0x55e68c403cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:29:15.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.428+0000 7f98b2000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e68c50f9b0 0x55e68c513280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:15.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.428+0000 7f98b2000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e68c50f9b0 0x55e68c513280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54806/0 (socket says 172.21.15.33:54806) 2024-05-03T18:29:15.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.428+0000 7f98b2000700 1 -- 172.21.15.33:0/1869395477 learned_addr learned my addr 172.21.15.33:0/1869395477 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:29:15.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.428+0000 7f98b2a00700 1 --2- 172.21.15.33:0/1869395477 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e68c50a5d0 0x55e68c515ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:15.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.428+0000 7f98b3400700 1 --2- 172.21.15.33:0/1869395477 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e68c5102d0 0x55e68c5137c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:15.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.429+0000 7f98b2000700 1 -- 172.21.15.33:0/1869395477 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e68c50a5d0 msgr2=0x55e68c515ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:29:15.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.429+0000 7f98b2000700 1 --2- 172.21.15.33:0/1869395477 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e68c50a5d0 0x55e68c515ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:15.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.429+0000 7f98b2000700 1 -- 172.21.15.33:0/1869395477 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e68c5102d0 msgr2=0x55e68c5137c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:29:15.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.429+0000 7f98b2000700 1 --2- 172.21.15.33:0/1869395477 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e68c5102d0 0x55e68c5137c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:15.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.429+0000 7f98b2000700 1 -- 172.21.15.33:0/1869395477 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e68c4224a0 con 0x55e68c50f9b0 2024-05-03T18:29:15.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.429+0000 7f98b3400700 1 --2- 172.21.15.33:0/1869395477 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e68c5102d0 0x55e68c5137c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:29:15.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.429+0000 7f98b2000700 1 --2- 172.21.15.33:0/1869395477 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e68c50f9b0 0x55e68c513280 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f98a00104d0 tx=0x7f98a00107e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb7d1f073910eb1 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:29:15.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.430+0000 7f989d600700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:29:15.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.430+0000 7f989d600700 1 -- 172.21.15.33:0/1869395477 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a0018070 con 0x55e68c50f9b0 2024-05-03T18:29:15.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.430+0000 7f98c47b4b80 1 -- 172.21.15.33:0/1869395477 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98a40aa400 con 0x55e68c50f9b0 2024-05-03T18:29:15.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.430+0000 7f989d600700 1 -- 172.21.15.33:0/1869395477 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f98a0014640 con 0x55e68c50f9b0 2024-05-03T18:29:15.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.431+0000 7f989d600700 1 -- 172.21.15.33:0/1869395477 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f98a0022050 con 0x55e68c50f9b0 2024-05-03T18:29:15.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.431+0000 7f98c47b4b80 10 client.15558 fetch_fsmap learned FSMap version 21 2024-05-03T18:29:15.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.431+0000 7f98c47b4b80 1 -- 172.21.15.33:0/1869395477 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55e68c521170 con 0x55e68c50f9b0 2024-05-03T18:29:15.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.432+0000 7f989d600700 1 -- 172.21.15.33:0/1869395477 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f98a40aa400 con 0x55e68c50f9b0 2024-05-03T18:29:15.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.432+0000 7f989d600700 1 -- 172.21.15.33:0/1869395477 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f98a001d870 con 0x55e68c50f9b0 2024-05-03T18:29:15.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.432+0000 7f98c47b4b80 10 client.15558 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:29:15.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.432+0000 7f98c47b4b80 10 client.15558 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:29:15.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.432+0000 7f98c47b4b80 20 client.15558 populate_metadata read hostname 'smithi033' 2024-05-03T18:29:15.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.432+0000 7f98c47b4b80 1 --2- 172.21.15.33:0/1869395477 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55e68c523750 0x55e68c525b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:15.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.432+0000 7f98c47b4b80 4 client.15558 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:29:15.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.432+0000 7f98c47b4b80 1 -- 172.21.15.33:0/1869395477 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e68c4d7070 con 0x55e68c523750 2024-05-03T18:29:15.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.434+0000 7f98b2a00700 1 --2- 172.21.15.33:0/1869395477 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55e68c523750 0x55e68c525b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:15.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.435+0000 7f98b2a00700 1 --2- 172.21.15.33:0/1869395477 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55e68c523750 0x55e68c525b80 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e14afe7080018fe server_cookie=d70a93d312b7210f in_seq=0 out_seq=0 2024-05-03T18:29:15.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.435+0000 7f989d600700 10 client.15558 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:29:15.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.437+0000 7f989d600700 1 -- 172.21.15.33:0/1869395477 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e68c4d7070 con 0x55e68c523750 2024-05-03T18:29:15.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.437+0000 7f989d600700 10 client.15558 handle_command_reply: tid=0 2024-05-03T18:29:15.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.437+0000 7f98c47b4b80 1 client.15558 shutdown 2024-05-03T18:29:15.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.437+0000 7f98c47b4b80 1 -- 172.21.15.33:0/1869395477 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e68c50f9b0 msgr2=0x55e68c513280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:15.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.437+0000 7f98c47b4b80 1 --2- 172.21.15.33:0/1869395477 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e68c50f9b0 0x55e68c513280 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f98a00104d0 tx=0x7f98a00107e0 comp rx=0 tx=0).stop 2024-05-03T18:29:15.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.437+0000 7f98c47b4b80 1 -- 172.21.15.33:0/1869395477 shutdown_connections 2024-05-03T18:29:15.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.437+0000 7f98c47b4b80 1 --2- 172.21.15.33:0/1869395477 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55e68c523750 0x55e68c525b80 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:15.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.437+0000 7f98c47b4b80 1 --2- 172.21.15.33:0/1869395477 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e68c5102d0 0x55e68c5137c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:15.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.437+0000 7f98c47b4b80 1 --2- 172.21.15.33:0/1869395477 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e68c50a5d0 0x55e68c515ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:15.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.437+0000 7f98c47b4b80 1 --2- 172.21.15.33:0/1869395477 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e68c50f9b0 0x55e68c513280 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:15.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.437+0000 7f98c47b4b80 1 -- 172.21.15.33:0/1869395477 >> 172.21.15.33:0/1869395477 conn(0x55e68c482750 msgr2=0x55e68c4ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:29:15.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.437+0000 7f989d600700 0 client.15558 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:29:15.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.438+0000 7f98c47b4b80 1 -- 172.21.15.33:0/1869395477 shutdown_connections 2024-05-03T18:29:15.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.438+0000 7f98c47b4b80 1 -- 172.21.15.33:0/1869395477 wait complete. 2024-05-03T18:29:15.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.438+0000 7f98c47b4b80 20 client.15558 trim_cache size 0 max 16384 2024-05-03T18:29:15.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.463+0000 7f98c47b4b80 1 Processor -- start 2024-05-03T18:29:15.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.463+0000 7f98c47b4b80 1 -- start start 2024-05-03T18:29:15.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.463+0000 7f98c47b4b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e68c50a5d0 0x55e68c509fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:15.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.463+0000 7f98c47b4b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e68c50f9b0 0x55e68c51f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:15.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.463+0000 7f98c47b4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e68c5102d0 0x55e68c510dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:15.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.463+0000 7f98c47b4b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f98b41ab280 con 0x55e68c50a5d0 2024-05-03T18:29:15.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.463+0000 7f98c47b4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f98b41adb60 con 0x55e68c5102d0 2024-05-03T18:29:15.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.463+0000 7f98c47b4b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f98b41ab950 con 0x55e68c50f9b0 2024-05-03T18:29:15.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.463+0000 7f98b2a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e68c50a5d0 0x55e68c509fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:15.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.464+0000 7f98b2a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e68c50a5d0 0x55e68c509fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54816/0 (socket says 172.21.15.33:54816) 2024-05-03T18:29:15.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.464+0000 7f98b2a00700 1 -- 172.21.15.33:0/804110212 learned_addr learned my addr 172.21.15.33:0/804110212 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:29:15.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.464+0000 7f98b2000700 1 --2- 172.21.15.33:0/804110212 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e68c50f9b0 0x55e68c51f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:15.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.464+0000 7f98b3400700 1 --2- 172.21.15.33:0/804110212 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e68c5102d0 0x55e68c510dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:15.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.464+0000 7f98b2a00700 1 -- 172.21.15.33:0/804110212 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e68c50f9b0 msgr2=0x55e68c51f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:29:15.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.464+0000 7f98b2a00700 1 --2- 172.21.15.33:0/804110212 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e68c50f9b0 0x55e68c51f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:15.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.464+0000 7f98b2a00700 1 -- 172.21.15.33:0/804110212 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e68c5102d0 msgr2=0x55e68c510dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:29:15.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.464+0000 7f98b2a00700 1 --2- 172.21.15.33:0/804110212 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e68c5102d0 0x55e68c510dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:15.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.464+0000 7f98b2a00700 1 -- 172.21.15.33:0/804110212 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e68c5166e0 con 0x55e68c50a5d0 2024-05-03T18:29:15.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.465+0000 7f98b2a00700 1 --2- 172.21.15.33:0/804110212 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e68c50a5d0 0x55e68c509fb0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f98a40e0d80 tx=0x7f98a401a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f10df9bc54d96f63 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:29:15.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.465+0000 7f989fe00700 1 -- 172.21.15.33:0/804110212 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a40aa400 con 0x55e68c50a5d0 2024-05-03T18:29:15.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.465+0000 7f989fe00700 1 -- 172.21.15.33:0/804110212 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f98a40aaff0 con 0x55e68c50a5d0 2024-05-03T18:29:15.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.465+0000 7f989fe00700 1 -- 172.21.15.33:0/804110212 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a40a6b80 con 0x55e68c50a5d0 2024-05-03T18:29:15.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.466+0000 7f98c47b4b80 1 -- 172.21.15.33:0/804110212 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e68c50a5d0 msgr2=0x55e68c509fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:15.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.466+0000 7f98c47b4b80 1 --2- 172.21.15.33:0/804110212 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e68c50a5d0 0x55e68c509fb0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f98a40e0d80 tx=0x7f98a401a040 comp rx=0 tx=0).stop 2024-05-03T18:29:15.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.466+0000 7f98c47b4b80 1 -- 172.21.15.33:0/804110212 shutdown_connections 2024-05-03T18:29:15.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.466+0000 7f98c47b4b80 1 --2- 172.21.15.33:0/804110212 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e68c5102d0 0x55e68c510dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:15.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.466+0000 7f98c47b4b80 1 --2- 172.21.15.33:0/804110212 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e68c50a5d0 0x55e68c509fb0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:15.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.466+0000 7f98c47b4b80 1 --2- 172.21.15.33:0/804110212 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e68c50f9b0 0x55e68c51f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:15.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.466+0000 7f98c47b4b80 1 -- 172.21.15.33:0/804110212 >> 172.21.15.33:0/804110212 conn(0x55e68c482750 msgr2=0x55e68c5106b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:29:15.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.466+0000 7f98c47b4b80 1 -- 172.21.15.33:0/804110212 shutdown_connections 2024-05-03T18:29:15.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.467+0000 7f98c47b4b80 1 -- 172.21.15.33:0/804110212 wait complete. 2024-05-03T18:29:15.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.467+0000 7f98c47b4b80 1 -- start start 2024-05-03T18:29:15.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.468+0000 7f98c47b4b80 1 Processor -- start 2024-05-03T18:29:15.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.468+0000 7f98c47b4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e68c50f9b0 0x55e68c518d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:15.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.468+0000 7f98c47b4b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e68c5102d0 0x55e68c519260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:15.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.468+0000 7f98c47b4b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e68c5197a0 0x55e68c58e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:15.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.468+0000 7f98c47b4b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f98b41ac020 con 0x55e68c5197a0 2024-05-03T18:29:15.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.468+0000 7f98c47b4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f98b41ad490 con 0x55e68c50f9b0 2024-05-03T18:29:15.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.468+0000 7f98c47b4b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f98b41ac6f0 con 0x55e68c5102d0 2024-05-03T18:29:15.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.468+0000 7f98c47b4b80 10 client.0 _ll_register_callbacks cb 0x55e68c403cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:29:15.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.469+0000 7f98b3400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e68c5197a0 0x55e68c58e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:15.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.469+0000 7f98b3400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e68c5197a0 0x55e68c58e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54824/0 (socket says 172.21.15.33:54824) 2024-05-03T18:29:15.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.469+0000 7f98b2a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e68c50f9b0 0x55e68c518d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:15.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.469+0000 7f98b2000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e68c5102d0 0x55e68c519260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:15.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.469+0000 7f98b2000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e68c5102d0 0x55e68c519260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:54106/0 (socket says 172.21.15.33:54106) 2024-05-03T18:29:15.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.469+0000 7f98b2000700 1 -- 172.21.15.33:0/611913750 learned_addr learned my addr 172.21.15.33:0/611913750 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:29:15.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.470+0000 7f98b3400700 1 -- 172.21.15.33:0/611913750 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e68c5102d0 msgr2=0x55e68c519260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:29:15.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.470+0000 7f98b3400700 1 --2- 172.21.15.33:0/611913750 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e68c5102d0 0x55e68c519260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:15.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.470+0000 7f98b3400700 1 -- 172.21.15.33:0/611913750 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e68c50f9b0 msgr2=0x55e68c518d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:29:15.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.470+0000 7f98b3400700 1 --2- 172.21.15.33:0/611913750 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e68c50f9b0 0x55e68c518d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:15.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.470+0000 7f98b3400700 1 -- 172.21.15.33:0/611913750 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e68c58e730 con 0x55e68c5197a0 2024-05-03T18:29:15.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.471+0000 7f98b3400700 1 --2- 172.21.15.33:0/611913750 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e68c5197a0 0x55e68c58e050 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f98ac009710 tx=0x7f98ac004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a624acc64b05fad8 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:29:15.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.471+0000 7f989d600700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:29:15.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.471+0000 7f989d600700 1 -- 172.21.15.33:0/611913750 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98ac028060 con 0x55e68c5197a0 2024-05-03T18:29:15.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.471+0000 7f989d600700 1 -- 172.21.15.33:0/611913750 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f98ac016be0 con 0x55e68c5197a0 2024-05-03T18:29:15.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.471+0000 7f98c47b4b80 1 -- 172.21.15.33:0/611913750 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98a40aaff0 con 0x55e68c5197a0 2024-05-03T18:29:15.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.472+0000 7f989d600700 1 -- 172.21.15.33:0/611913750 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f98ac016de0 con 0x55e68c5197a0 2024-05-03T18:29:15.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.472+0000 7f98c47b4b80 10 client.15564 fetch_fsmap learned FSMap version 21 2024-05-03T18:29:15.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.472+0000 7f98c47b4b80 1 -- 172.21.15.33:0/611913750 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55e68c58fab0 con 0x55e68c5197a0 2024-05-03T18:29:15.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.472+0000 7f989d600700 1 -- 172.21.15.33:0/611913750 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f98a40aaff0 con 0x55e68c5197a0 2024-05-03T18:29:15.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.472+0000 7f989d600700 1 -- 172.21.15.33:0/611913750 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f98ac022070 con 0x55e68c5197a0 2024-05-03T18:29:15.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.473+0000 7f98c47b4b80 10 client.15564 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:29:15.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.473+0000 7f98c47b4b80 10 client.15564 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:29:15.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.473+0000 7f98c47b4b80 20 client.15564 populate_metadata read hostname 'smithi033' 2024-05-03T18:29:15.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.473+0000 7f98c47b4b80 1 --2- 172.21.15.33:0/611913750 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55e68c5900f0 0x55e68c592510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:15.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.473+0000 7f98c47b4b80 4 client.15564 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-03T18:29:15.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.473+0000 7f98c47b4b80 1 -- 172.21.15.33:0/611913750 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e68c46a480 con 0x55e68c5900f0 2024-05-03T18:29:15.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.473+0000 7f98b2a00700 1 --2- 172.21.15.33:0/611913750 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55e68c5900f0 0x55e68c592510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:15.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.474+0000 7f98b2a00700 1 --2- 172.21.15.33:0/611913750 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55e68c5900f0 0x55e68c592510 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3d45a18857f49d2 server_cookie=3530f47bed42d9b4 in_seq=0 out_seq=0 2024-05-03T18:29:15.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.474+0000 7f989d600700 10 client.15564 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:29:15.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.477+0000 7f989d600700 1 -- 172.21.15.33:0/611913750 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e68c46a480 con 0x55e68c5900f0 2024-05-03T18:29:15.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.477+0000 7f989d600700 10 client.15564 handle_command_reply: tid=0 2024-05-03T18:29:15.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.477+0000 7f98c47b4b80 1 client.15564 shutdown 2024-05-03T18:29:15.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.477+0000 7f98c47b4b80 1 -- 172.21.15.33:0/611913750 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e68c5197a0 msgr2=0x55e68c58e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:15.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.477+0000 7f98c47b4b80 1 --2- 172.21.15.33:0/611913750 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e68c5197a0 0x55e68c58e050 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f98ac009710 tx=0x7f98ac004b40 comp rx=0 tx=0).stop 2024-05-03T18:29:15.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.478+0000 7f98c47b4b80 1 -- 172.21.15.33:0/611913750 shutdown_connections 2024-05-03T18:29:15.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.478+0000 7f98c47b4b80 1 --2- 172.21.15.33:0/611913750 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55e68c5900f0 0x55e68c592510 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:15.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.478+0000 7f98c47b4b80 1 --2- 172.21.15.33:0/611913750 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e68c5197a0 0x55e68c58e050 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:15.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.478+0000 7f98c47b4b80 1 --2- 172.21.15.33:0/611913750 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e68c50f9b0 0x55e68c518d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:15.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.478+0000 7f989d600700 0 client.15564 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:29:15.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.478+0000 7f98c47b4b80 1 --2- 172.21.15.33:0/611913750 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e68c5102d0 0x55e68c519260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:15.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.478+0000 7f98c47b4b80 1 -- 172.21.15.33:0/611913750 >> 172.21.15.33:0/611913750 conn(0x55e68c482750 msgr2=0x55e68c4feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:29:15.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.478+0000 7f98c47b4b80 1 -- 172.21.15.33:0/611913750 shutdown_connections 2024-05-03T18:29:15.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.478+0000 7f98c47b4b80 1 -- 172.21.15.33:0/611913750 wait complete. 2024-05-03T18:29:15.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.478+0000 7f98c47b4b80 20 client.15564 trim_cache size 0 max 16384 2024-05-03T18:29:15.479 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-03T18:29:15.479 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-05-03T18:29:15.480 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "dd98bf21-bd42-4616-a1cd-da57bea0a7c7", 2024-05-03T18:29:15.480 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-05-03T18:29:15.480 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-03T18:29:15.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.479+0000 7f98b9e00700 1 -- 172.21.15.33:0/1736935407 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f98980b29f0 msgr2=0x7f98980b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:15.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.479+0000 7f98b9e00700 1 --2- 172.21.15.33:0/1736935407 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f98980b29f0 0x7f98980b4eb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f98ac009cb0 tx=0x7f98ac0077e0 comp rx=0 tx=0).stop 2024-05-03T18:29:15.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.479+0000 7f98b9e00700 1 -- 172.21.15.33:0/1736935407 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98b41056c0 msgr2=0x7f98b41a1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:15.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.479+0000 7f98b9e00700 1 --2- 172.21.15.33:0/1736935407 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98b41056c0 0x7f98b41a1910 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f98a400b5c0 tx=0x7f98a4005990 comp rx=0 tx=0).stop 2024-05-03T18:29:15.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.480+0000 7f98b9e00700 1 -- 172.21.15.33:0/1736935407 shutdown_connections 2024-05-03T18:29:15.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.480+0000 7f98b9e00700 1 --2- 172.21.15.33:0/1736935407 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f98980b29f0 0x7f98980b4eb0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:15.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.480+0000 7f98b9e00700 1 --2- 172.21.15.33:0/1736935407 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f98b410b6c0 0x7f98b419bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:15.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.480+0000 7f98b9e00700 1 --2- 172.21.15.33:0/1736935407 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98b41056c0 0x7f98b41a1910 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:15.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.480+0000 7f98b9e00700 1 --2- 172.21.15.33:0/1736935407 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f98b4106080 0x7f98b41a1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:15.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.480+0000 7f98b9e00700 1 -- 172.21.15.33:0/1736935407 >> 172.21.15.33:0/1736935407 conn(0x7f98b41011d0 msgr2=0x7f98b41019c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:29:15.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.480+0000 7f98b9e00700 1 -- 172.21.15.33:0/1736935407 shutdown_connections 2024-05-03T18:29:15.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:15.480+0000 7f98b9e00700 1 -- 172.21.15.33:0/1736935407 wait complete. 2024-05-03T18:29:16.353 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-03T18:29:16.849 INFO:teuthology.orchestra.run.smithi033.stderr:1500000 of 3200000 tuples (46%) done (elapsed 53.96 s, remaining 61.15 s) 2024-05-03T18:29:16.863 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:29:17.187 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:29:16 smithi033 ceph-mon[28330]: scrub queued for path: / 2024-05-03T18:29:17.187 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:29:16 smithi033 ceph-mon[28330]: scrub summary: idle+waiting paths [/] 2024-05-03T18:29:17.187 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:29:16 smithi033 ceph-mon[28330]: scrub summary: active paths [/] 2024-05-03T18:29:17.187 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:29:16 smithi033 ceph-mon[28330]: scrub summary: idle+waiting paths [/] 2024-05-03T18:29:17.187 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:29:16 smithi033 ceph-mon[28330]: pgmap v65: 129 pgs: 129 active+clean; 381 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.7 MiB/s wr, 10 op/s 2024-05-03T18:29:17.187 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:29:16 smithi033 ceph-mon[28330]: scrub summary: idle 2024-05-03T18:29:17.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:29:16 smithi099 ceph-mon[31658]: scrub queued for path: / 2024-05-03T18:29:17.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:29:16 smithi099 ceph-mon[31658]: scrub summary: idle+waiting paths [/] 2024-05-03T18:29:17.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:29:16 smithi099 ceph-mon[31658]: scrub summary: active paths [/] 2024-05-03T18:29:17.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:29:16 smithi099 ceph-mon[31658]: scrub summary: idle+waiting paths [/] 2024-05-03T18:29:17.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:29:16 smithi099 ceph-mon[31658]: pgmap v65: 129 pgs: 129 active+clean; 381 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.7 MiB/s wr, 10 op/s 2024-05-03T18:29:17.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:29:16 smithi099 ceph-mon[31658]: scrub summary: idle 2024-05-03T18:29:17.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:29:16 smithi150 ceph-mon[32492]: scrub queued for path: / 2024-05-03T18:29:17.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:29:16 smithi150 ceph-mon[32492]: scrub summary: idle+waiting paths [/] 2024-05-03T18:29:17.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:29:16 smithi150 ceph-mon[32492]: scrub summary: active paths [/] 2024-05-03T18:29:17.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:29:16 smithi150 ceph-mon[32492]: scrub summary: idle+waiting paths [/] 2024-05-03T18:29:17.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:29:16 smithi150 ceph-mon[32492]: pgmap v65: 129 pgs: 129 active+clean; 381 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.7 MiB/s wr, 10 op/s 2024-05-03T18:29:17.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:29:16 smithi150 ceph-mon[32492]: scrub summary: idle 2024-05-03T18:29:19.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.226+0000 7f1b78a00700 1 -- 172.21.15.33:0/3804322762 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b74103eb0 msgr2=0x7f1b74110390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:19.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.226+0000 7f1b78a00700 1 --2- 172.21.15.33:0/3804322762 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b74103eb0 0x7f1b74110390 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c00b0a0 tx=0x7f1b6c00b3b0 comp rx=0 tx=0).stop 2024-05-03T18:29:19.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.226+0000 7f1b78a00700 1 -- 172.21.15.33:0/3804322762 shutdown_connections 2024-05-03T18:29:19.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.226+0000 7f1b78a00700 1 --2- 172.21.15.33:0/3804322762 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b74103eb0 0x7f1b74110390 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:19.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.226+0000 7f1b78a00700 1 --2- 172.21.15.33:0/3804322762 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b74109600 0x7f1b741099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:19.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.226+0000 7f1b78a00700 1 --2- 172.21.15.33:0/3804322762 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b741034f0 0x7f1b74103970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:19.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.226+0000 7f1b78a00700 1 -- 172.21.15.33:0/3804322762 >> 172.21.15.33:0/3804322762 conn(0x7f1b7407a650 msgr2=0x7f1b7407aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:29:19.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.227+0000 7f1b78a00700 1 -- 172.21.15.33:0/3804322762 shutdown_connections 2024-05-03T18:29:19.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.227+0000 7f1b78a00700 1 -- 172.21.15.33:0/3804322762 wait complete. 2024-05-03T18:29:19.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.228+0000 7f1b78a00700 1 Processor -- start 2024-05-03T18:29:19.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.228+0000 7f1b78a00700 1 -- start start 2024-05-03T18:29:19.229 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:29:18 smithi033 ceph-mon[28330]: pgmap v66: 129 pgs: 129 active+clean; 425 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 17 MiB/s wr, 15 op/s 2024-05-03T18:29:19.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.229+0000 7f1b78a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b741034f0 0x7f1b740757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:19.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.229+0000 7f1b78a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b74103eb0 0x7f1b74075ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:19.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.229+0000 7f1b78a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b74109600 0x7f1b74076220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:19.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.229+0000 7f1b78a00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1b74113570 con 0x7f1b741034f0 2024-05-03T18:29:19.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.229+0000 7f1b78a00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1b741133f0 con 0x7f1b74103eb0 2024-05-03T18:29:19.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.229+0000 7f1b78a00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f1b741136f0 con 0x7f1b74109600 2024-05-03T18:29:19.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.230+0000 7f1b71400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b74103eb0 0x7f1b74075ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:19.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.230+0000 7f1b71400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b74103eb0 0x7f1b74075ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:36962/0 (socket says 172.21.15.33:36962) 2024-05-03T18:29:19.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.230+0000 7f1b71400700 1 -- 172.21.15.33:0/1874399840 learned_addr learned my addr 172.21.15.33:0/1874399840 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:29:19.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.230+0000 7f1b72800700 1 --2- 172.21.15.33:0/1874399840 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b74109600 0x7f1b74076220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:19.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.230+0000 7f1b71e00700 1 --2- 172.21.15.33:0/1874399840 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b741034f0 0x7f1b740757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:19.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.231+0000 7f1b72800700 1 -- 172.21.15.33:0/1874399840 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b74103eb0 msgr2=0x7f1b74075ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:19.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.231+0000 7f1b72800700 1 --2- 172.21.15.33:0/1874399840 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b74103eb0 0x7f1b74075ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:19.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.231+0000 7f1b72800700 1 -- 172.21.15.33:0/1874399840 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b741034f0 msgr2=0x7f1b740757a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:19.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.231+0000 7f1b72800700 1 --2- 172.21.15.33:0/1874399840 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b741034f0 0x7f1b740757a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:19.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.231+0000 7f1b72800700 1 -- 172.21.15.33:0/1874399840 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b741a7f60 con 0x7f1b74109600 2024-05-03T18:29:19.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.231+0000 7f1b71e00700 1 --2- 172.21.15.33:0/1874399840 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b741034f0 0x7f1b740757a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:29:19.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.231+0000 7f1b71400700 1 --2- 172.21.15.33:0/1874399840 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b74103eb0 0x7f1b74075ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:29:19.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.231+0000 7f1b72800700 1 --2- 172.21.15.33:0/1874399840 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b74109600 0x7f1b74076220 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c00b070 tx=0x7f1b6c004c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:29:19.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.232+0000 7f1b6aa00700 1 -- 172.21.15.33:0/1874399840 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b6c021070 con 0x7f1b74109600 2024-05-03T18:29:19.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.232+0000 7f1b78a00700 1 -- 172.21.15.33:0/1874399840 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b741a81f0 con 0x7f1b74109600 2024-05-03T18:29:19.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.232+0000 7f1b78a00700 1 -- 172.21.15.33:0/1874399840 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b741a8640 con 0x7f1b74109600 2024-05-03T18:29:19.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.232+0000 7f1b6aa00700 1 -- 172.21.15.33:0/1874399840 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1b6c007b80 con 0x7f1b74109600 2024-05-03T18:29:19.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.232+0000 7f1b6aa00700 1 -- 172.21.15.33:0/1874399840 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b6c0052a0 con 0x7f1b74109600 2024-05-03T18:29:19.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.237+0000 7f1b6aa00700 1 -- 172.21.15.33:0/1874399840 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f1b6c024070 con 0x7f1b74109600 2024-05-03T18:29:19.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.238+0000 7f1b83352b80 1 Processor -- start 2024-05-03T18:29:19.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.238+0000 7f1b83352b80 1 -- start start 2024-05-03T18:29:19.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.238+0000 7f1b83352b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f6c8e7930 0x555f6c8e7d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:19.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.238+0000 7f1b83352b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555f6c8e8250 0x555f6c8e1f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:19.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.238+0000 7f1b83352b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f6c8e2550 0x555f6c8e29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:19.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.238+0000 7f1b83352b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x555f6c834a20 con 0x555f6c8e7930 2024-05-03T18:29:19.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.238+0000 7f1b83352b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x555f6c7d2760 con 0x555f6c8e2550 2024-05-03T18:29:19.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.238+0000 7f1b83352b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x555f6c8309e0 con 0x555f6c8e8250 2024-05-03T18:29:19.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.239+0000 7f1b71400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f6c8e7930 0x555f6c8e7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:19.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.239+0000 7f1b71400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f6c8e7930 0x555f6c8e7d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43870/0 (socket says 172.21.15.33:43870) 2024-05-03T18:29:19.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.239+0000 7f1b71400700 1 -- 172.21.15.33:0/3877786874 learned_addr learned my addr 172.21.15.33:0/3877786874 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:29:19.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.239+0000 7f1b71e00700 1 --2- 172.21.15.33:0/3877786874 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555f6c8e8250 0x555f6c8e1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:19.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.240+0000 7f1b71400700 1 -- 172.21.15.33:0/3877786874 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555f6c8e8250 msgr2=0x555f6c8e1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:29:19.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.240+0000 7f1b71400700 1 --2- 172.21.15.33:0/3877786874 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555f6c8e8250 0x555f6c8e1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:19.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.240+0000 7f1b71400700 1 -- 172.21.15.33:0/3877786874 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f6c8e2550 msgr2=0x555f6c8e29d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-03T18:29:19.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.240+0000 7f1b71400700 1 --2- 172.21.15.33:0/3877786874 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f6c8e2550 0x555f6c8e29d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:19.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.240+0000 7f1b71400700 1 -- 172.21.15.33:0/3877786874 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f6c7818d0 con 0x555f6c8e7930 2024-05-03T18:29:19.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.240+0000 7f1b6aa00700 1 --2- 172.21.15.33:0/1874399840 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f1b580b2a70 0x7f1b580b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:19.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.240+0000 7f1b71400700 1 --2- 172.21.15.33:0/3877786874 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f6c8e7930 0x555f6c8e7d10 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f1b6000bfd0 tx=0x7f1b60010890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7376d978b1a54503 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:29:19.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.240+0000 7f1b6aa00700 1 -- 172.21.15.33:0/1874399840 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f1b6c0e3890 con 0x7f1b74109600 2024-05-03T18:29:19.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.240+0000 7f1b71e00700 1 --2- 172.21.15.33:0/1874399840 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f1b580b2a70 0x7f1b580b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:19.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.241+0000 7f1b5f400700 1 -- 172.21.15.33:0/3877786874 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b60011700 con 0x555f6c8e7930 2024-05-03T18:29:19.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.241+0000 7f1b5f400700 1 -- 172.21.15.33:0/3877786874 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1b60011d30 con 0x555f6c8e7930 2024-05-03T18:29:19.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.241+0000 7f1b5f400700 1 -- 172.21.15.33:0/3877786874 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b60011700 con 0x555f6c8e7930 2024-05-03T18:29:19.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.242+0000 7f1b83352b80 1 -- 172.21.15.33:0/3877786874 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f6c8e7930 msgr2=0x555f6c8e7d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:19.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.242+0000 7f1b83352b80 1 --2- 172.21.15.33:0/3877786874 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f6c8e7930 0x555f6c8e7d10 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f1b6000bfd0 tx=0x7f1b60010890 comp rx=0 tx=0).stop 2024-05-03T18:29:19.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.243+0000 7f1b83352b80 1 -- 172.21.15.33:0/3877786874 shutdown_connections 2024-05-03T18:29:19.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.243+0000 7f1b83352b80 1 --2- 172.21.15.33:0/3877786874 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f6c8e2550 0x555f6c8e29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:19.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.243+0000 7f1b83352b80 1 --2- 172.21.15.33:0/3877786874 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f6c8e7930 0x555f6c8e7d10 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:19.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.243+0000 7f1b83352b80 1 --2- 172.21.15.33:0/3877786874 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555f6c8e8250 0x555f6c8e1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:19.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.243+0000 7f1b83352b80 1 -- 172.21.15.33:0/3877786874 >> 172.21.15.33:0/3877786874 conn(0x555f6c85a6d0 msgr2=0x555f6c8dc900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:29:19.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.243+0000 7f1b71e00700 1 --2- 172.21.15.33:0/1874399840 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f1b580b2a70 0x7f1b580b4f30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1b6400dd10 tx=0x7f1b6400d320 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:29:19.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.243+0000 7f1b83352b80 1 -- 172.21.15.33:0/3877786874 shutdown_connections 2024-05-03T18:29:19.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.243+0000 7f1b83352b80 1 -- 172.21.15.33:0/3877786874 wait complete. 2024-05-03T18:29:19.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.244+0000 7f1b83352b80 1 -- start start 2024-05-03T18:29:19.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.244+0000 7f1b83352b80 1 Processor -- start 2024-05-03T18:29:19.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.245+0000 7f1b83352b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555f6c8e2550 0x555f6c8ee050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:19.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.245+0000 7f1b83352b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f6c8e8250 0x555f6c8eb350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:19.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.245+0000 7f1b83352b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f6c8eb890 0x555f6c8ebd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:19.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.245+0000 7f1b83352b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x555f6c8eeda0 con 0x555f6c8e8250 2024-05-03T18:29:19.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.245+0000 7f1b83352b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x555f6c79ab20 con 0x555f6c8eb890 2024-05-03T18:29:19.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.245+0000 7f1b83352b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x555f6c8eef20 con 0x555f6c8e2550 2024-05-03T18:29:19.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.245+0000 7f1b83352b80 10 client.0 _ll_register_callbacks cb 0x555f6c7dbc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:29:19.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.245+0000 7f1b71400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f6c8e8250 0x555f6c8eb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:19.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.245+0000 7f1b71400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f6c8e8250 0x555f6c8eb350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43872/0 (socket says 172.21.15.33:43872) 2024-05-03T18:29:19.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.245+0000 7f1b71400700 1 -- 172.21.15.33:0/3862912462 learned_addr learned my addr 172.21.15.33:0/3862912462 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:29:19.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.246+0000 7f1b71e00700 1 --2- 172.21.15.33:0/3862912462 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555f6c8e2550 0x555f6c8ee050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:19.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.246+0000 7f1b72800700 1 --2- 172.21.15.33:0/3862912462 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f6c8eb890 0x555f6c8ebd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:19.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.246+0000 7f1b71400700 1 -- 172.21.15.33:0/3862912462 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555f6c8e2550 msgr2=0x555f6c8ee050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:29:19.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.246+0000 7f1b71400700 1 --2- 172.21.15.33:0/3862912462 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555f6c8e2550 0x555f6c8ee050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:19.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.246+0000 7f1b71400700 1 -- 172.21.15.33:0/3862912462 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f6c8eb890 msgr2=0x555f6c8ebd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:29:19.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.246+0000 7f1b71400700 1 --2- 172.21.15.33:0/3862912462 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f6c8eb890 0x555f6c8ebd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:19.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.246+0000 7f1b71400700 1 -- 172.21.15.33:0/3862912462 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f6c7fa420 con 0x555f6c8e8250 2024-05-03T18:29:19.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.246+0000 7f1b71400700 1 --2- 172.21.15.33:0/3862912462 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f6c8e8250 0x555f6c8eb350 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f1b600197b0 tx=0x7f1b60019480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9036d493bea34c9b server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:29:19.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.246+0000 7f1b53e00700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:29:19.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.247+0000 7f1b53e00700 1 -- 172.21.15.33:0/3862912462 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b6001f040 con 0x555f6c8e8250 2024-05-03T18:29:19.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.247+0000 7f1b53e00700 1 -- 172.21.15.33:0/3862912462 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f1b60019b40 con 0x555f6c8e8250 2024-05-03T18:29:19.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.247+0000 7f1b83352b80 1 -- 172.21.15.33:0/3862912462 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b60011d30 con 0x555f6c8e8250 2024-05-03T18:29:19.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.247+0000 7f1b53e00700 1 -- 172.21.15.33:0/3862912462 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f1b60019d80 con 0x555f6c8e8250 2024-05-03T18:29:19.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.247+0000 7f1b83352b80 10 client.15576 fetch_fsmap learned FSMap version 21 2024-05-03T18:29:19.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.247+0000 7f1b83352b80 1 -- 172.21.15.33:0/3862912462 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x555f6c8f9b30 con 0x555f6c8e8250 2024-05-03T18:29:19.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.248+0000 7f1b53e00700 1 -- 172.21.15.33:0/3862912462 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1b60011d30 con 0x555f6c8e8250 2024-05-03T18:29:19.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.249+0000 7f1b53e00700 1 -- 172.21.15.33:0/3862912462 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f1b6002b030 con 0x555f6c8e8250 2024-05-03T18:29:19.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.249+0000 7f1b83352b80 10 client.15576 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:29:19.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.249+0000 7f1b83352b80 10 client.15576 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:29:19.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.249+0000 7f1b83352b80 20 client.15576 populate_metadata read hostname 'smithi033' 2024-05-03T18:29:19.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.249+0000 7f1b83352b80 1 --2- 172.21.15.33:0/3862912462 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x555f6c8fc3d0 0x555f6c8fe7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:19.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.249+0000 7f1b83352b80 4 client.15576 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:29:19.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.249+0000 7f1b83352b80 1 -- 172.21.15.33:0/3862912462 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555f6c8aeff0 con 0x555f6c8fc3d0 2024-05-03T18:29:19.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.250+0000 7f1b71e00700 1 --2- 172.21.15.33:0/3862912462 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x555f6c8fc3d0 0x555f6c8fe7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:19.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.251+0000 7f1b71e00700 1 --2- 172.21.15.33:0/3862912462 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x555f6c8fc3d0 0x555f6c8fe7c0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75ce9b25c71f0c12 server_cookie=884f451ff74bb29d in_seq=0 out_seq=0 2024-05-03T18:29:19.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.251+0000 7f1b53e00700 10 client.15576 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:29:19.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.253+0000 7f1b53e00700 1 -- 172.21.15.33:0/3862912462 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555f6c8aeff0 con 0x555f6c8fc3d0 2024-05-03T18:29:19.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.253+0000 7f1b53e00700 10 client.15576 handle_command_reply: tid=0 2024-05-03T18:29:19.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.253+0000 7f1b83352b80 1 client.15576 shutdown 2024-05-03T18:29:19.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.253+0000 7f1b83352b80 1 -- 172.21.15.33:0/3862912462 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f6c8e8250 msgr2=0x555f6c8eb350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:19.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.253+0000 7f1b83352b80 1 --2- 172.21.15.33:0/3862912462 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f6c8e8250 0x555f6c8eb350 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f1b600197b0 tx=0x7f1b60019480 comp rx=0 tx=0).stop 2024-05-03T18:29:19.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.253+0000 7f1b83352b80 1 -- 172.21.15.33:0/3862912462 shutdown_connections 2024-05-03T18:29:19.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.253+0000 7f1b83352b80 1 --2- 172.21.15.33:0/3862912462 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x555f6c8fc3d0 0x555f6c8fe7c0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:19.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.253+0000 7f1b83352b80 1 --2- 172.21.15.33:0/3862912462 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f6c8eb890 0x555f6c8ebd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:19.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.253+0000 7f1b83352b80 1 --2- 172.21.15.33:0/3862912462 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555f6c8e2550 0x555f6c8ee050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:19.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.253+0000 7f1b83352b80 1 --2- 172.21.15.33:0/3862912462 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f6c8e8250 0x555f6c8eb350 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:19.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.253+0000 7f1b83352b80 1 -- 172.21.15.33:0/3862912462 >> 172.21.15.33:0/3862912462 conn(0x555f6c85a6d0 msgr2=0x555f6c8d73b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:29:19.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.254+0000 7f1b53e00700 0 client.15576 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:29:19.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.254+0000 7f1b83352b80 1 -- 172.21.15.33:0/3862912462 shutdown_connections 2024-05-03T18:29:19.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.254+0000 7f1b83352b80 1 -- 172.21.15.33:0/3862912462 wait complete. 2024-05-03T18:29:19.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.254+0000 7f1b83352b80 20 client.15576 trim_cache size 0 max 16384 2024-05-03T18:29:19.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.282+0000 7f1b83352b80 1 Processor -- start 2024-05-03T18:29:19.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.282+0000 7f1b83352b80 1 -- start start 2024-05-03T18:29:19.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.282+0000 7f1b83352b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f6c8e2550 0x555f6c8e1f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:19.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.282+0000 7f1b83352b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555f6c8eb890 0x555f6c8e8630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:19.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.282+0000 7f1b83352b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f6c8ea370 0x555f6c8e8b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:19.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.283+0000 7f1b83352b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1b5400cd90 con 0x555f6c8ea370 2024-05-03T18:29:19.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.283+0000 7f1b83352b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1b5400f610 con 0x555f6c8e2550 2024-05-03T18:29:19.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.283+0000 7f1b83352b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f1b5400d400 con 0x555f6c8eb890 2024-05-03T18:29:19.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.283+0000 7f1b72800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f6c8ea370 0x555f6c8e8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:19.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.283+0000 7f1b72800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f6c8ea370 0x555f6c8e8b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43888/0 (socket says 172.21.15.33:43888) 2024-05-03T18:29:19.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.283+0000 7f1b72800700 1 -- 172.21.15.33:0/2944272871 learned_addr learned my addr 172.21.15.33:0/2944272871 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:29:19.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.283+0000 7f1b71400700 1 --2- 172.21.15.33:0/2944272871 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555f6c8eb890 0x555f6c8e8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:19.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.283+0000 7f1b71e00700 1 --2- 172.21.15.33:0/2944272871 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f6c8e2550 0x555f6c8e1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:19.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.283+0000 7f1b72800700 1 -- 172.21.15.33:0/2944272871 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555f6c8eb890 msgr2=0x555f6c8e8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:29:19.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.283+0000 7f1b72800700 1 --2- 172.21.15.33:0/2944272871 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555f6c8eb890 0x555f6c8e8630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:19.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.283+0000 7f1b72800700 1 -- 172.21.15.33:0/2944272871 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f6c8e2550 msgr2=0x555f6c8e1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:29:19.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.283+0000 7f1b72800700 1 --2- 172.21.15.33:0/2944272871 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f6c8e2550 0x555f6c8e1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:19.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.284+0000 7f1b72800700 1 -- 172.21.15.33:0/2944272871 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f6c8e98a0 con 0x555f6c8ea370 2024-05-03T18:29:19.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.284+0000 7f1b72800700 1 --2- 172.21.15.33:0/2944272871 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f6c8ea370 0x555f6c8e8b70 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c00b0a0 tx=0x7f1b6c0a4f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee7780e595847995 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:29:19.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.284+0000 7f1b5ea00700 1 -- 172.21.15.33:0/2944272871 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b6c02b070 con 0x555f6c8ea370 2024-05-03T18:29:19.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.284+0000 7f1b5ea00700 1 -- 172.21.15.33:0/2944272871 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1b6c0aaf70 con 0x555f6c8ea370 2024-05-03T18:29:19.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.284+0000 7f1b5ea00700 1 -- 172.21.15.33:0/2944272871 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b6c0a6b00 con 0x555f6c8ea370 2024-05-03T18:29:19.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.285+0000 7f1b83352b80 1 -- 172.21.15.33:0/2944272871 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f6c8ea370 msgr2=0x555f6c8e8b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:19.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.285+0000 7f1b83352b80 1 --2- 172.21.15.33:0/2944272871 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f6c8ea370 0x555f6c8e8b70 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c00b0a0 tx=0x7f1b6c0a4f90 comp rx=0 tx=0).stop 2024-05-03T18:29:19.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.285+0000 7f1b83352b80 1 -- 172.21.15.33:0/2944272871 shutdown_connections 2024-05-03T18:29:19.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.285+0000 7f1b83352b80 1 --2- 172.21.15.33:0/2944272871 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f6c8ea370 0x555f6c8e8b70 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:19.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.285+0000 7f1b83352b80 1 --2- 172.21.15.33:0/2944272871 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f6c8e2550 0x555f6c8e1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:19.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.285+0000 7f1b83352b80 1 --2- 172.21.15.33:0/2944272871 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555f6c8eb890 0x555f6c8e8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:19.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.285+0000 7f1b83352b80 1 -- 172.21.15.33:0/2944272871 >> 172.21.15.33:0/2944272871 conn(0x555f6c85a6d0 msgr2=0x555f6c8e9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:29:19.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.285+0000 7f1b83352b80 1 -- 172.21.15.33:0/2944272871 shutdown_connections 2024-05-03T18:29:19.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.286+0000 7f1b83352b80 1 -- 172.21.15.33:0/2944272871 wait complete. 2024-05-03T18:29:19.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.286+0000 7f1b83352b80 1 -- start start 2024-05-03T18:29:19.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.286+0000 7f1b83352b80 1 Processor -- start 2024-05-03T18:29:19.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.287+0000 7f1b83352b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f6c8e2550 0x555f6c901800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:19.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.287+0000 7f1b83352b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555f6c8ea370 0x555f6c8ffc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:19.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.287+0000 7f1b83352b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f6c8eb890 0x555f6c900140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:19.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.287+0000 7f1b83352b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1b5400dad0 con 0x555f6c8e2550 2024-05-03T18:29:19.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.287+0000 7f1b83352b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1b5400ef40 con 0x555f6c8eb890 2024-05-03T18:29:19.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.287+0000 7f1b83352b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f1b5400e1a0 con 0x555f6c8ea370 2024-05-03T18:29:19.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.287+0000 7f1b83352b80 10 client.0 _ll_register_callbacks cb 0x555f6c7dbc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:29:19.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.287+0000 7f1b71e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f6c8e2550 0x555f6c901800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:19.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.287+0000 7f1b71e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f6c8e2550 0x555f6c901800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43896/0 (socket says 172.21.15.33:43896) 2024-05-03T18:29:19.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.287+0000 7f1b71e00700 1 -- 172.21.15.33:0/70465916 learned_addr learned my addr 172.21.15.33:0/70465916 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:29:19.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.288+0000 7f1b71e00700 1 -- 172.21.15.33:0/70465916 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555f6c8ea370 msgr2=0x555f6c8ffc00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-03T18:29:19.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.288+0000 7f1b71e00700 1 --2- 172.21.15.33:0/70465916 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555f6c8ea370 0x555f6c8ffc00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:19.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.288+0000 7f1b71e00700 1 -- 172.21.15.33:0/70465916 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f6c8eb890 msgr2=0x555f6c900140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:29:19.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.288+0000 7f1b71e00700 1 --2- 172.21.15.33:0/70465916 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f6c8eb890 0x555f6c900140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:19.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.288+0000 7f1b71e00700 1 -- 172.21.15.33:0/70465916 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f6c9008c0 con 0x555f6c8e2550 2024-05-03T18:29:19.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.288+0000 7f1b71e00700 1 --2- 172.21.15.33:0/70465916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f6c8e2550 0x555f6c901800 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x555f6c8eee90 tx=0x7f1b640047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ca67bc1f9746b1e server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:29:19.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.288+0000 7f1b53e00700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:29:19.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.288+0000 7f1b53e00700 1 -- 172.21.15.33:0/70465916 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b64029590 con 0x555f6c8e2550 2024-05-03T18:29:19.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.289+0000 7f1b83352b80 1 -- 172.21.15.33:0/70465916 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b6c0aaf70 con 0x555f6c8e2550 2024-05-03T18:29:19.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.289+0000 7f1b53e00700 1 -- 172.21.15.33:0/70465916 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f1b64029730 con 0x555f6c8e2550 2024-05-03T18:29:19.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.289+0000 7f1b53e00700 1 -- 172.21.15.33:0/70465916 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f1b6403a050 con 0x555f6c8e2550 2024-05-03T18:29:19.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.289+0000 7f1b83352b80 10 client.15582 fetch_fsmap learned FSMap version 21 2024-05-03T18:29:19.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.289+0000 7f1b83352b80 1 -- 172.21.15.33:0/70465916 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x555f6c967900 con 0x555f6c8e2550 2024-05-03T18:29:19.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.289+0000 7f1b53e00700 1 -- 172.21.15.33:0/70465916 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1b6c0aaf70 con 0x555f6c8e2550 2024-05-03T18:29:19.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.289+0000 7f1b53e00700 1 -- 172.21.15.33:0/70465916 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f1b64029070 con 0x555f6c8e2550 2024-05-03T18:29:19.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.290+0000 7f1b83352b80 10 client.15582 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:29:19.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.290+0000 7f1b83352b80 10 client.15582 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:29:19.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.290+0000 7f1b83352b80 20 client.15582 populate_metadata read hostname 'smithi033' 2024-05-03T18:29:19.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.290+0000 7f1b83352b80 1 --2- 172.21.15.33:0/70465916 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x555f6c968090 0x555f6c96a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:19.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.290+0000 7f1b83352b80 4 client.15582 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-03T18:29:19.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.290+0000 7f1b83352b80 1 -- 172.21.15.33:0/70465916 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555f6c842400 con 0x555f6c968090 2024-05-03T18:29:19.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.290+0000 7f1b71400700 1 --2- 172.21.15.33:0/70465916 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x555f6c968090 0x555f6c96a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:19.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.292+0000 7f1b71400700 1 --2- 172.21.15.33:0/70465916 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x555f6c968090 0x555f6c96a4f0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da38b88c1ad8ae72 server_cookie=b10502543ba66b0c in_seq=0 out_seq=0 2024-05-03T18:29:19.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.292+0000 7f1b53e00700 10 client.15582 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:29:19.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.292+0000 7f1b53e00700 1 -- 172.21.15.33:0/70465916 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x555f6c842400 con 0x555f6c968090 2024-05-03T18:29:19.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.292+0000 7f1b53e00700 10 client.15582 handle_command_reply: tid=0 2024-05-03T18:29:19.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.292+0000 7f1b83352b80 1 client.15582 shutdown 2024-05-03T18:29:19.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.293+0000 7f1b83352b80 1 -- 172.21.15.33:0/70465916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f6c8e2550 msgr2=0x555f6c901800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:19.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.293+0000 7f1b83352b80 1 --2- 172.21.15.33:0/70465916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f6c8e2550 0x555f6c901800 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x555f6c8eee90 tx=0x7f1b640047c0 comp rx=0 tx=0).stop 2024-05-03T18:29:19.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.293+0000 7f1b83352b80 1 -- 172.21.15.33:0/70465916 shutdown_connections 2024-05-03T18:29:19.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.293+0000 7f1b83352b80 1 --2- 172.21.15.33:0/70465916 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x555f6c968090 0x555f6c96a4f0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:19.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.293+0000 7f1b83352b80 1 --2- 172.21.15.33:0/70465916 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f6c8eb890 0x555f6c900140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:19.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.293+0000 7f1b83352b80 1 --2- 172.21.15.33:0/70465916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f6c8e2550 0x555f6c901800 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:19.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.293+0000 7f1b83352b80 1 --2- 172.21.15.33:0/70465916 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555f6c8ea370 0x555f6c8ffc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:19.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.293+0000 7f1b83352b80 1 -- 172.21.15.33:0/70465916 >> 172.21.15.33:0/70465916 conn(0x555f6c85a6d0 msgr2=0x555f6c8d6b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:29:19.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.293+0000 7f1b53e00700 0 client.15582 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:29:19.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.293+0000 7f1b83352b80 1 -- 172.21.15.33:0/70465916 shutdown_connections 2024-05-03T18:29:19.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.293+0000 7f1b83352b80 1 -- 172.21.15.33:0/70465916 wait complete. 2024-05-03T18:29:19.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.294+0000 7f1b83352b80 20 client.15582 trim_cache size 0 max 16384 2024-05-03T18:29:19.295 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-03T18:29:19.295 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-05-03T18:29:19.295 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-05-03T18:29:19.295 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-03T18:29:19.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.295+0000 7f1b78a00700 1 -- 172.21.15.33:0/1874399840 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f1b580b2a70 msgr2=0x7f1b580b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:19.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.295+0000 7f1b78a00700 1 --2- 172.21.15.33:0/1874399840 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f1b580b2a70 0x7f1b580b4f30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1b6400dd10 tx=0x7f1b6400d320 comp rx=0 tx=0).stop 2024-05-03T18:29:19.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.295+0000 7f1b78a00700 1 -- 172.21.15.33:0/1874399840 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b74109600 msgr2=0x7f1b74076220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:19.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.295+0000 7f1b78a00700 1 --2- 172.21.15.33:0/1874399840 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b74109600 0x7f1b74076220 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c00b070 tx=0x7f1b6c004c60 comp rx=0 tx=0).stop 2024-05-03T18:29:19.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.295+0000 7f1b78a00700 1 -- 172.21.15.33:0/1874399840 shutdown_connections 2024-05-03T18:29:19.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.295+0000 7f1b78a00700 1 --2- 172.21.15.33:0/1874399840 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b74109600 0x7f1b74076220 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:19.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.295+0000 7f1b78a00700 1 --2- 172.21.15.33:0/1874399840 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b741034f0 0x7f1b740757a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:19.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.295+0000 7f1b78a00700 1 --2- 172.21.15.33:0/1874399840 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f1b580b2a70 0x7f1b580b4f30 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:19.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.295+0000 7f1b78a00700 1 --2- 172.21.15.33:0/1874399840 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b74103eb0 0x7f1b74075ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:19.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.295+0000 7f1b78a00700 1 -- 172.21.15.33:0/1874399840 >> 172.21.15.33:0/1874399840 conn(0x7f1b7407a650 msgr2=0x7f1b74102140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:29:19.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.296+0000 7f1b78a00700 1 -- 172.21.15.33:0/1874399840 shutdown_connections 2024-05-03T18:29:19.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:19.296+0000 7f1b78a00700 1 -- 172.21.15.33:0/1874399840 wait complete. 2024-05-03T18:29:19.297 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:29:18 smithi099 ceph-mon[31658]: pgmap v66: 129 pgs: 129 active+clean; 425 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 17 MiB/s wr, 15 op/s 2024-05-03T18:29:19.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:29:18 smithi150 ceph-mon[32492]: pgmap v66: 129 pgs: 129 active+clean; 425 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 17 MiB/s wr, 15 op/s 2024-05-03T18:29:20.267 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-03T18:29:20.268 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T18:29:20.652 INFO:teuthology.orchestra.run.smithi033.stderr:1600000 of 3200000 tuples (50%) done (elapsed 57.76 s, remaining 57.76 s) 2024-05-03T18:29:20.751 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:29:21.156 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:29:20 smithi033 ceph-mon[28330]: pgmap v67: 129 pgs: 129 active+clean; 425 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 12 MiB/s wr, 11 op/s 2024-05-03T18:29:21.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:29:20 smithi099 ceph-mon[31658]: pgmap v67: 129 pgs: 129 active+clean; 425 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 12 MiB/s wr, 11 op/s 2024-05-03T18:29:21.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:29:20 smithi150 ceph-mon[32492]: pgmap v67: 129 pgs: 129 active+clean; 425 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 12 MiB/s wr, 11 op/s 2024-05-03T18:29:23.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:23.023+0000 7f767e000700 1 -- 172.21.15.33:0/2270097358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76781045d0 msgr2=0x7f76781049b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:23.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:23.023+0000 7f767e000700 1 --2- 172.21.15.33:0/2270097358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76781045d0 0x7f76781049b0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f766c009a10 tx=0x7f766c009d20 comp rx=0 tx=0).stop 2024-05-03T18:29:23.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:23.024+0000 7f767e000700 1 -- 172.21.15.33:0/2270097358 shutdown_connections 2024-05-03T18:29:23.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:23.024+0000 7f767e000700 1 --2- 172.21.15.33:0/2270097358 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f767810da00 0x7f767810fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:23.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:23.024+0000 7f767e000700 1 --2- 172.21.15.33:0/2270097358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76781045d0 0x7f76781049b0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:23.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:23.024+0000 7f767e000700 1 --2- 172.21.15.33:0/2270097358 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7678104ef0 0x7f767810d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:23.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:23.024+0000 7f767e000700 1 -- 172.21.15.33:0/2270097358 >> 172.21.15.33:0/2270097358 conn(0x7f76780fddc0 msgr2=0x7f76781001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:29:23.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:23.024+0000 7f767e000700 1 -- 172.21.15.33:0/2270097358 shutdown_connections 2024-05-03T18:29:23.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:23.024+0000 7f767e000700 1 -- 172.21.15.33:0/2270097358 wait complete. 2024-05-03T18:29:23.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:23.025+0000 7f767e000700 1 Processor -- start 2024-05-03T18:29:23.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:23.025+0000 7f767e000700 1 -- start start 2024-05-03T18:29:23.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:23.026+0000 7f767e000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76781045d0 0x7f767819d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:23.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:23.026+0000 7f767e000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7678104ef0 0x7f767819d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:23.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:23.026+0000 7f767e000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f767810da00 0x7f7678197670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:23.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:23.026+0000 7f767e000700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7678111330 con 0x7f76781045d0 2024-05-03T18:29:23.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:23.026+0000 7f767e000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f76781111b0 con 0x7f767810da00 2024-05-03T18:29:23.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:23.026+0000 7f767e000700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f76781114b0 con 0x7f7678104ef0 2024-05-03T18:29:23.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:23.026+0000 7f767cc00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76781045d0 0x7f767819d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:23.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:23.026+0000 7f767cc00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76781045d0 0x7f767819d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43922/0 (socket says 172.21.15.33:43922) 2024-05-03T18:29:23.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:23.026+0000 7f767cc00700 1 -- 172.21.15.33:0/3759495971 learned_addr learned my addr 172.21.15.33:0/3759495971 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:29:23.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:23.027+0000 7f7677e00700 1 --2- 172.21.15.33:0/3759495971 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7678104ef0 0x7f767819d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:23.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:23.027+0000 7f767cc00700 1 -- 172.21.15.33:0/3759495971 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7678104ef0 msgr2=0x7f767819d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:23.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:23.027+0000 7f767cc00700 1 --2- 172.21.15.33:0/3759495971 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7678104ef0 0x7f767819d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:23.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:23.027+0000 7f767cc00700 1 -- 172.21.15.33:0/3759495971 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f767810da00 msgr2=0x7f7678197670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:23.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:23.027+0000 7f767cc00700 1 --2- 172.21.15.33:0/3759495971 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f767810da00 0x7f7678197670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:23.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:23.027+0000 7f767cc00700 1 -- 172.21.15.33:0/3759495971 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7678197dd0 con 0x7f76781045d0 2024-05-03T18:29:23.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:23.027+0000 7f767cc00700 1 --2- 172.21.15.33:0/3759495971 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76781045d0 0x7f767819d460 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f766c01a040 tx=0x7f766c00b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:29:23.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:23.028+0000 7f7675600700 1 -- 172.21.15.33:0/3759495971 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f766c026070 con 0x7f76781045d0 2024-05-03T18:29:23.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:23.028+0000 7f767e000700 1 -- 172.21.15.33:0/3759495971 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7678198060 con 0x7f76781045d0 2024-05-03T18:29:23.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:23.028+0000 7f7675600700 1 -- 172.21.15.33:0/3759495971 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f766c010e30 con 0x7f76781045d0 2024-05-03T18:29:23.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:23.028+0000 7f7675600700 1 -- 172.21.15.33:0/3759495971 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f766c022680 con 0x7f76781045d0 2024-05-03T18:29:23.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:23.028+0000 7f767e000700 1 -- 172.21.15.33:0/3759495971 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76781a4330 con 0x7f76781045d0 2024-05-03T18:29:23.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:23.030+0000 7f767e000700 1 -- 172.21.15.33:0/3759495971 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f767806a190 con 0x7f76781045d0 2024-05-03T18:29:23.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:23.033+0000 7f7675600700 1 -- 172.21.15.33:0/3759495971 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f766c029070 con 0x7f76781045d0 2024-05-03T18:29:23.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:23.035+0000 7f7675600700 1 --2- 172.21.15.33:0/3759495971 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f76600b2a70 0x7f76600b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:23.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:23.035+0000 7f7675600700 1 -- 172.21.15.33:0/3759495971 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f766c0e7cf0 con 0x7f76781045d0 2024-05-03T18:29:23.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:23.035+0000 7f7677e00700 1 --2- 172.21.15.33:0/3759495971 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f76600b2a70 0x7f76600b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:23.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:23.036+0000 7f7677e00700 1 --2- 172.21.15.33:0/3759495971 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f76600b2a70 0x7f76600b4f30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f767810edc0 tx=0x7f766400b410 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:29:23.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:23.039+0000 7f7675600700 1 -- 172.21.15.33:0/3759495971 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f766c0af480 con 0x7f76781045d0 2024-05-03T18:29:23.237 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:29:22 smithi033 ceph-mon[28330]: pgmap v68: 129 pgs: 129 active+clean; 425 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 12 MiB/s wr, 11 op/s 2024-05-03T18:29:23.280 INFO:teuthology.orchestra.run.smithi033.stderr:1700000 of 3200000 tuples (53%) done (elapsed 60.39 s, remaining 53.28 s) 2024-05-03T18:29:23.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:29:22 smithi099 ceph-mon[31658]: pgmap v68: 129 pgs: 129 active+clean; 425 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 12 MiB/s wr, 11 op/s 2024-05-03T18:29:23.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:29:22 smithi150 ceph-mon[32492]: pgmap v68: 129 pgs: 129 active+clean; 425 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 12 MiB/s wr, 11 op/s 2024-05-03T18:29:23.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:23.437+0000 7f767e000700 1 -- 172.21.15.33:0/3759495971 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7678198b50 con 0x7f76781045d0 2024-05-03T18:29:23.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:23.438+0000 7f7675600700 1 -- 172.21.15.33:0/3759495971 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11801 (secure 0 0 0) 0x7f766c0aed80 con 0x7f76781045d0 2024-05-03T18:29:23.439 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:29:23.439 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1327393503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1327393503},{"type":"v1","addr":"172.21.15.33:6835","nonce":1327393503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/2461472948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":2461472948},{"type":"v1","addr":"172.21.15.33:6837","nonce":2461472948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14559,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/442269281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":442269281},{"type":"v1","addr":"172.21.15.33:6839","nonce":442269281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14571,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/645571756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":645571756},{"type":"v1","addr":"172.21.15.33:6841","nonce":645571756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/203598909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":203598909},{"type":"v1","addr":"172.21.15.150:6833","nonce":203598909}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/4029842493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":4029842493},{"type":"v1","addr":"172.21.15.150:6835","nonce":4029842493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3463664423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3463664423},{"type":"v1","addr":"172.21.15.150:6837","nonce":3463664423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24481,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6839/4051648018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":4051648018},{"type":"v1","addr":"172.21.15.150:6839","nonce":4051648018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34418,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1865105479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1865105479},{"type":"v1","addr":"172.21.15.99:6833","nonce":1865105479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34445,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/1206393914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1206393914},{"type":"v1","addr":"172.21.15.99:6835","nonce":1206393914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-03T18:22:28.837091+0000","modified":"2024-05-03T18:23:49.758398+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34508},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34472":{"gid":34472,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6837/3360970342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3360970342},{"type":"v1","addr":"172.21.15.99:6837","nonce":3360970342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34508":{"gid":34508,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/919907308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":919907308},{"type":"v1","addr":"172.21.15.99:6839","nonce":919907308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34508,"qdb_cluster":[34508]},"id":1}]} 2024-05-03T18:29:23.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:23.442+0000 7f767e000700 1 -- 172.21.15.33:0/3759495971 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f76600b2a70 msgr2=0x7f76600b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:23.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:23.442+0000 7f767e000700 1 --2- 172.21.15.33:0/3759495971 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f76600b2a70 0x7f76600b4f30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f767810edc0 tx=0x7f766400b410 comp rx=0 tx=0).stop 2024-05-03T18:29:23.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:23.442+0000 7f767e000700 1 -- 172.21.15.33:0/3759495971 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76781045d0 msgr2=0x7f767819d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:23.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:23.442+0000 7f767e000700 1 --2- 172.21.15.33:0/3759495971 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76781045d0 0x7f767819d460 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f766c01a040 tx=0x7f766c00b980 comp rx=0 tx=0).stop 2024-05-03T18:29:23.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:23.442+0000 7f767e000700 1 -- 172.21.15.33:0/3759495971 shutdown_connections 2024-05-03T18:29:23.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:23.442+0000 7f767e000700 1 --2- 172.21.15.33:0/3759495971 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f76600b2a70 0x7f76600b4f30 secure :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f767810edc0 tx=0x7f766400b410 comp rx=0 tx=0).stop 2024-05-03T18:29:23.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:23.442+0000 7f767e000700 1 --2- 172.21.15.33:0/3759495971 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f767810da00 0x7f7678197670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:23.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:23.442+0000 7f767e000700 1 --2- 172.21.15.33:0/3759495971 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76781045d0 0x7f767819d460 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:23.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:23.442+0000 7f767e000700 1 --2- 172.21.15.33:0/3759495971 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7678104ef0 0x7f767819d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:23.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:23.442+0000 7f767e000700 1 -- 172.21.15.33:0/3759495971 >> 172.21.15.33:0/3759495971 conn(0x7f76780fddc0 msgr2=0x7f76780feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:29:23.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:23.442+0000 7f767e000700 1 -- 172.21.15.33:0/3759495971 shutdown_connections 2024-05-03T18:29:23.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:23.442+0000 7f767e000700 1 -- 172.21.15.33:0/3759495971 wait complete. 2024-05-03T18:29:23.444 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 21 2024-05-03T18:29:24.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:29:23 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/3759495971' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:29:24.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:29:23 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/3759495971' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:29:24.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:29:23 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/3759495971' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:29:24.439 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-03T18:29:24.964 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:29:25.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:29:24 smithi033 ceph-mon[28330]: pgmap v69: 129 pgs: 129 active+clean; 453 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 17 MiB/s wr, 17 op/s 2024-05-03T18:29:25.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:29:24 smithi099 ceph-mon[31658]: pgmap v69: 129 pgs: 129 active+clean; 453 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 17 MiB/s wr, 17 op/s 2024-05-03T18:29:25.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:29:24 smithi150 ceph-mon[32492]: pgmap v69: 129 pgs: 129 active+clean; 453 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 17 MiB/s wr, 17 op/s 2024-05-03T18:29:26.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:29:25 smithi033 ceph-mon[28330]: pgmap v70: 129 pgs: 129 active+clean; 453 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 12 MiB/s wr, 12 op/s 2024-05-03T18:29:26.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:29:25 smithi099 ceph-mon[31658]: pgmap v70: 129 pgs: 129 active+clean; 453 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 12 MiB/s wr, 12 op/s 2024-05-03T18:29:26.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:29:25 smithi150 ceph-mon[32492]: pgmap v70: 129 pgs: 129 active+clean; 453 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 12 MiB/s wr, 12 op/s 2024-05-03T18:29:26.729 INFO:teuthology.orchestra.run.smithi033.stderr:1800000 of 3200000 tuples (56%) done (elapsed 63.84 s, remaining 49.65 s) 2024-05-03T18:29:27.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.302+0000 7f6d3fe00700 1 -- 172.21.15.33:0/2792142656 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d40107660 msgr2=0x7f6d4010f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:27.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.302+0000 7f6d3fe00700 1 --2- 172.21.15.33:0/2792142656 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d40107660 0x7f6d4010f230 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f6d3800b0d0 tx=0x7f6d3800b3e0 comp rx=0 tx=0).stop 2024-05-03T18:29:27.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.302+0000 7f6d3fe00700 1 -- 172.21.15.33:0/2792142656 shutdown_connections 2024-05-03T18:29:27.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.302+0000 7f6d3fe00700 1 --2- 172.21.15.33:0/2792142656 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d40107660 0x7f6d4010f230 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:27.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.302+0000 7f6d3fe00700 1 --2- 172.21.15.33:0/2792142656 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d40102580 0x7f6d40102960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:27.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.302+0000 7f6d3fe00700 1 --2- 172.21.15.33:0/2792142656 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d40102f30 0x7f6d40106f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:27.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.302+0000 7f6d3fe00700 1 -- 172.21.15.33:0/2792142656 >> 172.21.15.33:0/2792142656 conn(0x7f6d400fde30 msgr2=0x7f6d40100250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:29:27.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.303+0000 7f6d3fe00700 1 -- 172.21.15.33:0/2792142656 shutdown_connections 2024-05-03T18:29:27.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.303+0000 7f6d3fe00700 1 -- 172.21.15.33:0/2792142656 wait complete. 2024-05-03T18:29:27.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.304+0000 7f6d3fe00700 1 Processor -- start 2024-05-03T18:29:27.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.304+0000 7f6d3fe00700 1 -- start start 2024-05-03T18:29:27.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.305+0000 7f6d3fe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d40102580 0x7f6d401986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:27.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.305+0000 7f6d3fe00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d40102f30 0x7f6d40198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:27.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.305+0000 7f6d3fe00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d40107660 0x7f6d40199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:27.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.305+0000 7f6d3fe00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6d40112420 con 0x7f6d40107660 2024-05-03T18:29:27.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.305+0000 7f6d3fe00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6d401122a0 con 0x7f6d40102580 2024-05-03T18:29:27.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.305+0000 7f6d3fe00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6d401125a0 con 0x7f6d40102f30 2024-05-03T18:29:27.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.305+0000 7f6d3f400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d40107660 0x7f6d40199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:27.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.305+0000 7f6d3f400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d40107660 0x7f6d40199170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45662/0 (socket says 172.21.15.33:45662) 2024-05-03T18:29:27.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.305+0000 7f6d3f400700 1 -- 172.21.15.33:0/1910653322 learned_addr learned my addr 172.21.15.33:0/1910653322 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:29:27.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.305+0000 7f6d3ea00700 1 --2- 172.21.15.33:0/1910653322 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d40102580 0x7f6d401986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:27.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.306+0000 7f6d3e000700 1 --2- 172.21.15.33:0/1910653322 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d40102f30 0x7f6d40198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:27.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.306+0000 7f6d3f400700 1 -- 172.21.15.33:0/1910653322 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d40102f30 msgr2=0x7f6d40198c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:27.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.306+0000 7f6d3f400700 1 --2- 172.21.15.33:0/1910653322 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d40102f30 0x7f6d40198c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:27.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.306+0000 7f6d3f400700 1 -- 172.21.15.33:0/1910653322 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d40102580 msgr2=0x7f6d401986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:27.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.306+0000 7f6d3f400700 1 --2- 172.21.15.33:0/1910653322 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d40102580 0x7f6d401986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:27.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.306+0000 7f6d3f400700 1 -- 172.21.15.33:0/1910653322 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d401a7f00 con 0x7f6d40107660 2024-05-03T18:29:27.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.306+0000 7f6d3f400700 1 --2- 172.21.15.33:0/1910653322 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d40107660 0x7f6d40199170 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f6d38007d10 tx=0x7f6d38007d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:29:27.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.307+0000 7f6d37400700 1 -- 172.21.15.33:0/1910653322 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d3801f070 con 0x7f6d40107660 2024-05-03T18:29:27.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.307+0000 7f6d37400700 1 -- 172.21.15.33:0/1910653322 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6d38003e10 con 0x7f6d40107660 2024-05-03T18:29:27.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.307+0000 7f6d37400700 1 -- 172.21.15.33:0/1910653322 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d3801b720 con 0x7f6d40107660 2024-05-03T18:29:27.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.307+0000 7f6d3fe00700 1 -- 172.21.15.33:0/1910653322 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d401a8190 con 0x7f6d40107660 2024-05-03T18:29:27.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.307+0000 7f6d3fe00700 1 -- 172.21.15.33:0/1910653322 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d401a8590 con 0x7f6d40107660 2024-05-03T18:29:27.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.313+0000 7f6d37400700 1 -- 172.21.15.33:0/1910653322 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f6d38022070 con 0x7f6d40107660 2024-05-03T18:29:27.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.313+0000 7f6d50049b80 1 Processor -- start 2024-05-03T18:29:27.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.314+0000 7f6d50049b80 1 -- start start 2024-05-03T18:29:27.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.314+0000 7f6d50049b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f6fd9e5a40 0x55f6fd9e5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:27.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.314+0000 7f6d50049b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f6fd9e6360 0x55f6fd9dffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:27.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.314+0000 7f6d50049b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f6fd9e05d0 0x55f6fd9e0a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:27.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.314+0000 7f6d50049b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55f6fd932a20 con 0x55f6fd9e6360 2024-05-03T18:29:27.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.314+0000 7f6d50049b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55f6fd8d0760 con 0x55f6fd9e5a40 2024-05-03T18:29:27.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.314+0000 7f6d50049b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55f6fd92e9e0 con 0x55f6fd9e05d0 2024-05-03T18:29:27.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.315+0000 7f6d3ea00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f6fd9e6360 0x55f6fd9dffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:27.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.315+0000 7f6d3ea00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f6fd9e6360 0x55f6fd9dffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45668/0 (socket says 172.21.15.33:45668) 2024-05-03T18:29:27.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.315+0000 7f6d3ea00700 1 -- 172.21.15.33:0/2210203932 learned_addr learned my addr 172.21.15.33:0/2210203932 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:29:27.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.315+0000 7f6d37400700 1 --2- 172.21.15.33:0/1910653322 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f6d240b2a70 0x7f6d240b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:27.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.315+0000 7f6d3e000700 1 --2- 172.21.15.33:0/2210203932 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f6fd9e5a40 0x55f6fd9e5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:27.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.315+0000 7f6d3e000700 1 --2- 172.21.15.33:0/2210203932 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f6fd9e05d0 0x55f6fd9e0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:27.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.315+0000 7f6d37400700 1 -- 172.21.15.33:0/1910653322 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f6d3800f020 con 0x7f6d40107660 2024-05-03T18:29:27.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.315+0000 7f6d3ea00700 1 -- 172.21.15.33:0/2210203932 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f6fd9e05d0 msgr2=0x55f6fd9e0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:29:27.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.315+0000 7f6d3ea00700 1 --2- 172.21.15.33:0/2210203932 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f6fd9e05d0 0x55f6fd9e0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:27.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.315+0000 7f6d3ea00700 1 -- 172.21.15.33:0/2210203932 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f6fd9e5a40 msgr2=0x55f6fd9e5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:29:27.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.315+0000 7f6d3ea00700 1 --2- 172.21.15.33:0/2210203932 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f6fd9e5a40 0x55f6fd9e5e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:27.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.315+0000 7f6d3ea00700 1 -- 172.21.15.33:0/2210203932 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6fd87f8d0 con 0x55f6fd9e6360 2024-05-03T18:29:27.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.315+0000 7f6d3f400700 1 --2- 172.21.15.33:0/1910653322 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f6d240b2a70 0x7f6d240b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:27.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.315+0000 7f6d3e000700 1 --2- 172.21.15.33:0/2210203932 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f6fd9e05d0 0x55f6fd9e0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:29:27.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.316+0000 7f6d3ea00700 1 --2- 172.21.15.33:0/2210203932 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f6fd9e6360 0x55f6fd9dffb0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f6d3000b780 tx=0x7f6d3000ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f6ed4222c99167f server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:29:27.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.316+0000 7f6d3f400700 1 --2- 172.21.15.33:0/1910653322 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f6d240b2a70 0x7f6d240b4f30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6d380e1180 tx=0x7f6d3800b3e0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:29:27.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.316+0000 7f6d2be00700 1 -- 172.21.15.33:0/2210203932 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d30003a80 con 0x55f6fd9e6360 2024-05-03T18:29:27.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.316+0000 7f6d2be00700 1 -- 172.21.15.33:0/2210203932 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6d3001d070 con 0x55f6fd9e6360 2024-05-03T18:29:27.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.316+0000 7f6d2be00700 1 -- 172.21.15.33:0/2210203932 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d30005440 con 0x55f6fd9e6360 2024-05-03T18:29:27.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.317+0000 7f6d50049b80 1 -- 172.21.15.33:0/2210203932 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f6fd9e6360 msgr2=0x55f6fd9dffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:27.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.317+0000 7f6d50049b80 1 --2- 172.21.15.33:0/2210203932 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f6fd9e6360 0x55f6fd9dffb0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f6d3000b780 tx=0x7f6d3000ba90 comp rx=0 tx=0).stop 2024-05-03T18:29:27.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.317+0000 7f6d50049b80 1 -- 172.21.15.33:0/2210203932 shutdown_connections 2024-05-03T18:29:27.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.317+0000 7f6d50049b80 1 --2- 172.21.15.33:0/2210203932 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f6fd9e05d0 0x55f6fd9e0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:27.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.317+0000 7f6d50049b80 1 --2- 172.21.15.33:0/2210203932 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f6fd9e5a40 0x55f6fd9e5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:27.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.317+0000 7f6d50049b80 1 --2- 172.21.15.33:0/2210203932 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f6fd9e6360 0x55f6fd9dffb0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:27.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.317+0000 7f6d50049b80 1 -- 172.21.15.33:0/2210203932 >> 172.21.15.33:0/2210203932 conn(0x55f6fd90eb60 msgr2=0x55f6fd9da9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:29:27.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.318+0000 7f6d50049b80 1 -- 172.21.15.33:0/2210203932 shutdown_connections 2024-05-03T18:29:27.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.318+0000 7f6d50049b80 1 -- 172.21.15.33:0/2210203932 wait complete. 2024-05-03T18:29:27.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.318+0000 7f6d50049b80 1 -- start start 2024-05-03T18:29:27.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.318+0000 7f6d50049b80 1 Processor -- start 2024-05-03T18:29:27.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.319+0000 7f6d50049b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f6fd9e05d0 0x55f6fd9ec0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:27.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.319+0000 7f6d50049b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f6fd9e5a40 0x55f6fd9e9450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:27.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.319+0000 7f6d50049b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f6fd9e6360 0x55f6fd9e9990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:27.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.319+0000 7f6d50049b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55f6fd9ecdd0 con 0x55f6fd9e05d0 2024-05-03T18:29:27.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.319+0000 7f6d50049b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55f6fd898b20 con 0x55f6fd9e5a40 2024-05-03T18:29:27.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.319+0000 7f6d50049b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55f6fd9ecf50 con 0x55f6fd9e6360 2024-05-03T18:29:27.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.319+0000 7f6d50049b80 10 client.0 _ll_register_callbacks cb 0x55f6fd8d9c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:29:27.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.319+0000 7f6d3e000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f6fd9e05d0 0x55f6fd9ec0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:27.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.319+0000 7f6d3e000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f6fd9e05d0 0x55f6fd9ec0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45684/0 (socket says 172.21.15.33:45684) 2024-05-03T18:29:27.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.319+0000 7f6d3e000700 1 -- 172.21.15.33:0/3434977594 learned_addr learned my addr 172.21.15.33:0/3434977594 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:29:27.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.320+0000 7f6d3ea00700 1 --2- 172.21.15.33:0/3434977594 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f6fd9e5a40 0x55f6fd9e9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:27.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.320+0000 7f6d3e000700 1 --2- 172.21.15.33:0/3434977594 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f6fd9e6360 0x55f6fd9e9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:27.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.320+0000 7f6d3e000700 1 -- 172.21.15.33:0/3434977594 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f6fd9e6360 msgr2=0x55f6fd9e9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:29:27.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.320+0000 7f6d3e000700 1 --2- 172.21.15.33:0/3434977594 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f6fd9e6360 0x55f6fd9e9990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:27.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.320+0000 7f6d3e000700 1 -- 172.21.15.33:0/3434977594 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f6fd9e5a40 msgr2=0x55f6fd9e9450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:29:27.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.320+0000 7f6d3e000700 1 --2- 172.21.15.33:0/3434977594 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f6fd9e5a40 0x55f6fd9e9450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:27.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.320+0000 7f6d3e000700 1 -- 172.21.15.33:0/3434977594 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6fd8f8420 con 0x55f6fd9e05d0 2024-05-03T18:29:27.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.320+0000 7f6d3ea00700 1 --2- 172.21.15.33:0/3434977594 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f6fd9e5a40 0x55f6fd9e9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:29:27.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.321+0000 7f6d3e000700 1 --2- 172.21.15.33:0/3434977594 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f6fd9e05d0 0x55f6fd9ec0b0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c0044c0 tx=0x7f6d2c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c983131052a99bc server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:29:27.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.321+0000 7f6d28c00700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:29:27.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.321+0000 7f6d28c00700 1 -- 172.21.15.33:0/3434977594 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d2c004a30 con 0x55f6fd9e05d0 2024-05-03T18:29:27.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.321+0000 7f6d28c00700 1 -- 172.21.15.33:0/3434977594 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f6d2c004bd0 con 0x55f6fd9e05d0 2024-05-03T18:29:27.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.321+0000 7f6d50049b80 1 -- 172.21.15.33:0/3434977594 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d3001d070 con 0x55f6fd9e05d0 2024-05-03T18:29:27.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.321+0000 7f6d28c00700 1 -- 172.21.15.33:0/3434977594 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f6d2c027050 con 0x55f6fd9e05d0 2024-05-03T18:29:27.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.321+0000 7f6d50049b80 10 client.15600 fetch_fsmap learned FSMap version 21 2024-05-03T18:29:27.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.321+0000 7f6d50049b80 1 -- 172.21.15.33:0/3434977594 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55f6fd9f76a0 con 0x55f6fd9e05d0 2024-05-03T18:29:27.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.322+0000 7f6d28c00700 1 -- 172.21.15.33:0/3434977594 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6d3001d070 con 0x55f6fd9e05d0 2024-05-03T18:29:27.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.322+0000 7f6d28c00700 1 -- 172.21.15.33:0/3434977594 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f6d2c004e40 con 0x55f6fd9e05d0 2024-05-03T18:29:27.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.322+0000 7f6d50049b80 10 client.15600 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:29:27.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.322+0000 7f6d50049b80 10 client.15600 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:29:27.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.322+0000 7f6d50049b80 20 client.15600 populate_metadata read hostname 'smithi033' 2024-05-03T18:29:27.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.322+0000 7f6d50049b80 1 --2- 172.21.15.33:0/3434977594 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55f6fd9f9c30 0x55f6fd9fc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:27.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.322+0000 7f6d50049b80 4 client.15600 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:29:27.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.322+0000 7f6d50049b80 1 -- 172.21.15.33:0/3434977594 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f6fd9ad1d0 con 0x55f6fd9f9c30 2024-05-03T18:29:27.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.323+0000 7f6d3ea00700 1 --2- 172.21.15.33:0/3434977594 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55f6fd9f9c30 0x55f6fd9fc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:27.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.324+0000 7f6d3ea00700 1 --2- 172.21.15.33:0/3434977594 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55f6fd9f9c30 0x55f6fd9fc040 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aeaa7704b8307527 server_cookie=5d7be28c6abec89a in_seq=0 out_seq=0 2024-05-03T18:29:27.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.324+0000 7f6d28c00700 10 client.15600 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:29:27.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.326+0000 7f6d28c00700 1 -- 172.21.15.33:0/3434977594 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f6fd9ad1d0 con 0x55f6fd9f9c30 2024-05-03T18:29:27.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.326+0000 7f6d28c00700 10 client.15600 handle_command_reply: tid=0 2024-05-03T18:29:27.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.326+0000 7f6d50049b80 1 client.15600 shutdown 2024-05-03T18:29:27.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.327+0000 7f6d50049b80 1 -- 172.21.15.33:0/3434977594 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f6fd9e05d0 msgr2=0x55f6fd9ec0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:27.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.327+0000 7f6d50049b80 1 --2- 172.21.15.33:0/3434977594 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f6fd9e05d0 0x55f6fd9ec0b0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c0044c0 tx=0x7f6d2c0047c0 comp rx=0 tx=0).stop 2024-05-03T18:29:27.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.327+0000 7f6d50049b80 1 -- 172.21.15.33:0/3434977594 shutdown_connections 2024-05-03T18:29:27.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.327+0000 7f6d50049b80 1 --2- 172.21.15.33:0/3434977594 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55f6fd9f9c30 0x55f6fd9fc040 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:27.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.327+0000 7f6d50049b80 1 --2- 172.21.15.33:0/3434977594 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f6fd9e6360 0x55f6fd9e9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:27.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.327+0000 7f6d28c00700 0 client.15600 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:29:27.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.327+0000 7f6d50049b80 1 --2- 172.21.15.33:0/3434977594 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f6fd9e05d0 0x55f6fd9ec0b0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:27.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.327+0000 7f6d50049b80 1 --2- 172.21.15.33:0/3434977594 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f6fd9e5a40 0x55f6fd9e9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:27.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.327+0000 7f6d50049b80 1 -- 172.21.15.33:0/3434977594 >> 172.21.15.33:0/3434977594 conn(0x55f6fd90eb60 msgr2=0x55f6fd9da080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:29:27.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.329+0000 7f6d50049b80 1 -- 172.21.15.33:0/3434977594 shutdown_connections 2024-05-03T18:29:27.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.329+0000 7f6d50049b80 1 -- 172.21.15.33:0/3434977594 wait complete. 2024-05-03T18:29:27.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.329+0000 7f6d50049b80 20 client.15600 trim_cache size 0 max 16384 2024-05-03T18:29:27.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.350+0000 7f6d50049b80 1 Processor -- start 2024-05-03T18:29:27.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.350+0000 7f6d50049b80 1 -- start start 2024-05-03T18:29:27.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.351+0000 7f6d50049b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f6fd9e05d0 0x55f6fd9dffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:27.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.351+0000 7f6d50049b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f6fd9e5a40 0x55f6fd9f5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:27.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.351+0000 7f6d50049b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f6fd9e6360 0x55f6fd9e8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:27.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.351+0000 7f6d50049b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6d401aafe0 con 0x55f6fd9e5a40 2024-05-03T18:29:27.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.351+0000 7f6d50049b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6d401ad860 con 0x55f6fd9e6360 2024-05-03T18:29:27.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.351+0000 7f6d50049b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6d401ab650 con 0x55f6fd9e05d0 2024-05-03T18:29:27.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.351+0000 7f6d3ea00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f6fd9e5a40 0x55f6fd9f5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:27.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.351+0000 7f6d3ea00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f6fd9e5a40 0x55f6fd9f5e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45692/0 (socket says 172.21.15.33:45692) 2024-05-03T18:29:27.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.351+0000 7f6d3ea00700 1 -- 172.21.15.33:0/816632353 learned_addr learned my addr 172.21.15.33:0/816632353 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:29:27.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.352+0000 7f6d3e000700 1 --2- 172.21.15.33:0/816632353 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f6fd9e05d0 0x55f6fd9dffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:27.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.352+0000 7f6d3e000700 1 --2- 172.21.15.33:0/816632353 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f6fd9e6360 0x55f6fd9e8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:27.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.352+0000 7f6d3ea00700 1 -- 172.21.15.33:0/816632353 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f6fd9e05d0 msgr2=0x55f6fd9dffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:29:27.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.352+0000 7f6d3ea00700 1 --2- 172.21.15.33:0/816632353 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f6fd9e05d0 0x55f6fd9dffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:27.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.352+0000 7f6d3ea00700 1 -- 172.21.15.33:0/816632353 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f6fd9e6360 msgr2=0x55f6fd9e8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:29:27.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.352+0000 7f6d3ea00700 1 --2- 172.21.15.33:0/816632353 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f6fd9e6360 0x55f6fd9e8cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:27.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.352+0000 7f6d3ea00700 1 -- 172.21.15.33:0/816632353 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6fd9e9210 con 0x55f6fd9e5a40 2024-05-03T18:29:27.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.352+0000 7f6d3e000700 1 --2- 172.21.15.33:0/816632353 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f6fd9e6360 0x55f6fd9e8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:29:27.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.352+0000 7f6d3ea00700 1 --2- 172.21.15.33:0/816632353 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f6fd9e5a40 0x55f6fd9f5e60 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x55f6fd9eaa20 tx=0x7f6d3000b700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b563a7de6dcb64c server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:29:27.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.353+0000 7f6d2b400700 1 -- 172.21.15.33:0/816632353 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d30003c80 con 0x55f6fd9e5a40 2024-05-03T18:29:27.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.353+0000 7f6d2b400700 1 -- 172.21.15.33:0/816632353 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6d3001d040 con 0x55f6fd9e5a40 2024-05-03T18:29:27.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.353+0000 7f6d2b400700 1 -- 172.21.15.33:0/816632353 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d300052b0 con 0x55f6fd9e5a40 2024-05-03T18:29:27.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.353+0000 7f6d50049b80 1 -- 172.21.15.33:0/816632353 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f6fd9e5a40 msgr2=0x55f6fd9f5e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:27.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.353+0000 7f6d50049b80 1 --2- 172.21.15.33:0/816632353 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f6fd9e5a40 0x55f6fd9f5e60 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x55f6fd9eaa20 tx=0x7f6d3000b700 comp rx=0 tx=0).stop 2024-05-03T18:29:27.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.354+0000 7f6d50049b80 1 -- 172.21.15.33:0/816632353 shutdown_connections 2024-05-03T18:29:27.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.354+0000 7f6d50049b80 1 --2- 172.21.15.33:0/816632353 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f6fd9e6360 0x55f6fd9e8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:27.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.354+0000 7f6d50049b80 1 --2- 172.21.15.33:0/816632353 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f6fd9e05d0 0x55f6fd9dffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:27.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.354+0000 7f6d50049b80 1 --2- 172.21.15.33:0/816632353 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f6fd9e5a40 0x55f6fd9f5e60 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:27.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.354+0000 7f6d50049b80 1 -- 172.21.15.33:0/816632353 >> 172.21.15.33:0/816632353 conn(0x55f6fd90eb60 msgr2=0x55f6fd9e34e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:29:27.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.354+0000 7f6d50049b80 1 -- 172.21.15.33:0/816632353 shutdown_connections 2024-05-03T18:29:27.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.354+0000 7f6d50049b80 1 -- 172.21.15.33:0/816632353 wait complete. 2024-05-03T18:29:27.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.354+0000 7f6d50049b80 1 -- start start 2024-05-03T18:29:27.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.355+0000 7f6d50049b80 1 Processor -- start 2024-05-03T18:29:27.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.355+0000 7f6d50049b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f6fd9e05d0 0x55f6fd9f24e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:27.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.355+0000 7f6d50049b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f6fd9e5a40 0x55f6fda00d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:27.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.355+0000 7f6d50049b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f6fd9e6360 0x55f6fda012d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:27.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.355+0000 7f6d50049b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6d401abd20 con 0x55f6fd9e6360 2024-05-03T18:29:27.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.355+0000 7f6d50049b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6d401ad190 con 0x55f6fd9e5a40 2024-05-03T18:29:27.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.355+0000 7f6d50049b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6d401ac3f0 con 0x55f6fd9e05d0 2024-05-03T18:29:27.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.355+0000 7f6d50049b80 10 client.0 _ll_register_callbacks cb 0x55f6fd8d9c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:29:27.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.356+0000 7f6d3e000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f6fd9e6360 0x55f6fda012d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:27.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.356+0000 7f6d3e000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f6fd9e6360 0x55f6fda012d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45694/0 (socket says 172.21.15.33:45694) 2024-05-03T18:29:27.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.356+0000 7f6d3e000700 1 -- 172.21.15.33:0/817297749 learned_addr learned my addr 172.21.15.33:0/817297749 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:29:27.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.356+0000 7f6d3e000700 1 --2- 172.21.15.33:0/817297749 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f6fd9e05d0 0x55f6fd9f24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:27.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.356+0000 7f6d3ea00700 1 --2- 172.21.15.33:0/817297749 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f6fd9e5a40 0x55f6fda00d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:27.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.356+0000 7f6d3e000700 1 -- 172.21.15.33:0/817297749 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f6fd9e05d0 msgr2=0x55f6fd9f24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:29:27.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.356+0000 7f6d3e000700 1 --2- 172.21.15.33:0/817297749 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f6fd9e05d0 0x55f6fd9f24e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:27.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.356+0000 7f6d3e000700 1 -- 172.21.15.33:0/817297749 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f6fd9e5a40 msgr2=0x55f6fda00d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:29:27.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.357+0000 7f6d3e000700 1 --2- 172.21.15.33:0/817297749 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f6fd9e5a40 0x55f6fda00d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:27.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.357+0000 7f6d3e000700 1 -- 172.21.15.33:0/817297749 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6fda01890 con 0x55f6fd9e6360 2024-05-03T18:29:27.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.357+0000 7f6d3ea00700 1 --2- 172.21.15.33:0/817297749 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f6fd9e5a40 0x55f6fda00d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:29:27.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.357+0000 7f6d3e000700 1 --2- 172.21.15.33:0/817297749 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f6fd9e6360 0x55f6fda012d0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c004410 tx=0x7f6d2c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1867a515176fd1c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:29:27.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.357+0000 7f6d28c00700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:29:27.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.357+0000 7f6d28c00700 1 -- 172.21.15.33:0/817297749 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d2c005660 con 0x55f6fd9e6360 2024-05-03T18:29:27.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.357+0000 7f6d50049b80 1 -- 172.21.15.33:0/817297749 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d3001d040 con 0x55f6fd9e6360 2024-05-03T18:29:27.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.357+0000 7f6d28c00700 1 -- 172.21.15.33:0/817297749 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f6d2c011650 con 0x55f6fd9e6360 2024-05-03T18:29:27.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.358+0000 7f6d28c00700 1 -- 172.21.15.33:0/817297749 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f6d2c03c050 con 0x55f6fd9e6360 2024-05-03T18:29:27.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.358+0000 7f6d50049b80 10 client.15606 fetch_fsmap learned FSMap version 21 2024-05-03T18:29:27.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.358+0000 7f6d50049b80 1 -- 172.21.15.33:0/817297749 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55f6fda64250 con 0x55f6fd9e6360 2024-05-03T18:29:27.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.358+0000 7f6d28c00700 1 -- 172.21.15.33:0/817297749 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6d3001d040 con 0x55f6fd9e6360 2024-05-03T18:29:27.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.358+0000 7f6d28c00700 1 -- 172.21.15.33:0/817297749 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f6d2c004a60 con 0x55f6fd9e6360 2024-05-03T18:29:27.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.358+0000 7f6d50049b80 10 client.15606 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:29:27.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.358+0000 7f6d50049b80 10 client.15606 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:29:27.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.358+0000 7f6d50049b80 20 client.15606 populate_metadata read hostname 'smithi033' 2024-05-03T18:29:27.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.358+0000 7f6d50049b80 1 --2- 172.21.15.33:0/817297749 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55f6fd9f9c30 0x55f6fda669a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:27.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.358+0000 7f6d50049b80 4 client.15606 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-03T18:29:27.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.358+0000 7f6d50049b80 1 -- 172.21.15.33:0/817297749 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f6fd940400 con 0x55f6fd9f9c30 2024-05-03T18:29:27.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.359+0000 7f6d3ea00700 1 --2- 172.21.15.33:0/817297749 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55f6fd9f9c30 0x55f6fda669a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:27.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.360+0000 7f6d3ea00700 1 --2- 172.21.15.33:0/817297749 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55f6fd9f9c30 0x55f6fda669a0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38ad9999b7c8f1f4 server_cookie=e4bb58097c380c01 in_seq=0 out_seq=0 2024-05-03T18:29:27.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.360+0000 7f6d28c00700 10 client.15606 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:29:27.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.361+0000 7f6d28c00700 1 -- 172.21.15.33:0/817297749 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f6fd940400 con 0x55f6fd9f9c30 2024-05-03T18:29:27.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.361+0000 7f6d28c00700 10 client.15606 handle_command_reply: tid=0 2024-05-03T18:29:27.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.361+0000 7f6d50049b80 1 client.15606 shutdown 2024-05-03T18:29:27.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.361+0000 7f6d50049b80 1 -- 172.21.15.33:0/817297749 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f6fd9e6360 msgr2=0x55f6fda012d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:27.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.361+0000 7f6d50049b80 1 --2- 172.21.15.33:0/817297749 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f6fd9e6360 0x55f6fda012d0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c004410 tx=0x7f6d2c0047c0 comp rx=0 tx=0).stop 2024-05-03T18:29:27.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.361+0000 7f6d50049b80 1 -- 172.21.15.33:0/817297749 shutdown_connections 2024-05-03T18:29:27.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.361+0000 7f6d50049b80 1 --2- 172.21.15.33:0/817297749 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55f6fd9f9c30 0x55f6fda669a0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:27.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.361+0000 7f6d50049b80 1 --2- 172.21.15.33:0/817297749 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f6fd9e6360 0x55f6fda012d0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:27.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.361+0000 7f6d50049b80 1 --2- 172.21.15.33:0/817297749 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f6fd9e05d0 0x55f6fd9f24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:27.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.361+0000 7f6d50049b80 1 --2- 172.21.15.33:0/817297749 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f6fd9e5a40 0x55f6fda00d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:27.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.361+0000 7f6d28c00700 0 client.15606 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:29:27.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.361+0000 7f6d50049b80 1 -- 172.21.15.33:0/817297749 >> 172.21.15.33:0/817297749 conn(0x55f6fd90eb60 msgr2=0x55f6fd9e6740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:29:27.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.361+0000 7f6d50049b80 1 -- 172.21.15.33:0/817297749 shutdown_connections 2024-05-03T18:29:27.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.361+0000 7f6d50049b80 1 -- 172.21.15.33:0/817297749 wait complete. 2024-05-03T18:29:27.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.361+0000 7f6d50049b80 20 client.15606 trim_cache size 0 max 16384 2024-05-03T18:29:27.363 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-03T18:29:27.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.362+0000 7f6d3fe00700 1 -- 172.21.15.33:0/1910653322 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f6d240b2a70 msgr2=0x7f6d240b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:27.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.362+0000 7f6d3fe00700 1 --2- 172.21.15.33:0/1910653322 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f6d240b2a70 0x7f6d240b4f30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6d380e1180 tx=0x7f6d3800b3e0 comp rx=0 tx=0).stop 2024-05-03T18:29:27.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.362+0000 7f6d3fe00700 1 -- 172.21.15.33:0/1910653322 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d40107660 msgr2=0x7f6d40199170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:27.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.362+0000 7f6d3fe00700 1 --2- 172.21.15.33:0/1910653322 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d40107660 0x7f6d40199170 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f6d38007d10 tx=0x7f6d38007d40 comp rx=0 tx=0).stop 2024-05-03T18:29:27.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.362+0000 7f6d3fe00700 1 -- 172.21.15.33:0/1910653322 shutdown_connections 2024-05-03T18:29:27.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.362+0000 7f6d3fe00700 1 --2- 172.21.15.33:0/1910653322 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d40107660 0x7f6d40199170 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:27.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.362+0000 7f6d3fe00700 1 --2- 172.21.15.33:0/1910653322 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f6d240b2a70 0x7f6d240b4f30 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:27.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.362+0000 7f6d3fe00700 1 --2- 172.21.15.33:0/1910653322 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d40102580 0x7f6d401986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:27.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.362+0000 7f6d3fe00700 1 --2- 172.21.15.33:0/1910653322 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d40102f30 0x7f6d40198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:27.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.362+0000 7f6d3fe00700 1 -- 172.21.15.33:0/1910653322 >> 172.21.15.33:0/1910653322 conn(0x7f6d400fde30 msgr2=0x7f6d40100180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:29:27.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.363+0000 7f6d3fe00700 1 -- 172.21.15.33:0/1910653322 shutdown_connections 2024-05-03T18:29:27.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:27.363+0000 7f6d3fe00700 1 -- 172.21.15.33:0/1910653322 wait complete. 2024-05-03T18:29:29.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:29:28 smithi033 ceph-mon[28330]: pgmap v71: 129 pgs: 129 active+clean; 489 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 18 MiB/s wr, 18 op/s 2024-05-03T18:29:29.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:29:28 smithi099 ceph-mon[31658]: pgmap v71: 129 pgs: 129 active+clean; 489 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 18 MiB/s wr, 18 op/s 2024-05-03T18:29:29.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:29:28 smithi150 ceph-mon[32492]: pgmap v71: 129 pgs: 129 active+clean; 489 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 18 MiB/s wr, 18 op/s 2024-05-03T18:29:29.417 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-03T18:29:29.417 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-03T18:29:29.927 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:29:30.483 INFO:teuthology.orchestra.run.smithi033.stderr:1900000 of 3200000 tuples (59%) done (elapsed 67.59 s, remaining 46.25 s) 2024-05-03T18:29:31.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:29:30 smithi033 ceph-mon[28330]: pgmap v72: 129 pgs: 129 active+clean; 489 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 11 MiB/s wr, 12 op/s 2024-05-03T18:29:31.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:29:30 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:29:31.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:29:30 smithi099 ceph-mon[31658]: pgmap v72: 129 pgs: 129 active+clean; 489 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 11 MiB/s wr, 12 op/s 2024-05-03T18:29:31.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:29:30 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:29:31.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:29:30 smithi150 ceph-mon[32492]: pgmap v72: 129 pgs: 129 active+clean; 489 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 11 MiB/s wr, 12 op/s 2024-05-03T18:29:31.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:29:30 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:29:32.135 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:29:31 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:29:32.135 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:29:31 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:29:32.135 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:29:31 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:29:32.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:29:31 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:29:32.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:29:31 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:29:32.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:29:31 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:29:32.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.340+0000 7f01a8200700 1 -- 172.21.15.33:0/3753101098 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01a0075ff0 msgr2=0x7f01a010f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:32.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.340+0000 7f01a8200700 1 --2- 172.21.15.33:0/3753101098 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01a0075ff0 0x7f01a010f960 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f018c009990 tx=0x7f018c009ca0 comp rx=0 tx=0).stop 2024-05-03T18:29:32.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.340+0000 7f01a8200700 1 -- 172.21.15.33:0/3753101098 shutdown_connections 2024-05-03T18:29:32.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.340+0000 7f01a8200700 1 --2- 172.21.15.33:0/3753101098 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01a010fea0 0x7f01a0112290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:32.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.340+0000 7f01a8200700 1 --2- 172.21.15.33:0/3753101098 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f01a00756d0 0x7f01a0075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:32.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.340+0000 7f01a8200700 1 --2- 172.21.15.33:0/3753101098 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01a0075ff0 0x7f01a010f960 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:32.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.340+0000 7f01a8200700 1 -- 172.21.15.33:0/3753101098 >> 172.21.15.33:0/3753101098 conn(0x7f01a007a600 msgr2=0x7f01a007aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:29:32.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.340+0000 7f01a8200700 1 -- 172.21.15.33:0/3753101098 shutdown_connections 2024-05-03T18:29:32.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.340+0000 7f01a8200700 1 -- 172.21.15.33:0/3753101098 wait complete. 2024-05-03T18:29:32.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.341+0000 7f01a8200700 1 Processor -- start 2024-05-03T18:29:32.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.342+0000 7f01a8200700 1 -- start start 2024-05-03T18:29:32.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.342+0000 7f01a8200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f01a00756d0 0x7f01a01a1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:32.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.342+0000 7f01a8200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01a0075ff0 0x7f01a01a1e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:32.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.342+0000 7f01a8200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01a010fea0 0x7f01a019bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:32.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.342+0000 7f01a8200700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f01a0115730 con 0x7f01a0075ff0 2024-05-03T18:29:32.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.342+0000 7f01a8200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f01a01155b0 con 0x7f01a010fea0 2024-05-03T18:29:32.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.342+0000 7f01a8200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f01a01158b0 con 0x7f01a00756d0 2024-05-03T18:29:32.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.342+0000 7f01a4e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01a0075ff0 0x7f01a01a1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:32.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.343+0000 7f01a4e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01a0075ff0 0x7f01a01a1e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45710/0 (socket says 172.21.15.33:45710) 2024-05-03T18:29:32.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.343+0000 7f01a4e00700 1 -- 172.21.15.33:0/108970077 learned_addr learned my addr 172.21.15.33:0/108970077 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:29:32.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.343+0000 7f01a5800700 1 --2- 172.21.15.33:0/108970077 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f01a00756d0 0x7f01a01a1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:32.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.343+0000 7f01a6200700 1 --2- 172.21.15.33:0/108970077 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01a010fea0 0x7f01a019bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:32.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.343+0000 7f01a4e00700 1 -- 172.21.15.33:0/108970077 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f01a00756d0 msgr2=0x7f01a01a1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:32.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.343+0000 7f01a4e00700 1 --2- 172.21.15.33:0/108970077 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f01a00756d0 0x7f01a01a1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:32.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.343+0000 7f01a4e00700 1 -- 172.21.15.33:0/108970077 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01a010fea0 msgr2=0x7f01a019bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:32.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.343+0000 7f01a4e00700 1 --2- 172.21.15.33:0/108970077 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01a010fea0 0x7f01a019bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:32.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.343+0000 7f01a4e00700 1 -- 172.21.15.33:0/108970077 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01a019c220 con 0x7f01a0075ff0 2024-05-03T18:29:32.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.344+0000 7f01a4e00700 1 --2- 172.21.15.33:0/108970077 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01a0075ff0 0x7f01a01a1e40 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f018c0057c0 tx=0x7f018c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:29:32.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.344+0000 7f019e000700 1 -- 172.21.15.33:0/108970077 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f018c027070 con 0x7f01a0075ff0 2024-05-03T18:29:32.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.344+0000 7f019e000700 1 -- 172.21.15.33:0/108970077 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f018c00fc00 con 0x7f01a0075ff0 2024-05-03T18:29:32.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.344+0000 7f01a8200700 1 -- 172.21.15.33:0/108970077 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01a019c4b0 con 0x7f01a0075ff0 2024-05-03T18:29:32.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.344+0000 7f019e000700 1 -- 172.21.15.33:0/108970077 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f018c022b30 con 0x7f01a0075ff0 2024-05-03T18:29:32.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.344+0000 7f01a8200700 1 -- 172.21.15.33:0/108970077 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01a01a87d0 con 0x7f01a0075ff0 2024-05-03T18:29:32.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.348+0000 7f019e000700 1 -- 172.21.15.33:0/108970077 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f018c02a070 con 0x7f01a0075ff0 2024-05-03T18:29:32.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.349+0000 7f01b2b71b80 1 Processor -- start 2024-05-03T18:29:32.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.350+0000 7f01b2b71b80 1 -- start start 2024-05-03T18:29:32.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.350+0000 7f01b2b71b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5621c988b9b0 0x5621c988bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:32.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.350+0000 7f01b2b71b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621c988c2d0 0x5621c9885fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:32.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.350+0000 7f01b2b71b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621c98865d0 0x5621c9886a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:32.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.350+0000 7f01b2b71b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5621c97d8aa0 con 0x5621c98865d0 2024-05-03T18:29:32.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.350+0000 7f01b2b71b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5621c97767e0 con 0x5621c988c2d0 2024-05-03T18:29:32.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.350+0000 7f01b2b71b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5621c97d4a60 con 0x5621c988b9b0 2024-05-03T18:29:32.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.351+0000 7f01a4e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621c98865d0 0x5621c9886a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:32.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.351+0000 7f019e000700 1 --2- 172.21.15.33:0/108970077 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f01880b2a70 0x7f01880b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:32.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.351+0000 7f01a4e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621c98865d0 0x5621c9886a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45716/0 (socket says 172.21.15.33:45716) 2024-05-03T18:29:32.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.351+0000 7f01a4e00700 1 -- 172.21.15.33:0/147435843 learned_addr learned my addr 172.21.15.33:0/147435843 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:29:32.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.351+0000 7f01a6200700 1 --2- 172.21.15.33:0/147435843 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5621c988b9b0 0x5621c988bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:32.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.351+0000 7f01a5800700 1 --2- 172.21.15.33:0/147435843 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621c988c2d0 0x5621c9885fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:32.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.351+0000 7f01a6200700 1 --2- 172.21.15.33:0/108970077 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f01880b2a70 0x7f01880b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:32.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.351+0000 7f019e000700 1 -- 172.21.15.33:0/108970077 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f018c0e38f0 con 0x7f01a0075ff0 2024-05-03T18:29:32.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.352+0000 7f01a6200700 1 -- 172.21.15.33:0/147435843 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621c988c2d0 msgr2=0x5621c9885fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:29:32.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.352+0000 7f01a6200700 1 --2- 172.21.15.33:0/147435843 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621c988c2d0 0x5621c9885fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:32.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.352+0000 7f01a6200700 1 -- 172.21.15.33:0/147435843 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621c98865d0 msgr2=0x5621c9886a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:29:32.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.352+0000 7f01a6200700 1 --2- 172.21.15.33:0/147435843 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621c98865d0 0x5621c9886a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:32.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.352+0000 7f01a6200700 1 -- 172.21.15.33:0/147435843 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621c9725950 con 0x5621c988b9b0 2024-05-03T18:29:32.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.352+0000 7f01a6200700 1 --2- 172.21.15.33:0/147435843 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5621c988b9b0 0x5621c988bd90 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0198007950 tx=0x7f0198007c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2312fa24d9b0282 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:29:32.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.353+0000 7f01a6200700 1 --2- 172.21.15.33:0/108970077 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f01880b2a70 0x7f01880b4f30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f019801a000 tx=0x7f019801c000 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:29:32.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.353+0000 7f0192a00700 1 -- 172.21.15.33:0/147435843 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f019801d040 con 0x5621c988b9b0 2024-05-03T18:29:32.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.353+0000 7f0192a00700 1 -- 172.21.15.33:0/147435843 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0198009ca0 con 0x5621c988b9b0 2024-05-03T18:29:32.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.353+0000 7f0192a00700 1 -- 172.21.15.33:0/147435843 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0198012040 con 0x5621c988b9b0 2024-05-03T18:29:32.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.354+0000 7f01b2b71b80 1 -- 172.21.15.33:0/147435843 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5621c988b9b0 msgr2=0x5621c988bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:32.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.354+0000 7f01b2b71b80 1 --2- 172.21.15.33:0/147435843 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5621c988b9b0 0x5621c988bd90 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0198007950 tx=0x7f0198007c60 comp rx=0 tx=0).stop 2024-05-03T18:29:32.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.354+0000 7f01b2b71b80 1 -- 172.21.15.33:0/147435843 shutdown_connections 2024-05-03T18:29:32.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.354+0000 7f01b2b71b80 1 --2- 172.21.15.33:0/147435843 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621c98865d0 0x5621c9886a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:32.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.354+0000 7f01b2b71b80 1 --2- 172.21.15.33:0/147435843 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5621c988b9b0 0x5621c988bd90 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:32.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.354+0000 7f01b2b71b80 1 --2- 172.21.15.33:0/147435843 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621c988c2d0 0x5621c9885fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:32.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.354+0000 7f01b2b71b80 1 -- 172.21.15.33:0/147435843 >> 172.21.15.33:0/147435843 conn(0x5621c97fe750 msgr2=0x5621c9880980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:29:32.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.354+0000 7f01b2b71b80 1 -- 172.21.15.33:0/147435843 shutdown_connections 2024-05-03T18:29:32.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.354+0000 7f01b2b71b80 1 -- 172.21.15.33:0/147435843 wait complete. 2024-05-03T18:29:32.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.355+0000 7f01b2b71b80 1 -- start start 2024-05-03T18:29:32.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.355+0000 7f01b2b71b80 1 Processor -- start 2024-05-03T18:29:32.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.355+0000 7f01b2b71b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621c98865d0 0x5621c9891f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:32.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.356+0000 7f01b2b71b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5621c988b9b0 0x5621c988f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:32.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.356+0000 7f01b2b71b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621c988c2d0 0x5621c988f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:32.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.356+0000 7f01b2b71b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5621c9892d20 con 0x5621c98865d0 2024-05-03T18:29:32.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.356+0000 7f01b2b71b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5621c973eba0 con 0x5621c988c2d0 2024-05-03T18:29:32.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.356+0000 7f01b2b71b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5621c9892ea0 con 0x5621c988b9b0 2024-05-03T18:29:32.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.356+0000 7f01b2b71b80 10 client.0 _ll_register_callbacks cb 0x5621c977fcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:29:32.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.356+0000 7f01a5800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621c98865d0 0x5621c9891f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:32.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.356+0000 7f01a5800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621c98865d0 0x5621c9891f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45720/0 (socket says 172.21.15.33:45720) 2024-05-03T18:29:32.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.356+0000 7f01a5800700 1 -- 172.21.15.33:0/2452213074 learned_addr learned my addr 172.21.15.33:0/2452213074 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:29:32.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.356+0000 7f01a4e00700 1 --2- 172.21.15.33:0/2452213074 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5621c988b9b0 0x5621c988f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:32.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.356+0000 7f01a5800700 1 -- 172.21.15.33:0/2452213074 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5621c988b9b0 msgr2=0x5621c988f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:29:32.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.356+0000 7f01a5800700 1 --2- 172.21.15.33:0/2452213074 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5621c988b9b0 0x5621c988f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:32.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.356+0000 7f01a5800700 1 -- 172.21.15.33:0/2452213074 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621c988c2d0 msgr2=0x5621c988f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:29:32.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.356+0000 7f01a5800700 1 --2- 172.21.15.33:0/2452213074 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621c988c2d0 0x5621c988f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:32.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.356+0000 7f01a5800700 1 -- 172.21.15.33:0/2452213074 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621c979e4a0 con 0x5621c98865d0 2024-05-03T18:29:32.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.357+0000 7f01a5800700 1 --2- 172.21.15.33:0/2452213074 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621c98865d0 0x5621c9891f00 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f019400b9b0 tx=0x7f019400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=526205b77943e8a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:29:32.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.357+0000 7f0187400700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:29:32.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.357+0000 7f0187400700 1 -- 172.21.15.33:0/2452213074 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f019401b440 con 0x5621c98865d0 2024-05-03T18:29:32.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.357+0000 7f0187400700 1 -- 172.21.15.33:0/2452213074 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f0194011440 con 0x5621c98865d0 2024-05-03T18:29:32.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.357+0000 7f01b2b71b80 1 -- 172.21.15.33:0/2452213074 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0198009ca0 con 0x5621c98865d0 2024-05-03T18:29:32.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.358+0000 7f0187400700 1 -- 172.21.15.33:0/2452213074 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f0194011640 con 0x5621c98865d0 2024-05-03T18:29:32.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.358+0000 7f01b2b71b80 10 client.15618 fetch_fsmap learned FSMap version 21 2024-05-03T18:29:32.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.358+0000 7f01b2b71b80 1 -- 172.21.15.33:0/2452213074 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5621c989d510 con 0x5621c98865d0 2024-05-03T18:29:32.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.358+0000 7f0187400700 1 -- 172.21.15.33:0/2452213074 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0198009ca0 con 0x5621c98865d0 2024-05-03T18:29:32.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.358+0000 7f0187400700 1 -- 172.21.15.33:0/2452213074 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f0194019020 con 0x5621c98865d0 2024-05-03T18:29:32.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.359+0000 7f01b2b71b80 10 client.15618 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:29:32.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.359+0000 7f01b2b71b80 10 client.15618 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:29:32.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.359+0000 7f01b2b71b80 20 client.15618 populate_metadata read hostname 'smithi033' 2024-05-03T18:29:32.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.359+0000 7f01b2b71b80 1 --2- 172.21.15.33:0/2452213074 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5621c989fb50 0x5621c98a1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:32.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.359+0000 7f01b2b71b80 4 client.15618 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:29:32.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.359+0000 7f01b2b71b80 1 -- 172.21.15.33:0/2452213074 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5621c9853070 con 0x5621c989fb50 2024-05-03T18:29:32.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.359+0000 7f01a4e00700 1 --2- 172.21.15.33:0/2452213074 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5621c989fb50 0x5621c98a1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:32.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.360+0000 7f01a4e00700 1 --2- 172.21.15.33:0/2452213074 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5621c989fb50 0x5621c98a1f40 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=288088a0fb128e62 server_cookie=c1857374baab542b in_seq=0 out_seq=0 2024-05-03T18:29:32.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.360+0000 7f0187400700 10 client.15618 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:29:32.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.362+0000 7f0187400700 1 -- 172.21.15.33:0/2452213074 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5621c9853070 con 0x5621c989fb50 2024-05-03T18:29:32.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.362+0000 7f0187400700 10 client.15618 handle_command_reply: tid=0 2024-05-03T18:29:32.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.363+0000 7f01b2b71b80 1 client.15618 shutdown 2024-05-03T18:29:32.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.363+0000 7f01b2b71b80 1 -- 172.21.15.33:0/2452213074 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621c98865d0 msgr2=0x5621c9891f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:32.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.363+0000 7f01b2b71b80 1 --2- 172.21.15.33:0/2452213074 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621c98865d0 0x5621c9891f00 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f019400b9b0 tx=0x7f019400bd70 comp rx=0 tx=0).stop 2024-05-03T18:29:32.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.363+0000 7f01b2b71b80 1 -- 172.21.15.33:0/2452213074 shutdown_connections 2024-05-03T18:29:32.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.363+0000 7f01b2b71b80 1 --2- 172.21.15.33:0/2452213074 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5621c989fb50 0x5621c98a1f40 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:32.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.363+0000 7f01b2b71b80 1 --2- 172.21.15.33:0/2452213074 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621c988c2d0 0x5621c988f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:32.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.363+0000 7f01b2b71b80 1 --2- 172.21.15.33:0/2452213074 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621c98865d0 0x5621c9891f00 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:32.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.363+0000 7f01b2b71b80 1 --2- 172.21.15.33:0/2452213074 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5621c988b9b0 0x5621c988f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:32.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.363+0000 7f0187400700 0 client.15618 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:29:32.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.363+0000 7f01b2b71b80 1 -- 172.21.15.33:0/2452213074 >> 172.21.15.33:0/2452213074 conn(0x5621c97fe750 msgr2=0x5621c987b430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:29:32.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.363+0000 7f01b2b71b80 1 -- 172.21.15.33:0/2452213074 shutdown_connections 2024-05-03T18:29:32.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.364+0000 7f01b2b71b80 1 -- 172.21.15.33:0/2452213074 wait complete. 2024-05-03T18:29:32.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.365+0000 7f01b2b71b80 20 client.15618 trim_cache size 0 max 16384 2024-05-03T18:29:32.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:29:31 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:29:32.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:29:31 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:29:32.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:29:31 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:29:32.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.395+0000 7f01b2b71b80 1 Processor -- start 2024-05-03T18:29:32.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.395+0000 7f01b2b71b80 1 -- start start 2024-05-03T18:29:32.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.396+0000 7f01b2b71b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5621c98865d0 0x5621c988cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:32.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.396+0000 7f01b2b71b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621c988b9b0 0x5621c98a7800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:32.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.396+0000 7f01b2b71b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621c988c2d0 0x5621c98a7d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:32.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.396+0000 7f01b2b71b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f01a01af6d0 con 0x5621c988b9b0 2024-05-03T18:29:32.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.396+0000 7f01b2b71b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f01a01b1f50 con 0x5621c988c2d0 2024-05-03T18:29:32.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.396+0000 7f01b2b71b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f01a01afd40 con 0x5621c98865d0 2024-05-03T18:29:32.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.397+0000 7f01a4e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621c988b9b0 0x5621c98a7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:32.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.397+0000 7f01a5800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5621c98865d0 0x5621c988cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:32.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.397+0000 7f01a5800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5621c98865d0 0x5621c988cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:52118/0 (socket says 172.21.15.33:52118) 2024-05-03T18:29:32.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.397+0000 7f01a5800700 1 -- 172.21.15.33:0/1677093898 learned_addr learned my addr 172.21.15.33:0/1677093898 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:29:32.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.397+0000 7f01a4e00700 1 --2- 172.21.15.33:0/1677093898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621c988b9b0 0x5621c98a7800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45732/0 (socket says 172.21.15.33:45732) 2024-05-03T18:29:32.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.397+0000 7f01a6200700 1 --2- 172.21.15.33:0/1677093898 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621c988c2d0 0x5621c98a7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:32.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.397+0000 7f01a4e00700 1 -- 172.21.15.33:0/1677093898 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5621c98865d0 msgr2=0x5621c988cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:29:32.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.397+0000 7f01a4e00700 1 --2- 172.21.15.33:0/1677093898 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5621c98865d0 0x5621c988cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:32.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.397+0000 7f01a4e00700 1 -- 172.21.15.33:0/1677093898 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621c988c2d0 msgr2=0x5621c98a7d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:29:32.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.397+0000 7f01a4e00700 1 --2- 172.21.15.33:0/1677093898 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621c988c2d0 0x5621c98a7d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:32.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.397+0000 7f01a4e00700 1 -- 172.21.15.33:0/1677093898 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621c9892270 con 0x5621c988b9b0 2024-05-03T18:29:32.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.398+0000 7f01a4e00700 1 --2- 172.21.15.33:0/1677093898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621c988b9b0 0x5621c98a7800 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f018c0a7ae0 tx=0x7f018c02a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4be179f850d3fc9 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:29:32.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.398+0000 7f0192000700 1 -- 172.21.15.33:0/1677093898 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f018c0aaf70 con 0x5621c988b9b0 2024-05-03T18:29:32.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.398+0000 7f0192000700 1 -- 172.21.15.33:0/1677093898 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f018c0aa870 con 0x5621c988b9b0 2024-05-03T18:29:32.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.398+0000 7f0192000700 1 -- 172.21.15.33:0/1677093898 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f018c0b10b0 con 0x5621c988b9b0 2024-05-03T18:29:32.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.399+0000 7f01b2b71b80 1 -- 172.21.15.33:0/1677093898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621c988b9b0 msgr2=0x5621c98a7800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:32.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.399+0000 7f01b2b71b80 1 --2- 172.21.15.33:0/1677093898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621c988b9b0 0x5621c98a7800 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f018c0a7ae0 tx=0x7f018c02a000 comp rx=0 tx=0).stop 2024-05-03T18:29:32.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.399+0000 7f01b2b71b80 1 -- 172.21.15.33:0/1677093898 shutdown_connections 2024-05-03T18:29:32.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.399+0000 7f01b2b71b80 1 --2- 172.21.15.33:0/1677093898 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621c988c2d0 0x5621c98a7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:32.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.399+0000 7f01b2b71b80 1 --2- 172.21.15.33:0/1677093898 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5621c98865d0 0x5621c988cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:32.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.399+0000 7f01b2b71b80 1 --2- 172.21.15.33:0/1677093898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621c988b9b0 0x5621c98a7800 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:32.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.399+0000 7f01b2b71b80 1 -- 172.21.15.33:0/1677093898 >> 172.21.15.33:0/1677093898 conn(0x5621c97fe750 msgr2=0x5621c988c6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:29:32.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.399+0000 7f01b2b71b80 1 -- 172.21.15.33:0/1677093898 shutdown_connections 2024-05-03T18:29:32.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.400+0000 7f01b2b71b80 1 -- 172.21.15.33:0/1677093898 wait complete. 2024-05-03T18:29:32.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.400+0000 7f01b2b71b80 1 -- start start 2024-05-03T18:29:32.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.400+0000 7f01b2b71b80 1 Processor -- start 2024-05-03T18:29:32.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.401+0000 7f01b2b71b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5621c98865d0 0x5621c9895100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:32.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.401+0000 7f01b2b71b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621c988b9b0 0x5621c9895640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:32.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.401+0000 7f01b2b71b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621c988c2d0 0x5621c9895b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:32.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.401+0000 7f01b2b71b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f01a01b0410 con 0x5621c988b9b0 2024-05-03T18:29:32.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.401+0000 7f01b2b71b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f01a01b1880 con 0x5621c988c2d0 2024-05-03T18:29:32.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.401+0000 7f01b2b71b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f01a01b0ae0 con 0x5621c98865d0 2024-05-03T18:29:32.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.401+0000 7f01b2b71b80 10 client.0 _ll_register_callbacks cb 0x5621c977fcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:29:32.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.402+0000 7f01a4e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621c988b9b0 0x5621c9895640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:32.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.402+0000 7f01a5800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5621c98865d0 0x5621c9895100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:32.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.402+0000 7f01a6200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621c988c2d0 0x5621c9895b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:32.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.402+0000 7f01a5800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5621c98865d0 0x5621c9895100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:52130/0 (socket says 172.21.15.33:52130) 2024-05-03T18:29:32.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.402+0000 7f01a5800700 1 -- 172.21.15.33:0/2834602405 learned_addr learned my addr 172.21.15.33:0/2834602405 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:29:32.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.403+0000 7f01a5800700 1 -- 172.21.15.33:0/2834602405 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621c988c2d0 msgr2=0x5621c9895b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:29:32.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.403+0000 7f01a5800700 1 --2- 172.21.15.33:0/2834602405 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621c988c2d0 0x5621c9895b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:32.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.403+0000 7f01a5800700 1 -- 172.21.15.33:0/2834602405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621c988b9b0 msgr2=0x5621c9895640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:29:32.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.403+0000 7f01a5800700 1 --2- 172.21.15.33:0/2834602405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621c988b9b0 0x5621c9895640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:32.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.403+0000 7f01a5800700 1 -- 172.21.15.33:0/2834602405 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621c9912de0 con 0x5621c98865d0 2024-05-03T18:29:32.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.403+0000 7f01a4e00700 1 --2- 172.21.15.33:0/2834602405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621c988b9b0 0x5621c9895640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:29:32.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.404+0000 7f01a5800700 1 --2- 172.21.15.33:0/2834602405 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5621c98865d0 0x5621c9895100 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0194003a00 tx=0x7f0194003f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46bf6e64e8d4e75d server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:29:32.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.404+0000 7f0187400700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-03T18:29:32.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.404+0000 7f0187400700 1 -- 172.21.15.33:0/2834602405 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f019402a040 con 0x5621c98865d0 2024-05-03T18:29:32.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.404+0000 7f01b2b71b80 1 -- 172.21.15.33:0/2834602405 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f018c0aa870 con 0x5621c98865d0 2024-05-03T18:29:32.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.404+0000 7f0187400700 1 -- 172.21.15.33:0/2834602405 <== mon.1 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f019401b440 con 0x5621c98865d0 2024-05-03T18:29:32.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.405+0000 7f0187400700 1 -- 172.21.15.33:0/2834602405 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f019401b640 con 0x5621c98865d0 2024-05-03T18:29:32.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.405+0000 7f01b2b71b80 10 client.25306 fetch_fsmap learned FSMap version 21 2024-05-03T18:29:32.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.405+0000 7f01b2b71b80 1 -- 172.21.15.33:0/2834602405 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5621c99140e0 con 0x5621c98865d0 2024-05-03T18:29:32.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.405+0000 7f0187400700 1 -- 172.21.15.33:0/2834602405 <== mon.1 v2:172.21.15.150:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f018c0aa870 con 0x5621c98865d0 2024-05-03T18:29:32.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.406+0000 7f0187400700 1 -- 172.21.15.33:0/2834602405 <== mon.1 v2:172.21.15.150:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f0194019050 con 0x5621c98865d0 2024-05-03T18:29:32.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.406+0000 7f01b2b71b80 10 client.25306 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:29:32.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.406+0000 7f01b2b71b80 10 client.25306 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:29:32.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.406+0000 7f01b2b71b80 20 client.25306 populate_metadata read hostname 'smithi033' 2024-05-03T18:29:32.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.406+0000 7f01b2b71b80 1 --2- 172.21.15.33:0/2834602405 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5621c99147e0 0x5621c9916c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:32.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.406+0000 7f01b2b71b80 4 client.25306 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-03T18:29:32.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.406+0000 7f01b2b71b80 1 -- 172.21.15.33:0/2834602405 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5621c97e6480 con 0x5621c99147e0 2024-05-03T18:29:32.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.406+0000 7f01a4e00700 1 --2- 172.21.15.33:0/2834602405 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5621c99147e0 0x5621c9916c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:32.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.408+0000 7f01a4e00700 1 --2- 172.21.15.33:0/2834602405 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5621c99147e0 0x5621c9916c00 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2419ab4e92155885 server_cookie=a1c8ccb304cd77e2 in_seq=0 out_seq=0 2024-05-03T18:29:32.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.408+0000 7f0187400700 10 client.25306 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:29:32.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.411+0000 7f0187400700 1 -- 172.21.15.33:0/2834602405 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5621c97e6480 con 0x5621c99147e0 2024-05-03T18:29:32.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.411+0000 7f0187400700 10 client.25306 handle_command_reply: tid=0 2024-05-03T18:29:32.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.411+0000 7f01b2b71b80 1 client.25306 shutdown 2024-05-03T18:29:32.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.412+0000 7f01b2b71b80 1 -- 172.21.15.33:0/2834602405 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5621c98865d0 msgr2=0x5621c9895100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:32.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.412+0000 7f01b2b71b80 1 --2- 172.21.15.33:0/2834602405 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5621c98865d0 0x5621c9895100 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0194003a00 tx=0x7f0194003f90 comp rx=0 tx=0).stop 2024-05-03T18:29:32.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.412+0000 7f01b2b71b80 1 -- 172.21.15.33:0/2834602405 shutdown_connections 2024-05-03T18:29:32.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.412+0000 7f01b2b71b80 1 --2- 172.21.15.33:0/2834602405 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5621c99147e0 0x5621c9916c00 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:32.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.412+0000 7f01b2b71b80 1 --2- 172.21.15.33:0/2834602405 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621c988c2d0 0x5621c9895b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:32.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.412+0000 7f01b2b71b80 1 --2- 172.21.15.33:0/2834602405 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5621c98865d0 0x5621c9895100 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:32.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.412+0000 7f01b2b71b80 1 --2- 172.21.15.33:0/2834602405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621c988b9b0 0x5621c9895640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:32.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.412+0000 7f01b2b71b80 1 -- 172.21.15.33:0/2834602405 >> 172.21.15.33:0/2834602405 conn(0x5621c97fe750 msgr2=0x5621c987aba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:29:32.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.412+0000 7f0187400700 0 client.25306 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:29:32.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.412+0000 7f01b2b71b80 1 -- 172.21.15.33:0/2834602405 shutdown_connections 2024-05-03T18:29:32.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.412+0000 7f01b2b71b80 1 -- 172.21.15.33:0/2834602405 wait complete. 2024-05-03T18:29:32.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.413+0000 7f01b2b71b80 20 client.25306 trim_cache size 0 max 16384 2024-05-03T18:29:32.415 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-03T18:29:32.415 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-05-03T18:29:32.415 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "79763be4-de61-413f-8729-2cf150579498", 2024-05-03T18:29:32.415 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-05-03T18:29:32.415 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-03T18:29:32.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.413+0000 7f01a8200700 1 -- 172.21.15.33:0/108970077 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f01880b2a70 msgr2=0x7f01880b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:32.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.413+0000 7f01a8200700 1 --2- 172.21.15.33:0/108970077 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f01880b2a70 0x7f01880b4f30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f019801a000 tx=0x7f019801c000 comp rx=0 tx=0).stop 2024-05-03T18:29:32.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.414+0000 7f01a8200700 1 -- 172.21.15.33:0/108970077 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01a0075ff0 msgr2=0x7f01a01a1e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:32.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.414+0000 7f01a8200700 1 --2- 172.21.15.33:0/108970077 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01a0075ff0 0x7f01a01a1e40 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f018c0057c0 tx=0x7f018c0057f0 comp rx=0 tx=0).stop 2024-05-03T18:29:32.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.414+0000 7f01a8200700 1 -- 172.21.15.33:0/108970077 shutdown_connections 2024-05-03T18:29:32.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.414+0000 7f01a8200700 1 --2- 172.21.15.33:0/108970077 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f01880b2a70 0x7f01880b4f30 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:32.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.414+0000 7f01a8200700 1 --2- 172.21.15.33:0/108970077 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01a010fea0 0x7f01a019bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:32.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.414+0000 7f01a8200700 1 --2- 172.21.15.33:0/108970077 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f01a00756d0 0x7f01a01a1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:32.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.414+0000 7f01a8200700 1 --2- 172.21.15.33:0/108970077 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01a0075ff0 0x7f01a01a1e40 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:32.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.414+0000 7f01a8200700 1 -- 172.21.15.33:0/108970077 >> 172.21.15.33:0/108970077 conn(0x7f01a007a600 msgr2=0x7f01a01007d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:29:32.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.414+0000 7f01a8200700 1 -- 172.21.15.33:0/108970077 shutdown_connections 2024-05-03T18:29:32.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:32.414+0000 7f01a8200700 1 -- 172.21.15.33:0/108970077 wait complete. 2024-05-03T18:29:33.183 INFO:teuthology.orchestra.run.smithi033.stderr:2000000 of 3200000 tuples (62%) done (elapsed 70.29 s, remaining 42.17 s) 2024-05-03T18:29:33.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:29:32 smithi033 ceph-mon[28330]: pgmap v73: 129 pgs: 129 active+clean; 489 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 11 MiB/s wr, 12 op/s 2024-05-03T18:29:33.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:29:32 smithi099 ceph-mon[31658]: pgmap v73: 129 pgs: 129 active+clean; 489 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 11 MiB/s wr, 12 op/s 2024-05-03T18:29:33.376 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-03T18:29:33.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:29:32 smithi150 ceph-mon[32492]: pgmap v73: 129 pgs: 129 active+clean; 489 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 11 MiB/s wr, 12 op/s 2024-05-03T18:29:33.927 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:29:34.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:29:33 smithi033 ceph-mon[28330]: scrub queued for path: / 2024-05-03T18:29:34.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:29:33 smithi033 ceph-mon[28330]: scrub summary: idle+waiting paths [/] 2024-05-03T18:29:34.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:29:33 smithi033 ceph-mon[28330]: scrub summary: active paths [/] 2024-05-03T18:29:34.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:29:33 smithi033 ceph-mon[28330]: scrub summary: idle+waiting paths [/] 2024-05-03T18:29:34.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:29:33 smithi099 ceph-mon[31658]: scrub queued for path: / 2024-05-03T18:29:34.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:29:33 smithi099 ceph-mon[31658]: scrub summary: idle+waiting paths [/] 2024-05-03T18:29:34.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:29:33 smithi099 ceph-mon[31658]: scrub summary: active paths [/] 2024-05-03T18:29:34.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:29:33 smithi099 ceph-mon[31658]: scrub summary: idle+waiting paths [/] 2024-05-03T18:29:34.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:29:33 smithi150 ceph-mon[32492]: scrub queued for path: / 2024-05-03T18:29:34.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:29:33 smithi150 ceph-mon[32492]: scrub summary: idle+waiting paths [/] 2024-05-03T18:29:34.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:29:33 smithi150 ceph-mon[32492]: scrub summary: active paths [/] 2024-05-03T18:29:34.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:29:33 smithi150 ceph-mon[32492]: scrub summary: idle+waiting paths [/] 2024-05-03T18:29:35.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:29:34 smithi033 ceph-mon[28330]: pgmap v74: 129 pgs: 129 active+clean; 529 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 17 MiB/s wr, 19 op/s 2024-05-03T18:29:35.284 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:29:34 smithi150 ceph-mon[32492]: pgmap v74: 129 pgs: 129 active+clean; 529 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 17 MiB/s wr, 19 op/s 2024-05-03T18:29:35.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:29:34 smithi099 ceph-mon[31658]: pgmap v74: 129 pgs: 129 active+clean; 529 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 17 MiB/s wr, 19 op/s 2024-05-03T18:29:36.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.438+0000 7ff0ecc00700 1 -- 172.21.15.33:0/1562704907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0e81033d0 msgr2=0x7ff0e8103850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:36.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.438+0000 7ff0ecc00700 1 --2- 172.21.15.33:0/1562704907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0e81033d0 0x7ff0e8103850 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7ff0d0009990 tx=0x7ff0d0009ca0 comp rx=0 tx=0).stop 2024-05-03T18:29:36.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.439+0000 7ff0ecc00700 1 -- 172.21.15.33:0/1562704907 shutdown_connections 2024-05-03T18:29:36.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.439+0000 7ff0ecc00700 1 --2- 172.21.15.33:0/1562704907 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0e8103d90 0x7ff0e81102a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:36.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.439+0000 7ff0ecc00700 1 --2- 172.21.15.33:0/1562704907 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0e8109460 0x7ff0e8109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:36.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.439+0000 7ff0ecc00700 1 --2- 172.21.15.33:0/1562704907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0e81033d0 0x7ff0e8103850 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:36.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.439+0000 7ff0ecc00700 1 -- 172.21.15.33:0/1562704907 >> 172.21.15.33:0/1562704907 conn(0x7ff0e80fef20 msgr2=0x7ff0e8101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:29:36.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.439+0000 7ff0ecc00700 1 -- 172.21.15.33:0/1562704907 shutdown_connections 2024-05-03T18:29:36.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.439+0000 7ff0ecc00700 1 -- 172.21.15.33:0/1562704907 wait complete. 2024-05-03T18:29:36.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.440+0000 7ff0ecc00700 1 Processor -- start 2024-05-03T18:29:36.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.440+0000 7ff0ecc00700 1 -- start start 2024-05-03T18:29:36.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.441+0000 7ff0ecc00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0e81033d0 0x7ff0e81a1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:36.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.441+0000 7ff0ecc00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0e8103d90 0x7ff0e81a1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:36.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.441+0000 7ff0ecc00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0e8109460 0x7ff0e819ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:36.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.441+0000 7ff0ecc00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff0e8113470 con 0x7ff0e8103d90 2024-05-03T18:29:36.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.441+0000 7ff0ecc00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff0e81132f0 con 0x7ff0e81033d0 2024-05-03T18:29:36.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.441+0000 7ff0ecc00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff0e81135f0 con 0x7ff0e8109460 2024-05-03T18:29:36.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.441+0000 7ff0dea00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0e8103d90 0x7ff0e81a1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:36.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.441+0000 7ff0dea00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0e8103d90 0x7ff0e81a1d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45352/0 (socket says 172.21.15.33:45352) 2024-05-03T18:29:36.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.441+0000 7ff0dea00700 1 -- 172.21.15.33:0/3184027772 learned_addr learned my addr 172.21.15.33:0/3184027772 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:29:36.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.441+0000 7ff0e7400700 1 --2- 172.21.15.33:0/3184027772 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0e81033d0 0x7ff0e81a1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:36.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.441+0000 7ff0e7e00700 1 --2- 172.21.15.33:0/3184027772 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0e8109460 0x7ff0e819ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:36.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.442+0000 7ff0dea00700 1 -- 172.21.15.33:0/3184027772 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0e8109460 msgr2=0x7ff0e819ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:36.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.442+0000 7ff0dea00700 1 --2- 172.21.15.33:0/3184027772 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0e8109460 0x7ff0e819ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:36.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.442+0000 7ff0dea00700 1 -- 172.21.15.33:0/3184027772 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0e81033d0 msgr2=0x7ff0e81a1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:36.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.442+0000 7ff0dea00700 1 --2- 172.21.15.33:0/3184027772 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0e81033d0 0x7ff0e81a1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:36.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.442+0000 7ff0dea00700 1 -- 172.21.15.33:0/3184027772 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0e819c170 con 0x7ff0e8103d90 2024-05-03T18:29:36.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.442+0000 7ff0dea00700 1 --2- 172.21.15.33:0/3184027772 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0e8103d90 0x7ff0e81a1d90 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7ff0d0000c00 tx=0x7ff0d00057a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:29:36.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.443+0000 7ff0e4c00700 1 -- 172.21.15.33:0/3184027772 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0d0027070 con 0x7ff0e8103d90 2024-05-03T18:29:36.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.443+0000 7ff0e4c00700 1 -- 172.21.15.33:0/3184027772 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff0d000fb00 con 0x7ff0e8103d90 2024-05-03T18:29:36.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.443+0000 7ff0e4c00700 1 -- 172.21.15.33:0/3184027772 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0d0022b70 con 0x7ff0e8103d90 2024-05-03T18:29:36.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.443+0000 7ff0ecc00700 1 -- 172.21.15.33:0/3184027772 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0e819c400 con 0x7ff0e8103d90 2024-05-03T18:29:36.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.443+0000 7ff0ecc00700 1 -- 172.21.15.33:0/3184027772 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0e81a8720 con 0x7ff0e8103d90 2024-05-03T18:29:36.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.447+0000 7ff0e4c00700 1 -- 172.21.15.33:0/3184027772 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7ff0d0022d10 con 0x7ff0e8103d90 2024-05-03T18:29:36.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.448+0000 7ff0f8b3ab80 1 Processor -- start 2024-05-03T18:29:36.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.448+0000 7ff0f8b3ab80 1 -- start start 2024-05-03T18:29:36.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.448+0000 7ff0f8b3ab80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556fa4583a40 0x556fa4583e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:36.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.448+0000 7ff0f8b3ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556fa4584360 0x556fa457dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:36.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.448+0000 7ff0f8b3ab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556fa457e5d0 0x556fa457ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:36.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.449+0000 7ff0f8b3ab80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556fa44d0a20 con 0x556fa4583a40 2024-05-03T18:29:36.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.449+0000 7ff0f8b3ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x556fa446e760 con 0x556fa4584360 2024-05-03T18:29:36.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.449+0000 7ff0f8b3ab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x556fa44cc9e0 con 0x556fa457e5d0 2024-05-03T18:29:36.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.449+0000 7ff0e7e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556fa4583a40 0x556fa4583e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:36.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.449+0000 7ff0e7e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556fa4583a40 0x556fa4583e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45354/0 (socket says 172.21.15.33:45354) 2024-05-03T18:29:36.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.449+0000 7ff0e7e00700 1 -- 172.21.15.33:0/4250406309 learned_addr learned my addr 172.21.15.33:0/4250406309 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:29:36.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.449+0000 7ff0e4c00700 1 --2- 172.21.15.33:0/3184027772 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7ff0cc0b2a70 0x7ff0cc0b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:36.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.449+0000 7ff0e7400700 1 --2- 172.21.15.33:0/4250406309 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556fa4584360 0x556fa457dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:36.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.449+0000 7ff0dea00700 1 --2- 172.21.15.33:0/4250406309 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556fa457e5d0 0x556fa457ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:36.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.449+0000 7ff0e7e00700 1 --2- 172.21.15.33:0/3184027772 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7ff0cc0b2a70 0x7ff0cc0b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:36.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.450+0000 7ff0e4c00700 1 -- 172.21.15.33:0/3184027772 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7ff0d00e33a0 con 0x7ff0e8103d90 2024-05-03T18:29:36.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.450+0000 7ff0e7e00700 1 -- 172.21.15.33:0/4250406309 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556fa457e5d0 msgr2=0x556fa457ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:29:36.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.450+0000 7ff0e7e00700 1 --2- 172.21.15.33:0/4250406309 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556fa457e5d0 0x556fa457ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:36.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.450+0000 7ff0e7e00700 1 -- 172.21.15.33:0/4250406309 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556fa4584360 msgr2=0x556fa457dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:29:36.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.450+0000 7ff0e7e00700 1 --2- 172.21.15.33:0/4250406309 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556fa4584360 0x556fa457dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:36.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.450+0000 7ff0e7e00700 1 -- 172.21.15.33:0/4250406309 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556fa441d8d0 con 0x556fa4583a40 2024-05-03T18:29:36.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.450+0000 7ff0e7e00700 1 --2- 172.21.15.33:0/4250406309 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556fa4583a40 0x556fa4583e20 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7ff0e0007a20 tx=0x7ff0e0007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3f0a60e088bdf66 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:29:36.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.451+0000 7ff0e7e00700 1 --2- 172.21.15.33:0/3184027772 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7ff0cc0b2a70 0x7ff0cc0b4f30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff0e001a000 tx=0x7ff0e001c000 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:29:36.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.451+0000 7ff0dcc00700 1 -- 172.21.15.33:0/4250406309 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e001d040 con 0x556fa4583a40 2024-05-03T18:29:36.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.451+0000 7ff0dcc00700 1 -- 172.21.15.33:0/4250406309 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff0e0016e90 con 0x556fa4583a40 2024-05-03T18:29:36.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.451+0000 7ff0dcc00700 1 -- 172.21.15.33:0/4250406309 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e0012040 con 0x556fa4583a40 2024-05-03T18:29:36.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.452+0000 7ff0f8b3ab80 1 -- 172.21.15.33:0/4250406309 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556fa4583a40 msgr2=0x556fa4583e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:36.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.452+0000 7ff0f8b3ab80 1 --2- 172.21.15.33:0/4250406309 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556fa4583a40 0x556fa4583e20 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7ff0e0007a20 tx=0x7ff0e0007d30 comp rx=0 tx=0).stop 2024-05-03T18:29:36.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.452+0000 7ff0f8b3ab80 1 -- 172.21.15.33:0/4250406309 shutdown_connections 2024-05-03T18:29:36.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.452+0000 7ff0f8b3ab80 1 --2- 172.21.15.33:0/4250406309 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556fa457e5d0 0x556fa457ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:36.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.452+0000 7ff0f8b3ab80 1 --2- 172.21.15.33:0/4250406309 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556fa4583a40 0x556fa4583e20 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:36.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.452+0000 7ff0f8b3ab80 1 --2- 172.21.15.33:0/4250406309 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556fa4584360 0x556fa457dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:36.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.452+0000 7ff0f8b3ab80 1 -- 172.21.15.33:0/4250406309 >> 172.21.15.33:0/4250406309 conn(0x556fa44acb60 msgr2=0x556fa45789d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:29:36.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.452+0000 7ff0f8b3ab80 1 -- 172.21.15.33:0/4250406309 shutdown_connections 2024-05-03T18:29:36.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.452+0000 7ff0f8b3ab80 1 -- 172.21.15.33:0/4250406309 wait complete. 2024-05-03T18:29:36.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.453+0000 7ff0f8b3ab80 1 -- start start 2024-05-03T18:29:36.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.453+0000 7ff0f8b3ab80 1 Processor -- start 2024-05-03T18:29:36.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.454+0000 7ff0f8b3ab80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556fa457e5d0 0x556fa458a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:36.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.454+0000 7ff0f8b3ab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556fa4583a40 0x556fa4587510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:36.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.454+0000 7ff0f8b3ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556fa4584360 0x556fa4587a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:36.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.454+0000 7ff0f8b3ab80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556fa458ad70 con 0x556fa457e5d0 2024-05-03T18:29:36.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.454+0000 7ff0f8b3ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x556fa4436b20 con 0x556fa4584360 2024-05-03T18:29:36.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.454+0000 7ff0f8b3ab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x556fa458aef0 con 0x556fa4583a40 2024-05-03T18:29:36.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.454+0000 7ff0f8b3ab80 10 client.0 _ll_register_callbacks cb 0x556fa4477c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:29:36.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.454+0000 7ff0e7400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556fa457e5d0 0x556fa458a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:36.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.454+0000 7ff0e7400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556fa457e5d0 0x556fa458a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45362/0 (socket says 172.21.15.33:45362) 2024-05-03T18:29:36.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.454+0000 7ff0e7400700 1 -- 172.21.15.33:0/510386268 learned_addr learned my addr 172.21.15.33:0/510386268 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:29:36.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.454+0000 7ff0e7400700 1 -- 172.21.15.33:0/510386268 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556fa4583a40 msgr2=0x556fa4587510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:29:36.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.454+0000 7ff0e7400700 1 --2- 172.21.15.33:0/510386268 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556fa4583a40 0x556fa4587510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:36.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.455+0000 7ff0e7400700 1 -- 172.21.15.33:0/510386268 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556fa4584360 msgr2=0x556fa4587a50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-03T18:29:36.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.455+0000 7ff0e7400700 1 --2- 172.21.15.33:0/510386268 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556fa4584360 0x556fa4587a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:36.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.455+0000 7ff0e7400700 1 -- 172.21.15.33:0/510386268 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556fa4496420 con 0x556fa457e5d0 2024-05-03T18:29:36.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.455+0000 7ff0e7400700 1 --2- 172.21.15.33:0/510386268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556fa457e5d0 0x556fa458a170 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7ff0d800bfd0 tx=0x7ff0d80117d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eebaa83c86199ff server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:29:36.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.455+0000 7ff0d5600700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:29:36.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.455+0000 7ff0d5600700 1 -- 172.21.15.33:0/510386268 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0d8004030 con 0x556fa457e5d0 2024-05-03T18:29:36.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.455+0000 7ff0d5600700 1 -- 172.21.15.33:0/510386268 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7ff0d80041d0 con 0x556fa457e5d0 2024-05-03T18:29:36.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.455+0000 7ff0f8b3ab80 1 -- 172.21.15.33:0/510386268 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0e0016e90 con 0x556fa457e5d0 2024-05-03T18:29:36.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.456+0000 7ff0d5600700 1 -- 172.21.15.33:0/510386268 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7ff0d80043d0 con 0x556fa457e5d0 2024-05-03T18:29:36.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.456+0000 7ff0f8b3ab80 10 client.15636 fetch_fsmap learned FSMap version 21 2024-05-03T18:29:36.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.456+0000 7ff0f8b3ab80 1 -- 172.21.15.33:0/510386268 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x556fa4595750 con 0x556fa457e5d0 2024-05-03T18:29:36.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.456+0000 7ff0d5600700 1 -- 172.21.15.33:0/510386268 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff0e0016e90 con 0x556fa457e5d0 2024-05-03T18:29:36.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.456+0000 7ff0d5600700 1 -- 172.21.15.33:0/510386268 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7ff0d8003840 con 0x556fa457e5d0 2024-05-03T18:29:36.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.456+0000 7ff0f8b3ab80 10 client.15636 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:29:36.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.456+0000 7ff0f8b3ab80 10 client.15636 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:29:36.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.456+0000 7ff0f8b3ab80 20 client.15636 populate_metadata read hostname 'smithi033' 2024-05-03T18:29:36.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.456+0000 7ff0f8b3ab80 1 --2- 172.21.15.33:0/510386268 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x556fa4597d90 0x556fa459a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:36.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.456+0000 7ff0f8b3ab80 4 client.15636 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:29:36.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.456+0000 7ff0f8b3ab80 1 -- 172.21.15.33:0/510386268 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556fa454b1d0 con 0x556fa4597d90 2024-05-03T18:29:36.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.457+0000 7ff0dea00700 1 --2- 172.21.15.33:0/510386268 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x556fa4597d90 0x556fa459a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:36.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.458+0000 7ff0dea00700 1 --2- 172.21.15.33:0/510386268 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x556fa4597d90 0x556fa459a210 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c15a07baf60364e server_cookie=fdd03c2e6d01076d in_seq=0 out_seq=0 2024-05-03T18:29:36.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.458+0000 7ff0d5600700 10 client.15636 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:29:36.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.460+0000 7ff0d5600700 1 -- 172.21.15.33:0/510386268 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556fa454b1d0 con 0x556fa4597d90 2024-05-03T18:29:36.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.460+0000 7ff0d5600700 10 client.15636 handle_command_reply: tid=0 2024-05-03T18:29:36.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.460+0000 7ff0f8b3ab80 1 client.15636 shutdown 2024-05-03T18:29:36.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.461+0000 7ff0f8b3ab80 1 -- 172.21.15.33:0/510386268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556fa457e5d0 msgr2=0x556fa458a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:36.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.461+0000 7ff0f8b3ab80 1 --2- 172.21.15.33:0/510386268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556fa457e5d0 0x556fa458a170 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7ff0d800bfd0 tx=0x7ff0d80117d0 comp rx=0 tx=0).stop 2024-05-03T18:29:36.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.461+0000 7ff0f8b3ab80 1 -- 172.21.15.33:0/510386268 shutdown_connections 2024-05-03T18:29:36.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.461+0000 7ff0f8b3ab80 1 --2- 172.21.15.33:0/510386268 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x556fa4597d90 0x556fa459a210 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:36.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.461+0000 7ff0f8b3ab80 1 --2- 172.21.15.33:0/510386268 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556fa4584360 0x556fa4587a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:36.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.461+0000 7ff0f8b3ab80 1 --2- 172.21.15.33:0/510386268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556fa457e5d0 0x556fa458a170 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:36.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.461+0000 7ff0f8b3ab80 1 --2- 172.21.15.33:0/510386268 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556fa4583a40 0x556fa4587510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:36.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.461+0000 7ff0d5600700 0 client.15636 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:29:36.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.461+0000 7ff0f8b3ab80 1 -- 172.21.15.33:0/510386268 >> 172.21.15.33:0/510386268 conn(0x556fa44acb60 msgr2=0x556fa4578080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:29:36.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.461+0000 7ff0f8b3ab80 1 -- 172.21.15.33:0/510386268 shutdown_connections 2024-05-03T18:29:36.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.461+0000 7ff0f8b3ab80 1 -- 172.21.15.33:0/510386268 wait complete. 2024-05-03T18:29:36.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.461+0000 7ff0f8b3ab80 20 client.15636 trim_cache size 0 max 16384 2024-05-03T18:29:36.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.487+0000 7ff0f8b3ab80 1 Processor -- start 2024-05-03T18:29:36.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.487+0000 7ff0f8b3ab80 1 -- start start 2024-05-03T18:29:36.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.488+0000 7ff0f8b3ab80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556fa457e5d0 0x556fa457dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:36.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.488+0000 7ff0f8b3ab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556fa4583a40 0x556fa4581bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:36.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.488+0000 7ff0f8b3ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556fa4584360 0x556fa4593e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:36.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.488+0000 7ff0f8b3ab80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff0e81af2e0 con 0x556fa457e5d0 2024-05-03T18:29:36.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.488+0000 7ff0f8b3ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff0e81b1b60 con 0x556fa4584360 2024-05-03T18:29:36.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.488+0000 7ff0f8b3ab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff0e81af950 con 0x556fa4583a40 2024-05-03T18:29:36.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.488+0000 7ff0e7400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556fa457e5d0 0x556fa457dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:36.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.488+0000 7ff0e7400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556fa457e5d0 0x556fa457dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45370/0 (socket says 172.21.15.33:45370) 2024-05-03T18:29:36.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.488+0000 7ff0e7400700 1 -- 172.21.15.33:0/3646500177 learned_addr learned my addr 172.21.15.33:0/3646500177 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:29:36.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.489+0000 7ff0dea00700 1 --2- 172.21.15.33:0/3646500177 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556fa4583a40 0x556fa4581bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:36.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.489+0000 7ff0e7e00700 1 --2- 172.21.15.33:0/3646500177 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556fa4584360 0x556fa4593e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:36.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.489+0000 7ff0e7400700 1 -- 172.21.15.33:0/3646500177 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556fa4583a40 msgr2=0x556fa4581bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:29:36.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.489+0000 7ff0e7400700 1 --2- 172.21.15.33:0/3646500177 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556fa4583a40 0x556fa4581bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:36.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.489+0000 7ff0e7400700 1 -- 172.21.15.33:0/3646500177 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556fa4584360 msgr2=0x556fa4593e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:29:36.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.489+0000 7ff0e7400700 1 --2- 172.21.15.33:0/3646500177 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556fa4584360 0x556fa4593e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:36.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.489+0000 7ff0e7400700 1 -- 172.21.15.33:0/3646500177 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556fa4584070 con 0x556fa457e5d0 2024-05-03T18:29:36.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.490+0000 7ff0e7400700 1 --2- 172.21.15.33:0/3646500177 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556fa457e5d0 0x556fa457dfb0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7ff0d800bfa0 tx=0x7ff0d800b500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=303765cb491b4415 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:29:36.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.490+0000 7ff0d7e00700 1 -- 172.21.15.33:0/3646500177 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0d8003f20 con 0x556fa457e5d0 2024-05-03T18:29:36.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.490+0000 7ff0d7e00700 1 -- 172.21.15.33:0/3646500177 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff0d80040c0 con 0x556fa457e5d0 2024-05-03T18:29:36.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.490+0000 7ff0d7e00700 1 -- 172.21.15.33:0/3646500177 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0d8010700 con 0x556fa457e5d0 2024-05-03T18:29:36.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.491+0000 7ff0f8b3ab80 1 -- 172.21.15.33:0/3646500177 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556fa457e5d0 msgr2=0x556fa457dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:36.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.491+0000 7ff0f8b3ab80 1 --2- 172.21.15.33:0/3646500177 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556fa457e5d0 0x556fa457dfb0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7ff0d800bfa0 tx=0x7ff0d800b500 comp rx=0 tx=0).stop 2024-05-03T18:29:36.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.491+0000 7ff0f8b3ab80 1 -- 172.21.15.33:0/3646500177 shutdown_connections 2024-05-03T18:29:36.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.491+0000 7ff0f8b3ab80 1 --2- 172.21.15.33:0/3646500177 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556fa4584360 0x556fa4593e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:36.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.491+0000 7ff0f8b3ab80 1 --2- 172.21.15.33:0/3646500177 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556fa457e5d0 0x556fa457dfb0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:36.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.491+0000 7ff0f8b3ab80 1 --2- 172.21.15.33:0/3646500177 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556fa4583a40 0x556fa4581bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:36.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.491+0000 7ff0f8b3ab80 1 -- 172.21.15.33:0/3646500177 >> 172.21.15.33:0/3646500177 conn(0x556fa44acb60 msgr2=0x556fa45814e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:29:36.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.491+0000 7ff0f8b3ab80 1 -- 172.21.15.33:0/3646500177 shutdown_connections 2024-05-03T18:29:36.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.491+0000 7ff0f8b3ab80 1 -- 172.21.15.33:0/3646500177 wait complete. 2024-05-03T18:29:36.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.492+0000 7ff0f8b3ab80 1 -- start start 2024-05-03T18:29:36.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.492+0000 7ff0f8b3ab80 1 Processor -- start 2024-05-03T18:29:36.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.492+0000 7ff0f8b3ab80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556fa457e5d0 0x556fa4590830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:36.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.493+0000 7ff0f8b3ab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556fa4583a40 0x556fa459f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:36.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.493+0000 7ff0f8b3ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556fa4584360 0x556fa459f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:36.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.493+0000 7ff0f8b3ab80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff0e81b0020 con 0x556fa457e5d0 2024-05-03T18:29:36.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.493+0000 7ff0f8b3ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff0e81b1490 con 0x556fa4584360 2024-05-03T18:29:36.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.493+0000 7ff0f8b3ab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff0e81b06f0 con 0x556fa4583a40 2024-05-03T18:29:36.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.493+0000 7ff0f8b3ab80 10 client.0 _ll_register_callbacks cb 0x556fa4477c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:29:36.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.493+0000 7ff0e7400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556fa457e5d0 0x556fa4590830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:36.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.493+0000 7ff0e7400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556fa457e5d0 0x556fa4590830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45382/0 (socket says 172.21.15.33:45382) 2024-05-03T18:29:36.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.493+0000 7ff0e7400700 1 -- 172.21.15.33:0/3993277469 learned_addr learned my addr 172.21.15.33:0/3993277469 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:29:36.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.493+0000 7ff0e7e00700 1 --2- 172.21.15.33:0/3993277469 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556fa4584360 0x556fa459f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:36.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.493+0000 7ff0dea00700 1 --2- 172.21.15.33:0/3993277469 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556fa4583a40 0x556fa459f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:36.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.494+0000 7ff0e7400700 1 -- 172.21.15.33:0/3993277469 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556fa4583a40 msgr2=0x556fa459f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:29:36.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.494+0000 7ff0e7400700 1 --2- 172.21.15.33:0/3993277469 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556fa4583a40 0x556fa459f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:36.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.494+0000 7ff0e7400700 1 -- 172.21.15.33:0/3993277469 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556fa4584360 msgr2=0x556fa459f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:29:36.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.494+0000 7ff0e7400700 1 --2- 172.21.15.33:0/3993277469 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556fa4584360 0x556fa459f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:36.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.494+0000 7ff0e7400700 1 -- 172.21.15.33:0/3993277469 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556fa4601690 con 0x556fa457e5d0 2024-05-03T18:29:36.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.494+0000 7ff0e7e00700 1 --2- 172.21.15.33:0/3993277469 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556fa4584360 0x556fa459f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:29:36.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.494+0000 7ff0dea00700 1 --2- 172.21.15.33:0/3993277469 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556fa4583a40 0x556fa459f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:29:36.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.495+0000 7ff0e7400700 1 --2- 172.21.15.33:0/3993277469 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556fa457e5d0 0x556fa4590830 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7ff0d8011810 tx=0x7ff0d8004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aee8cc65affa8c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:29:36.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.495+0000 7ff0d5600700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:29:36.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.495+0000 7ff0d5600700 1 -- 172.21.15.33:0/3993277469 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0d8003950 con 0x556fa457e5d0 2024-05-03T18:29:36.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.495+0000 7ff0d5600700 1 -- 172.21.15.33:0/3993277469 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7ff0d8017040 con 0x556fa457e5d0 2024-05-03T18:29:36.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.495+0000 7ff0f8b3ab80 1 -- 172.21.15.33:0/3993277469 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0d80040c0 con 0x556fa457e5d0 2024-05-03T18:29:36.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.496+0000 7ff0d5600700 1 -- 172.21.15.33:0/3993277469 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7ff0d8010700 con 0x556fa457e5d0 2024-05-03T18:29:36.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.496+0000 7ff0f8b3ab80 10 client.15642 fetch_fsmap learned FSMap version 21 2024-05-03T18:29:36.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.496+0000 7ff0f8b3ab80 1 -- 172.21.15.33:0/3993277469 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x556fa4602860 con 0x556fa457e5d0 2024-05-03T18:29:36.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.497+0000 7ff0d5600700 1 -- 172.21.15.33:0/3993277469 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff0d80040c0 con 0x556fa457e5d0 2024-05-03T18:29:36.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.497+0000 7ff0d5600700 1 -- 172.21.15.33:0/3993277469 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7ff0d8044080 con 0x556fa457e5d0 2024-05-03T18:29:36.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.497+0000 7ff0f8b3ab80 10 client.15642 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:29:36.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.497+0000 7ff0f8b3ab80 10 client.15642 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:29:36.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.497+0000 7ff0f8b3ab80 20 client.15642 populate_metadata read hostname 'smithi033' 2024-05-03T18:29:36.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.497+0000 7ff0f8b3ab80 1 --2- 172.21.15.33:0/3993277469 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x556fa4602ca0 0x556fa46050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:36.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.497+0000 7ff0f8b3ab80 4 client.15642 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-03T18:29:36.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.497+0000 7ff0f8b3ab80 1 -- 172.21.15.33:0/3993277469 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556fa44de400 con 0x556fa4602ca0 2024-05-03T18:29:36.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.498+0000 7ff0dea00700 1 --2- 172.21.15.33:0/3993277469 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x556fa4602ca0 0x556fa46050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:36.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.499+0000 7ff0dea00700 1 --2- 172.21.15.33:0/3993277469 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x556fa4602ca0 0x556fa46050c0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e613f53eb85a147 server_cookie=56ea5bae0ca9ea0c in_seq=0 out_seq=0 2024-05-03T18:29:36.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.499+0000 7ff0d5600700 10 client.15642 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:29:36.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.499+0000 7ff0d5600700 1 -- 172.21.15.33:0/3993277469 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x556fa44de400 con 0x556fa4602ca0 2024-05-03T18:29:36.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.499+0000 7ff0d5600700 10 client.15642 handle_command_reply: tid=0 2024-05-03T18:29:36.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.499+0000 7ff0f8b3ab80 1 client.15642 shutdown 2024-05-03T18:29:36.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.499+0000 7ff0f8b3ab80 1 -- 172.21.15.33:0/3993277469 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556fa457e5d0 msgr2=0x556fa4590830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:36.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.499+0000 7ff0f8b3ab80 1 --2- 172.21.15.33:0/3993277469 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556fa457e5d0 0x556fa4590830 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7ff0d8011810 tx=0x7ff0d8004360 comp rx=0 tx=0).stop 2024-05-03T18:29:36.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.500+0000 7ff0f8b3ab80 1 -- 172.21.15.33:0/3993277469 shutdown_connections 2024-05-03T18:29:36.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.500+0000 7ff0f8b3ab80 1 --2- 172.21.15.33:0/3993277469 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x556fa4602ca0 0x556fa46050c0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:36.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.500+0000 7ff0f8b3ab80 1 --2- 172.21.15.33:0/3993277469 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556fa4584360 0x556fa459f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:36.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.500+0000 7ff0f8b3ab80 1 --2- 172.21.15.33:0/3993277469 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556fa457e5d0 0x556fa4590830 secure :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7ff0d8011810 tx=0x7ff0d8004360 comp rx=0 tx=0).stop 2024-05-03T18:29:36.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.500+0000 7ff0f8b3ab80 1 --2- 172.21.15.33:0/3993277469 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556fa4583a40 0x556fa459f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:36.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.500+0000 7ff0f8b3ab80 1 -- 172.21.15.33:0/3993277469 >> 172.21.15.33:0/3993277469 conn(0x556fa44acb60 msgr2=0x556fa4578080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:29:36.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.500+0000 7ff0d5600700 0 client.15642 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:29:36.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.500+0000 7ff0f8b3ab80 1 -- 172.21.15.33:0/3993277469 shutdown_connections 2024-05-03T18:29:36.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.500+0000 7ff0f8b3ab80 1 -- 172.21.15.33:0/3993277469 wait complete. 2024-05-03T18:29:36.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.500+0000 7ff0f8b3ab80 20 client.15642 trim_cache size 0 max 16384 2024-05-03T18:29:36.502 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-03T18:29:36.502 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-05-03T18:29:36.502 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-05-03T18:29:36.502 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-03T18:29:36.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.501+0000 7ff0ecc00700 1 -- 172.21.15.33:0/3184027772 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7ff0cc0b2a70 msgr2=0x7ff0cc0b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:36.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.501+0000 7ff0ecc00700 1 --2- 172.21.15.33:0/3184027772 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7ff0cc0b2a70 0x7ff0cc0b4f30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff0e001a000 tx=0x7ff0e001c000 comp rx=0 tx=0).stop 2024-05-03T18:29:36.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.501+0000 7ff0ecc00700 1 -- 172.21.15.33:0/3184027772 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0e8103d90 msgr2=0x7ff0e81a1d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:36.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.501+0000 7ff0ecc00700 1 --2- 172.21.15.33:0/3184027772 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0e8103d90 0x7ff0e81a1d90 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7ff0d0000c00 tx=0x7ff0d00057a0 comp rx=0 tx=0).stop 2024-05-03T18:29:36.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.501+0000 7ff0ecc00700 1 -- 172.21.15.33:0/3184027772 shutdown_connections 2024-05-03T18:29:36.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.501+0000 7ff0ecc00700 1 --2- 172.21.15.33:0/3184027772 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0e8109460 0x7ff0e819ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:36.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.501+0000 7ff0ecc00700 1 --2- 172.21.15.33:0/3184027772 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7ff0cc0b2a70 0x7ff0cc0b4f30 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:36.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.501+0000 7ff0ecc00700 1 --2- 172.21.15.33:0/3184027772 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0e81033d0 0x7ff0e81a1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:36.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.501+0000 7ff0ecc00700 1 --2- 172.21.15.33:0/3184027772 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0e8103d90 0x7ff0e81a1d90 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:36.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.501+0000 7ff0ecc00700 1 -- 172.21.15.33:0/3184027772 >> 172.21.15.33:0/3184027772 conn(0x7ff0e80fef20 msgr2=0x7ff0e810e660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:29:36.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.502+0000 7ff0ecc00700 1 -- 172.21.15.33:0/3184027772 shutdown_connections 2024-05-03T18:29:36.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:36.502+0000 7ff0ecc00700 1 -- 172.21.15.33:0/3184027772 wait complete. 2024-05-03T18:29:36.530 INFO:teuthology.orchestra.run.smithi033.stderr:2100000 of 3200000 tuples (65%) done (elapsed 73.64 s, remaining 38.57 s) 2024-05-03T18:29:37.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:29:36 smithi033 ceph-mon[28330]: pgmap v75: 129 pgs: 129 active+clean; 529 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 12 MiB/s wr, 12 op/s 2024-05-03T18:29:37.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:29:36 smithi033 ceph-mon[28330]: scrub summary: idle 2024-05-03T18:29:37.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:29:36 smithi099 ceph-mon[31658]: pgmap v75: 129 pgs: 129 active+clean; 529 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 12 MiB/s wr, 12 op/s 2024-05-03T18:29:37.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:29:36 smithi099 ceph-mon[31658]: scrub summary: idle 2024-05-03T18:29:37.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:29:36 smithi150 ceph-mon[32492]: pgmap v75: 129 pgs: 129 active+clean; 529 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 12 MiB/s wr, 12 op/s 2024-05-03T18:29:37.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:29:36 smithi150 ceph-mon[32492]: scrub summary: idle 2024-05-03T18:29:37.539 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-03T18:29:37.539 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T18:29:38.051 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:29:39.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:29:38 smithi033 ceph-mon[28330]: pgmap v76: 129 pgs: 129 active+clean; 561 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 18 MiB/s wr, 18 op/s 2024-05-03T18:29:39.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:29:38 smithi099 ceph-mon[31658]: pgmap v76: 129 pgs: 129 active+clean; 561 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 18 MiB/s wr, 18 op/s 2024-05-03T18:29:39.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:29:38 smithi150 ceph-mon[32492]: pgmap v76: 129 pgs: 129 active+clean; 561 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 18 MiB/s wr, 18 op/s 2024-05-03T18:29:40.933 INFO:teuthology.orchestra.run.smithi033.stderr:2200000 of 3200000 tuples (68%) done (elapsed 78.04 s, remaining 35.47 s) 2024-05-03T18:29:41.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:29:40 smithi033 ceph-mon[28330]: pgmap v77: 129 pgs: 129 active+clean; 561 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 12 MiB/s wr, 11 op/s 2024-05-03T18:29:41.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:29:40 smithi099 ceph-mon[31658]: pgmap v77: 129 pgs: 129 active+clean; 561 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 12 MiB/s wr, 11 op/s 2024-05-03T18:29:41.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:29:40 smithi150 ceph-mon[32492]: pgmap v77: 129 pgs: 129 active+clean; 561 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 12 MiB/s wr, 11 op/s 2024-05-03T18:29:41.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:41.454+0000 7ffb10200700 1 -- 172.21.15.33:0/3931290636 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb08109510 msgr2=0x7ffb081098f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:41.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:41.454+0000 7ffb10200700 1 --2- 172.21.15.33:0/3931290636 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb08109510 0x7ffb081098f0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7ffafc00b0a0 tx=0x7ffafc00b3b0 comp rx=0 tx=0).stop 2024-05-03T18:29:41.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:41.454+0000 7ffb10200700 1 -- 172.21.15.33:0/3931290636 shutdown_connections 2024-05-03T18:29:41.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:41.454+0000 7ffb10200700 1 --2- 172.21.15.33:0/3931290636 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb0806bb40 0x7ffb08110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:41.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:41.454+0000 7ffb10200700 1 --2- 172.21.15.33:0/3931290636 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb08109510 0x7ffb081098f0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:41.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:41.454+0000 7ffb10200700 1 --2- 172.21.15.33:0/3931290636 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffb0806b180 0x7ffb0806b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:41.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:41.454+0000 7ffb10200700 1 -- 172.21.15.33:0/3931290636 >> 172.21.15.33:0/3931290636 conn(0x7ffb08079350 msgr2=0x7ffb08079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:29:41.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:41.454+0000 7ffb10200700 1 -- 172.21.15.33:0/3931290636 shutdown_connections 2024-05-03T18:29:41.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:41.454+0000 7ffb10200700 1 -- 172.21.15.33:0/3931290636 wait complete. 2024-05-03T18:29:41.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:41.455+0000 7ffb10200700 1 Processor -- start 2024-05-03T18:29:41.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:41.455+0000 7ffb10200700 1 -- start start 2024-05-03T18:29:41.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:41.456+0000 7ffb10200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffb0806b180 0x7ffb08102470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:41.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:41.456+0000 7ffb10200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb0806bb40 0x7ffb081029b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:41.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:41.456+0000 7ffb10200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb08109510 0x7ffb08102ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:41.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:41.456+0000 7ffb10200700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ffb08113540 con 0x7ffb0806bb40 2024-05-03T18:29:41.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:41.456+0000 7ffb10200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ffb081133c0 con 0x7ffb0806b180 2024-05-03T18:29:41.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:41.456+0000 7ffb10200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ffb081136c0 con 0x7ffb08109510 2024-05-03T18:29:41.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:41.456+0000 7ffb0ce00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb0806bb40 0x7ffb081029b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:41.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:41.456+0000 7ffb0ce00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb0806bb40 0x7ffb081029b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45394/0 (socket says 172.21.15.33:45394) 2024-05-03T18:29:41.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:41.456+0000 7ffb0ce00700 1 -- 172.21.15.33:0/3414140856 learned_addr learned my addr 172.21.15.33:0/3414140856 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:29:41.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:41.456+0000 7ffb0d800700 1 --2- 172.21.15.33:0/3414140856 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffb0806b180 0x7ffb08102470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:41.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:41.457+0000 7ffb0e200700 1 --2- 172.21.15.33:0/3414140856 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb08109510 0x7ffb08102ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:41.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:41.457+0000 7ffb0ce00700 1 -- 172.21.15.33:0/3414140856 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb08109510 msgr2=0x7ffb08102ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:41.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:41.457+0000 7ffb0ce00700 1 --2- 172.21.15.33:0/3414140856 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb08109510 0x7ffb08102ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:41.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:41.457+0000 7ffb0ce00700 1 -- 172.21.15.33:0/3414140856 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffb0806b180 msgr2=0x7ffb08102470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:41.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:41.457+0000 7ffb0ce00700 1 --2- 172.21.15.33:0/3414140856 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffb0806b180 0x7ffb08102470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:41.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:41.457+0000 7ffb0ce00700 1 -- 172.21.15.33:0/3414140856 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb081a8020 con 0x7ffb0806bb40 2024-05-03T18:29:41.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:41.457+0000 7ffb0ce00700 1 --2- 172.21.15.33:0/3414140856 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb0806bb40 0x7ffb081029b0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7ffaf400e9a0 tx=0x7ffaf400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:29:41.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:41.458+0000 7ffb06000700 1 -- 172.21.15.33:0/3414140856 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaf400cbd0 con 0x7ffb0806bb40 2024-05-03T18:29:41.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:41.458+0000 7ffb06000700 1 -- 172.21.15.33:0/3414140856 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ffaf400cd70 con 0x7ffb0806bb40 2024-05-03T18:29:41.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:41.458+0000 7ffb06000700 1 -- 172.21.15.33:0/3414140856 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaf4018680 con 0x7ffb0806bb40 2024-05-03T18:29:41.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:41.458+0000 7ffb10200700 1 -- 172.21.15.33:0/3414140856 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb081a82b0 con 0x7ffb0806bb40 2024-05-03T18:29:41.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:41.458+0000 7ffb10200700 1 -- 172.21.15.33:0/3414140856 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb081a87a0 con 0x7ffb0806bb40 2024-05-03T18:29:41.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:41.460+0000 7ffb10200700 1 -- 172.21.15.33:0/3414140856 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb0810b9f0 con 0x7ffb0806bb40 2024-05-03T18:29:41.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:41.462+0000 7ffb06000700 1 -- 172.21.15.33:0/3414140856 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7ffaf4010430 con 0x7ffb0806bb40 2024-05-03T18:29:41.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:41.463+0000 7ffb06000700 1 --2- 172.21.15.33:0/3414140856 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7ffaf00b29f0 0x7ffaf00b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:41.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:41.464+0000 7ffb06000700 1 -- 172.21.15.33:0/3414140856 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7ffaf40e2c80 con 0x7ffb0806bb40 2024-05-03T18:29:41.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:41.464+0000 7ffb0d800700 1 --2- 172.21.15.33:0/3414140856 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7ffaf00b29f0 0x7ffaf00b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:41.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:41.465+0000 7ffb0d800700 1 --2- 172.21.15.33:0/3414140856 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7ffaf00b29f0 0x7ffaf00b4eb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ffafc009fd0 tx=0x7ffafc006210 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:29:41.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:41.469+0000 7ffb06000700 1 -- 172.21.15.33:0/3414140856 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7ffaf40aa270 con 0x7ffb0806bb40 2024-05-03T18:29:41.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:41.813+0000 7ffb10200700 1 -- 172.21.15.33:0/3414140856 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffb0810b3d0 con 0x7ffb0806bb40 2024-05-03T18:29:41.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:41.815+0000 7ffb06000700 1 -- 172.21.15.33:0/3414140856 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11801 (secure 0 0 0) 0x7ffaf40cae20 con 0x7ffb0806bb40 2024-05-03T18:29:41.816 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:29:41.817 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1327393503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1327393503},{"type":"v1","addr":"172.21.15.33:6835","nonce":1327393503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/2461472948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":2461472948},{"type":"v1","addr":"172.21.15.33:6837","nonce":2461472948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14559,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/442269281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":442269281},{"type":"v1","addr":"172.21.15.33:6839","nonce":442269281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14571,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/645571756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":645571756},{"type":"v1","addr":"172.21.15.33:6841","nonce":645571756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/203598909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":203598909},{"type":"v1","addr":"172.21.15.150:6833","nonce":203598909}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/4029842493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":4029842493},{"type":"v1","addr":"172.21.15.150:6835","nonce":4029842493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3463664423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3463664423},{"type":"v1","addr":"172.21.15.150:6837","nonce":3463664423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24481,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6839/4051648018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":4051648018},{"type":"v1","addr":"172.21.15.150:6839","nonce":4051648018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34418,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1865105479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1865105479},{"type":"v1","addr":"172.21.15.99:6833","nonce":1865105479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34445,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/1206393914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1206393914},{"type":"v1","addr":"172.21.15.99:6835","nonce":1206393914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-03T18:22:28.837091+0000","modified":"2024-05-03T18:23:49.758398+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34508},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34472":{"gid":34472,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6837/3360970342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3360970342},{"type":"v1","addr":"172.21.15.99:6837","nonce":3360970342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34508":{"gid":34508,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/919907308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":919907308},{"type":"v1","addr":"172.21.15.99:6839","nonce":919907308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34508,"qdb_cluster":[34508]},"id":1}]} 2024-05-03T18:29:41.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:41.818+0000 7ffb10200700 1 -- 172.21.15.33:0/3414140856 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7ffaf00b29f0 msgr2=0x7ffaf00b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:41.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:41.818+0000 7ffb10200700 1 --2- 172.21.15.33:0/3414140856 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7ffaf00b29f0 0x7ffaf00b4eb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ffafc009fd0 tx=0x7ffafc006210 comp rx=0 tx=0).stop 2024-05-03T18:29:41.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:41.818+0000 7ffb10200700 1 -- 172.21.15.33:0/3414140856 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb0806bb40 msgr2=0x7ffb081029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:41.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:41.818+0000 7ffb10200700 1 --2- 172.21.15.33:0/3414140856 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb0806bb40 0x7ffb081029b0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7ffaf400e9a0 tx=0x7ffaf400ecb0 comp rx=0 tx=0).stop 2024-05-03T18:29:41.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:41.818+0000 7ffb10200700 1 -- 172.21.15.33:0/3414140856 shutdown_connections 2024-05-03T18:29:41.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:41.818+0000 7ffb10200700 1 --2- 172.21.15.33:0/3414140856 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb08109510 0x7ffb08102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:41.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:41.818+0000 7ffb10200700 1 --2- 172.21.15.33:0/3414140856 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7ffaf00b29f0 0x7ffaf00b4eb0 secure :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ffafc009fd0 tx=0x7ffafc006210 comp rx=0 tx=0).stop 2024-05-03T18:29:41.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:41.818+0000 7ffb10200700 1 --2- 172.21.15.33:0/3414140856 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffb0806b180 0x7ffb08102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:41.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:41.818+0000 7ffb10200700 1 --2- 172.21.15.33:0/3414140856 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb0806bb40 0x7ffb081029b0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:41.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:41.818+0000 7ffb10200700 1 -- 172.21.15.33:0/3414140856 >> 172.21.15.33:0/3414140856 conn(0x7ffb08079350 msgr2=0x7ffb08101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:29:41.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:41.819+0000 7ffb10200700 1 -- 172.21.15.33:0/3414140856 shutdown_connections 2024-05-03T18:29:41.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:41.819+0000 7ffb10200700 1 -- 172.21.15.33:0/3414140856 wait complete. 2024-05-03T18:29:41.822 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 21 2024-05-03T18:29:42.198 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:29:41 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/3414140856' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:29:42.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:29:41 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/3414140856' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:29:42.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:29:41 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/3414140856' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:29:42.751 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-03T18:29:43.201 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:29:42 smithi099 ceph-mon[31658]: pgmap v78: 129 pgs: 129 active+clean; 561 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 12 MiB/s wr, 11 op/s 2024-05-03T18:29:43.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:29:42 smithi033 ceph-mon[28330]: pgmap v78: 129 pgs: 129 active+clean; 561 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 12 MiB/s wr, 11 op/s 2024-05-03T18:29:43.265 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:29:43.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:29:42 smithi150 ceph-mon[32492]: pgmap v78: 129 pgs: 129 active+clean; 561 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 12 MiB/s wr, 11 op/s 2024-05-03T18:29:43.441 INFO:teuthology.orchestra.run.smithi033.stderr:2300000 of 3200000 tuples (71%) done (elapsed 80.55 s, remaining 31.52 s) 2024-05-03T18:29:45.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:29:44 smithi033 ceph-mon[28330]: pgmap v79: 129 pgs: 129 active+clean; 598 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 17 MiB/s wr, 16 op/s 2024-05-03T18:29:45.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:29:44 smithi099 ceph-mon[31658]: pgmap v79: 129 pgs: 129 active+clean; 598 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 17 MiB/s wr, 16 op/s 2024-05-03T18:29:45.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:29:44 smithi150 ceph-mon[32492]: pgmap v79: 129 pgs: 129 active+clean; 598 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 17 MiB/s wr, 16 op/s 2024-05-03T18:29:45.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.849+0000 7f78a8c00700 1 -- 172.21.15.33:0/3012198121 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78a406bb40 msgr2=0x7f78a4110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:45.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.849+0000 7f78a8c00700 1 --2- 172.21.15.33:0/3012198121 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78a406bb40 0x7f78a4110330 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f789c00b0a0 tx=0x7f789c00b3b0 comp rx=0 tx=0).stop 2024-05-03T18:29:45.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.849+0000 7f78a8c00700 1 -- 172.21.15.33:0/3012198121 shutdown_connections 2024-05-03T18:29:45.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.849+0000 7f78a8c00700 1 --2- 172.21.15.33:0/3012198121 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78a406bb40 0x7f78a4110330 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:45.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.849+0000 7f78a8c00700 1 --2- 172.21.15.33:0/3012198121 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78a4109510 0x7f78a41098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:45.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.849+0000 7f78a8c00700 1 --2- 172.21.15.33:0/3012198121 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f78a406b180 0x7f78a406b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:45.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.849+0000 7f78a8c00700 1 -- 172.21.15.33:0/3012198121 >> 172.21.15.33:0/3012198121 conn(0x7f78a4079350 msgr2=0x7f78a4079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:29:45.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.851+0000 7f78a8c00700 1 -- 172.21.15.33:0/3012198121 shutdown_connections 2024-05-03T18:29:45.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.851+0000 7f78a8c00700 1 -- 172.21.15.33:0/3012198121 wait complete. 2024-05-03T18:29:45.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.852+0000 7f78a8c00700 1 Processor -- start 2024-05-03T18:29:45.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.852+0000 7f78a8c00700 1 -- start start 2024-05-03T18:29:45.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.852+0000 7f78a8c00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78a406b180 0x7f78a419f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:45.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.852+0000 7f78a8c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f78a406bb40 0x7f78a419fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:45.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.852+0000 7f78a8c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78a4109510 0x7f78a41997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:45.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.852+0000 7f78a8c00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f78a41134d0 con 0x7f78a406b180 2024-05-03T18:29:45.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.852+0000 7f78a8c00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f78a4113350 con 0x7f78a4109510 2024-05-03T18:29:45.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.852+0000 7f78a8c00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f78a4113650 con 0x7f78a406bb40 2024-05-03T18:29:45.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.853+0000 7f78a1e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78a406b180 0x7f78a419f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:45.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.853+0000 7f78a1e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78a406b180 0x7f78a419f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52054/0 (socket says 172.21.15.33:52054) 2024-05-03T18:29:45.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.853+0000 7f78a1e00700 1 -- 172.21.15.33:0/1002150557 learned_addr learned my addr 172.21.15.33:0/1002150557 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:29:45.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.853+0000 7f78a2800700 1 --2- 172.21.15.33:0/1002150557 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78a4109510 0x7f78a41997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:45.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.853+0000 7f78a1400700 1 --2- 172.21.15.33:0/1002150557 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f78a406bb40 0x7f78a419fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:45.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.854+0000 7f78a1e00700 1 -- 172.21.15.33:0/1002150557 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f78a406bb40 msgr2=0x7f78a419fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:45.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.854+0000 7f78a1e00700 1 --2- 172.21.15.33:0/1002150557 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f78a406bb40 0x7f78a419fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:45.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.854+0000 7f78a1e00700 1 -- 172.21.15.33:0/1002150557 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78a4109510 msgr2=0x7f78a41997b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:45.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.854+0000 7f78a1e00700 1 --2- 172.21.15.33:0/1002150557 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78a4109510 0x7f78a41997b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:45.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.854+0000 7f78a1e00700 1 -- 172.21.15.33:0/1002150557 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78a4199f90 con 0x7f78a406b180 2024-05-03T18:29:45.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.854+0000 7f78a1e00700 1 --2- 172.21.15.33:0/1002150557 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78a406b180 0x7f78a419f630 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f789400d950 tx=0x7f789400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:29:45.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.854+0000 7f789aa00700 1 -- 172.21.15.33:0/1002150557 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7894016070 con 0x7f78a406b180 2024-05-03T18:29:45.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.854+0000 7f789aa00700 1 -- 172.21.15.33:0/1002150557 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7894009dd0 con 0x7f78a406b180 2024-05-03T18:29:45.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.854+0000 7f789aa00700 1 -- 172.21.15.33:0/1002150557 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f789400f3c0 con 0x7f78a406b180 2024-05-03T18:29:45.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.854+0000 7f78a8c00700 1 -- 172.21.15.33:0/1002150557 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78a419a220 con 0x7f78a406b180 2024-05-03T18:29:45.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.855+0000 7f78a8c00700 1 -- 172.21.15.33:0/1002150557 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78a41a6490 con 0x7f78a406b180 2024-05-03T18:29:45.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.859+0000 7f789aa00700 1 -- 172.21.15.33:0/1002150557 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f7894019070 con 0x7f78a406b180 2024-05-03T18:29:45.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.860+0000 7f78b34c4b80 1 Processor -- start 2024-05-03T18:29:45.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.860+0000 7f78b34c4b80 1 -- start start 2024-05-03T18:29:45.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.861+0000 7f78b34c4b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593c10a5d40 0x5593c10a6120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:45.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.861+0000 7f78b34c4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5593c10a6660 0x5593c10a0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:45.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.861+0000 7f78b34c4b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593c10a0960 0x5593c10a0de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:45.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.861+0000 7f78b34c4b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5593c0feea20 con 0x5593c10a0960 2024-05-03T18:29:45.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.861+0000 7f78b34c4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5593c0f8c760 con 0x5593c10a6660 2024-05-03T18:29:45.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.861+0000 7f78b34c4b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5593c0fea9e0 con 0x5593c10a5d40 2024-05-03T18:29:45.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.861+0000 7f789aa00700 1 --2- 172.21.15.33:0/1002150557 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f78880b29f0 0x7f78880b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:45.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.861+0000 7f789aa00700 1 -- 172.21.15.33:0/1002150557 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f78940e25f0 con 0x7f78a406b180 2024-05-03T18:29:45.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.862+0000 7f78a2800700 1 --2- 172.21.15.33:0/1002150557 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f78880b29f0 0x7f78880b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:45.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.862+0000 7f78a2800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593c10a5d40 0x5593c10a6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:45.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.862+0000 7f78a1400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593c10a0960 0x5593c10a0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:45.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.862+0000 7f78a2800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593c10a5d40 0x5593c10a6120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:55398/0 (socket says 172.21.15.33:55398) 2024-05-03T18:29:45.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.862+0000 7f78a2800700 1 -- 172.21.15.33:0/500682586 learned_addr learned my addr 172.21.15.33:0/500682586 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:29:45.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.862+0000 7f78a1e00700 1 --2- 172.21.15.33:0/500682586 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5593c10a6660 0x5593c10a0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:45.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.863+0000 7f78a2800700 1 -- 172.21.15.33:0/500682586 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5593c10a6660 msgr2=0x5593c10a0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:29:45.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.863+0000 7f78a2800700 1 --2- 172.21.15.33:0/500682586 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5593c10a6660 0x5593c10a0340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:45.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.863+0000 7f78a2800700 1 -- 172.21.15.33:0/500682586 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593c10a0960 msgr2=0x5593c10a0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:29:45.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.863+0000 7f78a2800700 1 --2- 172.21.15.33:0/500682586 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593c10a0960 0x5593c10a0de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:45.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.863+0000 7f78a2800700 1 -- 172.21.15.33:0/500682586 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593c0f3b8d0 con 0x5593c10a5d40 2024-05-03T18:29:45.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.863+0000 7f78a1400700 1 --2- 172.21.15.33:0/500682586 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593c10a0960 0x5593c10a0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:29:45.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.863+0000 7f78a2800700 1 --2- 172.21.15.33:0/1002150557 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f78880b29f0 0x7f78880b4eb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f789c015000 tx=0x7f789c009ce0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:29:45.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.863+0000 7f78a2800700 1 --2- 172.21.15.33:0/500682586 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593c10a5d40 0x5593c10a6120 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f789c003ed0 tx=0x7f789c026000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b217251f52b29d79 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:29:45.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.864+0000 7f788f400700 1 -- 172.21.15.33:0/500682586 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f789c0091f0 con 0x5593c10a5d40 2024-05-03T18:29:45.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.864+0000 7f788f400700 1 -- 172.21.15.33:0/500682586 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f789c021460 con 0x5593c10a5d40 2024-05-03T18:29:45.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.864+0000 7f788f400700 1 -- 172.21.15.33:0/500682586 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f789c01e6c0 con 0x5593c10a5d40 2024-05-03T18:29:45.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.865+0000 7f78b34c4b80 1 -- 172.21.15.33:0/500682586 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593c10a5d40 msgr2=0x5593c10a6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:45.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.865+0000 7f78b34c4b80 1 --2- 172.21.15.33:0/500682586 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593c10a5d40 0x5593c10a6120 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f789c003ed0 tx=0x7f789c026000 comp rx=0 tx=0).stop 2024-05-03T18:29:45.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.865+0000 7f78b34c4b80 1 -- 172.21.15.33:0/500682586 shutdown_connections 2024-05-03T18:29:45.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.865+0000 7f78b34c4b80 1 --2- 172.21.15.33:0/500682586 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593c10a0960 0x5593c10a0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:45.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.865+0000 7f78b34c4b80 1 --2- 172.21.15.33:0/500682586 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593c10a5d40 0x5593c10a6120 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:45.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.865+0000 7f78b34c4b80 1 --2- 172.21.15.33:0/500682586 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5593c10a6660 0x5593c10a0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:45.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.865+0000 7f78b34c4b80 1 -- 172.21.15.33:0/500682586 >> 172.21.15.33:0/500682586 conn(0x5593c10146d0 msgr2=0x5593c1096900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:29:45.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.866+0000 7f78b34c4b80 1 -- 172.21.15.33:0/500682586 shutdown_connections 2024-05-03T18:29:45.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.866+0000 7f78b34c4b80 1 -- 172.21.15.33:0/500682586 wait complete. 2024-05-03T18:29:45.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.866+0000 7f78b34c4b80 1 -- start start 2024-05-03T18:29:45.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.867+0000 7f78b34c4b80 1 Processor -- start 2024-05-03T18:29:45.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.867+0000 7f78b34c4b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593c10a0960 0x5593c109d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:45.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.867+0000 7f78b34c4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5593c10a5d40 0x5593c109ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:45.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.867+0000 7f78b34c4b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593c10a6660 0x5593c1098f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:45.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.867+0000 7f78b34c4b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5593c10ad0b0 con 0x5593c10a0960 2024-05-03T18:29:45.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.867+0000 7f78b34c4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5593c0f54b20 con 0x5593c10a5d40 2024-05-03T18:29:45.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.867+0000 7f78b34c4b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5593c10ad230 con 0x5593c10a6660 2024-05-03T18:29:45.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.867+0000 7f78b34c4b80 10 client.0 _ll_register_callbacks cb 0x5593c0f95c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:29:45.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.868+0000 7f78a1e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593c10a0960 0x5593c109d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:45.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.868+0000 7f78a1e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593c10a0960 0x5593c109d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52072/0 (socket says 172.21.15.33:52072) 2024-05-03T18:29:45.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.868+0000 7f78a1e00700 1 -- 172.21.15.33:0/210595915 learned_addr learned my addr 172.21.15.33:0/210595915 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:29:45.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.868+0000 7f78a2800700 1 --2- 172.21.15.33:0/210595915 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593c10a6660 0x5593c1098f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:45.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.868+0000 7f78a1e00700 1 -- 172.21.15.33:0/210595915 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593c10a6660 msgr2=0x5593c1098f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:29:45.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.868+0000 7f78a1e00700 1 --2- 172.21.15.33:0/210595915 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593c10a6660 0x5593c1098f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:45.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.868+0000 7f78a1400700 1 --2- 172.21.15.33:0/210595915 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5593c10a5d40 0x5593c109ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:45.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.868+0000 7f78a1e00700 1 -- 172.21.15.33:0/210595915 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5593c10a5d40 msgr2=0x5593c109ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:29:45.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.868+0000 7f78a1e00700 1 --2- 172.21.15.33:0/210595915 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5593c10a5d40 0x5593c109ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:45.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.869+0000 7f78a1e00700 1 -- 172.21.15.33:0/210595915 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593c0fb4420 con 0x5593c10a0960 2024-05-03T18:29:45.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.869+0000 7f78a1400700 1 --2- 172.21.15.33:0/210595915 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5593c10a5d40 0x5593c109ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:29:45.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.869+0000 7f78a1e00700 1 --2- 172.21.15.33:0/210595915 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593c10a0960 0x5593c109d880 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f78940a6750 tx=0x7f78940108d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b686c2b9b89c402f server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:29:45.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.869+0000 7f7883e00700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:29:45.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.869+0000 7f7883e00700 1 -- 172.21.15.33:0/210595915 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7894020040 con 0x5593c10a0960 2024-05-03T18:29:45.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.869+0000 7f7883e00700 1 -- 172.21.15.33:0/210595915 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f78940a9740 con 0x5593c10a0960 2024-05-03T18:29:45.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.870+0000 7f78b34c4b80 1 -- 172.21.15.33:0/210595915 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f789c021460 con 0x5593c10a0960 2024-05-03T18:29:45.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.870+0000 7f7883e00700 1 -- 172.21.15.33:0/210595915 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f78940e23c0 con 0x5593c10a0960 2024-05-03T18:29:45.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.870+0000 7f78b34c4b80 10 client.15660 fetch_fsmap learned FSMap version 21 2024-05-03T18:29:45.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.870+0000 7f78b34c4b80 1 -- 172.21.15.33:0/210595915 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5593c10bbd40 con 0x5593c10a0960 2024-05-03T18:29:45.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.871+0000 7f7883e00700 1 -- 172.21.15.33:0/210595915 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f789c021460 con 0x5593c10a0960 2024-05-03T18:29:45.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.871+0000 7f7883e00700 1 -- 172.21.15.33:0/210595915 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f78940a8740 con 0x5593c10a0960 2024-05-03T18:29:45.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.871+0000 7f78b34c4b80 10 client.15660 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:29:45.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.871+0000 7f78b34c4b80 10 client.15660 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:29:45.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.871+0000 7f78b34c4b80 20 client.15660 populate_metadata read hostname 'smithi033' 2024-05-03T18:29:45.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.871+0000 7f78b34c4b80 1 --2- 172.21.15.33:0/210595915 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5593c10be260 0x5593c10c0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:45.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.871+0000 7f78b34c4b80 4 client.15660 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:29:45.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.871+0000 7f78b34c4b80 1 -- 172.21.15.33:0/210595915 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5593c1068ff0 con 0x5593c10be260 2024-05-03T18:29:45.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.872+0000 7f78a1400700 1 --2- 172.21.15.33:0/210595915 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5593c10be260 0x5593c10c0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:45.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.873+0000 7f78a1400700 1 --2- 172.21.15.33:0/210595915 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5593c10be260 0x5593c10c0650 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b08ae60c1be686c server_cookie=aebe755c791f02ac in_seq=0 out_seq=0 2024-05-03T18:29:45.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.873+0000 7f7883e00700 10 client.15660 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:29:45.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.875+0000 7f7883e00700 1 -- 172.21.15.33:0/210595915 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5593c1068ff0 con 0x5593c10be260 2024-05-03T18:29:45.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.875+0000 7f7883e00700 10 client.15660 handle_command_reply: tid=0 2024-05-03T18:29:45.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.875+0000 7f78b34c4b80 1 client.15660 shutdown 2024-05-03T18:29:45.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.875+0000 7f78b34c4b80 1 -- 172.21.15.33:0/210595915 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593c10a0960 msgr2=0x5593c109d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:45.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.875+0000 7f78b34c4b80 1 --2- 172.21.15.33:0/210595915 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593c10a0960 0x5593c109d880 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f78940a6750 tx=0x7f78940108d0 comp rx=0 tx=0).stop 2024-05-03T18:29:45.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.875+0000 7f78b34c4b80 1 -- 172.21.15.33:0/210595915 shutdown_connections 2024-05-03T18:29:45.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.875+0000 7f78b34c4b80 1 --2- 172.21.15.33:0/210595915 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5593c10be260 0x5593c10c0650 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:45.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.875+0000 7f78b34c4b80 1 --2- 172.21.15.33:0/210595915 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593c10a6660 0x5593c1098f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:45.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.875+0000 7f78b34c4b80 1 --2- 172.21.15.33:0/210595915 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593c10a0960 0x5593c109d880 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:45.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.875+0000 7f78b34c4b80 1 --2- 172.21.15.33:0/210595915 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5593c10a5d40 0x5593c109ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:45.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.875+0000 7f7883e00700 0 client.15660 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:29:45.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.875+0000 7f78b34c4b80 1 -- 172.21.15.33:0/210595915 >> 172.21.15.33:0/210595915 conn(0x5593c10146d0 msgr2=0x5593c10913b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:29:45.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.876+0000 7f78b34c4b80 1 -- 172.21.15.33:0/210595915 shutdown_connections 2024-05-03T18:29:45.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.876+0000 7f78b34c4b80 1 -- 172.21.15.33:0/210595915 wait complete. 2024-05-03T18:29:45.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.876+0000 7f78b34c4b80 20 client.15660 trim_cache size 0 max 16384 2024-05-03T18:29:45.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.896+0000 7f78b34c4b80 1 Processor -- start 2024-05-03T18:29:45.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.896+0000 7f78b34c4b80 1 -- start start 2024-05-03T18:29:45.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.897+0000 7f78b34c4b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593c10a0960 0x5593c10abb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:45.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.897+0000 7f78b34c4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5593c10a5d40 0x5593c10c5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:45.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.897+0000 7f78b34c4b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593c10a6660 0x5593c10c6420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:45.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.897+0000 7f78b34c4b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f78a41ab190 con 0x5593c10a6660 2024-05-03T18:29:45.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.897+0000 7f78b34c4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f78a41ada10 con 0x5593c10a5d40 2024-05-03T18:29:45.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.897+0000 7f78b34c4b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f78a41ab800 con 0x5593c10a0960 2024-05-03T18:29:45.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.897+0000 7f78a1e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593c10a0960 0x5593c10abb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:45.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.897+0000 7f78a1400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5593c10a5d40 0x5593c10c5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:45.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.897+0000 7f78a1e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593c10a0960 0x5593c10abb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:55418/0 (socket says 172.21.15.33:55418) 2024-05-03T18:29:45.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.897+0000 7f78a1e00700 1 -- 172.21.15.33:0/3496228631 learned_addr learned my addr 172.21.15.33:0/3496228631 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:29:45.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.898+0000 7f78a2800700 1 --2- 172.21.15.33:0/3496228631 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593c10a6660 0x5593c10c6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:45.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.898+0000 7f78a1e00700 1 -- 172.21.15.33:0/3496228631 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5593c10a5d40 msgr2=0x5593c10c5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:29:45.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.898+0000 7f78a1e00700 1 --2- 172.21.15.33:0/3496228631 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5593c10a5d40 0x5593c10c5ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:45.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.898+0000 7f78a1e00700 1 -- 172.21.15.33:0/3496228631 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593c10a6660 msgr2=0x5593c10c6420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:29:45.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.898+0000 7f78a1e00700 1 --2- 172.21.15.33:0/3496228631 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593c10a6660 0x5593c10c6420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:45.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.898+0000 7f78a1e00700 1 -- 172.21.15.33:0/3496228631 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593c109e110 con 0x5593c10a0960 2024-05-03T18:29:45.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.899+0000 7f78a1e00700 1 --2- 172.21.15.33:0/3496228631 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593c10a0960 0x5593c10abb50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f78940e2560 tx=0x7f7894000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b893150a9e71f80 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:29:45.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.899+0000 7f788ea00700 1 -- 172.21.15.33:0/3496228631 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7894020030 con 0x5593c10a0960 2024-05-03T18:29:45.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.899+0000 7f788ea00700 1 -- 172.21.15.33:0/3496228631 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f78940b2340 con 0x5593c10a0960 2024-05-03T18:29:45.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.899+0000 7f788ea00700 1 -- 172.21.15.33:0/3496228631 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78940a9740 con 0x5593c10a0960 2024-05-03T18:29:45.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.900+0000 7f78b34c4b80 1 -- 172.21.15.33:0/3496228631 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593c10a0960 msgr2=0x5593c10abb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:45.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.900+0000 7f78b34c4b80 1 --2- 172.21.15.33:0/3496228631 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593c10a0960 0x5593c10abb50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f78940e2560 tx=0x7f7894000bc0 comp rx=0 tx=0).stop 2024-05-03T18:29:45.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.900+0000 7f78b34c4b80 1 -- 172.21.15.33:0/3496228631 shutdown_connections 2024-05-03T18:29:45.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.900+0000 7f78b34c4b80 1 --2- 172.21.15.33:0/3496228631 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593c10a6660 0x5593c10c6420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:45.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.900+0000 7f78b34c4b80 1 --2- 172.21.15.33:0/3496228631 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593c10a0960 0x5593c10abb50 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:45.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.900+0000 7f78b34c4b80 1 --2- 172.21.15.33:0/3496228631 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5593c10a5d40 0x5593c10c5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:45.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.900+0000 7f78b34c4b80 1 -- 172.21.15.33:0/3496228631 >> 172.21.15.33:0/3496228631 conn(0x5593c10146d0 msgr2=0x5593c10ab440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:29:45.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.900+0000 7f78b34c4b80 1 -- 172.21.15.33:0/3496228631 shutdown_connections 2024-05-03T18:29:45.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.900+0000 7f78b34c4b80 1 -- 172.21.15.33:0/3496228631 wait complete. 2024-05-03T18:29:45.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.901+0000 7f78b34c4b80 1 -- start start 2024-05-03T18:29:45.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.901+0000 7f78b34c4b80 1 Processor -- start 2024-05-03T18:29:45.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.902+0000 7f78b34c4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5593c10a0960 0x5593c10bb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:45.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.902+0000 7f78b34c4b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593c10a5d40 0x5593c10b6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:45.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.902+0000 7f78b34c4b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593c10a6660 0x5593c10b6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:45.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.902+0000 7f78b34c4b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f78a41abed0 con 0x5593c10a5d40 2024-05-03T18:29:45.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.902+0000 7f78b34c4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f78a41ad340 con 0x5593c10a0960 2024-05-03T18:29:45.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.902+0000 7f78b34c4b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f78a41ac5a0 con 0x5593c10a6660 2024-05-03T18:29:45.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.902+0000 7f78b34c4b80 10 client.0 _ll_register_callbacks cb 0x5593c1099b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:29:45.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.902+0000 7f78a1400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593c10a5d40 0x5593c10b6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:45.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.902+0000 7f78a1400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593c10a5d40 0x5593c10b6a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52088/0 (socket says 172.21.15.33:52088) 2024-05-03T18:29:45.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.902+0000 7f78a1400700 1 -- 172.21.15.33:0/272711252 learned_addr learned my addr 172.21.15.33:0/272711252 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:29:45.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.902+0000 7f78a2800700 1 --2- 172.21.15.33:0/272711252 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593c10a6660 0x5593c10b6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:45.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.903+0000 7f78a1e00700 1 --2- 172.21.15.33:0/272711252 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5593c10a0960 0x5593c10bb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:45.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.903+0000 7f78a1400700 1 -- 172.21.15.33:0/272711252 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593c10a6660 msgr2=0x5593c10b6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:29:45.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.903+0000 7f78a1400700 1 --2- 172.21.15.33:0/272711252 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593c10a6660 0x5593c10b6fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:45.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.903+0000 7f78a1400700 1 -- 172.21.15.33:0/272711252 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5593c10a0960 msgr2=0x5593c10bb6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:29:45.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.903+0000 7f78a1400700 1 --2- 172.21.15.33:0/272711252 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5593c10a0960 0x5593c10bb6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:45.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.903+0000 7f78a1400700 1 -- 172.21.15.33:0/272711252 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593c10b7820 con 0x5593c10a5d40 2024-05-03T18:29:45.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.903+0000 7f78a2800700 1 --2- 172.21.15.33:0/272711252 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593c10a6660 0x5593c10b6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:29:45.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.903+0000 7f78a1400700 1 --2- 172.21.15.33:0/272711252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593c10a5d40 0x5593c10b6a40 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x5593c10c0e80 tx=0x7f789001a850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5333b9e38547070 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:29:45.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.903+0000 7f7883e00700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:29:45.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.903+0000 7f7883e00700 1 -- 172.21.15.33:0/272711252 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7890004230 con 0x5593c10a5d40 2024-05-03T18:29:45.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.904+0000 7f78b34c4b80 1 -- 172.21.15.33:0/272711252 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78940b2340 con 0x5593c10a5d40 2024-05-03T18:29:45.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.904+0000 7f7883e00700 1 -- 172.21.15.33:0/272711252 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f78900043d0 con 0x5593c10a5d40 2024-05-03T18:29:45.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.904+0000 7f7883e00700 1 -- 172.21.15.33:0/272711252 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f789002c050 con 0x5593c10a5d40 2024-05-03T18:29:45.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.904+0000 7f78b34c4b80 10 client.15666 fetch_fsmap learned FSMap version 21 2024-05-03T18:29:45.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.904+0000 7f78b34c4b80 1 -- 172.21.15.33:0/272711252 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5593c1128c10 con 0x5593c10a5d40 2024-05-03T18:29:45.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.905+0000 7f7883e00700 1 -- 172.21.15.33:0/272711252 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f78940b2340 con 0x5593c10a5d40 2024-05-03T18:29:45.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.905+0000 7f7883e00700 1 -- 172.21.15.33:0/272711252 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f7890022030 con 0x5593c10a5d40 2024-05-03T18:29:45.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.905+0000 7f78b34c4b80 10 client.15666 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:29:45.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.905+0000 7f78b34c4b80 10 client.15666 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:29:45.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.905+0000 7f78b34c4b80 20 client.15666 populate_metadata read hostname 'smithi033' 2024-05-03T18:29:45.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.905+0000 7f78b34c4b80 1 --2- 172.21.15.33:0/272711252 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5593c1129300 0x5593c112b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:45.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.905+0000 7f78b34c4b80 4 client.15666 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-03T18:29:45.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.905+0000 7f78b34c4b80 1 -- 172.21.15.33:0/272711252 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5593c0ffc400 con 0x5593c1129300 2024-05-03T18:29:45.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.906+0000 7f78a1e00700 1 --2- 172.21.15.33:0/272711252 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5593c1129300 0x5593c112b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:45.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.907+0000 7f78a1e00700 1 --2- 172.21.15.33:0/272711252 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5593c1129300 0x5593c112b720 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad5d2bc84a58bed4 server_cookie=b2536d143cf1c989 in_seq=0 out_seq=0 2024-05-03T18:29:45.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.907+0000 7f7883e00700 10 client.15666 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:29:45.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.907+0000 7f7883e00700 1 -- 172.21.15.33:0/272711252 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5593c0ffc400 con 0x5593c1129300 2024-05-03T18:29:45.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.907+0000 7f7883e00700 10 client.15666 handle_command_reply: tid=0 2024-05-03T18:29:45.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.907+0000 7f78b34c4b80 1 client.15666 shutdown 2024-05-03T18:29:45.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.908+0000 7f78b34c4b80 1 -- 172.21.15.33:0/272711252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593c10a5d40 msgr2=0x5593c10b6a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:45.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.908+0000 7f78b34c4b80 1 --2- 172.21.15.33:0/272711252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593c10a5d40 0x5593c10b6a40 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x5593c10c0e80 tx=0x7f789001a850 comp rx=0 tx=0).stop 2024-05-03T18:29:45.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.908+0000 7f78b34c4b80 1 -- 172.21.15.33:0/272711252 shutdown_connections 2024-05-03T18:29:45.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.908+0000 7f78b34c4b80 1 --2- 172.21.15.33:0/272711252 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5593c1129300 0x5593c112b720 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:45.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.908+0000 7f78b34c4b80 1 --2- 172.21.15.33:0/272711252 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593c10a6660 0x5593c10b6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:45.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.908+0000 7f78b34c4b80 1 --2- 172.21.15.33:0/272711252 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5593c10a0960 0x5593c10bb6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:45.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.908+0000 7f78b34c4b80 1 --2- 172.21.15.33:0/272711252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593c10a5d40 0x5593c10b6a40 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:45.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.908+0000 7f78b34c4b80 1 -- 172.21.15.33:0/272711252 >> 172.21.15.33:0/272711252 conn(0x5593c10146d0 msgr2=0x5593c1090b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:29:45.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.908+0000 7f7883e00700 0 client.15666 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:29:45.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.908+0000 7f78b34c4b80 1 -- 172.21.15.33:0/272711252 shutdown_connections 2024-05-03T18:29:45.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.909+0000 7f78b34c4b80 1 -- 172.21.15.33:0/272711252 wait complete. 2024-05-03T18:29:45.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.909+0000 7f78b34c4b80 20 client.15666 trim_cache size 0 max 16384 2024-05-03T18:29:45.911 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-03T18:29:45.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.909+0000 7f78a8c00700 1 -- 172.21.15.33:0/1002150557 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f78880b29f0 msgr2=0x7f78880b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:45.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.909+0000 7f78a8c00700 1 --2- 172.21.15.33:0/1002150557 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f78880b29f0 0x7f78880b4eb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f789c015000 tx=0x7f789c009ce0 comp rx=0 tx=0).stop 2024-05-03T18:29:45.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.910+0000 7f78a8c00700 1 -- 172.21.15.33:0/1002150557 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78a406b180 msgr2=0x7f78a419f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:45.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.910+0000 7f78a8c00700 1 --2- 172.21.15.33:0/1002150557 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78a406b180 0x7f78a419f630 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f789400d950 tx=0x7f789400dc60 comp rx=0 tx=0).stop 2024-05-03T18:29:45.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.910+0000 7f78a8c00700 1 -- 172.21.15.33:0/1002150557 shutdown_connections 2024-05-03T18:29:45.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.910+0000 7f78a8c00700 1 --2- 172.21.15.33:0/1002150557 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f78880b29f0 0x7f78880b4eb0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:45.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.910+0000 7f78a8c00700 1 --2- 172.21.15.33:0/1002150557 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78a4109510 0x7f78a41997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:45.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.910+0000 7f78a8c00700 1 --2- 172.21.15.33:0/1002150557 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78a406b180 0x7f78a419f630 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:45.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.910+0000 7f78a8c00700 1 --2- 172.21.15.33:0/1002150557 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f78a406bb40 0x7f78a419fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:45.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.910+0000 7f78a8c00700 1 -- 172.21.15.33:0/1002150557 >> 172.21.15.33:0/1002150557 conn(0x7f78a4079350 msgr2=0x7f78a4101030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:29:45.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.910+0000 7f78a8c00700 1 -- 172.21.15.33:0/1002150557 shutdown_connections 2024-05-03T18:29:45.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:45.910+0000 7f78a8c00700 1 -- 172.21.15.33:0/1002150557 wait complete. 2024-05-03T18:29:46.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:29:45 smithi033 ceph-mon[28330]: pgmap v80: 129 pgs: 129 active+clean; 598 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 11 MiB/s wr, 10 op/s 2024-05-03T18:29:46.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:29:45 smithi099 ceph-mon[31658]: pgmap v80: 129 pgs: 129 active+clean; 598 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 11 MiB/s wr, 10 op/s 2024-05-03T18:29:46.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:29:45 smithi150 ceph-mon[32492]: pgmap v80: 129 pgs: 129 active+clean; 598 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 11 MiB/s wr, 10 op/s 2024-05-03T18:29:47.100 INFO:teuthology.orchestra.run.smithi033.stderr:2400000 of 3200000 tuples (75%) done (elapsed 84.21 s, remaining 28.07 s) 2024-05-03T18:29:47.831 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-03T18:29:47.832 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-03T18:29:48.352 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:29:49.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:29:48 smithi033 ceph-mon[28330]: pgmap v81: 129 pgs: 129 active+clean; 630 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 17 MiB/s wr, 15 op/s 2024-05-03T18:29:49.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:29:48 smithi099 ceph-mon[31658]: pgmap v81: 129 pgs: 129 active+clean; 630 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 17 MiB/s wr, 15 op/s 2024-05-03T18:29:49.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:29:48 smithi150 ceph-mon[32492]: pgmap v81: 129 pgs: 129 active+clean; 630 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 17 MiB/s wr, 15 op/s 2024-05-03T18:29:50.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.673+0000 7f55dbe00700 1 -- 172.21.15.33:0/539935217 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55dc104590 msgr2=0x7f55dc104970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:50.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.673+0000 7f55dbe00700 1 --2- 172.21.15.33:0/539935217 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55dc104590 0x7f55dc104970 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f55cc009990 tx=0x7f55cc009ca0 comp rx=0 tx=0).stop 2024-05-03T18:29:50.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.674+0000 7f55dbe00700 1 -- 172.21.15.33:0/539935217 shutdown_connections 2024-05-03T18:29:50.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.674+0000 7f55dbe00700 1 --2- 172.21.15.33:0/539935217 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55dc10d9c0 0x7f55dc10fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:50.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.674+0000 7f55dbe00700 1 --2- 172.21.15.33:0/539935217 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55dc104590 0x7f55dc104970 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:50.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.674+0000 7f55dbe00700 1 --2- 172.21.15.33:0/539935217 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f55dc104eb0 0x7f55dc10d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:50.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.674+0000 7f55dbe00700 1 -- 172.21.15.33:0/539935217 >> 172.21.15.33:0/539935217 conn(0x7f55dc0fddc0 msgr2=0x7f55dc1001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:29:50.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.674+0000 7f55dbe00700 1 -- 172.21.15.33:0/539935217 shutdown_connections 2024-05-03T18:29:50.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.674+0000 7f55dbe00700 1 -- 172.21.15.33:0/539935217 wait complete. 2024-05-03T18:29:50.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.675+0000 7f55dbe00700 1 Processor -- start 2024-05-03T18:29:50.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.675+0000 7f55dbe00700 1 -- start start 2024-05-03T18:29:50.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.676+0000 7f55dbe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55dc104590 0x7f55dc105410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:50.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.676+0000 7f55dbe00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f55dc104eb0 0x7f55dc105950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:50.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.676+0000 7f55dbe00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55dc10d9c0 0x7f55dc105e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:50.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.676+0000 7f55dbe00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f55dc1134e0 con 0x7f55dc10d9c0 2024-05-03T18:29:50.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.676+0000 7f55dbe00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f55dc113360 con 0x7f55dc104590 2024-05-03T18:29:50.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.676+0000 7f55dbe00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f55dc113660 con 0x7f55dc104eb0 2024-05-03T18:29:50.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.676+0000 7f55db400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55dc10d9c0 0x7f55dc105e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:50.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.676+0000 7f55db400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55dc10d9c0 0x7f55dc105e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52116/0 (socket says 172.21.15.33:52116) 2024-05-03T18:29:50.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.676+0000 7f55db400700 1 -- 172.21.15.33:0/106680469 learned_addr learned my addr 172.21.15.33:0/106680469 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:29:50.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.677+0000 7f55da000700 1 --2- 172.21.15.33:0/106680469 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f55dc104eb0 0x7f55dc105950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:50.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.677+0000 7f55daa00700 1 --2- 172.21.15.33:0/106680469 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55dc104590 0x7f55dc105410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:50.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.677+0000 7f55db400700 1 -- 172.21.15.33:0/106680469 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f55dc104eb0 msgr2=0x7f55dc105950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:50.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.677+0000 7f55db400700 1 --2- 172.21.15.33:0/106680469 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f55dc104eb0 0x7f55dc105950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:50.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.677+0000 7f55db400700 1 -- 172.21.15.33:0/106680469 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55dc104590 msgr2=0x7f55dc105410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:50.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.677+0000 7f55db400700 1 --2- 172.21.15.33:0/106680469 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55dc104590 0x7f55dc105410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:50.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.677+0000 7f55db400700 1 -- 172.21.15.33:0/106680469 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55dc1ac270 con 0x7f55dc10d9c0 2024-05-03T18:29:50.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.678+0000 7f55db400700 1 --2- 172.21.15.33:0/106680469 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55dc10d9c0 0x7f55dc105e90 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f55d4007d70 tx=0x7f55d400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:29:50.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.678+0000 7f55d3400700 1 -- 172.21.15.33:0/106680469 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d4016020 con 0x7f55dc10d9c0 2024-05-03T18:29:50.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.678+0000 7f55d3400700 1 -- 172.21.15.33:0/106680469 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f55d400ac90 con 0x7f55dc10d9c0 2024-05-03T18:29:50.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.678+0000 7f55dbe00700 1 -- 172.21.15.33:0/106680469 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55dc1ac560 con 0x7f55dc10d9c0 2024-05-03T18:29:50.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.678+0000 7f55d3400700 1 -- 172.21.15.33:0/106680469 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d40115e0 con 0x7f55dc10d9c0 2024-05-03T18:29:50.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.678+0000 7f55dbe00700 1 -- 172.21.15.33:0/106680469 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55dc1acaa0 con 0x7f55dc10d9c0 2024-05-03T18:29:50.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.683+0000 7f55d3400700 1 -- 172.21.15.33:0/106680469 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f55d4019020 con 0x7f55dc10d9c0 2024-05-03T18:29:50.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.684+0000 7f55ec1f5b80 1 Processor -- start 2024-05-03T18:29:50.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.684+0000 7f55ec1f5b80 1 -- start start 2024-05-03T18:29:50.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.684+0000 7f55ec1f5b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c478371ac0 0x55c478371ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:50.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.684+0000 7f55ec1f5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c4783723e0 0x55c47836c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:50.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.684+0000 7f55ec1f5b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c47836c650 0x55c47836cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:50.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.684+0000 7f55ec1f5b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c4782beaa0 con 0x55c47836c650 2024-05-03T18:29:50.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.684+0000 7f55ec1f5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55c47825c7e0 con 0x55c4783723e0 2024-05-03T18:29:50.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.684+0000 7f55ec1f5b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55c4782baa60 con 0x55c478371ac0 2024-05-03T18:29:50.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.685+0000 7f55da000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c47836c650 0x55c47836cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:50.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.685+0000 7f55da000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c47836c650 0x55c47836cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52128/0 (socket says 172.21.15.33:52128) 2024-05-03T18:29:50.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.685+0000 7f55da000700 1 -- 172.21.15.33:0/3920834991 learned_addr learned my addr 172.21.15.33:0/3920834991 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:29:50.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.685+0000 7f55daa00700 1 --2- 172.21.15.33:0/3920834991 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c4783723e0 0x55c47836c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:50.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.685+0000 7f55da000700 1 --2- 172.21.15.33:0/3920834991 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c478371ac0 0x55c478371ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:50.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.685+0000 7f55d3400700 1 --2- 172.21.15.33:0/106680469 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f55c00b2a70 0x7f55c00b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:50.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.686+0000 7f55da000700 1 -- 172.21.15.33:0/3920834991 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c478371ac0 msgr2=0x55c478371ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:29:50.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.686+0000 7f55da000700 1 --2- 172.21.15.33:0/3920834991 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c478371ac0 0x55c478371ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:50.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.686+0000 7f55da000700 1 -- 172.21.15.33:0/3920834991 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c4783723e0 msgr2=0x55c47836c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:29:50.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.686+0000 7f55da000700 1 --2- 172.21.15.33:0/3920834991 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c4783723e0 0x55c47836c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:50.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.686+0000 7f55da000700 1 -- 172.21.15.33:0/3920834991 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c47820b950 con 0x55c47836c650 2024-05-03T18:29:50.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.686+0000 7f55db400700 1 --2- 172.21.15.33:0/106680469 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f55c00b2a70 0x7f55c00b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:50.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.686+0000 7f55da000700 1 --2- 172.21.15.33:0/3920834991 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c47836c650 0x55c47836cab0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f55c8012740 tx=0x7f55c8005d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e1179f76277ab02 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:29:50.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.686+0000 7f55d3400700 1 -- 172.21.15.33:0/106680469 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f55d40e2620 con 0x7f55dc10d9c0 2024-05-03T18:29:50.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.686+0000 7f55c7e00700 1 -- 172.21.15.33:0/3920834991 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c8004030 con 0x55c47836c650 2024-05-03T18:29:50.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.687+0000 7f55c7e00700 1 -- 172.21.15.33:0/3920834991 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f55c8004830 con 0x55c47836c650 2024-05-03T18:29:50.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.687+0000 7f55c7e00700 1 -- 172.21.15.33:0/3920834991 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c8005280 con 0x55c47836c650 2024-05-03T18:29:50.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.688+0000 7f55ec1f5b80 1 -- 172.21.15.33:0/3920834991 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c47836c650 msgr2=0x55c47836cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:50.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.688+0000 7f55ec1f5b80 1 --2- 172.21.15.33:0/3920834991 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c47836c650 0x55c47836cab0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f55c8012740 tx=0x7f55c8005d20 comp rx=0 tx=0).stop 2024-05-03T18:29:50.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.688+0000 7f55db400700 1 --2- 172.21.15.33:0/106680469 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f55c00b2a70 0x7f55c00b4f30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f55d40a9320 tx=0x7f55d40cc040 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:29:50.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.688+0000 7f55ec1f5b80 1 -- 172.21.15.33:0/3920834991 shutdown_connections 2024-05-03T18:29:50.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.688+0000 7f55ec1f5b80 1 --2- 172.21.15.33:0/3920834991 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c47836c650 0x55c47836cab0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:50.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.688+0000 7f55ec1f5b80 1 --2- 172.21.15.33:0/3920834991 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c478371ac0 0x55c478371ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:50.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.688+0000 7f55ec1f5b80 1 --2- 172.21.15.33:0/3920834991 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c4783723e0 0x55c47836c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:50.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.688+0000 7f55ec1f5b80 1 -- 172.21.15.33:0/3920834991 >> 172.21.15.33:0/3920834991 conn(0x55c47829abe0 msgr2=0x55c478366a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:29:50.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.688+0000 7f55ec1f5b80 1 -- 172.21.15.33:0/3920834991 shutdown_connections 2024-05-03T18:29:50.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.688+0000 7f55ec1f5b80 1 -- 172.21.15.33:0/3920834991 wait complete. 2024-05-03T18:29:50.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.689+0000 7f55ec1f5b80 1 -- start start 2024-05-03T18:29:50.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.689+0000 7f55ec1f5b80 1 Processor -- start 2024-05-03T18:29:50.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.690+0000 7f55ec1f5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c47836c650 0x55c4783780f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:50.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.690+0000 7f55ec1f5b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c478371ac0 0x55c478375490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:50.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.690+0000 7f55ec1f5b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c4783723e0 0x55c4783759d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:50.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.690+0000 7f55ec1f5b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c478378e50 con 0x55c478371ac0 2024-05-03T18:29:50.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.690+0000 7f55ec1f5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55c478224ba0 con 0x55c47836c650 2024-05-03T18:29:50.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.690+0000 7f55ec1f5b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55c478378fd0 con 0x55c4783723e0 2024-05-03T18:29:50.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.690+0000 7f55ec1f5b80 10 client.0 _ll_register_callbacks cb 0x55c478265cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:29:50.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.690+0000 7f55daa00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c478371ac0 0x55c478375490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:50.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.690+0000 7f55da000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c47836c650 0x55c4783780f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:50.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.690+0000 7f55daa00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c478371ac0 0x55c478375490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52138/0 (socket says 172.21.15.33:52138) 2024-05-03T18:29:50.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.690+0000 7f55daa00700 1 -- 172.21.15.33:0/2918168188 learned_addr learned my addr 172.21.15.33:0/2918168188 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:29:50.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.690+0000 7f55da000700 1 --2- 172.21.15.33:0/2918168188 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c4783723e0 0x55c4783759d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:50.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.691+0000 7f55daa00700 1 -- 172.21.15.33:0/2918168188 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c4783723e0 msgr2=0x55c4783759d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:29:50.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.691+0000 7f55daa00700 1 --2- 172.21.15.33:0/2918168188 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c4783723e0 0x55c4783759d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:50.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.691+0000 7f55daa00700 1 -- 172.21.15.33:0/2918168188 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c47836c650 msgr2=0x55c4783780f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:29:50.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.691+0000 7f55daa00700 1 --2- 172.21.15.33:0/2918168188 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c47836c650 0x55c4783780f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:50.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.691+0000 7f55daa00700 1 -- 172.21.15.33:0/2918168188 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4782844a0 con 0x55c478371ac0 2024-05-03T18:29:50.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.692+0000 7f55daa00700 1 --2- 172.21.15.33:0/2918168188 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c478371ac0 0x55c478375490 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f55cc003f80 tx=0x7f55cc005c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adf2a121f099b972 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:29:50.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.692+0000 7f55c4c00700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:29:50.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.692+0000 7f55c4c00700 1 -- 172.21.15.33:0/2918168188 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55cc028060 con 0x55c478371ac0 2024-05-03T18:29:50.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.692+0000 7f55c4c00700 1 -- 172.21.15.33:0/2918168188 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f55cc005240 con 0x55c478371ac0 2024-05-03T18:29:50.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.692+0000 7f55ec1f5b80 1 -- 172.21.15.33:0/2918168188 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55c8004830 con 0x55c478371ac0 2024-05-03T18:29:50.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.693+0000 7f55c4c00700 1 -- 172.21.15.33:0/2918168188 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f55cc02f050 con 0x55c478371ac0 2024-05-03T18:29:50.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.693+0000 7f55ec1f5b80 10 client.15678 fetch_fsmap learned FSMap version 21 2024-05-03T18:29:50.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.693+0000 7f55ec1f5b80 1 -- 172.21.15.33:0/2918168188 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55c478383750 con 0x55c478371ac0 2024-05-03T18:29:50.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.693+0000 7f55c4c00700 1 -- 172.21.15.33:0/2918168188 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f55c8004830 con 0x55c478371ac0 2024-05-03T18:29:50.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.694+0000 7f55c4c00700 1 -- 172.21.15.33:0/2918168188 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f55cc00f460 con 0x55c478371ac0 2024-05-03T18:29:50.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.694+0000 7f55ec1f5b80 10 client.15678 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:29:50.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.694+0000 7f55ec1f5b80 10 client.15678 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:29:50.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.694+0000 7f55ec1f5b80 20 client.15678 populate_metadata read hostname 'smithi033' 2024-05-03T18:29:50.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.694+0000 7f55ec1f5b80 1 --2- 172.21.15.33:0/2918168188 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55c478385d90 0x55c4783881a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:50.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.694+0000 7f55ec1f5b80 4 client.15678 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:29:50.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.694+0000 7f55ec1f5b80 1 -- 172.21.15.33:0/2918168188 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c478339250 con 0x55c478385d90 2024-05-03T18:29:50.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.695+0000 7f55da000700 1 --2- 172.21.15.33:0/2918168188 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55c478385d90 0x55c4783881a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:50.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.696+0000 7f55c4c00700 10 client.15678 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:29:50.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.696+0000 7f55da000700 1 --2- 172.21.15.33:0/2918168188 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55c478385d90 0x55c4783881a0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e749d39fa2b8840 server_cookie=6945e5a932bb9214 in_seq=0 out_seq=0 2024-05-03T18:29:50.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.698+0000 7f55c4c00700 1 -- 172.21.15.33:0/2918168188 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c478339250 con 0x55c478385d90 2024-05-03T18:29:50.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.698+0000 7f55c4c00700 10 client.15678 handle_command_reply: tid=0 2024-05-03T18:29:50.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.698+0000 7f55ec1f5b80 1 client.15678 shutdown 2024-05-03T18:29:50.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.699+0000 7f55ec1f5b80 1 -- 172.21.15.33:0/2918168188 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c478371ac0 msgr2=0x55c478375490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:50.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.699+0000 7f55ec1f5b80 1 --2- 172.21.15.33:0/2918168188 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c478371ac0 0x55c478375490 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f55cc003f80 tx=0x7f55cc005c40 comp rx=0 tx=0).stop 2024-05-03T18:29:50.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.699+0000 7f55ec1f5b80 1 -- 172.21.15.33:0/2918168188 shutdown_connections 2024-05-03T18:29:50.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.699+0000 7f55ec1f5b80 1 --2- 172.21.15.33:0/2918168188 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55c478385d90 0x55c4783881a0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:50.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.699+0000 7f55ec1f5b80 1 --2- 172.21.15.33:0/2918168188 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c4783723e0 0x55c4783759d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:50.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.699+0000 7f55ec1f5b80 1 --2- 172.21.15.33:0/2918168188 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c47836c650 0x55c4783780f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:50.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.699+0000 7f55ec1f5b80 1 --2- 172.21.15.33:0/2918168188 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c478371ac0 0x55c478375490 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:50.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.699+0000 7f55ec1f5b80 1 -- 172.21.15.33:0/2918168188 >> 172.21.15.33:0/2918168188 conn(0x55c47829abe0 msgr2=0x55c478366100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:29:50.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.699+0000 7f55c4c00700 0 client.15678 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:29:50.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.699+0000 7f55ec1f5b80 1 -- 172.21.15.33:0/2918168188 shutdown_connections 2024-05-03T18:29:50.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.699+0000 7f55ec1f5b80 1 -- 172.21.15.33:0/2918168188 wait complete. 2024-05-03T18:29:50.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.699+0000 7f55ec1f5b80 20 client.15678 trim_cache size 0 max 16384 2024-05-03T18:29:50.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.729+0000 7f55ec1f5b80 1 Processor -- start 2024-05-03T18:29:50.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.729+0000 7f55ec1f5b80 1 -- start start 2024-05-03T18:29:50.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.730+0000 7f55ec1f5b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c47836c650 0x55c47836c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:50.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.730+0000 7f55ec1f5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c478371ac0 0x55c478381f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:50.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.730+0000 7f55ec1f5b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4783723e0 0x55c47838dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:50.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.730+0000 7f55ec1f5b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f55bc00ccb0 con 0x55c4783723e0 2024-05-03T18:29:50.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.730+0000 7f55ec1f5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f55bc00f530 con 0x55c478371ac0 2024-05-03T18:29:50.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.730+0000 7f55ec1f5b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f55bc00d320 con 0x55c47836c650 2024-05-03T18:29:50.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.730+0000 7f55da000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4783723e0 0x55c47838dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:50.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.730+0000 7f55da000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4783723e0 0x55c47838dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52150/0 (socket says 172.21.15.33:52150) 2024-05-03T18:29:50.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.730+0000 7f55da000700 1 -- 172.21.15.33:0/3915616763 learned_addr learned my addr 172.21.15.33:0/3915616763 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:29:50.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.730+0000 7f55daa00700 1 --2- 172.21.15.33:0/3915616763 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c478371ac0 0x55c478381f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:50.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.730+0000 7f55da000700 1 --2- 172.21.15.33:0/3915616763 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c47836c650 0x55c47836c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:50.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.731+0000 7f55da000700 1 -- 172.21.15.33:0/3915616763 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c47836c650 msgr2=0x55c47836c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:29:50.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.731+0000 7f55da000700 1 --2- 172.21.15.33:0/3915616763 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c47836c650 0x55c47836c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:50.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.731+0000 7f55da000700 1 -- 172.21.15.33:0/3915616763 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c478371ac0 msgr2=0x55c478381f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:29:50.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.731+0000 7f55da000700 1 --2- 172.21.15.33:0/3915616763 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c478371ac0 0x55c478381f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:50.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.731+0000 7f55da000700 1 -- 172.21.15.33:0/3915616763 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c478368550 con 0x55c4783723e0 2024-05-03T18:29:50.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.731+0000 7f55daa00700 1 --2- 172.21.15.33:0/3915616763 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c478371ac0 0x55c478381f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:29:50.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.731+0000 7f55da000700 1 --2- 172.21.15.33:0/3915616763 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4783723e0 0x55c47838dab0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f55c8000c00 tx=0x7f55c8017040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d16802b8c190737 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:29:50.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.731+0000 7f55c7400700 1 -- 172.21.15.33:0/3915616763 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c8034070 con 0x55c4783723e0 2024-05-03T18:29:50.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.731+0000 7f55c7400700 1 -- 172.21.15.33:0/3915616763 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f55c802f350 con 0x55c4783723e0 2024-05-03T18:29:50.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.731+0000 7f55c7400700 1 -- 172.21.15.33:0/3915616763 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c8003680 con 0x55c4783723e0 2024-05-03T18:29:50.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.732+0000 7f55ec1f5b80 1 -- 172.21.15.33:0/3915616763 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4783723e0 msgr2=0x55c47838dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:50.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.732+0000 7f55ec1f5b80 1 --2- 172.21.15.33:0/3915616763 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4783723e0 0x55c47838dab0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f55c8000c00 tx=0x7f55c8017040 comp rx=0 tx=0).stop 2024-05-03T18:29:50.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.733+0000 7f55ec1f5b80 1 -- 172.21.15.33:0/3915616763 shutdown_connections 2024-05-03T18:29:50.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.733+0000 7f55ec1f5b80 1 --2- 172.21.15.33:0/3915616763 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4783723e0 0x55c47838dab0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:50.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.733+0000 7f55ec1f5b80 1 --2- 172.21.15.33:0/3915616763 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c47836c650 0x55c47836c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:50.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.733+0000 7f55ec1f5b80 1 --2- 172.21.15.33:0/3915616763 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c478371ac0 0x55c478381f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:50.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.733+0000 7f55ec1f5b80 1 -- 172.21.15.33:0/3915616763 >> 172.21.15.33:0/3915616763 conn(0x55c47829abe0 msgr2=0x55c478373d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:29:50.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.733+0000 7f55ec1f5b80 1 -- 172.21.15.33:0/3915616763 shutdown_connections 2024-05-03T18:29:50.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.733+0000 7f55ec1f5b80 1 -- 172.21.15.33:0/3915616763 wait complete. 2024-05-03T18:29:50.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.733+0000 7f55ec1f5b80 1 -- start start 2024-05-03T18:29:50.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.733+0000 7f55ec1f5b80 1 Processor -- start 2024-05-03T18:29:50.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.734+0000 7f55ec1f5b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c47836c650 0x55c47837e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:50.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.734+0000 7f55ec1f5b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c478371ac0 0x55c47837e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:50.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.734+0000 7f55ec1f5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c4783723e0 0x55c47837b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:50.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.734+0000 7f55ec1f5b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f55bc00d9f0 con 0x55c47836c650 2024-05-03T18:29:50.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.734+0000 7f55ec1f5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f55bc00ee60 con 0x55c4783723e0 2024-05-03T18:29:50.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.734+0000 7f55ec1f5b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f55bc00e0c0 con 0x55c478371ac0 2024-05-03T18:29:50.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.734+0000 7f55ec1f5b80 10 client.0 _ll_register_callbacks cb 0x55c478265cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:29:50.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.735+0000 7f55da000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c47836c650 0x55c47837e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:50.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.735+0000 7f55da000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c47836c650 0x55c47837e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52158/0 (socket says 172.21.15.33:52158) 2024-05-03T18:29:50.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.735+0000 7f55da000700 1 -- 172.21.15.33:0/1094466782 learned_addr learned my addr 172.21.15.33:0/1094466782 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:29:50.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.735+0000 7f55daa00700 1 --2- 172.21.15.33:0/1094466782 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c478371ac0 0x55c47837e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:50.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.735+0000 7f55da000700 1 -- 172.21.15.33:0/1094466782 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c478371ac0 msgr2=0x55c47837e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:29:50.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.735+0000 7f55da000700 1 --2- 172.21.15.33:0/1094466782 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c478371ac0 0x55c47837e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:50.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.735+0000 7f55da000700 1 -- 172.21.15.33:0/1094466782 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c4783723e0 msgr2=0x55c47837b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:29:50.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.735+0000 7f55da000700 1 --2- 172.21.15.33:0/1094466782 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c4783723e0 0x55c47837b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:50.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.735+0000 7f55da000700 1 -- 172.21.15.33:0/1094466782 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c47837bad0 con 0x55c47836c650 2024-05-03T18:29:50.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.736+0000 7f55da000700 1 --2- 172.21.15.33:0/1094466782 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c47836c650 0x55c47837e2c0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f55c800b710 tx=0x7f55c800b3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae091c83783c4838 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:29:50.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.736+0000 7f55c4c00700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:29:50.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.736+0000 7f55c4c00700 1 -- 172.21.15.33:0/1094466782 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c8034030 con 0x55c47836c650 2024-05-03T18:29:50.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.736+0000 7f55ec1f5b80 1 -- 172.21.15.33:0/1094466782 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55c802f350 con 0x55c47836c650 2024-05-03T18:29:50.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.736+0000 7f55c4c00700 1 -- 172.21.15.33:0/1094466782 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f55c801b040 con 0x55c47836c650 2024-05-03T18:29:50.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.737+0000 7f55c4c00700 1 -- 172.21.15.33:0/1094466782 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f55c803e050 con 0x55c47836c650 2024-05-03T18:29:50.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.737+0000 7f55ec1f5b80 10 client.15684 fetch_fsmap learned FSMap version 21 2024-05-03T18:29:50.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.737+0000 7f55ec1f5b80 1 -- 172.21.15.33:0/1094466782 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55c4783f1110 con 0x55c47836c650 2024-05-03T18:29:50.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.737+0000 7f55c4c00700 1 -- 172.21.15.33:0/1094466782 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f55c802f350 con 0x55c47836c650 2024-05-03T18:29:50.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.737+0000 7f55c4c00700 1 -- 172.21.15.33:0/1094466782 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f55c802d030 con 0x55c47836c650 2024-05-03T18:29:50.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.737+0000 7f55ec1f5b80 10 client.15684 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:29:50.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.737+0000 7f55ec1f5b80 10 client.15684 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:29:50.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.737+0000 7f55ec1f5b80 20 client.15684 populate_metadata read hostname 'smithi033' 2024-05-03T18:29:50.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.737+0000 7f55ec1f5b80 1 --2- 172.21.15.33:0/1094466782 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55c4783f15b0 0x55c4783f39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:50.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.737+0000 7f55ec1f5b80 4 client.15684 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-03T18:29:50.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.737+0000 7f55ec1f5b80 1 -- 172.21.15.33:0/1094466782 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55c4782cc480 con 0x55c4783f15b0 2024-05-03T18:29:50.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.738+0000 7f55daa00700 1 --2- 172.21.15.33:0/1094466782 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55c4783f15b0 0x55c4783f39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:50.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.739+0000 7f55daa00700 1 --2- 172.21.15.33:0/1094466782 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55c4783f15b0 0x55c4783f39d0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57c2a59e79055bef server_cookie=627970609ba01b3 in_seq=0 out_seq=0 2024-05-03T18:29:50.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.739+0000 7f55c4c00700 10 client.15684 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:29:50.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.741+0000 7f55c4c00700 1 -- 172.21.15.33:0/1094466782 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c4782cc480 con 0x55c4783f15b0 2024-05-03T18:29:50.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.741+0000 7f55c4c00700 10 client.15684 handle_command_reply: tid=0 2024-05-03T18:29:50.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.741+0000 7f55ec1f5b80 1 client.15684 shutdown 2024-05-03T18:29:50.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.742+0000 7f55ec1f5b80 1 -- 172.21.15.33:0/1094466782 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c47836c650 msgr2=0x55c47837e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:50.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.742+0000 7f55ec1f5b80 1 --2- 172.21.15.33:0/1094466782 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c47836c650 0x55c47837e2c0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f55c800b710 tx=0x7f55c800b3a0 comp rx=0 tx=0).stop 2024-05-03T18:29:50.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.742+0000 7f55ec1f5b80 1 -- 172.21.15.33:0/1094466782 shutdown_connections 2024-05-03T18:29:50.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.742+0000 7f55ec1f5b80 1 --2- 172.21.15.33:0/1094466782 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55c4783f15b0 0x55c4783f39d0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:50.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.742+0000 7f55ec1f5b80 1 --2- 172.21.15.33:0/1094466782 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c4783723e0 0x55c47837b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:50.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.742+0000 7f55ec1f5b80 1 --2- 172.21.15.33:0/1094466782 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c47836c650 0x55c47837e2c0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:50.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.742+0000 7f55ec1f5b80 1 --2- 172.21.15.33:0/1094466782 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c478371ac0 0x55c47837e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:50.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.742+0000 7f55ec1f5b80 1 -- 172.21.15.33:0/1094466782 >> 172.21.15.33:0/1094466782 conn(0x55c47829abe0 msgr2=0x55c4783727c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:29:50.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.742+0000 7f55c4c00700 0 client.15684 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:29:50.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.742+0000 7f55ec1f5b80 1 -- 172.21.15.33:0/1094466782 shutdown_connections 2024-05-03T18:29:50.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.742+0000 7f55ec1f5b80 1 -- 172.21.15.33:0/1094466782 wait complete. 2024-05-03T18:29:50.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.742+0000 7f55ec1f5b80 20 client.15684 trim_cache size 0 max 16384 2024-05-03T18:29:50.744 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-03T18:29:50.744 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-05-03T18:29:50.744 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "7aba77ee-9e20-4e15-b94d-13218dcfcf69", 2024-05-03T18:29:50.744 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-05-03T18:29:50.744 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-03T18:29:50.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.743+0000 7f55dbe00700 1 -- 172.21.15.33:0/106680469 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f55c00b2a70 msgr2=0x7f55c00b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:50.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.743+0000 7f55dbe00700 1 --2- 172.21.15.33:0/106680469 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f55c00b2a70 0x7f55c00b4f30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f55d40a9320 tx=0x7f55d40cc040 comp rx=0 tx=0).stop 2024-05-03T18:29:50.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.743+0000 7f55dbe00700 1 -- 172.21.15.33:0/106680469 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55dc10d9c0 msgr2=0x7f55dc105e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:50.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.743+0000 7f55dbe00700 1 --2- 172.21.15.33:0/106680469 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55dc10d9c0 0x7f55dc105e90 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f55d4007d70 tx=0x7f55d400a340 comp rx=0 tx=0).stop 2024-05-03T18:29:50.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.743+0000 7f55dbe00700 1 -- 172.21.15.33:0/106680469 shutdown_connections 2024-05-03T18:29:50.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.743+0000 7f55dbe00700 1 --2- 172.21.15.33:0/106680469 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55dc10d9c0 0x7f55dc105e90 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:50.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.743+0000 7f55dbe00700 1 --2- 172.21.15.33:0/106680469 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f55c00b2a70 0x7f55c00b4f30 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:50.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.744+0000 7f55dbe00700 1 --2- 172.21.15.33:0/106680469 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55dc104590 0x7f55dc105410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:50.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.744+0000 7f55dbe00700 1 --2- 172.21.15.33:0/106680469 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f55dc104eb0 0x7f55dc105950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:50.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.744+0000 7f55dbe00700 1 -- 172.21.15.33:0/106680469 >> 172.21.15.33:0/106680469 conn(0x7f55dc0fddc0 msgr2=0x7f55dc1001b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:29:50.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.744+0000 7f55dbe00700 1 -- 172.21.15.33:0/106680469 shutdown_connections 2024-05-03T18:29:50.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:50.744+0000 7f55dbe00700 1 -- 172.21.15.33:0/106680469 wait complete. 2024-05-03T18:29:50.769 INFO:teuthology.orchestra.run.smithi033.stderr:2500000 of 3200000 tuples (78%) done (elapsed 87.88 s, remaining 24.61 s) 2024-05-03T18:29:50.903 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:29:50 smithi033 ceph-mon[28330]: pgmap v82: 129 pgs: 129 active+clean; 630 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 MiB/s wr, 10 op/s 2024-05-03T18:29:51.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:29:50 smithi099 ceph-mon[31658]: pgmap v82: 129 pgs: 129 active+clean; 630 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 MiB/s wr, 10 op/s 2024-05-03T18:29:51.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:29:50 smithi150 ceph-mon[32492]: pgmap v82: 129 pgs: 129 active+clean; 630 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 MiB/s wr, 10 op/s 2024-05-03T18:29:51.622 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-03T18:29:52.133 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:29:52.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:29:51 smithi033 ceph-mon[28330]: scrub queued for path: / 2024-05-03T18:29:52.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:29:51 smithi033 ceph-mon[28330]: scrub summary: idle+waiting paths [/] 2024-05-03T18:29:52.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:29:51 smithi033 ceph-mon[28330]: scrub summary: active paths [/] 2024-05-03T18:29:52.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:29:51 smithi033 ceph-mon[28330]: scrub summary: idle+waiting paths [/] 2024-05-03T18:29:52.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:29:51 smithi033 ceph-mon[28330]: scrub summary: idle 2024-05-03T18:29:52.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:29:51 smithi099 ceph-mon[31658]: scrub queued for path: / 2024-05-03T18:29:52.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:29:51 smithi099 ceph-mon[31658]: scrub summary: idle+waiting paths [/] 2024-05-03T18:29:52.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:29:51 smithi099 ceph-mon[31658]: scrub summary: active paths [/] 2024-05-03T18:29:52.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:29:51 smithi099 ceph-mon[31658]: scrub summary: idle+waiting paths [/] 2024-05-03T18:29:52.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:29:51 smithi099 ceph-mon[31658]: scrub summary: idle 2024-05-03T18:29:52.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:29:51 smithi150 ceph-mon[32492]: scrub queued for path: / 2024-05-03T18:29:52.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:29:51 smithi150 ceph-mon[32492]: scrub summary: idle+waiting paths [/] 2024-05-03T18:29:52.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:29:51 smithi150 ceph-mon[32492]: scrub summary: active paths [/] 2024-05-03T18:29:52.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:29:51 smithi150 ceph-mon[32492]: scrub summary: idle+waiting paths [/] 2024-05-03T18:29:52.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:29:51 smithi150 ceph-mon[32492]: scrub summary: idle 2024-05-03T18:29:53.206 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:29:52 smithi033 ceph-mon[28330]: pgmap v83: 129 pgs: 129 active+clean; 630 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 MiB/s wr, 10 op/s 2024-05-03T18:29:53.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:29:52 smithi099 ceph-mon[31658]: pgmap v83: 129 pgs: 129 active+clean; 630 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 MiB/s wr, 10 op/s 2024-05-03T18:29:53.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:29:52 smithi150 ceph-mon[32492]: pgmap v83: 129 pgs: 129 active+clean; 630 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 MiB/s wr, 10 op/s 2024-05-03T18:29:53.405 INFO:teuthology.orchestra.run.smithi033.stderr:2600000 of 3200000 tuples (81%) done (elapsed 90.51 s, remaining 20.89 s) 2024-05-03T18:29:54.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.582+0000 7f4792c00700 1 -- 172.21.15.33:0/3670301815 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f478c10d9c0 msgr2=0x7f478c10fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:54.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.582+0000 7f4792c00700 1 --2- 172.21.15.33:0/3670301815 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f478c10d9c0 0x7f478c10fe80 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f478400b0a0 tx=0x7f478400b3b0 comp rx=0 tx=0).stop 2024-05-03T18:29:54.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.582+0000 7f4792c00700 1 -- 172.21.15.33:0/3670301815 shutdown_connections 2024-05-03T18:29:54.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.582+0000 7f4792c00700 1 --2- 172.21.15.33:0/3670301815 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f478c10d9c0 0x7f478c10fe80 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:54.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.582+0000 7f4792c00700 1 --2- 172.21.15.33:0/3670301815 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f478c104590 0x7f478c104970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:54.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.582+0000 7f4792c00700 1 --2- 172.21.15.33:0/3670301815 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f478c104eb0 0x7f478c10d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:54.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.582+0000 7f4792c00700 1 -- 172.21.15.33:0/3670301815 >> 172.21.15.33:0/3670301815 conn(0x7f478c0fddc0 msgr2=0x7f478c1001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:29:54.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.583+0000 7f4792c00700 1 -- 172.21.15.33:0/3670301815 shutdown_connections 2024-05-03T18:29:54.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.584+0000 7f4792c00700 1 -- 172.21.15.33:0/3670301815 wait complete. 2024-05-03T18:29:54.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.586+0000 7f4792c00700 1 Processor -- start 2024-05-03T18:29:54.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.586+0000 7f4792c00700 1 -- start start 2024-05-03T18:29:54.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.586+0000 7f4792c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f478c104590 0x7f478c1a1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:54.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.586+0000 7f4792c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f478c104eb0 0x7f478c1a1d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:54.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.586+0000 7f4792c00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f478c10d9c0 0x7f478c19ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:54.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.586+0000 7f4792c00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f478c1134c0 con 0x7f478c10d9c0 2024-05-03T18:29:54.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.586+0000 7f4792c00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f478c113340 con 0x7f478c104eb0 2024-05-03T18:29:54.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.586+0000 7f4792c00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f478c113640 con 0x7f478c104590 2024-05-03T18:29:54.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.587+0000 7f4792200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f478c10d9c0 0x7f478c19ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:54.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.587+0000 7f4792200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f478c10d9c0 0x7f478c19ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52176/0 (socket says 172.21.15.33:52176) 2024-05-03T18:29:54.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.587+0000 7f4792200700 1 -- 172.21.15.33:0/3121730306 learned_addr learned my addr 172.21.15.33:0/3121730306 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:29:54.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.587+0000 7f4790e00700 1 --2- 172.21.15.33:0/3121730306 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f478c104eb0 0x7f478c1a1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:54.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.587+0000 7f4791800700 1 --2- 172.21.15.33:0/3121730306 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f478c104590 0x7f478c1a1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:54.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.588+0000 7f4792200700 1 -- 172.21.15.33:0/3121730306 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f478c104590 msgr2=0x7f478c1a1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:54.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.588+0000 7f4792200700 1 --2- 172.21.15.33:0/3121730306 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f478c104590 0x7f478c1a1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:54.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.588+0000 7f4792200700 1 -- 172.21.15.33:0/3121730306 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f478c104eb0 msgr2=0x7f478c1a1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:54.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.588+0000 7f4792200700 1 --2- 172.21.15.33:0/3121730306 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f478c104eb0 0x7f478c1a1d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:54.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.588+0000 7f4792200700 1 -- 172.21.15.33:0/3121730306 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f478c19c160 con 0x7f478c10d9c0 2024-05-03T18:29:54.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.588+0000 7f4790e00700 1 --2- 172.21.15.33:0/3121730306 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f478c104eb0 0x7f478c1a1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:29:54.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.588+0000 7f4792200700 1 --2- 172.21.15.33:0/3121730306 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f478c10d9c0 0x7f478c19ba00 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f478400b070 tx=0x7f4784004d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:29:54.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.588+0000 7f478a000700 1 -- 172.21.15.33:0/3121730306 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4784021070 con 0x7f478c10d9c0 2024-05-03T18:29:54.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.589+0000 7f478a000700 1 -- 172.21.15.33:0/3121730306 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4784007c00 con 0x7f478c10d9c0 2024-05-03T18:29:54.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.589+0000 7f4792c00700 1 -- 172.21.15.33:0/3121730306 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f478c19c3f0 con 0x7f478c10d9c0 2024-05-03T18:29:54.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.589+0000 7f4792c00700 1 -- 172.21.15.33:0/3121730306 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f478c1a8710 con 0x7f478c10d9c0 2024-05-03T18:29:54.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.589+0000 7f478a000700 1 -- 172.21.15.33:0/3121730306 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4784004850 con 0x7f478c10d9c0 2024-05-03T18:29:54.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.594+0000 7f478a000700 1 -- 172.21.15.33:0/3121730306 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f4784003a00 con 0x7f478c10d9c0 2024-05-03T18:29:54.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.596+0000 7f479eaccb80 1 Processor -- start 2024-05-03T18:29:54.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.596+0000 7f479eaccb80 1 -- start start 2024-05-03T18:29:54.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.596+0000 7f479eaccb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ea44b6bd0 0x563ea44b6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:54.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.596+0000 7f479eaccb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563ea44b0f30 0x563ea44b13b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:54.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.596+0000 7f479eaccb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563ea44b18f0 0x563ea44bd990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:54.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.596+0000 7f479eaccb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563ea4403a20 con 0x563ea44b6bd0 2024-05-03T18:29:54.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.596+0000 7f479eaccb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x563ea43a1760 con 0x563ea44b18f0 2024-05-03T18:29:54.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.596+0000 7f479eaccb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x563ea43ff9e0 con 0x563ea44b0f30 2024-05-03T18:29:54.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.596+0000 7f478a000700 1 --2- 172.21.15.33:0/3121730306 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f47740b2b40 0x7f47740b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:54.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.597+0000 7f478a000700 1 -- 172.21.15.33:0/3121730306 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f478400f020 con 0x7f478c10d9c0 2024-05-03T18:29:54.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.597+0000 7f4790e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ea44b6bd0 0x563ea44b6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:54.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.597+0000 7f4790e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ea44b6bd0 0x563ea44b6fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52186/0 (socket says 172.21.15.33:52186) 2024-05-03T18:29:54.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.597+0000 7f4790e00700 1 -- 172.21.15.33:0/2869170637 learned_addr learned my addr 172.21.15.33:0/2869170637 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:29:54.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.597+0000 7f4792200700 1 --2- 172.21.15.33:0/3121730306 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f47740b2b40 0x7f47740b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:54.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.597+0000 7f4790e00700 1 --2- 172.21.15.33:0/2869170637 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563ea44b18f0 0x563ea44bd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:54.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.598+0000 7f4790e00700 1 -- 172.21.15.33:0/2869170637 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563ea44b0f30 msgr2=0x563ea44b13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:29:54.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.598+0000 7f4790e00700 1 --2- 172.21.15.33:0/2869170637 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563ea44b0f30 0x563ea44b13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:54.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.598+0000 7f4790e00700 1 -- 172.21.15.33:0/2869170637 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563ea44b18f0 msgr2=0x563ea44bd990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:29:54.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.598+0000 7f4790e00700 1 --2- 172.21.15.33:0/2869170637 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563ea44b18f0 0x563ea44bd990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:54.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.598+0000 7f4790e00700 1 -- 172.21.15.33:0/2869170637 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ea43508d0 con 0x563ea44b6bd0 2024-05-03T18:29:54.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.598+0000 7f4790e00700 1 --2- 172.21.15.33:0/2869170637 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ea44b6bd0 0x563ea44b6fb0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f477800bfd0 tx=0x7f477800b6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cadff6ee401c8b62 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:29:54.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.598+0000 7f477ea00700 1 -- 172.21.15.33:0/2869170637 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4778019030 con 0x563ea44b6bd0 2024-05-03T18:29:54.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.598+0000 7f4792200700 1 --2- 172.21.15.33:0/3121730306 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f47740b2b40 0x7f47740b5000 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f47840a6d60 tx=0x7f47840099e0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:29:54.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.598+0000 7f477ea00700 1 -- 172.21.15.33:0/2869170637 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4778013e00 con 0x563ea44b6bd0 2024-05-03T18:29:54.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.598+0000 7f477ea00700 1 -- 172.21.15.33:0/2869170637 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4778003d90 con 0x563ea44b6bd0 2024-05-03T18:29:54.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.599+0000 7f479eaccb80 1 -- 172.21.15.33:0/2869170637 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ea44b6bd0 msgr2=0x563ea44b6fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:54.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.599+0000 7f479eaccb80 1 --2- 172.21.15.33:0/2869170637 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ea44b6bd0 0x563ea44b6fb0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f477800bfd0 tx=0x7f477800b6a0 comp rx=0 tx=0).stop 2024-05-03T18:29:54.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.600+0000 7f479eaccb80 1 -- 172.21.15.33:0/2869170637 shutdown_connections 2024-05-03T18:29:54.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.600+0000 7f479eaccb80 1 --2- 172.21.15.33:0/2869170637 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563ea44b18f0 0x563ea44bd990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:54.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.600+0000 7f479eaccb80 1 --2- 172.21.15.33:0/2869170637 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ea44b6bd0 0x563ea44b6fb0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:54.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.600+0000 7f479eaccb80 1 --2- 172.21.15.33:0/2869170637 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563ea44b0f30 0x563ea44b13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:54.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.600+0000 7f479eaccb80 1 -- 172.21.15.33:0/2869170637 >> 172.21.15.33:0/2869170637 conn(0x563ea44296d0 msgr2=0x563ea44abbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:29:54.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.600+0000 7f479eaccb80 1 -- 172.21.15.33:0/2869170637 shutdown_connections 2024-05-03T18:29:54.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.600+0000 7f479eaccb80 1 -- 172.21.15.33:0/2869170637 wait complete. 2024-05-03T18:29:54.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.601+0000 7f479eaccb80 1 -- start start 2024-05-03T18:29:54.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.601+0000 7f479eaccb80 1 Processor -- start 2024-05-03T18:29:54.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.602+0000 7f479eaccb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ea44b0f30 0x563ea44bd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:54.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.602+0000 7f479eaccb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563ea44b18f0 0x563ea44ba930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:54.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.602+0000 7f479eaccb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563ea44b6bd0 0x563ea44bae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:54.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.602+0000 7f479eaccb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563ea44bdf30 con 0x563ea44b0f30 2024-05-03T18:29:54.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.602+0000 7f479eaccb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x563ea4369b20 con 0x563ea44b18f0 2024-05-03T18:29:54.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.602+0000 7f479eaccb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x563ea44be0b0 con 0x563ea44b6bd0 2024-05-03T18:29:54.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.602+0000 7f479eaccb80 10 client.0 _ll_register_callbacks cb 0x563ea43aac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:29:54.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.602+0000 7f4790e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ea44b0f30 0x563ea44bd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:54.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.602+0000 7f4790e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ea44b0f30 0x563ea44bd540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52196/0 (socket says 172.21.15.33:52196) 2024-05-03T18:29:54.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.602+0000 7f4790e00700 1 -- 172.21.15.33:0/625465739 learned_addr learned my addr 172.21.15.33:0/625465739 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:29:54.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.603+0000 7f4790e00700 1 -- 172.21.15.33:0/625465739 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563ea44b6bd0 msgr2=0x563ea44bae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:29:54.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.603+0000 7f4791800700 1 --2- 172.21.15.33:0/625465739 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563ea44b18f0 0x563ea44ba930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:54.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.603+0000 7f4790e00700 1 --2- 172.21.15.33:0/625465739 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563ea44b6bd0 0x563ea44bae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:54.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.603+0000 7f4790e00700 1 -- 172.21.15.33:0/625465739 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563ea44b18f0 msgr2=0x563ea44ba930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:29:54.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.603+0000 7f4790e00700 1 --2- 172.21.15.33:0/625465739 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563ea44b18f0 0x563ea44ba930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:54.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.603+0000 7f4790e00700 1 -- 172.21.15.33:0/625465739 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ea43c9420 con 0x563ea44b0f30 2024-05-03T18:29:54.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.603+0000 7f4791800700 1 --2- 172.21.15.33:0/625465739 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563ea44b18f0 0x563ea44ba930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:29:54.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.603+0000 7f4790e00700 1 --2- 172.21.15.33:0/625465739 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ea44b0f30 0x563ea44bd540 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f4778006010 tx=0x7f4778003750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b854072c47fe889 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:29:54.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.604+0000 7f4773400700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:29:54.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.604+0000 7f4773400700 1 -- 172.21.15.33:0/625465739 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4778019040 con 0x563ea44b0f30 2024-05-03T18:29:54.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.604+0000 7f4773400700 1 -- 172.21.15.33:0/625465739 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f47780055d0 con 0x563ea44b0f30 2024-05-03T18:29:54.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.604+0000 7f479eaccb80 1 -- 172.21.15.33:0/625465739 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4778013e00 con 0x563ea44b0f30 2024-05-03T18:29:54.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.605+0000 7f4773400700 1 -- 172.21.15.33:0/625465739 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f4778004750 con 0x563ea44b0f30 2024-05-03T18:29:54.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.605+0000 7f479eaccb80 10 client.15696 fetch_fsmap learned FSMap version 21 2024-05-03T18:29:54.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.605+0000 7f479eaccb80 1 -- 172.21.15.33:0/625465739 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x563ea44c86f0 con 0x563ea44b0f30 2024-05-03T18:29:54.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.605+0000 7f4773400700 1 -- 172.21.15.33:0/625465739 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4778013e00 con 0x563ea44b0f30 2024-05-03T18:29:54.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.605+0000 7f4773400700 1 -- 172.21.15.33:0/625465739 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f477801e8f0 con 0x563ea44b0f30 2024-05-03T18:29:54.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.605+0000 7f479eaccb80 10 client.15696 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:29:54.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.605+0000 7f479eaccb80 10 client.15696 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:29:54.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.606+0000 7f479eaccb80 20 client.15696 populate_metadata read hostname 'smithi033' 2024-05-03T18:29:54.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.606+0000 7f479eaccb80 1 --2- 172.21.15.33:0/625465739 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x563ea44cacd0 0x563ea44cd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:54.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.606+0000 7f479eaccb80 4 client.15696 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:29:54.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.606+0000 7f479eaccb80 1 -- 172.21.15.33:0/625465739 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563ea447dff0 con 0x563ea44cacd0 2024-05-03T18:29:54.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.606+0000 7f4791800700 1 --2- 172.21.15.33:0/625465739 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x563ea44cacd0 0x563ea44cd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:54.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.608+0000 7f4791800700 1 --2- 172.21.15.33:0/625465739 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x563ea44cacd0 0x563ea44cd0c0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=306948095e4c6434 server_cookie=fb5368a954aed16e in_seq=0 out_seq=0 2024-05-03T18:29:54.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.608+0000 7f4773400700 10 client.15696 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:29:54.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.609+0000 7f4773400700 1 -- 172.21.15.33:0/625465739 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563ea447dff0 con 0x563ea44cacd0 2024-05-03T18:29:54.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.609+0000 7f4773400700 10 client.15696 handle_command_reply: tid=0 2024-05-03T18:29:54.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.610+0000 7f479eaccb80 1 client.15696 shutdown 2024-05-03T18:29:54.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.610+0000 7f479eaccb80 1 -- 172.21.15.33:0/625465739 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ea44b0f30 msgr2=0x563ea44bd540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:54.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.610+0000 7f479eaccb80 1 --2- 172.21.15.33:0/625465739 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ea44b0f30 0x563ea44bd540 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f4778006010 tx=0x7f4778003750 comp rx=0 tx=0).stop 2024-05-03T18:29:54.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.610+0000 7f479eaccb80 1 -- 172.21.15.33:0/625465739 shutdown_connections 2024-05-03T18:29:54.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.610+0000 7f479eaccb80 1 --2- 172.21.15.33:0/625465739 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x563ea44cacd0 0x563ea44cd0c0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:54.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.610+0000 7f479eaccb80 1 --2- 172.21.15.33:0/625465739 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563ea44b6bd0 0x563ea44bae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:54.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.610+0000 7f479eaccb80 1 --2- 172.21.15.33:0/625465739 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ea44b0f30 0x563ea44bd540 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:54.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.610+0000 7f479eaccb80 1 --2- 172.21.15.33:0/625465739 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563ea44b18f0 0x563ea44ba930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:54.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.610+0000 7f479eaccb80 1 -- 172.21.15.33:0/625465739 >> 172.21.15.33:0/625465739 conn(0x563ea44296d0 msgr2=0x563ea44aff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:29:54.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.610+0000 7f4773400700 0 client.15696 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:29:54.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.610+0000 7f479eaccb80 1 -- 172.21.15.33:0/625465739 shutdown_connections 2024-05-03T18:29:54.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.610+0000 7f479eaccb80 1 -- 172.21.15.33:0/625465739 wait complete. 2024-05-03T18:29:54.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.611+0000 7f479eaccb80 20 client.15696 trim_cache size 0 max 16384 2024-05-03T18:29:54.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.644+0000 7f479eaccb80 1 Processor -- start 2024-05-03T18:29:54.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.644+0000 7f479eaccb80 1 -- start start 2024-05-03T18:29:54.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.644+0000 7f479eaccb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ea44b0f30 0x563ea44c6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:54.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.644+0000 7f479eaccb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563ea44b18f0 0x563ea44baea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:54.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.644+0000 7f479eaccb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563ea44b6bd0 0x563ea44d2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:54.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.644+0000 7f479eaccb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f476c00ccb0 con 0x563ea44b0f30 2024-05-03T18:29:54.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.644+0000 7f479eaccb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f476c00f530 con 0x563ea44b18f0 2024-05-03T18:29:54.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.644+0000 7f479eaccb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f476c00d320 con 0x563ea44b6bd0 2024-05-03T18:29:54.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.645+0000 7f4790e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ea44b0f30 0x563ea44c6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:54.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.645+0000 7f4790e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ea44b0f30 0x563ea44c6fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52208/0 (socket says 172.21.15.33:52208) 2024-05-03T18:29:54.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.645+0000 7f4790e00700 1 -- 172.21.15.33:0/1514872826 learned_addr learned my addr 172.21.15.33:0/1514872826 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:29:54.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.645+0000 7f4791800700 1 --2- 172.21.15.33:0/1514872826 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563ea44b18f0 0x563ea44baea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:54.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.645+0000 7f4790e00700 1 -- 172.21.15.33:0/1514872826 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563ea44b6bd0 msgr2=0x563ea44d2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:29:54.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.645+0000 7f4790e00700 1 --2- 172.21.15.33:0/1514872826 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563ea44b6bd0 0x563ea44d2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:54.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.645+0000 7f4790e00700 1 -- 172.21.15.33:0/1514872826 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563ea44b18f0 msgr2=0x563ea44baea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:29:54.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.645+0000 7f4790e00700 1 --2- 172.21.15.33:0/1514872826 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563ea44b18f0 0x563ea44baea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:54.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.645+0000 7f4790e00700 1 -- 172.21.15.33:0/1514872826 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ea44a5f40 con 0x563ea44b0f30 2024-05-03T18:29:54.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.645+0000 7f4791800700 1 --2- 172.21.15.33:0/1514872826 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563ea44b18f0 0x563ea44baea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:29:54.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.645+0000 7f4790e00700 1 --2- 172.21.15.33:0/1514872826 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ea44b0f30 0x563ea44c6fe0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f4778005950 tx=0x7f4778013530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af56e92ca873ed68 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:29:54.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.646+0000 7f477e000700 1 -- 172.21.15.33:0/1514872826 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f477803b070 con 0x563ea44b0f30 2024-05-03T18:29:54.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.646+0000 7f477e000700 1 -- 172.21.15.33:0/1514872826 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4778022070 con 0x563ea44b0f30 2024-05-03T18:29:54.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.646+0000 7f477e000700 1 -- 172.21.15.33:0/1514872826 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4778010e60 con 0x563ea44b0f30 2024-05-03T18:29:54.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.647+0000 7f479eaccb80 1 -- 172.21.15.33:0/1514872826 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ea44b0f30 msgr2=0x563ea44c6fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:54.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.647+0000 7f479eaccb80 1 --2- 172.21.15.33:0/1514872826 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ea44b0f30 0x563ea44c6fe0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f4778005950 tx=0x7f4778013530 comp rx=0 tx=0).stop 2024-05-03T18:29:54.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.647+0000 7f479eaccb80 1 -- 172.21.15.33:0/1514872826 shutdown_connections 2024-05-03T18:29:54.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.647+0000 7f479eaccb80 1 --2- 172.21.15.33:0/1514872826 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563ea44b6bd0 0x563ea44d2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:54.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.647+0000 7f479eaccb80 1 --2- 172.21.15.33:0/1514872826 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ea44b0f30 0x563ea44c6fe0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:54.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.647+0000 7f479eaccb80 1 --2- 172.21.15.33:0/1514872826 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563ea44b18f0 0x563ea44baea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:54.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.647+0000 7f479eaccb80 1 -- 172.21.15.33:0/1514872826 >> 172.21.15.33:0/1514872826 conn(0x563ea44296d0 msgr2=0x563ea44b9220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:29:54.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.647+0000 7f479eaccb80 1 -- 172.21.15.33:0/1514872826 shutdown_connections 2024-05-03T18:29:54.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.647+0000 7f479eaccb80 1 -- 172.21.15.33:0/1514872826 wait complete. 2024-05-03T18:29:54.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.648+0000 7f479eaccb80 1 -- start start 2024-05-03T18:29:54.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.648+0000 7f479eaccb80 1 Processor -- start 2024-05-03T18:29:54.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.648+0000 7f479eaccb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563ea44b0f30 0x563ea44d1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:54.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.649+0000 7f479eaccb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563ea44b18f0 0x563ea44d22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:54.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.649+0000 7f479eaccb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ea44b6bd0 0x563ea44d27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:54.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.649+0000 7f479eaccb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f476c00d9f0 con 0x563ea44b6bd0 2024-05-03T18:29:54.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.649+0000 7f479eaccb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f476c00ee60 con 0x563ea44b18f0 2024-05-03T18:29:54.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.649+0000 7f479eaccb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f476c00e0c0 con 0x563ea44b0f30 2024-05-03T18:29:54.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.649+0000 7f479eaccb80 10 client.0 _ll_register_callbacks cb 0x563ea43aac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:29:54.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.649+0000 7f4790e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ea44b6bd0 0x563ea44d27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:54.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.649+0000 7f4790e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ea44b6bd0 0x563ea44d27e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52224/0 (socket says 172.21.15.33:52224) 2024-05-03T18:29:54.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.649+0000 7f4790e00700 1 -- 172.21.15.33:0/1597471385 learned_addr learned my addr 172.21.15.33:0/1597471385 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:29:54.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.649+0000 7f4790e00700 1 --2- 172.21.15.33:0/1597471385 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563ea44b0f30 0x563ea44d1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:54.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.650+0000 7f4791800700 1 --2- 172.21.15.33:0/1597471385 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563ea44b18f0 0x563ea44d22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:54.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.650+0000 7f4790e00700 1 -- 172.21.15.33:0/1597471385 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563ea44b0f30 msgr2=0x563ea44d1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:29:54.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.650+0000 7f4790e00700 1 --2- 172.21.15.33:0/1597471385 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563ea44b0f30 0x563ea44d1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:54.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.650+0000 7f4790e00700 1 -- 172.21.15.33:0/1597471385 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563ea44b18f0 msgr2=0x563ea44d22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:29:54.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.650+0000 7f4790e00700 1 --2- 172.21.15.33:0/1597471385 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563ea44b18f0 0x563ea44d22a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:54.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.650+0000 7f4790e00700 1 -- 172.21.15.33:0/1597471385 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ea4534240 con 0x563ea44b6bd0 2024-05-03T18:29:54.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.650+0000 7f4791800700 1 --2- 172.21.15.33:0/1597471385 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563ea44b18f0 0x563ea44d22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:29:54.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.650+0000 7f4790e00700 1 --2- 172.21.15.33:0/1597471385 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ea44b6bd0 0x563ea44d27e0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f4778010fc0 tx=0x7f4778003750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3b3f39607acf0e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:29:54.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.650+0000 7f4773400700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:29:54.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.651+0000 7f4773400700 1 -- 172.21.15.33:0/1597471385 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4778019030 con 0x563ea44b6bd0 2024-05-03T18:29:54.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.651+0000 7f479eaccb80 1 -- 172.21.15.33:0/1597471385 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4778022070 con 0x563ea44b6bd0 2024-05-03T18:29:54.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.651+0000 7f4773400700 1 -- 172.21.15.33:0/1597471385 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f47780055d0 con 0x563ea44b6bd0 2024-05-03T18:29:54.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.651+0000 7f4773400700 1 -- 172.21.15.33:0/1597471385 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f477803b070 con 0x563ea44b6bd0 2024-05-03T18:29:54.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.651+0000 7f479eaccb80 10 client.15702 fetch_fsmap learned FSMap version 21 2024-05-03T18:29:54.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.651+0000 7f479eaccb80 1 -- 172.21.15.33:0/1597471385 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x563ea4535370 con 0x563ea44b6bd0 2024-05-03T18:29:54.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.652+0000 7f4773400700 1 -- 172.21.15.33:0/1597471385 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4778022070 con 0x563ea44b6bd0 2024-05-03T18:29:54.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.652+0000 7f4773400700 1 -- 172.21.15.33:0/1597471385 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f4778004c60 con 0x563ea44b6bd0 2024-05-03T18:29:54.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.652+0000 7f479eaccb80 10 client.15702 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:29:54.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.652+0000 7f479eaccb80 10 client.15702 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:29:54.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.652+0000 7f479eaccb80 20 client.15702 populate_metadata read hostname 'smithi033' 2024-05-03T18:29:54.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.652+0000 7f479eaccb80 1 --2- 172.21.15.33:0/1597471385 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x563ea4535760 0x563ea4537b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:54.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.652+0000 7f479eaccb80 4 client.15702 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-03T18:29:54.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.652+0000 7f479eaccb80 1 -- 172.21.15.33:0/1597471385 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563ea4411400 con 0x563ea4535760 2024-05-03T18:29:54.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.654+0000 7f4791800700 1 --2- 172.21.15.33:0/1597471385 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x563ea4535760 0x563ea4537b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:54.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.656+0000 7f4791800700 1 --2- 172.21.15.33:0/1597471385 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x563ea4535760 0x563ea4537b80 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59bb0cd027a1c869 server_cookie=4709837ae7094205 in_seq=0 out_seq=0 2024-05-03T18:29:54.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.656+0000 7f4773400700 10 client.15702 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:29:54.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.657+0000 7f4773400700 1 -- 172.21.15.33:0/1597471385 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563ea4411400 con 0x563ea4535760 2024-05-03T18:29:54.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.657+0000 7f4773400700 10 client.15702 handle_command_reply: tid=0 2024-05-03T18:29:54.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.660+0000 7f479eaccb80 1 client.15702 shutdown 2024-05-03T18:29:54.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.661+0000 7f479eaccb80 1 -- 172.21.15.33:0/1597471385 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ea44b6bd0 msgr2=0x563ea44d27e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:54.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.661+0000 7f479eaccb80 1 --2- 172.21.15.33:0/1597471385 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ea44b6bd0 0x563ea44d27e0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f4778010fc0 tx=0x7f4778003750 comp rx=0 tx=0).stop 2024-05-03T18:29:54.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.661+0000 7f479eaccb80 1 -- 172.21.15.33:0/1597471385 shutdown_connections 2024-05-03T18:29:54.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.661+0000 7f479eaccb80 1 --2- 172.21.15.33:0/1597471385 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x563ea4535760 0x563ea4537b80 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:54.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.661+0000 7f479eaccb80 1 --2- 172.21.15.33:0/1597471385 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ea44b6bd0 0x563ea44d27e0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:54.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.661+0000 7f479eaccb80 1 --2- 172.21.15.33:0/1597471385 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563ea44b0f30 0x563ea44d1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:54.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.661+0000 7f479eaccb80 1 --2- 172.21.15.33:0/1597471385 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563ea44b18f0 0x563ea44d22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:54.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.661+0000 7f479eaccb80 1 -- 172.21.15.33:0/1597471385 >> 172.21.15.33:0/1597471385 conn(0x563ea44296d0 msgr2=0x563ea44b5ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:29:54.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.661+0000 7f4773400700 0 client.15702 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:29:54.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.662+0000 7f479eaccb80 1 -- 172.21.15.33:0/1597471385 shutdown_connections 2024-05-03T18:29:54.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.662+0000 7f479eaccb80 1 -- 172.21.15.33:0/1597471385 wait complete. 2024-05-03T18:29:54.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.662+0000 7f479eaccb80 20 client.15702 trim_cache size 0 max 16384 2024-05-03T18:29:54.663 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-03T18:29:54.664 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-05-03T18:29:54.664 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-05-03T18:29:54.664 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-03T18:29:54.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.663+0000 7f4792c00700 1 -- 172.21.15.33:0/3121730306 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f47740b2b40 msgr2=0x7f47740b5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:54.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.663+0000 7f4792c00700 1 --2- 172.21.15.33:0/3121730306 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f47740b2b40 0x7f47740b5000 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f47840a6d60 tx=0x7f47840099e0 comp rx=0 tx=0).stop 2024-05-03T18:29:54.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.663+0000 7f4792c00700 1 -- 172.21.15.33:0/3121730306 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f478c10d9c0 msgr2=0x7f478c19ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:54.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.663+0000 7f4792c00700 1 --2- 172.21.15.33:0/3121730306 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f478c10d9c0 0x7f478c19ba00 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f478400b070 tx=0x7f4784004d30 comp rx=0 tx=0).stop 2024-05-03T18:29:54.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.663+0000 7f4792c00700 1 -- 172.21.15.33:0/3121730306 shutdown_connections 2024-05-03T18:29:54.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.663+0000 7f4792c00700 1 --2- 172.21.15.33:0/3121730306 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f478c10d9c0 0x7f478c19ba00 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:54.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.663+0000 7f4792c00700 1 --2- 172.21.15.33:0/3121730306 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f478c104590 0x7f478c1a1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:54.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.663+0000 7f4792c00700 1 --2- 172.21.15.33:0/3121730306 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f47740b2b40 0x7f47740b5000 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:54.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.663+0000 7f4792c00700 1 --2- 172.21.15.33:0/3121730306 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f478c104eb0 0x7f478c1a1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:54.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.663+0000 7f4792c00700 1 -- 172.21.15.33:0/3121730306 >> 172.21.15.33:0/3121730306 conn(0x7f478c0fddc0 msgr2=0x7f478c0fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:29:54.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.664+0000 7f4792c00700 1 -- 172.21.15.33:0/3121730306 shutdown_connections 2024-05-03T18:29:54.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:54.664+0000 7f4792c00700 1 -- 172.21.15.33:0/3121730306 wait complete. 2024-05-03T18:29:55.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:29:54 smithi033 ceph-mon[28330]: pgmap v84: 129 pgs: 129 active+clean; 666 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 17 MiB/s wr, 15 op/s 2024-05-03T18:29:55.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:29:54 smithi099 ceph-mon[31658]: pgmap v84: 129 pgs: 129 active+clean; 666 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 17 MiB/s wr, 15 op/s 2024-05-03T18:29:55.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:29:54 smithi150 ceph-mon[32492]: pgmap v84: 129 pgs: 129 active+clean; 666 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 17 MiB/s wr, 15 op/s 2024-05-03T18:29:55.667 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-03T18:29:55.667 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T18:29:56.164 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:29:57.089 INFO:teuthology.orchestra.run.smithi033.stderr:2700000 of 3200000 tuples (84%) done (elapsed 94.19 s, remaining 17.44 s) 2024-05-03T18:29:57.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:29:56 smithi033 ceph-mon[28330]: pgmap v85: 129 pgs: 129 active+clean; 666 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 MiB/s wr, 10 op/s 2024-05-03T18:29:57.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:29:56 smithi099 ceph-mon[31658]: pgmap v85: 129 pgs: 129 active+clean; 666 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 MiB/s wr, 10 op/s 2024-05-03T18:29:57.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:29:56 smithi150 ceph-mon[32492]: pgmap v85: 129 pgs: 129 active+clean; 666 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 MiB/s wr, 10 op/s 2024-05-03T18:29:58.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:58.654+0000 7f34bbe00700 1 -- 172.21.15.33:0/922413722 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34bc10b660 msgr2=0x7f34bc10ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:58.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:58.654+0000 7f34bbe00700 1 --2- 172.21.15.33:0/922413722 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34bc10b660 0x7f34bc10ba40 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f34ac009990 tx=0x7f34ac009ca0 comp rx=0 tx=0).stop 2024-05-03T18:29:58.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:58.654+0000 7f34bbe00700 1 -- 172.21.15.33:0/922413722 shutdown_connections 2024-05-03T18:29:58.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:58.654+0000 7f34bbe00700 1 --2- 172.21.15.33:0/922413722 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34bc105f90 0x7f34bc1124a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:58.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:58.654+0000 7f34bbe00700 1 --2- 172.21.15.33:0/922413722 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34bc10b660 0x7f34bc10ba40 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:58.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:58.654+0000 7f34bbe00700 1 --2- 172.21.15.33:0/922413722 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34bc1055d0 0x7f34bc105a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:58.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:58.654+0000 7f34bbe00700 1 -- 172.21.15.33:0/922413722 >> 172.21.15.33:0/922413722 conn(0x7f34bc1010c0 msgr2=0x7f34bc1034e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:29:58.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:58.654+0000 7f34bbe00700 1 -- 172.21.15.33:0/922413722 shutdown_connections 2024-05-03T18:29:58.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:58.654+0000 7f34bbe00700 1 -- 172.21.15.33:0/922413722 wait complete. 2024-05-03T18:29:58.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:58.656+0000 7f34bbe00700 1 Processor -- start 2024-05-03T18:29:58.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:58.656+0000 7f34bbe00700 1 -- start start 2024-05-03T18:29:58.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:58.656+0000 7f34bbe00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34bc1055d0 0x7f34bc112360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:58.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:58.656+0000 7f34bbe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34bc105f90 0x7f34bc10f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:58.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:58.656+0000 7f34bbe00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34bc10b660 0x7f34bc10f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:58.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:58.656+0000 7f34bbe00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f34bc115680 con 0x7f34bc10b660 2024-05-03T18:29:58.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:58.656+0000 7f34bbe00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f34bc115500 con 0x7f34bc105f90 2024-05-03T18:29:58.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:58.656+0000 7f34bbe00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f34bc115800 con 0x7f34bc1055d0 2024-05-03T18:29:58.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:58.657+0000 7f34bb400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34bc10b660 0x7f34bc10f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:58.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:58.657+0000 7f34bb400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34bc10b660 0x7f34bc10f9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52612/0 (socket says 172.21.15.33:52612) 2024-05-03T18:29:58.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:58.657+0000 7f34bb400700 1 -- 172.21.15.33:0/113551620 learned_addr learned my addr 172.21.15.33:0/113551620 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:29:58.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:58.657+0000 7f34baa00700 1 --2- 172.21.15.33:0/113551620 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34bc1055d0 0x7f34bc112360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:58.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:58.657+0000 7f34ba000700 1 --2- 172.21.15.33:0/113551620 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34bc105f90 0x7f34bc10f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:58.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:58.658+0000 7f34bb400700 1 -- 172.21.15.33:0/113551620 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34bc1055d0 msgr2=0x7f34bc112360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:58.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:58.658+0000 7f34bb400700 1 --2- 172.21.15.33:0/113551620 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34bc1055d0 0x7f34bc112360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:58.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:58.658+0000 7f34bb400700 1 -- 172.21.15.33:0/113551620 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34bc105f90 msgr2=0x7f34bc10f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:58.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:58.658+0000 7f34bb400700 1 --2- 172.21.15.33:0/113551620 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34bc105f90 0x7f34bc10f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:58.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:58.658+0000 7f34bb400700 1 -- 172.21.15.33:0/113551620 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34bc1a1b60 con 0x7f34bc10b660 2024-05-03T18:29:58.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:58.658+0000 7f34bb400700 1 --2- 172.21.15.33:0/113551620 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34bc10b660 0x7f34bc10f9e0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f34b4007ee0 tx=0x7f34b400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:29:58.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:58.658+0000 7f34b3400700 1 -- 172.21.15.33:0/113551620 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b4004bb0 con 0x7f34bc10b660 2024-05-03T18:29:58.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:58.658+0000 7f34bbe00700 1 -- 172.21.15.33:0/113551620 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34bc1a1e50 con 0x7f34bc10b660 2024-05-03T18:29:58.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:58.658+0000 7f34b3400700 1 -- 172.21.15.33:0/113551620 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f34b4004d50 con 0x7f34bc10b660 2024-05-03T18:29:58.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:58.658+0000 7f34b3400700 1 -- 172.21.15.33:0/113551620 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b40058b0 con 0x7f34bc10b660 2024-05-03T18:29:58.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:58.658+0000 7f34bbe00700 1 -- 172.21.15.33:0/113551620 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34bc1a2390 con 0x7f34bc10b660 2024-05-03T18:29:58.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:58.660+0000 7f34bbe00700 1 -- 172.21.15.33:0/113551620 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34bc106d10 con 0x7f34bc10b660 2024-05-03T18:29:58.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:58.662+0000 7f34b3400700 1 -- 172.21.15.33:0/113551620 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f34b401a020 con 0x7f34bc10b660 2024-05-03T18:29:58.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:58.664+0000 7f34b3400700 1 --2- 172.21.15.33:0/113551620 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f34a00b29f0 0x7f34a00b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:29:58.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:58.664+0000 7f34b3400700 1 -- 172.21.15.33:0/113551620 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f34b40e20a0 con 0x7f34bc10b660 2024-05-03T18:29:58.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:58.664+0000 7f34baa00700 1 --2- 172.21.15.33:0/113551620 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f34a00b29f0 0x7f34a00b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:29:58.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:58.665+0000 7f34baa00700 1 --2- 172.21.15.33:0/113551620 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f34a00b29f0 0x7f34a00b4eb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f34ac00b5c0 tx=0x7f34ac005960 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:29:58.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:58.669+0000 7f34b3400700 1 -- 172.21.15.33:0/113551620 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f34b4010070 con 0x7f34bc10b660 2024-05-03T18:29:58.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:58.976+0000 7f34bbe00700 1 -- 172.21.15.33:0/113551620 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f34bc110940 con 0x7f34bc10b660 2024-05-03T18:29:58.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:58.979+0000 7f34b3400700 1 -- 172.21.15.33:0/113551620 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11801 (secure 0 0 0) 0x7f34b40a9690 con 0x7f34bc10b660 2024-05-03T18:29:58.979 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:29:58.981 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1327393503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1327393503},{"type":"v1","addr":"172.21.15.33:6835","nonce":1327393503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/2461472948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":2461472948},{"type":"v1","addr":"172.21.15.33:6837","nonce":2461472948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14559,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/442269281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":442269281},{"type":"v1","addr":"172.21.15.33:6839","nonce":442269281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14571,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/645571756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":645571756},{"type":"v1","addr":"172.21.15.33:6841","nonce":645571756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/203598909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":203598909},{"type":"v1","addr":"172.21.15.150:6833","nonce":203598909}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/4029842493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":4029842493},{"type":"v1","addr":"172.21.15.150:6835","nonce":4029842493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3463664423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3463664423},{"type":"v1","addr":"172.21.15.150:6837","nonce":3463664423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24481,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6839/4051648018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":4051648018},{"type":"v1","addr":"172.21.15.150:6839","nonce":4051648018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34418,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1865105479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1865105479},{"type":"v1","addr":"172.21.15.99:6833","nonce":1865105479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34445,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/1206393914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1206393914},{"type":"v1","addr":"172.21.15.99:6835","nonce":1206393914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-03T18:22:28.837091+0000","modified":"2024-05-03T18:23:49.758398+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34508},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34472":{"gid":34472,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6837/3360970342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3360970342},{"type":"v1","addr":"172.21.15.99:6837","nonce":3360970342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34508":{"gid":34508,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/919907308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":919907308},{"type":"v1","addr":"172.21.15.99:6839","nonce":919907308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34508,"qdb_cluster":[34508]},"id":1}]} 2024-05-03T18:29:58.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:58.984+0000 7f34bbe00700 1 -- 172.21.15.33:0/113551620 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f34a00b29f0 msgr2=0x7f34a00b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:58.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:58.984+0000 7f34bbe00700 1 --2- 172.21.15.33:0/113551620 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f34a00b29f0 0x7f34a00b4eb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f34ac00b5c0 tx=0x7f34ac005960 comp rx=0 tx=0).stop 2024-05-03T18:29:58.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:58.984+0000 7f34bbe00700 1 -- 172.21.15.33:0/113551620 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34bc10b660 msgr2=0x7f34bc10f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:29:58.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:58.984+0000 7f34bbe00700 1 --2- 172.21.15.33:0/113551620 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34bc10b660 0x7f34bc10f9e0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f34b4007ee0 tx=0x7f34b400d3b0 comp rx=0 tx=0).stop 2024-05-03T18:29:58.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:58.985+0000 7f34bbe00700 1 -- 172.21.15.33:0/113551620 shutdown_connections 2024-05-03T18:29:58.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:58.985+0000 7f34bbe00700 1 --2- 172.21.15.33:0/113551620 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34bc10b660 0x7f34bc10f9e0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:58.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:58.985+0000 7f34bbe00700 1 --2- 172.21.15.33:0/113551620 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34bc1055d0 0x7f34bc112360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:58.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:58.985+0000 7f34bbe00700 1 --2- 172.21.15.33:0/113551620 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f34a00b29f0 0x7f34a00b4eb0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:58.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:58.985+0000 7f34bbe00700 1 --2- 172.21.15.33:0/113551620 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34bc105f90 0x7f34bc10f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:29:58.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:58.985+0000 7f34bbe00700 1 -- 172.21.15.33:0/113551620 >> 172.21.15.33:0/113551620 conn(0x7f34bc1010c0 msgr2=0x7f34bc1028d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:29:58.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:58.985+0000 7f34bbe00700 1 -- 172.21.15.33:0/113551620 shutdown_connections 2024-05-03T18:29:58.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:29:58.985+0000 7f34bbe00700 1 -- 172.21.15.33:0/113551620 wait complete. 2024-05-03T18:29:58.987 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 21 2024-05-03T18:29:59.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:29:58 smithi033 ceph-mon[28330]: pgmap v86: 129 pgs: 129 active+clean; 702 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 MiB/s wr, 15 op/s 2024-05-03T18:29:59.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:29:58 smithi099 ceph-mon[31658]: pgmap v86: 129 pgs: 129 active+clean; 702 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 MiB/s wr, 15 op/s 2024-05-03T18:29:59.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:29:58 smithi150 ceph-mon[32492]: pgmap v86: 129 pgs: 129 active+clean; 702 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 MiB/s wr, 15 op/s 2024-05-03T18:29:59.978 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-03T18:30:00.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:29:59 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/113551620' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:30:00.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:29:59 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/113551620' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:30:00.381 INFO:teuthology.orchestra.run.smithi033.stderr:2800000 of 3200000 tuples (87%) done (elapsed 97.49 s, remaining 13.93 s) 2024-05-03T18:30:00.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:29:59 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/113551620' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:30:00.517 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:30:01.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:30:00 smithi033 ceph-mon[28330]: pgmap v87: 129 pgs: 129 active+clean; 702 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 MiB/s wr, 10 op/s 2024-05-03T18:30:01.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:30:00 smithi033 ceph-mon[28330]: overall HEALTH_OK 2024-05-03T18:30:01.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:30:00 smithi099 ceph-mon[31658]: pgmap v87: 129 pgs: 129 active+clean; 702 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 MiB/s wr, 10 op/s 2024-05-03T18:30:01.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:30:00 smithi099 ceph-mon[31658]: overall HEALTH_OK 2024-05-03T18:30:01.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:30:00 smithi150 ceph-mon[32492]: pgmap v87: 129 pgs: 129 active+clean; 702 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 MiB/s wr, 10 op/s 2024-05-03T18:30:01.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:30:00 smithi150 ceph-mon[32492]: overall HEALTH_OK 2024-05-03T18:30:02.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:30:02 smithi099 ceph-mon[31658]: pgmap v88: 129 pgs: 129 active+clean; 702 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 MiB/s wr, 10 op/s 2024-05-03T18:30:02.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:30:02 smithi150 ceph-mon[32492]: pgmap v88: 129 pgs: 129 active+clean; 702 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 MiB/s wr, 10 op/s 2024-05-03T18:30:02.415 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:30:02 smithi033 ceph-mon[28330]: pgmap v88: 129 pgs: 129 active+clean; 702 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 MiB/s wr, 10 op/s 2024-05-03T18:30:02.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.964+0000 7f6a3fe00700 1 -- 172.21.15.33:0/2289303596 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a38102450 msgr2=0x7f6a38110340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:30:02.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.964+0000 7f6a3fe00700 1 --2- 172.21.15.33:0/2289303596 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a38102450 0x7f6a38110340 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f6a3000b0a0 tx=0x7f6a3000b3b0 comp rx=0 tx=0).stop 2024-05-03T18:30:02.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.964+0000 7f6a3fe00700 1 -- 172.21.15.33:0/2289303596 shutdown_connections 2024-05-03T18:30:02.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.964+0000 7f6a3fe00700 1 --2- 172.21.15.33:0/2289303596 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a38102450 0x7f6a38110340 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:02.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.964+0000 7f6a3fe00700 1 --2- 172.21.15.33:0/2289303596 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6a38109520 0x7f6a38109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:02.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.964+0000 7f6a3fe00700 1 --2- 172.21.15.33:0/2289303596 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a38101a90 0x7f6a38101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:02.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.964+0000 7f6a3fe00700 1 -- 172.21.15.33:0/2289303596 >> 172.21.15.33:0/2289303596 conn(0x7f6a38078a10 msgr2=0x7f6a380ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:30:02.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.964+0000 7f6a3fe00700 1 -- 172.21.15.33:0/2289303596 shutdown_connections 2024-05-03T18:30:02.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.964+0000 7f6a3fe00700 1 -- 172.21.15.33:0/2289303596 wait complete. 2024-05-03T18:30:02.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.965+0000 7f6a3fe00700 1 Processor -- start 2024-05-03T18:30:02.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.965+0000 7f6a3fe00700 1 -- start start 2024-05-03T18:30:02.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.966+0000 7f6a3fe00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a38101a90 0x7f6a381a17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:02.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.966+0000 7f6a3fe00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a38102450 0x7f6a381a1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:02.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.966+0000 7f6a3fe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6a38109520 0x7f6a3819b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:02.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.966+0000 7f6a3fe00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6a38113550 con 0x7f6a38101a90 2024-05-03T18:30:02.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.966+0000 7f6a3fe00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6a381133d0 con 0x7f6a38109520 2024-05-03T18:30:02.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.966+0000 7f6a3fe00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6a381136d0 con 0x7f6a38102450 2024-05-03T18:30:02.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.966+0000 7f6a3d400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a38101a90 0x7f6a381a17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:02.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.966+0000 7f6a3d400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a38101a90 0x7f6a381a17f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52638/0 (socket says 172.21.15.33:52638) 2024-05-03T18:30:02.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.966+0000 7f6a3d400700 1 -- 172.21.15.33:0/2080033175 learned_addr learned my addr 172.21.15.33:0/2080033175 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:30:02.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.967+0000 7f6a3ca00700 1 --2- 172.21.15.33:0/2080033175 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a38102450 0x7f6a381a1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:02.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.967+0000 7f6a3de00700 1 --2- 172.21.15.33:0/2080033175 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6a38109520 0x7f6a3819b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:02.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.967+0000 7f6a3d400700 1 -- 172.21.15.33:0/2080033175 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a38102450 msgr2=0x7f6a381a1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:30:02.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.967+0000 7f6a3d400700 1 --2- 172.21.15.33:0/2080033175 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a38102450 0x7f6a381a1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:02.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.967+0000 7f6a3d400700 1 -- 172.21.15.33:0/2080033175 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6a38109520 msgr2=0x7f6a3819b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:30:02.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.967+0000 7f6a3d400700 1 --2- 172.21.15.33:0/2080033175 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6a38109520 0x7f6a3819b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:02.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.967+0000 7f6a3d400700 1 -- 172.21.15.33:0/2080033175 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a3819c110 con 0x7f6a38101a90 2024-05-03T18:30:02.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.967+0000 7f6a3d400700 1 --2- 172.21.15.33:0/2080033175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a38101a90 0x7f6a381a17f0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c00c9b0 tx=0x7f6a2c00cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:30:02.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.968+0000 7f6a36000700 1 -- 172.21.15.33:0/2080033175 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c007b00 con 0x7f6a38101a90 2024-05-03T18:30:02.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.968+0000 7f6a36000700 1 -- 172.21.15.33:0/2080033175 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6a2c007ca0 con 0x7f6a38101a90 2024-05-03T18:30:02.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.968+0000 7f6a3fe00700 1 -- 172.21.15.33:0/2080033175 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a3819c3a0 con 0x7f6a38101a90 2024-05-03T18:30:02.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.968+0000 7f6a36000700 1 -- 172.21.15.33:0/2080033175 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c018660 con 0x7f6a38101a90 2024-05-03T18:30:02.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.968+0000 7f6a3fe00700 1 -- 172.21.15.33:0/2080033175 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a381a86c0 con 0x7f6a38101a90 2024-05-03T18:30:02.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.972+0000 7f6a36000700 1 -- 172.21.15.33:0/2080033175 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f6a2c00f450 con 0x7f6a38101a90 2024-05-03T18:30:02.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.973+0000 7f6a36000700 1 --2- 172.21.15.33:0/2080033175 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f6a200b29f0 0x7f6a200b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:02.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.973+0000 7f6a4a836b80 1 Processor -- start 2024-05-03T18:30:02.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.974+0000 7f6a36000700 1 -- 172.21.15.33:0/2080033175 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f6a2c0e2a50 con 0x7f6a38101a90 2024-05-03T18:30:02.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.974+0000 7f6a4a836b80 1 -- start start 2024-05-03T18:30:02.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.974+0000 7f6a3ca00700 1 --2- 172.21.15.33:0/2080033175 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f6a200b29f0 0x7f6a200b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:02.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.974+0000 7f6a4a836b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8dc1cbfc0 0x55f8dc1cc3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:02.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.974+0000 7f6a4a836b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f8dc1cc8e0 0x55f8dc1c77c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:02.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.974+0000 7f6a4a836b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f8dc1c7de0 0x55f8dc1c8240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:02.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.974+0000 7f6a4a836b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55f8dc111a20 con 0x55f8dc1cc8e0 2024-05-03T18:30:02.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.974+0000 7f6a4a836b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55f8dc0af760 con 0x55f8dc1cbfc0 2024-05-03T18:30:02.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.974+0000 7f6a4a836b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55f8dc10d9e0 con 0x55f8dc1c7de0 2024-05-03T18:30:02.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.975+0000 7f6a3ca00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f8dc1cc8e0 0x55f8dc1c77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:02.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.975+0000 7f6a3d400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8dc1cbfc0 0x55f8dc1cc3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:02.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.975+0000 7f6a3d400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8dc1cbfc0 0x55f8dc1cc3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:53650/0 (socket says 172.21.15.33:53650) 2024-05-03T18:30:02.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.975+0000 7f6a3d400700 1 -- 172.21.15.33:0/554809362 learned_addr learned my addr 172.21.15.33:0/554809362 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:30:02.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.975+0000 7f6a3de00700 1 --2- 172.21.15.33:0/554809362 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f8dc1c7de0 0x55f8dc1c8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:02.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.975+0000 7f6a3ca00700 1 --2- 172.21.15.33:0/2080033175 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f6a200b29f0 0x7f6a200b4eb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6a24005950 tx=0x7f6a240058e0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:30:02.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.976+0000 7f6a3d400700 1 -- 172.21.15.33:0/554809362 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f8dc1c7de0 msgr2=0x55f8dc1c8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:30:02.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.976+0000 7f6a3d400700 1 --2- 172.21.15.33:0/554809362 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f8dc1c7de0 0x55f8dc1c8240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:02.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.976+0000 7f6a3d400700 1 -- 172.21.15.33:0/554809362 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f8dc1cc8e0 msgr2=0x55f8dc1c77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:30:02.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.976+0000 7f6a3d400700 1 --2- 172.21.15.33:0/554809362 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f8dc1cc8e0 0x55f8dc1c77c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:02.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.976+0000 7f6a3d400700 1 -- 172.21.15.33:0/554809362 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8dc05e8d0 con 0x55f8dc1cbfc0 2024-05-03T18:30:02.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.976+0000 7f6a3ca00700 1 --2- 172.21.15.33:0/554809362 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f8dc1cc8e0 0x55f8dc1c77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:30:02.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.976+0000 7f6a3de00700 1 --2- 172.21.15.33:0/554809362 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f8dc1c7de0 0x55f8dc1c8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:30:02.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.976+0000 7f6a3d400700 1 --2- 172.21.15.33:0/554809362 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8dc1cbfc0 0x55f8dc1cc3a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c0a98d0 tx=0x7f6a2c0a9520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=926cede5113b54ef server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:30:02.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.976+0000 7f6a2aa00700 1 -- 172.21.15.33:0/554809362 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c08fe60 con 0x55f8dc1cbfc0 2024-05-03T18:30:02.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.977+0000 7f6a2aa00700 1 -- 172.21.15.33:0/554809362 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6a2c0adcf0 con 0x55f8dc1cbfc0 2024-05-03T18:30:02.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.977+0000 7f6a2aa00700 1 -- 172.21.15.33:0/554809362 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c0a5bd0 con 0x55f8dc1cbfc0 2024-05-03T18:30:02.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.977+0000 7f6a4a836b80 1 -- 172.21.15.33:0/554809362 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8dc1cbfc0 msgr2=0x55f8dc1cc3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:30:02.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.977+0000 7f6a4a836b80 1 --2- 172.21.15.33:0/554809362 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8dc1cbfc0 0x55f8dc1cc3a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c0a98d0 tx=0x7f6a2c0a9520 comp rx=0 tx=0).stop 2024-05-03T18:30:02.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.978+0000 7f6a4a836b80 1 -- 172.21.15.33:0/554809362 shutdown_connections 2024-05-03T18:30:02.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.978+0000 7f6a4a836b80 1 --2- 172.21.15.33:0/554809362 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f8dc1c7de0 0x55f8dc1c8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:02.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.978+0000 7f6a4a836b80 1 --2- 172.21.15.33:0/554809362 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8dc1cbfc0 0x55f8dc1cc3a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:02.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.978+0000 7f6a4a836b80 1 --2- 172.21.15.33:0/554809362 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f8dc1cc8e0 0x55f8dc1c77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:02.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.978+0000 7f6a4a836b80 1 -- 172.21.15.33:0/554809362 >> 172.21.15.33:0/554809362 conn(0x55f8dc0edb60 msgr2=0x55f8dc1b97f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:30:02.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.978+0000 7f6a4a836b80 1 -- 172.21.15.33:0/554809362 shutdown_connections 2024-05-03T18:30:02.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.978+0000 7f6a4a836b80 1 -- 172.21.15.33:0/554809362 wait complete. 2024-05-03T18:30:02.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.979+0000 7f6a4a836b80 1 -- start start 2024-05-03T18:30:02.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.979+0000 7f6a4a836b80 1 Processor -- start 2024-05-03T18:30:02.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.980+0000 7f6a4a836b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f8dc1c7de0 0x55f8dc1d3770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:02.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.980+0000 7f6a4a836b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8dc1cbfc0 0x55f8dc1d0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:02.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.980+0000 7f6a4a836b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f8dc1cc8e0 0x55f8dc1d1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:02.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.980+0000 7f6a4a836b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55f8dc1d4450 con 0x55f8dc1cc8e0 2024-05-03T18:30:02.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.980+0000 7f6a4a836b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55f8dc077b20 con 0x55f8dc1cbfc0 2024-05-03T18:30:02.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.980+0000 7f6a4a836b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55f8dc1d45d0 con 0x55f8dc1c7de0 2024-05-03T18:30:02.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.980+0000 7f6a4a836b80 10 client.0 _ll_register_callbacks cb 0x55f8dc0b8c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:30:02.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.980+0000 7f6a3de00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f8dc1cc8e0 0x55f8dc1d1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:02.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.980+0000 7f6a3de00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f8dc1cc8e0 0x55f8dc1d1050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52668/0 (socket says 172.21.15.33:52668) 2024-05-03T18:30:02.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.980+0000 7f6a3de00700 1 -- 172.21.15.33:0/2522815697 learned_addr learned my addr 172.21.15.33:0/2522815697 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:30:02.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.980+0000 7f6a3ca00700 1 --2- 172.21.15.33:0/2522815697 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8dc1cbfc0 0x55f8dc1d0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:02.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.981+0000 7f6a3d400700 1 --2- 172.21.15.33:0/2522815697 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f8dc1c7de0 0x55f8dc1d3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:02.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.981+0000 7f6a3de00700 1 -- 172.21.15.33:0/2522815697 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f8dc1c7de0 msgr2=0x55f8dc1d3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:30:02.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.981+0000 7f6a3de00700 1 --2- 172.21.15.33:0/2522815697 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f8dc1c7de0 0x55f8dc1d3770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:02.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.981+0000 7f6a3de00700 1 -- 172.21.15.33:0/2522815697 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8dc1cbfc0 msgr2=0x55f8dc1d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:30:02.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.981+0000 7f6a3de00700 1 --2- 172.21.15.33:0/2522815697 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8dc1cbfc0 0x55f8dc1d0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:02.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.981+0000 7f6a3de00700 1 -- 172.21.15.33:0/2522815697 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8dc0d7420 con 0x55f8dc1cc8e0 2024-05-03T18:30:02.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.981+0000 7f6a3d400700 1 --2- 172.21.15.33:0/2522815697 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f8dc1c7de0 0x55f8dc1d3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:30:02.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.981+0000 7f6a3de00700 1 --2- 172.21.15.33:0/2522815697 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f8dc1cc8e0 0x55f8dc1d1050 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f6a30015000 tx=0x7f6a30004d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5db6fd9dbaeb157 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:30:02.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.981+0000 7f6a1f400700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:30:02.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.981+0000 7f6a1f400700 1 -- 172.21.15.33:0/2522815697 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a30020070 con 0x55f8dc1cc8e0 2024-05-03T18:30:02.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.981+0000 7f6a1f400700 1 -- 172.21.15.33:0/2522815697 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f6a300078f0 con 0x55f8dc1cc8e0 2024-05-03T18:30:02.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.981+0000 7f6a4a836b80 1 -- 172.21.15.33:0/2522815697 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a2c0adcf0 con 0x55f8dc1cc8e0 2024-05-03T18:30:02.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.982+0000 7f6a1f400700 1 -- 172.21.15.33:0/2522815697 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f6a3002e050 con 0x55f8dc1cc8e0 2024-05-03T18:30:02.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.982+0000 7f6a4a836b80 10 client.15720 fetch_fsmap learned FSMap version 21 2024-05-03T18:30:02.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.982+0000 7f6a4a836b80 1 -- 172.21.15.33:0/2522815697 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55f8dc1ded20 con 0x55f8dc1cc8e0 2024-05-03T18:30:02.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.982+0000 7f6a1f400700 1 -- 172.21.15.33:0/2522815697 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6a2c0adcf0 con 0x55f8dc1cc8e0 2024-05-03T18:30:02.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.982+0000 7f6a1f400700 1 -- 172.21.15.33:0/2522815697 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f6a30011070 con 0x55f8dc1cc8e0 2024-05-03T18:30:02.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.983+0000 7f6a4a836b80 10 client.15720 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:30:02.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.983+0000 7f6a4a836b80 10 client.15720 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:30:02.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.983+0000 7f6a4a836b80 20 client.15720 populate_metadata read hostname 'smithi033' 2024-05-03T18:30:02.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.983+0000 7f6a4a836b80 1 --2- 172.21.15.33:0/2522815697 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55f8dc1e1360 0x55f8dc1e3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:02.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.983+0000 7f6a4a836b80 4 client.15720 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:30:02.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.983+0000 7f6a4a836b80 1 -- 172.21.15.33:0/2522815697 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f8dc18c1d0 con 0x55f8dc1e1360 2024-05-03T18:30:02.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.984+0000 7f6a3d400700 1 --2- 172.21.15.33:0/2522815697 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55f8dc1e1360 0x55f8dc1e3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:02.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.985+0000 7f6a3d400700 1 --2- 172.21.15.33:0/2522815697 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55f8dc1e1360 0x55f8dc1e3750 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e052ff8950d0be5 server_cookie=2f708318c99c6430 in_seq=0 out_seq=0 2024-05-03T18:30:02.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.985+0000 7f6a1f400700 10 client.15720 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:30:02.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.986+0000 7f6a1f400700 1 -- 172.21.15.33:0/2522815697 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f8dc18c1d0 con 0x55f8dc1e1360 2024-05-03T18:30:02.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.986+0000 7f6a1f400700 10 client.15720 handle_command_reply: tid=0 2024-05-03T18:30:02.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.987+0000 7f6a4a836b80 1 client.15720 shutdown 2024-05-03T18:30:02.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.987+0000 7f6a4a836b80 1 -- 172.21.15.33:0/2522815697 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f8dc1cc8e0 msgr2=0x55f8dc1d1050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:30:02.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.987+0000 7f6a4a836b80 1 --2- 172.21.15.33:0/2522815697 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f8dc1cc8e0 0x55f8dc1d1050 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f6a30015000 tx=0x7f6a30004d30 comp rx=0 tx=0).stop 2024-05-03T18:30:02.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.987+0000 7f6a4a836b80 1 -- 172.21.15.33:0/2522815697 shutdown_connections 2024-05-03T18:30:02.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.987+0000 7f6a4a836b80 1 --2- 172.21.15.33:0/2522815697 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55f8dc1e1360 0x55f8dc1e3750 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:02.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.987+0000 7f6a4a836b80 1 --2- 172.21.15.33:0/2522815697 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f8dc1cc8e0 0x55f8dc1d1050 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:02.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.987+0000 7f6a4a836b80 1 --2- 172.21.15.33:0/2522815697 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f8dc1c7de0 0x55f8dc1d3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:02.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.987+0000 7f6a4a836b80 1 --2- 172.21.15.33:0/2522815697 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8dc1cbfc0 0x55f8dc1d0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:02.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.987+0000 7f6a4a836b80 1 -- 172.21.15.33:0/2522815697 >> 172.21.15.33:0/2522815697 conn(0x55f8dc0edb60 msgr2=0x55f8dc1cd610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:30:02.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.987+0000 7f6a1f400700 0 client.15720 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:30:02.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.987+0000 7f6a4a836b80 1 -- 172.21.15.33:0/2522815697 shutdown_connections 2024-05-03T18:30:02.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.987+0000 7f6a4a836b80 1 -- 172.21.15.33:0/2522815697 wait complete. 2024-05-03T18:30:02.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:02.988+0000 7f6a4a836b80 20 client.15720 trim_cache size 0 max 16384 2024-05-03T18:30:03.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:03.007+0000 7f6a4a836b80 1 Processor -- start 2024-05-03T18:30:03.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:03.007+0000 7f6a4a836b80 1 -- start start 2024-05-03T18:30:03.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:03.008+0000 7f6a4a836b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f8dc1c7de0 0x55f8dc1d1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:03.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:03.008+0000 7f6a4a836b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8dc1cbfc0 0x55f8dc1dd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:03.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:03.008+0000 7f6a4a836b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f8dc1cc8e0 0x55f8dc1cb310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:03.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:03.008+0000 7f6a4a836b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6a381ab0a0 con 0x55f8dc1cc8e0 2024-05-03T18:30:03.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:03.008+0000 7f6a4a836b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6a381ad900 con 0x55f8dc1cbfc0 2024-05-03T18:30:03.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:03.008+0000 7f6a4a836b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6a381ab6f0 con 0x55f8dc1c7de0 2024-05-03T18:30:03.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:03.009+0000 7f6a3de00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f8dc1cc8e0 0x55f8dc1cb310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:03.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:03.009+0000 7f6a3de00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f8dc1cc8e0 0x55f8dc1cb310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52684/0 (socket says 172.21.15.33:52684) 2024-05-03T18:30:03.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:03.009+0000 7f6a3de00700 1 -- 172.21.15.33:0/603732969 learned_addr learned my addr 172.21.15.33:0/603732969 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:30:03.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:03.009+0000 7f6a3d400700 1 --2- 172.21.15.33:0/603732969 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f8dc1c7de0 0x55f8dc1d1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:03.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:03.009+0000 7f6a3ca00700 1 --2- 172.21.15.33:0/603732969 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8dc1cbfc0 0x55f8dc1dd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:03.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:03.009+0000 7f6a3de00700 1 -- 172.21.15.33:0/603732969 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f8dc1c7de0 msgr2=0x55f8dc1d1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:30:03.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:03.009+0000 7f6a3de00700 1 --2- 172.21.15.33:0/603732969 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f8dc1c7de0 0x55f8dc1d1140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:03.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:03.009+0000 7f6a3de00700 1 -- 172.21.15.33:0/603732969 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8dc1cbfc0 msgr2=0x55f8dc1dd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:30:03.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:03.009+0000 7f6a3de00700 1 --2- 172.21.15.33:0/603732969 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8dc1cbfc0 0x55f8dc1dd4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:03.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:03.009+0000 7f6a3de00700 1 -- 172.21.15.33:0/603732969 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8dc1cf0a0 con 0x55f8dc1cc8e0 2024-05-03T18:30:03.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:03.010+0000 7f6a3de00700 1 --2- 172.21.15.33:0/603732969 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f8dc1cc8e0 0x55f8dc1cb310 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f6a30025c90 tx=0x7f6a30004e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37c5e496e32e1528 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:30:03.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:03.010+0000 7f6a2a000700 1 -- 172.21.15.33:0/603732969 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a30020070 con 0x55f8dc1cc8e0 2024-05-03T18:30:03.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:03.010+0000 7f6a2a000700 1 -- 172.21.15.33:0/603732969 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6a300258f0 con 0x55f8dc1cc8e0 2024-05-03T18:30:03.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:03.010+0000 7f6a2a000700 1 -- 172.21.15.33:0/603732969 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a300077e0 con 0x55f8dc1cc8e0 2024-05-03T18:30:03.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:03.011+0000 7f6a4a836b80 1 -- 172.21.15.33:0/603732969 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f8dc1cc8e0 msgr2=0x55f8dc1cb310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:30:03.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:03.011+0000 7f6a4a836b80 1 --2- 172.21.15.33:0/603732969 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f8dc1cc8e0 0x55f8dc1cb310 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f6a30025c90 tx=0x7f6a30004e50 comp rx=0 tx=0).stop 2024-05-03T18:30:03.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:03.011+0000 7f6a4a836b80 1 -- 172.21.15.33:0/603732969 shutdown_connections 2024-05-03T18:30:03.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:03.011+0000 7f6a4a836b80 1 --2- 172.21.15.33:0/603732969 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f8dc1cc8e0 0x55f8dc1cb310 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:03.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:03.011+0000 7f6a4a836b80 1 --2- 172.21.15.33:0/603732969 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f8dc1c7de0 0x55f8dc1d1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:03.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:03.011+0000 7f6a4a836b80 1 --2- 172.21.15.33:0/603732969 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8dc1cbfc0 0x55f8dc1dd4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:03.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:03.011+0000 7f6a4a836b80 1 -- 172.21.15.33:0/603732969 >> 172.21.15.33:0/603732969 conn(0x55f8dc0edb60 msgr2=0x55f8dc1cac00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:30:03.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:03.012+0000 7f6a4a836b80 1 -- 172.21.15.33:0/603732969 shutdown_connections 2024-05-03T18:30:03.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:03.012+0000 7f6a4a836b80 1 -- 172.21.15.33:0/603732969 wait complete. 2024-05-03T18:30:03.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:03.012+0000 7f6a4a836b80 1 -- start start 2024-05-03T18:30:03.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:03.013+0000 7f6a4a836b80 1 Processor -- start 2024-05-03T18:30:03.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:03.013+0000 7f6a4a836b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f8dc1c7de0 0x55f8dc1d9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:03.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:03.013+0000 7f6a4a836b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f8dc1cbfc0 0x55f8dc1c5180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:03.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:03.013+0000 7f6a4a836b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8dc1cc8e0 0x55f8dc1c56c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:03.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:03.013+0000 7f6a4a836b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6a381abdc0 con 0x55f8dc1c7de0 2024-05-03T18:30:03.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:03.013+0000 7f6a4a836b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6a381ad230 con 0x55f8dc1cc8e0 2024-05-03T18:30:03.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:03.013+0000 7f6a4a836b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6a381ac490 con 0x55f8dc1cbfc0 2024-05-03T18:30:03.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:03.013+0000 7f6a4a836b80 10 client.0 _ll_register_callbacks cb 0x55f8dc0b8c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:30:03.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:03.014+0000 7f6a3d400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f8dc1c7de0 0x55f8dc1d9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:03.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:03.014+0000 7f6a3d400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f8dc1c7de0 0x55f8dc1d9de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52698/0 (socket says 172.21.15.33:52698) 2024-05-03T18:30:03.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:03.014+0000 7f6a3d400700 1 -- 172.21.15.33:0/1585143200 learned_addr learned my addr 172.21.15.33:0/1585143200 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:30:03.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:03.014+0000 7f6a3de00700 1 --2- 172.21.15.33:0/1585143200 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8dc1cc8e0 0x55f8dc1c56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:03.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:03.014+0000 7f6a3ca00700 1 --2- 172.21.15.33:0/1585143200 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f8dc1cbfc0 0x55f8dc1c5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:03.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:03.014+0000 7f6a3d400700 1 -- 172.21.15.33:0/1585143200 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f8dc1cbfc0 msgr2=0x55f8dc1c5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:30:03.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:03.014+0000 7f6a3d400700 1 --2- 172.21.15.33:0/1585143200 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f8dc1cbfc0 0x55f8dc1c5180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:03.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:03.014+0000 7f6a3d400700 1 -- 172.21.15.33:0/1585143200 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8dc1cc8e0 msgr2=0x55f8dc1c56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:30:03.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:03.014+0000 7f6a3d400700 1 --2- 172.21.15.33:0/1585143200 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8dc1cc8e0 0x55f8dc1c56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:03.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:03.014+0000 7f6a3d400700 1 -- 172.21.15.33:0/1585143200 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8dc1c5ed0 con 0x55f8dc1c7de0 2024-05-03T18:30:03.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:03.014+0000 7f6a3de00700 1 --2- 172.21.15.33:0/1585143200 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8dc1cc8e0 0x55f8dc1c56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:30:03.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:03.015+0000 7f6a3d400700 1 --2- 172.21.15.33:0/1585143200 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f8dc1c7de0 0x55f8dc1d9de0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c0a9470 tx=0x7f6a2c0a5930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f99c568d2782d6b server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:30:03.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:03.015+0000 7f6a1f400700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:30:03.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:03.015+0000 7f6a1f400700 1 -- 172.21.15.33:0/1585143200 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c08fe60 con 0x55f8dc1c7de0 2024-05-03T18:30:03.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:03.015+0000 7f6a1f400700 1 -- 172.21.15.33:0/1585143200 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f6a2c0a5bd0 con 0x55f8dc1c7de0 2024-05-03T18:30:03.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:03.016+0000 7f6a4a836b80 1 -- 172.21.15.33:0/1585143200 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a300258f0 con 0x55f8dc1c7de0 2024-05-03T18:30:03.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:03.016+0000 7f6a1f400700 1 -- 172.21.15.33:0/1585143200 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f6a2c0adcf0 con 0x55f8dc1c7de0 2024-05-03T18:30:03.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:03.016+0000 7f6a4a836b80 10 client.15726 fetch_fsmap learned FSMap version 21 2024-05-03T18:30:03.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:03.016+0000 7f6a4a836b80 1 -- 172.21.15.33:0/1585143200 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55f8dc24bc80 con 0x55f8dc1c7de0 2024-05-03T18:30:03.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:03.016+0000 7f6a1f400700 1 -- 172.21.15.33:0/1585143200 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6a300258f0 con 0x55f8dc1c7de0 2024-05-03T18:30:03.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:03.017+0000 7f6a1f400700 1 -- 172.21.15.33:0/1585143200 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f6a2c06f1e0 con 0x55f8dc1c7de0 2024-05-03T18:30:03.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:03.017+0000 7f6a4a836b80 10 client.15726 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:30:03.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:03.017+0000 7f6a4a836b80 10 client.15726 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:30:03.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:03.017+0000 7f6a4a836b80 20 client.15726 populate_metadata read hostname 'smithi033' 2024-05-03T18:30:03.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:03.017+0000 7f6a4a836b80 1 --2- 172.21.15.33:0/1585143200 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55f8dc24bf70 0x55f8dc24e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:03.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:03.017+0000 7f6a4a836b80 4 client.15726 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-03T18:30:03.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:03.017+0000 7f6a4a836b80 1 -- 172.21.15.33:0/1585143200 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f8dc11f400 con 0x55f8dc24bf70 2024-05-03T18:30:03.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:03.017+0000 7f6a3ca00700 1 --2- 172.21.15.33:0/1585143200 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55f8dc24bf70 0x55f8dc24e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:03.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:03.019+0000 7f6a1f400700 10 client.15726 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:30:03.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:03.019+0000 7f6a3ca00700 1 --2- 172.21.15.33:0/1585143200 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55f8dc24bf70 0x55f8dc24e360 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82f37dc0f3be2d02 server_cookie=a9e30e43cbf16576 in_seq=0 out_seq=0 2024-05-03T18:30:03.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:03.019+0000 7f6a1f400700 1 -- 172.21.15.33:0/1585143200 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f8dc11f400 con 0x55f8dc24bf70 2024-05-03T18:30:03.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:03.019+0000 7f6a1f400700 10 client.15726 handle_command_reply: tid=0 2024-05-03T18:30:03.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:03.019+0000 7f6a4a836b80 1 client.15726 shutdown 2024-05-03T18:30:03.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:03.019+0000 7f6a4a836b80 1 -- 172.21.15.33:0/1585143200 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f8dc1c7de0 msgr2=0x55f8dc1d9de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:30:03.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:03.019+0000 7f6a4a836b80 1 --2- 172.21.15.33:0/1585143200 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f8dc1c7de0 0x55f8dc1d9de0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c0a9470 tx=0x7f6a2c0a5930 comp rx=0 tx=0).stop 2024-05-03T18:30:03.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:03.020+0000 7f6a4a836b80 1 -- 172.21.15.33:0/1585143200 shutdown_connections 2024-05-03T18:30:03.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:03.020+0000 7f6a4a836b80 1 --2- 172.21.15.33:0/1585143200 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55f8dc24bf70 0x55f8dc24e360 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:03.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:03.020+0000 7f6a4a836b80 1 --2- 172.21.15.33:0/1585143200 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8dc1cc8e0 0x55f8dc1c56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:03.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:03.020+0000 7f6a4a836b80 1 --2- 172.21.15.33:0/1585143200 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f8dc1c7de0 0x55f8dc1d9de0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:03.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:03.020+0000 7f6a4a836b80 1 --2- 172.21.15.33:0/1585143200 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f8dc1cbfc0 0x55f8dc1c5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:03.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:03.020+0000 7f6a4a836b80 1 -- 172.21.15.33:0/1585143200 >> 172.21.15.33:0/1585143200 conn(0x55f8dc0edb60 msgr2=0x55f8dc1c6bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:30:03.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:03.020+0000 7f6a1f400700 0 client.15726 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:30:03.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:03.020+0000 7f6a4a836b80 1 -- 172.21.15.33:0/1585143200 shutdown_connections 2024-05-03T18:30:03.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:03.020+0000 7f6a4a836b80 1 -- 172.21.15.33:0/1585143200 wait complete. 2024-05-03T18:30:03.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:03.020+0000 7f6a4a836b80 20 client.15726 trim_cache size 0 max 16384 2024-05-03T18:30:03.022 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-03T18:30:03.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:03.021+0000 7f6a3fe00700 1 -- 172.21.15.33:0/2080033175 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f6a200b29f0 msgr2=0x7f6a200b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:30:03.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:03.021+0000 7f6a3fe00700 1 --2- 172.21.15.33:0/2080033175 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f6a200b29f0 0x7f6a200b4eb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6a24005950 tx=0x7f6a240058e0 comp rx=0 tx=0).stop 2024-05-03T18:30:03.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:03.021+0000 7f6a3fe00700 1 -- 172.21.15.33:0/2080033175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a38101a90 msgr2=0x7f6a381a17f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:30:03.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:03.021+0000 7f6a3fe00700 1 --2- 172.21.15.33:0/2080033175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a38101a90 0x7f6a381a17f0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c00c9b0 tx=0x7f6a2c00cd70 comp rx=0 tx=0).stop 2024-05-03T18:30:03.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:03.022+0000 7f6a3fe00700 1 -- 172.21.15.33:0/2080033175 shutdown_connections 2024-05-03T18:30:03.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:03.022+0000 7f6a3fe00700 1 --2- 172.21.15.33:0/2080033175 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f6a200b29f0 0x7f6a200b4eb0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:03.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:03.022+0000 7f6a3fe00700 1 --2- 172.21.15.33:0/2080033175 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6a38109520 0x7f6a3819b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:03.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:03.022+0000 7f6a3fe00700 1 --2- 172.21.15.33:0/2080033175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a38101a90 0x7f6a381a17f0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:03.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:03.022+0000 7f6a3fe00700 1 --2- 172.21.15.33:0/2080033175 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a38102450 0x7f6a381a1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:03.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:03.022+0000 7f6a3fe00700 1 -- 172.21.15.33:0/2080033175 >> 172.21.15.33:0/2080033175 conn(0x7f6a38078a10 msgr2=0x7f6a380ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:30:03.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:03.022+0000 7f6a3fe00700 1 -- 172.21.15.33:0/2080033175 shutdown_connections 2024-05-03T18:30:03.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:03.022+0000 7f6a3fe00700 1 -- 172.21.15.33:0/2080033175 wait complete. 2024-05-03T18:30:04.028 INFO:teuthology.orchestra.run.smithi033.stderr:2900000 of 3200000 tuples (90%) done (elapsed 101.14 s, remaining 10.46 s) 2024-05-03T18:30:05.021 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-03T18:30:05.021 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-03T18:30:05.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:30:04 smithi033 ceph-mon[28330]: pgmap v89: 129 pgs: 129 active+clean; 749 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 17 MiB/s wr, 30 op/s 2024-05-03T18:30:05.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:30:04 smithi099 ceph-mon[31658]: pgmap v89: 129 pgs: 129 active+clean; 749 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 17 MiB/s wr, 30 op/s 2024-05-03T18:30:05.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:30:04 smithi150 ceph-mon[32492]: pgmap v89: 129 pgs: 129 active+clean; 749 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 17 MiB/s wr, 30 op/s 2024-05-03T18:30:05.541 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:30:07.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:30:06 smithi033 ceph-mon[28330]: pgmap v90: 129 pgs: 129 active+clean; 749 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 11 MiB/s wr, 24 op/s 2024-05-03T18:30:07.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:30:06 smithi099 ceph-mon[31658]: pgmap v90: 129 pgs: 129 active+clean; 749 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 11 MiB/s wr, 24 op/s 2024-05-03T18:30:07.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:30:06 smithi150 ceph-mon[32492]: pgmap v90: 129 pgs: 129 active+clean; 749 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 11 MiB/s wr, 24 op/s 2024-05-03T18:30:07.895 INFO:teuthology.orchestra.run.smithi033.stderr:3000000 of 3200000 tuples (93%) done (elapsed 105.00 s, remaining 7.00 s) 2024-05-03T18:30:08.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.024+0000 7f3fc8e00700 1 -- 172.21.15.33:0/92990588 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fc4108850 msgr2=0x7f3fc41103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:30:08.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.024+0000 7f3fc8e00700 1 --2- 172.21.15.33:0/92990588 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fc4108850 0x7f3fc41103d0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc0099c0 tx=0x7f3fbc009cd0 comp rx=0 tx=0).stop 2024-05-03T18:30:08.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.024+0000 7f3fc8e00700 1 -- 172.21.15.33:0/92990588 shutdown_connections 2024-05-03T18:30:08.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.024+0000 7f3fc8e00700 1 --2- 172.21.15.33:0/92990588 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fc4108850 0x7f3fc41103d0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:08.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.024+0000 7f3fc8e00700 1 --2- 172.21.15.33:0/92990588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3fc4103750 0x7f3fc4103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:08.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.024+0000 7f3fc8e00700 1 --2- 172.21.15.33:0/92990588 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3fc4104100 0x7f3fc4108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:08.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.024+0000 7f3fc8e00700 1 -- 172.21.15.33:0/92990588 >> 172.21.15.33:0/92990588 conn(0x7f3fc407a600 msgr2=0x7f3fc407aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:30:08.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.024+0000 7f3fc8e00700 1 -- 172.21.15.33:0/92990588 shutdown_connections 2024-05-03T18:30:08.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.024+0000 7f3fc8e00700 1 -- 172.21.15.33:0/92990588 wait complete. 2024-05-03T18:30:08.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.026+0000 7f3fc8e00700 1 Processor -- start 2024-05-03T18:30:08.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.026+0000 7f3fc8e00700 1 -- start start 2024-05-03T18:30:08.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.026+0000 7f3fc8e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3fc4103750 0x7f3fc41a1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:08.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.026+0000 7f3fc8e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3fc4104100 0x7f3fc41a1c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:08.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.026+0000 7f3fc8e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fc4108850 0x7f3fc419b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:08.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.026+0000 7f3fc8e00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3fc4113520 con 0x7f3fc4108850 2024-05-03T18:30:08.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.026+0000 7f3fc8e00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3fc41133a0 con 0x7f3fc4103750 2024-05-03T18:30:08.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.026+0000 7f3fc8e00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3fc41136a0 con 0x7f3fc4104100 2024-05-03T18:30:08.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.027+0000 7f3fb9400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3fc4104100 0x7f3fc41a1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:08.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.027+0000 7f3fc2800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fc4108850 0x7f3fc419b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:08.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.027+0000 7f3fc1e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3fc4103750 0x7f3fc41a1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:08.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.027+0000 7f3fb9400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3fc4104100 0x7f3fc41a1c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:49126/0 (socket says 172.21.15.33:49126) 2024-05-03T18:30:08.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.027+0000 7f3fb9400700 1 -- 172.21.15.33:0/3641348333 learned_addr learned my addr 172.21.15.33:0/3641348333 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:30:08.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.028+0000 7f3fb9400700 1 -- 172.21.15.33:0/3641348333 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3fc4103750 msgr2=0x7f3fc41a1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:30:08.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.028+0000 7f3fb9400700 1 --2- 172.21.15.33:0/3641348333 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3fc4103750 0x7f3fc41a1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:08.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.028+0000 7f3fb9400700 1 -- 172.21.15.33:0/3641348333 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fc4108850 msgr2=0x7f3fc419b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:30:08.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.028+0000 7f3fb9400700 1 --2- 172.21.15.33:0/3641348333 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fc4108850 0x7f3fc419b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:08.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.028+0000 7f3fb9400700 1 -- 172.21.15.33:0/3641348333 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fc419c050 con 0x7f3fc4104100 2024-05-03T18:30:08.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.028+0000 7f3fc2800700 1 --2- 172.21.15.33:0/3641348333 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fc4108850 0x7f3fc419b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:30:08.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.028+0000 7f3fb9400700 1 --2- 172.21.15.33:0/3641348333 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3fc4104100 0x7f3fc41a1c70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3fac00c9b0 tx=0x7f3fac00ccc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:30:08.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.029+0000 7f3fbb400700 1 -- 172.21.15.33:0/3641348333 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fac007b00 con 0x7f3fc4104100 2024-05-03T18:30:08.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.029+0000 7f3fc8e00700 1 -- 172.21.15.33:0/3641348333 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fc419c340 con 0x7f3fc4104100 2024-05-03T18:30:08.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.029+0000 7f3fbb400700 1 -- 172.21.15.33:0/3641348333 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3fac00ce50 con 0x7f3fc4104100 2024-05-03T18:30:08.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.029+0000 7f3fc8e00700 1 -- 172.21.15.33:0/3641348333 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fc41a8600 con 0x7f3fc4104100 2024-05-03T18:30:08.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.029+0000 7f3fbb400700 1 -- 172.21.15.33:0/3641348333 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fac018660 con 0x7f3fc4104100 2024-05-03T18:30:08.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.035+0000 7f3fd370fb80 1 Processor -- start 2024-05-03T18:30:08.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.035+0000 7f3fd370fb80 1 -- start start 2024-05-03T18:30:08.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.035+0000 7f3fbb400700 1 -- 172.21.15.33:0/3641348333 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f3fac007ca0 con 0x7f3fc4104100 2024-05-03T18:30:08.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.035+0000 7f3fd370fb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558caaae99b0 0x558caaae9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:08.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.035+0000 7f3fd370fb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558caaaea2d0 0x558caaae3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:08.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.035+0000 7f3fd370fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558caaae45d0 0x558caaae4a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:08.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.035+0000 7f3fd370fb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x558caaa36aa0 con 0x558caaaea2d0 2024-05-03T18:30:08.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.035+0000 7f3fd370fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x558caa9d47e0 con 0x558caaae45d0 2024-05-03T18:30:08.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.035+0000 7f3fd370fb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x558caaa32a60 con 0x558caaae99b0 2024-05-03T18:30:08.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.036+0000 7f3fc1e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558caaaea2d0 0x558caaae3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:08.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.036+0000 7f3fc1e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558caaaea2d0 0x558caaae3fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43710/0 (socket says 172.21.15.33:43710) 2024-05-03T18:30:08.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.036+0000 7f3fc1e00700 1 -- 172.21.15.33:0/939998704 learned_addr learned my addr 172.21.15.33:0/939998704 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:30:08.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.036+0000 7f3fc2800700 1 --2- 172.21.15.33:0/939998704 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558caaae99b0 0x558caaae9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:08.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.036+0000 7f3fb9400700 1 --2- 172.21.15.33:0/939998704 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558caaae45d0 0x558caaae4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:08.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.036+0000 7f3fc1e00700 1 -- 172.21.15.33:0/939998704 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558caaae99b0 msgr2=0x558caaae9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:30:08.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.036+0000 7f3fc1e00700 1 --2- 172.21.15.33:0/939998704 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558caaae99b0 0x558caaae9d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:08.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.036+0000 7f3fc1e00700 1 -- 172.21.15.33:0/939998704 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558caaae45d0 msgr2=0x558caaae4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:30:08.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.037+0000 7f3fc1e00700 1 --2- 172.21.15.33:0/939998704 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558caaae45d0 0x558caaae4a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:08.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.037+0000 7f3fc1e00700 1 -- 172.21.15.33:0/939998704 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558caa983950 con 0x558caaaea2d0 2024-05-03T18:30:08.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.037+0000 7f3fc2800700 1 --2- 172.21.15.33:0/939998704 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558caaae99b0 0x558caaae9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:30:08.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.037+0000 7f3fb9400700 1 --2- 172.21.15.33:0/939998704 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558caaae45d0 0x558caaae4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:30:08.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.037+0000 7f3fc1e00700 1 --2- 172.21.15.33:0/939998704 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558caaaea2d0 0x558caaae3fb0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f3fb400f570 tx=0x7f3fb400f880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ade0245775660756 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:30:08.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.037+0000 7f3fb3400700 1 -- 172.21.15.33:0/939998704 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb4018020 con 0x558caaaea2d0 2024-05-03T18:30:08.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.037+0000 7f3fbb400700 1 --2- 172.21.15.33:0/3641348333 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f3fa80b2a70 0x7f3fa80b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:08.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.037+0000 7f3fb3400700 1 -- 172.21.15.33:0/939998704 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3fb4014b10 con 0x558caaaea2d0 2024-05-03T18:30:08.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.037+0000 7f3fb3400700 1 -- 172.21.15.33:0/939998704 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb4013ae0 con 0x558caaaea2d0 2024-05-03T18:30:08.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.038+0000 7f3fbb400700 1 -- 172.21.15.33:0/3641348333 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f3fac0e2780 con 0x7f3fc4104100 2024-05-03T18:30:08.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.038+0000 7f3fc2800700 1 --2- 172.21.15.33:0/3641348333 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f3fa80b2a70 0x7f3fa80b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:08.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.038+0000 7f3fd370fb80 1 -- 172.21.15.33:0/939998704 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558caaaea2d0 msgr2=0x558caaae3fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:30:08.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.038+0000 7f3fd370fb80 1 --2- 172.21.15.33:0/939998704 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558caaaea2d0 0x558caaae3fb0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f3fb400f570 tx=0x7f3fb400f880 comp rx=0 tx=0).stop 2024-05-03T18:30:08.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.039+0000 7f3fd370fb80 1 -- 172.21.15.33:0/939998704 shutdown_connections 2024-05-03T18:30:08.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.039+0000 7f3fd370fb80 1 --2- 172.21.15.33:0/939998704 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558caaae45d0 0x558caaae4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:08.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.039+0000 7f3fd370fb80 1 --2- 172.21.15.33:0/939998704 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558caaae99b0 0x558caaae9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:08.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.039+0000 7f3fd370fb80 1 --2- 172.21.15.33:0/939998704 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558caaaea2d0 0x558caaae3fb0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:08.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.039+0000 7f3fd370fb80 1 -- 172.21.15.33:0/939998704 >> 172.21.15.33:0/939998704 conn(0x558caaa5c750 msgr2=0x558caaade980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:30:08.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.039+0000 7f3fd370fb80 1 -- 172.21.15.33:0/939998704 shutdown_connections 2024-05-03T18:30:08.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.039+0000 7f3fc2800700 1 --2- 172.21.15.33:0/3641348333 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f3fa80b2a70 0x7f3fa80b4f30 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc009f50 tx=0x7f3fbc005fb0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:30:08.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.039+0000 7f3fd370fb80 1 -- 172.21.15.33:0/939998704 wait complete. 2024-05-03T18:30:08.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.040+0000 7f3fd370fb80 1 -- start start 2024-05-03T18:30:08.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.040+0000 7f3fd370fb80 1 Processor -- start 2024-05-03T18:30:08.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.041+0000 7f3fd370fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558caaae45d0 0x558caaaefe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:08.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.041+0000 7f3fd370fb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558caaae99b0 0x558caaaed220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:08.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.041+0000 7f3fd370fb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558caaaea2d0 0x558caaaed760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:08.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.041+0000 7f3fd370fb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x558caaaf0d80 con 0x558caaae99b0 2024-05-03T18:30:08.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.041+0000 7f3fd370fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x558caa99cba0 con 0x558caaae45d0 2024-05-03T18:30:08.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.041+0000 7f3fd370fb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x558caaaf0f00 con 0x558caaaea2d0 2024-05-03T18:30:08.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.041+0000 7f3fd370fb80 10 client.0 _ll_register_callbacks cb 0x558caa9ddcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:30:08.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.041+0000 7f3fc1e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558caaae45d0 0x558caaaefe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:08.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.041+0000 7f3fc1e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558caaae45d0 0x558caaaefe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:34494/0 (socket says 172.21.15.33:34494) 2024-05-03T18:30:08.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.041+0000 7f3fc1e00700 1 -- 172.21.15.33:0/140019675 learned_addr learned my addr 172.21.15.33:0/140019675 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:30:08.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.042+0000 7f3fb9400700 1 --2- 172.21.15.33:0/140019675 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558caaae99b0 0x558caaaed220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:08.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.042+0000 7f3fc2800700 1 --2- 172.21.15.33:0/140019675 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558caaaea2d0 0x558caaaed760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:08.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.042+0000 7f3fc1e00700 1 -- 172.21.15.33:0/140019675 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558caaaea2d0 msgr2=0x558caaaed760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:30:08.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.042+0000 7f3fc1e00700 1 --2- 172.21.15.33:0/140019675 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558caaaea2d0 0x558caaaed760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:08.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.042+0000 7f3fc1e00700 1 -- 172.21.15.33:0/140019675 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558caaae99b0 msgr2=0x558caaaed220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:30:08.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.042+0000 7f3fc1e00700 1 --2- 172.21.15.33:0/140019675 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558caaae99b0 0x558caaaed220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:08.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.042+0000 7f3fc1e00700 1 -- 172.21.15.33:0/140019675 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558caa9fc4a0 con 0x558caaae45d0 2024-05-03T18:30:08.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.043+0000 7f3fc1e00700 1 --2- 172.21.15.33:0/140019675 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558caaae45d0 0x558caaaefe80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3fb4007cf0 tx=0x7f3fb4013600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b849aa1df75b50e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:30:08.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.043+0000 7f3fa7e00700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-03T18:30:08.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.043+0000 7f3fa7e00700 1 -- 172.21.15.33:0/140019675 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb402a060 con 0x558caaae45d0 2024-05-03T18:30:08.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.043+0000 7f3fd370fb80 1 -- 172.21.15.33:0/140019675 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3fb4014b10 con 0x558caaae45d0 2024-05-03T18:30:08.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.043+0000 7f3fa7e00700 1 -- 172.21.15.33:0/140019675 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f3fb4004cd0 con 0x558caaae45d0 2024-05-03T18:30:08.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.044+0000 7f3fa7e00700 1 -- 172.21.15.33:0/140019675 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f3fb4031050 con 0x558caaae45d0 2024-05-03T18:30:08.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.044+0000 7f3fd370fb80 10 client.35363 fetch_fsmap learned FSMap version 21 2024-05-03T18:30:08.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.044+0000 7f3fd370fb80 1 -- 172.21.15.33:0/140019675 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x558caaafb460 con 0x558caaae45d0 2024-05-03T18:30:08.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.044+0000 7f3fa7e00700 1 -- 172.21.15.33:0/140019675 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3fb4014b10 con 0x558caaae45d0 2024-05-03T18:30:08.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.045+0000 7f3fa7e00700 1 -- 172.21.15.33:0/140019675 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f3fb40054d0 con 0x558caaae45d0 2024-05-03T18:30:08.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.045+0000 7f3fd370fb80 10 client.35363 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:30:08.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.045+0000 7f3fd370fb80 10 client.35363 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:30:08.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.045+0000 7f3fd370fb80 20 client.35363 populate_metadata read hostname 'smithi033' 2024-05-03T18:30:08.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.045+0000 7f3fd370fb80 1 --2- 172.21.15.33:0/140019675 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x558caaafd990 0x558caaaffd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:08.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.045+0000 7f3fd370fb80 4 client.35363 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:30:08.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.045+0000 7f3fd370fb80 1 -- 172.21.15.33:0/140019675 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558caaab1070 con 0x558caaafd990 2024-05-03T18:30:08.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.046+0000 7f3fb9400700 1 --2- 172.21.15.33:0/140019675 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x558caaafd990 0x558caaaffd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:08.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.047+0000 7f3fb9400700 1 --2- 172.21.15.33:0/140019675 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x558caaafd990 0x558caaaffd80 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bdd6a9ed39047712 server_cookie=9489b754e8a1e17a in_seq=0 out_seq=0 2024-05-03T18:30:08.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.047+0000 7f3fa7e00700 10 client.35363 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:30:08.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.048+0000 7f3fa7e00700 1 -- 172.21.15.33:0/140019675 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558caaab1070 con 0x558caaafd990 2024-05-03T18:30:08.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.048+0000 7f3fa7e00700 10 client.35363 handle_command_reply: tid=0 2024-05-03T18:30:08.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.049+0000 7f3fd370fb80 1 client.35363 shutdown 2024-05-03T18:30:08.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.049+0000 7f3fd370fb80 1 -- 172.21.15.33:0/140019675 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558caaae45d0 msgr2=0x558caaaefe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:30:08.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.049+0000 7f3fd370fb80 1 --2- 172.21.15.33:0/140019675 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558caaae45d0 0x558caaaefe80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3fb4007cf0 tx=0x7f3fb4013600 comp rx=0 tx=0).stop 2024-05-03T18:30:08.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.049+0000 7f3fd370fb80 1 -- 172.21.15.33:0/140019675 shutdown_connections 2024-05-03T18:30:08.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.049+0000 7f3fd370fb80 1 --2- 172.21.15.33:0/140019675 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x558caaafd990 0x558caaaffd80 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:08.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.049+0000 7f3fd370fb80 1 --2- 172.21.15.33:0/140019675 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558caaaea2d0 0x558caaaed760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:08.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.049+0000 7f3fd370fb80 1 --2- 172.21.15.33:0/140019675 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558caaae45d0 0x558caaaefe80 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:08.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.049+0000 7f3fa7e00700 0 client.35363 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:30:08.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.049+0000 7f3fd370fb80 1 --2- 172.21.15.33:0/140019675 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558caaae99b0 0x558caaaed220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:08.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.049+0000 7f3fd370fb80 1 -- 172.21.15.33:0/140019675 >> 172.21.15.33:0/140019675 conn(0x558caaa5c750 msgr2=0x558caaad9430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:30:08.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.050+0000 7f3fd370fb80 1 -- 172.21.15.33:0/140019675 shutdown_connections 2024-05-03T18:30:08.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.050+0000 7f3fd370fb80 1 -- 172.21.15.33:0/140019675 wait complete. 2024-05-03T18:30:08.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.050+0000 7f3fd370fb80 20 client.35363 trim_cache size 0 max 16384 2024-05-03T18:30:08.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.082+0000 7f3fd370fb80 1 Processor -- start 2024-05-03T18:30:08.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.082+0000 7f3fd370fb80 1 -- start start 2024-05-03T18:30:08.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.082+0000 7f3fd370fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558caaae45d0 0x558caab053e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:08.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.082+0000 7f3fd370fb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558caaae99b0 0x558caab05920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:08.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.082+0000 7f3fd370fb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558caaaea2d0 0x558caaaf6210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:08.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.082+0000 7f3fd370fb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3fa000cd90 con 0x558caaaea2d0 2024-05-03T18:30:08.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.082+0000 7f3fd370fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3fa000f610 con 0x558caaae45d0 2024-05-03T18:30:08.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.082+0000 7f3fd370fb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3fa000d400 con 0x558caaae99b0 2024-05-03T18:30:08.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.083+0000 7f3fc2800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558caaaea2d0 0x558caaaf6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:08.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.083+0000 7f3fc2800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558caaaea2d0 0x558caaaf6210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43734/0 (socket says 172.21.15.33:43734) 2024-05-03T18:30:08.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.083+0000 7f3fc2800700 1 -- 172.21.15.33:0/136460419 learned_addr learned my addr 172.21.15.33:0/136460419 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:30:08.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.083+0000 7f3fb9400700 1 --2- 172.21.15.33:0/136460419 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558caaae99b0 0x558caab05920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:08.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.083+0000 7f3fc1e00700 1 --2- 172.21.15.33:0/136460419 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558caaae45d0 0x558caab053e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:08.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.083+0000 7f3fc2800700 1 -- 172.21.15.33:0/136460419 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558caaae99b0 msgr2=0x558caab05920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:30:08.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.083+0000 7f3fc2800700 1 --2- 172.21.15.33:0/136460419 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558caaae99b0 0x558caab05920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:08.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.083+0000 7f3fc2800700 1 -- 172.21.15.33:0/136460419 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558caaae45d0 msgr2=0x558caab053e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:30:08.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.083+0000 7f3fc2800700 1 --2- 172.21.15.33:0/136460419 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558caaae45d0 0x558caab053e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:08.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.083+0000 7f3fc2800700 1 -- 172.21.15.33:0/136460419 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558caaadf870 con 0x558caaaea2d0 2024-05-03T18:30:08.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.084+0000 7f3fc2800700 1 --2- 172.21.15.33:0/136460419 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558caaaea2d0 0x558caaaf6210 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc0099c0 tx=0x7f3fbc021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cccd80b45c4c56e server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:30:08.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.085+0000 7f3fb2a00700 1 -- 172.21.15.33:0/136460419 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fbc003c60 con 0x558caaaea2d0 2024-05-03T18:30:08.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.085+0000 7f3fb2a00700 1 -- 172.21.15.33:0/136460419 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3fbc035070 con 0x558caaaea2d0 2024-05-03T18:30:08.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.085+0000 7f3fb2a00700 1 -- 172.21.15.33:0/136460419 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fbc025420 con 0x558caaaea2d0 2024-05-03T18:30:08.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.086+0000 7f3fd370fb80 1 -- 172.21.15.33:0/136460419 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558caaaea2d0 msgr2=0x558caaaf6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:30:08.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.086+0000 7f3fd370fb80 1 --2- 172.21.15.33:0/136460419 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558caaaea2d0 0x558caaaf6210 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc0099c0 tx=0x7f3fbc021000 comp rx=0 tx=0).stop 2024-05-03T18:30:08.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.086+0000 7f3fd370fb80 1 -- 172.21.15.33:0/136460419 shutdown_connections 2024-05-03T18:30:08.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.086+0000 7f3fd370fb80 1 --2- 172.21.15.33:0/136460419 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558caaaea2d0 0x558caaaf6210 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:08.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.086+0000 7f3fd370fb80 1 --2- 172.21.15.33:0/136460419 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558caaae45d0 0x558caab053e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:08.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.086+0000 7f3fd370fb80 1 --2- 172.21.15.33:0/136460419 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558caaae99b0 0x558caab05920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:08.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.086+0000 7f3fd370fb80 1 -- 172.21.15.33:0/136460419 >> 172.21.15.33:0/136460419 conn(0x558caaa5c750 msgr2=0x558caaaebb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:30:08.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.086+0000 7f3fd370fb80 1 -- 172.21.15.33:0/136460419 shutdown_connections 2024-05-03T18:30:08.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.086+0000 7f3fd370fb80 1 -- 172.21.15.33:0/136460419 wait complete. 2024-05-03T18:30:08.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.087+0000 7f3fd370fb80 1 -- start start 2024-05-03T18:30:08.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.087+0000 7f3fd370fb80 1 Processor -- start 2024-05-03T18:30:08.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.088+0000 7f3fd370fb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558caaae45d0 0x558caab63830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:08.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.088+0000 7f3fd370fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558caaae99b0 0x558caab61f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:08.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.088+0000 7f3fd370fb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558caaaea2d0 0x558caab624a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:08.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.088+0000 7f3fd370fb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3fa000dad0 con 0x558caaae45d0 2024-05-03T18:30:08.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.088+0000 7f3fd370fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3fa000ef40 con 0x558caaae99b0 2024-05-03T18:30:08.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.088+0000 7f3fd370fb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3fa000e1a0 con 0x558caaaea2d0 2024-05-03T18:30:08.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.088+0000 7f3fd370fb80 10 client.0 _ll_register_callbacks cb 0x558caa9ddcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:30:08.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.088+0000 7f3fc1e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558caaae45d0 0x558caab63830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:08.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.088+0000 7f3fc1e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558caaae45d0 0x558caab63830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43748/0 (socket says 172.21.15.33:43748) 2024-05-03T18:30:08.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.088+0000 7f3fc1e00700 1 -- 172.21.15.33:0/2238741068 learned_addr learned my addr 172.21.15.33:0/2238741068 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:30:08.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.088+0000 7f3fb9400700 1 --2- 172.21.15.33:0/2238741068 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558caaae99b0 0x558caab61f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:08.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.088+0000 7f3fc2800700 1 --2- 172.21.15.33:0/2238741068 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558caaaea2d0 0x558caab624a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:08.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.089+0000 7f3fc1e00700 1 -- 172.21.15.33:0/2238741068 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558caaaea2d0 msgr2=0x558caab624a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:30:08.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.089+0000 7f3fc1e00700 1 --2- 172.21.15.33:0/2238741068 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558caaaea2d0 0x558caab624a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:08.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.089+0000 7f3fc1e00700 1 -- 172.21.15.33:0/2238741068 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558caaae99b0 msgr2=0x558caab61f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:30:08.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.089+0000 7f3fc1e00700 1 --2- 172.21.15.33:0/2238741068 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558caaae99b0 0x558caab61f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:08.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.089+0000 7f3fc1e00700 1 -- 172.21.15.33:0/2238741068 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558caab62ce0 con 0x558caaae45d0 2024-05-03T18:30:08.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.089+0000 7f3fc1e00700 1 --2- 172.21.15.33:0/2238741068 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558caaae45d0 0x558caab63830 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f3fb40059c0 tx=0x7f3fb4007c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5059efaa3fa4dfce server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:30:08.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.089+0000 7f3fa7e00700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:30:08.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.089+0000 7f3fa7e00700 1 -- 172.21.15.33:0/2238741068 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb403a060 con 0x558caaae45d0 2024-05-03T18:30:08.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.089+0000 7f3fd370fb80 1 -- 172.21.15.33:0/2238741068 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3fbc035070 con 0x558caaae45d0 2024-05-03T18:30:08.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.089+0000 7f3fa7e00700 1 -- 172.21.15.33:0/2238741068 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f3fb4014d10 con 0x558caaae45d0 2024-05-03T18:30:08.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.090+0000 7f3fa7e00700 1 -- 172.21.15.33:0/2238741068 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f3fb4041050 con 0x558caaae45d0 2024-05-03T18:30:08.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.090+0000 7f3fd370fb80 10 client.15744 fetch_fsmap learned FSMap version 21 2024-05-03T18:30:08.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.090+0000 7f3fd370fb80 1 -- 172.21.15.33:0/2238741068 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x558caaaf6250 con 0x558caaae45d0 2024-05-03T18:30:08.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.090+0000 7f3fa7e00700 1 -- 172.21.15.33:0/2238741068 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3fbc035070 con 0x558caaae45d0 2024-05-03T18:30:08.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.090+0000 7f3fa7e00700 1 -- 172.21.15.33:0/2238741068 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f3fb4034070 con 0x558caaae45d0 2024-05-03T18:30:08.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.090+0000 7f3fd370fb80 10 client.15744 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:30:08.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.091+0000 7f3fd370fb80 10 client.15744 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:30:08.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.091+0000 7f3fd370fb80 20 client.15744 populate_metadata read hostname 'smithi033' 2024-05-03T18:30:08.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.091+0000 7f3fd370fb80 1 --2- 172.21.15.33:0/2238741068 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x558caaaf6960 0x558caab7cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:08.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.091+0000 7f3fd370fb80 4 client.15744 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-03T18:30:08.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.091+0000 7f3fd370fb80 1 -- 172.21.15.33:0/2238741068 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x558caaa44480 con 0x558caaaf6960 2024-05-03T18:30:08.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.091+0000 7f3fb9400700 1 --2- 172.21.15.33:0/2238741068 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x558caaaf6960 0x558caab7cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:08.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.092+0000 7f3fb9400700 1 --2- 172.21.15.33:0/2238741068 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x558caaaf6960 0x558caab7cfc0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eba68b5bd7782073 server_cookie=e06bcef98a1738f5 in_seq=0 out_seq=0 2024-05-03T18:30:08.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.092+0000 7f3fa7e00700 10 client.15744 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:30:08.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.094+0000 7f3fa7e00700 1 -- 172.21.15.33:0/2238741068 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x558caaa44480 con 0x558caaaf6960 2024-05-03T18:30:08.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.094+0000 7f3fa7e00700 10 client.15744 handle_command_reply: tid=0 2024-05-03T18:30:08.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.094+0000 7f3fd370fb80 1 client.15744 shutdown 2024-05-03T18:30:08.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.095+0000 7f3fd370fb80 1 -- 172.21.15.33:0/2238741068 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558caaae45d0 msgr2=0x558caab63830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:30:08.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.095+0000 7f3fd370fb80 1 --2- 172.21.15.33:0/2238741068 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558caaae45d0 0x558caab63830 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f3fb40059c0 tx=0x7f3fb4007c00 comp rx=0 tx=0).stop 2024-05-03T18:30:08.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.095+0000 7f3fd370fb80 1 -- 172.21.15.33:0/2238741068 shutdown_connections 2024-05-03T18:30:08.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.095+0000 7f3fd370fb80 1 --2- 172.21.15.33:0/2238741068 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x558caaaf6960 0x558caab7cfc0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:08.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.095+0000 7f3fd370fb80 1 --2- 172.21.15.33:0/2238741068 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558caaaea2d0 0x558caab624a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:08.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.095+0000 7f3fd370fb80 1 --2- 172.21.15.33:0/2238741068 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558caaae45d0 0x558caab63830 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:08.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.095+0000 7f3fd370fb80 1 --2- 172.21.15.33:0/2238741068 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558caaae99b0 0x558caab61f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:08.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.095+0000 7f3fd370fb80 1 -- 172.21.15.33:0/2238741068 >> 172.21.15.33:0/2238741068 conn(0x558caaa5c750 msgr2=0x558caaaecdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:30:08.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.095+0000 7f3fa7e00700 0 client.15744 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:30:08.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.095+0000 7f3fd370fb80 1 -- 172.21.15.33:0/2238741068 shutdown_connections 2024-05-03T18:30:08.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.095+0000 7f3fd370fb80 1 -- 172.21.15.33:0/2238741068 wait complete. 2024-05-03T18:30:08.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.096+0000 7f3fd370fb80 20 client.15744 trim_cache size 0 max 16384 2024-05-03T18:30:08.101 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-03T18:30:08.101 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-05-03T18:30:08.101 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "266d8104-c15c-4240-8d23-f0f93dc0b7e5", 2024-05-03T18:30:08.101 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-05-03T18:30:08.101 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-03T18:30:08.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.096+0000 7f3fc8e00700 1 -- 172.21.15.33:0/3641348333 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f3fa80b2a70 msgr2=0x7f3fa80b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:30:08.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.096+0000 7f3fc8e00700 1 --2- 172.21.15.33:0/3641348333 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f3fa80b2a70 0x7f3fa80b4f30 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc009f50 tx=0x7f3fbc005fb0 comp rx=0 tx=0).stop 2024-05-03T18:30:08.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.096+0000 7f3fc8e00700 1 -- 172.21.15.33:0/3641348333 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3fc4104100 msgr2=0x7f3fc41a1c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:30:08.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.096+0000 7f3fc8e00700 1 --2- 172.21.15.33:0/3641348333 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3fc4104100 0x7f3fc41a1c70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3fac00c9b0 tx=0x7f3fac00ccc0 comp rx=0 tx=0).stop 2024-05-03T18:30:08.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.096+0000 7f3fc8e00700 1 -- 172.21.15.33:0/3641348333 shutdown_connections 2024-05-03T18:30:08.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.096+0000 7f3fc8e00700 1 --2- 172.21.15.33:0/3641348333 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fc4108850 0x7f3fc419b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:08.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.096+0000 7f3fc8e00700 1 --2- 172.21.15.33:0/3641348333 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f3fa80b2a70 0x7f3fa80b4f30 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:08.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.096+0000 7f3fc8e00700 1 --2- 172.21.15.33:0/3641348333 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3fc4103750 0x7f3fc41a1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:08.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.096+0000 7f3fc8e00700 1 --2- 172.21.15.33:0/3641348333 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3fc4104100 0x7f3fc41a1c70 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:08.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.097+0000 7f3fc8e00700 1 -- 172.21.15.33:0/3641348333 >> 172.21.15.33:0/3641348333 conn(0x7f3fc407a600 msgr2=0x7f3fc4101660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:30:08.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.097+0000 7f3fc8e00700 1 -- 172.21.15.33:0/3641348333 shutdown_connections 2024-05-03T18:30:08.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:08.097+0000 7f3fc8e00700 1 -- 172.21.15.33:0/3641348333 wait complete. 2024-05-03T18:30:09.032 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-03T18:30:09.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:30:08 smithi033 ceph-mon[28330]: pgmap v91: 129 pgs: 129 active+clean; 796 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 16 MiB/s wr, 30 op/s 2024-05-03T18:30:09.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:30:08 smithi033 ceph-mon[28330]: scrub queued for path: / 2024-05-03T18:30:09.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:30:08 smithi033 ceph-mon[28330]: scrub summary: idle+waiting paths [/] 2024-05-03T18:30:09.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:30:08 smithi033 ceph-mon[28330]: scrub summary: active paths [/] 2024-05-03T18:30:09.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:30:08 smithi099 ceph-mon[31658]: pgmap v91: 129 pgs: 129 active+clean; 796 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 16 MiB/s wr, 30 op/s 2024-05-03T18:30:09.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:30:08 smithi099 ceph-mon[31658]: scrub queued for path: / 2024-05-03T18:30:09.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:30:08 smithi099 ceph-mon[31658]: scrub summary: idle+waiting paths [/] 2024-05-03T18:30:09.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:30:08 smithi099 ceph-mon[31658]: scrub summary: active paths [/] 2024-05-03T18:30:09.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:30:08 smithi150 ceph-mon[32492]: pgmap v91: 129 pgs: 129 active+clean; 796 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 16 MiB/s wr, 30 op/s 2024-05-03T18:30:09.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:30:08 smithi150 ceph-mon[32492]: scrub queued for path: / 2024-05-03T18:30:09.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:30:08 smithi150 ceph-mon[32492]: scrub summary: idle+waiting paths [/] 2024-05-03T18:30:09.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:30:08 smithi150 ceph-mon[32492]: scrub summary: active paths [/] 2024-05-03T18:30:09.557 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:30:09.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:30:09 smithi033 ceph-mon[28330]: scrub summary: idle+waiting paths [/] 2024-05-03T18:30:10.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:30:09 smithi099 ceph-mon[31658]: scrub summary: idle+waiting paths [/] 2024-05-03T18:30:10.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:30:09 smithi150 ceph-mon[32492]: scrub summary: idle+waiting paths [/] 2024-05-03T18:30:11.073 INFO:teuthology.orchestra.run.smithi033.stderr:3100000 of 3200000 tuples (96%) done (elapsed 108.18 s, remaining 3.49 s) 2024-05-03T18:30:11.140 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:30:10 smithi033 ceph-mon[28330]: pgmap v92: 129 pgs: 129 active+clean; 796 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 11 MiB/s wr, 25 op/s 2024-05-03T18:30:11.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:30:10 smithi099 ceph-mon[31658]: pgmap v92: 129 pgs: 129 active+clean; 796 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 11 MiB/s wr, 25 op/s 2024-05-03T18:30:11.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:30:10 smithi150 ceph-mon[32492]: pgmap v92: 129 pgs: 129 active+clean; 796 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 11 MiB/s wr, 25 op/s 2024-05-03T18:30:12.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:30:11 smithi033 ceph-mon[28330]: scrub summary: idle 2024-05-03T18:30:12.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:30:11 smithi099 ceph-mon[31658]: scrub summary: idle 2024-05-03T18:30:12.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:30:11 smithi150 ceph-mon[32492]: scrub summary: idle 2024-05-03T18:30:12.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.970+0000 7f7019800700 1 -- 172.21.15.33:0/3413802276 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f701406b180 msgr2=0x7f701406b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:30:12.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.970+0000 7f7019800700 1 --2- 172.21.15.33:0/3413802276 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f701406b180 0x7f701406b600 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f7000009990 tx=0x7f7000009ca0 comp rx=0 tx=0).stop 2024-05-03T18:30:12.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.971+0000 7f7019800700 1 -- 172.21.15.33:0/3413802276 shutdown_connections 2024-05-03T18:30:12.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.971+0000 7f7019800700 1 --2- 172.21.15.33:0/3413802276 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f701406bb40 0x7f7014110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:12.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.971+0000 7f7019800700 1 --2- 172.21.15.33:0/3413802276 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7014109510 0x7f70141098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:12.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.971+0000 7f7019800700 1 --2- 172.21.15.33:0/3413802276 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f701406b180 0x7f701406b600 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:12.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.971+0000 7f7019800700 1 -- 172.21.15.33:0/3413802276 >> 172.21.15.33:0/3413802276 conn(0x7f7014079350 msgr2=0x7f7014079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:30:12.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.972+0000 7f7019800700 1 -- 172.21.15.33:0/3413802276 shutdown_connections 2024-05-03T18:30:12.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.972+0000 7f7019800700 1 -- 172.21.15.33:0/3413802276 wait complete. 2024-05-03T18:30:12.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.973+0000 7f7019800700 1 Processor -- start 2024-05-03T18:30:12.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.973+0000 7f7019800700 1 -- start start 2024-05-03T18:30:12.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.974+0000 7f7019800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f701406b180 0x7f701419be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:12.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.974+0000 7f7019800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f701406bb40 0x7f701419c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:12.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.974+0000 7f7019800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7014109510 0x7f70141a0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:12.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.974+0000 7f7019800700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7014113540 con 0x7f701406bb40 2024-05-03T18:30:12.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.974+0000 7f7019800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f70141133c0 con 0x7f701406b180 2024-05-03T18:30:12.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.974+0000 7f7019800700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f70141136c0 con 0x7f7014109510 2024-05-03T18:30:12.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.974+0000 7f7012000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f701406bb40 0x7f701419c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:12.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.974+0000 7f7012000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f701406bb40 0x7f701419c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43758/0 (socket says 172.21.15.33:43758) 2024-05-03T18:30:12.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.974+0000 7f7012000700 1 -- 172.21.15.33:0/190359902 learned_addr learned my addr 172.21.15.33:0/190359902 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:30:12.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.974+0000 7f7012a00700 1 --2- 172.21.15.33:0/190359902 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f701406b180 0x7f701419be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:12.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.975+0000 7f7013400700 1 --2- 172.21.15.33:0/190359902 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7014109510 0x7f70141a0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:12.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.975+0000 7f7012000700 1 -- 172.21.15.33:0/190359902 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7014109510 msgr2=0x7f70141a0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:30:12.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.975+0000 7f7012000700 1 --2- 172.21.15.33:0/190359902 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7014109510 0x7f70141a0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:12.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.975+0000 7f7012000700 1 -- 172.21.15.33:0/190359902 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f701406b180 msgr2=0x7f701419be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:30:12.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.975+0000 7f7012000700 1 --2- 172.21.15.33:0/190359902 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f701406b180 0x7f701419be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:12.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.975+0000 7f7012000700 1 -- 172.21.15.33:0/190359902 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70141a0e60 con 0x7f701406bb40 2024-05-03T18:30:12.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.975+0000 7f7012000700 1 --2- 172.21.15.33:0/190359902 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f701406bb40 0x7f701419c390 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f7000005910 tx=0x7f70000047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:30:12.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.976+0000 7f700b400700 1 -- 172.21.15.33:0/190359902 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7000027070 con 0x7f701406bb40 2024-05-03T18:30:12.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.976+0000 7f7019800700 1 -- 172.21.15.33:0/190359902 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70141a10f0 con 0x7f701406bb40 2024-05-03T18:30:12.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.976+0000 7f7019800700 1 -- 172.21.15.33:0/190359902 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70141a89d0 con 0x7f701406bb40 2024-05-03T18:30:12.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.976+0000 7f700b400700 1 -- 172.21.15.33:0/190359902 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f700000fc00 con 0x7f701406bb40 2024-05-03T18:30:12.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.976+0000 7f700b400700 1 -- 172.21.15.33:0/190359902 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7000022d40 con 0x7f701406bb40 2024-05-03T18:30:12.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.980+0000 7f700b400700 1 -- 172.21.15.33:0/190359902 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f700002a070 con 0x7f701406bb40 2024-05-03T18:30:12.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.981+0000 7f7024153b80 1 Processor -- start 2024-05-03T18:30:12.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.981+0000 7f7024153b80 1 -- start start 2024-05-03T18:30:12.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.981+0000 7f7024153b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55daf4fc2930 0x55daf4fc2d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:12.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.981+0000 7f7024153b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55daf4fc3250 0x55daf4fbcf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:12.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.981+0000 7f7024153b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55daf4fbd550 0x55daf4fbd9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:12.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.982+0000 7f7024153b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55daf4f0fa20 con 0x55daf4fc2930 2024-05-03T18:30:12.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.982+0000 7f7024153b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55daf4ead760 con 0x55daf4fbd550 2024-05-03T18:30:12.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.982+0000 7f7024153b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55daf4f0b9e0 con 0x55daf4fc3250 2024-05-03T18:30:12.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.982+0000 7f7013400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55daf4fc2930 0x55daf4fc2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:12.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.982+0000 7f7013400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55daf4fc2930 0x55daf4fc2d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43766/0 (socket says 172.21.15.33:43766) 2024-05-03T18:30:12.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.982+0000 7f7013400700 1 -- 172.21.15.33:0/1376078878 learned_addr learned my addr 172.21.15.33:0/1376078878 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:30:12.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.982+0000 7f7012000700 1 --2- 172.21.15.33:0/1376078878 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55daf4fbd550 0x55daf4fbd9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:12.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.982+0000 7f700b400700 1 --2- 172.21.15.33:0/190359902 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f6ff80b2a70 0x7f6ff80b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:12.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.982+0000 7f7012a00700 1 --2- 172.21.15.33:0/1376078878 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55daf4fc3250 0x55daf4fbcf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:12.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.983+0000 7f7013400700 1 -- 172.21.15.33:0/1376078878 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55daf4fc3250 msgr2=0x55daf4fbcf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:30:12.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.983+0000 7f7013400700 1 --2- 172.21.15.33:0/1376078878 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55daf4fc3250 0x55daf4fbcf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:12.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.983+0000 7f7013400700 1 -- 172.21.15.33:0/1376078878 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55daf4fbd550 msgr2=0x55daf4fbd9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:30:12.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.983+0000 7f7013400700 1 --2- 172.21.15.33:0/1376078878 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55daf4fbd550 0x55daf4fbd9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:12.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.983+0000 7f7013400700 1 -- 172.21.15.33:0/1376078878 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55daf4e5c8d0 con 0x55daf4fc2930 2024-05-03T18:30:12.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.983+0000 7f7013400700 1 --2- 172.21.15.33:0/190359902 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f6ff80b2a70 0x7f6ff80b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:12.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.983+0000 7f700b400700 1 -- 172.21.15.33:0/190359902 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f70000e3610 con 0x7f701406bb40 2024-05-03T18:30:12.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.984+0000 7f7013400700 1 --2- 172.21.15.33:0/1376078878 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55daf4fc2930 0x55daf4fc2d10 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f700c009f60 tx=0x7f700c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b303c6a4adc05ddd server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:30:12.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.984+0000 7f6fffe00700 1 -- 172.21.15.33:0/1376078878 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f700c023440 con 0x55daf4fc2930 2024-05-03T18:30:12.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.984+0000 7f6fffe00700 1 -- 172.21.15.33:0/1376078878 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f700c023a70 con 0x55daf4fc2930 2024-05-03T18:30:12.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.984+0000 7f6fffe00700 1 -- 172.21.15.33:0/1376078878 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f700c017750 con 0x55daf4fc2930 2024-05-03T18:30:12.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.985+0000 7f7024153b80 1 -- 172.21.15.33:0/1376078878 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55daf4fc2930 msgr2=0x55daf4fc2d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:30:12.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.985+0000 7f7024153b80 1 --2- 172.21.15.33:0/1376078878 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55daf4fc2930 0x55daf4fc2d10 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f700c009f60 tx=0x7f700c009f90 comp rx=0 tx=0).stop 2024-05-03T18:30:12.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.985+0000 7f7013400700 1 --2- 172.21.15.33:0/190359902 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f6ff80b2a70 0x7f6ff80b4f30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f700c020000 tx=0x7f700c01d000 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:30:12.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.985+0000 7f7024153b80 1 -- 172.21.15.33:0/1376078878 shutdown_connections 2024-05-03T18:30:12.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.985+0000 7f7024153b80 1 --2- 172.21.15.33:0/1376078878 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55daf4fbd550 0x55daf4fbd9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:12.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.985+0000 7f7024153b80 1 --2- 172.21.15.33:0/1376078878 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55daf4fc2930 0x55daf4fc2d10 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:12.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.985+0000 7f7024153b80 1 --2- 172.21.15.33:0/1376078878 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55daf4fc3250 0x55daf4fbcf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:12.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.985+0000 7f7024153b80 1 -- 172.21.15.33:0/1376078878 >> 172.21.15.33:0/1376078878 conn(0x55daf4f356d0 msgr2=0x55daf4fb7900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:30:12.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.985+0000 7f7024153b80 1 -- 172.21.15.33:0/1376078878 shutdown_connections 2024-05-03T18:30:12.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.985+0000 7f7024153b80 1 -- 172.21.15.33:0/1376078878 wait complete. 2024-05-03T18:30:12.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.986+0000 7f7024153b80 1 -- start start 2024-05-03T18:30:12.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.986+0000 7f7024153b80 1 Processor -- start 2024-05-03T18:30:12.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.987+0000 7f7024153b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55daf4fbd550 0x55daf4fc8d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:12.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.987+0000 7f7024153b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55daf4fc2930 0x55daf4fc9270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:12.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.987+0000 7f7024153b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55daf4fc3250 0x55daf4fc61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:12.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.987+0000 7f7024153b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55daf4fc9d00 con 0x55daf4fc3250 2024-05-03T18:30:12.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.987+0000 7f7024153b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55daf4e75b20 con 0x55daf4fbd550 2024-05-03T18:30:12.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.987+0000 7f7024153b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55daf4fc9e80 con 0x55daf4fc2930 2024-05-03T18:30:12.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.987+0000 7f7024153b80 10 client.0 _ll_register_callbacks cb 0x55daf4eb6c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:30:12.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.987+0000 7f7013400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55daf4fc3250 0x55daf4fc61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:12.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.988+0000 7f7013400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55daf4fc3250 0x55daf4fc61a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43776/0 (socket says 172.21.15.33:43776) 2024-05-03T18:30:12.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.988+0000 7f7013400700 1 -- 172.21.15.33:0/4221147699 learned_addr learned my addr 172.21.15.33:0/4221147699 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:30:12.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.988+0000 7f7012a00700 1 --2- 172.21.15.33:0/4221147699 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55daf4fbd550 0x55daf4fc8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:12.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.988+0000 7f7012000700 1 --2- 172.21.15.33:0/4221147699 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55daf4fc2930 0x55daf4fc9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:12.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.988+0000 7f7013400700 1 -- 172.21.15.33:0/4221147699 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55daf4fc2930 msgr2=0x55daf4fc9270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:30:12.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.988+0000 7f7013400700 1 --2- 172.21.15.33:0/4221147699 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55daf4fc2930 0x55daf4fc9270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:12.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.988+0000 7f7013400700 1 -- 172.21.15.33:0/4221147699 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55daf4fbd550 msgr2=0x55daf4fc8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:30:12.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.988+0000 7f7013400700 1 --2- 172.21.15.33:0/4221147699 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55daf4fbd550 0x55daf4fc8d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:12.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.988+0000 7f7013400700 1 -- 172.21.15.33:0/4221147699 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55daf4ed5420 con 0x55daf4fc3250 2024-05-03T18:30:12.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.989+0000 7f7013400700 1 --2- 172.21.15.33:0/4221147699 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55daf4fc3250 0x55daf4fc61a0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f700c017410 tx=0x7f700c017680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f441a3fc31ea7d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:30:12.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.989+0000 7f6ffcc00700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:30:12.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.989+0000 7f6ffcc00700 1 -- 172.21.15.33:0/4221147699 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f700c032060 con 0x55daf4fc3250 2024-05-03T18:30:12.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.989+0000 7f6ffcc00700 1 -- 172.21.15.33:0/4221147699 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f700c017a70 con 0x55daf4fc3250 2024-05-03T18:30:12.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.989+0000 7f7024153b80 1 -- 172.21.15.33:0/4221147699 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f700c023a70 con 0x55daf4fc3250 2024-05-03T18:30:12.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.989+0000 7f6ffcc00700 1 -- 172.21.15.33:0/4221147699 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f700c039050 con 0x55daf4fc3250 2024-05-03T18:30:12.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.989+0000 7f7024153b80 10 client.15756 fetch_fsmap learned FSMap version 21 2024-05-03T18:30:12.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.989+0000 7f7024153b80 1 -- 172.21.15.33:0/4221147699 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55daf4fd4360 con 0x55daf4fc3250 2024-05-03T18:30:12.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.990+0000 7f6ffcc00700 1 -- 172.21.15.33:0/4221147699 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f700c023a70 con 0x55daf4fc3250 2024-05-03T18:30:12.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.990+0000 7f6ffcc00700 1 -- 172.21.15.33:0/4221147699 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f700c015040 con 0x55daf4fc3250 2024-05-03T18:30:12.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.990+0000 7f7024153b80 10 client.15756 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:30:12.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.990+0000 7f7024153b80 10 client.15756 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:30:12.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.990+0000 7f7024153b80 20 client.15756 populate_metadata read hostname 'smithi033' 2024-05-03T18:30:12.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.990+0000 7f7024153b80 1 --2- 172.21.15.33:0/4221147699 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55daf4fd6830 0x55daf4fd8c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:12.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.990+0000 7f7024153b80 4 client.15756 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:30:12.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.990+0000 7f7024153b80 1 -- 172.21.15.33:0/4221147699 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55daf4f89ff0 con 0x55daf4fd6830 2024-05-03T18:30:12.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.991+0000 7f7012a00700 1 --2- 172.21.15.33:0/4221147699 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55daf4fd6830 0x55daf4fd8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:12.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.992+0000 7f7012a00700 1 --2- 172.21.15.33:0/4221147699 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55daf4fd6830 0x55daf4fd8c20 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c910456b723d724e server_cookie=2c51ae959bf3a879 in_seq=0 out_seq=0 2024-05-03T18:30:12.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.992+0000 7f6ffcc00700 10 client.15756 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:30:12.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.994+0000 7f6ffcc00700 1 -- 172.21.15.33:0/4221147699 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55daf4f89ff0 con 0x55daf4fd6830 2024-05-03T18:30:12.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.994+0000 7f6ffcc00700 10 client.15756 handle_command_reply: tid=0 2024-05-03T18:30:12.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.994+0000 7f7024153b80 1 client.15756 shutdown 2024-05-03T18:30:12.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.994+0000 7f7024153b80 1 -- 172.21.15.33:0/4221147699 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55daf4fc3250 msgr2=0x55daf4fc61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:30:12.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.994+0000 7f7024153b80 1 --2- 172.21.15.33:0/4221147699 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55daf4fc3250 0x55daf4fc61a0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f700c017410 tx=0x7f700c017680 comp rx=0 tx=0).stop 2024-05-03T18:30:12.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.994+0000 7f7024153b80 1 -- 172.21.15.33:0/4221147699 shutdown_connections 2024-05-03T18:30:12.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.994+0000 7f7024153b80 1 --2- 172.21.15.33:0/4221147699 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55daf4fd6830 0x55daf4fd8c20 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:12.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.994+0000 7f7024153b80 1 --2- 172.21.15.33:0/4221147699 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55daf4fc3250 0x55daf4fc61a0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:12.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.994+0000 7f7024153b80 1 --2- 172.21.15.33:0/4221147699 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55daf4fbd550 0x55daf4fc8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:12.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.994+0000 7f6ffcc00700 0 client.15756 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:30:12.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.994+0000 7f7024153b80 1 --2- 172.21.15.33:0/4221147699 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55daf4fc2930 0x55daf4fc9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:12.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.994+0000 7f7024153b80 1 -- 172.21.15.33:0/4221147699 >> 172.21.15.33:0/4221147699 conn(0x55daf4f356d0 msgr2=0x55daf4fb23b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:30:12.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.994+0000 7f7024153b80 1 -- 172.21.15.33:0/4221147699 shutdown_connections 2024-05-03T18:30:12.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.995+0000 7f7024153b80 1 -- 172.21.15.33:0/4221147699 wait complete. 2024-05-03T18:30:12.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:12.995+0000 7f7024153b80 20 client.15756 trim_cache size 0 max 16384 2024-05-03T18:30:13.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:13.025+0000 7f7024153b80 1 Processor -- start 2024-05-03T18:30:13.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:13.026+0000 7f7024153b80 1 -- start start 2024-05-03T18:30:13.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:13.026+0000 7f7024153b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55daf4f356d0 0x55daf4fbcf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:13.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:13.026+0000 7f7024153b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55daf4fbd550 0x55daf4fde390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:13.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:13.026+0000 7f7024153b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55daf4fc2930 0x55daf4fde8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:13.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:13.026+0000 7f7024153b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff400cd90 con 0x55daf4fbd550 2024-05-03T18:30:13.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:13.026+0000 7f7024153b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff400f610 con 0x55daf4fc2930 2024-05-03T18:30:13.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:13.026+0000 7f7024153b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff400d400 con 0x55daf4f356d0 2024-05-03T18:30:13.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:13.026+0000 7f7012000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55daf4fbd550 0x55daf4fde390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:13.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:13.027+0000 7f7012000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55daf4fbd550 0x55daf4fde390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43778/0 (socket says 172.21.15.33:43778) 2024-05-03T18:30:13.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:13.027+0000 7f7012000700 1 -- 172.21.15.33:0/243303136 learned_addr learned my addr 172.21.15.33:0/243303136 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:30:13.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:13.027+0000 7f7012a00700 1 --2- 172.21.15.33:0/243303136 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55daf4f356d0 0x55daf4fbcf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:13.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:13.027+0000 7f7013400700 1 --2- 172.21.15.33:0/243303136 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55daf4fc2930 0x55daf4fde8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:13.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:13.027+0000 7f7012000700 1 -- 172.21.15.33:0/243303136 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55daf4f356d0 msgr2=0x55daf4fbcf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:30:13.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:13.027+0000 7f7012000700 1 --2- 172.21.15.33:0/243303136 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55daf4f356d0 0x55daf4fbcf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:13.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:13.027+0000 7f7012000700 1 -- 172.21.15.33:0/243303136 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55daf4fc2930 msgr2=0x55daf4fde8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:30:13.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:13.027+0000 7f7012000700 1 --2- 172.21.15.33:0/243303136 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55daf4fc2930 0x55daf4fde8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:13.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:13.027+0000 7f7012000700 1 -- 172.21.15.33:0/243303136 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55daf4fc95c0 con 0x55daf4fbd550 2024-05-03T18:30:13.028 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:30:12 smithi033 ceph-mon[28330]: pgmap v93: 129 pgs: 129 active+clean; 796 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 11 MiB/s wr, 25 op/s 2024-05-03T18:30:13.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:13.028+0000 7f7012000700 1 --2- 172.21.15.33:0/243303136 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55daf4fbd550 0x55daf4fde390 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f70000aa290 tx=0x7f70000a6450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94fcf7fd04088a80 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:30:13.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:13.028+0000 7f6fff400700 1 -- 172.21.15.33:0/243303136 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70000a9d00 con 0x55daf4fbd550 2024-05-03T18:30:13.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:13.028+0000 7f6fff400700 1 -- 172.21.15.33:0/243303136 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f70000aa0a0 con 0x55daf4fbd550 2024-05-03T18:30:13.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:13.028+0000 7f6fff400700 1 -- 172.21.15.33:0/243303136 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70000aaa50 con 0x55daf4fbd550 2024-05-03T18:30:13.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:13.029+0000 7f7024153b80 1 -- 172.21.15.33:0/243303136 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55daf4fbd550 msgr2=0x55daf4fde390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:30:13.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:13.029+0000 7f7024153b80 1 --2- 172.21.15.33:0/243303136 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55daf4fbd550 0x55daf4fde390 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f70000aa290 tx=0x7f70000a6450 comp rx=0 tx=0).stop 2024-05-03T18:30:13.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:13.029+0000 7f7024153b80 1 -- 172.21.15.33:0/243303136 shutdown_connections 2024-05-03T18:30:13.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:13.029+0000 7f7024153b80 1 --2- 172.21.15.33:0/243303136 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55daf4fc2930 0x55daf4fde8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:13.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:13.029+0000 7f7024153b80 1 --2- 172.21.15.33:0/243303136 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55daf4f356d0 0x55daf4fbcf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:13.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:13.029+0000 7f7024153b80 1 --2- 172.21.15.33:0/243303136 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55daf4fbd550 0x55daf4fde390 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:13.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:13.029+0000 7f7024153b80 1 -- 172.21.15.33:0/243303136 >> 172.21.15.33:0/243303136 conn(0x55daf4fd8ef0 msgr2=0x55daf4fc49c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:30:13.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:13.030+0000 7f7024153b80 1 -- 172.21.15.33:0/243303136 shutdown_connections 2024-05-03T18:30:13.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:13.030+0000 7f7024153b80 1 -- 172.21.15.33:0/243303136 wait complete. 2024-05-03T18:30:13.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:13.030+0000 7f7024153b80 1 -- start start 2024-05-03T18:30:13.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:13.030+0000 7f7024153b80 1 Processor -- start 2024-05-03T18:30:13.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:13.031+0000 7f7024153b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55daf4f356d0 0x55daf4fcf170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:13.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:13.031+0000 7f7024153b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55daf4fbd550 0x55daf4fcf6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:13.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:13.031+0000 7f7024153b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55daf4fc2930 0x55daf4fcc140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:13.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:13.031+0000 7f7024153b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff400dad0 con 0x55daf4fc2930 2024-05-03T18:30:13.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:13.031+0000 7f7024153b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff400ef40 con 0x55daf4fbd550 2024-05-03T18:30:13.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:13.031+0000 7f7024153b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff400e1a0 con 0x55daf4f356d0 2024-05-03T18:30:13.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:13.031+0000 7f7024153b80 10 client.0 _ll_register_callbacks cb 0x55daf4eb6c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:30:13.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:13.032+0000 7f7013400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55daf4fc2930 0x55daf4fcc140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:13.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:13.032+0000 7f7013400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55daf4fc2930 0x55daf4fcc140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43792/0 (socket says 172.21.15.33:43792) 2024-05-03T18:30:13.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:13.032+0000 7f7013400700 1 -- 172.21.15.33:0/3581775290 learned_addr learned my addr 172.21.15.33:0/3581775290 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:30:13.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:13.032+0000 7f7012000700 1 --2- 172.21.15.33:0/3581775290 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55daf4fbd550 0x55daf4fcf6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:13.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:13.032+0000 7f7013400700 1 -- 172.21.15.33:0/3581775290 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55daf4f356d0 msgr2=0x55daf4fcf170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:30:13.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:13.032+0000 7f7013400700 1 --2- 172.21.15.33:0/3581775290 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55daf4f356d0 0x55daf4fcf170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:13.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:13.032+0000 7f7013400700 1 -- 172.21.15.33:0/3581775290 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55daf4fbd550 msgr2=0x55daf4fcf6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:30:13.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:13.032+0000 7f7013400700 1 --2- 172.21.15.33:0/3581775290 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55daf4fbd550 0x55daf4fcf6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:13.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:13.032+0000 7f7013400700 1 -- 172.21.15.33:0/3581775290 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55daf4fcc730 con 0x55daf4fc2930 2024-05-03T18:30:13.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:13.032+0000 7f7012000700 1 --2- 172.21.15.33:0/3581775290 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55daf4fbd550 0x55daf4fcf6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:30:13.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:13.033+0000 7f7013400700 1 --2- 172.21.15.33:0/3581775290 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55daf4fc2930 0x55daf4fcc140 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f700c017410 tx=0x7f700c009540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a6a26667457f710 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:30:13.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:13.033+0000 7f6ffcc00700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:30:13.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:13.033+0000 7f6ffcc00700 1 -- 172.21.15.33:0/3581775290 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f700c032030 con 0x55daf4fc2930 2024-05-03T18:30:13.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:13.033+0000 7f6ffcc00700 1 -- 172.21.15.33:0/3581775290 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f700c017a70 con 0x55daf4fc2930 2024-05-03T18:30:13.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:13.033+0000 7f7024153b80 1 -- 172.21.15.33:0/3581775290 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70000aa0a0 con 0x55daf4fc2930 2024-05-03T18:30:13.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:13.033+0000 7f6ffcc00700 1 -- 172.21.15.33:0/3581775290 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f700c045050 con 0x55daf4fc2930 2024-05-03T18:30:13.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:13.033+0000 7f7024153b80 10 client.15762 fetch_fsmap learned FSMap version 21 2024-05-03T18:30:13.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:13.033+0000 7f7024153b80 1 -- 172.21.15.33:0/3581775290 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55daf504a030 con 0x55daf4fc2930 2024-05-03T18:30:13.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:13.034+0000 7f6ffcc00700 1 -- 172.21.15.33:0/3581775290 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f70000aa0a0 con 0x55daf4fc2930 2024-05-03T18:30:13.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:13.034+0000 7f6ffcc00700 1 -- 172.21.15.33:0/3581775290 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f700c023440 con 0x55daf4fc2930 2024-05-03T18:30:13.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:13.034+0000 7f7024153b80 10 client.15762 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:30:13.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:13.034+0000 7f7024153b80 10 client.15762 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:30:13.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:13.034+0000 7f7024153b80 20 client.15762 populate_metadata read hostname 'smithi033' 2024-05-03T18:30:13.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:13.034+0000 7f7024153b80 1 --2- 172.21.15.33:0/3581775290 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55daf4fc3250 0x55daf504c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:13.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:13.034+0000 7f7024153b80 4 client.15762 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-03T18:30:13.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:13.034+0000 7f7024153b80 1 -- 172.21.15.33:0/3581775290 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55daf4f1d400 con 0x55daf4fc3250 2024-05-03T18:30:13.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:13.035+0000 7f7012a00700 1 --2- 172.21.15.33:0/3581775290 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55daf4fc3250 0x55daf504c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:13.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:13.036+0000 7f7012a00700 1 --2- 172.21.15.33:0/3581775290 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55daf4fc3250 0x55daf504c670 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8caab31c0332f42 server_cookie=919c771fa831ee82 in_seq=0 out_seq=0 2024-05-03T18:30:13.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:13.036+0000 7f6ffcc00700 10 client.15762 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:30:13.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:13.036+0000 7f6ffcc00700 1 -- 172.21.15.33:0/3581775290 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55daf4f1d400 con 0x55daf4fc3250 2024-05-03T18:30:13.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:13.036+0000 7f6ffcc00700 10 client.15762 handle_command_reply: tid=0 2024-05-03T18:30:13.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:13.037+0000 7f7024153b80 1 client.15762 shutdown 2024-05-03T18:30:13.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:13.037+0000 7f7024153b80 1 -- 172.21.15.33:0/3581775290 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55daf4fc2930 msgr2=0x55daf4fcc140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:30:13.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:13.037+0000 7f7024153b80 1 --2- 172.21.15.33:0/3581775290 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55daf4fc2930 0x55daf4fcc140 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f700c017410 tx=0x7f700c009540 comp rx=0 tx=0).stop 2024-05-03T18:30:13.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:13.037+0000 7f7024153b80 1 -- 172.21.15.33:0/3581775290 shutdown_connections 2024-05-03T18:30:13.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:13.037+0000 7f7024153b80 1 --2- 172.21.15.33:0/3581775290 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55daf4fc3250 0x55daf504c670 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:13.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:13.037+0000 7f7024153b80 1 --2- 172.21.15.33:0/3581775290 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55daf4fc2930 0x55daf4fcc140 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:13.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:13.037+0000 7f7024153b80 1 --2- 172.21.15.33:0/3581775290 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55daf4f356d0 0x55daf4fcf170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:13.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:13.037+0000 7f7024153b80 1 --2- 172.21.15.33:0/3581775290 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55daf4fbd550 0x55daf4fcf6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:13.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:13.037+0000 7f7024153b80 1 -- 172.21.15.33:0/3581775290 >> 172.21.15.33:0/3581775290 conn(0x55daf4fd8ef0 msgr2=0x55daf4fc3630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:30:13.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:13.037+0000 7f6ffcc00700 0 client.15762 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:30:13.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:13.038+0000 7f7024153b80 1 -- 172.21.15.33:0/3581775290 shutdown_connections 2024-05-03T18:30:13.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:13.038+0000 7f7024153b80 1 -- 172.21.15.33:0/3581775290 wait complete. 2024-05-03T18:30:13.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:13.038+0000 7f7024153b80 20 client.15762 trim_cache size 0 max 16384 2024-05-03T18:30:13.039 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-03T18:30:13.039 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-05-03T18:30:13.040 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-05-03T18:30:13.040 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-03T18:30:13.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:13.039+0000 7f7019800700 1 -- 172.21.15.33:0/190359902 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f6ff80b2a70 msgr2=0x7f6ff80b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:30:13.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:13.039+0000 7f7019800700 1 --2- 172.21.15.33:0/190359902 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f6ff80b2a70 0x7f6ff80b4f30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f700c020000 tx=0x7f700c01d000 comp rx=0 tx=0).stop 2024-05-03T18:30:13.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:13.039+0000 7f7019800700 1 -- 172.21.15.33:0/190359902 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f701406bb40 msgr2=0x7f701419c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:30:13.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:13.039+0000 7f7019800700 1 --2- 172.21.15.33:0/190359902 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f701406bb40 0x7f701419c390 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f7000005910 tx=0x7f70000047b0 comp rx=0 tx=0).stop 2024-05-03T18:30:13.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:13.039+0000 7f7019800700 1 -- 172.21.15.33:0/190359902 shutdown_connections 2024-05-03T18:30:13.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:13.039+0000 7f7019800700 1 --2- 172.21.15.33:0/190359902 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7014109510 0x7f70141a0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:13.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:13.039+0000 7f7019800700 1 --2- 172.21.15.33:0/190359902 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f6ff80b2a70 0x7f6ff80b4f30 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:13.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:13.039+0000 7f7019800700 1 --2- 172.21.15.33:0/190359902 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f701406b180 0x7f701419be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:13.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:13.039+0000 7f7019800700 1 --2- 172.21.15.33:0/190359902 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f701406bb40 0x7f701419c390 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:13.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:13.039+0000 7f7019800700 1 -- 172.21.15.33:0/190359902 >> 172.21.15.33:0/190359902 conn(0x7f7014079350 msgr2=0x7f701410e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:30:13.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:13.040+0000 7f7019800700 1 -- 172.21.15.33:0/190359902 shutdown_connections 2024-05-03T18:30:13.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:13.040+0000 7f7019800700 1 -- 172.21.15.33:0/190359902 wait complete. 2024-05-03T18:30:13.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:30:12 smithi099 ceph-mon[31658]: pgmap v93: 129 pgs: 129 active+clean; 796 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 11 MiB/s wr, 25 op/s 2024-05-03T18:30:13.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:30:12 smithi150 ceph-mon[32492]: pgmap v93: 129 pgs: 129 active+clean; 796 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 11 MiB/s wr, 25 op/s 2024-05-03T18:30:13.888 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-03T18:30:13.889 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T18:30:14.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:30:13 smithi033 ceph-mon[28330]: pgmap v94: 129 pgs: 129 active+clean; 832 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 17 MiB/s wr, 30 op/s 2024-05-03T18:30:14.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:30:13 smithi099 ceph-mon[31658]: pgmap v94: 129 pgs: 129 active+clean; 832 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 17 MiB/s wr, 30 op/s 2024-05-03T18:30:14.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:30:13 smithi150 ceph-mon[32492]: pgmap v94: 129 pgs: 129 active+clean; 832 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 17 MiB/s wr, 30 op/s 2024-05-03T18:30:14.393 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:30:14.663 INFO:teuthology.orchestra.run.smithi033.stderr:3200000 of 3200000 tuples (100%) done (elapsed 111.77 s, remaining 0.00 s) 2024-05-03T18:30:14.982 INFO:teuthology.orchestra.run.smithi033.stderr:vacuuming... 2024-05-03T18:30:16.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:16.968+0000 7f8549400700 1 -- 172.21.15.33:0/952460239 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8544109510 msgr2=0x7f85441098f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:30:16.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:16.968+0000 7f8549400700 1 --2- 172.21.15.33:0/952460239 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8544109510 0x7f85441098f0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f8534009990 tx=0x7f8534009ca0 comp rx=0 tx=0).stop 2024-05-03T18:30:16.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:16.969+0000 7f8549400700 1 -- 172.21.15.33:0/952460239 shutdown_connections 2024-05-03T18:30:16.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:16.969+0000 7f8549400700 1 --2- 172.21.15.33:0/952460239 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f854406bb40 0x7f8544110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:16.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:16.969+0000 7f8549400700 1 --2- 172.21.15.33:0/952460239 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8544109510 0x7f85441098f0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:16.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:16.969+0000 7f8549400700 1 --2- 172.21.15.33:0/952460239 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f854406b180 0x7f854406b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:16.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:16.969+0000 7f8549400700 1 -- 172.21.15.33:0/952460239 >> 172.21.15.33:0/952460239 conn(0x7f8544079350 msgr2=0x7f8544079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:30:16.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:16.969+0000 7f8549400700 1 -- 172.21.15.33:0/952460239 shutdown_connections 2024-05-03T18:30:16.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:16.969+0000 7f8549400700 1 -- 172.21.15.33:0/952460239 wait complete. 2024-05-03T18:30:16.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:16.970+0000 7f8549400700 1 Processor -- start 2024-05-03T18:30:16.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:16.970+0000 7f8549400700 1 -- start start 2024-05-03T18:30:16.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:16.971+0000 7f8549400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f854406b180 0x7f85441a1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:16.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:16.971+0000 7f8549400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f854406bb40 0x7f85441a1e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:16.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:16.971+0000 7f8549400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8544109510 0x7f854419bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:16.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:16.971+0000 7f8549400700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8544113540 con 0x7f854406bb40 2024-05-03T18:30:16.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:16.971+0000 7f8549400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f85441133c0 con 0x7f854406b180 2024-05-03T18:30:16.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:16.971+0000 7f8549400700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f85441136c0 con 0x7f8544109510 2024-05-03T18:30:16.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:16.971+0000 7f8541e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f854406bb40 0x7f85441a1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:16.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:16.971+0000 7f8541e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f854406bb40 0x7f85441a1e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33500/0 (socket says 172.21.15.33:33500) 2024-05-03T18:30:16.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:16.971+0000 7f8541e00700 1 -- 172.21.15.33:0/406043726 learned_addr learned my addr 172.21.15.33:0/406043726 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:30:16.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:16.971+0000 7f8543200700 1 --2- 172.21.15.33:0/406043726 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8544109510 0x7f854419bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:16.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:16.972+0000 7f8542800700 1 --2- 172.21.15.33:0/406043726 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f854406b180 0x7f85441a1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:16.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:16.972+0000 7f8541e00700 1 -- 172.21.15.33:0/406043726 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8544109510 msgr2=0x7f854419bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:30:16.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:16.972+0000 7f8541e00700 1 --2- 172.21.15.33:0/406043726 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8544109510 0x7f854419bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:16.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:16.972+0000 7f8541e00700 1 -- 172.21.15.33:0/406043726 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f854406b180 msgr2=0x7f85441a1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:30:16.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:16.972+0000 7f8541e00700 1 --2- 172.21.15.33:0/406043726 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f854406b180 0x7f85441a1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:16.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:16.972+0000 7f8541e00700 1 -- 172.21.15.33:0/406043726 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f854419c240 con 0x7f854406bb40 2024-05-03T18:30:16.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:16.972+0000 7f8541e00700 1 --2- 172.21.15.33:0/406043726 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f854406bb40 0x7f85441a1e60 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f853000d980 tx=0x7f853000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:30:16.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:16.973+0000 7f853b400700 1 -- 172.21.15.33:0/406043726 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8530017070 con 0x7f854406bb40 2024-05-03T18:30:16.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:16.973+0000 7f853b400700 1 -- 172.21.15.33:0/406043726 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8530005020 con 0x7f854406bb40 2024-05-03T18:30:16.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:16.973+0000 7f8549400700 1 -- 172.21.15.33:0/406043726 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f854419c530 con 0x7f854406bb40 2024-05-03T18:30:16.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:16.973+0000 7f853b400700 1 -- 172.21.15.33:0/406043726 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f853000f980 con 0x7f854406bb40 2024-05-03T18:30:16.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:16.973+0000 7f8549400700 1 -- 172.21.15.33:0/406043726 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85441a87f0 con 0x7f854406bb40 2024-05-03T18:30:16.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:16.975+0000 7f8549400700 1 -- 172.21.15.33:0/406043726 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f854406a190 con 0x7f854406bb40 2024-05-03T18:30:16.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:16.980+0000 7f853b400700 1 -- 172.21.15.33:0/406043726 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f8530004a30 con 0x7f854406bb40 2024-05-03T18:30:16.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:16.983+0000 7f853b400700 1 --2- 172.21.15.33:0/406043726 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f85280b2a70 0x7f85280b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:16.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:16.983+0000 7f853b400700 1 -- 172.21.15.33:0/406043726 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f85300e2c00 con 0x7f854406bb40 2024-05-03T18:30:16.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:16.983+0000 7f8542800700 1 --2- 172.21.15.33:0/406043726 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f85280b2a70 0x7f85280b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:16.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:16.984+0000 7f853b400700 1 -- 172.21.15.33:0/406043726 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f8530020020 con 0x7f854406bb40 2024-05-03T18:30:16.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:16.984+0000 7f8542800700 1 --2- 172.21.15.33:0/406043726 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f85280b2a70 0x7f85280b4f30 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8534006010 tx=0x7f853400b540 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:30:17.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:30:16 smithi033 ceph-mon[28330]: pgmap v95: 129 pgs: 129 active+clean; 832 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 10 op/s 2024-05-03T18:30:17.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:30:16 smithi099 ceph-mon[31658]: pgmap v95: 129 pgs: 129 active+clean; 832 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 10 op/s 2024-05-03T18:30:17.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:17.335+0000 7f8549400700 1 -- 172.21.15.33:0/406043726 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f854419cfd0 con 0x7f854406bb40 2024-05-03T18:30:17.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:17.336+0000 7f853b400700 1 -- 172.21.15.33:0/406043726 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11801 (secure 0 0 0) 0x7f854419cfd0 con 0x7f854406bb40 2024-05-03T18:30:17.337 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:30:17.337 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1327393503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1327393503},{"type":"v1","addr":"172.21.15.33:6835","nonce":1327393503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/2461472948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":2461472948},{"type":"v1","addr":"172.21.15.33:6837","nonce":2461472948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14559,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/442269281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":442269281},{"type":"v1","addr":"172.21.15.33:6839","nonce":442269281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14571,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/645571756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":645571756},{"type":"v1","addr":"172.21.15.33:6841","nonce":645571756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/203598909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":203598909},{"type":"v1","addr":"172.21.15.150:6833","nonce":203598909}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/4029842493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":4029842493},{"type":"v1","addr":"172.21.15.150:6835","nonce":4029842493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3463664423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3463664423},{"type":"v1","addr":"172.21.15.150:6837","nonce":3463664423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24481,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6839/4051648018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":4051648018},{"type":"v1","addr":"172.21.15.150:6839","nonce":4051648018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34418,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1865105479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1865105479},{"type":"v1","addr":"172.21.15.99:6833","nonce":1865105479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34445,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/1206393914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1206393914},{"type":"v1","addr":"172.21.15.99:6835","nonce":1206393914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-03T18:22:28.837091+0000","modified":"2024-05-03T18:23:49.758398+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34508},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34472":{"gid":34472,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6837/3360970342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3360970342},{"type":"v1","addr":"172.21.15.99:6837","nonce":3360970342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34508":{"gid":34508,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/919907308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":919907308},{"type":"v1","addr":"172.21.15.99:6839","nonce":919907308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34508,"qdb_cluster":[34508]},"id":1}]} 2024-05-03T18:30:17.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:17.340+0000 7f8549400700 1 -- 172.21.15.33:0/406043726 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f85280b2a70 msgr2=0x7f85280b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:30:17.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:17.340+0000 7f8549400700 1 --2- 172.21.15.33:0/406043726 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f85280b2a70 0x7f85280b4f30 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8534006010 tx=0x7f853400b540 comp rx=0 tx=0).stop 2024-05-03T18:30:17.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:17.340+0000 7f8549400700 1 -- 172.21.15.33:0/406043726 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f854406bb40 msgr2=0x7f85441a1e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:30:17.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:17.340+0000 7f8549400700 1 --2- 172.21.15.33:0/406043726 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f854406bb40 0x7f85441a1e60 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f853000d980 tx=0x7f853000dc90 comp rx=0 tx=0).stop 2024-05-03T18:30:17.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:17.341+0000 7f8549400700 1 -- 172.21.15.33:0/406043726 shutdown_connections 2024-05-03T18:30:17.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:17.341+0000 7f8549400700 1 --2- 172.21.15.33:0/406043726 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8544109510 0x7f854419bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:17.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:17.341+0000 7f8549400700 1 --2- 172.21.15.33:0/406043726 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f85280b2a70 0x7f85280b4f30 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:17.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:17.341+0000 7f8549400700 1 --2- 172.21.15.33:0/406043726 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f854406b180 0x7f85441a1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:17.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:17.341+0000 7f8549400700 1 --2- 172.21.15.33:0/406043726 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f854406bb40 0x7f85441a1e60 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:17.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:17.341+0000 7f8549400700 1 -- 172.21.15.33:0/406043726 >> 172.21.15.33:0/406043726 conn(0x7f8544079350 msgr2=0x7f854410e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:30:17.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:17.341+0000 7f8549400700 1 -- 172.21.15.33:0/406043726 shutdown_connections 2024-05-03T18:30:17.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:17.341+0000 7f8549400700 1 -- 172.21.15.33:0/406043726 wait complete. 2024-05-03T18:30:17.342 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 21 2024-05-03T18:30:17.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:30:16 smithi150 ceph-mon[32492]: pgmap v95: 129 pgs: 129 active+clean; 832 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 10 op/s 2024-05-03T18:30:17.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:30:17 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/406043726' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:30:18.271 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-03T18:30:18.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:30:17 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/406043726' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:30:18.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:30:17 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/406043726' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:30:18.757 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:30:19.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:30:18 smithi033 ceph-mon[28330]: pgmap v96: 129 pgs: 129 active+clean; 872 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 16 MiB/s wr, 18 op/s 2024-05-03T18:30:19.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:30:18 smithi099 ceph-mon[31658]: pgmap v96: 129 pgs: 129 active+clean; 872 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 16 MiB/s wr, 18 op/s 2024-05-03T18:30:19.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:30:18 smithi150 ceph-mon[32492]: pgmap v96: 129 pgs: 129 active+clean; 872 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 16 MiB/s wr, 18 op/s 2024-05-03T18:30:21.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.170+0000 7f4448a00700 1 -- 172.21.15.33:0/1004273011 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4444108850 msgr2=0x7f44441103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:30:21.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.170+0000 7f4448a00700 1 --2- 172.21.15.33:0/1004273011 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4444108850 0x7f44441103d0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f443c00b0d0 tx=0x7f443c00b3e0 comp rx=0 tx=0).stop 2024-05-03T18:30:21.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.170+0000 7f4448a00700 1 -- 172.21.15.33:0/1004273011 shutdown_connections 2024-05-03T18:30:21.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.170+0000 7f4448a00700 1 --2- 172.21.15.33:0/1004273011 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4444108850 0x7f44441103d0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:21.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.170+0000 7f4448a00700 1 --2- 172.21.15.33:0/1004273011 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4444103750 0x7f4444103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:21.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.170+0000 7f4448a00700 1 --2- 172.21.15.33:0/1004273011 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4444104100 0x7f4444108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:21.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.170+0000 7f4448a00700 1 -- 172.21.15.33:0/1004273011 >> 172.21.15.33:0/1004273011 conn(0x7f444407a600 msgr2=0x7f444407aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:30:21.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.170+0000 7f4448a00700 1 -- 172.21.15.33:0/1004273011 shutdown_connections 2024-05-03T18:30:21.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.170+0000 7f4448a00700 1 -- 172.21.15.33:0/1004273011 wait complete. 2024-05-03T18:30:21.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.172+0000 7f4448a00700 1 Processor -- start 2024-05-03T18:30:21.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.172+0000 7f4448a00700 1 -- start start 2024-05-03T18:30:21.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.172+0000 7f4448a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4444103750 0x7f44441a1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:21.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.172+0000 7f4448a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4444104100 0x7f44441a1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:21.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.172+0000 7f4448a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4444108850 0x7f444419b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:21.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.173+0000 7f4448a00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4444113520 con 0x7f4444108850 2024-05-03T18:30:21.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.173+0000 7f4448a00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f44441133a0 con 0x7f4444103750 2024-05-03T18:30:21.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.173+0000 7f4448a00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f44441136a0 con 0x7f4444104100 2024-05-03T18:30:21.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.173+0000 7f4442800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4444108850 0x7f444419b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:21.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.173+0000 7f4442800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4444108850 0x7f444419b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33516/0 (socket says 172.21.15.33:33516) 2024-05-03T18:30:21.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.173+0000 7f4442800700 1 -- 172.21.15.33:0/2618490618 learned_addr learned my addr 172.21.15.33:0/2618490618 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:30:21.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.173+0000 7f4441e00700 1 --2- 172.21.15.33:0/2618490618 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4444103750 0x7f44441a1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:21.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.173+0000 7f4441400700 1 --2- 172.21.15.33:0/2618490618 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4444104100 0x7f44441a1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:21.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.173+0000 7f4442800700 1 -- 172.21.15.33:0/2618490618 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4444104100 msgr2=0x7f44441a1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:30:21.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.173+0000 7f4442800700 1 --2- 172.21.15.33:0/2618490618 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4444104100 0x7f44441a1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:21.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.173+0000 7f4442800700 1 -- 172.21.15.33:0/2618490618 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4444103750 msgr2=0x7f44441a1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:30:21.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.173+0000 7f4442800700 1 --2- 172.21.15.33:0/2618490618 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4444103750 0x7f44441a1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:21.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.174+0000 7f4442800700 1 -- 172.21.15.33:0/2618490618 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f444419c100 con 0x7f4444108850 2024-05-03T18:30:21.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.174+0000 7f4441e00700 1 --2- 172.21.15.33:0/2618490618 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4444103750 0x7f44441a1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:30:21.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.174+0000 7f4442800700 1 --2- 172.21.15.33:0/2618490618 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4444108850 0x7f444419b9a0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f443c007d10 tx=0x7f443c007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:30:21.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.174+0000 7f443aa00700 1 -- 172.21.15.33:0/2618490618 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f443c01f070 con 0x7f4444108850 2024-05-03T18:30:21.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.174+0000 7f443aa00700 1 -- 172.21.15.33:0/2618490618 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f443c003e10 con 0x7f4444108850 2024-05-03T18:30:21.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.174+0000 7f443aa00700 1 -- 172.21.15.33:0/2618490618 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f443c01b820 con 0x7f4444108850 2024-05-03T18:30:21.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.174+0000 7f4448a00700 1 -- 172.21.15.33:0/2618490618 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f444419c390 con 0x7f4444108850 2024-05-03T18:30:21.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.175+0000 7f4448a00700 1 -- 172.21.15.33:0/2618490618 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44441a8660 con 0x7f4444108850 2024-05-03T18:30:21.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.178+0000 7f443aa00700 1 -- 172.21.15.33:0/2618490618 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f443c022070 con 0x7f4444108850 2024-05-03T18:30:21.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.180+0000 7f44533a9b80 1 Processor -- start 2024-05-03T18:30:21.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.180+0000 7f44533a9b80 1 -- start start 2024-05-03T18:30:21.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.180+0000 7f443aa00700 1 --2- 172.21.15.33:0/2618490618 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f44280b29f0 0x7f44280b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:21.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.180+0000 7f44533a9b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555b748ac7b0 0x555b748acb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:21.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.180+0000 7f44533a9b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555b748ad0d0 0x555b748a7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:21.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.181+0000 7f44533a9b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b748a85d0 0x555b748a8a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:21.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.181+0000 7f44533a9b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x555b747faa20 con 0x555b748a85d0 2024-05-03T18:30:21.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.181+0000 7f44533a9b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x555b74798760 con 0x555b748ac7b0 2024-05-03T18:30:21.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.181+0000 7f44533a9b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x555b747f69e0 con 0x555b748ad0d0 2024-05-03T18:30:21.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.181+0000 7f443aa00700 1 -- 172.21.15.33:0/2618490618 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f443c00f020 con 0x7f4444108850 2024-05-03T18:30:21.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.181+0000 7f4442800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b748a85d0 0x555b748a8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:21.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.181+0000 7f4441400700 1 --2- 172.21.15.33:0/2618490618 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f44280b29f0 0x7f44280b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:21.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.181+0000 7f4442800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b748a85d0 0x555b748a8a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33528/0 (socket says 172.21.15.33:33528) 2024-05-03T18:30:21.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.181+0000 7f4442800700 1 -- 172.21.15.33:0/1750809001 learned_addr learned my addr 172.21.15.33:0/1750809001 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:30:21.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.181+0000 7f4441400700 1 --2- 172.21.15.33:0/1750809001 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555b748ad0d0 0x555b748a7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:21.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.182+0000 7f4442800700 1 -- 172.21.15.33:0/1750809001 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555b748ad0d0 msgr2=0x555b748a7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:30:21.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.182+0000 7f4442800700 1 --2- 172.21.15.33:0/1750809001 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555b748ad0d0 0x555b748a7fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:21.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.182+0000 7f4442800700 1 -- 172.21.15.33:0/1750809001 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555b748ac7b0 msgr2=0x555b748acb90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-03T18:30:21.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.182+0000 7f4442800700 1 --2- 172.21.15.33:0/1750809001 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555b748ac7b0 0x555b748acb90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:21.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.182+0000 7f4442800700 1 -- 172.21.15.33:0/1750809001 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b747478d0 con 0x555b748a85d0 2024-05-03T18:30:21.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.182+0000 7f4442800700 1 --2- 172.21.15.33:0/1750809001 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b748a85d0 0x555b748a8a30 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f443c0a6f60 tx=0x7f443c0aa650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b37565c6f3212df server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:30:21.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.182+0000 7f442f400700 1 -- 172.21.15.33:0/1750809001 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f443c0aaee0 con 0x555b748a85d0 2024-05-03T18:30:21.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.182+0000 7f4441400700 1 --2- 172.21.15.33:0/2618490618 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f44280b29f0 0x7f44280b4eb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4430005ea0 tx=0x7f443000a2b0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:30:21.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.182+0000 7f442f400700 1 -- 172.21.15.33:0/1750809001 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f443c0af290 con 0x555b748a85d0 2024-05-03T18:30:21.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.183+0000 7f442f400700 1 -- 172.21.15.33:0/1750809001 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f443c0e1540 con 0x555b748a85d0 2024-05-03T18:30:21.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.183+0000 7f44533a9b80 1 -- 172.21.15.33:0/1750809001 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b748a85d0 msgr2=0x555b748a8a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:30:21.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.183+0000 7f44533a9b80 1 --2- 172.21.15.33:0/1750809001 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b748a85d0 0x555b748a8a30 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f443c0a6f60 tx=0x7f443c0aa650 comp rx=0 tx=0).stop 2024-05-03T18:30:21.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.184+0000 7f44533a9b80 1 -- 172.21.15.33:0/1750809001 shutdown_connections 2024-05-03T18:30:21.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.184+0000 7f44533a9b80 1 --2- 172.21.15.33:0/1750809001 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b748a85d0 0x555b748a8a30 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:21.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.184+0000 7f44533a9b80 1 --2- 172.21.15.33:0/1750809001 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555b748ac7b0 0x555b748acb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:21.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.184+0000 7f44533a9b80 1 --2- 172.21.15.33:0/1750809001 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555b748ad0d0 0x555b748a7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:21.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.184+0000 7f44533a9b80 1 -- 172.21.15.33:0/1750809001 >> 172.21.15.33:0/1750809001 conn(0x555b747d6b60 msgr2=0x555b748a27f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:30:21.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.184+0000 7f44533a9b80 1 -- 172.21.15.33:0/1750809001 shutdown_connections 2024-05-03T18:30:21.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.184+0000 7f44533a9b80 1 -- 172.21.15.33:0/1750809001 wait complete. 2024-05-03T18:30:21.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.185+0000 7f44533a9b80 1 -- start start 2024-05-03T18:30:21.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.185+0000 7f44533a9b80 1 Processor -- start 2024-05-03T18:30:21.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.185+0000 7f44533a9b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555b748a85d0 0x555b748b3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:21.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.185+0000 7f44533a9b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555b748ac7b0 0x555b748b1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:21.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.185+0000 7f44533a9b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b748ad0d0 0x555b748b1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:21.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.185+0000 7f44533a9b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x555b748b4a60 con 0x555b748ad0d0 2024-05-03T18:30:21.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.185+0000 7f44533a9b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x555b74760b20 con 0x555b748ac7b0 2024-05-03T18:30:21.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.185+0000 7f44533a9b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x555b748b4be0 con 0x555b748a85d0 2024-05-03T18:30:21.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.186+0000 7f44533a9b80 10 client.0 _ll_register_callbacks cb 0x555b747a1c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:30:21.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.186+0000 7f4442800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b748ad0d0 0x555b748b1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:21.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.186+0000 7f4442800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b748ad0d0 0x555b748b1740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33534/0 (socket says 172.21.15.33:33534) 2024-05-03T18:30:21.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.186+0000 7f4442800700 1 -- 172.21.15.33:0/1030148047 learned_addr learned my addr 172.21.15.33:0/1030148047 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:30:21.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.186+0000 7f4441e00700 1 --2- 172.21.15.33:0/1030148047 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555b748a85d0 0x555b748b3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:21.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.186+0000 7f4441400700 1 --2- 172.21.15.33:0/1030148047 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555b748ac7b0 0x555b748b1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:21.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.186+0000 7f4442800700 1 -- 172.21.15.33:0/1030148047 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555b748a85d0 msgr2=0x555b748b3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:30:21.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.186+0000 7f4442800700 1 --2- 172.21.15.33:0/1030148047 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555b748a85d0 0x555b748b3e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:21.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.186+0000 7f4442800700 1 -- 172.21.15.33:0/1030148047 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555b748ac7b0 msgr2=0x555b748b1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:30:21.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.186+0000 7f4442800700 1 --2- 172.21.15.33:0/1030148047 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555b748ac7b0 0x555b748b1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:21.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.186+0000 7f4442800700 1 -- 172.21.15.33:0/1030148047 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b747c0420 con 0x555b748ad0d0 2024-05-03T18:30:21.188 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:30:20 smithi033 ceph-mon[28330]: pgmap v97: 129 pgs: 129 active+clean; 872 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 11 MiB/s wr, 13 op/s 2024-05-03T18:30:21.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.187+0000 7f4442800700 1 --2- 172.21.15.33:0/1030148047 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b748ad0d0 0x555b748b1740 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f443c0aab90 tx=0x7f443c0aaf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7391b1e5f627dd46 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:30:21.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.187+0000 7f4423e00700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:30:21.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.187+0000 7f4423e00700 1 -- 172.21.15.33:0/1030148047 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f443c0e1750 con 0x555b748ad0d0 2024-05-03T18:30:21.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.187+0000 7f4423e00700 1 -- 172.21.15.33:0/1030148047 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f443c0ab340 con 0x555b748ad0d0 2024-05-03T18:30:21.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.187+0000 7f44533a9b80 1 -- 172.21.15.33:0/1030148047 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f443c0af290 con 0x555b748ad0d0 2024-05-03T18:30:21.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.187+0000 7f4423e00700 1 -- 172.21.15.33:0/1030148047 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f443c0b3820 con 0x555b748ad0d0 2024-05-03T18:30:21.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.187+0000 7f44533a9b80 10 client.15780 fetch_fsmap learned FSMap version 21 2024-05-03T18:30:21.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.187+0000 7f44533a9b80 1 -- 172.21.15.33:0/1030148047 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x555b748b4a60 con 0x555b748ad0d0 2024-05-03T18:30:21.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.188+0000 7f4423e00700 1 -- 172.21.15.33:0/1030148047 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f443c0af290 con 0x555b748ad0d0 2024-05-03T18:30:21.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.188+0000 7f4423e00700 1 -- 172.21.15.33:0/1030148047 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f443c011070 con 0x555b748ad0d0 2024-05-03T18:30:21.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.188+0000 7f44533a9b80 10 client.15780 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:30:21.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.188+0000 7f44533a9b80 10 client.15780 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:30:21.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.188+0000 7f44533a9b80 20 client.15780 populate_metadata read hostname 'smithi033' 2024-05-03T18:30:21.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.188+0000 7f44533a9b80 1 --2- 172.21.15.33:0/1030148047 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x555b748c14b0 0x555b748c38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:21.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.188+0000 7f44533a9b80 4 client.15780 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:30:21.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.188+0000 7f44533a9b80 1 -- 172.21.15.33:0/1030148047 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555b748751d0 con 0x555b748c14b0 2024-05-03T18:30:21.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.189+0000 7f4441e00700 1 --2- 172.21.15.33:0/1030148047 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x555b748c14b0 0x555b748c38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:21.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.190+0000 7f4441e00700 1 --2- 172.21.15.33:0/1030148047 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x555b748c14b0 0x555b748c38a0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22f4d220624bc467 server_cookie=85b7cc5534b2e446 in_seq=0 out_seq=0 2024-05-03T18:30:21.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.190+0000 7f4423e00700 10 client.15780 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:30:21.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.192+0000 7f4423e00700 1 -- 172.21.15.33:0/1030148047 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555b748751d0 con 0x555b748c14b0 2024-05-03T18:30:21.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.192+0000 7f4423e00700 10 client.15780 handle_command_reply: tid=0 2024-05-03T18:30:21.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.192+0000 7f44533a9b80 1 client.15780 shutdown 2024-05-03T18:30:21.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.192+0000 7f44533a9b80 1 -- 172.21.15.33:0/1030148047 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b748ad0d0 msgr2=0x555b748b1740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:30:21.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.192+0000 7f44533a9b80 1 --2- 172.21.15.33:0/1030148047 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b748ad0d0 0x555b748b1740 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f443c0aab90 tx=0x7f443c0aaf90 comp rx=0 tx=0).stop 2024-05-03T18:30:21.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.192+0000 7f44533a9b80 1 -- 172.21.15.33:0/1030148047 shutdown_connections 2024-05-03T18:30:21.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.192+0000 7f44533a9b80 1 --2- 172.21.15.33:0/1030148047 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x555b748c14b0 0x555b748c38a0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:21.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.192+0000 7f44533a9b80 1 --2- 172.21.15.33:0/1030148047 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b748ad0d0 0x555b748b1740 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:21.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.192+0000 7f44533a9b80 1 --2- 172.21.15.33:0/1030148047 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555b748a85d0 0x555b748b3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:21.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.192+0000 7f44533a9b80 1 --2- 172.21.15.33:0/1030148047 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555b748ac7b0 0x555b748b1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:21.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.192+0000 7f4423e00700 0 client.15780 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:30:21.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.192+0000 7f44533a9b80 1 -- 172.21.15.33:0/1030148047 >> 172.21.15.33:0/1030148047 conn(0x555b747d6b60 msgr2=0x555b748adde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:30:21.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.193+0000 7f44533a9b80 1 -- 172.21.15.33:0/1030148047 shutdown_connections 2024-05-03T18:30:21.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.193+0000 7f44533a9b80 1 -- 172.21.15.33:0/1030148047 wait complete. 2024-05-03T18:30:21.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.193+0000 7f44533a9b80 20 client.15780 trim_cache size 0 max 16384 2024-05-03T18:30:21.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.217+0000 7f44533a9b80 1 Processor -- start 2024-05-03T18:30:21.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.217+0000 7f44533a9b80 1 -- start start 2024-05-03T18:30:21.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.218+0000 7f44533a9b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555b748a85d0 0x555b748bda00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:21.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.218+0000 7f44533a9b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b748ac7b0 0x555b748a89d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:21.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.218+0000 7f44533a9b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555b748ad0d0 0x555b748a2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:21.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.218+0000 7f44533a9b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f44441af3c0 con 0x555b748ac7b0 2024-05-03T18:30:21.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.218+0000 7f44533a9b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f44441b1c40 con 0x555b748a85d0 2024-05-03T18:30:21.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.218+0000 7f44533a9b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f44441afa30 con 0x555b748ad0d0 2024-05-03T18:30:21.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.218+0000 7f4441400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b748ac7b0 0x555b748a89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:21.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.218+0000 7f4441400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b748ac7b0 0x555b748a89d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33548/0 (socket says 172.21.15.33:33548) 2024-05-03T18:30:21.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.218+0000 7f4441400700 1 -- 172.21.15.33:0/919954722 learned_addr learned my addr 172.21.15.33:0/919954722 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:30:21.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.219+0000 7f4442800700 1 --2- 172.21.15.33:0/919954722 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555b748ad0d0 0x555b748a2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:21.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.219+0000 7f4441e00700 1 --2- 172.21.15.33:0/919954722 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555b748a85d0 0x555b748bda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:21.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.219+0000 7f4441400700 1 -- 172.21.15.33:0/919954722 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555b748ad0d0 msgr2=0x555b748a2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:30:21.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.219+0000 7f4441400700 1 --2- 172.21.15.33:0/919954722 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555b748ad0d0 0x555b748a2660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:21.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.219+0000 7f4441400700 1 -- 172.21.15.33:0/919954722 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555b748a85d0 msgr2=0x555b748bda00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:30:21.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.219+0000 7f4441400700 1 --2- 172.21.15.33:0/919954722 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555b748a85d0 0x555b748bda00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:21.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.219+0000 7f4441400700 1 -- 172.21.15.33:0/919954722 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b7489d7a0 con 0x555b748ac7b0 2024-05-03T18:30:21.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.219+0000 7f4441400700 1 --2- 172.21.15.33:0/919954722 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b748ac7b0 0x555b748a89d0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f4430013fc0 tx=0x7f4430012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb23d2fded55c492 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:30:21.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.220+0000 7f442ea00700 1 -- 172.21.15.33:0/919954722 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f443001a070 con 0x555b748ac7b0 2024-05-03T18:30:21.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.220+0000 7f442ea00700 1 -- 172.21.15.33:0/919954722 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4430012d20 con 0x555b748ac7b0 2024-05-03T18:30:21.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.220+0000 7f442ea00700 1 -- 172.21.15.33:0/919954722 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4430005560 con 0x555b748ac7b0 2024-05-03T18:30:21.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.220+0000 7f44533a9b80 1 -- 172.21.15.33:0/919954722 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b748ac7b0 msgr2=0x555b748a89d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:30:21.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.220+0000 7f44533a9b80 1 --2- 172.21.15.33:0/919954722 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b748ac7b0 0x555b748a89d0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f4430013fc0 tx=0x7f4430012000 comp rx=0 tx=0).stop 2024-05-03T18:30:21.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.221+0000 7f44533a9b80 1 -- 172.21.15.33:0/919954722 shutdown_connections 2024-05-03T18:30:21.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.221+0000 7f44533a9b80 1 --2- 172.21.15.33:0/919954722 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555b748ad0d0 0x555b748a2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:21.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.221+0000 7f44533a9b80 1 --2- 172.21.15.33:0/919954722 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555b748a85d0 0x555b748bda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:21.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.221+0000 7f44533a9b80 1 --2- 172.21.15.33:0/919954722 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b748ac7b0 0x555b748a89d0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:21.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.221+0000 7f44533a9b80 1 -- 172.21.15.33:0/919954722 >> 172.21.15.33:0/919954722 conn(0x555b747d6b60 msgr2=0x555b748abdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:30:21.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.221+0000 7f44533a9b80 1 -- 172.21.15.33:0/919954722 shutdown_connections 2024-05-03T18:30:21.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.221+0000 7f44533a9b80 1 -- 172.21.15.33:0/919954722 wait complete. 2024-05-03T18:30:21.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.221+0000 7f44533a9b80 1 -- start start 2024-05-03T18:30:21.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.222+0000 7f44533a9b80 1 Processor -- start 2024-05-03T18:30:21.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.222+0000 7f44533a9b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555b748a85d0 0x555b748ba340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:21.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.222+0000 7f44533a9b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555b748ac7b0 0x555b748a57f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:21.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.222+0000 7f44533a9b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b748ad0d0 0x555b748a5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:21.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.222+0000 7f44533a9b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f44441b0100 con 0x555b748ad0d0 2024-05-03T18:30:21.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.222+0000 7f44533a9b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f44441b1570 con 0x555b748ac7b0 2024-05-03T18:30:21.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.222+0000 7f44533a9b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f44441b07d0 con 0x555b748a85d0 2024-05-03T18:30:21.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.222+0000 7f44533a9b80 10 client.0 _ll_register_callbacks cb 0x555b747a1c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:30:21.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.223+0000 7f4442800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b748ad0d0 0x555b748a5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:21.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.223+0000 7f4442800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b748ad0d0 0x555b748a5db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33556/0 (socket says 172.21.15.33:33556) 2024-05-03T18:30:21.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.223+0000 7f4442800700 1 -- 172.21.15.33:0/3983230535 learned_addr learned my addr 172.21.15.33:0/3983230535 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:30:21.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.223+0000 7f4441400700 1 --2- 172.21.15.33:0/3983230535 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555b748ac7b0 0x555b748a57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:21.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.223+0000 7f4441e00700 1 --2- 172.21.15.33:0/3983230535 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555b748a85d0 0x555b748ba340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:21.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.224+0000 7f4442800700 1 -- 172.21.15.33:0/3983230535 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555b748a85d0 msgr2=0x555b748ba340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:30:21.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.224+0000 7f4442800700 1 --2- 172.21.15.33:0/3983230535 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555b748a85d0 0x555b748ba340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:21.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.224+0000 7f4442800700 1 -- 172.21.15.33:0/3983230535 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555b748ac7b0 msgr2=0x555b748a57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:30:21.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.224+0000 7f4442800700 1 --2- 172.21.15.33:0/3983230535 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555b748ac7b0 0x555b748a57f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:21.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.224+0000 7f4442800700 1 -- 172.21.15.33:0/3983230535 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b748a65f0 con 0x555b748ad0d0 2024-05-03T18:30:21.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.224+0000 7f4441400700 1 --2- 172.21.15.33:0/3983230535 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555b748ac7b0 0x555b748a57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:30:21.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.224+0000 7f4442800700 1 --2- 172.21.15.33:0/3983230535 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b748ad0d0 0x555b748a5db0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f443c0aa690 tx=0x7f443c0aa950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e1409e752deb0b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:30:21.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.224+0000 7f4423e00700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:30:21.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.225+0000 7f4423e00700 1 -- 172.21.15.33:0/3983230535 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f443c0e1750 con 0x555b748ad0d0 2024-05-03T18:30:21.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.225+0000 7f4423e00700 1 -- 172.21.15.33:0/3983230535 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f443c0ab340 con 0x555b748ad0d0 2024-05-03T18:30:21.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.225+0000 7f44533a9b80 1 -- 172.21.15.33:0/3983230535 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4430012d20 con 0x555b748ad0d0 2024-05-03T18:30:21.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.225+0000 7f4423e00700 1 -- 172.21.15.33:0/3983230535 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f443c0af290 con 0x555b748ad0d0 2024-05-03T18:30:21.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.225+0000 7f44533a9b80 10 client.15786 fetch_fsmap learned FSMap version 21 2024-05-03T18:30:21.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.225+0000 7f44533a9b80 1 -- 172.21.15.33:0/3983230535 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x555b748a65f0 con 0x555b748ad0d0 2024-05-03T18:30:21.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.226+0000 7f4423e00700 1 -- 172.21.15.33:0/3983230535 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4430012d20 con 0x555b748ad0d0 2024-05-03T18:30:21.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.226+0000 7f4423e00700 1 -- 172.21.15.33:0/3983230535 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f443c011040 con 0x555b748ad0d0 2024-05-03T18:30:21.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.226+0000 7f44533a9b80 10 client.15786 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:30:21.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.226+0000 7f44533a9b80 10 client.15786 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:30:21.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.226+0000 7f44533a9b80 20 client.15786 populate_metadata read hostname 'smithi033' 2024-05-03T18:30:21.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.226+0000 7f44533a9b80 1 --2- 172.21.15.33:0/3983230535 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x555b74934800 0x555b74936c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:21.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.226+0000 7f44533a9b80 4 client.15786 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-03T18:30:21.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.226+0000 7f44533a9b80 1 -- 172.21.15.33:0/3983230535 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555b74808400 con 0x555b74934800 2024-05-03T18:30:21.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.227+0000 7f4441e00700 1 --2- 172.21.15.33:0/3983230535 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x555b74934800 0x555b74936c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:21.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.228+0000 7f4441e00700 1 --2- 172.21.15.33:0/3983230535 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x555b74934800 0x555b74936c20 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5aa1b0f5f04c6cac server_cookie=fe64e797abb19105 in_seq=0 out_seq=0 2024-05-03T18:30:21.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.228+0000 7f4423e00700 10 client.15786 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:30:21.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.229+0000 7f4423e00700 1 -- 172.21.15.33:0/3983230535 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555b74808400 con 0x555b74934800 2024-05-03T18:30:21.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.229+0000 7f4423e00700 10 client.15786 handle_command_reply: tid=0 2024-05-03T18:30:21.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.229+0000 7f44533a9b80 1 client.15786 shutdown 2024-05-03T18:30:21.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.229+0000 7f44533a9b80 1 -- 172.21.15.33:0/3983230535 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b748ad0d0 msgr2=0x555b748a5db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:30:21.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.229+0000 7f44533a9b80 1 --2- 172.21.15.33:0/3983230535 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b748ad0d0 0x555b748a5db0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f443c0aa690 tx=0x7f443c0aa950 comp rx=0 tx=0).stop 2024-05-03T18:30:21.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.229+0000 7f44533a9b80 1 -- 172.21.15.33:0/3983230535 shutdown_connections 2024-05-03T18:30:21.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.229+0000 7f44533a9b80 1 --2- 172.21.15.33:0/3983230535 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x555b74934800 0x555b74936c20 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:21.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.229+0000 7f44533a9b80 1 --2- 172.21.15.33:0/3983230535 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b748ad0d0 0x555b748a5db0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:21.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.229+0000 7f44533a9b80 1 --2- 172.21.15.33:0/3983230535 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555b748a85d0 0x555b748ba340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:21.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.229+0000 7f44533a9b80 1 --2- 172.21.15.33:0/3983230535 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555b748ac7b0 0x555b748a57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:21.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.229+0000 7f44533a9b80 1 -- 172.21.15.33:0/3983230535 >> 172.21.15.33:0/3983230535 conn(0x555b747d6b60 msgr2=0x555b748afab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:30:21.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.229+0000 7f4423e00700 0 client.15786 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:30:21.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.230+0000 7f44533a9b80 1 -- 172.21.15.33:0/3983230535 shutdown_connections 2024-05-03T18:30:21.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.230+0000 7f44533a9b80 1 -- 172.21.15.33:0/3983230535 wait complete. 2024-05-03T18:30:21.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.230+0000 7f44533a9b80 20 client.15786 trim_cache size 0 max 16384 2024-05-03T18:30:21.231 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-03T18:30:21.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.231+0000 7f4448a00700 1 -- 172.21.15.33:0/2618490618 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f44280b29f0 msgr2=0x7f44280b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:30:21.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.231+0000 7f4448a00700 1 --2- 172.21.15.33:0/2618490618 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f44280b29f0 0x7f44280b4eb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4430005ea0 tx=0x7f443000a2b0 comp rx=0 tx=0).stop 2024-05-03T18:30:21.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.231+0000 7f4448a00700 1 -- 172.21.15.33:0/2618490618 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4444108850 msgr2=0x7f444419b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:30:21.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.231+0000 7f4448a00700 1 --2- 172.21.15.33:0/2618490618 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4444108850 0x7f444419b9a0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f443c007d10 tx=0x7f443c007df0 comp rx=0 tx=0).stop 2024-05-03T18:30:21.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.231+0000 7f4448a00700 1 -- 172.21.15.33:0/2618490618 shutdown_connections 2024-05-03T18:30:21.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.231+0000 7f4448a00700 1 --2- 172.21.15.33:0/2618490618 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4444108850 0x7f444419b9a0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:21.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.231+0000 7f4448a00700 1 --2- 172.21.15.33:0/2618490618 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f44280b29f0 0x7f44280b4eb0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:21.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.231+0000 7f4448a00700 1 --2- 172.21.15.33:0/2618490618 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4444103750 0x7f44441a1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:21.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.231+0000 7f4448a00700 1 --2- 172.21.15.33:0/2618490618 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4444104100 0x7f44441a1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:21.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.231+0000 7f4448a00700 1 -- 172.21.15.33:0/2618490618 >> 172.21.15.33:0/2618490618 conn(0x7f444407a600 msgr2=0x7f4444101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:30:21.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.232+0000 7f4448a00700 1 -- 172.21.15.33:0/2618490618 shutdown_connections 2024-05-03T18:30:21.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:21.232+0000 7f4448a00700 1 -- 172.21.15.33:0/2618490618 wait complete. 2024-05-03T18:30:21.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:30:20 smithi099 ceph-mon[31658]: pgmap v97: 129 pgs: 129 active+clean; 872 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 11 MiB/s wr, 13 op/s 2024-05-03T18:30:21.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:30:20 smithi150 ceph-mon[32492]: pgmap v97: 129 pgs: 129 active+clean; 872 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 11 MiB/s wr, 13 op/s 2024-05-03T18:30:23.206 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-03T18:30:23.206 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-03T18:30:23.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:30:22 smithi033 ceph-mon[28330]: pgmap v98: 129 pgs: 129 active+clean; 872 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 11 MiB/s wr, 13 op/s 2024-05-03T18:30:23.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:30:22 smithi099 ceph-mon[31658]: pgmap v98: 129 pgs: 129 active+clean; 872 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 11 MiB/s wr, 13 op/s 2024-05-03T18:30:23.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:30:22 smithi150 ceph-mon[32492]: pgmap v98: 129 pgs: 129 active+clean; 872 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 11 MiB/s wr, 13 op/s 2024-05-03T18:30:23.741 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:30:25.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:30:24 smithi033 ceph-mon[28330]: pgmap v99: 129 pgs: 129 active+clean; 932 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 16 MiB/s wr, 19 op/s 2024-05-03T18:30:25.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:30:24 smithi099 ceph-mon[31658]: pgmap v99: 129 pgs: 129 active+clean; 932 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 16 MiB/s wr, 19 op/s 2024-05-03T18:30:25.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:30:24 smithi150 ceph-mon[32492]: pgmap v99: 129 pgs: 129 active+clean; 932 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 16 MiB/s wr, 19 op/s 2024-05-03T18:30:26.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.279+0000 7f8a43e00700 1 -- 172.21.15.33:0/232683873 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a44104eb0 msgr2=0x7f8a4410d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:30:26.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.279+0000 7f8a43e00700 1 --2- 172.21.15.33:0/232683873 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a44104eb0 0x7f8a4410d3a0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f8a30009990 tx=0x7f8a30009ca0 comp rx=0 tx=0).stop 2024-05-03T18:30:26.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.279+0000 7f8a43e00700 1 -- 172.21.15.33:0/232683873 shutdown_connections 2024-05-03T18:30:26.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.279+0000 7f8a43e00700 1 --2- 172.21.15.33:0/232683873 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a4410d9c0 0x7f8a4410fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:26.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.279+0000 7f8a43e00700 1 --2- 172.21.15.33:0/232683873 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a44104590 0x7f8a44104970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:26.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.279+0000 7f8a43e00700 1 --2- 172.21.15.33:0/232683873 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a44104eb0 0x7f8a4410d3a0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:26.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.279+0000 7f8a43e00700 1 -- 172.21.15.33:0/232683873 >> 172.21.15.33:0/232683873 conn(0x7f8a440fddc0 msgr2=0x7f8a441001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:30:26.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.279+0000 7f8a43e00700 1 -- 172.21.15.33:0/232683873 shutdown_connections 2024-05-03T18:30:26.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.280+0000 7f8a43e00700 1 -- 172.21.15.33:0/232683873 wait complete. 2024-05-03T18:30:26.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.281+0000 7f8a43e00700 1 Processor -- start 2024-05-03T18:30:26.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.281+0000 7f8a43e00700 1 -- start start 2024-05-03T18:30:26.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.281+0000 7f8a43e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a44104590 0x7f8a441a18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:26.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.281+0000 7f8a43e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a44104eb0 0x7f8a441a1de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:26.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.281+0000 7f8a43e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a4410d9c0 0x7f8a4419ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:26.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.281+0000 7f8a43e00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8a44113460 con 0x7f8a44104590 2024-05-03T18:30:26.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.281+0000 7f8a43e00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8a441132e0 con 0x7f8a44104eb0 2024-05-03T18:30:26.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.281+0000 7f8a43e00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8a441135e0 con 0x7f8a4410d9c0 2024-05-03T18:30:26.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.282+0000 7f8a42a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a44104590 0x7f8a441a18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:26.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.282+0000 7f8a42a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a44104590 0x7f8a441a18a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33730/0 (socket says 172.21.15.33:33730) 2024-05-03T18:30:26.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.282+0000 7f8a42a00700 1 -- 172.21.15.33:0/679769951 learned_addr learned my addr 172.21.15.33:0/679769951 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:30:26.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.282+0000 7f8a42000700 1 --2- 172.21.15.33:0/679769951 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a44104eb0 0x7f8a441a1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:26.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.282+0000 7f8a43400700 1 --2- 172.21.15.33:0/679769951 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a4410d9c0 0x7f8a4419ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:26.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.283+0000 7f8a42a00700 1 -- 172.21.15.33:0/679769951 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a4410d9c0 msgr2=0x7f8a4419ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:30:26.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.283+0000 7f8a42a00700 1 --2- 172.21.15.33:0/679769951 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a4410d9c0 0x7f8a4419ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:26.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.283+0000 7f8a42a00700 1 -- 172.21.15.33:0/679769951 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a44104eb0 msgr2=0x7f8a441a1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:30:26.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.283+0000 7f8a42a00700 1 --2- 172.21.15.33:0/679769951 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a44104eb0 0x7f8a441a1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:26.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.283+0000 7f8a42a00700 1 -- 172.21.15.33:0/679769951 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a4419c1c0 con 0x7f8a44104590 2024-05-03T18:30:26.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.283+0000 7f8a42a00700 1 --2- 172.21.15.33:0/679769951 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a44104590 0x7f8a441a18a0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f8a3400d980 tx=0x7f8a3400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:30:26.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.283+0000 7f8a48a00700 1 -- 172.21.15.33:0/679769951 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a34017070 con 0x7f8a44104590 2024-05-03T18:30:26.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.283+0000 7f8a43e00700 1 -- 172.21.15.33:0/679769951 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a4419c4b0 con 0x7f8a44104590 2024-05-03T18:30:26.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.283+0000 7f8a48a00700 1 -- 172.21.15.33:0/679769951 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8a34005020 con 0x7f8a44104590 2024-05-03T18:30:26.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.284+0000 7f8a48a00700 1 -- 172.21.15.33:0/679769951 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a3400f980 con 0x7f8a44104590 2024-05-03T18:30:26.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.284+0000 7f8a43e00700 1 -- 172.21.15.33:0/679769951 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a441a8770 con 0x7f8a44104590 2024-05-03T18:30:26.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.287+0000 7f8a48a00700 1 -- 172.21.15.33:0/679769951 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f8a34004a30 con 0x7f8a44104590 2024-05-03T18:30:26.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.289+0000 7f8a48a00700 1 --2- 172.21.15.33:0/679769951 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f8a280b2a70 0x7f8a280b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:26.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.289+0000 7f8a54786b80 1 Processor -- start 2024-05-03T18:30:26.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.289+0000 7f8a48a00700 1 -- 172.21.15.33:0/679769951 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f8a340e2810 con 0x7f8a44104590 2024-05-03T18:30:26.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.290+0000 7f8a42000700 1 --2- 172.21.15.33:0/679769951 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f8a280b2a70 0x7f8a280b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:26.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.290+0000 7f8a54786b80 1 -- start start 2024-05-03T18:30:26.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.290+0000 7f8a54786b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c43b9d5ac0 0x55c43b9d5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:26.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.290+0000 7f8a54786b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c43b9d63e0 0x55c43b9d0030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:26.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.290+0000 7f8a54786b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c43b9d0650 0x55c43b9d0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:26.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.290+0000 7f8a54786b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c43b922aa0 con 0x55c43b9d5ac0 2024-05-03T18:30:26.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.290+0000 7f8a54786b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55c43b8c07e0 con 0x55c43b9d0650 2024-05-03T18:30:26.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.290+0000 7f8a54786b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55c43b91ea60 con 0x55c43b9d63e0 2024-05-03T18:30:26.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.291+0000 7f8a42000700 1 --2- 172.21.15.33:0/679769951 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f8a280b2a70 0x7f8a280b4f30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8a30006010 tx=0x7f8a30005960 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:30:26.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.291+0000 7f8a42a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c43b9d5ac0 0x55c43b9d5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:26.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.291+0000 7f8a42a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c43b9d5ac0 0x55c43b9d5ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33732/0 (socket says 172.21.15.33:33732) 2024-05-03T18:30:26.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.291+0000 7f8a42a00700 1 -- 172.21.15.33:0/3378590383 learned_addr learned my addr 172.21.15.33:0/3378590383 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:30:26.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.291+0000 7f8a42000700 1 --2- 172.21.15.33:0/3378590383 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c43b9d63e0 0x55c43b9d0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:26.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.291+0000 7f8a43400700 1 --2- 172.21.15.33:0/3378590383 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c43b9d0650 0x55c43b9d0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:26.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.291+0000 7f8a42a00700 1 -- 172.21.15.33:0/3378590383 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c43b9d63e0 msgr2=0x55c43b9d0030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:30:26.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.291+0000 7f8a42a00700 1 --2- 172.21.15.33:0/3378590383 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c43b9d63e0 0x55c43b9d0030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:26.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.291+0000 7f8a42a00700 1 -- 172.21.15.33:0/3378590383 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c43b9d0650 msgr2=0x55c43b9d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:30:26.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.291+0000 7f8a42a00700 1 --2- 172.21.15.33:0/3378590383 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c43b9d0650 0x55c43b9d0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:26.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.291+0000 7f8a42a00700 1 -- 172.21.15.33:0/3378590383 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c43b86f950 con 0x55c43b9d5ac0 2024-05-03T18:30:26.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.292+0000 7f8a42a00700 1 --2- 172.21.15.33:0/3378590383 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c43b9d5ac0 0x55c43b9d5ea0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f8a340a5780 tx=0x7f8a340cc050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1b884b9eb44a151 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:30:26.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.292+0000 7f8a38c00700 1 -- 172.21.15.33:0/3378590383 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a340a9700 con 0x55c43b9d5ac0 2024-05-03T18:30:26.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.292+0000 7f8a38c00700 1 -- 172.21.15.33:0/3378590383 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8a340adab0 con 0x55c43b9d5ac0 2024-05-03T18:30:26.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.292+0000 7f8a38c00700 1 -- 172.21.15.33:0/3378590383 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a340a9960 con 0x55c43b9d5ac0 2024-05-03T18:30:26.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.293+0000 7f8a54786b80 1 -- 172.21.15.33:0/3378590383 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c43b9d5ac0 msgr2=0x55c43b9d5ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:30:26.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.293+0000 7f8a54786b80 1 --2- 172.21.15.33:0/3378590383 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c43b9d5ac0 0x55c43b9d5ea0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f8a340a5780 tx=0x7f8a340cc050 comp rx=0 tx=0).stop 2024-05-03T18:30:26.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.293+0000 7f8a54786b80 1 -- 172.21.15.33:0/3378590383 shutdown_connections 2024-05-03T18:30:26.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.293+0000 7f8a54786b80 1 --2- 172.21.15.33:0/3378590383 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c43b9d0650 0x55c43b9d0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:26.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.293+0000 7f8a54786b80 1 --2- 172.21.15.33:0/3378590383 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c43b9d5ac0 0x55c43b9d5ea0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:26.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.293+0000 7f8a54786b80 1 --2- 172.21.15.33:0/3378590383 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c43b9d63e0 0x55c43b9d0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:26.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.293+0000 7f8a54786b80 1 -- 172.21.15.33:0/3378590383 >> 172.21.15.33:0/3378590383 conn(0x55c43b8febe0 msgr2=0x55c43b9caa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:30:26.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.293+0000 7f8a54786b80 1 -- 172.21.15.33:0/3378590383 shutdown_connections 2024-05-03T18:30:26.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.293+0000 7f8a54786b80 1 -- 172.21.15.33:0/3378590383 wait complete. 2024-05-03T18:30:26.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.294+0000 7f8a54786b80 1 -- start start 2024-05-03T18:30:26.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.294+0000 7f8a54786b80 1 Processor -- start 2024-05-03T18:30:26.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.295+0000 7f8a54786b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c43b9d0650 0x55c43b9dc140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:26.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.295+0000 7f8a54786b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c43b9d5ac0 0x55c43b9d94e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:26.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.295+0000 7f8a54786b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c43b9d63e0 0x55c43b9d9a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:26.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.295+0000 7f8a54786b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c43b9dce50 con 0x55c43b9d63e0 2024-05-03T18:30:26.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.295+0000 7f8a54786b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55c43b888ba0 con 0x55c43b9d0650 2024-05-03T18:30:26.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.295+0000 7f8a54786b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55c43b9dcfd0 con 0x55c43b9d5ac0 2024-05-03T18:30:26.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.295+0000 7f8a54786b80 10 client.0 _ll_register_callbacks cb 0x55c43b8c9cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:30:26.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.295+0000 7f8a43400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c43b9d63e0 0x55c43b9d9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:26.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.295+0000 7f8a43400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c43b9d63e0 0x55c43b9d9a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33748/0 (socket says 172.21.15.33:33748) 2024-05-03T18:30:26.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.295+0000 7f8a43400700 1 -- 172.21.15.33:0/3961016091 learned_addr learned my addr 172.21.15.33:0/3961016091 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:30:26.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.295+0000 7f8a42a00700 1 --2- 172.21.15.33:0/3961016091 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c43b9d0650 0x55c43b9dc140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:26.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.296+0000 7f8a42000700 1 --2- 172.21.15.33:0/3961016091 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c43b9d5ac0 0x55c43b9d94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:26.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.296+0000 7f8a42a00700 1 -- 172.21.15.33:0/3961016091 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c43b9d5ac0 msgr2=0x55c43b9d94e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:30:26.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.296+0000 7f8a42a00700 1 --2- 172.21.15.33:0/3961016091 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c43b9d5ac0 0x55c43b9d94e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:26.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.296+0000 7f8a42a00700 1 -- 172.21.15.33:0/3961016091 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c43b9d63e0 msgr2=0x55c43b9d9a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:30:26.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.296+0000 7f8a42a00700 1 --2- 172.21.15.33:0/3961016091 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c43b9d63e0 0x55c43b9d9a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:26.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.296+0000 7f8a42a00700 1 -- 172.21.15.33:0/3961016091 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c43b8e84a0 con 0x55c43b9d0650 2024-05-03T18:30:26.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.297+0000 7f8a42a00700 1 --2- 172.21.15.33:0/3961016091 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c43b9d0650 0x55c43b9dc140 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8a340a8fe0 tx=0x7f8a340a9370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d613aa9a71ffcc3d server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:30:26.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.297+0000 7f8a2d600700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-03T18:30:26.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.297+0000 7f8a2d600700 1 -- 172.21.15.33:0/3961016091 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a34014070 con 0x55c43b9d0650 2024-05-03T18:30:26.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.297+0000 7f8a2d600700 1 -- 172.21.15.33:0/3961016091 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f8a340a9b70 con 0x55c43b9d0650 2024-05-03T18:30:26.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.297+0000 7f8a54786b80 1 -- 172.21.15.33:0/3961016091 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a340adab0 con 0x55c43b9d0650 2024-05-03T18:30:26.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.297+0000 7f8a2d600700 1 -- 172.21.15.33:0/3961016091 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f8a34088050 con 0x55c43b9d0650 2024-05-03T18:30:26.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.298+0000 7f8a54786b80 10 client.35414 fetch_fsmap learned FSMap version 21 2024-05-03T18:30:26.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.298+0000 7f8a54786b80 1 -- 172.21.15.33:0/3961016091 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55c43b9efee0 con 0x55c43b9d0650 2024-05-03T18:30:26.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.298+0000 7f8a2d600700 1 -- 172.21.15.33:0/3961016091 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8a340adab0 con 0x55c43b9d0650 2024-05-03T18:30:26.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.298+0000 7f8a2d600700 1 -- 172.21.15.33:0/3961016091 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f8a340b2140 con 0x55c43b9d0650 2024-05-03T18:30:26.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.299+0000 7f8a54786b80 10 client.35414 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:30:26.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.299+0000 7f8a54786b80 10 client.35414 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:30:26.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.299+0000 7f8a54786b80 20 client.35414 populate_metadata read hostname 'smithi033' 2024-05-03T18:30:26.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.299+0000 7f8a54786b80 1 --2- 172.21.15.33:0/3961016091 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55c43b9f23b0 0x55c43b9f47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:26.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.299+0000 7f8a54786b80 4 client.35414 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:30:26.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.299+0000 7f8a54786b80 1 -- 172.21.15.33:0/3961016091 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c43b99d250 con 0x55c43b9f23b0 2024-05-03T18:30:26.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.299+0000 7f8a42000700 1 --2- 172.21.15.33:0/3961016091 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55c43b9f23b0 0x55c43b9f47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:26.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.300+0000 7f8a42000700 1 --2- 172.21.15.33:0/3961016091 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55c43b9f23b0 0x55c43b9f47a0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7168316f19a793c server_cookie=d0f55ffc770f5b51 in_seq=0 out_seq=0 2024-05-03T18:30:26.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.300+0000 7f8a2d600700 10 client.35414 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:30:26.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.302+0000 7f8a2d600700 1 -- 172.21.15.33:0/3961016091 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c43b99d250 con 0x55c43b9f23b0 2024-05-03T18:30:26.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.302+0000 7f8a2d600700 10 client.35414 handle_command_reply: tid=0 2024-05-03T18:30:26.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.302+0000 7f8a54786b80 1 client.35414 shutdown 2024-05-03T18:30:26.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.302+0000 7f8a54786b80 1 -- 172.21.15.33:0/3961016091 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c43b9d0650 msgr2=0x55c43b9dc140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:30:26.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.302+0000 7f8a54786b80 1 --2- 172.21.15.33:0/3961016091 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c43b9d0650 0x55c43b9dc140 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8a340a8fe0 tx=0x7f8a340a9370 comp rx=0 tx=0).stop 2024-05-03T18:30:26.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.303+0000 7f8a54786b80 1 -- 172.21.15.33:0/3961016091 shutdown_connections 2024-05-03T18:30:26.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.303+0000 7f8a54786b80 1 --2- 172.21.15.33:0/3961016091 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55c43b9f23b0 0x55c43b9f47a0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:26.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.303+0000 7f8a54786b80 1 --2- 172.21.15.33:0/3961016091 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c43b9d63e0 0x55c43b9d9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:26.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.303+0000 7f8a54786b80 1 --2- 172.21.15.33:0/3961016091 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c43b9d0650 0x55c43b9dc140 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:26.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.303+0000 7f8a54786b80 1 --2- 172.21.15.33:0/3961016091 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c43b9d5ac0 0x55c43b9d94e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:26.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.303+0000 7f8a2d600700 0 client.35414 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:30:26.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.303+0000 7f8a54786b80 1 -- 172.21.15.33:0/3961016091 >> 172.21.15.33:0/3961016091 conn(0x55c43b8febe0 msgr2=0x55c43b9ca100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:30:26.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.303+0000 7f8a54786b80 1 -- 172.21.15.33:0/3961016091 shutdown_connections 2024-05-03T18:30:26.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.303+0000 7f8a54786b80 1 -- 172.21.15.33:0/3961016091 wait complete. 2024-05-03T18:30:26.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.303+0000 7f8a54786b80 20 client.35414 trim_cache size 0 max 16384 2024-05-03T18:30:26.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.329+0000 7f8a54786b80 1 Processor -- start 2024-05-03T18:30:26.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.329+0000 7f8a54786b80 1 -- start start 2024-05-03T18:30:26.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.329+0000 7f8a54786b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c43b9d0650 0x55c43b9d0030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:26.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.330+0000 7f8a54786b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c43b9d5ac0 0x55c43b9db2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:26.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.330+0000 7f8a54786b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c43b9d63e0 0x55c43b9f77c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:26.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.330+0000 7f8a54786b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8a441ab070 con 0x55c43b9d5ac0 2024-05-03T18:30:26.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.330+0000 7f8a54786b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8a441ad8f0 con 0x55c43b9d63e0 2024-05-03T18:30:26.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.330+0000 7f8a54786b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8a441ab6e0 con 0x55c43b9d0650 2024-05-03T18:30:26.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.330+0000 7f8a42000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c43b9d5ac0 0x55c43b9db2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:26.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.330+0000 7f8a42000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c43b9d5ac0 0x55c43b9db2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33754/0 (socket says 172.21.15.33:33754) 2024-05-03T18:30:26.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.330+0000 7f8a42000700 1 -- 172.21.15.33:0/1224239408 learned_addr learned my addr 172.21.15.33:0/1224239408 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:30:26.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.330+0000 7f8a43400700 1 --2- 172.21.15.33:0/1224239408 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c43b9d63e0 0x55c43b9f77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:26.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.330+0000 7f8a42a00700 1 --2- 172.21.15.33:0/1224239408 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c43b9d0650 0x55c43b9d0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:26.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.331+0000 7f8a42000700 1 -- 172.21.15.33:0/1224239408 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c43b9d0650 msgr2=0x55c43b9d0030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:30:26.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.331+0000 7f8a42000700 1 --2- 172.21.15.33:0/1224239408 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c43b9d0650 0x55c43b9d0030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:26.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.331+0000 7f8a42000700 1 -- 172.21.15.33:0/1224239408 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c43b9d63e0 msgr2=0x55c43b9f77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:30:26.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.331+0000 7f8a42000700 1 --2- 172.21.15.33:0/1224239408 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c43b9d63e0 0x55c43b9f77c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:26.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.331+0000 7f8a42000700 1 -- 172.21.15.33:0/1224239408 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c43b9d8a80 con 0x55c43b9d5ac0 2024-05-03T18:30:26.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.331+0000 7f8a42a00700 1 --2- 172.21.15.33:0/1224239408 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c43b9d0650 0x55c43b9d0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:30:26.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.331+0000 7f8a43400700 1 --2- 172.21.15.33:0/1224239408 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c43b9d63e0 0x55c43b9f77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:30:26.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.331+0000 7f8a42000700 1 --2- 172.21.15.33:0/1224239408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c43b9d5ac0 0x55c43b9db2a0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f8a30009990 tx=0x7f8a30021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec59a450b1027557 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:30:26.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.331+0000 7f8a2fe00700 1 -- 172.21.15.33:0/1224239408 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a30038070 con 0x55c43b9d5ac0 2024-05-03T18:30:26.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.332+0000 7f8a2fe00700 1 -- 172.21.15.33:0/1224239408 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8a3000bc60 con 0x55c43b9d5ac0 2024-05-03T18:30:26.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.332+0000 7f8a2fe00700 1 -- 172.21.15.33:0/1224239408 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a300334d0 con 0x55c43b9d5ac0 2024-05-03T18:30:26.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.332+0000 7f8a54786b80 1 -- 172.21.15.33:0/1224239408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c43b9d5ac0 msgr2=0x55c43b9db2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:30:26.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.332+0000 7f8a54786b80 1 --2- 172.21.15.33:0/1224239408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c43b9d5ac0 0x55c43b9db2a0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f8a30009990 tx=0x7f8a30021000 comp rx=0 tx=0).stop 2024-05-03T18:30:26.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.333+0000 7f8a54786b80 1 -- 172.21.15.33:0/1224239408 shutdown_connections 2024-05-03T18:30:26.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.333+0000 7f8a54786b80 1 --2- 172.21.15.33:0/1224239408 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c43b9d63e0 0x55c43b9f77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:26.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.333+0000 7f8a54786b80 1 --2- 172.21.15.33:0/1224239408 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c43b9d0650 0x55c43b9d0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:26.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.333+0000 7f8a54786b80 1 --2- 172.21.15.33:0/1224239408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c43b9d5ac0 0x55c43b9db2a0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:26.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.333+0000 7f8a54786b80 1 -- 172.21.15.33:0/1224239408 >> 172.21.15.33:0/1224239408 conn(0x55c43b8febe0 msgr2=0x55c43b9dab90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:30:26.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.333+0000 7f8a54786b80 1 -- 172.21.15.33:0/1224239408 shutdown_connections 2024-05-03T18:30:26.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.333+0000 7f8a54786b80 1 -- 172.21.15.33:0/1224239408 wait complete. 2024-05-03T18:30:26.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.333+0000 7f8a54786b80 1 -- start start 2024-05-03T18:30:26.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.333+0000 7f8a54786b80 1 Processor -- start 2024-05-03T18:30:26.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.334+0000 7f8a54786b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c43b9d0650 0x55c43b9f7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:26.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.334+0000 7f8a54786b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c43b9d5ac0 0x55c43b9f62a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:26.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.334+0000 7f8a54786b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c43b9d63e0 0x55c43b9f69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:26.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.334+0000 7f8a54786b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8a441abdb0 con 0x55c43b9d5ac0 2024-05-03T18:30:26.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.334+0000 7f8a54786b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8a441ad220 con 0x55c43b9d63e0 2024-05-03T18:30:26.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.334+0000 7f8a54786b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8a441ac480 con 0x55c43b9d0650 2024-05-03T18:30:26.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.334+0000 7f8a54786b80 10 client.0 _ll_register_callbacks cb 0x55c43b9d40f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:30:26.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.334+0000 7f8a42000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c43b9d5ac0 0x55c43b9f62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:26.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.335+0000 7f8a42000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c43b9d5ac0 0x55c43b9f62a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33770/0 (socket says 172.21.15.33:33770) 2024-05-03T18:30:26.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.335+0000 7f8a42000700 1 -- 172.21.15.33:0/1749095574 learned_addr learned my addr 172.21.15.33:0/1749095574 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:30:26.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.335+0000 7f8a42a00700 1 --2- 172.21.15.33:0/1749095574 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c43b9d0650 0x55c43b9f7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:26.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.335+0000 7f8a43400700 1 --2- 172.21.15.33:0/1749095574 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c43b9d63e0 0x55c43b9f69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:26.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.335+0000 7f8a42000700 1 -- 172.21.15.33:0/1749095574 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c43b9d0650 msgr2=0x55c43b9f7ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:30:26.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.335+0000 7f8a42000700 1 --2- 172.21.15.33:0/1749095574 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c43b9d0650 0x55c43b9f7ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:26.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.335+0000 7f8a42000700 1 -- 172.21.15.33:0/1749095574 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c43b9d63e0 msgr2=0x55c43b9f69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:30:26.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.335+0000 7f8a42000700 1 --2- 172.21.15.33:0/1749095574 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c43b9d63e0 0x55c43b9f69e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:26.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.335+0000 7f8a42000700 1 -- 172.21.15.33:0/1749095574 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c43ba5c8e0 con 0x55c43b9d5ac0 2024-05-03T18:30:26.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.336+0000 7f8a42000700 1 --2- 172.21.15.33:0/1749095574 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c43b9d5ac0 0x55c43b9f62a0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f8a300098e0 tx=0x7f8a3000bfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1bde34b8c3619d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:30:26.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.336+0000 7f8a2d600700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:30:26.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.336+0000 7f8a2d600700 1 -- 172.21.15.33:0/1749095574 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a30038040 con 0x55c43b9d5ac0 2024-05-03T18:30:26.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.336+0000 7f8a54786b80 1 -- 172.21.15.33:0/1749095574 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a3000bc60 con 0x55c43b9d5ac0 2024-05-03T18:30:26.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.337+0000 7f8a2d600700 1 -- 172.21.15.33:0/1749095574 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f8a3000b7d0 con 0x55c43b9d5ac0 2024-05-03T18:30:26.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.337+0000 7f8a2d600700 1 -- 172.21.15.33:0/1749095574 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f8a3000b9d0 con 0x55c43b9d5ac0 2024-05-03T18:30:26.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.337+0000 7f8a54786b80 10 client.15804 fetch_fsmap learned FSMap version 21 2024-05-03T18:30:26.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.337+0000 7f8a54786b80 1 -- 172.21.15.33:0/1749095574 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55c43ba5db00 con 0x55c43b9d5ac0 2024-05-03T18:30:26.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.337+0000 7f8a2d600700 1 -- 172.21.15.33:0/1749095574 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8a3000bc60 con 0x55c43b9d5ac0 2024-05-03T18:30:26.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.338+0000 7f8a2d600700 1 -- 172.21.15.33:0/1749095574 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f8a30044030 con 0x55c43b9d5ac0 2024-05-03T18:30:26.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.338+0000 7f8a54786b80 10 client.15804 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:30:26.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.338+0000 7f8a54786b80 10 client.15804 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:30:26.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.338+0000 7f8a54786b80 20 client.15804 populate_metadata read hostname 'smithi033' 2024-05-03T18:30:26.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.338+0000 7f8a54786b80 1 --2- 172.21.15.33:0/1749095574 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55c43ba5e130 0x55c43ba60550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:26.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.338+0000 7f8a54786b80 4 client.15804 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-03T18:30:26.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.338+0000 7f8a54786b80 1 -- 172.21.15.33:0/1749095574 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55c43b930480 con 0x55c43ba5e130 2024-05-03T18:30:26.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.339+0000 7f8a42a00700 1 --2- 172.21.15.33:0/1749095574 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55c43ba5e130 0x55c43ba60550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:26.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.340+0000 7f8a42a00700 1 --2- 172.21.15.33:0/1749095574 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55c43ba5e130 0x55c43ba60550 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7153c590a0869d6 server_cookie=b75c44293791021 in_seq=0 out_seq=0 2024-05-03T18:30:26.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.340+0000 7f8a2d600700 10 client.15804 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:30:26.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.343+0000 7f8a2d600700 1 -- 172.21.15.33:0/1749095574 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c43b930480 con 0x55c43ba5e130 2024-05-03T18:30:26.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.343+0000 7f8a2d600700 10 client.15804 handle_command_reply: tid=0 2024-05-03T18:30:26.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.343+0000 7f8a54786b80 1 client.15804 shutdown 2024-05-03T18:30:26.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.343+0000 7f8a54786b80 1 -- 172.21.15.33:0/1749095574 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c43b9d5ac0 msgr2=0x55c43b9f62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:30:26.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.343+0000 7f8a54786b80 1 --2- 172.21.15.33:0/1749095574 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c43b9d5ac0 0x55c43b9f62a0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f8a300098e0 tx=0x7f8a3000bfa0 comp rx=0 tx=0).stop 2024-05-03T18:30:26.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.344+0000 7f8a54786b80 1 -- 172.21.15.33:0/1749095574 shutdown_connections 2024-05-03T18:30:26.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.344+0000 7f8a54786b80 1 --2- 172.21.15.33:0/1749095574 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55c43ba5e130 0x55c43ba60550 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:26.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.344+0000 7f8a2d600700 0 client.15804 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:30:26.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.344+0000 7f8a54786b80 1 --2- 172.21.15.33:0/1749095574 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c43b9d63e0 0x55c43b9f69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:26.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.344+0000 7f8a54786b80 1 --2- 172.21.15.33:0/1749095574 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c43b9d0650 0x55c43b9f7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:26.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.344+0000 7f8a54786b80 1 --2- 172.21.15.33:0/1749095574 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c43b9d5ac0 0x55c43b9f62a0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:26.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.344+0000 7f8a54786b80 1 -- 172.21.15.33:0/1749095574 >> 172.21.15.33:0/1749095574 conn(0x55c43b8febe0 msgr2=0x55c43b9d8300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:30:26.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.344+0000 7f8a54786b80 1 -- 172.21.15.33:0/1749095574 shutdown_connections 2024-05-03T18:30:26.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.344+0000 7f8a54786b80 1 -- 172.21.15.33:0/1749095574 wait complete. 2024-05-03T18:30:26.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.344+0000 7f8a54786b80 20 client.15804 trim_cache size 0 max 16384 2024-05-03T18:30:26.346 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-03T18:30:26.347 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-05-03T18:30:26.347 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "f77b958f-e7a0-46f6-8cab-f1c11b9092fe", 2024-05-03T18:30:26.347 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-05-03T18:30:26.347 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-03T18:30:26.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.345+0000 7f8a43e00700 1 -- 172.21.15.33:0/679769951 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f8a280b2a70 msgr2=0x7f8a280b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:30:26.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.345+0000 7f8a43e00700 1 --2- 172.21.15.33:0/679769951 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f8a280b2a70 0x7f8a280b4f30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8a30006010 tx=0x7f8a30005960 comp rx=0 tx=0).stop 2024-05-03T18:30:26.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.345+0000 7f8a43e00700 1 -- 172.21.15.33:0/679769951 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a44104590 msgr2=0x7f8a441a18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:30:26.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.345+0000 7f8a43e00700 1 --2- 172.21.15.33:0/679769951 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a44104590 0x7f8a441a18a0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f8a3400d980 tx=0x7f8a3400dc90 comp rx=0 tx=0).stop 2024-05-03T18:30:26.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.346+0000 7f8a43e00700 1 -- 172.21.15.33:0/679769951 shutdown_connections 2024-05-03T18:30:26.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.346+0000 7f8a43e00700 1 --2- 172.21.15.33:0/679769951 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a4410d9c0 0x7f8a4419ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:26.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.346+0000 7f8a43e00700 1 --2- 172.21.15.33:0/679769951 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a44104590 0x7f8a441a18a0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:26.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.346+0000 7f8a43e00700 1 --2- 172.21.15.33:0/679769951 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f8a280b2a70 0x7f8a280b4f30 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:26.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.346+0000 7f8a43e00700 1 --2- 172.21.15.33:0/679769951 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a44104eb0 0x7f8a441a1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:26.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.346+0000 7f8a43e00700 1 -- 172.21.15.33:0/679769951 >> 172.21.15.33:0/679769951 conn(0x7f8a440fddc0 msgr2=0x7f8a4410ed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:30:26.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.346+0000 7f8a43e00700 1 -- 172.21.15.33:0/679769951 shutdown_connections 2024-05-03T18:30:26.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:26.347+0000 7f8a43e00700 1 -- 172.21.15.33:0/679769951 wait complete. 2024-05-03T18:30:27.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:30:26 smithi033 ceph-mon[28330]: pgmap v100: 129 pgs: 129 active+clean; 932 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 10 MiB/s wr, 14 op/s 2024-05-03T18:30:27.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:30:26 smithi099 ceph-mon[31658]: pgmap v100: 129 pgs: 129 active+clean; 932 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 10 MiB/s wr, 14 op/s 2024-05-03T18:30:27.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:30:26 smithi150 ceph-mon[32492]: pgmap v100: 129 pgs: 129 active+clean; 932 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 10 MiB/s wr, 14 op/s 2024-05-03T18:30:27.442 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-03T18:30:27.995 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:30:28.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:30:27 smithi033 ceph-mon[28330]: scrub queued for path: / 2024-05-03T18:30:28.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:30:27 smithi033 ceph-mon[28330]: scrub summary: idle+waiting paths [/] 2024-05-03T18:30:28.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:30:27 smithi033 ceph-mon[28330]: scrub summary: active paths [/] 2024-05-03T18:30:28.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:30:27 smithi033 ceph-mon[28330]: scrub summary: idle+waiting paths [/] 2024-05-03T18:30:28.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:30:27 smithi099 ceph-mon[31658]: scrub queued for path: / 2024-05-03T18:30:28.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:30:27 smithi099 ceph-mon[31658]: scrub summary: idle+waiting paths [/] 2024-05-03T18:30:28.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:30:27 smithi099 ceph-mon[31658]: scrub summary: active paths [/] 2024-05-03T18:30:28.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:30:27 smithi099 ceph-mon[31658]: scrub summary: idle+waiting paths [/] 2024-05-03T18:30:28.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:30:27 smithi150 ceph-mon[32492]: scrub queued for path: / 2024-05-03T18:30:28.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:30:27 smithi150 ceph-mon[32492]: scrub summary: idle+waiting paths [/] 2024-05-03T18:30:28.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:30:27 smithi150 ceph-mon[32492]: scrub summary: active paths [/] 2024-05-03T18:30:28.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:30:27 smithi150 ceph-mon[32492]: scrub summary: idle+waiting paths [/] 2024-05-03T18:30:29.228 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:30:28 smithi033 ceph-mon[28330]: pgmap v101: 129 pgs: 129 active+clean; 992 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 15 MiB/s wr, 20 op/s 2024-05-03T18:30:29.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:30:28 smithi099 ceph-mon[31658]: pgmap v101: 129 pgs: 129 active+clean; 992 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 15 MiB/s wr, 20 op/s 2024-05-03T18:30:29.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:30:28 smithi150 ceph-mon[32492]: pgmap v101: 129 pgs: 129 active+clean; 992 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 15 MiB/s wr, 20 op/s 2024-05-03T18:30:30.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:30:29 smithi033 ceph-mon[28330]: pgmap v102: 129 pgs: 129 active+clean; 992 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 10 MiB/s wr, 12 op/s 2024-05-03T18:30:30.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:30:29 smithi099 ceph-mon[31658]: pgmap v102: 129 pgs: 129 active+clean; 992 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 10 MiB/s wr, 12 op/s 2024-05-03T18:30:30.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:30:29 smithi150 ceph-mon[32492]: pgmap v102: 129 pgs: 129 active+clean; 992 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 10 MiB/s wr, 12 op/s 2024-05-03T18:30:30.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.501+0000 7f209ac00700 1 -- 172.21.15.33:0/1551723644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f209406ba10 msgr2=0x7f209406be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:30:30.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.501+0000 7f209ac00700 1 --2- 172.21.15.33:0/1551723644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f209406ba10 0x7f209406be90 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f2080009990 tx=0x7f2080009ca0 comp rx=0 tx=0).stop 2024-05-03T18:30:30.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.501+0000 7f209ac00700 1 -- 172.21.15.33:0/1551723644 shutdown_connections 2024-05-03T18:30:30.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.501+0000 7f209ac00700 1 --2- 172.21.15.33:0/1551723644 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f209406c3d0 0x7f2094110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:30.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.501+0000 7f209ac00700 1 --2- 172.21.15.33:0/1551723644 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2094109500 0x7f20941098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:30.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.501+0000 7f209ac00700 1 --2- 172.21.15.33:0/1551723644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f209406ba10 0x7f209406be90 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:30.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.501+0000 7f209ac00700 1 -- 172.21.15.33:0/1551723644 >> 172.21.15.33:0/1551723644 conn(0x7f2094079310 msgr2=0x7f2094079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:30:30.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.501+0000 7f209ac00700 1 -- 172.21.15.33:0/1551723644 shutdown_connections 2024-05-03T18:30:30.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.501+0000 7f209ac00700 1 -- 172.21.15.33:0/1551723644 wait complete. 2024-05-03T18:30:30.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.502+0000 7f209ac00700 1 Processor -- start 2024-05-03T18:30:30.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.502+0000 7f209ac00700 1 -- start start 2024-05-03T18:30:30.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.503+0000 7f209ac00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f209406ba10 0x7f20941a1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:30.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.503+0000 7f209ac00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f209406c3d0 0x7f20941a1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:30.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.503+0000 7f209ac00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2094109500 0x7f209419bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:30.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.503+0000 7f209ac00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2094113510 con 0x7f209406ba10 2024-05-03T18:30:30.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.503+0000 7f209ac00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2094113390 con 0x7f2094109500 2024-05-03T18:30:30.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.503+0000 7f209ac00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2094113690 con 0x7f209406c3d0 2024-05-03T18:30:30.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.503+0000 7f2093e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f209406ba10 0x7f20941a1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:30.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.503+0000 7f2093e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f209406ba10 0x7f20941a1a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33790/0 (socket says 172.21.15.33:33790) 2024-05-03T18:30:30.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.503+0000 7f2093e00700 1 -- 172.21.15.33:0/2489796515 learned_addr learned my addr 172.21.15.33:0/2489796515 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:30:30.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.504+0000 7f2093400700 1 --2- 172.21.15.33:0/2489796515 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f209406c3d0 0x7f20941a1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:30.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.504+0000 7f2098c00700 1 --2- 172.21.15.33:0/2489796515 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2094109500 0x7f209419bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:30.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.504+0000 7f2093e00700 1 -- 172.21.15.33:0/2489796515 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f209406c3d0 msgr2=0x7f20941a1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:30:30.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.504+0000 7f2093e00700 1 --2- 172.21.15.33:0/2489796515 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f209406c3d0 0x7f20941a1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:30.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.504+0000 7f2093e00700 1 -- 172.21.15.33:0/2489796515 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2094109500 msgr2=0x7f209419bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:30:30.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.504+0000 7f2093e00700 1 --2- 172.21.15.33:0/2489796515 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2094109500 0x7f209419bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:30.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.504+0000 7f2093e00700 1 -- 172.21.15.33:0/2489796515 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f209419c3f0 con 0x7f209406ba10 2024-05-03T18:30:30.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.505+0000 7f2093e00700 1 --2- 172.21.15.33:0/2489796515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f209406ba10 0x7f20941a1a40 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f208400d950 tx=0x7f208400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:30:30.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.505+0000 7f2090c00700 1 -- 172.21.15.33:0/2489796515 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2084016070 con 0x7f209406ba10 2024-05-03T18:30:30.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.505+0000 7f2090c00700 1 -- 172.21.15.33:0/2489796515 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2084009dd0 con 0x7f209406ba10 2024-05-03T18:30:30.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.505+0000 7f209ac00700 1 -- 172.21.15.33:0/2489796515 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f209419c6e0 con 0x7f209406ba10 2024-05-03T18:30:30.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.505+0000 7f2090c00700 1 -- 172.21.15.33:0/2489796515 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f208400f3c0 con 0x7f209406ba10 2024-05-03T18:30:30.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.505+0000 7f209ac00700 1 -- 172.21.15.33:0/2489796515 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20941a89c0 con 0x7f209406ba10 2024-05-03T18:30:30.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.509+0000 7f2090c00700 1 -- 172.21.15.33:0/2489796515 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f2084019070 con 0x7f209406ba10 2024-05-03T18:30:30.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.511+0000 7f20a567ab80 1 Processor -- start 2024-05-03T18:30:30.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.511+0000 7f20a567ab80 1 -- start start 2024-05-03T18:30:30.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.511+0000 7f2090c00700 1 --2- 172.21.15.33:0/2489796515 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f20780b29f0 0x7f20780b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:30.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.512+0000 7f2090c00700 1 -- 172.21.15.33:0/2489796515 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f20840e2600 con 0x7f209406ba10 2024-05-03T18:30:30.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.512+0000 7f20a567ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d8aebe07b0 0x55d8aebe0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:30.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.512+0000 7f20a567ab80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8aebe10d0 0x55d8aebdbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:30.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.512+0000 7f20a567ab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d8aebdc5d0 0x55d8aebdca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:30.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.512+0000 7f20a567ab80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d8aeb2ea20 con 0x55d8aebe10d0 2024-05-03T18:30:30.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.512+0000 7f20a567ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55d8aeacc760 con 0x55d8aebe07b0 2024-05-03T18:30:30.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.512+0000 7f20a567ab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55d8aeb2a9e0 con 0x55d8aebdc5d0 2024-05-03T18:30:30.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.512+0000 7f2098c00700 1 --2- 172.21.15.33:0/2489796515 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f20780b29f0 0x7f20780b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:30.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.512+0000 7f2093400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8aebe10d0 0x55d8aebdbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:30.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.512+0000 7f2093e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d8aebe07b0 0x55d8aebe0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:30.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.513+0000 7f2093e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d8aebe07b0 0x55d8aebe0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:59490/0 (socket says 172.21.15.33:59490) 2024-05-03T18:30:30.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.513+0000 7f2093e00700 1 -- 172.21.15.33:0/1623260572 learned_addr learned my addr 172.21.15.33:0/1623260572 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:30:30.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.513+0000 7f2098c00700 1 --2- 172.21.15.33:0/1623260572 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d8aebdc5d0 0x55d8aebdca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:30.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.513+0000 7f2093400700 1 -- 172.21.15.33:0/1623260572 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d8aebdc5d0 msgr2=0x55d8aebdca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:30:30.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.513+0000 7f2093400700 1 --2- 172.21.15.33:0/1623260572 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d8aebdc5d0 0x55d8aebdca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:30.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.513+0000 7f2093400700 1 -- 172.21.15.33:0/1623260572 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d8aebe07b0 msgr2=0x55d8aebe0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:30:30.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.513+0000 7f2093400700 1 --2- 172.21.15.33:0/1623260572 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d8aebe07b0 0x55d8aebe0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:30.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.513+0000 7f2093400700 1 -- 172.21.15.33:0/1623260572 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8aea7b8d0 con 0x55d8aebe10d0 2024-05-03T18:30:30.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.514+0000 7f2093400700 1 --2- 172.21.15.33:0/1623260572 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8aebe10d0 0x55d8aebdbfb0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f2080006010 tx=0x7f2080009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2f19556d94374e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:30:30.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.514+0000 7f2098c00700 1 --2- 172.21.15.33:0/2489796515 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f20780b29f0 0x7f20780b4eb0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f208c007fd0 tx=0x7f208c012ad0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:30:30.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.514+0000 7f2089600700 1 -- 172.21.15.33:0/1623260572 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2080027070 con 0x55d8aebe10d0 2024-05-03T18:30:30.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.514+0000 7f2089600700 1 -- 172.21.15.33:0/1623260572 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f20800037c0 con 0x55d8aebe10d0 2024-05-03T18:30:30.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.514+0000 7f2089600700 1 -- 172.21.15.33:0/1623260572 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f208000ba30 con 0x55d8aebe10d0 2024-05-03T18:30:30.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.515+0000 7f20a567ab80 1 -- 172.21.15.33:0/1623260572 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8aebe10d0 msgr2=0x55d8aebdbfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:30:30.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.515+0000 7f20a567ab80 1 --2- 172.21.15.33:0/1623260572 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8aebe10d0 0x55d8aebdbfb0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f2080006010 tx=0x7f2080009f90 comp rx=0 tx=0).stop 2024-05-03T18:30:30.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.515+0000 7f20a567ab80 1 -- 172.21.15.33:0/1623260572 shutdown_connections 2024-05-03T18:30:30.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.515+0000 7f20a567ab80 1 --2- 172.21.15.33:0/1623260572 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d8aebdc5d0 0x55d8aebdca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:30.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.515+0000 7f20a567ab80 1 --2- 172.21.15.33:0/1623260572 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d8aebe07b0 0x55d8aebe0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:30.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.515+0000 7f20a567ab80 1 --2- 172.21.15.33:0/1623260572 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8aebe10d0 0x55d8aebdbfb0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:30.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.515+0000 7f20a567ab80 1 -- 172.21.15.33:0/1623260572 >> 172.21.15.33:0/1623260572 conn(0x55d8aeb0ab60 msgr2=0x55d8aebd67f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:30:30.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.515+0000 7f20a567ab80 1 -- 172.21.15.33:0/1623260572 shutdown_connections 2024-05-03T18:30:30.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.516+0000 7f20a567ab80 1 -- 172.21.15.33:0/1623260572 wait complete. 2024-05-03T18:30:30.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.516+0000 7f20a567ab80 1 -- start start 2024-05-03T18:30:30.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.516+0000 7f20a567ab80 1 Processor -- start 2024-05-03T18:30:30.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.517+0000 7f20a567ab80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8aebdc5d0 0x55d8aebe7e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:30.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.517+0000 7f20a567ab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d8aebe07b0 0x55d8aebe5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:30.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.517+0000 7f20a567ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d8aebe10d0 0x55d8aebe57a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:30.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.517+0000 7f20a567ab80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d8aebe8c40 con 0x55d8aebdc5d0 2024-05-03T18:30:30.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.517+0000 7f20a567ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55d8aea94b20 con 0x55d8aebe10d0 2024-05-03T18:30:30.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.517+0000 7f20a567ab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55d8aebe8dc0 con 0x55d8aebe07b0 2024-05-03T18:30:30.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.517+0000 7f20a567ab80 10 client.0 _ll_register_callbacks cb 0x55d8aead5c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:30:30.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.517+0000 7f2093e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8aebdc5d0 0x55d8aebe7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:30.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.517+0000 7f2093e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8aebdc5d0 0x55d8aebe7e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33808/0 (socket says 172.21.15.33:33808) 2024-05-03T18:30:30.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.517+0000 7f2093e00700 1 -- 172.21.15.33:0/2150368345 learned_addr learned my addr 172.21.15.33:0/2150368345 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:30:30.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.518+0000 7f2093400700 1 --2- 172.21.15.33:0/2150368345 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d8aebe07b0 0x55d8aebe5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:30.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.518+0000 7f2093e00700 1 -- 172.21.15.33:0/2150368345 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d8aebe07b0 msgr2=0x55d8aebe5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:30:30.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.518+0000 7f2093e00700 1 --2- 172.21.15.33:0/2150368345 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d8aebe07b0 0x55d8aebe5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:30.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.518+0000 7f2093e00700 1 -- 172.21.15.33:0/2150368345 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d8aebe10d0 msgr2=0x55d8aebe57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:30:30.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.518+0000 7f2093e00700 1 --2- 172.21.15.33:0/2150368345 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d8aebe10d0 0x55d8aebe57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:30.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.518+0000 7f2093e00700 1 -- 172.21.15.33:0/2150368345 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8aeaf4420 con 0x55d8aebdc5d0 2024-05-03T18:30:30.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.518+0000 7f2093e00700 1 --2- 172.21.15.33:0/2150368345 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8aebdc5d0 0x55d8aebe7e50 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f20840a6760 tx=0x7f20840a9cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4be913903699030 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:30:30.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.518+0000 7f207e000700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:30:30.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.518+0000 7f207e000700 1 -- 172.21.15.33:0/2150368345 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20840c3060 con 0x55d8aebdc5d0 2024-05-03T18:30:30.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.518+0000 7f207e000700 1 -- 172.21.15.33:0/2150368345 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f20840e2190 con 0x55d8aebdc5d0 2024-05-03T18:30:30.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.519+0000 7f20a567ab80 1 -- 172.21.15.33:0/2150368345 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20800037c0 con 0x55d8aebdc5d0 2024-05-03T18:30:30.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.519+0000 7f207e000700 1 -- 172.21.15.33:0/2150368345 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f2084020050 con 0x55d8aebdc5d0 2024-05-03T18:30:30.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.519+0000 7f20a567ab80 10 client.15816 fetch_fsmap learned FSMap version 21 2024-05-03T18:30:30.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.519+0000 7f20a567ab80 1 -- 172.21.15.33:0/2150368345 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55d8aebf3440 con 0x55d8aebdc5d0 2024-05-03T18:30:30.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.519+0000 7f207e000700 1 -- 172.21.15.33:0/2150368345 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f20800037c0 con 0x55d8aebdc5d0 2024-05-03T18:30:30.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.519+0000 7f207e000700 1 -- 172.21.15.33:0/2150368345 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f20840b1ec0 con 0x55d8aebdc5d0 2024-05-03T18:30:30.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.520+0000 7f20a567ab80 10 client.15816 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:30:30.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.520+0000 7f20a567ab80 10 client.15816 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:30:30.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.520+0000 7f20a567ab80 20 client.15816 populate_metadata read hostname 'smithi033' 2024-05-03T18:30:30.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.520+0000 7f20a567ab80 1 --2- 172.21.15.33:0/2150368345 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55d8aebf5a80 0x55d8aebf7e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:30.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.520+0000 7f20a567ab80 4 client.15816 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:30:30.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.520+0000 7f20a567ab80 1 -- 172.21.15.33:0/2150368345 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d8aeba91d0 con 0x55d8aebf5a80 2024-05-03T18:30:30.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.521+0000 7f2093400700 1 --2- 172.21.15.33:0/2150368345 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55d8aebf5a80 0x55d8aebf7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:30.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.522+0000 7f2093400700 1 --2- 172.21.15.33:0/2150368345 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55d8aebf5a80 0x55d8aebf7e70 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a63b52cc689d9bc server_cookie=a40792c7ca4f399c in_seq=0 out_seq=0 2024-05-03T18:30:30.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.522+0000 7f207e000700 10 client.15816 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:30:30.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.524+0000 7f207e000700 1 -- 172.21.15.33:0/2150368345 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d8aeba91d0 con 0x55d8aebf5a80 2024-05-03T18:30:30.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.524+0000 7f207e000700 10 client.15816 handle_command_reply: tid=0 2024-05-03T18:30:30.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.524+0000 7f20a567ab80 1 client.15816 shutdown 2024-05-03T18:30:30.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.524+0000 7f20a567ab80 1 -- 172.21.15.33:0/2150368345 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8aebdc5d0 msgr2=0x55d8aebe7e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:30:30.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.524+0000 7f20a567ab80 1 --2- 172.21.15.33:0/2150368345 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8aebdc5d0 0x55d8aebe7e50 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f20840a6760 tx=0x7f20840a9cf0 comp rx=0 tx=0).stop 2024-05-03T18:30:30.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.524+0000 7f20a567ab80 1 -- 172.21.15.33:0/2150368345 shutdown_connections 2024-05-03T18:30:30.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.524+0000 7f20a567ab80 1 --2- 172.21.15.33:0/2150368345 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55d8aebf5a80 0x55d8aebf7e70 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:30.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.524+0000 7f20a567ab80 1 --2- 172.21.15.33:0/2150368345 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d8aebe10d0 0x55d8aebe57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:30.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.524+0000 7f20a567ab80 1 --2- 172.21.15.33:0/2150368345 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8aebdc5d0 0x55d8aebe7e50 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:30.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.524+0000 7f20a567ab80 1 --2- 172.21.15.33:0/2150368345 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d8aebe07b0 0x55d8aebe5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:30.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.524+0000 7f20a567ab80 1 -- 172.21.15.33:0/2150368345 >> 172.21.15.33:0/2150368345 conn(0x55d8aeb0ab60 msgr2=0x55d8aebe1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:30:30.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.524+0000 7f207e000700 0 client.15816 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:30:30.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.524+0000 7f20a567ab80 1 -- 172.21.15.33:0/2150368345 shutdown_connections 2024-05-03T18:30:30.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.524+0000 7f20a567ab80 1 -- 172.21.15.33:0/2150368345 wait complete. 2024-05-03T18:30:30.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.525+0000 7f20a567ab80 20 client.15816 trim_cache size 0 max 16384 2024-05-03T18:30:30.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.552+0000 7f20a567ab80 1 Processor -- start 2024-05-03T18:30:30.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.552+0000 7f20a567ab80 1 -- start start 2024-05-03T18:30:30.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.552+0000 7f20a567ab80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8aebdc5d0 0x55d8aebf1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:30.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.553+0000 7f20a567ab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d8aebe07b0 0x55d8aebd6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:30.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.553+0000 7f20a567ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d8aebe10d0 0x55d8aebdfb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:30.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.553+0000 7f20a567ab80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f20941af500 con 0x55d8aebdc5d0 2024-05-03T18:30:30.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.553+0000 7f20a567ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f20941b1d80 con 0x55d8aebe10d0 2024-05-03T18:30:30.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.553+0000 7f20a567ab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f20941afb70 con 0x55d8aebe07b0 2024-05-03T18:30:30.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.553+0000 7f2093e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8aebdc5d0 0x55d8aebf1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:30.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.553+0000 7f2093e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8aebdc5d0 0x55d8aebf1cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33814/0 (socket says 172.21.15.33:33814) 2024-05-03T18:30:30.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.553+0000 7f2093e00700 1 -- 172.21.15.33:0/3448154679 learned_addr learned my addr 172.21.15.33:0/3448154679 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:30:30.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.553+0000 7f2093400700 1 --2- 172.21.15.33:0/3448154679 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d8aebe07b0 0x55d8aebd6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:30.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.553+0000 7f2093e00700 1 -- 172.21.15.33:0/3448154679 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d8aebe07b0 msgr2=0x55d8aebd6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:30:30.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.553+0000 7f2093e00700 1 --2- 172.21.15.33:0/3448154679 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d8aebe07b0 0x55d8aebd6660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:30.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.553+0000 7f2098c00700 1 --2- 172.21.15.33:0/3448154679 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d8aebe10d0 0x55d8aebdfb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:30.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.553+0000 7f2093e00700 1 -- 172.21.15.33:0/3448154679 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d8aebe10d0 msgr2=0x55d8aebdfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:30:30.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.554+0000 7f2093e00700 1 --2- 172.21.15.33:0/3448154679 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d8aebe10d0 0x55d8aebdfb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:30.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.554+0000 7f2093e00700 1 -- 172.21.15.33:0/3448154679 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8aebe5ab0 con 0x55d8aebdc5d0 2024-05-03T18:30:30.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.554+0000 7f2098c00700 1 --2- 172.21.15.33:0/3448154679 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d8aebe10d0 0x55d8aebdfb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:30:30.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.554+0000 7f2093e00700 1 --2- 172.21.15.33:0/3448154679 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8aebdc5d0 0x55d8aebf1cd0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f2084005950 tx=0x7f20840a9d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=191200a575893bb5 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:30:30.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.554+0000 7f2088c00700 1 -- 172.21.15.33:0/3448154679 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20840c3040 con 0x55d8aebdc5d0 2024-05-03T18:30:30.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.555+0000 7f2088c00700 1 -- 172.21.15.33:0/3448154679 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f20840b1ec0 con 0x55d8aebdc5d0 2024-05-03T18:30:30.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.555+0000 7f2088c00700 1 -- 172.21.15.33:0/3448154679 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20840e23f0 con 0x55d8aebdc5d0 2024-05-03T18:30:30.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.555+0000 7f20a567ab80 1 -- 172.21.15.33:0/3448154679 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8aebdc5d0 msgr2=0x55d8aebf1cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:30:30.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.555+0000 7f20a567ab80 1 --2- 172.21.15.33:0/3448154679 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8aebdc5d0 0x55d8aebf1cd0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f2084005950 tx=0x7f20840a9d60 comp rx=0 tx=0).stop 2024-05-03T18:30:30.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.556+0000 7f20a567ab80 1 -- 172.21.15.33:0/3448154679 shutdown_connections 2024-05-03T18:30:30.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.556+0000 7f20a567ab80 1 --2- 172.21.15.33:0/3448154679 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d8aebe10d0 0x55d8aebdfb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:30.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.556+0000 7f20a567ab80 1 --2- 172.21.15.33:0/3448154679 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8aebdc5d0 0x55d8aebf1cd0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:30.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.556+0000 7f20a567ab80 1 --2- 172.21.15.33:0/3448154679 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d8aebe07b0 0x55d8aebd6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:30.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.556+0000 7f20a567ab80 1 -- 172.21.15.33:0/3448154679 >> 172.21.15.33:0/3448154679 conn(0x55d8aeb0ab60 msgr2=0x55d8aebdf3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:30:30.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.556+0000 7f20a567ab80 1 -- 172.21.15.33:0/3448154679 shutdown_connections 2024-05-03T18:30:30.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.556+0000 7f20a567ab80 1 -- 172.21.15.33:0/3448154679 wait complete. 2024-05-03T18:30:30.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.556+0000 7f20a567ab80 1 -- start start 2024-05-03T18:30:30.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.556+0000 7f20a567ab80 1 Processor -- start 2024-05-03T18:30:30.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.557+0000 7f20a567ab80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8aebdc5d0 0x55d8aebee060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:30.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.557+0000 7f20a567ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d8aebe07b0 0x55d8aebee5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:30.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.557+0000 7f20a567ab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d8aebe10d0 0x55d8aebe9010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:30.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.557+0000 7f20a567ab80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f20941b0240 con 0x55d8aebdc5d0 2024-05-03T18:30:30.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.557+0000 7f20a567ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f20941b16b0 con 0x55d8aebe07b0 2024-05-03T18:30:30.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.557+0000 7f20a567ab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f20941b0910 con 0x55d8aebe10d0 2024-05-03T18:30:30.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.557+0000 7f20a567ab80 10 client.0 _ll_register_callbacks cb 0x55d8aead5c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:30:30.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.557+0000 7f2093e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8aebdc5d0 0x55d8aebee060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:30.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.558+0000 7f2093e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8aebdc5d0 0x55d8aebee060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33818/0 (socket says 172.21.15.33:33818) 2024-05-03T18:30:30.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.558+0000 7f2093e00700 1 -- 172.21.15.33:0/1937832211 learned_addr learned my addr 172.21.15.33:0/1937832211 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:30:30.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.558+0000 7f2093400700 1 --2- 172.21.15.33:0/1937832211 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d8aebe07b0 0x55d8aebee5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:30.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.558+0000 7f2098c00700 1 --2- 172.21.15.33:0/1937832211 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d8aebe10d0 0x55d8aebe9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:30.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.558+0000 7f2093e00700 1 -- 172.21.15.33:0/1937832211 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d8aebe10d0 msgr2=0x55d8aebe9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:30:30.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.558+0000 7f2093e00700 1 --2- 172.21.15.33:0/1937832211 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d8aebe10d0 0x55d8aebe9010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:30.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.558+0000 7f2093e00700 1 -- 172.21.15.33:0/1937832211 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d8aebe07b0 msgr2=0x55d8aebee5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:30:30.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.558+0000 7f2093e00700 1 --2- 172.21.15.33:0/1937832211 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d8aebe07b0 0x55d8aebee5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:30.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.558+0000 7f2093e00700 1 -- 172.21.15.33:0/1937832211 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8aebe9550 con 0x55d8aebdc5d0 2024-05-03T18:30:30.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.558+0000 7f2093400700 1 --2- 172.21.15.33:0/1937832211 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d8aebe07b0 0x55d8aebee5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:30:30.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.559+0000 7f2093e00700 1 --2- 172.21.15.33:0/1937832211 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8aebdc5d0 0x55d8aebee060 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f20840b22c0 tx=0x7f20840108b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bff258127f4877e server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:30:30.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.559+0000 7f207e000700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:30:30.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.559+0000 7f207e000700 1 -- 172.21.15.33:0/1937832211 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20840c3070 con 0x55d8aebdc5d0 2024-05-03T18:30:30.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.559+0000 7f207e000700 1 -- 172.21.15.33:0/1937832211 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f20840ca380 con 0x55d8aebdc5d0 2024-05-03T18:30:30.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.559+0000 7f20a567ab80 1 -- 172.21.15.33:0/1937832211 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20840b1ec0 con 0x55d8aebdc5d0 2024-05-03T18:30:30.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.560+0000 7f207e000700 1 -- 172.21.15.33:0/1937832211 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f20840b2070 con 0x55d8aebdc5d0 2024-05-03T18:30:30.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.560+0000 7f20a567ab80 10 client.15822 fetch_fsmap learned FSMap version 21 2024-05-03T18:30:30.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.560+0000 7f20a567ab80 1 -- 172.21.15.33:0/1937832211 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55d8aebe9550 con 0x55d8aebdc5d0 2024-05-03T18:30:30.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.560+0000 7f207e000700 1 -- 172.21.15.33:0/1937832211 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f20840b1ec0 con 0x55d8aebdc5d0 2024-05-03T18:30:30.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.560+0000 7f207e000700 1 -- 172.21.15.33:0/1937832211 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f20840a4e60 con 0x55d8aebdc5d0 2024-05-03T18:30:30.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.560+0000 7f20a567ab80 10 client.15822 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:30:30.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.560+0000 7f20a567ab80 10 client.15822 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:30:30.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.560+0000 7f20a567ab80 20 client.15822 populate_metadata read hostname 'smithi033' 2024-05-03T18:30:30.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.560+0000 7f20a567ab80 1 --2- 172.21.15.33:0/1937832211 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55d8aec69480 0x55d8aec6b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:30:30.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.560+0000 7f20a567ab80 4 client.15822 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-03T18:30:30.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.560+0000 7f20a567ab80 1 -- 172.21.15.33:0/1937832211 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d8aeb3c400 con 0x55d8aec69480 2024-05-03T18:30:30.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.562+0000 7f2093400700 1 --2- 172.21.15.33:0/1937832211 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55d8aec69480 0x55d8aec6b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:30:30.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.562+0000 7f2093400700 1 --2- 172.21.15.33:0/1937832211 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55d8aec69480 0x55d8aec6b870 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e13c9aeb1ecec21a server_cookie=9c75b6e876a244bd in_seq=0 out_seq=0 2024-05-03T18:30:30.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.562+0000 7f207e000700 10 client.15822 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:30:30.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.563+0000 7f207e000700 1 -- 172.21.15.33:0/1937832211 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55d8aeb3c400 con 0x55d8aec69480 2024-05-03T18:30:30.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.563+0000 7f207e000700 10 client.15822 handle_command_reply: tid=0 2024-05-03T18:30:30.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.563+0000 7f20a567ab80 1 client.15822 shutdown 2024-05-03T18:30:30.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.563+0000 7f20a567ab80 1 -- 172.21.15.33:0/1937832211 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8aebdc5d0 msgr2=0x55d8aebee060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:30:30.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.563+0000 7f20a567ab80 1 --2- 172.21.15.33:0/1937832211 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8aebdc5d0 0x55d8aebee060 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f20840b22c0 tx=0x7f20840108b0 comp rx=0 tx=0).stop 2024-05-03T18:30:30.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.564+0000 7f20a567ab80 1 -- 172.21.15.33:0/1937832211 shutdown_connections 2024-05-03T18:30:30.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.564+0000 7f20a567ab80 1 --2- 172.21.15.33:0/1937832211 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55d8aec69480 0x55d8aec6b870 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:30.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.564+0000 7f20a567ab80 1 --2- 172.21.15.33:0/1937832211 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d8aebe10d0 0x55d8aebe9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:30.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.564+0000 7f20a567ab80 1 --2- 172.21.15.33:0/1937832211 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8aebdc5d0 0x55d8aebee060 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:30.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.564+0000 7f20a567ab80 1 --2- 172.21.15.33:0/1937832211 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d8aebe07b0 0x55d8aebee5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:30.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.564+0000 7f20a567ab80 1 -- 172.21.15.33:0/1937832211 >> 172.21.15.33:0/1937832211 conn(0x55d8aeb0ab60 msgr2=0x55d8aebdfc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:30:30.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.564+0000 7f207e000700 0 client.15822 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:30:30.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.564+0000 7f20a567ab80 1 -- 172.21.15.33:0/1937832211 shutdown_connections 2024-05-03T18:30:30.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.564+0000 7f20a567ab80 1 -- 172.21.15.33:0/1937832211 wait complete. 2024-05-03T18:30:30.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.564+0000 7f20a567ab80 20 client.15822 trim_cache size 0 max 16384 2024-05-03T18:30:30.566 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-03T18:30:30.566 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-03T18:30:30.566 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-05-03T18:30:30.566 INFO:teuthology.orchestra.run.smithi033.stdout: "f77b958f-e7a0-46f6-8cab-f1c11b9092fe": { 2024-05-03T18:30:30.566 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-05-03T18:30:30.566 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "f77b958f-e7a0-46f6-8cab-f1c11b9092fe", 2024-05-03T18:30:30.567 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-05-03T18:30:30.567 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:30:30.567 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:30:30.567 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-03T18:30:30.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.565+0000 7f209ac00700 1 -- 172.21.15.33:0/2489796515 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f20780b29f0 msgr2=0x7f20780b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:30:30.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.565+0000 7f209ac00700 1 --2- 172.21.15.33:0/2489796515 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f20780b29f0 0x7f20780b4eb0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f208c007fd0 tx=0x7f208c012ad0 comp rx=0 tx=0).stop 2024-05-03T18:30:30.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.565+0000 7f209ac00700 1 -- 172.21.15.33:0/2489796515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f209406ba10 msgr2=0x7f20941a1a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:30:30.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.565+0000 7f209ac00700 1 --2- 172.21.15.33:0/2489796515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f209406ba10 0x7f20941a1a40 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f208400d950 tx=0x7f208400dc60 comp rx=0 tx=0).stop 2024-05-03T18:30:30.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.565+0000 7f209ac00700 1 -- 172.21.15.33:0/2489796515 shutdown_connections 2024-05-03T18:30:30.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.565+0000 7f209ac00700 1 --2- 172.21.15.33:0/2489796515 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f20780b29f0 0x7f20780b4eb0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:30.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.565+0000 7f209ac00700 1 --2- 172.21.15.33:0/2489796515 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2094109500 0x7f209419bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:30.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.565+0000 7f209ac00700 1 --2- 172.21.15.33:0/2489796515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f209406ba10 0x7f20941a1a40 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:30.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.565+0000 7f209ac00700 1 --2- 172.21.15.33:0/2489796515 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f209406c3d0 0x7f20941a1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:30:30.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.565+0000 7f209ac00700 1 -- 172.21.15.33:0/2489796515 >> 172.21.15.33:0/2489796515 conn(0x7f2094079310 msgr2=0x7f209410e650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:30:30.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.566+0000 7f209ac00700 1 -- 172.21.15.33:0/2489796515 shutdown_connections 2024-05-03T18:30:30.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:30:30.566+0000 7f209ac00700 1 -- 172.21.15.33:0/2489796515 wait complete. 2024-05-03T18:30:31.486 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:30:31 smithi033 ceph-mon[28330]: scrub summary: idle 2024-05-03T18:30:31.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:30:31 smithi099 ceph-mon[31658]: scrub summary: idle 2024-05-03T18:30:31.560 INFO:tasks.cephfs.filesystem:scrub status for tag:f77b958f-e7a0-46f6-8cab-f1c11b9092fe - {'path': '/', 'tag': 'f77b958f-e7a0-46f6-8cab-f1c11b9092fe', 'options': 'recursive,force'} 2024-05-03T18:30:31.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:30:31 smithi150 ceph-mon[32492]: scrub summary: idle 2024-05-03T18:30:32.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:30:32 smithi033 ceph-mon[28330]: pgmap v103: 129 pgs: 129 active+clean; 992 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 10 MiB/s wr, 12 op/s 2024-05-03T18:30:32.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:30:32 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:30:32.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:30:32 smithi099 ceph-mon[31658]: pgmap v103: 129 pgs: 129 active+clean; 992 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 10 MiB/s wr, 12 op/s 2024-05-03T18:30:32.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:30:32 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:30:32.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:30:32 smithi150 ceph-mon[32492]: pgmap v103: 129 pgs: 129 active+clean; 992 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 10 MiB/s wr, 12 op/s 2024-05-03T18:30:32.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:30:32 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:30:33.812 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:30:33 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:30:33.812 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:30:33 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:30:33.812 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:30:33 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:30:33.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:30:33 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:30:33.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:30:33 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:30:33.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:30:33 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:30:34.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:30:33 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:30:34.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:30:33 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:30:34.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:30:33 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:30:34.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:30:34 smithi150 ceph-mon[32492]: pgmap v104: 129 pgs: 129 active+clean; 1.0 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 15 MiB/s wr, 18 op/s 2024-05-03T18:30:34.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:30:34 smithi033 ceph-mon[28330]: pgmap v104: 129 pgs: 129 active+clean; 1.0 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 15 MiB/s wr, 18 op/s 2024-05-03T18:30:35.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:30:34 smithi099 ceph-mon[31658]: pgmap v104: 129 pgs: 129 active+clean; 1.0 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 15 MiB/s wr, 18 op/s 2024-05-03T18:30:37.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:30:36 smithi033 ceph-mon[28330]: pgmap v105: 129 pgs: 129 active+clean; 1.0 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 10 MiB/s wr, 12 op/s 2024-05-03T18:30:37.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:30:36 smithi099 ceph-mon[31658]: pgmap v105: 129 pgs: 129 active+clean; 1.0 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 10 MiB/s wr, 12 op/s 2024-05-03T18:30:37.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:30:36 smithi150 ceph-mon[32492]: pgmap v105: 129 pgs: 129 active+clean; 1.0 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 10 MiB/s wr, 12 op/s 2024-05-03T18:30:39.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:30:38 smithi033 ceph-mon[28330]: pgmap v106: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 14 MiB/s wr, 16 op/s 2024-05-03T18:30:39.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:30:38 smithi099 ceph-mon[31658]: pgmap v106: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 14 MiB/s wr, 16 op/s 2024-05-03T18:30:39.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:30:38 smithi150 ceph-mon[32492]: pgmap v106: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 14 MiB/s wr, 16 op/s 2024-05-03T18:30:41.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:30:40 smithi033 ceph-mon[28330]: pgmap v107: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 9.1 MiB/s wr, 10 op/s 2024-05-03T18:30:41.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:30:40 smithi099 ceph-mon[31658]: pgmap v107: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 9.1 MiB/s wr, 10 op/s 2024-05-03T18:30:41.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:30:40 smithi150 ceph-mon[32492]: pgmap v107: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 9.1 MiB/s wr, 10 op/s 2024-05-03T18:30:43.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:30:42 smithi033 ceph-mon[28330]: pgmap v108: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 9.1 MiB/s wr, 10 op/s 2024-05-03T18:30:43.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:30:42 smithi099 ceph-mon[31658]: pgmap v108: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 9.1 MiB/s wr, 10 op/s 2024-05-03T18:30:43.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:30:42 smithi150 ceph-mon[32492]: pgmap v108: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 9.1 MiB/s wr, 10 op/s 2024-05-03T18:30:45.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:30:44 smithi033 ceph-mon[28330]: pgmap v109: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 MiB/s wr, 14 op/s 2024-05-03T18:30:45.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:30:44 smithi099 ceph-mon[31658]: pgmap v109: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 MiB/s wr, 14 op/s 2024-05-03T18:30:45.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:30:44 smithi150 ceph-mon[32492]: pgmap v109: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 MiB/s wr, 14 op/s 2024-05-03T18:30:47.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:30:46 smithi033 ceph-mon[28330]: pgmap v110: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.7 MiB/s wr, 8 op/s 2024-05-03T18:30:47.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:30:46 smithi099 ceph-mon[31658]: pgmap v110: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.7 MiB/s wr, 8 op/s 2024-05-03T18:30:47.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:30:46 smithi150 ceph-mon[32492]: pgmap v110: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.7 MiB/s wr, 8 op/s 2024-05-03T18:30:49.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:30:48 smithi033 ceph-mon[28330]: pgmap v111: 129 pgs: 129 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 12 MiB/s wr, 13 op/s 2024-05-03T18:30:49.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:30:48 smithi099 ceph-mon[31658]: pgmap v111: 129 pgs: 129 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 12 MiB/s wr, 13 op/s 2024-05-03T18:30:49.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:30:48 smithi150 ceph-mon[32492]: pgmap v111: 129 pgs: 129 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 12 MiB/s wr, 13 op/s 2024-05-03T18:30:50.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:30:49 smithi033 ceph-mon[28330]: pgmap v112: 129 pgs: 129 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.4 MiB/s wr, 9 op/s 2024-05-03T18:30:50.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:30:49 smithi099 ceph-mon[31658]: pgmap v112: 129 pgs: 129 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.4 MiB/s wr, 9 op/s 2024-05-03T18:30:50.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:30:49 smithi150 ceph-mon[32492]: pgmap v112: 129 pgs: 129 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.4 MiB/s wr, 9 op/s 2024-05-03T18:30:52.998 INFO:teuthology.orchestra.run.smithi033.stderr:creating primary keys... 2024-05-03T18:30:53.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:30:52 smithi033 ceph-mon[28330]: pgmap v113: 129 pgs: 129 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.4 MiB/s wr, 9 op/s 2024-05-03T18:30:53.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:30:52 smithi099 ceph-mon[31658]: pgmap v113: 129 pgs: 129 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.4 MiB/s wr, 9 op/s 2024-05-03T18:30:53.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:30:52 smithi150 ceph-mon[32492]: pgmap v113: 129 pgs: 129 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.4 MiB/s wr, 9 op/s 2024-05-03T18:30:55.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:30:54 smithi033 ceph-mon[28330]: pgmap v114: 129 pgs: 129 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 13 MiB/s wr, 13 op/s 2024-05-03T18:30:55.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:30:54 smithi099 ceph-mon[31658]: pgmap v114: 129 pgs: 129 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 13 MiB/s wr, 13 op/s 2024-05-03T18:30:55.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:30:54 smithi150 ceph-mon[32492]: pgmap v114: 129 pgs: 129 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 13 MiB/s wr, 13 op/s 2024-05-03T18:30:57.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:30:56 smithi033 ceph-mon[28330]: pgmap v115: 129 pgs: 129 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.2 MiB/s wr, 9 op/s 2024-05-03T18:30:57.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:30:56 smithi099 ceph-mon[31658]: pgmap v115: 129 pgs: 129 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.2 MiB/s wr, 9 op/s 2024-05-03T18:30:57.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:30:56 smithi150 ceph-mon[32492]: pgmap v115: 129 pgs: 129 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.2 MiB/s wr, 9 op/s 2024-05-03T18:30:59.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:30:58 smithi033 ceph-mon[28330]: pgmap v116: 129 pgs: 129 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 14 MiB/s wr, 15 op/s 2024-05-03T18:30:59.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:30:58 smithi099 ceph-mon[31658]: pgmap v116: 129 pgs: 129 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 14 MiB/s wr, 15 op/s 2024-05-03T18:30:59.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:30:58 smithi150 ceph-mon[32492]: pgmap v116: 129 pgs: 129 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 14 MiB/s wr, 15 op/s 2024-05-03T18:31:01.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:31:00 smithi033 ceph-mon[28330]: pgmap v117: 129 pgs: 129 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 8.9 MiB/s wr, 11 op/s 2024-05-03T18:31:01.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:31:00 smithi099 ceph-mon[31658]: pgmap v117: 129 pgs: 129 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 8.9 MiB/s wr, 11 op/s 2024-05-03T18:31:01.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:31:00 smithi150 ceph-mon[32492]: pgmap v117: 129 pgs: 129 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 8.9 MiB/s wr, 11 op/s 2024-05-03T18:31:01.561 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-03T18:31:02.101 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:31:03.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:31:02 smithi033 ceph-mon[28330]: pgmap v118: 129 pgs: 129 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 8.9 MiB/s wr, 11 op/s 2024-05-03T18:31:03.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:31:02 smithi099 ceph-mon[31658]: pgmap v118: 129 pgs: 129 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 8.9 MiB/s wr, 11 op/s 2024-05-03T18:31:03.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:31:02 smithi150 ceph-mon[32492]: pgmap v118: 129 pgs: 129 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 8.9 MiB/s wr, 11 op/s 2024-05-03T18:31:04.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.678+0000 7f4824000700 1 -- 172.21.15.33:0/956706464 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f481c108850 msgr2=0x7f481c1103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:31:04.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.678+0000 7f4824000700 1 --2- 172.21.15.33:0/956706464 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f481c108850 0x7f481c1103d0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f481400b0d0 tx=0x7f481400b3e0 comp rx=0 tx=0).stop 2024-05-03T18:31:04.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.678+0000 7f4824000700 1 -- 172.21.15.33:0/956706464 shutdown_connections 2024-05-03T18:31:04.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.678+0000 7f4824000700 1 --2- 172.21.15.33:0/956706464 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f481c108850 0x7f481c1103d0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:04.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.678+0000 7f4824000700 1 --2- 172.21.15.33:0/956706464 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f481c103750 0x7f481c103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:04.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.678+0000 7f4824000700 1 --2- 172.21.15.33:0/956706464 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f481c104100 0x7f481c108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:04.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.678+0000 7f4824000700 1 -- 172.21.15.33:0/956706464 >> 172.21.15.33:0/956706464 conn(0x7f481c07a600 msgr2=0x7f481c07aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:31:04.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.678+0000 7f4824000700 1 -- 172.21.15.33:0/956706464 shutdown_connections 2024-05-03T18:31:04.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.678+0000 7f4824000700 1 -- 172.21.15.33:0/956706464 wait complete. 2024-05-03T18:31:04.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.680+0000 7f4824000700 1 Processor -- start 2024-05-03T18:31:04.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.680+0000 7f4824000700 1 -- start start 2024-05-03T18:31:04.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.680+0000 7f4824000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f481c103750 0x7f481c1a1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:04.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.680+0000 7f4824000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f481c104100 0x7f481c1a1d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:04.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.680+0000 7f4824000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f481c108850 0x7f481c19ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:04.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.680+0000 7f4824000700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f481c113520 con 0x7f481c104100 2024-05-03T18:31:04.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.680+0000 7f4824000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f481c1133a0 con 0x7f481c108850 2024-05-03T18:31:04.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.680+0000 7f4824000700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f481c1136a0 con 0x7f481c103750 2024-05-03T18:31:04.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.681+0000 7f4820c00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f481c104100 0x7f481c1a1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:04.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.681+0000 7f4820c00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f481c104100 0x7f481c1a1d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36790/0 (socket says 172.21.15.33:36790) 2024-05-03T18:31:04.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.681+0000 7f4820c00700 1 -- 172.21.15.33:0/3623993993 learned_addr learned my addr 172.21.15.33:0/3623993993 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:31:04.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.681+0000 7f4821600700 1 --2- 172.21.15.33:0/3623993993 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f481c103750 0x7f481c1a1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:04.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.681+0000 7f4822000700 1 --2- 172.21.15.33:0/3623993993 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f481c108850 0x7f481c19ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:04.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.682+0000 7f4820c00700 1 -- 172.21.15.33:0/3623993993 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f481c103750 msgr2=0x7f481c1a1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:31:04.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.682+0000 7f4820c00700 1 --2- 172.21.15.33:0/3623993993 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f481c103750 0x7f481c1a1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:04.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.682+0000 7f4820c00700 1 -- 172.21.15.33:0/3623993993 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f481c108850 msgr2=0x7f481c19ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:31:04.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.682+0000 7f4820c00700 1 --2- 172.21.15.33:0/3623993993 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f481c108850 0x7f481c19ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:04.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.682+0000 7f4820c00700 1 -- 172.21.15.33:0/3623993993 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f481c19c160 con 0x7f481c104100 2024-05-03T18:31:04.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.682+0000 7f4820c00700 1 --2- 172.21.15.33:0/3623993993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f481c104100 0x7f481c1a1d80 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f480800d950 tx=0x7f480800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:31:04.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.683+0000 7f481a000700 1 -- 172.21.15.33:0/3623993993 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4808016070 con 0x7f481c104100 2024-05-03T18:31:04.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.683+0000 7f481a000700 1 -- 172.21.15.33:0/3623993993 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4808009dd0 con 0x7f481c104100 2024-05-03T18:31:04.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.683+0000 7f481a000700 1 -- 172.21.15.33:0/3623993993 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f480800f3c0 con 0x7f481c104100 2024-05-03T18:31:04.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.683+0000 7f4824000700 1 -- 172.21.15.33:0/3623993993 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f481c19c450 con 0x7f481c104100 2024-05-03T18:31:04.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.683+0000 7f4824000700 1 -- 172.21.15.33:0/3623993993 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f481c1a8710 con 0x7f481c104100 2024-05-03T18:31:04.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.688+0000 7f481a000700 1 -- 172.21.15.33:0/3623993993 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f4808019070 con 0x7f481c104100 2024-05-03T18:31:04.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.690+0000 7f482ea0fb80 1 Processor -- start 2024-05-03T18:31:04.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.690+0000 7f482ea0fb80 1 -- start start 2024-05-03T18:31:04.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.690+0000 7f481a000700 1 --2- 172.21.15.33:0/3623993993 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f48040b2a70 0x7f48040b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:04.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.691+0000 7f482ea0fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633509ba930 0x5633509bad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:04.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.691+0000 7f482ea0fb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5633509bb250 0x5633509b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:04.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.691+0000 7f482ea0fb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5633509b5550 0x5633509b59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:04.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.691+0000 7f481a000700 1 -- 172.21.15.33:0/3623993993 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f48080e2600 con 0x7f481c104100 2024-05-03T18:31:04.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.691+0000 7f482ea0fb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563350907a20 con 0x5633509bb250 2024-05-03T18:31:04.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.691+0000 7f482ea0fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5633508a5760 con 0x5633509ba930 2024-05-03T18:31:04.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.691+0000 7f482ea0fb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5633509039e0 con 0x5633509b5550 2024-05-03T18:31:04.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.691+0000 7f4820c00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5633509bb250 0x5633509b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:04.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.691+0000 7f4822000700 1 --2- 172.21.15.33:0/3623993993 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f48040b2a70 0x7f48040b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:04.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.691+0000 7f4820c00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5633509bb250 0x5633509b4f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36798/0 (socket says 172.21.15.33:36798) 2024-05-03T18:31:04.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.691+0000 7f4820c00700 1 -- 172.21.15.33:0/3522369314 learned_addr learned my addr 172.21.15.33:0/3522369314 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:31:04.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.692+0000 7f4821600700 1 --2- 172.21.15.33:0/3522369314 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633509ba930 0x5633509bad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:04.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.692+0000 7f4822000700 1 --2- 172.21.15.33:0/3522369314 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5633509b5550 0x5633509b59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:04.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.692+0000 7f4820c00700 1 -- 172.21.15.33:0/3522369314 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5633509b5550 msgr2=0x5633509b59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:31:04.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.692+0000 7f4820c00700 1 --2- 172.21.15.33:0/3522369314 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5633509b5550 0x5633509b59d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:04.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.692+0000 7f4820c00700 1 -- 172.21.15.33:0/3522369314 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633509ba930 msgr2=0x5633509bad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:31:04.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.692+0000 7f4820c00700 1 --2- 172.21.15.33:0/3522369314 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633509ba930 0x5633509bad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:04.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.692+0000 7f4820c00700 1 -- 172.21.15.33:0/3522369314 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633508548d0 con 0x5633509bb250 2024-05-03T18:31:04.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.692+0000 7f4822000700 1 --2- 172.21.15.33:0/3623993993 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f48040b2a70 0x7f48040b4f30 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f48140062a0 tx=0x7f4814009c40 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:31:04.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.693+0000 7f4820c00700 1 --2- 172.21.15.33:0/3522369314 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5633509bb250 0x5633509b4f30 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f48080a5570 tx=0x7f48080a9d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=260ea5787ee928ab server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:31:04.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.693+0000 7f480ea00700 1 -- 172.21.15.33:0/3522369314 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4808020070 con 0x5633509bb250 2024-05-03T18:31:04.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.693+0000 7f480ea00700 1 -- 172.21.15.33:0/3522369314 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f48080ad8a0 con 0x5633509bb250 2024-05-03T18:31:04.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.693+0000 7f480ea00700 1 -- 172.21.15.33:0/3522369314 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48080a9750 con 0x5633509bb250 2024-05-03T18:31:04.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.694+0000 7f482ea0fb80 1 -- 172.21.15.33:0/3522369314 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5633509bb250 msgr2=0x5633509b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:31:04.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.694+0000 7f482ea0fb80 1 --2- 172.21.15.33:0/3522369314 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5633509bb250 0x5633509b4f30 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f48080a5570 tx=0x7f48080a9d20 comp rx=0 tx=0).stop 2024-05-03T18:31:04.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.694+0000 7f482ea0fb80 1 -- 172.21.15.33:0/3522369314 shutdown_connections 2024-05-03T18:31:04.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.694+0000 7f482ea0fb80 1 --2- 172.21.15.33:0/3522369314 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5633509b5550 0x5633509b59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:04.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.694+0000 7f482ea0fb80 1 --2- 172.21.15.33:0/3522369314 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633509ba930 0x5633509bad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:04.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.694+0000 7f482ea0fb80 1 --2- 172.21.15.33:0/3522369314 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5633509bb250 0x5633509b4f30 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:04.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.694+0000 7f482ea0fb80 1 -- 172.21.15.33:0/3522369314 >> 172.21.15.33:0/3522369314 conn(0x56335092d6d0 msgr2=0x5633509af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:31:04.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.694+0000 7f482ea0fb80 1 -- 172.21.15.33:0/3522369314 shutdown_connections 2024-05-03T18:31:04.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.694+0000 7f482ea0fb80 1 -- 172.21.15.33:0/3522369314 wait complete. 2024-05-03T18:31:04.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.695+0000 7f482ea0fb80 1 -- start start 2024-05-03T18:31:04.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.695+0000 7f482ea0fb80 1 Processor -- start 2024-05-03T18:31:04.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.696+0000 7f482ea0fb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5633509b5550 0x5633509c0e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:04.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.696+0000 7f482ea0fb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5633509ba930 0x5633509be1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:04.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.696+0000 7f482ea0fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633509bb250 0x5633509be700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:04.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.696+0000 7f482ea0fb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5633509c1c40 con 0x5633509ba930 2024-05-03T18:31:04.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.696+0000 7f482ea0fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56335086db20 con 0x5633509bb250 2024-05-03T18:31:04.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.696+0000 7f482ea0fb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5633509c1dc0 con 0x5633509b5550 2024-05-03T18:31:04.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.696+0000 7f482ea0fb80 10 client.0 _ll_register_callbacks cb 0x5633508aec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:31:04.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.697+0000 7f4821600700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5633509b5550 0x5633509c0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:04.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.697+0000 7f4821600700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5633509b5550 0x5633509c0e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:54524/0 (socket says 172.21.15.33:54524) 2024-05-03T18:31:04.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.697+0000 7f4821600700 1 -- 172.21.15.33:0/2302889773 learned_addr learned my addr 172.21.15.33:0/2302889773 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:31:04.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.697+0000 7f4820c00700 1 --2- 172.21.15.33:0/2302889773 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5633509ba930 0x5633509be1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:04.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.697+0000 7f4822000700 1 --2- 172.21.15.33:0/2302889773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633509bb250 0x5633509be700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:04.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.697+0000 7f4821600700 1 -- 172.21.15.33:0/2302889773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633509bb250 msgr2=0x5633509be700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:31:04.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.697+0000 7f4821600700 1 --2- 172.21.15.33:0/2302889773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633509bb250 0x5633509be700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:04.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.697+0000 7f4821600700 1 -- 172.21.15.33:0/2302889773 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5633509ba930 msgr2=0x5633509be1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:31:04.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.697+0000 7f4821600700 1 --2- 172.21.15.33:0/2302889773 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5633509ba930 0x5633509be1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:04.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.697+0000 7f4821600700 1 -- 172.21.15.33:0/2302889773 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633508cd420 con 0x5633509b5550 2024-05-03T18:31:04.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.698+0000 7f4821600700 1 --2- 172.21.15.33:0/2302889773 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5633509b5550 0x5633509c0e20 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f481000bb70 tx=0x7f481000be80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac1d6424cac01c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:31:04.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.698+0000 7f4803400700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-03T18:31:04.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.698+0000 7f4803400700 1 -- 172.21.15.33:0/2302889773 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4810003d00 con 0x5633509b5550 2024-05-03T18:31:04.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.698+0000 7f482ea0fb80 1 -- 172.21.15.33:0/2302889773 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48080ad8a0 con 0x5633509b5550 2024-05-03T18:31:04.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.698+0000 7f4803400700 1 -- 172.21.15.33:0/2302889773 <== mon.1 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f4810004830 con 0x5633509b5550 2024-05-03T18:31:04.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.699+0000 7f4803400700 1 -- 172.21.15.33:0/2302889773 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f4810024050 con 0x5633509b5550 2024-05-03T18:31:04.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.699+0000 7f482ea0fb80 10 client.25483 fetch_fsmap learned FSMap version 21 2024-05-03T18:31:04.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.699+0000 7f482ea0fb80 1 -- 172.21.15.33:0/2302889773 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5633509cc420 con 0x5633509b5550 2024-05-03T18:31:04.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.699+0000 7f4803400700 1 -- 172.21.15.33:0/2302889773 <== mon.1 v2:172.21.15.150:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f48080ad8a0 con 0x5633509b5550 2024-05-03T18:31:04.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.700+0000 7f4803400700 1 -- 172.21.15.33:0/2302889773 <== mon.1 v2:172.21.15.150:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f4810011440 con 0x5633509b5550 2024-05-03T18:31:04.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.700+0000 7f482ea0fb80 10 client.25483 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:31:04.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.700+0000 7f482ea0fb80 10 client.25483 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:31:04.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.700+0000 7f482ea0fb80 20 client.25483 populate_metadata read hostname 'smithi033' 2024-05-03T18:31:04.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.700+0000 7f482ea0fb80 1 --2- 172.21.15.33:0/2302889773 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5633509cea60 0x5633509d0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:04.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.700+0000 7f482ea0fb80 4 client.25483 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:31:04.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.700+0000 7f482ea0fb80 1 -- 172.21.15.33:0/2302889773 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563350981ff0 con 0x5633509cea60 2024-05-03T18:31:04.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.701+0000 7f4820c00700 1 --2- 172.21.15.33:0/2302889773 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5633509cea60 0x5633509d0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:04.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.702+0000 7f4803400700 10 client.25483 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:31:04.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.702+0000 7f4820c00700 1 --2- 172.21.15.33:0/2302889773 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5633509cea60 0x5633509d0e70 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6788fbd78679c53d server_cookie=2010e6646ac86918 in_seq=0 out_seq=0 2024-05-03T18:31:04.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.704+0000 7f4803400700 1 -- 172.21.15.33:0/2302889773 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563350981ff0 con 0x5633509cea60 2024-05-03T18:31:04.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.704+0000 7f4803400700 10 client.25483 handle_command_reply: tid=0 2024-05-03T18:31:04.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.704+0000 7f482ea0fb80 1 client.25483 shutdown 2024-05-03T18:31:04.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.704+0000 7f482ea0fb80 1 -- 172.21.15.33:0/2302889773 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5633509b5550 msgr2=0x5633509c0e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:31:04.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.704+0000 7f482ea0fb80 1 --2- 172.21.15.33:0/2302889773 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5633509b5550 0x5633509c0e20 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f481000bb70 tx=0x7f481000be80 comp rx=0 tx=0).stop 2024-05-03T18:31:04.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.704+0000 7f482ea0fb80 1 -- 172.21.15.33:0/2302889773 shutdown_connections 2024-05-03T18:31:04.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.704+0000 7f482ea0fb80 1 --2- 172.21.15.33:0/2302889773 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5633509cea60 0x5633509d0e70 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:04.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.704+0000 7f482ea0fb80 1 --2- 172.21.15.33:0/2302889773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633509bb250 0x5633509be700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:04.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.704+0000 7f482ea0fb80 1 --2- 172.21.15.33:0/2302889773 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5633509b5550 0x5633509c0e20 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:04.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.704+0000 7f482ea0fb80 1 --2- 172.21.15.33:0/2302889773 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5633509ba930 0x5633509be1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:04.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.704+0000 7f482ea0fb80 1 -- 172.21.15.33:0/2302889773 >> 172.21.15.33:0/2302889773 conn(0x56335092d6d0 msgr2=0x5633509aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:31:04.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.704+0000 7f4803400700 0 client.25483 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:31:04.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.704+0000 7f482ea0fb80 1 -- 172.21.15.33:0/2302889773 shutdown_connections 2024-05-03T18:31:04.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.705+0000 7f482ea0fb80 1 -- 172.21.15.33:0/2302889773 wait complete. 2024-05-03T18:31:04.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.705+0000 7f482ea0fb80 20 client.25483 trim_cache size 0 max 16384 2024-05-03T18:31:04.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.749+0000 7f482ea0fb80 1 Processor -- start 2024-05-03T18:31:04.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.750+0000 7f482ea0fb80 1 -- start start 2024-05-03T18:31:04.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.750+0000 7f482ea0fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633509b5550 0x5633509b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:04.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.750+0000 7f482ea0fb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5633509ba930 0x5633509d6740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:04.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.750+0000 7f482ea0fb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5633509bb250 0x5633509d6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:04.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.750+0000 7f482ea0fb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f481c1af630 con 0x5633509ba930 2024-05-03T18:31:04.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.750+0000 7f482ea0fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f481c1b1eb0 con 0x5633509b5550 2024-05-03T18:31:04.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.750+0000 7f482ea0fb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f481c1afca0 con 0x5633509bb250 2024-05-03T18:31:04.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.751+0000 7f4820c00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5633509ba930 0x5633509d6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:04.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.751+0000 7f4820c00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5633509ba930 0x5633509d6740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36816/0 (socket says 172.21.15.33:36816) 2024-05-03T18:31:04.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.751+0000 7f4820c00700 1 -- 172.21.15.33:0/706172990 learned_addr learned my addr 172.21.15.33:0/706172990 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:31:04.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.751+0000 7f4821600700 1 --2- 172.21.15.33:0/706172990 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633509b5550 0x5633509b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:04.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.751+0000 7f4822000700 1 --2- 172.21.15.33:0/706172990 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5633509bb250 0x5633509d6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:04.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.751+0000 7f4820c00700 1 -- 172.21.15.33:0/706172990 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5633509bb250 msgr2=0x5633509d6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:31:04.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.751+0000 7f4820c00700 1 --2- 172.21.15.33:0/706172990 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5633509bb250 0x5633509d6c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:04.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.751+0000 7f4820c00700 1 -- 172.21.15.33:0/706172990 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633509b5550 msgr2=0x5633509b4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:31:04.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.751+0000 7f4820c00700 1 --2- 172.21.15.33:0/706172990 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633509b5550 0x5633509b4f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:04.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.751+0000 7f4820c00700 1 -- 172.21.15.33:0/706172990 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633509bc730 con 0x5633509ba930 2024-05-03T18:31:04.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.751+0000 7f4822000700 1 --2- 172.21.15.33:0/706172990 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5633509bb250 0x5633509d6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:31:04.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.751+0000 7f4821600700 1 --2- 172.21.15.33:0/706172990 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633509b5550 0x5633509b4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:31:04.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.752+0000 7f4820c00700 1 --2- 172.21.15.33:0/706172990 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5633509ba930 0x5633509d6740 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f48080a91f0 tx=0x7f48080a8e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=970fd8a5c673f87b server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:31:04.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.752+0000 7f480e000700 1 -- 172.21.15.33:0/706172990 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4808020040 con 0x5633509ba930 2024-05-03T18:31:04.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.752+0000 7f480e000700 1 -- 172.21.15.33:0/706172990 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f48080ad8a0 con 0x5633509ba930 2024-05-03T18:31:04.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.752+0000 7f480e000700 1 -- 172.21.15.33:0/706172990 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48080a99b0 con 0x5633509ba930 2024-05-03T18:31:04.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.754+0000 7f482ea0fb80 1 -- 172.21.15.33:0/706172990 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5633509ba930 msgr2=0x5633509d6740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:31:04.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.754+0000 7f482ea0fb80 1 --2- 172.21.15.33:0/706172990 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5633509ba930 0x5633509d6740 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f48080a91f0 tx=0x7f48080a8e40 comp rx=0 tx=0).stop 2024-05-03T18:31:04.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.754+0000 7f482ea0fb80 1 -- 172.21.15.33:0/706172990 shutdown_connections 2024-05-03T18:31:04.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.754+0000 7f482ea0fb80 1 --2- 172.21.15.33:0/706172990 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5633509bb250 0x5633509d6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:04.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.754+0000 7f482ea0fb80 1 --2- 172.21.15.33:0/706172990 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633509b5550 0x5633509b4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:04.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.754+0000 7f482ea0fb80 1 --2- 172.21.15.33:0/706172990 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5633509ba930 0x5633509d6740 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:04.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.754+0000 7f482ea0fb80 1 -- 172.21.15.33:0/706172990 >> 172.21.15.33:0/706172990 conn(0x56335092d6d0 msgr2=0x5633509bcab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:31:04.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.754+0000 7f482ea0fb80 1 -- 172.21.15.33:0/706172990 shutdown_connections 2024-05-03T18:31:04.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.754+0000 7f482ea0fb80 1 -- 172.21.15.33:0/706172990 wait complete. 2024-05-03T18:31:04.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.754+0000 7f482ea0fb80 1 -- start start 2024-05-03T18:31:04.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.755+0000 7f482ea0fb80 1 Processor -- start 2024-05-03T18:31:04.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.756+0000 7f482ea0fb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5633509b5550 0x5633509c7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:04.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.756+0000 7f482ea0fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633509bb250 0x5633509c75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:04.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.756+0000 7f482ea0fb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5633509c4050 0x5633509c44d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:04.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.756+0000 7f482ea0fb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f481c1b0370 con 0x5633509c4050 2024-05-03T18:31:04.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.756+0000 7f482ea0fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f481c1b17e0 con 0x5633509bb250 2024-05-03T18:31:04.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.756+0000 7f482ea0fb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f481c1b0a40 con 0x5633509b5550 2024-05-03T18:31:04.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.756+0000 7f482ea0fb80 10 client.0 _ll_register_callbacks cb 0x5633508aec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:31:04.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.756+0000 7f4822000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5633509c4050 0x5633509c44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:04.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.756+0000 7f4822000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5633509c4050 0x5633509c44d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36830/0 (socket says 172.21.15.33:36830) 2024-05-03T18:31:04.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.756+0000 7f4822000700 1 -- 172.21.15.33:0/3938339156 learned_addr learned my addr 172.21.15.33:0/3938339156 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:31:04.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.756+0000 7f4820c00700 1 --2- 172.21.15.33:0/3938339156 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633509bb250 0x5633509c75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:04.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.756+0000 7f4821600700 1 --2- 172.21.15.33:0/3938339156 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5633509b5550 0x5633509c7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:04.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.757+0000 7f4822000700 1 -- 172.21.15.33:0/3938339156 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5633509b5550 msgr2=0x5633509c7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:31:04.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.757+0000 7f4822000700 1 --2- 172.21.15.33:0/3938339156 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5633509b5550 0x5633509c7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:04.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.757+0000 7f4822000700 1 -- 172.21.15.33:0/3938339156 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633509bb250 msgr2=0x5633509c75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:31:04.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.757+0000 7f4822000700 1 --2- 172.21.15.33:0/3938339156 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633509bb250 0x5633509c75c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:04.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.757+0000 7f4822000700 1 -- 172.21.15.33:0/3938339156 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633509c4a40 con 0x5633509c4050 2024-05-03T18:31:04.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.757+0000 7f4820c00700 1 --2- 172.21.15.33:0/3938339156 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633509bb250 0x5633509c75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:31:04.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.757+0000 7f4822000700 1 --2- 172.21.15.33:0/3938339156 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5633509c4050 0x5633509c44d0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f481400b010 tx=0x7f481400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ec4d107bbdbf5f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:31:04.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.757+0000 7f4803400700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:31:04.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.758+0000 7f4803400700 1 -- 172.21.15.33:0/3938339156 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4814033070 con 0x5633509c4050 2024-05-03T18:31:04.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.758+0000 7f4803400700 1 -- 172.21.15.33:0/3938339156 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f4814005250 con 0x5633509c4050 2024-05-03T18:31:04.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.758+0000 7f482ea0fb80 1 -- 172.21.15.33:0/3938339156 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48080ad8a0 con 0x5633509c4050 2024-05-03T18:31:04.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.758+0000 7f4803400700 1 -- 172.21.15.33:0/3938339156 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f4814005450 con 0x5633509c4050 2024-05-03T18:31:04.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.758+0000 7f482ea0fb80 10 client.15840 fetch_fsmap learned FSMap version 21 2024-05-03T18:31:04.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.758+0000 7f482ea0fb80 1 -- 172.21.15.33:0/3938339156 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x563350a42800 con 0x5633509c4050 2024-05-03T18:31:04.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.758+0000 7f4803400700 1 -- 172.21.15.33:0/3938339156 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f48080ad8a0 con 0x5633509c4050 2024-05-03T18:31:04.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.759+0000 7f4803400700 1 -- 172.21.15.33:0/3938339156 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f481402b040 con 0x5633509c4050 2024-05-03T18:31:04.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.759+0000 7f482ea0fb80 10 client.15840 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:31:04.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.759+0000 7f482ea0fb80 10 client.15840 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:31:04.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.759+0000 7f482ea0fb80 20 client.15840 populate_metadata read hostname 'smithi033' 2024-05-03T18:31:04.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.759+0000 7f482ea0fb80 1 --2- 172.21.15.33:0/3938339156 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x563350a42f10 0x563350a45330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:04.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.759+0000 7f482ea0fb80 4 client.15840 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-03T18:31:04.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.759+0000 7f482ea0fb80 1 -- 172.21.15.33:0/3938339156 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563350915400 con 0x563350a42f10 2024-05-03T18:31:04.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.760+0000 7f4821600700 1 --2- 172.21.15.33:0/3938339156 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x563350a42f10 0x563350a45330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:04.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.761+0000 7f4821600700 1 --2- 172.21.15.33:0/3938339156 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x563350a42f10 0x563350a45330 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7bed9cb3e9e12f65 server_cookie=f38cd7d7f92db865 in_seq=0 out_seq=0 2024-05-03T18:31:04.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.761+0000 7f4803400700 10 client.15840 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:31:04.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.761+0000 7f4803400700 1 -- 172.21.15.33:0/3938339156 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563350915400 con 0x563350a42f10 2024-05-03T18:31:04.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.761+0000 7f4803400700 10 client.15840 handle_command_reply: tid=0 2024-05-03T18:31:04.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.761+0000 7f482ea0fb80 1 client.15840 shutdown 2024-05-03T18:31:04.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.762+0000 7f482ea0fb80 1 -- 172.21.15.33:0/3938339156 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5633509c4050 msgr2=0x5633509c44d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:31:04.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.762+0000 7f482ea0fb80 1 --2- 172.21.15.33:0/3938339156 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5633509c4050 0x5633509c44d0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f481400b010 tx=0x7f481400b040 comp rx=0 tx=0).stop 2024-05-03T18:31:04.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.762+0000 7f482ea0fb80 1 -- 172.21.15.33:0/3938339156 shutdown_connections 2024-05-03T18:31:04.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.762+0000 7f482ea0fb80 1 --2- 172.21.15.33:0/3938339156 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x563350a42f10 0x563350a45330 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:04.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.762+0000 7f482ea0fb80 1 --2- 172.21.15.33:0/3938339156 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5633509c4050 0x5633509c44d0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:04.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.762+0000 7f482ea0fb80 1 --2- 172.21.15.33:0/3938339156 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5633509b5550 0x5633509c7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:04.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.762+0000 7f482ea0fb80 1 --2- 172.21.15.33:0/3938339156 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633509bb250 0x5633509c75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:04.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.762+0000 7f4803400700 0 client.15840 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:31:04.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.762+0000 7f482ea0fb80 1 -- 172.21.15.33:0/3938339156 >> 172.21.15.33:0/3938339156 conn(0x56335092d6d0 msgr2=0x5633509bb630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:31:04.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.762+0000 7f482ea0fb80 1 -- 172.21.15.33:0/3938339156 shutdown_connections 2024-05-03T18:31:04.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.762+0000 7f482ea0fb80 1 -- 172.21.15.33:0/3938339156 wait complete. 2024-05-03T18:31:04.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.762+0000 7f482ea0fb80 20 client.15840 trim_cache size 0 max 16384 2024-05-03T18:31:04.764 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-03T18:31:04.764 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-05-03T18:31:04.764 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-05-03T18:31:04.764 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-03T18:31:04.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.763+0000 7f4824000700 1 -- 172.21.15.33:0/3623993993 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f48040b2a70 msgr2=0x7f48040b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:31:04.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.763+0000 7f4824000700 1 --2- 172.21.15.33:0/3623993993 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f48040b2a70 0x7f48040b4f30 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f48140062a0 tx=0x7f4814009c40 comp rx=0 tx=0).stop 2024-05-03T18:31:04.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.763+0000 7f4824000700 1 -- 172.21.15.33:0/3623993993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f481c104100 msgr2=0x7f481c1a1d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:31:04.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.763+0000 7f4824000700 1 --2- 172.21.15.33:0/3623993993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f481c104100 0x7f481c1a1d80 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f480800d950 tx=0x7f480800dc60 comp rx=0 tx=0).stop 2024-05-03T18:31:04.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.764+0000 7f4824000700 1 -- 172.21.15.33:0/3623993993 shutdown_connections 2024-05-03T18:31:04.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.764+0000 7f4824000700 1 --2- 172.21.15.33:0/3623993993 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f48040b2a70 0x7f48040b4f30 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:04.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.764+0000 7f4824000700 1 --2- 172.21.15.33:0/3623993993 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f481c108850 0x7f481c19ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:04.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.764+0000 7f4824000700 1 --2- 172.21.15.33:0/3623993993 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f481c103750 0x7f481c1a1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:04.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.764+0000 7f4824000700 1 --2- 172.21.15.33:0/3623993993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f481c104100 0x7f481c1a1d80 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:04.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.764+0000 7f4824000700 1 -- 172.21.15.33:0/3623993993 >> 172.21.15.33:0/3623993993 conn(0x7f481c07a600 msgr2=0x7f481c101540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:31:04.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.764+0000 7f4824000700 1 -- 172.21.15.33:0/3623993993 shutdown_connections 2024-05-03T18:31:04.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:04.764+0000 7f4824000700 1 -- 172.21.15.33:0/3623993993 wait complete. 2024-05-03T18:31:05.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:31:04 smithi033 ceph-mon[28330]: pgmap v119: 129 pgs: 129 active+clean; 1.3 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 15 MiB/s wr, 30 op/s 2024-05-03T18:31:05.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:31:04 smithi099 ceph-mon[31658]: pgmap v119: 129 pgs: 129 active+clean; 1.3 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 15 MiB/s wr, 30 op/s 2024-05-03T18:31:05.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:31:04 smithi150 ceph-mon[32492]: pgmap v119: 129 pgs: 129 active+clean; 1.3 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 15 MiB/s wr, 30 op/s 2024-05-03T18:31:05.690 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-03T18:31:05.690 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T18:31:06.191 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:31:06.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:31:05 smithi033 ceph-mon[28330]: pgmap v120: 129 pgs: 129 active+clean; 1.3 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 10 MiB/s wr, 25 op/s 2024-05-03T18:31:06.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:31:05 smithi099 ceph-mon[31658]: pgmap v120: 129 pgs: 129 active+clean; 1.3 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 10 MiB/s wr, 25 op/s 2024-05-03T18:31:06.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:31:05 smithi150 ceph-mon[32492]: pgmap v120: 129 pgs: 129 active+clean; 1.3 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 10 MiB/s wr, 25 op/s 2024-05-03T18:31:09.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:09.190+0000 7f810b000700 1 -- 172.21.15.33:0/1029433259 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81040756d0 msgr2=0x7f8104075ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:31:09.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:09.190+0000 7f810b000700 1 --2- 172.21.15.33:0/1029433259 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81040756d0 0x7f8104075ab0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f80f4009990 tx=0x7f80f4009ca0 comp rx=0 tx=0).stop 2024-05-03T18:31:09.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:09.191+0000 7f810b000700 1 -- 172.21.15.33:0/1029433259 shutdown_connections 2024-05-03T18:31:09.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:09.191+0000 7f810b000700 1 --2- 172.21.15.33:0/1029433259 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f810410ff00 0x7f81041122f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:09.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:09.191+0000 7f810b000700 1 --2- 172.21.15.33:0/1029433259 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81040756d0 0x7f8104075ab0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:09.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:09.191+0000 7f810b000700 1 --2- 172.21.15.33:0/1029433259 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8104075ff0 0x7f810410f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:09.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:09.191+0000 7f810b000700 1 -- 172.21.15.33:0/1029433259 >> 172.21.15.33:0/1029433259 conn(0x7f810407a600 msgr2=0x7f810407aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:31:09.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:09.191+0000 7f810b000700 1 -- 172.21.15.33:0/1029433259 shutdown_connections 2024-05-03T18:31:09.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:09.191+0000 7f810b000700 1 -- 172.21.15.33:0/1029433259 wait complete. 2024-05-03T18:31:09.192 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:31:08 smithi033 ceph-mon[28330]: pgmap v121: 129 pgs: 129 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 17 MiB/s wr, 31 op/s 2024-05-03T18:31:09.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:09.192+0000 7f810b000700 1 Processor -- start 2024-05-03T18:31:09.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:09.193+0000 7f810b000700 1 -- start start 2024-05-03T18:31:09.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:09.193+0000 7f810b000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f81040756d0 0x7f81041a18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:09.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:09.193+0000 7f810b000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8104075ff0 0x7f81041a1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:09.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:09.193+0000 7f810b000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f810410ff00 0x7f810419bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:09.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:09.193+0000 7f810b000700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f81041157d0 con 0x7f810410ff00 2024-05-03T18:31:09.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:09.193+0000 7f810b000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8104115650 con 0x7f8104075ff0 2024-05-03T18:31:09.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:09.193+0000 7f810b000700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8104115950 con 0x7f81040756d0 2024-05-03T18:31:09.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:09.194+0000 7f8109000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f810410ff00 0x7f810419bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:09.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:09.194+0000 7f8103e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f81040756d0 0x7f81041a18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:09.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:09.194+0000 7f8109000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f810410ff00 0x7f810419bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36630/0 (socket says 172.21.15.33:36630) 2024-05-03T18:31:09.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:09.194+0000 7f8109000700 1 -- 172.21.15.33:0/3937985603 learned_addr learned my addr 172.21.15.33:0/3937985603 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:31:09.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:09.195+0000 7f8103400700 1 --2- 172.21.15.33:0/3937985603 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8104075ff0 0x7f81041a1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:09.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:09.195+0000 7f8109000700 1 -- 172.21.15.33:0/3937985603 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f81040756d0 msgr2=0x7f81041a18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:31:09.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:09.195+0000 7f8109000700 1 --2- 172.21.15.33:0/3937985603 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f81040756d0 0x7f81041a18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:09.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:09.195+0000 7f8109000700 1 -- 172.21.15.33:0/3937985603 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8104075ff0 msgr2=0x7f81041a1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:31:09.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:09.195+0000 7f8109000700 1 --2- 172.21.15.33:0/3937985603 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8104075ff0 0x7f81041a1e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:09.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:09.195+0000 7f8109000700 1 -- 172.21.15.33:0/3937985603 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f810419c210 con 0x7f810410ff00 2024-05-03T18:31:09.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:09.195+0000 7f8103e00700 1 --2- 172.21.15.33:0/3937985603 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f81040756d0 0x7f81041a18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:31:09.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:09.196+0000 7f8109000700 1 --2- 172.21.15.33:0/3937985603 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f810410ff00 0x7f810419bab0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f80fc007ee0 tx=0x7f80fc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:31:09.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:09.196+0000 7f8100c00700 1 -- 172.21.15.33:0/3937985603 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80fc004db0 con 0x7f810410ff00 2024-05-03T18:31:09.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:09.196+0000 7f8100c00700 1 -- 172.21.15.33:0/3937985603 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f80fc009e20 con 0x7f810410ff00 2024-05-03T18:31:09.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:09.196+0000 7f810b000700 1 -- 172.21.15.33:0/3937985603 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f810419c500 con 0x7f810410ff00 2024-05-03T18:31:09.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:09.196+0000 7f8100c00700 1 -- 172.21.15.33:0/3937985603 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80fc005a30 con 0x7f810410ff00 2024-05-03T18:31:09.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:09.196+0000 7f810b000700 1 -- 172.21.15.33:0/3937985603 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81041a87c0 con 0x7f810410ff00 2024-05-03T18:31:09.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:09.198+0000 7f810b000700 1 -- 172.21.15.33:0/3937985603 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f810406a190 con 0x7f810410ff00 2024-05-03T18:31:09.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:09.200+0000 7f8100c00700 1 -- 172.21.15.33:0/3937985603 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f80fc0051d0 con 0x7f810410ff00 2024-05-03T18:31:09.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:09.202+0000 7f8100c00700 1 --2- 172.21.15.33:0/3937985603 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f80e80b29f0 0x7f80e80b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:09.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:09.202+0000 7f8100c00700 1 -- 172.21.15.33:0/3937985603 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f80fc0e23d0 con 0x7f810410ff00 2024-05-03T18:31:09.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:09.202+0000 7f8103e00700 1 --2- 172.21.15.33:0/3937985603 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f80e80b29f0 0x7f80e80b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:09.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:09.203+0000 7f8103e00700 1 --2- 172.21.15.33:0/3937985603 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f80e80b29f0 0x7f80e80b4eb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f80f4003760 tx=0x7f80f400b540 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:31:09.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:09.207+0000 7f8100c00700 1 -- 172.21.15.33:0/3937985603 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f80fc010070 con 0x7f810410ff00 2024-05-03T18:31:09.297 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:31:08 smithi099 ceph-mon[31658]: pgmap v121: 129 pgs: 129 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 17 MiB/s wr, 31 op/s 2024-05-03T18:31:09.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:31:08 smithi150 ceph-mon[32492]: pgmap v121: 129 pgs: 129 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 17 MiB/s wr, 31 op/s 2024-05-03T18:31:09.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:09.542+0000 7f810b000700 1 -- 172.21.15.33:0/3937985603 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f810402e510 con 0x7f810410ff00 2024-05-03T18:31:09.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:09.543+0000 7f8100c00700 1 -- 172.21.15.33:0/3937985603 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11801 (secure 0 0 0) 0x7f80fc0a9920 con 0x7f810410ff00 2024-05-03T18:31:09.544 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:31:09.544 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1327393503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1327393503},{"type":"v1","addr":"172.21.15.33:6835","nonce":1327393503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/2461472948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":2461472948},{"type":"v1","addr":"172.21.15.33:6837","nonce":2461472948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14559,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/442269281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":442269281},{"type":"v1","addr":"172.21.15.33:6839","nonce":442269281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14571,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/645571756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":645571756},{"type":"v1","addr":"172.21.15.33:6841","nonce":645571756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/203598909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":203598909},{"type":"v1","addr":"172.21.15.150:6833","nonce":203598909}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/4029842493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":4029842493},{"type":"v1","addr":"172.21.15.150:6835","nonce":4029842493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3463664423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3463664423},{"type":"v1","addr":"172.21.15.150:6837","nonce":3463664423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24481,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6839/4051648018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":4051648018},{"type":"v1","addr":"172.21.15.150:6839","nonce":4051648018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34418,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1865105479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1865105479},{"type":"v1","addr":"172.21.15.99:6833","nonce":1865105479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34445,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/1206393914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1206393914},{"type":"v1","addr":"172.21.15.99:6835","nonce":1206393914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-03T18:22:28.837091+0000","modified":"2024-05-03T18:23:49.758398+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34508},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34472":{"gid":34472,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6837/3360970342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3360970342},{"type":"v1","addr":"172.21.15.99:6837","nonce":3360970342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34508":{"gid":34508,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/919907308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":919907308},{"type":"v1","addr":"172.21.15.99:6839","nonce":919907308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34508,"qdb_cluster":[34508]},"id":1}]} 2024-05-03T18:31:09.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:09.547+0000 7f810b000700 1 -- 172.21.15.33:0/3937985603 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f80e80b29f0 msgr2=0x7f80e80b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:31:09.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:09.547+0000 7f810b000700 1 --2- 172.21.15.33:0/3937985603 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f80e80b29f0 0x7f80e80b4eb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f80f4003760 tx=0x7f80f400b540 comp rx=0 tx=0).stop 2024-05-03T18:31:09.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:09.547+0000 7f810b000700 1 -- 172.21.15.33:0/3937985603 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f810410ff00 msgr2=0x7f810419bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:31:09.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:09.547+0000 7f810b000700 1 --2- 172.21.15.33:0/3937985603 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f810410ff00 0x7f810419bab0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f80fc007ee0 tx=0x7f80fc00d3b0 comp rx=0 tx=0).stop 2024-05-03T18:31:09.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:09.548+0000 7f810b000700 1 -- 172.21.15.33:0/3937985603 shutdown_connections 2024-05-03T18:31:09.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:09.548+0000 7f810b000700 1 --2- 172.21.15.33:0/3937985603 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f810410ff00 0x7f810419bab0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:09.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:09.548+0000 7f810b000700 1 --2- 172.21.15.33:0/3937985603 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f81040756d0 0x7f81041a18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:09.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:09.548+0000 7f810b000700 1 --2- 172.21.15.33:0/3937985603 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f80e80b29f0 0x7f80e80b4eb0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:09.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:09.548+0000 7f810b000700 1 --2- 172.21.15.33:0/3937985603 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8104075ff0 0x7f81041a1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:09.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:09.548+0000 7f810b000700 1 -- 172.21.15.33:0/3937985603 >> 172.21.15.33:0/3937985603 conn(0x7f810407a600 msgr2=0x7f8104100730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:31:09.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:09.548+0000 7f810b000700 1 -- 172.21.15.33:0/3937985603 shutdown_connections 2024-05-03T18:31:09.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:09.548+0000 7f810b000700 1 -- 172.21.15.33:0/3937985603 wait complete. 2024-05-03T18:31:09.549 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 21 2024-05-03T18:31:10.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:31:09 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/3937985603' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:31:10.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:31:09 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/3937985603' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:31:10.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:31:09 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/3937985603' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:31:10.457 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-03T18:31:10.973 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:31:11.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:31:10 smithi033 ceph-mon[28330]: pgmap v122: 129 pgs: 129 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 13 MiB/s wr, 24 op/s 2024-05-03T18:31:11.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:31:10 smithi099 ceph-mon[31658]: pgmap v122: 129 pgs: 129 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 13 MiB/s wr, 24 op/s 2024-05-03T18:31:11.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:31:10 smithi150 ceph-mon[32492]: pgmap v122: 129 pgs: 129 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 13 MiB/s wr, 24 op/s 2024-05-03T18:31:13.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:31:12 smithi033 ceph-mon[28330]: pgmap v123: 129 pgs: 129 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 13 MiB/s wr, 24 op/s 2024-05-03T18:31:13.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:31:12 smithi099 ceph-mon[31658]: pgmap v123: 129 pgs: 129 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 13 MiB/s wr, 24 op/s 2024-05-03T18:31:13.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:31:12 smithi150 ceph-mon[32492]: pgmap v123: 129 pgs: 129 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 13 MiB/s wr, 24 op/s 2024-05-03T18:31:14.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:31:14 smithi099 ceph-mon[31658]: pgmap v124: 129 pgs: 129 active+clean; 1.4 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 19 MiB/s wr, 30 op/s 2024-05-03T18:31:14.307 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:31:14 smithi033 ceph-mon[28330]: pgmap v124: 129 pgs: 129 active+clean; 1.4 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 19 MiB/s wr, 30 op/s 2024-05-03T18:31:14.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:31:14 smithi150 ceph-mon[32492]: pgmap v124: 129 pgs: 129 active+clean; 1.4 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 19 MiB/s wr, 30 op/s 2024-05-03T18:31:14.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.383+0000 7f404c200700 1 -- 172.21.15.33:0/754052034 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4044109510 msgr2=0x7f40441098f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:31:14.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.383+0000 7f404c200700 1 --2- 172.21.15.33:0/754052034 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4044109510 0x7f40441098f0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f4038009990 tx=0x7f4038009ca0 comp rx=0 tx=0).stop 2024-05-03T18:31:14.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.383+0000 7f404c200700 1 -- 172.21.15.33:0/754052034 shutdown_connections 2024-05-03T18:31:14.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.383+0000 7f404c200700 1 --2- 172.21.15.33:0/754052034 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f404406bb40 0x7f4044110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:14.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.383+0000 7f404c200700 1 --2- 172.21.15.33:0/754052034 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4044109510 0x7f40441098f0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:14.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.383+0000 7f404c200700 1 --2- 172.21.15.33:0/754052034 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f404406b180 0x7f404406b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:14.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.383+0000 7f404c200700 1 -- 172.21.15.33:0/754052034 >> 172.21.15.33:0/754052034 conn(0x7f4044079350 msgr2=0x7f4044079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:31:14.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.384+0000 7f404c200700 1 -- 172.21.15.33:0/754052034 shutdown_connections 2024-05-03T18:31:14.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.384+0000 7f404c200700 1 -- 172.21.15.33:0/754052034 wait complete. 2024-05-03T18:31:14.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.385+0000 7f404c200700 1 Processor -- start 2024-05-03T18:31:14.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.385+0000 7f404c200700 1 -- start start 2024-05-03T18:31:14.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.386+0000 7f404c200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f404406b180 0x7f4044102470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:14.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.386+0000 7f404c200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f404406bb40 0x7f40441029b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:14.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.386+0000 7f404c200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4044109510 0x7f4044102ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:14.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.386+0000 7f404c200700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4044113540 con 0x7f404406bb40 2024-05-03T18:31:14.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.386+0000 7f404c200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f40441133c0 con 0x7f404406b180 2024-05-03T18:31:14.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.386+0000 7f404c200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f40441136c0 con 0x7f4044109510 2024-05-03T18:31:14.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.386+0000 7f4048e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f404406bb40 0x7f40441029b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:14.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.386+0000 7f4048e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f404406bb40 0x7f40441029b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36642/0 (socket says 172.21.15.33:36642) 2024-05-03T18:31:14.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.386+0000 7f4048e00700 1 -- 172.21.15.33:0/3797417503 learned_addr learned my addr 172.21.15.33:0/3797417503 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:31:14.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.386+0000 7f4049800700 1 --2- 172.21.15.33:0/3797417503 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f404406b180 0x7f4044102470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:14.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.387+0000 7f404a200700 1 --2- 172.21.15.33:0/3797417503 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4044109510 0x7f4044102ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:14.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.387+0000 7f4048e00700 1 -- 172.21.15.33:0/3797417503 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4044109510 msgr2=0x7f4044102ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:31:14.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.387+0000 7f4048e00700 1 --2- 172.21.15.33:0/3797417503 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4044109510 0x7f4044102ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:14.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.387+0000 7f4048e00700 1 -- 172.21.15.33:0/3797417503 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f404406b180 msgr2=0x7f4044102470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:31:14.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.387+0000 7f4048e00700 1 --2- 172.21.15.33:0/3797417503 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f404406b180 0x7f4044102470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:14.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.387+0000 7f4048e00700 1 -- 172.21.15.33:0/3797417503 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40441a7f90 con 0x7f404406bb40 2024-05-03T18:31:14.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.387+0000 7f4048e00700 1 --2- 172.21.15.33:0/3797417503 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f404406bb40 0x7f40441029b0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f403000c9b0 tx=0x7f403000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:31:14.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.388+0000 7f4042000700 1 -- 172.21.15.33:0/3797417503 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4030007b00 con 0x7f404406bb40 2024-05-03T18:31:14.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.388+0000 7f4042000700 1 -- 172.21.15.33:0/3797417503 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4030007ca0 con 0x7f404406bb40 2024-05-03T18:31:14.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.388+0000 7f4042000700 1 -- 172.21.15.33:0/3797417503 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4030018660 con 0x7f404406bb40 2024-05-03T18:31:14.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.388+0000 7f404c200700 1 -- 172.21.15.33:0/3797417503 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40441a8280 con 0x7f404406bb40 2024-05-03T18:31:14.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.388+0000 7f404c200700 1 -- 172.21.15.33:0/3797417503 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40441a8810 con 0x7f404406bb40 2024-05-03T18:31:14.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.392+0000 7f4042000700 1 -- 172.21.15.33:0/3797417503 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f403000f450 con 0x7f404406bb40 2024-05-03T18:31:14.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.393+0000 7f4056b31b80 1 Processor -- start 2024-05-03T18:31:14.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.393+0000 7f4056b31b80 1 -- start start 2024-05-03T18:31:14.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.394+0000 7f4056b31b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5576ad3c27b0 0x5576ad3c2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:14.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.394+0000 7f4042000700 1 --2- 172.21.15.33:0/3797417503 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f402c0b2a70 0x7f402c0b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:14.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.394+0000 7f4056b31b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5576ad3c30d0 0x5576ad3bdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:14.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.394+0000 7f4056b31b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5576ad3be5d0 0x5576ad3bea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:14.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.394+0000 7f4056b31b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5576ad310a20 con 0x5576ad3be5d0 2024-05-03T18:31:14.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.394+0000 7f4056b31b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5576ad2ae760 con 0x5576ad3c27b0 2024-05-03T18:31:14.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.394+0000 7f4056b31b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5576ad30c9e0 con 0x5576ad3c30d0 2024-05-03T18:31:14.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.394+0000 7f4042000700 1 -- 172.21.15.33:0/3797417503 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f40300e2960 con 0x7f404406bb40 2024-05-03T18:31:14.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.394+0000 7f404a200700 1 --2- 172.21.15.33:0/3797417503 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f402c0b2a70 0x7f402c0b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:14.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.395+0000 7f4049800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5576ad3c27b0 0x5576ad3c2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:14.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.395+0000 7f404a200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5576ad3be5d0 0x5576ad3bea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:14.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.395+0000 7f4049800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5576ad3c27b0 0x5576ad3c2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:58424/0 (socket says 172.21.15.33:58424) 2024-05-03T18:31:14.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.395+0000 7f4049800700 1 -- 172.21.15.33:0/1259756268 learned_addr learned my addr 172.21.15.33:0/1259756268 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:31:14.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.395+0000 7f4048e00700 1 --2- 172.21.15.33:0/1259756268 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5576ad3c30d0 0x5576ad3bdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:14.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.395+0000 7f404a200700 1 -- 172.21.15.33:0/1259756268 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5576ad3c30d0 msgr2=0x5576ad3bdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:31:14.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.395+0000 7f404a200700 1 --2- 172.21.15.33:0/1259756268 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5576ad3c30d0 0x5576ad3bdfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:14.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.395+0000 7f404a200700 1 -- 172.21.15.33:0/1259756268 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5576ad3c27b0 msgr2=0x5576ad3c2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:31:14.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.395+0000 7f404a200700 1 --2- 172.21.15.33:0/1259756268 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5576ad3c27b0 0x5576ad3c2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:14.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.395+0000 7f404a200700 1 -- 172.21.15.33:0/1259756268 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576ad25d8d0 con 0x5576ad3be5d0 2024-05-03T18:31:14.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.395+0000 7f4049800700 1 --2- 172.21.15.33:0/1259756268 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5576ad3c27b0 0x5576ad3c2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:31:14.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.396+0000 7f404a200700 1 --2- 172.21.15.33:0/3797417503 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f402c0b2a70 0x7f402c0b4f30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f403c009710 tx=0x7f403c009bf0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:31:14.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.396+0000 7f404a200700 1 --2- 172.21.15.33:0/1259756268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5576ad3be5d0 0x5576ad3bea30 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f403c0060b0 tx=0x7f403c01b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94ad6bb3b840a215 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:31:14.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.396+0000 7f4036a00700 1 -- 172.21.15.33:0/1259756268 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f403c004bb0 con 0x5576ad3be5d0 2024-05-03T18:31:14.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.396+0000 7f4036a00700 1 -- 172.21.15.33:0/1259756268 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f403c01e070 con 0x5576ad3be5d0 2024-05-03T18:31:14.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.396+0000 7f4036a00700 1 -- 172.21.15.33:0/1259756268 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f403c022460 con 0x5576ad3be5d0 2024-05-03T18:31:14.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.397+0000 7f4056b31b80 1 -- 172.21.15.33:0/1259756268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5576ad3be5d0 msgr2=0x5576ad3bea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:31:14.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.397+0000 7f4056b31b80 1 --2- 172.21.15.33:0/1259756268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5576ad3be5d0 0x5576ad3bea30 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f403c0060b0 tx=0x7f403c01b000 comp rx=0 tx=0).stop 2024-05-03T18:31:14.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.397+0000 7f4056b31b80 1 -- 172.21.15.33:0/1259756268 shutdown_connections 2024-05-03T18:31:14.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.397+0000 7f4056b31b80 1 --2- 172.21.15.33:0/1259756268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5576ad3be5d0 0x5576ad3bea30 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:14.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.397+0000 7f4056b31b80 1 --2- 172.21.15.33:0/1259756268 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5576ad3c27b0 0x5576ad3c2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:14.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.397+0000 7f4056b31b80 1 --2- 172.21.15.33:0/1259756268 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5576ad3c30d0 0x5576ad3bdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:14.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.397+0000 7f4056b31b80 1 -- 172.21.15.33:0/1259756268 >> 172.21.15.33:0/1259756268 conn(0x5576ad2ecb60 msgr2=0x5576ad3b87f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:31:14.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.398+0000 7f4056b31b80 1 -- 172.21.15.33:0/1259756268 shutdown_connections 2024-05-03T18:31:14.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.398+0000 7f4056b31b80 1 -- 172.21.15.33:0/1259756268 wait complete. 2024-05-03T18:31:14.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.398+0000 7f4056b31b80 1 -- start start 2024-05-03T18:31:14.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.399+0000 7f4056b31b80 1 Processor -- start 2024-05-03T18:31:14.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.399+0000 7f4056b31b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5576ad3be5d0 0x5576ad3c9f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:14.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.399+0000 7f4056b31b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5576ad3c27b0 0x5576ad3c72f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:14.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.399+0000 7f4056b31b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5576ad3c30d0 0x5576ad3c7830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:14.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.399+0000 7f4056b31b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5576ad3cac40 con 0x5576ad3c27b0 2024-05-03T18:31:14.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.399+0000 7f4056b31b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5576ad276b20 con 0x5576ad3c30d0 2024-05-03T18:31:14.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.399+0000 7f4056b31b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5576ad3cadc0 con 0x5576ad3be5d0 2024-05-03T18:31:14.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.399+0000 7f4056b31b80 10 client.0 _ll_register_callbacks cb 0x5576ad2b7c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:31:14.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.400+0000 7f4048e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5576ad3c27b0 0x5576ad3c72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:14.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.400+0000 7f4049800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5576ad3be5d0 0x5576ad3c9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:14.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.400+0000 7f4049800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5576ad3be5d0 0x5576ad3c9f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:33946/0 (socket says 172.21.15.33:33946) 2024-05-03T18:31:14.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.400+0000 7f4049800700 1 -- 172.21.15.33:0/3226672912 learned_addr learned my addr 172.21.15.33:0/3226672912 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:31:14.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.400+0000 7f404a200700 1 --2- 172.21.15.33:0/3226672912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5576ad3c30d0 0x5576ad3c7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:14.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.400+0000 7f4048e00700 1 -- 172.21.15.33:0/3226672912 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5576ad3be5d0 msgr2=0x5576ad3c9f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:31:14.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.400+0000 7f4048e00700 1 --2- 172.21.15.33:0/3226672912 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5576ad3be5d0 0x5576ad3c9f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:14.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.400+0000 7f4048e00700 1 -- 172.21.15.33:0/3226672912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5576ad3c30d0 msgr2=0x5576ad3c7830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:31:14.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.400+0000 7f4048e00700 1 --2- 172.21.15.33:0/3226672912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5576ad3c30d0 0x5576ad3c7830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:14.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.400+0000 7f4048e00700 1 -- 172.21.15.33:0/3226672912 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576ad2d6420 con 0x5576ad3c27b0 2024-05-03T18:31:14.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.401+0000 7f4048e00700 1 --2- 172.21.15.33:0/3226672912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5576ad3c27b0 0x5576ad3c72f0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f40300dfcc0 tx=0x7f40300d18a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66b9dbf14d479847 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:31:14.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.401+0000 7f402b400700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:31:14.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.401+0000 7f402b400700 1 -- 172.21.15.33:0/3226672912 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f403000f950 con 0x5576ad3c27b0 2024-05-03T18:31:14.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.401+0000 7f402b400700 1 -- 172.21.15.33:0/3226672912 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f40300a9ab0 con 0x5576ad3c27b0 2024-05-03T18:31:14.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.401+0000 7f4056b31b80 1 -- 172.21.15.33:0/3226672912 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f403c01e070 con 0x5576ad3c27b0 2024-05-03T18:31:14.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.402+0000 7f402b400700 1 -- 172.21.15.33:0/3226672912 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f40300a9360 con 0x5576ad3c27b0 2024-05-03T18:31:14.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.402+0000 7f4056b31b80 10 client.15858 fetch_fsmap learned FSMap version 21 2024-05-03T18:31:14.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.402+0000 7f4056b31b80 1 -- 172.21.15.33:0/3226672912 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5576ad3d5510 con 0x5576ad3c27b0 2024-05-03T18:31:14.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.402+0000 7f402b400700 1 -- 172.21.15.33:0/3226672912 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f403c01e070 con 0x5576ad3c27b0 2024-05-03T18:31:14.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.403+0000 7f402b400700 1 -- 172.21.15.33:0/3226672912 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f40300b2220 con 0x5576ad3c27b0 2024-05-03T18:31:14.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.403+0000 7f4056b31b80 10 client.15858 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:31:14.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.403+0000 7f4056b31b80 10 client.15858 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:31:14.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.403+0000 7f4056b31b80 20 client.15858 populate_metadata read hostname 'smithi033' 2024-05-03T18:31:14.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.403+0000 7f4056b31b80 1 --2- 172.21.15.33:0/3226672912 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5576ad3d7aa0 0x5576ad3d9e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:14.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.403+0000 7f4056b31b80 4 client.15858 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:31:14.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.403+0000 7f4056b31b80 1 -- 172.21.15.33:0/3226672912 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5576ad38b1d0 con 0x5576ad3d7aa0 2024-05-03T18:31:14.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.404+0000 7f4049800700 1 --2- 172.21.15.33:0/3226672912 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5576ad3d7aa0 0x5576ad3d9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:14.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.405+0000 7f402b400700 10 client.15858 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:31:14.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.405+0000 7f4049800700 1 --2- 172.21.15.33:0/3226672912 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5576ad3d7aa0 0x5576ad3d9e90 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94a7a850c39a7e71 server_cookie=aac96ae26fed8959 in_seq=0 out_seq=0 2024-05-03T18:31:14.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.408+0000 7f402b400700 1 -- 172.21.15.33:0/3226672912 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5576ad38b1d0 con 0x5576ad3d7aa0 2024-05-03T18:31:14.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.408+0000 7f402b400700 10 client.15858 handle_command_reply: tid=0 2024-05-03T18:31:14.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.408+0000 7f4056b31b80 1 client.15858 shutdown 2024-05-03T18:31:14.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.408+0000 7f4056b31b80 1 -- 172.21.15.33:0/3226672912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5576ad3c27b0 msgr2=0x5576ad3c72f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:31:14.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.408+0000 7f4056b31b80 1 --2- 172.21.15.33:0/3226672912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5576ad3c27b0 0x5576ad3c72f0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f40300dfcc0 tx=0x7f40300d18a0 comp rx=0 tx=0).stop 2024-05-03T18:31:14.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.408+0000 7f4056b31b80 1 -- 172.21.15.33:0/3226672912 shutdown_connections 2024-05-03T18:31:14.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.408+0000 7f4056b31b80 1 --2- 172.21.15.33:0/3226672912 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5576ad3d7aa0 0x5576ad3d9e90 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:14.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.409+0000 7f4056b31b80 1 --2- 172.21.15.33:0/3226672912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5576ad3c30d0 0x5576ad3c7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:14.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.409+0000 7f4056b31b80 1 --2- 172.21.15.33:0/3226672912 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5576ad3be5d0 0x5576ad3c9f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:14.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.409+0000 7f402b400700 0 client.15858 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:31:14.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.409+0000 7f4056b31b80 1 --2- 172.21.15.33:0/3226672912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5576ad3c27b0 0x5576ad3c72f0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:14.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.409+0000 7f4056b31b80 1 -- 172.21.15.33:0/3226672912 >> 172.21.15.33:0/3226672912 conn(0x5576ad2ecb60 msgr2=0x5576ad3c3e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:31:14.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.409+0000 7f4056b31b80 1 -- 172.21.15.33:0/3226672912 shutdown_connections 2024-05-03T18:31:14.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.409+0000 7f4056b31b80 1 -- 172.21.15.33:0/3226672912 wait complete. 2024-05-03T18:31:14.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.409+0000 7f4056b31b80 20 client.15858 trim_cache size 0 max 16384 2024-05-03T18:31:14.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.433+0000 7f4056b31b80 1 Processor -- start 2024-05-03T18:31:14.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.433+0000 7f4056b31b80 1 -- start start 2024-05-03T18:31:14.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.434+0000 7f4056b31b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5576ad3be5d0 0x5576ad3d3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:14.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.434+0000 7f4056b31b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5576ad3c27b0 0x5576ad3c64b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:14.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.434+0000 7f4056b31b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5576ad3c30d0 0x5576ad3bdb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:14.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.434+0000 7f4056b31b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f40441af5d0 con 0x5576ad3c27b0 2024-05-03T18:31:14.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.434+0000 7f4056b31b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f40441b1e50 con 0x5576ad3c30d0 2024-05-03T18:31:14.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.434+0000 7f4056b31b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f40441afc40 con 0x5576ad3be5d0 2024-05-03T18:31:14.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.434+0000 7f4048e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5576ad3c27b0 0x5576ad3c64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:14.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.434+0000 7f4048e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5576ad3c27b0 0x5576ad3c64b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36678/0 (socket says 172.21.15.33:36678) 2024-05-03T18:31:14.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.434+0000 7f4048e00700 1 -- 172.21.15.33:0/2496054572 learned_addr learned my addr 172.21.15.33:0/2496054572 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:31:14.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.435+0000 7f4049800700 1 --2- 172.21.15.33:0/2496054572 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5576ad3be5d0 0x5576ad3d3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:14.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.435+0000 7f404a200700 1 --2- 172.21.15.33:0/2496054572 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5576ad3c30d0 0x5576ad3bdb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:14.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.435+0000 7f4048e00700 1 -- 172.21.15.33:0/2496054572 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5576ad3be5d0 msgr2=0x5576ad3d3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:31:14.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.435+0000 7f4048e00700 1 --2- 172.21.15.33:0/2496054572 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5576ad3be5d0 0x5576ad3d3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:14.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.435+0000 7f4048e00700 1 -- 172.21.15.33:0/2496054572 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5576ad3c30d0 msgr2=0x5576ad3bdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:31:14.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.435+0000 7f4048e00700 1 --2- 172.21.15.33:0/2496054572 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5576ad3c30d0 0x5576ad3bdb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:14.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.435+0000 7f4048e00700 1 -- 172.21.15.33:0/2496054572 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576ad3c7b60 con 0x5576ad3c27b0 2024-05-03T18:31:14.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.435+0000 7f4049800700 1 --2- 172.21.15.33:0/2496054572 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5576ad3be5d0 0x5576ad3d3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:31:14.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.435+0000 7f4048e00700 1 --2- 172.21.15.33:0/2496054572 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5576ad3c27b0 0x5576ad3c64b0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f40300a6ac0 tx=0x7f4030000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b1fb691719b1ec server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:31:14.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.435+0000 7f4036000700 1 -- 172.21.15.33:0/2496054572 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f403000f950 con 0x5576ad3c27b0 2024-05-03T18:31:14.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.436+0000 7f4036000700 1 -- 172.21.15.33:0/2496054572 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f40300a9d00 con 0x5576ad3c27b0 2024-05-03T18:31:14.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.436+0000 7f4036000700 1 -- 172.21.15.33:0/2496054572 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40300b2220 con 0x5576ad3c27b0 2024-05-03T18:31:14.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.436+0000 7f4056b31b80 1 -- 172.21.15.33:0/2496054572 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5576ad3c27b0 msgr2=0x5576ad3c64b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:31:14.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.436+0000 7f4056b31b80 1 --2- 172.21.15.33:0/2496054572 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5576ad3c27b0 0x5576ad3c64b0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f40300a6ac0 tx=0x7f4030000bc0 comp rx=0 tx=0).stop 2024-05-03T18:31:14.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.437+0000 7f4056b31b80 1 -- 172.21.15.33:0/2496054572 shutdown_connections 2024-05-03T18:31:14.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.437+0000 7f4056b31b80 1 --2- 172.21.15.33:0/2496054572 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5576ad3c30d0 0x5576ad3bdb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:14.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.437+0000 7f4056b31b80 1 --2- 172.21.15.33:0/2496054572 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5576ad3be5d0 0x5576ad3d3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:14.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.437+0000 7f4056b31b80 1 --2- 172.21.15.33:0/2496054572 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5576ad3c27b0 0x5576ad3c64b0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:14.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.437+0000 7f4056b31b80 1 -- 172.21.15.33:0/2496054572 >> 172.21.15.33:0/2496054572 conn(0x5576ad2ecb60 msgr2=0x5576ad3bd410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:31:14.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.437+0000 7f4056b31b80 1 -- 172.21.15.33:0/2496054572 shutdown_connections 2024-05-03T18:31:14.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.437+0000 7f4056b31b80 1 -- 172.21.15.33:0/2496054572 wait complete. 2024-05-03T18:31:14.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.437+0000 7f4056b31b80 1 -- start start 2024-05-03T18:31:14.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.438+0000 7f4056b31b80 1 Processor -- start 2024-05-03T18:31:14.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.438+0000 7f4056b31b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5576ad3be5d0 0x5576ad3d0520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:14.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.438+0000 7f4056b31b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5576ad3c27b0 0x5576ad3bb850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:14.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.438+0000 7f4056b31b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5576ad3c30d0 0x5576ad3bbd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:14.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.438+0000 7f4056b31b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f40441b0310 con 0x5576ad3be5d0 2024-05-03T18:31:14.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.438+0000 7f4056b31b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f40441b1780 con 0x5576ad3c30d0 2024-05-03T18:31:14.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.438+0000 7f4056b31b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f40441b09e0 con 0x5576ad3c27b0 2024-05-03T18:31:14.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.439+0000 7f4056b31b80 10 client.0 _ll_register_callbacks cb 0x5576ad2b7c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:31:14.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.439+0000 7f4049800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5576ad3be5d0 0x5576ad3d0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:14.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.439+0000 7f4049800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5576ad3be5d0 0x5576ad3d0520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36684/0 (socket says 172.21.15.33:36684) 2024-05-03T18:31:14.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.439+0000 7f4049800700 1 -- 172.21.15.33:0/3776062090 learned_addr learned my addr 172.21.15.33:0/3776062090 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:31:14.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.439+0000 7f4049800700 1 -- 172.21.15.33:0/3776062090 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5576ad3c27b0 msgr2=0x5576ad3bb850 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-03T18:31:14.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.439+0000 7f4049800700 1 --2- 172.21.15.33:0/3776062090 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5576ad3c27b0 0x5576ad3bb850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:14.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.439+0000 7f4049800700 1 -- 172.21.15.33:0/3776062090 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5576ad3c30d0 msgr2=0x5576ad3bbd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:31:14.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.439+0000 7f4049800700 1 --2- 172.21.15.33:0/3776062090 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5576ad3c30d0 0x5576ad3bbd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:14.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.440+0000 7f4049800700 1 -- 172.21.15.33:0/3776062090 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576ad3bc580 con 0x5576ad3be5d0 2024-05-03T18:31:14.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.440+0000 7f4049800700 1 --2- 172.21.15.33:0/3776062090 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5576ad3be5d0 0x5576ad3d0520 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f4038009750 tx=0x7f4038005530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78db19c50d870628 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:31:14.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.440+0000 7f402b400700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:31:14.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.440+0000 7f402b400700 1 -- 172.21.15.33:0/3776062090 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4038003bd0 con 0x5576ad3be5d0 2024-05-03T18:31:14.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.441+0000 7f4056b31b80 1 -- 172.21.15.33:0/3776062090 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40300a9d00 con 0x5576ad3be5d0 2024-05-03T18:31:14.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.441+0000 7f402b400700 1 -- 172.21.15.33:0/3776062090 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f4038003d70 con 0x5576ad3be5d0 2024-05-03T18:31:14.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.441+0000 7f402b400700 1 -- 172.21.15.33:0/3776062090 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f4038038050 con 0x5576ad3be5d0 2024-05-03T18:31:14.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.441+0000 7f4056b31b80 10 client.15864 fetch_fsmap learned FSMap version 21 2024-05-03T18:31:14.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.441+0000 7f4056b31b80 1 -- 172.21.15.33:0/3776062090 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5576ad44aa80 con 0x5576ad3be5d0 2024-05-03T18:31:14.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.441+0000 7f402b400700 1 -- 172.21.15.33:0/3776062090 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f40300a9d00 con 0x5576ad3be5d0 2024-05-03T18:31:14.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.442+0000 7f402b400700 1 -- 172.21.15.33:0/3776062090 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f4038029340 con 0x5576ad3be5d0 2024-05-03T18:31:14.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.442+0000 7f4056b31b80 10 client.15864 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:31:14.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.442+0000 7f4056b31b80 10 client.15864 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:31:14.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.442+0000 7f4056b31b80 20 client.15864 populate_metadata read hostname 'smithi033' 2024-05-03T18:31:14.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.442+0000 7f4056b31b80 1 --2- 172.21.15.33:0/3776062090 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5576ad44b0a0 0x5576ad44d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:14.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.442+0000 7f4056b31b80 4 client.15864 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-03T18:31:14.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.442+0000 7f4056b31b80 1 -- 172.21.15.33:0/3776062090 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5576ad31e400 con 0x5576ad44b0a0 2024-05-03T18:31:14.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.442+0000 7f4048e00700 1 --2- 172.21.15.33:0/3776062090 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5576ad44b0a0 0x5576ad44d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:14.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.443+0000 7f4048e00700 1 --2- 172.21.15.33:0/3776062090 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5576ad44b0a0 0x5576ad44d4e0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c5752fd80b72b9 server_cookie=34601e25d336e202 in_seq=0 out_seq=0 2024-05-03T18:31:14.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.443+0000 7f402b400700 10 client.15864 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:31:14.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.444+0000 7f402b400700 1 -- 172.21.15.33:0/3776062090 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5576ad31e400 con 0x5576ad44b0a0 2024-05-03T18:31:14.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.444+0000 7f402b400700 10 client.15864 handle_command_reply: tid=0 2024-05-03T18:31:14.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.444+0000 7f4056b31b80 1 client.15864 shutdown 2024-05-03T18:31:14.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.444+0000 7f4056b31b80 1 -- 172.21.15.33:0/3776062090 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5576ad3be5d0 msgr2=0x5576ad3d0520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:31:14.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.444+0000 7f4056b31b80 1 --2- 172.21.15.33:0/3776062090 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5576ad3be5d0 0x5576ad3d0520 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f4038009750 tx=0x7f4038005530 comp rx=0 tx=0).stop 2024-05-03T18:31:14.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.444+0000 7f4056b31b80 1 -- 172.21.15.33:0/3776062090 shutdown_connections 2024-05-03T18:31:14.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.444+0000 7f4056b31b80 1 --2- 172.21.15.33:0/3776062090 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5576ad44b0a0 0x5576ad44d4e0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:14.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.444+0000 7f4056b31b80 1 --2- 172.21.15.33:0/3776062090 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5576ad3c30d0 0x5576ad3bbd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:14.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.444+0000 7f4056b31b80 1 --2- 172.21.15.33:0/3776062090 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5576ad3be5d0 0x5576ad3d0520 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:14.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.444+0000 7f4056b31b80 1 --2- 172.21.15.33:0/3776062090 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5576ad3c27b0 0x5576ad3bb850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:14.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.444+0000 7f4056b31b80 1 -- 172.21.15.33:0/3776062090 >> 172.21.15.33:0/3776062090 conn(0x5576ad2ecb60 msgr2=0x5576ad3bdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:31:14.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.445+0000 7f402b400700 0 client.15864 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:31:14.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.445+0000 7f4056b31b80 1 -- 172.21.15.33:0/3776062090 shutdown_connections 2024-05-03T18:31:14.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.445+0000 7f4056b31b80 1 -- 172.21.15.33:0/3776062090 wait complete. 2024-05-03T18:31:14.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.445+0000 7f4056b31b80 20 client.15864 trim_cache size 0 max 16384 2024-05-03T18:31:14.446 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-03T18:31:14.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.446+0000 7f404c200700 1 -- 172.21.15.33:0/3797417503 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f402c0b2a70 msgr2=0x7f402c0b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:31:14.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.446+0000 7f404c200700 1 --2- 172.21.15.33:0/3797417503 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f402c0b2a70 0x7f402c0b4f30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f403c009710 tx=0x7f403c009bf0 comp rx=0 tx=0).stop 2024-05-03T18:31:14.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.446+0000 7f404c200700 1 -- 172.21.15.33:0/3797417503 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f404406bb40 msgr2=0x7f40441029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:31:14.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.446+0000 7f404c200700 1 --2- 172.21.15.33:0/3797417503 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f404406bb40 0x7f40441029b0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f403000c9b0 tx=0x7f403000ccc0 comp rx=0 tx=0).stop 2024-05-03T18:31:14.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.446+0000 7f404c200700 1 -- 172.21.15.33:0/3797417503 shutdown_connections 2024-05-03T18:31:14.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.446+0000 7f404c200700 1 --2- 172.21.15.33:0/3797417503 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4044109510 0x7f4044102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:14.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.446+0000 7f404c200700 1 --2- 172.21.15.33:0/3797417503 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f402c0b2a70 0x7f402c0b4f30 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:14.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.446+0000 7f404c200700 1 --2- 172.21.15.33:0/3797417503 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f404406b180 0x7f4044102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:14.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.446+0000 7f404c200700 1 --2- 172.21.15.33:0/3797417503 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f404406bb40 0x7f40441029b0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:14.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.446+0000 7f404c200700 1 -- 172.21.15.33:0/3797417503 >> 172.21.15.33:0/3797417503 conn(0x7f4044079350 msgr2=0x7f40441010f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:31:14.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.446+0000 7f404c200700 1 -- 172.21.15.33:0/3797417503 shutdown_connections 2024-05-03T18:31:14.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:14.446+0000 7f404c200700 1 -- 172.21.15.33:0/3797417503 wait complete. 2024-05-03T18:31:14.681 INFO:teuthology.orchestra.run.smithi033.stderr:done in 172.15 s (drop tables 0.03 s, create tables 0.21 s, client-side generate 112.22 s, vacuum 38.02 s, primary keys 21.68 s). 2024-05-03T18:31:14.696 DEBUG:teuthology.orchestra.run.smithi033:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900' 2024-05-03T18:31:15.110 INFO:teuthology.orchestra.run.smithi033.stderr:starting vacuum...end. 2024-05-03T18:31:16.337 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-03T18:31:16.338 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-03T18:31:16.977 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:31:17.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:31:16 smithi033 ceph-mon[28330]: pgmap v125: 129 pgs: 129 active+clean; 1.4 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 13 MiB/s wr, 11 op/s 2024-05-03T18:31:17.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:31:16 smithi099 ceph-mon[31658]: pgmap v125: 129 pgs: 129 active+clean; 1.4 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 13 MiB/s wr, 11 op/s 2024-05-03T18:31:17.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:31:16 smithi150 ceph-mon[32492]: pgmap v125: 129 pgs: 129 active+clean; 1.4 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 13 MiB/s wr, 11 op/s 2024-05-03T18:31:19.202 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:31:18 smithi033 ceph-mon[28330]: pgmap v126: 129 pgs: 129 active+clean; 1.4 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 15 MiB/s wr, 42 op/s 2024-05-03T18:31:19.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:31:18 smithi099 ceph-mon[31658]: pgmap v126: 129 pgs: 129 active+clean; 1.4 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 15 MiB/s wr, 42 op/s 2024-05-03T18:31:19.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:31:18 smithi150 ceph-mon[32492]: pgmap v126: 129 pgs: 129 active+clean; 1.4 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 15 MiB/s wr, 42 op/s 2024-05-03T18:31:19.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.639+0000 7f07bf400700 1 -- 172.21.15.33:0/2987149788 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07b810ff50 msgr2=0x7f07b8112340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:31:19.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.639+0000 7f07bf400700 1 --2- 172.21.15.33:0/2987149788 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07b810ff50 0x7f07b8112340 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f07b000b0a0 tx=0x7f07b000b3b0 comp rx=0 tx=0).stop 2024-05-03T18:31:19.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.640+0000 7f07bf400700 1 -- 172.21.15.33:0/2987149788 shutdown_connections 2024-05-03T18:31:19.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.640+0000 7f07bf400700 1 --2- 172.21.15.33:0/2987149788 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07b810ff50 0x7f07b8112340 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:19.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.640+0000 7f07bf400700 1 --2- 172.21.15.33:0/2987149788 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07b806b1b0 0x7f07b806b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:19.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.640+0000 7f07bf400700 1 --2- 172.21.15.33:0/2987149788 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f07b806bad0 0x7f07b810fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:19.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.640+0000 7f07bf400700 1 -- 172.21.15.33:0/2987149788 >> 172.21.15.33:0/2987149788 conn(0x7f07b8079350 msgr2=0x7f07b8079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:31:19.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.640+0000 7f07bf400700 1 -- 172.21.15.33:0/2987149788 shutdown_connections 2024-05-03T18:31:19.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.640+0000 7f07bf400700 1 -- 172.21.15.33:0/2987149788 wait complete. 2024-05-03T18:31:19.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.643+0000 7f07bf400700 1 Processor -- start 2024-05-03T18:31:19.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.643+0000 7f07bf400700 1 -- start start 2024-05-03T18:31:19.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.644+0000 7f07bf400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07b806b1b0 0x7f07b81a16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:19.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.644+0000 7f07bf400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f07b806bad0 0x7f07b81a1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:19.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.644+0000 7f07bf400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07b810ff50 0x7f07b819b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:19.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.644+0000 7f07bf400700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f07b8115770 con 0x7f07b806b1b0 2024-05-03T18:31:19.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.644+0000 7f07bf400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f07b81155f0 con 0x7f07b810ff50 2024-05-03T18:31:19.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.644+0000 7f07bf400700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f07b81158f0 con 0x7f07b806bad0 2024-05-03T18:31:19.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.645+0000 7f07bca00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07b806b1b0 0x7f07b81a16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:19.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.645+0000 7f07bca00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07b806b1b0 0x7f07b81a16b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46088/0 (socket says 172.21.15.33:46088) 2024-05-03T18:31:19.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.645+0000 7f07bca00700 1 -- 172.21.15.33:0/3066801408 learned_addr learned my addr 172.21.15.33:0/3066801408 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:31:19.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.645+0000 7f07bd400700 1 --2- 172.21.15.33:0/3066801408 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07b810ff50 0x7f07b819b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:19.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.645+0000 7f07b7e00700 1 --2- 172.21.15.33:0/3066801408 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f07b806bad0 0x7f07b81a1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:19.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.646+0000 7f07bca00700 1 -- 172.21.15.33:0/3066801408 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f07b806bad0 msgr2=0x7f07b81a1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:31:19.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.646+0000 7f07bca00700 1 --2- 172.21.15.33:0/3066801408 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f07b806bad0 0x7f07b81a1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:19.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.646+0000 7f07bca00700 1 -- 172.21.15.33:0/3066801408 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07b810ff50 msgr2=0x7f07b819b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:31:19.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.646+0000 7f07bca00700 1 --2- 172.21.15.33:0/3066801408 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07b810ff50 0x7f07b819b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:19.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.646+0000 7f07bca00700 1 -- 172.21.15.33:0/3066801408 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07b819bfd0 con 0x7f07b806b1b0 2024-05-03T18:31:19.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.646+0000 7f07bd400700 1 --2- 172.21.15.33:0/3066801408 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07b810ff50 0x7f07b819b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:31:19.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.647+0000 7f07bca00700 1 --2- 172.21.15.33:0/3066801408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07b806b1b0 0x7f07b81a16b0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f07ac00d950 tx=0x7f07ac00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:31:19.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.649+0000 7f07b5600700 1 -- 172.21.15.33:0/3066801408 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07ac016070 con 0x7f07b806b1b0 2024-05-03T18:31:19.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.650+0000 7f07bf400700 1 -- 172.21.15.33:0/3066801408 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07b819c260 con 0x7f07b806b1b0 2024-05-03T18:31:19.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.650+0000 7f07bf400700 1 -- 172.21.15.33:0/3066801408 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07b81a8580 con 0x7f07b806b1b0 2024-05-03T18:31:19.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.650+0000 7f07b5600700 1 -- 172.21.15.33:0/3066801408 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f07ac009dd0 con 0x7f07b806b1b0 2024-05-03T18:31:19.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.650+0000 7f07b5600700 1 -- 172.21.15.33:0/3066801408 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07ac00f3c0 con 0x7f07b806b1b0 2024-05-03T18:31:19.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.658+0000 7f07b5600700 1 -- 172.21.15.33:0/3066801408 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f07ac019070 con 0x7f07b806b1b0 2024-05-03T18:31:19.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.660+0000 7f07b5600700 1 --2- 172.21.15.33:0/3066801408 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f07a00b2c70 0x7f07a00b5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:19.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.660+0000 7f07b5600700 1 -- 172.21.15.33:0/3066801408 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f07ac010cd0 con 0x7f07b806b1b0 2024-05-03T18:31:19.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.661+0000 7f07c9dd2b80 1 Processor -- start 2024-05-03T18:31:19.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.661+0000 7f07b7e00700 1 --2- 172.21.15.33:0/3066801408 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f07a00b2c70 0x7f07a00b5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:19.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.662+0000 7f07c9dd2b80 1 -- start start 2024-05-03T18:31:19.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.662+0000 7f07c9dd2b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560d41bad1c0 0x560d41bad5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:19.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.662+0000 7f07c9dd2b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560d41badae0 0x560d41ba77c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:19.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.662+0000 7f07c9dd2b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d41ba7de0 0x560d41ba8260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:19.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.662+0000 7f07c9dd2b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560d41af1aa0 con 0x560d41ba7de0 2024-05-03T18:31:19.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.662+0000 7f07c9dd2b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x560d41a8f7e0 con 0x560d41badae0 2024-05-03T18:31:19.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.662+0000 7f07c9dd2b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x560d41aeda60 con 0x560d41bad1c0 2024-05-03T18:31:19.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.663+0000 7f07bca00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560d41bad1c0 0x560d41bad5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:19.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.663+0000 7f07bca00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560d41bad1c0 0x560d41bad5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:56212/0 (socket says 172.21.15.33:56212) 2024-05-03T18:31:19.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.663+0000 7f07bca00700 1 -- 172.21.15.33:0/1472692336 learned_addr learned my addr 172.21.15.33:0/1472692336 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:31:19.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.663+0000 7f07bd400700 1 --2- 172.21.15.33:0/1472692336 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d41ba7de0 0x560d41ba8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:19.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.664+0000 7f07bca00700 1 -- 172.21.15.33:0/1472692336 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560d41badae0 msgr2=0x560d41ba77c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-03T18:31:19.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.664+0000 7f07bca00700 1 --2- 172.21.15.33:0/1472692336 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560d41badae0 0x560d41ba77c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:19.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.664+0000 7f07bca00700 1 -- 172.21.15.33:0/1472692336 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d41ba7de0 msgr2=0x560d41ba8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:31:19.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.664+0000 7f07bca00700 1 --2- 172.21.15.33:0/1472692336 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d41ba7de0 0x560d41ba8260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:19.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.664+0000 7f07bca00700 1 -- 172.21.15.33:0/1472692336 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d41a3e950 con 0x560d41bad1c0 2024-05-03T18:31:19.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.664+0000 7f07b7e00700 1 --2- 172.21.15.33:0/3066801408 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f07a00b2c70 0x7f07a00b5130 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f07a4005fd0 tx=0x7f07a400af20 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:31:19.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.664+0000 7f07bca00700 1 --2- 172.21.15.33:0/1472692336 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560d41bad1c0 0x560d41bad5a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f07ac0a5330 tx=0x7f07ac0a9a30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cabc0088cea2a9b server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:31:19.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.665+0000 7f07aa000700 1 -- 172.21.15.33:0/1472692336 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07ac020040 con 0x560d41bad1c0 2024-05-03T18:31:19.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.665+0000 7f07aa000700 1 -- 172.21.15.33:0/1472692336 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f07ac0ad660 con 0x560d41bad1c0 2024-05-03T18:31:19.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.665+0000 7f07aa000700 1 -- 172.21.15.33:0/1472692336 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07ac0b1c80 con 0x560d41bad1c0 2024-05-03T18:31:19.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.667+0000 7f07c9dd2b80 1 -- 172.21.15.33:0/1472692336 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560d41bad1c0 msgr2=0x560d41bad5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:31:19.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.667+0000 7f07c9dd2b80 1 --2- 172.21.15.33:0/1472692336 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560d41bad1c0 0x560d41bad5a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f07ac0a5330 tx=0x7f07ac0a9a30 comp rx=0 tx=0).stop 2024-05-03T18:31:19.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.667+0000 7f07c9dd2b80 1 -- 172.21.15.33:0/1472692336 shutdown_connections 2024-05-03T18:31:19.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.667+0000 7f07c9dd2b80 1 --2- 172.21.15.33:0/1472692336 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d41ba7de0 0x560d41ba8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:19.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.667+0000 7f07c9dd2b80 1 --2- 172.21.15.33:0/1472692336 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560d41bad1c0 0x560d41bad5a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:19.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.667+0000 7f07c9dd2b80 1 --2- 172.21.15.33:0/1472692336 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560d41badae0 0x560d41ba77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:19.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.667+0000 7f07c9dd2b80 1 -- 172.21.15.33:0/1472692336 >> 172.21.15.33:0/1472692336 conn(0x560d41b17750 msgr2=0x560d41b99980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:31:19.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.668+0000 7f07c9dd2b80 1 -- 172.21.15.33:0/1472692336 shutdown_connections 2024-05-03T18:31:19.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.668+0000 7f07c9dd2b80 1 -- 172.21.15.33:0/1472692336 wait complete. 2024-05-03T18:31:19.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.669+0000 7f07c9dd2b80 1 -- start start 2024-05-03T18:31:19.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.669+0000 7f07c9dd2b80 1 Processor -- start 2024-05-03T18:31:19.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.670+0000 7f07c9dd2b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d41ba7de0 0x560d41bb36b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:19.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.670+0000 7f07c9dd2b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560d41bad1c0 0x560d41bb0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:19.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.670+0000 7f07c9dd2b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560d41badae0 0x560d41bb0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:19.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.670+0000 7f07c9dd2b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560d41bc55a0 con 0x560d41ba7de0 2024-05-03T18:31:19.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.670+0000 7f07c9dd2b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x560d41a57ba0 con 0x560d41bad1c0 2024-05-03T18:31:19.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.670+0000 7f07c9dd2b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x560d41bc5720 con 0x560d41badae0 2024-05-03T18:31:19.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.670+0000 7f07c9dd2b80 10 client.0 _ll_register_callbacks cb 0x560d41a98cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:31:19.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.670+0000 7f07bca00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d41ba7de0 0x560d41bb36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:19.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.671+0000 7f07bd400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560d41badae0 0x560d41bb0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:19.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.671+0000 7f07bd400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560d41badae0 0x560d41bb0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:56218/0 (socket says 172.21.15.33:56218) 2024-05-03T18:31:19.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.671+0000 7f07bd400700 1 -- 172.21.15.33:0/3754228511 learned_addr learned my addr 172.21.15.33:0/3754228511 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:31:19.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.671+0000 7f07b7e00700 1 --2- 172.21.15.33:0/3754228511 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560d41bad1c0 0x560d41bb0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:19.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.671+0000 7f07bd400700 1 -- 172.21.15.33:0/3754228511 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560d41bad1c0 msgr2=0x560d41bb0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:31:19.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.671+0000 7f07bd400700 1 --2- 172.21.15.33:0/3754228511 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560d41bad1c0 0x560d41bb0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:19.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.671+0000 7f07bd400700 1 -- 172.21.15.33:0/3754228511 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d41ba7de0 msgr2=0x560d41bb36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:31:19.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.672+0000 7f07bd400700 1 --2- 172.21.15.33:0/3754228511 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d41ba7de0 0x560d41bb36b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:19.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.672+0000 7f07bd400700 1 -- 172.21.15.33:0/3754228511 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d41ab74a0 con 0x560d41badae0 2024-05-03T18:31:19.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.672+0000 7f07bca00700 1 --2- 172.21.15.33:0/3754228511 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d41ba7de0 0x560d41bb36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:31:19.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.673+0000 7f07bd400700 1 --2- 172.21.15.33:0/3754228511 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560d41badae0 0x560d41bb0f90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f07b000b0a0 tx=0x7f07b0007a80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=684a6523b74419fb server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:31:19.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.673+0000 7f079ea00700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-03T18:31:19.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.673+0000 7f079ea00700 1 -- 172.21.15.33:0/3754228511 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b0007ca0 con 0x560d41badae0 2024-05-03T18:31:19.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.673+0000 7f079ea00700 1 -- 172.21.15.33:0/3754228511 <== mon.1 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f07b0026470 con 0x560d41badae0 2024-05-03T18:31:19.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.674+0000 7f07c9dd2b80 1 -- 172.21.15.33:0/3754228511 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07ac0ad660 con 0x560d41badae0 2024-05-03T18:31:19.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.675+0000 7f079ea00700 1 -- 172.21.15.33:0/3754228511 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f07b0026670 con 0x560d41badae0 2024-05-03T18:31:19.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.675+0000 7f07c9dd2b80 10 client.25516 fetch_fsmap learned FSMap version 21 2024-05-03T18:31:19.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.675+0000 7f07c9dd2b80 1 -- 172.21.15.33:0/3754228511 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x560d41bcfd00 con 0x560d41badae0 2024-05-03T18:31:19.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.677+0000 7f079ea00700 1 -- 172.21.15.33:0/3754228511 <== mon.1 v2:172.21.15.150:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f07ac0ad660 con 0x560d41badae0 2024-05-03T18:31:19.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.677+0000 7f079ea00700 1 -- 172.21.15.33:0/3754228511 <== mon.1 v2:172.21.15.150:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f07b0011070 con 0x560d41badae0 2024-05-03T18:31:19.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.677+0000 7f07c9dd2b80 10 client.25516 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:31:19.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.677+0000 7f07c9dd2b80 10 client.25516 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:31:19.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.677+0000 7f07c9dd2b80 20 client.25516 populate_metadata read hostname 'smithi033' 2024-05-03T18:31:19.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.677+0000 7f07c9dd2b80 1 --2- 172.21.15.33:0/3754228511 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x560d41bd2390 0x560d41bd4780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:19.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.677+0000 7f07c9dd2b80 4 client.25516 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:31:19.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.677+0000 7f07c9dd2b80 1 -- 172.21.15.33:0/3754228511 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560d41b6c070 con 0x560d41bd2390 2024-05-03T18:31:19.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.678+0000 7f07bca00700 1 --2- 172.21.15.33:0/3754228511 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x560d41bd2390 0x560d41bd4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:19.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.679+0000 7f07bca00700 1 --2- 172.21.15.33:0/3754228511 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x560d41bd2390 0x560d41bd4780 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6b1f0ba3da17bf1 server_cookie=f3001d0f4cecaa5 in_seq=0 out_seq=0 2024-05-03T18:31:19.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.679+0000 7f079ea00700 10 client.25516 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:31:19.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.682+0000 7f079ea00700 1 -- 172.21.15.33:0/3754228511 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560d41b6c070 con 0x560d41bd2390 2024-05-03T18:31:19.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.682+0000 7f079ea00700 10 client.25516 handle_command_reply: tid=0 2024-05-03T18:31:19.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.682+0000 7f07c9dd2b80 1 client.25516 shutdown 2024-05-03T18:31:19.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.683+0000 7f07c9dd2b80 1 -- 172.21.15.33:0/3754228511 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560d41badae0 msgr2=0x560d41bb0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:31:19.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.683+0000 7f07c9dd2b80 1 --2- 172.21.15.33:0/3754228511 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560d41badae0 0x560d41bb0f90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f07b000b0a0 tx=0x7f07b0007a80 comp rx=0 tx=0).stop 2024-05-03T18:31:19.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.683+0000 7f07c9dd2b80 1 -- 172.21.15.33:0/3754228511 shutdown_connections 2024-05-03T18:31:19.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.683+0000 7f07c9dd2b80 1 --2- 172.21.15.33:0/3754228511 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x560d41bd2390 0x560d41bd4780 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:19.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.683+0000 7f07c9dd2b80 1 --2- 172.21.15.33:0/3754228511 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560d41badae0 0x560d41bb0f90 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:19.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.683+0000 7f07c9dd2b80 1 --2- 172.21.15.33:0/3754228511 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d41ba7de0 0x560d41bb36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:19.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.683+0000 7f07c9dd2b80 1 --2- 172.21.15.33:0/3754228511 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560d41bad1c0 0x560d41bb0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:19.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.683+0000 7f07c9dd2b80 1 -- 172.21.15.33:0/3754228511 >> 172.21.15.33:0/3754228511 conn(0x560d41b17750 msgr2=0x560d41b94430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:31:19.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.683+0000 7f079ea00700 0 client.25516 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:31:19.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.684+0000 7f07c9dd2b80 1 -- 172.21.15.33:0/3754228511 shutdown_connections 2024-05-03T18:31:19.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.685+0000 7f07c9dd2b80 1 -- 172.21.15.33:0/3754228511 wait complete. 2024-05-03T18:31:19.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.685+0000 7f07c9dd2b80 20 client.25516 trim_cache size 0 max 16384 2024-05-03T18:31:19.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.722+0000 7f07c9dd2b80 1 Processor -- start 2024-05-03T18:31:19.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.723+0000 7f07c9dd2b80 1 -- start start 2024-05-03T18:31:19.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.723+0000 7f07c9dd2b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560d41ba7de0 0x560d41ba77c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:19.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.723+0000 7f07c9dd2b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560d41bad1c0 0x560d41bae5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:19.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.723+0000 7f07c9dd2b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d41badae0 0x560d41bda050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:19.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.723+0000 7f07c9dd2b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f079800cd90 con 0x560d41badae0 2024-05-03T18:31:19.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.723+0000 7f07c9dd2b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f079800f610 con 0x560d41bad1c0 2024-05-03T18:31:19.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.723+0000 7f07c9dd2b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f079800d400 con 0x560d41ba7de0 2024-05-03T18:31:19.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.724+0000 7f07b7e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560d41bad1c0 0x560d41bae5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:19.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.724+0000 7f07b7e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560d41bad1c0 0x560d41bae5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:39096/0 (socket says 172.21.15.33:39096) 2024-05-03T18:31:19.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.724+0000 7f07b7e00700 1 -- 172.21.15.33:0/4210636095 learned_addr learned my addr 172.21.15.33:0/4210636095 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:31:19.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.725+0000 7f07bd400700 1 --2- 172.21.15.33:0/4210636095 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d41badae0 0x560d41bda050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:19.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.725+0000 7f07bca00700 1 --2- 172.21.15.33:0/4210636095 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560d41ba7de0 0x560d41ba77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:19.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.725+0000 7f07b7e00700 1 -- 172.21.15.33:0/4210636095 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560d41ba7de0 msgr2=0x560d41ba77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:31:19.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.725+0000 7f07b7e00700 1 --2- 172.21.15.33:0/4210636095 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560d41ba7de0 0x560d41ba77c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:19.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.725+0000 7f07b7e00700 1 -- 172.21.15.33:0/4210636095 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d41badae0 msgr2=0x560d41bda050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:31:19.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.725+0000 7f07b7e00700 1 --2- 172.21.15.33:0/4210636095 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d41badae0 0x560d41bda050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:19.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.725+0000 7f07b7e00700 1 -- 172.21.15.33:0/4210636095 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d41bb32f0 con 0x560d41bad1c0 2024-05-03T18:31:19.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.726+0000 7f07bca00700 1 --2- 172.21.15.33:0/4210636095 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560d41ba7de0 0x560d41ba77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:31:19.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.726+0000 7f07b7e00700 1 --2- 172.21.15.33:0/4210636095 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560d41bad1c0 0x560d41bae5d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f07a400ed00 tx=0x7f07a400af90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=149a4a43e89a9a71 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:31:19.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.727+0000 7f07a9600700 1 -- 172.21.15.33:0/4210636095 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a4010d30 con 0x560d41bad1c0 2024-05-03T18:31:19.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.727+0000 7f07a9600700 1 -- 172.21.15.33:0/4210636095 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f07a401e8c0 con 0x560d41bad1c0 2024-05-03T18:31:19.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.727+0000 7f07a9600700 1 -- 172.21.15.33:0/4210636095 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a40267f0 con 0x560d41bad1c0 2024-05-03T18:31:19.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.728+0000 7f07c9dd2b80 1 -- 172.21.15.33:0/4210636095 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560d41bad1c0 msgr2=0x560d41bae5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:31:19.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.728+0000 7f07c9dd2b80 1 --2- 172.21.15.33:0/4210636095 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560d41bad1c0 0x560d41bae5d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f07a400ed00 tx=0x7f07a400af90 comp rx=0 tx=0).stop 2024-05-03T18:31:19.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.729+0000 7f07c9dd2b80 1 -- 172.21.15.33:0/4210636095 shutdown_connections 2024-05-03T18:31:19.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.729+0000 7f07c9dd2b80 1 --2- 172.21.15.33:0/4210636095 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d41badae0 0x560d41bda050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:19.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.729+0000 7f07c9dd2b80 1 --2- 172.21.15.33:0/4210636095 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560d41ba7de0 0x560d41ba77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:19.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.729+0000 7f07c9dd2b80 1 --2- 172.21.15.33:0/4210636095 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560d41bad1c0 0x560d41bae5d0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:19.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.729+0000 7f07c9dd2b80 1 -- 172.21.15.33:0/4210636095 >> 172.21.15.33:0/4210636095 conn(0x560d41b17750 msgr2=0x560d41badec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:31:19.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.729+0000 7f07c9dd2b80 1 -- 172.21.15.33:0/4210636095 shutdown_connections 2024-05-03T18:31:19.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.730+0000 7f07c9dd2b80 1 -- 172.21.15.33:0/4210636095 wait complete. 2024-05-03T18:31:19.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.730+0000 7f07c9dd2b80 1 -- start start 2024-05-03T18:31:19.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.731+0000 7f07c9dd2b80 1 Processor -- start 2024-05-03T18:31:19.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.732+0000 7f07c9dd2b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d41ba7de0 0x560d41bc79b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:19.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.733+0000 7f07c9dd2b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560d41bad1c0 0x560d41bc7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:19.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.733+0000 7f07c9dd2b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560d41badae0 0x560d41bc8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:19.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.733+0000 7f07c9dd2b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f079800dad0 con 0x560d41ba7de0 2024-05-03T18:31:19.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.733+0000 7f07c9dd2b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f079800ef40 con 0x560d41badae0 2024-05-03T18:31:19.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.733+0000 7f07c9dd2b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f079800e1a0 con 0x560d41bad1c0 2024-05-03T18:31:19.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.733+0000 7f07c9dd2b80 10 client.0 _ll_register_callbacks cb 0x560d41a98cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:31:19.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.734+0000 7f07b7e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560d41bad1c0 0x560d41bc7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:19.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.734+0000 7f07bca00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d41ba7de0 0x560d41bc79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:19.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.734+0000 7f07bca00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d41ba7de0 0x560d41bc79b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46120/0 (socket says 172.21.15.33:46120) 2024-05-03T18:31:19.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.734+0000 7f07bca00700 1 -- 172.21.15.33:0/1899472244 learned_addr learned my addr 172.21.15.33:0/1899472244 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:31:19.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.735+0000 7f07bd400700 1 --2- 172.21.15.33:0/1899472244 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560d41badae0 0x560d41bc8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:19.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.735+0000 7f07bca00700 1 -- 172.21.15.33:0/1899472244 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560d41bad1c0 msgr2=0x560d41bc7ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:31:19.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.735+0000 7f07bca00700 1 --2- 172.21.15.33:0/1899472244 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560d41bad1c0 0x560d41bc7ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:19.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.735+0000 7f07bca00700 1 -- 172.21.15.33:0/1899472244 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560d41badae0 msgr2=0x560d41bc8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:31:19.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.735+0000 7f07bca00700 1 --2- 172.21.15.33:0/1899472244 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560d41badae0 0x560d41bc8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:19.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.735+0000 7f07bca00700 1 -- 172.21.15.33:0/1899472244 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d41c3ce70 con 0x560d41ba7de0 2024-05-03T18:31:19.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.735+0000 7f07b7e00700 1 --2- 172.21.15.33:0/1899472244 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560d41bad1c0 0x560d41bc7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:31:19.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.736+0000 7f07bd400700 1 --2- 172.21.15.33:0/1899472244 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560d41badae0 0x560d41bc8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:31:19.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.736+0000 7f07bca00700 1 --2- 172.21.15.33:0/1899472244 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d41ba7de0 0x560d41bc79b0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f07ac0e2330 tx=0x7f07ac0dfc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5235fa01628cb2c server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:31:19.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.736+0000 7f079ea00700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:31:19.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.737+0000 7f079ea00700 1 -- 172.21.15.33:0/1899472244 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07ac06d060 con 0x560d41ba7de0 2024-05-03T18:31:19.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.737+0000 7f079ea00700 1 -- 172.21.15.33:0/1899472244 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f07ac0a9720 con 0x560d41ba7de0 2024-05-03T18:31:19.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.737+0000 7f07c9dd2b80 1 -- 172.21.15.33:0/1899472244 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07a401e8c0 con 0x560d41ba7de0 2024-05-03T18:31:19.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.738+0000 7f079ea00700 1 -- 172.21.15.33:0/1899472244 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f07ac0c2070 con 0x560d41ba7de0 2024-05-03T18:31:19.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.738+0000 7f07c9dd2b80 10 client.15879 fetch_fsmap learned FSMap version 21 2024-05-03T18:31:19.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.738+0000 7f07c9dd2b80 1 -- 172.21.15.33:0/1899472244 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x560d41c3e0e0 con 0x560d41ba7de0 2024-05-03T18:31:19.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.738+0000 7f079ea00700 1 -- 172.21.15.33:0/1899472244 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f07a401e8c0 con 0x560d41ba7de0 2024-05-03T18:31:19.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.738+0000 7f079ea00700 1 -- 172.21.15.33:0/1899472244 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f07ac0a8510 con 0x560d41ba7de0 2024-05-03T18:31:19.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.739+0000 7f07c9dd2b80 10 client.15879 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:31:19.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.739+0000 7f07c9dd2b80 10 client.15879 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:31:19.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.739+0000 7f07c9dd2b80 20 client.15879 populate_metadata read hostname 'smithi033' 2024-05-03T18:31:19.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.739+0000 7f07c9dd2b80 1 --2- 172.21.15.33:0/1899472244 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x560d41c3e3d0 0x560d41c407c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:19.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.739+0000 7f07c9dd2b80 4 client.15879 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-03T18:31:19.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.739+0000 7f07c9dd2b80 1 -- 172.21.15.33:0/1899472244 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x560d41aff480 con 0x560d41c3e3d0 2024-05-03T18:31:19.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.739+0000 7f07b7e00700 1 --2- 172.21.15.33:0/1899472244 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x560d41c3e3d0 0x560d41c407c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:19.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.740+0000 7f07b7e00700 1 --2- 172.21.15.33:0/1899472244 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x560d41c3e3d0 0x560d41c407c0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f587265021afd11c server_cookie=9269916e57f50b0 in_seq=0 out_seq=0 2024-05-03T18:31:19.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.741+0000 7f079ea00700 10 client.15879 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:31:19.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.747+0000 7f079ea00700 1 -- 172.21.15.33:0/1899472244 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560d41aff480 con 0x560d41c3e3d0 2024-05-03T18:31:19.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.747+0000 7f079ea00700 10 client.15879 handle_command_reply: tid=0 2024-05-03T18:31:19.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.747+0000 7f07c9dd2b80 1 client.15879 shutdown 2024-05-03T18:31:19.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.747+0000 7f07c9dd2b80 1 -- 172.21.15.33:0/1899472244 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d41ba7de0 msgr2=0x560d41bc79b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:31:19.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.747+0000 7f07c9dd2b80 1 --2- 172.21.15.33:0/1899472244 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d41ba7de0 0x560d41bc79b0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f07ac0e2330 tx=0x7f07ac0dfc60 comp rx=0 tx=0).stop 2024-05-03T18:31:19.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.747+0000 7f07c9dd2b80 1 -- 172.21.15.33:0/1899472244 shutdown_connections 2024-05-03T18:31:19.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.747+0000 7f07c9dd2b80 1 --2- 172.21.15.33:0/1899472244 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x560d41c3e3d0 0x560d41c407c0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:19.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.747+0000 7f07c9dd2b80 1 --2- 172.21.15.33:0/1899472244 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560d41badae0 0x560d41bc8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:19.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.747+0000 7f07c9dd2b80 1 --2- 172.21.15.33:0/1899472244 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d41ba7de0 0x560d41bc79b0 secure :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f07ac0e2330 tx=0x7f07ac0dfc60 comp rx=0 tx=0).stop 2024-05-03T18:31:19.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.747+0000 7f07c9dd2b80 1 --2- 172.21.15.33:0/1899472244 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560d41bad1c0 0x560d41bc7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:19.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.747+0000 7f07c9dd2b80 1 -- 172.21.15.33:0/1899472244 >> 172.21.15.33:0/1899472244 conn(0x560d41b17750 msgr2=0x560d41bae5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:31:19.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.748+0000 7f079ea00700 0 client.15879 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:31:19.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.748+0000 7f07c9dd2b80 1 -- 172.21.15.33:0/1899472244 shutdown_connections 2024-05-03T18:31:19.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.749+0000 7f07c9dd2b80 1 -- 172.21.15.33:0/1899472244 wait complete. 2024-05-03T18:31:19.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.749+0000 7f07c9dd2b80 20 client.15879 trim_cache size 0 max 16384 2024-05-03T18:31:19.751 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-03T18:31:19.751 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-05-03T18:31:19.751 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "d4de8331-08bc-466c-980b-29a9457212cd", 2024-05-03T18:31:19.751 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-05-03T18:31:19.751 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-03T18:31:19.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.751+0000 7f07bf400700 1 -- 172.21.15.33:0/3066801408 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f07a00b2c70 msgr2=0x7f07a00b5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:31:19.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.751+0000 7f07bf400700 1 --2- 172.21.15.33:0/3066801408 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f07a00b2c70 0x7f07a00b5130 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f07a4005fd0 tx=0x7f07a400af20 comp rx=0 tx=0).stop 2024-05-03T18:31:19.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.751+0000 7f07bf400700 1 -- 172.21.15.33:0/3066801408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07b806b1b0 msgr2=0x7f07b81a16b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:31:19.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.751+0000 7f07bf400700 1 --2- 172.21.15.33:0/3066801408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07b806b1b0 0x7f07b81a16b0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f07ac00d950 tx=0x7f07ac00dc60 comp rx=0 tx=0).stop 2024-05-03T18:31:19.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.752+0000 7f07bf400700 1 -- 172.21.15.33:0/3066801408 shutdown_connections 2024-05-03T18:31:19.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.752+0000 7f07bf400700 1 --2- 172.21.15.33:0/3066801408 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f07a00b2c70 0x7f07a00b5130 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:19.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.752+0000 7f07bf400700 1 --2- 172.21.15.33:0/3066801408 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07b810ff50 0x7f07b819b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:19.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.752+0000 7f07bf400700 1 --2- 172.21.15.33:0/3066801408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07b806b1b0 0x7f07b81a16b0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:19.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.752+0000 7f07bf400700 1 --2- 172.21.15.33:0/3066801408 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f07b806bad0 0x7f07b81a1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:19.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.752+0000 7f07bf400700 1 -- 172.21.15.33:0/3066801408 >> 172.21.15.33:0/3066801408 conn(0x7f07b8079350 msgr2=0x7f07b8111220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:31:19.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.752+0000 7f07bf400700 1 -- 172.21.15.33:0/3066801408 shutdown_connections 2024-05-03T18:31:19.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:19.753+0000 7f07bf400700 1 -- 172.21.15.33:0/3066801408 wait complete. 2024-05-03T18:31:20.110 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 5.0 s, 460.2 tps, lat 190.847 ms stddev 416.510 2024-05-03T18:31:20.800 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-03T18:31:21.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:31:20 smithi033 ceph-mon[28330]: scrub queued for path: / 2024-05-03T18:31:21.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:31:20 smithi033 ceph-mon[28330]: scrub summary: idle+waiting paths [/] 2024-05-03T18:31:21.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:31:20 smithi033 ceph-mon[28330]: scrub summary: active paths [/] 2024-05-03T18:31:21.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:31:20 smithi033 ceph-mon[28330]: pgmap v127: 129 pgs: 129 active+clean; 1.4 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 8.8 MiB/s wr, 36 op/s 2024-05-03T18:31:21.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:31:20 smithi033 ceph-mon[28330]: scrub summary: idle+waiting paths [/] 2024-05-03T18:31:21.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:31:20 smithi099 ceph-mon[31658]: scrub queued for path: / 2024-05-03T18:31:21.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:31:20 smithi099 ceph-mon[31658]: scrub summary: idle+waiting paths [/] 2024-05-03T18:31:21.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:31:20 smithi099 ceph-mon[31658]: scrub summary: active paths [/] 2024-05-03T18:31:21.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:31:20 smithi099 ceph-mon[31658]: pgmap v127: 129 pgs: 129 active+clean; 1.4 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 8.8 MiB/s wr, 36 op/s 2024-05-03T18:31:21.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:31:20 smithi099 ceph-mon[31658]: scrub summary: idle+waiting paths [/] 2024-05-03T18:31:21.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:31:20 smithi150 ceph-mon[32492]: scrub queued for path: / 2024-05-03T18:31:21.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:31:20 smithi150 ceph-mon[32492]: scrub summary: idle+waiting paths [/] 2024-05-03T18:31:21.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:31:20 smithi150 ceph-mon[32492]: scrub summary: active paths [/] 2024-05-03T18:31:21.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:31:20 smithi150 ceph-mon[32492]: pgmap v127: 129 pgs: 129 active+clean; 1.4 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 8.8 MiB/s wr, 36 op/s 2024-05-03T18:31:21.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:31:20 smithi150 ceph-mon[32492]: scrub summary: idle+waiting paths [/] 2024-05-03T18:31:21.566 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:31:22.093 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:31:21 smithi033 ceph-mon[28330]: scrub summary: idle 2024-05-03T18:31:22.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:31:21 smithi099 ceph-mon[31658]: scrub summary: idle 2024-05-03T18:31:22.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:31:21 smithi150 ceph-mon[32492]: scrub summary: idle 2024-05-03T18:31:22.974 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:31:22 smithi033 ceph-mon[28330]: pgmap v128: 129 pgs: 129 active+clean; 1.4 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 8.8 MiB/s wr, 36 op/s 2024-05-03T18:31:23.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:31:22 smithi099 ceph-mon[31658]: pgmap v128: 129 pgs: 129 active+clean; 1.4 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 8.8 MiB/s wr, 36 op/s 2024-05-03T18:31:23.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:31:22 smithi150 ceph-mon[32492]: pgmap v128: 129 pgs: 129 active+clean; 1.4 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 8.8 MiB/s wr, 36 op/s 2024-05-03T18:31:24.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.209+0000 7fc073200700 1 -- 172.21.15.33:0/3291370654 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc06c102d60 msgr2=0x7fc06c10f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:31:24.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.209+0000 7fc073200700 1 --2- 172.21.15.33:0/3291370654 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc06c102d60 0x7fc06c10f250 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fc06400b0a0 tx=0x7fc06400b3b0 comp rx=0 tx=0).stop 2024-05-03T18:31:24.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.210+0000 7fc073200700 1 -- 172.21.15.33:0/3291370654 shutdown_connections 2024-05-03T18:31:24.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.210+0000 7fc073200700 1 --2- 172.21.15.33:0/3291370654 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc06c102d60 0x7fc06c10f250 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:24.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.210+0000 7fc073200700 1 --2- 172.21.15.33:0/3291370654 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc06c108410 0x7fc06c1087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:24.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.210+0000 7fc073200700 1 --2- 172.21.15.33:0/3291370654 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc06c1023a0 0x7fc06c102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:24.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.210+0000 7fc073200700 1 -- 172.21.15.33:0/3291370654 >> 172.21.15.33:0/3291370654 conn(0x7fc06c079350 msgr2=0x7fc06c079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:31:24.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.211+0000 7fc073200700 1 -- 172.21.15.33:0/3291370654 shutdown_connections 2024-05-03T18:31:24.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.211+0000 7fc073200700 1 -- 172.21.15.33:0/3291370654 wait complete. 2024-05-03T18:31:24.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.212+0000 7fc073200700 1 Processor -- start 2024-05-03T18:31:24.214 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:31:23 smithi033 ceph-mon[28330]: pgmap v129: 129 pgs: 129 active+clean; 1.4 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 11 MiB/s wr, 79 op/s 2024-05-03T18:31:24.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.213+0000 7fc073200700 1 -- start start 2024-05-03T18:31:24.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.214+0000 7fc073200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc06c1023a0 0x7fc06c1a0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:24.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.214+0000 7fc073200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc06c102d60 0x7fc06c1a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:24.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.214+0000 7fc073200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc06c108410 0x7fc06c1a49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:24.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.215+0000 7fc073200700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc06c1113e0 con 0x7fc06c1023a0 2024-05-03T18:31:24.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.215+0000 7fc073200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc06c111260 con 0x7fc06c108410 2024-05-03T18:31:24.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.215+0000 7fc073200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc06c111560 con 0x7fc06c102d60 2024-05-03T18:31:24.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.215+0000 7fc071200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc06c108410 0x7fc06c1a49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:24.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.215+0000 7fc06b400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc06c102d60 0x7fc06c1a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:24.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.215+0000 7fc071200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc06c108410 0x7fc06c1a49d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:39118/0 (socket says 172.21.15.33:39118) 2024-05-03T18:31:24.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.215+0000 7fc071200700 1 -- 172.21.15.33:0/407399071 learned_addr learned my addr 172.21.15.33:0/407399071 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:31:24.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.215+0000 7fc06be00700 1 --2- 172.21.15.33:0/407399071 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc06c1023a0 0x7fc06c1a0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:24.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.216+0000 7fc06b400700 1 -- 172.21.15.33:0/407399071 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc06c108410 msgr2=0x7fc06c1a49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:31:24.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.216+0000 7fc06b400700 1 --2- 172.21.15.33:0/407399071 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc06c108410 0x7fc06c1a49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:24.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.216+0000 7fc06b400700 1 -- 172.21.15.33:0/407399071 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc06c1023a0 msgr2=0x7fc06c1a0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:31:24.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.216+0000 7fc06b400700 1 --2- 172.21.15.33:0/407399071 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc06c1023a0 0x7fc06c1a0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:24.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.216+0000 7fc06b400700 1 -- 172.21.15.33:0/407399071 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc06c1a5110 con 0x7fc06c102d60 2024-05-03T18:31:24.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.217+0000 7fc06b400700 1 --2- 172.21.15.33:0/407399071 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc06c102d60 0x7fc06c1a0640 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc05c00e9f0 tx=0x7fc05c00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:31:24.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.217+0000 7fc068c00700 1 -- 172.21.15.33:0/407399071 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc05c00cbd0 con 0x7fc06c102d60 2024-05-03T18:31:24.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.217+0000 7fc073200700 1 -- 172.21.15.33:0/407399071 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc06c1a5400 con 0x7fc06c102d60 2024-05-03T18:31:24.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.218+0000 7fc068c00700 1 -- 172.21.15.33:0/407399071 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc05c0044b0 con 0x7fc06c102d60 2024-05-03T18:31:24.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.218+0000 7fc068c00700 1 -- 172.21.15.33:0/407399071 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc05c00cbd0 con 0x7fc06c102d60 2024-05-03T18:31:24.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.218+0000 7fc073200700 1 -- 172.21.15.33:0/407399071 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc06c1acc80 con 0x7fc06c102d60 2024-05-03T18:31:24.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.226+0000 7fc07db96b80 1 Processor -- start 2024-05-03T18:31:24.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.227+0000 7fc07db96b80 1 -- start start 2024-05-03T18:31:24.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.227+0000 7fc068c00700 1 -- 172.21.15.33:0/407399071 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fc05c004da0 con 0x7fc06c102d60 2024-05-03T18:31:24.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.228+0000 7fc07db96b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d03d579930 0x55d03d579d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:24.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.228+0000 7fc07db96b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d03d57a250 0x55d03d573f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:24.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.228+0000 7fc07db96b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d03d574550 0x55d03d5749d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:24.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.228+0000 7fc07db96b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d03d4c6a20 con 0x55d03d574550 2024-05-03T18:31:24.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.228+0000 7fc07db96b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55d03d464760 con 0x55d03d57a250 2024-05-03T18:31:24.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.228+0000 7fc07db96b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55d03d4c29e0 con 0x55d03d579930 2024-05-03T18:31:24.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.228+0000 7fc06b400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d03d574550 0x55d03d5749d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:24.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.229+0000 7fc06be00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d03d57a250 0x55d03d573f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:24.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.229+0000 7fc06be00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d03d57a250 0x55d03d573f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:39130/0 (socket says 172.21.15.33:39130) 2024-05-03T18:31:24.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.229+0000 7fc06be00700 1 -- 172.21.15.33:0/2063959077 learned_addr learned my addr 172.21.15.33:0/2063959077 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:31:24.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.229+0000 7fc071200700 1 --2- 172.21.15.33:0/2063959077 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d03d579930 0x55d03d579d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:24.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.230+0000 7fc068c00700 1 --2- 172.21.15.33:0/407399071 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fc0500b2cd0 0x7fc0500b5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:24.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.230+0000 7fc068c00700 1 -- 172.21.15.33:0/407399071 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7fc05c0e2550 con 0x7fc06c102d60 2024-05-03T18:31:24.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.230+0000 7fc06be00700 1 -- 172.21.15.33:0/2063959077 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d03d579930 msgr2=0x55d03d579d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:31:24.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.230+0000 7fc06be00700 1 --2- 172.21.15.33:0/2063959077 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d03d579930 0x55d03d579d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:24.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.230+0000 7fc06be00700 1 -- 172.21.15.33:0/2063959077 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d03d574550 msgr2=0x55d03d5749d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:31:24.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.231+0000 7fc06be00700 1 --2- 172.21.15.33:0/2063959077 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d03d574550 0x55d03d5749d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:24.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.231+0000 7fc06be00700 1 -- 172.21.15.33:0/2063959077 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d03d4138d0 con 0x55d03d57a250 2024-05-03T18:31:24.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.231+0000 7fc071200700 1 --2- 172.21.15.33:0/2063959077 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d03d579930 0x55d03d579d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:31:24.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.231+0000 7fc071200700 1 --2- 172.21.15.33:0/407399071 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fc0500b2cd0 0x7fc0500b5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:24.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.232+0000 7fc06b400700 1 --2- 172.21.15.33:0/2063959077 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d03d574550 0x55d03d5749d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:31:24.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.232+0000 7fc06be00700 1 --2- 172.21.15.33:0/2063959077 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d03d57a250 0x55d03d573f30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc054007810 tx=0x7fc054007b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb613ecc6d42a09a server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:31:24.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.232+0000 7fc071200700 1 --2- 172.21.15.33:0/407399071 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fc0500b2cd0 0x7fc0500b5190 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc064009e60 tx=0x7fc0640099e0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:31:24.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.233+0000 7fc061600700 1 -- 172.21.15.33:0/2063959077 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0540125f0 con 0x55d03d57a250 2024-05-03T18:31:24.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.233+0000 7fc061600700 1 -- 172.21.15.33:0/2063959077 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc054018070 con 0x55d03d57a250 2024-05-03T18:31:24.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.233+0000 7fc061600700 1 -- 172.21.15.33:0/2063959077 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc054013b30 con 0x55d03d57a250 2024-05-03T18:31:24.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.234+0000 7fc07db96b80 1 -- 172.21.15.33:0/2063959077 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d03d57a250 msgr2=0x55d03d573f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:31:24.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.234+0000 7fc07db96b80 1 --2- 172.21.15.33:0/2063959077 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d03d57a250 0x55d03d573f30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc054007810 tx=0x7fc054007b20 comp rx=0 tx=0).stop 2024-05-03T18:31:24.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.234+0000 7fc07db96b80 1 -- 172.21.15.33:0/2063959077 shutdown_connections 2024-05-03T18:31:24.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.234+0000 7fc07db96b80 1 --2- 172.21.15.33:0/2063959077 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d03d574550 0x55d03d5749d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:24.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.234+0000 7fc07db96b80 1 --2- 172.21.15.33:0/2063959077 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d03d579930 0x55d03d579d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:24.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.234+0000 7fc07db96b80 1 --2- 172.21.15.33:0/2063959077 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d03d57a250 0x55d03d573f30 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:24.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.234+0000 7fc07db96b80 1 -- 172.21.15.33:0/2063959077 >> 172.21.15.33:0/2063959077 conn(0x55d03d4ec6d0 msgr2=0x55d03d56e900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:31:24.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.235+0000 7fc07db96b80 1 -- 172.21.15.33:0/2063959077 shutdown_connections 2024-05-03T18:31:24.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.235+0000 7fc07db96b80 1 -- 172.21.15.33:0/2063959077 wait complete. 2024-05-03T18:31:24.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.237+0000 7fc07db96b80 1 -- start start 2024-05-03T18:31:24.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.238+0000 7fc07db96b80 1 Processor -- start 2024-05-03T18:31:24.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.243+0000 7fc07db96b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d03d574550 0x55d03d57fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:24.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.243+0000 7fc07db96b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d03d579930 0x55d03d57d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:24.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.243+0000 7fc07db96b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d03d57a250 0x55d03d57d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:24.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.243+0000 7fc07db96b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d03d580d00 con 0x55d03d574550 2024-05-03T18:31:24.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.243+0000 7fc07db96b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55d03d42cb20 con 0x55d03d57a250 2024-05-03T18:31:24.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.243+0000 7fc07db96b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55d03d580e80 con 0x55d03d579930 2024-05-03T18:31:24.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.243+0000 7fc071200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d03d57a250 0x55d03d57d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:24.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.244+0000 7fc071200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d03d57a250 0x55d03d57d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:39146/0 (socket says 172.21.15.33:39146) 2024-05-03T18:31:24.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.244+0000 7fc071200700 1 -- 172.21.15.33:0/1852715219 learned_addr learned my addr 172.21.15.33:0/1852715219 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:31:24.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.244+0000 7fc06b400700 1 --2- 172.21.15.33:0/1852715219 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d03d579930 0x55d03d57d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:24.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.244+0000 7fc06be00700 1 --2- 172.21.15.33:0/1852715219 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d03d574550 0x55d03d57fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:24.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.244+0000 7fc071200700 1 -- 172.21.15.33:0/1852715219 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d03d579930 msgr2=0x55d03d57d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:31:24.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.244+0000 7fc071200700 1 --2- 172.21.15.33:0/1852715219 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d03d579930 0x55d03d57d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:24.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.244+0000 7fc071200700 1 -- 172.21.15.33:0/1852715219 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d03d574550 msgr2=0x55d03d57fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:31:24.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.245+0000 7fc071200700 1 --2- 172.21.15.33:0/1852715219 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d03d574550 0x55d03d57fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:24.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.245+0000 7fc071200700 1 -- 172.21.15.33:0/1852715219 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d03d48c420 con 0x55d03d57a250 2024-05-03T18:31:24.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.245+0000 7fc06be00700 1 --2- 172.21.15.33:0/1852715219 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d03d574550 0x55d03d57fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:31:24.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.245+0000 7fc071200700 1 --2- 172.21.15.33:0/1852715219 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d03d57a250 0x55d03d57d720 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc064007790 tx=0x7fc06400b3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed974f6f9b66ca93 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:31:24.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.246+0000 7fc05a000700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-03T18:31:24.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.246+0000 7fc07db96b80 10 client.0 _ll_register_callbacks cb 0x55d03d46dc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:31:24.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.246+0000 7fc07db96b80 1 -- 172.21.15.33:0/1852715219 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc054018070 con 0x55d03d57a250 2024-05-03T18:31:24.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.246+0000 7fc05a000700 1 -- 172.21.15.33:0/1852715219 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc064005240 con 0x55d03d57a250 2024-05-03T18:31:24.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.248+0000 7fc05a000700 1 -- 172.21.15.33:0/1852715219 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7fc0640053e0 con 0x55d03d57a250 2024-05-03T18:31:24.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.248+0000 7fc05a000700 1 -- 172.21.15.33:0/1852715219 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fc064005600 con 0x55d03d57a250 2024-05-03T18:31:24.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.248+0000 7fc07db96b80 10 client.35486 fetch_fsmap learned FSMap version 21 2024-05-03T18:31:24.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.248+0000 7fc07db96b80 1 -- 172.21.15.33:0/1852715219 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({fsmap=21}) -- 0x55d03d58b2c0 con 0x55d03d57a250 2024-05-03T18:31:24.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.249+0000 7fc05a000700 1 -- 172.21.15.33:0/1852715219 <== mon.2 v2:172.21.15.99:3300/0 4 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fc064011070 con 0x55d03d57a250 2024-05-03T18:31:24.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.250+0000 7fc07db96b80 10 client.35486 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:31:24.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.250+0000 7fc07db96b80 10 client.35486 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:31:24.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.250+0000 7fc07db96b80 20 client.35486 populate_metadata read hostname 'smithi033' 2024-05-03T18:31:24.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.250+0000 7fc07db96b80 1 --2- 172.21.15.33:0/1852715219 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55d03d58d790 0x55d03d58fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:24.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.250+0000 7fc07db96b80 4 client.35486 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:31:24.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.250+0000 7fc07db96b80 1 -- 172.21.15.33:0/1852715219 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d03d540ff0 con 0x55d03d58d790 2024-05-03T18:31:24.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.251+0000 7fc06be00700 1 --2- 172.21.15.33:0/1852715219 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55d03d58d790 0x55d03d58fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:24.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.253+0000 7fc06be00700 1 --2- 172.21.15.33:0/1852715219 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55d03d58d790 0x55d03d58fbc0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bcd2f6710919de8a server_cookie=774a2343eb7bc043 in_seq=0 out_seq=0 2024-05-03T18:31:24.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.253+0000 7fc05a000700 10 client.35486 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:31:24.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.256+0000 7fc05a000700 1 -- 172.21.15.33:0/1852715219 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d03d540ff0 con 0x55d03d58d790 2024-05-03T18:31:24.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.256+0000 7fc05a000700 10 client.35486 handle_command_reply: tid=0 2024-05-03T18:31:24.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.257+0000 7fc07db96b80 1 client.35486 shutdown 2024-05-03T18:31:24.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.257+0000 7fc07db96b80 1 -- 172.21.15.33:0/1852715219 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d03d57a250 msgr2=0x55d03d57d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:31:24.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.257+0000 7fc07db96b80 1 --2- 172.21.15.33:0/1852715219 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d03d57a250 0x55d03d57d720 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc064007790 tx=0x7fc06400b3b0 comp rx=0 tx=0).stop 2024-05-03T18:31:24.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.258+0000 7fc07db96b80 1 -- 172.21.15.33:0/1852715219 shutdown_connections 2024-05-03T18:31:24.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.258+0000 7fc07db96b80 1 --2- 172.21.15.33:0/1852715219 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55d03d58d790 0x55d03d58fbc0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:24.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.258+0000 7fc05a000700 0 client.35486 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:31:24.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.258+0000 7fc07db96b80 1 --2- 172.21.15.33:0/1852715219 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d03d57a250 0x55d03d57d720 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:24.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.258+0000 7fc07db96b80 1 --2- 172.21.15.33:0/1852715219 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d03d574550 0x55d03d57fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:24.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.258+0000 7fc07db96b80 1 --2- 172.21.15.33:0/1852715219 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d03d579930 0x55d03d57d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:24.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.258+0000 7fc07db96b80 1 -- 172.21.15.33:0/1852715219 >> 172.21.15.33:0/1852715219 conn(0x55d03d4ec6d0 msgr2=0x55d03d5693b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:31:24.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.259+0000 7fc07db96b80 1 -- 172.21.15.33:0/1852715219 shutdown_connections 2024-05-03T18:31:24.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.259+0000 7fc07db96b80 1 -- 172.21.15.33:0/1852715219 wait complete. 2024-05-03T18:31:24.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.259+0000 7fc07db96b80 20 client.35486 trim_cache size 0 max 16384 2024-05-03T18:31:24.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:31:23 smithi099 ceph-mon[31658]: pgmap v129: 129 pgs: 129 active+clean; 1.4 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 11 MiB/s wr, 79 op/s 2024-05-03T18:31:24.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.297+0000 7fc07db96b80 1 Processor -- start 2024-05-03T18:31:24.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.297+0000 7fc07db96b80 1 -- start start 2024-05-03T18:31:24.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.298+0000 7fc07db96b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d03d574550 0x55d03d57ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:24.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.298+0000 7fc07db96b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d03d579930 0x55d03d595490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:24.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.298+0000 7fc07db96b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d03d57a250 0x55d03d5959d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:24.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.298+0000 7fc07db96b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc06c1b3a00 con 0x55d03d57a250 2024-05-03T18:31:24.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.298+0000 7fc07db96b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc06c1b6280 con 0x55d03d579930 2024-05-03T18:31:24.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.298+0000 7fc07db96b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc06c1b4070 con 0x55d03d574550 2024-05-03T18:31:24.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.299+0000 7fc06be00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d03d574550 0x55d03d57ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:24.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.299+0000 7fc06b400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d03d579930 0x55d03d595490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:24.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.299+0000 7fc06be00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d03d574550 0x55d03d57ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:56292/0 (socket says 172.21.15.33:56292) 2024-05-03T18:31:24.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.299+0000 7fc06be00700 1 -- 172.21.15.33:0/1452725462 learned_addr learned my addr 172.21.15.33:0/1452725462 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:31:24.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.299+0000 7fc06be00700 1 -- 172.21.15.33:0/1452725462 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d03d579930 msgr2=0x55d03d595490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:31:24.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.300+0000 7fc06be00700 1 --2- 172.21.15.33:0/1452725462 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d03d579930 0x55d03d595490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:24.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.300+0000 7fc06be00700 1 -- 172.21.15.33:0/1452725462 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d03d57a250 msgr2=0x55d03d5959d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:31:24.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.300+0000 7fc06be00700 1 --2- 172.21.15.33:0/1452725462 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d03d57a250 0x55d03d5959d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:24.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.300+0000 7fc06be00700 1 -- 172.21.15.33:0/1452725462 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d03d574cf0 con 0x55d03d574550 2024-05-03T18:31:24.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.300+0000 7fc06b400700 1 --2- 172.21.15.33:0/1452725462 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d03d579930 0x55d03d595490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:31:24.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.300+0000 7fc06be00700 1 --2- 172.21.15.33:0/1452725462 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d03d574550 0x55d03d57ad40 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc054006f00 tx=0x7fc054013690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8dc00af2bf1f9e8a server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:31:24.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.301+0000 7fc060c00700 1 -- 172.21.15.33:0/1452725462 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc054004240 con 0x55d03d574550 2024-05-03T18:31:24.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.301+0000 7fc060c00700 1 -- 172.21.15.33:0/1452725462 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc054027830 con 0x55d03d574550 2024-05-03T18:31:24.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.301+0000 7fc060c00700 1 -- 172.21.15.33:0/1452725462 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc054003980 con 0x55d03d574550 2024-05-03T18:31:24.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.302+0000 7fc07db96b80 1 -- 172.21.15.33:0/1452725462 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d03d574550 msgr2=0x55d03d57ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:31:24.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.302+0000 7fc07db96b80 1 --2- 172.21.15.33:0/1452725462 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d03d574550 0x55d03d57ad40 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc054006f00 tx=0x7fc054013690 comp rx=0 tx=0).stop 2024-05-03T18:31:24.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.303+0000 7fc07db96b80 1 -- 172.21.15.33:0/1452725462 shutdown_connections 2024-05-03T18:31:24.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.303+0000 7fc07db96b80 1 --2- 172.21.15.33:0/1452725462 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d03d57a250 0x55d03d5959d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:24.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.303+0000 7fc07db96b80 1 --2- 172.21.15.33:0/1452725462 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d03d574550 0x55d03d57ad40 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:24.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.303+0000 7fc07db96b80 1 --2- 172.21.15.33:0/1452725462 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d03d579930 0x55d03d595490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:24.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.304+0000 7fc07db96b80 1 -- 172.21.15.33:0/1452725462 >> 172.21.15.33:0/1452725462 conn(0x55d03d4ec6d0 msgr2=0x55d03d57a630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:31:24.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.304+0000 7fc07db96b80 1 -- 172.21.15.33:0/1452725462 shutdown_connections 2024-05-03T18:31:24.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.304+0000 7fc07db96b80 1 -- 172.21.15.33:0/1452725462 wait complete. 2024-05-03T18:31:24.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.305+0000 7fc07db96b80 1 -- start start 2024-05-03T18:31:24.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.305+0000 7fc07db96b80 1 Processor -- start 2024-05-03T18:31:24.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.307+0000 7fc07db96b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d03d574550 0x55d03d594d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:24.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.307+0000 7fc07db96b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d03d579930 0x55d03d5952a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:24.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.307+0000 7fc07db96b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d03d57a250 0x55d03d5957e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:24.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.308+0000 7fc07db96b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc06c1b4740 con 0x55d03d57a250 2024-05-03T18:31:24.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.308+0000 7fc06be00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d03d574550 0x55d03d594d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:24.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.308+0000 7fc06be00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d03d574550 0x55d03d594d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:56300/0 (socket says 172.21.15.33:56300) 2024-05-03T18:31:24.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.308+0000 7fc06be00700 1 -- 172.21.15.33:0/947810532 learned_addr learned my addr 172.21.15.33:0/947810532 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:31:24.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.308+0000 7fc06b400700 1 --2- 172.21.15.33:0/947810532 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d03d579930 0x55d03d5952a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:24.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.309+0000 7fc07db96b80 1 -- 172.21.15.33:0/947810532 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc06c1b5bb0 con 0x55d03d579930 2024-05-03T18:31:24.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.309+0000 7fc071200700 1 --2- 172.21.15.33:0/947810532 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d03d57a250 0x55d03d5957e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:24.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.309+0000 7fc07db96b80 1 -- 172.21.15.33:0/947810532 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc06c1b4e10 con 0x55d03d574550 2024-05-03T18:31:24.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.309+0000 7fc07db96b80 10 client.0 _ll_register_callbacks cb 0x55d03d46dc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:31:24.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.311+0000 7fc06b400700 1 -- 172.21.15.33:0/947810532 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d03d574550 msgr2=0x55d03d594d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:31:24.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.311+0000 7fc06b400700 1 --2- 172.21.15.33:0/947810532 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d03d574550 0x55d03d594d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:24.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.311+0000 7fc06b400700 1 -- 172.21.15.33:0/947810532 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d03d57a250 msgr2=0x55d03d5957e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:31:24.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.311+0000 7fc06b400700 1 --2- 172.21.15.33:0/947810532 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d03d57a250 0x55d03d5957e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:24.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.311+0000 7fc06b400700 1 -- 172.21.15.33:0/947810532 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d03d5f72f0 con 0x55d03d579930 2024-05-03T18:31:24.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.311+0000 7fc071200700 1 --2- 172.21.15.33:0/947810532 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d03d57a250 0x55d03d5957e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:31:24.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.313+0000 7fc06b400700 1 --2- 172.21.15.33:0/947810532 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d03d579930 0x55d03d5952a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc05c0df8b0 tx=0x7fc05c01e5e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=269e94d24fff259d server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:31:24.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.313+0000 7fc05a000700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-03T18:31:24.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.314+0000 7fc05a000700 1 -- 172.21.15.33:0/947810532 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc05c01c040 con 0x55d03d579930 2024-05-03T18:31:24.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.314+0000 7fc07db96b80 1 -- 172.21.15.33:0/947810532 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc054027830 con 0x55d03d579930 2024-05-03T18:31:24.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.314+0000 7fc05a000700 1 -- 172.21.15.33:0/947810532 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7fc05c0b1e10 con 0x55d03d579930 2024-05-03T18:31:24.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.315+0000 7fc05a000700 1 -- 172.21.15.33:0/947810532 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fc05c0a5300 con 0x55d03d579930 2024-05-03T18:31:24.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.315+0000 7fc07db96b80 10 client.35492 fetch_fsmap learned FSMap version 21 2024-05-03T18:31:24.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.315+0000 7fc07db96b80 1 -- 172.21.15.33:0/947810532 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55d03d5f8630 con 0x55d03d579930 2024-05-03T18:31:24.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.316+0000 7fc05a000700 1 -- 172.21.15.33:0/947810532 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc054027830 con 0x55d03d579930 2024-05-03T18:31:24.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.316+0000 7fc05a000700 1 -- 172.21.15.33:0/947810532 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fc05c0a3b60 con 0x55d03d579930 2024-05-03T18:31:24.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.316+0000 7fc07db96b80 10 client.35492 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:31:24.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.316+0000 7fc07db96b80 10 client.35492 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:31:24.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.316+0000 7fc07db96b80 20 client.35492 populate_metadata read hostname 'smithi033' 2024-05-03T18:31:24.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.316+0000 7fc07db96b80 1 --2- 172.21.15.33:0/947810532 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55d03d58d790 0x55d03d5fac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:24.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.316+0000 7fc07db96b80 4 client.35492 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-03T18:31:24.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.316+0000 7fc07db96b80 1 -- 172.21.15.33:0/947810532 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d03d4d4400 con 0x55d03d58d790 2024-05-03T18:31:24.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.317+0000 7fc06be00700 1 --2- 172.21.15.33:0/947810532 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55d03d58d790 0x55d03d5fac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:24.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.319+0000 7fc06be00700 1 --2- 172.21.15.33:0/947810532 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55d03d58d790 0x55d03d5fac50 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7363210b14bf509 server_cookie=6f138c491c71a248 in_seq=0 out_seq=0 2024-05-03T18:31:24.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.319+0000 7fc05a000700 10 client.35492 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:31:24.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.319+0000 7fc05a000700 1 -- 172.21.15.33:0/947810532 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55d03d4d4400 con 0x55d03d58d790 2024-05-03T18:31:24.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.319+0000 7fc05a000700 10 client.35492 handle_command_reply: tid=0 2024-05-03T18:31:24.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.320+0000 7fc07db96b80 1 client.35492 shutdown 2024-05-03T18:31:24.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.321+0000 7fc07db96b80 1 -- 172.21.15.33:0/947810532 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d03d579930 msgr2=0x55d03d5952a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:31:24.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.321+0000 7fc07db96b80 1 --2- 172.21.15.33:0/947810532 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d03d579930 0x55d03d5952a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc05c0df8b0 tx=0x7fc05c01e5e0 comp rx=0 tx=0).stop 2024-05-03T18:31:24.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.321+0000 7fc07db96b80 1 -- 172.21.15.33:0/947810532 shutdown_connections 2024-05-03T18:31:24.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.321+0000 7fc07db96b80 1 --2- 172.21.15.33:0/947810532 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55d03d58d790 0x55d03d5fac50 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:24.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.321+0000 7fc07db96b80 1 --2- 172.21.15.33:0/947810532 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d03d57a250 0x55d03d5957e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:24.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.321+0000 7fc07db96b80 1 --2- 172.21.15.33:0/947810532 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d03d574550 0x55d03d594d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:24.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.321+0000 7fc07db96b80 1 --2- 172.21.15.33:0/947810532 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d03d579930 0x55d03d5952a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:24.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.321+0000 7fc07db96b80 1 -- 172.21.15.33:0/947810532 >> 172.21.15.33:0/947810532 conn(0x55d03d4ec6d0 msgr2=0x55d03d5693b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:31:24.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.321+0000 7fc05a000700 0 client.35492 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:31:24.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.322+0000 7fc07db96b80 1 -- 172.21.15.33:0/947810532 shutdown_connections 2024-05-03T18:31:24.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.322+0000 7fc07db96b80 1 -- 172.21.15.33:0/947810532 wait complete. 2024-05-03T18:31:24.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.322+0000 7fc07db96b80 20 client.35492 trim_cache size 0 max 16384 2024-05-03T18:31:24.324 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-03T18:31:24.324 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-05-03T18:31:24.324 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-05-03T18:31:24.324 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-03T18:31:24.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.323+0000 7fc073200700 1 -- 172.21.15.33:0/407399071 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fc0500b2cd0 msgr2=0x7fc0500b5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:31:24.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.323+0000 7fc073200700 1 --2- 172.21.15.33:0/407399071 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fc0500b2cd0 0x7fc0500b5190 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc064009e60 tx=0x7fc0640099e0 comp rx=0 tx=0).stop 2024-05-03T18:31:24.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.323+0000 7fc073200700 1 -- 172.21.15.33:0/407399071 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc06c102d60 msgr2=0x7fc06c1a0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:31:24.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.323+0000 7fc073200700 1 --2- 172.21.15.33:0/407399071 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc06c102d60 0x7fc06c1a0640 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc05c00e9f0 tx=0x7fc05c00ed00 comp rx=0 tx=0).stop 2024-05-03T18:31:24.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.324+0000 7fc073200700 1 -- 172.21.15.33:0/407399071 shutdown_connections 2024-05-03T18:31:24.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.324+0000 7fc073200700 1 --2- 172.21.15.33:0/407399071 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fc0500b2cd0 0x7fc0500b5190 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:24.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.324+0000 7fc073200700 1 --2- 172.21.15.33:0/407399071 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc06c108410 0x7fc06c1a49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:24.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.324+0000 7fc073200700 1 --2- 172.21.15.33:0/407399071 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc06c1023a0 0x7fc06c1a0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:24.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.324+0000 7fc073200700 1 --2- 172.21.15.33:0/407399071 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc06c102d60 0x7fc06c1a0640 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:24.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.324+0000 7fc073200700 1 -- 172.21.15.33:0/407399071 >> 172.21.15.33:0/407399071 conn(0x7fc06c079350 msgr2=0x7fc06c100e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:31:24.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.325+0000 7fc073200700 1 -- 172.21.15.33:0/407399071 shutdown_connections 2024-05-03T18:31:24.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:24.325+0000 7fc073200700 1 -- 172.21.15.33:0/407399071 wait complete. 2024-05-03T18:31:24.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:31:23 smithi150 ceph-mon[32492]: pgmap v129: 129 pgs: 129 active+clean; 1.4 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 11 MiB/s wr, 79 op/s 2024-05-03T18:31:25.110 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 10.0 s, 887.6 tps, lat 112.871 ms stddev 37.541 2024-05-03T18:31:25.326 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-03T18:31:25.326 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T18:31:26.050 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:31:26.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:31:26 smithi033 ceph-mon[28330]: pgmap v130: 129 pgs: 129 active+clean; 1.4 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 4.9 MiB/s wr, 73 op/s 2024-05-03T18:31:27.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:31:26 smithi099 ceph-mon[31658]: pgmap v130: 129 pgs: 129 active+clean; 1.4 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 4.9 MiB/s wr, 73 op/s 2024-05-03T18:31:27.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:31:26 smithi150 ceph-mon[32492]: pgmap v130: 129 pgs: 129 active+clean; 1.4 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 4.9 MiB/s wr, 73 op/s 2024-05-03T18:31:29.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:31:28 smithi033 ceph-mon[28330]: pgmap v131: 129 pgs: 129 active+clean; 1.4 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 5.6 MiB/s wr, 138 op/s 2024-05-03T18:31:29.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:31:28 smithi099 ceph-mon[31658]: pgmap v131: 129 pgs: 129 active+clean; 1.4 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 5.6 MiB/s wr, 138 op/s 2024-05-03T18:31:29.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:31:28 smithi150 ceph-mon[32492]: pgmap v131: 129 pgs: 129 active+clean; 1.4 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 5.6 MiB/s wr, 138 op/s 2024-05-03T18:31:29.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:29.481+0000 7f6027a00700 1 -- 172.21.15.33:0/3316481224 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f602006bb40 msgr2=0x7f6020110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:31:29.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:29.481+0000 7f6027a00700 1 --2- 172.21.15.33:0/3316481224 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f602006bb40 0x7f6020110330 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6018009f90 tx=0x7f601800b330 comp rx=0 tx=0).stop 2024-05-03T18:31:29.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:29.481+0000 7f6027a00700 1 -- 172.21.15.33:0/3316481224 shutdown_connections 2024-05-03T18:31:29.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:29.481+0000 7f6027a00700 1 --2- 172.21.15.33:0/3316481224 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f602006bb40 0x7f6020110330 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:29.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:29.481+0000 7f6027a00700 1 --2- 172.21.15.33:0/3316481224 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6020109510 0x7f60201098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:29.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:29.481+0000 7f6027a00700 1 --2- 172.21.15.33:0/3316481224 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f602006b180 0x7f602006b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:29.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:29.481+0000 7f6027a00700 1 -- 172.21.15.33:0/3316481224 >> 172.21.15.33:0/3316481224 conn(0x7f6020079350 msgr2=0x7f6020079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:31:29.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:29.483+0000 7f6027a00700 1 -- 172.21.15.33:0/3316481224 shutdown_connections 2024-05-03T18:31:29.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:29.483+0000 7f6027a00700 1 -- 172.21.15.33:0/3316481224 wait complete. 2024-05-03T18:31:29.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:29.484+0000 7f6027a00700 1 Processor -- start 2024-05-03T18:31:29.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:29.485+0000 7f6027a00700 1 -- start start 2024-05-03T18:31:29.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:29.486+0000 7f6027a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f602006b180 0x7f6020074d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:29.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:29.486+0000 7f6027a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f602006bb40 0x7f602006ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:29.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:29.486+0000 7f6027a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6020109510 0x7f602006e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:29.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:29.486+0000 7f6027a00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6020113520 con 0x7f602006bb40 2024-05-03T18:31:29.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:29.486+0000 7f6027a00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f60201133a0 con 0x7f602006b180 2024-05-03T18:31:29.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:29.486+0000 7f6027a00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f60201136a0 con 0x7f6020109510 2024-05-03T18:31:29.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:29.486+0000 7f6025000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f602006b180 0x7f6020074d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:29.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:29.486+0000 7f601fe00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f602006bb40 0x7f602006ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:29.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:29.486+0000 7f6025a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6020109510 0x7f602006e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:29.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:29.486+0000 7f601fe00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f602006bb40 0x7f602006ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54832/0 (socket says 172.21.15.33:54832) 2024-05-03T18:31:29.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:29.486+0000 7f6025000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f602006b180 0x7f6020074d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:44104/0 (socket says 172.21.15.33:44104) 2024-05-03T18:31:29.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:29.486+0000 7f601fe00700 1 -- 172.21.15.33:0/89137457 learned_addr learned my addr 172.21.15.33:0/89137457 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:31:29.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:29.487+0000 7f6025a00700 1 -- 172.21.15.33:0/89137457 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f602006b180 msgr2=0x7f6020074d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:31:29.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:29.487+0000 7f6025a00700 1 --2- 172.21.15.33:0/89137457 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f602006b180 0x7f6020074d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:29.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:29.487+0000 7f6025a00700 1 -- 172.21.15.33:0/89137457 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f602006bb40 msgr2=0x7f602006ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:31:29.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:29.487+0000 7f6025a00700 1 --2- 172.21.15.33:0/89137457 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f602006bb40 0x7f602006ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:29.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:29.487+0000 7f6025a00700 1 -- 172.21.15.33:0/89137457 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f602006e9d0 con 0x7f6020109510 2024-05-03T18:31:29.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:29.488+0000 7f6025000700 1 --2- 172.21.15.33:0/89137457 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f602006b180 0x7f6020074d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:31:29.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:29.488+0000 7f6025a00700 1 --2- 172.21.15.33:0/89137457 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6020109510 0x7f602006e330 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6018009f90 tx=0x7f6018004da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:31:29.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:29.490+0000 7f601d600700 1 -- 172.21.15.33:0/89137457 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6018021070 con 0x7f6020109510 2024-05-03T18:31:29.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:29.490+0000 7f601d600700 1 -- 172.21.15.33:0/89137457 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6018007ac0 con 0x7f6020109510 2024-05-03T18:31:29.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:29.490+0000 7f601d600700 1 -- 172.21.15.33:0/89137457 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60180052a0 con 0x7f6020109510 2024-05-03T18:31:29.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:29.490+0000 7f6027a00700 1 -- 172.21.15.33:0/89137457 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6020072840 con 0x7f6020109510 2024-05-03T18:31:29.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:29.491+0000 7f6027a00700 1 -- 172.21.15.33:0/89137457 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6020072d80 con 0x7f6020109510 2024-05-03T18:31:29.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:29.492+0000 7f6027a00700 1 -- 172.21.15.33:0/89137457 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6020104bd0 con 0x7f6020109510 2024-05-03T18:31:29.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:29.499+0000 7f601d600700 1 -- 172.21.15.33:0/89137457 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f6018024070 con 0x7f6020109510 2024-05-03T18:31:29.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:29.501+0000 7f601d600700 1 --2- 172.21.15.33:0/89137457 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f60080b2c00 0x7f60080b50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:29.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:29.502+0000 7f601d600700 1 -- 172.21.15.33:0/89137457 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f601800f080 con 0x7f6020109510 2024-05-03T18:31:29.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:29.502+0000 7f6025000700 1 --2- 172.21.15.33:0/89137457 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f60080b2c00 0x7f60080b50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:29.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:29.503+0000 7f6025000700 1 --2- 172.21.15.33:0/89137457 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f60080b2c00 0x7f60080b50c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6014005e10 tx=0x7f6014005d60 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:31:29.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:29.517+0000 7f601d600700 1 -- 172.21.15.33:0/89137457 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f60180ab2c0 con 0x7f6020109510 2024-05-03T18:31:29.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:29.997+0000 7f6027a00700 1 -- 172.21.15.33:0/89137457 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f602006f2d0 con 0x7f6020109510 2024-05-03T18:31:29.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:29.998+0000 7f601d600700 1 -- 172.21.15.33:0/89137457 <== mon.1 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11801 (secure 0 0 0) 0x7f60180aabc0 con 0x7f6020109510 2024-05-03T18:31:30.000 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:31:30.000 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1327393503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1327393503},{"type":"v1","addr":"172.21.15.33:6835","nonce":1327393503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/2461472948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":2461472948},{"type":"v1","addr":"172.21.15.33:6837","nonce":2461472948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14559,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/442269281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":442269281},{"type":"v1","addr":"172.21.15.33:6839","nonce":442269281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14571,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/645571756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":645571756},{"type":"v1","addr":"172.21.15.33:6841","nonce":645571756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/203598909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":203598909},{"type":"v1","addr":"172.21.15.150:6833","nonce":203598909}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/4029842493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":4029842493},{"type":"v1","addr":"172.21.15.150:6835","nonce":4029842493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3463664423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3463664423},{"type":"v1","addr":"172.21.15.150:6837","nonce":3463664423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24481,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6839/4051648018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":4051648018},{"type":"v1","addr":"172.21.15.150:6839","nonce":4051648018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34418,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1865105479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1865105479},{"type":"v1","addr":"172.21.15.99:6833","nonce":1865105479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34445,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/1206393914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1206393914},{"type":"v1","addr":"172.21.15.99:6835","nonce":1206393914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-03T18:22:28.837091+0000","modified":"2024-05-03T18:23:49.758398+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34508},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34472":{"gid":34472,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6837/3360970342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3360970342},{"type":"v1","addr":"172.21.15.99:6837","nonce":3360970342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34508":{"gid":34508,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/919907308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":919907308},{"type":"v1","addr":"172.21.15.99:6839","nonce":919907308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34508,"qdb_cluster":[34508]},"id":1}]} 2024-05-03T18:31:30.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:30.002+0000 7f6027a00700 1 -- 172.21.15.33:0/89137457 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f60080b2c00 msgr2=0x7f60080b50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:31:30.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:30.002+0000 7f6027a00700 1 --2- 172.21.15.33:0/89137457 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f60080b2c00 0x7f60080b50c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6014005e10 tx=0x7f6014005d60 comp rx=0 tx=0).stop 2024-05-03T18:31:30.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:30.002+0000 7f6027a00700 1 -- 172.21.15.33:0/89137457 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6020109510 msgr2=0x7f602006e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:31:30.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:30.002+0000 7f6027a00700 1 --2- 172.21.15.33:0/89137457 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6020109510 0x7f602006e330 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6018009f90 tx=0x7f6018004da0 comp rx=0 tx=0).stop 2024-05-03T18:31:30.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:30.003+0000 7f6027a00700 1 -- 172.21.15.33:0/89137457 shutdown_connections 2024-05-03T18:31:30.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:30.003+0000 7f6027a00700 1 --2- 172.21.15.33:0/89137457 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6020109510 0x7f602006e330 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:30.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:30.003+0000 7f6027a00700 1 --2- 172.21.15.33:0/89137457 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f60080b2c00 0x7f60080b50c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:30.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:30.003+0000 7f6027a00700 1 --2- 172.21.15.33:0/89137457 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f602006b180 0x7f6020074d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:30.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:30.003+0000 7f6027a00700 1 --2- 172.21.15.33:0/89137457 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f602006bb40 0x7f602006ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:30.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:30.003+0000 7f6027a00700 1 -- 172.21.15.33:0/89137457 >> 172.21.15.33:0/89137457 conn(0x7f6020079350 msgr2=0x7f60201010c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:31:30.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:30.004+0000 7f6027a00700 1 -- 172.21.15.33:0/89137457 shutdown_connections 2024-05-03T18:31:30.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:30.004+0000 7f6027a00700 1 -- 172.21.15.33:0/89137457 wait complete. 2024-05-03T18:31:30.010 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 21 2024-05-03T18:31:30.110 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 15.0 s, 880.0 tps, lat 112.897 ms stddev 34.952 2024-05-03T18:31:30.902 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-03T18:31:31.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:31:30 smithi033 ceph-mon[28330]: pgmap v132: 129 pgs: 129 active+clean; 1.4 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 2.9 MiB/s wr, 107 op/s 2024-05-03T18:31:31.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:31:30 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/89137457' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:31:31.297 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:31:30 smithi099 ceph-mon[31658]: pgmap v132: 129 pgs: 129 active+clean; 1.4 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 2.9 MiB/s wr, 107 op/s 2024-05-03T18:31:31.297 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:31:30 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/89137457' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:31:31.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:31:30 smithi150 ceph-mon[32492]: pgmap v132: 129 pgs: 129 active+clean; 1.4 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 2.9 MiB/s wr, 107 op/s 2024-05-03T18:31:31.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:31:30 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/89137457' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:31:31.654 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:31:33.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:31:32 smithi033 ceph-mon[28330]: pgmap v133: 129 pgs: 129 active+clean; 1.4 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 2.9 MiB/s wr, 107 op/s 2024-05-03T18:31:33.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:31:32 smithi099 ceph-mon[31658]: pgmap v133: 129 pgs: 129 active+clean; 1.4 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 2.9 MiB/s wr, 107 op/s 2024-05-03T18:31:33.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:31:32 smithi150 ceph-mon[32492]: pgmap v133: 129 pgs: 129 active+clean; 1.4 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 2.9 MiB/s wr, 107 op/s 2024-05-03T18:31:34.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:31:33 smithi033 ceph-mon[28330]: pgmap v134: 129 pgs: 129 active+clean; 1.4 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 5.9 MiB/s wr, 352 op/s 2024-05-03T18:31:34.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:31:33 smithi099 ceph-mon[31658]: pgmap v134: 129 pgs: 129 active+clean; 1.4 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 5.9 MiB/s wr, 352 op/s 2024-05-03T18:31:34.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:31:33 smithi150 ceph-mon[32492]: pgmap v134: 129 pgs: 129 active+clean; 1.4 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 5.9 MiB/s wr, 352 op/s 2024-05-03T18:31:35.110 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 20.0 s, 578.2 tps, lat 172.460 ms stddev 44.451 2024-05-03T18:31:35.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:31:34 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:31:35.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:31:34 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:31:35.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:31:34 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:31:36.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.194+0000 7f425e400700 1 -- 172.21.15.33:0/3288263350 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f425810a9e0 msgr2=0x7f425810ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:31:36.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.194+0000 7f425e400700 1 --2- 172.21.15.33:0/3288263350 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f425810a9e0 0x7f425810ae60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4248009f90 tx=0x7f424800b330 comp rx=0 tx=0).stop 2024-05-03T18:31:36.196 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:31:36 smithi033 ceph-mon[28330]: pgmap v135: 129 pgs: 129 active+clean; 1.4 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 3.7 MiB/s wr, 309 op/s 2024-05-03T18:31:36.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.195+0000 7f425e400700 1 -- 172.21.15.33:0/3288263350 shutdown_connections 2024-05-03T18:31:36.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.195+0000 7f425e400700 1 --2- 172.21.15.33:0/3288263350 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4258118530 0x7f425811a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:36.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.195+0000 7f425e400700 1 --2- 172.21.15.33:0/3288263350 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f425810db90 0x7f425810a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:36.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.195+0000 7f425e400700 1 --2- 172.21.15.33:0/3288263350 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f425810a9e0 0x7f425810ae60 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:36.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.195+0000 7f425e400700 1 -- 172.21.15.33:0/3288263350 >> 172.21.15.33:0/3288263350 conn(0x7f425806e9f0 msgr2=0x7f425806ee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:31:36.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.195+0000 7f425e400700 1 -- 172.21.15.33:0/3288263350 shutdown_connections 2024-05-03T18:31:36.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.195+0000 7f425e400700 1 -- 172.21.15.33:0/3288263350 wait complete. 2024-05-03T18:31:36.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.197+0000 7f425e400700 1 Processor -- start 2024-05-03T18:31:36.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.197+0000 7f425e400700 1 -- start start 2024-05-03T18:31:36.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.198+0000 7f425e400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f425810a9e0 0x7f42581a44f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:36.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.198+0000 7f4257400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f425810a9e0 0x7f42581a44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:36.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.198+0000 7f4257400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f425810a9e0 0x7f42581a44f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:34094/0 (socket says 172.21.15.33:34094) 2024-05-03T18:31:36.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.199+0000 7f425e400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f425810db90 0x7f42581a4a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:36.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.199+0000 7f4257400700 1 -- 172.21.15.33:0/2038357459 learned_addr learned my addr 172.21.15.33:0/2038357459 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:31:36.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.200+0000 7f425e400700 1 --2- 172.21.15.33:0/2038357459 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4258118530 0x7f42581a8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:36.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.200+0000 7f425e400700 1 -- 172.21.15.33:0/2038357459 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f425811dfc0 con 0x7f4258118530 2024-05-03T18:31:36.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.200+0000 7f4256a00700 1 --2- 172.21.15.33:0/2038357459 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f425810db90 0x7f42581a4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:36.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.201+0000 7f425e400700 1 -- 172.21.15.33:0/2038357459 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f425811de40 con 0x7f425810a9e0 2024-05-03T18:31:36.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.201+0000 7f425e400700 1 -- 172.21.15.33:0/2038357459 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f425811e140 con 0x7f425810db90 2024-05-03T18:31:36.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.202+0000 7f4257e00700 1 --2- 172.21.15.33:0/2038357459 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4258118530 0x7f42581a8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:36.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.203+0000 7f4257e00700 1 -- 172.21.15.33:0/2038357459 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f425810db90 msgr2=0x7f42581a4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:31:36.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.203+0000 7f4257e00700 1 --2- 172.21.15.33:0/2038357459 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f425810db90 0x7f42581a4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:36.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.203+0000 7f4257e00700 1 -- 172.21.15.33:0/2038357459 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f425810a9e0 msgr2=0x7f42581a44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:31:36.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.203+0000 7f4257e00700 1 --2- 172.21.15.33:0/2038357459 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f425810a9e0 0x7f42581a44f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:36.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.203+0000 7f4257e00700 1 -- 172.21.15.33:0/2038357459 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42581a9500 con 0x7f4258118530 2024-05-03T18:31:36.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.205+0000 7f4257e00700 1 --2- 172.21.15.33:0/2038357459 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4258118530 0x7f42581a8dc0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f424400e940 tx=0x7f424400ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:31:36.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.206+0000 7f424fe00700 1 -- 172.21.15.33:0/2038357459 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4244004220 con 0x7f4258118530 2024-05-03T18:31:36.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.206+0000 7f424fe00700 1 -- 172.21.15.33:0/2038357459 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f42440043c0 con 0x7f4258118530 2024-05-03T18:31:36.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.206+0000 7f425e400700 1 -- 172.21.15.33:0/2038357459 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42581a97f0 con 0x7f4258118530 2024-05-03T18:31:36.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.206+0000 7f425e400700 1 -- 172.21.15.33:0/2038357459 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42581b1070 con 0x7f4258118530 2024-05-03T18:31:36.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.209+0000 7f424fe00700 1 -- 172.21.15.33:0/2038357459 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4244005590 con 0x7f4258118530 2024-05-03T18:31:36.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.237+0000 7f424fe00700 1 -- 172.21.15.33:0/2038357459 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f4244004750 con 0x7f4258118530 2024-05-03T18:31:36.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.237+0000 7f4268e53b80 1 Processor -- start 2024-05-03T18:31:36.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.237+0000 7f4268e53b80 1 -- start start 2024-05-03T18:31:36.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.238+0000 7f4268e53b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b1390c7b0 0x556b1390cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:36.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.238+0000 7f4268e53b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b1390d0d0 0x556b13907fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:36.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.238+0000 7f4268e53b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b139085d0 0x556b13908a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:36.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.238+0000 7f4268e53b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556b1385aa20 con 0x556b1390c7b0 2024-05-03T18:31:36.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.238+0000 7f4268e53b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x556b137f8760 con 0x556b139085d0 2024-05-03T18:31:36.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.238+0000 7f4268e53b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x556b138569e0 con 0x556b1390d0d0 2024-05-03T18:31:36.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.240+0000 7f4257400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b1390d0d0 0x556b13907fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:36.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.240+0000 7f4257400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b1390d0d0 0x556b13907fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:33234/0 (socket says 172.21.15.33:33234) 2024-05-03T18:31:36.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.240+0000 7f4257400700 1 -- 172.21.15.33:0/44762324 learned_addr learned my addr 172.21.15.33:0/44762324 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:31:36.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.241+0000 7f4257400700 1 -- 172.21.15.33:0/44762324 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b139085d0 msgr2=0x556b13908a30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-03T18:31:36.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.241+0000 7f4257400700 1 --2- 172.21.15.33:0/44762324 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b139085d0 0x556b13908a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:36.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.241+0000 7f4257400700 1 -- 172.21.15.33:0/44762324 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b1390c7b0 msgr2=0x556b1390cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:31:36.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.241+0000 7f424fe00700 1 --2- 172.21.15.33:0/2038357459 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f423c0b2b00 0x7f423c0b4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:36.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.241+0000 7f4257400700 1 --2- 172.21.15.33:0/44762324 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b1390c7b0 0x556b1390cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:36.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.241+0000 7f4257400700 1 -- 172.21.15.33:0/44762324 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b137a78d0 con 0x556b1390d0d0 2024-05-03T18:31:36.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.241+0000 7f4257400700 1 --2- 172.21.15.33:0/44762324 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b1390d0d0 0x556b13907fb0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4250009e30 tx=0x7f4250011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3fa0a424986f2643 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:31:36.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.241+0000 7f424fe00700 1 -- 172.21.15.33:0/2038357459 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f42440e2b20 con 0x7f4258118530 2024-05-03T18:31:36.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.242+0000 7f4257e00700 1 --2- 172.21.15.33:0/2038357459 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f423c0b2b00 0x7f423c0b4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:36.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.242+0000 7f424cc00700 1 -- 172.21.15.33:0/44762324 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f425001e070 con 0x556b1390d0d0 2024-05-03T18:31:36.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.242+0000 7f424cc00700 1 -- 172.21.15.33:0/44762324 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4250019ac0 con 0x556b1390d0d0 2024-05-03T18:31:36.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.242+0000 7f424cc00700 1 -- 172.21.15.33:0/44762324 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f425000ea50 con 0x556b1390d0d0 2024-05-03T18:31:36.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.244+0000 7f4268e53b80 1 -- 172.21.15.33:0/44762324 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b1390d0d0 msgr2=0x556b13907fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:31:36.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.244+0000 7f4268e53b80 1 --2- 172.21.15.33:0/44762324 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b1390d0d0 0x556b13907fb0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4250009e30 tx=0x7f4250011720 comp rx=0 tx=0).stop 2024-05-03T18:31:36.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.245+0000 7f4268e53b80 1 -- 172.21.15.33:0/44762324 shutdown_connections 2024-05-03T18:31:36.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.245+0000 7f4268e53b80 1 --2- 172.21.15.33:0/44762324 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b139085d0 0x556b13908a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:36.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.245+0000 7f4268e53b80 1 --2- 172.21.15.33:0/44762324 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b1390c7b0 0x556b1390cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:36.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.245+0000 7f4268e53b80 1 --2- 172.21.15.33:0/44762324 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b1390d0d0 0x556b13907fb0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:36.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.245+0000 7f4268e53b80 1 -- 172.21.15.33:0/44762324 >> 172.21.15.33:0/44762324 conn(0x556b13836b60 msgr2=0x556b139027f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:31:36.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.245+0000 7f4257e00700 1 --2- 172.21.15.33:0/2038357459 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f423c0b2b00 0x7f423c0b4fc0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f42440dff90 tx=0x7f424408ff90 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:31:36.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.246+0000 7f4268e53b80 1 -- 172.21.15.33:0/44762324 shutdown_connections 2024-05-03T18:31:36.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.246+0000 7f4268e53b80 1 -- 172.21.15.33:0/44762324 wait complete. 2024-05-03T18:31:36.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.248+0000 7f4268e53b80 1 -- start start 2024-05-03T18:31:36.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.248+0000 7f4268e53b80 1 Processor -- start 2024-05-03T18:31:36.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.249+0000 7f4268e53b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b139085d0 0x556b13913e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:36.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.249+0000 7f4268e53b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b1390c7b0 0x556b13911270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:36.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.249+0000 7f4268e53b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b1390d0d0 0x556b139117b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:36.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.249+0000 7f4268e53b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556b13914ca0 con 0x556b139085d0 2024-05-03T18:31:36.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.249+0000 7f4268e53b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x556b137c0b20 con 0x556b1390d0d0 2024-05-03T18:31:36.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.249+0000 7f4268e53b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x556b13914e20 con 0x556b1390c7b0 2024-05-03T18:31:36.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.250+0000 7f4268e53b80 10 client.0 _ll_register_callbacks cb 0x556b13801c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:31:36.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.250+0000 7f4256a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b1390d0d0 0x556b139117b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:36.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.251+0000 7f4257400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b1390c7b0 0x556b13911270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:36.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.251+0000 7f4257400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b1390c7b0 0x556b13911270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:33242/0 (socket says 172.21.15.33:33242) 2024-05-03T18:31:36.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.251+0000 7f4257400700 1 -- 172.21.15.33:0/3109063628 learned_addr learned my addr 172.21.15.33:0/3109063628 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:31:36.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.251+0000 7f4256a00700 1 --2- 172.21.15.33:0/3109063628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b139085d0 0x556b13913e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:36.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.251+0000 7f4257400700 1 -- 172.21.15.33:0/3109063628 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b1390d0d0 msgr2=0x556b139117b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:31:36.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.251+0000 7f4257400700 1 --2- 172.21.15.33:0/3109063628 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b1390d0d0 0x556b139117b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:36.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.251+0000 7f4257400700 1 -- 172.21.15.33:0/3109063628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b139085d0 msgr2=0x556b13913e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:31:36.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.251+0000 7f4257400700 1 --2- 172.21.15.33:0/3109063628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b139085d0 0x556b13913e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:36.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.251+0000 7f4257400700 1 -- 172.21.15.33:0/3109063628 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b13820420 con 0x556b1390c7b0 2024-05-03T18:31:36.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.254+0000 7f4257400700 1 --2- 172.21.15.33:0/3109063628 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b1390c7b0 0x556b13911270 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f425000e630 tx=0x7f425000e4e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=101b25e08c3b2691 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:31:36.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.254+0000 7f4241600700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-03T18:31:36.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.254+0000 7f4241600700 1 -- 172.21.15.33:0/3109063628 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f425001e040 con 0x556b1390c7b0 2024-05-03T18:31:36.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.254+0000 7f4241600700 1 -- 172.21.15.33:0/3109063628 <== mon.1 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f4250004230 con 0x556b1390c7b0 2024-05-03T18:31:36.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.255+0000 7f4268e53b80 1 -- 172.21.15.33:0/3109063628 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4250019ac0 con 0x556b1390c7b0 2024-05-03T18:31:36.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.255+0000 7f4241600700 1 -- 172.21.15.33:0/3109063628 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f4250004430 con 0x556b1390c7b0 2024-05-03T18:31:36.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.256+0000 7f4268e53b80 10 client.25552 fetch_fsmap learned FSMap version 21 2024-05-03T18:31:36.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.256+0000 7f4268e53b80 1 -- 172.21.15.33:0/3109063628 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x556b13927d40 con 0x556b1390c7b0 2024-05-03T18:31:36.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.256+0000 7f4241600700 1 -- 172.21.15.33:0/3109063628 <== mon.1 v2:172.21.15.150:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4250019ac0 con 0x556b1390c7b0 2024-05-03T18:31:36.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.256+0000 7f4241600700 1 -- 172.21.15.33:0/3109063628 <== mon.1 v2:172.21.15.150:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f4250004980 con 0x556b1390c7b0 2024-05-03T18:31:36.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.256+0000 7f4268e53b80 10 client.25552 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:31:36.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.256+0000 7f4268e53b80 10 client.25552 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:31:36.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.256+0000 7f4268e53b80 20 client.25552 populate_metadata read hostname 'smithi033' 2024-05-03T18:31:36.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.257+0000 7f4268e53b80 1 --2- 172.21.15.33:0/3109063628 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x556b1392a330 0x556b1392c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:36.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.257+0000 7f4268e53b80 4 client.25552 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:31:36.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.257+0000 7f4268e53b80 1 -- 172.21.15.33:0/3109063628 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556b138d51d0 con 0x556b1392a330 2024-05-03T18:31:36.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.257+0000 7f4256a00700 1 --2- 172.21.15.33:0/3109063628 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x556b1392a330 0x556b1392c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:36.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.258+0000 7f4241600700 10 client.25552 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:31:36.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.258+0000 7f4256a00700 1 --2- 172.21.15.33:0/3109063628 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x556b1392a330 0x556b1392c750 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9af4aea0004c502 server_cookie=77bb3326674b3fb0 in_seq=0 out_seq=0 2024-05-03T18:31:36.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.259+0000 7f4241600700 1 -- 172.21.15.33:0/3109063628 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556b138d51d0 con 0x556b1392a330 2024-05-03T18:31:36.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.259+0000 7f4241600700 10 client.25552 handle_command_reply: tid=0 2024-05-03T18:31:36.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.259+0000 7f4268e53b80 1 client.25552 shutdown 2024-05-03T18:31:36.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.260+0000 7f4268e53b80 1 -- 172.21.15.33:0/3109063628 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b1390c7b0 msgr2=0x556b13911270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:31:36.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.260+0000 7f4268e53b80 1 --2- 172.21.15.33:0/3109063628 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b1390c7b0 0x556b13911270 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f425000e630 tx=0x7f425000e4e0 comp rx=0 tx=0).stop 2024-05-03T18:31:36.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.260+0000 7f4268e53b80 1 -- 172.21.15.33:0/3109063628 shutdown_connections 2024-05-03T18:31:36.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.260+0000 7f4268e53b80 1 --2- 172.21.15.33:0/3109063628 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x556b1392a330 0x556b1392c750 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:36.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.260+0000 7f4268e53b80 1 --2- 172.21.15.33:0/3109063628 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b1390d0d0 0x556b139117b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:36.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.260+0000 7f4268e53b80 1 --2- 172.21.15.33:0/3109063628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b139085d0 0x556b13913e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:36.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.260+0000 7f4268e53b80 1 --2- 172.21.15.33:0/3109063628 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b1390c7b0 0x556b13911270 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:36.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.260+0000 7f4268e53b80 1 -- 172.21.15.33:0/3109063628 >> 172.21.15.33:0/3109063628 conn(0x556b13836b60 msgr2=0x556b13901650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:31:36.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.260+0000 7f4241600700 0 client.25552 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:31:36.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.262+0000 7f4268e53b80 1 -- 172.21.15.33:0/3109063628 shutdown_connections 2024-05-03T18:31:36.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.262+0000 7f4268e53b80 1 -- 172.21.15.33:0/3109063628 wait complete. 2024-05-03T18:31:36.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.262+0000 7f4268e53b80 20 client.25552 trim_cache size 0 max 16384 2024-05-03T18:31:36.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.293+0000 7f4268e53b80 1 Processor -- start 2024-05-03T18:31:36.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.294+0000 7f4268e53b80 1 -- start start 2024-05-03T18:31:36.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.294+0000 7f4268e53b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b139085d0 0x556b13913e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:36.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:31:36 smithi099 ceph-mon[31658]: pgmap v135: 129 pgs: 129 active+clean; 1.4 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 3.7 MiB/s wr, 309 op/s 2024-05-03T18:31:36.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.294+0000 7f4268e53b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b1390c7b0 0x556b139117e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:36.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.294+0000 7f4268e53b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b1390d0d0 0x556b13902660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:36.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.294+0000 7f4268e53b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f423800ccc0 con 0x556b139085d0 2024-05-03T18:31:36.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.294+0000 7f4268e53b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f423800f540 con 0x556b1390d0d0 2024-05-03T18:31:36.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.294+0000 7f4268e53b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f423800d330 con 0x556b1390c7b0 2024-05-03T18:31:36.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.296+0000 7f4257400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b1390c7b0 0x556b139117e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:36.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.296+0000 7f4257400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b1390c7b0 0x556b139117e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:33248/0 (socket says 172.21.15.33:33248) 2024-05-03T18:31:36.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.296+0000 7f4257400700 1 -- 172.21.15.33:0/151530041 learned_addr learned my addr 172.21.15.33:0/151530041 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:31:36.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.297+0000 7f4257400700 1 -- 172.21.15.33:0/151530041 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b1390d0d0 msgr2=0x556b13902660 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-03T18:31:36.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.297+0000 7f4257400700 1 --2- 172.21.15.33:0/151530041 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b1390d0d0 0x556b13902660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:36.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.297+0000 7f4257400700 1 -- 172.21.15.33:0/151530041 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b139085d0 msgr2=0x556b13913e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:31:36.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.297+0000 7f4257400700 1 --2- 172.21.15.33:0/151530041 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b139085d0 0x556b13913e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:36.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.297+0000 7f4257400700 1 -- 172.21.15.33:0/151530041 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b13906fa0 con 0x556b1390c7b0 2024-05-03T18:31:36.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.298+0000 7f4257400700 1 --2- 172.21.15.33:0/151530041 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b1390c7b0 0x556b139117e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4250005950 tx=0x7f4250011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b643fed8fb137de0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:31:36.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.299+0000 7f4243e00700 1 -- 172.21.15.33:0/151530041 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4250039070 con 0x556b1390c7b0 2024-05-03T18:31:36.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.299+0000 7f4243e00700 1 -- 172.21.15.33:0/151530041 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4250019dc0 con 0x556b1390c7b0 2024-05-03T18:31:36.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.299+0000 7f4243e00700 1 -- 172.21.15.33:0/151530041 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f425001e040 con 0x556b1390c7b0 2024-05-03T18:31:36.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.300+0000 7f4268e53b80 1 -- 172.21.15.33:0/151530041 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b1390c7b0 msgr2=0x556b139117e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:31:36.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.300+0000 7f4268e53b80 1 --2- 172.21.15.33:0/151530041 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b1390c7b0 0x556b139117e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4250005950 tx=0x7f4250011720 comp rx=0 tx=0).stop 2024-05-03T18:31:36.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.300+0000 7f4268e53b80 1 -- 172.21.15.33:0/151530041 shutdown_connections 2024-05-03T18:31:36.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.301+0000 7f4268e53b80 1 --2- 172.21.15.33:0/151530041 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b1390d0d0 0x556b13902660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:36.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.301+0000 7f4268e53b80 1 --2- 172.21.15.33:0/151530041 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b139085d0 0x556b13913e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:36.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.301+0000 7f4268e53b80 1 --2- 172.21.15.33:0/151530041 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b1390c7b0 0x556b139117e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:36.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.301+0000 7f4268e53b80 1 -- 172.21.15.33:0/151530041 >> 172.21.15.33:0/151530041 conn(0x556b13836b60 msgr2=0x556b139128b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:31:36.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.301+0000 7f4268e53b80 1 -- 172.21.15.33:0/151530041 shutdown_connections 2024-05-03T18:31:36.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.302+0000 7f4268e53b80 1 -- 172.21.15.33:0/151530041 wait complete. 2024-05-03T18:31:36.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.302+0000 7f4268e53b80 1 -- start start 2024-05-03T18:31:36.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.303+0000 7f4268e53b80 1 Processor -- start 2024-05-03T18:31:36.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.304+0000 7f4268e53b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b139085d0 0x556b1391a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:36.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.305+0000 7f4268e53b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b1390c7b0 0x556b139157a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:36.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.305+0000 7f4268e53b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b1390d0d0 0x556b13915ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:36.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.305+0000 7f4268e53b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f423800da00 con 0x556b139085d0 2024-05-03T18:31:36.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.305+0000 7f4268e53b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f423800ee70 con 0x556b1390d0d0 2024-05-03T18:31:36.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.305+0000 7f4268e53b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f423800e0d0 con 0x556b1390c7b0 2024-05-03T18:31:36.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.305+0000 7f4268e53b80 10 client.0 _ll_register_callbacks cb 0x556b13801c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:31:36.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.305+0000 7f4256a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b139085d0 0x556b1391a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:36.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.305+0000 7f4256a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b139085d0 0x556b1391a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38388/0 (socket says 172.21.15.33:38388) 2024-05-03T18:31:36.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.305+0000 7f4256a00700 1 -- 172.21.15.33:0/1495564047 learned_addr learned my addr 172.21.15.33:0/1495564047 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:31:36.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.306+0000 7f4257400700 1 --2- 172.21.15.33:0/1495564047 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b1390c7b0 0x556b139157a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:36.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.306+0000 7f4256a00700 1 --2- 172.21.15.33:0/1495564047 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b1390d0d0 0x556b13915ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:36.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.306+0000 7f4257400700 1 -- 172.21.15.33:0/1495564047 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b1390d0d0 msgr2=0x556b13915ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:31:36.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.306+0000 7f4257400700 1 --2- 172.21.15.33:0/1495564047 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b1390d0d0 0x556b13915ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:36.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.306+0000 7f4257400700 1 -- 172.21.15.33:0/1495564047 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b139085d0 msgr2=0x556b1391a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:31:36.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.307+0000 7f4257400700 1 --2- 172.21.15.33:0/1495564047 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b139085d0 0x556b1391a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:36.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.307+0000 7f4257400700 1 -- 172.21.15.33:0/1495564047 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b139162a0 con 0x556b1390c7b0 2024-05-03T18:31:36.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.307+0000 7f4256a00700 1 --2- 172.21.15.33:0/1495564047 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b1390d0d0 0x556b13915ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:31:36.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.307+0000 7f4257400700 1 --2- 172.21.15.33:0/1495564047 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b1390c7b0 0x556b139157a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x556b13914520 tx=0x7f4250009dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=193df1207e4bb070 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:31:36.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.307+0000 7f4241600700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-03T18:31:36.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.308+0000 7f4241600700 1 -- 172.21.15.33:0/1495564047 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4250039070 con 0x556b1390c7b0 2024-05-03T18:31:36.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.308+0000 7f4268e53b80 1 -- 172.21.15.33:0/1495564047 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4250019dc0 con 0x556b1390c7b0 2024-05-03T18:31:36.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.308+0000 7f4241600700 1 -- 172.21.15.33:0/1495564047 <== mon.1 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f42500045d0 con 0x556b1390c7b0 2024-05-03T18:31:36.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.308+0000 7f4241600700 1 -- 172.21.15.33:0/1495564047 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f4250039070 con 0x556b1390c7b0 2024-05-03T18:31:36.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.308+0000 7f4268e53b80 10 client.25558 fetch_fsmap learned FSMap version 21 2024-05-03T18:31:36.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.308+0000 7f4268e53b80 1 -- 172.21.15.33:0/1495564047 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x556b139946d0 con 0x556b1390c7b0 2024-05-03T18:31:36.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.309+0000 7f4241600700 1 -- 172.21.15.33:0/1495564047 <== mon.1 v2:172.21.15.150:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4250019dc0 con 0x556b1390c7b0 2024-05-03T18:31:36.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.309+0000 7f4241600700 1 -- 172.21.15.33:0/1495564047 <== mon.1 v2:172.21.15.150:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f425000e630 con 0x556b1390c7b0 2024-05-03T18:31:36.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.309+0000 7f4268e53b80 10 client.25558 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:31:36.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.309+0000 7f4268e53b80 10 client.25558 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:31:36.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.309+0000 7f4268e53b80 20 client.25558 populate_metadata read hostname 'smithi033' 2024-05-03T18:31:36.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.309+0000 7f4268e53b80 1 --2- 172.21.15.33:0/1495564047 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x556b13994cc0 0x556b139970e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:36.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.309+0000 7f4268e53b80 4 client.25558 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-03T18:31:36.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.309+0000 7f4268e53b80 1 -- 172.21.15.33:0/1495564047 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556b13868400 con 0x556b13994cc0 2024-05-03T18:31:36.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.310+0000 7f4256a00700 1 --2- 172.21.15.33:0/1495564047 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x556b13994cc0 0x556b139970e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:36.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.311+0000 7f4256a00700 1 --2- 172.21.15.33:0/1495564047 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x556b13994cc0 0x556b139970e0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52cf9c182ce7d65 server_cookie=10d51f716cb27e3a in_seq=0 out_seq=0 2024-05-03T18:31:36.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.311+0000 7f4241600700 10 client.25558 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:31:36.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.311+0000 7f4241600700 1 -- 172.21.15.33:0/1495564047 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556b13868400 con 0x556b13994cc0 2024-05-03T18:31:36.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.311+0000 7f4241600700 10 client.25558 handle_command_reply: tid=0 2024-05-03T18:31:36.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.311+0000 7f4268e53b80 1 client.25558 shutdown 2024-05-03T18:31:36.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.311+0000 7f4268e53b80 1 -- 172.21.15.33:0/1495564047 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b1390c7b0 msgr2=0x556b139157a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:31:36.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.311+0000 7f4268e53b80 1 --2- 172.21.15.33:0/1495564047 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b1390c7b0 0x556b139157a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x556b13914520 tx=0x7f4250009dd0 comp rx=0 tx=0).stop 2024-05-03T18:31:36.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.312+0000 7f4268e53b80 1 -- 172.21.15.33:0/1495564047 shutdown_connections 2024-05-03T18:31:36.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.312+0000 7f4268e53b80 1 --2- 172.21.15.33:0/1495564047 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x556b13994cc0 0x556b139970e0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:36.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.312+0000 7f4268e53b80 1 --2- 172.21.15.33:0/1495564047 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b1390d0d0 0x556b13915ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:36.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.312+0000 7f4268e53b80 1 --2- 172.21.15.33:0/1495564047 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b139085d0 0x556b1391a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:36.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.312+0000 7f4268e53b80 1 --2- 172.21.15.33:0/1495564047 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b1390c7b0 0x556b139157a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:36.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.312+0000 7f4268e53b80 1 -- 172.21.15.33:0/1495564047 >> 172.21.15.33:0/1495564047 conn(0x556b13836b60 msgr2=0x556b1390bc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:31:36.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.312+0000 7f4241600700 0 client.25558 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:31:36.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.312+0000 7f4268e53b80 1 -- 172.21.15.33:0/1495564047 shutdown_connections 2024-05-03T18:31:36.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.312+0000 7f4268e53b80 1 -- 172.21.15.33:0/1495564047 wait complete. 2024-05-03T18:31:36.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.313+0000 7f4268e53b80 20 client.25558 trim_cache size 0 max 16384 2024-05-03T18:31:36.315 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-03T18:31:36.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.314+0000 7f425e400700 1 -- 172.21.15.33:0/2038357459 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f423c0b2b00 msgr2=0x7f423c0b4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:31:36.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.314+0000 7f425e400700 1 --2- 172.21.15.33:0/2038357459 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f423c0b2b00 0x7f423c0b4fc0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f42440dff90 tx=0x7f424408ff90 comp rx=0 tx=0).stop 2024-05-03T18:31:36.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.314+0000 7f425e400700 1 -- 172.21.15.33:0/2038357459 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4258118530 msgr2=0x7f42581a8dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:31:36.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.314+0000 7f425e400700 1 --2- 172.21.15.33:0/2038357459 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4258118530 0x7f42581a8dc0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f424400e940 tx=0x7f424400ec50 comp rx=0 tx=0).stop 2024-05-03T18:31:36.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.314+0000 7f425e400700 1 -- 172.21.15.33:0/2038357459 shutdown_connections 2024-05-03T18:31:36.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.314+0000 7f425e400700 1 --2- 172.21.15.33:0/2038357459 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4258118530 0x7f42581a8dc0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:36.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.314+0000 7f425e400700 1 --2- 172.21.15.33:0/2038357459 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f423c0b2b00 0x7f423c0b4fc0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:36.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.314+0000 7f425e400700 1 --2- 172.21.15.33:0/2038357459 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f425810a9e0 0x7f42581a44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:36.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.314+0000 7f425e400700 1 --2- 172.21.15.33:0/2038357459 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f425810db90 0x7f42581a4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:36.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.314+0000 7f425e400700 1 -- 172.21.15.33:0/2038357459 >> 172.21.15.33:0/2038357459 conn(0x7f425806e9f0 msgr2=0x7f4258119ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:31:36.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.314+0000 7f425e400700 1 -- 172.21.15.33:0/2038357459 shutdown_connections 2024-05-03T18:31:36.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:36.315+0000 7f425e400700 1 -- 172.21.15.33:0/2038357459 wait complete. 2024-05-03T18:31:36.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:31:36 smithi150 ceph-mon[32492]: pgmap v135: 129 pgs: 129 active+clean; 1.4 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 3.7 MiB/s wr, 309 op/s 2024-05-03T18:31:37.282 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:31:37 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:31:37.282 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:31:37 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:31:37.282 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:31:37 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:31:37.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:31:37 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:31:37.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:31:37 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:31:37.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:31:37 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:31:37.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:31:37 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:31:37.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:31:37 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:31:37.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:31:37 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:31:38.284 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-03T18:31:38.285 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-03T18:31:38.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:31:38 smithi099 ceph-mon[31658]: pgmap v136: 129 pgs: 129 active+clean; 1.4 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 6.8 MiB/s wr, 677 op/s 2024-05-03T18:31:38.378 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:31:38 smithi033 ceph-mon[28330]: pgmap v136: 129 pgs: 129 active+clean; 1.4 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 6.8 MiB/s wr, 677 op/s 2024-05-03T18:31:38.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:31:38 smithi150 ceph-mon[32492]: pgmap v136: 129 pgs: 129 active+clean; 1.4 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 6.8 MiB/s wr, 677 op/s 2024-05-03T18:31:38.978 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:31:40.110 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 25.0 s, 621.8 tps, lat 162.288 ms stddev 58.671 2024-05-03T18:31:41.193 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:31:40 smithi033 ceph-mon[28330]: pgmap v137: 129 pgs: 129 active+clean; 1.4 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.1 MiB/s wr, 612 op/s 2024-05-03T18:31:41.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:31:40 smithi099 ceph-mon[31658]: pgmap v137: 129 pgs: 129 active+clean; 1.4 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.1 MiB/s wr, 612 op/s 2024-05-03T18:31:41.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:31:40 smithi150 ceph-mon[32492]: pgmap v137: 129 pgs: 129 active+clean; 1.4 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.1 MiB/s wr, 612 op/s 2024-05-03T18:31:41.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.452+0000 7fb491c00700 1 -- 172.21.15.33:0/4186722332 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb48c1023a0 msgr2=0x7fb48c102820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:31:41.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.452+0000 7fb491c00700 1 --2- 172.21.15.33:0/4186722332 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb48c1023a0 0x7fb48c102820 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fb478009990 tx=0x7fb478009ca0 comp rx=0 tx=0).stop 2024-05-03T18:31:41.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.452+0000 7fb491c00700 1 -- 172.21.15.33:0/4186722332 shutdown_connections 2024-05-03T18:31:41.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.452+0000 7fb491c00700 1 --2- 172.21.15.33:0/4186722332 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb48c102d60 0x7fb48c10f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:41.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.452+0000 7fb491c00700 1 --2- 172.21.15.33:0/4186722332 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb48c108410 0x7fb48c1087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:41.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.452+0000 7fb491c00700 1 --2- 172.21.15.33:0/4186722332 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb48c1023a0 0x7fb48c102820 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:41.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.452+0000 7fb491c00700 1 -- 172.21.15.33:0/4186722332 >> 172.21.15.33:0/4186722332 conn(0x7fb48c079350 msgr2=0x7fb48c079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:31:41.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.454+0000 7fb491c00700 1 -- 172.21.15.33:0/4186722332 shutdown_connections 2024-05-03T18:31:41.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.454+0000 7fb491c00700 1 -- 172.21.15.33:0/4186722332 wait complete. 2024-05-03T18:31:41.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.456+0000 7fb491c00700 1 Processor -- start 2024-05-03T18:31:41.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.457+0000 7fb491c00700 1 -- start start 2024-05-03T18:31:41.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.458+0000 7fb491c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb48c1023a0 0x7fb48c1a02e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:41.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.458+0000 7fb491c00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb48c102d60 0x7fb48c1a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:41.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.458+0000 7fb491c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb48c108410 0x7fb48c1a49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:41.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.459+0000 7fb48aa00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb48c1023a0 0x7fb48c1a02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:41.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.459+0000 7fb48a000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb48c102d60 0x7fb48c1a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:41.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.459+0000 7fb48aa00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb48c1023a0 0x7fb48c1a02e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:33272/0 (socket says 172.21.15.33:33272) 2024-05-03T18:31:41.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.459+0000 7fb48aa00700 1 -- 172.21.15.33:0/2840613238 learned_addr learned my addr 172.21.15.33:0/2840613238 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:31:41.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.459+0000 7fb491c00700 1 -- 172.21.15.33:0/2840613238 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb48c112430 con 0x7fb48c102d60 2024-05-03T18:31:41.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.459+0000 7fb491c00700 1 -- 172.21.15.33:0/2840613238 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb48c1122b0 con 0x7fb48c108410 2024-05-03T18:31:41.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.459+0000 7fb491c00700 1 -- 172.21.15.33:0/2840613238 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb48c1125b0 con 0x7fb48c1023a0 2024-05-03T18:31:41.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.459+0000 7fb48b400700 1 --2- 172.21.15.33:0/2840613238 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb48c108410 0x7fb48c1a49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:41.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.460+0000 7fb48aa00700 1 -- 172.21.15.33:0/2840613238 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb48c108410 msgr2=0x7fb48c1a49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:31:41.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.460+0000 7fb48aa00700 1 --2- 172.21.15.33:0/2840613238 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb48c108410 0x7fb48c1a49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:41.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.460+0000 7fb48aa00700 1 -- 172.21.15.33:0/2840613238 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb48c102d60 msgr2=0x7fb48c1a0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:31:41.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.460+0000 7fb48aa00700 1 --2- 172.21.15.33:0/2840613238 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb48c102d60 0x7fb48c1a0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:41.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.460+0000 7fb48aa00700 1 -- 172.21.15.33:0/2840613238 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb48c1a50e0 con 0x7fb48c1023a0 2024-05-03T18:31:41.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.460+0000 7fb48a000700 1 --2- 172.21.15.33:0/2840613238 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb48c102d60 0x7fb48c1a0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:31:41.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.461+0000 7fb48aa00700 1 --2- 172.21.15.33:0/2840613238 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb48c1023a0 0x7fb48c1a02e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb47c00d950 tx=0x7fb47c00dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:31:41.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.462+0000 7fb483400700 1 -- 172.21.15.33:0/2840613238 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb47c016070 con 0x7fb48c1023a0 2024-05-03T18:31:41.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.462+0000 7fb483400700 1 -- 172.21.15.33:0/2840613238 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb47c00fd20 con 0x7fb48c1023a0 2024-05-03T18:31:41.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.462+0000 7fb483400700 1 -- 172.21.15.33:0/2840613238 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb47c00e3b0 con 0x7fb48c1023a0 2024-05-03T18:31:41.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.463+0000 7fb491c00700 1 -- 172.21.15.33:0/2840613238 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb48c1a53d0 con 0x7fb48c1023a0 2024-05-03T18:31:41.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.463+0000 7fb491c00700 1 -- 172.21.15.33:0/2840613238 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb48c1acc80 con 0x7fb48c1023a0 2024-05-03T18:31:41.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.472+0000 7fb483400700 1 -- 172.21.15.33:0/2840613238 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fb47c019070 con 0x7fb48c1023a0 2024-05-03T18:31:41.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.474+0000 7fb49c67ab80 1 Processor -- start 2024-05-03T18:31:41.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.475+0000 7fb483400700 1 --2- 172.21.15.33:0/2840613238 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fb4700b2af0 0x7fb4700b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:41.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.475+0000 7fb483400700 1 -- 172.21.15.33:0/2840613238 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7fb47c0e2370 con 0x7fb48c1023a0 2024-05-03T18:31:41.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.476+0000 7fb49c67ab80 1 -- start start 2024-05-03T18:31:41.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.476+0000 7fb49c67ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56036703c830 0x56036703cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:41.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.476+0000 7fb49c67ab80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56036703d150 0x560367038030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:41.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.476+0000 7fb49c67ab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560367038650 0x560367038ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:41.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.476+0000 7fb49c67ab80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560366f8aaa0 con 0x56036703d150 2024-05-03T18:31:41.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.476+0000 7fb49c67ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x560366f287e0 con 0x56036703c830 2024-05-03T18:31:41.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.476+0000 7fb49c67ab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x560366f86a60 con 0x560367038650 2024-05-03T18:31:41.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.477+0000 7fb48a000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56036703d150 0x560367038030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:41.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.477+0000 7fb48a000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56036703d150 0x560367038030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38420/0 (socket says 172.21.15.33:38420) 2024-05-03T18:31:41.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.477+0000 7fb48a000700 1 -- 172.21.15.33:0/857911547 learned_addr learned my addr 172.21.15.33:0/857911547 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:31:41.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.477+0000 7fb48b400700 1 --2- 172.21.15.33:0/2840613238 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fb4700b2af0 0x7fb4700b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:41.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.477+0000 7fb48b400700 1 --2- 172.21.15.33:0/857911547 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560367038650 0x560367038ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:41.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.477+0000 7fb48aa00700 1 --2- 172.21.15.33:0/857911547 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56036703c830 0x56036703cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:41.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.477+0000 7fb48b400700 1 -- 172.21.15.33:0/857911547 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56036703c830 msgr2=0x56036703cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:31:41.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.477+0000 7fb48b400700 1 --2- 172.21.15.33:0/857911547 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56036703c830 0x56036703cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:41.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.477+0000 7fb48b400700 1 -- 172.21.15.33:0/857911547 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56036703d150 msgr2=0x560367038030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:31:41.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.477+0000 7fb48b400700 1 --2- 172.21.15.33:0/857911547 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56036703d150 0x560367038030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:41.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.477+0000 7fb48b400700 1 -- 172.21.15.33:0/857911547 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560366ed7950 con 0x560367038650 2024-05-03T18:31:41.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.478+0000 7fb48b400700 1 --2- 172.21.15.33:0/857911547 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560367038650 0x560367038ab0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb484007e20 tx=0x7fb484019650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=812c9bcc2aa0bd03 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:31:41.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.478+0000 7fb48b400700 1 --2- 172.21.15.33:0/2840613238 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fb4700b2af0 0x7fb4700b4fb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb484009fd0 tx=0x7fb48400f000 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:31:41.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.479+0000 7fb477e00700 1 -- 172.21.15.33:0/857911547 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb48401f070 con 0x560367038650 2024-05-03T18:31:41.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.479+0000 7fb477e00700 1 -- 172.21.15.33:0/857911547 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb484009ab0 con 0x560367038650 2024-05-03T18:31:41.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.480+0000 7fb477e00700 1 -- 172.21.15.33:0/857911547 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb48401b990 con 0x560367038650 2024-05-03T18:31:41.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.481+0000 7fb49c67ab80 1 -- 172.21.15.33:0/857911547 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560367038650 msgr2=0x560367038ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:31:41.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.481+0000 7fb49c67ab80 1 --2- 172.21.15.33:0/857911547 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560367038650 0x560367038ab0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb484007e20 tx=0x7fb484019650 comp rx=0 tx=0).stop 2024-05-03T18:31:41.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.481+0000 7fb49c67ab80 1 -- 172.21.15.33:0/857911547 shutdown_connections 2024-05-03T18:31:41.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.481+0000 7fb49c67ab80 1 --2- 172.21.15.33:0/857911547 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560367038650 0x560367038ab0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:41.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.481+0000 7fb49c67ab80 1 --2- 172.21.15.33:0/857911547 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56036703c830 0x56036703cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:41.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.481+0000 7fb49c67ab80 1 --2- 172.21.15.33:0/857911547 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56036703d150 0x560367038030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:41.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.481+0000 7fb49c67ab80 1 -- 172.21.15.33:0/857911547 >> 172.21.15.33:0/857911547 conn(0x560366f66be0 msgr2=0x560367032870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:31:41.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.481+0000 7fb49c67ab80 1 -- 172.21.15.33:0/857911547 shutdown_connections 2024-05-03T18:31:41.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.482+0000 7fb49c67ab80 1 -- 172.21.15.33:0/857911547 wait complete. 2024-05-03T18:31:41.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.483+0000 7fb49c67ab80 1 -- start start 2024-05-03T18:31:41.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.483+0000 7fb49c67ab80 1 Processor -- start 2024-05-03T18:31:41.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.484+0000 7fb49c67ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560367038650 0x560367043f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:41.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.484+0000 7fb49c67ab80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56036703c830 0x5603670412e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:41.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.484+0000 7fb49c67ab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56036703d150 0x560367041820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:41.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.484+0000 7fb49c67ab80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560367044c60 con 0x56036703c830 2024-05-03T18:31:41.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.484+0000 7fb49c67ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x560366ef0ba0 con 0x560367038650 2024-05-03T18:31:41.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.484+0000 7fb49c67ab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x560367044de0 con 0x56036703d150 2024-05-03T18:31:41.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.484+0000 7fb49c67ab80 10 client.0 _ll_register_callbacks cb 0x560366f31cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:31:41.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.484+0000 7fb48a000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56036703c830 0x5603670412e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:41.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.484+0000 7fb48aa00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560367038650 0x560367043f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:41.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.484+0000 7fb48a000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56036703c830 0x5603670412e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38424/0 (socket says 172.21.15.33:38424) 2024-05-03T18:31:41.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.485+0000 7fb48a000700 1 -- 172.21.15.33:0/1232904403 learned_addr learned my addr 172.21.15.33:0/1232904403 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:31:41.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.485+0000 7fb48b400700 1 --2- 172.21.15.33:0/1232904403 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56036703d150 0x560367041820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:41.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.486+0000 7fb48b400700 1 -- 172.21.15.33:0/1232904403 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560367038650 msgr2=0x560367043f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:31:41.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.486+0000 7fb48b400700 1 --2- 172.21.15.33:0/1232904403 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560367038650 0x560367043f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:41.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.486+0000 7fb48b400700 1 -- 172.21.15.33:0/1232904403 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56036703c830 msgr2=0x5603670412e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:31:41.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.486+0000 7fb48b400700 1 --2- 172.21.15.33:0/1232904403 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56036703c830 0x5603670412e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:41.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.486+0000 7fb48b400700 1 -- 172.21.15.33:0/1232904403 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560366f504a0 con 0x56036703d150 2024-05-03T18:31:41.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.486+0000 7fb48a000700 1 --2- 172.21.15.33:0/1232904403 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56036703c830 0x5603670412e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:31:41.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.486+0000 7fb48b400700 1 --2- 172.21.15.33:0/1232904403 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56036703d150 0x560367041820 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb484007e20 tx=0x7fb484004a10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e90b7fee3fcd85fa server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:31:41.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.486+0000 7fb474c00700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-03T18:31:41.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.487+0000 7fb474c00700 1 -- 172.21.15.33:0/1232904403 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb484031060 con 0x56036703d150 2024-05-03T18:31:41.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.487+0000 7fb474c00700 1 -- 172.21.15.33:0/1232904403 <== mon.1 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7fb48401bc00 con 0x56036703d150 2024-05-03T18:31:41.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.487+0000 7fb49c67ab80 1 -- 172.21.15.33:0/1232904403 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb484009ab0 con 0x56036703d150 2024-05-03T18:31:41.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.488+0000 7fb474c00700 1 -- 172.21.15.33:0/1232904403 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fb48401f050 con 0x56036703d150 2024-05-03T18:31:41.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.488+0000 7fb49c67ab80 10 client.25570 fetch_fsmap learned FSMap version 21 2024-05-03T18:31:41.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.488+0000 7fb49c67ab80 1 -- 172.21.15.33:0/1232904403 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x56036704f4f0 con 0x56036703d150 2024-05-03T18:31:41.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.489+0000 7fb474c00700 1 -- 172.21.15.33:0/1232904403 <== mon.1 v2:172.21.15.150:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb484009ab0 con 0x56036703d150 2024-05-03T18:31:41.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.491+0000 7fb474c00700 1 -- 172.21.15.33:0/1232904403 <== mon.1 v2:172.21.15.150:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fb484018040 con 0x56036703d150 2024-05-03T18:31:41.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.491+0000 7fb49c67ab80 10 client.25570 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:31:41.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.491+0000 7fb49c67ab80 10 client.25570 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:31:41.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.491+0000 7fb49c67ab80 20 client.25570 populate_metadata read hostname 'smithi033' 2024-05-03T18:31:41.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.491+0000 7fb49c67ab80 1 --2- 172.21.15.33:0/1232904403 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x560367051a20 0x560367053e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:41.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.491+0000 7fb49c67ab80 4 client.25570 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:31:41.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.491+0000 7fb49c67ab80 1 -- 172.21.15.33:0/1232904403 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560367005250 con 0x560367051a20 2024-05-03T18:31:41.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.492+0000 7fb48aa00700 1 --2- 172.21.15.33:0/1232904403 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x560367051a20 0x560367053e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:41.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.493+0000 7fb48aa00700 1 --2- 172.21.15.33:0/1232904403 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x560367051a20 0x560367053e10 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44e15b5abae504e9 server_cookie=7cfe3ea1bb8cc949 in_seq=0 out_seq=0 2024-05-03T18:31:41.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.493+0000 7fb474c00700 10 client.25570 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:31:41.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.496+0000 7fb474c00700 1 -- 172.21.15.33:0/1232904403 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560367005250 con 0x560367051a20 2024-05-03T18:31:41.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.496+0000 7fb474c00700 10 client.25570 handle_command_reply: tid=0 2024-05-03T18:31:41.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.496+0000 7fb49c67ab80 1 client.25570 shutdown 2024-05-03T18:31:41.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.497+0000 7fb49c67ab80 1 -- 172.21.15.33:0/1232904403 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56036703d150 msgr2=0x560367041820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:31:41.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.497+0000 7fb49c67ab80 1 --2- 172.21.15.33:0/1232904403 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56036703d150 0x560367041820 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb484007e20 tx=0x7fb484004a10 comp rx=0 tx=0).stop 2024-05-03T18:31:41.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.497+0000 7fb49c67ab80 1 -- 172.21.15.33:0/1232904403 shutdown_connections 2024-05-03T18:31:41.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.497+0000 7fb49c67ab80 1 --2- 172.21.15.33:0/1232904403 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x560367051a20 0x560367053e10 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:41.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.497+0000 7fb49c67ab80 1 --2- 172.21.15.33:0/1232904403 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56036703d150 0x560367041820 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:41.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.497+0000 7fb49c67ab80 1 --2- 172.21.15.33:0/1232904403 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560367038650 0x560367043f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:41.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.497+0000 7fb49c67ab80 1 --2- 172.21.15.33:0/1232904403 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56036703c830 0x5603670412e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:41.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.497+0000 7fb49c67ab80 1 -- 172.21.15.33:0/1232904403 >> 172.21.15.33:0/1232904403 conn(0x560366f66be0 msgr2=0x56036703de40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:31:41.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.497+0000 7fb474c00700 0 client.25570 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:31:41.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.498+0000 7fb49c67ab80 1 -- 172.21.15.33:0/1232904403 shutdown_connections 2024-05-03T18:31:41.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.498+0000 7fb49c67ab80 1 -- 172.21.15.33:0/1232904403 wait complete. 2024-05-03T18:31:41.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.498+0000 7fb49c67ab80 20 client.25570 trim_cache size 0 max 16384 2024-05-03T18:31:41.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.542+0000 7fb49c67ab80 1 Processor -- start 2024-05-03T18:31:41.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.542+0000 7fb49c67ab80 1 -- start start 2024-05-03T18:31:41.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.543+0000 7fb49c67ab80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560367038650 0x5603670326e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:41.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.543+0000 7fb49c67ab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56036703c830 0x560367037a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:41.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.543+0000 7fb49c67ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56036703d150 0x560367037fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:41.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.543+0000 7fb49c67ab80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb46c00cc70 con 0x560367038650 2024-05-03T18:31:41.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.543+0000 7fb49c67ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb46c00f530 con 0x56036703d150 2024-05-03T18:31:41.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.543+0000 7fb49c67ab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb46c00d320 con 0x56036703c830 2024-05-03T18:31:41.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.544+0000 7fb48aa00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560367038650 0x5603670326e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:41.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.544+0000 7fb48aa00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560367038650 0x5603670326e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38440/0 (socket says 172.21.15.33:38440) 2024-05-03T18:31:41.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.545+0000 7fb48a000700 1 --2- 172.21.15.33:0/1855516482 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56036703c830 0x560367037a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:41.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.544+0000 7fb48aa00700 1 -- 172.21.15.33:0/1855516482 learned_addr learned my addr 172.21.15.33:0/1855516482 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:31:41.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.545+0000 7fb48b400700 1 --2- 172.21.15.33:0/1855516482 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56036703d150 0x560367037fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:41.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.546+0000 7fb48b400700 1 -- 172.21.15.33:0/1855516482 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56036703c830 msgr2=0x560367037a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:31:41.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.546+0000 7fb48b400700 1 --2- 172.21.15.33:0/1855516482 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56036703c830 0x560367037a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:41.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.546+0000 7fb48b400700 1 -- 172.21.15.33:0/1855516482 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560367038650 msgr2=0x5603670326e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:31:41.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.547+0000 7fb48b400700 1 --2- 172.21.15.33:0/1855516482 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560367038650 0x5603670326e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:41.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.547+0000 7fb48b400700 1 -- 172.21.15.33:0/1855516482 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56036703f850 con 0x56036703d150 2024-05-03T18:31:41.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.548+0000 7fb48b400700 1 --2- 172.21.15.33:0/1855516482 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56036703d150 0x560367037fd0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb484007cc0 tx=0x7fb484019650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eebdd5f02762d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:31:41.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.550+0000 7fb477400700 1 -- 172.21.15.33:0/1855516482 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb48403b080 con 0x56036703d150 2024-05-03T18:31:41.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.550+0000 7fb477400700 1 -- 172.21.15.33:0/1855516482 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb484009460 con 0x56036703d150 2024-05-03T18:31:41.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.550+0000 7fb477400700 1 -- 172.21.15.33:0/1855516482 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb48401f050 con 0x56036703d150 2024-05-03T18:31:41.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.552+0000 7fb49c67ab80 1 -- 172.21.15.33:0/1855516482 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56036703d150 msgr2=0x560367037fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:31:41.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.552+0000 7fb49c67ab80 1 --2- 172.21.15.33:0/1855516482 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56036703d150 0x560367037fd0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb484007cc0 tx=0x7fb484019650 comp rx=0 tx=0).stop 2024-05-03T18:31:41.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.552+0000 7fb49c67ab80 1 -- 172.21.15.33:0/1855516482 shutdown_connections 2024-05-03T18:31:41.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.552+0000 7fb49c67ab80 1 --2- 172.21.15.33:0/1855516482 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56036703d150 0x560367037fd0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:41.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.553+0000 7fb49c67ab80 1 --2- 172.21.15.33:0/1855516482 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560367038650 0x5603670326e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:41.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.553+0000 7fb49c67ab80 1 --2- 172.21.15.33:0/1855516482 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56036703c830 0x560367037a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:41.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.553+0000 7fb49c67ab80 1 -- 172.21.15.33:0/1855516482 >> 172.21.15.33:0/1855516482 conn(0x560366f66be0 msgr2=0x560367037380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:31:41.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.553+0000 7fb49c67ab80 1 -- 172.21.15.33:0/1855516482 shutdown_connections 2024-05-03T18:31:41.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.553+0000 7fb49c67ab80 1 -- 172.21.15.33:0/1855516482 wait complete. 2024-05-03T18:31:41.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.554+0000 7fb49c67ab80 1 -- start start 2024-05-03T18:31:41.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.554+0000 7fb49c67ab80 1 Processor -- start 2024-05-03T18:31:41.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.555+0000 7fb49c67ab80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560367038650 0x5603670450c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:41.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.556+0000 7fb49c67ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56036703c830 0x560367045600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:41.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.556+0000 7fb49c67ab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56036703d150 0x560367045b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:41.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.556+0000 7fb49c67ab80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb46c00d9f0 con 0x560367038650 2024-05-03T18:31:41.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.556+0000 7fb49c67ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb46c00ee60 con 0x56036703c830 2024-05-03T18:31:41.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.556+0000 7fb49c67ab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb46c00e0c0 con 0x56036703d150 2024-05-03T18:31:41.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.556+0000 7fb49c67ab80 10 client.0 _ll_register_callbacks cb 0x560366f31cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:31:41.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.556+0000 7fb48aa00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560367038650 0x5603670450c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:41.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.556+0000 7fb48b400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56036703d150 0x560367045b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:41.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.556+0000 7fb48b400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56036703d150 0x560367045b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:33294/0 (socket says 172.21.15.33:33294) 2024-05-03T18:31:41.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.556+0000 7fb48b400700 1 -- 172.21.15.33:0/4145861749 learned_addr learned my addr 172.21.15.33:0/4145861749 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:31:41.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.556+0000 7fb48a000700 1 --2- 172.21.15.33:0/4145861749 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56036703c830 0x560367045600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:41.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.557+0000 7fb48b400700 1 -- 172.21.15.33:0/4145861749 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56036703c830 msgr2=0x560367045600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:31:41.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.557+0000 7fb48b400700 1 --2- 172.21.15.33:0/4145861749 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56036703c830 0x560367045600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:41.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.557+0000 7fb48b400700 1 -- 172.21.15.33:0/4145861749 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560367038650 msgr2=0x5603670450c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:31:41.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.557+0000 7fb48b400700 1 --2- 172.21.15.33:0/4145861749 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560367038650 0x5603670450c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:41.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.557+0000 7fb48b400700 1 -- 172.21.15.33:0/4145861749 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603670cd000 con 0x56036703d150 2024-05-03T18:31:41.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.558+0000 7fb48b400700 1 --2- 172.21.15.33:0/4145861749 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56036703d150 0x560367045b40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb4840061e0 tx=0x7fb484019a30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6364d17cd258f1af server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:31:41.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.558+0000 7fb474c00700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-03T18:31:41.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.558+0000 7fb474c00700 1 -- 172.21.15.33:0/4145861749 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb48401f040 con 0x56036703d150 2024-05-03T18:31:41.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.558+0000 7fb474c00700 1 -- 172.21.15.33:0/4145861749 <== mon.1 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7fb484003c00 con 0x56036703d150 2024-05-03T18:31:41.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.559+0000 7fb49c67ab80 1 -- 172.21.15.33:0/4145861749 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb484009460 con 0x56036703d150 2024-05-03T18:31:41.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.559+0000 7fb474c00700 1 -- 172.21.15.33:0/4145861749 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fb484003e00 con 0x56036703d150 2024-05-03T18:31:41.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.559+0000 7fb49c67ab80 10 client.25576 fetch_fsmap learned FSMap version 21 2024-05-03T18:31:41.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.559+0000 7fb49c67ab80 1 -- 172.21.15.33:0/4145861749 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5603670ce320 con 0x56036703d150 2024-05-03T18:31:41.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.560+0000 7fb474c00700 1 -- 172.21.15.33:0/4145861749 <== mon.1 v2:172.21.15.150:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb484009460 con 0x56036703d150 2024-05-03T18:31:41.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.561+0000 7fb474c00700 1 -- 172.21.15.33:0/4145861749 <== mon.1 v2:172.21.15.150:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fb484048030 con 0x56036703d150 2024-05-03T18:31:41.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.561+0000 7fb49c67ab80 10 client.25576 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:31:41.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.561+0000 7fb49c67ab80 10 client.25576 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:31:41.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.561+0000 7fb49c67ab80 20 client.25576 populate_metadata read hostname 'smithi033' 2024-05-03T18:31:41.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.561+0000 7fb49c67ab80 1 --2- 172.21.15.33:0/4145861749 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5603670ce960 0x5603670d0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:41.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.561+0000 7fb49c67ab80 4 client.25576 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-03T18:31:41.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.561+0000 7fb49c67ab80 1 -- 172.21.15.33:0/4145861749 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x560366f98480 con 0x5603670ce960 2024-05-03T18:31:41.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.562+0000 7fb48aa00700 1 --2- 172.21.15.33:0/4145861749 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5603670ce960 0x5603670d0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:41.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.564+0000 7fb48aa00700 1 --2- 172.21.15.33:0/4145861749 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5603670ce960 0x5603670d0d50 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91c218b874c5fd2 server_cookie=72186372f5f25585 in_seq=0 out_seq=0 2024-05-03T18:31:41.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.564+0000 7fb474c00700 10 client.25576 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:31:41.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.567+0000 7fb474c00700 1 -- 172.21.15.33:0/4145861749 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560366f98480 con 0x5603670ce960 2024-05-03T18:31:41.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.567+0000 7fb474c00700 10 client.25576 handle_command_reply: tid=0 2024-05-03T18:31:41.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.567+0000 7fb49c67ab80 1 client.25576 shutdown 2024-05-03T18:31:41.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.568+0000 7fb49c67ab80 1 -- 172.21.15.33:0/4145861749 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56036703d150 msgr2=0x560367045b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:31:41.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.568+0000 7fb49c67ab80 1 --2- 172.21.15.33:0/4145861749 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56036703d150 0x560367045b40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb4840061e0 tx=0x7fb484019a30 comp rx=0 tx=0).stop 2024-05-03T18:31:41.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.568+0000 7fb49c67ab80 1 -- 172.21.15.33:0/4145861749 shutdown_connections 2024-05-03T18:31:41.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.568+0000 7fb49c67ab80 1 --2- 172.21.15.33:0/4145861749 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5603670ce960 0x5603670d0d50 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:41.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.568+0000 7fb49c67ab80 1 --2- 172.21.15.33:0/4145861749 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56036703d150 0x560367045b40 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:41.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.568+0000 7fb49c67ab80 1 --2- 172.21.15.33:0/4145861749 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560367038650 0x5603670450c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:41.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.568+0000 7fb49c67ab80 1 --2- 172.21.15.33:0/4145861749 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56036703c830 0x560367045600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:41.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.568+0000 7fb474c00700 0 client.25576 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:31:41.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.568+0000 7fb49c67ab80 1 -- 172.21.15.33:0/4145861749 >> 172.21.15.33:0/4145861749 conn(0x560366f66be0 msgr2=0x560367043170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:31:41.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.571+0000 7fb49c67ab80 1 -- 172.21.15.33:0/4145861749 shutdown_connections 2024-05-03T18:31:41.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.571+0000 7fb49c67ab80 1 -- 172.21.15.33:0/4145861749 wait complete. 2024-05-03T18:31:41.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.572+0000 7fb49c67ab80 20 client.25576 trim_cache size 0 max 16384 2024-05-03T18:31:41.574 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-03T18:31:41.574 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-05-03T18:31:41.574 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "fc07b1f0-3656-41e0-9c58-d4c804568704", 2024-05-03T18:31:41.574 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-05-03T18:31:41.574 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-03T18:31:41.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.573+0000 7fb491c00700 1 -- 172.21.15.33:0/2840613238 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fb4700b2af0 msgr2=0x7fb4700b4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:31:41.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.573+0000 7fb491c00700 1 --2- 172.21.15.33:0/2840613238 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fb4700b2af0 0x7fb4700b4fb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb484009fd0 tx=0x7fb48400f000 comp rx=0 tx=0).stop 2024-05-03T18:31:41.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.573+0000 7fb491c00700 1 -- 172.21.15.33:0/2840613238 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb48c1023a0 msgr2=0x7fb48c1a02e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:31:41.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.573+0000 7fb491c00700 1 --2- 172.21.15.33:0/2840613238 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb48c1023a0 0x7fb48c1a02e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb47c00d950 tx=0x7fb47c00dc60 comp rx=0 tx=0).stop 2024-05-03T18:31:41.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.574+0000 7fb491c00700 1 -- 172.21.15.33:0/2840613238 shutdown_connections 2024-05-03T18:31:41.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.574+0000 7fb491c00700 1 --2- 172.21.15.33:0/2840613238 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fb4700b2af0 0x7fb4700b4fb0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:41.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.574+0000 7fb491c00700 1 --2- 172.21.15.33:0/2840613238 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb48c108410 0x7fb48c1a49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:41.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.574+0000 7fb491c00700 1 --2- 172.21.15.33:0/2840613238 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb48c1023a0 0x7fb48c1a02e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:41.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.574+0000 7fb491c00700 1 --2- 172.21.15.33:0/2840613238 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb48c102d60 0x7fb48c1a0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:41.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.574+0000 7fb491c00700 1 -- 172.21.15.33:0/2840613238 >> 172.21.15.33:0/2840613238 conn(0x7fb48c079350 msgr2=0x7fb48c06b3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:31:41.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.575+0000 7fb491c00700 1 -- 172.21.15.33:0/2840613238 shutdown_connections 2024-05-03T18:31:41.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:41.575+0000 7fb491c00700 1 -- 172.21.15.33:0/2840613238 wait complete. 2024-05-03T18:31:42.534 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-03T18:31:43.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:31:42 smithi033 ceph-mon[28330]: scrub queued for path: / 2024-05-03T18:31:43.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:31:42 smithi033 ceph-mon[28330]: scrub summary: idle+waiting paths [/] 2024-05-03T18:31:43.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:31:42 smithi033 ceph-mon[28330]: scrub summary: active paths [/] 2024-05-03T18:31:43.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:31:42 smithi033 ceph-mon[28330]: scrub summary: idle+waiting paths [/] 2024-05-03T18:31:43.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:31:42 smithi033 ceph-mon[28330]: pgmap v138: 129 pgs: 129 active+clean; 1.4 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.1 MiB/s wr, 612 op/s 2024-05-03T18:31:43.248 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:31:43.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:31:42 smithi099 ceph-mon[31658]: scrub queued for path: / 2024-05-03T18:31:43.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:31:42 smithi099 ceph-mon[31658]: scrub summary: idle+waiting paths [/] 2024-05-03T18:31:43.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:31:42 smithi099 ceph-mon[31658]: scrub summary: active paths [/] 2024-05-03T18:31:43.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:31:42 smithi099 ceph-mon[31658]: scrub summary: idle+waiting paths [/] 2024-05-03T18:31:43.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:31:42 smithi099 ceph-mon[31658]: pgmap v138: 129 pgs: 129 active+clean; 1.4 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.1 MiB/s wr, 612 op/s 2024-05-03T18:31:43.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:31:42 smithi150 ceph-mon[32492]: scrub queued for path: / 2024-05-03T18:31:43.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:31:42 smithi150 ceph-mon[32492]: scrub summary: idle+waiting paths [/] 2024-05-03T18:31:43.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:31:42 smithi150 ceph-mon[32492]: scrub summary: active paths [/] 2024-05-03T18:31:43.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:31:42 smithi150 ceph-mon[32492]: scrub summary: idle+waiting paths [/] 2024-05-03T18:31:43.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:31:42 smithi150 ceph-mon[32492]: pgmap v138: 129 pgs: 129 active+clean; 1.4 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.1 MiB/s wr, 612 op/s 2024-05-03T18:31:45.110 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 30.0 s, 661.0 tps, lat 149.925 ms stddev 47.168 2024-05-03T18:31:45.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:31:44 smithi033 ceph-mon[28330]: pgmap v139: 129 pgs: 129 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 8.7 MiB/s wr, 923 op/s 2024-05-03T18:31:45.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:31:44 smithi099 ceph-mon[31658]: pgmap v139: 129 pgs: 129 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 8.7 MiB/s wr, 923 op/s 2024-05-03T18:31:45.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:31:44 smithi150 ceph-mon[32492]: pgmap v139: 129 pgs: 129 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 8.7 MiB/s wr, 923 op/s 2024-05-03T18:31:45.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:45.957+0000 7fe4cba00700 1 -- 172.21.15.33:0/1771657482 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4c410ff50 msgr2=0x7fe4c4112340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:31:45.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:45.957+0000 7fe4cba00700 1 --2- 172.21.15.33:0/1771657482 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4c410ff50 0x7fe4c4112340 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc00b0a0 tx=0x7fe4bc00b3b0 comp rx=0 tx=0).stop 2024-05-03T18:31:45.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:45.958+0000 7fe4cba00700 1 -- 172.21.15.33:0/1771657482 shutdown_connections 2024-05-03T18:31:45.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:45.958+0000 7fe4cba00700 1 --2- 172.21.15.33:0/1771657482 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4c410ff50 0x7fe4c4112340 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:45.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:45.958+0000 7fe4cba00700 1 --2- 172.21.15.33:0/1771657482 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4c406b1b0 0x7fe4c406b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:45.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:45.958+0000 7fe4cba00700 1 --2- 172.21.15.33:0/1771657482 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe4c406bad0 0x7fe4c410fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:45.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:45.958+0000 7fe4cba00700 1 -- 172.21.15.33:0/1771657482 >> 172.21.15.33:0/1771657482 conn(0x7fe4c4079350 msgr2=0x7fe4c4079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:31:45.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:45.959+0000 7fe4cba00700 1 -- 172.21.15.33:0/1771657482 shutdown_connections 2024-05-03T18:31:45.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:45.959+0000 7fe4cba00700 1 -- 172.21.15.33:0/1771657482 wait complete. 2024-05-03T18:31:45.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:45.961+0000 7fe4cba00700 1 Processor -- start 2024-05-03T18:31:45.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:45.961+0000 7fe4cba00700 1 -- start start 2024-05-03T18:31:45.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:45.962+0000 7fe4cba00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe4c406b1b0 0x7fe4c419bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:45.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:45.962+0000 7fe4cba00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4c406bad0 0x7fe4c419c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:45.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:45.962+0000 7fe4cba00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4c410ff50 0x7fe4c41a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:45.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:45.962+0000 7fe4cba00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c4115770 con 0x7fe4c406bad0 2024-05-03T18:31:45.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:45.962+0000 7fe4cba00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c41155f0 con 0x7fe4c410ff50 2024-05-03T18:31:45.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:45.962+0000 7fe4cba00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c41158f0 con 0x7fe4c406b1b0 2024-05-03T18:31:45.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:45.962+0000 7fe4c9000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe4c406b1b0 0x7fe4c419bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:45.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:45.962+0000 7fe4c9000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe4c406b1b0 0x7fe4c419bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:32896/0 (socket says 172.21.15.33:32896) 2024-05-03T18:31:45.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:45.962+0000 7fe4c9000700 1 -- 172.21.15.33:0/465175078 learned_addr learned my addr 172.21.15.33:0/465175078 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:31:45.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:45.963+0000 7fe4c9a00700 1 --2- 172.21.15.33:0/465175078 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4c410ff50 0x7fe4c41a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:45.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:45.963+0000 7fe4c3e00700 1 --2- 172.21.15.33:0/465175078 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4c406bad0 0x7fe4c419c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:45.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:45.963+0000 7fe4c9000700 1 -- 172.21.15.33:0/465175078 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4c410ff50 msgr2=0x7fe4c41a05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:31:45.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:45.963+0000 7fe4c9000700 1 --2- 172.21.15.33:0/465175078 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4c410ff50 0x7fe4c41a05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:45.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:45.963+0000 7fe4c9000700 1 -- 172.21.15.33:0/465175078 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4c406bad0 msgr2=0x7fe4c419c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:31:45.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:45.963+0000 7fe4c9000700 1 --2- 172.21.15.33:0/465175078 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4c406bad0 0x7fe4c419c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:45.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:45.963+0000 7fe4c9000700 1 -- 172.21.15.33:0/465175078 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4c41a0d20 con 0x7fe4c406b1b0 2024-05-03T18:31:45.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:45.964+0000 7fe4c9000700 1 --2- 172.21.15.33:0/465175078 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe4c406b1b0 0x7fe4c419bd10 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe4b800e9a0 tx=0x7fe4b800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:31:45.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:45.964+0000 7fe4c1600700 1 -- 172.21.15.33:0/465175078 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b800ca10 con 0x7fe4c406b1b0 2024-05-03T18:31:45.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:45.965+0000 7fe4cba00700 1 -- 172.21.15.33:0/465175078 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4c41a0fb0 con 0x7fe4c406b1b0 2024-05-03T18:31:45.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:45.965+0000 7fe4c1600700 1 -- 172.21.15.33:0/465175078 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe4b800cbb0 con 0x7fe4c406b1b0 2024-05-03T18:31:45.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:45.965+0000 7fe4cba00700 1 -- 172.21.15.33:0/465175078 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4c41a8890 con 0x7fe4c406b1b0 2024-05-03T18:31:45.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:45.966+0000 7fe4c1600700 1 -- 172.21.15.33:0/465175078 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b8018680 con 0x7fe4c406b1b0 2024-05-03T18:31:45.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:45.970+0000 7fe4c1600700 1 -- 172.21.15.33:0/465175078 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fe4b8018820 con 0x7fe4c406b1b0 2024-05-03T18:31:45.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:45.973+0000 7fe4c1600700 1 --2- 172.21.15.33:0/465175078 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fe4ac0b2af0 0x7fe4ac0b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:45.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:45.974+0000 7fe4c1600700 1 -- 172.21.15.33:0/465175078 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7fe4b80e3a50 con 0x7fe4c406b1b0 2024-05-03T18:31:45.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:45.974+0000 7fe4c3e00700 1 --2- 172.21.15.33:0/465175078 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fe4ac0b2af0 0x7fe4ac0b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:45.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:45.980+0000 7fe4c3e00700 1 --2- 172.21.15.33:0/465175078 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fe4ac0b2af0 0x7fe4ac0b4fb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe4c4101670 tx=0x7fe4b0009260 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:31:45.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:45.985+0000 7fe4d6318b80 1 Processor -- start 2024-05-03T18:31:45.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:45.986+0000 7fe4d6318b80 1 -- start start 2024-05-03T18:31:45.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:45.986+0000 7fe4d6318b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ce49534130 0x55ce49534510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:45.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:45.986+0000 7fe4d6318b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ce49534a50 0x55ce4952e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:45.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:45.986+0000 7fe4d6318b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce4952ede0 0x55ce4952f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:45.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:45.986+0000 7fe4d6318b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ce49478a20 con 0x55ce4952ede0 2024-05-03T18:31:45.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:45.986+0000 7fe4d6318b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55ce49416760 con 0x55ce49534130 2024-05-03T18:31:45.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:45.986+0000 7fe4d6318b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55ce494749e0 con 0x55ce49534a50 2024-05-03T18:31:45.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:45.987+0000 7fe4c3e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ce49534a50 0x55ce4952e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:45.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:45.987+0000 7fe4c3e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ce49534a50 0x55ce4952e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:32904/0 (socket says 172.21.15.33:32904) 2024-05-03T18:31:45.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:45.987+0000 7fe4c3e00700 1 -- 172.21.15.33:0/91192894 learned_addr learned my addr 172.21.15.33:0/91192894 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:31:45.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:45.987+0000 7fe4c9a00700 1 --2- 172.21.15.33:0/91192894 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce4952ede0 0x55ce4952f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:45.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:45.988+0000 7fe4c9000700 1 --2- 172.21.15.33:0/91192894 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ce49534130 0x55ce49534510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:45.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:45.988+0000 7fe4c9a00700 1 -- 172.21.15.33:0/91192894 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ce49534a50 msgr2=0x55ce4952e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:31:45.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:45.988+0000 7fe4c9a00700 1 --2- 172.21.15.33:0/91192894 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ce49534a50 0x55ce4952e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:45.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:45.988+0000 7fe4c9a00700 1 -- 172.21.15.33:0/91192894 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ce49534130 msgr2=0x55ce49534510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:31:45.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:45.988+0000 7fe4c9a00700 1 --2- 172.21.15.33:0/91192894 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ce49534130 0x55ce49534510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:45.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:45.988+0000 7fe4c9a00700 1 -- 172.21.15.33:0/91192894 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce493c58d0 con 0x55ce4952ede0 2024-05-03T18:31:45.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:45.989+0000 7fe4c9a00700 1 --2- 172.21.15.33:0/91192894 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce4952ede0 0x55ce4952f240 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc0062a0 tx=0x7fe4bc0092e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0ca9bce884f1bd9 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:31:45.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:45.990+0000 7fe4b6000700 1 -- 172.21.15.33:0/91192894 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4bc020070 con 0x55ce4952ede0 2024-05-03T18:31:45.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:45.991+0000 7fe4b6000700 1 -- 172.21.15.33:0/91192894 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe4bc00bcb0 con 0x55ce4952ede0 2024-05-03T18:31:45.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:45.991+0000 7fe4b6000700 1 -- 172.21.15.33:0/91192894 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4bc007ab0 con 0x55ce4952ede0 2024-05-03T18:31:45.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:45.992+0000 7fe4d6318b80 1 -- 172.21.15.33:0/91192894 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce4952ede0 msgr2=0x55ce4952f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:31:45.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:45.992+0000 7fe4d6318b80 1 --2- 172.21.15.33:0/91192894 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce4952ede0 0x55ce4952f240 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc0062a0 tx=0x7fe4bc0092e0 comp rx=0 tx=0).stop 2024-05-03T18:31:45.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:45.994+0000 7fe4d6318b80 1 -- 172.21.15.33:0/91192894 shutdown_connections 2024-05-03T18:31:45.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:45.994+0000 7fe4d6318b80 1 --2- 172.21.15.33:0/91192894 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce4952ede0 0x55ce4952f240 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:45.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:45.994+0000 7fe4d6318b80 1 --2- 172.21.15.33:0/91192894 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ce49534130 0x55ce49534510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:45.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:45.994+0000 7fe4d6318b80 1 --2- 172.21.15.33:0/91192894 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ce49534a50 0x55ce4952e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:45.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:45.994+0000 7fe4d6318b80 1 -- 172.21.15.33:0/91192894 >> 172.21.15.33:0/91192894 conn(0x55ce49454b60 msgr2=0x55ce495207f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:31:45.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:45.994+0000 7fe4d6318b80 1 -- 172.21.15.33:0/91192894 shutdown_connections 2024-05-03T18:31:45.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:45.994+0000 7fe4d6318b80 1 -- 172.21.15.33:0/91192894 wait complete. 2024-05-03T18:31:45.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:45.994+0000 7fe4d6318b80 1 -- start start 2024-05-03T18:31:45.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:45.995+0000 7fe4d6318b80 1 Processor -- start 2024-05-03T18:31:45.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:45.995+0000 7fe4d6318b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ce4952ede0 0x55ce49538010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:45.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:45.995+0000 7fe4d6318b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ce49534130 0x55ce49538550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:45.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:45.995+0000 7fe4d6318b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce49534a50 0x55ce49538a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:45.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:45.995+0000 7fe4d6318b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ce495262d0 con 0x55ce49534a50 2024-05-03T18:31:45.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:45.995+0000 7fe4d6318b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55ce493deb20 con 0x55ce4952ede0 2024-05-03T18:31:45.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:45.995+0000 7fe4d6318b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55ce49526450 con 0x55ce49534130 2024-05-03T18:31:45.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:45.996+0000 7fe4d6318b80 10 client.0 _ll_register_callbacks cb 0x55ce4941fc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:31:45.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:45.996+0000 7fe4c9a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce49534a50 0x55ce49538a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:45.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:45.996+0000 7fe4c9a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce49534a50 0x55ce49538a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55324/0 (socket says 172.21.15.33:55324) 2024-05-03T18:31:45.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:45.996+0000 7fe4c9a00700 1 -- 172.21.15.33:0/493702902 learned_addr learned my addr 172.21.15.33:0/493702902 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:31:45.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:45.997+0000 7fe4c3e00700 1 --2- 172.21.15.33:0/493702902 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ce49534130 0x55ce49538550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:45.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:45.997+0000 7fe4c9000700 1 --2- 172.21.15.33:0/493702902 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ce4952ede0 0x55ce49538010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:45.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:45.997+0000 7fe4c9a00700 1 -- 172.21.15.33:0/493702902 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ce49534130 msgr2=0x55ce49538550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:31:45.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:45.997+0000 7fe4c9a00700 1 --2- 172.21.15.33:0/493702902 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ce49534130 0x55ce49538550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:45.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:45.997+0000 7fe4c9a00700 1 -- 172.21.15.33:0/493702902 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ce4952ede0 msgr2=0x55ce49538010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:31:45.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:45.997+0000 7fe4c9a00700 1 --2- 172.21.15.33:0/493702902 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ce4952ede0 0x55ce49538010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:45.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:45.997+0000 7fe4c9a00700 1 -- 172.21.15.33:0/493702902 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce4943e420 con 0x55ce49534a50 2024-05-03T18:31:45.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:45.998+0000 7fe4c9a00700 1 --2- 172.21.15.33:0/493702902 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce49534a50 0x55ce49538a90 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc007900 tx=0x7fe4bc007930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5da858ae380c549c server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:31:46.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:45.998+0000 7fe4aaa00700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:31:46.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:45.998+0000 7fe4aaa00700 1 -- 172.21.15.33:0/493702902 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4bc020040 con 0x55ce49534a50 2024-05-03T18:31:46.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:45.998+0000 7fe4aaa00700 1 -- 172.21.15.33:0/493702902 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7fe4bc00bab0 con 0x55ce49534a50 2024-05-03T18:31:46.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:45.999+0000 7fe4d6318b80 1 -- 172.21.15.33:0/493702902 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4bc00bcb0 con 0x55ce49534a50 2024-05-03T18:31:46.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:45.999+0000 7fe4aaa00700 1 -- 172.21.15.33:0/493702902 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fe4bc00be60 con 0x55ce49534a50 2024-05-03T18:31:46.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:45.999+0000 7fe4d6318b80 10 client.15936 fetch_fsmap learned FSMap version 21 2024-05-03T18:31:46.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:45.999+0000 7fe4d6318b80 1 -- 172.21.15.33:0/493702902 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55ce4954e6b0 con 0x55ce49534a50 2024-05-03T18:31:46.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.000+0000 7fe4aaa00700 1 -- 172.21.15.33:0/493702902 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe4bc00bcb0 con 0x55ce49534a50 2024-05-03T18:31:46.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.000+0000 7fe4aaa00700 1 -- 172.21.15.33:0/493702902 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fe4bc011070 con 0x55ce49534a50 2024-05-03T18:31:46.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.000+0000 7fe4d6318b80 10 client.15936 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:31:46.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.000+0000 7fe4d6318b80 10 client.15936 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:31:46.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.000+0000 7fe4d6318b80 20 client.15936 populate_metadata read hostname 'smithi033' 2024-05-03T18:31:46.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.000+0000 7fe4d6318b80 1 --2- 172.21.15.33:0/493702902 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55ce49550b80 0x55ce49552f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:46.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.000+0000 7fe4d6318b80 4 client.15936 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:31:46.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.000+0000 7fe4d6318b80 1 -- 172.21.15.33:0/493702902 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ce494f31d0 con 0x55ce49550b80 2024-05-03T18:31:46.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.001+0000 7fe4c9000700 1 --2- 172.21.15.33:0/493702902 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55ce49550b80 0x55ce49552f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:46.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.002+0000 7fe4c9000700 1 --2- 172.21.15.33:0/493702902 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55ce49550b80 0x55ce49552f70 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6848fb3d7daacb7e server_cookie=1bc245e9efef51c4 in_seq=0 out_seq=0 2024-05-03T18:31:46.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.002+0000 7fe4aaa00700 10 client.15936 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:31:46.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.004+0000 7fe4aaa00700 1 -- 172.21.15.33:0/493702902 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ce494f31d0 con 0x55ce49550b80 2024-05-03T18:31:46.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.004+0000 7fe4aaa00700 10 client.15936 handle_command_reply: tid=0 2024-05-03T18:31:46.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.004+0000 7fe4d6318b80 1 client.15936 shutdown 2024-05-03T18:31:46.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.004+0000 7fe4d6318b80 1 -- 172.21.15.33:0/493702902 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce49534a50 msgr2=0x55ce49538a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:31:46.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.004+0000 7fe4d6318b80 1 --2- 172.21.15.33:0/493702902 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce49534a50 0x55ce49538a90 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc007900 tx=0x7fe4bc007930 comp rx=0 tx=0).stop 2024-05-03T18:31:46.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.004+0000 7fe4d6318b80 1 -- 172.21.15.33:0/493702902 shutdown_connections 2024-05-03T18:31:46.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.004+0000 7fe4d6318b80 1 --2- 172.21.15.33:0/493702902 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55ce49550b80 0x55ce49552f70 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:46.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.004+0000 7fe4d6318b80 1 --2- 172.21.15.33:0/493702902 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce49534a50 0x55ce49538a90 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:46.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.004+0000 7fe4d6318b80 1 --2- 172.21.15.33:0/493702902 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ce4952ede0 0x55ce49538010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:46.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.004+0000 7fe4d6318b80 1 --2- 172.21.15.33:0/493702902 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ce49534130 0x55ce49538550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:46.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.004+0000 7fe4d6318b80 1 -- 172.21.15.33:0/493702902 >> 172.21.15.33:0/493702902 conn(0x55ce49454b60 msgr2=0x55ce49533540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:31:46.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.004+0000 7fe4aaa00700 0 client.15936 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:31:46.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.005+0000 7fe4d6318b80 1 -- 172.21.15.33:0/493702902 shutdown_connections 2024-05-03T18:31:46.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.005+0000 7fe4d6318b80 1 -- 172.21.15.33:0/493702902 wait complete. 2024-05-03T18:31:46.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.005+0000 7fe4d6318b80 20 client.15936 trim_cache size 0 max 16384 2024-05-03T18:31:46.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.043+0000 7fe4d6318b80 1 Processor -- start 2024-05-03T18:31:46.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.043+0000 7fe4d6318b80 1 -- start start 2024-05-03T18:31:46.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.043+0000 7fe4d6318b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ce4952ede0 0x55ce495256b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:46.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.043+0000 7fe4d6318b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce49534130 0x55ce49533050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:46.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.043+0000 7fe4d6318b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ce49534a50 0x55ce49533590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:46.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.043+0000 7fe4d6318b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe4a400cc60 con 0x55ce49534130 2024-05-03T18:31:46.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.043+0000 7fe4d6318b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe4a400f4e0 con 0x55ce49534a50 2024-05-03T18:31:46.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.043+0000 7fe4d6318b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe4a400d2d0 con 0x55ce4952ede0 2024-05-03T18:31:46.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.044+0000 7fe4c9000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ce4952ede0 0x55ce495256b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:46.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.044+0000 7fe4c3e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce49534130 0x55ce49533050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:46.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.044+0000 7fe4c9000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ce4952ede0 0x55ce495256b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:32924/0 (socket says 172.21.15.33:32924) 2024-05-03T18:31:46.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.044+0000 7fe4c9000700 1 -- 172.21.15.33:0/2196176489 learned_addr learned my addr 172.21.15.33:0/2196176489 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:31:46.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.045+0000 7fe4c9a00700 1 --2- 172.21.15.33:0/2196176489 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ce49534a50 0x55ce49533590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:46.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.045+0000 7fe4c9000700 1 -- 172.21.15.33:0/2196176489 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ce49534a50 msgr2=0x55ce49533590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:31:46.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.045+0000 7fe4c9000700 1 --2- 172.21.15.33:0/2196176489 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ce49534a50 0x55ce49533590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:46.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.045+0000 7fe4c9000700 1 -- 172.21.15.33:0/2196176489 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce49534130 msgr2=0x55ce49533050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:31:46.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.045+0000 7fe4c9000700 1 --2- 172.21.15.33:0/2196176489 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce49534130 0x55ce49533050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:46.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.045+0000 7fe4c9000700 1 -- 172.21.15.33:0/2196176489 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce49521550 con 0x55ce4952ede0 2024-05-03T18:31:46.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.045+0000 7fe4c3e00700 1 --2- 172.21.15.33:0/2196176489 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce49534130 0x55ce49533050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:31:46.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.046+0000 7fe4c9000700 1 --2- 172.21.15.33:0/2196176489 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ce4952ede0 0x55ce495256b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe4b8010d00 tx=0x7fe4b800c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d94e569a66f9878d server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:31:46.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.047+0000 7fe4b5600700 1 -- 172.21.15.33:0/2196176489 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b8090e60 con 0x55ce4952ede0 2024-05-03T18:31:46.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.047+0000 7fe4b5600700 1 -- 172.21.15.33:0/2196176489 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe4b80aa940 con 0x55ce4952ede0 2024-05-03T18:31:46.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.047+0000 7fe4b5600700 1 -- 172.21.15.33:0/2196176489 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b8090e60 con 0x55ce4952ede0 2024-05-03T18:31:46.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.048+0000 7fe4d6318b80 1 -- 172.21.15.33:0/2196176489 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ce4952ede0 msgr2=0x55ce495256b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:31:46.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.048+0000 7fe4d6318b80 1 --2- 172.21.15.33:0/2196176489 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ce4952ede0 0x55ce495256b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe4b8010d00 tx=0x7fe4b800c480 comp rx=0 tx=0).stop 2024-05-03T18:31:46.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.049+0000 7fe4d6318b80 1 -- 172.21.15.33:0/2196176489 shutdown_connections 2024-05-03T18:31:46.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.049+0000 7fe4d6318b80 1 --2- 172.21.15.33:0/2196176489 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ce49534a50 0x55ce49533590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:46.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.049+0000 7fe4d6318b80 1 --2- 172.21.15.33:0/2196176489 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ce4952ede0 0x55ce495256b0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:46.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.049+0000 7fe4d6318b80 1 --2- 172.21.15.33:0/2196176489 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce49534130 0x55ce49533050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:46.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.049+0000 7fe4d6318b80 1 -- 172.21.15.33:0/2196176489 >> 172.21.15.33:0/2196176489 conn(0x55ce49454b60 msgr2=0x55ce49524fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:31:46.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.049+0000 7fe4d6318b80 1 -- 172.21.15.33:0/2196176489 shutdown_connections 2024-05-03T18:31:46.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.049+0000 7fe4d6318b80 1 -- 172.21.15.33:0/2196176489 wait complete. 2024-05-03T18:31:46.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.050+0000 7fe4d6318b80 1 -- start start 2024-05-03T18:31:46.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.050+0000 7fe4d6318b80 1 Processor -- start 2024-05-03T18:31:46.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.051+0000 7fe4d6318b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce4952ede0 0x55ce49528ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:46.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.052+0000 7fe4d6318b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ce49534130 0x55ce49529430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:46.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.052+0000 7fe4c9000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce4952ede0 0x55ce49528ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:46.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.053+0000 7fe4c9000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce4952ede0 0x55ce49528ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55334/0 (socket says 172.21.15.33:55334) 2024-05-03T18:31:46.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.053+0000 7fe4d6318b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ce49534a50 0x55ce495cb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:46.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.053+0000 7fe4d6318b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe4a400d9a0 con 0x55ce4952ede0 2024-05-03T18:31:46.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.053+0000 7fe4d6318b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe4a400ee10 con 0x55ce49534130 2024-05-03T18:31:46.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.053+0000 7fe4d6318b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe4a400e070 con 0x55ce49534a50 2024-05-03T18:31:46.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.053+0000 7fe4d6318b80 10 client.0 _ll_register_callbacks cb 0x55ce4941fc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:31:46.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.054+0000 7fe4c3e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ce49534130 0x55ce49529430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:46.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.054+0000 7fe4c3e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ce49534130 0x55ce49529430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:36874/0 (socket says 172.21.15.33:36874) 2024-05-03T18:31:46.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.054+0000 7fe4c3e00700 1 -- 172.21.15.33:0/2825627921 learned_addr learned my addr 172.21.15.33:0/2825627921 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:31:46.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.054+0000 7fe4c9a00700 1 --2- 172.21.15.33:0/2825627921 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ce49534a50 0x55ce495cb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:46.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.056+0000 7fe4c9000700 1 -- 172.21.15.33:0/2825627921 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ce49534a50 msgr2=0x55ce495cb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:31:46.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.056+0000 7fe4c9000700 1 --2- 172.21.15.33:0/2825627921 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ce49534a50 0x55ce495cb0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:46.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.056+0000 7fe4c9000700 1 -- 172.21.15.33:0/2825627921 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ce49534130 msgr2=0x55ce49529430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:31:46.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.056+0000 7fe4c9000700 1 --2- 172.21.15.33:0/2825627921 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ce49534130 0x55ce49529430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:46.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.056+0000 7fe4c9000700 1 -- 172.21.15.33:0/2825627921 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce495cb5e0 con 0x55ce4952ede0 2024-05-03T18:31:46.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.056+0000 7fe4c9a00700 1 --2- 172.21.15.33:0/2825627921 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ce49534a50 0x55ce495cb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:31:46.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.056+0000 7fe4c3e00700 1 --2- 172.21.15.33:0/2825627921 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ce49534130 0x55ce49529430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:31:46.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.057+0000 7fe4c9000700 1 --2- 172.21.15.33:0/2825627921 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce4952ede0 0x55ce49528ef0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fe4b80b3960 tx=0x7fe4b80e12e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=244945a6fbd33fcb server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:31:46.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.057+0000 7fe4aaa00700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:31:46.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.057+0000 7fe4aaa00700 1 -- 172.21.15.33:0/2825627921 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b8090e60 con 0x55ce4952ede0 2024-05-03T18:31:46.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.058+0000 7fe4d6318b80 1 -- 172.21.15.33:0/2825627921 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4b80aa940 con 0x55ce4952ede0 2024-05-03T18:31:46.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.058+0000 7fe4aaa00700 1 -- 172.21.15.33:0/2825627921 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7fe4b80aadb0 con 0x55ce4952ede0 2024-05-03T18:31:46.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.058+0000 7fe4aaa00700 1 -- 172.21.15.33:0/2825627921 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fe4b80b37a0 con 0x55ce4952ede0 2024-05-03T18:31:46.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.059+0000 7fe4d6318b80 10 client.15942 fetch_fsmap learned FSMap version 21 2024-05-03T18:31:46.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.059+0000 7fe4d6318b80 1 -- 172.21.15.33:0/2825627921 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55ce495cc870 con 0x55ce4952ede0 2024-05-03T18:31:46.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.060+0000 7fe4aaa00700 1 -- 172.21.15.33:0/2825627921 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe4b80aa940 con 0x55ce4952ede0 2024-05-03T18:31:46.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.060+0000 7fe4aaa00700 1 -- 172.21.15.33:0/2825627921 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fe4b8018c80 con 0x55ce4952ede0 2024-05-03T18:31:46.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.060+0000 7fe4d6318b80 10 client.15942 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:31:46.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.060+0000 7fe4d6318b80 10 client.15942 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:31:46.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.060+0000 7fe4d6318b80 20 client.15942 populate_metadata read hostname 'smithi033' 2024-05-03T18:31:46.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.060+0000 7fe4d6318b80 1 --2- 172.21.15.33:0/2825627921 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55ce495cce70 0x55ce495cf260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:31:46.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.060+0000 7fe4d6318b80 4 client.15942 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-03T18:31:46.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.060+0000 7fe4d6318b80 1 -- 172.21.15.33:0/2825627921 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ce49486400 con 0x55ce495cce70 2024-05-03T18:31:46.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.061+0000 7fe4c3e00700 1 --2- 172.21.15.33:0/2825627921 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55ce495cce70 0x55ce495cf260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:31:46.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.062+0000 7fe4c3e00700 1 --2- 172.21.15.33:0/2825627921 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55ce495cce70 0x55ce495cf260 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c64087985415074 server_cookie=e41db71d255153b in_seq=0 out_seq=0 2024-05-03T18:31:46.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.062+0000 7fe4aaa00700 10 client.15942 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:31:46.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.063+0000 7fe4aaa00700 1 -- 172.21.15.33:0/2825627921 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55ce49486400 con 0x55ce495cce70 2024-05-03T18:31:46.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.063+0000 7fe4aaa00700 10 client.15942 handle_command_reply: tid=0 2024-05-03T18:31:46.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.063+0000 7fe4d6318b80 1 client.15942 shutdown 2024-05-03T18:31:46.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.064+0000 7fe4d6318b80 1 -- 172.21.15.33:0/2825627921 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce4952ede0 msgr2=0x55ce49528ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:31:46.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.064+0000 7fe4d6318b80 1 --2- 172.21.15.33:0/2825627921 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce4952ede0 0x55ce49528ef0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fe4b80b3960 tx=0x7fe4b80e12e0 comp rx=0 tx=0).stop 2024-05-03T18:31:46.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.064+0000 7fe4d6318b80 1 -- 172.21.15.33:0/2825627921 shutdown_connections 2024-05-03T18:31:46.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.064+0000 7fe4d6318b80 1 --2- 172.21.15.33:0/2825627921 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55ce495cce70 0x55ce495cf260 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:46.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.064+0000 7fe4d6318b80 1 --2- 172.21.15.33:0/2825627921 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ce49534a50 0x55ce495cb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:46.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.064+0000 7fe4d6318b80 1 --2- 172.21.15.33:0/2825627921 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce4952ede0 0x55ce49528ef0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:46.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.064+0000 7fe4d6318b80 1 --2- 172.21.15.33:0/2825627921 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ce49534130 0x55ce49529430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:46.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.064+0000 7fe4d6318b80 1 -- 172.21.15.33:0/2825627921 >> 172.21.15.33:0/2825627921 conn(0x55ce49454b60 msgr2=0x55ce49525830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:31:46.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.065+0000 7fe4aaa00700 0 client.15942 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:31:46.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.065+0000 7fe4d6318b80 1 -- 172.21.15.33:0/2825627921 shutdown_connections 2024-05-03T18:31:46.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.065+0000 7fe4d6318b80 1 -- 172.21.15.33:0/2825627921 wait complete. 2024-05-03T18:31:46.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.066+0000 7fe4d6318b80 20 client.15942 trim_cache size 0 max 16384 2024-05-03T18:31:46.067 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-03T18:31:46.067 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-03T18:31:46.068 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-05-03T18:31:46.068 INFO:teuthology.orchestra.run.smithi033.stdout: "fc07b1f0-3656-41e0-9c58-d4c804568704": { 2024-05-03T18:31:46.068 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-05-03T18:31:46.068 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "fc07b1f0-3656-41e0-9c58-d4c804568704", 2024-05-03T18:31:46.068 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-05-03T18:31:46.068 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:31:46.068 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:31:46.068 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-03T18:31:46.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.067+0000 7fe4cba00700 1 -- 172.21.15.33:0/465175078 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fe4ac0b2af0 msgr2=0x7fe4ac0b4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:31:46.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.067+0000 7fe4cba00700 1 --2- 172.21.15.33:0/465175078 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fe4ac0b2af0 0x7fe4ac0b4fb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe4c4101670 tx=0x7fe4b0009260 comp rx=0 tx=0).stop 2024-05-03T18:31:46.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.067+0000 7fe4cba00700 1 -- 172.21.15.33:0/465175078 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe4c406b1b0 msgr2=0x7fe4c419bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:31:46.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.067+0000 7fe4cba00700 1 --2- 172.21.15.33:0/465175078 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe4c406b1b0 0x7fe4c419bd10 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe4b800e9a0 tx=0x7fe4b800ecb0 comp rx=0 tx=0).stop 2024-05-03T18:31:46.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.068+0000 7fe4cba00700 1 -- 172.21.15.33:0/465175078 shutdown_connections 2024-05-03T18:31:46.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.068+0000 7fe4cba00700 1 --2- 172.21.15.33:0/465175078 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fe4ac0b2af0 0x7fe4ac0b4fb0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:46.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.068+0000 7fe4cba00700 1 --2- 172.21.15.33:0/465175078 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4c410ff50 0x7fe4c41a05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:46.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.068+0000 7fe4cba00700 1 --2- 172.21.15.33:0/465175078 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe4c406b1b0 0x7fe4c419bd10 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:46.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.068+0000 7fe4cba00700 1 --2- 172.21.15.33:0/465175078 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4c406bad0 0x7fe4c419c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:31:46.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.068+0000 7fe4cba00700 1 -- 172.21.15.33:0/465175078 >> 172.21.15.33:0/465175078 conn(0x7fe4c4079350 msgr2=0x7fe4c41114e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:31:46.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.068+0000 7fe4cba00700 1 -- 172.21.15.33:0/465175078 shutdown_connections 2024-05-03T18:31:46.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:31:46.069+0000 7fe4cba00700 1 -- 172.21.15.33:0/465175078 wait complete. 2024-05-03T18:31:47.048 INFO:tasks.cephfs.filesystem:scrub status for tag:fc07b1f0-3656-41e0-9c58-d4c804568704 - {'path': '/', 'tag': 'fc07b1f0-3656-41e0-9c58-d4c804568704', 'options': 'recursive,force'} 2024-05-03T18:31:47.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:31:46 smithi033 ceph-mon[28330]: pgmap v140: 129 pgs: 129 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.7 MiB/s wr, 679 op/s 2024-05-03T18:31:47.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:31:46 smithi033 ceph-mon[28330]: scrub summary: idle 2024-05-03T18:31:47.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:31:46 smithi099 ceph-mon[31658]: pgmap v140: 129 pgs: 129 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.7 MiB/s wr, 679 op/s 2024-05-03T18:31:47.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:31:46 smithi099 ceph-mon[31658]: scrub summary: idle 2024-05-03T18:31:47.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:31:46 smithi150 ceph-mon[32492]: pgmap v140: 129 pgs: 129 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.7 MiB/s wr, 679 op/s 2024-05-03T18:31:47.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:31:46 smithi150 ceph-mon[32492]: scrub summary: idle 2024-05-03T18:31:48.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:31:48 smithi099 ceph-mon[31658]: pgmap v141: 129 pgs: 129 active+clean; 1.5 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 8.4 MiB/s wr, 986 op/s 2024-05-03T18:31:48.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:31:48 smithi150 ceph-mon[32492]: pgmap v141: 129 pgs: 129 active+clean; 1.5 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 8.4 MiB/s wr, 986 op/s 2024-05-03T18:31:48.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:31:48 smithi033 ceph-mon[28330]: pgmap v141: 129 pgs: 129 active+clean; 1.5 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 8.4 MiB/s wr, 986 op/s 2024-05-03T18:31:50.110 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 35.0 s, 625.7 tps, lat 160.522 ms stddev 40.803 2024-05-03T18:31:51.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:31:50 smithi033 ceph-mon[28330]: pgmap v142: 129 pgs: 129 active+clean; 1.5 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 5.3 MiB/s wr, 618 op/s 2024-05-03T18:31:51.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:31:50 smithi099 ceph-mon[31658]: pgmap v142: 129 pgs: 129 active+clean; 1.5 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 5.3 MiB/s wr, 618 op/s 2024-05-03T18:31:51.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:31:50 smithi150 ceph-mon[32492]: pgmap v142: 129 pgs: 129 active+clean; 1.5 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 5.3 MiB/s wr, 618 op/s 2024-05-03T18:31:53.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:31:52 smithi033 ceph-mon[28330]: pgmap v143: 129 pgs: 129 active+clean; 1.5 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 5.3 MiB/s wr, 618 op/s 2024-05-03T18:31:53.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:31:52 smithi099 ceph-mon[31658]: pgmap v143: 129 pgs: 129 active+clean; 1.5 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 5.3 MiB/s wr, 618 op/s 2024-05-03T18:31:53.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:31:52 smithi150 ceph-mon[32492]: pgmap v143: 129 pgs: 129 active+clean; 1.5 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 5.3 MiB/s wr, 618 op/s 2024-05-03T18:31:55.110 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 40.0 s, 616.3 tps, lat 162.415 ms stddev 38.456 2024-05-03T18:31:55.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:31:54 smithi033 ceph-mon[28330]: pgmap v144: 129 pgs: 129 active+clean; 1.5 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 8.1 MiB/s wr, 952 op/s 2024-05-03T18:31:55.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:31:54 smithi099 ceph-mon[31658]: pgmap v144: 129 pgs: 129 active+clean; 1.5 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 8.1 MiB/s wr, 952 op/s 2024-05-03T18:31:55.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:31:54 smithi150 ceph-mon[32492]: pgmap v144: 129 pgs: 129 active+clean; 1.5 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 8.1 MiB/s wr, 952 op/s 2024-05-03T18:31:56.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:31:56 smithi099 ceph-mon[31658]: pgmap v145: 129 pgs: 129 active+clean; 1.5 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 5.5 MiB/s wr, 641 op/s 2024-05-03T18:31:56.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:31:56 smithi150 ceph-mon[32492]: pgmap v145: 129 pgs: 129 active+clean; 1.5 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 5.5 MiB/s wr, 641 op/s 2024-05-03T18:31:56.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:31:56 smithi033 ceph-mon[28330]: pgmap v145: 129 pgs: 129 active+clean; 1.5 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 5.5 MiB/s wr, 641 op/s 2024-05-03T18:31:59.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:31:58 smithi033 ceph-mon[28330]: pgmap v146: 129 pgs: 129 active+clean; 1.5 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 7.8 MiB/s wr, 908 op/s 2024-05-03T18:31:59.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:31:58 smithi099 ceph-mon[31658]: pgmap v146: 129 pgs: 129 active+clean; 1.5 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 7.8 MiB/s wr, 908 op/s 2024-05-03T18:31:59.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:31:58 smithi150 ceph-mon[32492]: pgmap v146: 129 pgs: 129 active+clean; 1.5 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 7.8 MiB/s wr, 908 op/s 2024-05-03T18:32:00.110 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 45.0 s, 576.9 tps, lat 172.909 ms stddev 72.281 2024-05-03T18:32:01.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:32:00 smithi033 ceph-mon[28330]: pgmap v147: 129 pgs: 129 active+clean; 1.5 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.1 MiB/s wr, 601 op/s 2024-05-03T18:32:01.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:32:00 smithi099 ceph-mon[31658]: pgmap v147: 129 pgs: 129 active+clean; 1.5 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.1 MiB/s wr, 601 op/s 2024-05-03T18:32:01.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:32:00 smithi150 ceph-mon[32492]: pgmap v147: 129 pgs: 129 active+clean; 1.5 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.1 MiB/s wr, 601 op/s 2024-05-03T18:32:03.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:32:02 smithi033 ceph-mon[28330]: pgmap v148: 129 pgs: 129 active+clean; 1.5 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.1 MiB/s wr, 601 op/s 2024-05-03T18:32:03.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:32:02 smithi099 ceph-mon[31658]: pgmap v148: 129 pgs: 129 active+clean; 1.5 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.1 MiB/s wr, 601 op/s 2024-05-03T18:32:03.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:32:02 smithi150 ceph-mon[32492]: pgmap v148: 129 pgs: 129 active+clean; 1.5 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.1 MiB/s wr, 601 op/s 2024-05-03T18:32:04.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:32:04 smithi099 ceph-mon[31658]: pgmap v149: 129 pgs: 129 active+clean; 1.5 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 7.7 MiB/s wr, 918 op/s 2024-05-03T18:32:04.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:32:04 smithi150 ceph-mon[32492]: pgmap v149: 129 pgs: 129 active+clean; 1.5 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 7.7 MiB/s wr, 918 op/s 2024-05-03T18:32:04.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:32:04 smithi033 ceph-mon[28330]: pgmap v149: 129 pgs: 129 active+clean; 1.5 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 7.7 MiB/s wr, 918 op/s 2024-05-03T18:32:05.110 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 50.0 s, 548.7 tps, lat 173.229 ms stddev 106.093 2024-05-03T18:32:07.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:32:06 smithi033 ceph-mon[28330]: pgmap v150: 129 pgs: 129 active+clean; 1.5 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.9 MiB/s wr, 584 op/s 2024-05-03T18:32:07.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:32:06 smithi099 ceph-mon[31658]: pgmap v150: 129 pgs: 129 active+clean; 1.5 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.9 MiB/s wr, 584 op/s 2024-05-03T18:32:07.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:32:06 smithi150 ceph-mon[32492]: pgmap v150: 129 pgs: 129 active+clean; 1.5 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.9 MiB/s wr, 584 op/s 2024-05-03T18:32:09.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:32:08 smithi033 ceph-mon[28330]: pgmap v151: 129 pgs: 129 active+clean; 1.6 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 8.9 MiB/s wr, 875 op/s 2024-05-03T18:32:09.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:32:08 smithi099 ceph-mon[31658]: pgmap v151: 129 pgs: 129 active+clean; 1.6 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 8.9 MiB/s wr, 875 op/s 2024-05-03T18:32:09.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:32:08 smithi150 ceph-mon[32492]: pgmap v151: 129 pgs: 129 active+clean; 1.6 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 8.9 MiB/s wr, 875 op/s 2024-05-03T18:32:10.110 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 55.0 s, 624.4 tps, lat 169.016 ms stddev 205.690 2024-05-03T18:32:11.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:32:10 smithi033 ceph-mon[28330]: pgmap v152: 129 pgs: 129 active+clean; 1.6 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 6.6 MiB/s wr, 608 op/s 2024-05-03T18:32:11.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:32:10 smithi099 ceph-mon[31658]: pgmap v152: 129 pgs: 129 active+clean; 1.6 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 6.6 MiB/s wr, 608 op/s 2024-05-03T18:32:11.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:32:10 smithi150 ceph-mon[32492]: pgmap v152: 129 pgs: 129 active+clean; 1.6 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 6.6 MiB/s wr, 608 op/s 2024-05-03T18:32:13.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:32:12 smithi033 ceph-mon[28330]: pgmap v153: 129 pgs: 129 active+clean; 1.6 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 6.6 MiB/s wr, 608 op/s 2024-05-03T18:32:13.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:32:12 smithi099 ceph-mon[31658]: pgmap v153: 129 pgs: 129 active+clean; 1.6 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 6.6 MiB/s wr, 608 op/s 2024-05-03T18:32:13.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:32:12 smithi150 ceph-mon[32492]: pgmap v153: 129 pgs: 129 active+clean; 1.6 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 6.6 MiB/s wr, 608 op/s 2024-05-03T18:32:14.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:32:14 smithi099 ceph-mon[31658]: pgmap v154: 129 pgs: 129 active+clean; 1.6 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 9.5 MiB/s wr, 943 op/s 2024-05-03T18:32:14.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:32:14 smithi150 ceph-mon[32492]: pgmap v154: 129 pgs: 129 active+clean; 1.6 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 9.5 MiB/s wr, 943 op/s 2024-05-03T18:32:14.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:32:14 smithi033 ceph-mon[28330]: pgmap v154: 129 pgs: 129 active+clean; 1.6 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 9.5 MiB/s wr, 943 op/s 2024-05-03T18:32:15.110 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 60.0 s, 679.0 tps, lat 146.439 ms stddev 42.063 2024-05-03T18:32:17.050 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-03T18:32:17.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:32:16 smithi033 ceph-mon[28330]: pgmap v155: 129 pgs: 129 active+clean; 1.6 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 6.9 MiB/s wr, 625 op/s 2024-05-03T18:32:17.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:32:16 smithi099 ceph-mon[31658]: pgmap v155: 129 pgs: 129 active+clean; 1.6 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 6.9 MiB/s wr, 625 op/s 2024-05-03T18:32:17.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:32:16 smithi150 ceph-mon[32492]: pgmap v155: 129 pgs: 129 active+clean; 1.6 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 6.9 MiB/s wr, 625 op/s 2024-05-03T18:32:17.785 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:32:19.236 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:32:18 smithi033 ceph-mon[28330]: pgmap v156: 129 pgs: 129 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 9.6 MiB/s wr, 944 op/s 2024-05-03T18:32:19.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:32:18 smithi099 ceph-mon[31658]: pgmap v156: 129 pgs: 129 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 9.6 MiB/s wr, 944 op/s 2024-05-03T18:32:19.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:32:18 smithi150 ceph-mon[32492]: pgmap v156: 129 pgs: 129 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 9.6 MiB/s wr, 944 op/s 2024-05-03T18:32:20.110 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 65.0 s, 622.2 tps, lat 160.196 ms stddev 42.034 2024-05-03T18:32:21.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:32:20 smithi033 ceph-mon[28330]: pgmap v157: 129 pgs: 129 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 5.6 MiB/s wr, 653 op/s 2024-05-03T18:32:21.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:32:20 smithi099 ceph-mon[31658]: pgmap v157: 129 pgs: 129 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 5.6 MiB/s wr, 653 op/s 2024-05-03T18:32:21.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:32:20 smithi150 ceph-mon[32492]: pgmap v157: 129 pgs: 129 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 5.6 MiB/s wr, 653 op/s 2024-05-03T18:32:22.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.421+0000 7fdc0fa00700 1 -- 172.21.15.33:0/1761895555 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdc0806bb40 msgr2=0x7fdc08110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:32:22.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.421+0000 7fdc0fa00700 1 --2- 172.21.15.33:0/1761895555 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdc0806bb40 0x7fdc08110330 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fdbf800b0a0 tx=0x7fdbf800b3b0 comp rx=0 tx=0).stop 2024-05-03T18:32:22.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.421+0000 7fdc0fa00700 1 -- 172.21.15.33:0/1761895555 shutdown_connections 2024-05-03T18:32:22.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.421+0000 7fdc0fa00700 1 --2- 172.21.15.33:0/1761895555 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdc0806bb40 0x7fdc08110330 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:22.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.421+0000 7fdc0fa00700 1 --2- 172.21.15.33:0/1761895555 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc08109510 0x7fdc081098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:22.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.421+0000 7fdc0fa00700 1 --2- 172.21.15.33:0/1761895555 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc0806b180 0x7fdc0806b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:22.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.421+0000 7fdc0fa00700 1 -- 172.21.15.33:0/1761895555 >> 172.21.15.33:0/1761895555 conn(0x7fdc08079350 msgr2=0x7fdc08079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:32:22.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.423+0000 7fdc0fa00700 1 -- 172.21.15.33:0/1761895555 shutdown_connections 2024-05-03T18:32:22.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.423+0000 7fdc0fa00700 1 -- 172.21.15.33:0/1761895555 wait complete. 2024-05-03T18:32:22.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.424+0000 7fdc0fa00700 1 Processor -- start 2024-05-03T18:32:22.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.425+0000 7fdc0fa00700 1 -- start start 2024-05-03T18:32:22.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.425+0000 7fdc0fa00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc0806b180 0x7fdc081a18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:32:22.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.425+0000 7fdc0fa00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc0806bb40 0x7fdc081a1e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:32:22.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.425+0000 7fdc0fa00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdc08109510 0x7fdc0819ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:32:22.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.425+0000 7fdc0fa00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fdc08103d70 con 0x7fdc0806bb40 2024-05-03T18:32:22.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.425+0000 7fdc0fa00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fdc08103bf0 con 0x7fdc0806b180 2024-05-03T18:32:22.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.425+0000 7fdc0fa00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fdc08103ef0 con 0x7fdc08109510 2024-05-03T18:32:22.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.426+0000 7fdc0da00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdc08109510 0x7fdc0819ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:32:22.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.426+0000 7fdc0da00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdc08109510 0x7fdc0819ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:39166/0 (socket says 172.21.15.33:39166) 2024-05-03T18:32:22.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.426+0000 7fdc0da00700 1 -- 172.21.15.33:0/377295817 learned_addr learned my addr 172.21.15.33:0/377295817 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:32:22.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.426+0000 7fdc07e00700 1 --2- 172.21.15.33:0/377295817 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc0806bb40 0x7fdc081a1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:32:22.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.426+0000 7fdc0d000700 1 --2- 172.21.15.33:0/377295817 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc0806b180 0x7fdc081a18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:32:22.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.427+0000 7fdc0da00700 1 -- 172.21.15.33:0/377295817 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc0806b180 msgr2=0x7fdc081a18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:32:22.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.427+0000 7fdc0da00700 1 --2- 172.21.15.33:0/377295817 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc0806b180 0x7fdc081a18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:22.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.427+0000 7fdc0da00700 1 -- 172.21.15.33:0/377295817 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc0806bb40 msgr2=0x7fdc081a1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:32:22.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.427+0000 7fdc0da00700 1 --2- 172.21.15.33:0/377295817 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc0806bb40 0x7fdc081a1e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:22.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.427+0000 7fdc0da00700 1 -- 172.21.15.33:0/377295817 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc0819c1e0 con 0x7fdc08109510 2024-05-03T18:32:22.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.428+0000 7fdc0da00700 1 --2- 172.21.15.33:0/377295817 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdc08109510 0x7fdc0819ba80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fdbf800b070 tx=0x7fdbf8004d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:32:22.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.428+0000 7fdc05600700 1 -- 172.21.15.33:0/377295817 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbf8021070 con 0x7fdc08109510 2024-05-03T18:32:22.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.429+0000 7fdc0fa00700 1 -- 172.21.15.33:0/377295817 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc0819c470 con 0x7fdc08109510 2024-05-03T18:32:22.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.429+0000 7fdc05600700 1 -- 172.21.15.33:0/377295817 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdbf8007c00 con 0x7fdc08109510 2024-05-03T18:32:22.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.429+0000 7fdc05600700 1 -- 172.21.15.33:0/377295817 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbf8004850 con 0x7fdc08109510 2024-05-03T18:32:22.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.430+0000 7fdc0fa00700 1 -- 172.21.15.33:0/377295817 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc08111b50 con 0x7fdc08109510 2024-05-03T18:32:22.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.444+0000 7fdc05600700 1 -- 172.21.15.33:0/377295817 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fdbf8003fb0 con 0x7fdc08109510 2024-05-03T18:32:22.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.446+0000 7fdc1a2beb80 1 Processor -- start 2024-05-03T18:32:22.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.447+0000 7fdc1a2beb80 1 -- start start 2024-05-03T18:32:22.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.448+0000 7fdc1a2beb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ed9bf337b0 0x55ed9bf33b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:32:22.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.448+0000 7fdc1a2beb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ed9bf340d0 0x55ed9bf2efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:32:22.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.448+0000 7fdc1a2beb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed9bf2f5d0 0x55ed9bf2fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:32:22.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.448+0000 7fdc1a2beb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ed9be81a20 con 0x55ed9bf2f5d0 2024-05-03T18:32:22.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.448+0000 7fdc1a2beb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55ed9be1f760 con 0x55ed9bf337b0 2024-05-03T18:32:22.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.448+0000 7fdc1a2beb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55ed9be7d9e0 con 0x55ed9bf340d0 2024-05-03T18:32:22.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.449+0000 7fdc0d000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ed9bf340d0 0x55ed9bf2efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:32:22.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.449+0000 7fdc0d000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ed9bf340d0 0x55ed9bf2efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:39172/0 (socket says 172.21.15.33:39172) 2024-05-03T18:32:22.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.449+0000 7fdc0d000700 1 -- 172.21.15.33:0/176225049 learned_addr learned my addr 172.21.15.33:0/176225049 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:32:22.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.449+0000 7fdc07e00700 1 --2- 172.21.15.33:0/176225049 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ed9bf337b0 0x55ed9bf33b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:32:22.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.450+0000 7fdc0d000700 1 -- 172.21.15.33:0/176225049 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ed9bf337b0 msgr2=0x55ed9bf33b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:32:22.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.450+0000 7fdc0d000700 1 --2- 172.21.15.33:0/176225049 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ed9bf337b0 0x55ed9bf33b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:22.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.450+0000 7fdc0d000700 1 -- 172.21.15.33:0/176225049 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed9bf2f5d0 msgr2=0x55ed9bf2fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:32:22.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.450+0000 7fdc0d000700 1 --2- 172.21.15.33:0/176225049 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed9bf2f5d0 0x55ed9bf2fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:22.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.450+0000 7fdc0d000700 1 -- 172.21.15.33:0/176225049 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed9bdce8d0 con 0x55ed9bf340d0 2024-05-03T18:32:22.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.450+0000 7fdc05600700 1 --2- 172.21.15.33:0/377295817 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fdbe80b2af0 0x7fdbe80b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:32:22.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.451+0000 7fdc05600700 1 -- 172.21.15.33:0/377295817 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7fdbf80e3a20 con 0x7fdc08109510 2024-05-03T18:32:22.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.451+0000 7fdc07e00700 1 --2- 172.21.15.33:0/176225049 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ed9bf337b0 0x55ed9bf33b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:32:22.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.452+0000 7fdc0d000700 1 --2- 172.21.15.33:0/176225049 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ed9bf340d0 0x55ed9bf2efb0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fdc00009c80 tx=0x7fdc00009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b5340c220343aee3 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:32:22.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.452+0000 7fdbfe000700 1 -- 172.21.15.33:0/176225049 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc00010c90 con 0x55ed9bf340d0 2024-05-03T18:32:22.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.453+0000 7fdbfe000700 1 -- 172.21.15.33:0/176225049 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdc00010e30 con 0x55ed9bf340d0 2024-05-03T18:32:22.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.453+0000 7fdbfe000700 1 -- 172.21.15.33:0/176225049 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc000117c0 con 0x55ed9bf340d0 2024-05-03T18:32:22.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.454+0000 7fdc1a2beb80 1 -- 172.21.15.33:0/176225049 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ed9bf340d0 msgr2=0x55ed9bf2efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:32:22.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.454+0000 7fdc1a2beb80 1 --2- 172.21.15.33:0/176225049 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ed9bf340d0 0x55ed9bf2efb0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fdc00009c80 tx=0x7fdc00009f90 comp rx=0 tx=0).stop 2024-05-03T18:32:22.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.454+0000 7fdc07e00700 1 --2- 172.21.15.33:0/377295817 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fdbe80b2af0 0x7fdbe80b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:32:22.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.454+0000 7fdc1a2beb80 1 -- 172.21.15.33:0/176225049 shutdown_connections 2024-05-03T18:32:22.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.454+0000 7fdc1a2beb80 1 --2- 172.21.15.33:0/176225049 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed9bf2f5d0 0x55ed9bf2fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:22.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.454+0000 7fdc1a2beb80 1 --2- 172.21.15.33:0/176225049 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ed9bf337b0 0x55ed9bf33b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:22.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.454+0000 7fdc1a2beb80 1 --2- 172.21.15.33:0/176225049 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ed9bf340d0 0x55ed9bf2efb0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:22.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.454+0000 7fdc1a2beb80 1 -- 172.21.15.33:0/176225049 >> 172.21.15.33:0/176225049 conn(0x55ed9be5db60 msgr2=0x55ed9bf297f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:32:22.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.454+0000 7fdc1a2beb80 1 -- 172.21.15.33:0/176225049 shutdown_connections 2024-05-03T18:32:22.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.455+0000 7fdc1a2beb80 1 -- 172.21.15.33:0/176225049 wait complete. 2024-05-03T18:32:22.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.455+0000 7fdc1a2beb80 1 -- start start 2024-05-03T18:32:22.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.456+0000 7fdc1a2beb80 1 Processor -- start 2024-05-03T18:32:22.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.456+0000 7fdc07e00700 1 --2- 172.21.15.33:0/377295817 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fdbe80b2af0 0x7fdbe80b4fb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fdbf00046c0 tx=0x7fdbf0004750 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:32:22.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.458+0000 7fdc1a2beb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ed9bf2f5d0 0x55ed9bf3aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:32:22.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.458+0000 7fdc1a2beb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ed9bf337b0 0x55ed9bf38250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:32:22.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.458+0000 7fdc1a2beb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed9bf340d0 0x55ed9bf38790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:32:22.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.458+0000 7fdc1a2beb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ed9bf3bca0 con 0x55ed9bf340d0 2024-05-03T18:32:22.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.458+0000 7fdc1a2beb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55ed9bde7b20 con 0x55ed9bf2f5d0 2024-05-03T18:32:22.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.458+0000 7fdc1a2beb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55ed9bf3be20 con 0x55ed9bf337b0 2024-05-03T18:32:22.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.458+0000 7fdc0d000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ed9bf2f5d0 0x55ed9bf3aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:32:22.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.458+0000 7fdc0d000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ed9bf2f5d0 0x55ed9bf3aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:34306/0 (socket says 172.21.15.33:34306) 2024-05-03T18:32:22.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.458+0000 7fdc0d000700 1 -- 172.21.15.33:0/2947265189 learned_addr learned my addr 172.21.15.33:0/2947265189 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:32:22.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.458+0000 7fdc07e00700 1 --2- 172.21.15.33:0/2947265189 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ed9bf337b0 0x55ed9bf38250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:32:22.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.459+0000 7fdc1a2beb80 10 client.0 _ll_register_callbacks cb 0x55ed9be28c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:32:22.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.459+0000 7fdc0da00700 1 --2- 172.21.15.33:0/2947265189 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed9bf340d0 0x55ed9bf38790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:32:22.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.460+0000 7fdc0d000700 1 -- 172.21.15.33:0/2947265189 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ed9bf337b0 msgr2=0x55ed9bf38250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:32:22.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.460+0000 7fdc0d000700 1 --2- 172.21.15.33:0/2947265189 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ed9bf337b0 0x55ed9bf38250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:22.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.460+0000 7fdc0d000700 1 -- 172.21.15.33:0/2947265189 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed9bf340d0 msgr2=0x55ed9bf38790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:32:22.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.460+0000 7fdc0d000700 1 --2- 172.21.15.33:0/2947265189 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed9bf340d0 0x55ed9bf38790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:22.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.460+0000 7fdc0d000700 1 -- 172.21.15.33:0/2947265189 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed9be47420 con 0x55ed9bf2f5d0 2024-05-03T18:32:22.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.461+0000 7fdc0d000700 1 --2- 172.21.15.33:0/2947265189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ed9bf2f5d0 0x55ed9bf3aeb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fdc00012a40 tx=0x7fdc00012a70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b92f07f698b83c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:32:22.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.461+0000 7fdbf6a00700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-03T18:32:22.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.462+0000 7fdbf6a00700 1 -- 172.21.15.33:0/2947265189 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc000117c0 con 0x55ed9bf2f5d0 2024-05-03T18:32:22.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.462+0000 7fdbf6a00700 1 -- 172.21.15.33:0/2947265189 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7fdc00011560 con 0x55ed9bf2f5d0 2024-05-03T18:32:22.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.462+0000 7fdc1a2beb80 1 -- 172.21.15.33:0/2947265189 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc00010e30 con 0x55ed9bf2f5d0 2024-05-03T18:32:22.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.463+0000 7fdbf6a00700 1 -- 172.21.15.33:0/2947265189 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fdc00032050 con 0x55ed9bf2f5d0 2024-05-03T18:32:22.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.463+0000 7fdc1a2beb80 10 client.35549 fetch_fsmap learned FSMap version 21 2024-05-03T18:32:22.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.464+0000 7fdc1a2beb80 1 -- 172.21.15.33:0/2947265189 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55ed9bf463d0 con 0x55ed9bf2f5d0 2024-05-03T18:32:22.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.464+0000 7fdbf6a00700 1 -- 172.21.15.33:0/2947265189 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdc00010e30 con 0x55ed9bf2f5d0 2024-05-03T18:32:22.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.465+0000 7fdbf6a00700 1 -- 172.21.15.33:0/2947265189 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fdc00004b80 con 0x55ed9bf2f5d0 2024-05-03T18:32:22.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.466+0000 7fdc1a2beb80 10 client.35549 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:32:22.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.466+0000 7fdc1a2beb80 10 client.35549 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:32:22.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.466+0000 7fdc1a2beb80 20 client.35549 populate_metadata read hostname 'smithi033' 2024-05-03T18:32:22.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.466+0000 7fdc1a2beb80 1 --2- 172.21.15.33:0/2947265189 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55ed9bf48950 0x55ed9bf4ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:32:22.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.466+0000 7fdc1a2beb80 4 client.35549 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:32:22.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.466+0000 7fdc1a2beb80 1 -- 172.21.15.33:0/2947265189 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ed9befc1d0 con 0x55ed9bf48950 2024-05-03T18:32:22.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.467+0000 7fdc07e00700 1 --2- 172.21.15.33:0/2947265189 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55ed9bf48950 0x55ed9bf4ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:32:22.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.469+0000 7fdc07e00700 1 --2- 172.21.15.33:0/2947265189 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55ed9bf48950 0x55ed9bf4ad40 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2675fcdd7e811b47 server_cookie=3d34fe03cf4fc53 in_seq=0 out_seq=0 2024-05-03T18:32:22.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.469+0000 7fdbf6a00700 10 client.35549 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:32:22.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.475+0000 7fdbf6a00700 1 -- 172.21.15.33:0/2947265189 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ed9befc1d0 con 0x55ed9bf48950 2024-05-03T18:32:22.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.475+0000 7fdbf6a00700 10 client.35549 handle_command_reply: tid=0 2024-05-03T18:32:22.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.475+0000 7fdc1a2beb80 1 client.35549 shutdown 2024-05-03T18:32:22.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.475+0000 7fdc1a2beb80 1 -- 172.21.15.33:0/2947265189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ed9bf2f5d0 msgr2=0x55ed9bf3aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:32:22.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.475+0000 7fdc1a2beb80 1 --2- 172.21.15.33:0/2947265189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ed9bf2f5d0 0x55ed9bf3aeb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fdc00012a40 tx=0x7fdc00012a70 comp rx=0 tx=0).stop 2024-05-03T18:32:22.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.475+0000 7fdc1a2beb80 1 -- 172.21.15.33:0/2947265189 shutdown_connections 2024-05-03T18:32:22.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.475+0000 7fdc1a2beb80 1 --2- 172.21.15.33:0/2947265189 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55ed9bf48950 0x55ed9bf4ad40 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:22.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.475+0000 7fdc1a2beb80 1 --2- 172.21.15.33:0/2947265189 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed9bf340d0 0x55ed9bf38790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:22.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.475+0000 7fdc1a2beb80 1 --2- 172.21.15.33:0/2947265189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ed9bf2f5d0 0x55ed9bf3aeb0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:22.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.476+0000 7fdc1a2beb80 1 --2- 172.21.15.33:0/2947265189 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ed9bf337b0 0x55ed9bf38250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:22.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.476+0000 7fdc1a2beb80 1 -- 172.21.15.33:0/2947265189 >> 172.21.15.33:0/2947265189 conn(0x55ed9be5db60 msgr2=0x55ed9bf28ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:32:22.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.476+0000 7fdbf6a00700 0 client.35549 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:32:22.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.476+0000 7fdc1a2beb80 1 -- 172.21.15.33:0/2947265189 shutdown_connections 2024-05-03T18:32:22.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.476+0000 7fdc1a2beb80 1 -- 172.21.15.33:0/2947265189 wait complete. 2024-05-03T18:32:22.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.476+0000 7fdc1a2beb80 20 client.35549 trim_cache size 0 max 16384 2024-05-03T18:32:22.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.518+0000 7fdc1a2beb80 1 Processor -- start 2024-05-03T18:32:22.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.519+0000 7fdc1a2beb80 1 -- start start 2024-05-03T18:32:22.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.519+0000 7fdc1a2beb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ed9bf2f5d0 0x55ed9bf29660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:32:22.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.519+0000 7fdc1a2beb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ed9bf337b0 0x55ed9bf2ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:32:22.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.519+0000 7fdc1a2beb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed9bf340d0 0x55ed9bf50310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:32:22.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.520+0000 7fdc1a2beb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fdc081b3940 con 0x55ed9bf340d0 2024-05-03T18:32:22.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.520+0000 7fdc1a2beb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fdc081b61c0 con 0x55ed9bf2f5d0 2024-05-03T18:32:22.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.520+0000 7fdc1a2beb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fdc081b3fb0 con 0x55ed9bf337b0 2024-05-03T18:32:22.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.520+0000 7fdc07e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ed9bf337b0 0x55ed9bf2ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:32:22.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.520+0000 7fdc07e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ed9bf337b0 0x55ed9bf2ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:39190/0 (socket says 172.21.15.33:39190) 2024-05-03T18:32:22.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.520+0000 7fdc07e00700 1 -- 172.21.15.33:0/2469769392 learned_addr learned my addr 172.21.15.33:0/2469769392 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:32:22.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.520+0000 7fdc07e00700 1 -- 172.21.15.33:0/2469769392 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ed9bf2f5d0 msgr2=0x55ed9bf29660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:32:22.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.520+0000 7fdc0d000700 1 --2- 172.21.15.33:0/2469769392 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ed9bf2f5d0 0x55ed9bf29660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:32:22.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.520+0000 7fdc07e00700 1 --2- 172.21.15.33:0/2469769392 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ed9bf2f5d0 0x55ed9bf29660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:22.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.521+0000 7fdc07e00700 1 -- 172.21.15.33:0/2469769392 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed9bf340d0 msgr2=0x55ed9bf50310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:32:22.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.521+0000 7fdc07e00700 1 --2- 172.21.15.33:0/2469769392 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed9bf340d0 0x55ed9bf50310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:22.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.521+0000 7fdc07e00700 1 -- 172.21.15.33:0/2469769392 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed9bf44d30 con 0x55ed9bf337b0 2024-05-03T18:32:22.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.521+0000 7fdc0d000700 1 --2- 172.21.15.33:0/2469769392 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ed9bf2f5d0 0x55ed9bf29660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:32:22.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.521+0000 7fdc07e00700 1 --2- 172.21.15.33:0/2469769392 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ed9bf337b0 0x55ed9bf2ea80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x55ed9bf4b2b0 tx=0x7fdbf0013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b194f3cc8de82ad server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:32:22.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.522+0000 7fdbfd600700 1 -- 172.21.15.33:0/2469769392 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbf00207a0 con 0x55ed9bf337b0 2024-05-03T18:32:22.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.522+0000 7fdbfd600700 1 -- 172.21.15.33:0/2469769392 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdbf002c070 con 0x55ed9bf337b0 2024-05-03T18:32:22.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.522+0000 7fdbfd600700 1 -- 172.21.15.33:0/2469769392 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbf00279b0 con 0x55ed9bf337b0 2024-05-03T18:32:22.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.523+0000 7fdc1a2beb80 1 -- 172.21.15.33:0/2469769392 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ed9bf337b0 msgr2=0x55ed9bf2ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:32:22.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.523+0000 7fdc1a2beb80 1 --2- 172.21.15.33:0/2469769392 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ed9bf337b0 0x55ed9bf2ea80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x55ed9bf4b2b0 tx=0x7fdbf0013000 comp rx=0 tx=0).stop 2024-05-03T18:32:22.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.524+0000 7fdc1a2beb80 1 -- 172.21.15.33:0/2469769392 shutdown_connections 2024-05-03T18:32:22.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.524+0000 7fdc1a2beb80 1 --2- 172.21.15.33:0/2469769392 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed9bf340d0 0x55ed9bf50310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:22.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.524+0000 7fdc1a2beb80 1 --2- 172.21.15.33:0/2469769392 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ed9bf2f5d0 0x55ed9bf29660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:22.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.524+0000 7fdc1a2beb80 1 --2- 172.21.15.33:0/2469769392 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ed9bf337b0 0x55ed9bf2ea80 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:22.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.524+0000 7fdc1a2beb80 1 -- 172.21.15.33:0/2469769392 >> 172.21.15.33:0/2469769392 conn(0x55ed9be5db60 msgr2=0x55ed9bf2e370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:32:22.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.524+0000 7fdc1a2beb80 1 -- 172.21.15.33:0/2469769392 shutdown_connections 2024-05-03T18:32:22.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.524+0000 7fdc1a2beb80 1 -- 172.21.15.33:0/2469769392 wait complete. 2024-05-03T18:32:22.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.525+0000 7fdc1a2beb80 1 -- start start 2024-05-03T18:32:22.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.525+0000 7fdc1a2beb80 1 Processor -- start 2024-05-03T18:32:22.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.526+0000 7fdc1a2beb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed9bf2f5d0 0x55ed9bf410b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:32:22.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.526+0000 7fdc1a2beb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ed9bf337b0 0x55ed9bf415f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:32:22.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.526+0000 7fdc1a2beb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ed9bf340d0 0x55ed9bf3e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:32:22.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.527+0000 7fdc1a2beb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fdc081b4680 con 0x55ed9bf2f5d0 2024-05-03T18:32:22.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.527+0000 7fdc1a2beb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fdc081b5af0 con 0x55ed9bf337b0 2024-05-03T18:32:22.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.528+0000 7fdc0d000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed9bf2f5d0 0x55ed9bf410b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:32:22.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.528+0000 7fdc1a2beb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fdc081b4d50 con 0x55ed9bf340d0 2024-05-03T18:32:22.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.528+0000 7fdc0d000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed9bf2f5d0 0x55ed9bf410b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58858/0 (socket says 172.21.15.33:58858) 2024-05-03T18:32:22.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.528+0000 7fdc1a2beb80 10 client.0 _ll_register_callbacks cb 0x55ed9be28c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:32:22.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.528+0000 7fdc0d000700 1 -- 172.21.15.33:0/1312363359 learned_addr learned my addr 172.21.15.33:0/1312363359 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:32:22.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.528+0000 7fdc07e00700 1 --2- 172.21.15.33:0/1312363359 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ed9bf337b0 0x55ed9bf415f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:32:22.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.528+0000 7fdc0da00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ed9bf340d0 0x55ed9bf3e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:32:22.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.529+0000 7fdc0d000700 1 -- 172.21.15.33:0/1312363359 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ed9bf340d0 msgr2=0x55ed9bf3e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:32:22.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.529+0000 7fdc0d000700 1 --2- 172.21.15.33:0/1312363359 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ed9bf340d0 0x55ed9bf3e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:22.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.529+0000 7fdc0d000700 1 -- 172.21.15.33:0/1312363359 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ed9bf337b0 msgr2=0x55ed9bf415f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:32:22.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.529+0000 7fdc0d000700 1 --2- 172.21.15.33:0/1312363359 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ed9bf337b0 0x55ed9bf415f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:22.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.529+0000 7fdc0d000700 1 -- 172.21.15.33:0/1312363359 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed9bf3e5c0 con 0x55ed9bf2f5d0 2024-05-03T18:32:22.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.530+0000 7fdc0d000700 1 --2- 172.21.15.33:0/1312363359 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed9bf2f5d0 0x55ed9bf410b0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fdc00006010 tx=0x7fdc00005980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc754a7b7f088f4b server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:32:22.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.530+0000 7fdbf6a00700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:32:22.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.530+0000 7fdbf6a00700 1 -- 172.21.15.33:0/1312363359 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc0001ca90 con 0x55ed9bf2f5d0 2024-05-03T18:32:22.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.530+0000 7fdbf6a00700 1 -- 172.21.15.33:0/1312363359 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7fdc00004710 con 0x55ed9bf2f5d0 2024-05-03T18:32:22.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.530+0000 7fdc1a2beb80 1 -- 172.21.15.33:0/1312363359 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbf002c070 con 0x55ed9bf2f5d0 2024-05-03T18:32:22.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.531+0000 7fdbf6a00700 1 -- 172.21.15.33:0/1312363359 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fdc00004910 con 0x55ed9bf2f5d0 2024-05-03T18:32:22.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.531+0000 7fdc1a2beb80 10 client.15954 fetch_fsmap learned FSMap version 21 2024-05-03T18:32:22.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.531+0000 7fdc1a2beb80 1 -- 172.21.15.33:0/1312363359 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55ed9bfb35f0 con 0x55ed9bf2f5d0 2024-05-03T18:32:22.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.532+0000 7fdbf6a00700 1 -- 172.21.15.33:0/1312363359 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdbf002c070 con 0x55ed9bf2f5d0 2024-05-03T18:32:22.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.532+0000 7fdbf6a00700 1 -- 172.21.15.33:0/1312363359 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fdc00012070 con 0x55ed9bf2f5d0 2024-05-03T18:32:22.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.532+0000 7fdc1a2beb80 10 client.15954 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:32:22.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.532+0000 7fdc1a2beb80 10 client.15954 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:32:22.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.532+0000 7fdc1a2beb80 20 client.15954 populate_metadata read hostname 'smithi033' 2024-05-03T18:32:22.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.532+0000 7fdc1a2beb80 1 --2- 172.21.15.33:0/1312363359 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55ed9bfb3d00 0x55ed9bfb6120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:32:22.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.532+0000 7fdc1a2beb80 4 client.15954 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-03T18:32:22.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.532+0000 7fdc1a2beb80 1 -- 172.21.15.33:0/1312363359 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ed9bf33420 con 0x55ed9bfb3d00 2024-05-03T18:32:22.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.533+0000 7fdc07e00700 1 --2- 172.21.15.33:0/1312363359 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55ed9bfb3d00 0x55ed9bfb6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:32:22.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.534+0000 7fdc07e00700 1 --2- 172.21.15.33:0/1312363359 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55ed9bfb3d00 0x55ed9bfb6120 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d218db1e73535a50 server_cookie=429047b31268279 in_seq=0 out_seq=0 2024-05-03T18:32:22.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.534+0000 7fdbf6a00700 10 client.15954 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:32:22.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.535+0000 7fdbf6a00700 1 -- 172.21.15.33:0/1312363359 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55ed9bf33420 con 0x55ed9bfb3d00 2024-05-03T18:32:22.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.535+0000 7fdbf6a00700 10 client.15954 handle_command_reply: tid=0 2024-05-03T18:32:22.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.535+0000 7fdc1a2beb80 1 client.15954 shutdown 2024-05-03T18:32:22.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.535+0000 7fdc1a2beb80 1 -- 172.21.15.33:0/1312363359 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed9bf2f5d0 msgr2=0x55ed9bf410b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:32:22.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.535+0000 7fdc1a2beb80 1 --2- 172.21.15.33:0/1312363359 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed9bf2f5d0 0x55ed9bf410b0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fdc00006010 tx=0x7fdc00005980 comp rx=0 tx=0).stop 2024-05-03T18:32:22.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.536+0000 7fdc1a2beb80 1 -- 172.21.15.33:0/1312363359 shutdown_connections 2024-05-03T18:32:22.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.536+0000 7fdc1a2beb80 1 --2- 172.21.15.33:0/1312363359 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55ed9bfb3d00 0x55ed9bfb6120 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:22.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.536+0000 7fdc1a2beb80 1 --2- 172.21.15.33:0/1312363359 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ed9bf340d0 0x55ed9bf3e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:22.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.536+0000 7fdc1a2beb80 1 --2- 172.21.15.33:0/1312363359 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed9bf2f5d0 0x55ed9bf410b0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:22.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.536+0000 7fdc1a2beb80 1 --2- 172.21.15.33:0/1312363359 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ed9bf337b0 0x55ed9bf415f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:22.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.536+0000 7fdc1a2beb80 1 -- 172.21.15.33:0/1312363359 >> 172.21.15.33:0/1312363359 conn(0x55ed9be5db60 msgr2=0x55ed9bf39850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:32:22.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.537+0000 7fdbf6a00700 0 client.15954 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:32:22.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.537+0000 7fdc1a2beb80 1 -- 172.21.15.33:0/1312363359 shutdown_connections 2024-05-03T18:32:22.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.537+0000 7fdc1a2beb80 1 -- 172.21.15.33:0/1312363359 wait complete. 2024-05-03T18:32:22.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.538+0000 7fdc1a2beb80 20 client.15954 trim_cache size 0 max 16384 2024-05-03T18:32:22.542 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-03T18:32:22.542 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-05-03T18:32:22.542 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-05-03T18:32:22.542 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-03T18:32:22.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.539+0000 7fdc0fa00700 1 -- 172.21.15.33:0/377295817 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fdbe80b2af0 msgr2=0x7fdbe80b4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:32:22.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.539+0000 7fdc0fa00700 1 --2- 172.21.15.33:0/377295817 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fdbe80b2af0 0x7fdbe80b4fb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fdbf00046c0 tx=0x7fdbf0004750 comp rx=0 tx=0).stop 2024-05-03T18:32:22.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.539+0000 7fdc0fa00700 1 -- 172.21.15.33:0/377295817 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdc08109510 msgr2=0x7fdc0819ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:32:22.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.539+0000 7fdc0fa00700 1 --2- 172.21.15.33:0/377295817 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdc08109510 0x7fdc0819ba80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fdbf800b070 tx=0x7fdbf8004d30 comp rx=0 tx=0).stop 2024-05-03T18:32:22.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.539+0000 7fdc0fa00700 1 -- 172.21.15.33:0/377295817 shutdown_connections 2024-05-03T18:32:22.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.539+0000 7fdc0fa00700 1 --2- 172.21.15.33:0/377295817 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdc08109510 0x7fdc0819ba80 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:22.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.539+0000 7fdc0fa00700 1 --2- 172.21.15.33:0/377295817 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fdbe80b2af0 0x7fdbe80b4fb0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:22.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.539+0000 7fdc0fa00700 1 --2- 172.21.15.33:0/377295817 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc0806b180 0x7fdc081a18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:22.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.539+0000 7fdc0fa00700 1 --2- 172.21.15.33:0/377295817 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc0806bb40 0x7fdc081a1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:22.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.539+0000 7fdc0fa00700 1 -- 172.21.15.33:0/377295817 >> 172.21.15.33:0/377295817 conn(0x7fdc08079350 msgr2=0x7fdc08101090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:32:22.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.541+0000 7fdc0fa00700 1 -- 172.21.15.33:0/377295817 shutdown_connections 2024-05-03T18:32:22.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:22.541+0000 7fdc0fa00700 1 -- 172.21.15.33:0/377295817 wait complete. 2024-05-03T18:32:23.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:32:22 smithi099 ceph-mon[31658]: pgmap v158: 129 pgs: 129 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 5.6 MiB/s wr, 653 op/s 2024-05-03T18:32:23.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:32:22 smithi150 ceph-mon[32492]: pgmap v158: 129 pgs: 129 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 5.6 MiB/s wr, 653 op/s 2024-05-03T18:32:23.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:32:22 smithi033 ceph-mon[28330]: pgmap v158: 129 pgs: 129 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 5.6 MiB/s wr, 653 op/s 2024-05-03T18:32:24.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:32:23 smithi033 ceph-mon[28330]: pgmap v159: 129 pgs: 129 active+clean; 1.6 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 8.0 MiB/s wr, 939 op/s 2024-05-03T18:32:24.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:32:23 smithi099 ceph-mon[31658]: pgmap v159: 129 pgs: 129 active+clean; 1.6 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 8.0 MiB/s wr, 939 op/s 2024-05-03T18:32:24.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:32:23 smithi150 ceph-mon[32492]: pgmap v159: 129 pgs: 129 active+clean; 1.6 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 8.0 MiB/s wr, 939 op/s 2024-05-03T18:32:24.999 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-03T18:32:24.999 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T18:32:25.111 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 70.0 s, 529.7 tps, lat 188.554 ms stddev 97.336 2024-05-03T18:32:25.717 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:32:27.236 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:32:26 smithi033 ceph-mon[28330]: pgmap v160: 129 pgs: 129 active+clean; 1.6 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.1 MiB/s wr, 604 op/s 2024-05-03T18:32:27.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:32:26 smithi099 ceph-mon[31658]: pgmap v160: 129 pgs: 129 active+clean; 1.6 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.1 MiB/s wr, 604 op/s 2024-05-03T18:32:27.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:32:26 smithi150 ceph-mon[32492]: pgmap v160: 129 pgs: 129 active+clean; 1.6 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.1 MiB/s wr, 604 op/s 2024-05-03T18:32:29.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:32:28 smithi033 ceph-mon[28330]: pgmap v161: 129 pgs: 129 active+clean; 1.7 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 7.6 MiB/s wr, 901 op/s 2024-05-03T18:32:29.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:32:28 smithi099 ceph-mon[31658]: pgmap v161: 129 pgs: 129 active+clean; 1.7 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 7.6 MiB/s wr, 901 op/s 2024-05-03T18:32:29.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:32:28 smithi150 ceph-mon[32492]: pgmap v161: 129 pgs: 129 active+clean; 1.7 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 7.6 MiB/s wr, 901 op/s 2024-05-03T18:32:30.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:30.015+0000 7fef2fe00700 1 -- 172.21.15.33:0/3938066756 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef2810ff50 msgr2=0x7fef28112340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:32:30.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:30.015+0000 7fef2fe00700 1 --2- 172.21.15.33:0/3938066756 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef2810ff50 0x7fef28112340 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fef20009990 tx=0x7fef20009ca0 comp rx=0 tx=0).stop 2024-05-03T18:32:30.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:30.015+0000 7fef2fe00700 1 -- 172.21.15.33:0/3938066756 shutdown_connections 2024-05-03T18:32:30.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:30.015+0000 7fef2fe00700 1 --2- 172.21.15.33:0/3938066756 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef2810ff50 0x7fef28112340 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:30.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:30.015+0000 7fef2fe00700 1 --2- 172.21.15.33:0/3938066756 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef2806b1b0 0x7fef2806b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:30.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:30.015+0000 7fef2fe00700 1 --2- 172.21.15.33:0/3938066756 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef2806bad0 0x7fef2810fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:30.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:30.015+0000 7fef2fe00700 1 -- 172.21.15.33:0/3938066756 >> 172.21.15.33:0/3938066756 conn(0x7fef28079350 msgr2=0x7fef28079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:32:30.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:30.016+0000 7fef2fe00700 1 -- 172.21.15.33:0/3938066756 shutdown_connections 2024-05-03T18:32:30.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:30.016+0000 7fef2fe00700 1 -- 172.21.15.33:0/3938066756 wait complete. 2024-05-03T18:32:30.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:30.017+0000 7fef2fe00700 1 Processor -- start 2024-05-03T18:32:30.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:30.018+0000 7fef2fe00700 1 -- start start 2024-05-03T18:32:30.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:30.018+0000 7fef2fe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef2806b1b0 0x7fef28074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:32:30.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:30.018+0000 7fef2fe00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef2806bad0 0x7fef2806dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:32:30.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:30.018+0000 7fef2fe00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef2810ff50 0x7fef2806e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:32:30.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:30.018+0000 7fef2fe00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fef28115770 con 0x7fef2806bad0 2024-05-03T18:32:30.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:30.018+0000 7fef2fe00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fef281155f0 con 0x7fef2806b1b0 2024-05-03T18:32:30.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:30.018+0000 7fef2fe00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fef281158f0 con 0x7fef2810ff50 2024-05-03T18:32:30.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:30.019+0000 7fef2de00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef2810ff50 0x7fef2806e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:32:30.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:30.019+0000 7fef2de00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef2810ff50 0x7fef2806e2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:33658/0 (socket says 172.21.15.33:33658) 2024-05-03T18:32:30.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:30.019+0000 7fef2de00700 1 -- 172.21.15.33:0/3737741181 learned_addr learned my addr 172.21.15.33:0/3737741181 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:32:30.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:30.019+0000 7fef2d400700 1 --2- 172.21.15.33:0/3737741181 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef2806b1b0 0x7fef28074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:32:30.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:30.019+0000 7fef2ca00700 1 --2- 172.21.15.33:0/3737741181 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef2806bad0 0x7fef2806dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:32:30.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:30.019+0000 7fef2de00700 1 -- 172.21.15.33:0/3737741181 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef2806b1b0 msgr2=0x7fef28074d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:32:30.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:30.019+0000 7fef2de00700 1 --2- 172.21.15.33:0/3737741181 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef2806b1b0 0x7fef28074d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:30.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:30.019+0000 7fef2de00700 1 -- 172.21.15.33:0/3737741181 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef2806bad0 msgr2=0x7fef2806dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:32:30.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:30.019+0000 7fef2de00700 1 --2- 172.21.15.33:0/3737741181 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef2806bad0 0x7fef2806dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:30.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:30.019+0000 7fef2de00700 1 -- 172.21.15.33:0/3737741181 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef2806e940 con 0x7fef2810ff50 2024-05-03T18:32:30.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:30.020+0000 7fef2de00700 1 --2- 172.21.15.33:0/3737741181 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef2810ff50 0x7fef2806e2d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fef200047b0 tx=0x7fef20005ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:32:30.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:30.020+0000 7fef26000700 1 -- 172.21.15.33:0/3737741181 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef20026070 con 0x7fef2810ff50 2024-05-03T18:32:30.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:30.020+0000 7fef26000700 1 -- 172.21.15.33:0/3737741181 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fef20021410 con 0x7fef2810ff50 2024-05-03T18:32:30.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:30.020+0000 7fef26000700 1 -- 172.21.15.33:0/3737741181 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef20017650 con 0x7fef2810ff50 2024-05-03T18:32:30.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:30.021+0000 7fef2fe00700 1 -- 172.21.15.33:0/3737741181 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef2806ebd0 con 0x7fef2810ff50 2024-05-03T18:32:30.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:30.021+0000 7fef2fe00700 1 -- 172.21.15.33:0/3737741181 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef28072a60 con 0x7fef2810ff50 2024-05-03T18:32:30.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:30.025+0000 7fef2fe00700 1 -- 172.21.15.33:0/3737741181 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef281029c0 con 0x7fef2810ff50 2024-05-03T18:32:30.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:30.029+0000 7fef26000700 1 -- 172.21.15.33:0/3737741181 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fef20029020 con 0x7fef2810ff50 2024-05-03T18:32:30.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:30.032+0000 7fef26000700 1 --2- 172.21.15.33:0/3737741181 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fef100b2aa0 0x7fef100b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:32:30.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:30.032+0000 7fef26000700 1 -- 172.21.15.33:0/3737741181 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7fef200e3790 con 0x7fef2810ff50 2024-05-03T18:32:30.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:30.032+0000 7fef2d400700 1 --2- 172.21.15.33:0/3737741181 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fef100b2aa0 0x7fef100b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:32:30.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:30.034+0000 7fef2d400700 1 --2- 172.21.15.33:0/3737741181 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fef100b2aa0 0x7fef100b4f60 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fef1c009380 tx=0x7fef1c00b040 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:32:30.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:30.037+0000 7fef26000700 1 -- 172.21.15.33:0/3737741181 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fef20013070 con 0x7fef2810ff50 2024-05-03T18:32:30.111 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 75.0 s, 531.7 tps, lat 179.654 ms stddev 80.006 2024-05-03T18:32:30.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:30.582+0000 7fef2fe00700 1 -- 172.21.15.33:0/3737741181 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fef280735c0 con 0x7fef2810ff50 2024-05-03T18:32:30.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:30.584+0000 7fef26000700 1 -- 172.21.15.33:0/3737741181 <== mon.1 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11801 (secure 0 0 0) 0x7fef200215c0 con 0x7fef2810ff50 2024-05-03T18:32:30.586 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:32:30.586 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1327393503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1327393503},{"type":"v1","addr":"172.21.15.33:6835","nonce":1327393503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/2461472948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":2461472948},{"type":"v1","addr":"172.21.15.33:6837","nonce":2461472948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14559,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/442269281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":442269281},{"type":"v1","addr":"172.21.15.33:6839","nonce":442269281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14571,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/645571756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":645571756},{"type":"v1","addr":"172.21.15.33:6841","nonce":645571756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/203598909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":203598909},{"type":"v1","addr":"172.21.15.150:6833","nonce":203598909}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/4029842493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":4029842493},{"type":"v1","addr":"172.21.15.150:6835","nonce":4029842493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3463664423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3463664423},{"type":"v1","addr":"172.21.15.150:6837","nonce":3463664423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24481,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6839/4051648018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":4051648018},{"type":"v1","addr":"172.21.15.150:6839","nonce":4051648018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34418,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1865105479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1865105479},{"type":"v1","addr":"172.21.15.99:6833","nonce":1865105479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34445,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/1206393914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1206393914},{"type":"v1","addr":"172.21.15.99:6835","nonce":1206393914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-03T18:22:28.837091+0000","modified":"2024-05-03T18:23:49.758398+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34508},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34472":{"gid":34472,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6837/3360970342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3360970342},{"type":"v1","addr":"172.21.15.99:6837","nonce":3360970342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34508":{"gid":34508,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/919907308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":919907308},{"type":"v1","addr":"172.21.15.99:6839","nonce":919907308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34508,"qdb_cluster":[34508]},"id":1}]} 2024-05-03T18:32:30.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:30.588+0000 7fef2fe00700 1 -- 172.21.15.33:0/3737741181 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fef100b2aa0 msgr2=0x7fef100b4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:32:30.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:30.588+0000 7fef2fe00700 1 --2- 172.21.15.33:0/3737741181 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fef100b2aa0 0x7fef100b4f60 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fef1c009380 tx=0x7fef1c00b040 comp rx=0 tx=0).stop 2024-05-03T18:32:30.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:30.588+0000 7fef2fe00700 1 -- 172.21.15.33:0/3737741181 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef2810ff50 msgr2=0x7fef2806e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:32:30.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:30.588+0000 7fef2fe00700 1 --2- 172.21.15.33:0/3737741181 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef2810ff50 0x7fef2806e2d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fef200047b0 tx=0x7fef20005ad0 comp rx=0 tx=0).stop 2024-05-03T18:32:30.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:30.589+0000 7fef2fe00700 1 -- 172.21.15.33:0/3737741181 shutdown_connections 2024-05-03T18:32:30.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:30.589+0000 7fef2fe00700 1 --2- 172.21.15.33:0/3737741181 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef2810ff50 0x7fef2806e2d0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:30.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:30.589+0000 7fef2fe00700 1 --2- 172.21.15.33:0/3737741181 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fef100b2aa0 0x7fef100b4f60 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:30.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:30.589+0000 7fef2fe00700 1 --2- 172.21.15.33:0/3737741181 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef2806b1b0 0x7fef28074d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:30.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:30.589+0000 7fef2fe00700 1 --2- 172.21.15.33:0/3737741181 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef2806bad0 0x7fef2806dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:30.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:30.589+0000 7fef2fe00700 1 -- 172.21.15.33:0/3737741181 >> 172.21.15.33:0/3737741181 conn(0x7fef28079350 msgr2=0x7fef28111460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:32:30.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:30.589+0000 7fef2fe00700 1 -- 172.21.15.33:0/3737741181 shutdown_connections 2024-05-03T18:32:30.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:30.590+0000 7fef2fe00700 1 -- 172.21.15.33:0/3737741181 wait complete. 2024-05-03T18:32:30.593 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 21 2024-05-03T18:32:31.262 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:32:30 smithi033 ceph-mon[28330]: pgmap v162: 129 pgs: 129 active+clean; 1.7 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 4.9 MiB/s wr, 582 op/s 2024-05-03T18:32:31.262 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:32:30 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/3737741181' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:32:31.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:32:30 smithi099 ceph-mon[31658]: pgmap v162: 129 pgs: 129 active+clean; 1.7 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 4.9 MiB/s wr, 582 op/s 2024-05-03T18:32:31.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:32:30 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/3737741181' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:32:31.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:32:30 smithi150 ceph-mon[32492]: pgmap v162: 129 pgs: 129 active+clean; 1.7 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 4.9 MiB/s wr, 582 op/s 2024-05-03T18:32:31.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:32:30 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/3737741181' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:32:31.743 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-03T18:32:32.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:32:32 smithi150 ceph-mon[32492]: pgmap v163: 129 pgs: 129 active+clean; 1.7 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 4.9 MiB/s wr, 582 op/s 2024-05-03T18:32:32.442 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:32:32.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:32:32 smithi033 ceph-mon[28330]: pgmap v163: 129 pgs: 129 active+clean; 1.7 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 4.9 MiB/s wr, 582 op/s 2024-05-03T18:32:32.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:32:32 smithi099 ceph-mon[31658]: pgmap v163: 129 pgs: 129 active+clean; 1.7 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 4.9 MiB/s wr, 582 op/s 2024-05-03T18:32:35.111 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 80.0 s, 572.1 tps, lat 183.045 ms stddev 527.276 2024-05-03T18:32:35.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:32:34 smithi033 ceph-mon[28330]: pgmap v164: 129 pgs: 129 active+clean; 1.7 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 7.9 MiB/s wr, 907 op/s 2024-05-03T18:32:35.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.291+0000 7f27b8c00700 1 -- 172.21.15.33:0/944942206 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f27b41023a0 msgr2=0x7f27b4102820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:32:35.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.291+0000 7f27b8c00700 1 --2- 172.21.15.33:0/944942206 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f27b41023a0 0x7f27b4102820 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f27a00098f0 tx=0x7f27a0009c00 comp rx=0 tx=0).stop 2024-05-03T18:32:35.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.291+0000 7f27b8c00700 1 -- 172.21.15.33:0/944942206 shutdown_connections 2024-05-03T18:32:35.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.291+0000 7f27b8c00700 1 --2- 172.21.15.33:0/944942206 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27b4102d60 0x7f27b410f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:35.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.291+0000 7f27b8c00700 1 --2- 172.21.15.33:0/944942206 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27b4108410 0x7f27b41087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:35.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.291+0000 7f27b8c00700 1 --2- 172.21.15.33:0/944942206 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f27b41023a0 0x7f27b4102820 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:35.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.291+0000 7f27b8c00700 1 -- 172.21.15.33:0/944942206 >> 172.21.15.33:0/944942206 conn(0x7f27b4079350 msgr2=0x7f27b4079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:32:35.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.291+0000 7f27b8c00700 1 -- 172.21.15.33:0/944942206 shutdown_connections 2024-05-03T18:32:35.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.292+0000 7f27b8c00700 1 -- 172.21.15.33:0/944942206 wait complete. 2024-05-03T18:32:35.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.293+0000 7f27b8c00700 1 Processor -- start 2024-05-03T18:32:35.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.293+0000 7f27b8c00700 1 -- start start 2024-05-03T18:32:35.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:32:34 smithi099 ceph-mon[31658]: pgmap v164: 129 pgs: 129 active+clean; 1.7 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 7.9 MiB/s wr, 907 op/s 2024-05-03T18:32:35.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.294+0000 7f27b8c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27b41023a0 0x7f27b410d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:32:35.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.294+0000 7f27b8c00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27b4102d60 0x7f27b4074f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:32:35.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.294+0000 7f27b8c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f27b4108410 0x7f27b410db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:32:35.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.294+0000 7f27b8c00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f27b41109a0 con 0x7f27b4102d60 2024-05-03T18:32:35.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.294+0000 7f27b8c00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f27b4110820 con 0x7f27b41023a0 2024-05-03T18:32:35.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.294+0000 7f27b8c00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f27b4110b20 con 0x7f27b4108410 2024-05-03T18:32:35.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.295+0000 7f27b2800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f27b4108410 0x7f27b410db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:32:35.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.295+0000 7f27b2800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f27b4108410 0x7f27b410db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:33682/0 (socket says 172.21.15.33:33682) 2024-05-03T18:32:35.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.295+0000 7f27b2800700 1 -- 172.21.15.33:0/948846216 learned_addr learned my addr 172.21.15.33:0/948846216 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:32:35.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.295+0000 7f27b1e00700 1 --2- 172.21.15.33:0/948846216 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27b41023a0 0x7f27b410d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:32:35.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.295+0000 7f27b1400700 1 --2- 172.21.15.33:0/948846216 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27b4102d60 0x7f27b4074f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:32:35.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.296+0000 7f27b1e00700 1 -- 172.21.15.33:0/948846216 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f27b4108410 msgr2=0x7f27b410db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:32:35.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.296+0000 7f27b1e00700 1 --2- 172.21.15.33:0/948846216 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f27b4108410 0x7f27b410db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:35.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.296+0000 7f27b1e00700 1 -- 172.21.15.33:0/948846216 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27b4102d60 msgr2=0x7f27b4074f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:32:35.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.296+0000 7f27b1e00700 1 --2- 172.21.15.33:0/948846216 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27b4102d60 0x7f27b4074f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:35.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.296+0000 7f27b1e00700 1 -- 172.21.15.33:0/948846216 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27b410e150 con 0x7f27b41023a0 2024-05-03T18:32:35.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.297+0000 7f27b1e00700 1 --2- 172.21.15.33:0/948846216 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27b41023a0 0x7f27b410d620 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f27a40097d0 tx=0x7f27a4009ae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:32:35.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.297+0000 7f27aaa00700 1 -- 172.21.15.33:0/948846216 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27a4013790 con 0x7f27b41023a0 2024-05-03T18:32:35.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.297+0000 7f27aaa00700 1 -- 172.21.15.33:0/948846216 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f27a4019070 con 0x7f27b41023a0 2024-05-03T18:32:35.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.297+0000 7f27aaa00700 1 -- 172.21.15.33:0/948846216 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27a4014d80 con 0x7f27b41023a0 2024-05-03T18:32:35.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.298+0000 7f27b8c00700 1 -- 172.21.15.33:0/948846216 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27b410e440 con 0x7f27b41023a0 2024-05-03T18:32:35.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.298+0000 7f27b8c00700 1 -- 172.21.15.33:0/948846216 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27b41b5480 con 0x7f27b41023a0 2024-05-03T18:32:35.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.302+0000 7f27aaa00700 1 -- 172.21.15.33:0/948846216 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f27a4013930 con 0x7f27b41023a0 2024-05-03T18:32:35.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.304+0000 7f27aaa00700 1 --2- 172.21.15.33:0/948846216 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f27980b2bc0 0x7f27980b5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:32:35.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.304+0000 7f27aaa00700 1 -- 172.21.15.33:0/948846216 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f27a40e20b0 con 0x7f27b41023a0 2024-05-03T18:32:35.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.304+0000 7f27b1400700 1 --2- 172.21.15.33:0/948846216 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f27980b2bc0 0x7f27980b5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:32:35.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.305+0000 7f27b1400700 1 --2- 172.21.15.33:0/948846216 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f27980b2bc0 0x7f27980b5080 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f27a00098c0 tx=0x7f27a0019040 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:32:35.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.308+0000 7f27c34c2b80 1 Processor -- start 2024-05-03T18:32:35.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.308+0000 7f27c34c2b80 1 -- start start 2024-05-03T18:32:35.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.309+0000 7f27c34c2b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3d336f7b0 0x55e3d336fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:32:35.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.309+0000 7f27c34c2b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e3d33700d0 0x55e3d336afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:32:35.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.309+0000 7f27c34c2b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3d336b5d0 0x55e3d336ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:32:35.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.309+0000 7f27c34c2b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e3d32bda20 con 0x55e3d336f7b0 2024-05-03T18:32:35.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.309+0000 7f27c34c2b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55e3d325b760 con 0x55e3d336b5d0 2024-05-03T18:32:35.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.309+0000 7f27c34c2b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55e3d32b99e0 con 0x55e3d33700d0 2024-05-03T18:32:35.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.310+0000 7f27b1400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e3d33700d0 0x55e3d336afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:32:35.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.310+0000 7f27b1400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e3d33700d0 0x55e3d336afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:33690/0 (socket says 172.21.15.33:33690) 2024-05-03T18:32:35.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.310+0000 7f27b1400700 1 -- 172.21.15.33:0/3732861338 learned_addr learned my addr 172.21.15.33:0/3732861338 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:32:35.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.310+0000 7f27b1e00700 1 --2- 172.21.15.33:0/3732861338 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3d336f7b0 0x55e3d336fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:32:35.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.310+0000 7f27b2800700 1 --2- 172.21.15.33:0/3732861338 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3d336b5d0 0x55e3d336ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:32:35.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.312+0000 7f27b1400700 1 -- 172.21.15.33:0/3732861338 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3d336b5d0 msgr2=0x55e3d336ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:32:35.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.312+0000 7f27b1400700 1 --2- 172.21.15.33:0/3732861338 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3d336b5d0 0x55e3d336ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:35.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.312+0000 7f27b1400700 1 -- 172.21.15.33:0/3732861338 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3d336f7b0 msgr2=0x55e3d336fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:32:35.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.312+0000 7f27b1400700 1 --2- 172.21.15.33:0/3732861338 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3d336f7b0 0x55e3d336fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:35.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.312+0000 7f27b1400700 1 -- 172.21.15.33:0/3732861338 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3d320a8d0 con 0x55e3d33700d0 2024-05-03T18:32:35.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.312+0000 7f27b1e00700 1 --2- 172.21.15.33:0/3732861338 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3d336f7b0 0x55e3d336fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:32:35.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.315+0000 7f27b1400700 1 --2- 172.21.15.33:0/3732861338 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e3d33700d0 0x55e3d336afb0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f27a00098f0 tx=0x7f27a001e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b36cb8677fcd09a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:32:35.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.315+0000 7f279f400700 1 -- 172.21.15.33:0/3732861338 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27a000f6e0 con 0x55e3d33700d0 2024-05-03T18:32:35.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.315+0000 7f279f400700 1 -- 172.21.15.33:0/3732861338 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f27a000f880 con 0x55e3d33700d0 2024-05-03T18:32:35.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.316+0000 7f279f400700 1 -- 172.21.15.33:0/3732861338 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27a0003cc0 con 0x55e3d33700d0 2024-05-03T18:32:35.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.316+0000 7f27c34c2b80 1 -- 172.21.15.33:0/3732861338 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e3d33700d0 msgr2=0x55e3d336afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:32:35.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.316+0000 7f27c34c2b80 1 --2- 172.21.15.33:0/3732861338 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e3d33700d0 0x55e3d336afb0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f27a00098f0 tx=0x7f27a001e000 comp rx=0 tx=0).stop 2024-05-03T18:32:35.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.317+0000 7f27c34c2b80 1 -- 172.21.15.33:0/3732861338 shutdown_connections 2024-05-03T18:32:35.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.317+0000 7f27c34c2b80 1 --2- 172.21.15.33:0/3732861338 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3d336b5d0 0x55e3d336ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:35.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.317+0000 7f27c34c2b80 1 --2- 172.21.15.33:0/3732861338 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3d336f7b0 0x55e3d336fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:35.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.317+0000 7f27c34c2b80 1 --2- 172.21.15.33:0/3732861338 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e3d33700d0 0x55e3d336afb0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:35.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.317+0000 7f27c34c2b80 1 -- 172.21.15.33:0/3732861338 >> 172.21.15.33:0/3732861338 conn(0x55e3d3299b60 msgr2=0x55e3d33657f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:32:35.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.317+0000 7f27c34c2b80 1 -- 172.21.15.33:0/3732861338 shutdown_connections 2024-05-03T18:32:35.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.317+0000 7f27c34c2b80 1 -- 172.21.15.33:0/3732861338 wait complete. 2024-05-03T18:32:35.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.318+0000 7f27c34c2b80 1 -- start start 2024-05-03T18:32:35.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.319+0000 7f27c34c2b80 1 Processor -- start 2024-05-03T18:32:35.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.321+0000 7f27c34c2b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e3d336b5d0 0x55e3d3377050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:32:35.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.321+0000 7f27c34c2b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3d336f7b0 0x55e3d3374350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:32:35.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.321+0000 7f27c34c2b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3d3374890 0x55e3d3374d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:32:35.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.321+0000 7f27c34c2b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e3d3377c40 con 0x55e3d336f7b0 2024-05-03T18:32:35.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.321+0000 7f27c34c2b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55e3d3223b20 con 0x55e3d3374890 2024-05-03T18:32:35.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.321+0000 7f27c34c2b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55e3d3377dc0 con 0x55e3d336b5d0 2024-05-03T18:32:35.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.322+0000 7f27b1400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3d336f7b0 0x55e3d3374350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:32:35.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.322+0000 7f27c34c2b80 10 client.0 _ll_register_callbacks cb 0x55e3d3264c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:32:35.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.322+0000 7f27b1400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3d336f7b0 0x55e3d3374350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34690/0 (socket says 172.21.15.33:34690) 2024-05-03T18:32:35.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.322+0000 7f27b1400700 1 -- 172.21.15.33:0/3064063507 learned_addr learned my addr 172.21.15.33:0/3064063507 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:32:35.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.322+0000 7f27b2800700 1 --2- 172.21.15.33:0/3064063507 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3d3374890 0x55e3d3374d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:32:35.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.323+0000 7f27b2800700 1 -- 172.21.15.33:0/3064063507 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e3d336b5d0 msgr2=0x55e3d3377050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:32:35.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.323+0000 7f27b2800700 1 --2- 172.21.15.33:0/3064063507 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e3d336b5d0 0x55e3d3377050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:35.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.323+0000 7f27b2800700 1 -- 172.21.15.33:0/3064063507 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3d336f7b0 msgr2=0x55e3d3374350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:32:35.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.323+0000 7f27b2800700 1 --2- 172.21.15.33:0/3064063507 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3d336f7b0 0x55e3d3374350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:35.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.323+0000 7f27b2800700 1 -- 172.21.15.33:0/3064063507 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3d3283420 con 0x55e3d3374890 2024-05-03T18:32:35.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.324+0000 7f27b2800700 1 --2- 172.21.15.33:0/3064063507 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3d3374890 0x55e3d3374d10 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f27ac0062a0 tx=0x7f27ac014600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16162358b67d1b09 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:32:35.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.324+0000 7f2793e00700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-03T18:32:35.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.324+0000 7f2793e00700 1 -- 172.21.15.33:0/3064063507 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27ac004bb0 con 0x55e3d3374890 2024-05-03T18:32:35.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.324+0000 7f2793e00700 1 -- 172.21.15.33:0/3064063507 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f27ac004d50 con 0x55e3d3374890 2024-05-03T18:32:35.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.325+0000 7f27c34c2b80 1 -- 172.21.15.33:0/3064063507 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27a000f880 con 0x55e3d3374890 2024-05-03T18:32:35.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.325+0000 7f2793e00700 1 -- 172.21.15.33:0/3064063507 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f27ac025050 con 0x55e3d3374890 2024-05-03T18:32:35.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.326+0000 7f27c34c2b80 10 client.35570 fetch_fsmap learned FSMap version 21 2024-05-03T18:32:35.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.326+0000 7f27c34c2b80 1 -- 172.21.15.33:0/3064063507 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({fsmap=21}) -- 0x55e3d3382960 con 0x55e3d3374890 2024-05-03T18:32:35.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.327+0000 7f2793e00700 1 -- 172.21.15.33:0/3064063507 <== mon.2 v2:172.21.15.99:3300/0 4 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f27ac020470 con 0x55e3d3374890 2024-05-03T18:32:35.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.327+0000 7f27c34c2b80 10 client.35570 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:32:35.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.327+0000 7f27c34c2b80 10 client.35570 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:32:35.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.327+0000 7f27c34c2b80 20 client.35570 populate_metadata read hostname 'smithi033' 2024-05-03T18:32:35.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.327+0000 7f27c34c2b80 1 --2- 172.21.15.33:0/3064063507 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55e3d3384e90 0x55e3d33872d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:32:35.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.327+0000 7f27c34c2b80 4 client.35570 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:32:35.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.327+0000 7f27c34c2b80 1 -- 172.21.15.33:0/3064063507 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e3d33381d0 con 0x55e3d3384e90 2024-05-03T18:32:35.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.328+0000 7f27b1e00700 1 --2- 172.21.15.33:0/3064063507 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55e3d3384e90 0x55e3d33872d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:32:35.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.328+0000 7f2793e00700 10 client.35570 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:32:35.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.328+0000 7f27b1e00700 1 --2- 172.21.15.33:0/3064063507 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55e3d3384e90 0x55e3d33872d0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f763abb983c96dd server_cookie=dd3502479180f435 in_seq=0 out_seq=0 2024-05-03T18:32:35.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.330+0000 7f2793e00700 1 -- 172.21.15.33:0/3064063507 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e3d33381d0 con 0x55e3d3384e90 2024-05-03T18:32:35.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.330+0000 7f2793e00700 10 client.35570 handle_command_reply: tid=0 2024-05-03T18:32:35.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.330+0000 7f27c34c2b80 1 client.35570 shutdown 2024-05-03T18:32:35.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.331+0000 7f27c34c2b80 1 -- 172.21.15.33:0/3064063507 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3d3374890 msgr2=0x55e3d3374d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:32:35.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.331+0000 7f27c34c2b80 1 --2- 172.21.15.33:0/3064063507 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3d3374890 0x55e3d3374d10 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f27ac0062a0 tx=0x7f27ac014600 comp rx=0 tx=0).stop 2024-05-03T18:32:35.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.331+0000 7f27c34c2b80 1 -- 172.21.15.33:0/3064063507 shutdown_connections 2024-05-03T18:32:35.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.331+0000 7f27c34c2b80 1 --2- 172.21.15.33:0/3064063507 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55e3d3384e90 0x55e3d33872d0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:35.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.331+0000 7f27c34c2b80 1 --2- 172.21.15.33:0/3064063507 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3d3374890 0x55e3d3374d10 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:35.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.331+0000 7f2793e00700 0 client.35570 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:32:35.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.331+0000 7f27c34c2b80 1 --2- 172.21.15.33:0/3064063507 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e3d336b5d0 0x55e3d3377050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:35.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.331+0000 7f27c34c2b80 1 --2- 172.21.15.33:0/3064063507 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3d336f7b0 0x55e3d3374350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:35.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.331+0000 7f27c34c2b80 1 -- 172.21.15.33:0/3064063507 >> 172.21.15.33:0/3064063507 conn(0x55e3d3299b60 msgr2=0x55e3d3370e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:32:35.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.332+0000 7f27c34c2b80 1 -- 172.21.15.33:0/3064063507 shutdown_connections 2024-05-03T18:32:35.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.333+0000 7f27c34c2b80 1 -- 172.21.15.33:0/3064063507 wait complete. 2024-05-03T18:32:35.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.333+0000 7f27c34c2b80 20 client.35570 trim_cache size 0 max 16384 2024-05-03T18:32:35.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.367+0000 7f27c34c2b80 1 Processor -- start 2024-05-03T18:32:35.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.367+0000 7f27c34c2b80 1 -- start start 2024-05-03T18:32:35.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.367+0000 7f27c34c2b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3d336b5d0 0x55e3d33657c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:32:35.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.367+0000 7f27c34c2b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e3d336f7b0 0x55e3d338cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:32:35.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.367+0000 7f27c34c2b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3d3374890 0x55e3d338d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:32:35.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.367+0000 7f27c34c2b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f27b41bc0d0 con 0x55e3d3374890 2024-05-03T18:32:35.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.367+0000 7f27c34c2b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f27b41be950 con 0x55e3d336b5d0 2024-05-03T18:32:35.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.367+0000 7f27c34c2b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f27b41bc740 con 0x55e3d336f7b0 2024-05-03T18:32:35.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.368+0000 7f27b1400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e3d336f7b0 0x55e3d338cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:32:35.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.368+0000 7f27b1400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e3d336f7b0 0x55e3d338cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:33696/0 (socket says 172.21.15.33:33696) 2024-05-03T18:32:35.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.368+0000 7f27b1400700 1 -- 172.21.15.33:0/1467223982 learned_addr learned my addr 172.21.15.33:0/1467223982 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:32:35.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.368+0000 7f27b1e00700 1 --2- 172.21.15.33:0/1467223982 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3d336b5d0 0x55e3d33657c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:32:35.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.369+0000 7f27b1400700 1 -- 172.21.15.33:0/1467223982 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3d336b5d0 msgr2=0x55e3d33657c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:32:35.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.369+0000 7f27b1400700 1 --2- 172.21.15.33:0/1467223982 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3d336b5d0 0x55e3d33657c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:35.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.369+0000 7f27b1400700 1 -- 172.21.15.33:0/1467223982 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3d3374890 msgr2=0x55e3d338d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:32:35.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.369+0000 7f27b1400700 1 --2- 172.21.15.33:0/1467223982 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3d3374890 0x55e3d338d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:35.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.369+0000 7f27b1400700 1 -- 172.21.15.33:0/1467223982 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3d3365d00 con 0x55e3d336f7b0 2024-05-03T18:32:35.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.370+0000 7f27b1400700 1 --2- 172.21.15.33:0/1467223982 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e3d336f7b0 0x55e3d338cba0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f27a00098f0 tx=0x7f27a000f670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7dca03163bfb3147 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:32:35.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.370+0000 7f279ea00700 1 -- 172.21.15.33:0/1467223982 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27a00050d0 con 0x55e3d336f7b0 2024-05-03T18:32:35.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.371+0000 7f279ea00700 1 -- 172.21.15.33:0/1467223982 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f27a0003680 con 0x55e3d336f7b0 2024-05-03T18:32:35.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.371+0000 7f279ea00700 1 -- 172.21.15.33:0/1467223982 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27a000b7c0 con 0x55e3d336f7b0 2024-05-03T18:32:35.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.372+0000 7f27c34c2b80 1 -- 172.21.15.33:0/1467223982 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e3d336f7b0 msgr2=0x55e3d338cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:32:35.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.372+0000 7f27c34c2b80 1 --2- 172.21.15.33:0/1467223982 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e3d336f7b0 0x55e3d338cba0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f27a00098f0 tx=0x7f27a000f670 comp rx=0 tx=0).stop 2024-05-03T18:32:35.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.372+0000 7f27c34c2b80 1 -- 172.21.15.33:0/1467223982 shutdown_connections 2024-05-03T18:32:35.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.372+0000 7f27c34c2b80 1 --2- 172.21.15.33:0/1467223982 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3d3374890 0x55e3d338d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:35.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.372+0000 7f27c34c2b80 1 --2- 172.21.15.33:0/1467223982 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3d336b5d0 0x55e3d33657c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:35.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.372+0000 7f27c34c2b80 1 --2- 172.21.15.33:0/1467223982 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e3d336f7b0 0x55e3d338cba0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:35.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.372+0000 7f27c34c2b80 1 -- 172.21.15.33:0/1467223982 >> 172.21.15.33:0/1467223982 conn(0x55e3d3299b60 msgr2=0x55e3d33650b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:32:35.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.372+0000 7f27c34c2b80 1 -- 172.21.15.33:0/1467223982 shutdown_connections 2024-05-03T18:32:35.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.372+0000 7f27c34c2b80 1 -- 172.21.15.33:0/1467223982 wait complete. 2024-05-03T18:32:35.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.373+0000 7f27c34c2b80 1 -- start start 2024-05-03T18:32:35.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.373+0000 7f27c34c2b80 1 Processor -- start 2024-05-03T18:32:35.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.375+0000 7f27c34c2b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3d336b5d0 0x55e3d3368e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:32:35.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.375+0000 7f27c34c2b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3d336f7b0 0x55e3d3369350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:32:35.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.375+0000 7f27c34c2b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e3d3374890 0x55e3d3369890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:32:35.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.375+0000 7f27c34c2b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f27b41bce10 con 0x55e3d336b5d0 2024-05-03T18:32:35.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.375+0000 7f27c34c2b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f27b41be280 con 0x55e3d336f7b0 2024-05-03T18:32:35.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.375+0000 7f27c34c2b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f27b41bd4e0 con 0x55e3d3374890 2024-05-03T18:32:35.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.375+0000 7f27c34c2b80 10 client.0 _ll_register_callbacks cb 0x55e3d3264c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:32:35.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.376+0000 7f27b2800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e3d3374890 0x55e3d3369890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:32:35.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.376+0000 7f27b2800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e3d3374890 0x55e3d3369890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:33698/0 (socket says 172.21.15.33:33698) 2024-05-03T18:32:35.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.376+0000 7f27b2800700 1 -- 172.21.15.33:0/2304067684 learned_addr learned my addr 172.21.15.33:0/2304067684 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:32:35.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.376+0000 7f27b1400700 1 --2- 172.21.15.33:0/2304067684 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3d336f7b0 0x55e3d3369350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:32:35.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.376+0000 7f27b1e00700 1 --2- 172.21.15.33:0/2304067684 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3d336b5d0 0x55e3d3368e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:32:35.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.376+0000 7f27b2800700 1 -- 172.21.15.33:0/2304067684 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3d336f7b0 msgr2=0x55e3d3369350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:32:35.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.376+0000 7f27b2800700 1 --2- 172.21.15.33:0/2304067684 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3d336f7b0 0x55e3d3369350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:35.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.376+0000 7f27b2800700 1 -- 172.21.15.33:0/2304067684 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3d336b5d0 msgr2=0x55e3d3368e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:32:35.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.377+0000 7f27b2800700 1 --2- 172.21.15.33:0/2304067684 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3d336b5d0 0x55e3d3368e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:35.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.377+0000 7f27b2800700 1 -- 172.21.15.33:0/2304067684 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3d337b8b0 con 0x55e3d3374890 2024-05-03T18:32:35.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.377+0000 7f27b1e00700 1 --2- 172.21.15.33:0/2304067684 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3d336b5d0 0x55e3d3368e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:32:35.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.377+0000 7f27b2800700 1 --2- 172.21.15.33:0/2304067684 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e3d3374890 0x55e3d3369890 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f27ac002230 tx=0x7f27ac004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=804e3f3820fa3d57 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:32:35.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.377+0000 7f2793e00700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-03T18:32:35.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.379+0000 7f2793e00700 1 -- 172.21.15.33:0/2304067684 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27ac014e80 con 0x55e3d3374890 2024-05-03T18:32:35.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.379+0000 7f2793e00700 1 -- 172.21.15.33:0/2304067684 <== mon.1 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f27ac005c60 con 0x55e3d3374890 2024-05-03T18:32:35.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.379+0000 7f27c34c2b80 1 -- 172.21.15.33:0/2304067684 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27a0003680 con 0x55e3d3374890 2024-05-03T18:32:35.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.380+0000 7f2793e00700 1 -- 172.21.15.33:0/2304067684 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f27ac005e60 con 0x55e3d3374890 2024-05-03T18:32:35.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.380+0000 7f27c34c2b80 10 client.25633 fetch_fsmap learned FSMap version 21 2024-05-03T18:32:35.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.380+0000 7f27c34c2b80 1 -- 172.21.15.33:0/2304067684 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55e3d33efcc0 con 0x55e3d3374890 2024-05-03T18:32:35.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.381+0000 7f2793e00700 1 -- 172.21.15.33:0/2304067684 <== mon.1 v2:172.21.15.150:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f27a0003680 con 0x55e3d3374890 2024-05-03T18:32:35.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:32:34 smithi150 ceph-mon[32492]: pgmap v164: 129 pgs: 129 active+clean; 1.7 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 7.9 MiB/s wr, 907 op/s 2024-05-03T18:32:35.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.381+0000 7f2793e00700 1 -- 172.21.15.33:0/2304067684 <== mon.1 v2:172.21.15.150:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f27ac03cc90 con 0x55e3d3374890 2024-05-03T18:32:35.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.381+0000 7f27c34c2b80 10 client.25633 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:32:35.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.381+0000 7f27c34c2b80 10 client.25633 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:32:35.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.381+0000 7f27c34c2b80 20 client.25633 populate_metadata read hostname 'smithi033' 2024-05-03T18:32:35.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.381+0000 7f27c34c2b80 1 --2- 172.21.15.33:0/2304067684 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55e3d33f0370 0x55e3d33f2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:32:35.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.381+0000 7f27c34c2b80 4 client.25633 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-03T18:32:35.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.381+0000 7f27c34c2b80 1 -- 172.21.15.33:0/2304067684 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e3d32cb400 con 0x55e3d33f0370 2024-05-03T18:32:35.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.382+0000 7f27b1e00700 1 --2- 172.21.15.33:0/2304067684 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55e3d33f0370 0x55e3d33f2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:32:35.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.383+0000 7f27b1e00700 1 --2- 172.21.15.33:0/2304067684 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55e3d33f0370 0x55e3d33f2760 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffbea9f77e1aa082 server_cookie=8baadd7b4ebc515c in_seq=0 out_seq=0 2024-05-03T18:32:35.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.383+0000 7f2793e00700 10 client.25633 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:32:35.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.385+0000 7f2793e00700 1 -- 172.21.15.33:0/2304067684 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e3d32cb400 con 0x55e3d33f0370 2024-05-03T18:32:35.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.385+0000 7f2793e00700 10 client.25633 handle_command_reply: tid=0 2024-05-03T18:32:35.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.385+0000 7f27c34c2b80 1 client.25633 shutdown 2024-05-03T18:32:35.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.385+0000 7f27c34c2b80 1 -- 172.21.15.33:0/2304067684 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e3d3374890 msgr2=0x55e3d3369890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:32:35.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.385+0000 7f27c34c2b80 1 --2- 172.21.15.33:0/2304067684 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e3d3374890 0x55e3d3369890 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f27ac002230 tx=0x7f27ac004ad0 comp rx=0 tx=0).stop 2024-05-03T18:32:35.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.385+0000 7f27c34c2b80 1 -- 172.21.15.33:0/2304067684 shutdown_connections 2024-05-03T18:32:35.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.385+0000 7f27c34c2b80 1 --2- 172.21.15.33:0/2304067684 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55e3d33f0370 0x55e3d33f2760 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:35.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.385+0000 7f27c34c2b80 1 --2- 172.21.15.33:0/2304067684 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e3d3374890 0x55e3d3369890 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:35.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.385+0000 7f27c34c2b80 1 --2- 172.21.15.33:0/2304067684 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3d336b5d0 0x55e3d3368e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:35.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.385+0000 7f27c34c2b80 1 --2- 172.21.15.33:0/2304067684 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3d336f7b0 0x55e3d3369350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:35.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.385+0000 7f27c34c2b80 1 -- 172.21.15.33:0/2304067684 >> 172.21.15.33:0/2304067684 conn(0x55e3d3299b60 msgr2=0x55e3d336a410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:32:35.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.385+0000 7f2793e00700 0 client.25633 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:32:35.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.386+0000 7f27c34c2b80 1 -- 172.21.15.33:0/2304067684 shutdown_connections 2024-05-03T18:32:35.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.386+0000 7f27c34c2b80 1 -- 172.21.15.33:0/2304067684 wait complete. 2024-05-03T18:32:35.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.386+0000 7f27c34c2b80 20 client.25633 trim_cache size 0 max 16384 2024-05-03T18:32:35.388 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-03T18:32:35.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.387+0000 7f27b8c00700 1 -- 172.21.15.33:0/948846216 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f27980b2bc0 msgr2=0x7f27980b5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:32:35.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.387+0000 7f27b8c00700 1 --2- 172.21.15.33:0/948846216 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f27980b2bc0 0x7f27980b5080 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f27a00098c0 tx=0x7f27a0019040 comp rx=0 tx=0).stop 2024-05-03T18:32:35.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.387+0000 7f27b8c00700 1 -- 172.21.15.33:0/948846216 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27b41023a0 msgr2=0x7f27b410d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:32:35.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.387+0000 7f27b8c00700 1 --2- 172.21.15.33:0/948846216 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27b41023a0 0x7f27b410d620 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f27a40097d0 tx=0x7f27a4009ae0 comp rx=0 tx=0).stop 2024-05-03T18:32:35.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.388+0000 7f27b8c00700 1 -- 172.21.15.33:0/948846216 shutdown_connections 2024-05-03T18:32:35.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.388+0000 7f27b8c00700 1 --2- 172.21.15.33:0/948846216 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f27b4108410 0x7f27b410db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:35.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.388+0000 7f27b8c00700 1 --2- 172.21.15.33:0/948846216 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f27980b2bc0 0x7f27980b5080 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:35.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.388+0000 7f27b8c00700 1 --2- 172.21.15.33:0/948846216 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27b41023a0 0x7f27b410d620 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:35.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.388+0000 7f27b8c00700 1 --2- 172.21.15.33:0/948846216 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27b4102d60 0x7f27b4074f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:35.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.388+0000 7f27b8c00700 1 -- 172.21.15.33:0/948846216 >> 172.21.15.33:0/948846216 conn(0x7f27b4079350 msgr2=0x7f27b4101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:32:35.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.388+0000 7f27b8c00700 1 -- 172.21.15.33:0/948846216 shutdown_connections 2024-05-03T18:32:35.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:35.388+0000 7f27b8c00700 1 -- 172.21.15.33:0/948846216 wait complete. 2024-05-03T18:32:37.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:32:36 smithi033 ceph-mon[28330]: pgmap v165: 129 pgs: 129 active+clean; 1.7 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 5.5 MiB/s wr, 621 op/s 2024-05-03T18:32:37.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:32:36 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:32:37.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:32:36 smithi099 ceph-mon[31658]: pgmap v165: 129 pgs: 129 active+clean; 1.7 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 5.5 MiB/s wr, 621 op/s 2024-05-03T18:32:37.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:32:36 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:32:37.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:32:36 smithi150 ceph-mon[32492]: pgmap v165: 129 pgs: 129 active+clean; 1.7 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 5.5 MiB/s wr, 621 op/s 2024-05-03T18:32:37.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:32:36 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:32:38.153 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-03T18:32:38.154 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-03T18:32:38.872 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:32:39.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:32:38 smithi099 ceph-mon[31658]: pgmap v166: 129 pgs: 129 active+clean; 1.7 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 8.2 MiB/s wr, 922 op/s 2024-05-03T18:32:39.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:32:38 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:32:39.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:32:38 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:32:39.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:32:38 smithi150 ceph-mon[32492]: pgmap v166: 129 pgs: 129 active+clean; 1.7 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 8.2 MiB/s wr, 922 op/s 2024-05-03T18:32:39.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:32:38 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:32:39.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:32:38 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:32:39.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:32:38 smithi033 ceph-mon[28330]: pgmap v166: 129 pgs: 129 active+clean; 1.7 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 8.2 MiB/s wr, 922 op/s 2024-05-03T18:32:39.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:32:38 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:32:39.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:32:38 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:32:40.110 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 85.0 s, 632.9 tps, lat 158.328 ms stddev 48.652 2024-05-03T18:32:41.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:32:40 smithi099 ceph-mon[31658]: pgmap v167: 129 pgs: 129 active+clean; 1.7 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.7 MiB/s wr, 625 op/s 2024-05-03T18:32:41.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:32:40 smithi150 ceph-mon[32492]: pgmap v167: 129 pgs: 129 active+clean; 1.7 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.7 MiB/s wr, 625 op/s 2024-05-03T18:32:41.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:32:40 smithi033 ceph-mon[28330]: pgmap v167: 129 pgs: 129 active+clean; 1.7 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.7 MiB/s wr, 625 op/s 2024-05-03T18:32:42.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:32:41 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:32:42.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:32:41 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:32:42.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:32:41 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:32:42.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:32:41 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:32:42.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:32:41 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:32:42.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:32:41 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:32:42.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:32:41 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:32:42.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:32:41 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:32:42.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:32:41 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:32:43.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.200+0000 7f8a20600700 1 -- 172.21.15.33:0/3897179071 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a18108410 msgr2=0x7f8a181087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:32:43.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.200+0000 7f8a20600700 1 --2- 172.21.15.33:0/3897179071 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a18108410 0x7f8a181087f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c00d960 tx=0x7f8a0c00dc70 comp rx=0 tx=0).stop 2024-05-03T18:32:43.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.200+0000 7f8a20600700 1 -- 172.21.15.33:0/3897179071 shutdown_connections 2024-05-03T18:32:43.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.200+0000 7f8a20600700 1 --2- 172.21.15.33:0/3897179071 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a18102d60 0x7f8a1810f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:43.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.200+0000 7f8a20600700 1 --2- 172.21.15.33:0/3897179071 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a18108410 0x7f8a181087f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:43.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.200+0000 7f8a20600700 1 --2- 172.21.15.33:0/3897179071 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a181023a0 0x7f8a18102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:43.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.200+0000 7f8a20600700 1 -- 172.21.15.33:0/3897179071 >> 172.21.15.33:0/3897179071 conn(0x7f8a18079350 msgr2=0x7f8a18079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:32:43.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.201+0000 7f8a20600700 1 -- 172.21.15.33:0/3897179071 shutdown_connections 2024-05-03T18:32:43.203 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:32:42 smithi033 ceph-mon[28330]: pgmap v168: 129 pgs: 129 active+clean; 1.7 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.7 MiB/s wr, 625 op/s 2024-05-03T18:32:43.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.202+0000 7f8a20600700 1 -- 172.21.15.33:0/3897179071 wait complete. 2024-05-03T18:32:43.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.203+0000 7f8a20600700 1 Processor -- start 2024-05-03T18:32:43.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.204+0000 7f8a20600700 1 -- start start 2024-05-03T18:32:43.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.204+0000 7f8a20600700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a181023a0 0x7f8a18074d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:32:43.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.204+0000 7f8a20600700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a18102d60 0x7f8a1806de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:32:43.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.204+0000 7f8a20600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a18108410 0x7f8a1806e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:32:43.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.204+0000 7f8a20600700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8a18111340 con 0x7f8a18102d60 2024-05-03T18:32:43.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.204+0000 7f8a20600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8a181111c0 con 0x7f8a18108410 2024-05-03T18:32:43.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.204+0000 7f8a20600700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8a181114c0 con 0x7f8a181023a0 2024-05-03T18:32:43.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.205+0000 7f8a1dc00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a181023a0 0x7f8a18074d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:32:43.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.205+0000 7f8a1dc00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a181023a0 0x7f8a18074d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:53478/0 (socket says 172.21.15.33:53478) 2024-05-03T18:32:43.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.205+0000 7f8a1dc00700 1 -- 172.21.15.33:0/2755300410 learned_addr learned my addr 172.21.15.33:0/2755300410 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:32:43.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.205+0000 7f8a1e600700 1 --2- 172.21.15.33:0/2755300410 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a18108410 0x7f8a1806e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:32:43.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.205+0000 7f8a1dc00700 1 -- 172.21.15.33:0/2755300410 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a18108410 msgr2=0x7f8a1806e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:32:43.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.205+0000 7f8a1dc00700 1 --2- 172.21.15.33:0/2755300410 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a18108410 0x7f8a1806e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:43.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.206+0000 7f8a1dc00700 1 -- 172.21.15.33:0/2755300410 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a18102d60 msgr2=0x7f8a1806de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:32:43.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.206+0000 7f8a1dc00700 1 --2- 172.21.15.33:0/2755300410 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a18102d60 0x7f8a1806de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:43.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.206+0000 7f8a1dc00700 1 -- 172.21.15.33:0/2755300410 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a1806ea80 con 0x7f8a181023a0 2024-05-03T18:32:43.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.206+0000 7f8a1dc00700 1 --2- 172.21.15.33:0/2755300410 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a181023a0 0x7f8a18074d80 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c005300 tx=0x7f8a0c005330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:32:43.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.207+0000 7f8a16000700 1 -- 172.21.15.33:0/2755300410 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a0c01a070 con 0x7f8a181023a0 2024-05-03T18:32:43.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.207+0000 7f8a20600700 1 -- 172.21.15.33:0/2755300410 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a180728b0 con 0x7f8a181023a0 2024-05-03T18:32:43.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.207+0000 7f8a20600700 1 -- 172.21.15.33:0/2755300410 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a18072de0 con 0x7f8a181023a0 2024-05-03T18:32:43.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.207+0000 7f8a16000700 1 -- 172.21.15.33:0/2755300410 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8a0c00c330 con 0x7f8a181023a0 2024-05-03T18:32:43.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.207+0000 7f8a16000700 1 -- 172.21.15.33:0/2755300410 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a0c003970 con 0x7f8a181023a0 2024-05-03T18:32:43.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.214+0000 7f8a16000700 1 -- 172.21.15.33:0/2755300410 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f8a0c00c040 con 0x7f8a181023a0 2024-05-03T18:32:43.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.216+0000 7f8a16000700 1 --2- 172.21.15.33:0/2755300410 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f8a000b6e80 0x7f8a000b9340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:32:43.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.217+0000 7f8a16000700 1 -- 172.21.15.33:0/2755300410 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f8a0c0e30d0 con 0x7f8a181023a0 2024-05-03T18:32:43.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.217+0000 7f8a1d200700 1 --2- 172.21.15.33:0/2755300410 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f8a000b6e80 0x7f8a000b9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:32:43.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.220+0000 7f8a1d200700 1 --2- 172.21.15.33:0/2755300410 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f8a000b6e80 0x7f8a000b9340 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8a04005e50 tx=0x7f8a04005da0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:32:43.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.224+0000 7f8a2afedb80 1 Processor -- start 2024-05-03T18:32:43.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.224+0000 7f8a2afedb80 1 -- start start 2024-05-03T18:32:43.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.225+0000 7f8a2afedb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b0ef2521a0 0x55b0ef252580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:32:43.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.226+0000 7f8a2afedb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b0ef252ac0 0x55b0ef24c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:32:43.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.226+0000 7f8a2afedb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0ef24cde0 0x55b0ef24d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:32:43.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.226+0000 7f8a2afedb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55b0ef196aa0 con 0x55b0ef24cde0 2024-05-03T18:32:43.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.226+0000 7f8a2afedb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55b0ef1347e0 con 0x55b0ef252ac0 2024-05-03T18:32:43.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.226+0000 7f8a2afedb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55b0ef192a60 con 0x55b0ef2521a0 2024-05-03T18:32:43.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.226+0000 7f8a1d200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b0ef252ac0 0x55b0ef24c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:32:43.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.226+0000 7f8a1dc00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b0ef2521a0 0x55b0ef252580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:32:43.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.226+0000 7f8a1d200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b0ef252ac0 0x55b0ef24c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:35900/0 (socket says 172.21.15.33:35900) 2024-05-03T18:32:43.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.226+0000 7f8a1d200700 1 -- 172.21.15.33:0/3577349012 learned_addr learned my addr 172.21.15.33:0/3577349012 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:32:43.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.228+0000 7f8a1dc00700 1 -- 172.21.15.33:0/3577349012 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b0ef252ac0 msgr2=0x55b0ef24c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:32:43.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.228+0000 7f8a1dc00700 1 --2- 172.21.15.33:0/3577349012 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b0ef252ac0 0x55b0ef24c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:43.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.228+0000 7f8a1dc00700 1 -- 172.21.15.33:0/3577349012 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0ef24cde0 msgr2=0x55b0ef24d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:32:43.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.228+0000 7f8a1dc00700 1 --2- 172.21.15.33:0/3577349012 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0ef24cde0 0x55b0ef24d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:43.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.228+0000 7f8a1dc00700 1 -- 172.21.15.33:0/3577349012 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0ef0e3950 con 0x55b0ef2521a0 2024-05-03T18:32:43.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.229+0000 7f8a1dc00700 1 --2- 172.21.15.33:0/3577349012 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b0ef2521a0 0x55b0ef252580 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c0e0490 tx=0x7f8a0c009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d95c4a2b0eaa01e server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:32:43.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.230+0000 7f8a0aa00700 1 -- 172.21.15.33:0/3577349012 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a0c0a9bc0 con 0x55b0ef2521a0 2024-05-03T18:32:43.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.230+0000 7f8a0aa00700 1 -- 172.21.15.33:0/3577349012 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8a0c0aa7b0 con 0x55b0ef2521a0 2024-05-03T18:32:43.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.230+0000 7f8a0aa00700 1 -- 172.21.15.33:0/3577349012 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a0c0e0de0 con 0x55b0ef2521a0 2024-05-03T18:32:43.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.231+0000 7f8a2afedb80 1 -- 172.21.15.33:0/3577349012 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b0ef2521a0 msgr2=0x55b0ef252580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:32:43.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.231+0000 7f8a2afedb80 1 --2- 172.21.15.33:0/3577349012 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b0ef2521a0 0x55b0ef252580 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c0e0490 tx=0x7f8a0c009260 comp rx=0 tx=0).stop 2024-05-03T18:32:43.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.231+0000 7f8a2afedb80 1 -- 172.21.15.33:0/3577349012 shutdown_connections 2024-05-03T18:32:43.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.231+0000 7f8a2afedb80 1 --2- 172.21.15.33:0/3577349012 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0ef24cde0 0x55b0ef24d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:43.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.231+0000 7f8a2afedb80 1 --2- 172.21.15.33:0/3577349012 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b0ef2521a0 0x55b0ef252580 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:43.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.231+0000 7f8a2afedb80 1 --2- 172.21.15.33:0/3577349012 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b0ef252ac0 0x55b0ef24c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:43.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.231+0000 7f8a2afedb80 1 -- 172.21.15.33:0/3577349012 >> 172.21.15.33:0/3577349012 conn(0x55b0ef1bc750 msgr2=0x55b0ef23e980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:32:43.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.232+0000 7f8a2afedb80 1 -- 172.21.15.33:0/3577349012 shutdown_connections 2024-05-03T18:32:43.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.232+0000 7f8a2afedb80 1 -- 172.21.15.33:0/3577349012 wait complete. 2024-05-03T18:32:43.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.233+0000 7f8a2afedb80 1 -- start start 2024-05-03T18:32:43.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.233+0000 7f8a2afedb80 1 Processor -- start 2024-05-03T18:32:43.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.234+0000 7f8a2afedb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b0ef24cde0 0x55b0ef2587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:32:43.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.234+0000 7f8a2afedb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0ef2521a0 0x55b0ef255b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:32:43.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.234+0000 7f8a2afedb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b0ef252ac0 0x55b0ef2560b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:32:43.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.234+0000 7f8a2afedb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55b0ef192a60 con 0x55b0ef2521a0 2024-05-03T18:32:43.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.234+0000 7f8a2afedb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55b0ef0fcba0 con 0x55b0ef24cde0 2024-05-03T18:32:43.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.234+0000 7f8a2afedb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55b0ef259380 con 0x55b0ef252ac0 2024-05-03T18:32:43.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.234+0000 7f8a2afedb80 10 client.0 _ll_register_callbacks cb 0x55b0ef13dcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:32:43.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.235+0000 7f8a1dc00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b0ef24cde0 0x55b0ef2587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:32:43.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.235+0000 7f8a1d200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0ef2521a0 0x55b0ef255b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:32:43.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.235+0000 7f8a1d200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0ef2521a0 0x55b0ef255b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60204/0 (socket says 172.21.15.33:60204) 2024-05-03T18:32:43.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.235+0000 7f8a1d200700 1 -- 172.21.15.33:0/3831306868 learned_addr learned my addr 172.21.15.33:0/3831306868 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:32:43.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.236+0000 7f8a1dc00700 1 -- 172.21.15.33:0/3831306868 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b0ef252ac0 msgr2=0x55b0ef2560b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:32:43.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.236+0000 7f8a1dc00700 1 --2- 172.21.15.33:0/3831306868 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b0ef252ac0 0x55b0ef2560b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:43.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.236+0000 7f8a1dc00700 1 -- 172.21.15.33:0/3831306868 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0ef2521a0 msgr2=0x55b0ef255b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:32:43.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.236+0000 7f8a1dc00700 1 --2- 172.21.15.33:0/3831306868 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0ef2521a0 0x55b0ef255b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:43.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.236+0000 7f8a1dc00700 1 -- 172.21.15.33:0/3831306868 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0ef15c4a0 con 0x55b0ef24cde0 2024-05-03T18:32:43.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.237+0000 7f8a1dc00700 1 --2- 172.21.15.33:0/3831306868 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b0ef24cde0 0x55b0ef2587d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c0a9d60 tx=0x7f8a0c0e0e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9203d8dbad176344 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:32:43.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.237+0000 7f89ff400700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-03T18:32:43.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.237+0000 7f89ff400700 1 -- 172.21.15.33:0/3831306868 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a0c0c3070 con 0x55b0ef24cde0 2024-05-03T18:32:43.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.237+0000 7f89ff400700 1 -- 172.21.15.33:0/3831306868 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f8a0c0aa520 con 0x55b0ef24cde0 2024-05-03T18:32:43.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.237+0000 7f8a2afedb80 1 -- 172.21.15.33:0/3831306868 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a0c0aa7b0 con 0x55b0ef24cde0 2024-05-03T18:32:43.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.238+0000 7f89ff400700 1 -- 172.21.15.33:0/3831306868 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f8a0c0aa0b0 con 0x55b0ef24cde0 2024-05-03T18:32:43.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.238+0000 7f8a2afedb80 10 client.35588 fetch_fsmap learned FSMap version 21 2024-05-03T18:32:43.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.238+0000 7f8a2afedb80 1 -- 172.21.15.33:0/3831306868 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55b0ef263930 con 0x55b0ef24cde0 2024-05-03T18:32:43.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.238+0000 7f89ff400700 1 -- 172.21.15.33:0/3831306868 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8a0c0aa7b0 con 0x55b0ef24cde0 2024-05-03T18:32:43.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.239+0000 7f89ff400700 1 -- 172.21.15.33:0/3831306868 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f8a0c003b10 con 0x55b0ef24cde0 2024-05-03T18:32:43.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.239+0000 7f8a2afedb80 10 client.35588 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:32:43.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.239+0000 7f8a2afedb80 10 client.35588 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:32:43.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.239+0000 7f8a2afedb80 20 client.35588 populate_metadata read hostname 'smithi033' 2024-05-03T18:32:43.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.239+0000 7f8a2afedb80 1 --2- 172.21.15.33:0/3831306868 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55b0ef265e00 0x55b0ef268240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:32:43.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.239+0000 7f8a2afedb80 4 client.35588 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:32:43.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.239+0000 7f8a2afedb80 1 -- 172.21.15.33:0/3831306868 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b0ef211070 con 0x55b0ef265e00 2024-05-03T18:32:43.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.240+0000 7f8a1d200700 1 --2- 172.21.15.33:0/3831306868 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55b0ef265e00 0x55b0ef268240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:32:43.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.240+0000 7f8a1d200700 1 --2- 172.21.15.33:0/3831306868 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55b0ef265e00 0x55b0ef268240 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ec3ed15c6f68439 server_cookie=5f9ccf3a61603f15 in_seq=0 out_seq=0 2024-05-03T18:32:43.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.240+0000 7f89ff400700 10 client.35588 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:32:43.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.241+0000 7f89ff400700 1 -- 172.21.15.33:0/3831306868 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b0ef211070 con 0x55b0ef265e00 2024-05-03T18:32:43.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.241+0000 7f89ff400700 10 client.35588 handle_command_reply: tid=0 2024-05-03T18:32:43.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.241+0000 7f8a2afedb80 1 client.35588 shutdown 2024-05-03T18:32:43.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.242+0000 7f8a2afedb80 1 -- 172.21.15.33:0/3831306868 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b0ef24cde0 msgr2=0x55b0ef2587d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:32:43.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.242+0000 7f8a2afedb80 1 --2- 172.21.15.33:0/3831306868 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b0ef24cde0 0x55b0ef2587d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c0a9d60 tx=0x7f8a0c0e0e90 comp rx=0 tx=0).stop 2024-05-03T18:32:43.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.242+0000 7f8a2afedb80 1 -- 172.21.15.33:0/3831306868 shutdown_connections 2024-05-03T18:32:43.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.242+0000 7f8a2afedb80 1 --2- 172.21.15.33:0/3831306868 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55b0ef265e00 0x55b0ef268240 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:43.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.242+0000 7f8a2afedb80 1 --2- 172.21.15.33:0/3831306868 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b0ef252ac0 0x55b0ef2560b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:43.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.242+0000 7f8a2afedb80 1 --2- 172.21.15.33:0/3831306868 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b0ef24cde0 0x55b0ef2587d0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:43.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.242+0000 7f8a2afedb80 1 --2- 172.21.15.33:0/3831306868 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0ef2521a0 0x55b0ef255b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:43.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.242+0000 7f8a2afedb80 1 -- 172.21.15.33:0/3831306868 >> 172.21.15.33:0/3831306868 conn(0x55b0ef1bc750 msgr2=0x55b0ef239430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:32:43.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.242+0000 7f89ff400700 0 client.35588 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:32:43.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.243+0000 7f8a2afedb80 1 -- 172.21.15.33:0/3831306868 shutdown_connections 2024-05-03T18:32:43.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.243+0000 7f8a2afedb80 1 -- 172.21.15.33:0/3831306868 wait complete. 2024-05-03T18:32:43.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.243+0000 7f8a2afedb80 20 client.35588 trim_cache size 0 max 16384 2024-05-03T18:32:43.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.279+0000 7f8a2afedb80 1 Processor -- start 2024-05-03T18:32:43.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.279+0000 7f8a2afedb80 1 -- start start 2024-05-03T18:32:43.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.280+0000 7f8a2afedb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b0ef24cde0 0x55b0ef2621a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:32:43.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.280+0000 7f8a2afedb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b0ef2521a0 0x55b0ef253510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:32:43.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.280+0000 7f8a2afedb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0ef252ac0 0x55b0ef26db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:32:43.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.280+0000 7f8a2afedb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8a181b3aa0 con 0x55b0ef252ac0 2024-05-03T18:32:43.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.280+0000 7f8a2afedb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8a181b6320 con 0x55b0ef2521a0 2024-05-03T18:32:43.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.280+0000 7f8a2afedb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8a181b4110 con 0x55b0ef24cde0 2024-05-03T18:32:43.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.281+0000 7f8a1dc00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b0ef24cde0 0x55b0ef2621a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:32:43.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.281+0000 7f8a1dc00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b0ef24cde0 0x55b0ef2621a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:53502/0 (socket says 172.21.15.33:53502) 2024-05-03T18:32:43.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.281+0000 7f8a1dc00700 1 -- 172.21.15.33:0/2435131325 learned_addr learned my addr 172.21.15.33:0/2435131325 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:32:43.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.281+0000 7f8a1d200700 1 --2- 172.21.15.33:0/2435131325 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b0ef2521a0 0x55b0ef253510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:32:43.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.281+0000 7f8a1e600700 1 --2- 172.21.15.33:0/2435131325 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0ef252ac0 0x55b0ef26db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:32:43.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.281+0000 7f8a1d200700 1 -- 172.21.15.33:0/2435131325 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b0ef24cde0 msgr2=0x55b0ef2621a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:32:43.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.281+0000 7f8a1d200700 1 --2- 172.21.15.33:0/2435131325 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b0ef24cde0 0x55b0ef2621a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:43.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.281+0000 7f8a1d200700 1 -- 172.21.15.33:0/2435131325 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0ef252ac0 msgr2=0x55b0ef26db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:32:43.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.282+0000 7f8a1d200700 1 --2- 172.21.15.33:0/2435131325 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0ef252ac0 0x55b0ef26db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:43.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.282+0000 7f8a1d200700 1 -- 172.21.15.33:0/2435131325 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0ef23b170 con 0x55b0ef2521a0 2024-05-03T18:32:43.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.283+0000 7f8a1d200700 1 --2- 172.21.15.33:0/2435131325 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b0ef2521a0 0x55b0ef253510 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x55b0ef268a80 tx=0x7f8a04011000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5505ff64256d5cb7 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:32:43.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.283+0000 7f8a0a000700 1 -- 172.21.15.33:0/2435131325 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a04004a30 con 0x55b0ef2521a0 2024-05-03T18:32:43.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.283+0000 7f8a0a000700 1 -- 172.21.15.33:0/2435131325 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8a0400ee60 con 0x55b0ef2521a0 2024-05-03T18:32:43.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.283+0000 7f8a0a000700 1 -- 172.21.15.33:0/2435131325 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a040038e0 con 0x55b0ef2521a0 2024-05-03T18:32:43.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.284+0000 7f8a2afedb80 1 -- 172.21.15.33:0/2435131325 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b0ef2521a0 msgr2=0x55b0ef253510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:32:43.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.284+0000 7f8a2afedb80 1 --2- 172.21.15.33:0/2435131325 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b0ef2521a0 0x55b0ef253510 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x55b0ef268a80 tx=0x7f8a04011000 comp rx=0 tx=0).stop 2024-05-03T18:32:43.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.284+0000 7f8a2afedb80 1 -- 172.21.15.33:0/2435131325 shutdown_connections 2024-05-03T18:32:43.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.284+0000 7f8a2afedb80 1 --2- 172.21.15.33:0/2435131325 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0ef252ac0 0x55b0ef26db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:43.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.284+0000 7f8a2afedb80 1 --2- 172.21.15.33:0/2435131325 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b0ef24cde0 0x55b0ef2621a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:43.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.284+0000 7f8a2afedb80 1 --2- 172.21.15.33:0/2435131325 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b0ef2521a0 0x55b0ef253510 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:43.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.284+0000 7f8a2afedb80 1 -- 172.21.15.33:0/2435131325 >> 172.21.15.33:0/2435131325 conn(0x55b0ef1bc750 msgr2=0x55b0ef239430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:32:43.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.285+0000 7f8a2afedb80 1 -- 172.21.15.33:0/2435131325 shutdown_connections 2024-05-03T18:32:43.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.285+0000 7f8a2afedb80 1 -- 172.21.15.33:0/2435131325 wait complete. 2024-05-03T18:32:43.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.285+0000 7f8a2afedb80 1 -- start start 2024-05-03T18:32:43.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.286+0000 7f8a2afedb80 1 Processor -- start 2024-05-03T18:32:43.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.287+0000 7f8a2afedb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b0ef24cde0 0x55b0ef25e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:32:43.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.287+0000 7f8a2afedb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b0ef2521a0 0x55b0ef25e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:32:43.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.287+0000 7f8a2afedb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0ef252ac0 0x55b0ef25b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:32:43.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.287+0000 7f8a2afedb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8a181b47e0 con 0x55b0ef252ac0 2024-05-03T18:32:43.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.287+0000 7f8a2afedb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8a181b5c50 con 0x55b0ef2521a0 2024-05-03T18:32:43.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.287+0000 7f8a2afedb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8a181b4eb0 con 0x55b0ef24cde0 2024-05-03T18:32:43.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.287+0000 7f8a2afedb80 10 client.0 _ll_register_callbacks cb 0x55b0ef13dcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:32:43.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.288+0000 7f8a1d200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b0ef2521a0 0x55b0ef25e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:32:43.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.288+0000 7f8a1d200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b0ef2521a0 0x55b0ef25e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:35928/0 (socket says 172.21.15.33:35928) 2024-05-03T18:32:43.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.288+0000 7f8a1d200700 1 -- 172.21.15.33:0/3067493203 learned_addr learned my addr 172.21.15.33:0/3067493203 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:32:43.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.288+0000 7f8a1dc00700 1 --2- 172.21.15.33:0/3067493203 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b0ef24cde0 0x55b0ef25e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:32:43.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.289+0000 7f8a1d200700 1 -- 172.21.15.33:0/3067493203 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b0ef24cde0 msgr2=0x55b0ef25e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:32:43.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.289+0000 7f8a1d200700 1 --2- 172.21.15.33:0/3067493203 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b0ef24cde0 0x55b0ef25e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:43.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.289+0000 7f8a1d200700 1 -- 172.21.15.33:0/3067493203 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0ef252ac0 msgr2=0x55b0ef25b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:32:43.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.289+0000 7f8a1d200700 1 --2- 172.21.15.33:0/3067493203 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0ef252ac0 0x55b0ef25b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:43.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.289+0000 7f8a1d200700 1 -- 172.21.15.33:0/3067493203 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0ef25bff0 con 0x55b0ef2521a0 2024-05-03T18:32:43.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.290+0000 7f8a1dc00700 1 --2- 172.21.15.33:0/3067493203 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b0ef24cde0 0x55b0ef25e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:32:43.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.292+0000 7f8a1d200700 1 --2- 172.21.15.33:0/3067493203 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b0ef2521a0 0x55b0ef25e7d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8a04000c00 tx=0x7f8a04009560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf2e4123eeb711b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:32:43.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.292+0000 7f89ff400700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-03T18:32:43.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.294+0000 7f89ff400700 1 -- 172.21.15.33:0/3067493203 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a04003d40 con 0x55b0ef2521a0 2024-05-03T18:32:43.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.294+0000 7f89ff400700 1 -- 172.21.15.33:0/3067493203 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f8a04034480 con 0x55b0ef2521a0 2024-05-03T18:32:43.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.294+0000 7f8a2afedb80 1 -- 172.21.15.33:0/3067493203 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a0400ee60 con 0x55b0ef2521a0 2024-05-03T18:32:43.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.294+0000 7f89ff400700 1 -- 172.21.15.33:0/3067493203 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f8a04042050 con 0x55b0ef2521a0 2024-05-03T18:32:43.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.294+0000 7f8a2afedb80 10 client.35594 fetch_fsmap learned FSMap version 21 2024-05-03T18:32:43.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.294+0000 7f8a2afedb80 1 -- 172.21.15.33:0/3067493203 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55b0ef2d21d0 con 0x55b0ef2521a0 2024-05-03T18:32:43.297 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:32:42 smithi099 ceph-mon[31658]: pgmap v168: 129 pgs: 129 active+clean; 1.7 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.7 MiB/s wr, 625 op/s 2024-05-03T18:32:43.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.295+0000 7f89ff400700 1 -- 172.21.15.33:0/3067493203 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8a0400ee60 con 0x55b0ef2521a0 2024-05-03T18:32:43.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.295+0000 7f89ff400700 1 -- 172.21.15.33:0/3067493203 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f8a04029660 con 0x55b0ef2521a0 2024-05-03T18:32:43.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.295+0000 7f8a2afedb80 10 client.35594 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:32:43.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.295+0000 7f8a2afedb80 10 client.35594 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:32:43.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.295+0000 7f8a2afedb80 20 client.35594 populate_metadata read hostname 'smithi033' 2024-05-03T18:32:43.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.295+0000 7f8a2afedb80 1 --2- 172.21.15.33:0/3067493203 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55b0ef2d2850 0x55b0ef2d4c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:32:43.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.295+0000 7f8a2afedb80 4 client.35594 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-03T18:32:43.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.296+0000 7f8a2afedb80 1 -- 172.21.15.33:0/3067493203 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55b0ef1a4480 con 0x55b0ef2d2850 2024-05-03T18:32:43.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.296+0000 7f8a1dc00700 1 --2- 172.21.15.33:0/3067493203 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55b0ef2d2850 0x55b0ef2d4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:32:43.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.298+0000 7f8a1dc00700 1 --2- 172.21.15.33:0/3067493203 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55b0ef2d2850 0x55b0ef2d4c70 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2e1fd625a00713f server_cookie=83a69eff5188c54b in_seq=0 out_seq=0 2024-05-03T18:32:43.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.298+0000 7f89ff400700 10 client.35594 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:32:43.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.300+0000 7f89ff400700 1 -- 172.21.15.33:0/3067493203 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b0ef1a4480 con 0x55b0ef2d2850 2024-05-03T18:32:43.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.300+0000 7f89ff400700 10 client.35594 handle_command_reply: tid=0 2024-05-03T18:32:43.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.300+0000 7f8a2afedb80 1 client.35594 shutdown 2024-05-03T18:32:43.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.300+0000 7f8a2afedb80 1 -- 172.21.15.33:0/3067493203 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b0ef2521a0 msgr2=0x55b0ef25e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:32:43.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.300+0000 7f8a2afedb80 1 --2- 172.21.15.33:0/3067493203 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b0ef2521a0 0x55b0ef25e7d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8a04000c00 tx=0x7f8a04009560 comp rx=0 tx=0).stop 2024-05-03T18:32:43.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.301+0000 7f8a2afedb80 1 -- 172.21.15.33:0/3067493203 shutdown_connections 2024-05-03T18:32:43.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.301+0000 7f8a2afedb80 1 --2- 172.21.15.33:0/3067493203 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55b0ef2d2850 0x55b0ef2d4c70 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:43.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.301+0000 7f8a2afedb80 1 --2- 172.21.15.33:0/3067493203 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0ef252ac0 0x55b0ef25b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:43.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.301+0000 7f89ff400700 0 client.35594 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:32:43.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.301+0000 7f8a2afedb80 1 --2- 172.21.15.33:0/3067493203 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b0ef24cde0 0x55b0ef25e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:43.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.301+0000 7f8a2afedb80 1 --2- 172.21.15.33:0/3067493203 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b0ef2521a0 0x55b0ef25e7d0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:43.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.301+0000 7f8a2afedb80 1 -- 172.21.15.33:0/3067493203 >> 172.21.15.33:0/3067493203 conn(0x55b0ef1bc750 msgr2=0x55b0ef239430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:32:43.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.302+0000 7f8a2afedb80 1 -- 172.21.15.33:0/3067493203 shutdown_connections 2024-05-03T18:32:43.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.302+0000 7f8a2afedb80 1 -- 172.21.15.33:0/3067493203 wait complete. 2024-05-03T18:32:43.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.303+0000 7f8a2afedb80 20 client.35594 trim_cache size 0 max 16384 2024-05-03T18:32:43.306 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-03T18:32:43.306 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-05-03T18:32:43.306 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "3802f056-7ebb-4a4f-8c91-4fb07c233345", 2024-05-03T18:32:43.306 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-05-03T18:32:43.306 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-03T18:32:43.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.305+0000 7f8a20600700 1 -- 172.21.15.33:0/2755300410 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f8a000b6e80 msgr2=0x7f8a000b9340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:32:43.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.305+0000 7f8a20600700 1 --2- 172.21.15.33:0/2755300410 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f8a000b6e80 0x7f8a000b9340 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8a04005e50 tx=0x7f8a04005da0 comp rx=0 tx=0).stop 2024-05-03T18:32:43.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.305+0000 7f8a20600700 1 -- 172.21.15.33:0/2755300410 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a181023a0 msgr2=0x7f8a18074d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:32:43.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.305+0000 7f8a20600700 1 --2- 172.21.15.33:0/2755300410 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a181023a0 0x7f8a18074d80 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c005300 tx=0x7f8a0c005330 comp rx=0 tx=0).stop 2024-05-03T18:32:43.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.305+0000 7f8a20600700 1 -- 172.21.15.33:0/2755300410 shutdown_connections 2024-05-03T18:32:43.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.305+0000 7f8a20600700 1 --2- 172.21.15.33:0/2755300410 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f8a000b6e80 0x7f8a000b9340 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:43.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.305+0000 7f8a20600700 1 --2- 172.21.15.33:0/2755300410 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a18108410 0x7f8a1806e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:43.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.305+0000 7f8a20600700 1 --2- 172.21.15.33:0/2755300410 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a181023a0 0x7f8a18074d80 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:43.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.305+0000 7f8a20600700 1 --2- 172.21.15.33:0/2755300410 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a18102d60 0x7f8a1806de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:43.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.305+0000 7f8a20600700 1 -- 172.21.15.33:0/2755300410 >> 172.21.15.33:0/2755300410 conn(0x7f8a18079350 msgr2=0x7f8a18101450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:32:43.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.306+0000 7f8a20600700 1 -- 172.21.15.33:0/2755300410 shutdown_connections 2024-05-03T18:32:43.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:43.306+0000 7f8a20600700 1 -- 172.21.15.33:0/2755300410 wait complete. 2024-05-03T18:32:43.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:32:42 smithi150 ceph-mon[32492]: pgmap v168: 129 pgs: 129 active+clean; 1.7 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.7 MiB/s wr, 625 op/s 2024-05-03T18:32:44.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:32:44 smithi099 ceph-mon[31658]: pgmap v169: 129 pgs: 129 active+clean; 1.7 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 8.2 MiB/s wr, 922 op/s 2024-05-03T18:32:44.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:32:44 smithi150 ceph-mon[32492]: pgmap v169: 129 pgs: 129 active+clean; 1.7 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 8.2 MiB/s wr, 922 op/s 2024-05-03T18:32:44.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:32:44 smithi033 ceph-mon[28330]: pgmap v169: 129 pgs: 129 active+clean; 1.7 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 8.2 MiB/s wr, 922 op/s 2024-05-03T18:32:45.057 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-03T18:32:45.110 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 90.0 s, 643.2 tps, lat 155.083 ms stddev 56.962 2024-05-03T18:32:45.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:32:45 smithi033 ceph-mon[28330]: scrub queued for path: / 2024-05-03T18:32:45.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:32:45 smithi033 ceph-mon[28330]: scrub summary: idle+waiting paths [/] 2024-05-03T18:32:45.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:32:45 smithi033 ceph-mon[28330]: scrub summary: active paths [/] 2024-05-03T18:32:45.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:32:45 smithi033 ceph-mon[28330]: scrub summary: idle+waiting paths [/] 2024-05-03T18:32:45.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:32:45 smithi099 ceph-mon[31658]: scrub queued for path: / 2024-05-03T18:32:45.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:32:45 smithi099 ceph-mon[31658]: scrub summary: idle+waiting paths [/] 2024-05-03T18:32:45.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:32:45 smithi099 ceph-mon[31658]: scrub summary: active paths [/] 2024-05-03T18:32:45.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:32:45 smithi099 ceph-mon[31658]: scrub summary: idle+waiting paths [/] 2024-05-03T18:32:45.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:32:45 smithi150 ceph-mon[32492]: scrub queued for path: / 2024-05-03T18:32:45.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:32:45 smithi150 ceph-mon[32492]: scrub summary: idle+waiting paths [/] 2024-05-03T18:32:45.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:32:45 smithi150 ceph-mon[32492]: scrub summary: active paths [/] 2024-05-03T18:32:45.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:32:45 smithi150 ceph-mon[32492]: scrub summary: idle+waiting paths [/] 2024-05-03T18:32:45.784 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:32:46.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:32:46 smithi033 ceph-mon[28330]: pgmap v170: 129 pgs: 129 active+clean; 1.7 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.2 MiB/s wr, 598 op/s 2024-05-03T18:32:46.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:32:46 smithi099 ceph-mon[31658]: pgmap v170: 129 pgs: 129 active+clean; 1.7 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.2 MiB/s wr, 598 op/s 2024-05-03T18:32:46.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:32:46 smithi150 ceph-mon[32492]: pgmap v170: 129 pgs: 129 active+clean; 1.7 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.2 MiB/s wr, 598 op/s 2024-05-03T18:32:47.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:32:47 smithi099 ceph-mon[31658]: scrub summary: idle 2024-05-03T18:32:47.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:32:47 smithi150 ceph-mon[32492]: scrub summary: idle 2024-05-03T18:32:47.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:32:47 smithi033 ceph-mon[28330]: scrub summary: idle 2024-05-03T18:32:48.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:32:48 smithi150 ceph-mon[32492]: pgmap v171: 129 pgs: 129 active+clean; 1.7 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 7.8 MiB/s wr, 910 op/s 2024-05-03T18:32:48.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:32:48 smithi033 ceph-mon[28330]: pgmap v171: 129 pgs: 129 active+clean; 1.7 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 7.8 MiB/s wr, 910 op/s 2024-05-03T18:32:48.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:32:48 smithi099 ceph-mon[31658]: pgmap v171: 129 pgs: 129 active+clean; 1.7 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 7.8 MiB/s wr, 910 op/s 2024-05-03T18:32:48.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.732+0000 7f6814a00700 1 -- 172.21.15.33:0/3213986240 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f680800b760 con 0x7f6818108410 2024-05-03T18:32:48.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.732+0000 7f681ce00700 1 -- 172.21.15.33:0/3213986240 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6818108410 msgr2=0x7f68181087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:32:48.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.732+0000 7f681ce00700 1 --2- 172.21.15.33:0/3213986240 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6818108410 0x7f68181087f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f68080098f0 tx=0x7f6808009c00 comp rx=0 tx=0).stop 2024-05-03T18:32:48.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.732+0000 7f681ce00700 1 -- 172.21.15.33:0/3213986240 shutdown_connections 2024-05-03T18:32:48.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.732+0000 7f681ce00700 1 --2- 172.21.15.33:0/3213986240 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6818102d60 0x7f681810f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:48.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.732+0000 7f681ce00700 1 --2- 172.21.15.33:0/3213986240 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6818108410 0x7f68181087f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:48.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.732+0000 7f681ce00700 1 --2- 172.21.15.33:0/3213986240 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68181023a0 0x7f6818102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:48.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.732+0000 7f681ce00700 1 -- 172.21.15.33:0/3213986240 >> 172.21.15.33:0/3213986240 conn(0x7f6818079350 msgr2=0x7f6818079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:32:48.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.733+0000 7f681ce00700 1 -- 172.21.15.33:0/3213986240 shutdown_connections 2024-05-03T18:32:48.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.733+0000 7f681ce00700 1 -- 172.21.15.33:0/3213986240 wait complete. 2024-05-03T18:32:48.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.737+0000 7f681ce00700 1 Processor -- start 2024-05-03T18:32:48.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.737+0000 7f681ce00700 1 -- start start 2024-05-03T18:32:48.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.738+0000 7f681ce00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68181023a0 0x7f68181986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:32:48.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.738+0000 7f681ce00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6818102d60 0x7f6818198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:32:48.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.738+0000 7f681ce00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6818108410 0x7f681819d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:32:48.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.738+0000 7f681ce00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6818112480 con 0x7f6818102d60 2024-05-03T18:32:48.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.738+0000 7f681ce00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6818112300 con 0x7f6818108410 2024-05-03T18:32:48.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.738+0000 7f681ce00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6818112600 con 0x7f68181023a0 2024-05-03T18:32:48.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.738+0000 7f6815e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68181023a0 0x7f68181986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:32:48.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.738+0000 7f6815e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68181023a0 0x7f68181986f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:60370/0 (socket says 172.21.15.33:60370) 2024-05-03T18:32:48.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.738+0000 7f6815e00700 1 -- 172.21.15.33:0/2445527372 learned_addr learned my addr 172.21.15.33:0/2445527372 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:32:48.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.739+0000 7f6815400700 1 --2- 172.21.15.33:0/2445527372 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6818102d60 0x7f6818198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:32:48.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.739+0000 7f6816800700 1 --2- 172.21.15.33:0/2445527372 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6818108410 0x7f681819d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:32:48.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.739+0000 7f6815e00700 1 -- 172.21.15.33:0/2445527372 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6818108410 msgr2=0x7f681819d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:32:48.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.739+0000 7f6815e00700 1 --2- 172.21.15.33:0/2445527372 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6818108410 0x7f681819d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:48.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.739+0000 7f6815e00700 1 -- 172.21.15.33:0/2445527372 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6818102d60 msgr2=0x7f6818198c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:32:48.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.740+0000 7f6815e00700 1 --2- 172.21.15.33:0/2445527372 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6818102d60 0x7f6818198c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:48.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.740+0000 7f6815e00700 1 -- 172.21.15.33:0/2445527372 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f681819dc80 con 0x7f68181023a0 2024-05-03T18:32:48.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.740+0000 7f6815400700 1 --2- 172.21.15.33:0/2445527372 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6818102d60 0x7f6818198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:32:48.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.740+0000 7f6815e00700 1 --2- 172.21.15.33:0/2445527372 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68181023a0 0x7f68181986f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6808000c00 tx=0x7f680800b610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:32:48.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.741+0000 7f680ea00700 1 -- 172.21.15.33:0/2445527372 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6808025070 con 0x7f68181023a0 2024-05-03T18:32:48.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.741+0000 7f680ea00700 1 -- 172.21.15.33:0/2445527372 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f68080054d0 con 0x7f68181023a0 2024-05-03T18:32:48.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.741+0000 7f680ea00700 1 -- 172.21.15.33:0/2445527372 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6808003e10 con 0x7f68181023a0 2024-05-03T18:32:48.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.741+0000 7f681ce00700 1 -- 172.21.15.33:0/2445527372 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f681819df10 con 0x7f68181023a0 2024-05-03T18:32:48.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.742+0000 7f681ce00700 1 -- 172.21.15.33:0/2445527372 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f681819e2e0 con 0x7f68181023a0 2024-05-03T18:32:48.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.751+0000 7f6827885b80 1 Processor -- start 2024-05-03T18:32:48.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.751+0000 7f680ea00700 1 -- 172.21.15.33:0/2445527372 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f6808028070 con 0x7f68181023a0 2024-05-03T18:32:48.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.752+0000 7f6827885b80 1 -- start start 2024-05-03T18:32:48.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.752+0000 7f6827885b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa54a0e7b0 0x55aa54a0eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:32:48.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.752+0000 7f6827885b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aa54a0f0d0 0x55aa54a09fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:32:48.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.752+0000 7f6827885b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa54a0a5d0 0x55aa54a0aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:32:48.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.752+0000 7f6827885b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55aa5495ca20 con 0x55aa54a0f0d0 2024-05-03T18:32:48.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.752+0000 7f6827885b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55aa548fa760 con 0x55aa54a0e7b0 2024-05-03T18:32:48.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.752+0000 7f6827885b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55aa549589e0 con 0x55aa54a0a5d0 2024-05-03T18:32:48.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.753+0000 7f6815e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aa54a0f0d0 0x55aa54a09fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:32:48.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.753+0000 7f680ea00700 1 --2- 172.21.15.33:0/2445527372 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f67fc0b2af0 0x7f67fc0b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:32:48.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.753+0000 7f6815e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aa54a0f0d0 0x55aa54a09fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57796/0 (socket says 172.21.15.33:57796) 2024-05-03T18:32:48.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.753+0000 7f6815e00700 1 -- 172.21.15.33:0/1476911733 learned_addr learned my addr 172.21.15.33:0/1476911733 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:32:48.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.753+0000 7f6815400700 1 --2- 172.21.15.33:0/1476911733 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa54a0a5d0 0x55aa54a0aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:32:48.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.753+0000 7f680ea00700 1 -- 172.21.15.33:0/2445527372 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f68080e3330 con 0x7f68181023a0 2024-05-03T18:32:48.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.754+0000 7f6815e00700 1 -- 172.21.15.33:0/1476911733 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa54a0a5d0 msgr2=0x55aa54a0aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:32:48.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.754+0000 7f6815e00700 1 --2- 172.21.15.33:0/1476911733 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa54a0a5d0 0x55aa54a0aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:48.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.754+0000 7f6815e00700 1 -- 172.21.15.33:0/1476911733 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa54a0e7b0 msgr2=0x55aa54a0eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:32:48.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.754+0000 7f6815e00700 1 --2- 172.21.15.33:0/1476911733 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa54a0e7b0 0x55aa54a0eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:48.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.754+0000 7f6815e00700 1 -- 172.21.15.33:0/1476911733 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa548a98d0 con 0x55aa54a0f0d0 2024-05-03T18:32:48.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.754+0000 7f6816800700 1 --2- 172.21.15.33:0/2445527372 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f67fc0b2af0 0x7f67fc0b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:32:48.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.754+0000 7f6815e00700 1 --2- 172.21.15.33:0/1476911733 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aa54a0f0d0 0x55aa54a09fb0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f68080e0600 tx=0x7f6808009480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39f7476f4b09737d server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:32:48.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.755+0000 7f6803400700 1 -- 172.21.15.33:0/1476911733 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68080a9dc0 con 0x55aa54a0f0d0 2024-05-03T18:32:48.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.755+0000 7f6803400700 1 -- 172.21.15.33:0/1476911733 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f68080d4040 con 0x55aa54a0f0d0 2024-05-03T18:32:48.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.755+0000 7f6803400700 1 -- 172.21.15.33:0/1476911733 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68080aa510 con 0x55aa54a0f0d0 2024-05-03T18:32:48.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.756+0000 7f6827885b80 1 -- 172.21.15.33:0/1476911733 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aa54a0f0d0 msgr2=0x55aa54a09fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:32:48.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.756+0000 7f6827885b80 1 --2- 172.21.15.33:0/1476911733 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aa54a0f0d0 0x55aa54a09fb0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f68080e0600 tx=0x7f6808009480 comp rx=0 tx=0).stop 2024-05-03T18:32:48.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.756+0000 7f6816800700 1 --2- 172.21.15.33:0/2445527372 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f67fc0b2af0 0x7f67fc0b4fb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6810006130 tx=0x7f68100077f0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:32:48.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.757+0000 7f6827885b80 1 -- 172.21.15.33:0/1476911733 shutdown_connections 2024-05-03T18:32:48.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.757+0000 7f6827885b80 1 --2- 172.21.15.33:0/1476911733 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa54a0a5d0 0x55aa54a0aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:48.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.757+0000 7f6827885b80 1 --2- 172.21.15.33:0/1476911733 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa54a0e7b0 0x55aa54a0eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:48.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.757+0000 7f6827885b80 1 --2- 172.21.15.33:0/1476911733 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aa54a0f0d0 0x55aa54a09fb0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:48.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.757+0000 7f6827885b80 1 -- 172.21.15.33:0/1476911733 >> 172.21.15.33:0/1476911733 conn(0x55aa54938b60 msgr2=0x55aa54a047f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:32:48.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.757+0000 7f6827885b80 1 -- 172.21.15.33:0/1476911733 shutdown_connections 2024-05-03T18:32:48.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.757+0000 7f6827885b80 1 -- 172.21.15.33:0/1476911733 wait complete. 2024-05-03T18:32:48.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.758+0000 7f6827885b80 1 -- start start 2024-05-03T18:32:48.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.758+0000 7f6827885b80 1 Processor -- start 2024-05-03T18:32:48.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.759+0000 7f6827885b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa54a0a5d0 0x55aa54a15f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:32:48.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.759+0000 7f6827885b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aa54a0e7b0 0x55aa54a13300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:32:48.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.759+0000 7f6827885b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa54a0f0d0 0x55aa54a13840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:32:48.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.759+0000 7f6827885b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55aa5495ca20 con 0x55aa54a0e7b0 2024-05-03T18:32:48.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.759+0000 7f6827885b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55aa548c2b20 con 0x55aa54a0f0d0 2024-05-03T18:32:48.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.759+0000 7f6827885b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55aa54a16a60 con 0x55aa54a0a5d0 2024-05-03T18:32:48.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.759+0000 7f6827885b80 10 client.0 _ll_register_callbacks cb 0x55aa54903c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:32:48.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.760+0000 7f6815400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aa54a0e7b0 0x55aa54a13300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:32:48.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.760+0000 7f6815400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aa54a0e7b0 0x55aa54a13300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57810/0 (socket says 172.21.15.33:57810) 2024-05-03T18:32:48.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.760+0000 7f6815400700 1 -- 172.21.15.33:0/2863579158 learned_addr learned my addr 172.21.15.33:0/2863579158 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:32:48.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.760+0000 7f6816800700 1 --2- 172.21.15.33:0/2863579158 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa54a0f0d0 0x55aa54a13840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:32:48.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.761+0000 7f6815400700 1 -- 172.21.15.33:0/2863579158 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa54a0a5d0 msgr2=0x55aa54a15f60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-03T18:32:48.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.761+0000 7f6815400700 1 --2- 172.21.15.33:0/2863579158 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa54a0a5d0 0x55aa54a15f60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:48.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.761+0000 7f6815400700 1 -- 172.21.15.33:0/2863579158 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa54a0f0d0 msgr2=0x55aa54a13840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:32:48.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.761+0000 7f6815400700 1 --2- 172.21.15.33:0/2863579158 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa54a0f0d0 0x55aa54a13840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:48.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.761+0000 7f6815400700 1 -- 172.21.15.33:0/2863579158 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa54922420 con 0x55aa54a0e7b0 2024-05-03T18:32:48.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.761+0000 7f6815400700 1 --2- 172.21.15.33:0/2863579158 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aa54a0e7b0 0x55aa54a13300 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f68040126a0 tx=0x7f68040129b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f7b179976a9b6ff server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:32:48.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.761+0000 7f67f7e00700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:32:48.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.762+0000 7f67f7e00700 1 -- 172.21.15.33:0/2863579158 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f680401f060 con 0x55aa54a0e7b0 2024-05-03T18:32:48.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.762+0000 7f67f7e00700 1 -- 172.21.15.33:0/2863579158 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f6804004460 con 0x55aa54a0e7b0 2024-05-03T18:32:48.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.762+0000 7f6827885b80 1 -- 172.21.15.33:0/2863579158 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68080d4040 con 0x55aa54a0e7b0 2024-05-03T18:32:48.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.762+0000 7f67f7e00700 1 -- 172.21.15.33:0/2863579158 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f6804026050 con 0x55aa54a0e7b0 2024-05-03T18:32:48.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.762+0000 7f6827885b80 10 client.15987 fetch_fsmap learned FSMap version 21 2024-05-03T18:32:48.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.762+0000 7f6827885b80 1 -- 172.21.15.33:0/2863579158 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55aa54a32110 con 0x55aa54a0e7b0 2024-05-03T18:32:48.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.763+0000 7f67f7e00700 1 -- 172.21.15.33:0/2863579158 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f68080d4040 con 0x55aa54a0e7b0 2024-05-03T18:32:48.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.763+0000 7f67f7e00700 1 -- 172.21.15.33:0/2863579158 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f6804019030 con 0x55aa54a0e7b0 2024-05-03T18:32:48.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.763+0000 7f6827885b80 10 client.15987 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:32:48.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.763+0000 7f6827885b80 10 client.15987 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:32:48.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.763+0000 7f6827885b80 20 client.15987 populate_metadata read hostname 'smithi033' 2024-05-03T18:32:48.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.763+0000 7f6827885b80 1 --2- 172.21.15.33:0/2863579158 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55aa54a34750 0x55aa54a36b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:32:48.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.763+0000 7f6827885b80 4 client.15987 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:32:48.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.763+0000 7f6827885b80 1 -- 172.21.15.33:0/2863579158 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aa549d71d0 con 0x55aa54a34750 2024-05-03T18:32:48.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.765+0000 7f6815e00700 1 --2- 172.21.15.33:0/2863579158 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55aa54a34750 0x55aa54a36b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:32:48.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.766+0000 7f6815e00700 1 --2- 172.21.15.33:0/2863579158 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55aa54a34750 0x55aa54a36b40 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3d347ba98c9f10e server_cookie=4955d85d208973b4 in_seq=0 out_seq=0 2024-05-03T18:32:48.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.766+0000 7f67f7e00700 10 client.15987 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:32:48.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.767+0000 7f67f7e00700 1 -- 172.21.15.33:0/2863579158 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55aa549d71d0 con 0x55aa54a34750 2024-05-03T18:32:48.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.767+0000 7f67f7e00700 10 client.15987 handle_command_reply: tid=0 2024-05-03T18:32:48.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.768+0000 7f6827885b80 1 client.15987 shutdown 2024-05-03T18:32:48.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.768+0000 7f6827885b80 1 -- 172.21.15.33:0/2863579158 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aa54a0e7b0 msgr2=0x55aa54a13300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:32:48.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.768+0000 7f6827885b80 1 --2- 172.21.15.33:0/2863579158 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aa54a0e7b0 0x55aa54a13300 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f68040126a0 tx=0x7f68040129b0 comp rx=0 tx=0).stop 2024-05-03T18:32:48.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.768+0000 7f6827885b80 1 -- 172.21.15.33:0/2863579158 shutdown_connections 2024-05-03T18:32:48.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.768+0000 7f6827885b80 1 --2- 172.21.15.33:0/2863579158 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55aa54a34750 0x55aa54a36b40 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:48.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.769+0000 7f67f7e00700 0 client.15987 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:32:48.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.769+0000 7f6827885b80 1 --2- 172.21.15.33:0/2863579158 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa54a0f0d0 0x55aa54a13840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:48.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.769+0000 7f6827885b80 1 --2- 172.21.15.33:0/2863579158 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa54a0a5d0 0x55aa54a15f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:48.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.769+0000 7f6827885b80 1 --2- 172.21.15.33:0/2863579158 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aa54a0e7b0 0x55aa54a13300 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:48.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.769+0000 7f6827885b80 1 -- 172.21.15.33:0/2863579158 >> 172.21.15.33:0/2863579158 conn(0x55aa54938b60 msgr2=0x55aa54a0fe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:32:48.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.769+0000 7f6827885b80 1 -- 172.21.15.33:0/2863579158 shutdown_connections 2024-05-03T18:32:48.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.769+0000 7f6827885b80 1 -- 172.21.15.33:0/2863579158 wait complete. 2024-05-03T18:32:48.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.770+0000 7f6827885b80 20 client.15987 trim_cache size 0 max 16384 2024-05-03T18:32:48.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.815+0000 7f6827885b80 1 Processor -- start 2024-05-03T18:32:48.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.815+0000 7f6827885b80 1 -- start start 2024-05-03T18:32:48.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.816+0000 7f6827885b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa54a0a5d0 0x55aa54a03d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:32:48.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.816+0000 7f6827885b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa54a0e7b0 0x55aa54a3c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:32:48.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.816+0000 7f6827885b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aa54a0f0d0 0x55aa54a3c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:32:48.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.816+0000 7f6827885b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f68180738e0 con 0x55aa54a0f0d0 2024-05-03T18:32:48.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.816+0000 7f6827885b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f681819f060 con 0x55aa54a0a5d0 2024-05-03T18:32:48.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.816+0000 7f6827885b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6818073f50 con 0x55aa54a0e7b0 2024-05-03T18:32:48.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.816+0000 7f6815400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa54a0e7b0 0x55aa54a3c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:32:48.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.816+0000 7f6815400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa54a0e7b0 0x55aa54a3c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:60390/0 (socket says 172.21.15.33:60390) 2024-05-03T18:32:48.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.816+0000 7f6815400700 1 -- 172.21.15.33:0/2983617089 learned_addr learned my addr 172.21.15.33:0/2983617089 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:32:48.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.817+0000 7f6816800700 1 --2- 172.21.15.33:0/2983617089 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aa54a0f0d0 0x55aa54a3c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:32:48.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.817+0000 7f6815400700 1 -- 172.21.15.33:0/2983617089 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa54a0a5d0 msgr2=0x55aa54a03d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:32:48.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.817+0000 7f6815400700 1 --2- 172.21.15.33:0/2983617089 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa54a0a5d0 0x55aa54a03d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:48.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.817+0000 7f6815400700 1 -- 172.21.15.33:0/2983617089 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aa54a0f0d0 msgr2=0x55aa54a3c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:32:48.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.817+0000 7f6815400700 1 --2- 172.21.15.33:0/2983617089 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aa54a0f0d0 0x55aa54a3c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:48.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.817+0000 7f6815400700 1 -- 172.21.15.33:0/2983617089 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa54a30ce0 con 0x55aa54a0e7b0 2024-05-03T18:32:48.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.819+0000 7f6815400700 1 --2- 172.21.15.33:0/2983617089 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa54a0e7b0 0x55aa54a3c450 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f680401a840 tx=0x7f6804012db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ccd01a45b1be24 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:32:48.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.820+0000 7f6802a00700 1 -- 172.21.15.33:0/2983617089 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f680402f070 con 0x55aa54a0e7b0 2024-05-03T18:32:48.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.821+0000 7f6802a00700 1 -- 172.21.15.33:0/2983617089 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f680401ab30 con 0x55aa54a0e7b0 2024-05-03T18:32:48.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.821+0000 7f6802a00700 1 -- 172.21.15.33:0/2983617089 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6804003b10 con 0x55aa54a0e7b0 2024-05-03T18:32:48.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.822+0000 7f6827885b80 1 -- 172.21.15.33:0/2983617089 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa54a0e7b0 msgr2=0x55aa54a3c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:32:48.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.822+0000 7f6827885b80 1 --2- 172.21.15.33:0/2983617089 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa54a0e7b0 0x55aa54a3c450 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f680401a840 tx=0x7f6804012db0 comp rx=0 tx=0).stop 2024-05-03T18:32:48.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.822+0000 7f6827885b80 1 -- 172.21.15.33:0/2983617089 shutdown_connections 2024-05-03T18:32:48.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.822+0000 7f6827885b80 1 --2- 172.21.15.33:0/2983617089 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aa54a0f0d0 0x55aa54a3c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:48.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.822+0000 7f6827885b80 1 --2- 172.21.15.33:0/2983617089 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa54a0a5d0 0x55aa54a03d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:48.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.822+0000 7f6827885b80 1 --2- 172.21.15.33:0/2983617089 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa54a0e7b0 0x55aa54a3c450 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:48.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.823+0000 7f6827885b80 1 -- 172.21.15.33:0/2983617089 >> 172.21.15.33:0/2983617089 conn(0x55aa54938b60 msgr2=0x55aa54a03650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:32:48.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.823+0000 7f6827885b80 1 -- 172.21.15.33:0/2983617089 shutdown_connections 2024-05-03T18:32:48.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.823+0000 7f6827885b80 1 -- 172.21.15.33:0/2983617089 wait complete. 2024-05-03T18:32:48.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.824+0000 7f6827885b80 1 -- start start 2024-05-03T18:32:48.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.824+0000 7f6827885b80 1 Processor -- start 2024-05-03T18:32:48.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.825+0000 7f6827885b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aa54a0a5d0 0x55aa54a1bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:32:48.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.826+0000 7f6827885b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa54a0e7b0 0x55aa54a1c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:32:48.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.826+0000 7f6827885b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa54a0f0d0 0x55aa54a16c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:32:48.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.826+0000 7f6827885b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6818074620 con 0x55aa54a0a5d0 2024-05-03T18:32:48.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.826+0000 7f6827885b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f681819e990 con 0x55aa54a0e7b0 2024-05-03T18:32:48.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.826+0000 7f6827885b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6818074cf0 con 0x55aa54a0f0d0 2024-05-03T18:32:48.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.826+0000 7f6827885b80 10 client.0 _ll_register_callbacks cb 0x55aa54903c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:32:48.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.826+0000 7f6816800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa54a0f0d0 0x55aa54a16c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:32:48.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.826+0000 7f6816800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa54a0f0d0 0x55aa54a16c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:60400/0 (socket says 172.21.15.33:60400) 2024-05-03T18:32:48.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.826+0000 7f6816800700 1 -- 172.21.15.33:0/3379357933 learned_addr learned my addr 172.21.15.33:0/3379357933 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:32:48.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.827+0000 7f6815e00700 1 --2- 172.21.15.33:0/3379357933 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aa54a0a5d0 0x55aa54a1bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:32:48.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.828+0000 7f6816800700 1 -- 172.21.15.33:0/3379357933 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa54a0e7b0 msgr2=0x55aa54a1c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:32:48.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.828+0000 7f6816800700 1 --2- 172.21.15.33:0/3379357933 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa54a0e7b0 0x55aa54a1c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:48.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.828+0000 7f6816800700 1 -- 172.21.15.33:0/3379357933 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aa54a0a5d0 msgr2=0x55aa54a1bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:32:48.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.828+0000 7f6816800700 1 --2- 172.21.15.33:0/3379357933 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aa54a0a5d0 0x55aa54a1bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:48.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.828+0000 7f6816800700 1 -- 172.21.15.33:0/3379357933 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa54a171b0 con 0x55aa54a0f0d0 2024-05-03T18:32:48.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.829+0000 7f67f7e00700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-03T18:32:48.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.829+0000 7f6816800700 1 --2- 172.21.15.33:0/3379357933 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa54a0f0d0 0x55aa54a16c70 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6810013df0 tx=0x7f6810004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b87ec6693430216f server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:32:48.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.829+0000 7f67f7e00700 1 -- 172.21.15.33:0/3379357933 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6810025020 con 0x55aa54a0f0d0 2024-05-03T18:32:48.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.830+0000 7f6827885b80 1 -- 172.21.15.33:0/3379357933 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f680401ab30 con 0x55aa54a0f0d0 2024-05-03T18:32:48.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.830+0000 7f67f7e00700 1 -- 172.21.15.33:0/3379357933 <== mon.1 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f681000fe40 con 0x55aa54a0f0d0 2024-05-03T18:32:48.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.831+0000 7f67f7e00700 1 -- 172.21.15.33:0/3379357933 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f6810030050 con 0x55aa54a0f0d0 2024-05-03T18:32:48.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.831+0000 7f6827885b80 10 client.25657 fetch_fsmap learned FSMap version 21 2024-05-03T18:32:48.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.831+0000 7f6827885b80 1 -- 172.21.15.33:0/3379357933 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55aa54a9f8d0 con 0x55aa54a0f0d0 2024-05-03T18:32:48.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.832+0000 7f67f7e00700 1 -- 172.21.15.33:0/3379357933 <== mon.1 v2:172.21.15.150:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f681001a070 con 0x55aa54a0f0d0 2024-05-03T18:32:48.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.832+0000 7f67f7e00700 1 -- 172.21.15.33:0/3379357933 <== mon.1 v2:172.21.15.150:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f6810021a40 con 0x55aa54a0f0d0 2024-05-03T18:32:48.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.832+0000 7f6827885b80 10 client.25657 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:32:48.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.832+0000 7f6827885b80 10 client.25657 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:32:48.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.832+0000 7f6827885b80 20 client.25657 populate_metadata read hostname 'smithi033' 2024-05-03T18:32:48.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.832+0000 7f6827885b80 1 --2- 172.21.15.33:0/3379357933 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55aa54a34750 0x55aa54aa2080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:32:48.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.832+0000 7f6827885b80 4 client.25657 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-03T18:32:48.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.832+0000 7f6827885b80 1 -- 172.21.15.33:0/3379357933 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f680401ab30 con 0x55aa54a34750 2024-05-03T18:32:48.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.833+0000 7f6815e00700 1 --2- 172.21.15.33:0/3379357933 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55aa54a34750 0x55aa54aa2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:32:48.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.834+0000 7f6815e00700 1 --2- 172.21.15.33:0/3379357933 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55aa54a34750 0x55aa54aa2080 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a359cf2aacfb5688 server_cookie=5a76da8ef833bdba in_seq=0 out_seq=0 2024-05-03T18:32:48.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.834+0000 7f67f7e00700 10 client.25657 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:32:48.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.835+0000 7f67f7e00700 1 -- 172.21.15.33:0/3379357933 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f680401ab30 con 0x55aa54a34750 2024-05-03T18:32:48.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.835+0000 7f67f7e00700 10 client.25657 handle_command_reply: tid=0 2024-05-03T18:32:48.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.835+0000 7f6827885b80 1 client.25657 shutdown 2024-05-03T18:32:48.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.836+0000 7f6827885b80 1 -- 172.21.15.33:0/3379357933 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa54a0f0d0 msgr2=0x55aa54a16c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:32:48.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.836+0000 7f6827885b80 1 --2- 172.21.15.33:0/3379357933 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa54a0f0d0 0x55aa54a16c70 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6810013df0 tx=0x7f6810004b40 comp rx=0 tx=0).stop 2024-05-03T18:32:48.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.836+0000 7f6827885b80 1 -- 172.21.15.33:0/3379357933 shutdown_connections 2024-05-03T18:32:48.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.836+0000 7f6827885b80 1 --2- 172.21.15.33:0/3379357933 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55aa54a34750 0x55aa54aa2080 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:48.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.836+0000 7f6827885b80 1 --2- 172.21.15.33:0/3379357933 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa54a0f0d0 0x55aa54a16c70 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:48.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.836+0000 7f67f7e00700 0 client.25657 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:32:48.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.836+0000 7f6827885b80 1 --2- 172.21.15.33:0/3379357933 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aa54a0a5d0 0x55aa54a1bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:48.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.836+0000 7f6827885b80 1 --2- 172.21.15.33:0/3379357933 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa54a0e7b0 0x55aa54a1c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:48.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.836+0000 7f6827885b80 1 -- 172.21.15.33:0/3379357933 >> 172.21.15.33:0/3379357933 conn(0x55aa54938b60 msgr2=0x55aa54a03ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:32:48.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.837+0000 7f6827885b80 1 -- 172.21.15.33:0/3379357933 shutdown_connections 2024-05-03T18:32:48.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.838+0000 7f6827885b80 1 -- 172.21.15.33:0/3379357933 wait complete. 2024-05-03T18:32:48.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.838+0000 7f6827885b80 20 client.25657 trim_cache size 0 max 16384 2024-05-03T18:32:48.839 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-03T18:32:48.839 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-05-03T18:32:48.839 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-05-03T18:32:48.839 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-03T18:32:48.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.839+0000 7f681ce00700 1 -- 172.21.15.33:0/2445527372 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f67fc0b2af0 msgr2=0x7f67fc0b4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:32:48.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.839+0000 7f681ce00700 1 --2- 172.21.15.33:0/2445527372 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f67fc0b2af0 0x7f67fc0b4fb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6810006130 tx=0x7f68100077f0 comp rx=0 tx=0).stop 2024-05-03T18:32:48.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.839+0000 7f681ce00700 1 -- 172.21.15.33:0/2445527372 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68181023a0 msgr2=0x7f68181986f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:32:48.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.839+0000 7f681ce00700 1 --2- 172.21.15.33:0/2445527372 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68181023a0 0x7f68181986f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6808000c00 tx=0x7f680800b610 comp rx=0 tx=0).stop 2024-05-03T18:32:48.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.839+0000 7f681ce00700 1 -- 172.21.15.33:0/2445527372 shutdown_connections 2024-05-03T18:32:48.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.839+0000 7f681ce00700 1 --2- 172.21.15.33:0/2445527372 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f67fc0b2af0 0x7f67fc0b4fb0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:48.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.839+0000 7f681ce00700 1 --2- 172.21.15.33:0/2445527372 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6818108410 0x7f681819d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:48.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.839+0000 7f681ce00700 1 --2- 172.21.15.33:0/2445527372 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68181023a0 0x7f68181986f0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:48.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.839+0000 7f681ce00700 1 --2- 172.21.15.33:0/2445527372 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6818102d60 0x7f6818198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:48.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.839+0000 7f681ce00700 1 -- 172.21.15.33:0/2445527372 >> 172.21.15.33:0/2445527372 conn(0x7f6818079350 msgr2=0x7f6818101100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:32:48.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.840+0000 7f681ce00700 1 -- 172.21.15.33:0/2445527372 shutdown_connections 2024-05-03T18:32:48.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:48.840+0000 7f681ce00700 1 -- 172.21.15.33:0/2445527372 wait complete. 2024-05-03T18:32:49.803 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-03T18:32:49.804 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T18:32:50.111 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 95.0 s, 616.2 tps, lat 161.397 ms stddev 46.931 2024-05-03T18:32:50.563 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:32:51.115 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:32:50 smithi033 ceph-mon[28330]: pgmap v172: 129 pgs: 129 active+clean; 1.7 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.2 MiB/s wr, 609 op/s 2024-05-03T18:32:51.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:32:50 smithi099 ceph-mon[31658]: pgmap v172: 129 pgs: 129 active+clean; 1.7 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.2 MiB/s wr, 609 op/s 2024-05-03T18:32:51.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:32:50 smithi150 ceph-mon[32492]: pgmap v172: 129 pgs: 129 active+clean; 1.7 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.2 MiB/s wr, 609 op/s 2024-05-03T18:32:53.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:32:52 smithi099 ceph-mon[31658]: pgmap v173: 129 pgs: 129 active+clean; 1.7 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.2 MiB/s wr, 609 op/s 2024-05-03T18:32:53.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:32:52 smithi150 ceph-mon[32492]: pgmap v173: 129 pgs: 129 active+clean; 1.7 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.2 MiB/s wr, 609 op/s 2024-05-03T18:32:53.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:32:52 smithi033 ceph-mon[28330]: pgmap v173: 129 pgs: 129 active+clean; 1.7 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.2 MiB/s wr, 609 op/s 2024-05-03T18:32:54.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:54.866+0000 7fe382400700 1 -- 172.21.15.33:0/3791467410 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe37c0756d0 msgr2=0x7fe37c075ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:32:54.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:54.866+0000 7fe382400700 1 --2- 172.21.15.33:0/3791467410 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe37c0756d0 0x7fe37c075ab0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe36c0098f0 tx=0x7fe36c009c00 comp rx=0 tx=0).stop 2024-05-03T18:32:54.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:54.868+0000 7fe382400700 1 -- 172.21.15.33:0/3791467410 shutdown_connections 2024-05-03T18:32:54.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:54.868+0000 7fe382400700 1 --2- 172.21.15.33:0/3791467410 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe37c10ff00 0x7fe37c1122f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:54.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:54.868+0000 7fe382400700 1 --2- 172.21.15.33:0/3791467410 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe37c0756d0 0x7fe37c075ab0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:54.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:54.868+0000 7fe382400700 1 --2- 172.21.15.33:0/3791467410 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe37c075ff0 0x7fe37c10f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:54.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:54.868+0000 7fe382400700 1 -- 172.21.15.33:0/3791467410 >> 172.21.15.33:0/3791467410 conn(0x7fe37c07a600 msgr2=0x7fe37c07aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:32:54.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:54.868+0000 7fe382400700 1 -- 172.21.15.33:0/3791467410 shutdown_connections 2024-05-03T18:32:54.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:54.868+0000 7fe382400700 1 -- 172.21.15.33:0/3791467410 wait complete. 2024-05-03T18:32:54.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:54.870+0000 7fe382400700 1 Processor -- start 2024-05-03T18:32:54.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:54.870+0000 7fe382400700 1 -- start start 2024-05-03T18:32:54.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:54.870+0000 7fe382400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe37c0756d0 0x7fe37c1a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:32:54.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:54.870+0000 7fe382400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe37c075ff0 0x7fe37c1a1c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:32:54.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:54.870+0000 7fe382400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe37c10ff00 0x7fe37c19b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:32:54.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:54.870+0000 7fe382400700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe37c1157c0 con 0x7fe37c075ff0 2024-05-03T18:32:54.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:54.870+0000 7fe382400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe37c115640 con 0x7fe37c0756d0 2024-05-03T18:32:54.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:54.870+0000 7fe382400700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe37c115940 con 0x7fe37c10ff00 2024-05-03T18:32:54.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:54.871+0000 7fe37aa00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe37c075ff0 0x7fe37c1a1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:32:54.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:54.872+0000 7fe37be00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe37c10ff00 0x7fe37c19b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:32:54.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:54.872+0000 7fe37aa00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe37c075ff0 0x7fe37c1a1c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57844/0 (socket says 172.21.15.33:57844) 2024-05-03T18:32:54.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:54.872+0000 7fe37aa00700 1 -- 172.21.15.33:0/4087805411 learned_addr learned my addr 172.21.15.33:0/4087805411 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:32:54.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:54.873+0000 7fe37b400700 1 --2- 172.21.15.33:0/4087805411 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe37c0756d0 0x7fe37c1a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:32:54.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:54.874+0000 7fe37be00700 1 -- 172.21.15.33:0/4087805411 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe37c0756d0 msgr2=0x7fe37c1a1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:32:54.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:54.874+0000 7fe37be00700 1 --2- 172.21.15.33:0/4087805411 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe37c0756d0 0x7fe37c1a1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:54.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:54.874+0000 7fe37be00700 1 -- 172.21.15.33:0/4087805411 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe37c075ff0 msgr2=0x7fe37c1a1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:32:54.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:54.874+0000 7fe37be00700 1 --2- 172.21.15.33:0/4087805411 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe37c075ff0 0x7fe37c1a1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:54.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:54.874+0000 7fe37be00700 1 -- 172.21.15.33:0/4087805411 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe37c19c080 con 0x7fe37c10ff00 2024-05-03T18:32:54.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:54.874+0000 7fe37be00700 1 --2- 172.21.15.33:0/4087805411 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe37c10ff00 0x7fe37c19b920 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe37400bdf0 tx=0x7fe374009e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:32:54.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:54.875+0000 7fe373e00700 1 -- 172.21.15.33:0/4087805411 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe374019070 con 0x7fe37c10ff00 2024-05-03T18:32:54.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:54.875+0000 7fe373e00700 1 -- 172.21.15.33:0/4087805411 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe374014440 con 0x7fe37c10ff00 2024-05-03T18:32:54.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:54.875+0000 7fe382400700 1 -- 172.21.15.33:0/4087805411 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe37c19c370 con 0x7fe37c10ff00 2024-05-03T18:32:54.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:54.876+0000 7fe373e00700 1 -- 172.21.15.33:0/4087805411 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3740125a0 con 0x7fe37c10ff00 2024-05-03T18:32:54.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:54.876+0000 7fe382400700 1 -- 172.21.15.33:0/4087805411 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe37c1a8660 con 0x7fe37c10ff00 2024-05-03T18:32:54.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:54.877+0000 7fe382400700 1 -- 172.21.15.33:0/4087805411 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe35c005330 con 0x7fe37c10ff00 2024-05-03T18:32:54.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:54.881+0000 7fe373e00700 1 -- 172.21.15.33:0/4087805411 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fe37401c070 con 0x7fe37c10ff00 2024-05-03T18:32:54.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:54.885+0000 7fe373e00700 1 --2- 172.21.15.33:0/4087805411 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fe3600b2a70 0x7fe3600b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:32:54.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:54.885+0000 7fe373e00700 1 -- 172.21.15.33:0/4087805411 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7fe3740e2560 con 0x7fe37c10ff00 2024-05-03T18:32:54.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:54.886+0000 7fe37b400700 1 --2- 172.21.15.33:0/4087805411 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fe3600b2a70 0x7fe3600b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:32:54.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:54.888+0000 7fe37b400700 1 --2- 172.21.15.33:0/4087805411 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fe3600b2a70 0x7fe3600b4f30 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe36c0094f0 tx=0x7fe36c019040 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:32:54.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:54.897+0000 7fe373e00700 1 -- 172.21.15.33:0/4087805411 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fe374023070 con 0x7fe37c10ff00 2024-05-03T18:32:55.110 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 100.0 s, 551.9 tps, lat 180.890 ms stddev 97.334 2024-05-03T18:32:55.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:32:54 smithi033 ceph-mon[28330]: pgmap v174: 129 pgs: 129 active+clean; 1.8 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.6 MiB/s wr, 906 op/s 2024-05-03T18:32:55.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:32:54 smithi099 ceph-mon[31658]: pgmap v174: 129 pgs: 129 active+clean; 1.8 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.6 MiB/s wr, 906 op/s 2024-05-03T18:32:55.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:32:54 smithi150 ceph-mon[32492]: pgmap v174: 129 pgs: 129 active+clean; 1.8 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.6 MiB/s wr, 906 op/s 2024-05-03T18:32:55.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:55.448+0000 7fe382400700 1 -- 172.21.15.33:0/4087805411 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe35c005620 con 0x7fe37c10ff00 2024-05-03T18:32:55.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:55.451+0000 7fe373e00700 1 -- 172.21.15.33:0/4087805411 <== mon.1 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11801 (secure 0 0 0) 0x7fe3740a9c20 con 0x7fe37c10ff00 2024-05-03T18:32:55.452 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:32:55.452 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1327393503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1327393503},{"type":"v1","addr":"172.21.15.33:6835","nonce":1327393503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/2461472948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":2461472948},{"type":"v1","addr":"172.21.15.33:6837","nonce":2461472948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14559,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/442269281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":442269281},{"type":"v1","addr":"172.21.15.33:6839","nonce":442269281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14571,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/645571756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":645571756},{"type":"v1","addr":"172.21.15.33:6841","nonce":645571756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/203598909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":203598909},{"type":"v1","addr":"172.21.15.150:6833","nonce":203598909}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/4029842493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":4029842493},{"type":"v1","addr":"172.21.15.150:6835","nonce":4029842493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3463664423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3463664423},{"type":"v1","addr":"172.21.15.150:6837","nonce":3463664423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24481,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6839/4051648018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":4051648018},{"type":"v1","addr":"172.21.15.150:6839","nonce":4051648018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34418,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1865105479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1865105479},{"type":"v1","addr":"172.21.15.99:6833","nonce":1865105479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34445,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/1206393914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1206393914},{"type":"v1","addr":"172.21.15.99:6835","nonce":1206393914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-03T18:22:28.837091+0000","modified":"2024-05-03T18:23:49.758398+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34508},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34472":{"gid":34472,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6837/3360970342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3360970342},{"type":"v1","addr":"172.21.15.99:6837","nonce":3360970342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34508":{"gid":34508,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/919907308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":919907308},{"type":"v1","addr":"172.21.15.99:6839","nonce":919907308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34508,"qdb_cluster":[34508]},"id":1}]} 2024-05-03T18:32:55.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:55.455+0000 7fe382400700 1 -- 172.21.15.33:0/4087805411 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fe3600b2a70 msgr2=0x7fe3600b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:32:55.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:55.455+0000 7fe382400700 1 --2- 172.21.15.33:0/4087805411 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fe3600b2a70 0x7fe3600b4f30 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe36c0094f0 tx=0x7fe36c019040 comp rx=0 tx=0).stop 2024-05-03T18:32:55.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:55.455+0000 7fe382400700 1 -- 172.21.15.33:0/4087805411 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe37c10ff00 msgr2=0x7fe37c19b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:32:55.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:55.455+0000 7fe382400700 1 --2- 172.21.15.33:0/4087805411 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe37c10ff00 0x7fe37c19b920 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe37400bdf0 tx=0x7fe374009e20 comp rx=0 tx=0).stop 2024-05-03T18:32:55.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:55.460+0000 7fe382400700 1 -- 172.21.15.33:0/4087805411 shutdown_connections 2024-05-03T18:32:55.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:55.460+0000 7fe382400700 1 --2- 172.21.15.33:0/4087805411 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe37c10ff00 0x7fe37c19b920 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:55.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:55.460+0000 7fe382400700 1 --2- 172.21.15.33:0/4087805411 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fe3600b2a70 0x7fe3600b4f30 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:55.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:55.460+0000 7fe382400700 1 --2- 172.21.15.33:0/4087805411 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe37c0756d0 0x7fe37c1a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:55.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:55.460+0000 7fe382400700 1 --2- 172.21.15.33:0/4087805411 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe37c075ff0 0x7fe37c1a1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:32:55.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:55.460+0000 7fe382400700 1 -- 172.21.15.33:0/4087805411 >> 172.21.15.33:0/4087805411 conn(0x7fe37c07a600 msgr2=0x7fe37c1005d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:32:55.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:55.462+0000 7fe382400700 1 -- 172.21.15.33:0/4087805411 shutdown_connections 2024-05-03T18:32:55.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:32:55.463+0000 7fe382400700 1 -- 172.21.15.33:0/4087805411 wait complete. 2024-05-03T18:32:55.466 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 21 2024-05-03T18:32:56.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:32:56 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/4087805411' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:32:56.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:32:56 smithi033 ceph-mon[28330]: pgmap v175: 129 pgs: 129 active+clean; 1.8 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.1 MiB/s wr, 609 op/s 2024-05-03T18:32:56.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:32:56 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/4087805411' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:32:56.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:32:56 smithi099 ceph-mon[31658]: pgmap v175: 129 pgs: 129 active+clean; 1.8 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.1 MiB/s wr, 609 op/s 2024-05-03T18:32:56.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:32:56 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/4087805411' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:32:56.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:32:56 smithi150 ceph-mon[32492]: pgmap v175: 129 pgs: 129 active+clean; 1.8 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.1 MiB/s wr, 609 op/s 2024-05-03T18:32:56.925 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-03T18:32:57.691 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:32:59.225 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:32:58 smithi033 ceph-mon[28330]: pgmap v176: 129 pgs: 129 active+clean; 1.8 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.4 MiB/s wr, 871 op/s 2024-05-03T18:32:59.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:32:58 smithi099 ceph-mon[31658]: pgmap v176: 129 pgs: 129 active+clean; 1.8 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.4 MiB/s wr, 871 op/s 2024-05-03T18:32:59.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:32:58 smithi150 ceph-mon[32492]: pgmap v176: 129 pgs: 129 active+clean; 1.8 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.4 MiB/s wr, 871 op/s 2024-05-03T18:33:00.110 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 105.0 s, 606.4 tps, lat 165.561 ms stddev 54.620 2024-05-03T18:33:00.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.285+0000 7fe87cc00700 1 -- 172.21.15.33:0/1037179455 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe8781165f0 msgr2=0x7fe8781189e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:33:00.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.285+0000 7fe87cc00700 1 --2- 172.21.15.33:0/1037179455 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe8781165f0 0x7fe8781189e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe87000b0a0 tx=0x7fe87000b3b0 comp rx=0 tx=0).stop 2024-05-03T18:33:00.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.287+0000 7fe87cc00700 1 -- 172.21.15.33:0/1037179455 shutdown_connections 2024-05-03T18:33:00.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.287+0000 7fe87cc00700 1 --2- 172.21.15.33:0/1037179455 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe8781165f0 0x7fe8781189e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:00.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.287+0000 7fe87cc00700 1 --2- 172.21.15.33:0/1037179455 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe87810a4a0 0x7fe87810a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:00.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.287+0000 7fe87cc00700 1 --2- 172.21.15.33:0/1037179455 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe87810adc0 0x7fe8781160b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:00.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.287+0000 7fe87cc00700 1 -- 172.21.15.33:0/1037179455 >> 172.21.15.33:0/1037179455 conn(0x7fe87806e610 msgr2=0x7fe87806ea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:33:00.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.288+0000 7fe87cc00700 1 -- 172.21.15.33:0/1037179455 shutdown_connections 2024-05-03T18:33:00.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.288+0000 7fe87cc00700 1 -- 172.21.15.33:0/1037179455 wait complete. 2024-05-03T18:33:00.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.289+0000 7fe87cc00700 1 Processor -- start 2024-05-03T18:33:00.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.292+0000 7fe87cc00700 1 -- start start 2024-05-03T18:33:00.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.292+0000 7fe87cc00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe87810a4a0 0x7fe87810bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:00.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.292+0000 7fe87cc00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe87810adc0 0x7fe87810c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:00.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.292+0000 7fe87cc00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe8781165f0 0x7fe87810c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:00.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.293+0000 7fe87cc00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe87811be50 con 0x7fe87810a4a0 2024-05-03T18:33:00.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.293+0000 7fe875400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe87810adc0 0x7fe87810c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:00.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.293+0000 7fe875400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe87810adc0 0x7fe87810c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:40396/0 (socket says 172.21.15.33:40396) 2024-05-03T18:33:00.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.293+0000 7fe875400700 1 -- 172.21.15.33:0/3631114509 learned_addr learned my addr 172.21.15.33:0/3631114509 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:33:00.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.293+0000 7fe87cc00700 1 -- 172.21.15.33:0/3631114509 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe87811bcd0 con 0x7fe8781165f0 2024-05-03T18:33:00.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.293+0000 7fe87cc00700 1 -- 172.21.15.33:0/3631114509 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe87811bfd0 con 0x7fe87810adc0 2024-05-03T18:33:00.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.293+0000 7fe875e00700 1 --2- 172.21.15.33:0/3631114509 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe87810a4a0 0x7fe87810bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:00.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.294+0000 7fe875400700 1 -- 172.21.15.33:0/3631114509 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe8781165f0 msgr2=0x7fe87810c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:33:00.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.294+0000 7fe875400700 1 --2- 172.21.15.33:0/3631114509 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe8781165f0 0x7fe87810c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:00.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.294+0000 7fe875400700 1 -- 172.21.15.33:0/3631114509 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe87810a4a0 msgr2=0x7fe87810bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:33:00.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.294+0000 7fe875400700 1 --2- 172.21.15.33:0/3631114509 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe87810a4a0 0x7fe87810bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:00.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.294+0000 7fe875400700 1 -- 172.21.15.33:0/3631114509 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8781b0890 con 0x7fe87810adc0 2024-05-03T18:33:00.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.294+0000 7fe875e00700 1 --2- 172.21.15.33:0/3631114509 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe87810a4a0 0x7fe87810bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:33:00.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.295+0000 7fe875400700 1 --2- 172.21.15.33:0/3631114509 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe87810adc0 0x7fe87810c450 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe86400b550 tx=0x7fe86400b860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:33:00.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.296+0000 7fe86ea00700 1 -- 172.21.15.33:0/3631114509 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe864015020 con 0x7fe87810adc0 2024-05-03T18:33:00.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.297+0000 7fe87cc00700 1 -- 172.21.15.33:0/3631114509 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8781b0b80 con 0x7fe87810adc0 2024-05-03T18:33:00.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.297+0000 7fe86ea00700 1 -- 172.21.15.33:0/3631114509 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe864011cc0 con 0x7fe87810adc0 2024-05-03T18:33:00.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.297+0000 7fe87cc00700 1 -- 172.21.15.33:0/3631114509 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8781b1070 con 0x7fe87810adc0 2024-05-03T18:33:00.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.297+0000 7fe86ea00700 1 -- 172.21.15.33:0/3631114509 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe864010d10 con 0x7fe87810adc0 2024-05-03T18:33:00.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.306+0000 7fe8875efb80 1 Processor -- start 2024-05-03T18:33:00.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.307+0000 7fe8875efb80 1 -- start start 2024-05-03T18:33:00.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.308+0000 7fe8875efb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5606d37d3930 0x5606d37d3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:00.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.308+0000 7fe8875efb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606d37d4250 0x5606d37cdf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:00.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.308+0000 7fe8875efb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5606d37ce550 0x5606d37ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:00.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.308+0000 7fe8875efb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5606d3720a20 con 0x5606d37d4250 2024-05-03T18:33:00.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.308+0000 7fe8875efb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5606d36be760 con 0x5606d37d3930 2024-05-03T18:33:00.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.308+0000 7fe8875efb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5606d371c9e0 con 0x5606d37ce550 2024-05-03T18:33:00.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.309+0000 7fe86ea00700 1 -- 172.21.15.33:0/3631114509 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fe864018020 con 0x7fe87810adc0 2024-05-03T18:33:00.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.309+0000 7fe876800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5606d37d3930 0x5606d37d3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:00.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.309+0000 7fe876800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5606d37d3930 0x5606d37d3d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:60844/0 (socket says 172.21.15.33:60844) 2024-05-03T18:33:00.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.309+0000 7fe876800700 1 -- 172.21.15.33:0/1406931935 learned_addr learned my addr 172.21.15.33:0/1406931935 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:33:00.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.310+0000 7fe876800700 1 -- 172.21.15.33:0/1406931935 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5606d37ce550 msgr2=0x5606d37ce9d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-03T18:33:00.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.310+0000 7fe876800700 1 --2- 172.21.15.33:0/1406931935 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5606d37ce550 0x5606d37ce9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:00.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.310+0000 7fe876800700 1 -- 172.21.15.33:0/1406931935 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606d37d4250 msgr2=0x5606d37cdf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:33:00.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.310+0000 7fe876800700 1 --2- 172.21.15.33:0/1406931935 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606d37d4250 0x5606d37cdf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:00.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.310+0000 7fe876800700 1 -- 172.21.15.33:0/1406931935 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606d366d8d0 con 0x5606d37d3930 2024-05-03T18:33:00.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.312+0000 7fe876800700 1 --2- 172.21.15.33:0/1406931935 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5606d37d3930 0x5606d37d3d10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe87000cf10 tx=0x7fe87000cf40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1bfa94055066632 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:33:00.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.312+0000 7fe863400700 1 -- 172.21.15.33:0/1406931935 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe870009350 con 0x5606d37d3930 2024-05-03T18:33:00.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.312+0000 7fe863400700 1 -- 172.21.15.33:0/1406931935 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe8700074e0 con 0x5606d37d3930 2024-05-03T18:33:00.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.312+0000 7fe863400700 1 -- 172.21.15.33:0/1406931935 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe87001d690 con 0x5606d37d3930 2024-05-03T18:33:00.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.312+0000 7fe86ea00700 1 --2- 172.21.15.33:0/3631114509 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fe85c0bb230 0x7fe85c0bd6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:00.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.313+0000 7fe875e00700 1 --2- 172.21.15.33:0/3631114509 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fe85c0bb230 0x7fe85c0bd6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:00.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.313+0000 7fe8875efb80 1 -- 172.21.15.33:0/1406931935 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5606d37d3930 msgr2=0x5606d37d3d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:33:00.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.313+0000 7fe8875efb80 1 --2- 172.21.15.33:0/1406931935 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5606d37d3930 0x5606d37d3d10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe87000cf10 tx=0x7fe87000cf40 comp rx=0 tx=0).stop 2024-05-03T18:33:00.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.314+0000 7fe8875efb80 1 -- 172.21.15.33:0/1406931935 shutdown_connections 2024-05-03T18:33:00.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.314+0000 7fe8875efb80 1 --2- 172.21.15.33:0/1406931935 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5606d37ce550 0x5606d37ce9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:00.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.314+0000 7fe8875efb80 1 --2- 172.21.15.33:0/1406931935 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5606d37d3930 0x5606d37d3d10 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:00.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.314+0000 7fe8875efb80 1 --2- 172.21.15.33:0/1406931935 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606d37d4250 0x5606d37cdf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:00.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.314+0000 7fe8875efb80 1 -- 172.21.15.33:0/1406931935 >> 172.21.15.33:0/1406931935 conn(0x5606d37466d0 msgr2=0x5606d37c8900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:33:00.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.314+0000 7fe86ea00700 1 -- 172.21.15.33:0/3631114509 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7fe8640e1fe0 con 0x7fe87810adc0 2024-05-03T18:33:00.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.317+0000 7fe875e00700 1 --2- 172.21.15.33:0/3631114509 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fe85c0bb230 0x7fe85c0bd6f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe868005d90 tx=0x7fe868005d20 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:33:00.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.317+0000 7fe8875efb80 1 -- 172.21.15.33:0/1406931935 shutdown_connections 2024-05-03T18:33:00.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.317+0000 7fe8875efb80 1 -- 172.21.15.33:0/1406931935 wait complete. 2024-05-03T18:33:00.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.318+0000 7fe8875efb80 1 -- start start 2024-05-03T18:33:00.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.318+0000 7fe8875efb80 1 Processor -- start 2024-05-03T18:33:00.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.320+0000 7fe8875efb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5606d37ce550 0x5606d37d9e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:00.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.320+0000 7fe8875efb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606d37d3930 0x5606d37d71e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:00.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.320+0000 7fe875e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5606d37ce550 0x5606d37d9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:00.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.320+0000 7fe875e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5606d37ce550 0x5606d37d9e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:60846/0 (socket says 172.21.15.33:60846) 2024-05-03T18:33:00.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.321+0000 7fe875e00700 1 -- 172.21.15.33:0/2639874507 learned_addr learned my addr 172.21.15.33:0/2639874507 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:33:00.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.321+0000 7fe8875efb80 1 --2- 172.21.15.33:0/2639874507 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5606d37d4250 0x5606d37d7720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:00.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.321+0000 7fe8875efb80 1 -- 172.21.15.33:0/2639874507 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5606d37daca0 con 0x5606d37d3930 2024-05-03T18:33:00.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.321+0000 7fe8875efb80 1 -- 172.21.15.33:0/2639874507 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5606d3686b20 con 0x5606d37ce550 2024-05-03T18:33:00.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.321+0000 7fe8875efb80 1 -- 172.21.15.33:0/2639874507 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5606d37dae20 con 0x5606d37d4250 2024-05-03T18:33:00.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.321+0000 7fe8875efb80 10 client.0 _ll_register_callbacks cb 0x5606d36c7c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:33:00.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.321+0000 7fe875400700 1 --2- 172.21.15.33:0/2639874507 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606d37d3930 0x5606d37d71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:00.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.322+0000 7fe876800700 1 --2- 172.21.15.33:0/2639874507 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5606d37d4250 0x5606d37d7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:00.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.322+0000 7fe875400700 1 -- 172.21.15.33:0/2639874507 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5606d37d4250 msgr2=0x5606d37d7720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:33:00.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.322+0000 7fe875400700 1 --2- 172.21.15.33:0/2639874507 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5606d37d4250 0x5606d37d7720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:00.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.323+0000 7fe875400700 1 -- 172.21.15.33:0/2639874507 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5606d37ce550 msgr2=0x5606d37d9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:33:00.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.323+0000 7fe875400700 1 --2- 172.21.15.33:0/2639874507 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5606d37ce550 0x5606d37d9e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:00.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.323+0000 7fe875400700 1 -- 172.21.15.33:0/2639874507 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606d36e6420 con 0x5606d37d3930 2024-05-03T18:33:00.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.323+0000 7fe876800700 1 --2- 172.21.15.33:0/2639874507 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5606d37d4250 0x5606d37d7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:33:00.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.323+0000 7fe875e00700 1 --2- 172.21.15.33:0/2639874507 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5606d37ce550 0x5606d37d9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:33:00.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.324+0000 7fe875400700 1 --2- 172.21.15.33:0/2639874507 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606d37d3930 0x5606d37d71e0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fe8640df4b0 tx=0x7fe8640a8e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c39581f50abb48ee server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:33:00.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.324+0000 7fe857e00700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:33:00.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.325+0000 7fe857e00700 1 -- 172.21.15.33:0/2639874507 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8640a9000 con 0x5606d37d3930 2024-05-03T18:33:00.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.325+0000 7fe857e00700 1 -- 172.21.15.33:0/2639874507 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7fe8640b1c20 con 0x5606d37d3930 2024-05-03T18:33:00.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.325+0000 7fe8875efb80 1 -- 172.21.15.33:0/2639874507 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8700074e0 con 0x5606d37d3930 2024-05-03T18:33:00.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.326+0000 7fe857e00700 1 -- 172.21.15.33:0/2639874507 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fe8640a9bf0 con 0x5606d37d3930 2024-05-03T18:33:00.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.327+0000 7fe8875efb80 10 client.16002 fetch_fsmap learned FSMap version 21 2024-05-03T18:33:00.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.327+0000 7fe8875efb80 1 -- 172.21.15.33:0/2639874507 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5606d37e5430 con 0x5606d37d3930 2024-05-03T18:33:00.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.328+0000 7fe857e00700 1 -- 172.21.15.33:0/2639874507 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe8700074e0 con 0x5606d37d3930 2024-05-03T18:33:00.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.328+0000 7fe857e00700 1 -- 172.21.15.33:0/2639874507 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fe8640cb070 con 0x5606d37d3930 2024-05-03T18:33:00.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.328+0000 7fe8875efb80 10 client.16002 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:33:00.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.328+0000 7fe8875efb80 10 client.16002 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:33:00.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.328+0000 7fe8875efb80 20 client.16002 populate_metadata read hostname 'smithi033' 2024-05-03T18:33:00.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.328+0000 7fe8875efb80 1 --2- 172.21.15.33:0/2639874507 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5606d37e7e20 0x5606d37ea260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:00.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.328+0000 7fe8875efb80 4 client.16002 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:33:00.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.328+0000 7fe8875efb80 1 -- 172.21.15.33:0/2639874507 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5606d379aff0 con 0x5606d37e7e20 2024-05-03T18:33:00.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.329+0000 7fe875e00700 1 --2- 172.21.15.33:0/2639874507 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5606d37e7e20 0x5606d37ea260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:00.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.333+0000 7fe875e00700 1 --2- 172.21.15.33:0/2639874507 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5606d37e7e20 0x5606d37ea260 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50ed1d07ba669446 server_cookie=eb9aa4e4bd185b5b in_seq=0 out_seq=0 2024-05-03T18:33:00.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.333+0000 7fe857e00700 10 client.16002 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:33:00.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.335+0000 7fe857e00700 1 -- 172.21.15.33:0/2639874507 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5606d379aff0 con 0x5606d37e7e20 2024-05-03T18:33:00.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.335+0000 7fe857e00700 10 client.16002 handle_command_reply: tid=0 2024-05-03T18:33:00.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.335+0000 7fe8875efb80 1 client.16002 shutdown 2024-05-03T18:33:00.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.336+0000 7fe8875efb80 1 -- 172.21.15.33:0/2639874507 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606d37d3930 msgr2=0x5606d37d71e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:33:00.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.337+0000 7fe8875efb80 1 --2- 172.21.15.33:0/2639874507 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606d37d3930 0x5606d37d71e0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fe8640df4b0 tx=0x7fe8640a8e40 comp rx=0 tx=0).stop 2024-05-03T18:33:00.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.337+0000 7fe8875efb80 1 -- 172.21.15.33:0/2639874507 shutdown_connections 2024-05-03T18:33:00.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.337+0000 7fe8875efb80 1 --2- 172.21.15.33:0/2639874507 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5606d37e7e20 0x5606d37ea260 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:00.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.337+0000 7fe8875efb80 1 --2- 172.21.15.33:0/2639874507 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5606d37d4250 0x5606d37d7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:00.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.337+0000 7fe8875efb80 1 --2- 172.21.15.33:0/2639874507 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5606d37ce550 0x5606d37d9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:00.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.337+0000 7fe8875efb80 1 --2- 172.21.15.33:0/2639874507 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606d37d3930 0x5606d37d71e0 secure :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fe8640df4b0 tx=0x7fe8640a8e40 comp rx=0 tx=0).stop 2024-05-03T18:33:00.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.337+0000 7fe857e00700 0 client.16002 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:33:00.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.337+0000 7fe8875efb80 1 -- 172.21.15.33:0/2639874507 >> 172.21.15.33:0/2639874507 conn(0x5606d37466d0 msgr2=0x5606d37c33b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:33:00.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.338+0000 7fe8875efb80 1 -- 172.21.15.33:0/2639874507 shutdown_connections 2024-05-03T18:33:00.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.338+0000 7fe8875efb80 1 -- 172.21.15.33:0/2639874507 wait complete. 2024-05-03T18:33:00.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.339+0000 7fe8875efb80 20 client.16002 trim_cache size 0 max 16384 2024-05-03T18:33:00.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.373+0000 7fe8875efb80 1 Processor -- start 2024-05-03T18:33:00.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.373+0000 7fe8875efb80 1 -- start start 2024-05-03T18:33:00.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.374+0000 7fe8875efb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5606d37ce550 0x5606d37d6a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:00.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.374+0000 7fe8875efb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606d37d4250 0x5606d37d5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:00.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.374+0000 7fe8875efb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5606d37e3d60 0x5606d37d6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:00.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.374+0000 7fe8875efb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe8781b3ab0 con 0x5606d37d4250 2024-05-03T18:33:00.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.374+0000 7fe8875efb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe8781b6330 con 0x5606d37e3d60 2024-05-03T18:33:00.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.374+0000 7fe8875efb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe8781b4120 con 0x5606d37ce550 2024-05-03T18:33:00.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.374+0000 7fe875e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5606d37ce550 0x5606d37d6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:00.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.374+0000 7fe875e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5606d37ce550 0x5606d37d6a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:40432/0 (socket says 172.21.15.33:40432) 2024-05-03T18:33:00.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.374+0000 7fe875e00700 1 -- 172.21.15.33:0/1214991720 learned_addr learned my addr 172.21.15.33:0/1214991720 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:33:00.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.374+0000 7fe876800700 1 --2- 172.21.15.33:0/1214991720 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5606d37e3d60 0x5606d37d6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:00.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.375+0000 7fe876800700 1 -- 172.21.15.33:0/1214991720 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5606d37ce550 msgr2=0x5606d37d6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:33:00.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.375+0000 7fe876800700 1 --2- 172.21.15.33:0/1214991720 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5606d37ce550 0x5606d37d6a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:00.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.375+0000 7fe876800700 1 -- 172.21.15.33:0/1214991720 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606d37d4250 msgr2=0x5606d37d5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:33:00.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.375+0000 7fe875400700 1 --2- 172.21.15.33:0/1214991720 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606d37d4250 0x5606d37d5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:00.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.376+0000 7fe876800700 1 --2- 172.21.15.33:0/1214991720 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606d37d4250 0x5606d37d5ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:00.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.377+0000 7fe876800700 1 -- 172.21.15.33:0/1214991720 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606d37ea8e0 con 0x5606d37e3d60 2024-05-03T18:33:00.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.377+0000 7fe875400700 1 --2- 172.21.15.33:0/1214991720 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606d37d4250 0x5606d37d5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:33:00.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.377+0000 7fe876800700 1 --2- 172.21.15.33:0/1214991720 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5606d37e3d60 0x5606d37d6010 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe87000cf10 tx=0x7fe870009350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16899cb46ef80272 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:33:00.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.378+0000 7fe862a00700 1 -- 172.21.15.33:0/1214991720 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe870003a00 con 0x5606d37e3d60 2024-05-03T18:33:00.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.378+0000 7fe862a00700 1 -- 172.21.15.33:0/1214991720 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe870003ba0 con 0x5606d37e3d60 2024-05-03T18:33:00.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.378+0000 7fe862a00700 1 -- 172.21.15.33:0/1214991720 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8700074a0 con 0x5606d37e3d60 2024-05-03T18:33:00.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.379+0000 7fe8875efb80 1 -- 172.21.15.33:0/1214991720 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5606d37e3d60 msgr2=0x5606d37d6010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:33:00.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.379+0000 7fe8875efb80 1 --2- 172.21.15.33:0/1214991720 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5606d37e3d60 0x5606d37d6010 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe87000cf10 tx=0x7fe870009350 comp rx=0 tx=0).stop 2024-05-03T18:33:00.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.379+0000 7fe8875efb80 1 -- 172.21.15.33:0/1214991720 shutdown_connections 2024-05-03T18:33:00.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.379+0000 7fe8875efb80 1 --2- 172.21.15.33:0/1214991720 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5606d37e3d60 0x5606d37d6010 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:00.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.379+0000 7fe8875efb80 1 --2- 172.21.15.33:0/1214991720 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5606d37ce550 0x5606d37d6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:00.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.379+0000 7fe8875efb80 1 --2- 172.21.15.33:0/1214991720 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606d37d4250 0x5606d37d5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:00.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.379+0000 7fe8875efb80 1 -- 172.21.15.33:0/1214991720 >> 172.21.15.33:0/1214991720 conn(0x5606d37466d0 msgr2=0x5606d37d71a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:33:00.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.380+0000 7fe8875efb80 1 -- 172.21.15.33:0/1214991720 shutdown_connections 2024-05-03T18:33:00.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.380+0000 7fe8875efb80 1 -- 172.21.15.33:0/1214991720 wait complete. 2024-05-03T18:33:00.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.381+0000 7fe8875efb80 1 -- start start 2024-05-03T18:33:00.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.381+0000 7fe8875efb80 1 Processor -- start 2024-05-03T18:33:00.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.382+0000 7fe8875efb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5606d37ce550 0x5606d37ef670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:00.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.382+0000 7fe8875efb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606d37d4250 0x5606d37efbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:00.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.382+0000 7fe8875efb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5606d37e3d60 0x5606d37f0130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:00.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.383+0000 7fe875e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5606d37ce550 0x5606d37ef670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:00.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.384+0000 7fe876800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5606d37e3d60 0x5606d37f0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:00.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.384+0000 7fe876800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5606d37e3d60 0x5606d37f0130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:60862/0 (socket says 172.21.15.33:60862) 2024-05-03T18:33:00.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.384+0000 7fe876800700 1 -- 172.21.15.33:0/3266893148 learned_addr learned my addr 172.21.15.33:0/3266893148 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:33:00.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.383+0000 7fe875e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5606d37ce550 0x5606d37ef670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:40434/0 (socket says 172.21.15.33:40434) 2024-05-03T18:33:00.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.384+0000 7fe8875efb80 1 -- 172.21.15.33:0/3266893148 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe8781b47f0 con 0x5606d37d4250 2024-05-03T18:33:00.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.384+0000 7fe8875efb80 1 -- 172.21.15.33:0/3266893148 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe8781b5c60 con 0x5606d37e3d60 2024-05-03T18:33:00.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.384+0000 7fe8875efb80 1 -- 172.21.15.33:0/3266893148 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe8781b4ec0 con 0x5606d37ce550 2024-05-03T18:33:00.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.385+0000 7fe8875efb80 10 client.0 _ll_register_callbacks cb 0x5606d36c7c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:33:00.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.385+0000 7fe876800700 1 -- 172.21.15.33:0/3266893148 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5606d37ce550 msgr2=0x5606d37ef670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:33:00.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.385+0000 7fe876800700 1 --2- 172.21.15.33:0/3266893148 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5606d37ce550 0x5606d37ef670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:00.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.385+0000 7fe876800700 1 -- 172.21.15.33:0/3266893148 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606d37d4250 msgr2=0x5606d37efbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:33:00.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.385+0000 7fe876800700 1 --2- 172.21.15.33:0/3266893148 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606d37d4250 0x5606d37efbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:00.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.385+0000 7fe876800700 1 -- 172.21.15.33:0/3266893148 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606d37ced70 con 0x5606d37e3d60 2024-05-03T18:33:00.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.386+0000 7fe876800700 1 --2- 172.21.15.33:0/3266893148 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5606d37e3d60 0x5606d37f0130 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe870007930 tx=0x7fe870007960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9dbc733d0773b26 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:33:00.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.386+0000 7fe857e00700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-03T18:33:00.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.386+0000 7fe857e00700 1 -- 172.21.15.33:0/3266893148 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe870033030 con 0x5606d37e3d60 2024-05-03T18:33:00.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.387+0000 7fe857e00700 1 -- 172.21.15.33:0/3266893148 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7fe870009440 con 0x5606d37e3d60 2024-05-03T18:33:00.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.387+0000 7fe8875efb80 1 -- 172.21.15.33:0/3266893148 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe870003ba0 con 0x5606d37e3d60 2024-05-03T18:33:00.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.387+0000 7fe857e00700 1 -- 172.21.15.33:0/3266893148 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fe870007dc0 con 0x5606d37e3d60 2024-05-03T18:33:00.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.387+0000 7fe8875efb80 10 client.35621 fetch_fsmap learned FSMap version 21 2024-05-03T18:33:00.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.388+0000 7fe8875efb80 1 -- 172.21.15.33:0/3266893148 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5606d37e7c00 con 0x5606d37e3d60 2024-05-03T18:33:00.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.388+0000 7fe857e00700 1 -- 172.21.15.33:0/3266893148 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe870003ba0 con 0x5606d37e3d60 2024-05-03T18:33:00.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.388+0000 7fe857e00700 1 -- 172.21.15.33:0/3266893148 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fe870011070 con 0x5606d37e3d60 2024-05-03T18:33:00.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.389+0000 7fe8875efb80 10 client.35621 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:33:00.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.389+0000 7fe8875efb80 10 client.35621 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:33:00.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.389+0000 7fe8875efb80 20 client.35621 populate_metadata read hostname 'smithi033' 2024-05-03T18:33:00.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.389+0000 7fe8875efb80 1 --2- 172.21.15.33:0/3266893148 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5606d3852ae0 0x5606d3854ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:00.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.389+0000 7fe8875efb80 4 client.35621 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-03T18:33:00.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.389+0000 7fe8875efb80 1 -- 172.21.15.33:0/3266893148 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5606d372e400 con 0x5606d3852ae0 2024-05-03T18:33:00.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.390+0000 7fe875e00700 1 --2- 172.21.15.33:0/3266893148 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5606d3852ae0 0x5606d3854ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:00.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.391+0000 7fe875e00700 1 --2- 172.21.15.33:0/3266893148 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5606d3852ae0 0x5606d3854ed0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e60e7847e7c6cac server_cookie=f22f1342987a6347 in_seq=0 out_seq=0 2024-05-03T18:33:00.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.391+0000 7fe857e00700 10 client.35621 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:33:00.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.391+0000 7fe857e00700 1 -- 172.21.15.33:0/3266893148 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5606d372e400 con 0x5606d3852ae0 2024-05-03T18:33:00.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.391+0000 7fe857e00700 10 client.35621 handle_command_reply: tid=0 2024-05-03T18:33:00.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.391+0000 7fe8875efb80 1 client.35621 shutdown 2024-05-03T18:33:00.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.392+0000 7fe8875efb80 1 -- 172.21.15.33:0/3266893148 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5606d37e3d60 msgr2=0x5606d37f0130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:33:00.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.392+0000 7fe8875efb80 1 --2- 172.21.15.33:0/3266893148 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5606d37e3d60 0x5606d37f0130 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe870007930 tx=0x7fe870007960 comp rx=0 tx=0).stop 2024-05-03T18:33:00.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.393+0000 7fe8875efb80 1 -- 172.21.15.33:0/3266893148 shutdown_connections 2024-05-03T18:33:00.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.393+0000 7fe8875efb80 1 --2- 172.21.15.33:0/3266893148 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5606d3852ae0 0x5606d3854ed0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:00.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.393+0000 7fe8875efb80 1 --2- 172.21.15.33:0/3266893148 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5606d37e3d60 0x5606d37f0130 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:00.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.393+0000 7fe8875efb80 1 --2- 172.21.15.33:0/3266893148 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5606d37ce550 0x5606d37ef670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:00.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.393+0000 7fe8875efb80 1 --2- 172.21.15.33:0/3266893148 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606d37d4250 0x5606d37efbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:00.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.393+0000 7fe8875efb80 1 -- 172.21.15.33:0/3266893148 >> 172.21.15.33:0/3266893148 conn(0x5606d37466d0 msgr2=0x5606d37d5a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:33:00.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.393+0000 7fe857e00700 0 client.35621 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:33:00.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.393+0000 7fe8875efb80 1 -- 172.21.15.33:0/3266893148 shutdown_connections 2024-05-03T18:33:00.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.393+0000 7fe8875efb80 1 -- 172.21.15.33:0/3266893148 wait complete. 2024-05-03T18:33:00.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.394+0000 7fe8875efb80 20 client.35621 trim_cache size 0 max 16384 2024-05-03T18:33:00.395 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-03T18:33:00.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.395+0000 7fe87cc00700 1 -- 172.21.15.33:0/3631114509 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fe85c0bb230 msgr2=0x7fe85c0bd6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:33:00.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.395+0000 7fe87cc00700 1 --2- 172.21.15.33:0/3631114509 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fe85c0bb230 0x7fe85c0bd6f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe868005d90 tx=0x7fe868005d20 comp rx=0 tx=0).stop 2024-05-03T18:33:00.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.395+0000 7fe87cc00700 1 -- 172.21.15.33:0/3631114509 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe87810adc0 msgr2=0x7fe87810c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:33:00.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.395+0000 7fe87cc00700 1 --2- 172.21.15.33:0/3631114509 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe87810adc0 0x7fe87810c450 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe86400b550 tx=0x7fe86400b860 comp rx=0 tx=0).stop 2024-05-03T18:33:00.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.395+0000 7fe87cc00700 1 -- 172.21.15.33:0/3631114509 shutdown_connections 2024-05-03T18:33:00.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.395+0000 7fe87cc00700 1 --2- 172.21.15.33:0/3631114509 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fe85c0bb230 0x7fe85c0bd6f0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:00.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.395+0000 7fe87cc00700 1 --2- 172.21.15.33:0/3631114509 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe8781165f0 0x7fe87810c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:00.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.395+0000 7fe87cc00700 1 --2- 172.21.15.33:0/3631114509 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe87810a4a0 0x7fe87810bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:00.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.395+0000 7fe87cc00700 1 --2- 172.21.15.33:0/3631114509 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe87810adc0 0x7fe87810c450 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:00.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.395+0000 7fe87cc00700 1 -- 172.21.15.33:0/3631114509 >> 172.21.15.33:0/3631114509 conn(0x7fe87806e610 msgr2=0x7fe878072280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:33:00.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.395+0000 7fe87cc00700 1 -- 172.21.15.33:0/3631114509 shutdown_connections 2024-05-03T18:33:00.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:00.396+0000 7fe87cc00700 1 -- 172.21.15.33:0/3631114509 wait complete. 2024-05-03T18:33:01.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:33:00 smithi033 ceph-mon[28330]: pgmap v177: 129 pgs: 129 active+clean; 1.8 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.8 MiB/s wr, 558 op/s 2024-05-03T18:33:01.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:33:00 smithi099 ceph-mon[31658]: pgmap v177: 129 pgs: 129 active+clean; 1.8 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.8 MiB/s wr, 558 op/s 2024-05-03T18:33:01.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:33:00 smithi150 ceph-mon[32492]: pgmap v177: 129 pgs: 129 active+clean; 1.8 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.8 MiB/s wr, 558 op/s 2024-05-03T18:33:02.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:33:02 smithi150 ceph-mon[32492]: pgmap v178: 129 pgs: 129 active+clean; 1.8 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.8 MiB/s wr, 558 op/s 2024-05-03T18:33:02.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:33:02 smithi033 ceph-mon[28330]: pgmap v178: 129 pgs: 129 active+clean; 1.8 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.8 MiB/s wr, 558 op/s 2024-05-03T18:33:02.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:33:02 smithi099 ceph-mon[31658]: pgmap v178: 129 pgs: 129 active+clean; 1.8 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.8 MiB/s wr, 558 op/s 2024-05-03T18:33:04.496 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-03T18:33:04.497 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-03T18:33:05.110 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 110.0 s, 547.6 tps, lat 182.958 ms stddev 301.925 2024-05-03T18:33:05.190 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:33:05.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:33:04 smithi099 ceph-mon[31658]: pgmap v179: 129 pgs: 129 active+clean; 1.8 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 7.9 MiB/s wr, 903 op/s 2024-05-03T18:33:05.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:33:04 smithi150 ceph-mon[32492]: pgmap v179: 129 pgs: 129 active+clean; 1.8 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 7.9 MiB/s wr, 903 op/s 2024-05-03T18:33:05.387 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:33:04 smithi033 ceph-mon[28330]: pgmap v179: 129 pgs: 129 active+clean; 1.8 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 7.9 MiB/s wr, 903 op/s 2024-05-03T18:33:07.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:33:06 smithi033 ceph-mon[28330]: pgmap v180: 129 pgs: 129 active+clean; 1.8 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.4 MiB/s wr, 606 op/s 2024-05-03T18:33:07.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:33:07 smithi099 ceph-mon[31658]: pgmap v180: 129 pgs: 129 active+clean; 1.8 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.4 MiB/s wr, 606 op/s 2024-05-03T18:33:07.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:33:07 smithi150 ceph-mon[32492]: pgmap v180: 129 pgs: 129 active+clean; 1.8 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.4 MiB/s wr, 606 op/s 2024-05-03T18:33:07.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.867+0000 7feff3000700 1 -- 172.21.15.33:0/2172163252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fefec102450 msgr2=0x7fefec110340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:33:07.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.867+0000 7feff3000700 1 --2- 172.21.15.33:0/2172163252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fefec102450 0x7fefec110340 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fefe400b0a0 tx=0x7fefe400b3b0 comp rx=0 tx=0).stop 2024-05-03T18:33:07.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.867+0000 7feff3000700 1 -- 172.21.15.33:0/2172163252 shutdown_connections 2024-05-03T18:33:07.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.868+0000 7feff3000700 1 --2- 172.21.15.33:0/2172163252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fefec102450 0x7fefec110340 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:07.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.868+0000 7feff3000700 1 --2- 172.21.15.33:0/2172163252 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fefec109520 0x7fefec109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:07.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.868+0000 7feff3000700 1 --2- 172.21.15.33:0/2172163252 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fefec101a90 0x7fefec101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:07.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.868+0000 7feff3000700 1 -- 172.21.15.33:0/2172163252 >> 172.21.15.33:0/2172163252 conn(0x7fefec078a10 msgr2=0x7fefec0ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:33:07.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.868+0000 7feff3000700 1 -- 172.21.15.33:0/2172163252 shutdown_connections 2024-05-03T18:33:07.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.869+0000 7feff3000700 1 -- 172.21.15.33:0/2172163252 wait complete. 2024-05-03T18:33:07.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.870+0000 7feff3000700 1 Processor -- start 2024-05-03T18:33:07.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.871+0000 7feff3000700 1 -- start start 2024-05-03T18:33:07.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.871+0000 7feff3000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fefec101a90 0x7fefec1a18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:07.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.871+0000 7feff3000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fefec102450 0x7fefec1a1df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:07.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.871+0000 7feff3000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fefec109520 0x7fefec19ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:07.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.871+0000 7feff3000700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fefec113530 con 0x7fefec102450 2024-05-03T18:33:07.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.871+0000 7feff3000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fefec1133b0 con 0x7fefec101a90 2024-05-03T18:33:07.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.871+0000 7feff3000700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fefec1136b0 con 0x7fefec109520 2024-05-03T18:33:07.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.872+0000 7fefebe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fefec101a90 0x7fefec1a18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:07.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.872+0000 7fefebe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fefec101a90 0x7fefec1a18b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:60386/0 (socket says 172.21.15.33:60386) 2024-05-03T18:33:07.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.872+0000 7fefebe00700 1 -- 172.21.15.33:0/712144089 learned_addr learned my addr 172.21.15.33:0/712144089 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:33:07.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.872+0000 7fefeb400700 1 --2- 172.21.15.33:0/712144089 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fefec102450 0x7fefec1a1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:07.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.872+0000 7feff1000700 1 --2- 172.21.15.33:0/712144089 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fefec109520 0x7fefec19ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:07.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.873+0000 7fefebe00700 1 -- 172.21.15.33:0/712144089 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fefec109520 msgr2=0x7fefec19ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:33:07.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.873+0000 7fefebe00700 1 --2- 172.21.15.33:0/712144089 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fefec109520 0x7fefec19ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:07.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.873+0000 7fefebe00700 1 -- 172.21.15.33:0/712144089 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fefec102450 msgr2=0x7fefec1a1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:33:07.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.873+0000 7fefebe00700 1 --2- 172.21.15.33:0/712144089 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fefec102450 0x7fefec1a1df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:07.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.873+0000 7fefebe00700 1 -- 172.21.15.33:0/712144089 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefec19c1d0 con 0x7fefec101a90 2024-05-03T18:33:07.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.875+0000 7fefebe00700 1 --2- 172.21.15.33:0/712144089 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fefec101a90 0x7fefec1a18b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fefdc00e9a0 tx=0x7fefdc00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:33:07.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.875+0000 7fefe8c00700 1 -- 172.21.15.33:0/712144089 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefdc00ca10 con 0x7fefec101a90 2024-05-03T18:33:07.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.875+0000 7fefe8c00700 1 -- 172.21.15.33:0/712144089 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fefdc00cbb0 con 0x7fefec101a90 2024-05-03T18:33:07.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.875+0000 7fefe8c00700 1 -- 172.21.15.33:0/712144089 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefdc021e60 con 0x7fefec101a90 2024-05-03T18:33:07.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.876+0000 7feff3000700 1 -- 172.21.15.33:0/712144089 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefec19c460 con 0x7fefec101a90 2024-05-03T18:33:07.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.876+0000 7feff3000700 1 -- 172.21.15.33:0/712144089 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefec06cec0 con 0x7fefec101a90 2024-05-03T18:33:07.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.884+0000 7feffd9afb80 1 Processor -- start 2024-05-03T18:33:07.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.888+0000 7feffd9afb80 1 -- start start 2024-05-03T18:33:07.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.889+0000 7feffd9afb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2ee4d3830 0x55e2ee4d3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:07.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.889+0000 7feffd9afb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2ee4d4150 0x55e2ee4cf030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:07.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.889+0000 7feffd9afb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e2ee4cf650 0x55e2ee4cfab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:07.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.889+0000 7feffd9afb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e2ee421aa0 con 0x55e2ee4d4150 2024-05-03T18:33:07.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.889+0000 7feffd9afb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55e2ee3bf7e0 con 0x55e2ee4d3830 2024-05-03T18:33:07.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.889+0000 7feffd9afb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55e2ee41da60 con 0x55e2ee4cf650 2024-05-03T18:33:07.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.890+0000 7fefe8c00700 1 -- 172.21.15.33:0/712144089 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fefdc030020 con 0x7fefec101a90 2024-05-03T18:33:07.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.890+0000 7fefeb400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e2ee4cf650 0x55e2ee4cfab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:07.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.890+0000 7feff1000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2ee4d3830 0x55e2ee4d3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:07.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.890+0000 7feff1000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2ee4d3830 0x55e2ee4d3c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:60390/0 (socket says 172.21.15.33:60390) 2024-05-03T18:33:07.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.890+0000 7feff1000700 1 -- 172.21.15.33:0/2525294502 learned_addr learned my addr 172.21.15.33:0/2525294502 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:33:07.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.891+0000 7fefebe00700 1 --2- 172.21.15.33:0/2525294502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2ee4d4150 0x55e2ee4cf030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:07.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.891+0000 7feff1000700 1 -- 172.21.15.33:0/2525294502 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e2ee4cf650 msgr2=0x55e2ee4cfab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:33:07.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.891+0000 7feff1000700 1 --2- 172.21.15.33:0/2525294502 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e2ee4cf650 0x55e2ee4cfab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:07.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.891+0000 7feff1000700 1 -- 172.21.15.33:0/2525294502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2ee4d4150 msgr2=0x55e2ee4cf030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:33:07.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.891+0000 7feff1000700 1 --2- 172.21.15.33:0/2525294502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2ee4d4150 0x55e2ee4cf030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:07.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.891+0000 7feff1000700 1 -- 172.21.15.33:0/2525294502 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2ee36e950 con 0x55e2ee4d3830 2024-05-03T18:33:07.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.891+0000 7fefebe00700 1 --2- 172.21.15.33:0/2525294502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2ee4d4150 0x55e2ee4cf030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:33:07.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.893+0000 7fefe8c00700 1 --2- 172.21.15.33:0/712144089 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fefd00c3ab0 0x7fefd00c5f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:07.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.893+0000 7fefe8c00700 1 -- 172.21.15.33:0/712144089 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7fefdc0f3870 con 0x7fefec101a90 2024-05-03T18:33:07.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.894+0000 7feff1000700 1 --2- 172.21.15.33:0/2525294502 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2ee4d3830 0x55e2ee4d3c10 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fefe40093c0 tx=0x7fefe40093f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2efe9b81df2d2d15 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:33:07.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.894+0000 7fefebe00700 1 --2- 172.21.15.33:0/712144089 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fefd00c3ab0 0x7fefd00c5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:07.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.895+0000 7fefe1600700 1 -- 172.21.15.33:0/2525294502 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe400bcb0 con 0x55e2ee4d3830 2024-05-03T18:33:07.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.895+0000 7fefe1600700 1 -- 172.21.15.33:0/2525294502 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fefe400be50 con 0x55e2ee4d3830 2024-05-03T18:33:07.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.895+0000 7fefe1600700 1 -- 172.21.15.33:0/2525294502 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe4007d30 con 0x55e2ee4d3830 2024-05-03T18:33:07.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.896+0000 7fefebe00700 1 --2- 172.21.15.33:0/712144089 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fefd00c3ab0 0x7fefd00c5f70 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fefdc0f0ce0 tx=0x7fefdc0b6040 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:33:07.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.899+0000 7feffd9afb80 1 -- 172.21.15.33:0/2525294502 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2ee4d3830 msgr2=0x55e2ee4d3c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:33:07.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.899+0000 7feffd9afb80 1 --2- 172.21.15.33:0/2525294502 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2ee4d3830 0x55e2ee4d3c10 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fefe40093c0 tx=0x7fefe40093f0 comp rx=0 tx=0).stop 2024-05-03T18:33:07.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.900+0000 7feffd9afb80 1 -- 172.21.15.33:0/2525294502 shutdown_connections 2024-05-03T18:33:07.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.900+0000 7feffd9afb80 1 --2- 172.21.15.33:0/2525294502 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e2ee4cf650 0x55e2ee4cfab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:07.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.900+0000 7feffd9afb80 1 --2- 172.21.15.33:0/2525294502 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2ee4d3830 0x55e2ee4d3c10 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:07.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.900+0000 7feffd9afb80 1 --2- 172.21.15.33:0/2525294502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2ee4d4150 0x55e2ee4cf030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:07.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.900+0000 7feffd9afb80 1 -- 172.21.15.33:0/2525294502 >> 172.21.15.33:0/2525294502 conn(0x55e2ee3fdbe0 msgr2=0x55e2ee4c9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:33:07.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.900+0000 7feffd9afb80 1 -- 172.21.15.33:0/2525294502 shutdown_connections 2024-05-03T18:33:07.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.901+0000 7feffd9afb80 1 -- 172.21.15.33:0/2525294502 wait complete. 2024-05-03T18:33:07.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.901+0000 7feffd9afb80 1 -- start start 2024-05-03T18:33:07.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.902+0000 7feffd9afb80 1 Processor -- start 2024-05-03T18:33:07.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.903+0000 7feffd9afb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2ee4cf650 0x55e2ee4daf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:07.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.903+0000 7feffd9afb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2ee4d3830 0x55e2ee4d8370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:07.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.903+0000 7feffd9afb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e2ee4d4150 0x55e2ee4d88b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:07.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.903+0000 7feffd9afb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e2ee4dbcc0 con 0x55e2ee4cf650 2024-05-03T18:33:07.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.903+0000 7feffd9afb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55e2ee387ba0 con 0x55e2ee4d3830 2024-05-03T18:33:07.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.903+0000 7feffd9afb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55e2ee4dbe40 con 0x55e2ee4d4150 2024-05-03T18:33:07.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.903+0000 7feffd9afb80 10 client.0 _ll_register_callbacks cb 0x55e2ee3c8cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:33:07.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.904+0000 7fefeb400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2ee4d3830 0x55e2ee4d8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:07.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.904+0000 7fefeb400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2ee4d3830 0x55e2ee4d8370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:60392/0 (socket says 172.21.15.33:60392) 2024-05-03T18:33:07.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.904+0000 7fefeb400700 1 -- 172.21.15.33:0/51407153 learned_addr learned my addr 172.21.15.33:0/51407153 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:33:07.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.905+0000 7feff1000700 1 --2- 172.21.15.33:0/51407153 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2ee4cf650 0x55e2ee4daf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:07.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.905+0000 7fefeb400700 1 -- 172.21.15.33:0/51407153 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e2ee4d4150 msgr2=0x55e2ee4d88b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:33:07.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.905+0000 7fefeb400700 1 --2- 172.21.15.33:0/51407153 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e2ee4d4150 0x55e2ee4d88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:07.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.905+0000 7fefeb400700 1 -- 172.21.15.33:0/51407153 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2ee4cf650 msgr2=0x55e2ee4daf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:33:07.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.906+0000 7fefeb400700 1 --2- 172.21.15.33:0/51407153 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2ee4cf650 0x55e2ee4daf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:07.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.906+0000 7fefeb400700 1 -- 172.21.15.33:0/51407153 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2ee3e74a0 con 0x55e2ee4d3830 2024-05-03T18:33:07.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.906+0000 7feff1000700 1 --2- 172.21.15.33:0/51407153 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2ee4cf650 0x55e2ee4daf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:33:07.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.906+0000 7fefeb400700 1 --2- 172.21.15.33:0/51407153 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2ee4d3830 0x55e2ee4d8370 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fefd800b8c0 tx=0x7fefd800bbd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4e3cd0d5bcafbbc server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:33:07.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.906+0000 7fefd6000700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-03T18:33:07.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.907+0000 7fefd6000700 1 -- 172.21.15.33:0/51407153 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd8003d90 con 0x55e2ee4d3830 2024-05-03T18:33:07.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.907+0000 7fefd6000700 1 -- 172.21.15.33:0/51407153 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7fefd8004830 con 0x55e2ee4d3830 2024-05-03T18:33:07.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.907+0000 7feffd9afb80 1 -- 172.21.15.33:0/51407153 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fefe400be50 con 0x55e2ee4d3830 2024-05-03T18:33:07.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.908+0000 7fefd6000700 1 -- 172.21.15.33:0/51407153 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fefd8004a30 con 0x55e2ee4d3830 2024-05-03T18:33:07.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.908+0000 7feffd9afb80 10 client.35633 fetch_fsmap learned FSMap version 21 2024-05-03T18:33:07.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.908+0000 7feffd9afb80 1 -- 172.21.15.33:0/51407153 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55e2ee4e6500 con 0x55e2ee4d3830 2024-05-03T18:33:07.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.909+0000 7fefd6000700 1 -- 172.21.15.33:0/51407153 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fefe400be50 con 0x55e2ee4d3830 2024-05-03T18:33:07.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.909+0000 7fefd6000700 1 -- 172.21.15.33:0/51407153 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fefd8019030 con 0x55e2ee4d3830 2024-05-03T18:33:07.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.909+0000 7feffd9afb80 10 client.35633 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:33:07.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.909+0000 7feffd9afb80 10 client.35633 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:33:07.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.909+0000 7feffd9afb80 20 client.35633 populate_metadata read hostname 'smithi033' 2024-05-03T18:33:07.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.909+0000 7feffd9afb80 1 --2- 172.21.15.33:0/51407153 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55e2ee4e8b40 0x55e2ee4eaf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:07.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.909+0000 7feffd9afb80 4 client.35633 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:33:07.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.909+0000 7feffd9afb80 1 -- 172.21.15.33:0/51407153 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e2ee49c250 con 0x55e2ee4e8b40 2024-05-03T18:33:07.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.910+0000 7feff1000700 1 --2- 172.21.15.33:0/51407153 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55e2ee4e8b40 0x55e2ee4eaf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:07.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.912+0000 7feff1000700 1 --2- 172.21.15.33:0/51407153 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55e2ee4e8b40 0x55e2ee4eaf30 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab3429ed2e8a9f1 server_cookie=ea528ccf79eea4f9 in_seq=0 out_seq=0 2024-05-03T18:33:07.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.912+0000 7fefd6000700 10 client.35633 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:33:07.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.913+0000 7fefd6000700 1 -- 172.21.15.33:0/51407153 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e2ee49c250 con 0x55e2ee4e8b40 2024-05-03T18:33:07.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.913+0000 7fefd6000700 10 client.35633 handle_command_reply: tid=0 2024-05-03T18:33:07.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.914+0000 7feffd9afb80 1 client.35633 shutdown 2024-05-03T18:33:07.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.914+0000 7feffd9afb80 1 -- 172.21.15.33:0/51407153 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2ee4d3830 msgr2=0x55e2ee4d8370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:33:07.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.914+0000 7feffd9afb80 1 --2- 172.21.15.33:0/51407153 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2ee4d3830 0x55e2ee4d8370 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fefd800b8c0 tx=0x7fefd800bbd0 comp rx=0 tx=0).stop 2024-05-03T18:33:07.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.915+0000 7feffd9afb80 1 -- 172.21.15.33:0/51407153 shutdown_connections 2024-05-03T18:33:07.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.915+0000 7feffd9afb80 1 --2- 172.21.15.33:0/51407153 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55e2ee4e8b40 0x55e2ee4eaf30 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:07.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.915+0000 7feffd9afb80 1 --2- 172.21.15.33:0/51407153 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e2ee4d4150 0x55e2ee4d88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:07.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.915+0000 7feffd9afb80 1 --2- 172.21.15.33:0/51407153 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2ee4cf650 0x55e2ee4daf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:07.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.915+0000 7feffd9afb80 1 --2- 172.21.15.33:0/51407153 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2ee4d3830 0x55e2ee4d8370 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:07.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.915+0000 7feffd9afb80 1 -- 172.21.15.33:0/51407153 >> 172.21.15.33:0/51407153 conn(0x55e2ee3fdbe0 msgr2=0x55e2ee4d4e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:33:07.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.915+0000 7fefd6000700 0 client.35633 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:33:07.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.915+0000 7feffd9afb80 1 -- 172.21.15.33:0/51407153 shutdown_connections 2024-05-03T18:33:07.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.916+0000 7feffd9afb80 1 -- 172.21.15.33:0/51407153 wait complete. 2024-05-03T18:33:07.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.916+0000 7feffd9afb80 20 client.35633 trim_cache size 0 max 16384 2024-05-03T18:33:07.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.948+0000 7feffd9afb80 1 Processor -- start 2024-05-03T18:33:07.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.948+0000 7feffd9afb80 1 -- start start 2024-05-03T18:33:07.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.949+0000 7feffd9afb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2ee4cf650 0x55e2ee4d2d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:07.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.949+0000 7feffd9afb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e2ee4d3830 0x55e2ee4c96e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:07.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.949+0000 7feffd9afb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2ee4d4150 0x55e2ee4cead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:07.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.949+0000 7feffd9afb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fefec06f870 con 0x55e2ee4cf650 2024-05-03T18:33:07.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.949+0000 7feffd9afb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fefec0720f0 con 0x55e2ee4d4150 2024-05-03T18:33:07.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.949+0000 7feffd9afb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fefec06fee0 con 0x55e2ee4d3830 2024-05-03T18:33:07.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.949+0000 7fefeb400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e2ee4d3830 0x55e2ee4c96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:07.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.950+0000 7fefeb400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e2ee4d3830 0x55e2ee4c96e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:38046/0 (socket says 172.21.15.33:38046) 2024-05-03T18:33:07.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.950+0000 7fefeb400700 1 -- 172.21.15.33:0/3447240557 learned_addr learned my addr 172.21.15.33:0/3447240557 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:33:07.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.950+0000 7feff1000700 1 --2- 172.21.15.33:0/3447240557 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2ee4cf650 0x55e2ee4d2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:07.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.951+0000 7fefeb400700 1 -- 172.21.15.33:0/3447240557 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2ee4d4150 msgr2=0x55e2ee4cead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:33:07.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.951+0000 7fefeb400700 1 --2- 172.21.15.33:0/3447240557 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2ee4d4150 0x55e2ee4cead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:07.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.951+0000 7fefeb400700 1 -- 172.21.15.33:0/3447240557 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2ee4cf650 msgr2=0x55e2ee4d2d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:33:07.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.951+0000 7fefeb400700 1 --2- 172.21.15.33:0/3447240557 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2ee4cf650 0x55e2ee4d2d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:07.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.951+0000 7fefeb400700 1 -- 172.21.15.33:0/3447240557 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2ee4d72d0 con 0x55e2ee4d3830 2024-05-03T18:33:07.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.951+0000 7feff1000700 1 --2- 172.21.15.33:0/3447240557 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2ee4cf650 0x55e2ee4d2d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:33:07.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.952+0000 7fefeb400700 1 --2- 172.21.15.33:0/3447240557 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e2ee4d3830 0x55e2ee4c96e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fefd800c010 tx=0x7fefd8005b00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26ab56f3db0667d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:33:07.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.954+0000 7fefe0c00700 1 -- 172.21.15.33:0/3447240557 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd80247e0 con 0x55e2ee4d3830 2024-05-03T18:33:07.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.954+0000 7fefe0c00700 1 -- 172.21.15.33:0/3447240557 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fefd8024980 con 0x55e2ee4d3830 2024-05-03T18:33:07.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.954+0000 7fefe0c00700 1 -- 172.21.15.33:0/3447240557 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd80118d0 con 0x55e2ee4d3830 2024-05-03T18:33:07.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.955+0000 7feffd9afb80 1 -- 172.21.15.33:0/3447240557 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e2ee4d3830 msgr2=0x55e2ee4c96e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:33:07.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.955+0000 7feffd9afb80 1 --2- 172.21.15.33:0/3447240557 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e2ee4d3830 0x55e2ee4c96e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fefd800c010 tx=0x7fefd8005b00 comp rx=0 tx=0).stop 2024-05-03T18:33:07.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.955+0000 7feffd9afb80 1 -- 172.21.15.33:0/3447240557 shutdown_connections 2024-05-03T18:33:07.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.955+0000 7feffd9afb80 1 --2- 172.21.15.33:0/3447240557 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2ee4d4150 0x55e2ee4cead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:07.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.955+0000 7feffd9afb80 1 --2- 172.21.15.33:0/3447240557 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2ee4cf650 0x55e2ee4d2d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:07.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.955+0000 7feffd9afb80 1 --2- 172.21.15.33:0/3447240557 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e2ee4d3830 0x55e2ee4c96e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:07.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.955+0000 7feffd9afb80 1 -- 172.21.15.33:0/3447240557 >> 172.21.15.33:0/3447240557 conn(0x55e2ee3fdbe0 msgr2=0x55e2ee4ce3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:33:07.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.955+0000 7feffd9afb80 1 -- 172.21.15.33:0/3447240557 shutdown_connections 2024-05-03T18:33:07.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.956+0000 7feffd9afb80 1 -- 172.21.15.33:0/3447240557 wait complete. 2024-05-03T18:33:07.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.956+0000 7feffd9afb80 1 -- start start 2024-05-03T18:33:07.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.956+0000 7feffd9afb80 1 Processor -- start 2024-05-03T18:33:07.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.957+0000 7feffd9afb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2ee4cf650 0x55e2ee4e16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:07.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.957+0000 7feffd9afb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e2ee4d3830 0x55e2ee4dc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:07.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.958+0000 7feffd9afb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2ee4d4150 0x55e2ee4dc620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:07.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.958+0000 7feffd9afb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fefec0705b0 con 0x55e2ee4cf650 2024-05-03T18:33:07.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.958+0000 7feffd9afb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fefec071a20 con 0x55e2ee4d4150 2024-05-03T18:33:07.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.958+0000 7feffd9afb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fefec070c80 con 0x55e2ee4d3830 2024-05-03T18:33:07.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.958+0000 7feff1000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2ee4cf650 0x55e2ee4e16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:07.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.958+0000 7feffd9afb80 10 client.0 _ll_register_callbacks cb 0x55e2ee3c8cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:33:07.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.958+0000 7feff1000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2ee4cf650 0x55e2ee4e16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33248/0 (socket says 172.21.15.33:33248) 2024-05-03T18:33:07.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.958+0000 7feff1000700 1 -- 172.21.15.33:0/3053058128 learned_addr learned my addr 172.21.15.33:0/3053058128 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:33:07.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.958+0000 7fefeb400700 1 --2- 172.21.15.33:0/3053058128 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e2ee4d3830 0x55e2ee4dc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:07.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.959+0000 7feff1000700 1 -- 172.21.15.33:0/3053058128 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e2ee4d3830 msgr2=0x55e2ee4dc0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:33:07.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.959+0000 7feff1000700 1 --2- 172.21.15.33:0/3053058128 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e2ee4d3830 0x55e2ee4dc0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:07.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.959+0000 7feff1000700 1 -- 172.21.15.33:0/3053058128 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2ee4d4150 msgr2=0x55e2ee4dc620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:33:07.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.959+0000 7feff1000700 1 --2- 172.21.15.33:0/3053058128 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2ee4d4150 0x55e2ee4dc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:07.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.959+0000 7feff1000700 1 -- 172.21.15.33:0/3053058128 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2ee4dce10 con 0x55e2ee4cf650 2024-05-03T18:33:07.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.960+0000 7feff1000700 1 --2- 172.21.15.33:0/3053058128 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2ee4cf650 0x55e2ee4e16e0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fefe4000f80 tx=0x7fefe4007500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6c27974586081d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:33:07.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.960+0000 7fefd6000700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:33:07.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.960+0000 7fefd6000700 1 -- 172.21.15.33:0/3053058128 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe4007b00 con 0x55e2ee4cf650 2024-05-03T18:33:07.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.961+0000 7fefd6000700 1 -- 172.21.15.33:0/3053058128 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7fefe4007ca0 con 0x55e2ee4cf650 2024-05-03T18:33:07.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.961+0000 7feffd9afb80 1 -- 172.21.15.33:0/3053058128 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fefd8024980 con 0x55e2ee4cf650 2024-05-03T18:33:07.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.962+0000 7fefd6000700 1 -- 172.21.15.33:0/3053058128 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fefe4039050 con 0x55e2ee4cf650 2024-05-03T18:33:07.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.963+0000 7feffd9afb80 10 client.16017 fetch_fsmap learned FSMap version 21 2024-05-03T18:33:07.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.963+0000 7feffd9afb80 1 -- 172.21.15.33:0/3053058128 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55e2ee5544b0 con 0x55e2ee4cf650 2024-05-03T18:33:07.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.963+0000 7fefd6000700 1 -- 172.21.15.33:0/3053058128 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fefe4031e50 con 0x55e2ee4cf650 2024-05-03T18:33:07.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.963+0000 7fefd6000700 1 -- 172.21.15.33:0/3053058128 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fefe4011070 con 0x55e2ee4cf650 2024-05-03T18:33:07.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.964+0000 7feffd9afb80 10 client.16017 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:33:07.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.964+0000 7feffd9afb80 10 client.16017 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:33:07.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.964+0000 7feffd9afb80 20 client.16017 populate_metadata read hostname 'smithi033' 2024-05-03T18:33:07.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.964+0000 7feffd9afb80 1 --2- 172.21.15.33:0/3053058128 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55e2ee5546e0 0x55e2ee556ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:07.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.964+0000 7feffd9afb80 4 client.16017 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-03T18:33:07.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.964+0000 7feffd9afb80 1 -- 172.21.15.33:0/3053058128 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7fefd8024980 con 0x55e2ee5546e0 2024-05-03T18:33:07.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.964+0000 7fefeb400700 1 --2- 172.21.15.33:0/3053058128 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55e2ee5546e0 0x55e2ee556ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:07.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.966+0000 7fefd6000700 10 client.16017 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:33:07.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.966+0000 7fefeb400700 1 --2- 172.21.15.33:0/3053058128 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55e2ee5546e0 0x55e2ee556ad0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa05af0c8e3abab4 server_cookie=4633a583b8a1711e in_seq=0 out_seq=0 2024-05-03T18:33:07.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.968+0000 7fefd6000700 1 -- 172.21.15.33:0/3053058128 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7fefd8024980 con 0x55e2ee5546e0 2024-05-03T18:33:07.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.968+0000 7fefd6000700 10 client.16017 handle_command_reply: tid=0 2024-05-03T18:33:07.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.968+0000 7feffd9afb80 1 client.16017 shutdown 2024-05-03T18:33:07.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.968+0000 7feffd9afb80 1 -- 172.21.15.33:0/3053058128 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2ee4cf650 msgr2=0x55e2ee4e16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:33:07.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.968+0000 7feffd9afb80 1 --2- 172.21.15.33:0/3053058128 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2ee4cf650 0x55e2ee4e16e0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fefe4000f80 tx=0x7fefe4007500 comp rx=0 tx=0).stop 2024-05-03T18:33:07.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.968+0000 7feffd9afb80 1 -- 172.21.15.33:0/3053058128 shutdown_connections 2024-05-03T18:33:07.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.968+0000 7feffd9afb80 1 --2- 172.21.15.33:0/3053058128 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55e2ee5546e0 0x55e2ee556ad0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:07.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.968+0000 7feffd9afb80 1 --2- 172.21.15.33:0/3053058128 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2ee4d4150 0x55e2ee4dc620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:07.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.968+0000 7feffd9afb80 1 --2- 172.21.15.33:0/3053058128 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2ee4cf650 0x55e2ee4e16e0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:07.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.968+0000 7feffd9afb80 1 --2- 172.21.15.33:0/3053058128 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e2ee4d3830 0x55e2ee4dc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:07.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.968+0000 7feffd9afb80 1 -- 172.21.15.33:0/3053058128 >> 172.21.15.33:0/3053058128 conn(0x55e2ee3fdbe0 msgr2=0x55e2ee4d2b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:33:07.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.968+0000 7fefd6000700 0 client.16017 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:33:07.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.969+0000 7feffd9afb80 1 -- 172.21.15.33:0/3053058128 shutdown_connections 2024-05-03T18:33:07.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.969+0000 7feffd9afb80 1 -- 172.21.15.33:0/3053058128 wait complete. 2024-05-03T18:33:07.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.969+0000 7feffd9afb80 20 client.16017 trim_cache size 0 max 16384 2024-05-03T18:33:07.972 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-03T18:33:07.972 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-05-03T18:33:07.973 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "8cf4a378-4288-4e5a-8127-e8669bd61ed0", 2024-05-03T18:33:07.973 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-05-03T18:33:07.973 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-03T18:33:07.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.970+0000 7feff3000700 1 -- 172.21.15.33:0/712144089 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fefd00c3ab0 msgr2=0x7fefd00c5f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:33:07.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.970+0000 7feff3000700 1 --2- 172.21.15.33:0/712144089 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fefd00c3ab0 0x7fefd00c5f70 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fefdc0f0ce0 tx=0x7fefdc0b6040 comp rx=0 tx=0).stop 2024-05-03T18:33:07.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.970+0000 7feff3000700 1 -- 172.21.15.33:0/712144089 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fefec101a90 msgr2=0x7fefec1a18b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:33:07.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.970+0000 7feff3000700 1 --2- 172.21.15.33:0/712144089 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fefec101a90 0x7fefec1a18b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fefdc00e9a0 tx=0x7fefdc00ecb0 comp rx=0 tx=0).stop 2024-05-03T18:33:07.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.970+0000 7feff3000700 1 -- 172.21.15.33:0/712144089 shutdown_connections 2024-05-03T18:33:07.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.970+0000 7feff3000700 1 --2- 172.21.15.33:0/712144089 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fefec109520 0x7fefec19ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:07.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.970+0000 7feff3000700 1 --2- 172.21.15.33:0/712144089 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fefd00c3ab0 0x7fefd00c5f70 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:07.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.970+0000 7feff3000700 1 --2- 172.21.15.33:0/712144089 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fefec101a90 0x7fefec1a18b0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:07.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.970+0000 7feff3000700 1 --2- 172.21.15.33:0/712144089 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fefec102450 0x7fefec1a1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:07.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.970+0000 7feff3000700 1 -- 172.21.15.33:0/712144089 >> 172.21.15.33:0/712144089 conn(0x7fefec078a10 msgr2=0x7fefec0fffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:33:07.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.974+0000 7feff3000700 1 -- 172.21.15.33:0/712144089 shutdown_connections 2024-05-03T18:33:07.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:07.974+0000 7feff3000700 1 -- 172.21.15.33:0/712144089 wait complete. 2024-05-03T18:33:08.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:33:08 smithi033 ceph-mon[28330]: pgmap v181: 129 pgs: 129 active+clean; 1.8 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 8.2 MiB/s wr, 812 op/s 2024-05-03T18:33:08.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:33:08 smithi099 ceph-mon[31658]: pgmap v181: 129 pgs: 129 active+clean; 1.8 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 8.2 MiB/s wr, 812 op/s 2024-05-03T18:33:08.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:33:08 smithi150 ceph-mon[32492]: pgmap v181: 129 pgs: 129 active+clean; 1.8 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 8.2 MiB/s wr, 812 op/s 2024-05-03T18:33:09.090 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-03T18:33:09.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:33:09 smithi099 ceph-mon[31658]: scrub queued for path: / 2024-05-03T18:33:09.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:33:09 smithi099 ceph-mon[31658]: scrub summary: idle+waiting paths [/] 2024-05-03T18:33:09.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:33:09 smithi099 ceph-mon[31658]: scrub summary: active paths [/] 2024-05-03T18:33:09.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:33:09 smithi150 ceph-mon[32492]: scrub queued for path: / 2024-05-03T18:33:09.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:33:09 smithi150 ceph-mon[32492]: scrub summary: idle+waiting paths [/] 2024-05-03T18:33:09.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:33:09 smithi150 ceph-mon[32492]: scrub summary: active paths [/] 2024-05-03T18:33:09.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:33:09 smithi033 ceph-mon[28330]: scrub queued for path: / 2024-05-03T18:33:09.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:33:09 smithi033 ceph-mon[28330]: scrub summary: idle+waiting paths [/] 2024-05-03T18:33:09.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:33:09 smithi033 ceph-mon[28330]: scrub summary: active paths [/] 2024-05-03T18:33:09.834 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:33:10.110 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 115.0 s, 578.6 tps, lat 172.534 ms stddev 45.760 2024-05-03T18:33:10.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:33:10 smithi099 ceph-mon[31658]: scrub summary: idle+waiting paths [/] 2024-05-03T18:33:10.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:33:10 smithi099 ceph-mon[31658]: pgmap v182: 129 pgs: 129 active+clean; 1.8 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 5.9 MiB/s wr, 551 op/s 2024-05-03T18:33:10.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:33:10 smithi150 ceph-mon[32492]: scrub summary: idle+waiting paths [/] 2024-05-03T18:33:10.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:33:10 smithi150 ceph-mon[32492]: pgmap v182: 129 pgs: 129 active+clean; 1.8 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 5.9 MiB/s wr, 551 op/s 2024-05-03T18:33:10.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:33:10 smithi033 ceph-mon[28330]: scrub summary: idle+waiting paths [/] 2024-05-03T18:33:10.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:33:10 smithi033 ceph-mon[28330]: pgmap v182: 129 pgs: 129 active+clean; 1.8 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 5.9 MiB/s wr, 551 op/s 2024-05-03T18:33:11.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:33:11 smithi099 ceph-mon[31658]: scrub summary: idle 2024-05-03T18:33:11.554 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:33:11 smithi033 ceph-mon[28330]: scrub summary: idle 2024-05-03T18:33:11.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:33:11 smithi150 ceph-mon[32492]: scrub summary: idle 2024-05-03T18:33:12.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:33:12 smithi033 ceph-mon[28330]: pgmap v183: 129 pgs: 129 active+clean; 1.8 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 5.9 MiB/s wr, 551 op/s 2024-05-03T18:33:12.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:33:12 smithi099 ceph-mon[31658]: pgmap v183: 129 pgs: 129 active+clean; 1.8 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 5.9 MiB/s wr, 551 op/s 2024-05-03T18:33:12.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:33:12 smithi150 ceph-mon[32492]: pgmap v183: 129 pgs: 129 active+clean; 1.8 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 5.9 MiB/s wr, 551 op/s 2024-05-03T18:33:14.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.285+0000 7fe2fac00700 1 -- 172.21.15.33:0/13812879 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe2f4102b30 msgr2=0x7fe2f410f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:33:14.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.286+0000 7fe2fac00700 1 --2- 172.21.15.33:0/13812879 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe2f4102b30 0x7fe2f410f7f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe2e4009990 tx=0x7fe2e4009ca0 comp rx=0 tx=0).stop 2024-05-03T18:33:14.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.286+0000 7fe2fac00700 1 -- 172.21.15.33:0/13812879 shutdown_connections 2024-05-03T18:33:14.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.286+0000 7fe2fac00700 1 --2- 172.21.15.33:0/13812879 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2f410fd30 0x7fe2f4112120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:14.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.286+0000 7fe2fac00700 1 --2- 172.21.15.33:0/13812879 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2f4102180 0x7fe2f4102560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:14.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.286+0000 7fe2fac00700 1 --2- 172.21.15.33:0/13812879 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe2f4102b30 0x7fe2f410f7f0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:14.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.286+0000 7fe2fac00700 1 -- 172.21.15.33:0/13812879 >> 172.21.15.33:0/13812879 conn(0x7fe2f40fddc0 msgr2=0x7fe2f41001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:33:14.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.288+0000 7fe2fac00700 1 -- 172.21.15.33:0/13812879 shutdown_connections 2024-05-03T18:33:14.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.289+0000 7fe2fac00700 1 -- 172.21.15.33:0/13812879 wait complete. 2024-05-03T18:33:14.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.290+0000 7fe2fac00700 1 Processor -- start 2024-05-03T18:33:14.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.291+0000 7fe2fac00700 1 -- start start 2024-05-03T18:33:14.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.291+0000 7fe2fac00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe2f4102180 0x7fe2f41a1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:14.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.291+0000 7fe2fac00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2f4102b30 0x7fe2f41a1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:14.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.291+0000 7fe2fac00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2f410fd30 0x7fe2f419ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:14.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.291+0000 7fe2fac00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe2f4115690 con 0x7fe2f410fd30 2024-05-03T18:33:14.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.291+0000 7fe2fac00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe2f4115510 con 0x7fe2f4102b30 2024-05-03T18:33:14.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.291+0000 7fe2fac00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe2f4115810 con 0x7fe2f4102180 2024-05-03T18:33:14.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.292+0000 7fe2f9800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe2f4102180 0x7fe2f41a1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:14.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.292+0000 7fe2f9800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe2f4102180 0x7fe2f41a1850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:38074/0 (socket says 172.21.15.33:38074) 2024-05-03T18:33:14.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.292+0000 7fe2f9800700 1 -- 172.21.15.33:0/3215390563 learned_addr learned my addr 172.21.15.33:0/3215390563 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:33:14.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.292+0000 7fe2f8e00700 1 --2- 172.21.15.33:0/3215390563 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2f4102b30 0x7fe2f41a1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:14.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.292+0000 7fe2fa200700 1 --2- 172.21.15.33:0/3215390563 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2f410fd30 0x7fe2f419ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:14.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.293+0000 7fe2f9800700 1 -- 172.21.15.33:0/3215390563 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2f4102b30 msgr2=0x7fe2f41a1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:33:14.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.293+0000 7fe2f9800700 1 --2- 172.21.15.33:0/3215390563 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2f4102b30 0x7fe2f41a1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:14.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.293+0000 7fe2f9800700 1 -- 172.21.15.33:0/3215390563 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2f410fd30 msgr2=0x7fe2f419ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:33:14.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.293+0000 7fe2f9800700 1 --2- 172.21.15.33:0/3215390563 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2f410fd30 0x7fe2f419ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:14.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.293+0000 7fe2f9800700 1 -- 172.21.15.33:0/3215390563 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2f419c170 con 0x7fe2f4102180 2024-05-03T18:33:14.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.294+0000 7fe2f9800700 1 --2- 172.21.15.33:0/3215390563 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe2f4102180 0x7fe2f41a1850 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe2e800e9a0 tx=0x7fe2e800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:33:14.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.294+0000 7fe2f2000700 1 -- 172.21.15.33:0/3215390563 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e800ca10 con 0x7fe2f4102180 2024-05-03T18:33:14.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.294+0000 7fe2fac00700 1 -- 172.21.15.33:0/3215390563 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2f419c460 con 0x7fe2f4102180 2024-05-03T18:33:14.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.294+0000 7fe2f2000700 1 -- 172.21.15.33:0/3215390563 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe2e800cbb0 con 0x7fe2f4102180 2024-05-03T18:33:14.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.294+0000 7fe2fac00700 1 -- 172.21.15.33:0/3215390563 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2f41a8720 con 0x7fe2f4102180 2024-05-03T18:33:14.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.294+0000 7fe2f2000700 1 -- 172.21.15.33:0/3215390563 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e8018680 con 0x7fe2f4102180 2024-05-03T18:33:14.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.305+0000 7fe2f2000700 1 -- 172.21.15.33:0/3215390563 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fe2e8018820 con 0x7fe2f4102180 2024-05-03T18:33:14.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.308+0000 7fe306c5bb80 1 Processor -- start 2024-05-03T18:33:14.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.308+0000 7fe306c5bb80 1 -- start start 2024-05-03T18:33:14.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.309+0000 7fe306c5bb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5565a98c6c60 0x5565a98c7040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:14.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.309+0000 7fe306c5bb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565a98bef10 0x5565a98bf390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:14.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.309+0000 7fe306c5bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565a98bf8d0 0x5565a98df770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:14.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.309+0000 7fe306c5bb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5565a9814a20 con 0x5565a98c6c60 2024-05-03T18:33:14.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.309+0000 7fe306c5bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5565a97b2760 con 0x5565a98bf8d0 2024-05-03T18:33:14.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.309+0000 7fe306c5bb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5565a98109e0 con 0x5565a98bef10 2024-05-03T18:33:14.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.309+0000 7fe2fa200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5565a98c6c60 0x5565a98c7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:14.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.309+0000 7fe2fa200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5565a98c6c60 0x5565a98c7040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33268/0 (socket says 172.21.15.33:33268) 2024-05-03T18:33:14.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.309+0000 7fe2fa200700 1 -- 172.21.15.33:0/955505404 learned_addr learned my addr 172.21.15.33:0/955505404 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:33:14.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.310+0000 7fe2f8e00700 1 --2- 172.21.15.33:0/955505404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565a98bf8d0 0x5565a98df770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:14.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.310+0000 7fe2f9800700 1 --2- 172.21.15.33:0/955505404 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565a98bef10 0x5565a98bf390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:14.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.310+0000 7fe2fa200700 1 -- 172.21.15.33:0/955505404 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565a98bef10 msgr2=0x5565a98bf390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:33:14.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.310+0000 7fe2fa200700 1 --2- 172.21.15.33:0/955505404 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565a98bef10 0x5565a98bf390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:14.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.310+0000 7fe2fa200700 1 -- 172.21.15.33:0/955505404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565a98bf8d0 msgr2=0x5565a98df770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:33:14.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.310+0000 7fe2fa200700 1 --2- 172.21.15.33:0/955505404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565a98bf8d0 0x5565a98df770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:14.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.310+0000 7fe2fa200700 1 -- 172.21.15.33:0/955505404 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565a97618d0 con 0x5565a98c6c60 2024-05-03T18:33:14.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.310+0000 7fe2f8e00700 1 --2- 172.21.15.33:0/955505404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565a98bf8d0 0x5565a98df770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:33:14.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.311+0000 7fe2fa200700 1 --2- 172.21.15.33:0/955505404 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5565a98c6c60 0x5565a98c7040 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec003f40 tx=0x7fe2ec009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8791ce9d4ce266fc server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:33:14.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.311+0000 7fe2e2a00700 1 -- 172.21.15.33:0/955505404 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2ec004bb0 con 0x5565a98c6c60 2024-05-03T18:33:14.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.312+0000 7fe2e2a00700 1 -- 172.21.15.33:0/955505404 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe2ec004d50 con 0x5565a98c6c60 2024-05-03T18:33:14.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.312+0000 7fe2e2a00700 1 -- 172.21.15.33:0/955505404 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2ec007da0 con 0x5565a98c6c60 2024-05-03T18:33:14.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.312+0000 7fe2f2000700 1 --2- 172.21.15.33:0/3215390563 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fe2dc0b29f0 0x7fe2dc0b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:14.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.313+0000 7fe2f2000700 1 -- 172.21.15.33:0/3215390563 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7fe2e80e2bb0 con 0x7fe2f4102180 2024-05-03T18:33:14.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.313+0000 7fe306c5bb80 1 -- 172.21.15.33:0/955505404 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5565a98c6c60 msgr2=0x5565a98c7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:33:14.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.313+0000 7fe306c5bb80 1 --2- 172.21.15.33:0/955505404 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5565a98c6c60 0x5565a98c7040 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec003f40 tx=0x7fe2ec009ba0 comp rx=0 tx=0).stop 2024-05-03T18:33:14.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.313+0000 7fe306c5bb80 1 -- 172.21.15.33:0/955505404 shutdown_connections 2024-05-03T18:33:14.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.313+0000 7fe306c5bb80 1 --2- 172.21.15.33:0/955505404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565a98bf8d0 0x5565a98df770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:14.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.313+0000 7fe306c5bb80 1 --2- 172.21.15.33:0/955505404 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5565a98c6c60 0x5565a98c7040 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:14.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.313+0000 7fe306c5bb80 1 --2- 172.21.15.33:0/955505404 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565a98bef10 0x5565a98bf390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:14.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.313+0000 7fe306c5bb80 1 -- 172.21.15.33:0/955505404 >> 172.21.15.33:0/955505404 conn(0x5565a983a6d0 msgr2=0x5565a98bcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:33:14.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.316+0000 7fe2f9800700 1 --2- 172.21.15.33:0/3215390563 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fe2dc0b29f0 0x7fe2dc0b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:14.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.316+0000 7fe306c5bb80 1 -- 172.21.15.33:0/955505404 shutdown_connections 2024-05-03T18:33:14.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.316+0000 7fe306c5bb80 1 -- 172.21.15.33:0/955505404 wait complete. 2024-05-03T18:33:14.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.317+0000 7fe306c5bb80 1 -- start start 2024-05-03T18:33:14.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.317+0000 7fe306c5bb80 1 Processor -- start 2024-05-03T18:33:14.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.318+0000 7fe2f9800700 1 --2- 172.21.15.33:0/3215390563 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fe2dc0b29f0 0x7fe2dc0b4eb0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe2e80aa6c0 tx=0x7fe2e801f000 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:33:14.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.319+0000 7fe306c5bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565a98bef10 0x5565a98df2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:14.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.319+0000 7fe306c5bb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565a98bf8d0 0x5565a98dc650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:14.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.319+0000 7fe306c5bb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5565a98c6c60 0x5565a98dcb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:14.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.319+0000 7fe306c5bb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5565a98dff50 con 0x5565a98c6c60 2024-05-03T18:33:14.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.319+0000 7fe306c5bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5565a977ab20 con 0x5565a98bef10 2024-05-03T18:33:14.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.319+0000 7fe306c5bb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5565a98e00d0 con 0x5565a98bf8d0 2024-05-03T18:33:14.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.319+0000 7fe306c5bb80 10 client.0 _ll_register_callbacks cb 0x5565a97bbc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:33:14.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.320+0000 7fe2fa200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565a98bef10 0x5565a98df2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:14.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.320+0000 7fe2fa200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565a98bef10 0x5565a98df2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:60450/0 (socket says 172.21.15.33:60450) 2024-05-03T18:33:14.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.320+0000 7fe2fa200700 1 -- 172.21.15.33:0/2067462712 learned_addr learned my addr 172.21.15.33:0/2067462712 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:33:14.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.320+0000 7fe2f8e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565a98bf8d0 0x5565a98dc650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:14.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.321+0000 7fe2fa200700 1 --2- 172.21.15.33:0/2067462712 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5565a98c6c60 0x5565a98dcb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:14.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.322+0000 7fe2f8e00700 1 -- 172.21.15.33:0/2067462712 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565a98bef10 msgr2=0x5565a98df2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:33:14.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.322+0000 7fe2f8e00700 1 --2- 172.21.15.33:0/2067462712 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565a98bef10 0x5565a98df2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:14.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.322+0000 7fe2f8e00700 1 -- 172.21.15.33:0/2067462712 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5565a98c6c60 msgr2=0x5565a98dcb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:33:14.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.322+0000 7fe2f8e00700 1 --2- 172.21.15.33:0/2067462712 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5565a98c6c60 0x5565a98dcb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:14.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.323+0000 7fe2f8e00700 1 -- 172.21.15.33:0/2067462712 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565a97da420 con 0x5565a98bf8d0 2024-05-03T18:33:14.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.323+0000 7fe2fa200700 1 --2- 172.21.15.33:0/2067462712 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5565a98c6c60 0x5565a98dcb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:33:14.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.324+0000 7fe2f8e00700 1 --2- 172.21.15.33:0/2067462712 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565a98bf8d0 0x5565a98dc650 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe2e4006010 tx=0x7fe2e40093a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1c8929186fe9cb4 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:33:14.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.324+0000 7fe2d7400700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-03T18:33:14.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.324+0000 7fe2d7400700 1 -- 172.21.15.33:0/2067462712 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e4027060 con 0x5565a98bf8d0 2024-05-03T18:33:14.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.325+0000 7fe2d7400700 1 -- 172.21.15.33:0/2067462712 <== mon.1 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7fe2e400b6b0 con 0x5565a98bf8d0 2024-05-03T18:33:14.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.325+0000 7fe306c5bb80 1 -- 172.21.15.33:0/2067462712 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2ec004d50 con 0x5565a98bf8d0 2024-05-03T18:33:14.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.327+0000 7fe2d7400700 1 -- 172.21.15.33:0/2067462712 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fe2e402e050 con 0x5565a98bf8d0 2024-05-03T18:33:14.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.327+0000 7fe306c5bb80 10 client.25702 fetch_fsmap learned FSMap version 21 2024-05-03T18:33:14.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.327+0000 7fe306c5bb80 1 -- 172.21.15.33:0/2067462712 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5565a98ea6f0 con 0x5565a98bf8d0 2024-05-03T18:33:14.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.328+0000 7fe2d7400700 1 -- 172.21.15.33:0/2067462712 <== mon.1 v2:172.21.15.150:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe2ec004d50 con 0x5565a98bf8d0 2024-05-03T18:33:14.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.329+0000 7fe2d7400700 1 -- 172.21.15.33:0/2067462712 <== mon.1 v2:172.21.15.150:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fe2e400b8b0 con 0x5565a98bf8d0 2024-05-03T18:33:14.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.329+0000 7fe306c5bb80 10 client.25702 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:33:14.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.329+0000 7fe306c5bb80 10 client.25702 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:33:14.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.329+0000 7fe306c5bb80 20 client.25702 populate_metadata read hostname 'smithi033' 2024-05-03T18:33:14.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.329+0000 7fe306c5bb80 1 --2- 172.21.15.33:0/2067462712 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5565a98ecd30 0x5565a98ef120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:14.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.329+0000 7fe306c5bb80 4 client.25702 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:33:14.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.329+0000 7fe306c5bb80 1 -- 172.21.15.33:0/2067462712 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5565a988eff0 con 0x5565a98ecd30 2024-05-03T18:33:14.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.330+0000 7fe2fa200700 1 --2- 172.21.15.33:0/2067462712 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5565a98ecd30 0x5565a98ef120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:14.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.331+0000 7fe2d7400700 10 client.25702 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:33:14.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.331+0000 7fe2fa200700 1 --2- 172.21.15.33:0/2067462712 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5565a98ecd30 0x5565a98ef120 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f89b400d4ec565b server_cookie=c76f8f07bc75575 in_seq=0 out_seq=0 2024-05-03T18:33:14.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.337+0000 7fe2d7400700 1 -- 172.21.15.33:0/2067462712 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5565a988eff0 con 0x5565a98ecd30 2024-05-03T18:33:14.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.337+0000 7fe2d7400700 10 client.25702 handle_command_reply: tid=0 2024-05-03T18:33:14.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.337+0000 7fe306c5bb80 1 client.25702 shutdown 2024-05-03T18:33:14.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.338+0000 7fe306c5bb80 1 -- 172.21.15.33:0/2067462712 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565a98bf8d0 msgr2=0x5565a98dc650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:33:14.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.338+0000 7fe306c5bb80 1 --2- 172.21.15.33:0/2067462712 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565a98bf8d0 0x5565a98dc650 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe2e4006010 tx=0x7fe2e40093a0 comp rx=0 tx=0).stop 2024-05-03T18:33:14.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.338+0000 7fe306c5bb80 1 -- 172.21.15.33:0/2067462712 shutdown_connections 2024-05-03T18:33:14.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.338+0000 7fe306c5bb80 1 --2- 172.21.15.33:0/2067462712 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5565a98ecd30 0x5565a98ef120 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:14.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.338+0000 7fe306c5bb80 1 --2- 172.21.15.33:0/2067462712 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5565a98c6c60 0x5565a98dcb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:14.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.338+0000 7fe306c5bb80 1 --2- 172.21.15.33:0/2067462712 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565a98bef10 0x5565a98df2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:14.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.338+0000 7fe306c5bb80 1 --2- 172.21.15.33:0/2067462712 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565a98bf8d0 0x5565a98dc650 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:14.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.338+0000 7fe2d7400700 0 client.25702 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:33:14.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.338+0000 7fe306c5bb80 1 -- 172.21.15.33:0/2067462712 >> 172.21.15.33:0/2067462712 conn(0x5565a983a6d0 msgr2=0x5565a98c9730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:33:14.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.339+0000 7fe306c5bb80 1 -- 172.21.15.33:0/2067462712 shutdown_connections 2024-05-03T18:33:14.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.339+0000 7fe306c5bb80 1 -- 172.21.15.33:0/2067462712 wait complete. 2024-05-03T18:33:14.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.339+0000 7fe306c5bb80 20 client.25702 trim_cache size 0 max 16384 2024-05-03T18:33:14.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.378+0000 7fe306c5bb80 1 Processor -- start 2024-05-03T18:33:14.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.378+0000 7fe306c5bb80 1 -- start start 2024-05-03T18:33:14.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.378+0000 7fe306c5bb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565a98bef10 0x5565a98c4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:14.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.378+0000 7fe306c5bb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5565a98c6c60 0x5565a98bc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:14.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.378+0000 7fe306c5bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565a98c54e0 0x5565a98bcbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:14.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.378+0000 7fe306c5bb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d800ccb0 con 0x5565a98c6c60 2024-05-03T18:33:14.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.378+0000 7fe306c5bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d800f530 con 0x5565a98c54e0 2024-05-03T18:33:14.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.378+0000 7fe306c5bb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d800d320 con 0x5565a98bef10 2024-05-03T18:33:14.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.379+0000 7fe2fa200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565a98bef10 0x5565a98c4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:14.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.379+0000 7fe2fa200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565a98bef10 0x5565a98c4fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:38104/0 (socket says 172.21.15.33:38104) 2024-05-03T18:33:14.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.379+0000 7fe2fa200700 1 -- 172.21.15.33:0/104158717 learned_addr learned my addr 172.21.15.33:0/104158717 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:33:14.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.379+0000 7fe2fa200700 1 --2- 172.21.15.33:0/104158717 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565a98c54e0 0x5565a98bcbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:14.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.380+0000 7fe2fa200700 1 -- 172.21.15.33:0/104158717 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565a98c54e0 msgr2=0x5565a98bcbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:33:14.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.380+0000 7fe2fa200700 1 --2- 172.21.15.33:0/104158717 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565a98c54e0 0x5565a98bcbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:14.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.380+0000 7fe2f8e00700 1 --2- 172.21.15.33:0/104158717 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5565a98c6c60 0x5565a98bc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:14.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.380+0000 7fe2fa200700 1 -- 172.21.15.33:0/104158717 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5565a98c6c60 msgr2=0x5565a98bc6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:33:14.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.380+0000 7fe2fa200700 1 --2- 172.21.15.33:0/104158717 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5565a98c6c60 0x5565a98bc6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:14.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.380+0000 7fe2fa200700 1 -- 172.21.15.33:0/104158717 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565a98db9a0 con 0x5565a98bef10 2024-05-03T18:33:14.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.380+0000 7fe2f8e00700 1 --2- 172.21.15.33:0/104158717 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5565a98c6c60 0x5565a98bc6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:33:14.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.381+0000 7fe2fa200700 1 --2- 172.21.15.33:0/104158717 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565a98bef10 0x5565a98c4fa0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec003f40 tx=0x7fe2ec000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc0c14da2a2cf6ce server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:33:14.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.381+0000 7fe2e2000700 1 -- 172.21.15.33:0/104158717 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2ec007780 con 0x5565a98bef10 2024-05-03T18:33:14.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.382+0000 7fe2e2000700 1 -- 172.21.15.33:0/104158717 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe2ec004960 con 0x5565a98bef10 2024-05-03T18:33:14.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.382+0000 7fe2e2000700 1 -- 172.21.15.33:0/104158717 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2ec029650 con 0x5565a98bef10 2024-05-03T18:33:14.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.382+0000 7fe306c5bb80 1 -- 172.21.15.33:0/104158717 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565a98bef10 msgr2=0x5565a98c4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:33:14.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.382+0000 7fe306c5bb80 1 --2- 172.21.15.33:0/104158717 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565a98bef10 0x5565a98c4fa0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec003f40 tx=0x7fe2ec000f40 comp rx=0 tx=0).stop 2024-05-03T18:33:14.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.383+0000 7fe306c5bb80 1 -- 172.21.15.33:0/104158717 shutdown_connections 2024-05-03T18:33:14.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.383+0000 7fe306c5bb80 1 --2- 172.21.15.33:0/104158717 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565a98c54e0 0x5565a98bcbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:14.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.383+0000 7fe306c5bb80 1 --2- 172.21.15.33:0/104158717 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565a98bef10 0x5565a98c4fa0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:14.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.383+0000 7fe306c5bb80 1 --2- 172.21.15.33:0/104158717 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5565a98c6c60 0x5565a98bc6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:14.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.383+0000 7fe306c5bb80 1 -- 172.21.15.33:0/104158717 >> 172.21.15.33:0/104158717 conn(0x5565a983a6d0 msgr2=0x5565a98c5910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:33:14.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.383+0000 7fe306c5bb80 1 -- 172.21.15.33:0/104158717 shutdown_connections 2024-05-03T18:33:14.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.383+0000 7fe306c5bb80 1 -- 172.21.15.33:0/104158717 wait complete. 2024-05-03T18:33:14.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.384+0000 7fe306c5bb80 1 -- start start 2024-05-03T18:33:14.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.385+0000 7fe306c5bb80 1 Processor -- start 2024-05-03T18:33:14.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.385+0000 7fe306c5bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565a98bef10 0x5565a98e5680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:14.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.385+0000 7fe306c5bb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565a98c54e0 0x5565a98e2390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:14.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.385+0000 7fe306c5bb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5565a98c6c60 0x5565a98e28d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:14.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.385+0000 7fe306c5bb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d800d9f0 con 0x5565a98c6c60 2024-05-03T18:33:14.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.385+0000 7fe306c5bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d800ee60 con 0x5565a98bef10 2024-05-03T18:33:14.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.385+0000 7fe306c5bb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d800e0c0 con 0x5565a98c54e0 2024-05-03T18:33:14.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.385+0000 7fe306c5bb80 10 client.0 _ll_register_callbacks cb 0x5565a97bbc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:33:14.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.386+0000 7fe2fa200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565a98bef10 0x5565a98e5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:14.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.386+0000 7fe2fa200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565a98bef10 0x5565a98e5680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:60462/0 (socket says 172.21.15.33:60462) 2024-05-03T18:33:14.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.386+0000 7fe2fa200700 1 -- 172.21.15.33:0/252884219 learned_addr learned my addr 172.21.15.33:0/252884219 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:33:14.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.386+0000 7fe2f8e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565a98c54e0 0x5565a98e2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:14.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.387+0000 7fe2fa200700 1 --2- 172.21.15.33:0/252884219 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5565a98c6c60 0x5565a98e28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:14.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.388+0000 7fe2f8e00700 1 -- 172.21.15.33:0/252884219 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565a98bef10 msgr2=0x5565a98e5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:33:14.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.388+0000 7fe2f8e00700 1 --2- 172.21.15.33:0/252884219 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565a98bef10 0x5565a98e5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:14.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.388+0000 7fe2f8e00700 1 -- 172.21.15.33:0/252884219 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5565a98c6c60 msgr2=0x5565a98e28d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:33:14.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.388+0000 7fe2f8e00700 1 --2- 172.21.15.33:0/252884219 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5565a98c6c60 0x5565a98e28d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:14.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.388+0000 7fe2f8e00700 1 -- 172.21.15.33:0/252884219 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565a98e2e40 con 0x5565a98c54e0 2024-05-03T18:33:14.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.390+0000 7fe2f8e00700 1 --2- 172.21.15.33:0/252884219 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565a98c54e0 0x5565a98e2390 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x5565a98dd6e0 tx=0x7fe2e4005f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a01eeb101b4f516 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:33:14.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.390+0000 7fe2d7400700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-03T18:33:14.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.391+0000 7fe2d7400700 1 -- 172.21.15.33:0/252884219 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e4027040 con 0x5565a98c54e0 2024-05-03T18:33:14.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.391+0000 7fe306c5bb80 1 -- 172.21.15.33:0/252884219 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2ec004960 con 0x5565a98c54e0 2024-05-03T18:33:14.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.391+0000 7fe2d7400700 1 -- 172.21.15.33:0/252884219 <== mon.1 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7fe2e400b6b0 con 0x5565a98c54e0 2024-05-03T18:33:14.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.392+0000 7fe2d7400700 1 -- 172.21.15.33:0/252884219 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fe2e403d050 con 0x5565a98c54e0 2024-05-03T18:33:14.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.392+0000 7fe306c5bb80 10 client.25708 fetch_fsmap learned FSMap version 21 2024-05-03T18:33:14.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.392+0000 7fe306c5bb80 1 -- 172.21.15.33:0/252884219 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5565a99582a0 con 0x5565a98c54e0 2024-05-03T18:33:14.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.395+0000 7fe2d7400700 1 -- 172.21.15.33:0/252884219 <== mon.1 v2:172.21.15.150:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe2ec004960 con 0x5565a98c54e0 2024-05-03T18:33:14.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.395+0000 7fe2d7400700 1 -- 172.21.15.33:0/252884219 <== mon.1 v2:172.21.15.150:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fe2e400fb00 con 0x5565a98c54e0 2024-05-03T18:33:14.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.396+0000 7fe306c5bb80 10 client.25708 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:33:14.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.396+0000 7fe306c5bb80 10 client.25708 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:33:14.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.396+0000 7fe306c5bb80 20 client.25708 populate_metadata read hostname 'smithi033' 2024-05-03T18:33:14.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.397+0000 7fe306c5bb80 1 --2- 172.21.15.33:0/252884219 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5565a99588e0 0x5565a995ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:14.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.397+0000 7fe306c5bb80 4 client.25708 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-03T18:33:14.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.397+0000 7fe306c5bb80 1 -- 172.21.15.33:0/252884219 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5565a9822400 con 0x5565a99588e0 2024-05-03T18:33:14.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.398+0000 7fe2fa200700 1 --2- 172.21.15.33:0/252884219 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5565a99588e0 0x5565a995ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:14.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.399+0000 7fe2fa200700 1 --2- 172.21.15.33:0/252884219 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5565a99588e0 0x5565a995ad00 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c11dc6170efc5e3a server_cookie=9dcd0de58b6523e5 in_seq=0 out_seq=0 2024-05-03T18:33:14.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.399+0000 7fe2d7400700 10 client.25708 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:33:14.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.401+0000 7fe2d7400700 1 -- 172.21.15.33:0/252884219 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5565a9822400 con 0x5565a99588e0 2024-05-03T18:33:14.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.401+0000 7fe2d7400700 10 client.25708 handle_command_reply: tid=0 2024-05-03T18:33:14.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.401+0000 7fe306c5bb80 1 client.25708 shutdown 2024-05-03T18:33:14.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.401+0000 7fe306c5bb80 1 -- 172.21.15.33:0/252884219 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565a98c54e0 msgr2=0x5565a98e2390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:33:14.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.401+0000 7fe306c5bb80 1 --2- 172.21.15.33:0/252884219 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565a98c54e0 0x5565a98e2390 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x5565a98dd6e0 tx=0x7fe2e4005f00 comp rx=0 tx=0).stop 2024-05-03T18:33:14.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.402+0000 7fe306c5bb80 1 -- 172.21.15.33:0/252884219 shutdown_connections 2024-05-03T18:33:14.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.402+0000 7fe306c5bb80 1 --2- 172.21.15.33:0/252884219 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5565a99588e0 0x5565a995ad00 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:14.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.402+0000 7fe306c5bb80 1 --2- 172.21.15.33:0/252884219 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5565a98c6c60 0x5565a98e28d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:14.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.402+0000 7fe306c5bb80 1 --2- 172.21.15.33:0/252884219 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565a98bef10 0x5565a98e5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:14.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.402+0000 7fe306c5bb80 1 --2- 172.21.15.33:0/252884219 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565a98c54e0 0x5565a98e2390 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:14.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.402+0000 7fe306c5bb80 1 -- 172.21.15.33:0/252884219 >> 172.21.15.33:0/252884219 conn(0x5565a983a6d0 msgr2=0x5565a98c5910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:33:14.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.402+0000 7fe2d7400700 0 client.25708 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:33:14.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.403+0000 7fe306c5bb80 1 -- 172.21.15.33:0/252884219 shutdown_connections 2024-05-03T18:33:14.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.403+0000 7fe306c5bb80 1 -- 172.21.15.33:0/252884219 wait complete. 2024-05-03T18:33:14.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.403+0000 7fe306c5bb80 20 client.25708 trim_cache size 0 max 16384 2024-05-03T18:33:14.404 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-03T18:33:14.404 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-05-03T18:33:14.404 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-05-03T18:33:14.404 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-03T18:33:14.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.405+0000 7fe2fac00700 1 -- 172.21.15.33:0/3215390563 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fe2dc0b29f0 msgr2=0x7fe2dc0b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:33:14.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.405+0000 7fe2fac00700 1 --2- 172.21.15.33:0/3215390563 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fe2dc0b29f0 0x7fe2dc0b4eb0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe2e80aa6c0 tx=0x7fe2e801f000 comp rx=0 tx=0).stop 2024-05-03T18:33:14.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.405+0000 7fe2fac00700 1 -- 172.21.15.33:0/3215390563 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe2f4102180 msgr2=0x7fe2f41a1850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:33:14.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.405+0000 7fe2fac00700 1 --2- 172.21.15.33:0/3215390563 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe2f4102180 0x7fe2f41a1850 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe2e800e9a0 tx=0x7fe2e800ecb0 comp rx=0 tx=0).stop 2024-05-03T18:33:14.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.405+0000 7fe2fac00700 1 -- 172.21.15.33:0/3215390563 shutdown_connections 2024-05-03T18:33:14.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.405+0000 7fe2fac00700 1 --2- 172.21.15.33:0/3215390563 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2f410fd30 0x7fe2f419ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:14.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.405+0000 7fe2fac00700 1 --2- 172.21.15.33:0/3215390563 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe2f4102180 0x7fe2f41a1850 secure :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe2e800e9a0 tx=0x7fe2e800ecb0 comp rx=0 tx=0).stop 2024-05-03T18:33:14.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.405+0000 7fe2fac00700 1 --2- 172.21.15.33:0/3215390563 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fe2dc0b29f0 0x7fe2dc0b4eb0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:14.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.405+0000 7fe2fac00700 1 --2- 172.21.15.33:0/3215390563 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2f4102b30 0x7fe2f41a1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:14.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.405+0000 7fe2fac00700 1 -- 172.21.15.33:0/3215390563 >> 172.21.15.33:0/3215390563 conn(0x7fe2f40fddc0 msgr2=0x7fe2f41001b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:33:14.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.406+0000 7fe2fac00700 1 -- 172.21.15.33:0/3215390563 shutdown_connections 2024-05-03T18:33:14.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:14.407+0000 7fe2fac00700 1 -- 172.21.15.33:0/3215390563 wait complete. 2024-05-03T18:33:15.110 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 120.0 s, 573.8 tps, lat 174.553 ms stddev 79.061 2024-05-03T18:33:15.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:33:14 smithi033 ceph-mon[28330]: pgmap v184: 129 pgs: 129 active+clean; 1.9 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 8.1 MiB/s wr, 812 op/s 2024-05-03T18:33:15.251 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:33:15 smithi099 ceph-mon[31658]: pgmap v184: 129 pgs: 129 active+clean; 1.9 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 8.1 MiB/s wr, 812 op/s 2024-05-03T18:33:15.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:33:14 smithi150 ceph-mon[32492]: pgmap v184: 129 pgs: 129 active+clean; 1.9 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 8.1 MiB/s wr, 812 op/s 2024-05-03T18:33:15.569 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-03T18:33:15.569 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T18:33:16.336 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:33:16.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:33:16 smithi150 ceph-mon[32492]: pgmap v185: 129 pgs: 129 active+clean; 1.9 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 5.0 MiB/s wr, 467 op/s 2024-05-03T18:33:16.486 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:33:16 smithi033 ceph-mon[28330]: pgmap v185: 129 pgs: 129 active+clean; 1.9 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 5.0 MiB/s wr, 467 op/s 2024-05-03T18:33:16.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:33:16 smithi099 ceph-mon[31658]: pgmap v185: 129 pgs: 129 active+clean; 1.9 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 5.0 MiB/s wr, 467 op/s 2024-05-03T18:33:19.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:33:19 smithi099 ceph-mon[31658]: pgmap v186: 129 pgs: 129 active+clean; 1.9 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 7.4 MiB/s wr, 751 op/s 2024-05-03T18:33:19.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:33:18 smithi150 ceph-mon[32492]: pgmap v186: 129 pgs: 129 active+clean; 1.9 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 7.4 MiB/s wr, 751 op/s 2024-05-03T18:33:19.486 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:33:18 smithi033 ceph-mon[28330]: pgmap v186: 129 pgs: 129 active+clean; 1.9 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 7.4 MiB/s wr, 751 op/s 2024-05-03T18:33:20.110 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 125.0 s, 579.4 tps, lat 172.393 ms stddev 85.126 2024-05-03T18:33:20.316 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:33:20 smithi033 ceph-mon[28330]: pgmap v187: 129 pgs: 129 active+clean; 1.9 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.6 MiB/s wr, 545 op/s 2024-05-03T18:33:20.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:33:20 smithi150 ceph-mon[32492]: pgmap v187: 129 pgs: 129 active+clean; 1.9 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.6 MiB/s wr, 545 op/s 2024-05-03T18:33:20.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:33:20 smithi099 ceph-mon[31658]: pgmap v187: 129 pgs: 129 active+clean; 1.9 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.6 MiB/s wr, 545 op/s 2024-05-03T18:33:20.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:20.669+0000 7f7466e00700 1 -- 172.21.15.33:0/3476120730 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f74601083d0 msgr2=0x7f746006b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:33:20.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:20.669+0000 7f7466e00700 1 --2- 172.21.15.33:0/3476120730 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f74601083d0 0x7f746006b1b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7450009990 tx=0x7f7450009ca0 comp rx=0 tx=0).stop 2024-05-03T18:33:20.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:20.669+0000 7f7466e00700 1 -- 172.21.15.33:0/3476120730 shutdown_connections 2024-05-03T18:33:20.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:20.669+0000 7f7466e00700 1 --2- 172.21.15.33:0/3476120730 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7460105ee0 0x7f7460106370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:20.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:20.669+0000 7f7466e00700 1 --2- 172.21.15.33:0/3476120730 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f74601083d0 0x7f746006b1b0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:20.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:20.669+0000 7f7466e00700 1 --2- 172.21.15.33:0/3476120730 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f746006b6f0 0x7f746006bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:20.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:20.670+0000 7f7466e00700 1 -- 172.21.15.33:0/3476120730 >> 172.21.15.33:0/3476120730 conn(0x7f7460079350 msgr2=0x7f7460079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:33:20.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:20.672+0000 7f7466e00700 1 -- 172.21.15.33:0/3476120730 shutdown_connections 2024-05-03T18:33:20.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:20.672+0000 7f7466e00700 1 -- 172.21.15.33:0/3476120730 wait complete. 2024-05-03T18:33:20.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:20.674+0000 7f7466e00700 1 Processor -- start 2024-05-03T18:33:20.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:20.674+0000 7f7466e00700 1 -- start start 2024-05-03T18:33:20.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:20.675+0000 7f7466e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f746006b6f0 0x7f7460074d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:20.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:20.675+0000 7f7466e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7460105ee0 0x7f746006de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:20.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:20.675+0000 7f7466e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74601083d0 0x7f746006e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:20.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:20.675+0000 7f7466e00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7460119b10 con 0x7f74601083d0 2024-05-03T18:33:20.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:20.675+0000 7f7466e00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7460119990 con 0x7f746006b6f0 2024-05-03T18:33:20.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:20.675+0000 7f7466e00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f7460119c90 con 0x7f7460105ee0 2024-05-03T18:33:20.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:20.676+0000 7f745f400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7460105ee0 0x7f746006de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:20.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:20.676+0000 7f745f400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7460105ee0 0x7f746006de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:52000/0 (socket says 172.21.15.33:52000) 2024-05-03T18:33:20.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:20.676+0000 7f745f400700 1 -- 172.21.15.33:0/4109571035 learned_addr learned my addr 172.21.15.33:0/4109571035 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:33:20.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:20.676+0000 7f7464e00700 1 --2- 172.21.15.33:0/4109571035 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74601083d0 0x7f746006e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:20.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:20.676+0000 7f745fe00700 1 --2- 172.21.15.33:0/4109571035 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f746006b6f0 0x7f7460074d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:20.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:20.677+0000 7f745f400700 1 -- 172.21.15.33:0/4109571035 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f746006b6f0 msgr2=0x7f7460074d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:33:20.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:20.677+0000 7f745f400700 1 --2- 172.21.15.33:0/4109571035 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f746006b6f0 0x7f7460074d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:20.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:20.677+0000 7f745f400700 1 -- 172.21.15.33:0/4109571035 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74601083d0 msgr2=0x7f746006e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:33:20.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:20.677+0000 7f745f400700 1 --2- 172.21.15.33:0/4109571035 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74601083d0 0x7f746006e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:20.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:20.677+0000 7f745f400700 1 -- 172.21.15.33:0/4109571035 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f746006eab0 con 0x7f7460105ee0 2024-05-03T18:33:20.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:20.679+0000 7f745f400700 1 --2- 172.21.15.33:0/4109571035 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7460105ee0 0x7f746006de40 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f744c00d950 tx=0x7f744c00dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:33:20.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:20.680+0000 7f745cc00700 1 -- 172.21.15.33:0/4109571035 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f744c016070 con 0x7f7460105ee0 2024-05-03T18:33:20.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:20.680+0000 7f745cc00700 1 -- 172.21.15.33:0/4109571035 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f744c009dd0 con 0x7f7460105ee0 2024-05-03T18:33:20.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:20.680+0000 7f745cc00700 1 -- 172.21.15.33:0/4109571035 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f744c00f3c0 con 0x7f7460105ee0 2024-05-03T18:33:20.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:20.681+0000 7f7466e00700 1 -- 172.21.15.33:0/4109571035 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7460072890 con 0x7f7460105ee0 2024-05-03T18:33:20.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:20.681+0000 7f7466e00700 1 -- 172.21.15.33:0/4109571035 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7460072d50 con 0x7f7460105ee0 2024-05-03T18:33:20.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:20.687+0000 7f7466e00700 1 -- 172.21.15.33:0/4109571035 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74601029c0 con 0x7f7460105ee0 2024-05-03T18:33:20.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:20.691+0000 7f745cc00700 1 -- 172.21.15.33:0/4109571035 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f744c019070 con 0x7f7460105ee0 2024-05-03T18:33:20.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:20.693+0000 7f745cc00700 1 --2- 172.21.15.33:0/4109571035 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f74440b2b40 0x7f74440b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:20.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:20.694+0000 7f745cc00700 1 -- 172.21.15.33:0/4109571035 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f744c0e2390 con 0x7f7460105ee0 2024-05-03T18:33:20.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:20.695+0000 7f745fe00700 1 --2- 172.21.15.33:0/4109571035 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f74440b2b40 0x7f74440b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:20.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:20.698+0000 7f745fe00700 1 --2- 172.21.15.33:0/4109571035 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f74440b2b40 0x7f74440b5000 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7450006010 tx=0x7f745000b540 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:33:20.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:20.698+0000 7f745cc00700 1 -- 172.21.15.33:0/4109571035 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f744c020070 con 0x7f7460105ee0 2024-05-03T18:33:21.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:21.210+0000 7f7466e00700 1 -- 172.21.15.33:0/4109571035 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f746006f320 con 0x7f7460105ee0 2024-05-03T18:33:21.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:21.212+0000 7f745cc00700 1 -- 172.21.15.33:0/4109571035 <== mon.1 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11801 (secure 0 0 0) 0x7f744c0a99e0 con 0x7f7460105ee0 2024-05-03T18:33:21.217 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:33:21.217 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1327393503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1327393503},{"type":"v1","addr":"172.21.15.33:6835","nonce":1327393503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/2461472948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":2461472948},{"type":"v1","addr":"172.21.15.33:6837","nonce":2461472948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14559,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/442269281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":442269281},{"type":"v1","addr":"172.21.15.33:6839","nonce":442269281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14571,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/645571756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":645571756},{"type":"v1","addr":"172.21.15.33:6841","nonce":645571756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/203598909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":203598909},{"type":"v1","addr":"172.21.15.150:6833","nonce":203598909}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/4029842493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":4029842493},{"type":"v1","addr":"172.21.15.150:6835","nonce":4029842493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3463664423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3463664423},{"type":"v1","addr":"172.21.15.150:6837","nonce":3463664423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24481,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6839/4051648018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":4051648018},{"type":"v1","addr":"172.21.15.150:6839","nonce":4051648018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34418,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1865105479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1865105479},{"type":"v1","addr":"172.21.15.99:6833","nonce":1865105479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34445,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/1206393914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1206393914},{"type":"v1","addr":"172.21.15.99:6835","nonce":1206393914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-03T18:22:28.837091+0000","modified":"2024-05-03T18:23:49.758398+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34508},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34472":{"gid":34472,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6837/3360970342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3360970342},{"type":"v1","addr":"172.21.15.99:6837","nonce":3360970342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34508":{"gid":34508,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/919907308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":919907308},{"type":"v1","addr":"172.21.15.99:6839","nonce":919907308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34508,"qdb_cluster":[34508]},"id":1}]} 2024-05-03T18:33:21.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:21.216+0000 7f7466e00700 1 -- 172.21.15.33:0/4109571035 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f74440b2b40 msgr2=0x7f74440b5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:33:21.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:21.216+0000 7f7466e00700 1 --2- 172.21.15.33:0/4109571035 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f74440b2b40 0x7f74440b5000 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7450006010 tx=0x7f745000b540 comp rx=0 tx=0).stop 2024-05-03T18:33:21.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:21.216+0000 7f7466e00700 1 -- 172.21.15.33:0/4109571035 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7460105ee0 msgr2=0x7f746006de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:33:21.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:21.216+0000 7f7466e00700 1 --2- 172.21.15.33:0/4109571035 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7460105ee0 0x7f746006de40 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f744c00d950 tx=0x7f744c00dc60 comp rx=0 tx=0).stop 2024-05-03T18:33:21.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:21.216+0000 7f7466e00700 1 -- 172.21.15.33:0/4109571035 shutdown_connections 2024-05-03T18:33:21.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:21.216+0000 7f7466e00700 1 --2- 172.21.15.33:0/4109571035 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74601083d0 0x7f746006e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:21.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:21.216+0000 7f7466e00700 1 --2- 172.21.15.33:0/4109571035 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f74440b2b40 0x7f74440b5000 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:21.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:21.216+0000 7f7466e00700 1 --2- 172.21.15.33:0/4109571035 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f746006b6f0 0x7f7460074d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:21.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:21.216+0000 7f7466e00700 1 --2- 172.21.15.33:0/4109571035 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7460105ee0 0x7f746006de40 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:21.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:21.216+0000 7f7466e00700 1 -- 172.21.15.33:0/4109571035 >> 172.21.15.33:0/4109571035 conn(0x7f7460079350 msgr2=0x7f7460101390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:33:21.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:21.217+0000 7f7466e00700 1 -- 172.21.15.33:0/4109571035 shutdown_connections 2024-05-03T18:33:21.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:21.217+0000 7f7466e00700 1 -- 172.21.15.33:0/4109571035 wait complete. 2024-05-03T18:33:21.220 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 21 2024-05-03T18:33:21.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:33:21 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/4109571035' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:33:21.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:33:21 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/4109571035' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:33:21.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:33:21 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/4109571035' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:33:22.428 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-03T18:33:22.521 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:33:22 smithi033 ceph-mon[28330]: pgmap v188: 129 pgs: 129 active+clean; 1.9 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.6 MiB/s wr, 545 op/s 2024-05-03T18:33:22.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:33:22 smithi099 ceph-mon[31658]: pgmap v188: 129 pgs: 129 active+clean; 1.9 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.6 MiB/s wr, 545 op/s 2024-05-03T18:33:22.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:33:22 smithi150 ceph-mon[32492]: pgmap v188: 129 pgs: 129 active+clean; 1.9 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.6 MiB/s wr, 545 op/s 2024-05-03T18:33:23.161 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:33:25.114 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 130.0 s, 425.3 tps, lat 177.411 ms stddev 59.127 2024-05-03T18:33:25.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:33:25 smithi099 ceph-mon[31658]: pgmap v189: 129 pgs: 129 active+clean; 1.9 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 7.4 MiB/s wr, 830 op/s 2024-05-03T18:33:25.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:33:25 smithi150 ceph-mon[32492]: pgmap v189: 129 pgs: 129 active+clean; 1.9 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 7.4 MiB/s wr, 830 op/s 2024-05-03T18:33:25.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:33:25 smithi033 ceph-mon[28330]: pgmap v189: 129 pgs: 129 active+clean; 1.9 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 7.4 MiB/s wr, 830 op/s 2024-05-03T18:33:26.139 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:33:26 smithi033 ceph-mon[28330]: pgmap v190: 129 pgs: 129 active+clean; 1.9 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.2 MiB/s wr, 568 op/s 2024-05-03T18:33:26.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.250+0000 7f46bdc00700 1 -- 172.21.15.33:0/1736227405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46b8075840 msgr2=0x7f46b8075c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:33:26.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.250+0000 7f46bdc00700 1 --2- 172.21.15.33:0/1736227405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46b8075840 0x7f46b8075c20 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f46b0009990 tx=0x7f46b0009ca0 comp rx=0 tx=0).stop 2024-05-03T18:33:26.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.250+0000 7f46bdc00700 1 -- 172.21.15.33:0/1736227405 shutdown_connections 2024-05-03T18:33:26.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.250+0000 7f46bdc00700 1 --2- 172.21.15.33:0/1736227405 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f46b81143e0 0x7f46b81167d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:26.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.250+0000 7f46bdc00700 1 --2- 172.21.15.33:0/1736227405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46b8075840 0x7f46b8075c20 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:26.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.250+0000 7f46bdc00700 1 --2- 172.21.15.33:0/1736227405 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46b8076160 0x7f46b8113ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:26.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.250+0000 7f46bdc00700 1 -- 172.21.15.33:0/1736227405 >> 172.21.15.33:0/1736227405 conn(0x7f46b80fef20 msgr2=0x7f46b8101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:33:26.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.250+0000 7f46bdc00700 1 -- 172.21.15.33:0/1736227405 shutdown_connections 2024-05-03T18:33:26.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.250+0000 7f46bdc00700 1 -- 172.21.15.33:0/1736227405 wait complete. 2024-05-03T18:33:26.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.252+0000 7f46bdc00700 1 Processor -- start 2024-05-03T18:33:26.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.252+0000 7f46bdc00700 1 -- start start 2024-05-03T18:33:26.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.253+0000 7f46bdc00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46b8075840 0x7f46b819fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:26.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.253+0000 7f46bdc00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46b8076160 0x7f46b81a0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:26.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.253+0000 7f46bdc00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f46b81143e0 0x7f46b81a48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:26.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.253+0000 7f46bdc00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f46b8119ac0 con 0x7f46b8076160 2024-05-03T18:33:26.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.253+0000 7f46bdc00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f46b8119940 con 0x7f46b8075840 2024-05-03T18:33:26.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.253+0000 7f46bdc00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f46b8119c40 con 0x7f46b81143e0 2024-05-03T18:33:26.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.254+0000 7f46b7e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46b8075840 0x7f46b819fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:26.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.254+0000 7f46b7400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46b8076160 0x7f46b81a0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:26.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.254+0000 7f46b7400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46b8076160 0x7f46b81a0530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43052/0 (socket says 172.21.15.33:43052) 2024-05-03T18:33:26.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.254+0000 7f46b7400700 1 -- 172.21.15.33:0/3107866039 learned_addr learned my addr 172.21.15.33:0/3107866039 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:33:26.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.254+0000 7f46bd200700 1 --2- 172.21.15.33:0/3107866039 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f46b81143e0 0x7f46b81a48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:26.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.255+0000 7f46b7400700 1 -- 172.21.15.33:0/3107866039 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f46b81143e0 msgr2=0x7f46b81a48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:33:26.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.255+0000 7f46b7400700 1 --2- 172.21.15.33:0/3107866039 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f46b81143e0 0x7f46b81a48c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:26.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.255+0000 7f46b7400700 1 -- 172.21.15.33:0/3107866039 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46b8075840 msgr2=0x7f46b819fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:33:26.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.255+0000 7f46b7400700 1 --2- 172.21.15.33:0/3107866039 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46b8075840 0x7f46b819fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:26.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.255+0000 7f46b7400700 1 -- 172.21.15.33:0/3107866039 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46b81a5000 con 0x7f46b8076160 2024-05-03T18:33:26.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.256+0000 7f46b7400700 1 --2- 172.21.15.33:0/3107866039 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46b8076160 0x7f46b81a0530 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f46a00060b0 tx=0x7f46a0007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:33:26.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.257+0000 7f46b4c00700 1 -- 172.21.15.33:0/3107866039 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a001a070 con 0x7f46b8076160 2024-05-03T18:33:26.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.257+0000 7f46b4c00700 1 -- 172.21.15.33:0/3107866039 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f46a000dd60 con 0x7f46b8076160 2024-05-03T18:33:26.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.257+0000 7f46b4c00700 1 -- 172.21.15.33:0/3107866039 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a00145e0 con 0x7f46b8076160 2024-05-03T18:33:26.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.257+0000 7f46bdc00700 1 -- 172.21.15.33:0/3107866039 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46b81a52f0 con 0x7f46b8076160 2024-05-03T18:33:26.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.258+0000 7f46bdc00700 1 -- 172.21.15.33:0/3107866039 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46b81acb70 con 0x7f46b8076160 2024-05-03T18:33:26.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.263+0000 7f46c9b57b80 1 Processor -- start 2024-05-03T18:33:26.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.264+0000 7f46c9b57b80 1 -- start start 2024-05-03T18:33:26.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.265+0000 7f46c9b57b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f50a28fbd0 0x55f50a28ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:26.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.265+0000 7f46c9b57b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f50a289f30 0x55f50a28a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:26.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.265+0000 7f46c9b57b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f50a28a8f0 0x55f50a296990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:26.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.265+0000 7f46c9b57b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55f50a1dca20 con 0x55f50a28a8f0 2024-05-03T18:33:26.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.265+0000 7f46c9b57b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55f50a17a760 con 0x55f50a289f30 2024-05-03T18:33:26.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.265+0000 7f46c9b57b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55f50a1d89e0 con 0x55f50a28fbd0 2024-05-03T18:33:26.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.265+0000 7f46b4c00700 1 -- 172.21.15.33:0/3107866039 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f46a001d070 con 0x7f46b8076160 2024-05-03T18:33:26.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.266+0000 7f46b7e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f50a289f30 0x55f50a28a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:26.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.266+0000 7f46b7e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f50a289f30 0x55f50a28a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:50298/0 (socket says 172.21.15.33:50298) 2024-05-03T18:33:26.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.266+0000 7f46b7e00700 1 -- 172.21.15.33:0/4257309617 learned_addr learned my addr 172.21.15.33:0/4257309617 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:33:26.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.266+0000 7f46b7400700 1 --2- 172.21.15.33:0/4257309617 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f50a28a8f0 0x55f50a296990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:26.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.266+0000 7f46b7e00700 1 -- 172.21.15.33:0/4257309617 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f50a28fbd0 msgr2=0x55f50a28ffb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-03T18:33:26.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.267+0000 7f46bd200700 1 --2- 172.21.15.33:0/4257309617 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f50a28fbd0 0x55f50a28ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:26.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.267+0000 7f46b7e00700 1 --2- 172.21.15.33:0/4257309617 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f50a28fbd0 0x55f50a28ffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:26.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.267+0000 7f46b7e00700 1 -- 172.21.15.33:0/4257309617 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f50a28a8f0 msgr2=0x55f50a296990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:33:26.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.267+0000 7f46b7e00700 1 --2- 172.21.15.33:0/4257309617 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f50a28a8f0 0x55f50a296990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:26.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.267+0000 7f46b7e00700 1 -- 172.21.15.33:0/4257309617 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f50a1298d0 con 0x55f50a289f30 2024-05-03T18:33:26.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.267+0000 7f46b7400700 1 --2- 172.21.15.33:0/4257309617 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f50a28a8f0 0x55f50a296990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:33:26.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.267+0000 7f46b4c00700 1 --2- 172.21.15.33:0/3107866039 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f469c0b2a70 0x7f469c0b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:26.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.267+0000 7f46b4c00700 1 -- 172.21.15.33:0/3107866039 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f46a00e21b0 con 0x7f46b8076160 2024-05-03T18:33:26.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.267+0000 7f46bd200700 1 --2- 172.21.15.33:0/3107866039 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f469c0b2a70 0x7f469c0b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:26.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.267+0000 7f46b7e00700 1 --2- 172.21.15.33:0/4257309617 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f50a289f30 0x55f50a28a3b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f46b00096d0 tx=0x7f46b0009700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7bc555e5c238702 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:33:26.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.268+0000 7f46ad600700 1 -- 172.21.15.33:0/4257309617 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b0026070 con 0x55f50a289f30 2024-05-03T18:33:26.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.268+0000 7f46ad600700 1 -- 172.21.15.33:0/4257309617 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f46b0005550 con 0x55f50a289f30 2024-05-03T18:33:26.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.268+0000 7f46ad600700 1 -- 172.21.15.33:0/4257309617 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b000ba30 con 0x55f50a289f30 2024-05-03T18:33:26.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.269+0000 7f46c9b57b80 1 -- 172.21.15.33:0/4257309617 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f50a289f30 msgr2=0x55f50a28a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:33:26.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.269+0000 7f46c9b57b80 1 --2- 172.21.15.33:0/4257309617 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f50a289f30 0x55f50a28a3b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f46b00096d0 tx=0x7f46b0009700 comp rx=0 tx=0).stop 2024-05-03T18:33:26.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.269+0000 7f46bd200700 1 --2- 172.21.15.33:0/3107866039 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f469c0b2a70 0x7f469c0b4f30 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f46b806bdb0 tx=0x7f46a8007320 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:33:26.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.269+0000 7f46c9b57b80 1 -- 172.21.15.33:0/4257309617 shutdown_connections 2024-05-03T18:33:26.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.269+0000 7f46c9b57b80 1 --2- 172.21.15.33:0/4257309617 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f50a28a8f0 0x55f50a296990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:26.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.269+0000 7f46c9b57b80 1 --2- 172.21.15.33:0/4257309617 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f50a28fbd0 0x55f50a28ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:26.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.269+0000 7f46c9b57b80 1 --2- 172.21.15.33:0/4257309617 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f50a289f30 0x55f50a28a3b0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:26.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.269+0000 7f46c9b57b80 1 -- 172.21.15.33:0/4257309617 >> 172.21.15.33:0/4257309617 conn(0x55f50a2026d0 msgr2=0x55f50a284bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:33:26.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.270+0000 7f46c9b57b80 1 -- 172.21.15.33:0/4257309617 shutdown_connections 2024-05-03T18:33:26.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.270+0000 7f46c9b57b80 1 -- 172.21.15.33:0/4257309617 wait complete. 2024-05-03T18:33:26.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.270+0000 7f46c9b57b80 1 -- start start 2024-05-03T18:33:26.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.271+0000 7f46c9b57b80 1 Processor -- start 2024-05-03T18:33:26.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.271+0000 7f46c9b57b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f50a289f30 0x55f50a296590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:26.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.271+0000 7f46c9b57b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f50a28a8f0 0x55f50a293980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:26.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.271+0000 7f46c9b57b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f50a28fbd0 0x55f50a293ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:26.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.271+0000 7f46c9b57b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55f50a296ed0 con 0x55f50a289f30 2024-05-03T18:33:26.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.271+0000 7f46c9b57b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55f50a142b20 con 0x55f50a28fbd0 2024-05-03T18:33:26.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.271+0000 7f46c9b57b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55f50a297050 con 0x55f50a28a8f0 2024-05-03T18:33:26.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.271+0000 7f46c9b57b80 10 client.0 _ll_register_callbacks cb 0x55f50a183c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:33:26.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.272+0000 7f46b7400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f50a28a8f0 0x55f50a293980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:26.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.272+0000 7f46b7400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f50a28a8f0 0x55f50a293980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:53014/0 (socket says 172.21.15.33:53014) 2024-05-03T18:33:26.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.272+0000 7f46b7400700 1 -- 172.21.15.33:0/700683693 learned_addr learned my addr 172.21.15.33:0/700683693 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:33:26.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.272+0000 7f46bd200700 1 --2- 172.21.15.33:0/700683693 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f50a28fbd0 0x55f50a293ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:26.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.273+0000 7f46b7e00700 1 --2- 172.21.15.33:0/700683693 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f50a289f30 0x55f50a296590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:26.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.273+0000 7f46b7400700 1 -- 172.21.15.33:0/700683693 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f50a28fbd0 msgr2=0x55f50a293ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:33:26.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.273+0000 7f46b7400700 1 --2- 172.21.15.33:0/700683693 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f50a28fbd0 0x55f50a293ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:26.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.273+0000 7f46b7400700 1 -- 172.21.15.33:0/700683693 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f50a289f30 msgr2=0x55f50a296590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:33:26.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.273+0000 7f46b7400700 1 --2- 172.21.15.33:0/700683693 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f50a289f30 0x55f50a296590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:26.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.273+0000 7f46b7400700 1 -- 172.21.15.33:0/700683693 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f50a1a2420 con 0x55f50a28a8f0 2024-05-03T18:33:26.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.273+0000 7f46b7e00700 1 --2- 172.21.15.33:0/700683693 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f50a289f30 0x55f50a296590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:33:26.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.274+0000 7f46b7400700 1 --2- 172.21.15.33:0/700683693 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f50a28a8f0 0x55f50a293980 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f46a00a8e00 tx=0x7f46a00a5610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c68cbf1b6a8dd48 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:33:26.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.274+0000 7f46a6000700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-03T18:33:26.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.275+0000 7f46a6000700 1 -- 172.21.15.33:0/700683693 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a00a9800 con 0x55f50a28a8f0 2024-05-03T18:33:26.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.275+0000 7f46a6000700 1 -- 172.21.15.33:0/700683693 <== mon.1 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f46a00b1ce0 con 0x55f50a28a8f0 2024-05-03T18:33:26.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.275+0000 7f46c9b57b80 1 -- 172.21.15.33:0/700683693 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46b0005550 con 0x55f50a28a8f0 2024-05-03T18:33:26.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.276+0000 7f46a6000700 1 -- 172.21.15.33:0/700683693 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f46a00a9100 con 0x55f50a28a8f0 2024-05-03T18:33:26.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.276+0000 7f46c9b57b80 10 client.25723 fetch_fsmap learned FSMap version 21 2024-05-03T18:33:26.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.276+0000 7f46c9b57b80 1 -- 172.21.15.33:0/700683693 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55f50a2a1650 con 0x55f50a28a8f0 2024-05-03T18:33:26.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.276+0000 7f46a6000700 1 -- 172.21.15.33:0/700683693 <== mon.1 v2:172.21.15.150:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f46b0005550 con 0x55f50a28a8f0 2024-05-03T18:33:26.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.277+0000 7f46a6000700 1 -- 172.21.15.33:0/700683693 <== mon.1 v2:172.21.15.150:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f46a00a4a70 con 0x55f50a28a8f0 2024-05-03T18:33:26.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.277+0000 7f46c9b57b80 10 client.25723 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:33:26.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.277+0000 7f46c9b57b80 10 client.25723 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:33:26.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.277+0000 7f46c9b57b80 20 client.25723 populate_metadata read hostname 'smithi033' 2024-05-03T18:33:26.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.277+0000 7f46c9b57b80 1 --2- 172.21.15.33:0/700683693 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55f50a2a3c90 0x55f50a2a6080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:26.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.277+0000 7f46c9b57b80 4 client.25723 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:33:26.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.277+0000 7f46c9b57b80 1 -- 172.21.15.33:0/700683693 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f50a256ff0 con 0x55f50a2a3c90 2024-05-03T18:33:26.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.278+0000 7f46b7e00700 1 --2- 172.21.15.33:0/700683693 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55f50a2a3c90 0x55f50a2a6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:26.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.279+0000 7f46b7e00700 1 --2- 172.21.15.33:0/700683693 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55f50a2a3c90 0x55f50a2a6080 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13d8393561c846a server_cookie=4150679c064410b0 in_seq=0 out_seq=0 2024-05-03T18:33:26.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.279+0000 7f46a6000700 10 client.25723 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:33:26.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.283+0000 7f46a6000700 1 -- 172.21.15.33:0/700683693 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f50a256ff0 con 0x55f50a2a3c90 2024-05-03T18:33:26.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.283+0000 7f46a6000700 10 client.25723 handle_command_reply: tid=0 2024-05-03T18:33:26.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.284+0000 7f46c9b57b80 1 client.25723 shutdown 2024-05-03T18:33:26.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.284+0000 7f46c9b57b80 1 -- 172.21.15.33:0/700683693 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f50a28a8f0 msgr2=0x55f50a293980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:33:26.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.284+0000 7f46c9b57b80 1 --2- 172.21.15.33:0/700683693 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f50a28a8f0 0x55f50a293980 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f46a00a8e00 tx=0x7f46a00a5610 comp rx=0 tx=0).stop 2024-05-03T18:33:26.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.284+0000 7f46c9b57b80 1 -- 172.21.15.33:0/700683693 shutdown_connections 2024-05-03T18:33:26.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.284+0000 7f46c9b57b80 1 --2- 172.21.15.33:0/700683693 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55f50a2a3c90 0x55f50a2a6080 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:26.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.284+0000 7f46c9b57b80 1 --2- 172.21.15.33:0/700683693 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f50a28fbd0 0x55f50a293ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:26.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.284+0000 7f46c9b57b80 1 --2- 172.21.15.33:0/700683693 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f50a289f30 0x55f50a296590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:26.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.284+0000 7f46c9b57b80 1 --2- 172.21.15.33:0/700683693 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f50a28a8f0 0x55f50a293980 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:26.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.284+0000 7f46c9b57b80 1 -- 172.21.15.33:0/700683693 >> 172.21.15.33:0/700683693 conn(0x55f50a2026d0 msgr2=0x55f50a288f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:33:26.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.285+0000 7f46a6000700 0 client.25723 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:33:26.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.286+0000 7f46c9b57b80 1 -- 172.21.15.33:0/700683693 shutdown_connections 2024-05-03T18:33:26.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.286+0000 7f46c9b57b80 1 -- 172.21.15.33:0/700683693 wait complete. 2024-05-03T18:33:26.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.286+0000 7f46c9b57b80 20 client.25723 trim_cache size 0 max 16384 2024-05-03T18:33:26.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:33:26 smithi099 ceph-mon[31658]: pgmap v190: 129 pgs: 129 active+clean; 1.9 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.2 MiB/s wr, 568 op/s 2024-05-03T18:33:26.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.314+0000 7f46c9b57b80 1 Processor -- start 2024-05-03T18:33:26.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.315+0000 7f46c9b57b80 1 -- start start 2024-05-03T18:33:26.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.315+0000 7f46c9b57b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f50a289f30 0x55f50a29ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:26.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.315+0000 7f46c9b57b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f50a28a8f0 0x55f50a292270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:26.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.315+0000 7f46c9b57b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f50a28fbd0 0x55f50a2927b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:26.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.315+0000 7f46c9b57b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f46b81b3930 con 0x55f50a28a8f0 2024-05-03T18:33:26.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.315+0000 7f46c9b57b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f46b81b61b0 con 0x55f50a28fbd0 2024-05-03T18:33:26.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.315+0000 7f46c9b57b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f46b81b3fa0 con 0x55f50a289f30 2024-05-03T18:33:26.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.316+0000 7f46b7400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f50a28a8f0 0x55f50a292270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:26.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.316+0000 7f46b7e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f50a289f30 0x55f50a29ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:26.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.316+0000 7f46b7400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f50a28a8f0 0x55f50a292270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43080/0 (socket says 172.21.15.33:43080) 2024-05-03T18:33:26.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.316+0000 7f46bd200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f50a28fbd0 0x55f50a2927b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:26.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.316+0000 7f46b7400700 1 -- 172.21.15.33:0/2254895473 learned_addr learned my addr 172.21.15.33:0/2254895473 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:33:26.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.317+0000 7f46b7400700 1 -- 172.21.15.33:0/2254895473 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f50a289f30 msgr2=0x55f50a29ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:33:26.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.317+0000 7f46b7400700 1 --2- 172.21.15.33:0/2254895473 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f50a289f30 0x55f50a29ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:26.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.317+0000 7f46b7400700 1 -- 172.21.15.33:0/2254895473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f50a28fbd0 msgr2=0x55f50a2927b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:33:26.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.317+0000 7f46b7400700 1 --2- 172.21.15.33:0/2254895473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f50a28fbd0 0x55f50a2927b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:26.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.317+0000 7f46b7400700 1 -- 172.21.15.33:0/2254895473 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f50a256fd0 con 0x55f50a28a8f0 2024-05-03T18:33:26.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.318+0000 7f46b7400700 1 --2- 172.21.15.33:0/2254895473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f50a28a8f0 0x55f50a292270 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f46a00a5180 tx=0x7f46a0009490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69fcb55973daed55 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:33:26.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.318+0000 7f46acc00700 1 -- 172.21.15.33:0/2254895473 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a00b1f60 con 0x55f50a28a8f0 2024-05-03T18:33:26.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.318+0000 7f46acc00700 1 -- 172.21.15.33:0/2254895473 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f46a00ad4b0 con 0x55f50a28a8f0 2024-05-03T18:33:26.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.318+0000 7f46acc00700 1 -- 172.21.15.33:0/2254895473 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a00a4a70 con 0x55f50a28a8f0 2024-05-03T18:33:26.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.320+0000 7f46c9b57b80 1 -- 172.21.15.33:0/2254895473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f50a28a8f0 msgr2=0x55f50a292270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:33:26.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.320+0000 7f46c9b57b80 1 --2- 172.21.15.33:0/2254895473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f50a28a8f0 0x55f50a292270 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f46a00a5180 tx=0x7f46a0009490 comp rx=0 tx=0).stop 2024-05-03T18:33:26.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.320+0000 7f46c9b57b80 1 -- 172.21.15.33:0/2254895473 shutdown_connections 2024-05-03T18:33:26.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.320+0000 7f46c9b57b80 1 --2- 172.21.15.33:0/2254895473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f50a28fbd0 0x55f50a2927b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:26.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.320+0000 7f46c9b57b80 1 --2- 172.21.15.33:0/2254895473 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f50a289f30 0x55f50a29ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:26.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.320+0000 7f46c9b57b80 1 --2- 172.21.15.33:0/2254895473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f50a28a8f0 0x55f50a292270 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:26.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.320+0000 7f46c9b57b80 1 -- 172.21.15.33:0/2254895473 >> 172.21.15.33:0/2254895473 conn(0x55f50a2026d0 msgr2=0x55f50a2846b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:33:26.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.321+0000 7f46c9b57b80 1 -- 172.21.15.33:0/2254895473 shutdown_connections 2024-05-03T18:33:26.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.321+0000 7f46c9b57b80 1 -- 172.21.15.33:0/2254895473 wait complete. 2024-05-03T18:33:26.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.322+0000 7f46c9b57b80 1 -- start start 2024-05-03T18:33:26.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.322+0000 7f46c9b57b80 1 Processor -- start 2024-05-03T18:33:26.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.323+0000 7f46c9b57b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f50a289f30 0x55f50a29c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:26.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.323+0000 7f46c9b57b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f50a28a8f0 0x55f50a2aadc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:26.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.323+0000 7f46c9b57b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f50a28fbd0 0x55f50a2ab300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:26.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.323+0000 7f46c9b57b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f46b81b4670 con 0x55f50a289f30 2024-05-03T18:33:26.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.323+0000 7f46c9b57b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f46b81b5ae0 con 0x55f50a28fbd0 2024-05-03T18:33:26.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.323+0000 7f46c9b57b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f46b81b4d40 con 0x55f50a28a8f0 2024-05-03T18:33:26.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.324+0000 7f46c9b57b80 10 client.0 _ll_register_callbacks cb 0x55f50a183c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:33:26.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.324+0000 7f46bd200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f50a28fbd0 0x55f50a2ab300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:26.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.324+0000 7f46b7400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f50a28a8f0 0x55f50a2aadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:26.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.324+0000 7f46b7400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f50a28a8f0 0x55f50a2aadc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:53030/0 (socket says 172.21.15.33:53030) 2024-05-03T18:33:26.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.324+0000 7f46b7400700 1 -- 172.21.15.33:0/2368780975 learned_addr learned my addr 172.21.15.33:0/2368780975 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:33:26.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.325+0000 7f46b7e00700 1 --2- 172.21.15.33:0/2368780975 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f50a289f30 0x55f50a29c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:26.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.325+0000 7f46b7400700 1 -- 172.21.15.33:0/2368780975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f50a28fbd0 msgr2=0x55f50a2ab300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:33:26.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.325+0000 7f46b7400700 1 --2- 172.21.15.33:0/2368780975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f50a28fbd0 0x55f50a2ab300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:26.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.325+0000 7f46b7400700 1 -- 172.21.15.33:0/2368780975 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f50a289f30 msgr2=0x55f50a29c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:33:26.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.325+0000 7f46b7400700 1 --2- 172.21.15.33:0/2368780975 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f50a289f30 0x55f50a29c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:26.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.325+0000 7f46b7400700 1 -- 172.21.15.33:0/2368780975 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f50a2814a0 con 0x55f50a28a8f0 2024-05-03T18:33:26.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.326+0000 7f46b7400700 1 --2- 172.21.15.33:0/2368780975 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f50a28a8f0 0x55f50a2aadc0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f46a00a8e00 tx=0x7f46a00098c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=673afc8986490e61 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:33:26.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.326+0000 7f46a6000700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-03T18:33:26.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.326+0000 7f46a6000700 1 -- 172.21.15.33:0/2368780975 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a00c7020 con 0x55f50a28a8f0 2024-05-03T18:33:26.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.326+0000 7f46c9b57b80 1 -- 172.21.15.33:0/2368780975 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46a00ad4b0 con 0x55f50a28a8f0 2024-05-03T18:33:26.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.326+0000 7f46a6000700 1 -- 172.21.15.33:0/2368780975 <== mon.1 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f46a00a56f0 con 0x55f50a28a8f0 2024-05-03T18:33:26.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.327+0000 7f46a6000700 1 -- 172.21.15.33:0/2368780975 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f46a00b1f60 con 0x55f50a28a8f0 2024-05-03T18:33:26.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.327+0000 7f46c9b57b80 10 client.25729 fetch_fsmap learned FSMap version 21 2024-05-03T18:33:26.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.328+0000 7f46c9b57b80 1 -- 172.21.15.33:0/2368780975 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55f50a2814a0 con 0x55f50a28a8f0 2024-05-03T18:33:26.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.328+0000 7f46a6000700 1 -- 172.21.15.33:0/2368780975 <== mon.1 v2:172.21.15.150:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f46a00ad4b0 con 0x55f50a28a8f0 2024-05-03T18:33:26.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.329+0000 7f46a6000700 1 -- 172.21.15.33:0/2368780975 <== mon.1 v2:172.21.15.150:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f46a00ae640 con 0x55f50a28a8f0 2024-05-03T18:33:26.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.329+0000 7f46c9b57b80 10 client.25729 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:33:26.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.329+0000 7f46c9b57b80 10 client.25729 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:33:26.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.329+0000 7f46c9b57b80 20 client.25729 populate_metadata read hostname 'smithi033' 2024-05-03T18:33:26.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.329+0000 7f46c9b57b80 1 --2- 172.21.15.33:0/2368780975 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55f50a30e400 0x55f50a310820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:26.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.329+0000 7f46c9b57b80 4 client.25729 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-03T18:33:26.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.329+0000 7f46c9b57b80 1 -- 172.21.15.33:0/2368780975 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f50a1ea400 con 0x55f50a30e400 2024-05-03T18:33:26.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.330+0000 7f46b7e00700 1 --2- 172.21.15.33:0/2368780975 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55f50a30e400 0x55f50a310820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:26.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.331+0000 7f46b7e00700 1 --2- 172.21.15.33:0/2368780975 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55f50a30e400 0x55f50a310820 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e571c85f0c7045f3 server_cookie=d4f0a5ac325a9447 in_seq=0 out_seq=0 2024-05-03T18:33:26.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.331+0000 7f46a6000700 10 client.25729 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:33:26.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.331+0000 7f46a6000700 1 -- 172.21.15.33:0/2368780975 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f50a1ea400 con 0x55f50a30e400 2024-05-03T18:33:26.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.332+0000 7f46a6000700 10 client.25729 handle_command_reply: tid=0 2024-05-03T18:33:26.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.332+0000 7f46c9b57b80 1 client.25729 shutdown 2024-05-03T18:33:26.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.332+0000 7f46c9b57b80 1 -- 172.21.15.33:0/2368780975 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f50a28a8f0 msgr2=0x55f50a2aadc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:33:26.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.332+0000 7f46c9b57b80 1 --2- 172.21.15.33:0/2368780975 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f50a28a8f0 0x55f50a2aadc0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f46a00a8e00 tx=0x7f46a00098c0 comp rx=0 tx=0).stop 2024-05-03T18:33:26.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.332+0000 7f46c9b57b80 1 -- 172.21.15.33:0/2368780975 shutdown_connections 2024-05-03T18:33:26.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.332+0000 7f46c9b57b80 1 --2- 172.21.15.33:0/2368780975 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55f50a30e400 0x55f50a310820 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:26.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.332+0000 7f46c9b57b80 1 --2- 172.21.15.33:0/2368780975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f50a28fbd0 0x55f50a2ab300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:26.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.332+0000 7f46c9b57b80 1 --2- 172.21.15.33:0/2368780975 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f50a289f30 0x55f50a29c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:26.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.332+0000 7f46c9b57b80 1 --2- 172.21.15.33:0/2368780975 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f50a28a8f0 0x55f50a2aadc0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:26.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.332+0000 7f46c9b57b80 1 -- 172.21.15.33:0/2368780975 >> 172.21.15.33:0/2368780975 conn(0x55f50a2026d0 msgr2=0x55f50a2846b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:33:26.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.332+0000 7f46a6000700 0 client.25729 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:33:26.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.333+0000 7f46c9b57b80 1 -- 172.21.15.33:0/2368780975 shutdown_connections 2024-05-03T18:33:26.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.333+0000 7f46c9b57b80 1 -- 172.21.15.33:0/2368780975 wait complete. 2024-05-03T18:33:26.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.333+0000 7f46c9b57b80 20 client.25729 trim_cache size 0 max 16384 2024-05-03T18:33:26.337 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-03T18:33:26.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.334+0000 7f46bdc00700 1 -- 172.21.15.33:0/3107866039 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f469c0b2a70 msgr2=0x7f469c0b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:33:26.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.334+0000 7f46bdc00700 1 --2- 172.21.15.33:0/3107866039 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f469c0b2a70 0x7f469c0b4f30 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f46b806bdb0 tx=0x7f46a8007320 comp rx=0 tx=0).stop 2024-05-03T18:33:26.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.334+0000 7f46bdc00700 1 -- 172.21.15.33:0/3107866039 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46b8076160 msgr2=0x7f46b81a0530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:33:26.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.334+0000 7f46bdc00700 1 --2- 172.21.15.33:0/3107866039 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46b8076160 0x7f46b81a0530 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f46a00060b0 tx=0x7f46a0007f90 comp rx=0 tx=0).stop 2024-05-03T18:33:26.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.335+0000 7f46bdc00700 1 -- 172.21.15.33:0/3107866039 shutdown_connections 2024-05-03T18:33:26.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.335+0000 7f46bdc00700 1 --2- 172.21.15.33:0/3107866039 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f46b81143e0 0x7f46b81a48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:26.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.335+0000 7f46bdc00700 1 --2- 172.21.15.33:0/3107866039 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f469c0b2a70 0x7f469c0b4f30 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:26.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.335+0000 7f46bdc00700 1 --2- 172.21.15.33:0/3107866039 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46b8075840 0x7f46b819fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:26.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.335+0000 7f46bdc00700 1 --2- 172.21.15.33:0/3107866039 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46b8076160 0x7f46b81a0530 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:26.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.335+0000 7f46bdc00700 1 -- 172.21.15.33:0/3107866039 >> 172.21.15.33:0/3107866039 conn(0x7f46b80fef20 msgr2=0x7f46b8100780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:33:26.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.335+0000 7f46bdc00700 1 -- 172.21.15.33:0/3107866039 shutdown_connections 2024-05-03T18:33:26.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:26.336+0000 7f46bdc00700 1 -- 172.21.15.33:0/3107866039 wait complete. 2024-05-03T18:33:26.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:33:26 smithi150 ceph-mon[32492]: pgmap v190: 129 pgs: 129 active+clean; 1.9 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.2 MiB/s wr, 568 op/s 2024-05-03T18:33:28.965 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-03T18:33:28.966 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-03T18:33:29.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:33:29 smithi099 ceph-mon[31658]: pgmap v191: 129 pgs: 129 active+clean; 1.9 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 11 MiB/s wr, 859 op/s 2024-05-03T18:33:29.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:33:29 smithi150 ceph-mon[32492]: pgmap v191: 129 pgs: 129 active+clean; 1.9 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 11 MiB/s wr, 859 op/s 2024-05-03T18:33:29.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:33:29 smithi033 ceph-mon[28330]: pgmap v191: 129 pgs: 129 active+clean; 1.9 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 11 MiB/s wr, 859 op/s 2024-05-03T18:33:29.585 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:33:30.111 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 135.0 s, 604.3 tps, lat 206.904 ms stddev 341.365 2024-05-03T18:33:30.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:33:30 smithi099 ceph-mon[31658]: pgmap v192: 129 pgs: 129 active+clean; 1.9 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 8.9 MiB/s wr, 630 op/s 2024-05-03T18:33:30.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:33:30 smithi150 ceph-mon[32492]: pgmap v192: 129 pgs: 129 active+clean; 1.9 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 8.9 MiB/s wr, 630 op/s 2024-05-03T18:33:30.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:33:30 smithi033 ceph-mon[28330]: pgmap v192: 129 pgs: 129 active+clean; 1.9 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 8.9 MiB/s wr, 630 op/s 2024-05-03T18:33:32.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.770+0000 7f7f78200700 1 -- 172.21.15.33:0/1647120917 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f7010ff00 msgr2=0x7f7f701122f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:33:32.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.770+0000 7f7f78200700 1 --2- 172.21.15.33:0/1647120917 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f7010ff00 0x7f7f701122f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7f68009f90 tx=0x7f7f6800b330 comp rx=0 tx=0).stop 2024-05-03T18:33:32.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.771+0000 7f7f78200700 1 -- 172.21.15.33:0/1647120917 shutdown_connections 2024-05-03T18:33:32.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.771+0000 7f7f78200700 1 --2- 172.21.15.33:0/1647120917 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f7010ff00 0x7f7f701122f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:32.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.771+0000 7f7f78200700 1 --2- 172.21.15.33:0/1647120917 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f700756d0 0x7f7f70075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:32.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.771+0000 7f7f78200700 1 --2- 172.21.15.33:0/1647120917 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f70075ff0 0x7f7f7010f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:32.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.771+0000 7f7f78200700 1 -- 172.21.15.33:0/1647120917 >> 172.21.15.33:0/1647120917 conn(0x7f7f7007a600 msgr2=0x7f7f7007aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:33:32.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.772+0000 7f7f78200700 1 -- 172.21.15.33:0/1647120917 shutdown_connections 2024-05-03T18:33:32.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.772+0000 7f7f78200700 1 -- 172.21.15.33:0/1647120917 wait complete. 2024-05-03T18:33:32.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.774+0000 7f7f78200700 1 Processor -- start 2024-05-03T18:33:32.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.775+0000 7f7f78200700 1 -- start start 2024-05-03T18:33:32.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.775+0000 7f7f78200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f700756d0 0x7f7f701a1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:32.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.775+0000 7f7f78200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f70075ff0 0x7f7f701a1d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:32.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.775+0000 7f7f78200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f7010ff00 0x7f7f7019b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:32.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.775+0000 7f7f78200700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7f701157b0 con 0x7f7f7010ff00 2024-05-03T18:33:32.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.775+0000 7f7f78200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7f70115630 con 0x7f7f700756d0 2024-05-03T18:33:32.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.775+0000 7f7f78200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f7f70115930 con 0x7f7f70075ff0 2024-05-03T18:33:32.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.776+0000 7f7f75800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f700756d0 0x7f7f701a1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:32.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.776+0000 7f7f76200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f7010ff00 0x7f7f7019b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:32.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.776+0000 7f7f75800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f700756d0 0x7f7f701a1830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:50330/0 (socket says 172.21.15.33:50330) 2024-05-03T18:33:32.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.776+0000 7f7f75800700 1 -- 172.21.15.33:0/4020744458 learned_addr learned my addr 172.21.15.33:0/4020744458 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:33:32.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.777+0000 7f7f74e00700 1 --2- 172.21.15.33:0/4020744458 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f70075ff0 0x7f7f701a1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:32.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.778+0000 7f7f76200700 1 -- 172.21.15.33:0/4020744458 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f70075ff0 msgr2=0x7f7f701a1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:33:32.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.778+0000 7f7f76200700 1 --2- 172.21.15.33:0/4020744458 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f70075ff0 0x7f7f701a1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:32.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.778+0000 7f7f76200700 1 -- 172.21.15.33:0/4020744458 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f700756d0 msgr2=0x7f7f701a1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:33:32.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.778+0000 7f7f76200700 1 --2- 172.21.15.33:0/4020744458 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f700756d0 0x7f7f701a1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:32.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.778+0000 7f7f76200700 1 -- 172.21.15.33:0/4020744458 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f7019c150 con 0x7f7f7010ff00 2024-05-03T18:33:32.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.778+0000 7f7f75800700 1 --2- 172.21.15.33:0/4020744458 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f700756d0 0x7f7f701a1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:33:32.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.779+0000 7f7f76200700 1 --2- 172.21.15.33:0/4020744458 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f7010ff00 0x7f7f7019b9f0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f7f7006bcd0 tx=0x7f7f6800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:33:32.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.779+0000 7f7f6e000700 1 -- 172.21.15.33:0/4020744458 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f68021070 con 0x7f7f7010ff00 2024-05-03T18:33:32.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.779+0000 7f7f78200700 1 -- 172.21.15.33:0/4020744458 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f7019c3e0 con 0x7f7f7010ff00 2024-05-03T18:33:32.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.779+0000 7f7f78200700 1 -- 172.21.15.33:0/4020744458 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f701a8700 con 0x7f7f7010ff00 2024-05-03T18:33:32.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.779+0000 7f7f6e000700 1 -- 172.21.15.33:0/4020744458 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7f68009be0 con 0x7f7f7010ff00 2024-05-03T18:33:32.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.780+0000 7f7f6e000700 1 -- 172.21.15.33:0/4020744458 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f6801d8d0 con 0x7f7f7010ff00 2024-05-03T18:33:32.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.786+0000 7f7f6e000700 1 -- 172.21.15.33:0/4020744458 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f7f68024070 con 0x7f7f7010ff00 2024-05-03T18:33:32.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.791+0000 7f7f6e000700 1 --2- 172.21.15.33:0/4020744458 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f7f580b2c70 0x7f7f580b5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:32.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.791+0000 7f7f6e000700 1 -- 172.21.15.33:0/4020744458 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f7f6800f020 con 0x7f7f7010ff00 2024-05-03T18:33:32.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.791+0000 7f7f75800700 1 --2- 172.21.15.33:0/4020744458 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f7f580b2c70 0x7f7f580b5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:32.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.792+0000 7f7f82b4eb80 1 Processor -- start 2024-05-03T18:33:32.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.793+0000 7f7f82b4eb80 1 -- start start 2024-05-03T18:33:32.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.794+0000 7f7f82b4eb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b3d948b9b0 0x55b3d948bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:32.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.794+0000 7f7f82b4eb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b3d948c2d0 0x55b3d9485fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:32.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.794+0000 7f7f82b4eb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3d94865d0 0x55b3d9486a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:32.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.794+0000 7f7f82b4eb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55b3d93d8aa0 con 0x55b3d94865d0 2024-05-03T18:33:32.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.794+0000 7f7f82b4eb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55b3d93767e0 con 0x55b3d948c2d0 2024-05-03T18:33:32.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.794+0000 7f7f82b4eb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55b3d93d4a60 con 0x55b3d948b9b0 2024-05-03T18:33:32.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.794+0000 7f7f75800700 1 --2- 172.21.15.33:0/4020744458 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f7f580b2c70 0x7f7f580b5130 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7f7019cb30 tx=0x7f7f64009310 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:33:32.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.794+0000 7f7f74e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b3d948c2d0 0x55b3d9485fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:32.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.794+0000 7f7f74e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b3d948c2d0 0x55b3d9485fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:50340/0 (socket says 172.21.15.33:50340) 2024-05-03T18:33:32.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.794+0000 7f7f74e00700 1 -- 172.21.15.33:0/142704852 learned_addr learned my addr 172.21.15.33:0/142704852 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:33:32.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.794+0000 7f7f75800700 1 --2- 172.21.15.33:0/142704852 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b3d948b9b0 0x55b3d948bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:32.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.795+0000 7f7f74e00700 1 -- 172.21.15.33:0/142704852 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b3d948b9b0 msgr2=0x55b3d948bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:33:32.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.795+0000 7f7f74e00700 1 --2- 172.21.15.33:0/142704852 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b3d948b9b0 0x55b3d948bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:32.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.795+0000 7f7f74e00700 1 -- 172.21.15.33:0/142704852 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3d94865d0 msgr2=0x55b3d9486a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:33:32.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.795+0000 7f7f76200700 1 --2- 172.21.15.33:0/142704852 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3d94865d0 0x55b3d9486a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:32.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.795+0000 7f7f74e00700 1 --2- 172.21.15.33:0/142704852 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3d94865d0 0x55b3d9486a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:32.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.795+0000 7f7f74e00700 1 -- 172.21.15.33:0/142704852 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3d9325950 con 0x55b3d948c2d0 2024-05-03T18:33:32.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.795+0000 7f7f75800700 1 --2- 172.21.15.33:0/142704852 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b3d948b9b0 0x55b3d948bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:33:32.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.796+0000 7f7f76200700 1 --2- 172.21.15.33:0/142704852 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3d94865d0 0x55b3d9486a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:33:32.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.796+0000 7f7f74e00700 1 --2- 172.21.15.33:0/142704852 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b3d948c2d0 0x55b3d9485fb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c00acd0 tx=0x7f7f5c00f3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd92991670ba275c server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:33:32.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.797+0000 7f7f62a00700 1 -- 172.21.15.33:0/142704852 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c00fe50 con 0x55b3d948c2d0 2024-05-03T18:33:32.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.797+0000 7f7f62a00700 1 -- 172.21.15.33:0/142704852 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7f5c004cc0 con 0x55b3d948c2d0 2024-05-03T18:33:32.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.797+0000 7f7f62a00700 1 -- 172.21.15.33:0/142704852 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c0186a0 con 0x55b3d948c2d0 2024-05-03T18:33:32.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.798+0000 7f7f82b4eb80 1 -- 172.21.15.33:0/142704852 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b3d948c2d0 msgr2=0x55b3d9485fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:33:32.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.798+0000 7f7f82b4eb80 1 --2- 172.21.15.33:0/142704852 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b3d948c2d0 0x55b3d9485fb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c00acd0 tx=0x7f7f5c00f3c0 comp rx=0 tx=0).stop 2024-05-03T18:33:32.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.800+0000 7f7f82b4eb80 1 -- 172.21.15.33:0/142704852 shutdown_connections 2024-05-03T18:33:32.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.800+0000 7f7f82b4eb80 1 --2- 172.21.15.33:0/142704852 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3d94865d0 0x55b3d9486a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:32.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.800+0000 7f7f82b4eb80 1 --2- 172.21.15.33:0/142704852 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b3d948b9b0 0x55b3d948bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:32.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.800+0000 7f7f82b4eb80 1 --2- 172.21.15.33:0/142704852 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b3d948c2d0 0x55b3d9485fb0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:32.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.800+0000 7f7f82b4eb80 1 -- 172.21.15.33:0/142704852 >> 172.21.15.33:0/142704852 conn(0x55b3d93fe750 msgr2=0x55b3d9480980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:33:32.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.801+0000 7f7f82b4eb80 1 -- 172.21.15.33:0/142704852 shutdown_connections 2024-05-03T18:33:32.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.801+0000 7f7f82b4eb80 1 -- 172.21.15.33:0/142704852 wait complete. 2024-05-03T18:33:32.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.801+0000 7f7f82b4eb80 1 -- start start 2024-05-03T18:33:32.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.802+0000 7f7f82b4eb80 1 Processor -- start 2024-05-03T18:33:32.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.802+0000 7f7f82b4eb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b3d94865d0 0x55b3d9491e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:32.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.802+0000 7f7f82b4eb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3d948b9b0 0x55b3d948f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:32.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.802+0000 7f7f82b4eb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b3d948c2d0 0x55b3d948f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:32.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.802+0000 7f7f82b4eb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55b3d9492d80 con 0x55b3d948b9b0 2024-05-03T18:33:32.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.802+0000 7f7f82b4eb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55b3d933eba0 con 0x55b3d948c2d0 2024-05-03T18:33:32.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.802+0000 7f7f82b4eb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55b3d9492f00 con 0x55b3d94865d0 2024-05-03T18:33:32.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.802+0000 7f7f82b4eb80 10 client.0 _ll_register_callbacks cb 0x55b3d937fcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:33:32.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.803+0000 7f7f74e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3d948b9b0 0x55b3d948f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:32.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.803+0000 7f7f76200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b3d948c2d0 0x55b3d948f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:32.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.803+0000 7f7f74e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3d948b9b0 0x55b3d948f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43116/0 (socket says 172.21.15.33:43116) 2024-05-03T18:33:32.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.803+0000 7f7f74e00700 1 -- 172.21.15.33:0/3975819761 learned_addr learned my addr 172.21.15.33:0/3975819761 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:33:32.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.804+0000 7f7f75800700 1 --2- 172.21.15.33:0/3975819761 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b3d94865d0 0x55b3d9491e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:32.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.804+0000 7f7f74e00700 1 -- 172.21.15.33:0/3975819761 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b3d94865d0 msgr2=0x55b3d9491e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:33:32.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.804+0000 7f7f74e00700 1 --2- 172.21.15.33:0/3975819761 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b3d94865d0 0x55b3d9491e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:32.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.804+0000 7f7f74e00700 1 -- 172.21.15.33:0/3975819761 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b3d948c2d0 msgr2=0x55b3d948f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:33:32.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.804+0000 7f7f74e00700 1 --2- 172.21.15.33:0/3975819761 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b3d948c2d0 0x55b3d948f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:32.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.804+0000 7f7f74e00700 1 -- 172.21.15.33:0/3975819761 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3d939e4a0 con 0x55b3d948b9b0 2024-05-03T18:33:32.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.805+0000 7f7f74e00700 1 --2- 172.21.15.33:0/3975819761 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3d948b9b0 0x55b3d948f220 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c00ff10 tx=0x7f7f5c00ff40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcfdb7f3b67c7bb4 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:33:32.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.806+0000 7f7f57400700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:33:32.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.806+0000 7f7f57400700 1 -- 172.21.15.33:0/3975819761 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c026020 con 0x55b3d948b9b0 2024-05-03T18:33:32.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.806+0000 7f7f57400700 1 -- 172.21.15.33:0/3975819761 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f7f5c004950 con 0x55b3d948b9b0 2024-05-03T18:33:32.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.806+0000 7f7f82b4eb80 1 -- 172.21.15.33:0/3975819761 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f5c004cc0 con 0x55b3d948b9b0 2024-05-03T18:33:32.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.806+0000 7f7f57400700 1 -- 172.21.15.33:0/3975819761 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f7f5c004040 con 0x55b3d948b9b0 2024-05-03T18:33:32.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.806+0000 7f7f82b4eb80 10 client.16053 fetch_fsmap learned FSMap version 21 2024-05-03T18:33:32.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.806+0000 7f7f82b4eb80 1 -- 172.21.15.33:0/3975819761 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55b3d949d460 con 0x55b3d948b9b0 2024-05-03T18:33:32.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.807+0000 7f7f57400700 1 -- 172.21.15.33:0/3975819761 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7f5c004cc0 con 0x55b3d948b9b0 2024-05-03T18:33:32.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.807+0000 7f7f57400700 1 -- 172.21.15.33:0/3975819761 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f7f5c022460 con 0x55b3d948b9b0 2024-05-03T18:33:32.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.807+0000 7f7f82b4eb80 10 client.16053 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:33:32.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.807+0000 7f7f82b4eb80 10 client.16053 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:33:32.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.808+0000 7f7f82b4eb80 20 client.16053 populate_metadata read hostname 'smithi033' 2024-05-03T18:33:32.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.808+0000 7f7f82b4eb80 1 --2- 172.21.15.33:0/3975819761 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55b3d949f930 0x55b3d94a1d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:32.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.808+0000 7f7f82b4eb80 4 client.16053 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:33:32.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.808+0000 7f7f82b4eb80 1 -- 172.21.15.33:0/3975819761 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b3d9453070 con 0x55b3d949f930 2024-05-03T18:33:32.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.809+0000 7f7f75800700 1 --2- 172.21.15.33:0/3975819761 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55b3d949f930 0x55b3d94a1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:32.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.811+0000 7f7f75800700 1 --2- 172.21.15.33:0/3975819761 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55b3d949f930 0x55b3d94a1d20 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8d8dfd869404f79 server_cookie=d218414ea471742f in_seq=0 out_seq=0 2024-05-03T18:33:32.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.811+0000 7f7f57400700 10 client.16053 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:33:32.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.815+0000 7f7f57400700 1 -- 172.21.15.33:0/3975819761 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b3d9453070 con 0x55b3d949f930 2024-05-03T18:33:32.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.815+0000 7f7f57400700 10 client.16053 handle_command_reply: tid=0 2024-05-03T18:33:32.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.815+0000 7f7f82b4eb80 1 client.16053 shutdown 2024-05-03T18:33:32.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.815+0000 7f7f82b4eb80 1 -- 172.21.15.33:0/3975819761 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3d948b9b0 msgr2=0x55b3d948f220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:33:32.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.815+0000 7f7f82b4eb80 1 --2- 172.21.15.33:0/3975819761 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3d948b9b0 0x55b3d948f220 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c00ff10 tx=0x7f7f5c00ff40 comp rx=0 tx=0).stop 2024-05-03T18:33:32.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.816+0000 7f7f82b4eb80 1 -- 172.21.15.33:0/3975819761 shutdown_connections 2024-05-03T18:33:32.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.816+0000 7f7f82b4eb80 1 --2- 172.21.15.33:0/3975819761 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55b3d949f930 0x55b3d94a1d20 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:32.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.816+0000 7f7f82b4eb80 1 --2- 172.21.15.33:0/3975819761 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b3d948c2d0 0x55b3d948f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:32.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.816+0000 7f7f82b4eb80 1 --2- 172.21.15.33:0/3975819761 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b3d94865d0 0x55b3d9491e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:32.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.816+0000 7f7f82b4eb80 1 --2- 172.21.15.33:0/3975819761 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3d948b9b0 0x55b3d948f220 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:32.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.816+0000 7f7f82b4eb80 1 -- 172.21.15.33:0/3975819761 >> 172.21.15.33:0/3975819761 conn(0x55b3d93fe750 msgr2=0x55b3d947b430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:33:32.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.816+0000 7f7f57400700 0 client.16053 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:33:32.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.816+0000 7f7f82b4eb80 1 -- 172.21.15.33:0/3975819761 shutdown_connections 2024-05-03T18:33:32.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.817+0000 7f7f82b4eb80 1 -- 172.21.15.33:0/3975819761 wait complete. 2024-05-03T18:33:32.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.817+0000 7f7f82b4eb80 20 client.16053 trim_cache size 0 max 16384 2024-05-03T18:33:32.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.854+0000 7f7f82b4eb80 1 Processor -- start 2024-05-03T18:33:32.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.854+0000 7f7f82b4eb80 1 -- start start 2024-05-03T18:33:32.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.854+0000 7f7f82b4eb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b3d94865d0 0x55b3d9485fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:32.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.855+0000 7f7f82b4eb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3d948b9b0 0x55b3d948aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:32.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.855+0000 7f7f82b4eb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b3d948c2d0 0x55b3d948afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:32.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.855+0000 7f7f82b4eb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7f5000ce60 con 0x55b3d948b9b0 2024-05-03T18:33:32.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.855+0000 7f7f82b4eb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7f5000f6e0 con 0x55b3d94865d0 2024-05-03T18:33:32.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.855+0000 7f7f82b4eb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f7f5000d4d0 con 0x55b3d948c2d0 2024-05-03T18:33:32.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.856+0000 7f7f75800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b3d94865d0 0x55b3d9485fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:32.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.856+0000 7f7f75800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b3d94865d0 0x55b3d9485fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:50354/0 (socket says 172.21.15.33:50354) 2024-05-03T18:33:32.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.856+0000 7f7f75800700 1 -- 172.21.15.33:0/3076315724 learned_addr learned my addr 172.21.15.33:0/3076315724 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:33:32.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.856+0000 7f7f75800700 1 -- 172.21.15.33:0/3076315724 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b3d948c2d0 msgr2=0x55b3d948afb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-03T18:33:32.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.856+0000 7f7f75800700 1 --2- 172.21.15.33:0/3076315724 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b3d948c2d0 0x55b3d948afb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:32.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.856+0000 7f7f75800700 1 -- 172.21.15.33:0/3076315724 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3d948b9b0 msgr2=0x55b3d948aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:33:32.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.857+0000 7f7f74e00700 1 --2- 172.21.15.33:0/3076315724 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3d948b9b0 0x55b3d948aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:32.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.857+0000 7f7f75800700 1 --2- 172.21.15.33:0/3076315724 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3d948b9b0 0x55b3d948aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:32.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.857+0000 7f7f75800700 1 -- 172.21.15.33:0/3076315724 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3d9480ec0 con 0x55b3d94865d0 2024-05-03T18:33:32.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.858+0000 7f7f75800700 1 --2- 172.21.15.33:0/3076315724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b3d94865d0 0x55b3d9485fb0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7f64009790 tx=0x7f7f64014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8591ea17c6fa05d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:33:32.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.860+0000 7f7f62000700 1 -- 172.21.15.33:0/3076315724 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f6402b030 con 0x55b3d94865d0 2024-05-03T18:33:32.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.860+0000 7f7f62000700 1 -- 172.21.15.33:0/3076315724 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7f64005020 con 0x55b3d94865d0 2024-05-03T18:33:32.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.860+0000 7f7f62000700 1 -- 172.21.15.33:0/3076315724 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f6402b430 con 0x55b3d94865d0 2024-05-03T18:33:32.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.861+0000 7f7f82b4eb80 1 -- 172.21.15.33:0/3076315724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b3d94865d0 msgr2=0x55b3d9485fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:33:32.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.861+0000 7f7f82b4eb80 1 --2- 172.21.15.33:0/3076315724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b3d94865d0 0x55b3d9485fb0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7f64009790 tx=0x7f7f64014000 comp rx=0 tx=0).stop 2024-05-03T18:33:32.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.861+0000 7f7f82b4eb80 1 -- 172.21.15.33:0/3076315724 shutdown_connections 2024-05-03T18:33:32.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.861+0000 7f7f82b4eb80 1 --2- 172.21.15.33:0/3076315724 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b3d948c2d0 0x55b3d948afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:32.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.861+0000 7f7f82b4eb80 1 --2- 172.21.15.33:0/3076315724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b3d94865d0 0x55b3d9485fb0 secure :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7f64009790 tx=0x7f7f64014000 comp rx=0 tx=0).stop 2024-05-03T18:33:32.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.861+0000 7f7f82b4eb80 1 --2- 172.21.15.33:0/3076315724 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3d948b9b0 0x55b3d948aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:32.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.861+0000 7f7f82b4eb80 1 -- 172.21.15.33:0/3076315724 >> 172.21.15.33:0/3076315724 conn(0x55b3d93fe750 msgr2=0x55b3d948a360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:33:32.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.862+0000 7f7f82b4eb80 1 -- 172.21.15.33:0/3076315724 shutdown_connections 2024-05-03T18:33:32.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.864+0000 7f7f82b4eb80 1 -- 172.21.15.33:0/3076315724 wait complete. 2024-05-03T18:33:32.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.864+0000 7f7f82b4eb80 1 -- start start 2024-05-03T18:33:32.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.864+0000 7f7f82b4eb80 1 Processor -- start 2024-05-03T18:33:32.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.865+0000 7f7f82b4eb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3d948b9b0 0x55b3d9495790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:32.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.865+0000 7f7f82b4eb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b3d948c2d0 0x55b3d9495cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:32.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.865+0000 7f7f82b4eb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b3d9497df0 0x55b3d9498270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:32.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.865+0000 7f7f82b4eb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7f5000dba0 con 0x55b3d948b9b0 2024-05-03T18:33:32.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.865+0000 7f7f82b4eb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7f5000f010 con 0x55b3d9497df0 2024-05-03T18:33:32.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.865+0000 7f7f82b4eb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f7f5000e270 con 0x55b3d948c2d0 2024-05-03T18:33:32.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.865+0000 7f7f82b4eb80 10 client.0 _ll_register_callbacks cb 0x55b3d937fcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:33:32.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.866+0000 7f7f74e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b3d948c2d0 0x55b3d9495cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:32.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.866+0000 7f7f75800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3d948b9b0 0x55b3d9495790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:32.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.866+0000 7f7f74e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b3d948c2d0 0x55b3d9495cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:53094/0 (socket says 172.21.15.33:53094) 2024-05-03T18:33:32.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.866+0000 7f7f74e00700 1 -- 172.21.15.33:0/490039455 learned_addr learned my addr 172.21.15.33:0/490039455 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:33:32.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.866+0000 7f7f76200700 1 --2- 172.21.15.33:0/490039455 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b3d9497df0 0x55b3d9498270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:32.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.867+0000 7f7f75800700 1 -- 172.21.15.33:0/490039455 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b3d948c2d0 msgr2=0x55b3d9495cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:33:32.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.867+0000 7f7f75800700 1 --2- 172.21.15.33:0/490039455 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b3d948c2d0 0x55b3d9495cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:32.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.867+0000 7f7f75800700 1 -- 172.21.15.33:0/490039455 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b3d9497df0 msgr2=0x55b3d9498270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:33:32.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.867+0000 7f7f75800700 1 --2- 172.21.15.33:0/490039455 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b3d9497df0 0x55b3d9498270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:32.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.867+0000 7f7f75800700 1 -- 172.21.15.33:0/490039455 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3d9498860 con 0x55b3d948b9b0 2024-05-03T18:33:32.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.867+0000 7f7f75800700 1 --2- 172.21.15.33:0/490039455 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3d948b9b0 0x55b3d9495790 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x55b3d94930c0 tx=0x7f7f640053e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a93af56d16a47349 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:33:32.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.867+0000 7f7f57400700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:33:32.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.870+0000 7f7f57400700 1 -- 172.21.15.33:0/490039455 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f64039020 con 0x55b3d948b9b0 2024-05-03T18:33:32.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.870+0000 7f7f82b4eb80 1 -- 172.21.15.33:0/490039455 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f64005020 con 0x55b3d948b9b0 2024-05-03T18:33:32.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.870+0000 7f7f57400700 1 -- 172.21.15.33:0/490039455 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f7f64004750 con 0x55b3d948b9b0 2024-05-03T18:33:32.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.871+0000 7f7f57400700 1 -- 172.21.15.33:0/490039455 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f7f6402b040 con 0x55b3d948b9b0 2024-05-03T18:33:32.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.871+0000 7f7f82b4eb80 10 client.16056 fetch_fsmap learned FSMap version 21 2024-05-03T18:33:32.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.871+0000 7f7f82b4eb80 1 -- 172.21.15.33:0/490039455 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55b3d9514620 con 0x55b3d948b9b0 2024-05-03T18:33:32.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.872+0000 7f7f57400700 1 -- 172.21.15.33:0/490039455 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7f64005020 con 0x55b3d948b9b0 2024-05-03T18:33:32.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.872+0000 7f7f57400700 1 -- 172.21.15.33:0/490039455 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f7f64026040 con 0x55b3d948b9b0 2024-05-03T18:33:32.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.873+0000 7f7f82b4eb80 10 client.16056 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:33:32.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.873+0000 7f7f82b4eb80 10 client.16056 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:33:32.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.873+0000 7f7f82b4eb80 20 client.16056 populate_metadata read hostname 'smithi033' 2024-05-03T18:33:32.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.873+0000 7f7f82b4eb80 1 --2- 172.21.15.33:0/490039455 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55b3d9514c50 0x55b3d9517070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:32.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.873+0000 7f7f82b4eb80 4 client.16056 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-03T18:33:32.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.873+0000 7f7f82b4eb80 1 -- 172.21.15.33:0/490039455 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55b3d93e6480 con 0x55b3d9514c50 2024-05-03T18:33:32.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.874+0000 7f7f74e00700 1 --2- 172.21.15.33:0/490039455 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55b3d9514c50 0x55b3d9517070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:32.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.875+0000 7f7f74e00700 1 --2- 172.21.15.33:0/490039455 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55b3d9514c50 0x55b3d9517070 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fcf730ce1d719ccf server_cookie=65edda162d5bfd28 in_seq=0 out_seq=0 2024-05-03T18:33:32.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.875+0000 7f7f57400700 10 client.16056 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:33:32.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.878+0000 7f7f57400700 1 -- 172.21.15.33:0/490039455 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b3d93e6480 con 0x55b3d9514c50 2024-05-03T18:33:32.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.878+0000 7f7f57400700 10 client.16056 handle_command_reply: tid=0 2024-05-03T18:33:32.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.878+0000 7f7f82b4eb80 1 client.16056 shutdown 2024-05-03T18:33:32.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.878+0000 7f7f82b4eb80 1 -- 172.21.15.33:0/490039455 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3d948b9b0 msgr2=0x55b3d9495790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:33:32.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.878+0000 7f7f82b4eb80 1 --2- 172.21.15.33:0/490039455 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3d948b9b0 0x55b3d9495790 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x55b3d94930c0 tx=0x7f7f640053e0 comp rx=0 tx=0).stop 2024-05-03T18:33:32.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.879+0000 7f7f82b4eb80 1 -- 172.21.15.33:0/490039455 shutdown_connections 2024-05-03T18:33:32.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.879+0000 7f7f82b4eb80 1 --2- 172.21.15.33:0/490039455 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55b3d9514c50 0x55b3d9517070 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:32.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.879+0000 7f7f82b4eb80 1 --2- 172.21.15.33:0/490039455 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b3d9497df0 0x55b3d9498270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:32.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.879+0000 7f7f82b4eb80 1 --2- 172.21.15.33:0/490039455 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3d948b9b0 0x55b3d9495790 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:32.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.879+0000 7f7f82b4eb80 1 --2- 172.21.15.33:0/490039455 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b3d948c2d0 0x55b3d9495cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:32.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.879+0000 7f7f82b4eb80 1 -- 172.21.15.33:0/490039455 >> 172.21.15.33:0/490039455 conn(0x55b3d93fe750 msgr2=0x55b3d948abf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:33:32.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.879+0000 7f7f57400700 0 client.16056 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:33:32.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.879+0000 7f7f82b4eb80 1 -- 172.21.15.33:0/490039455 shutdown_connections 2024-05-03T18:33:32.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.880+0000 7f7f82b4eb80 1 -- 172.21.15.33:0/490039455 wait complete. 2024-05-03T18:33:32.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.880+0000 7f7f82b4eb80 20 client.16056 trim_cache size 0 max 16384 2024-05-03T18:33:32.883 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-03T18:33:32.884 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-05-03T18:33:32.884 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "089e9e1a-c06d-4322-983f-43dbda4a66c2", 2024-05-03T18:33:32.884 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-05-03T18:33:32.884 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-03T18:33:32.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.881+0000 7f7f78200700 1 -- 172.21.15.33:0/4020744458 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f7f580b2c70 msgr2=0x7f7f580b5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:33:32.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.881+0000 7f7f78200700 1 --2- 172.21.15.33:0/4020744458 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f7f580b2c70 0x7f7f580b5130 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7f7019cb30 tx=0x7f7f64009310 comp rx=0 tx=0).stop 2024-05-03T18:33:32.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.881+0000 7f7f78200700 1 -- 172.21.15.33:0/4020744458 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f7010ff00 msgr2=0x7f7f7019b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:33:32.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.881+0000 7f7f78200700 1 --2- 172.21.15.33:0/4020744458 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f7010ff00 0x7f7f7019b9f0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f7f7006bcd0 tx=0x7f7f6800cc30 comp rx=0 tx=0).stop 2024-05-03T18:33:32.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.881+0000 7f7f78200700 1 -- 172.21.15.33:0/4020744458 shutdown_connections 2024-05-03T18:33:32.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.881+0000 7f7f78200700 1 --2- 172.21.15.33:0/4020744458 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f7010ff00 0x7f7f7019b9f0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:32.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.881+0000 7f7f78200700 1 --2- 172.21.15.33:0/4020744458 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f7f580b2c70 0x7f7f580b5130 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:32.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.881+0000 7f7f78200700 1 --2- 172.21.15.33:0/4020744458 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f700756d0 0x7f7f701a1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:32.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.881+0000 7f7f78200700 1 --2- 172.21.15.33:0/4020744458 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f70075ff0 0x7f7f701a1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:32.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.881+0000 7f7f78200700 1 -- 172.21.15.33:0/4020744458 >> 172.21.15.33:0/4020744458 conn(0x7f7f7007a600 msgr2=0x7f7f70100690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:33:32.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.882+0000 7f7f78200700 1 -- 172.21.15.33:0/4020744458 shutdown_connections 2024-05-03T18:33:32.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:32.882+0000 7f7f78200700 1 -- 172.21.15.33:0/4020744458 wait complete. 2024-05-03T18:33:33.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:33:32 smithi033 ceph-mon[28330]: pgmap v193: 129 pgs: 129 active+clean; 1.9 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 8.9 MiB/s wr, 630 op/s 2024-05-03T18:33:33.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:33:32 smithi099 ceph-mon[31658]: pgmap v193: 129 pgs: 129 active+clean; 1.9 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 8.9 MiB/s wr, 630 op/s 2024-05-03T18:33:33.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:33:32 smithi150 ceph-mon[32492]: pgmap v193: 129 pgs: 129 active+clean; 1.9 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 8.9 MiB/s wr, 630 op/s 2024-05-03T18:33:33.761 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-03T18:33:34.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:33:34 smithi099 ceph-mon[31658]: scrub queued for path: / 2024-05-03T18:33:34.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:33:34 smithi099 ceph-mon[31658]: scrub summary: idle+waiting paths [/] 2024-05-03T18:33:34.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:33:34 smithi099 ceph-mon[31658]: scrub summary: active paths [/] 2024-05-03T18:33:34.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:33:34 smithi099 ceph-mon[31658]: scrub summary: idle+waiting paths [/] 2024-05-03T18:33:34.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:33:34 smithi150 ceph-mon[32492]: scrub queued for path: / 2024-05-03T18:33:34.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:33:34 smithi150 ceph-mon[32492]: scrub summary: idle+waiting paths [/] 2024-05-03T18:33:34.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:33:34 smithi150 ceph-mon[32492]: scrub summary: active paths [/] 2024-05-03T18:33:34.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:33:34 smithi150 ceph-mon[32492]: scrub summary: idle+waiting paths [/] 2024-05-03T18:33:34.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:33:34 smithi033 ceph-mon[28330]: scrub queued for path: / 2024-05-03T18:33:34.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:33:34 smithi033 ceph-mon[28330]: scrub summary: idle+waiting paths [/] 2024-05-03T18:33:34.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:33:34 smithi033 ceph-mon[28330]: scrub summary: active paths [/] 2024-05-03T18:33:34.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:33:34 smithi033 ceph-mon[28330]: scrub summary: idle+waiting paths [/] 2024-05-03T18:33:34.530 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:33:35.112 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 140.0 s, 439.3 tps, lat 174.374 ms stddev 267.890 2024-05-03T18:33:35.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:33:35 smithi099 ceph-mon[31658]: pgmap v194: 129 pgs: 129 active+clean; 2.0 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 16 MiB/s wr, 878 op/s 2024-05-03T18:33:35.379 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:33:35 smithi033 ceph-mon[28330]: pgmap v194: 129 pgs: 129 active+clean; 2.0 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 16 MiB/s wr, 878 op/s 2024-05-03T18:33:35.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:33:35 smithi150 ceph-mon[32492]: pgmap v194: 129 pgs: 129 active+clean; 2.0 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 16 MiB/s wr, 878 op/s 2024-05-03T18:33:36.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:33:36 smithi033 ceph-mon[28330]: pgmap v195: 129 pgs: 129 active+clean; 2.0 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 13 MiB/s wr, 593 op/s 2024-05-03T18:33:36.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:33:36 smithi099 ceph-mon[31658]: pgmap v195: 129 pgs: 129 active+clean; 2.0 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 13 MiB/s wr, 593 op/s 2024-05-03T18:33:36.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:33:36 smithi150 ceph-mon[32492]: pgmap v195: 129 pgs: 129 active+clean; 2.0 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 13 MiB/s wr, 593 op/s 2024-05-03T18:33:37.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:33:37 smithi099 ceph-mon[31658]: scrub summary: idle 2024-05-03T18:33:37.296 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:33:37 smithi033 ceph-mon[28330]: scrub summary: idle 2024-05-03T18:33:37.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.295+0000 7fe9fb200700 1 -- 172.21.15.33:0/554952480 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9f4104100 msgr2=0x7fe9f4108120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:33:37.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.295+0000 7fe9fb200700 1 --2- 172.21.15.33:0/554952480 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9f4104100 0x7fe9f4108120 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc009920 tx=0x7fe9dc009c30 comp rx=0 tx=0).stop 2024-05-03T18:33:37.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.295+0000 7fe9fb200700 1 -- 172.21.15.33:0/554952480 shutdown_connections 2024-05-03T18:33:37.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.295+0000 7fe9fb200700 1 --2- 172.21.15.33:0/554952480 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe9f4108850 0x7fe9f41103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:37.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.295+0000 7fe9fb200700 1 --2- 172.21.15.33:0/554952480 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9f4103750 0x7fe9f4103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:37.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.295+0000 7fe9fb200700 1 --2- 172.21.15.33:0/554952480 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9f4104100 0x7fe9f4108120 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:37.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.295+0000 7fe9fb200700 1 -- 172.21.15.33:0/554952480 >> 172.21.15.33:0/554952480 conn(0x7fe9f407a600 msgr2=0x7fe9f407aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:33:37.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.295+0000 7fe9fb200700 1 -- 172.21.15.33:0/554952480 shutdown_connections 2024-05-03T18:33:37.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.296+0000 7fe9fb200700 1 -- 172.21.15.33:0/554952480 wait complete. 2024-05-03T18:33:37.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.297+0000 7fe9fb200700 1 Processor -- start 2024-05-03T18:33:37.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.298+0000 7fe9fb200700 1 -- start start 2024-05-03T18:33:37.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.299+0000 7fe9fb200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9f4103750 0x7fe9f4110340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:37.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.299+0000 7fe9fb200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe9f4104100 0x7fe9f410d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:37.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.299+0000 7fe9fb200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9f4108850 0x7fe9f410dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:37.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.299+0000 7fe9fb200700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe9f40770c0 con 0x7fe9f4108850 2024-05-03T18:33:37.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.299+0000 7fe9fb200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe9f4076f40 con 0x7fe9f4103750 2024-05-03T18:33:37.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.299+0000 7fe9fb200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe9f4077240 con 0x7fe9f4104100 2024-05-03T18:33:37.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.299+0000 7fe9eb400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe9f4104100 0x7fe9f410d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:37.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.299+0000 7fe9f3e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9f4103750 0x7fe9f4110340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:37.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.299+0000 7fe9f9200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9f4108850 0x7fe9f410dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:37.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.299+0000 7fe9f9200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9f4108850 0x7fe9f410dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46192/0 (socket says 172.21.15.33:46192) 2024-05-03T18:33:37.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.299+0000 7fe9f9200700 1 -- 172.21.15.33:0/1926893092 learned_addr learned my addr 172.21.15.33:0/1926893092 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:33:37.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.300+0000 7fe9f9200700 1 -- 172.21.15.33:0/1926893092 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe9f4104100 msgr2=0x7fe9f410d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:33:37.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.300+0000 7fe9f9200700 1 --2- 172.21.15.33:0/1926893092 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe9f4104100 0x7fe9f410d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:37.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.300+0000 7fe9f9200700 1 -- 172.21.15.33:0/1926893092 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9f4103750 msgr2=0x7fe9f4110340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:33:37.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.300+0000 7fe9f9200700 1 --2- 172.21.15.33:0/1926893092 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9f4103750 0x7fe9f4110340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:37.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.301+0000 7fe9f9200700 1 -- 172.21.15.33:0/1926893092 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9f41a1c90 con 0x7fe9f4108850 2024-05-03T18:33:37.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.301+0000 7fe9eb400700 1 --2- 172.21.15.33:0/1926893092 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe9f4104100 0x7fe9f410d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:33:37.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.301+0000 7fe9f9200700 1 --2- 172.21.15.33:0/1926893092 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9f4108850 0x7fe9f410dbb0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec00e970 tx=0x7fe9ec00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:33:37.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.302+0000 7fe9f1600700 1 -- 172.21.15.33:0/1926893092 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9ec004a30 con 0x7fe9f4108850 2024-05-03T18:33:37.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.302+0000 7fe9f1600700 1 -- 172.21.15.33:0/1926893092 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe9ec0054b0 con 0x7fe9f4108850 2024-05-03T18:33:37.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.302+0000 7fe9f1600700 1 -- 172.21.15.33:0/1926893092 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9ec010c50 con 0x7fe9f4108850 2024-05-03T18:33:37.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.302+0000 7fe9fb200700 1 -- 172.21.15.33:0/1926893092 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9f41a1f80 con 0x7fe9f4108850 2024-05-03T18:33:37.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.303+0000 7fe9fb200700 1 -- 172.21.15.33:0/1926893092 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9f41a2440 con 0x7fe9f4108850 2024-05-03T18:33:37.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.312+0000 7fea05ac2b80 1 Processor -- start 2024-05-03T18:33:37.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.312+0000 7fe9f1600700 1 -- 172.21.15.33:0/1926893092 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fe9ec004f30 con 0x7fe9f4108850 2024-05-03T18:33:37.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.313+0000 7fea05ac2b80 1 -- start start 2024-05-03T18:33:37.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.313+0000 7fea05ac2b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e916767b0 0x557e91676b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:37.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.313+0000 7fea05ac2b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e916770d0 0x557e91671fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:37.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.313+0000 7fea05ac2b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e916725d0 0x557e91672a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:37.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.313+0000 7fea05ac2b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x557e915c4a20 con 0x557e916770d0 2024-05-03T18:33:37.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.313+0000 7fea05ac2b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x557e91562760 con 0x557e916725d0 2024-05-03T18:33:37.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.313+0000 7fea05ac2b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x557e915c09e0 con 0x557e916767b0 2024-05-03T18:33:37.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.315+0000 7fe9eb400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e916725d0 0x557e91672a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:37.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.315+0000 7fe9eb400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e916767b0 0x557e91676b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:37.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.315+0000 7fe9eb400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e916767b0 0x557e91676b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:38996/0 (socket says 172.21.15.33:38996) 2024-05-03T18:33:37.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.315+0000 7fe9eb400700 1 -- 172.21.15.33:0/2785229339 learned_addr learned my addr 172.21.15.33:0/2785229339 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:33:37.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.315+0000 7fe9f3e00700 1 --2- 172.21.15.33:0/2785229339 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e916770d0 0x557e91671fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:37.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.316+0000 7fe9f3e00700 1 -- 172.21.15.33:0/2785229339 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e916767b0 msgr2=0x557e91676b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:33:37.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.316+0000 7fe9f3e00700 1 --2- 172.21.15.33:0/2785229339 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e916767b0 0x557e91676b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:37.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.316+0000 7fe9f3e00700 1 -- 172.21.15.33:0/2785229339 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e916725d0 msgr2=0x557e91672a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:33:37.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.316+0000 7fe9f3e00700 1 --2- 172.21.15.33:0/2785229339 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e916725d0 0x557e91672a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:37.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.316+0000 7fe9f3e00700 1 -- 172.21.15.33:0/2785229339 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e915118d0 con 0x557e916770d0 2024-05-03T18:33:37.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.317+0000 7fe9f1600700 1 --2- 172.21.15.33:0/1926893092 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fe9d80b2bc0 0x7fe9d80b5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:37.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.317+0000 7fe9f1600700 1 -- 172.21.15.33:0/1926893092 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7fe9ec0e2b60 con 0x7fe9f4108850 2024-05-03T18:33:37.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.318+0000 7fe9eb400700 1 --2- 172.21.15.33:0/1926893092 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fe9d80b2bc0 0x7fe9d80b5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:37.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.318+0000 7fe9f3e00700 1 --2- 172.21.15.33:0/2785229339 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e916770d0 0x557e91671fb0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fe9e4003fa0 tx=0x7fe9e4009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fabfc3091c401097 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:33:37.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.320+0000 7fe9eb400700 1 --2- 172.21.15.33:0/1926893092 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fe9d80b2bc0 0x7fe9d80b5080 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc009230 tx=0x7fe9dc019040 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:33:37.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.320+0000 7fe9e9600700 1 -- 172.21.15.33:0/2785229339 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9e4004bb0 con 0x557e916770d0 2024-05-03T18:33:37.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.320+0000 7fe9e9600700 1 -- 172.21.15.33:0/2785229339 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe9e4004d50 con 0x557e916770d0 2024-05-03T18:33:37.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.320+0000 7fe9e9600700 1 -- 172.21.15.33:0/2785229339 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9e4007c90 con 0x557e916770d0 2024-05-03T18:33:37.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.321+0000 7fea05ac2b80 1 -- 172.21.15.33:0/2785229339 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e916770d0 msgr2=0x557e91671fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:33:37.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.321+0000 7fea05ac2b80 1 --2- 172.21.15.33:0/2785229339 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e916770d0 0x557e91671fb0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fe9e4003fa0 tx=0x7fe9e4009ba0 comp rx=0 tx=0).stop 2024-05-03T18:33:37.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.321+0000 7fea05ac2b80 1 -- 172.21.15.33:0/2785229339 shutdown_connections 2024-05-03T18:33:37.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.321+0000 7fea05ac2b80 1 --2- 172.21.15.33:0/2785229339 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e916725d0 0x557e91672a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:37.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.321+0000 7fea05ac2b80 1 --2- 172.21.15.33:0/2785229339 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e916767b0 0x557e91676b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:37.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.321+0000 7fea05ac2b80 1 --2- 172.21.15.33:0/2785229339 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e916770d0 0x557e91671fb0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:37.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.321+0000 7fea05ac2b80 1 -- 172.21.15.33:0/2785229339 >> 172.21.15.33:0/2785229339 conn(0x557e915a0b60 msgr2=0x557e9166c7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:33:37.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.323+0000 7fea05ac2b80 1 -- 172.21.15.33:0/2785229339 shutdown_connections 2024-05-03T18:33:37.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.323+0000 7fea05ac2b80 1 -- 172.21.15.33:0/2785229339 wait complete. 2024-05-03T18:33:37.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.325+0000 7fea05ac2b80 1 -- start start 2024-05-03T18:33:37.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.326+0000 7fea05ac2b80 1 Processor -- start 2024-05-03T18:33:37.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.328+0000 7fea05ac2b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e916725d0 0x557e9167de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:37.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.328+0000 7fea05ac2b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e916767b0 0x557e9167b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:37.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.328+0000 7fea05ac2b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e916770d0 0x557e9167b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:37.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.328+0000 7fea05ac2b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x557e9167eca0 con 0x557e916725d0 2024-05-03T18:33:37.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.328+0000 7fea05ac2b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x557e9152ab20 con 0x557e916770d0 2024-05-03T18:33:37.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.328+0000 7fea05ac2b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x557e9167ee20 con 0x557e916767b0 2024-05-03T18:33:37.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.328+0000 7fea05ac2b80 10 client.0 _ll_register_callbacks cb 0x557e9156bc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:33:37.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.329+0000 7fe9f3e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e916725d0 0x557e9167de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:37.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.329+0000 7fe9eb400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e916767b0 0x557e9167b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:37.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.329+0000 7fe9eb400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e916767b0 0x557e9167b220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:39008/0 (socket says 172.21.15.33:39008) 2024-05-03T18:33:37.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.329+0000 7fe9eb400700 1 -- 172.21.15.33:0/3850260313 learned_addr learned my addr 172.21.15.33:0/3850260313 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:33:37.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.329+0000 7fe9f9200700 1 --2- 172.21.15.33:0/3850260313 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e916770d0 0x557e9167b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:37.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.330+0000 7fe9eb400700 1 -- 172.21.15.33:0/3850260313 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e916770d0 msgr2=0x557e9167b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:33:37.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.330+0000 7fe9eb400700 1 --2- 172.21.15.33:0/3850260313 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e916770d0 0x557e9167b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:37.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.330+0000 7fe9eb400700 1 -- 172.21.15.33:0/3850260313 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e916725d0 msgr2=0x557e9167de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:33:37.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.330+0000 7fe9eb400700 1 --2- 172.21.15.33:0/3850260313 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e916725d0 0x557e9167de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:37.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.330+0000 7fe9eb400700 1 -- 172.21.15.33:0/3850260313 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e9158a420 con 0x557e916767b0 2024-05-03T18:33:37.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.331+0000 7fe9eb400700 1 --2- 172.21.15.33:0/3850260313 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e916767b0 0x557e9167b220 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc010f90 tx=0x7fe9dc009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ed05989a60bf190 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:33:37.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.331+0000 7fe9f3e00700 1 --2- 172.21.15.33:0/3850260313 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e916725d0 0x557e9167de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:33:37.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.332+0000 7fe9e2000700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-03T18:33:37.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.332+0000 7fe9e2000700 1 -- 172.21.15.33:0/3850260313 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9dc003680 con 0x557e916767b0 2024-05-03T18:33:37.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.332+0000 7fe9e2000700 1 -- 172.21.15.33:0/3850260313 <== mon.1 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7fe9dc003820 con 0x557e916767b0 2024-05-03T18:33:37.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.332+0000 7fea05ac2b80 1 -- 172.21.15.33:0/3850260313 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9e4004d50 con 0x557e916767b0 2024-05-03T18:33:37.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.334+0000 7fe9e2000700 1 -- 172.21.15.33:0/3850260313 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fe9dc003a60 con 0x557e916767b0 2024-05-03T18:33:37.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.334+0000 7fea05ac2b80 10 client.25756 fetch_fsmap learned FSMap version 21 2024-05-03T18:33:37.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.334+0000 7fea05ac2b80 1 -- 172.21.15.33:0/3850260313 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x557e91689470 con 0x557e916767b0 2024-05-03T18:33:37.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.335+0000 7fe9e2000700 1 -- 172.21.15.33:0/3850260313 <== mon.1 v2:172.21.15.150:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe9e4004d50 con 0x557e916767b0 2024-05-03T18:33:37.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.338+0000 7fe9e2000700 1 -- 172.21.15.33:0/3850260313 <== mon.1 v2:172.21.15.150:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fe9dc031070 con 0x557e916767b0 2024-05-03T18:33:37.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.338+0000 7fea05ac2b80 10 client.25756 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:33:37.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.338+0000 7fea05ac2b80 10 client.25756 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:33:37.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.338+0000 7fea05ac2b80 20 client.25756 populate_metadata read hostname 'smithi033' 2024-05-03T18:33:37.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.338+0000 7fea05ac2b80 1 --2- 172.21.15.33:0/3850260313 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x557e9168bab0 0x557e9168dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:37.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.338+0000 7fea05ac2b80 4 client.25756 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:33:37.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.338+0000 7fea05ac2b80 1 -- 172.21.15.33:0/3850260313 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557e9163f1d0 con 0x557e9168bab0 2024-05-03T18:33:37.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.339+0000 7fe9f3e00700 1 --2- 172.21.15.33:0/3850260313 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x557e9168bab0 0x557e9168dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:37.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.342+0000 7fe9e2000700 10 client.25756 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:33:37.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.342+0000 7fe9f3e00700 1 --2- 172.21.15.33:0/3850260313 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x557e9168bab0 0x557e9168dea0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0cb0ec203424c9d server_cookie=40aeb6fb1a249e80 in_seq=0 out_seq=0 2024-05-03T18:33:37.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.345+0000 7fe9e2000700 1 -- 172.21.15.33:0/3850260313 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557e9163f1d0 con 0x557e9168bab0 2024-05-03T18:33:37.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.345+0000 7fe9e2000700 10 client.25756 handle_command_reply: tid=0 2024-05-03T18:33:37.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.345+0000 7fea05ac2b80 1 client.25756 shutdown 2024-05-03T18:33:37.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.346+0000 7fea05ac2b80 1 -- 172.21.15.33:0/3850260313 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e916767b0 msgr2=0x557e9167b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:33:37.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.346+0000 7fea05ac2b80 1 --2- 172.21.15.33:0/3850260313 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e916767b0 0x557e9167b220 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc010f90 tx=0x7fe9dc009260 comp rx=0 tx=0).stop 2024-05-03T18:33:37.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.347+0000 7fea05ac2b80 1 -- 172.21.15.33:0/3850260313 shutdown_connections 2024-05-03T18:33:37.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.347+0000 7fea05ac2b80 1 --2- 172.21.15.33:0/3850260313 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x557e9168bab0 0x557e9168dea0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:37.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.347+0000 7fea05ac2b80 1 --2- 172.21.15.33:0/3850260313 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e916770d0 0x557e9167b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:37.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.347+0000 7fea05ac2b80 1 --2- 172.21.15.33:0/3850260313 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e916725d0 0x557e9167de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:37.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.347+0000 7fe9e2000700 0 client.25756 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:33:37.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.347+0000 7fea05ac2b80 1 --2- 172.21.15.33:0/3850260313 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e916767b0 0x557e9167b220 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:37.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.347+0000 7fea05ac2b80 1 -- 172.21.15.33:0/3850260313 >> 172.21.15.33:0/3850260313 conn(0x557e915a0b60 msgr2=0x557e9166b650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:33:37.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.347+0000 7fea05ac2b80 1 -- 172.21.15.33:0/3850260313 shutdown_connections 2024-05-03T18:33:37.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.347+0000 7fea05ac2b80 1 -- 172.21.15.33:0/3850260313 wait complete. 2024-05-03T18:33:37.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.348+0000 7fea05ac2b80 20 client.25756 trim_cache size 0 max 16384 2024-05-03T18:33:37.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:33:37 smithi150 ceph-mon[32492]: scrub summary: idle 2024-05-03T18:33:37.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.392+0000 7fea05ac2b80 1 Processor -- start 2024-05-03T18:33:37.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.393+0000 7fea05ac2b80 1 -- start start 2024-05-03T18:33:37.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.393+0000 7fea05ac2b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e916725d0 0x557e9166c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:37.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.393+0000 7fea05ac2b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e916767b0 0x557e9167cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:37.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.393+0000 7fea05ac2b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e916770d0 0x557e91693770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:37.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.393+0000 7fea05ac2b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe9f41bc2f0 con 0x557e916770d0 2024-05-03T18:33:37.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.393+0000 7fea05ac2b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe9f41beb70 con 0x557e916767b0 2024-05-03T18:33:37.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.393+0000 7fea05ac2b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe9f41bc960 con 0x557e916725d0 2024-05-03T18:33:37.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.394+0000 7fe9f9200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e916770d0 0x557e91693770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:37.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.394+0000 7fe9f9200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e916770d0 0x557e91693770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46210/0 (socket says 172.21.15.33:46210) 2024-05-03T18:33:37.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.394+0000 7fe9f9200700 1 -- 172.21.15.33:0/2731278409 learned_addr learned my addr 172.21.15.33:0/2731278409 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:33:37.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.395+0000 7fe9f3e00700 1 --2- 172.21.15.33:0/2731278409 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e916725d0 0x557e9166c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:37.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.395+0000 7fe9f9200700 1 -- 172.21.15.33:0/2731278409 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e916725d0 msgr2=0x557e9166c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:33:37.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.395+0000 7fe9f9200700 1 --2- 172.21.15.33:0/2731278409 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e916725d0 0x557e9166c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:37.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.395+0000 7fe9f9200700 1 -- 172.21.15.33:0/2731278409 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e916767b0 msgr2=0x557e9167cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:33:37.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.395+0000 7fe9f9200700 1 --2- 172.21.15.33:0/2731278409 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e916767b0 0x557e9167cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:37.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.395+0000 7fe9f9200700 1 -- 172.21.15.33:0/2731278409 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e916677a0 con 0x557e916770d0 2024-05-03T18:33:37.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.396+0000 7fe9f9200700 1 --2- 172.21.15.33:0/2731278409 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e916770d0 0x557e91693770 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec0dff20 tx=0x7fe9ec0a5250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fc98e4fb4183083 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:33:37.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.397+0000 7fe9e8c00700 1 -- 172.21.15.33:0/2731278409 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9ec0a5040 con 0x557e916770d0 2024-05-03T18:33:37.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.398+0000 7fe9e8c00700 1 -- 172.21.15.33:0/2731278409 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe9ec0c2030 con 0x557e916770d0 2024-05-03T18:33:37.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.398+0000 7fe9e8c00700 1 -- 172.21.15.33:0/2731278409 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9ec004c60 con 0x557e916770d0 2024-05-03T18:33:37.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.399+0000 7fea05ac2b80 1 -- 172.21.15.33:0/2731278409 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e916770d0 msgr2=0x557e91693770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:33:37.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.399+0000 7fea05ac2b80 1 --2- 172.21.15.33:0/2731278409 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e916770d0 0x557e91693770 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec0dff20 tx=0x7fe9ec0a5250 comp rx=0 tx=0).stop 2024-05-03T18:33:37.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.400+0000 7fea05ac2b80 1 -- 172.21.15.33:0/2731278409 shutdown_connections 2024-05-03T18:33:37.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.400+0000 7fea05ac2b80 1 --2- 172.21.15.33:0/2731278409 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e916770d0 0x557e91693770 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:37.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.400+0000 7fea05ac2b80 1 --2- 172.21.15.33:0/2731278409 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e916725d0 0x557e9166c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:37.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.400+0000 7fea05ac2b80 1 --2- 172.21.15.33:0/2731278409 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e916767b0 0x557e9167cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:37.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.400+0000 7fea05ac2b80 1 -- 172.21.15.33:0/2731278409 >> 172.21.15.33:0/2731278409 conn(0x557e915a0b60 msgr2=0x557e9167c8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:33:37.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.401+0000 7fea05ac2b80 1 -- 172.21.15.33:0/2731278409 shutdown_connections 2024-05-03T18:33:37.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.401+0000 7fea05ac2b80 1 -- 172.21.15.33:0/2731278409 wait complete. 2024-05-03T18:33:37.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.402+0000 7fea05ac2b80 1 -- start start 2024-05-03T18:33:37.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.403+0000 7fea05ac2b80 1 Processor -- start 2024-05-03T18:33:37.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.403+0000 7fea05ac2b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e916725d0 0x557e916840e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:37.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.403+0000 7fea05ac2b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e916767b0 0x557e91684620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:37.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.403+0000 7fea05ac2b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e916770d0 0x557e916ef9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:37.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.404+0000 7fe9f9200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e916770d0 0x557e916ef9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:37.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.404+0000 7fe9f3e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e916725d0 0x557e916840e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:37.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.404+0000 7fea05ac2b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe9f41bd030 con 0x557e916770d0 2024-05-03T18:33:37.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.404+0000 7fea05ac2b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe9f41be4a0 con 0x557e916767b0 2024-05-03T18:33:37.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.404+0000 7fea05ac2b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe9f41bd700 con 0x557e916725d0 2024-05-03T18:33:37.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.404+0000 7fe9eb400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e916767b0 0x557e91684620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:37.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.404+0000 7fe9eb400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e916767b0 0x557e91684620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:42316/0 (socket says 172.21.15.33:42316) 2024-05-03T18:33:37.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.404+0000 7fe9eb400700 1 -- 172.21.15.33:0/2378753697 learned_addr learned my addr 172.21.15.33:0/2378753697 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:33:37.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.405+0000 7fea05ac2b80 10 client.0 _ll_register_callbacks cb 0x557e9156bc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:33:37.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.406+0000 7fe9f3e00700 1 -- 172.21.15.33:0/2378753697 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e916767b0 msgr2=0x557e91684620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:33:37.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.406+0000 7fe9f3e00700 1 --2- 172.21.15.33:0/2378753697 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e916767b0 0x557e91684620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:37.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.406+0000 7fe9f3e00700 1 -- 172.21.15.33:0/2378753697 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e916770d0 msgr2=0x557e916ef9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:33:37.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.406+0000 7fe9f3e00700 1 --2- 172.21.15.33:0/2378753697 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e916770d0 0x557e916ef9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:37.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.406+0000 7fe9f3e00700 1 -- 172.21.15.33:0/2378753697 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e916f01d0 con 0x557e916725d0 2024-05-03T18:33:37.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.407+0000 7fe9eb400700 1 --2- 172.21.15.33:0/2378753697 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e916767b0 0x557e91684620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:33:37.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.407+0000 7fe9f9200700 1 --2- 172.21.15.33:0/2378753697 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e916770d0 0x557e916ef9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:33:37.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.408+0000 7fe9f3e00700 1 --2- 172.21.15.33:0/2378753697 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e916725d0 0x557e916840e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x557e9168e710 tx=0x7fe9e4007470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2065771d497b2e server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:33:37.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.409+0000 7fe9e2000700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-03T18:33:37.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.409+0000 7fe9e2000700 1 -- 172.21.15.33:0/2378753697 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9e4003a00 con 0x557e916725d0 2024-05-03T18:33:37.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.409+0000 7fe9e2000700 1 -- 172.21.15.33:0/2378753697 <== mon.1 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7fe9e4004d00 con 0x557e916725d0 2024-05-03T18:33:37.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.410+0000 7fea05ac2b80 1 -- 172.21.15.33:0/2378753697 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9ec0c2030 con 0x557e916725d0 2024-05-03T18:33:37.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.410+0000 7fe9e2000700 1 -- 172.21.15.33:0/2378753697 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fe9e4004510 con 0x557e916725d0 2024-05-03T18:33:37.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.410+0000 7fea05ac2b80 10 client.25759 fetch_fsmap learned FSMap version 21 2024-05-03T18:33:37.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.411+0000 7fea05ac2b80 1 -- 172.21.15.33:0/2378753697 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x557e91707f80 con 0x557e916725d0 2024-05-03T18:33:37.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.411+0000 7fe9e2000700 1 -- 172.21.15.33:0/2378753697 <== mon.1 v2:172.21.15.150:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe9ec0c2030 con 0x557e916725d0 2024-05-03T18:33:37.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.412+0000 7fe9e2000700 1 -- 172.21.15.33:0/2378753697 <== mon.1 v2:172.21.15.150:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fe9e4015480 con 0x557e916725d0 2024-05-03T18:33:37.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.412+0000 7fea05ac2b80 10 client.25759 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:33:37.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.412+0000 7fea05ac2b80 10 client.25759 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:33:37.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.412+0000 7fea05ac2b80 20 client.25759 populate_metadata read hostname 'smithi033' 2024-05-03T18:33:37.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.412+0000 7fea05ac2b80 1 --2- 172.21.15.33:0/2378753697 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x557e917083e0 0x557e9170a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:37.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.412+0000 7fea05ac2b80 4 client.25759 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-03T18:33:37.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.412+0000 7fea05ac2b80 1 -- 172.21.15.33:0/2378753697 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557e915d2400 con 0x557e917083e0 2024-05-03T18:33:37.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.413+0000 7fe9eb400700 1 --2- 172.21.15.33:0/2378753697 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x557e917083e0 0x557e9170a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:37.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.414+0000 7fe9eb400700 1 --2- 172.21.15.33:0/2378753697 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x557e917083e0 0x557e9170a800 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=458af679b64cb98d server_cookie=16c5f2d4607b47f0 in_seq=0 out_seq=0 2024-05-03T18:33:37.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.414+0000 7fe9e2000700 10 client.25759 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:33:37.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.414+0000 7fe9e2000700 1 -- 172.21.15.33:0/2378753697 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x557e915d2400 con 0x557e917083e0 2024-05-03T18:33:37.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.414+0000 7fe9e2000700 10 client.25759 handle_command_reply: tid=0 2024-05-03T18:33:37.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.414+0000 7fea05ac2b80 1 client.25759 shutdown 2024-05-03T18:33:37.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.415+0000 7fea05ac2b80 1 -- 172.21.15.33:0/2378753697 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e916725d0 msgr2=0x557e916840e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:33:37.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.415+0000 7fea05ac2b80 1 --2- 172.21.15.33:0/2378753697 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e916725d0 0x557e916840e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x557e9168e710 tx=0x7fe9e4007470 comp rx=0 tx=0).stop 2024-05-03T18:33:37.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.416+0000 7fea05ac2b80 1 -- 172.21.15.33:0/2378753697 shutdown_connections 2024-05-03T18:33:37.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.416+0000 7fea05ac2b80 1 --2- 172.21.15.33:0/2378753697 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x557e917083e0 0x557e9170a800 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:37.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.416+0000 7fea05ac2b80 1 --2- 172.21.15.33:0/2378753697 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e916770d0 0x557e916ef9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:37.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.416+0000 7fea05ac2b80 1 --2- 172.21.15.33:0/2378753697 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e916725d0 0x557e916840e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:37.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.416+0000 7fea05ac2b80 1 --2- 172.21.15.33:0/2378753697 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e916767b0 0x557e91684620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:37.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.416+0000 7fe9e2000700 0 client.25759 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:33:37.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.416+0000 7fea05ac2b80 1 -- 172.21.15.33:0/2378753697 >> 172.21.15.33:0/2378753697 conn(0x557e915a0b60 msgr2=0x557e91675c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:33:37.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.417+0000 7fea05ac2b80 1 -- 172.21.15.33:0/2378753697 shutdown_connections 2024-05-03T18:33:37.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.417+0000 7fea05ac2b80 1 -- 172.21.15.33:0/2378753697 wait complete. 2024-05-03T18:33:37.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.417+0000 7fea05ac2b80 20 client.25759 trim_cache size 0 max 16384 2024-05-03T18:33:37.419 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-03T18:33:37.419 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-05-03T18:33:37.419 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-05-03T18:33:37.419 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-03T18:33:37.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.419+0000 7fe9fb200700 1 -- 172.21.15.33:0/1926893092 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fe9d80b2bc0 msgr2=0x7fe9d80b5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:33:37.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.419+0000 7fe9fb200700 1 --2- 172.21.15.33:0/1926893092 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fe9d80b2bc0 0x7fe9d80b5080 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc009230 tx=0x7fe9dc019040 comp rx=0 tx=0).stop 2024-05-03T18:33:37.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.419+0000 7fe9fb200700 1 -- 172.21.15.33:0/1926893092 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9f4108850 msgr2=0x7fe9f410dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:33:37.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.419+0000 7fe9fb200700 1 --2- 172.21.15.33:0/1926893092 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9f4108850 0x7fe9f410dbb0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec00e970 tx=0x7fe9ec00ec80 comp rx=0 tx=0).stop 2024-05-03T18:33:37.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.419+0000 7fe9fb200700 1 -- 172.21.15.33:0/1926893092 shutdown_connections 2024-05-03T18:33:37.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.419+0000 7fe9fb200700 1 --2- 172.21.15.33:0/1926893092 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9f4108850 0x7fe9f410dbb0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:37.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.419+0000 7fe9fb200700 1 --2- 172.21.15.33:0/1926893092 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fe9d80b2bc0 0x7fe9d80b5080 secure :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc009230 tx=0x7fe9dc019040 comp rx=0 tx=0).stop 2024-05-03T18:33:37.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.419+0000 7fe9fb200700 1 --2- 172.21.15.33:0/1926893092 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9f4103750 0x7fe9f4110340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:37.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.419+0000 7fe9fb200700 1 --2- 172.21.15.33:0/1926893092 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe9f4104100 0x7fe9f410d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:37.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.419+0000 7fe9fb200700 1 -- 172.21.15.33:0/1926893092 >> 172.21.15.33:0/1926893092 conn(0x7fe9f407a600 msgr2=0x7fe9f4101870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:33:37.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.422+0000 7fe9fb200700 1 -- 172.21.15.33:0/1926893092 shutdown_connections 2024-05-03T18:33:37.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:37.422+0000 7fe9fb200700 1 -- 172.21.15.33:0/1926893092 wait complete. 2024-05-03T18:33:38.185 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:33:38 smithi033 ceph-mon[28330]: pgmap v196: 129 pgs: 129 active+clean; 2.0 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 18 MiB/s wr, 835 op/s 2024-05-03T18:33:38.231 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-03T18:33:38.231 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T18:33:38.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:33:38 smithi099 ceph-mon[31658]: pgmap v196: 129 pgs: 129 active+clean; 2.0 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 18 MiB/s wr, 835 op/s 2024-05-03T18:33:38.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:33:38 smithi150 ceph-mon[32492]: pgmap v196: 129 pgs: 129 active+clean; 2.0 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 18 MiB/s wr, 835 op/s 2024-05-03T18:33:38.874 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:33:40.110 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 145.0 s, 751.7 tps, lat 163.499 ms stddev 296.964 2024-05-03T18:33:41.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:33:41 smithi099 ceph-mon[31658]: pgmap v197: 129 pgs: 129 active+clean; 2.0 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 13 MiB/s wr, 588 op/s 2024-05-03T18:33:41.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:33:41 smithi150 ceph-mon[32492]: pgmap v197: 129 pgs: 129 active+clean; 2.0 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 13 MiB/s wr, 588 op/s 2024-05-03T18:33:41.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:33:41 smithi033 ceph-mon[28330]: pgmap v197: 129 pgs: 129 active+clean; 2.0 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 13 MiB/s wr, 588 op/s 2024-05-03T18:33:42.259 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:33:42 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:33:42.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:33:42 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:33:42.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:33:42 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:33:42.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:42.657+0000 7f20b3800700 1 -- 172.21.15.33:0/1761957163 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20ac109510 msgr2=0x7f20ac1098f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:33:42.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:42.657+0000 7f20b3800700 1 --2- 172.21.15.33:0/1761957163 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20ac109510 0x7f20ac1098f0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f209c00b0a0 tx=0x7f209c00b3b0 comp rx=0 tx=0).stop 2024-05-03T18:33:42.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:42.657+0000 7f20b3800700 1 -- 172.21.15.33:0/1761957163 shutdown_connections 2024-05-03T18:33:42.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:42.657+0000 7f20b3800700 1 --2- 172.21.15.33:0/1761957163 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20ac06bb40 0x7f20ac110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:42.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:42.657+0000 7f20b3800700 1 --2- 172.21.15.33:0/1761957163 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20ac109510 0x7f20ac1098f0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:42.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:42.657+0000 7f20b3800700 1 --2- 172.21.15.33:0/1761957163 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20ac06b180 0x7f20ac06b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:42.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:42.657+0000 7f20b3800700 1 -- 172.21.15.33:0/1761957163 >> 172.21.15.33:0/1761957163 conn(0x7f20ac079350 msgr2=0x7f20ac079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:33:42.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:42.658+0000 7f20b3800700 1 -- 172.21.15.33:0/1761957163 shutdown_connections 2024-05-03T18:33:42.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:42.658+0000 7f20b3800700 1 -- 172.21.15.33:0/1761957163 wait complete. 2024-05-03T18:33:42.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:42.660+0000 7f20b3800700 1 Processor -- start 2024-05-03T18:33:42.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:42.660+0000 7f20b3800700 1 -- start start 2024-05-03T18:33:42.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:42.660+0000 7f20b3800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20ac06b180 0x7f20ac117800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:42.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:42.660+0000 7f20b3800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20ac06bb40 0x7f20ac117d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:42.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:42.660+0000 7f20b3800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20ac109510 0x7f20ac112920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:42.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:42.661+0000 7f20b3800700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f20ac103700 con 0x7f20ac06b180 2024-05-03T18:33:42.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:42.661+0000 7f20b3800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f20ac103580 con 0x7f20ac109510 2024-05-03T18:33:42.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:42.661+0000 7f20b3800700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f20ac103880 con 0x7f20ac06bb40 2024-05-03T18:33:42.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:42.661+0000 7f20b0e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20ac06b180 0x7f20ac117800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:42.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:42.661+0000 7f20b0e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20ac06b180 0x7f20ac117800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46234/0 (socket says 172.21.15.33:46234) 2024-05-03T18:33:42.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:42.661+0000 7f20b0e00700 1 -- 172.21.15.33:0/663206835 learned_addr learned my addr 172.21.15.33:0/663206835 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:33:42.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:42.661+0000 7f20a3e00700 1 --2- 172.21.15.33:0/663206835 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20ac06bb40 0x7f20ac117d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:42.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:42.662+0000 7f20b1800700 1 --2- 172.21.15.33:0/663206835 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20ac109510 0x7f20ac112920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:42.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:42.662+0000 7f20b0e00700 1 -- 172.21.15.33:0/663206835 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20ac06bb40 msgr2=0x7f20ac117d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:33:42.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:42.662+0000 7f20b0e00700 1 --2- 172.21.15.33:0/663206835 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20ac06bb40 0x7f20ac117d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:42.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:42.662+0000 7f20b0e00700 1 -- 172.21.15.33:0/663206835 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20ac109510 msgr2=0x7f20ac112920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:33:42.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:42.662+0000 7f20b0e00700 1 --2- 172.21.15.33:0/663206835 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20ac109510 0x7f20ac112920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:42.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:42.662+0000 7f20b0e00700 1 -- 172.21.15.33:0/663206835 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20ac113220 con 0x7f20ac06b180 2024-05-03T18:33:42.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:42.662+0000 7f20b1800700 1 --2- 172.21.15.33:0/663206835 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20ac109510 0x7f20ac112920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:33:42.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:42.663+0000 7f20b0e00700 1 --2- 172.21.15.33:0/663206835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20ac06b180 0x7f20ac117800 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f209c0062a0 tx=0x7f209c004da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:33:42.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:42.664+0000 7f20aa000700 1 -- 172.21.15.33:0/663206835 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f209c021070 con 0x7f20ac06b180 2024-05-03T18:33:42.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:42.664+0000 7f20b3800700 1 -- 172.21.15.33:0/663206835 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20ac1134b0 con 0x7f20ac06b180 2024-05-03T18:33:42.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:42.665+0000 7f20b3800700 1 -- 172.21.15.33:0/663206835 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20ac11d6f0 con 0x7f20ac06b180 2024-05-03T18:33:42.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:42.665+0000 7f20aa000700 1 -- 172.21.15.33:0/663206835 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f209c009c40 con 0x7f20ac06b180 2024-05-03T18:33:42.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:42.665+0000 7f20aa000700 1 -- 172.21.15.33:0/663206835 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f209c004330 con 0x7f20ac06b180 2024-05-03T18:33:42.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:42.669+0000 7f20b3800700 1 -- 172.21.15.33:0/663206835 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2088005330 con 0x7f20ac06b180 2024-05-03T18:33:42.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:42.671+0000 7f20aa000700 1 -- 172.21.15.33:0/663206835 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f209c0044d0 con 0x7f20ac06b180 2024-05-03T18:33:42.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:42.672+0000 7f20aa000700 1 --2- 172.21.15.33:0/663206835 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f20900b2b20 0x7f20900b4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:42.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:42.672+0000 7f20aa000700 1 -- 172.21.15.33:0/663206835 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f209c00f020 con 0x7f20ac06b180 2024-05-03T18:33:42.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:42.673+0000 7f20a3e00700 1 --2- 172.21.15.33:0/663206835 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f20900b2b20 0x7f20900b4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:42.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:42.674+0000 7f20a3e00700 1 --2- 172.21.15.33:0/663206835 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f20900b2b20 0x7f20900b4fe0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2094005ea0 tx=0x7f2094005e30 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:33:42.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:42.683+0000 7f20aa000700 1 -- 172.21.15.33:0/663206835 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f209c02a020 con 0x7f20ac06b180 2024-05-03T18:33:43.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:43.171+0000 7f20b3800700 1 -- 172.21.15.33:0/663206835 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f20880058f0 con 0x7f20ac06b180 2024-05-03T18:33:43.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:43.176+0000 7f20aa000700 1 -- 172.21.15.33:0/663206835 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11801 (secure 0 0 0) 0x7f209c007c00 con 0x7f20ac06b180 2024-05-03T18:33:43.179 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:33:43.179 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1327393503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1327393503},{"type":"v1","addr":"172.21.15.33:6835","nonce":1327393503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/2461472948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":2461472948},{"type":"v1","addr":"172.21.15.33:6837","nonce":2461472948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14559,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/442269281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":442269281},{"type":"v1","addr":"172.21.15.33:6839","nonce":442269281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14571,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/645571756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":645571756},{"type":"v1","addr":"172.21.15.33:6841","nonce":645571756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/203598909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":203598909},{"type":"v1","addr":"172.21.15.150:6833","nonce":203598909}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/4029842493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":4029842493},{"type":"v1","addr":"172.21.15.150:6835","nonce":4029842493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3463664423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3463664423},{"type":"v1","addr":"172.21.15.150:6837","nonce":3463664423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24481,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6839/4051648018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":4051648018},{"type":"v1","addr":"172.21.15.150:6839","nonce":4051648018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34418,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1865105479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1865105479},{"type":"v1","addr":"172.21.15.99:6833","nonce":1865105479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34445,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/1206393914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1206393914},{"type":"v1","addr":"172.21.15.99:6835","nonce":1206393914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-03T18:22:28.837091+0000","modified":"2024-05-03T18:23:49.758398+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34508},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34472":{"gid":34472,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6837/3360970342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3360970342},{"type":"v1","addr":"172.21.15.99:6837","nonce":3360970342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34508":{"gid":34508,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/919907308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":919907308},{"type":"v1","addr":"172.21.15.99:6839","nonce":919907308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34508,"qdb_cluster":[34508]},"id":1}]} 2024-05-03T18:33:43.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:43.181+0000 7f20b3800700 1 -- 172.21.15.33:0/663206835 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f20900b2b20 msgr2=0x7f20900b4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:33:43.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:43.181+0000 7f20b3800700 1 --2- 172.21.15.33:0/663206835 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f20900b2b20 0x7f20900b4fe0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2094005ea0 tx=0x7f2094005e30 comp rx=0 tx=0).stop 2024-05-03T18:33:43.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:43.181+0000 7f20b3800700 1 -- 172.21.15.33:0/663206835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20ac06b180 msgr2=0x7f20ac117800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:33:43.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:43.181+0000 7f20b3800700 1 --2- 172.21.15.33:0/663206835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20ac06b180 0x7f20ac117800 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f209c0062a0 tx=0x7f209c004da0 comp rx=0 tx=0).stop 2024-05-03T18:33:43.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:43.181+0000 7f20b3800700 1 -- 172.21.15.33:0/663206835 shutdown_connections 2024-05-03T18:33:43.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:43.181+0000 7f20b3800700 1 --2- 172.21.15.33:0/663206835 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f20900b2b20 0x7f20900b4fe0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:43.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:43.181+0000 7f20b3800700 1 --2- 172.21.15.33:0/663206835 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20ac109510 0x7f20ac112920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:43.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:43.181+0000 7f20b3800700 1 --2- 172.21.15.33:0/663206835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20ac06b180 0x7f20ac117800 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:43.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:43.181+0000 7f20b3800700 1 --2- 172.21.15.33:0/663206835 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20ac06bb40 0x7f20ac117d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:43.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:43.181+0000 7f20b3800700 1 -- 172.21.15.33:0/663206835 >> 172.21.15.33:0/663206835 conn(0x7f20ac079350 msgr2=0x7f20ac107060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:33:43.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:43.183+0000 7f20b3800700 1 -- 172.21.15.33:0/663206835 shutdown_connections 2024-05-03T18:33:43.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:43.188+0000 7f20b3800700 1 -- 172.21.15.33:0/663206835 wait complete. 2024-05-03T18:33:43.195 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 21 2024-05-03T18:33:43.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:33:43 smithi099 ceph-mon[31658]: pgmap v198: 129 pgs: 129 active+clean; 2.0 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 13 MiB/s wr, 532 op/s 2024-05-03T18:33:43.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:33:43 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:33:43.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:33:43 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:33:43.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:33:43 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:33:43.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:33:43 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:33:43.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:33:43 smithi150 ceph-mon[32492]: pgmap v198: 129 pgs: 129 active+clean; 2.0 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 13 MiB/s wr, 532 op/s 2024-05-03T18:33:43.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:33:43 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:33:43.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:33:43 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:33:43.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:33:43 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:33:43.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:33:43 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:33:43.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:33:43 smithi033 ceph-mon[28330]: pgmap v198: 129 pgs: 129 active+clean; 2.0 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 13 MiB/s wr, 532 op/s 2024-05-03T18:33:43.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:33:43 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:33:43.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:33:43 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:33:43.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:33:43 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:33:43.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:33:43 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:33:44.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:33:44 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/663206835' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:33:44.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:33:44 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:33:44.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:33:44 smithi099 ceph-mon[31658]: pgmap v199: 129 pgs: 129 active+clean; 2.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 18 MiB/s wr, 713 op/s 2024-05-03T18:33:44.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:33:44 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:33:44.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:33:44 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:33:44.300 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:33:44 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/663206835' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:33:44.300 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:33:44 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:33:44.300 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:33:44 smithi033 ceph-mon[28330]: pgmap v199: 129 pgs: 129 active+clean; 2.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 18 MiB/s wr, 713 op/s 2024-05-03T18:33:44.300 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:33:44 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:33:44.300 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:33:44 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:33:44.304 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-03T18:33:44.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:33:44 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/663206835' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:33:44.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:33:44 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:33:44.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:33:44 smithi150 ceph-mon[32492]: pgmap v199: 129 pgs: 129 active+clean; 2.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 18 MiB/s wr, 713 op/s 2024-05-03T18:33:44.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:33:44 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:33:44.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:33:44 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:33:45.078 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:33:45.110 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 150.0 s, 428.2 tps, lat 210.206 ms stddev 287.689 2024-05-03T18:33:47.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:33:47 smithi033 ceph-mon[28330]: pgmap v200: 129 pgs: 129 active+clean; 2.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 12 MiB/s wr, 465 op/s 2024-05-03T18:33:47.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:33:47 smithi099 ceph-mon[31658]: pgmap v200: 129 pgs: 129 active+clean; 2.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 12 MiB/s wr, 465 op/s 2024-05-03T18:33:47.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:33:47 smithi150 ceph-mon[32492]: pgmap v200: 129 pgs: 129 active+clean; 2.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 12 MiB/s wr, 465 op/s 2024-05-03T18:33:48.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:33:48 smithi099 ceph-mon[31658]: pgmap v201: 129 pgs: 129 active+clean; 2.1 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 17 MiB/s wr, 678 op/s 2024-05-03T18:33:48.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:33:48 smithi150 ceph-mon[32492]: pgmap v201: 129 pgs: 129 active+clean; 2.1 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 17 MiB/s wr, 678 op/s 2024-05-03T18:33:48.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:33:48 smithi033 ceph-mon[28330]: pgmap v201: 129 pgs: 129 active+clean; 2.1 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 17 MiB/s wr, 678 op/s 2024-05-03T18:33:48.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:48.979+0000 7f3365600700 1 -- 172.21.15.33:0/12643746 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3360003b40 con 0x7f3368111b90 2024-05-03T18:33:48.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:48.980+0000 7f3367e00700 1 -- 172.21.15.33:0/12643746 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3368111b90 msgr2=0x7f336810a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:33:48.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:48.980+0000 7f3367e00700 1 --2- 172.21.15.33:0/12643746 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3368111b90 0x7f336810a4a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3360009990 tx=0x7f3360009ca0 comp rx=0 tx=0).stop 2024-05-03T18:33:48.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:48.981+0000 7f3367e00700 1 -- 172.21.15.33:0/12643746 shutdown_connections 2024-05-03T18:33:48.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:48.981+0000 7f3367e00700 1 --2- 172.21.15.33:0/12643746 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f336811cc80 0x7f336811f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:48.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:48.981+0000 7f3367e00700 1 --2- 172.21.15.33:0/12643746 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3368111b90 0x7f336810a4a0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:48.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:48.981+0000 7f3367e00700 1 --2- 172.21.15.33:0/12643746 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3368112000 0x7f336810a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:48.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:48.981+0000 7f3367e00700 1 -- 172.21.15.33:0/12643746 >> 172.21.15.33:0/12643746 conn(0x7f336806e480 msgr2=0x7f336806e890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:33:48.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:48.981+0000 7f3367e00700 1 -- 172.21.15.33:0/12643746 shutdown_connections 2024-05-03T18:33:48.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:48.982+0000 7f3367e00700 1 -- 172.21.15.33:0/12643746 wait complete. 2024-05-03T18:33:48.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:48.985+0000 7f3367e00700 1 Processor -- start 2024-05-03T18:33:48.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:48.987+0000 7f3367e00700 1 -- start start 2024-05-03T18:33:48.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:48.987+0000 7f3367e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3368111b90 0x7f33681a86d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:48.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:48.987+0000 7f3367e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3368112000 0x7f33681a8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:48.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:48.987+0000 7f3367e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f336811cc80 0x7f33681acfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:48.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:48.987+0000 7f3367e00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f33681223c0 con 0x7f3368112000 2024-05-03T18:33:48.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:48.987+0000 7f3367e00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3368122240 con 0x7f336811cc80 2024-05-03T18:33:48.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:48.987+0000 7f3367e00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3368122540 con 0x7f3368111b90 2024-05-03T18:33:48.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:48.988+0000 7f3366a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3368111b90 0x7f33681a86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:48.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:48.988+0000 7f3366a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3368111b90 0x7f33681a86d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:40580/0 (socket says 172.21.15.33:40580) 2024-05-03T18:33:48.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:48.988+0000 7f3366a00700 1 -- 172.21.15.33:0/507773014 learned_addr learned my addr 172.21.15.33:0/507773014 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:33:48.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:48.989+0000 7f3366000700 1 --2- 172.21.15.33:0/507773014 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3368112000 0x7f33681a8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:48.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:48.988+0000 7f3367400700 1 --2- 172.21.15.33:0/507773014 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f336811cc80 0x7f33681acfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:48.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:48.990+0000 7f3366a00700 1 -- 172.21.15.33:0/507773014 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f336811cc80 msgr2=0x7f33681acfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:33:48.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:48.990+0000 7f3366a00700 1 --2- 172.21.15.33:0/507773014 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f336811cc80 0x7f33681acfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:48.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:48.990+0000 7f3366a00700 1 -- 172.21.15.33:0/507773014 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3368112000 msgr2=0x7f33681a8c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:33:48.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:48.990+0000 7f3366a00700 1 --2- 172.21.15.33:0/507773014 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3368112000 0x7f33681a8c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:48.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:48.990+0000 7f3366a00700 1 -- 172.21.15.33:0/507773014 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33681ad6e0 con 0x7f3368111b90 2024-05-03T18:33:48.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:48.991+0000 7f3366a00700 1 --2- 172.21.15.33:0/507773014 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3368111b90 0x7f33681a86d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3360005d10 tx=0x7f336000f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:33:48.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:48.991+0000 7f335f400700 1 -- 172.21.15.33:0/507773014 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3360027070 con 0x7f3368111b90 2024-05-03T18:33:48.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:48.991+0000 7f335f400700 1 -- 172.21.15.33:0/507773014 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f336000fbf0 con 0x7f3368111b90 2024-05-03T18:33:48.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:48.991+0000 7f3367e00700 1 -- 172.21.15.33:0/507773014 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33681ad970 con 0x7f3368111b90 2024-05-03T18:33:48.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:48.994+0000 7f335f400700 1 -- 172.21.15.33:0/507773014 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3360022e00 con 0x7f3368111b90 2024-05-03T18:33:48.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:48.996+0000 7f3367e00700 1 -- 172.21.15.33:0/507773014 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33681b5250 con 0x7f3368111b90 2024-05-03T18:33:49.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.000+0000 7f335f400700 1 -- 172.21.15.33:0/507773014 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f336002a070 con 0x7f3368111b90 2024-05-03T18:33:49.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.004+0000 7f335f400700 1 --2- 172.21.15.33:0/507773014 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f334c0b2a70 0x7f334c0b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:49.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.004+0000 7f335f400700 1 -- 172.21.15.33:0/507773014 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f33600e3810 con 0x7f3368111b90 2024-05-03T18:33:49.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.006+0000 7f3377eb3b80 1 Processor -- start 2024-05-03T18:33:49.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.006+0000 7f3366000700 1 --2- 172.21.15.33:0/507773014 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f334c0b2a70 0x7f334c0b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:49.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.007+0000 7f3377eb3b80 1 -- start start 2024-05-03T18:33:49.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.007+0000 7f3366000700 1 --2- 172.21.15.33:0/507773014 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f334c0b2a70 0x7f334c0b4f30 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3354005950 tx=0x7f33540058e0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:33:49.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.008+0000 7f3377eb3b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555be518ea40 0x555be518ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:49.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.008+0000 7f3377eb3b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555be518f360 0x555be5188fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:49.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.008+0000 7f3377eb3b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555be51895d0 0x555be5189a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:49.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.008+0000 7f3366a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555be518ea40 0x555be518ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:49.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.008+0000 7f3366a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555be518ea40 0x555be518ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:40588/0 (socket says 172.21.15.33:40588) 2024-05-03T18:33:49.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.009+0000 7f3377eb3b80 1 -- 172.21.15.33:0/2149690155 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x555be50dba20 con 0x555be51895d0 2024-05-03T18:33:49.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.009+0000 7f3377eb3b80 1 -- 172.21.15.33:0/2149690155 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x555be5079760 con 0x555be518f360 2024-05-03T18:33:49.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.009+0000 7f3377eb3b80 1 -- 172.21.15.33:0/2149690155 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x555be50d79e0 con 0x555be518ea40 2024-05-03T18:33:49.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.009+0000 7f3366a00700 1 -- 172.21.15.33:0/2149690155 learned_addr learned my addr 172.21.15.33:0/2149690155 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:33:49.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.009+0000 7f3366000700 1 --2- 172.21.15.33:0/2149690155 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555be518f360 0x555be5188fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:49.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.009+0000 7f3367400700 1 --2- 172.21.15.33:0/2149690155 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555be51895d0 0x555be5189a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:49.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.010+0000 7f3366a00700 1 -- 172.21.15.33:0/2149690155 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555be518f360 msgr2=0x555be5188fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:33:49.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.010+0000 7f3366a00700 1 --2- 172.21.15.33:0/2149690155 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555be518f360 0x555be5188fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:49.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.010+0000 7f3366a00700 1 -- 172.21.15.33:0/2149690155 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555be51895d0 msgr2=0x555be5189a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:33:49.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.010+0000 7f3366a00700 1 --2- 172.21.15.33:0/2149690155 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555be51895d0 0x555be5189a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:49.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.010+0000 7f3366a00700 1 -- 172.21.15.33:0/2149690155 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555be50288d0 con 0x555be518ea40 2024-05-03T18:33:49.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.010+0000 7f3366a00700 1 --2- 172.21.15.33:0/2149690155 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555be518ea40 0x555be518ee20 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f33600aa780 tx=0x7f33600aa300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b47adb8409dac679 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:33:49.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.010+0000 7f3353e00700 1 -- 172.21.15.33:0/2149690155 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33600b10e0 con 0x555be518ea40 2024-05-03T18:33:49.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.011+0000 7f3353e00700 1 -- 172.21.15.33:0/2149690155 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f33600aa7f0 con 0x555be518ea40 2024-05-03T18:33:49.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.012+0000 7f3377eb3b80 1 -- 172.21.15.33:0/2149690155 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555be518ea40 msgr2=0x555be518ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:33:49.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.012+0000 7f3353e00700 1 -- 172.21.15.33:0/2149690155 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33600b10e0 con 0x555be518ea40 2024-05-03T18:33:49.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.012+0000 7f3377eb3b80 1 --2- 172.21.15.33:0/2149690155 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555be518ea40 0x555be518ee20 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f33600aa780 tx=0x7f33600aa300 comp rx=0 tx=0).stop 2024-05-03T18:33:49.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.012+0000 7f3377eb3b80 1 -- 172.21.15.33:0/2149690155 shutdown_connections 2024-05-03T18:33:49.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.012+0000 7f3377eb3b80 1 --2- 172.21.15.33:0/2149690155 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555be51895d0 0x555be5189a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:49.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.012+0000 7f3377eb3b80 1 --2- 172.21.15.33:0/2149690155 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555be518ea40 0x555be518ee20 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:49.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.012+0000 7f3377eb3b80 1 --2- 172.21.15.33:0/2149690155 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555be518f360 0x555be5188fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:49.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.012+0000 7f3377eb3b80 1 -- 172.21.15.33:0/2149690155 >> 172.21.15.33:0/2149690155 conn(0x555be50b7b60 msgr2=0x555be51839d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:33:49.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.013+0000 7f3377eb3b80 1 -- 172.21.15.33:0/2149690155 shutdown_connections 2024-05-03T18:33:49.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.013+0000 7f3377eb3b80 1 -- 172.21.15.33:0/2149690155 wait complete. 2024-05-03T18:33:49.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.013+0000 7f3377eb3b80 1 -- start start 2024-05-03T18:33:49.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.013+0000 7f3377eb3b80 1 Processor -- start 2024-05-03T18:33:49.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.015+0000 7f3377eb3b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555be51895d0 0x555be5195050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:49.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.015+0000 7f3377eb3b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555be518ea40 0x555be51923f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:49.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.015+0000 7f3377eb3b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555be518f360 0x555be5192930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:49.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.016+0000 7f3377eb3b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x555be5195b90 con 0x555be51895d0 2024-05-03T18:33:49.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.016+0000 7f3377eb3b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x555be5041b20 con 0x555be518f360 2024-05-03T18:33:49.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.016+0000 7f3377eb3b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x555be5195d10 con 0x555be518ea40 2024-05-03T18:33:49.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.016+0000 7f3377eb3b80 10 client.0 _ll_register_callbacks cb 0x555be5082c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:33:49.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.016+0000 7f3366a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555be51895d0 0x555be5195050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:49.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.016+0000 7f3367400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555be518f360 0x555be5192930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:49.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.016+0000 7f3366000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555be518ea40 0x555be51923f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:49.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.016+0000 7f3366000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555be518ea40 0x555be51923f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:40602/0 (socket says 172.21.15.33:40602) 2024-05-03T18:33:49.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.016+0000 7f3366000700 1 -- 172.21.15.33:0/1405815780 learned_addr learned my addr 172.21.15.33:0/1405815780 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:33:49.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.017+0000 7f3366000700 1 -- 172.21.15.33:0/1405815780 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555be518f360 msgr2=0x555be5192930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:33:49.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.017+0000 7f3366000700 1 --2- 172.21.15.33:0/1405815780 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555be518f360 0x555be5192930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:49.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.017+0000 7f3366000700 1 -- 172.21.15.33:0/1405815780 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555be51895d0 msgr2=0x555be5195050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:33:49.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.017+0000 7f3366000700 1 --2- 172.21.15.33:0/1405815780 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555be51895d0 0x555be5195050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:49.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.017+0000 7f3366000700 1 -- 172.21.15.33:0/1405815780 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555be50a1420 con 0x555be518ea40 2024-05-03T18:33:49.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.017+0000 7f3367400700 1 --2- 172.21.15.33:0/1405815780 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555be518f360 0x555be5192930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:33:49.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.017+0000 7f3366a00700 1 --2- 172.21.15.33:0/1405815780 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555be51895d0 0x555be5195050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:33:49.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.018+0000 7f3366000700 1 --2- 172.21.15.33:0/1405815780 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555be518ea40 0x555be51923f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f335400fed0 tx=0x7f3354004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ec0ea955bd019fa server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:33:49.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.018+0000 7f3350c00700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-03T18:33:49.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.018+0000 7f3350c00700 1 -- 172.21.15.33:0/1405815780 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f335401f630 con 0x555be518ea40 2024-05-03T18:33:49.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.018+0000 7f3350c00700 1 -- 172.21.15.33:0/1405815780 <== mon.1 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f335401f7d0 con 0x555be518ea40 2024-05-03T18:33:49.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.018+0000 7f3377eb3b80 1 -- 172.21.15.33:0/1405815780 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33600aa7f0 con 0x555be518ea40 2024-05-03T18:33:49.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.019+0000 7f3350c00700 1 -- 172.21.15.33:0/1405815780 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f335401f9d0 con 0x555be518ea40 2024-05-03T18:33:49.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.019+0000 7f3377eb3b80 10 client.25774 fetch_fsmap learned FSMap version 21 2024-05-03T18:33:49.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.019+0000 7f3377eb3b80 1 -- 172.21.15.33:0/1405815780 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x555be51a0420 con 0x555be518ea40 2024-05-03T18:33:49.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.019+0000 7f3350c00700 1 -- 172.21.15.33:0/1405815780 <== mon.1 v2:172.21.15.150:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f33600aa7f0 con 0x555be518ea40 2024-05-03T18:33:49.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.020+0000 7f3350c00700 1 -- 172.21.15.33:0/1405815780 <== mon.1 v2:172.21.15.150:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f33540319b0 con 0x555be518ea40 2024-05-03T18:33:49.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.020+0000 7f3377eb3b80 10 client.25774 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:33:49.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.020+0000 7f3377eb3b80 10 client.25774 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:33:49.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.020+0000 7f3377eb3b80 20 client.25774 populate_metadata read hostname 'smithi033' 2024-05-03T18:33:49.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.020+0000 7f3377eb3b80 1 --2- 172.21.15.33:0/1405815780 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x555be51a2a60 0x555be51a4e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:49.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.020+0000 7f3377eb3b80 4 client.25774 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:33:49.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.020+0000 7f3377eb3b80 1 -- 172.21.15.33:0/1405815780 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555be51561d0 con 0x555be51a2a60 2024-05-03T18:33:49.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.020+0000 7f3366a00700 1 --2- 172.21.15.33:0/1405815780 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x555be51a2a60 0x555be51a4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:49.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.021+0000 7f3350c00700 10 client.25774 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:33:49.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.022+0000 7f3366a00700 1 --2- 172.21.15.33:0/1405815780 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x555be51a2a60 0x555be51a4e50 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e3a964877d8542e server_cookie=946a1669b0251c6c in_seq=0 out_seq=0 2024-05-03T18:33:49.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.023+0000 7f3350c00700 1 -- 172.21.15.33:0/1405815780 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555be51561d0 con 0x555be51a2a60 2024-05-03T18:33:49.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.023+0000 7f3350c00700 10 client.25774 handle_command_reply: tid=0 2024-05-03T18:33:49.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.023+0000 7f3377eb3b80 1 client.25774 shutdown 2024-05-03T18:33:49.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.023+0000 7f3377eb3b80 1 -- 172.21.15.33:0/1405815780 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555be518ea40 msgr2=0x555be51923f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:33:49.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.023+0000 7f3377eb3b80 1 --2- 172.21.15.33:0/1405815780 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555be518ea40 0x555be51923f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f335400fed0 tx=0x7f3354004750 comp rx=0 tx=0).stop 2024-05-03T18:33:49.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.023+0000 7f3377eb3b80 1 -- 172.21.15.33:0/1405815780 shutdown_connections 2024-05-03T18:33:49.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.023+0000 7f3377eb3b80 1 --2- 172.21.15.33:0/1405815780 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x555be51a2a60 0x555be51a4e50 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:49.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.023+0000 7f3377eb3b80 1 --2- 172.21.15.33:0/1405815780 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555be518f360 0x555be5192930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:49.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.023+0000 7f3377eb3b80 1 --2- 172.21.15.33:0/1405815780 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555be51895d0 0x555be5195050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:49.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.023+0000 7f3377eb3b80 1 --2- 172.21.15.33:0/1405815780 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555be518ea40 0x555be51923f0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:49.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.023+0000 7f3377eb3b80 1 -- 172.21.15.33:0/1405815780 >> 172.21.15.33:0/1405815780 conn(0x555be50b7b60 msgr2=0x555be5183080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:33:49.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.024+0000 7f3350c00700 0 client.25774 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:33:49.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.024+0000 7f3377eb3b80 1 -- 172.21.15.33:0/1405815780 shutdown_connections 2024-05-03T18:33:49.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.024+0000 7f3377eb3b80 1 -- 172.21.15.33:0/1405815780 wait complete. 2024-05-03T18:33:49.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.024+0000 7f3377eb3b80 20 client.25774 trim_cache size 0 max 16384 2024-05-03T18:33:49.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.043+0000 7f3377eb3b80 1 Processor -- start 2024-05-03T18:33:49.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.043+0000 7f3377eb3b80 1 -- start start 2024-05-03T18:33:49.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.044+0000 7f3377eb3b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555be51895d0 0x555be5188fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:49.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.044+0000 7f3377eb3b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555be518ea40 0x555be519ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:49.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.044+0000 7f3377eb3b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555be518f360 0x555be5190ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:49.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.044+0000 7f3377eb3b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f33681bbe60 con 0x555be518ea40 2024-05-03T18:33:49.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.044+0000 7f3377eb3b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f33681be6e0 con 0x555be51895d0 2024-05-03T18:33:49.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.044+0000 7f3377eb3b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f33681bc4d0 con 0x555be518f360 2024-05-03T18:33:49.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.044+0000 7f3366000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555be518ea40 0x555be519ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:49.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.044+0000 7f3366000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555be518ea40 0x555be519ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54910/0 (socket says 172.21.15.33:54910) 2024-05-03T18:33:49.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.044+0000 7f3366000700 1 -- 172.21.15.33:0/4042748152 learned_addr learned my addr 172.21.15.33:0/4042748152 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:33:49.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.044+0000 7f3366a00700 1 --2- 172.21.15.33:0/4042748152 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555be51895d0 0x555be5188fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:49.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.044+0000 7f3367400700 1 --2- 172.21.15.33:0/4042748152 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555be518f360 0x555be5190ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:49.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.045+0000 7f3366000700 1 -- 172.21.15.33:0/4042748152 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555be518f360 msgr2=0x555be5190ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:33:49.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.045+0000 7f3366000700 1 --2- 172.21.15.33:0/4042748152 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555be518f360 0x555be5190ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:49.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.045+0000 7f3366000700 1 -- 172.21.15.33:0/4042748152 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555be51895d0 msgr2=0x555be5188fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:33:49.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.045+0000 7f3366000700 1 --2- 172.21.15.33:0/4042748152 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555be51895d0 0x555be5188fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:49.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.045+0000 7f3366000700 1 -- 172.21.15.33:0/4042748152 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555be5190960 con 0x555be518ea40 2024-05-03T18:33:49.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.045+0000 7f3366000700 1 --2- 172.21.15.33:0/4042748152 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555be518ea40 0x555be519ec30 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f335400b570 tx=0x7f335400b9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bac877329315b0e server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:33:49.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.046+0000 7f3353400700 1 -- 172.21.15.33:0/4042748152 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3354041070 con 0x555be518ea40 2024-05-03T18:33:49.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.046+0000 7f3353400700 1 -- 172.21.15.33:0/4042748152 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3354027600 con 0x555be518ea40 2024-05-03T18:33:49.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.046+0000 7f3353400700 1 -- 172.21.15.33:0/4042748152 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3354036070 con 0x555be518ea40 2024-05-03T18:33:49.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.047+0000 7f3377eb3b80 1 -- 172.21.15.33:0/4042748152 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555be518ea40 msgr2=0x555be519ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:33:49.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.047+0000 7f3377eb3b80 1 --2- 172.21.15.33:0/4042748152 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555be518ea40 0x555be519ec30 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f335400b570 tx=0x7f335400b9e0 comp rx=0 tx=0).stop 2024-05-03T18:33:49.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.047+0000 7f3377eb3b80 1 -- 172.21.15.33:0/4042748152 shutdown_connections 2024-05-03T18:33:49.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.047+0000 7f3377eb3b80 1 --2- 172.21.15.33:0/4042748152 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555be518f360 0x555be5190ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:49.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.047+0000 7f3377eb3b80 1 --2- 172.21.15.33:0/4042748152 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555be51895d0 0x555be5188fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:49.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.047+0000 7f3377eb3b80 1 --2- 172.21.15.33:0/4042748152 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555be518ea40 0x555be519ec30 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:49.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.047+0000 7f3377eb3b80 1 -- 172.21.15.33:0/4042748152 >> 172.21.15.33:0/4042748152 conn(0x555be50b7b60 msgr2=0x555be51923b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:33:49.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.047+0000 7f3377eb3b80 1 -- 172.21.15.33:0/4042748152 shutdown_connections 2024-05-03T18:33:49.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.047+0000 7f3377eb3b80 1 -- 172.21.15.33:0/4042748152 wait complete. 2024-05-03T18:33:49.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.047+0000 7f3377eb3b80 1 -- start start 2024-05-03T18:33:49.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.048+0000 7f3377eb3b80 1 Processor -- start 2024-05-03T18:33:49.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.048+0000 7f3377eb3b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555be51895d0 0x555be51a9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:49.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.048+0000 7f3377eb3b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555be518ea40 0x555be51aa300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:49.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.048+0000 7f3377eb3b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555be518f360 0x555be51aa840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:49.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.049+0000 7f3377eb3b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f33681bcba0 con 0x555be518f360 2024-05-03T18:33:49.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.049+0000 7f3377eb3b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f33681be010 con 0x555be518ea40 2024-05-03T18:33:49.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.049+0000 7f3377eb3b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f33681bd270 con 0x555be51895d0 2024-05-03T18:33:49.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.049+0000 7f3377eb3b80 10 client.0 _ll_register_callbacks cb 0x555be5082c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:33:49.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.049+0000 7f3366a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555be51895d0 0x555be51a9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:49.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.049+0000 7f3367400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555be518f360 0x555be51aa840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:49.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.049+0000 7f3366a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555be51895d0 0x555be51a9dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:40632/0 (socket says 172.21.15.33:40632) 2024-05-03T18:33:49.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.049+0000 7f3366a00700 1 -- 172.21.15.33:0/216864784 learned_addr learned my addr 172.21.15.33:0/216864784 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:33:49.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.049+0000 7f3366000700 1 --2- 172.21.15.33:0/216864784 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555be518ea40 0x555be51aa300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:49.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.050+0000 7f3367400700 1 -- 172.21.15.33:0/216864784 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555be51895d0 msgr2=0x555be51a9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:33:49.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.050+0000 7f3367400700 1 --2- 172.21.15.33:0/216864784 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555be51895d0 0x555be51a9dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:49.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.050+0000 7f3367400700 1 -- 172.21.15.33:0/216864784 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555be518ea40 msgr2=0x555be51aa300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:33:49.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.050+0000 7f3367400700 1 --2- 172.21.15.33:0/216864784 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555be518ea40 0x555be51aa300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:49.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.050+0000 7f3367400700 1 -- 172.21.15.33:0/216864784 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555be520c2a0 con 0x555be518f360 2024-05-03T18:33:49.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.050+0000 7f3367400700 1 --2- 172.21.15.33:0/216864784 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555be518f360 0x555be51aa840 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x555be50d7640 tx=0x7f335800af80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3bdecea0525f562 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:33:49.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.050+0000 7f3350c00700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:33:49.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.050+0000 7f3350c00700 1 -- 172.21.15.33:0/216864784 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f335801e070 con 0x555be518f360 2024-05-03T18:33:49.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.051+0000 7f3377eb3b80 1 -- 172.21.15.33:0/216864784 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3354027600 con 0x555be518f360 2024-05-03T18:33:49.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.051+0000 7f3350c00700 1 -- 172.21.15.33:0/216864784 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f3358015940 con 0x555be518f360 2024-05-03T18:33:49.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.051+0000 7f3350c00700 1 -- 172.21.15.33:0/216864784 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f3358015b40 con 0x555be518f360 2024-05-03T18:33:49.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.051+0000 7f3377eb3b80 10 client.16089 fetch_fsmap learned FSMap version 21 2024-05-03T18:33:49.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.052+0000 7f3377eb3b80 1 -- 172.21.15.33:0/216864784 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x555be520d510 con 0x555be518f360 2024-05-03T18:33:49.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.052+0000 7f3350c00700 1 -- 172.21.15.33:0/216864784 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3354027600 con 0x555be518f360 2024-05-03T18:33:49.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.052+0000 7f3350c00700 1 -- 172.21.15.33:0/216864784 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f33580223b0 con 0x555be518f360 2024-05-03T18:33:49.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.052+0000 7f3377eb3b80 10 client.16089 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:33:49.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.052+0000 7f3377eb3b80 10 client.16089 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:33:49.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.052+0000 7f3377eb3b80 20 client.16089 populate_metadata read hostname 'smithi033' 2024-05-03T18:33:49.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.052+0000 7f3377eb3b80 1 --2- 172.21.15.33:0/216864784 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x555be520dbd0 0x555be520fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:49.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.052+0000 7f3377eb3b80 4 client.16089 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-03T18:33:49.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.052+0000 7f3377eb3b80 1 -- 172.21.15.33:0/216864784 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555be50e9400 con 0x555be520dbd0 2024-05-03T18:33:49.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.053+0000 7f3366a00700 1 --2- 172.21.15.33:0/216864784 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x555be520dbd0 0x555be520fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:49.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.053+0000 7f3366a00700 1 --2- 172.21.15.33:0/216864784 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x555be520dbd0 0x555be520fff0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8929c9a208448711 server_cookie=dd699db38c3e25a3 in_seq=0 out_seq=0 2024-05-03T18:33:49.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.054+0000 7f3350c00700 10 client.16089 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:33:49.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.054+0000 7f3350c00700 1 -- 172.21.15.33:0/216864784 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555be50e9400 con 0x555be520dbd0 2024-05-03T18:33:49.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.054+0000 7f3350c00700 10 client.16089 handle_command_reply: tid=0 2024-05-03T18:33:49.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.054+0000 7f3377eb3b80 1 client.16089 shutdown 2024-05-03T18:33:49.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.054+0000 7f3377eb3b80 1 -- 172.21.15.33:0/216864784 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555be518f360 msgr2=0x555be51aa840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:33:49.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.054+0000 7f3377eb3b80 1 --2- 172.21.15.33:0/216864784 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555be518f360 0x555be51aa840 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x555be50d7640 tx=0x7f335800af80 comp rx=0 tx=0).stop 2024-05-03T18:33:49.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.054+0000 7f3377eb3b80 1 -- 172.21.15.33:0/216864784 shutdown_connections 2024-05-03T18:33:49.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.054+0000 7f3377eb3b80 1 --2- 172.21.15.33:0/216864784 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x555be520dbd0 0x555be520fff0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:49.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.055+0000 7f3377eb3b80 1 --2- 172.21.15.33:0/216864784 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555be518f360 0x555be51aa840 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:49.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.055+0000 7f3377eb3b80 1 --2- 172.21.15.33:0/216864784 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555be51895d0 0x555be51a9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:49.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.055+0000 7f3377eb3b80 1 --2- 172.21.15.33:0/216864784 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555be518ea40 0x555be51aa300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:49.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.055+0000 7f3377eb3b80 1 -- 172.21.15.33:0/216864784 >> 172.21.15.33:0/216864784 conn(0x555be50b7b60 msgr2=0x555be5190bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:33:49.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.055+0000 7f3350c00700 0 client.16089 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:33:49.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.055+0000 7f3377eb3b80 1 -- 172.21.15.33:0/216864784 shutdown_connections 2024-05-03T18:33:49.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.055+0000 7f3377eb3b80 1 -- 172.21.15.33:0/216864784 wait complete. 2024-05-03T18:33:49.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.055+0000 7f3377eb3b80 20 client.16089 trim_cache size 0 max 16384 2024-05-03T18:33:49.057 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-03T18:33:49.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.056+0000 7f3367e00700 1 -- 172.21.15.33:0/507773014 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f334c0b2a70 msgr2=0x7f334c0b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:33:49.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.056+0000 7f3367e00700 1 --2- 172.21.15.33:0/507773014 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f334c0b2a70 0x7f334c0b4f30 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3354005950 tx=0x7f33540058e0 comp rx=0 tx=0).stop 2024-05-03T18:33:49.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.056+0000 7f3367e00700 1 -- 172.21.15.33:0/507773014 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3368111b90 msgr2=0x7f33681a86d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:33:49.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.056+0000 7f3367e00700 1 --2- 172.21.15.33:0/507773014 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3368111b90 0x7f33681a86d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3360005d10 tx=0x7f336000f690 comp rx=0 tx=0).stop 2024-05-03T18:33:49.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.056+0000 7f3367e00700 1 -- 172.21.15.33:0/507773014 shutdown_connections 2024-05-03T18:33:49.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.056+0000 7f3367e00700 1 --2- 172.21.15.33:0/507773014 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f334c0b2a70 0x7f334c0b4f30 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:49.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.056+0000 7f3367e00700 1 --2- 172.21.15.33:0/507773014 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f336811cc80 0x7f33681acfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:49.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.056+0000 7f3367e00700 1 --2- 172.21.15.33:0/507773014 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3368111b90 0x7f33681a86d0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:49.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.056+0000 7f3367e00700 1 --2- 172.21.15.33:0/507773014 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3368112000 0x7f33681a8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:49.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.056+0000 7f3367e00700 1 -- 172.21.15.33:0/507773014 >> 172.21.15.33:0/507773014 conn(0x7f336806e480 msgr2=0x7f336810f7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:33:49.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.057+0000 7f3367e00700 1 -- 172.21.15.33:0/507773014 shutdown_connections 2024-05-03T18:33:49.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:49.057+0000 7f3367e00700 1 -- 172.21.15.33:0/507773014 wait complete. 2024-05-03T18:33:50.110 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 155.0 s, 624.6 tps, lat 177.099 ms stddev 307.544 2024-05-03T18:33:50.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:33:50 smithi150 ceph-mon[32492]: pgmap v202: 129 pgs: 129 active+clean; 2.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 12 MiB/s wr, 472 op/s 2024-05-03T18:33:50.482 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:33:50 smithi033 ceph-mon[28330]: pgmap v202: 129 pgs: 129 active+clean; 2.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 12 MiB/s wr, 472 op/s 2024-05-03T18:33:50.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:33:50 smithi099 ceph-mon[31658]: pgmap v202: 129 pgs: 129 active+clean; 2.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 12 MiB/s wr, 472 op/s 2024-05-03T18:33:51.544 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-03T18:33:51.545 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-03T18:33:52.322 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:33:52.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:33:52 smithi150 ceph-mon[32492]: pgmap v203: 129 pgs: 129 active+clean; 2.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 11 MiB/s wr, 429 op/s 2024-05-03T18:33:52.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:33:52 smithi033 ceph-mon[28330]: pgmap v203: 129 pgs: 129 active+clean; 2.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 11 MiB/s wr, 429 op/s 2024-05-03T18:33:52.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:33:52 smithi099 ceph-mon[31658]: pgmap v203: 129 pgs: 129 active+clean; 2.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 11 MiB/s wr, 429 op/s 2024-05-03T18:33:54.059 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:33:54 smithi033 ceph-mon[28330]: pgmap v204: 129 pgs: 129 active+clean; 2.2 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 18 MiB/s wr, 741 op/s 2024-05-03T18:33:54.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:33:54 smithi150 ceph-mon[32492]: pgmap v204: 129 pgs: 129 active+clean; 2.2 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 18 MiB/s wr, 741 op/s 2024-05-03T18:33:54.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:33:54 smithi099 ceph-mon[31658]: pgmap v204: 129 pgs: 129 active+clean; 2.2 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 18 MiB/s wr, 741 op/s 2024-05-03T18:33:54.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.849+0000 7f321a400700 1 -- 172.21.15.33:0/2102078858 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f321406bb40 msgr2=0x7f3214110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:33:54.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.849+0000 7f321a400700 1 --2- 172.21.15.33:0/2102078858 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f321406bb40 0x7f3214110330 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f320c00b0a0 tx=0x7f320c00b3b0 comp rx=0 tx=0).stop 2024-05-03T18:33:54.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.850+0000 7f321a400700 1 -- 172.21.15.33:0/2102078858 shutdown_connections 2024-05-03T18:33:54.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.850+0000 7f321a400700 1 --2- 172.21.15.33:0/2102078858 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f321406bb40 0x7f3214110330 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:54.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.850+0000 7f321a400700 1 --2- 172.21.15.33:0/2102078858 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3214109510 0x7f32141098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:54.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.850+0000 7f321a400700 1 --2- 172.21.15.33:0/2102078858 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f321406b180 0x7f321406b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:54.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.850+0000 7f321a400700 1 -- 172.21.15.33:0/2102078858 >> 172.21.15.33:0/2102078858 conn(0x7f3214079350 msgr2=0x7f3214079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:33:54.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.850+0000 7f321a400700 1 -- 172.21.15.33:0/2102078858 shutdown_connections 2024-05-03T18:33:54.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.850+0000 7f321a400700 1 -- 172.21.15.33:0/2102078858 wait complete. 2024-05-03T18:33:54.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.852+0000 7f321a400700 1 Processor -- start 2024-05-03T18:33:54.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.855+0000 7f321a400700 1 -- start start 2024-05-03T18:33:54.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.855+0000 7f321a400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f321406b180 0x7f321419c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:54.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.855+0000 7f321a400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f321406bb40 0x7f321419c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:54.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.855+0000 7f321a400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3214109510 0x7f321410e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:54.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.855+0000 7f321a400700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3214113570 con 0x7f321406b180 2024-05-03T18:33:54.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.855+0000 7f321a400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f32141133f0 con 0x7f3214109510 2024-05-03T18:33:54.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.855+0000 7f321a400700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f32141136f0 con 0x7f321406bb40 2024-05-03T18:33:54.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.856+0000 7f3213e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3214109510 0x7f321410e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:54.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.856+0000 7f3213e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3214109510 0x7f321410e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:43400/0 (socket says 172.21.15.33:43400) 2024-05-03T18:33:54.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.856+0000 7f3213e00700 1 -- 172.21.15.33:0/429460201 learned_addr learned my addr 172.21.15.33:0/429460201 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:33:54.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.857+0000 7f3213e00700 1 -- 172.21.15.33:0/429460201 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f321406bb40 msgr2=0x7f321419c9b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-03T18:33:54.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.857+0000 7f3213e00700 1 --2- 172.21.15.33:0/429460201 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f321406bb40 0x7f321419c9b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:54.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.857+0000 7f3213e00700 1 -- 172.21.15.33:0/429460201 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f321406b180 msgr2=0x7f321419c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:33:54.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.857+0000 7f3213e00700 1 --2- 172.21.15.33:0/429460201 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f321406b180 0x7f321419c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:54.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.857+0000 7f3213e00700 1 -- 172.21.15.33:0/429460201 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f321410eae0 con 0x7f3214109510 2024-05-03T18:33:54.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.858+0000 7f3213e00700 1 --2- 172.21.15.33:0/429460201 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3214109510 0x7f321410e1e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f320c0060b0 tx=0x7f320c007770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:33:54.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.858+0000 7f3210c00700 1 -- 172.21.15.33:0/429460201 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f320c01f030 con 0x7f3214109510 2024-05-03T18:33:54.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.858+0000 7f321a400700 1 -- 172.21.15.33:0/429460201 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f321410ed70 con 0x7f3214109510 2024-05-03T18:33:54.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.858+0000 7f321a400700 1 -- 172.21.15.33:0/429460201 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32141a0180 con 0x7f3214109510 2024-05-03T18:33:54.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.859+0000 7f3210c00700 1 -- 172.21.15.33:0/429460201 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f320c007c10 con 0x7f3214109510 2024-05-03T18:33:54.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.859+0000 7f3210c00700 1 -- 172.21.15.33:0/429460201 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f320c01b760 con 0x7f3214109510 2024-05-03T18:33:54.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.868+0000 7f3224dabb80 1 Processor -- start 2024-05-03T18:33:54.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.870+0000 7f3210c00700 1 -- 172.21.15.33:0/429460201 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f320c022030 con 0x7f3214109510 2024-05-03T18:33:54.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.870+0000 7f3224dabb80 1 -- start start 2024-05-03T18:33:54.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.870+0000 7f3224dabb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cd12a0830 0x556cd12a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:54.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.870+0000 7f3224dabb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556cd12a1150 0x556cd129c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:54.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.870+0000 7f3224dabb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556cd129c650 0x556cd129cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:54.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.870+0000 7f3224dabb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556cd11eeaa0 con 0x556cd12a1150 2024-05-03T18:33:54.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.870+0000 7f3224dabb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x556cd118c7e0 con 0x556cd12a0830 2024-05-03T18:33:54.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.870+0000 7f3224dabb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x556cd11eaa60 con 0x556cd129c650 2024-05-03T18:33:54.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.871+0000 7f3213400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cd12a0830 0x556cd12a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:54.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.871+0000 7f3213400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cd12a0830 0x556cd12a0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:43402/0 (socket says 172.21.15.33:43402) 2024-05-03T18:33:54.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.871+0000 7f3213400700 1 -- 172.21.15.33:0/1097386327 learned_addr learned my addr 172.21.15.33:0/1097386327 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:33:54.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.871+0000 7f320b400700 1 --2- 172.21.15.33:0/1097386327 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556cd12a1150 0x556cd129c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:54.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.872+0000 7f3213400700 1 --2- 172.21.15.33:0/1097386327 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556cd129c650 0x556cd129cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:54.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.873+0000 7f3210c00700 1 --2- 172.21.15.33:0/429460201 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f31f80b2b30 0x7f31f80b4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:54.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.873+0000 7f3210c00700 1 -- 172.21.15.33:0/429460201 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f320c0e3450 con 0x7f3214109510 2024-05-03T18:33:54.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.873+0000 7f320b400700 1 -- 172.21.15.33:0/1097386327 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556cd129c650 msgr2=0x556cd129cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:33:54.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.873+0000 7f320b400700 1 --2- 172.21.15.33:0/1097386327 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556cd129c650 0x556cd129cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:54.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.873+0000 7f320b400700 1 -- 172.21.15.33:0/1097386327 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cd12a0830 msgr2=0x556cd12a0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:33:54.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.873+0000 7f320b400700 1 --2- 172.21.15.33:0/1097386327 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cd12a0830 0x556cd12a0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:54.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.873+0000 7f320b400700 1 -- 172.21.15.33:0/1097386327 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556cd113b950 con 0x556cd12a1150 2024-05-03T18:33:54.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.873+0000 7f3213400700 1 --2- 172.21.15.33:0/1097386327 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cd12a0830 0x556cd12a0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:33:54.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.874+0000 7f3213e00700 1 --2- 172.21.15.33:0/429460201 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f31f80b2b30 0x7f31f80b4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:54.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.874+0000 7f3213e00700 1 --2- 172.21.15.33:0/429460201 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f31f80b2b30 0x7f31f80b4ff0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f320c0a7640 tx=0x7f320c005040 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:33:54.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.875+0000 7f320b400700 1 --2- 172.21.15.33:0/1097386327 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556cd12a1150 0x556cd129c030 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f3204009b50 tx=0x7f3204009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3e618cdbf3861dd server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:33:54.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.875+0000 7f3208c00700 1 -- 172.21.15.33:0/1097386327 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3204004230 con 0x556cd12a1150 2024-05-03T18:33:54.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.876+0000 7f3208c00700 1 -- 172.21.15.33:0/1097386327 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f32040043d0 con 0x556cd12a1150 2024-05-03T18:33:54.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.876+0000 7f3208c00700 1 -- 172.21.15.33:0/1097386327 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3204004230 con 0x556cd12a1150 2024-05-03T18:33:54.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.877+0000 7f3224dabb80 1 -- 172.21.15.33:0/1097386327 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556cd12a1150 msgr2=0x556cd129c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:33:54.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.877+0000 7f3224dabb80 1 --2- 172.21.15.33:0/1097386327 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556cd12a1150 0x556cd129c030 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f3204009b50 tx=0x7f3204009e60 comp rx=0 tx=0).stop 2024-05-03T18:33:54.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.877+0000 7f3224dabb80 1 -- 172.21.15.33:0/1097386327 shutdown_connections 2024-05-03T18:33:54.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.877+0000 7f3224dabb80 1 --2- 172.21.15.33:0/1097386327 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556cd129c650 0x556cd129cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:54.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.878+0000 7f3224dabb80 1 --2- 172.21.15.33:0/1097386327 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cd12a0830 0x556cd12a0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:54.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.878+0000 7f3224dabb80 1 --2- 172.21.15.33:0/1097386327 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556cd12a1150 0x556cd129c030 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:54.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.878+0000 7f3224dabb80 1 -- 172.21.15.33:0/1097386327 >> 172.21.15.33:0/1097386327 conn(0x556cd11cabe0 msgr2=0x556cd1296870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:33:54.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.879+0000 7f3224dabb80 1 -- 172.21.15.33:0/1097386327 shutdown_connections 2024-05-03T18:33:54.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.879+0000 7f3224dabb80 1 -- 172.21.15.33:0/1097386327 wait complete. 2024-05-03T18:33:54.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.880+0000 7f3224dabb80 1 -- start start 2024-05-03T18:33:54.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.880+0000 7f3224dabb80 1 Processor -- start 2024-05-03T18:33:54.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.881+0000 7f3224dabb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cd129c650 0x556cd12a7f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:54.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.881+0000 7f3224dabb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556cd12a0830 0x556cd12a5310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:54.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.881+0000 7f3224dabb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556cd12a1150 0x556cd12a5850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:54.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.881+0000 7f3224dabb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556cd12a8d20 con 0x556cd12a0830 2024-05-03T18:33:54.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.881+0000 7f3224dabb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x556cd1154ba0 con 0x556cd129c650 2024-05-03T18:33:54.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.881+0000 7f3224dabb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x556cd12a8ea0 con 0x556cd12a1150 2024-05-03T18:33:54.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.881+0000 7f3224dabb80 10 client.0 _ll_register_callbacks cb 0x556cd1195cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:33:54.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.882+0000 7f3213400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cd129c650 0x556cd12a7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:54.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.882+0000 7f320b400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556cd12a0830 0x556cd12a5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:54.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.882+0000 7f3213400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cd129c650 0x556cd12a7f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:43412/0 (socket says 172.21.15.33:43412) 2024-05-03T18:33:54.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.882+0000 7f3213400700 1 -- 172.21.15.33:0/2973859394 learned_addr learned my addr 172.21.15.33:0/2973859394 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:33:54.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.882+0000 7f3213400700 1 --2- 172.21.15.33:0/2973859394 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556cd12a1150 0x556cd12a5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:54.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.883+0000 7f3213400700 1 -- 172.21.15.33:0/2973859394 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556cd12a1150 msgr2=0x556cd12a5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:33:54.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.883+0000 7f3213400700 1 --2- 172.21.15.33:0/2973859394 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556cd12a1150 0x556cd12a5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:54.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.883+0000 7f3213400700 1 -- 172.21.15.33:0/2973859394 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556cd12a0830 msgr2=0x556cd12a5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:33:54.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.883+0000 7f3213400700 1 --2- 172.21.15.33:0/2973859394 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556cd12a0830 0x556cd12a5310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:54.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.883+0000 7f3213400700 1 -- 172.21.15.33:0/2973859394 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556cd11b44a0 con 0x556cd129c650 2024-05-03T18:33:54.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.884+0000 7f3213400700 1 --2- 172.21.15.33:0/2973859394 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cd129c650 0x556cd12a7f70 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f31fc005920 tx=0x7f31fc014040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fbac4a52e9d9c948 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:33:54.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.884+0000 7f3201600700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-03T18:33:54.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.885+0000 7f3201600700 1 -- 172.21.15.33:0/2973859394 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31fc00bac0 con 0x556cd129c650 2024-05-03T18:33:54.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.885+0000 7f3201600700 1 -- 172.21.15.33:0/2973859394 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f31fc00bc60 con 0x556cd129c650 2024-05-03T18:33:54.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.885+0000 7f3224dabb80 1 -- 172.21.15.33:0/2973859394 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32040043d0 con 0x556cd129c650 2024-05-03T18:33:54.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.886+0000 7f3201600700 1 -- 172.21.15.33:0/2973859394 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f31fc00be60 con 0x556cd129c650 2024-05-03T18:33:54.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.886+0000 7f3224dabb80 10 client.35738 fetch_fsmap learned FSMap version 21 2024-05-03T18:33:54.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.886+0000 7f3224dabb80 1 -- 172.21.15.33:0/2973859394 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x556cd12b33c0 con 0x556cd129c650 2024-05-03T18:33:54.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.888+0000 7f3201600700 1 -- 172.21.15.33:0/2973859394 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f32040043d0 con 0x556cd129c650 2024-05-03T18:33:54.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.888+0000 7f3201600700 1 -- 172.21.15.33:0/2973859394 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f31fc020410 con 0x556cd129c650 2024-05-03T18:33:54.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.888+0000 7f3224dabb80 10 client.35738 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:33:54.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.888+0000 7f3224dabb80 10 client.35738 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:33:54.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.888+0000 7f3224dabb80 20 client.35738 populate_metadata read hostname 'smithi033' 2024-05-03T18:33:54.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.888+0000 7f3224dabb80 1 --2- 172.21.15.33:0/2973859394 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x556cd12b5950 0x556cd12b7d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:54.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.888+0000 7f3224dabb80 4 client.35738 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:33:54.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.888+0000 7f3224dabb80 1 -- 172.21.15.33:0/2973859394 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556cd1269250 con 0x556cd12b5950 2024-05-03T18:33:54.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.890+0000 7f320b400700 1 --2- 172.21.15.33:0/2973859394 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x556cd12b5950 0x556cd12b7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:54.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.892+0000 7f320b400700 1 --2- 172.21.15.33:0/2973859394 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x556cd12b5950 0x556cd12b7d40 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2943e553abfab243 server_cookie=ab0aec1487f97810 in_seq=0 out_seq=0 2024-05-03T18:33:54.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.892+0000 7f3201600700 10 client.35738 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:33:54.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.895+0000 7f3201600700 1 -- 172.21.15.33:0/2973859394 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556cd1269250 con 0x556cd12b5950 2024-05-03T18:33:54.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.895+0000 7f3201600700 10 client.35738 handle_command_reply: tid=0 2024-05-03T18:33:54.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.896+0000 7f3224dabb80 1 client.35738 shutdown 2024-05-03T18:33:54.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.897+0000 7f3224dabb80 1 -- 172.21.15.33:0/2973859394 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cd129c650 msgr2=0x556cd12a7f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:33:54.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.897+0000 7f3224dabb80 1 --2- 172.21.15.33:0/2973859394 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cd129c650 0x556cd12a7f70 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f31fc005920 tx=0x7f31fc014040 comp rx=0 tx=0).stop 2024-05-03T18:33:54.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.897+0000 7f3224dabb80 1 -- 172.21.15.33:0/2973859394 shutdown_connections 2024-05-03T18:33:54.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.897+0000 7f3224dabb80 1 --2- 172.21.15.33:0/2973859394 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x556cd12b5950 0x556cd12b7d40 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:54.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.897+0000 7f3224dabb80 1 --2- 172.21.15.33:0/2973859394 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556cd12a1150 0x556cd12a5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:54.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.897+0000 7f3224dabb80 1 --2- 172.21.15.33:0/2973859394 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cd129c650 0x556cd12a7f70 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:54.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.897+0000 7f3224dabb80 1 --2- 172.21.15.33:0/2973859394 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556cd12a0830 0x556cd12a5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:54.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.897+0000 7f3224dabb80 1 -- 172.21.15.33:0/2973859394 >> 172.21.15.33:0/2973859394 conn(0x556cd11cabe0 msgr2=0x556cd12a1e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:33:54.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.898+0000 7f3201600700 0 client.35738 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:33:54.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.898+0000 7f3224dabb80 1 -- 172.21.15.33:0/2973859394 shutdown_connections 2024-05-03T18:33:54.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.899+0000 7f3224dabb80 1 -- 172.21.15.33:0/2973859394 wait complete. 2024-05-03T18:33:54.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.899+0000 7f3224dabb80 20 client.35738 trim_cache size 0 max 16384 2024-05-03T18:33:54.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.941+0000 7f3224dabb80 1 Processor -- start 2024-05-03T18:33:54.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.941+0000 7f3224dabb80 1 -- start start 2024-05-03T18:33:54.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.942+0000 7f3224dabb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556cd11cabe0 0x556cd12b1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:54.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.942+0000 7f3224dabb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556cd129c650 0x556cd12966e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:54.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.942+0000 7f3224dabb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cd12a0830 0x556cd12bd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:54.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.942+0000 7f3224dabb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f32141bc100 con 0x556cd129c650 2024-05-03T18:33:54.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.942+0000 7f3224dabb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f32141be9c0 con 0x556cd12a0830 2024-05-03T18:33:54.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.942+0000 7f3224dabb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f32141bc7b0 con 0x556cd11cabe0 2024-05-03T18:33:54.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.943+0000 7f320b400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556cd129c650 0x556cd12966e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:54.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.943+0000 7f3213400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556cd11cabe0 0x556cd12b1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:54.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.943+0000 7f320b400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556cd129c650 0x556cd12966e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54978/0 (socket says 172.21.15.33:54978) 2024-05-03T18:33:54.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.943+0000 7f320b400700 1 -- 172.21.15.33:0/137947969 learned_addr learned my addr 172.21.15.33:0/137947969 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:33:54.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.943+0000 7f3213400700 1 --2- 172.21.15.33:0/137947969 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cd12a0830 0x556cd12bd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:54.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.943+0000 7f320b400700 1 -- 172.21.15.33:0/137947969 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556cd11cabe0 msgr2=0x556cd12b1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:33:54.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.943+0000 7f320b400700 1 --2- 172.21.15.33:0/137947969 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556cd11cabe0 0x556cd12b1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:54.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.943+0000 7f320b400700 1 -- 172.21.15.33:0/137947969 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cd12a0830 msgr2=0x556cd12bd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:33:54.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.943+0000 7f320b400700 1 --2- 172.21.15.33:0/137947969 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cd12a0830 0x556cd12bd550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:54.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.943+0000 7f320b400700 1 -- 172.21.15.33:0/137947969 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556cd12a3880 con 0x556cd129c650 2024-05-03T18:33:54.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.945+0000 7f320b400700 1 --2- 172.21.15.33:0/137947969 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556cd129c650 0x556cd12966e0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f320400e630 tx=0x7f32040058e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8077715841c3d44 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:33:54.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.946+0000 7f3203e00700 1 -- 172.21.15.33:0/137947969 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f320400e880 con 0x556cd129c650 2024-05-03T18:33:54.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.946+0000 7f3203e00700 1 -- 172.21.15.33:0/137947969 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3204028070 con 0x556cd129c650 2024-05-03T18:33:54.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.946+0000 7f3203e00700 1 -- 172.21.15.33:0/137947969 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3204003a70 con 0x556cd129c650 2024-05-03T18:33:54.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.947+0000 7f3224dabb80 1 -- 172.21.15.33:0/137947969 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556cd129c650 msgr2=0x556cd12966e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:33:54.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.947+0000 7f3224dabb80 1 --2- 172.21.15.33:0/137947969 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556cd129c650 0x556cd12966e0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f320400e630 tx=0x7f32040058e0 comp rx=0 tx=0).stop 2024-05-03T18:33:54.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.947+0000 7f3224dabb80 1 -- 172.21.15.33:0/137947969 shutdown_connections 2024-05-03T18:33:54.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.947+0000 7f3224dabb80 1 --2- 172.21.15.33:0/137947969 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cd12a0830 0x556cd12bd550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:54.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.947+0000 7f3224dabb80 1 --2- 172.21.15.33:0/137947969 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556cd11cabe0 0x556cd12b1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:54.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.947+0000 7f3224dabb80 1 --2- 172.21.15.33:0/137947969 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556cd129c650 0x556cd12966e0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:54.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.947+0000 7f3224dabb80 1 -- 172.21.15.33:0/137947969 >> 172.21.15.33:0/137947969 conn(0x556cd12a6200 msgr2=0x556cd129bc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:33:54.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.948+0000 7f3224dabb80 1 -- 172.21.15.33:0/137947969 shutdown_connections 2024-05-03T18:33:54.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.948+0000 7f3224dabb80 1 -- 172.21.15.33:0/137947969 wait complete. 2024-05-03T18:33:54.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.949+0000 7f3224dabb80 1 -- start start 2024-05-03T18:33:54.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.949+0000 7f3224dabb80 1 Processor -- start 2024-05-03T18:33:54.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.950+0000 7f3224dabb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556cd11cabe0 0x556cd12a2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:54.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.950+0000 7f3224dabb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cd129c650 0x556cd12ae160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:54.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.950+0000 7f3224dabb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556cd12a0830 0x556cd12ae6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:54.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.950+0000 7f3224dabb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f32141bce80 con 0x556cd12a0830 2024-05-03T18:33:54.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.950+0000 7f3224dabb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f32141be2f0 con 0x556cd129c650 2024-05-03T18:33:54.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.950+0000 7f3224dabb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f32141bd550 con 0x556cd11cabe0 2024-05-03T18:33:54.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.951+0000 7f3224dabb80 10 client.0 _ll_register_callbacks cb 0x556cd1195cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:33:54.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.951+0000 7f3213400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556cd11cabe0 0x556cd12a2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:54.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.951+0000 7f3213400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556cd11cabe0 0x556cd12a2e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:40656/0 (socket says 172.21.15.33:40656) 2024-05-03T18:33:54.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.951+0000 7f3213400700 1 -- 172.21.15.33:0/1193666811 learned_addr learned my addr 172.21.15.33:0/1193666811 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:33:54.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.951+0000 7f320b400700 1 --2- 172.21.15.33:0/1193666811 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cd129c650 0x556cd12ae160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:54.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.951+0000 7f3213400700 1 --2- 172.21.15.33:0/1193666811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556cd12a0830 0x556cd12ae6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:54.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.952+0000 7f3213400700 1 -- 172.21.15.33:0/1193666811 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cd129c650 msgr2=0x556cd12ae160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:33:54.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.952+0000 7f3213400700 1 --2- 172.21.15.33:0/1193666811 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cd129c650 0x556cd12ae160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:54.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.952+0000 7f3213400700 1 -- 172.21.15.33:0/1193666811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556cd12a0830 msgr2=0x556cd12ae6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:33:54.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.952+0000 7f3213400700 1 --2- 172.21.15.33:0/1193666811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556cd12a0830 0x556cd12ae6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:54.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.952+0000 7f3213400700 1 -- 172.21.15.33:0/1193666811 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556cd12a1670 con 0x556cd11cabe0 2024-05-03T18:33:54.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.953+0000 7f3213400700 1 --2- 172.21.15.33:0/1193666811 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556cd11cabe0 0x556cd12a2e20 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f31fc00b010 tx=0x7f31fc014040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a65f637c677e2f56 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:33:54.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.953+0000 7f3201600700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-03T18:33:54.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.953+0000 7f3201600700 1 -- 172.21.15.33:0/1193666811 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31fc00be60 con 0x556cd11cabe0 2024-05-03T18:33:54.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.953+0000 7f3201600700 1 -- 172.21.15.33:0/1193666811 <== mon.1 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f31fc01f3b0 con 0x556cd11cabe0 2024-05-03T18:33:54.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.953+0000 7f3224dabb80 1 -- 172.21.15.33:0/1193666811 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3204028070 con 0x556cd11cabe0 2024-05-03T18:33:54.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.955+0000 7f3201600700 1 -- 172.21.15.33:0/1193666811 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f31fc00de70 con 0x556cd11cabe0 2024-05-03T18:33:54.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.955+0000 7f3224dabb80 10 client.25795 fetch_fsmap learned FSMap version 21 2024-05-03T18:33:54.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.955+0000 7f3224dabb80 1 -- 172.21.15.33:0/1193666811 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x556cd13214a0 con 0x556cd11cabe0 2024-05-03T18:33:54.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.956+0000 7f3201600700 1 -- 172.21.15.33:0/1193666811 <== mon.1 v2:172.21.15.150:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3204028070 con 0x556cd11cabe0 2024-05-03T18:33:54.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.956+0000 7f3201600700 1 -- 172.21.15.33:0/1193666811 <== mon.1 v2:172.21.15.150:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f31fc036070 con 0x556cd11cabe0 2024-05-03T18:33:54.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.957+0000 7f3224dabb80 10 client.25795 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:33:54.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.957+0000 7f3224dabb80 10 client.25795 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:33:54.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.957+0000 7f3224dabb80 20 client.25795 populate_metadata read hostname 'smithi033' 2024-05-03T18:33:54.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.957+0000 7f3224dabb80 1 --2- 172.21.15.33:0/1193666811 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x556cd12a1150 0x556cd1323b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:33:54.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.957+0000 7f3224dabb80 4 client.25795 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-03T18:33:54.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.957+0000 7f3224dabb80 1 -- 172.21.15.33:0/1193666811 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x556cd11fc480 con 0x556cd12a1150 2024-05-03T18:33:54.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.958+0000 7f320b400700 1 --2- 172.21.15.33:0/1193666811 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x556cd12a1150 0x556cd1323b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:33:54.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.959+0000 7f3201600700 10 client.25795 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:33:54.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.960+0000 7f320b400700 1 --2- 172.21.15.33:0/1193666811 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x556cd12a1150 0x556cd1323b00 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1632f56f7e535927 server_cookie=b82224beb2af2647 in_seq=0 out_seq=0 2024-05-03T18:33:54.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.963+0000 7f3201600700 1 -- 172.21.15.33:0/1193666811 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x556cd11fc480 con 0x556cd12a1150 2024-05-03T18:33:54.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.963+0000 7f3201600700 10 client.25795 handle_command_reply: tid=0 2024-05-03T18:33:54.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.963+0000 7f3224dabb80 1 client.25795 shutdown 2024-05-03T18:33:54.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.963+0000 7f3224dabb80 1 -- 172.21.15.33:0/1193666811 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556cd11cabe0 msgr2=0x556cd12a2e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:33:54.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.963+0000 7f3224dabb80 1 --2- 172.21.15.33:0/1193666811 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556cd11cabe0 0x556cd12a2e20 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f31fc00b010 tx=0x7f31fc014040 comp rx=0 tx=0).stop 2024-05-03T18:33:54.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.963+0000 7f3224dabb80 1 -- 172.21.15.33:0/1193666811 shutdown_connections 2024-05-03T18:33:54.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.963+0000 7f3224dabb80 1 --2- 172.21.15.33:0/1193666811 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x556cd12a1150 0x556cd1323b00 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:54.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.963+0000 7f3224dabb80 1 --2- 172.21.15.33:0/1193666811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556cd12a0830 0x556cd12ae6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:54.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.963+0000 7f3224dabb80 1 --2- 172.21.15.33:0/1193666811 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556cd11cabe0 0x556cd12a2e20 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:54.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.963+0000 7f3224dabb80 1 --2- 172.21.15.33:0/1193666811 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cd129c650 0x556cd12ae160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:54.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.963+0000 7f3201600700 0 client.25795 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:33:54.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.963+0000 7f3224dabb80 1 -- 172.21.15.33:0/1193666811 >> 172.21.15.33:0/1193666811 conn(0x556cd12a6200 msgr2=0x556cd12a67a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:33:54.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.964+0000 7f3224dabb80 1 -- 172.21.15.33:0/1193666811 shutdown_connections 2024-05-03T18:33:54.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.964+0000 7f3224dabb80 1 -- 172.21.15.33:0/1193666811 wait complete. 2024-05-03T18:33:54.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.965+0000 7f3224dabb80 20 client.25795 trim_cache size 0 max 16384 2024-05-03T18:33:54.966 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-03T18:33:54.966 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-05-03T18:33:54.966 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "c62420b9-005c-42bc-96d8-05fcf58b66ec", 2024-05-03T18:33:54.966 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-05-03T18:33:54.966 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-03T18:33:54.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.966+0000 7f321a400700 1 -- 172.21.15.33:0/429460201 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f31f80b2b30 msgr2=0x7f31f80b4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:33:54.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.966+0000 7f321a400700 1 --2- 172.21.15.33:0/429460201 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f31f80b2b30 0x7f31f80b4ff0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f320c0a7640 tx=0x7f320c005040 comp rx=0 tx=0).stop 2024-05-03T18:33:54.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.966+0000 7f321a400700 1 -- 172.21.15.33:0/429460201 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3214109510 msgr2=0x7f321410e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:33:54.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.966+0000 7f321a400700 1 --2- 172.21.15.33:0/429460201 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3214109510 0x7f321410e1e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f320c0060b0 tx=0x7f320c007770 comp rx=0 tx=0).stop 2024-05-03T18:33:54.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.967+0000 7f321a400700 1 -- 172.21.15.33:0/429460201 shutdown_connections 2024-05-03T18:33:54.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.967+0000 7f321a400700 1 --2- 172.21.15.33:0/429460201 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f31f80b2b30 0x7f31f80b4ff0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:54.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.967+0000 7f321a400700 1 --2- 172.21.15.33:0/429460201 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3214109510 0x7f321410e1e0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:54.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.967+0000 7f321a400700 1 --2- 172.21.15.33:0/429460201 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f321406b180 0x7f321419c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:54.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.967+0000 7f321a400700 1 --2- 172.21.15.33:0/429460201 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f321406bb40 0x7f321419c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:33:54.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.967+0000 7f321a400700 1 -- 172.21.15.33:0/429460201 >> 172.21.15.33:0/429460201 conn(0x7f3214079350 msgr2=0x7f3214102dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:33:54.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.967+0000 7f321a400700 1 -- 172.21.15.33:0/429460201 shutdown_connections 2024-05-03T18:33:54.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:33:54.967+0000 7f321a400700 1 -- 172.21.15.33:0/429460201 wait complete. 2024-05-03T18:33:55.110 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 160.0 s, 549.2 tps, lat 181.447 ms stddev 265.982 2024-05-03T18:33:55.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:33:55 smithi099 ceph-mon[31658]: scrub queued for path: / 2024-05-03T18:33:55.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:33:55 smithi099 ceph-mon[31658]: scrub summary: idle+waiting paths [/] 2024-05-03T18:33:55.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:33:55 smithi099 ceph-mon[31658]: scrub summary: active paths [/] 2024-05-03T18:33:55.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:33:55 smithi150 ceph-mon[32492]: scrub queued for path: / 2024-05-03T18:33:55.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:33:55 smithi150 ceph-mon[32492]: scrub summary: idle+waiting paths [/] 2024-05-03T18:33:55.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:33:55 smithi150 ceph-mon[32492]: scrub summary: active paths [/] 2024-05-03T18:33:55.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:33:55 smithi033 ceph-mon[28330]: scrub queued for path: / 2024-05-03T18:33:55.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:33:55 smithi033 ceph-mon[28330]: scrub summary: idle+waiting paths [/] 2024-05-03T18:33:55.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:33:55 smithi033 ceph-mon[28330]: scrub summary: active paths [/] 2024-05-03T18:33:56.547 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:33:56 smithi099 ceph-mon[31658]: pgmap v205: 129 pgs: 129 active+clean; 2.2 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 12 MiB/s wr, 561 op/s 2024-05-03T18:33:56.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:33:56 smithi150 ceph-mon[32492]: pgmap v205: 129 pgs: 129 active+clean; 2.2 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 12 MiB/s wr, 561 op/s 2024-05-03T18:33:56.736 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:33:56 smithi033 ceph-mon[28330]: pgmap v205: 129 pgs: 129 active+clean; 2.2 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 12 MiB/s wr, 561 op/s 2024-05-03T18:33:57.097 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-03T18:33:57.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:33:57 smithi033 ceph-mon[28330]: scrub summary: idle+waiting paths [/] 2024-05-03T18:33:57.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:33:57 smithi099 ceph-mon[31658]: scrub summary: idle+waiting paths [/] 2024-05-03T18:33:57.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:33:57 smithi150 ceph-mon[32492]: scrub summary: idle+waiting paths [/] 2024-05-03T18:33:57.654 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:33:58.346 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:33:58 smithi033 ceph-mon[28330]: pgmap v206: 129 pgs: 129 active+clean; 2.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 17 MiB/s wr, 782 op/s 2024-05-03T18:33:58.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:33:58 smithi150 ceph-mon[32492]: pgmap v206: 129 pgs: 129 active+clean; 2.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 17 MiB/s wr, 782 op/s 2024-05-03T18:33:58.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:33:58 smithi099 ceph-mon[31658]: pgmap v206: 129 pgs: 129 active+clean; 2.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 17 MiB/s wr, 782 op/s 2024-05-03T18:34:00.110 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 165.0 s, 602.6 tps, lat 165.055 ms stddev 231.542 2024-05-03T18:34:00.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:34:00 smithi033 ceph-mon[28330]: pgmap v207: 129 pgs: 129 active+clean; 2.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 12 MiB/s wr, 604 op/s 2024-05-03T18:34:00.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:34:00 smithi150 ceph-mon[32492]: pgmap v207: 129 pgs: 129 active+clean; 2.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 12 MiB/s wr, 604 op/s 2024-05-03T18:34:00.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:34:00 smithi099 ceph-mon[31658]: pgmap v207: 129 pgs: 129 active+clean; 2.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 12 MiB/s wr, 604 op/s 2024-05-03T18:34:00.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.543+0000 7f567e200700 1 -- 172.21.15.33:0/1255034037 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f567806b180 msgr2=0x7f567806b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:34:00.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.543+0000 7f567e200700 1 --2- 172.21.15.33:0/1255034037 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f567806b180 0x7f567806b600 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f5664009990 tx=0x7f5664009ca0 comp rx=0 tx=0).stop 2024-05-03T18:34:00.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.544+0000 7f567e200700 1 -- 172.21.15.33:0/1255034037 shutdown_connections 2024-05-03T18:34:00.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.544+0000 7f567e200700 1 --2- 172.21.15.33:0/1255034037 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f567806bb40 0x7f5678110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:00.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.544+0000 7f567e200700 1 --2- 172.21.15.33:0/1255034037 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5678109510 0x7f56781098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:00.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.544+0000 7f567e200700 1 --2- 172.21.15.33:0/1255034037 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f567806b180 0x7f567806b600 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:00.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.544+0000 7f567e200700 1 -- 172.21.15.33:0/1255034037 >> 172.21.15.33:0/1255034037 conn(0x7f5678079350 msgr2=0x7f5678079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:34:00.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.544+0000 7f567e200700 1 -- 172.21.15.33:0/1255034037 shutdown_connections 2024-05-03T18:34:00.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.544+0000 7f567e200700 1 -- 172.21.15.33:0/1255034037 wait complete. 2024-05-03T18:34:00.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.547+0000 7f567e200700 1 Processor -- start 2024-05-03T18:34:00.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.547+0000 7f567e200700 1 -- start start 2024-05-03T18:34:00.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.548+0000 7f567e200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f567806b180 0x7f567810f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:34:00.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.549+0000 7f567e200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f567806bb40 0x7f567810fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:34:00.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.549+0000 7f567e200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5678109510 0x7f56781a49f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:34:00.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.549+0000 7f567e200700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5678113520 con 0x7f567806b180 2024-05-03T18:34:00.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.549+0000 7f567e200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f56781133a0 con 0x7f5678109510 2024-05-03T18:34:00.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.549+0000 7f567e200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f56781136a0 con 0x7f567806bb40 2024-05-03T18:34:00.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.549+0000 7f5677400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f567806b180 0x7f567810f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:34:00.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.549+0000 7f5677e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5678109510 0x7f56781a49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:34:00.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.549+0000 7f5677e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5678109510 0x7f56781a49f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:35786/0 (socket says 172.21.15.33:35786) 2024-05-03T18:34:00.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.549+0000 7f5677e00700 1 -- 172.21.15.33:0/771990999 learned_addr learned my addr 172.21.15.33:0/771990999 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:34:00.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.550+0000 7f5676a00700 1 --2- 172.21.15.33:0/771990999 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f567806bb40 0x7f567810fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:34:00.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.550+0000 7f5677e00700 1 -- 172.21.15.33:0/771990999 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f567806bb40 msgr2=0x7f567810fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:34:00.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.550+0000 7f5677e00700 1 --2- 172.21.15.33:0/771990999 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f567806bb40 0x7f567810fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:00.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.550+0000 7f5677e00700 1 -- 172.21.15.33:0/771990999 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f567806b180 msgr2=0x7f567810f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:34:00.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.550+0000 7f5677e00700 1 --2- 172.21.15.33:0/771990999 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f567806b180 0x7f567810f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:00.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.550+0000 7f5677e00700 1 -- 172.21.15.33:0/771990999 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56781a50f0 con 0x7f5678109510 2024-05-03T18:34:00.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.550+0000 7f5677e00700 1 --2- 172.21.15.33:0/771990999 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5678109510 0x7f56781a49f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f567000ef30 tx=0x7f567000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:34:00.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.551+0000 7f566fe00700 1 -- 172.21.15.33:0/771990999 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56700092e0 con 0x7f5678109510 2024-05-03T18:34:00.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.551+0000 7f566fe00700 1 -- 172.21.15.33:0/771990999 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5670004bb0 con 0x7f5678109510 2024-05-03T18:34:00.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.551+0000 7f566fe00700 1 -- 172.21.15.33:0/771990999 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5670004020 con 0x7f5678109510 2024-05-03T18:34:00.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.552+0000 7f567e200700 1 -- 172.21.15.33:0/771990999 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56781a5380 con 0x7f5678109510 2024-05-03T18:34:00.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.552+0000 7f567e200700 1 -- 172.21.15.33:0/771990999 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56781a57b0 con 0x7f5678109510 2024-05-03T18:34:00.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.557+0000 7f566fe00700 1 -- 172.21.15.33:0/771990999 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f567001b030 con 0x7f5678109510 2024-05-03T18:34:00.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.558+0000 7f5688c7bb80 1 Processor -- start 2024-05-03T18:34:00.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.559+0000 7f5688c7bb80 1 -- start start 2024-05-03T18:34:00.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.559+0000 7f5688c7bb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5564297647b0 0x556429764b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:34:00.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.559+0000 7f5688c7bb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5564297650d0 0x55642975ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:34:00.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.559+0000 7f5688c7bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5564297605d0 0x556429760a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:34:00.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.560+0000 7f5676a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5564297647b0 0x556429764b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:34:00.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.560+0000 7f5688c7bb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5564296b2a20 con 0x5564297650d0 2024-05-03T18:34:00.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.560+0000 7f5688c7bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x556429650760 con 0x5564297605d0 2024-05-03T18:34:00.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.560+0000 7f5688c7bb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5564296ae9e0 con 0x5564297647b0 2024-05-03T18:34:00.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.560+0000 7f5676a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5564297647b0 0x556429764b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:55646/0 (socket says 172.21.15.33:55646) 2024-05-03T18:34:00.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.560+0000 7f5676a00700 1 -- 172.21.15.33:0/1091405961 learned_addr learned my addr 172.21.15.33:0/1091405961 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:34:00.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.560+0000 7f566fe00700 1 --2- 172.21.15.33:0/771990999 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f565c0b2a70 0x7f565c0b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:34:00.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.560+0000 7f5677400700 1 --2- 172.21.15.33:0/1091405961 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5564297650d0 0x55642975ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:34:00.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.560+0000 7f5676a00700 1 --2- 172.21.15.33:0/1091405961 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5564297605d0 0x556429760a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:34:00.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.560+0000 7f566fe00700 1 -- 172.21.15.33:0/771990999 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f56700e2460 con 0x7f5678109510 2024-05-03T18:34:00.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.561+0000 7f5676a00700 1 -- 172.21.15.33:0/1091405961 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5564297605d0 msgr2=0x556429760a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:34:00.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.561+0000 7f5676a00700 1 --2- 172.21.15.33:0/1091405961 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5564297605d0 0x556429760a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:00.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.561+0000 7f5676a00700 1 -- 172.21.15.33:0/1091405961 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5564297650d0 msgr2=0x55642975ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:34:00.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.561+0000 7f5676a00700 1 --2- 172.21.15.33:0/1091405961 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5564297650d0 0x55642975ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:00.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.561+0000 7f5676a00700 1 -- 172.21.15.33:0/1091405961 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564295ff8d0 con 0x5564297647b0 2024-05-03T18:34:00.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.561+0000 7f5677e00700 1 --2- 172.21.15.33:0/771990999 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f565c0b2a70 0x7f565c0b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:34:00.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.561+0000 7f5677400700 1 --2- 172.21.15.33:0/1091405961 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5564297650d0 0x55642975ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:34:00.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.562+0000 7f5676a00700 1 --2- 172.21.15.33:0/1091405961 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5564297647b0 0x556429764b90 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f566400b2e0 tx=0x7f5664005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d72d298a4f74637 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:34:00.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.562+0000 7f566cc00700 1 -- 172.21.15.33:0/1091405961 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5664021040 con 0x5564297647b0 2024-05-03T18:34:00.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.563+0000 7f566cc00700 1 -- 172.21.15.33:0/1091405961 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5664003680 con 0x5564297647b0 2024-05-03T18:34:00.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.563+0000 7f566cc00700 1 -- 172.21.15.33:0/1091405961 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f566400f6c0 con 0x5564297647b0 2024-05-03T18:34:00.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.564+0000 7f5688c7bb80 1 -- 172.21.15.33:0/1091405961 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5564297647b0 msgr2=0x556429764b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:34:00.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.564+0000 7f5677e00700 1 --2- 172.21.15.33:0/771990999 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f565c0b2a70 0x7f565c0b4f30 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f56700df970 tx=0x7f5670009040 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:34:00.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.564+0000 7f5688c7bb80 1 --2- 172.21.15.33:0/1091405961 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5564297647b0 0x556429764b90 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f566400b2e0 tx=0x7f5664005c00 comp rx=0 tx=0).stop 2024-05-03T18:34:00.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.564+0000 7f5688c7bb80 1 -- 172.21.15.33:0/1091405961 shutdown_connections 2024-05-03T18:34:00.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.564+0000 7f5688c7bb80 1 --2- 172.21.15.33:0/1091405961 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5564297605d0 0x556429760a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:00.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.564+0000 7f5688c7bb80 1 --2- 172.21.15.33:0/1091405961 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5564297647b0 0x556429764b90 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:00.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.564+0000 7f5688c7bb80 1 --2- 172.21.15.33:0/1091405961 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5564297650d0 0x55642975ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:00.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.564+0000 7f5688c7bb80 1 -- 172.21.15.33:0/1091405961 >> 172.21.15.33:0/1091405961 conn(0x55642968eb60 msgr2=0x55642975a7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:34:00.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.565+0000 7f5688c7bb80 1 -- 172.21.15.33:0/1091405961 shutdown_connections 2024-05-03T18:34:00.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.565+0000 7f5688c7bb80 1 -- 172.21.15.33:0/1091405961 wait complete. 2024-05-03T18:34:00.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.565+0000 7f5688c7bb80 1 -- start start 2024-05-03T18:34:00.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.566+0000 7f5688c7bb80 1 Processor -- start 2024-05-03T18:34:00.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.566+0000 7f5688c7bb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5564297605d0 0x55642976bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:34:00.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.566+0000 7f5688c7bb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5564297647b0 0x556429769310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:34:00.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.566+0000 7f5688c7bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5564297650d0 0x556429769850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:34:00.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.566+0000 7f5688c7bb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55642976cc40 con 0x5564297605d0 2024-05-03T18:34:00.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.566+0000 7f5688c7bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x556429618b20 con 0x5564297650d0 2024-05-03T18:34:00.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.566+0000 7f5688c7bb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55642976cdc0 con 0x5564297647b0 2024-05-03T18:34:00.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.566+0000 7f5688c7bb80 10 client.0 _ll_register_callbacks cb 0x556429659c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:34:00.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.567+0000 7f5677400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5564297647b0 0x556429769310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:34:00.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.567+0000 7f5677400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5564297647b0 0x556429769310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:55648/0 (socket says 172.21.15.33:55648) 2024-05-03T18:34:00.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.567+0000 7f5677400700 1 -- 172.21.15.33:0/2161521688 learned_addr learned my addr 172.21.15.33:0/2161521688 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:34:00.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.567+0000 7f5676a00700 1 --2- 172.21.15.33:0/2161521688 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5564297605d0 0x55642976bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:34:00.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.567+0000 7f5676a00700 1 --2- 172.21.15.33:0/2161521688 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5564297650d0 0x556429769850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:34:00.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.569+0000 7f5677400700 1 -- 172.21.15.33:0/2161521688 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5564297650d0 msgr2=0x556429769850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:34:00.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.569+0000 7f5677400700 1 --2- 172.21.15.33:0/2161521688 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5564297650d0 0x556429769850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:00.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.569+0000 7f5677400700 1 -- 172.21.15.33:0/2161521688 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5564297605d0 msgr2=0x55642976bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:34:00.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.569+0000 7f5677400700 1 --2- 172.21.15.33:0/2161521688 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5564297605d0 0x55642976bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:00.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.569+0000 7f5677400700 1 -- 172.21.15.33:0/2161521688 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556429678420 con 0x5564297647b0 2024-05-03T18:34:00.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.570+0000 7f5677400700 1 --2- 172.21.15.33:0/2161521688 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5564297647b0 0x556429769310 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5668010840 tx=0x7f5668010b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a78d4242b7bfe7c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:34:00.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.570+0000 7f5661600700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-03T18:34:00.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.571+0000 7f5661600700 1 -- 172.21.15.33:0/2161521688 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5668004020 con 0x5564297647b0 2024-05-03T18:34:00.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.571+0000 7f5688c7bb80 1 -- 172.21.15.33:0/2161521688 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5664003680 con 0x5564297647b0 2024-05-03T18:34:00.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.571+0000 7f5661600700 1 -- 172.21.15.33:0/2161521688 <== mon.1 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f56680041c0 con 0x5564297647b0 2024-05-03T18:34:00.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.571+0000 7f5661600700 1 -- 172.21.15.33:0/2161521688 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f5668024050 con 0x5564297647b0 2024-05-03T18:34:00.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.572+0000 7f5688c7bb80 10 client.25804 fetch_fsmap learned FSMap version 21 2024-05-03T18:34:00.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.572+0000 7f5688c7bb80 1 -- 172.21.15.33:0/2161521688 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5564297775b0 con 0x5564297647b0 2024-05-03T18:34:00.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.572+0000 7f5661600700 1 -- 172.21.15.33:0/2161521688 <== mon.1 v2:172.21.15.150:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5664003680 con 0x5564297647b0 2024-05-03T18:34:00.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.573+0000 7f5661600700 1 -- 172.21.15.33:0/2161521688 <== mon.1 v2:172.21.15.150:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f5668003840 con 0x5564297647b0 2024-05-03T18:34:00.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.574+0000 7f5688c7bb80 10 client.25804 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:34:00.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.574+0000 7f5688c7bb80 10 client.25804 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:34:00.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.574+0000 7f5688c7bb80 20 client.25804 populate_metadata read hostname 'smithi033' 2024-05-03T18:34:00.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.574+0000 7f5688c7bb80 1 --2- 172.21.15.33:0/2161521688 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x556429779b40 0x55642977bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:34:00.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.574+0000 7f5688c7bb80 4 client.25804 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:34:00.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.574+0000 7f5688c7bb80 1 -- 172.21.15.33:0/2161521688 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55642972d1d0 con 0x556429779b40 2024-05-03T18:34:00.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.574+0000 7f5676a00700 1 --2- 172.21.15.33:0/2161521688 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x556429779b40 0x55642977bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:34:00.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.575+0000 7f5676a00700 1 --2- 172.21.15.33:0/2161521688 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x556429779b40 0x55642977bf30 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ee6b2be5c287e19 server_cookie=782b02513ea02391 in_seq=0 out_seq=0 2024-05-03T18:34:00.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.576+0000 7f5661600700 10 client.25804 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:34:00.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.577+0000 7f5661600700 1 -- 172.21.15.33:0/2161521688 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55642972d1d0 con 0x556429779b40 2024-05-03T18:34:00.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.577+0000 7f5661600700 10 client.25804 handle_command_reply: tid=0 2024-05-03T18:34:00.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.577+0000 7f5688c7bb80 1 client.25804 shutdown 2024-05-03T18:34:00.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.578+0000 7f5688c7bb80 1 -- 172.21.15.33:0/2161521688 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5564297647b0 msgr2=0x556429769310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:34:00.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.578+0000 7f5688c7bb80 1 --2- 172.21.15.33:0/2161521688 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5564297647b0 0x556429769310 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5668010840 tx=0x7f5668010b50 comp rx=0 tx=0).stop 2024-05-03T18:34:00.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.578+0000 7f5688c7bb80 1 -- 172.21.15.33:0/2161521688 shutdown_connections 2024-05-03T18:34:00.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.578+0000 7f5688c7bb80 1 --2- 172.21.15.33:0/2161521688 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x556429779b40 0x55642977bf30 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:00.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.578+0000 7f5688c7bb80 1 --2- 172.21.15.33:0/2161521688 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5564297650d0 0x556429769850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:00.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.578+0000 7f5688c7bb80 1 --2- 172.21.15.33:0/2161521688 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5564297605d0 0x55642976bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:00.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.578+0000 7f5688c7bb80 1 --2- 172.21.15.33:0/2161521688 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5564297647b0 0x556429769310 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:00.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.578+0000 7f5688c7bb80 1 -- 172.21.15.33:0/2161521688 >> 172.21.15.33:0/2161521688 conn(0x55642968eb60 msgr2=0x556429759650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:34:00.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.578+0000 7f5661600700 0 client.25804 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:34:00.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.578+0000 7f5688c7bb80 1 -- 172.21.15.33:0/2161521688 shutdown_connections 2024-05-03T18:34:00.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.578+0000 7f5688c7bb80 1 -- 172.21.15.33:0/2161521688 wait complete. 2024-05-03T18:34:00.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.579+0000 7f5688c7bb80 20 client.25804 trim_cache size 0 max 16384 2024-05-03T18:34:00.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.614+0000 7f5688c7bb80 1 Processor -- start 2024-05-03T18:34:00.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.615+0000 7f5688c7bb80 1 -- start start 2024-05-03T18:34:00.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.615+0000 7f5688c7bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5564297605d0 0x556429775d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:34:00.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.615+0000 7f5688c7bb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5564297647b0 0x5564297817c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:34:00.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.615+0000 7f5688c7bb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5564297650d0 0x556429781d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:34:00.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.615+0000 7f5688c7bb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f56781b3a30 con 0x5564297650d0 2024-05-03T18:34:00.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.615+0000 7f5688c7bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f56781b62b0 con 0x5564297605d0 2024-05-03T18:34:00.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.615+0000 7f5688c7bb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f56781b40a0 con 0x5564297647b0 2024-05-03T18:34:00.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.616+0000 7f5677400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5564297647b0 0x5564297817c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:34:00.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.616+0000 7f5677400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5564297647b0 0x5564297817c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:55656/0 (socket says 172.21.15.33:55656) 2024-05-03T18:34:00.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.616+0000 7f5677400700 1 -- 172.21.15.33:0/437130684 learned_addr learned my addr 172.21.15.33:0/437130684 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:34:00.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.617+0000 7f5676a00700 1 --2- 172.21.15.33:0/437130684 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5564297605d0 0x556429775d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:34:00.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.617+0000 7f5676a00700 1 --2- 172.21.15.33:0/437130684 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5564297650d0 0x556429781d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:34:00.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.617+0000 7f5677400700 1 -- 172.21.15.33:0/437130684 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5564297605d0 msgr2=0x556429775d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:34:00.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.618+0000 7f5677400700 1 --2- 172.21.15.33:0/437130684 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5564297605d0 0x556429775d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:00.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.618+0000 7f5677400700 1 -- 172.21.15.33:0/437130684 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5564297650d0 msgr2=0x556429781d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:34:00.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.618+0000 7f5677400700 1 --2- 172.21.15.33:0/437130684 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5564297650d0 0x556429781d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:00.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.618+0000 7f5677400700 1 -- 172.21.15.33:0/437130684 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556429764400 con 0x5564297647b0 2024-05-03T18:34:00.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.618+0000 7f5676a00700 1 --2- 172.21.15.33:0/437130684 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5564297605d0 0x556429775d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:34:00.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.619+0000 7f5677400700 1 --2- 172.21.15.33:0/437130684 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5564297647b0 0x5564297817c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5668003c30 tx=0x7f5668005800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5922af9283e019e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:34:00.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.619+0000 7f5663e00700 1 -- 172.21.15.33:0/437130684 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f566802c070 con 0x5564297647b0 2024-05-03T18:34:00.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.619+0000 7f5663e00700 1 -- 172.21.15.33:0/437130684 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f566800f830 con 0x5564297647b0 2024-05-03T18:34:00.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.621+0000 7f5663e00700 1 -- 172.21.15.33:0/437130684 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5668004100 con 0x5564297647b0 2024-05-03T18:34:00.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.622+0000 7f5688c7bb80 1 -- 172.21.15.33:0/437130684 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5564297647b0 msgr2=0x5564297817c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:34:00.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.622+0000 7f5688c7bb80 1 --2- 172.21.15.33:0/437130684 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5564297647b0 0x5564297817c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5668003c30 tx=0x7f5668005800 comp rx=0 tx=0).stop 2024-05-03T18:34:00.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.622+0000 7f5688c7bb80 1 -- 172.21.15.33:0/437130684 shutdown_connections 2024-05-03T18:34:00.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.622+0000 7f5688c7bb80 1 --2- 172.21.15.33:0/437130684 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5564297650d0 0x556429781d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:00.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.622+0000 7f5688c7bb80 1 --2- 172.21.15.33:0/437130684 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5564297605d0 0x556429775d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:00.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.622+0000 7f5688c7bb80 1 --2- 172.21.15.33:0/437130684 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5564297647b0 0x5564297817c0 secure :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5668003c30 tx=0x7f5668005800 comp rx=0 tx=0).stop 2024-05-03T18:34:00.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.622+0000 7f5688c7bb80 1 -- 172.21.15.33:0/437130684 >> 172.21.15.33:0/437130684 conn(0x55642968eb60 msgr2=0x556429759ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:34:00.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.623+0000 7f5688c7bb80 1 -- 172.21.15.33:0/437130684 shutdown_connections 2024-05-03T18:34:00.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.624+0000 7f5688c7bb80 1 -- 172.21.15.33:0/437130684 wait complete. 2024-05-03T18:34:00.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.625+0000 7f5688c7bb80 1 -- start start 2024-05-03T18:34:00.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.625+0000 7f5688c7bb80 1 Processor -- start 2024-05-03T18:34:00.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.626+0000 7f5688c7bb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5564297605d0 0x556429772690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:34:00.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.626+0000 7f5688c7bb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5564297650d0 0x55642976e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:34:00.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.626+0000 7f5688c7bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55642976eec0 0x55642976f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:34:00.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.626+0000 7f5688c7bb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f56781b4770 con 0x5564297650d0 2024-05-03T18:34:00.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.626+0000 7f5688c7bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f56781b5be0 con 0x55642976eec0 2024-05-03T18:34:00.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.626+0000 7f5688c7bb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f56781b4e40 con 0x5564297605d0 2024-05-03T18:34:00.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.626+0000 7f5688c7bb80 10 client.0 _ll_register_callbacks cb 0x556429659c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:34:00.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.627+0000 7f5676a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5564297605d0 0x556429772690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:34:00.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.627+0000 7f5676a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5564297605d0 0x556429772690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:55658/0 (socket says 172.21.15.33:55658) 2024-05-03T18:34:00.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.627+0000 7f5676a00700 1 -- 172.21.15.33:0/3226134770 learned_addr learned my addr 172.21.15.33:0/3226134770 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:34:00.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.627+0000 7f5676a00700 1 --2- 172.21.15.33:0/3226134770 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55642976eec0 0x55642976f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:34:00.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.628+0000 7f5676a00700 1 -- 172.21.15.33:0/3226134770 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55642976eec0 msgr2=0x55642976f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:34:00.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.628+0000 7f5676a00700 1 --2- 172.21.15.33:0/3226134770 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55642976eec0 0x55642976f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:00.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.628+0000 7f5676a00700 1 -- 172.21.15.33:0/3226134770 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5564297650d0 msgr2=0x55642976e980 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-03T18:34:00.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.630+0000 7f5676a00700 1 --2- 172.21.15.33:0/3226134770 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5564297650d0 0x55642976e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:00.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.631+0000 7f5676a00700 1 -- 172.21.15.33:0/3226134770 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564297e4550 con 0x5564297605d0 2024-05-03T18:34:00.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.631+0000 7f5676a00700 1 --2- 172.21.15.33:0/3226134770 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5564297605d0 0x556429772690 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f566400b2b0 tx=0x7f5664019040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3084cb367a605652 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:34:00.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.631+0000 7f5661600700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-03T18:34:00.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.632+0000 7f5661600700 1 -- 172.21.15.33:0/3226134770 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5664005470 con 0x5564297605d0 2024-05-03T18:34:00.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.632+0000 7f5661600700 1 -- 172.21.15.33:0/3226134770 <== mon.1 v2:172.21.15.150:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9430+0+0 (secure 0 0 0) 0x7f5664037c70 con 0x5564297605d0 2024-05-03T18:34:00.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.632+0000 7f5688c7bb80 1 -- 172.21.15.33:0/3226134770 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f566800f830 con 0x5564297605d0 2024-05-03T18:34:00.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.632+0000 7f5661600700 1 -- 172.21.15.33:0/3226134770 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f5664037380 con 0x5564297605d0 2024-05-03T18:34:00.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.632+0000 7f5688c7bb80 10 client.25810 fetch_fsmap learned FSMap version 21 2024-05-03T18:34:00.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.632+0000 7f5688c7bb80 1 -- 172.21.15.33:0/3226134770 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5564297e5860 con 0x5564297605d0 2024-05-03T18:34:00.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.633+0000 7f5661600700 1 -- 172.21.15.33:0/3226134770 <== mon.1 v2:172.21.15.150:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f566800f830 con 0x5564297605d0 2024-05-03T18:34:00.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.633+0000 7f5661600700 1 -- 172.21.15.33:0/3226134770 <== mon.1 v2:172.21.15.150:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f56640039f0 con 0x5564297605d0 2024-05-03T18:34:00.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.633+0000 7f5688c7bb80 10 client.25810 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:34:00.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.633+0000 7f5688c7bb80 10 client.25810 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:34:00.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.633+0000 7f5688c7bb80 20 client.25810 populate_metadata read hostname 'smithi033' 2024-05-03T18:34:00.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.633+0000 7f5688c7bb80 1 --2- 172.21.15.33:0/3226134770 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5564297e5af0 0x5564297e7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:34:00.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.633+0000 7f5688c7bb80 4 client.25810 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-03T18:34:00.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.633+0000 7f5688c7bb80 1 -- 172.21.15.33:0/3226134770 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5564296c0400 con 0x5564297e5af0 2024-05-03T18:34:00.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.634+0000 7f5677400700 1 --2- 172.21.15.33:0/3226134770 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5564297e5af0 0x5564297e7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:34:00.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.636+0000 7f5677400700 1 --2- 172.21.15.33:0/3226134770 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5564297e5af0 0x5564297e7ee0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92e00680981804d8 server_cookie=86de1ab223df2356 in_seq=0 out_seq=0 2024-05-03T18:34:00.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.636+0000 7f5661600700 10 client.25810 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:34:00.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.638+0000 7f5661600700 1 -- 172.21.15.33:0/3226134770 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5564296c0400 con 0x5564297e5af0 2024-05-03T18:34:00.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.638+0000 7f5661600700 10 client.25810 handle_command_reply: tid=0 2024-05-03T18:34:00.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.638+0000 7f5688c7bb80 1 client.25810 shutdown 2024-05-03T18:34:00.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.638+0000 7f5688c7bb80 1 -- 172.21.15.33:0/3226134770 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5564297605d0 msgr2=0x556429772690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:34:00.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.638+0000 7f5688c7bb80 1 --2- 172.21.15.33:0/3226134770 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5564297605d0 0x556429772690 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f566400b2b0 tx=0x7f5664019040 comp rx=0 tx=0).stop 2024-05-03T18:34:00.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.639+0000 7f5688c7bb80 1 -- 172.21.15.33:0/3226134770 shutdown_connections 2024-05-03T18:34:00.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.639+0000 7f5688c7bb80 1 --2- 172.21.15.33:0/3226134770 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5564297e5af0 0x5564297e7ee0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:00.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.639+0000 7f5688c7bb80 1 --2- 172.21.15.33:0/3226134770 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55642976eec0 0x55642976f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:00.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.639+0000 7f5688c7bb80 1 --2- 172.21.15.33:0/3226134770 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5564297605d0 0x556429772690 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:00.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.639+0000 7f5688c7bb80 1 --2- 172.21.15.33:0/3226134770 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5564297650d0 0x55642976e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:00.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.639+0000 7f5688c7bb80 1 -- 172.21.15.33:0/3226134770 >> 172.21.15.33:0/3226134770 conn(0x55642968eb60 msgr2=0x55642975efa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:34:00.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.639+0000 7f5661600700 0 client.25810 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:34:00.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.639+0000 7f5688c7bb80 1 -- 172.21.15.33:0/3226134770 shutdown_connections 2024-05-03T18:34:00.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.640+0000 7f5688c7bb80 1 -- 172.21.15.33:0/3226134770 wait complete. 2024-05-03T18:34:00.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.641+0000 7f5688c7bb80 20 client.25810 trim_cache size 0 max 16384 2024-05-03T18:34:00.643 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-03T18:34:00.643 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-03T18:34:00.643 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-05-03T18:34:00.643 INFO:teuthology.orchestra.run.smithi033.stdout: "c62420b9-005c-42bc-96d8-05fcf58b66ec": { 2024-05-03T18:34:00.643 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-05-03T18:34:00.643 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "c62420b9-005c-42bc-96d8-05fcf58b66ec", 2024-05-03T18:34:00.643 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-05-03T18:34:00.643 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:34:00.643 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:34:00.643 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-03T18:34:00.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.643+0000 7f567e200700 1 -- 172.21.15.33:0/771990999 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f565c0b2a70 msgr2=0x7f565c0b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:34:00.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.643+0000 7f567e200700 1 --2- 172.21.15.33:0/771990999 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f565c0b2a70 0x7f565c0b4f30 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f56700df970 tx=0x7f5670009040 comp rx=0 tx=0).stop 2024-05-03T18:34:00.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.643+0000 7f567e200700 1 -- 172.21.15.33:0/771990999 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5678109510 msgr2=0x7f56781a49f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:34:00.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.643+0000 7f567e200700 1 --2- 172.21.15.33:0/771990999 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5678109510 0x7f56781a49f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f567000ef30 tx=0x7f567000c3a0 comp rx=0 tx=0).stop 2024-05-03T18:34:00.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.643+0000 7f567e200700 1 -- 172.21.15.33:0/771990999 shutdown_connections 2024-05-03T18:34:00.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.644+0000 7f567e200700 1 --2- 172.21.15.33:0/771990999 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f565c0b2a70 0x7f565c0b4f30 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:00.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.644+0000 7f567e200700 1 --2- 172.21.15.33:0/771990999 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5678109510 0x7f56781a49f0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:00.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.644+0000 7f567e200700 1 --2- 172.21.15.33:0/771990999 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f567806b180 0x7f567810f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:00.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.644+0000 7f567e200700 1 --2- 172.21.15.33:0/771990999 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f567806bb40 0x7f567810fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:00.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.644+0000 7f567e200700 1 -- 172.21.15.33:0/771990999 >> 172.21.15.33:0/771990999 conn(0x7f5678079350 msgr2=0x7f5678100f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:34:00.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.644+0000 7f567e200700 1 -- 172.21.15.33:0/771990999 shutdown_connections 2024-05-03T18:34:00.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:00.644+0000 7f567e200700 1 -- 172.21.15.33:0/771990999 wait complete. 2024-05-03T18:34:01.608 INFO:tasks.cephfs.filesystem:scrub status for tag:c62420b9-005c-42bc-96d8-05fcf58b66ec - {'path': '/', 'tag': 'c62420b9-005c-42bc-96d8-05fcf58b66ec', 'options': 'recursive,force'} 2024-05-03T18:34:02.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:34:02 smithi033 ceph-mon[28330]: scrub summary: idle 2024-05-03T18:34:02.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:34:02 smithi033 ceph-mon[28330]: osdmap e91: 12 total, 12 up, 12 in 2024-05-03T18:34:02.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:34:02 smithi033 ceph-mon[28330]: pgmap v209: 129 pgs: 129 active+clean; 2.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 14 MiB/s wr, 682 op/s 2024-05-03T18:34:02.547 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:34:02 smithi099 ceph-mon[31658]: scrub summary: idle 2024-05-03T18:34:02.547 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:34:02 smithi099 ceph-mon[31658]: osdmap e91: 12 total, 12 up, 12 in 2024-05-03T18:34:02.547 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:34:02 smithi099 ceph-mon[31658]: pgmap v209: 129 pgs: 129 active+clean; 2.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 14 MiB/s wr, 682 op/s 2024-05-03T18:34:02.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:34:02 smithi150 ceph-mon[32492]: scrub summary: idle 2024-05-03T18:34:02.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:34:02 smithi150 ceph-mon[32492]: osdmap e91: 12 total, 12 up, 12 in 2024-05-03T18:34:02.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:34:02 smithi150 ceph-mon[32492]: pgmap v209: 129 pgs: 129 active+clean; 2.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 14 MiB/s wr, 682 op/s 2024-05-03T18:34:05.110 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 170.0 s, 582.6 tps, lat 169.947 ms stddev 246.882 2024-05-03T18:34:05.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:34:05 smithi099 ceph-mon[31658]: pgmap v210: 129 pgs: 129 active+clean; 2.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 12 MiB/s wr, 602 op/s 2024-05-03T18:34:05.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:34:05 smithi150 ceph-mon[32492]: pgmap v210: 129 pgs: 129 active+clean; 2.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 12 MiB/s wr, 602 op/s 2024-05-03T18:34:05.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:34:05 smithi033 ceph-mon[28330]: pgmap v210: 129 pgs: 129 active+clean; 2.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 12 MiB/s wr, 602 op/s 2024-05-03T18:34:07.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:34:07 smithi099 ceph-mon[31658]: pgmap v211: 129 pgs: 129 active+clean; 2.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 12 MiB/s wr, 602 op/s 2024-05-03T18:34:07.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:34:07 smithi150 ceph-mon[32492]: pgmap v211: 129 pgs: 129 active+clean; 2.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 12 MiB/s wr, 602 op/s 2024-05-03T18:34:07.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:34:07 smithi033 ceph-mon[28330]: pgmap v211: 129 pgs: 129 active+clean; 2.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 12 MiB/s wr, 602 op/s 2024-05-03T18:34:08.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:34:08 smithi033 ceph-mon[28330]: osdmap e92: 12 total, 12 up, 12 in 2024-05-03T18:34:08.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:34:08 smithi033 ceph-mon[28330]: pgmap v213: 129 pgs: 129 active+clean; 2.2 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 14 MiB/s wr, 652 op/s 2024-05-03T18:34:08.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:34:08 smithi099 ceph-mon[31658]: osdmap e92: 12 total, 12 up, 12 in 2024-05-03T18:34:08.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:34:08 smithi099 ceph-mon[31658]: pgmap v213: 129 pgs: 129 active+clean; 2.2 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 14 MiB/s wr, 652 op/s 2024-05-03T18:34:08.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:34:08 smithi150 ceph-mon[32492]: osdmap e92: 12 total, 12 up, 12 in 2024-05-03T18:34:08.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:34:08 smithi150 ceph-mon[32492]: pgmap v213: 129 pgs: 129 active+clean; 2.2 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 14 MiB/s wr, 652 op/s 2024-05-03T18:34:10.110 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 175.0 s, 545.4 tps, lat 164.871 ms stddev 250.667 2024-05-03T18:34:11.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:34:11 smithi099 ceph-mon[31658]: pgmap v214: 129 pgs: 129 active+clean; 2.2 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 13 MiB/s wr, 656 op/s 2024-05-03T18:34:11.335 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:34:11 smithi150 ceph-mon[32492]: pgmap v214: 129 pgs: 129 active+clean; 2.2 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 13 MiB/s wr, 656 op/s 2024-05-03T18:34:11.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:34:11 smithi033 ceph-mon[28330]: pgmap v214: 129 pgs: 129 active+clean; 2.2 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 13 MiB/s wr, 656 op/s 2024-05-03T18:34:12.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:34:12 smithi150 ceph-mon[32492]: pgmap v215: 129 pgs: 129 active+clean; 2.2 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 12 MiB/s wr, 576 op/s 2024-05-03T18:34:12.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:34:12 smithi033 ceph-mon[28330]: pgmap v215: 129 pgs: 129 active+clean; 2.2 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 12 MiB/s wr, 576 op/s 2024-05-03T18:34:12.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:34:12 smithi099 ceph-mon[31658]: pgmap v215: 129 pgs: 129 active+clean; 2.2 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 12 MiB/s wr, 576 op/s 2024-05-03T18:34:15.111 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 180.0 s, 567.6 tps, lat 160.214 ms stddev 233.262 2024-05-03T18:34:15.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:34:15 smithi099 ceph-mon[31658]: pgmap v216: 129 pgs: 129 active+clean; 2.3 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 12 MiB/s wr, 655 op/s 2024-05-03T18:34:15.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:34:15 smithi150 ceph-mon[32492]: pgmap v216: 129 pgs: 129 active+clean; 2.3 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 12 MiB/s wr, 655 op/s 2024-05-03T18:34:15.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:34:15 smithi033 ceph-mon[28330]: pgmap v216: 129 pgs: 129 active+clean; 2.3 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 12 MiB/s wr, 655 op/s 2024-05-03T18:34:16.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:34:16 smithi150 ceph-mon[32492]: pgmap v217: 129 pgs: 129 active+clean; 2.3 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 12 MiB/s wr, 655 op/s 2024-05-03T18:34:16.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:34:16 smithi033 ceph-mon[28330]: pgmap v217: 129 pgs: 129 active+clean; 2.3 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 12 MiB/s wr, 655 op/s 2024-05-03T18:34:16.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:34:16 smithi099 ceph-mon[31658]: pgmap v217: 129 pgs: 129 active+clean; 2.3 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 12 MiB/s wr, 655 op/s 2024-05-03T18:34:19.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:34:19 smithi099 ceph-mon[31658]: pgmap v218: 129 pgs: 129 active+clean; 2.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 17 MiB/s wr, 880 op/s 2024-05-03T18:34:19.384 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:34:19 smithi150 ceph-mon[32492]: pgmap v218: 129 pgs: 129 active+clean; 2.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 17 MiB/s wr, 880 op/s 2024-05-03T18:34:19.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:34:19 smithi033 ceph-mon[28330]: pgmap v218: 129 pgs: 129 active+clean; 2.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 17 MiB/s wr, 880 op/s 2024-05-03T18:34:20.111 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 185.0 s, 624.6 tps, lat 165.529 ms stddev 224.616 2024-05-03T18:34:20.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:34:20 smithi150 ceph-mon[32492]: pgmap v219: 129 pgs: 129 active+clean; 2.3 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 11 MiB/s wr, 642 op/s 2024-05-03T18:34:20.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:34:20 smithi033 ceph-mon[28330]: pgmap v219: 129 pgs: 129 active+clean; 2.3 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 11 MiB/s wr, 642 op/s 2024-05-03T18:34:20.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:34:20 smithi099 ceph-mon[31658]: pgmap v219: 129 pgs: 129 active+clean; 2.3 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 11 MiB/s wr, 642 op/s 2024-05-03T18:34:23.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:34:23 smithi099 ceph-mon[31658]: pgmap v220: 129 pgs: 129 active+clean; 2.3 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 10 MiB/s wr, 597 op/s 2024-05-03T18:34:23.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:34:23 smithi150 ceph-mon[32492]: pgmap v220: 129 pgs: 129 active+clean; 2.3 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 10 MiB/s wr, 597 op/s 2024-05-03T18:34:23.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:34:23 smithi033 ceph-mon[28330]: pgmap v220: 129 pgs: 129 active+clean; 2.3 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 10 MiB/s wr, 597 op/s 2024-05-03T18:34:24.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:34:24 smithi150 ceph-mon[32492]: pgmap v221: 129 pgs: 129 active+clean; 2.4 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 15 MiB/s wr, 870 op/s 2024-05-03T18:34:24.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:34:24 smithi033 ceph-mon[28330]: pgmap v221: 129 pgs: 129 active+clean; 2.4 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 15 MiB/s wr, 870 op/s 2024-05-03T18:34:24.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:34:24 smithi099 ceph-mon[31658]: pgmap v221: 129 pgs: 129 active+clean; 2.4 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 15 MiB/s wr, 870 op/s 2024-05-03T18:34:25.111 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 190.0 s, 652.9 tps, lat 178.099 ms stddev 224.793 2024-05-03T18:34:27.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:34:27 smithi099 ceph-mon[31658]: pgmap v222: 129 pgs: 129 active+clean; 2.4 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 9.8 MiB/s wr, 558 op/s 2024-05-03T18:34:27.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:34:27 smithi150 ceph-mon[32492]: pgmap v222: 129 pgs: 129 active+clean; 2.4 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 9.8 MiB/s wr, 558 op/s 2024-05-03T18:34:27.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:34:27 smithi033 ceph-mon[28330]: pgmap v222: 129 pgs: 129 active+clean; 2.4 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 9.8 MiB/s wr, 558 op/s 2024-05-03T18:34:28.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:34:28 smithi150 ceph-mon[32492]: pgmap v223: 129 pgs: 129 active+clean; 2.4 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 14 MiB/s wr, 817 op/s 2024-05-03T18:34:28.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:34:28 smithi033 ceph-mon[28330]: pgmap v223: 129 pgs: 129 active+clean; 2.4 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 14 MiB/s wr, 817 op/s 2024-05-03T18:34:28.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:34:28 smithi099 ceph-mon[31658]: pgmap v223: 129 pgs: 129 active+clean; 2.4 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 14 MiB/s wr, 817 op/s 2024-05-03T18:34:30.110 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 195.0 s, 538.6 tps, lat 185.436 ms stddev 240.155 2024-05-03T18:34:31.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:34:31 smithi099 ceph-mon[31658]: pgmap v224: 129 pgs: 129 active+clean; 2.4 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 10 MiB/s wr, 624 op/s 2024-05-03T18:34:31.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:34:31 smithi150 ceph-mon[32492]: pgmap v224: 129 pgs: 129 active+clean; 2.4 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 10 MiB/s wr, 624 op/s 2024-05-03T18:34:31.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:34:31 smithi033 ceph-mon[28330]: pgmap v224: 129 pgs: 129 active+clean; 2.4 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 10 MiB/s wr, 624 op/s 2024-05-03T18:34:31.608 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-03T18:34:32.281 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:34:32.295 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:34:32 smithi099 ceph-mon[31658]: pgmap v225: 129 pgs: 129 active+clean; 2.4 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 9.7 MiB/s wr, 579 op/s 2024-05-03T18:34:32.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:34:32 smithi150 ceph-mon[32492]: pgmap v225: 129 pgs: 129 active+clean; 2.4 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 9.7 MiB/s wr, 579 op/s 2024-05-03T18:34:32.432 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:34:32 smithi033 ceph-mon[28330]: pgmap v225: 129 pgs: 129 active+clean; 2.4 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 9.7 MiB/s wr, 579 op/s 2024-05-03T18:34:34.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:34:34 smithi150 ceph-mon[32492]: pgmap v226: 129 pgs: 129 active+clean; 2.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 15 MiB/s wr, 902 op/s 2024-05-03T18:34:34.392 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:34:34 smithi033 ceph-mon[28330]: pgmap v226: 129 pgs: 129 active+clean; 2.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 15 MiB/s wr, 902 op/s 2024-05-03T18:34:34.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:34:34 smithi099 ceph-mon[31658]: pgmap v226: 129 pgs: 129 active+clean; 2.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 15 MiB/s wr, 902 op/s 2024-05-03T18:34:34.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.950+0000 7f1657a00700 1 -- 172.21.15.33:0/3925991889 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f165006b180 msgr2=0x7f165006b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:34:34.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.950+0000 7f1657a00700 1 --2- 172.21.15.33:0/3925991889 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f165006b180 0x7f165006b600 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1638009990 tx=0x7f1638009ca0 comp rx=0 tx=0).stop 2024-05-03T18:34:34.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.950+0000 7f1657a00700 1 -- 172.21.15.33:0/3925991889 shutdown_connections 2024-05-03T18:34:34.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.950+0000 7f1657a00700 1 --2- 172.21.15.33:0/3925991889 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f165006bb40 0x7f1650110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:34.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.950+0000 7f1657a00700 1 --2- 172.21.15.33:0/3925991889 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1650109510 0x7f16501098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:34.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.950+0000 7f1657a00700 1 --2- 172.21.15.33:0/3925991889 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f165006b180 0x7f165006b600 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:34.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.950+0000 7f1657a00700 1 -- 172.21.15.33:0/3925991889 >> 172.21.15.33:0/3925991889 conn(0x7f1650079350 msgr2=0x7f1650079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:34:34.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.951+0000 7f1657a00700 1 -- 172.21.15.33:0/3925991889 shutdown_connections 2024-05-03T18:34:34.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.951+0000 7f1657a00700 1 -- 172.21.15.33:0/3925991889 wait complete. 2024-05-03T18:34:34.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.952+0000 7f1657a00700 1 Processor -- start 2024-05-03T18:34:34.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.953+0000 7f1657a00700 1 -- start start 2024-05-03T18:34:34.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.953+0000 7f1657a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f165006b180 0x7f1650110230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:34:34.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.953+0000 7f1657a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f165006bb40 0x7f165010d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:34:34.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.953+0000 7f1657a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1650109510 0x7f165010d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:34:34.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.953+0000 7f1657a00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1650103d70 con 0x7f165006b180 2024-05-03T18:34:34.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.953+0000 7f1657a00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1650103bf0 con 0x7f165006bb40 2024-05-03T18:34:34.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.953+0000 7f1657a00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f1650103ef0 con 0x7f1650109510 2024-05-03T18:34:34.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.954+0000 7f164fe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f165006bb40 0x7f165010d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:34:34.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.954+0000 7f164fe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f165006bb40 0x7f165010d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:52748/0 (socket says 172.21.15.33:52748) 2024-05-03T18:34:34.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.954+0000 7f164fe00700 1 -- 172.21.15.33:0/4104428777 learned_addr learned my addr 172.21.15.33:0/4104428777 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:34:34.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.954+0000 7f1655a00700 1 --2- 172.21.15.33:0/4104428777 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1650109510 0x7f165010d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:34:34.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.954+0000 7f164fe00700 1 -- 172.21.15.33:0/4104428777 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1650109510 msgr2=0x7f165010d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:34:34.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.954+0000 7f164fe00700 1 --2- 172.21.15.33:0/4104428777 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1650109510 0x7f165010d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:34.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.954+0000 7f164fe00700 1 -- 172.21.15.33:0/4104428777 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f165006b180 msgr2=0x7f1650110230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:34:34.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.954+0000 7f164fe00700 1 --2- 172.21.15.33:0/4104428777 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f165006b180 0x7f1650110230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:34.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.955+0000 7f164fe00700 1 -- 172.21.15.33:0/4104428777 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1650115f60 con 0x7f165006bb40 2024-05-03T18:34:34.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.955+0000 7f1655a00700 1 --2- 172.21.15.33:0/4104428777 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1650109510 0x7f165010d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:34:34.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.955+0000 7f164fe00700 1 --2- 172.21.15.33:0/4104428777 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f165006bb40 0x7f165010d2e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f16380098e0 tx=0x7f1638009550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:34:34.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.955+0000 7f164d600700 1 -- 172.21.15.33:0/4104428777 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1638027070 con 0x7f165006bb40 2024-05-03T18:34:34.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.955+0000 7f1657a00700 1 -- 172.21.15.33:0/4104428777 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16501161f0 con 0x7f165006bb40 2024-05-03T18:34:34.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.956+0000 7f1657a00700 1 -- 172.21.15.33:0/4104428777 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1650116720 con 0x7f165006bb40 2024-05-03T18:34:34.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.956+0000 7f164d600700 1 -- 172.21.15.33:0/4104428777 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1638022470 con 0x7f165006bb40 2024-05-03T18:34:34.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.956+0000 7f164d600700 1 -- 172.21.15.33:0/4104428777 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1638017650 con 0x7f165006bb40 2024-05-03T18:34:34.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.963+0000 7f164d600700 1 -- 172.21.15.33:0/4104428777 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f163802a070 con 0x7f165006bb40 2024-05-03T18:34:34.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.965+0000 7f164d600700 1 --2- 172.21.15.33:0/4104428777 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f16340b2af0 0x7f16340b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:34:34.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.965+0000 7f164d600700 1 -- 172.21.15.33:0/4104428777 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9518+0+0 (secure 0 0 0) 0x7f16380e3a20 con 0x7f165006bb40 2024-05-03T18:34:34.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.966+0000 7f1655000700 1 --2- 172.21.15.33:0/4104428777 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f16340b2af0 0x7f16340b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:34:34.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.968+0000 7f1655000700 1 --2- 172.21.15.33:0/4104428777 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f16340b2af0 0x7f16340b4fb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1640009b60 tx=0x7f1640009620 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:34:34.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.968+0000 7f166245db80 1 Processor -- start 2024-05-03T18:34:34.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.969+0000 7f166245db80 1 -- start start 2024-05-03T18:34:34.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.969+0000 7f166245db80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557d763d930 0x5557d763dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:34:34.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.969+0000 7f166245db80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5557d763e250 0x5557d7637f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:34:34.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.969+0000 7f166245db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5557d7638550 0x5557d76389d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:34:34.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.969+0000 7f166245db80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5557d758aa20 con 0x5557d763d930 2024-05-03T18:34:34.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.969+0000 7f166245db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5557d7528760 con 0x5557d7638550 2024-05-03T18:34:34.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.969+0000 7f166245db80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5557d75869e0 con 0x5557d763e250 2024-05-03T18:34:34.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.970+0000 7f1655000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557d763d930 0x5557d763dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:34:34.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.970+0000 7f1655a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5557d7638550 0x5557d76389d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:34:34.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.970+0000 7f1655a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5557d7638550 0x5557d76389d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:52758/0 (socket says 172.21.15.33:52758) 2024-05-03T18:34:34.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.970+0000 7f1655a00700 1 -- 172.21.15.33:0/3399573937 learned_addr learned my addr 172.21.15.33:0/3399573937 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:34:34.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.970+0000 7f1655a00700 1 -- 172.21.15.33:0/3399573937 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5557d763e250 msgr2=0x5557d7637f30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-03T18:34:34.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.970+0000 7f1655a00700 1 --2- 172.21.15.33:0/3399573937 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5557d763e250 0x5557d7637f30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:34.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.970+0000 7f1655a00700 1 -- 172.21.15.33:0/3399573937 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557d763d930 msgr2=0x5557d763dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:34:34.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.970+0000 7f1655a00700 1 --2- 172.21.15.33:0/3399573937 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557d763d930 0x5557d763dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:34.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.970+0000 7f1655a00700 1 -- 172.21.15.33:0/3399573937 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557d74d78d0 con 0x5557d7638550 2024-05-03T18:34:34.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.971+0000 7f1655a00700 1 --2- 172.21.15.33:0/3399573937 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5557d7638550 0x5557d76389d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1648009d20 tx=0x7f16480077e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8af7db0a00bdc8fb server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:34:34.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.972+0000 7f1646000700 1 -- 172.21.15.33:0/3399573937 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1648004db0 con 0x5557d7638550 2024-05-03T18:34:34.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.972+0000 7f1646000700 1 -- 172.21.15.33:0/3399573937 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f16480054b0 con 0x5557d7638550 2024-05-03T18:34:34.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.972+0000 7f1646000700 1 -- 172.21.15.33:0/3399573937 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1648004db0 con 0x5557d7638550 2024-05-03T18:34:34.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.974+0000 7f166245db80 1 -- 172.21.15.33:0/3399573937 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5557d7638550 msgr2=0x5557d76389d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:34:34.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.974+0000 7f166245db80 1 --2- 172.21.15.33:0/3399573937 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5557d7638550 0x5557d76389d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1648009d20 tx=0x7f16480077e0 comp rx=0 tx=0).stop 2024-05-03T18:34:34.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.975+0000 7f166245db80 1 -- 172.21.15.33:0/3399573937 shutdown_connections 2024-05-03T18:34:34.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.975+0000 7f166245db80 1 --2- 172.21.15.33:0/3399573937 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5557d7638550 0x5557d76389d0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:34.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.975+0000 7f166245db80 1 --2- 172.21.15.33:0/3399573937 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557d763d930 0x5557d763dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:34.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.975+0000 7f166245db80 1 --2- 172.21.15.33:0/3399573937 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5557d763e250 0x5557d7637f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:34.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.975+0000 7f166245db80 1 -- 172.21.15.33:0/3399573937 >> 172.21.15.33:0/3399573937 conn(0x5557d75b06d0 msgr2=0x5557d7632900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:34:34.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.976+0000 7f166245db80 1 -- 172.21.15.33:0/3399573937 shutdown_connections 2024-05-03T18:34:34.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.976+0000 7f166245db80 1 -- 172.21.15.33:0/3399573937 wait complete. 2024-05-03T18:34:34.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.978+0000 7f166245db80 1 -- start start 2024-05-03T18:34:34.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.978+0000 7f166245db80 1 Processor -- start 2024-05-03T18:34:34.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.979+0000 7f166245db80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557d7638550 0x5557d7643f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:34:34.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.979+0000 7f166245db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5557d763d930 0x5557d76412e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:34:34.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.979+0000 7f166245db80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5557d763e250 0x5557d7641820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:34:34.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.979+0000 7f166245db80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5557d7655d10 con 0x5557d7638550 2024-05-03T18:34:34.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.979+0000 7f166245db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5557d74f0b20 con 0x5557d763d930 2024-05-03T18:34:34.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.979+0000 7f166245db80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5557d7655e90 con 0x5557d763e250 2024-05-03T18:34:34.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.979+0000 7f166245db80 10 client.0 _ll_register_callbacks cb 0x5557d7531c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:34:34.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.980+0000 7f1655a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5557d763e250 0x5557d7641820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:34:34.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.980+0000 7f1655a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5557d763e250 0x5557d7641820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:45500/0 (socket says 172.21.15.33:45500) 2024-05-03T18:34:34.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.980+0000 7f1655a00700 1 -- 172.21.15.33:0/3561908212 learned_addr learned my addr 172.21.15.33:0/3561908212 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:34:34.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.980+0000 7f164fe00700 1 --2- 172.21.15.33:0/3561908212 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5557d763d930 0x5557d76412e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:34:34.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.980+0000 7f1655000700 1 --2- 172.21.15.33:0/3561908212 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557d7638550 0x5557d7643f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:34:34.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.981+0000 7f1655a00700 1 -- 172.21.15.33:0/3561908212 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5557d763d930 msgr2=0x5557d76412e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:34:34.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.981+0000 7f1655a00700 1 --2- 172.21.15.33:0/3561908212 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5557d763d930 0x5557d76412e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:34.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.981+0000 7f1655a00700 1 -- 172.21.15.33:0/3561908212 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557d7638550 msgr2=0x5557d7643f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:34:34.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.981+0000 7f1655a00700 1 --2- 172.21.15.33:0/3561908212 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557d7638550 0x5557d7643f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:34.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.981+0000 7f1655a00700 1 -- 172.21.15.33:0/3561908212 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557d7550420 con 0x5557d763e250 2024-05-03T18:34:34.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.982+0000 7f1655a00700 1 --2- 172.21.15.33:0/3561908212 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5557d763e250 0x5557d7641820 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1648005000 tx=0x7f1648005b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ceb4a99c331c46d server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:34:34.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.983+0000 7f163ea00700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-03T18:34:34.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.983+0000 7f163ea00700 1 -- 172.21.15.33:0/3561908212 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1648022070 con 0x5557d763e250 2024-05-03T18:34:34.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.983+0000 7f163ea00700 1 -- 172.21.15.33:0/3561908212 <== mon.1 v2:172.21.15.150:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9518+0+0 (secure 0 0 0) 0x7f164801e870 con 0x5557d763e250 2024-05-03T18:34:34.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.984+0000 7f166245db80 1 -- 172.21.15.33:0/3561908212 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16480054b0 con 0x5557d763e250 2024-05-03T18:34:34.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.986+0000 7f163ea00700 1 -- 172.21.15.33:0/3561908212 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f164802c050 con 0x5557d763e250 2024-05-03T18:34:34.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.986+0000 7f166245db80 10 client.25816 fetch_fsmap learned FSMap version 21 2024-05-03T18:34:34.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.986+0000 7f166245db80 1 -- 172.21.15.33:0/3561908212 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5557d7660300 con 0x5557d763e250 2024-05-03T18:34:34.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.987+0000 7f163ea00700 1 -- 172.21.15.33:0/3561908212 <== mon.1 v2:172.21.15.150:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f16480054b0 con 0x5557d763e250 2024-05-03T18:34:34.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.989+0000 7f163ea00700 1 -- 172.21.15.33:0/3561908212 <== mon.1 v2:172.21.15.150:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f1648027690 con 0x5557d763e250 2024-05-03T18:34:34.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.989+0000 7f166245db80 10 client.25816 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:34:34.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.989+0000 7f166245db80 10 client.25816 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:34:34.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.989+0000 7f166245db80 20 client.25816 populate_metadata read hostname 'smithi033' 2024-05-03T18:34:34.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.989+0000 7f166245db80 1 --2- 172.21.15.33:0/3561908212 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5557d76628e0 0x5557d7664cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:34:34.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.989+0000 7f166245db80 4 client.25816 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:34:34.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.989+0000 7f166245db80 1 -- 172.21.15.33:0/3561908212 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5557d7604ff0 con 0x5557d76628e0 2024-05-03T18:34:34.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.990+0000 7f1655000700 1 --2- 172.21.15.33:0/3561908212 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5557d76628e0 0x5557d7664cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:34:34.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.991+0000 7f1655000700 1 --2- 172.21.15.33:0/3561908212 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5557d76628e0 0x5557d7664cd0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a963a7b4f5be030 server_cookie=19dd318aaa4acf5d in_seq=0 out_seq=0 2024-05-03T18:34:34.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.991+0000 7f163ea00700 10 client.25816 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:34:34.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.993+0000 7f163ea00700 1 -- 172.21.15.33:0/3561908212 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5557d7604ff0 con 0x5557d76628e0 2024-05-03T18:34:34.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.993+0000 7f163ea00700 10 client.25816 handle_command_reply: tid=0 2024-05-03T18:34:34.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.993+0000 7f166245db80 1 client.25816 shutdown 2024-05-03T18:34:34.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.994+0000 7f166245db80 1 -- 172.21.15.33:0/3561908212 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5557d763e250 msgr2=0x5557d7641820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:34:34.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.994+0000 7f166245db80 1 --2- 172.21.15.33:0/3561908212 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5557d763e250 0x5557d7641820 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1648005000 tx=0x7f1648005b60 comp rx=0 tx=0).stop 2024-05-03T18:34:34.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.994+0000 7f166245db80 1 -- 172.21.15.33:0/3561908212 shutdown_connections 2024-05-03T18:34:34.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.994+0000 7f166245db80 1 --2- 172.21.15.33:0/3561908212 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5557d76628e0 0x5557d7664cd0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:34.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.994+0000 7f166245db80 1 --2- 172.21.15.33:0/3561908212 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5557d763e250 0x5557d7641820 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:34.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.994+0000 7f166245db80 1 --2- 172.21.15.33:0/3561908212 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557d7638550 0x5557d7643f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:34.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.994+0000 7f166245db80 1 --2- 172.21.15.33:0/3561908212 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5557d763d930 0x5557d76412e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:34.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.994+0000 7f166245db80 1 -- 172.21.15.33:0/3561908212 >> 172.21.15.33:0/3561908212 conn(0x5557d75b06d0 msgr2=0x5557d762d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:34:34.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.994+0000 7f163ea00700 0 client.25816 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:34:34.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.994+0000 7f166245db80 1 -- 172.21.15.33:0/3561908212 shutdown_connections 2024-05-03T18:34:34.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.994+0000 7f166245db80 1 -- 172.21.15.33:0/3561908212 wait complete. 2024-05-03T18:34:34.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:34.994+0000 7f166245db80 20 client.25816 trim_cache size 0 max 16384 2024-05-03T18:34:35.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:35.034+0000 7f166245db80 1 Processor -- start 2024-05-03T18:34:35.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:35.035+0000 7f166245db80 1 -- start start 2024-05-03T18:34:35.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:35.035+0000 7f166245db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5557d7638550 0x5557d7637f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:34:35.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:35.035+0000 7f166245db80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557d763d930 0x5557d765eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:34:35.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:35.035+0000 7f166245db80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5557d763e250 0x5557d763ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:34:35.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:35.035+0000 7f166245db80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f165011d660 con 0x5557d763d930 2024-05-03T18:34:35.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:35.035+0000 7f166245db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f165006e610 con 0x5557d7638550 2024-05-03T18:34:35.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:35.035+0000 7f166245db80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f165011dcb0 con 0x5557d763e250 2024-05-03T18:34:35.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:35.036+0000 7f1655000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5557d7638550 0x5557d7637f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:34:35.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:35.036+0000 7f1655000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5557d7638550 0x5557d7637f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:52780/0 (socket says 172.21.15.33:52780) 2024-05-03T18:34:35.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:35.036+0000 7f1655a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5557d763e250 0x5557d763ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:34:35.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:35.036+0000 7f1655a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5557d763e250 0x5557d763ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:45502/0 (socket says 172.21.15.33:45502) 2024-05-03T18:34:35.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:35.036+0000 7f1655a00700 1 -- 172.21.15.33:0/1768055262 learned_addr learned my addr 172.21.15.33:0/1768055262 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:34:35.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:35.037+0000 7f1655a00700 1 -- 172.21.15.33:0/1768055262 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5557d7638550 msgr2=0x5557d7637f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:34:35.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:35.037+0000 7f1655a00700 1 --2- 172.21.15.33:0/1768055262 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5557d7638550 0x5557d7637f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:35.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:35.037+0000 7f1655a00700 1 -- 172.21.15.33:0/1768055262 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557d763d930 msgr2=0x5557d765eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:34:35.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:35.037+0000 7f1655a00700 1 --2- 172.21.15.33:0/1768055262 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557d763d930 0x5557d765eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:35.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:35.037+0000 7f1655a00700 1 -- 172.21.15.33:0/1768055262 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557d7634570 con 0x5557d763e250 2024-05-03T18:34:35.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:35.038+0000 7f1655a00700 1 --2- 172.21.15.33:0/1768055262 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5557d763e250 0x5557d763ed60 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1648007c60 tx=0x7f1648014400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=195b940ab13ac7d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:34:35.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:35.038+0000 7f1645600700 1 -- 172.21.15.33:0/1768055262 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1648022040 con 0x5557d763e250 2024-05-03T18:34:35.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:35.038+0000 7f1645600700 1 -- 172.21.15.33:0/1768055262 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1648036070 con 0x5557d763e250 2024-05-03T18:34:35.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:35.038+0000 7f1645600700 1 -- 172.21.15.33:0/1768055262 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1648027730 con 0x5557d763e250 2024-05-03T18:34:35.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:35.039+0000 7f166245db80 1 -- 172.21.15.33:0/1768055262 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5557d763e250 msgr2=0x5557d763ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:34:35.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:35.039+0000 7f166245db80 1 --2- 172.21.15.33:0/1768055262 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5557d763e250 0x5557d763ed60 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1648007c60 tx=0x7f1648014400 comp rx=0 tx=0).stop 2024-05-03T18:34:35.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:35.040+0000 7f166245db80 1 -- 172.21.15.33:0/1768055262 shutdown_connections 2024-05-03T18:34:35.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:35.040+0000 7f166245db80 1 --2- 172.21.15.33:0/1768055262 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5557d763e250 0x5557d763ed60 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:35.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:35.040+0000 7f166245db80 1 --2- 172.21.15.33:0/1768055262 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5557d7638550 0x5557d7637f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:35.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:35.040+0000 7f166245db80 1 --2- 172.21.15.33:0/1768055262 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557d763d930 0x5557d765eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:35.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:35.040+0000 7f166245db80 1 -- 172.21.15.33:0/1768055262 >> 172.21.15.33:0/1768055262 conn(0x5557d75b06d0 msgr2=0x5557d763e650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:34:35.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:35.040+0000 7f166245db80 1 -- 172.21.15.33:0/1768055262 shutdown_connections 2024-05-03T18:34:35.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:35.041+0000 7f166245db80 1 -- 172.21.15.33:0/1768055262 wait complete. 2024-05-03T18:34:35.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:35.041+0000 7f166245db80 1 -- start start 2024-05-03T18:34:35.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:35.042+0000 7f166245db80 1 Processor -- start 2024-05-03T18:34:35.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:35.043+0000 7f166245db80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557d7638550 0x5557d7669ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:34:35.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:35.043+0000 7f166245db80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5557d763d930 0x5557d7669ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:34:35.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:35.044+0000 7f166245db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5557d763e250 0x5557d766a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:34:35.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:35.044+0000 7f166245db80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f165011e380 con 0x5557d7638550 2024-05-03T18:34:35.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:35.044+0000 7f166245db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f165006df40 con 0x5557d763e250 2024-05-03T18:34:35.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:35.044+0000 7f166245db80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f165006d1a0 con 0x5557d763d930 2024-05-03T18:34:35.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:35.044+0000 7f166245db80 10 client.0 _ll_register_callbacks cb 0x5557d7531c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:34:35.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:35.044+0000 7f1655000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557d7638550 0x5557d7669ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:34:35.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:35.045+0000 7f164fe00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5557d763d930 0x5557d7669ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:34:35.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:35.045+0000 7f1655a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5557d763e250 0x5557d766a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:34:35.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:35.045+0000 7f164fe00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5557d763d930 0x5557d7669ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:45516/0 (socket says 172.21.15.33:45516) 2024-05-03T18:34:35.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:35.045+0000 7f164fe00700 1 -- 172.21.15.33:0/3580605671 learned_addr learned my addr 172.21.15.33:0/3580605671 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:34:35.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:35.045+0000 7f1655000700 1 -- 172.21.15.33:0/3580605671 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5557d763d930 msgr2=0x5557d7669ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:34:35.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:35.045+0000 7f1655000700 1 --2- 172.21.15.33:0/3580605671 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5557d763d930 0x5557d7669ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:35.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:35.045+0000 7f1655000700 1 -- 172.21.15.33:0/3580605671 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5557d763e250 msgr2=0x5557d766a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:34:35.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:35.045+0000 7f1655000700 1 --2- 172.21.15.33:0/3580605671 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5557d763e250 0x5557d766a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:35.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:35.046+0000 7f1655000700 1 -- 172.21.15.33:0/3580605671 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557d76cbeb0 con 0x5557d7638550 2024-05-03T18:34:35.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:35.047+0000 7f1655000700 1 --2- 172.21.15.33:0/3580605671 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557d7638550 0x5557d7669ab0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f1640009fd0 tx=0x7f1640009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74f86bca6ef98843 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:34:35.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:35.047+0000 7f163ea00700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:34:35.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:35.048+0000 7f163ea00700 1 -- 172.21.15.33:0/3580605671 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1640004a30 con 0x5557d7638550 2024-05-03T18:34:35.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:35.048+0000 7f163ea00700 1 -- 172.21.15.33:0/3580605671 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9518+0+0 (secure 0 0 0) 0x7f1640004bd0 con 0x5557d7638550 2024-05-03T18:34:35.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:35.048+0000 7f166245db80 1 -- 172.21.15.33:0/3580605671 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1648036070 con 0x5557d7638550 2024-05-03T18:34:35.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:35.049+0000 7f163ea00700 1 -- 172.21.15.33:0/3580605671 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f164000e980 con 0x5557d7638550 2024-05-03T18:34:35.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:35.049+0000 7f166245db80 10 client.16131 fetch_fsmap learned FSMap version 21 2024-05-03T18:34:35.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:35.049+0000 7f166245db80 1 -- 172.21.15.33:0/3580605671 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5557d76cd1d0 con 0x5557d7638550 2024-05-03T18:34:35.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:35.049+0000 7f163ea00700 1 -- 172.21.15.33:0/3580605671 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1648036070 con 0x5557d7638550 2024-05-03T18:34:35.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:35.049+0000 7f163ea00700 1 -- 172.21.15.33:0/3580605671 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f16400052d0 con 0x5557d7638550 2024-05-03T18:34:35.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:35.050+0000 7f166245db80 10 client.16131 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:34:35.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:35.050+0000 7f166245db80 10 client.16131 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:34:35.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:35.050+0000 7f166245db80 20 client.16131 populate_metadata read hostname 'smithi033' 2024-05-03T18:34:35.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:35.050+0000 7f166245db80 1 --2- 172.21.15.33:0/3580605671 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5557d76cd890 0x5557d76cfcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:34:35.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:35.050+0000 7f166245db80 4 client.16131 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-03T18:34:35.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:35.050+0000 7f166245db80 1 -- 172.21.15.33:0/3580605671 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5557d7598400 con 0x5557d76cd890 2024-05-03T18:34:35.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:35.050+0000 7f164fe00700 1 --2- 172.21.15.33:0/3580605671 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5557d76cd890 0x5557d76cfcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:34:35.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:35.051+0000 7f163ea00700 10 client.16131 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:34:35.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:35.052+0000 7f164fe00700 1 --2- 172.21.15.33:0/3580605671 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5557d76cd890 0x5557d76cfcb0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=161627fab8dd57dd server_cookie=980c3b8d360506e9 in_seq=0 out_seq=0 2024-05-03T18:34:35.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:35.052+0000 7f163ea00700 1 -- 172.21.15.33:0/3580605671 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5557d7598400 con 0x5557d76cd890 2024-05-03T18:34:35.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:35.052+0000 7f163ea00700 10 client.16131 handle_command_reply: tid=0 2024-05-03T18:34:35.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:35.053+0000 7f166245db80 1 client.16131 shutdown 2024-05-03T18:34:35.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:35.054+0000 7f166245db80 1 -- 172.21.15.33:0/3580605671 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557d7638550 msgr2=0x5557d7669ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:34:35.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:35.054+0000 7f166245db80 1 --2- 172.21.15.33:0/3580605671 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557d7638550 0x5557d7669ab0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f1640009fd0 tx=0x7f1640009260 comp rx=0 tx=0).stop 2024-05-03T18:34:35.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:35.054+0000 7f166245db80 1 -- 172.21.15.33:0/3580605671 shutdown_connections 2024-05-03T18:34:35.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:35.054+0000 7f166245db80 1 --2- 172.21.15.33:0/3580605671 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5557d76cd890 0x5557d76cfcb0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:35.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:35.055+0000 7f166245db80 1 --2- 172.21.15.33:0/3580605671 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5557d763e250 0x5557d766a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:35.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:35.055+0000 7f166245db80 1 --2- 172.21.15.33:0/3580605671 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557d7638550 0x5557d7669ab0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:35.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:35.055+0000 7f166245db80 1 --2- 172.21.15.33:0/3580605671 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5557d763d930 0x5557d7669ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:35.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:35.055+0000 7f166245db80 1 -- 172.21.15.33:0/3580605671 >> 172.21.15.33:0/3580605671 conn(0x5557d75b06d0 msgr2=0x5557d762cb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:34:35.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:35.055+0000 7f163ea00700 0 client.16131 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:34:35.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:35.055+0000 7f166245db80 1 -- 172.21.15.33:0/3580605671 shutdown_connections 2024-05-03T18:34:35.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:35.055+0000 7f166245db80 1 -- 172.21.15.33:0/3580605671 wait complete. 2024-05-03T18:34:35.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:35.056+0000 7f166245db80 20 client.16131 trim_cache size 0 max 16384 2024-05-03T18:34:35.058 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-03T18:34:35.058 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-05-03T18:34:35.058 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-05-03T18:34:35.058 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-03T18:34:35.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:35.057+0000 7f1657a00700 1 -- 172.21.15.33:0/4104428777 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f16340b2af0 msgr2=0x7f16340b4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:34:35.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:35.057+0000 7f1657a00700 1 --2- 172.21.15.33:0/4104428777 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f16340b2af0 0x7f16340b4fb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1640009b60 tx=0x7f1640009620 comp rx=0 tx=0).stop 2024-05-03T18:34:35.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:35.057+0000 7f1657a00700 1 -- 172.21.15.33:0/4104428777 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f165006bb40 msgr2=0x7f165010d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:34:35.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:35.057+0000 7f1657a00700 1 --2- 172.21.15.33:0/4104428777 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f165006bb40 0x7f165010d2e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f16380098e0 tx=0x7f1638009550 comp rx=0 tx=0).stop 2024-05-03T18:34:35.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:35.057+0000 7f1657a00700 1 -- 172.21.15.33:0/4104428777 shutdown_connections 2024-05-03T18:34:35.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:35.057+0000 7f1657a00700 1 --2- 172.21.15.33:0/4104428777 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1650109510 0x7f165010d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:35.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:35.057+0000 7f1657a00700 1 --2- 172.21.15.33:0/4104428777 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f165006b180 0x7f1650110230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:35.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:35.057+0000 7f1657a00700 1 --2- 172.21.15.33:0/4104428777 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f16340b2af0 0x7f16340b4fb0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:35.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:35.057+0000 7f1657a00700 1 --2- 172.21.15.33:0/4104428777 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f165006bb40 0x7f165010d2e0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:35.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:35.057+0000 7f1657a00700 1 -- 172.21.15.33:0/4104428777 >> 172.21.15.33:0/4104428777 conn(0x7f1650079350 msgr2=0x7f16501011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:34:35.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:35.058+0000 7f1657a00700 1 -- 172.21.15.33:0/4104428777 shutdown_connections 2024-05-03T18:34:35.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:35.058+0000 7f1657a00700 1 -- 172.21.15.33:0/4104428777 wait complete. 2024-05-03T18:34:35.110 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 200.0 s, 600.0 tps, lat 167.484 ms stddev 225.088 2024-05-03T18:34:36.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:34:36 smithi150 ceph-mon[32492]: pgmap v227: 129 pgs: 129 active+clean; 2.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 10 MiB/s wr, 629 op/s 2024-05-03T18:34:36.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:34:36 smithi033 ceph-mon[28330]: pgmap v227: 129 pgs: 129 active+clean; 2.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 10 MiB/s wr, 629 op/s 2024-05-03T18:34:36.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:34:36 smithi099 ceph-mon[31658]: pgmap v227: 129 pgs: 129 active+clean; 2.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 10 MiB/s wr, 629 op/s 2024-05-03T18:34:36.692 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-03T18:34:36.692 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T18:34:37.184 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:34:38.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:34:38 smithi033 ceph-mon[28330]: pgmap v228: 129 pgs: 129 active+clean; 2.5 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 13 MiB/s wr, 823 op/s 2024-05-03T18:34:38.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:34:38 smithi150 ceph-mon[32492]: pgmap v228: 129 pgs: 129 active+clean; 2.5 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 13 MiB/s wr, 823 op/s 2024-05-03T18:34:38.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:34:38 smithi099 ceph-mon[31658]: pgmap v228: 129 pgs: 129 active+clean; 2.5 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 13 MiB/s wr, 823 op/s 2024-05-03T18:34:40.110 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 205.0 s, 544.8 tps, lat 184.800 ms stddev 284.615 2024-05-03T18:34:40.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:34:40 smithi150 ceph-mon[32492]: pgmap v229: 129 pgs: 129 active+clean; 2.5 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 9.1 MiB/s wr, 608 op/s 2024-05-03T18:34:40.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:34:40 smithi033 ceph-mon[28330]: pgmap v229: 129 pgs: 129 active+clean; 2.5 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 9.1 MiB/s wr, 608 op/s 2024-05-03T18:34:40.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:34:40 smithi099 ceph-mon[31658]: pgmap v229: 129 pgs: 129 active+clean; 2.5 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 9.1 MiB/s wr, 608 op/s 2024-05-03T18:34:42.221 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:34:42 smithi033 ceph-mon[28330]: pgmap v230: 129 pgs: 129 active+clean; 2.5 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 8.2 MiB/s wr, 560 op/s 2024-05-03T18:34:42.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:34:42 smithi150 ceph-mon[32492]: pgmap v230: 129 pgs: 129 active+clean; 2.5 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 8.2 MiB/s wr, 560 op/s 2024-05-03T18:34:42.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:42.456+0000 7f92aec00700 1 -- 172.21.15.33:0/1357657307 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92a811cc80 msgr2=0x7f92a811f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:34:42.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:42.456+0000 7f92aec00700 1 --2- 172.21.15.33:0/1357657307 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92a811cc80 0x7f92a811f070 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f92a000b0a0 tx=0x7f92a000b3b0 comp rx=0 tx=0).stop 2024-05-03T18:34:42.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:42.456+0000 7f92aec00700 1 -- 172.21.15.33:0/1357657307 shutdown_connections 2024-05-03T18:34:42.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:42.456+0000 7f92aec00700 1 --2- 172.21.15.33:0/1357657307 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92a811cc80 0x7f92a811f070 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:42.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:42.456+0000 7f92aec00700 1 --2- 172.21.15.33:0/1357657307 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f92a8111c80 0x7f92a810a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:42.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:42.456+0000 7f92aec00700 1 --2- 172.21.15.33:0/1357657307 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92a810a9e0 0x7f92a810ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:42.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:42.456+0000 7f92aec00700 1 -- 172.21.15.33:0/1357657307 >> 172.21.15.33:0/1357657307 conn(0x7f92a806e610 msgr2=0x7f92a806ea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:34:42.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:42.458+0000 7f92aec00700 1 -- 172.21.15.33:0/1357657307 shutdown_connections 2024-05-03T18:34:42.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:42.458+0000 7f92aec00700 1 -- 172.21.15.33:0/1357657307 wait complete. 2024-05-03T18:34:42.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:42.459+0000 7f92aec00700 1 Processor -- start 2024-05-03T18:34:42.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:42.460+0000 7f92aec00700 1 -- start start 2024-05-03T18:34:42.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:42.461+0000 7f92aec00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92a810a9e0 0x7f92a81ae650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:34:42.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:42.461+0000 7f92aec00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f92a8111c80 0x7f92a81aeb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:34:42.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:42.461+0000 7f92aec00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92a811cc80 0x7f92a81a8720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:34:42.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:42.461+0000 7f92aec00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f92a81223c0 con 0x7f92a8111c80 2024-05-03T18:34:42.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:42.461+0000 7f92aec00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f92a8122240 con 0x7f92a811cc80 2024-05-03T18:34:42.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:42.461+0000 7f92aec00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f92a8122540 con 0x7f92a810a9e0 2024-05-03T18:34:42.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:42.462+0000 7f92a7e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92a810a9e0 0x7f92a81ae650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:34:42.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:42.462+0000 7f92a7e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92a810a9e0 0x7f92a81ae650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:49396/0 (socket says 172.21.15.33:49396) 2024-05-03T18:34:42.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:42.462+0000 7f92a7e00700 1 -- 172.21.15.33:0/232249350 learned_addr learned my addr 172.21.15.33:0/232249350 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:34:42.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:42.462+0000 7f92a7400700 1 --2- 172.21.15.33:0/232249350 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f92a8111c80 0x7f92a81aeb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:34:42.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:42.462+0000 7f92acc00700 1 --2- 172.21.15.33:0/232249350 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92a811cc80 0x7f92a81a8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:34:42.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:42.463+0000 7f92a7e00700 1 -- 172.21.15.33:0/232249350 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92a811cc80 msgr2=0x7f92a81a8720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:34:42.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:42.463+0000 7f92a7e00700 1 --2- 172.21.15.33:0/232249350 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92a811cc80 0x7f92a81a8720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:42.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:42.463+0000 7f92a7e00700 1 -- 172.21.15.33:0/232249350 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f92a8111c80 msgr2=0x7f92a81aeb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:34:42.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:42.463+0000 7f92a7e00700 1 --2- 172.21.15.33:0/232249350 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f92a8111c80 0x7f92a81aeb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:42.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:42.463+0000 7f92a7e00700 1 -- 172.21.15.33:0/232249350 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92a81a9000 con 0x7f92a810a9e0 2024-05-03T18:34:42.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:42.464+0000 7f92a7e00700 1 --2- 172.21.15.33:0/232249350 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92a810a9e0 0x7f92a81ae650 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f929800d980 tx=0x7f929800dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:34:42.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:42.464+0000 7f92a4c00700 1 -- 172.21.15.33:0/232249350 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9298016070 con 0x7f92a810a9e0 2024-05-03T18:34:42.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:42.464+0000 7f92a4c00700 1 -- 172.21.15.33:0/232249350 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9298004830 con 0x7f92a810a9e0 2024-05-03T18:34:42.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:42.464+0000 7f92a4c00700 1 -- 172.21.15.33:0/232249350 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9298005440 con 0x7f92a810a9e0 2024-05-03T18:34:42.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:42.464+0000 7f92aec00700 1 -- 172.21.15.33:0/232249350 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92a81a9290 con 0x7f92a810a9e0 2024-05-03T18:34:42.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:42.464+0000 7f92aec00700 1 -- 172.21.15.33:0/232249350 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92a81b5400 con 0x7f92a810a9e0 2024-05-03T18:34:42.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:42.467+0000 7f92aec00700 1 -- 172.21.15.33:0/232249350 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92a806fba0 con 0x7f92a810a9e0 2024-05-03T18:34:42.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:42.470+0000 7f92a4c00700 1 -- 172.21.15.33:0/232249350 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f92980055e0 con 0x7f92a810a9e0 2024-05-03T18:34:42.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:42.473+0000 7f92a4c00700 1 --2- 172.21.15.33:0/232249350 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f928c0b2af0 0x7f928c0b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:34:42.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:42.473+0000 7f92a4c00700 1 -- 172.21.15.33:0/232249350 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9518+0+0 (secure 0 0 0) 0x7f92980e2720 con 0x7f92a810a9e0 2024-05-03T18:34:42.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:42.474+0000 7f92a7400700 1 --2- 172.21.15.33:0/232249350 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f928c0b2af0 0x7f928c0b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:34:42.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:42.475+0000 7f92a7400700 1 --2- 172.21.15.33:0/232249350 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f928c0b2af0 0x7f928c0b4fb0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9294005950 tx=0x7f92940058e0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:34:42.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:42.480+0000 7f92a4c00700 1 -- 172.21.15.33:0/232249350 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f929801f020 con 0x7f92a810a9e0 2024-05-03T18:34:42.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:34:42 smithi099 ceph-mon[31658]: pgmap v230: 129 pgs: 129 active+clean; 2.5 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 8.2 MiB/s wr, 560 op/s 2024-05-03T18:34:42.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:42.980+0000 7f92aec00700 1 -- 172.21.15.33:0/232249350 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f92a806a190 con 0x7f92a810a9e0 2024-05-03T18:34:42.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:42.982+0000 7f92a4c00700 1 -- 172.21.15.33:0/232249350 <== mon.1 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11801 (secure 0 0 0) 0x7f92980a9cc0 con 0x7f92a810a9e0 2024-05-03T18:34:42.984 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:34:42.984 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1327393503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1327393503},{"type":"v1","addr":"172.21.15.33:6835","nonce":1327393503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/2461472948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":2461472948},{"type":"v1","addr":"172.21.15.33:6837","nonce":2461472948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14559,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/442269281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":442269281},{"type":"v1","addr":"172.21.15.33:6839","nonce":442269281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14571,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/645571756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":645571756},{"type":"v1","addr":"172.21.15.33:6841","nonce":645571756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/203598909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":203598909},{"type":"v1","addr":"172.21.15.150:6833","nonce":203598909}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/4029842493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":4029842493},{"type":"v1","addr":"172.21.15.150:6835","nonce":4029842493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3463664423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3463664423},{"type":"v1","addr":"172.21.15.150:6837","nonce":3463664423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24481,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6839/4051648018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":4051648018},{"type":"v1","addr":"172.21.15.150:6839","nonce":4051648018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34418,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1865105479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1865105479},{"type":"v1","addr":"172.21.15.99:6833","nonce":1865105479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34445,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/1206393914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1206393914},{"type":"v1","addr":"172.21.15.99:6835","nonce":1206393914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-03T18:22:28.837091+0000","modified":"2024-05-03T18:23:49.758398+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34508},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34472":{"gid":34472,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6837/3360970342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3360970342},{"type":"v1","addr":"172.21.15.99:6837","nonce":3360970342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34508":{"gid":34508,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/919907308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":919907308},{"type":"v1","addr":"172.21.15.99:6839","nonce":919907308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34508,"qdb_cluster":[34508]},"id":1}]} 2024-05-03T18:34:42.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:42.987+0000 7f92aec00700 1 -- 172.21.15.33:0/232249350 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f928c0b2af0 msgr2=0x7f928c0b4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:34:42.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:42.987+0000 7f92aec00700 1 --2- 172.21.15.33:0/232249350 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f928c0b2af0 0x7f928c0b4fb0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9294005950 tx=0x7f92940058e0 comp rx=0 tx=0).stop 2024-05-03T18:34:42.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:42.987+0000 7f92aec00700 1 -- 172.21.15.33:0/232249350 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92a810a9e0 msgr2=0x7f92a81ae650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:34:42.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:42.987+0000 7f92aec00700 1 --2- 172.21.15.33:0/232249350 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92a810a9e0 0x7f92a81ae650 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f929800d980 tx=0x7f929800dc90 comp rx=0 tx=0).stop 2024-05-03T18:34:42.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:42.987+0000 7f92aec00700 1 -- 172.21.15.33:0/232249350 shutdown_connections 2024-05-03T18:34:42.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:42.987+0000 7f92aec00700 1 --2- 172.21.15.33:0/232249350 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f928c0b2af0 0x7f928c0b4fb0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:42.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:42.987+0000 7f92aec00700 1 --2- 172.21.15.33:0/232249350 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92a811cc80 0x7f92a81a8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:42.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:42.987+0000 7f92aec00700 1 --2- 172.21.15.33:0/232249350 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92a810a9e0 0x7f92a81ae650 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:42.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:42.987+0000 7f92aec00700 1 --2- 172.21.15.33:0/232249350 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f92a8111c80 0x7f92a81aeb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:42.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:42.987+0000 7f92aec00700 1 -- 172.21.15.33:0/232249350 >> 172.21.15.33:0/232249350 conn(0x7f92a806e610 msgr2=0x7f92a811ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:34:42.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:42.988+0000 7f92aec00700 1 -- 172.21.15.33:0/232249350 shutdown_connections 2024-05-03T18:34:42.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:42.988+0000 7f92aec00700 1 -- 172.21.15.33:0/232249350 wait complete. 2024-05-03T18:34:42.991 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 21 2024-05-03T18:34:43.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:34:43 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/232249350' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:34:43.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:34:43 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/232249350' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:34:43.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:34:43 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/232249350' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:34:43.939 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-03T18:34:44.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:34:44 smithi033 ceph-mon[28330]: pgmap v231: 129 pgs: 129 active+clean; 2.5 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 13 MiB/s wr, 848 op/s 2024-05-03T18:34:44.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:34:44 smithi150 ceph-mon[32492]: pgmap v231: 129 pgs: 129 active+clean; 2.5 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 13 MiB/s wr, 848 op/s 2024-05-03T18:34:44.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:34:44 smithi099 ceph-mon[31658]: pgmap v231: 129 pgs: 129 active+clean; 2.5 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 13 MiB/s wr, 848 op/s 2024-05-03T18:34:44.703 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:34:45.111 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 210.0 s, 551.7 tps, lat 138.552 ms stddev 41.960 2024-05-03T18:34:45.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:34:45 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:34:45.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:34:45 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:34:45.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:34:45 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:34:46.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:34:46 smithi150 ceph-mon[32492]: pgmap v232: 129 pgs: 129 active+clean; 2.5 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 7.8 MiB/s wr, 525 op/s 2024-05-03T18:34:46.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:34:46 smithi033 ceph-mon[28330]: pgmap v232: 129 pgs: 129 active+clean; 2.5 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 7.8 MiB/s wr, 525 op/s 2024-05-03T18:34:46.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:34:46 smithi099 ceph-mon[31658]: pgmap v232: 129 pgs: 129 active+clean; 2.5 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 7.8 MiB/s wr, 525 op/s 2024-05-03T18:34:47.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:34:47 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:34:47.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:34:47 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:34:47.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:34:47 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:34:47.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:34:47 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:34:47.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:34:47 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:34:47.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:34:47 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:34:47.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:34:47 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:34:47.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:34:47 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:34:47.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:34:47 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:34:47.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.582+0000 7f9a24c00700 1 -- 172.21.15.33:0/1764210278 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a2006bad0 msgr2=0x7f9a2010fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:34:47.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.582+0000 7f9a24c00700 1 --2- 172.21.15.33:0/1764210278 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a2006bad0 0x7f9a2010fa10 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f9a080098f0 tx=0x7f9a08009c00 comp rx=0 tx=0).stop 2024-05-03T18:34:47.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.582+0000 7f9a24c00700 1 -- 172.21.15.33:0/1764210278 shutdown_connections 2024-05-03T18:34:47.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.582+0000 7f9a24c00700 1 --2- 172.21.15.33:0/1764210278 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a2010ff50 0x7f9a20112340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:47.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.582+0000 7f9a24c00700 1 --2- 172.21.15.33:0/1764210278 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a2006b1b0 0x7f9a2006b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:47.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.582+0000 7f9a24c00700 1 --2- 172.21.15.33:0/1764210278 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a2006bad0 0x7f9a2010fa10 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:47.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.582+0000 7f9a24c00700 1 -- 172.21.15.33:0/1764210278 >> 172.21.15.33:0/1764210278 conn(0x7f9a20079350 msgr2=0x7f9a20079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:34:47.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.582+0000 7f9a24c00700 1 -- 172.21.15.33:0/1764210278 shutdown_connections 2024-05-03T18:34:47.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.583+0000 7f9a24c00700 1 -- 172.21.15.33:0/1764210278 wait complete. 2024-05-03T18:34:47.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.584+0000 7f9a24c00700 1 Processor -- start 2024-05-03T18:34:47.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.584+0000 7f9a24c00700 1 -- start start 2024-05-03T18:34:47.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.585+0000 7f9a24c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a2006b1b0 0x7f9a2019bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:34:47.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.586+0000 7f9a24c00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a2006bad0 0x7f9a2019c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:34:47.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.586+0000 7f9a24c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a2010ff50 0x7f9a201a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:34:47.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.586+0000 7f9a24c00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9a20115760 con 0x7f9a2006bad0 2024-05-03T18:34:47.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.586+0000 7f9a24c00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9a201155e0 con 0x7f9a2010ff50 2024-05-03T18:34:47.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.586+0000 7f9a24c00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9a201158e0 con 0x7f9a2006b1b0 2024-05-03T18:34:47.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.586+0000 7f9a17e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a2006bad0 0x7f9a2019c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:34:47.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.586+0000 7f9a1de00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a2006b1b0 0x7f9a2019bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:34:47.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.586+0000 7f9a17e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a2006bad0 0x7f9a2019c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49704/0 (socket says 172.21.15.33:49704) 2024-05-03T18:34:47.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.586+0000 7f9a17e00700 1 -- 172.21.15.33:0/2563879249 learned_addr learned my addr 172.21.15.33:0/2563879249 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:34:47.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.587+0000 7f9a1de00700 1 -- 172.21.15.33:0/2563879249 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a2010ff50 msgr2=0x7f9a201a0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:34:47.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.587+0000 7f9a1de00700 1 --2- 172.21.15.33:0/2563879249 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a2010ff50 0x7f9a201a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:47.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.587+0000 7f9a1de00700 1 -- 172.21.15.33:0/2563879249 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a2006bad0 msgr2=0x7f9a2019c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:34:47.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.587+0000 7f9a1de00700 1 --2- 172.21.15.33:0/2563879249 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a2006bad0 0x7f9a2019c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:47.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.587+0000 7f9a1de00700 1 -- 172.21.15.33:0/2563879249 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a201a0da0 con 0x7f9a2006b1b0 2024-05-03T18:34:47.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.587+0000 7f9a17e00700 1 --2- 172.21.15.33:0/2563879249 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a2006bad0 0x7f9a2019c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:34:47.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.588+0000 7f9a1de00700 1 --2- 172.21.15.33:0/2563879249 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a2006b1b0 0x7f9a2019bd90 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9a1000ef10 tx=0x7f9a1000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:34:47.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.588+0000 7f9a16a00700 1 -- 172.21.15.33:0/2563879249 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a1000ca70 con 0x7f9a2006b1b0 2024-05-03T18:34:47.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.589+0000 7f9a16a00700 1 -- 172.21.15.33:0/2563879249 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9a10019070 con 0x7f9a2006b1b0 2024-05-03T18:34:47.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.589+0000 7f9a16a00700 1 -- 172.21.15.33:0/2563879249 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a100146e0 con 0x7f9a2006b1b0 2024-05-03T18:34:47.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.589+0000 7f9a24c00700 1 -- 172.21.15.33:0/2563879249 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a201a1030 con 0x7f9a2006b1b0 2024-05-03T18:34:47.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.589+0000 7f9a24c00700 1 -- 172.21.15.33:0/2563879249 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a201a8910 con 0x7f9a2006b1b0 2024-05-03T18:34:47.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.600+0000 7f9a16a00700 1 -- 172.21.15.33:0/2563879249 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f9a10012050 con 0x7f9a2006b1b0 2024-05-03T18:34:47.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.602+0000 7f9a2f4c8b80 1 Processor -- start 2024-05-03T18:34:47.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.602+0000 7f9a2f4c8b80 1 -- start start 2024-05-03T18:34:47.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.602+0000 7f9a2f4c8b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d82ea1c130 0x55d82ea1c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:34:47.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.602+0000 7f9a2f4c8b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d82ea1ca50 0x55d82ea167c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:34:47.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.603+0000 7f9a2f4c8b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d82ea16de0 0x55d82ea17240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:34:47.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.603+0000 7f9a2f4c8b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d82e960a20 con 0x55d82ea16de0 2024-05-03T18:34:47.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.603+0000 7f9a2f4c8b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55d82e8fe760 con 0x55d82ea1ca50 2024-05-03T18:34:47.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.603+0000 7f9a2f4c8b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55d82e95c9e0 con 0x55d82ea1c130 2024-05-03T18:34:47.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.603+0000 7f9a1e800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d82ea1c130 0x55d82ea1c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:34:47.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.604+0000 7f9a1e800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d82ea1c130 0x55d82ea1c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:40836/0 (socket says 172.21.15.33:40836) 2024-05-03T18:34:47.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.604+0000 7f9a1e800700 1 -- 172.21.15.33:0/1771983429 learned_addr learned my addr 172.21.15.33:0/1771983429 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:34:47.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.604+0000 7f9a17e00700 1 --2- 172.21.15.33:0/1771983429 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d82ea16de0 0x55d82ea17240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:34:47.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.604+0000 7f9a1de00700 1 --2- 172.21.15.33:0/1771983429 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d82ea1ca50 0x55d82ea167c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:34:47.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.604+0000 7f9a1e800700 1 -- 172.21.15.33:0/1771983429 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d82ea1ca50 msgr2=0x55d82ea167c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:34:47.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.604+0000 7f9a1e800700 1 --2- 172.21.15.33:0/1771983429 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d82ea1ca50 0x55d82ea167c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:47.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.604+0000 7f9a1e800700 1 -- 172.21.15.33:0/1771983429 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d82ea16de0 msgr2=0x55d82ea17240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:34:47.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.604+0000 7f9a1e800700 1 --2- 172.21.15.33:0/1771983429 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d82ea16de0 0x55d82ea17240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:47.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.604+0000 7f9a1e800700 1 -- 172.21.15.33:0/1771983429 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d82e8ad8d0 con 0x55d82ea1c130 2024-05-03T18:34:47.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.605+0000 7f9a1e800700 1 --2- 172.21.15.33:0/1771983429 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d82ea1c130 0x55d82ea1c510 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9a18009be0 tx=0x7f9a18009ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6681974df20b10e server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:34:47.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.606+0000 7f9a16a00700 1 --2- 172.21.15.33:0/2563879249 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f9a040b2a70 0x7f9a040b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:34:47.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.606+0000 7f9a16a00700 1 -- 172.21.15.33:0/2563879249 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9518+0+0 (secure 0 0 0) 0x7f9a100e29a0 con 0x7f9a2006b1b0 2024-05-03T18:34:47.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.607+0000 7f9a0f400700 1 -- 172.21.15.33:0/1771983429 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a1800ebf0 con 0x55d82ea1c130 2024-05-03T18:34:47.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.607+0000 7f9a0f400700 1 -- 172.21.15.33:0/1771983429 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9a180044c0 con 0x55d82ea1c130 2024-05-03T18:34:47.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.607+0000 7f9a0f400700 1 -- 172.21.15.33:0/1771983429 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a18003e00 con 0x55d82ea1c130 2024-05-03T18:34:47.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.609+0000 7f9a2f4c8b80 1 -- 172.21.15.33:0/1771983429 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d82ea1c130 msgr2=0x55d82ea1c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:34:47.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.609+0000 7f9a2f4c8b80 1 --2- 172.21.15.33:0/1771983429 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d82ea1c130 0x55d82ea1c510 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9a18009be0 tx=0x7f9a18009ef0 comp rx=0 tx=0).stop 2024-05-03T18:34:47.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.609+0000 7f9a2f4c8b80 1 -- 172.21.15.33:0/1771983429 shutdown_connections 2024-05-03T18:34:47.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.609+0000 7f9a2f4c8b80 1 --2- 172.21.15.33:0/1771983429 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d82ea16de0 0x55d82ea17240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:47.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.609+0000 7f9a2f4c8b80 1 --2- 172.21.15.33:0/1771983429 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d82ea1c130 0x55d82ea1c510 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:47.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.609+0000 7f9a2f4c8b80 1 --2- 172.21.15.33:0/1771983429 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d82ea1ca50 0x55d82ea167c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:47.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.609+0000 7f9a2f4c8b80 1 -- 172.21.15.33:0/1771983429 >> 172.21.15.33:0/1771983429 conn(0x55d82e93cb60 msgr2=0x55d82ea087f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:34:47.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.609+0000 7f9a1de00700 1 --2- 172.21.15.33:0/2563879249 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f9a040b2a70 0x7f9a040b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:34:47.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.610+0000 7f9a2f4c8b80 1 -- 172.21.15.33:0/1771983429 shutdown_connections 2024-05-03T18:34:47.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.610+0000 7f9a2f4c8b80 1 -- 172.21.15.33:0/1771983429 wait complete. 2024-05-03T18:34:47.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.611+0000 7f9a2f4c8b80 1 -- start start 2024-05-03T18:34:47.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.611+0000 7f9a2f4c8b80 1 Processor -- start 2024-05-03T18:34:47.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.613+0000 7f9a1de00700 1 --2- 172.21.15.33:0/2563879249 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f9a040b2a70 0x7f9a040b4f30 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9a100dfdc0 tx=0x7f9a100e2ca0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:34:47.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.614+0000 7f9a2f4c8b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d82ea16de0 0x55d82ea22810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:34:47.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.614+0000 7f9a2f4c8b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d82ea1c130 0x55d82ea1fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:34:47.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.614+0000 7f9a2f4c8b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d82ea1ca50 0x55d82ea200f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:34:47.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.614+0000 7f9a2f4c8b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d82ea23520 con 0x55d82ea1ca50 2024-05-03T18:34:47.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.614+0000 7f9a2f4c8b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55d82e8c6b20 con 0x55d82ea1c130 2024-05-03T18:34:47.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.614+0000 7f9a2f4c8b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55d82ea236a0 con 0x55d82ea16de0 2024-05-03T18:34:47.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.614+0000 7f9a2f4c8b80 10 client.0 _ll_register_callbacks cb 0x55d82e907c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:34:47.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.615+0000 7f9a17e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d82ea1c130 0x55d82ea1fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:34:47.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.615+0000 7f9a17e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d82ea1c130 0x55d82ea1fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:50378/0 (socket says 172.21.15.33:50378) 2024-05-03T18:34:47.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.615+0000 7f9a17e00700 1 -- 172.21.15.33:0/2609901891 learned_addr learned my addr 172.21.15.33:0/2609901891 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:34:47.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.615+0000 7f9a17e00700 1 -- 172.21.15.33:0/2609901891 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d82ea16de0 msgr2=0x55d82ea22810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:34:47.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.615+0000 7f9a17e00700 1 --2- 172.21.15.33:0/2609901891 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d82ea16de0 0x55d82ea22810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:47.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.615+0000 7f9a17e00700 1 -- 172.21.15.33:0/2609901891 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d82ea1ca50 msgr2=0x55d82ea200f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:34:47.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.615+0000 7f9a17e00700 1 --2- 172.21.15.33:0/2609901891 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d82ea1ca50 0x55d82ea200f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:47.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.615+0000 7f9a17e00700 1 -- 172.21.15.33:0/2609901891 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d82e926420 con 0x55d82ea1c130 2024-05-03T18:34:47.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.617+0000 7f9a17e00700 1 --2- 172.21.15.33:0/2609901891 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d82ea1c130 0x55d82ea1fbb0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f9a08005950 tx=0x7f9a0800f8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a60da6d07c51a14a server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:34:47.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.617+0000 7f9a03e00700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-03T18:34:47.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.619+0000 7f9a03e00700 1 -- 172.21.15.33:0/2609901891 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a080054d0 con 0x55d82ea1c130 2024-05-03T18:34:47.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.619+0000 7f9a03e00700 1 -- 172.21.15.33:0/2609901891 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9518+0+0 (secure 0 0 0) 0x7f9a08005670 con 0x55d82ea1c130 2024-05-03T18:34:47.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.619+0000 7f9a2f4c8b80 1 -- 172.21.15.33:0/2609901891 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a180044c0 con 0x55d82ea1c130 2024-05-03T18:34:47.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.619+0000 7f9a03e00700 1 -- 172.21.15.33:0/2609901891 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f9a08030050 con 0x55d82ea1c130 2024-05-03T18:34:47.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.620+0000 7f9a2f4c8b80 10 client.35792 fetch_fsmap learned FSMap version 21 2024-05-03T18:34:47.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.620+0000 7f9a2f4c8b80 1 -- 172.21.15.33:0/2609901891 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55d82ea0f410 con 0x55d82ea1c130 2024-05-03T18:34:47.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.620+0000 7f9a03e00700 1 -- 172.21.15.33:0/2609901891 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9a180044c0 con 0x55d82ea1c130 2024-05-03T18:34:47.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.621+0000 7f9a03e00700 1 -- 172.21.15.33:0/2609901891 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f9a0800b760 con 0x55d82ea1c130 2024-05-03T18:34:47.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.621+0000 7f9a2f4c8b80 10 client.35792 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:34:47.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.621+0000 7f9a2f4c8b80 10 client.35792 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:34:47.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.621+0000 7f9a2f4c8b80 20 client.35792 populate_metadata read hostname 'smithi033' 2024-05-03T18:34:47.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.621+0000 7f9a2f4c8b80 1 --2- 172.21.15.33:0/2609901891 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55d82ea11ee0 0x55d82ea14320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:34:47.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.621+0000 7f9a2f4c8b80 4 client.35792 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:34:47.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.621+0000 7f9a2f4c8b80 1 -- 172.21.15.33:0/2609901891 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d82e9db1d0 con 0x55d82ea11ee0 2024-05-03T18:34:47.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.621+0000 7f9a1e800700 1 --2- 172.21.15.33:0/2609901891 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55d82ea11ee0 0x55d82ea14320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:34:47.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.623+0000 7f9a03e00700 10 client.35792 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:34:47.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.623+0000 7f9a1e800700 1 --2- 172.21.15.33:0/2609901891 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55d82ea11ee0 0x55d82ea14320 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a23ae9c50aaa158b server_cookie=41ee45637c63437d in_seq=0 out_seq=0 2024-05-03T18:34:47.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.626+0000 7f9a03e00700 1 -- 172.21.15.33:0/2609901891 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d82e9db1d0 con 0x55d82ea11ee0 2024-05-03T18:34:47.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.626+0000 7f9a03e00700 10 client.35792 handle_command_reply: tid=0 2024-05-03T18:34:47.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.626+0000 7f9a2f4c8b80 1 client.35792 shutdown 2024-05-03T18:34:47.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.626+0000 7f9a2f4c8b80 1 -- 172.21.15.33:0/2609901891 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d82ea1c130 msgr2=0x55d82ea1fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:34:47.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.626+0000 7f9a2f4c8b80 1 --2- 172.21.15.33:0/2609901891 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d82ea1c130 0x55d82ea1fbb0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f9a08005950 tx=0x7f9a0800f8d0 comp rx=0 tx=0).stop 2024-05-03T18:34:47.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.627+0000 7f9a2f4c8b80 1 -- 172.21.15.33:0/2609901891 shutdown_connections 2024-05-03T18:34:47.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.627+0000 7f9a2f4c8b80 1 --2- 172.21.15.33:0/2609901891 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55d82ea11ee0 0x55d82ea14320 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:47.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.627+0000 7f9a2f4c8b80 1 --2- 172.21.15.33:0/2609901891 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d82ea1ca50 0x55d82ea200f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:47.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.627+0000 7f9a2f4c8b80 1 --2- 172.21.15.33:0/2609901891 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d82ea16de0 0x55d82ea22810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:47.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.627+0000 7f9a2f4c8b80 1 --2- 172.21.15.33:0/2609901891 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d82ea1c130 0x55d82ea1fbb0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:47.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.627+0000 7f9a2f4c8b80 1 -- 172.21.15.33:0/2609901891 >> 172.21.15.33:0/2609901891 conn(0x55d82e93cb60 msgr2=0x55d82ea1b540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:34:47.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.627+0000 7f9a03e00700 0 client.35792 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:34:47.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.628+0000 7f9a2f4c8b80 1 -- 172.21.15.33:0/2609901891 shutdown_connections 2024-05-03T18:34:47.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.628+0000 7f9a2f4c8b80 1 -- 172.21.15.33:0/2609901891 wait complete. 2024-05-03T18:34:47.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.628+0000 7f9a2f4c8b80 20 client.35792 trim_cache size 0 max 16384 2024-05-03T18:34:47.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.660+0000 7f9a2f4c8b80 1 Processor -- start 2024-05-03T18:34:47.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.660+0000 7f9a2f4c8b80 1 -- start start 2024-05-03T18:34:47.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.660+0000 7f9a2f4c8b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d82ea16de0 0x55d82ea1eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:34:47.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.660+0000 7f9a2f4c8b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d82ea1c130 0x55d82ea07ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:34:47.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.661+0000 7f9a2f4c8b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d82ea1ca50 0x55d82ea1d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:34:47.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.661+0000 7f9a2f4c8b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f99fc00ced0 con 0x55d82ea1ca50 2024-05-03T18:34:47.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.661+0000 7f9a2f4c8b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f99fc00f730 con 0x55d82ea1c130 2024-05-03T18:34:47.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.661+0000 7f9a2f4c8b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f99fc00d520 con 0x55d82ea16de0 2024-05-03T18:34:47.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.662+0000 7f9a17e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d82ea1c130 0x55d82ea07ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:34:47.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.662+0000 7f9a17e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d82ea1c130 0x55d82ea07ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:50394/0 (socket says 172.21.15.33:50394) 2024-05-03T18:34:47.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.662+0000 7f9a17e00700 1 -- 172.21.15.33:0/3726465769 learned_addr learned my addr 172.21.15.33:0/3726465769 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:34:47.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.662+0000 7f9a1e800700 1 --2- 172.21.15.33:0/3726465769 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d82ea16de0 0x55d82ea1eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:34:47.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.662+0000 7f9a1e800700 1 --2- 172.21.15.33:0/3726465769 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d82ea1ca50 0x55d82ea1d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:34:47.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.663+0000 7f9a17e00700 1 -- 172.21.15.33:0/3726465769 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d82ea16de0 msgr2=0x55d82ea1eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:34:47.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.663+0000 7f9a17e00700 1 --2- 172.21.15.33:0/3726465769 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d82ea16de0 0x55d82ea1eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:47.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.663+0000 7f9a17e00700 1 -- 172.21.15.33:0/3726465769 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d82ea1ca50 msgr2=0x55d82ea1d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:34:47.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.663+0000 7f9a17e00700 1 --2- 172.21.15.33:0/3726465769 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d82ea1ca50 0x55d82ea1d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:47.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.663+0000 7f9a17e00700 1 -- 172.21.15.33:0/3726465769 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d82ea149a0 con 0x55d82ea1c130 2024-05-03T18:34:47.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.664+0000 7f9a17e00700 1 --2- 172.21.15.33:0/3726465769 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d82ea1c130 0x55d82ea07ee0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9a08009c40 tx=0x7f9a0800b770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc39cccb1c8475a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:34:47.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.664+0000 7f9a0ea00700 1 -- 172.21.15.33:0/3726465769 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a08038030 con 0x55d82ea1c130 2024-05-03T18:34:47.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.664+0000 7f9a0ea00700 1 -- 172.21.15.33:0/3726465769 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9a08027070 con 0x55d82ea1c130 2024-05-03T18:34:47.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.664+0000 7f9a0ea00700 1 -- 172.21.15.33:0/3726465769 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a080226d0 con 0x55d82ea1c130 2024-05-03T18:34:47.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.665+0000 7f9a2f4c8b80 1 -- 172.21.15.33:0/3726465769 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d82ea1c130 msgr2=0x55d82ea07ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:34:47.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.665+0000 7f9a2f4c8b80 1 --2- 172.21.15.33:0/3726465769 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d82ea1c130 0x55d82ea07ee0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9a08009c40 tx=0x7f9a0800b770 comp rx=0 tx=0).stop 2024-05-03T18:34:47.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.666+0000 7f9a2f4c8b80 1 -- 172.21.15.33:0/3726465769 shutdown_connections 2024-05-03T18:34:47.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.666+0000 7f9a2f4c8b80 1 --2- 172.21.15.33:0/3726465769 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d82ea1ca50 0x55d82ea1d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:47.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.666+0000 7f9a2f4c8b80 1 --2- 172.21.15.33:0/3726465769 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d82ea16de0 0x55d82ea1eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:47.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.666+0000 7f9a2f4c8b80 1 --2- 172.21.15.33:0/3726465769 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d82ea1c130 0x55d82ea07ee0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:47.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.666+0000 7f9a2f4c8b80 1 -- 172.21.15.33:0/3726465769 >> 172.21.15.33:0/3726465769 conn(0x55d82e93cb60 msgr2=0x55d82ea1ce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:34:47.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.666+0000 7f9a2f4c8b80 1 -- 172.21.15.33:0/3726465769 shutdown_connections 2024-05-03T18:34:47.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.666+0000 7f9a2f4c8b80 1 -- 172.21.15.33:0/3726465769 wait complete. 2024-05-03T18:34:47.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.667+0000 7f9a2f4c8b80 1 -- start start 2024-05-03T18:34:47.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.668+0000 7f9a2f4c8b80 1 Processor -- start 2024-05-03T18:34:47.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.668+0000 7f9a2f4c8b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d82ea16de0 0x55d82ea421b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:34:47.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.669+0000 7f9a2f4c8b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d82ea1c130 0x55d82ea426f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:34:47.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.669+0000 7f9a2f4c8b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d82ea1ca50 0x55d82ea42c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:34:47.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.669+0000 7f9a2f4c8b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f99fc00dbf0 con 0x55d82ea1ca50 2024-05-03T18:34:47.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.669+0000 7f9a2f4c8b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f99fc00f060 con 0x55d82ea1c130 2024-05-03T18:34:47.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.669+0000 7f9a2f4c8b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f99fc00e2c0 con 0x55d82ea16de0 2024-05-03T18:34:47.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.669+0000 7f9a2f4c8b80 10 client.0 _ll_register_callbacks cb 0x55d82e907c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:34:47.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.670+0000 7f9a1e800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d82ea16de0 0x55d82ea421b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:34:47.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.670+0000 7f9a17e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d82ea1c130 0x55d82ea426f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:34:47.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.670+0000 7f9a17e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d82ea1c130 0x55d82ea426f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:50410/0 (socket says 172.21.15.33:50410) 2024-05-03T18:34:47.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.670+0000 7f9a17e00700 1 -- 172.21.15.33:0/24880691 learned_addr learned my addr 172.21.15.33:0/24880691 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:34:47.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.670+0000 7f9a17e00700 1 -- 172.21.15.33:0/24880691 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d82ea16de0 msgr2=0x55d82ea421b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:34:47.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.671+0000 7f9a17e00700 1 --2- 172.21.15.33:0/24880691 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d82ea16de0 0x55d82ea421b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:47.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.671+0000 7f9a17e00700 1 -- 172.21.15.33:0/24880691 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d82ea1ca50 msgr2=0x55d82ea42c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:34:47.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.671+0000 7f9a17e00700 1 --2- 172.21.15.33:0/24880691 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d82ea1ca50 0x55d82ea42c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:47.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.671+0000 7f9a17e00700 1 -- 172.21.15.33:0/24880691 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d82eaa25e0 con 0x55d82ea1c130 2024-05-03T18:34:47.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.672+0000 7f9a17e00700 1 --2- 172.21.15.33:0/24880691 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d82ea1c130 0x55d82ea426f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9a0800ba60 tx=0x7f9a08005bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff28b70d1774aa45 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:34:47.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.672+0000 7f9a03e00700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-03T18:34:47.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.674+0000 7f9a03e00700 1 -- 172.21.15.33:0/24880691 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a08041060 con 0x55d82ea1c130 2024-05-03T18:34:47.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.675+0000 7f9a03e00700 1 -- 172.21.15.33:0/24880691 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9518+0+0 (secure 0 0 0) 0x7f9a080054d0 con 0x55d82ea1c130 2024-05-03T18:34:47.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.675+0000 7f9a2f4c8b80 1 -- 172.21.15.33:0/24880691 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a08027070 con 0x55d82ea1c130 2024-05-03T18:34:47.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.675+0000 7f9a03e00700 1 -- 172.21.15.33:0/24880691 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f9a080056d0 con 0x55d82ea1c130 2024-05-03T18:34:47.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.676+0000 7f9a2f4c8b80 10 client.35798 fetch_fsmap learned FSMap version 21 2024-05-03T18:34:47.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.676+0000 7f9a2f4c8b80 1 -- 172.21.15.33:0/24880691 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55d82eaa3840 con 0x55d82ea1c130 2024-05-03T18:34:47.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.677+0000 7f9a03e00700 1 -- 172.21.15.33:0/24880691 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9a08027070 con 0x55d82ea1c130 2024-05-03T18:34:47.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.678+0000 7f9a03e00700 1 -- 172.21.15.33:0/24880691 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f9a080039c0 con 0x55d82ea1c130 2024-05-03T18:34:47.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.678+0000 7f9a2f4c8b80 10 client.35798 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:34:47.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.678+0000 7f9a2f4c8b80 10 client.35798 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:34:47.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.678+0000 7f9a2f4c8b80 20 client.35798 populate_metadata read hostname 'smithi033' 2024-05-03T18:34:47.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.678+0000 7f9a2f4c8b80 1 --2- 172.21.15.33:0/24880691 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55d82eaa3bd0 0x55d82eaa5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:34:47.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.678+0000 7f9a2f4c8b80 4 client.35798 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-03T18:34:47.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.678+0000 7f9a2f4c8b80 1 -- 172.21.15.33:0/24880691 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d82e96e400 con 0x55d82eaa3bd0 2024-05-03T18:34:47.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.679+0000 7f9a1e800700 1 --2- 172.21.15.33:0/24880691 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55d82eaa3bd0 0x55d82eaa5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:34:47.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.681+0000 7f9a1e800700 1 --2- 172.21.15.33:0/24880691 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55d82eaa3bd0 0x55d82eaa5ff0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c47585d2ed7c230 server_cookie=d6f8d4be78ce6790 in_seq=0 out_seq=0 2024-05-03T18:34:47.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.681+0000 7f9a03e00700 10 client.35798 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:34:47.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.681+0000 7f9a03e00700 1 -- 172.21.15.33:0/24880691 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d82e96e400 con 0x55d82eaa3bd0 2024-05-03T18:34:47.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.681+0000 7f9a03e00700 10 client.35798 handle_command_reply: tid=0 2024-05-03T18:34:47.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.681+0000 7f9a2f4c8b80 1 client.35798 shutdown 2024-05-03T18:34:47.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.683+0000 7f9a2f4c8b80 1 -- 172.21.15.33:0/24880691 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d82ea1c130 msgr2=0x55d82ea426f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:34:47.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.683+0000 7f9a2f4c8b80 1 --2- 172.21.15.33:0/24880691 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d82ea1c130 0x55d82ea426f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9a0800ba60 tx=0x7f9a08005bf0 comp rx=0 tx=0).stop 2024-05-03T18:34:47.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.684+0000 7f9a2f4c8b80 1 -- 172.21.15.33:0/24880691 shutdown_connections 2024-05-03T18:34:47.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.684+0000 7f9a2f4c8b80 1 --2- 172.21.15.33:0/24880691 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55d82eaa3bd0 0x55d82eaa5ff0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:47.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.684+0000 7f9a2f4c8b80 1 --2- 172.21.15.33:0/24880691 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d82ea1ca50 0x55d82ea42c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:47.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.684+0000 7f9a2f4c8b80 1 --2- 172.21.15.33:0/24880691 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d82ea16de0 0x55d82ea421b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:47.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.684+0000 7f9a2f4c8b80 1 --2- 172.21.15.33:0/24880691 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d82ea1c130 0x55d82ea426f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:47.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.684+0000 7f9a2f4c8b80 1 -- 172.21.15.33:0/24880691 >> 172.21.15.33:0/24880691 conn(0x55d82e93cb60 msgr2=0x55d82ea1d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:34:47.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.684+0000 7f9a03e00700 0 client.35798 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:34:47.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.684+0000 7f9a2f4c8b80 1 -- 172.21.15.33:0/24880691 shutdown_connections 2024-05-03T18:34:47.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.684+0000 7f9a2f4c8b80 1 -- 172.21.15.33:0/24880691 wait complete. 2024-05-03T18:34:47.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.685+0000 7f9a2f4c8b80 20 client.35798 trim_cache size 0 max 16384 2024-05-03T18:34:47.686 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-03T18:34:47.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.686+0000 7f9a24c00700 1 -- 172.21.15.33:0/2563879249 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f9a040b2a70 msgr2=0x7f9a040b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:34:47.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.686+0000 7f9a24c00700 1 --2- 172.21.15.33:0/2563879249 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f9a040b2a70 0x7f9a040b4f30 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9a100dfdc0 tx=0x7f9a100e2ca0 comp rx=0 tx=0).stop 2024-05-03T18:34:47.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.687+0000 7f9a24c00700 1 -- 172.21.15.33:0/2563879249 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a2006b1b0 msgr2=0x7f9a2019bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:34:47.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.687+0000 7f9a24c00700 1 --2- 172.21.15.33:0/2563879249 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a2006b1b0 0x7f9a2019bd90 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9a1000ef10 tx=0x7f9a1000c3a0 comp rx=0 tx=0).stop 2024-05-03T18:34:47.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.687+0000 7f9a24c00700 1 -- 172.21.15.33:0/2563879249 shutdown_connections 2024-05-03T18:34:47.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.687+0000 7f9a24c00700 1 --2- 172.21.15.33:0/2563879249 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f9a040b2a70 0x7f9a040b4f30 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:47.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.687+0000 7f9a24c00700 1 --2- 172.21.15.33:0/2563879249 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a2010ff50 0x7f9a201a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:47.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.687+0000 7f9a24c00700 1 --2- 172.21.15.33:0/2563879249 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a2006b1b0 0x7f9a2019bd90 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:47.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.687+0000 7f9a24c00700 1 --2- 172.21.15.33:0/2563879249 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a2006bad0 0x7f9a2019c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:47.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.687+0000 7f9a24c00700 1 -- 172.21.15.33:0/2563879249 >> 172.21.15.33:0/2563879249 conn(0x7f9a20079350 msgr2=0x7f9a20111790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:34:47.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.687+0000 7f9a24c00700 1 -- 172.21.15.33:0/2563879249 shutdown_connections 2024-05-03T18:34:47.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:47.687+0000 7f9a24c00700 1 -- 172.21.15.33:0/2563879249 wait complete. 2024-05-03T18:34:48.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:34:48 smithi150 ceph-mon[32492]: pgmap v233: 129 pgs: 129 active+clean; 2.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 12 MiB/s wr, 733 op/s 2024-05-03T18:34:48.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:34:48 smithi033 ceph-mon[28330]: pgmap v233: 129 pgs: 129 active+clean; 2.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 12 MiB/s wr, 733 op/s 2024-05-03T18:34:48.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:34:48 smithi099 ceph-mon[31658]: pgmap v233: 129 pgs: 129 active+clean; 2.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 12 MiB/s wr, 733 op/s 2024-05-03T18:34:50.111 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 215.0 s, 656.3 tps, lat 187.231 ms stddev 253.915 2024-05-03T18:34:50.199 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-03T18:34:50.200 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-03T18:34:50.348 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:34:50 smithi099 ceph-mon[31658]: pgmap v234: 129 pgs: 129 active+clean; 2.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 9.5 MiB/s wr, 580 op/s 2024-05-03T18:34:50.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:34:50 smithi150 ceph-mon[32492]: pgmap v234: 129 pgs: 129 active+clean; 2.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 9.5 MiB/s wr, 580 op/s 2024-05-03T18:34:50.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:34:50 smithi033 ceph-mon[28330]: pgmap v234: 129 pgs: 129 active+clean; 2.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 9.5 MiB/s wr, 580 op/s 2024-05-03T18:34:50.906 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:34:52.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:34:52 smithi150 ceph-mon[32492]: pgmap v235: 129 pgs: 129 active+clean; 2.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 9.0 MiB/s wr, 537 op/s 2024-05-03T18:34:52.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:34:52 smithi033 ceph-mon[28330]: pgmap v235: 129 pgs: 129 active+clean; 2.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 9.0 MiB/s wr, 537 op/s 2024-05-03T18:34:52.548 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:34:52 smithi099 ceph-mon[31658]: pgmap v235: 129 pgs: 129 active+clean; 2.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 9.0 MiB/s wr, 537 op/s 2024-05-03T18:34:53.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.640+0000 7f878ee00700 1 -- 172.21.15.33:0/433868182 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8788106050 msgr2=0x7f8788112560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:34:53.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.640+0000 7f878ee00700 1 --2- 172.21.15.33:0/433868182 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8788106050 0x7f8788112560 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f878000b0a0 tx=0x7f878000b3b0 comp rx=0 tx=0).stop 2024-05-03T18:34:53.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.642+0000 7f878ee00700 1 -- 172.21.15.33:0/433868182 shutdown_connections 2024-05-03T18:34:53.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.642+0000 7f878ee00700 1 --2- 172.21.15.33:0/433868182 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8788106050 0x7f8788112560 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:53.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.642+0000 7f878ee00700 1 --2- 172.21.15.33:0/433868182 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f878810b690 0x7f878810ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:53.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.642+0000 7f878ee00700 1 --2- 172.21.15.33:0/433868182 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8788105690 0x7f8788105b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:53.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.642+0000 7f878ee00700 1 -- 172.21.15.33:0/433868182 >> 172.21.15.33:0/433868182 conn(0x7f8788101180 msgr2=0x7f87881035a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:34:53.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.642+0000 7f878ee00700 1 -- 172.21.15.33:0/433868182 shutdown_connections 2024-05-03T18:34:53.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.643+0000 7f878ee00700 1 -- 172.21.15.33:0/433868182 wait complete. 2024-05-03T18:34:53.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.644+0000 7f878ee00700 1 Processor -- start 2024-05-03T18:34:53.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.644+0000 7f878ee00700 1 -- start start 2024-05-03T18:34:53.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.645+0000 7f878ee00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8788105690 0x7f87881a18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:34:53.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.645+0000 7f878ee00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8788106050 0x7f87881a1e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:34:53.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.645+0000 7f878ee00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f878810b690 0x7f878819baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:34:53.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.645+0000 7f878ee00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8788115790 con 0x7f8788105690 2024-05-03T18:34:53.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.645+0000 7f878ee00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8788115610 con 0x7f878810b690 2024-05-03T18:34:53.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.645+0000 7f878ee00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8788115910 con 0x7f8788106050 2024-05-03T18:34:53.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.646+0000 7f8787400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8788106050 0x7f87881a1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:34:53.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.646+0000 7f878ce00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f878810b690 0x7f878819baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:34:53.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.646+0000 7f878ce00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f878810b690 0x7f878819baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:50416/0 (socket says 172.21.15.33:50416) 2024-05-03T18:34:53.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.646+0000 7f878ce00700 1 -- 172.21.15.33:0/1373145410 learned_addr learned my addr 172.21.15.33:0/1373145410 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:34:53.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.646+0000 7f8787400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8788106050 0x7f87881a1e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:40878/0 (socket says 172.21.15.33:40878) 2024-05-03T18:34:53.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.646+0000 7f8787e00700 1 --2- 172.21.15.33:0/1373145410 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8788105690 0x7f87881a18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:34:53.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.647+0000 7f878ce00700 1 -- 172.21.15.33:0/1373145410 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8788106050 msgr2=0x7f87881a1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:34:53.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.647+0000 7f878ce00700 1 --2- 172.21.15.33:0/1373145410 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8788106050 0x7f87881a1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:53.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.647+0000 7f878ce00700 1 -- 172.21.15.33:0/1373145410 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8788105690 msgr2=0x7f87881a18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:34:53.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.647+0000 7f878ce00700 1 --2- 172.21.15.33:0/1373145410 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8788105690 0x7f87881a18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:53.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.647+0000 7f878ce00700 1 -- 172.21.15.33:0/1373145410 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f878819c250 con 0x7f878810b690 2024-05-03T18:34:53.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.648+0000 7f8787e00700 1 --2- 172.21.15.33:0/1373145410 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8788105690 0x7f87881a18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:34:53.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.649+0000 7f878ce00700 1 --2- 172.21.15.33:0/1373145410 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f878810b690 0x7f878819baf0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f878000b070 tx=0x7f8780004d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:34:53.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.650+0000 7f8784c00700 1 -- 172.21.15.33:0/1373145410 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8780021070 con 0x7f878810b690 2024-05-03T18:34:53.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.650+0000 7f878ee00700 1 -- 172.21.15.33:0/1373145410 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f878819c4e0 con 0x7f878810b690 2024-05-03T18:34:53.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.650+0000 7f878ee00700 1 -- 172.21.15.33:0/1373145410 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87881a87b0 con 0x7f878810b690 2024-05-03T18:34:53.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.651+0000 7f8784c00700 1 -- 172.21.15.33:0/1373145410 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8780007b80 con 0x7f878810b690 2024-05-03T18:34:53.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.651+0000 7f8784c00700 1 -- 172.21.15.33:0/1373145410 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8780004280 con 0x7f878810b690 2024-05-03T18:34:53.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.658+0000 7f87997ffb80 1 Processor -- start 2024-05-03T18:34:53.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.658+0000 7f8784c00700 1 -- 172.21.15.33:0/1373145410 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f8780003a00 con 0x7f878810b690 2024-05-03T18:34:53.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.659+0000 7f87997ffb80 1 -- start start 2024-05-03T18:34:53.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.660+0000 7f87997ffb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bd266569b0 0x55bd26656d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:34:53.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.660+0000 7f87997ffb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd266572d0 0x55bd26650fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:34:53.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.660+0000 7f87997ffb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd266515d0 0x55bd26651a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:34:53.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.660+0000 7f87997ffb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55bd265a3aa0 con 0x55bd266572d0 2024-05-03T18:34:53.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.660+0000 7f87997ffb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55bd265417e0 con 0x55bd266569b0 2024-05-03T18:34:53.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.660+0000 7f87997ffb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55bd2659fa60 con 0x55bd266515d0 2024-05-03T18:34:53.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.661+0000 7f8784c00700 1 --2- 172.21.15.33:0/1373145410 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f876c0b2af0 0x7f876c0b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:34:53.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.661+0000 7f8787400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bd266569b0 0x55bd26656d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:34:53.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.661+0000 7f8787400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bd266569b0 0x55bd26656d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:50430/0 (socket says 172.21.15.33:50430) 2024-05-03T18:34:53.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.661+0000 7f8787400700 1 -- 172.21.15.33:0/2799620220 learned_addr learned my addr 172.21.15.33:0/2799620220 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:34:53.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.661+0000 7f8787400700 1 --2- 172.21.15.33:0/2799620220 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd266515d0 0x55bd26651a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:34:53.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.661+0000 7f8787e00700 1 --2- 172.21.15.33:0/2799620220 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd266572d0 0x55bd26650fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:34:53.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.661+0000 7f8784c00700 1 -- 172.21.15.33:0/1373145410 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9518+0+0 (secure 0 0 0) 0x7f878000f020 con 0x7f878810b690 2024-05-03T18:34:53.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.661+0000 7f878ce00700 1 --2- 172.21.15.33:0/1373145410 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f876c0b2af0 0x7f876c0b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:34:53.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.662+0000 7f8787e00700 1 -- 172.21.15.33:0/2799620220 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd266515d0 msgr2=0x55bd26651a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:34:53.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.662+0000 7f8787e00700 1 --2- 172.21.15.33:0/2799620220 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd266515d0 0x55bd26651a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:53.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.662+0000 7f8787e00700 1 -- 172.21.15.33:0/2799620220 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bd266569b0 msgr2=0x55bd26656d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:34:53.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.662+0000 7f8787e00700 1 --2- 172.21.15.33:0/2799620220 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bd266569b0 0x55bd26656d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:53.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.662+0000 7f8787e00700 1 -- 172.21.15.33:0/2799620220 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd264f0950 con 0x55bd266572d0 2024-05-03T18:34:53.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.662+0000 7f8787e00700 1 --2- 172.21.15.33:0/2799620220 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd266572d0 0x55bd26650fb0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f877800bce0 tx=0x7f877800f360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e091dedd8f4d7da server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:34:53.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.662+0000 7f878ce00700 1 --2- 172.21.15.33:0/1373145410 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f876c0b2af0 0x7f876c0b4fb0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f8780009fd0 tx=0x7f878000b3b0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:34:53.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.663+0000 7f877d600700 1 -- 172.21.15.33:0/2799620220 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f877800fbe0 con 0x55bd266572d0 2024-05-03T18:34:53.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.663+0000 7f877d600700 1 -- 172.21.15.33:0/2799620220 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f877800fd80 con 0x55bd266572d0 2024-05-03T18:34:53.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.663+0000 7f877d600700 1 -- 172.21.15.33:0/2799620220 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f877801b670 con 0x55bd266572d0 2024-05-03T18:34:53.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.665+0000 7f87997ffb80 1 -- 172.21.15.33:0/2799620220 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd266572d0 msgr2=0x55bd26650fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:34:53.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.665+0000 7f87997ffb80 1 --2- 172.21.15.33:0/2799620220 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd266572d0 0x55bd26650fb0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f877800bce0 tx=0x7f877800f360 comp rx=0 tx=0).stop 2024-05-03T18:34:53.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.665+0000 7f87997ffb80 1 -- 172.21.15.33:0/2799620220 shutdown_connections 2024-05-03T18:34:53.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.665+0000 7f87997ffb80 1 --2- 172.21.15.33:0/2799620220 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd266515d0 0x55bd26651a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:53.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.665+0000 7f87997ffb80 1 --2- 172.21.15.33:0/2799620220 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bd266569b0 0x55bd26656d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:53.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.665+0000 7f87997ffb80 1 --2- 172.21.15.33:0/2799620220 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd266572d0 0x55bd26650fb0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:53.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.665+0000 7f87997ffb80 1 -- 172.21.15.33:0/2799620220 >> 172.21.15.33:0/2799620220 conn(0x55bd265c9750 msgr2=0x55bd2664b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:34:53.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.666+0000 7f87997ffb80 1 -- 172.21.15.33:0/2799620220 shutdown_connections 2024-05-03T18:34:53.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.666+0000 7f87997ffb80 1 -- 172.21.15.33:0/2799620220 wait complete. 2024-05-03T18:34:53.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.667+0000 7f87997ffb80 1 -- start start 2024-05-03T18:34:53.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.667+0000 7f87997ffb80 1 Processor -- start 2024-05-03T18:34:53.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.668+0000 7f87997ffb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd266515d0 0x55bd2665ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:34:53.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.668+0000 7f87997ffb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd266569b0 0x55bd2665d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:34:53.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.668+0000 7f87997ffb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bd266572d0 0x55bd2665a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:34:53.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.668+0000 7f87997ffb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55bd2665dd80 con 0x55bd266515d0 2024-05-03T18:34:53.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.668+0000 7f87997ffb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55bd26509ba0 con 0x55bd266572d0 2024-05-03T18:34:53.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.668+0000 7f87997ffb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55bd2665df00 con 0x55bd266569b0 2024-05-03T18:34:53.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.668+0000 7f87997ffb80 10 client.0 _ll_register_callbacks cb 0x55bd2654acf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:34:53.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.668+0000 7f8787400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd266515d0 0x55bd2665ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:34:53.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.669+0000 7f8787400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd266515d0 0x55bd2665ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49786/0 (socket says 172.21.15.33:49786) 2024-05-03T18:34:53.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.669+0000 7f8787400700 1 -- 172.21.15.33:0/1467329970 learned_addr learned my addr 172.21.15.33:0/1467329970 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:34:53.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.669+0000 7f8787e00700 1 --2- 172.21.15.33:0/1467329970 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd266569b0 0x55bd2665d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:34:53.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.669+0000 7f8787400700 1 --2- 172.21.15.33:0/1467329970 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bd266572d0 0x55bd2665a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:34:53.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.669+0000 7f8787400700 1 -- 172.21.15.33:0/1467329970 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd266569b0 msgr2=0x55bd2665d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:34:53.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.669+0000 7f8787400700 1 --2- 172.21.15.33:0/1467329970 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd266569b0 0x55bd2665d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:53.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.669+0000 7f8787400700 1 -- 172.21.15.33:0/1467329970 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bd266572d0 msgr2=0x55bd2665a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:34:53.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.669+0000 7f8787400700 1 --2- 172.21.15.33:0/1467329970 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bd266572d0 0x55bd2665a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:53.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.669+0000 7f8787400700 1 -- 172.21.15.33:0/1467329970 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd265694a0 con 0x55bd266515d0 2024-05-03T18:34:53.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.670+0000 7f8787400700 1 --2- 172.21.15.33:0/1467329970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd266515d0 0x55bd2665ce40 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f8774012ce0 tx=0x7f8774014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34cac677062cfcd9 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:34:53.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.670+0000 7f8772000700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:34:53.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.670+0000 7f8772000700 1 -- 172.21.15.33:0/1467329970 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8774010b60 con 0x55bd266515d0 2024-05-03T18:34:53.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.670+0000 7f8772000700 1 -- 172.21.15.33:0/1467329970 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9518+0+0 (secure 0 0 0) 0x7f8774010d00 con 0x55bd266515d0 2024-05-03T18:34:53.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.671+0000 7f87997ffb80 1 -- 172.21.15.33:0/1467329970 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f877800fd80 con 0x55bd266515d0 2024-05-03T18:34:53.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.672+0000 7f8772000700 1 -- 172.21.15.33:0/1467329970 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f877402e050 con 0x55bd266515d0 2024-05-03T18:34:53.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.672+0000 7f87997ffb80 10 client.16161 fetch_fsmap learned FSMap version 21 2024-05-03T18:34:53.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.672+0000 7f87997ffb80 1 -- 172.21.15.33:0/1467329970 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55bd266684e0 con 0x55bd266515d0 2024-05-03T18:34:53.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.672+0000 7f8772000700 1 -- 172.21.15.33:0/1467329970 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f877800fd80 con 0x55bd266515d0 2024-05-03T18:34:53.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.673+0000 7f8772000700 1 -- 172.21.15.33:0/1467329970 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f87740293b0 con 0x55bd266515d0 2024-05-03T18:34:53.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.673+0000 7f87997ffb80 10 client.16161 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:34:53.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.673+0000 7f87997ffb80 10 client.16161 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:34:53.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.673+0000 7f87997ffb80 20 client.16161 populate_metadata read hostname 'smithi033' 2024-05-03T18:34:53.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.673+0000 7f87997ffb80 1 --2- 172.21.15.33:0/1467329970 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55bd2666ab20 0x55bd2666cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:34:53.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.673+0000 7f87997ffb80 4 client.16161 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:34:53.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.673+0000 7f87997ffb80 1 -- 172.21.15.33:0/1467329970 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bd2661e070 con 0x55bd2666ab20 2024-05-03T18:34:53.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.674+0000 7f8787e00700 1 --2- 172.21.15.33:0/1467329970 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55bd2666ab20 0x55bd2666cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:34:53.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.676+0000 7f8787e00700 1 --2- 172.21.15.33:0/1467329970 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55bd2666ab20 0x55bd2666cf50 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2d4cb2e357e9da2 server_cookie=f20f4d4eccff9044 in_seq=0 out_seq=0 2024-05-03T18:34:53.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.676+0000 7f8772000700 10 client.16161 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:34:53.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.678+0000 7f8772000700 1 -- 172.21.15.33:0/1467329970 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55bd2661e070 con 0x55bd2666ab20 2024-05-03T18:34:53.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.678+0000 7f8772000700 10 client.16161 handle_command_reply: tid=0 2024-05-03T18:34:53.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.678+0000 7f87997ffb80 1 client.16161 shutdown 2024-05-03T18:34:53.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.679+0000 7f87997ffb80 1 -- 172.21.15.33:0/1467329970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd266515d0 msgr2=0x55bd2665ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:34:53.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.679+0000 7f87997ffb80 1 --2- 172.21.15.33:0/1467329970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd266515d0 0x55bd2665ce40 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f8774012ce0 tx=0x7f8774014040 comp rx=0 tx=0).stop 2024-05-03T18:34:53.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.679+0000 7f87997ffb80 1 -- 172.21.15.33:0/1467329970 shutdown_connections 2024-05-03T18:34:53.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.679+0000 7f87997ffb80 1 --2- 172.21.15.33:0/1467329970 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55bd2666ab20 0x55bd2666cf50 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:53.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.679+0000 7f87997ffb80 1 --2- 172.21.15.33:0/1467329970 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bd266572d0 0x55bd2665a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:53.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.679+0000 7f87997ffb80 1 --2- 172.21.15.33:0/1467329970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd266515d0 0x55bd2665ce40 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:53.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.679+0000 7f87997ffb80 1 --2- 172.21.15.33:0/1467329970 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd266569b0 0x55bd2665d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:53.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.679+0000 7f87997ffb80 1 -- 172.21.15.33:0/1467329970 >> 172.21.15.33:0/1467329970 conn(0x55bd265c9750 msgr2=0x55bd26646430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:34:53.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.680+0000 7f8772000700 0 client.16161 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:34:53.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.680+0000 7f87997ffb80 1 -- 172.21.15.33:0/1467329970 shutdown_connections 2024-05-03T18:34:53.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.680+0000 7f87997ffb80 1 -- 172.21.15.33:0/1467329970 wait complete. 2024-05-03T18:34:53.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.680+0000 7f87997ffb80 20 client.16161 trim_cache size 0 max 16384 2024-05-03T18:34:53.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.718+0000 7f87997ffb80 1 Processor -- start 2024-05-03T18:34:53.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.718+0000 7f87997ffb80 1 -- start start 2024-05-03T18:34:53.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.719+0000 7f87997ffb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd266515d0 0x55bd26650fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:34:53.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.719+0000 7f87997ffb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd266569b0 0x55bd266728b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:34:53.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.719+0000 7f87997ffb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bd266572d0 0x55bd26672df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:34:53.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.719+0000 7f87997ffb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f876800cfa0 con 0x55bd266569b0 2024-05-03T18:34:53.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.719+0000 7f87997ffb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f876800f800 con 0x55bd266572d0 2024-05-03T18:34:53.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.719+0000 7f87997ffb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f876800d5f0 con 0x55bd266515d0 2024-05-03T18:34:53.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.720+0000 7f8787400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd266515d0 0x55bd26650fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:34:53.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.720+0000 7f8787e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd266569b0 0x55bd266728b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:34:53.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.720+0000 7f8787400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd266515d0 0x55bd26650fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:40894/0 (socket says 172.21.15.33:40894) 2024-05-03T18:34:53.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.720+0000 7f8787400700 1 -- 172.21.15.33:0/439319780 learned_addr learned my addr 172.21.15.33:0/439319780 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:34:53.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.720+0000 7f8787400700 1 --2- 172.21.15.33:0/439319780 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bd266572d0 0x55bd26672df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:34:53.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.721+0000 7f8787e00700 1 -- 172.21.15.33:0/439319780 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd266515d0 msgr2=0x55bd26650fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:34:53.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.721+0000 7f8787e00700 1 --2- 172.21.15.33:0/439319780 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd266515d0 0x55bd26650fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:53.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.721+0000 7f8787e00700 1 -- 172.21.15.33:0/439319780 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bd266572d0 msgr2=0x55bd26672df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:34:53.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.721+0000 7f8787e00700 1 --2- 172.21.15.33:0/439319780 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bd266572d0 0x55bd26672df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:53.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.721+0000 7f8787e00700 1 -- 172.21.15.33:0/439319780 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd2664ca70 con 0x55bd266569b0 2024-05-03T18:34:53.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.721+0000 7f8787400700 1 --2- 172.21.15.33:0/439319780 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bd266572d0 0x55bd26672df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:34:53.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.721+0000 7f8787e00700 1 --2- 172.21.15.33:0/439319780 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd266569b0 0x55bd266728b0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x55bd2665b7b0 tx=0x7f877800b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad2329428a49b382 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:34:53.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.722+0000 7f877cc00700 1 -- 172.21.15.33:0/439319780 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8778005020 con 0x55bd266569b0 2024-05-03T18:34:53.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.722+0000 7f877cc00700 1 -- 172.21.15.33:0/439319780 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55bd2661e070 con 0x55bd266569b0 2024-05-03T18:34:53.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.722+0000 7f877cc00700 1 -- 172.21.15.33:0/439319780 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8778004b80 con 0x55bd266569b0 2024-05-03T18:34:53.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.723+0000 7f87997ffb80 1 -- 172.21.15.33:0/439319780 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd266569b0 msgr2=0x55bd266728b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:34:53.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.723+0000 7f87997ffb80 1 --2- 172.21.15.33:0/439319780 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd266569b0 0x55bd266728b0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x55bd2665b7b0 tx=0x7f877800b410 comp rx=0 tx=0).stop 2024-05-03T18:34:53.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.724+0000 7f87997ffb80 1 -- 172.21.15.33:0/439319780 shutdown_connections 2024-05-03T18:34:53.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.724+0000 7f87997ffb80 1 --2- 172.21.15.33:0/439319780 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bd266572d0 0x55bd26672df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:53.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.724+0000 7f87997ffb80 1 --2- 172.21.15.33:0/439319780 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd266515d0 0x55bd26650fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:53.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.724+0000 7f87997ffb80 1 --2- 172.21.15.33:0/439319780 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd266569b0 0x55bd266728b0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:53.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.724+0000 7f87997ffb80 1 -- 172.21.15.33:0/439319780 >> 172.21.15.33:0/439319780 conn(0x55bd265c9750 msgr2=0x55bd26658ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:34:53.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.725+0000 7f87997ffb80 1 -- 172.21.15.33:0/439319780 shutdown_connections 2024-05-03T18:34:53.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.725+0000 7f87997ffb80 1 -- 172.21.15.33:0/439319780 wait complete. 2024-05-03T18:34:53.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.725+0000 7f87997ffb80 1 -- start start 2024-05-03T18:34:53.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.726+0000 7f87997ffb80 1 Processor -- start 2024-05-03T18:34:53.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.727+0000 7f87997ffb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd266515d0 0x55bd26663220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:34:53.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.727+0000 7f87997ffb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bd266569b0 0x55bd26663760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:34:53.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.727+0000 7f87997ffb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd266572d0 0x55bd266601f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:34:53.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.727+0000 7f87997ffb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f876800dcc0 con 0x55bd266515d0 2024-05-03T18:34:53.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.727+0000 7f87997ffb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f876800f130 con 0x55bd266569b0 2024-05-03T18:34:53.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.727+0000 7f87997ffb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f876800e390 con 0x55bd266572d0 2024-05-03T18:34:53.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.727+0000 7f87997ffb80 10 client.0 _ll_register_callbacks cb 0x55bd2654acf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:34:53.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.727+0000 7f8787400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd266515d0 0x55bd26663220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:34:53.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.727+0000 7f8787400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd266515d0 0x55bd26663220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49794/0 (socket says 172.21.15.33:49794) 2024-05-03T18:34:53.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.727+0000 7f8787400700 1 -- 172.21.15.33:0/194737967 learned_addr learned my addr 172.21.15.33:0/194737967 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:34:53.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.728+0000 7f8787400700 1 --2- 172.21.15.33:0/194737967 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd266572d0 0x55bd266601f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:34:53.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.728+0000 7f8787e00700 1 --2- 172.21.15.33:0/194737967 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bd266569b0 0x55bd26663760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:34:53.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.728+0000 7f8787400700 1 -- 172.21.15.33:0/194737967 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd266572d0 msgr2=0x55bd266601f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:34:53.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.728+0000 7f8787400700 1 --2- 172.21.15.33:0/194737967 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd266572d0 0x55bd266601f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:53.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.728+0000 7f8787400700 1 -- 172.21.15.33:0/194737967 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bd266569b0 msgr2=0x55bd26663760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:34:53.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.728+0000 7f8787400700 1 --2- 172.21.15.33:0/194737967 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bd266569b0 0x55bd26663760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:53.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.728+0000 7f8787400700 1 -- 172.21.15.33:0/194737967 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd26660a30 con 0x55bd266515d0 2024-05-03T18:34:53.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.729+0000 7f8787400700 1 --2- 172.21.15.33:0/194737967 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd266515d0 0x55bd26663220 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f877400b720 tx=0x7f8774014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa4fbc2b08592151 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:34:53.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.729+0000 7f8772000700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:34:53.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.729+0000 7f8772000700 1 -- 172.21.15.33:0/194737967 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8774039030 con 0x55bd266515d0 2024-05-03T18:34:53.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.729+0000 7f87997ffb80 1 -- 172.21.15.33:0/194737967 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55bd2661e070 con 0x55bd266515d0 2024-05-03T18:34:53.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.730+0000 7f8772000700 1 -- 172.21.15.33:0/194737967 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9518+0+0 (secure 0 0 0) 0x7f877401b030 con 0x55bd266515d0 2024-05-03T18:34:53.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.730+0000 7f8772000700 1 -- 172.21.15.33:0/194737967 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f8774000e60 con 0x55bd266515d0 2024-05-03T18:34:53.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.730+0000 7f87997ffb80 10 client.16167 fetch_fsmap learned FSMap version 21 2024-05-03T18:34:53.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.730+0000 7f87997ffb80 1 -- 172.21.15.33:0/194737967 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55bd266de860 con 0x55bd266515d0 2024-05-03T18:34:53.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.731+0000 7f8772000700 1 -- 172.21.15.33:0/194737967 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55bd2661e070 con 0x55bd266515d0 2024-05-03T18:34:53.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.731+0000 7f8772000700 1 -- 172.21.15.33:0/194737967 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f8774004830 con 0x55bd266515d0 2024-05-03T18:34:53.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.732+0000 7f87997ffb80 10 client.16167 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:34:53.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.732+0000 7f87997ffb80 10 client.16167 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:34:53.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.732+0000 7f87997ffb80 20 client.16167 populate_metadata read hostname 'smithi033' 2024-05-03T18:34:53.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.732+0000 7f87997ffb80 1 --2- 172.21.15.33:0/194737967 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55bd266deef0 0x55bd266e1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:34:53.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.732+0000 7f87997ffb80 4 client.16167 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-03T18:34:53.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.732+0000 7f87997ffb80 1 -- 172.21.15.33:0/194737967 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55bd265b1480 con 0x55bd266deef0 2024-05-03T18:34:53.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.733+0000 7f8787e00700 1 --2- 172.21.15.33:0/194737967 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55bd266deef0 0x55bd266e1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:34:53.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.734+0000 7f8787e00700 1 --2- 172.21.15.33:0/194737967 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55bd266deef0 0x55bd266e1310 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b18c8e863f959be server_cookie=a53ba55bc8de55a0 in_seq=0 out_seq=0 2024-05-03T18:34:53.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.734+0000 7f8772000700 10 client.16167 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:34:53.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.738+0000 7f8772000700 1 -- 172.21.15.33:0/194737967 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55bd265b1480 con 0x55bd266deef0 2024-05-03T18:34:53.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.738+0000 7f8772000700 10 client.16167 handle_command_reply: tid=0 2024-05-03T18:34:53.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.738+0000 7f87997ffb80 1 client.16167 shutdown 2024-05-03T18:34:53.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.739+0000 7f87997ffb80 1 -- 172.21.15.33:0/194737967 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd266515d0 msgr2=0x55bd26663220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:34:53.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.739+0000 7f87997ffb80 1 --2- 172.21.15.33:0/194737967 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd266515d0 0x55bd26663220 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f877400b720 tx=0x7f8774014040 comp rx=0 tx=0).stop 2024-05-03T18:34:53.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.739+0000 7f87997ffb80 1 -- 172.21.15.33:0/194737967 shutdown_connections 2024-05-03T18:34:53.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.739+0000 7f87997ffb80 1 --2- 172.21.15.33:0/194737967 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55bd266deef0 0x55bd266e1310 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:53.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.739+0000 7f8772000700 0 client.16167 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:34:53.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.739+0000 7f87997ffb80 1 --2- 172.21.15.33:0/194737967 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bd266572d0 0x55bd266601f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:53.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.739+0000 7f87997ffb80 1 --2- 172.21.15.33:0/194737967 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd266515d0 0x55bd26663220 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:53.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.739+0000 7f87997ffb80 1 --2- 172.21.15.33:0/194737967 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bd266569b0 0x55bd26663760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:53.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.739+0000 7f87997ffb80 1 -- 172.21.15.33:0/194737967 >> 172.21.15.33:0/194737967 conn(0x55bd265c9750 msgr2=0x55bd26658930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:34:53.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.739+0000 7f87997ffb80 1 -- 172.21.15.33:0/194737967 shutdown_connections 2024-05-03T18:34:53.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.739+0000 7f87997ffb80 1 -- 172.21.15.33:0/194737967 wait complete. 2024-05-03T18:34:53.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.739+0000 7f87997ffb80 20 client.16167 trim_cache size 0 max 16384 2024-05-03T18:34:53.742 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-03T18:34:53.742 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-05-03T18:34:53.742 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "7d480fc7-aa6e-43ee-8111-4ee8be142722", 2024-05-03T18:34:53.742 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-05-03T18:34:53.742 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-03T18:34:53.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.741+0000 7f878ee00700 1 -- 172.21.15.33:0/1373145410 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f876c0b2af0 msgr2=0x7f876c0b4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:34:53.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.741+0000 7f878ee00700 1 --2- 172.21.15.33:0/1373145410 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f876c0b2af0 0x7f876c0b4fb0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f8780009fd0 tx=0x7f878000b3b0 comp rx=0 tx=0).stop 2024-05-03T18:34:53.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.741+0000 7f878ee00700 1 -- 172.21.15.33:0/1373145410 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f878810b690 msgr2=0x7f878819baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:34:53.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.741+0000 7f878ee00700 1 --2- 172.21.15.33:0/1373145410 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f878810b690 0x7f878819baf0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f878000b070 tx=0x7f8780004d50 comp rx=0 tx=0).stop 2024-05-03T18:34:53.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.741+0000 7f878ee00700 1 -- 172.21.15.33:0/1373145410 shutdown_connections 2024-05-03T18:34:53.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.741+0000 7f878ee00700 1 --2- 172.21.15.33:0/1373145410 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f876c0b2af0 0x7f876c0b4fb0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:53.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.741+0000 7f878ee00700 1 --2- 172.21.15.33:0/1373145410 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f878810b690 0x7f878819baf0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:53.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.741+0000 7f878ee00700 1 --2- 172.21.15.33:0/1373145410 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8788105690 0x7f87881a18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:53.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.741+0000 7f878ee00700 1 --2- 172.21.15.33:0/1373145410 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8788106050 0x7f87881a1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:53.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.741+0000 7f878ee00700 1 -- 172.21.15.33:0/1373145410 >> 172.21.15.33:0/1373145410 conn(0x7f8788101180 msgr2=0x7f87881018a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:34:53.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.741+0000 7f878ee00700 1 -- 172.21.15.33:0/1373145410 shutdown_connections 2024-05-03T18:34:53.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:53.741+0000 7f878ee00700 1 -- 172.21.15.33:0/1373145410 wait complete. 2024-05-03T18:34:54.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:34:54 smithi150 ceph-mon[32492]: pgmap v236: 129 pgs: 129 active+clean; 2.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 14 MiB/s wr, 857 op/s 2024-05-03T18:34:54.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:34:54 smithi033 ceph-mon[28330]: pgmap v236: 129 pgs: 129 active+clean; 2.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 14 MiB/s wr, 857 op/s 2024-05-03T18:34:54.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:34:54 smithi099 ceph-mon[31658]: pgmap v236: 129 pgs: 129 active+clean; 2.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 14 MiB/s wr, 857 op/s 2024-05-03T18:34:55.013 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-03T18:34:55.110 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 220.0 s, 539.2 tps, lat 184.679 ms stddev 260.190 2024-05-03T18:34:55.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:34:55 smithi033 ceph-mon[28330]: scrub queued for path: / 2024-05-03T18:34:55.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:34:55 smithi033 ceph-mon[28330]: scrub summary: idle+waiting paths [/] 2024-05-03T18:34:55.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:34:55 smithi033 ceph-mon[28330]: scrub summary: active paths [/] 2024-05-03T18:34:55.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:34:55 smithi033 ceph-mon[28330]: scrub summary: idle+waiting paths [/] 2024-05-03T18:34:55.384 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:34:55 smithi150 ceph-mon[32492]: scrub queued for path: / 2024-05-03T18:34:55.384 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:34:55 smithi150 ceph-mon[32492]: scrub summary: idle+waiting paths [/] 2024-05-03T18:34:55.385 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:34:55 smithi150 ceph-mon[32492]: scrub summary: active paths [/] 2024-05-03T18:34:55.385 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:34:55 smithi150 ceph-mon[32492]: scrub summary: idle+waiting paths [/] 2024-05-03T18:34:55.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:34:55 smithi099 ceph-mon[31658]: scrub queued for path: / 2024-05-03T18:34:55.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:34:55 smithi099 ceph-mon[31658]: scrub summary: idle+waiting paths [/] 2024-05-03T18:34:55.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:34:55 smithi099 ceph-mon[31658]: scrub summary: active paths [/] 2024-05-03T18:34:55.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:34:55 smithi099 ceph-mon[31658]: scrub summary: idle+waiting paths [/] 2024-05-03T18:34:55.751 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:34:56.182 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:34:56 smithi033 ceph-mon[28330]: pgmap v237: 129 pgs: 129 active+clean; 2.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 9.0 MiB/s wr, 569 op/s 2024-05-03T18:34:56.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:34:56 smithi150 ceph-mon[32492]: pgmap v237: 129 pgs: 129 active+clean; 2.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 9.0 MiB/s wr, 569 op/s 2024-05-03T18:34:56.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:34:56 smithi099 ceph-mon[31658]: pgmap v237: 129 pgs: 129 active+clean; 2.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 9.0 MiB/s wr, 569 op/s 2024-05-03T18:34:57.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:34:57 smithi033 ceph-mon[28330]: scrub summary: idle 2024-05-03T18:34:57.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:34:57 smithi150 ceph-mon[32492]: scrub summary: idle 2024-05-03T18:34:57.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:34:57 smithi099 ceph-mon[31658]: scrub summary: idle 2024-05-03T18:34:58.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:34:58 smithi150 ceph-mon[32492]: pgmap v238: 129 pgs: 129 active+clean; 2.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 11 MiB/s wr, 732 op/s 2024-05-03T18:34:58.408 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:34:58 smithi033 ceph-mon[28330]: pgmap v238: 129 pgs: 129 active+clean; 2.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 11 MiB/s wr, 732 op/s 2024-05-03T18:34:58.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:34:58 smithi099 ceph-mon[31658]: pgmap v238: 129 pgs: 129 active+clean; 2.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 11 MiB/s wr, 732 op/s 2024-05-03T18:34:58.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.628+0000 7f199ee00700 1 -- 172.21.15.33:0/3530806758 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1998103ee0 msgr2=0x7f19981103c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:34:58.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.628+0000 7f199ee00700 1 --2- 172.21.15.33:0/3530806758 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1998103ee0 0x7f19981103c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1990009f90 tx=0x7f199000b330 comp rx=0 tx=0).stop 2024-05-03T18:34:58.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.628+0000 7f199ee00700 1 -- 172.21.15.33:0/3530806758 shutdown_connections 2024-05-03T18:34:58.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.628+0000 7f199ee00700 1 --2- 172.21.15.33:0/3530806758 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1998103ee0 0x7f19981103c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:58.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.628+0000 7f199ee00700 1 --2- 172.21.15.33:0/3530806758 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1998109630 0x7f1998109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:58.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.628+0000 7f199ee00700 1 --2- 172.21.15.33:0/3530806758 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1998103520 0x7f19981039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:58.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.628+0000 7f199ee00700 1 -- 172.21.15.33:0/3530806758 >> 172.21.15.33:0/3530806758 conn(0x7f199807a650 msgr2=0x7f199807aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:34:58.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.628+0000 7f199ee00700 1 -- 172.21.15.33:0/3530806758 shutdown_connections 2024-05-03T18:34:58.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.628+0000 7f199ee00700 1 -- 172.21.15.33:0/3530806758 wait complete. 2024-05-03T18:34:58.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.630+0000 7f199ee00700 1 Processor -- start 2024-05-03T18:34:58.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.630+0000 7f199ee00700 1 -- start start 2024-05-03T18:34:58.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.631+0000 7f199ee00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1998103520 0x7f199819f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:34:58.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.631+0000 7f199ee00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1998103ee0 0x7f199819fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:34:58.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.631+0000 7f199ee00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1998109630 0x7f1998199920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:34:58.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.631+0000 7f1997e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1998103520 0x7f199819f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:34:58.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.631+0000 7f1997e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1998103520 0x7f199819f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:36218/0 (socket says 172.21.15.33:36218) 2024-05-03T18:34:58.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.631+0000 7f1997e00700 1 -- 172.21.15.33:0/877281870 learned_addr learned my addr 172.21.15.33:0/877281870 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:34:58.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.631+0000 7f1997400700 1 --2- 172.21.15.33:0/877281870 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1998103ee0 0x7f199819fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:34:58.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.631+0000 7f199ee00700 1 -- 172.21.15.33:0/877281870 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1998113600 con 0x7f1998103ee0 2024-05-03T18:34:58.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.632+0000 7f199ee00700 1 -- 172.21.15.33:0/877281870 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1998113480 con 0x7f1998109630 2024-05-03T18:34:58.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.632+0000 7f199ee00700 1 -- 172.21.15.33:0/877281870 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f1998113780 con 0x7f1998103520 2024-05-03T18:34:58.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.632+0000 7f199ce00700 1 --2- 172.21.15.33:0/877281870 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1998109630 0x7f1998199920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:34:58.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.633+0000 7f1997e00700 1 -- 172.21.15.33:0/877281870 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1998109630 msgr2=0x7f1998199920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:34:58.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.633+0000 7f1997e00700 1 --2- 172.21.15.33:0/877281870 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1998109630 0x7f1998199920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:58.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.633+0000 7f1997e00700 1 -- 172.21.15.33:0/877281870 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1998103ee0 msgr2=0x7f199819fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:34:58.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.633+0000 7f1997e00700 1 --2- 172.21.15.33:0/877281870 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1998103ee0 0x7f199819fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:58.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.633+0000 7f1997e00700 1 -- 172.21.15.33:0/877281870 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f199819a080 con 0x7f1998103520 2024-05-03T18:34:58.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.634+0000 7f1997e00700 1 --2- 172.21.15.33:0/877281870 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1998103520 0x7f199819f760 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f198800d980 tx=0x7f198800dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:34:58.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.635+0000 7f1994c00700 1 -- 172.21.15.33:0/877281870 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1988016070 con 0x7f1998103520 2024-05-03T18:34:58.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.635+0000 7f1994c00700 1 -- 172.21.15.33:0/877281870 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f198800fd20 con 0x7f1998103520 2024-05-03T18:34:58.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.635+0000 7f1994c00700 1 -- 172.21.15.33:0/877281870 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19880093a0 con 0x7f1998103520 2024-05-03T18:34:58.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.635+0000 7f199ee00700 1 -- 172.21.15.33:0/877281870 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f199819a370 con 0x7f1998103520 2024-05-03T18:34:58.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.635+0000 7f199ee00700 1 -- 172.21.15.33:0/877281870 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19980756d0 con 0x7f1998103520 2024-05-03T18:34:58.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.646+0000 7f19a977bb80 1 Processor -- start 2024-05-03T18:34:58.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.647+0000 7f19a977bb80 1 -- start start 2024-05-03T18:34:58.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.648+0000 7f19a977bb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f4855667b0 0x55f485566b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:34:58.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.648+0000 7f19a977bb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f4855670d0 0x55f485561fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:34:58.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.648+0000 7f19a977bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4855625d0 0x55f485562a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:34:58.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.648+0000 7f19a977bb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55f4854b4a20 con 0x55f4855670d0 2024-05-03T18:34:58.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.648+0000 7f19a977bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55f485452760 con 0x55f4855625d0 2024-05-03T18:34:58.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.648+0000 7f19a977bb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55f4854b09e0 con 0x55f4855667b0 2024-05-03T18:34:58.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.648+0000 7f199ce00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f4855667b0 0x55f485566b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:34:58.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.648+0000 7f199ce00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f4855667b0 0x55f485566b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:36228/0 (socket says 172.21.15.33:36228) 2024-05-03T18:34:58.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.648+0000 7f199ce00700 1 -- 172.21.15.33:0/823724184 learned_addr learned my addr 172.21.15.33:0/823724184 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:34:58.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.649+0000 7f1994c00700 1 -- 172.21.15.33:0/877281870 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f1988019070 con 0x7f1998103520 2024-05-03T18:34:58.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.649+0000 7f199ce00700 1 -- 172.21.15.33:0/823724184 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4855625d0 msgr2=0x55f485562a30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-03T18:34:58.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.649+0000 7f199ce00700 1 --2- 172.21.15.33:0/823724184 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4855625d0 0x55f485562a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:58.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.649+0000 7f199ce00700 1 -- 172.21.15.33:0/823724184 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f4855670d0 msgr2=0x55f485561fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:34:58.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.650+0000 7f199ce00700 1 --2- 172.21.15.33:0/823724184 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f4855670d0 0x55f485561fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:58.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.650+0000 7f199ce00700 1 -- 172.21.15.33:0/823724184 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4854018d0 con 0x55f4855667b0 2024-05-03T18:34:58.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.650+0000 7f199ce00700 1 --2- 172.21.15.33:0/823724184 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f4855667b0 0x55f485566b90 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1990009da0 tx=0x7f19900094e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fdfbecaa54f17f72 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:34:58.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.652+0000 7f1994c00700 1 --2- 172.21.15.33:0/877281870 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f197c0b2a70 0x7f197c0b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:34:58.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.653+0000 7f198d600700 1 -- 172.21.15.33:0/823724184 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1990020070 con 0x55f4855667b0 2024-05-03T18:34:58.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.653+0000 7f198d600700 1 -- 172.21.15.33:0/823724184 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1990007420 con 0x55f4855667b0 2024-05-03T18:34:58.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.653+0000 7f198d600700 1 -- 172.21.15.33:0/823724184 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19900052a0 con 0x55f4855667b0 2024-05-03T18:34:58.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.655+0000 7f19a977bb80 1 -- 172.21.15.33:0/823724184 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f4855667b0 msgr2=0x55f485566b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:34:58.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.655+0000 7f19a977bb80 1 --2- 172.21.15.33:0/823724184 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f4855667b0 0x55f485566b90 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1990009da0 tx=0x7f19900094e0 comp rx=0 tx=0).stop 2024-05-03T18:34:58.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.655+0000 7f1994c00700 1 -- 172.21.15.33:0/877281870 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9518+0+0 (secure 0 0 0) 0x7f19880e2730 con 0x7f1998103520 2024-05-03T18:34:58.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.656+0000 7f19a977bb80 1 -- 172.21.15.33:0/823724184 shutdown_connections 2024-05-03T18:34:58.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.656+0000 7f19a977bb80 1 --2- 172.21.15.33:0/823724184 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4855625d0 0x55f485562a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:58.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.656+0000 7f19a977bb80 1 --2- 172.21.15.33:0/823724184 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f4855667b0 0x55f485566b90 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:58.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.656+0000 7f19a977bb80 1 --2- 172.21.15.33:0/823724184 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f4855670d0 0x55f485561fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:58.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.656+0000 7f19a977bb80 1 -- 172.21.15.33:0/823724184 >> 172.21.15.33:0/823724184 conn(0x55f485490b60 msgr2=0x55f48555c7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:34:58.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.657+0000 7f1997e00700 1 --2- 172.21.15.33:0/877281870 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f197c0b2a70 0x7f197c0b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:34:58.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.658+0000 7f19a977bb80 1 -- 172.21.15.33:0/823724184 shutdown_connections 2024-05-03T18:34:58.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.659+0000 7f19a977bb80 1 -- 172.21.15.33:0/823724184 wait complete. 2024-05-03T18:34:58.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.659+0000 7f1997e00700 1 --2- 172.21.15.33:0/877281870 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f197c0b2a70 0x7f197c0b4f30 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f19880a68a0 tx=0x7f1988000bc0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:34:58.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.660+0000 7f19a977bb80 1 -- start start 2024-05-03T18:34:58.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.660+0000 7f19a977bb80 1 Processor -- start 2024-05-03T18:34:58.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.661+0000 7f19a977bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4855625d0 0x55f48556df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:34:58.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.661+0000 7f19a977bb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f4855667b0 0x55f48556b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:34:58.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.661+0000 7f19a977bb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f4855670d0 0x55f48556b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:34:58.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.661+0000 7f19a977bb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55f48556ec40 con 0x55f4855667b0 2024-05-03T18:34:58.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.661+0000 7f19a977bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55f48541ab20 con 0x55f4855625d0 2024-05-03T18:34:58.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.662+0000 7f19a977bb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55f48556edc0 con 0x55f4855670d0 2024-05-03T18:34:58.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.662+0000 7f1997400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f4855667b0 0x55f48556b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:34:58.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.662+0000 7f19a977bb80 10 client.0 _ll_register_callbacks cb 0x55f48545bc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:34:58.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.663+0000 7f1997400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f4855667b0 0x55f48556b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33070/0 (socket says 172.21.15.33:33070) 2024-05-03T18:34:58.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.663+0000 7f1997400700 1 -- 172.21.15.33:0/4069712621 learned_addr learned my addr 172.21.15.33:0/4069712621 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:34:58.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.663+0000 7f199ce00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f4855670d0 0x55f48556b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:34:58.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.664+0000 7f1997400700 1 -- 172.21.15.33:0/4069712621 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f4855670d0 msgr2=0x55f48556b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:34:58.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.664+0000 7f1997400700 1 --2- 172.21.15.33:0/4069712621 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f4855670d0 0x55f48556b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:58.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.664+0000 7f199ce00700 1 --2- 172.21.15.33:0/4069712621 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4855625d0 0x55f48556df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:34:58.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.664+0000 7f1997400700 1 -- 172.21.15.33:0/4069712621 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4855625d0 msgr2=0x55f48556df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:34:58.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.664+0000 7f1997400700 1 --2- 172.21.15.33:0/4069712621 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4855625d0 0x55f48556df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:58.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.664+0000 7f1997400700 1 -- 172.21.15.33:0/4069712621 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f48547a420 con 0x55f4855667b0 2024-05-03T18:34:58.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.664+0000 7f199ce00700 1 --2- 172.21.15.33:0/4069712621 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4855625d0 0x55f48556df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:34:58.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.665+0000 7f1997400700 1 --2- 172.21.15.33:0/4069712621 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f4855667b0 0x55f48556b2f0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f1984009a60 tx=0x7f1984009e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d95b19a2729d359 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:34:58.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.665+0000 7f1982000700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:34:58.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.665+0000 7f1982000700 1 -- 172.21.15.33:0/4069712621 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f198400f040 con 0x55f4855667b0 2024-05-03T18:34:58.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.665+0000 7f1982000700 1 -- 172.21.15.33:0/4069712621 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9518+0+0 (secure 0 0 0) 0x7f19840135f0 con 0x55f4855667b0 2024-05-03T18:34:58.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.666+0000 7f19a977bb80 1 -- 172.21.15.33:0/4069712621 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1990007420 con 0x55f4855667b0 2024-05-03T18:34:58.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.666+0000 7f1982000700 1 -- 172.21.15.33:0/4069712621 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f1984021050 con 0x55f4855667b0 2024-05-03T18:34:58.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.666+0000 7f19a977bb80 10 client.16173 fetch_fsmap learned FSMap version 21 2024-05-03T18:34:58.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.666+0000 7f19a977bb80 1 -- 172.21.15.33:0/4069712621 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55f48558a550 con 0x55f4855667b0 2024-05-03T18:34:58.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.667+0000 7f1982000700 1 -- 172.21.15.33:0/4069712621 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1990007420 con 0x55f4855667b0 2024-05-03T18:34:58.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.667+0000 7f1982000700 1 -- 172.21.15.33:0/4069712621 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f1984012030 con 0x55f4855667b0 2024-05-03T18:34:58.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.667+0000 7f19a977bb80 10 client.16173 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:34:58.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.667+0000 7f19a977bb80 10 client.16173 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:34:58.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.667+0000 7f19a977bb80 20 client.16173 populate_metadata read hostname 'smithi033' 2024-05-03T18:34:58.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.667+0000 7f19a977bb80 1 --2- 172.21.15.33:0/4069712621 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55f48558cb90 0x55f48558ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:34:58.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.667+0000 7f19a977bb80 4 client.16173 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:34:58.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.667+0000 7f19a977bb80 1 -- 172.21.15.33:0/4069712621 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f48552f1d0 con 0x55f48558cb90 2024-05-03T18:34:58.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.668+0000 7f199ce00700 1 --2- 172.21.15.33:0/4069712621 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55f48558cb90 0x55f48558ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:34:58.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.671+0000 7f199ce00700 1 --2- 172.21.15.33:0/4069712621 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55f48558cb90 0x55f48558ef80 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff4b47d35187d5a4 server_cookie=a6c9ae5d6a88f7cd in_seq=0 out_seq=0 2024-05-03T18:34:58.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.671+0000 7f1982000700 10 client.16173 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:34:58.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.675+0000 7f1982000700 1 -- 172.21.15.33:0/4069712621 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f48552f1d0 con 0x55f48558cb90 2024-05-03T18:34:58.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.675+0000 7f1982000700 10 client.16173 handle_command_reply: tid=0 2024-05-03T18:34:58.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.675+0000 7f19a977bb80 1 client.16173 shutdown 2024-05-03T18:34:58.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.675+0000 7f19a977bb80 1 -- 172.21.15.33:0/4069712621 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f4855667b0 msgr2=0x55f48556b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:34:58.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.675+0000 7f19a977bb80 1 --2- 172.21.15.33:0/4069712621 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f4855667b0 0x55f48556b2f0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f1984009a60 tx=0x7f1984009e20 comp rx=0 tx=0).stop 2024-05-03T18:34:58.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.676+0000 7f19a977bb80 1 -- 172.21.15.33:0/4069712621 shutdown_connections 2024-05-03T18:34:58.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.676+0000 7f19a977bb80 1 --2- 172.21.15.33:0/4069712621 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55f48558cb90 0x55f48558ef80 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:58.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.676+0000 7f19a977bb80 1 --2- 172.21.15.33:0/4069712621 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f4855670d0 0x55f48556b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:58.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.676+0000 7f19a977bb80 1 --2- 172.21.15.33:0/4069712621 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4855625d0 0x55f48556df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:58.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.676+0000 7f19a977bb80 1 --2- 172.21.15.33:0/4069712621 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f4855667b0 0x55f48556b2f0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:58.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.676+0000 7f19a977bb80 1 -- 172.21.15.33:0/4069712621 >> 172.21.15.33:0/4069712621 conn(0x55f485490b60 msgr2=0x55f485567de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:34:58.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.677+0000 7f1982000700 0 client.16173 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:34:58.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.677+0000 7f19a977bb80 1 -- 172.21.15.33:0/4069712621 shutdown_connections 2024-05-03T18:34:58.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.677+0000 7f19a977bb80 1 -- 172.21.15.33:0/4069712621 wait complete. 2024-05-03T18:34:58.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.678+0000 7f19a977bb80 20 client.16173 trim_cache size 0 max 16384 2024-05-03T18:34:58.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.722+0000 7f19a977bb80 1 Processor -- start 2024-05-03T18:34:58.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.722+0000 7f19a977bb80 1 -- start start 2024-05-03T18:34:58.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.723+0000 7f19a977bb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f4855625d0 0x55f48556b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:34:58.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.723+0000 7f19a977bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4855667b0 0x55f48555c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:34:58.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.723+0000 7f19a977bb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f4855670d0 0x55f485561aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:34:58.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.723+0000 7f19a977bb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f19981ab3f0 con 0x55f4855670d0 2024-05-03T18:34:58.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.723+0000 7f19a977bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f19981adc50 con 0x55f4855667b0 2024-05-03T18:34:58.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.723+0000 7f19a977bb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f19981aba40 con 0x55f4855625d0 2024-05-03T18:34:58.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.723+0000 7f199ce00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f4855625d0 0x55f48556b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:34:58.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.724+0000 7f199ce00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f4855625d0 0x55f48556b920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:36242/0 (socket says 172.21.15.33:36242) 2024-05-03T18:34:58.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.724+0000 7f199ce00700 1 -- 172.21.15.33:0/4075533811 learned_addr learned my addr 172.21.15.33:0/4075533811 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:34:58.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.724+0000 7f1997400700 1 --2- 172.21.15.33:0/4075533811 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4855667b0 0x55f48555c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:34:58.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.724+0000 7f199ce00700 1 -- 172.21.15.33:0/4075533811 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4855667b0 msgr2=0x55f48555c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:34:58.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.725+0000 7f199ce00700 1 --2- 172.21.15.33:0/4075533811 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4855667b0 0x55f48555c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:58.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.725+0000 7f199ce00700 1 -- 172.21.15.33:0/4075533811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f4855670d0 msgr2=0x55f485561aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:34:58.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.725+0000 7f199ce00700 1 --2- 172.21.15.33:0/4075533811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f4855670d0 0x55f485561aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:58.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.725+0000 7f199ce00700 1 -- 172.21.15.33:0/4075533811 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f485589150 con 0x55f4855625d0 2024-05-03T18:34:58.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.725+0000 7f199ce00700 1 --2- 172.21.15.33:0/4075533811 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f4855625d0 0x55f48556b920 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f1990007d80 tx=0x7f1990007ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0fb0f1d2150f81c server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:34:58.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.726+0000 7f198cc00700 1 -- 172.21.15.33:0/4075533811 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1990020040 con 0x55f4855625d0 2024-05-03T18:34:58.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.726+0000 7f198cc00700 1 -- 172.21.15.33:0/4075533811 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1990007b40 con 0x55f4855625d0 2024-05-03T18:34:58.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.726+0000 7f198cc00700 1 -- 172.21.15.33:0/4075533811 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1990005440 con 0x55f4855625d0 2024-05-03T18:34:58.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.727+0000 7f19a977bb80 1 -- 172.21.15.33:0/4075533811 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f4855625d0 msgr2=0x55f48556b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:34:58.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.727+0000 7f19a977bb80 1 --2- 172.21.15.33:0/4075533811 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f4855625d0 0x55f48556b920 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f1990007d80 tx=0x7f1990007ad0 comp rx=0 tx=0).stop 2024-05-03T18:34:58.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.727+0000 7f19a977bb80 1 -- 172.21.15.33:0/4075533811 shutdown_connections 2024-05-03T18:34:58.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.727+0000 7f19a977bb80 1 --2- 172.21.15.33:0/4075533811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f4855670d0 0x55f485561aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:58.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.727+0000 7f19a977bb80 1 --2- 172.21.15.33:0/4075533811 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f4855625d0 0x55f48556b920 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:58.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.727+0000 7f19a977bb80 1 --2- 172.21.15.33:0/4075533811 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4855667b0 0x55f48555c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:58.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.727+0000 7f19a977bb80 1 -- 172.21.15.33:0/4075533811 >> 172.21.15.33:0/4075533811 conn(0x55f485490b60 msgr2=0x55f485561390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:34:58.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.728+0000 7f19a977bb80 1 -- 172.21.15.33:0/4075533811 shutdown_connections 2024-05-03T18:34:58.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.729+0000 7f19a977bb80 1 -- 172.21.15.33:0/4075533811 wait complete. 2024-05-03T18:34:58.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.729+0000 7f19a977bb80 1 -- start start 2024-05-03T18:34:58.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.729+0000 7f19a977bb80 1 Processor -- start 2024-05-03T18:34:58.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.731+0000 7f19a977bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4855625d0 0x55f485574390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:34:58.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.731+0000 7f19a977bb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f4855667b0 0x55f48555f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:34:58.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.731+0000 7f19a977bb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f4855670d0 0x55f48555fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:34:58.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.731+0000 7f19a977bb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f19981ac110 con 0x55f4855670d0 2024-05-03T18:34:58.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.731+0000 7f19a977bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f19981ad580 con 0x55f4855625d0 2024-05-03T18:34:58.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.731+0000 7f19a977bb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f19981ac7e0 con 0x55f4855667b0 2024-05-03T18:34:58.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.731+0000 7f19a977bb80 10 client.0 _ll_register_callbacks cb 0x55f48545bc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:34:58.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.731+0000 7f199ce00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4855625d0 0x55f485574390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:34:58.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.731+0000 7f199ce00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4855625d0 0x55f485574390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:54200/0 (socket says 172.21.15.33:54200) 2024-05-03T18:34:58.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.731+0000 7f199ce00700 1 -- 172.21.15.33:0/3863177374 learned_addr learned my addr 172.21.15.33:0/3863177374 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:34:58.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.731+0000 7f199ce00700 1 --2- 172.21.15.33:0/3863177374 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f4855670d0 0x55f48555fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:34:58.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.732+0000 7f1997400700 1 --2- 172.21.15.33:0/3863177374 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f4855667b0 0x55f48555f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:34:58.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.732+0000 7f199ce00700 1 -- 172.21.15.33:0/3863177374 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f4855667b0 msgr2=0x55f48555f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:34:58.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.732+0000 7f199ce00700 1 --2- 172.21.15.33:0/3863177374 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f4855667b0 0x55f48555f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:58.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.732+0000 7f199ce00700 1 -- 172.21.15.33:0/3863177374 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f4855670d0 msgr2=0x55f48555fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:34:58.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.732+0000 7f199ce00700 1 --2- 172.21.15.33:0/3863177374 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f4855670d0 0x55f48555fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:58.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.732+0000 7f199ce00700 1 -- 172.21.15.33:0/3863177374 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f485566400 con 0x55f4855625d0 2024-05-03T18:34:58.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.735+0000 7f199ce00700 1 --2- 172.21.15.33:0/3863177374 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4855625d0 0x55f485574390 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f199000b420 tx=0x7f199000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac8c51572e31c654 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:34:58.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.735+0000 7f1982000700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-03T18:34:58.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.735+0000 7f1982000700 1 -- 172.21.15.33:0/3863177374 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1990036020 con 0x55f4855625d0 2024-05-03T18:34:58.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.735+0000 7f19a977bb80 1 -- 172.21.15.33:0/3863177374 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1990007b40 con 0x55f4855625d0 2024-05-03T18:34:58.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.736+0000 7f1982000700 1 -- 172.21.15.33:0/3863177374 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9518+0+0 (secure 0 0 0) 0x7f19900050c0 con 0x55f4855625d0 2024-05-03T18:34:58.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.737+0000 7f1982000700 1 -- 172.21.15.33:0/3863177374 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f1990020040 con 0x55f4855625d0 2024-05-03T18:34:58.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.737+0000 7f19a977bb80 10 client.35828 fetch_fsmap learned FSMap version 21 2024-05-03T18:34:58.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.737+0000 7f19a977bb80 1 -- 172.21.15.33:0/3863177374 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55f48556e2c0 con 0x55f4855625d0 2024-05-03T18:34:58.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.738+0000 7f1982000700 1 -- 172.21.15.33:0/3863177374 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1990007b40 con 0x55f4855625d0 2024-05-03T18:34:58.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.738+0000 7f1982000700 1 -- 172.21.15.33:0/3863177374 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f199002e070 con 0x55f4855625d0 2024-05-03T18:34:58.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.738+0000 7f19a977bb80 10 client.35828 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:34:58.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.738+0000 7f19a977bb80 10 client.35828 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:34:58.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.738+0000 7f19a977bb80 20 client.35828 populate_metadata read hostname 'smithi033' 2024-05-03T18:34:58.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.738+0000 7f19a977bb80 1 --2- 172.21.15.33:0/3863177374 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55f4855f7400 0x55f4855f97f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:34:58.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.738+0000 7f19a977bb80 4 client.35828 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-03T18:34:58.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.738+0000 7f19a977bb80 1 -- 172.21.15.33:0/3863177374 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f4854c2400 con 0x55f4855f7400 2024-05-03T18:34:58.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.740+0000 7f1997400700 1 --2- 172.21.15.33:0/3863177374 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55f4855f7400 0x55f4855f97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:34:58.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.741+0000 7f1997400700 1 --2- 172.21.15.33:0/3863177374 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55f4855f7400 0x55f4855f97f0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=305a65653c04b317 server_cookie=d7d76afa20064827 in_seq=0 out_seq=0 2024-05-03T18:34:58.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.741+0000 7f1982000700 10 client.35828 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:34:58.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.742+0000 7f1982000700 1 -- 172.21.15.33:0/3863177374 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55f4854c2400 con 0x55f4855f7400 2024-05-03T18:34:58.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.742+0000 7f1982000700 10 client.35828 handle_command_reply: tid=0 2024-05-03T18:34:58.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.742+0000 7f19a977bb80 1 client.35828 shutdown 2024-05-03T18:34:58.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.743+0000 7f19a977bb80 1 -- 172.21.15.33:0/3863177374 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4855625d0 msgr2=0x55f485574390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:34:58.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.743+0000 7f19a977bb80 1 --2- 172.21.15.33:0/3863177374 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4855625d0 0x55f485574390 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f199000b420 tx=0x7f199000b330 comp rx=0 tx=0).stop 2024-05-03T18:34:58.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.744+0000 7f19a977bb80 1 -- 172.21.15.33:0/3863177374 shutdown_connections 2024-05-03T18:34:58.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.744+0000 7f19a977bb80 1 --2- 172.21.15.33:0/3863177374 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55f4855f7400 0x55f4855f97f0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:58.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.744+0000 7f19a977bb80 1 --2- 172.21.15.33:0/3863177374 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f4855670d0 0x55f48555fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:58.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.744+0000 7f19a977bb80 1 --2- 172.21.15.33:0/3863177374 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4855625d0 0x55f485574390 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:58.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.744+0000 7f19a977bb80 1 --2- 172.21.15.33:0/3863177374 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f4855667b0 0x55f48555f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:58.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.744+0000 7f19a977bb80 1 -- 172.21.15.33:0/3863177374 >> 172.21.15.33:0/3863177374 conn(0x55f485490b60 msgr2=0x55f485565c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:34:58.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.744+0000 7f1982000700 0 client.35828 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:34:58.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.744+0000 7f19a977bb80 1 -- 172.21.15.33:0/3863177374 shutdown_connections 2024-05-03T18:34:58.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.746+0000 7f19a977bb80 1 -- 172.21.15.33:0/3863177374 wait complete. 2024-05-03T18:34:58.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.746+0000 7f19a977bb80 20 client.35828 trim_cache size 0 max 16384 2024-05-03T18:34:58.747 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-03T18:34:58.748 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-05-03T18:34:58.748 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-05-03T18:34:58.748 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-03T18:34:58.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.748+0000 7f199ee00700 1 -- 172.21.15.33:0/877281870 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f197c0b2a70 msgr2=0x7f197c0b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:34:58.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.748+0000 7f199ee00700 1 --2- 172.21.15.33:0/877281870 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f197c0b2a70 0x7f197c0b4f30 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f19880a68a0 tx=0x7f1988000bc0 comp rx=0 tx=0).stop 2024-05-03T18:34:58.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.748+0000 7f199ee00700 1 -- 172.21.15.33:0/877281870 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1998103520 msgr2=0x7f199819f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:34:58.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.748+0000 7f199ee00700 1 --2- 172.21.15.33:0/877281870 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1998103520 0x7f199819f760 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f198800d980 tx=0x7f198800dc90 comp rx=0 tx=0).stop 2024-05-03T18:34:58.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.749+0000 7f199ee00700 1 -- 172.21.15.33:0/877281870 shutdown_connections 2024-05-03T18:34:58.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.749+0000 7f199ee00700 1 --2- 172.21.15.33:0/877281870 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f197c0b2a70 0x7f197c0b4f30 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:58.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.749+0000 7f199ee00700 1 --2- 172.21.15.33:0/877281870 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1998109630 0x7f1998199920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:58.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.749+0000 7f199ee00700 1 --2- 172.21.15.33:0/877281870 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1998103520 0x7f199819f760 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:58.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.749+0000 7f199ee00700 1 --2- 172.21.15.33:0/877281870 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1998103ee0 0x7f199819fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:34:58.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.749+0000 7f199ee00700 1 -- 172.21.15.33:0/877281870 >> 172.21.15.33:0/877281870 conn(0x7f199807a650 msgr2=0x7f1998102280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:34:58.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.749+0000 7f199ee00700 1 -- 172.21.15.33:0/877281870 shutdown_connections 2024-05-03T18:34:58.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:34:58.750+0000 7f199ee00700 1 -- 172.21.15.33:0/877281870 wait complete. 2024-05-03T18:34:59.734 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-03T18:34:59.735 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T18:35:00.111 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 225.0 s, 526.0 tps, lat 152.062 ms stddev 45.323 2024-05-03T18:35:00.375 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:35:00.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:35:00 smithi150 ceph-mon[32492]: pgmap v239: 129 pgs: 129 active+clean; 2.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 8.1 MiB/s wr, 607 op/s 2024-05-03T18:35:00.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:35:00 smithi033 ceph-mon[28330]: pgmap v239: 129 pgs: 129 active+clean; 2.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 8.1 MiB/s wr, 607 op/s 2024-05-03T18:35:00.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:35:00 smithi099 ceph-mon[31658]: pgmap v239: 129 pgs: 129 active+clean; 2.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 8.1 MiB/s wr, 607 op/s 2024-05-03T18:35:02.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:35:02 smithi033 ceph-mon[28330]: osdmap e93: 12 total, 12 up, 12 in 2024-05-03T18:35:02.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:35:02 smithi033 ceph-mon[28330]: pgmap v241: 129 pgs: 129 active+clean; 2.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 9.3 MiB/s wr, 679 op/s 2024-05-03T18:35:02.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:35:02 smithi099 ceph-mon[31658]: osdmap e93: 12 total, 12 up, 12 in 2024-05-03T18:35:02.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:35:02 smithi099 ceph-mon[31658]: pgmap v241: 129 pgs: 129 active+clean; 2.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 9.3 MiB/s wr, 679 op/s 2024-05-03T18:35:02.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:35:02 smithi150 ceph-mon[32492]: osdmap e93: 12 total, 12 up, 12 in 2024-05-03T18:35:02.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:35:02 smithi150 ceph-mon[32492]: pgmap v241: 129 pgs: 129 active+clean; 2.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 9.3 MiB/s wr, 679 op/s 2024-05-03T18:35:04.086 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:35:04 smithi033 ceph-mon[28330]: pgmap v242: 129 pgs: 129 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 8.8 MiB/s wr, 624 op/s 2024-05-03T18:35:04.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:35:04 smithi150 ceph-mon[32492]: pgmap v242: 129 pgs: 129 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 8.8 MiB/s wr, 624 op/s 2024-05-03T18:35:04.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:35:04 smithi099 ceph-mon[31658]: pgmap v242: 129 pgs: 129 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 8.8 MiB/s wr, 624 op/s 2024-05-03T18:35:04.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:04.985+0000 7f1b27e00700 1 -- 172.21.15.33:0/3596862387 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b2010ff50 msgr2=0x7f1b20112340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:35:04.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:04.985+0000 7f1b27e00700 1 --2- 172.21.15.33:0/3596862387 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b2010ff50 0x7f1b20112340 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1b1800b0a0 tx=0x7f1b1800b3b0 comp rx=0 tx=0).stop 2024-05-03T18:35:04.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:04.985+0000 7f1b27e00700 1 -- 172.21.15.33:0/3596862387 shutdown_connections 2024-05-03T18:35:04.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:04.985+0000 7f1b27e00700 1 --2- 172.21.15.33:0/3596862387 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b2010ff50 0x7f1b20112340 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:04.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:04.985+0000 7f1b27e00700 1 --2- 172.21.15.33:0/3596862387 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b2006b1b0 0x7f1b2006b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:04.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:04.985+0000 7f1b27e00700 1 --2- 172.21.15.33:0/3596862387 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b2006bad0 0x7f1b2010fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:04.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:04.986+0000 7f1b27e00700 1 -- 172.21.15.33:0/3596862387 >> 172.21.15.33:0/3596862387 conn(0x7f1b20079350 msgr2=0x7f1b20079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:35:04.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:04.986+0000 7f1b27e00700 1 -- 172.21.15.33:0/3596862387 shutdown_connections 2024-05-03T18:35:04.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:04.986+0000 7f1b27e00700 1 -- 172.21.15.33:0/3596862387 wait complete. 2024-05-03T18:35:04.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:04.987+0000 7f1b27e00700 1 Processor -- start 2024-05-03T18:35:04.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:04.988+0000 7f1b27e00700 1 -- start start 2024-05-03T18:35:04.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:04.989+0000 7f1b27e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b2006b1b0 0x7f1b2010f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:04.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:04.990+0000 7f1b27e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b2006bad0 0x7f1b2010d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:04.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:04.990+0000 7f1b27e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b2010ff50 0x7f1b2010ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:04.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:04.990+0000 7f1b27e00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1b20104cd0 con 0x7f1b2006bad0 2024-05-03T18:35:04.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:04.990+0000 7f1b25400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b2006b1b0 0x7f1b2010f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:04.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:04.990+0000 7f1b24a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b2006bad0 0x7f1b2010d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:04.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:04.990+0000 7f1b27e00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1b20104b50 con 0x7f1b2010ff50 2024-05-03T18:35:04.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:04.990+0000 7f1b27e00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f1b20104e50 con 0x7f1b2006b1b0 2024-05-03T18:35:04.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:04.990+0000 7f1b24a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b2006bad0 0x7f1b2010d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33094/0 (socket says 172.21.15.33:33094) 2024-05-03T18:35:04.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:04.990+0000 7f1b24a00700 1 -- 172.21.15.33:0/1044217060 learned_addr learned my addr 172.21.15.33:0/1044217060 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:35:04.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:04.991+0000 7f1b25e00700 1 --2- 172.21.15.33:0/1044217060 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b2010ff50 0x7f1b2010ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:04.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:04.991+0000 7f1b24a00700 1 -- 172.21.15.33:0/1044217060 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b2006b1b0 msgr2=0x7f1b2010f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:35:04.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:04.991+0000 7f1b24a00700 1 --2- 172.21.15.33:0/1044217060 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b2006b1b0 0x7f1b2010f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:04.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:04.991+0000 7f1b24a00700 1 -- 172.21.15.33:0/1044217060 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b2010ff50 msgr2=0x7f1b2010ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:35:04.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:04.991+0000 7f1b24a00700 1 --2- 172.21.15.33:0/1044217060 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b2010ff50 0x7f1b2010ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:04.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:04.991+0000 7f1b24a00700 1 -- 172.21.15.33:0/1044217060 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b2010e6c0 con 0x7f1b2006bad0 2024-05-03T18:35:04.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:04.992+0000 7f1b25400700 1 --2- 172.21.15.33:0/1044217060 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b2006b1b0 0x7f1b2010f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:35:04.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:04.992+0000 7f1b24a00700 1 --2- 172.21.15.33:0/1044217060 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b2006bad0 0x7f1b2010d880 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f1b1400d980 tx=0x7f1b1400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:35:04.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:04.992+0000 7f1b1e000700 1 -- 172.21.15.33:0/1044217060 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b14016070 con 0x7f1b2006bad0 2024-05-03T18:35:04.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:04.992+0000 7f1b1e000700 1 -- 172.21.15.33:0/1044217060 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1b14004830 con 0x7f1b2006bad0 2024-05-03T18:35:04.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:04.992+0000 7f1b1e000700 1 -- 172.21.15.33:0/1044217060 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b14005480 con 0x7f1b2006bad0 2024-05-03T18:35:04.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:04.992+0000 7f1b27e00700 1 -- 172.21.15.33:0/1044217060 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b201ac5f0 con 0x7f1b2006bad0 2024-05-03T18:35:04.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:04.993+0000 7f1b27e00700 1 -- 172.21.15.33:0/1044217060 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b201acae0 con 0x7f1b2006bad0 2024-05-03T18:35:04.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:04.994+0000 7f1b27e00700 1 -- 172.21.15.33:0/1044217060 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b2006a190 con 0x7f1b2006bad0 2024-05-03T18:35:04.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:04.998+0000 7f1b1e000700 1 -- 172.21.15.33:0/1044217060 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f1b14005620 con 0x7f1b2006bad0 2024-05-03T18:35:05.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:05.000+0000 7f1b1e000700 1 --2- 172.21.15.33:0/1044217060 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f1b080b2bc0 0x7f1b080b5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:05.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:05.000+0000 7f1b1e000700 1 -- 172.21.15.33:0/1044217060 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9606+0+0 (secure 0 0 0) 0x7f1b140e2650 con 0x7f1b2006bad0 2024-05-03T18:35:05.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:05.001+0000 7f1b25400700 1 --2- 172.21.15.33:0/1044217060 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f1b080b2bc0 0x7f1b080b5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:05.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:05.002+0000 7f1b25400700 1 --2- 172.21.15.33:0/1044217060 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f1b080b2bc0 0x7f1b080b5080 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1b10009680 tx=0x7f1b10009260 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:35:05.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:05.011+0000 7f1b1e000700 1 -- 172.21.15.33:0/1044217060 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f1b1401f020 con 0x7f1b2006bad0 2024-05-03T18:35:05.113 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 230.0 s, 609.7 tps, lat 196.298 ms stddev 283.628 2024-05-03T18:35:05.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:05.540+0000 7f1b27e00700 1 -- 172.21.15.33:0/1044217060 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1b2002e510 con 0x7f1b2006bad0 2024-05-03T18:35:05.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:05.542+0000 7f1b1e000700 1 -- 172.21.15.33:0/1044217060 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11801 (secure 0 0 0) 0x7f1b140a9b90 con 0x7f1b2006bad0 2024-05-03T18:35:05.544 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:35:05.544 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1327393503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1327393503},{"type":"v1","addr":"172.21.15.33:6835","nonce":1327393503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/2461472948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":2461472948},{"type":"v1","addr":"172.21.15.33:6837","nonce":2461472948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14559,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/442269281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":442269281},{"type":"v1","addr":"172.21.15.33:6839","nonce":442269281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14571,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/645571756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":645571756},{"type":"v1","addr":"172.21.15.33:6841","nonce":645571756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/203598909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":203598909},{"type":"v1","addr":"172.21.15.150:6833","nonce":203598909}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/4029842493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":4029842493},{"type":"v1","addr":"172.21.15.150:6835","nonce":4029842493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3463664423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3463664423},{"type":"v1","addr":"172.21.15.150:6837","nonce":3463664423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24481,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6839/4051648018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":4051648018},{"type":"v1","addr":"172.21.15.150:6839","nonce":4051648018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34418,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1865105479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1865105479},{"type":"v1","addr":"172.21.15.99:6833","nonce":1865105479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34445,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/1206393914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1206393914},{"type":"v1","addr":"172.21.15.99:6835","nonce":1206393914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-03T18:22:28.837091+0000","modified":"2024-05-03T18:23:49.758398+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34508},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34472":{"gid":34472,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6837/3360970342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3360970342},{"type":"v1","addr":"172.21.15.99:6837","nonce":3360970342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34508":{"gid":34508,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/919907308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":919907308},{"type":"v1","addr":"172.21.15.99:6839","nonce":919907308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34508,"qdb_cluster":[34508]},"id":1}]} 2024-05-03T18:35:05.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:05.547+0000 7f1b27e00700 1 -- 172.21.15.33:0/1044217060 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f1b080b2bc0 msgr2=0x7f1b080b5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:35:05.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:05.547+0000 7f1b27e00700 1 --2- 172.21.15.33:0/1044217060 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f1b080b2bc0 0x7f1b080b5080 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1b10009680 tx=0x7f1b10009260 comp rx=0 tx=0).stop 2024-05-03T18:35:05.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:05.547+0000 7f1b27e00700 1 -- 172.21.15.33:0/1044217060 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b2006bad0 msgr2=0x7f1b2010d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:35:05.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:05.547+0000 7f1b27e00700 1 --2- 172.21.15.33:0/1044217060 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b2006bad0 0x7f1b2010d880 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f1b1400d980 tx=0x7f1b1400dc90 comp rx=0 tx=0).stop 2024-05-03T18:35:05.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:05.547+0000 7f1b27e00700 1 -- 172.21.15.33:0/1044217060 shutdown_connections 2024-05-03T18:35:05.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:05.547+0000 7f1b27e00700 1 --2- 172.21.15.33:0/1044217060 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f1b080b2bc0 0x7f1b080b5080 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:05.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:05.547+0000 7f1b27e00700 1 --2- 172.21.15.33:0/1044217060 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b2010ff50 0x7f1b2010ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:05.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:05.547+0000 7f1b27e00700 1 --2- 172.21.15.33:0/1044217060 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b2006b1b0 0x7f1b2010f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:05.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:05.547+0000 7f1b27e00700 1 --2- 172.21.15.33:0/1044217060 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b2006bad0 0x7f1b2010d880 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:05.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:05.547+0000 7f1b27e00700 1 -- 172.21.15.33:0/1044217060 >> 172.21.15.33:0/1044217060 conn(0x7f1b20079350 msgr2=0x7f1b20111480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:35:05.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:05.547+0000 7f1b27e00700 1 -- 172.21.15.33:0/1044217060 shutdown_connections 2024-05-03T18:35:05.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:05.548+0000 7f1b27e00700 1 -- 172.21.15.33:0/1044217060 wait complete. 2024-05-03T18:35:05.552 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 21 2024-05-03T18:35:05.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:35:05 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/1044217060' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:35:05.970 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:35:05 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/1044217060' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:35:06.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:35:05 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/1044217060' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:35:06.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:35:06 smithi150 ceph-mon[32492]: pgmap v243: 129 pgs: 129 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 8.8 MiB/s wr, 624 op/s 2024-05-03T18:35:06.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:35:06 smithi033 ceph-mon[28330]: pgmap v243: 129 pgs: 129 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 8.8 MiB/s wr, 624 op/s 2024-05-03T18:35:07.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:35:06 smithi099 ceph-mon[31658]: pgmap v243: 129 pgs: 129 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 8.8 MiB/s wr, 624 op/s 2024-05-03T18:35:08.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:35:08 smithi150 ceph-mon[32492]: osdmap e94: 12 total, 12 up, 12 in 2024-05-03T18:35:08.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:35:08 smithi150 ceph-mon[32492]: pgmap v245: 129 pgs: 129 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 6.5 MiB/s wr, 422 op/s 2024-05-03T18:35:08.735 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:35:08 smithi033 ceph-mon[28330]: osdmap e94: 12 total, 12 up, 12 in 2024-05-03T18:35:08.735 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:35:08 smithi033 ceph-mon[28330]: pgmap v245: 129 pgs: 129 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 6.5 MiB/s wr, 422 op/s 2024-05-03T18:35:08.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:35:08 smithi099 ceph-mon[31658]: osdmap e94: 12 total, 12 up, 12 in 2024-05-03T18:35:08.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:35:08 smithi099 ceph-mon[31658]: pgmap v245: 129 pgs: 129 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 6.5 MiB/s wr, 422 op/s 2024-05-03T18:35:09.811 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-03T18:35:10.111 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 235.0 s, 572.2 tps, lat 175.729 ms stddev 104.258 2024-05-03T18:35:10.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:35:10 smithi150 ceph-mon[32492]: pgmap v246: 129 pgs: 129 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 9.5 MiB/s wr, 702 op/s 2024-05-03T18:35:10.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:35:10 smithi033 ceph-mon[28330]: pgmap v246: 129 pgs: 129 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 9.5 MiB/s wr, 702 op/s 2024-05-03T18:35:10.521 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:35:10.547 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:35:10 smithi099 ceph-mon[31658]: pgmap v246: 129 pgs: 129 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 9.5 MiB/s wr, 702 op/s 2024-05-03T18:35:12.351 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:35:12 smithi033 ceph-mon[28330]: pgmap v247: 129 pgs: 129 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 8.4 MiB/s wr, 618 op/s 2024-05-03T18:35:12.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:35:12 smithi150 ceph-mon[32492]: pgmap v247: 129 pgs: 129 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 8.4 MiB/s wr, 618 op/s 2024-05-03T18:35:12.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:35:12 smithi099 ceph-mon[31658]: pgmap v247: 129 pgs: 129 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 8.4 MiB/s wr, 618 op/s 2024-05-03T18:35:12.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:12.987+0000 7f32a8a00700 1 -- 172.21.15.33:0/2166937341 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32a4101880 msgr2=0x7f32a4101c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:35:12.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:12.987+0000 7f32a8a00700 1 --2- 172.21.15.33:0/2166937341 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32a4101880 0x7f32a4101c60 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3294009990 tx=0x7f3294009ca0 comp rx=0 tx=0).stop 2024-05-03T18:35:12.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:12.990+0000 7f32a8a00700 1 -- 172.21.15.33:0/2166937341 shutdown_connections 2024-05-03T18:35:12.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:12.990+0000 7f32a8a00700 1 --2- 172.21.15.33:0/2166937341 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32a410fc90 0x7f32a4112130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:12.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:12.990+0000 7f32a8a00700 1 --2- 172.21.15.33:0/2166937341 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32a4101880 0x7f32a4101c60 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:12.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:12.990+0000 7f32a8a00700 1 --2- 172.21.15.33:0/2166937341 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32a41021a0 0x7f32a410f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:12.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:12.990+0000 7f32a8a00700 1 -- 172.21.15.33:0/2166937341 >> 172.21.15.33:0/2166937341 conn(0x7f32a40785a0 msgr2=0x7f32a40ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:35:12.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:12.991+0000 7f32a8a00700 1 -- 172.21.15.33:0/2166937341 shutdown_connections 2024-05-03T18:35:12.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:12.991+0000 7f32a8a00700 1 -- 172.21.15.33:0/2166937341 wait complete. 2024-05-03T18:35:12.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:12.992+0000 7f32a8a00700 1 Processor -- start 2024-05-03T18:35:12.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:12.993+0000 7f32a8a00700 1 -- start start 2024-05-03T18:35:12.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:12.993+0000 7f32a8a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32a4101880 0x7f32a4105260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:12.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:12.993+0000 7f32a8a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32a41021a0 0x7f32a41057a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:12.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:12.993+0000 7f32a8a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32a410fc90 0x7f32a4105ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:12.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:12.994+0000 7f32a8a00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f32a41157b0 con 0x7f32a41021a0 2024-05-03T18:35:12.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:12.994+0000 7f32a8a00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f32a4115630 con 0x7f32a410fc90 2024-05-03T18:35:12.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:12.994+0000 7f32a8a00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f32a4115930 con 0x7f32a4101880 2024-05-03T18:35:12.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:12.994+0000 7f32a1e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32a4101880 0x7f32a4105260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:12.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:12.994+0000 7f32a1e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32a4101880 0x7f32a4105260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:36766/0 (socket says 172.21.15.33:36766) 2024-05-03T18:35:12.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:12.994+0000 7f32a1e00700 1 -- 172.21.15.33:0/2505469964 learned_addr learned my addr 172.21.15.33:0/2505469964 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:35:12.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:12.994+0000 7f32a2800700 1 --2- 172.21.15.33:0/2505469964 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32a410fc90 0x7f32a4105ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:12.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:12.994+0000 7f32a1400700 1 --2- 172.21.15.33:0/2505469964 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32a41021a0 0x7f32a41057a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:12.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:12.995+0000 7f32a1e00700 1 -- 172.21.15.33:0/2505469964 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32a410fc90 msgr2=0x7f32a4105ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:35:12.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:12.995+0000 7f32a1e00700 1 --2- 172.21.15.33:0/2505469964 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32a410fc90 0x7f32a4105ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:12.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:12.995+0000 7f32a1e00700 1 -- 172.21.15.33:0/2505469964 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32a41021a0 msgr2=0x7f32a41057a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:35:12.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:12.995+0000 7f32a1e00700 1 --2- 172.21.15.33:0/2505469964 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32a41021a0 0x7f32a41057a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:12.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:12.995+0000 7f32a1e00700 1 -- 172.21.15.33:0/2505469964 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32a4073d50 con 0x7f32a4101880 2024-05-03T18:35:12.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:12.996+0000 7f32a1e00700 1 --2- 172.21.15.33:0/2505469964 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32a4101880 0x7f32a4105260 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f3294000c00 tx=0x7f32940047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:35:12.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:12.996+0000 7f329aa00700 1 -- 172.21.15.33:0/2505469964 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329400fa90 con 0x7f32a4101880 2024-05-03T18:35:12.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:12.997+0000 7f32a8a00700 1 -- 172.21.15.33:0/2505469964 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32a4073fe0 con 0x7f32a4101880 2024-05-03T18:35:12.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:12.997+0000 7f32a8a00700 1 -- 172.21.15.33:0/2505469964 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32a4074510 con 0x7f32a4101880 2024-05-03T18:35:12.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:12.997+0000 7f329aa00700 1 -- 172.21.15.33:0/2505469964 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f329400fc30 con 0x7f32a4101880 2024-05-03T18:35:12.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:12.997+0000 7f329aa00700 1 -- 172.21.15.33:0/2505469964 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329400fa90 con 0x7f32a4101880 2024-05-03T18:35:13.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.004+0000 7f32b33f5b80 1 Processor -- start 2024-05-03T18:35:13.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.005+0000 7f32b33f5b80 1 -- start start 2024-05-03T18:35:13.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.006+0000 7f32b33f5b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a6661eb930 0x55a6661ebd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:13.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.006+0000 7f32b33f5b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6661ec250 0x55a6661e5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:13.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.006+0000 7f32b33f5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6661e6550 0x55a6661e69d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:13.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.006+0000 7f32b33f5b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a666138a20 con 0x55a6661ec250 2024-05-03T18:35:13.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.006+0000 7f32b33f5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55a6660d6760 con 0x55a6661e6550 2024-05-03T18:35:13.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.006+0000 7f32b33f5b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55a6661349e0 con 0x55a6661eb930 2024-05-03T18:35:13.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.007+0000 7f32a2800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a6661eb930 0x55a6661ebd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:13.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.007+0000 7f32a2800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a6661eb930 0x55a6661ebd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:36770/0 (socket says 172.21.15.33:36770) 2024-05-03T18:35:13.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.007+0000 7f32a2800700 1 -- 172.21.15.33:0/4159405170 learned_addr learned my addr 172.21.15.33:0/4159405170 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:35:13.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.007+0000 7f32a1400700 1 --2- 172.21.15.33:0/4159405170 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6661e6550 0x55a6661e69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:13.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.008+0000 7f32a2800700 1 -- 172.21.15.33:0/4159405170 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6661e6550 msgr2=0x55a6661e69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:35:13.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.008+0000 7f32a2800700 1 --2- 172.21.15.33:0/4159405170 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6661e6550 0x55a6661e69d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:13.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.008+0000 7f32a2800700 1 -- 172.21.15.33:0/4159405170 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6661ec250 msgr2=0x55a6661e5f30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-03T18:35:13.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.008+0000 7f32a2800700 1 --2- 172.21.15.33:0/4159405170 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6661ec250 0x55a6661e5f30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:13.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.008+0000 7f32a2800700 1 -- 172.21.15.33:0/4159405170 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6660858d0 con 0x55a6661eb930 2024-05-03T18:35:13.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.009+0000 7f32a2800700 1 --2- 172.21.15.33:0/4159405170 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a6661eb930 0x55a6661ebd10 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f329c009440 tx=0x7f329c009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5cd32106deef86d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:35:13.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.010+0000 7f329aa00700 1 -- 172.21.15.33:0/2505469964 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f3294009d10 con 0x7f32a4101880 2024-05-03T18:35:13.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.010+0000 7f328f400700 1 -- 172.21.15.33:0/4159405170 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329c013810 con 0x55a6661eb930 2024-05-03T18:35:13.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.011+0000 7f328f400700 1 -- 172.21.15.33:0/4159405170 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f329c013e40 con 0x55a6661eb930 2024-05-03T18:35:13.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.011+0000 7f328f400700 1 -- 172.21.15.33:0/4159405170 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329c013810 con 0x55a6661eb930 2024-05-03T18:35:13.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.012+0000 7f32b33f5b80 1 -- 172.21.15.33:0/4159405170 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a6661eb930 msgr2=0x55a6661ebd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:35:13.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.012+0000 7f32b33f5b80 1 --2- 172.21.15.33:0/4159405170 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a6661eb930 0x55a6661ebd10 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f329c009440 tx=0x7f329c009920 comp rx=0 tx=0).stop 2024-05-03T18:35:13.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.014+0000 7f329aa00700 1 --2- 172.21.15.33:0/2505469964 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f32880b2ca0 0x7f32880b5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:13.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.015+0000 7f32a2800700 1 --2- 172.21.15.33:0/2505469964 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f32880b2ca0 0x7f32880b5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:13.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.015+0000 7f32b33f5b80 1 -- 172.21.15.33:0/4159405170 shutdown_connections 2024-05-03T18:35:13.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.015+0000 7f32b33f5b80 1 --2- 172.21.15.33:0/4159405170 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6661e6550 0x55a6661e69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:13.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.015+0000 7f32b33f5b80 1 --2- 172.21.15.33:0/4159405170 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a6661eb930 0x55a6661ebd10 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:13.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.015+0000 7f32b33f5b80 1 --2- 172.21.15.33:0/4159405170 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6661ec250 0x55a6661e5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:13.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.015+0000 7f32b33f5b80 1 -- 172.21.15.33:0/4159405170 >> 172.21.15.33:0/4159405170 conn(0x55a66615e6d0 msgr2=0x55a6661e0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:35:13.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.016+0000 7f32b33f5b80 1 -- 172.21.15.33:0/4159405170 shutdown_connections 2024-05-03T18:35:13.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.016+0000 7f32b33f5b80 1 -- 172.21.15.33:0/4159405170 wait complete. 2024-05-03T18:35:13.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.016+0000 7f329aa00700 1 -- 172.21.15.33:0/2505469964 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9518+0+0 (secure 0 0 0) 0x7f32940e3700 con 0x7f32a4101880 2024-05-03T18:35:13.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.017+0000 7f32b33f5b80 1 -- start start 2024-05-03T18:35:13.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.017+0000 7f32a2800700 1 --2- 172.21.15.33:0/2505469964 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f32880b2ca0 0x7f32880b5160 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f329c009440 tx=0x7f329c015bd0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:35:13.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.017+0000 7f32b33f5b80 1 Processor -- start 2024-05-03T18:35:13.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.018+0000 7f32b33f5b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6661e6550 0x55a6661f1e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:13.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.019+0000 7f32b33f5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6661eb930 0x55a6661ef200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:13.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.019+0000 7f32b33f5b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a6661ec250 0x55a6661ef740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:13.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.019+0000 7f32b33f5b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a6661f2d50 con 0x55a6661e6550 2024-05-03T18:35:13.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.019+0000 7f32b33f5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55a66609eb20 con 0x55a6661eb930 2024-05-03T18:35:13.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.019+0000 7f32b33f5b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55a6661f2ed0 con 0x55a6661ec250 2024-05-03T18:35:13.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.020+0000 7f32a2800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a6661ec250 0x55a6661ef740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:13.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.020+0000 7f32a2800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a6661ec250 0x55a6661ef740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:36778/0 (socket says 172.21.15.33:36778) 2024-05-03T18:35:13.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.020+0000 7f32a2800700 1 -- 172.21.15.33:0/3425484811 learned_addr learned my addr 172.21.15.33:0/3425484811 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:35:13.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.020+0000 7f32a1400700 1 --2- 172.21.15.33:0/3425484811 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6661eb930 0x55a6661ef200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:13.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.020+0000 7f32b33f5b80 10 client.0 _ll_register_callbacks cb 0x55a6660dfc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:35:13.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.021+0000 7f32a1e00700 1 --2- 172.21.15.33:0/3425484811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6661e6550 0x55a6661f1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:13.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.021+0000 7f32a2800700 1 -- 172.21.15.33:0/3425484811 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6661eb930 msgr2=0x55a6661ef200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:35:13.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.021+0000 7f32a2800700 1 --2- 172.21.15.33:0/3425484811 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6661eb930 0x55a6661ef200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:13.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.021+0000 7f32a2800700 1 -- 172.21.15.33:0/3425484811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6661e6550 msgr2=0x55a6661f1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:35:13.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.021+0000 7f32a2800700 1 --2- 172.21.15.33:0/3425484811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6661e6550 0x55a6661f1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:13.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.021+0000 7f32a2800700 1 -- 172.21.15.33:0/3425484811 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6660fe420 con 0x55a6661ec250 2024-05-03T18:35:13.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.021+0000 7f32a1e00700 1 --2- 172.21.15.33:0/3425484811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6661e6550 0x55a6661f1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:35:13.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.022+0000 7f32a2800700 1 --2- 172.21.15.33:0/3425484811 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a6661ec250 0x55a6661ef740 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f329c016ff0 tx=0x7f329c00b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57b50b15ff02d6ec server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:35:13.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.022+0000 7f3283e00700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-03T18:35:13.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.024+0000 7f3283e00700 1 -- 172.21.15.33:0/3425484811 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329c015640 con 0x55a6661ec250 2024-05-03T18:35:13.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.024+0000 7f3283e00700 1 -- 172.21.15.33:0/3425484811 <== mon.1 v2:172.21.15.150:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9518+0+0 (secure 0 0 0) 0x7f329c003a00 con 0x55a6661ec250 2024-05-03T18:35:13.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.024+0000 7f32b33f5b80 1 -- 172.21.15.33:0/3425484811 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a6661b2ff0 con 0x55a6661ec250 2024-05-03T18:35:13.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.025+0000 7f3283e00700 1 -- 172.21.15.33:0/3425484811 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f329c03d050 con 0x55a6661ec250 2024-05-03T18:35:13.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.025+0000 7f32b33f5b80 10 client.25897 fetch_fsmap learned FSMap version 21 2024-05-03T18:35:13.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.025+0000 7f32b33f5b80 1 -- 172.21.15.33:0/3425484811 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55a6661fd510 con 0x55a6661ec250 2024-05-03T18:35:13.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.025+0000 7f3283e00700 1 -- 172.21.15.33:0/3425484811 <== mon.1 v2:172.21.15.150:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55a6661b2ff0 con 0x55a6661ec250 2024-05-03T18:35:13.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.026+0000 7f3283e00700 1 -- 172.21.15.33:0/3425484811 <== mon.1 v2:172.21.15.150:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f329c028040 con 0x55a6661ec250 2024-05-03T18:35:13.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.026+0000 7f32b33f5b80 10 client.25897 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:35:13.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.026+0000 7f32b33f5b80 10 client.25897 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:35:13.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.026+0000 7f32b33f5b80 20 client.25897 populate_metadata read hostname 'smithi033' 2024-05-03T18:35:13.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.026+0000 7f32b33f5b80 1 --2- 172.21.15.33:0/3425484811 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55a6661ffa40 0x55a666201e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:13.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.026+0000 7f32b33f5b80 4 client.25897 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:35:13.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.026+0000 7f32b33f5b80 1 -- 172.21.15.33:0/3425484811 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a666146400 con 0x55a6661ffa40 2024-05-03T18:35:13.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.027+0000 7f32a1e00700 1 --2- 172.21.15.33:0/3425484811 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55a6661ffa40 0x55a666201e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:13.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.028+0000 7f32a1e00700 1 --2- 172.21.15.33:0/3425484811 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55a6661ffa40 0x55a666201e30 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2953ce0dd4d2e5a server_cookie=42ade53c42934256 in_seq=0 out_seq=0 2024-05-03T18:35:13.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.029+0000 7f3283e00700 10 client.25897 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:35:13.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.030+0000 7f3283e00700 1 -- 172.21.15.33:0/3425484811 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a666146400 con 0x55a6661ffa40 2024-05-03T18:35:13.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.030+0000 7f3283e00700 10 client.25897 handle_command_reply: tid=0 2024-05-03T18:35:13.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.030+0000 7f32b33f5b80 1 client.25897 shutdown 2024-05-03T18:35:13.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.031+0000 7f32b33f5b80 1 -- 172.21.15.33:0/3425484811 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a6661ec250 msgr2=0x55a6661ef740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:35:13.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.031+0000 7f32b33f5b80 1 --2- 172.21.15.33:0/3425484811 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a6661ec250 0x55a6661ef740 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f329c016ff0 tx=0x7f329c00b040 comp rx=0 tx=0).stop 2024-05-03T18:35:13.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.032+0000 7f32b33f5b80 1 -- 172.21.15.33:0/3425484811 shutdown_connections 2024-05-03T18:35:13.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.032+0000 7f32b33f5b80 1 --2- 172.21.15.33:0/3425484811 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55a6661ffa40 0x55a666201e30 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:13.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.032+0000 7f32b33f5b80 1 --2- 172.21.15.33:0/3425484811 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a6661ec250 0x55a6661ef740 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:13.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.032+0000 7f3283e00700 0 client.25897 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:35:13.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.032+0000 7f32b33f5b80 1 --2- 172.21.15.33:0/3425484811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6661e6550 0x55a6661f1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:13.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.032+0000 7f32b33f5b80 1 --2- 172.21.15.33:0/3425484811 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6661eb930 0x55a6661ef200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:13.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.032+0000 7f32b33f5b80 1 -- 172.21.15.33:0/3425484811 >> 172.21.15.33:0/3425484811 conn(0x55a66615e6d0 msgr2=0x55a6661db3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:35:13.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.033+0000 7f32b33f5b80 1 -- 172.21.15.33:0/3425484811 shutdown_connections 2024-05-03T18:35:13.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.033+0000 7f32b33f5b80 1 -- 172.21.15.33:0/3425484811 wait complete. 2024-05-03T18:35:13.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.033+0000 7f32b33f5b80 20 client.25897 trim_cache size 0 max 16384 2024-05-03T18:35:13.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.065+0000 7f32b33f5b80 1 Processor -- start 2024-05-03T18:35:13.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.065+0000 7f32b33f5b80 1 -- start start 2024-05-03T18:35:13.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.066+0000 7f32b33f5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6661e6550 0x55a6661efd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:13.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.066+0000 7f32b33f5b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a6661eb930 0x55a666207490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:13.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.066+0000 7f32b33f5b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6661ec250 0x55a6662079d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:13.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.066+0000 7f32b33f5b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f32a41bc3d0 con 0x55a6661ec250 2024-05-03T18:35:13.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.066+0000 7f32b33f5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f32a41bec30 con 0x55a6661e6550 2024-05-03T18:35:13.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.066+0000 7f32b33f5b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f32a41bca20 con 0x55a6661eb930 2024-05-03T18:35:13.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.067+0000 7f32a1400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a6661eb930 0x55a666207490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:13.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.067+0000 7f32a2800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6661ec250 0x55a6662079d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:13.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.067+0000 7f32a1400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a6661eb930 0x55a666207490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:36784/0 (socket says 172.21.15.33:36784) 2024-05-03T18:35:13.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.067+0000 7f32a1400700 1 -- 172.21.15.33:0/3771444225 learned_addr learned my addr 172.21.15.33:0/3771444225 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:35:13.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.067+0000 7f32a1e00700 1 --2- 172.21.15.33:0/3771444225 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6661e6550 0x55a6661efd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:13.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.068+0000 7f32a2800700 1 -- 172.21.15.33:0/3771444225 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a6661eb930 msgr2=0x55a666207490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:35:13.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.068+0000 7f32a2800700 1 --2- 172.21.15.33:0/3771444225 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a6661eb930 0x55a666207490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:13.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.068+0000 7f32a2800700 1 -- 172.21.15.33:0/3771444225 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6661e6550 msgr2=0x55a6661efd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:35:13.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.068+0000 7f32a2800700 1 --2- 172.21.15.33:0/3771444225 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6661e6550 0x55a6661efd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:13.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.068+0000 7f32a2800700 1 -- 172.21.15.33:0/3771444225 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6661e6360 con 0x55a6661ec250 2024-05-03T18:35:13.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.068+0000 7f32a2800700 1 --2- 172.21.15.33:0/3771444225 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6661ec250 0x55a6662079d0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f329c007ac0 tx=0x7f329c009ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6a7678e5664512e server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:35:13.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.069+0000 7f328ea00700 1 -- 172.21.15.33:0/3771444225 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329c0370e0 con 0x55a6661ec250 2024-05-03T18:35:13.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.069+0000 7f328ea00700 1 -- 172.21.15.33:0/3771444225 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f329c046070 con 0x55a6661ec250 2024-05-03T18:35:13.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.069+0000 7f328ea00700 1 -- 172.21.15.33:0/3771444225 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329c02b450 con 0x55a6661ec250 2024-05-03T18:35:13.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.070+0000 7f32b33f5b80 1 -- 172.21.15.33:0/3771444225 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6661ec250 msgr2=0x55a6662079d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:35:13.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.070+0000 7f32b33f5b80 1 --2- 172.21.15.33:0/3771444225 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6661ec250 0x55a6662079d0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f329c007ac0 tx=0x7f329c009ab0 comp rx=0 tx=0).stop 2024-05-03T18:35:13.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.070+0000 7f32b33f5b80 1 -- 172.21.15.33:0/3771444225 shutdown_connections 2024-05-03T18:35:13.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.070+0000 7f32b33f5b80 1 --2- 172.21.15.33:0/3771444225 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6661ec250 0x55a6662079d0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:13.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.071+0000 7f32b33f5b80 1 --2- 172.21.15.33:0/3771444225 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6661e6550 0x55a6661efd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:13.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.071+0000 7f32b33f5b80 1 --2- 172.21.15.33:0/3771444225 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a6661eb930 0x55a666207490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:13.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.071+0000 7f32b33f5b80 1 -- 172.21.15.33:0/3771444225 >> 172.21.15.33:0/3771444225 conn(0x55a66615e6d0 msgr2=0x55a6661ef1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:35:13.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.072+0000 7f32b33f5b80 1 -- 172.21.15.33:0/3771444225 shutdown_connections 2024-05-03T18:35:13.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.073+0000 7f32b33f5b80 1 -- 172.21.15.33:0/3771444225 wait complete. 2024-05-03T18:35:13.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.074+0000 7f32b33f5b80 1 -- start start 2024-05-03T18:35:13.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.074+0000 7f32b33f5b80 1 Processor -- start 2024-05-03T18:35:13.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.076+0000 7f32b33f5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6661e6550 0x55a666207b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:13.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.076+0000 7f32b33f5b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a6661eb930 0x55a6661f81b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:13.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.077+0000 7f32b33f5b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6661ec250 0x55a666206480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:13.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.077+0000 7f32b33f5b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f32a41bd0f0 con 0x55a6661ec250 2024-05-03T18:35:13.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.077+0000 7f32b33f5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f32a41be560 con 0x55a6661e6550 2024-05-03T18:35:13.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.077+0000 7f32b33f5b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f32a41bd7c0 con 0x55a6661eb930 2024-05-03T18:35:13.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.077+0000 7f32b33f5b80 10 client.0 _ll_register_callbacks cb 0x55a6660dfc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:35:13.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.077+0000 7f32a1e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6661e6550 0x55a666207b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:13.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.077+0000 7f32a1e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6661e6550 0x55a666207b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:46900/0 (socket says 172.21.15.33:46900) 2024-05-03T18:35:13.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.077+0000 7f32a1e00700 1 -- 172.21.15.33:0/3612147460 learned_addr learned my addr 172.21.15.33:0/3612147460 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:35:13.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.077+0000 7f32a1400700 1 --2- 172.21.15.33:0/3612147460 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a6661eb930 0x55a6661f81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:13.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.078+0000 7f32a2800700 1 --2- 172.21.15.33:0/3612147460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6661ec250 0x55a666206480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:13.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.078+0000 7f32a1e00700 1 -- 172.21.15.33:0/3612147460 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a6661eb930 msgr2=0x55a6661f81b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:35:13.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.078+0000 7f32a1e00700 1 --2- 172.21.15.33:0/3612147460 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a6661eb930 0x55a6661f81b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:13.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.078+0000 7f32a1e00700 1 -- 172.21.15.33:0/3612147460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6661ec250 msgr2=0x55a666206480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:35:13.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.078+0000 7f32a1e00700 1 --2- 172.21.15.33:0/3612147460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6661ec250 0x55a666206480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:13.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.078+0000 7f32a1e00700 1 -- 172.21.15.33:0/3612147460 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6661f8bb0 con 0x55a6661e6550 2024-05-03T18:35:13.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.078+0000 7f32a2800700 1 --2- 172.21.15.33:0/3612147460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6661ec250 0x55a666206480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:35:13.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.078+0000 7f32a1e00700 1 --2- 172.21.15.33:0/3612147460 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6661e6550 0x55a666207b40 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f32940e0a70 tx=0x7f3294009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b40a871f6e46b27 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:35:13.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.079+0000 7f3283e00700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-03T18:35:13.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.079+0000 7f3283e00700 1 -- 172.21.15.33:0/3612147460 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32940c3030 con 0x55a6661e6550 2024-05-03T18:35:13.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.079+0000 7f3283e00700 1 -- 172.21.15.33:0/3612147460 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9518+0+0 (secure 0 0 0) 0x7f32940a6c20 con 0x55a6661e6550 2024-05-03T18:35:13.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.079+0000 7f32b33f5b80 1 -- 172.21.15.33:0/3612147460 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f329c046070 con 0x55a6661e6550 2024-05-03T18:35:13.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.080+0000 7f3283e00700 1 -- 172.21.15.33:0/3612147460 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f32940e3040 con 0x55a6661e6550 2024-05-03T18:35:13.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.081+0000 7f32b33f5b80 10 client.35852 fetch_fsmap learned FSMap version 21 2024-05-03T18:35:13.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.081+0000 7f32b33f5b80 1 -- 172.21.15.33:0/3612147460 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55a6662695b0 con 0x55a6661e6550 2024-05-03T18:35:13.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.081+0000 7f3283e00700 1 -- 172.21.15.33:0/3612147460 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f329c046070 con 0x55a6661e6550 2024-05-03T18:35:13.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.082+0000 7f3283e00700 1 -- 172.21.15.33:0/3612147460 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f32940aa890 con 0x55a6661e6550 2024-05-03T18:35:13.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.082+0000 7f32b33f5b80 10 client.35852 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:35:13.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.082+0000 7f32b33f5b80 10 client.35852 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:35:13.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.082+0000 7f32b33f5b80 20 client.35852 populate_metadata read hostname 'smithi033' 2024-05-03T18:35:13.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.082+0000 7f32b33f5b80 1 --2- 172.21.15.33:0/3612147460 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55a6661ffa40 0x55a66626bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:13.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.082+0000 7f32b33f5b80 4 client.35852 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-03T18:35:13.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.082+0000 7f32b33f5b80 1 -- 172.21.15.33:0/3612147460 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a6661404a0 con 0x55a6661ffa40 2024-05-03T18:35:13.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.083+0000 7f32a1400700 1 --2- 172.21.15.33:0/3612147460 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55a6661ffa40 0x55a66626bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:13.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.084+0000 7f3283e00700 10 client.35852 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:35:13.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.084+0000 7f32a1400700 1 --2- 172.21.15.33:0/3612147460 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55a6661ffa40 0x55a66626bc50 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3ea414e41247fdf server_cookie=6792036c27f6f3b3 in_seq=0 out_seq=0 2024-05-03T18:35:13.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.085+0000 7f3283e00700 1 -- 172.21.15.33:0/3612147460 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a6661404a0 con 0x55a6661ffa40 2024-05-03T18:35:13.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.085+0000 7f3283e00700 10 client.35852 handle_command_reply: tid=0 2024-05-03T18:35:13.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.085+0000 7f32b33f5b80 1 client.35852 shutdown 2024-05-03T18:35:13.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.085+0000 7f32b33f5b80 1 -- 172.21.15.33:0/3612147460 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6661e6550 msgr2=0x55a666207b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:35:13.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.085+0000 7f32b33f5b80 1 --2- 172.21.15.33:0/3612147460 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6661e6550 0x55a666207b40 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f32940e0a70 tx=0x7f3294009310 comp rx=0 tx=0).stop 2024-05-03T18:35:13.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.085+0000 7f32b33f5b80 1 -- 172.21.15.33:0/3612147460 shutdown_connections 2024-05-03T18:35:13.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.085+0000 7f32b33f5b80 1 --2- 172.21.15.33:0/3612147460 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55a6661ffa40 0x55a66626bc50 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:13.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.085+0000 7f32b33f5b80 1 --2- 172.21.15.33:0/3612147460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6661ec250 0x55a666206480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:13.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.085+0000 7f3283e00700 0 client.35852 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:35:13.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.085+0000 7f32b33f5b80 1 --2- 172.21.15.33:0/3612147460 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6661e6550 0x55a666207b40 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:13.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.085+0000 7f32b33f5b80 1 --2- 172.21.15.33:0/3612147460 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a6661eb930 0x55a6661f81b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:13.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.085+0000 7f32b33f5b80 1 -- 172.21.15.33:0/3612147460 >> 172.21.15.33:0/3612147460 conn(0x55a66615e6d0 msgr2=0x55a6661ec630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:35:13.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.087+0000 7f32b33f5b80 1 -- 172.21.15.33:0/3612147460 shutdown_connections 2024-05-03T18:35:13.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.087+0000 7f32b33f5b80 1 -- 172.21.15.33:0/3612147460 wait complete. 2024-05-03T18:35:13.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.087+0000 7f32b33f5b80 20 client.35852 trim_cache size 0 max 16384 2024-05-03T18:35:13.088 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-03T18:35:13.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.088+0000 7f32a8a00700 1 -- 172.21.15.33:0/2505469964 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f32880b2ca0 msgr2=0x7f32880b5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:35:13.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.088+0000 7f32a8a00700 1 --2- 172.21.15.33:0/2505469964 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f32880b2ca0 0x7f32880b5160 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f329c009440 tx=0x7f329c015bd0 comp rx=0 tx=0).stop 2024-05-03T18:35:13.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.088+0000 7f32a8a00700 1 -- 172.21.15.33:0/2505469964 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32a4101880 msgr2=0x7f32a4105260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:35:13.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.088+0000 7f32a8a00700 1 --2- 172.21.15.33:0/2505469964 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32a4101880 0x7f32a4105260 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f3294000c00 tx=0x7f32940047b0 comp rx=0 tx=0).stop 2024-05-03T18:35:13.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.088+0000 7f32a8a00700 1 -- 172.21.15.33:0/2505469964 shutdown_connections 2024-05-03T18:35:13.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.088+0000 7f32a8a00700 1 --2- 172.21.15.33:0/2505469964 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f32880b2ca0 0x7f32880b5160 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:13.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.088+0000 7f32a8a00700 1 --2- 172.21.15.33:0/2505469964 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32a410fc90 0x7f32a4105ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:13.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.088+0000 7f32a8a00700 1 --2- 172.21.15.33:0/2505469964 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32a4101880 0x7f32a4105260 secure :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f3294000c00 tx=0x7f32940047b0 comp rx=0 tx=0).stop 2024-05-03T18:35:13.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.088+0000 7f32a8a00700 1 --2- 172.21.15.33:0/2505469964 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32a41021a0 0x7f32a41057a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:13.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.088+0000 7f32a8a00700 1 -- 172.21.15.33:0/2505469964 >> 172.21.15.33:0/2505469964 conn(0x7f32a40785a0 msgr2=0x7f32a40ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:35:13.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.089+0000 7f32a8a00700 1 -- 172.21.15.33:0/2505469964 shutdown_connections 2024-05-03T18:35:13.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:13.089+0000 7f32a8a00700 1 -- 172.21.15.33:0/2505469964 wait complete. 2024-05-03T18:35:14.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:35:14 smithi150 ceph-mon[32492]: pgmap v248: 129 pgs: 129 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 6.8 MiB/s wr, 608 op/s 2024-05-03T18:35:14.486 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:35:14 smithi033 ceph-mon[28330]: pgmap v248: 129 pgs: 129 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 6.8 MiB/s wr, 608 op/s 2024-05-03T18:35:14.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:35:14 smithi099 ceph-mon[31658]: pgmap v248: 129 pgs: 129 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 6.8 MiB/s wr, 608 op/s 2024-05-03T18:35:15.110 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 240.0 s, 634.0 tps, lat 157.130 ms stddev 46.387 2024-05-03T18:35:16.063 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-03T18:35:16.063 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-03T18:35:16.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:35:16 smithi150 ceph-mon[32492]: pgmap v249: 129 pgs: 129 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 6.8 MiB/s wr, 608 op/s 2024-05-03T18:35:16.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:35:16 smithi033 ceph-mon[28330]: pgmap v249: 129 pgs: 129 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 6.8 MiB/s wr, 608 op/s 2024-05-03T18:35:16.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:35:16 smithi099 ceph-mon[31658]: pgmap v249: 129 pgs: 129 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 6.8 MiB/s wr, 608 op/s 2024-05-03T18:35:16.730 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:35:18.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:35:18 smithi033 ceph-mon[28330]: pgmap v250: 129 pgs: 129 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 6.4 MiB/s wr, 569 op/s 2024-05-03T18:35:18.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:35:18 smithi150 ceph-mon[32492]: pgmap v250: 129 pgs: 129 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 6.4 MiB/s wr, 569 op/s 2024-05-03T18:35:18.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:35:18 smithi099 ceph-mon[31658]: pgmap v250: 129 pgs: 129 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 6.4 MiB/s wr, 569 op/s 2024-05-03T18:35:20.110 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 245.0 s, 637.4 tps, lat 157.183 ms stddev 69.333 2024-05-03T18:35:20.348 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:35:20 smithi099 ceph-mon[31658]: pgmap v251: 129 pgs: 129 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 8.5 MiB/s wr, 774 op/s 2024-05-03T18:35:20.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:35:20 smithi150 ceph-mon[32492]: pgmap v251: 129 pgs: 129 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 8.5 MiB/s wr, 774 op/s 2024-05-03T18:35:20.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:35:20 smithi033 ceph-mon[28330]: pgmap v251: 129 pgs: 129 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 8.5 MiB/s wr, 774 op/s 2024-05-03T18:35:21.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.355+0000 7f43ef400700 1 -- 172.21.15.33:0/101750423 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43e81083d0 msgr2=0x7f43e806b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:35:21.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.355+0000 7f43ef400700 1 --2- 172.21.15.33:0/101750423 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43e81083d0 0x7f43e806b1b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f43d80098f0 tx=0x7f43d8009c00 comp rx=0 tx=0).stop 2024-05-03T18:35:21.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.355+0000 7f43ef400700 1 -- 172.21.15.33:0/101750423 shutdown_connections 2024-05-03T18:35:21.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.355+0000 7f43ef400700 1 --2- 172.21.15.33:0/101750423 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43e8105ee0 0x7f43e8106370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:21.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.355+0000 7f43ef400700 1 --2- 172.21.15.33:0/101750423 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43e81083d0 0x7f43e806b1b0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:21.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.355+0000 7f43ef400700 1 --2- 172.21.15.33:0/101750423 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43e806b6f0 0x7f43e806bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:21.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.355+0000 7f43ef400700 1 -- 172.21.15.33:0/101750423 >> 172.21.15.33:0/101750423 conn(0x7f43e8079350 msgr2=0x7f43e8079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:35:21.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.356+0000 7f43ef400700 1 -- 172.21.15.33:0/101750423 shutdown_connections 2024-05-03T18:35:21.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.356+0000 7f43ef400700 1 -- 172.21.15.33:0/101750423 wait complete. 2024-05-03T18:35:21.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.358+0000 7f43ef400700 1 Processor -- start 2024-05-03T18:35:21.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.358+0000 7f43ef400700 1 -- start start 2024-05-03T18:35:21.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.359+0000 7f43ef400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43e806b6f0 0x7f43e81a01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:21.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.359+0000 7f43ef400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43e8105ee0 0x7f43e81a06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:21.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.359+0000 7f43ef400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43e81083d0 0x7f43e81a4a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:21.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.359+0000 7f43ef400700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f43e8119ae0 con 0x7f43e81083d0 2024-05-03T18:35:21.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.359+0000 7f43ef400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f43e8119960 con 0x7f43e8105ee0 2024-05-03T18:35:21.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.359+0000 7f43ef400700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f43e8119c60 con 0x7f43e806b6f0 2024-05-03T18:35:21.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.360+0000 7f43e7e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43e8105ee0 0x7f43e81a06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:21.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.360+0000 7f43e7e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43e8105ee0 0x7f43e81a06e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:40430/0 (socket says 172.21.15.33:40430) 2024-05-03T18:35:21.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.360+0000 7f43ed400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43e81083d0 0x7f43e81a4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:21.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.360+0000 7f43e7e00700 1 -- 172.21.15.33:0/2369519418 learned_addr learned my addr 172.21.15.33:0/2369519418 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:35:21.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.361+0000 7f43eca00700 1 --2- 172.21.15.33:0/2369519418 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43e806b6f0 0x7f43e81a01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:21.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.362+0000 7f43ed400700 1 -- 172.21.15.33:0/2369519418 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43e806b6f0 msgr2=0x7f43e81a01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:35:21.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.362+0000 7f43ed400700 1 --2- 172.21.15.33:0/2369519418 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43e806b6f0 0x7f43e81a01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:21.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.362+0000 7f43ed400700 1 -- 172.21.15.33:0/2369519418 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43e8105ee0 msgr2=0x7f43e81a06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:35:21.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.362+0000 7f43ed400700 1 --2- 172.21.15.33:0/2369519418 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43e8105ee0 0x7f43e81a06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:21.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.362+0000 7f43ed400700 1 -- 172.21.15.33:0/2369519418 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43e81a51b0 con 0x7f43e81083d0 2024-05-03T18:35:21.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.362+0000 7f43eca00700 1 --2- 172.21.15.33:0/2369519418 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43e806b6f0 0x7f43e81a01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:35:21.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.363+0000 7f43ed400700 1 --2- 172.21.15.33:0/2369519418 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43e81083d0 0x7f43e81a4a70 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f43e000efc0 tx=0x7f43e000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:35:21.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.364+0000 7f43e5600700 1 -- 172.21.15.33:0/2369519418 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43e00092e0 con 0x7f43e81083d0 2024-05-03T18:35:21.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.364+0000 7f43e5600700 1 -- 172.21.15.33:0/2369519418 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f43e0004bb0 con 0x7f43e81083d0 2024-05-03T18:35:21.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.364+0000 7f43ef400700 1 -- 172.21.15.33:0/2369519418 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43e81a5440 con 0x7f43e81083d0 2024-05-03T18:35:21.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.365+0000 7f43ef400700 1 -- 172.21.15.33:0/2369519418 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43e81acd20 con 0x7f43e81083d0 2024-05-03T18:35:21.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.364+0000 7f43e5600700 1 -- 172.21.15.33:0/2369519418 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43e0004020 con 0x7f43e81083d0 2024-05-03T18:35:21.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.372+0000 7f43f9cfeb80 1 Processor -- start 2024-05-03T18:35:21.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.372+0000 7f43e5600700 1 -- 172.21.15.33:0/2369519418 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f43e001b030 con 0x7f43e81083d0 2024-05-03T18:35:21.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.373+0000 7f43f9cfeb80 1 -- start start 2024-05-03T18:35:21.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.374+0000 7f43f9cfeb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f7b1939830 0x55f7b1939c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:21.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.374+0000 7f43f9cfeb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f7b193a150 0x55f7b1935030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:21.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.374+0000 7f43f9cfeb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f7b1935650 0x55f7b1935ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:21.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.374+0000 7f43f9cfeb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55f7b1887aa0 con 0x55f7b1935650 2024-05-03T18:35:21.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.374+0000 7f43f9cfeb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55f7b18257e0 con 0x55f7b1939830 2024-05-03T18:35:21.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.374+0000 7f43f9cfeb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55f7b1883a60 con 0x55f7b193a150 2024-05-03T18:35:21.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.375+0000 7f43e7e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f7b1939830 0x55f7b1939c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:21.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.375+0000 7f43e7e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f7b1939830 0x55f7b1939c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:40434/0 (socket says 172.21.15.33:40434) 2024-05-03T18:35:21.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.375+0000 7f43e7e00700 1 -- 172.21.15.33:0/3515233504 learned_addr learned my addr 172.21.15.33:0/3515233504 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:35:21.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.375+0000 7f43e7e00700 1 --2- 172.21.15.33:0/3515233504 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f7b1935650 0x55f7b1935ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:21.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.376+0000 7f43e7e00700 1 -- 172.21.15.33:0/3515233504 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f7b193a150 msgr2=0x55f7b1935030 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-03T18:35:21.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.376+0000 7f43e7e00700 1 --2- 172.21.15.33:0/3515233504 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f7b193a150 0x55f7b1935030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:21.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.376+0000 7f43e7e00700 1 -- 172.21.15.33:0/3515233504 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f7b1935650 msgr2=0x55f7b1935ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:35:21.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.376+0000 7f43e7e00700 1 --2- 172.21.15.33:0/3515233504 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f7b1935650 0x55f7b1935ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:21.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.376+0000 7f43e7e00700 1 -- 172.21.15.33:0/3515233504 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7b17d4950 con 0x55f7b1939830 2024-05-03T18:35:21.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.377+0000 7f43e5600700 1 --2- 172.21.15.33:0/2369519418 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f43d00b2bc0 0x7f43d00b5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:21.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.377+0000 7f43e7e00700 1 --2- 172.21.15.33:0/3515233504 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f7b1939830 0x55f7b1939c10 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f43dc0046e0 tx=0x7f43dc0096e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=994a86f5f29d4de4 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:35:21.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.378+0000 7f43d6000700 1 -- 172.21.15.33:0/3515233504 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43dc01f440 con 0x55f7b1939830 2024-05-03T18:35:21.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.378+0000 7f43eca00700 1 --2- 172.21.15.33:0/2369519418 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f43d00b2bc0 0x7f43d00b5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:21.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.378+0000 7f43d6000700 1 -- 172.21.15.33:0/3515233504 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f43dc00f910 con 0x55f7b1939830 2024-05-03T18:35:21.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.378+0000 7f43d6000700 1 -- 172.21.15.33:0/3515233504 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43dc00e6d0 con 0x55f7b1939830 2024-05-03T18:35:21.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.378+0000 7f43e5600700 1 -- 172.21.15.33:0/2369519418 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9518+0+0 (secure 0 0 0) 0x7f43e00e2430 con 0x7f43e81083d0 2024-05-03T18:35:21.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.379+0000 7f43f9cfeb80 1 -- 172.21.15.33:0/3515233504 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f7b1939830 msgr2=0x55f7b1939c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:35:21.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.379+0000 7f43f9cfeb80 1 --2- 172.21.15.33:0/3515233504 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f7b1939830 0x55f7b1939c10 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f43dc0046e0 tx=0x7f43dc0096e0 comp rx=0 tx=0).stop 2024-05-03T18:35:21.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.380+0000 7f43f9cfeb80 1 -- 172.21.15.33:0/3515233504 shutdown_connections 2024-05-03T18:35:21.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.380+0000 7f43f9cfeb80 1 --2- 172.21.15.33:0/3515233504 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f7b1935650 0x55f7b1935ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:21.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.380+0000 7f43f9cfeb80 1 --2- 172.21.15.33:0/3515233504 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f7b1939830 0x55f7b1939c10 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:21.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.380+0000 7f43f9cfeb80 1 --2- 172.21.15.33:0/3515233504 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f7b193a150 0x55f7b1935030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:21.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.380+0000 7f43f9cfeb80 1 -- 172.21.15.33:0/3515233504 >> 172.21.15.33:0/3515233504 conn(0x55f7b1863be0 msgr2=0x55f7b192f870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:35:21.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.382+0000 7f43f9cfeb80 1 -- 172.21.15.33:0/3515233504 shutdown_connections 2024-05-03T18:35:21.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.382+0000 7f43eca00700 1 --2- 172.21.15.33:0/2369519418 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f43d00b2bc0 0x7f43d00b5080 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f43d8009fd0 tx=0x7f43d8019040 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:35:21.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.382+0000 7f43f9cfeb80 1 -- 172.21.15.33:0/3515233504 wait complete. 2024-05-03T18:35:21.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.383+0000 7f43f9cfeb80 1 -- start start 2024-05-03T18:35:21.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.383+0000 7f43f9cfeb80 1 Processor -- start 2024-05-03T18:35:21.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.388+0000 7f43f9cfeb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f7b1935650 0x55f7b1940ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:21.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.388+0000 7f43f9cfeb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f7b1939830 0x55f7b193e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:21.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.389+0000 7f43f9cfeb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f7b193a150 0x55f7b193e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:21.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.389+0000 7f43eca00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f7b1935650 0x55f7b1940ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:21.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.389+0000 7f43eca00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f7b1935650 0x55f7b1940ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:40446/0 (socket says 172.21.15.33:40446) 2024-05-03T18:35:21.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.389+0000 7f43f9cfeb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55f7b1941d20 con 0x55f7b193a150 2024-05-03T18:35:21.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.389+0000 7f43f9cfeb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55f7b17edba0 con 0x55f7b1935650 2024-05-03T18:35:21.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.389+0000 7f43eca00700 1 -- 172.21.15.33:0/196405361 learned_addr learned my addr 172.21.15.33:0/196405361 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:35:21.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.389+0000 7f43e7e00700 1 --2- 172.21.15.33:0/196405361 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f7b1939830 0x55f7b193e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:21.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.389+0000 7f43f9cfeb80 1 -- 172.21.15.33:0/196405361 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55f7b1941ea0 con 0x55f7b1939830 2024-05-03T18:35:21.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.389+0000 7f43f9cfeb80 10 client.0 _ll_register_callbacks cb 0x55f7b182ecf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:35:21.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.390+0000 7f43ed400700 1 --2- 172.21.15.33:0/196405361 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f7b193a150 0x55f7b193e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:21.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.390+0000 7f43eca00700 1 -- 172.21.15.33:0/196405361 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f7b1939830 msgr2=0x55f7b193e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:35:21.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.390+0000 7f43eca00700 1 --2- 172.21.15.33:0/196405361 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f7b1939830 0x55f7b193e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:21.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.390+0000 7f43eca00700 1 -- 172.21.15.33:0/196405361 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f7b193a150 msgr2=0x55f7b193e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:35:21.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.390+0000 7f43eca00700 1 --2- 172.21.15.33:0/196405361 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f7b193a150 0x55f7b193e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:21.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.390+0000 7f43eca00700 1 -- 172.21.15.33:0/196405361 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7b184d4a0 con 0x55f7b1935650 2024-05-03T18:35:21.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.391+0000 7f43eca00700 1 --2- 172.21.15.33:0/196405361 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f7b1935650 0x55f7b1940ff0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f43d8016000 tx=0x7f43d8009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8dad9da0cc168e server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:35:21.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.391+0000 7f43caa00700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-03T18:35:21.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.392+0000 7f43caa00700 1 -- 172.21.15.33:0/196405361 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d8032060 con 0x55f7b1935650 2024-05-03T18:35:21.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.392+0000 7f43caa00700 1 -- 172.21.15.33:0/196405361 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9518+0+0 (secure 0 0 0) 0x7f43d8005250 con 0x55f7b1935650 2024-05-03T18:35:21.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.392+0000 7f43f9cfeb80 1 -- 172.21.15.33:0/196405361 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43dc00f910 con 0x55f7b1935650 2024-05-03T18:35:21.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.392+0000 7f43caa00700 1 -- 172.21.15.33:0/196405361 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f43d8039050 con 0x55f7b1935650 2024-05-03T18:35:21.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.392+0000 7f43f9cfeb80 10 client.35864 fetch_fsmap learned FSMap version 21 2024-05-03T18:35:21.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.392+0000 7f43f9cfeb80 1 -- 172.21.15.33:0/196405361 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55f7b194c630 con 0x55f7b1935650 2024-05-03T18:35:21.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.393+0000 7f43caa00700 1 -- 172.21.15.33:0/196405361 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f43dc00f910 con 0x55f7b1935650 2024-05-03T18:35:21.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.393+0000 7f43caa00700 1 -- 172.21.15.33:0/196405361 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f43d800b560 con 0x55f7b1935650 2024-05-03T18:35:21.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.393+0000 7f43f9cfeb80 10 client.35864 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:35:21.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.393+0000 7f43f9cfeb80 10 client.35864 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:35:21.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.394+0000 7f43f9cfeb80 20 client.35864 populate_metadata read hostname 'smithi033' 2024-05-03T18:35:21.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.394+0000 7f43f9cfeb80 1 --2- 172.21.15.33:0/196405361 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55f7b194ed60 0x55f7b19511e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:21.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.394+0000 7f43f9cfeb80 4 client.35864 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:35:21.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.394+0000 7f43f9cfeb80 1 -- 172.21.15.33:0/196405361 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f7b1902250 con 0x55f7b194ed60 2024-05-03T18:35:21.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.397+0000 7f43e7e00700 1 --2- 172.21.15.33:0/196405361 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55f7b194ed60 0x55f7b19511e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:21.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.398+0000 7f43e7e00700 1 --2- 172.21.15.33:0/196405361 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55f7b194ed60 0x55f7b19511e0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e936cf3275e27d8 server_cookie=d752c7d41a32aca3 in_seq=0 out_seq=0 2024-05-03T18:35:21.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.398+0000 7f43caa00700 10 client.35864 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:35:21.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.402+0000 7f43caa00700 1 -- 172.21.15.33:0/196405361 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f7b1902250 con 0x55f7b194ed60 2024-05-03T18:35:21.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.402+0000 7f43caa00700 10 client.35864 handle_command_reply: tid=0 2024-05-03T18:35:21.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.402+0000 7f43f9cfeb80 1 client.35864 shutdown 2024-05-03T18:35:21.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.403+0000 7f43f9cfeb80 1 -- 172.21.15.33:0/196405361 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f7b1935650 msgr2=0x55f7b1940ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:35:21.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.403+0000 7f43f9cfeb80 1 --2- 172.21.15.33:0/196405361 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f7b1935650 0x55f7b1940ff0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f43d8016000 tx=0x7f43d8009260 comp rx=0 tx=0).stop 2024-05-03T18:35:21.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.403+0000 7f43f9cfeb80 1 -- 172.21.15.33:0/196405361 shutdown_connections 2024-05-03T18:35:21.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.403+0000 7f43f9cfeb80 1 --2- 172.21.15.33:0/196405361 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55f7b194ed60 0x55f7b19511e0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:21.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.404+0000 7f43f9cfeb80 1 --2- 172.21.15.33:0/196405361 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f7b193a150 0x55f7b193e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:21.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.404+0000 7f43f9cfeb80 1 --2- 172.21.15.33:0/196405361 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f7b1935650 0x55f7b1940ff0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:21.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.404+0000 7f43f9cfeb80 1 --2- 172.21.15.33:0/196405361 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f7b1939830 0x55f7b193e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:21.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.404+0000 7f43f9cfeb80 1 -- 172.21.15.33:0/196405361 >> 172.21.15.33:0/196405361 conn(0x55f7b1863be0 msgr2=0x55f7b192e6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:35:21.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.404+0000 7f43caa00700 0 client.35864 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:35:21.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.404+0000 7f43f9cfeb80 1 -- 172.21.15.33:0/196405361 shutdown_connections 2024-05-03T18:35:21.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.404+0000 7f43f9cfeb80 1 -- 172.21.15.33:0/196405361 wait complete. 2024-05-03T18:35:21.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.405+0000 7f43f9cfeb80 20 client.35864 trim_cache size 0 max 16384 2024-05-03T18:35:21.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.454+0000 7f43f9cfeb80 1 Processor -- start 2024-05-03T18:35:21.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.454+0000 7f43f9cfeb80 1 -- start start 2024-05-03T18:35:21.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.455+0000 7f43f9cfeb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f7b1935650 0x55f7b194ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:21.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.455+0000 7f43f9cfeb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f7b1939830 0x55f7b193e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:21.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.455+0000 7f43f9cfeb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f7b193a150 0x55f7b1956af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:21.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.455+0000 7f43f9cfeb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f43e81b3ac0 con 0x55f7b193a150 2024-05-03T18:35:21.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.455+0000 7f43f9cfeb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f43e81b6320 con 0x55f7b1939830 2024-05-03T18:35:21.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.455+0000 7f43f9cfeb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f43e81b4110 con 0x55f7b1935650 2024-05-03T18:35:21.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.455+0000 7f43ed400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f7b193a150 0x55f7b1956af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:21.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.455+0000 7f43ed400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f7b193a150 0x55f7b1956af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40990/0 (socket says 172.21.15.33:40990) 2024-05-03T18:35:21.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.455+0000 7f43ed400700 1 -- 172.21.15.33:0/3579717193 learned_addr learned my addr 172.21.15.33:0/3579717193 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:35:21.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.455+0000 7f43e7e00700 1 --2- 172.21.15.33:0/3579717193 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f7b1939830 0x55f7b193e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:21.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.456+0000 7f43ed400700 1 -- 172.21.15.33:0/3579717193 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f7b1935650 msgr2=0x55f7b194ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:35:21.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.456+0000 7f43eca00700 1 --2- 172.21.15.33:0/3579717193 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f7b1935650 0x55f7b194ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:21.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.456+0000 7f43ed400700 1 --2- 172.21.15.33:0/3579717193 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f7b1935650 0x55f7b194ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:21.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.456+0000 7f43ed400700 1 -- 172.21.15.33:0/3579717193 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f7b1939830 msgr2=0x55f7b193e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:35:21.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.456+0000 7f43ed400700 1 --2- 172.21.15.33:0/3579717193 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f7b1939830 0x55f7b193e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:21.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.456+0000 7f43ed400700 1 -- 172.21.15.33:0/3579717193 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7b1941360 con 0x55f7b193a150 2024-05-03T18:35:21.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.456+0000 7f43ed400700 1 --2- 172.21.15.33:0/3579717193 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f7b193a150 0x55f7b1956af0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f43e00a9060 tx=0x7f43e00a5220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dd1e995c9360a12 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:35:21.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.457+0000 7f43d5600700 1 -- 172.21.15.33:0/3579717193 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43e00dfbd0 con 0x55f7b193a150 2024-05-03T18:35:21.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.457+0000 7f43d5600700 1 -- 172.21.15.33:0/3579717193 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f43e00c3070 con 0x55f7b193a150 2024-05-03T18:35:21.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.457+0000 7f43d5600700 1 -- 172.21.15.33:0/3579717193 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43e00b1ca0 con 0x55f7b193a150 2024-05-03T18:35:21.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.458+0000 7f43f9cfeb80 1 -- 172.21.15.33:0/3579717193 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f7b193a150 msgr2=0x55f7b1956af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:35:21.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.458+0000 7f43f9cfeb80 1 --2- 172.21.15.33:0/3579717193 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f7b193a150 0x55f7b1956af0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f43e00a9060 tx=0x7f43e00a5220 comp rx=0 tx=0).stop 2024-05-03T18:35:21.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.458+0000 7f43f9cfeb80 1 -- 172.21.15.33:0/3579717193 shutdown_connections 2024-05-03T18:35:21.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.458+0000 7f43f9cfeb80 1 --2- 172.21.15.33:0/3579717193 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f7b193a150 0x55f7b1956af0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:21.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.458+0000 7f43f9cfeb80 1 --2- 172.21.15.33:0/3579717193 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f7b1935650 0x55f7b194ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:21.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.458+0000 7f43f9cfeb80 1 --2- 172.21.15.33:0/3579717193 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f7b1939830 0x55f7b193e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:21.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.458+0000 7f43f9cfeb80 1 -- 172.21.15.33:0/3579717193 >> 172.21.15.33:0/3579717193 conn(0x55f7b1863be0 msgr2=0x55f7b192ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:35:21.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.458+0000 7f43f9cfeb80 1 -- 172.21.15.33:0/3579717193 shutdown_connections 2024-05-03T18:35:21.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.459+0000 7f43f9cfeb80 1 -- 172.21.15.33:0/3579717193 wait complete. 2024-05-03T18:35:21.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.459+0000 7f43f9cfeb80 1 -- start start 2024-05-03T18:35:21.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.460+0000 7f43f9cfeb80 1 Processor -- start 2024-05-03T18:35:21.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.461+0000 7f43f9cfeb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f7b1935650 0x55f7b19473d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:21.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.461+0000 7f43f9cfeb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f7b1939830 0x55f7b1945780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:21.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.461+0000 7f43f9cfeb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f7b193a150 0x55f7b1945cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:21.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.462+0000 7f43eca00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f7b1935650 0x55f7b19473d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:21.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.462+0000 7f43e7e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f7b1939830 0x55f7b1945780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:21.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.463+0000 7f43ed400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f7b193a150 0x55f7b1945cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:21.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.463+0000 7f43e7e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f7b1939830 0x55f7b1945780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:40460/0 (socket says 172.21.15.33:40460) 2024-05-03T18:35:21.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.463+0000 7f43e7e00700 1 -- 172.21.15.33:0/4212168179 learned_addr learned my addr 172.21.15.33:0/4212168179 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:35:21.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.463+0000 7f43f9cfeb80 1 -- 172.21.15.33:0/4212168179 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f43e81b47e0 con 0x55f7b1935650 2024-05-03T18:35:21.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.463+0000 7f43f9cfeb80 1 -- 172.21.15.33:0/4212168179 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f43e81b5c50 con 0x55f7b1939830 2024-05-03T18:35:21.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.463+0000 7f43f9cfeb80 1 -- 172.21.15.33:0/4212168179 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f43e81b4eb0 con 0x55f7b193a150 2024-05-03T18:35:21.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.463+0000 7f43f9cfeb80 10 client.0 _ll_register_callbacks cb 0x55f7b182ecf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:35:21.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.466+0000 7f43eca00700 1 -- 172.21.15.33:0/4212168179 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f7b193a150 msgr2=0x55f7b1945cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:35:21.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.466+0000 7f43eca00700 1 --2- 172.21.15.33:0/4212168179 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f7b193a150 0x55f7b1945cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:21.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.466+0000 7f43eca00700 1 -- 172.21.15.33:0/4212168179 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f7b1939830 msgr2=0x55f7b1945780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:35:21.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.466+0000 7f43eca00700 1 --2- 172.21.15.33:0/4212168179 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f7b1939830 0x55f7b1945780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:21.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.466+0000 7f43eca00700 1 -- 172.21.15.33:0/4212168179 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7b1939480 con 0x55f7b1935650 2024-05-03T18:35:21.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.466+0000 7f43e7e00700 1 --2- 172.21.15.33:0/4212168179 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f7b1939830 0x55f7b1945780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:35:21.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.469+0000 7f43eca00700 1 --2- 172.21.15.33:0/4212168179 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f7b1935650 0x55f7b19473d0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f43d800b930 tx=0x7f43d800b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90fbd97ec91b3bf9 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:35:21.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.469+0000 7f43caa00700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:35:21.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.469+0000 7f43caa00700 1 -- 172.21.15.33:0/4212168179 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d8042060 con 0x55f7b1935650 2024-05-03T18:35:21.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.469+0000 7f43f9cfeb80 1 -- 172.21.15.33:0/4212168179 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43e00c3070 con 0x55f7b1935650 2024-05-03T18:35:21.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.469+0000 7f43caa00700 1 -- 172.21.15.33:0/4212168179 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9518+0+0 (secure 0 0 0) 0x7f43d8005250 con 0x55f7b1935650 2024-05-03T18:35:21.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.470+0000 7f43caa00700 1 -- 172.21.15.33:0/4212168179 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f43d8032030 con 0x55f7b1935650 2024-05-03T18:35:21.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.471+0000 7f43f9cfeb80 10 client.16209 fetch_fsmap learned FSMap version 21 2024-05-03T18:35:21.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.471+0000 7f43f9cfeb80 1 -- 172.21.15.33:0/4212168179 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55f7b19ba280 con 0x55f7b1935650 2024-05-03T18:35:21.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.473+0000 7f43caa00700 1 -- 172.21.15.33:0/4212168179 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f43e00c3070 con 0x55f7b1935650 2024-05-03T18:35:21.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.473+0000 7f43caa00700 1 -- 172.21.15.33:0/4212168179 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f43d8023440 con 0x55f7b1935650 2024-05-03T18:35:21.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.474+0000 7f43f9cfeb80 10 client.16209 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:35:21.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.475+0000 7f43f9cfeb80 10 client.16209 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:35:21.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.475+0000 7f43f9cfeb80 20 client.16209 populate_metadata read hostname 'smithi033' 2024-05-03T18:35:21.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.475+0000 7f43f9cfeb80 1 --2- 172.21.15.33:0/4212168179 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55f7b19ba8e0 0x55f7b19bccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:21.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.475+0000 7f43f9cfeb80 4 client.16209 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-03T18:35:21.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.475+0000 7f43f9cfeb80 1 -- 172.21.15.33:0/4212168179 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f7b1895480 con 0x55f7b19ba8e0 2024-05-03T18:35:21.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.476+0000 7f43e7e00700 1 --2- 172.21.15.33:0/4212168179 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55f7b19ba8e0 0x55f7b19bccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:21.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.480+0000 7f43e7e00700 1 --2- 172.21.15.33:0/4212168179 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55f7b19ba8e0 0x55f7b19bccd0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0da80df4f1e0fa5 server_cookie=c24343ec9336ff9e in_seq=0 out_seq=0 2024-05-03T18:35:21.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.480+0000 7f43caa00700 10 client.16209 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:35:21.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.484+0000 7f43caa00700 1 -- 172.21.15.33:0/4212168179 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f7b1895480 con 0x55f7b19ba8e0 2024-05-03T18:35:21.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.484+0000 7f43caa00700 10 client.16209 handle_command_reply: tid=0 2024-05-03T18:35:21.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.484+0000 7f43f9cfeb80 1 client.16209 shutdown 2024-05-03T18:35:21.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.484+0000 7f43f9cfeb80 1 -- 172.21.15.33:0/4212168179 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f7b1935650 msgr2=0x55f7b19473d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:35:21.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.484+0000 7f43f9cfeb80 1 --2- 172.21.15.33:0/4212168179 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f7b1935650 0x55f7b19473d0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f43d800b930 tx=0x7f43d800b790 comp rx=0 tx=0).stop 2024-05-03T18:35:21.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.484+0000 7f43f9cfeb80 1 -- 172.21.15.33:0/4212168179 shutdown_connections 2024-05-03T18:35:21.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.484+0000 7f43f9cfeb80 1 --2- 172.21.15.33:0/4212168179 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55f7b19ba8e0 0x55f7b19bccd0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:21.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.484+0000 7f43f9cfeb80 1 --2- 172.21.15.33:0/4212168179 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f7b193a150 0x55f7b1945cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:21.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.484+0000 7f43f9cfeb80 1 --2- 172.21.15.33:0/4212168179 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f7b1935650 0x55f7b19473d0 secure :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f43d800b930 tx=0x7f43d800b790 comp rx=0 tx=0).stop 2024-05-03T18:35:21.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.484+0000 7f43f9cfeb80 1 --2- 172.21.15.33:0/4212168179 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f7b1939830 0x55f7b1945780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:21.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.484+0000 7f43f9cfeb80 1 -- 172.21.15.33:0/4212168179 >> 172.21.15.33:0/4212168179 conn(0x55f7b1863be0 msgr2=0x55f7b1938470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:35:21.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.484+0000 7f43caa00700 0 client.16209 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:35:21.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.485+0000 7f43f9cfeb80 1 -- 172.21.15.33:0/4212168179 shutdown_connections 2024-05-03T18:35:21.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.485+0000 7f43f9cfeb80 1 -- 172.21.15.33:0/4212168179 wait complete. 2024-05-03T18:35:21.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.485+0000 7f43f9cfeb80 20 client.16209 trim_cache size 0 max 16384 2024-05-03T18:35:21.490 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-03T18:35:21.490 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-05-03T18:35:21.490 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "db215cbd-4224-4b09-9ec6-a39c9e6ac59e", 2024-05-03T18:35:21.490 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-05-03T18:35:21.490 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-03T18:35:21.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.489+0000 7f43ef400700 1 -- 172.21.15.33:0/2369519418 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f43d00b2bc0 msgr2=0x7f43d00b5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:35:21.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.489+0000 7f43ef400700 1 --2- 172.21.15.33:0/2369519418 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f43d00b2bc0 0x7f43d00b5080 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f43d8009fd0 tx=0x7f43d8019040 comp rx=0 tx=0).stop 2024-05-03T18:35:21.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.489+0000 7f43ef400700 1 -- 172.21.15.33:0/2369519418 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43e81083d0 msgr2=0x7f43e81a4a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:35:21.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.489+0000 7f43ef400700 1 --2- 172.21.15.33:0/2369519418 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43e81083d0 0x7f43e81a4a70 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f43e000efc0 tx=0x7f43e000c3a0 comp rx=0 tx=0).stop 2024-05-03T18:35:21.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.489+0000 7f43ef400700 1 -- 172.21.15.33:0/2369519418 shutdown_connections 2024-05-03T18:35:21.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.489+0000 7f43ef400700 1 --2- 172.21.15.33:0/2369519418 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43e81083d0 0x7f43e81a4a70 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:21.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.489+0000 7f43ef400700 1 --2- 172.21.15.33:0/2369519418 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43e806b6f0 0x7f43e81a01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:21.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.489+0000 7f43ef400700 1 --2- 172.21.15.33:0/2369519418 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f43d00b2bc0 0x7f43d00b5080 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:21.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.489+0000 7f43ef400700 1 --2- 172.21.15.33:0/2369519418 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43e8105ee0 0x7f43e81a06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:21.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.489+0000 7f43ef400700 1 -- 172.21.15.33:0/2369519418 >> 172.21.15.33:0/2369519418 conn(0x7f43e8079350 msgr2=0x7f43e81045d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:35:21.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.490+0000 7f43ef400700 1 -- 172.21.15.33:0/2369519418 shutdown_connections 2024-05-03T18:35:21.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:21.490+0000 7f43ef400700 1 -- 172.21.15.33:0/2369519418 wait complete. 2024-05-03T18:35:22.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:35:22 smithi033 ceph-mon[28330]: pgmap v252: 129 pgs: 129 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 5.9 MiB/s wr, 540 op/s 2024-05-03T18:35:22.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:35:22 smithi150 ceph-mon[32492]: pgmap v252: 129 pgs: 129 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 5.9 MiB/s wr, 540 op/s 2024-05-03T18:35:22.426 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-03T18:35:22.547 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:35:22 smithi099 ceph-mon[31658]: pgmap v252: 129 pgs: 129 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 5.9 MiB/s wr, 540 op/s 2024-05-03T18:35:23.159 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:35:23.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:35:23 smithi150 ceph-mon[32492]: scrub queued for path: / 2024-05-03T18:35:23.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:35:23 smithi150 ceph-mon[32492]: scrub summary: idle+waiting paths [/] 2024-05-03T18:35:23.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:35:23 smithi150 ceph-mon[32492]: scrub summary: active paths [/] 2024-05-03T18:35:23.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:35:23 smithi150 ceph-mon[32492]: scrub summary: idle+waiting paths [/] 2024-05-03T18:35:23.393 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:35:23 smithi033 ceph-mon[28330]: scrub queued for path: / 2024-05-03T18:35:23.393 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:35:23 smithi033 ceph-mon[28330]: scrub summary: idle+waiting paths [/] 2024-05-03T18:35:23.393 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:35:23 smithi033 ceph-mon[28330]: scrub summary: active paths [/] 2024-05-03T18:35:23.393 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:35:23 smithi033 ceph-mon[28330]: scrub summary: idle+waiting paths [/] 2024-05-03T18:35:23.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:35:23 smithi099 ceph-mon[31658]: scrub queued for path: / 2024-05-03T18:35:23.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:35:23 smithi099 ceph-mon[31658]: scrub summary: idle+waiting paths [/] 2024-05-03T18:35:23.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:35:23 smithi099 ceph-mon[31658]: scrub summary: active paths [/] 2024-05-03T18:35:23.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:35:23 smithi099 ceph-mon[31658]: scrub summary: idle+waiting paths [/] 2024-05-03T18:35:24.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:35:24 smithi150 ceph-mon[32492]: pgmap v253: 129 pgs: 129 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 8.9 MiB/s wr, 818 op/s 2024-05-03T18:35:24.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:35:24 smithi033 ceph-mon[28330]: pgmap v253: 129 pgs: 129 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 8.9 MiB/s wr, 818 op/s 2024-05-03T18:35:24.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:35:24 smithi099 ceph-mon[31658]: pgmap v253: 129 pgs: 129 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 8.9 MiB/s wr, 818 op/s 2024-05-03T18:35:25.110 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 250.0 s, 541.0 tps, lat 183.848 ms stddev 81.132 2024-05-03T18:35:26.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:35:26 smithi150 ceph-mon[32492]: pgmap v254: 129 pgs: 129 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 5.9 MiB/s wr, 550 op/s 2024-05-03T18:35:26.472 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:35:26 smithi033 ceph-mon[28330]: pgmap v254: 129 pgs: 129 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 5.9 MiB/s wr, 550 op/s 2024-05-03T18:35:26.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:35:26 smithi099 ceph-mon[31658]: pgmap v254: 129 pgs: 129 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 5.9 MiB/s wr, 550 op/s 2024-05-03T18:35:27.207 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:35:27 smithi033 ceph-mon[28330]: scrub summary: idle 2024-05-03T18:35:27.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:35:27 smithi150 ceph-mon[32492]: scrub summary: idle 2024-05-03T18:35:27.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.464+0000 7f1449c00700 1 -- 172.21.15.33:0/2467136869 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1444108410 msgr2=0x7f14441087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:35:27.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.464+0000 7f1449c00700 1 --2- 172.21.15.33:0/2467136869 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1444108410 0x7f14441087f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f14340098f0 tx=0x7f1434009c00 comp rx=0 tx=0).stop 2024-05-03T18:35:27.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.464+0000 7f1449c00700 1 -- 172.21.15.33:0/2467136869 shutdown_connections 2024-05-03T18:35:27.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.464+0000 7f1449c00700 1 --2- 172.21.15.33:0/2467136869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1444102d60 0x7f144410f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:27.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.464+0000 7f1449c00700 1 --2- 172.21.15.33:0/2467136869 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1444108410 0x7f14441087f0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:27.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.464+0000 7f1449c00700 1 --2- 172.21.15.33:0/2467136869 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f14441023a0 0x7f1444102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:27.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.464+0000 7f1449c00700 1 -- 172.21.15.33:0/2467136869 >> 172.21.15.33:0/2467136869 conn(0x7f1444079350 msgr2=0x7f1444079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:35:27.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.465+0000 7f1449c00700 1 -- 172.21.15.33:0/2467136869 shutdown_connections 2024-05-03T18:35:27.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.465+0000 7f1449c00700 1 -- 172.21.15.33:0/2467136869 wait complete. 2024-05-03T18:35:27.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.467+0000 7f1449c00700 1 Processor -- start 2024-05-03T18:35:27.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.468+0000 7f1449c00700 1 -- start start 2024-05-03T18:35:27.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.469+0000 7f1449c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f14441023a0 0x7f1444117b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:27.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.470+0000 7f1449c00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1444102d60 0x7f1444118070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:27.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.470+0000 7f1449c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1444108410 0x7f1444111c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:27.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.470+0000 7f1449c00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f14441104b0 con 0x7f1444102d60 2024-05-03T18:35:27.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.470+0000 7f1449c00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1444110330 con 0x7f14441023a0 2024-05-03T18:35:27.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.470+0000 7f1449c00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f1444110630 con 0x7f1444108410 2024-05-03T18:35:27.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.470+0000 7f1442a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f14441023a0 0x7f1444117b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:27.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.470+0000 7f1442a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f14441023a0 0x7f1444117b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:45376/0 (socket says 172.21.15.33:45376) 2024-05-03T18:35:27.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.470+0000 7f1442a00700 1 -- 172.21.15.33:0/572000632 learned_addr learned my addr 172.21.15.33:0/572000632 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:35:27.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.471+0000 7f1442000700 1 --2- 172.21.15.33:0/572000632 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1444102d60 0x7f1444118070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:27.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.471+0000 7f1442a00700 1 -- 172.21.15.33:0/572000632 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1444108410 msgr2=0x7f1444111c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:35:27.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.471+0000 7f1443400700 1 --2- 172.21.15.33:0/572000632 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1444108410 0x7f1444111c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:27.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.472+0000 7f1442a00700 1 --2- 172.21.15.33:0/572000632 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1444108410 0x7f1444111c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:27.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.472+0000 7f1442a00700 1 -- 172.21.15.33:0/572000632 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1444102d60 msgr2=0x7f1444118070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:35:27.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.472+0000 7f1442a00700 1 --2- 172.21.15.33:0/572000632 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1444102d60 0x7f1444118070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:27.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.472+0000 7f1442a00700 1 -- 172.21.15.33:0/572000632 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1444112550 con 0x7f14441023a0 2024-05-03T18:35:27.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.472+0000 7f1443400700 1 --2- 172.21.15.33:0/572000632 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1444108410 0x7f1444111c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:35:27.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.473+0000 7f1442a00700 1 --2- 172.21.15.33:0/572000632 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f14441023a0 0x7f1444117b30 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f143400fa00 tx=0x7f143400b610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:35:27.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.475+0000 7f143b400700 1 -- 172.21.15.33:0/572000632 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1434027070 con 0x7f14441023a0 2024-05-03T18:35:27.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.475+0000 7f143b400700 1 -- 172.21.15.33:0/572000632 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1434005040 con 0x7f14441023a0 2024-05-03T18:35:27.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.475+0000 7f143b400700 1 -- 172.21.15.33:0/572000632 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14340049c0 con 0x7f14441023a0 2024-05-03T18:35:27.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.475+0000 7f1449c00700 1 -- 172.21.15.33:0/572000632 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14441127e0 con 0x7f14441023a0 2024-05-03T18:35:27.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.475+0000 7f1449c00700 1 -- 172.21.15.33:0/572000632 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14441b5400 con 0x7f14441023a0 2024-05-03T18:35:27.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.483+0000 7f14545deb80 1 Processor -- start 2024-05-03T18:35:27.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.488+0000 7f14545deb80 1 -- start start 2024-05-03T18:35:27.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.488+0000 7f14545deb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557de3467930 0x557de3467d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:27.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.488+0000 7f14545deb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557de3468250 0x557de3461f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:27.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.488+0000 7f14545deb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557de3462550 0x557de34629d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:27.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.488+0000 7f14545deb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x557de33b4a20 con 0x557de3467930 2024-05-03T18:35:27.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.488+0000 7f14545deb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x557de3352760 con 0x557de3462550 2024-05-03T18:35:27.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.489+0000 7f14545deb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x557de33b09e0 con 0x557de3468250 2024-05-03T18:35:27.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.491+0000 7f1443400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557de3467930 0x557de3467d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:27.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.491+0000 7f143b400700 1 -- 172.21.15.33:0/572000632 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f143402a070 con 0x7f14441023a0 2024-05-03T18:35:27.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.491+0000 7f1442000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557de3462550 0x557de34629d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:27.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.491+0000 7f1442000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557de3462550 0x557de34629d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:45390/0 (socket says 172.21.15.33:45390) 2024-05-03T18:35:27.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.491+0000 7f1442000700 1 -- 172.21.15.33:0/1104647908 learned_addr learned my addr 172.21.15.33:0/1104647908 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:35:27.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.492+0000 7f1442000700 1 -- 172.21.15.33:0/1104647908 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557de3468250 msgr2=0x557de3461f30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-03T18:35:27.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.492+0000 7f1442000700 1 --2- 172.21.15.33:0/1104647908 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557de3468250 0x557de3461f30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:27.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.492+0000 7f1442000700 1 -- 172.21.15.33:0/1104647908 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557de3467930 msgr2=0x557de3467d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:35:27.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.493+0000 7f1442000700 1 --2- 172.21.15.33:0/1104647908 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557de3467930 0x557de3467d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:27.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.493+0000 7f1442000700 1 -- 172.21.15.33:0/1104647908 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557de33018d0 con 0x557de3462550 2024-05-03T18:35:27.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.494+0000 7f1443400700 1 --2- 172.21.15.33:0/1104647908 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557de3467930 0x557de3467d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:35:27.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.496+0000 7f1442000700 1 --2- 172.21.15.33:0/1104647908 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557de3462550 0x557de34629d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1430009c80 tx=0x7f1430009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5fb128ddd120d2ed server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:35:27.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.496+0000 7f142fe00700 1 -- 172.21.15.33:0/1104647908 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1430011bf0 con 0x557de3462550 2024-05-03T18:35:27.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.497+0000 7f142fe00700 1 -- 172.21.15.33:0/1104647908 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f14300044c0 con 0x557de3462550 2024-05-03T18:35:27.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.497+0000 7f142fe00700 1 -- 172.21.15.33:0/1104647908 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1430003dd0 con 0x557de3462550 2024-05-03T18:35:27.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.498+0000 7f14545deb80 1 -- 172.21.15.33:0/1104647908 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557de3462550 msgr2=0x557de34629d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:35:27.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.498+0000 7f14545deb80 1 --2- 172.21.15.33:0/1104647908 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557de3462550 0x557de34629d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1430009c80 tx=0x7f1430009f90 comp rx=0 tx=0).stop 2024-05-03T18:35:27.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.499+0000 7f14545deb80 1 -- 172.21.15.33:0/1104647908 shutdown_connections 2024-05-03T18:35:27.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.499+0000 7f14545deb80 1 --2- 172.21.15.33:0/1104647908 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557de3462550 0x557de34629d0 secure :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1430009c80 tx=0x7f1430009f90 comp rx=0 tx=0).stop 2024-05-03T18:35:27.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.499+0000 7f14545deb80 1 --2- 172.21.15.33:0/1104647908 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557de3467930 0x557de3467d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:27.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.499+0000 7f14545deb80 1 --2- 172.21.15.33:0/1104647908 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557de3468250 0x557de3461f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:27.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.499+0000 7f14545deb80 1 -- 172.21.15.33:0/1104647908 >> 172.21.15.33:0/1104647908 conn(0x557de33da6d0 msgr2=0x557de345c900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:35:27.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.499+0000 7f143b400700 1 --2- 172.21.15.33:0/572000632 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f14280b2b20 0x7f14280b4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:27.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.499+0000 7f143b400700 1 -- 172.21.15.33:0/572000632 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9518+0+0 (secure 0 0 0) 0x7f14340e34e0 con 0x7f14441023a0 2024-05-03T18:35:27.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.499+0000 7f1443400700 1 --2- 172.21.15.33:0/572000632 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f14280b2b20 0x7f14280b4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:27.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.500+0000 7f14545deb80 1 -- 172.21.15.33:0/1104647908 shutdown_connections 2024-05-03T18:35:27.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.500+0000 7f1443400700 1 --2- 172.21.15.33:0/572000632 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f14280b2b20 0x7f14280b4fe0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f143c009cd0 tx=0x7f143c009040 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:35:27.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.500+0000 7f14545deb80 1 -- 172.21.15.33:0/1104647908 wait complete. 2024-05-03T18:35:27.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.501+0000 7f14545deb80 1 -- start start 2024-05-03T18:35:27.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.501+0000 7f14545deb80 1 Processor -- start 2024-05-03T18:35:27.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.502+0000 7f14545deb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557de3467930 0x557de346e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:27.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.502+0000 7f14545deb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557de3468250 0x557de346b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:27.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.502+0000 7f14545deb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557de346b860 0x557de346bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:27.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.502+0000 7f14545deb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x557de347fd10 con 0x557de3467930 2024-05-03T18:35:27.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.502+0000 7f14545deb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x557de331ab20 con 0x557de3468250 2024-05-03T18:35:27.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.502+0000 7f14545deb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x557de347fe90 con 0x557de346b860 2024-05-03T18:35:27.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.502+0000 7f14545deb80 10 client.0 _ll_register_callbacks cb 0x557de335bc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:35:27.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.503+0000 7f1443400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557de346b860 0x557de346bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:27.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.503+0000 7f1443400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557de346b860 0x557de346bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:40484/0 (socket says 172.21.15.33:40484) 2024-05-03T18:35:27.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.503+0000 7f1443400700 1 -- 172.21.15.33:0/1462015675 learned_addr learned my addr 172.21.15.33:0/1462015675 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:35:27.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.503+0000 7f1442000700 1 --2- 172.21.15.33:0/1462015675 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557de3468250 0x557de346b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:27.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.504+0000 7f1443400700 1 -- 172.21.15.33:0/1462015675 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557de3468250 msgr2=0x557de346b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:35:27.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.504+0000 7f1443400700 1 --2- 172.21.15.33:0/1462015675 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557de3468250 0x557de346b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:27.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.504+0000 7f1443400700 1 -- 172.21.15.33:0/1462015675 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557de3467930 msgr2=0x557de346e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:35:27.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.505+0000 7f1443400700 1 --2- 172.21.15.33:0/1462015675 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557de3467930 0x557de346e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:27.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.505+0000 7f1443400700 1 -- 172.21.15.33:0/1462015675 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557de337a420 con 0x557de346b860 2024-05-03T18:35:27.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.505+0000 7f1443400700 1 --2- 172.21.15.33:0/1462015675 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557de346b860 0x557de346bce0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f143c016e40 tx=0x7f143c004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=446ad7ecf361fe49 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:35:27.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.505+0000 7f142cc00700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-03T18:35:27.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.505+0000 7f142cc00700 1 -- 172.21.15.33:0/1462015675 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f143c01d050 con 0x557de346b860 2024-05-03T18:35:27.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.506+0000 7f14545deb80 1 -- 172.21.15.33:0/1462015675 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14300044c0 con 0x557de346b860 2024-05-03T18:35:27.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.506+0000 7f142cc00700 1 -- 172.21.15.33:0/1462015675 <== mon.1 v2:172.21.15.150:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9518+0+0 (secure 0 0 0) 0x7f143c012040 con 0x557de346b860 2024-05-03T18:35:27.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.506+0000 7f142cc00700 1 -- 172.21.15.33:0/1462015675 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f143c02f050 con 0x557de346b860 2024-05-03T18:35:27.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.506+0000 7f14545deb80 10 client.25921 fetch_fsmap learned FSMap version 21 2024-05-03T18:35:27.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.506+0000 7f14545deb80 1 -- 172.21.15.33:0/1462015675 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x557de348ab00 con 0x557de346b860 2024-05-03T18:35:27.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.507+0000 7f142cc00700 1 -- 172.21.15.33:0/1462015675 <== mon.1 v2:172.21.15.150:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f14300044c0 con 0x557de346b860 2024-05-03T18:35:27.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.507+0000 7f142cc00700 1 -- 172.21.15.33:0/1462015675 <== mon.1 v2:172.21.15.150:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f143c022680 con 0x557de346b860 2024-05-03T18:35:27.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.507+0000 7f14545deb80 10 client.25921 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:35:27.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.507+0000 7f14545deb80 10 client.25921 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:35:27.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.508+0000 7f14545deb80 20 client.25921 populate_metadata read hostname 'smithi033' 2024-05-03T18:35:27.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.508+0000 7f14545deb80 1 --2- 172.21.15.33:0/1462015675 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x557de348d140 0x557de348f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:27.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.508+0000 7f14545deb80 4 client.25921 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:35:27.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.508+0000 7f14545deb80 1 -- 172.21.15.33:0/1462015675 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557de342eff0 con 0x557de348d140 2024-05-03T18:35:27.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.509+0000 7f1442a00700 1 --2- 172.21.15.33:0/1462015675 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x557de348d140 0x557de348f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:27.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.510+0000 7f1442a00700 1 --2- 172.21.15.33:0/1462015675 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x557de348d140 0x557de348f5e0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0bf185b00b6f181 server_cookie=91168f25fbc8335a in_seq=0 out_seq=0 2024-05-03T18:35:27.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.510+0000 7f142cc00700 10 client.25921 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:35:27.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.512+0000 7f142cc00700 1 -- 172.21.15.33:0/1462015675 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557de342eff0 con 0x557de348d140 2024-05-03T18:35:27.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.512+0000 7f142cc00700 10 client.25921 handle_command_reply: tid=0 2024-05-03T18:35:27.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.513+0000 7f14545deb80 1 client.25921 shutdown 2024-05-03T18:35:27.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.513+0000 7f14545deb80 1 -- 172.21.15.33:0/1462015675 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557de346b860 msgr2=0x557de346bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:35:27.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.513+0000 7f14545deb80 1 --2- 172.21.15.33:0/1462015675 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557de346b860 0x557de346bce0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f143c016e40 tx=0x7f143c004b40 comp rx=0 tx=0).stop 2024-05-03T18:35:27.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.514+0000 7f14545deb80 1 -- 172.21.15.33:0/1462015675 shutdown_connections 2024-05-03T18:35:27.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.514+0000 7f14545deb80 1 --2- 172.21.15.33:0/1462015675 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x557de348d140 0x557de348f5e0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:27.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.515+0000 7f14545deb80 1 --2- 172.21.15.33:0/1462015675 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557de346b860 0x557de346bce0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:27.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.515+0000 7f14545deb80 1 --2- 172.21.15.33:0/1462015675 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557de3467930 0x557de346e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:27.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.515+0000 7f14545deb80 1 --2- 172.21.15.33:0/1462015675 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557de3468250 0x557de346b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:27.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.515+0000 7f14545deb80 1 -- 172.21.15.33:0/1462015675 >> 172.21.15.33:0/1462015675 conn(0x557de33da6d0 msgr2=0x557de34573b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:35:27.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.515+0000 7f142cc00700 0 client.25921 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:35:27.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.516+0000 7f14545deb80 1 -- 172.21.15.33:0/1462015675 shutdown_connections 2024-05-03T18:35:27.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.517+0000 7f14545deb80 1 -- 172.21.15.33:0/1462015675 wait complete. 2024-05-03T18:35:27.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.517+0000 7f14545deb80 20 client.25921 trim_cache size 0 max 16384 2024-05-03T18:35:27.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:35:27 smithi099 ceph-mon[31658]: scrub summary: idle 2024-05-03T18:35:27.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.559+0000 7f14545deb80 1 Processor -- start 2024-05-03T18:35:27.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.560+0000 7f14545deb80 1 -- start start 2024-05-03T18:35:27.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.560+0000 7f14545deb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557de3467930 0x557de3461f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:27.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.560+0000 7f14545deb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557de3468250 0x557de3462970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:27.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.560+0000 7f14545deb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557de346b860 0x557de3468da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:27.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.560+0000 7f14545deb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1444073b00 con 0x557de3468250 2024-05-03T18:35:27.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.560+0000 7f14545deb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f14441b6260 con 0x557de346b860 2024-05-03T18:35:27.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.560+0000 7f14545deb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f1444074150 con 0x557de3467930 2024-05-03T18:35:27.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.561+0000 7f1442a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557de3467930 0x557de3461f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:27.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.561+0000 7f1442a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557de3467930 0x557de3461f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:40500/0 (socket says 172.21.15.33:40500) 2024-05-03T18:35:27.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.561+0000 7f1442a00700 1 -- 172.21.15.33:0/1923604228 learned_addr learned my addr 172.21.15.33:0/1923604228 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:35:27.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.561+0000 7f1442000700 1 --2- 172.21.15.33:0/1923604228 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557de3468250 0x557de3462970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:27.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.561+0000 7f1443400700 1 --2- 172.21.15.33:0/1923604228 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557de346b860 0x557de3468da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:27.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.562+0000 7f1442a00700 1 -- 172.21.15.33:0/1923604228 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557de346b860 msgr2=0x557de3468da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:35:27.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.562+0000 7f1442a00700 1 --2- 172.21.15.33:0/1923604228 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557de346b860 0x557de3468da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:27.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.562+0000 7f1442a00700 1 -- 172.21.15.33:0/1923604228 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557de3468250 msgr2=0x557de3462970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:35:27.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.562+0000 7f1442a00700 1 --2- 172.21.15.33:0/1923604228 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557de3468250 0x557de3462970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:27.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.562+0000 7f1442a00700 1 -- 172.21.15.33:0/1923604228 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557de34698b0 con 0x557de3467930 2024-05-03T18:35:27.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.562+0000 7f1443400700 1 --2- 172.21.15.33:0/1923604228 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557de346b860 0x557de3468da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:35:27.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.562+0000 7f1442000700 1 --2- 172.21.15.33:0/1923604228 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557de3468250 0x557de3462970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:35:27.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.563+0000 7f1442a00700 1 --2- 172.21.15.33:0/1923604228 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557de3467930 0x557de3461f30 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1434011040 tx=0x7f1434005440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=862ed88ac25bccb7 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:35:27.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.563+0000 7f142f400700 1 -- 172.21.15.33:0/1923604228 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14340aa410 con 0x557de3467930 2024-05-03T18:35:27.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.564+0000 7f142f400700 1 -- 172.21.15.33:0/1923604228 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f14340ae7c0 con 0x557de3467930 2024-05-03T18:35:27.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.564+0000 7f142f400700 1 -- 172.21.15.33:0/1923604228 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14340e30b0 con 0x557de3467930 2024-05-03T18:35:27.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.565+0000 7f14545deb80 1 -- 172.21.15.33:0/1923604228 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557de3467930 msgr2=0x557de3461f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:35:27.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.565+0000 7f14545deb80 1 --2- 172.21.15.33:0/1923604228 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557de3467930 0x557de3461f30 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1434011040 tx=0x7f1434005440 comp rx=0 tx=0).stop 2024-05-03T18:35:27.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.565+0000 7f14545deb80 1 -- 172.21.15.33:0/1923604228 shutdown_connections 2024-05-03T18:35:27.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.565+0000 7f14545deb80 1 --2- 172.21.15.33:0/1923604228 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557de346b860 0x557de3468da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:27.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.565+0000 7f14545deb80 1 --2- 172.21.15.33:0/1923604228 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557de3467930 0x557de3461f30 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:27.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.565+0000 7f14545deb80 1 --2- 172.21.15.33:0/1923604228 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557de3468250 0x557de3462970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:27.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.565+0000 7f14545deb80 1 -- 172.21.15.33:0/1923604228 >> 172.21.15.33:0/1923604228 conn(0x557de33da6d0 msgr2=0x557de3468690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:35:27.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.567+0000 7f14545deb80 1 -- 172.21.15.33:0/1923604228 shutdown_connections 2024-05-03T18:35:27.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.567+0000 7f14545deb80 1 -- 172.21.15.33:0/1923604228 wait complete. 2024-05-03T18:35:27.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.567+0000 7f14545deb80 1 -- start start 2024-05-03T18:35:27.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.569+0000 7f14545deb80 1 Processor -- start 2024-05-03T18:35:27.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.570+0000 7f14545deb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557de3467930 0x557de3494580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:27.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.570+0000 7f14545deb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557de3468250 0x557de3494ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:27.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.570+0000 7f14545deb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557de346b860 0x557de3495000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:27.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.570+0000 7f14545deb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1444074820 con 0x557de3467930 2024-05-03T18:35:27.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.570+0000 7f14545deb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f14441b5b90 con 0x557de346b860 2024-05-03T18:35:27.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.570+0000 7f14545deb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f1444074ef0 con 0x557de3468250 2024-05-03T18:35:27.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.570+0000 7f14545deb80 10 client.0 _ll_register_callbacks cb 0x557de335bc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:35:27.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.571+0000 7f1442000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557de3468250 0x557de3494ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:27.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.572+0000 7f1442000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557de3468250 0x557de3494ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:40504/0 (socket says 172.21.15.33:40504) 2024-05-03T18:35:27.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.572+0000 7f1442a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557de3467930 0x557de3494580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:27.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.572+0000 7f1443400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557de346b860 0x557de3495000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:27.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.572+0000 7f1442000700 1 -- 172.21.15.33:0/2018712432 learned_addr learned my addr 172.21.15.33:0/2018712432 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:35:27.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.573+0000 7f1442a00700 1 -- 172.21.15.33:0/2018712432 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557de3468250 msgr2=0x557de3494ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:35:27.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.574+0000 7f1442a00700 1 --2- 172.21.15.33:0/2018712432 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557de3468250 0x557de3494ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:27.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.574+0000 7f1442a00700 1 -- 172.21.15.33:0/2018712432 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557de346b860 msgr2=0x557de3495000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:35:27.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.574+0000 7f1442a00700 1 --2- 172.21.15.33:0/2018712432 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557de346b860 0x557de3495000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:27.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.574+0000 7f1442a00700 1 -- 172.21.15.33:0/2018712432 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557de34f6a60 con 0x557de3467930 2024-05-03T18:35:27.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.574+0000 7f1442000700 1 --2- 172.21.15.33:0/2018712432 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557de3468250 0x557de3494ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:35:27.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.574+0000 7f1442a00700 1 --2- 172.21.15.33:0/2018712432 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557de3467930 0x557de3494580 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f1434009c40 tx=0x7f14340a9f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71b789d74a98dbaf server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:35:27.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.574+0000 7f1443400700 1 --2- 172.21.15.33:0/2018712432 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557de346b860 0x557de3495000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:35:27.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.574+0000 7f142cc00700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:35:27.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.575+0000 7f142cc00700 1 -- 172.21.15.33:0/2018712432 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14340c8020 con 0x557de3467930 2024-05-03T18:35:27.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.575+0000 7f142cc00700 1 -- 172.21.15.33:0/2018712432 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9518+0+0 (secure 0 0 0) 0x7f14340054d0 con 0x557de3467930 2024-05-03T18:35:27.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.575+0000 7f14545deb80 1 -- 172.21.15.33:0/2018712432 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14340ae7c0 con 0x557de3467930 2024-05-03T18:35:27.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.575+0000 7f142cc00700 1 -- 172.21.15.33:0/2018712432 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f14340b31e0 con 0x557de3467930 2024-05-03T18:35:27.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.575+0000 7f14545deb80 10 client.16224 fetch_fsmap learned FSMap version 21 2024-05-03T18:35:27.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.575+0000 7f14545deb80 1 -- 172.21.15.33:0/2018712432 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x557de34f7ba0 con 0x557de3467930 2024-05-03T18:35:27.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.575+0000 7f142cc00700 1 -- 172.21.15.33:0/2018712432 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f14340ae7c0 con 0x557de3467930 2024-05-03T18:35:27.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.576+0000 7f142cc00700 1 -- 172.21.15.33:0/2018712432 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f14340aa670 con 0x557de3467930 2024-05-03T18:35:27.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.576+0000 7f14545deb80 10 client.16224 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:35:27.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.576+0000 7f14545deb80 10 client.16224 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:35:27.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.576+0000 7f14545deb80 20 client.16224 populate_metadata read hostname 'smithi033' 2024-05-03T18:35:27.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.576+0000 7f14545deb80 1 --2- 172.21.15.33:0/2018712432 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x557de34f8230 0x557de34fa650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:27.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.576+0000 7f14545deb80 4 client.16224 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-03T18:35:27.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.576+0000 7f14545deb80 1 -- 172.21.15.33:0/2018712432 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557de33c2400 con 0x557de34f8230 2024-05-03T18:35:27.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.578+0000 7f1442000700 1 --2- 172.21.15.33:0/2018712432 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x557de34f8230 0x557de34fa650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:27.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.581+0000 7f1442000700 1 --2- 172.21.15.33:0/2018712432 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x557de34f8230 0x557de34fa650 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=296a8fcd0064260a server_cookie=389ad9b91a2d57d6 in_seq=0 out_seq=0 2024-05-03T18:35:27.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.582+0000 7f142cc00700 10 client.16224 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:35:27.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.582+0000 7f142cc00700 1 -- 172.21.15.33:0/2018712432 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x557de33c2400 con 0x557de34f8230 2024-05-03T18:35:27.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.582+0000 7f142cc00700 10 client.16224 handle_command_reply: tid=0 2024-05-03T18:35:27.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.582+0000 7f14545deb80 1 client.16224 shutdown 2024-05-03T18:35:27.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.582+0000 7f14545deb80 1 -- 172.21.15.33:0/2018712432 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557de3467930 msgr2=0x557de3494580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:35:27.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.582+0000 7f14545deb80 1 --2- 172.21.15.33:0/2018712432 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557de3467930 0x557de3494580 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f1434009c40 tx=0x7f14340a9f20 comp rx=0 tx=0).stop 2024-05-03T18:35:27.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.583+0000 7f14545deb80 1 -- 172.21.15.33:0/2018712432 shutdown_connections 2024-05-03T18:35:27.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.583+0000 7f14545deb80 1 --2- 172.21.15.33:0/2018712432 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x557de34f8230 0x557de34fa650 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:27.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.583+0000 7f14545deb80 1 --2- 172.21.15.33:0/2018712432 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557de346b860 0x557de3495000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:27.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.583+0000 7f14545deb80 1 --2- 172.21.15.33:0/2018712432 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557de3467930 0x557de3494580 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:27.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.583+0000 7f14545deb80 1 --2- 172.21.15.33:0/2018712432 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557de3468250 0x557de3494ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:27.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.583+0000 7f142cc00700 0 client.16224 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:35:27.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.583+0000 7f14545deb80 1 -- 172.21.15.33:0/2018712432 >> 172.21.15.33:0/2018712432 conn(0x557de33da6d0 msgr2=0x557de3468ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:35:27.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.583+0000 7f14545deb80 1 -- 172.21.15.33:0/2018712432 shutdown_connections 2024-05-03T18:35:27.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.583+0000 7f14545deb80 1 -- 172.21.15.33:0/2018712432 wait complete. 2024-05-03T18:35:27.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.584+0000 7f14545deb80 20 client.16224 trim_cache size 0 max 16384 2024-05-03T18:35:27.588 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-03T18:35:27.588 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-05-03T18:35:27.588 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-05-03T18:35:27.588 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-03T18:35:27.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.585+0000 7f1449c00700 1 -- 172.21.15.33:0/572000632 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f14280b2b20 msgr2=0x7f14280b4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:35:27.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.585+0000 7f1449c00700 1 --2- 172.21.15.33:0/572000632 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f14280b2b20 0x7f14280b4fe0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f143c009cd0 tx=0x7f143c009040 comp rx=0 tx=0).stop 2024-05-03T18:35:27.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.585+0000 7f1449c00700 1 -- 172.21.15.33:0/572000632 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f14441023a0 msgr2=0x7f1444117b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:35:27.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.585+0000 7f1449c00700 1 --2- 172.21.15.33:0/572000632 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f14441023a0 0x7f1444117b30 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f143400fa00 tx=0x7f143400b610 comp rx=0 tx=0).stop 2024-05-03T18:35:27.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.585+0000 7f1449c00700 1 -- 172.21.15.33:0/572000632 shutdown_connections 2024-05-03T18:35:27.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.585+0000 7f1449c00700 1 --2- 172.21.15.33:0/572000632 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1444108410 0x7f1444111c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:27.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.585+0000 7f1449c00700 1 --2- 172.21.15.33:0/572000632 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f14280b2b20 0x7f14280b4fe0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:27.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.586+0000 7f1449c00700 1 --2- 172.21.15.33:0/572000632 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f14441023a0 0x7f1444117b30 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:27.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.586+0000 7f1449c00700 1 --2- 172.21.15.33:0/572000632 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1444102d60 0x7f1444118070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:27.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.586+0000 7f1449c00700 1 -- 172.21.15.33:0/572000632 >> 172.21.15.33:0/572000632 conn(0x7f1444079350 msgr2=0x7f1444101320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:35:27.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.586+0000 7f1449c00700 1 -- 172.21.15.33:0/572000632 shutdown_connections 2024-05-03T18:35:27.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:27.586+0000 7f1449c00700 1 -- 172.21.15.33:0/572000632 wait complete. 2024-05-03T18:35:28.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:35:28 smithi150 ceph-mon[32492]: pgmap v255: 129 pgs: 129 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 5.9 MiB/s wr, 551 op/s 2024-05-03T18:35:28.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:35:28 smithi033 ceph-mon[28330]: pgmap v255: 129 pgs: 129 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 5.9 MiB/s wr, 551 op/s 2024-05-03T18:35:28.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:35:28 smithi099 ceph-mon[31658]: pgmap v255: 129 pgs: 129 active+clean; 2.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 5.9 MiB/s wr, 551 op/s 2024-05-03T18:35:28.571 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-03T18:35:28.571 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T18:35:29.319 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:35:30.110 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 255.0 s, 530.8 tps, lat 189.101 ms stddev 88.178 2024-05-03T18:35:30.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:35:30 smithi150 ceph-mon[32492]: pgmap v256: 129 pgs: 129 active+clean; 2.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 8.4 MiB/s wr, 776 op/s 2024-05-03T18:35:30.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:35:30 smithi033 ceph-mon[28330]: pgmap v256: 129 pgs: 129 active+clean; 2.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 8.4 MiB/s wr, 776 op/s 2024-05-03T18:35:30.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:35:30 smithi099 ceph-mon[31658]: pgmap v256: 129 pgs: 129 active+clean; 2.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 8.4 MiB/s wr, 776 op/s 2024-05-03T18:35:32.343 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:35:32 smithi033 ceph-mon[28330]: pgmap v257: 129 pgs: 129 active+clean; 2.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 5.5 MiB/s wr, 503 op/s 2024-05-03T18:35:32.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:35:32 smithi150 ceph-mon[32492]: pgmap v257: 129 pgs: 129 active+clean; 2.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 5.5 MiB/s wr, 503 op/s 2024-05-03T18:35:32.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:35:32 smithi099 ceph-mon[31658]: pgmap v257: 129 pgs: 129 active+clean; 2.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 5.5 MiB/s wr, 503 op/s 2024-05-03T18:35:32.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:32.715+0000 7f968bc00700 1 -- 172.21.15.33:0/3879365906 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96841056b0 msgr2=0x7f9684105b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:35:32.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:32.715+0000 7f968bc00700 1 --2- 172.21.15.33:0/3879365906 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96841056b0 0x7f9684105b30 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f96780098f0 tx=0x7f9678009c00 comp rx=0 tx=0).stop 2024-05-03T18:35:32.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:32.716+0000 7f968bc00700 1 -- 172.21.15.33:0/3879365906 shutdown_connections 2024-05-03T18:35:32.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:32.716+0000 7f968bc00700 1 --2- 172.21.15.33:0/3879365906 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9684106070 0x7f9684112580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:32.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:32.716+0000 7f968bc00700 1 --2- 172.21.15.33:0/3879365906 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f968410b6b0 0x7f968410ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:32.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:32.716+0000 7f968bc00700 1 --2- 172.21.15.33:0/3879365906 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96841056b0 0x7f9684105b30 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:32.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:32.716+0000 7f968bc00700 1 -- 172.21.15.33:0/3879365906 >> 172.21.15.33:0/3879365906 conn(0x7f9684101180 msgr2=0x7f96841035a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:35:32.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:32.717+0000 7f968bc00700 1 -- 172.21.15.33:0/3879365906 shutdown_connections 2024-05-03T18:35:32.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:32.717+0000 7f968bc00700 1 -- 172.21.15.33:0/3879365906 wait complete. 2024-05-03T18:35:32.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:32.719+0000 7f968bc00700 1 Processor -- start 2024-05-03T18:35:32.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:32.719+0000 7f968bc00700 1 -- start start 2024-05-03T18:35:32.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:32.719+0000 7f968bc00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96841056b0 0x7f96840779b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:32.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:32.719+0000 7f968bc00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9684106070 0x7f9684077ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:32.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:32.719+0000 7f968bc00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f968410b6b0 0x7f9684078430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:32.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:32.719+0000 7f968bc00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9684115740 con 0x7f96841056b0 2024-05-03T18:35:32.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:32.719+0000 7f968bc00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f96841155c0 con 0x7f9684106070 2024-05-03T18:35:32.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:32.719+0000 7f968bc00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f96841158c0 con 0x7f968410b6b0 2024-05-03T18:35:32.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:32.720+0000 7f9689c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f968410b6b0 0x7f9684078430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:32.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:32.720+0000 7f9689c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f968410b6b0 0x7f9684078430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:40528/0 (socket says 172.21.15.33:40528) 2024-05-03T18:35:32.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:32.720+0000 7f9689c00700 1 -- 172.21.15.33:0/2700181724 learned_addr learned my addr 172.21.15.33:0/2700181724 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:35:32.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:32.721+0000 7f9683e00700 1 --2- 172.21.15.33:0/2700181724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9684106070 0x7f9684077ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:32.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:32.721+0000 7f9689c00700 1 -- 172.21.15.33:0/2700181724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9684106070 msgr2=0x7f9684077ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:35:32.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:32.721+0000 7f9689c00700 1 --2- 172.21.15.33:0/2700181724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9684106070 0x7f9684077ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:32.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:32.721+0000 7f9689c00700 1 -- 172.21.15.33:0/2700181724 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96841056b0 msgr2=0x7f96840779b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:35:32.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:32.721+0000 7f9689c00700 1 --2- 172.21.15.33:0/2700181724 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96841056b0 0x7f96840779b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:32.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:32.721+0000 7f9689c00700 1 -- 172.21.15.33:0/2700181724 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f968419ba70 con 0x7f968410b6b0 2024-05-03T18:35:32.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:32.723+0000 7f9689c00700 1 --2- 172.21.15.33:0/2700181724 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f968410b6b0 0x7f9684078430 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f967c00ef60 tx=0x7f967c00ef90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:35:32.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:32.724+0000 7f9681600700 1 -- 172.21.15.33:0/2700181724 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f967c0092e0 con 0x7f968410b6b0 2024-05-03T18:35:32.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:32.724+0000 7f9681600700 1 -- 172.21.15.33:0/2700181724 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f967c004bb0 con 0x7f968410b6b0 2024-05-03T18:35:32.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:32.724+0000 7f9681600700 1 -- 172.21.15.33:0/2700181724 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f967c004020 con 0x7f968410b6b0 2024-05-03T18:35:32.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:32.724+0000 7f968bc00700 1 -- 172.21.15.33:0/2700181724 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f968419bd00 con 0x7f968410b6b0 2024-05-03T18:35:32.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:32.725+0000 7f968bc00700 1 -- 172.21.15.33:0/2700181724 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f968419c1c0 con 0x7f968410b6b0 2024-05-03T18:35:32.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:32.726+0000 7f968bc00700 1 -- 172.21.15.33:0/2700181724 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9684106df0 con 0x7f968410b6b0 2024-05-03T18:35:32.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:32.730+0000 7f9681600700 1 -- 172.21.15.33:0/2700181724 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f967c01b030 con 0x7f968410b6b0 2024-05-03T18:35:32.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:32.733+0000 7f9681600700 1 --2- 172.21.15.33:0/2700181724 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f966c0b2a70 0x7f966c0b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:32.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:32.733+0000 7f9681600700 1 -- 172.21.15.33:0/2700181724 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9518+0+0 (secure 0 0 0) 0x7f967c0e2460 con 0x7f968410b6b0 2024-05-03T18:35:32.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:32.734+0000 7f9689200700 1 --2- 172.21.15.33:0/2700181724 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f966c0b2a70 0x7f966c0b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:32.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:32.736+0000 7f9689200700 1 --2- 172.21.15.33:0/2700181724 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f966c0b2a70 0x7f966c0b4f30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9674005fd0 tx=0x7f9674009690 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:35:32.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:32.737+0000 7f9681600700 1 -- 172.21.15.33:0/2700181724 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f967c022020 con 0x7f968410b6b0 2024-05-03T18:35:33.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:33.225+0000 7f968bc00700 1 -- 172.21.15.33:0/2700181724 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f968402e510 con 0x7f968410b6b0 2024-05-03T18:35:33.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:33.227+0000 7f9681600700 1 -- 172.21.15.33:0/2700181724 <== mon.1 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11801 (secure 0 0 0) 0x7f967c010040 con 0x7f968410b6b0 2024-05-03T18:35:33.229 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:35:33.229 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1327393503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1327393503},{"type":"v1","addr":"172.21.15.33:6835","nonce":1327393503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/2461472948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":2461472948},{"type":"v1","addr":"172.21.15.33:6837","nonce":2461472948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14559,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/442269281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":442269281},{"type":"v1","addr":"172.21.15.33:6839","nonce":442269281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14571,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/645571756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":645571756},{"type":"v1","addr":"172.21.15.33:6841","nonce":645571756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/203598909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":203598909},{"type":"v1","addr":"172.21.15.150:6833","nonce":203598909}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/4029842493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":4029842493},{"type":"v1","addr":"172.21.15.150:6835","nonce":4029842493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3463664423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3463664423},{"type":"v1","addr":"172.21.15.150:6837","nonce":3463664423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24481,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6839/4051648018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":4051648018},{"type":"v1","addr":"172.21.15.150:6839","nonce":4051648018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34418,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1865105479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1865105479},{"type":"v1","addr":"172.21.15.99:6833","nonce":1865105479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34445,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/1206393914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1206393914},{"type":"v1","addr":"172.21.15.99:6835","nonce":1206393914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-03T18:22:28.837091+0000","modified":"2024-05-03T18:23:49.758398+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34508},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34472":{"gid":34472,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6837/3360970342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3360970342},{"type":"v1","addr":"172.21.15.99:6837","nonce":3360970342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34508":{"gid":34508,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/919907308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":919907308},{"type":"v1","addr":"172.21.15.99:6839","nonce":919907308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34508,"qdb_cluster":[34508]},"id":1}]} 2024-05-03T18:35:33.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:33.230+0000 7f968bc00700 1 -- 172.21.15.33:0/2700181724 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f966c0b2a70 msgr2=0x7f966c0b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:35:33.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:33.230+0000 7f968bc00700 1 --2- 172.21.15.33:0/2700181724 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f966c0b2a70 0x7f966c0b4f30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9674005fd0 tx=0x7f9674009690 comp rx=0 tx=0).stop 2024-05-03T18:35:33.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:33.230+0000 7f968bc00700 1 -- 172.21.15.33:0/2700181724 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f968410b6b0 msgr2=0x7f9684078430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:35:33.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:33.230+0000 7f968bc00700 1 --2- 172.21.15.33:0/2700181724 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f968410b6b0 0x7f9684078430 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f967c00ef60 tx=0x7f967c00ef90 comp rx=0 tx=0).stop 2024-05-03T18:35:33.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:33.231+0000 7f968bc00700 1 -- 172.21.15.33:0/2700181724 shutdown_connections 2024-05-03T18:35:33.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:33.231+0000 7f968bc00700 1 --2- 172.21.15.33:0/2700181724 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f968410b6b0 0x7f9684078430 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:33.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:33.231+0000 7f968bc00700 1 --2- 172.21.15.33:0/2700181724 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96841056b0 0x7f96840779b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:33.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:33.231+0000 7f968bc00700 1 --2- 172.21.15.33:0/2700181724 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f966c0b2a70 0x7f966c0b4f30 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:33.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:33.231+0000 7f968bc00700 1 --2- 172.21.15.33:0/2700181724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9684106070 0x7f9684077ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:33.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:33.231+0000 7f968bc00700 1 -- 172.21.15.33:0/2700181724 >> 172.21.15.33:0/2700181724 conn(0x7f9684101180 msgr2=0x7f9684101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:35:33.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:33.231+0000 7f968bc00700 1 -- 172.21.15.33:0/2700181724 shutdown_connections 2024-05-03T18:35:33.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:33.231+0000 7f968bc00700 1 -- 172.21.15.33:0/2700181724 wait complete. 2024-05-03T18:35:33.234 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 21 2024-05-03T18:35:33.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:35:33 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/2700181724' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:35:33.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:35:33 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/2700181724' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:35:33.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:35:33 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/2700181724' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:35:34.069 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-03T18:35:34.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:35:34 smithi150 ceph-mon[32492]: pgmap v258: 129 pgs: 129 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 8.0 MiB/s wr, 745 op/s 2024-05-03T18:35:34.736 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:35:34 smithi033 ceph-mon[28330]: pgmap v258: 129 pgs: 129 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 8.0 MiB/s wr, 745 op/s 2024-05-03T18:35:34.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:35:34 smithi099 ceph-mon[31658]: pgmap v258: 129 pgs: 129 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 8.0 MiB/s wr, 745 op/s 2024-05-03T18:35:34.865 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:35:35.110 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 260.0 s, 588.2 tps, lat 170.751 ms stddev 84.938 2024-05-03T18:35:36.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:35:36 smithi150 ceph-mon[32492]: pgmap v259: 129 pgs: 129 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 5.0 MiB/s wr, 467 op/s 2024-05-03T18:35:36.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:35:36 smithi033 ceph-mon[28330]: pgmap v259: 129 pgs: 129 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 5.0 MiB/s wr, 467 op/s 2024-05-03T18:35:36.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:35:36 smithi099 ceph-mon[31658]: pgmap v259: 129 pgs: 129 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 5.0 MiB/s wr, 467 op/s 2024-05-03T18:35:38.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.073+0000 7f9439e00700 1 -- 172.21.15.33:0/2850450879 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f943410b6b0 msgr2=0x7f943410ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:35:38.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.073+0000 7f9439e00700 1 --2- 172.21.15.33:0/2850450879 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f943410b6b0 0x7f943410ba90 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f9424009f90 tx=0x7f942400b330 comp rx=0 tx=0).stop 2024-05-03T18:35:38.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.073+0000 7f9439e00700 1 -- 172.21.15.33:0/2850450879 shutdown_connections 2024-05-03T18:35:38.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.073+0000 7f9439e00700 1 --2- 172.21.15.33:0/2850450879 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9434106070 0x7f9434112580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:38.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.073+0000 7f9439e00700 1 --2- 172.21.15.33:0/2850450879 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f943410b6b0 0x7f943410ba90 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:38.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.073+0000 7f9439e00700 1 --2- 172.21.15.33:0/2850450879 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f94341056b0 0x7f9434105b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:38.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.073+0000 7f9439e00700 1 -- 172.21.15.33:0/2850450879 >> 172.21.15.33:0/2850450879 conn(0x7f94341011e0 msgr2=0x7f9434103600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:35:38.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.074+0000 7f9439e00700 1 -- 172.21.15.33:0/2850450879 shutdown_connections 2024-05-03T18:35:38.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.075+0000 7f9439e00700 1 -- 172.21.15.33:0/2850450879 wait complete. 2024-05-03T18:35:38.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.077+0000 7f9439e00700 1 Processor -- start 2024-05-03T18:35:38.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.078+0000 7f9439e00700 1 -- start start 2024-05-03T18:35:38.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.079+0000 7f9439e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f94341056b0 0x7f94341124e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:38.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.079+0000 7f9439e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9434106070 0x7f943411cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:38.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.080+0000 7f9432a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f94341056b0 0x7f94341124e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:38.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.080+0000 7f9432a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f94341056b0 0x7f94341124e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:34210/0 (socket says 172.21.15.33:34210) 2024-05-03T18:35:38.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.080+0000 7f9439e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f943410b6b0 0x7f94341105f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:38.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.080+0000 7f9432a00700 1 -- 172.21.15.33:0/1204941714 learned_addr learned my addr 172.21.15.33:0/1204941714 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:35:38.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.080+0000 7f9439e00700 1 -- 172.21.15.33:0/1204941714 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f94340781a0 con 0x7f9434106070 2024-05-03T18:35:38.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.080+0000 7f9439e00700 1 -- 172.21.15.33:0/1204941714 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9434078020 con 0x7f943410b6b0 2024-05-03T18:35:38.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.080+0000 7f9439e00700 1 -- 172.21.15.33:0/1204941714 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9434078320 con 0x7f94341056b0 2024-05-03T18:35:38.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.080+0000 7f9433400700 1 --2- 172.21.15.33:0/1204941714 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f943410b6b0 0x7f94341105f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:38.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.080+0000 7f9432a00700 1 -- 172.21.15.33:0/1204941714 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f943410b6b0 msgr2=0x7f94341105f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:35:38.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.081+0000 7f9432a00700 1 --2- 172.21.15.33:0/1204941714 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f943410b6b0 0x7f94341105f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:38.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.081+0000 7f9432a00700 1 -- 172.21.15.33:0/1204941714 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9434106070 msgr2=0x7f943411cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:35:38.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.081+0000 7f9432a00700 1 --2- 172.21.15.33:0/1204941714 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9434106070 0x7f943411cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:38.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.081+0000 7f9432a00700 1 -- 172.21.15.33:0/1204941714 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9434110e30 con 0x7f94341056b0 2024-05-03T18:35:38.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.081+0000 7f9433400700 1 --2- 172.21.15.33:0/1204941714 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f943410b6b0 0x7f94341105f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:35:38.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.082+0000 7f9432a00700 1 --2- 172.21.15.33:0/1204941714 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f94341056b0 0x7f94341124e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9424004e20 tx=0x7f9424004f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:35:38.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.082+0000 7f9438a00700 1 -- 172.21.15.33:0/1204941714 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9424021070 con 0x7f94341056b0 2024-05-03T18:35:38.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.083+0000 7f9438a00700 1 -- 172.21.15.33:0/1204941714 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f942401d8e0 con 0x7f94341056b0 2024-05-03T18:35:38.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.083+0000 7f9439e00700 1 -- 172.21.15.33:0/1204941714 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94341110c0 con 0x7f94341056b0 2024-05-03T18:35:38.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.083+0000 7f9438a00700 1 -- 172.21.15.33:0/1204941714 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9424005420 con 0x7f94341056b0 2024-05-03T18:35:38.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.083+0000 7f9439e00700 1 -- 172.21.15.33:0/1204941714 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94341b5640 con 0x7f94341056b0 2024-05-03T18:35:38.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.096+0000 7f9438a00700 1 -- 172.21.15.33:0/1204941714 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f9424024070 con 0x7f94341056b0 2024-05-03T18:35:38.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.100+0000 7f94446c7b80 1 Processor -- start 2024-05-03T18:35:38.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.101+0000 7f94446c7b80 1 -- start start 2024-05-03T18:35:38.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.101+0000 7f9438a00700 1 --2- 172.21.15.33:0/1204941714 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f94180b2a70 0x7f94180b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:38.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.101+0000 7f94446c7b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d474885930 0x55d474885d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:38.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.101+0000 7f94446c7b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d474886250 0x55d47487ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:38.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.101+0000 7f94446c7b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d474880550 0x55d4748809d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:38.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.101+0000 7f94446c7b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d4747d2a20 con 0x55d474886250 2024-05-03T18:35:38.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.101+0000 7f94446c7b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55d474770760 con 0x55d474885930 2024-05-03T18:35:38.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.101+0000 7f94446c7b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55d4747ce9e0 con 0x55d474880550 2024-05-03T18:35:38.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.101+0000 7f9438a00700 1 -- 172.21.15.33:0/1204941714 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9518+0+0 (secure 0 0 0) 0x7f942400f020 con 0x7f94341056b0 2024-05-03T18:35:38.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.102+0000 7f9433400700 1 --2- 172.21.15.33:0/1204941714 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f94180b2a70 0x7f94180b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:38.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.102+0000 7f9432a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d474885930 0x55d474885d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:38.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.102+0000 7f9433400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d474880550 0x55d4748809d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:38.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.102+0000 7f9432a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d474885930 0x55d474885d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:52666/0 (socket says 172.21.15.33:52666) 2024-05-03T18:35:38.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.102+0000 7f9432a00700 1 -- 172.21.15.33:0/770722555 learned_addr learned my addr 172.21.15.33:0/770722555 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:35:38.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.102+0000 7f9433400700 1 --2- 172.21.15.33:0/1204941714 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f94180b2a70 0x7f94180b4f30 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f942c005e50 tx=0x7f942c009790 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:35:38.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.103+0000 7f9433400700 1 -- 172.21.15.33:0/770722555 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d474885930 msgr2=0x55d474885d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:35:38.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.103+0000 7f9433400700 1 --2- 172.21.15.33:0/770722555 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d474885930 0x55d474885d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:38.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.103+0000 7f9433400700 1 -- 172.21.15.33:0/770722555 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d474886250 msgr2=0x55d47487ff30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-03T18:35:38.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.103+0000 7f9433400700 1 --2- 172.21.15.33:0/770722555 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d474886250 0x55d47487ff30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:38.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.103+0000 7f9433400700 1 -- 172.21.15.33:0/770722555 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d47471f8d0 con 0x55d474880550 2024-05-03T18:35:38.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.103+0000 7f9433400700 1 --2- 172.21.15.33:0/770722555 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d474880550 0x55d4748809d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f942c012f70 tx=0x7f942c019000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be6a3e86e8b7f888 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:35:38.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.104+0000 7f9428c00700 1 -- 172.21.15.33:0/770722555 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f942c013cf0 con 0x55d474880550 2024-05-03T18:35:38.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.104+0000 7f9428c00700 1 -- 172.21.15.33:0/770722555 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f942c022070 con 0x55d474880550 2024-05-03T18:35:38.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.104+0000 7f9428c00700 1 -- 172.21.15.33:0/770722555 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f942c00f640 con 0x55d474880550 2024-05-03T18:35:38.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.105+0000 7f94446c7b80 1 -- 172.21.15.33:0/770722555 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d474880550 msgr2=0x55d4748809d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:35:38.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.105+0000 7f94446c7b80 1 --2- 172.21.15.33:0/770722555 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d474880550 0x55d4748809d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f942c012f70 tx=0x7f942c019000 comp rx=0 tx=0).stop 2024-05-03T18:35:38.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.106+0000 7f94446c7b80 1 -- 172.21.15.33:0/770722555 shutdown_connections 2024-05-03T18:35:38.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.106+0000 7f94446c7b80 1 --2- 172.21.15.33:0/770722555 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d474880550 0x55d4748809d0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:38.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.106+0000 7f94446c7b80 1 --2- 172.21.15.33:0/770722555 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d474885930 0x55d474885d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:38.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.106+0000 7f94446c7b80 1 --2- 172.21.15.33:0/770722555 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d474886250 0x55d47487ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:38.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.106+0000 7f94446c7b80 1 -- 172.21.15.33:0/770722555 >> 172.21.15.33:0/770722555 conn(0x55d4747f86d0 msgr2=0x55d47487a900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:35:38.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.106+0000 7f94446c7b80 1 -- 172.21.15.33:0/770722555 shutdown_connections 2024-05-03T18:35:38.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.106+0000 7f94446c7b80 1 -- 172.21.15.33:0/770722555 wait complete. 2024-05-03T18:35:38.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.107+0000 7f94446c7b80 1 -- start start 2024-05-03T18:35:38.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.108+0000 7f94446c7b80 1 Processor -- start 2024-05-03T18:35:38.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.108+0000 7f94446c7b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d474880550 0x55d47488be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:38.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.108+0000 7f94446c7b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d474885930 0x55d4748891c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:38.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.108+0000 7f94446c7b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d474886250 0x55d474889700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:38.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.109+0000 7f9432a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d474880550 0x55d47488be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:38.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.109+0000 7f942a000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d474885930 0x55d4748891c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:38.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.109+0000 7f942a000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d474885930 0x55d4748891c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32964/0 (socket says 172.21.15.33:32964) 2024-05-03T18:35:38.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.109+0000 7f9432a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d474880550 0x55d47488be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:34228/0 (socket says 172.21.15.33:34228) 2024-05-03T18:35:38.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.110+0000 7f94446c7b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d47488cca0 con 0x55d474885930 2024-05-03T18:35:38.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.110+0000 7f942a000700 1 -- 172.21.15.33:0/3588198283 learned_addr learned my addr 172.21.15.33:0/3588198283 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:35:38.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.110+0000 7f94446c7b80 1 -- 172.21.15.33:0/3588198283 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55d474738b20 con 0x55d474886250 2024-05-03T18:35:38.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.110+0000 7f94446c7b80 1 -- 172.21.15.33:0/3588198283 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55d47488ce20 con 0x55d474880550 2024-05-03T18:35:38.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.110+0000 7f94446c7b80 10 client.0 _ll_register_callbacks cb 0x55d474779c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:35:38.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.111+0000 7f9433400700 1 --2- 172.21.15.33:0/3588198283 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d474886250 0x55d474889700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:38.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.111+0000 7f9432a00700 1 -- 172.21.15.33:0/3588198283 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d474886250 msgr2=0x55d474889700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:35:38.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.111+0000 7f9432a00700 1 --2- 172.21.15.33:0/3588198283 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d474886250 0x55d474889700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:38.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.111+0000 7f9432a00700 1 -- 172.21.15.33:0/3588198283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d474885930 msgr2=0x55d4748891c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:35:38.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.112+0000 7f9432a00700 1 --2- 172.21.15.33:0/3588198283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d474885930 0x55d4748891c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:38.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.112+0000 7f9432a00700 1 -- 172.21.15.33:0/3588198283 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d474798420 con 0x55d474880550 2024-05-03T18:35:38.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.112+0000 7f942a000700 1 --2- 172.21.15.33:0/3588198283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d474885930 0x55d4748891c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:35:38.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.113+0000 7f9432a00700 1 --2- 172.21.15.33:0/3588198283 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d474880550 0x55d47488be20 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f94240aaf50 tx=0x7f942402e930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3665ad8a4132c4a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:35:38.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.113+0000 7f9421600700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-03T18:35:38.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.114+0000 7f9421600700 1 -- 172.21.15.33:0/3588198283 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94240b3d90 con 0x55d474880550 2024-05-03T18:35:38.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.114+0000 7f9421600700 1 -- 172.21.15.33:0/3588198283 <== mon.1 v2:172.21.15.150:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9518+0+0 (secure 0 0 0) 0x7f942401dd90 con 0x55d474880550 2024-05-03T18:35:38.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.114+0000 7f94446c7b80 1 -- 172.21.15.33:0/3588198283 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f942c022070 con 0x55d474880550 2024-05-03T18:35:38.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.115+0000 7f9421600700 1 -- 172.21.15.33:0/3588198283 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f9424011d00 con 0x55d474880550 2024-05-03T18:35:38.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.115+0000 7f94446c7b80 10 client.25942 fetch_fsmap learned FSMap version 21 2024-05-03T18:35:38.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.115+0000 7f94446c7b80 1 -- 172.21.15.33:0/3588198283 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55d47488cd00 con 0x55d474880550 2024-05-03T18:35:38.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.116+0000 7f9421600700 1 -- 172.21.15.33:0/3588198283 <== mon.1 v2:172.21.15.150:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f942c022070 con 0x55d474880550 2024-05-03T18:35:38.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.116+0000 7f9421600700 1 -- 172.21.15.33:0/3588198283 <== mon.1 v2:172.21.15.150:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f94240ad030 con 0x55d474880550 2024-05-03T18:35:38.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.116+0000 7f94446c7b80 10 client.25942 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:35:38.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.116+0000 7f94446c7b80 10 client.25942 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:35:38.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.116+0000 7f94446c7b80 20 client.25942 populate_metadata read hostname 'smithi033' 2024-05-03T18:35:38.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.116+0000 7f94446c7b80 1 --2- 172.21.15.33:0/3588198283 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55d474899990 0x55d47489bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:38.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.116+0000 7f94446c7b80 4 client.25942 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:35:38.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.116+0000 7f94446c7b80 1 -- 172.21.15.33:0/3588198283 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d47484cff0 con 0x55d474899990 2024-05-03T18:35:38.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.119+0000 7f942a000700 1 --2- 172.21.15.33:0/3588198283 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55d474899990 0x55d47489bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:38.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.120+0000 7f942a000700 1 --2- 172.21.15.33:0/3588198283 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55d474899990 0x55d47489bdf0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83de26f0cc26a5d3 server_cookie=1446e3c8a30dc7e8 in_seq=0 out_seq=0 2024-05-03T18:35:38.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.120+0000 7f9421600700 10 client.25942 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:35:38.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.122+0000 7f9421600700 1 -- 172.21.15.33:0/3588198283 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d47484cff0 con 0x55d474899990 2024-05-03T18:35:38.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.122+0000 7f9421600700 10 client.25942 handle_command_reply: tid=0 2024-05-03T18:35:38.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.122+0000 7f94446c7b80 1 client.25942 shutdown 2024-05-03T18:35:38.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.123+0000 7f94446c7b80 1 -- 172.21.15.33:0/3588198283 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d474880550 msgr2=0x55d47488be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:35:38.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.123+0000 7f94446c7b80 1 --2- 172.21.15.33:0/3588198283 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d474880550 0x55d47488be20 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f94240aaf50 tx=0x7f942402e930 comp rx=0 tx=0).stop 2024-05-03T18:35:38.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.123+0000 7f94446c7b80 1 -- 172.21.15.33:0/3588198283 shutdown_connections 2024-05-03T18:35:38.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.123+0000 7f94446c7b80 1 --2- 172.21.15.33:0/3588198283 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55d474899990 0x55d47489bdf0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:38.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.123+0000 7f94446c7b80 1 --2- 172.21.15.33:0/3588198283 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d474886250 0x55d474889700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:38.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.123+0000 7f94446c7b80 1 --2- 172.21.15.33:0/3588198283 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d474880550 0x55d47488be20 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:38.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.123+0000 7f94446c7b80 1 --2- 172.21.15.33:0/3588198283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d474885930 0x55d4748891c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:38.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.123+0000 7f94446c7b80 1 -- 172.21.15.33:0/3588198283 >> 172.21.15.33:0/3588198283 conn(0x55d4747f86d0 msgr2=0x55d4748753b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:35:38.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.123+0000 7f9421600700 0 client.25942 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:35:38.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.123+0000 7f94446c7b80 1 -- 172.21.15.33:0/3588198283 shutdown_connections 2024-05-03T18:35:38.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.123+0000 7f94446c7b80 1 -- 172.21.15.33:0/3588198283 wait complete. 2024-05-03T18:35:38.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.124+0000 7f94446c7b80 20 client.25942 trim_cache size 0 max 16384 2024-05-03T18:35:38.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.159+0000 7f94446c7b80 1 Processor -- start 2024-05-03T18:35:38.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.159+0000 7f94446c7b80 1 -- start start 2024-05-03T18:35:38.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.159+0000 7f94446c7b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d474880550 0x55d47487ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:38.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.159+0000 7f94446c7b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d474885930 0x55d474880990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:38.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.159+0000 7f94446c7b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d474886250 0x55d474889180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:38.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.160+0000 7f94446c7b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f94341bc470 con 0x55d474885930 2024-05-03T18:35:38.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.160+0000 7f94446c7b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f94341becd0 con 0x55d474886250 2024-05-03T18:35:38.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.160+0000 7f94446c7b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f94341bcac0 con 0x55d474880550 2024-05-03T18:35:38.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.161+0000 7f9432a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d474880550 0x55d47487ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:38.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.161+0000 7f9432a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d474880550 0x55d47487ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:34234/0 (socket says 172.21.15.33:34234) 2024-05-03T18:35:38.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.161+0000 7f9432a00700 1 -- 172.21.15.33:0/1462516770 learned_addr learned my addr 172.21.15.33:0/1462516770 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:35:38.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.161+0000 7f942a000700 1 --2- 172.21.15.33:0/1462516770 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d474885930 0x55d474880990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:38.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.161+0000 7f9433400700 1 --2- 172.21.15.33:0/1462516770 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d474886250 0x55d474889180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:38.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.162+0000 7f942a000700 1 -- 172.21.15.33:0/1462516770 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d474880550 msgr2=0x55d47487ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:35:38.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.162+0000 7f942a000700 1 --2- 172.21.15.33:0/1462516770 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d474880550 0x55d47487ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:38.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.162+0000 7f942a000700 1 -- 172.21.15.33:0/1462516770 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d474886250 msgr2=0x55d474889180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:35:38.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.162+0000 7f942a000700 1 --2- 172.21.15.33:0/1462516770 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d474886250 0x55d474889180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:38.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.162+0000 7f942a000700 1 -- 172.21.15.33:0/1462516770 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4748770f0 con 0x55d474885930 2024-05-03T18:35:38.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.163+0000 7f942a000700 1 --2- 172.21.15.33:0/1462516770 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d474885930 0x55d474880990 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x55d47489c640 tx=0x7f941c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6951def0d6c1374b server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:35:38.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.163+0000 7f9423e00700 1 -- 172.21.15.33:0/1462516770 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f941c017370 con 0x55d474885930 2024-05-03T18:35:38.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.163+0000 7f9423e00700 1 -- 172.21.15.33:0/1462516770 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f941c01b030 con 0x55d474885930 2024-05-03T18:35:38.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.163+0000 7f9423e00700 1 -- 172.21.15.33:0/1462516770 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f941c0149b0 con 0x55d474885930 2024-05-03T18:35:38.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.164+0000 7f94446c7b80 1 -- 172.21.15.33:0/1462516770 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d474885930 msgr2=0x55d474880990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:35:38.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.164+0000 7f94446c7b80 1 --2- 172.21.15.33:0/1462516770 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d474885930 0x55d474880990 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x55d47489c640 tx=0x7f941c008040 comp rx=0 tx=0).stop 2024-05-03T18:35:38.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.165+0000 7f94446c7b80 1 -- 172.21.15.33:0/1462516770 shutdown_connections 2024-05-03T18:35:38.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.165+0000 7f94446c7b80 1 --2- 172.21.15.33:0/1462516770 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d474886250 0x55d474889180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:38.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.165+0000 7f94446c7b80 1 --2- 172.21.15.33:0/1462516770 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d474880550 0x55d47487ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:38.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.165+0000 7f94446c7b80 1 --2- 172.21.15.33:0/1462516770 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d474885930 0x55d474880990 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:38.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.165+0000 7f94446c7b80 1 -- 172.21.15.33:0/1462516770 >> 172.21.15.33:0/1462516770 conn(0x55d4747f86d0 msgr2=0x55d47487a870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:35:38.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.165+0000 7f94446c7b80 1 -- 172.21.15.33:0/1462516770 shutdown_connections 2024-05-03T18:35:38.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.165+0000 7f94446c7b80 1 -- 172.21.15.33:0/1462516770 wait complete. 2024-05-03T18:35:38.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.166+0000 7f94446c7b80 1 -- start start 2024-05-03T18:35:38.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.166+0000 7f94446c7b80 1 Processor -- start 2024-05-03T18:35:38.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.167+0000 7f94446c7b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d474880550 0x55d4748920e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:38.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.167+0000 7f94446c7b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d474885930 0x55d474892620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:38.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.167+0000 7f94446c7b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d474886250 0x55d4748a06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:38.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.168+0000 7f9432a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d474880550 0x55d4748920e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:38.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.168+0000 7f94446c7b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f94341bd190 con 0x55d474880550 2024-05-03T18:35:38.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.168+0000 7f94446c7b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f94341be600 con 0x55d474886250 2024-05-03T18:35:38.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.168+0000 7f94446c7b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f94341bd860 con 0x55d474885930 2024-05-03T18:35:38.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.168+0000 7f94446c7b80 10 client.0 _ll_register_callbacks cb 0x55d47488a200 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:35:38.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.169+0000 7f9433400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d474886250 0x55d4748a06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:38.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.169+0000 7f9432a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d474880550 0x55d4748920e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32988/0 (socket says 172.21.15.33:32988) 2024-05-03T18:35:38.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.169+0000 7f9433400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d474886250 0x55d4748a06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:52708/0 (socket says 172.21.15.33:52708) 2024-05-03T18:35:38.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.169+0000 7f9433400700 1 -- 172.21.15.33:0/3603171280 learned_addr learned my addr 172.21.15.33:0/3603171280 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:35:38.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.169+0000 7f942a000700 1 --2- 172.21.15.33:0/3603171280 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d474885930 0x55d474892620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:38.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.170+0000 7f9433400700 1 -- 172.21.15.33:0/3603171280 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d474885930 msgr2=0x55d474892620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:35:38.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.170+0000 7f9433400700 1 --2- 172.21.15.33:0/3603171280 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d474885930 0x55d474892620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:38.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.170+0000 7f9433400700 1 -- 172.21.15.33:0/3603171280 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d474880550 msgr2=0x55d4748920e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:35:38.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.170+0000 7f9433400700 1 --2- 172.21.15.33:0/3603171280 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d474880550 0x55d4748920e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:38.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.170+0000 7f9433400700 1 -- 172.21.15.33:0/3603171280 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4748a0c30 con 0x55d474886250 2024-05-03T18:35:38.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.170+0000 7f9433400700 1 --2- 172.21.15.33:0/3603171280 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d474886250 0x55d4748a06f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f942c009510 tx=0x7f942c009a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=689409657b9b5b5b server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:35:38.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.170+0000 7f9421600700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-03T18:35:38.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.171+0000 7f9421600700 1 -- 172.21.15.33:0/3603171280 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f942c01d450 con 0x55d474886250 2024-05-03T18:35:38.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.171+0000 7f9421600700 1 -- 172.21.15.33:0/3603171280 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9518+0+0 (secure 0 0 0) 0x7f942c01d5f0 con 0x55d474886250 2024-05-03T18:35:38.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.171+0000 7f94446c7b80 1 -- 172.21.15.33:0/3603171280 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f941c01b030 con 0x55d474886250 2024-05-03T18:35:38.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.171+0000 7f9421600700 1 -- 172.21.15.33:0/3603171280 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f942c01d7f0 con 0x55d474886250 2024-05-03T18:35:38.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.171+0000 7f94446c7b80 10 client.35906 fetch_fsmap learned FSMap version 21 2024-05-03T18:35:38.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.171+0000 7f94446c7b80 1 -- 172.21.15.33:0/3603171280 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55d474903a50 con 0x55d474886250 2024-05-03T18:35:38.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.172+0000 7f9421600700 1 -- 172.21.15.33:0/3603171280 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f941c01b030 con 0x55d474886250 2024-05-03T18:35:38.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.172+0000 7f9421600700 1 -- 172.21.15.33:0/3603171280 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f942c02f020 con 0x55d474886250 2024-05-03T18:35:38.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.174+0000 7f94446c7b80 10 client.35906 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:35:38.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.174+0000 7f94446c7b80 10 client.35906 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:35:38.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.174+0000 7f94446c7b80 20 client.35906 populate_metadata read hostname 'smithi033' 2024-05-03T18:35:38.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.174+0000 7f94446c7b80 1 --2- 172.21.15.33:0/3603171280 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55d474903dd0 0x55d4749061c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:38.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.174+0000 7f94446c7b80 4 client.35906 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-03T18:35:38.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.174+0000 7f94446c7b80 1 -- 172.21.15.33:0/3603171280 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d4747e0400 con 0x55d474903dd0 2024-05-03T18:35:38.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.175+0000 7f9432a00700 1 --2- 172.21.15.33:0/3603171280 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55d474903dd0 0x55d4749061c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:38.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.176+0000 7f9432a00700 1 --2- 172.21.15.33:0/3603171280 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55d474903dd0 0x55d4749061c0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24d731b8c4a671eb server_cookie=952364ee38e39cf7 in_seq=0 out_seq=0 2024-05-03T18:35:38.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.176+0000 7f9421600700 10 client.35906 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:35:38.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.176+0000 7f9421600700 1 -- 172.21.15.33:0/3603171280 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d4747e0400 con 0x55d474903dd0 2024-05-03T18:35:38.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.176+0000 7f9421600700 10 client.35906 handle_command_reply: tid=0 2024-05-03T18:35:38.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.176+0000 7f94446c7b80 1 client.35906 shutdown 2024-05-03T18:35:38.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.177+0000 7f94446c7b80 1 -- 172.21.15.33:0/3603171280 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d474886250 msgr2=0x55d4748a06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:35:38.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.177+0000 7f94446c7b80 1 --2- 172.21.15.33:0/3603171280 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d474886250 0x55d4748a06f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f942c009510 tx=0x7f942c009a40 comp rx=0 tx=0).stop 2024-05-03T18:35:38.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.177+0000 7f94446c7b80 1 -- 172.21.15.33:0/3603171280 shutdown_connections 2024-05-03T18:35:38.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.177+0000 7f94446c7b80 1 --2- 172.21.15.33:0/3603171280 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55d474903dd0 0x55d4749061c0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:38.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.177+0000 7f94446c7b80 1 --2- 172.21.15.33:0/3603171280 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d474886250 0x55d4748a06f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:38.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.177+0000 7f94446c7b80 1 --2- 172.21.15.33:0/3603171280 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d474880550 0x55d4748920e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:38.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.177+0000 7f94446c7b80 1 --2- 172.21.15.33:0/3603171280 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d474885930 0x55d474892620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:38.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.177+0000 7f94446c7b80 1 -- 172.21.15.33:0/3603171280 >> 172.21.15.33:0/3603171280 conn(0x55d4747f86d0 msgr2=0x55d474889180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:35:38.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.177+0000 7f9421600700 0 client.35906 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:35:38.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.177+0000 7f94446c7b80 1 -- 172.21.15.33:0/3603171280 shutdown_connections 2024-05-03T18:35:38.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.177+0000 7f94446c7b80 1 -- 172.21.15.33:0/3603171280 wait complete. 2024-05-03T18:35:38.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.179+0000 7f94446c7b80 20 client.35906 trim_cache size 0 max 16384 2024-05-03T18:35:38.181 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-03T18:35:38.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.180+0000 7f9439e00700 1 -- 172.21.15.33:0/1204941714 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f94180b2a70 msgr2=0x7f94180b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:35:38.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.180+0000 7f9439e00700 1 --2- 172.21.15.33:0/1204941714 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f94180b2a70 0x7f94180b4f30 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f942c005e50 tx=0x7f942c009790 comp rx=0 tx=0).stop 2024-05-03T18:35:38.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.180+0000 7f9439e00700 1 -- 172.21.15.33:0/1204941714 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f94341056b0 msgr2=0x7f94341124e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:35:38.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.180+0000 7f9439e00700 1 --2- 172.21.15.33:0/1204941714 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f94341056b0 0x7f94341124e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9424004e20 tx=0x7f9424004f00 comp rx=0 tx=0).stop 2024-05-03T18:35:38.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.181+0000 7f9439e00700 1 -- 172.21.15.33:0/1204941714 shutdown_connections 2024-05-03T18:35:38.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.181+0000 7f9439e00700 1 --2- 172.21.15.33:0/1204941714 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f94180b2a70 0x7f94180b4f30 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:38.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.181+0000 7f9439e00700 1 --2- 172.21.15.33:0/1204941714 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f943410b6b0 0x7f94341105f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:38.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.181+0000 7f9439e00700 1 --2- 172.21.15.33:0/1204941714 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f94341056b0 0x7f94341124e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:38.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.181+0000 7f9439e00700 1 --2- 172.21.15.33:0/1204941714 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9434106070 0x7f943411cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:38.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.181+0000 7f9439e00700 1 -- 172.21.15.33:0/1204941714 >> 172.21.15.33:0/1204941714 conn(0x7f94341011e0 msgr2=0x7f94341019a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:35:38.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.181+0000 7f9439e00700 1 -- 172.21.15.33:0/1204941714 shutdown_connections 2024-05-03T18:35:38.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:38.181+0000 7f9439e00700 1 -- 172.21.15.33:0/1204941714 wait complete. 2024-05-03T18:35:38.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:35:38 smithi150 ceph-mon[32492]: pgmap v260: 129 pgs: 129 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 5.0 MiB/s wr, 468 op/s 2024-05-03T18:35:38.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:35:38 smithi033 ceph-mon[28330]: pgmap v260: 129 pgs: 129 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 5.0 MiB/s wr, 468 op/s 2024-05-03T18:35:38.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:35:38 smithi099 ceph-mon[31658]: pgmap v260: 129 pgs: 129 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 5.0 MiB/s wr, 468 op/s 2024-05-03T18:35:40.037 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-03T18:35:40.037 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-03T18:35:40.111 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 265.0 s, 532.9 tps, lat 186.903 ms stddev 147.332 2024-05-03T18:35:40.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:35:40 smithi150 ceph-mon[32492]: pgmap v261: 129 pgs: 129 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 7.8 MiB/s wr, 746 op/s 2024-05-03T18:35:40.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:35:40 smithi033 ceph-mon[28330]: pgmap v261: 129 pgs: 129 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 7.8 MiB/s wr, 746 op/s 2024-05-03T18:35:40.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:35:40 smithi099 ceph-mon[31658]: pgmap v261: 129 pgs: 129 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 7.8 MiB/s wr, 746 op/s 2024-05-03T18:35:40.797 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:35:42.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:35:42 smithi150 ceph-mon[32492]: pgmap v262: 129 pgs: 129 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 5.3 MiB/s wr, 521 op/s 2024-05-03T18:35:42.399 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:35:42 smithi033 ceph-mon[28330]: pgmap v262: 129 pgs: 129 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 5.3 MiB/s wr, 521 op/s 2024-05-03T18:35:42.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:35:42 smithi099 ceph-mon[31658]: pgmap v262: 129 pgs: 129 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 5.3 MiB/s wr, 521 op/s 2024-05-03T18:35:43.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.756+0000 7f7b9dc00700 1 -- 172.21.15.33:0/741897304 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b9810fea0 msgr2=0x7f7b98112290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:35:43.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.756+0000 7f7b9dc00700 1 --2- 172.21.15.33:0/741897304 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b9810fea0 0x7f7b98112290 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f7b80009990 tx=0x7f7b80009ca0 comp rx=0 tx=0).stop 2024-05-03T18:35:43.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.756+0000 7f7b9dc00700 1 -- 172.21.15.33:0/741897304 shutdown_connections 2024-05-03T18:35:43.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.756+0000 7f7b9dc00700 1 --2- 172.21.15.33:0/741897304 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b9810fea0 0x7f7b98112290 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:43.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.756+0000 7f7b9dc00700 1 --2- 172.21.15.33:0/741897304 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b980756d0 0x7f7b98075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:43.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.756+0000 7f7b9dc00700 1 --2- 172.21.15.33:0/741897304 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b98075ff0 0x7f7b9810f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:43.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.757+0000 7f7b9dc00700 1 -- 172.21.15.33:0/741897304 >> 172.21.15.33:0/741897304 conn(0x7f7b9807a600 msgr2=0x7f7b9807aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:35:43.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.757+0000 7f7b9dc00700 1 -- 172.21.15.33:0/741897304 shutdown_connections 2024-05-03T18:35:43.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.757+0000 7f7b9dc00700 1 -- 172.21.15.33:0/741897304 wait complete. 2024-05-03T18:35:43.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.759+0000 7f7b9dc00700 1 Processor -- start 2024-05-03T18:35:43.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.759+0000 7f7b9dc00700 1 -- start start 2024-05-03T18:35:43.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.760+0000 7f7b9dc00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b980756d0 0x7f7b9810a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:43.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.760+0000 7f7b9dc00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b98075ff0 0x7f7b9810a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:43.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.760+0000 7f7b9dc00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b9810fea0 0x7f7b98105400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:43.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.761+0000 7f7b96a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b980756d0 0x7f7b9810a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:43.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.761+0000 7f7b96a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b980756d0 0x7f7b9810a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:34254/0 (socket says 172.21.15.33:34254) 2024-05-03T18:35:43.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.761+0000 7f7b9dc00700 1 -- 172.21.15.33:0/3964409441 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7b98104050 con 0x7f7b98075ff0 2024-05-03T18:35:43.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.761+0000 7f7b9dc00700 1 -- 172.21.15.33:0/3964409441 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7b98103ed0 con 0x7f7b9810fea0 2024-05-03T18:35:43.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.761+0000 7f7b9dc00700 1 -- 172.21.15.33:0/3964409441 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f7b981041d0 con 0x7f7b980756d0 2024-05-03T18:35:43.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.761+0000 7f7b96a00700 1 -- 172.21.15.33:0/3964409441 learned_addr learned my addr 172.21.15.33:0/3964409441 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:35:43.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.762+0000 7f7b96000700 1 --2- 172.21.15.33:0/3964409441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b98075ff0 0x7f7b9810a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:43.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.762+0000 7f7b97400700 1 --2- 172.21.15.33:0/3964409441 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b9810fea0 0x7f7b98105400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:43.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.762+0000 7f7b96a00700 1 -- 172.21.15.33:0/3964409441 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b9810fea0 msgr2=0x7f7b98105400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:35:43.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.762+0000 7f7b96a00700 1 --2- 172.21.15.33:0/3964409441 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b9810fea0 0x7f7b98105400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:43.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.762+0000 7f7b96a00700 1 -- 172.21.15.33:0/3964409441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b98075ff0 msgr2=0x7f7b9810a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:35:43.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.762+0000 7f7b96a00700 1 --2- 172.21.15.33:0/3964409441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b98075ff0 0x7f7b9810a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:43.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.762+0000 7f7b96a00700 1 -- 172.21.15.33:0/3964409441 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b98105d00 con 0x7f7b980756d0 2024-05-03T18:35:43.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.763+0000 7f7b96a00700 1 --2- 172.21.15.33:0/3964409441 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b980756d0 0x7f7b9810a290 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7b9000ba50 tx=0x7f7b9000bd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:35:43.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.763+0000 7f7b8f400700 1 -- 172.21.15.33:0/3964409441 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b90019070 con 0x7f7b980756d0 2024-05-03T18:35:43.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.764+0000 7f7b8f400700 1 -- 172.21.15.33:0/3964409441 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7b9000ce40 con 0x7f7b980756d0 2024-05-03T18:35:43.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.764+0000 7f7b9dc00700 1 -- 172.21.15.33:0/3964409441 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b98105ff0 con 0x7f7b980756d0 2024-05-03T18:35:43.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.764+0000 7f7b9dc00700 1 -- 172.21.15.33:0/3964409441 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b9810e170 con 0x7f7b980756d0 2024-05-03T18:35:43.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.764+0000 7f7b8f400700 1 -- 172.21.15.33:0/3964409441 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b90007370 con 0x7f7b980756d0 2024-05-03T18:35:43.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.769+0000 7f7b8f400700 1 -- 172.21.15.33:0/3964409441 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f7b9001c070 con 0x7f7b980756d0 2024-05-03T18:35:43.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.772+0000 7f7b8f400700 1 --2- 172.21.15.33:0/3964409441 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f7b7c0b2c00 0x7f7b7c0b50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:43.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.772+0000 7f7b8f400700 1 -- 172.21.15.33:0/3964409441 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9518+0+0 (secure 0 0 0) 0x7f7b900e1d20 con 0x7f7b980756d0 2024-05-03T18:35:43.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.772+0000 7f7b96000700 1 --2- 172.21.15.33:0/3964409441 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f7b7c0b2c00 0x7f7b7c0b50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:43.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.775+0000 7f7ba8545b80 1 Processor -- start 2024-05-03T18:35:43.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.775+0000 7f7ba8545b80 1 -- start start 2024-05-03T18:35:43.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.775+0000 7f7ba8545b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560aaa4989b0 0x560aaa498d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:43.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.776+0000 7f7ba8545b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560aaa4992d0 0x560aaa492fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:43.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.776+0000 7f7ba8545b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560aaa4935d0 0x560aaa493a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:43.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.776+0000 7f7ba8545b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560aaa3e5aa0 con 0x560aaa4989b0 2024-05-03T18:35:43.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.776+0000 7f7ba8545b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x560aaa3837e0 con 0x560aaa4935d0 2024-05-03T18:35:43.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.776+0000 7f7ba8545b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x560aaa3e1a60 con 0x560aaa4992d0 2024-05-03T18:35:43.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.777+0000 7f7b96a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560aaa4989b0 0x560aaa498d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:43.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.777+0000 7f7b97400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560aaa4935d0 0x560aaa493a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:43.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.778+0000 7f7b96000700 1 --2- 172.21.15.33:0/3964409441 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f7b7c0b2c00 0x7f7b7c0b50c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7b98106b70 tx=0x7f7b8800b410 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:35:43.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.778+0000 7f7b96a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560aaa4989b0 0x560aaa498d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33002/0 (socket says 172.21.15.33:33002) 2024-05-03T18:35:43.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.778+0000 7f7b96a00700 1 -- 172.21.15.33:0/3470653246 learned_addr learned my addr 172.21.15.33:0/3470653246 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:35:43.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.778+0000 7f7b96000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560aaa4992d0 0x560aaa492fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:43.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.779+0000 7f7b96000700 1 -- 172.21.15.33:0/3470653246 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560aaa4935d0 msgr2=0x560aaa493a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:35:43.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.779+0000 7f7b96000700 1 --2- 172.21.15.33:0/3470653246 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560aaa4935d0 0x560aaa493a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:43.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.779+0000 7f7b96000700 1 -- 172.21.15.33:0/3470653246 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560aaa4989b0 msgr2=0x560aaa498d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:35:43.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.779+0000 7f7b96000700 1 --2- 172.21.15.33:0/3470653246 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560aaa4989b0 0x560aaa498d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:43.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.779+0000 7f7b96000700 1 -- 172.21.15.33:0/3470653246 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560aaa332950 con 0x560aaa4992d0 2024-05-03T18:35:43.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.780+0000 7f7b97400700 1 --2- 172.21.15.33:0/3470653246 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560aaa4935d0 0x560aaa493a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:35:43.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.780+0000 7f7b96a00700 1 --2- 172.21.15.33:0/3470653246 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560aaa4989b0 0x560aaa498d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:35:43.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.781+0000 7f7b96000700 1 --2- 172.21.15.33:0/3470653246 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560aaa4992d0 0x560aaa492fb0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7b980769d0 tx=0x7f7b88013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1beecd634621a791 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:35:43.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.781+0000 7f7b87e00700 1 -- 172.21.15.33:0/3470653246 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b88011bb0 con 0x560aaa4992d0 2024-05-03T18:35:43.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.781+0000 7f7b87e00700 1 -- 172.21.15.33:0/3470653246 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7b880044b0 con 0x560aaa4992d0 2024-05-03T18:35:43.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.781+0000 7f7b87e00700 1 -- 172.21.15.33:0/3470653246 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b88003dd0 con 0x560aaa4992d0 2024-05-03T18:35:43.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.782+0000 7f7ba8545b80 1 -- 172.21.15.33:0/3470653246 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560aaa4992d0 msgr2=0x560aaa492fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:35:43.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.782+0000 7f7ba8545b80 1 --2- 172.21.15.33:0/3470653246 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560aaa4992d0 0x560aaa492fb0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7b980769d0 tx=0x7f7b88013000 comp rx=0 tx=0).stop 2024-05-03T18:35:43.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.783+0000 7f7ba8545b80 1 -- 172.21.15.33:0/3470653246 shutdown_connections 2024-05-03T18:35:43.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.783+0000 7f7ba8545b80 1 --2- 172.21.15.33:0/3470653246 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560aaa4935d0 0x560aaa493a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:43.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.783+0000 7f7ba8545b80 1 --2- 172.21.15.33:0/3470653246 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560aaa4989b0 0x560aaa498d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:43.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.783+0000 7f7ba8545b80 1 --2- 172.21.15.33:0/3470653246 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560aaa4992d0 0x560aaa492fb0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:43.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.783+0000 7f7ba8545b80 1 -- 172.21.15.33:0/3470653246 >> 172.21.15.33:0/3470653246 conn(0x560aaa40b750 msgr2=0x560aaa48d980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:35:43.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.783+0000 7f7ba8545b80 1 -- 172.21.15.33:0/3470653246 shutdown_connections 2024-05-03T18:35:43.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.783+0000 7f7ba8545b80 1 -- 172.21.15.33:0/3470653246 wait complete. 2024-05-03T18:35:43.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.784+0000 7f7ba8545b80 1 -- start start 2024-05-03T18:35:43.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.785+0000 7f7ba8545b80 1 Processor -- start 2024-05-03T18:35:43.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.787+0000 7f7ba8545b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560aaa4935d0 0x560aaa49eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:43.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.787+0000 7f7ba8545b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560aaa4989b0 0x560aaa49c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:43.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.787+0000 7f7ba8545b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560aaa4992d0 0x560aaa49c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:43.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.787+0000 7f7ba8545b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560aaa49fd80 con 0x560aaa4989b0 2024-05-03T18:35:43.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.787+0000 7f7ba8545b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x560aaa34bba0 con 0x560aaa4935d0 2024-05-03T18:35:43.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.787+0000 7f7ba8545b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x560aaa49ff00 con 0x560aaa4992d0 2024-05-03T18:35:43.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.787+0000 7f7ba8545b80 10 client.0 _ll_register_callbacks cb 0x560aaa38ccf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:35:43.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.787+0000 7f7b96000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560aaa4989b0 0x560aaa49c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:43.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.788+0000 7f7b96000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560aaa4989b0 0x560aaa49c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33008/0 (socket says 172.21.15.33:33008) 2024-05-03T18:35:43.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.788+0000 7f7b96000700 1 -- 172.21.15.33:0/4041200580 learned_addr learned my addr 172.21.15.33:0/4041200580 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:35:43.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.788+0000 7f7b96a00700 1 --2- 172.21.15.33:0/4041200580 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560aaa4935d0 0x560aaa49eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:43.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.788+0000 7f7b97400700 1 --2- 172.21.15.33:0/4041200580 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560aaa4992d0 0x560aaa49c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:43.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.789+0000 7f7b97400700 1 -- 172.21.15.33:0/4041200580 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560aaa4935d0 msgr2=0x560aaa49eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:35:43.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.789+0000 7f7b97400700 1 --2- 172.21.15.33:0/4041200580 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560aaa4935d0 0x560aaa49eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:43.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.789+0000 7f7b97400700 1 -- 172.21.15.33:0/4041200580 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560aaa4989b0 msgr2=0x560aaa49c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:35:43.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.789+0000 7f7b97400700 1 --2- 172.21.15.33:0/4041200580 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560aaa4989b0 0x560aaa49c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:43.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.789+0000 7f7b97400700 1 -- 172.21.15.33:0/4041200580 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560aaa3ab4a0 con 0x560aaa4992d0 2024-05-03T18:35:43.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.790+0000 7f7b96000700 1 --2- 172.21.15.33:0/4041200580 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560aaa4989b0 0x560aaa49c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:35:43.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.790+0000 7f7b97400700 1 --2- 172.21.15.33:0/4041200580 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560aaa4992d0 0x560aaa49c780 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7b800098e0 tx=0x7f7b80005810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=642b4b169bf1e6ff server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:35:43.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.790+0000 7f7b84c00700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-03T18:35:43.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.791+0000 7f7b84c00700 1 -- 172.21.15.33:0/4041200580 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b800039c0 con 0x560aaa4992d0 2024-05-03T18:35:43.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.791+0000 7f7ba8545b80 1 -- 172.21.15.33:0/4041200580 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b880044b0 con 0x560aaa4992d0 2024-05-03T18:35:43.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.791+0000 7f7b84c00700 1 -- 172.21.15.33:0/4041200580 <== mon.1 v2:172.21.15.150:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9518+0+0 (secure 0 0 0) 0x7f7b800037c0 con 0x560aaa4992d0 2024-05-03T18:35:43.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.792+0000 7f7b84c00700 1 -- 172.21.15.33:0/4041200580 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f7b80030050 con 0x560aaa4992d0 2024-05-03T18:35:43.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.792+0000 7f7ba8545b80 10 client.25960 fetch_fsmap learned FSMap version 21 2024-05-03T18:35:43.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.792+0000 7f7ba8545b80 1 -- 172.21.15.33:0/4041200580 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x560aaa4aa500 con 0x560aaa4992d0 2024-05-03T18:35:43.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.793+0000 7f7b84c00700 1 -- 172.21.15.33:0/4041200580 <== mon.1 v2:172.21.15.150:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7b880044b0 con 0x560aaa4992d0 2024-05-03T18:35:43.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.793+0000 7f7b84c00700 1 -- 172.21.15.33:0/4041200580 <== mon.1 v2:172.21.15.150:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f7b8000f460 con 0x560aaa4992d0 2024-05-03T18:35:43.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.793+0000 7f7ba8545b80 10 client.25960 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:35:43.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.793+0000 7f7ba8545b80 10 client.25960 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:35:43.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.793+0000 7f7ba8545b80 20 client.25960 populate_metadata read hostname 'smithi033' 2024-05-03T18:35:43.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.793+0000 7f7ba8545b80 1 --2- 172.21.15.33:0/4041200580 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x560aaa4acb30 0x560aaa4aef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:43.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.793+0000 7f7ba8545b80 4 client.25960 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:35:43.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.793+0000 7f7ba8545b80 1 -- 172.21.15.33:0/4041200580 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560aaa460070 con 0x560aaa4acb30 2024-05-03T18:35:43.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.794+0000 7f7b96a00700 1 --2- 172.21.15.33:0/4041200580 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x560aaa4acb30 0x560aaa4aef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:43.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.795+0000 7f7b84c00700 10 client.25960 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:35:43.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.796+0000 7f7b96a00700 1 --2- 172.21.15.33:0/4041200580 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x560aaa4acb30 0x560aaa4aef50 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c06bb3bebc1fc66e server_cookie=c81b6d67e0eebfc in_seq=0 out_seq=0 2024-05-03T18:35:43.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.798+0000 7f7b84c00700 1 -- 172.21.15.33:0/4041200580 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560aaa460070 con 0x560aaa4acb30 2024-05-03T18:35:43.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.798+0000 7f7b84c00700 10 client.25960 handle_command_reply: tid=0 2024-05-03T18:35:43.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.798+0000 7f7ba8545b80 1 client.25960 shutdown 2024-05-03T18:35:43.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.799+0000 7f7ba8545b80 1 -- 172.21.15.33:0/4041200580 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560aaa4992d0 msgr2=0x560aaa49c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:35:43.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.799+0000 7f7ba8545b80 1 --2- 172.21.15.33:0/4041200580 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560aaa4992d0 0x560aaa49c780 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7b800098e0 tx=0x7f7b80005810 comp rx=0 tx=0).stop 2024-05-03T18:35:43.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.800+0000 7f7ba8545b80 1 -- 172.21.15.33:0/4041200580 shutdown_connections 2024-05-03T18:35:43.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.800+0000 7f7ba8545b80 1 --2- 172.21.15.33:0/4041200580 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x560aaa4acb30 0x560aaa4aef50 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:43.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.800+0000 7f7ba8545b80 1 --2- 172.21.15.33:0/4041200580 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560aaa4992d0 0x560aaa49c780 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:43.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.800+0000 7f7ba8545b80 1 --2- 172.21.15.33:0/4041200580 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560aaa4935d0 0x560aaa49eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:43.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.800+0000 7f7ba8545b80 1 --2- 172.21.15.33:0/4041200580 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560aaa4989b0 0x560aaa49c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:43.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.800+0000 7f7ba8545b80 1 -- 172.21.15.33:0/4041200580 >> 172.21.15.33:0/4041200580 conn(0x560aaa40b750 msgr2=0x560aaa488430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:35:43.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.800+0000 7f7b84c00700 0 client.25960 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:35:43.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.800+0000 7f7ba8545b80 1 -- 172.21.15.33:0/4041200580 shutdown_connections 2024-05-03T18:35:43.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.801+0000 7f7ba8545b80 1 -- 172.21.15.33:0/4041200580 wait complete. 2024-05-03T18:35:43.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.801+0000 7f7ba8545b80 20 client.25960 trim_cache size 0 max 16384 2024-05-03T18:35:43.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.843+0000 7f7ba8545b80 1 Processor -- start 2024-05-03T18:35:43.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.843+0000 7f7ba8545b80 1 -- start start 2024-05-03T18:35:43.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.843+0000 7f7ba8545b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560aaa4935d0 0x560aaa492fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:43.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.844+0000 7f7ba8545b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560aaa4989b0 0x560aaa49ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:43.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.844+0000 7f7ba8545b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560aaa4992d0 0x560aaa49b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:43.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.844+0000 7f7ba8545b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7b7400d0c0 con 0x560aaa4935d0 2024-05-03T18:35:43.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.844+0000 7f7ba8545b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7b7400f960 con 0x560aaa4989b0 2024-05-03T18:35:43.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.844+0000 7f7ba8545b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f7b7400d750 con 0x560aaa4992d0 2024-05-03T18:35:43.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.844+0000 7f7b97400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560aaa4992d0 0x560aaa49b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:43.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.844+0000 7f7b97400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560aaa4992d0 0x560aaa49b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:34290/0 (socket says 172.21.15.33:34290) 2024-05-03T18:35:43.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.844+0000 7f7b97400700 1 -- 172.21.15.33:0/511941387 learned_addr learned my addr 172.21.15.33:0/511941387 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:35:43.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.845+0000 7f7b97400700 1 -- 172.21.15.33:0/511941387 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560aaa4989b0 msgr2=0x560aaa49ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:35:43.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.845+0000 7f7b96000700 1 --2- 172.21.15.33:0/511941387 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560aaa4989b0 0x560aaa49ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:43.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.845+0000 7f7b97400700 1 --2- 172.21.15.33:0/511941387 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560aaa4989b0 0x560aaa49ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:43.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.845+0000 7f7b97400700 1 -- 172.21.15.33:0/511941387 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560aaa4935d0 msgr2=0x560aaa492fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:35:43.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.845+0000 7f7b97400700 1 --2- 172.21.15.33:0/511941387 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560aaa4935d0 0x560aaa492fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:43.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.845+0000 7f7b97400700 1 -- 172.21.15.33:0/511941387 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560aaa49a7b0 con 0x560aaa4992d0 2024-05-03T18:35:43.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.847+0000 7f7b97400700 1 --2- 172.21.15.33:0/511941387 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560aaa4992d0 0x560aaa49b070 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7b80005f00 tx=0x7f7b80003950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bff5b783b9f26a40 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:35:43.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.848+0000 7f7b87400700 1 -- 172.21.15.33:0/511941387 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b80039070 con 0x560aaa4992d0 2024-05-03T18:35:43.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.848+0000 7f7b87400700 1 -- 172.21.15.33:0/511941387 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7b80027070 con 0x560aaa4992d0 2024-05-03T18:35:43.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.848+0000 7f7b87400700 1 -- 172.21.15.33:0/511941387 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b80030050 con 0x560aaa4992d0 2024-05-03T18:35:43.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.849+0000 7f7ba8545b80 1 -- 172.21.15.33:0/511941387 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560aaa4992d0 msgr2=0x560aaa49b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:35:43.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.849+0000 7f7ba8545b80 1 --2- 172.21.15.33:0/511941387 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560aaa4992d0 0x560aaa49b070 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7b80005f00 tx=0x7f7b80003950 comp rx=0 tx=0).stop 2024-05-03T18:35:43.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.850+0000 7f7ba8545b80 1 -- 172.21.15.33:0/511941387 shutdown_connections 2024-05-03T18:35:43.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.850+0000 7f7ba8545b80 1 --2- 172.21.15.33:0/511941387 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560aaa4992d0 0x560aaa49b070 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:43.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.850+0000 7f7ba8545b80 1 --2- 172.21.15.33:0/511941387 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560aaa4935d0 0x560aaa492fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:43.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.850+0000 7f7ba8545b80 1 --2- 172.21.15.33:0/511941387 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560aaa4989b0 0x560aaa49ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:43.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.850+0000 7f7ba8545b80 1 -- 172.21.15.33:0/511941387 >> 172.21.15.33:0/511941387 conn(0x560aaa40b750 msgr2=0x560aaa49c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:35:43.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.850+0000 7f7ba8545b80 1 -- 172.21.15.33:0/511941387 shutdown_connections 2024-05-03T18:35:43.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.850+0000 7f7ba8545b80 1 -- 172.21.15.33:0/511941387 wait complete. 2024-05-03T18:35:43.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.851+0000 7f7ba8545b80 1 -- start start 2024-05-03T18:35:43.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.852+0000 7f7ba8545b80 1 Processor -- start 2024-05-03T18:35:43.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.854+0000 7f7ba8545b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560aaa4935d0 0x560aaa4b42e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:43.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.854+0000 7f7ba8545b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560aaa4989b0 0x560aaa4b4820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:43.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.854+0000 7f7ba8545b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560aaa4992d0 0x560aaa4b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:43.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.854+0000 7f7ba8545b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7b7400de20 con 0x560aaa4935d0 2024-05-03T18:35:43.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.854+0000 7f7ba8545b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7b7400f290 con 0x560aaa4992d0 2024-05-03T18:35:43.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.854+0000 7f7ba8545b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f7b7400e4f0 con 0x560aaa4989b0 2024-05-03T18:35:43.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.855+0000 7f7ba8545b80 10 client.0 _ll_register_callbacks cb 0x560aaa38ccf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:35:43.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.855+0000 7f7b96000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560aaa4989b0 0x560aaa4b4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:43.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.856+0000 7f7b96000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560aaa4989b0 0x560aaa4b4820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:34302/0 (socket says 172.21.15.33:34302) 2024-05-03T18:35:43.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.856+0000 7f7b96000700 1 -- 172.21.15.33:0/339080262 learned_addr learned my addr 172.21.15.33:0/339080262 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:35:43.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.856+0000 7f7b97400700 1 --2- 172.21.15.33:0/339080262 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560aaa4992d0 0x560aaa4b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:43.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.856+0000 7f7b96000700 1 -- 172.21.15.33:0/339080262 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560aaa4992d0 msgr2=0x560aaa4b4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:35:43.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.856+0000 7f7b96000700 1 --2- 172.21.15.33:0/339080262 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560aaa4992d0 0x560aaa4b4d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:43.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.856+0000 7f7b96000700 1 -- 172.21.15.33:0/339080262 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560aaa4935d0 msgr2=0x560aaa4b42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:35:43.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.856+0000 7f7b96000700 1 --2- 172.21.15.33:0/339080262 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560aaa4935d0 0x560aaa4b42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:43.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.856+0000 7f7b96000700 1 -- 172.21.15.33:0/339080262 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560aaa5166a0 con 0x560aaa4989b0 2024-05-03T18:35:43.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.857+0000 7f7b96000700 1 --2- 172.21.15.33:0/339080262 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560aaa4989b0 0x560aaa4b4820 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x560aaa49d810 tx=0x7f7b88011bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73ed59fa77a8f111 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:35:43.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.857+0000 7f7b84c00700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-03T18:35:43.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.858+0000 7f7b84c00700 1 -- 172.21.15.33:0/339080262 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b88035060 con 0x560aaa4989b0 2024-05-03T18:35:43.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.858+0000 7f7b84c00700 1 -- 172.21.15.33:0/339080262 <== mon.1 v2:172.21.15.150:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9518+0+0 (secure 0 0 0) 0x7f7b88026650 con 0x560aaa4989b0 2024-05-03T18:35:43.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.858+0000 7f7ba8545b80 1 -- 172.21.15.33:0/339080262 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b80027070 con 0x560aaa4989b0 2024-05-03T18:35:43.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.860+0000 7f7b84c00700 1 -- 172.21.15.33:0/339080262 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f7b88026850 con 0x560aaa4989b0 2024-05-03T18:35:43.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.860+0000 7f7ba8545b80 10 client.25966 fetch_fsmap learned FSMap version 21 2024-05-03T18:35:43.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.860+0000 7f7ba8545b80 1 -- 172.21.15.33:0/339080262 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x560aaa5179e0 con 0x560aaa4989b0 2024-05-03T18:35:43.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.860+0000 7f7b84c00700 1 -- 172.21.15.33:0/339080262 <== mon.1 v2:172.21.15.150:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7b80027070 con 0x560aaa4989b0 2024-05-03T18:35:43.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.861+0000 7f7b84c00700 1 -- 172.21.15.33:0/339080262 <== mon.1 v2:172.21.15.150:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f7b8801eb20 con 0x560aaa4989b0 2024-05-03T18:35:43.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.861+0000 7f7ba8545b80 10 client.25966 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:35:43.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.861+0000 7f7ba8545b80 10 client.25966 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:35:43.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.861+0000 7f7ba8545b80 20 client.25966 populate_metadata read hostname 'smithi033' 2024-05-03T18:35:43.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.861+0000 7f7ba8545b80 1 --2- 172.21.15.33:0/339080262 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x560aaa517cb0 0x560aaa51a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:43.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.861+0000 7f7ba8545b80 4 client.25966 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-03T18:35:43.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.861+0000 7f7ba8545b80 1 -- 172.21.15.33:0/339080262 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x560aaa3f3480 con 0x560aaa517cb0 2024-05-03T18:35:43.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.862+0000 7f7b96a00700 1 --2- 172.21.15.33:0/339080262 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x560aaa517cb0 0x560aaa51a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:43.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.863+0000 7f7b96a00700 1 --2- 172.21.15.33:0/339080262 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x560aaa517cb0 0x560aaa51a0a0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a381f584c45c2d1 server_cookie=ddf0abb04f7066aa in_seq=0 out_seq=0 2024-05-03T18:35:43.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.863+0000 7f7b84c00700 10 client.25966 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:35:43.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.865+0000 7f7b84c00700 1 -- 172.21.15.33:0/339080262 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560aaa3f3480 con 0x560aaa517cb0 2024-05-03T18:35:43.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.865+0000 7f7b84c00700 10 client.25966 handle_command_reply: tid=0 2024-05-03T18:35:43.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.866+0000 7f7ba8545b80 1 client.25966 shutdown 2024-05-03T18:35:43.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.866+0000 7f7ba8545b80 1 -- 172.21.15.33:0/339080262 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560aaa4989b0 msgr2=0x560aaa4b4820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:35:43.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.866+0000 7f7ba8545b80 1 --2- 172.21.15.33:0/339080262 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560aaa4989b0 0x560aaa4b4820 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x560aaa49d810 tx=0x7f7b88011bb0 comp rx=0 tx=0).stop 2024-05-03T18:35:43.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.867+0000 7f7ba8545b80 1 -- 172.21.15.33:0/339080262 shutdown_connections 2024-05-03T18:35:43.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.867+0000 7f7ba8545b80 1 --2- 172.21.15.33:0/339080262 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x560aaa517cb0 0x560aaa51a0a0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:43.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.867+0000 7f7ba8545b80 1 --2- 172.21.15.33:0/339080262 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560aaa4992d0 0x560aaa4b4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:43.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.867+0000 7f7ba8545b80 1 --2- 172.21.15.33:0/339080262 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560aaa4935d0 0x560aaa4b42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:43.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.867+0000 7f7ba8545b80 1 --2- 172.21.15.33:0/339080262 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560aaa4989b0 0x560aaa4b4820 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:43.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.867+0000 7f7ba8545b80 1 -- 172.21.15.33:0/339080262 >> 172.21.15.33:0/339080262 conn(0x560aaa40b750 msgr2=0x560aaa49a9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:35:43.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.867+0000 7f7b84c00700 0 client.25966 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:35:43.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.868+0000 7f7ba8545b80 1 -- 172.21.15.33:0/339080262 shutdown_connections 2024-05-03T18:35:43.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.869+0000 7f7ba8545b80 1 -- 172.21.15.33:0/339080262 wait complete. 2024-05-03T18:35:43.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.871+0000 7f7ba8545b80 20 client.25966 trim_cache size 0 max 16384 2024-05-03T18:35:43.874 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-03T18:35:43.874 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-05-03T18:35:43.874 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "8326764e-fcfb-420d-8635-8531184274df", 2024-05-03T18:35:43.874 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-05-03T18:35:43.874 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-03T18:35:43.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.874+0000 7f7b9dc00700 1 -- 172.21.15.33:0/3964409441 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f7b7c0b2c00 msgr2=0x7f7b7c0b50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:35:43.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.874+0000 7f7b9dc00700 1 --2- 172.21.15.33:0/3964409441 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f7b7c0b2c00 0x7f7b7c0b50c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7b98106b70 tx=0x7f7b8800b410 comp rx=0 tx=0).stop 2024-05-03T18:35:43.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.874+0000 7f7b9dc00700 1 -- 172.21.15.33:0/3964409441 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b980756d0 msgr2=0x7f7b9810a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:35:43.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.874+0000 7f7b9dc00700 1 --2- 172.21.15.33:0/3964409441 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b980756d0 0x7f7b9810a290 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7b9000ba50 tx=0x7f7b9000bd60 comp rx=0 tx=0).stop 2024-05-03T18:35:43.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.875+0000 7f7b9dc00700 1 -- 172.21.15.33:0/3964409441 shutdown_connections 2024-05-03T18:35:43.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.875+0000 7f7b9dc00700 1 --2- 172.21.15.33:0/3964409441 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f7b7c0b2c00 0x7f7b7c0b50c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:43.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.875+0000 7f7b9dc00700 1 --2- 172.21.15.33:0/3964409441 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b9810fea0 0x7f7b98105400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:43.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.875+0000 7f7b9dc00700 1 --2- 172.21.15.33:0/3964409441 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b980756d0 0x7f7b9810a290 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:43.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.875+0000 7f7b9dc00700 1 --2- 172.21.15.33:0/3964409441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b98075ff0 0x7f7b9810a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:43.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.875+0000 7f7b9dc00700 1 -- 172.21.15.33:0/3964409441 >> 172.21.15.33:0/3964409441 conn(0x7f7b9807a600 msgr2=0x7f7b98100670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:35:43.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.875+0000 7f7b9dc00700 1 -- 172.21.15.33:0/3964409441 shutdown_connections 2024-05-03T18:35:43.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:43.875+0000 7f7b9dc00700 1 -- 172.21.15.33:0/3964409441 wait complete. 2024-05-03T18:35:44.374 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:35:44 smithi033 ceph-mon[28330]: pgmap v263: 129 pgs: 129 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 8.3 MiB/s wr, 777 op/s 2024-05-03T18:35:44.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:35:44 smithi150 ceph-mon[32492]: pgmap v263: 129 pgs: 129 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 8.3 MiB/s wr, 777 op/s 2024-05-03T18:35:44.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:35:44 smithi099 ceph-mon[31658]: pgmap v263: 129 pgs: 129 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 8.3 MiB/s wr, 777 op/s 2024-05-03T18:35:44.735 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-03T18:35:45.112 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 270.0 s, 616.9 tps, lat 161.290 ms stddev 46.242 2024-05-03T18:35:45.378 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:35:45 smithi033 ceph-mon[28330]: scrub queued for path: / 2024-05-03T18:35:45.379 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:35:45 smithi033 ceph-mon[28330]: scrub summary: idle+waiting paths [/] 2024-05-03T18:35:45.379 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:35:45 smithi033 ceph-mon[28330]: scrub summary: active paths [/] 2024-05-03T18:35:45.379 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:35:45 smithi033 ceph-mon[28330]: scrub summary: idle+waiting paths [/] 2024-05-03T18:35:45.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:35:45 smithi150 ceph-mon[32492]: scrub queued for path: / 2024-05-03T18:35:45.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:35:45 smithi150 ceph-mon[32492]: scrub summary: idle+waiting paths [/] 2024-05-03T18:35:45.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:35:45 smithi150 ceph-mon[32492]: scrub summary: active paths [/] 2024-05-03T18:35:45.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:35:45 smithi150 ceph-mon[32492]: scrub summary: idle+waiting paths [/] 2024-05-03T18:35:45.526 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:35:45.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:35:45 smithi099 ceph-mon[31658]: scrub queued for path: / 2024-05-03T18:35:45.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:35:45 smithi099 ceph-mon[31658]: scrub summary: idle+waiting paths [/] 2024-05-03T18:35:45.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:35:45 smithi099 ceph-mon[31658]: scrub summary: active paths [/] 2024-05-03T18:35:45.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:35:45 smithi099 ceph-mon[31658]: scrub summary: idle+waiting paths [/] 2024-05-03T18:35:46.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:35:46 smithi033 ceph-mon[28330]: pgmap v264: 129 pgs: 129 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 5.8 MiB/s wr, 535 op/s 2024-05-03T18:35:46.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:35:46 smithi150 ceph-mon[32492]: pgmap v264: 129 pgs: 129 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 5.8 MiB/s wr, 535 op/s 2024-05-03T18:35:46.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:35:46 smithi099 ceph-mon[31658]: pgmap v264: 129 pgs: 129 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 5.8 MiB/s wr, 535 op/s 2024-05-03T18:35:47.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:35:47 smithi150 ceph-mon[32492]: scrub summary: idle 2024-05-03T18:35:47.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:35:47 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:35:47.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:35:47 smithi033 ceph-mon[28330]: scrub summary: idle 2024-05-03T18:35:47.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:35:47 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:35:47.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:35:47 smithi099 ceph-mon[31658]: scrub summary: idle 2024-05-03T18:35:47.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:35:47 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:35:48.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:35:48 smithi150 ceph-mon[32492]: pgmap v265: 129 pgs: 129 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 5.8 MiB/s wr, 536 op/s 2024-05-03T18:35:48.446 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:35:48 smithi033 ceph-mon[28330]: pgmap v265: 129 pgs: 129 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 5.8 MiB/s wr, 536 op/s 2024-05-03T18:35:48.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:35:48 smithi099 ceph-mon[31658]: pgmap v265: 129 pgs: 129 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 5.8 MiB/s wr, 536 op/s 2024-05-03T18:35:49.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:35:49 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:35:49.384 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:35:49 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:35:49.384 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:35:49 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:35:49.486 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:35:49 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:35:49.486 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:35:49 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:35:49.486 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:35:49 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:35:49.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.501+0000 7f2a77c00700 1 -- 172.21.15.33:0/213002857 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2a70075ff0 msgr2=0x7f2a7010f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:35:49.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.501+0000 7f2a77c00700 1 --2- 172.21.15.33:0/213002857 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2a70075ff0 0x7f2a7010f9c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c009990 tx=0x7f2a5c009ca0 comp rx=0 tx=0).stop 2024-05-03T18:35:49.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.501+0000 7f2a77c00700 1 -- 172.21.15.33:0/213002857 shutdown_connections 2024-05-03T18:35:49.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.501+0000 7f2a77c00700 1 --2- 172.21.15.33:0/213002857 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a7010ff00 0x7f2a701122f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:49.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.501+0000 7f2a77c00700 1 --2- 172.21.15.33:0/213002857 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a700756d0 0x7f2a70075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:49.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.501+0000 7f2a77c00700 1 --2- 172.21.15.33:0/213002857 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2a70075ff0 0x7f2a7010f9c0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:49.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.501+0000 7f2a77c00700 1 -- 172.21.15.33:0/213002857 >> 172.21.15.33:0/213002857 conn(0x7f2a7007a600 msgr2=0x7f2a7007aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:35:49.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.502+0000 7f2a77c00700 1 -- 172.21.15.33:0/213002857 shutdown_connections 2024-05-03T18:35:49.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.502+0000 7f2a77c00700 1 -- 172.21.15.33:0/213002857 wait complete. 2024-05-03T18:35:49.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.504+0000 7f2a77c00700 1 Processor -- start 2024-05-03T18:35:49.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.504+0000 7f2a77c00700 1 -- start start 2024-05-03T18:35:49.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.505+0000 7f2a77c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a700756d0 0x7f2a701a17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:49.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.505+0000 7f2a77c00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a70075ff0 0x7f2a701a1d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:49.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.506+0000 7f2a77c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2a7010ff00 0x7f2a7019b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:49.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.506+0000 7f2a77c00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2a701157f0 con 0x7f2a70075ff0 2024-05-03T18:35:49.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.506+0000 7f2a77c00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2a70115670 con 0x7f2a700756d0 2024-05-03T18:35:49.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.506+0000 7f2a77c00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2a70115970 con 0x7f2a7010ff00 2024-05-03T18:35:49.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.506+0000 7f2a75200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a700756d0 0x7f2a701a17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:49.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.506+0000 7f2a75200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a700756d0 0x7f2a701a17e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:39028/0 (socket says 172.21.15.33:39028) 2024-05-03T18:35:49.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.506+0000 7f2a75200700 1 -- 172.21.15.33:0/3942553141 learned_addr learned my addr 172.21.15.33:0/3942553141 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:35:49.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.507+0000 7f2a6fe00700 1 --2- 172.21.15.33:0/3942553141 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a70075ff0 0x7f2a701a1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:49.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.507+0000 7f2a75200700 1 -- 172.21.15.33:0/3942553141 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2a7010ff00 msgr2=0x7f2a7019b9f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-03T18:35:49.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.507+0000 7f2a75200700 1 --2- 172.21.15.33:0/3942553141 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2a7010ff00 0x7f2a7019b9f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:49.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.507+0000 7f2a75200700 1 -- 172.21.15.33:0/3942553141 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a70075ff0 msgr2=0x7f2a701a1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:35:49.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.507+0000 7f2a75200700 1 --2- 172.21.15.33:0/3942553141 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a70075ff0 0x7f2a701a1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:49.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.507+0000 7f2a75200700 1 -- 172.21.15.33:0/3942553141 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a7019c150 con 0x7f2a700756d0 2024-05-03T18:35:49.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.508+0000 7f2a75200700 1 --2- 172.21.15.33:0/3942553141 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a700756d0 0x7f2a701a17e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2a6400c9b0 tx=0x7f2a6400cd70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:35:49.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.510+0000 7f2a6d600700 1 -- 172.21.15.33:0/3942553141 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a64007b00 con 0x7f2a700756d0 2024-05-03T18:35:49.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.510+0000 7f2a6d600700 1 -- 172.21.15.33:0/3942553141 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2a64007ca0 con 0x7f2a700756d0 2024-05-03T18:35:49.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.510+0000 7f2a77c00700 1 -- 172.21.15.33:0/3942553141 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a7019c440 con 0x7f2a700756d0 2024-05-03T18:35:49.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.510+0000 7f2a6d600700 1 -- 172.21.15.33:0/3942553141 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a64018660 con 0x7f2a700756d0 2024-05-03T18:35:49.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.510+0000 7f2a77c00700 1 -- 172.21.15.33:0/3942553141 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a701a86b0 con 0x7f2a700756d0 2024-05-03T18:35:49.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.520+0000 7f2a8250ab80 1 Processor -- start 2024-05-03T18:35:49.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.521+0000 7f2a8250ab80 1 -- start start 2024-05-03T18:35:49.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.521+0000 7f2a6d600700 1 -- 172.21.15.33:0/3942553141 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f2a6400f450 con 0x7f2a700756d0 2024-05-03T18:35:49.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.522+0000 7f2a8250ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560ab0986140 0x560ab0986520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:49.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.522+0000 7f2a8250ab80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ab0986a60 0x560ab0980740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:49.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.522+0000 7f2a8250ab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560ab0980d60 0x560ab09811e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:49.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.522+0000 7f2a8250ab80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560ab08caa20 con 0x560ab0986a60 2024-05-03T18:35:49.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.522+0000 7f2a8250ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x560ab0868760 con 0x560ab0986140 2024-05-03T18:35:49.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.522+0000 7f2a8250ab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x560ab08c69e0 con 0x560ab0980d60 2024-05-03T18:35:49.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.523+0000 7f2a75c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560ab0986140 0x560ab0986520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:49.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.524+0000 7f2a75c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560ab0986140 0x560ab0986520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:39030/0 (socket says 172.21.15.33:39030) 2024-05-03T18:35:49.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.524+0000 7f2a75c00700 1 -- 172.21.15.33:0/3674257421 learned_addr learned my addr 172.21.15.33:0/3674257421 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:35:49.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.524+0000 7f2a75200700 1 --2- 172.21.15.33:0/3674257421 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ab0986a60 0x560ab0980740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:49.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.524+0000 7f2a75c00700 1 -- 172.21.15.33:0/3674257421 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560ab0980d60 msgr2=0x560ab09811e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-03T18:35:49.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.524+0000 7f2a75c00700 1 --2- 172.21.15.33:0/3674257421 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560ab0980d60 0x560ab09811e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:49.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.524+0000 7f2a75c00700 1 -- 172.21.15.33:0/3674257421 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ab0986a60 msgr2=0x560ab0980740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:35:49.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.524+0000 7f2a75c00700 1 --2- 172.21.15.33:0/3674257421 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ab0986a60 0x560ab0980740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:49.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.524+0000 7f2a75c00700 1 -- 172.21.15.33:0/3674257421 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ab08178d0 con 0x560ab0986140 2024-05-03T18:35:49.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.525+0000 7f2a75200700 1 --2- 172.21.15.33:0/3674257421 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ab0986a60 0x560ab0980740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:35:49.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.526+0000 7f2a6d600700 1 --2- 172.21.15.33:0/3942553141 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f2a580b2aa0 0x7f2a580b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:49.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.526+0000 7f2a6d600700 1 -- 172.21.15.33:0/3942553141 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9518+0+0 (secure 0 0 0) 0x7f2a640e29b0 con 0x7f2a700756d0 2024-05-03T18:35:49.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.526+0000 7f2a75c00700 1 --2- 172.21.15.33:0/3674257421 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560ab0986140 0x560ab0986520 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2a68008f40 tx=0x7f2a6800d3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67d0d3a7dead97a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:35:49.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.527+0000 7f2a75200700 1 --2- 172.21.15.33:0/3942553141 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f2a580b2aa0 0x7f2a580b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:49.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.527+0000 7f2a62000700 1 -- 172.21.15.33:0/3674257421 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a68013420 con 0x560ab0986140 2024-05-03T18:35:49.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.527+0000 7f2a62000700 1 -- 172.21.15.33:0/3674257421 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2a68018070 con 0x560ab0986140 2024-05-03T18:35:49.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.528+0000 7f2a62000700 1 -- 172.21.15.33:0/3674257421 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a68013420 con 0x560ab0986140 2024-05-03T18:35:49.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.529+0000 7f2a8250ab80 1 -- 172.21.15.33:0/3674257421 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560ab0986140 msgr2=0x560ab0986520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:35:49.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.529+0000 7f2a75200700 1 --2- 172.21.15.33:0/3942553141 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f2a580b2aa0 0x7f2a580b4f60 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2a640dfdd0 tx=0x7f2a64000bc0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:35:49.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.529+0000 7f2a8250ab80 1 --2- 172.21.15.33:0/3674257421 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560ab0986140 0x560ab0986520 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2a68008f40 tx=0x7f2a6800d3c0 comp rx=0 tx=0).stop 2024-05-03T18:35:49.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.530+0000 7f2a8250ab80 1 -- 172.21.15.33:0/3674257421 shutdown_connections 2024-05-03T18:35:49.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.530+0000 7f2a8250ab80 1 --2- 172.21.15.33:0/3674257421 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560ab0980d60 0x560ab09811e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:49.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.530+0000 7f2a8250ab80 1 --2- 172.21.15.33:0/3674257421 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560ab0986140 0x560ab0986520 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:49.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.530+0000 7f2a8250ab80 1 --2- 172.21.15.33:0/3674257421 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ab0986a60 0x560ab0980740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:49.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.530+0000 7f2a8250ab80 1 -- 172.21.15.33:0/3674257421 >> 172.21.15.33:0/3674257421 conn(0x560ab08f06d0 msgr2=0x560ab0972900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:35:49.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.531+0000 7f2a8250ab80 1 -- 172.21.15.33:0/3674257421 shutdown_connections 2024-05-03T18:35:49.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.531+0000 7f2a8250ab80 1 -- 172.21.15.33:0/3674257421 wait complete. 2024-05-03T18:35:49.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.532+0000 7f2a8250ab80 1 -- start start 2024-05-03T18:35:49.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.533+0000 7f2a8250ab80 1 Processor -- start 2024-05-03T18:35:49.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.534+0000 7f2a8250ab80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ab0980d60 0x560ab098c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:49.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.534+0000 7f2a8250ab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560ab0986140 0x560ab098cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:49.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.534+0000 7f2a8250ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560ab0986a60 0x560ab0989b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:49.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.534+0000 7f2a8250ab80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560ab098d510 con 0x560ab0980d60 2024-05-03T18:35:49.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.534+0000 7f2a8250ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x560ab0830b20 con 0x560ab0986a60 2024-05-03T18:35:49.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.534+0000 7f2a8250ab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x560ab098d690 con 0x560ab0986140 2024-05-03T18:35:49.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.534+0000 7f2a8250ab80 10 client.0 _ll_register_callbacks cb 0x560ab0871c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:35:49.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.534+0000 7f2a6fe00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560ab0986140 0x560ab098cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:49.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.535+0000 7f2a75c00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ab0980d60 0x560ab098c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:49.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.535+0000 7f2a75c00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ab0980d60 0x560ab098c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38460/0 (socket says 172.21.15.33:38460) 2024-05-03T18:35:49.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.535+0000 7f2a75c00700 1 -- 172.21.15.33:0/4009120019 learned_addr learned my addr 172.21.15.33:0/4009120019 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:35:49.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.535+0000 7f2a75c00700 1 --2- 172.21.15.33:0/4009120019 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560ab0986a60 0x560ab0989b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:49.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.535+0000 7f2a75c00700 1 -- 172.21.15.33:0/4009120019 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560ab0986140 msgr2=0x560ab098cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:35:49.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.536+0000 7f2a75c00700 1 --2- 172.21.15.33:0/4009120019 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560ab0986140 0x560ab098cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:49.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.536+0000 7f2a75c00700 1 -- 172.21.15.33:0/4009120019 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560ab0986a60 msgr2=0x560ab0989b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:35:49.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.536+0000 7f2a75c00700 1 --2- 172.21.15.33:0/4009120019 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560ab0986a60 0x560ab0989b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:49.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.536+0000 7f2a75c00700 1 -- 172.21.15.33:0/4009120019 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ab0890420 con 0x560ab0980d60 2024-05-03T18:35:49.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.537+0000 7f2a75c00700 1 --2- 172.21.15.33:0/4009120019 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ab0980d60 0x560ab098c5c0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f2a680085a0 tx=0x7f2a68013f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f6726793c826205 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:35:49.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.537+0000 7f2a56a00700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:35:49.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.538+0000 7f2a56a00700 1 -- 172.21.15.33:0/4009120019 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a68013a60 con 0x560ab0980d60 2024-05-03T18:35:49.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.538+0000 7f2a56a00700 1 -- 172.21.15.33:0/4009120019 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9518+0+0 (secure 0 0 0) 0x7f2a68013c00 con 0x560ab0980d60 2024-05-03T18:35:49.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.538+0000 7f2a8250ab80 1 -- 172.21.15.33:0/4009120019 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a68018070 con 0x560ab0980d60 2024-05-03T18:35:49.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.538+0000 7f2a56a00700 1 -- 172.21.15.33:0/4009120019 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f2a680050c0 con 0x560ab0980d60 2024-05-03T18:35:49.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.538+0000 7f2a8250ab80 10 client.16260 fetch_fsmap learned FSMap version 21 2024-05-03T18:35:49.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.538+0000 7f2a8250ab80 1 -- 172.21.15.33:0/4009120019 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x560ab0997c20 con 0x560ab0980d60 2024-05-03T18:35:49.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.539+0000 7f2a56a00700 1 -- 172.21.15.33:0/4009120019 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2a68018070 con 0x560ab0980d60 2024-05-03T18:35:49.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.540+0000 7f2a56a00700 1 -- 172.21.15.33:0/4009120019 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f2a6801d490 con 0x560ab0980d60 2024-05-03T18:35:49.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.540+0000 7f2a8250ab80 10 client.16260 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:35:49.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.540+0000 7f2a8250ab80 10 client.16260 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:35:49.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.540+0000 7f2a8250ab80 20 client.16260 populate_metadata read hostname 'smithi033' 2024-05-03T18:35:49.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.540+0000 7f2a8250ab80 1 --2- 172.21.15.33:0/4009120019 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x560ab099a150 0x560ab099c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:49.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.540+0000 7f2a8250ab80 4 client.16260 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:35:49.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.540+0000 7f2a8250ab80 1 -- 172.21.15.33:0/4009120019 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560ab0944ff0 con 0x560ab099a150 2024-05-03T18:35:49.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.541+0000 7f2a6fe00700 1 --2- 172.21.15.33:0/4009120019 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x560ab099a150 0x560ab099c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:49.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.541+0000 7f2a6fe00700 1 --2- 172.21.15.33:0/4009120019 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x560ab099a150 0x560ab099c540 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94b86fb55e7c8f0f server_cookie=88f180312a6a792d in_seq=0 out_seq=0 2024-05-03T18:35:49.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.542+0000 7f2a56a00700 10 client.16260 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:35:49.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.543+0000 7f2a56a00700 1 -- 172.21.15.33:0/4009120019 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560ab0944ff0 con 0x560ab099a150 2024-05-03T18:35:49.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.543+0000 7f2a56a00700 10 client.16260 handle_command_reply: tid=0 2024-05-03T18:35:49.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.543+0000 7f2a8250ab80 1 client.16260 shutdown 2024-05-03T18:35:49.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.544+0000 7f2a8250ab80 1 -- 172.21.15.33:0/4009120019 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ab0980d60 msgr2=0x560ab098c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:35:49.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.544+0000 7f2a8250ab80 1 --2- 172.21.15.33:0/4009120019 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ab0980d60 0x560ab098c5c0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f2a680085a0 tx=0x7f2a68013f10 comp rx=0 tx=0).stop 2024-05-03T18:35:49.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:35:49 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:35:49.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:35:49 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:35:49.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:35:49 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:35:49.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.545+0000 7f2a8250ab80 1 -- 172.21.15.33:0/4009120019 shutdown_connections 2024-05-03T18:35:49.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.545+0000 7f2a8250ab80 1 --2- 172.21.15.33:0/4009120019 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x560ab099a150 0x560ab099c540 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:49.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.545+0000 7f2a8250ab80 1 --2- 172.21.15.33:0/4009120019 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560ab0986a60 0x560ab0989b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:49.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.545+0000 7f2a8250ab80 1 --2- 172.21.15.33:0/4009120019 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ab0980d60 0x560ab098c5c0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:49.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.545+0000 7f2a8250ab80 1 --2- 172.21.15.33:0/4009120019 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560ab0986140 0x560ab098cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:49.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.545+0000 7f2a8250ab80 1 -- 172.21.15.33:0/4009120019 >> 172.21.15.33:0/4009120019 conn(0x560ab08f06d0 msgr2=0x560ab096d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:35:49.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.545+0000 7f2a56a00700 0 client.16260 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:35:49.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.545+0000 7f2a8250ab80 1 -- 172.21.15.33:0/4009120019 shutdown_connections 2024-05-03T18:35:49.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.545+0000 7f2a8250ab80 1 -- 172.21.15.33:0/4009120019 wait complete. 2024-05-03T18:35:49.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.545+0000 7f2a8250ab80 20 client.16260 trim_cache size 0 max 16384 2024-05-03T18:35:49.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.584+0000 7f2a8250ab80 1 Processor -- start 2024-05-03T18:35:49.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.584+0000 7f2a8250ab80 1 -- start start 2024-05-03T18:35:49.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.584+0000 7f2a8250ab80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ab0980d60 0x560ab09a1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:49.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.584+0000 7f2a8250ab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560ab0986140 0x560ab09a20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:49.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.584+0000 7f2a8250ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560ab0986a60 0x560ab09929a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:49.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.584+0000 7f2a8250ab80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2a5000cfa0 con 0x560ab0980d60 2024-05-03T18:35:49.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.584+0000 7f2a8250ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2a5000f840 con 0x560ab0986a60 2024-05-03T18:35:49.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.584+0000 7f2a8250ab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2a5000d630 con 0x560ab0986140 2024-05-03T18:35:49.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.585+0000 7f2a6fe00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560ab0986140 0x560ab09a20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:49.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.585+0000 7f2a75c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560ab0986a60 0x560ab09929a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:49.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.586+0000 7f2a75c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560ab0986a60 0x560ab09929a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:39056/0 (socket says 172.21.15.33:39056) 2024-05-03T18:35:49.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.586+0000 7f2a75c00700 1 -- 172.21.15.33:0/2879717377 learned_addr learned my addr 172.21.15.33:0/2879717377 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:35:49.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.586+0000 7f2a75c00700 1 --2- 172.21.15.33:0/2879717377 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ab0980d60 0x560ab09a1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:49.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.588+0000 7f2a75c00700 1 -- 172.21.15.33:0/2879717377 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560ab0986140 msgr2=0x560ab09a20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:35:49.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.588+0000 7f2a75c00700 1 --2- 172.21.15.33:0/2879717377 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560ab0986140 0x560ab09a20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:49.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.588+0000 7f2a75c00700 1 -- 172.21.15.33:0/2879717377 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560ab0986a60 msgr2=0x560ab09929a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:35:49.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.588+0000 7f2a75c00700 1 --2- 172.21.15.33:0/2879717377 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560ab0986a60 0x560ab09929a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:49.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.588+0000 7f2a75c00700 1 -- 172.21.15.33:0/2879717377 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ab098a160 con 0x560ab0980d60 2024-05-03T18:35:49.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.589+0000 7f2a75c00700 1 --2- 172.21.15.33:0/2879717377 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ab0980d60 0x560ab09a1b70 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f2a68008e60 tx=0x7f2a6801c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86c0d02c1e45067c server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:35:49.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.590+0000 7f2a61600700 1 -- 172.21.15.33:0/2879717377 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a680050c0 con 0x560ab0980d60 2024-05-03T18:35:49.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.590+0000 7f2a61600700 1 -- 172.21.15.33:0/2879717377 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2a68018040 con 0x560ab0980d60 2024-05-03T18:35:49.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.590+0000 7f2a61600700 1 -- 172.21.15.33:0/2879717377 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a6802ae60 con 0x560ab0980d60 2024-05-03T18:35:49.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.591+0000 7f2a8250ab80 1 -- 172.21.15.33:0/2879717377 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ab0980d60 msgr2=0x560ab09a1b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:35:49.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.591+0000 7f2a8250ab80 1 --2- 172.21.15.33:0/2879717377 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ab0980d60 0x560ab09a1b70 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f2a68008e60 tx=0x7f2a6801c390 comp rx=0 tx=0).stop 2024-05-03T18:35:49.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.592+0000 7f2a8250ab80 1 -- 172.21.15.33:0/2879717377 shutdown_connections 2024-05-03T18:35:49.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.592+0000 7f2a8250ab80 1 --2- 172.21.15.33:0/2879717377 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560ab0986a60 0x560ab09929a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:49.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.592+0000 7f2a8250ab80 1 --2- 172.21.15.33:0/2879717377 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ab0980d60 0x560ab09a1b70 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:49.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.592+0000 7f2a8250ab80 1 --2- 172.21.15.33:0/2879717377 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560ab0986140 0x560ab09a20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:49.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.592+0000 7f2a8250ab80 1 -- 172.21.15.33:0/2879717377 >> 172.21.15.33:0/2879717377 conn(0x560ab08f06d0 msgr2=0x560ab0988250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:35:49.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.593+0000 7f2a8250ab80 1 -- 172.21.15.33:0/2879717377 shutdown_connections 2024-05-03T18:35:49.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.593+0000 7f2a8250ab80 1 -- 172.21.15.33:0/2879717377 wait complete. 2024-05-03T18:35:49.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.594+0000 7f2a8250ab80 1 -- start start 2024-05-03T18:35:49.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.594+0000 7f2a8250ab80 1 Processor -- start 2024-05-03T18:35:49.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.595+0000 7f2a8250ab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560ab0980d60 0x560ab0992ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:49.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.596+0000 7f2a8250ab80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ab0986140 0x560ab098f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:49.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.596+0000 7f2a8250ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560ab0986a60 0x560ab098fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:49.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.596+0000 7f2a8250ab80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2a5000dd00 con 0x560ab0986140 2024-05-03T18:35:49.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.596+0000 7f2a8250ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2a5000f170 con 0x560ab0986a60 2024-05-03T18:35:49.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.596+0000 7f2a8250ab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2a5000e3d0 con 0x560ab0980d60 2024-05-03T18:35:49.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.596+0000 7f2a8250ab80 10 client.0 _ll_register_callbacks cb 0x560ab0871c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:35:49.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.596+0000 7f2a75c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560ab0980d60 0x560ab0992ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:49.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.596+0000 7f2a75c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560ab0980d60 0x560ab0992ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:51428/0 (socket says 172.21.15.33:51428) 2024-05-03T18:35:49.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.596+0000 7f2a75c00700 1 -- 172.21.15.33:0/873110469 learned_addr learned my addr 172.21.15.33:0/873110469 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:35:49.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.597+0000 7f2a75c00700 1 --2- 172.21.15.33:0/873110469 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560ab0986a60 0x560ab098fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:49.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.597+0000 7f2a75c00700 1 -- 172.21.15.33:0/873110469 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560ab0986a60 msgr2=0x560ab098fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:35:49.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.597+0000 7f2a75c00700 1 --2- 172.21.15.33:0/873110469 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560ab0986a60 0x560ab098fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:49.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.597+0000 7f2a75c00700 1 -- 172.21.15.33:0/873110469 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ab0986140 msgr2=0x560ab098f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:35:49.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.597+0000 7f2a6fe00700 1 --2- 172.21.15.33:0/873110469 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ab0986140 0x560ab098f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:49.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.597+0000 7f2a75c00700 1 --2- 172.21.15.33:0/873110469 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ab0986140 0x560ab098f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:49.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.597+0000 7f2a75c00700 1 -- 172.21.15.33:0/873110469 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ab0990730 con 0x560ab0980d60 2024-05-03T18:35:49.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.597+0000 7f2a6fe00700 1 --2- 172.21.15.33:0/873110469 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ab0986140 0x560ab098f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:35:49.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.598+0000 7f2a75c00700 1 --2- 172.21.15.33:0/873110469 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560ab0980d60 0x560ab0992ea0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2a6800dea0 tx=0x7f2a68000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28897d0212b6d575 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:35:49.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.598+0000 7f2a56a00700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-03T18:35:49.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.599+0000 7f2a56a00700 1 -- 172.21.15.33:0/873110469 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a6802ae10 con 0x560ab0980d60 2024-05-03T18:35:49.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.599+0000 7f2a56a00700 1 -- 172.21.15.33:0/873110469 <== mon.1 v2:172.21.15.150:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9518+0+0 (secure 0 0 0) 0x7f2a680295b0 con 0x560ab0980d60 2024-05-03T18:35:49.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.599+0000 7f2a8250ab80 1 -- 172.21.15.33:0/873110469 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a68018040 con 0x560ab0980d60 2024-05-03T18:35:49.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.600+0000 7f2a56a00700 1 -- 172.21.15.33:0/873110469 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f2a6803e050 con 0x560ab0980d60 2024-05-03T18:35:49.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.600+0000 7f2a8250ab80 10 client.25978 fetch_fsmap learned FSMap version 21 2024-05-03T18:35:49.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.600+0000 7f2a8250ab80 1 -- 172.21.15.33:0/873110469 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x560ab0a05740 con 0x560ab0980d60 2024-05-03T18:35:49.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.600+0000 7f2a56a00700 1 -- 172.21.15.33:0/873110469 <== mon.1 v2:172.21.15.150:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2a68018040 con 0x560ab0980d60 2024-05-03T18:35:49.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.601+0000 7f2a56a00700 1 -- 172.21.15.33:0/873110469 <== mon.1 v2:172.21.15.150:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f2a6801d490 con 0x560ab0980d60 2024-05-03T18:35:49.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.601+0000 7f2a8250ab80 10 client.25978 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:35:49.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.601+0000 7f2a8250ab80 10 client.25978 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:35:49.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.601+0000 7f2a8250ab80 20 client.25978 populate_metadata read hostname 'smithi033' 2024-05-03T18:35:49.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.601+0000 7f2a8250ab80 1 --2- 172.21.15.33:0/873110469 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x560ab0a05a80 0x560ab0a07ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:49.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.601+0000 7f2a8250ab80 4 client.25978 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-03T18:35:49.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.602+0000 7f2a8250ab80 1 -- 172.21.15.33:0/873110469 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560ab08d8400 con 0x560ab0a05a80 2024-05-03T18:35:49.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.603+0000 7f2a6fe00700 1 --2- 172.21.15.33:0/873110469 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x560ab0a05a80 0x560ab0a07ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:49.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.605+0000 7f2a6fe00700 1 --2- 172.21.15.33:0/873110469 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x560ab0a05a80 0x560ab0a07ea0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa2121c8c1e171cf server_cookie=d3da14d3d5b5eb14 in_seq=0 out_seq=0 2024-05-03T18:35:49.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.605+0000 7f2a56a00700 10 client.25978 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:35:49.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.606+0000 7f2a56a00700 1 -- 172.21.15.33:0/873110469 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x560ab08d8400 con 0x560ab0a05a80 2024-05-03T18:35:49.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.606+0000 7f2a56a00700 10 client.25978 handle_command_reply: tid=0 2024-05-03T18:35:49.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.606+0000 7f2a8250ab80 1 client.25978 shutdown 2024-05-03T18:35:49.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.607+0000 7f2a8250ab80 1 -- 172.21.15.33:0/873110469 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560ab0980d60 msgr2=0x560ab0992ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:35:49.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.607+0000 7f2a8250ab80 1 --2- 172.21.15.33:0/873110469 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560ab0980d60 0x560ab0992ea0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2a6800dea0 tx=0x7f2a68000f40 comp rx=0 tx=0).stop 2024-05-03T18:35:49.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.607+0000 7f2a8250ab80 1 -- 172.21.15.33:0/873110469 shutdown_connections 2024-05-03T18:35:49.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.607+0000 7f2a8250ab80 1 --2- 172.21.15.33:0/873110469 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x560ab0a05a80 0x560ab0a07ea0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:49.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.607+0000 7f2a8250ab80 1 --2- 172.21.15.33:0/873110469 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560ab0986a60 0x560ab098fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:49.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.607+0000 7f2a8250ab80 1 --2- 172.21.15.33:0/873110469 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560ab0980d60 0x560ab0992ea0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:49.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.607+0000 7f2a8250ab80 1 --2- 172.21.15.33:0/873110469 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ab0986140 0x560ab098f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:49.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.607+0000 7f2a8250ab80 1 -- 172.21.15.33:0/873110469 >> 172.21.15.33:0/873110469 conn(0x560ab08f06d0 msgr2=0x560ab0986e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:35:49.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.607+0000 7f2a56a00700 0 client.25978 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:35:49.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.608+0000 7f2a8250ab80 1 -- 172.21.15.33:0/873110469 shutdown_connections 2024-05-03T18:35:49.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.608+0000 7f2a8250ab80 1 -- 172.21.15.33:0/873110469 wait complete. 2024-05-03T18:35:49.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.609+0000 7f2a8250ab80 20 client.25978 trim_cache size 0 max 16384 2024-05-03T18:35:49.610 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-03T18:35:49.610 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-05-03T18:35:49.610 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-05-03T18:35:49.610 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-03T18:35:49.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.610+0000 7f2a77c00700 1 -- 172.21.15.33:0/3942553141 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f2a580b2aa0 msgr2=0x7f2a580b4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:35:49.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.610+0000 7f2a77c00700 1 --2- 172.21.15.33:0/3942553141 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f2a580b2aa0 0x7f2a580b4f60 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2a640dfdd0 tx=0x7f2a64000bc0 comp rx=0 tx=0).stop 2024-05-03T18:35:49.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.610+0000 7f2a77c00700 1 -- 172.21.15.33:0/3942553141 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a700756d0 msgr2=0x7f2a701a17e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:35:49.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.610+0000 7f2a77c00700 1 --2- 172.21.15.33:0/3942553141 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a700756d0 0x7f2a701a17e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2a6400c9b0 tx=0x7f2a6400cd70 comp rx=0 tx=0).stop 2024-05-03T18:35:49.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.610+0000 7f2a77c00700 1 -- 172.21.15.33:0/3942553141 shutdown_connections 2024-05-03T18:35:49.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.610+0000 7f2a77c00700 1 --2- 172.21.15.33:0/3942553141 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2a7010ff00 0x7f2a7019b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:49.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.610+0000 7f2a77c00700 1 --2- 172.21.15.33:0/3942553141 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f2a580b2aa0 0x7f2a580b4f60 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:49.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.610+0000 7f2a77c00700 1 --2- 172.21.15.33:0/3942553141 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a700756d0 0x7f2a701a17e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:49.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.610+0000 7f2a77c00700 1 --2- 172.21.15.33:0/3942553141 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a70075ff0 0x7f2a701a1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:49.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.610+0000 7f2a77c00700 1 -- 172.21.15.33:0/3942553141 >> 172.21.15.33:0/3942553141 conn(0x7f2a7007a600 msgr2=0x7f2a701005d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:35:49.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.610+0000 7f2a77c00700 1 -- 172.21.15.33:0/3942553141 shutdown_connections 2024-05-03T18:35:49.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:49.610+0000 7f2a77c00700 1 -- 172.21.15.33:0/3942553141 wait complete. 2024-05-03T18:35:50.110 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 275.0 s, 605.8 tps, lat 165.885 ms stddev 48.631 2024-05-03T18:35:50.448 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:35:50 smithi099 ceph-mon[31658]: pgmap v266: 129 pgs: 129 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 8.4 MiB/s wr, 787 op/s 2024-05-03T18:35:50.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:35:50 smithi033 ceph-mon[28330]: pgmap v266: 129 pgs: 129 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 8.4 MiB/s wr, 787 op/s 2024-05-03T18:35:50.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:35:50 smithi150 ceph-mon[32492]: pgmap v266: 129 pgs: 129 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 8.4 MiB/s wr, 787 op/s 2024-05-03T18:35:51.153 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-03T18:35:51.153 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T18:35:51.887 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:35:52.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:35:52 smithi150 ceph-mon[32492]: pgmap v267: 129 pgs: 129 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 5.5 MiB/s wr, 508 op/s 2024-05-03T18:35:52.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:35:52 smithi033 ceph-mon[28330]: pgmap v267: 129 pgs: 129 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 5.5 MiB/s wr, 508 op/s 2024-05-03T18:35:52.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:35:52 smithi099 ceph-mon[31658]: pgmap v267: 129 pgs: 129 active+clean; 2.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 5.5 MiB/s wr, 508 op/s 2024-05-03T18:35:54.215 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:35:54 smithi033 ceph-mon[28330]: pgmap v268: 129 pgs: 129 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 8.2 MiB/s wr, 763 op/s 2024-05-03T18:35:54.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:35:54 smithi150 ceph-mon[32492]: pgmap v268: 129 pgs: 129 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 8.2 MiB/s wr, 763 op/s 2024-05-03T18:35:54.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:35:54 smithi099 ceph-mon[31658]: pgmap v268: 129 pgs: 129 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 8.2 MiB/s wr, 763 op/s 2024-05-03T18:35:55.110 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 280.0 s, 607.2 tps, lat 164.621 ms stddev 89.566 2024-05-03T18:35:56.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:35:56 smithi150 ceph-mon[32492]: pgmap v269: 129 pgs: 129 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 5.2 MiB/s wr, 506 op/s 2024-05-03T18:35:56.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:56.457+0000 7f644d600700 1 -- 172.21.15.33:0/336796893 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f643c00bab0 con 0x7f64501056c0 2024-05-03T18:35:56.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:56.458+0000 7f6455c00700 1 -- 172.21.15.33:0/336796893 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64501056c0 msgr2=0x7f6450105b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:35:56.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:56.458+0000 7f6455c00700 1 --2- 172.21.15.33:0/336796893 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64501056c0 0x7f6450105b40 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f643c009990 tx=0x7f643c009ca0 comp rx=0 tx=0).stop 2024-05-03T18:35:56.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:56.458+0000 7f6455c00700 1 -- 172.21.15.33:0/336796893 shutdown_connections 2024-05-03T18:35:56.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:56.458+0000 7f6455c00700 1 --2- 172.21.15.33:0/336796893 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6450106080 0x7f6450112590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:56.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:56.458+0000 7f6455c00700 1 --2- 172.21.15.33:0/336796893 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f645010b6c0 0x7f645010baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:56.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:56.458+0000 7f6455c00700 1 --2- 172.21.15.33:0/336796893 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64501056c0 0x7f6450105b40 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:56.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:56.458+0000 7f6455c00700 1 -- 172.21.15.33:0/336796893 >> 172.21.15.33:0/336796893 conn(0x7f64501011b0 msgr2=0x7f64501035d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:35:56.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:56.458+0000 7f6455c00700 1 -- 172.21.15.33:0/336796893 shutdown_connections 2024-05-03T18:35:56.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:56.458+0000 7f6455c00700 1 -- 172.21.15.33:0/336796893 wait complete. 2024-05-03T18:35:56.461 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:35:56 smithi033 ceph-mon[28330]: pgmap v269: 129 pgs: 129 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 5.2 MiB/s wr, 506 op/s 2024-05-03T18:35:56.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:56.460+0000 7f6455c00700 1 Processor -- start 2024-05-03T18:35:56.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:56.460+0000 7f6455c00700 1 -- start start 2024-05-03T18:35:56.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:56.460+0000 7f6455c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f64501056c0 0x7f64501a1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:56.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:56.461+0000 7f6455c00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6450106080 0x7f64501a1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:56.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:56.461+0000 7f6455c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f645010b6c0 0x7f645019baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:56.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:56.461+0000 7f6455c00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6450115850 con 0x7f6450106080 2024-05-03T18:35:56.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:56.461+0000 7f6455c00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f64501156d0 con 0x7f64501056c0 2024-05-03T18:35:56.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:56.461+0000 7f6455c00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f64501159d0 con 0x7f645010b6c0 2024-05-03T18:35:56.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:56.461+0000 7f644ea00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f64501056c0 0x7f64501a1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:56.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:56.461+0000 7f644ea00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f64501056c0 0x7f64501a1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:44064/0 (socket says 172.21.15.33:44064) 2024-05-03T18:35:56.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:56.461+0000 7f644ea00700 1 -- 172.21.15.33:0/3350748535 learned_addr learned my addr 172.21.15.33:0/3350748535 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:35:56.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:56.462+0000 7f644e000700 1 --2- 172.21.15.33:0/3350748535 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6450106080 0x7f64501a1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:56.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:56.462+0000 7f644f400700 1 --2- 172.21.15.33:0/3350748535 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f645010b6c0 0x7f645019baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:56.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:56.463+0000 7f644e000700 1 -- 172.21.15.33:0/3350748535 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f645010b6c0 msgr2=0x7f645019baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:35:56.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:56.463+0000 7f644e000700 1 --2- 172.21.15.33:0/3350748535 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f645010b6c0 0x7f645019baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:56.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:56.463+0000 7f644e000700 1 -- 172.21.15.33:0/3350748535 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f64501056c0 msgr2=0x7f64501a1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:35:56.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:56.463+0000 7f644e000700 1 --2- 172.21.15.33:0/3350748535 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f64501056c0 0x7f64501a1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:56.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:56.463+0000 7f644e000700 1 -- 172.21.15.33:0/3350748535 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f645019c250 con 0x7f6450106080 2024-05-03T18:35:56.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:56.464+0000 7f644e000700 1 --2- 172.21.15.33:0/3350748535 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6450106080 0x7f64501a1e70 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f643c009990 tx=0x7f643c0057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:35:56.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:56.465+0000 7f6447400700 1 -- 172.21.15.33:0/3350748535 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f643c026070 con 0x7f6450106080 2024-05-03T18:35:56.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:56.465+0000 7f6447400700 1 -- 172.21.15.33:0/3350748535 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f643c021410 con 0x7f6450106080 2024-05-03T18:35:56.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:56.465+0000 7f6447400700 1 -- 172.21.15.33:0/3350748535 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f643c017650 con 0x7f6450106080 2024-05-03T18:35:56.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:56.465+0000 7f6455c00700 1 -- 172.21.15.33:0/3350748535 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f645019c4e0 con 0x7f6450106080 2024-05-03T18:35:56.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:56.465+0000 7f6455c00700 1 -- 172.21.15.33:0/3350748535 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64501a8800 con 0x7f6450106080 2024-05-03T18:35:56.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:56.468+0000 7f6455c00700 1 -- 172.21.15.33:0/3350748535 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f645010dc50 con 0x7f6450106080 2024-05-03T18:35:56.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:56.473+0000 7f6447400700 1 -- 172.21.15.33:0/3350748535 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f643c009d10 con 0x7f6450106080 2024-05-03T18:35:56.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:56.476+0000 7f6447400700 1 --2- 172.21.15.33:0/3350748535 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f64340b2b40 0x7f64340b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:35:56.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:56.477+0000 7f6447400700 1 -- 172.21.15.33:0/3350748535 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9518+0+0 (secure 0 0 0) 0x7f643c0e3270 con 0x7f6450106080 2024-05-03T18:35:56.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:56.478+0000 7f644ea00700 1 --2- 172.21.15.33:0/3350748535 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f64340b2b40 0x7f64340b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:35:56.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:56.479+0000 7f644ea00700 1 --2- 172.21.15.33:0/3350748535 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f64340b2b40 0x7f64340b5000 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6440009c90 tx=0x7f6440009260 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:35:56.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:56.491+0000 7f6447400700 1 -- 172.21.15.33:0/3350748535 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f643c013070 con 0x7f6450106080 2024-05-03T18:35:56.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:35:56 smithi099 ceph-mon[31658]: pgmap v269: 129 pgs: 129 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 5.2 MiB/s wr, 506 op/s 2024-05-03T18:35:57.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:57.011+0000 7f6455c00700 1 -- 172.21.15.33:0/3350748535 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f645006a190 con 0x7f6450106080 2024-05-03T18:35:57.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:57.015+0000 7f6447400700 1 -- 172.21.15.33:0/3350748535 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11801 (secure 0 0 0) 0x7f643c0aa900 con 0x7f6450106080 2024-05-03T18:35:57.017 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:35:57.018 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1327393503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1327393503},{"type":"v1","addr":"172.21.15.33:6835","nonce":1327393503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/2461472948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":2461472948},{"type":"v1","addr":"172.21.15.33:6837","nonce":2461472948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14559,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/442269281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":442269281},{"type":"v1","addr":"172.21.15.33:6839","nonce":442269281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14571,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/645571756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":645571756},{"type":"v1","addr":"172.21.15.33:6841","nonce":645571756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/203598909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":203598909},{"type":"v1","addr":"172.21.15.150:6833","nonce":203598909}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/4029842493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":4029842493},{"type":"v1","addr":"172.21.15.150:6835","nonce":4029842493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3463664423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3463664423},{"type":"v1","addr":"172.21.15.150:6837","nonce":3463664423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24481,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6839/4051648018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":4051648018},{"type":"v1","addr":"172.21.15.150:6839","nonce":4051648018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34418,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1865105479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1865105479},{"type":"v1","addr":"172.21.15.99:6833","nonce":1865105479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34445,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/1206393914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1206393914},{"type":"v1","addr":"172.21.15.99:6835","nonce":1206393914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-03T18:22:28.837091+0000","modified":"2024-05-03T18:23:49.758398+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34508},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34472":{"gid":34472,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6837/3360970342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3360970342},{"type":"v1","addr":"172.21.15.99:6837","nonce":3360970342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34508":{"gid":34508,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/919907308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":919907308},{"type":"v1","addr":"172.21.15.99:6839","nonce":919907308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34508,"qdb_cluster":[34508]},"id":1}]} 2024-05-03T18:35:57.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:57.019+0000 7f6455c00700 1 -- 172.21.15.33:0/3350748535 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f64340b2b40 msgr2=0x7f64340b5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:35:57.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:57.019+0000 7f6455c00700 1 --2- 172.21.15.33:0/3350748535 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f64340b2b40 0x7f64340b5000 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6440009c90 tx=0x7f6440009260 comp rx=0 tx=0).stop 2024-05-03T18:35:57.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:57.019+0000 7f6455c00700 1 -- 172.21.15.33:0/3350748535 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6450106080 msgr2=0x7f64501a1e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:35:57.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:57.019+0000 7f6455c00700 1 --2- 172.21.15.33:0/3350748535 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6450106080 0x7f64501a1e70 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f643c009990 tx=0x7f643c0057c0 comp rx=0 tx=0).stop 2024-05-03T18:35:57.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:57.020+0000 7f6455c00700 1 -- 172.21.15.33:0/3350748535 shutdown_connections 2024-05-03T18:35:57.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:57.020+0000 7f6455c00700 1 --2- 172.21.15.33:0/3350748535 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f645010b6c0 0x7f645019baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:57.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:57.020+0000 7f6455c00700 1 --2- 172.21.15.33:0/3350748535 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f64340b2b40 0x7f64340b5000 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:57.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:57.020+0000 7f6455c00700 1 --2- 172.21.15.33:0/3350748535 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f64501056c0 0x7f64501a1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:57.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:57.020+0000 7f6455c00700 1 --2- 172.21.15.33:0/3350748535 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6450106080 0x7f64501a1e70 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:35:57.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:57.020+0000 7f6455c00700 1 -- 172.21.15.33:0/3350748535 >> 172.21.15.33:0/3350748535 conn(0x7f64501011b0 msgr2=0x7f64501019b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:35:57.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:57.020+0000 7f6455c00700 1 -- 172.21.15.33:0/3350748535 shutdown_connections 2024-05-03T18:35:57.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:35:57.020+0000 7f6455c00700 1 -- 172.21.15.33:0/3350748535 wait complete. 2024-05-03T18:35:57.024 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 21 2024-05-03T18:35:57.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:35:57 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/3350748535' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:35:57.482 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:35:57 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/3350748535' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:35:57.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:35:57 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/3350748535' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:35:57.941 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-03T18:35:58.106 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:35:58 smithi033 ceph-mon[28330]: pgmap v270: 129 pgs: 129 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 5.2 MiB/s wr, 507 op/s 2024-05-03T18:35:58.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:35:58 smithi150 ceph-mon[32492]: pgmap v270: 129 pgs: 129 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 5.2 MiB/s wr, 507 op/s 2024-05-03T18:35:58.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:35:58 smithi099 ceph-mon[31658]: pgmap v270: 129 pgs: 129 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 5.2 MiB/s wr, 507 op/s 2024-05-03T18:35:58.716 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:36:00.110 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 285.0 s, 590.4 tps, lat 169.975 ms stddev 48.231 2024-05-03T18:36:00.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:36:00 smithi150 ceph-mon[32492]: pgmap v271: 129 pgs: 129 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 8.0 MiB/s wr, 785 op/s 2024-05-03T18:36:00.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:36:00 smithi033 ceph-mon[28330]: pgmap v271: 129 pgs: 129 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 8.0 MiB/s wr, 785 op/s 2024-05-03T18:36:00.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:36:00 smithi099 ceph-mon[31658]: pgmap v271: 129 pgs: 129 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 8.0 MiB/s wr, 785 op/s 2024-05-03T18:36:01.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.407+0000 7f2c21800700 1 -- 172.21.15.33:0/3020054263 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c1c10f230 msgr2=0x7f2c1c116e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:36:01.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.407+0000 7f2c21800700 1 --2- 172.21.15.33:0/3020054263 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c1c10f230 0x7f2c1c116e00 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f2c040074e0 tx=0x7f2c0400af90 comp rx=0 tx=0).stop 2024-05-03T18:36:01.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.407+0000 7f2c21800700 1 -- 172.21.15.33:0/3020054263 shutdown_connections 2024-05-03T18:36:01.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.407+0000 7f2c21800700 1 --2- 172.21.15.33:0/3020054263 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c1c10f230 0x7f2c1c116e00 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:01.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.408+0000 7f2c21800700 1 --2- 172.21.15.33:0/3020054263 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2c1c10a700 0x7f2c1c10aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:01.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.408+0000 7f2c21800700 1 --2- 172.21.15.33:0/3020054263 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2c1c10b0b0 0x7f2c1c10eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:01.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.408+0000 7f2c21800700 1 -- 172.21.15.33:0/3020054263 >> 172.21.15.33:0/3020054263 conn(0x7f2c1c06e610 msgr2=0x7f2c1c06ea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:36:01.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.409+0000 7f2c21800700 1 -- 172.21.15.33:0/3020054263 shutdown_connections 2024-05-03T18:36:01.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.409+0000 7f2c21800700 1 -- 172.21.15.33:0/3020054263 wait complete. 2024-05-03T18:36:01.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.410+0000 7f2c21800700 1 Processor -- start 2024-05-03T18:36:01.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.411+0000 7f2c21800700 1 -- start start 2024-05-03T18:36:01.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.411+0000 7f2c21800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2c1c10a700 0x7f2c1c1ae600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:01.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.411+0000 7f2c21800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2c1c10b0b0 0x7f2c1c1aeb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:01.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.411+0000 7f2c21800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c1c10f230 0x7f2c1c1a87d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:01.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.411+0000 7f2c21800700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2c1c119bb0 con 0x7f2c1c10f230 2024-05-03T18:36:01.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.411+0000 7f2c21800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2c1c119a30 con 0x7f2c1c10a700 2024-05-03T18:36:01.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.411+0000 7f2c21800700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2c1c119d30 con 0x7f2c1c10b0b0 2024-05-03T18:36:01.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.412+0000 7f2c1aa00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2c1c10a700 0x7f2c1c1ae600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:01.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.412+0000 7f2c1b400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c1c10f230 0x7f2c1c1a87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:01.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.412+0000 7f2c1b400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c1c10f230 0x7f2c1c1a87d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46984/0 (socket says 172.21.15.33:46984) 2024-05-03T18:36:01.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.412+0000 7f2c1b400700 1 -- 172.21.15.33:0/1851397427 learned_addr learned my addr 172.21.15.33:0/1851397427 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:36:01.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.413+0000 7f2c1a000700 1 --2- 172.21.15.33:0/1851397427 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2c1c10b0b0 0x7f2c1c1aeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:01.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.414+0000 7f2c1b400700 1 -- 172.21.15.33:0/1851397427 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2c1c10b0b0 msgr2=0x7f2c1c1aeb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:36:01.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.414+0000 7f2c1b400700 1 --2- 172.21.15.33:0/1851397427 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2c1c10b0b0 0x7f2c1c1aeb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:01.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.414+0000 7f2c1b400700 1 -- 172.21.15.33:0/1851397427 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2c1c10a700 msgr2=0x7f2c1c1ae600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:36:01.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.414+0000 7f2c1b400700 1 --2- 172.21.15.33:0/1851397427 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2c1c10a700 0x7f2c1c1ae600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:01.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.414+0000 7f2c1b400700 1 -- 172.21.15.33:0/1851397427 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c1c1a8fb0 con 0x7f2c1c10f230 2024-05-03T18:36:01.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.414+0000 7f2c1aa00700 1 --2- 172.21.15.33:0/1851397427 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2c1c10a700 0x7f2c1c1ae600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:36:01.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.415+0000 7f2c1b400700 1 --2- 172.21.15.33:0/1851397427 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c1c10f230 0x7f2c1c1a87d0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f2c04006010 tx=0x7f2c04003920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:36:01.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.415+0000 7f2c13400700 1 -- 172.21.15.33:0/1851397427 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c04018070 con 0x7f2c1c10f230 2024-05-03T18:36:01.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.416+0000 7f2c21800700 1 -- 172.21.15.33:0/1851397427 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c1c1a9240 con 0x7f2c1c10f230 2024-05-03T18:36:01.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.416+0000 7f2c21800700 1 -- 172.21.15.33:0/1851397427 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c1c1b5460 con 0x7f2c1c10f230 2024-05-03T18:36:01.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.416+0000 7f2c13400700 1 -- 172.21.15.33:0/1851397427 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2c040056d0 con 0x7f2c1c10f230 2024-05-03T18:36:01.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.418+0000 7f2c13400700 1 -- 172.21.15.33:0/1851397427 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c0402d060 con 0x7f2c1c10f230 2024-05-03T18:36:01.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.422+0000 7f2c13400700 1 -- 172.21.15.33:0/1851397427 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f2c0400cbb0 con 0x7f2c1c10f230 2024-05-03T18:36:01.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.424+0000 7f2c2c1bbb80 1 Processor -- start 2024-05-03T18:36:01.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.425+0000 7f2c13400700 1 --2- 172.21.15.33:0/1851397427 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f2c000b2a50 0x7f2c000b4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:01.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.425+0000 7f2c2c1bbb80 1 -- start start 2024-05-03T18:36:01.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.425+0000 7f2c2c1bbb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b434497b0 0x562b43449b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:01.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.425+0000 7f2c2c1bbb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b4344a0d0 0x562b43444fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:01.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.425+0000 7f2c2c1bbb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562b434455d0 0x562b43445a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:01.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.425+0000 7f2c2c1bbb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x562b43397a20 con 0x562b434497b0 2024-05-03T18:36:01.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.425+0000 7f2c2c1bbb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x562b43335760 con 0x562b4344a0d0 2024-05-03T18:36:01.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.425+0000 7f2c2c1bbb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x562b433939e0 con 0x562b434455d0 2024-05-03T18:36:01.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.425+0000 7f2c13400700 1 -- 172.21.15.33:0/1851397427 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9518+0+0 (secure 0 0 0) 0x7f2c040e2a10 con 0x7f2c1c10f230 2024-05-03T18:36:01.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.426+0000 7f2c1a000700 1 --2- 172.21.15.33:0/1851397427 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f2c000b2a50 0x7f2c000b4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:01.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.427+0000 7f2c1b400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562b434455d0 0x562b43445a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:01.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.427+0000 7f2c1b400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562b434455d0 0x562b43445a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:32840/0 (socket says 172.21.15.33:32840) 2024-05-03T18:36:01.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.427+0000 7f2c1b400700 1 -- 172.21.15.33:0/2250057755 learned_addr learned my addr 172.21.15.33:0/2250057755 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:36:01.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.427+0000 7f2c1b400700 1 -- 172.21.15.33:0/2250057755 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b4344a0d0 msgr2=0x562b43444fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:36:01.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.427+0000 7f2c1b400700 1 --2- 172.21.15.33:0/2250057755 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b4344a0d0 0x562b43444fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:01.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.427+0000 7f2c1aa00700 1 --2- 172.21.15.33:0/2250057755 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b434497b0 0x562b43449b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:01.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.427+0000 7f2c1b400700 1 -- 172.21.15.33:0/2250057755 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b434497b0 msgr2=0x562b43449b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:36:01.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.427+0000 7f2c1b400700 1 --2- 172.21.15.33:0/2250057755 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b434497b0 0x562b43449b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:01.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.427+0000 7f2c1b400700 1 -- 172.21.15.33:0/2250057755 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b432e48d0 con 0x562b434455d0 2024-05-03T18:36:01.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.427+0000 7f2c1aa00700 1 --2- 172.21.15.33:0/2250057755 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b434497b0 0x562b43449b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:36:01.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.428+0000 7f2c1b400700 1 --2- 172.21.15.33:0/2250057755 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562b434455d0 0x562b43445a30 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2c040a7b20 tx=0x7f2c040aa780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e9c2555803e4ebc server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:36:01.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.428+0000 7f2c1a000700 1 --2- 172.21.15.33:0/1851397427 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f2c000b2a50 0x7f2c000b4f10 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c074e00 tx=0x7f2c0c0103d0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:36:01.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.428+0000 7f2c0be00700 1 -- 172.21.15.33:0/2250057755 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c040aa3c0 con 0x562b434455d0 2024-05-03T18:36:01.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.429+0000 7f2c0be00700 1 -- 172.21.15.33:0/2250057755 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2c040aafb0 con 0x562b434455d0 2024-05-03T18:36:01.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.429+0000 7f2c0be00700 1 -- 172.21.15.33:0/2250057755 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c040aa3c0 con 0x562b434455d0 2024-05-03T18:36:01.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.430+0000 7f2c2c1bbb80 1 -- 172.21.15.33:0/2250057755 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562b434455d0 msgr2=0x562b43445a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:36:01.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.430+0000 7f2c2c1bbb80 1 --2- 172.21.15.33:0/2250057755 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562b434455d0 0x562b43445a30 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2c040a7b20 tx=0x7f2c040aa780 comp rx=0 tx=0).stop 2024-05-03T18:36:01.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.430+0000 7f2c2c1bbb80 1 -- 172.21.15.33:0/2250057755 shutdown_connections 2024-05-03T18:36:01.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.430+0000 7f2c2c1bbb80 1 --2- 172.21.15.33:0/2250057755 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562b434455d0 0x562b43445a30 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:01.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.430+0000 7f2c2c1bbb80 1 --2- 172.21.15.33:0/2250057755 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b434497b0 0x562b43449b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:01.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.430+0000 7f2c2c1bbb80 1 --2- 172.21.15.33:0/2250057755 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b4344a0d0 0x562b43444fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:01.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.430+0000 7f2c2c1bbb80 1 -- 172.21.15.33:0/2250057755 >> 172.21.15.33:0/2250057755 conn(0x562b43373b60 msgr2=0x562b4343f7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:36:01.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.430+0000 7f2c2c1bbb80 1 -- 172.21.15.33:0/2250057755 shutdown_connections 2024-05-03T18:36:01.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.431+0000 7f2c2c1bbb80 1 -- 172.21.15.33:0/2250057755 wait complete. 2024-05-03T18:36:01.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.431+0000 7f2c2c1bbb80 1 -- start start 2024-05-03T18:36:01.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.432+0000 7f2c2c1bbb80 1 Processor -- start 2024-05-03T18:36:01.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.433+0000 7f2c2c1bbb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562b434455d0 0x562b43450eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:01.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.433+0000 7f2c2c1bbb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b434497b0 0x562b4344e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:01.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.434+0000 7f2c2c1bbb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b4344a0d0 0x562b4344e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:01.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.434+0000 7f2c1aa00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562b434455d0 0x562b43450eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:01.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.434+0000 7f2c1aa00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562b434455d0 0x562b43450eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:32846/0 (socket says 172.21.15.33:32846) 2024-05-03T18:36:01.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.434+0000 7f2c1a000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b434497b0 0x562b4344e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:01.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.434+0000 7f2c1a000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b434497b0 0x562b4344e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:44102/0 (socket says 172.21.15.33:44102) 2024-05-03T18:36:01.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.434+0000 7f2c1b400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b4344a0d0 0x562b4344e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:01.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.435+0000 7f2c1aa00700 1 -- 172.21.15.33:0/3462923884 learned_addr learned my addr 172.21.15.33:0/3462923884 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:36:01.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.435+0000 7f2c2c1bbb80 1 -- 172.21.15.33:0/3462923884 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x562b43451a60 con 0x562b4344a0d0 2024-05-03T18:36:01.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.435+0000 7f2c2c1bbb80 1 -- 172.21.15.33:0/3462923884 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x562b432fdb20 con 0x562b434497b0 2024-05-03T18:36:01.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.435+0000 7f2c2c1bbb80 1 -- 172.21.15.33:0/3462923884 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x562b43451be0 con 0x562b434455d0 2024-05-03T18:36:01.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.435+0000 7f2c2c1bbb80 10 client.0 _ll_register_callbacks cb 0x562b4333ec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:36:01.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.436+0000 7f2c1a000700 1 -- 172.21.15.33:0/3462923884 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562b434455d0 msgr2=0x562b43450eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:36:01.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.436+0000 7f2c1a000700 1 --2- 172.21.15.33:0/3462923884 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562b434455d0 0x562b43450eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:01.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.436+0000 7f2c1a000700 1 -- 172.21.15.33:0/3462923884 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b4344a0d0 msgr2=0x562b4344e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:36:01.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.436+0000 7f2c1a000700 1 --2- 172.21.15.33:0/3462923884 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b4344a0d0 0x562b4344e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:01.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.436+0000 7f2c1a000700 1 -- 172.21.15.33:0/3462923884 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b4335d420 con 0x562b434497b0 2024-05-03T18:36:01.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.436+0000 7f2c1b400700 1 --2- 172.21.15.33:0/3462923884 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b4344a0d0 0x562b4344e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:36:01.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.437+0000 7f2c1a000700 1 --2- 172.21.15.33:0/3462923884 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b434497b0 0x562b4344e2c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c06cc10 tx=0x7f2c0c0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6efaa907d8f5ae65 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:36:01.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.437+0000 7f2c08c00700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-03T18:36:01.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.437+0000 7f2c08c00700 1 -- 172.21.15.33:0/3462923884 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c0c024020 con 0x562b434497b0 2024-05-03T18:36:01.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.438+0000 7f2c08c00700 1 -- 172.21.15.33:0/3462923884 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9518+0+0 (secure 0 0 0) 0x7f2c0c020860 con 0x562b434497b0 2024-05-03T18:36:01.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.438+0000 7f2c2c1bbb80 1 -- 172.21.15.33:0/3462923884 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c040aafb0 con 0x562b434497b0 2024-05-03T18:36:01.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.438+0000 7f2c08c00700 1 -- 172.21.15.33:0/3462923884 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f2c0c011e30 con 0x562b434497b0 2024-05-03T18:36:01.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.439+0000 7f2c2c1bbb80 10 client.35945 fetch_fsmap learned FSMap version 21 2024-05-03T18:36:01.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.439+0000 7f2c2c1bbb80 1 -- 172.21.15.33:0/3462923884 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x562b4345c020 con 0x562b434497b0 2024-05-03T18:36:01.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.439+0000 7f2c08c00700 1 -- 172.21.15.33:0/3462923884 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2c040aafb0 con 0x562b434497b0 2024-05-03T18:36:01.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.440+0000 7f2c08c00700 1 -- 172.21.15.33:0/3462923884 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f2c0c031020 con 0x562b434497b0 2024-05-03T18:36:01.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.440+0000 7f2c2c1bbb80 10 client.35945 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:36:01.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.440+0000 7f2c2c1bbb80 10 client.35945 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:36:01.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.440+0000 7f2c2c1bbb80 20 client.35945 populate_metadata read hostname 'smithi033' 2024-05-03T18:36:01.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.441+0000 7f2c2c1bbb80 1 --2- 172.21.15.33:0/3462923884 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x562b4345e660 0x562b43460a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:01.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.441+0000 7f2c2c1bbb80 4 client.35945 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:36:01.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.441+0000 7f2c2c1bbb80 1 -- 172.21.15.33:0/3462923884 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562b434121d0 con 0x562b4345e660 2024-05-03T18:36:01.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.443+0000 7f2c1aa00700 1 --2- 172.21.15.33:0/3462923884 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x562b4345e660 0x562b43460a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:01.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.444+0000 7f2c1aa00700 1 --2- 172.21.15.33:0/3462923884 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x562b4345e660 0x562b43460a50 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e100bb1669f60f4a server_cookie=c110076c82720163 in_seq=0 out_seq=0 2024-05-03T18:36:01.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.444+0000 7f2c08c00700 10 client.35945 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:36:01.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.446+0000 7f2c08c00700 1 -- 172.21.15.33:0/3462923884 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562b434121d0 con 0x562b4345e660 2024-05-03T18:36:01.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.446+0000 7f2c08c00700 10 client.35945 handle_command_reply: tid=0 2024-05-03T18:36:01.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.446+0000 7f2c2c1bbb80 1 client.35945 shutdown 2024-05-03T18:36:01.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.447+0000 7f2c2c1bbb80 1 -- 172.21.15.33:0/3462923884 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b434497b0 msgr2=0x562b4344e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:36:01.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.447+0000 7f2c2c1bbb80 1 --2- 172.21.15.33:0/3462923884 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b434497b0 0x562b4344e2c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c06cc10 tx=0x7f2c0c0047c0 comp rx=0 tx=0).stop 2024-05-03T18:36:01.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.447+0000 7f2c2c1bbb80 1 -- 172.21.15.33:0/3462923884 shutdown_connections 2024-05-03T18:36:01.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.447+0000 7f2c2c1bbb80 1 --2- 172.21.15.33:0/3462923884 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x562b4345e660 0x562b43460a50 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:01.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.447+0000 7f2c2c1bbb80 1 --2- 172.21.15.33:0/3462923884 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b4344a0d0 0x562b4344e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:01.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.447+0000 7f2c2c1bbb80 1 --2- 172.21.15.33:0/3462923884 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562b434455d0 0x562b43450eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:01.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.447+0000 7f2c2c1bbb80 1 --2- 172.21.15.33:0/3462923884 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b434497b0 0x562b4344e2c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:01.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.447+0000 7f2c2c1bbb80 1 -- 172.21.15.33:0/3462923884 >> 172.21.15.33:0/3462923884 conn(0x562b43373b60 msgr2=0x562b4344ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:36:01.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.447+0000 7f2c08c00700 0 client.35945 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:36:01.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.447+0000 7f2c2c1bbb80 1 -- 172.21.15.33:0/3462923884 shutdown_connections 2024-05-03T18:36:01.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.447+0000 7f2c2c1bbb80 1 -- 172.21.15.33:0/3462923884 wait complete. 2024-05-03T18:36:01.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.448+0000 7f2c2c1bbb80 20 client.35945 trim_cache size 0 max 16384 2024-05-03T18:36:01.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.481+0000 7f2c2c1bbb80 1 Processor -- start 2024-05-03T18:36:01.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.482+0000 7f2c2c1bbb80 1 -- start start 2024-05-03T18:36:01.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.482+0000 7f2c2c1bbb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562b434455d0 0x562b43450eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:01.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.482+0000 7f2c2c1bbb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b434497b0 0x562b4344e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:01.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.482+0000 7f2c2c1bbb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b4344a0d0 0x562b43448b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:01.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.482+0000 7f2c2c1bbb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf800cdf0 con 0x562b434497b0 2024-05-03T18:36:01.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.482+0000 7f2c2c1bbb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf800f650 con 0x562b4344a0d0 2024-05-03T18:36:01.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.482+0000 7f2c2c1bbb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf800d440 con 0x562b434455d0 2024-05-03T18:36:01.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.483+0000 7f2c1b400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b4344a0d0 0x562b43448b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:01.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.483+0000 7f2c1aa00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562b434455d0 0x562b43450eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:01.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.483+0000 7f2c1b400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b4344a0d0 0x562b43448b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:44110/0 (socket says 172.21.15.33:44110) 2024-05-03T18:36:01.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.483+0000 7f2c1b400700 1 -- 172.21.15.33:0/2830229882 learned_addr learned my addr 172.21.15.33:0/2830229882 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:36:01.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.483+0000 7f2c1a000700 1 --2- 172.21.15.33:0/2830229882 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b434497b0 0x562b4344e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:01.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.484+0000 7f2c1aa00700 1 -- 172.21.15.33:0/2830229882 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b4344a0d0 msgr2=0x562b43448b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:36:01.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.484+0000 7f2c1aa00700 1 --2- 172.21.15.33:0/2830229882 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b4344a0d0 0x562b43448b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:01.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.484+0000 7f2c1aa00700 1 -- 172.21.15.33:0/2830229882 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b434497b0 msgr2=0x562b4344e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:36:01.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.484+0000 7f2c1aa00700 1 --2- 172.21.15.33:0/2830229882 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b434497b0 0x562b4344e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:01.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.484+0000 7f2c1aa00700 1 -- 172.21.15.33:0/2830229882 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b43461080 con 0x562b434455d0 2024-05-03T18:36:01.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.485+0000 7f2c1aa00700 1 --2- 172.21.15.33:0/2830229882 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562b434455d0 0x562b43450eb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x562b43461250 tx=0x7f2c14009040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7aa6375c82621c4a server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:36:01.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.486+0000 7f2c0b400700 1 -- 172.21.15.33:0/2830229882 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c14023020 con 0x562b434455d0 2024-05-03T18:36:01.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.486+0000 7f2c0b400700 1 -- 172.21.15.33:0/2830229882 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2c14007e10 con 0x562b434455d0 2024-05-03T18:36:01.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.486+0000 7f2c0b400700 1 -- 172.21.15.33:0/2830229882 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c1400e4f0 con 0x562b434455d0 2024-05-03T18:36:01.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.487+0000 7f2c2c1bbb80 1 -- 172.21.15.33:0/2830229882 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562b434455d0 msgr2=0x562b43450eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:36:01.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.487+0000 7f2c2c1bbb80 1 --2- 172.21.15.33:0/2830229882 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562b434455d0 0x562b43450eb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x562b43461250 tx=0x7f2c14009040 comp rx=0 tx=0).stop 2024-05-03T18:36:01.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.487+0000 7f2c2c1bbb80 1 -- 172.21.15.33:0/2830229882 shutdown_connections 2024-05-03T18:36:01.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.487+0000 7f2c2c1bbb80 1 --2- 172.21.15.33:0/2830229882 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b4344a0d0 0x562b43448b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:01.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.487+0000 7f2c2c1bbb80 1 --2- 172.21.15.33:0/2830229882 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562b434455d0 0x562b43450eb0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:01.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.487+0000 7f2c2c1bbb80 1 --2- 172.21.15.33:0/2830229882 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b434497b0 0x562b4344e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:01.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.487+0000 7f2c2c1bbb80 1 -- 172.21.15.33:0/2830229882 >> 172.21.15.33:0/2830229882 conn(0x562b43373b60 msgr2=0x562b434483f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:36:01.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.487+0000 7f2c2c1bbb80 1 -- 172.21.15.33:0/2830229882 shutdown_connections 2024-05-03T18:36:01.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.487+0000 7f2c2c1bbb80 1 -- 172.21.15.33:0/2830229882 wait complete. 2024-05-03T18:36:01.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.488+0000 7f2c2c1bbb80 1 -- start start 2024-05-03T18:36:01.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.489+0000 7f2c2c1bbb80 1 Processor -- start 2024-05-03T18:36:01.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.490+0000 7f2c2c1bbb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b434455d0 0x562b434c6420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:01.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.490+0000 7f2c2c1bbb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b434497b0 0x562b434c6960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:01.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.490+0000 7f2c2c1bbb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562b4344a0d0 0x562b43456fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:01.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.490+0000 7f2c2c1bbb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf800db10 con 0x562b434497b0 2024-05-03T18:36:01.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.490+0000 7f2c2c1bbb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf800ef80 con 0x562b434455d0 2024-05-03T18:36:01.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.490+0000 7f2c2c1bbb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf800e1e0 con 0x562b4344a0d0 2024-05-03T18:36:01.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.490+0000 7f2c2c1bbb80 10 client.0 _ll_register_callbacks cb 0x562b4333ec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:36:01.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.491+0000 7f2c1a000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b434497b0 0x562b434c6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:01.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.492+0000 7f2c1aa00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b434455d0 0x562b434c6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:01.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.492+0000 7f2c1aa00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b434455d0 0x562b434c6420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:44122/0 (socket says 172.21.15.33:44122) 2024-05-03T18:36:01.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.492+0000 7f2c1aa00700 1 -- 172.21.15.33:0/635969472 learned_addr learned my addr 172.21.15.33:0/635969472 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:36:01.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.492+0000 7f2c1a000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b434497b0 0x562b434c6960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47028/0 (socket says 172.21.15.33:47028) 2024-05-03T18:36:01.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.493+0000 7f2c1b400700 1 --2- 172.21.15.33:0/635969472 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562b4344a0d0 0x562b43456fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:01.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.493+0000 7f2c1a000700 1 -- 172.21.15.33:0/635969472 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562b4344a0d0 msgr2=0x562b43456fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:36:01.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.493+0000 7f2c1a000700 1 --2- 172.21.15.33:0/635969472 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562b4344a0d0 0x562b43456fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:01.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.493+0000 7f2c1a000700 1 -- 172.21.15.33:0/635969472 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b434455d0 msgr2=0x562b434c6420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:36:01.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.493+0000 7f2c1a000700 1 --2- 172.21.15.33:0/635969472 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b434455d0 0x562b434c6420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:01.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.493+0000 7f2c1a000700 1 -- 172.21.15.33:0/635969472 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b43457840 con 0x562b434497b0 2024-05-03T18:36:01.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.494+0000 7f2c1a000700 1 --2- 172.21.15.33:0/635969472 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b434497b0 0x562b434c6960 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f2c0c011fd0 tx=0x7f2c0c020910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad29264f9c82804a server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:36:01.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.495+0000 7f2c08c00700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:36:01.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.495+0000 7f2c08c00700 1 -- 172.21.15.33:0/635969472 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c0c040060 con 0x562b434497b0 2024-05-03T18:36:01.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.495+0000 7f2c08c00700 1 -- 172.21.15.33:0/635969472 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9518+0+0 (secure 0 0 0) 0x7f2c0c020dc0 con 0x562b434497b0 2024-05-03T18:36:01.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.496+0000 7f2c2c1bbb80 1 -- 172.21.15.33:0/635969472 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c14007e10 con 0x562b434497b0 2024-05-03T18:36:01.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.497+0000 7f2c08c00700 1 -- 172.21.15.33:0/635969472 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f2c0c024040 con 0x562b434497b0 2024-05-03T18:36:01.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.497+0000 7f2c2c1bbb80 10 client.16284 fetch_fsmap learned FSMap version 21 2024-05-03T18:36:01.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.497+0000 7f2c2c1bbb80 1 -- 172.21.15.33:0/635969472 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x562b434d9f90 con 0x562b434497b0 2024-05-03T18:36:01.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.498+0000 7f2c08c00700 1 -- 172.21.15.33:0/635969472 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2c14007e10 con 0x562b434497b0 2024-05-03T18:36:01.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.498+0000 7f2c08c00700 1 -- 172.21.15.33:0/635969472 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f2c0c031040 con 0x562b434497b0 2024-05-03T18:36:01.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.499+0000 7f2c2c1bbb80 10 client.16284 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:36:01.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.499+0000 7f2c2c1bbb80 10 client.16284 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:36:01.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.499+0000 7f2c2c1bbb80 20 client.16284 populate_metadata read hostname 'smithi033' 2024-05-03T18:36:01.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.499+0000 7f2c2c1bbb80 1 --2- 172.21.15.33:0/635969472 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x562b434da610 0x562b434dca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:01.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.499+0000 7f2c2c1bbb80 4 client.16284 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-03T18:36:01.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.499+0000 7f2c2c1bbb80 1 -- 172.21.15.33:0/635969472 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562b433a5400 con 0x562b434da610 2024-05-03T18:36:01.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.500+0000 7f2c1aa00700 1 --2- 172.21.15.33:0/635969472 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x562b434da610 0x562b434dca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:01.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.501+0000 7f2c1aa00700 1 --2- 172.21.15.33:0/635969472 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x562b434da610 0x562b434dca30 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e81715bae6ceb942 server_cookie=1f9a206a8124405a in_seq=0 out_seq=0 2024-05-03T18:36:01.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.501+0000 7f2c08c00700 10 client.16284 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:36:01.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.501+0000 7f2c08c00700 1 -- 172.21.15.33:0/635969472 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562b433a5400 con 0x562b434da610 2024-05-03T18:36:01.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.501+0000 7f2c08c00700 10 client.16284 handle_command_reply: tid=0 2024-05-03T18:36:01.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.501+0000 7f2c2c1bbb80 1 client.16284 shutdown 2024-05-03T18:36:01.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.502+0000 7f2c2c1bbb80 1 -- 172.21.15.33:0/635969472 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b434497b0 msgr2=0x562b434c6960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:36:01.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.502+0000 7f2c2c1bbb80 1 --2- 172.21.15.33:0/635969472 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b434497b0 0x562b434c6960 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f2c0c011fd0 tx=0x7f2c0c020910 comp rx=0 tx=0).stop 2024-05-03T18:36:01.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.502+0000 7f2c2c1bbb80 1 -- 172.21.15.33:0/635969472 shutdown_connections 2024-05-03T18:36:01.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.502+0000 7f2c2c1bbb80 1 --2- 172.21.15.33:0/635969472 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x562b434da610 0x562b434dca30 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:01.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.502+0000 7f2c2c1bbb80 1 --2- 172.21.15.33:0/635969472 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562b4344a0d0 0x562b43456fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:01.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.502+0000 7f2c2c1bbb80 1 --2- 172.21.15.33:0/635969472 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b434455d0 0x562b434c6420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:01.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.502+0000 7f2c2c1bbb80 1 --2- 172.21.15.33:0/635969472 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b434497b0 0x562b434c6960 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:01.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.502+0000 7f2c2c1bbb80 1 -- 172.21.15.33:0/635969472 >> 172.21.15.33:0/635969472 conn(0x562b43373b60 msgr2=0x562b4344f870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:36:01.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.503+0000 7f2c08c00700 0 client.16284 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:36:01.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.503+0000 7f2c2c1bbb80 1 -- 172.21.15.33:0/635969472 shutdown_connections 2024-05-03T18:36:01.505 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-03T18:36:01.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.503+0000 7f2c2c1bbb80 1 -- 172.21.15.33:0/635969472 wait complete. 2024-05-03T18:36:01.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.503+0000 7f2c2c1bbb80 20 client.16284 trim_cache size 0 max 16384 2024-05-03T18:36:01.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.504+0000 7f2c21800700 1 -- 172.21.15.33:0/1851397427 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f2c000b2a50 msgr2=0x7f2c000b4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:36:01.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.505+0000 7f2c21800700 1 --2- 172.21.15.33:0/1851397427 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f2c000b2a50 0x7f2c000b4f10 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c074e00 tx=0x7f2c0c0103d0 comp rx=0 tx=0).stop 2024-05-03T18:36:01.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.505+0000 7f2c21800700 1 -- 172.21.15.33:0/1851397427 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c1c10f230 msgr2=0x7f2c1c1a87d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:36:01.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.505+0000 7f2c21800700 1 --2- 172.21.15.33:0/1851397427 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c1c10f230 0x7f2c1c1a87d0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f2c04006010 tx=0x7f2c04003920 comp rx=0 tx=0).stop 2024-05-03T18:36:01.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.505+0000 7f2c21800700 1 -- 172.21.15.33:0/1851397427 shutdown_connections 2024-05-03T18:36:01.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.505+0000 7f2c21800700 1 --2- 172.21.15.33:0/1851397427 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c1c10f230 0x7f2c1c1a87d0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:01.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.505+0000 7f2c21800700 1 --2- 172.21.15.33:0/1851397427 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f2c000b2a50 0x7f2c000b4f10 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:01.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.505+0000 7f2c21800700 1 --2- 172.21.15.33:0/1851397427 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2c1c10a700 0x7f2c1c1ae600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:01.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.505+0000 7f2c21800700 1 --2- 172.21.15.33:0/1851397427 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2c1c10b0b0 0x7f2c1c1aeb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:01.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.505+0000 7f2c21800700 1 -- 172.21.15.33:0/1851397427 >> 172.21.15.33:0/1851397427 conn(0x7f2c1c06e610 msgr2=0x7f2c1c073570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:36:01.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.505+0000 7f2c21800700 1 -- 172.21.15.33:0/1851397427 shutdown_connections 2024-05-03T18:36:01.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:01.505+0000 7f2c21800700 1 -- 172.21.15.33:0/1851397427 wait complete. 2024-05-03T18:36:02.237 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:36:02 smithi033 ceph-mon[28330]: pgmap v272: 129 pgs: 129 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 5.5 MiB/s wr, 533 op/s 2024-05-03T18:36:02.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:36:02 smithi150 ceph-mon[32492]: pgmap v272: 129 pgs: 129 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 5.5 MiB/s wr, 533 op/s 2024-05-03T18:36:02.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:36:02 smithi099 ceph-mon[31658]: pgmap v272: 129 pgs: 129 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 5.5 MiB/s wr, 533 op/s 2024-05-03T18:36:03.439 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-03T18:36:03.439 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-03T18:36:04.234 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:36:04.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:36:04 smithi150 ceph-mon[32492]: pgmap v273: 129 pgs: 129 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 8.0 MiB/s wr, 803 op/s 2024-05-03T18:36:04.417 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:36:04 smithi033 ceph-mon[28330]: pgmap v273: 129 pgs: 129 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 8.0 MiB/s wr, 803 op/s 2024-05-03T18:36:04.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:36:04 smithi099 ceph-mon[31658]: pgmap v273: 129 pgs: 129 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 8.0 MiB/s wr, 803 op/s 2024-05-03T18:36:05.111 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 290.0 s, 595.5 tps, lat 167.702 ms stddev 85.528 2024-05-03T18:36:06.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:36:06 smithi150 ceph-mon[32492]: pgmap v274: 129 pgs: 129 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 5.4 MiB/s wr, 548 op/s 2024-05-03T18:36:06.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:36:06 smithi033 ceph-mon[28330]: pgmap v274: 129 pgs: 129 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 5.4 MiB/s wr, 548 op/s 2024-05-03T18:36:06.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:36:06 smithi099 ceph-mon[31658]: pgmap v274: 129 pgs: 129 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 5.4 MiB/s wr, 548 op/s 2024-05-03T18:36:08.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:36:08 smithi150 ceph-mon[32492]: osdmap e95: 12 total, 12 up, 12 in 2024-05-03T18:36:08.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:36:08 smithi150 ceph-mon[32492]: pgmap v276: 129 pgs: 129 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 6.5 MiB/s wr, 658 op/s 2024-05-03T18:36:08.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:36:08 smithi033 ceph-mon[28330]: osdmap e95: 12 total, 12 up, 12 in 2024-05-03T18:36:08.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:36:08 smithi033 ceph-mon[28330]: pgmap v276: 129 pgs: 129 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 6.5 MiB/s wr, 658 op/s 2024-05-03T18:36:08.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:36:08 smithi099 ceph-mon[31658]: osdmap e95: 12 total, 12 up, 12 in 2024-05-03T18:36:08.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:36:08 smithi099 ceph-mon[31658]: pgmap v276: 129 pgs: 129 active+clean; 2.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 6.5 MiB/s wr, 658 op/s 2024-05-03T18:36:10.110 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 295.0 s, 604.6 tps, lat 165.209 ms stddev 62.287 2024-05-03T18:36:10.278 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:36:10 smithi033 ceph-mon[28330]: pgmap v277: 129 pgs: 129 active+clean; 2.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 6.2 MiB/s wr, 639 op/s 2024-05-03T18:36:10.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.277+0000 7f2cdf800700 1 -- 172.21.15.33:0/184499420 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2cd806bad0 msgr2=0x7f2cd810fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:36:10.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.277+0000 7f2cdf800700 1 --2- 172.21.15.33:0/184499420 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2cd806bad0 0x7f2cd810fa10 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2cc4009990 tx=0x7f2cc4009ca0 comp rx=0 tx=0).stop 2024-05-03T18:36:10.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.278+0000 7f2cdf800700 1 -- 172.21.15.33:0/184499420 shutdown_connections 2024-05-03T18:36:10.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.278+0000 7f2cdf800700 1 --2- 172.21.15.33:0/184499420 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2cd810ff50 0x7f2cd8112340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:10.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.278+0000 7f2cdf800700 1 --2- 172.21.15.33:0/184499420 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2cd806b1b0 0x7f2cd806b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:10.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.278+0000 7f2cdf800700 1 --2- 172.21.15.33:0/184499420 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2cd806bad0 0x7f2cd810fa10 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:10.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.278+0000 7f2cdf800700 1 -- 172.21.15.33:0/184499420 >> 172.21.15.33:0/184499420 conn(0x7f2cd8079350 msgr2=0x7f2cd8079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:36:10.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.279+0000 7f2cdf800700 1 -- 172.21.15.33:0/184499420 shutdown_connections 2024-05-03T18:36:10.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.279+0000 7f2cdf800700 1 -- 172.21.15.33:0/184499420 wait complete. 2024-05-03T18:36:10.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.281+0000 7f2cdf800700 1 Processor -- start 2024-05-03T18:36:10.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.281+0000 7f2cdf800700 1 -- start start 2024-05-03T18:36:10.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.283+0000 7f2cdf800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2cd806b1b0 0x7f2cd810e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:10.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.283+0000 7f2cdf800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2cd806bad0 0x7f2cd810e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:10.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.283+0000 7f2cdf800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2cd810ff50 0x7f2cd81abca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:10.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.283+0000 7f2cdf800700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd8105600 con 0x7f2cd806b1b0 2024-05-03T18:36:10.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.283+0000 7f2cdf800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd8105480 con 0x7f2cd810ff50 2024-05-03T18:36:10.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.283+0000 7f2cdf800700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd8105780 con 0x7f2cd806bad0 2024-05-03T18:36:10.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.283+0000 7f2cd7e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2cd806bad0 0x7f2cd810e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:10.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.283+0000 7f2cd7e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2cd806bad0 0x7f2cd810e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:37030/0 (socket says 172.21.15.33:37030) 2024-05-03T18:36:10.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.284+0000 7f2cd7e00700 1 -- 172.21.15.33:0/1870517474 learned_addr learned my addr 172.21.15.33:0/1870517474 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:36:10.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.284+0000 7f2cd7e00700 1 -- 172.21.15.33:0/1870517474 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2cd810ff50 msgr2=0x7f2cd81abca0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-03T18:36:10.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.285+0000 7f2cd7e00700 1 --2- 172.21.15.33:0/1870517474 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2cd810ff50 0x7f2cd81abca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:10.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.285+0000 7f2cd7e00700 1 -- 172.21.15.33:0/1870517474 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2cd806b1b0 msgr2=0x7f2cd810e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:36:10.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.285+0000 7f2cd7e00700 1 --2- 172.21.15.33:0/1870517474 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2cd806b1b0 0x7f2cd810e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:10.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.285+0000 7f2cd7e00700 1 -- 172.21.15.33:0/1870517474 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cd81ac430 con 0x7f2cd806bad0 2024-05-03T18:36:10.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.285+0000 7f2cd7e00700 1 --2- 172.21.15.33:0/1870517474 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2cd806bad0 0x7f2cd810e980 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2cc4006010 tx=0x7f2cc4005ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:36:10.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.286+0000 7f2cd5600700 1 -- 172.21.15.33:0/1870517474 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc4027070 con 0x7f2cd806bad0 2024-05-03T18:36:10.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.286+0000 7f2cd5600700 1 -- 172.21.15.33:0/1870517474 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2cc400fb80 con 0x7f2cd806bad0 2024-05-03T18:36:10.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.286+0000 7f2cd5600700 1 -- 172.21.15.33:0/1870517474 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc4022c00 con 0x7f2cd806bad0 2024-05-03T18:36:10.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.286+0000 7f2cdf800700 1 -- 172.21.15.33:0/1870517474 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cd81ac6c0 con 0x7f2cd806bad0 2024-05-03T18:36:10.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.286+0000 7f2cdf800700 1 -- 172.21.15.33:0/1870517474 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cd81acac0 con 0x7f2cd806bad0 2024-05-03T18:36:10.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.292+0000 7f2cd5600700 1 -- 172.21.15.33:0/1870517474 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f2cc402a070 con 0x7f2cd806bad0 2024-05-03T18:36:10.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.297+0000 7f2cea1e5b80 1 Processor -- start 2024-05-03T18:36:10.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.301+0000 7f2cea1e5b80 1 -- start start 2024-05-03T18:36:10.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.301+0000 7f2cd5600700 1 --2- 172.21.15.33:0/1870517474 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f2cc00b2a70 0x7f2cc00b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:10.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.301+0000 7f2cea1e5b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f6df3d9b0 0x557f6df3dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:10.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.302+0000 7f2cea1e5b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557f6df3e2d0 0x557f6df37fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:10.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.302+0000 7f2cea1e5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557f6df385d0 0x557f6df38a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:10.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.302+0000 7f2cea1e5b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x557f6de8aaa0 con 0x557f6df3e2d0 2024-05-03T18:36:10.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.302+0000 7f2cea1e5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x557f6de287e0 con 0x557f6df385d0 2024-05-03T18:36:10.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.302+0000 7f2cea1e5b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x557f6de86a60 con 0x557f6df3d9b0 2024-05-03T18:36:10.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.303+0000 7f2cd7e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557f6df3e2d0 0x557f6df37fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:10.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.303+0000 7f2cd7e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557f6df3e2d0 0x557f6df37fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50520/0 (socket says 172.21.15.33:50520) 2024-05-03T18:36:10.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.303+0000 7f2cd7e00700 1 -- 172.21.15.33:0/1826367186 learned_addr learned my addr 172.21.15.33:0/1826367186 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:36:10.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.303+0000 7f2cdd800700 1 --2- 172.21.15.33:0/1870517474 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f2cc00b2a70 0x7f2cc00b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:10.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.303+0000 7f2cdce00700 1 --2- 172.21.15.33:0/1826367186 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f6df3d9b0 0x557f6df3dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:10.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.304+0000 7f2cdd800700 1 --2- 172.21.15.33:0/1826367186 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557f6df385d0 0x557f6df38a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:10.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.305+0000 7f2cd7e00700 1 -- 172.21.15.33:0/1826367186 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f6df3d9b0 msgr2=0x557f6df3dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:36:10.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.305+0000 7f2cd7e00700 1 --2- 172.21.15.33:0/1826367186 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f6df3d9b0 0x557f6df3dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:10.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.305+0000 7f2cd7e00700 1 -- 172.21.15.33:0/1826367186 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557f6df385d0 msgr2=0x557f6df38a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:36:10.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.305+0000 7f2cd5600700 1 -- 172.21.15.33:0/1870517474 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9606+0+0 (secure 0 0 0) 0x7f2cc40e3a70 con 0x7f2cd806bad0 2024-05-03T18:36:10.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.305+0000 7f2cd7e00700 1 --2- 172.21.15.33:0/1826367186 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557f6df385d0 0x557f6df38a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:10.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.305+0000 7f2cd7e00700 1 -- 172.21.15.33:0/1826367186 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f6ddd7950 con 0x557f6df3e2d0 2024-05-03T18:36:10.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.305+0000 7f2cdce00700 1 --2- 172.21.15.33:0/1826367186 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f6df3d9b0 0x557f6df3dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:36:10.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.305+0000 7f2cdd800700 1 --2- 172.21.15.33:0/1826367186 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557f6df385d0 0x557f6df38a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:36:10.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.306+0000 7f2cd7e00700 1 --2- 172.21.15.33:0/1826367186 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557f6df3e2d0 0x557f6df37fb0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f2cc40e0cf0 tx=0x7f2cc4009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d32c9f9de09766c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:36:10.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.307+0000 7f2cca000700 1 -- 172.21.15.33:0/1826367186 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc40aa2f0 con 0x557f6df3e2d0 2024-05-03T18:36:10.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.307+0000 7f2cca000700 1 -- 172.21.15.33:0/1826367186 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2cc40e71c0 con 0x557f6df3e2d0 2024-05-03T18:36:10.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.307+0000 7f2cca000700 1 -- 172.21.15.33:0/1826367186 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc40a6130 con 0x557f6df3e2d0 2024-05-03T18:36:10.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.308+0000 7f2cea1e5b80 1 -- 172.21.15.33:0/1826367186 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557f6df3e2d0 msgr2=0x557f6df37fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:36:10.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.308+0000 7f2cdd800700 1 --2- 172.21.15.33:0/1870517474 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f2cc00b2a70 0x7f2cc00b4f30 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2cd0009710 tx=0x7f2cd0009620 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:36:10.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.308+0000 7f2cea1e5b80 1 --2- 172.21.15.33:0/1826367186 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557f6df3e2d0 0x557f6df37fb0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f2cc40e0cf0 tx=0x7f2cc4009310 comp rx=0 tx=0).stop 2024-05-03T18:36:10.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.309+0000 7f2cea1e5b80 1 -- 172.21.15.33:0/1826367186 shutdown_connections 2024-05-03T18:36:10.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.309+0000 7f2cea1e5b80 1 --2- 172.21.15.33:0/1826367186 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557f6df385d0 0x557f6df38a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:10.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.309+0000 7f2cea1e5b80 1 --2- 172.21.15.33:0/1826367186 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f6df3d9b0 0x557f6df3dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:10.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.309+0000 7f2cea1e5b80 1 --2- 172.21.15.33:0/1826367186 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557f6df3e2d0 0x557f6df37fb0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:10.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.309+0000 7f2cea1e5b80 1 -- 172.21.15.33:0/1826367186 >> 172.21.15.33:0/1826367186 conn(0x557f6deb0750 msgr2=0x557f6df32980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:36:10.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.309+0000 7f2cea1e5b80 1 -- 172.21.15.33:0/1826367186 shutdown_connections 2024-05-03T18:36:10.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.310+0000 7f2cea1e5b80 1 -- 172.21.15.33:0/1826367186 wait complete. 2024-05-03T18:36:10.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.311+0000 7f2cea1e5b80 1 -- start start 2024-05-03T18:36:10.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.313+0000 7f2cea1e5b80 1 Processor -- start 2024-05-03T18:36:10.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.314+0000 7f2cea1e5b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f6df385d0 0x557f6df43ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:10.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.314+0000 7f2cea1e5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557f6df3d9b0 0x557f6df41280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:10.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.314+0000 7f2cea1e5b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557f6df3e2d0 0x557f6df417c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:10.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.315+0000 7f2cd7e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557f6df3d9b0 0x557f6df41280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:10.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.315+0000 7f2cd7e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557f6df3d9b0 0x557f6df41280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:46236/0 (socket says 172.21.15.33:46236) 2024-05-03T18:36:10.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.315+0000 7f2cdd800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557f6df3e2d0 0x557f6df417c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:10.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.315+0000 7f2cdd800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557f6df3e2d0 0x557f6df417c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50524/0 (socket says 172.21.15.33:50524) 2024-05-03T18:36:10.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.316+0000 7f2cea1e5b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x557f6de8aaa0 con 0x557f6df3e2d0 2024-05-03T18:36:10.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.316+0000 7f2cea1e5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x557f6ddf0ba0 con 0x557f6df3d9b0 2024-05-03T18:36:10.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.316+0000 7f2cea1e5b80 1 -- 172.21.15.33:0/3677152659 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x557f6df44b40 con 0x557f6df385d0 2024-05-03T18:36:10.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.316+0000 7f2cd7e00700 1 -- 172.21.15.33:0/3677152659 learned_addr learned my addr 172.21.15.33:0/3677152659 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:36:10.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.316+0000 7f2cea1e5b80 10 client.0 _ll_register_callbacks cb 0x557f6de31cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:36:10.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.317+0000 7f2cdd800700 1 -- 172.21.15.33:0/3677152659 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f6df385d0 msgr2=0x557f6df43ee0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-03T18:36:10.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.318+0000 7f2cdd800700 1 --2- 172.21.15.33:0/3677152659 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f6df385d0 0x557f6df43ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:10.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.318+0000 7f2cdd800700 1 -- 172.21.15.33:0/3677152659 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557f6df3d9b0 msgr2=0x557f6df41280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:36:10.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.318+0000 7f2cdd800700 1 --2- 172.21.15.33:0/3677152659 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557f6df3d9b0 0x557f6df41280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:10.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.318+0000 7f2cdd800700 1 -- 172.21.15.33:0/3677152659 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f6de504a0 con 0x557f6df3e2d0 2024-05-03T18:36:10.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.318+0000 7f2cd7e00700 1 --2- 172.21.15.33:0/3677152659 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557f6df3d9b0 0x557f6df41280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:36:10.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.320+0000 7f2cdd800700 1 --2- 172.21.15.33:0/3677152659 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557f6df3e2d0 0x557f6df417c0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x557f6df38f90 tx=0x7f2cd0004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7442ff31e42a28a server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:36:10.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.320+0000 7f2cbea00700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:36:10.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.321+0000 7f2cbea00700 1 -- 172.21.15.33:0/3677152659 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd00180d0 con 0x557f6df3e2d0 2024-05-03T18:36:10.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.321+0000 7f2cbea00700 1 -- 172.21.15.33:0/3677152659 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9606+0+0 (secure 0 0 0) 0x7f2cd0004bb0 con 0x557f6df3e2d0 2024-05-03T18:36:10.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.321+0000 7f2cea1e5b80 1 -- 172.21.15.33:0/3677152659 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557f6df05070 con 0x557f6df3e2d0 2024-05-03T18:36:10.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.322+0000 7f2cbea00700 1 -- 172.21.15.33:0/3677152659 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f2cd0018270 con 0x557f6df3e2d0 2024-05-03T18:36:10.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.322+0000 7f2cea1e5b80 10 client.16293 fetch_fsmap learned FSMap version 21 2024-05-03T18:36:10.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.322+0000 7f2cea1e5b80 1 -- 172.21.15.33:0/3677152659 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x557f6df4ef20 con 0x557f6df3e2d0 2024-05-03T18:36:10.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.322+0000 7f2cbea00700 1 -- 172.21.15.33:0/3677152659 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x557f6df05070 con 0x557f6df3e2d0 2024-05-03T18:36:10.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.323+0000 7f2cbea00700 1 -- 172.21.15.33:0/3677152659 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f2cd002b630 con 0x557f6df3e2d0 2024-05-03T18:36:10.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.323+0000 7f2cea1e5b80 10 client.16293 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:36:10.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.323+0000 7f2cea1e5b80 10 client.16293 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:36:10.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.323+0000 7f2cea1e5b80 20 client.16293 populate_metadata read hostname 'smithi033' 2024-05-03T18:36:10.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.323+0000 7f2cea1e5b80 1 --2- 172.21.15.33:0/3677152659 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x557f6df514b0 0x557f6df53920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:10.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.323+0000 7f2cea1e5b80 4 client.16293 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:36:10.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.323+0000 7f2cea1e5b80 1 -- 172.21.15.33:0/3677152659 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557f6de98480 con 0x557f6df514b0 2024-05-03T18:36:10.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.324+0000 7f2cdce00700 1 --2- 172.21.15.33:0/3677152659 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x557f6df514b0 0x557f6df53920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:10.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.324+0000 7f2cbea00700 10 client.16293 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:36:10.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.326+0000 7f2cdce00700 1 --2- 172.21.15.33:0/3677152659 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x557f6df514b0 0x557f6df53920 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad913f0e7f4364eb server_cookie=aac21ad8afdb486d in_seq=0 out_seq=0 2024-05-03T18:36:10.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.327+0000 7f2cbea00700 1 -- 172.21.15.33:0/3677152659 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557f6de98480 con 0x557f6df514b0 2024-05-03T18:36:10.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.327+0000 7f2cbea00700 10 client.16293 handle_command_reply: tid=0 2024-05-03T18:36:10.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.328+0000 7f2cea1e5b80 1 client.16293 shutdown 2024-05-03T18:36:10.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.328+0000 7f2cea1e5b80 1 -- 172.21.15.33:0/3677152659 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557f6df3e2d0 msgr2=0x557f6df417c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:36:10.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.328+0000 7f2cea1e5b80 1 --2- 172.21.15.33:0/3677152659 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557f6df3e2d0 0x557f6df417c0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x557f6df38f90 tx=0x7f2cd0004b40 comp rx=0 tx=0).stop 2024-05-03T18:36:10.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.329+0000 7f2cea1e5b80 1 -- 172.21.15.33:0/3677152659 shutdown_connections 2024-05-03T18:36:10.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.329+0000 7f2cea1e5b80 1 --2- 172.21.15.33:0/3677152659 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x557f6df514b0 0x557f6df53920 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:10.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.329+0000 7f2cea1e5b80 1 --2- 172.21.15.33:0/3677152659 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557f6df3e2d0 0x557f6df417c0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:10.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.329+0000 7f2cea1e5b80 1 --2- 172.21.15.33:0/3677152659 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f6df385d0 0x557f6df43ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:10.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.329+0000 7f2cea1e5b80 1 --2- 172.21.15.33:0/3677152659 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557f6df3d9b0 0x557f6df41280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:10.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.329+0000 7f2cea1e5b80 1 -- 172.21.15.33:0/3677152659 >> 172.21.15.33:0/3677152659 conn(0x557f6deb0750 msgr2=0x557f6df2d430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:36:10.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.329+0000 7f2cbea00700 0 client.16293 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:36:10.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.329+0000 7f2cea1e5b80 1 -- 172.21.15.33:0/3677152659 shutdown_connections 2024-05-03T18:36:10.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.329+0000 7f2cea1e5b80 1 -- 172.21.15.33:0/3677152659 wait complete. 2024-05-03T18:36:10.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.330+0000 7f2cea1e5b80 20 client.16293 trim_cache size 0 max 16384 2024-05-03T18:36:10.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.370+0000 7f2cea1e5b80 1 Processor -- start 2024-05-03T18:36:10.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.371+0000 7f2cea1e5b80 1 -- start start 2024-05-03T18:36:10.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.371+0000 7f2cea1e5b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f6deb0750 0x557f6df37fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:10.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.371+0000 7f2cea1e5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557f6df385d0 0x557f6df40280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:10.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.371+0000 7f2cea1e5b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557f6df3d9b0 0x557f6df591f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:10.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.371+0000 7f2cea1e5b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd81b3c60 con 0x557f6df3d9b0 2024-05-03T18:36:10.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.371+0000 7f2cea1e5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd81b64c0 con 0x557f6df385d0 2024-05-03T18:36:10.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.371+0000 7f2cea1e5b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd81b42b0 con 0x557f6deb0750 2024-05-03T18:36:10.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.372+0000 7f2cd7e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557f6df385d0 0x557f6df40280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:10.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.372+0000 7f2cd7e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557f6df385d0 0x557f6df40280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:46244/0 (socket says 172.21.15.33:46244) 2024-05-03T18:36:10.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.372+0000 7f2cd7e00700 1 -- 172.21.15.33:0/3380854368 learned_addr learned my addr 172.21.15.33:0/3380854368 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:36:10.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.374+0000 7f2cd7e00700 1 -- 172.21.15.33:0/3380854368 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f6deb0750 msgr2=0x557f6df37fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:36:10.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.374+0000 7f2cd7e00700 1 --2- 172.21.15.33:0/3380854368 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f6deb0750 0x557f6df37fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:10.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.374+0000 7f2cd7e00700 1 -- 172.21.15.33:0/3380854368 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557f6df3d9b0 msgr2=0x557f6df591f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:36:10.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.374+0000 7f2cd7e00700 1 --2- 172.21.15.33:0/3380854368 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557f6df3d9b0 0x557f6df591f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:10.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.374+0000 7f2cd7e00700 1 -- 172.21.15.33:0/3380854368 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f6df3f7f0 con 0x557f6df385d0 2024-05-03T18:36:10.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.375+0000 7f2cd7e00700 1 --2- 172.21.15.33:0/3380854368 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557f6df385d0 0x557f6df40280 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2cc40a6f40 tx=0x7f2cc40cd040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d45ed4f3ea6a729 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:36:10.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.376+0000 7f2cc9600700 1 -- 172.21.15.33:0/3380854368 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc40aa110 con 0x557f6df385d0 2024-05-03T18:36:10.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.376+0000 7f2cc9600700 1 -- 172.21.15.33:0/3380854368 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2cc40ab0a0 con 0x557f6df385d0 2024-05-03T18:36:10.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.376+0000 7f2cc9600700 1 -- 172.21.15.33:0/3380854368 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc40aac00 con 0x557f6df385d0 2024-05-03T18:36:10.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.377+0000 7f2cea1e5b80 1 -- 172.21.15.33:0/3380854368 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557f6df385d0 msgr2=0x557f6df40280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:36:10.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.377+0000 7f2cea1e5b80 1 --2- 172.21.15.33:0/3380854368 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557f6df385d0 0x557f6df40280 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2cc40a6f40 tx=0x7f2cc40cd040 comp rx=0 tx=0).stop 2024-05-03T18:36:10.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.378+0000 7f2cea1e5b80 1 -- 172.21.15.33:0/3380854368 shutdown_connections 2024-05-03T18:36:10.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.378+0000 7f2cea1e5b80 1 --2- 172.21.15.33:0/3380854368 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557f6df3d9b0 0x557f6df591f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:10.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.378+0000 7f2cea1e5b80 1 --2- 172.21.15.33:0/3380854368 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f6deb0750 0x557f6df37fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:10.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.378+0000 7f2cea1e5b80 1 --2- 172.21.15.33:0/3380854368 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557f6df385d0 0x557f6df40280 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:10.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.378+0000 7f2cea1e5b80 1 -- 172.21.15.33:0/3380854368 >> 172.21.15.33:0/3380854368 conn(0x557f6df34880 msgr2=0x557f6df3fb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:36:10.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.380+0000 7f2cea1e5b80 1 -- 172.21.15.33:0/3380854368 shutdown_connections 2024-05-03T18:36:10.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.380+0000 7f2cea1e5b80 1 -- 172.21.15.33:0/3380854368 wait complete. 2024-05-03T18:36:10.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.380+0000 7f2cea1e5b80 1 -- start start 2024-05-03T18:36:10.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.381+0000 7f2cea1e5b80 1 Processor -- start 2024-05-03T18:36:10.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:36:10 smithi150 ceph-mon[32492]: pgmap v277: 129 pgs: 129 active+clean; 2.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 6.2 MiB/s wr, 639 op/s 2024-05-03T18:36:10.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.381+0000 7f2cea1e5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557f6deb0750 0x557f6df4a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:10.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.381+0000 7f2cea1e5b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f6df385d0 0x557f6dfb6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:10.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.381+0000 7f2cea1e5b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557f6df3d9b0 0x557f6dfb7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:10.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.381+0000 7f2cea1e5b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd81b4980 con 0x557f6df3d9b0 2024-05-03T18:36:10.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.381+0000 7f2cea1e5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd81b5df0 con 0x557f6deb0750 2024-05-03T18:36:10.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.381+0000 7f2cea1e5b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd81b5050 con 0x557f6df385d0 2024-05-03T18:36:10.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.381+0000 7f2cea1e5b80 10 client.0 _ll_register_callbacks cb 0x557f6de31cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:36:10.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.382+0000 7f2cdd800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557f6df3d9b0 0x557f6dfb7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:10.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.382+0000 7f2cd7e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f6df385d0 0x557f6dfb6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:10.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.382+0000 7f2cd7e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f6df385d0 0x557f6dfb6e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:37060/0 (socket says 172.21.15.33:37060) 2024-05-03T18:36:10.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.382+0000 7f2cd7e00700 1 -- 172.21.15.33:0/788449706 learned_addr learned my addr 172.21.15.33:0/788449706 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:36:10.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.383+0000 7f2cdce00700 1 --2- 172.21.15.33:0/788449706 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557f6deb0750 0x557f6df4a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:10.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.383+0000 7f2cd7e00700 1 -- 172.21.15.33:0/788449706 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557f6deb0750 msgr2=0x557f6df4a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:36:10.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.383+0000 7f2cd7e00700 1 --2- 172.21.15.33:0/788449706 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557f6deb0750 0x557f6df4a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:10.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.383+0000 7f2cd7e00700 1 -- 172.21.15.33:0/788449706 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557f6df3d9b0 msgr2=0x557f6dfb7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:36:10.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.383+0000 7f2cd7e00700 1 --2- 172.21.15.33:0/788449706 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557f6df3d9b0 0x557f6dfb7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:10.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.383+0000 7f2cd7e00700 1 -- 172.21.15.33:0/788449706 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f6dfb79c0 con 0x557f6df385d0 2024-05-03T18:36:10.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.383+0000 7f2cdd800700 1 --2- 172.21.15.33:0/788449706 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557f6df3d9b0 0x557f6dfb7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:36:10.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.383+0000 7f2cdce00700 1 --2- 172.21.15.33:0/788449706 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557f6deb0750 0x557f6df4a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:36:10.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.384+0000 7f2cd7e00700 1 --2- 172.21.15.33:0/788449706 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f6df385d0 0x557f6dfb6e10 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2cc40aa6a0 tx=0x7f2cc40b1300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=410063b6fd24c12c server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:36:10.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.384+0000 7f2cbea00700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-03T18:36:10.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.384+0000 7f2cbea00700 1 -- 172.21.15.33:0/788449706 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc40c3070 con 0x557f6df385d0 2024-05-03T18:36:10.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.384+0000 7f2cbea00700 1 -- 172.21.15.33:0/788449706 <== mon.1 v2:172.21.15.150:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9606+0+0 (secure 0 0 0) 0x7f2cc40aae10 con 0x557f6df385d0 2024-05-03T18:36:10.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.384+0000 7f2cea1e5b80 1 -- 172.21.15.33:0/788449706 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2cc40ab0a0 con 0x557f6df385d0 2024-05-03T18:36:10.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.385+0000 7f2cbea00700 1 -- 172.21.15.33:0/788449706 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f2cc40aa9a0 con 0x557f6df385d0 2024-05-03T18:36:10.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.385+0000 7f2cea1e5b80 10 client.26008 fetch_fsmap learned FSMap version 21 2024-05-03T18:36:10.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.385+0000 7f2cea1e5b80 1 -- 172.21.15.33:0/788449706 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x557f6df48cc0 con 0x557f6df385d0 2024-05-03T18:36:10.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.385+0000 7f2cbea00700 1 -- 172.21.15.33:0/788449706 <== mon.1 v2:172.21.15.150:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2cc40ab0a0 con 0x557f6df385d0 2024-05-03T18:36:10.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.385+0000 7f2cbea00700 1 -- 172.21.15.33:0/788449706 <== mon.1 v2:172.21.15.150:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f2cc40cbcd0 con 0x557f6df385d0 2024-05-03T18:36:10.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.385+0000 7f2cea1e5b80 10 client.26008 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:36:10.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.385+0000 7f2cea1e5b80 10 client.26008 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:36:10.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.385+0000 7f2cea1e5b80 20 client.26008 populate_metadata read hostname 'smithi033' 2024-05-03T18:36:10.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.385+0000 7f2cea1e5b80 1 --2- 172.21.15.33:0/788449706 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x557f6df3e2d0 0x557f6df49150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:10.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.385+0000 7f2cea1e5b80 4 client.26008 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-03T18:36:10.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.385+0000 7f2cea1e5b80 1 -- 172.21.15.33:0/788449706 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x557f6de92520 con 0x557f6df3e2d0 2024-05-03T18:36:10.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.386+0000 7f2cdce00700 1 --2- 172.21.15.33:0/788449706 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x557f6df3e2d0 0x557f6df49150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:10.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.387+0000 7f2cdce00700 1 --2- 172.21.15.33:0/788449706 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x557f6df3e2d0 0x557f6df49150 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3fde640543787ee7 server_cookie=4d3b072507b95583 in_seq=0 out_seq=0 2024-05-03T18:36:10.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.387+0000 7f2cbea00700 10 client.26008 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:36:10.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.390+0000 7f2cbea00700 1 -- 172.21.15.33:0/788449706 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x557f6de92520 con 0x557f6df3e2d0 2024-05-03T18:36:10.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.390+0000 7f2cbea00700 10 client.26008 handle_command_reply: tid=0 2024-05-03T18:36:10.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.390+0000 7f2cea1e5b80 1 client.26008 shutdown 2024-05-03T18:36:10.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.391+0000 7f2cea1e5b80 1 -- 172.21.15.33:0/788449706 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f6df385d0 msgr2=0x557f6dfb6e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:36:10.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.391+0000 7f2cea1e5b80 1 --2- 172.21.15.33:0/788449706 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f6df385d0 0x557f6dfb6e10 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2cc40aa6a0 tx=0x7f2cc40b1300 comp rx=0 tx=0).stop 2024-05-03T18:36:10.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.391+0000 7f2cea1e5b80 1 -- 172.21.15.33:0/788449706 shutdown_connections 2024-05-03T18:36:10.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.391+0000 7f2cea1e5b80 1 --2- 172.21.15.33:0/788449706 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x557f6df3e2d0 0x557f6df49150 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:10.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.391+0000 7f2cea1e5b80 1 --2- 172.21.15.33:0/788449706 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557f6df3d9b0 0x557f6dfb7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:10.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.391+0000 7f2cea1e5b80 1 --2- 172.21.15.33:0/788449706 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557f6deb0750 0x557f6df4a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:10.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.391+0000 7f2cea1e5b80 1 --2- 172.21.15.33:0/788449706 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f6df385d0 0x557f6dfb6e10 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:10.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.391+0000 7f2cea1e5b80 1 -- 172.21.15.33:0/788449706 >> 172.21.15.33:0/788449706 conn(0x557f6df34880 msgr2=0x557f6df2d430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:36:10.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.392+0000 7f2cbea00700 0 client.26008 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:36:10.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.392+0000 7f2cea1e5b80 1 -- 172.21.15.33:0/788449706 shutdown_connections 2024-05-03T18:36:10.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.392+0000 7f2cea1e5b80 1 -- 172.21.15.33:0/788449706 wait complete. 2024-05-03T18:36:10.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.393+0000 7f2cea1e5b80 20 client.26008 trim_cache size 0 max 16384 2024-05-03T18:36:10.394 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-03T18:36:10.394 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-05-03T18:36:10.394 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "4b42d3ec-28c5-448b-bb2d-8104e29ff29b", 2024-05-03T18:36:10.394 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-05-03T18:36:10.394 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-03T18:36:10.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.395+0000 7f2cdf800700 1 -- 172.21.15.33:0/1870517474 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f2cc00b2a70 msgr2=0x7f2cc00b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:36:10.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.395+0000 7f2cdf800700 1 --2- 172.21.15.33:0/1870517474 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f2cc00b2a70 0x7f2cc00b4f30 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2cd0009710 tx=0x7f2cd0009620 comp rx=0 tx=0).stop 2024-05-03T18:36:10.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.395+0000 7f2cdf800700 1 -- 172.21.15.33:0/1870517474 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2cd806bad0 msgr2=0x7f2cd810e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:36:10.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.395+0000 7f2cdf800700 1 --2- 172.21.15.33:0/1870517474 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2cd806bad0 0x7f2cd810e980 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2cc4006010 tx=0x7f2cc4005ef0 comp rx=0 tx=0).stop 2024-05-03T18:36:10.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.395+0000 7f2cdf800700 1 -- 172.21.15.33:0/1870517474 shutdown_connections 2024-05-03T18:36:10.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.395+0000 7f2cdf800700 1 --2- 172.21.15.33:0/1870517474 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f2cc00b2a70 0x7f2cc00b4f30 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:10.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.395+0000 7f2cdf800700 1 --2- 172.21.15.33:0/1870517474 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2cd810ff50 0x7f2cd81abca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:10.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.395+0000 7f2cdf800700 1 --2- 172.21.15.33:0/1870517474 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2cd806b1b0 0x7f2cd810e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:10.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.395+0000 7f2cdf800700 1 --2- 172.21.15.33:0/1870517474 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2cd806bad0 0x7f2cd810e980 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:10.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.395+0000 7f2cdf800700 1 -- 172.21.15.33:0/1870517474 >> 172.21.15.33:0/1870517474 conn(0x7f2cd8079350 msgr2=0x7f2cd8111690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:36:10.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.397+0000 7f2cdf800700 1 -- 172.21.15.33:0/1870517474 shutdown_connections 2024-05-03T18:36:10.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:10.397+0000 7f2cdf800700 1 -- 172.21.15.33:0/1870517474 wait complete. 2024-05-03T18:36:10.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:36:10 smithi099 ceph-mon[31658]: pgmap v277: 129 pgs: 129 active+clean; 2.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 6.2 MiB/s wr, 639 op/s 2024-05-03T18:36:11.408 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-03T18:36:11.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:36:11 smithi099 ceph-mon[31658]: scrub queued for path: / 2024-05-03T18:36:11.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:36:11 smithi099 ceph-mon[31658]: scrub summary: idle+waiting paths [/] 2024-05-03T18:36:11.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:36:11 smithi099 ceph-mon[31658]: scrub summary: active paths [/] 2024-05-03T18:36:11.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:36:11 smithi099 ceph-mon[31658]: scrub summary: idle+waiting paths [/] 2024-05-03T18:36:11.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:36:11 smithi099 ceph-mon[31658]: scrub summary: idle 2024-05-03T18:36:11.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:36:11 smithi150 ceph-mon[32492]: scrub queued for path: / 2024-05-03T18:36:11.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:36:11 smithi150 ceph-mon[32492]: scrub summary: idle+waiting paths [/] 2024-05-03T18:36:11.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:36:11 smithi150 ceph-mon[32492]: scrub summary: active paths [/] 2024-05-03T18:36:11.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:36:11 smithi150 ceph-mon[32492]: scrub summary: idle+waiting paths [/] 2024-05-03T18:36:11.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:36:11 smithi150 ceph-mon[32492]: scrub summary: idle 2024-05-03T18:36:11.738 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:36:11 smithi033 ceph-mon[28330]: scrub queued for path: / 2024-05-03T18:36:11.738 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:36:11 smithi033 ceph-mon[28330]: scrub summary: idle+waiting paths [/] 2024-05-03T18:36:11.738 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:36:11 smithi033 ceph-mon[28330]: scrub summary: active paths [/] 2024-05-03T18:36:11.738 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:36:11 smithi033 ceph-mon[28330]: scrub summary: idle+waiting paths [/] 2024-05-03T18:36:11.738 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:36:11 smithi033 ceph-mon[28330]: scrub summary: idle 2024-05-03T18:36:12.145 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:36:12.555 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:36:12 smithi033 ceph-mon[28330]: pgmap v278: 129 pgs: 129 active+clean; 2.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 6.2 MiB/s wr, 639 op/s 2024-05-03T18:36:12.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:36:12 smithi150 ceph-mon[32492]: pgmap v278: 129 pgs: 129 active+clean; 2.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 6.2 MiB/s wr, 639 op/s 2024-05-03T18:36:12.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:36:12 smithi099 ceph-mon[31658]: pgmap v278: 129 pgs: 129 active+clean; 2.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 6.2 MiB/s wr, 639 op/s 2024-05-03T18:36:13.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:36:13 smithi150 ceph-mon[32492]: osdmap e96: 12 total, 12 up, 12 in 2024-05-03T18:36:13.680 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:36:13 smithi033 ceph-mon[28330]: osdmap e96: 12 total, 12 up, 12 in 2024-05-03T18:36:13.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:36:13 smithi099 ceph-mon[31658]: osdmap e96: 12 total, 12 up, 12 in 2024-05-03T18:36:14.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:36:14 smithi150 ceph-mon[32492]: pgmap v280: 129 pgs: 129 active+clean; 2.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 7.9 MiB/s wr, 791 op/s 2024-05-03T18:36:14.698 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:36:14 smithi033 ceph-mon[28330]: pgmap v280: 129 pgs: 129 active+clean; 2.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 7.9 MiB/s wr, 791 op/s 2024-05-03T18:36:14.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:36:14 smithi099 ceph-mon[31658]: pgmap v280: 129 pgs: 129 active+clean; 2.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 7.9 MiB/s wr, 791 op/s 2024-05-03T18:36:15.110 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 300.0 s, 626.6 tps, lat 159.238 ms stddev 44.018 2024-05-03T18:36:15.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.313+0000 7f59fec00700 1 -- 172.21.15.33:0/3645284435 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59f8103550 msgr2=0x7f59f81039b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:36:15.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.313+0000 7f59fec00700 1 --2- 172.21.15.33:0/3645284435 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59f8103550 0x7f59f81039b0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f59e8009990 tx=0x7f59e8009ca0 comp rx=0 tx=0).stop 2024-05-03T18:36:15.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.313+0000 7f59fec00700 1 -- 172.21.15.33:0/3645284435 shutdown_connections 2024-05-03T18:36:15.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.313+0000 7f59fec00700 1 --2- 172.21.15.33:0/3645284435 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f59f8103ef0 0x7f59f81103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:15.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.313+0000 7f59fec00700 1 --2- 172.21.15.33:0/3645284435 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f59f8108460 0x7f59f8108840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:15.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.313+0000 7f59fec00700 1 --2- 172.21.15.33:0/3645284435 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59f8103550 0x7f59f81039b0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:15.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.313+0000 7f59fec00700 1 -- 172.21.15.33:0/3645284435 >> 172.21.15.33:0/3645284435 conn(0x7f59f807a600 msgr2=0x7f59f807aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:36:15.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.314+0000 7f59fec00700 1 -- 172.21.15.33:0/3645284435 shutdown_connections 2024-05-03T18:36:15.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.314+0000 7f59fec00700 1 -- 172.21.15.33:0/3645284435 wait complete. 2024-05-03T18:36:15.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.315+0000 7f59fec00700 1 Processor -- start 2024-05-03T18:36:15.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.316+0000 7f59fec00700 1 -- start start 2024-05-03T18:36:15.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.317+0000 7f59fec00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f59f8103550 0x7f59f81a07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:15.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.317+0000 7f59fec00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59f8103ef0 0x7f59f81a0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:15.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.317+0000 7f59fec00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f59f8108460 0x7f59f810f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:15.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.317+0000 7f59fec00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f59f8113590 con 0x7f59f8103ef0 2024-05-03T18:36:15.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.317+0000 7f59fec00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f59f8113410 con 0x7f59f8108460 2024-05-03T18:36:15.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.317+0000 7f59fec00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f59f8113710 con 0x7f59f8103550 2024-05-03T18:36:15.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.317+0000 7f59f7e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f59f8103550 0x7f59f81a07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:15.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.317+0000 7f59f7e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f59f8103550 0x7f59f81a07a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:37078/0 (socket says 172.21.15.33:37078) 2024-05-03T18:36:15.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.317+0000 7f59f7e00700 1 -- 172.21.15.33:0/2482685518 learned_addr learned my addr 172.21.15.33:0/2482685518 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:36:15.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.318+0000 7f59ef400700 1 --2- 172.21.15.33:0/2482685518 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59f8103ef0 0x7f59f81a0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:15.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.318+0000 7f59fcc00700 1 --2- 172.21.15.33:0/2482685518 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f59f8108460 0x7f59f810f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:15.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.319+0000 7f59fcc00700 1 -- 172.21.15.33:0/2482685518 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f59f8103550 msgr2=0x7f59f81a07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:36:15.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.319+0000 7f59fcc00700 1 --2- 172.21.15.33:0/2482685518 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f59f8103550 0x7f59f81a07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:15.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.319+0000 7f59fcc00700 1 -- 172.21.15.33:0/2482685518 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59f8103ef0 msgr2=0x7f59f81a0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:36:15.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.319+0000 7f59fcc00700 1 --2- 172.21.15.33:0/2482685518 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59f8103ef0 0x7f59f81a0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:15.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.319+0000 7f59fcc00700 1 -- 172.21.15.33:0/2482685518 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59f810ffb0 con 0x7f59f8108460 2024-05-03T18:36:15.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.319+0000 7f59f7e00700 1 --2- 172.21.15.33:0/2482685518 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f59f8103550 0x7f59f81a07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:36:15.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.320+0000 7f59fcc00700 1 --2- 172.21.15.33:0/2482685518 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f59f8108460 0x7f59f810f6b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f59f000efc0 tx=0x7f59f000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:36:15.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.320+0000 7f59f5600700 1 -- 172.21.15.33:0/2482685518 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59f00092e0 con 0x7f59f8108460 2024-05-03T18:36:15.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.320+0000 7f59fec00700 1 -- 172.21.15.33:0/2482685518 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59f8110240 con 0x7f59f8108460 2024-05-03T18:36:15.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.320+0000 7f59fec00700 1 -- 172.21.15.33:0/2482685518 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59f819c350 con 0x7f59f8108460 2024-05-03T18:36:15.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.321+0000 7f59f5600700 1 -- 172.21.15.33:0/2482685518 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f59f0004bb0 con 0x7f59f8108460 2024-05-03T18:36:15.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.321+0000 7f59f5600700 1 -- 172.21.15.33:0/2482685518 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59f0004020 con 0x7f59f8108460 2024-05-03T18:36:15.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.326+0000 7f59f5600700 1 -- 172.21.15.33:0/2482685518 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f59f001b030 con 0x7f59f8108460 2024-05-03T18:36:15.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.328+0000 7f5a09689b80 1 Processor -- start 2024-05-03T18:36:15.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.328+0000 7f5a09689b80 1 -- start start 2024-05-03T18:36:15.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.329+0000 7f59f5600700 1 --2- 172.21.15.33:0/2482685518 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f59dc0b2af0 0x7f59dc0b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:15.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.329+0000 7f5a09689b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fe477137b0 0x55fe47713b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:15.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.329+0000 7f59f5600700 1 -- 172.21.15.33:0/2482685518 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9518+0+0 (secure 0 0 0) 0x7f59f00e2390 con 0x7f59f8108460 2024-05-03T18:36:15.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.329+0000 7f5a09689b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fe477140d0 0x55fe4770efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:15.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.329+0000 7f59ef400700 1 --2- 172.21.15.33:0/2482685518 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f59dc0b2af0 0x7f59dc0b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:15.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.329+0000 7f5a09689b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe4770f5d0 0x55fe4770fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:15.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.329+0000 7f5a09689b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55fe47661a20 con 0x55fe477140d0 2024-05-03T18:36:15.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.329+0000 7f5a09689b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55fe475ff760 con 0x55fe4770f5d0 2024-05-03T18:36:15.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.329+0000 7f5a09689b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55fe4765d9e0 con 0x55fe477137b0 2024-05-03T18:36:15.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.330+0000 7f59f7e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fe477137b0 0x55fe47713b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:15.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.330+0000 7f59f7e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fe477137b0 0x55fe47713b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:37082/0 (socket says 172.21.15.33:37082) 2024-05-03T18:36:15.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.330+0000 7f59f7e00700 1 -- 172.21.15.33:0/1461115579 learned_addr learned my addr 172.21.15.33:0/1461115579 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:36:15.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.330+0000 7f59fcc00700 1 --2- 172.21.15.33:0/1461115579 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe4770f5d0 0x55fe4770fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:15.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.331+0000 7f59ef400700 1 --2- 172.21.15.33:0/1461115579 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fe477140d0 0x55fe4770efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:15.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.331+0000 7f59f7e00700 1 -- 172.21.15.33:0/1461115579 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe4770f5d0 msgr2=0x55fe4770fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:36:15.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.331+0000 7f59f7e00700 1 --2- 172.21.15.33:0/1461115579 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe4770f5d0 0x55fe4770fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:15.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.331+0000 7f59f7e00700 1 -- 172.21.15.33:0/1461115579 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fe477140d0 msgr2=0x55fe4770efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:36:15.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.331+0000 7f59f7e00700 1 --2- 172.21.15.33:0/1461115579 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fe477140d0 0x55fe4770efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:15.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.331+0000 7f59f7e00700 1 -- 172.21.15.33:0/1461115579 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe475ae8d0 con 0x55fe477137b0 2024-05-03T18:36:15.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.331+0000 7f59ef400700 1 --2- 172.21.15.33:0/1461115579 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fe477140d0 0x55fe4770efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:36:15.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.331+0000 7f59ef400700 1 --2- 172.21.15.33:0/2482685518 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f59dc0b2af0 0x7f59dc0b4fb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f59f81107d0 tx=0x7f59e80058e0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:36:15.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.331+0000 7f59f7e00700 1 --2- 172.21.15.33:0/1461115579 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fe477137b0 0x55fe47713b90 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f59e00046e0 tx=0x7f59e00118c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84099625129ca8ad server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:36:15.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.332+0000 7f59ed600700 1 -- 172.21.15.33:0/1461115579 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e0004830 con 0x55fe477137b0 2024-05-03T18:36:15.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.332+0000 7f59ed600700 1 -- 172.21.15.33:0/1461115579 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f59e0005020 con 0x55fe477137b0 2024-05-03T18:36:15.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.332+0000 7f59ed600700 1 -- 172.21.15.33:0/1461115579 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e000e6b0 con 0x55fe477137b0 2024-05-03T18:36:15.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.333+0000 7f5a09689b80 1 -- 172.21.15.33:0/1461115579 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fe477137b0 msgr2=0x55fe47713b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:36:15.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.333+0000 7f5a09689b80 1 --2- 172.21.15.33:0/1461115579 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fe477137b0 0x55fe47713b90 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f59e00046e0 tx=0x7f59e00118c0 comp rx=0 tx=0).stop 2024-05-03T18:36:15.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.333+0000 7f5a09689b80 1 -- 172.21.15.33:0/1461115579 shutdown_connections 2024-05-03T18:36:15.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.333+0000 7f5a09689b80 1 --2- 172.21.15.33:0/1461115579 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe4770f5d0 0x55fe4770fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:15.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.333+0000 7f5a09689b80 1 --2- 172.21.15.33:0/1461115579 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fe477137b0 0x55fe47713b90 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:15.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.333+0000 7f5a09689b80 1 --2- 172.21.15.33:0/1461115579 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fe477140d0 0x55fe4770efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:15.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.333+0000 7f5a09689b80 1 -- 172.21.15.33:0/1461115579 >> 172.21.15.33:0/1461115579 conn(0x55fe4763db60 msgr2=0x55fe477097f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:36:15.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.333+0000 7f5a09689b80 1 -- 172.21.15.33:0/1461115579 shutdown_connections 2024-05-03T18:36:15.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.334+0000 7f5a09689b80 1 -- 172.21.15.33:0/1461115579 wait complete. 2024-05-03T18:36:15.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.334+0000 7f5a09689b80 1 -- start start 2024-05-03T18:36:15.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.334+0000 7f5a09689b80 1 Processor -- start 2024-05-03T18:36:15.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.335+0000 7f5a09689b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fe4770f5d0 0x55fe4771ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:15.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.335+0000 7f5a09689b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe477137b0 0x55fe47718220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:15.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.335+0000 7f5a09689b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fe477140d0 0x55fe47718760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:15.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.335+0000 7f5a09689b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55fe4771bca0 con 0x55fe477140d0 2024-05-03T18:36:15.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.335+0000 7f5a09689b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55fe475c7b20 con 0x55fe477137b0 2024-05-03T18:36:15.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.335+0000 7f5a09689b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55fe4771be20 con 0x55fe4770f5d0 2024-05-03T18:36:15.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.335+0000 7f5a09689b80 10 client.0 _ll_register_callbacks cb 0x55fe47608c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:36:15.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.336+0000 7f59f7e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fe4770f5d0 0x55fe4771ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:15.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.336+0000 7f59ef400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe477137b0 0x55fe47718220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:15.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.336+0000 7f59f7e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fe4770f5d0 0x55fe4771ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:37084/0 (socket says 172.21.15.33:37084) 2024-05-03T18:36:15.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.336+0000 7f59f7e00700 1 -- 172.21.15.33:0/3293416399 learned_addr learned my addr 172.21.15.33:0/3293416399 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:36:15.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.336+0000 7f59fcc00700 1 --2- 172.21.15.33:0/3293416399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fe477140d0 0x55fe47718760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:15.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.337+0000 7f59f7e00700 1 -- 172.21.15.33:0/3293416399 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe477137b0 msgr2=0x55fe47718220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:36:15.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.337+0000 7f59f7e00700 1 --2- 172.21.15.33:0/3293416399 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe477137b0 0x55fe47718220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:15.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.337+0000 7f59f7e00700 1 -- 172.21.15.33:0/3293416399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fe477140d0 msgr2=0x55fe47718760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:36:15.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.337+0000 7f59f7e00700 1 --2- 172.21.15.33:0/3293416399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fe477140d0 0x55fe47718760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:15.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.337+0000 7f59f7e00700 1 -- 172.21.15.33:0/3293416399 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe47627420 con 0x55fe4770f5d0 2024-05-03T18:36:15.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.337+0000 7f59ef400700 1 --2- 172.21.15.33:0/3293416399 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe477137b0 0x55fe47718220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:36:15.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.337+0000 7f59fcc00700 1 --2- 172.21.15.33:0/3293416399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fe477140d0 0x55fe47718760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:36:15.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.338+0000 7f59f7e00700 1 --2- 172.21.15.33:0/3293416399 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fe4770f5d0 0x55fe4771ae80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f59e001b040 tx=0x7f59e0004980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e137f797bde3d172 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:36:15.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.338+0000 7f59e6000700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-03T18:36:15.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.339+0000 7f59e6000700 1 -- 172.21.15.33:0/3293416399 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e0026070 con 0x55fe4770f5d0 2024-05-03T18:36:15.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.339+0000 7f5a09689b80 1 -- 172.21.15.33:0/3293416399 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59e0005020 con 0x55fe4770f5d0 2024-05-03T18:36:15.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.339+0000 7f59e6000700 1 -- 172.21.15.33:0/3293416399 <== mon.1 v2:172.21.15.150:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9518+0+0 (secure 0 0 0) 0x7f59e000e470 con 0x55fe4770f5d0 2024-05-03T18:36:15.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.339+0000 7f59e6000700 1 -- 172.21.15.33:0/3293416399 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f59e0032050 con 0x55fe4770f5d0 2024-05-03T18:36:15.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.339+0000 7f5a09689b80 10 client.26020 fetch_fsmap learned FSMap version 21 2024-05-03T18:36:15.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.339+0000 7f5a09689b80 1 -- 172.21.15.33:0/3293416399 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55fe47726440 con 0x55fe4770f5d0 2024-05-03T18:36:15.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.340+0000 7f59e6000700 1 -- 172.21.15.33:0/3293416399 <== mon.1 v2:172.21.15.150:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f59e0005020 con 0x55fe4770f5d0 2024-05-03T18:36:15.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.341+0000 7f59e6000700 1 -- 172.21.15.33:0/3293416399 <== mon.1 v2:172.21.15.150:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f59e00224e0 con 0x55fe4770f5d0 2024-05-03T18:36:15.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.341+0000 7f5a09689b80 10 client.26020 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:36:15.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.341+0000 7f5a09689b80 10 client.26020 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:36:15.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.341+0000 7f5a09689b80 20 client.26020 populate_metadata read hostname 'smithi033' 2024-05-03T18:36:15.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.341+0000 7f5a09689b80 1 --2- 172.21.15.33:0/3293416399 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55fe47728970 0x55fe4772ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:15.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.341+0000 7f5a09689b80 4 client.26020 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:36:15.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.341+0000 7f5a09689b80 1 -- 172.21.15.33:0/3293416399 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fe476dc1d0 con 0x55fe47728970 2024-05-03T18:36:15.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.342+0000 7f59ef400700 1 --2- 172.21.15.33:0/3293416399 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55fe47728970 0x55fe4772ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:15.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.344+0000 7f59ef400700 1 --2- 172.21.15.33:0/3293416399 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55fe47728970 0x55fe4772ad60 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef1a472190027b94 server_cookie=17135c1a93ba887b in_seq=0 out_seq=0 2024-05-03T18:36:15.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.344+0000 7f59e6000700 10 client.26020 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:36:15.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.346+0000 7f59e6000700 1 -- 172.21.15.33:0/3293416399 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55fe476dc1d0 con 0x55fe47728970 2024-05-03T18:36:15.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.346+0000 7f59e6000700 10 client.26020 handle_command_reply: tid=0 2024-05-03T18:36:15.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.346+0000 7f5a09689b80 1 client.26020 shutdown 2024-05-03T18:36:15.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.347+0000 7f5a09689b80 1 -- 172.21.15.33:0/3293416399 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fe4770f5d0 msgr2=0x55fe4771ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:36:15.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.347+0000 7f5a09689b80 1 --2- 172.21.15.33:0/3293416399 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fe4770f5d0 0x55fe4771ae80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f59e001b040 tx=0x7f59e0004980 comp rx=0 tx=0).stop 2024-05-03T18:36:15.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.347+0000 7f5a09689b80 1 -- 172.21.15.33:0/3293416399 shutdown_connections 2024-05-03T18:36:15.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.347+0000 7f5a09689b80 1 --2- 172.21.15.33:0/3293416399 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55fe47728970 0x55fe4772ad60 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:15.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.347+0000 7f5a09689b80 1 --2- 172.21.15.33:0/3293416399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fe477140d0 0x55fe47718760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:15.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.348+0000 7f5a09689b80 1 --2- 172.21.15.33:0/3293416399 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fe4770f5d0 0x55fe4771ae80 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:15.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.348+0000 7f5a09689b80 1 --2- 172.21.15.33:0/3293416399 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe477137b0 0x55fe47718220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:15.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.348+0000 7f5a09689b80 1 -- 172.21.15.33:0/3293416399 >> 172.21.15.33:0/3293416399 conn(0x55fe4763db60 msgr2=0x55fe47708650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:36:15.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.348+0000 7f59e6000700 0 client.26020 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:36:15.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.348+0000 7f5a09689b80 1 -- 172.21.15.33:0/3293416399 shutdown_connections 2024-05-03T18:36:15.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.348+0000 7f5a09689b80 1 -- 172.21.15.33:0/3293416399 wait complete. 2024-05-03T18:36:15.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.349+0000 7f5a09689b80 20 client.26020 trim_cache size 0 max 16384 2024-05-03T18:36:15.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.386+0000 7f5a09689b80 1 Processor -- start 2024-05-03T18:36:15.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.386+0000 7f5a09689b80 1 -- start start 2024-05-03T18:36:15.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.387+0000 7f5a09689b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fe4770f5d0 0x55fe47724dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:15.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.387+0000 7f5a09689b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe477137b0 0x55fe477304a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:15.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.387+0000 7f5a09689b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fe477140d0 0x55fe477309e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:15.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.387+0000 7f5a09689b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f59d400cdf0 con 0x55fe4770f5d0 2024-05-03T18:36:15.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.387+0000 7f5a09689b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f59d400f650 con 0x55fe477137b0 2024-05-03T18:36:15.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.387+0000 7f5a09689b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f59d400d440 con 0x55fe477140d0 2024-05-03T18:36:15.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.387+0000 7f59f7e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fe4770f5d0 0x55fe47724dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:15.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.387+0000 7f59fcc00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fe477140d0 0x55fe477309e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:15.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.387+0000 7f59f7e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fe4770f5d0 0x55fe47724dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50590/0 (socket says 172.21.15.33:50590) 2024-05-03T18:36:15.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.387+0000 7f59f7e00700 1 -- 172.21.15.33:0/47212738 learned_addr learned my addr 172.21.15.33:0/47212738 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:36:15.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.387+0000 7f59ef400700 1 --2- 172.21.15.33:0/47212738 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe477137b0 0x55fe477304a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:15.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.388+0000 7f59fcc00700 1 -- 172.21.15.33:0/47212738 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe477137b0 msgr2=0x55fe477304a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:36:15.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.388+0000 7f59fcc00700 1 --2- 172.21.15.33:0/47212738 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe477137b0 0x55fe477304a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:15.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.388+0000 7f59fcc00700 1 -- 172.21.15.33:0/47212738 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fe4770f5d0 msgr2=0x55fe47724dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:36:15.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.388+0000 7f59fcc00700 1 --2- 172.21.15.33:0/47212738 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fe4770f5d0 0x55fe47724dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:15.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.388+0000 7f59fcc00700 1 -- 172.21.15.33:0/47212738 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe47713400 con 0x55fe477140d0 2024-05-03T18:36:15.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.389+0000 7f59f7e00700 1 --2- 172.21.15.33:0/47212738 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fe4770f5d0 0x55fe47724dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:36:15.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.389+0000 7f59fcc00700 1 --2- 172.21.15.33:0/47212738 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fe477140d0 0x55fe477309e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f59f00df6a0 tx=0x7f59f000ce80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=616adc4bd79ffc7e server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:36:15.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.390+0000 7f59ecc00700 1 -- 172.21.15.33:0/47212738 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59f00a9920 con 0x55fe477140d0 2024-05-03T18:36:15.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.390+0000 7f59ecc00700 1 -- 172.21.15.33:0/47212738 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f59f00a9220 con 0x55fe477140d0 2024-05-03T18:36:15.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.390+0000 7f59ecc00700 1 -- 172.21.15.33:0/47212738 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59f00a9480 con 0x55fe477140d0 2024-05-03T18:36:15.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.391+0000 7f5a09689b80 1 -- 172.21.15.33:0/47212738 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fe477140d0 msgr2=0x55fe477309e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:36:15.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.391+0000 7f5a09689b80 1 --2- 172.21.15.33:0/47212738 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fe477140d0 0x55fe477309e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f59f00df6a0 tx=0x7f59f000ce80 comp rx=0 tx=0).stop 2024-05-03T18:36:15.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.392+0000 7f5a09689b80 1 -- 172.21.15.33:0/47212738 shutdown_connections 2024-05-03T18:36:15.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.392+0000 7f5a09689b80 1 --2- 172.21.15.33:0/47212738 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fe477140d0 0x55fe477309e0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:15.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.392+0000 7f5a09689b80 1 --2- 172.21.15.33:0/47212738 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fe4770f5d0 0x55fe47724dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:15.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.392+0000 7f5a09689b80 1 --2- 172.21.15.33:0/47212738 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe477137b0 0x55fe477304a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:15.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.392+0000 7f5a09689b80 1 -- 172.21.15.33:0/47212738 >> 172.21.15.33:0/47212738 conn(0x55fe4763db60 msgr2=0x55fe47708ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:36:15.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.393+0000 7f5a09689b80 1 -- 172.21.15.33:0/47212738 shutdown_connections 2024-05-03T18:36:15.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.393+0000 7f5a09689b80 1 -- 172.21.15.33:0/47212738 wait complete. 2024-05-03T18:36:15.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.394+0000 7f5a09689b80 1 -- start start 2024-05-03T18:36:15.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.394+0000 7f5a09689b80 1 Processor -- start 2024-05-03T18:36:15.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.396+0000 7f5a09689b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe4770f5d0 0x55fe47721320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:15.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.396+0000 7f5a09689b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fe477137b0 0x55fe47721860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:15.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.396+0000 7f5a09689b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fe477140d0 0x55fe4771e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:15.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.396+0000 7f5a09689b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f59d400db10 con 0x55fe477140d0 2024-05-03T18:36:15.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.396+0000 7f5a09689b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f59d400ef80 con 0x55fe4770f5d0 2024-05-03T18:36:15.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.396+0000 7f5a09689b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f59d400e1e0 con 0x55fe477137b0 2024-05-03T18:36:15.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.396+0000 7f5a09689b80 10 client.0 _ll_register_callbacks cb 0x55fe47608c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:36:15.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.396+0000 7f59f7e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe4770f5d0 0x55fe47721320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:15.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.397+0000 7f59fcc00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fe477140d0 0x55fe4771e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:15.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.397+0000 7f59fcc00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fe477140d0 0x55fe4771e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50594/0 (socket says 172.21.15.33:50594) 2024-05-03T18:36:15.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.397+0000 7f59fcc00700 1 -- 172.21.15.33:0/2808375591 learned_addr learned my addr 172.21.15.33:0/2808375591 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:36:15.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.397+0000 7f59ef400700 1 --2- 172.21.15.33:0/2808375591 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fe477137b0 0x55fe47721860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:15.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.398+0000 7f59f7e00700 1 -- 172.21.15.33:0/2808375591 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fe477137b0 msgr2=0x55fe47721860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:36:15.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.398+0000 7f59f7e00700 1 --2- 172.21.15.33:0/2808375591 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fe477137b0 0x55fe47721860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:15.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.398+0000 7f59f7e00700 1 -- 172.21.15.33:0/2808375591 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fe477140d0 msgr2=0x55fe4771e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:36:15.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.398+0000 7f59f7e00700 1 --2- 172.21.15.33:0/2808375591 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fe477140d0 0x55fe4771e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:15.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.398+0000 7f59f7e00700 1 -- 172.21.15.33:0/2808375591 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe4771e950 con 0x55fe4770f5d0 2024-05-03T18:36:15.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.399+0000 7f59f7e00700 1 --2- 172.21.15.33:0/2808375591 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe4770f5d0 0x55fe47721320 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f59e00228e0 tx=0x7f59e0023f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19201edd33149ff4 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:36:15.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.399+0000 7f59e6000700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-03T18:36:15.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.399+0000 7f59e6000700 1 -- 172.21.15.33:0/2808375591 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e0026040 con 0x55fe4770f5d0 2024-05-03T18:36:15.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.400+0000 7f5a09689b80 1 -- 172.21.15.33:0/2808375591 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59f00a9220 con 0x55fe4770f5d0 2024-05-03T18:36:15.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.400+0000 7f59e6000700 1 -- 172.21.15.33:0/2808375591 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9518+0+0 (secure 0 0 0) 0x7f59e0004d30 con 0x55fe4770f5d0 2024-05-03T18:36:15.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.400+0000 7f59e6000700 1 -- 172.21.15.33:0/2808375591 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f59e0011cc0 con 0x55fe4770f5d0 2024-05-03T18:36:15.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.400+0000 7f5a09689b80 10 client.35978 fetch_fsmap learned FSMap version 21 2024-05-03T18:36:15.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.401+0000 7f5a09689b80 1 -- 172.21.15.33:0/2808375591 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55fe47793bb0 con 0x55fe4770f5d0 2024-05-03T18:36:15.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.402+0000 7f59e6000700 1 -- 172.21.15.33:0/2808375591 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f59f00a9220 con 0x55fe4770f5d0 2024-05-03T18:36:15.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.402+0000 7f59e6000700 1 -- 172.21.15.33:0/2808375591 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f59e0003680 con 0x55fe4770f5d0 2024-05-03T18:36:15.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.402+0000 7f5a09689b80 10 client.35978 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:36:15.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.402+0000 7f5a09689b80 10 client.35978 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:36:15.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.402+0000 7f5a09689b80 20 client.35978 populate_metadata read hostname 'smithi033' 2024-05-03T18:36:15.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.402+0000 7f5a09689b80 1 --2- 172.21.15.33:0/2808375591 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55fe477941f0 0x55fe47796610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:15.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.402+0000 7f5a09689b80 4 client.35978 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-03T18:36:15.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.402+0000 7f5a09689b80 1 -- 172.21.15.33:0/2808375591 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fe4766f400 con 0x55fe477941f0 2024-05-03T18:36:15.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.403+0000 7f59ef400700 1 --2- 172.21.15.33:0/2808375591 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55fe477941f0 0x55fe47796610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:15.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.405+0000 7f59ef400700 1 --2- 172.21.15.33:0/2808375591 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55fe477941f0 0x55fe47796610 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d06359ec13bf2876 server_cookie=5a07d8cb8a363045 in_seq=0 out_seq=0 2024-05-03T18:36:15.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.405+0000 7f59e6000700 10 client.35978 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:36:15.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.406+0000 7f59e6000700 1 -- 172.21.15.33:0/2808375591 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f59e8032e50 con 0x55fe477941f0 2024-05-03T18:36:15.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.406+0000 7f59e6000700 10 client.35978 handle_command_reply: tid=0 2024-05-03T18:36:15.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.406+0000 7f5a09689b80 1 client.35978 shutdown 2024-05-03T18:36:15.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.408+0000 7f5a09689b80 1 -- 172.21.15.33:0/2808375591 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe4770f5d0 msgr2=0x55fe47721320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:36:15.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.408+0000 7f5a09689b80 1 --2- 172.21.15.33:0/2808375591 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe4770f5d0 0x55fe47721320 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f59e00228e0 tx=0x7f59e0023f80 comp rx=0 tx=0).stop 2024-05-03T18:36:15.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.408+0000 7f5a09689b80 1 -- 172.21.15.33:0/2808375591 shutdown_connections 2024-05-03T18:36:15.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.408+0000 7f5a09689b80 1 --2- 172.21.15.33:0/2808375591 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55fe477941f0 0x55fe47796610 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:15.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.408+0000 7f59e6000700 0 client.35978 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:36:15.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.408+0000 7f5a09689b80 1 --2- 172.21.15.33:0/2808375591 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fe477140d0 0x55fe4771e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:15.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.408+0000 7f5a09689b80 1 --2- 172.21.15.33:0/2808375591 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe4770f5d0 0x55fe47721320 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:15.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.408+0000 7f5a09689b80 1 --2- 172.21.15.33:0/2808375591 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fe477137b0 0x55fe47721860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:15.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.408+0000 7f5a09689b80 1 -- 172.21.15.33:0/2808375591 >> 172.21.15.33:0/2808375591 conn(0x55fe4763db60 msgr2=0x55fe477123f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:36:15.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.409+0000 7f5a09689b80 1 -- 172.21.15.33:0/2808375591 shutdown_connections 2024-05-03T18:36:15.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.409+0000 7f5a09689b80 1 -- 172.21.15.33:0/2808375591 wait complete. 2024-05-03T18:36:15.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.409+0000 7f5a09689b80 20 client.35978 trim_cache size 0 max 16384 2024-05-03T18:36:15.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.410+0000 7f59fec00700 1 -- 172.21.15.33:0/2482685518 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f59dc0b2af0 msgr2=0x7f59dc0b4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:36:15.411 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-03T18:36:15.411 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-05-03T18:36:15.411 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-05-03T18:36:15.411 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-03T18:36:15.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.410+0000 7f59fec00700 1 --2- 172.21.15.33:0/2482685518 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f59dc0b2af0 0x7f59dc0b4fb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f59f81107d0 tx=0x7f59e80058e0 comp rx=0 tx=0).stop 2024-05-03T18:36:15.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.410+0000 7f59fec00700 1 -- 172.21.15.33:0/2482685518 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f59f8108460 msgr2=0x7f59f810f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:36:15.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.410+0000 7f59fec00700 1 --2- 172.21.15.33:0/2482685518 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f59f8108460 0x7f59f810f6b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f59f000efc0 tx=0x7f59f000c3a0 comp rx=0 tx=0).stop 2024-05-03T18:36:15.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.411+0000 7f59fec00700 1 -- 172.21.15.33:0/2482685518 shutdown_connections 2024-05-03T18:36:15.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.411+0000 7f59fec00700 1 --2- 172.21.15.33:0/2482685518 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f59dc0b2af0 0x7f59dc0b4fb0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:15.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.411+0000 7f59fec00700 1 --2- 172.21.15.33:0/2482685518 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f59f8108460 0x7f59f810f6b0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:15.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.411+0000 7f59fec00700 1 --2- 172.21.15.33:0/2482685518 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f59f8103550 0x7f59f81a07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:15.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.411+0000 7f59fec00700 1 --2- 172.21.15.33:0/2482685518 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59f8103ef0 0x7f59f81a0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:15.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.411+0000 7f59fec00700 1 -- 172.21.15.33:0/2482685518 >> 172.21.15.33:0/2482685518 conn(0x7f59f807a600 msgr2=0x7f59f81013c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:36:15.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.411+0000 7f59fec00700 1 -- 172.21.15.33:0/2482685518 shutdown_connections 2024-05-03T18:36:15.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:15.411+0000 7f59fec00700 1 -- 172.21.15.33:0/2482685518 wait complete. 2024-05-03T18:36:16.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:36:16 smithi033 ceph-mon[28330]: pgmap v281: 129 pgs: 129 active+clean; 2.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 7.1 MiB/s wr, 710 op/s 2024-05-03T18:36:16.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:36:16 smithi150 ceph-mon[32492]: pgmap v281: 129 pgs: 129 active+clean; 2.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 7.1 MiB/s wr, 710 op/s 2024-05-03T18:36:16.452 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-03T18:36:16.452 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T18:36:16.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:36:16 smithi099 ceph-mon[31658]: pgmap v281: 129 pgs: 129 active+clean; 2.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 7.1 MiB/s wr, 710 op/s 2024-05-03T18:36:17.180 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:36:18.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:36:18 smithi150 ceph-mon[32492]: pgmap v282: 129 pgs: 129 active+clean; 2.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 6.3 MiB/s wr, 633 op/s 2024-05-03T18:36:18.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:36:18 smithi033 ceph-mon[28330]: pgmap v282: 129 pgs: 129 active+clean; 2.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 6.3 MiB/s wr, 633 op/s 2024-05-03T18:36:18.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:36:18 smithi099 ceph-mon[31658]: pgmap v282: 129 pgs: 129 active+clean; 2.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 6.3 MiB/s wr, 633 op/s 2024-05-03T18:36:20.110 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 305.0 s, 618.4 tps, lat 161.655 ms stddev 46.858 2024-05-03T18:36:20.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:36:20 smithi150 ceph-mon[32492]: pgmap v283: 129 pgs: 129 active+clean; 2.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 6.2 MiB/s wr, 638 op/s 2024-05-03T18:36:20.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:36:20 smithi033 ceph-mon[28330]: pgmap v283: 129 pgs: 129 active+clean; 2.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 6.2 MiB/s wr, 638 op/s 2024-05-03T18:36:20.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:36:20 smithi099 ceph-mon[31658]: pgmap v283: 129 pgs: 129 active+clean; 2.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 6.2 MiB/s wr, 638 op/s 2024-05-03T18:36:21.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:21.485+0000 7f553f200700 1 -- 172.21.15.33:0/2707830351 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5538111c80 msgr2=0x7f553810a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:36:21.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:21.485+0000 7f553f200700 1 --2- 172.21.15.33:0/2707830351 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5538111c80 0x7f553810a4a0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f5528009990 tx=0x7f5528009ca0 comp rx=0 tx=0).stop 2024-05-03T18:36:21.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:21.486+0000 7f553f200700 1 -- 172.21.15.33:0/2707830351 shutdown_connections 2024-05-03T18:36:21.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:21.486+0000 7f553f200700 1 --2- 172.21.15.33:0/2707830351 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f553811cc80 0x7f553811f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:21.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:21.486+0000 7f553f200700 1 --2- 172.21.15.33:0/2707830351 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5538111c80 0x7f553810a4a0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:21.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:21.486+0000 7f553f200700 1 --2- 172.21.15.33:0/2707830351 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f553810a9e0 0x7f553810ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:21.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:21.486+0000 7f553f200700 1 -- 172.21.15.33:0/2707830351 >> 172.21.15.33:0/2707830351 conn(0x7f553806e610 msgr2=0x7f553806ea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:36:21.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:21.486+0000 7f553f200700 1 -- 172.21.15.33:0/2707830351 shutdown_connections 2024-05-03T18:36:21.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:21.486+0000 7f553f200700 1 -- 172.21.15.33:0/2707830351 wait complete. 2024-05-03T18:36:21.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:21.487+0000 7f553f200700 1 Processor -- start 2024-05-03T18:36:21.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:21.487+0000 7f553f200700 1 -- start start 2024-05-03T18:36:21.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:21.488+0000 7f553f200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f553810a9e0 0x7f55381ae640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:21.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:21.488+0000 7f553f200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5538111c80 0x7f55381aeb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:21.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:21.488+0000 7f553f200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f553811cc80 0x7f55381a87c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:21.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:21.488+0000 7f553f200700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5538122460 con 0x7f553810a9e0 2024-05-03T18:36:21.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:21.488+0000 7f553f200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f55381222e0 con 0x7f5538111c80 2024-05-03T18:36:21.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:21.488+0000 7f553f200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f55381225e0 con 0x7f553811cc80 2024-05-03T18:36:21.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:21.489+0000 7f5537e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f553810a9e0 0x7f55381ae640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:21.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:21.489+0000 7f553d200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f553811cc80 0x7f55381a87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:21.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:21.489+0000 7f553d200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f553811cc80 0x7f55381a87c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:34144/0 (socket says 172.21.15.33:34144) 2024-05-03T18:36:21.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:21.489+0000 7f553d200700 1 -- 172.21.15.33:0/1133028828 learned_addr learned my addr 172.21.15.33:0/1133028828 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:36:21.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:21.489+0000 7f5537400700 1 --2- 172.21.15.33:0/1133028828 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5538111c80 0x7f55381aeb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:21.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:21.490+0000 7f553d200700 1 -- 172.21.15.33:0/1133028828 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5538111c80 msgr2=0x7f55381aeb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:36:21.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:21.490+0000 7f553d200700 1 --2- 172.21.15.33:0/1133028828 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5538111c80 0x7f55381aeb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:21.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:21.490+0000 7f553d200700 1 -- 172.21.15.33:0/1133028828 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f553810a9e0 msgr2=0x7f55381ae640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:36:21.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:21.490+0000 7f553d200700 1 --2- 172.21.15.33:0/1133028828 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f553810a9e0 0x7f55381ae640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:21.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:21.490+0000 7f553d200700 1 -- 172.21.15.33:0/1133028828 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55381a8fa0 con 0x7f553811cc80 2024-05-03T18:36:21.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:21.491+0000 7f553d200700 1 --2- 172.21.15.33:0/1133028828 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f553811cc80 0x7f55381a87c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f553000b9e0 tx=0x7f553000bcf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:36:21.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:21.491+0000 7f5534c00700 1 -- 172.21.15.33:0/1133028828 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f553000d8e0 con 0x7f553811cc80 2024-05-03T18:36:21.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:21.491+0000 7f5534c00700 1 -- 172.21.15.33:0/1133028828 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5530015420 con 0x7f553811cc80 2024-05-03T18:36:21.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:21.491+0000 7f5534c00700 1 -- 172.21.15.33:0/1133028828 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5530014630 con 0x7f553811cc80 2024-05-03T18:36:21.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:21.492+0000 7f553f200700 1 -- 172.21.15.33:0/1133028828 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55381a9290 con 0x7f553811cc80 2024-05-03T18:36:21.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:21.492+0000 7f553f200700 1 -- 172.21.15.33:0/1133028828 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55381b54a0 con 0x7f553811cc80 2024-05-03T18:36:21.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:21.494+0000 7f553f200700 1 -- 172.21.15.33:0/1133028828 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f553806fba0 con 0x7f553811cc80 2024-05-03T18:36:21.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:21.498+0000 7f5534c00700 1 -- 172.21.15.33:0/1133028828 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f55300155d0 con 0x7f553811cc80 2024-05-03T18:36:21.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:21.500+0000 7f5534c00700 1 --2- 172.21.15.33:0/1133028828 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f551c0b2af0 0x7f551c0b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:21.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:21.501+0000 7f5534c00700 1 -- 172.21.15.33:0/1133028828 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9518+0+0 (secure 0 0 0) 0x7f55300e2620 con 0x7f553811cc80 2024-05-03T18:36:21.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:21.501+0000 7f5537e00700 1 --2- 172.21.15.33:0/1133028828 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f551c0b2af0 0x7f551c0b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:21.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:21.502+0000 7f5534c00700 1 -- 172.21.15.33:0/1133028828 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f55300a9bc0 con 0x7f553811cc80 2024-05-03T18:36:21.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:21.502+0000 7f5537e00700 1 --2- 172.21.15.33:0/1133028828 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f551c0b2af0 0x7f551c0b4fb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5528006010 tx=0x7f552800b560 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:36:22.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:22.052+0000 7f553f200700 1 -- 172.21.15.33:0/1133028828 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f553806a190 con 0x7f553811cc80 2024-05-03T18:36:22.056 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:36:22.056 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1327393503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1327393503},{"type":"v1","addr":"172.21.15.33:6835","nonce":1327393503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/2461472948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":2461472948},{"type":"v1","addr":"172.21.15.33:6837","nonce":2461472948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14559,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/442269281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":442269281},{"type":"v1","addr":"172.21.15.33:6839","nonce":442269281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14571,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/645571756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":645571756},{"type":"v1","addr":"172.21.15.33:6841","nonce":645571756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/203598909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":203598909},{"type":"v1","addr":"172.21.15.150:6833","nonce":203598909}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/4029842493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":4029842493},{"type":"v1","addr":"172.21.15.150:6835","nonce":4029842493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3463664423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3463664423},{"type":"v1","addr":"172.21.15.150:6837","nonce":3463664423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24481,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6839/4051648018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":4051648018},{"type":"v1","addr":"172.21.15.150:6839","nonce":4051648018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34418,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1865105479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1865105479},{"type":"v1","addr":"172.21.15.99:6833","nonce":1865105479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34445,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/1206393914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1206393914},{"type":"v1","addr":"172.21.15.99:6835","nonce":1206393914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-03T18:22:28.837091+0000","modified":"2024-05-03T18:23:49.758398+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34508},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34472":{"gid":34472,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6837/3360970342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3360970342},{"type":"v1","addr":"172.21.15.99:6837","nonce":3360970342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34508":{"gid":34508,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/919907308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":919907308},{"type":"v1","addr":"172.21.15.99:6839","nonce":919907308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34508,"qdb_cluster":[34508]},"id":1}]} 2024-05-03T18:36:22.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:22.054+0000 7f5534c00700 1 -- 172.21.15.33:0/1133028828 <== mon.1 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11801 (secure 0 0 0) 0x7f55300a94c0 con 0x7f553811cc80 2024-05-03T18:36:22.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:22.059+0000 7f553f200700 1 -- 172.21.15.33:0/1133028828 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f551c0b2af0 msgr2=0x7f551c0b4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:36:22.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:22.059+0000 7f553f200700 1 --2- 172.21.15.33:0/1133028828 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f551c0b2af0 0x7f551c0b4fb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5528006010 tx=0x7f552800b560 comp rx=0 tx=0).stop 2024-05-03T18:36:22.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:22.059+0000 7f553f200700 1 -- 172.21.15.33:0/1133028828 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f553811cc80 msgr2=0x7f55381a87c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:36:22.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:22.059+0000 7f553f200700 1 --2- 172.21.15.33:0/1133028828 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f553811cc80 0x7f55381a87c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f553000b9e0 tx=0x7f553000bcf0 comp rx=0 tx=0).stop 2024-05-03T18:36:22.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:22.059+0000 7f553f200700 1 -- 172.21.15.33:0/1133028828 shutdown_connections 2024-05-03T18:36:22.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:22.059+0000 7f553f200700 1 --2- 172.21.15.33:0/1133028828 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f553811cc80 0x7f55381a87c0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:22.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:22.059+0000 7f553f200700 1 --2- 172.21.15.33:0/1133028828 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f553810a9e0 0x7f55381ae640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:22.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:22.059+0000 7f553f200700 1 --2- 172.21.15.33:0/1133028828 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f551c0b2af0 0x7f551c0b4fb0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:22.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:22.059+0000 7f553f200700 1 --2- 172.21.15.33:0/1133028828 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5538111c80 0x7f55381aeb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:22.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:22.059+0000 7f553f200700 1 -- 172.21.15.33:0/1133028828 >> 172.21.15.33:0/1133028828 conn(0x7f553806e610 msgr2=0x7f553811ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:36:22.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:22.060+0000 7f553f200700 1 -- 172.21.15.33:0/1133028828 shutdown_connections 2024-05-03T18:36:22.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:22.060+0000 7f553f200700 1 -- 172.21.15.33:0/1133028828 wait complete. 2024-05-03T18:36:22.064 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 21 2024-05-03T18:36:22.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:36:22 smithi150 ceph-mon[32492]: pgmap v284: 129 pgs: 129 active+clean; 2.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 6.2 MiB/s wr, 638 op/s 2024-05-03T18:36:22.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:36:22 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/1133028828' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:36:22.466 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:36:22 smithi033 ceph-mon[28330]: pgmap v284: 129 pgs: 129 active+clean; 2.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 6.2 MiB/s wr, 638 op/s 2024-05-03T18:36:22.466 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:36:22 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/1133028828' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:36:22.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:36:22 smithi099 ceph-mon[31658]: pgmap v284: 129 pgs: 129 active+clean; 2.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 6.2 MiB/s wr, 638 op/s 2024-05-03T18:36:22.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:36:22 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/1133028828' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:36:23.385 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-03T18:36:24.130 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:36:24.352 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:36:24 smithi033 ceph-mon[28330]: pgmap v285: 129 pgs: 129 active+clean; 2.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 8.1 MiB/s wr, 821 op/s 2024-05-03T18:36:24.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:36:24 smithi150 ceph-mon[32492]: pgmap v285: 129 pgs: 129 active+clean; 2.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 8.1 MiB/s wr, 821 op/s 2024-05-03T18:36:24.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:36:24 smithi099 ceph-mon[31658]: pgmap v285: 129 pgs: 129 active+clean; 2.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 8.1 MiB/s wr, 821 op/s 2024-05-03T18:36:25.111 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 310.0 s, 586.1 tps, lat 171.039 ms stddev 49.832 2024-05-03T18:36:26.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:36:26 smithi150 ceph-mon[32492]: pgmap v286: 129 pgs: 129 active+clean; 2.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 5.3 MiB/s wr, 536 op/s 2024-05-03T18:36:26.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:36:26 smithi033 ceph-mon[28330]: pgmap v286: 129 pgs: 129 active+clean; 2.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 5.3 MiB/s wr, 536 op/s 2024-05-03T18:36:26.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:36:26 smithi099 ceph-mon[31658]: pgmap v286: 129 pgs: 129 active+clean; 2.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 5.3 MiB/s wr, 536 op/s 2024-05-03T18:36:28.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.113+0000 7f443a200700 1 -- 172.21.15.33:0/17723427 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4434103ef0 msgr2=0x7f44341103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:36:28.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.113+0000 7f443a200700 1 --2- 172.21.15.33:0/17723427 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4434103ef0 0x7f44341103d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4424009990 tx=0x7f4424009ca0 comp rx=0 tx=0).stop 2024-05-03T18:36:28.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.113+0000 7f443a200700 1 -- 172.21.15.33:0/17723427 shutdown_connections 2024-05-03T18:36:28.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.113+0000 7f443a200700 1 --2- 172.21.15.33:0/17723427 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4434103ef0 0x7f44341103d0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:28.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.113+0000 7f443a200700 1 --2- 172.21.15.33:0/17723427 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4434108460 0x7f4434108840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:28.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.113+0000 7f443a200700 1 --2- 172.21.15.33:0/17723427 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4434103550 0x7f44341039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:28.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.113+0000 7f443a200700 1 -- 172.21.15.33:0/17723427 >> 172.21.15.33:0/17723427 conn(0x7f443407a600 msgr2=0x7f443407aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:36:28.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.114+0000 7f443a200700 1 -- 172.21.15.33:0/17723427 shutdown_connections 2024-05-03T18:36:28.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.114+0000 7f443a200700 1 -- 172.21.15.33:0/17723427 wait complete. 2024-05-03T18:36:28.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.115+0000 7f443a200700 1 Processor -- start 2024-05-03T18:36:28.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.115+0000 7f443a200700 1 -- start start 2024-05-03T18:36:28.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.116+0000 7f443a200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4434103550 0x7f4434073070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:28.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.116+0000 7f443a200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4434103ef0 0x7f44340735b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:28.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.116+0000 7f443a200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4434108460 0x7f44341a8e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:28.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.117+0000 7f443a200700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f443406d390 con 0x7f4434108460 2024-05-03T18:36:28.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.117+0000 7f443a200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f443406d210 con 0x7f4434103ef0 2024-05-03T18:36:28.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.117+0000 7f443a200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f443406d510 con 0x7f4434103550 2024-05-03T18:36:28.120 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:36:28 smithi033 ceph-mon[28330]: pgmap v287: 129 pgs: 129 active+clean; 2.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 5.3 MiB/s wr, 536 op/s 2024-05-03T18:36:28.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.117+0000 7f4432a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4434103ef0 0x7f44340735b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:28.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.117+0000 7f4432a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4434103ef0 0x7f44340735b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:40232/0 (socket says 172.21.15.33:40232) 2024-05-03T18:36:28.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.117+0000 7f4432a00700 1 -- 172.21.15.33:0/2079381048 learned_addr learned my addr 172.21.15.33:0/2079381048 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:36:28.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.118+0000 7f4432a00700 1 -- 172.21.15.33:0/2079381048 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4434103550 msgr2=0x7f4434073070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:36:28.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.117+0000 7f4433400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4434103550 0x7f4434073070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:28.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.119+0000 7f4432a00700 1 --2- 172.21.15.33:0/2079381048 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4434103550 0x7f4434073070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:28.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.119+0000 7f4432a00700 1 -- 172.21.15.33:0/2079381048 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4434108460 msgr2=0x7f44341a8e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:36:28.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.119+0000 7f4432a00700 1 --2- 172.21.15.33:0/2079381048 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4434108460 0x7f44341a8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:28.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.120+0000 7f4432a00700 1 -- 172.21.15.33:0/2079381048 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44341a93b0 con 0x7f4434103ef0 2024-05-03T18:36:28.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.121+0000 7f4433400700 1 --2- 172.21.15.33:0/2079381048 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4434103550 0x7f4434073070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:36:28.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.123+0000 7f4432a00700 1 --2- 172.21.15.33:0/2079381048 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4434103ef0 0x7f44340735b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f442000d950 tx=0x7f442000dd10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:36:28.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.123+0000 7f442be00700 1 -- 172.21.15.33:0/2079381048 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4420016070 con 0x7f4434103ef0 2024-05-03T18:36:28.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.124+0000 7f442be00700 1 -- 172.21.15.33:0/2079381048 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4420010460 con 0x7f4434103ef0 2024-05-03T18:36:28.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.124+0000 7f442be00700 1 -- 172.21.15.33:0/2079381048 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f442000f620 con 0x7f4434103ef0 2024-05-03T18:36:28.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.124+0000 7f443a200700 1 -- 172.21.15.33:0/2079381048 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44341a96a0 con 0x7f4434103ef0 2024-05-03T18:36:28.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.124+0000 7f443a200700 1 -- 172.21.15.33:0/2079381048 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44341a9b90 con 0x7f4434103ef0 2024-05-03T18:36:28.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.133+0000 7f4444b98b80 1 Processor -- start 2024-05-03T18:36:28.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.134+0000 7f4444b98b80 1 -- start start 2024-05-03T18:36:28.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.134+0000 7f4444b98b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dcea34b930 0x55dcea34bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:28.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.134+0000 7f4444b98b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dcea34c250 0x55dcea345f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:28.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.134+0000 7f4444b98b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dcea346550 0x55dcea3469d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:28.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.134+0000 7f4444b98b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55dcea298a20 con 0x55dcea34b930 2024-05-03T18:36:28.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.134+0000 7f4444b98b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55dcea236760 con 0x55dcea346550 2024-05-03T18:36:28.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.134+0000 7f4444b98b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55dcea2949e0 con 0x55dcea34c250 2024-05-03T18:36:28.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.135+0000 7f442be00700 1 -- 172.21.15.33:0/2079381048 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f4420019070 con 0x7f4434103ef0 2024-05-03T18:36:28.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.135+0000 7f4433e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dcea34b930 0x55dcea34bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:28.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.135+0000 7f4433e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dcea34b930 0x55dcea34bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37658/0 (socket says 172.21.15.33:37658) 2024-05-03T18:36:28.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.135+0000 7f4433e00700 1 -- 172.21.15.33:0/3611138960 learned_addr learned my addr 172.21.15.33:0/3611138960 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:36:28.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.135+0000 7f4433400700 1 --2- 172.21.15.33:0/3611138960 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dcea34c250 0x55dcea345f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:28.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.135+0000 7f4432a00700 1 --2- 172.21.15.33:0/3611138960 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dcea346550 0x55dcea3469d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:28.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.136+0000 7f4433400700 1 -- 172.21.15.33:0/3611138960 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dcea346550 msgr2=0x55dcea3469d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:36:28.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.136+0000 7f4433400700 1 --2- 172.21.15.33:0/3611138960 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dcea346550 0x55dcea3469d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:28.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.136+0000 7f4433400700 1 -- 172.21.15.33:0/3611138960 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dcea34b930 msgr2=0x55dcea34bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:36:28.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.136+0000 7f4433400700 1 --2- 172.21.15.33:0/3611138960 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dcea34b930 0x55dcea34bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:28.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.136+0000 7f4433400700 1 -- 172.21.15.33:0/3611138960 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcea1e58d0 con 0x55dcea34c250 2024-05-03T18:36:28.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.137+0000 7f4433400700 1 --2- 172.21.15.33:0/3611138960 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dcea34c250 0x55dcea345f30 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f442c007ec0 tx=0x7f442c00c5a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6db004c7c75530ec server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:36:28.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.138+0000 7f442be00700 1 --2- 172.21.15.33:0/2079381048 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f44180b2b20 0x7f44180b4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:28.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.138+0000 7f442be00700 1 -- 172.21.15.33:0/2079381048 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9518+0+0 (secure 0 0 0) 0x7f44200e2320 con 0x7f4434103ef0 2024-05-03T18:36:28.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.138+0000 7f4428c00700 1 -- 172.21.15.33:0/3611138960 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f442c014070 con 0x55dcea34c250 2024-05-03T18:36:28.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.138+0000 7f4428c00700 1 -- 172.21.15.33:0/3611138960 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f442c00ce10 con 0x55dcea34c250 2024-05-03T18:36:28.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.138+0000 7f4428c00700 1 -- 172.21.15.33:0/3611138960 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f442c00e860 con 0x55dcea34c250 2024-05-03T18:36:28.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.139+0000 7f4433e00700 1 --2- 172.21.15.33:0/2079381048 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f44180b2b20 0x7f44180b4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:28.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.139+0000 7f4444b98b80 1 -- 172.21.15.33:0/3611138960 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dcea34c250 msgr2=0x55dcea345f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:36:28.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.139+0000 7f4444b98b80 1 --2- 172.21.15.33:0/3611138960 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dcea34c250 0x55dcea345f30 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f442c007ec0 tx=0x7f442c00c5a0 comp rx=0 tx=0).stop 2024-05-03T18:36:28.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.140+0000 7f4444b98b80 1 -- 172.21.15.33:0/3611138960 shutdown_connections 2024-05-03T18:36:28.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.140+0000 7f4444b98b80 1 --2- 172.21.15.33:0/3611138960 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dcea346550 0x55dcea3469d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:28.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.140+0000 7f4444b98b80 1 --2- 172.21.15.33:0/3611138960 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dcea34b930 0x55dcea34bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:28.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.140+0000 7f4433e00700 1 --2- 172.21.15.33:0/2079381048 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f44180b2b20 0x7f44180b4fe0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4424006010 tx=0x7f4424009510 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:36:28.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.140+0000 7f4444b98b80 1 --2- 172.21.15.33:0/3611138960 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dcea34c250 0x55dcea345f30 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:28.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.140+0000 7f4444b98b80 1 -- 172.21.15.33:0/3611138960 >> 172.21.15.33:0/3611138960 conn(0x55dcea2be6d0 msgr2=0x55dcea340900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:36:28.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.140+0000 7f4444b98b80 1 -- 172.21.15.33:0/3611138960 shutdown_connections 2024-05-03T18:36:28.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.141+0000 7f4444b98b80 1 -- 172.21.15.33:0/3611138960 wait complete. 2024-05-03T18:36:28.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.142+0000 7f4444b98b80 1 -- start start 2024-05-03T18:36:28.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.142+0000 7f4444b98b80 1 Processor -- start 2024-05-03T18:36:28.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.143+0000 7f4444b98b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dcea346550 0x55dcea351e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:28.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.143+0000 7f4444b98b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dcea34b930 0x55dcea34f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:28.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.143+0000 7f4444b98b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dcea34c250 0x55dcea34f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:28.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.143+0000 7f4444b98b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55dcea352d00 con 0x55dcea34c250 2024-05-03T18:36:28.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.143+0000 7f4444b98b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55dcea1feb20 con 0x55dcea34b930 2024-05-03T18:36:28.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.143+0000 7f4444b98b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55dcea352e80 con 0x55dcea346550 2024-05-03T18:36:28.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.143+0000 7f4444b98b80 10 client.0 _ll_register_callbacks cb 0x55dcea23fc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:36:28.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.144+0000 7f4433400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dcea346550 0x55dcea351e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:28.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.144+0000 7f4432a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dcea34b930 0x55dcea34f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:28.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.144+0000 7f4433400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dcea346550 0x55dcea351e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:57898/0 (socket says 172.21.15.33:57898) 2024-05-03T18:36:28.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.144+0000 7f4433400700 1 -- 172.21.15.33:0/1935490916 learned_addr learned my addr 172.21.15.33:0/1935490916 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:36:28.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.145+0000 7f4433e00700 1 --2- 172.21.15.33:0/1935490916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dcea34c250 0x55dcea34f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:28.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.145+0000 7f4433400700 1 -- 172.21.15.33:0/1935490916 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dcea34b930 msgr2=0x55dcea34f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:36:28.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.145+0000 7f4433400700 1 --2- 172.21.15.33:0/1935490916 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dcea34b930 0x55dcea34f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:28.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.145+0000 7f4433400700 1 -- 172.21.15.33:0/1935490916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dcea34c250 msgr2=0x55dcea34f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:36:28.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.145+0000 7f4433400700 1 --2- 172.21.15.33:0/1935490916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dcea34c250 0x55dcea34f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:28.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.145+0000 7f4433400700 1 -- 172.21.15.33:0/1935490916 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dcea25e420 con 0x55dcea346550 2024-05-03T18:36:28.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.146+0000 7f441d600700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-03T18:36:28.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.146+0000 7f4433400700 1 --2- 172.21.15.33:0/1935490916 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dcea346550 0x55dcea351e00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f442c00fbd0 tx=0x7f442c00fc00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62582294872da98c server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:36:28.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.147+0000 7f441d600700 1 -- 172.21.15.33:0/1935490916 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f442c014040 con 0x55dcea346550 2024-05-03T18:36:28.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.147+0000 7f441d600700 1 -- 172.21.15.33:0/1935490916 <== mon.1 v2:172.21.15.150:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9518+0+0 (secure 0 0 0) 0x7f442c00e6e0 con 0x55dcea346550 2024-05-03T18:36:28.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.147+0000 7f4444b98b80 1 -- 172.21.15.33:0/1935490916 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f442c00ce10 con 0x55dcea346550 2024-05-03T18:36:28.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.147+0000 7f441d600700 1 -- 172.21.15.33:0/1935490916 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f442c00e8e0 con 0x55dcea346550 2024-05-03T18:36:28.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.147+0000 7f4444b98b80 10 client.26041 fetch_fsmap learned FSMap version 21 2024-05-03T18:36:28.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.147+0000 7f4444b98b80 1 -- 172.21.15.33:0/1935490916 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55dcea35d230 con 0x55dcea346550 2024-05-03T18:36:28.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.148+0000 7f441d600700 1 -- 172.21.15.33:0/1935490916 <== mon.1 v2:172.21.15.150:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f442c00ce10 con 0x55dcea346550 2024-05-03T18:36:28.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.149+0000 7f441d600700 1 -- 172.21.15.33:0/1935490916 <== mon.1 v2:172.21.15.150:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f442c019a30 con 0x55dcea346550 2024-05-03T18:36:28.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.150+0000 7f4444b98b80 10 client.26041 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:36:28.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.150+0000 7f4444b98b80 10 client.26041 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:36:28.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.150+0000 7f4444b98b80 20 client.26041 populate_metadata read hostname 'smithi033' 2024-05-03T18:36:28.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.150+0000 7f4444b98b80 1 --2- 172.21.15.33:0/1935490916 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55dcea35f700 0x55dcea361af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:28.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.150+0000 7f4444b98b80 4 client.26041 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:36:28.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.150+0000 7f4444b98b80 1 -- 172.21.15.33:0/1935490916 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dcea312ff0 con 0x55dcea35f700 2024-05-03T18:36:28.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.150+0000 7f4432a00700 1 --2- 172.21.15.33:0/1935490916 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55dcea35f700 0x55dcea361af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:28.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.151+0000 7f441d600700 10 client.26041 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:36:28.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.151+0000 7f4432a00700 1 --2- 172.21.15.33:0/1935490916 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55dcea35f700 0x55dcea361af0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2bccaeba6694a1a3 server_cookie=e53e6b90351984e8 in_seq=0 out_seq=0 2024-05-03T18:36:28.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.152+0000 7f441d600700 1 -- 172.21.15.33:0/1935490916 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55dcea312ff0 con 0x55dcea35f700 2024-05-03T18:36:28.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.152+0000 7f441d600700 10 client.26041 handle_command_reply: tid=0 2024-05-03T18:36:28.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.152+0000 7f4444b98b80 1 client.26041 shutdown 2024-05-03T18:36:28.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.153+0000 7f4444b98b80 1 -- 172.21.15.33:0/1935490916 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dcea346550 msgr2=0x55dcea351e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:36:28.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.153+0000 7f4444b98b80 1 --2- 172.21.15.33:0/1935490916 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dcea346550 0x55dcea351e00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f442c00fbd0 tx=0x7f442c00fc00 comp rx=0 tx=0).stop 2024-05-03T18:36:28.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.154+0000 7f4444b98b80 1 -- 172.21.15.33:0/1935490916 shutdown_connections 2024-05-03T18:36:28.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.154+0000 7f4444b98b80 1 --2- 172.21.15.33:0/1935490916 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55dcea35f700 0x55dcea361af0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:28.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.154+0000 7f4444b98b80 1 --2- 172.21.15.33:0/1935490916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dcea34c250 0x55dcea34f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:28.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.154+0000 7f4444b98b80 1 --2- 172.21.15.33:0/1935490916 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dcea346550 0x55dcea351e00 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:28.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.154+0000 7f4444b98b80 1 --2- 172.21.15.33:0/1935490916 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dcea34b930 0x55dcea34f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:28.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.154+0000 7f4444b98b80 1 -- 172.21.15.33:0/1935490916 >> 172.21.15.33:0/1935490916 conn(0x55dcea2be6d0 msgr2=0x55dcea33b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:36:28.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.154+0000 7f441d600700 0 client.26041 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:36:28.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.155+0000 7f4444b98b80 1 -- 172.21.15.33:0/1935490916 shutdown_connections 2024-05-03T18:36:28.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.155+0000 7f4444b98b80 1 -- 172.21.15.33:0/1935490916 wait complete. 2024-05-03T18:36:28.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.155+0000 7f4444b98b80 20 client.26041 trim_cache size 0 max 16384 2024-05-03T18:36:28.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.185+0000 7f4444b98b80 1 Processor -- start 2024-05-03T18:36:28.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.185+0000 7f4444b98b80 1 -- start start 2024-05-03T18:36:28.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.186+0000 7f4444b98b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dcea346550 0x55dcea345f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:28.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.186+0000 7f4444b98b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dcea34b930 0x55dcea361af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:28.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.186+0000 7f4444b98b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dcea34c250 0x55dcea34ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:28.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.186+0000 7f4444b98b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f441400ceb0 con 0x55dcea34c250 2024-05-03T18:36:28.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.186+0000 7f4444b98b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f441400f710 con 0x55dcea346550 2024-05-03T18:36:28.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.186+0000 7f4444b98b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f441400d500 con 0x55dcea34b930 2024-05-03T18:36:28.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.187+0000 7f4433400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dcea346550 0x55dcea345f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:28.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.187+0000 7f4433e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dcea34c250 0x55dcea34ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:28.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.187+0000 7f4433400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dcea346550 0x55dcea345f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:40264/0 (socket says 172.21.15.33:40264) 2024-05-03T18:36:28.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.187+0000 7f4432a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dcea34b930 0x55dcea361af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:28.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.187+0000 7f4432a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dcea34b930 0x55dcea361af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:57904/0 (socket says 172.21.15.33:57904) 2024-05-03T18:36:28.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.187+0000 7f4432a00700 1 -- 172.21.15.33:0/4081298558 learned_addr learned my addr 172.21.15.33:0/4081298558 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:36:28.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.188+0000 7f4432a00700 1 -- 172.21.15.33:0/4081298558 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dcea346550 msgr2=0x55dcea345f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:36:28.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.188+0000 7f4432a00700 1 --2- 172.21.15.33:0/4081298558 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dcea346550 0x55dcea345f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:28.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.188+0000 7f4432a00700 1 -- 172.21.15.33:0/4081298558 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dcea34c250 msgr2=0x55dcea34ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:36:28.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.188+0000 7f4432a00700 1 --2- 172.21.15.33:0/4081298558 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dcea34c250 0x55dcea34ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:28.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.188+0000 7f4432a00700 1 -- 172.21.15.33:0/4081298558 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcea34d730 con 0x55dcea34b930 2024-05-03T18:36:28.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.188+0000 7f4433e00700 1 --2- 172.21.15.33:0/4081298558 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dcea34c250 0x55dcea34ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:36:28.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.189+0000 7f4432a00700 1 --2- 172.21.15.33:0/4081298558 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dcea34b930 0x55dcea361af0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f44200a6430 tx=0x7f4420000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8b04c24f9efabeb server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:36:28.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.190+0000 7f441fe00700 1 -- 172.21.15.33:0/4081298558 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44200a8b10 con 0x55dcea34b930 2024-05-03T18:36:28.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.190+0000 7f441fe00700 1 -- 172.21.15.33:0/4081298558 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f44200a8cd0 con 0x55dcea34b930 2024-05-03T18:36:28.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.190+0000 7f441fe00700 1 -- 172.21.15.33:0/4081298558 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44200a8930 con 0x55dcea34b930 2024-05-03T18:36:28.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.192+0000 7f4444b98b80 1 -- 172.21.15.33:0/4081298558 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dcea34b930 msgr2=0x55dcea361af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:36:28.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.192+0000 7f4444b98b80 1 --2- 172.21.15.33:0/4081298558 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dcea34b930 0x55dcea361af0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f44200a6430 tx=0x7f4420000bc0 comp rx=0 tx=0).stop 2024-05-03T18:36:28.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.192+0000 7f4444b98b80 1 -- 172.21.15.33:0/4081298558 shutdown_connections 2024-05-03T18:36:28.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.192+0000 7f4444b98b80 1 --2- 172.21.15.33:0/4081298558 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dcea34c250 0x55dcea34ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:28.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.192+0000 7f4444b98b80 1 --2- 172.21.15.33:0/4081298558 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dcea346550 0x55dcea345f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:28.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.192+0000 7f4444b98b80 1 --2- 172.21.15.33:0/4081298558 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dcea34b930 0x55dcea361af0 secure :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f44200a6430 tx=0x7f4420000bc0 comp rx=0 tx=0).stop 2024-05-03T18:36:28.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.192+0000 7f4444b98b80 1 -- 172.21.15.33:0/4081298558 >> 172.21.15.33:0/4081298558 conn(0x55dcea2be6d0 msgr2=0x55dcea34c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:36:28.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.193+0000 7f4444b98b80 1 -- 172.21.15.33:0/4081298558 shutdown_connections 2024-05-03T18:36:28.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.195+0000 7f4444b98b80 1 -- 172.21.15.33:0/4081298558 wait complete. 2024-05-03T18:36:28.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.195+0000 7f4444b98b80 1 -- start start 2024-05-03T18:36:28.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.196+0000 7f4444b98b80 1 Processor -- start 2024-05-03T18:36:28.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.196+0000 7f4444b98b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dcea346550 0x55dcea366780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:28.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.196+0000 7f4444b98b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dcea34c250 0x55dcea366cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:28.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.196+0000 7f4444b98b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dcea367200 0x55dcea3c8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:28.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.196+0000 7f4444b98b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f441400dbd0 con 0x55dcea346550 2024-05-03T18:36:28.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.196+0000 7f4444b98b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f441400f040 con 0x55dcea34c250 2024-05-03T18:36:28.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.196+0000 7f4444b98b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f441400e2a0 con 0x55dcea367200 2024-05-03T18:36:28.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.196+0000 7f4444b98b80 10 client.0 _ll_register_callbacks cb 0x55dcea23fc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:36:28.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.197+0000 7f4433e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dcea367200 0x55dcea3c8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:28.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.197+0000 7f4433e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dcea367200 0x55dcea3c8ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:57914/0 (socket says 172.21.15.33:57914) 2024-05-03T18:36:28.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.197+0000 7f4433e00700 1 -- 172.21.15.33:0/1245660701 learned_addr learned my addr 172.21.15.33:0/1245660701 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:36:28.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.197+0000 7f4432a00700 1 --2- 172.21.15.33:0/1245660701 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dcea34c250 0x55dcea366cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:28.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.197+0000 7f4433400700 1 --2- 172.21.15.33:0/1245660701 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dcea346550 0x55dcea366780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:28.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.198+0000 7f4433e00700 1 -- 172.21.15.33:0/1245660701 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dcea34c250 msgr2=0x55dcea366cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:36:28.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.198+0000 7f4433e00700 1 --2- 172.21.15.33:0/1245660701 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dcea34c250 0x55dcea366cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:28.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.198+0000 7f4433e00700 1 -- 172.21.15.33:0/1245660701 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dcea346550 msgr2=0x55dcea366780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:36:28.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.198+0000 7f4433e00700 1 --2- 172.21.15.33:0/1245660701 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dcea346550 0x55dcea366780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:28.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.198+0000 7f4433e00700 1 -- 172.21.15.33:0/1245660701 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dcea358bc0 con 0x55dcea367200 2024-05-03T18:36:28.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.198+0000 7f4433e00700 1 --2- 172.21.15.33:0/1245660701 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dcea367200 0x55dcea3c8ca0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4424005910 tx=0x7f4424005be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7f6000bcf25957b server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:36:28.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.199+0000 7f441d600700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-03T18:36:28.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.199+0000 7f441d600700 1 -- 172.21.15.33:0/1245660701 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f442400bdf0 con 0x55dcea367200 2024-05-03T18:36:28.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.199+0000 7f441d600700 1 -- 172.21.15.33:0/1245660701 <== mon.1 v2:172.21.15.150:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9518+0+0 (secure 0 0 0) 0x7f4424023460 con 0x55dcea367200 2024-05-03T18:36:28.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.200+0000 7f4444b98b80 1 -- 172.21.15.33:0/1245660701 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44200a8cd0 con 0x55dcea367200 2024-05-03T18:36:28.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.200+0000 7f441d600700 1 -- 172.21.15.33:0/1245660701 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f4424023660 con 0x55dcea367200 2024-05-03T18:36:28.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.201+0000 7f4444b98b80 10 client.26047 fetch_fsmap learned FSMap version 21 2024-05-03T18:36:28.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.201+0000 7f4444b98b80 1 -- 172.21.15.33:0/1245660701 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55dcea3ca2a0 con 0x55dcea367200 2024-05-03T18:36:28.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.201+0000 7f441d600700 1 -- 172.21.15.33:0/1245660701 <== mon.1 v2:172.21.15.150:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f44200a8cd0 con 0x55dcea367200 2024-05-03T18:36:28.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.202+0000 7f441d600700 1 -- 172.21.15.33:0/1245660701 <== mon.1 v2:172.21.15.150:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f4424039910 con 0x55dcea367200 2024-05-03T18:36:28.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.202+0000 7f4444b98b80 10 client.26047 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:36:28.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.202+0000 7f4444b98b80 10 client.26047 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:36:28.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.202+0000 7f4444b98b80 20 client.26047 populate_metadata read hostname 'smithi033' 2024-05-03T18:36:28.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.202+0000 7f4444b98b80 1 --2- 172.21.15.33:0/1245660701 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55dcea3caa30 0x55dcea3cce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:28.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.202+0000 7f4444b98b80 4 client.26047 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-03T18:36:28.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.202+0000 7f4444b98b80 1 -- 172.21.15.33:0/1245660701 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dcea2a6400 con 0x55dcea3caa30 2024-05-03T18:36:28.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.203+0000 7f4433400700 1 --2- 172.21.15.33:0/1245660701 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55dcea3caa30 0x55dcea3cce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:28.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.204+0000 7f4433400700 1 --2- 172.21.15.33:0/1245660701 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55dcea3caa30 0x55dcea3cce50 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a7fd168911c6522 server_cookie=6747e9362721bdcb in_seq=0 out_seq=0 2024-05-03T18:36:28.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.204+0000 7f441d600700 10 client.26047 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:36:28.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.204+0000 7f441d600700 1 -- 172.21.15.33:0/1245660701 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dcea2a6400 con 0x55dcea3caa30 2024-05-03T18:36:28.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.204+0000 7f441d600700 10 client.26047 handle_command_reply: tid=0 2024-05-03T18:36:28.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.204+0000 7f4444b98b80 1 client.26047 shutdown 2024-05-03T18:36:28.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.205+0000 7f4444b98b80 1 -- 172.21.15.33:0/1245660701 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dcea367200 msgr2=0x55dcea3c8ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:36:28.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.205+0000 7f4444b98b80 1 --2- 172.21.15.33:0/1245660701 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dcea367200 0x55dcea3c8ca0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4424005910 tx=0x7f4424005be0 comp rx=0 tx=0).stop 2024-05-03T18:36:28.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.205+0000 7f4444b98b80 1 -- 172.21.15.33:0/1245660701 shutdown_connections 2024-05-03T18:36:28.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.205+0000 7f4444b98b80 1 --2- 172.21.15.33:0/1245660701 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55dcea3caa30 0x55dcea3cce50 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:28.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.205+0000 7f441d600700 0 client.26047 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:36:28.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.205+0000 7f4444b98b80 1 --2- 172.21.15.33:0/1245660701 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dcea367200 0x55dcea3c8ca0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:28.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.205+0000 7f4444b98b80 1 --2- 172.21.15.33:0/1245660701 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dcea346550 0x55dcea366780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:28.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.205+0000 7f4444b98b80 1 --2- 172.21.15.33:0/1245660701 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dcea34c250 0x55dcea366cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:28.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.205+0000 7f4444b98b80 1 -- 172.21.15.33:0/1245660701 >> 172.21.15.33:0/1245660701 conn(0x55dcea2be6d0 msgr2=0x55dcea33ab20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:36:28.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.205+0000 7f4444b98b80 1 -- 172.21.15.33:0/1245660701 shutdown_connections 2024-05-03T18:36:28.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.206+0000 7f4444b98b80 1 -- 172.21.15.33:0/1245660701 wait complete. 2024-05-03T18:36:28.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.206+0000 7f4444b98b80 20 client.26047 trim_cache size 0 max 16384 2024-05-03T18:36:28.210 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-03T18:36:28.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.207+0000 7f443a200700 1 -- 172.21.15.33:0/2079381048 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f44180b2b20 msgr2=0x7f44180b4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:36:28.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.207+0000 7f443a200700 1 --2- 172.21.15.33:0/2079381048 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f44180b2b20 0x7f44180b4fe0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4424006010 tx=0x7f4424009510 comp rx=0 tx=0).stop 2024-05-03T18:36:28.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.207+0000 7f443a200700 1 -- 172.21.15.33:0/2079381048 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4434103ef0 msgr2=0x7f44340735b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:36:28.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.207+0000 7f443a200700 1 --2- 172.21.15.33:0/2079381048 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4434103ef0 0x7f44340735b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f442000d950 tx=0x7f442000dd10 comp rx=0 tx=0).stop 2024-05-03T18:36:28.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.207+0000 7f443a200700 1 -- 172.21.15.33:0/2079381048 shutdown_connections 2024-05-03T18:36:28.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.207+0000 7f443a200700 1 --2- 172.21.15.33:0/2079381048 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4434108460 0x7f44341a8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:28.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.207+0000 7f443a200700 1 --2- 172.21.15.33:0/2079381048 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4434103550 0x7f4434073070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:28.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.207+0000 7f443a200700 1 --2- 172.21.15.33:0/2079381048 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f44180b2b20 0x7f44180b4fe0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:28.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.207+0000 7f443a200700 1 --2- 172.21.15.33:0/2079381048 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4434103ef0 0x7f44340735b0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:28.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.207+0000 7f443a200700 1 -- 172.21.15.33:0/2079381048 >> 172.21.15.33:0/2079381048 conn(0x7f443407a600 msgr2=0x7f44341016a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:36:28.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.208+0000 7f443a200700 1 -- 172.21.15.33:0/2079381048 shutdown_connections 2024-05-03T18:36:28.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:28.208+0000 7f443a200700 1 -- 172.21.15.33:0/2079381048 wait complete. 2024-05-03T18:36:28.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:36:28 smithi150 ceph-mon[32492]: pgmap v287: 129 pgs: 129 active+clean; 2.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 5.3 MiB/s wr, 536 op/s 2024-05-03T18:36:28.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:36:28 smithi099 ceph-mon[31658]: pgmap v287: 129 pgs: 129 active+clean; 2.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 5.3 MiB/s wr, 536 op/s 2024-05-03T18:36:30.110 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:36:30 smithi033 ceph-mon[28330]: pgmap v288: 129 pgs: 129 active+clean; 2.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 7.8 MiB/s wr, 811 op/s 2024-05-03T18:36:30.111 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 315.0 s, 580.7 tps, lat 171.391 ms stddev 62.007 2024-05-03T18:36:30.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:36:30 smithi150 ceph-mon[32492]: pgmap v288: 129 pgs: 129 active+clean; 2.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 7.8 MiB/s wr, 811 op/s 2024-05-03T18:36:30.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:36:30 smithi099 ceph-mon[31658]: pgmap v288: 129 pgs: 129 active+clean; 2.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 7.8 MiB/s wr, 811 op/s 2024-05-03T18:36:31.648 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-03T18:36:31.649 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-03T18:36:32.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:36:32 smithi150 ceph-mon[32492]: pgmap v289: 129 pgs: 129 active+clean; 2.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 5.2 MiB/s wr, 543 op/s 2024-05-03T18:36:32.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:36:32 smithi099 ceph-mon[31658]: pgmap v289: 129 pgs: 129 active+clean; 2.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 5.2 MiB/s wr, 543 op/s 2024-05-03T18:36:32.693 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:36:32 smithi033 ceph-mon[28330]: pgmap v289: 129 pgs: 129 active+clean; 2.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 5.2 MiB/s wr, 543 op/s 2024-05-03T18:36:34.025 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:36:34.376 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:36:34 smithi033 ceph-mon[28330]: pgmap v290: 129 pgs: 129 active+clean; 2.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 7.7 MiB/s wr, 796 op/s 2024-05-03T18:36:34.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:36:34 smithi150 ceph-mon[32492]: pgmap v290: 129 pgs: 129 active+clean; 2.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 7.7 MiB/s wr, 796 op/s 2024-05-03T18:36:34.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:36:34 smithi099 ceph-mon[31658]: pgmap v290: 129 pgs: 129 active+clean; 2.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 7.7 MiB/s wr, 796 op/s 2024-05-03T18:36:35.111 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 320.0 s, 588.2 tps, lat 169.965 ms stddev 93.731 2024-05-03T18:36:36.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:36:36 smithi150 ceph-mon[32492]: pgmap v291: 129 pgs: 129 active+clean; 2.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 5.0 MiB/s wr, 529 op/s 2024-05-03T18:36:36.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:36:36 smithi099 ceph-mon[31658]: pgmap v291: 129 pgs: 129 active+clean; 2.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 5.0 MiB/s wr, 529 op/s 2024-05-03T18:36:36.607 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:36:36 smithi033 ceph-mon[28330]: pgmap v291: 129 pgs: 129 active+clean; 2.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 5.0 MiB/s wr, 529 op/s 2024-05-03T18:36:38.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:36:38 smithi150 ceph-mon[32492]: pgmap v292: 129 pgs: 129 active+clean; 2.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 5.0 MiB/s wr, 529 op/s 2024-05-03T18:36:38.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:36:38 smithi033 ceph-mon[28330]: pgmap v292: 129 pgs: 129 active+clean; 2.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 5.0 MiB/s wr, 529 op/s 2024-05-03T18:36:38.547 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:36:38 smithi099 ceph-mon[31658]: pgmap v292: 129 pgs: 129 active+clean; 2.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 5.0 MiB/s wr, 529 op/s 2024-05-03T18:36:39.920 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-03T18:36:39.920 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-05-03T18:36:39.920 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "aeb6d998-abc5-4b47-9595-96f38bd2adb1", 2024-05-03T18:36:39.920 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-05-03T18:36:39.920 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-03T18:36:39.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.701+0000 7fca33e00700 1 -- 172.21.15.33:0/3151685833 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca34109460 msgr2=0x7fca34109840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:36:39.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.701+0000 7fca33e00700 1 --2- 172.21.15.33:0/3151685833 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca34109460 0x7fca34109840 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fca24009990 tx=0x7fca24009ca0 comp rx=0 tx=0).stop 2024-05-03T18:36:39.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.701+0000 7fca33e00700 1 -- 172.21.15.33:0/3151685833 shutdown_connections 2024-05-03T18:36:39.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.701+0000 7fca33e00700 1 --2- 172.21.15.33:0/3151685833 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fca34103d90 0x7fca341102a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:39.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.701+0000 7fca33e00700 1 --2- 172.21.15.33:0/3151685833 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca34109460 0x7fca34109840 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:39.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.701+0000 7fca33e00700 1 --2- 172.21.15.33:0/3151685833 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca341033d0 0x7fca34103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:39.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.701+0000 7fca33e00700 1 -- 172.21.15.33:0/3151685833 >> 172.21.15.33:0/3151685833 conn(0x7fca340fef20 msgr2=0x7fca34101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:36:39.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.701+0000 7fca33e00700 1 -- 172.21.15.33:0/3151685833 shutdown_connections 2024-05-03T18:36:39.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.701+0000 7fca33e00700 1 -- 172.21.15.33:0/3151685833 wait complete. 2024-05-03T18:36:39.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.702+0000 7fca33e00700 1 Processor -- start 2024-05-03T18:36:39.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.702+0000 7fca33e00700 1 -- start start 2024-05-03T18:36:39.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.703+0000 7fca33e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fca341033d0 0x7fca341a17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:39.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.703+0000 7fca33e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca34103d90 0x7fca341a1d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:39.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.703+0000 7fca33e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca34109460 0x7fca3419b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:39.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.703+0000 7fca33e00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fca34113470 con 0x7fca34109460 2024-05-03T18:36:39.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.703+0000 7fca33e00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fca341132f0 con 0x7fca34103d90 2024-05-03T18:36:39.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.703+0000 7fca33e00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fca341135f0 con 0x7fca341033d0 2024-05-03T18:36:39.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.703+0000 7fca33400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca34109460 0x7fca3419b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:39.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.703+0000 7fca33400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca34109460 0x7fca3419b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48054/0 (socket says 172.21.15.33:48054) 2024-05-03T18:36:39.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.703+0000 7fca33400700 1 -- 172.21.15.33:0/4224974045 learned_addr learned my addr 172.21.15.33:0/4224974045 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:36:39.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.703+0000 7fca32a00700 1 --2- 172.21.15.33:0/4224974045 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fca341033d0 0x7fca341a17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:39.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.703+0000 7fca32000700 1 --2- 172.21.15.33:0/4224974045 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca34103d90 0x7fca341a1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:39.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.703+0000 7fca33400700 1 -- 172.21.15.33:0/4224974045 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fca341033d0 msgr2=0x7fca341a17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:36:39.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.703+0000 7fca33400700 1 --2- 172.21.15.33:0/4224974045 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fca341033d0 0x7fca341a17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:39.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.703+0000 7fca33400700 1 -- 172.21.15.33:0/4224974045 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca34103d90 msgr2=0x7fca341a1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:36:39.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.703+0000 7fca33400700 1 --2- 172.21.15.33:0/4224974045 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca34103d90 0x7fca341a1d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:39.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.703+0000 7fca33400700 1 -- 172.21.15.33:0/4224974045 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca3419c0e0 con 0x7fca34109460 2024-05-03T18:36:39.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.703+0000 7fca32000700 1 --2- 172.21.15.33:0/4224974045 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca34103d90 0x7fca341a1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:36:39.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.704+0000 7fca33400700 1 --2- 172.21.15.33:0/4224974045 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca34109460 0x7fca3419b980 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fca2c007d70 tx=0x7fca2c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:36:39.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.704+0000 7fca2b400700 1 -- 172.21.15.33:0/4224974045 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca2c016020 con 0x7fca34109460 2024-05-03T18:36:39.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.704+0000 7fca2b400700 1 -- 172.21.15.33:0/4224974045 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fca2c00ad40 con 0x7fca34109460 2024-05-03T18:36:39.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.704+0000 7fca2b400700 1 -- 172.21.15.33:0/4224974045 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca2c0115e0 con 0x7fca34109460 2024-05-03T18:36:39.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.704+0000 7fca33e00700 1 -- 172.21.15.33:0/4224974045 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca3419c3d0 con 0x7fca34109460 2024-05-03T18:36:39.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.704+0000 7fca33e00700 1 -- 172.21.15.33:0/4224974045 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca341a8690 con 0x7fca34109460 2024-05-03T18:36:39.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.708+0000 7fca2b400700 1 -- 172.21.15.33:0/4224974045 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fca2c019020 con 0x7fca34109460 2024-05-03T18:36:39.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.708+0000 7fca4416ab80 1 Processor -- start 2024-05-03T18:36:39.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.708+0000 7fca4416ab80 1 -- start start 2024-05-03T18:36:39.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.709+0000 7fca4416ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560b2f377c50 0x560b2f378030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:39.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.709+0000 7fca4416ab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560b2f371fb0 0x560b2f372430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:39.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.709+0000 7fca4416ab80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b2f372970 0x560b2f37ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:39.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.709+0000 7fca4416ab80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560b2f2c4aa0 con 0x560b2f372970 2024-05-03T18:36:39.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.709+0000 7fca4416ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x560b2f2627e0 con 0x560b2f377c50 2024-05-03T18:36:39.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.709+0000 7fca4416ab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x560b2f2c0a60 con 0x560b2f371fb0 2024-05-03T18:36:39.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.709+0000 7fca2b400700 1 --2- 172.21.15.33:0/4224974045 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fca180b2a70 0x7fca180b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:39.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.709+0000 7fca2b400700 1 -- 172.21.15.33:0/4224974045 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9518+0+0 (secure 0 0 0) 0x7fca2c0e28b0 con 0x7fca34109460 2024-05-03T18:36:39.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.709+0000 7fca33400700 1 --2- 172.21.15.33:0/4224974045 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fca180b2a70 0x7fca180b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:39.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.709+0000 7fca32000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b2f372970 0x560b2f37ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:39.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.709+0000 7fca32000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b2f372970 0x560b2f37ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48064/0 (socket says 172.21.15.33:48064) 2024-05-03T18:36:39.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.709+0000 7fca32000700 1 -- 172.21.15.33:0/4102224766 learned_addr learned my addr 172.21.15.33:0/4102224766 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:36:39.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.709+0000 7fca32000700 1 --2- 172.21.15.33:0/4102224766 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560b2f377c50 0x560b2f378030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:39.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.709+0000 7fca32a00700 1 --2- 172.21.15.33:0/4102224766 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560b2f371fb0 0x560b2f372430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:39.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.710+0000 7fca32000700 1 -- 172.21.15.33:0/4102224766 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560b2f371fb0 msgr2=0x560b2f372430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:36:39.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.710+0000 7fca32000700 1 --2- 172.21.15.33:0/4102224766 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560b2f371fb0 0x560b2f372430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:39.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.710+0000 7fca32000700 1 -- 172.21.15.33:0/4102224766 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560b2f377c50 msgr2=0x560b2f378030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:36:39.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.710+0000 7fca32000700 1 --2- 172.21.15.33:0/4102224766 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560b2f377c50 0x560b2f378030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:39.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.710+0000 7fca32000700 1 -- 172.21.15.33:0/4102224766 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b2f211950 con 0x560b2f372970 2024-05-03T18:36:39.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.710+0000 7fca33400700 1 --2- 172.21.15.33:0/4224974045 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fca180b2a70 0x7fca180b4f30 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fca2c007d40 tx=0x7fca2c0cc040 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:36:39.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.710+0000 7fca32000700 1 --2- 172.21.15.33:0/4102224766 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b2f372970 0x560b2f37ea10 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fca200125b0 tx=0x7fca2000d440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a03b35ae9d9c5a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:36:39.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.710+0000 7fca1fe00700 1 -- 172.21.15.33:0/4102224766 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca2001c600 con 0x560b2f372970 2024-05-03T18:36:39.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.710+0000 7fca1fe00700 1 -- 172.21.15.33:0/4102224766 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fca20010860 con 0x560b2f372970 2024-05-03T18:36:39.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.710+0000 7fca1fe00700 1 -- 172.21.15.33:0/4102224766 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca2000f6e0 con 0x560b2f372970 2024-05-03T18:36:39.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.711+0000 7fca4416ab80 1 -- 172.21.15.33:0/4102224766 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b2f372970 msgr2=0x560b2f37ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:36:39.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.711+0000 7fca4416ab80 1 --2- 172.21.15.33:0/4102224766 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b2f372970 0x560b2f37ea10 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fca200125b0 tx=0x7fca2000d440 comp rx=0 tx=0).stop 2024-05-03T18:36:39.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.711+0000 7fca4416ab80 1 -- 172.21.15.33:0/4102224766 shutdown_connections 2024-05-03T18:36:39.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.711+0000 7fca4416ab80 1 --2- 172.21.15.33:0/4102224766 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b2f372970 0x560b2f37ea10 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:39.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.711+0000 7fca4416ab80 1 --2- 172.21.15.33:0/4102224766 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560b2f377c50 0x560b2f378030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:39.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.711+0000 7fca4416ab80 1 --2- 172.21.15.33:0/4102224766 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560b2f371fb0 0x560b2f372430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:39.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.711+0000 7fca4416ab80 1 -- 172.21.15.33:0/4102224766 >> 172.21.15.33:0/4102224766 conn(0x560b2f2ea750 msgr2=0x560b2f36cc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:36:39.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.711+0000 7fca4416ab80 1 -- 172.21.15.33:0/4102224766 shutdown_connections 2024-05-03T18:36:39.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.711+0000 7fca4416ab80 1 -- 172.21.15.33:0/4102224766 wait complete. 2024-05-03T18:36:39.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.712+0000 7fca4416ab80 1 -- start start 2024-05-03T18:36:39.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.712+0000 7fca4416ab80 1 Processor -- start 2024-05-03T18:36:39.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.713+0000 7fca4416ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560b2f371fb0 0x560b2f37e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:39.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.713+0000 7fca4416ab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560b2f372970 0x560b2f37ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:39.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.713+0000 7fca4416ab80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b2f377c50 0x560b2f37bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:39.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.713+0000 7fca4416ab80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560b2f37efb0 con 0x560b2f377c50 2024-05-03T18:36:39.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.713+0000 7fca4416ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x560b2f22aba0 con 0x560b2f371fb0 2024-05-03T18:36:39.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.713+0000 7fca4416ab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x560b2f37f130 con 0x560b2f372970 2024-05-03T18:36:39.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.713+0000 7fca4416ab80 10 client.0 _ll_register_callbacks cb 0x560b2f26bcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:36:39.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.713+0000 7fca32000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b2f377c50 0x560b2f37bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:39.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.713+0000 7fca32000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b2f377c50 0x560b2f37bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48070/0 (socket says 172.21.15.33:48070) 2024-05-03T18:36:39.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.713+0000 7fca32000700 1 -- 172.21.15.33:0/3786139999 learned_addr learned my addr 172.21.15.33:0/3786139999 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:36:39.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.713+0000 7fca32000700 1 --2- 172.21.15.33:0/3786139999 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560b2f371fb0 0x560b2f37e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:39.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.713+0000 7fca32000700 1 -- 172.21.15.33:0/3786139999 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560b2f372970 msgr2=0x560b2f37ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:36:39.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.713+0000 7fca32000700 1 --2- 172.21.15.33:0/3786139999 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560b2f372970 0x560b2f37ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:39.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.713+0000 7fca32000700 1 -- 172.21.15.33:0/3786139999 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560b2f371fb0 msgr2=0x560b2f37e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:36:39.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.713+0000 7fca32000700 1 --2- 172.21.15.33:0/3786139999 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560b2f371fb0 0x560b2f37e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:39.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.713+0000 7fca32000700 1 -- 172.21.15.33:0/3786139999 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b2f28a4a0 con 0x560b2f377c50 2024-05-03T18:36:39.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.714+0000 7fca32000700 1 --2- 172.21.15.33:0/3786139999 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b2f377c50 0x560b2f37bfb0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fca2000d410 tx=0x7fca20014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f07cb569d4e57775 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:36:39.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.714+0000 7fca1cc00700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:36:39.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.714+0000 7fca1cc00700 1 -- 172.21.15.33:0/3786139999 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca2001c400 con 0x560b2f377c50 2024-05-03T18:36:39.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.714+0000 7fca1cc00700 1 -- 172.21.15.33:0/3786139999 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9518+0+0 (secure 0 0 0) 0x7fca2001c5a0 con 0x560b2f377c50 2024-05-03T18:36:39.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.714+0000 7fca4416ab80 1 -- 172.21.15.33:0/3786139999 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca20010860 con 0x560b2f377c50 2024-05-03T18:36:39.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.714+0000 7fca1cc00700 1 -- 172.21.15.33:0/3786139999 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fca20030050 con 0x560b2f377c50 2024-05-03T18:36:39.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.714+0000 7fca4416ab80 10 client.16338 fetch_fsmap learned FSMap version 21 2024-05-03T18:36:39.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.714+0000 7fca4416ab80 1 -- 172.21.15.33:0/3786139999 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x560b2f389820 con 0x560b2f377c50 2024-05-03T18:36:39.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.714+0000 7fca1cc00700 1 -- 172.21.15.33:0/3786139999 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fca20010860 con 0x560b2f377c50 2024-05-03T18:36:39.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.714+0000 7fca1cc00700 1 -- 172.21.15.33:0/3786139999 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fca2000f7f0 con 0x560b2f377c50 2024-05-03T18:36:39.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.715+0000 7fca4416ab80 10 client.16338 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:36:39.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.715+0000 7fca4416ab80 10 client.16338 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:36:39.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.715+0000 7fca4416ab80 20 client.16338 populate_metadata read hostname 'smithi033' 2024-05-03T18:36:39.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.715+0000 7fca4416ab80 1 --2- 172.21.15.33:0/3786139999 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x560b2f38bcf0 0x560b2f38e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:39.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.715+0000 7fca4416ab80 4 client.16338 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:36:39.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.715+0000 7fca4416ab80 1 -- 172.21.15.33:0/3786139999 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560b2f33f070 con 0x560b2f38bcf0 2024-05-03T18:36:39.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.715+0000 7fca32a00700 1 --2- 172.21.15.33:0/3786139999 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x560b2f38bcf0 0x560b2f38e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:39.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.716+0000 7fca1cc00700 10 client.16338 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:36:39.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.716+0000 7fca32a00700 1 --2- 172.21.15.33:0/3786139999 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x560b2f38bcf0 0x560b2f38e0e0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc4b87c1145010c5 server_cookie=52144a141bc70092 in_seq=0 out_seq=0 2024-05-03T18:36:39.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.718+0000 7fca1cc00700 1 -- 172.21.15.33:0/3786139999 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560b2f33f070 con 0x560b2f38bcf0 2024-05-03T18:36:39.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.718+0000 7fca1cc00700 10 client.16338 handle_command_reply: tid=0 2024-05-03T18:36:39.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.718+0000 7fca4416ab80 1 client.16338 shutdown 2024-05-03T18:36:39.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.718+0000 7fca4416ab80 1 -- 172.21.15.33:0/3786139999 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b2f377c50 msgr2=0x560b2f37bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:36:39.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.718+0000 7fca4416ab80 1 --2- 172.21.15.33:0/3786139999 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b2f377c50 0x560b2f37bfb0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fca2000d410 tx=0x7fca20014040 comp rx=0 tx=0).stop 2024-05-03T18:36:39.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.718+0000 7fca4416ab80 1 -- 172.21.15.33:0/3786139999 shutdown_connections 2024-05-03T18:36:39.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.718+0000 7fca4416ab80 1 --2- 172.21.15.33:0/3786139999 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x560b2f38bcf0 0x560b2f38e0e0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:39.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.718+0000 7fca4416ab80 1 --2- 172.21.15.33:0/3786139999 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b2f377c50 0x560b2f37bfb0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:39.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.718+0000 7fca4416ab80 1 --2- 172.21.15.33:0/3786139999 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560b2f371fb0 0x560b2f37e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:39.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.718+0000 7fca4416ab80 1 --2- 172.21.15.33:0/3786139999 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560b2f372970 0x560b2f37ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:39.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.718+0000 7fca4416ab80 1 -- 172.21.15.33:0/3786139999 >> 172.21.15.33:0/3786139999 conn(0x560b2f2ea750 msgr2=0x560b2f370fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:36:39.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.718+0000 7fca1cc00700 0 client.16338 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:36:39.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.718+0000 7fca4416ab80 1 -- 172.21.15.33:0/3786139999 shutdown_connections 2024-05-03T18:36:39.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.718+0000 7fca4416ab80 1 -- 172.21.15.33:0/3786139999 wait complete. 2024-05-03T18:36:39.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.719+0000 7fca4416ab80 20 client.16338 trim_cache size 0 max 16384 2024-05-03T18:36:39.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.740+0000 7fca4416ab80 1 Processor -- start 2024-05-03T18:36:39.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.740+0000 7fca4416ab80 1 -- start start 2024-05-03T18:36:39.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.740+0000 7fca4416ab80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b2f371fb0 0x560b2f37df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:39.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.740+0000 7fca4416ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560b2f372970 0x560b2f388000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:39.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.740+0000 7fca4416ab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560b2f377c50 0x560b2f37aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:39.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.740+0000 7fca4416ab80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fca341af580 con 0x560b2f371fb0 2024-05-03T18:36:39.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.740+0000 7fca4416ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fca341b1de0 con 0x560b2f372970 2024-05-03T18:36:39.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.740+0000 7fca4416ab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fca341afbd0 con 0x560b2f377c50 2024-05-03T18:36:39.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.741+0000 7fca32000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b2f371fb0 0x560b2f37df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:39.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.741+0000 7fca32000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b2f371fb0 0x560b2f37df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48080/0 (socket says 172.21.15.33:48080) 2024-05-03T18:36:39.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.741+0000 7fca32000700 1 -- 172.21.15.33:0/518452533 learned_addr learned my addr 172.21.15.33:0/518452533 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:36:39.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.741+0000 7fca32000700 1 -- 172.21.15.33:0/518452533 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560b2f377c50 msgr2=0x560b2f37aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:36:39.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.741+0000 7fca32000700 1 --2- 172.21.15.33:0/518452533 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560b2f377c50 0x560b2f37aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:39.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.741+0000 7fca32000700 1 -- 172.21.15.33:0/518452533 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560b2f372970 msgr2=0x560b2f388000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:36:39.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.741+0000 7fca32000700 1 --2- 172.21.15.33:0/518452533 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560b2f372970 0x560b2f388000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:39.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.741+0000 7fca32000700 1 -- 172.21.15.33:0/518452533 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b2f33f050 con 0x560b2f371fb0 2024-05-03T18:36:39.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.741+0000 7fca32000700 1 --2- 172.21.15.33:0/518452533 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b2f371fb0 0x560b2f37df10 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fca2000fd60 tx=0x7fca2000dd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb33daaba5e606ef server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:36:39.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.742+0000 7fca1f400700 1 -- 172.21.15.33:0/518452533 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca2001b3b0 con 0x560b2f371fb0 2024-05-03T18:36:39.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.742+0000 7fca1f400700 1 -- 172.21.15.33:0/518452533 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fca20039070 con 0x560b2f371fb0 2024-05-03T18:36:39.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.742+0000 7fca1f400700 1 -- 172.21.15.33:0/518452533 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca2001b5f0 con 0x560b2f371fb0 2024-05-03T18:36:39.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.742+0000 7fca4416ab80 1 -- 172.21.15.33:0/518452533 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b2f371fb0 msgr2=0x560b2f37df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:36:39.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.742+0000 7fca4416ab80 1 --2- 172.21.15.33:0/518452533 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b2f371fb0 0x560b2f37df10 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fca2000fd60 tx=0x7fca2000dd60 comp rx=0 tx=0).stop 2024-05-03T18:36:39.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.742+0000 7fca4416ab80 1 -- 172.21.15.33:0/518452533 shutdown_connections 2024-05-03T18:36:39.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.742+0000 7fca4416ab80 1 --2- 172.21.15.33:0/518452533 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560b2f377c50 0x560b2f37aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:39.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.742+0000 7fca4416ab80 1 --2- 172.21.15.33:0/518452533 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b2f371fb0 0x560b2f37df10 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:39.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.742+0000 7fca4416ab80 1 --2- 172.21.15.33:0/518452533 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560b2f372970 0x560b2f388000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:39.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.743+0000 7fca4416ab80 1 -- 172.21.15.33:0/518452533 >> 172.21.15.33:0/518452533 conn(0x560b2f2ea750 msgr2=0x560b2f37a360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:36:39.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.743+0000 7fca4416ab80 1 -- 172.21.15.33:0/518452533 shutdown_connections 2024-05-03T18:36:39.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.743+0000 7fca4416ab80 1 -- 172.21.15.33:0/518452533 wait complete. 2024-05-03T18:36:39.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.743+0000 7fca4416ab80 1 -- start start 2024-05-03T18:36:39.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.743+0000 7fca4416ab80 1 Processor -- start 2024-05-03T18:36:39.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.744+0000 7fca4416ab80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b2f371fb0 0x560b2f381070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:39.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.744+0000 7fca4416ab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560b2f372970 0x560b2f3815b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:39.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.744+0000 7fca4416ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560b2f377c50 0x560b2f381af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:39.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.744+0000 7fca4416ab80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fca341b02a0 con 0x560b2f371fb0 2024-05-03T18:36:39.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.744+0000 7fca4416ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fca341b1710 con 0x560b2f377c50 2024-05-03T18:36:39.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.744+0000 7fca4416ab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fca341b0970 con 0x560b2f372970 2024-05-03T18:36:39.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.744+0000 7fca4416ab80 10 client.0 _ll_register_callbacks cb 0x560b2f26bcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:36:39.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.744+0000 7fca32000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b2f371fb0 0x560b2f381070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:39.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.744+0000 7fca32000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b2f371fb0 0x560b2f381070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48094/0 (socket says 172.21.15.33:48094) 2024-05-03T18:36:39.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.744+0000 7fca32000700 1 -- 172.21.15.33:0/3092050465 learned_addr learned my addr 172.21.15.33:0/3092050465 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:36:39.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.745+0000 7fca32a00700 1 --2- 172.21.15.33:0/3092050465 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560b2f372970 0x560b2f3815b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:39.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.745+0000 7fca32000700 1 -- 172.21.15.33:0/3092050465 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560b2f372970 msgr2=0x560b2f3815b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:36:39.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.745+0000 7fca32000700 1 --2- 172.21.15.33:0/3092050465 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560b2f372970 0x560b2f3815b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:39.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.745+0000 7fca32000700 1 -- 172.21.15.33:0/3092050465 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560b2f377c50 msgr2=0x560b2f381af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:36:39.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.745+0000 7fca32000700 1 --2- 172.21.15.33:0/3092050465 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560b2f377c50 0x560b2f381af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:39.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.745+0000 7fca32000700 1 -- 172.21.15.33:0/3092050465 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b2f372f40 con 0x560b2f371fb0 2024-05-03T18:36:39.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.745+0000 7fca32a00700 1 --2- 172.21.15.33:0/3092050465 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560b2f372970 0x560b2f3815b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:36:39.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.745+0000 7fca32000700 1 --2- 172.21.15.33:0/3092050465 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b2f371fb0 0x560b2f381070 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fca2000dce0 tx=0x7fca200047a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1da17b883310874 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:36:39.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.745+0000 7fca1cc00700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:36:39.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.745+0000 7fca1cc00700 1 -- 172.21.15.33:0/3092050465 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca2000d720 con 0x560b2f371fb0 2024-05-03T18:36:39.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.745+0000 7fca1cc00700 1 -- 172.21.15.33:0/3092050465 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9518+0+0 (secure 0 0 0) 0x7fca20010760 con 0x560b2f371fb0 2024-05-03T18:36:39.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.745+0000 7fca4416ab80 1 -- 172.21.15.33:0/3092050465 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca20039070 con 0x560b2f371fb0 2024-05-03T18:36:39.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.746+0000 7fca1cc00700 1 -- 172.21.15.33:0/3092050465 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fca2000f3c0 con 0x560b2f371fb0 2024-05-03T18:36:39.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.746+0000 7fca4416ab80 10 client.16344 fetch_fsmap learned FSMap version 21 2024-05-03T18:36:39.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.746+0000 7fca4416ab80 1 -- 172.21.15.33:0/3092050465 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x560b2f3f7500 con 0x560b2f371fb0 2024-05-03T18:36:39.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.746+0000 7fca1cc00700 1 -- 172.21.15.33:0/3092050465 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fca20039070 con 0x560b2f371fb0 2024-05-03T18:36:39.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.746+0000 7fca1cc00700 1 -- 172.21.15.33:0/3092050465 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fca2001b9d0 con 0x560b2f371fb0 2024-05-03T18:36:39.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.746+0000 7fca4416ab80 10 client.16344 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:36:39.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.746+0000 7fca4416ab80 10 client.16344 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:36:39.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.746+0000 7fca4416ab80 20 client.16344 populate_metadata read hostname 'smithi033' 2024-05-03T18:36:39.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.746+0000 7fca4416ab80 1 --2- 172.21.15.33:0/3092050465 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x560b2f3f7b20 0x560b2f3f9f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:39.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.746+0000 7fca4416ab80 4 client.16344 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-03T18:36:39.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.746+0000 7fca4416ab80 1 -- 172.21.15.33:0/3092050465 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x560b2f2d2480 con 0x560b2f3f7b20 2024-05-03T18:36:39.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.747+0000 7fca32a00700 1 --2- 172.21.15.33:0/3092050465 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x560b2f3f7b20 0x560b2f3f9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:39.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.748+0000 7fca32a00700 1 --2- 172.21.15.33:0/3092050465 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x560b2f3f7b20 0x560b2f3f9f40 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47b4207ce6b2ab48 server_cookie=e13401f1b60add72 in_seq=0 out_seq=0 2024-05-03T18:36:39.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.748+0000 7fca1cc00700 10 client.16344 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:36:39.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.751+0000 7fca1cc00700 1 -- 172.21.15.33:0/3092050465 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560b2f2d2480 con 0x560b2f3f7b20 2024-05-03T18:36:39.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.751+0000 7fca1cc00700 10 client.16344 handle_command_reply: tid=0 2024-05-03T18:36:39.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.751+0000 7fca4416ab80 1 client.16344 shutdown 2024-05-03T18:36:39.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.751+0000 7fca4416ab80 1 -- 172.21.15.33:0/3092050465 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b2f371fb0 msgr2=0x560b2f381070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:36:39.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.751+0000 7fca4416ab80 1 --2- 172.21.15.33:0/3092050465 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b2f371fb0 0x560b2f381070 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fca2000dce0 tx=0x7fca200047a0 comp rx=0 tx=0).stop 2024-05-03T18:36:39.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.751+0000 7fca4416ab80 1 -- 172.21.15.33:0/3092050465 shutdown_connections 2024-05-03T18:36:39.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.751+0000 7fca4416ab80 1 --2- 172.21.15.33:0/3092050465 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x560b2f3f7b20 0x560b2f3f9f40 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:39.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.751+0000 7fca4416ab80 1 --2- 172.21.15.33:0/3092050465 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560b2f377c50 0x560b2f381af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:39.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.751+0000 7fca4416ab80 1 --2- 172.21.15.33:0/3092050465 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b2f371fb0 0x560b2f381070 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:39.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.751+0000 7fca4416ab80 1 --2- 172.21.15.33:0/3092050465 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560b2f372970 0x560b2f3815b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:39.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.751+0000 7fca1cc00700 0 client.16344 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:36:39.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.751+0000 7fca4416ab80 1 -- 172.21.15.33:0/3092050465 >> 172.21.15.33:0/3092050465 conn(0x560b2f2ea750 msgr2=0x560b2f375f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:36:39.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.752+0000 7fca4416ab80 1 -- 172.21.15.33:0/3092050465 shutdown_connections 2024-05-03T18:36:39.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.752+0000 7fca4416ab80 1 -- 172.21.15.33:0/3092050465 wait complete. 2024-05-03T18:36:39.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.752+0000 7fca4416ab80 20 client.16344 trim_cache size 0 max 16384 2024-05-03T18:36:39.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.752+0000 7fca33e00700 1 -- 172.21.15.33:0/4224974045 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fca180b2a70 msgr2=0x7fca180b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:36:39.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.752+0000 7fca33e00700 1 --2- 172.21.15.33:0/4224974045 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fca180b2a70 0x7fca180b4f30 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fca2c007d40 tx=0x7fca2c0cc040 comp rx=0 tx=0).stop 2024-05-03T18:36:39.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.752+0000 7fca33e00700 1 -- 172.21.15.33:0/4224974045 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca34109460 msgr2=0x7fca3419b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:36:39.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.752+0000 7fca33e00700 1 --2- 172.21.15.33:0/4224974045 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca34109460 0x7fca3419b980 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fca2c007d70 tx=0x7fca2c00a340 comp rx=0 tx=0).stop 2024-05-03T18:36:39.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.752+0000 7fca33e00700 1 -- 172.21.15.33:0/4224974045 shutdown_connections 2024-05-03T18:36:39.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.752+0000 7fca33e00700 1 --2- 172.21.15.33:0/4224974045 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca34109460 0x7fca3419b980 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:39.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.752+0000 7fca33e00700 1 --2- 172.21.15.33:0/4224974045 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fca341033d0 0x7fca341a17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:39.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.752+0000 7fca33e00700 1 --2- 172.21.15.33:0/4224974045 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fca180b2a70 0x7fca180b4f30 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:39.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.752+0000 7fca33e00700 1 --2- 172.21.15.33:0/4224974045 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca34103d90 0x7fca341a1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:39.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.752+0000 7fca33e00700 1 -- 172.21.15.33:0/4224974045 >> 172.21.15.33:0/4224974045 conn(0x7fca340fef20 msgr2=0x7fca3410ff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:36:39.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.753+0000 7fca33e00700 1 -- 172.21.15.33:0/4224974045 shutdown_connections 2024-05-03T18:36:39.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:39.753+0000 7fca33e00700 1 -- 172.21.15.33:0/4224974045 wait complete. 2024-05-03T18:36:40.109 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:36:40 smithi033 ceph-mon[28330]: pgmap v293: 129 pgs: 129 active+clean; 2.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 7.7 MiB/s wr, 816 op/s 2024-05-03T18:36:40.110 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 325.0 s, 628.1 tps, lat 157.947 ms stddev 106.259 2024-05-03T18:36:40.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:36:40 smithi150 ceph-mon[32492]: pgmap v293: 129 pgs: 129 active+clean; 2.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 7.7 MiB/s wr, 816 op/s 2024-05-03T18:36:40.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:36:40 smithi099 ceph-mon[31658]: pgmap v293: 129 pgs: 129 active+clean; 2.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 7.7 MiB/s wr, 816 op/s 2024-05-03T18:36:41.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:36:41 smithi150 ceph-mon[32492]: scrub queued for path: / 2024-05-03T18:36:41.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:36:41 smithi150 ceph-mon[32492]: scrub summary: idle+waiting paths [/] 2024-05-03T18:36:41.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:36:41 smithi150 ceph-mon[32492]: scrub summary: active paths [/] 2024-05-03T18:36:41.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:36:41 smithi150 ceph-mon[32492]: scrub summary: idle+waiting paths [/] 2024-05-03T18:36:41.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:36:41 smithi033 ceph-mon[28330]: scrub queued for path: / 2024-05-03T18:36:41.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:36:41 smithi033 ceph-mon[28330]: scrub summary: idle+waiting paths [/] 2024-05-03T18:36:41.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:36:41 smithi033 ceph-mon[28330]: scrub summary: active paths [/] 2024-05-03T18:36:41.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:36:41 smithi033 ceph-mon[28330]: scrub summary: idle+waiting paths [/] 2024-05-03T18:36:41.548 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:36:41 smithi099 ceph-mon[31658]: scrub queued for path: / 2024-05-03T18:36:41.548 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:36:41 smithi099 ceph-mon[31658]: scrub summary: idle+waiting paths [/] 2024-05-03T18:36:41.548 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:36:41 smithi099 ceph-mon[31658]: scrub summary: active paths [/] 2024-05-03T18:36:41.548 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:36:41 smithi099 ceph-mon[31658]: scrub summary: idle+waiting paths [/] 2024-05-03T18:36:42.230 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-03T18:36:42.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:36:42 smithi150 ceph-mon[32492]: scrub summary: idle 2024-05-03T18:36:42.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:36:42 smithi150 ceph-mon[32492]: pgmap v294: 129 pgs: 129 active+clean; 2.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 5.1 MiB/s wr, 541 op/s 2024-05-03T18:36:42.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:36:42 smithi033 ceph-mon[28330]: scrub summary: idle 2024-05-03T18:36:42.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:36:42 smithi033 ceph-mon[28330]: pgmap v294: 129 pgs: 129 active+clean; 2.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 5.1 MiB/s wr, 541 op/s 2024-05-03T18:36:42.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:36:42 smithi099 ceph-mon[31658]: scrub summary: idle 2024-05-03T18:36:42.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:36:42 smithi099 ceph-mon[31658]: pgmap v294: 129 pgs: 129 active+clean; 2.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 5.1 MiB/s wr, 541 op/s 2024-05-03T18:36:43.270 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:36:44.111 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:36:44 smithi033 ceph-mon[28330]: pgmap v295: 129 pgs: 129 active+clean; 2.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 7.4 MiB/s wr, 766 op/s 2024-05-03T18:36:44.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:36:44 smithi150 ceph-mon[32492]: pgmap v295: 129 pgs: 129 active+clean; 2.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 7.4 MiB/s wr, 766 op/s 2024-05-03T18:36:44.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:36:44 smithi099 ceph-mon[31658]: pgmap v295: 129 pgs: 129 active+clean; 2.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 7.4 MiB/s wr, 766 op/s 2024-05-03T18:36:45.110 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 330.0 s, 588.1 tps, lat 172.049 ms stddev 94.983 2024-05-03T18:36:46.238 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:36:46 smithi033 ceph-mon[28330]: pgmap v296: 129 pgs: 129 active+clean; 2.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 5.0 MiB/s wr, 513 op/s 2024-05-03T18:36:46.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:36:46 smithi150 ceph-mon[32492]: pgmap v296: 129 pgs: 129 active+clean; 2.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 5.0 MiB/s wr, 513 op/s 2024-05-03T18:36:46.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:36:46 smithi099 ceph-mon[31658]: pgmap v296: 129 pgs: 129 active+clean; 2.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 5.0 MiB/s wr, 513 op/s 2024-05-03T18:36:48.199 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:36:48 smithi033 ceph-mon[28330]: pgmap v297: 129 pgs: 129 active+clean; 2.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 5.0 MiB/s wr, 513 op/s 2024-05-03T18:36:48.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:36:48 smithi150 ceph-mon[32492]: pgmap v297: 129 pgs: 129 active+clean; 2.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 5.0 MiB/s wr, 513 op/s 2024-05-03T18:36:48.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.524+0000 7f9d85e00700 1 -- 172.21.15.33:0/135523410 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d80109460 msgr2=0x7f9d80109840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:36:48.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.524+0000 7f9d85e00700 1 --2- 172.21.15.33:0/135523410 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d80109460 0x7f9d80109840 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9d74009990 tx=0x7f9d74009ca0 comp rx=0 tx=0).stop 2024-05-03T18:36:48.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.524+0000 7f9d85e00700 1 -- 172.21.15.33:0/135523410 shutdown_connections 2024-05-03T18:36:48.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.524+0000 7f9d85e00700 1 --2- 172.21.15.33:0/135523410 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d80103d90 0x7f9d801102a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:48.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.524+0000 7f9d85e00700 1 --2- 172.21.15.33:0/135523410 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d80109460 0x7f9d80109840 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:48.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.524+0000 7f9d85e00700 1 --2- 172.21.15.33:0/135523410 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d801033d0 0x7f9d80103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:48.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.524+0000 7f9d85e00700 1 -- 172.21.15.33:0/135523410 >> 172.21.15.33:0/135523410 conn(0x7f9d800fef20 msgr2=0x7f9d80101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:36:48.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.524+0000 7f9d85e00700 1 -- 172.21.15.33:0/135523410 shutdown_connections 2024-05-03T18:36:48.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.524+0000 7f9d85e00700 1 -- 172.21.15.33:0/135523410 wait complete. 2024-05-03T18:36:48.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.527+0000 7f9d85e00700 1 Processor -- start 2024-05-03T18:36:48.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.527+0000 7f9d85e00700 1 -- start start 2024-05-03T18:36:48.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.528+0000 7f9d85e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d801033d0 0x7f9d8010fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:48.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.528+0000 7f9d85e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d80103d90 0x7f9d8010df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:48.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.528+0000 7f9d85e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d80109460 0x7f9d8010e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:48.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.528+0000 7f9d85e00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9d801112f0 con 0x7f9d80109460 2024-05-03T18:36:48.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.528+0000 7f9d85e00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9d80111170 con 0x7f9d801033d0 2024-05-03T18:36:48.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.528+0000 7f9d85e00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9d80111470 con 0x7f9d80103d90 2024-05-03T18:36:48.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.528+0000 7f9d84a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d801033d0 0x7f9d8010fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:48.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.529+0000 7f9d7fe00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d80103d90 0x7f9d8010df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:48.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.529+0000 7f9d84a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d801033d0 0x7f9d8010fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:48984/0 (socket says 172.21.15.33:48984) 2024-05-03T18:36:48.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.529+0000 7f9d84a00700 1 -- 172.21.15.33:0/214733292 learned_addr learned my addr 172.21.15.33:0/214733292 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:36:48.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.530+0000 7f9d84a00700 1 -- 172.21.15.33:0/214733292 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d80103d90 msgr2=0x7f9d8010df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:36:48.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.530+0000 7f9d84a00700 1 --2- 172.21.15.33:0/214733292 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d80103d90 0x7f9d8010df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:48.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.530+0000 7f9d84a00700 1 -- 172.21.15.33:0/214733292 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d80109460 msgr2=0x7f9d8010e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:36:48.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.530+0000 7f9d84a00700 1 --2- 172.21.15.33:0/214733292 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d80109460 0x7f9d8010e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:48.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.530+0000 7f9d84a00700 1 -- 172.21.15.33:0/214733292 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d8010eae0 con 0x7f9d801033d0 2024-05-03T18:36:48.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.531+0000 7f9d84a00700 1 --2- 172.21.15.33:0/214733292 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d801033d0 0x7f9d8010fdf0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9d740092e0 tx=0x7f9d74009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:36:48.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.532+0000 7f9d7d600700 1 -- 172.21.15.33:0/214733292 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d74004830 con 0x7f9d801033d0 2024-05-03T18:36:48.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.532+0000 7f9d7d600700 1 -- 172.21.15.33:0/214733292 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9d74005020 con 0x7f9d801033d0 2024-05-03T18:36:48.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.532+0000 7f9d7d600700 1 -- 172.21.15.33:0/214733292 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d74003cc0 con 0x7f9d801033d0 2024-05-03T18:36:48.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.533+0000 7f9d85e00700 1 -- 172.21.15.33:0/214733292 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d801ac830 con 0x7f9d801033d0 2024-05-03T18:36:48.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.533+0000 7f9d85e00700 1 -- 172.21.15.33:0/214733292 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d801acbd0 con 0x7f9d801033d0 2024-05-03T18:36:48.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.543+0000 7f9d7d600700 1 -- 172.21.15.33:0/214733292 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f9d740054d0 con 0x7f9d801033d0 2024-05-03T18:36:48.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.543+0000 7f9d91e65b80 1 Processor -- start 2024-05-03T18:36:48.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.544+0000 7f9d91e65b80 1 -- start start 2024-05-03T18:36:48.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.544+0000 7f9d91e65b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560fa4bf9bd0 0x560fa4bf9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:48.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.544+0000 7f9d91e65b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560fa4bf3f30 0x560fa4bf43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:48.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.544+0000 7f9d91e65b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fa4bf48f0 0x560fa4c00990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:48.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.544+0000 7f9d91e65b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560fa4b46a20 con 0x560fa4bf48f0 2024-05-03T18:36:48.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.544+0000 7f9d91e65b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x560fa4ae4760 con 0x560fa4bf3f30 2024-05-03T18:36:48.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.544+0000 7f9d91e65b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x560fa4b429e0 con 0x560fa4bf9bd0 2024-05-03T18:36:48.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:36:48 smithi099 ceph-mon[31658]: pgmap v297: 129 pgs: 129 active+clean; 2.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 5.0 MiB/s wr, 513 op/s 2024-05-03T18:36:48.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.545+0000 7f9d7fe00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fa4bf48f0 0x560fa4c00990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:48.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.545+0000 7f9d7fe00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fa4bf48f0 0x560fa4c00990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35210/0 (socket says 172.21.15.33:35210) 2024-05-03T18:36:48.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.545+0000 7f9d7fe00700 1 -- 172.21.15.33:0/3065112116 learned_addr learned my addr 172.21.15.33:0/3065112116 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:36:48.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.546+0000 7f9d85400700 1 --2- 172.21.15.33:0/3065112116 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560fa4bf9bd0 0x560fa4bf9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:48.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.546+0000 7f9d7fe00700 1 -- 172.21.15.33:0/3065112116 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560fa4bf9bd0 msgr2=0x560fa4bf9fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:36:48.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.546+0000 7f9d7fe00700 1 --2- 172.21.15.33:0/3065112116 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560fa4bf9bd0 0x560fa4bf9fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:48.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.546+0000 7f9d7fe00700 1 -- 172.21.15.33:0/3065112116 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560fa4bf3f30 msgr2=0x560fa4bf43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:36:48.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.546+0000 7f9d7fe00700 1 --2- 172.21.15.33:0/3065112116 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560fa4bf3f30 0x560fa4bf43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:48.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.546+0000 7f9d7fe00700 1 -- 172.21.15.33:0/3065112116 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fa4a938d0 con 0x560fa4bf48f0 2024-05-03T18:36:48.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.546+0000 7f9d85400700 1 --2- 172.21.15.33:0/3065112116 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560fa4bf9bd0 0x560fa4bf9fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:36:48.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.547+0000 7f9d7d600700 1 --2- 172.21.15.33:0/214733292 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f9d680b2a70 0x7f9d680b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:48.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.547+0000 7f9d7fe00700 1 --2- 172.21.15.33:0/3065112116 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fa4bf48f0 0x560fa4c00990 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c00bce0 tx=0x7f9d6c00f360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be635f2de270ddc7 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:36:48.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.547+0000 7f9d7d600700 1 -- 172.21.15.33:0/214733292 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9518+0+0 (secure 0 0 0) 0x7f9d740e38f0 con 0x7f9d801033d0 2024-05-03T18:36:48.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.547+0000 7f9d72000700 1 -- 172.21.15.33:0/3065112116 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d6c015070 con 0x560fa4bf48f0 2024-05-03T18:36:48.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.547+0000 7f9d72000700 1 -- 172.21.15.33:0/3065112116 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9d6c00fc90 con 0x560fa4bf48f0 2024-05-03T18:36:48.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.547+0000 7f9d72000700 1 -- 172.21.15.33:0/3065112116 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d6c010630 con 0x560fa4bf48f0 2024-05-03T18:36:48.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.548+0000 7f9d85400700 1 --2- 172.21.15.33:0/214733292 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f9d680b2a70 0x7f9d680b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:48.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.548+0000 7f9d91e65b80 1 -- 172.21.15.33:0/3065112116 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fa4bf48f0 msgr2=0x560fa4c00990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:36:48.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.549+0000 7f9d91e65b80 1 --2- 172.21.15.33:0/3065112116 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fa4bf48f0 0x560fa4c00990 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c00bce0 tx=0x7f9d6c00f360 comp rx=0 tx=0).stop 2024-05-03T18:36:48.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.550+0000 7f9d91e65b80 1 -- 172.21.15.33:0/3065112116 shutdown_connections 2024-05-03T18:36:48.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.550+0000 7f9d91e65b80 1 --2- 172.21.15.33:0/3065112116 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fa4bf48f0 0x560fa4c00990 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:48.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.550+0000 7f9d91e65b80 1 --2- 172.21.15.33:0/3065112116 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560fa4bf9bd0 0x560fa4bf9fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:48.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.550+0000 7f9d91e65b80 1 --2- 172.21.15.33:0/3065112116 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560fa4bf3f30 0x560fa4bf43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:48.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.550+0000 7f9d91e65b80 1 -- 172.21.15.33:0/3065112116 >> 172.21.15.33:0/3065112116 conn(0x560fa4b6c6d0 msgr2=0x560fa4beebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:36:48.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.550+0000 7f9d85400700 1 --2- 172.21.15.33:0/214733292 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f9d680b2a70 0x7f9d680b4f30 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9d78009ec0 tx=0x7f9d78009450 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:36:48.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.551+0000 7f9d91e65b80 1 -- 172.21.15.33:0/3065112116 shutdown_connections 2024-05-03T18:36:48.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.551+0000 7f9d91e65b80 1 -- 172.21.15.33:0/3065112116 wait complete. 2024-05-03T18:36:48.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.552+0000 7f9d91e65b80 1 -- start start 2024-05-03T18:36:48.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.552+0000 7f9d91e65b80 1 Processor -- start 2024-05-03T18:36:48.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.553+0000 7f9d91e65b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fa4bf3f30 0x560fa4c00660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:48.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.554+0000 7f9d91e65b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560fa4bf48f0 0x560fa4bfda50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:48.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.554+0000 7f9d91e65b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560fa4bf9bd0 0x560fa4bfdf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:48.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.554+0000 7f9d84a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fa4bf3f30 0x560fa4c00660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:48.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.554+0000 7f9d91e65b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560fa4c00f30 con 0x560fa4bf3f30 2024-05-03T18:36:48.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.554+0000 7f9d91e65b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x560fa4aacb20 con 0x560fa4bf9bd0 2024-05-03T18:36:48.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.554+0000 7f9d91e65b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x560fa4c010b0 con 0x560fa4bf48f0 2024-05-03T18:36:48.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.555+0000 7f9d91e65b80 10 client.0 _ll_register_callbacks cb 0x560fa4aedc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:36:48.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.555+0000 7f9d7fe00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560fa4bf48f0 0x560fa4bfda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:48.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.555+0000 7f9d85400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560fa4bf9bd0 0x560fa4bfdf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:48.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.555+0000 7f9d85400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560fa4bf9bd0 0x560fa4bfdf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:49002/0 (socket says 172.21.15.33:49002) 2024-05-03T18:36:48.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.555+0000 7f9d85400700 1 -- 172.21.15.33:0/3666321728 learned_addr learned my addr 172.21.15.33:0/3666321728 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:36:48.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.556+0000 7f9d85400700 1 -- 172.21.15.33:0/3666321728 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560fa4bf48f0 msgr2=0x560fa4bfda50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:36:48.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.556+0000 7f9d85400700 1 --2- 172.21.15.33:0/3666321728 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560fa4bf48f0 0x560fa4bfda50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:48.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.556+0000 7f9d85400700 1 -- 172.21.15.33:0/3666321728 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fa4bf3f30 msgr2=0x560fa4c00660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:36:48.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.556+0000 7f9d85400700 1 --2- 172.21.15.33:0/3666321728 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fa4bf3f30 0x560fa4c00660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:48.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.556+0000 7f9d85400700 1 -- 172.21.15.33:0/3666321728 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560fa4b0c420 con 0x560fa4bf9bd0 2024-05-03T18:36:48.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.557+0000 7f9d85400700 1 --2- 172.21.15.33:0/3666321728 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560fa4bf9bd0 0x560fa4bfdf90 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9d780097d0 tx=0x7f9d78004c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=adda9fe021049645 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:36:48.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.557+0000 7f9d66a00700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-03T18:36:48.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.557+0000 7f9d66a00700 1 -- 172.21.15.33:0/3666321728 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d78027060 con 0x560fa4bf9bd0 2024-05-03T18:36:48.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.557+0000 7f9d66a00700 1 -- 172.21.15.33:0/3666321728 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9518+0+0 (secure 0 0 0) 0x7f9d78012030 con 0x560fa4bf9bd0 2024-05-03T18:36:48.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.558+0000 7f9d91e65b80 1 -- 172.21.15.33:0/3666321728 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d6c00fc90 con 0x560fa4bf9bd0 2024-05-03T18:36:48.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.558+0000 7f9d66a00700 1 -- 172.21.15.33:0/3666321728 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f9d78022420 con 0x560fa4bf9bd0 2024-05-03T18:36:48.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.559+0000 7f9d91e65b80 10 client.36008 fetch_fsmap learned FSMap version 21 2024-05-03T18:36:48.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.559+0000 7f9d91e65b80 1 -- 172.21.15.33:0/3666321728 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x560fa4c0b8a0 con 0x560fa4bf9bd0 2024-05-03T18:36:48.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.561+0000 7f9d66a00700 1 -- 172.21.15.33:0/3666321728 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9d6c00fc90 con 0x560fa4bf9bd0 2024-05-03T18:36:48.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.562+0000 7f9d66a00700 1 -- 172.21.15.33:0/3666321728 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f9d7802b530 con 0x560fa4bf9bd0 2024-05-03T18:36:48.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.562+0000 7f9d91e65b80 10 client.36008 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:36:48.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.562+0000 7f9d91e65b80 10 client.36008 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:36:48.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.562+0000 7f9d91e65b80 20 client.36008 populate_metadata read hostname 'smithi033' 2024-05-03T18:36:48.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.562+0000 7f9d91e65b80 1 --2- 172.21.15.33:0/3666321728 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x560fa4c0de30 0x560fa4c10220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:48.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.562+0000 7f9d91e65b80 4 client.36008 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:36:48.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.562+0000 7f9d91e65b80 1 -- 172.21.15.33:0/3666321728 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560fa4bc0ff0 con 0x560fa4c0de30 2024-05-03T18:36:48.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.566+0000 7f9d84a00700 1 --2- 172.21.15.33:0/3666321728 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x560fa4c0de30 0x560fa4c10220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:48.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.568+0000 7f9d66a00700 10 client.36008 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:36:48.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.568+0000 7f9d84a00700 1 --2- 172.21.15.33:0/3666321728 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x560fa4c0de30 0x560fa4c10220 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=836c9d2691d586d server_cookie=be12aa3a7e83f491 in_seq=0 out_seq=0 2024-05-03T18:36:48.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.573+0000 7f9d66a00700 1 -- 172.21.15.33:0/3666321728 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560fa4bc0ff0 con 0x560fa4c0de30 2024-05-03T18:36:48.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.573+0000 7f9d66a00700 10 client.36008 handle_command_reply: tid=0 2024-05-03T18:36:48.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.573+0000 7f9d91e65b80 1 client.36008 shutdown 2024-05-03T18:36:48.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.574+0000 7f9d91e65b80 1 -- 172.21.15.33:0/3666321728 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560fa4bf9bd0 msgr2=0x560fa4bfdf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:36:48.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.574+0000 7f9d91e65b80 1 --2- 172.21.15.33:0/3666321728 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560fa4bf9bd0 0x560fa4bfdf90 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9d780097d0 tx=0x7f9d78004c50 comp rx=0 tx=0).stop 2024-05-03T18:36:48.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.574+0000 7f9d91e65b80 1 -- 172.21.15.33:0/3666321728 shutdown_connections 2024-05-03T18:36:48.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.574+0000 7f9d91e65b80 1 --2- 172.21.15.33:0/3666321728 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x560fa4c0de30 0x560fa4c10220 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:48.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.574+0000 7f9d91e65b80 1 --2- 172.21.15.33:0/3666321728 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560fa4bf9bd0 0x560fa4bfdf90 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:48.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.574+0000 7f9d91e65b80 1 --2- 172.21.15.33:0/3666321728 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fa4bf3f30 0x560fa4c00660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:48.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.574+0000 7f9d91e65b80 1 --2- 172.21.15.33:0/3666321728 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560fa4bf48f0 0x560fa4bfda50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:48.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.574+0000 7f9d91e65b80 1 -- 172.21.15.33:0/3666321728 >> 172.21.15.33:0/3666321728 conn(0x560fa4b6c6d0 msgr2=0x560fa4bf2f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:36:48.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.574+0000 7f9d66a00700 0 client.36008 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:36:48.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.575+0000 7f9d91e65b80 1 -- 172.21.15.33:0/3666321728 shutdown_connections 2024-05-03T18:36:48.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.576+0000 7f9d91e65b80 1 -- 172.21.15.33:0/3666321728 wait complete. 2024-05-03T18:36:48.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.576+0000 7f9d91e65b80 20 client.36008 trim_cache size 0 max 16384 2024-05-03T18:36:48.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.617+0000 7f9d91e65b80 1 Processor -- start 2024-05-03T18:36:48.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.618+0000 7f9d91e65b80 1 -- start start 2024-05-03T18:36:48.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.619+0000 7f9d91e65b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fa4bf3f30 0x560fa4beed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:48.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.619+0000 7f9d91e65b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560fa4bf48f0 0x560fa4c09fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:48.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.619+0000 7f9d91e65b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560fa4bf9bd0 0x560fa4c15b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:48.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.619+0000 7f9d91e65b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9d801b39f0 con 0x560fa4bf3f30 2024-05-03T18:36:48.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.619+0000 7f9d91e65b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9d801b6250 con 0x560fa4bf9bd0 2024-05-03T18:36:48.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.619+0000 7f9d91e65b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9d801b4040 con 0x560fa4bf48f0 2024-05-03T18:36:48.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.620+0000 7f9d7fe00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560fa4bf48f0 0x560fa4c09fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:48.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.620+0000 7f9d7fe00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560fa4bf48f0 0x560fa4c09fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:42854/0 (socket says 172.21.15.33:42854) 2024-05-03T18:36:48.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.620+0000 7f9d7fe00700 1 -- 172.21.15.33:0/573064009 learned_addr learned my addr 172.21.15.33:0/573064009 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:36:48.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.620+0000 7f9d85400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560fa4bf9bd0 0x560fa4c15b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:48.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.621+0000 7f9d7fe00700 1 -- 172.21.15.33:0/573064009 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560fa4bf9bd0 msgr2=0x560fa4c15b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:36:48.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.621+0000 7f9d7fe00700 1 --2- 172.21.15.33:0/573064009 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560fa4bf9bd0 0x560fa4c15b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:48.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.621+0000 7f9d7fe00700 1 -- 172.21.15.33:0/573064009 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fa4bf3f30 msgr2=0x560fa4beed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:36:48.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.621+0000 7f9d7fe00700 1 --2- 172.21.15.33:0/573064009 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fa4bf3f30 0x560fa4beed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:48.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.621+0000 7f9d7fe00700 1 -- 172.21.15.33:0/573064009 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fa4bc0fd0 con 0x560fa4bf48f0 2024-05-03T18:36:48.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.621+0000 7f9d7fe00700 1 --2- 172.21.15.33:0/573064009 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560fa4bf48f0 0x560fa4c09fb0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x560fa4befa90 tx=0x7f9d6c00fcd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc4692fa099f3894 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:36:48.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.623+0000 7f9d71600700 1 -- 172.21.15.33:0/573064009 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d6c015040 con 0x560fa4bf48f0 2024-05-03T18:36:48.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.623+0000 7f9d71600700 1 -- 172.21.15.33:0/573064009 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9d6c004030 con 0x560fa4bf48f0 2024-05-03T18:36:48.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.623+0000 7f9d71600700 1 -- 172.21.15.33:0/573064009 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d6c004cd0 con 0x560fa4bf48f0 2024-05-03T18:36:48.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.624+0000 7f9d91e65b80 1 -- 172.21.15.33:0/573064009 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560fa4bf48f0 msgr2=0x560fa4c09fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:36:48.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.624+0000 7f9d91e65b80 1 --2- 172.21.15.33:0/573064009 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560fa4bf48f0 0x560fa4c09fb0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x560fa4befa90 tx=0x7f9d6c00fcd0 comp rx=0 tx=0).stop 2024-05-03T18:36:48.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.624+0000 7f9d91e65b80 1 -- 172.21.15.33:0/573064009 shutdown_connections 2024-05-03T18:36:48.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.624+0000 7f9d91e65b80 1 --2- 172.21.15.33:0/573064009 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560fa4bf9bd0 0x560fa4c15b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:48.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.624+0000 7f9d91e65b80 1 --2- 172.21.15.33:0/573064009 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fa4bf3f30 0x560fa4beed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:48.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.624+0000 7f9d91e65b80 1 --2- 172.21.15.33:0/573064009 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560fa4bf48f0 0x560fa4c09fb0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:48.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.624+0000 7f9d91e65b80 1 -- 172.21.15.33:0/573064009 >> 172.21.15.33:0/573064009 conn(0x560fa4b6c6d0 msgr2=0x560fa4bfda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:36:48.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.625+0000 7f9d91e65b80 1 -- 172.21.15.33:0/573064009 shutdown_connections 2024-05-03T18:36:48.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.625+0000 7f9d91e65b80 1 -- 172.21.15.33:0/573064009 wait complete. 2024-05-03T18:36:48.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.626+0000 7f9d91e65b80 1 -- start start 2024-05-03T18:36:48.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.626+0000 7f9d91e65b80 1 Processor -- start 2024-05-03T18:36:48.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.628+0000 7f9d91e65b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560fa4bf3f30 0x560fa4c06510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:48.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.628+0000 7f9d91e65b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560fa4bf48f0 0x560fa4c06a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:48.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.628+0000 7f9d91e65b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fa4bf9bd0 0x560fa4c03500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:48.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.629+0000 7f9d7fe00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560fa4bf48f0 0x560fa4c06a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:48.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.629+0000 7f9d7fe00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560fa4bf48f0 0x560fa4c06a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:42864/0 (socket says 172.21.15.33:42864) 2024-05-03T18:36:48.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.629+0000 7f9d7fe00700 1 -- 172.21.15.33:0/3966754223 learned_addr learned my addr 172.21.15.33:0/3966754223 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:36:48.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.629+0000 7f9d91e65b80 1 -- 172.21.15.33:0/3966754223 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9d801b4710 con 0x560fa4bf9bd0 2024-05-03T18:36:48.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.629+0000 7f9d85400700 1 --2- 172.21.15.33:0/3966754223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fa4bf9bd0 0x560fa4c03500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:48.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.629+0000 7f9d91e65b80 1 -- 172.21.15.33:0/3966754223 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9d801b5b80 con 0x560fa4bf3f30 2024-05-03T18:36:48.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.629+0000 7f9d91e65b80 1 -- 172.21.15.33:0/3966754223 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9d801b4de0 con 0x560fa4bf48f0 2024-05-03T18:36:48.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.629+0000 7f9d91e65b80 10 client.0 _ll_register_callbacks cb 0x560fa4aedc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:36:48.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.629+0000 7f9d84a00700 1 --2- 172.21.15.33:0/3966754223 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560fa4bf3f30 0x560fa4c06510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:48.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.630+0000 7f9d7fe00700 1 -- 172.21.15.33:0/3966754223 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560fa4bf3f30 msgr2=0x560fa4c06510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:36:48.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.630+0000 7f9d7fe00700 1 --2- 172.21.15.33:0/3966754223 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560fa4bf3f30 0x560fa4c06510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:48.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.630+0000 7f9d7fe00700 1 -- 172.21.15.33:0/3966754223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fa4bf9bd0 msgr2=0x560fa4c03500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:36:48.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.630+0000 7f9d7fe00700 1 --2- 172.21.15.33:0/3966754223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fa4bf9bd0 0x560fa4c03500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:48.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.630+0000 7f9d7fe00700 1 -- 172.21.15.33:0/3966754223 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560fa4c03d40 con 0x560fa4bf48f0 2024-05-03T18:36:48.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.630+0000 7f9d85400700 1 --2- 172.21.15.33:0/3966754223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fa4bf9bd0 0x560fa4c03500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:36:48.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.631+0000 7f9d66a00700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-03T18:36:48.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.631+0000 7f9d7fe00700 1 --2- 172.21.15.33:0/3966754223 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560fa4bf48f0 0x560fa4c06a50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c00b910 tx=0x7f9d6c00fa90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=537ca7b26bda2d55 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:36:48.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.631+0000 7f9d66a00700 1 -- 172.21.15.33:0/3966754223 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d6c015040 con 0x560fa4bf48f0 2024-05-03T18:36:48.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.631+0000 7f9d91e65b80 1 -- 172.21.15.33:0/3966754223 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d6c004030 con 0x560fa4bf48f0 2024-05-03T18:36:48.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.632+0000 7f9d66a00700 1 -- 172.21.15.33:0/3966754223 <== mon.1 v2:172.21.15.150:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9518+0+0 (secure 0 0 0) 0x7f9d6c029070 con 0x560fa4bf48f0 2024-05-03T18:36:48.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.632+0000 7f9d66a00700 1 -- 172.21.15.33:0/3966754223 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f9d6c015040 con 0x560fa4bf48f0 2024-05-03T18:36:48.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.632+0000 7f9d91e65b80 10 client.26071 fetch_fsmap learned FSMap version 21 2024-05-03T18:36:48.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.632+0000 7f9d91e65b80 1 -- 172.21.15.33:0/3966754223 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x560fa4c79320 con 0x560fa4bf48f0 2024-05-03T18:36:48.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.633+0000 7f9d66a00700 1 -- 172.21.15.33:0/3966754223 <== mon.1 v2:172.21.15.150:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9d6c004030 con 0x560fa4bf48f0 2024-05-03T18:36:48.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.633+0000 7f9d66a00700 1 -- 172.21.15.33:0/3966754223 <== mon.1 v2:172.21.15.150:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f9d6c003680 con 0x560fa4bf48f0 2024-05-03T18:36:48.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.634+0000 7f9d91e65b80 10 client.26071 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:36:48.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.634+0000 7f9d91e65b80 10 client.26071 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:36:48.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.634+0000 7f9d91e65b80 20 client.26071 populate_metadata read hostname 'smithi033' 2024-05-03T18:36:48.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.634+0000 7f9d91e65b80 1 --2- 172.21.15.33:0/3966754223 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x560fa4c79730 0x560fa4c7bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:48.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.634+0000 7f9d91e65b80 4 client.26071 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-03T18:36:48.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.634+0000 7f9d91e65b80 1 -- 172.21.15.33:0/3966754223 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560fa4b54400 con 0x560fa4c79730 2024-05-03T18:36:48.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.636+0000 7f9d84a00700 1 --2- 172.21.15.33:0/3966754223 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x560fa4c79730 0x560fa4c7bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:48.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.639+0000 7f9d66a00700 10 client.26071 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:36:48.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.639+0000 7f9d84a00700 1 --2- 172.21.15.33:0/3966754223 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x560fa4c79730 0x560fa4c7bb50 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2369c798ddb76334 server_cookie=f2675f4c77a97c65 in_seq=0 out_seq=0 2024-05-03T18:36:48.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.640+0000 7f9d66a00700 1 -- 172.21.15.33:0/3966754223 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x560fa4b54400 con 0x560fa4c79730 2024-05-03T18:36:48.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.640+0000 7f9d66a00700 10 client.26071 handle_command_reply: tid=0 2024-05-03T18:36:48.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.640+0000 7f9d91e65b80 1 client.26071 shutdown 2024-05-03T18:36:48.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.640+0000 7f9d91e65b80 1 -- 172.21.15.33:0/3966754223 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560fa4bf48f0 msgr2=0x560fa4c06a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:36:48.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.640+0000 7f9d91e65b80 1 --2- 172.21.15.33:0/3966754223 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560fa4bf48f0 0x560fa4c06a50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c00b910 tx=0x7f9d6c00fa90 comp rx=0 tx=0).stop 2024-05-03T18:36:48.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.640+0000 7f9d91e65b80 1 -- 172.21.15.33:0/3966754223 shutdown_connections 2024-05-03T18:36:48.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.640+0000 7f9d91e65b80 1 --2- 172.21.15.33:0/3966754223 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x560fa4c79730 0x560fa4c7bb50 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:48.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.640+0000 7f9d91e65b80 1 --2- 172.21.15.33:0/3966754223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fa4bf9bd0 0x560fa4c03500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:48.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.640+0000 7f9d66a00700 0 client.26071 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:36:48.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.640+0000 7f9d91e65b80 1 --2- 172.21.15.33:0/3966754223 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560fa4bf3f30 0x560fa4c06510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:48.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.640+0000 7f9d91e65b80 1 --2- 172.21.15.33:0/3966754223 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560fa4bf48f0 0x560fa4c06a50 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:48.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.640+0000 7f9d91e65b80 1 -- 172.21.15.33:0/3966754223 >> 172.21.15.33:0/3966754223 conn(0x560fa4b6c6d0 msgr2=0x560fa4bf2f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:36:48.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.641+0000 7f9d91e65b80 1 -- 172.21.15.33:0/3966754223 shutdown_connections 2024-05-03T18:36:48.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.641+0000 7f9d91e65b80 1 -- 172.21.15.33:0/3966754223 wait complete. 2024-05-03T18:36:48.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.641+0000 7f9d91e65b80 20 client.26071 trim_cache size 0 max 16384 2024-05-03T18:36:48.642 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-03T18:36:48.642 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-05-03T18:36:48.642 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-05-03T18:36:48.642 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-03T18:36:48.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.642+0000 7f9d85e00700 1 -- 172.21.15.33:0/214733292 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f9d680b2a70 msgr2=0x7f9d680b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:36:48.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.642+0000 7f9d85e00700 1 --2- 172.21.15.33:0/214733292 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f9d680b2a70 0x7f9d680b4f30 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9d78009ec0 tx=0x7f9d78009450 comp rx=0 tx=0).stop 2024-05-03T18:36:48.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.642+0000 7f9d85e00700 1 -- 172.21.15.33:0/214733292 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d801033d0 msgr2=0x7f9d8010fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:36:48.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.642+0000 7f9d85e00700 1 --2- 172.21.15.33:0/214733292 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d801033d0 0x7f9d8010fdf0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9d740092e0 tx=0x7f9d74009f90 comp rx=0 tx=0).stop 2024-05-03T18:36:48.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.642+0000 7f9d85e00700 1 -- 172.21.15.33:0/214733292 shutdown_connections 2024-05-03T18:36:48.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.642+0000 7f9d85e00700 1 --2- 172.21.15.33:0/214733292 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d80109460 0x7f9d8010e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:48.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.642+0000 7f9d85e00700 1 --2- 172.21.15.33:0/214733292 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f9d680b2a70 0x7f9d680b4f30 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:48.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.642+0000 7f9d85e00700 1 --2- 172.21.15.33:0/214733292 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d801033d0 0x7f9d8010fdf0 secure :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9d740092e0 tx=0x7f9d74009f90 comp rx=0 tx=0).stop 2024-05-03T18:36:48.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.642+0000 7f9d85e00700 1 --2- 172.21.15.33:0/214733292 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d80103d90 0x7f9d8010df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:48.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.642+0000 7f9d85e00700 1 -- 172.21.15.33:0/214733292 >> 172.21.15.33:0/214733292 conn(0x7f9d800fef20 msgr2=0x7f9d800ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:36:48.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.643+0000 7f9d85e00700 1 -- 172.21.15.33:0/214733292 shutdown_connections 2024-05-03T18:36:48.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:48.643+0000 7f9d85e00700 1 -- 172.21.15.33:0/214733292 wait complete. 2024-05-03T18:36:49.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:36:49 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:36:49.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:36:49 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:36:49.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:36:49 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:36:50.110 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 335.0 s, 591.6 tps, lat 168.194 ms stddev 96.261 2024-05-03T18:36:50.523 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:36:50 smithi033 ceph-mon[28330]: pgmap v298: 129 pgs: 129 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 7.2 MiB/s wr, 753 op/s 2024-05-03T18:36:50.529 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-03T18:36:50.529 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T18:36:50.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:36:50 smithi099 ceph-mon[31658]: pgmap v298: 129 pgs: 129 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 7.2 MiB/s wr, 753 op/s 2024-05-03T18:36:50.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:36:50 smithi150 ceph-mon[32492]: pgmap v298: 129 pgs: 129 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 7.2 MiB/s wr, 753 op/s 2024-05-03T18:36:51.627 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:36:52.362 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:36:52 smithi033 ceph-mon[28330]: pgmap v299: 129 pgs: 129 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 4.5 MiB/s wr, 466 op/s 2024-05-03T18:36:52.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:36:52 smithi150 ceph-mon[32492]: pgmap v299: 129 pgs: 129 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 4.5 MiB/s wr, 466 op/s 2024-05-03T18:36:52.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:36:52 smithi099 ceph-mon[31658]: pgmap v299: 129 pgs: 129 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 4.5 MiB/s wr, 466 op/s 2024-05-03T18:36:53.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:36:53 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:36:53.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:36:53 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:36:53.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:36:53 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:36:53.460 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:36:53 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:36:53.584 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:36:53 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:36:53.584 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:36:53 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:36:53.584 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:36:53 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:36:53.585 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:36:53 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:36:53.585 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:36:53 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:36:54.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:36:54 smithi150 ceph-mon[32492]: pgmap v300: 129 pgs: 129 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 6.6 MiB/s wr, 685 op/s 2024-05-03T18:36:54.394 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:36:54 smithi033 ceph-mon[28330]: pgmap v300: 129 pgs: 129 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 6.6 MiB/s wr, 685 op/s 2024-05-03T18:36:54.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:36:54 smithi099 ceph-mon[31658]: pgmap v300: 129 pgs: 129 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 6.6 MiB/s wr, 685 op/s 2024-05-03T18:36:55.110 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 340.0 s, 541.6 tps, lat 185.276 ms stddev 77.925 2024-05-03T18:36:56.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:36:56 smithi150 ceph-mon[32492]: pgmap v301: 129 pgs: 129 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 4.3 MiB/s wr, 459 op/s 2024-05-03T18:36:56.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:36:56 smithi033 ceph-mon[28330]: pgmap v301: 129 pgs: 129 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 4.3 MiB/s wr, 459 op/s 2024-05-03T18:36:56.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:56.508+0000 7fa369200700 1 -- 172.21.15.33:0/1989801261 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3641083d0 msgr2=0x7fa36406b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:36:56.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:56.508+0000 7fa369200700 1 --2- 172.21.15.33:0/1989801261 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3641083d0 0x7fa36406b1b0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fa354009990 tx=0x7fa354009ca0 comp rx=0 tx=0).stop 2024-05-03T18:36:56.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:56.509+0000 7fa369200700 1 -- 172.21.15.33:0/1989801261 shutdown_connections 2024-05-03T18:36:56.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:56.509+0000 7fa369200700 1 --2- 172.21.15.33:0/1989801261 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa364105ee0 0x7fa364106370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:56.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:56.509+0000 7fa369200700 1 --2- 172.21.15.33:0/1989801261 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3641083d0 0x7fa36406b1b0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:56.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:56.509+0000 7fa369200700 1 --2- 172.21.15.33:0/1989801261 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa36406b6f0 0x7fa36406bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:56.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:56.509+0000 7fa369200700 1 -- 172.21.15.33:0/1989801261 >> 172.21.15.33:0/1989801261 conn(0x7fa364079350 msgr2=0x7fa364079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:36:56.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:56.510+0000 7fa369200700 1 -- 172.21.15.33:0/1989801261 shutdown_connections 2024-05-03T18:36:56.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:56.510+0000 7fa369200700 1 -- 172.21.15.33:0/1989801261 wait complete. 2024-05-03T18:36:56.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:56.512+0000 7fa369200700 1 Processor -- start 2024-05-03T18:36:56.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:56.514+0000 7fa369200700 1 -- start start 2024-05-03T18:36:56.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:56.515+0000 7fa369200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa36406b6f0 0x7fa364074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:56.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:56.515+0000 7fa369200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa364105ee0 0x7fa36406dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:56.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:56.515+0000 7fa369200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3641083d0 0x7fa36406e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:56.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:56.515+0000 7fa369200700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa364119ac0 con 0x7fa364105ee0 2024-05-03T18:36:56.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:56.515+0000 7fa369200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa364119940 con 0x7fa3641083d0 2024-05-03T18:36:56.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:56.515+0000 7fa369200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa364119c40 con 0x7fa36406b6f0 2024-05-03T18:36:56.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:56.516+0000 7fa361400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa364105ee0 0x7fa36406dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:56.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:56.516+0000 7fa361400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa364105ee0 0x7fa36406dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45862/0 (socket says 172.21.15.33:45862) 2024-05-03T18:36:56.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:56.516+0000 7fa361400700 1 -- 172.21.15.33:0/2064110385 learned_addr learned my addr 172.21.15.33:0/2064110385 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:36:56.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:56.516+0000 7fa362800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3641083d0 0x7fa36406e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:56.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:56.516+0000 7fa361e00700 1 --2- 172.21.15.33:0/2064110385 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa36406b6f0 0x7fa364074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:56.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:56.517+0000 7fa362800700 1 -- 172.21.15.33:0/2064110385 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa36406b6f0 msgr2=0x7fa364074d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:36:56.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:56.517+0000 7fa362800700 1 --2- 172.21.15.33:0/2064110385 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa36406b6f0 0x7fa364074d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:56.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:56.517+0000 7fa362800700 1 -- 172.21.15.33:0/2064110385 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa364105ee0 msgr2=0x7fa36406dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:36:56.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:56.517+0000 7fa362800700 1 --2- 172.21.15.33:0/2064110385 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa364105ee0 0x7fa36406dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:56.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:56.517+0000 7fa362800700 1 -- 172.21.15.33:0/2064110385 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa36406e940 con 0x7fa3641083d0 2024-05-03T18:36:56.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:56.520+0000 7fa362800700 1 --2- 172.21.15.33:0/2064110385 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3641083d0 0x7fa36406e2d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa35c007f10 tx=0x7fa35c00d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:36:56.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:56.520+0000 7fa35aa00700 1 -- 172.21.15.33:0/2064110385 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa35c004db0 con 0x7fa3641083d0 2024-05-03T18:36:56.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:56.520+0000 7fa35aa00700 1 -- 172.21.15.33:0/2064110385 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa35c009e20 con 0x7fa3641083d0 2024-05-03T18:36:56.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:56.520+0000 7fa35aa00700 1 -- 172.21.15.33:0/2064110385 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa35c015400 con 0x7fa3641083d0 2024-05-03T18:36:56.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:56.521+0000 7fa369200700 1 -- 172.21.15.33:0/2064110385 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa364072830 con 0x7fa3641083d0 2024-05-03T18:36:56.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:56.521+0000 7fa369200700 1 -- 172.21.15.33:0/2064110385 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa364072d20 con 0x7fa3641083d0 2024-05-03T18:36:56.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:56.524+0000 7fa369200700 1 -- 172.21.15.33:0/2064110385 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa36406ebd0 con 0x7fa3641083d0 2024-05-03T18:36:56.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:56.527+0000 7fa35aa00700 1 -- 172.21.15.33:0/2064110385 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fa35c0155a0 con 0x7fa3641083d0 2024-05-03T18:36:56.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:56.530+0000 7fa35aa00700 1 --2- 172.21.15.33:0/2064110385 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fa3480b2af0 0x7fa3480b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:56.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:56.530+0000 7fa35aa00700 1 -- 172.21.15.33:0/2064110385 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9518+0+0 (secure 0 0 0) 0x7fa35c0e22f0 con 0x7fa3641083d0 2024-05-03T18:36:56.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:56.531+0000 7fa361e00700 1 --2- 172.21.15.33:0/2064110385 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fa3480b2af0 0x7fa3480b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:56.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:56.532+0000 7fa361e00700 1 --2- 172.21.15.33:0/2064110385 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fa3480b2af0 0x7fa3480b4fb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa35400b5c0 tx=0x7fa354005960 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:36:56.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:56.542+0000 7fa35aa00700 1 -- 172.21.15.33:0/2064110385 <== mon.2 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fa35c010070 con 0x7fa3641083d0 2024-05-03T18:36:56.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:36:56 smithi099 ceph-mon[31658]: pgmap v301: 129 pgs: 129 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 4.3 MiB/s wr, 459 op/s 2024-05-03T18:36:57.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:57.034+0000 7fa369200700 1 -- 172.21.15.33:0/2064110385 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa364072f80 con 0x7fa3641083d0 2024-05-03T18:36:57.038 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:36:57.039 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1327393503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1327393503},{"type":"v1","addr":"172.21.15.33:6835","nonce":1327393503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/2461472948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":2461472948},{"type":"v1","addr":"172.21.15.33:6837","nonce":2461472948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14559,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/442269281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":442269281},{"type":"v1","addr":"172.21.15.33:6839","nonce":442269281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14571,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/645571756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":645571756},{"type":"v1","addr":"172.21.15.33:6841","nonce":645571756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/203598909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":203598909},{"type":"v1","addr":"172.21.15.150:6833","nonce":203598909}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/4029842493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":4029842493},{"type":"v1","addr":"172.21.15.150:6835","nonce":4029842493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3463664423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3463664423},{"type":"v1","addr":"172.21.15.150:6837","nonce":3463664423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24481,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6839/4051648018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":4051648018},{"type":"v1","addr":"172.21.15.150:6839","nonce":4051648018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34418,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1865105479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1865105479},{"type":"v1","addr":"172.21.15.99:6833","nonce":1865105479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34445,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/1206393914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1206393914},{"type":"v1","addr":"172.21.15.99:6835","nonce":1206393914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-03T18:22:28.837091+0000","modified":"2024-05-03T18:23:49.758398+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34508},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34472":{"gid":34472,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6837/3360970342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3360970342},{"type":"v1","addr":"172.21.15.99:6837","nonce":3360970342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34508":{"gid":34508,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/919907308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":919907308},{"type":"v1","addr":"172.21.15.99:6839","nonce":919907308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34508,"qdb_cluster":[34508]},"id":1}]} 2024-05-03T18:36:57.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:57.036+0000 7fa35aa00700 1 -- 172.21.15.33:0/2064110385 <== mon.2 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11801 (secure 0 0 0) 0x7fa35c0a9890 con 0x7fa3641083d0 2024-05-03T18:36:57.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:57.040+0000 7fa369200700 1 -- 172.21.15.33:0/2064110385 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fa3480b2af0 msgr2=0x7fa3480b4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:36:57.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:57.040+0000 7fa369200700 1 --2- 172.21.15.33:0/2064110385 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fa3480b2af0 0x7fa3480b4fb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa35400b5c0 tx=0x7fa354005960 comp rx=0 tx=0).stop 2024-05-03T18:36:57.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:57.040+0000 7fa369200700 1 -- 172.21.15.33:0/2064110385 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3641083d0 msgr2=0x7fa36406e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:36:57.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:57.040+0000 7fa369200700 1 --2- 172.21.15.33:0/2064110385 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3641083d0 0x7fa36406e2d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa35c007f10 tx=0x7fa35c00d3b0 comp rx=0 tx=0).stop 2024-05-03T18:36:57.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:57.040+0000 7fa369200700 1 -- 172.21.15.33:0/2064110385 shutdown_connections 2024-05-03T18:36:57.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:57.040+0000 7fa369200700 1 --2- 172.21.15.33:0/2064110385 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fa3480b2af0 0x7fa3480b4fb0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:57.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:57.040+0000 7fa369200700 1 --2- 172.21.15.33:0/2064110385 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3641083d0 0x7fa36406e2d0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:57.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:57.040+0000 7fa369200700 1 --2- 172.21.15.33:0/2064110385 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa36406b6f0 0x7fa364074d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:57.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:57.040+0000 7fa369200700 1 --2- 172.21.15.33:0/2064110385 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa364105ee0 0x7fa36406dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:57.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:57.040+0000 7fa369200700 1 -- 172.21.15.33:0/2064110385 >> 172.21.15.33:0/2064110385 conn(0x7fa364079350 msgr2=0x7fa3641045d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:36:57.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:57.042+0000 7fa369200700 1 -- 172.21.15.33:0/2064110385 shutdown_connections 2024-05-03T18:36:57.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:36:57.042+0000 7fa369200700 1 -- 172.21.15.33:0/2064110385 wait complete. 2024-05-03T18:36:57.047 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 21 2024-05-03T18:36:57.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:36:57 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/2064110385' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:36:57.461 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:36:57 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/2064110385' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:36:57.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:36:57 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/2064110385' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:36:58.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:36:58 smithi150 ceph-mon[32492]: pgmap v302: 129 pgs: 129 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 4.3 MiB/s wr, 459 op/s 2024-05-03T18:36:58.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:36:58 smithi033 ceph-mon[28330]: pgmap v302: 129 pgs: 129 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 4.3 MiB/s wr, 459 op/s 2024-05-03T18:36:58.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:36:58 smithi099 ceph-mon[31658]: pgmap v302: 129 pgs: 129 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 4.3 MiB/s wr, 459 op/s 2024-05-03T18:36:58.768 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-03T18:36:59.779 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:37:00.111 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 345.0 s, 563.2 tps, lat 178.154 ms stddev 80.183 2024-05-03T18:37:00.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:37:00 smithi033 ceph-mon[28330]: pgmap v303: 129 pgs: 129 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 6.7 MiB/s wr, 710 op/s 2024-05-03T18:37:00.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:37:00 smithi150 ceph-mon[32492]: pgmap v303: 129 pgs: 129 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 6.7 MiB/s wr, 710 op/s 2024-05-03T18:37:00.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:37:00 smithi099 ceph-mon[31658]: pgmap v303: 129 pgs: 129 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 6.7 MiB/s wr, 710 op/s 2024-05-03T18:37:02.384 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:37:02 smithi150 ceph-mon[32492]: pgmap v304: 129 pgs: 129 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 4.4 MiB/s wr, 470 op/s 2024-05-03T18:37:02.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:37:02 smithi033 ceph-mon[28330]: pgmap v304: 129 pgs: 129 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 4.4 MiB/s wr, 470 op/s 2024-05-03T18:37:02.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:37:02 smithi099 ceph-mon[31658]: pgmap v304: 129 pgs: 129 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 4.4 MiB/s wr, 470 op/s 2024-05-03T18:37:04.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:37:04 smithi150 ceph-mon[32492]: pgmap v305: 129 pgs: 129 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 6.7 MiB/s wr, 726 op/s 2024-05-03T18:37:04.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:37:04 smithi033 ceph-mon[28330]: pgmap v305: 129 pgs: 129 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 6.7 MiB/s wr, 726 op/s 2024-05-03T18:37:04.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:37:04 smithi099 ceph-mon[31658]: pgmap v305: 129 pgs: 129 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 6.7 MiB/s wr, 726 op/s 2024-05-03T18:37:05.111 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 350.0 s, 601.7 tps, lat 165.418 ms stddev 113.551 2024-05-03T18:37:06.342 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:37:06 smithi033 ceph-mon[28330]: pgmap v306: 129 pgs: 129 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 4.7 MiB/s wr, 507 op/s 2024-05-03T18:37:06.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:37:06 smithi150 ceph-mon[32492]: pgmap v306: 129 pgs: 129 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 4.7 MiB/s wr, 507 op/s 2024-05-03T18:37:06.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:37:06 smithi099 ceph-mon[31658]: pgmap v306: 129 pgs: 129 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 4.7 MiB/s wr, 507 op/s 2024-05-03T18:37:06.598 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-03T18:37:06.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.341+0000 7fcf9ca00700 1 -- 172.21.15.33:0/2341397364 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcf9810d9c0 msgr2=0x7fcf9810fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:06.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.341+0000 7fcf9ca00700 1 --2- 172.21.15.33:0/2341397364 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcf9810d9c0 0x7fcf9810fe80 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fcf9000b0a0 tx=0x7fcf9000b3b0 comp rx=0 tx=0).stop 2024-05-03T18:37:06.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.341+0000 7fcf9ca00700 1 -- 172.21.15.33:0/2341397364 shutdown_connections 2024-05-03T18:37:06.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.341+0000 7fcf9ca00700 1 --2- 172.21.15.33:0/2341397364 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcf9810d9c0 0x7fcf9810fe80 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:06.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.341+0000 7fcf9ca00700 1 --2- 172.21.15.33:0/2341397364 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf98104590 0x7fcf98104970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:06.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.341+0000 7fcf9ca00700 1 --2- 172.21.15.33:0/2341397364 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf98104eb0 0x7fcf9810d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:06.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.341+0000 7fcf9ca00700 1 -- 172.21.15.33:0/2341397364 >> 172.21.15.33:0/2341397364 conn(0x7fcf980fddc0 msgr2=0x7fcf981001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:37:06.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.343+0000 7fcf9ca00700 1 -- 172.21.15.33:0/2341397364 shutdown_connections 2024-05-03T18:37:06.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.344+0000 7fcf9ca00700 1 -- 172.21.15.33:0/2341397364 wait complete. 2024-05-03T18:37:06.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.346+0000 7fcf9ca00700 1 Processor -- start 2024-05-03T18:37:06.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.347+0000 7fcf9ca00700 1 -- start start 2024-05-03T18:37:06.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.348+0000 7fcf9ca00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcf98104590 0x7fcf9819f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:06.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.348+0000 7fcf9ca00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf98104eb0 0x7fcf9819fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:06.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.348+0000 7fcf9ca00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf9810d9c0 0x7fcf98199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:06.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.349+0000 7fcf96800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcf98104590 0x7fcf9819f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:06.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.349+0000 7fcf96800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcf98104590 0x7fcf9819f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:60722/0 (socket says 172.21.15.33:60722) 2024-05-03T18:37:06.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.349+0000 7fcf95e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf98104eb0 0x7fcf9819fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:06.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.349+0000 7fcf95e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf98104eb0 0x7fcf9819fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34686/0 (socket says 172.21.15.33:34686) 2024-05-03T18:37:06.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.350+0000 7fcf95e00700 1 -- 172.21.15.33:0/596445685 learned_addr learned my addr 172.21.15.33:0/596445685 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:37:06.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.350+0000 7fcf9ca00700 1 -- 172.21.15.33:0/596445685 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcf981134c0 con 0x7fcf98104eb0 2024-05-03T18:37:06.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.350+0000 7fcf9ca00700 1 -- 172.21.15.33:0/596445685 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fcf98113340 con 0x7fcf9810d9c0 2024-05-03T18:37:06.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.350+0000 7fcf9ca00700 1 -- 172.21.15.33:0/596445685 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fcf98113640 con 0x7fcf98104590 2024-05-03T18:37:06.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.350+0000 7fcf97200700 1 --2- 172.21.15.33:0/596445685 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf9810d9c0 0x7fcf98199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:06.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.351+0000 7fcf97200700 1 -- 172.21.15.33:0/596445685 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcf98104590 msgr2=0x7fcf9819f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:06.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.351+0000 7fcf97200700 1 --2- 172.21.15.33:0/596445685 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcf98104590 0x7fcf9819f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:06.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.351+0000 7fcf97200700 1 -- 172.21.15.33:0/596445685 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf98104eb0 msgr2=0x7fcf9819fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:06.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.351+0000 7fcf97200700 1 --2- 172.21.15.33:0/596445685 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf98104eb0 0x7fcf9819fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:06.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.351+0000 7fcf97200700 1 -- 172.21.15.33:0/596445685 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf98199fe0 con 0x7fcf9810d9c0 2024-05-03T18:37:06.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.353+0000 7fcf97200700 1 --2- 172.21.15.33:0/596445685 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf9810d9c0 0x7fcf98199800 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fcf9000b070 tx=0x7fcf90009c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:06.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.354+0000 7fcf8f400700 1 -- 172.21.15.33:0/596445685 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf90022070 con 0x7fcf9810d9c0 2024-05-03T18:37:06.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.354+0000 7fcf8f400700 1 -- 172.21.15.33:0/596445685 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcf90007b80 con 0x7fcf9810d9c0 2024-05-03T18:37:06.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.354+0000 7fcf9ca00700 1 -- 172.21.15.33:0/596445685 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf9819a270 con 0x7fcf9810d9c0 2024-05-03T18:37:06.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.354+0000 7fcf8f400700 1 -- 172.21.15.33:0/596445685 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf900053b0 con 0x7fcf9810d9c0 2024-05-03T18:37:06.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.354+0000 7fcf9ca00700 1 -- 172.21.15.33:0/596445685 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf98102120 con 0x7fcf9810d9c0 2024-05-03T18:37:06.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.368+0000 7fcf8f400700 1 -- 172.21.15.33:0/596445685 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fcf90005550 con 0x7fcf9810d9c0 2024-05-03T18:37:06.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.368+0000 7fcfa7e32b80 1 Processor -- start 2024-05-03T18:37:06.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.369+0000 7fcfa7e32b80 1 -- start start 2024-05-03T18:37:06.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.370+0000 7fcfa7e32b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e47c9a09a0 0x55e47c9a0d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:06.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.370+0000 7fcfa7e32b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e47c9a12c0 0x55e47c9a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:06.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.370+0000 7fcfa7e32b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e47c9a53f0 0x55e47c9a5830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:06.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.370+0000 7fcfa7e32b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e47c8eaa20 con 0x55e47c9a53f0 2024-05-03T18:37:06.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.370+0000 7fcfa7e32b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55e47c888760 con 0x55e47c9a09a0 2024-05-03T18:37:06.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.370+0000 7fcfa7e32b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55e47c8e69e0 con 0x55e47c9a12c0 2024-05-03T18:37:06.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.371+0000 7fcf96800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e47c9a12c0 0x55e47c9a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:06.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.371+0000 7fcf96800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e47c9a12c0 0x55e47c9a4e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:60736/0 (socket says 172.21.15.33:60736) 2024-05-03T18:37:06.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.371+0000 7fcf96800700 1 -- 172.21.15.33:0/2918400563 learned_addr learned my addr 172.21.15.33:0/2918400563 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:37:06.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.372+0000 7fcf8f400700 1 --2- 172.21.15.33:0/596445685 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fcf7c0b2af0 0x7fcf7c0b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:06.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.372+0000 7fcf96800700 1 -- 172.21.15.33:0/2918400563 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e47c9a09a0 msgr2=0x55e47c9a0d80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-03T18:37:06.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.372+0000 7fcf96800700 1 --2- 172.21.15.33:0/2918400563 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e47c9a09a0 0x55e47c9a0d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:06.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.372+0000 7fcf96800700 1 -- 172.21.15.33:0/2918400563 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e47c9a53f0 msgr2=0x55e47c9a5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:37:06.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.372+0000 7fcf95e00700 1 --2- 172.21.15.33:0/2918400563 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e47c9a53f0 0x55e47c9a5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:06.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.372+0000 7fcf96800700 1 --2- 172.21.15.33:0/2918400563 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e47c9a53f0 0x55e47c9a5830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:06.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.373+0000 7fcf96800700 1 -- 172.21.15.33:0/2918400563 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e47c8378d0 con 0x55e47c9a12c0 2024-05-03T18:37:06.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.373+0000 7fcf8f400700 1 -- 172.21.15.33:0/596445685 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9518+0+0 (secure 0 0 0) 0x7fcf9000f030 con 0x7fcf9810d9c0 2024-05-03T18:37:06.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.373+0000 7fcf97200700 1 --2- 172.21.15.33:0/596445685 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fcf7c0b2af0 0x7fcf7c0b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:06.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.373+0000 7fcf95e00700 1 --2- 172.21.15.33:0/2918400563 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e47c9a53f0 0x55e47c9a5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:37:06.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.373+0000 7fcf96800700 1 --2- 172.21.15.33:0/2918400563 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e47c9a12c0 0x55e47c9a4e60 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fcf88009d80 tx=0x7fcf88011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67ca492c83ec9f12 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:06.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.374+0000 7fcf83e00700 1 -- 172.21.15.33:0/2918400563 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf88004230 con 0x55e47c9a12c0 2024-05-03T18:37:06.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.374+0000 7fcf83e00700 1 -- 172.21.15.33:0/2918400563 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcf880043d0 con 0x55e47c9a12c0 2024-05-03T18:37:06.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.374+0000 7fcf83e00700 1 -- 172.21.15.33:0/2918400563 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf88019780 con 0x55e47c9a12c0 2024-05-03T18:37:06.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.375+0000 7fcfa7e32b80 1 -- 172.21.15.33:0/2918400563 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e47c9a12c0 msgr2=0x55e47c9a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:06.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.375+0000 7fcfa7e32b80 1 --2- 172.21.15.33:0/2918400563 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e47c9a12c0 0x55e47c9a4e60 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fcf88009d80 tx=0x7fcf88011720 comp rx=0 tx=0).stop 2024-05-03T18:37:06.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.375+0000 7fcf97200700 1 --2- 172.21.15.33:0/596445685 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fcf7c0b2af0 0x7fcf7c0b4fb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fcf900aa250 tx=0x7fcf90009040 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:06.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.377+0000 7fcfa7e32b80 1 -- 172.21.15.33:0/2918400563 shutdown_connections 2024-05-03T18:37:06.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.377+0000 7fcfa7e32b80 1 --2- 172.21.15.33:0/2918400563 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e47c9a53f0 0x55e47c9a5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:06.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.377+0000 7fcfa7e32b80 1 --2- 172.21.15.33:0/2918400563 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e47c9a09a0 0x55e47c9a0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:06.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.377+0000 7fcfa7e32b80 1 --2- 172.21.15.33:0/2918400563 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e47c9a12c0 0x55e47c9a4e60 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:06.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.378+0000 7fcfa7e32b80 1 -- 172.21.15.33:0/2918400563 >> 172.21.15.33:0/2918400563 conn(0x55e47c8c6b60 msgr2=0x55e47c9929d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:37:06.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.378+0000 7fcfa7e32b80 1 -- 172.21.15.33:0/2918400563 shutdown_connections 2024-05-03T18:37:06.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.378+0000 7fcfa7e32b80 1 -- 172.21.15.33:0/2918400563 wait complete. 2024-05-03T18:37:06.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.381+0000 7fcfa7e32b80 1 -- start start 2024-05-03T18:37:06.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.382+0000 7fcfa7e32b80 1 Processor -- start 2024-05-03T18:37:06.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.383+0000 7fcfa7e32b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e47c9a09a0 0x55e47c9ac8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:06.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.383+0000 7fcfa7e32b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e47c9a12c0 0x55e47c9a9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:06.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.384+0000 7fcfa7e32b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e47c9a53f0 0x55e47c9aa240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:06.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.384+0000 7fcf95e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e47c9a09a0 0x55e47c9ac8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:06.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.384+0000 7fcf95e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e47c9a09a0 0x55e47c9ac8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:60746/0 (socket says 172.21.15.33:60746) 2024-05-03T18:37:06.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.384+0000 7fcf95e00700 1 -- 172.21.15.33:0/2980971066 learned_addr learned my addr 172.21.15.33:0/2980971066 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:37:06.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.384+0000 7fcfa7e32b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e47c9ad610 con 0x55e47c9a12c0 2024-05-03T18:37:06.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.384+0000 7fcfa7e32b80 1 -- 172.21.15.33:0/2980971066 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55e47c850b20 con 0x55e47c9a53f0 2024-05-03T18:37:06.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.384+0000 7fcfa7e32b80 1 -- 172.21.15.33:0/2980971066 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55e47c9ad790 con 0x55e47c9a09a0 2024-05-03T18:37:06.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.384+0000 7fcfa7e32b80 10 client.0 _ll_register_callbacks cb 0x55e47c891c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:37:06.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.385+0000 7fcf96800700 1 --2- 172.21.15.33:0/2980971066 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e47c9a12c0 0x55e47c9a9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:06.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.385+0000 7fcf95e00700 1 --2- 172.21.15.33:0/2980971066 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e47c9a53f0 0x55e47c9aa240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:06.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.385+0000 7fcf95e00700 1 -- 172.21.15.33:0/2980971066 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e47c9a53f0 msgr2=0x55e47c9aa240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:37:06.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.385+0000 7fcf95e00700 1 --2- 172.21.15.33:0/2980971066 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e47c9a53f0 0x55e47c9aa240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:06.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.385+0000 7fcf95e00700 1 -- 172.21.15.33:0/2980971066 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e47c9a12c0 msgr2=0x55e47c9a9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:37:06.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.385+0000 7fcf95e00700 1 --2- 172.21.15.33:0/2980971066 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e47c9a12c0 0x55e47c9a9d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:06.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.385+0000 7fcf95e00700 1 -- 172.21.15.33:0/2980971066 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e47c8b0420 con 0x55e47c9a09a0 2024-05-03T18:37:06.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.386+0000 7fcf95e00700 1 --2- 172.21.15.33:0/2980971066 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e47c9a09a0 0x55e47c9ac8f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fcf84014720 tx=0x7fcf840045d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26f4abf176b6f391 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:06.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.386+0000 7fcf80c00700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-03T18:37:06.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.387+0000 7fcf80c00700 1 -- 172.21.15.33:0/2980971066 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf8401a020 con 0x55e47c9a09a0 2024-05-03T18:37:06.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.387+0000 7fcf80c00700 1 -- 172.21.15.33:0/2980971066 <== mon.1 v2:172.21.15.150:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9518+0+0 (secure 0 0 0) 0x7fcf8400b620 con 0x55e47c9a09a0 2024-05-03T18:37:06.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.387+0000 7fcfa7e32b80 1 -- 172.21.15.33:0/2980971066 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf880043d0 con 0x55e47c9a09a0 2024-05-03T18:37:06.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.388+0000 7fcf80c00700 1 -- 172.21.15.33:0/2980971066 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fcf84025050 con 0x55e47c9a09a0 2024-05-03T18:37:06.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.388+0000 7fcfa7e32b80 10 client.26089 fetch_fsmap learned FSMap version 21 2024-05-03T18:37:06.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.388+0000 7fcfa7e32b80 1 -- 172.21.15.33:0/2980971066 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55e47c9b7f40 con 0x55e47c9a09a0 2024-05-03T18:37:06.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.389+0000 7fcf80c00700 1 -- 172.21.15.33:0/2980971066 <== mon.1 v2:172.21.15.150:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcf880043d0 con 0x55e47c9a09a0 2024-05-03T18:37:06.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.389+0000 7fcf80c00700 1 -- 172.21.15.33:0/2980971066 <== mon.1 v2:172.21.15.150:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fcf8400f040 con 0x55e47c9a09a0 2024-05-03T18:37:06.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.389+0000 7fcfa7e32b80 10 client.26089 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:37:06.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.389+0000 7fcfa7e32b80 10 client.26089 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:37:06.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.389+0000 7fcfa7e32b80 20 client.26089 populate_metadata read hostname 'smithi033' 2024-05-03T18:37:06.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.389+0000 7fcfa7e32b80 1 --2- 172.21.15.33:0/2980971066 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55e47c9ba580 0x55e47c9bc970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:06.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.389+0000 7fcfa7e32b80 4 client.26089 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:37:06.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.389+0000 7fcfa7e32b80 1 -- 172.21.15.33:0/2980971066 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e47c9651d0 con 0x55e47c9ba580 2024-05-03T18:37:06.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.391+0000 7fcf96800700 1 --2- 172.21.15.33:0/2980971066 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55e47c9ba580 0x55e47c9bc970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:06.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.392+0000 7fcf96800700 1 --2- 172.21.15.33:0/2980971066 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55e47c9ba580 0x55e47c9bc970 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=445489f9830bdb4f server_cookie=4872431e6961533a in_seq=0 out_seq=0 2024-05-03T18:37:06.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.393+0000 7fcf80c00700 10 client.26089 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:37:06.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.394+0000 7fcf80c00700 1 -- 172.21.15.33:0/2980971066 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e47c9651d0 con 0x55e47c9ba580 2024-05-03T18:37:06.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.394+0000 7fcf80c00700 10 client.26089 handle_command_reply: tid=0 2024-05-03T18:37:06.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.395+0000 7fcfa7e32b80 1 client.26089 shutdown 2024-05-03T18:37:06.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.396+0000 7fcfa7e32b80 1 -- 172.21.15.33:0/2980971066 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e47c9a09a0 msgr2=0x55e47c9ac8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:06.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.396+0000 7fcfa7e32b80 1 --2- 172.21.15.33:0/2980971066 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e47c9a09a0 0x55e47c9ac8f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fcf84014720 tx=0x7fcf840045d0 comp rx=0 tx=0).stop 2024-05-03T18:37:06.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.398+0000 7fcfa7e32b80 1 -- 172.21.15.33:0/2980971066 shutdown_connections 2024-05-03T18:37:06.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.398+0000 7fcfa7e32b80 1 --2- 172.21.15.33:0/2980971066 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55e47c9ba580 0x55e47c9bc970 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:06.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.398+0000 7fcfa7e32b80 1 --2- 172.21.15.33:0/2980971066 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e47c9a53f0 0x55e47c9aa240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:06.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.398+0000 7fcfa7e32b80 1 --2- 172.21.15.33:0/2980971066 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e47c9a09a0 0x55e47c9ac8f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:06.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.398+0000 7fcfa7e32b80 1 --2- 172.21.15.33:0/2980971066 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e47c9a12c0 0x55e47c9a9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:06.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.398+0000 7fcfa7e32b80 1 -- 172.21.15.33:0/2980971066 >> 172.21.15.33:0/2980971066 conn(0x55e47c8c6b60 msgr2=0x55e47c992080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:37:06.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.398+0000 7fcf80c00700 0 client.26089 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:37:06.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.398+0000 7fcfa7e32b80 1 -- 172.21.15.33:0/2980971066 shutdown_connections 2024-05-03T18:37:06.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.399+0000 7fcfa7e32b80 1 -- 172.21.15.33:0/2980971066 wait complete. 2024-05-03T18:37:06.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.399+0000 7fcfa7e32b80 20 client.26089 trim_cache size 0 max 16384 2024-05-03T18:37:06.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.431+0000 7fcfa7e32b80 1 Processor -- start 2024-05-03T18:37:06.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.432+0000 7fcfa7e32b80 1 -- start start 2024-05-03T18:37:06.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.432+0000 7fcfa7e32b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e47c9a09a0 0x55e47c9b66d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:06.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.432+0000 7fcfa7e32b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e47c9a12c0 0x55e47c9c2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:06.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.432+0000 7fcfa7e32b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e47c9a53f0 0x55e47c9c2750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:06.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.432+0000 7fcfa7e32b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcf981ab1c0 con 0x55e47c9a53f0 2024-05-03T18:37:06.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.432+0000 7fcfa7e32b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fcf981ada00 con 0x55e47c9a09a0 2024-05-03T18:37:06.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.432+0000 7fcfa7e32b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fcf981ab810 con 0x55e47c9a12c0 2024-05-03T18:37:06.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.433+0000 7fcf96800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e47c9a12c0 0x55e47c9c2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:06.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.433+0000 7fcf96800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e47c9a12c0 0x55e47c9c2210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:60762/0 (socket says 172.21.15.33:60762) 2024-05-03T18:37:06.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.433+0000 7fcf96800700 1 -- 172.21.15.33:0/2674411133 learned_addr learned my addr 172.21.15.33:0/2674411133 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:37:06.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.433+0000 7fcf95e00700 1 --2- 172.21.15.33:0/2674411133 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e47c9a53f0 0x55e47c9c2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:06.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.433+0000 7fcf95e00700 1 --2- 172.21.15.33:0/2674411133 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e47c9a09a0 0x55e47c9b66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:06.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.434+0000 7fcf96800700 1 -- 172.21.15.33:0/2674411133 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e47c9a09a0 msgr2=0x55e47c9b66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:37:06.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.434+0000 7fcf96800700 1 --2- 172.21.15.33:0/2674411133 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e47c9a09a0 0x55e47c9b66d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:06.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.434+0000 7fcf96800700 1 -- 172.21.15.33:0/2674411133 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e47c9a53f0 msgr2=0x55e47c9c2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:37:06.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.434+0000 7fcf96800700 1 --2- 172.21.15.33:0/2674411133 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e47c9a53f0 0x55e47c9c2750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:06.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.434+0000 7fcf96800700 1 -- 172.21.15.33:0/2674411133 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e47c9a0fd0 con 0x55e47c9a12c0 2024-05-03T18:37:06.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.434+0000 7fcf95e00700 1 --2- 172.21.15.33:0/2674411133 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e47c9a09a0 0x55e47c9b66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:37:06.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.434+0000 7fcf96800700 1 --2- 172.21.15.33:0/2674411133 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e47c9a12c0 0x55e47c9c2210 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fcf88006010 tx=0x7fcf88005fa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e17504da0d78e41 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:06.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.435+0000 7fcf83400700 1 -- 172.21.15.33:0/2674411133 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf88004290 con 0x55e47c9a12c0 2024-05-03T18:37:06.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.435+0000 7fcf83400700 1 -- 172.21.15.33:0/2674411133 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcf88004430 con 0x55e47c9a12c0 2024-05-03T18:37:06.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.436+0000 7fcfa7e32b80 1 -- 172.21.15.33:0/2674411133 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e47c9a12c0 msgr2=0x55e47c9c2210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:06.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.436+0000 7fcf83400700 1 -- 172.21.15.33:0/2674411133 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf88003a40 con 0x55e47c9a12c0 2024-05-03T18:37:06.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.436+0000 7fcfa7e32b80 1 --2- 172.21.15.33:0/2674411133 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e47c9a12c0 0x55e47c9c2210 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fcf88006010 tx=0x7fcf88005fa0 comp rx=0 tx=0).stop 2024-05-03T18:37:06.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.436+0000 7fcfa7e32b80 1 -- 172.21.15.33:0/2674411133 shutdown_connections 2024-05-03T18:37:06.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.436+0000 7fcfa7e32b80 1 --2- 172.21.15.33:0/2674411133 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e47c9a53f0 0x55e47c9c2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:06.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.436+0000 7fcfa7e32b80 1 --2- 172.21.15.33:0/2674411133 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e47c9a09a0 0x55e47c9b66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:06.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.436+0000 7fcfa7e32b80 1 --2- 172.21.15.33:0/2674411133 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e47c9a12c0 0x55e47c9c2210 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:06.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.436+0000 7fcfa7e32b80 1 -- 172.21.15.33:0/2674411133 >> 172.21.15.33:0/2674411133 conn(0x55e47c8c6b60 msgr2=0x55e47c996fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:37:06.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.437+0000 7fcfa7e32b80 1 -- 172.21.15.33:0/2674411133 shutdown_connections 2024-05-03T18:37:06.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.437+0000 7fcfa7e32b80 1 -- 172.21.15.33:0/2674411133 wait complete. 2024-05-03T18:37:06.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.437+0000 7fcfa7e32b80 1 -- start start 2024-05-03T18:37:06.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.438+0000 7fcfa7e32b80 1 Processor -- start 2024-05-03T18:37:06.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.440+0000 7fcfa7e32b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e47c9a09a0 0x55e47c9b2c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:06.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.440+0000 7fcfa7e32b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e47c9a12c0 0x55e47c9b3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:06.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.440+0000 7fcfa7e32b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e47c9a53f0 0x55e47c9b11d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:06.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.440+0000 7fcfa7e32b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcf981abec0 con 0x55e47c9a53f0 2024-05-03T18:37:06.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.440+0000 7fcfa7e32b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fcf981ad330 con 0x55e47c9a12c0 2024-05-03T18:37:06.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.440+0000 7fcfa7e32b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fcf981ac590 con 0x55e47c9a09a0 2024-05-03T18:37:06.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.440+0000 7fcfa7e32b80 10 client.0 _ll_register_callbacks cb 0x55e47c891c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:37:06.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.441+0000 7fcf95e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e47c9a09a0 0x55e47c9b2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:06.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.441+0000 7fcf96800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e47c9a12c0 0x55e47c9b3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:06.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.441+0000 7fcf95e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e47c9a09a0 0x55e47c9b2c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:60778/0 (socket says 172.21.15.33:60778) 2024-05-03T18:37:06.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.441+0000 7fcf95e00700 1 -- 172.21.15.33:0/1643222263 learned_addr learned my addr 172.21.15.33:0/1643222263 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:37:06.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.441+0000 7fcf96800700 1 -- 172.21.15.33:0/1643222263 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e47c9a09a0 msgr2=0x55e47c9b2c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:37:06.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.441+0000 7fcf96800700 1 --2- 172.21.15.33:0/1643222263 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e47c9a09a0 0x55e47c9b2c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:06.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.441+0000 7fcf96800700 1 -- 172.21.15.33:0/1643222263 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e47c9a53f0 msgr2=0x55e47c9b11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:37:06.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.441+0000 7fcf96800700 1 --2- 172.21.15.33:0/1643222263 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e47c9a53f0 0x55e47c9b11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:06.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.441+0000 7fcf96800700 1 -- 172.21.15.33:0/1643222263 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e47c9b1a10 con 0x55e47c9a12c0 2024-05-03T18:37:06.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.442+0000 7fcf96800700 1 --2- 172.21.15.33:0/1643222263 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e47c9a12c0 0x55e47c9b3160 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x55e47c8e65d0 tx=0x7fcf880040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b22b78bfbf5ff8b server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:06.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.442+0000 7fcf80c00700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-03T18:37:06.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.443+0000 7fcf80c00700 1 -- 172.21.15.33:0/1643222263 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf88038070 con 0x55e47c9a12c0 2024-05-03T18:37:06.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.444+0000 7fcf80c00700 1 -- 172.21.15.33:0/1643222263 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9518+0+0 (secure 0 0 0) 0x7fcf88005610 con 0x55e47c9a12c0 2024-05-03T18:37:06.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.445+0000 7fcfa7e32b80 1 -- 172.21.15.33:0/1643222263 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf88004430 con 0x55e47c9a12c0 2024-05-03T18:37:06.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.446+0000 7fcf80c00700 1 -- 172.21.15.33:0/1643222263 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fcf88042050 con 0x55e47c9a12c0 2024-05-03T18:37:06.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.447+0000 7fcfa7e32b80 10 client.36032 fetch_fsmap learned FSMap version 21 2024-05-03T18:37:06.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.447+0000 7fcfa7e32b80 1 -- 172.21.15.33:0/1643222263 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55e47c99fc20 con 0x55e47c9a12c0 2024-05-03T18:37:06.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.448+0000 7fcf80c00700 1 -- 172.21.15.33:0/1643222263 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcf88004430 con 0x55e47c9a12c0 2024-05-03T18:37:06.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.448+0000 7fcf80c00700 1 -- 172.21.15.33:0/1643222263 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fcf8802b720 con 0x55e47c9a12c0 2024-05-03T18:37:06.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.450+0000 7fcfa7e32b80 10 client.36032 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:37:06.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.450+0000 7fcfa7e32b80 10 client.36032 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:37:06.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.450+0000 7fcfa7e32b80 20 client.36032 populate_metadata read hostname 'smithi033' 2024-05-03T18:37:06.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.450+0000 7fcfa7e32b80 1 --2- 172.21.15.33:0/1643222263 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55e47c9a0090 0x55e47ca2fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:06.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.450+0000 7fcfa7e32b80 4 client.36032 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-03T18:37:06.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.450+0000 7fcfa7e32b80 1 -- 172.21.15.33:0/1643222263 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e47c8f8400 con 0x55e47c9a0090 2024-05-03T18:37:06.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.451+0000 7fcf95e00700 1 --2- 172.21.15.33:0/1643222263 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55e47c9a0090 0x55e47ca2fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:06.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.454+0000 7fcf95e00700 1 --2- 172.21.15.33:0/1643222263 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55e47c9a0090 0x55e47ca2fda0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5fc9721bc5a0b8f2 server_cookie=10ede415207b0016 in_seq=0 out_seq=0 2024-05-03T18:37:06.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.456+0000 7fcf80c00700 10 client.36032 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:37:06.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.456+0000 7fcf80c00700 1 -- 172.21.15.33:0/1643222263 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e47c8f8400 con 0x55e47c9a0090 2024-05-03T18:37:06.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.456+0000 7fcf80c00700 10 client.36032 handle_command_reply: tid=0 2024-05-03T18:37:06.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.456+0000 7fcfa7e32b80 1 client.36032 shutdown 2024-05-03T18:37:06.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.457+0000 7fcfa7e32b80 1 -- 172.21.15.33:0/1643222263 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e47c9a12c0 msgr2=0x55e47c9b3160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:06.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.457+0000 7fcfa7e32b80 1 --2- 172.21.15.33:0/1643222263 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e47c9a12c0 0x55e47c9b3160 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x55e47c8e65d0 tx=0x7fcf880040c0 comp rx=0 tx=0).stop 2024-05-03T18:37:06.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.457+0000 7fcfa7e32b80 1 -- 172.21.15.33:0/1643222263 shutdown_connections 2024-05-03T18:37:06.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.457+0000 7fcfa7e32b80 1 --2- 172.21.15.33:0/1643222263 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55e47c9a0090 0x55e47ca2fda0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:06.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.458+0000 7fcfa7e32b80 1 --2- 172.21.15.33:0/1643222263 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e47c9a53f0 0x55e47c9b11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:06.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.458+0000 7fcfa7e32b80 1 --2- 172.21.15.33:0/1643222263 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e47c9a09a0 0x55e47c9b2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:06.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.458+0000 7fcfa7e32b80 1 --2- 172.21.15.33:0/1643222263 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e47c9a12c0 0x55e47c9b3160 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:06.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.458+0000 7fcfa7e32b80 1 -- 172.21.15.33:0/1643222263 >> 172.21.15.33:0/1643222263 conn(0x55e47c8c6b60 msgr2=0x55e47c9a4b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:37:06.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.458+0000 7fcf80c00700 0 client.36032 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:37:06.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.459+0000 7fcfa7e32b80 1 -- 172.21.15.33:0/1643222263 shutdown_connections 2024-05-03T18:37:06.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.459+0000 7fcfa7e32b80 1 -- 172.21.15.33:0/1643222263 wait complete. 2024-05-03T18:37:06.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.459+0000 7fcfa7e32b80 20 client.36032 trim_cache size 0 max 16384 2024-05-03T18:37:06.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.460+0000 7fcf9ca00700 1 -- 172.21.15.33:0/596445685 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fcf7c0b2af0 msgr2=0x7fcf7c0b4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:06.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.460+0000 7fcf9ca00700 1 --2- 172.21.15.33:0/596445685 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fcf7c0b2af0 0x7fcf7c0b4fb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fcf900aa250 tx=0x7fcf90009040 comp rx=0 tx=0).stop 2024-05-03T18:37:06.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.460+0000 7fcf9ca00700 1 -- 172.21.15.33:0/596445685 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf9810d9c0 msgr2=0x7fcf98199800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:06.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.460+0000 7fcf9ca00700 1 --2- 172.21.15.33:0/596445685 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf9810d9c0 0x7fcf98199800 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fcf9000b070 tx=0x7fcf90009c50 comp rx=0 tx=0).stop 2024-05-03T18:37:06.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.460+0000 7fcf9ca00700 1 -- 172.21.15.33:0/596445685 shutdown_connections 2024-05-03T18:37:06.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.460+0000 7fcf9ca00700 1 --2- 172.21.15.33:0/596445685 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fcf7c0b2af0 0x7fcf7c0b4fb0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:06.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.460+0000 7fcf9ca00700 1 --2- 172.21.15.33:0/596445685 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf9810d9c0 0x7fcf98199800 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:06.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.460+0000 7fcf9ca00700 1 --2- 172.21.15.33:0/596445685 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcf98104590 0x7fcf9819f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:06.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.460+0000 7fcf9ca00700 1 --2- 172.21.15.33:0/596445685 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf98104eb0 0x7fcf9819fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:06.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.460+0000 7fcf9ca00700 1 -- 172.21.15.33:0/596445685 >> 172.21.15.33:0/596445685 conn(0x7fcf980fddc0 msgr2=0x7fcf980feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:37:06.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.462+0000 7fcf9ca00700 1 -- 172.21.15.33:0/596445685 shutdown_connections 2024-05-03T18:37:06.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:06.462+0000 7fcf9ca00700 1 -- 172.21.15.33:0/596445685 wait complete. 2024-05-03T18:37:08.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:37:08 smithi150 ceph-mon[32492]: pgmap v307: 129 pgs: 129 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 4.7 MiB/s wr, 507 op/s 2024-05-03T18:37:08.415 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:37:08 smithi033 ceph-mon[28330]: pgmap v307: 129 pgs: 129 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 4.7 MiB/s wr, 507 op/s 2024-05-03T18:37:08.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:37:08 smithi099 ceph-mon[31658]: pgmap v307: 129 pgs: 129 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 4.7 MiB/s wr, 507 op/s 2024-05-03T18:37:09.518 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-03T18:37:09.518 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-03T18:37:10.110 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 355.0 s, 596.3 tps, lat 166.915 ms stddev 62.887 2024-05-03T18:37:10.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:37:10 smithi150 ceph-mon[32492]: pgmap v308: 129 pgs: 129 active+clean; 2.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 6.9 MiB/s wr, 747 op/s 2024-05-03T18:37:10.410 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:37:10.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:37:10 smithi033 ceph-mon[28330]: pgmap v308: 129 pgs: 129 active+clean; 2.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 6.9 MiB/s wr, 747 op/s 2024-05-03T18:37:10.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:37:10 smithi099 ceph-mon[31658]: pgmap v308: 129 pgs: 129 active+clean; 2.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 6.9 MiB/s wr, 747 op/s 2024-05-03T18:37:12.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:37:12 smithi033 ceph-mon[28330]: osdmap e97: 12 total, 12 up, 12 in 2024-05-03T18:37:12.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:37:12 smithi033 ceph-mon[28330]: pgmap v310: 129 pgs: 129 active+clean; 2.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 5.5 MiB/s wr, 595 op/s 2024-05-03T18:37:12.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:37:12 smithi099 ceph-mon[31658]: osdmap e97: 12 total, 12 up, 12 in 2024-05-03T18:37:12.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:37:12 smithi099 ceph-mon[31658]: pgmap v310: 129 pgs: 129 active+clean; 2.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 5.5 MiB/s wr, 595 op/s 2024-05-03T18:37:12.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:37:12 smithi150 ceph-mon[32492]: osdmap e97: 12 total, 12 up, 12 in 2024-05-03T18:37:12.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:37:12 smithi150 ceph-mon[32492]: pgmap v310: 129 pgs: 129 active+clean; 2.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 5.5 MiB/s wr, 595 op/s 2024-05-03T18:37:14.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:37:14 smithi150 ceph-mon[32492]: pgmap v311: 129 pgs: 129 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 5.3 MiB/s wr, 576 op/s 2024-05-03T18:37:14.385 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:37:14 smithi033 ceph-mon[28330]: pgmap v311: 129 pgs: 129 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 5.3 MiB/s wr, 576 op/s 2024-05-03T18:37:14.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:37:14 smithi099 ceph-mon[31658]: pgmap v311: 129 pgs: 129 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 5.3 MiB/s wr, 576 op/s 2024-05-03T18:37:14.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.851+0000 7f1379a00700 1 -- 172.21.15.33:0/823969315 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1374109510 msgr2=0x7f13741098f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:14.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.851+0000 7f1379a00700 1 --2- 172.21.15.33:0/823969315 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1374109510 0x7f13741098f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1364009990 tx=0x7f1364009ca0 comp rx=0 tx=0).stop 2024-05-03T18:37:14.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.852+0000 7f1379a00700 1 -- 172.21.15.33:0/823969315 shutdown_connections 2024-05-03T18:37:14.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.852+0000 7f1379a00700 1 --2- 172.21.15.33:0/823969315 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f137406bb40 0x7f1374110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:14.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.852+0000 7f1379a00700 1 --2- 172.21.15.33:0/823969315 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1374109510 0x7f13741098f0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:14.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.852+0000 7f1379a00700 1 --2- 172.21.15.33:0/823969315 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f137406b180 0x7f137406b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:14.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.852+0000 7f1379a00700 1 -- 172.21.15.33:0/823969315 >> 172.21.15.33:0/823969315 conn(0x7f1374079350 msgr2=0x7f1374079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:37:14.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.852+0000 7f1379a00700 1 -- 172.21.15.33:0/823969315 shutdown_connections 2024-05-03T18:37:14.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.852+0000 7f1379a00700 1 -- 172.21.15.33:0/823969315 wait complete. 2024-05-03T18:37:14.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.855+0000 7f1379a00700 1 Processor -- start 2024-05-03T18:37:14.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.855+0000 7f1379a00700 1 -- start start 2024-05-03T18:37:14.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.856+0000 7f1379a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f137406b180 0x7f13741a4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:14.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.856+0000 7f1379a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f137406bb40 0x7f13741a51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:14.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.856+0000 7f1379a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1374109510 0x7f137410f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:14.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.856+0000 7f1379a00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1374113570 con 0x7f137406b180 2024-05-03T18:37:14.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.856+0000 7f1379a00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f13741133f0 con 0x7f1374109510 2024-05-03T18:37:14.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.856+0000 7f1379a00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f13741136f0 con 0x7f137406bb40 2024-05-03T18:37:14.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.856+0000 7f1372a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f137406b180 0x7f13741a4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:14.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.857+0000 7f1372a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f137406b180 0x7f13741a4c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34728/0 (socket says 172.21.15.33:34728) 2024-05-03T18:37:14.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.857+0000 7f1372a00700 1 -- 172.21.15.33:0/490023949 learned_addr learned my addr 172.21.15.33:0/490023949 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:37:14.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.857+0000 7f1372000700 1 --2- 172.21.15.33:0/490023949 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f137406bb40 0x7f13741a51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:14.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.857+0000 7f1373400700 1 --2- 172.21.15.33:0/490023949 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1374109510 0x7f137410f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:14.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.857+0000 7f1372a00700 1 -- 172.21.15.33:0/490023949 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f137406bb40 msgr2=0x7f13741a51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:14.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.857+0000 7f1372a00700 1 --2- 172.21.15.33:0/490023949 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f137406bb40 0x7f13741a51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:14.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.857+0000 7f1372a00700 1 -- 172.21.15.33:0/490023949 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1374109510 msgr2=0x7f137410f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:14.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.857+0000 7f1372a00700 1 --2- 172.21.15.33:0/490023949 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1374109510 0x7f137410f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:14.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.857+0000 7f1372a00700 1 -- 172.21.15.33:0/490023949 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f137410fc20 con 0x7f137406b180 2024-05-03T18:37:14.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.857+0000 7f1372a00700 1 --2- 172.21.15.33:0/490023949 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f137406b180 0x7f13741a4c60 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f136400b650 tx=0x7f136400bfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:14.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.858+0000 7f136b400700 1 -- 172.21.15.33:0/490023949 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f136400fb10 con 0x7f137406b180 2024-05-03T18:37:14.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.858+0000 7f1379a00700 1 -- 172.21.15.33:0/490023949 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f137410feb0 con 0x7f137406b180 2024-05-03T18:37:14.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.858+0000 7f1379a00700 1 -- 172.21.15.33:0/490023949 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13741acb90 con 0x7f137406b180 2024-05-03T18:37:14.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.858+0000 7f136b400700 1 -- 172.21.15.33:0/490023949 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f136400fcb0 con 0x7f137406b180 2024-05-03T18:37:14.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.858+0000 7f136b400700 1 -- 172.21.15.33:0/490023949 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1364021620 con 0x7f137406b180 2024-05-03T18:37:14.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.864+0000 7f136b400700 1 -- 172.21.15.33:0/490023949 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f1364009d10 con 0x7f137406b180 2024-05-03T18:37:14.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.867+0000 7f136b400700 1 --2- 172.21.15.33:0/490023949 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f13580b2c20 0x7f13580b50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:14.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.867+0000 7f136b400700 1 -- 172.21.15.33:0/490023949 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9606+0+0 (secure 0 0 0) 0x7f13640213c0 con 0x7f137406b180 2024-05-03T18:37:14.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.868+0000 7f1372000700 1 --2- 172.21.15.33:0/490023949 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f13580b2c20 0x7f13580b50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:14.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.869+0000 7f1372000700 1 --2- 172.21.15.33:0/490023949 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f13580b2c20 0x7f13580b50e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1360005ea0 tx=0x7f1360005e30 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:14.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.878+0000 7f1384310b80 1 Processor -- start 2024-05-03T18:37:14.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.880+0000 7f1384310b80 1 -- start start 2024-05-03T18:37:14.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.881+0000 7f1384310b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56103075f9b0 0x56103075fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:14.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.881+0000 7f1384310b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5610307602d0 0x561030759fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:14.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.882+0000 7f1372a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56103075f9b0 0x56103075fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:14.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.882+0000 7f1384310b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56103075a5d0 0x56103075aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:14.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.882+0000 7f1372a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56103075f9b0 0x56103075fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:60798/0 (socket says 172.21.15.33:60798) 2024-05-03T18:37:14.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.883+0000 7f1372000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5610307602d0 0x561030759fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:14.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.883+0000 7f1372000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5610307602d0 0x561030759fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:57148/0 (socket says 172.21.15.33:57148) 2024-05-03T18:37:14.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.883+0000 7f1384310b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5610306acaa0 con 0x56103075a5d0 2024-05-03T18:37:14.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.883+0000 7f1384310b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56103064a7e0 con 0x5610307602d0 2024-05-03T18:37:14.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.883+0000 7f1384310b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5610306a8a60 con 0x56103075f9b0 2024-05-03T18:37:14.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.884+0000 7f1373400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56103075a5d0 0x56103075aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:14.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.884+0000 7f1373400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56103075a5d0 0x56103075aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34732/0 (socket says 172.21.15.33:34732) 2024-05-03T18:37:14.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.884+0000 7f1373400700 1 -- 172.21.15.33:0/2622115877 learned_addr learned my addr 172.21.15.33:0/2622115877 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:37:14.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.886+0000 7f1373400700 1 -- 172.21.15.33:0/2622115877 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56103075f9b0 msgr2=0x56103075fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:37:14.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.886+0000 7f1373400700 1 --2- 172.21.15.33:0/2622115877 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56103075f9b0 0x56103075fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:14.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.886+0000 7f1373400700 1 -- 172.21.15.33:0/2622115877 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5610307602d0 msgr2=0x561030759fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:37:14.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.886+0000 7f1373400700 1 --2- 172.21.15.33:0/2622115877 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5610307602d0 0x561030759fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:14.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.886+0000 7f1373400700 1 -- 172.21.15.33:0/2622115877 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610305f9950 con 0x56103075a5d0 2024-05-03T18:37:14.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.886+0000 7f1372a00700 1 --2- 172.21.15.33:0/2622115877 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56103075f9b0 0x56103075fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-05-03T18:37:14.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.886+0000 7f1372000700 1 --2- 172.21.15.33:0/2622115877 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5610307602d0 0x561030759fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-05-03T18:37:14.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.887+0000 7f1373400700 1 --2- 172.21.15.33:0/2622115877 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56103075a5d0 0x56103075aa50 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f136c0093a0 tx=0x7f136c009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64be3b08f7345b2f server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:14.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.888+0000 7f135fe00700 1 -- 172.21.15.33:0/2622115877 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f136c01a070 con 0x56103075a5d0 2024-05-03T18:37:14.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.889+0000 7f135fe00700 1 -- 172.21.15.33:0/2622115877 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f136c013b90 con 0x56103075a5d0 2024-05-03T18:37:14.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.889+0000 7f135fe00700 1 -- 172.21.15.33:0/2622115877 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f136c015ae0 con 0x56103075a5d0 2024-05-03T18:37:14.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.891+0000 7f1384310b80 1 -- 172.21.15.33:0/2622115877 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56103075a5d0 msgr2=0x56103075aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:14.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.891+0000 7f1384310b80 1 --2- 172.21.15.33:0/2622115877 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56103075a5d0 0x56103075aa50 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f136c0093a0 tx=0x7f136c009920 comp rx=0 tx=0).stop 2024-05-03T18:37:14.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.892+0000 7f1384310b80 1 -- 172.21.15.33:0/2622115877 shutdown_connections 2024-05-03T18:37:14.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.892+0000 7f1384310b80 1 --2- 172.21.15.33:0/2622115877 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56103075a5d0 0x56103075aa50 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:14.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.892+0000 7f1384310b80 1 --2- 172.21.15.33:0/2622115877 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56103075f9b0 0x56103075fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:14.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.893+0000 7f1384310b80 1 --2- 172.21.15.33:0/2622115877 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5610307602d0 0x561030759fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:14.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.893+0000 7f1384310b80 1 -- 172.21.15.33:0/2622115877 >> 172.21.15.33:0/2622115877 conn(0x5610306d2750 msgr2=0x561030754980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:37:14.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.900+0000 7f1384310b80 1 -- 172.21.15.33:0/2622115877 shutdown_connections 2024-05-03T18:37:14.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.901+0000 7f1384310b80 1 -- 172.21.15.33:0/2622115877 wait complete. 2024-05-03T18:37:14.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.901+0000 7f1384310b80 1 -- start start 2024-05-03T18:37:14.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.902+0000 7f1384310b80 1 Processor -- start 2024-05-03T18:37:14.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.905+0000 7f1384310b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56103075a5d0 0x561030765ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:14.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.906+0000 7f1384310b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56103075f9b0 0x561030763260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:14.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.906+0000 7f1384310b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5610307602d0 0x5610307637a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:14.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.906+0000 7f1372000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56103075f9b0 0x561030763260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:14.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.906+0000 7f1372000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56103075f9b0 0x561030763260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:57156/0 (socket says 172.21.15.33:57156) 2024-05-03T18:37:14.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.907+0000 7f1373400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5610307602d0 0x5610307637a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:14.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.907+0000 7f1384310b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x561030766d80 con 0x5610307602d0 2024-05-03T18:37:14.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.907+0000 7f1372a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56103075a5d0 0x561030765ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:14.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.908+0000 7f1372a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56103075a5d0 0x561030765ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:60806/0 (socket says 172.21.15.33:60806) 2024-05-03T18:37:14.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.908+0000 7f1373400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5610307602d0 0x5610307637a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34748/0 (socket says 172.21.15.33:34748) 2024-05-03T18:37:14.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.908+0000 7f1384310b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x561030612ba0 con 0x56103075f9b0 2024-05-03T18:37:14.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.908+0000 7f1384310b80 1 -- 172.21.15.33:0/1158922165 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x561030766f00 con 0x56103075a5d0 2024-05-03T18:37:14.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.908+0000 7f1384310b80 10 client.0 _ll_register_callbacks cb 0x561030653cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:37:14.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.908+0000 7f1372000700 1 -- 172.21.15.33:0/1158922165 learned_addr learned my addr 172.21.15.33:0/1158922165 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:37:14.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.910+0000 7f1373400700 1 -- 172.21.15.33:0/1158922165 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56103075a5d0 msgr2=0x561030765ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:37:14.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.910+0000 7f1373400700 1 --2- 172.21.15.33:0/1158922165 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56103075a5d0 0x561030765ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:14.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.910+0000 7f1373400700 1 -- 172.21.15.33:0/1158922165 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56103075f9b0 msgr2=0x561030763260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:37:14.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.910+0000 7f1373400700 1 --2- 172.21.15.33:0/1158922165 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56103075f9b0 0x561030763260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:14.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.910+0000 7f1373400700 1 -- 172.21.15.33:0/1158922165 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610306724a0 con 0x5610307602d0 2024-05-03T18:37:14.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.910+0000 7f1372000700 1 --2- 172.21.15.33:0/1158922165 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56103075f9b0 0x561030763260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:37:14.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.911+0000 7f1372a00700 1 --2- 172.21.15.33:0/1158922165 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56103075a5d0 0x561030765ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-05-03T18:37:14.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.911+0000 7f1373400700 1 --2- 172.21.15.33:0/1158922165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5610307602d0 0x5610307637a0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f136c009370 tx=0x7f136c0135b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cde62b024c8e19ea server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:14.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.912+0000 7f135cc00700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:37:14.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.913+0000 7f135cc00700 1 -- 172.21.15.33:0/1158922165 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f136c024020 con 0x5610307602d0 2024-05-03T18:37:14.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.914+0000 7f135cc00700 1 -- 172.21.15.33:0/1158922165 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9606+0+0 (secure 0 0 0) 0x7f136c015760 con 0x5610307602d0 2024-05-03T18:37:14.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.915+0000 7f1384310b80 1 -- 172.21.15.33:0/1158922165 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f136c013b90 con 0x5610307602d0 2024-05-03T18:37:14.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.916+0000 7f135cc00700 1 -- 172.21.15.33:0/1158922165 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f136c01a040 con 0x5610307602d0 2024-05-03T18:37:14.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.917+0000 7f1384310b80 10 client.16380 fetch_fsmap learned FSMap version 21 2024-05-03T18:37:14.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.917+0000 7f1384310b80 1 -- 172.21.15.33:0/1158922165 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5610307713f0 con 0x5610307602d0 2024-05-03T18:37:14.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.918+0000 7f135cc00700 1 -- 172.21.15.33:0/1158922165 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f136c013b90 con 0x5610307602d0 2024-05-03T18:37:14.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.918+0000 7f135cc00700 1 -- 172.21.15.33:0/1158922165 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f136c014070 con 0x5610307602d0 2024-05-03T18:37:14.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.919+0000 7f1384310b80 10 client.16380 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:37:14.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.919+0000 7f1384310b80 10 client.16380 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:37:14.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.919+0000 7f1384310b80 20 client.16380 populate_metadata read hostname 'smithi033' 2024-05-03T18:37:14.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.919+0000 7f1384310b80 1 --2- 172.21.15.33:0/1158922165 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5610307738c0 0x561030775cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:14.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.919+0000 7f1384310b80 4 client.16380 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:37:14.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.919+0000 7f1384310b80 1 -- 172.21.15.33:0/1158922165 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561030727070 con 0x5610307738c0 2024-05-03T18:37:14.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.922+0000 7f1372a00700 1 --2- 172.21.15.33:0/1158922165 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5610307738c0 0x561030775cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:14.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.923+0000 7f135cc00700 10 client.16380 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:37:14.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.923+0000 7f1372a00700 1 --2- 172.21.15.33:0/1158922165 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5610307738c0 0x561030775cb0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5bc44285f4c73176 server_cookie=970dfe3d53275cc4 in_seq=0 out_seq=0 2024-05-03T18:37:14.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.926+0000 7f135cc00700 1 -- 172.21.15.33:0/1158922165 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561030727070 con 0x5610307738c0 2024-05-03T18:37:14.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.926+0000 7f135cc00700 10 client.16380 handle_command_reply: tid=0 2024-05-03T18:37:14.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.927+0000 7f1384310b80 1 client.16380 shutdown 2024-05-03T18:37:14.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.927+0000 7f1384310b80 1 -- 172.21.15.33:0/1158922165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5610307602d0 msgr2=0x5610307637a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:14.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.927+0000 7f1384310b80 1 --2- 172.21.15.33:0/1158922165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5610307602d0 0x5610307637a0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f136c009370 tx=0x7f136c0135b0 comp rx=0 tx=0).stop 2024-05-03T18:37:14.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.927+0000 7f1384310b80 1 -- 172.21.15.33:0/1158922165 shutdown_connections 2024-05-03T18:37:14.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.927+0000 7f1384310b80 1 --2- 172.21.15.33:0/1158922165 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5610307738c0 0x561030775cb0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:14.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.928+0000 7f1384310b80 1 --2- 172.21.15.33:0/1158922165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5610307602d0 0x5610307637a0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:14.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.928+0000 7f1384310b80 1 --2- 172.21.15.33:0/1158922165 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56103075a5d0 0x561030765ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:14.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.928+0000 7f1384310b80 1 --2- 172.21.15.33:0/1158922165 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56103075f9b0 0x561030763260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:14.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.928+0000 7f1384310b80 1 -- 172.21.15.33:0/1158922165 >> 172.21.15.33:0/1158922165 conn(0x5610306d2750 msgr2=0x56103074f430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:37:14.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.928+0000 7f135cc00700 0 client.16380 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:37:14.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.928+0000 7f1384310b80 1 -- 172.21.15.33:0/1158922165 shutdown_connections 2024-05-03T18:37:14.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.928+0000 7f1384310b80 1 -- 172.21.15.33:0/1158922165 wait complete. 2024-05-03T18:37:14.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.929+0000 7f1384310b80 20 client.16380 trim_cache size 0 max 16384 2024-05-03T18:37:14.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.980+0000 7f1384310b80 1 Processor -- start 2024-05-03T18:37:14.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.980+0000 7f1384310b80 1 -- start start 2024-05-03T18:37:14.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.981+0000 7f1384310b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56103075a5d0 0x561030759fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:14.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.981+0000 7f1384310b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56103075f9b0 0x561030760e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:14.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.981+0000 7f1384310b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5610307602d0 0x56103077b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:14.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.981+0000 7f1384310b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f13741b3b90 con 0x56103075a5d0 2024-05-03T18:37:14.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.981+0000 7f1384310b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f13741b63f0 con 0x5610307602d0 2024-05-03T18:37:14.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.981+0000 7f1384310b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f13741b41e0 con 0x56103075f9b0 2024-05-03T18:37:14.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.982+0000 7f1373400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5610307602d0 0x56103077b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:14.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.982+0000 7f1372000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56103075f9b0 0x561030760e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:14.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.982+0000 7f1372000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56103075f9b0 0x561030760e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:60812/0 (socket says 172.21.15.33:60812) 2024-05-03T18:37:14.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.982+0000 7f1372000700 1 -- 172.21.15.33:0/1228643944 learned_addr learned my addr 172.21.15.33:0/1228643944 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:37:14.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.983+0000 7f1373400700 1 -- 172.21.15.33:0/1228643944 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56103075f9b0 msgr2=0x561030760e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:37:14.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.983+0000 7f1373400700 1 --2- 172.21.15.33:0/1228643944 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56103075f9b0 0x561030760e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:14.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.983+0000 7f1373400700 1 -- 172.21.15.33:0/1228643944 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56103075a5d0 msgr2=0x561030759fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:37:14.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.983+0000 7f1373400700 1 --2- 172.21.15.33:0/1228643944 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56103075a5d0 0x561030759fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:14.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.983+0000 7f1373400700 1 -- 172.21.15.33:0/1228643944 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561030754730 con 0x5610307602d0 2024-05-03T18:37:14.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.983+0000 7f1373400700 1 --2- 172.21.15.33:0/1228643944 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5610307602d0 0x56103077b2e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f136c015690 tx=0x7f136c009490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54a8e11363ff6793 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:14.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.983+0000 7f135f400700 1 -- 172.21.15.33:0/1228643944 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f136c024050 con 0x5610307602d0 2024-05-03T18:37:14.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.984+0000 7f135f400700 1 -- 172.21.15.33:0/1228643944 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f136c013b60 con 0x5610307602d0 2024-05-03T18:37:14.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.984+0000 7f135f400700 1 -- 172.21.15.33:0/1228643944 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f136c015730 con 0x5610307602d0 2024-05-03T18:37:14.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.986+0000 7f1384310b80 1 -- 172.21.15.33:0/1228643944 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5610307602d0 msgr2=0x56103077b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:14.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.986+0000 7f1384310b80 1 --2- 172.21.15.33:0/1228643944 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5610307602d0 0x56103077b2e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f136c015690 tx=0x7f136c009490 comp rx=0 tx=0).stop 2024-05-03T18:37:14.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.986+0000 7f1384310b80 1 -- 172.21.15.33:0/1228643944 shutdown_connections 2024-05-03T18:37:14.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.987+0000 7f1384310b80 1 --2- 172.21.15.33:0/1228643944 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5610307602d0 0x56103077b2e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:14.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.988+0000 7f1384310b80 1 --2- 172.21.15.33:0/1228643944 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56103075a5d0 0x561030759fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:14.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.988+0000 7f1384310b80 1 --2- 172.21.15.33:0/1228643944 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56103075f9b0 0x561030760e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:14.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.988+0000 7f1384310b80 1 -- 172.21.15.33:0/1228643944 >> 172.21.15.33:0/1228643944 conn(0x5610306d2750 msgr2=0x561030760770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:37:14.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.988+0000 7f1384310b80 1 -- 172.21.15.33:0/1228643944 shutdown_connections 2024-05-03T18:37:14.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.988+0000 7f1384310b80 1 -- 172.21.15.33:0/1228643944 wait complete. 2024-05-03T18:37:14.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.989+0000 7f1384310b80 1 -- start start 2024-05-03T18:37:14.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.990+0000 7f1384310b80 1 Processor -- start 2024-05-03T18:37:14.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.991+0000 7f1384310b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56103075a5d0 0x56103077ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:14.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.991+0000 7f1384310b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56103075f9b0 0x56103077b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:14.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.991+0000 7f1384310b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5610307602d0 0x5610307edea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:14.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.991+0000 7f1384310b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f13741b48b0 con 0x5610307602d0 2024-05-03T18:37:14.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.991+0000 7f1384310b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f13741b5d20 con 0x56103075f9b0 2024-05-03T18:37:14.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.991+0000 7f1384310b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f13741b4f80 con 0x56103075a5d0 2024-05-03T18:37:14.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.991+0000 7f1384310b80 10 client.0 _ll_register_callbacks cb 0x561030653cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:37:14.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.992+0000 7f1373400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5610307602d0 0x5610307edea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:14.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.992+0000 7f1372000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56103075f9b0 0x56103077b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:14.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.993+0000 7f1373400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5610307602d0 0x5610307edea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34772/0 (socket says 172.21.15.33:34772) 2024-05-03T18:37:14.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.993+0000 7f1373400700 1 -- 172.21.15.33:0/1841333693 learned_addr learned my addr 172.21.15.33:0/1841333693 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:37:14.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.993+0000 7f1373400700 1 -- 172.21.15.33:0/1841333693 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56103075a5d0 msgr2=0x56103077ae20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-03T18:37:14.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.993+0000 7f1373400700 1 --2- 172.21.15.33:0/1841333693 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56103075a5d0 0x56103077ae20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:14.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.993+0000 7f1373400700 1 -- 172.21.15.33:0/1841333693 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56103075f9b0 msgr2=0x56103077b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:37:14.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.994+0000 7f1373400700 1 --2- 172.21.15.33:0/1841333693 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56103075f9b0 0x56103077b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:14.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.994+0000 7f1373400700 1 -- 172.21.15.33:0/1841333693 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610307ee3e0 con 0x5610307602d0 2024-05-03T18:37:14.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.994+0000 7f1372000700 1 --2- 172.21.15.33:0/1841333693 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56103075f9b0 0x56103077b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:37:14.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.995+0000 7f1373400700 1 --2- 172.21.15.33:0/1841333693 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5610307602d0 0x5610307edea0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f136c0135f0 tx=0x7f136c020730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6a6a19ff0e356fa server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:14.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.995+0000 7f135cc00700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:37:14.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.996+0000 7f135cc00700 1 -- 172.21.15.33:0/1841333693 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f136c024040 con 0x5610307602d0 2024-05-03T18:37:14.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.996+0000 7f1384310b80 1 -- 172.21.15.33:0/1841333693 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f136c013b60 con 0x5610307602d0 2024-05-03T18:37:14.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.997+0000 7f135cc00700 1 -- 172.21.15.33:0/1841333693 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9606+0+0 (secure 0 0 0) 0x7f136c015ad0 con 0x5610307602d0 2024-05-03T18:37:14.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.998+0000 7f135cc00700 1 -- 172.21.15.33:0/1841333693 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f136c015cd0 con 0x5610307602d0 2024-05-03T18:37:14.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.998+0000 7f1384310b80 10 client.16383 fetch_fsmap learned FSMap version 21 2024-05-03T18:37:14.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:14.998+0000 7f1384310b80 1 -- 172.21.15.33:0/1841333693 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5610307ef510 con 0x5610307602d0 2024-05-03T18:37:15.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:15.000+0000 7f135cc00700 1 -- 172.21.15.33:0/1841333693 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f136c013b60 con 0x5610307602d0 2024-05-03T18:37:15.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:15.000+0000 7f135cc00700 1 -- 172.21.15.33:0/1841333693 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f136c029400 con 0x5610307602d0 2024-05-03T18:37:15.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:15.000+0000 7f1384310b80 10 client.16383 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:37:15.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:15.000+0000 7f1384310b80 10 client.16383 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:37:15.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:15.000+0000 7f1384310b80 20 client.16383 populate_metadata read hostname 'smithi033' 2024-05-03T18:37:15.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:15.000+0000 7f1384310b80 1 --2- 172.21.15.33:0/1841333693 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5610307efb00 0x5610307f1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:15.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:15.000+0000 7f1384310b80 4 client.16383 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-03T18:37:15.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:15.000+0000 7f1384310b80 1 -- 172.21.15.33:0/1841333693 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5610306ba480 con 0x5610307efb00 2024-05-03T18:37:15.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:15.001+0000 7f1372a00700 1 --2- 172.21.15.33:0/1841333693 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5610307efb00 0x5610307f1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:15.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:15.003+0000 7f1372a00700 1 --2- 172.21.15.33:0/1841333693 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5610307efb00 0x5610307f1f20 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a9ea5af20a572a9 server_cookie=444a71c4d5bdbcf2 in_seq=0 out_seq=0 2024-05-03T18:37:15.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:15.003+0000 7f135cc00700 10 client.16383 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:37:15.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:15.007+0000 7f135cc00700 1 -- 172.21.15.33:0/1841333693 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5610306ba480 con 0x5610307efb00 2024-05-03T18:37:15.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:15.007+0000 7f135cc00700 10 client.16383 handle_command_reply: tid=0 2024-05-03T18:37:15.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:15.007+0000 7f1384310b80 1 client.16383 shutdown 2024-05-03T18:37:15.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:15.008+0000 7f1384310b80 1 -- 172.21.15.33:0/1841333693 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5610307602d0 msgr2=0x5610307edea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:15.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:15.008+0000 7f1384310b80 1 --2- 172.21.15.33:0/1841333693 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5610307602d0 0x5610307edea0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f136c0135f0 tx=0x7f136c020730 comp rx=0 tx=0).stop 2024-05-03T18:37:15.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:15.010+0000 7f1384310b80 1 -- 172.21.15.33:0/1841333693 shutdown_connections 2024-05-03T18:37:15.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:15.010+0000 7f1384310b80 1 --2- 172.21.15.33:0/1841333693 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5610307efb00 0x5610307f1f20 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:15.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:15.010+0000 7f1384310b80 1 --2- 172.21.15.33:0/1841333693 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5610307602d0 0x5610307edea0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:15.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:15.010+0000 7f1384310b80 1 --2- 172.21.15.33:0/1841333693 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56103075a5d0 0x56103077ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:15.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:15.010+0000 7f1384310b80 1 --2- 172.21.15.33:0/1841333693 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56103075f9b0 0x56103077b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:15.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:15.010+0000 7f1384310b80 1 -- 172.21.15.33:0/1841333693 >> 172.21.15.33:0/1841333693 conn(0x5610306d2750 msgr2=0x56103074eba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:37:15.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:15.010+0000 7f135cc00700 0 client.16383 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:37:15.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:15.010+0000 7f1384310b80 1 -- 172.21.15.33:0/1841333693 shutdown_connections 2024-05-03T18:37:15.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:15.011+0000 7f1384310b80 1 -- 172.21.15.33:0/1841333693 wait complete. 2024-05-03T18:37:15.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:15.011+0000 7f1384310b80 20 client.16383 trim_cache size 0 max 16384 2024-05-03T18:37:15.014 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-03T18:37:15.014 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-05-03T18:37:15.014 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "f64c6d59-ac3d-4e0a-a9ec-9d72c8e507f5", 2024-05-03T18:37:15.014 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-05-03T18:37:15.014 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-03T18:37:15.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:15.014+0000 7f1379a00700 1 -- 172.21.15.33:0/490023949 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f13580b2c20 msgr2=0x7f13580b50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:15.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:15.014+0000 7f1379a00700 1 --2- 172.21.15.33:0/490023949 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f13580b2c20 0x7f13580b50e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1360005ea0 tx=0x7f1360005e30 comp rx=0 tx=0).stop 2024-05-03T18:37:15.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:15.014+0000 7f1379a00700 1 -- 172.21.15.33:0/490023949 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f137406b180 msgr2=0x7f13741a4c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:15.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:15.014+0000 7f1379a00700 1 --2- 172.21.15.33:0/490023949 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f137406b180 0x7f13741a4c60 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f136400b650 tx=0x7f136400bfc0 comp rx=0 tx=0).stop 2024-05-03T18:37:15.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:15.015+0000 7f1379a00700 1 -- 172.21.15.33:0/490023949 shutdown_connections 2024-05-03T18:37:15.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:15.015+0000 7f1379a00700 1 --2- 172.21.15.33:0/490023949 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f13580b2c20 0x7f13580b50e0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:15.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:15.015+0000 7f1379a00700 1 --2- 172.21.15.33:0/490023949 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1374109510 0x7f137410f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:15.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:15.015+0000 7f1379a00700 1 --2- 172.21.15.33:0/490023949 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f137406b180 0x7f13741a4c60 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:15.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:15.015+0000 7f1379a00700 1 --2- 172.21.15.33:0/490023949 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f137406bb40 0x7f13741a51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:15.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:15.015+0000 7f1379a00700 1 -- 172.21.15.33:0/490023949 >> 172.21.15.33:0/490023949 conn(0x7f1374079350 msgr2=0x7f1374100e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:37:15.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:15.016+0000 7f1379a00700 1 -- 172.21.15.33:0/490023949 shutdown_connections 2024-05-03T18:37:15.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:15.017+0000 7f1379a00700 1 -- 172.21.15.33:0/490023949 wait complete. 2024-05-03T18:37:15.110 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 360.0 s, 530.8 tps, lat 187.361 ms stddev 95.319 2024-05-03T18:37:15.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:37:15 smithi150 ceph-mon[32492]: scrub queued for path: / 2024-05-03T18:37:15.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:37:15 smithi150 ceph-mon[32492]: scrub summary: idle+waiting paths [/] 2024-05-03T18:37:15.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:37:15 smithi150 ceph-mon[32492]: scrub summary: active paths [/] 2024-05-03T18:37:15.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:37:15 smithi033 ceph-mon[28330]: scrub queued for path: / 2024-05-03T18:37:15.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:37:15 smithi033 ceph-mon[28330]: scrub summary: idle+waiting paths [/] 2024-05-03T18:37:15.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:37:15 smithi033 ceph-mon[28330]: scrub summary: active paths [/] 2024-05-03T18:37:15.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:37:15 smithi099 ceph-mon[31658]: scrub queued for path: / 2024-05-03T18:37:15.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:37:15 smithi099 ceph-mon[31658]: scrub summary: idle+waiting paths [/] 2024-05-03T18:37:15.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:37:15 smithi099 ceph-mon[31658]: scrub summary: active paths [/] 2024-05-03T18:37:16.573 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:37:16 smithi033 ceph-mon[28330]: scrub summary: idle+waiting paths [/] 2024-05-03T18:37:16.573 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:37:16 smithi033 ceph-mon[28330]: pgmap v312: 129 pgs: 129 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 5.3 MiB/s wr, 576 op/s 2024-05-03T18:37:16.573 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:37:16 smithi033 ceph-mon[28330]: scrub summary: idle 2024-05-03T18:37:16.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:37:16 smithi150 ceph-mon[32492]: scrub summary: idle+waiting paths [/] 2024-05-03T18:37:16.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:37:16 smithi150 ceph-mon[32492]: pgmap v312: 129 pgs: 129 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 5.3 MiB/s wr, 576 op/s 2024-05-03T18:37:16.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:37:16 smithi150 ceph-mon[32492]: scrub summary: idle 2024-05-03T18:37:16.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:37:16 smithi099 ceph-mon[31658]: scrub summary: idle+waiting paths [/] 2024-05-03T18:37:16.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:37:16 smithi099 ceph-mon[31658]: pgmap v312: 129 pgs: 129 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 5.3 MiB/s wr, 576 op/s 2024-05-03T18:37:16.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:37:16 smithi099 ceph-mon[31658]: scrub summary: idle 2024-05-03T18:37:17.675 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-03T18:37:18.404 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:37:18.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:37:18 smithi150 ceph-mon[32492]: osdmap e98: 12 total, 12 up, 12 in 2024-05-03T18:37:18.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:37:18 smithi150 ceph-mon[32492]: pgmap v314: 129 pgs: 129 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 3.2 MiB/s wr, 359 op/s 2024-05-03T18:37:18.713 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:37:18 smithi033 ceph-mon[28330]: osdmap e98: 12 total, 12 up, 12 in 2024-05-03T18:37:18.713 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:37:18 smithi033 ceph-mon[28330]: pgmap v314: 129 pgs: 129 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 3.2 MiB/s wr, 359 op/s 2024-05-03T18:37:18.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:37:18 smithi099 ceph-mon[31658]: osdmap e98: 12 total, 12 up, 12 in 2024-05-03T18:37:18.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:37:18 smithi099 ceph-mon[31658]: pgmap v314: 129 pgs: 129 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 3.2 MiB/s wr, 359 op/s 2024-05-03T18:37:20.110 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 365.0 s, 539.4 tps, lat 185.293 ms stddev 79.084 2024-05-03T18:37:20.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:37:20 smithi150 ceph-mon[32492]: pgmap v315: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 5.6 MiB/s wr, 615 op/s 2024-05-03T18:37:20.398 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:37:20 smithi033 ceph-mon[28330]: pgmap v315: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 5.6 MiB/s wr, 615 op/s 2024-05-03T18:37:20.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:37:20 smithi099 ceph-mon[31658]: pgmap v315: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 5.6 MiB/s wr, 615 op/s 2024-05-03T18:37:22.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:37:22 smithi033 ceph-mon[28330]: pgmap v316: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 5.0 MiB/s wr, 544 op/s 2024-05-03T18:37:22.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:37:22 smithi099 ceph-mon[31658]: pgmap v316: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 5.0 MiB/s wr, 544 op/s 2024-05-03T18:37:22.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:37:22 smithi150 ceph-mon[32492]: pgmap v316: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 5.0 MiB/s wr, 544 op/s 2024-05-03T18:37:23.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.060+0000 7f77e0600700 1 -- 172.21.15.33:0/1243275260 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f77d8102d60 msgr2=0x7f77d810f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:23.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.060+0000 7f77e0600700 1 --2- 172.21.15.33:0/1243275260 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f77d8102d60 0x7f77d810f250 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f77d000b0a0 tx=0x7f77d000b3b0 comp rx=0 tx=0).stop 2024-05-03T18:37:23.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.061+0000 7f77e0600700 1 -- 172.21.15.33:0/1243275260 shutdown_connections 2024-05-03T18:37:23.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.061+0000 7f77e0600700 1 --2- 172.21.15.33:0/1243275260 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f77d8102d60 0x7f77d810f250 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:23.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.061+0000 7f77e0600700 1 --2- 172.21.15.33:0/1243275260 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77d8108410 0x7f77d81087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:23.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.061+0000 7f77e0600700 1 --2- 172.21.15.33:0/1243275260 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77d81023a0 0x7f77d8102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:23.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.061+0000 7f77e0600700 1 -- 172.21.15.33:0/1243275260 >> 172.21.15.33:0/1243275260 conn(0x7f77d8079350 msgr2=0x7f77d8079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:37:23.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.062+0000 7f77e0600700 1 -- 172.21.15.33:0/1243275260 shutdown_connections 2024-05-03T18:37:23.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.062+0000 7f77e0600700 1 -- 172.21.15.33:0/1243275260 wait complete. 2024-05-03T18:37:23.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.063+0000 7f77e0600700 1 Processor -- start 2024-05-03T18:37:23.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.064+0000 7f77e0600700 1 -- start start 2024-05-03T18:37:23.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.065+0000 7f77e0600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77d81023a0 0x7f77d819d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:23.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.065+0000 7f77e0600700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77d8102d60 0x7f77d819dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:23.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.065+0000 7f77e0600700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f77d8108410 0x7f77d8197820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:23.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.065+0000 7f77e0600700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f77d8111440 con 0x7f77d8102d60 2024-05-03T18:37:23.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.065+0000 7f77e0600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f77d81112c0 con 0x7f77d81023a0 2024-05-03T18:37:23.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.065+0000 7f77e0600700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f77d81115c0 con 0x7f77d8108410 2024-05-03T18:37:23.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.066+0000 7f77de600700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f77d8108410 0x7f77d8197820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:23.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.066+0000 7f77de600700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f77d8108410 0x7f77d8197820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:37866/0 (socket says 172.21.15.33:37866) 2024-05-03T18:37:23.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.066+0000 7f77de600700 1 -- 172.21.15.33:0/2453709976 learned_addr learned my addr 172.21.15.33:0/2453709976 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:37:23.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.067+0000 7f77de600700 1 -- 172.21.15.33:0/2453709976 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77d81023a0 msgr2=0x7f77d819d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:23.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.067+0000 7f77de600700 1 --2- 172.21.15.33:0/2453709976 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77d81023a0 0x7f77d819d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:23.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.067+0000 7f77de600700 1 -- 172.21.15.33:0/2453709976 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77d8102d60 msgr2=0x7f77d819dbe0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-03T18:37:23.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.067+0000 7f77de600700 1 --2- 172.21.15.33:0/2453709976 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77d8102d60 0x7f77d819dbe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:23.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.067+0000 7f77de600700 1 -- 172.21.15.33:0/2453709976 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77d8198000 con 0x7f77d8108410 2024-05-03T18:37:23.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.068+0000 7f77de600700 1 --2- 172.21.15.33:0/2453709976 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f77d8108410 0x7f77d8197820 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f77d000b070 tx=0x7f77d000cf20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:23.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.068+0000 7f77d6000700 1 -- 172.21.15.33:0/2453709976 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d0021030 con 0x7f77d8108410 2024-05-03T18:37:23.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.068+0000 7f77d6000700 1 -- 172.21.15.33:0/2453709976 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f77d0007bc0 con 0x7f77d8108410 2024-05-03T18:37:23.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.068+0000 7f77d6000700 1 -- 172.21.15.33:0/2453709976 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d00046e0 con 0x7f77d8108410 2024-05-03T18:37:23.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.068+0000 7f77e0600700 1 -- 172.21.15.33:0/2453709976 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77d8198290 con 0x7f77d8108410 2024-05-03T18:37:23.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.069+0000 7f77e0600700 1 -- 172.21.15.33:0/2453709976 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77d806cec0 con 0x7f77d8108410 2024-05-03T18:37:23.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.074+0000 7f77d6000700 1 -- 172.21.15.33:0/2453709976 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f77d0024070 con 0x7f77d8108410 2024-05-03T18:37:23.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.076+0000 7f77eafd4b80 1 Processor -- start 2024-05-03T18:37:23.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.076+0000 7f77eafd4b80 1 -- start start 2024-05-03T18:37:23.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.076+0000 7f77eafd4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609dd2367b0 0x5609dd236b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:23.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.077+0000 7f77d6000700 1 --2- 172.21.15.33:0/2453709976 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f77c00b6f60 0x7f77c00b9420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:23.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.078+0000 7f77d6000700 1 -- 172.21.15.33:0/2453709976 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9518+0+0 (secure 0 0 0) 0x7f77d000f020 con 0x7f77d8108410 2024-05-03T18:37:23.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.079+0000 7f77ddc00700 1 --2- 172.21.15.33:0/2453709976 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f77c00b6f60 0x7f77c00b9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:23.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.079+0000 7f77eafd4b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5609dd2370d0 0x5609dd231fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:23.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.080+0000 7f77eafd4b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609dd2325d0 0x5609dd232a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:23.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.080+0000 7f77eafd4b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5609dd184a20 con 0x5609dd2370d0 2024-05-03T18:37:23.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.080+0000 7f77eafd4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5609dd122760 con 0x5609dd2367b0 2024-05-03T18:37:23.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.080+0000 7f77eafd4b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5609dd1809e0 con 0x5609dd2325d0 2024-05-03T18:37:23.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.080+0000 7f77de600700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609dd2325d0 0x5609dd232a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:23.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.080+0000 7f77de600700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609dd2325d0 0x5609dd232a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:37870/0 (socket says 172.21.15.33:37870) 2024-05-03T18:37:23.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.080+0000 7f77de600700 1 -- 172.21.15.33:0/3841486895 learned_addr learned my addr 172.21.15.33:0/3841486895 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:37:23.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.081+0000 7f77dd200700 1 --2- 172.21.15.33:0/3841486895 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609dd2367b0 0x5609dd236b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:23.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.082+0000 7f77de600700 1 -- 172.21.15.33:0/3841486895 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609dd2367b0 msgr2=0x5609dd236b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:37:23.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.082+0000 7f77de600700 1 --2- 172.21.15.33:0/3841486895 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609dd2367b0 0x5609dd236b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:23.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.082+0000 7f77de600700 1 -- 172.21.15.33:0/3841486895 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5609dd2370d0 msgr2=0x5609dd231fb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-03T18:37:23.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.082+0000 7f77de600700 1 --2- 172.21.15.33:0/3841486895 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5609dd2370d0 0x5609dd231fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:23.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.082+0000 7f77de600700 1 -- 172.21.15.33:0/3841486895 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609dd0d18d0 con 0x5609dd2325d0 2024-05-03T18:37:23.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.083+0000 7f77ddc00700 1 --2- 172.21.15.33:0/2453709976 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f77c00b6f60 0x7f77c00b9420 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f77cc0097b0 tx=0x7f77cc006d20 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:23.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.083+0000 7f77de600700 1 --2- 172.21.15.33:0/3841486895 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609dd2325d0 0x5609dd232a30 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f77d00e0c90 tx=0x7f77d00aa750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4731d88f163b15f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:23.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.084+0000 7f77caa00700 1 -- 172.21.15.33:0/3841486895 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d0004880 con 0x5609dd2325d0 2024-05-03T18:37:23.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.085+0000 7f77caa00700 1 -- 172.21.15.33:0/3841486895 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f77d00aa360 con 0x5609dd2325d0 2024-05-03T18:37:23.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.085+0000 7f77caa00700 1 -- 172.21.15.33:0/3841486895 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d00074e0 con 0x5609dd2325d0 2024-05-03T18:37:23.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.086+0000 7f77eafd4b80 1 -- 172.21.15.33:0/3841486895 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609dd2325d0 msgr2=0x5609dd232a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:23.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.086+0000 7f77eafd4b80 1 --2- 172.21.15.33:0/3841486895 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609dd2325d0 0x5609dd232a30 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f77d00e0c90 tx=0x7f77d00aa750 comp rx=0 tx=0).stop 2024-05-03T18:37:23.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.087+0000 7f77eafd4b80 1 -- 172.21.15.33:0/3841486895 shutdown_connections 2024-05-03T18:37:23.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.087+0000 7f77eafd4b80 1 --2- 172.21.15.33:0/3841486895 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609dd2325d0 0x5609dd232a30 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:23.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.087+0000 7f77eafd4b80 1 --2- 172.21.15.33:0/3841486895 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609dd2367b0 0x5609dd236b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:23.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.087+0000 7f77eafd4b80 1 --2- 172.21.15.33:0/3841486895 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5609dd2370d0 0x5609dd231fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:23.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.087+0000 7f77eafd4b80 1 -- 172.21.15.33:0/3841486895 >> 172.21.15.33:0/3841486895 conn(0x5609dd160b60 msgr2=0x5609dd22c7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:37:23.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.088+0000 7f77eafd4b80 1 -- 172.21.15.33:0/3841486895 shutdown_connections 2024-05-03T18:37:23.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.088+0000 7f77eafd4b80 1 -- 172.21.15.33:0/3841486895 wait complete. 2024-05-03T18:37:23.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.089+0000 7f77eafd4b80 1 -- start start 2024-05-03T18:37:23.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.089+0000 7f77eafd4b80 1 Processor -- start 2024-05-03T18:37:23.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.090+0000 7f77eafd4b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5609dd2325d0 0x5609dd23de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:23.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.090+0000 7f77eafd4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609dd2367b0 0x5609dd23b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:23.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.090+0000 7f77eafd4b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609dd2370d0 0x5609dd23b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:23.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.090+0000 7f77eafd4b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5609dd23ea60 con 0x5609dd2325d0 2024-05-03T18:37:23.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.090+0000 7f77eafd4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5609dd0eab20 con 0x5609dd2367b0 2024-05-03T18:37:23.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.090+0000 7f77eafd4b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5609dd23ebe0 con 0x5609dd2370d0 2024-05-03T18:37:23.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.090+0000 7f77eafd4b80 10 client.0 _ll_register_callbacks cb 0x5609dd12bc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:37:23.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.092+0000 7f77dd200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609dd2367b0 0x5609dd23b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:23.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.092+0000 7f77ddc00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5609dd2325d0 0x5609dd23de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:23.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.092+0000 7f77dd200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609dd2367b0 0x5609dd23b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:40288/0 (socket says 172.21.15.33:40288) 2024-05-03T18:37:23.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.092+0000 7f77dd200700 1 -- 172.21.15.33:0/3587141546 learned_addr learned my addr 172.21.15.33:0/3587141546 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:37:23.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.092+0000 7f77de600700 1 --2- 172.21.15.33:0/3587141546 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609dd2370d0 0x5609dd23b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:23.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.093+0000 7f77dd200700 1 -- 172.21.15.33:0/3587141546 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609dd2370d0 msgr2=0x5609dd23b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:37:23.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.093+0000 7f77dd200700 1 --2- 172.21.15.33:0/3587141546 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609dd2370d0 0x5609dd23b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:23.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.093+0000 7f77dd200700 1 -- 172.21.15.33:0/3587141546 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5609dd2325d0 msgr2=0x5609dd23de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:37:23.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.093+0000 7f77dd200700 1 --2- 172.21.15.33:0/3587141546 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5609dd2325d0 0x5609dd23de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:23.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.093+0000 7f77dd200700 1 -- 172.21.15.33:0/3587141546 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609dd14a420 con 0x5609dd2367b0 2024-05-03T18:37:23.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.094+0000 7f77ddc00700 1 --2- 172.21.15.33:0/3587141546 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5609dd2325d0 0x5609dd23de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:37:23.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.095+0000 7f77dd200700 1 --2- 172.21.15.33:0/3587141546 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609dd2367b0 0x5609dd23b290 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f77c400c180 tx=0x7f77c400c540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=834f1a55ef59af72 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:23.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.095+0000 7f77bf400700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-03T18:37:23.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.095+0000 7f77bf400700 1 -- 172.21.15.33:0/3587141546 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c4015070 con 0x5609dd2367b0 2024-05-03T18:37:23.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.095+0000 7f77bf400700 1 -- 172.21.15.33:0/3587141546 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9518+0+0 (secure 0 0 0) 0x7f77c4004220 con 0x5609dd2367b0 2024-05-03T18:37:23.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.096+0000 7f77eafd4b80 1 -- 172.21.15.33:0/3587141546 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77d00aa360 con 0x5609dd2367b0 2024-05-03T18:37:23.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.099+0000 7f77bf400700 1 -- 172.21.15.33:0/3587141546 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f77c401e050 con 0x5609dd2367b0 2024-05-03T18:37:23.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.099+0000 7f77eafd4b80 10 client.36038 fetch_fsmap learned FSMap version 21 2024-05-03T18:37:23.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.099+0000 7f77eafd4b80 1 -- 172.21.15.33:0/3587141546 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5609dd249290 con 0x5609dd2367b0 2024-05-03T18:37:23.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.100+0000 7f77bf400700 1 -- 172.21.15.33:0/3587141546 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f77d00aa360 con 0x5609dd2367b0 2024-05-03T18:37:23.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.100+0000 7f77bf400700 1 -- 172.21.15.33:0/3587141546 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f77c40193f0 con 0x5609dd2367b0 2024-05-03T18:37:23.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.100+0000 7f77eafd4b80 10 client.36038 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:37:23.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.100+0000 7f77eafd4b80 10 client.36038 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:37:23.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.100+0000 7f77eafd4b80 20 client.36038 populate_metadata read hostname 'smithi033' 2024-05-03T18:37:23.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.100+0000 7f77eafd4b80 1 --2- 172.21.15.33:0/3587141546 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5609dd24b8d0 0x5609dd24dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:23.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.100+0000 7f77eafd4b80 4 client.36038 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:37:23.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.100+0000 7f77eafd4b80 1 -- 172.21.15.33:0/3587141546 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5609dd1ff1d0 con 0x5609dd24b8d0 2024-05-03T18:37:23.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.101+0000 7f77ddc00700 1 --2- 172.21.15.33:0/3587141546 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5609dd24b8d0 0x5609dd24dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:23.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.104+0000 7f77ddc00700 1 --2- 172.21.15.33:0/3587141546 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5609dd24b8d0 0x5609dd24dcc0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=188c470960e6c014 server_cookie=cd030ee8d386ed22 in_seq=0 out_seq=0 2024-05-03T18:37:23.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.104+0000 7f77bf400700 10 client.36038 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:37:23.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.106+0000 7f77bf400700 1 -- 172.21.15.33:0/3587141546 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5609dd1ff1d0 con 0x5609dd24b8d0 2024-05-03T18:37:23.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.106+0000 7f77bf400700 10 client.36038 handle_command_reply: tid=0 2024-05-03T18:37:23.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.106+0000 7f77eafd4b80 1 client.36038 shutdown 2024-05-03T18:37:23.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.106+0000 7f77eafd4b80 1 -- 172.21.15.33:0/3587141546 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609dd2367b0 msgr2=0x5609dd23b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:23.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.106+0000 7f77eafd4b80 1 --2- 172.21.15.33:0/3587141546 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609dd2367b0 0x5609dd23b290 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f77c400c180 tx=0x7f77c400c540 comp rx=0 tx=0).stop 2024-05-03T18:37:23.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.106+0000 7f77eafd4b80 1 -- 172.21.15.33:0/3587141546 shutdown_connections 2024-05-03T18:37:23.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.106+0000 7f77eafd4b80 1 --2- 172.21.15.33:0/3587141546 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5609dd24b8d0 0x5609dd24dcc0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:23.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.106+0000 7f77eafd4b80 1 --2- 172.21.15.33:0/3587141546 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609dd2370d0 0x5609dd23b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:23.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.106+0000 7f77eafd4b80 1 --2- 172.21.15.33:0/3587141546 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5609dd2325d0 0x5609dd23de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:23.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.106+0000 7f77eafd4b80 1 --2- 172.21.15.33:0/3587141546 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609dd2367b0 0x5609dd23b290 secure :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f77c400c180 tx=0x7f77c400c540 comp rx=0 tx=0).stop 2024-05-03T18:37:23.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.106+0000 7f77eafd4b80 1 -- 172.21.15.33:0/3587141546 >> 172.21.15.33:0/3587141546 conn(0x5609dd160b60 msgr2=0x5609dd237e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:37:23.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.107+0000 7f77bf400700 0 client.36038 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:37:23.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.107+0000 7f77eafd4b80 1 -- 172.21.15.33:0/3587141546 shutdown_connections 2024-05-03T18:37:23.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.107+0000 7f77eafd4b80 1 -- 172.21.15.33:0/3587141546 wait complete. 2024-05-03T18:37:23.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.107+0000 7f77eafd4b80 20 client.36038 trim_cache size 0 max 16384 2024-05-03T18:37:23.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.145+0000 7f77eafd4b80 1 Processor -- start 2024-05-03T18:37:23.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.145+0000 7f77eafd4b80 1 -- start start 2024-05-03T18:37:23.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.146+0000 7f77eafd4b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609dd2325d0 0x5609dd253590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:23.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.146+0000 7f77eafd4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609dd2370d0 0x5609dd253ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:23.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.146+0000 7f77eafd4b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5609dd23a290 0x5609dd22ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:23.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.147+0000 7f77eafd4b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f77b800ce00 con 0x5609dd23a290 2024-05-03T18:37:23.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.147+0000 7f77eafd4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f77b800f660 con 0x5609dd2370d0 2024-05-03T18:37:23.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.147+0000 7f77eafd4b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f77b800d450 con 0x5609dd2325d0 2024-05-03T18:37:23.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.147+0000 7f77dd200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609dd2370d0 0x5609dd253ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:23.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.147+0000 7f77ddc00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609dd2325d0 0x5609dd253590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:23.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.147+0000 7f77de600700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5609dd23a290 0x5609dd22ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:23.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.147+0000 7f77de600700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5609dd23a290 0x5609dd22ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50420/0 (socket says 172.21.15.33:50420) 2024-05-03T18:37:23.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.147+0000 7f77dd200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609dd2370d0 0x5609dd253ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:40294/0 (socket says 172.21.15.33:40294) 2024-05-03T18:37:23.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.147+0000 7f77de600700 1 -- 172.21.15.33:0/3296031891 learned_addr learned my addr 172.21.15.33:0/3296031891 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:37:23.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.148+0000 7f77de600700 1 -- 172.21.15.33:0/3296031891 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609dd2325d0 msgr2=0x5609dd253590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:37:23.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.148+0000 7f77de600700 1 --2- 172.21.15.33:0/3296031891 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609dd2325d0 0x5609dd253590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:23.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.148+0000 7f77de600700 1 -- 172.21.15.33:0/3296031891 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609dd2370d0 msgr2=0x5609dd253ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:37:23.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.148+0000 7f77de600700 1 --2- 172.21.15.33:0/3296031891 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609dd2370d0 0x5609dd253ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:23.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.148+0000 7f77de600700 1 -- 172.21.15.33:0/3296031891 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609dd247ab0 con 0x5609dd23a290 2024-05-03T18:37:23.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.148+0000 7f77de600700 1 --2- 172.21.15.33:0/3296031891 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5609dd23a290 0x5609dd22ffa0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f77d00e1230 tx=0x7f77d000b830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=841bdb17712cae59 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:23.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.149+0000 7f77ca000700 1 -- 172.21.15.33:0/3296031891 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d00aa850 con 0x5609dd23a290 2024-05-03T18:37:23.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.149+0000 7f77ca000700 1 -- 172.21.15.33:0/3296031891 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f77d00aec00 con 0x5609dd23a290 2024-05-03T18:37:23.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.149+0000 7f77ca000700 1 -- 172.21.15.33:0/3296031891 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d00a6ae0 con 0x5609dd23a290 2024-05-03T18:37:23.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.150+0000 7f77eafd4b80 1 -- 172.21.15.33:0/3296031891 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5609dd23a290 msgr2=0x5609dd22ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:23.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.150+0000 7f77eafd4b80 1 --2- 172.21.15.33:0/3296031891 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5609dd23a290 0x5609dd22ffa0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f77d00e1230 tx=0x7f77d000b830 comp rx=0 tx=0).stop 2024-05-03T18:37:23.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.150+0000 7f77eafd4b80 1 -- 172.21.15.33:0/3296031891 shutdown_connections 2024-05-03T18:37:23.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.150+0000 7f77eafd4b80 1 --2- 172.21.15.33:0/3296031891 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5609dd23a290 0x5609dd22ffa0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:23.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.150+0000 7f77eafd4b80 1 --2- 172.21.15.33:0/3296031891 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609dd2325d0 0x5609dd253590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:23.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.150+0000 7f77eafd4b80 1 --2- 172.21.15.33:0/3296031891 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609dd2370d0 0x5609dd253ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:23.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.150+0000 7f77eafd4b80 1 -- 172.21.15.33:0/3296031891 >> 172.21.15.33:0/3296031891 conn(0x5609dd160b60 msgr2=0x5609dd239b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:37:23.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.150+0000 7f77eafd4b80 1 -- 172.21.15.33:0/3296031891 shutdown_connections 2024-05-03T18:37:23.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.150+0000 7f77eafd4b80 1 -- 172.21.15.33:0/3296031891 wait complete. 2024-05-03T18:37:23.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.151+0000 7f77eafd4b80 1 -- start start 2024-05-03T18:37:23.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.151+0000 7f77eafd4b80 1 Processor -- start 2024-05-03T18:37:23.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.152+0000 7f77eafd4b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5609dd2325d0 0x5609dd23edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:23.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.152+0000 7f77eafd4b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609dd2370d0 0x5609dd23f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:23.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.152+0000 7f77eafd4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609dd23a290 0x5609dd23f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:23.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.152+0000 7f77eafd4b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f77b800db20 con 0x5609dd2325d0 2024-05-03T18:37:23.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.152+0000 7f77eafd4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f77b800ef90 con 0x5609dd23a290 2024-05-03T18:37:23.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.152+0000 7f77eafd4b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f77b800e1f0 con 0x5609dd2370d0 2024-05-03T18:37:23.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.152+0000 7f77eafd4b80 10 client.0 _ll_register_callbacks cb 0x5609dd12bc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:37:23.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.152+0000 7f77dd200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609dd2370d0 0x5609dd23f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:23.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.152+0000 7f77dd200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609dd2370d0 0x5609dd23f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:37896/0 (socket says 172.21.15.33:37896) 2024-05-03T18:37:23.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.152+0000 7f77dd200700 1 -- 172.21.15.33:0/3548719087 learned_addr learned my addr 172.21.15.33:0/3548719087 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:37:23.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.152+0000 7f77de600700 1 --2- 172.21.15.33:0/3548719087 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609dd23a290 0x5609dd23f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:23.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.153+0000 7f77dd200700 1 -- 172.21.15.33:0/3548719087 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609dd23a290 msgr2=0x5609dd23f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:37:23.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.153+0000 7f77dd200700 1 --2- 172.21.15.33:0/3548719087 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609dd23a290 0x5609dd23f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:23.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.153+0000 7f77dd200700 1 -- 172.21.15.33:0/3548719087 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5609dd2325d0 msgr2=0x5609dd23edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:37:23.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.153+0000 7f77dd200700 1 --2- 172.21.15.33:0/3548719087 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5609dd2325d0 0x5609dd23edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:23.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.153+0000 7f77dd200700 1 -- 172.21.15.33:0/3548719087 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609dd2bea50 con 0x5609dd2370d0 2024-05-03T18:37:23.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.153+0000 7f77dd200700 1 --2- 172.21.15.33:0/3548719087 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609dd2370d0 0x5609dd23f330 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f77c4000c00 tx=0x7f77c4006db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a537daa25708cf7d server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:23.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.154+0000 7f77bf400700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-03T18:37:23.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.154+0000 7f77bf400700 1 -- 172.21.15.33:0/3548719087 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c4015040 con 0x5609dd2370d0 2024-05-03T18:37:23.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.154+0000 7f77bf400700 1 -- 172.21.15.33:0/3548719087 <== mon.1 v2:172.21.15.150:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9518+0+0 (secure 0 0 0) 0x7f77c4004450 con 0x5609dd2370d0 2024-05-03T18:37:23.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.154+0000 7f77eafd4b80 1 -- 172.21.15.33:0/3548719087 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5609dd192400 con 0x5609dd2370d0 2024-05-03T18:37:23.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.155+0000 7f77bf400700 1 -- 172.21.15.33:0/3548719087 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f77c4004670 con 0x5609dd2370d0 2024-05-03T18:37:23.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.155+0000 7f77eafd4b80 10 client.26122 fetch_fsmap learned FSMap version 21 2024-05-03T18:37:23.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.155+0000 7f77eafd4b80 1 -- 172.21.15.33:0/3548719087 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5609dd2bfd60 con 0x5609dd2370d0 2024-05-03T18:37:23.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.155+0000 7f77bf400700 1 -- 172.21.15.33:0/3548719087 <== mon.1 v2:172.21.15.150:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x5609dd192400 con 0x5609dd2370d0 2024-05-03T18:37:23.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.155+0000 7f77bf400700 1 -- 172.21.15.33:0/3548719087 <== mon.1 v2:172.21.15.150:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f77c402a070 con 0x5609dd2370d0 2024-05-03T18:37:23.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.156+0000 7f77eafd4b80 10 client.26122 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:37:23.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.156+0000 7f77eafd4b80 10 client.26122 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:37:23.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.156+0000 7f77eafd4b80 20 client.26122 populate_metadata read hostname 'smithi033' 2024-05-03T18:37:23.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.156+0000 7f77eafd4b80 1 --2- 172.21.15.33:0/3548719087 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5609dd2c0470 0x5609dd2c2890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:23.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.156+0000 7f77eafd4b80 4 client.26122 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-03T18:37:23.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.156+0000 7f77eafd4b80 1 -- 172.21.15.33:0/3548719087 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5609dd18c4a0 con 0x5609dd2c0470 2024-05-03T18:37:23.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.157+0000 7f77ddc00700 1 --2- 172.21.15.33:0/3548719087 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5609dd2c0470 0x5609dd2c2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:23.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.158+0000 7f77bf400700 10 client.26122 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:37:23.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.158+0000 7f77ddc00700 1 --2- 172.21.15.33:0/3548719087 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5609dd2c0470 0x5609dd2c2890 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29b203e1a57231c0 server_cookie=dcaa0a21cc59c6aa in_seq=0 out_seq=0 2024-05-03T18:37:23.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.161+0000 7f77bf400700 1 -- 172.21.15.33:0/3548719087 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5609dd18c4a0 con 0x5609dd2c0470 2024-05-03T18:37:23.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.161+0000 7f77bf400700 10 client.26122 handle_command_reply: tid=0 2024-05-03T18:37:23.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.162+0000 7f77eafd4b80 1 client.26122 shutdown 2024-05-03T18:37:23.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.162+0000 7f77eafd4b80 1 -- 172.21.15.33:0/3548719087 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609dd2370d0 msgr2=0x5609dd23f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:23.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.162+0000 7f77eafd4b80 1 --2- 172.21.15.33:0/3548719087 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609dd2370d0 0x5609dd23f330 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f77c4000c00 tx=0x7f77c4006db0 comp rx=0 tx=0).stop 2024-05-03T18:37:23.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.162+0000 7f77eafd4b80 1 -- 172.21.15.33:0/3548719087 shutdown_connections 2024-05-03T18:37:23.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.163+0000 7f77eafd4b80 1 --2- 172.21.15.33:0/3548719087 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5609dd2c0470 0x5609dd2c2890 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:23.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.163+0000 7f77eafd4b80 1 --2- 172.21.15.33:0/3548719087 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609dd23a290 0x5609dd23f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:23.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.163+0000 7f77eafd4b80 1 --2- 172.21.15.33:0/3548719087 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5609dd2325d0 0x5609dd23edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:23.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.163+0000 7f77eafd4b80 1 --2- 172.21.15.33:0/3548719087 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609dd2370d0 0x5609dd23f330 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:23.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.163+0000 7f77bf400700 0 client.26122 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:37:23.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.163+0000 7f77eafd4b80 1 -- 172.21.15.33:0/3548719087 >> 172.21.15.33:0/3548719087 conn(0x5609dd160b60 msgr2=0x5609dd23b6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:37:23.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.163+0000 7f77eafd4b80 1 -- 172.21.15.33:0/3548719087 shutdown_connections 2024-05-03T18:37:23.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.164+0000 7f77eafd4b80 1 -- 172.21.15.33:0/3548719087 wait complete. 2024-05-03T18:37:23.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.165+0000 7f77eafd4b80 20 client.26122 trim_cache size 0 max 16384 2024-05-03T18:37:23.171 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-03T18:37:23.171 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-05-03T18:37:23.171 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-05-03T18:37:23.171 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-03T18:37:23.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.170+0000 7f77e0600700 1 -- 172.21.15.33:0/2453709976 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f77c00b6f60 msgr2=0x7f77c00b9420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:23.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.170+0000 7f77e0600700 1 --2- 172.21.15.33:0/2453709976 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f77c00b6f60 0x7f77c00b9420 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f77cc0097b0 tx=0x7f77cc006d20 comp rx=0 tx=0).stop 2024-05-03T18:37:23.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.170+0000 7f77e0600700 1 -- 172.21.15.33:0/2453709976 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f77d8108410 msgr2=0x7f77d8197820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:23.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.170+0000 7f77e0600700 1 --2- 172.21.15.33:0/2453709976 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f77d8108410 0x7f77d8197820 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f77d000b070 tx=0x7f77d000cf20 comp rx=0 tx=0).stop 2024-05-03T18:37:23.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.171+0000 7f77e0600700 1 -- 172.21.15.33:0/2453709976 shutdown_connections 2024-05-03T18:37:23.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.171+0000 7f77e0600700 1 --2- 172.21.15.33:0/2453709976 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f77d8108410 0x7f77d8197820 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:23.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.171+0000 7f77e0600700 1 --2- 172.21.15.33:0/2453709976 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f77c00b6f60 0x7f77c00b9420 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:23.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.171+0000 7f77e0600700 1 --2- 172.21.15.33:0/2453709976 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77d81023a0 0x7f77d819d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:23.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.171+0000 7f77e0600700 1 --2- 172.21.15.33:0/2453709976 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77d8102d60 0x7f77d819dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:23.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.171+0000 7f77e0600700 1 -- 172.21.15.33:0/2453709976 >> 172.21.15.33:0/2453709976 conn(0x7f77d8079350 msgr2=0x7f77d81010d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:37:23.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.171+0000 7f77e0600700 1 -- 172.21.15.33:0/2453709976 shutdown_connections 2024-05-03T18:37:23.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:23.171+0000 7f77e0600700 1 -- 172.21.15.33:0/2453709976 wait complete. 2024-05-03T18:37:24.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:37:24 smithi150 ceph-mon[32492]: pgmap v317: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 5.0 MiB/s wr, 545 op/s 2024-05-03T18:37:24.468 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:37:24 smithi033 ceph-mon[28330]: pgmap v317: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 5.0 MiB/s wr, 545 op/s 2024-05-03T18:37:24.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:37:24 smithi099 ceph-mon[31658]: pgmap v317: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 5.0 MiB/s wr, 545 op/s 2024-05-03T18:37:24.558 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-03T18:37:24.558 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T18:37:25.112 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 370.0 s, 553.8 tps, lat 181.227 ms stddev 79.766 2024-05-03T18:37:25.555 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:37:26.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:37:26 smithi150 ceph-mon[32492]: pgmap v318: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 5.0 MiB/s wr, 545 op/s 2024-05-03T18:37:26.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:37:26 smithi033 ceph-mon[28330]: pgmap v318: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 5.0 MiB/s wr, 545 op/s 2024-05-03T18:37:26.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:37:26 smithi099 ceph-mon[31658]: pgmap v318: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 5.0 MiB/s wr, 545 op/s 2024-05-03T18:37:28.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:37:28 smithi033 ceph-mon[28330]: pgmap v319: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 4.7 MiB/s wr, 511 op/s 2024-05-03T18:37:28.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:37:28 smithi150 ceph-mon[32492]: pgmap v319: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 4.7 MiB/s wr, 511 op/s 2024-05-03T18:37:28.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:37:28 smithi099 ceph-mon[31658]: pgmap v319: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 4.7 MiB/s wr, 511 op/s 2024-05-03T18:37:30.110 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 375.0 s, 550.8 tps, lat 182.427 ms stddev 66.681 2024-05-03T18:37:30.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:37:30 smithi150 ceph-mon[32492]: pgmap v320: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 6.3 MiB/s wr, 682 op/s 2024-05-03T18:37:30.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:37:30 smithi033 ceph-mon[28330]: pgmap v320: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 6.3 MiB/s wr, 682 op/s 2024-05-03T18:37:30.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:30.480+0000 7f8283a00700 1 -- 172.21.15.33:0/1117744624 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f827c10fb60 msgr2=0x7f827c10ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:30.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:30.480+0000 7f8283a00700 1 --2- 172.21.15.33:0/1117744624 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f827c10fb60 0x7f827c10ff40 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f826c00b0a0 tx=0x7f826c00b3b0 comp rx=0 tx=0).stop 2024-05-03T18:37:30.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:30.480+0000 7f8283a00700 1 -- 172.21.15.33:0/1117744624 shutdown_connections 2024-05-03T18:37:30.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:30.480+0000 7f8283a00700 1 --2- 172.21.15.33:0/1117744624 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f827c10a490 0x7f827c1169a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:30.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:30.480+0000 7f8283a00700 1 --2- 172.21.15.33:0/1117744624 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f827c10fb60 0x7f827c10ff40 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:30.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:30.480+0000 7f8283a00700 1 --2- 172.21.15.33:0/1117744624 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f827c109ad0 0x7f827c109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:30.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:30.480+0000 7f8283a00700 1 -- 172.21.15.33:0/1117744624 >> 172.21.15.33:0/1117744624 conn(0x7f827c076c70 msgr2=0x7f827c079090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:37:30.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:30.481+0000 7f8283a00700 1 -- 172.21.15.33:0/1117744624 shutdown_connections 2024-05-03T18:37:30.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:30.481+0000 7f8283a00700 1 -- 172.21.15.33:0/1117744624 wait complete. 2024-05-03T18:37:30.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:30.482+0000 7f8283a00700 1 Processor -- start 2024-05-03T18:37:30.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:30.483+0000 7f8283a00700 1 -- start start 2024-05-03T18:37:30.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:30.484+0000 7f8283a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f827c109ad0 0x7f827c1a0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:30.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:30.484+0000 7f8283a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f827c10a490 0x7f827c1a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:30.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:30.485+0000 7f8283a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f827c10fb60 0x7f827c1a4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:30.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:30.485+0000 7f8283a00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f827c119ce0 con 0x7f827c109ad0 2024-05-03T18:37:30.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:30.485+0000 7f8283a00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f827c119b60 con 0x7f827c10fb60 2024-05-03T18:37:30.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:30.485+0000 7f8283a00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f827c119e60 con 0x7f827c10a490 2024-05-03T18:37:30.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:30.485+0000 7f8273e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f827c10a490 0x7f827c1a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:30.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:30.485+0000 7f8273e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f827c10a490 0x7f827c1a0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:45796/0 (socket says 172.21.15.33:45796) 2024-05-03T18:37:30.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:30.485+0000 7f8273e00700 1 -- 172.21.15.33:0/463114161 learned_addr learned my addr 172.21.15.33:0/463114161 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:37:30.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:30.486+0000 7f8281000700 1 --2- 172.21.15.33:0/463114161 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f827c109ad0 0x7f827c1a0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:30.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:30.486+0000 7f8281a00700 1 --2- 172.21.15.33:0/463114161 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f827c10fb60 0x7f827c1a4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:30.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:30.486+0000 7f8273e00700 1 -- 172.21.15.33:0/463114161 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f827c10fb60 msgr2=0x7f827c1a4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:30.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:30.486+0000 7f8273e00700 1 --2- 172.21.15.33:0/463114161 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f827c10fb60 0x7f827c1a4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:30.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:30.486+0000 7f8273e00700 1 -- 172.21.15.33:0/463114161 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f827c109ad0 msgr2=0x7f827c1a0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:30.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:30.486+0000 7f8273e00700 1 --2- 172.21.15.33:0/463114161 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f827c109ad0 0x7f827c1a0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:30.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:30.486+0000 7f8273e00700 1 -- 172.21.15.33:0/463114161 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f827c1a5160 con 0x7f827c10a490 2024-05-03T18:37:30.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:30.487+0000 7f8281000700 1 --2- 172.21.15.33:0/463114161 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f827c109ad0 0x7f827c1a0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:37:30.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:30.488+0000 7f8273e00700 1 --2- 172.21.15.33:0/463114161 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f827c10a490 0x7f827c1a0690 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f826400d950 tx=0x7f826400dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:30.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:30.488+0000 7f827a000700 1 -- 172.21.15.33:0/463114161 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8264016070 con 0x7f827c10a490 2024-05-03T18:37:30.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:30.488+0000 7f8283a00700 1 -- 172.21.15.33:0/463114161 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f827c1a53f0 con 0x7f827c10a490 2024-05-03T18:37:30.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:30.488+0000 7f8283a00700 1 -- 172.21.15.33:0/463114161 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f827c1accd0 con 0x7f827c10a490 2024-05-03T18:37:30.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:30.490+0000 7f8283a00700 1 -- 172.21.15.33:0/463114161 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f827c06a190 con 0x7f827c10a490 2024-05-03T18:37:30.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:30.491+0000 7f827a000700 1 -- 172.21.15.33:0/463114161 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f826400fd20 con 0x7f827c10a490 2024-05-03T18:37:30.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:30.491+0000 7f827a000700 1 -- 172.21.15.33:0/463114161 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f826400e3b0 con 0x7f827c10a490 2024-05-03T18:37:30.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:30.497+0000 7f827a000700 1 -- 172.21.15.33:0/463114161 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f8264019070 con 0x7f827c10a490 2024-05-03T18:37:30.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:30.499+0000 7f827a000700 1 --2- 172.21.15.33:0/463114161 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f82600b2aa0 0x7f82600b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:30.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:30.499+0000 7f827a000700 1 -- 172.21.15.33:0/463114161 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9518+0+0 (secure 0 0 0) 0x7f82640e2860 con 0x7f827c10a490 2024-05-03T18:37:30.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:30.499+0000 7f8281000700 1 --2- 172.21.15.33:0/463114161 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f82600b2aa0 0x7f82600b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:30.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:30.501+0000 7f8281000700 1 --2- 172.21.15.33:0/463114161 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f82600b2aa0 0x7f82600b4f60 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f826c009e60 tx=0x7f826c00b7b0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:30.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:30.503+0000 7f827a000700 1 -- 172.21.15.33:0/463114161 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f82640a9e60 con 0x7f827c10a490 2024-05-03T18:37:30.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:37:30 smithi099 ceph-mon[31658]: pgmap v320: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 6.3 MiB/s wr, 682 op/s 2024-05-03T18:37:31.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:31.008+0000 7f8283a00700 1 -- 172.21.15.33:0/463114161 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f827c02e510 con 0x7f827c10a490 2024-05-03T18:37:31.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:31.010+0000 7f827a000700 1 -- 172.21.15.33:0/463114161 <== mon.1 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11801 (secure 0 0 0) 0x7f82640a9760 con 0x7f827c10a490 2024-05-03T18:37:31.015 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:37:31.015 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1327393503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1327393503},{"type":"v1","addr":"172.21.15.33:6835","nonce":1327393503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/2461472948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":2461472948},{"type":"v1","addr":"172.21.15.33:6837","nonce":2461472948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14559,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/442269281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":442269281},{"type":"v1","addr":"172.21.15.33:6839","nonce":442269281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14571,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/645571756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":645571756},{"type":"v1","addr":"172.21.15.33:6841","nonce":645571756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/203598909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":203598909},{"type":"v1","addr":"172.21.15.150:6833","nonce":203598909}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/4029842493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":4029842493},{"type":"v1","addr":"172.21.15.150:6835","nonce":4029842493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3463664423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3463664423},{"type":"v1","addr":"172.21.15.150:6837","nonce":3463664423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24481,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6839/4051648018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":4051648018},{"type":"v1","addr":"172.21.15.150:6839","nonce":4051648018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34418,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1865105479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1865105479},{"type":"v1","addr":"172.21.15.99:6833","nonce":1865105479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34445,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/1206393914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1206393914},{"type":"v1","addr":"172.21.15.99:6835","nonce":1206393914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-03T18:22:28.837091+0000","modified":"2024-05-03T18:23:49.758398+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34508},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34472":{"gid":34472,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6837/3360970342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3360970342},{"type":"v1","addr":"172.21.15.99:6837","nonce":3360970342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34508":{"gid":34508,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/919907308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":919907308},{"type":"v1","addr":"172.21.15.99:6839","nonce":919907308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34508,"qdb_cluster":[34508]},"id":1}]} 2024-05-03T18:37:31.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:31.016+0000 7f8283a00700 1 -- 172.21.15.33:0/463114161 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f82600b2aa0 msgr2=0x7f82600b4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:31.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:31.016+0000 7f8283a00700 1 --2- 172.21.15.33:0/463114161 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f82600b2aa0 0x7f82600b4f60 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f826c009e60 tx=0x7f826c00b7b0 comp rx=0 tx=0).stop 2024-05-03T18:37:31.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:31.016+0000 7f8283a00700 1 -- 172.21.15.33:0/463114161 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f827c10a490 msgr2=0x7f827c1a0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:31.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:31.016+0000 7f8283a00700 1 --2- 172.21.15.33:0/463114161 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f827c10a490 0x7f827c1a0690 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f826400d950 tx=0x7f826400dc60 comp rx=0 tx=0).stop 2024-05-03T18:37:31.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:31.018+0000 7f8283a00700 1 -- 172.21.15.33:0/463114161 shutdown_connections 2024-05-03T18:37:31.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:31.018+0000 7f8283a00700 1 --2- 172.21.15.33:0/463114161 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f82600b2aa0 0x7f82600b4f60 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:31.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:31.018+0000 7f8283a00700 1 --2- 172.21.15.33:0/463114161 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f827c10fb60 0x7f827c1a4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:31.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:31.018+0000 7f8283a00700 1 --2- 172.21.15.33:0/463114161 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f827c109ad0 0x7f827c1a0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:31.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:31.018+0000 7f8283a00700 1 --2- 172.21.15.33:0/463114161 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f827c10a490 0x7f827c1a0690 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:31.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:31.018+0000 7f8283a00700 1 -- 172.21.15.33:0/463114161 >> 172.21.15.33:0/463114161 conn(0x7f827c076c70 msgr2=0x7f827c0771b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:37:31.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:31.020+0000 7f8283a00700 1 -- 172.21.15.33:0/463114161 shutdown_connections 2024-05-03T18:37:31.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:31.021+0000 7f8283a00700 1 -- 172.21.15.33:0/463114161 wait complete. 2024-05-03T18:37:31.028 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 21 2024-05-03T18:37:31.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:37:31 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/463114161' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:37:31.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:37:31 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/463114161' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:37:31.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:37:31 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/463114161' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:37:32.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:37:32 smithi033 ceph-mon[28330]: pgmap v321: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 4.3 MiB/s wr, 468 op/s 2024-05-03T18:37:32.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:37:32 smithi099 ceph-mon[31658]: pgmap v321: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 4.3 MiB/s wr, 468 op/s 2024-05-03T18:37:32.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:37:32 smithi150 ceph-mon[32492]: pgmap v321: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 4.3 MiB/s wr, 468 op/s 2024-05-03T18:37:33.704 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-03T18:37:34.439 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:37:34 smithi033 ceph-mon[28330]: pgmap v322: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 6.7 MiB/s wr, 721 op/s 2024-05-03T18:37:34.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:37:34 smithi099 ceph-mon[31658]: pgmap v322: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 6.7 MiB/s wr, 721 op/s 2024-05-03T18:37:34.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:37:34 smithi150 ceph-mon[32492]: pgmap v322: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 6.7 MiB/s wr, 721 op/s 2024-05-03T18:37:34.709 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:37:35.110 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 380.0 s, 504.6 tps, lat 196.706 ms stddev 140.796 2024-05-03T18:37:36.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:37:36 smithi150 ceph-mon[32492]: pgmap v323: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 4.5 MiB/s wr, 481 op/s 2024-05-03T18:37:36.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:37:36 smithi033 ceph-mon[28330]: pgmap v323: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 4.5 MiB/s wr, 481 op/s 2024-05-03T18:37:36.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:37:36 smithi099 ceph-mon[31658]: pgmap v323: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 4.5 MiB/s wr, 481 op/s 2024-05-03T18:37:38.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:37:38 smithi150 ceph-mon[32492]: pgmap v324: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 4.5 MiB/s wr, 481 op/s 2024-05-03T18:37:38.486 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:37:38 smithi033 ceph-mon[28330]: pgmap v324: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 4.5 MiB/s wr, 481 op/s 2024-05-03T18:37:38.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:37:38 smithi099 ceph-mon[31658]: pgmap v324: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 4.5 MiB/s wr, 481 op/s 2024-05-03T18:37:40.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.053+0000 7f22f2a00700 1 -- 172.21.15.33:0/1263653083 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f22ec109510 msgr2=0x7f22ec1098f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:40.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.053+0000 7f22f2a00700 1 --2- 172.21.15.33:0/1263653083 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f22ec109510 0x7f22ec1098f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f22e0009990 tx=0x7f22e0009ca0 comp rx=0 tx=0).stop 2024-05-03T18:37:40.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.054+0000 7f22f2a00700 1 -- 172.21.15.33:0/1263653083 shutdown_connections 2024-05-03T18:37:40.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.054+0000 7f22f2a00700 1 --2- 172.21.15.33:0/1263653083 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22ec06bb40 0x7f22ec110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:40.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.054+0000 7f22f2a00700 1 --2- 172.21.15.33:0/1263653083 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f22ec109510 0x7f22ec1098f0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:40.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.054+0000 7f22f2a00700 1 --2- 172.21.15.33:0/1263653083 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f22ec06b180 0x7f22ec06b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:40.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.054+0000 7f22f2a00700 1 -- 172.21.15.33:0/1263653083 >> 172.21.15.33:0/1263653083 conn(0x7f22ec079350 msgr2=0x7f22ec079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:37:40.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.055+0000 7f22f2a00700 1 -- 172.21.15.33:0/1263653083 shutdown_connections 2024-05-03T18:37:40.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.055+0000 7f22f2a00700 1 -- 172.21.15.33:0/1263653083 wait complete. 2024-05-03T18:37:40.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.059+0000 7f22f2a00700 1 Processor -- start 2024-05-03T18:37:40.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.060+0000 7f22f2a00700 1 -- start start 2024-05-03T18:37:40.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.061+0000 7f22f2a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f22ec06b180 0x7f22ec1a4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:40.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.061+0000 7f22f2a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22ec06bb40 0x7f22ec1a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:40.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.061+0000 7f22f2a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f22ec109510 0x7f22ec10f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:40.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.061+0000 7f22f2a00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f22ec1135c0 con 0x7f22ec109510 2024-05-03T18:37:40.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.062+0000 7f22f2a00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f22ec113440 con 0x7f22ec06b180 2024-05-03T18:37:40.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.062+0000 7f22f2a00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f22ec113740 con 0x7f22ec06bb40 2024-05-03T18:37:40.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.062+0000 7f22eb400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22ec06bb40 0x7f22ec1a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:40.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.063+0000 7f22eb400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22ec06bb40 0x7f22ec1a5240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:41000/0 (socket says 172.21.15.33:41000) 2024-05-03T18:37:40.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.063+0000 7f22eb400700 1 -- 172.21.15.33:0/2887265050 learned_addr learned my addr 172.21.15.33:0/2887265050 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:37:40.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.063+0000 7f22eb400700 1 -- 172.21.15.33:0/2887265050 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f22ec06b180 msgr2=0x7f22ec1a4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:40.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.063+0000 7f22eb400700 1 --2- 172.21.15.33:0/2887265050 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f22ec06b180 0x7f22ec1a4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:40.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.063+0000 7f22eb400700 1 -- 172.21.15.33:0/2887265050 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f22ec109510 msgr2=0x7f22ec10f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:40.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.063+0000 7f22eb400700 1 --2- 172.21.15.33:0/2887265050 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f22ec109510 0x7f22ec10f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:40.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.063+0000 7f22eb400700 1 -- 172.21.15.33:0/2887265050 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22ec10fcc0 con 0x7f22ec06bb40 2024-05-03T18:37:40.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.064+0000 7f22eb400700 1 --2- 172.21.15.33:0/2887265050 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22ec06bb40 0x7f22ec1a5240 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f22d800c9b0 tx=0x7f22d800cd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:40.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.065+0000 7f22e8c00700 1 -- 172.21.15.33:0/2887265050 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d8007b00 con 0x7f22ec06bb40 2024-05-03T18:37:40.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.065+0000 7f22f2a00700 1 -- 172.21.15.33:0/2887265050 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22ec10ffb0 con 0x7f22ec06bb40 2024-05-03T18:37:40.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.065+0000 7f22e8c00700 1 -- 172.21.15.33:0/2887265050 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f22d8007ca0 con 0x7f22ec06bb40 2024-05-03T18:37:40.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.065+0000 7f22e8c00700 1 -- 172.21.15.33:0/2887265050 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d8018660 con 0x7f22ec06bb40 2024-05-03T18:37:40.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.065+0000 7f22f2a00700 1 -- 172.21.15.33:0/2887265050 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22ec1acc30 con 0x7f22ec06bb40 2024-05-03T18:37:40.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.078+0000 7f22e8c00700 1 -- 172.21.15.33:0/2887265050 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f22d800f450 con 0x7f22ec06bb40 2024-05-03T18:37:40.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.078+0000 7f22fd45db80 1 Processor -- start 2024-05-03T18:37:40.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.080+0000 7f22fd45db80 1 -- start start 2024-05-03T18:37:40.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.080+0000 7f22fd45db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560e4588d7b0 0x560e4588db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:40.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.080+0000 7f22fd45db80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560e4588e0d0 0x560e45888fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:40.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.080+0000 7f22fd45db80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560e458895d0 0x560e45889a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:40.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.080+0000 7f22fd45db80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560e457dba20 con 0x560e458895d0 2024-05-03T18:37:40.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.080+0000 7f22fd45db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x560e45779760 con 0x560e4588d7b0 2024-05-03T18:37:40.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.080+0000 7f22fd45db80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x560e457d79e0 con 0x560e4588e0d0 2024-05-03T18:37:40.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.081+0000 7f22ebe00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560e4588e0d0 0x560e45888fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:40.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.081+0000 7f22ebe00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560e4588e0d0 0x560e45888fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:41004/0 (socket says 172.21.15.33:41004) 2024-05-03T18:37:40.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.081+0000 7f22ebe00700 1 -- 172.21.15.33:0/1562478369 learned_addr learned my addr 172.21.15.33:0/1562478369 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:37:40.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.081+0000 7f22f0a00700 1 --2- 172.21.15.33:0/1562478369 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560e4588d7b0 0x560e4588db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:40.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.081+0000 7f22e8c00700 1 --2- 172.21.15.33:0/2887265050 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f22d40b2a70 0x7f22d40b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:40.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.082+0000 7f22f0a00700 1 --2- 172.21.15.33:0/2887265050 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f22d40b2a70 0x7f22d40b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:40.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.082+0000 7f22ebe00700 1 -- 172.21.15.33:0/1562478369 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560e4588d7b0 msgr2=0x560e4588db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:37:40.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.082+0000 7f22ebe00700 1 --2- 172.21.15.33:0/1562478369 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560e4588d7b0 0x560e4588db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:40.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.083+0000 7f22ebe00700 1 -- 172.21.15.33:0/1562478369 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560e458895d0 msgr2=0x560e45889a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:37:40.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.082+0000 7f22e8c00700 1 -- 172.21.15.33:0/2887265050 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9518+0+0 (secure 0 0 0) 0x7f22d80e2b30 con 0x7f22ec06bb40 2024-05-03T18:37:40.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.083+0000 7f22ebe00700 1 --2- 172.21.15.33:0/1562478369 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560e458895d0 0x560e45889a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:40.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.083+0000 7f22ebe00700 1 -- 172.21.15.33:0/1562478369 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e457288d0 con 0x560e4588e0d0 2024-05-03T18:37:40.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.083+0000 7f22f0a00700 1 --2- 172.21.15.33:0/1562478369 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560e4588d7b0 0x560e4588db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:37:40.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.084+0000 7f22ebe00700 1 --2- 172.21.15.33:0/1562478369 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560e4588e0d0 0x560e45888fb0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f22e0006010 tx=0x7f22e0009f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7200f6bf728d0f0a server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:40.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.085+0000 7f22f0a00700 1 --2- 172.21.15.33:0/2887265050 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f22d40b2a70 0x7f22d40b4f30 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f22e400d8e0 tx=0x7f22e4004bd0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:40.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.085+0000 7f22dd600700 1 -- 172.21.15.33:0/1562478369 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e0003850 con 0x560e4588e0d0 2024-05-03T18:37:40.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.085+0000 7f22dd600700 1 -- 172.21.15.33:0/1562478369 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f22e00050c0 con 0x560e4588e0d0 2024-05-03T18:37:40.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.085+0000 7f22dd600700 1 -- 172.21.15.33:0/1562478369 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e0003850 con 0x560e4588e0d0 2024-05-03T18:37:40.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.088+0000 7f22fd45db80 1 -- 172.21.15.33:0/1562478369 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560e4588e0d0 msgr2=0x560e45888fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:40.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.088+0000 7f22fd45db80 1 --2- 172.21.15.33:0/1562478369 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560e4588e0d0 0x560e45888fb0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f22e0006010 tx=0x7f22e0009f80 comp rx=0 tx=0).stop 2024-05-03T18:37:40.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.088+0000 7f22fd45db80 1 -- 172.21.15.33:0/1562478369 shutdown_connections 2024-05-03T18:37:40.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.088+0000 7f22fd45db80 1 --2- 172.21.15.33:0/1562478369 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560e458895d0 0x560e45889a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:40.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.088+0000 7f22fd45db80 1 --2- 172.21.15.33:0/1562478369 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560e4588d7b0 0x560e4588db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:40.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.088+0000 7f22fd45db80 1 --2- 172.21.15.33:0/1562478369 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560e4588e0d0 0x560e45888fb0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:40.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.088+0000 7f22fd45db80 1 -- 172.21.15.33:0/1562478369 >> 172.21.15.33:0/1562478369 conn(0x560e457b7b60 msgr2=0x560e458837f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:37:40.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.088+0000 7f22fd45db80 1 -- 172.21.15.33:0/1562478369 shutdown_connections 2024-05-03T18:37:40.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.088+0000 7f22fd45db80 1 -- 172.21.15.33:0/1562478369 wait complete. 2024-05-03T18:37:40.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.089+0000 7f22fd45db80 1 -- start start 2024-05-03T18:37:40.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.090+0000 7f22fd45db80 1 Processor -- start 2024-05-03T18:37:40.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.090+0000 7f22fd45db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560e458895d0 0x560e45894ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:40.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.090+0000 7f22fd45db80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560e4588d7b0 0x560e45892270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:40.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.090+0000 7f22fd45db80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560e4588e0d0 0x560e458927b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:40.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.091+0000 7f22fd45db80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560e45895c40 con 0x560e4588e0d0 2024-05-03T18:37:40.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.091+0000 7f22fd45db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x560e45741b20 con 0x560e458895d0 2024-05-03T18:37:40.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.091+0000 7f22fd45db80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x560e45895dc0 con 0x560e4588d7b0 2024-05-03T18:37:40.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.091+0000 7f22fd45db80 10 client.0 _ll_register_callbacks cb 0x560e45782c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:37:40.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.091+0000 7f22eb400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560e4588d7b0 0x560e45892270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:40.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.091+0000 7f22eb400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560e4588d7b0 0x560e45892270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:41018/0 (socket says 172.21.15.33:41018) 2024-05-03T18:37:40.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.091+0000 7f22eb400700 1 -- 172.21.15.33:0/4074599151 learned_addr learned my addr 172.21.15.33:0/4074599151 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:37:40.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.092+0000 7f22ebe00700 1 --2- 172.21.15.33:0/4074599151 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560e458895d0 0x560e45894ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:40.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.092+0000 7f22eb400700 1 -- 172.21.15.33:0/4074599151 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560e458895d0 msgr2=0x560e45894ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:37:40.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.092+0000 7f22eb400700 1 --2- 172.21.15.33:0/4074599151 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560e458895d0 0x560e45894ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:40.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.092+0000 7f22eb400700 1 -- 172.21.15.33:0/4074599151 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560e4588e0d0 msgr2=0x560e458927b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:37:40.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.092+0000 7f22eb400700 1 --2- 172.21.15.33:0/4074599151 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560e4588e0d0 0x560e458927b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:40.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.093+0000 7f22eb400700 1 -- 172.21.15.33:0/4074599151 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e457a1420 con 0x560e4588d7b0 2024-05-03T18:37:40.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.093+0000 7f22eb400700 1 --2- 172.21.15.33:0/4074599151 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560e4588d7b0 0x560e45892270 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f22d80a96d0 tx=0x7f22d80e02f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b9030c680a90b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:40.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.093+0000 7f22d2000700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-03T18:37:40.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.093+0000 7f22d2000700 1 -- 172.21.15.33:0/4074599151 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d80aa0d0 con 0x560e4588d7b0 2024-05-03T18:37:40.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.094+0000 7f22d2000700 1 -- 172.21.15.33:0/4074599151 <== mon.1 v2:172.21.15.150:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9518+0+0 (secure 0 0 0) 0x7f22d80b23a0 con 0x560e4588d7b0 2024-05-03T18:37:40.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.094+0000 7f22fd45db80 1 -- 172.21.15.33:0/4074599151 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22e00050c0 con 0x560e4588d7b0 2024-05-03T18:37:40.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.094+0000 7f22d2000700 1 -- 172.21.15.33:0/4074599151 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f22d806e050 con 0x560e4588d7b0 2024-05-03T18:37:40.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.094+0000 7f22fd45db80 10 client.26137 fetch_fsmap learned FSMap version 21 2024-05-03T18:37:40.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.094+0000 7f22fd45db80 1 -- 172.21.15.33:0/4074599151 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x560e458a04a0 con 0x560e4588d7b0 2024-05-03T18:37:40.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.095+0000 7f22d2000700 1 -- 172.21.15.33:0/4074599151 <== mon.1 v2:172.21.15.150:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f22e00050c0 con 0x560e4588d7b0 2024-05-03T18:37:40.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.095+0000 7f22d2000700 1 -- 172.21.15.33:0/4074599151 <== mon.1 v2:172.21.15.150:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f22d806f240 con 0x560e4588d7b0 2024-05-03T18:37:40.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.096+0000 7f22fd45db80 10 client.26137 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:37:40.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.096+0000 7f22fd45db80 10 client.26137 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:37:40.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.096+0000 7f22fd45db80 20 client.26137 populate_metadata read hostname 'smithi033' 2024-05-03T18:37:40.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.096+0000 7f22fd45db80 1 --2- 172.21.15.33:0/4074599151 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x560e458a2ae0 0x560e458a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:40.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.096+0000 7f22fd45db80 4 client.26137 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:37:40.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.096+0000 7f22fd45db80 1 -- 172.21.15.33:0/4074599151 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560e458561d0 con 0x560e458a2ae0 2024-05-03T18:37:40.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.097+0000 7f22ebe00700 1 --2- 172.21.15.33:0/4074599151 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x560e458a2ae0 0x560e458a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:40.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.099+0000 7f22ebe00700 1 --2- 172.21.15.33:0/4074599151 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x560e458a2ae0 0x560e458a4ed0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a904ae20409b56a server_cookie=1391fbc1dc0761ce in_seq=0 out_seq=0 2024-05-03T18:37:40.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.099+0000 7f22d2000700 10 client.26137 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:37:40.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.103+0000 7f22d2000700 1 -- 172.21.15.33:0/4074599151 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560e458561d0 con 0x560e458a2ae0 2024-05-03T18:37:40.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.103+0000 7f22d2000700 10 client.26137 handle_command_reply: tid=0 2024-05-03T18:37:40.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.103+0000 7f22fd45db80 1 client.26137 shutdown 2024-05-03T18:37:40.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.104+0000 7f22fd45db80 1 -- 172.21.15.33:0/4074599151 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560e4588d7b0 msgr2=0x560e45892270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:40.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.104+0000 7f22fd45db80 1 --2- 172.21.15.33:0/4074599151 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560e4588d7b0 0x560e45892270 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f22d80a96d0 tx=0x7f22d80e02f0 comp rx=0 tx=0).stop 2024-05-03T18:37:40.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.105+0000 7f22fd45db80 1 -- 172.21.15.33:0/4074599151 shutdown_connections 2024-05-03T18:37:40.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.105+0000 7f22fd45db80 1 --2- 172.21.15.33:0/4074599151 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x560e458a2ae0 0x560e458a4ed0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:40.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.105+0000 7f22fd45db80 1 --2- 172.21.15.33:0/4074599151 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560e4588e0d0 0x560e458927b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:40.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.105+0000 7f22fd45db80 1 --2- 172.21.15.33:0/4074599151 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560e458895d0 0x560e45894ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:40.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.105+0000 7f22fd45db80 1 --2- 172.21.15.33:0/4074599151 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560e4588d7b0 0x560e45892270 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:40.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.105+0000 7f22fd45db80 1 -- 172.21.15.33:0/4074599151 >> 172.21.15.33:0/4074599151 conn(0x560e457b7b60 msgr2=0x560e4588ee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:37:40.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.106+0000 7f22d2000700 0 client.26137 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:37:40.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.106+0000 7f22fd45db80 1 -- 172.21.15.33:0/4074599151 shutdown_connections 2024-05-03T18:37:40.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.106+0000 7f22fd45db80 1 -- 172.21.15.33:0/4074599151 wait complete. 2024-05-03T18:37:40.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.107+0000 7f22fd45db80 20 client.26137 trim_cache size 0 max 16384 2024-05-03T18:37:40.113 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 385.0 s, 603.2 tps, lat 164.885 ms stddev 115.381 2024-05-03T18:37:40.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.138+0000 7f22fd45db80 1 Processor -- start 2024-05-03T18:37:40.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.139+0000 7f22fd45db80 1 -- start start 2024-05-03T18:37:40.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.139+0000 7f22fd45db80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560e458895d0 0x560e458854f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:40.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.139+0000 7f22fd45db80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560e4588d7b0 0x560e4589ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:40.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.139+0000 7f22fd45db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560e4588e0d0 0x560e4588cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:40.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.139+0000 7f22fd45db80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f22ec19f5b0 con 0x560e458895d0 2024-05-03T18:37:40.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.139+0000 7f22fd45db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f22ec1ada70 con 0x560e4588e0d0 2024-05-03T18:37:40.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.139+0000 7f22fd45db80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f22ec19fc00 con 0x560e4588d7b0 2024-05-03T18:37:40.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.140+0000 7f22ebe00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560e458895d0 0x560e458854f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:40.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.140+0000 7f22ebe00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560e458895d0 0x560e458854f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57664/0 (socket says 172.21.15.33:57664) 2024-05-03T18:37:40.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.140+0000 7f22ebe00700 1 -- 172.21.15.33:0/2185551216 learned_addr learned my addr 172.21.15.33:0/2185551216 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:37:40.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.140+0000 7f22eb400700 1 --2- 172.21.15.33:0/2185551216 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560e4588d7b0 0x560e4589ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:40.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.141+0000 7f22f0a00700 1 --2- 172.21.15.33:0/2185551216 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560e4588e0d0 0x560e4588cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:40.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.142+0000 7f22ebe00700 1 -- 172.21.15.33:0/2185551216 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560e4588d7b0 msgr2=0x560e4589ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:37:40.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.142+0000 7f22ebe00700 1 --2- 172.21.15.33:0/2185551216 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560e4588d7b0 0x560e4589ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:40.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.142+0000 7f22ebe00700 1 -- 172.21.15.33:0/2185551216 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560e4588e0d0 msgr2=0x560e4588cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:37:40.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.142+0000 7f22ebe00700 1 --2- 172.21.15.33:0/2185551216 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560e4588e0d0 0x560e4588cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:40.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.142+0000 7f22ebe00700 1 -- 172.21.15.33:0/2185551216 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e458907e0 con 0x560e458895d0 2024-05-03T18:37:40.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.144+0000 7f22ebe00700 1 --2- 172.21.15.33:0/2185551216 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560e458895d0 0x560e458854f0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f22e0005f50 tx=0x7f22e0009ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f0565bb5aa014d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:40.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.145+0000 7f22dcc00700 1 -- 172.21.15.33:0/2185551216 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e00053b0 con 0x560e458895d0 2024-05-03T18:37:40.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.145+0000 7f22dcc00700 1 -- 172.21.15.33:0/2185551216 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f22e00050b0 con 0x560e458895d0 2024-05-03T18:37:40.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.145+0000 7f22dcc00700 1 -- 172.21.15.33:0/2185551216 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e000b950 con 0x560e458895d0 2024-05-03T18:37:40.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.146+0000 7f22fd45db80 1 -- 172.21.15.33:0/2185551216 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560e458895d0 msgr2=0x560e458854f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:40.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.146+0000 7f22fd45db80 1 --2- 172.21.15.33:0/2185551216 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560e458895d0 0x560e458854f0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f22e0005f50 tx=0x7f22e0009ee0 comp rx=0 tx=0).stop 2024-05-03T18:37:40.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.147+0000 7f22fd45db80 1 -- 172.21.15.33:0/2185551216 shutdown_connections 2024-05-03T18:37:40.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.147+0000 7f22fd45db80 1 --2- 172.21.15.33:0/2185551216 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560e4588e0d0 0x560e4588cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:40.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.147+0000 7f22fd45db80 1 --2- 172.21.15.33:0/2185551216 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560e458895d0 0x560e458854f0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:40.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.147+0000 7f22fd45db80 1 --2- 172.21.15.33:0/2185551216 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560e4588d7b0 0x560e4589ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:40.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.147+0000 7f22fd45db80 1 -- 172.21.15.33:0/2185551216 >> 172.21.15.33:0/2185551216 conn(0x560e457b7b60 msgr2=0x560e4588c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:37:40.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.147+0000 7f22fd45db80 1 -- 172.21.15.33:0/2185551216 shutdown_connections 2024-05-03T18:37:40.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.147+0000 7f22fd45db80 1 -- 172.21.15.33:0/2185551216 wait complete. 2024-05-03T18:37:40.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.148+0000 7f22fd45db80 1 -- start start 2024-05-03T18:37:40.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.148+0000 7f22fd45db80 1 Processor -- start 2024-05-03T18:37:40.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.149+0000 7f22fd45db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560e458895d0 0x560e45886a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:40.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.149+0000 7f22fd45db80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560e4588d7b0 0x560e45886f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:40.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.149+0000 7f22fd45db80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560e4588e0d0 0x560e458874b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:40.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.149+0000 7f22fd45db80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f22ec1a02d0 con 0x560e4588d7b0 2024-05-03T18:37:40.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.149+0000 7f22fd45db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f22ec1ad3a0 con 0x560e458895d0 2024-05-03T18:37:40.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.149+0000 7f22fd45db80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f22ec1a09a0 con 0x560e4588e0d0 2024-05-03T18:37:40.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.149+0000 7f22fd45db80 10 client.0 _ll_register_callbacks cb 0x560e45782c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:37:40.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.150+0000 7f22ebe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560e458895d0 0x560e45886a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:40.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.150+0000 7f22eb400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560e4588d7b0 0x560e45886f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:40.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.150+0000 7f22eb400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560e4588d7b0 0x560e45886f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57672/0 (socket says 172.21.15.33:57672) 2024-05-03T18:37:40.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.150+0000 7f22ebe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560e458895d0 0x560e45886a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:46638/0 (socket says 172.21.15.33:46638) 2024-05-03T18:37:40.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.150+0000 7f22eb400700 1 -- 172.21.15.33:0/393791467 learned_addr learned my addr 172.21.15.33:0/393791467 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:37:40.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.150+0000 7f22f0a00700 1 --2- 172.21.15.33:0/393791467 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560e4588e0d0 0x560e458874b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:40.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.150+0000 7f22eb400700 1 -- 172.21.15.33:0/393791467 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560e4588e0d0 msgr2=0x560e458874b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:37:40.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.150+0000 7f22eb400700 1 --2- 172.21.15.33:0/393791467 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560e4588e0d0 0x560e458874b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:40.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.150+0000 7f22eb400700 1 -- 172.21.15.33:0/393791467 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560e458895d0 msgr2=0x560e45886a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:37:40.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.150+0000 7f22eb400700 1 --2- 172.21.15.33:0/393791467 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560e458895d0 0x560e45886a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:40.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.150+0000 7f22eb400700 1 -- 172.21.15.33:0/393791467 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e45899380 con 0x560e4588d7b0 2024-05-03T18:37:40.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.151+0000 7f22ebe00700 1 --2- 172.21.15.33:0/393791467 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560e458895d0 0x560e45886a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:37:40.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.151+0000 7f22eb400700 1 --2- 172.21.15.33:0/393791467 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560e4588d7b0 0x560e45886f70 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f22d80a5f70 tx=0x7f22d80b25d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f88692bab40af30 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:40.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.151+0000 7f22d2000700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:37:40.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.152+0000 7f22d2000700 1 -- 172.21.15.33:0/393791467 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d80aa0d0 con 0x560e4588d7b0 2024-05-03T18:37:40.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.152+0000 7f22d2000700 1 -- 172.21.15.33:0/393791467 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9518+0+0 (secure 0 0 0) 0x7f22d80a5620 con 0x560e4588d7b0 2024-05-03T18:37:40.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.152+0000 7f22fd45db80 1 -- 172.21.15.33:0/393791467 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22e00050b0 con 0x560e4588d7b0 2024-05-03T18:37:40.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.152+0000 7f22d2000700 1 -- 172.21.15.33:0/393791467 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f22d80add80 con 0x560e4588d7b0 2024-05-03T18:37:40.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.152+0000 7f22fd45db80 10 client.16398 fetch_fsmap learned FSMap version 21 2024-05-03T18:37:40.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.152+0000 7f22fd45db80 1 -- 172.21.15.33:0/393791467 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x560e4590d690 con 0x560e4588d7b0 2024-05-03T18:37:40.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.153+0000 7f22d2000700 1 -- 172.21.15.33:0/393791467 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f22e00050b0 con 0x560e4588d7b0 2024-05-03T18:37:40.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.154+0000 7f22d2000700 1 -- 172.21.15.33:0/393791467 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f22d80a86a0 con 0x560e4588d7b0 2024-05-03T18:37:40.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.155+0000 7f22fd45db80 10 client.16398 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:37:40.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.155+0000 7f22fd45db80 10 client.16398 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:37:40.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.155+0000 7f22fd45db80 20 client.16398 populate_metadata read hostname 'smithi033' 2024-05-03T18:37:40.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.155+0000 7f22fd45db80 1 --2- 172.21.15.33:0/393791467 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x560e4590db60 0x560e4590ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:40.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.155+0000 7f22fd45db80 4 client.16398 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-03T18:37:40.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.155+0000 7f22fd45db80 1 -- 172.21.15.33:0/393791467 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560e457e9400 con 0x560e4590db60 2024-05-03T18:37:40.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.156+0000 7f22ebe00700 1 --2- 172.21.15.33:0/393791467 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x560e4590db60 0x560e4590ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:40.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.157+0000 7f22ebe00700 1 --2- 172.21.15.33:0/393791467 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x560e4590db60 0x560e4590ff50 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba42e92d94ac7be6 server_cookie=a2b827cec59ce14a in_seq=0 out_seq=0 2024-05-03T18:37:40.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.158+0000 7f22d2000700 10 client.16398 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:37:40.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.158+0000 7f22d2000700 1 -- 172.21.15.33:0/393791467 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560e457e9400 con 0x560e4590db60 2024-05-03T18:37:40.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.158+0000 7f22d2000700 10 client.16398 handle_command_reply: tid=0 2024-05-03T18:37:40.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.159+0000 7f22fd45db80 1 client.16398 shutdown 2024-05-03T18:37:40.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.159+0000 7f22fd45db80 1 -- 172.21.15.33:0/393791467 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560e4588d7b0 msgr2=0x560e45886f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:40.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.159+0000 7f22fd45db80 1 --2- 172.21.15.33:0/393791467 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560e4588d7b0 0x560e45886f70 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f22d80a5f70 tx=0x7f22d80b25d0 comp rx=0 tx=0).stop 2024-05-03T18:37:40.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.159+0000 7f22fd45db80 1 -- 172.21.15.33:0/393791467 shutdown_connections 2024-05-03T18:37:40.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.159+0000 7f22fd45db80 1 --2- 172.21.15.33:0/393791467 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x560e4590db60 0x560e4590ff50 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:40.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.160+0000 7f22d2000700 0 client.16398 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:37:40.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.160+0000 7f22fd45db80 1 --2- 172.21.15.33:0/393791467 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560e4588e0d0 0x560e458874b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:40.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.160+0000 7f22fd45db80 1 --2- 172.21.15.33:0/393791467 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560e458895d0 0x560e45886a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:40.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.160+0000 7f22fd45db80 1 --2- 172.21.15.33:0/393791467 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560e4588d7b0 0x560e45886f70 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:40.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.160+0000 7f22fd45db80 1 -- 172.21.15.33:0/393791467 >> 172.21.15.33:0/393791467 conn(0x560e457b7b60 msgr2=0x560e45893870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:37:40.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.161+0000 7f22fd45db80 1 -- 172.21.15.33:0/393791467 shutdown_connections 2024-05-03T18:37:40.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.161+0000 7f22fd45db80 1 -- 172.21.15.33:0/393791467 wait complete. 2024-05-03T18:37:40.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.161+0000 7f22fd45db80 20 client.16398 trim_cache size 0 max 16384 2024-05-03T18:37:40.163 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-03T18:37:40.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.162+0000 7f22f2a00700 1 -- 172.21.15.33:0/2887265050 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f22d40b2a70 msgr2=0x7f22d40b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:40.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.162+0000 7f22f2a00700 1 --2- 172.21.15.33:0/2887265050 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f22d40b2a70 0x7f22d40b4f30 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f22e400d8e0 tx=0x7f22e4004bd0 comp rx=0 tx=0).stop 2024-05-03T18:37:40.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.162+0000 7f22f2a00700 1 -- 172.21.15.33:0/2887265050 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22ec06bb40 msgr2=0x7f22ec1a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:40.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.162+0000 7f22f2a00700 1 --2- 172.21.15.33:0/2887265050 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22ec06bb40 0x7f22ec1a5240 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f22d800c9b0 tx=0x7f22d800cd70 comp rx=0 tx=0).stop 2024-05-03T18:37:40.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.162+0000 7f22f2a00700 1 -- 172.21.15.33:0/2887265050 shutdown_connections 2024-05-03T18:37:40.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.162+0000 7f22f2a00700 1 --2- 172.21.15.33:0/2887265050 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f22ec109510 0x7f22ec10f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:40.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.162+0000 7f22f2a00700 1 --2- 172.21.15.33:0/2887265050 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f22d40b2a70 0x7f22d40b4f30 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:40.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.162+0000 7f22f2a00700 1 --2- 172.21.15.33:0/2887265050 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f22ec06b180 0x7f22ec1a4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:40.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.162+0000 7f22f2a00700 1 --2- 172.21.15.33:0/2887265050 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22ec06bb40 0x7f22ec1a5240 secure :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f22d800c9b0 tx=0x7f22d800cd70 comp rx=0 tx=0).stop 2024-05-03T18:37:40.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.162+0000 7f22f2a00700 1 -- 172.21.15.33:0/2887265050 >> 172.21.15.33:0/2887265050 conn(0x7f22ec079350 msgr2=0x7f22ec100f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:37:40.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.163+0000 7f22f2a00700 1 -- 172.21.15.33:0/2887265050 shutdown_connections 2024-05-03T18:37:40.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:40.163+0000 7f22f2a00700 1 -- 172.21.15.33:0/2887265050 wait complete. 2024-05-03T18:37:40.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:37:40 smithi150 ceph-mon[32492]: pgmap v325: 129 pgs: 129 active+clean; 2.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 6.8 MiB/s wr, 711 op/s 2024-05-03T18:37:40.486 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:37:40 smithi033 ceph-mon[28330]: pgmap v325: 129 pgs: 129 active+clean; 2.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 6.8 MiB/s wr, 711 op/s 2024-05-03T18:37:40.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:37:40 smithi099 ceph-mon[31658]: pgmap v325: 129 pgs: 129 active+clean; 2.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 6.8 MiB/s wr, 711 op/s 2024-05-03T18:37:42.188 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-03T18:37:42.188 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-03T18:37:42.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:37:42 smithi033 ceph-mon[28330]: pgmap v326: 129 pgs: 129 active+clean; 2.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 4.7 MiB/s wr, 483 op/s 2024-05-03T18:37:42.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:37:42 smithi099 ceph-mon[31658]: pgmap v326: 129 pgs: 129 active+clean; 2.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 4.7 MiB/s wr, 483 op/s 2024-05-03T18:37:42.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:37:42 smithi150 ceph-mon[32492]: pgmap v326: 129 pgs: 129 active+clean; 2.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 4.7 MiB/s wr, 483 op/s 2024-05-03T18:37:43.040 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:37:44.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:37:44 smithi033 ceph-mon[28330]: pgmap v327: 129 pgs: 129 active+clean; 2.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 6.8 MiB/s wr, 711 op/s 2024-05-03T18:37:44.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:37:44 smithi099 ceph-mon[31658]: pgmap v327: 129 pgs: 129 active+clean; 2.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 6.8 MiB/s wr, 711 op/s 2024-05-03T18:37:44.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:37:44 smithi150 ceph-mon[32492]: pgmap v327: 129 pgs: 129 active+clean; 2.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 6.8 MiB/s wr, 711 op/s 2024-05-03T18:37:45.110 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 390.0 s, 605.9 tps, lat 166.321 ms stddev 104.164 2024-05-03T18:37:46.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:37:46 smithi033 ceph-mon[28330]: pgmap v328: 129 pgs: 129 active+clean; 2.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 4.4 MiB/s wr, 458 op/s 2024-05-03T18:37:46.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:37:46 smithi099 ceph-mon[31658]: pgmap v328: 129 pgs: 129 active+clean; 2.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 4.4 MiB/s wr, 458 op/s 2024-05-03T18:37:46.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:37:46 smithi150 ceph-mon[32492]: pgmap v328: 129 pgs: 129 active+clean; 2.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 4.4 MiB/s wr, 458 op/s 2024-05-03T18:37:48.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.237+0000 7fc195a00700 1 -- 172.21.15.33:0/2376471536 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1901056e0 msgr2=0x7fc190105b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:48.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.237+0000 7fc195a00700 1 --2- 172.21.15.33:0/2376471536 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1901056e0 0x7fc190105b60 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc17c0098f0 tx=0x7fc17c009c00 comp rx=0 tx=0).stop 2024-05-03T18:37:48.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.237+0000 7fc195a00700 1 -- 172.21.15.33:0/2376471536 shutdown_connections 2024-05-03T18:37:48.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.237+0000 7fc195a00700 1 --2- 172.21.15.33:0/2376471536 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1901060a0 0x7fc1901125b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:48.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.237+0000 7fc195a00700 1 --2- 172.21.15.33:0/2376471536 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc19010b6e0 0x7fc19010bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:48.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.237+0000 7fc195a00700 1 --2- 172.21.15.33:0/2376471536 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1901056e0 0x7fc190105b60 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:48.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.237+0000 7fc195a00700 1 -- 172.21.15.33:0/2376471536 >> 172.21.15.33:0/2376471536 conn(0x7fc1901011d0 msgr2=0x7fc1901035f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:37:48.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.238+0000 7fc195a00700 1 -- 172.21.15.33:0/2376471536 shutdown_connections 2024-05-03T18:37:48.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.238+0000 7fc195a00700 1 -- 172.21.15.33:0/2376471536 wait complete. 2024-05-03T18:37:48.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.240+0000 7fc195a00700 1 Processor -- start 2024-05-03T18:37:48.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.240+0000 7fc195a00700 1 -- start start 2024-05-03T18:37:48.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.241+0000 7fc195a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1901056e0 0x7fc19019bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:48.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.241+0000 7fc195a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1901060a0 0x7fc19019c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:48.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.242+0000 7fc195a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc19010b6e0 0x7fc1901a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:48.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.242+0000 7fc18ea00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1901056e0 0x7fc19019bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:48.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.242+0000 7fc195a00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc190115830 con 0x7fc1901060a0 2024-05-03T18:37:48.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.242+0000 7fc195a00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc1901156b0 con 0x7fc19010b6e0 2024-05-03T18:37:48.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.242+0000 7fc195a00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc1901159b0 con 0x7fc1901056e0 2024-05-03T18:37:48.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.242+0000 7fc18ea00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1901056e0 0x7fc19019bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:47550/0 (socket says 172.21.15.33:47550) 2024-05-03T18:37:48.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.242+0000 7fc18ea00700 1 -- 172.21.15.33:0/146310206 learned_addr learned my addr 172.21.15.33:0/146310206 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:37:48.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.242+0000 7fc18e000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1901060a0 0x7fc19019c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:48.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.242+0000 7fc18f400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc19010b6e0 0x7fc1901a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:48.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.243+0000 7fc18ea00700 1 -- 172.21.15.33:0/146310206 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc19010b6e0 msgr2=0x7fc1901a0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:48.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.243+0000 7fc18ea00700 1 --2- 172.21.15.33:0/146310206 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc19010b6e0 0x7fc1901a0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:48.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.243+0000 7fc18ea00700 1 -- 172.21.15.33:0/146310206 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1901060a0 msgr2=0x7fc19019c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:48.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.243+0000 7fc18ea00700 1 --2- 172.21.15.33:0/146310206 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1901060a0 0x7fc19019c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:48.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.243+0000 7fc18ea00700 1 -- 172.21.15.33:0/146310206 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1901a0d70 con 0x7fc1901056e0 2024-05-03T18:37:48.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.244+0000 7fc18ea00700 1 --2- 172.21.15.33:0/146310206 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1901056e0 0x7fc19019bd60 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc18000e9f0 tx=0x7fc18000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:48.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.244+0000 7fc187400700 1 -- 172.21.15.33:0/146310206 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc180004a30 con 0x7fc1901056e0 2024-05-03T18:37:48.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.245+0000 7fc187400700 1 -- 172.21.15.33:0/146310206 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc18000cc50 con 0x7fc1901056e0 2024-05-03T18:37:48.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.245+0000 7fc187400700 1 -- 172.21.15.33:0/146310206 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1800055b0 con 0x7fc1901056e0 2024-05-03T18:37:48.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.246+0000 7fc195a00700 1 -- 172.21.15.33:0/146310206 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1901a1060 con 0x7fc1901056e0 2024-05-03T18:37:48.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.246+0000 7fc195a00700 1 -- 172.21.15.33:0/146310206 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1901a88e0 con 0x7fc1901056e0 2024-05-03T18:37:48.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.260+0000 7fc1a0357b80 1 Processor -- start 2024-05-03T18:37:48.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.260+0000 7fc187400700 1 -- 172.21.15.33:0/146310206 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fc180021430 con 0x7fc1901056e0 2024-05-03T18:37:48.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.261+0000 7fc1a0357b80 1 -- start start 2024-05-03T18:37:48.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.263+0000 7fc1a0357b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ed809881c0 0x55ed809885a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:48.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.263+0000 7fc1a0357b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed80988ae0 0x55ed809827c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:48.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.263+0000 7fc1a0357b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ed80982de0 0x55ed80983260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:48.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.263+0000 7fc1a0357b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ed808ccaa0 con 0x55ed80988ae0 2024-05-03T18:37:48.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.263+0000 7fc1a0357b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55ed8086a7e0 con 0x55ed809881c0 2024-05-03T18:37:48.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.263+0000 7fc1a0357b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55ed808c8a60 con 0x55ed80982de0 2024-05-03T18:37:48.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.263+0000 7fc18f400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ed809881c0 0x55ed809885a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:48.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.263+0000 7fc18f400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ed809881c0 0x55ed809885a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:44008/0 (socket says 172.21.15.33:44008) 2024-05-03T18:37:48.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.263+0000 7fc18f400700 1 -- 172.21.15.33:0/3473055697 learned_addr learned my addr 172.21.15.33:0/3473055697 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:37:48.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.264+0000 7fc18e000700 1 --2- 172.21.15.33:0/3473055697 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ed80982de0 0x55ed80983260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:48.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.264+0000 7fc18f400700 1 -- 172.21.15.33:0/3473055697 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ed80982de0 msgr2=0x55ed80983260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:37:48.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.264+0000 7fc18f400700 1 --2- 172.21.15.33:0/3473055697 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ed80982de0 0x55ed80983260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:48.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.264+0000 7fc18f400700 1 -- 172.21.15.33:0/3473055697 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed80988ae0 msgr2=0x55ed809827c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:37:48.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.264+0000 7fc18f400700 1 --2- 172.21.15.33:0/3473055697 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed80988ae0 0x55ed809827c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:48.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.264+0000 7fc18f400700 1 -- 172.21.15.33:0/3473055697 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed80819950 con 0x55ed809881c0 2024-05-03T18:37:48.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.265+0000 7fc187400700 1 --2- 172.21.15.33:0/146310206 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fc1740b2c00 0x7fc1740b50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:48.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.265+0000 7fc187400700 1 -- 172.21.15.33:0/146310206 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9518+0+0 (secure 0 0 0) 0x7fc1800e2d40 con 0x7fc1901056e0 2024-05-03T18:37:48.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.265+0000 7fc18f400700 1 --2- 172.21.15.33:0/3473055697 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ed809881c0 0x55ed809885a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc188011680 tx=0x7fc188009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c57cc116ece1f0b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:48.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.266+0000 7fc17be00700 1 -- 172.21.15.33:0/3473055697 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc188019030 con 0x55ed809881c0 2024-05-03T18:37:48.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.266+0000 7fc17be00700 1 -- 172.21.15.33:0/3473055697 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc1880093a0 con 0x55ed809881c0 2024-05-03T18:37:48.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.266+0000 7fc17be00700 1 -- 172.21.15.33:0/3473055697 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc188007500 con 0x55ed809881c0 2024-05-03T18:37:48.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.268+0000 7fc1a0357b80 1 -- 172.21.15.33:0/3473055697 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ed809881c0 msgr2=0x55ed809885a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:48.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.268+0000 7fc1a0357b80 1 --2- 172.21.15.33:0/3473055697 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ed809881c0 0x55ed809885a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc188011680 tx=0x7fc188009920 comp rx=0 tx=0).stop 2024-05-03T18:37:48.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.268+0000 7fc18ea00700 1 --2- 172.21.15.33:0/146310206 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fc1740b2c00 0x7fc1740b50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:48.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.268+0000 7fc1a0357b80 1 -- 172.21.15.33:0/3473055697 shutdown_connections 2024-05-03T18:37:48.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.268+0000 7fc1a0357b80 1 --2- 172.21.15.33:0/3473055697 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ed80982de0 0x55ed80983260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:48.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.268+0000 7fc1a0357b80 1 --2- 172.21.15.33:0/3473055697 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ed809881c0 0x55ed809885a0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:48.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.268+0000 7fc1a0357b80 1 --2- 172.21.15.33:0/3473055697 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed80988ae0 0x55ed809827c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:48.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.268+0000 7fc1a0357b80 1 -- 172.21.15.33:0/3473055697 >> 172.21.15.33:0/3473055697 conn(0x55ed808f2750 msgr2=0x55ed80974980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:37:48.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.268+0000 7fc1a0357b80 1 -- 172.21.15.33:0/3473055697 shutdown_connections 2024-05-03T18:37:48.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.268+0000 7fc1a0357b80 1 -- 172.21.15.33:0/3473055697 wait complete. 2024-05-03T18:37:48.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.269+0000 7fc1a0357b80 1 -- start start 2024-05-03T18:37:48.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.270+0000 7fc1a0357b80 1 Processor -- start 2024-05-03T18:37:48.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.270+0000 7fc18ea00700 1 --2- 172.21.15.33:0/146310206 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fc1740b2c00 0x7fc1740b50c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc180004fb0 tx=0x7fc1800057e0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:48.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.273+0000 7fc1a0357b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ed80982de0 0x55ed8098e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:48.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.274+0000 7fc1a0357b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ed809881c0 0x55ed8098baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:48.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.274+0000 7fc1a0357b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed80988ae0 0x55ed8098c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:48.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.274+0000 7fc1a0357b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ed8098f590 con 0x55ed80988ae0 2024-05-03T18:37:48.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.274+0000 7fc1a0357b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55ed80832ba0 con 0x55ed80982de0 2024-05-03T18:37:48.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.274+0000 7fc1a0357b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55ed8098f710 con 0x55ed809881c0 2024-05-03T18:37:48.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.275+0000 7fc18e000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ed809881c0 0x55ed8098baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:48.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.275+0000 7fc1a0357b80 10 client.0 _ll_register_callbacks cb 0x55ed80873cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:37:48.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.275+0000 7fc18e000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ed809881c0 0x55ed8098baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:47564/0 (socket says 172.21.15.33:47564) 2024-05-03T18:37:48.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.275+0000 7fc18e000700 1 -- 172.21.15.33:0/3433613465 learned_addr learned my addr 172.21.15.33:0/3433613465 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:37:48.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.275+0000 7fc18f400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed80988ae0 0x55ed8098c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:48.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.276+0000 7fc18f400700 1 --2- 172.21.15.33:0/3433613465 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ed80982de0 0x55ed8098e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:48.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.276+0000 7fc18f400700 1 -- 172.21.15.33:0/3433613465 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ed809881c0 msgr2=0x55ed8098baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:37:48.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.276+0000 7fc18f400700 1 --2- 172.21.15.33:0/3433613465 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ed809881c0 0x55ed8098baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:48.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.276+0000 7fc18f400700 1 -- 172.21.15.33:0/3433613465 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed80988ae0 msgr2=0x55ed8098c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:37:48.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.276+0000 7fc18f400700 1 --2- 172.21.15.33:0/3433613465 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed80988ae0 0x55ed8098c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:48.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.276+0000 7fc18f400700 1 -- 172.21.15.33:0/3433613465 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed808924a0 con 0x55ed80982de0 2024-05-03T18:37:48.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.277+0000 7fc18f400700 1 --2- 172.21.15.33:0/3433613465 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ed80982de0 0x55ed8098e750 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc188011650 tx=0x7fc188000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d71fb7edea4d61f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:48.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.277+0000 7fc178c00700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-03T18:37:48.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.278+0000 7fc178c00700 1 -- 172.21.15.33:0/3433613465 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc188004ad0 con 0x55ed80982de0 2024-05-03T18:37:48.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.278+0000 7fc178c00700 1 -- 172.21.15.33:0/3433613465 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9518+0+0 (secure 0 0 0) 0x7fc188025030 con 0x55ed80982de0 2024-05-03T18:37:48.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.278+0000 7fc1a0357b80 1 -- 172.21.15.33:0/3433613465 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ed80947070 con 0x55ed80982de0 2024-05-03T18:37:48.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.281+0000 7fc178c00700 1 -- 172.21.15.33:0/3433613465 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fc188004c70 con 0x55ed80982de0 2024-05-03T18:37:48.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.281+0000 7fc1a0357b80 10 client.36077 fetch_fsmap learned FSMap version 21 2024-05-03T18:37:48.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.281+0000 7fc1a0357b80 1 -- 172.21.15.33:0/3433613465 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55ed80999d10 con 0x55ed80982de0 2024-05-03T18:37:48.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.281+0000 7fc178c00700 1 -- 172.21.15.33:0/3433613465 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55ed80947070 con 0x55ed80982de0 2024-05-03T18:37:48.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.282+0000 7fc178c00700 1 -- 172.21.15.33:0/3433613465 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fc188007ce0 con 0x55ed80982de0 2024-05-03T18:37:48.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.283+0000 7fc1a0357b80 10 client.36077 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:37:48.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.283+0000 7fc1a0357b80 10 client.36077 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:37:48.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.283+0000 7fc1a0357b80 20 client.36077 populate_metadata read hostname 'smithi033' 2024-05-03T18:37:48.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.283+0000 7fc1a0357b80 1 --2- 172.21.15.33:0/3433613465 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55ed8099c2f0 0x55ed8099e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:48.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.283+0000 7fc1a0357b80 4 client.36077 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:37:48.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.283+0000 7fc1a0357b80 1 -- 172.21.15.33:0/3433613465 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ed808da480 con 0x55ed8099c2f0 2024-05-03T18:37:48.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.284+0000 7fc18e000700 1 --2- 172.21.15.33:0/3433613465 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55ed8099c2f0 0x55ed8099e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:48.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.287+0000 7fc18e000700 1 --2- 172.21.15.33:0/3433613465 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55ed8099c2f0 0x55ed8099e6e0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e128e2a681d070b9 server_cookie=cb484c4b41f3cfaf in_seq=0 out_seq=0 2024-05-03T18:37:48.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.287+0000 7fc178c00700 10 client.36077 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:37:48.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.289+0000 7fc178c00700 1 -- 172.21.15.33:0/3433613465 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ed808da480 con 0x55ed8099c2f0 2024-05-03T18:37:48.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.289+0000 7fc178c00700 10 client.36077 handle_command_reply: tid=0 2024-05-03T18:37:48.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.290+0000 7fc1a0357b80 1 client.36077 shutdown 2024-05-03T18:37:48.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.290+0000 7fc1a0357b80 1 -- 172.21.15.33:0/3433613465 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ed80982de0 msgr2=0x55ed8098e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:48.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.290+0000 7fc1a0357b80 1 --2- 172.21.15.33:0/3433613465 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ed80982de0 0x55ed8098e750 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc188011650 tx=0x7fc188000f40 comp rx=0 tx=0).stop 2024-05-03T18:37:48.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.290+0000 7fc1a0357b80 1 -- 172.21.15.33:0/3433613465 shutdown_connections 2024-05-03T18:37:48.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.290+0000 7fc1a0357b80 1 --2- 172.21.15.33:0/3433613465 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55ed8099c2f0 0x55ed8099e6e0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:48.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.290+0000 7fc1a0357b80 1 --2- 172.21.15.33:0/3433613465 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed80988ae0 0x55ed8098c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:48.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.290+0000 7fc1a0357b80 1 --2- 172.21.15.33:0/3433613465 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ed80982de0 0x55ed8098e750 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:48.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.290+0000 7fc1a0357b80 1 --2- 172.21.15.33:0/3433613465 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ed809881c0 0x55ed8098baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:48.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.291+0000 7fc1a0357b80 1 -- 172.21.15.33:0/3433613465 >> 172.21.15.33:0/3433613465 conn(0x55ed808f2750 msgr2=0x55ed8096f430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:37:48.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.291+0000 7fc178c00700 0 client.36077 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:37:48.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.291+0000 7fc1a0357b80 1 -- 172.21.15.33:0/3433613465 shutdown_connections 2024-05-03T18:37:48.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.292+0000 7fc1a0357b80 1 -- 172.21.15.33:0/3433613465 wait complete. 2024-05-03T18:37:48.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.292+0000 7fc1a0357b80 20 client.36077 trim_cache size 0 max 16384 2024-05-03T18:37:48.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.330+0000 7fc1a0357b80 1 Processor -- start 2024-05-03T18:37:48.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.330+0000 7fc1a0357b80 1 -- start start 2024-05-03T18:37:48.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.330+0000 7fc1a0357b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ed80982de0 0x55ed80998620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:48.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.330+0000 7fc1a0357b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ed809881c0 0x55ed809827c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:48.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.330+0000 7fc1a0357b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed80988ae0 0x55ed8098acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:48.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.330+0000 7fc1a0357b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc1901ab550 con 0x55ed80988ae0 2024-05-03T18:37:48.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.330+0000 7fc1a0357b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc1901addb0 con 0x55ed809881c0 2024-05-03T18:37:48.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.330+0000 7fc1a0357b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc1901abba0 con 0x55ed80982de0 2024-05-03T18:37:48.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.331+0000 7fc18e000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ed809881c0 0x55ed809827c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:48.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.331+0000 7fc18e000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ed809881c0 0x55ed809827c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:44016/0 (socket says 172.21.15.33:44016) 2024-05-03T18:37:48.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.331+0000 7fc18e000700 1 -- 172.21.15.33:0/435661943 learned_addr learned my addr 172.21.15.33:0/435661943 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:37:48.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.331+0000 7fc18f400700 1 --2- 172.21.15.33:0/435661943 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ed80982de0 0x55ed80998620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:48.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.332+0000 7fc18e000700 1 -- 172.21.15.33:0/435661943 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ed80982de0 msgr2=0x55ed80998620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:37:48.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.332+0000 7fc18e000700 1 --2- 172.21.15.33:0/435661943 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ed80982de0 0x55ed80998620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:48.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.332+0000 7fc18e000700 1 -- 172.21.15.33:0/435661943 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed80988ae0 msgr2=0x55ed8098acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:37:48.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.332+0000 7fc18e000700 1 --2- 172.21.15.33:0/435661943 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed80988ae0 0x55ed8098acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:48.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.332+0000 7fc18e000700 1 -- 172.21.15.33:0/435661943 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed8098e390 con 0x55ed809881c0 2024-05-03T18:37:48.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.333+0000 7fc18e000700 1 --2- 172.21.15.33:0/435661943 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ed809881c0 0x55ed809827c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc17c000c00 tx=0x7fc17c0097f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a4c8eb5a72a7a5e server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:48.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.334+0000 7fc17b400700 1 -- 172.21.15.33:0/435661943 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc17c02c020 con 0x55ed809881c0 2024-05-03T18:37:48.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.334+0000 7fc17b400700 1 -- 172.21.15.33:0/435661943 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc17c0054d0 con 0x55ed809881c0 2024-05-03T18:37:48.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.334+0000 7fc17b400700 1 -- 172.21.15.33:0/435661943 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc17c004930 con 0x55ed809881c0 2024-05-03T18:37:48.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.336+0000 7fc1a0357b80 1 -- 172.21.15.33:0/435661943 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ed809881c0 msgr2=0x55ed809827c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:48.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.336+0000 7fc1a0357b80 1 --2- 172.21.15.33:0/435661943 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ed809881c0 0x55ed809827c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc17c000c00 tx=0x7fc17c0097f0 comp rx=0 tx=0).stop 2024-05-03T18:37:48.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.336+0000 7fc1a0357b80 1 -- 172.21.15.33:0/435661943 shutdown_connections 2024-05-03T18:37:48.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.336+0000 7fc1a0357b80 1 --2- 172.21.15.33:0/435661943 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed80988ae0 0x55ed8098acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:48.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.336+0000 7fc1a0357b80 1 --2- 172.21.15.33:0/435661943 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ed80982de0 0x55ed80998620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:48.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.336+0000 7fc1a0357b80 1 --2- 172.21.15.33:0/435661943 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ed809881c0 0x55ed809827c0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:48.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.336+0000 7fc1a0357b80 1 -- 172.21.15.33:0/435661943 >> 172.21.15.33:0/435661943 conn(0x55ed808f2750 msgr2=0x55ed80989530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:37:48.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.337+0000 7fc1a0357b80 1 -- 172.21.15.33:0/435661943 shutdown_connections 2024-05-03T18:37:48.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.337+0000 7fc1a0357b80 1 -- 172.21.15.33:0/435661943 wait complete. 2024-05-03T18:37:48.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.337+0000 7fc1a0357b80 1 -- start start 2024-05-03T18:37:48.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.338+0000 7fc1a0357b80 1 Processor -- start 2024-05-03T18:37:48.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.339+0000 7fc1a0357b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed80982de0 0x55ed809949a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:48.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.339+0000 7fc1a0357b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ed809881c0 0x55ed80994ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:48.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.339+0000 7fc1a0357b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ed80988ae0 0x55ed80991970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:48.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.339+0000 7fc1a0357b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc1901ac270 con 0x55ed80982de0 2024-05-03T18:37:48.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.339+0000 7fc1a0357b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc1901ad6e0 con 0x55ed80988ae0 2024-05-03T18:37:48.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.339+0000 7fc1a0357b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc1901ac940 con 0x55ed809881c0 2024-05-03T18:37:48.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.339+0000 7fc1a0357b80 10 client.0 _ll_register_callbacks cb 0x55ed80873cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:37:48.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.340+0000 7fc18e000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ed809881c0 0x55ed80994ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:48.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.340+0000 7fc18e000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ed809881c0 0x55ed80994ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:47574/0 (socket says 172.21.15.33:47574) 2024-05-03T18:37:48.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.340+0000 7fc18e000700 1 -- 172.21.15.33:0/113867921 learned_addr learned my addr 172.21.15.33:0/113867921 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:37:48.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.340+0000 7fc18f400700 1 --2- 172.21.15.33:0/113867921 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed80982de0 0x55ed809949a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:48.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.340+0000 7fc18f400700 1 --2- 172.21.15.33:0/113867921 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ed80988ae0 0x55ed80991970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:48.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.340+0000 7fc18e000700 1 -- 172.21.15.33:0/113867921 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ed80988ae0 msgr2=0x55ed80991970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:37:48.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.340+0000 7fc18e000700 1 --2- 172.21.15.33:0/113867921 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ed80988ae0 0x55ed80991970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:48.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.340+0000 7fc18e000700 1 -- 172.21.15.33:0/113867921 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed80982de0 msgr2=0x55ed809949a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:37:48.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.340+0000 7fc18e000700 1 --2- 172.21.15.33:0/113867921 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed80982de0 0x55ed809949a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:48.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.340+0000 7fc18e000700 1 -- 172.21.15.33:0/113867921 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed80991fe0 con 0x55ed809881c0 2024-05-03T18:37:48.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.340+0000 7fc18f400700 1 --2- 172.21.15.33:0/113867921 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ed80988ae0 0x55ed80991970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:37:48.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.341+0000 7fc18e000700 1 --2- 172.21.15.33:0/113867921 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ed809881c0 0x55ed80994ee0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc17c009530 tx=0x7fc17c00f510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=585a04339a2cf29c server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:48.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.341+0000 7fc178c00700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-03T18:37:48.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.341+0000 7fc178c00700 1 -- 172.21.15.33:0/113867921 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc17c037020 con 0x55ed809881c0 2024-05-03T18:37:48.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.341+0000 7fc1a0357b80 1 -- 172.21.15.33:0/113867921 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc17c0054d0 con 0x55ed809881c0 2024-05-03T18:37:48.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.341+0000 7fc178c00700 1 -- 172.21.15.33:0/113867921 <== mon.1 v2:172.21.15.150:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9518+0+0 (secure 0 0 0) 0x7fc17c00b760 con 0x55ed809881c0 2024-05-03T18:37:48.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.342+0000 7fc178c00700 1 -- 172.21.15.33:0/113867921 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fc17c042050 con 0x55ed809881c0 2024-05-03T18:37:48.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.342+0000 7fc1a0357b80 10 client.26161 fetch_fsmap learned FSMap version 21 2024-05-03T18:37:48.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.342+0000 7fc1a0357b80 1 -- 172.21.15.33:0/113867921 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55ed80a072b0 con 0x55ed809881c0 2024-05-03T18:37:48.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.342+0000 7fc178c00700 1 -- 172.21.15.33:0/113867921 <== mon.1 v2:172.21.15.150:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc17c0054d0 con 0x55ed809881c0 2024-05-03T18:37:48.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.343+0000 7fc178c00700 1 -- 172.21.15.33:0/113867921 <== mon.1 v2:172.21.15.150:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fc17c0285b0 con 0x55ed809881c0 2024-05-03T18:37:48.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.343+0000 7fc1a0357b80 10 client.26161 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:37:48.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.343+0000 7fc1a0357b80 10 client.26161 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:37:48.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.343+0000 7fc1a0357b80 20 client.26161 populate_metadata read hostname 'smithi033' 2024-05-03T18:37:48.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.343+0000 7fc1a0357b80 1 --2- 172.21.15.33:0/113867921 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55ed80a078b0 0x55ed80a09cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:48.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.343+0000 7fc1a0357b80 4 client.26161 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-03T18:37:48.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.343+0000 7fc1a0357b80 1 -- 172.21.15.33:0/113867921 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ed808d4520 con 0x55ed80a078b0 2024-05-03T18:37:48.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.344+0000 7fc18f400700 1 --2- 172.21.15.33:0/113867921 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55ed80a078b0 0x55ed80a09cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:48.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.345+0000 7fc18f400700 1 --2- 172.21.15.33:0/113867921 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55ed80a078b0 0x55ed80a09cd0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a639193f446f40e7 server_cookie=8a585ce7b517aade in_seq=0 out_seq=0 2024-05-03T18:37:48.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.345+0000 7fc178c00700 10 client.26161 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:37:48.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.349+0000 7fc178c00700 1 -- 172.21.15.33:0/113867921 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ed808d4520 con 0x55ed80a078b0 2024-05-03T18:37:48.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.349+0000 7fc178c00700 10 client.26161 handle_command_reply: tid=0 2024-05-03T18:37:48.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.349+0000 7fc1a0357b80 1 client.26161 shutdown 2024-05-03T18:37:48.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.349+0000 7fc1a0357b80 1 -- 172.21.15.33:0/113867921 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ed809881c0 msgr2=0x55ed80994ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:48.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.349+0000 7fc1a0357b80 1 --2- 172.21.15.33:0/113867921 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ed809881c0 0x55ed80994ee0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc17c009530 tx=0x7fc17c00f510 comp rx=0 tx=0).stop 2024-05-03T18:37:48.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.350+0000 7fc1a0357b80 1 -- 172.21.15.33:0/113867921 shutdown_connections 2024-05-03T18:37:48.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.350+0000 7fc1a0357b80 1 --2- 172.21.15.33:0/113867921 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55ed80a078b0 0x55ed80a09cd0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:48.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.350+0000 7fc1a0357b80 1 --2- 172.21.15.33:0/113867921 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ed80988ae0 0x55ed80991970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:48.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.350+0000 7fc1a0357b80 1 --2- 172.21.15.33:0/113867921 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed80982de0 0x55ed809949a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:48.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.350+0000 7fc178c00700 0 client.26161 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:37:48.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.350+0000 7fc1a0357b80 1 --2- 172.21.15.33:0/113867921 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ed809881c0 0x55ed80994ee0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:48.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.350+0000 7fc1a0357b80 1 -- 172.21.15.33:0/113867921 >> 172.21.15.33:0/113867921 conn(0x55ed808f2750 msgr2=0x55ed8096eba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:37:48.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.350+0000 7fc1a0357b80 1 -- 172.21.15.33:0/113867921 shutdown_connections 2024-05-03T18:37:48.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.350+0000 7fc1a0357b80 1 -- 172.21.15.33:0/113867921 wait complete. 2024-05-03T18:37:48.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.350+0000 7fc1a0357b80 20 client.26161 trim_cache size 0 max 16384 2024-05-03T18:37:48.354 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-03T18:37:48.354 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-05-03T18:37:48.354 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "cc0714ec-64a9-41a5-8ee2-54691c991a65", 2024-05-03T18:37:48.354 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-05-03T18:37:48.354 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-03T18:37:48.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.353+0000 7fc195a00700 1 -- 172.21.15.33:0/146310206 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fc1740b2c00 msgr2=0x7fc1740b50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:48.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.353+0000 7fc195a00700 1 --2- 172.21.15.33:0/146310206 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fc1740b2c00 0x7fc1740b50c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc180004fb0 tx=0x7fc1800057e0 comp rx=0 tx=0).stop 2024-05-03T18:37:48.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.353+0000 7fc195a00700 1 -- 172.21.15.33:0/146310206 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1901056e0 msgr2=0x7fc19019bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:48.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.353+0000 7fc195a00700 1 --2- 172.21.15.33:0/146310206 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1901056e0 0x7fc19019bd60 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc18000e9f0 tx=0x7fc18000ed00 comp rx=0 tx=0).stop 2024-05-03T18:37:48.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.354+0000 7fc195a00700 1 -- 172.21.15.33:0/146310206 shutdown_connections 2024-05-03T18:37:48.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.354+0000 7fc195a00700 1 --2- 172.21.15.33:0/146310206 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fc1740b2c00 0x7fc1740b50c0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:48.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.354+0000 7fc195a00700 1 --2- 172.21.15.33:0/146310206 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc19010b6e0 0x7fc1901a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:48.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.354+0000 7fc195a00700 1 --2- 172.21.15.33:0/146310206 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc1901056e0 0x7fc19019bd60 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:48.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.354+0000 7fc195a00700 1 --2- 172.21.15.33:0/146310206 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1901060a0 0x7fc19019c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:48.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.354+0000 7fc195a00700 1 -- 172.21.15.33:0/146310206 >> 172.21.15.33:0/146310206 conn(0x7fc1901011d0 msgr2=0x7fc190101920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:37:48.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.354+0000 7fc195a00700 1 -- 172.21.15.33:0/146310206 shutdown_connections 2024-05-03T18:37:48.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:48.355+0000 7fc195a00700 1 -- 172.21.15.33:0/146310206 wait complete. 2024-05-03T18:37:48.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:37:48 smithi150 ceph-mon[32492]: pgmap v329: 129 pgs: 129 active+clean; 2.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 4.4 MiB/s wr, 458 op/s 2024-05-03T18:37:48.385 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:37:48 smithi033 ceph-mon[28330]: pgmap v329: 129 pgs: 129 active+clean; 2.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 4.4 MiB/s wr, 458 op/s 2024-05-03T18:37:48.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:37:48 smithi099 ceph-mon[31658]: pgmap v329: 129 pgs: 129 active+clean; 2.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 4.4 MiB/s wr, 458 op/s 2024-05-03T18:37:49.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:37:49 smithi150 ceph-mon[32492]: scrub queued for path: / 2024-05-03T18:37:49.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:37:49 smithi150 ceph-mon[32492]: scrub summary: idle+waiting paths [/] 2024-05-03T18:37:49.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:37:49 smithi150 ceph-mon[32492]: scrub summary: active paths [/] 2024-05-03T18:37:49.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:37:49 smithi150 ceph-mon[32492]: scrub summary: idle+waiting paths [/] 2024-05-03T18:37:49.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:37:49 smithi033 ceph-mon[28330]: scrub queued for path: / 2024-05-03T18:37:49.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:37:49 smithi033 ceph-mon[28330]: scrub summary: idle+waiting paths [/] 2024-05-03T18:37:49.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:37:49 smithi033 ceph-mon[28330]: scrub summary: active paths [/] 2024-05-03T18:37:49.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:37:49 smithi033 ceph-mon[28330]: scrub summary: idle+waiting paths [/] 2024-05-03T18:37:49.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:37:49 smithi099 ceph-mon[31658]: scrub queued for path: / 2024-05-03T18:37:49.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:37:49 smithi099 ceph-mon[31658]: scrub summary: idle+waiting paths [/] 2024-05-03T18:37:49.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:37:49 smithi099 ceph-mon[31658]: scrub summary: active paths [/] 2024-05-03T18:37:49.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:37:49 smithi099 ceph-mon[31658]: scrub summary: idle+waiting paths [/] 2024-05-03T18:37:50.111 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 395.0 s, 573.4 tps, lat 173.784 ms stddev 94.024 2024-05-03T18:37:50.623 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:37:50 smithi033 ceph-mon[28330]: pgmap v330: 129 pgs: 129 active+clean; 2.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 6.4 MiB/s wr, 677 op/s 2024-05-03T18:37:50.628 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-03T18:37:50.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:37:50 smithi150 ceph-mon[32492]: pgmap v330: 129 pgs: 129 active+clean; 2.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 6.4 MiB/s wr, 677 op/s 2024-05-03T18:37:50.697 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:37:50 smithi099 ceph-mon[31658]: pgmap v330: 129 pgs: 129 active+clean; 2.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 6.4 MiB/s wr, 677 op/s 2024-05-03T18:37:51.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:37:51 smithi150 ceph-mon[32492]: scrub summary: idle 2024-05-03T18:37:51.668 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:37:51.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:37:51 smithi033 ceph-mon[28330]: scrub summary: idle 2024-05-03T18:37:51.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:37:51 smithi099 ceph-mon[31658]: scrub summary: idle 2024-05-03T18:37:52.591 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:37:52 smithi033 ceph-mon[28330]: pgmap v331: 129 pgs: 129 active+clean; 2.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 4.0 MiB/s wr, 446 op/s 2024-05-03T18:37:52.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:37:52 smithi150 ceph-mon[32492]: pgmap v331: 129 pgs: 129 active+clean; 2.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 4.0 MiB/s wr, 446 op/s 2024-05-03T18:37:52.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:37:52 smithi099 ceph-mon[31658]: pgmap v331: 129 pgs: 129 active+clean; 2.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 4.0 MiB/s wr, 446 op/s 2024-05-03T18:37:53.798 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:37:53 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:37:53.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:37:53 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:37:53.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:37:53 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:37:54.596 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:37:54 smithi033 ceph-mon[28330]: pgmap v332: 129 pgs: 129 active+clean; 2.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 6.2 MiB/s wr, 688 op/s 2024-05-03T18:37:54.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:37:54 smithi099 ceph-mon[31658]: pgmap v332: 129 pgs: 129 active+clean; 2.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 6.2 MiB/s wr, 688 op/s 2024-05-03T18:37:54.864 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:37:54 smithi150 ceph-mon[32492]: pgmap v332: 129 pgs: 129 active+clean; 2.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 6.2 MiB/s wr, 688 op/s 2024-05-03T18:37:55.110 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 400.0 s, 605.8 tps, lat 167.153 ms stddev 65.646 2024-05-03T18:37:55.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.836+0000 7f444f600700 1 -- 172.21.15.33:0/1228398663 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4448111c80 msgr2=0x7f444810a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:55.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.836+0000 7f444f600700 1 --2- 172.21.15.33:0/1228398663 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4448111c80 0x7f444810a4a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f443c009990 tx=0x7f443c009ca0 comp rx=0 tx=0).stop 2024-05-03T18:37:55.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.836+0000 7f444f600700 1 -- 172.21.15.33:0/1228398663 shutdown_connections 2024-05-03T18:37:55.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.836+0000 7f444f600700 1 --2- 172.21.15.33:0/1228398663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f444811cc80 0x7f444811f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:55.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.836+0000 7f444f600700 1 --2- 172.21.15.33:0/1228398663 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4448111c80 0x7f444810a4a0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:55.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.836+0000 7f444f600700 1 --2- 172.21.15.33:0/1228398663 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f444810a9e0 0x7f444810ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:55.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.837+0000 7f444f600700 1 -- 172.21.15.33:0/1228398663 >> 172.21.15.33:0/1228398663 conn(0x7f444806e610 msgr2=0x7f444806ea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:37:55.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.839+0000 7f444f600700 1 -- 172.21.15.33:0/1228398663 shutdown_connections 2024-05-03T18:37:55.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.840+0000 7f444f600700 1 -- 172.21.15.33:0/1228398663 wait complete. 2024-05-03T18:37:55.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.842+0000 7f444f600700 1 Processor -- start 2024-05-03T18:37:55.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.843+0000 7f444f600700 1 -- start start 2024-05-03T18:37:55.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.843+0000 7f444f600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f444810a9e0 0x7f44481ae6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:55.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.843+0000 7f444f600700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4448111c80 0x7f44481aec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:55.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.844+0000 7f444f600700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f444811cc80 0x7f44481a8850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:55.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.844+0000 7f444f600700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f44481224b0 con 0x7f444811cc80 2024-05-03T18:37:55.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.844+0000 7f444f600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4448122330 con 0x7f444810a9e0 2024-05-03T18:37:55.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.844+0000 7f444f600700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f4448122630 con 0x7f4448111c80 2024-05-03T18:37:55.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.844+0000 7f444cc00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f444810a9e0 0x7f44481ae6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:55.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.844+0000 7f444cc00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f444810a9e0 0x7f44481ae6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:49852/0 (socket says 172.21.15.33:49852) 2024-05-03T18:37:55.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.844+0000 7f444cc00700 1 -- 172.21.15.33:0/4260628646 learned_addr learned my addr 172.21.15.33:0/4260628646 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:37:55.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.844+0000 7f4447e00700 1 --2- 172.21.15.33:0/4260628646 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4448111c80 0x7f44481aec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:55.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.845+0000 7f444d600700 1 --2- 172.21.15.33:0/4260628646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f444811cc80 0x7f44481a8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:55.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.845+0000 7f444cc00700 1 -- 172.21.15.33:0/4260628646 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4448111c80 msgr2=0x7f44481aec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:55.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.845+0000 7f444cc00700 1 --2- 172.21.15.33:0/4260628646 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4448111c80 0x7f44481aec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:55.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.845+0000 7f444cc00700 1 -- 172.21.15.33:0/4260628646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f444811cc80 msgr2=0x7f44481a8850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:55.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.845+0000 7f444cc00700 1 --2- 172.21.15.33:0/4260628646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f444811cc80 0x7f44481a8850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:55.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.845+0000 7f444cc00700 1 -- 172.21.15.33:0/4260628646 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44481a9030 con 0x7f444810a9e0 2024-05-03T18:37:55.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.846+0000 7f444cc00700 1 --2- 172.21.15.33:0/4260628646 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f444810a9e0 0x7f44481ae6d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f443c000c00 tx=0x7f443c009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:55.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.846+0000 7f4445600700 1 -- 172.21.15.33:0/4260628646 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f443c004830 con 0x7f444810a9e0 2024-05-03T18:37:55.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.846+0000 7f444f600700 1 -- 172.21.15.33:0/4260628646 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44481a92c0 con 0x7f444810a9e0 2024-05-03T18:37:55.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.847+0000 7f444f600700 1 -- 172.21.15.33:0/4260628646 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44481b5530 con 0x7f444810a9e0 2024-05-03T18:37:55.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.847+0000 7f4445600700 1 -- 172.21.15.33:0/4260628646 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f443c005020 con 0x7f444810a9e0 2024-05-03T18:37:55.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.847+0000 7f4445600700 1 -- 172.21.15.33:0/4260628646 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f443c00f8d0 con 0x7f444810a9e0 2024-05-03T18:37:55.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.854+0000 7f4445600700 1 -- 172.21.15.33:0/4260628646 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f443c0049d0 con 0x7f444810a9e0 2024-05-03T18:37:55.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.857+0000 7f4445600700 1 --2- 172.21.15.33:0/4260628646 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f44300b2b20 0x7f44300b4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:55.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.857+0000 7f4445600700 1 -- 172.21.15.33:0/4260628646 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9518+0+0 (secure 0 0 0) 0x7f443c0e3370 con 0x7f444810a9e0 2024-05-03T18:37:55.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.859+0000 7f4447e00700 1 --2- 172.21.15.33:0/4260628646 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f44300b2b20 0x7f44300b4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:55.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.860+0000 7f4459ff5b80 1 Processor -- start 2024-05-03T18:37:55.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.861+0000 7f4447e00700 1 --2- 172.21.15.33:0/4260628646 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f44300b2b20 0x7f44300b4fe0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f444811f3e0 tx=0x7f443400b410 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:55.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.861+0000 7f4459ff5b80 1 -- start start 2024-05-03T18:37:55.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.862+0000 7f4459ff5b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56452952e7b0 0x56452952eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:55.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.862+0000 7f4459ff5b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56452952f0d0 0x564529529fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:55.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.862+0000 7f4459ff5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56452952a5d0 0x56452952aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:55.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.863+0000 7f444cc00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56452952e7b0 0x56452952eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:55.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.863+0000 7f444cc00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56452952e7b0 0x56452952eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:47708/0 (socket says 172.21.15.33:47708) 2024-05-03T18:37:55.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.863+0000 7f4459ff5b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56452947ca20 con 0x56452952f0d0 2024-05-03T18:37:55.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.863+0000 7f4459ff5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56452941a760 con 0x56452952a5d0 2024-05-03T18:37:55.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.863+0000 7f4459ff5b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5645294789e0 con 0x56452952e7b0 2024-05-03T18:37:55.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.863+0000 7f444cc00700 1 -- 172.21.15.33:0/367429933 learned_addr learned my addr 172.21.15.33:0/367429933 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:37:55.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.864+0000 7f444d600700 1 --2- 172.21.15.33:0/367429933 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56452952a5d0 0x56452952aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:55.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.864+0000 7f444cc00700 1 -- 172.21.15.33:0/367429933 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56452952a5d0 msgr2=0x56452952aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:37:55.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.864+0000 7f444cc00700 1 --2- 172.21.15.33:0/367429933 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56452952a5d0 0x56452952aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:55.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.864+0000 7f444cc00700 1 -- 172.21.15.33:0/367429933 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56452952f0d0 msgr2=0x564529529fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:37:55.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.864+0000 7f444cc00700 1 --2- 172.21.15.33:0/367429933 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56452952f0d0 0x564529529fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:55.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.864+0000 7f444cc00700 1 -- 172.21.15.33:0/367429933 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645293c98d0 con 0x56452952e7b0 2024-05-03T18:37:55.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.865+0000 7f444cc00700 1 --2- 172.21.15.33:0/367429933 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56452952e7b0 0x56452952eb90 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f443c0a6320 tx=0x7f443c0aa1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51637bd96f237374 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:55.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.865+0000 7f443a000700 1 -- 172.21.15.33:0/367429933 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f443c0aa9a0 con 0x56452952e7b0 2024-05-03T18:37:55.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.865+0000 7f443a000700 1 -- 172.21.15.33:0/367429933 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f443c0aa2a0 con 0x56452952e7b0 2024-05-03T18:37:55.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.865+0000 7f443a000700 1 -- 172.21.15.33:0/367429933 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f443c0a5c10 con 0x56452952e7b0 2024-05-03T18:37:55.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.866+0000 7f4459ff5b80 1 -- 172.21.15.33:0/367429933 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56452952e7b0 msgr2=0x56452952eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:55.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.866+0000 7f4459ff5b80 1 --2- 172.21.15.33:0/367429933 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56452952e7b0 0x56452952eb90 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f443c0a6320 tx=0x7f443c0aa1a0 comp rx=0 tx=0).stop 2024-05-03T18:37:55.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.866+0000 7f4459ff5b80 1 -- 172.21.15.33:0/367429933 shutdown_connections 2024-05-03T18:37:55.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.866+0000 7f4459ff5b80 1 --2- 172.21.15.33:0/367429933 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56452952a5d0 0x56452952aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:55.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.866+0000 7f4459ff5b80 1 --2- 172.21.15.33:0/367429933 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56452952e7b0 0x56452952eb90 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:55.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.866+0000 7f4459ff5b80 1 --2- 172.21.15.33:0/367429933 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56452952f0d0 0x564529529fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:55.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.866+0000 7f4459ff5b80 1 -- 172.21.15.33:0/367429933 >> 172.21.15.33:0/367429933 conn(0x564529458b60 msgr2=0x5645295247f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:37:55.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.869+0000 7f4459ff5b80 1 -- 172.21.15.33:0/367429933 shutdown_connections 2024-05-03T18:37:55.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.869+0000 7f4459ff5b80 1 -- 172.21.15.33:0/367429933 wait complete. 2024-05-03T18:37:55.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.870+0000 7f4459ff5b80 1 -- start start 2024-05-03T18:37:55.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.870+0000 7f4459ff5b80 1 Processor -- start 2024-05-03T18:37:55.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.871+0000 7f4459ff5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56452952a5d0 0x564529535f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:55.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.871+0000 7f4459ff5b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56452952e7b0 0x564529533310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:55.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.871+0000 7f4459ff5b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56452952f0d0 0x564529533850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:55.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.871+0000 7f4459ff5b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x564529536a60 con 0x56452952e7b0 2024-05-03T18:37:55.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.871+0000 7f4459ff5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5645293e2b20 con 0x56452952a5d0 2024-05-03T18:37:55.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.871+0000 7f4459ff5b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x564529536be0 con 0x56452952f0d0 2024-05-03T18:37:55.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.871+0000 7f4459ff5b80 10 client.0 _ll_register_callbacks cb 0x564529423c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:37:55.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.873+0000 7f444cc00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56452952a5d0 0x564529535f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:55.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.873+0000 7f444d600700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56452952f0d0 0x564529533850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:55.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.873+0000 7f444d600700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56452952f0d0 0x564529533850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:47722/0 (socket says 172.21.15.33:47722) 2024-05-03T18:37:55.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.873+0000 7f444d600700 1 -- 172.21.15.33:0/2904495802 learned_addr learned my addr 172.21.15.33:0/2904495802 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:37:55.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.874+0000 7f444cc00700 1 -- 172.21.15.33:0/2904495802 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56452952f0d0 msgr2=0x564529533850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:37:55.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.874+0000 7f444cc00700 1 --2- 172.21.15.33:0/2904495802 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56452952f0d0 0x564529533850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:55.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.874+0000 7f444cc00700 1 -- 172.21.15.33:0/2904495802 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56452952e7b0 msgr2=0x564529533310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:37:55.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.874+0000 7f444cc00700 1 --2- 172.21.15.33:0/2904495802 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56452952e7b0 0x564529533310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:55.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.874+0000 7f444cc00700 1 -- 172.21.15.33:0/2904495802 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564529442420 con 0x56452952a5d0 2024-05-03T18:37:55.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.875+0000 7f442ea00700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-03T18:37:55.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.875+0000 7f444cc00700 1 --2- 172.21.15.33:0/2904495802 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56452952a5d0 0x564529535f70 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f443c0e0c80 tx=0x7f443c009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=760c765b802cca48 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:55.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.876+0000 7f442ea00700 1 -- 172.21.15.33:0/2904495802 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f443c0c4060 con 0x56452952a5d0 2024-05-03T18:37:55.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.876+0000 7f442ea00700 1 -- 172.21.15.33:0/2904495802 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9518+0+0 (secure 0 0 0) 0x7f443c0a6890 con 0x56452952a5d0 2024-05-03T18:37:55.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.876+0000 7f4459ff5b80 1 -- 172.21.15.33:0/2904495802 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f443c0aa2a0 con 0x56452952a5d0 2024-05-03T18:37:55.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.877+0000 7f442ea00700 1 -- 172.21.15.33:0/2904495802 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f443c0c4060 con 0x56452952a5d0 2024-05-03T18:37:55.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.877+0000 7f4459ff5b80 10 client.36095 fetch_fsmap learned FSMap version 21 2024-05-03T18:37:55.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.877+0000 7f4459ff5b80 1 -- 172.21.15.33:0/2904495802 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x564529536a60 con 0x56452952a5d0 2024-05-03T18:37:55.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.878+0000 7f442ea00700 1 -- 172.21.15.33:0/2904495802 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f443c0b3070 con 0x56452952a5d0 2024-05-03T18:37:55.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.879+0000 7f442ea00700 1 -- 172.21.15.33:0/2904495802 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f443c0a5c10 con 0x56452952a5d0 2024-05-03T18:37:55.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.879+0000 7f4459ff5b80 10 client.36095 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:37:55.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.879+0000 7f4459ff5b80 10 client.36095 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:37:55.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.879+0000 7f4459ff5b80 20 client.36095 populate_metadata read hostname 'smithi033' 2024-05-03T18:37:55.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.879+0000 7f4459ff5b80 1 --2- 172.21.15.33:0/2904495802 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5645295436f0 0x564529545b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:55.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.879+0000 7f4459ff5b80 4 client.36095 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:37:55.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.879+0000 7f4459ff5b80 1 -- 172.21.15.33:0/2904495802 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f443c0aa2a0 con 0x5645295436f0 2024-05-03T18:37:55.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.879+0000 7f4447e00700 1 --2- 172.21.15.33:0/2904495802 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5645295436f0 0x564529545b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:55.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.880+0000 7f442ea00700 10 client.36095 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:37:55.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.880+0000 7f4447e00700 1 --2- 172.21.15.33:0/2904495802 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5645295436f0 0x564529545b10 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad193e987456e054 server_cookie=d7da72032d802d6a in_seq=0 out_seq=0 2024-05-03T18:37:55.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.881+0000 7f442ea00700 1 -- 172.21.15.33:0/2904495802 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f443c0aa2a0 con 0x5645295436f0 2024-05-03T18:37:55.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.881+0000 7f442ea00700 10 client.36095 handle_command_reply: tid=0 2024-05-03T18:37:55.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.882+0000 7f4459ff5b80 1 client.36095 shutdown 2024-05-03T18:37:55.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.883+0000 7f4459ff5b80 1 -- 172.21.15.33:0/2904495802 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56452952a5d0 msgr2=0x564529535f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:55.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.883+0000 7f4459ff5b80 1 --2- 172.21.15.33:0/2904495802 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56452952a5d0 0x564529535f70 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f443c0e0c80 tx=0x7f443c009310 comp rx=0 tx=0).stop 2024-05-03T18:37:55.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.883+0000 7f4459ff5b80 1 -- 172.21.15.33:0/2904495802 shutdown_connections 2024-05-03T18:37:55.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.883+0000 7f4459ff5b80 1 --2- 172.21.15.33:0/2904495802 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5645295436f0 0x564529545b10 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:55.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.884+0000 7f442ea00700 0 client.36095 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:37:55.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.884+0000 7f4459ff5b80 1 --2- 172.21.15.33:0/2904495802 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56452952f0d0 0x564529533850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:55.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.884+0000 7f4459ff5b80 1 --2- 172.21.15.33:0/2904495802 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56452952a5d0 0x564529535f70 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:55.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.884+0000 7f4459ff5b80 1 --2- 172.21.15.33:0/2904495802 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56452952e7b0 0x564529533310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:55.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.884+0000 7f4459ff5b80 1 -- 172.21.15.33:0/2904495802 >> 172.21.15.33:0/2904495802 conn(0x564529458b60 msgr2=0x564529523650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:37:55.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.884+0000 7f4459ff5b80 1 -- 172.21.15.33:0/2904495802 shutdown_connections 2024-05-03T18:37:55.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.884+0000 7f4459ff5b80 1 -- 172.21.15.33:0/2904495802 wait complete. 2024-05-03T18:37:55.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.888+0000 7f4459ff5b80 20 client.36095 trim_cache size 0 max 16384 2024-05-03T18:37:55.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.923+0000 7f4459ff5b80 1 Processor -- start 2024-05-03T18:37:55.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.924+0000 7f4459ff5b80 1 -- start start 2024-05-03T18:37:55.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.924+0000 7f4459ff5b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56452952a5d0 0x5645295338a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:55.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.924+0000 7f4459ff5b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56452952e7b0 0x56452953fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:55.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.924+0000 7f4459ff5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56452952f0d0 0x564529529c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:55.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.924+0000 7f4459ff5b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f442800cff0 con 0x56452952a5d0 2024-05-03T18:37:55.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.925+0000 7f4459ff5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f442800f890 con 0x56452952f0d0 2024-05-03T18:37:55.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.925+0000 7f4459ff5b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f442800d680 con 0x56452952e7b0 2024-05-03T18:37:55.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.925+0000 7f444d600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56452952f0d0 0x564529529c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:55.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.925+0000 7f444d600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56452952f0d0 0x564529529c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:49896/0 (socket says 172.21.15.33:49896) 2024-05-03T18:37:55.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.925+0000 7f444d600700 1 -- 172.21.15.33:0/4236282774 learned_addr learned my addr 172.21.15.33:0/4236282774 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:37:55.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.925+0000 7f4447e00700 1 --2- 172.21.15.33:0/4236282774 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56452952e7b0 0x56452953fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:55.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.925+0000 7f444cc00700 1 --2- 172.21.15.33:0/4236282774 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56452952a5d0 0x5645295338a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:55.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.926+0000 7f4447e00700 1 -- 172.21.15.33:0/4236282774 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56452952f0d0 msgr2=0x564529529c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:37:55.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.926+0000 7f4447e00700 1 --2- 172.21.15.33:0/4236282774 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56452952f0d0 0x564529529c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:55.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.926+0000 7f4447e00700 1 -- 172.21.15.33:0/4236282774 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56452952a5d0 msgr2=0x5645295338a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:37:55.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.926+0000 7f4447e00700 1 --2- 172.21.15.33:0/4236282774 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56452952a5d0 0x5645295338a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:55.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.926+0000 7f4447e00700 1 -- 172.21.15.33:0/4236282774 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56452952e400 con 0x56452952e7b0 2024-05-03T18:37:55.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.927+0000 7f444d600700 1 --2- 172.21.15.33:0/4236282774 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56452952f0d0 0x564529529c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:37:55.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.927+0000 7f4447e00700 1 --2- 172.21.15.33:0/4236282774 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56452952e7b0 0x56452953fab0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x564529546380 tx=0x7f44340047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b69bf9fd9a05c447 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:55.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.928+0000 7f4439600700 1 -- 172.21.15.33:0/4236282774 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4434011820 con 0x56452952e7b0 2024-05-03T18:37:55.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.928+0000 7f4439600700 1 -- 172.21.15.33:0/4236282774 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4434011e50 con 0x56452952e7b0 2024-05-03T18:37:55.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.928+0000 7f4439600700 1 -- 172.21.15.33:0/4236282774 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f443401e420 con 0x56452952e7b0 2024-05-03T18:37:55.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.929+0000 7f4459ff5b80 1 -- 172.21.15.33:0/4236282774 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56452952e7b0 msgr2=0x56452953fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:55.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.929+0000 7f4459ff5b80 1 --2- 172.21.15.33:0/4236282774 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56452952e7b0 0x56452953fab0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x564529546380 tx=0x7f44340047c0 comp rx=0 tx=0).stop 2024-05-03T18:37:55.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.930+0000 7f4459ff5b80 1 -- 172.21.15.33:0/4236282774 shutdown_connections 2024-05-03T18:37:55.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.930+0000 7f4459ff5b80 1 --2- 172.21.15.33:0/4236282774 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56452952f0d0 0x564529529c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:55.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.930+0000 7f4459ff5b80 1 --2- 172.21.15.33:0/4236282774 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56452952a5d0 0x5645295338a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:55.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.930+0000 7f4459ff5b80 1 --2- 172.21.15.33:0/4236282774 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56452952e7b0 0x56452953fab0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:55.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.930+0000 7f4459ff5b80 1 -- 172.21.15.33:0/4236282774 >> 172.21.15.33:0/4236282774 conn(0x564529458b60 msgr2=0x564529523ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:37:55.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.930+0000 7f4459ff5b80 1 -- 172.21.15.33:0/4236282774 shutdown_connections 2024-05-03T18:37:55.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.931+0000 7f4459ff5b80 1 -- 172.21.15.33:0/4236282774 wait complete. 2024-05-03T18:37:55.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.931+0000 7f4459ff5b80 1 -- start start 2024-05-03T18:37:55.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.932+0000 7f4459ff5b80 1 Processor -- start 2024-05-03T18:37:55.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.935+0000 7f4459ff5b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56452952a5d0 0x564529537be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:55.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.936+0000 7f4459ff5b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56452952e7b0 0x564529538120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:55.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.936+0000 7f4459ff5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56452952f0d0 0x564529538660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:55.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.936+0000 7f4459ff5b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f442800dd50 con 0x56452952e7b0 2024-05-03T18:37:55.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.936+0000 7f4459ff5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f442800f1c0 con 0x56452952f0d0 2024-05-03T18:37:55.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.936+0000 7f4459ff5b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f442800e420 con 0x56452952a5d0 2024-05-03T18:37:55.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.936+0000 7f4459ff5b80 10 client.0 _ll_register_callbacks cb 0x564529423c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:37:55.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.937+0000 7f444cc00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56452952a5d0 0x564529537be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:55.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.937+0000 7f444cc00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56452952a5d0 0x564529537be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:47738/0 (socket says 172.21.15.33:47738) 2024-05-03T18:37:55.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.937+0000 7f444cc00700 1 -- 172.21.15.33:0/295783270 learned_addr learned my addr 172.21.15.33:0/295783270 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:37:55.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.938+0000 7f444d600700 1 --2- 172.21.15.33:0/295783270 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56452952f0d0 0x564529538660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:55.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.938+0000 7f4447e00700 1 --2- 172.21.15.33:0/295783270 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56452952e7b0 0x564529538120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:55.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.939+0000 7f444cc00700 1 -- 172.21.15.33:0/295783270 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56452952f0d0 msgr2=0x564529538660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:37:55.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.939+0000 7f444cc00700 1 --2- 172.21.15.33:0/295783270 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56452952f0d0 0x564529538660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:55.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.939+0000 7f444cc00700 1 -- 172.21.15.33:0/295783270 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56452952e7b0 msgr2=0x564529538120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:37:55.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.939+0000 7f444cc00700 1 --2- 172.21.15.33:0/295783270 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56452952e7b0 0x564529538120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:55.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.939+0000 7f444cc00700 1 -- 172.21.15.33:0/295783270 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645295ad1f0 con 0x56452952a5d0 2024-05-03T18:37:55.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.939+0000 7f444cc00700 1 --2- 172.21.15.33:0/295783270 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56452952a5d0 0x564529537be0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f443c0ae810 tx=0x7f443c0ae270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56c05a4c01d05d8d server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:55.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.939+0000 7f442ea00700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-03T18:37:55.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.940+0000 7f442ea00700 1 -- 172.21.15.33:0/295783270 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f443c0c4060 con 0x56452952a5d0 2024-05-03T18:37:55.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.940+0000 7f442ea00700 1 -- 172.21.15.33:0/295783270 <== mon.1 v2:172.21.15.150:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9518+0+0 (secure 0 0 0) 0x7f443c0c4b00 con 0x56452952a5d0 2024-05-03T18:37:55.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.940+0000 7f4459ff5b80 1 -- 172.21.15.33:0/295783270 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4434011e50 con 0x56452952a5d0 2024-05-03T18:37:55.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.941+0000 7f442ea00700 1 -- 172.21.15.33:0/295783270 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f443c0aa2a0 con 0x56452952a5d0 2024-05-03T18:37:55.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.941+0000 7f4459ff5b80 10 client.26179 fetch_fsmap learned FSMap version 21 2024-05-03T18:37:55.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.941+0000 7f4459ff5b80 1 -- 172.21.15.33:0/295783270 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5645295ad1f0 con 0x56452952a5d0 2024-05-03T18:37:55.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.943+0000 7f442ea00700 1 -- 172.21.15.33:0/295783270 <== mon.1 v2:172.21.15.150:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4434011e50 con 0x56452952a5d0 2024-05-03T18:37:55.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.943+0000 7f442ea00700 1 -- 172.21.15.33:0/295783270 <== mon.1 v2:172.21.15.150:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f443c00fa70 con 0x56452952a5d0 2024-05-03T18:37:55.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.944+0000 7f4459ff5b80 10 client.26179 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:37:55.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.944+0000 7f4459ff5b80 10 client.26179 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:37:55.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.944+0000 7f4459ff5b80 20 client.26179 populate_metadata read hostname 'smithi033' 2024-05-03T18:37:55.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.944+0000 7f4459ff5b80 1 --2- 172.21.15.33:0/295783270 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5645295436f0 0x5645295b0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:55.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.944+0000 7f4459ff5b80 4 client.26179 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-03T18:37:55.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.944+0000 7f4459ff5b80 1 -- 172.21.15.33:0/295783270 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5645294f71d0 con 0x5645295436f0 2024-05-03T18:37:55.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.944+0000 7f4447e00700 1 --2- 172.21.15.33:0/295783270 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5645295436f0 0x5645295b0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:55.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.945+0000 7f442ea00700 10 client.26179 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:37:55.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.945+0000 7f4447e00700 1 --2- 172.21.15.33:0/295783270 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5645295436f0 0x5645295b0760 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e73d72574949517a server_cookie=d3f5727fde8e02e8 in_seq=0 out_seq=0 2024-05-03T18:37:55.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.946+0000 7f442ea00700 1 -- 172.21.15.33:0/295783270 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5645294f71d0 con 0x5645295436f0 2024-05-03T18:37:55.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.946+0000 7f442ea00700 10 client.26179 handle_command_reply: tid=0 2024-05-03T18:37:55.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.946+0000 7f4459ff5b80 1 client.26179 shutdown 2024-05-03T18:37:55.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.948+0000 7f4459ff5b80 1 -- 172.21.15.33:0/295783270 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56452952a5d0 msgr2=0x564529537be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:55.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.948+0000 7f4459ff5b80 1 --2- 172.21.15.33:0/295783270 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56452952a5d0 0x564529537be0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f443c0ae810 tx=0x7f443c0ae270 comp rx=0 tx=0).stop 2024-05-03T18:37:55.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.948+0000 7f4459ff5b80 1 -- 172.21.15.33:0/295783270 shutdown_connections 2024-05-03T18:37:55.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.948+0000 7f4459ff5b80 1 --2- 172.21.15.33:0/295783270 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5645295436f0 0x5645295b0760 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:55.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.948+0000 7f442ea00700 0 client.26179 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:37:55.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.948+0000 7f4459ff5b80 1 --2- 172.21.15.33:0/295783270 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56452952f0d0 0x564529538660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:55.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.948+0000 7f4459ff5b80 1 --2- 172.21.15.33:0/295783270 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56452952a5d0 0x564529537be0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:55.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.948+0000 7f4459ff5b80 1 --2- 172.21.15.33:0/295783270 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56452952e7b0 0x564529538120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:55.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.948+0000 7f4459ff5b80 1 -- 172.21.15.33:0/295783270 >> 172.21.15.33:0/295783270 conn(0x564529458b60 msgr2=0x56452952d3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:37:55.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.949+0000 7f4459ff5b80 1 -- 172.21.15.33:0/295783270 shutdown_connections 2024-05-03T18:37:55.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.949+0000 7f4459ff5b80 1 -- 172.21.15.33:0/295783270 wait complete. 2024-05-03T18:37:55.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.949+0000 7f4459ff5b80 20 client.26179 trim_cache size 0 max 16384 2024-05-03T18:37:55.950 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-03T18:37:55.950 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-05-03T18:37:55.950 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-05-03T18:37:55.950 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-03T18:37:55.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.950+0000 7f444f600700 1 -- 172.21.15.33:0/4260628646 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f44300b2b20 msgr2=0x7f44300b4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:55.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.950+0000 7f444f600700 1 --2- 172.21.15.33:0/4260628646 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f44300b2b20 0x7f44300b4fe0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f444811f3e0 tx=0x7f443400b410 comp rx=0 tx=0).stop 2024-05-03T18:37:55.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.950+0000 7f444f600700 1 -- 172.21.15.33:0/4260628646 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f444810a9e0 msgr2=0x7f44481ae6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:55.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.950+0000 7f444f600700 1 --2- 172.21.15.33:0/4260628646 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f444810a9e0 0x7f44481ae6d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f443c000c00 tx=0x7f443c009580 comp rx=0 tx=0).stop 2024-05-03T18:37:55.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.950+0000 7f444f600700 1 -- 172.21.15.33:0/4260628646 shutdown_connections 2024-05-03T18:37:55.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.950+0000 7f444f600700 1 --2- 172.21.15.33:0/4260628646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f444811cc80 0x7f44481a8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:55.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.950+0000 7f444f600700 1 --2- 172.21.15.33:0/4260628646 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f44300b2b20 0x7f44300b4fe0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:55.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.950+0000 7f444f600700 1 --2- 172.21.15.33:0/4260628646 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f444810a9e0 0x7f44481ae6d0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:55.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.950+0000 7f444f600700 1 --2- 172.21.15.33:0/4260628646 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4448111c80 0x7f44481aec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:55.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.950+0000 7f444f600700 1 -- 172.21.15.33:0/4260628646 >> 172.21.15.33:0/4260628646 conn(0x7f444806e610 msgr2=0x7f444811ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:37:55.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.951+0000 7f444f600700 1 -- 172.21.15.33:0/4260628646 shutdown_connections 2024-05-03T18:37:55.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:37:55.951+0000 7f444f600700 1 -- 172.21.15.33:0/4260628646 wait complete. 2024-05-03T18:37:56.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:37:56 smithi033 ceph-mon[28330]: pgmap v333: 129 pgs: 129 active+clean; 2.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 4.2 MiB/s wr, 460 op/s 2024-05-03T18:37:56.547 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:37:56 smithi099 ceph-mon[31658]: pgmap v333: 129 pgs: 129 active+clean; 2.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 4.2 MiB/s wr, 460 op/s 2024-05-03T18:37:56.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:37:56 smithi150 ceph-mon[32492]: pgmap v333: 129 pgs: 129 active+clean; 2.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 4.2 MiB/s wr, 460 op/s 2024-05-03T18:37:57.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:37:57 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:37:57.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:37:57 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:37:57.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:37:57 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:37:57.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:37:57 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:37:57.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:37:57 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:37:57.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:37:57 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:37:57.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:37:57 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:37:57.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:37:57 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:37:57.888 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:37:57 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:37:57.889 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:37:57 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:37:57.889 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:37:57 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:37:57.889 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:37:57 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:37:58.798 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:37:58 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:37:58.798 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:37:58 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:37:58.798 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:37:58 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:37:58.798 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:37:58 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:37:58.798 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:37:58 smithi099 ceph-mon[31658]: Adjusting osd_memory_target on smithi150 to 176.4M 2024-05-03T18:37:58.798 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:37:58 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:37:58.798 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:37:58 smithi099 ceph-mon[31658]: Unable to set osd_memory_target on smithi150 to 185064857: error parsing value: Value '185064857' is below minimum 939524096 2024-05-03T18:37:58.799 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:37:58 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:37:58.799 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:37:58 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:37:58.799 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:37:58 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:37:58.799 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:37:58 smithi099 ceph-mon[31658]: Adjusting osd_memory_target on smithi099 to 176.4M 2024-05-03T18:37:58.799 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:37:58 smithi099 ceph-mon[31658]: Unable to set osd_memory_target on smithi099 to 185040486: error parsing value: Value '185040486' is below minimum 939524096 2024-05-03T18:37:58.799 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:37:58 smithi099 ceph-mon[31658]: pgmap v334: 129 pgs: 129 active+clean; 2.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 4.2 MiB/s wr, 461 op/s 2024-05-03T18:37:58.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:37:58 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:37:58.884 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:37:58 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:37:58.884 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:37:58 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:37:58.884 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:37:58 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:37:58.884 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:37:58 smithi150 ceph-mon[32492]: Adjusting osd_memory_target on smithi150 to 176.4M 2024-05-03T18:37:58.884 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:37:58 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:37:58.884 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:37:58 smithi150 ceph-mon[32492]: Unable to set osd_memory_target on smithi150 to 185064857: error parsing value: Value '185064857' is below minimum 939524096 2024-05-03T18:37:58.884 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:37:58 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:37:58.884 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:37:58 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:37:58.884 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:37:58 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:37:58.884 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:37:58 smithi150 ceph-mon[32492]: Adjusting osd_memory_target on smithi099 to 176.4M 2024-05-03T18:37:58.884 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:37:58 smithi150 ceph-mon[32492]: Unable to set osd_memory_target on smithi099 to 185040486: error parsing value: Value '185040486' is below minimum 939524096 2024-05-03T18:37:58.884 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:37:58 smithi150 ceph-mon[32492]: pgmap v334: 129 pgs: 129 active+clean; 2.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 4.2 MiB/s wr, 461 op/s 2024-05-03T18:37:58.913 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:37:58 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:37:58.913 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:37:58 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:37:58.913 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:37:58 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:37:58.913 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:37:58 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:37:58.913 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:37:58 smithi033 ceph-mon[28330]: Adjusting osd_memory_target on smithi150 to 176.4M 2024-05-03T18:37:58.913 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:37:58 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:37:58.913 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:37:58 smithi033 ceph-mon[28330]: Unable to set osd_memory_target on smithi150 to 185064857: error parsing value: Value '185064857' is below minimum 939524096 2024-05-03T18:37:58.913 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:37:58 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:37:58.913 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:37:58 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:37:58.913 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:37:58 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:37:58.913 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:37:58 smithi033 ceph-mon[28330]: Adjusting osd_memory_target on smithi099 to 176.4M 2024-05-03T18:37:58.913 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:37:58 smithi033 ceph-mon[28330]: Unable to set osd_memory_target on smithi099 to 185040486: error parsing value: Value '185040486' is below minimum 939524096 2024-05-03T18:37:58.914 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:37:58 smithi033 ceph-mon[28330]: pgmap v334: 129 pgs: 129 active+clean; 2.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 4.2 MiB/s wr, 461 op/s 2024-05-03T18:37:59.701 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-03T18:37:59.702 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T18:38:00.110 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 405.0 s, 583.8 tps, lat 170.531 ms stddev 57.952 2024-05-03T18:38:00.478 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:38:00.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:38:00 smithi033 ceph-mon[28330]: pgmap v335: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 6.3 MiB/s wr, 693 op/s 2024-05-03T18:38:00.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:38:00 smithi099 ceph-mon[31658]: pgmap v335: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 6.3 MiB/s wr, 693 op/s 2024-05-03T18:38:00.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:38:00 smithi150 ceph-mon[32492]: pgmap v335: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 6.3 MiB/s wr, 693 op/s 2024-05-03T18:38:02.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:38:02 smithi033 ceph-mon[28330]: pgmap v336: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 4.3 MiB/s wr, 474 op/s 2024-05-03T18:38:02.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:38:02 smithi099 ceph-mon[31658]: pgmap v336: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 4.3 MiB/s wr, 474 op/s 2024-05-03T18:38:02.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:38:02 smithi150 ceph-mon[32492]: pgmap v336: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 4.3 MiB/s wr, 474 op/s 2024-05-03T18:38:04.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:38:04 smithi033 ceph-mon[28330]: pgmap v337: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 6.4 MiB/s wr, 714 op/s 2024-05-03T18:38:04.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:38:04 smithi099 ceph-mon[31658]: pgmap v337: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 6.4 MiB/s wr, 714 op/s 2024-05-03T18:38:04.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:38:04 smithi150 ceph-mon[32492]: pgmap v337: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 6.4 MiB/s wr, 714 op/s 2024-05-03T18:38:05.110 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 410.0 s, 480.6 tps, lat 205.239 ms stddev 180.390 2024-05-03T18:38:06.422 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:38:06 smithi033 ceph-mon[28330]: pgmap v338: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 4.2 MiB/s wr, 473 op/s 2024-05-03T18:38:06.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:38:06 smithi099 ceph-mon[31658]: pgmap v338: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 4.2 MiB/s wr, 473 op/s 2024-05-03T18:38:06.634 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:38:06 smithi150 ceph-mon[32492]: pgmap v338: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 4.2 MiB/s wr, 473 op/s 2024-05-03T18:38:07.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:07.310+0000 7f0a25a00700 1 -- 172.21.15.33:0/1677612402 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a20102d60 msgr2=0x7f0a2010f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:07.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:07.311+0000 7f0a25a00700 1 --2- 172.21.15.33:0/1677612402 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a20102d60 0x7f0a2010f250 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0a18009f90 tx=0x7f0a1800b330 comp rx=0 tx=0).stop 2024-05-03T18:38:07.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:07.311+0000 7f0a25a00700 1 -- 172.21.15.33:0/1677612402 shutdown_connections 2024-05-03T18:38:07.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:07.312+0000 7f0a25a00700 1 --2- 172.21.15.33:0/1677612402 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a20102d60 0x7f0a2010f250 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:07.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:07.312+0000 7f0a25a00700 1 --2- 172.21.15.33:0/1677612402 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a20108410 0x7f0a201087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:07.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:07.312+0000 7f0a25a00700 1 --2- 172.21.15.33:0/1677612402 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a201023a0 0x7f0a20102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:07.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:07.312+0000 7f0a25a00700 1 -- 172.21.15.33:0/1677612402 >> 172.21.15.33:0/1677612402 conn(0x7f0a20079350 msgr2=0x7f0a20079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:38:07.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:07.313+0000 7f0a25a00700 1 -- 172.21.15.33:0/1677612402 shutdown_connections 2024-05-03T18:38:07.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:07.313+0000 7f0a25a00700 1 -- 172.21.15.33:0/1677612402 wait complete. 2024-05-03T18:38:07.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:07.315+0000 7f0a25a00700 1 Processor -- start 2024-05-03T18:38:07.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:07.316+0000 7f0a25a00700 1 -- start start 2024-05-03T18:38:07.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:07.317+0000 7f0a25a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a201023a0 0x7f0a201a0080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:07.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:07.317+0000 7f0a25a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a20102d60 0x7f0a201a05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:07.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:07.317+0000 7f0a25a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a20108410 0x7f0a201a4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:07.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:07.317+0000 7f0a25a00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0a20111340 con 0x7f0a201023a0 2024-05-03T18:38:07.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:07.317+0000 7f0a25a00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0a201111c0 con 0x7f0a20102d60 2024-05-03T18:38:07.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:07.317+0000 7f0a25a00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f0a201114c0 con 0x7f0a20108410 2024-05-03T18:38:07.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:07.317+0000 7f0a1ea00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a201023a0 0x7f0a201a0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:07.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:07.318+0000 7f0a1ea00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a201023a0 0x7f0a201a0080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47326/0 (socket says 172.21.15.33:47326) 2024-05-03T18:38:07.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:07.318+0000 7f0a1ea00700 1 -- 172.21.15.33:0/2716089301 learned_addr learned my addr 172.21.15.33:0/2716089301 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:38:07.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:07.318+0000 7f0a16000700 1 --2- 172.21.15.33:0/2716089301 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a20102d60 0x7f0a201a05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:07.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:07.319+0000 7f0a1f400700 1 --2- 172.21.15.33:0/2716089301 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a20108410 0x7f0a201a4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:07.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:07.319+0000 7f0a1ea00700 1 -- 172.21.15.33:0/2716089301 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a20108410 msgr2=0x7f0a201a4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:07.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:07.319+0000 7f0a1ea00700 1 --2- 172.21.15.33:0/2716089301 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a20108410 0x7f0a201a4950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:07.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:07.320+0000 7f0a1ea00700 1 -- 172.21.15.33:0/2716089301 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a20102d60 msgr2=0x7f0a201a05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:07.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:07.320+0000 7f0a1ea00700 1 --2- 172.21.15.33:0/2716089301 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a20102d60 0x7f0a201a05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:07.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:07.320+0000 7f0a1ea00700 1 -- 172.21.15.33:0/2716089301 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a201a5090 con 0x7f0a201023a0 2024-05-03T18:38:07.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:07.320+0000 7f0a16000700 1 --2- 172.21.15.33:0/2716089301 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a20102d60 0x7f0a201a05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:38:07.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:07.321+0000 7f0a1ea00700 1 --2- 172.21.15.33:0/2716089301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a201023a0 0x7f0a201a0080 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f0a1000e9c0 tx=0x7f0a1000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:38:07.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:07.321+0000 7f0a17e00700 1 -- 172.21.15.33:0/2716089301 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a1000cbd0 con 0x7f0a201023a0 2024-05-03T18:38:07.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:07.322+0000 7f0a17e00700 1 -- 172.21.15.33:0/2716089301 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0a1000cd70 con 0x7f0a201023a0 2024-05-03T18:38:07.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:07.322+0000 7f0a17e00700 1 -- 172.21.15.33:0/2716089301 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a100054e0 con 0x7f0a201023a0 2024-05-03T18:38:07.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:07.322+0000 7f0a25a00700 1 -- 172.21.15.33:0/2716089301 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a201a5380 con 0x7f0a201023a0 2024-05-03T18:38:07.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:07.322+0000 7f0a25a00700 1 -- 172.21.15.33:0/2716089301 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a201acc00 con 0x7f0a201023a0 2024-05-03T18:38:07.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:07.325+0000 7f0a25a00700 1 -- 172.21.15.33:0/2716089301 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09fc005330 con 0x7f0a201023a0 2024-05-03T18:38:07.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:07.329+0000 7f0a17e00700 1 -- 172.21.15.33:0/2716089301 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f0a10004750 con 0x7f0a201023a0 2024-05-03T18:38:07.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:07.331+0000 7f0a17e00700 1 --2- 172.21.15.33:0/2716089301 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f0a040b2a50 0x7f0a040b4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:07.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:07.332+0000 7f0a17e00700 1 -- 172.21.15.33:0/2716089301 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9518+0+0 (secure 0 0 0) 0x7f0a100e29b0 con 0x7f0a201023a0 2024-05-03T18:38:07.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:07.334+0000 7f0a16000700 1 --2- 172.21.15.33:0/2716089301 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f0a040b2a50 0x7f0a040b4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:07.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:07.335+0000 7f0a16000700 1 --2- 172.21.15.33:0/2716089301 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f0a040b2a50 0x7f0a040b4f10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0a08005fd0 tx=0x7f0a08005ee0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:38:07.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:07.342+0000 7f0a17e00700 1 -- 172.21.15.33:0/2716089301 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f0a100a9fb0 con 0x7f0a201023a0 2024-05-03T18:38:07.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:07.911+0000 7f0a25a00700 1 -- 172.21.15.33:0/2716089301 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f09fc0058f0 con 0x7f0a201023a0 2024-05-03T18:38:07.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:07.917+0000 7f0a17e00700 1 -- 172.21.15.33:0/2716089301 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11801 (secure 0 0 0) 0x7f0a100a98b0 con 0x7f0a201023a0 2024-05-03T18:38:07.920 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:38:07.920 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1327393503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1327393503},{"type":"v1","addr":"172.21.15.33:6835","nonce":1327393503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/2461472948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":2461472948},{"type":"v1","addr":"172.21.15.33:6837","nonce":2461472948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14559,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/442269281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":442269281},{"type":"v1","addr":"172.21.15.33:6839","nonce":442269281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14571,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/645571756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":645571756},{"type":"v1","addr":"172.21.15.33:6841","nonce":645571756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/203598909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":203598909},{"type":"v1","addr":"172.21.15.150:6833","nonce":203598909}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/4029842493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":4029842493},{"type":"v1","addr":"172.21.15.150:6835","nonce":4029842493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3463664423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3463664423},{"type":"v1","addr":"172.21.15.150:6837","nonce":3463664423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24481,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6839/4051648018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":4051648018},{"type":"v1","addr":"172.21.15.150:6839","nonce":4051648018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34418,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1865105479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1865105479},{"type":"v1","addr":"172.21.15.99:6833","nonce":1865105479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34445,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/1206393914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1206393914},{"type":"v1","addr":"172.21.15.99:6835","nonce":1206393914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-03T18:22:28.837091+0000","modified":"2024-05-03T18:23:49.758398+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34508},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34472":{"gid":34472,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6837/3360970342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3360970342},{"type":"v1","addr":"172.21.15.99:6837","nonce":3360970342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34508":{"gid":34508,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/919907308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":919907308},{"type":"v1","addr":"172.21.15.99:6839","nonce":919907308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34508,"qdb_cluster":[34508]},"id":1}]} 2024-05-03T18:38:07.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:07.922+0000 7f0a25a00700 1 -- 172.21.15.33:0/2716089301 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f0a040b2a50 msgr2=0x7f0a040b4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:07.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:07.922+0000 7f0a25a00700 1 --2- 172.21.15.33:0/2716089301 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f0a040b2a50 0x7f0a040b4f10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0a08005fd0 tx=0x7f0a08005ee0 comp rx=0 tx=0).stop 2024-05-03T18:38:07.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:07.922+0000 7f0a25a00700 1 -- 172.21.15.33:0/2716089301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a201023a0 msgr2=0x7f0a201a0080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:07.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:07.922+0000 7f0a25a00700 1 --2- 172.21.15.33:0/2716089301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a201023a0 0x7f0a201a0080 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f0a1000e9c0 tx=0x7f0a1000ecd0 comp rx=0 tx=0).stop 2024-05-03T18:38:07.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:07.923+0000 7f0a25a00700 1 -- 172.21.15.33:0/2716089301 shutdown_connections 2024-05-03T18:38:07.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:07.923+0000 7f0a25a00700 1 --2- 172.21.15.33:0/2716089301 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a20108410 0x7f0a201a4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:07.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:07.923+0000 7f0a25a00700 1 --2- 172.21.15.33:0/2716089301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a201023a0 0x7f0a201a0080 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:07.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:07.923+0000 7f0a25a00700 1 --2- 172.21.15.33:0/2716089301 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f0a040b2a50 0x7f0a040b4f10 secure :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0a08005fd0 tx=0x7f0a08005ee0 comp rx=0 tx=0).stop 2024-05-03T18:38:07.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:07.923+0000 7f0a25a00700 1 --2- 172.21.15.33:0/2716089301 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a20102d60 0x7f0a201a05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:07.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:07.923+0000 7f0a25a00700 1 -- 172.21.15.33:0/2716089301 >> 172.21.15.33:0/2716089301 conn(0x7f0a20079350 msgr2=0x7f0a20101010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:38:07.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:07.924+0000 7f0a25a00700 1 -- 172.21.15.33:0/2716089301 shutdown_connections 2024-05-03T18:38:07.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:07.925+0000 7f0a25a00700 1 -- 172.21.15.33:0/2716089301 wait complete. 2024-05-03T18:38:07.935 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 21 2024-05-03T18:38:08.738 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:38:08 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:38:08.738 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:38:08 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:38:08.738 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:38:08 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/2716089301' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:38:08.738 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:38:08 smithi033 ceph-mon[28330]: pgmap v339: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 4.2 MiB/s wr, 473 op/s 2024-05-03T18:38:08.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:38:08 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:38:08.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:38:08 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:38:08.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:38:08 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/2716089301' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:38:08.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:38:08 smithi099 ceph-mon[31658]: pgmap v339: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 4.2 MiB/s wr, 473 op/s 2024-05-03T18:38:08.884 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:38:08 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:38:08.884 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:38:08 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:38:08.884 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:38:08 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/2716089301' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:38:08.884 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:38:08 smithi150 ceph-mon[32492]: pgmap v339: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 4.2 MiB/s wr, 473 op/s 2024-05-03T18:38:09.361 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-03T18:38:09.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:38:09 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:38:09.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:38:09 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:38:09.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:38:09 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:38:09.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:38:09 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:38:09.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:38:09 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:38:09.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:38:09 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:38:09.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:38:09 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:38:10.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:38:09 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:38:10.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:38:09 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:38:10.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:38:09 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:38:10.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:38:09 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:38:10.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:38:09 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:38:10.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:38:09 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:38:10.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:38:09 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:38:10.110 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 415.0 s, 524.0 tps, lat 194.081 ms stddev 105.890 2024-05-03T18:38:10.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:38:09 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:38:10.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:38:09 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:38:10.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:38:09 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:38:10.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:38:09 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:38:10.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:38:09 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:38:10.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:38:09 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:38:10.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:38:09 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:38:10.166 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:38:10.720 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:38:10 smithi033 ceph-mon[28330]: Adjusting osd_memory_target on smithi033 to 177.5M 2024-05-03T18:38:10.720 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:38:10 smithi033 ceph-mon[28330]: Unable to set osd_memory_target on smithi033 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-05-03T18:38:10.720 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:38:10 smithi033 ceph-mon[28330]: pgmap v340: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 6.1 MiB/s wr, 684 op/s 2024-05-03T18:38:11.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:38:10 smithi099 ceph-mon[31658]: Adjusting osd_memory_target on smithi033 to 177.5M 2024-05-03T18:38:11.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:38:10 smithi099 ceph-mon[31658]: Unable to set osd_memory_target on smithi033 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-05-03T18:38:11.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:38:10 smithi099 ceph-mon[31658]: pgmap v340: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 6.1 MiB/s wr, 684 op/s 2024-05-03T18:38:11.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:38:10 smithi150 ceph-mon[32492]: Adjusting osd_memory_target on smithi033 to 177.5M 2024-05-03T18:38:11.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:38:10 smithi150 ceph-mon[32492]: Unable to set osd_memory_target on smithi033 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-05-03T18:38:11.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:38:10 smithi150 ceph-mon[32492]: pgmap v340: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 6.1 MiB/s wr, 684 op/s 2024-05-03T18:38:13.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:38:12 smithi099 ceph-mon[31658]: osdmap e99: 12 total, 12 up, 12 in 2024-05-03T18:38:13.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:38:12 smithi099 ceph-mon[31658]: pgmap v342: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 4.8 MiB/s wr, 541 op/s 2024-05-03T18:38:13.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:38:12 smithi150 ceph-mon[32492]: osdmap e99: 12 total, 12 up, 12 in 2024-05-03T18:38:13.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:38:12 smithi150 ceph-mon[32492]: pgmap v342: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 4.8 MiB/s wr, 541 op/s 2024-05-03T18:38:13.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:38:12 smithi033 ceph-mon[28330]: osdmap e99: 12 total, 12 up, 12 in 2024-05-03T18:38:13.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:38:12 smithi033 ceph-mon[28330]: pgmap v342: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 4.8 MiB/s wr, 541 op/s 2024-05-03T18:38:14.381 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:38:14 smithi150 ceph-mon[32492]: pgmap v343: 129 pgs: 129 active+clean; 2.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 4.5 MiB/s wr, 503 op/s 2024-05-03T18:38:14.443 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:38:14 smithi033 ceph-mon[28330]: pgmap v343: 129 pgs: 129 active+clean; 2.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 4.5 MiB/s wr, 503 op/s 2024-05-03T18:38:14.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:38:14 smithi099 ceph-mon[31658]: pgmap v343: 129 pgs: 129 active+clean; 2.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 4.5 MiB/s wr, 503 op/s 2024-05-03T18:38:15.111 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 420.0 s, 588.9 tps, lat 169.002 ms stddev 94.098 2024-05-03T18:38:15.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.154+0000 7fbf19c00700 1 -- 172.21.15.33:0/2460021346 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf1406b6f0 msgr2=0x7fbf1406bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:15.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.154+0000 7fbf19c00700 1 --2- 172.21.15.33:0/2460021346 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf1406b6f0 0x7fbf1406bb50 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fbf040098f0 tx=0x7fbf04009c00 comp rx=0 tx=0).stop 2024-05-03T18:38:15.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.156+0000 7fbf19c00700 1 -- 172.21.15.33:0/2460021346 shutdown_connections 2024-05-03T18:38:15.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.156+0000 7fbf19c00700 1 --2- 172.21.15.33:0/2460021346 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbf14105ee0 0x7fbf14106370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:15.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.157+0000 7fbf19c00700 1 --2- 172.21.15.33:0/2460021346 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf141083d0 0x7fbf1406b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:15.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.157+0000 7fbf19c00700 1 --2- 172.21.15.33:0/2460021346 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf1406b6f0 0x7fbf1406bb50 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:15.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.157+0000 7fbf19c00700 1 -- 172.21.15.33:0/2460021346 >> 172.21.15.33:0/2460021346 conn(0x7fbf14079350 msgr2=0x7fbf14079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:38:15.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.157+0000 7fbf19c00700 1 -- 172.21.15.33:0/2460021346 shutdown_connections 2024-05-03T18:38:15.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.157+0000 7fbf19c00700 1 -- 172.21.15.33:0/2460021346 wait complete. 2024-05-03T18:38:15.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.160+0000 7fbf19c00700 1 Processor -- start 2024-05-03T18:38:15.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.160+0000 7fbf19c00700 1 -- start start 2024-05-03T18:38:15.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.160+0000 7fbf19c00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf1406b6f0 0x7fbf141a0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:15.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.160+0000 7fbf19c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf14105ee0 0x7fbf141a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:15.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.160+0000 7fbf19c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbf141083d0 0x7fbf141a49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:15.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.161+0000 7fbf12a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf1406b6f0 0x7fbf141a0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:15.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.162+0000 7fbf12000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf14105ee0 0x7fbf141a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:15.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.162+0000 7fbf12000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf14105ee0 0x7fbf141a0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:35214/0 (socket says 172.21.15.33:35214) 2024-05-03T18:38:15.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.162+0000 7fbf12000700 1 -- 172.21.15.33:0/3187238000 learned_addr learned my addr 172.21.15.33:0/3187238000 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:38:15.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.162+0000 7fbf19c00700 1 -- 172.21.15.33:0/3187238000 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbf14119b80 con 0x7fbf1406b6f0 2024-05-03T18:38:15.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.162+0000 7fbf19c00700 1 -- 172.21.15.33:0/3187238000 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbf14119a00 con 0x7fbf141083d0 2024-05-03T18:38:15.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.162+0000 7fbf19c00700 1 -- 172.21.15.33:0/3187238000 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fbf14119d00 con 0x7fbf14105ee0 2024-05-03T18:38:15.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.162+0000 7fbf13400700 1 --2- 172.21.15.33:0/3187238000 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbf141083d0 0x7fbf141a49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:15.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.163+0000 7fbf12000700 1 -- 172.21.15.33:0/3187238000 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbf141083d0 msgr2=0x7fbf141a49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:15.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.163+0000 7fbf12000700 1 --2- 172.21.15.33:0/3187238000 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbf141083d0 0x7fbf141a49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:15.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.163+0000 7fbf12000700 1 -- 172.21.15.33:0/3187238000 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf1406b6f0 msgr2=0x7fbf141a0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:15.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.163+0000 7fbf12000700 1 --2- 172.21.15.33:0/3187238000 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf1406b6f0 0x7fbf141a0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:15.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.163+0000 7fbf12000700 1 -- 172.21.15.33:0/3187238000 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf141a5110 con 0x7fbf14105ee0 2024-05-03T18:38:15.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.167+0000 7fbf12000700 1 --2- 172.21.15.33:0/3187238000 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf14105ee0 0x7fbf141a0640 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fbf0400c010 tx=0x7fbf0400b560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:38:15.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.167+0000 7fbf0b400700 1 -- 172.21.15.33:0/3187238000 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf0400b780 con 0x7fbf14105ee0 2024-05-03T18:38:15.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.168+0000 7fbf19c00700 1 -- 172.21.15.33:0/3187238000 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf141a53a0 con 0x7fbf14105ee0 2024-05-03T18:38:15.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.168+0000 7fbf19c00700 1 -- 172.21.15.33:0/3187238000 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf141acc80 con 0x7fbf14105ee0 2024-05-03T18:38:15.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.168+0000 7fbf0b400700 1 -- 172.21.15.33:0/3187238000 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbf040094c0 con 0x7fbf14105ee0 2024-05-03T18:38:15.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.168+0000 7fbf0b400700 1 -- 172.21.15.33:0/3187238000 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf0400b780 con 0x7fbf14105ee0 2024-05-03T18:38:15.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.176+0000 7fbf0b400700 1 -- 172.21.15.33:0/3187238000 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fbf04003680 con 0x7fbf14105ee0 2024-05-03T18:38:15.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.179+0000 7fbf0b400700 1 --2- 172.21.15.33:0/3187238000 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fbef80b2e40 0x7fbef80b5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:15.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.180+0000 7fbf0b400700 1 -- 172.21.15.33:0/3187238000 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9606+0+0 (secure 0 0 0) 0x7fbf0401d070 con 0x7fbf14105ee0 2024-05-03T18:38:15.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.180+0000 7fbf12a00700 1 --2- 172.21.15.33:0/3187238000 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fbef80b2e40 0x7fbef80b5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:15.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.386+0000 7fbf12a00700 1 --2- 172.21.15.33:0/3187238000 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fbef80b2e40 0x7fbef80b5300 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fbefc005950 tx=0x7fbefc0058e0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:38:15.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.388+0000 7fbf244d4b80 1 Processor -- start 2024-05-03T18:38:15.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.389+0000 7fbf244d4b80 1 -- start start 2024-05-03T18:38:15.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.389+0000 7fbf244d4b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ca614907b0 0x55ca61490b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:15.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.390+0000 7fbf244d4b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ca614910d0 0x55ca6148bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:15.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.391+0000 7fbf244d4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ca6148c5d0 0x55ca6148ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:15.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.391+0000 7fbf12a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ca614907b0 0x55ca61490b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:15.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.391+0000 7fbf12a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ca614907b0 0x55ca61490b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:35220/0 (socket says 172.21.15.33:35220) 2024-05-03T18:38:15.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.393+0000 7fbf12a00700 1 -- 172.21.15.33:0/676648115 learned_addr learned my addr 172.21.15.33:0/676648115 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:38:15.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.392+0000 7fbf13400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ca6148c5d0 0x55ca6148ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:15.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.392+0000 7fbf12000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ca614910d0 0x55ca6148bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:15.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.394+0000 7fbf244d4b80 1 -- 172.21.15.33:0/676648115 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ca613dea20 con 0x55ca614910d0 2024-05-03T18:38:15.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.394+0000 7fbf244d4b80 1 -- 172.21.15.33:0/676648115 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55ca6137c760 con 0x55ca6148c5d0 2024-05-03T18:38:15.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.394+0000 7fbf244d4b80 1 -- 172.21.15.33:0/676648115 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55ca613da9e0 con 0x55ca614907b0 2024-05-03T18:38:15.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.395+0000 7fbf12a00700 1 -- 172.21.15.33:0/676648115 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ca6148c5d0 msgr2=0x55ca6148ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:38:15.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.395+0000 7fbf12a00700 1 --2- 172.21.15.33:0/676648115 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ca6148c5d0 0x55ca6148ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:15.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.395+0000 7fbf12a00700 1 -- 172.21.15.33:0/676648115 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ca614910d0 msgr2=0x55ca6148bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:38:15.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.395+0000 7fbf12a00700 1 --2- 172.21.15.33:0/676648115 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ca614910d0 0x55ca6148bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:15.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.395+0000 7fbf12a00700 1 -- 172.21.15.33:0/676648115 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca6132b8d0 con 0x55ca614907b0 2024-05-03T18:38:15.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.396+0000 7fbf12a00700 1 --2- 172.21.15.33:0/676648115 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ca614907b0 0x55ca61490b90 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fbefc00fcd0 tx=0x7fbefc012000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38f27a5d0ab5b398 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:38:15.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.396+0000 7fbf03e00700 1 -- 172.21.15.33:0/676648115 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbefc022070 con 0x55ca614907b0 2024-05-03T18:38:15.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.396+0000 7fbf03e00700 1 -- 172.21.15.33:0/676648115 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbefc011b90 con 0x55ca614907b0 2024-05-03T18:38:15.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.396+0000 7fbf03e00700 1 -- 172.21.15.33:0/676648115 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbefc01d610 con 0x55ca614907b0 2024-05-03T18:38:15.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.397+0000 7fbf244d4b80 1 -- 172.21.15.33:0/676648115 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ca614907b0 msgr2=0x55ca61490b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:15.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.397+0000 7fbf244d4b80 1 --2- 172.21.15.33:0/676648115 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ca614907b0 0x55ca61490b90 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fbefc00fcd0 tx=0x7fbefc012000 comp rx=0 tx=0).stop 2024-05-03T18:38:15.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.399+0000 7fbf244d4b80 1 -- 172.21.15.33:0/676648115 shutdown_connections 2024-05-03T18:38:15.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.399+0000 7fbf244d4b80 1 --2- 172.21.15.33:0/676648115 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ca6148c5d0 0x55ca6148ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:15.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.399+0000 7fbf244d4b80 1 --2- 172.21.15.33:0/676648115 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ca614907b0 0x55ca61490b90 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:15.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.399+0000 7fbf244d4b80 1 --2- 172.21.15.33:0/676648115 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ca614910d0 0x55ca6148bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:15.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.399+0000 7fbf244d4b80 1 -- 172.21.15.33:0/676648115 >> 172.21.15.33:0/676648115 conn(0x55ca613bab60 msgr2=0x55ca614867f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:38:15.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.399+0000 7fbf244d4b80 1 -- 172.21.15.33:0/676648115 shutdown_connections 2024-05-03T18:38:15.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.399+0000 7fbf244d4b80 1 -- 172.21.15.33:0/676648115 wait complete. 2024-05-03T18:38:15.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.400+0000 7fbf244d4b80 1 -- start start 2024-05-03T18:38:15.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.400+0000 7fbf244d4b80 1 Processor -- start 2024-05-03T18:38:15.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.402+0000 7fbf244d4b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ca6148c5d0 0x55ca61497e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:15.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.403+0000 7fbf244d4b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ca614907b0 0x55ca61495240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:15.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.403+0000 7fbf244d4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ca614910d0 0x55ca61495780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:15.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.403+0000 7fbf244d4b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ca61498cf0 con 0x55ca614907b0 2024-05-03T18:38:15.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.403+0000 7fbf244d4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55ca61344b20 con 0x55ca614910d0 2024-05-03T18:38:15.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.403+0000 7fbf244d4b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55ca61498e70 con 0x55ca6148c5d0 2024-05-03T18:38:15.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.403+0000 7fbf244d4b80 10 client.0 _ll_register_callbacks cb 0x55ca61385c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:38:15.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.404+0000 7fbf12a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ca6148c5d0 0x55ca61497e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:15.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.405+0000 7fbf12000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ca614907b0 0x55ca61495240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:15.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.405+0000 7fbf12a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ca6148c5d0 0x55ca61497e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:35232/0 (socket says 172.21.15.33:35232) 2024-05-03T18:38:15.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.405+0000 7fbf12a00700 1 -- 172.21.15.33:0/3464228416 learned_addr learned my addr 172.21.15.33:0/3464228416 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:38:15.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.405+0000 7fbf12a00700 1 -- 172.21.15.33:0/3464228416 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ca614910d0 msgr2=0x55ca61495780 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-03T18:38:15.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.405+0000 7fbf12a00700 1 --2- 172.21.15.33:0/3464228416 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ca614910d0 0x55ca61495780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:15.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.405+0000 7fbf12a00700 1 -- 172.21.15.33:0/3464228416 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ca614907b0 msgr2=0x55ca61495240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:38:15.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.405+0000 7fbf12a00700 1 --2- 172.21.15.33:0/3464228416 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ca614907b0 0x55ca61495240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:15.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.406+0000 7fbf12a00700 1 -- 172.21.15.33:0/3464228416 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca613a4420 con 0x55ca6148c5d0 2024-05-03T18:38:15.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.408+0000 7fbf12a00700 1 --2- 172.21.15.33:0/3464228416 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ca6148c5d0 0x55ca61497e50 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fbefc00b3f0 tx=0x7fbefc011a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7a187b4bd26a977 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:38:15.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.408+0000 7fbf00c00700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-03T18:38:15.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.408+0000 7fbf00c00700 1 -- 172.21.15.33:0/3464228416 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbefc047060 con 0x55ca6148c5d0 2024-05-03T18:38:15.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.409+0000 7fbf00c00700 1 -- 172.21.15.33:0/3464228416 <== mon.1 v2:172.21.15.150:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9606+0+0 (secure 0 0 0) 0x7fbefc01e510 con 0x55ca6148c5d0 2024-05-03T18:38:15.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.409+0000 7fbf244d4b80 1 -- 172.21.15.33:0/3464228416 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbefc011b90 con 0x55ca6148c5d0 2024-05-03T18:38:15.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.410+0000 7fbf00c00700 1 -- 172.21.15.33:0/3464228416 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fbefc047060 con 0x55ca6148c5d0 2024-05-03T18:38:15.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.410+0000 7fbf244d4b80 10 client.26197 fetch_fsmap learned FSMap version 21 2024-05-03T18:38:15.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.410+0000 7fbf244d4b80 1 -- 172.21.15.33:0/3464228416 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55ca614a3500 con 0x55ca6148c5d0 2024-05-03T18:38:15.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.411+0000 7fbf00c00700 1 -- 172.21.15.33:0/3464228416 <== mon.1 v2:172.21.15.150:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbefc011b90 con 0x55ca6148c5d0 2024-05-03T18:38:15.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.411+0000 7fbf00c00700 1 -- 172.21.15.33:0/3464228416 <== mon.1 v2:172.21.15.150:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fbefc027aa0 con 0x55ca6148c5d0 2024-05-03T18:38:15.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.415+0000 7fbf244d4b80 10 client.26197 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:38:15.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.415+0000 7fbf244d4b80 10 client.26197 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:38:15.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.415+0000 7fbf244d4b80 20 client.26197 populate_metadata read hostname 'smithi033' 2024-05-03T18:38:15.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.415+0000 7fbf244d4b80 1 --2- 172.21.15.33:0/3464228416 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55ca614a59d0 0x55ca614a7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:15.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.415+0000 7fbf244d4b80 4 client.26197 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:38:15.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.415+0000 7fbf244d4b80 1 -- 172.21.15.33:0/3464228416 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ca614591d0 con 0x55ca614a59d0 2024-05-03T18:38:15.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.415+0000 7fbf12000700 1 --2- 172.21.15.33:0/3464228416 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55ca614a59d0 0x55ca614a7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:15.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.417+0000 7fbf12000700 1 --2- 172.21.15.33:0/3464228416 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55ca614a59d0 0x55ca614a7dc0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29df456d5b11c867 server_cookie=713aabc140db66d3 in_seq=0 out_seq=0 2024-05-03T18:38:15.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.418+0000 7fbf00c00700 10 client.26197 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:38:15.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.419+0000 7fbf00c00700 1 -- 172.21.15.33:0/3464228416 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ca614591d0 con 0x55ca614a59d0 2024-05-03T18:38:15.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.419+0000 7fbf00c00700 10 client.26197 handle_command_reply: tid=0 2024-05-03T18:38:15.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.419+0000 7fbf244d4b80 1 client.26197 shutdown 2024-05-03T18:38:15.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.419+0000 7fbf244d4b80 1 -- 172.21.15.33:0/3464228416 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ca6148c5d0 msgr2=0x55ca61497e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:15.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.419+0000 7fbf244d4b80 1 --2- 172.21.15.33:0/3464228416 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ca6148c5d0 0x55ca61497e50 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fbefc00b3f0 tx=0x7fbefc011a40 comp rx=0 tx=0).stop 2024-05-03T18:38:15.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.419+0000 7fbf244d4b80 1 -- 172.21.15.33:0/3464228416 shutdown_connections 2024-05-03T18:38:15.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.419+0000 7fbf244d4b80 1 --2- 172.21.15.33:0/3464228416 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55ca614a59d0 0x55ca614a7dc0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:15.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.419+0000 7fbf244d4b80 1 --2- 172.21.15.33:0/3464228416 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ca614910d0 0x55ca61495780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:15.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.419+0000 7fbf244d4b80 1 --2- 172.21.15.33:0/3464228416 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ca6148c5d0 0x55ca61497e50 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:15.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.419+0000 7fbf244d4b80 1 --2- 172.21.15.33:0/3464228416 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ca614907b0 0x55ca61495240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:15.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.419+0000 7fbf244d4b80 1 -- 172.21.15.33:0/3464228416 >> 172.21.15.33:0/3464228416 conn(0x55ca613bab60 msgr2=0x55ca61491e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:38:15.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.419+0000 7fbf00c00700 0 client.26197 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:38:15.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.420+0000 7fbf244d4b80 1 -- 172.21.15.33:0/3464228416 shutdown_connections 2024-05-03T18:38:15.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.420+0000 7fbf244d4b80 1 -- 172.21.15.33:0/3464228416 wait complete. 2024-05-03T18:38:15.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.420+0000 7fbf244d4b80 20 client.26197 trim_cache size 0 max 16384 2024-05-03T18:38:15.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.455+0000 7fbf244d4b80 1 Processor -- start 2024-05-03T18:38:15.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.455+0000 7fbf244d4b80 1 -- start start 2024-05-03T18:38:15.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.456+0000 7fbf244d4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ca614907b0 0x55ca614a7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:15.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.456+0000 7fbf244d4b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ca614910d0 0x55ca61497660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:15.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.456+0000 7fbf244d4b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ca614a1e10 0x55ca61486640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:15.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.456+0000 7fbf244d4b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbef000cfd0 con 0x55ca614a1e10 2024-05-03T18:38:15.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.456+0000 7fbf244d4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbef000f830 con 0x55ca614907b0 2024-05-03T18:38:15.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.456+0000 7fbf244d4b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fbef000d620 con 0x55ca614910d0 2024-05-03T18:38:15.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.456+0000 7fbf12a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ca614907b0 0x55ca614a7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:15.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.456+0000 7fbf12a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ca614907b0 0x55ca614a7dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:39838/0 (socket says 172.21.15.33:39838) 2024-05-03T18:38:15.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.456+0000 7fbf12a00700 1 -- 172.21.15.33:0/872841115 learned_addr learned my addr 172.21.15.33:0/872841115 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:38:15.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.457+0000 7fbf12a00700 1 -- 172.21.15.33:0/872841115 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ca614910d0 msgr2=0x55ca61497660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:38:15.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.457+0000 7fbf12a00700 1 --2- 172.21.15.33:0/872841115 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ca614910d0 0x55ca61497660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:15.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.457+0000 7fbf12a00700 1 -- 172.21.15.33:0/872841115 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ca614a1e10 msgr2=0x55ca61486640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:38:15.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.457+0000 7fbf12a00700 1 --2- 172.21.15.33:0/872841115 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ca614a1e10 0x55ca61486640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:15.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.457+0000 7fbf12a00700 1 -- 172.21.15.33:0/872841115 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca614879c0 con 0x55ca614907b0 2024-05-03T18:38:15.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.457+0000 7fbf12a00700 1 --2- 172.21.15.33:0/872841115 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ca614907b0 0x55ca614a7dc0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fbefc00bfd0 tx=0x7fbefc005ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98fa5b0a54f718ff server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:38:15.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.458+0000 7fbf03400700 1 -- 172.21.15.33:0/872841115 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbefc015050 con 0x55ca614907b0 2024-05-03T18:38:15.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.458+0000 7fbf03400700 1 -- 172.21.15.33:0/872841115 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbefc011b60 con 0x55ca614907b0 2024-05-03T18:38:15.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.458+0000 7fbf03400700 1 -- 172.21.15.33:0/872841115 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbefc051080 con 0x55ca614907b0 2024-05-03T18:38:15.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.459+0000 7fbf244d4b80 1 -- 172.21.15.33:0/872841115 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ca614907b0 msgr2=0x55ca614a7dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:15.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.459+0000 7fbf244d4b80 1 --2- 172.21.15.33:0/872841115 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ca614907b0 0x55ca614a7dc0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fbefc00bfd0 tx=0x7fbefc005ce0 comp rx=0 tx=0).stop 2024-05-03T18:38:15.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.459+0000 7fbf244d4b80 1 -- 172.21.15.33:0/872841115 shutdown_connections 2024-05-03T18:38:15.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.459+0000 7fbf244d4b80 1 --2- 172.21.15.33:0/872841115 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ca614a1e10 0x55ca61486640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:15.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.459+0000 7fbf244d4b80 1 --2- 172.21.15.33:0/872841115 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ca614907b0 0x55ca614a7dc0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:15.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.459+0000 7fbf244d4b80 1 --2- 172.21.15.33:0/872841115 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ca614910d0 0x55ca61497660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:15.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.459+0000 7fbf244d4b80 1 -- 172.21.15.33:0/872841115 >> 172.21.15.33:0/872841115 conn(0x55ca613bab60 msgr2=0x55ca61485f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:38:15.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.460+0000 7fbf244d4b80 1 -- 172.21.15.33:0/872841115 shutdown_connections 2024-05-03T18:38:15.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.460+0000 7fbf244d4b80 1 -- 172.21.15.33:0/872841115 wait complete. 2024-05-03T18:38:15.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.461+0000 7fbf244d4b80 1 -- start start 2024-05-03T18:38:15.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.461+0000 7fbf244d4b80 1 Processor -- start 2024-05-03T18:38:15.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.462+0000 7fbf244d4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ca614907b0 0x55ca6149e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:15.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.462+0000 7fbf244d4b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ca614910d0 0x55ca614927b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:15.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.462+0000 7fbf244d4b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ca614a1e10 0x55ca61492cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:15.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.462+0000 7fbf244d4b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbef000dcf0 con 0x55ca614910d0 2024-05-03T18:38:15.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.462+0000 7fbf244d4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbef000f160 con 0x55ca614907b0 2024-05-03T18:38:15.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.462+0000 7fbf244d4b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fbef000e3c0 con 0x55ca614a1e10 2024-05-03T18:38:15.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.462+0000 7fbf244d4b80 10 client.0 _ll_register_callbacks cb 0x55ca61385c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:38:15.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.463+0000 7fbf12000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ca614910d0 0x55ca614927b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:15.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.464+0000 7fbf12a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ca614907b0 0x55ca6149e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:15.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.464+0000 7fbf12a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ca614907b0 0x55ca6149e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:39846/0 (socket says 172.21.15.33:39846) 2024-05-03T18:38:15.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.464+0000 7fbf12a00700 1 -- 172.21.15.33:0/2890883520 learned_addr learned my addr 172.21.15.33:0/2890883520 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:38:15.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.465+0000 7fbf13400700 1 --2- 172.21.15.33:0/2890883520 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ca614a1e10 0x55ca61492cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:15.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.465+0000 7fbf13400700 1 -- 172.21.15.33:0/2890883520 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ca614907b0 msgr2=0x55ca6149e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:38:15.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.465+0000 7fbf13400700 1 --2- 172.21.15.33:0/2890883520 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ca614907b0 0x55ca6149e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:15.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.465+0000 7fbf13400700 1 -- 172.21.15.33:0/2890883520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ca614910d0 msgr2=0x55ca614927b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:38:15.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.465+0000 7fbf13400700 1 --2- 172.21.15.33:0/2890883520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ca614910d0 0x55ca614927b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:15.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.465+0000 7fbf13400700 1 -- 172.21.15.33:0/2890883520 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca61489900 con 0x55ca614a1e10 2024-05-03T18:38:15.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.467+0000 7fbf13400700 1 --2- 172.21.15.33:0/2890883520 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ca614a1e10 0x55ca61492cf0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c014dd0 tx=0x7fbf0c012580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf6a084216e33c42 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:38:15.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.467+0000 7fbf00c00700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-03T18:38:15.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.468+0000 7fbf00c00700 1 -- 172.21.15.33:0/2890883520 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf0c012b10 con 0x55ca614a1e10 2024-05-03T18:38:15.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.468+0000 7fbf00c00700 1 -- 172.21.15.33:0/2890883520 <== mon.1 v2:172.21.15.150:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9606+0+0 (secure 0 0 0) 0x7fbf0c012cb0 con 0x55ca614a1e10 2024-05-03T18:38:15.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.468+0000 7fbf244d4b80 1 -- 172.21.15.33:0/2890883520 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbefc011b60 con 0x55ca614a1e10 2024-05-03T18:38:15.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.468+0000 7fbf00c00700 1 -- 172.21.15.33:0/2890883520 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fbf0c028050 con 0x55ca614a1e10 2024-05-03T18:38:15.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.468+0000 7fbf244d4b80 10 client.26200 fetch_fsmap learned FSMap version 21 2024-05-03T18:38:15.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.468+0000 7fbf244d4b80 1 -- 172.21.15.33:0/2890883520 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55ca61510170 con 0x55ca614a1e10 2024-05-03T18:38:15.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.469+0000 7fbf00c00700 1 -- 172.21.15.33:0/2890883520 <== mon.1 v2:172.21.15.150:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbefc011b60 con 0x55ca614a1e10 2024-05-03T18:38:15.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.469+0000 7fbf00c00700 1 -- 172.21.15.33:0/2890883520 <== mon.1 v2:172.21.15.150:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fbf0c01f480 con 0x55ca614a1e10 2024-05-03T18:38:15.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.469+0000 7fbf244d4b80 10 client.26200 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:38:15.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.469+0000 7fbf244d4b80 10 client.26200 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:38:15.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.469+0000 7fbf244d4b80 20 client.26200 populate_metadata read hostname 'smithi033' 2024-05-03T18:38:15.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.469+0000 7fbf244d4b80 1 --2- 172.21.15.33:0/2890883520 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55ca61510870 0x55ca61512c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:15.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.469+0000 7fbf244d4b80 4 client.26200 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-03T18:38:15.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.469+0000 7fbf244d4b80 1 -- 172.21.15.33:0/2890883520 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ca613ec400 con 0x55ca61510870 2024-05-03T18:38:15.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.470+0000 7fbf12a00700 1 --2- 172.21.15.33:0/2890883520 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55ca61510870 0x55ca61512c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:15.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.471+0000 7fbf12a00700 1 --2- 172.21.15.33:0/2890883520 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55ca61510870 0x55ca61512c90 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80f4040ca49de0ee server_cookie=4cdc4bfaf29246f in_seq=0 out_seq=0 2024-05-03T18:38:15.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.471+0000 7fbf00c00700 10 client.26200 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:38:15.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.472+0000 7fbf00c00700 1 -- 172.21.15.33:0/2890883520 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ca613ec400 con 0x55ca61510870 2024-05-03T18:38:15.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.472+0000 7fbf00c00700 10 client.26200 handle_command_reply: tid=0 2024-05-03T18:38:15.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.472+0000 7fbf244d4b80 1 client.26200 shutdown 2024-05-03T18:38:15.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.473+0000 7fbf244d4b80 1 -- 172.21.15.33:0/2890883520 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ca614a1e10 msgr2=0x55ca61492cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:15.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.473+0000 7fbf244d4b80 1 --2- 172.21.15.33:0/2890883520 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ca614a1e10 0x55ca61492cf0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c014dd0 tx=0x7fbf0c012580 comp rx=0 tx=0).stop 2024-05-03T18:38:15.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.473+0000 7fbf244d4b80 1 -- 172.21.15.33:0/2890883520 shutdown_connections 2024-05-03T18:38:15.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.473+0000 7fbf244d4b80 1 --2- 172.21.15.33:0/2890883520 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55ca61510870 0x55ca61512c90 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:15.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.473+0000 7fbf244d4b80 1 --2- 172.21.15.33:0/2890883520 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ca614a1e10 0x55ca61492cf0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:15.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.473+0000 7fbf244d4b80 1 --2- 172.21.15.33:0/2890883520 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ca614907b0 0x55ca6149e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:15.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.473+0000 7fbf244d4b80 1 --2- 172.21.15.33:0/2890883520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ca614910d0 0x55ca614927b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:15.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.473+0000 7fbf00c00700 0 client.26200 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:38:15.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.473+0000 7fbf244d4b80 1 -- 172.21.15.33:0/2890883520 >> 172.21.15.33:0/2890883520 conn(0x55ca613bab60 msgr2=0x55ca614a1690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:38:15.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.473+0000 7fbf244d4b80 1 -- 172.21.15.33:0/2890883520 shutdown_connections 2024-05-03T18:38:15.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.473+0000 7fbf244d4b80 1 -- 172.21.15.33:0/2890883520 wait complete. 2024-05-03T18:38:15.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.474+0000 7fbf244d4b80 20 client.26200 trim_cache size 0 max 16384 2024-05-03T18:38:15.479 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-03T18:38:15.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.479+0000 7fbf19c00700 1 -- 172.21.15.33:0/3187238000 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fbef80b2e40 msgr2=0x7fbef80b5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:15.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.479+0000 7fbf19c00700 1 --2- 172.21.15.33:0/3187238000 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fbef80b2e40 0x7fbef80b5300 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fbefc005950 tx=0x7fbefc0058e0 comp rx=0 tx=0).stop 2024-05-03T18:38:15.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.479+0000 7fbf19c00700 1 -- 172.21.15.33:0/3187238000 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf14105ee0 msgr2=0x7fbf141a0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:15.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.479+0000 7fbf19c00700 1 --2- 172.21.15.33:0/3187238000 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf14105ee0 0x7fbf141a0640 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fbf0400c010 tx=0x7fbf0400b560 comp rx=0 tx=0).stop 2024-05-03T18:38:15.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.479+0000 7fbf19c00700 1 -- 172.21.15.33:0/3187238000 shutdown_connections 2024-05-03T18:38:15.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.479+0000 7fbf19c00700 1 --2- 172.21.15.33:0/3187238000 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fbef80b2e40 0x7fbef80b5300 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:15.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.479+0000 7fbf19c00700 1 --2- 172.21.15.33:0/3187238000 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbf141083d0 0x7fbf141a49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:15.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.479+0000 7fbf19c00700 1 --2- 172.21.15.33:0/3187238000 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf1406b6f0 0x7fbf141a0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:15.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.479+0000 7fbf19c00700 1 --2- 172.21.15.33:0/3187238000 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf14105ee0 0x7fbf141a0640 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:15.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.479+0000 7fbf19c00700 1 -- 172.21.15.33:0/3187238000 >> 172.21.15.33:0/3187238000 conn(0x7fbf14079350 msgr2=0x7fbf14102eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:38:15.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.481+0000 7fbf19c00700 1 -- 172.21.15.33:0/3187238000 shutdown_connections 2024-05-03T18:38:15.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:15.481+0000 7fbf19c00700 1 -- 172.21.15.33:0/3187238000 wait complete. 2024-05-03T18:38:16.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:38:16 smithi150 ceph-mon[32492]: pgmap v344: 129 pgs: 129 active+clean; 2.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 4.5 MiB/s wr, 503 op/s 2024-05-03T18:38:16.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:38:16 smithi033 ceph-mon[28330]: pgmap v344: 129 pgs: 129 active+clean; 2.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 4.5 MiB/s wr, 503 op/s 2024-05-03T18:38:16.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:38:16 smithi099 ceph-mon[31658]: pgmap v344: 129 pgs: 129 active+clean; 2.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 4.5 MiB/s wr, 503 op/s 2024-05-03T18:38:18.259 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-03T18:38:18.260 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-03T18:38:18.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:38:18 smithi033 ceph-mon[28330]: osdmap e100: 12 total, 12 up, 12 in 2024-05-03T18:38:18.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:38:18 smithi033 ceph-mon[28330]: pgmap v346: 129 pgs: 129 active+clean; 2.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 2.8 MiB/s wr, 312 op/s 2024-05-03T18:38:18.797 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:38:18 smithi099 ceph-mon[31658]: osdmap e100: 12 total, 12 up, 12 in 2024-05-03T18:38:18.797 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:38:18 smithi099 ceph-mon[31658]: pgmap v346: 129 pgs: 129 active+clean; 2.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 2.8 MiB/s wr, 312 op/s 2024-05-03T18:38:18.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:38:18 smithi150 ceph-mon[32492]: osdmap e100: 12 total, 12 up, 12 in 2024-05-03T18:38:18.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:38:18 smithi150 ceph-mon[32492]: pgmap v346: 129 pgs: 129 active+clean; 2.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 2.8 MiB/s wr, 312 op/s 2024-05-03T18:38:19.189 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:38:20.111 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 425.0 s, 606.3 tps, lat 164.889 ms stddev 53.944 2024-05-03T18:38:20.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:38:20 smithi033 ceph-mon[28330]: pgmap v347: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 5.9 MiB/s wr, 655 op/s 2024-05-03T18:38:20.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:38:20 smithi099 ceph-mon[31658]: pgmap v347: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 5.9 MiB/s wr, 655 op/s 2024-05-03T18:38:20.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:38:20 smithi150 ceph-mon[32492]: pgmap v347: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 5.9 MiB/s wr, 655 op/s 2024-05-03T18:38:23.395 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:38:23 smithi099 ceph-mon[31658]: pgmap v348: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 4.9 MiB/s wr, 545 op/s 2024-05-03T18:38:23.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.441+0000 7f44f9600700 1 -- 172.21.15.33:0/3464038309 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44f4106090 msgr2=0x7f44f41125a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:23.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.441+0000 7f44f9600700 1 --2- 172.21.15.33:0/3464038309 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44f4106090 0x7f44f41125a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f44ec00b0a0 tx=0x7f44ec00b3b0 comp rx=0 tx=0).stop 2024-05-03T18:38:23.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.442+0000 7f44f9600700 1 -- 172.21.15.33:0/3464038309 shutdown_connections 2024-05-03T18:38:23.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.442+0000 7f44f9600700 1 --2- 172.21.15.33:0/3464038309 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44f4106090 0x7f44f41125a0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:23.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.442+0000 7f44f9600700 1 --2- 172.21.15.33:0/3464038309 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44f410b6d0 0x7f44f410bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:23.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.442+0000 7f44f9600700 1 --2- 172.21.15.33:0/3464038309 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44f41056d0 0x7f44f4105b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:23.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.443+0000 7f44f9600700 1 -- 172.21.15.33:0/3464038309 >> 172.21.15.33:0/3464038309 conn(0x7f44f41011c0 msgr2=0x7f44f41035e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:38:23.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.443+0000 7f44f9600700 1 -- 172.21.15.33:0/3464038309 shutdown_connections 2024-05-03T18:38:23.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.443+0000 7f44f9600700 1 -- 172.21.15.33:0/3464038309 wait complete. 2024-05-03T18:38:23.447 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:38:23 smithi033 ceph-mon[28330]: pgmap v348: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 4.9 MiB/s wr, 545 op/s 2024-05-03T18:38:23.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.445+0000 7f44f9600700 1 Processor -- start 2024-05-03T18:38:23.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.446+0000 7f44f9600700 1 -- start start 2024-05-03T18:38:23.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.446+0000 7f44f9600700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44f41056d0 0x7f44f419bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:23.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.447+0000 7f44f9600700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44f4106090 0x7f44f419c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:23.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.447+0000 7f44f9600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44f410b6d0 0x7f44f41a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:23.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.447+0000 7f44f9600700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f44f4115870 con 0x7f44f4106090 2024-05-03T18:38:23.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.447+0000 7f44f9600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f44f41156f0 con 0x7f44f410b6d0 2024-05-03T18:38:23.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.447+0000 7f44f9600700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f44f41159f0 con 0x7f44f41056d0 2024-05-03T18:38:23.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.447+0000 7f44ebe00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44f4106090 0x7f44f419c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:23.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.447+0000 7f44f2a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44f41056d0 0x7f44f419bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:23.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.447+0000 7f44f2a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44f41056d0 0x7f44f419bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:41532/0 (socket says 172.21.15.33:41532) 2024-05-03T18:38:23.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.447+0000 7f44f2a00700 1 -- 172.21.15.33:0/1706368351 learned_addr learned my addr 172.21.15.33:0/1706368351 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:38:23.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.447+0000 7f44f3400700 1 --2- 172.21.15.33:0/1706368351 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44f410b6d0 0x7f44f41a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:23.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.448+0000 7f44f2a00700 1 -- 172.21.15.33:0/1706368351 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44f410b6d0 msgr2=0x7f44f41a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:23.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.448+0000 7f44f2a00700 1 --2- 172.21.15.33:0/1706368351 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44f410b6d0 0x7f44f41a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:23.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.448+0000 7f44f2a00700 1 -- 172.21.15.33:0/1706368351 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44f4106090 msgr2=0x7f44f419c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:23.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.448+0000 7f44f2a00700 1 --2- 172.21.15.33:0/1706368351 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44f4106090 0x7f44f419c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:23.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.448+0000 7f44f2a00700 1 -- 172.21.15.33:0/1706368351 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44f41a0df0 con 0x7f44f41056d0 2024-05-03T18:38:23.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.450+0000 7f44f2a00700 1 --2- 172.21.15.33:0/1706368351 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44f41056d0 0x7f44f419bde0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f44e400b530 tx=0x7f44e400b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:38:23.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.450+0000 7f44eb400700 1 -- 172.21.15.33:0/1706368351 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e4015020 con 0x7f44f41056d0 2024-05-03T18:38:23.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.450+0000 7f44eb400700 1 -- 172.21.15.33:0/1706368351 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f44e4011cc0 con 0x7f44f41056d0 2024-05-03T18:38:23.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.450+0000 7f44eb400700 1 -- 172.21.15.33:0/1706368351 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e4010ba0 con 0x7f44f41056d0 2024-05-03T18:38:23.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.451+0000 7f44f9600700 1 -- 172.21.15.33:0/1706368351 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44f41a1080 con 0x7f44f41056d0 2024-05-03T18:38:23.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.451+0000 7f44f9600700 1 -- 172.21.15.33:0/1706368351 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44f40778e0 con 0x7f44f41056d0 2024-05-03T18:38:23.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.463+0000 7f44eb400700 1 -- 172.21.15.33:0/1706368351 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f44e4018020 con 0x7f44f41056d0 2024-05-03T18:38:23.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.466+0000 7f44eb400700 1 --2- 172.21.15.33:0/1706368351 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f44d80b2a20 0x7f44d80b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:23.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.468+0000 7f44eb400700 1 -- 172.21.15.33:0/1706368351 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9518+0+0 (secure 0 0 0) 0x7f44e40e1fa0 con 0x7f44f41056d0 2024-05-03T18:38:23.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.469+0000 7f44ebe00700 1 --2- 172.21.15.33:0/1706368351 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f44d80b2a20 0x7f44d80b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:23.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.470+0000 7f44ebe00700 1 --2- 172.21.15.33:0/1706368351 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f44d80b2a20 0x7f44d80b4ee0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f44f41038d0 tx=0x7f44dc008040 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:38:23.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.485+0000 7f4503f22b80 1 Processor -- start 2024-05-03T18:38:23.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.485+0000 7f4503f22b80 1 -- start start 2024-05-03T18:38:23.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.485+0000 7f4503f22b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5565b9d64830 0x5565b9d64c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:23.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.486+0000 7f4503f22b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565b9d65150 0x5565b9d60030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:23.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.486+0000 7f4503f22b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565b9d60650 0x5565b9d60ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:23.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.486+0000 7f4503f22b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5565b9cb2aa0 con 0x5565b9d64830 2024-05-03T18:38:23.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.486+0000 7f4503f22b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5565b9c507e0 con 0x5565b9d60650 2024-05-03T18:38:23.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.486+0000 7f4503f22b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5565b9caea60 con 0x5565b9d65150 2024-05-03T18:38:23.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.486+0000 7f44f2a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5565b9d64830 0x5565b9d64c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:23.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.486+0000 7f44f2a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5565b9d64830 0x5565b9d64c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56164/0 (socket says 172.21.15.33:56164) 2024-05-03T18:38:23.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.486+0000 7f44f2a00700 1 -- 172.21.15.33:0/340138751 learned_addr learned my addr 172.21.15.33:0/340138751 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:38:23.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.486+0000 7f44f3400700 1 --2- 172.21.15.33:0/340138751 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565b9d60650 0x5565b9d60ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:23.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.486+0000 7f44ebe00700 1 --2- 172.21.15.33:0/340138751 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565b9d65150 0x5565b9d60030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:23.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.487+0000 7f44f2a00700 1 -- 172.21.15.33:0/340138751 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565b9d65150 msgr2=0x5565b9d60030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:38:23.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.487+0000 7f44f2a00700 1 --2- 172.21.15.33:0/340138751 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565b9d65150 0x5565b9d60030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:23.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.487+0000 7f44f2a00700 1 -- 172.21.15.33:0/340138751 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565b9d60650 msgr2=0x5565b9d60ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:38:23.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.487+0000 7f44f2a00700 1 --2- 172.21.15.33:0/340138751 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565b9d60650 0x5565b9d60ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:23.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.487+0000 7f44f2a00700 1 -- 172.21.15.33:0/340138751 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565b9bff950 con 0x5565b9d64830 2024-05-03T18:38:23.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.487+0000 7f44f2a00700 1 --2- 172.21.15.33:0/340138751 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5565b9d64830 0x5565b9d64c10 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f44e40a4fd0 tx=0x7f44e40a87c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25905b083a501224 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:38:23.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.487+0000 7f44e3e00700 1 -- 172.21.15.33:0/340138751 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e40ad300 con 0x5565b9d64830 2024-05-03T18:38:23.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.488+0000 7f44e3e00700 1 -- 172.21.15.33:0/340138751 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f44e40b1db0 con 0x5565b9d64830 2024-05-03T18:38:23.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.488+0000 7f44e3e00700 1 -- 172.21.15.33:0/340138751 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e40ad300 con 0x5565b9d64830 2024-05-03T18:38:23.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.488+0000 7f4503f22b80 1 -- 172.21.15.33:0/340138751 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5565b9d64830 msgr2=0x5565b9d64c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:23.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.488+0000 7f4503f22b80 1 --2- 172.21.15.33:0/340138751 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5565b9d64830 0x5565b9d64c10 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f44e40a4fd0 tx=0x7f44e40a87c0 comp rx=0 tx=0).stop 2024-05-03T18:38:23.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.488+0000 7f4503f22b80 1 -- 172.21.15.33:0/340138751 shutdown_connections 2024-05-03T18:38:23.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.488+0000 7f4503f22b80 1 --2- 172.21.15.33:0/340138751 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565b9d60650 0x5565b9d60ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:23.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.489+0000 7f4503f22b80 1 --2- 172.21.15.33:0/340138751 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5565b9d64830 0x5565b9d64c10 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:23.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.489+0000 7f4503f22b80 1 --2- 172.21.15.33:0/340138751 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565b9d65150 0x5565b9d60030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:23.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.489+0000 7f4503f22b80 1 -- 172.21.15.33:0/340138751 >> 172.21.15.33:0/340138751 conn(0x5565b9c8ebe0 msgr2=0x5565b9d5a870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:38:23.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.489+0000 7f4503f22b80 1 -- 172.21.15.33:0/340138751 shutdown_connections 2024-05-03T18:38:23.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.489+0000 7f4503f22b80 1 -- 172.21.15.33:0/340138751 wait complete. 2024-05-03T18:38:23.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.490+0000 7f4503f22b80 1 -- start start 2024-05-03T18:38:23.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.491+0000 7f4503f22b80 1 Processor -- start 2024-05-03T18:38:23.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.492+0000 7f4503f22b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565b9d60650 0x5565b9d6c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:23.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.492+0000 7f4503f22b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5565b9d64830 0x5565b9d693a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:23.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.492+0000 7f4503f22b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565b9d65150 0x5565b9d698e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:23.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.492+0000 7f4503f22b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5565b9d6cd20 con 0x5565b9d64830 2024-05-03T18:38:23.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.492+0000 7f4503f22b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5565b9c18ba0 con 0x5565b9d60650 2024-05-03T18:38:23.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.492+0000 7f4503f22b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5565b9d6cea0 con 0x5565b9d65150 2024-05-03T18:38:23.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.492+0000 7f4503f22b80 10 client.0 _ll_register_callbacks cb 0x5565b9c59cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:38:23.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.493+0000 7f44f2a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565b9d60650 0x5565b9d6c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:23.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.493+0000 7f44f2a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565b9d60650 0x5565b9d6c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:35686/0 (socket says 172.21.15.33:35686) 2024-05-03T18:38:23.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.493+0000 7f44f2a00700 1 -- 172.21.15.33:0/62621784 learned_addr learned my addr 172.21.15.33:0/62621784 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:38:23.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.493+0000 7f44ebe00700 1 --2- 172.21.15.33:0/62621784 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5565b9d64830 0x5565b9d693a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:23.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.494+0000 7f44f2a00700 1 -- 172.21.15.33:0/62621784 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565b9d65150 msgr2=0x5565b9d698e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:38:23.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.494+0000 7f44f3400700 1 --2- 172.21.15.33:0/62621784 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565b9d65150 0x5565b9d698e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:23.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.494+0000 7f44f2a00700 1 --2- 172.21.15.33:0/62621784 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565b9d65150 0x5565b9d698e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:23.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.494+0000 7f44f2a00700 1 -- 172.21.15.33:0/62621784 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5565b9d64830 msgr2=0x5565b9d693a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:38:23.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.494+0000 7f44f2a00700 1 --2- 172.21.15.33:0/62621784 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5565b9d64830 0x5565b9d693a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:23.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.494+0000 7f44f2a00700 1 -- 172.21.15.33:0/62621784 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565b9c784a0 con 0x5565b9d60650 2024-05-03T18:38:23.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.494+0000 7f44ebe00700 1 --2- 172.21.15.33:0/62621784 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5565b9d64830 0x5565b9d693a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:38:23.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.494+0000 7f44f2a00700 1 --2- 172.21.15.33:0/62621784 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565b9d60650 0x5565b9d6c000 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f44e400b500 tx=0x7f44e400bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26fad848a13049bd server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:38:23.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.494+0000 7f44e0c00700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-03T18:38:23.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.495+0000 7f44e0c00700 1 -- 172.21.15.33:0/62621784 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e40c7020 con 0x5565b9d60650 2024-05-03T18:38:23.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.495+0000 7f4503f22b80 1 -- 172.21.15.33:0/62621784 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44e40b1db0 con 0x5565b9d60650 2024-05-03T18:38:23.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.495+0000 7f44e0c00700 1 -- 172.21.15.33:0/62621784 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9518+0+0 (secure 0 0 0) 0x7f44e40a93b0 con 0x5565b9d60650 2024-05-03T18:38:23.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.495+0000 7f44e0c00700 1 -- 172.21.15.33:0/62621784 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f44e4011a40 con 0x5565b9d60650 2024-05-03T18:38:23.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.496+0000 7f4503f22b80 10 client.36128 fetch_fsmap learned FSMap version 21 2024-05-03T18:38:23.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.496+0000 7f4503f22b80 1 -- 172.21.15.33:0/62621784 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5565b9d6cd20 con 0x5565b9d60650 2024-05-03T18:38:23.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.496+0000 7f44e0c00700 1 -- 172.21.15.33:0/62621784 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f44e40b1db0 con 0x5565b9d60650 2024-05-03T18:38:23.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.497+0000 7f44e0c00700 1 -- 172.21.15.33:0/62621784 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f44e40a7c20 con 0x5565b9d60650 2024-05-03T18:38:23.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.497+0000 7f4503f22b80 10 client.36128 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:38:23.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.497+0000 7f4503f22b80 10 client.36128 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:38:23.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.497+0000 7f4503f22b80 20 client.36128 populate_metadata read hostname 'smithi033' 2024-05-03T18:38:23.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.497+0000 7f4503f22b80 1 --2- 172.21.15.33:0/62621784 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5565b9d798d0 0x5565b9d7bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:23.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.497+0000 7f4503f22b80 4 client.36128 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:38:23.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.497+0000 7f4503f22b80 1 -- 172.21.15.33:0/62621784 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5565b9d2d250 con 0x5565b9d798d0 2024-05-03T18:38:23.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.498+0000 7f44ebe00700 1 --2- 172.21.15.33:0/62621784 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5565b9d798d0 0x5565b9d7bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:23.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.499+0000 7f44ebe00700 1 --2- 172.21.15.33:0/62621784 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5565b9d798d0 0x5565b9d7bcc0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e63b9348099a291c server_cookie=bb7dfef62be6a444 in_seq=0 out_seq=0 2024-05-03T18:38:23.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.499+0000 7f44e0c00700 10 client.36128 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:38:23.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.508+0000 7f44e0c00700 1 -- 172.21.15.33:0/62621784 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5565b9d2d250 con 0x5565b9d798d0 2024-05-03T18:38:23.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.508+0000 7f44e0c00700 10 client.36128 handle_command_reply: tid=0 2024-05-03T18:38:23.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.508+0000 7f4503f22b80 1 client.36128 shutdown 2024-05-03T18:38:23.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.510+0000 7f4503f22b80 1 -- 172.21.15.33:0/62621784 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565b9d60650 msgr2=0x5565b9d6c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:23.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.510+0000 7f4503f22b80 1 --2- 172.21.15.33:0/62621784 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565b9d60650 0x5565b9d6c000 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f44e400b500 tx=0x7f44e400bf60 comp rx=0 tx=0).stop 2024-05-03T18:38:23.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.510+0000 7f4503f22b80 1 -- 172.21.15.33:0/62621784 shutdown_connections 2024-05-03T18:38:23.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.510+0000 7f4503f22b80 1 --2- 172.21.15.33:0/62621784 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5565b9d798d0 0x5565b9d7bcc0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:23.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.510+0000 7f4503f22b80 1 --2- 172.21.15.33:0/62621784 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565b9d65150 0x5565b9d698e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:23.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.510+0000 7f4503f22b80 1 --2- 172.21.15.33:0/62621784 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565b9d60650 0x5565b9d6c000 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:23.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.510+0000 7f4503f22b80 1 --2- 172.21.15.33:0/62621784 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5565b9d64830 0x5565b9d693a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:23.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.510+0000 7f4503f22b80 1 -- 172.21.15.33:0/62621784 >> 172.21.15.33:0/62621784 conn(0x5565b9c8ebe0 msgr2=0x5565b9d65e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:38:23.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.510+0000 7f44e0c00700 0 client.36128 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:38:23.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.510+0000 7f4503f22b80 1 -- 172.21.15.33:0/62621784 shutdown_connections 2024-05-03T18:38:23.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.511+0000 7f4503f22b80 1 -- 172.21.15.33:0/62621784 wait complete. 2024-05-03T18:38:23.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.511+0000 7f4503f22b80 20 client.36128 trim_cache size 0 max 16384 2024-05-03T18:38:23.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.557+0000 7f4503f22b80 1 Processor -- start 2024-05-03T18:38:23.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.557+0000 7f4503f22b80 1 -- start start 2024-05-03T18:38:23.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.558+0000 7f4503f22b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5565b9d60650 0x5565b9d75db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:23.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.558+0000 7f4503f22b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565b9d64830 0x5565b9d5fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:23.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.558+0000 7f4503f22b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565b9d65150 0x5565b9d81350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:23.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.558+0000 7f4503f22b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f44f41ab4d0 con 0x5565b9d60650 2024-05-03T18:38:23.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.558+0000 7f4503f22b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f44f41add10 con 0x5565b9d65150 2024-05-03T18:38:23.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.558+0000 7f4503f22b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f44f41abb20 con 0x5565b9d64830 2024-05-03T18:38:23.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.558+0000 7f44f3400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565b9d65150 0x5565b9d81350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:23.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.558+0000 7f44f3400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565b9d65150 0x5565b9d81350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:35696/0 (socket says 172.21.15.33:35696) 2024-05-03T18:38:23.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.558+0000 7f44f3400700 1 -- 172.21.15.33:0/3693015936 learned_addr learned my addr 172.21.15.33:0/3693015936 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:38:23.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.559+0000 7f44ebe00700 1 --2- 172.21.15.33:0/3693015936 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565b9d64830 0x5565b9d5fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:23.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.559+0000 7f44f3400700 1 -- 172.21.15.33:0/3693015936 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565b9d64830 msgr2=0x5565b9d5fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:38:23.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.559+0000 7f44f3400700 1 --2- 172.21.15.33:0/3693015936 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565b9d64830 0x5565b9d5fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:23.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.559+0000 7f44f3400700 1 -- 172.21.15.33:0/3693015936 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5565b9d60650 msgr2=0x5565b9d75db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:38:23.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.559+0000 7f44f3400700 1 --2- 172.21.15.33:0/3693015936 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5565b9d60650 0x5565b9d75db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:23.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.559+0000 7f44f3400700 1 -- 172.21.15.33:0/3693015936 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565b9d67910 con 0x5565b9d65150 2024-05-03T18:38:23.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.559+0000 7f44ebe00700 1 --2- 172.21.15.33:0/3693015936 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565b9d64830 0x5565b9d5fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:38:23.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.560+0000 7f44f3400700 1 --2- 172.21.15.33:0/3693015936 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565b9d65150 0x5565b9d81350 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f44ec009370 tx=0x7f44ec0093a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b50ac6b6d8cfdccb server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:38:23.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.561+0000 7f44e3400700 1 -- 172.21.15.33:0/3693015936 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44ec003c00 con 0x5565b9d65150 2024-05-03T18:38:23.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.561+0000 7f44e3400700 1 -- 172.21.15.33:0/3693015936 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f44ec020070 con 0x5565b9d65150 2024-05-03T18:38:23.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.561+0000 7f44e3400700 1 -- 172.21.15.33:0/3693015936 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44ec0257f0 con 0x5565b9d65150 2024-05-03T18:38:23.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.563+0000 7f4503f22b80 1 -- 172.21.15.33:0/3693015936 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565b9d65150 msgr2=0x5565b9d81350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:23.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.563+0000 7f4503f22b80 1 --2- 172.21.15.33:0/3693015936 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565b9d65150 0x5565b9d81350 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f44ec009370 tx=0x7f44ec0093a0 comp rx=0 tx=0).stop 2024-05-03T18:38:23.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.563+0000 7f4503f22b80 1 -- 172.21.15.33:0/3693015936 shutdown_connections 2024-05-03T18:38:23.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.563+0000 7f4503f22b80 1 --2- 172.21.15.33:0/3693015936 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565b9d65150 0x5565b9d81350 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:23.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.563+0000 7f4503f22b80 1 --2- 172.21.15.33:0/3693015936 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5565b9d60650 0x5565b9d75db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:23.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.563+0000 7f4503f22b80 1 --2- 172.21.15.33:0/3693015936 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565b9d64830 0x5565b9d5fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:23.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.564+0000 7f4503f22b80 1 -- 172.21.15.33:0/3693015936 >> 172.21.15.33:0/3693015936 conn(0x5565b9c8ebe0 msgr2=0x5565b9d5f460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:38:23.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.566+0000 7f4503f22b80 1 -- 172.21.15.33:0/3693015936 shutdown_connections 2024-05-03T18:38:23.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.566+0000 7f4503f22b80 1 -- 172.21.15.33:0/3693015936 wait complete. 2024-05-03T18:38:23.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.567+0000 7f4503f22b80 1 -- start start 2024-05-03T18:38:23.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.567+0000 7f4503f22b80 1 Processor -- start 2024-05-03T18:38:23.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.568+0000 7f4503f22b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565b9d60650 0x5565b9d72620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:23.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.568+0000 7f4503f22b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565b9d64830 0x5565b9d6f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:23.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.568+0000 7f4503f22b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5565b9d65150 0x5565b9d6f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:23.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.568+0000 7f4503f22b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f44f41ac1d0 con 0x5565b9d65150 2024-05-03T18:38:23.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.568+0000 7f4503f22b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f44f41ad640 con 0x5565b9d64830 2024-05-03T18:38:23.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.568+0000 7f4503f22b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f44f41ac8a0 con 0x5565b9d60650 2024-05-03T18:38:23.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.568+0000 7f4503f22b80 10 client.0 _ll_register_callbacks cb 0x5565b9c59cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:38:23.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.569+0000 7f44f2a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565b9d60650 0x5565b9d72620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:23.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.569+0000 7f44f2a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565b9d60650 0x5565b9d72620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:41568/0 (socket says 172.21.15.33:41568) 2024-05-03T18:38:23.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.569+0000 7f44f2a00700 1 -- 172.21.15.33:0/1463427570 learned_addr learned my addr 172.21.15.33:0/1463427570 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:38:23.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.569+0000 7f44ebe00700 1 --2- 172.21.15.33:0/1463427570 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565b9d64830 0x5565b9d6f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:23.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.570+0000 7f44f2a00700 1 -- 172.21.15.33:0/1463427570 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565b9d64830 msgr2=0x5565b9d6f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:38:23.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.570+0000 7f44f2a00700 1 --2- 172.21.15.33:0/1463427570 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565b9d64830 0x5565b9d6f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:23.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.570+0000 7f44f2a00700 1 -- 172.21.15.33:0/1463427570 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5565b9d65150 msgr2=0x5565b9d6f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:38:23.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.570+0000 7f44f2a00700 1 --2- 172.21.15.33:0/1463427570 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5565b9d65150 0x5565b9d6f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:23.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.570+0000 7f44f2a00700 1 -- 172.21.15.33:0/1463427570 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565b9d6fe10 con 0x5565b9d60650 2024-05-03T18:38:23.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.570+0000 7f44ebe00700 1 --2- 172.21.15.33:0/1463427570 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565b9d64830 0x5565b9d6f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:38:23.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.570+0000 7f44f2a00700 1 --2- 172.21.15.33:0/1463427570 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565b9d60650 0x5565b9d72620 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f44e40a8c50 tx=0x7f44e40b1be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b469861926225479 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:38:23.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.570+0000 7f44e0c00700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-03T18:38:23.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.573+0000 7f44e0c00700 1 -- 172.21.15.33:0/1463427570 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e40c7070 con 0x5565b9d60650 2024-05-03T18:38:23.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.573+0000 7f44e0c00700 1 -- 172.21.15.33:0/1463427570 <== mon.1 v2:172.21.15.150:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9518+0+0 (secure 0 0 0) 0x7f44e40a91b0 con 0x5565b9d60650 2024-05-03T18:38:23.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.573+0000 7f4503f22b80 1 -- 172.21.15.33:0/1463427570 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44ec020070 con 0x5565b9d60650 2024-05-03T18:38:23.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.573+0000 7f44e0c00700 1 -- 172.21.15.33:0/1463427570 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f44e40b1db0 con 0x5565b9d60650 2024-05-03T18:38:23.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.574+0000 7f4503f22b80 10 client.26215 fetch_fsmap learned FSMap version 21 2024-05-03T18:38:23.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.574+0000 7f4503f22b80 1 -- 172.21.15.33:0/1463427570 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5565b9d6fe10 con 0x5565b9d60650 2024-05-03T18:38:23.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.574+0000 7f44e0c00700 1 -- 172.21.15.33:0/1463427570 <== mon.1 v2:172.21.15.150:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f44ec020070 con 0x5565b9d60650 2024-05-03T18:38:23.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.574+0000 7f44e0c00700 1 -- 172.21.15.33:0/1463427570 <== mon.1 v2:172.21.15.150:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f44e40a48c0 con 0x5565b9d60650 2024-05-03T18:38:23.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.575+0000 7f4503f22b80 10 client.26215 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:38:23.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.575+0000 7f4503f22b80 10 client.26215 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:38:23.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.575+0000 7f4503f22b80 20 client.26215 populate_metadata read hostname 'smithi033' 2024-05-03T18:38:23.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.575+0000 7f4503f22b80 1 --2- 172.21.15.33:0/1463427570 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5565b9de5180 0x5565b9de7570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:23.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.575+0000 7f4503f22b80 4 client.26215 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-03T18:38:23.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.575+0000 7f4503f22b80 1 -- 172.21.15.33:0/1463427570 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5565b9cc0480 con 0x5565b9de5180 2024-05-03T18:38:23.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.576+0000 7f44ebe00700 1 --2- 172.21.15.33:0/1463427570 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5565b9de5180 0x5565b9de7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:23.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.577+0000 7f44ebe00700 1 --2- 172.21.15.33:0/1463427570 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5565b9de5180 0x5565b9de7570 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79d57d3c76cdee34 server_cookie=316cf32909f00eb in_seq=0 out_seq=0 2024-05-03T18:38:23.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.577+0000 7f44e0c00700 10 client.26215 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:38:23.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.580+0000 7f44e0c00700 1 -- 172.21.15.33:0/1463427570 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5565b9cc0480 con 0x5565b9de5180 2024-05-03T18:38:23.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.580+0000 7f44e0c00700 10 client.26215 handle_command_reply: tid=0 2024-05-03T18:38:23.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.580+0000 7f4503f22b80 1 client.26215 shutdown 2024-05-03T18:38:23.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.582+0000 7f4503f22b80 1 -- 172.21.15.33:0/1463427570 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565b9d60650 msgr2=0x5565b9d72620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:23.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.582+0000 7f4503f22b80 1 --2- 172.21.15.33:0/1463427570 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565b9d60650 0x5565b9d72620 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f44e40a8c50 tx=0x7f44e40b1be0 comp rx=0 tx=0).stop 2024-05-03T18:38:23.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.583+0000 7f4503f22b80 1 -- 172.21.15.33:0/1463427570 shutdown_connections 2024-05-03T18:38:23.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.583+0000 7f4503f22b80 1 --2- 172.21.15.33:0/1463427570 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5565b9de5180 0x5565b9de7570 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:23.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.583+0000 7f4503f22b80 1 --2- 172.21.15.33:0/1463427570 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5565b9d65150 0x5565b9d6f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:23.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.583+0000 7f4503f22b80 1 --2- 172.21.15.33:0/1463427570 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565b9d60650 0x5565b9d72620 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:23.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.583+0000 7f4503f22b80 1 --2- 172.21.15.33:0/1463427570 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565b9d64830 0x5565b9d6f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:23.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.583+0000 7f44e0c00700 0 client.26215 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:38:23.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.583+0000 7f4503f22b80 1 -- 172.21.15.33:0/1463427570 >> 172.21.15.33:0/1463427570 conn(0x5565b9c8ebe0 msgr2=0x5565b9d596d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:38:23.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.584+0000 7f4503f22b80 1 -- 172.21.15.33:0/1463427570 shutdown_connections 2024-05-03T18:38:23.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.584+0000 7f4503f22b80 1 -- 172.21.15.33:0/1463427570 wait complete. 2024-05-03T18:38:23.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.584+0000 7f4503f22b80 20 client.26215 trim_cache size 0 max 16384 2024-05-03T18:38:23.587 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-03T18:38:23.587 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-05-03T18:38:23.587 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "bf5b567c-8bfe-4561-8fcf-8fb1fbf5176f", 2024-05-03T18:38:23.587 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-05-03T18:38:23.587 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-03T18:38:23.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.586+0000 7f44f9600700 1 -- 172.21.15.33:0/1706368351 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f44d80b2a20 msgr2=0x7f44d80b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:23.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.586+0000 7f44f9600700 1 --2- 172.21.15.33:0/1706368351 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f44d80b2a20 0x7f44d80b4ee0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f44f41038d0 tx=0x7f44dc008040 comp rx=0 tx=0).stop 2024-05-03T18:38:23.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.586+0000 7f44f9600700 1 -- 172.21.15.33:0/1706368351 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44f41056d0 msgr2=0x7f44f419bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:23.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.586+0000 7f44f9600700 1 --2- 172.21.15.33:0/1706368351 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44f41056d0 0x7f44f419bde0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f44e400b530 tx=0x7f44e400b840 comp rx=0 tx=0).stop 2024-05-03T18:38:23.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.586+0000 7f44f9600700 1 -- 172.21.15.33:0/1706368351 shutdown_connections 2024-05-03T18:38:23.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.586+0000 7f44f9600700 1 --2- 172.21.15.33:0/1706368351 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f44d80b2a20 0x7f44d80b4ee0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:23.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.586+0000 7f44f9600700 1 --2- 172.21.15.33:0/1706368351 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44f410b6d0 0x7f44f41a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:23.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.586+0000 7f44f9600700 1 --2- 172.21.15.33:0/1706368351 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44f41056d0 0x7f44f419bde0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:23.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.586+0000 7f44f9600700 1 --2- 172.21.15.33:0/1706368351 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44f4106090 0x7f44f419c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:23.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.586+0000 7f44f9600700 1 -- 172.21.15.33:0/1706368351 >> 172.21.15.33:0/1706368351 conn(0x7f44f41011c0 msgr2=0x7f44f4101ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:38:23.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.586+0000 7f44f9600700 1 -- 172.21.15.33:0/1706368351 shutdown_connections 2024-05-03T18:38:23.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:23.586+0000 7f44f9600700 1 -- 172.21.15.33:0/1706368351 wait complete. 2024-05-03T18:38:23.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:38:23 smithi150 ceph-mon[32492]: pgmap v348: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 4.9 MiB/s wr, 545 op/s 2024-05-03T18:38:24.237 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:38:24 smithi033 ceph-mon[28330]: pgmap v349: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 5.5 MiB/s wr, 570 op/s 2024-05-03T18:38:24.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:38:24 smithi099 ceph-mon[31658]: pgmap v349: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 5.5 MiB/s wr, 570 op/s 2024-05-03T18:38:24.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:38:24 smithi150 ceph-mon[32492]: pgmap v349: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 5.5 MiB/s wr, 570 op/s 2024-05-03T18:38:25.111 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 430.0 s, 552.4 tps, lat 180.604 ms stddev 76.158 2024-05-03T18:38:25.438 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:38:25 smithi033 ceph-mon[28330]: scrub queued for path: / 2024-05-03T18:38:25.438 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:38:25 smithi033 ceph-mon[28330]: scrub summary: idle+waiting paths [/] 2024-05-03T18:38:25.438 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:38:25 smithi033 ceph-mon[28330]: scrub summary: active paths [/] 2024-05-03T18:38:25.438 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:38:25 smithi033 ceph-mon[28330]: scrub summary: idle+waiting paths [/] 2024-05-03T18:38:25.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:38:25 smithi099 ceph-mon[31658]: scrub queued for path: / 2024-05-03T18:38:25.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:38:25 smithi099 ceph-mon[31658]: scrub summary: idle+waiting paths [/] 2024-05-03T18:38:25.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:38:25 smithi099 ceph-mon[31658]: scrub summary: active paths [/] 2024-05-03T18:38:25.547 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:38:25 smithi099 ceph-mon[31658]: scrub summary: idle+waiting paths [/] 2024-05-03T18:38:25.552 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-03T18:38:25.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:38:25 smithi150 ceph-mon[32492]: scrub queued for path: / 2024-05-03T18:38:25.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:38:25 smithi150 ceph-mon[32492]: scrub summary: idle+waiting paths [/] 2024-05-03T18:38:25.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:38:25 smithi150 ceph-mon[32492]: scrub summary: active paths [/] 2024-05-03T18:38:25.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:38:25 smithi150 ceph-mon[32492]: scrub summary: idle+waiting paths [/] 2024-05-03T18:38:26.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:38:26 smithi150 ceph-mon[32492]: pgmap v350: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 5.5 MiB/s wr, 570 op/s 2024-05-03T18:38:26.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:38:26 smithi033 ceph-mon[28330]: pgmap v350: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 5.5 MiB/s wr, 570 op/s 2024-05-03T18:38:26.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:38:26 smithi099 ceph-mon[31658]: pgmap v350: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 5.5 MiB/s wr, 570 op/s 2024-05-03T18:38:26.582 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:38:27.486 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:38:27 smithi033 ceph-mon[28330]: scrub summary: idle 2024-05-03T18:38:27.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:38:27 smithi099 ceph-mon[31658]: scrub summary: idle 2024-05-03T18:38:27.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:38:27 smithi150 ceph-mon[32492]: scrub summary: idle 2024-05-03T18:38:28.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:38:28 smithi033 ceph-mon[28330]: pgmap v351: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 5.2 MiB/s wr, 534 op/s 2024-05-03T18:38:28.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:38:28 smithi099 ceph-mon[31658]: pgmap v351: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 5.2 MiB/s wr, 534 op/s 2024-05-03T18:38:28.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:38:28 smithi150 ceph-mon[32492]: pgmap v351: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 5.2 MiB/s wr, 534 op/s 2024-05-03T18:38:30.111 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 435.0 s, 504.0 tps, lat 198.540 ms stddev 120.259 2024-05-03T18:38:30.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:38:30 smithi033 ceph-mon[28330]: pgmap v352: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 8.7 MiB/s wr, 733 op/s 2024-05-03T18:38:30.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:38:30 smithi099 ceph-mon[31658]: pgmap v352: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 8.7 MiB/s wr, 733 op/s 2024-05-03T18:38:30.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:38:30 smithi150 ceph-mon[32492]: pgmap v352: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 8.7 MiB/s wr, 733 op/s 2024-05-03T18:38:32.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:38:32 smithi033 ceph-mon[28330]: pgmap v353: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 6.5 MiB/s wr, 487 op/s 2024-05-03T18:38:32.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:38:32 smithi099 ceph-mon[31658]: pgmap v353: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 6.5 MiB/s wr, 487 op/s 2024-05-03T18:38:32.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:38:32 smithi150 ceph-mon[32492]: pgmap v353: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 6.5 MiB/s wr, 487 op/s 2024-05-03T18:38:33.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.382+0000 7f4a88e00700 1 -- 172.21.15.33:0/3160627230 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a84076160 msgr2=0x7f4a84113ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:33.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.382+0000 7f4a88e00700 1 --2- 172.21.15.33:0/3160627230 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a84076160 0x7f4a84113ec0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f4a70009990 tx=0x7f4a70009ca0 comp rx=0 tx=0).stop 2024-05-03T18:38:33.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.383+0000 7f4a88e00700 1 -- 172.21.15.33:0/3160627230 shutdown_connections 2024-05-03T18:38:33.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.383+0000 7f4a88e00700 1 --2- 172.21.15.33:0/3160627230 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a84114400 0x7f4a841167f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:33.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.383+0000 7f4a88e00700 1 --2- 172.21.15.33:0/3160627230 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a84075840 0x7f4a84075c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:33.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.383+0000 7f4a88e00700 1 --2- 172.21.15.33:0/3160627230 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a84076160 0x7f4a84113ec0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:33.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.383+0000 7f4a88e00700 1 -- 172.21.15.33:0/3160627230 >> 172.21.15.33:0/3160627230 conn(0x7f4a840feee0 msgr2=0x7f4a84101300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:38:33.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.383+0000 7f4a88e00700 1 -- 172.21.15.33:0/3160627230 shutdown_connections 2024-05-03T18:38:33.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.384+0000 7f4a88e00700 1 -- 172.21.15.33:0/3160627230 wait complete. 2024-05-03T18:38:33.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.387+0000 7f4a88e00700 1 Processor -- start 2024-05-03T18:38:33.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.387+0000 7f4a88e00700 1 -- start start 2024-05-03T18:38:33.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.388+0000 7f4a88e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a84075840 0x7f4a84074ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:33.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.388+0000 7f4a88e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a84076160 0x7f4a84075020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:33.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.388+0000 7f4a88e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a84114400 0x7f4a8406dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:33.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.388+0000 7f4a88e00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4a84119ae0 con 0x7f4a84114400 2024-05-03T18:38:33.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.388+0000 7f4a88e00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4a84119960 con 0x7f4a84075840 2024-05-03T18:38:33.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.388+0000 7f4a88e00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f4a84119c60 con 0x7f4a84076160 2024-05-03T18:38:33.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.389+0000 7f4a81400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a84076160 0x7f4a84075020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:33.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.389+0000 7f4a81e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a84075840 0x7f4a84074ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:33.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.389+0000 7f4a81e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a84075840 0x7f4a84074ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:53126/0 (socket says 172.21.15.33:53126) 2024-05-03T18:38:33.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.389+0000 7f4a81e00700 1 -- 172.21.15.33:0/275877686 learned_addr learned my addr 172.21.15.33:0/275877686 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:38:33.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.390+0000 7f4a81e00700 1 -- 172.21.15.33:0/275877686 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a84076160 msgr2=0x7f4a84075020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:33.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.390+0000 7f4a81e00700 1 --2- 172.21.15.33:0/275877686 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a84076160 0x7f4a84075020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:33.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.390+0000 7f4a81e00700 1 -- 172.21.15.33:0/275877686 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a84114400 msgr2=0x7f4a8406dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:33.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.390+0000 7f4a81e00700 1 --2- 172.21.15.33:0/275877686 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a84114400 0x7f4a8406dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:33.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.390+0000 7f4a81e00700 1 -- 172.21.15.33:0/275877686 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a8406e600 con 0x7f4a84075840 2024-05-03T18:38:33.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.390+0000 7f4a81e00700 1 --2- 172.21.15.33:0/275877686 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a84075840 0x7f4a84074ae0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f4a7400d980 tx=0x7f4a7400dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:38:33.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.391+0000 7f4a7aa00700 1 -- 172.21.15.33:0/275877686 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a74017070 con 0x7f4a84075840 2024-05-03T18:38:33.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.391+0000 7f4a7aa00700 1 -- 172.21.15.33:0/275877686 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4a74005020 con 0x7f4a84075840 2024-05-03T18:38:33.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.391+0000 7f4a7aa00700 1 -- 172.21.15.33:0/275877686 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a74009ab0 con 0x7f4a84075840 2024-05-03T18:38:33.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.391+0000 7f4a88e00700 1 -- 172.21.15.33:0/275877686 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a8406e890 con 0x7f4a84075840 2024-05-03T18:38:33.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.391+0000 7f4a88e00700 1 -- 172.21.15.33:0/275877686 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a840727e0 con 0x7f4a84075840 2024-05-03T18:38:33.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.403+0000 7f4a7aa00700 1 -- 172.21.15.33:0/275877686 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f4a740051d0 con 0x7f4a84075840 2024-05-03T18:38:33.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.406+0000 7f4a7aa00700 1 --2- 172.21.15.33:0/275877686 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f4a680b2af0 0x7f4a680b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:33.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.406+0000 7f4a7aa00700 1 -- 172.21.15.33:0/275877686 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9518+0+0 (secure 0 0 0) 0x7f4a740e2900 con 0x7f4a84075840 2024-05-03T18:38:33.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.406+0000 7f4a81400700 1 --2- 172.21.15.33:0/275877686 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f4a680b2af0 0x7f4a680b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:33.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.408+0000 7f4a81400700 1 --2- 172.21.15.33:0/275877686 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f4a680b2af0 0x7f4a680b4fb0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f4a70009350 tx=0x7f4a7000b560 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:38:33.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.440+0000 7f4a93841b80 1 Processor -- start 2024-05-03T18:38:33.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.441+0000 7f4a93841b80 1 -- start start 2024-05-03T18:38:33.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.441+0000 7f4a93841b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55567495fa40 0x55567495fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:33.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.441+0000 7f4a93841b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555674960360 0x555674959fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:33.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.441+0000 7f4a93841b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55567495a5d0 0x55567495aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:33.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.441+0000 7f4a93841b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5556748aca20 con 0x555674960360 2024-05-03T18:38:33.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.441+0000 7f4a93841b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55567484a760 con 0x55567495fa40 2024-05-03T18:38:33.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.441+0000 7f4a93841b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5556748a89e0 con 0x55567495a5d0 2024-05-03T18:38:33.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.442+0000 7f4a81e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55567495fa40 0x55567495fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:33.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.442+0000 7f4a81e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55567495fa40 0x55567495fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:53140/0 (socket says 172.21.15.33:53140) 2024-05-03T18:38:33.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.442+0000 7f4a81e00700 1 -- 172.21.15.33:0/2232776293 learned_addr learned my addr 172.21.15.33:0/2232776293 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:38:33.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.442+0000 7f4a82800700 1 --2- 172.21.15.33:0/2232776293 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55567495a5d0 0x55567495aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:33.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.443+0000 7f4a81e00700 1 -- 172.21.15.33:0/2232776293 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55567495a5d0 msgr2=0x55567495aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:38:33.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.443+0000 7f4a81e00700 1 --2- 172.21.15.33:0/2232776293 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55567495a5d0 0x55567495aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:33.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.443+0000 7f4a81e00700 1 -- 172.21.15.33:0/2232776293 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555674960360 msgr2=0x555674959fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:38:33.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.443+0000 7f4a81e00700 1 --2- 172.21.15.33:0/2232776293 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555674960360 0x555674959fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:33.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.443+0000 7f4a81e00700 1 -- 172.21.15.33:0/2232776293 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556747f98d0 con 0x55567495fa40 2024-05-03T18:38:33.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.443+0000 7f4a81e00700 1 --2- 172.21.15.33:0/2232776293 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55567495fa40 0x55567495fe20 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4a740a5820 tx=0x7f4a740a9ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=260875d852c042bf server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:38:33.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.446+0000 7f4a6f400700 1 -- 172.21.15.33:0/2232776293 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a740e00a0 con 0x55567495fa40 2024-05-03T18:38:33.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.446+0000 7f4a6f400700 1 -- 172.21.15.33:0/2232776293 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4a740adb50 con 0x55567495fa40 2024-05-03T18:38:33.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.446+0000 7f4a6f400700 1 -- 172.21.15.33:0/2232776293 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a740a9c60 con 0x55567495fa40 2024-05-03T18:38:33.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.448+0000 7f4a93841b80 1 -- 172.21.15.33:0/2232776293 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55567495fa40 msgr2=0x55567495fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:33.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.448+0000 7f4a93841b80 1 --2- 172.21.15.33:0/2232776293 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55567495fa40 0x55567495fe20 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4a740a5820 tx=0x7f4a740a9ea0 comp rx=0 tx=0).stop 2024-05-03T18:38:33.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.448+0000 7f4a93841b80 1 -- 172.21.15.33:0/2232776293 shutdown_connections 2024-05-03T18:38:33.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.448+0000 7f4a93841b80 1 --2- 172.21.15.33:0/2232776293 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55567495a5d0 0x55567495aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:33.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.448+0000 7f4a93841b80 1 --2- 172.21.15.33:0/2232776293 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55567495fa40 0x55567495fe20 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:33.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.448+0000 7f4a93841b80 1 --2- 172.21.15.33:0/2232776293 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555674960360 0x555674959fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:33.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.448+0000 7f4a93841b80 1 -- 172.21.15.33:0/2232776293 >> 172.21.15.33:0/2232776293 conn(0x555674888b60 msgr2=0x5556749549d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:38:33.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.449+0000 7f4a93841b80 1 -- 172.21.15.33:0/2232776293 shutdown_connections 2024-05-03T18:38:33.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.449+0000 7f4a93841b80 1 -- 172.21.15.33:0/2232776293 wait complete. 2024-05-03T18:38:33.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.449+0000 7f4a93841b80 1 -- start start 2024-05-03T18:38:33.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.450+0000 7f4a93841b80 1 Processor -- start 2024-05-03T18:38:33.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.451+0000 7f4a93841b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55567495a5d0 0x555674966070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:33.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.452+0000 7f4a93841b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55567495fa40 0x555674963410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:33.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.452+0000 7f4a93841b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555674960360 0x555674963950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:33.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.452+0000 7f4a81e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55567495a5d0 0x555674966070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:33.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.452+0000 7f4a81e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55567495a5d0 0x555674966070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:53144/0 (socket says 172.21.15.33:53144) 2024-05-03T18:38:33.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.452+0000 7f4a93841b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x555674966d10 con 0x555674960360 2024-05-03T18:38:33.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.452+0000 7f4a93841b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x555674812b20 con 0x55567495a5d0 2024-05-03T18:38:33.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.452+0000 7f4a93841b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x555674966e90 con 0x55567495fa40 2024-05-03T18:38:33.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.452+0000 7f4a93841b80 10 client.0 _ll_register_callbacks cb 0x555674853c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:38:33.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.453+0000 7f4a81e00700 1 -- 172.21.15.33:0/1563507643 learned_addr learned my addr 172.21.15.33:0/1563507643 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:38:33.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.453+0000 7f4a81400700 1 --2- 172.21.15.33:0/1563507643 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55567495fa40 0x555674963410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:33.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.453+0000 7f4a81e00700 1 -- 172.21.15.33:0/1563507643 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55567495fa40 msgr2=0x555674963410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:38:33.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.453+0000 7f4a81e00700 1 --2- 172.21.15.33:0/1563507643 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55567495fa40 0x555674963410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:33.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.453+0000 7f4a81e00700 1 -- 172.21.15.33:0/1563507643 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555674960360 msgr2=0x555674963950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:38:33.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.453+0000 7f4a81e00700 1 --2- 172.21.15.33:0/1563507643 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555674960360 0x555674963950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:33.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.453+0000 7f4a81e00700 1 -- 172.21.15.33:0/1563507643 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555674872420 con 0x55567495a5d0 2024-05-03T18:38:33.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.454+0000 7f4a81e00700 1 --2- 172.21.15.33:0/1563507643 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55567495a5d0 0x555674966070 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4a74012040 tx=0x7f4a740913d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b260beb1918eb7f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:38:33.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.454+0000 7f4a63e00700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-03T18:38:33.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.454+0000 7f4a63e00700 1 -- 172.21.15.33:0/1563507643 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a740c2070 con 0x55567495a5d0 2024-05-03T18:38:33.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.454+0000 7f4a63e00700 1 -- 172.21.15.33:0/1563507643 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9518+0+0 (secure 0 0 0) 0x7f4a740a9c10 con 0x55567495a5d0 2024-05-03T18:38:33.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.454+0000 7f4a93841b80 1 -- 172.21.15.33:0/1563507643 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a740adb50 con 0x55567495a5d0 2024-05-03T18:38:33.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.455+0000 7f4a63e00700 1 -- 172.21.15.33:0/1563507643 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f4a74083e60 con 0x55567495a5d0 2024-05-03T18:38:33.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.455+0000 7f4a93841b80 10 client.36146 fetch_fsmap learned FSMap version 21 2024-05-03T18:38:33.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.455+0000 7f4a93841b80 1 -- 172.21.15.33:0/1563507643 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x555674971550 con 0x55567495a5d0 2024-05-03T18:38:33.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.455+0000 7f4a63e00700 1 -- 172.21.15.33:0/1563507643 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4a740adb50 con 0x55567495a5d0 2024-05-03T18:38:33.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.455+0000 7f4a63e00700 1 -- 172.21.15.33:0/1563507643 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f4a740a8a00 con 0x55567495a5d0 2024-05-03T18:38:33.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.455+0000 7f4a93841b80 10 client.36146 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:38:33.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.455+0000 7f4a93841b80 10 client.36146 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:38:33.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.455+0000 7f4a93841b80 20 client.36146 populate_metadata read hostname 'smithi033' 2024-05-03T18:38:33.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.455+0000 7f4a93841b80 1 --2- 172.21.15.33:0/1563507643 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x555674973a20 0x555674975e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:33.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.456+0000 7f4a93841b80 4 client.36146 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:38:33.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.456+0000 7f4a93841b80 1 -- 172.21.15.33:0/1563507643 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5556749271d0 con 0x555674973a20 2024-05-03T18:38:33.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.456+0000 7f4a81400700 1 --2- 172.21.15.33:0/1563507643 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x555674973a20 0x555674975e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:33.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.457+0000 7f4a81400700 1 --2- 172.21.15.33:0/1563507643 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x555674973a20 0x555674975e10 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1182e5430b27e1fb server_cookie=961afe1d33d8f28b in_seq=0 out_seq=0 2024-05-03T18:38:33.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.457+0000 7f4a63e00700 10 client.36146 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:38:33.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.458+0000 7f4a63e00700 1 -- 172.21.15.33:0/1563507643 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5556749271d0 con 0x555674973a20 2024-05-03T18:38:33.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.458+0000 7f4a63e00700 10 client.36146 handle_command_reply: tid=0 2024-05-03T18:38:33.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.458+0000 7f4a93841b80 1 client.36146 shutdown 2024-05-03T18:38:33.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.459+0000 7f4a93841b80 1 -- 172.21.15.33:0/1563507643 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55567495a5d0 msgr2=0x555674966070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:33.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.459+0000 7f4a93841b80 1 --2- 172.21.15.33:0/1563507643 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55567495a5d0 0x555674966070 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4a74012040 tx=0x7f4a740913d0 comp rx=0 tx=0).stop 2024-05-03T18:38:33.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.459+0000 7f4a93841b80 1 -- 172.21.15.33:0/1563507643 shutdown_connections 2024-05-03T18:38:33.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.459+0000 7f4a93841b80 1 --2- 172.21.15.33:0/1563507643 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x555674973a20 0x555674975e10 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:33.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.459+0000 7f4a93841b80 1 --2- 172.21.15.33:0/1563507643 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555674960360 0x555674963950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:33.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.459+0000 7f4a93841b80 1 --2- 172.21.15.33:0/1563507643 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55567495a5d0 0x555674966070 secure :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4a74012040 tx=0x7f4a740913d0 comp rx=0 tx=0).stop 2024-05-03T18:38:33.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.459+0000 7f4a93841b80 1 --2- 172.21.15.33:0/1563507643 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55567495fa40 0x555674963410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:33.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.459+0000 7f4a93841b80 1 -- 172.21.15.33:0/1563507643 >> 172.21.15.33:0/1563507643 conn(0x555674888b60 msgr2=0x555674954080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:38:33.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.459+0000 7f4a63e00700 0 client.36146 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:38:33.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.461+0000 7f4a93841b80 1 -- 172.21.15.33:0/1563507643 shutdown_connections 2024-05-03T18:38:33.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.461+0000 7f4a93841b80 1 -- 172.21.15.33:0/1563507643 wait complete. 2024-05-03T18:38:33.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.461+0000 7f4a93841b80 20 client.36146 trim_cache size 0 max 16384 2024-05-03T18:38:33.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.500+0000 7f4a93841b80 1 Processor -- start 2024-05-03T18:38:33.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.500+0000 7f4a93841b80 1 -- start start 2024-05-03T18:38:33.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.500+0000 7f4a93841b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55567495fa40 0x555674959fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:33.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.500+0000 7f4a93841b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555674960360 0x555674963740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:33.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.500+0000 7f4a93841b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55567495daf0 0x555674961090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:33.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.500+0000 7f4a93841b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4a841bc160 con 0x55567495fa40 2024-05-03T18:38:33.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.500+0000 7f4a93841b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4a841be9c0 con 0x555674960360 2024-05-03T18:38:33.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.500+0000 7f4a93841b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f4a841bc7b0 con 0x55567495daf0 2024-05-03T18:38:33.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.501+0000 7f4a81400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555674960360 0x555674963740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:33.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.501+0000 7f4a81400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555674960360 0x555674963740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:53148/0 (socket says 172.21.15.33:53148) 2024-05-03T18:38:33.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.501+0000 7f4a81400700 1 -- 172.21.15.33:0/2263823013 learned_addr learned my addr 172.21.15.33:0/2263823013 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:38:33.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.501+0000 7f4a82800700 1 --2- 172.21.15.33:0/2263823013 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55567495daf0 0x555674961090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:33.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.501+0000 7f4a81400700 1 -- 172.21.15.33:0/2263823013 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55567495daf0 msgr2=0x555674961090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:38:33.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.501+0000 7f4a81400700 1 --2- 172.21.15.33:0/2263823013 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55567495daf0 0x555674961090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:33.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.501+0000 7f4a81400700 1 -- 172.21.15.33:0/2263823013 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55567495fa40 msgr2=0x555674959fb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-03T18:38:33.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.501+0000 7f4a81400700 1 --2- 172.21.15.33:0/2263823013 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55567495fa40 0x555674959fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:33.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.501+0000 7f4a81400700 1 -- 172.21.15.33:0/2263823013 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55567494f770 con 0x555674960360 2024-05-03T18:38:33.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.501+0000 7f4a82800700 1 --2- 172.21.15.33:0/2263823013 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55567495daf0 0x555674961090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:38:33.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.502+0000 7f4a81400700 1 --2- 172.21.15.33:0/2263823013 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555674960360 0x555674963740 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4a70005f50 tx=0x7f4a70029040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ae929f0b7dde6ed server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:38:33.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.502+0000 7f4a6ea00700 1 -- 172.21.15.33:0/2263823013 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a70037070 con 0x555674960360 2024-05-03T18:38:33.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.502+0000 7f4a6ea00700 1 -- 172.21.15.33:0/2263823013 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x5556749271d0 con 0x555674960360 2024-05-03T18:38:33.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.502+0000 7f4a6ea00700 1 -- 172.21.15.33:0/2263823013 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a7000ba30 con 0x555674960360 2024-05-03T18:38:33.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.503+0000 7f4a93841b80 1 -- 172.21.15.33:0/2263823013 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555674960360 msgr2=0x555674963740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:33.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.503+0000 7f4a93841b80 1 --2- 172.21.15.33:0/2263823013 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555674960360 0x555674963740 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4a70005f50 tx=0x7f4a70029040 comp rx=0 tx=0).stop 2024-05-03T18:38:33.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.504+0000 7f4a93841b80 1 -- 172.21.15.33:0/2263823013 shutdown_connections 2024-05-03T18:38:33.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.504+0000 7f4a93841b80 1 --2- 172.21.15.33:0/2263823013 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55567495daf0 0x555674961090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:33.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.504+0000 7f4a93841b80 1 --2- 172.21.15.33:0/2263823013 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55567495fa40 0x555674959fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:33.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.504+0000 7f4a93841b80 1 --2- 172.21.15.33:0/2263823013 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555674960360 0x555674963740 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:33.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.504+0000 7f4a93841b80 1 -- 172.21.15.33:0/2263823013 >> 172.21.15.33:0/2263823013 conn(0x555674888b60 msgr2=0x555674961d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:38:33.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.504+0000 7f4a93841b80 1 -- 172.21.15.33:0/2263823013 shutdown_connections 2024-05-03T18:38:33.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.505+0000 7f4a93841b80 1 -- 172.21.15.33:0/2263823013 wait complete. 2024-05-03T18:38:33.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.505+0000 7f4a93841b80 1 -- start start 2024-05-03T18:38:33.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.505+0000 7f4a93841b80 1 Processor -- start 2024-05-03T18:38:33.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.506+0000 7f4a93841b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55567495daf0 0x555674978c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:33.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.506+0000 7f4a93841b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55567495fa40 0x555674979160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:33.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.506+0000 7f4a93841b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555674960360 0x5556749771b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:33.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.506+0000 7f4a93841b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4a841bce80 con 0x55567495daf0 2024-05-03T18:38:33.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.506+0000 7f4a93841b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4a841be2f0 con 0x555674960360 2024-05-03T18:38:33.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.506+0000 7f4a93841b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f4a841bd550 con 0x55567495fa40 2024-05-03T18:38:33.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.506+0000 7f4a93841b80 10 client.0 _ll_register_callbacks cb 0x555674853c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:38:33.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.507+0000 7f4a81e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55567495daf0 0x555674978c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:33.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.508+0000 7f4a82800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555674960360 0x5556749771b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:33.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.508+0000 7f4a81400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55567495fa40 0x555674979160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:33.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.508+0000 7f4a81400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55567495fa40 0x555674979160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:40264/0 (socket says 172.21.15.33:40264) 2024-05-03T18:38:33.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.508+0000 7f4a81400700 1 -- 172.21.15.33:0/4277066033 learned_addr learned my addr 172.21.15.33:0/4277066033 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:38:33.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.509+0000 7f4a82800700 1 -- 172.21.15.33:0/4277066033 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55567495fa40 msgr2=0x555674979160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:38:33.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.509+0000 7f4a82800700 1 --2- 172.21.15.33:0/4277066033 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55567495fa40 0x555674979160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:33.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.509+0000 7f4a82800700 1 -- 172.21.15.33:0/4277066033 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55567495daf0 msgr2=0x555674978c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:38:33.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.509+0000 7f4a82800700 1 --2- 172.21.15.33:0/4277066033 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55567495daf0 0x555674978c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:33.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.509+0000 7f4a82800700 1 -- 172.21.15.33:0/4277066033 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556749779f0 con 0x555674960360 2024-05-03T18:38:33.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.510+0000 7f4a82800700 1 --2- 172.21.15.33:0/4277066033 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555674960360 0x5556749771b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c0060b0 tx=0x7f4a7c007920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=994da9b60e4d9967 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:38:33.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.510+0000 7f4a63e00700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-03T18:38:33.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.510+0000 7f4a63e00700 1 -- 172.21.15.33:0/4277066033 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a7c004bb0 con 0x555674960360 2024-05-03T18:38:33.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.510+0000 7f4a63e00700 1 -- 172.21.15.33:0/4277066033 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9518+0+0 (secure 0 0 0) 0x7f4a7c012040 con 0x555674960360 2024-05-03T18:38:33.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.510+0000 7f4a93841b80 1 -- 172.21.15.33:0/4277066033 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5556749271d0 con 0x555674960360 2024-05-03T18:38:33.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.511+0000 7f4a63e00700 1 -- 172.21.15.33:0/4277066033 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f4a7c022050 con 0x555674960360 2024-05-03T18:38:33.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.511+0000 7f4a93841b80 10 client.36152 fetch_fsmap learned FSMap version 21 2024-05-03T18:38:33.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.511+0000 7f4a93841b80 1 -- 172.21.15.33:0/4277066033 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5556749debf0 con 0x555674960360 2024-05-03T18:38:33.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.512+0000 7f4a63e00700 1 -- 172.21.15.33:0/4277066033 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x5556749271d0 con 0x555674960360 2024-05-03T18:38:33.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.512+0000 7f4a63e00700 1 -- 172.21.15.33:0/4277066033 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f4a7c015650 con 0x555674960360 2024-05-03T18:38:33.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.513+0000 7f4a93841b80 10 client.36152 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:38:33.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.513+0000 7f4a93841b80 10 client.36152 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:38:33.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.513+0000 7f4a93841b80 20 client.36152 populate_metadata read hostname 'smithi033' 2024-05-03T18:38:33.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.513+0000 7f4a93841b80 1 --2- 172.21.15.33:0/4277066033 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5556749df3b0 0x5556749e17d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:33.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.513+0000 7f4a93841b80 4 client.36152 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-03T18:38:33.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.513+0000 7f4a93841b80 1 -- 172.21.15.33:0/4277066033 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5556748ba400 con 0x5556749df3b0 2024-05-03T18:38:33.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.514+0000 7f4a81e00700 1 --2- 172.21.15.33:0/4277066033 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5556749df3b0 0x5556749e17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:33.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.515+0000 7f4a63e00700 10 client.36152 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:38:33.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.515+0000 7f4a81e00700 1 --2- 172.21.15.33:0/4277066033 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5556749df3b0 0x5556749e17d0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed2548b2a0fe207c server_cookie=1aa239dbb680aa91 in_seq=0 out_seq=0 2024-05-03T18:38:33.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.516+0000 7f4a63e00700 1 -- 172.21.15.33:0/4277066033 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5556748ba400 con 0x5556749df3b0 2024-05-03T18:38:33.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.516+0000 7f4a63e00700 10 client.36152 handle_command_reply: tid=0 2024-05-03T18:38:33.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.516+0000 7f4a93841b80 1 client.36152 shutdown 2024-05-03T18:38:33.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.517+0000 7f4a93841b80 1 -- 172.21.15.33:0/4277066033 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555674960360 msgr2=0x5556749771b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:33.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.517+0000 7f4a93841b80 1 --2- 172.21.15.33:0/4277066033 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555674960360 0x5556749771b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c0060b0 tx=0x7f4a7c007920 comp rx=0 tx=0).stop 2024-05-03T18:38:33.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.517+0000 7f4a93841b80 1 -- 172.21.15.33:0/4277066033 shutdown_connections 2024-05-03T18:38:33.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.517+0000 7f4a93841b80 1 --2- 172.21.15.33:0/4277066033 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5556749df3b0 0x5556749e17d0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:33.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.517+0000 7f4a93841b80 1 --2- 172.21.15.33:0/4277066033 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555674960360 0x5556749771b0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:33.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.517+0000 7f4a93841b80 1 --2- 172.21.15.33:0/4277066033 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55567495daf0 0x555674978c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:33.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.517+0000 7f4a93841b80 1 --2- 172.21.15.33:0/4277066033 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55567495fa40 0x555674979160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:33.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.517+0000 7f4a63e00700 0 client.36152 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:38:33.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.517+0000 7f4a93841b80 1 -- 172.21.15.33:0/4277066033 >> 172.21.15.33:0/4277066033 conn(0x555674888b60 msgr2=0x555674962590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:38:33.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.517+0000 7f4a93841b80 1 -- 172.21.15.33:0/4277066033 shutdown_connections 2024-05-03T18:38:33.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.517+0000 7f4a93841b80 1 -- 172.21.15.33:0/4277066033 wait complete. 2024-05-03T18:38:33.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.518+0000 7f4a93841b80 20 client.36152 trim_cache size 0 max 16384 2024-05-03T18:38:33.520 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-03T18:38:33.520 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-05-03T18:38:33.520 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-05-03T18:38:33.520 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-03T18:38:33.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.519+0000 7f4a88e00700 1 -- 172.21.15.33:0/275877686 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f4a680b2af0 msgr2=0x7f4a680b4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:33.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.519+0000 7f4a88e00700 1 --2- 172.21.15.33:0/275877686 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f4a680b2af0 0x7f4a680b4fb0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f4a70009350 tx=0x7f4a7000b560 comp rx=0 tx=0).stop 2024-05-03T18:38:33.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.519+0000 7f4a88e00700 1 -- 172.21.15.33:0/275877686 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a84075840 msgr2=0x7f4a84074ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:33.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.519+0000 7f4a88e00700 1 --2- 172.21.15.33:0/275877686 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a84075840 0x7f4a84074ae0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f4a7400d980 tx=0x7f4a7400dc90 comp rx=0 tx=0).stop 2024-05-03T18:38:33.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.520+0000 7f4a88e00700 1 -- 172.21.15.33:0/275877686 shutdown_connections 2024-05-03T18:38:33.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.520+0000 7f4a88e00700 1 --2- 172.21.15.33:0/275877686 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a84114400 0x7f4a8406dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:33.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.520+0000 7f4a88e00700 1 --2- 172.21.15.33:0/275877686 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f4a680b2af0 0x7f4a680b4fb0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:33.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.520+0000 7f4a88e00700 1 --2- 172.21.15.33:0/275877686 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a84075840 0x7f4a84074ae0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:33.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.520+0000 7f4a88e00700 1 --2- 172.21.15.33:0/275877686 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a84076160 0x7f4a84075020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:33.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.520+0000 7f4a88e00700 1 -- 172.21.15.33:0/275877686 >> 172.21.15.33:0/275877686 conn(0x7f4a840feee0 msgr2=0x7f4a840ffb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:38:33.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.520+0000 7f4a88e00700 1 -- 172.21.15.33:0/275877686 shutdown_connections 2024-05-03T18:38:33.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:33.520+0000 7f4a88e00700 1 -- 172.21.15.33:0/275877686 wait complete. 2024-05-03T18:38:34.442 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:38:34 smithi033 ceph-mon[28330]: pgmap v354: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 11 MiB/s wr, 758 op/s 2024-05-03T18:38:34.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:38:34 smithi099 ceph-mon[31658]: pgmap v354: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 11 MiB/s wr, 758 op/s 2024-05-03T18:38:34.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:38:34 smithi150 ceph-mon[32492]: pgmap v354: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 11 MiB/s wr, 758 op/s 2024-05-03T18:38:35.022 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-03T18:38:35.023 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T18:38:35.110 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 440.0 s, 569.8 tps, lat 175.571 ms stddev 93.797 2024-05-03T18:38:35.901 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:38:36.171 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:38:36 smithi033 ceph-mon[28330]: pgmap v355: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 8.3 MiB/s wr, 530 op/s 2024-05-03T18:38:36.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:38:36 smithi099 ceph-mon[31658]: pgmap v355: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 8.3 MiB/s wr, 530 op/s 2024-05-03T18:38:36.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:38:36 smithi150 ceph-mon[32492]: pgmap v355: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 8.3 MiB/s wr, 530 op/s 2024-05-03T18:38:38.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:38:38 smithi033 ceph-mon[28330]: pgmap v356: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 8.3 MiB/s wr, 530 op/s 2024-05-03T18:38:38.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:38:38 smithi099 ceph-mon[31658]: pgmap v356: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 8.3 MiB/s wr, 530 op/s 2024-05-03T18:38:38.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:38:38 smithi150 ceph-mon[32492]: pgmap v356: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 8.3 MiB/s wr, 530 op/s 2024-05-03T18:38:40.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:40.107+0000 7fb3a1c00700 1 -- 172.21.15.33:0/2699456080 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb39c109510 msgr2=0x7fb39c1098f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:40.111 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 445.0 s, 751.6 tps, lat 133.340 ms stddev 66.309 2024-05-03T18:38:40.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:40.107+0000 7fb3a1c00700 1 --2- 172.21.15.33:0/2699456080 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb39c109510 0x7fb39c1098f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb38c009990 tx=0x7fb38c009ca0 comp rx=0 tx=0).stop 2024-05-03T18:38:40.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:40.108+0000 7fb3a1c00700 1 -- 172.21.15.33:0/2699456080 shutdown_connections 2024-05-03T18:38:40.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:40.108+0000 7fb3a1c00700 1 --2- 172.21.15.33:0/2699456080 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb39c06bb40 0x7fb39c110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:40.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:40.108+0000 7fb3a1c00700 1 --2- 172.21.15.33:0/2699456080 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb39c109510 0x7fb39c1098f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:40.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:40.108+0000 7fb3a1c00700 1 --2- 172.21.15.33:0/2699456080 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb39c06b180 0x7fb39c06b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:40.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:40.108+0000 7fb3a1c00700 1 -- 172.21.15.33:0/2699456080 >> 172.21.15.33:0/2699456080 conn(0x7fb39c079350 msgr2=0x7fb39c079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:38:40.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:40.109+0000 7fb3a1c00700 1 -- 172.21.15.33:0/2699456080 shutdown_connections 2024-05-03T18:38:40.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:40.109+0000 7fb3a1c00700 1 -- 172.21.15.33:0/2699456080 wait complete. 2024-05-03T18:38:40.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:40.111+0000 7fb3a1c00700 1 Processor -- start 2024-05-03T18:38:40.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:40.111+0000 7fb3a1c00700 1 -- start start 2024-05-03T18:38:40.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:40.112+0000 7fb3a1c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb39c06b180 0x7fb39c102470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:40.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:40.112+0000 7fb3a1c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb39c06bb40 0x7fb39c1029b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:40.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:40.112+0000 7fb3a1c00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb39c109510 0x7fb39c102ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:40.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:40.112+0000 7fb3a1c00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb39c113520 con 0x7fb39c109510 2024-05-03T18:38:40.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:40.112+0000 7fb3a1c00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb39c1133a0 con 0x7fb39c06bb40 2024-05-03T18:38:40.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:40.112+0000 7fb3a1c00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb39c1136a0 con 0x7fb39c06b180 2024-05-03T18:38:40.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:40.113+0000 7fb39a000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb39c06bb40 0x7fb39c1029b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:40.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:40.113+0000 7fb39aa00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb39c06b180 0x7fb39c102470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:40.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:40.113+0000 7fb39a000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb39c06bb40 0x7fb39c1029b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:53378/0 (socket says 172.21.15.33:53378) 2024-05-03T18:38:40.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:40.113+0000 7fb39a000700 1 -- 172.21.15.33:0/1745744070 learned_addr learned my addr 172.21.15.33:0/1745744070 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:38:40.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:40.113+0000 7fb39b400700 1 --2- 172.21.15.33:0/1745744070 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb39c109510 0x7fb39c102ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:40.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:40.113+0000 7fb39a000700 1 -- 172.21.15.33:0/1745744070 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb39c06b180 msgr2=0x7fb39c102470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:40.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:40.113+0000 7fb39a000700 1 --2- 172.21.15.33:0/1745744070 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb39c06b180 0x7fb39c102470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:40.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:40.113+0000 7fb39a000700 1 -- 172.21.15.33:0/1745744070 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb39c109510 msgr2=0x7fb39c102ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:40.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:40.113+0000 7fb39a000700 1 --2- 172.21.15.33:0/1745744070 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb39c109510 0x7fb39c102ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:40.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:40.113+0000 7fb39a000700 1 -- 172.21.15.33:0/1745744070 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb39c1a8070 con 0x7fb39c06bb40 2024-05-03T18:38:40.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:40.114+0000 7fb39a000700 1 --2- 172.21.15.33:0/1745744070 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb39c06bb40 0x7fb39c1029b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb38800d980 tx=0x7fb38800dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:38:40.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:40.114+0000 7fb393400700 1 -- 172.21.15.33:0/1745744070 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb388016070 con 0x7fb39c06bb40 2024-05-03T18:38:40.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:40.114+0000 7fb393400700 1 -- 172.21.15.33:0/1745744070 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb388004830 con 0x7fb39c06bb40 2024-05-03T18:38:40.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:40.114+0000 7fb393400700 1 -- 172.21.15.33:0/1745744070 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb388005440 con 0x7fb39c06bb40 2024-05-03T18:38:40.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:40.114+0000 7fb3a1c00700 1 -- 172.21.15.33:0/1745744070 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb39c1a8360 con 0x7fb39c06bb40 2024-05-03T18:38:40.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:40.114+0000 7fb3a1c00700 1 -- 172.21.15.33:0/1745744070 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb39c1a8790 con 0x7fb39c06bb40 2024-05-03T18:38:40.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:40.116+0000 7fb3a1c00700 1 -- 172.21.15.33:0/1745744070 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb39c10b9f0 con 0x7fb39c06bb40 2024-05-03T18:38:40.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:40.122+0000 7fb393400700 1 -- 172.21.15.33:0/1745744070 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fb3880055e0 con 0x7fb39c06bb40 2024-05-03T18:38:40.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:40.124+0000 7fb393400700 1 --2- 172.21.15.33:0/1745744070 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fb3800b2a70 0x7fb3800b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:40.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:40.124+0000 7fb393400700 1 -- 172.21.15.33:0/1745744070 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9518+0+0 (secure 0 0 0) 0x7fb3880e2d60 con 0x7fb39c06bb40 2024-05-03T18:38:40.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:40.125+0000 7fb39aa00700 1 --2- 172.21.15.33:0/1745744070 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fb3800b2a70 0x7fb3800b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:40.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:40.126+0000 7fb39aa00700 1 --2- 172.21.15.33:0/1745744070 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fb3800b2a70 0x7fb3800b4f30 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb38c006010 tx=0x7fb38c00b560 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:38:40.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:40.130+0000 7fb393400700 1 -- 172.21.15.33:0/1745744070 <== mon.2 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fb38801f020 con 0x7fb39c06bb40 2024-05-03T18:38:40.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:38:40 smithi033 ceph-mon[28330]: pgmap v357: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 13 MiB/s wr, 776 op/s 2024-05-03T18:38:40.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:38:40 smithi099 ceph-mon[31658]: pgmap v357: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 13 MiB/s wr, 776 op/s 2024-05-03T18:38:40.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:40.614+0000 7fb3a1c00700 1 -- 172.21.15.33:0/1745744070 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb39c06a190 con 0x7fb39c06bb40 2024-05-03T18:38:40.619 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:38:40.619 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1327393503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1327393503},{"type":"v1","addr":"172.21.15.33:6835","nonce":1327393503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/2461472948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":2461472948},{"type":"v1","addr":"172.21.15.33:6837","nonce":2461472948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14559,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/442269281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":442269281},{"type":"v1","addr":"172.21.15.33:6839","nonce":442269281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14571,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/645571756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":645571756},{"type":"v1","addr":"172.21.15.33:6841","nonce":645571756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/203598909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":203598909},{"type":"v1","addr":"172.21.15.150:6833","nonce":203598909}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/4029842493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":4029842493},{"type":"v1","addr":"172.21.15.150:6835","nonce":4029842493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3463664423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3463664423},{"type":"v1","addr":"172.21.15.150:6837","nonce":3463664423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24481,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6839/4051648018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":4051648018},{"type":"v1","addr":"172.21.15.150:6839","nonce":4051648018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34418,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1865105479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1865105479},{"type":"v1","addr":"172.21.15.99:6833","nonce":1865105479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34445,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/1206393914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1206393914},{"type":"v1","addr":"172.21.15.99:6835","nonce":1206393914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-03T18:22:28.837091+0000","modified":"2024-05-03T18:23:49.758398+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34508},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34472":{"gid":34472,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6837/3360970342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3360970342},{"type":"v1","addr":"172.21.15.99:6837","nonce":3360970342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34508":{"gid":34508,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/919907308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":919907308},{"type":"v1","addr":"172.21.15.99:6839","nonce":919907308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34508,"qdb_cluster":[34508]},"id":1}]} 2024-05-03T18:38:40.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:40.616+0000 7fb393400700 1 -- 172.21.15.33:0/1745744070 <== mon.2 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11801 (secure 0 0 0) 0x7fb388055e20 con 0x7fb39c06bb40 2024-05-03T18:38:40.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:40.621+0000 7fb3a1c00700 1 -- 172.21.15.33:0/1745744070 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fb3800b2a70 msgr2=0x7fb3800b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:40.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:40.621+0000 7fb3a1c00700 1 --2- 172.21.15.33:0/1745744070 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fb3800b2a70 0x7fb3800b4f30 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb38c006010 tx=0x7fb38c00b560 comp rx=0 tx=0).stop 2024-05-03T18:38:40.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:40.621+0000 7fb3a1c00700 1 -- 172.21.15.33:0/1745744070 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb39c06bb40 msgr2=0x7fb39c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:40.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:40.621+0000 7fb3a1c00700 1 --2- 172.21.15.33:0/1745744070 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb39c06bb40 0x7fb39c1029b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb38800d980 tx=0x7fb38800dc90 comp rx=0 tx=0).stop 2024-05-03T18:38:40.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:40.621+0000 7fb3a1c00700 1 -- 172.21.15.33:0/1745744070 shutdown_connections 2024-05-03T18:38:40.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:40.621+0000 7fb3a1c00700 1 --2- 172.21.15.33:0/1745744070 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb39c109510 0x7fb39c102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:40.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:40.621+0000 7fb3a1c00700 1 --2- 172.21.15.33:0/1745744070 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb39c06b180 0x7fb39c102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:40.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:40.621+0000 7fb3a1c00700 1 --2- 172.21.15.33:0/1745744070 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fb3800b2a70 0x7fb3800b4f30 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:40.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:40.621+0000 7fb3a1c00700 1 --2- 172.21.15.33:0/1745744070 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb39c06bb40 0x7fb39c1029b0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:40.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:40.621+0000 7fb3a1c00700 1 -- 172.21.15.33:0/1745744070 >> 172.21.15.33:0/1745744070 conn(0x7fb39c079350 msgr2=0x7fb39c101180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:38:40.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:40.623+0000 7fb3a1c00700 1 -- 172.21.15.33:0/1745744070 shutdown_connections 2024-05-03T18:38:40.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:40.623+0000 7fb3a1c00700 1 -- 172.21.15.33:0/1745744070 wait complete. 2024-05-03T18:38:40.630 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 21 2024-05-03T18:38:40.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:38:40 smithi150 ceph-mon[32492]: pgmap v357: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 13 MiB/s wr, 776 op/s 2024-05-03T18:38:41.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:38:41 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/1745744070' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:38:41.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:38:41 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/1745744070' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:38:41.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:38:41 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/1745744070' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:38:42.422 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:38:42 smithi033 ceph-mon[28330]: pgmap v358: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 8.5 MiB/s wr, 517 op/s 2024-05-03T18:38:42.429 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-03T18:38:42.534 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:38:42 smithi150 ceph-mon[32492]: pgmap v358: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 8.5 MiB/s wr, 517 op/s 2024-05-03T18:38:42.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:38:42 smithi099 ceph-mon[31658]: pgmap v358: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 8.5 MiB/s wr, 517 op/s 2024-05-03T18:38:43.351 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:38:44.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:38:44 smithi033 ceph-mon[28330]: pgmap v359: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 12 MiB/s wr, 765 op/s 2024-05-03T18:38:44.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:38:44 smithi099 ceph-mon[31658]: pgmap v359: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 12 MiB/s wr, 765 op/s 2024-05-03T18:38:44.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:38:44 smithi150 ceph-mon[32492]: pgmap v359: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 12 MiB/s wr, 765 op/s 2024-05-03T18:38:45.111 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 450.0 s, 609.9 tps, lat 162.264 ms stddev 97.158 2024-05-03T18:38:46.366 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:38:46 smithi033 ceph-mon[28330]: pgmap v360: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 8.2 MiB/s wr, 494 op/s 2024-05-03T18:38:46.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:38:46 smithi099 ceph-mon[31658]: pgmap v360: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 8.2 MiB/s wr, 494 op/s 2024-05-03T18:38:46.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:38:46 smithi150 ceph-mon[32492]: pgmap v360: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 8.2 MiB/s wr, 494 op/s 2024-05-03T18:38:47.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.420+0000 7fab70800700 1 -- 172.21.15.33:0/341891150 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab6806b180 msgr2=0x7fab6806b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:47.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.420+0000 7fab70800700 1 --2- 172.21.15.33:0/341891150 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab6806b180 0x7fab6806b600 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fab5c009990 tx=0x7fab5c009ca0 comp rx=0 tx=0).stop 2024-05-03T18:38:47.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.420+0000 7fab70800700 1 -- 172.21.15.33:0/341891150 shutdown_connections 2024-05-03T18:38:47.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.420+0000 7fab70800700 1 --2- 172.21.15.33:0/341891150 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab6806bb40 0x7fab68110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:47.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.420+0000 7fab70800700 1 --2- 172.21.15.33:0/341891150 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab68109510 0x7fab681098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:47.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.420+0000 7fab70800700 1 --2- 172.21.15.33:0/341891150 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab6806b180 0x7fab6806b600 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:47.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.420+0000 7fab70800700 1 -- 172.21.15.33:0/341891150 >> 172.21.15.33:0/341891150 conn(0x7fab68079350 msgr2=0x7fab68079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:38:47.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.420+0000 7fab70800700 1 -- 172.21.15.33:0/341891150 shutdown_connections 2024-05-03T18:38:47.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.420+0000 7fab70800700 1 -- 172.21.15.33:0/341891150 wait complete. 2024-05-03T18:38:47.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.422+0000 7fab70800700 1 Processor -- start 2024-05-03T18:38:47.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.422+0000 7fab70800700 1 -- start start 2024-05-03T18:38:47.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.423+0000 7fab70800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab6806b180 0x7fab68110230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:47.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.423+0000 7fab70800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab6806bb40 0x7fab6810d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:47.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.424+0000 7fab70800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab68109510 0x7fab6810d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:47.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.424+0000 7fab6de00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab6806b180 0x7fab68110230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:47.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.424+0000 7fab6de00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab6806b180 0x7fab68110230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60710/0 (socket says 172.21.15.33:60710) 2024-05-03T18:38:47.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.424+0000 7fab70800700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fab68103d70 con 0x7fab6806b180 2024-05-03T18:38:47.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.425+0000 7fab6de00700 1 -- 172.21.15.33:0/2089299567 learned_addr learned my addr 172.21.15.33:0/2089299567 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:38:47.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.425+0000 7fab70800700 1 -- 172.21.15.33:0/2089299567 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fab68103bf0 con 0x7fab68109510 2024-05-03T18:38:47.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.425+0000 7fab70800700 1 -- 172.21.15.33:0/2089299567 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fab68103ef0 con 0x7fab6806bb40 2024-05-03T18:38:47.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.424+0000 7fab6d400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab6806bb40 0x7fab6810d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:47.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.425+0000 7fab6e800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab68109510 0x7fab6810d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:47.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.426+0000 7fab6d400700 1 -- 172.21.15.33:0/2089299567 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab68109510 msgr2=0x7fab6810d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:47.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.426+0000 7fab6d400700 1 --2- 172.21.15.33:0/2089299567 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab68109510 0x7fab6810d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:47.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.426+0000 7fab6d400700 1 -- 172.21.15.33:0/2089299567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab6806b180 msgr2=0x7fab68110230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:47.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.426+0000 7fab6d400700 1 --2- 172.21.15.33:0/2089299567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab6806b180 0x7fab68110230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:47.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.426+0000 7fab6d400700 1 -- 172.21.15.33:0/2089299567 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab68073f10 con 0x7fab6806bb40 2024-05-03T18:38:47.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.426+0000 7fab6d400700 1 --2- 172.21.15.33:0/2089299567 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab6806bb40 0x7fab6810d2e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fab5c000c00 tx=0x7fab5c0057c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:38:47.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.427+0000 7fab62a00700 1 -- 172.21.15.33:0/2089299567 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab5c026070 con 0x7fab6806bb40 2024-05-03T18:38:47.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.427+0000 7fab70800700 1 -- 172.21.15.33:0/2089299567 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab680741a0 con 0x7fab6806bb40 2024-05-03T18:38:47.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.427+0000 7fab70800700 1 -- 172.21.15.33:0/2089299567 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab680746d0 con 0x7fab6806bb40 2024-05-03T18:38:47.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.427+0000 7fab62a00700 1 -- 172.21.15.33:0/2089299567 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fab5c021410 con 0x7fab6806bb40 2024-05-03T18:38:47.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.427+0000 7fab62a00700 1 -- 172.21.15.33:0/2089299567 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab5c017650 con 0x7fab6806bb40 2024-05-03T18:38:47.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.433+0000 7fab62a00700 1 -- 172.21.15.33:0/2089299567 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fab5c009d10 con 0x7fab6806bb40 2024-05-03T18:38:47.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.436+0000 7fab62a00700 1 --2- 172.21.15.33:0/2089299567 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fab500b2a70 0x7fab500b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:47.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.436+0000 7fab62a00700 1 -- 172.21.15.33:0/2089299567 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9518+0+0 (secure 0 0 0) 0x7fab5c0e4060 con 0x7fab6806bb40 2024-05-03T18:38:47.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.438+0000 7fab6de00700 1 --2- 172.21.15.33:0/2089299567 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fab500b2a70 0x7fab500b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:47.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.468+0000 7fab6de00700 1 --2- 172.21.15.33:0/2089299567 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fab500b2a70 0x7fab500b4f30 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fab580046c0 tx=0x7fab58004040 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:38:47.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.472+0000 7fab7b14eb80 1 Processor -- start 2024-05-03T18:38:47.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.473+0000 7fab7b14eb80 1 -- start start 2024-05-03T18:38:47.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.473+0000 7fab7b14eb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56238dc167b0 0x56238dc16b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:47.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.474+0000 7fab7b14eb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56238dc170d0 0x56238dc11fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:47.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.474+0000 7fab7b14eb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56238dc125d0 0x56238dc12a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:47.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.474+0000 7fab6de00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56238dc167b0 0x56238dc16b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:47.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.474+0000 7fab6de00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56238dc167b0 0x56238dc16b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:45826/0 (socket says 172.21.15.33:45826) 2024-05-03T18:38:47.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.474+0000 7fab6de00700 1 -- 172.21.15.33:0/2241214349 learned_addr learned my addr 172.21.15.33:0/2241214349 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:38:47.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.475+0000 7fab6e800700 1 --2- 172.21.15.33:0/2241214349 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56238dc125d0 0x56238dc12a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:47.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.475+0000 7fab6d400700 1 --2- 172.21.15.33:0/2241214349 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56238dc170d0 0x56238dc11fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:47.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.475+0000 7fab7b14eb80 1 -- 172.21.15.33:0/2241214349 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56238db64a20 con 0x56238dc170d0 2024-05-03T18:38:47.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.475+0000 7fab7b14eb80 1 -- 172.21.15.33:0/2241214349 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56238db02760 con 0x56238dc167b0 2024-05-03T18:38:47.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.475+0000 7fab7b14eb80 1 -- 172.21.15.33:0/2241214349 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56238db609e0 con 0x56238dc125d0 2024-05-03T18:38:47.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.476+0000 7fab6e800700 1 -- 172.21.15.33:0/2241214349 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56238dc167b0 msgr2=0x56238dc16b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:38:47.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.477+0000 7fab6e800700 1 --2- 172.21.15.33:0/2241214349 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56238dc167b0 0x56238dc16b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:47.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.477+0000 7fab6e800700 1 -- 172.21.15.33:0/2241214349 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56238dc170d0 msgr2=0x56238dc11fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:38:47.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.477+0000 7fab6e800700 1 --2- 172.21.15.33:0/2241214349 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56238dc170d0 0x56238dc11fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:47.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.477+0000 7fab6e800700 1 -- 172.21.15.33:0/2241214349 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56238dab18d0 con 0x56238dc125d0 2024-05-03T18:38:47.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.477+0000 7fab6e800700 1 --2- 172.21.15.33:0/2241214349 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56238dc125d0 0x56238dc12a30 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fab64011580 tx=0x7fab64009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7432ab1a8435c498 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:38:47.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.480+0000 7fab57400700 1 -- 172.21.15.33:0/2241214349 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab64011e80 con 0x56238dc125d0 2024-05-03T18:38:47.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.480+0000 7fab57400700 1 -- 172.21.15.33:0/2241214349 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fab64004bb0 con 0x56238dc125d0 2024-05-03T18:38:47.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.480+0000 7fab57400700 1 -- 172.21.15.33:0/2241214349 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab64004660 con 0x56238dc125d0 2024-05-03T18:38:47.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.481+0000 7fab7b14eb80 1 -- 172.21.15.33:0/2241214349 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56238dc125d0 msgr2=0x56238dc12a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:47.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.481+0000 7fab7b14eb80 1 --2- 172.21.15.33:0/2241214349 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56238dc125d0 0x56238dc12a30 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fab64011580 tx=0x7fab64009920 comp rx=0 tx=0).stop 2024-05-03T18:38:47.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.481+0000 7fab7b14eb80 1 -- 172.21.15.33:0/2241214349 shutdown_connections 2024-05-03T18:38:47.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.481+0000 7fab7b14eb80 1 --2- 172.21.15.33:0/2241214349 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56238dc125d0 0x56238dc12a30 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:47.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.481+0000 7fab7b14eb80 1 --2- 172.21.15.33:0/2241214349 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56238dc167b0 0x56238dc16b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:47.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.481+0000 7fab7b14eb80 1 --2- 172.21.15.33:0/2241214349 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56238dc170d0 0x56238dc11fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:47.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.481+0000 7fab7b14eb80 1 -- 172.21.15.33:0/2241214349 >> 172.21.15.33:0/2241214349 conn(0x56238db40b60 msgr2=0x56238dc0c7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:38:47.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.482+0000 7fab7b14eb80 1 -- 172.21.15.33:0/2241214349 shutdown_connections 2024-05-03T18:38:47.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.482+0000 7fab7b14eb80 1 -- 172.21.15.33:0/2241214349 wait complete. 2024-05-03T18:38:47.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.483+0000 7fab7b14eb80 1 -- start start 2024-05-03T18:38:47.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.483+0000 7fab7b14eb80 1 Processor -- start 2024-05-03T18:38:47.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.484+0000 7fab7b14eb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56238dc125d0 0x56238dc1de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:47.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.484+0000 7fab7b14eb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56238dc167b0 0x56238dc1b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:47.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.484+0000 7fab7b14eb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56238dc170d0 0x56238dc1b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:47.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.484+0000 7fab7b14eb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56238dc1eca0 con 0x56238dc167b0 2024-05-03T18:38:47.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.484+0000 7fab7b14eb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56238dacab20 con 0x56238dc170d0 2024-05-03T18:38:47.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.484+0000 7fab7b14eb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56238dc1ee20 con 0x56238dc125d0 2024-05-03T18:38:47.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.484+0000 7fab7b14eb80 10 client.0 _ll_register_callbacks cb 0x56238db0bc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:38:47.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.484+0000 7fab6de00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56238dc125d0 0x56238dc1de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:47.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.484+0000 7fab6de00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56238dc125d0 0x56238dc1de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:51828/0 (socket says 172.21.15.33:51828) 2024-05-03T18:38:47.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.484+0000 7fab6d400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56238dc167b0 0x56238dc1b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:47.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.484+0000 7fab6de00700 1 -- 172.21.15.33:0/1660146609 learned_addr learned my addr 172.21.15.33:0/1660146609 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:38:47.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.485+0000 7fab6de00700 1 -- 172.21.15.33:0/1660146609 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56238dc170d0 msgr2=0x56238dc1b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:38:47.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.485+0000 7fab6de00700 1 --2- 172.21.15.33:0/1660146609 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56238dc170d0 0x56238dc1b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:47.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.485+0000 7fab6de00700 1 -- 172.21.15.33:0/1660146609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56238dc167b0 msgr2=0x56238dc1b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:38:47.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.485+0000 7fab6de00700 1 --2- 172.21.15.33:0/1660146609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56238dc167b0 0x56238dc1b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:47.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.485+0000 7fab6de00700 1 -- 172.21.15.33:0/1660146609 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56238db2a420 con 0x56238dc125d0 2024-05-03T18:38:47.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.485+0000 7fab6de00700 1 --2- 172.21.15.33:0/1660146609 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56238dc125d0 0x56238dc1de60 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fab5800fe50 tx=0x7fab58004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba7396d68b9c29f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:38:47.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.485+0000 7fab4be00700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-03T18:38:47.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.486+0000 7fab4be00700 1 -- 172.21.15.33:0/1660146609 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab5801f630 con 0x56238dc125d0 2024-05-03T18:38:47.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.486+0000 7fab7b14eb80 1 -- 172.21.15.33:0/1660146609 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab64004bb0 con 0x56238dc125d0 2024-05-03T18:38:47.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.486+0000 7fab4be00700 1 -- 172.21.15.33:0/1660146609 <== mon.1 v2:172.21.15.150:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9518+0+0 (secure 0 0 0) 0x7fab5801f7d0 con 0x56238dc125d0 2024-05-03T18:38:47.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.486+0000 7fab4be00700 1 -- 172.21.15.33:0/1660146609 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fab58030050 con 0x56238dc125d0 2024-05-03T18:38:47.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.486+0000 7fab7b14eb80 10 client.26248 fetch_fsmap learned FSMap version 21 2024-05-03T18:38:47.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.486+0000 7fab7b14eb80 1 -- 172.21.15.33:0/1660146609 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x56238dc29560 con 0x56238dc125d0 2024-05-03T18:38:47.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.487+0000 7fab4be00700 1 -- 172.21.15.33:0/1660146609 <== mon.1 v2:172.21.15.150:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fab58023030 con 0x56238dc125d0 2024-05-03T18:38:47.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.487+0000 7fab4be00700 1 -- 172.21.15.33:0/1660146609 <== mon.1 v2:172.21.15.150:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fab580287b0 con 0x56238dc125d0 2024-05-03T18:38:47.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.487+0000 7fab7b14eb80 10 client.26248 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:38:47.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.487+0000 7fab7b14eb80 10 client.26248 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:38:47.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.487+0000 7fab7b14eb80 20 client.26248 populate_metadata read hostname 'smithi033' 2024-05-03T18:38:47.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.487+0000 7fab7b14eb80 1 --2- 172.21.15.33:0/1660146609 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x56238dc2bb40 0x56238dc2dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:47.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.487+0000 7fab7b14eb80 4 client.26248 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:38:47.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.487+0000 7fab7b14eb80 1 -- 172.21.15.33:0/1660146609 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fab64004bb0 con 0x56238dc2bb40 2024-05-03T18:38:47.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.491+0000 7fab6d400700 1 --2- 172.21.15.33:0/1660146609 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x56238dc2bb40 0x56238dc2dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:47.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.491+0000 7fab6d400700 1 --2- 172.21.15.33:0/1660146609 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x56238dc2bb40 0x56238dc2dfc0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82dbb0663ec78141 server_cookie=b123087a2179a427 in_seq=0 out_seq=0 2024-05-03T18:38:47.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.492+0000 7fab4be00700 10 client.26248 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:38:47.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.493+0000 7fab4be00700 1 -- 172.21.15.33:0/1660146609 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7fab64004bb0 con 0x56238dc2bb40 2024-05-03T18:38:47.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.494+0000 7fab4be00700 10 client.26248 handle_command_reply: tid=0 2024-05-03T18:38:47.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.494+0000 7fab7b14eb80 1 client.26248 shutdown 2024-05-03T18:38:47.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.495+0000 7fab7b14eb80 1 -- 172.21.15.33:0/1660146609 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56238dc125d0 msgr2=0x56238dc1de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:47.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.495+0000 7fab7b14eb80 1 --2- 172.21.15.33:0/1660146609 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56238dc125d0 0x56238dc1de60 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fab5800fe50 tx=0x7fab58004750 comp rx=0 tx=0).stop 2024-05-03T18:38:47.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.495+0000 7fab7b14eb80 1 -- 172.21.15.33:0/1660146609 shutdown_connections 2024-05-03T18:38:47.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.495+0000 7fab7b14eb80 1 --2- 172.21.15.33:0/1660146609 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x56238dc2bb40 0x56238dc2dfc0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:47.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.495+0000 7fab7b14eb80 1 --2- 172.21.15.33:0/1660146609 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56238dc170d0 0x56238dc1b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:47.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.495+0000 7fab7b14eb80 1 --2- 172.21.15.33:0/1660146609 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56238dc125d0 0x56238dc1de60 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:47.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.495+0000 7fab7b14eb80 1 --2- 172.21.15.33:0/1660146609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56238dc167b0 0x56238dc1b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:47.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.495+0000 7fab7b14eb80 1 -- 172.21.15.33:0/1660146609 >> 172.21.15.33:0/1660146609 conn(0x56238db40b60 msgr2=0x56238dc17e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:38:47.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.495+0000 7fab4be00700 0 client.26248 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:38:47.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.497+0000 7fab7b14eb80 1 -- 172.21.15.33:0/1660146609 shutdown_connections 2024-05-03T18:38:47.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.497+0000 7fab7b14eb80 1 -- 172.21.15.33:0/1660146609 wait complete. 2024-05-03T18:38:47.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.497+0000 7fab7b14eb80 20 client.26248 trim_cache size 0 max 16384 2024-05-03T18:38:47.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.530+0000 7fab7b14eb80 1 Processor -- start 2024-05-03T18:38:47.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.530+0000 7fab7b14eb80 1 -- start start 2024-05-03T18:38:47.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.530+0000 7fab7b14eb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56238dc125d0 0x56238dc27dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:47.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.530+0000 7fab7b14eb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56238dc167b0 0x56238dc2df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:47.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.530+0000 7fab7b14eb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56238dc170d0 0x56238dc19af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:47.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.530+0000 7fab7b14eb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fab4c00cdf0 con 0x56238dc167b0 2024-05-03T18:38:47.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.530+0000 7fab7b14eb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fab4c00f650 con 0x56238dc125d0 2024-05-03T18:38:47.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.530+0000 7fab7b14eb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fab4c00d440 con 0x56238dc170d0 2024-05-03T18:38:47.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.530+0000 7fab6e800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56238dc170d0 0x56238dc19af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:47.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.530+0000 7fab6e800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56238dc170d0 0x56238dc19af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:51830/0 (socket says 172.21.15.33:51830) 2024-05-03T18:38:47.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.530+0000 7fab6e800700 1 -- 172.21.15.33:0/3878274603 learned_addr learned my addr 172.21.15.33:0/3878274603 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:38:47.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.532+0000 7fab6de00700 1 --2- 172.21.15.33:0/3878274603 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56238dc125d0 0x56238dc27dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:47.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.532+0000 7fab6e800700 1 -- 172.21.15.33:0/3878274603 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56238dc125d0 msgr2=0x56238dc27dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:38:47.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.532+0000 7fab6e800700 1 --2- 172.21.15.33:0/3878274603 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56238dc125d0 0x56238dc27dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:47.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.532+0000 7fab6e800700 1 -- 172.21.15.33:0/3878274603 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56238dc167b0 msgr2=0x56238dc2df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:38:47.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.532+0000 7fab6e800700 1 --2- 172.21.15.33:0/3878274603 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56238dc167b0 0x56238dc2df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:47.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.532+0000 7fab6e800700 1 -- 172.21.15.33:0/3878274603 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56238dc15d40 con 0x56238dc170d0 2024-05-03T18:38:47.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.532+0000 7fab6e800700 1 --2- 172.21.15.33:0/3878274603 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56238dc170d0 0x56238dc19af0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fab64012010 tx=0x7fab64005b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f56baf0b0bb7080e server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:38:47.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.534+0000 7fab56a00700 1 -- 172.21.15.33:0/3878274603 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab64004c90 con 0x56238dc170d0 2024-05-03T18:38:47.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.534+0000 7fab56a00700 1 -- 172.21.15.33:0/3878274603 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fab64021070 con 0x56238dc170d0 2024-05-03T18:38:47.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.534+0000 7fab56a00700 1 -- 172.21.15.33:0/3878274603 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab64007500 con 0x56238dc170d0 2024-05-03T18:38:47.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.534+0000 7fab7b14eb80 1 -- 172.21.15.33:0/3878274603 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56238dc170d0 msgr2=0x56238dc19af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:47.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.534+0000 7fab7b14eb80 1 --2- 172.21.15.33:0/3878274603 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56238dc170d0 0x56238dc19af0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fab64012010 tx=0x7fab64005b50 comp rx=0 tx=0).stop 2024-05-03T18:38:47.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.536+0000 7fab7b14eb80 1 -- 172.21.15.33:0/3878274603 shutdown_connections 2024-05-03T18:38:47.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.536+0000 7fab7b14eb80 1 --2- 172.21.15.33:0/3878274603 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56238dc170d0 0x56238dc19af0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:47.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.536+0000 7fab7b14eb80 1 --2- 172.21.15.33:0/3878274603 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56238dc125d0 0x56238dc27dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:47.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.536+0000 7fab7b14eb80 1 --2- 172.21.15.33:0/3878274603 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56238dc167b0 0x56238dc2df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:47.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.536+0000 7fab7b14eb80 1 -- 172.21.15.33:0/3878274603 >> 172.21.15.33:0/3878274603 conn(0x56238db40b60 msgr2=0x56238dc1b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:38:47.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.536+0000 7fab7b14eb80 1 -- 172.21.15.33:0/3878274603 shutdown_connections 2024-05-03T18:38:47.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.536+0000 7fab7b14eb80 1 -- 172.21.15.33:0/3878274603 wait complete. 2024-05-03T18:38:47.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.538+0000 7fab7b14eb80 1 -- start start 2024-05-03T18:38:47.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.538+0000 7fab7b14eb80 1 Processor -- start 2024-05-03T18:38:47.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.540+0000 7fab7b14eb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56238dc125d0 0x56238dc24140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:47.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.540+0000 7fab7b14eb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56238dc167b0 0x56238dc24680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:47.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.540+0000 7fab7b14eb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56238dc170d0 0x56238dc0fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:47.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.540+0000 7fab7b14eb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fab4c00db10 con 0x56238dc125d0 2024-05-03T18:38:47.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.540+0000 7fab7b14eb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fab4c00ef80 con 0x56238dc167b0 2024-05-03T18:38:47.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.540+0000 7fab7b14eb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fab4c00e1e0 con 0x56238dc170d0 2024-05-03T18:38:47.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.540+0000 7fab7b14eb80 10 client.0 _ll_register_callbacks cb 0x56238db0bc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:38:47.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.542+0000 7fab6d400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56238dc167b0 0x56238dc24680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:47.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.542+0000 7fab6d400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56238dc167b0 0x56238dc24680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:45848/0 (socket says 172.21.15.33:45848) 2024-05-03T18:38:47.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.542+0000 7fab6d400700 1 -- 172.21.15.33:0/2522388711 learned_addr learned my addr 172.21.15.33:0/2522388711 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:38:47.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.542+0000 7fab6e800700 1 --2- 172.21.15.33:0/2522388711 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56238dc170d0 0x56238dc0fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:47.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.542+0000 7fab6d400700 1 -- 172.21.15.33:0/2522388711 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56238dc170d0 msgr2=0x56238dc0fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:38:47.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.542+0000 7fab6d400700 1 --2- 172.21.15.33:0/2522388711 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56238dc170d0 0x56238dc0fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:47.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.542+0000 7fab6d400700 1 -- 172.21.15.33:0/2522388711 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56238dc125d0 msgr2=0x56238dc24140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:38:47.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.542+0000 7fab6d400700 1 --2- 172.21.15.33:0/2522388711 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56238dc125d0 0x56238dc24140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:47.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.542+0000 7fab6d400700 1 -- 172.21.15.33:0/2522388711 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56238dc102c0 con 0x56238dc167b0 2024-05-03T18:38:47.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.544+0000 7fab6d400700 1 --2- 172.21.15.33:0/2522388711 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56238dc167b0 0x56238dc24680 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fab5c0a7010 tx=0x7fab5c0a7450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58fbb9f788eec9ea server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:38:47.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.544+0000 7fab4be00700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-03T18:38:47.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.544+0000 7fab4be00700 1 -- 172.21.15.33:0/2522388711 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab5c0af340 con 0x56238dc167b0 2024-05-03T18:38:47.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.545+0000 7fab7b14eb80 1 -- 172.21.15.33:0/2522388711 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab64021070 con 0x56238dc167b0 2024-05-03T18:38:47.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.545+0000 7fab4be00700 1 -- 172.21.15.33:0/2522388711 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9518+0+0 (secure 0 0 0) 0x7fab5c0ab400 con 0x56238dc167b0 2024-05-03T18:38:47.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.545+0000 7fab4be00700 1 -- 172.21.15.33:0/2522388711 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fab5c0b3d60 con 0x56238dc167b0 2024-05-03T18:38:47.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.545+0000 7fab7b14eb80 10 client.36173 fetch_fsmap learned FSMap version 21 2024-05-03T18:38:47.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.545+0000 7fab7b14eb80 1 -- 172.21.15.33:0/2522388711 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x56238dc96bf0 con 0x56238dc167b0 2024-05-03T18:38:47.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.545+0000 7fab4be00700 1 -- 172.21.15.33:0/2522388711 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fab64021070 con 0x56238dc167b0 2024-05-03T18:38:47.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.545+0000 7fab4be00700 1 -- 172.21.15.33:0/2522388711 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fab5c0a6900 con 0x56238dc167b0 2024-05-03T18:38:47.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.545+0000 7fab7b14eb80 10 client.36173 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:38:47.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.545+0000 7fab7b14eb80 10 client.36173 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:38:47.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.545+0000 7fab7b14eb80 20 client.36173 populate_metadata read hostname 'smithi033' 2024-05-03T18:38:47.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.547+0000 7fab7b14eb80 1 --2- 172.21.15.33:0/2522388711 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x56238dc97280 0x56238dc996a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:47.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.547+0000 7fab7b14eb80 4 client.36173 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-03T18:38:47.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.547+0000 7fab7b14eb80 1 -- 172.21.15.33:0/2522388711 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56238dbdf1d0 con 0x56238dc97280 2024-05-03T18:38:47.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.547+0000 7fab6de00700 1 --2- 172.21.15.33:0/2522388711 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x56238dc97280 0x56238dc996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:47.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.550+0000 7fab4be00700 10 client.36173 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:38:47.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.550+0000 7fab6de00700 1 --2- 172.21.15.33:0/2522388711 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x56238dc97280 0x56238dc996a0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0b630c9cfa43ad8 server_cookie=851507683ae2ff0 in_seq=0 out_seq=0 2024-05-03T18:38:47.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.551+0000 7fab4be00700 1 -- 172.21.15.33:0/2522388711 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56238dbdf1d0 con 0x56238dc97280 2024-05-03T18:38:47.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.551+0000 7fab4be00700 10 client.36173 handle_command_reply: tid=0 2024-05-03T18:38:47.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.551+0000 7fab7b14eb80 1 client.36173 shutdown 2024-05-03T18:38:47.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.551+0000 7fab7b14eb80 1 -- 172.21.15.33:0/2522388711 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56238dc167b0 msgr2=0x56238dc24680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:47.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.551+0000 7fab7b14eb80 1 --2- 172.21.15.33:0/2522388711 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56238dc167b0 0x56238dc24680 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fab5c0a7010 tx=0x7fab5c0a7450 comp rx=0 tx=0).stop 2024-05-03T18:38:47.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.551+0000 7fab7b14eb80 1 -- 172.21.15.33:0/2522388711 shutdown_connections 2024-05-03T18:38:47.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.551+0000 7fab7b14eb80 1 --2- 172.21.15.33:0/2522388711 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x56238dc97280 0x56238dc996a0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:47.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.551+0000 7fab7b14eb80 1 --2- 172.21.15.33:0/2522388711 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56238dc170d0 0x56238dc0fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:47.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.551+0000 7fab4be00700 0 client.36173 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:38:47.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.551+0000 7fab7b14eb80 1 --2- 172.21.15.33:0/2522388711 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56238dc125d0 0x56238dc24140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:47.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.551+0000 7fab7b14eb80 1 --2- 172.21.15.33:0/2522388711 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56238dc167b0 0x56238dc24680 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:47.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.551+0000 7fab7b14eb80 1 -- 172.21.15.33:0/2522388711 >> 172.21.15.33:0/2522388711 conn(0x56238db40b60 msgr2=0x56238dc19a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:38:47.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.553+0000 7fab7b14eb80 1 -- 172.21.15.33:0/2522388711 shutdown_connections 2024-05-03T18:38:47.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.553+0000 7fab7b14eb80 1 -- 172.21.15.33:0/2522388711 wait complete. 2024-05-03T18:38:47.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.553+0000 7fab7b14eb80 20 client.36173 trim_cache size 0 max 16384 2024-05-03T18:38:47.555 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-03T18:38:47.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.554+0000 7fab70800700 1 -- 172.21.15.33:0/2089299567 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fab500b2a70 msgr2=0x7fab500b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:47.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.554+0000 7fab70800700 1 --2- 172.21.15.33:0/2089299567 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fab500b2a70 0x7fab500b4f30 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fab580046c0 tx=0x7fab58004040 comp rx=0 tx=0).stop 2024-05-03T18:38:47.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.554+0000 7fab70800700 1 -- 172.21.15.33:0/2089299567 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab6806bb40 msgr2=0x7fab6810d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:47.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.554+0000 7fab70800700 1 --2- 172.21.15.33:0/2089299567 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab6806bb40 0x7fab6810d2e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fab5c000c00 tx=0x7fab5c0057c0 comp rx=0 tx=0).stop 2024-05-03T18:38:47.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.555+0000 7fab70800700 1 -- 172.21.15.33:0/2089299567 shutdown_connections 2024-05-03T18:38:47.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.555+0000 7fab70800700 1 --2- 172.21.15.33:0/2089299567 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fab500b2a70 0x7fab500b4f30 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:47.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.555+0000 7fab70800700 1 --2- 172.21.15.33:0/2089299567 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab68109510 0x7fab6810d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:47.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.555+0000 7fab70800700 1 --2- 172.21.15.33:0/2089299567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab6806b180 0x7fab68110230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:47.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.555+0000 7fab70800700 1 --2- 172.21.15.33:0/2089299567 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab6806bb40 0x7fab6810d2e0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:47.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.555+0000 7fab70800700 1 -- 172.21.15.33:0/2089299567 >> 172.21.15.33:0/2089299567 conn(0x7fab68079350 msgr2=0x7fab68101040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:38:47.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.555+0000 7fab70800700 1 -- 172.21.15.33:0/2089299567 shutdown_connections 2024-05-03T18:38:47.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:47.555+0000 7fab70800700 1 -- 172.21.15.33:0/2089299567 wait complete. 2024-05-03T18:38:48.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:38:48 smithi033 ceph-mon[28330]: pgmap v361: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 8.2 MiB/s wr, 495 op/s 2024-05-03T18:38:48.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:38:48 smithi099 ceph-mon[31658]: pgmap v361: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 8.2 MiB/s wr, 495 op/s 2024-05-03T18:38:48.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:38:48 smithi150 ceph-mon[32492]: pgmap v361: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 8.2 MiB/s wr, 495 op/s 2024-05-03T18:38:49.801 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-03T18:38:49.801 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-03T18:38:50.114 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 455.0 s, 608.0 tps, lat 165.429 ms stddev 54.203 2024-05-03T18:38:50.162 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:38:50 smithi033 ceph-mon[28330]: pgmap v362: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 506 KiB/s rd, 12 MiB/s wr, 758 op/s 2024-05-03T18:38:50.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:38:50 smithi099 ceph-mon[31658]: pgmap v362: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 506 KiB/s rd, 12 MiB/s wr, 758 op/s 2024-05-03T18:38:50.591 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:38:50.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:38:50 smithi150 ceph-mon[32492]: pgmap v362: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 506 KiB/s rd, 12 MiB/s wr, 758 op/s 2024-05-03T18:38:52.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:38:52 smithi033 ceph-mon[28330]: pgmap v363: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 7.9 MiB/s wr, 512 op/s 2024-05-03T18:38:52.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:38:52 smithi099 ceph-mon[31658]: pgmap v363: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 7.9 MiB/s wr, 512 op/s 2024-05-03T18:38:52.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:38:52 smithi150 ceph-mon[32492]: pgmap v363: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 7.9 MiB/s wr, 512 op/s 2024-05-03T18:38:54.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.064+0000 7feb41600700 1 -- 172.21.15.33:0/3374646048 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb3c103ef0 msgr2=0x7feb3c1103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:54.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.064+0000 7feb41600700 1 --2- 172.21.15.33:0/3374646048 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb3c103ef0 0x7feb3c1103d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7feb2c00b0a0 tx=0x7feb2c00b3b0 comp rx=0 tx=0).stop 2024-05-03T18:38:54.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.064+0000 7feb41600700 1 -- 172.21.15.33:0/3374646048 shutdown_connections 2024-05-03T18:38:54.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.064+0000 7feb41600700 1 --2- 172.21.15.33:0/3374646048 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb3c103ef0 0x7feb3c1103d0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:54.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.064+0000 7feb41600700 1 --2- 172.21.15.33:0/3374646048 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb3c108460 0x7feb3c108840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:54.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.064+0000 7feb41600700 1 --2- 172.21.15.33:0/3374646048 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb3c103550 0x7feb3c1039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:54.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.064+0000 7feb41600700 1 -- 172.21.15.33:0/3374646048 >> 172.21.15.33:0/3374646048 conn(0x7feb3c07a600 msgr2=0x7feb3c07aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:38:54.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.064+0000 7feb41600700 1 -- 172.21.15.33:0/3374646048 shutdown_connections 2024-05-03T18:38:54.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.064+0000 7feb41600700 1 -- 172.21.15.33:0/3374646048 wait complete. 2024-05-03T18:38:54.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.066+0000 7feb41600700 1 Processor -- start 2024-05-03T18:38:54.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.066+0000 7feb41600700 1 -- start start 2024-05-03T18:38:54.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.067+0000 7feb41600700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb3c103550 0x7feb3c19f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:54.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.067+0000 7feb41600700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb3c103ef0 0x7feb3c19fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:54.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.068+0000 7feb41600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb3c108460 0x7feb3c199860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:54.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.068+0000 7feb3aa00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb3c103550 0x7feb3c19f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:54.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.068+0000 7feb3aa00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb3c103550 0x7feb3c19f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60782/0 (socket says 172.21.15.33:60782) 2024-05-03T18:38:54.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.068+0000 7feb41600700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7feb3c1135a0 con 0x7feb3c103550 2024-05-03T18:38:54.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.068+0000 7feb41600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7feb3c113420 con 0x7feb3c108460 2024-05-03T18:38:54.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.068+0000 7feb41600700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7feb3c113720 con 0x7feb3c103ef0 2024-05-03T18:38:54.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.068+0000 7feb3aa00700 1 -- 172.21.15.33:0/127035974 learned_addr learned my addr 172.21.15.33:0/127035974 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:38:54.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.068+0000 7feb3a000700 1 --2- 172.21.15.33:0/127035974 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb3c103ef0 0x7feb3c19fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:54.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.069+0000 7feb3aa00700 1 -- 172.21.15.33:0/127035974 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb3c103ef0 msgr2=0x7feb3c19fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:54.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.069+0000 7feb3aa00700 1 --2- 172.21.15.33:0/127035974 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb3c103ef0 0x7feb3c19fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:54.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.069+0000 7feb3aa00700 1 -- 172.21.15.33:0/127035974 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb3c108460 msgr2=0x7feb3c199860 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-03T18:38:54.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.069+0000 7feb3aa00700 1 --2- 172.21.15.33:0/127035974 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb3c108460 0x7feb3c199860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:54.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.069+0000 7feb3aa00700 1 -- 172.21.15.33:0/127035974 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb3c199fc0 con 0x7feb3c103550 2024-05-03T18:38:54.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.070+0000 7feb3aa00700 1 --2- 172.21.15.33:0/127035974 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb3c103550 0x7feb3c19f6a0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7feb3400e970 tx=0x7feb3400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:38:54.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.071+0000 7feb33400700 1 -- 172.21.15.33:0/127035974 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb34013be0 con 0x7feb3c103550 2024-05-03T18:38:54.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.072+0000 7feb41600700 1 -- 172.21.15.33:0/127035974 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb3c19a2b0 con 0x7feb3c103550 2024-05-03T18:38:54.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.072+0000 7feb41600700 1 -- 172.21.15.33:0/127035974 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb3c0756d0 con 0x7feb3c103550 2024-05-03T18:38:54.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.075+0000 7feb33400700 1 -- 172.21.15.33:0/127035974 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7feb34013d80 con 0x7feb3c103550 2024-05-03T18:38:54.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.075+0000 7feb33400700 1 -- 172.21.15.33:0/127035974 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb340249e0 con 0x7feb3c103550 2024-05-03T18:38:54.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.078+0000 7feb4bedab80 1 Processor -- start 2024-05-03T18:38:54.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.080+0000 7feb4bedab80 1 -- start start 2024-05-03T18:38:54.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.080+0000 7feb4bedab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562dae4ed1a0 0x562dae4ed580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:54.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.080+0000 7feb4bedab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562dae4edac0 0x562dae4e77c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:54.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.080+0000 7feb4bedab80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562dae4e7de0 0x562dae4e8240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:54.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.080+0000 7feb4bedab80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x562dae431aa0 con 0x562dae4e7de0 2024-05-03T18:38:54.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.080+0000 7feb4bedab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x562dae3cf7e0 con 0x562dae4edac0 2024-05-03T18:38:54.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.080+0000 7feb4bedab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x562dae42da60 con 0x562dae4ed1a0 2024-05-03T18:38:54.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.081+0000 7feb3a000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562dae4e7de0 0x562dae4e8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:54.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.081+0000 7feb3aa00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562dae4edac0 0x562dae4e77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:54.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.082+0000 7feb3a000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562dae4e7de0 0x562dae4e8240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60788/0 (socket says 172.21.15.33:60788) 2024-05-03T18:38:54.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.082+0000 7feb3a000700 1 -- 172.21.15.33:0/2560944154 learned_addr learned my addr 172.21.15.33:0/2560944154 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:38:54.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.083+0000 7feb3b400700 1 --2- 172.21.15.33:0/2560944154 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562dae4ed1a0 0x562dae4ed580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:54.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.084+0000 7feb3b400700 1 -- 172.21.15.33:0/2560944154 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562dae4edac0 msgr2=0x562dae4e77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:38:54.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.084+0000 7feb3b400700 1 --2- 172.21.15.33:0/2560944154 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562dae4edac0 0x562dae4e77c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:54.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.084+0000 7feb3b400700 1 -- 172.21.15.33:0/2560944154 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562dae4e7de0 msgr2=0x562dae4e8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:38:54.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.084+0000 7feb3b400700 1 --2- 172.21.15.33:0/2560944154 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562dae4e7de0 0x562dae4e8240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:54.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.084+0000 7feb3b400700 1 -- 172.21.15.33:0/2560944154 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562dae37e950 con 0x562dae4ed1a0 2024-05-03T18:38:54.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.085+0000 7feb3b400700 1 --2- 172.21.15.33:0/2560944154 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562dae4ed1a0 0x562dae4ed580 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7feb2c00b070 tx=0x7feb2c0092e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8cc04e9417ceceb3 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:38:54.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.086+0000 7feb33400700 1 -- 172.21.15.33:0/127035974 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7feb34022020 con 0x7feb3c103550 2024-05-03T18:38:54.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.086+0000 7feb27e00700 1 -- 172.21.15.33:0/2560944154 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb2c00bcb0 con 0x562dae4ed1a0 2024-05-03T18:38:54.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.086+0000 7feb27e00700 1 -- 172.21.15.33:0/2560944154 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7feb2c009490 con 0x562dae4ed1a0 2024-05-03T18:38:54.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.086+0000 7feb27e00700 1 -- 172.21.15.33:0/2560944154 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb2c01d470 con 0x562dae4ed1a0 2024-05-03T18:38:54.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.087+0000 7feb33400700 1 --2- 172.21.15.33:0/127035974 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7feb200b2a70 0x7feb200b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:54.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.088+0000 7feb33400700 1 -- 172.21.15.33:0/127035974 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9518+0+0 (secure 0 0 0) 0x7feb340e5610 con 0x7feb3c103550 2024-05-03T18:38:54.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.088+0000 7feb3aa00700 1 --2- 172.21.15.33:0/127035974 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7feb200b2a70 0x7feb200b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:54.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.088+0000 7feb4bedab80 1 -- 172.21.15.33:0/2560944154 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562dae4ed1a0 msgr2=0x562dae4ed580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:54.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.088+0000 7feb4bedab80 1 --2- 172.21.15.33:0/2560944154 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562dae4ed1a0 0x562dae4ed580 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7feb2c00b070 tx=0x7feb2c0092e0 comp rx=0 tx=0).stop 2024-05-03T18:38:54.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.090+0000 7feb4bedab80 1 -- 172.21.15.33:0/2560944154 shutdown_connections 2024-05-03T18:38:54.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.090+0000 7feb4bedab80 1 --2- 172.21.15.33:0/2560944154 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562dae4e7de0 0x562dae4e8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:54.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.090+0000 7feb4bedab80 1 --2- 172.21.15.33:0/2560944154 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562dae4ed1a0 0x562dae4ed580 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:54.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.090+0000 7feb4bedab80 1 --2- 172.21.15.33:0/2560944154 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562dae4edac0 0x562dae4e77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:54.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.090+0000 7feb4bedab80 1 -- 172.21.15.33:0/2560944154 >> 172.21.15.33:0/2560944154 conn(0x562dae457750 msgr2=0x562dae4d9980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:38:54.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.090+0000 7feb3aa00700 1 --2- 172.21.15.33:0/127035974 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7feb200b2a70 0x7feb200b4f30 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7feb340245b0 tx=0x7feb340cf050 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:38:54.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.091+0000 7feb4bedab80 1 -- 172.21.15.33:0/2560944154 shutdown_connections 2024-05-03T18:38:54.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.093+0000 7feb4bedab80 1 -- 172.21.15.33:0/2560944154 wait complete. 2024-05-03T18:38:54.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.094+0000 7feb4bedab80 1 -- start start 2024-05-03T18:38:54.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.094+0000 7feb4bedab80 1 Processor -- start 2024-05-03T18:38:54.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.096+0000 7feb4bedab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562dae4e7de0 0x562dae4f0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:54.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.096+0000 7feb4bedab80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562dae4ed1a0 0x562dae4f14f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:54.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.096+0000 7feb4bedab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562dae4edac0 0x562dae4f1a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:54.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.096+0000 7feb4bedab80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x562dae4df350 con 0x562dae4ed1a0 2024-05-03T18:38:54.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.096+0000 7feb4bedab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x562dae397ba0 con 0x562dae4e7de0 2024-05-03T18:38:54.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.096+0000 7feb4bedab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x562dae4df4d0 con 0x562dae4edac0 2024-05-03T18:38:54.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.096+0000 7feb4bedab80 10 client.0 _ll_register_callbacks cb 0x562dae3d8cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:38:54.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.098+0000 7feb3a000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562dae4ed1a0 0x562dae4f14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:54.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.098+0000 7feb3b400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562dae4e7de0 0x562dae4f0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:54.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.098+0000 7feb3b400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562dae4e7de0 0x562dae4f0fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:45898/0 (socket says 172.21.15.33:45898) 2024-05-03T18:38:54.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.098+0000 7feb3b400700 1 -- 172.21.15.33:0/1063268865 learned_addr learned my addr 172.21.15.33:0/1063268865 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:38:54.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.098+0000 7feb3b400700 1 --2- 172.21.15.33:0/1063268865 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562dae4edac0 0x562dae4f1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:54.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.098+0000 7feb3a000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562dae4ed1a0 0x562dae4f14f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60802/0 (socket says 172.21.15.33:60802) 2024-05-03T18:38:54.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.099+0000 7feb3a000700 1 -- 172.21.15.33:0/1063268865 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562dae4edac0 msgr2=0x562dae4f1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:38:54.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.100+0000 7feb3a000700 1 --2- 172.21.15.33:0/1063268865 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562dae4edac0 0x562dae4f1a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:54.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.100+0000 7feb3a000700 1 -- 172.21.15.33:0/1063268865 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562dae4e7de0 msgr2=0x562dae4f0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:38:54.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.100+0000 7feb3a000700 1 --2- 172.21.15.33:0/1063268865 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562dae4e7de0 0x562dae4f0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:54.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.100+0000 7feb3a000700 1 -- 172.21.15.33:0/1063268865 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562dae3f74a0 con 0x562dae4ed1a0 2024-05-03T18:38:54.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.100+0000 7feb3a000700 1 --2- 172.21.15.33:0/1063268865 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562dae4ed1a0 0x562dae4f14f0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7feb280116c0 tx=0x7feb280119d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37109468e6bd5d22 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:38:54.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.100+0000 7feb24c00700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:38:54.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.101+0000 7feb24c00700 1 -- 172.21.15.33:0/1063268865 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb2801e020 con 0x562dae4ed1a0 2024-05-03T18:38:54.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.101+0000 7feb4bedab80 1 -- 172.21.15.33:0/1063268865 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb2c009490 con 0x562dae4ed1a0 2024-05-03T18:38:54.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.101+0000 7feb24c00700 1 -- 172.21.15.33:0/1063268865 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9518+0+0 (secure 0 0 0) 0x7feb28004830 con 0x562dae4ed1a0 2024-05-03T18:38:54.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.105+0000 7feb24c00700 1 -- 172.21.15.33:0/1063268865 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7feb28004a30 con 0x562dae4ed1a0 2024-05-03T18:38:54.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.105+0000 7feb4bedab80 10 client.16476 fetch_fsmap learned FSMap version 21 2024-05-03T18:38:54.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.105+0000 7feb4bedab80 1 -- 172.21.15.33:0/1063268865 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x562dae5186e0 con 0x562dae4ed1a0 2024-05-03T18:38:54.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.105+0000 7feb24c00700 1 -- 172.21.15.33:0/1063268865 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7feb2c009490 con 0x562dae4ed1a0 2024-05-03T18:38:54.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.106+0000 7feb24c00700 1 -- 172.21.15.33:0/1063268865 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7feb28019070 con 0x562dae4ed1a0 2024-05-03T18:38:54.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.106+0000 7feb4bedab80 10 client.16476 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:38:54.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.106+0000 7feb4bedab80 10 client.16476 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:38:54.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.106+0000 7feb4bedab80 20 client.16476 populate_metadata read hostname 'smithi033' 2024-05-03T18:38:54.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.106+0000 7feb4bedab80 1 --2- 172.21.15.33:0/1063268865 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x562dae51ac80 0x562dae51d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:54.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.106+0000 7feb4bedab80 4 client.16476 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:38:54.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.106+0000 7feb4bedab80 1 -- 172.21.15.33:0/1063268865 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562dae4ac070 con 0x562dae51ac80 2024-05-03T18:38:54.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.109+0000 7feb3b400700 1 --2- 172.21.15.33:0/1063268865 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x562dae51ac80 0x562dae51d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:54.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.111+0000 7feb3b400700 1 --2- 172.21.15.33:0/1063268865 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x562dae51ac80 0x562dae51d070 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18904240eb9e1a2 server_cookie=21055b39c93b704a in_seq=0 out_seq=0 2024-05-03T18:38:54.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.112+0000 7feb24c00700 10 client.16476 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:38:54.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.113+0000 7feb24c00700 1 -- 172.21.15.33:0/1063268865 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562dae4ac070 con 0x562dae51ac80 2024-05-03T18:38:54.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.113+0000 7feb24c00700 10 client.16476 handle_command_reply: tid=0 2024-05-03T18:38:54.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.113+0000 7feb4bedab80 1 client.16476 shutdown 2024-05-03T18:38:54.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.114+0000 7feb4bedab80 1 -- 172.21.15.33:0/1063268865 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562dae4ed1a0 msgr2=0x562dae4f14f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:54.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.114+0000 7feb4bedab80 1 --2- 172.21.15.33:0/1063268865 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562dae4ed1a0 0x562dae4f14f0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7feb280116c0 tx=0x7feb280119d0 comp rx=0 tx=0).stop 2024-05-03T18:38:54.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.115+0000 7feb4bedab80 1 -- 172.21.15.33:0/1063268865 shutdown_connections 2024-05-03T18:38:54.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.115+0000 7feb4bedab80 1 --2- 172.21.15.33:0/1063268865 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x562dae51ac80 0x562dae51d070 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:54.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.115+0000 7feb4bedab80 1 --2- 172.21.15.33:0/1063268865 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562dae4edac0 0x562dae4f1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:54.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.115+0000 7feb4bedab80 1 --2- 172.21.15.33:0/1063268865 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562dae4e7de0 0x562dae4f0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:54.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.115+0000 7feb4bedab80 1 --2- 172.21.15.33:0/1063268865 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562dae4ed1a0 0x562dae4f14f0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:54.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.115+0000 7feb4bedab80 1 -- 172.21.15.33:0/1063268865 >> 172.21.15.33:0/1063268865 conn(0x562dae457750 msgr2=0x562dae4d4430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:38:54.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.115+0000 7feb24c00700 0 client.16476 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:38:54.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.116+0000 7feb4bedab80 1 -- 172.21.15.33:0/1063268865 shutdown_connections 2024-05-03T18:38:54.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.116+0000 7feb4bedab80 1 -- 172.21.15.33:0/1063268865 wait complete. 2024-05-03T18:38:54.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.116+0000 7feb4bedab80 20 client.16476 trim_cache size 0 max 16384 2024-05-03T18:38:54.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.161+0000 7feb4bedab80 1 Processor -- start 2024-05-03T18:38:54.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.161+0000 7feb4bedab80 1 -- start start 2024-05-03T18:38:54.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.162+0000 7feb4bedab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562dae4e7de0 0x562dae4d42b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:54.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.162+0000 7feb4bedab80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562dae4ed1a0 0x562dae522820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:54.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.162+0000 7feb4bedab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562dae4edac0 0x562dae522d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:54.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.162+0000 7feb4bedab80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7feb1c00cf40 con 0x562dae4ed1a0 2024-05-03T18:38:54.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.162+0000 7feb4bedab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7feb1c00f820 con 0x562dae4edac0 2024-05-03T18:38:54.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.162+0000 7feb4bedab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7feb1c00d610 con 0x562dae4e7de0 2024-05-03T18:38:54.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.162+0000 7feb3a000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562dae4ed1a0 0x562dae522820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:54.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.163+0000 7feb3a000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562dae4ed1a0 0x562dae522820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60816/0 (socket says 172.21.15.33:60816) 2024-05-03T18:38:54.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.163+0000 7feb3a000700 1 -- 172.21.15.33:0/4261020853 learned_addr learned my addr 172.21.15.33:0/4261020853 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:38:54.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.163+0000 7feb3b400700 1 --2- 172.21.15.33:0/4261020853 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562dae4e7de0 0x562dae4d42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:54.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.163+0000 7feb3a000700 1 -- 172.21.15.33:0/4261020853 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562dae4e7de0 msgr2=0x562dae4d42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:38:54.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.163+0000 7feb3a000700 1 --2- 172.21.15.33:0/4261020853 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562dae4e7de0 0x562dae4d42b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:54.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.164+0000 7feb3a000700 1 -- 172.21.15.33:0/4261020853 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562dae4edac0 msgr2=0x562dae522d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:38:54.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.164+0000 7feb3a000700 1 --2- 172.21.15.33:0/4261020853 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562dae4edac0 0x562dae522d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:54.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.164+0000 7feb3a000700 1 -- 172.21.15.33:0/4261020853 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562dae4ef540 con 0x562dae4ed1a0 2024-05-03T18:38:54.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.164+0000 7feb3b400700 1 --2- 172.21.15.33:0/4261020853 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562dae4e7de0 0x562dae4d42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:38:54.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.164+0000 7feb3a000700 1 --2- 172.21.15.33:0/4261020853 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562dae4ed1a0 0x562dae522820 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7feb28018ea0 tx=0x7feb28011650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bdea6d5bdbd832a server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:38:54.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.169+0000 7feb27400700 1 -- 172.21.15.33:0/4261020853 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb28031070 con 0x562dae4ed1a0 2024-05-03T18:38:54.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.169+0000 7feb27400700 1 -- 172.21.15.33:0/4261020853 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7feb2800e3b0 con 0x562dae4ed1a0 2024-05-03T18:38:54.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.169+0000 7feb27400700 1 -- 172.21.15.33:0/4261020853 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb28031070 con 0x562dae4ed1a0 2024-05-03T18:38:54.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.172+0000 7feb4bedab80 1 -- 172.21.15.33:0/4261020853 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562dae4ed1a0 msgr2=0x562dae522820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:54.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.172+0000 7feb4bedab80 1 --2- 172.21.15.33:0/4261020853 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562dae4ed1a0 0x562dae522820 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7feb28018ea0 tx=0x7feb28011650 comp rx=0 tx=0).stop 2024-05-03T18:38:54.174 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:38:54 smithi033 ceph-mon[28330]: pgmap v364: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 12 MiB/s wr, 773 op/s 2024-05-03T18:38:54.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.173+0000 7feb4bedab80 1 -- 172.21.15.33:0/4261020853 shutdown_connections 2024-05-03T18:38:54.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.173+0000 7feb4bedab80 1 --2- 172.21.15.33:0/4261020853 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562dae4edac0 0x562dae522d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:54.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.173+0000 7feb4bedab80 1 --2- 172.21.15.33:0/4261020853 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562dae4e7de0 0x562dae4d42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:54.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.173+0000 7feb4bedab80 1 --2- 172.21.15.33:0/4261020853 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562dae4ed1a0 0x562dae522820 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:54.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.173+0000 7feb4bedab80 1 -- 172.21.15.33:0/4261020853 >> 172.21.15.33:0/4261020853 conn(0x562dae457750 msgr2=0x562dae4d3ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:38:54.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.174+0000 7feb4bedab80 1 -- 172.21.15.33:0/4261020853 shutdown_connections 2024-05-03T18:38:54.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.175+0000 7feb4bedab80 1 -- 172.21.15.33:0/4261020853 wait complete. 2024-05-03T18:38:54.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.176+0000 7feb4bedab80 1 -- start start 2024-05-03T18:38:54.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.176+0000 7feb4bedab80 1 Processor -- start 2024-05-03T18:38:54.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.178+0000 7feb4bedab80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562dae4e7de0 0x562dae4e5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:54.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.179+0000 7feb3b400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562dae4e7de0 0x562dae4e5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:54.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.179+0000 7feb3b400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562dae4e7de0 0x562dae4e5ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60828/0 (socket says 172.21.15.33:60828) 2024-05-03T18:38:54.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.179+0000 7feb3b400700 1 -- 172.21.15.33:0/523995356 learned_addr learned my addr 172.21.15.33:0/523995356 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:38:54.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.179+0000 7feb4bedab80 1 --2- 172.21.15.33:0/523995356 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562dae4ed1a0 0x562dae4e5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:54.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.180+0000 7feb4bedab80 1 --2- 172.21.15.33:0/523995356 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562dae4edac0 0x562dae4e1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:54.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.180+0000 7feb4bedab80 1 -- 172.21.15.33:0/523995356 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7feb1c00dce0 con 0x562dae4e7de0 2024-05-03T18:38:54.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.180+0000 7feb4bedab80 1 -- 172.21.15.33:0/523995356 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7feb1c00f150 con 0x562dae4ed1a0 2024-05-03T18:38:54.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.180+0000 7feb4bedab80 1 -- 172.21.15.33:0/523995356 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7feb1c00e3b0 con 0x562dae4edac0 2024-05-03T18:38:54.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.181+0000 7feb4bedab80 10 client.0 _ll_register_callbacks cb 0x562dae3d8cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:38:54.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.181+0000 7feb3a000700 1 --2- 172.21.15.33:0/523995356 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562dae4ed1a0 0x562dae4e5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:54.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.182+0000 7feb3b400700 1 --2- 172.21.15.33:0/523995356 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562dae4edac0 0x562dae4e1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:54.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.183+0000 7feb3b400700 1 -- 172.21.15.33:0/523995356 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562dae4edac0 msgr2=0x562dae4e1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:38:54.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.183+0000 7feb3b400700 1 --2- 172.21.15.33:0/523995356 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562dae4edac0 0x562dae4e1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:54.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.183+0000 7feb3b400700 1 -- 172.21.15.33:0/523995356 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562dae4ed1a0 msgr2=0x562dae4e5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:38:54.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.183+0000 7feb3b400700 1 --2- 172.21.15.33:0/523995356 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562dae4ed1a0 0x562dae4e5ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:54.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.183+0000 7feb3b400700 1 -- 172.21.15.33:0/523995356 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562dae4e2770 con 0x562dae4e7de0 2024-05-03T18:38:54.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.186+0000 7feb3b400700 1 --2- 172.21.15.33:0/523995356 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562dae4e7de0 0x562dae4e5ab0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7feb2c000f80 tx=0x7feb2c00b740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18c1ea2962e1478b server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:38:54.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.187+0000 7feb24c00700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:38:54.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.187+0000 7feb24c00700 1 -- 172.21.15.33:0/523995356 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb2c003c10 con 0x562dae4e7de0 2024-05-03T18:38:54.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.187+0000 7feb4bedab80 1 -- 172.21.15.33:0/523995356 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562dae43f480 con 0x562dae4e7de0 2024-05-03T18:38:54.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.188+0000 7feb24c00700 1 -- 172.21.15.33:0/523995356 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9518+0+0 (secure 0 0 0) 0x7feb2c003db0 con 0x562dae4e7de0 2024-05-03T18:38:54.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.189+0000 7feb24c00700 1 -- 172.21.15.33:0/523995356 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7feb2c0074a0 con 0x562dae4e7de0 2024-05-03T18:38:54.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.189+0000 7feb4bedab80 10 client.16482 fetch_fsmap learned FSMap version 21 2024-05-03T18:38:54.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.189+0000 7feb4bedab80 1 -- 172.21.15.33:0/523995356 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x562dae585e10 con 0x562dae4e7de0 2024-05-03T18:38:54.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.191+0000 7feb24c00700 1 -- 172.21.15.33:0/523995356 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x562dae43f480 con 0x562dae4e7de0 2024-05-03T18:38:54.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.191+0000 7feb24c00700 1 -- 172.21.15.33:0/523995356 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7feb2c011070 con 0x562dae4e7de0 2024-05-03T18:38:54.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.191+0000 7feb4bedab80 10 client.16482 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:38:54.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.191+0000 7feb4bedab80 10 client.16482 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:38:54.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.191+0000 7feb4bedab80 20 client.16482 populate_metadata read hostname 'smithi033' 2024-05-03T18:38:54.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.192+0000 7feb4bedab80 1 --2- 172.21.15.33:0/523995356 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x562dae586150 0x562dae588540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:54.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.192+0000 7feb4bedab80 4 client.16482 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-03T18:38:54.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.192+0000 7feb4bedab80 1 -- 172.21.15.33:0/523995356 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562dae439520 con 0x562dae586150 2024-05-03T18:38:54.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.193+0000 7feb3a000700 1 --2- 172.21.15.33:0/523995356 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x562dae586150 0x562dae588540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:54.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.196+0000 7feb3a000700 1 --2- 172.21.15.33:0/523995356 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x562dae586150 0x562dae588540 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12390ccb760a29ab server_cookie=fdd19baa21431547 in_seq=0 out_seq=0 2024-05-03T18:38:54.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.196+0000 7feb24c00700 10 client.16482 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:38:54.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.199+0000 7feb24c00700 1 -- 172.21.15.33:0/523995356 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562dae439520 con 0x562dae586150 2024-05-03T18:38:54.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.199+0000 7feb24c00700 10 client.16482 handle_command_reply: tid=0 2024-05-03T18:38:54.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.201+0000 7feb4bedab80 1 client.16482 shutdown 2024-05-03T18:38:54.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.201+0000 7feb4bedab80 1 -- 172.21.15.33:0/523995356 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562dae4e7de0 msgr2=0x562dae4e5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:54.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.201+0000 7feb4bedab80 1 --2- 172.21.15.33:0/523995356 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562dae4e7de0 0x562dae4e5ab0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7feb2c000f80 tx=0x7feb2c00b740 comp rx=0 tx=0).stop 2024-05-03T18:38:54.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.201+0000 7feb4bedab80 1 -- 172.21.15.33:0/523995356 shutdown_connections 2024-05-03T18:38:54.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.201+0000 7feb4bedab80 1 --2- 172.21.15.33:0/523995356 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x562dae586150 0x562dae588540 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:54.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.201+0000 7feb4bedab80 1 --2- 172.21.15.33:0/523995356 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562dae4edac0 0x562dae4e1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:54.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.201+0000 7feb4bedab80 1 --2- 172.21.15.33:0/523995356 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562dae4e7de0 0x562dae4e5ab0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:54.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.201+0000 7feb4bedab80 1 --2- 172.21.15.33:0/523995356 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562dae4ed1a0 0x562dae4e5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:54.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.201+0000 7feb4bedab80 1 -- 172.21.15.33:0/523995356 >> 172.21.15.33:0/523995356 conn(0x562dae457750 msgr2=0x562dae4d4430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:38:54.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.202+0000 7feb24c00700 0 client.16482 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:38:54.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.202+0000 7feb4bedab80 1 -- 172.21.15.33:0/523995356 shutdown_connections 2024-05-03T18:38:54.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.202+0000 7feb4bedab80 1 -- 172.21.15.33:0/523995356 wait complete. 2024-05-03T18:38:54.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.203+0000 7feb4bedab80 20 client.16482 trim_cache size 0 max 16384 2024-05-03T18:38:54.204 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-03T18:38:54.204 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-05-03T18:38:54.205 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "f6c096c8-c202-44eb-ae61-9846d97661a6", 2024-05-03T18:38:54.205 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-05-03T18:38:54.205 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-03T18:38:54.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.204+0000 7feb41600700 1 -- 172.21.15.33:0/127035974 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7feb200b2a70 msgr2=0x7feb200b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:54.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.204+0000 7feb41600700 1 --2- 172.21.15.33:0/127035974 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7feb200b2a70 0x7feb200b4f30 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7feb340245b0 tx=0x7feb340cf050 comp rx=0 tx=0).stop 2024-05-03T18:38:54.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.204+0000 7feb41600700 1 -- 172.21.15.33:0/127035974 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb3c103550 msgr2=0x7feb3c19f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:54.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.204+0000 7feb41600700 1 --2- 172.21.15.33:0/127035974 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb3c103550 0x7feb3c19f6a0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7feb3400e970 tx=0x7feb3400ec80 comp rx=0 tx=0).stop 2024-05-03T18:38:54.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.205+0000 7feb41600700 1 -- 172.21.15.33:0/127035974 shutdown_connections 2024-05-03T18:38:54.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.205+0000 7feb41600700 1 --2- 172.21.15.33:0/127035974 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7feb200b2a70 0x7feb200b4f30 secure :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7feb340245b0 tx=0x7feb340cf050 comp rx=0 tx=0).stop 2024-05-03T18:38:54.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.205+0000 7feb41600700 1 --2- 172.21.15.33:0/127035974 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb3c108460 0x7feb3c199860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:54.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.205+0000 7feb41600700 1 --2- 172.21.15.33:0/127035974 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb3c103550 0x7feb3c19f6a0 secure :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7feb3400e970 tx=0x7feb3400ec80 comp rx=0 tx=0).stop 2024-05-03T18:38:54.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.205+0000 7feb41600700 1 --2- 172.21.15.33:0/127035974 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb3c103ef0 0x7feb3c19fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:54.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.205+0000 7feb41600700 1 -- 172.21.15.33:0/127035974 >> 172.21.15.33:0/127035974 conn(0x7feb3c07a600 msgr2=0x7feb3c1016a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:38:54.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.205+0000 7feb41600700 1 -- 172.21.15.33:0/127035974 shutdown_connections 2024-05-03T18:38:54.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:54.206+0000 7feb41600700 1 -- 172.21.15.33:0/127035974 wait complete. 2024-05-03T18:38:54.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:38:54 smithi099 ceph-mon[31658]: pgmap v364: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 12 MiB/s wr, 773 op/s 2024-05-03T18:38:54.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:38:54 smithi150 ceph-mon[32492]: pgmap v364: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 12 MiB/s wr, 773 op/s 2024-05-03T18:38:55.111 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 460.0 s, 618.3 tps, lat 161.691 ms stddev 85.428 2024-05-03T18:38:55.112 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-03T18:38:55.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:38:55 smithi033 ceph-mon[28330]: scrub queued for path: / 2024-05-03T18:38:55.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:38:55 smithi033 ceph-mon[28330]: scrub summary: idle+waiting paths [/] 2024-05-03T18:38:55.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:38:55 smithi033 ceph-mon[28330]: scrub summary: active paths [/] 2024-05-03T18:38:55.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:38:55 smithi099 ceph-mon[31658]: scrub queued for path: / 2024-05-03T18:38:55.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:38:55 smithi099 ceph-mon[31658]: scrub summary: idle+waiting paths [/] 2024-05-03T18:38:55.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:38:55 smithi099 ceph-mon[31658]: scrub summary: active paths [/] 2024-05-03T18:38:55.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:38:55 smithi150 ceph-mon[32492]: scrub queued for path: / 2024-05-03T18:38:55.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:38:55 smithi150 ceph-mon[32492]: scrub summary: idle+waiting paths [/] 2024-05-03T18:38:55.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:38:55 smithi150 ceph-mon[32492]: scrub summary: active paths [/] 2024-05-03T18:38:55.803 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:38:56.212 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:38:56 smithi033 ceph-mon[28330]: scrub summary: idle+waiting paths [/] 2024-05-03T18:38:56.212 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:38:56 smithi033 ceph-mon[28330]: pgmap v365: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 7.8 MiB/s wr, 525 op/s 2024-05-03T18:38:56.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:38:56 smithi099 ceph-mon[31658]: scrub summary: idle+waiting paths [/] 2024-05-03T18:38:56.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:38:56 smithi099 ceph-mon[31658]: pgmap v365: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 7.8 MiB/s wr, 525 op/s 2024-05-03T18:38:56.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:38:56 smithi150 ceph-mon[32492]: scrub summary: idle+waiting paths [/] 2024-05-03T18:38:56.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:38:56 smithi150 ceph-mon[32492]: pgmap v365: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 7.8 MiB/s wr, 525 op/s 2024-05-03T18:38:57.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:38:57 smithi033 ceph-mon[28330]: scrub summary: idle 2024-05-03T18:38:57.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:38:57 smithi099 ceph-mon[31658]: scrub summary: idle 2024-05-03T18:38:57.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:38:57 smithi150 ceph-mon[32492]: scrub summary: idle 2024-05-03T18:38:58.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.307+0000 7f5612400700 1 -- 172.21.15.33:0/681287014 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f560c06b180 msgr2=0x7f560c06b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:58.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.307+0000 7f5612400700 1 --2- 172.21.15.33:0/681287014 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f560c06b180 0x7f560c06b600 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f55f8009990 tx=0x7f55f8009ca0 comp rx=0 tx=0).stop 2024-05-03T18:38:58.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.310+0000 7f5612400700 1 -- 172.21.15.33:0/681287014 shutdown_connections 2024-05-03T18:38:58.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.310+0000 7f5612400700 1 --2- 172.21.15.33:0/681287014 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f560c06bb40 0x7f560c110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:58.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.310+0000 7f5612400700 1 --2- 172.21.15.33:0/681287014 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f560c109510 0x7f560c1098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:58.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.310+0000 7f5612400700 1 --2- 172.21.15.33:0/681287014 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f560c06b180 0x7f560c06b600 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:58.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.310+0000 7f5612400700 1 -- 172.21.15.33:0/681287014 >> 172.21.15.33:0/681287014 conn(0x7f560c079350 msgr2=0x7f560c079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:38:58.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.311+0000 7f5612400700 1 -- 172.21.15.33:0/681287014 shutdown_connections 2024-05-03T18:38:58.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.311+0000 7f5612400700 1 -- 172.21.15.33:0/681287014 wait complete. 2024-05-03T18:38:58.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.312+0000 7f5612400700 1 Processor -- start 2024-05-03T18:38:58.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.313+0000 7f5612400700 1 -- start start 2024-05-03T18:38:58.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.313+0000 7f5612400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f560c06b180 0x7f560c10a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:58.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.313+0000 7f5612400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f560c06bb40 0x7f560c10a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:58.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.313+0000 7f5612400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f560c109510 0x7f560c10ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:58.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.313+0000 7f5612400700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f560c113570 con 0x7f560c06bb40 2024-05-03T18:38:58.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.313+0000 7f5612400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f560c1133f0 con 0x7f560c06b180 2024-05-03T18:38:58.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.313+0000 7f5612400700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f560c1136f0 con 0x7f560c109510 2024-05-03T18:38:58.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.314+0000 7f560b400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f560c06b180 0x7f560c10a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:58.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.314+0000 7f560b400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f560c06b180 0x7f560c10a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:45672/0 (socket says 172.21.15.33:45672) 2024-05-03T18:38:58.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.314+0000 7f560b400700 1 -- 172.21.15.33:0/2723076666 learned_addr learned my addr 172.21.15.33:0/2723076666 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:38:58.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.315+0000 7f560b400700 1 -- 172.21.15.33:0/2723076666 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f560c109510 msgr2=0x7f560c10ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:58.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.315+0000 7f560b400700 1 --2- 172.21.15.33:0/2723076666 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f560c109510 0x7f560c10ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:58.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.315+0000 7f560b400700 1 -- 172.21.15.33:0/2723076666 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f560c06bb40 msgr2=0x7f560c10a860 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-03T18:38:58.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.315+0000 7f560b400700 1 --2- 172.21.15.33:0/2723076666 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f560c06bb40 0x7f560c10a860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:58.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.315+0000 7f560b400700 1 -- 172.21.15.33:0/2723076666 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f560c1ac410 con 0x7f560c06b180 2024-05-03T18:38:58.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.316+0000 7f560b400700 1 --2- 172.21.15.33:0/2723076666 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f560c06b180 0x7f560c10a320 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f55fc00e9d0 tx=0x7f55fc00ed90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:38:58.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.316+0000 7f5603e00700 1 -- 172.21.15.33:0/2723076666 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55fc00cbd0 con 0x7f560c06b180 2024-05-03T18:38:58.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.317+0000 7f5603e00700 1 -- 172.21.15.33:0/2723076666 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f55fc00cd70 con 0x7f560c06b180 2024-05-03T18:38:58.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.317+0000 7f5603e00700 1 -- 172.21.15.33:0/2723076666 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55fc018680 con 0x7f560c06b180 2024-05-03T18:38:58.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.317+0000 7f5612400700 1 -- 172.21.15.33:0/2723076666 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f560c1ac700 con 0x7f560c06b180 2024-05-03T18:38:58.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.317+0000 7f5612400700 1 -- 172.21.15.33:0/2723076666 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f560c1acc90 con 0x7f560c06b180 2024-05-03T18:38:58.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.327+0000 7f561cd12b80 1 Processor -- start 2024-05-03T18:38:58.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.327+0000 7f561cd12b80 1 -- start start 2024-05-03T18:38:58.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.328+0000 7f561cd12b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5561d6b97930 0x5561d6b97d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:58.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.328+0000 7f561cd12b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561d6b98250 0x5561d6b91f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:58.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.328+0000 7f561cd12b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5561d6b92550 0x5561d6b929d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:58.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.328+0000 7f561cd12b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5561d6ae4a20 con 0x5561d6b98250 2024-05-03T18:38:58.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.328+0000 7f561cd12b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5561d6a82760 con 0x5561d6b97930 2024-05-03T18:38:58.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.328+0000 7f561cd12b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5561d6ae09e0 con 0x5561d6b92550 2024-05-03T18:38:58.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.328+0000 7f5603e00700 1 -- 172.21.15.33:0/2723076666 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f55fc010430 con 0x7f560c06b180 2024-05-03T18:38:58.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.328+0000 7f560b400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561d6b98250 0x5561d6b91f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:58.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.329+0000 7f560b400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561d6b98250 0x5561d6b91f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39090/0 (socket says 172.21.15.33:39090) 2024-05-03T18:38:58.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.329+0000 7f560b400700 1 -- 172.21.15.33:0/4292696473 learned_addr learned my addr 172.21.15.33:0/4292696473 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:38:58.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.329+0000 7f560be00700 1 --2- 172.21.15.33:0/4292696473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5561d6b97930 0x5561d6b97d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:58.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.330+0000 7f560b400700 1 -- 172.21.15.33:0/4292696473 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5561d6b92550 msgr2=0x5561d6b929d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-03T18:38:58.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.330+0000 7f560b400700 1 --2- 172.21.15.33:0/4292696473 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5561d6b92550 0x5561d6b929d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:58.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.330+0000 7f560b400700 1 -- 172.21.15.33:0/4292696473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5561d6b97930 msgr2=0x5561d6b97d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:38:58.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.330+0000 7f560b400700 1 --2- 172.21.15.33:0/4292696473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5561d6b97930 0x5561d6b97d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:58.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.330+0000 7f560b400700 1 -- 172.21.15.33:0/4292696473 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561d6a318d0 con 0x5561d6b98250 2024-05-03T18:38:58.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.330+0000 7f560b400700 1 --2- 172.21.15.33:0/4292696473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561d6b98250 0x5561d6b91f30 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f55fc0dfe10 tx=0x7f55fc0e6f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=230216b2b0388e4e server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:38:58.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.331+0000 7f5603e00700 1 --2- 172.21.15.33:0/2723076666 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f55f00b2b20 0x7f55f00b4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:58.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.331+0000 7f5603e00700 1 -- 172.21.15.33:0/2723076666 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9518+0+0 (secure 0 0 0) 0x7f55fc0e2bc0 con 0x7f560c06b180 2024-05-03T18:38:58.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.331+0000 7f5600c00700 1 -- 172.21.15.33:0/4292696473 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55fc0f2020 con 0x5561d6b98250 2024-05-03T18:38:58.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.331+0000 7f5600c00700 1 -- 172.21.15.33:0/4292696473 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f55fc0e7040 con 0x5561d6b98250 2024-05-03T18:38:58.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.331+0000 7f5600c00700 1 -- 172.21.15.33:0/4292696473 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55fc0f2020 con 0x5561d6b98250 2024-05-03T18:38:58.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.332+0000 7f560be00700 1 --2- 172.21.15.33:0/2723076666 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f55f00b2b20 0x7f55f00b4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:58.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.333+0000 7f561cd12b80 1 -- 172.21.15.33:0/4292696473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561d6b98250 msgr2=0x5561d6b91f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:58.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.333+0000 7f561cd12b80 1 --2- 172.21.15.33:0/4292696473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561d6b98250 0x5561d6b91f30 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f55fc0dfe10 tx=0x7f55fc0e6f90 comp rx=0 tx=0).stop 2024-05-03T18:38:58.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.333+0000 7f561cd12b80 1 -- 172.21.15.33:0/4292696473 shutdown_connections 2024-05-03T18:38:58.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.333+0000 7f561cd12b80 1 --2- 172.21.15.33:0/4292696473 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5561d6b92550 0x5561d6b929d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:58.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.333+0000 7f561cd12b80 1 --2- 172.21.15.33:0/4292696473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5561d6b97930 0x5561d6b97d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:58.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.333+0000 7f561cd12b80 1 --2- 172.21.15.33:0/4292696473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561d6b98250 0x5561d6b91f30 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:58.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.333+0000 7f561cd12b80 1 -- 172.21.15.33:0/4292696473 >> 172.21.15.33:0/4292696473 conn(0x5561d6b0a6d0 msgr2=0x5561d6b8c900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:38:58.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.334+0000 7f561cd12b80 1 -- 172.21.15.33:0/4292696473 shutdown_connections 2024-05-03T18:38:58.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.334+0000 7f561cd12b80 1 -- 172.21.15.33:0/4292696473 wait complete. 2024-05-03T18:38:58.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.334+0000 7f560be00700 1 --2- 172.21.15.33:0/2723076666 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f55f00b2b20 0x7f55f00b4fe0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f56040093a0 tx=0x7f560400b040 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:38:58.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.334+0000 7f561cd12b80 1 -- start start 2024-05-03T18:38:58.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.335+0000 7f561cd12b80 1 Processor -- start 2024-05-03T18:38:58.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.335+0000 7f561cd12b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5561d6b92550 0x5561d6b9de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:58.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.336+0000 7f561cd12b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561d6b97930 0x5561d6b9b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:58.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.336+0000 7f561cd12b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5561d6b98250 0x5561d6b9b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:58.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.336+0000 7f561cd12b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5561d6bafd10 con 0x5561d6b97930 2024-05-03T18:38:58.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.336+0000 7f561cd12b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5561d6a4ab20 con 0x5561d6b98250 2024-05-03T18:38:58.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.336+0000 7f561cd12b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5561d6bafe90 con 0x5561d6b92550 2024-05-03T18:38:58.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.336+0000 7f561cd12b80 10 client.0 _ll_register_callbacks cb 0x5561d6a8bc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:38:58.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.336+0000 7f560b400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5561d6b92550 0x5561d6b9de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:58.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.336+0000 7f560b400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5561d6b92550 0x5561d6b9de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:54934/0 (socket says 172.21.15.33:54934) 2024-05-03T18:38:58.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.336+0000 7f560b400700 1 -- 172.21.15.33:0/3947297246 learned_addr learned my addr 172.21.15.33:0/3947297246 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:38:58.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.336+0000 7f560be00700 1 --2- 172.21.15.33:0/3947297246 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5561d6b98250 0x5561d6b9b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:58.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.337+0000 7f560b400700 1 -- 172.21.15.33:0/3947297246 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5561d6b98250 msgr2=0x5561d6b9b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:38:58.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.337+0000 7f560b400700 1 --2- 172.21.15.33:0/3947297246 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5561d6b98250 0x5561d6b9b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:58.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.337+0000 7f560b400700 1 -- 172.21.15.33:0/3947297246 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561d6b97930 msgr2=0x5561d6b9b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:38:58.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.337+0000 7f560b400700 1 --2- 172.21.15.33:0/3947297246 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561d6b97930 0x5561d6b9b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:58.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.337+0000 7f560b400700 1 -- 172.21.15.33:0/3947297246 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561d6aaa420 con 0x5561d6b92550 2024-05-03T18:38:58.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.337+0000 7f560b400700 1 --2- 172.21.15.33:0/3947297246 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5561d6b92550 0x5561d6b9de90 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f55fc000c00 tx=0x7f55fc01f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92d5ec1de9abfcd5 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:38:58.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.338+0000 7f55f5600700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-03T18:38:58.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.343+0000 7f55f5600700 1 -- 172.21.15.33:0/3947297246 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55fc0f2070 con 0x5561d6b92550 2024-05-03T18:38:58.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.343+0000 7f55f5600700 1 -- 172.21.15.33:0/3947297246 <== mon.1 v2:172.21.15.150:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9518+0+0 (secure 0 0 0) 0x7f55fc0b2430 con 0x5561d6b92550 2024-05-03T18:38:58.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.343+0000 7f561cd12b80 1 -- 172.21.15.33:0/3947297246 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55fc0e7040 con 0x5561d6b92550 2024-05-03T18:38:58.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.344+0000 7f55f5600700 1 -- 172.21.15.33:0/3947297246 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f55fc0a93b0 con 0x5561d6b92550 2024-05-03T18:38:58.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.344+0000 7f561cd12b80 10 client.26275 fetch_fsmap learned FSMap version 21 2024-05-03T18:38:58.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.344+0000 7f561cd12b80 1 -- 172.21.15.33:0/3947297246 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5561d6bafd10 con 0x5561d6b92550 2024-05-03T18:38:58.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.346+0000 7f55f5600700 1 -- 172.21.15.33:0/3947297246 <== mon.1 v2:172.21.15.150:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f55fc0e7040 con 0x5561d6b92550 2024-05-03T18:38:58.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.347+0000 7f55f5600700 1 -- 172.21.15.33:0/3947297246 <== mon.1 v2:172.21.15.150:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f55fc0ebbe0 con 0x5561d6b92550 2024-05-03T18:38:58.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.348+0000 7f561cd12b80 10 client.26275 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:38:58.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.348+0000 7f561cd12b80 10 client.26275 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:38:58.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.348+0000 7f561cd12b80 20 client.26275 populate_metadata read hostname 'smithi033' 2024-05-03T18:38:58.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.348+0000 7f561cd12b80 1 --2- 172.21.15.33:0/3947297246 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5561d6bbc560 0x5561d6bbe950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:58.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.348+0000 7f561cd12b80 4 client.26275 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:38:58.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.348+0000 7f561cd12b80 1 -- 172.21.15.33:0/3947297246 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5561d6b5eff0 con 0x5561d6bbc560 2024-05-03T18:38:58.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.352+0000 7f560aa00700 1 --2- 172.21.15.33:0/3947297246 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5561d6bbc560 0x5561d6bbe950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:58.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.355+0000 7f560aa00700 1 --2- 172.21.15.33:0/3947297246 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5561d6bbc560 0x5561d6bbe950 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=362d3fd1d0bda393 server_cookie=8ea5d033157aa59b in_seq=0 out_seq=0 2024-05-03T18:38:58.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.355+0000 7f55f5600700 10 client.26275 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:38:58.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.359+0000 7f55f5600700 1 -- 172.21.15.33:0/3947297246 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5561d6b5eff0 con 0x5561d6bbc560 2024-05-03T18:38:58.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.359+0000 7f55f5600700 10 client.26275 handle_command_reply: tid=0 2024-05-03T18:38:58.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.360+0000 7f561cd12b80 1 client.26275 shutdown 2024-05-03T18:38:58.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.362+0000 7f561cd12b80 1 -- 172.21.15.33:0/3947297246 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5561d6b92550 msgr2=0x5561d6b9de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:58.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.362+0000 7f561cd12b80 1 --2- 172.21.15.33:0/3947297246 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5561d6b92550 0x5561d6b9de90 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f55fc000c00 tx=0x7f55fc01f000 comp rx=0 tx=0).stop 2024-05-03T18:38:58.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.362+0000 7f561cd12b80 1 -- 172.21.15.33:0/3947297246 shutdown_connections 2024-05-03T18:38:58.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.362+0000 7f561cd12b80 1 --2- 172.21.15.33:0/3947297246 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5561d6bbc560 0x5561d6bbe950 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:58.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.362+0000 7f561cd12b80 1 --2- 172.21.15.33:0/3947297246 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5561d6b98250 0x5561d6b9b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:58.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.362+0000 7f561cd12b80 1 --2- 172.21.15.33:0/3947297246 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5561d6b92550 0x5561d6b9de90 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:58.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.362+0000 7f561cd12b80 1 --2- 172.21.15.33:0/3947297246 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561d6b97930 0x5561d6b9b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:58.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.362+0000 7f561cd12b80 1 -- 172.21.15.33:0/3947297246 >> 172.21.15.33:0/3947297246 conn(0x5561d6b0a6d0 msgr2=0x5561d6b873b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:38:58.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.363+0000 7f55f5600700 0 client.26275 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:38:58.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.363+0000 7f561cd12b80 1 -- 172.21.15.33:0/3947297246 shutdown_connections 2024-05-03T18:38:58.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.363+0000 7f561cd12b80 1 -- 172.21.15.33:0/3947297246 wait complete. 2024-05-03T18:38:58.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.364+0000 7f561cd12b80 20 client.26275 trim_cache size 0 max 16384 2024-05-03T18:38:58.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.401+0000 7f561cd12b80 1 Processor -- start 2024-05-03T18:38:58.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.401+0000 7f561cd12b80 1 -- start start 2024-05-03T18:38:58.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.401+0000 7f561cd12b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5561d6b0a6d0 0x5561d6b91f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:58.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.401+0000 7f561cd12b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5561d6b92550 0x5561d6b98e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:58.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.401+0000 7f561cd12b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561d6b97930 0x5561d6bc3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:58.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.402+0000 7f561cd12b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f560c1a2980 con 0x5561d6b97930 2024-05-03T18:38:58.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.402+0000 7f561cd12b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f560c1adee0 con 0x5561d6b92550 2024-05-03T18:38:58.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.402+0000 7f561cd12b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f560c1a2fd0 con 0x5561d6b0a6d0 2024-05-03T18:38:58.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.402+0000 7f560be00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561d6b97930 0x5561d6bc3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:58.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.402+0000 7f560be00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561d6b97930 0x5561d6bc3fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39112/0 (socket says 172.21.15.33:39112) 2024-05-03T18:38:58.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.402+0000 7f560be00700 1 -- 172.21.15.33:0/62958861 learned_addr learned my addr 172.21.15.33:0/62958861 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:38:58.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.403+0000 7f560b400700 1 --2- 172.21.15.33:0/62958861 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5561d6b0a6d0 0x5561d6b91f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:58.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.403+0000 7f560be00700 1 -- 172.21.15.33:0/62958861 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5561d6b0a6d0 msgr2=0x5561d6b91f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:38:58.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.403+0000 7f560be00700 1 --2- 172.21.15.33:0/62958861 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5561d6b0a6d0 0x5561d6b91f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:58.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.403+0000 7f560be00700 1 -- 172.21.15.33:0/62958861 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5561d6b92550 msgr2=0x5561d6b98e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:38:58.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.403+0000 7f560be00700 1 --2- 172.21.15.33:0/62958861 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5561d6b92550 0x5561d6b98e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:58.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.403+0000 7f560be00700 1 -- 172.21.15.33:0/62958861 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561d6b9bfd0 con 0x5561d6b97930 2024-05-03T18:38:58.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.404+0000 7f560be00700 1 --2- 172.21.15.33:0/62958861 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561d6b97930 0x5561d6bc3fb0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f56040155d0 tx=0x7f5604013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b025d842489d4c99 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:38:58.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.404+0000 7f55f7e00700 1 -- 172.21.15.33:0/62958861 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5604026070 con 0x5561d6b97930 2024-05-03T18:38:58.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.405+0000 7f55f7e00700 1 -- 172.21.15.33:0/62958861 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f560401d040 con 0x5561d6b97930 2024-05-03T18:38:58.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.405+0000 7f561cd12b80 1 -- 172.21.15.33:0/62958861 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561d6b97930 msgr2=0x5561d6bc3fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:58.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.405+0000 7f561cd12b80 1 --2- 172.21.15.33:0/62958861 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561d6b97930 0x5561d6bc3fb0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f56040155d0 tx=0x7f5604013000 comp rx=0 tx=0).stop 2024-05-03T18:38:58.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.406+0000 7f561cd12b80 1 -- 172.21.15.33:0/62958861 shutdown_connections 2024-05-03T18:38:58.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.406+0000 7f561cd12b80 1 --2- 172.21.15.33:0/62958861 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561d6b97930 0x5561d6bc3fb0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:58.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.406+0000 7f561cd12b80 1 --2- 172.21.15.33:0/62958861 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5561d6b0a6d0 0x5561d6b91f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:58.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.406+0000 7f561cd12b80 1 --2- 172.21.15.33:0/62958861 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5561d6b92550 0x5561d6b98e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:58.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.406+0000 7f561cd12b80 1 -- 172.21.15.33:0/62958861 >> 172.21.15.33:0/62958861 conn(0x5561d6b8e800 msgr2=0x5561d6b986f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:38:58.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.406+0000 7f561cd12b80 1 -- 172.21.15.33:0/62958861 shutdown_connections 2024-05-03T18:38:58.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.406+0000 7f561cd12b80 1 -- 172.21.15.33:0/62958861 wait complete. 2024-05-03T18:38:58.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.407+0000 7f561cd12b80 1 -- start start 2024-05-03T18:38:58.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.407+0000 7f561cd12b80 1 Processor -- start 2024-05-03T18:38:58.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.408+0000 7f561cd12b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5561d6b0a6d0 0x5561d6bb4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:58.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.408+0000 7f561cd12b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5561d6b92550 0x5561d6bb4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:58.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.408+0000 7f561cd12b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561d6b97930 0x5561d6bb4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:58.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.408+0000 7f561cd12b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f560c1a36a0 con 0x5561d6b97930 2024-05-03T18:38:58.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.408+0000 7f561cd12b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f560c1ad810 con 0x5561d6b92550 2024-05-03T18:38:58.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.408+0000 7f561cd12b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f560c1a3d70 con 0x5561d6b0a6d0 2024-05-03T18:38:58.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.408+0000 7f561cd12b80 10 client.0 _ll_register_callbacks cb 0x5561d6a8bc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:38:58.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.409+0000 7f560b400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5561d6b0a6d0 0x5561d6bb4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:58.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.409+0000 7f560b400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5561d6b0a6d0 0x5561d6bb4120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:54948/0 (socket says 172.21.15.33:54948) 2024-05-03T18:38:58.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.409+0000 7f560be00700 1 --2- 172.21.15.33:0/4069232599 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561d6b97930 0x5561d6bb4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:58.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.409+0000 7f560aa00700 1 --2- 172.21.15.33:0/4069232599 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5561d6b92550 0x5561d6bb4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:58.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.409+0000 7f560b400700 1 -- 172.21.15.33:0/4069232599 learned_addr learned my addr 172.21.15.33:0/4069232599 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:38:58.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.410+0000 7f560be00700 1 -- 172.21.15.33:0/4069232599 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5561d6b0a6d0 msgr2=0x5561d6bb4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:38:58.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.410+0000 7f560be00700 1 --2- 172.21.15.33:0/4069232599 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5561d6b0a6d0 0x5561d6bb4120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:58.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.410+0000 7f560be00700 1 -- 172.21.15.33:0/4069232599 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5561d6b92550 msgr2=0x5561d6bb4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:38:58.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.410+0000 7f560be00700 1 --2- 172.21.15.33:0/4069232599 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5561d6b92550 0x5561d6bb4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:58.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.410+0000 7f560be00700 1 -- 172.21.15.33:0/4069232599 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561d6bb5160 con 0x5561d6b97930 2024-05-03T18:38:58.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.411+0000 7f560be00700 1 --2- 172.21.15.33:0/4069232599 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561d6b97930 0x5561d6bb4ba0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f5604015fd0 tx=0x7f5604017c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37cc2f371d2f876e server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:38:58.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.411+0000 7f55f5600700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:38:58.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.412+0000 7f55f5600700 1 -- 172.21.15.33:0/4069232599 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5604026040 con 0x5561d6b97930 2024-05-03T18:38:58.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.412+0000 7f55f5600700 1 -- 172.21.15.33:0/4069232599 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9518+0+0 (secure 0 0 0) 0x7f5604022600 con 0x5561d6b97930 2024-05-03T18:38:58.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.412+0000 7f561cd12b80 1 -- 172.21.15.33:0/4069232599 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f560401d040 con 0x5561d6b97930 2024-05-03T18:38:58.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.413+0000 7f55f5600700 1 -- 172.21.15.33:0/4069232599 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f5604017e60 con 0x5561d6b97930 2024-05-03T18:38:58.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.413+0000 7f561cd12b80 10 client.16494 fetch_fsmap learned FSMap version 21 2024-05-03T18:38:58.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.413+0000 7f561cd12b80 1 -- 172.21.15.33:0/4069232599 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5561d6c38240 con 0x5561d6b97930 2024-05-03T18:38:58.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.414+0000 7f55f5600700 1 -- 172.21.15.33:0/4069232599 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f560401d040 con 0x5561d6b97930 2024-05-03T18:38:58.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.414+0000 7f55f5600700 1 -- 172.21.15.33:0/4069232599 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f560402b7c0 con 0x5561d6b97930 2024-05-03T18:38:58.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.414+0000 7f561cd12b80 10 client.16494 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:38:58.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.414+0000 7f561cd12b80 10 client.16494 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:38:58.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.414+0000 7f561cd12b80 20 client.16494 populate_metadata read hostname 'smithi033' 2024-05-03T18:38:58.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.414+0000 7f561cd12b80 1 --2- 172.21.15.33:0/4069232599 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5561d6b98250 0x5561d6c3a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:58.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.414+0000 7f561cd12b80 4 client.16494 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-03T18:38:58.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.414+0000 7f561cd12b80 1 -- 172.21.15.33:0/4069232599 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5561d6af2400 con 0x5561d6b98250 2024-05-03T18:38:58.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.415+0000 7f560b400700 1 --2- 172.21.15.33:0/4069232599 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5561d6b98250 0x5561d6c3a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:58.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.417+0000 7f560b400700 1 --2- 172.21.15.33:0/4069232599 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5561d6b98250 0x5561d6c3a880 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b524a9cf1f80c9d5 server_cookie=8530aa0867159279 in_seq=0 out_seq=0 2024-05-03T18:38:58.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.417+0000 7f55f5600700 10 client.16494 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:38:58.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.418+0000 7f55f5600700 1 -- 172.21.15.33:0/4069232599 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5561d6af2400 con 0x5561d6b98250 2024-05-03T18:38:58.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.418+0000 7f55f5600700 10 client.16494 handle_command_reply: tid=0 2024-05-03T18:38:58.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.418+0000 7f561cd12b80 1 client.16494 shutdown 2024-05-03T18:38:58.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.418+0000 7f561cd12b80 1 -- 172.21.15.33:0/4069232599 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561d6b97930 msgr2=0x5561d6bb4ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:58.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.418+0000 7f561cd12b80 1 --2- 172.21.15.33:0/4069232599 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561d6b97930 0x5561d6bb4ba0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f5604015fd0 tx=0x7f5604017c70 comp rx=0 tx=0).stop 2024-05-03T18:38:58.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.419+0000 7f561cd12b80 1 -- 172.21.15.33:0/4069232599 shutdown_connections 2024-05-03T18:38:58.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.419+0000 7f561cd12b80 1 --2- 172.21.15.33:0/4069232599 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5561d6b98250 0x5561d6c3a880 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:58.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.419+0000 7f561cd12b80 1 --2- 172.21.15.33:0/4069232599 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561d6b97930 0x5561d6bb4ba0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:58.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.419+0000 7f561cd12b80 1 --2- 172.21.15.33:0/4069232599 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5561d6b0a6d0 0x5561d6bb4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:58.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.419+0000 7f561cd12b80 1 --2- 172.21.15.33:0/4069232599 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5561d6b92550 0x5561d6bb4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:58.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.419+0000 7f561cd12b80 1 -- 172.21.15.33:0/4069232599 >> 172.21.15.33:0/4069232599 conn(0x5561d6b8e800 msgr2=0x5561d6b86b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:38:58.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.419+0000 7f55f5600700 0 client.16494 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:38:58.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.420+0000 7f561cd12b80 1 -- 172.21.15.33:0/4069232599 shutdown_connections 2024-05-03T18:38:58.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.420+0000 7f561cd12b80 1 -- 172.21.15.33:0/4069232599 wait complete. 2024-05-03T18:38:58.423 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-03T18:38:58.424 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-05-03T18:38:58.424 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-05-03T18:38:58.424 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-03T18:38:58.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.421+0000 7f561cd12b80 20 client.16494 trim_cache size 0 max 16384 2024-05-03T18:38:58.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.422+0000 7f5612400700 1 -- 172.21.15.33:0/2723076666 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f55f00b2b20 msgr2=0x7f55f00b4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:58.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.422+0000 7f5612400700 1 --2- 172.21.15.33:0/2723076666 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f55f00b2b20 0x7f55f00b4fe0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f56040093a0 tx=0x7f560400b040 comp rx=0 tx=0).stop 2024-05-03T18:38:58.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.422+0000 7f5612400700 1 -- 172.21.15.33:0/2723076666 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f560c06b180 msgr2=0x7f560c10a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:58.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.422+0000 7f5612400700 1 --2- 172.21.15.33:0/2723076666 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f560c06b180 0x7f560c10a320 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f55fc00e9d0 tx=0x7f55fc00ed90 comp rx=0 tx=0).stop 2024-05-03T18:38:58.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.422+0000 7f5612400700 1 -- 172.21.15.33:0/2723076666 shutdown_connections 2024-05-03T18:38:58.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.422+0000 7f5612400700 1 --2- 172.21.15.33:0/2723076666 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f560c109510 0x7f560c10ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:58.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.422+0000 7f5612400700 1 --2- 172.21.15.33:0/2723076666 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f55f00b2b20 0x7f55f00b4fe0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:58.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.422+0000 7f5612400700 1 --2- 172.21.15.33:0/2723076666 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f560c06b180 0x7f560c10a320 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:58.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.422+0000 7f5612400700 1 --2- 172.21.15.33:0/2723076666 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f560c06bb40 0x7f560c10a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:58.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.422+0000 7f5612400700 1 -- 172.21.15.33:0/2723076666 >> 172.21.15.33:0/2723076666 conn(0x7f560c079350 msgr2=0x7f560c10e660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:38:58.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.423+0000 7f5612400700 1 -- 172.21.15.33:0/2723076666 shutdown_connections 2024-05-03T18:38:58.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:38:58.424+0000 7f5612400700 1 -- 172.21.15.33:0/2723076666 wait complete. 2024-05-03T18:38:58.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:38:58 smithi033 ceph-mon[28330]: pgmap v366: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 7.8 MiB/s wr, 525 op/s 2024-05-03T18:38:58.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:38:58 smithi099 ceph-mon[31658]: pgmap v366: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 7.8 MiB/s wr, 525 op/s 2024-05-03T18:38:58.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:38:58 smithi150 ceph-mon[32492]: pgmap v366: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 7.8 MiB/s wr, 525 op/s 2024-05-03T18:39:00.110 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 465.0 s, 605.8 tps, lat 164.913 ms stddev 80.187 2024-05-03T18:39:00.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:39:00 smithi033 ceph-mon[28330]: pgmap v367: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 11 MiB/s wr, 775 op/s 2024-05-03T18:39:00.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:39:00 smithi099 ceph-mon[31658]: pgmap v367: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 11 MiB/s wr, 775 op/s 2024-05-03T18:39:00.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:39:00 smithi150 ceph-mon[32492]: pgmap v367: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 11 MiB/s wr, 775 op/s 2024-05-03T18:39:01.850 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-03T18:39:01.851 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T18:39:02.150 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:39:02 smithi033 ceph-mon[28330]: pgmap v368: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 7.3 MiB/s wr, 511 op/s 2024-05-03T18:39:02.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:39:02 smithi099 ceph-mon[31658]: pgmap v368: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 7.3 MiB/s wr, 511 op/s 2024-05-03T18:39:02.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:39:02 smithi150 ceph-mon[32492]: pgmap v368: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 7.3 MiB/s wr, 511 op/s 2024-05-03T18:39:02.671 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:39:04.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:39:04 smithi033 ceph-mon[28330]: pgmap v369: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 11 MiB/s wr, 760 op/s 2024-05-03T18:39:04.547 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:39:04 smithi099 ceph-mon[31658]: pgmap v369: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 11 MiB/s wr, 760 op/s 2024-05-03T18:39:04.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:39:04 smithi150 ceph-mon[32492]: pgmap v369: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 11 MiB/s wr, 760 op/s 2024-05-03T18:39:05.111 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 470.0 s, 553.2 tps, lat 178.527 ms stddev 124.601 2024-05-03T18:39:07.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:39:07 smithi033 ceph-mon[28330]: pgmap v370: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 6.8 MiB/s wr, 499 op/s 2024-05-03T18:39:07.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:39:07 smithi033 ceph-mon[28330]: osdmap e101: 12 total, 12 up, 12 in 2024-05-03T18:39:07.545 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:39:07 smithi099 ceph-mon[31658]: pgmap v370: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 6.8 MiB/s wr, 499 op/s 2024-05-03T18:39:07.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:39:07 smithi099 ceph-mon[31658]: osdmap e101: 12 total, 12 up, 12 in 2024-05-03T18:39:08.221 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:39:07 smithi150 ceph-mon[32492]: pgmap v370: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 6.8 MiB/s wr, 499 op/s 2024-05-03T18:39:08.221 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:39:07 smithi150 ceph-mon[32492]: osdmap e101: 12 total, 12 up, 12 in 2024-05-03T18:39:09.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:39:09 smithi033 ceph-mon[28330]: pgmap v372: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 8.2 MiB/s wr, 599 op/s 2024-05-03T18:39:09.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:39:09 smithi099 ceph-mon[31658]: pgmap v372: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 8.2 MiB/s wr, 599 op/s 2024-05-03T18:39:09.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:39:09 smithi150 ceph-mon[32492]: pgmap v372: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 8.2 MiB/s wr, 599 op/s 2024-05-03T18:39:10.110 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 475.0 s, 567.8 tps, lat 178.172 ms stddev 98.298 2024-05-03T18:39:10.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:39:10 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:39:10.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:39:10 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:39:10.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:39:10 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:39:11.486 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:39:11 smithi033 ceph-mon[28330]: pgmap v373: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 7.5 MiB/s wr, 559 op/s 2024-05-03T18:39:11.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:39:11 smithi099 ceph-mon[31658]: pgmap v373: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 7.5 MiB/s wr, 559 op/s 2024-05-03T18:39:11.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:39:11 smithi150 ceph-mon[32492]: pgmap v373: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 7.5 MiB/s wr, 559 op/s 2024-05-03T18:39:11.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:11.663+0000 7f77ed200700 1 -- 172.21.15.33:0/2637075404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77e811cc80 msgr2=0x7f77e811f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:39:11.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:11.663+0000 7f77ed200700 1 --2- 172.21.15.33:0/2637075404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77e811cc80 0x7f77e811f070 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f77d8009990 tx=0x7f77d8009ca0 comp rx=0 tx=0).stop 2024-05-03T18:39:11.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:11.664+0000 7f77ed200700 1 -- 172.21.15.33:0/2637075404 shutdown_connections 2024-05-03T18:39:11.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:11.664+0000 7f77ed200700 1 --2- 172.21.15.33:0/2637075404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77e811cc80 0x7f77e811f070 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:11.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:11.664+0000 7f77ed200700 1 --2- 172.21.15.33:0/2637075404 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f77e8111b90 0x7f77e810a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:11.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:11.664+0000 7f77ed200700 1 --2- 172.21.15.33:0/2637075404 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77e8112000 0x7f77e810a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:11.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:11.664+0000 7f77ed200700 1 -- 172.21.15.33:0/2637075404 >> 172.21.15.33:0/2637075404 conn(0x7f77e806e480 msgr2=0x7f77e806e890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:39:11.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:11.664+0000 7f77ed200700 1 -- 172.21.15.33:0/2637075404 shutdown_connections 2024-05-03T18:39:11.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:11.664+0000 7f77ed200700 1 -- 172.21.15.33:0/2637075404 wait complete. 2024-05-03T18:39:11.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:11.666+0000 7f77ed200700 1 Processor -- start 2024-05-03T18:39:11.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:11.666+0000 7f77ed200700 1 -- start start 2024-05-03T18:39:11.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:11.669+0000 7f77ed200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77e8111b90 0x7f77e811a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:39:11.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:11.669+0000 7f77ed200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f77e8112000 0x7f77e811a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:39:11.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:11.669+0000 7f77ed200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77e811cc80 0x7f77e8113470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:39:11.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:11.669+0000 7f77e7400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77e8111b90 0x7f77e811a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:39:11.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:11.670+0000 7f77e6a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f77e8112000 0x7f77e811a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:39:11.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:11.670+0000 7f77e6a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f77e8112000 0x7f77e811a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:43586/0 (socket says 172.21.15.33:43586) 2024-05-03T18:39:11.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:11.670+0000 7f77e7400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77e8111b90 0x7f77e811a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54628/0 (socket says 172.21.15.33:54628) 2024-05-03T18:39:11.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:11.670+0000 7f77ed200700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f77e8122350 con 0x7f77e8111b90 2024-05-03T18:39:11.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:11.670+0000 7f77ed200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f77e81221d0 con 0x7f77e811cc80 2024-05-03T18:39:11.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:11.671+0000 7f77ed200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f77e81224d0 con 0x7f77e8112000 2024-05-03T18:39:11.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:11.671+0000 7f77e6a00700 1 -- 172.21.15.33:0/5970359 learned_addr learned my addr 172.21.15.33:0/5970359 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:39:11.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:11.672+0000 7f77e7e00700 1 --2- 172.21.15.33:0/5970359 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77e811cc80 0x7f77e8113470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:39:11.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:11.672+0000 7f77e6a00700 1 -- 172.21.15.33:0/5970359 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77e811cc80 msgr2=0x7f77e8113470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:39:11.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:11.672+0000 7f77e6a00700 1 --2- 172.21.15.33:0/5970359 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77e811cc80 0x7f77e8113470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:11.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:11.672+0000 7f77e6a00700 1 -- 172.21.15.33:0/5970359 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77e8111b90 msgr2=0x7f77e811a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:39:11.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:11.672+0000 7f77e6a00700 1 --2- 172.21.15.33:0/5970359 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77e8111b90 0x7f77e811a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:11.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:11.673+0000 7f77e6a00700 1 -- 172.21.15.33:0/5970359 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77e8113d70 con 0x7f77e8112000 2024-05-03T18:39:11.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:11.673+0000 7f77e7e00700 1 --2- 172.21.15.33:0/5970359 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77e811cc80 0x7f77e8113470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:39:11.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:11.673+0000 7f77e7400700 1 --2- 172.21.15.33:0/5970359 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77e8111b90 0x7f77e811a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:39:11.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:11.673+0000 7f77e6a00700 1 --2- 172.21.15.33:0/5970359 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f77e8112000 0x7f77e811a720 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f77d400d950 tx=0x7f77d400dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:39:11.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:11.674+0000 7f77dfe00700 1 -- 172.21.15.33:0/5970359 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d4016070 con 0x7f77e8112000 2024-05-03T18:39:11.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:11.675+0000 7f77dfe00700 1 -- 172.21.15.33:0/5970359 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f77d4009dd0 con 0x7f77e8112000 2024-05-03T18:39:11.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:11.675+0000 7f77ed200700 1 -- 172.21.15.33:0/5970359 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77e8114060 con 0x7f77e8112000 2024-05-03T18:39:11.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:11.675+0000 7f77ed200700 1 -- 172.21.15.33:0/5970359 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77e8118030 con 0x7f77e8112000 2024-05-03T18:39:11.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:11.676+0000 7f77dfe00700 1 -- 172.21.15.33:0/5970359 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d400f3c0 con 0x7f77e8112000 2024-05-03T18:39:11.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:11.678+0000 7f77ed200700 1 -- 172.21.15.33:0/5970359 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77c8005360 con 0x7f77e8112000 2024-05-03T18:39:11.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:11.687+0000 7f77dfe00700 1 -- 172.21.15.33:0/5970359 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f77d4019070 con 0x7f77e8112000 2024-05-03T18:39:11.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:11.689+0000 7f77dfe00700 1 --2- 172.21.15.33:0/5970359 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f77cc0b2b40 0x7f77cc0b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:39:11.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:11.690+0000 7f77e7400700 1 --2- 172.21.15.33:0/5970359 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f77cc0b2b40 0x7f77cc0b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:39:11.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:11.690+0000 7f77dfe00700 1 -- 172.21.15.33:0/5970359 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9606+0+0 (secure 0 0 0) 0x7f77d40e24c0 con 0x7f77e8112000 2024-05-03T18:39:11.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:11.692+0000 7f77e7400700 1 --2- 172.21.15.33:0/5970359 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f77cc0b2b40 0x7f77cc0b5000 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f77e000a7f0 tx=0x7f77e000d040 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:39:11.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:11.700+0000 7f77dfe00700 1 -- 172.21.15.33:0/5970359 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f77d40a9a60 con 0x7f77e8112000 2024-05-03T18:39:12.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:12.194+0000 7f77ed200700 1 -- 172.21.15.33:0/5970359 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f77c8005920 con 0x7f77e8112000 2024-05-03T18:39:12.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:12.196+0000 7f77dfe00700 1 -- 172.21.15.33:0/5970359 <== mon.1 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11801 (secure 0 0 0) 0x7f77d40a9360 con 0x7f77e8112000 2024-05-03T18:39:12.197 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:39:12.197 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1327393503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1327393503},{"type":"v1","addr":"172.21.15.33:6835","nonce":1327393503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/2461472948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":2461472948},{"type":"v1","addr":"172.21.15.33:6837","nonce":2461472948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14559,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/442269281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":442269281},{"type":"v1","addr":"172.21.15.33:6839","nonce":442269281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14571,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/645571756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":645571756},{"type":"v1","addr":"172.21.15.33:6841","nonce":645571756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/203598909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":203598909},{"type":"v1","addr":"172.21.15.150:6833","nonce":203598909}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/4029842493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":4029842493},{"type":"v1","addr":"172.21.15.150:6835","nonce":4029842493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3463664423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3463664423},{"type":"v1","addr":"172.21.15.150:6837","nonce":3463664423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24481,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6839/4051648018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":4051648018},{"type":"v1","addr":"172.21.15.150:6839","nonce":4051648018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34418,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1865105479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1865105479},{"type":"v1","addr":"172.21.15.99:6833","nonce":1865105479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34445,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/1206393914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1206393914},{"type":"v1","addr":"172.21.15.99:6835","nonce":1206393914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-03T18:22:28.837091+0000","modified":"2024-05-03T18:23:49.758398+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34508},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34472":{"gid":34472,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6837/3360970342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3360970342},{"type":"v1","addr":"172.21.15.99:6837","nonce":3360970342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34508":{"gid":34508,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/919907308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":919907308},{"type":"v1","addr":"172.21.15.99:6839","nonce":919907308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34508,"qdb_cluster":[34508]},"id":1}]} 2024-05-03T18:39:12.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:12.201+0000 7f77ed200700 1 -- 172.21.15.33:0/5970359 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f77cc0b2b40 msgr2=0x7f77cc0b5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:39:12.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:12.201+0000 7f77ed200700 1 --2- 172.21.15.33:0/5970359 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f77cc0b2b40 0x7f77cc0b5000 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f77e000a7f0 tx=0x7f77e000d040 comp rx=0 tx=0).stop 2024-05-03T18:39:12.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:12.201+0000 7f77ed200700 1 -- 172.21.15.33:0/5970359 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f77e8112000 msgr2=0x7f77e811a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:39:12.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:12.201+0000 7f77ed200700 1 --2- 172.21.15.33:0/5970359 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f77e8112000 0x7f77e811a720 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f77d400d950 tx=0x7f77d400dc60 comp rx=0 tx=0).stop 2024-05-03T18:39:12.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:12.202+0000 7f77ed200700 1 -- 172.21.15.33:0/5970359 shutdown_connections 2024-05-03T18:39:12.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:12.202+0000 7f77ed200700 1 --2- 172.21.15.33:0/5970359 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f77cc0b2b40 0x7f77cc0b5000 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:12.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:12.202+0000 7f77ed200700 1 --2- 172.21.15.33:0/5970359 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77e811cc80 0x7f77e8113470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:12.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:12.202+0000 7f77ed200700 1 --2- 172.21.15.33:0/5970359 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77e8111b90 0x7f77e811a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:12.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:12.202+0000 7f77ed200700 1 --2- 172.21.15.33:0/5970359 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f77e8112000 0x7f77e811a720 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:12.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:12.202+0000 7f77ed200700 1 -- 172.21.15.33:0/5970359 >> 172.21.15.33:0/5970359 conn(0x7f77e806e480 msgr2=0x7f77e810f7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:39:12.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:12.202+0000 7f77ed200700 1 -- 172.21.15.33:0/5970359 shutdown_connections 2024-05-03T18:39:12.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:12.203+0000 7f77ed200700 1 -- 172.21.15.33:0/5970359 wait complete. 2024-05-03T18:39:12.207 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 21 2024-05-03T18:39:12.487 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:39:12 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:39:12.487 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:39:12 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:39:12.487 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:39:12 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:39:12.487 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:39:12 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/5970359' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:39:12.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:39:12 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:39:12.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:39:12 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:39:12.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:39:12 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:39:12.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:39:12 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/5970359' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:39:12.969 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:39:12 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:39:12.970 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:39:12 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:39:12.970 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:39:12 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:39:12.970 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:39:12 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/5970359' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:39:13.418 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:39:13 smithi033 ceph-mon[28330]: pgmap v374: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 7.5 MiB/s wr, 559 op/s 2024-05-03T18:39:13.418 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:39:13 smithi033 ceph-mon[28330]: osdmap e102: 12 total, 12 up, 12 in 2024-05-03T18:39:13.588 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-03T18:39:13.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:39:13 smithi099 ceph-mon[31658]: pgmap v374: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 7.5 MiB/s wr, 559 op/s 2024-05-03T18:39:13.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:39:13 smithi099 ceph-mon[31658]: osdmap e102: 12 total, 12 up, 12 in 2024-05-03T18:39:13.890 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:39:13 smithi150 ceph-mon[32492]: pgmap v374: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 7.5 MiB/s wr, 559 op/s 2024-05-03T18:39:13.890 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:39:13 smithi150 ceph-mon[32492]: osdmap e102: 12 total, 12 up, 12 in 2024-05-03T18:39:14.340 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:39:15.111 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 480.0 s, 491.4 tps, lat 199.079 ms stddev 163.395 2024-05-03T18:39:15.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:39:15 smithi033 ceph-mon[28330]: pgmap v376: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 9.4 MiB/s wr, 685 op/s 2024-05-03T18:39:15.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:39:15 smithi099 ceph-mon[31658]: pgmap v376: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 9.4 MiB/s wr, 685 op/s 2024-05-03T18:39:15.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:39:15 smithi150 ceph-mon[32492]: pgmap v376: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 9.4 MiB/s wr, 685 op/s 2024-05-03T18:39:17.723 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:39:17 smithi033 ceph-mon[28330]: pgmap v377: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 7.6 MiB/s wr, 553 op/s 2024-05-03T18:39:17.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:39:17 smithi099 ceph-mon[31658]: pgmap v377: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 7.6 MiB/s wr, 553 op/s 2024-05-03T18:39:17.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:39:17 smithi150 ceph-mon[32492]: pgmap v377: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 7.6 MiB/s wr, 553 op/s 2024-05-03T18:39:19.735 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:39:19 smithi033 ceph-mon[28330]: pgmap v378: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 7.5 MiB/s wr, 548 op/s 2024-05-03T18:39:19.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:39:19 smithi099 ceph-mon[31658]: pgmap v378: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 7.5 MiB/s wr, 548 op/s 2024-05-03T18:39:19.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:39:19 smithi150 ceph-mon[32492]: pgmap v378: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 7.5 MiB/s wr, 548 op/s 2024-05-03T18:39:20.111 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 485.0 s, 608.0 tps, lat 168.239 ms stddev 132.760 2024-05-03T18:39:20.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.498+0000 7fae06400700 1 -- 172.21.15.33:0/2078627395 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fae000779d0 msgr2=0x7fae00077db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:39:20.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.498+0000 7fae06400700 1 --2- 172.21.15.33:0/2078627395 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fae000779d0 0x7fae00077db0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fadf40098f0 tx=0x7fadf4009c00 comp rx=0 tx=0).stop 2024-05-03T18:39:20.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.499+0000 7fae06400700 1 -- 172.21.15.33:0/2078627395 shutdown_connections 2024-05-03T18:39:20.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.499+0000 7fae06400700 1 --2- 172.21.15.33:0/2078627395 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fae00114250 0x7fae00116640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:20.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.499+0000 7fae06400700 1 --2- 172.21.15.33:0/2078627395 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fae000779d0 0x7fae00077db0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:20.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.499+0000 7fae06400700 1 --2- 172.21.15.33:0/2078627395 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae000782f0 0x7fae00113d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:20.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.499+0000 7fae06400700 1 -- 172.21.15.33:0/2078627395 >> 172.21.15.33:0/2078627395 conn(0x7fae00101080 msgr2=0x7fae001034a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:39:20.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.499+0000 7fae06400700 1 -- 172.21.15.33:0/2078627395 shutdown_connections 2024-05-03T18:39:20.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.499+0000 7fae06400700 1 -- 172.21.15.33:0/2078627395 wait complete. 2024-05-03T18:39:20.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.500+0000 7fae06400700 1 Processor -- start 2024-05-03T18:39:20.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.501+0000 7fae06400700 1 -- start start 2024-05-03T18:39:20.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.501+0000 7fae06400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fae000779d0 0x7fae001a0020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:39:20.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.501+0000 7fae06400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae000782f0 0x7fae001a0560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:39:20.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.501+0000 7fae06400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fae00114250 0x7fae001a48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:39:20.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.501+0000 7fae06400700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fae00119ad0 con 0x7fae000782f0 2024-05-03T18:39:20.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.502+0000 7fae06400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fae00119950 con 0x7fae000779d0 2024-05-03T18:39:20.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.502+0000 7fae06400700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fae00119c50 con 0x7fae00114250 2024-05-03T18:39:20.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.502+0000 7fae05a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fae00114250 0x7fae001a48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:39:20.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.502+0000 7fae05a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fae00114250 0x7fae001a48f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:54700/0 (socket says 172.21.15.33:54700) 2024-05-03T18:39:20.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.502+0000 7fae05a00700 1 -- 172.21.15.33:0/2123358113 learned_addr learned my addr 172.21.15.33:0/2123358113 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:39:20.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.503+0000 7fae05000700 1 --2- 172.21.15.33:0/2123358113 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fae000779d0 0x7fae001a0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:39:20.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.503+0000 7fae05a00700 1 -- 172.21.15.33:0/2123358113 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fae000779d0 msgr2=0x7fae001a0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:39:20.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.503+0000 7fae05a00700 1 --2- 172.21.15.33:0/2123358113 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fae000779d0 0x7fae001a0020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:20.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.503+0000 7fae05a00700 1 -- 172.21.15.33:0/2123358113 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae000782f0 msgr2=0x7fae001a0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:39:20.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.503+0000 7fae05a00700 1 --2- 172.21.15.33:0/2123358113 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae000782f0 0x7fae001a0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:20.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.503+0000 7fae05a00700 1 -- 172.21.15.33:0/2123358113 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae001a5030 con 0x7fae00114250 2024-05-03T18:39:20.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.504+0000 7fae05a00700 1 --2- 172.21.15.33:0/2123358113 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fae00114250 0x7fae001a48f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fadf800b9e0 tx=0x7fadf800bda0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:39:20.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.504+0000 7fadfd600700 1 -- 172.21.15.33:0/2123358113 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf8019070 con 0x7fae00114250 2024-05-03T18:39:20.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.504+0000 7fadfd600700 1 -- 172.21.15.33:0/2123358113 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fadf800ce40 con 0x7fae00114250 2024-05-03T18:39:20.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.504+0000 7fadfd600700 1 -- 172.21.15.33:0/2123358113 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf8012340 con 0x7fae00114250 2024-05-03T18:39:20.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.505+0000 7fae06400700 1 -- 172.21.15.33:0/2123358113 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae001a5320 con 0x7fae00114250 2024-05-03T18:39:20.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.505+0000 7fae06400700 1 -- 172.21.15.33:0/2123358113 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae001acba0 con 0x7fae00114250 2024-05-03T18:39:20.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.514+0000 7fadfd600700 1 -- 172.21.15.33:0/2123358113 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fadf801c070 con 0x7fae00114250 2024-05-03T18:39:20.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.519+0000 7fadfd600700 1 --2- 172.21.15.33:0/2123358113 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fade80b2a70 0x7fade80b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:39:20.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.519+0000 7fadfd600700 1 -- 172.21.15.33:0/2123358113 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9518+0+0 (secure 0 0 0) 0x7fadf80e1eb0 con 0x7fae00114250 2024-05-03T18:39:20.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.520+0000 7fae05000700 1 --2- 172.21.15.33:0/2123358113 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fade80b2a70 0x7fade80b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:39:20.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.522+0000 7fae05000700 1 --2- 172.21.15.33:0/2123358113 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fade80b2a70 0x7fade80b4f30 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fadf4009fd0 tx=0x7fadf4019040 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:39:20.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.592+0000 7fae12366b80 1 Processor -- start 2024-05-03T18:39:20.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.592+0000 7fae12366b80 1 -- start start 2024-05-03T18:39:20.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.593+0000 7fae12366b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55727cf8ea40 0x55727cf8ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:39:20.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.593+0000 7fae12366b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55727cf8f360 0x55727cf88fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:39:20.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.593+0000 7fae12366b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55727cf895d0 0x55727cf89a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:39:20.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.593+0000 7fae12366b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55727cedba20 con 0x55727cf895d0 2024-05-03T18:39:20.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.593+0000 7fae12366b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55727ce79760 con 0x55727cf8f360 2024-05-03T18:39:20.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.593+0000 7fae12366b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55727ced79e0 con 0x55727cf8ea40 2024-05-03T18:39:20.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.594+0000 7fae05000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55727cf8ea40 0x55727cf8ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:39:20.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.594+0000 7fae05000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55727cf8ea40 0x55727cf8ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:54714/0 (socket says 172.21.15.33:54714) 2024-05-03T18:39:20.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.594+0000 7fae05000700 1 -- 172.21.15.33:0/1857954158 learned_addr learned my addr 172.21.15.33:0/1857954158 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:39:20.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.594+0000 7fae05a00700 1 --2- 172.21.15.33:0/1857954158 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55727cf895d0 0x55727cf89a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:39:20.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.595+0000 7fae05000700 1 -- 172.21.15.33:0/1857954158 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55727cf8f360 msgr2=0x55727cf88fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:39:20.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.595+0000 7fae05000700 1 --2- 172.21.15.33:0/1857954158 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55727cf8f360 0x55727cf88fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:20.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.595+0000 7fae05000700 1 -- 172.21.15.33:0/1857954158 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55727cf895d0 msgr2=0x55727cf89a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:39:20.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.595+0000 7fae05000700 1 --2- 172.21.15.33:0/1857954158 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55727cf895d0 0x55727cf89a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:20.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.595+0000 7fae05000700 1 -- 172.21.15.33:0/1857954158 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55727ce288d0 con 0x55727cf8ea40 2024-05-03T18:39:20.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.596+0000 7fae05000700 1 --2- 172.21.15.33:0/1857954158 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55727cf8ea40 0x55727cf8ee20 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fadf4016000 tx=0x7fadf401d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2e943e6352faae7 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:39:20.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.596+0000 7fadf2000700 1 -- 172.21.15.33:0/1857954158 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf4032070 con 0x55727cf8ea40 2024-05-03T18:39:20.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.596+0000 7fadf2000700 1 -- 172.21.15.33:0/1857954158 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fadf40054d0 con 0x55727cf8ea40 2024-05-03T18:39:20.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.596+0000 7fadf2000700 1 -- 172.21.15.33:0/1857954158 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf400b560 con 0x55727cf8ea40 2024-05-03T18:39:20.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.597+0000 7fae12366b80 1 -- 172.21.15.33:0/1857954158 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55727cf8ea40 msgr2=0x55727cf8ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:39:20.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.597+0000 7fae12366b80 1 --2- 172.21.15.33:0/1857954158 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55727cf8ea40 0x55727cf8ee20 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fadf4016000 tx=0x7fadf401d000 comp rx=0 tx=0).stop 2024-05-03T18:39:20.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.598+0000 7fae12366b80 1 -- 172.21.15.33:0/1857954158 shutdown_connections 2024-05-03T18:39:20.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.598+0000 7fae12366b80 1 --2- 172.21.15.33:0/1857954158 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55727cf895d0 0x55727cf89a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:20.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.598+0000 7fae12366b80 1 --2- 172.21.15.33:0/1857954158 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55727cf8ea40 0x55727cf8ee20 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:20.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.598+0000 7fae12366b80 1 --2- 172.21.15.33:0/1857954158 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55727cf8f360 0x55727cf88fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:20.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.598+0000 7fae12366b80 1 -- 172.21.15.33:0/1857954158 >> 172.21.15.33:0/1857954158 conn(0x55727ceb7b60 msgr2=0x55727cf839d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:39:20.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.599+0000 7fae12366b80 1 -- 172.21.15.33:0/1857954158 shutdown_connections 2024-05-03T18:39:20.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.599+0000 7fae12366b80 1 -- 172.21.15.33:0/1857954158 wait complete. 2024-05-03T18:39:20.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.600+0000 7fae12366b80 1 -- start start 2024-05-03T18:39:20.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.601+0000 7fae12366b80 1 Processor -- start 2024-05-03T18:39:20.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.603+0000 7fae12366b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55727cf895d0 0x55727cf950c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:39:20.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.603+0000 7fae12366b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55727cf8ea40 0x55727cf92460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:39:20.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.604+0000 7fae12366b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55727cf8f360 0x55727cf929a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:39:20.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.604+0000 7fae05000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55727cf895d0 0x55727cf950c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:39:20.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.604+0000 7fae05000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55727cf895d0 0x55727cf950c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:54720/0 (socket says 172.21.15.33:54720) 2024-05-03T18:39:20.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.604+0000 7fae05000700 1 -- 172.21.15.33:0/1338179644 learned_addr learned my addr 172.21.15.33:0/1338179644 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:39:20.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.604+0000 7fadffe00700 1 --2- 172.21.15.33:0/1338179644 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55727cf8ea40 0x55727cf92460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:39:20.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.604+0000 7fae12366b80 1 -- 172.21.15.33:0/1338179644 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55727cf95d70 con 0x55727cf8f360 2024-05-03T18:39:20.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.604+0000 7fae12366b80 1 -- 172.21.15.33:0/1338179644 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55727ce41b20 con 0x55727cf8ea40 2024-05-03T18:39:20.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.605+0000 7fae12366b80 1 -- 172.21.15.33:0/1338179644 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55727cf95ef0 con 0x55727cf895d0 2024-05-03T18:39:20.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.605+0000 7fae12366b80 10 client.0 _ll_register_callbacks cb 0x55727ce82c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:39:20.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.606+0000 7fae05000700 1 -- 172.21.15.33:0/1338179644 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55727cf8ea40 msgr2=0x55727cf92460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:39:20.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.606+0000 7fae05000700 1 --2- 172.21.15.33:0/1338179644 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55727cf8ea40 0x55727cf92460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:20.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.606+0000 7fae05000700 1 -- 172.21.15.33:0/1338179644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55727cf8f360 msgr2=0x55727cf929a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:39:20.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.606+0000 7fae05000700 1 --2- 172.21.15.33:0/1338179644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55727cf8f360 0x55727cf929a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:20.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.606+0000 7fae05000700 1 -- 172.21.15.33:0/1338179644 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55727cea1420 con 0x55727cf895d0 2024-05-03T18:39:20.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.606+0000 7fadffe00700 1 --2- 172.21.15.33:0/1338179644 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55727cf8ea40 0x55727cf92460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:39:20.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.607+0000 7fae05000700 1 --2- 172.21.15.33:0/1338179644 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55727cf895d0 0x55727cf950c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fadf4004900 tx=0x7fadf4004f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6042a485ed758a12 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:39:20.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.607+0000 7fade6a00700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-03T18:39:20.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.607+0000 7fade6a00700 1 -- 172.21.15.33:0/1338179644 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf4032040 con 0x55727cf895d0 2024-05-03T18:39:20.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.607+0000 7fade6a00700 1 -- 172.21.15.33:0/1338179644 <== mon.1 v2:172.21.15.150:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9518+0+0 (secure 0 0 0) 0x7fadf4005040 con 0x55727cf895d0 2024-05-03T18:39:20.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.607+0000 7fae12366b80 1 -- 172.21.15.33:0/1338179644 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fadf40054d0 con 0x55727cf895d0 2024-05-03T18:39:20.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.608+0000 7fade6a00700 1 -- 172.21.15.33:0/1338179644 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fadf4043050 con 0x55727cf895d0 2024-05-03T18:39:20.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.608+0000 7fae12366b80 10 client.26296 fetch_fsmap learned FSMap version 21 2024-05-03T18:39:20.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.608+0000 7fae12366b80 1 -- 172.21.15.33:0/1338179644 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55727cf95d70 con 0x55727cf895d0 2024-05-03T18:39:20.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.609+0000 7fade6a00700 1 -- 172.21.15.33:0/1338179644 <== mon.1 v2:172.21.15.150:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fadf40054d0 con 0x55727cf895d0 2024-05-03T18:39:20.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.609+0000 7fade6a00700 1 -- 172.21.15.33:0/1338179644 <== mon.1 v2:172.21.15.150:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fadf403e030 con 0x55727cf895d0 2024-05-03T18:39:20.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.609+0000 7fae12366b80 10 client.26296 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:39:20.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.609+0000 7fae12366b80 10 client.26296 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:39:20.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.609+0000 7fae12366b80 20 client.26296 populate_metadata read hostname 'smithi033' 2024-05-03T18:39:20.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.609+0000 7fae12366b80 1 --2- 172.21.15.33:0/1338179644 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55727cfa28d0 0x55727cfa4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:39:20.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.609+0000 7fae12366b80 4 client.26296 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:39:20.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.609+0000 7fae12366b80 1 -- 172.21.15.33:0/1338179644 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55727cf561d0 con 0x55727cfa28d0 2024-05-03T18:39:20.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.610+0000 7fadffe00700 1 --2- 172.21.15.33:0/1338179644 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55727cfa28d0 0x55727cfa4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:39:20.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.611+0000 7fadffe00700 1 --2- 172.21.15.33:0/1338179644 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55727cfa28d0 0x55727cfa4cc0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2fd7730b145ae6e server_cookie=5b5bfcb47ae8cdfc in_seq=0 out_seq=0 2024-05-03T18:39:20.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.611+0000 7fade6a00700 10 client.26296 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:39:20.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.613+0000 7fade6a00700 1 -- 172.21.15.33:0/1338179644 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55727cf561d0 con 0x55727cfa28d0 2024-05-03T18:39:20.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.613+0000 7fade6a00700 10 client.26296 handle_command_reply: tid=0 2024-05-03T18:39:20.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.613+0000 7fae12366b80 1 client.26296 shutdown 2024-05-03T18:39:20.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.614+0000 7fae12366b80 1 -- 172.21.15.33:0/1338179644 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55727cf895d0 msgr2=0x55727cf950c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:39:20.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.614+0000 7fae12366b80 1 --2- 172.21.15.33:0/1338179644 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55727cf895d0 0x55727cf950c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fadf4004900 tx=0x7fadf4004f90 comp rx=0 tx=0).stop 2024-05-03T18:39:20.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.614+0000 7fae12366b80 1 -- 172.21.15.33:0/1338179644 shutdown_connections 2024-05-03T18:39:20.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.614+0000 7fae12366b80 1 --2- 172.21.15.33:0/1338179644 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55727cfa28d0 0x55727cfa4cc0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:20.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.614+0000 7fae12366b80 1 --2- 172.21.15.33:0/1338179644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55727cf8f360 0x55727cf929a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:20.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.614+0000 7fae12366b80 1 --2- 172.21.15.33:0/1338179644 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55727cf895d0 0x55727cf950c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:20.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.614+0000 7fae12366b80 1 --2- 172.21.15.33:0/1338179644 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55727cf8ea40 0x55727cf92460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:20.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.614+0000 7fae12366b80 1 -- 172.21.15.33:0/1338179644 >> 172.21.15.33:0/1338179644 conn(0x55727ceb7b60 msgr2=0x55727cf83080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:39:20.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.614+0000 7fade6a00700 0 client.26296 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:39:20.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.615+0000 7fae12366b80 1 -- 172.21.15.33:0/1338179644 shutdown_connections 2024-05-03T18:39:20.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.615+0000 7fae12366b80 1 -- 172.21.15.33:0/1338179644 wait complete. 2024-05-03T18:39:20.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.615+0000 7fae12366b80 20 client.26296 trim_cache size 0 max 16384 2024-05-03T18:39:20.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.645+0000 7fae12366b80 1 Processor -- start 2024-05-03T18:39:20.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.645+0000 7fae12366b80 1 -- start start 2024-05-03T18:39:20.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.645+0000 7fae12366b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55727cf895d0 0x55727cf9ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:39:20.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.645+0000 7fae12366b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55727cf8ea40 0x55727cf83790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:39:20.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.645+0000 7fae12366b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55727cf8f360 0x55727cf8d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:39:20.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.646+0000 7fae12366b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fae001b39e0 con 0x55727cf895d0 2024-05-03T18:39:20.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.646+0000 7fae12366b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fae001b6240 con 0x55727cf8ea40 2024-05-03T18:39:20.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.646+0000 7fae12366b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fae001b4030 con 0x55727cf8f360 2024-05-03T18:39:20.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.646+0000 7fae05000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55727cf895d0 0x55727cf9ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:39:20.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.646+0000 7fae05000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55727cf895d0 0x55727cf9ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59040/0 (socket says 172.21.15.33:59040) 2024-05-03T18:39:20.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.646+0000 7fae05000700 1 -- 172.21.15.33:0/3514010741 learned_addr learned my addr 172.21.15.33:0/3514010741 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:39:20.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.646+0000 7fadffe00700 1 --2- 172.21.15.33:0/3514010741 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55727cf8ea40 0x55727cf83790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:39:20.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.647+0000 7fae05000700 1 -- 172.21.15.33:0/3514010741 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55727cf8f360 msgr2=0x55727cf8d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:39:20.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.647+0000 7fae05000700 1 --2- 172.21.15.33:0/3514010741 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55727cf8f360 0x55727cf8d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:20.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.647+0000 7fae05000700 1 -- 172.21.15.33:0/3514010741 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55727cf8ea40 msgr2=0x55727cf83790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:39:20.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.647+0000 7fae05000700 1 --2- 172.21.15.33:0/3514010741 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55727cf8ea40 0x55727cf83790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:20.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.647+0000 7fae05000700 1 -- 172.21.15.33:0/3514010741 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55727cf909f0 con 0x55727cf895d0 2024-05-03T18:39:20.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.647+0000 7fae05000700 1 --2- 172.21.15.33:0/3514010741 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55727cf895d0 0x55727cf9ee00 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fadf400f6c0 tx=0x7fadf400fb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=447015e3239ac96a server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:39:20.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.648+0000 7fadf1600700 1 -- 172.21.15.33:0/3514010741 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf4032040 con 0x55727cf895d0 2024-05-03T18:39:20.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.648+0000 7fadf1600700 1 -- 172.21.15.33:0/3514010741 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fadf4022e40 con 0x55727cf895d0 2024-05-03T18:39:20.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.648+0000 7fadf1600700 1 -- 172.21.15.33:0/3514010741 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf400b560 con 0x55727cf895d0 2024-05-03T18:39:20.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.649+0000 7fae12366b80 1 -- 172.21.15.33:0/3514010741 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55727cf895d0 msgr2=0x55727cf9ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:39:20.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.649+0000 7fae12366b80 1 --2- 172.21.15.33:0/3514010741 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55727cf895d0 0x55727cf9ee00 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fadf400f6c0 tx=0x7fadf400fb20 comp rx=0 tx=0).stop 2024-05-03T18:39:20.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.649+0000 7fae12366b80 1 -- 172.21.15.33:0/3514010741 shutdown_connections 2024-05-03T18:39:20.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.649+0000 7fae12366b80 1 --2- 172.21.15.33:0/3514010741 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55727cf8f360 0x55727cf8d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:20.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.650+0000 7fae12366b80 1 --2- 172.21.15.33:0/3514010741 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55727cf895d0 0x55727cf9ee00 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:20.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.650+0000 7fae12366b80 1 --2- 172.21.15.33:0/3514010741 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55727cf8ea40 0x55727cf83790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:20.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.650+0000 7fae12366b80 1 -- 172.21.15.33:0/3514010741 >> 172.21.15.33:0/3514010741 conn(0x55727ceb7b60 msgr2=0x55727cf83080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:39:20.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.650+0000 7fae12366b80 1 -- 172.21.15.33:0/3514010741 shutdown_connections 2024-05-03T18:39:20.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.650+0000 7fae12366b80 1 -- 172.21.15.33:0/3514010741 wait complete. 2024-05-03T18:39:20.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.650+0000 7fae12366b80 1 -- start start 2024-05-03T18:39:20.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.651+0000 7fae12366b80 1 Processor -- start 2024-05-03T18:39:20.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.652+0000 7fae12366b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55727cf895d0 0x55727cf9b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:39:20.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.652+0000 7fae12366b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55727cf8ea40 0x55727cf96ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:39:20.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.652+0000 7fae12366b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55727cf8f360 0x55727cf97080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:39:20.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.653+0000 7fae12366b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fae001b4700 con 0x55727cf8f360 2024-05-03T18:39:20.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.653+0000 7fae12366b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fae001b5b70 con 0x55727cf8ea40 2024-05-03T18:39:20.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.653+0000 7fae12366b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fae001b4dd0 con 0x55727cf895d0 2024-05-03T18:39:20.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.653+0000 7fae12366b80 10 client.0 _ll_register_callbacks cb 0x55727ce82c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:39:20.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.653+0000 7fae05a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55727cf8f360 0x55727cf97080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:39:20.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.653+0000 7fadffe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55727cf8ea40 0x55727cf96ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:39:20.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.653+0000 7fae05000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55727cf895d0 0x55727cf9b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:39:20.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.653+0000 7fadffe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55727cf8ea40 0x55727cf96ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:34276/0 (socket says 172.21.15.33:34276) 2024-05-03T18:39:20.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.653+0000 7fadffe00700 1 -- 172.21.15.33:0/3519514849 learned_addr learned my addr 172.21.15.33:0/3519514849 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:39:20.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.654+0000 7fae05a00700 1 -- 172.21.15.33:0/3519514849 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55727cf895d0 msgr2=0x55727cf9b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:39:20.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.654+0000 7fae05a00700 1 --2- 172.21.15.33:0/3519514849 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55727cf895d0 0x55727cf9b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:20.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.654+0000 7fae05a00700 1 -- 172.21.15.33:0/3519514849 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55727cf8ea40 msgr2=0x55727cf96ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:39:20.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.654+0000 7fae05a00700 1 --2- 172.21.15.33:0/3519514849 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55727cf8ea40 0x55727cf96ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:20.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.655+0000 7fae05a00700 1 -- 172.21.15.33:0/3519514849 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55727cf978c0 con 0x55727cf8f360 2024-05-03T18:39:20.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.655+0000 7fade6a00700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:39:20.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.655+0000 7fae05a00700 1 --2- 172.21.15.33:0/3519514849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55727cf8f360 0x55727cf97080 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fadf80df330 tx=0x7fadf80a9400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50b1e4d35b85b902 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:39:20.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.656+0000 7fade6a00700 1 -- 172.21.15.33:0/3519514849 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf80cb070 con 0x55727cf8f360 2024-05-03T18:39:20.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.656+0000 7fade6a00700 1 -- 172.21.15.33:0/3519514849 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9518+0+0 (secure 0 0 0) 0x7fadf80a54b0 con 0x55727cf8f360 2024-05-03T18:39:20.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.656+0000 7fae12366b80 1 -- 172.21.15.33:0/3519514849 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55727cee9400 con 0x55727cf8f360 2024-05-03T18:39:20.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.657+0000 7fade6a00700 1 -- 172.21.15.33:0/3519514849 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fadf80cb070 con 0x55727cf8f360 2024-05-03T18:39:20.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.658+0000 7fae12366b80 10 client.16509 fetch_fsmap learned FSMap version 21 2024-05-03T18:39:20.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.658+0000 7fae12366b80 1 -- 172.21.15.33:0/3519514849 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55727d00d000 con 0x55727cf8f360 2024-05-03T18:39:20.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.659+0000 7fade6a00700 1 -- 172.21.15.33:0/3519514849 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55727cee9400 con 0x55727cf8f360 2024-05-03T18:39:20.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.659+0000 7fade6a00700 1 -- 172.21.15.33:0/3519514849 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fadf80a4830 con 0x55727cf8f360 2024-05-03T18:39:20.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.659+0000 7fae12366b80 10 client.16509 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:39:20.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.659+0000 7fae12366b80 10 client.16509 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:39:20.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.659+0000 7fae12366b80 20 client.16509 populate_metadata read hostname 'smithi033' 2024-05-03T18:39:20.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.659+0000 7fae12366b80 1 --2- 172.21.15.33:0/3519514849 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55727d00d6c0 0x55727d00fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:39:20.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.660+0000 7fae12366b80 4 client.16509 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-03T18:39:20.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.661+0000 7fae12366b80 1 -- 172.21.15.33:0/3519514849 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55727cee34a0 con 0x55727d00d6c0 2024-05-03T18:39:20.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.661+0000 7fae05000700 1 --2- 172.21.15.33:0/3519514849 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55727d00d6c0 0x55727d00fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:39:20.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.662+0000 7fae05000700 1 --2- 172.21.15.33:0/3519514849 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55727d00d6c0 0x55727d00fae0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fba1020420ba9de server_cookie=5a416b1b10b929fb in_seq=0 out_seq=0 2024-05-03T18:39:20.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.662+0000 7fade6a00700 10 client.16509 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:39:20.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.663+0000 7fade6a00700 1 -- 172.21.15.33:0/3519514849 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55727cee34a0 con 0x55727d00d6c0 2024-05-03T18:39:20.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.663+0000 7fade6a00700 10 client.16509 handle_command_reply: tid=0 2024-05-03T18:39:20.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.663+0000 7fae12366b80 1 client.16509 shutdown 2024-05-03T18:39:20.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.663+0000 7fae12366b80 1 -- 172.21.15.33:0/3519514849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55727cf8f360 msgr2=0x55727cf97080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:39:20.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.663+0000 7fae12366b80 1 --2- 172.21.15.33:0/3519514849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55727cf8f360 0x55727cf97080 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fadf80df330 tx=0x7fadf80a9400 comp rx=0 tx=0).stop 2024-05-03T18:39:20.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.664+0000 7fae12366b80 1 -- 172.21.15.33:0/3519514849 shutdown_connections 2024-05-03T18:39:20.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.664+0000 7fae12366b80 1 --2- 172.21.15.33:0/3519514849 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55727d00d6c0 0x55727d00fae0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:20.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.664+0000 7fae12366b80 1 --2- 172.21.15.33:0/3519514849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55727cf8f360 0x55727cf97080 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:20.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.664+0000 7fae12366b80 1 --2- 172.21.15.33:0/3519514849 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55727cf895d0 0x55727cf9b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:20.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.664+0000 7fae12366b80 1 --2- 172.21.15.33:0/3519514849 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55727cf8ea40 0x55727cf96ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:20.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.664+0000 7fae12366b80 1 -- 172.21.15.33:0/3519514849 >> 172.21.15.33:0/3519514849 conn(0x55727ceb7b60 msgr2=0x55727cf83910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:39:20.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.664+0000 7fade6a00700 0 client.16509 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:39:20.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.664+0000 7fae12366b80 1 -- 172.21.15.33:0/3519514849 shutdown_connections 2024-05-03T18:39:20.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.665+0000 7fae12366b80 1 -- 172.21.15.33:0/3519514849 wait complete. 2024-05-03T18:39:20.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.665+0000 7fae12366b80 20 client.16509 trim_cache size 0 max 16384 2024-05-03T18:39:20.666 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-03T18:39:20.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.668+0000 7fae06400700 1 -- 172.21.15.33:0/2123358113 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fade80b2a70 msgr2=0x7fade80b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:39:20.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.668+0000 7fae06400700 1 --2- 172.21.15.33:0/2123358113 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fade80b2a70 0x7fade80b4f30 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fadf4009fd0 tx=0x7fadf4019040 comp rx=0 tx=0).stop 2024-05-03T18:39:20.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.668+0000 7fae06400700 1 -- 172.21.15.33:0/2123358113 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fae00114250 msgr2=0x7fae001a48f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:39:20.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.668+0000 7fae06400700 1 --2- 172.21.15.33:0/2123358113 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fae00114250 0x7fae001a48f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fadf800b9e0 tx=0x7fadf800bda0 comp rx=0 tx=0).stop 2024-05-03T18:39:20.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.668+0000 7fae06400700 1 -- 172.21.15.33:0/2123358113 shutdown_connections 2024-05-03T18:39:20.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.668+0000 7fae06400700 1 --2- 172.21.15.33:0/2123358113 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fae00114250 0x7fae001a48f0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:20.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.668+0000 7fae06400700 1 --2- 172.21.15.33:0/2123358113 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fade80b2a70 0x7fade80b4f30 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:20.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.668+0000 7fae06400700 1 --2- 172.21.15.33:0/2123358113 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fae000779d0 0x7fae001a0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:20.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.668+0000 7fae06400700 1 --2- 172.21.15.33:0/2123358113 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae000782f0 0x7fae001a0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:20.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.668+0000 7fae06400700 1 -- 172.21.15.33:0/2123358113 >> 172.21.15.33:0/2123358113 conn(0x7fae00101080 msgr2=0x7fae00103470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:39:20.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.669+0000 7fae06400700 1 -- 172.21.15.33:0/2123358113 shutdown_connections 2024-05-03T18:39:20.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:20.669+0000 7fae06400700 1 -- 172.21.15.33:0/2123358113 wait complete. 2024-05-03T18:39:21.700 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:39:21 smithi033 ceph-mon[28330]: pgmap v379: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 7.6 MiB/s wr, 562 op/s 2024-05-03T18:39:21.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:39:21 smithi099 ceph-mon[31658]: pgmap v379: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 7.6 MiB/s wr, 562 op/s 2024-05-03T18:39:21.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:39:21 smithi150 ceph-mon[32492]: pgmap v379: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 7.6 MiB/s wr, 562 op/s 2024-05-03T18:39:23.102 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-03T18:39:23.102 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-03T18:39:23.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:39:23 smithi033 ceph-mon[28330]: pgmap v380: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 7.6 MiB/s wr, 562 op/s 2024-05-03T18:39:23.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:39:23 smithi099 ceph-mon[31658]: pgmap v380: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 7.6 MiB/s wr, 562 op/s 2024-05-03T18:39:23.847 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:39:23.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:39:23 smithi150 ceph-mon[32492]: pgmap v380: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 7.6 MiB/s wr, 562 op/s 2024-05-03T18:39:25.114 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 490.0 s, 521.6 tps, lat 190.746 ms stddev 278.177 2024-05-03T18:39:25.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:39:25 smithi033 ceph-mon[28330]: pgmap v381: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 11 MiB/s wr, 737 op/s 2024-05-03T18:39:25.795 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:39:25 smithi099 ceph-mon[31658]: pgmap v381: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 11 MiB/s wr, 737 op/s 2024-05-03T18:39:25.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:39:25 smithi150 ceph-mon[32492]: pgmap v381: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 11 MiB/s wr, 737 op/s 2024-05-03T18:39:27.470 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:39:27 smithi033 ceph-mon[28330]: pgmap v382: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 7.5 MiB/s wr, 478 op/s 2024-05-03T18:39:27.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:39:27 smithi099 ceph-mon[31658]: pgmap v382: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 7.5 MiB/s wr, 478 op/s 2024-05-03T18:39:27.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:39:27 smithi150 ceph-mon[32492]: pgmap v382: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 7.5 MiB/s wr, 478 op/s 2024-05-03T18:39:29.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.693+0000 7f9b67600700 1 -- 172.21.15.33:0/1322467266 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b60103520 msgr2=0x7f9b601039a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:39:29.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.693+0000 7f9b67600700 1 --2- 172.21.15.33:0/1322467266 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b60103520 0x7f9b601039a0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f9b540098f0 tx=0x7f9b54009c00 comp rx=0 tx=0).stop 2024-05-03T18:39:29.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.693+0000 7f9b67600700 1 -- 172.21.15.33:0/1322467266 shutdown_connections 2024-05-03T18:39:29.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.693+0000 7f9b67600700 1 --2- 172.21.15.33:0/1322467266 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b60103ee0 0x7f9b60110370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:29.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.693+0000 7f9b67600700 1 --2- 172.21.15.33:0/1322467266 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b60109530 0x7f9b60109910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:29.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.693+0000 7f9b67600700 1 --2- 172.21.15.33:0/1322467266 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b60103520 0x7f9b601039a0 secure :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f9b540098f0 tx=0x7f9b54009c00 comp rx=0 tx=0).stop 2024-05-03T18:39:29.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.693+0000 7f9b67600700 1 -- 172.21.15.33:0/1322467266 >> 172.21.15.33:0/1322467266 conn(0x7f9b6007a570 msgr2=0x7f9b6007a980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:39:29.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.694+0000 7f9b67600700 1 -- 172.21.15.33:0/1322467266 shutdown_connections 2024-05-03T18:39:29.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.695+0000 7f9b67600700 1 -- 172.21.15.33:0/1322467266 wait complete. 2024-05-03T18:39:29.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.696+0000 7f9b67600700 1 Processor -- start 2024-05-03T18:39:29.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.696+0000 7f9b67600700 1 -- start start 2024-05-03T18:39:29.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.696+0000 7f9b67600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b60103ee0 0x7f9b60199cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:39:29.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.697+0000 7f9b67600700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b60109530 0x7f9b6019a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:39:29.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.697+0000 7f9b67600700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b6019d4d0 0x7f9b6019d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:39:29.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.697+0000 7f9b67600700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9b60113510 con 0x7f9b60109530 2024-05-03T18:39:29.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.697+0000 7f9b67600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9b60113390 con 0x7f9b60103ee0 2024-05-03T18:39:29.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.697+0000 7f9b67600700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9b60113690 con 0x7f9b6019d4d0 2024-05-03T18:39:29.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.697+0000 7f9b5fe00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b60109530 0x7f9b6019a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:39:29.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.697+0000 7f9b5fe00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b60109530 0x7f9b6019a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56468/0 (socket says 172.21.15.33:56468) 2024-05-03T18:39:29.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.697+0000 7f9b5fe00700 1 -- 172.21.15.33:0/1804504963 learned_addr learned my addr 172.21.15.33:0/1804504963 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:39:29.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.697+0000 7f9b64c00700 1 --2- 172.21.15.33:0/1804504963 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b60103ee0 0x7f9b60199cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:39:29.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.698+0000 7f9b5fe00700 1 -- 172.21.15.33:0/1804504963 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b6019d4d0 msgr2=0x7f9b6019d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:39:29.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.698+0000 7f9b5fe00700 1 --2- 172.21.15.33:0/1804504963 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b6019d4d0 0x7f9b6019d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:29.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.698+0000 7f9b5fe00700 1 -- 172.21.15.33:0/1804504963 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b60103ee0 msgr2=0x7f9b60199cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:39:29.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.698+0000 7f9b5fe00700 1 --2- 172.21.15.33:0/1804504963 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b60103ee0 0x7f9b60199cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:29.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.698+0000 7f9b5fe00700 1 -- 172.21.15.33:0/1804504963 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b6019dff0 con 0x7f9b60109530 2024-05-03T18:39:29.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.699+0000 7f9b5fe00700 1 --2- 172.21.15.33:0/1804504963 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b60109530 0x7f9b6019a1f0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f9b54000c00 tx=0x7f9b5400fa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:39:29.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.699+0000 7f9b5d600700 1 -- 172.21.15.33:0/1804504963 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b5400b660 con 0x7f9b60109530 2024-05-03T18:39:29.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.699+0000 7f9b67600700 1 -- 172.21.15.33:0/1804504963 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b6019e280 con 0x7f9b60109530 2024-05-03T18:39:29.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.700+0000 7f9b67600700 1 -- 172.21.15.33:0/1804504963 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b6006d090 con 0x7f9b60109530 2024-05-03T18:39:29.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.701+0000 7f9b5d600700 1 -- 172.21.15.33:0/1804504963 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9b54009480 con 0x7f9b60109530 2024-05-03T18:39:29.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.701+0000 7f9b5d600700 1 -- 172.21.15.33:0/1804504963 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b540055e0 con 0x7f9b60109530 2024-05-03T18:39:29.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.705+0000 7f9b5d600700 1 -- 172.21.15.33:0/1804504963 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f9b54003680 con 0x7f9b60109530 2024-05-03T18:39:29.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.707+0000 7f9b5d600700 1 --2- 172.21.15.33:0/1804504963 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f9b480b29f0 0x7f9b480b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:39:29.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.707+0000 7f9b5d600700 1 -- 172.21.15.33:0/1804504963 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9518+0+0 (secure 0 0 0) 0x7f9b540e47f0 con 0x7f9b60109530 2024-05-03T18:39:29.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.707+0000 7f9b64c00700 1 --2- 172.21.15.33:0/1804504963 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f9b480b29f0 0x7f9b480b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:39:29.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.714+0000 7f9b64c00700 1 --2- 172.21.15.33:0/1804504963 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f9b480b29f0 0x7f9b480b4eb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c009bd0 tx=0x7f9b4c009260 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:39:29.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.718+0000 7f9b71f03b80 1 Processor -- start 2024-05-03T18:39:29.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.719+0000 7f9b71f03b80 1 -- start start 2024-05-03T18:39:29.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.719+0000 7f9b71f03b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e0bd649e20 0x55e0bd64a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:39:29.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.719+0000 7f9b71f03b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e0bd64a740 0x55e0bd6443c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:39:29.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.719+0000 7f9b71f03b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e0bd6449e0 0x55e0bd644e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:39:29.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.719+0000 7f9b71f03b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e0bd592aa0 con 0x55e0bd6449e0 2024-05-03T18:39:29.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.719+0000 7f9b71f03b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55e0bd5307e0 con 0x55e0bd649e20 2024-05-03T18:39:29.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.719+0000 7f9b71f03b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55e0bd58ea60 con 0x55e0bd64a740 2024-05-03T18:39:29.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.720+0000 7f9b65600700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e0bd6449e0 0x55e0bd644e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:39:29.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.720+0000 7f9b5fe00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e0bd64a740 0x55e0bd6443c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:39:29.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.720+0000 7f9b5fe00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e0bd64a740 0x55e0bd6443c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:34822/0 (socket says 172.21.15.33:34822) 2024-05-03T18:39:29.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.720+0000 7f9b5fe00700 1 -- 172.21.15.33:0/3578689124 learned_addr learned my addr 172.21.15.33:0/3578689124 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:39:29.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.720+0000 7f9b64c00700 1 --2- 172.21.15.33:0/3578689124 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e0bd649e20 0x55e0bd64a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:39:29.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.721+0000 7f9b5fe00700 1 -- 172.21.15.33:0/3578689124 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e0bd649e20 msgr2=0x55e0bd64a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:39:29.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.722+0000 7f9b5fe00700 1 --2- 172.21.15.33:0/3578689124 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e0bd649e20 0x55e0bd64a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:29.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.722+0000 7f9b5fe00700 1 -- 172.21.15.33:0/3578689124 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e0bd6449e0 msgr2=0x55e0bd644e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:39:29.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.722+0000 7f9b5fe00700 1 --2- 172.21.15.33:0/3578689124 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e0bd6449e0 0x55e0bd644e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:29.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.722+0000 7f9b5fe00700 1 -- 172.21.15.33:0/3578689124 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0bd4df950 con 0x55e0bd64a740 2024-05-03T18:39:29.724 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:39:29 smithi033 ceph-mon[28330]: pgmap v383: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 7.5 MiB/s wr, 478 op/s 2024-05-03T18:39:29.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.723+0000 7f9b65600700 1 --2- 172.21.15.33:0/3578689124 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e0bd6449e0 0x55e0bd644e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:39:29.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.723+0000 7f9b5fe00700 1 --2- 172.21.15.33:0/3578689124 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e0bd64a740 0x55e0bd6443c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9b54005e90 tx=0x7f9b540ab190 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4c95d154cdab5f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:39:29.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.726+0000 7f9b52000700 1 -- 172.21.15.33:0/3578689124 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b540abe20 con 0x55e0bd64a740 2024-05-03T18:39:29.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.726+0000 7f9b52000700 1 -- 172.21.15.33:0/3578689124 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9b540ab720 con 0x55e0bd64a740 2024-05-03T18:39:29.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.726+0000 7f9b52000700 1 -- 172.21.15.33:0/3578689124 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b54093110 con 0x55e0bd64a740 2024-05-03T18:39:29.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.727+0000 7f9b71f03b80 1 -- 172.21.15.33:0/3578689124 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e0bd64a740 msgr2=0x55e0bd6443c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:39:29.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.727+0000 7f9b71f03b80 1 --2- 172.21.15.33:0/3578689124 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e0bd64a740 0x55e0bd6443c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9b54005e90 tx=0x7f9b540ab190 comp rx=0 tx=0).stop 2024-05-03T18:39:29.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.731+0000 7f9b71f03b80 1 -- 172.21.15.33:0/3578689124 shutdown_connections 2024-05-03T18:39:29.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.731+0000 7f9b71f03b80 1 --2- 172.21.15.33:0/3578689124 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e0bd6449e0 0x55e0bd644e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:29.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.731+0000 7f9b71f03b80 1 --2- 172.21.15.33:0/3578689124 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e0bd649e20 0x55e0bd64a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:29.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.731+0000 7f9b71f03b80 1 --2- 172.21.15.33:0/3578689124 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e0bd64a740 0x55e0bd6443c0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:29.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.731+0000 7f9b71f03b80 1 -- 172.21.15.33:0/3578689124 >> 172.21.15.33:0/3578689124 conn(0x55e0bd5b8750 msgr2=0x55e0bd63a980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:39:29.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.731+0000 7f9b71f03b80 1 -- 172.21.15.33:0/3578689124 shutdown_connections 2024-05-03T18:39:29.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.731+0000 7f9b71f03b80 1 -- 172.21.15.33:0/3578689124 wait complete. 2024-05-03T18:39:29.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.732+0000 7f9b71f03b80 1 -- start start 2024-05-03T18:39:29.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.734+0000 7f9b71f03b80 1 Processor -- start 2024-05-03T18:39:29.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.735+0000 7f9b71f03b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e0bd6449e0 0x55e0bd6500c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:39:29.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.735+0000 7f9b71f03b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e0bd649e20 0x55e0bd650600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:39:29.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.735+0000 7f9b71f03b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e0bd64d6b0 0x55e0bd64db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:39:29.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.735+0000 7f9b71f03b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e0bd650fa0 con 0x55e0bd64d6b0 2024-05-03T18:39:29.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.736+0000 7f9b71f03b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55e0bd4f8ba0 con 0x55e0bd6449e0 2024-05-03T18:39:29.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.736+0000 7f9b71f03b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55e0bd651120 con 0x55e0bd649e20 2024-05-03T18:39:29.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.736+0000 7f9b71f03b80 10 client.0 _ll_register_callbacks cb 0x55e0bd539cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:39:29.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.736+0000 7f9b64c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e0bd6449e0 0x55e0bd6500c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:39:29.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.736+0000 7f9b64c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e0bd6449e0 0x55e0bd6500c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:46716/0 (socket says 172.21.15.33:46716) 2024-05-03T18:39:29.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.736+0000 7f9b64c00700 1 -- 172.21.15.33:0/3239731101 learned_addr learned my addr 172.21.15.33:0/3239731101 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:39:29.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.737+0000 7f9b65600700 1 --2- 172.21.15.33:0/3239731101 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e0bd64d6b0 0x55e0bd64db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:39:29.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.737+0000 7f9b5fe00700 1 --2- 172.21.15.33:0/3239731101 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e0bd649e20 0x55e0bd650600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:39:29.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.738+0000 7f9b64c00700 1 -- 172.21.15.33:0/3239731101 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e0bd649e20 msgr2=0x55e0bd650600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:39:29.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.738+0000 7f9b64c00700 1 --2- 172.21.15.33:0/3239731101 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e0bd649e20 0x55e0bd650600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:29.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.738+0000 7f9b64c00700 1 -- 172.21.15.33:0/3239731101 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e0bd64d6b0 msgr2=0x55e0bd64db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:39:29.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.738+0000 7f9b64c00700 1 --2- 172.21.15.33:0/3239731101 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e0bd64d6b0 0x55e0bd64db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:29.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.738+0000 7f9b64c00700 1 -- 172.21.15.33:0/3239731101 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0bd5584a0 con 0x55e0bd6449e0 2024-05-03T18:39:29.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.739+0000 7f9b64c00700 1 --2- 172.21.15.33:0/3239731101 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e0bd6449e0 0x55e0bd6500c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c0118f0 tx=0x7f9b4c0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3caffd591984c9ec server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:39:29.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.739+0000 7f9b46a00700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-03T18:39:29.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.740+0000 7f9b46a00700 1 -- 172.21.15.33:0/3239731101 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b4c028670 con 0x55e0bd6449e0 2024-05-03T18:39:29.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.740+0000 7f9b46a00700 1 -- 172.21.15.33:0/3239731101 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9518+0+0 (secure 0 0 0) 0x7f9b4c028810 con 0x55e0bd6449e0 2024-05-03T18:39:29.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.741+0000 7f9b71f03b80 1 -- 172.21.15.33:0/3239731101 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b540ab720 con 0x55e0bd6449e0 2024-05-03T18:39:29.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.741+0000 7f9b46a00700 1 -- 172.21.15.33:0/3239731101 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f9b4c028a10 con 0x55e0bd6449e0 2024-05-03T18:39:29.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.741+0000 7f9b71f03b80 10 client.36224 fetch_fsmap learned FSMap version 21 2024-05-03T18:39:29.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.741+0000 7f9b71f03b80 1 -- 172.21.15.33:0/3239731101 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55e0bd643e00 con 0x55e0bd6449e0 2024-05-03T18:39:29.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.742+0000 7f9b46a00700 1 -- 172.21.15.33:0/3239731101 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9b540ab720 con 0x55e0bd6449e0 2024-05-03T18:39:29.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.743+0000 7f9b46a00700 1 -- 172.21.15.33:0/3239731101 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f9b4c032420 con 0x55e0bd6449e0 2024-05-03T18:39:29.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.743+0000 7f9b71f03b80 10 client.36224 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:39:29.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.743+0000 7f9b71f03b80 10 client.36224 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:39:29.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.743+0000 7f9b71f03b80 20 client.36224 populate_metadata read hostname 'smithi033' 2024-05-03T18:39:29.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.743+0000 7f9b71f03b80 1 --2- 172.21.15.33:0/3239731101 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55e0bd6624a0 0x55e0bd6648c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:39:29.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.743+0000 7f9b71f03b80 4 client.36224 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:39:29.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.743+0000 7f9b71f03b80 1 -- 172.21.15.33:0/3239731101 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e0bd60d070 con 0x55e0bd6624a0 2024-05-03T18:39:29.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.744+0000 7f9b5fe00700 1 --2- 172.21.15.33:0/3239731101 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55e0bd6624a0 0x55e0bd6648c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:39:29.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.745+0000 7f9b5fe00700 1 --2- 172.21.15.33:0/3239731101 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55e0bd6624a0 0x55e0bd6648c0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d1e4be3eb9a089d server_cookie=eb58db4bffeb0b83 in_seq=0 out_seq=0 2024-05-03T18:39:29.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.745+0000 7f9b46a00700 10 client.36224 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:39:29.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.747+0000 7f9b46a00700 1 -- 172.21.15.33:0/3239731101 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e0bd60d070 con 0x55e0bd6624a0 2024-05-03T18:39:29.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.747+0000 7f9b46a00700 10 client.36224 handle_command_reply: tid=0 2024-05-03T18:39:29.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.747+0000 7f9b71f03b80 1 client.36224 shutdown 2024-05-03T18:39:29.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.747+0000 7f9b71f03b80 1 -- 172.21.15.33:0/3239731101 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e0bd6449e0 msgr2=0x55e0bd6500c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:39:29.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.747+0000 7f9b71f03b80 1 --2- 172.21.15.33:0/3239731101 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e0bd6449e0 0x55e0bd6500c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c0118f0 tx=0x7f9b4c0047c0 comp rx=0 tx=0).stop 2024-05-03T18:39:29.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.747+0000 7f9b71f03b80 1 -- 172.21.15.33:0/3239731101 shutdown_connections 2024-05-03T18:39:29.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.747+0000 7f9b71f03b80 1 --2- 172.21.15.33:0/3239731101 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55e0bd6624a0 0x55e0bd6648c0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:29.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.748+0000 7f9b71f03b80 1 --2- 172.21.15.33:0/3239731101 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e0bd64d6b0 0x55e0bd64db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:29.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.748+0000 7f9b71f03b80 1 --2- 172.21.15.33:0/3239731101 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e0bd6449e0 0x55e0bd6500c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:29.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.748+0000 7f9b71f03b80 1 --2- 172.21.15.33:0/3239731101 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e0bd649e20 0x55e0bd650600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:29.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.748+0000 7f9b71f03b80 1 -- 172.21.15.33:0/3239731101 >> 172.21.15.33:0/3239731101 conn(0x55e0bd5b8750 msgr2=0x55e0bd635430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:39:29.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.748+0000 7f9b46a00700 0 client.36224 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:39:29.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.748+0000 7f9b71f03b80 1 -- 172.21.15.33:0/3239731101 shutdown_connections 2024-05-03T18:39:29.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.748+0000 7f9b71f03b80 1 -- 172.21.15.33:0/3239731101 wait complete. 2024-05-03T18:39:29.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.748+0000 7f9b71f03b80 20 client.36224 trim_cache size 0 max 16384 2024-05-03T18:39:29.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.783+0000 7f9b71f03b80 1 Processor -- start 2024-05-03T18:39:29.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.783+0000 7f9b71f03b80 1 -- start start 2024-05-03T18:39:29.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.784+0000 7f9b71f03b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e0bd6449e0 0x55e0bd64b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:39:29.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.784+0000 7f9b71f03b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e0bd649e20 0x55e0bd66a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:39:29.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.784+0000 7f9b71f03b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e0bd64d6b0 0x55e0bd66a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:39:29.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.784+0000 7f9b71f03b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9b4000ce00 con 0x55e0bd6449e0 2024-05-03T18:39:29.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.784+0000 7f9b64c00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e0bd6449e0 0x55e0bd64b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:39:29.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.784+0000 7f9b64c00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e0bd6449e0 0x55e0bd64b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56504/0 (socket says 172.21.15.33:56504) 2024-05-03T18:39:29.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.784+0000 7f9b64c00700 1 -- 172.21.15.33:0/2329410295 learned_addr learned my addr 172.21.15.33:0/2329410295 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:39:29.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.784+0000 7f9b71f03b80 1 -- 172.21.15.33:0/2329410295 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9b4000f5d0 con 0x55e0bd649e20 2024-05-03T18:39:29.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.784+0000 7f9b71f03b80 1 -- 172.21.15.33:0/2329410295 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9b4000d3c0 con 0x55e0bd64d6b0 2024-05-03T18:39:29.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.784+0000 7f9b5fe00700 1 --2- 172.21.15.33:0/2329410295 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e0bd649e20 0x55e0bd66a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:39:29.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.784+0000 7f9b65600700 1 --2- 172.21.15.33:0/2329410295 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e0bd64d6b0 0x55e0bd66a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:39:29.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.785+0000 7f9b64c00700 1 -- 172.21.15.33:0/2329410295 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e0bd64d6b0 msgr2=0x55e0bd66a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:39:29.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.785+0000 7f9b64c00700 1 --2- 172.21.15.33:0/2329410295 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e0bd64d6b0 0x55e0bd66a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:29.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.785+0000 7f9b64c00700 1 -- 172.21.15.33:0/2329410295 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e0bd649e20 msgr2=0x55e0bd66a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:39:29.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.785+0000 7f9b64c00700 1 --2- 172.21.15.33:0/2329410295 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e0bd649e20 0x55e0bd66a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:29.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.785+0000 7f9b64c00700 1 -- 172.21.15.33:0/2329410295 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0bd649680 con 0x55e0bd6449e0 2024-05-03T18:39:29.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.785+0000 7f9b65600700 1 --2- 172.21.15.33:0/2329410295 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e0bd64d6b0 0x55e0bd66a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:39:29.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.786+0000 7f9b64c00700 1 --2- 172.21.15.33:0/2329410295 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e0bd6449e0 0x55e0bd64b240 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c010760 tx=0x7f9b4c0046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8ace4159ab45d75 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:39:29.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.788+0000 7f9b51600700 1 -- 172.21.15.33:0/2329410295 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b4c010a50 con 0x55e0bd6449e0 2024-05-03T18:39:29.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.789+0000 7f9b51600700 1 -- 172.21.15.33:0/2329410295 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9b4c023070 con 0x55e0bd6449e0 2024-05-03T18:39:29.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.789+0000 7f9b51600700 1 -- 172.21.15.33:0/2329410295 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b4c032d60 con 0x55e0bd6449e0 2024-05-03T18:39:29.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.790+0000 7f9b71f03b80 1 -- 172.21.15.33:0/2329410295 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e0bd6449e0 msgr2=0x55e0bd64b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:39:29.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.790+0000 7f9b71f03b80 1 --2- 172.21.15.33:0/2329410295 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e0bd6449e0 0x55e0bd64b240 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c010760 tx=0x7f9b4c0046d0 comp rx=0 tx=0).stop 2024-05-03T18:39:29.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.790+0000 7f9b71f03b80 1 -- 172.21.15.33:0/2329410295 shutdown_connections 2024-05-03T18:39:29.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.790+0000 7f9b71f03b80 1 --2- 172.21.15.33:0/2329410295 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e0bd64d6b0 0x55e0bd66a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:29.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.790+0000 7f9b71f03b80 1 --2- 172.21.15.33:0/2329410295 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e0bd6449e0 0x55e0bd64b240 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:29.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.790+0000 7f9b71f03b80 1 --2- 172.21.15.33:0/2329410295 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e0bd649e20 0x55e0bd66a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:29.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.790+0000 7f9b71f03b80 1 -- 172.21.15.33:0/2329410295 >> 172.21.15.33:0/2329410295 conn(0x55e0bd5b8750 msgr2=0x55e0bd64ab30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:39:29.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.791+0000 7f9b71f03b80 1 -- 172.21.15.33:0/2329410295 shutdown_connections 2024-05-03T18:39:29.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.791+0000 7f9b71f03b80 1 -- 172.21.15.33:0/2329410295 wait complete. 2024-05-03T18:39:29.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.792+0000 7f9b71f03b80 1 -- start start 2024-05-03T18:39:29.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.792+0000 7f9b71f03b80 1 Processor -- start 2024-05-03T18:39:29.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:39:29 smithi099 ceph-mon[31658]: pgmap v383: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 7.5 MiB/s wr, 478 op/s 2024-05-03T18:39:29.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.793+0000 7f9b71f03b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e0bd6449e0 0x55e0bd65f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:39:29.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.793+0000 7f9b71f03b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e0bd649e20 0x55e0bd65d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:39:29.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.793+0000 7f9b71f03b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e0bd64d6b0 0x55e0bd65dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:39:29.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.793+0000 7f9b71f03b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9b4000da90 con 0x55e0bd649e20 2024-05-03T18:39:29.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.793+0000 7f9b71f03b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9b4000ef00 con 0x55e0bd64d6b0 2024-05-03T18:39:29.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.793+0000 7f9b71f03b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9b4000e160 con 0x55e0bd6449e0 2024-05-03T18:39:29.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.793+0000 7f9b71f03b80 10 client.0 _ll_register_callbacks cb 0x55e0bd539cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:39:29.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.794+0000 7f9b65600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e0bd64d6b0 0x55e0bd65dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:39:29.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.794+0000 7f9b65600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e0bd64d6b0 0x55e0bd65dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:46732/0 (socket says 172.21.15.33:46732) 2024-05-03T18:39:29.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.794+0000 7f9b65600700 1 -- 172.21.15.33:0/305673567 learned_addr learned my addr 172.21.15.33:0/305673567 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:39:29.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.795+0000 7f9b5fe00700 1 --2- 172.21.15.33:0/305673567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e0bd649e20 0x55e0bd65d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:39:29.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.795+0000 7f9b64c00700 1 --2- 172.21.15.33:0/305673567 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e0bd6449e0 0x55e0bd65f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:39:29.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.795+0000 7f9b65600700 1 -- 172.21.15.33:0/305673567 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e0bd6449e0 msgr2=0x55e0bd65f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:39:29.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.795+0000 7f9b65600700 1 --2- 172.21.15.33:0/305673567 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e0bd6449e0 0x55e0bd65f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:29.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.795+0000 7f9b65600700 1 -- 172.21.15.33:0/305673567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e0bd649e20 msgr2=0x55e0bd65d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:39:29.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.795+0000 7f9b65600700 1 --2- 172.21.15.33:0/305673567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e0bd649e20 0x55e0bd65d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:29.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.796+0000 7f9b65600700 1 -- 172.21.15.33:0/305673567 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0bd65e2c0 con 0x55e0bd64d6b0 2024-05-03T18:39:29.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.797+0000 7f9b65600700 1 --2- 172.21.15.33:0/305673567 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e0bd64d6b0 0x55e0bd65dc40 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9b58007cd0 tx=0x7f9b58007d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8d17e25c01f05d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:39:29.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.797+0000 7f9b46a00700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-03T18:39:29.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.798+0000 7f9b46a00700 1 -- 172.21.15.33:0/305673567 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b5801b070 con 0x55e0bd64d6b0 2024-05-03T18:39:29.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.798+0000 7f9b71f03b80 1 -- 172.21.15.33:0/305673567 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b4c023070 con 0x55e0bd64d6b0 2024-05-03T18:39:29.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.798+0000 7f9b46a00700 1 -- 172.21.15.33:0/305673567 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9518+0+0 (secure 0 0 0) 0x7f9b58004bb0 con 0x55e0bd64d6b0 2024-05-03T18:39:29.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.801+0000 7f9b46a00700 1 -- 172.21.15.33:0/305673567 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f9b58028050 con 0x55e0bd64d6b0 2024-05-03T18:39:29.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.801+0000 7f9b71f03b80 10 client.36230 fetch_fsmap learned FSMap version 21 2024-05-03T18:39:29.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.801+0000 7f9b71f03b80 1 -- 172.21.15.33:0/305673567 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55e0bd6cdc70 con 0x55e0bd64d6b0 2024-05-03T18:39:29.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.801+0000 7f9b46a00700 1 -- 172.21.15.33:0/305673567 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9b58003e10 con 0x55e0bd64d6b0 2024-05-03T18:39:29.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.802+0000 7f9b46a00700 1 -- 172.21.15.33:0/305673567 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f9b58004210 con 0x55e0bd64d6b0 2024-05-03T18:39:29.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.802+0000 7f9b71f03b80 10 client.36230 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:39:29.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.802+0000 7f9b71f03b80 10 client.36230 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:39:29.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.802+0000 7f9b71f03b80 20 client.36230 populate_metadata read hostname 'smithi033' 2024-05-03T18:39:29.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.802+0000 7f9b71f03b80 1 --2- 172.21.15.33:0/305673567 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55e0bd6ce3b0 0x55e0bd6d07d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:39:29.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.802+0000 7f9b71f03b80 4 client.36230 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-03T18:39:29.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.802+0000 7f9b71f03b80 1 -- 172.21.15.33:0/305673567 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f9b4c023070 con 0x55e0bd6ce3b0 2024-05-03T18:39:29.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.803+0000 7f9b64c00700 1 --2- 172.21.15.33:0/305673567 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55e0bd6ce3b0 0x55e0bd6d07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:39:29.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.805+0000 7f9b46a00700 10 client.36230 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:39:29.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.805+0000 7f9b64c00700 1 --2- 172.21.15.33:0/305673567 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55e0bd6ce3b0 0x55e0bd6d07d0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76793f4dca72f851 server_cookie=194a24273fef0583 in_seq=0 out_seq=0 2024-05-03T18:39:29.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.819+0000 7f9b46a00700 1 -- 172.21.15.33:0/305673567 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f9b4c023070 con 0x55e0bd6ce3b0 2024-05-03T18:39:29.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.819+0000 7f9b46a00700 10 client.36230 handle_command_reply: tid=0 2024-05-03T18:39:29.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.820+0000 7f9b71f03b80 1 client.36230 shutdown 2024-05-03T18:39:29.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.823+0000 7f9b71f03b80 1 -- 172.21.15.33:0/305673567 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e0bd64d6b0 msgr2=0x55e0bd65dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:39:29.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.823+0000 7f9b71f03b80 1 --2- 172.21.15.33:0/305673567 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e0bd64d6b0 0x55e0bd65dc40 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9b58007cd0 tx=0x7f9b58007d00 comp rx=0 tx=0).stop 2024-05-03T18:39:29.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.823+0000 7f9b71f03b80 1 -- 172.21.15.33:0/305673567 shutdown_connections 2024-05-03T18:39:29.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.823+0000 7f9b71f03b80 1 --2- 172.21.15.33:0/305673567 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55e0bd6ce3b0 0x55e0bd6d07d0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:29.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.823+0000 7f9b46a00700 0 client.36230 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:39:29.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.823+0000 7f9b71f03b80 1 --2- 172.21.15.33:0/305673567 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e0bd64d6b0 0x55e0bd65dc40 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:29.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.823+0000 7f9b71f03b80 1 --2- 172.21.15.33:0/305673567 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e0bd6449e0 0x55e0bd65f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:29.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.823+0000 7f9b71f03b80 1 --2- 172.21.15.33:0/305673567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e0bd649e20 0x55e0bd65d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:29.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.823+0000 7f9b71f03b80 1 -- 172.21.15.33:0/305673567 >> 172.21.15.33:0/305673567 conn(0x55e0bd5b8750 msgr2=0x55e0bd634ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:39:29.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.824+0000 7f9b71f03b80 1 -- 172.21.15.33:0/305673567 shutdown_connections 2024-05-03T18:39:29.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.824+0000 7f9b71f03b80 1 -- 172.21.15.33:0/305673567 wait complete. 2024-05-03T18:39:29.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.824+0000 7f9b71f03b80 20 client.36230 trim_cache size 0 max 16384 2024-05-03T18:39:29.826 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-03T18:39:29.827 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-05-03T18:39:29.827 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "27abcbbb-0c64-41c2-acde-d6a70f794d7f", 2024-05-03T18:39:29.827 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-05-03T18:39:29.827 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-03T18:39:29.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.825+0000 7f9b67600700 1 -- 172.21.15.33:0/1804504963 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f9b480b29f0 msgr2=0x7f9b480b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:39:29.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.826+0000 7f9b67600700 1 --2- 172.21.15.33:0/1804504963 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f9b480b29f0 0x7f9b480b4eb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c009bd0 tx=0x7f9b4c009260 comp rx=0 tx=0).stop 2024-05-03T18:39:29.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.826+0000 7f9b67600700 1 -- 172.21.15.33:0/1804504963 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b60109530 msgr2=0x7f9b6019a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:39:29.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.826+0000 7f9b67600700 1 --2- 172.21.15.33:0/1804504963 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b60109530 0x7f9b6019a1f0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f9b54000c00 tx=0x7f9b5400fa40 comp rx=0 tx=0).stop 2024-05-03T18:39:29.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.826+0000 7f9b67600700 1 -- 172.21.15.33:0/1804504963 shutdown_connections 2024-05-03T18:39:29.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.826+0000 7f9b67600700 1 --2- 172.21.15.33:0/1804504963 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b6019d4d0 0x7f9b6019d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:29.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.826+0000 7f9b67600700 1 --2- 172.21.15.33:0/1804504963 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f9b480b29f0 0x7f9b480b4eb0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:29.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.826+0000 7f9b67600700 1 --2- 172.21.15.33:0/1804504963 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b60103ee0 0x7f9b60199cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:29.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.826+0000 7f9b67600700 1 --2- 172.21.15.33:0/1804504963 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b60109530 0x7f9b6019a1f0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:29.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.826+0000 7f9b67600700 1 -- 172.21.15.33:0/1804504963 >> 172.21.15.33:0/1804504963 conn(0x7f9b6007a570 msgr2=0x7f9b60102490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:39:29.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.827+0000 7f9b67600700 1 -- 172.21.15.33:0/1804504963 shutdown_connections 2024-05-03T18:39:29.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:29.828+0000 7f9b67600700 1 -- 172.21.15.33:0/1804504963 wait complete. 2024-05-03T18:39:29.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:39:29 smithi150 ceph-mon[32492]: pgmap v383: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 7.5 MiB/s wr, 478 op/s 2024-05-03T18:39:30.111 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 495.0 s, 496.0 tps, lat 200.664 ms stddev 276.875 2024-05-03T18:39:30.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:39:30 smithi099 ceph-mon[31658]: scrub queued for path: / 2024-05-03T18:39:30.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:39:30 smithi099 ceph-mon[31658]: scrub summary: idle+waiting paths [/] 2024-05-03T18:39:30.797 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:39:30 smithi099 ceph-mon[31658]: scrub summary: active paths [/] 2024-05-03T18:39:30.797 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:39:30 smithi099 ceph-mon[31658]: scrub summary: idle+waiting paths [/] 2024-05-03T18:39:30.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:39:30 smithi150 ceph-mon[32492]: scrub queued for path: / 2024-05-03T18:39:30.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:39:30 smithi150 ceph-mon[32492]: scrub summary: idle+waiting paths [/] 2024-05-03T18:39:30.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:39:30 smithi150 ceph-mon[32492]: scrub summary: active paths [/] 2024-05-03T18:39:30.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:39:30 smithi150 ceph-mon[32492]: scrub summary: idle+waiting paths [/] 2024-05-03T18:39:30.904 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:39:30 smithi033 ceph-mon[28330]: scrub queued for path: / 2024-05-03T18:39:30.904 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:39:30 smithi033 ceph-mon[28330]: scrub summary: idle+waiting paths [/] 2024-05-03T18:39:30.904 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:39:30 smithi033 ceph-mon[28330]: scrub summary: active paths [/] 2024-05-03T18:39:30.904 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:39:30 smithi033 ceph-mon[28330]: scrub summary: idle+waiting paths [/] 2024-05-03T18:39:31.006 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-03T18:39:31.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:39:31 smithi099 ceph-mon[31658]: pgmap v384: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 11 MiB/s wr, 771 op/s 2024-05-03T18:39:31.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:39:31 smithi099 ceph-mon[31658]: scrub summary: idle 2024-05-03T18:39:31.863 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:39:31.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:39:31 smithi150 ceph-mon[32492]: pgmap v384: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 11 MiB/s wr, 771 op/s 2024-05-03T18:39:31.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:39:31 smithi150 ceph-mon[32492]: scrub summary: idle 2024-05-03T18:39:31.986 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:39:31 smithi033 ceph-mon[28330]: pgmap v384: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 11 MiB/s wr, 771 op/s 2024-05-03T18:39:31.986 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:39:31 smithi033 ceph-mon[28330]: scrub summary: idle 2024-05-03T18:39:33.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:39:33 smithi099 ceph-mon[31658]: pgmap v385: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 8.2 MiB/s wr, 541 op/s 2024-05-03T18:39:33.884 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:39:33 smithi150 ceph-mon[32492]: pgmap v385: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 8.2 MiB/s wr, 541 op/s 2024-05-03T18:39:33.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:39:33 smithi033 ceph-mon[28330]: pgmap v385: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 8.2 MiB/s wr, 541 op/s 2024-05-03T18:39:35.111 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 500.0 s, 583.9 tps, lat 143.059 ms stddev 77.963 2024-05-03T18:39:35.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:39:35 smithi099 ceph-mon[31658]: pgmap v386: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 12 MiB/s wr, 717 op/s 2024-05-03T18:39:35.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:39:35 smithi150 ceph-mon[32492]: pgmap v386: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 12 MiB/s wr, 717 op/s 2024-05-03T18:39:35.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:39:35 smithi033 ceph-mon[28330]: pgmap v386: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 12 MiB/s wr, 717 op/s 2024-05-03T18:39:37.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.179+0000 7f142a600700 1 -- 172.21.15.33:0/2840568406 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f142410b6e0 msgr2=0x7f142410bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:39:37.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.179+0000 7f142a600700 1 --2- 172.21.15.33:0/2840568406 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f142410b6e0 0x7f142410bac0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f1414009990 tx=0x7f1414009ca0 comp rx=0 tx=0).stop 2024-05-03T18:39:37.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.179+0000 7f142a600700 1 -- 172.21.15.33:0/2840568406 shutdown_connections 2024-05-03T18:39:37.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.179+0000 7f142a600700 1 --2- 172.21.15.33:0/2840568406 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f14241060a0 0x7f14241125b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:37.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.179+0000 7f142a600700 1 --2- 172.21.15.33:0/2840568406 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f142410b6e0 0x7f142410bac0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:37.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.179+0000 7f142a600700 1 --2- 172.21.15.33:0/2840568406 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f14241056e0 0x7f1424105b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:37.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.179+0000 7f142a600700 1 -- 172.21.15.33:0/2840568406 >> 172.21.15.33:0/2840568406 conn(0x7f14241011d0 msgr2=0x7f14241035f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:39:37.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.180+0000 7f142a600700 1 -- 172.21.15.33:0/2840568406 shutdown_connections 2024-05-03T18:39:37.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.180+0000 7f142a600700 1 -- 172.21.15.33:0/2840568406 wait complete. 2024-05-03T18:39:37.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.182+0000 7f142a600700 1 Processor -- start 2024-05-03T18:39:37.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.182+0000 7f142a600700 1 -- start start 2024-05-03T18:39:37.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.183+0000 7f142a600700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f14241056e0 0x7f14241a1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:39:37.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.183+0000 7f142a600700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14241060a0 0x7f14241a1e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:39:37.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.183+0000 7f142a600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f142410b6e0 0x7f142419bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:39:37.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.183+0000 7f142a600700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1424115810 con 0x7f14241060a0 2024-05-03T18:39:37.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.183+0000 7f142a600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1424115690 con 0x7f142410b6e0 2024-05-03T18:39:37.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.183+0000 7f142a600700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f1424115990 con 0x7f14241056e0 2024-05-03T18:39:37.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.183+0000 7f1423400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f14241056e0 0x7f14241a1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:39:37.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.184+0000 7f1423400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f14241056e0 0x7f14241a1910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:50158/0 (socket says 172.21.15.33:50158) 2024-05-03T18:39:37.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.184+0000 7f1423400700 1 -- 172.21.15.33:0/2263850305 learned_addr learned my addr 172.21.15.33:0/2263850305 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:39:37.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.184+0000 7f1422a00700 1 --2- 172.21.15.33:0/2263850305 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14241060a0 0x7f14241a1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:39:37.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.184+0000 7f1423e00700 1 --2- 172.21.15.33:0/2263850305 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f142410b6e0 0x7f142419bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:39:37.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.184+0000 7f1423400700 1 -- 172.21.15.33:0/2263850305 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f142410b6e0 msgr2=0x7f142419bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:39:37.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.184+0000 7f1423400700 1 --2- 172.21.15.33:0/2263850305 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f142410b6e0 0x7f142419bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:37.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.184+0000 7f1423400700 1 -- 172.21.15.33:0/2263850305 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14241060a0 msgr2=0x7f14241a1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:39:37.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.185+0000 7f1423400700 1 --2- 172.21.15.33:0/2263850305 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14241060a0 0x7f14241a1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:37.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.185+0000 7f1423400700 1 -- 172.21.15.33:0/2263850305 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f142419c230 con 0x7f14241056e0 2024-05-03T18:39:37.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.185+0000 7f1422a00700 1 --2- 172.21.15.33:0/2263850305 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14241060a0 0x7f14241a1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:39:37.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.186+0000 7f1423400700 1 --2- 172.21.15.33:0/2263850305 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f14241056e0 0x7f14241a1910 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f14140098e0 tx=0x7f14140057c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:39:37.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.187+0000 7f141be00700 1 -- 172.21.15.33:0/2263850305 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1414026070 con 0x7f14241056e0 2024-05-03T18:39:37.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.187+0000 7f142a600700 1 -- 172.21.15.33:0/2263850305 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f142419c4c0 con 0x7f14241056e0 2024-05-03T18:39:37.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.187+0000 7f142a600700 1 -- 172.21.15.33:0/2263850305 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14240778e0 con 0x7f14241056e0 2024-05-03T18:39:37.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.187+0000 7f141be00700 1 -- 172.21.15.33:0/2263850305 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f141400fb00 con 0x7f14241056e0 2024-05-03T18:39:37.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.187+0000 7f141be00700 1 -- 172.21.15.33:0/2263850305 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1414021bc0 con 0x7f14241056e0 2024-05-03T18:39:37.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.193+0000 7f141be00700 1 -- 172.21.15.33:0/2263850305 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f1414029070 con 0x7f14241056e0 2024-05-03T18:39:37.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.195+0000 7f141be00700 1 --2- 172.21.15.33:0/2263850305 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f14080b2a20 0x7f14080b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:39:37.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.196+0000 7f141be00700 1 -- 172.21.15.33:0/2263850305 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9518+0+0 (secure 0 0 0) 0x7f14140e36f0 con 0x7f14241056e0 2024-05-03T18:39:37.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.196+0000 7f1422a00700 1 --2- 172.21.15.33:0/2263850305 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f14080b2a20 0x7f14080b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:39:37.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.198+0000 7f1422a00700 1 --2- 172.21.15.33:0/2263850305 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f14080b2a20 0x7f14080b4ee0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1410005fd0 tx=0x7f1410005ee0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:39:37.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.215+0000 7f143506ab80 1 Processor -- start 2024-05-03T18:39:37.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.216+0000 7f143506ab80 1 -- start start 2024-05-03T18:39:37.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.217+0000 7f143506ab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55834a4227b0 0x55834a422b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:39:37.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.217+0000 7f143506ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55834a4230d0 0x55834a41dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:39:37.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.218+0000 7f143506ab80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55834a41e5d0 0x55834a41ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:39:37.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.218+0000 7f143506ab80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55834a370a20 con 0x55834a41e5d0 2024-05-03T18:39:37.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.218+0000 7f143506ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55834a30e760 con 0x55834a4230d0 2024-05-03T18:39:37.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.218+0000 7f143506ab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55834a36c9e0 con 0x55834a4227b0 2024-05-03T18:39:37.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.218+0000 7f1423e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55834a41e5d0 0x55834a41ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:39:37.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.218+0000 7f1422a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55834a4230d0 0x55834a41dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:39:37.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.218+0000 7f1423400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55834a4227b0 0x55834a422b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:39:37.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.218+0000 7f1423400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55834a4227b0 0x55834a422b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:50168/0 (socket says 172.21.15.33:50168) 2024-05-03T18:39:37.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.218+0000 7f1423400700 1 -- 172.21.15.33:0/1183905058 learned_addr learned my addr 172.21.15.33:0/1183905058 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:39:37.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.219+0000 7f1423400700 1 -- 172.21.15.33:0/1183905058 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55834a4230d0 msgr2=0x55834a41dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:39:37.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.219+0000 7f1423400700 1 --2- 172.21.15.33:0/1183905058 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55834a4230d0 0x55834a41dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:37.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.219+0000 7f1423400700 1 -- 172.21.15.33:0/1183905058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55834a41e5d0 msgr2=0x55834a41ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:39:37.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.220+0000 7f1423400700 1 --2- 172.21.15.33:0/1183905058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55834a41e5d0 0x55834a41ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:37.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.220+0000 7f1423400700 1 -- 172.21.15.33:0/1183905058 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55834a2bd8d0 con 0x55834a4227b0 2024-05-03T18:39:37.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.220+0000 7f1423e00700 1 --2- 172.21.15.33:0/1183905058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55834a41e5d0 0x55834a41ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:39:37.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.222+0000 7f1423400700 1 --2- 172.21.15.33:0/1183905058 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55834a4227b0 0x55834a422b90 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f14140aa380 tx=0x7f14140a9e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c01636f0648c06e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:39:37.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.222+0000 7f1418c00700 1 -- 172.21.15.33:0/1183905058 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14140aad80 con 0x55834a4227b0 2024-05-03T18:39:37.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.223+0000 7f1418c00700 1 -- 172.21.15.33:0/1183905058 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f14140aa680 con 0x55834a4227b0 2024-05-03T18:39:37.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.223+0000 7f1418c00700 1 -- 172.21.15.33:0/1183905058 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14140b2fc0 con 0x55834a4227b0 2024-05-03T18:39:37.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.224+0000 7f143506ab80 1 -- 172.21.15.33:0/1183905058 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55834a4227b0 msgr2=0x55834a422b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:39:37.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.224+0000 7f143506ab80 1 --2- 172.21.15.33:0/1183905058 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55834a4227b0 0x55834a422b90 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f14140aa380 tx=0x7f14140a9e70 comp rx=0 tx=0).stop 2024-05-03T18:39:37.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.224+0000 7f143506ab80 1 -- 172.21.15.33:0/1183905058 shutdown_connections 2024-05-03T18:39:37.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.224+0000 7f143506ab80 1 --2- 172.21.15.33:0/1183905058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55834a41e5d0 0x55834a41ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:37.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.224+0000 7f143506ab80 1 --2- 172.21.15.33:0/1183905058 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55834a4227b0 0x55834a422b90 secure :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f14140aa380 tx=0x7f14140a9e70 comp rx=0 tx=0).stop 2024-05-03T18:39:37.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.224+0000 7f143506ab80 1 --2- 172.21.15.33:0/1183905058 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55834a4230d0 0x55834a41dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:37.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.224+0000 7f143506ab80 1 -- 172.21.15.33:0/1183905058 >> 172.21.15.33:0/1183905058 conn(0x55834a34cb60 msgr2=0x55834a4187f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:39:37.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.224+0000 7f143506ab80 1 -- 172.21.15.33:0/1183905058 shutdown_connections 2024-05-03T18:39:37.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.225+0000 7f143506ab80 1 -- 172.21.15.33:0/1183905058 wait complete. 2024-05-03T18:39:37.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.226+0000 7f143506ab80 1 -- start start 2024-05-03T18:39:37.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.226+0000 7f143506ab80 1 Processor -- start 2024-05-03T18:39:37.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.227+0000 7f143506ab80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55834a41e5d0 0x55834a42a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:39:37.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.227+0000 7f143506ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55834a4230d0 0x55834a427490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:39:37.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.227+0000 7f143506ab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55834a4279d0 0x55834a427e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:39:37.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.227+0000 7f143506ab80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55834a42aa60 con 0x55834a41e5d0 2024-05-03T18:39:37.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.227+0000 7f143506ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55834a2d6b20 con 0x55834a4230d0 2024-05-03T18:39:37.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.227+0000 7f143506ab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55834a42abe0 con 0x55834a4279d0 2024-05-03T18:39:37.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.227+0000 7f143506ab80 10 client.0 _ll_register_callbacks cb 0x55834a317c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:39:37.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.227+0000 7f1423400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55834a41e5d0 0x55834a42a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:39:37.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.227+0000 7f1423400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55834a41e5d0 0x55834a42a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54460/0 (socket says 172.21.15.33:54460) 2024-05-03T18:39:37.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.227+0000 7f1423400700 1 -- 172.21.15.33:0/2764259979 learned_addr learned my addr 172.21.15.33:0/2764259979 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:39:37.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.227+0000 7f1423e00700 1 --2- 172.21.15.33:0/2764259979 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55834a4279d0 0x55834a427e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:39:37.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.228+0000 7f1422a00700 1 --2- 172.21.15.33:0/2764259979 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55834a4230d0 0x55834a427490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:39:37.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.229+0000 7f1423400700 1 -- 172.21.15.33:0/2764259979 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55834a4279d0 msgr2=0x55834a427e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:39:37.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.229+0000 7f1423400700 1 --2- 172.21.15.33:0/2764259979 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55834a4279d0 0x55834a427e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:37.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.229+0000 7f1423400700 1 -- 172.21.15.33:0/2764259979 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55834a4230d0 msgr2=0x55834a427490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:39:37.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.229+0000 7f1423400700 1 --2- 172.21.15.33:0/2764259979 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55834a4230d0 0x55834a427490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:37.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.229+0000 7f1423400700 1 -- 172.21.15.33:0/2764259979 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55834a336420 con 0x55834a41e5d0 2024-05-03T18:39:37.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.230+0000 7f1423400700 1 --2- 172.21.15.33:0/2764259979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55834a41e5d0 0x55834a42a190 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f14140a6700 tx=0x7f14140aaaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b0925b19ef06fd8 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:39:37.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.230+0000 7f140d600700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:39:37.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.232+0000 7f140d600700 1 -- 172.21.15.33:0/2764259979 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14140aea30 con 0x55834a41e5d0 2024-05-03T18:39:37.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.232+0000 7f140d600700 1 -- 172.21.15.33:0/2764259979 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9518+0+0 (secure 0 0 0) 0x7f14140b31d0 con 0x55834a41e5d0 2024-05-03T18:39:37.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.232+0000 7f143506ab80 1 -- 172.21.15.33:0/2764259979 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14140aa680 con 0x55834a41e5d0 2024-05-03T18:39:37.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.233+0000 7f140d600700 1 -- 172.21.15.33:0/2764259979 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f14140b3450 con 0x55834a41e5d0 2024-05-03T18:39:37.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.233+0000 7f143506ab80 10 client.16536 fetch_fsmap learned FSMap version 21 2024-05-03T18:39:37.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.233+0000 7f143506ab80 1 -- 172.21.15.33:0/2764259979 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55834a42aa60 con 0x55834a41e5d0 2024-05-03T18:39:37.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.233+0000 7f140d600700 1 -- 172.21.15.33:0/2764259979 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f14140aa680 con 0x55834a41e5d0 2024-05-03T18:39:37.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.234+0000 7f140d600700 1 -- 172.21.15.33:0/2764259979 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f1414090ba0 con 0x55834a41e5d0 2024-05-03T18:39:37.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.234+0000 7f143506ab80 10 client.16536 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:39:37.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.234+0000 7f143506ab80 10 client.16536 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:39:37.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.234+0000 7f143506ab80 20 client.16536 populate_metadata read hostname 'smithi033' 2024-05-03T18:39:37.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.234+0000 7f143506ab80 1 --2- 172.21.15.33:0/2764259979 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55834a448b40 0x55834a44af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:39:37.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.234+0000 7f143506ab80 4 client.16536 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:39:37.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.234+0000 7f143506ab80 1 -- 172.21.15.33:0/2764259979 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55834a3eb1d0 con 0x55834a448b40 2024-05-03T18:39:37.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.235+0000 7f1422a00700 1 --2- 172.21.15.33:0/2764259979 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55834a448b40 0x55834a44af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:39:37.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.236+0000 7f1422a00700 1 --2- 172.21.15.33:0/2764259979 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55834a448b40 0x55834a44af60 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98cf27cfad435a6a server_cookie=6bd76b28e81cabb7 in_seq=0 out_seq=0 2024-05-03T18:39:37.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.236+0000 7f140d600700 10 client.16536 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:39:37.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.238+0000 7f140d600700 1 -- 172.21.15.33:0/2764259979 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55834a3eb1d0 con 0x55834a448b40 2024-05-03T18:39:37.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.238+0000 7f140d600700 10 client.16536 handle_command_reply: tid=0 2024-05-03T18:39:37.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.239+0000 7f143506ab80 1 client.16536 shutdown 2024-05-03T18:39:37.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.239+0000 7f143506ab80 1 -- 172.21.15.33:0/2764259979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55834a41e5d0 msgr2=0x55834a42a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:39:37.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.239+0000 7f143506ab80 1 --2- 172.21.15.33:0/2764259979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55834a41e5d0 0x55834a42a190 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f14140a6700 tx=0x7f14140aaaf0 comp rx=0 tx=0).stop 2024-05-03T18:39:37.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.240+0000 7f143506ab80 1 -- 172.21.15.33:0/2764259979 shutdown_connections 2024-05-03T18:39:37.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.240+0000 7f143506ab80 1 --2- 172.21.15.33:0/2764259979 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55834a448b40 0x55834a44af60 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:37.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.240+0000 7f143506ab80 1 --2- 172.21.15.33:0/2764259979 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55834a4279d0 0x55834a427e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:37.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.240+0000 7f143506ab80 1 --2- 172.21.15.33:0/2764259979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55834a41e5d0 0x55834a42a190 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:37.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.240+0000 7f143506ab80 1 --2- 172.21.15.33:0/2764259979 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55834a4230d0 0x55834a427490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:37.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.240+0000 7f143506ab80 1 -- 172.21.15.33:0/2764259979 >> 172.21.15.33:0/2764259979 conn(0x55834a34cb60 msgr2=0x55834a423e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:39:37.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.240+0000 7f140d600700 0 client.16536 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:39:37.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.240+0000 7f143506ab80 1 -- 172.21.15.33:0/2764259979 shutdown_connections 2024-05-03T18:39:37.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.240+0000 7f143506ab80 1 -- 172.21.15.33:0/2764259979 wait complete. 2024-05-03T18:39:37.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.240+0000 7f143506ab80 20 client.16536 trim_cache size 0 max 16384 2024-05-03T18:39:37.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.276+0000 7f143506ab80 1 Processor -- start 2024-05-03T18:39:37.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.277+0000 7f143506ab80 1 -- start start 2024-05-03T18:39:37.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.277+0000 7f143506ab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55834a41e5d0 0x55834a41dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:39:37.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.277+0000 7f143506ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55834a4230d0 0x55834a418660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:39:37.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.277+0000 7f143506ab80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55834a4279d0 0x55834a4293d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:39:37.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.277+0000 7f143506ab80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f142406f8f0 con 0x55834a4279d0 2024-05-03T18:39:37.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.277+0000 7f143506ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1424072150 con 0x55834a4230d0 2024-05-03T18:39:37.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.277+0000 7f143506ab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f142406ff40 con 0x55834a41e5d0 2024-05-03T18:39:37.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.278+0000 7f1422a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55834a4230d0 0x55834a418660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:39:37.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.278+0000 7f1423e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55834a4279d0 0x55834a4293d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:39:37.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.278+0000 7f1423400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55834a41e5d0 0x55834a41dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:39:37.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.278+0000 7f1422a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55834a4230d0 0x55834a418660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:46100/0 (socket says 172.21.15.33:46100) 2024-05-03T18:39:37.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.278+0000 7f1422a00700 1 -- 172.21.15.33:0/3761257998 learned_addr learned my addr 172.21.15.33:0/3761257998 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:39:37.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.278+0000 7f1422a00700 1 -- 172.21.15.33:0/3761257998 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55834a41e5d0 msgr2=0x55834a41dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:39:37.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.278+0000 7f1422a00700 1 --2- 172.21.15.33:0/3761257998 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55834a41e5d0 0x55834a41dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:37.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.279+0000 7f1422a00700 1 -- 172.21.15.33:0/3761257998 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55834a4279d0 msgr2=0x55834a4293d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:39:37.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.279+0000 7f1422a00700 1 --2- 172.21.15.33:0/3761257998 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55834a4279d0 0x55834a4293d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:37.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.279+0000 7f1422a00700 1 -- 172.21.15.33:0/3761257998 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55834a419250 con 0x55834a4230d0 2024-05-03T18:39:37.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.279+0000 7f1423e00700 1 --2- 172.21.15.33:0/3761257998 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55834a4279d0 0x55834a4293d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:39:37.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.279+0000 7f1422a00700 1 --2- 172.21.15.33:0/3761257998 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55834a4230d0 0x55834a418660 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x55834a428a60 tx=0x7f1410014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c2bb7d443b6a88a server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:39:37.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.280+0000 7f140fe00700 1 -- 172.21.15.33:0/3761257998 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1410004020 con 0x55834a4230d0 2024-05-03T18:39:37.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.280+0000 7f140fe00700 1 -- 172.21.15.33:0/3761257998 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1410017040 con 0x55834a4230d0 2024-05-03T18:39:37.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.280+0000 7f140fe00700 1 -- 172.21.15.33:0/3761257998 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1410004c50 con 0x55834a4230d0 2024-05-03T18:39:37.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.281+0000 7f143506ab80 1 -- 172.21.15.33:0/3761257998 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55834a4230d0 msgr2=0x55834a418660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:39:37.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.281+0000 7f143506ab80 1 --2- 172.21.15.33:0/3761257998 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55834a4230d0 0x55834a418660 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x55834a428a60 tx=0x7f1410014000 comp rx=0 tx=0).stop 2024-05-03T18:39:37.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.282+0000 7f143506ab80 1 -- 172.21.15.33:0/3761257998 shutdown_connections 2024-05-03T18:39:37.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.282+0000 7f143506ab80 1 --2- 172.21.15.33:0/3761257998 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55834a4279d0 0x55834a4293d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:37.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.282+0000 7f143506ab80 1 --2- 172.21.15.33:0/3761257998 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55834a41e5d0 0x55834a41dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:37.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.282+0000 7f143506ab80 1 --2- 172.21.15.33:0/3761257998 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55834a4230d0 0x55834a418660 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:37.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.282+0000 7f143506ab80 1 -- 172.21.15.33:0/3761257998 >> 172.21.15.33:0/3761257998 conn(0x55834a34cb60 msgr2=0x55834a428cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:39:37.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.282+0000 7f143506ab80 1 -- 172.21.15.33:0/3761257998 shutdown_connections 2024-05-03T18:39:37.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.282+0000 7f143506ab80 1 -- 172.21.15.33:0/3761257998 wait complete. 2024-05-03T18:39:37.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.283+0000 7f143506ab80 1 -- start start 2024-05-03T18:39:37.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.283+0000 7f143506ab80 1 Processor -- start 2024-05-03T18:39:37.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.284+0000 7f143506ab80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55834a41e5d0 0x55834a450270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:39:37.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.284+0000 7f143506ab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55834a4230d0 0x55834a4507b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:39:37.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.284+0000 7f143506ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55834a4279d0 0x55834a450cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:39:37.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.284+0000 7f143506ab80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1424070610 con 0x55834a41e5d0 2024-05-03T18:39:37.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.284+0000 7f143506ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1424071a80 con 0x55834a4279d0 2024-05-03T18:39:37.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.284+0000 7f143506ab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f1424070ce0 con 0x55834a4230d0 2024-05-03T18:39:37.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.284+0000 7f143506ab80 10 client.0 _ll_register_callbacks cb 0x55834a317c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:39:37.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.284+0000 7f1423400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55834a41e5d0 0x55834a450270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:39:37.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.284+0000 7f1423400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55834a41e5d0 0x55834a450270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54470/0 (socket says 172.21.15.33:54470) 2024-05-03T18:39:37.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.284+0000 7f1423400700 1 -- 172.21.15.33:0/843087187 learned_addr learned my addr 172.21.15.33:0/843087187 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:39:37.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.285+0000 7f1423e00700 1 --2- 172.21.15.33:0/843087187 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55834a4279d0 0x55834a450cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:39:37.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.285+0000 7f1422a00700 1 --2- 172.21.15.33:0/843087187 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55834a4230d0 0x55834a4507b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:39:37.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.285+0000 7f1423400700 1 -- 172.21.15.33:0/843087187 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55834a4230d0 msgr2=0x55834a4507b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:39:37.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.285+0000 7f1423400700 1 --2- 172.21.15.33:0/843087187 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55834a4230d0 0x55834a4507b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:37.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.285+0000 7f1423400700 1 -- 172.21.15.33:0/843087187 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55834a4279d0 msgr2=0x55834a450cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:39:37.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.285+0000 7f1423400700 1 --2- 172.21.15.33:0/843087187 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55834a4279d0 0x55834a450cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:37.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.285+0000 7f1423400700 1 -- 172.21.15.33:0/843087187 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55834a422400 con 0x55834a41e5d0 2024-05-03T18:39:37.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.286+0000 7f1423400700 1 --2- 172.21.15.33:0/843087187 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55834a41e5d0 0x55834a450270 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f14140e3590 tx=0x7f14140095f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3be26a52abdbf5b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:39:37.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.286+0000 7f140d600700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:39:37.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.287+0000 7f140d600700 1 -- 172.21.15.33:0/843087187 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14140aad80 con 0x55834a41e5d0 2024-05-03T18:39:37.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.287+0000 7f143506ab80 1 -- 172.21.15.33:0/843087187 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1410017040 con 0x55834a41e5d0 2024-05-03T18:39:37.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.287+0000 7f140d600700 1 -- 172.21.15.33:0/843087187 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9518+0+0 (secure 0 0 0) 0x7f14140afbc0 con 0x55834a41e5d0 2024-05-03T18:39:37.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.288+0000 7f140d600700 1 -- 172.21.15.33:0/843087187 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f14140aea30 con 0x55834a41e5d0 2024-05-03T18:39:37.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.288+0000 7f143506ab80 10 client.16542 fetch_fsmap learned FSMap version 21 2024-05-03T18:39:37.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.288+0000 7f143506ab80 1 -- 172.21.15.33:0/843087187 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55834a422400 con 0x55834a41e5d0 2024-05-03T18:39:37.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.289+0000 7f140d600700 1 -- 172.21.15.33:0/843087187 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1410017040 con 0x55834a41e5d0 2024-05-03T18:39:37.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.289+0000 7f140d600700 1 -- 172.21.15.33:0/843087187 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f14140a98e0 con 0x55834a41e5d0 2024-05-03T18:39:37.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.289+0000 7f143506ab80 10 client.16542 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:39:37.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.289+0000 7f143506ab80 10 client.16542 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:39:37.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.289+0000 7f143506ab80 20 client.16542 populate_metadata read hostname 'smithi033' 2024-05-03T18:39:37.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.289+0000 7f143506ab80 1 --2- 172.21.15.33:0/843087187 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55834a4b3a70 0x55834a4b5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:39:37.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.289+0000 7f143506ab80 4 client.16542 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-03T18:39:37.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.289+0000 7f143506ab80 1 -- 172.21.15.33:0/843087187 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55834a37e400 con 0x55834a4b3a70 2024-05-03T18:39:37.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.290+0000 7f1422a00700 1 --2- 172.21.15.33:0/843087187 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55834a4b3a70 0x55834a4b5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:39:37.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.292+0000 7f1422a00700 1 --2- 172.21.15.33:0/843087187 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55834a4b3a70 0x55834a4b5e90 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe7585b7b169a656 server_cookie=e32fe1d52c6b01be in_seq=0 out_seq=0 2024-05-03T18:39:37.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.292+0000 7f140d600700 10 client.16542 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:39:37.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.292+0000 7f140d600700 1 -- 172.21.15.33:0/843087187 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55834a37e400 con 0x55834a4b3a70 2024-05-03T18:39:37.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.292+0000 7f140d600700 10 client.16542 handle_command_reply: tid=0 2024-05-03T18:39:37.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.293+0000 7f143506ab80 1 client.16542 shutdown 2024-05-03T18:39:37.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.293+0000 7f143506ab80 1 -- 172.21.15.33:0/843087187 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55834a41e5d0 msgr2=0x55834a450270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:39:37.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.293+0000 7f143506ab80 1 --2- 172.21.15.33:0/843087187 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55834a41e5d0 0x55834a450270 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f14140e3590 tx=0x7f14140095f0 comp rx=0 tx=0).stop 2024-05-03T18:39:37.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.293+0000 7f143506ab80 1 -- 172.21.15.33:0/843087187 shutdown_connections 2024-05-03T18:39:37.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.293+0000 7f143506ab80 1 --2- 172.21.15.33:0/843087187 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55834a4b3a70 0x55834a4b5e90 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:37.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.293+0000 7f140d600700 0 client.16542 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:39:37.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.293+0000 7f143506ab80 1 --2- 172.21.15.33:0/843087187 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55834a4279d0 0x55834a450cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:37.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.293+0000 7f143506ab80 1 --2- 172.21.15.33:0/843087187 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55834a41e5d0 0x55834a450270 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:37.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.293+0000 7f143506ab80 1 --2- 172.21.15.33:0/843087187 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55834a4230d0 0x55834a4507b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:37.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.293+0000 7f143506ab80 1 -- 172.21.15.33:0/843087187 >> 172.21.15.33:0/843087187 conn(0x55834a34cb60 msgr2=0x55834a421c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:39:37.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.294+0000 7f143506ab80 1 -- 172.21.15.33:0/843087187 shutdown_connections 2024-05-03T18:39:37.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.294+0000 7f143506ab80 1 -- 172.21.15.33:0/843087187 wait complete. 2024-05-03T18:39:37.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.294+0000 7f143506ab80 20 client.16542 trim_cache size 0 max 16384 2024-05-03T18:39:37.297 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-03T18:39:37.297 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-05-03T18:39:37.297 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-05-03T18:39:37.297 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-03T18:39:37.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.296+0000 7f142a600700 1 -- 172.21.15.33:0/2263850305 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f14080b2a20 msgr2=0x7f14080b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:39:37.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.296+0000 7f142a600700 1 --2- 172.21.15.33:0/2263850305 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f14080b2a20 0x7f14080b4ee0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1410005fd0 tx=0x7f1410005ee0 comp rx=0 tx=0).stop 2024-05-03T18:39:37.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.296+0000 7f142a600700 1 -- 172.21.15.33:0/2263850305 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f14241056e0 msgr2=0x7f14241a1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:39:37.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.296+0000 7f142a600700 1 --2- 172.21.15.33:0/2263850305 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f14241056e0 0x7f14241a1910 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f14140098e0 tx=0x7f14140057c0 comp rx=0 tx=0).stop 2024-05-03T18:39:37.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.296+0000 7f142a600700 1 -- 172.21.15.33:0/2263850305 shutdown_connections 2024-05-03T18:39:37.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.296+0000 7f142a600700 1 --2- 172.21.15.33:0/2263850305 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f14080b2a20 0x7f14080b4ee0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:37.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.296+0000 7f142a600700 1 --2- 172.21.15.33:0/2263850305 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f142410b6e0 0x7f142419bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:37.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.296+0000 7f142a600700 1 --2- 172.21.15.33:0/2263850305 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f14241056e0 0x7f14241a1910 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:37.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.296+0000 7f142a600700 1 --2- 172.21.15.33:0/2263850305 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14241060a0 0x7f14241a1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:37.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.296+0000 7f142a600700 1 -- 172.21.15.33:0/2263850305 >> 172.21.15.33:0/2263850305 conn(0x7f14241011d0 msgr2=0x7f14241019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:39:37.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.296+0000 7f142a600700 1 -- 172.21.15.33:0/2263850305 shutdown_connections 2024-05-03T18:39:37.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:37.297+0000 7f142a600700 1 -- 172.21.15.33:0/2263850305 wait complete. 2024-05-03T18:39:37.773 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:39:37 smithi033 ceph-mon[28330]: pgmap v387: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 7.5 MiB/s wr, 468 op/s 2024-05-03T18:39:37.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:39:37 smithi099 ceph-mon[31658]: pgmap v387: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 7.5 MiB/s wr, 468 op/s 2024-05-03T18:39:37.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:39:37 smithi150 ceph-mon[32492]: pgmap v387: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 7.5 MiB/s wr, 468 op/s 2024-05-03T18:39:38.176 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-03T18:39:38.176 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T18:39:38.938 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:39:39.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:39:39 smithi099 ceph-mon[31658]: pgmap v388: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 7.5 MiB/s wr, 469 op/s 2024-05-03T18:39:39.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:39:39 smithi150 ceph-mon[32492]: pgmap v388: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 7.5 MiB/s wr, 469 op/s 2024-05-03T18:39:39.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:39:39 smithi033 ceph-mon[28330]: pgmap v388: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 7.5 MiB/s wr, 469 op/s 2024-05-03T18:39:40.110 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 505.0 s, 659.9 tps, lat 177.383 ms stddev 247.809 2024-05-03T18:39:41.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:39:41 smithi099 ceph-mon[31658]: pgmap v389: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 12 MiB/s wr, 696 op/s 2024-05-03T18:39:41.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:39:41 smithi150 ceph-mon[32492]: pgmap v389: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 12 MiB/s wr, 696 op/s 2024-05-03T18:39:41.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:39:41 smithi033 ceph-mon[28330]: pgmap v389: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 12 MiB/s wr, 696 op/s 2024-05-03T18:39:43.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:39:43 smithi150 ceph-mon[32492]: pgmap v390: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 7.9 MiB/s wr, 402 op/s 2024-05-03T18:39:43.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:39:43 smithi033 ceph-mon[28330]: pgmap v390: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 7.9 MiB/s wr, 402 op/s 2024-05-03T18:39:44.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:39:43 smithi099 ceph-mon[31658]: pgmap v390: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 7.9 MiB/s wr, 402 op/s 2024-05-03T18:39:44.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:44.544+0000 7f9992a00700 1 -- 172.21.15.33:0/3366886059 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f998c06bb40 msgr2=0x7f998c110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:39:44.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:44.544+0000 7f9992a00700 1 --2- 172.21.15.33:0/3366886059 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f998c06bb40 0x7f998c110330 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f9984009f90 tx=0x7f998400b330 comp rx=0 tx=0).stop 2024-05-03T18:39:44.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:44.545+0000 7f9992a00700 1 -- 172.21.15.33:0/3366886059 shutdown_connections 2024-05-03T18:39:44.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:44.545+0000 7f9992a00700 1 --2- 172.21.15.33:0/3366886059 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f998c06bb40 0x7f998c110330 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:44.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:44.545+0000 7f9992a00700 1 --2- 172.21.15.33:0/3366886059 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f998c109510 0x7f998c1098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:44.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:44.545+0000 7f9992a00700 1 --2- 172.21.15.33:0/3366886059 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f998c06b180 0x7f998c06b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:44.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:44.545+0000 7f9992a00700 1 -- 172.21.15.33:0/3366886059 >> 172.21.15.33:0/3366886059 conn(0x7f998c079350 msgr2=0x7f998c079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:39:44.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:44.546+0000 7f9992a00700 1 -- 172.21.15.33:0/3366886059 shutdown_connections 2024-05-03T18:39:44.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:44.546+0000 7f9992a00700 1 -- 172.21.15.33:0/3366886059 wait complete. 2024-05-03T18:39:44.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:44.547+0000 7f9992a00700 1 Processor -- start 2024-05-03T18:39:44.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:44.548+0000 7f9992a00700 1 -- start start 2024-05-03T18:39:44.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:44.548+0000 7f9992a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f998c06b180 0x7f998c19f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:39:44.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:44.549+0000 7f9992a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f998c06bb40 0x7f998c19fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:39:44.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:44.549+0000 7f9992a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f998c109510 0x7f998c199920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:39:44.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:44.549+0000 7f9992a00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f998c1135e0 con 0x7f998c109510 2024-05-03T18:39:44.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:44.549+0000 7f9992a00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f998c113460 con 0x7f998c06b180 2024-05-03T18:39:44.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:44.549+0000 7f9992a00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f998c113760 con 0x7f998c06bb40 2024-05-03T18:39:44.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:44.549+0000 7f998be00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f998c06b180 0x7f998c19f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:39:44.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:44.549+0000 7f998b400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f998c06bb40 0x7f998c19fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:39:44.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:44.549+0000 7f998be00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f998c06b180 0x7f998c19f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:46124/0 (socket says 172.21.15.33:46124) 2024-05-03T18:39:44.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:44.549+0000 7f998be00700 1 -- 172.21.15.33:0/820852791 learned_addr learned my addr 172.21.15.33:0/820852791 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:39:44.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:44.549+0000 7f9990a00700 1 --2- 172.21.15.33:0/820852791 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f998c109510 0x7f998c199920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:39:44.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:44.550+0000 7f998b400700 1 -- 172.21.15.33:0/820852791 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f998c06b180 msgr2=0x7f998c19f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:39:44.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:44.550+0000 7f998b400700 1 --2- 172.21.15.33:0/820852791 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f998c06b180 0x7f998c19f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:44.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:44.550+0000 7f998b400700 1 -- 172.21.15.33:0/820852791 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f998c109510 msgr2=0x7f998c199920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:39:44.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:44.550+0000 7f998b400700 1 --2- 172.21.15.33:0/820852791 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f998c109510 0x7f998c199920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:44.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:44.550+0000 7f998b400700 1 -- 172.21.15.33:0/820852791 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f998c19a080 con 0x7f998c06bb40 2024-05-03T18:39:44.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:44.551+0000 7f998b400700 1 --2- 172.21.15.33:0/820852791 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f998c06bb40 0x7f998c19fca0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9978006fd0 tx=0x7f997800b730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:39:44.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:44.551+0000 7f9988c00700 1 -- 172.21.15.33:0/820852791 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9978015020 con 0x7f998c06bb40 2024-05-03T18:39:44.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:44.551+0000 7f9988c00700 1 -- 172.21.15.33:0/820852791 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9978011cc0 con 0x7f998c06bb40 2024-05-03T18:39:44.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:44.551+0000 7f9988c00700 1 -- 172.21.15.33:0/820852791 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9978010bd0 con 0x7f998c06bb40 2024-05-03T18:39:44.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:44.552+0000 7f9992a00700 1 -- 172.21.15.33:0/820852791 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f998c19a370 con 0x7f998c06bb40 2024-05-03T18:39:44.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:44.552+0000 7f9992a00700 1 -- 172.21.15.33:0/820852791 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f998c1023a0 con 0x7f998c06bb40 2024-05-03T18:39:44.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:44.554+0000 7f9992a00700 1 -- 172.21.15.33:0/820852791 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f998c10b9f0 con 0x7f998c06bb40 2024-05-03T18:39:44.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:44.559+0000 7f9988c00700 1 -- 172.21.15.33:0/820852791 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f9978018020 con 0x7f998c06bb40 2024-05-03T18:39:44.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:44.562+0000 7f9988c00700 1 --2- 172.21.15.33:0/820852791 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f99700b6e80 0x7f99700b9340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:39:44.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:44.562+0000 7f9988c00700 1 -- 172.21.15.33:0/820852791 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9518+0+0 (secure 0 0 0) 0x7f99780e26f0 con 0x7f998c06bb40 2024-05-03T18:39:44.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:44.562+0000 7f998be00700 1 --2- 172.21.15.33:0/820852791 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f99700b6e80 0x7f99700b9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:39:44.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:44.563+0000 7f998be00700 1 --2- 172.21.15.33:0/820852791 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f99700b6e80 0x7f99700b9340 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f998c19aa60 tx=0x7f997c009310 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:39:44.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:44.567+0000 7f9988c00700 1 -- 172.21.15.33:0/820852791 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f99780a9da0 con 0x7f998c06bb40 2024-05-03T18:39:45.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:45.047+0000 7f9992a00700 1 -- 172.21.15.33:0/820852791 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f998c06a190 con 0x7f998c06bb40 2024-05-03T18:39:45.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:45.049+0000 7f9988c00700 1 -- 172.21.15.33:0/820852791 <== mon.1 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11801 (secure 0 0 0) 0x7f99780a96a0 con 0x7f998c06bb40 2024-05-03T18:39:45.051 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:39:45.051 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1327393503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1327393503},{"type":"v1","addr":"172.21.15.33:6835","nonce":1327393503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/2461472948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":2461472948},{"type":"v1","addr":"172.21.15.33:6837","nonce":2461472948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14559,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/442269281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":442269281},{"type":"v1","addr":"172.21.15.33:6839","nonce":442269281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14571,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/645571756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":645571756},{"type":"v1","addr":"172.21.15.33:6841","nonce":645571756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/203598909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":203598909},{"type":"v1","addr":"172.21.15.150:6833","nonce":203598909}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/4029842493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":4029842493},{"type":"v1","addr":"172.21.15.150:6835","nonce":4029842493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3463664423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3463664423},{"type":"v1","addr":"172.21.15.150:6837","nonce":3463664423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24481,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6839/4051648018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":4051648018},{"type":"v1","addr":"172.21.15.150:6839","nonce":4051648018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34418,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1865105479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1865105479},{"type":"v1","addr":"172.21.15.99:6833","nonce":1865105479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34445,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/1206393914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1206393914},{"type":"v1","addr":"172.21.15.99:6835","nonce":1206393914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-03T18:22:28.837091+0000","modified":"2024-05-03T18:23:49.758398+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34508},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34472":{"gid":34472,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6837/3360970342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3360970342},{"type":"v1","addr":"172.21.15.99:6837","nonce":3360970342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34508":{"gid":34508,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/919907308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":919907308},{"type":"v1","addr":"172.21.15.99:6839","nonce":919907308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34508,"qdb_cluster":[34508]},"id":1}]} 2024-05-03T18:39:45.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:45.053+0000 7f9992a00700 1 -- 172.21.15.33:0/820852791 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f99700b6e80 msgr2=0x7f99700b9340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:39:45.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:45.053+0000 7f9992a00700 1 --2- 172.21.15.33:0/820852791 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f99700b6e80 0x7f99700b9340 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f998c19aa60 tx=0x7f997c009310 comp rx=0 tx=0).stop 2024-05-03T18:39:45.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:45.053+0000 7f9992a00700 1 -- 172.21.15.33:0/820852791 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f998c06bb40 msgr2=0x7f998c19fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:39:45.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:45.053+0000 7f9992a00700 1 --2- 172.21.15.33:0/820852791 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f998c06bb40 0x7f998c19fca0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9978006fd0 tx=0x7f997800b730 comp rx=0 tx=0).stop 2024-05-03T18:39:45.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:45.053+0000 7f9992a00700 1 -- 172.21.15.33:0/820852791 shutdown_connections 2024-05-03T18:39:45.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:45.053+0000 7f9992a00700 1 --2- 172.21.15.33:0/820852791 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f998c109510 0x7f998c199920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:45.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:45.053+0000 7f9992a00700 1 --2- 172.21.15.33:0/820852791 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f99700b6e80 0x7f99700b9340 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:45.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:45.053+0000 7f9992a00700 1 --2- 172.21.15.33:0/820852791 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f998c06b180 0x7f998c19f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:45.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:45.053+0000 7f9992a00700 1 --2- 172.21.15.33:0/820852791 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f998c06bb40 0x7f998c19fca0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:45.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:45.053+0000 7f9992a00700 1 -- 172.21.15.33:0/820852791 >> 172.21.15.33:0/820852791 conn(0x7f998c079350 msgr2=0x7f998c1011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:39:45.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:45.054+0000 7f9992a00700 1 -- 172.21.15.33:0/820852791 shutdown_connections 2024-05-03T18:39:45.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:45.054+0000 7f9992a00700 1 -- 172.21.15.33:0/820852791 wait complete. 2024-05-03T18:39:45.057 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 21 2024-05-03T18:39:45.110 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 510.0 s, 428.8 tps, lat 234.842 ms stddev 351.496 2024-05-03T18:39:45.735 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:39:45 smithi033 ceph-mon[28330]: pgmap v391: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 11 MiB/s wr, 659 op/s 2024-05-03T18:39:45.735 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:39:45 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/820852791' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:39:45.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:39:45 smithi150 ceph-mon[32492]: pgmap v391: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 11 MiB/s wr, 659 op/s 2024-05-03T18:39:45.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:39:45 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/820852791' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:39:46.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:39:45 smithi099 ceph-mon[31658]: pgmap v391: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 11 MiB/s wr, 659 op/s 2024-05-03T18:39:46.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:39:45 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/820852791' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:39:47.250 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-03T18:39:47.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:39:47 smithi150 ceph-mon[32492]: pgmap v392: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 7.2 MiB/s wr, 484 op/s 2024-05-03T18:39:47.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:39:47 smithi033 ceph-mon[28330]: pgmap v392: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 7.2 MiB/s wr, 484 op/s 2024-05-03T18:39:48.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:39:47 smithi099 ceph-mon[31658]: pgmap v392: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 7.2 MiB/s wr, 484 op/s 2024-05-03T18:39:48.051 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:39:49.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:39:49 smithi150 ceph-mon[32492]: pgmap v393: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 7.2 MiB/s wr, 484 op/s 2024-05-03T18:39:49.899 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:39:49 smithi033 ceph-mon[28330]: pgmap v393: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 7.2 MiB/s wr, 484 op/s 2024-05-03T18:39:50.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:39:49 smithi099 ceph-mon[31658]: pgmap v393: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 7.2 MiB/s wr, 484 op/s 2024-05-03T18:39:50.110 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 515.0 s, 598.4 tps, lat 166.188 ms stddev 70.829 2024-05-03T18:39:51.844 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:39:51 smithi033 ceph-mon[28330]: pgmap v394: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 11 MiB/s wr, 645 op/s 2024-05-03T18:39:51.848 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:39:51 smithi099 ceph-mon[31658]: pgmap v394: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 11 MiB/s wr, 645 op/s 2024-05-03T18:39:51.884 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:39:51 smithi150 ceph-mon[32492]: pgmap v394: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 11 MiB/s wr, 645 op/s 2024-05-03T18:39:53.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.257+0000 7f2048c00700 1 -- 172.21.15.33:0/3333851009 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f204410b6e0 msgr2=0x7f204410bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:39:53.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.257+0000 7f2048c00700 1 --2- 172.21.15.33:0/3333851009 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f204410b6e0 0x7f204410bac0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2034009990 tx=0x7f2034009ca0 comp rx=0 tx=0).stop 2024-05-03T18:39:53.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.259+0000 7f2048c00700 1 -- 172.21.15.33:0/3333851009 shutdown_connections 2024-05-03T18:39:53.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.259+0000 7f2048c00700 1 --2- 172.21.15.33:0/3333851009 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20441060a0 0x7f20441125b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:53.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.259+0000 7f2048c00700 1 --2- 172.21.15.33:0/3333851009 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f204410b6e0 0x7f204410bac0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:53.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.259+0000 7f2048c00700 1 --2- 172.21.15.33:0/3333851009 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20441056e0 0x7f2044105b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:53.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.260+0000 7f2048c00700 1 -- 172.21.15.33:0/3333851009 >> 172.21.15.33:0/3333851009 conn(0x7f20441011d0 msgr2=0x7f20441035f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:39:53.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.260+0000 7f2048c00700 1 -- 172.21.15.33:0/3333851009 shutdown_connections 2024-05-03T18:39:53.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.261+0000 7f2048c00700 1 -- 172.21.15.33:0/3333851009 wait complete. 2024-05-03T18:39:53.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.262+0000 7f2048c00700 1 Processor -- start 2024-05-03T18:39:53.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.263+0000 7f2048c00700 1 -- start start 2024-05-03T18:39:53.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.264+0000 7f2048c00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20441056e0 0x7f20441a1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:39:53.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.264+0000 7f2048c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20441060a0 0x7f20441a1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:39:53.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.265+0000 7f2048c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f204410b6e0 0x7f204419bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:39:53.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.265+0000 7f2041e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20441056e0 0x7f20441a1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:39:53.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.266+0000 7f2041e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20441056e0 0x7f20441a1940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54820/0 (socket says 172.21.15.33:54820) 2024-05-03T18:39:53.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.266+0000 7f2041e00700 1 -- 172.21.15.33:0/1060906477 learned_addr learned my addr 172.21.15.33:0/1060906477 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:39:53.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.266+0000 7f2048c00700 1 -- 172.21.15.33:0/1060906477 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2044115810 con 0x7f20441056e0 2024-05-03T18:39:53.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.266+0000 7f2048c00700 1 -- 172.21.15.33:0/1060906477 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2044115690 con 0x7f20441060a0 2024-05-03T18:39:53.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.266+0000 7f2048c00700 1 -- 172.21.15.33:0/1060906477 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2044115990 con 0x7f204410b6e0 2024-05-03T18:39:53.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.267+0000 7f2042800700 1 --2- 172.21.15.33:0/1060906477 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f204410b6e0 0x7f204419bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:39:53.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.267+0000 7f2041400700 1 --2- 172.21.15.33:0/1060906477 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20441060a0 0x7f20441a1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:39:53.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.267+0000 7f2042800700 1 -- 172.21.15.33:0/1060906477 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20441060a0 msgr2=0x7f20441a1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:39:53.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.267+0000 7f2042800700 1 --2- 172.21.15.33:0/1060906477 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20441060a0 0x7f20441a1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:53.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.267+0000 7f2042800700 1 -- 172.21.15.33:0/1060906477 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20441056e0 msgr2=0x7f20441a1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:39:53.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.268+0000 7f2042800700 1 --2- 172.21.15.33:0/1060906477 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20441056e0 0x7f20441a1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:53.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.268+0000 7f2042800700 1 -- 172.21.15.33:0/1060906477 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f204419c2b0 con 0x7f204410b6e0 2024-05-03T18:39:53.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.268+0000 7f2041e00700 1 --2- 172.21.15.33:0/1060906477 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20441056e0 0x7f20441a1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:39:53.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.269+0000 7f2042800700 1 --2- 172.21.15.33:0/1060906477 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f204410b6e0 0x7f204419bb50 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f203c00efc0 tx=0x7f203c00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:39:53.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.269+0000 7f203aa00700 1 -- 172.21.15.33:0/1060906477 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f203c0092e0 con 0x7f204410b6e0 2024-05-03T18:39:53.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.269+0000 7f203aa00700 1 -- 172.21.15.33:0/1060906477 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f203c004bb0 con 0x7f204410b6e0 2024-05-03T18:39:53.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.269+0000 7f203aa00700 1 -- 172.21.15.33:0/1060906477 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f203c004280 con 0x7f204410b6e0 2024-05-03T18:39:53.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.269+0000 7f2048c00700 1 -- 172.21.15.33:0/1060906477 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f204419c540 con 0x7f204410b6e0 2024-05-03T18:39:53.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.269+0000 7f2048c00700 1 -- 172.21.15.33:0/1060906477 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20441a8810 con 0x7f204410b6e0 2024-05-03T18:39:53.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.275+0000 7f203aa00700 1 -- 172.21.15.33:0/1060906477 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f203c01b030 con 0x7f204410b6e0 2024-05-03T18:39:53.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.277+0000 7f203aa00700 1 --2- 172.21.15.33:0/1060906477 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f20280b2aa0 0x7f20280b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:39:53.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.278+0000 7f203aa00700 1 -- 172.21.15.33:0/1060906477 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9518+0+0 (secure 0 0 0) 0x7f203c0e2540 con 0x7f204410b6e0 2024-05-03T18:39:53.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.279+0000 7f2041e00700 1 --2- 172.21.15.33:0/1060906477 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f20280b2aa0 0x7f20280b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:39:53.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.280+0000 7f2041e00700 1 --2- 172.21.15.33:0/1060906477 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f20280b2aa0 0x7f20280b4f60 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f203400b2e0 tx=0x7f203401a040 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:39:53.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.304+0000 7f20535ebb80 1 Processor -- start 2024-05-03T18:39:53.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.305+0000 7f20535ebb80 1 -- start start 2024-05-03T18:39:53.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.305+0000 7f20535ebb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ea86074930 0x55ea86074d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:39:53.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.305+0000 7f20535ebb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ea86075250 0x55ea8606ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:39:53.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.305+0000 7f20535ebb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea8606f550 0x55ea8606f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:39:53.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.305+0000 7f20535ebb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ea85fc1a20 con 0x55ea86074930 2024-05-03T18:39:53.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.305+0000 7f20535ebb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55ea85f5f760 con 0x55ea8606f550 2024-05-03T18:39:53.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.305+0000 7f20535ebb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55ea85fbd9e0 con 0x55ea86075250 2024-05-03T18:39:53.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.306+0000 7f2041400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ea86075250 0x55ea8606ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:39:53.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.306+0000 7f2042800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea8606f550 0x55ea8606f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:39:53.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.306+0000 7f2041400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ea86075250 0x55ea8606ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:45278/0 (socket says 172.21.15.33:45278) 2024-05-03T18:39:53.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.306+0000 7f2041400700 1 -- 172.21.15.33:0/3200722669 learned_addr learned my addr 172.21.15.33:0/3200722669 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:39:53.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.306+0000 7f2041e00700 1 --2- 172.21.15.33:0/3200722669 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ea86074930 0x55ea86074d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:39:53.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.307+0000 7f2041400700 1 -- 172.21.15.33:0/3200722669 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea8606f550 msgr2=0x55ea8606f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:39:53.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.307+0000 7f2041400700 1 --2- 172.21.15.33:0/3200722669 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea8606f550 0x55ea8606f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:53.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.307+0000 7f2041400700 1 -- 172.21.15.33:0/3200722669 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ea86074930 msgr2=0x55ea86074d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:39:53.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.307+0000 7f2041400700 1 --2- 172.21.15.33:0/3200722669 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ea86074930 0x55ea86074d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:53.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.307+0000 7f2041400700 1 -- 172.21.15.33:0/3200722669 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea85f0e8d0 con 0x55ea86075250 2024-05-03T18:39:53.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.307+0000 7f2042800700 1 --2- 172.21.15.33:0/3200722669 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea8606f550 0x55ea8606f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:39:53.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.307+0000 7f2041e00700 1 --2- 172.21.15.33:0/3200722669 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ea86074930 0x55ea86074d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:39:53.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.307+0000 7f2041400700 1 --2- 172.21.15.33:0/3200722669 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ea86075250 0x55ea8606ef30 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f202c009cb0 tx=0x7f202c0113e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c4203e89316d1ff server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:39:53.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.308+0000 7f2033400700 1 -- 172.21.15.33:0/3200722669 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f202c011c60 con 0x55ea86075250 2024-05-03T18:39:53.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.308+0000 7f2033400700 1 -- 172.21.15.33:0/3200722669 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f202c011e00 con 0x55ea86075250 2024-05-03T18:39:53.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.308+0000 7f2033400700 1 -- 172.21.15.33:0/3200722669 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f202c00f680 con 0x55ea86075250 2024-05-03T18:39:53.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.309+0000 7f20535ebb80 1 -- 172.21.15.33:0/3200722669 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ea86075250 msgr2=0x55ea8606ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:39:53.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.309+0000 7f20535ebb80 1 --2- 172.21.15.33:0/3200722669 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ea86075250 0x55ea8606ef30 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f202c009cb0 tx=0x7f202c0113e0 comp rx=0 tx=0).stop 2024-05-03T18:39:53.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.309+0000 7f20535ebb80 1 -- 172.21.15.33:0/3200722669 shutdown_connections 2024-05-03T18:39:53.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.309+0000 7f20535ebb80 1 --2- 172.21.15.33:0/3200722669 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea8606f550 0x55ea8606f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:53.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.309+0000 7f20535ebb80 1 --2- 172.21.15.33:0/3200722669 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ea86074930 0x55ea86074d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:53.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.309+0000 7f20535ebb80 1 --2- 172.21.15.33:0/3200722669 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ea86075250 0x55ea8606ef30 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:53.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.309+0000 7f20535ebb80 1 -- 172.21.15.33:0/3200722669 >> 172.21.15.33:0/3200722669 conn(0x55ea85fe76d0 msgr2=0x55ea86069900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:39:53.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.310+0000 7f20535ebb80 1 -- 172.21.15.33:0/3200722669 shutdown_connections 2024-05-03T18:39:53.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.310+0000 7f20535ebb80 1 -- 172.21.15.33:0/3200722669 wait complete. 2024-05-03T18:39:53.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.310+0000 7f20535ebb80 1 -- start start 2024-05-03T18:39:53.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.311+0000 7f20535ebb80 1 Processor -- start 2024-05-03T18:39:53.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.312+0000 7f20535ebb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ea8606f550 0x55ea8607ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:39:53.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.312+0000 7f20535ebb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea86074930 0x55ea86078230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:39:53.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.313+0000 7f2041e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ea8606f550 0x55ea8607ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:39:53.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.313+0000 7f20535ebb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ea86075250 0x55ea86078770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:39:53.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.313+0000 7f2041e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ea8606f550 0x55ea8607ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:45290/0 (socket says 172.21.15.33:45290) 2024-05-03T18:39:53.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.313+0000 7f20535ebb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ea8607bd00 con 0x55ea86075250 2024-05-03T18:39:53.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.313+0000 7f20535ebb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55ea85f27b20 con 0x55ea86074930 2024-05-03T18:39:53.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.313+0000 7f20535ebb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55ea8607be80 con 0x55ea8606f550 2024-05-03T18:39:53.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.313+0000 7f2041e00700 1 -- 172.21.15.33:0/1120662436 learned_addr learned my addr 172.21.15.33:0/1120662436 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:39:53.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.313+0000 7f20535ebb80 10 client.0 _ll_register_callbacks cb 0x55ea85f68c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:39:53.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.313+0000 7f2041400700 1 --2- 172.21.15.33:0/1120662436 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea86074930 0x55ea86078230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:39:53.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.314+0000 7f2042800700 1 --2- 172.21.15.33:0/1120662436 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ea86075250 0x55ea86078770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:39:53.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.315+0000 7f2041e00700 1 -- 172.21.15.33:0/1120662436 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea86074930 msgr2=0x55ea86078230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:39:53.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.315+0000 7f2041e00700 1 --2- 172.21.15.33:0/1120662436 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea86074930 0x55ea86078230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:53.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.315+0000 7f2041e00700 1 -- 172.21.15.33:0/1120662436 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ea86075250 msgr2=0x55ea86078770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:39:53.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.315+0000 7f2041e00700 1 --2- 172.21.15.33:0/1120662436 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ea86075250 0x55ea86078770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:53.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.315+0000 7f2041e00700 1 -- 172.21.15.33:0/1120662436 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea85f87420 con 0x55ea8606f550 2024-05-03T18:39:53.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.315+0000 7f2042800700 1 --2- 172.21.15.33:0/1120662436 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ea86075250 0x55ea86078770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:39:53.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.316+0000 7f2041e00700 1 --2- 172.21.15.33:0/1120662436 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ea8606f550 0x55ea8607ae90 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f203400f6e0 tx=0x7f2034009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b882083f5fd1ab4e server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:39:53.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.316+0000 7f2027e00700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-03T18:39:53.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.316+0000 7f2027e00700 1 -- 172.21.15.33:0/1120662436 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2034035060 con 0x55ea8606f550 2024-05-03T18:39:53.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.317+0000 7f20535ebb80 1 -- 172.21.15.33:0/1120662436 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f202c011e00 con 0x55ea8606f550 2024-05-03T18:39:53.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.317+0000 7f2027e00700 1 -- 172.21.15.33:0/1120662436 <== mon.1 v2:172.21.15.150:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9518+0+0 (secure 0 0 0) 0x7f2034004c20 con 0x55ea8606f550 2024-05-03T18:39:53.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.317+0000 7f2027e00700 1 -- 172.21.15.33:0/1120662436 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f2034004e60 con 0x55ea8606f550 2024-05-03T18:39:53.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.317+0000 7f20535ebb80 10 client.26347 fetch_fsmap learned FSMap version 21 2024-05-03T18:39:53.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.317+0000 7f20535ebb80 1 -- 172.21.15.33:0/1120662436 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55ea8608eca0 con 0x55ea8606f550 2024-05-03T18:39:53.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.318+0000 7f2027e00700 1 -- 172.21.15.33:0/1120662436 <== mon.1 v2:172.21.15.150:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f202c011e00 con 0x55ea8606f550 2024-05-03T18:39:53.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.318+0000 7f2027e00700 1 -- 172.21.15.33:0/1120662436 <== mon.1 v2:172.21.15.150:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f2034022070 con 0x55ea8606f550 2024-05-03T18:39:53.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.319+0000 7f20535ebb80 10 client.26347 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:39:53.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.319+0000 7f20535ebb80 10 client.26347 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:39:53.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.319+0000 7f20535ebb80 20 client.26347 populate_metadata read hostname 'smithi033' 2024-05-03T18:39:53.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.319+0000 7f20535ebb80 1 --2- 172.21.15.33:0/1120662436 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55ea86091230 0x55ea86093620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:39:53.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.319+0000 7f20535ebb80 4 client.26347 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:39:53.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.319+0000 7f20535ebb80 1 -- 172.21.15.33:0/1120662436 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ea8603bff0 con 0x55ea86091230 2024-05-03T18:39:53.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.319+0000 7f2041400700 1 --2- 172.21.15.33:0/1120662436 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55ea86091230 0x55ea86093620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:39:53.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.320+0000 7f2041400700 1 --2- 172.21.15.33:0/1120662436 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55ea86091230 0x55ea86093620 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e1f6a91e89f3d68 server_cookie=e30aad9817eff94 in_seq=0 out_seq=0 2024-05-03T18:39:53.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.320+0000 7f2027e00700 10 client.26347 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:39:53.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.323+0000 7f2027e00700 1 -- 172.21.15.33:0/1120662436 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ea8603bff0 con 0x55ea86091230 2024-05-03T18:39:53.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.323+0000 7f2027e00700 10 client.26347 handle_command_reply: tid=0 2024-05-03T18:39:53.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.323+0000 7f20535ebb80 1 client.26347 shutdown 2024-05-03T18:39:53.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.324+0000 7f20535ebb80 1 -- 172.21.15.33:0/1120662436 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ea8606f550 msgr2=0x55ea8607ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:39:53.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.324+0000 7f20535ebb80 1 --2- 172.21.15.33:0/1120662436 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ea8606f550 0x55ea8607ae90 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f203400f6e0 tx=0x7f2034009ca0 comp rx=0 tx=0).stop 2024-05-03T18:39:53.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.324+0000 7f20535ebb80 1 -- 172.21.15.33:0/1120662436 shutdown_connections 2024-05-03T18:39:53.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.324+0000 7f20535ebb80 1 --2- 172.21.15.33:0/1120662436 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55ea86091230 0x55ea86093620 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:53.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.324+0000 7f20535ebb80 1 --2- 172.21.15.33:0/1120662436 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ea86075250 0x55ea86078770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:53.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.324+0000 7f20535ebb80 1 --2- 172.21.15.33:0/1120662436 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ea8606f550 0x55ea8607ae90 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:53.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.324+0000 7f20535ebb80 1 --2- 172.21.15.33:0/1120662436 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea86074930 0x55ea86078230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:53.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.324+0000 7f20535ebb80 1 -- 172.21.15.33:0/1120662436 >> 172.21.15.33:0/1120662436 conn(0x55ea85fe76d0 msgr2=0x55ea860643b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:39:53.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.324+0000 7f2027e00700 0 client.26347 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:39:53.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.324+0000 7f20535ebb80 1 -- 172.21.15.33:0/1120662436 shutdown_connections 2024-05-03T18:39:53.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.325+0000 7f20535ebb80 1 -- 172.21.15.33:0/1120662436 wait complete. 2024-05-03T18:39:53.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.325+0000 7f20535ebb80 20 client.26347 trim_cache size 0 max 16384 2024-05-03T18:39:53.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.353+0000 7f20535ebb80 1 Processor -- start 2024-05-03T18:39:53.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.354+0000 7f20535ebb80 1 -- start start 2024-05-03T18:39:53.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.354+0000 7f20535ebb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ea8606f550 0x55ea8606ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:39:53.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.354+0000 7f20535ebb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ea86074930 0x55ea86076b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:39:53.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.354+0000 7f20535ebb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea86075250 0x55ea86077080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:39:53.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.354+0000 7f20535ebb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f20441ab3e0 con 0x55ea8606f550 2024-05-03T18:39:53.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.354+0000 7f20535ebb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f20441adc20 con 0x55ea86075250 2024-05-03T18:39:53.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.354+0000 7f20535ebb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f20441aba30 con 0x55ea86074930 2024-05-03T18:39:53.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.355+0000 7f2041e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ea8606f550 0x55ea8606ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:39:53.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.355+0000 7f2042800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea86075250 0x55ea86077080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:39:53.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.355+0000 7f2041e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ea8606f550 0x55ea8606ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54842/0 (socket says 172.21.15.33:54842) 2024-05-03T18:39:53.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.355+0000 7f2042800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea86075250 0x55ea86077080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:55908/0 (socket says 172.21.15.33:55908) 2024-05-03T18:39:53.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.355+0000 7f2041e00700 1 -- 172.21.15.33:0/3376509180 learned_addr learned my addr 172.21.15.33:0/3376509180 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:39:53.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.355+0000 7f2041400700 1 --2- 172.21.15.33:0/3376509180 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ea86074930 0x55ea86076b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:39:53.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.355+0000 7f2041e00700 1 -- 172.21.15.33:0/3376509180 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ea86074930 msgr2=0x55ea86076b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:39:53.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.355+0000 7f2041e00700 1 --2- 172.21.15.33:0/3376509180 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ea86074930 0x55ea86076b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:53.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.355+0000 7f2041e00700 1 -- 172.21.15.33:0/3376509180 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea86075250 msgr2=0x55ea86077080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:39:53.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.355+0000 7f2041e00700 1 --2- 172.21.15.33:0/3376509180 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea86075250 0x55ea86077080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:53.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.355+0000 7f2041e00700 1 -- 172.21.15.33:0/3376509180 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea86078fd0 con 0x55ea8606f550 2024-05-03T18:39:53.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.356+0000 7f2041e00700 1 --2- 172.21.15.33:0/3376509180 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ea8606f550 0x55ea8606ef30 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f203400b650 tx=0x7f203400fcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c9b4300382afea4 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:39:53.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.356+0000 7f2032a00700 1 -- 172.21.15.33:0/3376509180 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2034035070 con 0x55ea8606f550 2024-05-03T18:39:53.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.357+0000 7f2032a00700 1 -- 172.21.15.33:0/3376509180 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f20340309d0 con 0x55ea8606f550 2024-05-03T18:39:53.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.357+0000 7f2032a00700 1 -- 172.21.15.33:0/3376509180 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2034004c80 con 0x55ea8606f550 2024-05-03T18:39:53.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.358+0000 7f20535ebb80 1 -- 172.21.15.33:0/3376509180 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ea8606f550 msgr2=0x55ea8606ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:39:53.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.358+0000 7f20535ebb80 1 --2- 172.21.15.33:0/3376509180 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ea8606f550 0x55ea8606ef30 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f203400b650 tx=0x7f203400fcd0 comp rx=0 tx=0).stop 2024-05-03T18:39:53.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.358+0000 7f20535ebb80 1 -- 172.21.15.33:0/3376509180 shutdown_connections 2024-05-03T18:39:53.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.358+0000 7f20535ebb80 1 --2- 172.21.15.33:0/3376509180 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea86075250 0x55ea86077080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:53.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.358+0000 7f20535ebb80 1 --2- 172.21.15.33:0/3376509180 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ea8606f550 0x55ea8606ef30 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:53.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.358+0000 7f20535ebb80 1 --2- 172.21.15.33:0/3376509180 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ea86074930 0x55ea86076b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:53.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.358+0000 7f20535ebb80 1 -- 172.21.15.33:0/3376509180 >> 172.21.15.33:0/3376509180 conn(0x55ea85fe76d0 msgr2=0x55ea860781f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:39:53.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.358+0000 7f20535ebb80 1 -- 172.21.15.33:0/3376509180 shutdown_connections 2024-05-03T18:39:53.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.358+0000 7f20535ebb80 1 -- 172.21.15.33:0/3376509180 wait complete. 2024-05-03T18:39:53.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.359+0000 7f20535ebb80 1 -- start start 2024-05-03T18:39:53.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.360+0000 7f20535ebb80 1 Processor -- start 2024-05-03T18:39:53.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.361+0000 7f20535ebb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ea8606f550 0x55ea8608d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:39:53.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.361+0000 7f20535ebb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea86074930 0x55ea86098af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:39:53.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.361+0000 7f20535ebb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ea86075250 0x55ea86099230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:39:53.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.361+0000 7f20535ebb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f20441ac0e0 con 0x55ea8606f550 2024-05-03T18:39:53.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.361+0000 7f20535ebb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f20441ad550 con 0x55ea86074930 2024-05-03T18:39:53.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.361+0000 7f20535ebb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f20441ac7b0 con 0x55ea86075250 2024-05-03T18:39:53.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.361+0000 7f20535ebb80 10 client.0 _ll_register_callbacks cb 0x55ea85f68c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:39:53.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.362+0000 7f2041e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ea8606f550 0x55ea8608d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:39:53.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.362+0000 7f2041e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ea8606f550 0x55ea8608d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54844/0 (socket says 172.21.15.33:54844) 2024-05-03T18:39:53.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.362+0000 7f2041e00700 1 -- 172.21.15.33:0/3824933764 learned_addr learned my addr 172.21.15.33:0/3824933764 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:39:53.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.362+0000 7f2041400700 1 --2- 172.21.15.33:0/3824933764 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea86074930 0x55ea86098af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:39:53.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.362+0000 7f2042800700 1 --2- 172.21.15.33:0/3824933764 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ea86075250 0x55ea86099230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:39:53.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.363+0000 7f2041400700 1 -- 172.21.15.33:0/3824933764 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ea86075250 msgr2=0x55ea86099230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:39:53.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.363+0000 7f2041400700 1 --2- 172.21.15.33:0/3824933764 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ea86075250 0x55ea86099230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:53.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.363+0000 7f2041400700 1 -- 172.21.15.33:0/3824933764 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ea8606f550 msgr2=0x55ea8608d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:39:53.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.363+0000 7f2041400700 1 --2- 172.21.15.33:0/3824933764 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ea8606f550 0x55ea8608d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:53.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.363+0000 7f2041400700 1 -- 172.21.15.33:0/3824933764 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea8607f180 con 0x55ea86074930 2024-05-03T18:39:53.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.363+0000 7f2041e00700 1 --2- 172.21.15.33:0/3824933764 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ea8606f550 0x55ea8608d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:39:53.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.364+0000 7f2041400700 1 --2- 172.21.15.33:0/3824933764 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea86074930 0x55ea86098af0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f202c012010 tx=0x7f202c012ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2adb1ae48251089 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:39:53.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.364+0000 7f2027e00700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-03T18:39:53.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.365+0000 7f2027e00700 1 -- 172.21.15.33:0/3824933764 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f202c005240 con 0x55ea86074930 2024-05-03T18:39:53.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.366+0000 7f20535ebb80 1 -- 172.21.15.33:0/3824933764 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20340309d0 con 0x55ea86074930 2024-05-03T18:39:53.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.366+0000 7f2027e00700 1 -- 172.21.15.33:0/3824933764 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9518+0+0 (secure 0 0 0) 0x7f202c0053e0 con 0x55ea86074930 2024-05-03T18:39:53.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.366+0000 7f2027e00700 1 -- 172.21.15.33:0/3824933764 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f202c039050 con 0x55ea86074930 2024-05-03T18:39:53.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.367+0000 7f20535ebb80 10 client.36269 fetch_fsmap learned FSMap version 21 2024-05-03T18:39:53.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.367+0000 7f20535ebb80 1 -- 172.21.15.33:0/3824933764 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55ea86080460 con 0x55ea86074930 2024-05-03T18:39:53.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.368+0000 7f2027e00700 1 -- 172.21.15.33:0/3824933764 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f20340309d0 con 0x55ea86074930 2024-05-03T18:39:53.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.369+0000 7f2027e00700 1 -- 172.21.15.33:0/3824933764 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f202c02cce0 con 0x55ea86074930 2024-05-03T18:39:53.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.369+0000 7f20535ebb80 10 client.36269 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:39:53.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.369+0000 7f20535ebb80 10 client.36269 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:39:53.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.369+0000 7f20535ebb80 20 client.36269 populate_metadata read hostname 'smithi033' 2024-05-03T18:39:53.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.369+0000 7f20535ebb80 1 --2- 172.21.15.33:0/3824933764 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55ea86080740 0x55ea86082b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:39:53.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.369+0000 7f20535ebb80 4 client.36269 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-03T18:39:53.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.370+0000 7f20535ebb80 1 -- 172.21.15.33:0/3824933764 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ea85fcf400 con 0x55ea86080740 2024-05-03T18:39:53.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.370+0000 7f2041e00700 1 --2- 172.21.15.33:0/3824933764 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55ea86080740 0x55ea86082b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:39:53.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.371+0000 7f2027e00700 10 client.36269 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:39:53.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.371+0000 7f2041e00700 1 --2- 172.21.15.33:0/3824933764 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55ea86080740 0x55ea86082b60 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4a4b1961e7e718e server_cookie=669a27a80adc200d in_seq=0 out_seq=0 2024-05-03T18:39:53.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.372+0000 7f2027e00700 1 -- 172.21.15.33:0/3824933764 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ea85fcf400 con 0x55ea86080740 2024-05-03T18:39:53.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.372+0000 7f2027e00700 10 client.36269 handle_command_reply: tid=0 2024-05-03T18:39:53.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.372+0000 7f20535ebb80 1 client.36269 shutdown 2024-05-03T18:39:53.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.372+0000 7f20535ebb80 1 -- 172.21.15.33:0/3824933764 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea86074930 msgr2=0x55ea86098af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:39:53.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.373+0000 7f20535ebb80 1 --2- 172.21.15.33:0/3824933764 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea86074930 0x55ea86098af0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f202c012010 tx=0x7f202c012ef0 comp rx=0 tx=0).stop 2024-05-03T18:39:53.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.373+0000 7f20535ebb80 1 -- 172.21.15.33:0/3824933764 shutdown_connections 2024-05-03T18:39:53.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.373+0000 7f20535ebb80 1 --2- 172.21.15.33:0/3824933764 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55ea86080740 0x55ea86082b60 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:53.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.373+0000 7f20535ebb80 1 --2- 172.21.15.33:0/3824933764 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ea86075250 0x55ea86099230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:53.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.373+0000 7f20535ebb80 1 --2- 172.21.15.33:0/3824933764 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ea8606f550 0x55ea8608d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:53.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.373+0000 7f20535ebb80 1 --2- 172.21.15.33:0/3824933764 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea86074930 0x55ea86098af0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:53.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.373+0000 7f20535ebb80 1 -- 172.21.15.33:0/3824933764 >> 172.21.15.33:0/3824933764 conn(0x55ea85fe76d0 msgr2=0x55ea86076a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:39:53.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.373+0000 7f2027e00700 0 client.36269 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:39:53.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.373+0000 7f20535ebb80 1 -- 172.21.15.33:0/3824933764 shutdown_connections 2024-05-03T18:39:53.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.374+0000 7f20535ebb80 1 -- 172.21.15.33:0/3824933764 wait complete. 2024-05-03T18:39:53.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.374+0000 7f20535ebb80 20 client.36269 trim_cache size 0 max 16384 2024-05-03T18:39:53.375 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-03T18:39:53.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.375+0000 7f2048c00700 1 -- 172.21.15.33:0/1060906477 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f20280b2aa0 msgr2=0x7f20280b4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:39:53.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.375+0000 7f2048c00700 1 --2- 172.21.15.33:0/1060906477 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f20280b2aa0 0x7f20280b4f60 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f203400b2e0 tx=0x7f203401a040 comp rx=0 tx=0).stop 2024-05-03T18:39:53.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.375+0000 7f2048c00700 1 -- 172.21.15.33:0/1060906477 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f204410b6e0 msgr2=0x7f204419bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:39:53.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.375+0000 7f2048c00700 1 --2- 172.21.15.33:0/1060906477 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f204410b6e0 0x7f204419bb50 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f203c00efc0 tx=0x7f203c00c3a0 comp rx=0 tx=0).stop 2024-05-03T18:39:53.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.375+0000 7f2048c00700 1 -- 172.21.15.33:0/1060906477 shutdown_connections 2024-05-03T18:39:53.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.375+0000 7f2048c00700 1 --2- 172.21.15.33:0/1060906477 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f204410b6e0 0x7f204419bb50 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:53.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.375+0000 7f2048c00700 1 --2- 172.21.15.33:0/1060906477 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20441056e0 0x7f20441a1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:53.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.375+0000 7f2048c00700 1 --2- 172.21.15.33:0/1060906477 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f20280b2aa0 0x7f20280b4f60 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:53.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.375+0000 7f2048c00700 1 --2- 172.21.15.33:0/1060906477 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20441060a0 0x7f20441a1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:53.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.375+0000 7f2048c00700 1 -- 172.21.15.33:0/1060906477 >> 172.21.15.33:0/1060906477 conn(0x7f20441011d0 msgr2=0x7f20441019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:39:53.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.376+0000 7f2048c00700 1 -- 172.21.15.33:0/1060906477 shutdown_connections 2024-05-03T18:39:53.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:39:53.376+0000 7f2048c00700 1 -- 172.21.15.33:0/1060906477 wait complete. 2024-05-03T18:39:53.593 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:39:53 smithi033 ceph-mon[28330]: pgmap v395: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 6.4 MiB/s wr, 418 op/s 2024-05-03T18:39:53.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:39:53 smithi150 ceph-mon[32492]: pgmap v395: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 6.4 MiB/s wr, 418 op/s 2024-05-03T18:39:54.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:39:53 smithi099 ceph-mon[31658]: pgmap v395: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 6.4 MiB/s wr, 418 op/s 2024-05-03T18:39:55.110 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 520.0 s, 520.6 tps, lat 192.501 ms stddev 286.944 2024-05-03T18:39:55.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:39:55 smithi150 ceph-mon[32492]: pgmap v396: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 10 MiB/s wr, 637 op/s 2024-05-03T18:39:55.924 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-03T18:39:55.925 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-03T18:39:55.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:39:55 smithi033 ceph-mon[28330]: pgmap v396: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 10 MiB/s wr, 637 op/s 2024-05-03T18:39:56.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:39:55 smithi099 ceph-mon[31658]: pgmap v396: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 10 MiB/s wr, 637 op/s 2024-05-03T18:39:56.695 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:39:57.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:39:57 smithi033 ceph-mon[28330]: pgmap v397: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 7.3 MiB/s wr, 380 op/s 2024-05-03T18:39:57.884 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:39:57 smithi150 ceph-mon[32492]: pgmap v397: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 7.3 MiB/s wr, 380 op/s 2024-05-03T18:39:58.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:39:57 smithi099 ceph-mon[31658]: pgmap v397: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 7.3 MiB/s wr, 380 op/s 2024-05-03T18:39:59.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:39:59 smithi150 ceph-mon[32492]: pgmap v398: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 7.3 MiB/s wr, 381 op/s 2024-05-03T18:39:59.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:39:59 smithi033 ceph-mon[28330]: pgmap v398: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 7.3 MiB/s wr, 381 op/s 2024-05-03T18:40:00.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:39:59 smithi099 ceph-mon[31658]: pgmap v398: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 7.3 MiB/s wr, 381 op/s 2024-05-03T18:40:00.111 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 525.0 s, 496.1 tps, lat 163.890 ms stddev 54.514 2024-05-03T18:40:00.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:40:00 smithi033 ceph-mon[28330]: overall HEALTH_OK 2024-05-03T18:40:01.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:40:00 smithi099 ceph-mon[31658]: overall HEALTH_OK 2024-05-03T18:40:01.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:40:00 smithi150 ceph-mon[32492]: overall HEALTH_OK 2024-05-03T18:40:01.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.207+0000 7f8715400700 1 -- 172.21.15.33:0/4115554892 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f871006bb40 msgr2=0x7f8710110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:40:01.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.207+0000 7f8715400700 1 --2- 172.21.15.33:0/4115554892 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f871006bb40 0x7f8710110330 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f870800b0a0 tx=0x7f870800b3b0 comp rx=0 tx=0).stop 2024-05-03T18:40:01.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.209+0000 7f8715400700 1 -- 172.21.15.33:0/4115554892 shutdown_connections 2024-05-03T18:40:01.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.209+0000 7f8715400700 1 --2- 172.21.15.33:0/4115554892 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f871006bb40 0x7f8710110330 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:01.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.209+0000 7f8715400700 1 --2- 172.21.15.33:0/4115554892 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8710109510 0x7f87101098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:01.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.209+0000 7f8715400700 1 --2- 172.21.15.33:0/4115554892 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f871006b180 0x7f871006b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:01.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.209+0000 7f8715400700 1 -- 172.21.15.33:0/4115554892 >> 172.21.15.33:0/4115554892 conn(0x7f8710079350 msgr2=0x7f8710079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:40:01.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.211+0000 7f8715400700 1 -- 172.21.15.33:0/4115554892 shutdown_connections 2024-05-03T18:40:01.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.211+0000 7f8715400700 1 -- 172.21.15.33:0/4115554892 wait complete. 2024-05-03T18:40:01.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.213+0000 7f8715400700 1 Processor -- start 2024-05-03T18:40:01.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.213+0000 7f8715400700 1 -- start start 2024-05-03T18:40:01.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.214+0000 7f8715400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f871006b180 0x7f871019f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:01.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.214+0000 7f8715400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f871006bb40 0x7f871019fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:01.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.215+0000 7f8715400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8710109510 0x7f8710199880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:01.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.215+0000 7f8715400700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8710113520 con 0x7f8710109510 2024-05-03T18:40:01.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.215+0000 7f8715400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f87101133a0 con 0x7f871006b180 2024-05-03T18:40:01.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.215+0000 7f8715400700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f87101136a0 con 0x7f871006bb40 2024-05-03T18:40:01.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.216+0000 7f870f200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8710109510 0x7f8710199880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:01.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.216+0000 7f870f200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8710109510 0x7f8710199880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41502/0 (socket says 172.21.15.33:41502) 2024-05-03T18:40:01.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.216+0000 7f870f200700 1 -- 172.21.15.33:0/1513337444 learned_addr learned my addr 172.21.15.33:0/1513337444 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:40:01.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.216+0000 7f870e800700 1 --2- 172.21.15.33:0/1513337444 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f871006b180 0x7f871019f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:01.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.216+0000 7f870de00700 1 --2- 172.21.15.33:0/1513337444 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f871006bb40 0x7f871019fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:01.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.217+0000 7f870e800700 1 -- 172.21.15.33:0/1513337444 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f871006bb40 msgr2=0x7f871019fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:40:01.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.217+0000 7f870e800700 1 --2- 172.21.15.33:0/1513337444 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f871006bb40 0x7f871019fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:01.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.217+0000 7f870e800700 1 -- 172.21.15.33:0/1513337444 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8710109510 msgr2=0x7f8710199880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:40:01.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.217+0000 7f870e800700 1 --2- 172.21.15.33:0/1513337444 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8710109510 0x7f8710199880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:01.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.217+0000 7f870e800700 1 -- 172.21.15.33:0/1513337444 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8710199fe0 con 0x7f871006b180 2024-05-03T18:40:01.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.218+0000 7f870e800700 1 --2- 172.21.15.33:0/1513337444 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f871006b180 0x7f871019f6c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f870000c970 tx=0x7f870000cc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:40:01.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.219+0000 7f8707400700 1 -- 172.21.15.33:0/1513337444 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8700014070 con 0x7f871006b180 2024-05-03T18:40:01.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.219+0000 7f8715400700 1 -- 172.21.15.33:0/1513337444 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f871019a270 con 0x7f871006b180 2024-05-03T18:40:01.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.219+0000 7f8715400700 1 -- 172.21.15.33:0/1513337444 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87101023a0 con 0x7f871006b180 2024-05-03T18:40:01.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.219+0000 7f8707400700 1 -- 172.21.15.33:0/1513337444 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8700004440 con 0x7f871006b180 2024-05-03T18:40:01.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.219+0000 7f8707400700 1 -- 172.21.15.33:0/1513337444 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8700007940 con 0x7f871006b180 2024-05-03T18:40:01.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.226+0000 7f8707400700 1 -- 172.21.15.33:0/1513337444 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f8700017070 con 0x7f871006b180 2024-05-03T18:40:01.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.228+0000 7f8707400700 1 --2- 172.21.15.33:0/1513337444 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f86f40b2af0 0x7f86f40b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:01.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.228+0000 7f8707400700 1 -- 172.21.15.33:0/1513337444 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9518+0+0 (secure 0 0 0) 0x7f87000e2480 con 0x7f871006b180 2024-05-03T18:40:01.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.229+0000 7f870de00700 1 --2- 172.21.15.33:0/1513337444 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f86f40b2af0 0x7f86f40b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:01.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.230+0000 7f870de00700 1 --2- 172.21.15.33:0/1513337444 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f86f40b2af0 0x7f86f40b4fb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8710079a70 tx=0x7f86fc00b410 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:40:01.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.243+0000 7f871fdb9b80 1 Processor -- start 2024-05-03T18:40:01.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.243+0000 7f871fdb9b80 1 -- start start 2024-05-03T18:40:01.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.243+0000 7f871fdb9b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e23b429b0 0x558e23b42d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:01.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.243+0000 7f871fdb9b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558e23b432d0 0x558e23b3cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:01.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.243+0000 7f871fdb9b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558e23b3d5d0 0x558e23b3da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:01.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.243+0000 7f871fdb9b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x558e23a8faa0 con 0x558e23b429b0 2024-05-03T18:40:01.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.243+0000 7f871fdb9b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x558e23a2d7e0 con 0x558e23b3d5d0 2024-05-03T18:40:01.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.243+0000 7f871fdb9b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x558e23a8ba60 con 0x558e23b432d0 2024-05-03T18:40:01.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.244+0000 7f870e800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e23b429b0 0x558e23b42d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:01.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.244+0000 7f870e800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e23b429b0 0x558e23b42d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41508/0 (socket says 172.21.15.33:41508) 2024-05-03T18:40:01.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.244+0000 7f870e800700 1 -- 172.21.15.33:0/4016444542 learned_addr learned my addr 172.21.15.33:0/4016444542 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:40:01.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.245+0000 7f870f200700 1 --2- 172.21.15.33:0/4016444542 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558e23b3d5d0 0x558e23b3da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:01.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.245+0000 7f870e800700 1 -- 172.21.15.33:0/4016444542 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558e23b432d0 msgr2=0x558e23b3cfb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-03T18:40:01.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.245+0000 7f870de00700 1 --2- 172.21.15.33:0/4016444542 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558e23b432d0 0x558e23b3cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:01.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.245+0000 7f870e800700 1 --2- 172.21.15.33:0/4016444542 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558e23b432d0 0x558e23b3cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:01.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.245+0000 7f870e800700 1 -- 172.21.15.33:0/4016444542 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558e23b3d5d0 msgr2=0x558e23b3da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:40:01.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.245+0000 7f870e800700 1 --2- 172.21.15.33:0/4016444542 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558e23b3d5d0 0x558e23b3da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:01.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.245+0000 7f870e800700 1 -- 172.21.15.33:0/4016444542 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e239dc950 con 0x558e23b429b0 2024-05-03T18:40:01.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.245+0000 7f870de00700 1 --2- 172.21.15.33:0/4016444542 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558e23b432d0 0x558e23b3cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:40:01.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.245+0000 7f870f200700 1 --2- 172.21.15.33:0/4016444542 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558e23b3d5d0 0x558e23b3da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:40:01.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.246+0000 7f870e800700 1 --2- 172.21.15.33:0/4016444542 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e23b429b0 0x558e23b42d90 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f87000a53a0 tx=0x7f87000a9220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afa9fe96446d5c62 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:40:01.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.247+0000 7f86fbe00700 1 -- 172.21.15.33:0/4016444542 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87000dfc20 con 0x558e23b429b0 2024-05-03T18:40:01.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.247+0000 7f86fbe00700 1 -- 172.21.15.33:0/4016444542 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f87000ad6d0 con 0x558e23b429b0 2024-05-03T18:40:01.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.247+0000 7f86fbe00700 1 -- 172.21.15.33:0/4016444542 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87000b1cf0 con 0x558e23b429b0 2024-05-03T18:40:01.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.248+0000 7f871fdb9b80 1 -- 172.21.15.33:0/4016444542 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e23b429b0 msgr2=0x558e23b42d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:40:01.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.249+0000 7f871fdb9b80 1 --2- 172.21.15.33:0/4016444542 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e23b429b0 0x558e23b42d90 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f87000a53a0 tx=0x7f87000a9220 comp rx=0 tx=0).stop 2024-05-03T18:40:01.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.249+0000 7f871fdb9b80 1 -- 172.21.15.33:0/4016444542 shutdown_connections 2024-05-03T18:40:01.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.249+0000 7f871fdb9b80 1 --2- 172.21.15.33:0/4016444542 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558e23b3d5d0 0x558e23b3da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:01.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.249+0000 7f871fdb9b80 1 --2- 172.21.15.33:0/4016444542 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e23b429b0 0x558e23b42d90 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:01.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.249+0000 7f871fdb9b80 1 --2- 172.21.15.33:0/4016444542 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558e23b432d0 0x558e23b3cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:01.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.249+0000 7f871fdb9b80 1 -- 172.21.15.33:0/4016444542 >> 172.21.15.33:0/4016444542 conn(0x558e23ab5750 msgr2=0x558e23b37980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:40:01.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.249+0000 7f871fdb9b80 1 -- 172.21.15.33:0/4016444542 shutdown_connections 2024-05-03T18:40:01.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.250+0000 7f871fdb9b80 1 -- 172.21.15.33:0/4016444542 wait complete. 2024-05-03T18:40:01.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.250+0000 7f871fdb9b80 1 -- start start 2024-05-03T18:40:01.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.251+0000 7f871fdb9b80 1 Processor -- start 2024-05-03T18:40:01.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.251+0000 7f871fdb9b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558e23b3d5d0 0x558e23b48ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:01.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.251+0000 7f871fdb9b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558e23b429b0 0x558e23b46240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:01.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.251+0000 7f871fdb9b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e23b432d0 0x558e23b46780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:01.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.251+0000 7f871fdb9b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x558e23b49d80 con 0x558e23b432d0 2024-05-03T18:40:01.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.251+0000 7f871fdb9b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x558e239f5ba0 con 0x558e23b3d5d0 2024-05-03T18:40:01.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.251+0000 7f871fdb9b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x558e23b49f00 con 0x558e23b429b0 2024-05-03T18:40:01.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.251+0000 7f871fdb9b80 10 client.0 _ll_register_callbacks cb 0x558e23a36cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:40:01.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.252+0000 7f870e800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558e23b3d5d0 0x558e23b48ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:01.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.252+0000 7f870e800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558e23b3d5d0 0x558e23b48ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:51734/0 (socket says 172.21.15.33:51734) 2024-05-03T18:40:01.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.252+0000 7f870e800700 1 -- 172.21.15.33:0/4133335056 learned_addr learned my addr 172.21.15.33:0/4133335056 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:40:01.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.252+0000 7f870f200700 1 --2- 172.21.15.33:0/4133335056 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e23b432d0 0x558e23b46780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:01.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.252+0000 7f870de00700 1 --2- 172.21.15.33:0/4133335056 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558e23b429b0 0x558e23b46240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:01.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.253+0000 7f870e800700 1 -- 172.21.15.33:0/4133335056 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558e23b429b0 msgr2=0x558e23b46240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:40:01.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.253+0000 7f870e800700 1 --2- 172.21.15.33:0/4133335056 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558e23b429b0 0x558e23b46240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:01.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.253+0000 7f870e800700 1 -- 172.21.15.33:0/4133335056 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e23b432d0 msgr2=0x558e23b46780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:40:01.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.253+0000 7f870e800700 1 --2- 172.21.15.33:0/4133335056 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e23b432d0 0x558e23b46780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:01.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.253+0000 7f870e800700 1 -- 172.21.15.33:0/4133335056 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e23a554a0 con 0x558e23b3d5d0 2024-05-03T18:40:01.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.254+0000 7f870e800700 1 --2- 172.21.15.33:0/4133335056 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558e23b3d5d0 0x558e23b48ea0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8700000c00 tx=0x7f87000a9830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a31316aee8228e67 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:40:01.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.254+0000 7f86f8c00700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-03T18:40:01.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.254+0000 7f86f8c00700 1 -- 172.21.15.33:0/4133335056 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87000c3060 con 0x558e23b3d5d0 2024-05-03T18:40:01.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.254+0000 7f86f8c00700 1 -- 172.21.15.33:0/4133335056 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9518+0+0 (secure 0 0 0) 0x7f87000b1f00 con 0x558e23b3d5d0 2024-05-03T18:40:01.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.255+0000 7f871fdb9b80 1 -- 172.21.15.33:0/4133335056 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87000ad6d0 con 0x558e23b3d5d0 2024-05-03T18:40:01.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.255+0000 7f86f8c00700 1 -- 172.21.15.33:0/4133335056 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f87000c8050 con 0x558e23b3d5d0 2024-05-03T18:40:01.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.255+0000 7f871fdb9b80 10 client.36281 fetch_fsmap learned FSMap version 21 2024-05-03T18:40:01.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.255+0000 7f871fdb9b80 1 -- 172.21.15.33:0/4133335056 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x558e23b54490 con 0x558e23b3d5d0 2024-05-03T18:40:01.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.256+0000 7f86f8c00700 1 -- 172.21.15.33:0/4133335056 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f87000ad6d0 con 0x558e23b3d5d0 2024-05-03T18:40:01.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.256+0000 7f86f8c00700 1 -- 172.21.15.33:0/4133335056 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f87000a4c90 con 0x558e23b3d5d0 2024-05-03T18:40:01.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.256+0000 7f871fdb9b80 10 client.36281 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:40:01.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.256+0000 7f871fdb9b80 10 client.36281 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:40:01.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.256+0000 7f871fdb9b80 20 client.36281 populate_metadata read hostname 'smithi033' 2024-05-03T18:40:01.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.256+0000 7f871fdb9b80 1 --2- 172.21.15.33:0/4133335056 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x558e23b56960 0x558e23b58d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:01.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.256+0000 7f871fdb9b80 4 client.36281 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:40:01.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.256+0000 7f871fdb9b80 1 -- 172.21.15.33:0/4133335056 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558e23b0a070 con 0x558e23b56960 2024-05-03T18:40:01.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.257+0000 7f870de00700 1 --2- 172.21.15.33:0/4133335056 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x558e23b56960 0x558e23b58d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:01.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.258+0000 7f870de00700 1 --2- 172.21.15.33:0/4133335056 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x558e23b56960 0x558e23b58d50 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dfd726104caee85d server_cookie=a3ba8633c699da85 in_seq=0 out_seq=0 2024-05-03T18:40:01.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.258+0000 7f86f8c00700 10 client.36281 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:40:01.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.259+0000 7f86f8c00700 1 -- 172.21.15.33:0/4133335056 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558e23b0a070 con 0x558e23b56960 2024-05-03T18:40:01.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.259+0000 7f86f8c00700 10 client.36281 handle_command_reply: tid=0 2024-05-03T18:40:01.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.259+0000 7f871fdb9b80 1 client.36281 shutdown 2024-05-03T18:40:01.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.259+0000 7f871fdb9b80 1 -- 172.21.15.33:0/4133335056 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558e23b3d5d0 msgr2=0x558e23b48ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:40:01.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.259+0000 7f871fdb9b80 1 --2- 172.21.15.33:0/4133335056 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558e23b3d5d0 0x558e23b48ea0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8700000c00 tx=0x7f87000a9830 comp rx=0 tx=0).stop 2024-05-03T18:40:01.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.260+0000 7f871fdb9b80 1 -- 172.21.15.33:0/4133335056 shutdown_connections 2024-05-03T18:40:01.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.260+0000 7f871fdb9b80 1 --2- 172.21.15.33:0/4133335056 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x558e23b56960 0x558e23b58d50 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:01.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.260+0000 7f871fdb9b80 1 --2- 172.21.15.33:0/4133335056 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e23b432d0 0x558e23b46780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:01.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.260+0000 7f871fdb9b80 1 --2- 172.21.15.33:0/4133335056 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558e23b3d5d0 0x558e23b48ea0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:01.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.260+0000 7f871fdb9b80 1 --2- 172.21.15.33:0/4133335056 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558e23b429b0 0x558e23b46240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:01.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.260+0000 7f871fdb9b80 1 -- 172.21.15.33:0/4133335056 >> 172.21.15.33:0/4133335056 conn(0x558e23ab5750 msgr2=0x558e23b32430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:40:01.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.260+0000 7f86f8c00700 0 client.36281 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:40:01.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.260+0000 7f871fdb9b80 1 -- 172.21.15.33:0/4133335056 shutdown_connections 2024-05-03T18:40:01.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.260+0000 7f871fdb9b80 1 -- 172.21.15.33:0/4133335056 wait complete. 2024-05-03T18:40:01.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.260+0000 7f871fdb9b80 20 client.36281 trim_cache size 0 max 16384 2024-05-03T18:40:01.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.293+0000 7f871fdb9b80 1 Processor -- start 2024-05-03T18:40:01.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.294+0000 7f871fdb9b80 1 -- start start 2024-05-03T18:40:01.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.294+0000 7f871fdb9b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558e23b3d5d0 0x558e23b453c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:01.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.294+0000 7f871fdb9b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558e23b429b0 0x558e23b322b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:01.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.294+0000 7f871fdb9b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e23b432d0 0x558e23b5bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:01.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.294+0000 7f871fdb9b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f86f000cfa0 con 0x558e23b432d0 2024-05-03T18:40:01.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.294+0000 7f871fdb9b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f86f000f800 con 0x558e23b429b0 2024-05-03T18:40:01.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.294+0000 7f871fdb9b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f86f000d5f0 con 0x558e23b3d5d0 2024-05-03T18:40:01.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.296+0000 7f870de00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558e23b429b0 0x558e23b322b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:01.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.296+0000 7f870f200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e23b432d0 0x558e23b5bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:01.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.296+0000 7f870de00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558e23b429b0 0x558e23b322b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:51746/0 (socket says 172.21.15.33:51746) 2024-05-03T18:40:01.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.296+0000 7f870de00700 1 -- 172.21.15.33:0/2289409143 learned_addr learned my addr 172.21.15.33:0/2289409143 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:40:01.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.297+0000 7f870e800700 1 --2- 172.21.15.33:0/2289409143 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558e23b3d5d0 0x558e23b453c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:01.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.298+0000 7f870de00700 1 -- 172.21.15.33:0/2289409143 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558e23b3d5d0 msgr2=0x558e23b453c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:40:01.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.298+0000 7f870de00700 1 --2- 172.21.15.33:0/2289409143 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558e23b3d5d0 0x558e23b453c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:01.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.298+0000 7f870de00700 1 -- 172.21.15.33:0/2289409143 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e23b432d0 msgr2=0x558e23b5bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:40:01.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.298+0000 7f870de00700 1 --2- 172.21.15.33:0/2289409143 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e23b432d0 0x558e23b5bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:01.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.298+0000 7f870de00700 1 -- 172.21.15.33:0/2289409143 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e23b49210 con 0x558e23b429b0 2024-05-03T18:40:01.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.298+0000 7f870e800700 1 --2- 172.21.15.33:0/2289409143 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558e23b3d5d0 0x558e23b453c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:40:01.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.298+0000 7f870f200700 1 --2- 172.21.15.33:0/2289409143 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e23b432d0 0x558e23b5bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:40:01.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.298+0000 7f870de00700 1 --2- 172.21.15.33:0/2289409143 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558e23b429b0 0x558e23b322b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f86fc005950 tx=0x7f86fc013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6081cc8e5164713 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:40:01.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.299+0000 7f86fb400700 1 -- 172.21.15.33:0/2289409143 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86fc011b60 con 0x558e23b429b0 2024-05-03T18:40:01.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.299+0000 7f86fb400700 1 -- 172.21.15.33:0/2289409143 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f86fc011d00 con 0x558e23b429b0 2024-05-03T18:40:01.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.299+0000 7f86fb400700 1 -- 172.21.15.33:0/2289409143 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86fc0055c0 con 0x558e23b429b0 2024-05-03T18:40:01.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.300+0000 7f871fdb9b80 1 -- 172.21.15.33:0/2289409143 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558e23b429b0 msgr2=0x558e23b322b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:40:01.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.300+0000 7f871fdb9b80 1 --2- 172.21.15.33:0/2289409143 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558e23b429b0 0x558e23b322b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f86fc005950 tx=0x7f86fc013000 comp rx=0 tx=0).stop 2024-05-03T18:40:01.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.300+0000 7f871fdb9b80 1 -- 172.21.15.33:0/2289409143 shutdown_connections 2024-05-03T18:40:01.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.300+0000 7f871fdb9b80 1 --2- 172.21.15.33:0/2289409143 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e23b432d0 0x558e23b5bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:01.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.300+0000 7f871fdb9b80 1 --2- 172.21.15.33:0/2289409143 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558e23b3d5d0 0x558e23b453c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:01.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.300+0000 7f871fdb9b80 1 --2- 172.21.15.33:0/2289409143 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558e23b429b0 0x558e23b322b0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:01.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.300+0000 7f871fdb9b80 1 -- 172.21.15.33:0/2289409143 >> 172.21.15.33:0/2289409143 conn(0x558e23ab5750 msgr2=0x558e23b31ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:40:01.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.301+0000 7f871fdb9b80 1 -- 172.21.15.33:0/2289409143 shutdown_connections 2024-05-03T18:40:01.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.301+0000 7f871fdb9b80 1 -- 172.21.15.33:0/2289409143 wait complete. 2024-05-03T18:40:01.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.301+0000 7f871fdb9b80 1 -- start start 2024-05-03T18:40:01.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.302+0000 7f871fdb9b80 1 Processor -- start 2024-05-03T18:40:01.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.303+0000 7f871fdb9b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558e23b3d5d0 0x558e23b5c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:01.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.303+0000 7f871fdb9b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e23b429b0 0x558e23b5a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:01.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.303+0000 7f871fdb9b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558e23b432d0 0x558e23b5acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:01.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.303+0000 7f871fdb9b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f86f000dcc0 con 0x558e23b429b0 2024-05-03T18:40:01.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.303+0000 7f871fdb9b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f86f000f130 con 0x558e23b432d0 2024-05-03T18:40:01.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.303+0000 7f871fdb9b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f86f000e390 con 0x558e23b3d5d0 2024-05-03T18:40:01.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.303+0000 7f871fdb9b80 10 client.0 _ll_register_callbacks cb 0x558e23a36cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:40:01.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.303+0000 7f870de00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e23b429b0 0x558e23b5a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:01.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.303+0000 7f870de00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e23b429b0 0x558e23b5a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41536/0 (socket says 172.21.15.33:41536) 2024-05-03T18:40:01.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.303+0000 7f870de00700 1 -- 172.21.15.33:0/1811472565 learned_addr learned my addr 172.21.15.33:0/1811472565 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:40:01.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.304+0000 7f870e800700 1 --2- 172.21.15.33:0/1811472565 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558e23b3d5d0 0x558e23b5c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:01.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.304+0000 7f870de00700 1 -- 172.21.15.33:0/1811472565 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558e23b3d5d0 msgr2=0x558e23b5c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:40:01.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.304+0000 7f870de00700 1 --2- 172.21.15.33:0/1811472565 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558e23b3d5d0 0x558e23b5c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:01.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.304+0000 7f870de00700 1 -- 172.21.15.33:0/1811472565 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558e23b432d0 msgr2=0x558e23b5acd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-03T18:40:01.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.304+0000 7f870de00700 1 --2- 172.21.15.33:0/1811472565 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558e23b432d0 0x558e23b5acd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:01.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.304+0000 7f870de00700 1 -- 172.21.15.33:0/1811472565 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e23b5b510 con 0x558e23b429b0 2024-05-03T18:40:01.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.306+0000 7f870de00700 1 --2- 172.21.15.33:0/1811472565 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e23b429b0 0x558e23b5a750 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f86fc00ba80 tx=0x7f86fc011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b10ae0b03e220750 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:40:01.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.306+0000 7f86f8c00700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:40:01.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.307+0000 7f86f8c00700 1 -- 172.21.15.33:0/1811472565 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86fc034020 con 0x558e23b429b0 2024-05-03T18:40:01.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.307+0000 7f86f8c00700 1 -- 172.21.15.33:0/1811472565 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9518+0+0 (secure 0 0 0) 0x7f86fc005480 con 0x558e23b429b0 2024-05-03T18:40:01.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.308+0000 7f871fdb9b80 1 -- 172.21.15.33:0/1811472565 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86fc011d00 con 0x558e23b429b0 2024-05-03T18:40:01.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.308+0000 7f86f8c00700 1 -- 172.21.15.33:0/1811472565 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f86fc02c040 con 0x558e23b429b0 2024-05-03T18:40:01.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.308+0000 7f871fdb9b80 10 client.16581 fetch_fsmap learned FSMap version 21 2024-05-03T18:40:01.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.308+0000 7f871fdb9b80 1 -- 172.21.15.33:0/1811472565 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x558e23bc1ef0 con 0x558e23b429b0 2024-05-03T18:40:01.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.309+0000 7f86f8c00700 1 -- 172.21.15.33:0/1811472565 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f86fc011d00 con 0x558e23b429b0 2024-05-03T18:40:01.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.310+0000 7f86f8c00700 1 -- 172.21.15.33:0/1811472565 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f86fc02f030 con 0x558e23b429b0 2024-05-03T18:40:01.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.310+0000 7f871fdb9b80 10 client.16581 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:40:01.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.310+0000 7f871fdb9b80 10 client.16581 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:40:01.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.310+0000 7f871fdb9b80 20 client.16581 populate_metadata read hostname 'smithi033' 2024-05-03T18:40:01.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.310+0000 7f871fdb9b80 1 --2- 172.21.15.33:0/1811472565 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x558e23bc2550 0x558e23bc4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:01.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.310+0000 7f871fdb9b80 4 client.16581 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-03T18:40:01.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.310+0000 7f871fdb9b80 1 -- 172.21.15.33:0/1811472565 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x558e23a9d480 con 0x558e23bc2550 2024-05-03T18:40:01.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.311+0000 7f870e800700 1 --2- 172.21.15.33:0/1811472565 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x558e23bc2550 0x558e23bc4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:01.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.312+0000 7f870e800700 1 --2- 172.21.15.33:0/1811472565 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x558e23bc2550 0x558e23bc4970 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a21a508c52eb2a7 server_cookie=98ac657559c2dc9d in_seq=0 out_seq=0 2024-05-03T18:40:01.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.312+0000 7f86f8c00700 10 client.16581 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:40:01.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.314+0000 7f86f8c00700 1 -- 172.21.15.33:0/1811472565 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x558e23a9d480 con 0x558e23bc2550 2024-05-03T18:40:01.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.314+0000 7f86f8c00700 10 client.16581 handle_command_reply: tid=0 2024-05-03T18:40:01.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.314+0000 7f871fdb9b80 1 client.16581 shutdown 2024-05-03T18:40:01.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.315+0000 7f871fdb9b80 1 -- 172.21.15.33:0/1811472565 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e23b429b0 msgr2=0x558e23b5a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:40:01.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.315+0000 7f871fdb9b80 1 --2- 172.21.15.33:0/1811472565 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e23b429b0 0x558e23b5a750 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f86fc00ba80 tx=0x7f86fc011b60 comp rx=0 tx=0).stop 2024-05-03T18:40:01.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.315+0000 7f871fdb9b80 1 -- 172.21.15.33:0/1811472565 shutdown_connections 2024-05-03T18:40:01.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.315+0000 7f871fdb9b80 1 --2- 172.21.15.33:0/1811472565 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x558e23bc2550 0x558e23bc4970 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:01.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.315+0000 7f871fdb9b80 1 --2- 172.21.15.33:0/1811472565 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558e23b432d0 0x558e23b5acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:01.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.315+0000 7f86f8c00700 0 client.16581 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:40:01.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.315+0000 7f871fdb9b80 1 --2- 172.21.15.33:0/1811472565 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558e23b3d5d0 0x558e23b5c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:01.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.315+0000 7f871fdb9b80 1 --2- 172.21.15.33:0/1811472565 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e23b429b0 0x558e23b5a750 secure :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f86fc00ba80 tx=0x7f86fc011b60 comp rx=0 tx=0).stop 2024-05-03T18:40:01.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.315+0000 7f871fdb9b80 1 -- 172.21.15.33:0/1811472565 >> 172.21.15.33:0/1811472565 conn(0x558e23ab5750 msgr2=0x558e23b32430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:40:01.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.315+0000 7f871fdb9b80 1 -- 172.21.15.33:0/1811472565 shutdown_connections 2024-05-03T18:40:01.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.315+0000 7f871fdb9b80 1 -- 172.21.15.33:0/1811472565 wait complete. 2024-05-03T18:40:01.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.316+0000 7f871fdb9b80 20 client.16581 trim_cache size 0 max 16384 2024-05-03T18:40:01.318 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-03T18:40:01.318 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-05-03T18:40:01.318 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "6a0cda57-09e8-4ec5-ba59-91a633631327", 2024-05-03T18:40:01.319 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-05-03T18:40:01.319 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-03T18:40:01.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.317+0000 7f8715400700 1 -- 172.21.15.33:0/1513337444 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f86f40b2af0 msgr2=0x7f86f40b4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:40:01.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.317+0000 7f8715400700 1 --2- 172.21.15.33:0/1513337444 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f86f40b2af0 0x7f86f40b4fb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8710079a70 tx=0x7f86fc00b410 comp rx=0 tx=0).stop 2024-05-03T18:40:01.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.317+0000 7f8715400700 1 -- 172.21.15.33:0/1513337444 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f871006b180 msgr2=0x7f871019f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:40:01.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.317+0000 7f8715400700 1 --2- 172.21.15.33:0/1513337444 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f871006b180 0x7f871019f6c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f870000c970 tx=0x7f870000cc80 comp rx=0 tx=0).stop 2024-05-03T18:40:01.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.318+0000 7f8715400700 1 -- 172.21.15.33:0/1513337444 shutdown_connections 2024-05-03T18:40:01.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.318+0000 7f8715400700 1 --2- 172.21.15.33:0/1513337444 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8710109510 0x7f8710199880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:01.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.318+0000 7f8715400700 1 --2- 172.21.15.33:0/1513337444 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f86f40b2af0 0x7f86f40b4fb0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:01.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.318+0000 7f8715400700 1 --2- 172.21.15.33:0/1513337444 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f871006b180 0x7f871019f6c0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:01.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.318+0000 7f8715400700 1 --2- 172.21.15.33:0/1513337444 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f871006bb40 0x7f871019fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:01.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.318+0000 7f8715400700 1 -- 172.21.15.33:0/1513337444 >> 172.21.15.33:0/1513337444 conn(0x7f8710079350 msgr2=0x7f87101011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:40:01.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.319+0000 7f8715400700 1 -- 172.21.15.33:0/1513337444 shutdown_connections 2024-05-03T18:40:01.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:01.319+0000 7f8715400700 1 -- 172.21.15.33:0/1513337444 wait complete. 2024-05-03T18:40:01.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:40:01 smithi033 ceph-mon[28330]: pgmap v399: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 10 MiB/s wr, 622 op/s 2024-05-03T18:40:02.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:40:01 smithi099 ceph-mon[31658]: pgmap v399: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 10 MiB/s wr, 622 op/s 2024-05-03T18:40:02.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:40:01 smithi150 ceph-mon[32492]: pgmap v399: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 10 MiB/s wr, 622 op/s 2024-05-03T18:40:02.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:40:02 smithi033 ceph-mon[28330]: scrub queued for path: / 2024-05-03T18:40:02.986 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:40:02 smithi033 ceph-mon[28330]: scrub summary: idle+waiting paths [/] 2024-05-03T18:40:02.986 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:40:02 smithi033 ceph-mon[28330]: scrub summary: active paths [/] 2024-05-03T18:40:02.986 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:40:02 smithi033 ceph-mon[28330]: scrub summary: idle+waiting paths [/] 2024-05-03T18:40:02.986 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:40:02 smithi033 ceph-mon[28330]: osdmap e103: 12 total, 12 up, 12 in 2024-05-03T18:40:03.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:40:02 smithi099 ceph-mon[31658]: scrub queued for path: / 2024-05-03T18:40:03.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:40:02 smithi099 ceph-mon[31658]: scrub summary: idle+waiting paths [/] 2024-05-03T18:40:03.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:40:02 smithi099 ceph-mon[31658]: scrub summary: active paths [/] 2024-05-03T18:40:03.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:40:02 smithi099 ceph-mon[31658]: scrub summary: idle+waiting paths [/] 2024-05-03T18:40:03.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:40:02 smithi099 ceph-mon[31658]: osdmap e103: 12 total, 12 up, 12 in 2024-05-03T18:40:03.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:40:02 smithi150 ceph-mon[32492]: scrub queued for path: / 2024-05-03T18:40:03.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:40:02 smithi150 ceph-mon[32492]: scrub summary: idle+waiting paths [/] 2024-05-03T18:40:03.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:40:02 smithi150 ceph-mon[32492]: scrub summary: active paths [/] 2024-05-03T18:40:03.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:40:02 smithi150 ceph-mon[32492]: scrub summary: idle+waiting paths [/] 2024-05-03T18:40:03.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:40:02 smithi150 ceph-mon[32492]: osdmap e103: 12 total, 12 up, 12 in 2024-05-03T18:40:03.763 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-03T18:40:03.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:40:03 smithi033 ceph-mon[28330]: pgmap v401: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 8.1 MiB/s wr, 552 op/s 2024-05-03T18:40:04.045 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:40:03 smithi099 ceph-mon[31658]: pgmap v401: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 8.1 MiB/s wr, 552 op/s 2024-05-03T18:40:04.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:40:03 smithi150 ceph-mon[32492]: pgmap v401: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 8.1 MiB/s wr, 552 op/s 2024-05-03T18:40:04.577 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:40:05.112 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 530.0 s, 534.7 tps, lat 221.237 ms stddev 344.679 2024-05-03T18:40:05.986 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:40:05 smithi033 ceph-mon[28330]: pgmap v402: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 7.6 MiB/s wr, 521 op/s 2024-05-03T18:40:06.049 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:40:05 smithi099 ceph-mon[31658]: pgmap v402: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 7.6 MiB/s wr, 521 op/s 2024-05-03T18:40:06.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:40:05 smithi150 ceph-mon[32492]: pgmap v402: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 7.6 MiB/s wr, 521 op/s 2024-05-03T18:40:06.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:40:06 smithi033 ceph-mon[28330]: scrub summary: idle 2024-05-03T18:40:07.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:40:06 smithi099 ceph-mon[31658]: scrub summary: idle 2024-05-03T18:40:07.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:40:06 smithi150 ceph-mon[32492]: scrub summary: idle 2024-05-03T18:40:07.892 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:40:07 smithi033 ceph-mon[28330]: pgmap v403: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 7.5 MiB/s wr, 520 op/s 2024-05-03T18:40:07.892 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:40:07 smithi033 ceph-mon[28330]: osdmap e104: 12 total, 12 up, 12 in 2024-05-03T18:40:08.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:40:07 smithi099 ceph-mon[31658]: pgmap v403: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 7.5 MiB/s wr, 520 op/s 2024-05-03T18:40:08.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:40:07 smithi099 ceph-mon[31658]: osdmap e104: 12 total, 12 up, 12 in 2024-05-03T18:40:08.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:40:07 smithi150 ceph-mon[32492]: pgmap v403: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 7.5 MiB/s wr, 520 op/s 2024-05-03T18:40:08.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:40:07 smithi150 ceph-mon[32492]: osdmap e104: 12 total, 12 up, 12 in 2024-05-03T18:40:09.987 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:40:09 smithi033 ceph-mon[28330]: pgmap v405: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 5.2 MiB/s wr, 290 op/s 2024-05-03T18:40:10.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:40:09 smithi099 ceph-mon[31658]: pgmap v405: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 5.2 MiB/s wr, 290 op/s 2024-05-03T18:40:10.110 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 535.0 s, 499.2 tps, lat 188.406 ms stddev 103.500 2024-05-03T18:40:10.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:40:09 smithi150 ceph-mon[32492]: pgmap v405: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 5.2 MiB/s wr, 290 op/s 2024-05-03T18:40:11.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.750+0000 7f67e8a00700 1 -- 172.21.15.33:0/3490918815 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67e4075ff0 msgr2=0x7f67e410f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:40:11.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.751+0000 7f67e8a00700 1 --2- 172.21.15.33:0/3490918815 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67e4075ff0 0x7f67e410f9c0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f67d00098f0 tx=0x7f67d0009c00 comp rx=0 tx=0).stop 2024-05-03T18:40:11.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.751+0000 7f67e8a00700 1 -- 172.21.15.33:0/3490918815 shutdown_connections 2024-05-03T18:40:11.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.751+0000 7f67e8a00700 1 --2- 172.21.15.33:0/3490918815 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67e410ff00 0x7f67e41122f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:11.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.751+0000 7f67e8a00700 1 --2- 172.21.15.33:0/3490918815 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f67e40756d0 0x7f67e4075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:11.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.751+0000 7f67e8a00700 1 --2- 172.21.15.33:0/3490918815 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67e4075ff0 0x7f67e410f9c0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:11.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.751+0000 7f67e8a00700 1 -- 172.21.15.33:0/3490918815 >> 172.21.15.33:0/3490918815 conn(0x7f67e407a600 msgr2=0x7f67e407aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:40:11.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.752+0000 7f67e8a00700 1 -- 172.21.15.33:0/3490918815 shutdown_connections 2024-05-03T18:40:11.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.752+0000 7f67e8a00700 1 -- 172.21.15.33:0/3490918815 wait complete. 2024-05-03T18:40:11.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.753+0000 7f67e8a00700 1 Processor -- start 2024-05-03T18:40:11.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.753+0000 7f67e8a00700 1 -- start start 2024-05-03T18:40:11.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.754+0000 7f67e8a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67e40756d0 0x7f67e41a1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:11.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.754+0000 7f67e8a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67e4075ff0 0x7f67e41a1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:11.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.754+0000 7f67e8a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f67e410ff00 0x7f67e419bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:11.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.754+0000 7f67e8a00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f67e41157a0 con 0x7f67e4075ff0 2024-05-03T18:40:11.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.754+0000 7f67e8a00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f67e4115620 con 0x7f67e40756d0 2024-05-03T18:40:11.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.754+0000 7f67e8a00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f67e4115920 con 0x7f67e410ff00 2024-05-03T18:40:11.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.754+0000 7f67e1400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67e4075ff0 0x7f67e41a1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:11.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.755+0000 7f67e1400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67e4075ff0 0x7f67e41a1e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45214/0 (socket says 172.21.15.33:45214) 2024-05-03T18:40:11.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.755+0000 7f67e1400700 1 -- 172.21.15.33:0/1372182190 learned_addr learned my addr 172.21.15.33:0/1372182190 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:40:11.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.755+0000 7f67e2800700 1 --2- 172.21.15.33:0/1372182190 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f67e410ff00 0x7f67e419bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:11.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.755+0000 7f67e1e00700 1 --2- 172.21.15.33:0/1372182190 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67e40756d0 0x7f67e41a1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:11.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.756+0000 7f67e1400700 1 -- 172.21.15.33:0/1372182190 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f67e410ff00 msgr2=0x7f67e419bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:40:11.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.756+0000 7f67e1400700 1 --2- 172.21.15.33:0/1372182190 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f67e410ff00 0x7f67e419bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:11.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.756+0000 7f67e1400700 1 -- 172.21.15.33:0/1372182190 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67e40756d0 msgr2=0x7f67e41a1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:40:11.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.756+0000 7f67e1400700 1 --2- 172.21.15.33:0/1372182190 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67e40756d0 0x7f67e41a1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:11.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.756+0000 7f67e1400700 1 -- 172.21.15.33:0/1372182190 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67e419c2a0 con 0x7f67e4075ff0 2024-05-03T18:40:11.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.756+0000 7f67e2800700 1 --2- 172.21.15.33:0/1372182190 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f67e410ff00 0x7f67e419bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:40:11.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.756+0000 7f67e1e00700 1 --2- 172.21.15.33:0/1372182190 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67e40756d0 0x7f67e41a1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:40:11.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.756+0000 7f67e1400700 1 --2- 172.21.15.33:0/1372182190 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67e4075ff0 0x7f67e41a1e70 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f67d0005950 tx=0x7f67d000b560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:40:11.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.757+0000 7f67daa00700 1 -- 172.21.15.33:0/1372182190 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d00228d0 con 0x7f67e4075ff0 2024-05-03T18:40:11.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.757+0000 7f67daa00700 1 -- 172.21.15.33:0/1372182190 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f67d0026070 con 0x7f67e4075ff0 2024-05-03T18:40:11.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.757+0000 7f67daa00700 1 -- 172.21.15.33:0/1372182190 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d0003680 con 0x7f67e4075ff0 2024-05-03T18:40:11.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.757+0000 7f67e8a00700 1 -- 172.21.15.33:0/1372182190 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67e419c530 con 0x7f67e4075ff0 2024-05-03T18:40:11.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.757+0000 7f67e8a00700 1 -- 172.21.15.33:0/1372182190 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67e4103550 con 0x7f67e4075ff0 2024-05-03T18:40:11.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.762+0000 7f67daa00700 1 -- 172.21.15.33:0/1372182190 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f67d0003820 con 0x7f67e4075ff0 2024-05-03T18:40:11.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.764+0000 7f67daa00700 1 --2- 172.21.15.33:0/1372182190 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f67c80b2b40 0x7f67c80b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:11.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.764+0000 7f67daa00700 1 -- 172.21.15.33:0/1372182190 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9518+0+0 (secure 0 0 0) 0x7f67d00eca30 con 0x7f67e4075ff0 2024-05-03T18:40:11.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.764+0000 7f67e1e00700 1 --2- 172.21.15.33:0/1372182190 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f67c80b2b40 0x7f67c80b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:11.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.789+0000 7f67e1e00700 1 --2- 172.21.15.33:0/1372182190 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f67c80b2b40 0x7f67c80b5000 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f67d4005fd0 tx=0x7f67d4009310 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:40:11.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.792+0000 7f67f347cb80 1 Processor -- start 2024-05-03T18:40:11.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.792+0000 7f67f347cb80 1 -- start start 2024-05-03T18:40:11.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.793+0000 7f67f347cb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584c99de930 0x5584c99ded10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:11.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.793+0000 7f67f347cb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5584c99df250 0x5584c99d8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:11.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.793+0000 7f67f347cb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5584c99d9550 0x5584c99d99d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:11.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.794+0000 7f67e1e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584c99de930 0x5584c99ded10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:11.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.794+0000 7f67e1400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5584c99df250 0x5584c99d8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:11.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.794+0000 7f67e1400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5584c99df250 0x5584c99d8f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45230/0 (socket says 172.21.15.33:45230) 2024-05-03T18:40:11.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.794+0000 7f67e1e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584c99de930 0x5584c99ded10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:57186/0 (socket says 172.21.15.33:57186) 2024-05-03T18:40:11.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.795+0000 7f67e1400700 1 -- 172.21.15.33:0/1015547471 learned_addr learned my addr 172.21.15.33:0/1015547471 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:40:11.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.795+0000 7f67f347cb80 1 -- 172.21.15.33:0/1015547471 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5584c992ba20 con 0x5584c99df250 2024-05-03T18:40:11.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.795+0000 7f67f347cb80 1 -- 172.21.15.33:0/1015547471 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5584c98c9760 con 0x5584c99de930 2024-05-03T18:40:11.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.795+0000 7f67f347cb80 1 -- 172.21.15.33:0/1015547471 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5584c99279e0 con 0x5584c99d9550 2024-05-03T18:40:11.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.795+0000 7f67e2800700 1 --2- 172.21.15.33:0/1015547471 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5584c99d9550 0x5584c99d99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:11.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.796+0000 7f67e1400700 1 -- 172.21.15.33:0/1015547471 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5584c99d9550 msgr2=0x5584c99d99d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:40:11.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.796+0000 7f67e1400700 1 --2- 172.21.15.33:0/1015547471 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5584c99d9550 0x5584c99d99d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:11.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.796+0000 7f67e1400700 1 -- 172.21.15.33:0/1015547471 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584c99de930 msgr2=0x5584c99ded10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:40:11.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.796+0000 7f67e1400700 1 --2- 172.21.15.33:0/1015547471 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584c99de930 0x5584c99ded10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:11.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.796+0000 7f67e1400700 1 -- 172.21.15.33:0/1015547471 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584c98788d0 con 0x5584c99df250 2024-05-03T18:40:11.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.796+0000 7f67e2800700 1 --2- 172.21.15.33:0/1015547471 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5584c99d9550 0x5584c99d99d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:40:11.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.797+0000 7f67e1400700 1 --2- 172.21.15.33:0/1015547471 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5584c99df250 0x5584c99d8f30 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f67d00afb50 tx=0x7f67d0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a5c822cd55bb93c server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:40:11.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.797+0000 7f67cf400700 1 -- 172.21.15.33:0/1015547471 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d00bc050 con 0x5584c99df250 2024-05-03T18:40:11.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.798+0000 7f67cf400700 1 -- 172.21.15.33:0/1015547471 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f67d00bc1f0 con 0x5584c99df250 2024-05-03T18:40:11.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.798+0000 7f67cf400700 1 -- 172.21.15.33:0/1015547471 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d0032040 con 0x5584c99df250 2024-05-03T18:40:11.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.799+0000 7f67f347cb80 1 -- 172.21.15.33:0/1015547471 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5584c99df250 msgr2=0x5584c99d8f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:40:11.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.799+0000 7f67f347cb80 1 --2- 172.21.15.33:0/1015547471 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5584c99df250 0x5584c99d8f30 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f67d00afb50 tx=0x7f67d0009c00 comp rx=0 tx=0).stop 2024-05-03T18:40:11.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.799+0000 7f67f347cb80 1 -- 172.21.15.33:0/1015547471 shutdown_connections 2024-05-03T18:40:11.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.799+0000 7f67f347cb80 1 --2- 172.21.15.33:0/1015547471 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5584c99d9550 0x5584c99d99d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:11.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.799+0000 7f67f347cb80 1 --2- 172.21.15.33:0/1015547471 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584c99de930 0x5584c99ded10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:11.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.799+0000 7f67f347cb80 1 --2- 172.21.15.33:0/1015547471 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5584c99df250 0x5584c99d8f30 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:11.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.799+0000 7f67f347cb80 1 -- 172.21.15.33:0/1015547471 >> 172.21.15.33:0/1015547471 conn(0x5584c99516d0 msgr2=0x5584c99d3900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:40:11.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.800+0000 7f67f347cb80 1 -- 172.21.15.33:0/1015547471 shutdown_connections 2024-05-03T18:40:11.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.800+0000 7f67f347cb80 1 -- 172.21.15.33:0/1015547471 wait complete. 2024-05-03T18:40:11.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.800+0000 7f67f347cb80 1 -- start start 2024-05-03T18:40:11.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.801+0000 7f67f347cb80 1 Processor -- start 2024-05-03T18:40:11.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.802+0000 7f67f347cb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5584c99d9550 0x5584c99e4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:11.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.802+0000 7f67f347cb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584c99de930 0x5584c99e2250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:11.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.802+0000 7f67f347cb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5584c99df250 0x5584c99e2790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:11.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.802+0000 7f67f347cb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5584c99e5ac0 con 0x5584c99df250 2024-05-03T18:40:11.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.802+0000 7f67f347cb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5584c9891b20 con 0x5584c99de930 2024-05-03T18:40:11.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.802+0000 7f67f347cb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5584c99e5c40 con 0x5584c99d9550 2024-05-03T18:40:11.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.802+0000 7f67f347cb80 10 client.0 _ll_register_callbacks cb 0x5584c98d2c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:40:11.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.802+0000 7f67e2800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5584c99df250 0x5584c99e2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:11.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.803+0000 7f67e1400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584c99de930 0x5584c99e2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:11.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.803+0000 7f67e2800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5584c99df250 0x5584c99e2790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45242/0 (socket says 172.21.15.33:45242) 2024-05-03T18:40:11.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.803+0000 7f67e2800700 1 -- 172.21.15.33:0/443320028 learned_addr learned my addr 172.21.15.33:0/443320028 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:40:11.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.803+0000 7f67e1e00700 1 --2- 172.21.15.33:0/443320028 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5584c99d9550 0x5584c99e4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:11.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.803+0000 7f67e2800700 1 -- 172.21.15.33:0/443320028 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5584c99d9550 msgr2=0x5584c99e4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:40:11.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.803+0000 7f67e2800700 1 --2- 172.21.15.33:0/443320028 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5584c99d9550 0x5584c99e4eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:11.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.803+0000 7f67e2800700 1 -- 172.21.15.33:0/443320028 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584c99de930 msgr2=0x5584c99e2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:40:11.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.803+0000 7f67e2800700 1 --2- 172.21.15.33:0/443320028 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584c99de930 0x5584c99e2250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:11.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.803+0000 7f67e2800700 1 -- 172.21.15.33:0/443320028 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584c98f1420 con 0x5584c99df250 2024-05-03T18:40:11.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.804+0000 7f67e1e00700 1 --2- 172.21.15.33:0/443320028 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5584c99d9550 0x5584c99e4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:40:11.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.804+0000 7f67e2800700 1 --2- 172.21.15.33:0/443320028 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5584c99df250 0x5584c99e2790 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f67dc007b70 tx=0x7f67dc007e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9647ba1124f4e19b server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:40:11.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.804+0000 7f67c3e00700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:40:11.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.805+0000 7f67c3e00700 1 -- 172.21.15.33:0/443320028 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67dc01b020 con 0x5584c99df250 2024-05-03T18:40:11.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.805+0000 7f67c3e00700 1 -- 172.21.15.33:0/443320028 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9518+0+0 (secure 0 0 0) 0x7f67dc011930 con 0x5584c99df250 2024-05-03T18:40:11.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.805+0000 7f67f347cb80 1 -- 172.21.15.33:0/443320028 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67d00bc1f0 con 0x5584c99df250 2024-05-03T18:40:11.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.805+0000 7f67c3e00700 1 -- 172.21.15.33:0/443320028 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f67dc026050 con 0x5584c99df250 2024-05-03T18:40:11.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.805+0000 7f67f347cb80 10 client.16593 fetch_fsmap learned FSMap version 21 2024-05-03T18:40:11.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.806+0000 7f67f347cb80 1 -- 172.21.15.33:0/443320028 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5584c99f01e0 con 0x5584c99df250 2024-05-03T18:40:11.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.806+0000 7f67c3e00700 1 -- 172.21.15.33:0/443320028 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f67d00bc1f0 con 0x5584c99df250 2024-05-03T18:40:11.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.807+0000 7f67c3e00700 1 -- 172.21.15.33:0/443320028 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f67dc016b70 con 0x5584c99df250 2024-05-03T18:40:11.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.807+0000 7f67f347cb80 10 client.16593 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:40:11.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.807+0000 7f67f347cb80 10 client.16593 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:40:11.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.807+0000 7f67f347cb80 20 client.16593 populate_metadata read hostname 'smithi033' 2024-05-03T18:40:11.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.807+0000 7f67f347cb80 1 --2- 172.21.15.33:0/443320028 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5584c99f2820 0x5584c99f4c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:11.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.807+0000 7f67f347cb80 4 client.16593 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:40:11.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.807+0000 7f67f347cb80 1 -- 172.21.15.33:0/443320028 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5584c99a5ff0 con 0x5584c99f2820 2024-05-03T18:40:11.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.808+0000 7f67e1e00700 1 --2- 172.21.15.33:0/443320028 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5584c99f2820 0x5584c99f4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:11.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.810+0000 7f67e1e00700 1 --2- 172.21.15.33:0/443320028 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5584c99f2820 0x5584c99f4c90 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c87e0d067536557 server_cookie=1b2ab04f32900300 in_seq=0 out_seq=0 2024-05-03T18:40:11.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.810+0000 7f67c3e00700 10 client.16593 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:40:11.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.812+0000 7f67c3e00700 1 -- 172.21.15.33:0/443320028 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5584c99a5ff0 con 0x5584c99f2820 2024-05-03T18:40:11.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.812+0000 7f67c3e00700 10 client.16593 handle_command_reply: tid=0 2024-05-03T18:40:11.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.812+0000 7f67f347cb80 1 client.16593 shutdown 2024-05-03T18:40:11.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.813+0000 7f67f347cb80 1 -- 172.21.15.33:0/443320028 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5584c99df250 msgr2=0x5584c99e2790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:40:11.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.813+0000 7f67f347cb80 1 --2- 172.21.15.33:0/443320028 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5584c99df250 0x5584c99e2790 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f67dc007b70 tx=0x7f67dc007e80 comp rx=0 tx=0).stop 2024-05-03T18:40:11.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.814+0000 7f67f347cb80 1 -- 172.21.15.33:0/443320028 shutdown_connections 2024-05-03T18:40:11.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.814+0000 7f67f347cb80 1 --2- 172.21.15.33:0/443320028 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5584c99f2820 0x5584c99f4c90 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:11.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.814+0000 7f67f347cb80 1 --2- 172.21.15.33:0/443320028 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5584c99df250 0x5584c99e2790 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:11.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.814+0000 7f67f347cb80 1 --2- 172.21.15.33:0/443320028 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5584c99d9550 0x5584c99e4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:11.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.814+0000 7f67f347cb80 1 --2- 172.21.15.33:0/443320028 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584c99de930 0x5584c99e2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:11.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.814+0000 7f67f347cb80 1 -- 172.21.15.33:0/443320028 >> 172.21.15.33:0/443320028 conn(0x5584c99516d0 msgr2=0x5584c99ce3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:40:11.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.814+0000 7f67c3e00700 0 client.16593 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:40:11.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.814+0000 7f67f347cb80 1 -- 172.21.15.33:0/443320028 shutdown_connections 2024-05-03T18:40:11.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.814+0000 7f67f347cb80 1 -- 172.21.15.33:0/443320028 wait complete. 2024-05-03T18:40:11.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.814+0000 7f67f347cb80 20 client.16593 trim_cache size 0 max 16384 2024-05-03T18:40:11.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.852+0000 7f67f347cb80 1 Processor -- start 2024-05-03T18:40:11.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.852+0000 7f67f347cb80 1 -- start start 2024-05-03T18:40:11.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.852+0000 7f67f347cb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5584c99d9550 0x5584c99d8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:11.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.852+0000 7f67f347cb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5584c99de930 0x5584c99f4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:11.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.852+0000 7f67f347cb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584c99df250 0x5584c99dfe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:11.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.852+0000 7f67f347cb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f67e41ab2d0 con 0x5584c99d9550 2024-05-03T18:40:11.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.852+0000 7f67f347cb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f67e41adb10 con 0x5584c99df250 2024-05-03T18:40:11.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.852+0000 7f67f347cb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f67e41ab920 con 0x5584c99de930 2024-05-03T18:40:11.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.853+0000 7f67e1e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5584c99d9550 0x5584c99d8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:11.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.853+0000 7f67e1400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5584c99de930 0x5584c99f4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:11.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.853+0000 7f67e1e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5584c99d9550 0x5584c99d8f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45250/0 (socket says 172.21.15.33:45250) 2024-05-03T18:40:11.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.853+0000 7f67e2800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584c99df250 0x5584c99dfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:11.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.853+0000 7f67e1e00700 1 -- 172.21.15.33:0/2321946311 learned_addr learned my addr 172.21.15.33:0/2321946311 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:40:11.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.853+0000 7f67e1400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5584c99de930 0x5584c99f4ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:47372/0 (socket says 172.21.15.33:47372) 2024-05-03T18:40:11.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.855+0000 7f67e1e00700 1 -- 172.21.15.33:0/2321946311 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5584c99de930 msgr2=0x5584c99f4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:40:11.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.855+0000 7f67e1e00700 1 --2- 172.21.15.33:0/2321946311 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5584c99de930 0x5584c99f4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:11.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.855+0000 7f67e1e00700 1 -- 172.21.15.33:0/2321946311 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584c99df250 msgr2=0x5584c99dfe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:40:11.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.855+0000 7f67e1e00700 1 --2- 172.21.15.33:0/2321946311 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584c99df250 0x5584c99dfe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:11.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.855+0000 7f67e1e00700 1 -- 172.21.15.33:0/2321946311 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584c99de3d0 con 0x5584c99d9550 2024-05-03T18:40:11.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.856+0000 7f67e1e00700 1 --2- 172.21.15.33:0/2321946311 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5584c99d9550 0x5584c99d8f30 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f67d4012610 tx=0x7f67d4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7050dbe257e6326d server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:40:11.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.857+0000 7f67cea00700 1 -- 172.21.15.33:0/2321946311 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d4012c90 con 0x5584c99d9550 2024-05-03T18:40:11.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.857+0000 7f67cea00700 1 -- 172.21.15.33:0/2321946311 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f67d402d070 con 0x5584c99d9550 2024-05-03T18:40:11.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.857+0000 7f67cea00700 1 -- 172.21.15.33:0/2321946311 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d401f8b0 con 0x5584c99d9550 2024-05-03T18:40:11.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.858+0000 7f67f347cb80 1 -- 172.21.15.33:0/2321946311 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5584c99d9550 msgr2=0x5584c99d8f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:40:11.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.858+0000 7f67f347cb80 1 --2- 172.21.15.33:0/2321946311 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5584c99d9550 0x5584c99d8f30 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f67d4012610 tx=0x7f67d4017000 comp rx=0 tx=0).stop 2024-05-03T18:40:11.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.858+0000 7f67f347cb80 1 -- 172.21.15.33:0/2321946311 shutdown_connections 2024-05-03T18:40:11.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.858+0000 7f67f347cb80 1 --2- 172.21.15.33:0/2321946311 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584c99df250 0x5584c99dfe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:11.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.858+0000 7f67f347cb80 1 --2- 172.21.15.33:0/2321946311 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5584c99d9550 0x5584c99d8f30 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:11.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.858+0000 7f67f347cb80 1 --2- 172.21.15.33:0/2321946311 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5584c99de930 0x5584c99f4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:11.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.858+0000 7f67f347cb80 1 -- 172.21.15.33:0/2321946311 >> 172.21.15.33:0/2321946311 conn(0x5584c99516d0 msgr2=0x5584c99df6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:40:11.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.859+0000 7f67f347cb80 1 -- 172.21.15.33:0/2321946311 shutdown_connections 2024-05-03T18:40:11.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.859+0000 7f67f347cb80 1 -- 172.21.15.33:0/2321946311 wait complete. 2024-05-03T18:40:11.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.859+0000 7f67f347cb80 1 -- start start 2024-05-03T18:40:11.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.860+0000 7f67f347cb80 1 Processor -- start 2024-05-03T18:40:11.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.860+0000 7f67f347cb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584c99de930 0x5584c99eae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:11.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.860+0000 7f67f347cb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5584c99df250 0x5584c99eb3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:11.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.860+0000 7f67f347cb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5584c9a58c30 0x5584c99f9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:11.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.860+0000 7f67f347cb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f67e41abfd0 con 0x5584c9a58c30 2024-05-03T18:40:11.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.860+0000 7f67f347cb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f67e41ad440 con 0x5584c99de930 2024-05-03T18:40:11.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.860+0000 7f67f347cb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f67e41ac6a0 con 0x5584c99df250 2024-05-03T18:40:11.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.860+0000 7f67f347cb80 10 client.0 _ll_register_callbacks cb 0x5584c98d2c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:40:11.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.861+0000 7f67e1e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584c99de930 0x5584c99eae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:11.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.861+0000 7f67e1e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584c99de930 0x5584c99eae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:57216/0 (socket says 172.21.15.33:57216) 2024-05-03T18:40:11.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.861+0000 7f67e1e00700 1 -- 172.21.15.33:0/2525242905 learned_addr learned my addr 172.21.15.33:0/2525242905 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:40:11.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.861+0000 7f67e1400700 1 --2- 172.21.15.33:0/2525242905 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5584c99df250 0x5584c99eb3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:11.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.862+0000 7f67e1e00700 1 -- 172.21.15.33:0/2525242905 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5584c99df250 msgr2=0x5584c99eb3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:40:11.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.862+0000 7f67e1e00700 1 --2- 172.21.15.33:0/2525242905 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5584c99df250 0x5584c99eb3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:11.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.862+0000 7f67e1e00700 1 -- 172.21.15.33:0/2525242905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5584c9a58c30 msgr2=0x5584c99f9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:40:11.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.862+0000 7f67e1e00700 1 --2- 172.21.15.33:0/2525242905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5584c9a58c30 0x5584c99f9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:11.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.862+0000 7f67e1e00700 1 -- 172.21.15.33:0/2525242905 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584c99fa720 con 0x5584c99de930 2024-05-03T18:40:11.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.862+0000 7f67e1e00700 1 --2- 172.21.15.33:0/2525242905 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584c99de930 0x5584c99eae90 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f67d4000c00 tx=0x7f67d4012b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=760d875f0c307fcc server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:40:11.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.863+0000 7f67c3e00700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-03T18:40:11.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.863+0000 7f67c3e00700 1 -- 172.21.15.33:0/2525242905 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d4005240 con 0x5584c99de930 2024-05-03T18:40:11.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.864+0000 7f67c3e00700 1 -- 172.21.15.33:0/2525242905 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9518+0+0 (secure 0 0 0) 0x7f67d40053e0 con 0x5584c99de930 2024-05-03T18:40:11.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.864+0000 7f67f347cb80 1 -- 172.21.15.33:0/2525242905 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67d402d070 con 0x5584c99de930 2024-05-03T18:40:11.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.864+0000 7f67c3e00700 1 -- 172.21.15.33:0/2525242905 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f67d40055e0 con 0x5584c99de930 2024-05-03T18:40:11.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.865+0000 7f67f347cb80 10 client.36302 fetch_fsmap learned FSMap version 21 2024-05-03T18:40:11.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.865+0000 7f67f347cb80 1 -- 172.21.15.33:0/2525242905 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5584c9a7fb40 con 0x5584c99de930 2024-05-03T18:40:11.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.865+0000 7f67c3e00700 1 -- 172.21.15.33:0/2525242905 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f67d402d070 con 0x5584c99de930 2024-05-03T18:40:11.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.866+0000 7f67c3e00700 1 -- 172.21.15.33:0/2525242905 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f67d4028ce0 con 0x5584c99de930 2024-05-03T18:40:11.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.867+0000 7f67f347cb80 10 client.36302 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:40:11.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.867+0000 7f67f347cb80 10 client.36302 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:40:11.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.867+0000 7f67f347cb80 20 client.36302 populate_metadata read hostname 'smithi033' 2024-05-03T18:40:11.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.867+0000 7f67f347cb80 1 --2- 172.21.15.33:0/2525242905 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5584c9a80180 0x5584c9a825a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:11.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.867+0000 7f67f347cb80 4 client.36302 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-03T18:40:11.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.867+0000 7f67f347cb80 1 -- 172.21.15.33:0/2525242905 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5584c9939400 con 0x5584c9a80180 2024-05-03T18:40:11.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.868+0000 7f67e1400700 1 --2- 172.21.15.33:0/2525242905 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5584c9a80180 0x5584c9a825a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:11.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.870+0000 7f67e1400700 1 --2- 172.21.15.33:0/2525242905 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5584c9a80180 0x5584c9a825a0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=756da4368cbb1bf3 server_cookie=935c33bec72414e9 in_seq=0 out_seq=0 2024-05-03T18:40:11.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.870+0000 7f67c3e00700 10 client.36302 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:40:11.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.870+0000 7f67c3e00700 1 -- 172.21.15.33:0/2525242905 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5584c9939400 con 0x5584c9a80180 2024-05-03T18:40:11.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.870+0000 7f67c3e00700 10 client.36302 handle_command_reply: tid=0 2024-05-03T18:40:11.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.870+0000 7f67f347cb80 1 client.36302 shutdown 2024-05-03T18:40:11.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.871+0000 7f67f347cb80 1 -- 172.21.15.33:0/2525242905 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584c99de930 msgr2=0x5584c99eae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:40:11.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.871+0000 7f67f347cb80 1 --2- 172.21.15.33:0/2525242905 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584c99de930 0x5584c99eae90 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f67d4000c00 tx=0x7f67d4012b40 comp rx=0 tx=0).stop 2024-05-03T18:40:11.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.871+0000 7f67f347cb80 1 -- 172.21.15.33:0/2525242905 shutdown_connections 2024-05-03T18:40:11.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.871+0000 7f67f347cb80 1 --2- 172.21.15.33:0/2525242905 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5584c9a80180 0x5584c9a825a0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:11.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.871+0000 7f67c3e00700 0 client.36302 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:40:11.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.871+0000 7f67f347cb80 1 --2- 172.21.15.33:0/2525242905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5584c9a58c30 0x5584c99f9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:11.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.871+0000 7f67f347cb80 1 --2- 172.21.15.33:0/2525242905 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584c99de930 0x5584c99eae90 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:11.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.871+0000 7f67f347cb80 1 --2- 172.21.15.33:0/2525242905 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5584c99df250 0x5584c99eb3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:11.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.871+0000 7f67f347cb80 1 -- 172.21.15.33:0/2525242905 >> 172.21.15.33:0/2525242905 conn(0x5584c99516d0 msgr2=0x5584c99cdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:40:11.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.872+0000 7f67f347cb80 1 -- 172.21.15.33:0/2525242905 shutdown_connections 2024-05-03T18:40:11.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.872+0000 7f67f347cb80 1 -- 172.21.15.33:0/2525242905 wait complete. 2024-05-03T18:40:11.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.872+0000 7f67f347cb80 20 client.36302 trim_cache size 0 max 16384 2024-05-03T18:40:11.874 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-03T18:40:11.874 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-05-03T18:40:11.874 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-05-03T18:40:11.874 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-03T18:40:11.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.873+0000 7f67e8a00700 1 -- 172.21.15.33:0/1372182190 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f67c80b2b40 msgr2=0x7f67c80b5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:40:11.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.873+0000 7f67e8a00700 1 --2- 172.21.15.33:0/1372182190 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f67c80b2b40 0x7f67c80b5000 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f67d4005fd0 tx=0x7f67d4009310 comp rx=0 tx=0).stop 2024-05-03T18:40:11.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.873+0000 7f67e8a00700 1 -- 172.21.15.33:0/1372182190 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67e4075ff0 msgr2=0x7f67e41a1e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:40:11.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.873+0000 7f67e8a00700 1 --2- 172.21.15.33:0/1372182190 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67e4075ff0 0x7f67e41a1e70 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f67d0005950 tx=0x7f67d000b560 comp rx=0 tx=0).stop 2024-05-03T18:40:11.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.874+0000 7f67e8a00700 1 -- 172.21.15.33:0/1372182190 shutdown_connections 2024-05-03T18:40:11.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.874+0000 7f67e8a00700 1 --2- 172.21.15.33:0/1372182190 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f67e410ff00 0x7f67e419bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:11.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.874+0000 7f67e8a00700 1 --2- 172.21.15.33:0/1372182190 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f67c80b2b40 0x7f67c80b5000 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:11.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.874+0000 7f67e8a00700 1 --2- 172.21.15.33:0/1372182190 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67e40756d0 0x7f67e41a1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:11.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.874+0000 7f67e8a00700 1 --2- 172.21.15.33:0/1372182190 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67e4075ff0 0x7f67e41a1e70 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:11.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.874+0000 7f67e8a00700 1 -- 172.21.15.33:0/1372182190 >> 172.21.15.33:0/1372182190 conn(0x7f67e407a600 msgr2=0x7f67e41007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:40:11.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.875+0000 7f67e8a00700 1 -- 172.21.15.33:0/1372182190 shutdown_connections 2024-05-03T18:40:11.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:11.875+0000 7f67e8a00700 1 -- 172.21.15.33:0/1372182190 wait complete. 2024-05-03T18:40:12.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:40:11 smithi099 ceph-mon[31658]: pgmap v406: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 8.5 MiB/s wr, 586 op/s 2024-05-03T18:40:12.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:40:11 smithi150 ceph-mon[32492]: pgmap v406: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 8.5 MiB/s wr, 586 op/s 2024-05-03T18:40:12.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:40:11 smithi033 ceph-mon[28330]: pgmap v406: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 8.5 MiB/s wr, 586 op/s 2024-05-03T18:40:12.845 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-03T18:40:12.846 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T18:40:13.676 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:40:14.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:40:13 smithi099 ceph-mon[31658]: pgmap v407: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 7.2 MiB/s wr, 496 op/s 2024-05-03T18:40:14.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:40:13 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:40:14.090 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:40:13 smithi033 ceph-mon[28330]: pgmap v407: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 7.2 MiB/s wr, 496 op/s 2024-05-03T18:40:14.090 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:40:13 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:40:14.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:40:13 smithi150 ceph-mon[32492]: pgmap v407: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 7.2 MiB/s wr, 496 op/s 2024-05-03T18:40:14.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:40:13 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:40:15.110 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 540.0 s, 556.4 tps, lat 191.120 ms stddev 309.026 2024-05-03T18:40:16.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:40:15 smithi099 ceph-mon[31658]: pgmap v408: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 7.4 MiB/s wr, 513 op/s 2024-05-03T18:40:16.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:40:15 smithi150 ceph-mon[32492]: pgmap v408: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 7.4 MiB/s wr, 513 op/s 2024-05-03T18:40:16.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:40:15 smithi033 ceph-mon[28330]: pgmap v408: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 7.4 MiB/s wr, 513 op/s 2024-05-03T18:40:17.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:17.508+0000 7f866e800700 1 -- 172.21.15.33:0/1156840580 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86681083d0 msgr2=0x7f866806b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:40:17.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:17.508+0000 7f866e800700 1 --2- 172.21.15.33:0/1156840580 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86681083d0 0x7f866806b1b0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f86500098f0 tx=0x7f8650009c00 comp rx=0 tx=0).stop 2024-05-03T18:40:17.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:17.508+0000 7f866e800700 1 -- 172.21.15.33:0/1156840580 shutdown_connections 2024-05-03T18:40:17.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:17.508+0000 7f866e800700 1 --2- 172.21.15.33:0/1156840580 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8668105ee0 0x7f8668106370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:17.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:17.508+0000 7f866e800700 1 --2- 172.21.15.33:0/1156840580 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86681083d0 0x7f866806b1b0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:17.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:17.508+0000 7f866e800700 1 --2- 172.21.15.33:0/1156840580 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f866806b6f0 0x7f866806bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:17.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:17.508+0000 7f866e800700 1 -- 172.21.15.33:0/1156840580 >> 172.21.15.33:0/1156840580 conn(0x7f8668079350 msgr2=0x7f8668079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:40:17.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:17.508+0000 7f866e800700 1 -- 172.21.15.33:0/1156840580 shutdown_connections 2024-05-03T18:40:17.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:17.509+0000 7f866e800700 1 -- 172.21.15.33:0/1156840580 wait complete. 2024-05-03T18:40:17.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:17.510+0000 7f866e800700 1 Processor -- start 2024-05-03T18:40:17.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:17.510+0000 7f866e800700 1 -- start start 2024-05-03T18:40:17.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:17.511+0000 7f866e800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f866806b6f0 0x7f8668074b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:17.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:17.511+0000 7f866e800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8668105ee0 0x7f86680750a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:17.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:17.512+0000 7f866e800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86681083d0 0x7f866806dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:17.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:17.512+0000 7f866e800700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8668119b10 con 0x7f8668105ee0 2024-05-03T18:40:17.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:17.512+0000 7f8667400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f866806b6f0 0x7f8668074b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:17.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:17.512+0000 7f8667400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f866806b6f0 0x7f8668074b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:54744/0 (socket says 172.21.15.33:54744) 2024-05-03T18:40:17.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:17.512+0000 7f8667400700 1 -- 172.21.15.33:0/1911157065 learned_addr learned my addr 172.21.15.33:0/1911157065 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:40:17.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:17.512+0000 7f866e800700 1 -- 172.21.15.33:0/1911157065 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8668119990 con 0x7f866806b6f0 2024-05-03T18:40:17.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:17.512+0000 7f866e800700 1 -- 172.21.15.33:0/1911157065 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8668119c90 con 0x7f86681083d0 2024-05-03T18:40:17.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:17.513+0000 7f8667e00700 1 --2- 172.21.15.33:0/1911157065 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86681083d0 0x7f866806dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:17.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:17.514+0000 7f8667e00700 1 -- 172.21.15.33:0/1911157065 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f866806b6f0 msgr2=0x7f8668074b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:40:17.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:17.514+0000 7f8667e00700 1 --2- 172.21.15.33:0/1911157065 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f866806b6f0 0x7f8668074b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:17.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:17.514+0000 7f8667e00700 1 -- 172.21.15.33:0/1911157065 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8668105ee0 msgr2=0x7f86680750a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:40:17.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:17.514+0000 7f8667e00700 1 --2- 172.21.15.33:0/1911157065 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8668105ee0 0x7f86680750a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:17.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:17.514+0000 7f8667e00700 1 -- 172.21.15.33:0/1911157065 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f866806e600 con 0x7f86681083d0 2024-05-03T18:40:17.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:17.514+0000 7f8667e00700 1 --2- 172.21.15.33:0/1911157065 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86681083d0 0x7f866806dd30 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8660007f10 tx=0x7f866000d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:40:17.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:17.515+0000 7f8664c00700 1 -- 172.21.15.33:0/1911157065 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f866000ddd0 con 0x7f86681083d0 2024-05-03T18:40:17.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:17.515+0000 7f8664c00700 1 -- 172.21.15.33:0/1911157065 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8660009e20 con 0x7f86681083d0 2024-05-03T18:40:17.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:17.515+0000 7f866e800700 1 -- 172.21.15.33:0/1911157065 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f866806e8f0 con 0x7f86681083d0 2024-05-03T18:40:17.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:17.516+0000 7f866e800700 1 -- 172.21.15.33:0/1911157065 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86680728c0 con 0x7f86681083d0 2024-05-03T18:40:17.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:17.516+0000 7f8664c00700 1 -- 172.21.15.33:0/1911157065 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8660015400 con 0x7f86681083d0 2024-05-03T18:40:17.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:17.517+0000 7f866e800700 1 -- 172.21.15.33:0/1911157065 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86681029c0 con 0x7f86681083d0 2024-05-03T18:40:17.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:17.522+0000 7f8664c00700 1 -- 172.21.15.33:0/1911157065 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f8660005020 con 0x7f86681083d0 2024-05-03T18:40:17.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:17.524+0000 7f8664c00700 1 --2- 172.21.15.33:0/1911157065 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f864c0b2a70 0x7f864c0b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:17.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:17.524+0000 7f8664c00700 1 -- 172.21.15.33:0/1911157065 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9518+0+0 (secure 0 0 0) 0x7f86600e2bc0 con 0x7f86681083d0 2024-05-03T18:40:17.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:17.526+0000 7f8667400700 1 --2- 172.21.15.33:0/1911157065 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f864c0b2a70 0x7f864c0b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:17.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:17.531+0000 7f8667400700 1 --2- 172.21.15.33:0/1911157065 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f864c0b2a70 0x7f864c0b4f30 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8650009fd0 tx=0x7f8650019040 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:40:17.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:17.534+0000 7f8664c00700 1 -- 172.21.15.33:0/1911157065 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f8660010070 con 0x7f86681083d0 2024-05-03T18:40:18.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:40:17 smithi099 ceph-mon[31658]: pgmap v409: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 7.5 MiB/s wr, 513 op/s 2024-05-03T18:40:18.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:40:17 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:40:18.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:40:17 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:40:18.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:40:17 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:40:18.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:18.077+0000 7f866e800700 1 -- 172.21.15.33:0/1911157065 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8668072c80 con 0x7f86681083d0 2024-05-03T18:40:18.079 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:40:17 smithi033 ceph-mon[28330]: pgmap v409: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 7.5 MiB/s wr, 513 op/s 2024-05-03T18:40:18.079 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:40:17 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:40:18.079 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:40:17 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:40:18.079 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:40:17 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:40:18.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:18.079+0000 7f8664c00700 1 -- 172.21.15.33:0/1911157065 <== mon.1 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11801 (secure 0 0 0) 0x7f86600aa160 con 0x7f86681083d0 2024-05-03T18:40:18.084 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:40:18.085 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1327393503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1327393503},{"type":"v1","addr":"172.21.15.33:6835","nonce":1327393503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/2461472948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":2461472948},{"type":"v1","addr":"172.21.15.33:6837","nonce":2461472948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14559,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/442269281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":442269281},{"type":"v1","addr":"172.21.15.33:6839","nonce":442269281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14571,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/645571756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":645571756},{"type":"v1","addr":"172.21.15.33:6841","nonce":645571756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/203598909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":203598909},{"type":"v1","addr":"172.21.15.150:6833","nonce":203598909}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/4029842493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":4029842493},{"type":"v1","addr":"172.21.15.150:6835","nonce":4029842493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3463664423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3463664423},{"type":"v1","addr":"172.21.15.150:6837","nonce":3463664423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24481,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6839/4051648018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":4051648018},{"type":"v1","addr":"172.21.15.150:6839","nonce":4051648018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34418,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1865105479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1865105479},{"type":"v1","addr":"172.21.15.99:6833","nonce":1865105479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34445,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/1206393914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1206393914},{"type":"v1","addr":"172.21.15.99:6835","nonce":1206393914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-03T18:22:28.837091+0000","modified":"2024-05-03T18:23:49.758398+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34508},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34472":{"gid":34472,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6837/3360970342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3360970342},{"type":"v1","addr":"172.21.15.99:6837","nonce":3360970342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34508":{"gid":34508,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/919907308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":919907308},{"type":"v1","addr":"172.21.15.99:6839","nonce":919907308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34508,"qdb_cluster":[34508]},"id":1}]} 2024-05-03T18:40:18.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:18.085+0000 7f866e800700 1 -- 172.21.15.33:0/1911157065 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f864c0b2a70 msgr2=0x7f864c0b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:40:18.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:18.085+0000 7f866e800700 1 --2- 172.21.15.33:0/1911157065 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f864c0b2a70 0x7f864c0b4f30 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8650009fd0 tx=0x7f8650019040 comp rx=0 tx=0).stop 2024-05-03T18:40:18.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:18.085+0000 7f866e800700 1 -- 172.21.15.33:0/1911157065 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86681083d0 msgr2=0x7f866806dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:40:18.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:18.085+0000 7f866e800700 1 --2- 172.21.15.33:0/1911157065 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86681083d0 0x7f866806dd30 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8660007f10 tx=0x7f866000d3b0 comp rx=0 tx=0).stop 2024-05-03T18:40:18.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:18.088+0000 7f866e800700 1 -- 172.21.15.33:0/1911157065 shutdown_connections 2024-05-03T18:40:18.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:18.088+0000 7f866e800700 1 --2- 172.21.15.33:0/1911157065 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86681083d0 0x7f866806dd30 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:18.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:18.088+0000 7f866e800700 1 --2- 172.21.15.33:0/1911157065 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f864c0b2a70 0x7f864c0b4f30 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:18.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:18.088+0000 7f866e800700 1 --2- 172.21.15.33:0/1911157065 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f866806b6f0 0x7f8668074b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:18.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:18.088+0000 7f866e800700 1 --2- 172.21.15.33:0/1911157065 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8668105ee0 0x7f86680750a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:18.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:18.088+0000 7f866e800700 1 -- 172.21.15.33:0/1911157065 >> 172.21.15.33:0/1911157065 conn(0x7f8668079350 msgr2=0x7f8668104ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:40:18.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:18.090+0000 7f866e800700 1 -- 172.21.15.33:0/1911157065 shutdown_connections 2024-05-03T18:40:18.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:18.091+0000 7f866e800700 1 -- 172.21.15.33:0/1911157065 wait complete. 2024-05-03T18:40:18.096 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 21 2024-05-03T18:40:18.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:40:17 smithi150 ceph-mon[32492]: pgmap v409: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 7.5 MiB/s wr, 513 op/s 2024-05-03T18:40:18.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:40:17 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:40:18.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:40:17 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:40:18.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:40:17 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:40:19.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:40:18 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/1911157065' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:40:19.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:40:18 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/1911157065' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:40:19.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:40:18 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/1911157065' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:40:20.049 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:40:19 smithi099 ceph-mon[31658]: pgmap v410: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 7.1 MiB/s wr, 482 op/s 2024-05-03T18:40:20.110 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 545.0 s, 563.8 tps, lat 171.434 ms stddev 62.715 2024-05-03T18:40:20.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:40:19 smithi150 ceph-mon[32492]: pgmap v410: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 7.1 MiB/s wr, 482 op/s 2024-05-03T18:40:20.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:40:19 smithi033 ceph-mon[28330]: pgmap v410: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 7.1 MiB/s wr, 482 op/s 2024-05-03T18:40:20.557 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-03T18:40:21.260 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:40:21.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:40:21 smithi033 ceph-mon[28330]: pgmap v411: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 8.8 MiB/s wr, 656 op/s 2024-05-03T18:40:22.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:40:21 smithi099 ceph-mon[31658]: pgmap v411: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 8.8 MiB/s wr, 656 op/s 2024-05-03T18:40:22.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:40:21 smithi150 ceph-mon[32492]: pgmap v411: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 8.8 MiB/s wr, 656 op/s 2024-05-03T18:40:24.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:40:23 smithi099 ceph-mon[31658]: pgmap v412: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 6.2 MiB/s wr, 436 op/s 2024-05-03T18:40:24.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:40:23 smithi150 ceph-mon[32492]: pgmap v412: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 6.2 MiB/s wr, 436 op/s 2024-05-03T18:40:24.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:40:23 smithi033 ceph-mon[28330]: pgmap v412: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 6.2 MiB/s wr, 436 op/s 2024-05-03T18:40:25.110 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 550.0 s, 521.0 tps, lat 197.586 ms stddev 323.071 2024-05-03T18:40:26.135 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:40:25 smithi150 ceph-mon[32492]: pgmap v413: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 10 MiB/s wr, 668 op/s 2024-05-03T18:40:26.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:40:25 smithi033 ceph-mon[28330]: pgmap v413: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 10 MiB/s wr, 668 op/s 2024-05-03T18:40:26.297 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:40:25 smithi099 ceph-mon[31658]: pgmap v413: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 10 MiB/s wr, 668 op/s 2024-05-03T18:40:26.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.439+0000 7f042fe00700 1 -- 172.21.15.33:0/860082598 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f043010aef0 msgr2=0x7f043010e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:40:26.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.439+0000 7f042fe00700 1 --2- 172.21.15.33:0/860082598 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f043010aef0 0x7f043010e6b0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f04200099c0 tx=0x7f0420009cd0 comp rx=0 tx=0).stop 2024-05-03T18:40:26.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.440+0000 7f042fe00700 1 -- 172.21.15.33:0/860082598 shutdown_connections 2024-05-03T18:40:26.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.440+0000 7f042fe00700 1 --2- 172.21.15.33:0/860082598 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f043010ede0 0x7f0430116900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:26.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.440+0000 7f042fe00700 1 --2- 172.21.15.33:0/860082598 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f043010a540 0x7f043010a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:26.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.440+0000 7f042fe00700 1 --2- 172.21.15.33:0/860082598 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f043010aef0 0x7f043010e6b0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:26.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.440+0000 7f042fe00700 1 -- 172.21.15.33:0/860082598 >> 172.21.15.33:0/860082598 conn(0x7f04300791d0 msgr2=0x7f043007b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:40:26.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.440+0000 7f042fe00700 1 -- 172.21.15.33:0/860082598 shutdown_connections 2024-05-03T18:40:26.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.440+0000 7f042fe00700 1 -- 172.21.15.33:0/860082598 wait complete. 2024-05-03T18:40:26.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.442+0000 7f042fe00700 1 Processor -- start 2024-05-03T18:40:26.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.442+0000 7f042fe00700 1 -- start start 2024-05-03T18:40:26.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.442+0000 7f042fe00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f043010a540 0x7f0430085e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:26.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.443+0000 7f042fe00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f043010aef0 0x7f043007dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:26.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.443+0000 7f042fe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f043010ede0 0x7f043007e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:26.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.443+0000 7f042fe00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0430119a80 con 0x7f043010aef0 2024-05-03T18:40:26.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.443+0000 7f042fe00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0430119900 con 0x7f043010ede0 2024-05-03T18:40:26.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.443+0000 7f042fe00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f0430119c00 con 0x7f043010a540 2024-05-03T18:40:26.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.443+0000 7f042ea00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f043010a540 0x7f0430085e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:26.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.443+0000 7f042ea00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f043010a540 0x7f0430085e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:53034/0 (socket says 172.21.15.33:53034) 2024-05-03T18:40:26.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.443+0000 7f042ea00700 1 -- 172.21.15.33:0/139415928 learned_addr learned my addr 172.21.15.33:0/139415928 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:40:26.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.444+0000 7f042f400700 1 --2- 172.21.15.33:0/139415928 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f043010ede0 0x7f043007e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:26.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.444+0000 7f042e000700 1 --2- 172.21.15.33:0/139415928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f043010aef0 0x7f043007dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:26.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.444+0000 7f042ea00700 1 -- 172.21.15.33:0/139415928 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f043010ede0 msgr2=0x7f043007e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:40:26.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.444+0000 7f042ea00700 1 --2- 172.21.15.33:0/139415928 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f043010ede0 0x7f043007e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:26.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.444+0000 7f042ea00700 1 -- 172.21.15.33:0/139415928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f043010aef0 msgr2=0x7f043007dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:40:26.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.444+0000 7f042ea00700 1 --2- 172.21.15.33:0/139415928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f043010aef0 0x7f043007dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:26.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.444+0000 7f042ea00700 1 -- 172.21.15.33:0/139415928 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f043007eb90 con 0x7f043010a540 2024-05-03T18:40:26.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.445+0000 7f042ea00700 1 --2- 172.21.15.33:0/139415928 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f043010a540 0x7f0430085e10 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f041c00c970 tx=0x7f041c00cc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:40:26.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.446+0000 7f0427400700 1 -- 172.21.15.33:0/139415928 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f041c004220 con 0x7f043010a540 2024-05-03T18:40:26.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.446+0000 7f042fe00700 1 -- 172.21.15.33:0/139415928 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04301b4e60 con 0x7f043010a540 2024-05-03T18:40:26.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.446+0000 7f0427400700 1 -- 172.21.15.33:0/139415928 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f041c007d30 con 0x7f043010a540 2024-05-03T18:40:26.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.447+0000 7f042fe00700 1 -- 172.21.15.33:0/139415928 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04301b5260 con 0x7f043010a540 2024-05-03T18:40:26.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.447+0000 7f0427400700 1 -- 172.21.15.33:0/139415928 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f041c003c40 con 0x7f043010a540 2024-05-03T18:40:26.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.457+0000 7f0427400700 1 -- 172.21.15.33:0/139415928 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f041c0078a0 con 0x7f043010a540 2024-05-03T18:40:26.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.460+0000 7f0427400700 1 --2- 172.21.15.33:0/139415928 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f04140b2a70 0x7f04140b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:26.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.460+0000 7f0427400700 1 -- 172.21.15.33:0/139415928 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9518+0+0 (secure 0 0 0) 0x7f041c0e2480 con 0x7f043010a540 2024-05-03T18:40:26.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.462+0000 7f042e000700 1 --2- 172.21.15.33:0/139415928 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f04140b2a70 0x7f04140b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:26.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.464+0000 7f042e000700 1 --2- 172.21.15.33:0/139415928 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f04140b2a70 0x7f04140b4f30 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f04200058e0 tx=0x7f042001a040 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:40:26.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.505+0000 7f0440506b80 1 Processor -- start 2024-05-03T18:40:26.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.505+0000 7f0440506b80 1 -- start start 2024-05-03T18:40:26.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.506+0000 7f0440506b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564524e22a40 0x564524e22e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:26.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.506+0000 7f0440506b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564524e23360 0x564524e1cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:26.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.506+0000 7f0440506b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564524e1d5d0 0x564524e1da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:26.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.506+0000 7f0440506b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x564524d6fa20 con 0x564524e23360 2024-05-03T18:40:26.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.506+0000 7f0440506b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x564524d0d760 con 0x564524e22a40 2024-05-03T18:40:26.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.506+0000 7f0440506b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x564524d6b9e0 con 0x564524e1d5d0 2024-05-03T18:40:26.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.507+0000 7f042f400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564524e1d5d0 0x564524e1da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:26.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.507+0000 7f042ea00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564524e22a40 0x564524e22e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:26.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.507+0000 7f042ea00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564524e22a40 0x564524e22e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:49700/0 (socket says 172.21.15.33:49700) 2024-05-03T18:40:26.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.507+0000 7f042ea00700 1 -- 172.21.15.33:0/1729415043 learned_addr learned my addr 172.21.15.33:0/1729415043 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:40:26.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.507+0000 7f042e000700 1 --2- 172.21.15.33:0/1729415043 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564524e23360 0x564524e1cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:26.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.508+0000 7f042e000700 1 -- 172.21.15.33:0/1729415043 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564524e1d5d0 msgr2=0x564524e1da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:40:26.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.508+0000 7f042e000700 1 --2- 172.21.15.33:0/1729415043 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564524e1d5d0 0x564524e1da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:26.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.508+0000 7f042e000700 1 -- 172.21.15.33:0/1729415043 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564524e22a40 msgr2=0x564524e22e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:40:26.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.508+0000 7f042e000700 1 --2- 172.21.15.33:0/1729415043 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564524e22a40 0x564524e22e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:26.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.508+0000 7f042e000700 1 -- 172.21.15.33:0/1729415043 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564524cbc8d0 con 0x564524e23360 2024-05-03T18:40:26.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.509+0000 7f042e000700 1 --2- 172.21.15.33:0/1729415043 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564524e23360 0x564524e1cfb0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f042000b580 tx=0x7f042001e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0f6781a46b81077 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:40:26.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.510+0000 7f041be00700 1 -- 172.21.15.33:0/1729415043 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0420033070 con 0x564524e23360 2024-05-03T18:40:26.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.510+0000 7f041be00700 1 -- 172.21.15.33:0/1729415043 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f042000bda0 con 0x564524e23360 2024-05-03T18:40:26.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.510+0000 7f041be00700 1 -- 172.21.15.33:0/1729415043 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f042002eb50 con 0x564524e23360 2024-05-03T18:40:26.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.511+0000 7f0440506b80 1 -- 172.21.15.33:0/1729415043 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564524e23360 msgr2=0x564524e1cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:40:26.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.511+0000 7f0440506b80 1 --2- 172.21.15.33:0/1729415043 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564524e23360 0x564524e1cfb0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f042000b580 tx=0x7f042001e000 comp rx=0 tx=0).stop 2024-05-03T18:40:26.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.511+0000 7f0440506b80 1 -- 172.21.15.33:0/1729415043 shutdown_connections 2024-05-03T18:40:26.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.511+0000 7f0440506b80 1 --2- 172.21.15.33:0/1729415043 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564524e1d5d0 0x564524e1da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:26.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.511+0000 7f0440506b80 1 --2- 172.21.15.33:0/1729415043 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564524e22a40 0x564524e22e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:26.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.511+0000 7f0440506b80 1 --2- 172.21.15.33:0/1729415043 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564524e23360 0x564524e1cfb0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:26.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.511+0000 7f0440506b80 1 -- 172.21.15.33:0/1729415043 >> 172.21.15.33:0/1729415043 conn(0x564524d4bb60 msgr2=0x564524e179d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:40:26.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.514+0000 7f0440506b80 1 -- 172.21.15.33:0/1729415043 shutdown_connections 2024-05-03T18:40:26.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.514+0000 7f0440506b80 1 -- 172.21.15.33:0/1729415043 wait complete. 2024-05-03T18:40:26.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.515+0000 7f0440506b80 1 -- start start 2024-05-03T18:40:26.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.515+0000 7f0440506b80 1 Processor -- start 2024-05-03T18:40:26.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.517+0000 7f0440506b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564524e1d5d0 0x564524e28fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:26.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.517+0000 7f0440506b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564524e22a40 0x564524e26380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:26.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.517+0000 7f0440506b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564524e23360 0x564524e268c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:26.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.518+0000 7f0440506b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x564524e29d70 con 0x564524e22a40 2024-05-03T18:40:26.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.518+0000 7f0440506b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x564524cd5b20 con 0x564524e23360 2024-05-03T18:40:26.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.518+0000 7f0440506b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x564524e29ef0 con 0x564524e1d5d0 2024-05-03T18:40:26.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.518+0000 7f0440506b80 10 client.0 _ll_register_callbacks cb 0x564524d16c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:40:26.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.518+0000 7f042f400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564524e23360 0x564524e268c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:26.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.519+0000 7f042f400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564524e23360 0x564524e268c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:49712/0 (socket says 172.21.15.33:49712) 2024-05-03T18:40:26.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.519+0000 7f042f400700 1 -- 172.21.15.33:0/470082082 learned_addr learned my addr 172.21.15.33:0/470082082 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:40:26.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.520+0000 7f042ea00700 1 --2- 172.21.15.33:0/470082082 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564524e1d5d0 0x564524e28fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:26.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.520+0000 7f042e000700 1 --2- 172.21.15.33:0/470082082 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564524e22a40 0x564524e26380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:26.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.520+0000 7f042f400700 1 -- 172.21.15.33:0/470082082 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564524e1d5d0 msgr2=0x564524e28fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:40:26.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.520+0000 7f042f400700 1 --2- 172.21.15.33:0/470082082 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564524e1d5d0 0x564524e28fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:26.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.520+0000 7f042f400700 1 -- 172.21.15.33:0/470082082 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564524e22a40 msgr2=0x564524e26380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:40:26.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.520+0000 7f042f400700 1 --2- 172.21.15.33:0/470082082 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564524e22a40 0x564524e26380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:26.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.520+0000 7f042f400700 1 -- 172.21.15.33:0/470082082 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564524d35420 con 0x564524e23360 2024-05-03T18:40:26.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.521+0000 7f042e000700 1 --2- 172.21.15.33:0/470082082 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564524e22a40 0x564524e26380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:40:26.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.521+0000 7f042ea00700 1 --2- 172.21.15.33:0/470082082 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564524e1d5d0 0x564524e28fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:40:26.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.521+0000 7f042f400700 1 --2- 172.21.15.33:0/470082082 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564524e23360 0x564524e268c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0428004070 tx=0x7f0428004380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c7584b69ce3c9cf server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:40:26.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.523+0000 7f0418c00700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-03T18:40:26.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.523+0000 7f0418c00700 1 -- 172.21.15.33:0/470082082 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f042801a020 con 0x564524e23360 2024-05-03T18:40:26.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.523+0000 7f0418c00700 1 -- 172.21.15.33:0/470082082 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9518+0+0 (secure 0 0 0) 0x7f04280079d0 con 0x564524e23360 2024-05-03T18:40:26.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.523+0000 7f0440506b80 1 -- 172.21.15.33:0/470082082 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f042000bda0 con 0x564524e23360 2024-05-03T18:40:26.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.523+0000 7f0418c00700 1 -- 172.21.15.33:0/470082082 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f0428007bd0 con 0x564524e23360 2024-05-03T18:40:26.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.523+0000 7f0440506b80 10 client.36317 fetch_fsmap learned FSMap version 21 2024-05-03T18:40:26.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.523+0000 7f0440506b80 1 -- 172.21.15.33:0/470082082 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x564524e34630 con 0x564524e23360 2024-05-03T18:40:26.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.524+0000 7f0418c00700 1 -- 172.21.15.33:0/470082082 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f042000bda0 con 0x564524e23360 2024-05-03T18:40:26.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.526+0000 7f0418c00700 1 -- 172.21.15.33:0/470082082 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f0428027020 con 0x564524e23360 2024-05-03T18:40:26.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.527+0000 7f0440506b80 10 client.36317 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:40:26.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.527+0000 7f0440506b80 10 client.36317 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:40:26.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.527+0000 7f0440506b80 20 client.36317 populate_metadata read hostname 'smithi033' 2024-05-03T18:40:26.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.527+0000 7f0440506b80 1 --2- 172.21.15.33:0/470082082 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x564524e36cc0 0x564524e39100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:26.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.527+0000 7f0440506b80 4 client.36317 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:40:26.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.527+0000 7f0440506b80 1 -- 172.21.15.33:0/470082082 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564524dea1d0 con 0x564524e36cc0 2024-05-03T18:40:26.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.527+0000 7f042ea00700 1 --2- 172.21.15.33:0/470082082 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x564524e36cc0 0x564524e39100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:26.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.530+0000 7f042ea00700 1 --2- 172.21.15.33:0/470082082 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x564524e36cc0 0x564524e39100 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d44ad56728d8cf05 server_cookie=bb50a97b6c0dfd87 in_seq=0 out_seq=0 2024-05-03T18:40:26.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.530+0000 7f0418c00700 10 client.36317 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:40:26.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.531+0000 7f0418c00700 1 -- 172.21.15.33:0/470082082 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564524dea1d0 con 0x564524e36cc0 2024-05-03T18:40:26.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.531+0000 7f0418c00700 10 client.36317 handle_command_reply: tid=0 2024-05-03T18:40:26.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.532+0000 7f0440506b80 1 client.36317 shutdown 2024-05-03T18:40:26.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.532+0000 7f0440506b80 1 -- 172.21.15.33:0/470082082 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564524e23360 msgr2=0x564524e268c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:40:26.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.532+0000 7f0440506b80 1 --2- 172.21.15.33:0/470082082 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564524e23360 0x564524e268c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0428004070 tx=0x7f0428004380 comp rx=0 tx=0).stop 2024-05-03T18:40:26.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.533+0000 7f0440506b80 1 -- 172.21.15.33:0/470082082 shutdown_connections 2024-05-03T18:40:26.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.533+0000 7f0440506b80 1 --2- 172.21.15.33:0/470082082 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x564524e36cc0 0x564524e39100 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:26.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.533+0000 7f0440506b80 1 --2- 172.21.15.33:0/470082082 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564524e23360 0x564524e268c0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:26.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.533+0000 7f0440506b80 1 --2- 172.21.15.33:0/470082082 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564524e1d5d0 0x564524e28fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:26.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.533+0000 7f0440506b80 1 --2- 172.21.15.33:0/470082082 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564524e22a40 0x564524e26380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:26.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.533+0000 7f0440506b80 1 -- 172.21.15.33:0/470082082 >> 172.21.15.33:0/470082082 conn(0x564524d4bb60 msgr2=0x564524e17080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:40:26.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.533+0000 7f0418c00700 0 client.36317 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:40:26.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.533+0000 7f0440506b80 1 -- 172.21.15.33:0/470082082 shutdown_connections 2024-05-03T18:40:26.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.534+0000 7f0440506b80 1 -- 172.21.15.33:0/470082082 wait complete. 2024-05-03T18:40:26.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.534+0000 7f0440506b80 20 client.36317 trim_cache size 0 max 16384 2024-05-03T18:40:26.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.566+0000 7f0440506b80 1 Processor -- start 2024-05-03T18:40:26.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.567+0000 7f0440506b80 1 -- start start 2024-05-03T18:40:26.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.568+0000 7f0440506b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564524e1d5d0 0x564524e1cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:26.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.568+0000 7f0440506b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564524e22a40 0x564524e26910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:26.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.568+0000 7f0440506b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564524e23360 0x564524e23f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:26.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.568+0000 7f0440506b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f041000cdf0 con 0x564524e1d5d0 2024-05-03T18:40:26.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.568+0000 7f0440506b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f041000f5c0 con 0x564524e22a40 2024-05-03T18:40:26.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.568+0000 7f0440506b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f041000d3b0 con 0x564524e23360 2024-05-03T18:40:26.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.569+0000 7f042ea00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564524e1d5d0 0x564524e1cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:26.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.569+0000 7f042f400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564524e23360 0x564524e23f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:26.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.569+0000 7f042f400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564524e23360 0x564524e23f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:53060/0 (socket says 172.21.15.33:53060) 2024-05-03T18:40:26.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.569+0000 7f042f400700 1 -- 172.21.15.33:0/3112235350 learned_addr learned my addr 172.21.15.33:0/3112235350 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:40:26.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.569+0000 7f042e000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564524e22a40 0x564524e26910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:26.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.570+0000 7f042f400700 1 -- 172.21.15.33:0/3112235350 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564524e22a40 msgr2=0x564524e26910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:40:26.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.570+0000 7f042f400700 1 --2- 172.21.15.33:0/3112235350 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564524e22a40 0x564524e26910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:26.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.570+0000 7f042f400700 1 -- 172.21.15.33:0/3112235350 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564524e1d5d0 msgr2=0x564524e1cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:40:26.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.570+0000 7f042f400700 1 --2- 172.21.15.33:0/3112235350 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564524e1d5d0 0x564524e1cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:26.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.570+0000 7f042f400700 1 -- 172.21.15.33:0/3112235350 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564524e23160 con 0x564524e23360 2024-05-03T18:40:26.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.570+0000 7f042ea00700 1 --2- 172.21.15.33:0/3112235350 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564524e1d5d0 0x564524e1cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:40:26.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.571+0000 7f042f400700 1 --2- 172.21.15.33:0/3112235350 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564524e23360 0x564524e23f10 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0428002230 tx=0x7f0428004850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=906f072d10c679b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:40:26.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.571+0000 7f041b400700 1 -- 172.21.15.33:0/3112235350 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f042801a070 con 0x564524e23360 2024-05-03T18:40:26.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.571+0000 7f041b400700 1 -- 172.21.15.33:0/3112235350 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f042802c020 con 0x564524e23360 2024-05-03T18:40:26.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.572+0000 7f041b400700 1 -- 172.21.15.33:0/3112235350 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0428005c60 con 0x564524e23360 2024-05-03T18:40:26.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.572+0000 7f0440506b80 1 -- 172.21.15.33:0/3112235350 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564524e23360 msgr2=0x564524e23f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:40:26.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.572+0000 7f0440506b80 1 --2- 172.21.15.33:0/3112235350 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564524e23360 0x564524e23f10 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0428002230 tx=0x7f0428004850 comp rx=0 tx=0).stop 2024-05-03T18:40:26.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.573+0000 7f0440506b80 1 -- 172.21.15.33:0/3112235350 shutdown_connections 2024-05-03T18:40:26.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.573+0000 7f0440506b80 1 --2- 172.21.15.33:0/3112235350 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564524e23360 0x564524e23f10 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:26.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.573+0000 7f0440506b80 1 --2- 172.21.15.33:0/3112235350 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564524e1d5d0 0x564524e1cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:26.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.573+0000 7f0440506b80 1 --2- 172.21.15.33:0/3112235350 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564524e22a40 0x564524e26910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:26.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.573+0000 7f0440506b80 1 -- 172.21.15.33:0/3112235350 >> 172.21.15.33:0/3112235350 conn(0x564524d4bb60 msgr2=0x564524e23800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:40:26.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.573+0000 7f0440506b80 1 -- 172.21.15.33:0/3112235350 shutdown_connections 2024-05-03T18:40:26.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.573+0000 7f0440506b80 1 -- 172.21.15.33:0/3112235350 wait complete. 2024-05-03T18:40:26.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.574+0000 7f0440506b80 1 -- start start 2024-05-03T18:40:26.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.574+0000 7f0440506b80 1 Processor -- start 2024-05-03T18:40:26.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.576+0000 7f0440506b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564524e1d5d0 0x564524e2f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:26.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.576+0000 7f0440506b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564524e22a40 0x564524e2f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:26.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.577+0000 7f0440506b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564524e23360 0x564524e3da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:26.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.577+0000 7f0440506b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f041000da80 con 0x564524e1d5d0 2024-05-03T18:40:26.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.577+0000 7f042f400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564524e23360 0x564524e3da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:26.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.577+0000 7f042f400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564524e23360 0x564524e3da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:53074/0 (socket says 172.21.15.33:53074) 2024-05-03T18:40:26.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.577+0000 7f042f400700 1 -- 172.21.15.33:0/2398107961 learned_addr learned my addr 172.21.15.33:0/2398107961 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:40:26.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.578+0000 7f0440506b80 1 -- 172.21.15.33:0/2398107961 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f041000eef0 con 0x564524e22a40 2024-05-03T18:40:26.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.578+0000 7f0440506b80 1 -- 172.21.15.33:0/2398107961 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f041000e150 con 0x564524e23360 2024-05-03T18:40:26.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.577+0000 7f042e000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564524e22a40 0x564524e2f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:26.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.578+0000 7f0440506b80 10 client.0 _ll_register_callbacks cb 0x564524d16c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:40:26.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.578+0000 7f042f400700 1 -- 172.21.15.33:0/2398107961 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564524e22a40 msgr2=0x564524e2f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:40:26.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.579+0000 7f042f400700 1 --2- 172.21.15.33:0/2398107961 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564524e22a40 0x564524e2f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:26.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.579+0000 7f042f400700 1 -- 172.21.15.33:0/2398107961 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564524e1d5d0 msgr2=0x564524e2f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:40:26.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.579+0000 7f042f400700 1 --2- 172.21.15.33:0/2398107961 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564524e1d5d0 0x564524e2f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:26.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.579+0000 7f042f400700 1 -- 172.21.15.33:0/2398107961 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564524e3df40 con 0x564524e23360 2024-05-03T18:40:26.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.579+0000 7f042e000700 1 --2- 172.21.15.33:0/2398107961 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564524e22a40 0x564524e2f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:40:26.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.580+0000 7f042f400700 1 --2- 172.21.15.33:0/2398107961 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564524e23360 0x564524e3da00 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0428003c40 tx=0x7f0428005fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2103072e178dc5d server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:40:26.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.580+0000 7f0418c00700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-03T18:40:26.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.581+0000 7f0418c00700 1 -- 172.21.15.33:0/2398107961 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0428036060 con 0x564524e23360 2024-05-03T18:40:26.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.581+0000 7f0440506b80 1 -- 172.21.15.33:0/2398107961 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f042802c020 con 0x564524e23360 2024-05-03T18:40:26.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.581+0000 7f0418c00700 1 -- 172.21.15.33:0/2398107961 <== mon.1 v2:172.21.15.150:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9518+0+0 (secure 0 0 0) 0x7f0428002030 con 0x564524e23360 2024-05-03T18:40:26.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.581+0000 7f0418c00700 1 -- 172.21.15.33:0/2398107961 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f042801a040 con 0x564524e23360 2024-05-03T18:40:26.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.582+0000 7f0440506b80 10 client.26404 fetch_fsmap learned FSMap version 21 2024-05-03T18:40:26.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.582+0000 7f0440506b80 1 -- 172.21.15.33:0/2398107961 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x564524ea0c20 con 0x564524e23360 2024-05-03T18:40:26.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.582+0000 7f0418c00700 1 -- 172.21.15.33:0/2398107961 <== mon.1 v2:172.21.15.150:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f042802c020 con 0x564524e23360 2024-05-03T18:40:26.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.582+0000 7f0418c00700 1 -- 172.21.15.33:0/2398107961 <== mon.1 v2:172.21.15.150:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f0428027030 con 0x564524e23360 2024-05-03T18:40:26.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.583+0000 7f0440506b80 10 client.26404 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:40:26.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.583+0000 7f0440506b80 10 client.26404 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:40:26.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.583+0000 7f0440506b80 20 client.26404 populate_metadata read hostname 'smithi033' 2024-05-03T18:40:26.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.583+0000 7f0440506b80 1 --2- 172.21.15.33:0/2398107961 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x564524ea1260 0x564524ea36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:26.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.583+0000 7f0440506b80 4 client.26404 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-03T18:40:26.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.583+0000 7f0440506b80 1 -- 172.21.15.33:0/2398107961 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564524d7d400 con 0x564524ea1260 2024-05-03T18:40:26.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.585+0000 7f042ea00700 1 --2- 172.21.15.33:0/2398107961 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x564524ea1260 0x564524ea36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:26.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.587+0000 7f0418c00700 10 client.26404 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:40:26.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.587+0000 7f042ea00700 1 --2- 172.21.15.33:0/2398107961 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x564524ea1260 0x564524ea36a0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b703a447c439bc0 server_cookie=80ef207422bd0d54 in_seq=0 out_seq=0 2024-05-03T18:40:26.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.587+0000 7f0418c00700 1 -- 172.21.15.33:0/2398107961 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564524d7d400 con 0x564524ea1260 2024-05-03T18:40:26.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.587+0000 7f0418c00700 10 client.26404 handle_command_reply: tid=0 2024-05-03T18:40:26.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.588+0000 7f0440506b80 1 client.26404 shutdown 2024-05-03T18:40:26.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.590+0000 7f0440506b80 1 -- 172.21.15.33:0/2398107961 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564524e23360 msgr2=0x564524e3da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:40:26.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.590+0000 7f0440506b80 1 --2- 172.21.15.33:0/2398107961 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564524e23360 0x564524e3da00 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0428003c40 tx=0x7f0428005fb0 comp rx=0 tx=0).stop 2024-05-03T18:40:26.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.590+0000 7f0440506b80 1 -- 172.21.15.33:0/2398107961 shutdown_connections 2024-05-03T18:40:26.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.590+0000 7f0440506b80 1 --2- 172.21.15.33:0/2398107961 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x564524ea1260 0x564524ea36a0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:26.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.590+0000 7f0440506b80 1 --2- 172.21.15.33:0/2398107961 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564524e23360 0x564524e3da00 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:26.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.590+0000 7f0440506b80 1 --2- 172.21.15.33:0/2398107961 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564524e1d5d0 0x564524e2f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:26.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.590+0000 7f0440506b80 1 --2- 172.21.15.33:0/2398107961 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564524e22a40 0x564524e2f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:26.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.590+0000 7f0440506b80 1 -- 172.21.15.33:0/2398107961 >> 172.21.15.33:0/2398107961 conn(0x564524d4bb60 msgr2=0x564524e214c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:40:26.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.590+0000 7f0418c00700 0 client.26404 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:40:26.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.591+0000 7f0440506b80 1 -- 172.21.15.33:0/2398107961 shutdown_connections 2024-05-03T18:40:26.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.591+0000 7f0440506b80 1 -- 172.21.15.33:0/2398107961 wait complete. 2024-05-03T18:40:26.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.591+0000 7f0440506b80 20 client.26404 trim_cache size 0 max 16384 2024-05-03T18:40:26.594 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-03T18:40:26.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.593+0000 7f042fe00700 1 -- 172.21.15.33:0/139415928 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f04140b2a70 msgr2=0x7f04140b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:40:26.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.593+0000 7f042fe00700 1 --2- 172.21.15.33:0/139415928 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f04140b2a70 0x7f04140b4f30 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f04200058e0 tx=0x7f042001a040 comp rx=0 tx=0).stop 2024-05-03T18:40:26.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.593+0000 7f042fe00700 1 -- 172.21.15.33:0/139415928 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f043010a540 msgr2=0x7f0430085e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:40:26.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.593+0000 7f042fe00700 1 --2- 172.21.15.33:0/139415928 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f043010a540 0x7f0430085e10 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f041c00c970 tx=0x7f041c00cc80 comp rx=0 tx=0).stop 2024-05-03T18:40:26.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.593+0000 7f042fe00700 1 -- 172.21.15.33:0/139415928 shutdown_connections 2024-05-03T18:40:26.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.593+0000 7f042fe00700 1 --2- 172.21.15.33:0/139415928 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f04140b2a70 0x7f04140b4f30 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:26.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.593+0000 7f042fe00700 1 --2- 172.21.15.33:0/139415928 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f043010ede0 0x7f043007e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:26.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.593+0000 7f042fe00700 1 --2- 172.21.15.33:0/139415928 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f043010a540 0x7f0430085e10 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:26.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.593+0000 7f042fe00700 1 --2- 172.21.15.33:0/139415928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f043010aef0 0x7f043007dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:26.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.593+0000 7f042fe00700 1 -- 172.21.15.33:0/139415928 >> 172.21.15.33:0/139415928 conn(0x7f04300791d0 msgr2=0x7f0430079950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:40:26.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.594+0000 7f042fe00700 1 -- 172.21.15.33:0/139415928 shutdown_connections 2024-05-03T18:40:26.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:26.594+0000 7f042fe00700 1 -- 172.21.15.33:0/139415928 wait complete. 2024-05-03T18:40:28.137 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:40:27 smithi150 ceph-mon[32492]: pgmap v414: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 6.4 MiB/s wr, 462 op/s 2024-05-03T18:40:28.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:40:27 smithi033 ceph-mon[28330]: pgmap v414: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 6.4 MiB/s wr, 462 op/s 2024-05-03T18:40:28.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:40:27 smithi099 ceph-mon[31658]: pgmap v414: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 6.4 MiB/s wr, 462 op/s 2024-05-03T18:40:28.731 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-03T18:40:28.732 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-03T18:40:29.373 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:40:30.111 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 555.0 s, 554.2 tps, lat 179.743 ms stddev 90.498 2024-05-03T18:40:30.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:40:29 smithi150 ceph-mon[32492]: pgmap v415: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 6.5 MiB/s wr, 464 op/s 2024-05-03T18:40:30.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:40:29 smithi033 ceph-mon[28330]: pgmap v415: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 6.5 MiB/s wr, 464 op/s 2024-05-03T18:40:30.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:40:29 smithi099 ceph-mon[31658]: pgmap v415: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 6.5 MiB/s wr, 464 op/s 2024-05-03T18:40:32.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:40:31 smithi150 ceph-mon[32492]: pgmap v416: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 9.2 MiB/s wr, 741 op/s 2024-05-03T18:40:32.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:40:31 smithi033 ceph-mon[28330]: pgmap v416: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 9.2 MiB/s wr, 741 op/s 2024-05-03T18:40:32.297 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:40:31 smithi099 ceph-mon[31658]: pgmap v416: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 9.2 MiB/s wr, 741 op/s 2024-05-03T18:40:33.842 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:40:33 smithi033 ceph-mon[28330]: pgmap v417: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 6.7 MiB/s wr, 512 op/s 2024-05-03T18:40:34.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:40:33 smithi150 ceph-mon[32492]: pgmap v417: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 6.7 MiB/s wr, 512 op/s 2024-05-03T18:40:34.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:40:33 smithi099 ceph-mon[31658]: pgmap v417: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 6.7 MiB/s wr, 512 op/s 2024-05-03T18:40:35.111 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 560.0 s, 537.3 tps, lat 187.667 ms stddev 77.405 2024-05-03T18:40:35.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.386+0000 7fdafea00700 1 -- 172.21.15.33:0/4236857771 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdaf806b180 msgr2=0x7fdaf806b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:40:35.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.386+0000 7fdafea00700 1 --2- 172.21.15.33:0/4236857771 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdaf806b180 0x7fdaf806b600 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fdae00098f0 tx=0x7fdae0009c00 comp rx=0 tx=0).stop 2024-05-03T18:40:35.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.387+0000 7fdafea00700 1 -- 172.21.15.33:0/4236857771 shutdown_connections 2024-05-03T18:40:35.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.387+0000 7fdafea00700 1 --2- 172.21.15.33:0/4236857771 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdaf806bb40 0x7fdaf8110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:35.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.387+0000 7fdafea00700 1 --2- 172.21.15.33:0/4236857771 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdaf8109510 0x7fdaf81098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:35.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.387+0000 7fdafea00700 1 --2- 172.21.15.33:0/4236857771 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdaf806b180 0x7fdaf806b600 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:35.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.387+0000 7fdafea00700 1 -- 172.21.15.33:0/4236857771 >> 172.21.15.33:0/4236857771 conn(0x7fdaf8079350 msgr2=0x7fdaf8079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:40:35.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.388+0000 7fdafea00700 1 -- 172.21.15.33:0/4236857771 shutdown_connections 2024-05-03T18:40:35.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.388+0000 7fdafea00700 1 -- 172.21.15.33:0/4236857771 wait complete. 2024-05-03T18:40:35.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.390+0000 7fdafea00700 1 Processor -- start 2024-05-03T18:40:35.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.390+0000 7fdafea00700 1 -- start start 2024-05-03T18:40:35.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.391+0000 7fdafea00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdaf806b180 0x7fdaf8074d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:35.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.391+0000 7fdafea00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdaf806bb40 0x7fdaf806de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:35.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.391+0000 7fdafea00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdaf8109510 0x7fdaf806e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:35.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.392+0000 7fdafea00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf8113570 con 0x7fdaf8109510 2024-05-03T18:40:35.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.392+0000 7fdafea00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf81133f0 con 0x7fdaf806b180 2024-05-03T18:40:35.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.392+0000 7fdafea00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf81136f0 con 0x7fdaf806bb40 2024-05-03T18:40:35.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.392+0000 7fdaf7e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdaf806b180 0x7fdaf8074d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:35.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.392+0000 7fdafca00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdaf8109510 0x7fdaf806e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:35.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.392+0000 7fdafca00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdaf8109510 0x7fdaf806e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46506/0 (socket says 172.21.15.33:46506) 2024-05-03T18:40:35.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.392+0000 7fdafca00700 1 -- 172.21.15.33:0/2279124733 learned_addr learned my addr 172.21.15.33:0/2279124733 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:40:35.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.392+0000 7fdaf7e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdaf806b180 0x7fdaf8074d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:49746/0 (socket says 172.21.15.33:49746) 2024-05-03T18:40:35.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.393+0000 7fdafca00700 1 -- 172.21.15.33:0/2279124733 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdaf806bb40 msgr2=0x7fdaf806de20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-03T18:40:35.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.393+0000 7fdafca00700 1 --2- 172.21.15.33:0/2279124733 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdaf806bb40 0x7fdaf806de20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:35.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.393+0000 7fdafca00700 1 -- 172.21.15.33:0/2279124733 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdaf806b180 msgr2=0x7fdaf8074d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:40:35.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.394+0000 7fdafca00700 1 --2- 172.21.15.33:0/2279124733 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdaf806b180 0x7fdaf8074d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:35.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.394+0000 7fdafca00700 1 -- 172.21.15.33:0/2279124733 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaf806ea60 con 0x7fdaf8109510 2024-05-03T18:40:35.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.394+0000 7fdaf7e00700 1 --2- 172.21.15.33:0/2279124733 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdaf806b180 0x7fdaf8074d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:40:35.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.395+0000 7fdafca00700 1 --2- 172.21.15.33:0/2279124733 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdaf8109510 0x7fdaf806e360 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fdaf000e970 tx=0x7fdaf000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:40:35.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.395+0000 7fdaf5600700 1 -- 172.21.15.33:0/2279124733 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf000cbd0 con 0x7fdaf8109510 2024-05-03T18:40:35.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.395+0000 7fdaf5600700 1 -- 172.21.15.33:0/2279124733 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdaf000ee50 con 0x7fdaf8109510 2024-05-03T18:40:35.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.395+0000 7fdaf5600700 1 -- 172.21.15.33:0/2279124733 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf0005550 con 0x7fdaf8109510 2024-05-03T18:40:35.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.395+0000 7fdafea00700 1 -- 172.21.15.33:0/2279124733 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaf80728c0 con 0x7fdaf8109510 2024-05-03T18:40:35.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.395+0000 7fdafea00700 1 -- 172.21.15.33:0/2279124733 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaf8072db0 con 0x7fdaf8109510 2024-05-03T18:40:35.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.401+0000 7fdaf5600700 1 -- 172.21.15.33:0/2279124733 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fdaf0004750 con 0x7fdaf8109510 2024-05-03T18:40:35.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.404+0000 7fdaf5600700 1 --2- 172.21.15.33:0/2279124733 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fdadc0b2b80 0x7fdadc0b5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:35.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.404+0000 7fdaf5600700 1 -- 172.21.15.33:0/2279124733 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9518+0+0 (secure 0 0 0) 0x7fdaf00e2b60 con 0x7fdaf8109510 2024-05-03T18:40:35.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.405+0000 7fdaf7e00700 1 --2- 172.21.15.33:0/2279124733 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fdadc0b2b80 0x7fdadc0b5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:35.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.407+0000 7fdaf7e00700 1 --2- 172.21.15.33:0/2279124733 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fdadc0b2b80 0x7fdadc0b5040 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fdae8009c50 tx=0x7fdae8009040 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:40:35.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.437+0000 7fdb0946db80 1 Processor -- start 2024-05-03T18:40:35.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.437+0000 7fdb0946db80 1 -- start start 2024-05-03T18:40:35.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.438+0000 7fdb0946db80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ca0333830 0x559ca0333c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:35.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.438+0000 7fdb0946db80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559ca0334150 0x559ca032f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:35.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.438+0000 7fdb0946db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ca032f650 0x559ca032fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:35.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.438+0000 7fdb0946db80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x559ca0281aa0 con 0x559ca0333830 2024-05-03T18:40:35.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.438+0000 7fdb0946db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x559ca021f7e0 con 0x559ca032f650 2024-05-03T18:40:35.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.438+0000 7fdb0946db80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x559ca027da60 con 0x559ca0334150 2024-05-03T18:40:35.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.438+0000 7fdaef400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559ca0334150 0x559ca032f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:35.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.438+0000 7fdaef400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559ca0334150 0x559ca032f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:53106/0 (socket says 172.21.15.33:53106) 2024-05-03T18:40:35.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.438+0000 7fdaef400700 1 -- 172.21.15.33:0/934161965 learned_addr learned my addr 172.21.15.33:0/934161965 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:40:35.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.439+0000 7fdaf7e00700 1 --2- 172.21.15.33:0/934161965 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ca0333830 0x559ca0333c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:35.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.439+0000 7fdafca00700 1 --2- 172.21.15.33:0/934161965 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ca032f650 0x559ca032fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:35.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.439+0000 7fdaef400700 1 -- 172.21.15.33:0/934161965 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ca032f650 msgr2=0x559ca032fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:40:35.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.440+0000 7fdaef400700 1 --2- 172.21.15.33:0/934161965 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ca032f650 0x559ca032fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:35.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.440+0000 7fdaef400700 1 -- 172.21.15.33:0/934161965 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ca0333830 msgr2=0x559ca0333c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:40:35.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.440+0000 7fdaef400700 1 --2- 172.21.15.33:0/934161965 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ca0333830 0x559ca0333c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:35.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.440+0000 7fdaef400700 1 -- 172.21.15.33:0/934161965 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ca01ce950 con 0x559ca0334150 2024-05-03T18:40:35.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.440+0000 7fdaef400700 1 --2- 172.21.15.33:0/934161965 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559ca0334150 0x559ca032f030 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fdae0009230 tx=0x7fdae00095d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14c7ff255960484f server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:40:35.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.440+0000 7fdaed600700 1 -- 172.21.15.33:0/934161965 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae0010c70 con 0x559ca0334150 2024-05-03T18:40:35.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.441+0000 7fdaed600700 1 -- 172.21.15.33:0/934161965 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdae000f8f0 con 0x559ca0334150 2024-05-03T18:40:35.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.442+0000 7fdb0946db80 1 -- 172.21.15.33:0/934161965 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559ca0334150 msgr2=0x559ca032f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:40:35.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.443+0000 7fdb0946db80 1 --2- 172.21.15.33:0/934161965 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559ca0334150 0x559ca032f030 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fdae0009230 tx=0x7fdae00095d0 comp rx=0 tx=0).stop 2024-05-03T18:40:35.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.443+0000 7fdaed600700 1 -- 172.21.15.33:0/934161965 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae0003680 con 0x559ca0334150 2024-05-03T18:40:35.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.443+0000 7fdb0946db80 1 -- 172.21.15.33:0/934161965 shutdown_connections 2024-05-03T18:40:35.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.443+0000 7fdb0946db80 1 --2- 172.21.15.33:0/934161965 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ca032f650 0x559ca032fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:35.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.443+0000 7fdb0946db80 1 --2- 172.21.15.33:0/934161965 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ca0333830 0x559ca0333c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:35.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.443+0000 7fdb0946db80 1 --2- 172.21.15.33:0/934161965 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559ca0334150 0x559ca032f030 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:35.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.443+0000 7fdb0946db80 1 -- 172.21.15.33:0/934161965 >> 172.21.15.33:0/934161965 conn(0x559ca025dbe0 msgr2=0x559ca0329870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:40:35.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.443+0000 7fdb0946db80 1 -- 172.21.15.33:0/934161965 shutdown_connections 2024-05-03T18:40:35.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.443+0000 7fdb0946db80 1 -- 172.21.15.33:0/934161965 wait complete. 2024-05-03T18:40:35.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.444+0000 7fdb0946db80 1 -- start start 2024-05-03T18:40:35.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.444+0000 7fdb0946db80 1 Processor -- start 2024-05-03T18:40:35.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.446+0000 7fdb0946db80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ca032f650 0x559ca033aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:35.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.446+0000 7fdb0946db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ca0333830 0x559ca03382e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:35.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.446+0000 7fdb0946db80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559ca0334150 0x559ca0338820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:35.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.446+0000 7fdb0946db80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x559ca033bcc0 con 0x559ca032f650 2024-05-03T18:40:35.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.446+0000 7fdb0946db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x559ca01e7ba0 con 0x559ca0333830 2024-05-03T18:40:35.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.446+0000 7fdb0946db80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x559ca033be40 con 0x559ca0334150 2024-05-03T18:40:35.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.446+0000 7fdb0946db80 10 client.0 _ll_register_callbacks cb 0x559ca0228cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:40:35.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.447+0000 7fdaf7e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ca032f650 0x559ca033aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:35.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.447+0000 7fdaf7e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ca032f650 0x559ca033aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46532/0 (socket says 172.21.15.33:46532) 2024-05-03T18:40:35.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.447+0000 7fdaf7e00700 1 -- 172.21.15.33:0/2222559052 learned_addr learned my addr 172.21.15.33:0/2222559052 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:40:35.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.447+0000 7fdafca00700 1 --2- 172.21.15.33:0/2222559052 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559ca0334150 0x559ca0338820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:35.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.447+0000 7fdaef400700 1 --2- 172.21.15.33:0/2222559052 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ca0333830 0x559ca03382e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:35.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.448+0000 7fdaf7e00700 1 -- 172.21.15.33:0/2222559052 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559ca0334150 msgr2=0x559ca0338820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:40:35.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.448+0000 7fdaf7e00700 1 --2- 172.21.15.33:0/2222559052 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559ca0334150 0x559ca0338820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:35.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.448+0000 7fdaf7e00700 1 -- 172.21.15.33:0/2222559052 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ca0333830 msgr2=0x559ca03382e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:40:35.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.448+0000 7fdaf7e00700 1 --2- 172.21.15.33:0/2222559052 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ca0333830 0x559ca03382e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:35.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.448+0000 7fdaf7e00700 1 -- 172.21.15.33:0/2222559052 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ca02474a0 con 0x559ca032f650 2024-05-03T18:40:35.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.448+0000 7fdaef400700 1 --2- 172.21.15.33:0/2222559052 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ca0333830 0x559ca03382e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:40:35.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.448+0000 7fdafca00700 1 --2- 172.21.15.33:0/2222559052 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559ca0334150 0x559ca0338820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:40:35.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.448+0000 7fdaf7e00700 1 --2- 172.21.15.33:0/2222559052 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ca032f650 0x559ca033aef0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fdae8016e20 tx=0x7fdae8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faabbec658488c24 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:40:35.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.448+0000 7fdae6000700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:40:35.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.449+0000 7fdae6000700 1 -- 172.21.15.33:0/2222559052 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae8027060 con 0x559ca032f650 2024-05-03T18:40:35.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.449+0000 7fdb0946db80 1 -- 172.21.15.33:0/2222559052 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdae000f8f0 con 0x559ca032f650 2024-05-03T18:40:35.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.449+0000 7fdae6000700 1 -- 172.21.15.33:0/2222559052 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9518+0+0 (secure 0 0 0) 0x7fdae8015d50 con 0x559ca032f650 2024-05-03T18:40:35.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.449+0000 7fdae6000700 1 -- 172.21.15.33:0/2222559052 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fdae802e050 con 0x559ca032f650 2024-05-03T18:40:35.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.449+0000 7fdb0946db80 10 client.16620 fetch_fsmap learned FSMap version 21 2024-05-03T18:40:35.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.449+0000 7fdb0946db80 1 -- 172.21.15.33:0/2222559052 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x559ca034a8f0 con 0x559ca032f650 2024-05-03T18:40:35.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.450+0000 7fdae6000700 1 -- 172.21.15.33:0/2222559052 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdae000f8f0 con 0x559ca032f650 2024-05-03T18:40:35.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.450+0000 7fdae6000700 1 -- 172.21.15.33:0/2222559052 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fdae8021070 con 0x559ca032f650 2024-05-03T18:40:35.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.450+0000 7fdb0946db80 10 client.16620 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:40:35.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.450+0000 7fdb0946db80 10 client.16620 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:40:35.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.451+0000 7fdb0946db80 20 client.16620 populate_metadata read hostname 'smithi033' 2024-05-03T18:40:35.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.451+0000 7fdb0946db80 1 --2- 172.21.15.33:0/2222559052 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x559ca034cf30 0x559ca034f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:35.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.451+0000 7fdb0946db80 4 client.16620 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:40:35.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.451+0000 7fdb0946db80 1 -- 172.21.15.33:0/2222559052 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559ca02fc250 con 0x559ca034cf30 2024-05-03T18:40:35.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.451+0000 7fdaef400700 1 --2- 172.21.15.33:0/2222559052 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x559ca034cf30 0x559ca034f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:35.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.452+0000 7fdae6000700 10 client.16620 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:40:35.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.454+0000 7fdaef400700 1 --2- 172.21.15.33:0/2222559052 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x559ca034cf30 0x559ca034f360 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30d2a08fe50ccf01 server_cookie=a33486cd0e2c92db in_seq=0 out_seq=0 2024-05-03T18:40:35.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.454+0000 7fdae6000700 1 -- 172.21.15.33:0/2222559052 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559ca02fc250 con 0x559ca034cf30 2024-05-03T18:40:35.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.454+0000 7fdae6000700 10 client.16620 handle_command_reply: tid=0 2024-05-03T18:40:35.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.455+0000 7fdb0946db80 1 client.16620 shutdown 2024-05-03T18:40:35.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.455+0000 7fdb0946db80 1 -- 172.21.15.33:0/2222559052 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ca032f650 msgr2=0x559ca033aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:40:35.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.455+0000 7fdb0946db80 1 --2- 172.21.15.33:0/2222559052 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ca032f650 0x559ca033aef0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fdae8016e20 tx=0x7fdae8004ad0 comp rx=0 tx=0).stop 2024-05-03T18:40:35.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.456+0000 7fdb0946db80 1 -- 172.21.15.33:0/2222559052 shutdown_connections 2024-05-03T18:40:35.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.456+0000 7fdb0946db80 1 --2- 172.21.15.33:0/2222559052 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x559ca034cf30 0x559ca034f360 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:35.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.456+0000 7fdb0946db80 1 --2- 172.21.15.33:0/2222559052 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559ca0334150 0x559ca0338820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:35.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.456+0000 7fdb0946db80 1 --2- 172.21.15.33:0/2222559052 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ca032f650 0x559ca033aef0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:35.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.456+0000 7fdb0946db80 1 --2- 172.21.15.33:0/2222559052 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ca0333830 0x559ca03382e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:35.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.456+0000 7fdb0946db80 1 -- 172.21.15.33:0/2222559052 >> 172.21.15.33:0/2222559052 conn(0x559ca025dbe0 msgr2=0x559ca0334e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:40:35.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.456+0000 7fdae6000700 0 client.16620 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:40:35.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.456+0000 7fdb0946db80 1 -- 172.21.15.33:0/2222559052 shutdown_connections 2024-05-03T18:40:35.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.457+0000 7fdb0946db80 1 -- 172.21.15.33:0/2222559052 wait complete. 2024-05-03T18:40:35.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.457+0000 7fdb0946db80 20 client.16620 trim_cache size 0 max 16384 2024-05-03T18:40:35.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.502+0000 7fdb0946db80 1 Processor -- start 2024-05-03T18:40:35.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.503+0000 7fdb0946db80 1 -- start start 2024-05-03T18:40:35.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.503+0000 7fdb0946db80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559ca032f650 0x559ca03296e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:35.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.503+0000 7fdb0946db80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ca0333830 0x559ca0338420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:35.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.503+0000 7fdb0946db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ca0334150 0x559ca0354c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:35.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.503+0000 7fdb0946db80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf81b39f0 con 0x559ca0333830 2024-05-03T18:40:35.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.503+0000 7fdb0946db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf81b6250 con 0x559ca0334150 2024-05-03T18:40:35.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.504+0000 7fdb0946db80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf81b4040 con 0x559ca032f650 2024-05-03T18:40:35.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.504+0000 7fdafca00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ca0334150 0x559ca0354c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:35.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.504+0000 7fdafca00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ca0334150 0x559ca0354c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:49754/0 (socket says 172.21.15.33:49754) 2024-05-03T18:40:35.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.504+0000 7fdafca00700 1 -- 172.21.15.33:0/2828518673 learned_addr learned my addr 172.21.15.33:0/2828518673 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:40:35.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.504+0000 7fdaef400700 1 --2- 172.21.15.33:0/2828518673 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ca0333830 0x559ca0338420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:35.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.504+0000 7fdaf7e00700 1 --2- 172.21.15.33:0/2828518673 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559ca032f650 0x559ca03296e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:35.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.505+0000 7fdaf7e00700 1 -- 172.21.15.33:0/2828518673 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ca0334150 msgr2=0x559ca0354c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:40:35.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.505+0000 7fdaf7e00700 1 --2- 172.21.15.33:0/2828518673 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ca0334150 0x559ca0354c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:35.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.505+0000 7fdaf7e00700 1 -- 172.21.15.33:0/2828518673 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ca0333830 msgr2=0x559ca0338420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:40:35.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.505+0000 7fdaf7e00700 1 --2- 172.21.15.33:0/2828518673 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ca0333830 0x559ca0338420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:35.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.505+0000 7fdaf7e00700 1 -- 172.21.15.33:0/2828518673 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ca03372a0 con 0x559ca032f650 2024-05-03T18:40:35.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.505+0000 7fdaf7e00700 1 --2- 172.21.15.33:0/2828518673 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559ca032f650 0x559ca03296e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fdae8000f80 tx=0x7fdae80218e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1651c59d4bc31093 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:40:35.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.506+0000 7fdaecc00700 1 -- 172.21.15.33:0/2828518673 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae8027040 con 0x559ca032f650 2024-05-03T18:40:35.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.506+0000 7fdaecc00700 1 -- 172.21.15.33:0/2828518673 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdae8005e60 con 0x559ca032f650 2024-05-03T18:40:35.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.506+0000 7fdaecc00700 1 -- 172.21.15.33:0/2828518673 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae8021c80 con 0x559ca032f650 2024-05-03T18:40:35.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.507+0000 7fdb0946db80 1 -- 172.21.15.33:0/2828518673 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559ca032f650 msgr2=0x559ca03296e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:40:35.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.507+0000 7fdb0946db80 1 --2- 172.21.15.33:0/2828518673 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559ca032f650 0x559ca03296e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fdae8000f80 tx=0x7fdae80218e0 comp rx=0 tx=0).stop 2024-05-03T18:40:35.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.508+0000 7fdb0946db80 1 -- 172.21.15.33:0/2828518673 shutdown_connections 2024-05-03T18:40:35.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.508+0000 7fdb0946db80 1 --2- 172.21.15.33:0/2828518673 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ca0334150 0x559ca0354c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:35.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.508+0000 7fdb0946db80 1 --2- 172.21.15.33:0/2828518673 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559ca032f650 0x559ca03296e0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:35.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.508+0000 7fdb0946db80 1 --2- 172.21.15.33:0/2828518673 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ca0333830 0x559ca0338420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:35.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.508+0000 7fdb0946db80 1 -- 172.21.15.33:0/2828518673 >> 172.21.15.33:0/2828518673 conn(0x559ca025dbe0 msgr2=0x559ca0332ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:40:35.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.508+0000 7fdb0946db80 1 -- 172.21.15.33:0/2828518673 shutdown_connections 2024-05-03T18:40:35.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.508+0000 7fdb0946db80 1 -- 172.21.15.33:0/2828518673 wait complete. 2024-05-03T18:40:35.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.509+0000 7fdb0946db80 1 -- start start 2024-05-03T18:40:35.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.509+0000 7fdb0946db80 1 Processor -- start 2024-05-03T18:40:35.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.510+0000 7fdb0946db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ca032f650 0x559ca0349130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:35.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.511+0000 7fdb0946db80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559ca0333830 0x559ca0341110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:35.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.511+0000 7fdb0946db80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ca0334150 0x559ca03444b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:35.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.511+0000 7fdaf7e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ca032f650 0x559ca0349130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:35.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.511+0000 7fdaf7e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ca032f650 0x559ca0349130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:49762/0 (socket says 172.21.15.33:49762) 2024-05-03T18:40:35.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.511+0000 7fdaef400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559ca0333830 0x559ca0341110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:35.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.511+0000 7fdaef400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559ca0333830 0x559ca0341110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:53128/0 (socket says 172.21.15.33:53128) 2024-05-03T18:40:35.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.512+0000 7fdafca00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ca0334150 0x559ca03444b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:35.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.512+0000 7fdb0946db80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf81b4710 con 0x559ca0334150 2024-05-03T18:40:35.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.512+0000 7fdb0946db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf81b5b80 con 0x559ca032f650 2024-05-03T18:40:35.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.512+0000 7fdb0946db80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf81b4de0 con 0x559ca0333830 2024-05-03T18:40:35.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.512+0000 7fdb0946db80 10 client.0 _ll_register_callbacks cb 0x559ca0228cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:40:35.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.513+0000 7fdafca00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ca0334150 0x559ca03444b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46546/0 (socket says 172.21.15.33:46546) 2024-05-03T18:40:35.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.513+0000 7fdafca00700 1 -- 172.21.15.33:0/4155755139 learned_addr learned my addr 172.21.15.33:0/4155755139 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:40:35.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.515+0000 7fdaf7e00700 1 -- 172.21.15.33:0/4155755139 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559ca0333830 msgr2=0x559ca0341110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:40:35.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.516+0000 7fdaf7e00700 1 --2- 172.21.15.33:0/4155755139 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559ca0333830 0x559ca0341110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:35.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.516+0000 7fdaf7e00700 1 -- 172.21.15.33:0/4155755139 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ca0334150 msgr2=0x559ca03444b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:40:35.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.516+0000 7fdaf7e00700 1 --2- 172.21.15.33:0/4155755139 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ca0334150 0x559ca03444b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:35.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.516+0000 7fdaf7e00700 1 -- 172.21.15.33:0/4155755139 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ca0341b50 con 0x559ca032f650 2024-05-03T18:40:35.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.516+0000 7fdafca00700 1 --2- 172.21.15.33:0/4155755139 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ca0334150 0x559ca03444b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:40:35.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.517+0000 7fdaf7e00700 1 --2- 172.21.15.33:0/4155755139 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ca032f650 0x559ca0349130 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fdae80213e0 tx=0x7fdae8009480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80c204a180764acd server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:40:35.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.517+0000 7fdae6000700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-03T18:40:35.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.520+0000 7fdae6000700 1 -- 172.21.15.33:0/4155755139 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae8036020 con 0x559ca032f650 2024-05-03T18:40:35.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.520+0000 7fdae6000700 1 -- 172.21.15.33:0/4155755139 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9518+0+0 (secure 0 0 0) 0x7fdae8015a20 con 0x559ca032f650 2024-05-03T18:40:35.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.520+0000 7fdb0946db80 1 -- 172.21.15.33:0/4155755139 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdae8005e60 con 0x559ca032f650 2024-05-03T18:40:35.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.521+0000 7fdae6000700 1 -- 172.21.15.33:0/4155755139 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fdae8041050 con 0x559ca032f650 2024-05-03T18:40:35.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.521+0000 7fdb0946db80 10 client.36335 fetch_fsmap learned FSMap version 21 2024-05-03T18:40:35.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.521+0000 7fdb0946db80 1 -- 172.21.15.33:0/4155755139 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x559ca03bc5b0 con 0x559ca032f650 2024-05-03T18:40:35.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.523+0000 7fdae6000700 1 -- 172.21.15.33:0/4155755139 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdae8005e60 con 0x559ca032f650 2024-05-03T18:40:35.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.523+0000 7fdae6000700 1 -- 172.21.15.33:0/4155755139 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fdae8032590 con 0x559ca032f650 2024-05-03T18:40:35.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.523+0000 7fdb0946db80 10 client.36335 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:40:35.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.523+0000 7fdb0946db80 10 client.36335 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:40:35.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.523+0000 7fdb0946db80 20 client.36335 populate_metadata read hostname 'smithi033' 2024-05-03T18:40:35.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.523+0000 7fdb0946db80 1 --2- 172.21.15.33:0/4155755139 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x559ca03bcb70 0x559ca03bef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:35.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.523+0000 7fdb0946db80 4 client.36335 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-03T18:40:35.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.523+0000 7fdb0946db80 1 -- 172.21.15.33:0/4155755139 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559ca028f480 con 0x559ca03bcb70 2024-05-03T18:40:35.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.526+0000 7fdaef400700 1 --2- 172.21.15.33:0/4155755139 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x559ca03bcb70 0x559ca03bef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:35.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.529+0000 7fdaef400700 1 --2- 172.21.15.33:0/4155755139 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x559ca03bcb70 0x559ca03bef90 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=517a496020e6afec server_cookie=9cd349bdf3d1f754 in_seq=0 out_seq=0 2024-05-03T18:40:35.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.529+0000 7fdae6000700 10 client.36335 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:40:35.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.531+0000 7fdae6000700 1 -- 172.21.15.33:0/4155755139 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559ca028f480 con 0x559ca03bcb70 2024-05-03T18:40:35.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.531+0000 7fdae6000700 10 client.36335 handle_command_reply: tid=0 2024-05-03T18:40:35.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.531+0000 7fdb0946db80 1 client.36335 shutdown 2024-05-03T18:40:35.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.532+0000 7fdb0946db80 1 -- 172.21.15.33:0/4155755139 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ca032f650 msgr2=0x559ca0349130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:40:35.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.532+0000 7fdb0946db80 1 --2- 172.21.15.33:0/4155755139 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ca032f650 0x559ca0349130 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fdae80213e0 tx=0x7fdae8009480 comp rx=0 tx=0).stop 2024-05-03T18:40:35.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.532+0000 7fdb0946db80 1 -- 172.21.15.33:0/4155755139 shutdown_connections 2024-05-03T18:40:35.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.532+0000 7fdb0946db80 1 --2- 172.21.15.33:0/4155755139 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x559ca03bcb70 0x559ca03bef90 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:35.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.532+0000 7fdb0946db80 1 --2- 172.21.15.33:0/4155755139 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ca0334150 0x559ca03444b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:35.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.532+0000 7fdb0946db80 1 --2- 172.21.15.33:0/4155755139 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ca032f650 0x559ca0349130 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:35.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.532+0000 7fdb0946db80 1 --2- 172.21.15.33:0/4155755139 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559ca0333830 0x559ca0341110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:35.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.532+0000 7fdb0946db80 1 -- 172.21.15.33:0/4155755139 >> 172.21.15.33:0/4155755139 conn(0x559ca025dbe0 msgr2=0x559ca032ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:40:35.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.532+0000 7fdae6000700 0 client.36335 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:40:35.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.533+0000 7fdb0946db80 1 -- 172.21.15.33:0/4155755139 shutdown_connections 2024-05-03T18:40:35.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.533+0000 7fdb0946db80 1 -- 172.21.15.33:0/4155755139 wait complete. 2024-05-03T18:40:35.536 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-03T18:40:35.536 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-05-03T18:40:35.536 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "6ea6969b-445b-4f90-b1a8-2cb9a1b651ff", 2024-05-03T18:40:35.536 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-05-03T18:40:35.536 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-03T18:40:35.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.533+0000 7fdb0946db80 20 client.36335 trim_cache size 0 max 16384 2024-05-03T18:40:35.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.535+0000 7fdafea00700 1 -- 172.21.15.33:0/2279124733 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fdadc0b2b80 msgr2=0x7fdadc0b5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:40:35.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.535+0000 7fdafea00700 1 --2- 172.21.15.33:0/2279124733 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fdadc0b2b80 0x7fdadc0b5040 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fdae8009c50 tx=0x7fdae8009040 comp rx=0 tx=0).stop 2024-05-03T18:40:35.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.535+0000 7fdafea00700 1 -- 172.21.15.33:0/2279124733 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdaf8109510 msgr2=0x7fdaf806e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:40:35.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.535+0000 7fdafea00700 1 --2- 172.21.15.33:0/2279124733 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdaf8109510 0x7fdaf806e360 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fdaf000e970 tx=0x7fdaf000ed30 comp rx=0 tx=0).stop 2024-05-03T18:40:35.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.535+0000 7fdafea00700 1 -- 172.21.15.33:0/2279124733 shutdown_connections 2024-05-03T18:40:35.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.535+0000 7fdafea00700 1 --2- 172.21.15.33:0/2279124733 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdaf8109510 0x7fdaf806e360 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:35.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.535+0000 7fdafea00700 1 --2- 172.21.15.33:0/2279124733 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fdadc0b2b80 0x7fdadc0b5040 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:35.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.535+0000 7fdafea00700 1 --2- 172.21.15.33:0/2279124733 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdaf806b180 0x7fdaf8074d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:35.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.535+0000 7fdafea00700 1 --2- 172.21.15.33:0/2279124733 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdaf806bb40 0x7fdaf806de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:35.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.535+0000 7fdafea00700 1 -- 172.21.15.33:0/2279124733 >> 172.21.15.33:0/2279124733 conn(0x7fdaf8079350 msgr2=0x7fdaf8101140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:40:35.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.536+0000 7fdafea00700 1 -- 172.21.15.33:0/2279124733 shutdown_connections 2024-05-03T18:40:35.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:35.536+0000 7fdafea00700 1 -- 172.21.15.33:0/2279124733 wait complete. 2024-05-03T18:40:35.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:40:35 smithi033 ceph-mon[28330]: pgmap v418: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 578 KiB/s rd, 9.3 MiB/s wr, 699 op/s 2024-05-03T18:40:36.138 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:40:35 smithi150 ceph-mon[32492]: pgmap v418: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 578 KiB/s rd, 9.3 MiB/s wr, 699 op/s 2024-05-03T18:40:36.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:40:35 smithi099 ceph-mon[31658]: pgmap v418: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 578 KiB/s rd, 9.3 MiB/s wr, 699 op/s 2024-05-03T18:40:36.580 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-03T18:40:36.865 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:40:36 smithi033 ceph-mon[28330]: scrub queued for path: / 2024-05-03T18:40:36.865 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:40:36 smithi033 ceph-mon[28330]: scrub summary: idle+waiting paths [/] 2024-05-03T18:40:36.866 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:40:36 smithi033 ceph-mon[28330]: scrub summary: active paths [/] 2024-05-03T18:40:36.866 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:40:36 smithi033 ceph-mon[28330]: scrub summary: idle+waiting paths [/] 2024-05-03T18:40:36.866 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:40:36 smithi033 ceph-mon[28330]: scrub summary: idle 2024-05-03T18:40:37.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:40:36 smithi150 ceph-mon[32492]: scrub queued for path: / 2024-05-03T18:40:37.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:40:36 smithi150 ceph-mon[32492]: scrub summary: idle+waiting paths [/] 2024-05-03T18:40:37.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:40:36 smithi150 ceph-mon[32492]: scrub summary: active paths [/] 2024-05-03T18:40:37.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:40:36 smithi150 ceph-mon[32492]: scrub summary: idle+waiting paths [/] 2024-05-03T18:40:37.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:40:36 smithi150 ceph-mon[32492]: scrub summary: idle 2024-05-03T18:40:37.145 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:40:37.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:40:36 smithi099 ceph-mon[31658]: scrub queued for path: / 2024-05-03T18:40:37.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:40:36 smithi099 ceph-mon[31658]: scrub summary: idle+waiting paths [/] 2024-05-03T18:40:37.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:40:36 smithi099 ceph-mon[31658]: scrub summary: active paths [/] 2024-05-03T18:40:37.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:40:36 smithi099 ceph-mon[31658]: scrub summary: idle+waiting paths [/] 2024-05-03T18:40:37.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:40:36 smithi099 ceph-mon[31658]: scrub summary: idle 2024-05-03T18:40:37.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:40:37 smithi033 ceph-mon[28330]: pgmap v419: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 548 KiB/s rd, 5.5 MiB/s wr, 467 op/s 2024-05-03T18:40:38.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:40:37 smithi150 ceph-mon[32492]: pgmap v419: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 548 KiB/s rd, 5.5 MiB/s wr, 467 op/s 2024-05-03T18:40:38.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:40:37 smithi099 ceph-mon[31658]: pgmap v419: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 548 KiB/s rd, 5.5 MiB/s wr, 467 op/s 2024-05-03T18:40:40.112 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 565.0 s, 637.1 tps, lat 155.453 ms stddev 91.951 2024-05-03T18:40:40.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:40:39 smithi150 ceph-mon[32492]: pgmap v420: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 677 KiB/s rd, 5.6 MiB/s wr, 469 op/s 2024-05-03T18:40:40.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:40:39 smithi033 ceph-mon[28330]: pgmap v420: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 677 KiB/s rd, 5.6 MiB/s wr, 469 op/s 2024-05-03T18:40:40.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:40:39 smithi099 ceph-mon[31658]: pgmap v420: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 677 KiB/s rd, 5.6 MiB/s wr, 469 op/s 2024-05-03T18:40:41.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.568+0000 7f2bfb200700 1 -- 172.21.15.33:0/3350692871 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2bf406efe0 msgr2=0x7f2bf406f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:40:41.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.568+0000 7f2bfb200700 1 --2- 172.21.15.33:0/3350692871 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2bf406efe0 0x7f2bf406f3c0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f2be4009990 tx=0x7f2be4009ca0 comp rx=0 tx=0).stop 2024-05-03T18:40:41.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.568+0000 7f2bfb200700 1 -- 172.21.15.33:0/3350692871 shutdown_connections 2024-05-03T18:40:41.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.568+0000 7f2bfb200700 1 --2- 172.21.15.33:0/3350692871 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2bf411cb50 0x7f2bf411ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:41.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.568+0000 7f2bfb200700 1 --2- 172.21.15.33:0/3350692871 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2bf406efe0 0x7f2bf406f3c0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:41.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.568+0000 7f2bfb200700 1 --2- 172.21.15.33:0/3350692871 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2bf406f900 0x7f2bf411c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:41.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.568+0000 7f2bfb200700 1 -- 172.21.15.33:0/3350692871 >> 172.21.15.33:0/3350692871 conn(0x7f2bf406e6c0 msgr2=0x7f2bf410c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:40:41.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.569+0000 7f2bfb200700 1 -- 172.21.15.33:0/3350692871 shutdown_connections 2024-05-03T18:40:41.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.569+0000 7f2bfb200700 1 -- 172.21.15.33:0/3350692871 wait complete. 2024-05-03T18:40:41.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.570+0000 7f2bfb200700 1 Processor -- start 2024-05-03T18:40:41.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.571+0000 7f2bfb200700 1 -- start start 2024-05-03T18:40:41.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.571+0000 7f2bfb200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2bf406efe0 0x7f2bf41ae720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:41.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.572+0000 7f2bfb200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2bf406f900 0x7f2bf41aec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:41.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.572+0000 7f2bfb200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2bf411cb50 0x7f2bf41a88e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:41.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.572+0000 7f2bfb200700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf41224f0 con 0x7f2bf411cb50 2024-05-03T18:40:41.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.572+0000 7f2bfb200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf4122370 con 0x7f2bf406efe0 2024-05-03T18:40:41.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.572+0000 7f2bfb200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf4122670 con 0x7f2bf406f900 2024-05-03T18:40:41.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.572+0000 7f2bf3e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2bf406efe0 0x7f2bf41ae720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:41.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.572+0000 7f2bf9200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2bf411cb50 0x7f2bf41a88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:41.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.572+0000 7f2bf3e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2bf406efe0 0x7f2bf41ae720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:41382/0 (socket says 172.21.15.33:41382) 2024-05-03T18:40:41.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.572+0000 7f2bf3e00700 1 -- 172.21.15.33:0/3892319881 learned_addr learned my addr 172.21.15.33:0/3892319881 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:40:41.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.573+0000 7f2bf9200700 1 -- 172.21.15.33:0/3892319881 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2bf406f900 msgr2=0x7f2bf41aec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:40:41.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.573+0000 7f2bf9200700 1 --2- 172.21.15.33:0/3892319881 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2bf406f900 0x7f2bf41aec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:41.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.573+0000 7f2bf9200700 1 -- 172.21.15.33:0/3892319881 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2bf406efe0 msgr2=0x7f2bf41ae720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:40:41.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.573+0000 7f2bf9200700 1 --2- 172.21.15.33:0/3892319881 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2bf406efe0 0x7f2bf41ae720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:41.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.573+0000 7f2bf9200700 1 -- 172.21.15.33:0/3892319881 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bf41a9040 con 0x7f2bf411cb50 2024-05-03T18:40:41.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.574+0000 7f2bf9200700 1 --2- 172.21.15.33:0/3892319881 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2bf411cb50 0x7f2bf41a88e0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f2bf40732d0 tx=0x7f2bec009e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:40:41.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.576+0000 7f2bf0c00700 1 -- 172.21.15.33:0/3892319881 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bec019070 con 0x7f2bf411cb50 2024-05-03T18:40:41.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.576+0000 7f2bfb200700 1 -- 172.21.15.33:0/3892319881 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bf41a9330 con 0x7f2bf411cb50 2024-05-03T18:40:41.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.576+0000 7f2bf0c00700 1 -- 172.21.15.33:0/3892319881 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2bec014440 con 0x7f2bf411cb50 2024-05-03T18:40:41.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.576+0000 7f2bfb200700 1 -- 172.21.15.33:0/3892319881 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bf41b55f0 con 0x7f2bf411cb50 2024-05-03T18:40:41.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.577+0000 7f2bf0c00700 1 -- 172.21.15.33:0/3892319881 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bec019070 con 0x7f2bf411cb50 2024-05-03T18:40:41.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.582+0000 7f2c05c8cb80 1 Processor -- start 2024-05-03T18:40:41.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.584+0000 7f2bf0c00700 1 -- 172.21.15.33:0/3892319881 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f2bec014610 con 0x7f2bf411cb50 2024-05-03T18:40:41.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.584+0000 7f2c05c8cb80 1 -- start start 2024-05-03T18:40:41.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.584+0000 7f2c05c8cb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e61cc4d7b0 0x55e61cc4db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:41.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.584+0000 7f2c05c8cb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e61cc4e0d0 0x55e61cc48fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:41.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.584+0000 7f2c05c8cb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e61cc495d0 0x55e61cc49a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:41.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.584+0000 7f2c05c8cb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e61cb9ba20 con 0x55e61cc495d0 2024-05-03T18:40:41.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.584+0000 7f2c05c8cb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55e61cb39760 con 0x55e61cc4e0d0 2024-05-03T18:40:41.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.584+0000 7f2c05c8cb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55e61cb979e0 con 0x55e61cc4d7b0 2024-05-03T18:40:41.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.587+0000 7f2bf3e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e61cc4e0d0 0x55e61cc48fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:41.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.587+0000 7f2bf3e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e61cc4e0d0 0x55e61cc48fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:41390/0 (socket says 172.21.15.33:41390) 2024-05-03T18:40:41.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.587+0000 7f2bf3e00700 1 -- 172.21.15.33:0/2523038364 learned_addr learned my addr 172.21.15.33:0/2523038364 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:40:41.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.588+0000 7f2bf3e00700 1 -- 172.21.15.33:0/2523038364 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e61cc4d7b0 msgr2=0x55e61cc4db90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-03T18:40:41.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.588+0000 7f2bf3e00700 1 --2- 172.21.15.33:0/2523038364 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e61cc4d7b0 0x55e61cc4db90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:41.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.588+0000 7f2bf3e00700 1 -- 172.21.15.33:0/2523038364 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e61cc495d0 msgr2=0x55e61cc49a30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-03T18:40:41.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.588+0000 7f2bf0c00700 1 --2- 172.21.15.33:0/3892319881 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f2bd80b2ec0 0x7f2bd80b5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:41.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.589+0000 7f2bf0c00700 1 -- 172.21.15.33:0/3892319881 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9518+0+0 (secure 0 0 0) 0x7f2bec00c930 con 0x7f2bf411cb50 2024-05-03T18:40:41.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.591+0000 7f2bf3400700 1 --2- 172.21.15.33:0/2523038364 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e61cc495d0 0x55e61cc49a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:41.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.591+0000 7f2bf3e00700 1 --2- 172.21.15.33:0/2523038364 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e61cc495d0 0x55e61cc49a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:41.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.591+0000 7f2bf3e00700 1 -- 172.21.15.33:0/2523038364 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e61cae88d0 con 0x55e61cc4e0d0 2024-05-03T18:40:41.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.592+0000 7f2bf3400700 1 --2- 172.21.15.33:0/2523038364 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e61cc495d0 0x55e61cc49a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:40:41.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.592+0000 7f2bf3e00700 1 --2- 172.21.15.33:0/2523038364 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e61cc4e0d0 0x55e61cc48fb0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2be4003760 tx=0x7f2be4003790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c079a16a60c53c8f server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:40:41.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.593+0000 7f2be9600700 1 -- 172.21.15.33:0/2523038364 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be4026070 con 0x55e61cc4e0d0 2024-05-03T18:40:41.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.593+0000 7f2be9600700 1 -- 172.21.15.33:0/2523038364 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2be4003850 con 0x55e61cc4e0d0 2024-05-03T18:40:41.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.593+0000 7f2be9600700 1 -- 172.21.15.33:0/2523038364 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be4005570 con 0x55e61cc4e0d0 2024-05-03T18:40:41.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.594+0000 7f2bf3400700 1 --2- 172.21.15.33:0/3892319881 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f2bd80b2ec0 0x7f2bd80b5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:41.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.594+0000 7f2c05c8cb80 1 -- 172.21.15.33:0/2523038364 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e61cc4e0d0 msgr2=0x55e61cc48fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:40:41.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.594+0000 7f2c05c8cb80 1 --2- 172.21.15.33:0/2523038364 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e61cc4e0d0 0x55e61cc48fb0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2be4003760 tx=0x7f2be4003790 comp rx=0 tx=0).stop 2024-05-03T18:40:41.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.595+0000 7f2c05c8cb80 1 -- 172.21.15.33:0/2523038364 shutdown_connections 2024-05-03T18:40:41.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.595+0000 7f2c05c8cb80 1 --2- 172.21.15.33:0/2523038364 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e61cc495d0 0x55e61cc49a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:41.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.595+0000 7f2c05c8cb80 1 --2- 172.21.15.33:0/2523038364 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e61cc4d7b0 0x55e61cc4db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:41.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.595+0000 7f2c05c8cb80 1 --2- 172.21.15.33:0/2523038364 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e61cc4e0d0 0x55e61cc48fb0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:41.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.595+0000 7f2c05c8cb80 1 -- 172.21.15.33:0/2523038364 >> 172.21.15.33:0/2523038364 conn(0x55e61cb77b60 msgr2=0x55e61cc437f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:40:41.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.597+0000 7f2c05c8cb80 1 -- 172.21.15.33:0/2523038364 shutdown_connections 2024-05-03T18:40:41.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.597+0000 7f2bf3400700 1 --2- 172.21.15.33:0/3892319881 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f2bd80b2ec0 0x7f2bd80b5380 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2bf4072680 tx=0x7f2bdc000fd0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:40:41.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.597+0000 7f2c05c8cb80 1 -- 172.21.15.33:0/2523038364 wait complete. 2024-05-03T18:40:41.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.598+0000 7f2c05c8cb80 1 -- start start 2024-05-03T18:40:41.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.600+0000 7f2c05c8cb80 1 Processor -- start 2024-05-03T18:40:41.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.601+0000 7f2c05c8cb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e61cc495d0 0x55e61cc54e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:41.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.601+0000 7f2c05c8cb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e61cc4d7b0 0x55e61cc521d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:41.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.601+0000 7f2c05c8cb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e61cc4e0d0 0x55e61cc52710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:41.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.601+0000 7f2c05c8cb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e61cc55c40 con 0x55e61cc4d7b0 2024-05-03T18:40:41.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.601+0000 7f2c05c8cb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55e61cb01b20 con 0x55e61cc4e0d0 2024-05-03T18:40:41.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.601+0000 7f2c05c8cb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55e61cc55dc0 con 0x55e61cc495d0 2024-05-03T18:40:41.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.602+0000 7f2c05c8cb80 10 client.0 _ll_register_callbacks cb 0x55e61cb42c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:40:41.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.602+0000 7f2bf9200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e61cc4e0d0 0x55e61cc52710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:41.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.602+0000 7f2bf3e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e61cc495d0 0x55e61cc54e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:41.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.602+0000 7f2bf3e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e61cc495d0 0x55e61cc54e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:55752/0 (socket says 172.21.15.33:55752) 2024-05-03T18:40:41.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.602+0000 7f2bf3e00700 1 -- 172.21.15.33:0/1350747843 learned_addr learned my addr 172.21.15.33:0/1350747843 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:40:41.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.603+0000 7f2bf3400700 1 --2- 172.21.15.33:0/1350747843 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e61cc4d7b0 0x55e61cc521d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:41.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.603+0000 7f2bf9200700 1 -- 172.21.15.33:0/1350747843 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e61cc495d0 msgr2=0x55e61cc54e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:40:41.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.603+0000 7f2bf9200700 1 --2- 172.21.15.33:0/1350747843 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e61cc495d0 0x55e61cc54e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:41.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.603+0000 7f2bf9200700 1 -- 172.21.15.33:0/1350747843 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e61cc4d7b0 msgr2=0x55e61cc521d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:40:41.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.604+0000 7f2bf9200700 1 --2- 172.21.15.33:0/1350747843 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e61cc4d7b0 0x55e61cc521d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:41.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.604+0000 7f2bf9200700 1 -- 172.21.15.33:0/1350747843 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e61cb61420 con 0x55e61cc4e0d0 2024-05-03T18:40:41.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.604+0000 7f2bf3400700 1 --2- 172.21.15.33:0/1350747843 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e61cc4d7b0 0x55e61cc521d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:40:41.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.605+0000 7f2bf9200700 1 --2- 172.21.15.33:0/1350747843 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e61cc4e0d0 0x55e61cc52710 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2bec0de700 tx=0x7f2bec0a7a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e05b5c2f9aaa9ff server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:40:41.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.605+0000 7f2be2000700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-03T18:40:41.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.608+0000 7f2be2000700 1 -- 172.21.15.33:0/1350747843 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bec019030 con 0x55e61cc4e0d0 2024-05-03T18:40:41.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.608+0000 7f2be2000700 1 -- 172.21.15.33:0/1350747843 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9518+0+0 (secure 0 0 0) 0x7f2bec00cd40 con 0x55e61cc4e0d0 2024-05-03T18:40:41.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.608+0000 7f2c05c8cb80 1 -- 172.21.15.33:0/1350747843 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2be4003850 con 0x55e61cc4e0d0 2024-05-03T18:40:41.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.612+0000 7f2be2000700 1 -- 172.21.15.33:0/1350747843 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f2bec0a8280 con 0x55e61cc4e0d0 2024-05-03T18:40:41.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.612+0000 7f2c05c8cb80 10 client.36344 fetch_fsmap learned FSMap version 21 2024-05-03T18:40:41.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.612+0000 7f2c05c8cb80 1 -- 172.21.15.33:0/1350747843 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55e61cc603f0 con 0x55e61cc4e0d0 2024-05-03T18:40:41.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.612+0000 7f2be2000700 1 -- 172.21.15.33:0/1350747843 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2be4003850 con 0x55e61cc4e0d0 2024-05-03T18:40:41.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.613+0000 7f2be2000700 1 -- 172.21.15.33:0/1350747843 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f2bec01c030 con 0x55e61cc4e0d0 2024-05-03T18:40:41.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.613+0000 7f2c05c8cb80 10 client.36344 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:40:41.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.613+0000 7f2c05c8cb80 10 client.36344 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:40:41.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.613+0000 7f2c05c8cb80 20 client.36344 populate_metadata read hostname 'smithi033' 2024-05-03T18:40:41.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.613+0000 7f2c05c8cb80 1 --2- 172.21.15.33:0/1350747843 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55e61cc62ea0 0x55e61cc65300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:41.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.613+0000 7f2c05c8cb80 4 client.36344 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:40:41.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.613+0000 7f2c05c8cb80 1 -- 172.21.15.33:0/1350747843 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e61cc161d0 con 0x55e61cc62ea0 2024-05-03T18:40:41.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.614+0000 7f2bf3e00700 1 --2- 172.21.15.33:0/1350747843 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55e61cc62ea0 0x55e61cc65300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:41.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.615+0000 7f2bf3e00700 1 --2- 172.21.15.33:0/1350747843 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55e61cc62ea0 0x55e61cc65300 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8f4b39b2ea84ef4 server_cookie=6479882dbbd254f5 in_seq=0 out_seq=0 2024-05-03T18:40:41.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.615+0000 7f2be2000700 10 client.36344 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:40:41.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.617+0000 7f2be2000700 1 -- 172.21.15.33:0/1350747843 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e61cc161d0 con 0x55e61cc62ea0 2024-05-03T18:40:41.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.617+0000 7f2be2000700 10 client.36344 handle_command_reply: tid=0 2024-05-03T18:40:41.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.618+0000 7f2c05c8cb80 1 client.36344 shutdown 2024-05-03T18:40:41.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.619+0000 7f2c05c8cb80 1 -- 172.21.15.33:0/1350747843 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e61cc4e0d0 msgr2=0x55e61cc52710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:40:41.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.619+0000 7f2c05c8cb80 1 --2- 172.21.15.33:0/1350747843 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e61cc4e0d0 0x55e61cc52710 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2bec0de700 tx=0x7f2bec0a7a60 comp rx=0 tx=0).stop 2024-05-03T18:40:41.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.619+0000 7f2c05c8cb80 1 -- 172.21.15.33:0/1350747843 shutdown_connections 2024-05-03T18:40:41.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.619+0000 7f2c05c8cb80 1 --2- 172.21.15.33:0/1350747843 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55e61cc62ea0 0x55e61cc65300 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:41.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.619+0000 7f2c05c8cb80 1 --2- 172.21.15.33:0/1350747843 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e61cc4e0d0 0x55e61cc52710 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:41.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.619+0000 7f2c05c8cb80 1 --2- 172.21.15.33:0/1350747843 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e61cc495d0 0x55e61cc54e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:41.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.619+0000 7f2c05c8cb80 1 --2- 172.21.15.33:0/1350747843 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e61cc4d7b0 0x55e61cc521d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:41.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.619+0000 7f2be2000700 0 client.36344 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:40:41.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.619+0000 7f2c05c8cb80 1 -- 172.21.15.33:0/1350747843 >> 172.21.15.33:0/1350747843 conn(0x55e61cb77b60 msgr2=0x55e61cc4ee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:40:41.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.620+0000 7f2c05c8cb80 1 -- 172.21.15.33:0/1350747843 shutdown_connections 2024-05-03T18:40:41.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.621+0000 7f2c05c8cb80 1 -- 172.21.15.33:0/1350747843 wait complete. 2024-05-03T18:40:41.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.621+0000 7f2c05c8cb80 20 client.36344 trim_cache size 0 max 16384 2024-05-03T18:40:41.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.664+0000 7f2c05c8cb80 1 Processor -- start 2024-05-03T18:40:41.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.664+0000 7f2c05c8cb80 1 -- start start 2024-05-03T18:40:41.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.665+0000 7f2c05c8cb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e61cc495d0 0x55e61cc5ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:41.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.665+0000 7f2c05c8cb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e61cc4d7b0 0x55e61cc50ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:41.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.665+0000 7f2c05c8cb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e61cc4e0d0 0x55e61cc51000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:41.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.665+0000 7f2c05c8cb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf41bc340 con 0x55e61cc4d7b0 2024-05-03T18:40:41.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.665+0000 7f2c05c8cb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf41beba0 con 0x55e61cc4e0d0 2024-05-03T18:40:41.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.665+0000 7f2c05c8cb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf41bc990 con 0x55e61cc495d0 2024-05-03T18:40:41.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.665+0000 7f2bf3e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e61cc495d0 0x55e61cc5ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:41.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.665+0000 7f2bf3e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e61cc495d0 0x55e61cc5ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:55760/0 (socket says 172.21.15.33:55760) 2024-05-03T18:40:41.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.665+0000 7f2bf3e00700 1 -- 172.21.15.33:0/1819874698 learned_addr learned my addr 172.21.15.33:0/1819874698 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:40:41.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.666+0000 7f2bf3400700 1 --2- 172.21.15.33:0/1819874698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e61cc4d7b0 0x55e61cc50ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:41.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.666+0000 7f2bf9200700 1 --2- 172.21.15.33:0/1819874698 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e61cc4e0d0 0x55e61cc51000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:41.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.666+0000 7f2bf3e00700 1 -- 172.21.15.33:0/1819874698 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e61cc4e0d0 msgr2=0x55e61cc51000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:40:41.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.666+0000 7f2bf3e00700 1 --2- 172.21.15.33:0/1819874698 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e61cc4e0d0 0x55e61cc51000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:41.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.666+0000 7f2bf3e00700 1 -- 172.21.15.33:0/1819874698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e61cc4d7b0 msgr2=0x55e61cc50ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:40:41.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.666+0000 7f2bf3e00700 1 --2- 172.21.15.33:0/1819874698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e61cc4d7b0 0x55e61cc50ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:41.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.667+0000 7f2bf3e00700 1 -- 172.21.15.33:0/1819874698 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e61cc55dd0 con 0x55e61cc495d0 2024-05-03T18:40:41.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.668+0000 7f2bf3e00700 1 --2- 172.21.15.33:0/1819874698 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e61cc495d0 0x55e61cc5ed00 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f2be400b880 tx=0x7f2be4005040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=286c4c109a3b1b08 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:40:41.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.669+0000 7f2be8c00700 1 -- 172.21.15.33:0/1819874698 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be4033070 con 0x55e61cc495d0 2024-05-03T18:40:41.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.669+0000 7f2be8c00700 1 -- 172.21.15.33:0/1819874698 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2be400be40 con 0x55e61cc495d0 2024-05-03T18:40:41.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.669+0000 7f2be8c00700 1 -- 172.21.15.33:0/1819874698 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be4026040 con 0x55e61cc495d0 2024-05-03T18:40:41.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.670+0000 7f2c05c8cb80 1 -- 172.21.15.33:0/1819874698 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e61cc495d0 msgr2=0x55e61cc5ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:40:41.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.670+0000 7f2c05c8cb80 1 --2- 172.21.15.33:0/1819874698 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e61cc495d0 0x55e61cc5ed00 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f2be400b880 tx=0x7f2be4005040 comp rx=0 tx=0).stop 2024-05-03T18:40:41.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.670+0000 7f2c05c8cb80 1 -- 172.21.15.33:0/1819874698 shutdown_connections 2024-05-03T18:40:41.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.670+0000 7f2c05c8cb80 1 --2- 172.21.15.33:0/1819874698 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e61cc4e0d0 0x55e61cc51000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:41.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.670+0000 7f2c05c8cb80 1 --2- 172.21.15.33:0/1819874698 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e61cc495d0 0x55e61cc5ed00 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:41.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.670+0000 7f2c05c8cb80 1 --2- 172.21.15.33:0/1819874698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e61cc4d7b0 0x55e61cc50ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:41.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.670+0000 7f2c05c8cb80 1 -- 172.21.15.33:0/1819874698 >> 172.21.15.33:0/1819874698 conn(0x55e61cb77b60 msgr2=0x55e61cc52190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:40:41.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.671+0000 7f2c05c8cb80 1 -- 172.21.15.33:0/1819874698 shutdown_connections 2024-05-03T18:40:41.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.671+0000 7f2c05c8cb80 1 -- 172.21.15.33:0/1819874698 wait complete. 2024-05-03T18:40:41.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.672+0000 7f2c05c8cb80 1 -- start start 2024-05-03T18:40:41.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.672+0000 7f2c05c8cb80 1 Processor -- start 2024-05-03T18:40:41.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.673+0000 7f2c05c8cb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e61cc495d0 0x55e61cc5b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:41.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.673+0000 7f2c05c8cb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e61cc4d7b0 0x55e61cc4f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:41.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.673+0000 7f2c05c8cb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e61cc4e0d0 0x55e61cc4fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:41.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.673+0000 7f2c05c8cb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf41bd060 con 0x55e61cc495d0 2024-05-03T18:40:41.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.673+0000 7f2c05c8cb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf41be4d0 con 0x55e61cc4e0d0 2024-05-03T18:40:41.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.673+0000 7f2c05c8cb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf41bd730 con 0x55e61cc4d7b0 2024-05-03T18:40:41.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.673+0000 7f2c05c8cb80 10 client.0 _ll_register_callbacks cb 0x55e61cb42c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:40:41.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.674+0000 7f2bf9200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e61cc4e0d0 0x55e61cc4fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:41.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.674+0000 7f2bf9200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e61cc4e0d0 0x55e61cc4fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:41422/0 (socket says 172.21.15.33:41422) 2024-05-03T18:40:41.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.674+0000 7f2bf9200700 1 -- 172.21.15.33:0/4213563466 learned_addr learned my addr 172.21.15.33:0/4213563466 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:40:41.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.674+0000 7f2bf3400700 1 --2- 172.21.15.33:0/4213563466 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e61cc4d7b0 0x55e61cc4f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:41.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.676+0000 7f2bf3e00700 1 --2- 172.21.15.33:0/4213563466 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e61cc495d0 0x55e61cc5b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:41.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.677+0000 7f2bf9200700 1 -- 172.21.15.33:0/4213563466 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e61cc4d7b0 msgr2=0x55e61cc4f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:40:41.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.677+0000 7f2bf9200700 1 --2- 172.21.15.33:0/4213563466 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e61cc4d7b0 0x55e61cc4f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:41.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.677+0000 7f2bf9200700 1 -- 172.21.15.33:0/4213563466 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e61cc495d0 msgr2=0x55e61cc5b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:40:41.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.677+0000 7f2bf9200700 1 --2- 172.21.15.33:0/4213563466 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e61cc495d0 0x55e61cc5b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:41.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.677+0000 7f2bf9200700 1 -- 172.21.15.33:0/4213563466 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e61cc46930 con 0x55e61cc4e0d0 2024-05-03T18:40:41.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.679+0000 7f2bf9200700 1 --2- 172.21.15.33:0/4213563466 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e61cc4e0d0 0x55e61cc4fcd0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2bec0a4300 tx=0x7f2bec0a3e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a92ada875438fabc server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:40:41.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.679+0000 7f2be2000700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-03T18:40:41.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.680+0000 7f2be2000700 1 -- 172.21.15.33:0/4213563466 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bec06b020 con 0x55e61cc4e0d0 2024-05-03T18:40:41.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.681+0000 7f2be2000700 1 -- 172.21.15.33:0/4213563466 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9518+0+0 (secure 0 0 0) 0x7f2bec0a7160 con 0x55e61cc4e0d0 2024-05-03T18:40:41.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.681+0000 7f2c05c8cb80 1 -- 172.21.15.33:0/4213563466 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2be400be40 con 0x55e61cc4e0d0 2024-05-03T18:40:41.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.682+0000 7f2be2000700 1 -- 172.21.15.33:0/4213563466 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f2bec019070 con 0x55e61cc4e0d0 2024-05-03T18:40:41.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.682+0000 7f2c05c8cb80 10 client.36350 fetch_fsmap learned FSMap version 21 2024-05-03T18:40:41.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.682+0000 7f2c05c8cb80 1 -- 172.21.15.33:0/4213563466 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55e61cccdaa0 con 0x55e61cc4e0d0 2024-05-03T18:40:41.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.683+0000 7f2be2000700 1 -- 172.21.15.33:0/4213563466 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2be400be40 con 0x55e61cc4e0d0 2024-05-03T18:40:41.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.685+0000 7f2be2000700 1 -- 172.21.15.33:0/4213563466 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f2bec01c070 con 0x55e61cc4e0d0 2024-05-03T18:40:41.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.686+0000 7f2c05c8cb80 10 client.36350 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:40:41.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.686+0000 7f2c05c8cb80 10 client.36350 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:40:41.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.686+0000 7f2c05c8cb80 20 client.36350 populate_metadata read hostname 'smithi033' 2024-05-03T18:40:41.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.686+0000 7f2c05c8cb80 1 --2- 172.21.15.33:0/4213563466 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55e61cccdf80 0x55e61ccd03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:41.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.686+0000 7f2c05c8cb80 4 client.36350 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-03T18:40:41.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.686+0000 7f2c05c8cb80 1 -- 172.21.15.33:0/4213563466 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e61cba9400 con 0x55e61cccdf80 2024-05-03T18:40:41.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.687+0000 7f2bf3e00700 1 --2- 172.21.15.33:0/4213563466 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55e61cccdf80 0x55e61ccd03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:41.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.689+0000 7f2bf3e00700 1 --2- 172.21.15.33:0/4213563466 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55e61cccdf80 0x55e61ccd03a0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7238aedd85dfa1ef server_cookie=38c0044129af7cf2 in_seq=0 out_seq=0 2024-05-03T18:40:41.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.689+0000 7f2be2000700 10 client.36350 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:40:41.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.689+0000 7f2be2000700 1 -- 172.21.15.33:0/4213563466 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55e61cba9400 con 0x55e61cccdf80 2024-05-03T18:40:41.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.689+0000 7f2be2000700 10 client.36350 handle_command_reply: tid=0 2024-05-03T18:40:41.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.689+0000 7f2c05c8cb80 1 client.36350 shutdown 2024-05-03T18:40:41.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.689+0000 7f2c05c8cb80 1 -- 172.21.15.33:0/4213563466 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e61cc4e0d0 msgr2=0x55e61cc4fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:40:41.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.689+0000 7f2c05c8cb80 1 --2- 172.21.15.33:0/4213563466 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e61cc4e0d0 0x55e61cc4fcd0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2bec0a4300 tx=0x7f2bec0a3e50 comp rx=0 tx=0).stop 2024-05-03T18:40:41.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.690+0000 7f2c05c8cb80 1 -- 172.21.15.33:0/4213563466 shutdown_connections 2024-05-03T18:40:41.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.690+0000 7f2c05c8cb80 1 --2- 172.21.15.33:0/4213563466 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55e61cccdf80 0x55e61ccd03a0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:41.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.690+0000 7f2c05c8cb80 1 --2- 172.21.15.33:0/4213563466 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e61cc4e0d0 0x55e61cc4fcd0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:41.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.690+0000 7f2be2000700 0 client.36350 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:40:41.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.690+0000 7f2c05c8cb80 1 --2- 172.21.15.33:0/4213563466 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e61cc495d0 0x55e61cc5b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:41.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.690+0000 7f2c05c8cb80 1 --2- 172.21.15.33:0/4213563466 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e61cc4d7b0 0x55e61cc4f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:41.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.690+0000 7f2c05c8cb80 1 -- 172.21.15.33:0/4213563466 >> 172.21.15.33:0/4213563466 conn(0x55e61cb77b60 msgr2=0x55e61cc50a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:40:41.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.691+0000 7f2c05c8cb80 1 -- 172.21.15.33:0/4213563466 shutdown_connections 2024-05-03T18:40:41.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.692+0000 7f2c05c8cb80 1 -- 172.21.15.33:0/4213563466 wait complete. 2024-05-03T18:40:41.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.692+0000 7f2c05c8cb80 20 client.36350 trim_cache size 0 max 16384 2024-05-03T18:40:41.695 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-03T18:40:41.695 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-05-03T18:40:41.696 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-05-03T18:40:41.696 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-03T18:40:41.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.694+0000 7f2bfb200700 1 -- 172.21.15.33:0/3892319881 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f2bd80b2ec0 msgr2=0x7f2bd80b5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:40:41.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.694+0000 7f2bfb200700 1 --2- 172.21.15.33:0/3892319881 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f2bd80b2ec0 0x7f2bd80b5380 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2bf4072680 tx=0x7f2bdc000fd0 comp rx=0 tx=0).stop 2024-05-03T18:40:41.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.694+0000 7f2bfb200700 1 -- 172.21.15.33:0/3892319881 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2bf411cb50 msgr2=0x7f2bf41a88e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:40:41.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.694+0000 7f2bfb200700 1 --2- 172.21.15.33:0/3892319881 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2bf411cb50 0x7f2bf41a88e0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f2bf40732d0 tx=0x7f2bec009e20 comp rx=0 tx=0).stop 2024-05-03T18:40:41.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.695+0000 7f2bfb200700 1 -- 172.21.15.33:0/3892319881 shutdown_connections 2024-05-03T18:40:41.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.695+0000 7f2bfb200700 1 --2- 172.21.15.33:0/3892319881 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2bf411cb50 0x7f2bf41a88e0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:41.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.695+0000 7f2bfb200700 1 --2- 172.21.15.33:0/3892319881 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f2bd80b2ec0 0x7f2bd80b5380 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:41.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.695+0000 7f2bfb200700 1 --2- 172.21.15.33:0/3892319881 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2bf406efe0 0x7f2bf41ae720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:41.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.695+0000 7f2bfb200700 1 --2- 172.21.15.33:0/3892319881 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2bf406f900 0x7f2bf41aec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:41.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.695+0000 7f2bfb200700 1 -- 172.21.15.33:0/3892319881 >> 172.21.15.33:0/3892319881 conn(0x7f2bf406e6c0 msgr2=0x7f2bf410c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:40:41.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.696+0000 7f2bfb200700 1 -- 172.21.15.33:0/3892319881 shutdown_connections 2024-05-03T18:40:41.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:41.696+0000 7f2bfb200700 1 -- 172.21.15.33:0/3892319881 wait complete. 2024-05-03T18:40:42.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:40:41 smithi150 ceph-mon[32492]: pgmap v421: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 551 KiB/s rd, 8.3 MiB/s wr, 728 op/s 2024-05-03T18:40:42.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:40:41 smithi033 ceph-mon[28330]: pgmap v421: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 551 KiB/s rd, 8.3 MiB/s wr, 728 op/s 2024-05-03T18:40:42.297 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:40:41 smithi099 ceph-mon[31658]: pgmap v421: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 551 KiB/s rd, 8.3 MiB/s wr, 728 op/s 2024-05-03T18:40:43.263 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-03T18:40:43.264 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T18:40:43.983 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:40:44.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:40:43 smithi033 ceph-mon[28330]: pgmap v422: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 5.5 MiB/s wr, 449 op/s 2024-05-03T18:40:44.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:40:43 smithi099 ceph-mon[31658]: pgmap v422: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 5.5 MiB/s wr, 449 op/s 2024-05-03T18:40:44.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:40:43 smithi150 ceph-mon[32492]: pgmap v422: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 5.5 MiB/s wr, 449 op/s 2024-05-03T18:40:45.111 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 570.0 s, 551.5 tps, lat 181.791 ms stddev 92.372 2024-05-03T18:40:46.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:40:45 smithi033 ceph-mon[28330]: pgmap v423: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 7.6 MiB/s wr, 657 op/s 2024-05-03T18:40:46.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:40:45 smithi099 ceph-mon[31658]: pgmap v423: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 7.6 MiB/s wr, 657 op/s 2024-05-03T18:40:46.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:40:45 smithi150 ceph-mon[32492]: pgmap v423: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 7.6 MiB/s wr, 657 op/s 2024-05-03T18:40:48.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:40:47 smithi033 ceph-mon[28330]: pgmap v424: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 5.0 MiB/s wr, 470 op/s 2024-05-03T18:40:48.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:40:47 smithi099 ceph-mon[31658]: pgmap v424: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 5.0 MiB/s wr, 470 op/s 2024-05-03T18:40:48.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:40:47 smithi150 ceph-mon[32492]: pgmap v424: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 5.0 MiB/s wr, 470 op/s 2024-05-03T18:40:49.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:49.313+0000 7fd0cfe00700 1 -- 172.21.15.33:0/1288272437 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0c8094f80 msgr2=0x7fd0c80a65b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:40:49.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:49.313+0000 7fd0cfe00700 1 --2- 172.21.15.33:0/1288272437 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0c8094f80 0x7fd0c80a65b0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc009990 tx=0x7fd0bc009ca0 comp rx=0 tx=0).stop 2024-05-03T18:40:49.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:49.314+0000 7fd0cfe00700 1 -- 172.21.15.33:0/1288272437 shutdown_connections 2024-05-03T18:40:49.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:49.314+0000 7fd0cfe00700 1 --2- 172.21.15.33:0/1288272437 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd0c80a6af0 0x7fd0c80a8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:49.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:49.314+0000 7fd0cfe00700 1 --2- 172.21.15.33:0/1288272437 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0c8094660 0x7fd0c8094a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:49.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:49.314+0000 7fd0cfe00700 1 --2- 172.21.15.33:0/1288272437 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0c8094f80 0x7fd0c80a65b0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:49.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:49.314+0000 7fd0cfe00700 1 -- 172.21.15.33:0/1288272437 >> 172.21.15.33:0/1288272437 conn(0x7fd0c8009f20 msgr2=0x7fd0c800a330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:40:49.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:49.314+0000 7fd0cfe00700 1 -- 172.21.15.33:0/1288272437 shutdown_connections 2024-05-03T18:40:49.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:49.314+0000 7fd0cfe00700 1 -- 172.21.15.33:0/1288272437 wait complete. 2024-05-03T18:40:49.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:49.315+0000 7fd0cfe00700 1 Processor -- start 2024-05-03T18:40:49.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:49.316+0000 7fd0cfe00700 1 -- start start 2024-05-03T18:40:49.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:49.316+0000 7fd0cfe00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd0c8094660 0x7fd0c8138610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:49.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:49.316+0000 7fd0cfe00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0c8094f80 0x7fd0c8138b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:49.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:49.316+0000 7fd0cfe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0c80a6af0 0x7fd0c8132790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:49.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:49.316+0000 7fd0cfe00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c80ac490 con 0x7fd0c8094f80 2024-05-03T18:40:49.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:49.316+0000 7fd0cfe00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c80ac310 con 0x7fd0c80a6af0 2024-05-03T18:40:49.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:49.316+0000 7fd0cfe00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c80ac610 con 0x7fd0c8094660 2024-05-03T18:40:49.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:49.317+0000 7fd0cea00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd0c8094660 0x7fd0c8138610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:49.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:49.317+0000 7fd0cea00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd0c8094660 0x7fd0c8138610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:41472/0 (socket says 172.21.15.33:41472) 2024-05-03T18:40:49.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:49.317+0000 7fd0cea00700 1 -- 172.21.15.33:0/1170654308 learned_addr learned my addr 172.21.15.33:0/1170654308 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:40:49.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:49.317+0000 7fd0cf400700 1 --2- 172.21.15.33:0/1170654308 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0c80a6af0 0x7fd0c8132790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:49.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:49.318+0000 7fd0cf400700 1 -- 172.21.15.33:0/1170654308 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd0c8094660 msgr2=0x7fd0c8138610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:40:49.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:49.318+0000 7fd0cf400700 1 --2- 172.21.15.33:0/1170654308 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd0c8094660 0x7fd0c8138610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:49.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:49.318+0000 7fd0cf400700 1 -- 172.21.15.33:0/1170654308 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0c8094f80 msgr2=0x7fd0c8138b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:40:49.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:49.318+0000 7fd0cf400700 1 --2- 172.21.15.33:0/1170654308 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0c8094f80 0x7fd0c8138b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:49.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:49.318+0000 7fd0cf400700 1 -- 172.21.15.33:0/1170654308 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0c8132f70 con 0x7fd0c80a6af0 2024-05-03T18:40:49.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:49.319+0000 7fd0cf400700 1 --2- 172.21.15.33:0/1170654308 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0c80a6af0 0x7fd0c8132790 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd0d0051fe0 tx=0x7fd0d006cbc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:40:49.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:49.319+0000 7fd0c7400700 1 -- 172.21.15.33:0/1170654308 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0d0087070 con 0x7fd0c80a6af0 2024-05-03T18:40:49.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:49.319+0000 7fd0c7400700 1 -- 172.21.15.33:0/1170654308 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd0d0075da0 con 0x7fd0c80a6af0 2024-05-03T18:40:49.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:49.319+0000 7fd0cfe00700 1 -- 172.21.15.33:0/1170654308 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0c8133260 con 0x7fd0c80a6af0 2024-05-03T18:40:49.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:49.319+0000 7fd0c7400700 1 -- 172.21.15.33:0/1170654308 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0d0081750 con 0x7fd0c80a6af0 2024-05-03T18:40:49.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:49.319+0000 7fd0cfe00700 1 -- 172.21.15.33:0/1170654308 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0c813f470 con 0x7fd0c80a6af0 2024-05-03T18:40:49.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:49.321+0000 7fd0cfe00700 1 -- 172.21.15.33:0/1170654308 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0b0005330 con 0x7fd0c80a6af0 2024-05-03T18:40:49.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:49.325+0000 7fd0c7400700 1 -- 172.21.15.33:0/1170654308 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fd0d008a070 con 0x7fd0c80a6af0 2024-05-03T18:40:49.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:49.329+0000 7fd0c7400700 1 --2- 172.21.15.33:0/1170654308 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fd0b40b2a70 0x7fd0b40b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:49.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:49.330+0000 7fd0c7400700 1 -- 172.21.15.33:0/1170654308 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9518+0+0 (secure 0 0 0) 0x7fd0d014f5c0 con 0x7fd0c80a6af0 2024-05-03T18:40:49.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:49.331+0000 7fd0cea00700 1 --2- 172.21.15.33:0/1170654308 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fd0b40b2a70 0x7fd0b40b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:49.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:49.331+0000 7fd0c7400700 1 -- 172.21.15.33:0/1170654308 <== mon.2 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fd0d0116cd0 con 0x7fd0c80a6af0 2024-05-03T18:40:49.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:49.332+0000 7fd0cea00700 1 --2- 172.21.15.33:0/1170654308 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fd0b40b2a70 0x7fd0b40b4f30 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd0c0009ad0 tx=0x7fd0c0009290 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:40:49.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:49.863+0000 7fd0cfe00700 1 -- 172.21.15.33:0/1170654308 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd0b0005620 con 0x7fd0c80a6af0 2024-05-03T18:40:49.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:49.866+0000 7fd0c7400700 1 -- 172.21.15.33:0/1170654308 <== mon.2 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11801 (secure 0 0 0) 0x7fd0d01165d0 con 0x7fd0c80a6af0 2024-05-03T18:40:49.869 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:40:49.869 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1327393503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1327393503},{"type":"v1","addr":"172.21.15.33:6835","nonce":1327393503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/2461472948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":2461472948},{"type":"v1","addr":"172.21.15.33:6837","nonce":2461472948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14559,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/442269281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":442269281},{"type":"v1","addr":"172.21.15.33:6839","nonce":442269281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14571,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/645571756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":645571756},{"type":"v1","addr":"172.21.15.33:6841","nonce":645571756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/203598909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":203598909},{"type":"v1","addr":"172.21.15.150:6833","nonce":203598909}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/4029842493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":4029842493},{"type":"v1","addr":"172.21.15.150:6835","nonce":4029842493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3463664423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3463664423},{"type":"v1","addr":"172.21.15.150:6837","nonce":3463664423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24481,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6839/4051648018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":4051648018},{"type":"v1","addr":"172.21.15.150:6839","nonce":4051648018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34418,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1865105479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1865105479},{"type":"v1","addr":"172.21.15.99:6833","nonce":1865105479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34445,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/1206393914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1206393914},{"type":"v1","addr":"172.21.15.99:6835","nonce":1206393914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-03T18:22:28.837091+0000","modified":"2024-05-03T18:23:49.758398+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34508},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34472":{"gid":34472,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6837/3360970342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3360970342},{"type":"v1","addr":"172.21.15.99:6837","nonce":3360970342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34508":{"gid":34508,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/919907308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":919907308},{"type":"v1","addr":"172.21.15.99:6839","nonce":919907308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34508,"qdb_cluster":[34508]},"id":1}]} 2024-05-03T18:40:49.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:49.870+0000 7fd0cfe00700 1 -- 172.21.15.33:0/1170654308 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fd0b40b2a70 msgr2=0x7fd0b40b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:40:49.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:49.870+0000 7fd0cfe00700 1 --2- 172.21.15.33:0/1170654308 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fd0b40b2a70 0x7fd0b40b4f30 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd0c0009ad0 tx=0x7fd0c0009290 comp rx=0 tx=0).stop 2024-05-03T18:40:49.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:49.870+0000 7fd0cfe00700 1 -- 172.21.15.33:0/1170654308 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0c80a6af0 msgr2=0x7fd0c8132790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:40:49.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:49.870+0000 7fd0cfe00700 1 --2- 172.21.15.33:0/1170654308 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0c80a6af0 0x7fd0c8132790 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd0d0051fe0 tx=0x7fd0d006cbc0 comp rx=0 tx=0).stop 2024-05-03T18:40:49.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:49.870+0000 7fd0cfe00700 1 -- 172.21.15.33:0/1170654308 shutdown_connections 2024-05-03T18:40:49.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:49.870+0000 7fd0cfe00700 1 --2- 172.21.15.33:0/1170654308 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fd0b40b2a70 0x7fd0b40b4f30 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:49.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:49.870+0000 7fd0cfe00700 1 --2- 172.21.15.33:0/1170654308 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0c80a6af0 0x7fd0c8132790 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:49.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:49.870+0000 7fd0cfe00700 1 --2- 172.21.15.33:0/1170654308 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd0c8094660 0x7fd0c8138610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:49.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:49.870+0000 7fd0cfe00700 1 --2- 172.21.15.33:0/1170654308 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0c8094f80 0x7fd0c8138b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:49.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:49.870+0000 7fd0cfe00700 1 -- 172.21.15.33:0/1170654308 >> 172.21.15.33:0/1170654308 conn(0x7fd0c8009f20 msgr2=0x7fd0c80a45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:40:49.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:49.872+0000 7fd0cfe00700 1 -- 172.21.15.33:0/1170654308 shutdown_connections 2024-05-03T18:40:49.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:49.872+0000 7fd0cfe00700 1 -- 172.21.15.33:0/1170654308 wait complete. 2024-05-03T18:40:49.875 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 21 2024-05-03T18:40:50.111 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 575.0 s, 563.0 tps, lat 176.463 ms stddev 55.926 2024-05-03T18:40:50.165 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:40:49 smithi033 ceph-mon[28330]: pgmap v425: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 5.1 MiB/s wr, 471 op/s 2024-05-03T18:40:50.165 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:40:49 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/1170654308' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:40:50.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:40:49 smithi099 ceph-mon[31658]: pgmap v425: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 5.1 MiB/s wr, 471 op/s 2024-05-03T18:40:50.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:40:49 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/1170654308' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:40:50.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:40:49 smithi150 ceph-mon[32492]: pgmap v425: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 5.1 MiB/s wr, 471 op/s 2024-05-03T18:40:50.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:40:49 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/1170654308' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:40:50.810 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-03T18:40:51.594 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:40:51.987 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:40:51 smithi033 ceph-mon[28330]: pgmap v426: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 7.4 MiB/s wr, 701 op/s 2024-05-03T18:40:52.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:40:51 smithi099 ceph-mon[31658]: pgmap v426: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 7.4 MiB/s wr, 701 op/s 2024-05-03T18:40:52.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:40:51 smithi150 ceph-mon[32492]: pgmap v426: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 7.4 MiB/s wr, 701 op/s 2024-05-03T18:40:54.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:40:53 smithi033 ceph-mon[28330]: pgmap v427: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 4.7 MiB/s wr, 441 op/s 2024-05-03T18:40:54.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:40:53 smithi099 ceph-mon[31658]: pgmap v427: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 4.7 MiB/s wr, 441 op/s 2024-05-03T18:40:54.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:40:53 smithi150 ceph-mon[32492]: pgmap v427: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 4.7 MiB/s wr, 441 op/s 2024-05-03T18:40:55.111 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 580.0 s, 565.6 tps, lat 178.718 ms stddev 109.120 2024-05-03T18:40:55.975 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:40:55 smithi033 ceph-mon[28330]: pgmap v428: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 7.2 MiB/s wr, 682 op/s 2024-05-03T18:40:56.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:40:55 smithi099 ceph-mon[31658]: pgmap v428: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 7.2 MiB/s wr, 682 op/s 2024-05-03T18:40:56.384 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:40:55 smithi150 ceph-mon[32492]: pgmap v428: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 7.2 MiB/s wr, 682 op/s 2024-05-03T18:40:56.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.503+0000 7f3b7dc00700 1 -- 172.21.15.33:0/409710620 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3b7810fb60 msgr2=0x7f3b7810ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:40:56.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.503+0000 7f3b7dc00700 1 --2- 172.21.15.33:0/409710620 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3b7810fb60 0x7f3b7810ff40 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f3b68009990 tx=0x7f3b68009ca0 comp rx=0 tx=0).stop 2024-05-03T18:40:56.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.504+0000 7f3b7dc00700 1 -- 172.21.15.33:0/409710620 shutdown_connections 2024-05-03T18:40:56.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.504+0000 7f3b7dc00700 1 --2- 172.21.15.33:0/409710620 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b7810a490 0x7f3b781169a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:56.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.504+0000 7f3b7dc00700 1 --2- 172.21.15.33:0/409710620 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3b7810fb60 0x7f3b7810ff40 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:56.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.504+0000 7f3b7dc00700 1 --2- 172.21.15.33:0/409710620 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3b78109ad0 0x7f3b78109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:56.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.504+0000 7f3b7dc00700 1 -- 172.21.15.33:0/409710620 >> 172.21.15.33:0/409710620 conn(0x7f3b78076c70 msgr2=0x7f3b781082c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:40:56.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.505+0000 7f3b7dc00700 1 -- 172.21.15.33:0/409710620 shutdown_connections 2024-05-03T18:40:56.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.505+0000 7f3b7dc00700 1 -- 172.21.15.33:0/409710620 wait complete. 2024-05-03T18:40:56.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.506+0000 7f3b7dc00700 1 Processor -- start 2024-05-03T18:40:56.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.507+0000 7f3b7dc00700 1 -- start start 2024-05-03T18:40:56.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.507+0000 7f3b7dc00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3b78109ad0 0x7f3b781a00c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:56.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.508+0000 7f3b7dc00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b7810a490 0x7f3b781a0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:56.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.508+0000 7f3b7dc00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3b7810fb60 0x7f3b781a4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:56.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.508+0000 7f3b76a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3b78109ad0 0x7f3b781a00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:56.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.509+0000 7f3b7dc00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3b78119bb0 con 0x7f3b7810a490 2024-05-03T18:40:56.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.509+0000 7f3b7dc00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3b78119a30 con 0x7f3b78109ad0 2024-05-03T18:40:56.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.509+0000 7f3b77400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3b7810fb60 0x7f3b781a4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:56.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.509+0000 7f3b77400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3b7810fb60 0x7f3b781a4940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:59310/0 (socket says 172.21.15.33:59310) 2024-05-03T18:40:56.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.509+0000 7f3b77400700 1 -- 172.21.15.33:0/3265498391 learned_addr learned my addr 172.21.15.33:0/3265498391 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:40:56.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.509+0000 7f3b7dc00700 1 -- 172.21.15.33:0/3265498391 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3b78119d30 con 0x7f3b7810fb60 2024-05-03T18:40:56.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.510+0000 7f3b76a00700 1 -- 172.21.15.33:0/3265498391 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3b7810fb60 msgr2=0x7f3b781a4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:40:56.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.510+0000 7f3b76a00700 1 --2- 172.21.15.33:0/3265498391 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3b7810fb60 0x7f3b781a4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:56.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.510+0000 7f3b76a00700 1 -- 172.21.15.33:0/3265498391 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b7810a490 msgr2=0x7f3b781a0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:40:56.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.510+0000 7f3b76a00700 1 --2- 172.21.15.33:0/3265498391 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b7810a490 0x7f3b781a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:56.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.510+0000 7f3b76a00700 1 -- 172.21.15.33:0/3265498391 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b781a5080 con 0x7f3b78109ad0 2024-05-03T18:40:56.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.510+0000 7f3b77400700 1 --2- 172.21.15.33:0/3265498391 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3b7810fb60 0x7f3b781a4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:40:56.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.511+0000 7f3b76a00700 1 --2- 172.21.15.33:0/3265498391 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3b78109ad0 0x7f3b781a00c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3b680098e0 tx=0x7f3b680047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:40:56.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.512+0000 7f3b6f400700 1 -- 172.21.15.33:0/3265498391 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b68026070 con 0x7f3b78109ad0 2024-05-03T18:40:56.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.513+0000 7f3b6f400700 1 -- 172.21.15.33:0/3265498391 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3b68021410 con 0x7f3b78109ad0 2024-05-03T18:40:56.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.513+0000 7f3b7dc00700 1 -- 172.21.15.33:0/3265498391 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b781a5310 con 0x7f3b78109ad0 2024-05-03T18:40:56.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.513+0000 7f3b7dc00700 1 -- 172.21.15.33:0/3265498391 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b781acbf0 con 0x7f3b78109ad0 2024-05-03T18:40:56.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.513+0000 7f3b6f400700 1 -- 172.21.15.33:0/3265498391 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b68017650 con 0x7f3b78109ad0 2024-05-03T18:40:56.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.522+0000 7f3b6f400700 1 -- 172.21.15.33:0/3265498391 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f3b68029070 con 0x7f3b78109ad0 2024-05-03T18:40:56.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.524+0000 7f3b6f400700 1 --2- 172.21.15.33:0/3265498391 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f3b5c0b2bc0 0x7f3b5c0b5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:56.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.525+0000 7f3b6f400700 1 -- 172.21.15.33:0/3265498391 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9518+0+0 (secure 0 0 0) 0x7f3b680e46a0 con 0x7f3b78109ad0 2024-05-03T18:40:56.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.529+0000 7f3b76000700 1 --2- 172.21.15.33:0/3265498391 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f3b5c0b2bc0 0x7f3b5c0b5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:56.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.850+0000 7f3b76000700 1 --2- 172.21.15.33:0/3265498391 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f3b5c0b2bc0 0x7f3b5c0b5080 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3b60005950 tx=0x7f3b600058e0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:40:56.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.852+0000 7f3b8860eb80 1 Processor -- start 2024-05-03T18:40:56.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.853+0000 7f3b8860eb80 1 -- start start 2024-05-03T18:40:56.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.854+0000 7f3b8860eb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55788aed07b0 0x55788aed0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:56.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.854+0000 7f3b8860eb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55788aed10d0 0x55788aecbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:56.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.855+0000 7f3b8860eb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55788aecc5d0 0x55788aecca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:56.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.855+0000 7f3b76a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55788aed07b0 0x55788aed0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:56.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.855+0000 7f3b76a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55788aed07b0 0x55788aed0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:59320/0 (socket says 172.21.15.33:59320) 2024-05-03T18:40:56.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.856+0000 7f3b8860eb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55788ae1ea20 con 0x55788aed10d0 2024-05-03T18:40:56.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.856+0000 7f3b8860eb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55788adbc760 con 0x55788aecc5d0 2024-05-03T18:40:56.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.856+0000 7f3b8860eb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55788ae1a9e0 con 0x55788aed07b0 2024-05-03T18:40:56.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.856+0000 7f3b76a00700 1 -- 172.21.15.33:0/964344512 learned_addr learned my addr 172.21.15.33:0/964344512 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:40:56.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.857+0000 7f3b76a00700 1 -- 172.21.15.33:0/964344512 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55788aecc5d0 msgr2=0x55788aecca30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-03T18:40:56.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.857+0000 7f3b76a00700 1 --2- 172.21.15.33:0/964344512 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55788aecc5d0 0x55788aecca30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:56.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.857+0000 7f3b76a00700 1 -- 172.21.15.33:0/964344512 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55788aed10d0 msgr2=0x55788aecbfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:40:56.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.857+0000 7f3b76a00700 1 --2- 172.21.15.33:0/964344512 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55788aed10d0 0x55788aecbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:56.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.857+0000 7f3b76a00700 1 -- 172.21.15.33:0/964344512 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55788ad6b8d0 con 0x55788aed07b0 2024-05-03T18:40:56.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.858+0000 7f3b76a00700 1 --2- 172.21.15.33:0/964344512 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55788aed07b0 0x55788aed0b90 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f3b6800b650 tx=0x7f3b680aad40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87bb9d7a3dd34a09 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:40:56.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.859+0000 7f3b67e00700 1 -- 172.21.15.33:0/964344512 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b680ab5d0 con 0x55788aed07b0 2024-05-03T18:40:56.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.860+0000 7f3b67e00700 1 -- 172.21.15.33:0/964344512 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3b680af980 con 0x55788aed07b0 2024-05-03T18:40:56.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.860+0000 7f3b67e00700 1 -- 172.21.15.33:0/964344512 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b680abac0 con 0x55788aed07b0 2024-05-03T18:40:56.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.861+0000 7f3b8860eb80 1 -- 172.21.15.33:0/964344512 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55788aed07b0 msgr2=0x55788aed0b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:40:56.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.861+0000 7f3b8860eb80 1 --2- 172.21.15.33:0/964344512 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55788aed07b0 0x55788aed0b90 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f3b6800b650 tx=0x7f3b680aad40 comp rx=0 tx=0).stop 2024-05-03T18:40:56.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.862+0000 7f3b8860eb80 1 -- 172.21.15.33:0/964344512 shutdown_connections 2024-05-03T18:40:56.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.862+0000 7f3b8860eb80 1 --2- 172.21.15.33:0/964344512 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55788aecc5d0 0x55788aecca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:56.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.862+0000 7f3b8860eb80 1 --2- 172.21.15.33:0/964344512 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55788aed07b0 0x55788aed0b90 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:56.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.862+0000 7f3b8860eb80 1 --2- 172.21.15.33:0/964344512 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55788aed10d0 0x55788aecbfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:56.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.862+0000 7f3b8860eb80 1 -- 172.21.15.33:0/964344512 >> 172.21.15.33:0/964344512 conn(0x55788adfab60 msgr2=0x55788aec67f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:40:56.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.862+0000 7f3b8860eb80 1 -- 172.21.15.33:0/964344512 shutdown_connections 2024-05-03T18:40:56.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.862+0000 7f3b8860eb80 1 -- 172.21.15.33:0/964344512 wait complete. 2024-05-03T18:40:56.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.863+0000 7f3b8860eb80 1 -- start start 2024-05-03T18:40:56.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.863+0000 7f3b8860eb80 1 Processor -- start 2024-05-03T18:40:56.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.865+0000 7f3b8860eb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55788aecc5d0 0x55788aed7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:56.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.865+0000 7f3b8860eb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55788aed07b0 0x55788aed5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:56.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.866+0000 7f3b8860eb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55788aed10d0 0x55788aed57a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:56.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.866+0000 7f3b76000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55788aed07b0 0x55788aed5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:56.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.866+0000 7f3b76000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55788aed07b0 0x55788aed5260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:41886/0 (socket says 172.21.15.33:41886) 2024-05-03T18:40:56.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.866+0000 7f3b76a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55788aecc5d0 0x55788aed7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:56.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.867+0000 7f3b76000700 1 -- 172.21.15.33:0/2497824468 learned_addr learned my addr 172.21.15.33:0/2497824468 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:40:56.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.867+0000 7f3b8860eb80 1 -- 172.21.15.33:0/2497824468 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55788aed8a60 con 0x55788aed10d0 2024-05-03T18:40:56.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.867+0000 7f3b8860eb80 1 -- 172.21.15.33:0/2497824468 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55788ad84b20 con 0x55788aed07b0 2024-05-03T18:40:56.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.867+0000 7f3b8860eb80 1 -- 172.21.15.33:0/2497824468 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55788aed8be0 con 0x55788aecc5d0 2024-05-03T18:40:56.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.867+0000 7f3b8860eb80 10 client.0 _ll_register_callbacks cb 0x55788adc5c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:40:56.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.867+0000 7f3b77400700 1 --2- 172.21.15.33:0/2497824468 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55788aed10d0 0x55788aed57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:56.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.867+0000 7f3b76000700 1 -- 172.21.15.33:0/2497824468 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55788aecc5d0 msgr2=0x55788aed7ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:40:56.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.867+0000 7f3b76000700 1 --2- 172.21.15.33:0/2497824468 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55788aecc5d0 0x55788aed7ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:56.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.867+0000 7f3b76000700 1 -- 172.21.15.33:0/2497824468 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55788aed10d0 msgr2=0x55788aed57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:40:56.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.867+0000 7f3b76000700 1 --2- 172.21.15.33:0/2497824468 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55788aed10d0 0x55788aed57a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:56.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.867+0000 7f3b76000700 1 -- 172.21.15.33:0/2497824468 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55788ade4420 con 0x55788aed07b0 2024-05-03T18:40:56.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.868+0000 7f3b76000700 1 --2- 172.21.15.33:0/2497824468 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55788aed07b0 0x55788aed5260 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3b6000ddd0 tx=0x7f3b60010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9de9e08c53b6783 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:40:56.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.868+0000 7f3b64c00700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-03T18:40:56.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.869+0000 7f3b64c00700 1 -- 172.21.15.33:0/2497824468 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b60021020 con 0x55788aed07b0 2024-05-03T18:40:56.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.869+0000 7f3b64c00700 1 -- 172.21.15.33:0/2497824468 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9518+0+0 (secure 0 0 0) 0x7f3b60010c70 con 0x55788aed07b0 2024-05-03T18:40:56.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.869+0000 7f3b8860eb80 1 -- 172.21.15.33:0/2497824468 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b680af980 con 0x55788aed07b0 2024-05-03T18:40:56.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.871+0000 7f3b64c00700 1 -- 172.21.15.33:0/2497824468 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f3b6001d440 con 0x55788aed07b0 2024-05-03T18:40:56.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.872+0000 7f3b8860eb80 10 client.36365 fetch_fsmap learned FSMap version 21 2024-05-03T18:40:56.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.872+0000 7f3b8860eb80 1 -- 172.21.15.33:0/2497824468 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55788aee3210 con 0x55788aed07b0 2024-05-03T18:40:56.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.874+0000 7f3b64c00700 1 -- 172.21.15.33:0/2497824468 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3b680af980 con 0x55788aed07b0 2024-05-03T18:40:56.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.874+0000 7f3b64c00700 1 -- 172.21.15.33:0/2497824468 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f3b6002e020 con 0x55788aed07b0 2024-05-03T18:40:56.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.874+0000 7f3b8860eb80 10 client.36365 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:40:56.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.874+0000 7f3b8860eb80 10 client.36365 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:40:56.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.874+0000 7f3b8860eb80 20 client.36365 populate_metadata read hostname 'smithi033' 2024-05-03T18:40:56.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.874+0000 7f3b8860eb80 1 --2- 172.21.15.33:0/2497824468 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55788aee5850 0x55788aee7c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:56.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.874+0000 7f3b8860eb80 4 client.36365 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:40:56.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.874+0000 7f3b8860eb80 1 -- 172.21.15.33:0/2497824468 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55788ae991d0 con 0x55788aee5850 2024-05-03T18:40:56.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.875+0000 7f3b76a00700 1 --2- 172.21.15.33:0/2497824468 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55788aee5850 0x55788aee7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:56.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.877+0000 7f3b76a00700 1 --2- 172.21.15.33:0/2497824468 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55788aee5850 0x55788aee7c40 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57593847e7987d02 server_cookie=c40b3cb2eb523cad in_seq=0 out_seq=0 2024-05-03T18:40:56.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.877+0000 7f3b64c00700 10 client.36365 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:40:56.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.878+0000 7f3b64c00700 1 -- 172.21.15.33:0/2497824468 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55788ae991d0 con 0x55788aee5850 2024-05-03T18:40:56.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.878+0000 7f3b64c00700 10 client.36365 handle_command_reply: tid=0 2024-05-03T18:40:56.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.879+0000 7f3b8860eb80 1 client.36365 shutdown 2024-05-03T18:40:56.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.881+0000 7f3b8860eb80 1 -- 172.21.15.33:0/2497824468 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55788aed07b0 msgr2=0x55788aed5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:40:56.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.881+0000 7f3b8860eb80 1 --2- 172.21.15.33:0/2497824468 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55788aed07b0 0x55788aed5260 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3b6000ddd0 tx=0x7f3b60010740 comp rx=0 tx=0).stop 2024-05-03T18:40:56.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.881+0000 7f3b8860eb80 1 -- 172.21.15.33:0/2497824468 shutdown_connections 2024-05-03T18:40:56.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.881+0000 7f3b8860eb80 1 --2- 172.21.15.33:0/2497824468 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55788aee5850 0x55788aee7c40 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:56.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.881+0000 7f3b8860eb80 1 --2- 172.21.15.33:0/2497824468 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55788aed10d0 0x55788aed57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:56.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.881+0000 7f3b8860eb80 1 --2- 172.21.15.33:0/2497824468 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55788aecc5d0 0x55788aed7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:56.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.881+0000 7f3b8860eb80 1 --2- 172.21.15.33:0/2497824468 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55788aed07b0 0x55788aed5260 secure :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3b6000ddd0 tx=0x7f3b60010740 comp rx=0 tx=0).stop 2024-05-03T18:40:56.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.881+0000 7f3b8860eb80 1 -- 172.21.15.33:0/2497824468 >> 172.21.15.33:0/2497824468 conn(0x55788adfab60 msgr2=0x55788aed1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:40:56.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.881+0000 7f3b64c00700 0 client.36365 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:40:56.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.882+0000 7f3b8860eb80 1 -- 172.21.15.33:0/2497824468 shutdown_connections 2024-05-03T18:40:56.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.882+0000 7f3b8860eb80 1 -- 172.21.15.33:0/2497824468 wait complete. 2024-05-03T18:40:56.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.882+0000 7f3b8860eb80 20 client.36365 trim_cache size 0 max 16384 2024-05-03T18:40:56.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.913+0000 7f3b8860eb80 1 Processor -- start 2024-05-03T18:40:56.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.913+0000 7f3b8860eb80 1 -- start start 2024-05-03T18:40:56.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.913+0000 7f3b8860eb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55788aecc5d0 0x55788aec84f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:56.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.913+0000 7f3b8860eb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55788aed10d0 0x55788aed43e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:56.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.913+0000 7f3b8860eb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55788aee1ab0 0x55788aec6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:56.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.913+0000 7f3b8860eb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3b5400cf60 con 0x55788aee1ab0 2024-05-03T18:40:56.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.913+0000 7f3b8860eb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3b5400f7c0 con 0x55788aecc5d0 2024-05-03T18:40:56.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.913+0000 7f3b8860eb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3b5400d5b0 con 0x55788aed10d0 2024-05-03T18:40:56.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.914+0000 7f3b76000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55788aed10d0 0x55788aed43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:56.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.914+0000 7f3b76000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55788aed10d0 0x55788aed43e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:59346/0 (socket says 172.21.15.33:59346) 2024-05-03T18:40:56.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.914+0000 7f3b76000700 1 -- 172.21.15.33:0/802566701 learned_addr learned my addr 172.21.15.33:0/802566701 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:40:56.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.914+0000 7f3b76a00700 1 --2- 172.21.15.33:0/802566701 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55788aecc5d0 0x55788aec84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:56.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.914+0000 7f3b77400700 1 --2- 172.21.15.33:0/802566701 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55788aee1ab0 0x55788aec6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:56.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.915+0000 7f3b76000700 1 -- 172.21.15.33:0/802566701 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55788aecc5d0 msgr2=0x55788aec84f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:40:56.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.915+0000 7f3b76000700 1 --2- 172.21.15.33:0/802566701 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55788aecc5d0 0x55788aec84f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:56.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.915+0000 7f3b76000700 1 -- 172.21.15.33:0/802566701 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55788aee1ab0 msgr2=0x55788aec6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:40:56.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.915+0000 7f3b76000700 1 --2- 172.21.15.33:0/802566701 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55788aee1ab0 0x55788aec6660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:56.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.916+0000 7f3b76000700 1 -- 172.21.15.33:0/802566701 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55788aed3740 con 0x55788aed10d0 2024-05-03T18:40:56.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.919+0000 7f3b77400700 1 --2- 172.21.15.33:0/802566701 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55788aee1ab0 0x55788aec6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:40:56.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.920+0000 7f3b76000700 1 --2- 172.21.15.33:0/802566701 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55788aed10d0 0x55788aed43e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f3b60005d90 tx=0x7f3b60000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3077b91f4692937 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:40:56.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.920+0000 7f3b67400700 1 -- 172.21.15.33:0/802566701 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b60021050 con 0x55788aed10d0 2024-05-03T18:40:56.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.920+0000 7f3b67400700 1 -- 172.21.15.33:0/802566701 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3b60035070 con 0x55788aed10d0 2024-05-03T18:40:56.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.920+0000 7f3b67400700 1 -- 172.21.15.33:0/802566701 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b6001da70 con 0x55788aed10d0 2024-05-03T18:40:56.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.921+0000 7f3b8860eb80 1 -- 172.21.15.33:0/802566701 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55788aed10d0 msgr2=0x55788aed43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:40:56.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.921+0000 7f3b8860eb80 1 --2- 172.21.15.33:0/802566701 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55788aed10d0 0x55788aed43e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f3b60005d90 tx=0x7f3b60000bc0 comp rx=0 tx=0).stop 2024-05-03T18:40:56.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.922+0000 7f3b8860eb80 1 -- 172.21.15.33:0/802566701 shutdown_connections 2024-05-03T18:40:56.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.922+0000 7f3b8860eb80 1 --2- 172.21.15.33:0/802566701 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55788aee1ab0 0x55788aec6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:56.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.922+0000 7f3b8860eb80 1 --2- 172.21.15.33:0/802566701 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55788aecc5d0 0x55788aec84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:56.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.922+0000 7f3b8860eb80 1 --2- 172.21.15.33:0/802566701 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55788aed10d0 0x55788aed43e0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:56.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.922+0000 7f3b8860eb80 1 -- 172.21.15.33:0/802566701 >> 172.21.15.33:0/802566701 conn(0x55788adfab60 msgr2=0x55788aecb2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:40:56.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.922+0000 7f3b8860eb80 1 -- 172.21.15.33:0/802566701 shutdown_connections 2024-05-03T18:40:56.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.922+0000 7f3b8860eb80 1 -- 172.21.15.33:0/802566701 wait complete. 2024-05-03T18:40:56.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.923+0000 7f3b8860eb80 1 -- start start 2024-05-03T18:40:56.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.923+0000 7f3b8860eb80 1 Processor -- start 2024-05-03T18:40:56.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.924+0000 7f3b8860eb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55788aecc5d0 0x55788aed2f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:56.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.924+0000 7f3b8860eb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55788aed10d0 0x55788aedde30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:56.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.924+0000 7f3b8860eb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55788aee1ab0 0x55788aec9c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:56.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.924+0000 7f3b8860eb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3b5400dc80 con 0x55788aed10d0 2024-05-03T18:40:56.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.924+0000 7f3b8860eb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3b5400f0f0 con 0x55788aecc5d0 2024-05-03T18:40:56.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.924+0000 7f3b8860eb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3b5400e350 con 0x55788aee1ab0 2024-05-03T18:40:56.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.924+0000 7f3b8860eb80 10 client.0 _ll_register_callbacks cb 0x55788adc5c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:40:56.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.924+0000 7f3b76000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55788aed10d0 0x55788aedde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:56.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.924+0000 7f3b77400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55788aee1ab0 0x55788aec9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:56.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.925+0000 7f3b76000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55788aed10d0 0x55788aedde30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36772/0 (socket says 172.21.15.33:36772) 2024-05-03T18:40:56.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.925+0000 7f3b76000700 1 -- 172.21.15.33:0/4110462561 learned_addr learned my addr 172.21.15.33:0/4110462561 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:40:56.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.925+0000 7f3b76000700 1 -- 172.21.15.33:0/4110462561 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55788aee1ab0 msgr2=0x55788aec9c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:40:56.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.925+0000 7f3b76000700 1 --2- 172.21.15.33:0/4110462561 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55788aee1ab0 0x55788aec9c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:56.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.925+0000 7f3b76000700 1 -- 172.21.15.33:0/4110462561 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55788aecc5d0 msgr2=0x55788aed2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:40:56.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.925+0000 7f3b76000700 1 --2- 172.21.15.33:0/4110462561 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55788aecc5d0 0x55788aed2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:56.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.925+0000 7f3b76000700 1 -- 172.21.15.33:0/4110462561 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55788aede8a0 con 0x55788aed10d0 2024-05-03T18:40:56.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.926+0000 7f3b76000700 1 --2- 172.21.15.33:0/4110462561 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55788aed10d0 0x55788aedde30 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f3b600267b0 tx=0x7f3b60010df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=265a5cc9d39a9962 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:40:56.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.926+0000 7f3b64c00700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:40:56.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.927+0000 7f3b64c00700 1 -- 172.21.15.33:0/4110462561 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b60040070 con 0x55788aed10d0 2024-05-03T18:40:56.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.927+0000 7f3b8860eb80 1 -- 172.21.15.33:0/4110462561 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b60035070 con 0x55788aed10d0 2024-05-03T18:40:56.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.929+0000 7f3b64c00700 1 -- 172.21.15.33:0/4110462561 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9518+0+0 (secure 0 0 0) 0x7f3b6001d9f0 con 0x55788aed10d0 2024-05-03T18:40:56.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.929+0000 7f3b64c00700 1 -- 172.21.15.33:0/4110462561 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f3b60021040 con 0x55788aed10d0 2024-05-03T18:40:56.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.929+0000 7f3b8860eb80 10 client.16653 fetch_fsmap learned FSMap version 21 2024-05-03T18:40:56.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.929+0000 7f3b8860eb80 1 -- 172.21.15.33:0/4110462561 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55788af505d0 con 0x55788aed10d0 2024-05-03T18:40:56.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.929+0000 7f3b64c00700 1 -- 172.21.15.33:0/4110462561 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3b60035070 con 0x55788aed10d0 2024-05-03T18:40:56.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.930+0000 7f3b64c00700 1 -- 172.21.15.33:0/4110462561 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f3b6002e030 con 0x55788aed10d0 2024-05-03T18:40:56.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.930+0000 7f3b8860eb80 10 client.16653 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:40:56.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.930+0000 7f3b8860eb80 10 client.16653 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:40:56.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.930+0000 7f3b8860eb80 20 client.16653 populate_metadata read hostname 'smithi033' 2024-05-03T18:40:56.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.930+0000 7f3b8860eb80 1 --2- 172.21.15.33:0/4110462561 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55788af50ba0 0x55788af52fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:56.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.930+0000 7f3b8860eb80 4 client.16653 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-03T18:40:56.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.930+0000 7f3b8860eb80 1 -- 172.21.15.33:0/4110462561 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55788ae2c400 con 0x55788af50ba0 2024-05-03T18:40:56.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.931+0000 7f3b76a00700 1 --2- 172.21.15.33:0/4110462561 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55788af50ba0 0x55788af52fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:56.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.932+0000 7f3b76a00700 1 --2- 172.21.15.33:0/4110462561 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55788af50ba0 0x55788af52fc0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3efeb1ac8a8ba6ed server_cookie=c59fffdc0030e4cc in_seq=0 out_seq=0 2024-05-03T18:40:56.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.932+0000 7f3b64c00700 10 client.16653 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:40:56.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.932+0000 7f3b64c00700 1 -- 172.21.15.33:0/4110462561 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55788ae2c400 con 0x55788af50ba0 2024-05-03T18:40:56.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.932+0000 7f3b64c00700 10 client.16653 handle_command_reply: tid=0 2024-05-03T18:40:56.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.932+0000 7f3b8860eb80 1 client.16653 shutdown 2024-05-03T18:40:56.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.935+0000 7f3b8860eb80 1 -- 172.21.15.33:0/4110462561 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55788aed10d0 msgr2=0x55788aedde30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:40:56.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.935+0000 7f3b8860eb80 1 --2- 172.21.15.33:0/4110462561 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55788aed10d0 0x55788aedde30 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f3b600267b0 tx=0x7f3b60010df0 comp rx=0 tx=0).stop 2024-05-03T18:40:56.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.935+0000 7f3b8860eb80 1 -- 172.21.15.33:0/4110462561 shutdown_connections 2024-05-03T18:40:56.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.935+0000 7f3b8860eb80 1 --2- 172.21.15.33:0/4110462561 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55788af50ba0 0x55788af52fc0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:56.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.935+0000 7f3b8860eb80 1 --2- 172.21.15.33:0/4110462561 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55788aee1ab0 0x55788aec9c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:56.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.935+0000 7f3b8860eb80 1 --2- 172.21.15.33:0/4110462561 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55788aecc5d0 0x55788aed2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:56.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.935+0000 7f3b8860eb80 1 --2- 172.21.15.33:0/4110462561 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55788aed10d0 0x55788aedde30 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:56.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.935+0000 7f3b64c00700 0 client.16653 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:40:56.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.935+0000 7f3b8860eb80 1 -- 172.21.15.33:0/4110462561 >> 172.21.15.33:0/4110462561 conn(0x55788adfab60 msgr2=0x55788aee1330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:40:56.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.936+0000 7f3b8860eb80 1 -- 172.21.15.33:0/4110462561 shutdown_connections 2024-05-03T18:40:56.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.936+0000 7f3b8860eb80 1 -- 172.21.15.33:0/4110462561 wait complete. 2024-05-03T18:40:56.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.937+0000 7f3b8860eb80 20 client.16653 trim_cache size 0 max 16384 2024-05-03T18:40:56.938 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-03T18:40:56.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.941+0000 7f3b7dc00700 1 -- 172.21.15.33:0/3265498391 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f3b5c0b2bc0 msgr2=0x7f3b5c0b5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:40:56.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.941+0000 7f3b7dc00700 1 --2- 172.21.15.33:0/3265498391 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f3b5c0b2bc0 0x7f3b5c0b5080 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3b60005950 tx=0x7f3b600058e0 comp rx=0 tx=0).stop 2024-05-03T18:40:56.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.941+0000 7f3b7dc00700 1 -- 172.21.15.33:0/3265498391 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3b78109ad0 msgr2=0x7f3b781a00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:40:56.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.941+0000 7f3b7dc00700 1 --2- 172.21.15.33:0/3265498391 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3b78109ad0 0x7f3b781a00c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3b680098e0 tx=0x7f3b680047b0 comp rx=0 tx=0).stop 2024-05-03T18:40:56.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.941+0000 7f3b7dc00700 1 -- 172.21.15.33:0/3265498391 shutdown_connections 2024-05-03T18:40:56.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.941+0000 7f3b7dc00700 1 --2- 172.21.15.33:0/3265498391 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3b7810fb60 0x7f3b781a4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:56.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.941+0000 7f3b7dc00700 1 --2- 172.21.15.33:0/3265498391 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f3b5c0b2bc0 0x7f3b5c0b5080 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:56.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.941+0000 7f3b7dc00700 1 --2- 172.21.15.33:0/3265498391 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3b78109ad0 0x7f3b781a00c0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:56.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.941+0000 7f3b7dc00700 1 --2- 172.21.15.33:0/3265498391 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b7810a490 0x7f3b781a0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:56.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.941+0000 7f3b7dc00700 1 -- 172.21.15.33:0/3265498391 >> 172.21.15.33:0/3265498391 conn(0x7f3b78076c70 msgr2=0x7f3b780772d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:40:56.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.941+0000 7f3b7dc00700 1 -- 172.21.15.33:0/3265498391 shutdown_connections 2024-05-03T18:40:56.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:40:56.942+0000 7f3b7dc00700 1 -- 172.21.15.33:0/3265498391 wait complete. 2024-05-03T18:40:58.117 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:40:58 smithi033 ceph-mon[28330]: pgmap v429: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 5.0 MiB/s wr, 474 op/s 2024-05-03T18:40:58.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:40:58 smithi150 ceph-mon[32492]: pgmap v429: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 5.0 MiB/s wr, 474 op/s 2024-05-03T18:40:58.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:40:58 smithi099 ceph-mon[31658]: pgmap v429: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 5.0 MiB/s wr, 474 op/s 2024-05-03T18:40:58.956 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-03T18:40:58.957 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-03T18:40:59.697 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:41:00.113 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 585.0 s, 579.8 tps, lat 171.352 ms stddev 63.820 2024-05-03T18:41:00.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:41:00 smithi033 ceph-mon[28330]: pgmap v430: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 5.0 MiB/s wr, 475 op/s 2024-05-03T18:41:00.384 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:41:00 smithi150 ceph-mon[32492]: pgmap v430: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 5.0 MiB/s wr, 475 op/s 2024-05-03T18:41:00.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:41:00 smithi099 ceph-mon[31658]: pgmap v430: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 5.0 MiB/s wr, 475 op/s 2024-05-03T18:41:02.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:41:02 smithi150 ceph-mon[32492]: pgmap v431: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 7.1 MiB/s wr, 686 op/s 2024-05-03T18:41:02.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:41:02 smithi033 ceph-mon[28330]: pgmap v431: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 7.1 MiB/s wr, 686 op/s 2024-05-03T18:41:02.547 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:41:02 smithi099 ceph-mon[31658]: pgmap v431: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 7.1 MiB/s wr, 686 op/s 2024-05-03T18:41:03.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.878+0000 7f5074600700 1 -- 172.21.15.33:0/3722651228 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f506c10b6e0 msgr2=0x7f506c10bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:41:03.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.878+0000 7f5074600700 1 --2- 172.21.15.33:0/3722651228 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f506c10b6e0 0x7f506c10bac0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f5060009990 tx=0x7f5060009ca0 comp rx=0 tx=0).stop 2024-05-03T18:41:03.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.879+0000 7f5074600700 1 -- 172.21.15.33:0/3722651228 shutdown_connections 2024-05-03T18:41:03.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.879+0000 7f5074600700 1 --2- 172.21.15.33:0/3722651228 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f506c1060a0 0x7f506c1125b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:03.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.879+0000 7f5074600700 1 --2- 172.21.15.33:0/3722651228 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f506c10b6e0 0x7f506c10bac0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:03.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.879+0000 7f5074600700 1 --2- 172.21.15.33:0/3722651228 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f506c1056e0 0x7f506c105b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:03.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.879+0000 7f5074600700 1 -- 172.21.15.33:0/3722651228 >> 172.21.15.33:0/3722651228 conn(0x7f506c1011d0 msgr2=0x7f506c1035f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:41:03.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.879+0000 7f5074600700 1 -- 172.21.15.33:0/3722651228 shutdown_connections 2024-05-03T18:41:03.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.879+0000 7f5074600700 1 -- 172.21.15.33:0/3722651228 wait complete. 2024-05-03T18:41:03.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.880+0000 7f5074600700 1 Processor -- start 2024-05-03T18:41:03.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.880+0000 7f5074600700 1 -- start start 2024-05-03T18:41:03.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.881+0000 7f5074600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f506c1056e0 0x7f506c1124a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:03.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.881+0000 7f5074600700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f506c1060a0 0x7f506c10f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:03.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.881+0000 7f5074600700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f506c10b6e0 0x7f506c10faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:03.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.881+0000 7f5074600700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f506c1157e0 con 0x7f506c1060a0 2024-05-03T18:41:03.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.881+0000 7f5074600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f506c115660 con 0x7f506c1056e0 2024-05-03T18:41:03.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.881+0000 7f5074600700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f506c115960 con 0x7f506c10b6e0 2024-05-03T18:41:03.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.881+0000 7f5071c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f506c1056e0 0x7f506c1124a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:03.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.882+0000 7f5072600700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f506c10b6e0 0x7f506c10faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:03.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.882+0000 7f5071200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f506c1060a0 0x7f506c10f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:03.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.882+0000 7f5071c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f506c1056e0 0x7f506c1124a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:41936/0 (socket says 172.21.15.33:41936) 2024-05-03T18:41:03.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.882+0000 7f5071c00700 1 -- 172.21.15.33:0/3286197192 learned_addr learned my addr 172.21.15.33:0/3286197192 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:41:03.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.883+0000 7f5072600700 1 -- 172.21.15.33:0/3286197192 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f506c1056e0 msgr2=0x7f506c1124a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:41:03.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.883+0000 7f5072600700 1 --2- 172.21.15.33:0/3286197192 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f506c1056e0 0x7f506c1124a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:03.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.883+0000 7f5072600700 1 -- 172.21.15.33:0/3286197192 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f506c1060a0 msgr2=0x7f506c10f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:41:03.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.884+0000 7f5072600700 1 --2- 172.21.15.33:0/3286197192 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f506c1060a0 0x7f506c10f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:03.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.884+0000 7f5072600700 1 -- 172.21.15.33:0/3286197192 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f506c1a3cd0 con 0x7f506c10b6e0 2024-05-03T18:41:03.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.884+0000 7f5071200700 1 --2- 172.21.15.33:0/3286197192 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f506c1060a0 0x7f506c10f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:41:03.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.886+0000 7f5072600700 1 --2- 172.21.15.33:0/3286197192 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f506c10b6e0 0x7f506c10faf0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f506400efc0 tx=0x7f506400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:41:03.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.886+0000 7f506a000700 1 -- 172.21.15.33:0/3286197192 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50640092e0 con 0x7f506c10b6e0 2024-05-03T18:41:03.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.886+0000 7f506a000700 1 -- 172.21.15.33:0/3286197192 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5064004bb0 con 0x7f506c10b6e0 2024-05-03T18:41:03.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.886+0000 7f506a000700 1 -- 172.21.15.33:0/3286197192 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5064004020 con 0x7f506c10b6e0 2024-05-03T18:41:03.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.886+0000 7f5074600700 1 -- 172.21.15.33:0/3286197192 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f506c1a3f60 con 0x7f506c10b6e0 2024-05-03T18:41:03.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.887+0000 7f5074600700 1 -- 172.21.15.33:0/3286197192 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f506c1a4450 con 0x7f506c10b6e0 2024-05-03T18:41:03.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.896+0000 7f506a000700 1 -- 172.21.15.33:0/3286197192 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f506401b030 con 0x7f506c10b6e0 2024-05-03T18:41:03.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.899+0000 7f506a000700 1 --2- 172.21.15.33:0/3286197192 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f50540b2bc0 0x7f50540b5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:03.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.899+0000 7f506a000700 1 -- 172.21.15.33:0/3286197192 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9518+0+0 (secure 0 0 0) 0x7f50640e2430 con 0x7f506c10b6e0 2024-05-03T18:41:03.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.900+0000 7f5071c00700 1 --2- 172.21.15.33:0/3286197192 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f50540b2bc0 0x7f50540b5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:03.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.901+0000 7f5071c00700 1 --2- 172.21.15.33:0/3286197192 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f50540b2bc0 0x7f50540b5080 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5060006010 tx=0x7f506001a040 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:41:03.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.938+0000 7f507ef7db80 1 Processor -- start 2024-05-03T18:41:03.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.938+0000 7f507ef7db80 1 -- start start 2024-05-03T18:41:03.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.940+0000 7f507ef7db80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557898831830 0x557898831c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:03.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.940+0000 7f507ef7db80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557898832150 0x55789882d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:03.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.940+0000 7f507ef7db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55789882d650 0x55789882dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:03.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.941+0000 7f507ef7db80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55789877faa0 con 0x557898831830 2024-05-03T18:41:03.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.941+0000 7f507ef7db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55789871d7e0 con 0x55789882d650 2024-05-03T18:41:03.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.941+0000 7f507ef7db80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55789877ba60 con 0x557898832150 2024-05-03T18:41:03.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.941+0000 7f5072600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55789882d650 0x55789882dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:03.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.941+0000 7f5072600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55789882d650 0x55789882dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:41940/0 (socket says 172.21.15.33:41940) 2024-05-03T18:41:03.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.941+0000 7f5072600700 1 -- 172.21.15.33:0/4524889 learned_addr learned my addr 172.21.15.33:0/4524889 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:41:03.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.944+0000 7f5071c00700 1 --2- 172.21.15.33:0/4524889 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557898831830 0x557898831c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:03.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.943+0000 7f5072600700 1 -- 172.21.15.33:0/4524889 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557898832150 msgr2=0x55789882d030 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-03T18:41:03.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.944+0000 7f5072600700 1 --2- 172.21.15.33:0/4524889 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557898832150 0x55789882d030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:03.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.944+0000 7f5072600700 1 -- 172.21.15.33:0/4524889 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557898831830 msgr2=0x557898831c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:41:03.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.944+0000 7f5072600700 1 --2- 172.21.15.33:0/4524889 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557898831830 0x557898831c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:03.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.944+0000 7f5072600700 1 -- 172.21.15.33:0/4524889 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578986cc950 con 0x55789882d650 2024-05-03T18:41:03.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.944+0000 7f5071c00700 1 --2- 172.21.15.33:0/4524889 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557898831830 0x557898831c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:41:03.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.945+0000 7f5072600700 1 --2- 172.21.15.33:0/4524889 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55789882d650 0x55789882dab0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f50640a92f0 tx=0x7f5064004de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a45cbe9cf88196e server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:41:03.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.945+0000 7f505ea00700 1 -- 172.21.15.33:0/4524889 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50640a9360 con 0x55789882d650 2024-05-03T18:41:03.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.945+0000 7f505ea00700 1 -- 172.21.15.33:0/4524889 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f50640ad710 con 0x55789882d650 2024-05-03T18:41:03.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.945+0000 7f505ea00700 1 -- 172.21.15.33:0/4524889 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50640a4cd0 con 0x55789882d650 2024-05-03T18:41:03.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.947+0000 7f507ef7db80 1 -- 172.21.15.33:0/4524889 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55789882d650 msgr2=0x55789882dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:41:03.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.947+0000 7f507ef7db80 1 --2- 172.21.15.33:0/4524889 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55789882d650 0x55789882dab0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f50640a92f0 tx=0x7f5064004de0 comp rx=0 tx=0).stop 2024-05-03T18:41:03.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.947+0000 7f507ef7db80 1 -- 172.21.15.33:0/4524889 shutdown_connections 2024-05-03T18:41:03.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.947+0000 7f507ef7db80 1 --2- 172.21.15.33:0/4524889 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55789882d650 0x55789882dab0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:03.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.948+0000 7f507ef7db80 1 --2- 172.21.15.33:0/4524889 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557898831830 0x557898831c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:03.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.948+0000 7f507ef7db80 1 --2- 172.21.15.33:0/4524889 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557898832150 0x55789882d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:03.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.948+0000 7f507ef7db80 1 -- 172.21.15.33:0/4524889 >> 172.21.15.33:0/4524889 conn(0x55789875bbe0 msgr2=0x557898827870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:41:03.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.948+0000 7f507ef7db80 1 -- 172.21.15.33:0/4524889 shutdown_connections 2024-05-03T18:41:03.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.949+0000 7f507ef7db80 1 -- 172.21.15.33:0/4524889 wait complete. 2024-05-03T18:41:03.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.950+0000 7f507ef7db80 1 -- start start 2024-05-03T18:41:03.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.950+0000 7f507ef7db80 1 Processor -- start 2024-05-03T18:41:03.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.951+0000 7f507ef7db80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55789882d650 0x557898839000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:03.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.951+0000 7f507ef7db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557898831830 0x5578988363a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:03.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.951+0000 7f507ef7db80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557898832150 0x5578988368e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:03.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.951+0000 7f507ef7db80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x557898839cc0 con 0x557898832150 2024-05-03T18:41:03.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.951+0000 7f507ef7db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5578986e5ba0 con 0x557898831830 2024-05-03T18:41:03.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.951+0000 7f507ef7db80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x557898839e40 con 0x55789882d650 2024-05-03T18:41:03.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.951+0000 7f507ef7db80 10 client.0 _ll_register_callbacks cb 0x557898726cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:41:03.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.952+0000 7f5071c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55789882d650 0x557898839000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:03.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.952+0000 7f5071c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55789882d650 0x557898839000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:59424/0 (socket says 172.21.15.33:59424) 2024-05-03T18:41:03.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.952+0000 7f5071c00700 1 -- 172.21.15.33:0/3076732367 learned_addr learned my addr 172.21.15.33:0/3076732367 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:41:03.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.953+0000 7f5071200700 1 --2- 172.21.15.33:0/3076732367 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557898831830 0x5578988363a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:03.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.954+0000 7f5072600700 1 --2- 172.21.15.33:0/3076732367 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557898832150 0x5578988368e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:03.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.954+0000 7f5071c00700 1 -- 172.21.15.33:0/3076732367 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557898831830 msgr2=0x5578988363a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:41:03.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.954+0000 7f5071c00700 1 --2- 172.21.15.33:0/3076732367 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557898831830 0x5578988363a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:03.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.954+0000 7f5071c00700 1 -- 172.21.15.33:0/3076732367 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557898832150 msgr2=0x5578988368e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:41:03.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.954+0000 7f5071c00700 1 --2- 172.21.15.33:0/3076732367 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557898832150 0x5578988368e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:03.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.954+0000 7f5071c00700 1 -- 172.21.15.33:0/3076732367 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578987454a0 con 0x55789882d650 2024-05-03T18:41:03.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.955+0000 7f5071c00700 1 --2- 172.21.15.33:0/3076732367 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55789882d650 0x557898839000 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f506000f710 tx=0x7f5060009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4fd2002a08cae29d server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:41:03.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.955+0000 7f5053400700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-03T18:41:03.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.955+0000 7f5053400700 1 -- 172.21.15.33:0/3076732367 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5060033060 con 0x55789882d650 2024-05-03T18:41:03.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.956+0000 7f5053400700 1 -- 172.21.15.33:0/3076732367 <== mon.1 v2:172.21.15.150:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9518+0+0 (secure 0 0 0) 0x7f5060023820 con 0x55789882d650 2024-05-03T18:41:03.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.956+0000 7f507ef7db80 1 -- 172.21.15.33:0/3076732367 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50640ad710 con 0x55789882d650 2024-05-03T18:41:03.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.956+0000 7f5053400700 1 -- 172.21.15.33:0/3076732367 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f50600095c0 con 0x55789882d650 2024-05-03T18:41:03.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.957+0000 7f507ef7db80 10 client.26458 fetch_fsmap learned FSMap version 21 2024-05-03T18:41:03.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.957+0000 7f507ef7db80 1 -- 172.21.15.33:0/3076732367 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5578988554f0 con 0x55789882d650 2024-05-03T18:41:03.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.961+0000 7f5053400700 1 -- 172.21.15.33:0/3076732367 <== mon.1 v2:172.21.15.150:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f50640ad710 con 0x55789882d650 2024-05-03T18:41:03.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.961+0000 7f5053400700 1 -- 172.21.15.33:0/3076732367 <== mon.1 v2:172.21.15.150:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f506002ec60 con 0x55789882d650 2024-05-03T18:41:03.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.962+0000 7f507ef7db80 10 client.26458 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:41:03.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.962+0000 7f507ef7db80 10 client.26458 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:41:03.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.962+0000 7f507ef7db80 20 client.26458 populate_metadata read hostname 'smithi033' 2024-05-03T18:41:03.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.962+0000 7f507ef7db80 1 --2- 172.21.15.33:0/3076732367 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x557898857a80 0x557898859e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:03.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.962+0000 7f507ef7db80 4 client.26458 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:41:03.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.962+0000 7f507ef7db80 1 -- 172.21.15.33:0/3076732367 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5578987fa250 con 0x557898857a80 2024-05-03T18:41:03.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.963+0000 7f5071200700 1 --2- 172.21.15.33:0/3076732367 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x557898857a80 0x557898859e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:03.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.965+0000 7f5071200700 1 --2- 172.21.15.33:0/3076732367 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x557898857a80 0x557898859e70 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5013424b00b6cad server_cookie=a3b5a73bc7187ed0 in_seq=0 out_seq=0 2024-05-03T18:41:03.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.965+0000 7f5053400700 10 client.26458 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:41:03.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.966+0000 7f5053400700 1 -- 172.21.15.33:0/3076732367 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5578987fa250 con 0x557898857a80 2024-05-03T18:41:03.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.966+0000 7f5053400700 10 client.26458 handle_command_reply: tid=0 2024-05-03T18:41:03.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.966+0000 7f507ef7db80 1 client.26458 shutdown 2024-05-03T18:41:03.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.967+0000 7f507ef7db80 1 -- 172.21.15.33:0/3076732367 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55789882d650 msgr2=0x557898839000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:41:03.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.967+0000 7f507ef7db80 1 --2- 172.21.15.33:0/3076732367 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55789882d650 0x557898839000 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f506000f710 tx=0x7f5060009ca0 comp rx=0 tx=0).stop 2024-05-03T18:41:03.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.969+0000 7f507ef7db80 1 -- 172.21.15.33:0/3076732367 shutdown_connections 2024-05-03T18:41:03.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.969+0000 7f507ef7db80 1 --2- 172.21.15.33:0/3076732367 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x557898857a80 0x557898859e70 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:03.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.969+0000 7f5053400700 0 client.26458 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:41:03.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.970+0000 7f507ef7db80 1 --2- 172.21.15.33:0/3076732367 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557898832150 0x5578988368e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:03.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.970+0000 7f507ef7db80 1 --2- 172.21.15.33:0/3076732367 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55789882d650 0x557898839000 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:03.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.970+0000 7f507ef7db80 1 --2- 172.21.15.33:0/3076732367 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557898831830 0x5578988363a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:03.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.970+0000 7f507ef7db80 1 -- 172.21.15.33:0/3076732367 >> 172.21.15.33:0/3076732367 conn(0x55789875bbe0 msgr2=0x557898826f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:41:03.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.971+0000 7f507ef7db80 1 -- 172.21.15.33:0/3076732367 shutdown_connections 2024-05-03T18:41:03.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.971+0000 7f507ef7db80 1 -- 172.21.15.33:0/3076732367 wait complete. 2024-05-03T18:41:03.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:03.972+0000 7f507ef7db80 20 client.26458 trim_cache size 0 max 16384 2024-05-03T18:41:04.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:04.012+0000 7f507ef7db80 1 Processor -- start 2024-05-03T18:41:04.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:04.012+0000 7f507ef7db80 1 -- start start 2024-05-03T18:41:04.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:04.012+0000 7f507ef7db80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55789875bbe0 0x557898835300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:04.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:04.012+0000 7f507ef7db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55789882d650 0x5578988276e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:04.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:04.012+0000 7f507ef7db80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557898831830 0x55789885f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:04.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:04.012+0000 7f507ef7db80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f506c073c10 con 0x557898831830 2024-05-03T18:41:04.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:04.012+0000 7f507ef7db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f506c1a5510 con 0x55789882d650 2024-05-03T18:41:04.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:04.012+0000 7f507ef7db80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f506c074260 con 0x55789875bbe0 2024-05-03T18:41:04.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:04.013+0000 7f5071c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55789875bbe0 0x557898835300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:04.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:04.013+0000 7f5071c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55789875bbe0 0x557898835300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:59428/0 (socket says 172.21.15.33:59428) 2024-05-03T18:41:04.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:04.013+0000 7f5071c00700 1 -- 172.21.15.33:0/4120739799 learned_addr learned my addr 172.21.15.33:0/4120739799 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:41:04.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:04.014+0000 7f5072600700 1 --2- 172.21.15.33:0/4120739799 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557898831830 0x55789885f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:04.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:04.014+0000 7f5071c00700 1 -- 172.21.15.33:0/4120739799 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55789882d650 msgr2=0x5578988276e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:41:04.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:04.014+0000 7f5071c00700 1 --2- 172.21.15.33:0/4120739799 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55789882d650 0x5578988276e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:04.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:04.015+0000 7f5071c00700 1 -- 172.21.15.33:0/4120739799 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557898831830 msgr2=0x55789885f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:41:04.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:04.017+0000 7f5071c00700 1 --2- 172.21.15.33:0/4120739799 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557898831830 0x55789885f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:04.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:04.017+0000 7f5071c00700 1 -- 172.21.15.33:0/4120739799 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55789882d330 con 0x55789875bbe0 2024-05-03T18:41:04.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:04.017+0000 7f5072600700 1 --2- 172.21.15.33:0/4120739799 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557898831830 0x55789885f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:41:04.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:04.018+0000 7f5071c00700 1 --2- 172.21.15.33:0/4120739799 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55789875bbe0 0x557898835300 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f50600173f0 tx=0x7f5060009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12b7d0af78ec7615 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:41:04.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:04.019+0000 7f505e000700 1 -- 172.21.15.33:0/4120739799 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5060033040 con 0x55789875bbe0 2024-05-03T18:41:04.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:04.019+0000 7f505e000700 1 -- 172.21.15.33:0/4120739799 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5060017ce0 con 0x55789875bbe0 2024-05-03T18:41:04.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:04.019+0000 7f505e000700 1 -- 172.21.15.33:0/4120739799 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5060033040 con 0x55789875bbe0 2024-05-03T18:41:04.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:04.021+0000 7f507ef7db80 1 -- 172.21.15.33:0/4120739799 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55789875bbe0 msgr2=0x557898835300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:41:04.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:04.021+0000 7f507ef7db80 1 --2- 172.21.15.33:0/4120739799 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55789875bbe0 0x557898835300 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f50600173f0 tx=0x7f5060009310 comp rx=0 tx=0).stop 2024-05-03T18:41:04.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:04.021+0000 7f507ef7db80 1 -- 172.21.15.33:0/4120739799 shutdown_connections 2024-05-03T18:41:04.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:04.022+0000 7f507ef7db80 1 --2- 172.21.15.33:0/4120739799 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557898831830 0x55789885f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:04.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:04.022+0000 7f507ef7db80 1 --2- 172.21.15.33:0/4120739799 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55789875bbe0 0x557898835300 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:04.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:04.022+0000 7f507ef7db80 1 --2- 172.21.15.33:0/4120739799 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55789882d650 0x5578988276e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:04.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:04.022+0000 7f507ef7db80 1 -- 172.21.15.33:0/4120739799 >> 172.21.15.33:0/4120739799 conn(0x557898831270 msgr2=0x557898838230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:41:04.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:04.023+0000 7f507ef7db80 1 -- 172.21.15.33:0/4120739799 shutdown_connections 2024-05-03T18:41:04.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:04.023+0000 7f507ef7db80 1 -- 172.21.15.33:0/4120739799 wait complete. 2024-05-03T18:41:04.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:04.024+0000 7f507ef7db80 1 -- start start 2024-05-03T18:41:04.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:04.024+0000 7f507ef7db80 1 Processor -- start 2024-05-03T18:41:04.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:04.026+0000 7f507ef7db80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55789875bbe0 0x55789883c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:04.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:04.027+0000 7f507ef7db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55789882d650 0x55789883c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:04.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:04.027+0000 7f507ef7db80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557898831830 0x5578988c22b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:04.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:04.027+0000 7f5071c00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55789875bbe0 0x55789883c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:04.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:04.028+0000 7f507ef7db80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f506c074930 con 0x55789875bbe0 2024-05-03T18:41:04.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:04.028+0000 7f5071c00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55789875bbe0 0x55789883c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36842/0 (socket says 172.21.15.33:36842) 2024-05-03T18:41:04.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:04.028+0000 7f5071c00700 1 -- 172.21.15.33:0/940657095 learned_addr learned my addr 172.21.15.33:0/940657095 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:41:04.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:04.028+0000 7f5072600700 1 --2- 172.21.15.33:0/940657095 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557898831830 0x5578988c22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:04.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:04.028+0000 7f507ef7db80 1 -- 172.21.15.33:0/940657095 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f506c1a4e40 con 0x55789882d650 2024-05-03T18:41:04.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:04.028+0000 7f507ef7db80 1 -- 172.21.15.33:0/940657095 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f506c075000 con 0x557898831830 2024-05-03T18:41:04.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:04.029+0000 7f507ef7db80 10 client.0 _ll_register_callbacks cb 0x557898726cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:41:04.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:04.029+0000 7f5071200700 1 --2- 172.21.15.33:0/940657095 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55789882d650 0x55789883c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:04.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:04.029+0000 7f5071c00700 1 -- 172.21.15.33:0/940657095 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557898831830 msgr2=0x5578988c22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:41:04.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:04.029+0000 7f5071c00700 1 --2- 172.21.15.33:0/940657095 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557898831830 0x5578988c22b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:04.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:04.029+0000 7f5071c00700 1 -- 172.21.15.33:0/940657095 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55789882d650 msgr2=0x55789883c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:41:04.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:04.029+0000 7f5071c00700 1 --2- 172.21.15.33:0/940657095 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55789882d650 0x55789883c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:04.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:04.029+0000 7f5071c00700 1 -- 172.21.15.33:0/940657095 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578988c27f0 con 0x55789875bbe0 2024-05-03T18:41:04.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:04.030+0000 7f5071c00700 1 --2- 172.21.15.33:0/940657095 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55789875bbe0 0x55789883c0a0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f506000b4c0 tx=0x7f5060023980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2cb651183869445 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:41:04.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:04.030+0000 7f5053400700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:41:04.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:04.031+0000 7f5053400700 1 -- 172.21.15.33:0/940657095 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5060033070 con 0x55789875bbe0 2024-05-03T18:41:04.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:04.031+0000 7f507ef7db80 1 -- 172.21.15.33:0/940657095 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5060017ce0 con 0x55789875bbe0 2024-05-03T18:41:04.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:04.031+0000 7f5053400700 1 -- 172.21.15.33:0/940657095 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9518+0+0 (secure 0 0 0) 0x7f5060042880 con 0x55789875bbe0 2024-05-03T18:41:04.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:04.031+0000 7f5053400700 1 -- 172.21.15.33:0/940657095 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f506003d080 con 0x55789875bbe0 2024-05-03T18:41:04.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:04.032+0000 7f507ef7db80 10 client.16662 fetch_fsmap learned FSMap version 21 2024-05-03T18:41:04.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:04.032+0000 7f507ef7db80 1 -- 172.21.15.33:0/940657095 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5578988c38f0 con 0x55789875bbe0 2024-05-03T18:41:04.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:04.033+0000 7f5053400700 1 -- 172.21.15.33:0/940657095 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5060017ce0 con 0x55789875bbe0 2024-05-03T18:41:04.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:04.033+0000 7f5053400700 1 -- 172.21.15.33:0/940657095 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f506002e870 con 0x55789875bbe0 2024-05-03T18:41:04.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:04.033+0000 7f507ef7db80 10 client.16662 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:41:04.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:04.033+0000 7f507ef7db80 10 client.16662 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:41:04.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:04.033+0000 7f507ef7db80 20 client.16662 populate_metadata read hostname 'smithi033' 2024-05-03T18:41:04.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:04.033+0000 7f507ef7db80 1 --2- 172.21.15.33:0/940657095 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x557898832150 0x5578988c5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:04.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:04.033+0000 7f507ef7db80 4 client.16662 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-03T18:41:04.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:04.033+0000 7f507ef7db80 1 -- 172.21.15.33:0/940657095 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55789878d480 con 0x557898832150 2024-05-03T18:41:04.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:04.034+0000 7f5071200700 1 --2- 172.21.15.33:0/940657095 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x557898832150 0x5578988c5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:04.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:04.036+0000 7f5071200700 1 --2- 172.21.15.33:0/940657095 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x557898832150 0x5578988c5cb0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8809fe116fe8bca5 server_cookie=ab7117056d7b3523 in_seq=0 out_seq=0 2024-05-03T18:41:04.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:04.036+0000 7f5053400700 10 client.16662 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:41:04.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:04.040+0000 7f5053400700 1 -- 172.21.15.33:0/940657095 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55789878d480 con 0x557898832150 2024-05-03T18:41:04.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:04.040+0000 7f5053400700 10 client.16662 handle_command_reply: tid=0 2024-05-03T18:41:04.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:04.040+0000 7f507ef7db80 1 client.16662 shutdown 2024-05-03T18:41:04.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:04.040+0000 7f507ef7db80 1 -- 172.21.15.33:0/940657095 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55789875bbe0 msgr2=0x55789883c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:41:04.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:04.040+0000 7f507ef7db80 1 --2- 172.21.15.33:0/940657095 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55789875bbe0 0x55789883c0a0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f506000b4c0 tx=0x7f5060023980 comp rx=0 tx=0).stop 2024-05-03T18:41:04.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:04.040+0000 7f507ef7db80 1 -- 172.21.15.33:0/940657095 shutdown_connections 2024-05-03T18:41:04.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:04.040+0000 7f507ef7db80 1 --2- 172.21.15.33:0/940657095 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x557898832150 0x5578988c5cb0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:04.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:04.041+0000 7f507ef7db80 1 --2- 172.21.15.33:0/940657095 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557898831830 0x5578988c22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:04.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:04.041+0000 7f507ef7db80 1 --2- 172.21.15.33:0/940657095 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55789875bbe0 0x55789883c0a0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:04.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:04.041+0000 7f507ef7db80 1 --2- 172.21.15.33:0/940657095 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55789882d650 0x55789883c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:04.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:04.041+0000 7f507ef7db80 1 -- 172.21.15.33:0/940657095 >> 172.21.15.33:0/940657095 conn(0x557898831270 msgr2=0x557898841d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:41:04.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:04.041+0000 7f5053400700 0 client.16662 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:41:04.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:04.043+0000 7f507ef7db80 1 -- 172.21.15.33:0/940657095 shutdown_connections 2024-05-03T18:41:04.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:04.043+0000 7f507ef7db80 1 -- 172.21.15.33:0/940657095 wait complete. 2024-05-03T18:41:04.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:04.043+0000 7f507ef7db80 20 client.16662 trim_cache size 0 max 16384 2024-05-03T18:41:04.045 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-03T18:41:04.045 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-05-03T18:41:04.045 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "1592e90a-ae6a-455c-9193-5220e70322cf", 2024-05-03T18:41:04.046 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-05-03T18:41:04.046 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-03T18:41:04.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:04.045+0000 7f5074600700 1 -- 172.21.15.33:0/3286197192 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f50540b2bc0 msgr2=0x7f50540b5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:41:04.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:04.045+0000 7f5074600700 1 --2- 172.21.15.33:0/3286197192 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f50540b2bc0 0x7f50540b5080 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5060006010 tx=0x7f506001a040 comp rx=0 tx=0).stop 2024-05-03T18:41:04.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:04.045+0000 7f5074600700 1 -- 172.21.15.33:0/3286197192 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f506c10b6e0 msgr2=0x7f506c10faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:41:04.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:04.045+0000 7f5074600700 1 --2- 172.21.15.33:0/3286197192 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f506c10b6e0 0x7f506c10faf0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f506400efc0 tx=0x7f506400c3a0 comp rx=0 tx=0).stop 2024-05-03T18:41:04.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:04.046+0000 7f5074600700 1 -- 172.21.15.33:0/3286197192 shutdown_connections 2024-05-03T18:41:04.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:04.046+0000 7f5074600700 1 --2- 172.21.15.33:0/3286197192 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f506c10b6e0 0x7f506c10faf0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:04.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:04.046+0000 7f5074600700 1 --2- 172.21.15.33:0/3286197192 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f50540b2bc0 0x7f50540b5080 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:04.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:04.046+0000 7f5074600700 1 --2- 172.21.15.33:0/3286197192 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f506c1056e0 0x7f506c1124a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:04.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:04.046+0000 7f5074600700 1 --2- 172.21.15.33:0/3286197192 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f506c1060a0 0x7f506c10f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:04.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:04.046+0000 7f5074600700 1 -- 172.21.15.33:0/3286197192 >> 172.21.15.33:0/3286197192 conn(0x7f506c1011d0 msgr2=0x7f506c1018e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:41:04.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:04.047+0000 7f5074600700 1 -- 172.21.15.33:0/3286197192 shutdown_connections 2024-05-03T18:41:04.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:04.047+0000 7f5074600700 1 -- 172.21.15.33:0/3286197192 wait complete. 2024-05-03T18:41:04.160 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:41:04 smithi033 ceph-mon[28330]: pgmap v432: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 4.8 MiB/s wr, 454 op/s 2024-05-03T18:41:04.547 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:41:04 smithi099 ceph-mon[31658]: pgmap v432: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 4.8 MiB/s wr, 454 op/s 2024-05-03T18:41:04.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:41:04 smithi150 ceph-mon[32492]: pgmap v432: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 4.8 MiB/s wr, 454 op/s 2024-05-03T18:41:05.110 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 590.0 s, 578.6 tps, lat 173.255 ms stddev 127.743 2024-05-03T18:41:05.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:41:05 smithi033 ceph-mon[28330]: scrub queued for path: / 2024-05-03T18:41:05.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:41:05 smithi033 ceph-mon[28330]: scrub summary: idle+waiting paths [/] 2024-05-03T18:41:05.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:41:05 smithi033 ceph-mon[28330]: scrub summary: active paths [/] 2024-05-03T18:41:05.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:41:05 smithi099 ceph-mon[31658]: scrub queued for path: / 2024-05-03T18:41:05.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:41:05 smithi099 ceph-mon[31658]: scrub summary: idle+waiting paths [/] 2024-05-03T18:41:05.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:41:05 smithi099 ceph-mon[31658]: scrub summary: active paths [/] 2024-05-03T18:41:05.634 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:41:05 smithi150 ceph-mon[32492]: scrub queued for path: / 2024-05-03T18:41:05.634 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:41:05 smithi150 ceph-mon[32492]: scrub summary: idle+waiting paths [/] 2024-05-03T18:41:05.634 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:41:05 smithi150 ceph-mon[32492]: scrub summary: active paths [/] 2024-05-03T18:41:06.267 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-03T18:41:06.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:41:06 smithi033 ceph-mon[28330]: pgmap v433: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 6.6 MiB/s wr, 648 op/s 2024-05-03T18:41:06.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:41:06 smithi033 ceph-mon[28330]: scrub summary: idle+waiting paths [/] 2024-05-03T18:41:06.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:41:06 smithi099 ceph-mon[31658]: pgmap v433: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 6.6 MiB/s wr, 648 op/s 2024-05-03T18:41:06.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:41:06 smithi099 ceph-mon[31658]: scrub summary: idle+waiting paths [/] 2024-05-03T18:41:06.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:41:06 smithi150 ceph-mon[32492]: pgmap v433: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 6.6 MiB/s wr, 648 op/s 2024-05-03T18:41:06.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:41:06 smithi150 ceph-mon[32492]: scrub summary: idle+waiting paths [/] 2024-05-03T18:41:07.061 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:41:07.451 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:41:07 smithi033 ceph-mon[28330]: scrub summary: idle 2024-05-03T18:41:07.451 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:41:07 smithi033 ceph-mon[28330]: osdmap e105: 12 total, 12 up, 12 in 2024-05-03T18:41:07.452 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:41:07 smithi033 ceph-mon[28330]: pgmap v435: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 5.8 MiB/s wr, 566 op/s 2024-05-03T18:41:07.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:41:07 smithi099 ceph-mon[31658]: scrub summary: idle 2024-05-03T18:41:07.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:41:07 smithi099 ceph-mon[31658]: osdmap e105: 12 total, 12 up, 12 in 2024-05-03T18:41:07.547 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:41:07 smithi099 ceph-mon[31658]: pgmap v435: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 5.8 MiB/s wr, 566 op/s 2024-05-03T18:41:07.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:41:07 smithi150 ceph-mon[32492]: scrub summary: idle 2024-05-03T18:41:07.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:41:07 smithi150 ceph-mon[32492]: osdmap e105: 12 total, 12 up, 12 in 2024-05-03T18:41:07.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:41:07 smithi150 ceph-mon[32492]: pgmap v435: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 5.8 MiB/s wr, 566 op/s 2024-05-03T18:41:09.477 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:41:09 smithi033 ceph-mon[28330]: pgmap v436: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 5.8 MiB/s wr, 566 op/s 2024-05-03T18:41:09.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:41:09 smithi099 ceph-mon[31658]: pgmap v436: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 5.8 MiB/s wr, 566 op/s 2024-05-03T18:41:09.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:41:09 smithi150 ceph-mon[32492]: pgmap v436: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 5.8 MiB/s wr, 566 op/s 2024-05-03T18:41:10.111 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 595.0 s, 512.6 tps, lat 194.739 ms stddev 153.112 2024-05-03T18:41:10.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:10.982+0000 7fee35600700 1 -- 172.21.15.33:0/4141523286 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee30102d60 msgr2=0x7fee3010f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:41:10.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:10.982+0000 7fee35600700 1 --2- 172.21.15.33:0/4141523286 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee30102d60 0x7fee3010f250 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fee280098f0 tx=0x7fee28009c00 comp rx=0 tx=0).stop 2024-05-03T18:41:10.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:10.983+0000 7fee35600700 1 -- 172.21.15.33:0/4141523286 shutdown_connections 2024-05-03T18:41:10.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:10.983+0000 7fee35600700 1 --2- 172.21.15.33:0/4141523286 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee30102d60 0x7fee3010f250 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:10.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:10.983+0000 7fee35600700 1 --2- 172.21.15.33:0/4141523286 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fee30108410 0x7fee301087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:10.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:10.983+0000 7fee35600700 1 --2- 172.21.15.33:0/4141523286 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee301023a0 0x7fee30102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:10.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:10.983+0000 7fee35600700 1 -- 172.21.15.33:0/4141523286 >> 172.21.15.33:0/4141523286 conn(0x7fee30079350 msgr2=0x7fee30079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:41:10.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:10.983+0000 7fee35600700 1 -- 172.21.15.33:0/4141523286 shutdown_connections 2024-05-03T18:41:10.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:10.983+0000 7fee35600700 1 -- 172.21.15.33:0/4141523286 wait complete. 2024-05-03T18:41:10.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:10.985+0000 7fee35600700 1 Processor -- start 2024-05-03T18:41:10.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:10.985+0000 7fee35600700 1 -- start start 2024-05-03T18:41:10.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:10.985+0000 7fee35600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee301023a0 0x7fee3006ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:10.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:10.985+0000 7fee35600700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fee30102d60 0x7fee3006f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:10.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:10.985+0000 7fee35600700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee30108410 0x7fee30073c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:10.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:10.985+0000 7fee35600700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fee3006d620 con 0x7fee30108410 2024-05-03T18:41:10.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:10.985+0000 7fee35600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fee3006d4a0 con 0x7fee301023a0 2024-05-03T18:41:10.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:10.985+0000 7fee35600700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fee3006d7a0 con 0x7fee30102d60 2024-05-03T18:41:10.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:10.986+0000 7fee2ea00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee301023a0 0x7fee3006ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:10.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:10.986+0000 7fee26000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fee30102d60 0x7fee3006f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:10.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:10.986+0000 7fee26000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fee30102d60 0x7fee3006f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:49202/0 (socket says 172.21.15.33:49202) 2024-05-03T18:41:10.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:10.986+0000 7fee26000700 1 -- 172.21.15.33:0/3879595843 learned_addr learned my addr 172.21.15.33:0/3879595843 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:41:10.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:10.986+0000 7fee2f400700 1 --2- 172.21.15.33:0/3879595843 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee30108410 0x7fee30073c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:10.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:10.987+0000 7fee2ea00700 1 -- 172.21.15.33:0/3879595843 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fee30102d60 msgr2=0x7fee3006f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:41:10.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:10.987+0000 7fee2ea00700 1 --2- 172.21.15.33:0/3879595843 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fee30102d60 0x7fee3006f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:10.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:10.987+0000 7fee2ea00700 1 -- 172.21.15.33:0/3879595843 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee30108410 msgr2=0x7fee30073c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:41:10.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:10.987+0000 7fee2ea00700 1 --2- 172.21.15.33:0/3879595843 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee30108410 0x7fee30073c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:10.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:10.987+0000 7fee2ea00700 1 -- 172.21.15.33:0/3879595843 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee30074170 con 0x7fee301023a0 2024-05-03T18:41:10.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:10.988+0000 7fee26000700 1 --2- 172.21.15.33:0/3879595843 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fee30102d60 0x7fee3006f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:41:10.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:10.988+0000 7fee2ea00700 1 --2- 172.21.15.33:0/3879595843 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee301023a0 0x7fee3006ebe0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fee20009fd0 tx=0x7fee2000c620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:41:10.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:10.988+0000 7fee27e00700 1 -- 172.21.15.33:0/3879595843 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee20018020 con 0x7fee301023a0 2024-05-03T18:41:10.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:10.989+0000 7fee35600700 1 -- 172.21.15.33:0/3879595843 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee30074460 con 0x7fee301023a0 2024-05-03T18:41:10.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:10.989+0000 7fee35600700 1 -- 172.21.15.33:0/3879595843 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee300749f0 con 0x7fee301023a0 2024-05-03T18:41:10.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:10.989+0000 7fee27e00700 1 -- 172.21.15.33:0/3879595843 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fee20003a00 con 0x7fee301023a0 2024-05-03T18:41:10.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:10.989+0000 7fee27e00700 1 -- 172.21.15.33:0/3879595843 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee200079e0 con 0x7fee301023a0 2024-05-03T18:41:10.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:10.996+0000 7fee27e00700 1 -- 172.21.15.33:0/3879595843 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fee20012050 con 0x7fee301023a0 2024-05-03T18:41:10.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:10.996+0000 7fee3ffe5b80 1 Processor -- start 2024-05-03T18:41:11.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:10.997+0000 7fee3ffe5b80 1 -- start start 2024-05-03T18:41:11.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.000+0000 7fee27e00700 1 --2- 172.21.15.33:0/3879595843 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fee140b2af0 0x7fee140b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:11.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.000+0000 7fee27e00700 1 -- 172.21.15.33:0/3879595843 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9606+0+0 (secure 0 0 0) 0x7fee200e2680 con 0x7fee301023a0 2024-05-03T18:41:11.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.001+0000 7fee3ffe5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbf29027b0 0x55bbf2902b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:11.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.001+0000 7fee3ffe5b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbf29030d0 0x55bbf28fdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:11.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.001+0000 7fee3ffe5b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbf28fe5d0 0x55bbf28fea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:11.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.001+0000 7fee3ffe5b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55bbf2850a20 con 0x55bbf28fe5d0 2024-05-03T18:41:11.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.001+0000 7fee3ffe5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55bbf27ee760 con 0x55bbf29027b0 2024-05-03T18:41:11.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.001+0000 7fee3ffe5b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55bbf284c9e0 con 0x55bbf29030d0 2024-05-03T18:41:11.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.002+0000 7fee2ea00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbf29027b0 0x55bbf2902b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:11.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.002+0000 7fee2ea00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbf29027b0 0x55bbf2902b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:49920/0 (socket says 172.21.15.33:49920) 2024-05-03T18:41:11.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.002+0000 7fee2ea00700 1 -- 172.21.15.33:0/357208271 learned_addr learned my addr 172.21.15.33:0/357208271 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:41:11.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.002+0000 7fee2f400700 1 --2- 172.21.15.33:0/3879595843 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fee140b2af0 0x7fee140b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:11.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.002+0000 7fee26000700 1 --2- 172.21.15.33:0/357208271 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbf29030d0 0x55bbf28fdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:11.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.004+0000 7fee2ea00700 1 -- 172.21.15.33:0/357208271 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbf29030d0 msgr2=0x55bbf28fdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:41:11.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.004+0000 7fee2f400700 1 --2- 172.21.15.33:0/357208271 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbf28fe5d0 0x55bbf28fea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:11.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.004+0000 7fee2ea00700 1 --2- 172.21.15.33:0/357208271 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbf29030d0 0x55bbf28fdfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:11.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.004+0000 7fee2ea00700 1 -- 172.21.15.33:0/357208271 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbf28fe5d0 msgr2=0x55bbf28fea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:41:11.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.005+0000 7fee2ea00700 1 --2- 172.21.15.33:0/357208271 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbf28fe5d0 0x55bbf28fea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:11.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.005+0000 7fee2ea00700 1 -- 172.21.15.33:0/357208271 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbf279d8d0 con 0x55bbf29027b0 2024-05-03T18:41:11.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.005+0000 7fee26000700 1 --2- 172.21.15.33:0/357208271 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbf29030d0 0x55bbf28fdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:41:11.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.005+0000 7fee2f400700 1 --2- 172.21.15.33:0/357208271 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbf28fe5d0 0x55bbf28fea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:41:11.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.005+0000 7fee2f400700 1 --2- 172.21.15.33:0/3879595843 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fee140b2af0 0x7fee140b4fb0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fee28009fd0 tx=0x7fee28019040 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:41:11.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.006+0000 7fee2ea00700 1 --2- 172.21.15.33:0/357208271 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbf29027b0 0x55bbf2902b90 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fee200a55d0 tx=0x7fee200a8fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5307cc057da925db server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:41:11.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.007+0000 7fee1fe00700 1 -- 172.21.15.33:0/357208271 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee200dfdc0 con 0x55bbf29027b0 2024-05-03T18:41:11.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.007+0000 7fee1fe00700 1 -- 172.21.15.33:0/357208271 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fee200a9550 con 0x55bbf29027b0 2024-05-03T18:41:11.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.008+0000 7fee1fe00700 1 -- 172.21.15.33:0/357208271 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee200b20f0 con 0x55bbf29027b0 2024-05-03T18:41:11.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.009+0000 7fee3ffe5b80 1 -- 172.21.15.33:0/357208271 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbf29027b0 msgr2=0x55bbf2902b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:41:11.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.009+0000 7fee3ffe5b80 1 --2- 172.21.15.33:0/357208271 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbf29027b0 0x55bbf2902b90 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fee200a55d0 tx=0x7fee200a8fa0 comp rx=0 tx=0).stop 2024-05-03T18:41:11.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.009+0000 7fee3ffe5b80 1 -- 172.21.15.33:0/357208271 shutdown_connections 2024-05-03T18:41:11.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.009+0000 7fee3ffe5b80 1 --2- 172.21.15.33:0/357208271 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbf28fe5d0 0x55bbf28fea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:11.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.009+0000 7fee3ffe5b80 1 --2- 172.21.15.33:0/357208271 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbf29027b0 0x55bbf2902b90 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:11.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.009+0000 7fee3ffe5b80 1 --2- 172.21.15.33:0/357208271 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbf29030d0 0x55bbf28fdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:11.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.009+0000 7fee3ffe5b80 1 -- 172.21.15.33:0/357208271 >> 172.21.15.33:0/357208271 conn(0x55bbf282cb60 msgr2=0x55bbf28f87f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:41:11.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.009+0000 7fee3ffe5b80 1 -- 172.21.15.33:0/357208271 shutdown_connections 2024-05-03T18:41:11.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.010+0000 7fee3ffe5b80 1 -- 172.21.15.33:0/357208271 wait complete. 2024-05-03T18:41:11.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.011+0000 7fee3ffe5b80 1 -- start start 2024-05-03T18:41:11.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.011+0000 7fee3ffe5b80 1 Processor -- start 2024-05-03T18:41:11.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.013+0000 7fee3ffe5b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbf28fe5d0 0x55bbf2909f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:11.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.013+0000 7fee3ffe5b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbf29027b0 0x55bbf29072d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:11.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.013+0000 7fee3ffe5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbf29030d0 0x55bbf2907810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:11.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.014+0000 7fee3ffe5b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55bbf290ac40 con 0x55bbf29027b0 2024-05-03T18:41:11.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.014+0000 7fee3ffe5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55bbf27b6b20 con 0x55bbf29030d0 2024-05-03T18:41:11.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.014+0000 7fee3ffe5b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55bbf290adc0 con 0x55bbf28fe5d0 2024-05-03T18:41:11.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.014+0000 7fee3ffe5b80 10 client.0 _ll_register_callbacks cb 0x55bbf27f7c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:41:11.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.014+0000 7fee2f400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbf29030d0 0x55bbf2907810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:11.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.014+0000 7fee2f400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbf29030d0 0x55bbf2907810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:49936/0 (socket says 172.21.15.33:49936) 2024-05-03T18:41:11.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.014+0000 7fee2f400700 1 -- 172.21.15.33:0/1979247919 learned_addr learned my addr 172.21.15.33:0/1979247919 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:41:11.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.015+0000 7fee26000700 1 --2- 172.21.15.33:0/1979247919 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbf29027b0 0x55bbf29072d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:11.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.015+0000 7fee2f400700 1 -- 172.21.15.33:0/1979247919 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbf28fe5d0 msgr2=0x55bbf2909f30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-03T18:41:11.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.015+0000 7fee2f400700 1 --2- 172.21.15.33:0/1979247919 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbf28fe5d0 0x55bbf2909f30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:11.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.015+0000 7fee2f400700 1 -- 172.21.15.33:0/1979247919 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbf29027b0 msgr2=0x55bbf29072d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:41:11.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.015+0000 7fee2f400700 1 --2- 172.21.15.33:0/1979247919 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbf29027b0 0x55bbf29072d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:11.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.015+0000 7fee2f400700 1 -- 172.21.15.33:0/1979247919 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbf2816420 con 0x55bbf29030d0 2024-05-03T18:41:11.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.016+0000 7fee26000700 1 --2- 172.21.15.33:0/1979247919 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbf29027b0 0x55bbf29072d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:41:11.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.016+0000 7fee2f400700 1 --2- 172.21.15.33:0/1979247919 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbf29030d0 0x55bbf2907810 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fee28016000 tx=0x7fee28009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ffae3f7012d8c06c server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:41:11.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.017+0000 7fee1cc00700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-03T18:41:11.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.017+0000 7fee1cc00700 1 -- 172.21.15.33:0/1979247919 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee28033060 con 0x55bbf29030d0 2024-05-03T18:41:11.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.017+0000 7fee1cc00700 1 -- 172.21.15.33:0/1979247919 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9606+0+0 (secure 0 0 0) 0x7fee28020040 con 0x55bbf29030d0 2024-05-03T18:41:11.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.018+0000 7fee3ffe5b80 1 -- 172.21.15.33:0/1979247919 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee200a9550 con 0x55bbf29030d0 2024-05-03T18:41:11.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.020+0000 7fee1cc00700 1 -- 172.21.15.33:0/1979247919 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fee2800f920 con 0x55bbf29030d0 2024-05-03T18:41:11.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.020+0000 7fee3ffe5b80 10 client.36392 fetch_fsmap learned FSMap version 21 2024-05-03T18:41:11.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.020+0000 7fee3ffe5b80 1 -- 172.21.15.33:0/1979247919 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55bbf29154d0 con 0x55bbf29030d0 2024-05-03T18:41:11.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.021+0000 7fee1cc00700 1 -- 172.21.15.33:0/1979247919 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fee200a9550 con 0x55bbf29030d0 2024-05-03T18:41:11.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.021+0000 7fee1cc00700 1 -- 172.21.15.33:0/1979247919 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fee28023070 con 0x55bbf29030d0 2024-05-03T18:41:11.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.021+0000 7fee3ffe5b80 10 client.36392 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:41:11.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.021+0000 7fee3ffe5b80 10 client.36392 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:41:11.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.021+0000 7fee3ffe5b80 20 client.36392 populate_metadata read hostname 'smithi033' 2024-05-03T18:41:11.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.021+0000 7fee3ffe5b80 1 --2- 172.21.15.33:0/1979247919 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55bbf2917b00 0x55bbf2919f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:11.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.021+0000 7fee3ffe5b80 4 client.36392 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:41:11.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.021+0000 7fee3ffe5b80 1 -- 172.21.15.33:0/1979247919 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bbf28cb1d0 con 0x55bbf2917b00 2024-05-03T18:41:11.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.022+0000 7fee2ea00700 1 --2- 172.21.15.33:0/1979247919 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55bbf2917b00 0x55bbf2919f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:11.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.023+0000 7fee2ea00700 1 --2- 172.21.15.33:0/1979247919 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55bbf2917b00 0x55bbf2919f20 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=747adcb58a766cb0 server_cookie=f10da89e2b3f6111 in_seq=0 out_seq=0 2024-05-03T18:41:11.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.023+0000 7fee1cc00700 10 client.36392 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:41:11.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.024+0000 7fee1cc00700 1 -- 172.21.15.33:0/1979247919 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55bbf28cb1d0 con 0x55bbf2917b00 2024-05-03T18:41:11.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.024+0000 7fee1cc00700 10 client.36392 handle_command_reply: tid=0 2024-05-03T18:41:11.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.025+0000 7fee3ffe5b80 1 client.36392 shutdown 2024-05-03T18:41:11.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.025+0000 7fee3ffe5b80 1 -- 172.21.15.33:0/1979247919 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbf29030d0 msgr2=0x55bbf2907810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:41:11.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.025+0000 7fee3ffe5b80 1 --2- 172.21.15.33:0/1979247919 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbf29030d0 0x55bbf2907810 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fee28016000 tx=0x7fee28009260 comp rx=0 tx=0).stop 2024-05-03T18:41:11.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.025+0000 7fee3ffe5b80 1 -- 172.21.15.33:0/1979247919 shutdown_connections 2024-05-03T18:41:11.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.025+0000 7fee3ffe5b80 1 --2- 172.21.15.33:0/1979247919 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55bbf2917b00 0x55bbf2919f20 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:11.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.025+0000 7fee3ffe5b80 1 --2- 172.21.15.33:0/1979247919 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbf29030d0 0x55bbf2907810 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:11.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.025+0000 7fee3ffe5b80 1 --2- 172.21.15.33:0/1979247919 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbf28fe5d0 0x55bbf2909f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:11.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.025+0000 7fee3ffe5b80 1 --2- 172.21.15.33:0/1979247919 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbf29027b0 0x55bbf29072d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:11.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.025+0000 7fee3ffe5b80 1 -- 172.21.15.33:0/1979247919 >> 172.21.15.33:0/1979247919 conn(0x55bbf282cb60 msgr2=0x55bbf28f7650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:41:11.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.025+0000 7fee1cc00700 0 client.36392 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:41:11.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.026+0000 7fee3ffe5b80 1 -- 172.21.15.33:0/1979247919 shutdown_connections 2024-05-03T18:41:11.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.026+0000 7fee3ffe5b80 1 -- 172.21.15.33:0/1979247919 wait complete. 2024-05-03T18:41:11.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.026+0000 7fee3ffe5b80 20 client.36392 trim_cache size 0 max 16384 2024-05-03T18:41:11.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.063+0000 7fee3ffe5b80 1 Processor -- start 2024-05-03T18:41:11.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.063+0000 7fee3ffe5b80 1 -- start start 2024-05-03T18:41:11.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.064+0000 7fee3ffe5b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbf28fe5d0 0x55bbf28fdc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:11.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.064+0000 7fee3ffe5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbf29027b0 0x55bbf28f8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:11.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.064+0000 7fee3ffe5b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbf29030d0 0x55bbf2901b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:11.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.064+0000 7fee3ffe5b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fee0c00cf10 con 0x55bbf29030d0 2024-05-03T18:41:11.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.064+0000 7fee3ffe5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fee0c00f770 con 0x55bbf29027b0 2024-05-03T18:41:11.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.064+0000 7fee3ffe5b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fee0c00d560 con 0x55bbf28fe5d0 2024-05-03T18:41:11.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.064+0000 7fee2f400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbf29030d0 0x55bbf2901b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:11.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.064+0000 7fee2f400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbf29030d0 0x55bbf2901b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60444/0 (socket says 172.21.15.33:60444) 2024-05-03T18:41:11.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.064+0000 7fee2f400700 1 -- 172.21.15.33:0/2733806952 learned_addr learned my addr 172.21.15.33:0/2733806952 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:41:11.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.065+0000 7fee2ea00700 1 --2- 172.21.15.33:0/2733806952 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbf28fe5d0 0x55bbf28fdc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:11.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.065+0000 7fee26000700 1 --2- 172.21.15.33:0/2733806952 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbf29027b0 0x55bbf28f8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:11.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.065+0000 7fee2f400700 1 -- 172.21.15.33:0/2733806952 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbf28fe5d0 msgr2=0x55bbf28fdc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:41:11.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.065+0000 7fee2f400700 1 --2- 172.21.15.33:0/2733806952 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbf28fe5d0 0x55bbf28fdc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:11.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.065+0000 7fee2f400700 1 -- 172.21.15.33:0/2733806952 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbf29027b0 msgr2=0x55bbf28f8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:41:11.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.065+0000 7fee2f400700 1 --2- 172.21.15.33:0/2733806952 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbf29027b0 0x55bbf28f8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:11.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.065+0000 7fee2f400700 1 -- 172.21.15.33:0/2733806952 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbf2905840 con 0x55bbf29030d0 2024-05-03T18:41:11.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.066+0000 7fee26000700 1 --2- 172.21.15.33:0/2733806952 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbf29027b0 0x55bbf28f8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:41:11.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.066+0000 7fee2ea00700 1 --2- 172.21.15.33:0/2733806952 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbf28fe5d0 0x55bbf28fdc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:41:11.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.066+0000 7fee2f400700 1 --2- 172.21.15.33:0/2733806952 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbf29030d0 0x55bbf2901b00 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fee28009810 tx=0x7fee2800f940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e53eb164d8e971d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:41:11.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.067+0000 7fee1f400700 1 -- 172.21.15.33:0/2733806952 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee2803d080 con 0x55bbf29030d0 2024-05-03T18:41:11.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.067+0000 7fee1f400700 1 -- 172.21.15.33:0/2733806952 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fee2800b820 con 0x55bbf29030d0 2024-05-03T18:41:11.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.067+0000 7fee1f400700 1 -- 172.21.15.33:0/2733806952 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee2803d080 con 0x55bbf29030d0 2024-05-03T18:41:11.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.068+0000 7fee3ffe5b80 1 -- 172.21.15.33:0/2733806952 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbf29030d0 msgr2=0x55bbf2901b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:41:11.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.068+0000 7fee3ffe5b80 1 --2- 172.21.15.33:0/2733806952 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbf29030d0 0x55bbf2901b00 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fee28009810 tx=0x7fee2800f940 comp rx=0 tx=0).stop 2024-05-03T18:41:11.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.069+0000 7fee3ffe5b80 1 -- 172.21.15.33:0/2733806952 shutdown_connections 2024-05-03T18:41:11.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.069+0000 7fee3ffe5b80 1 --2- 172.21.15.33:0/2733806952 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbf29030d0 0x55bbf2901b00 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:11.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.069+0000 7fee3ffe5b80 1 --2- 172.21.15.33:0/2733806952 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbf28fe5d0 0x55bbf28fdc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:11.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.069+0000 7fee3ffe5b80 1 --2- 172.21.15.33:0/2733806952 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbf29027b0 0x55bbf28f8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:11.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.069+0000 7fee3ffe5b80 1 -- 172.21.15.33:0/2733806952 >> 172.21.15.33:0/2733806952 conn(0x55bbf282cb60 msgr2=0x55bbf29013f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:41:11.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.069+0000 7fee3ffe5b80 1 -- 172.21.15.33:0/2733806952 shutdown_connections 2024-05-03T18:41:11.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.069+0000 7fee3ffe5b80 1 -- 172.21.15.33:0/2733806952 wait complete. 2024-05-03T18:41:11.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.070+0000 7fee3ffe5b80 1 -- start start 2024-05-03T18:41:11.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.070+0000 7fee3ffe5b80 1 Processor -- start 2024-05-03T18:41:11.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.071+0000 7fee3ffe5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbf28fe5d0 0x55bbf2910330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:11.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.071+0000 7fee3ffe5b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbf29027b0 0x55bbf290c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:11.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.071+0000 7fee3ffe5b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbf29030d0 0x55bbf290cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:11.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.071+0000 7fee3ffe5b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fee0c00dc30 con 0x55bbf29027b0 2024-05-03T18:41:11.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.071+0000 7fee3ffe5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fee0c00f0a0 con 0x55bbf28fe5d0 2024-05-03T18:41:11.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.071+0000 7fee3ffe5b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fee0c00e300 con 0x55bbf29030d0 2024-05-03T18:41:11.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.071+0000 7fee3ffe5b80 10 client.0 _ll_register_callbacks cb 0x55bbf27f7c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:41:11.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.071+0000 7fee26000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbf29027b0 0x55bbf290c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:11.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.071+0000 7fee26000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbf29027b0 0x55bbf290c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60452/0 (socket says 172.21.15.33:60452) 2024-05-03T18:41:11.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.071+0000 7fee26000700 1 -- 172.21.15.33:0/839425452 learned_addr learned my addr 172.21.15.33:0/839425452 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:41:11.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.072+0000 7fee2ea00700 1 --2- 172.21.15.33:0/839425452 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbf28fe5d0 0x55bbf2910330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:11.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.073+0000 7fee26000700 1 -- 172.21.15.33:0/839425452 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbf29030d0 msgr2=0x55bbf290cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:41:11.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.073+0000 7fee26000700 1 --2- 172.21.15.33:0/839425452 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbf29030d0 0x55bbf290cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:11.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.073+0000 7fee26000700 1 -- 172.21.15.33:0/839425452 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbf28fe5d0 msgr2=0x55bbf2910330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:41:11.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.073+0000 7fee26000700 1 --2- 172.21.15.33:0/839425452 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbf28fe5d0 0x55bbf2910330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:11.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.073+0000 7fee26000700 1 -- 172.21.15.33:0/839425452 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbf290d140 con 0x55bbf29027b0 2024-05-03T18:41:11.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.074+0000 7fee26000700 1 --2- 172.21.15.33:0/839425452 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbf29027b0 0x55bbf290c6c0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fee3006bcf0 tx=0x7fee18010710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b9eb12eb08aa292 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:41:11.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.075+0000 7fee1cc00700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:41:11.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.075+0000 7fee1cc00700 1 -- 172.21.15.33:0/839425452 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee18021020 con 0x55bbf29027b0 2024-05-03T18:41:11.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.075+0000 7fee3ffe5b80 1 -- 172.21.15.33:0/839425452 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee2800b820 con 0x55bbf29027b0 2024-05-03T18:41:11.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.076+0000 7fee1cc00700 1 -- 172.21.15.33:0/839425452 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9606+0+0 (secure 0 0 0) 0x7fee18004220 con 0x55bbf29027b0 2024-05-03T18:41:11.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.077+0000 7fee1cc00700 1 -- 172.21.15.33:0/839425452 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fee18029050 con 0x55bbf29027b0 2024-05-03T18:41:11.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.077+0000 7fee3ffe5b80 10 client.16674 fetch_fsmap learned FSMap version 21 2024-05-03T18:41:11.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.077+0000 7fee3ffe5b80 1 -- 172.21.15.33:0/839425452 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55bbf2983210 con 0x55bbf29027b0 2024-05-03T18:41:11.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.078+0000 7fee1cc00700 1 -- 172.21.15.33:0/839425452 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fee2800b820 con 0x55bbf29027b0 2024-05-03T18:41:11.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.078+0000 7fee1cc00700 1 -- 172.21.15.33:0/839425452 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fee1801c070 con 0x55bbf29027b0 2024-05-03T18:41:11.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.078+0000 7fee3ffe5b80 10 client.16674 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:41:11.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.078+0000 7fee3ffe5b80 10 client.16674 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:41:11.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.078+0000 7fee3ffe5b80 20 client.16674 populate_metadata read hostname 'smithi033' 2024-05-03T18:41:11.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.078+0000 7fee3ffe5b80 1 --2- 172.21.15.33:0/839425452 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55bbf2983980 0x55bbf2985da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:11.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.078+0000 7fee3ffe5b80 4 client.16674 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-03T18:41:11.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.078+0000 7fee3ffe5b80 1 -- 172.21.15.33:0/839425452 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bbf285e400 con 0x55bbf2983980 2024-05-03T18:41:11.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.079+0000 7fee2ea00700 1 --2- 172.21.15.33:0/839425452 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55bbf2983980 0x55bbf2985da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:11.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.080+0000 7fee2ea00700 1 --2- 172.21.15.33:0/839425452 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55bbf2983980 0x55bbf2985da0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45c23cdf8f4d4542 server_cookie=54e6721af7022a7d in_seq=0 out_seq=0 2024-05-03T18:41:11.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.080+0000 7fee1cc00700 10 client.16674 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:41:11.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.080+0000 7fee1cc00700 1 -- 172.21.15.33:0/839425452 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55bbf285e400 con 0x55bbf2983980 2024-05-03T18:41:11.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.080+0000 7fee1cc00700 10 client.16674 handle_command_reply: tid=0 2024-05-03T18:41:11.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.080+0000 7fee3ffe5b80 1 client.16674 shutdown 2024-05-03T18:41:11.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.080+0000 7fee3ffe5b80 1 -- 172.21.15.33:0/839425452 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbf29027b0 msgr2=0x55bbf290c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:41:11.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.080+0000 7fee3ffe5b80 1 --2- 172.21.15.33:0/839425452 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbf29027b0 0x55bbf290c6c0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fee3006bcf0 tx=0x7fee18010710 comp rx=0 tx=0).stop 2024-05-03T18:41:11.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.081+0000 7fee3ffe5b80 1 -- 172.21.15.33:0/839425452 shutdown_connections 2024-05-03T18:41:11.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.081+0000 7fee3ffe5b80 1 --2- 172.21.15.33:0/839425452 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55bbf2983980 0x55bbf2985da0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:11.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.081+0000 7fee3ffe5b80 1 --2- 172.21.15.33:0/839425452 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbf29030d0 0x55bbf290cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:11.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.081+0000 7fee3ffe5b80 1 --2- 172.21.15.33:0/839425452 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbf28fe5d0 0x55bbf2910330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:11.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.081+0000 7fee3ffe5b80 1 --2- 172.21.15.33:0/839425452 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbf29027b0 0x55bbf290c6c0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:11.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.081+0000 7fee3ffe5b80 1 -- 172.21.15.33:0/839425452 >> 172.21.15.33:0/839425452 conn(0x55bbf282cb60 msgr2=0x55bbf2901c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:41:11.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.081+0000 7fee1cc00700 0 client.16674 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:41:11.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.081+0000 7fee3ffe5b80 1 -- 172.21.15.33:0/839425452 shutdown_connections 2024-05-03T18:41:11.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.081+0000 7fee3ffe5b80 1 -- 172.21.15.33:0/839425452 wait complete. 2024-05-03T18:41:11.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.081+0000 7fee3ffe5b80 20 client.16674 trim_cache size 0 max 16384 2024-05-03T18:41:11.083 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-03T18:41:11.083 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-05-03T18:41:11.083 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-05-03T18:41:11.083 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-03T18:41:11.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.082+0000 7fee35600700 1 -- 172.21.15.33:0/3879595843 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fee140b2af0 msgr2=0x7fee140b4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:41:11.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.082+0000 7fee35600700 1 --2- 172.21.15.33:0/3879595843 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fee140b2af0 0x7fee140b4fb0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fee28009fd0 tx=0x7fee28019040 comp rx=0 tx=0).stop 2024-05-03T18:41:11.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.082+0000 7fee35600700 1 -- 172.21.15.33:0/3879595843 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee301023a0 msgr2=0x7fee3006ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:41:11.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.082+0000 7fee35600700 1 --2- 172.21.15.33:0/3879595843 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee301023a0 0x7fee3006ebe0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fee20009fd0 tx=0x7fee2000c620 comp rx=0 tx=0).stop 2024-05-03T18:41:11.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.083+0000 7fee35600700 1 -- 172.21.15.33:0/3879595843 shutdown_connections 2024-05-03T18:41:11.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.083+0000 7fee35600700 1 --2- 172.21.15.33:0/3879595843 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee30108410 0x7fee30073c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:11.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.083+0000 7fee35600700 1 --2- 172.21.15.33:0/3879595843 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fee140b2af0 0x7fee140b4fb0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:11.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.083+0000 7fee35600700 1 --2- 172.21.15.33:0/3879595843 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee301023a0 0x7fee3006ebe0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:11.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.083+0000 7fee35600700 1 --2- 172.21.15.33:0/3879595843 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fee30102d60 0x7fee3006f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:11.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.083+0000 7fee35600700 1 -- 172.21.15.33:0/3879595843 >> 172.21.15.33:0/3879595843 conn(0x7fee30079350 msgr2=0x7fee30101070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:41:11.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.083+0000 7fee35600700 1 -- 172.21.15.33:0/3879595843 shutdown_connections 2024-05-03T18:41:11.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:11.083+0000 7fee35600700 1 -- 172.21.15.33:0/3879595843 wait complete. 2024-05-03T18:41:11.435 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:41:11 smithi033 ceph-mon[28330]: pgmap v437: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 5.4 MiB/s wr, 545 op/s 2024-05-03T18:41:11.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:41:11 smithi099 ceph-mon[31658]: pgmap v437: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 5.4 MiB/s wr, 545 op/s 2024-05-03T18:41:11.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:41:11 smithi150 ceph-mon[32492]: pgmap v437: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 5.4 MiB/s wr, 545 op/s 2024-05-03T18:41:13.735 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:41:13 smithi033 ceph-mon[28330]: pgmap v438: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 5.4 MiB/s wr, 545 op/s 2024-05-03T18:41:13.735 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:41:13 smithi033 ceph-mon[28330]: osdmap e106: 12 total, 12 up, 12 in 2024-05-03T18:41:13.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:41:13 smithi099 ceph-mon[31658]: pgmap v438: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 5.4 MiB/s wr, 545 op/s 2024-05-03T18:41:13.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:41:13 smithi099 ceph-mon[31658]: osdmap e106: 12 total, 12 up, 12 in 2024-05-03T18:41:13.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:41:13 smithi150 ceph-mon[32492]: pgmap v438: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 5.4 MiB/s wr, 545 op/s 2024-05-03T18:41:13.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:41:13 smithi150 ceph-mon[32492]: osdmap e106: 12 total, 12 up, 12 in 2024-05-03T18:41:14.797 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-03T18:41:14.798 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T18:41:15.110 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 600.0 s, 536.4 tps, lat 185.812 ms stddev 118.899 2024-05-03T18:41:15.554 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:41:15.735 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:41:15 smithi033 ceph-mon[28330]: pgmap v440: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 5.8 MiB/s wr, 579 op/s 2024-05-03T18:41:15.797 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:41:15 smithi099 ceph-mon[31658]: pgmap v440: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 5.8 MiB/s wr, 579 op/s 2024-05-03T18:41:15.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:41:15 smithi150 ceph-mon[32492]: pgmap v440: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 5.8 MiB/s wr, 579 op/s 2024-05-03T18:41:17.735 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:41:17 smithi033 ceph-mon[28330]: pgmap v441: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 4.9 MiB/s wr, 502 op/s 2024-05-03T18:41:17.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:41:17 smithi099 ceph-mon[31658]: pgmap v441: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 4.9 MiB/s wr, 502 op/s 2024-05-03T18:41:17.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:41:17 smithi150 ceph-mon[32492]: pgmap v441: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 4.9 MiB/s wr, 502 op/s 2024-05-03T18:41:18.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:41:18 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:41:18.859 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:41:18 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:41:18.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:41:18 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:41:19.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:19.720+0000 7ff323c00700 1 -- 172.21.15.33:0/102084242 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff31c1153f0 msgr2=0x7ff31c1177e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:41:19.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:19.720+0000 7ff323c00700 1 --2- 172.21.15.33:0/102084242 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff31c1153f0 0x7ff31c1177e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff30c009990 tx=0x7ff30c009ca0 comp rx=0 tx=0).stop 2024-05-03T18:41:19.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:19.720+0000 7ff323c00700 1 -- 172.21.15.33:0/102084242 shutdown_connections 2024-05-03T18:41:19.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:19.720+0000 7ff323c00700 1 --2- 172.21.15.33:0/102084242 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff31c1153f0 0x7ff31c1177e0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:19.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:19.720+0000 7ff323c00700 1 --2- 172.21.15.33:0/102084242 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff31c10be40 0x7ff31c10c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:19.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:19.720+0000 7ff323c00700 1 --2- 172.21.15.33:0/102084242 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff31c10c760 0x7ff31c114eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:19.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:19.720+0000 7ff323c00700 1 -- 172.21.15.33:0/102084242 >> 172.21.15.33:0/102084242 conn(0x7ff31c06e610 msgr2=0x7ff31c06ea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:41:19.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:19.721+0000 7ff323c00700 1 -- 172.21.15.33:0/102084242 shutdown_connections 2024-05-03T18:41:19.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:19.724+0000 7ff323c00700 1 -- 172.21.15.33:0/102084242 wait complete. 2024-05-03T18:41:19.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:19.726+0000 7ff323c00700 1 Processor -- start 2024-05-03T18:41:19.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:19.727+0000 7ff323c00700 1 -- start start 2024-05-03T18:41:19.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:19.727+0000 7ff323c00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff31c10be40 0x7ff31c114c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:19.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:19.727+0000 7ff323c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff31c10c760 0x7ff31c110d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:19.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:19.727+0000 7ff323c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff31c1153f0 0x7ff31c1112a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:19.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:19.727+0000 7ff323c00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff31c119bc0 con 0x7ff31c10be40 2024-05-03T18:41:19.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:19.727+0000 7ff323c00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff31c119a40 con 0x7ff31c10c760 2024-05-03T18:41:19.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:19.727+0000 7ff323c00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff31c119d40 con 0x7ff31c1153f0 2024-05-03T18:41:19.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:19.728+0000 7ff31be00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff31c10c760 0x7ff31c110d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:19.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:19.728+0000 7ff31be00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff31c10c760 0x7ff31c110d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:42538/0 (socket says 172.21.15.33:42538) 2024-05-03T18:41:19.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:19.728+0000 7ff31be00700 1 -- 172.21.15.33:0/3746449629 learned_addr learned my addr 172.21.15.33:0/3746449629 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:41:19.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:19.728+0000 7ff321200700 1 --2- 172.21.15.33:0/3746449629 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff31c10be40 0x7ff31c114c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:19.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:19.729+0000 7ff321c00700 1 --2- 172.21.15.33:0/3746449629 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff31c1153f0 0x7ff31c1112a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:19.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:19.729+0000 7ff31be00700 1 -- 172.21.15.33:0/3746449629 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff31c1153f0 msgr2=0x7ff31c1112a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:41:19.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:19.729+0000 7ff31be00700 1 --2- 172.21.15.33:0/3746449629 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff31c1153f0 0x7ff31c1112a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:19.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:19.729+0000 7ff31be00700 1 -- 172.21.15.33:0/3746449629 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff31c10be40 msgr2=0x7ff31c114c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:41:19.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:19.729+0000 7ff31be00700 1 --2- 172.21.15.33:0/3746449629 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff31c10be40 0x7ff31c114c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:19.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:19.729+0000 7ff31be00700 1 -- 172.21.15.33:0/3746449629 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff31c111ba0 con 0x7ff31c10c760 2024-05-03T18:41:19.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:19.730+0000 7ff31be00700 1 --2- 172.21.15.33:0/3746449629 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff31c10c760 0x7ff31c110d60 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff304007f10 tx=0x7ff30400d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:41:19.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:19.730+0000 7ff319600700 1 -- 172.21.15.33:0/3746449629 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff30400ddd0 con 0x7ff31c10c760 2024-05-03T18:41:19.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:19.734+0000 7ff319600700 1 -- 172.21.15.33:0/3746449629 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff304004020 con 0x7ff31c10c760 2024-05-03T18:41:19.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:19.734+0000 7ff323c00700 1 -- 172.21.15.33:0/3746449629 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff31c1b4ef0 con 0x7ff31c10c760 2024-05-03T18:41:19.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:19.734+0000 7ff323c00700 1 -- 172.21.15.33:0/3746449629 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff31c1b5350 con 0x7ff31c10c760 2024-05-03T18:41:19.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:19.734+0000 7ff319600700 1 -- 172.21.15.33:0/3746449629 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff304015640 con 0x7ff31c10c760 2024-05-03T18:41:19.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:19.737+0000 7ff323c00700 1 -- 172.21.15.33:0/3746449629 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff31c06fba0 con 0x7ff31c10c760 2024-05-03T18:41:19.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:19.741+0000 7ff319600700 1 -- 172.21.15.33:0/3746449629 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7ff304003a00 con 0x7ff31c10c760 2024-05-03T18:41:19.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:19.744+0000 7ff319600700 1 --2- 172.21.15.33:0/3746449629 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7ff3000b2b20 0x7ff3000b4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:19.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:19.744+0000 7ff319600700 1 -- 172.21.15.33:0/3746449629 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9518+0+0 (secure 0 0 0) 0x7ff3040e25a0 con 0x7ff31c10c760 2024-05-03T18:41:19.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:19.745+0000 7ff321200700 1 --2- 172.21.15.33:0/3746449629 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7ff3000b2b20 0x7ff3000b4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:19.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:19.746+0000 7ff321200700 1 --2- 172.21.15.33:0/3746449629 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7ff3000b2b20 0x7ff3000b4fe0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff31400ba60 tx=0x7ff314005d50 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:41:19.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:19.759+0000 7ff319600700 1 -- 172.21.15.33:0/3746449629 <== mon.2 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7ff304051d70 con 0x7ff31c10c760 2024-05-03T18:41:19.761 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:41:19 smithi033 ceph-mon[28330]: pgmap v442: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 4.9 MiB/s wr, 503 op/s 2024-05-03T18:41:19.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:41:19 smithi099 ceph-mon[31658]: pgmap v442: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 4.9 MiB/s wr, 503 op/s 2024-05-03T18:41:19.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:41:19 smithi150 ceph-mon[32492]: pgmap v442: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 4.9 MiB/s wr, 503 op/s 2024-05-03T18:41:20.110 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 605.0 s, 583.2 tps, lat 171.819 ms stddev 50.188 2024-05-03T18:41:20.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:20.290+0000 7ff323c00700 1 -- 172.21.15.33:0/3746449629 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff31c112270 con 0x7ff31c10c760 2024-05-03T18:41:20.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:20.294+0000 7ff319600700 1 -- 172.21.15.33:0/3746449629 <== mon.2 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11801 (secure 0 0 0) 0x7ff31c112270 con 0x7ff31c10c760 2024-05-03T18:41:20.295 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:41:20.296 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1327393503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1327393503},{"type":"v1","addr":"172.21.15.33:6835","nonce":1327393503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/2461472948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":2461472948},{"type":"v1","addr":"172.21.15.33:6837","nonce":2461472948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14559,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/442269281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":442269281},{"type":"v1","addr":"172.21.15.33:6839","nonce":442269281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14571,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/645571756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":645571756},{"type":"v1","addr":"172.21.15.33:6841","nonce":645571756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/203598909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":203598909},{"type":"v1","addr":"172.21.15.150:6833","nonce":203598909}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/4029842493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":4029842493},{"type":"v1","addr":"172.21.15.150:6835","nonce":4029842493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3463664423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3463664423},{"type":"v1","addr":"172.21.15.150:6837","nonce":3463664423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24481,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6839/4051648018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":4051648018},{"type":"v1","addr":"172.21.15.150:6839","nonce":4051648018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34418,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1865105479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1865105479},{"type":"v1","addr":"172.21.15.99:6833","nonce":1865105479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34445,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/1206393914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1206393914},{"type":"v1","addr":"172.21.15.99:6835","nonce":1206393914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-03T18:22:28.837091+0000","modified":"2024-05-03T18:23:49.758398+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34508},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34472":{"gid":34472,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6837/3360970342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3360970342},{"type":"v1","addr":"172.21.15.99:6837","nonce":3360970342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34508":{"gid":34508,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/919907308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":919907308},{"type":"v1","addr":"172.21.15.99:6839","nonce":919907308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34508,"qdb_cluster":[34508]},"id":1}]} 2024-05-03T18:41:20.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:20.298+0000 7ff323c00700 1 -- 172.21.15.33:0/3746449629 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7ff3000b2b20 msgr2=0x7ff3000b4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:41:20.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:20.298+0000 7ff323c00700 1 --2- 172.21.15.33:0/3746449629 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7ff3000b2b20 0x7ff3000b4fe0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff31400ba60 tx=0x7ff314005d50 comp rx=0 tx=0).stop 2024-05-03T18:41:20.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:20.298+0000 7ff323c00700 1 -- 172.21.15.33:0/3746449629 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff31c10c760 msgr2=0x7ff31c110d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:41:20.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:20.298+0000 7ff323c00700 1 --2- 172.21.15.33:0/3746449629 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff31c10c760 0x7ff31c110d60 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff304007f10 tx=0x7ff30400d3b0 comp rx=0 tx=0).stop 2024-05-03T18:41:20.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:20.299+0000 7ff323c00700 1 -- 172.21.15.33:0/3746449629 shutdown_connections 2024-05-03T18:41:20.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:20.299+0000 7ff323c00700 1 --2- 172.21.15.33:0/3746449629 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff31c1153f0 0x7ff31c1112a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:20.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:20.299+0000 7ff323c00700 1 --2- 172.21.15.33:0/3746449629 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff31c10be40 0x7ff31c114c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:20.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:20.299+0000 7ff323c00700 1 --2- 172.21.15.33:0/3746449629 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7ff3000b2b20 0x7ff3000b4fe0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:20.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:20.299+0000 7ff323c00700 1 --2- 172.21.15.33:0/3746449629 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff31c10c760 0x7ff31c110d60 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:20.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:20.299+0000 7ff323c00700 1 -- 172.21.15.33:0/3746449629 >> 172.21.15.33:0/3746449629 conn(0x7ff31c06e610 msgr2=0x7ff31c0736a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:41:20.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:20.299+0000 7ff323c00700 1 -- 172.21.15.33:0/3746449629 shutdown_connections 2024-05-03T18:41:20.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:20.300+0000 7ff323c00700 1 -- 172.21.15.33:0/3746449629 wait complete. 2024-05-03T18:41:20.302 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 21 2024-05-03T18:41:20.735 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:41:20 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:41:20.735 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:41:20 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:41:20.735 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:41:20 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:41:20.735 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:41:20 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/3746449629' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:41:20.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:41:20 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:41:20.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:41:20 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:41:20.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:41:20 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:41:20.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:41:20 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/3746449629' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:41:20.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:41:20 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:41:20.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:41:20 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:41:20.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:41:20 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:41:20.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:41:20 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/3746449629' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:41:21.837 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-03T18:41:22.114 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:41:21 smithi033 ceph-mon[28330]: pgmap v443: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 5.5 MiB/s wr, 557 op/s 2024-05-03T18:41:22.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:41:21 smithi099 ceph-mon[31658]: pgmap v443: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 5.5 MiB/s wr, 557 op/s 2024-05-03T18:41:22.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:41:21 smithi150 ceph-mon[32492]: pgmap v443: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 5.5 MiB/s wr, 557 op/s 2024-05-03T18:41:22.759 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:41:24.214 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:41:23 smithi033 ceph-mon[28330]: pgmap v444: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 5.5 MiB/s wr, 557 op/s 2024-05-03T18:41:24.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:41:23 smithi099 ceph-mon[31658]: pgmap v444: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 5.5 MiB/s wr, 557 op/s 2024-05-03T18:41:24.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:41:23 smithi150 ceph-mon[32492]: pgmap v444: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 5.5 MiB/s wr, 557 op/s 2024-05-03T18:41:25.111 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 610.0 s, 487.4 tps, lat 205.709 ms stddev 144.892 2024-05-03T18:41:26.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:41:25 smithi033 ceph-mon[28330]: pgmap v445: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 7.0 MiB/s wr, 689 op/s 2024-05-03T18:41:26.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:41:25 smithi099 ceph-mon[31658]: pgmap v445: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 7.0 MiB/s wr, 689 op/s 2024-05-03T18:41:26.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:41:25 smithi150 ceph-mon[32492]: pgmap v445: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 7.0 MiB/s wr, 689 op/s 2024-05-03T18:41:28.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.216+0000 7fc83ae00700 1 -- 172.21.15.33:0/2258702502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc834109500 msgr2=0x7fc8341098e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:41:28.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.216+0000 7fc83ae00700 1 --2- 172.21.15.33:0/2258702502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc834109500 0x7fc8341098e0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fc8240098f0 tx=0x7fc824009c00 comp rx=0 tx=0).stop 2024-05-03T18:41:28.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.217+0000 7fc83ae00700 1 -- 172.21.15.33:0/2258702502 shutdown_connections 2024-05-03T18:41:28.219 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:41:27 smithi033 ceph-mon[28330]: pgmap v446: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 5.2 MiB/s wr, 513 op/s 2024-05-03T18:41:28.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.217+0000 7fc83ae00700 1 --2- 172.21.15.33:0/2258702502 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc83406c3d0 0x7fc834110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:28.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.217+0000 7fc83ae00700 1 --2- 172.21.15.33:0/2258702502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc834109500 0x7fc8341098e0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:28.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.217+0000 7fc83ae00700 1 --2- 172.21.15.33:0/2258702502 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc83406ba10 0x7fc83406be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:28.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.217+0000 7fc83ae00700 1 -- 172.21.15.33:0/2258702502 >> 172.21.15.33:0/2258702502 conn(0x7fc834079310 msgr2=0x7fc834079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:41:28.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.218+0000 7fc83ae00700 1 -- 172.21.15.33:0/2258702502 shutdown_connections 2024-05-03T18:41:28.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.218+0000 7fc83ae00700 1 -- 172.21.15.33:0/2258702502 wait complete. 2024-05-03T18:41:28.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.220+0000 7fc83ae00700 1 Processor -- start 2024-05-03T18:41:28.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.220+0000 7fc83ae00700 1 -- start start 2024-05-03T18:41:28.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.221+0000 7fc83ae00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc83406ba10 0x7fc83419bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:28.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.221+0000 7fc83ae00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc83406c3d0 0x7fc83419c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:28.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.221+0000 7fc83ae00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc834109500 0x7fc8341a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:28.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.221+0000 7fc83ae00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc834115840 con 0x7fc83406ba10 2024-05-03T18:41:28.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.221+0000 7fc83ae00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc8341156c0 con 0x7fc83406c3d0 2024-05-03T18:41:28.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.221+0000 7fc83ae00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc8341159c0 con 0x7fc834109500 2024-05-03T18:41:28.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.221+0000 7fc82be00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc83406c3d0 0x7fc83419c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:28.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.221+0000 7fc838e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc834109500 0x7fc8341a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:28.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.221+0000 7fc82be00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc83406c3d0 0x7fc83419c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:56854/0 (socket says 172.21.15.33:56854) 2024-05-03T18:41:28.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.221+0000 7fc82be00700 1 -- 172.21.15.33:0/3172263770 learned_addr learned my addr 172.21.15.33:0/3172263770 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:41:28.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.222+0000 7fc82be00700 1 -- 172.21.15.33:0/3172263770 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc834109500 msgr2=0x7fc8341a0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:41:28.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.222+0000 7fc82be00700 1 --2- 172.21.15.33:0/3172263770 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc834109500 0x7fc8341a0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:28.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.222+0000 7fc82be00700 1 -- 172.21.15.33:0/3172263770 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc83406ba10 msgr2=0x7fc83419bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:41:28.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.222+0000 7fc82be00700 1 --2- 172.21.15.33:0/3172263770 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc83406ba10 0x7fc83419bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:28.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.222+0000 7fc82be00700 1 -- 172.21.15.33:0/3172263770 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8341a0d80 con 0x7fc83406c3d0 2024-05-03T18:41:28.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.223+0000 7fc82be00700 1 --2- 172.21.15.33:0/3172263770 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc83406c3d0 0x7fc83419c2b0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc81c00c0c0 tx=0x7fc81c00c3d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:41:28.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.224+0000 7fc831600700 1 -- 172.21.15.33:0/3172263770 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc81c00cdd0 con 0x7fc83406c3d0 2024-05-03T18:41:28.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.224+0000 7fc831600700 1 -- 172.21.15.33:0/3172263770 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc81c0075c0 con 0x7fc83406c3d0 2024-05-03T18:41:28.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.224+0000 7fc831600700 1 -- 172.21.15.33:0/3172263770 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc81c0146e0 con 0x7fc83406c3d0 2024-05-03T18:41:28.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.224+0000 7fc83ae00700 1 -- 172.21.15.33:0/3172263770 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8341a1010 con 0x7fc83406c3d0 2024-05-03T18:41:28.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.225+0000 7fc83ae00700 1 -- 172.21.15.33:0/3172263770 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc834110b60 con 0x7fc83406c3d0 2024-05-03T18:41:28.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.233+0000 7fc831600700 1 -- 172.21.15.33:0/3172263770 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fc81c01b020 con 0x7fc83406c3d0 2024-05-03T18:41:28.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.235+0000 7fc831600700 1 --2- 172.21.15.33:0/3172263770 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fc8180b2a70 0x7fc8180b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:28.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.236+0000 7fc831600700 1 -- 172.21.15.33:0/3172263770 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9518+0+0 (secure 0 0 0) 0x7fc81c0e36c0 con 0x7fc83406c3d0 2024-05-03T18:41:28.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.237+0000 7fc833e00700 1 --2- 172.21.15.33:0/3172263770 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fc8180b2a70 0x7fc8180b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:28.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.254+0000 7fc833e00700 1 --2- 172.21.15.33:0/3172263770 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fc8180b2a70 0x7fc8180b4f30 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc824009680 tx=0x7fc824009f50 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:41:28.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.259+0000 7fc84585bb80 1 Processor -- start 2024-05-03T18:41:28.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.259+0000 7fc84585bb80 1 -- start start 2024-05-03T18:41:28.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.260+0000 7fc84585bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5597abfa3970 0x5597abfa3d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:28.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.260+0000 7fc84585bb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5597abfa4290 0x5597abf9dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:28.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.260+0000 7fc84585bb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5597abf9e5e0 0x5597abf9ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:28.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.260+0000 7fc84585bb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5597abedfa20 con 0x5597abfa4290 2024-05-03T18:41:28.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.260+0000 7fc84585bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5597abe7d760 con 0x5597abfa3970 2024-05-03T18:41:28.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.260+0000 7fc84585bb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5597abedb9e0 con 0x5597abf9e5e0 2024-05-03T18:41:28.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.260+0000 7fc833e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5597abfa3970 0x5597abfa3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:28.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.260+0000 7fc833e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5597abfa3970 0x5597abfa3d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:56858/0 (socket says 172.21.15.33:56858) 2024-05-03T18:41:28.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.260+0000 7fc833e00700 1 -- 172.21.15.33:0/3186570302 learned_addr learned my addr 172.21.15.33:0/3186570302 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:41:28.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.261+0000 7fc838e00700 1 --2- 172.21.15.33:0/3186570302 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5597abf9e5e0 0x5597abf9ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:28.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.262+0000 7fc833e00700 1 -- 172.21.15.33:0/3186570302 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5597abf9e5e0 msgr2=0x5597abf9ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:41:28.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.262+0000 7fc833e00700 1 --2- 172.21.15.33:0/3186570302 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5597abf9e5e0 0x5597abf9ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:28.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.262+0000 7fc833e00700 1 -- 172.21.15.33:0/3186570302 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5597abfa4290 msgr2=0x5597abf9dfc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-03T18:41:28.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.262+0000 7fc833e00700 1 --2- 172.21.15.33:0/3186570302 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5597abfa4290 0x5597abf9dfc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:28.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.262+0000 7fc833e00700 1 -- 172.21.15.33:0/3186570302 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597abe2c8d0 con 0x5597abfa3970 2024-05-03T18:41:28.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.263+0000 7fc833e00700 1 --2- 172.21.15.33:0/3186570302 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5597abfa3970 0x5597abfa3d50 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc82400fc80 tx=0x7fc82401c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e2e894289b156fb server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:41:28.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.264+0000 7fc829600700 1 -- 172.21.15.33:0/3186570302 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc82400f560 con 0x5597abfa3970 2024-05-03T18:41:28.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.264+0000 7fc829600700 1 -- 172.21.15.33:0/3186570302 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc8240054d0 con 0x5597abfa3970 2024-05-03T18:41:28.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.266+0000 7fc829600700 1 -- 172.21.15.33:0/3186570302 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc82400f560 con 0x5597abfa3970 2024-05-03T18:41:28.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.266+0000 7fc84585bb80 1 -- 172.21.15.33:0/3186570302 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5597abfa3970 msgr2=0x5597abfa3d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:41:28.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.266+0000 7fc84585bb80 1 --2- 172.21.15.33:0/3186570302 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5597abfa3970 0x5597abfa3d50 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc82400fc80 tx=0x7fc82401c000 comp rx=0 tx=0).stop 2024-05-03T18:41:28.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.268+0000 7fc84585bb80 1 -- 172.21.15.33:0/3186570302 shutdown_connections 2024-05-03T18:41:28.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.268+0000 7fc84585bb80 1 --2- 172.21.15.33:0/3186570302 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5597abf9e5e0 0x5597abf9ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:28.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.268+0000 7fc84585bb80 1 --2- 172.21.15.33:0/3186570302 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5597abfa3970 0x5597abfa3d50 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:28.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.268+0000 7fc84585bb80 1 --2- 172.21.15.33:0/3186570302 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5597abfa4290 0x5597abf9dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:28.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.268+0000 7fc84585bb80 1 -- 172.21.15.33:0/3186570302 >> 172.21.15.33:0/3186570302 conn(0x5597abebbb60 msgr2=0x5597abf877f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:41:28.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.268+0000 7fc84585bb80 1 -- 172.21.15.33:0/3186570302 shutdown_connections 2024-05-03T18:41:28.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.268+0000 7fc84585bb80 1 -- 172.21.15.33:0/3186570302 wait complete. 2024-05-03T18:41:28.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.269+0000 7fc84585bb80 1 -- start start 2024-05-03T18:41:28.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.269+0000 7fc84585bb80 1 Processor -- start 2024-05-03T18:41:28.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.273+0000 7fc84585bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5597abf9e5e0 0x5597abfa7830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:28.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.273+0000 7fc84585bb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5597abfa3970 0x5597abfa7d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:28.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.273+0000 7fc84585bb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5597abfa4290 0x5597abfa82b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:28.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.273+0000 7fc84585bb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5597abf8d1e0 con 0x5597abfa4290 2024-05-03T18:41:28.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.273+0000 7fc84585bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5597abe45b20 con 0x5597abf9e5e0 2024-05-03T18:41:28.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.273+0000 7fc84585bb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5597abf8d360 con 0x5597abfa3970 2024-05-03T18:41:28.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.273+0000 7fc84585bb80 10 client.0 _ll_register_callbacks cb 0x5597abe86c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:41:28.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.274+0000 7fc838e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5597abfa4290 0x5597abfa82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:28.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.274+0000 7fc838e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5597abfa4290 0x5597abfa82b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33120/0 (socket says 172.21.15.33:33120) 2024-05-03T18:41:28.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.274+0000 7fc838e00700 1 -- 172.21.15.33:0/1625506197 learned_addr learned my addr 172.21.15.33:0/1625506197 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:41:28.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.275+0000 7fc82be00700 1 --2- 172.21.15.33:0/1625506197 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5597abfa3970 0x5597abfa7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:28.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.276+0000 7fc82be00700 1 -- 172.21.15.33:0/1625506197 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5597abf9e5e0 msgr2=0x5597abfa7830 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-03T18:41:28.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.276+0000 7fc82be00700 1 --2- 172.21.15.33:0/1625506197 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5597abf9e5e0 0x5597abfa7830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:28.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.276+0000 7fc82be00700 1 -- 172.21.15.33:0/1625506197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5597abfa4290 msgr2=0x5597abfa82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:41:28.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.276+0000 7fc82be00700 1 --2- 172.21.15.33:0/1625506197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5597abfa4290 0x5597abfa82b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:28.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.276+0000 7fc82be00700 1 -- 172.21.15.33:0/1625506197 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597abea5420 con 0x5597abfa3970 2024-05-03T18:41:28.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.276+0000 7fc822000700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-03T18:41:28.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.276+0000 7fc82be00700 1 --2- 172.21.15.33:0/1625506197 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5597abfa3970 0x5597abfa7d70 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc81c0e0a30 tx=0x7fc81c0a9e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4406876d016819b server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:41:28.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.277+0000 7fc822000700 1 -- 172.21.15.33:0/1625506197 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc81c0ae970 con 0x5597abfa3970 2024-05-03T18:41:28.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.277+0000 7fc84585bb80 1 -- 172.21.15.33:0/1625506197 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8240054d0 con 0x5597abfa3970 2024-05-03T18:41:28.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.278+0000 7fc822000700 1 -- 172.21.15.33:0/1625506197 <== mon.1 v2:172.21.15.150:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9518+0+0 (secure 0 0 0) 0x7fc81c0a6bb0 con 0x5597abfa3970 2024-05-03T18:41:28.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.278+0000 7fc822000700 1 -- 172.21.15.33:0/1625506197 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fc81c0ae970 con 0x5597abfa3970 2024-05-03T18:41:28.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.279+0000 7fc84585bb80 10 client.26491 fetch_fsmap learned FSMap version 21 2024-05-03T18:41:28.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.279+0000 7fc84585bb80 1 -- 172.21.15.33:0/1625506197 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5597abf97d80 con 0x5597abfa3970 2024-05-03T18:41:28.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.280+0000 7fc822000700 1 -- 172.21.15.33:0/1625506197 <== mon.1 v2:172.21.15.150:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc81c0e2070 con 0x5597abfa3970 2024-05-03T18:41:28.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.282+0000 7fc822000700 1 -- 172.21.15.33:0/1625506197 <== mon.1 v2:172.21.15.150:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fc81c0a4ce0 con 0x5597abfa3970 2024-05-03T18:41:28.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.282+0000 7fc84585bb80 10 client.26491 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:41:28.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.282+0000 7fc84585bb80 10 client.26491 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:41:28.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.282+0000 7fc84585bb80 20 client.26491 populate_metadata read hostname 'smithi033' 2024-05-03T18:41:28.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.282+0000 7fc84585bb80 1 --2- 172.21.15.33:0/1625506197 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5597abf9a750 0x5597abf9cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:28.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.282+0000 7fc84585bb80 4 client.26491 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:41:28.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.282+0000 7fc84585bb80 1 -- 172.21.15.33:0/1625506197 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fc8240054d0 con 0x5597abf9a750 2024-05-03T18:41:28.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.283+0000 7fc833e00700 1 --2- 172.21.15.33:0/1625506197 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5597abf9a750 0x5597abf9cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:28.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.288+0000 7fc822000700 10 client.26491 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:41:28.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.288+0000 7fc833e00700 1 --2- 172.21.15.33:0/1625506197 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5597abf9a750 0x5597abf9cc10 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dfba6a6d105c4222 server_cookie=bee3e04249a3abb8 in_seq=0 out_seq=0 2024-05-03T18:41:28.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.289+0000 7fc822000700 1 -- 172.21.15.33:0/1625506197 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7fc8240054d0 con 0x5597abf9a750 2024-05-03T18:41:28.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.289+0000 7fc822000700 10 client.26491 handle_command_reply: tid=0 2024-05-03T18:41:28.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.289+0000 7fc84585bb80 1 client.26491 shutdown 2024-05-03T18:41:28.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.291+0000 7fc84585bb80 1 -- 172.21.15.33:0/1625506197 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5597abfa3970 msgr2=0x5597abfa7d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:41:28.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.291+0000 7fc84585bb80 1 --2- 172.21.15.33:0/1625506197 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5597abfa3970 0x5597abfa7d70 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc81c0e0a30 tx=0x7fc81c0a9e50 comp rx=0 tx=0).stop 2024-05-03T18:41:28.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.291+0000 7fc84585bb80 1 -- 172.21.15.33:0/1625506197 shutdown_connections 2024-05-03T18:41:28.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.291+0000 7fc84585bb80 1 --2- 172.21.15.33:0/1625506197 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5597abf9a750 0x5597abf9cc10 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:28.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.291+0000 7fc822000700 0 client.26491 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:41:28.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.291+0000 7fc84585bb80 1 --2- 172.21.15.33:0/1625506197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5597abfa4290 0x5597abfa82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:28.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.291+0000 7fc84585bb80 1 --2- 172.21.15.33:0/1625506197 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5597abf9e5e0 0x5597abfa7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:28.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.291+0000 7fc84585bb80 1 --2- 172.21.15.33:0/1625506197 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5597abfa3970 0x5597abfa7d70 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:28.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.291+0000 7fc84585bb80 1 -- 172.21.15.33:0/1625506197 >> 172.21.15.33:0/1625506197 conn(0x5597abebbb60 msgr2=0x5597abfa2d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:41:28.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:41:27 smithi099 ceph-mon[31658]: pgmap v446: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 5.2 MiB/s wr, 513 op/s 2024-05-03T18:41:28.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.292+0000 7fc84585bb80 1 -- 172.21.15.33:0/1625506197 shutdown_connections 2024-05-03T18:41:28.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.295+0000 7fc84585bb80 1 -- 172.21.15.33:0/1625506197 wait complete. 2024-05-03T18:41:28.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.295+0000 7fc84585bb80 20 client.26491 trim_cache size 0 max 16384 2024-05-03T18:41:28.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.327+0000 7fc84585bb80 1 Processor -- start 2024-05-03T18:41:28.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.327+0000 7fc84585bb80 1 -- start start 2024-05-03T18:41:28.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.328+0000 7fc84585bb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5597abf9a750 0x5597abf86ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:28.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.328+0000 7fc84585bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5597abf9e5e0 0x5597abf9dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:28.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.328+0000 7fc84585bb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5597abfa3970 0x5597abfa6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:28.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.328+0000 7fc84585bb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc83406fc20 con 0x5597abfa3970 2024-05-03T18:41:28.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.328+0000 7fc84585bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc834072500 con 0x5597abf9e5e0 2024-05-03T18:41:28.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.328+0000 7fc84585bb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc8340702f0 con 0x5597abf9a750 2024-05-03T18:41:28.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.329+0000 7fc82be00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5597abf9e5e0 0x5597abf9dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:28.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.329+0000 7fc82be00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5597abf9e5e0 0x5597abf9dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:56872/0 (socket says 172.21.15.33:56872) 2024-05-03T18:41:28.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.329+0000 7fc82be00700 1 -- 172.21.15.33:0/4061619509 learned_addr learned my addr 172.21.15.33:0/4061619509 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:41:28.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.329+0000 7fc833e00700 1 --2- 172.21.15.33:0/4061619509 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5597abf9a750 0x5597abf86ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:28.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.330+0000 7fc833e00700 1 -- 172.21.15.33:0/4061619509 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5597abf9e5e0 msgr2=0x5597abf9dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:41:28.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.330+0000 7fc833e00700 1 --2- 172.21.15.33:0/4061619509 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5597abf9e5e0 0x5597abf9dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:28.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.330+0000 7fc833e00700 1 -- 172.21.15.33:0/4061619509 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5597abfa3970 msgr2=0x5597abfa6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:41:28.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.330+0000 7fc833e00700 1 --2- 172.21.15.33:0/4061619509 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5597abfa3970 0x5597abfa6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:28.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.330+0000 7fc833e00700 1 -- 172.21.15.33:0/4061619509 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597abf88550 con 0x5597abf9a750 2024-05-03T18:41:28.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.330+0000 7fc833e00700 1 --2- 172.21.15.33:0/4061619509 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5597abf9a750 0x5597abf86ee0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x5597abf9d480 tx=0x7fc82401c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bdf46cdbfc5edba1 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:41:28.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.331+0000 7fc828c00700 1 -- 172.21.15.33:0/4061619509 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc82405c070 con 0x5597abf9a750 2024-05-03T18:41:28.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.331+0000 7fc828c00700 1 -- 172.21.15.33:0/4061619509 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc8240054d0 con 0x5597abf9a750 2024-05-03T18:41:28.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.331+0000 7fc828c00700 1 -- 172.21.15.33:0/4061619509 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc82400f860 con 0x5597abf9a750 2024-05-03T18:41:28.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.332+0000 7fc84585bb80 1 -- 172.21.15.33:0/4061619509 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5597abf9a750 msgr2=0x5597abf86ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:41:28.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.332+0000 7fc84585bb80 1 --2- 172.21.15.33:0/4061619509 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5597abf9a750 0x5597abf86ee0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x5597abf9d480 tx=0x7fc82401c000 comp rx=0 tx=0).stop 2024-05-03T18:41:28.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.333+0000 7fc84585bb80 1 -- 172.21.15.33:0/4061619509 shutdown_connections 2024-05-03T18:41:28.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.333+0000 7fc84585bb80 1 --2- 172.21.15.33:0/4061619509 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5597abfa3970 0x5597abfa6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:28.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.333+0000 7fc84585bb80 1 --2- 172.21.15.33:0/4061619509 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5597abf9a750 0x5597abf86ee0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:28.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.333+0000 7fc84585bb80 1 --2- 172.21.15.33:0/4061619509 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5597abf9e5e0 0x5597abf9dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:28.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.333+0000 7fc84585bb80 1 -- 172.21.15.33:0/4061619509 >> 172.21.15.33:0/4061619509 conn(0x5597abebbb60 msgr2=0x5597abf9d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:41:28.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.333+0000 7fc84585bb80 1 -- 172.21.15.33:0/4061619509 shutdown_connections 2024-05-03T18:41:28.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.333+0000 7fc84585bb80 1 -- 172.21.15.33:0/4061619509 wait complete. 2024-05-03T18:41:28.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.334+0000 7fc84585bb80 1 -- start start 2024-05-03T18:41:28.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.334+0000 7fc84585bb80 1 Processor -- start 2024-05-03T18:41:28.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.335+0000 7fc84585bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5597abf9a750 0x5597abf92640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:28.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.335+0000 7fc84585bb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5597abf9e5e0 0x5597abf92b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:28.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.335+0000 7fc84585bb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5597abfa3970 0x5597abf956b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:28.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.335+0000 7fc84585bb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc8340709c0 con 0x5597abfa3970 2024-05-03T18:41:28.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.335+0000 7fc84585bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc834071e30 con 0x5597abf9a750 2024-05-03T18:41:28.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.335+0000 7fc84585bb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc834071090 con 0x5597abf9e5e0 2024-05-03T18:41:28.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.335+0000 7fc84585bb80 10 client.0 _ll_register_callbacks cb 0x5597abe86c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:41:28.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.336+0000 7fc833e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5597abf9a750 0x5597abf92640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:28.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.336+0000 7fc833e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5597abf9a750 0x5597abf92640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:56880/0 (socket says 172.21.15.33:56880) 2024-05-03T18:41:28.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.336+0000 7fc833e00700 1 -- 172.21.15.33:0/3049253088 learned_addr learned my addr 172.21.15.33:0/3049253088 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:41:28.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.336+0000 7fc838e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5597abfa3970 0x5597abf956b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:28.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.336+0000 7fc833e00700 1 -- 172.21.15.33:0/3049253088 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5597abf9e5e0 msgr2=0x5597abf92b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:41:28.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.336+0000 7fc82be00700 1 --2- 172.21.15.33:0/3049253088 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5597abf9e5e0 0x5597abf92b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:28.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.336+0000 7fc833e00700 1 --2- 172.21.15.33:0/3049253088 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5597abf9e5e0 0x5597abf92b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:28.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.337+0000 7fc833e00700 1 -- 172.21.15.33:0/3049253088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5597abfa3970 msgr2=0x5597abf956b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:41:28.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.337+0000 7fc833e00700 1 --2- 172.21.15.33:0/3049253088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5597abfa3970 0x5597abf956b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:28.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.337+0000 7fc833e00700 1 -- 172.21.15.33:0/3049253088 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597abf95cc0 con 0x5597abf9a750 2024-05-03T18:41:28.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.337+0000 7fc833e00700 1 --2- 172.21.15.33:0/3049253088 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5597abf9a750 0x5597abf92640 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc82400f9b0 tx=0x7fc82400fa90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c28372ecfdd1d785 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:41:28.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.337+0000 7fc822000700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-03T18:41:28.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.338+0000 7fc822000700 1 -- 172.21.15.33:0/3049253088 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc82405c020 con 0x5597abf9a750 2024-05-03T18:41:28.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.338+0000 7fc822000700 1 -- 172.21.15.33:0/3049253088 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9518+0+0 (secure 0 0 0) 0x7fc824055040 con 0x5597abf9a750 2024-05-03T18:41:28.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.338+0000 7fc84585bb80 1 -- 172.21.15.33:0/3049253088 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8240054d0 con 0x5597abf9a750 2024-05-03T18:41:28.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.339+0000 7fc822000700 1 -- 172.21.15.33:0/3049253088 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fc824003a50 con 0x5597abf9a750 2024-05-03T18:41:28.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.339+0000 7fc84585bb80 10 client.36419 fetch_fsmap learned FSMap version 21 2024-05-03T18:41:28.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.339+0000 7fc84585bb80 1 -- 172.21.15.33:0/3049253088 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5597ac021ff0 con 0x5597abf9a750 2024-05-03T18:41:28.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.340+0000 7fc822000700 1 -- 172.21.15.33:0/3049253088 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc8240054d0 con 0x5597abf9a750 2024-05-03T18:41:28.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.341+0000 7fc822000700 1 -- 172.21.15.33:0/3049253088 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fc82400b560 con 0x5597abf9a750 2024-05-03T18:41:28.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.341+0000 7fc84585bb80 10 client.36419 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:41:28.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.341+0000 7fc84585bb80 10 client.36419 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:41:28.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.341+0000 7fc84585bb80 20 client.36419 populate_metadata read hostname 'smithi033' 2024-05-03T18:41:28.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.341+0000 7fc84585bb80 1 --2- 172.21.15.33:0/3049253088 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5597abfa4290 0x5597ac024280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:28.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.341+0000 7fc84585bb80 4 client.36419 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-03T18:41:28.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.341+0000 7fc84585bb80 1 -- 172.21.15.33:0/3049253088 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5597abf5a1d0 con 0x5597abfa4290 2024-05-03T18:41:28.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.342+0000 7fc82be00700 1 --2- 172.21.15.33:0/3049253088 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5597abfa4290 0x5597ac024280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:28.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.343+0000 7fc82be00700 1 --2- 172.21.15.33:0/3049253088 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5597abfa4290 0x5597ac024280 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a971908617ce279 server_cookie=7c7acd8552ac8105 in_seq=0 out_seq=0 2024-05-03T18:41:28.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.343+0000 7fc822000700 10 client.36419 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:41:28.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.343+0000 7fc822000700 1 -- 172.21.15.33:0/3049253088 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5597abf5a1d0 con 0x5597abfa4290 2024-05-03T18:41:28.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.343+0000 7fc822000700 10 client.36419 handle_command_reply: tid=0 2024-05-03T18:41:28.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.343+0000 7fc84585bb80 1 client.36419 shutdown 2024-05-03T18:41:28.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.347+0000 7fc84585bb80 1 -- 172.21.15.33:0/3049253088 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5597abf9a750 msgr2=0x5597abf92640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:41:28.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.347+0000 7fc84585bb80 1 --2- 172.21.15.33:0/3049253088 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5597abf9a750 0x5597abf92640 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc82400f9b0 tx=0x7fc82400fa90 comp rx=0 tx=0).stop 2024-05-03T18:41:28.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.347+0000 7fc84585bb80 1 -- 172.21.15.33:0/3049253088 shutdown_connections 2024-05-03T18:41:28.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.347+0000 7fc84585bb80 1 --2- 172.21.15.33:0/3049253088 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5597abfa4290 0x5597ac024280 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:28.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.347+0000 7fc822000700 0 client.36419 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:41:28.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.347+0000 7fc84585bb80 1 --2- 172.21.15.33:0/3049253088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5597abfa3970 0x5597abf956b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:28.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.347+0000 7fc84585bb80 1 --2- 172.21.15.33:0/3049253088 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5597abf9a750 0x5597abf92640 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:28.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.347+0000 7fc84585bb80 1 --2- 172.21.15.33:0/3049253088 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5597abf9e5e0 0x5597abf92b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:28.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.347+0000 7fc84585bb80 1 -- 172.21.15.33:0/3049253088 >> 172.21.15.33:0/3049253088 conn(0x5597abebbb60 msgr2=0x5597abf8c830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:41:28.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.349+0000 7fc84585bb80 1 -- 172.21.15.33:0/3049253088 shutdown_connections 2024-05-03T18:41:28.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.349+0000 7fc84585bb80 1 -- 172.21.15.33:0/3049253088 wait complete. 2024-05-03T18:41:28.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.349+0000 7fc84585bb80 20 client.36419 trim_cache size 0 max 16384 2024-05-03T18:41:28.353 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-03T18:41:28.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.352+0000 7fc83ae00700 1 -- 172.21.15.33:0/3172263770 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fc8180b2a70 msgr2=0x7fc8180b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:41:28.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.352+0000 7fc83ae00700 1 --2- 172.21.15.33:0/3172263770 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fc8180b2a70 0x7fc8180b4f30 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc824009680 tx=0x7fc824009f50 comp rx=0 tx=0).stop 2024-05-03T18:41:28.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.352+0000 7fc83ae00700 1 -- 172.21.15.33:0/3172263770 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc83406c3d0 msgr2=0x7fc83419c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:41:28.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.352+0000 7fc83ae00700 1 --2- 172.21.15.33:0/3172263770 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc83406c3d0 0x7fc83419c2b0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc81c00c0c0 tx=0x7fc81c00c3d0 comp rx=0 tx=0).stop 2024-05-03T18:41:28.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.352+0000 7fc83ae00700 1 -- 172.21.15.33:0/3172263770 shutdown_connections 2024-05-03T18:41:28.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.352+0000 7fc83ae00700 1 --2- 172.21.15.33:0/3172263770 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc834109500 0x7fc8341a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:28.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.352+0000 7fc83ae00700 1 --2- 172.21.15.33:0/3172263770 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc83406ba10 0x7fc83419bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:28.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.352+0000 7fc83ae00700 1 --2- 172.21.15.33:0/3172263770 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fc8180b2a70 0x7fc8180b4f30 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:28.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.352+0000 7fc83ae00700 1 --2- 172.21.15.33:0/3172263770 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc83406c3d0 0x7fc83419c2b0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:28.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.352+0000 7fc83ae00700 1 -- 172.21.15.33:0/3172263770 >> 172.21.15.33:0/3172263770 conn(0x7fc834079310 msgr2=0x7fc834101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:41:28.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.352+0000 7fc83ae00700 1 -- 172.21.15.33:0/3172263770 shutdown_connections 2024-05-03T18:41:28.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:28.352+0000 7fc83ae00700 1 -- 172.21.15.33:0/3172263770 wait complete. 2024-05-03T18:41:28.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:41:27 smithi150 ceph-mon[32492]: pgmap v446: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 5.2 MiB/s wr, 513 op/s 2024-05-03T18:41:30.110 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 615.0 s, 446.6 tps, lat 224.962 ms stddev 192.386 2024-05-03T18:41:30.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:41:29 smithi033 ceph-mon[28330]: pgmap v447: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 4.9 MiB/s wr, 481 op/s 2024-05-03T18:41:30.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:41:29 smithi099 ceph-mon[31658]: pgmap v447: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 4.9 MiB/s wr, 481 op/s 2024-05-03T18:41:30.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:41:29 smithi150 ceph-mon[32492]: pgmap v447: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 4.9 MiB/s wr, 481 op/s 2024-05-03T18:41:30.504 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-03T18:41:30.504 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-03T18:41:31.302 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:41:32.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:41:31 smithi033 ceph-mon[28330]: pgmap v448: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 7.4 MiB/s wr, 729 op/s 2024-05-03T18:41:32.297 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:41:31 smithi099 ceph-mon[31658]: pgmap v448: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 7.4 MiB/s wr, 729 op/s 2024-05-03T18:41:32.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:41:31 smithi150 ceph-mon[32492]: pgmap v448: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 7.4 MiB/s wr, 729 op/s 2024-05-03T18:41:34.217 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:41:33 smithi033 ceph-mon[28330]: pgmap v449: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 5.0 MiB/s wr, 488 op/s 2024-05-03T18:41:34.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:41:33 smithi099 ceph-mon[31658]: pgmap v449: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 5.0 MiB/s wr, 488 op/s 2024-05-03T18:41:34.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:41:33 smithi150 ceph-mon[32492]: pgmap v449: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 5.0 MiB/s wr, 488 op/s 2024-05-03T18:41:34.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:34.773+0000 7f2eabe00700 1 -- 172.21.15.33:0/299507138 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ea41056e0 msgr2=0x7f2ea4105b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:41:34.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:34.773+0000 7f2eabe00700 1 --2- 172.21.15.33:0/299507138 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ea41056e0 0x7f2ea4105b60 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2e9400b0a0 tx=0x7f2e9400b3b0 comp rx=0 tx=0).stop 2024-05-03T18:41:34.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:34.773+0000 7f2eabe00700 1 -- 172.21.15.33:0/299507138 shutdown_connections 2024-05-03T18:41:34.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:34.773+0000 7f2eabe00700 1 --2- 172.21.15.33:0/299507138 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ea41060a0 0x7f2ea41125b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:34.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:34.773+0000 7f2eabe00700 1 --2- 172.21.15.33:0/299507138 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ea410b6e0 0x7f2ea410bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:34.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:34.773+0000 7f2eabe00700 1 --2- 172.21.15.33:0/299507138 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ea41056e0 0x7f2ea4105b60 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:34.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:34.773+0000 7f2eabe00700 1 -- 172.21.15.33:0/299507138 >> 172.21.15.33:0/299507138 conn(0x7f2ea41011d0 msgr2=0x7f2ea41035f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:41:34.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:34.774+0000 7f2eabe00700 1 -- 172.21.15.33:0/299507138 shutdown_connections 2024-05-03T18:41:34.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:34.774+0000 7f2eabe00700 1 -- 172.21.15.33:0/299507138 wait complete. 2024-05-03T18:41:34.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:34.775+0000 7f2eabe00700 1 Processor -- start 2024-05-03T18:41:34.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:34.776+0000 7f2eabe00700 1 -- start start 2024-05-03T18:41:34.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:34.776+0000 7f2eabe00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ea41056e0 0x7f2ea41124c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:34.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:34.776+0000 7f2eabe00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ea41060a0 0x7f2ea410f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:34.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:34.776+0000 7f2eabe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ea410b6e0 0x7f2ea410fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:34.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:34.776+0000 7f2eabe00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2ea4115820 con 0x7f2ea41060a0 2024-05-03T18:41:34.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:34.776+0000 7f2eabe00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2ea41156a0 con 0x7f2ea410b6e0 2024-05-03T18:41:34.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:34.776+0000 7f2eabe00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2ea41159a0 con 0x7f2ea41056e0 2024-05-03T18:41:34.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:34.776+0000 7f2ea9400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ea41056e0 0x7f2ea41124c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:34.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:34.777+0000 7f2ea9400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ea41056e0 0x7f2ea41124c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:50792/0 (socket says 172.21.15.33:50792) 2024-05-03T18:41:34.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:34.777+0000 7f2ea9400700 1 -- 172.21.15.33:0/2092873885 learned_addr learned my addr 172.21.15.33:0/2092873885 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:41:34.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:34.777+0000 7f2ea9400700 1 -- 172.21.15.33:0/2092873885 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ea410b6e0 msgr2=0x7f2ea410fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:41:34.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:34.777+0000 7f2ea9400700 1 --2- 172.21.15.33:0/2092873885 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ea410b6e0 0x7f2ea410fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:34.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:34.777+0000 7f2ea9400700 1 -- 172.21.15.33:0/2092873885 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ea41060a0 msgr2=0x7f2ea410f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:41:34.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:34.777+0000 7f2ea9400700 1 --2- 172.21.15.33:0/2092873885 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ea41060a0 0x7f2ea410f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:34.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:34.777+0000 7f2ea9400700 1 -- 172.21.15.33:0/2092873885 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ea41ac500 con 0x7f2ea41056e0 2024-05-03T18:41:34.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:34.778+0000 7f2ea9400700 1 --2- 172.21.15.33:0/2092873885 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ea41056e0 0x7f2ea41124c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c00e9c0 tx=0x7f2e9c00ed80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:41:34.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:34.778+0000 7f2ea2000700 1 -- 172.21.15.33:0/2092873885 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e9c00cbd0 con 0x7f2ea41056e0 2024-05-03T18:41:34.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:34.778+0000 7f2ea2000700 1 -- 172.21.15.33:0/2092873885 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2e9c00cd70 con 0x7f2ea41056e0 2024-05-03T18:41:34.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:34.778+0000 7f2ea2000700 1 -- 172.21.15.33:0/2092873885 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e9c005610 con 0x7f2ea41056e0 2024-05-03T18:41:34.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:34.778+0000 7f2eabe00700 1 -- 172.21.15.33:0/2092873885 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ea41ac6d0 con 0x7f2ea41056e0 2024-05-03T18:41:34.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:34.778+0000 7f2eabe00700 1 -- 172.21.15.33:0/2092873885 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ea41acb90 con 0x7f2ea41056e0 2024-05-03T18:41:34.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:34.787+0000 7f2ea2000700 1 -- 172.21.15.33:0/2092873885 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f2e9c004750 con 0x7f2ea41056e0 2024-05-03T18:41:34.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:34.789+0000 7f2ea2000700 1 --2- 172.21.15.33:0/2092873885 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f2e880b2a70 0x7f2e880b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:34.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:34.790+0000 7f2ea2000700 1 -- 172.21.15.33:0/2092873885 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9518+0+0 (secure 0 0 0) 0x7f2e9c0e2d20 con 0x7f2ea41056e0 2024-05-03T18:41:34.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:34.790+0000 7f2ea8a00700 1 --2- 172.21.15.33:0/2092873885 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f2e880b2a70 0x7f2e880b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:34.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:34.792+0000 7f2ea8a00700 1 --2- 172.21.15.33:0/2092873885 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f2e880b2a70 0x7f2e880b4f30 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2e94009e60 tx=0x7f2e9400b700 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:41:35.111 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 620.0 s, 497.0 tps, lat 200.943 ms stddev 242.876 2024-05-03T18:41:35.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.546+0000 7f2eb6864b80 1 Processor -- start 2024-05-03T18:41:35.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.546+0000 7f2eb6864b80 1 -- start start 2024-05-03T18:41:35.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.547+0000 7f2eb6864b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3271859b0 0x55f327185d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:35.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.547+0000 7f2eb6864b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f3271862d0 0x55f32717ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:35.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.547+0000 7f2eb6864b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f3271805d0 0x55f327180a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:35.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.547+0000 7f2ea9e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f3271805d0 0x55f327180a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:35.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.548+0000 7f2ea9e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f3271805d0 0x55f327180a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33166/0 (socket says 172.21.15.33:33166) 2024-05-03T18:41:35.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.548+0000 7f2ea9e00700 1 -- 172.21.15.33:0/41124436 learned_addr learned my addr 172.21.15.33:0/41124436 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:41:35.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.548+0000 7f2ea9400700 1 --2- 172.21.15.33:0/41124436 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3271859b0 0x55f327185d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:35.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.548+0000 7f2eb6864b80 1 -- 172.21.15.33:0/41124436 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55f3270d2aa0 con 0x55f3271805d0 2024-05-03T18:41:35.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.548+0000 7f2eb6864b80 1 -- 172.21.15.33:0/41124436 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55f3270707e0 con 0x55f3271859b0 2024-05-03T18:41:35.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.548+0000 7f2eb6864b80 1 -- 172.21.15.33:0/41124436 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55f3270cea60 con 0x55f3271862d0 2024-05-03T18:41:35.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.548+0000 7f2ea8a00700 1 --2- 172.21.15.33:0/41124436 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f3271862d0 0x55f32717ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:35.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.549+0000 7f2ea9e00700 1 -- 172.21.15.33:0/41124436 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f3271862d0 msgr2=0x55f32717ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:41:35.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.549+0000 7f2ea9e00700 1 --2- 172.21.15.33:0/41124436 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f3271862d0 0x55f32717ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:35.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.549+0000 7f2ea9e00700 1 -- 172.21.15.33:0/41124436 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3271859b0 msgr2=0x55f327185d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:41:35.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.549+0000 7f2ea9e00700 1 --2- 172.21.15.33:0/41124436 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3271859b0 0x55f327185d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:35.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.549+0000 7f2ea9e00700 1 -- 172.21.15.33:0/41124436 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f32701f950 con 0x55f3271805d0 2024-05-03T18:41:35.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.550+0000 7f2ea9e00700 1 --2- 172.21.15.33:0/41124436 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f3271805d0 0x55f327180a50 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c009cd0 tx=0x7f2e8c00e350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54c722b26852bca8 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:41:35.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.551+0000 7f2e9aa00700 1 -- 172.21.15.33:0/41124436 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e8c00ec00 con 0x55f3271805d0 2024-05-03T18:41:35.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.551+0000 7f2e9aa00700 1 -- 172.21.15.33:0/41124436 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2e8c00eda0 con 0x55f3271805d0 2024-05-03T18:41:35.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.551+0000 7f2e9aa00700 1 -- 172.21.15.33:0/41124436 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e8c0056c0 con 0x55f3271805d0 2024-05-03T18:41:35.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.552+0000 7f2eb6864b80 1 -- 172.21.15.33:0/41124436 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f3271805d0 msgr2=0x55f327180a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:41:35.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.552+0000 7f2eb6864b80 1 --2- 172.21.15.33:0/41124436 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f3271805d0 0x55f327180a50 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c009cd0 tx=0x7f2e8c00e350 comp rx=0 tx=0).stop 2024-05-03T18:41:35.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.552+0000 7f2eb6864b80 1 -- 172.21.15.33:0/41124436 shutdown_connections 2024-05-03T18:41:35.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.552+0000 7f2eb6864b80 1 --2- 172.21.15.33:0/41124436 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f3271805d0 0x55f327180a50 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:35.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.552+0000 7f2eb6864b80 1 --2- 172.21.15.33:0/41124436 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3271859b0 0x55f327185d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:35.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.552+0000 7f2eb6864b80 1 --2- 172.21.15.33:0/41124436 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f3271862d0 0x55f32717ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:35.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.552+0000 7f2eb6864b80 1 -- 172.21.15.33:0/41124436 >> 172.21.15.33:0/41124436 conn(0x55f3270f8750 msgr2=0x55f32717a980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:41:35.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.553+0000 7f2eb6864b80 1 -- 172.21.15.33:0/41124436 shutdown_connections 2024-05-03T18:41:35.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.553+0000 7f2eb6864b80 1 -- 172.21.15.33:0/41124436 wait complete. 2024-05-03T18:41:35.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.555+0000 7f2eb6864b80 1 -- start start 2024-05-03T18:41:35.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.555+0000 7f2eb6864b80 1 Processor -- start 2024-05-03T18:41:35.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.556+0000 7f2eb6864b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f3271805d0 0x55f32718bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:35.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.556+0000 7f2eb6864b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3271859b0 0x55f327189310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:35.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.556+0000 7f2eb6864b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f3271862d0 0x55f327189850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:35.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.556+0000 7f2eb6864b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55f32718cd80 con 0x55f3271805d0 2024-05-03T18:41:35.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.556+0000 7f2eb6864b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55f327038ba0 con 0x55f3271859b0 2024-05-03T18:41:35.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.556+0000 7f2eb6864b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55f32718cf00 con 0x55f3271862d0 2024-05-03T18:41:35.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.556+0000 7f2eb6864b80 10 client.0 _ll_register_callbacks cb 0x55f327079cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:41:35.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.557+0000 7f2ea8a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3271859b0 0x55f327189310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:35.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.557+0000 7f2ea8a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3271859b0 0x55f327189310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:56928/0 (socket says 172.21.15.33:56928) 2024-05-03T18:41:35.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.557+0000 7f2ea8a00700 1 -- 172.21.15.33:0/435205569 learned_addr learned my addr 172.21.15.33:0/435205569 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:41:35.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.557+0000 7f2ea9e00700 1 --2- 172.21.15.33:0/435205569 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f3271862d0 0x55f327189850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:35.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.557+0000 7f2ea9400700 1 --2- 172.21.15.33:0/435205569 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f3271805d0 0x55f32718bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:35.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.558+0000 7f2ea8a00700 1 -- 172.21.15.33:0/435205569 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f3271862d0 msgr2=0x55f327189850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:41:35.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.558+0000 7f2ea8a00700 1 --2- 172.21.15.33:0/435205569 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f3271862d0 0x55f327189850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:35.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.558+0000 7f2ea8a00700 1 -- 172.21.15.33:0/435205569 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f3271805d0 msgr2=0x55f32718bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:41:35.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.558+0000 7f2ea8a00700 1 --2- 172.21.15.33:0/435205569 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f3271805d0 0x55f32718bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:35.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.558+0000 7f2ea8a00700 1 -- 172.21.15.33:0/435205569 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3270984a0 con 0x55f3271859b0 2024-05-03T18:41:35.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.558+0000 7f2ea8a00700 1 --2- 172.21.15.33:0/435205569 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3271859b0 0x55f327189310 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2e94015000 tx=0x7f2e940099e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7721c513379db495 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:41:35.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.558+0000 7f2e93400700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-03T18:41:35.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.559+0000 7f2e93400700 1 -- 172.21.15.33:0/435205569 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e94007970 con 0x55f3271859b0 2024-05-03T18:41:35.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.559+0000 7f2eb6864b80 1 -- 172.21.15.33:0/435205569 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e8c00eda0 con 0x55f3271859b0 2024-05-03T18:41:35.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.559+0000 7f2e93400700 1 -- 172.21.15.33:0/435205569 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9518+0+0 (secure 0 0 0) 0x7f2e94007b10 con 0x55f3271859b0 2024-05-03T18:41:35.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.560+0000 7f2e93400700 1 -- 172.21.15.33:0/435205569 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f2e94007d10 con 0x55f3271859b0 2024-05-03T18:41:35.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.560+0000 7f2eb6864b80 10 client.36428 fetch_fsmap learned FSMap version 21 2024-05-03T18:41:35.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.560+0000 7f2eb6864b80 1 -- 172.21.15.33:0/435205569 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55f327197550 con 0x55f3271859b0 2024-05-03T18:41:35.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.561+0000 7f2e93400700 1 -- 172.21.15.33:0/435205569 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2e8c00eda0 con 0x55f3271859b0 2024-05-03T18:41:35.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.562+0000 7f2e93400700 1 -- 172.21.15.33:0/435205569 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f2e94011070 con 0x55f3271859b0 2024-05-03T18:41:35.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.562+0000 7f2eb6864b80 10 client.36428 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:41:35.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.562+0000 7f2eb6864b80 10 client.36428 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:41:35.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.562+0000 7f2eb6864b80 20 client.36428 populate_metadata read hostname 'smithi033' 2024-05-03T18:41:35.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.562+0000 7f2eb6864b80 1 --2- 172.21.15.33:0/435205569 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55f327199ae0 0x55f32719bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:35.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.562+0000 7f2eb6864b80 4 client.36428 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:41:35.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.562+0000 7f2eb6864b80 1 -- 172.21.15.33:0/435205569 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f32714d070 con 0x55f327199ae0 2024-05-03T18:41:35.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.563+0000 7f2ea9400700 1 --2- 172.21.15.33:0/435205569 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55f327199ae0 0x55f32719bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:35.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.564+0000 7f2ea9400700 1 --2- 172.21.15.33:0/435205569 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55f327199ae0 0x55f32719bed0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c7da36466eefd9c server_cookie=90765213ce365582 in_seq=0 out_seq=0 2024-05-03T18:41:35.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.564+0000 7f2e93400700 10 client.36428 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:41:35.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.567+0000 7f2e93400700 1 -- 172.21.15.33:0/435205569 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f32714d070 con 0x55f327199ae0 2024-05-03T18:41:35.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.567+0000 7f2e93400700 10 client.36428 handle_command_reply: tid=0 2024-05-03T18:41:35.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.567+0000 7f2eb6864b80 1 client.36428 shutdown 2024-05-03T18:41:35.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.567+0000 7f2eb6864b80 1 -- 172.21.15.33:0/435205569 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3271859b0 msgr2=0x55f327189310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:41:35.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.567+0000 7f2eb6864b80 1 --2- 172.21.15.33:0/435205569 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3271859b0 0x55f327189310 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2e94015000 tx=0x7f2e940099e0 comp rx=0 tx=0).stop 2024-05-03T18:41:35.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.568+0000 7f2eb6864b80 1 -- 172.21.15.33:0/435205569 shutdown_connections 2024-05-03T18:41:35.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.568+0000 7f2eb6864b80 1 --2- 172.21.15.33:0/435205569 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55f327199ae0 0x55f32719bed0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:35.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.569+0000 7f2e93400700 0 client.36428 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:41:35.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.569+0000 7f2eb6864b80 1 --2- 172.21.15.33:0/435205569 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f3271862d0 0x55f327189850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:35.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.569+0000 7f2eb6864b80 1 --2- 172.21.15.33:0/435205569 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f3271805d0 0x55f32718bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:35.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.569+0000 7f2eb6864b80 1 --2- 172.21.15.33:0/435205569 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3271859b0 0x55f327189310 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:35.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.569+0000 7f2eb6864b80 1 -- 172.21.15.33:0/435205569 >> 172.21.15.33:0/435205569 conn(0x55f3270f8750 msgr2=0x55f327175430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:41:35.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.569+0000 7f2eb6864b80 1 -- 172.21.15.33:0/435205569 shutdown_connections 2024-05-03T18:41:35.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.570+0000 7f2eb6864b80 1 -- 172.21.15.33:0/435205569 wait complete. 2024-05-03T18:41:35.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.570+0000 7f2eb6864b80 20 client.36428 trim_cache size 0 max 16384 2024-05-03T18:41:35.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.611+0000 7f2eb6864b80 1 Processor -- start 2024-05-03T18:41:35.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.612+0000 7f2eb6864b80 1 -- start start 2024-05-03T18:41:35.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.612+0000 7f2eb6864b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3271805d0 0x55f32717ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:35.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.612+0000 7f2eb6864b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f3271859b0 0x55f327186f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:35.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.612+0000 7f2eb6864b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f3271862d0 0x55f3271752b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:35.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.612+0000 7f2eb6864b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2ea41b3cb0 con 0x55f3271862d0 2024-05-03T18:41:35.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.612+0000 7f2eb6864b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2ea41b6510 con 0x55f3271805d0 2024-05-03T18:41:35.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.612+0000 7f2eb6864b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2ea41b4300 con 0x55f3271859b0 2024-05-03T18:41:35.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.613+0000 7f2ea9e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f3271862d0 0x55f3271752b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:35.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.613+0000 7f2ea9e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f3271862d0 0x55f3271752b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33180/0 (socket says 172.21.15.33:33180) 2024-05-03T18:41:35.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.613+0000 7f2ea9e00700 1 -- 172.21.15.33:0/3450669618 learned_addr learned my addr 172.21.15.33:0/3450669618 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:41:35.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.613+0000 7f2ea9e00700 1 -- 172.21.15.33:0/3450669618 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f3271859b0 msgr2=0x55f327186f40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-03T18:41:35.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.613+0000 7f2ea9e00700 1 --2- 172.21.15.33:0/3450669618 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f3271859b0 0x55f327186f40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:35.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.613+0000 7f2ea9e00700 1 -- 172.21.15.33:0/3450669618 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3271805d0 msgr2=0x55f32717ffb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-03T18:41:35.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.615+0000 7f2ea9400700 1 --2- 172.21.15.33:0/3450669618 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3271805d0 0x55f32717ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:35.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.615+0000 7f2ea9e00700 1 --2- 172.21.15.33:0/3450669618 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3271805d0 0x55f32717ffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:35.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.615+0000 7f2ea9e00700 1 -- 172.21.15.33:0/3450669618 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3271890e0 con 0x55f3271862d0 2024-05-03T18:41:35.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.615+0000 7f2ea9e00700 1 --2- 172.21.15.33:0/3450669618 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f3271862d0 0x55f3271752b0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x55f327180f90 tx=0x7f2e8c005260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d699567e7f9b779 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:41:35.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.616+0000 7f2e9a000700 1 -- 172.21.15.33:0/3450669618 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e8c027020 con 0x55f3271862d0 2024-05-03T18:41:35.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.616+0000 7f2e9a000700 1 -- 172.21.15.33:0/3450669618 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2e8c00ec00 con 0x55f3271862d0 2024-05-03T18:41:35.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.616+0000 7f2e9a000700 1 -- 172.21.15.33:0/3450669618 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e8c027020 con 0x55f3271862d0 2024-05-03T18:41:35.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.617+0000 7f2eb6864b80 1 -- 172.21.15.33:0/3450669618 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f3271862d0 msgr2=0x55f3271752b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:41:35.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.617+0000 7f2eb6864b80 1 --2- 172.21.15.33:0/3450669618 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f3271862d0 0x55f3271752b0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x55f327180f90 tx=0x7f2e8c005260 comp rx=0 tx=0).stop 2024-05-03T18:41:35.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.619+0000 7f2eb6864b80 1 -- 172.21.15.33:0/3450669618 shutdown_connections 2024-05-03T18:41:35.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.619+0000 7f2eb6864b80 1 --2- 172.21.15.33:0/3450669618 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f3271862d0 0x55f3271752b0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:35.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.619+0000 7f2eb6864b80 1 --2- 172.21.15.33:0/3450669618 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3271805d0 0x55f32717ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:35.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.619+0000 7f2eb6864b80 1 --2- 172.21.15.33:0/3450669618 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f3271859b0 0x55f327186f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:35.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.619+0000 7f2eb6864b80 1 -- 172.21.15.33:0/3450669618 >> 172.21.15.33:0/3450669618 conn(0x55f3270f8750 msgr2=0x55f327174ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:41:35.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.620+0000 7f2eb6864b80 1 -- 172.21.15.33:0/3450669618 shutdown_connections 2024-05-03T18:41:35.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.620+0000 7f2eb6864b80 1 -- 172.21.15.33:0/3450669618 wait complete. 2024-05-03T18:41:35.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.622+0000 7f2eb6864b80 1 -- start start 2024-05-03T18:41:35.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.622+0000 7f2eb6864b80 1 Processor -- start 2024-05-03T18:41:35.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.623+0000 7f2eb6864b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f3271805d0 0x55f32718f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:35.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.623+0000 7f2eb6864b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3271859b0 0x55f32718f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:35.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.623+0000 7f2eb6864b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f3271862d0 0x55f327204360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:35.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.624+0000 7f2eb6864b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2ea41b49d0 con 0x55f3271805d0 2024-05-03T18:41:35.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.624+0000 7f2eb6864b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2ea41b5e40 con 0x55f3271859b0 2024-05-03T18:41:35.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.624+0000 7f2eb6864b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2ea41b50a0 con 0x55f3271862d0 2024-05-03T18:41:35.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.624+0000 7f2eb6864b80 10 client.0 _ll_register_callbacks cb 0x55f327079cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:41:35.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.624+0000 7f2ea9400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f3271805d0 0x55f32718f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:35.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.625+0000 7f2ea9400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f3271805d0 0x55f32718f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33182/0 (socket says 172.21.15.33:33182) 2024-05-03T18:41:35.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.625+0000 7f2ea9400700 1 -- 172.21.15.33:0/447245636 learned_addr learned my addr 172.21.15.33:0/447245636 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:41:35.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.625+0000 7f2ea9e00700 1 --2- 172.21.15.33:0/447245636 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f3271862d0 0x55f327204360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:35.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.626+0000 7f2ea9400700 1 -- 172.21.15.33:0/447245636 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f3271862d0 msgr2=0x55f327204360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:41:35.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.626+0000 7f2ea9400700 1 --2- 172.21.15.33:0/447245636 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f3271862d0 0x55f327204360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:35.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.626+0000 7f2ea9400700 1 -- 172.21.15.33:0/447245636 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3271859b0 msgr2=0x55f32718f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:41:35.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.626+0000 7f2ea9400700 1 --2- 172.21.15.33:0/447245636 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3271859b0 0x55f32718f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:35.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.626+0000 7f2ea9400700 1 -- 172.21.15.33:0/447245636 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f32718ff70 con 0x55f3271805d0 2024-05-03T18:41:35.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.627+0000 7f2ea9400700 1 --2- 172.21.15.33:0/447245636 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f3271805d0 0x55f32718f160 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c0e0690 tx=0x7f2e9c0a9440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a738570c2aec90b server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:41:35.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.627+0000 7f2e93400700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:41:35.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.627+0000 7f2e93400700 1 -- 172.21.15.33:0/447245636 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e9c0c2020 con 0x55f3271805d0 2024-05-03T18:41:35.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.627+0000 7f2e93400700 1 -- 172.21.15.33:0/447245636 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9518+0+0 (secure 0 0 0) 0x7f2e9c0a5eb0 con 0x55f3271805d0 2024-05-03T18:41:35.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.628+0000 7f2eb6864b80 1 -- 172.21.15.33:0/447245636 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e8c00ec00 con 0x55f3271805d0 2024-05-03T18:41:35.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.628+0000 7f2e93400700 1 -- 172.21.15.33:0/447245636 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f2e9c0a9c20 con 0x55f3271805d0 2024-05-03T18:41:35.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.628+0000 7f2eb6864b80 10 client.16698 fetch_fsmap learned FSMap version 21 2024-05-03T18:41:35.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.628+0000 7f2eb6864b80 1 -- 172.21.15.33:0/447245636 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55f327216a50 con 0x55f3271805d0 2024-05-03T18:41:35.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.628+0000 7f2e93400700 1 -- 172.21.15.33:0/447245636 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2e8c00ec00 con 0x55f3271805d0 2024-05-03T18:41:35.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.629+0000 7f2e93400700 1 -- 172.21.15.33:0/447245636 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f2e9c016330 con 0x55f3271805d0 2024-05-03T18:41:35.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.629+0000 7f2eb6864b80 10 client.16698 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:41:35.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.629+0000 7f2eb6864b80 10 client.16698 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:41:35.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.629+0000 7f2eb6864b80 20 client.16698 populate_metadata read hostname 'smithi033' 2024-05-03T18:41:35.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.629+0000 7f2eb6864b80 1 --2- 172.21.15.33:0/447245636 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55f327199ae0 0x55f327219120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:35.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.629+0000 7f2eb6864b80 4 client.16698 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-03T18:41:35.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.629+0000 7f2eb6864b80 1 -- 172.21.15.33:0/447245636 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f3270e0480 con 0x55f327199ae0 2024-05-03T18:41:35.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.630+0000 7f2ea8a00700 1 --2- 172.21.15.33:0/447245636 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55f327199ae0 0x55f327219120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:35.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.631+0000 7f2ea8a00700 1 --2- 172.21.15.33:0/447245636 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55f327199ae0 0x55f327219120 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3b705919ac4dcaf server_cookie=b23af46e3021b5ce in_seq=0 out_seq=0 2024-05-03T18:41:35.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.631+0000 7f2e93400700 10 client.16698 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:41:35.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.633+0000 7f2e93400700 1 -- 172.21.15.33:0/447245636 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f3270e0480 con 0x55f327199ae0 2024-05-03T18:41:35.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.633+0000 7f2e93400700 10 client.16698 handle_command_reply: tid=0 2024-05-03T18:41:35.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.634+0000 7f2eb6864b80 1 client.16698 shutdown 2024-05-03T18:41:35.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.634+0000 7f2eb6864b80 1 -- 172.21.15.33:0/447245636 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f3271805d0 msgr2=0x55f32718f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:41:35.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.634+0000 7f2eb6864b80 1 --2- 172.21.15.33:0/447245636 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f3271805d0 0x55f32718f160 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c0e0690 tx=0x7f2e9c0a9440 comp rx=0 tx=0).stop 2024-05-03T18:41:35.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.635+0000 7f2eb6864b80 1 -- 172.21.15.33:0/447245636 shutdown_connections 2024-05-03T18:41:35.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.635+0000 7f2eb6864b80 1 --2- 172.21.15.33:0/447245636 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55f327199ae0 0x55f327219120 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:35.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.635+0000 7f2eb6864b80 1 --2- 172.21.15.33:0/447245636 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f3271862d0 0x55f327204360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:35.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.635+0000 7f2eb6864b80 1 --2- 172.21.15.33:0/447245636 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f3271805d0 0x55f32718f160 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:35.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.635+0000 7f2eb6864b80 1 --2- 172.21.15.33:0/447245636 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3271859b0 0x55f32718f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:35.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.635+0000 7f2e93400700 0 client.16698 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:41:35.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.635+0000 7f2eb6864b80 1 -- 172.21.15.33:0/447245636 >> 172.21.15.33:0/447245636 conn(0x55f3270f8750 msgr2=0x55f327175430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:41:35.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.637+0000 7f2eb6864b80 1 -- 172.21.15.33:0/447245636 shutdown_connections 2024-05-03T18:41:35.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.637+0000 7f2eb6864b80 1 -- 172.21.15.33:0/447245636 wait complete. 2024-05-03T18:41:35.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.637+0000 7f2eb6864b80 20 client.16698 trim_cache size 0 max 16384 2024-05-03T18:41:35.640 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-03T18:41:35.640 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-05-03T18:41:35.640 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "4b9bebf8-2da9-48dd-a7ac-ac96c4845742", 2024-05-03T18:41:35.640 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-05-03T18:41:35.640 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-03T18:41:35.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.638+0000 7f2eabe00700 1 -- 172.21.15.33:0/2092873885 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f2e880b2a70 msgr2=0x7f2e880b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:41:35.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.638+0000 7f2eabe00700 1 --2- 172.21.15.33:0/2092873885 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f2e880b2a70 0x7f2e880b4f30 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2e94009e60 tx=0x7f2e9400b700 comp rx=0 tx=0).stop 2024-05-03T18:41:35.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.638+0000 7f2eabe00700 1 -- 172.21.15.33:0/2092873885 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ea41056e0 msgr2=0x7f2ea41124c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:41:35.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.638+0000 7f2eabe00700 1 --2- 172.21.15.33:0/2092873885 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ea41056e0 0x7f2ea41124c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c00e9c0 tx=0x7f2e9c00ed80 comp rx=0 tx=0).stop 2024-05-03T18:41:35.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.638+0000 7f2eabe00700 1 -- 172.21.15.33:0/2092873885 shutdown_connections 2024-05-03T18:41:35.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.638+0000 7f2eabe00700 1 --2- 172.21.15.33:0/2092873885 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f2e880b2a70 0x7f2e880b4f30 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:35.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.638+0000 7f2eabe00700 1 --2- 172.21.15.33:0/2092873885 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ea410b6e0 0x7f2ea410fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:35.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.638+0000 7f2eabe00700 1 --2- 172.21.15.33:0/2092873885 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ea41056e0 0x7f2ea41124c0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:35.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.638+0000 7f2eabe00700 1 --2- 172.21.15.33:0/2092873885 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ea41060a0 0x7f2ea410f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:35.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.638+0000 7f2eabe00700 1 -- 172.21.15.33:0/2092873885 >> 172.21.15.33:0/2092873885 conn(0x7f2ea41011d0 msgr2=0x7f2ea4101a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:41:35.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.639+0000 7f2eabe00700 1 -- 172.21.15.33:0/2092873885 shutdown_connections 2024-05-03T18:41:35.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:35.639+0000 7f2eabe00700 1 -- 172.21.15.33:0/2092873885 wait complete. 2024-05-03T18:41:36.298 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:41:36 smithi099 ceph-mon[31658]: pgmap v450: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 7.3 MiB/s wr, 694 op/s 2024-05-03T18:41:36.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:41:36 smithi150 ceph-mon[32492]: pgmap v450: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 7.3 MiB/s wr, 694 op/s 2024-05-03T18:41:36.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:41:36 smithi033 ceph-mon[28330]: pgmap v450: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 7.3 MiB/s wr, 694 op/s 2024-05-03T18:41:37.291 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:41:37 smithi033 ceph-mon[28330]: scrub queued for path: / 2024-05-03T18:41:37.291 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:41:37 smithi033 ceph-mon[28330]: scrub summary: idle+waiting paths [/] 2024-05-03T18:41:37.291 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:41:37 smithi033 ceph-mon[28330]: scrub summary: active paths [/] 2024-05-03T18:41:37.291 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:41:37 smithi033 ceph-mon[28330]: scrub summary: idle+waiting paths [/] 2024-05-03T18:41:37.291 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:41:37 smithi033 ceph-mon[28330]: scrub summary: idle 2024-05-03T18:41:37.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:41:37 smithi099 ceph-mon[31658]: scrub queued for path: / 2024-05-03T18:41:37.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:41:37 smithi099 ceph-mon[31658]: scrub summary: idle+waiting paths [/] 2024-05-03T18:41:37.297 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:41:37 smithi099 ceph-mon[31658]: scrub summary: active paths [/] 2024-05-03T18:41:37.297 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:41:37 smithi099 ceph-mon[31658]: scrub summary: idle+waiting paths [/] 2024-05-03T18:41:37.297 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:41:37 smithi099 ceph-mon[31658]: scrub summary: idle 2024-05-03T18:41:37.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:41:37 smithi150 ceph-mon[32492]: scrub queued for path: / 2024-05-03T18:41:37.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:41:37 smithi150 ceph-mon[32492]: scrub summary: idle+waiting paths [/] 2024-05-03T18:41:37.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:41:37 smithi150 ceph-mon[32492]: scrub summary: active paths [/] 2024-05-03T18:41:37.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:41:37 smithi150 ceph-mon[32492]: scrub summary: idle+waiting paths [/] 2024-05-03T18:41:37.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:41:37 smithi150 ceph-mon[32492]: scrub summary: idle 2024-05-03T18:41:37.891 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-03T18:41:38.236 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:41:38 smithi033 ceph-mon[28330]: pgmap v451: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 5.6 MiB/s wr, 543 op/s 2024-05-03T18:41:38.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:41:38 smithi099 ceph-mon[31658]: pgmap v451: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 5.6 MiB/s wr, 543 op/s 2024-05-03T18:41:38.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:41:38 smithi150 ceph-mon[32492]: pgmap v451: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 5.6 MiB/s wr, 543 op/s 2024-05-03T18:41:38.643 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:41:40.111 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 625.0 s, 686.1 tps, lat 144.887 ms stddev 77.600 2024-05-03T18:41:40.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:41:40 smithi099 ceph-mon[31658]: pgmap v452: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 5.3 MiB/s wr, 512 op/s 2024-05-03T18:41:40.384 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:41:40 smithi150 ceph-mon[32492]: pgmap v452: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 5.3 MiB/s wr, 512 op/s 2024-05-03T18:41:40.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:41:40 smithi033 ceph-mon[28330]: pgmap v452: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 5.3 MiB/s wr, 512 op/s 2024-05-03T18:41:42.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:41:42 smithi099 ceph-mon[31658]: pgmap v453: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 8.5 MiB/s wr, 753 op/s 2024-05-03T18:41:42.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:41:42 smithi150 ceph-mon[32492]: pgmap v453: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 8.5 MiB/s wr, 753 op/s 2024-05-03T18:41:42.456 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:41:42 smithi033 ceph-mon[28330]: pgmap v453: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 8.5 MiB/s wr, 753 op/s 2024-05-03T18:41:43.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.347+0000 7f2242a00700 1 -- 172.21.15.33:0/2641995398 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f223c10a9e0 msgr2=0x7f223c10ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:41:43.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.347+0000 7f2242a00700 1 --2- 172.21.15.33:0/2641995398 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f223c10a9e0 0x7f223c10ae60 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f223400b0a0 tx=0x7f223400b3b0 comp rx=0 tx=0).stop 2024-05-03T18:41:43.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.348+0000 7f2242a00700 1 -- 172.21.15.33:0/2641995398 shutdown_connections 2024-05-03T18:41:43.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.348+0000 7f2242a00700 1 --2- 172.21.15.33:0/2641995398 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f223c1184a0 0x7f223c11a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:43.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.348+0000 7f2242a00700 1 --2- 172.21.15.33:0/2641995398 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f223c10dab0 0x7f223c10a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:43.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.348+0000 7f2242a00700 1 --2- 172.21.15.33:0/2641995398 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f223c10a9e0 0x7f223c10ae60 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:43.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.348+0000 7f2242a00700 1 -- 172.21.15.33:0/2641995398 >> 172.21.15.33:0/2641995398 conn(0x7f223c06e8f0 msgr2=0x7f223c06ed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:41:43.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.348+0000 7f2242a00700 1 -- 172.21.15.33:0/2641995398 shutdown_connections 2024-05-03T18:41:43.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.348+0000 7f2242a00700 1 -- 172.21.15.33:0/2641995398 wait complete. 2024-05-03T18:41:43.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.350+0000 7f2242a00700 1 Processor -- start 2024-05-03T18:41:43.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.351+0000 7f2242a00700 1 -- start start 2024-05-03T18:41:43.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.352+0000 7f2242a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f223c10a9e0 0x7f223c1a44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:43.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.352+0000 7f2242a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f223c10dab0 0x7f223c1a49f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:43.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.352+0000 7f2242a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f223c1184a0 0x7f223c1a8d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:43.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.352+0000 7f2242a00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f223c11de80 con 0x7f223c10a9e0 2024-05-03T18:41:43.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.352+0000 7f2242a00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f223c11dd00 con 0x7f223c1184a0 2024-05-03T18:41:43.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.352+0000 7f2242a00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f223c11e000 con 0x7f223c10dab0 2024-05-03T18:41:43.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.353+0000 7f2240c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f223c10dab0 0x7f223c1a49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:43.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.353+0000 7f2240c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f223c10dab0 0x7f223c1a49f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:41644/0 (socket says 172.21.15.33:41644) 2024-05-03T18:41:43.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.353+0000 7f2240c00700 1 -- 172.21.15.33:0/3076000612 learned_addr learned my addr 172.21.15.33:0/3076000612 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:41:43.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.353+0000 7f2241600700 1 --2- 172.21.15.33:0/3076000612 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f223c10a9e0 0x7f223c1a44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:43.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.353+0000 7f2242000700 1 --2- 172.21.15.33:0/3076000612 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f223c1184a0 0x7f223c1a8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:43.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.354+0000 7f2240c00700 1 -- 172.21.15.33:0/3076000612 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f223c1184a0 msgr2=0x7f223c1a8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:41:43.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.354+0000 7f2240c00700 1 --2- 172.21.15.33:0/3076000612 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f223c1184a0 0x7f223c1a8d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:43.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.354+0000 7f2240c00700 1 -- 172.21.15.33:0/3076000612 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f223c10a9e0 msgr2=0x7f223c1a44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:41:43.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.354+0000 7f2240c00700 1 --2- 172.21.15.33:0/3076000612 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f223c10a9e0 0x7f223c1a44b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:43.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.354+0000 7f2240c00700 1 -- 172.21.15.33:0/3076000612 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f223c1a94c0 con 0x7f223c10dab0 2024-05-03T18:41:43.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.355+0000 7f2240c00700 1 --2- 172.21.15.33:0/3076000612 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f223c10dab0 0x7f223c1a49f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f223400b070 tx=0x7f2234004e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:41:43.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.355+0000 7f2242000700 1 --2- 172.21.15.33:0/3076000612 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f223c1184a0 0x7f223c1a8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:41:43.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.356+0000 7f223a000700 1 -- 172.21.15.33:0/3076000612 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2234021070 con 0x7f223c10dab0 2024-05-03T18:41:43.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.356+0000 7f223a000700 1 -- 172.21.15.33:0/3076000612 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2234007d40 con 0x7f223c10dab0 2024-05-03T18:41:43.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.356+0000 7f223a000700 1 -- 172.21.15.33:0/3076000612 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22340047b0 con 0x7f223c10dab0 2024-05-03T18:41:43.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.356+0000 7f2242a00700 1 -- 172.21.15.33:0/3076000612 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f223c1a9750 con 0x7f223c10dab0 2024-05-03T18:41:43.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.357+0000 7f2242a00700 1 -- 172.21.15.33:0/3076000612 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f223c1b1030 con 0x7f223c10dab0 2024-05-03T18:41:43.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.364+0000 7f223a000700 1 -- 172.21.15.33:0/3076000612 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f2234003a00 con 0x7f223c10dab0 2024-05-03T18:41:43.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.368+0000 7f223a000700 1 --2- 172.21.15.33:0/3076000612 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f22240b2b40 0x7f22240b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:43.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.368+0000 7f223a000700 1 -- 172.21.15.33:0/3076000612 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9518+0+0 (secure 0 0 0) 0x7f223400f020 con 0x7f223c10dab0 2024-05-03T18:41:43.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.369+0000 7f2241600700 1 --2- 172.21.15.33:0/3076000612 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f22240b2b40 0x7f22240b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:43.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.371+0000 7f2241600700 1 --2- 172.21.15.33:0/3076000612 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f22240b2b40 0x7f22240b5000 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2230005950 tx=0x7f22300058e0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:41:43.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.501+0000 7f224e9c5b80 1 Processor -- start 2024-05-03T18:41:43.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.504+0000 7f224e9c5b80 1 -- start start 2024-05-03T18:41:43.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.505+0000 7f224e9c5b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ee5e25e250 0x55ee5e25e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:43.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.505+0000 7f224e9c5b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee5e25eb70 0x55ee5e2587c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:43.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.505+0000 7f224e9c5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee5e258de0 0x55ee5e259240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:43.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.505+0000 7f224e9c5b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ee5e1a2a20 con 0x55ee5e25eb70 2024-05-03T18:41:43.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.505+0000 7f224e9c5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55ee5e140760 con 0x55ee5e258de0 2024-05-03T18:41:43.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.505+0000 7f224e9c5b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55ee5e19e9e0 con 0x55ee5e25e250 2024-05-03T18:41:43.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.505+0000 7f2241600700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ee5e25e250 0x55ee5e25e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:43.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.505+0000 7f2241600700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ee5e25e250 0x55ee5e25e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:41654/0 (socket says 172.21.15.33:41654) 2024-05-03T18:41:43.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.505+0000 7f2241600700 1 -- 172.21.15.33:0/2571551192 learned_addr learned my addr 172.21.15.33:0/2571551192 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:41:43.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.506+0000 7f2242000700 1 --2- 172.21.15.33:0/2571551192 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee5e258de0 0x55ee5e259240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:43.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.506+0000 7f2241600700 1 -- 172.21.15.33:0/2571551192 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee5e258de0 msgr2=0x55ee5e259240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:41:43.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.506+0000 7f2241600700 1 --2- 172.21.15.33:0/2571551192 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee5e258de0 0x55ee5e259240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:43.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.506+0000 7f2241600700 1 -- 172.21.15.33:0/2571551192 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee5e25eb70 msgr2=0x55ee5e2587c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:41:43.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.506+0000 7f2241600700 1 --2- 172.21.15.33:0/2571551192 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee5e25eb70 0x55ee5e2587c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:43.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.506+0000 7f2241600700 1 -- 172.21.15.33:0/2571551192 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee5e0ef8d0 con 0x55ee5e25e250 2024-05-03T18:41:43.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.508+0000 7f2241600700 1 --2- 172.21.15.33:0/2571551192 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ee5e25e250 0x55ee5e25e630 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f223000fd70 tx=0x7f2230012000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d8fac8710d29c3d server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:41:43.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.509+0000 7f222ea00700 1 -- 172.21.15.33:0/2571551192 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2230022070 con 0x55ee5e25e250 2024-05-03T18:41:43.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.509+0000 7f222ea00700 1 -- 172.21.15.33:0/2571551192 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2230011c60 con 0x55ee5e25e250 2024-05-03T18:41:43.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.509+0000 7f222ea00700 1 -- 172.21.15.33:0/2571551192 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f223001d610 con 0x55ee5e25e250 2024-05-03T18:41:43.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.511+0000 7f224e9c5b80 1 -- 172.21.15.33:0/2571551192 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ee5e25e250 msgr2=0x55ee5e25e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:41:43.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.511+0000 7f224e9c5b80 1 --2- 172.21.15.33:0/2571551192 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ee5e25e250 0x55ee5e25e630 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f223000fd70 tx=0x7f2230012000 comp rx=0 tx=0).stop 2024-05-03T18:41:43.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.512+0000 7f224e9c5b80 1 -- 172.21.15.33:0/2571551192 shutdown_connections 2024-05-03T18:41:43.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.512+0000 7f224e9c5b80 1 --2- 172.21.15.33:0/2571551192 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee5e258de0 0x55ee5e259240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:43.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.512+0000 7f224e9c5b80 1 --2- 172.21.15.33:0/2571551192 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ee5e25e250 0x55ee5e25e630 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:43.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.512+0000 7f224e9c5b80 1 --2- 172.21.15.33:0/2571551192 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee5e25eb70 0x55ee5e2587c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:43.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.512+0000 7f224e9c5b80 1 -- 172.21.15.33:0/2571551192 >> 172.21.15.33:0/2571551192 conn(0x55ee5e17eb60 msgr2=0x55ee5e24a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:41:43.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.513+0000 7f224e9c5b80 1 -- 172.21.15.33:0/2571551192 shutdown_connections 2024-05-03T18:41:43.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.513+0000 7f224e9c5b80 1 -- 172.21.15.33:0/2571551192 wait complete. 2024-05-03T18:41:43.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.514+0000 7f224e9c5b80 1 -- start start 2024-05-03T18:41:43.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.514+0000 7f224e9c5b80 1 Processor -- start 2024-05-03T18:41:43.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.518+0000 7f224e9c5b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ee5e258de0 0x55ee5e264880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:43.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.518+0000 7f224e9c5b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee5e25e250 0x55ee5e261c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:43.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.519+0000 7f224e9c5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee5e25eb70 0x55ee5e262160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:43.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.519+0000 7f224e9c5b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ee5e265630 con 0x55ee5e25e250 2024-05-03T18:41:43.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.519+0000 7f224e9c5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55ee5e108b20 con 0x55ee5e25eb70 2024-05-03T18:41:43.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.519+0000 7f224e9c5b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55ee5e2657b0 con 0x55ee5e258de0 2024-05-03T18:41:43.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.519+0000 7f224e9c5b80 10 client.0 _ll_register_callbacks cb 0x55ee5e149c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:41:43.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.521+0000 7f2242000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee5e25eb70 0x55ee5e262160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:43.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.521+0000 7f2242000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee5e25eb70 0x55ee5e262160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:33828/0 (socket says 172.21.15.33:33828) 2024-05-03T18:41:43.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.521+0000 7f2242000700 1 -- 172.21.15.33:0/829043684 learned_addr learned my addr 172.21.15.33:0/829043684 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:41:43.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.522+0000 7f2240c00700 1 --2- 172.21.15.33:0/829043684 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee5e25e250 0x55ee5e261c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:43.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.522+0000 7f2242000700 1 -- 172.21.15.33:0/829043684 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ee5e258de0 msgr2=0x55ee5e264880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:41:43.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.523+0000 7f2241600700 1 --2- 172.21.15.33:0/829043684 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ee5e258de0 0x55ee5e264880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:43.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.523+0000 7f2242000700 1 --2- 172.21.15.33:0/829043684 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ee5e258de0 0x55ee5e264880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:43.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.523+0000 7f2242000700 1 -- 172.21.15.33:0/829043684 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee5e25e250 msgr2=0x55ee5e261c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:41:43.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.523+0000 7f2242000700 1 --2- 172.21.15.33:0/829043684 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee5e25e250 0x55ee5e261c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:43.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.523+0000 7f2242000700 1 -- 172.21.15.33:0/829043684 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee5e168420 con 0x55ee5e25eb70 2024-05-03T18:41:43.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.523+0000 7f2240c00700 1 --2- 172.21.15.33:0/829043684 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee5e25e250 0x55ee5e261c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:41:43.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.523+0000 7f2241600700 1 --2- 172.21.15.33:0/829043684 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ee5e258de0 0x55ee5e264880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:41:43.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.524+0000 7f2242000700 1 --2- 172.21.15.33:0/829043684 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee5e25eb70 0x55ee5e262160 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f22280107f0 tx=0x7f2228010bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=facbd809ce7d6448 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:41:43.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.524+0000 7f2223400700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-03T18:41:43.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.524+0000 7f2223400700 1 -- 172.21.15.33:0/829043684 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f222800c980 con 0x55ee5e25eb70 2024-05-03T18:41:43.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.525+0000 7f2223400700 1 -- 172.21.15.33:0/829043684 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9518+0+0 (secure 0 0 0) 0x7f222800cb20 con 0x55ee5e25eb70 2024-05-03T18:41:43.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.525+0000 7f224e9c5b80 1 -- 172.21.15.33:0/829043684 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2230011c60 con 0x55ee5e25eb70 2024-05-03T18:41:43.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.526+0000 7f2223400700 1 -- 172.21.15.33:0/829043684 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f2228025050 con 0x55ee5e25eb70 2024-05-03T18:41:43.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.526+0000 7f224e9c5b80 10 client.36440 fetch_fsmap learned FSMap version 21 2024-05-03T18:41:43.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.526+0000 7f224e9c5b80 1 -- 172.21.15.33:0/829043684 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55ee5e26fed0 con 0x55ee5e25eb70 2024-05-03T18:41:43.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.527+0000 7f2223400700 1 -- 172.21.15.33:0/829043684 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2230011c60 con 0x55ee5e25eb70 2024-05-03T18:41:43.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.527+0000 7f2223400700 1 -- 172.21.15.33:0/829043684 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f2228011440 con 0x55ee5e25eb70 2024-05-03T18:41:43.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.528+0000 7f224e9c5b80 10 client.36440 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:41:43.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.528+0000 7f224e9c5b80 10 client.36440 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:41:43.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.528+0000 7f224e9c5b80 20 client.36440 populate_metadata read hostname 'smithi033' 2024-05-03T18:41:43.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.528+0000 7f224e9c5b80 1 --2- 172.21.15.33:0/829043684 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55ee5e272510 0x55ee5e274900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:43.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.528+0000 7f224e9c5b80 4 client.36440 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:41:43.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.528+0000 7f224e9c5b80 1 -- 172.21.15.33:0/829043684 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ee5e21d1d0 con 0x55ee5e272510 2024-05-03T18:41:43.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.529+0000 7f2241600700 1 --2- 172.21.15.33:0/829043684 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55ee5e272510 0x55ee5e274900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:43.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.530+0000 7f2241600700 1 --2- 172.21.15.33:0/829043684 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55ee5e272510 0x55ee5e274900 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b225b53acacac3 server_cookie=ff1fcd870563217 in_seq=0 out_seq=0 2024-05-03T18:41:43.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.532+0000 7f2223400700 10 client.36440 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:41:43.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.532+0000 7f2223400700 1 -- 172.21.15.33:0/829043684 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ee5e21d1d0 con 0x55ee5e272510 2024-05-03T18:41:43.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.532+0000 7f2223400700 10 client.36440 handle_command_reply: tid=0 2024-05-03T18:41:43.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.533+0000 7f224e9c5b80 1 client.36440 shutdown 2024-05-03T18:41:43.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.534+0000 7f224e9c5b80 1 -- 172.21.15.33:0/829043684 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee5e25eb70 msgr2=0x55ee5e262160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:41:43.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.534+0000 7f224e9c5b80 1 --2- 172.21.15.33:0/829043684 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee5e25eb70 0x55ee5e262160 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f22280107f0 tx=0x7f2228010bb0 comp rx=0 tx=0).stop 2024-05-03T18:41:43.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.536+0000 7f224e9c5b80 1 -- 172.21.15.33:0/829043684 shutdown_connections 2024-05-03T18:41:43.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.536+0000 7f224e9c5b80 1 --2- 172.21.15.33:0/829043684 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55ee5e272510 0x55ee5e274900 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:43.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.536+0000 7f224e9c5b80 1 --2- 172.21.15.33:0/829043684 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee5e25eb70 0x55ee5e262160 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:43.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.536+0000 7f224e9c5b80 1 --2- 172.21.15.33:0/829043684 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ee5e258de0 0x55ee5e264880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:43.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.536+0000 7f224e9c5b80 1 --2- 172.21.15.33:0/829043684 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee5e25e250 0x55ee5e261c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:43.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.536+0000 7f2223400700 0 client.36440 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:41:43.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.536+0000 7f224e9c5b80 1 -- 172.21.15.33:0/829043684 >> 172.21.15.33:0/829043684 conn(0x55ee5e17eb60 msgr2=0x55ee5e24a080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:41:43.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.536+0000 7f224e9c5b80 1 -- 172.21.15.33:0/829043684 shutdown_connections 2024-05-03T18:41:43.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.537+0000 7f224e9c5b80 1 -- 172.21.15.33:0/829043684 wait complete. 2024-05-03T18:41:43.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.537+0000 7f224e9c5b80 20 client.36440 trim_cache size 0 max 16384 2024-05-03T18:41:43.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.582+0000 7f224e9c5b80 1 Processor -- start 2024-05-03T18:41:43.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.582+0000 7f224e9c5b80 1 -- start start 2024-05-03T18:41:43.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.582+0000 7f224e9c5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee5e258de0 0x55ee5e2587c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:43.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.582+0000 7f224e9c5b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee5e25e250 0x55ee5e26e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:43.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.582+0000 7f224e9c5b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ee5e25eb70 0x55ee5e27a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:43.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.583+0000 7f224e9c5b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f223c1b3950 con 0x55ee5e25e250 2024-05-03T18:41:43.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.583+0000 7f224e9c5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f223c1b6170 con 0x55ee5e258de0 2024-05-03T18:41:43.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.583+0000 7f224e9c5b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f223c1b3fa0 con 0x55ee5e25eb70 2024-05-03T18:41:43.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.583+0000 7f2242000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ee5e25eb70 0x55ee5e27a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:43.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.583+0000 7f2240c00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee5e25e250 0x55ee5e26e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:43.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.583+0000 7f2242000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ee5e25eb70 0x55ee5e27a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:41666/0 (socket says 172.21.15.33:41666) 2024-05-03T18:41:43.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.583+0000 7f2242000700 1 -- 172.21.15.33:0/921153195 learned_addr learned my addr 172.21.15.33:0/921153195 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:41:43.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.583+0000 7f2241600700 1 --2- 172.21.15.33:0/921153195 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee5e258de0 0x55ee5e2587c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:43.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.584+0000 7f2242000700 1 -- 172.21.15.33:0/921153195 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee5e258de0 msgr2=0x55ee5e2587c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:41:43.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.584+0000 7f2242000700 1 --2- 172.21.15.33:0/921153195 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee5e258de0 0x55ee5e2587c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:43.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.584+0000 7f2242000700 1 -- 172.21.15.33:0/921153195 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee5e25e250 msgr2=0x55ee5e26e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:41:43.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.584+0000 7f2242000700 1 --2- 172.21.15.33:0/921153195 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee5e25e250 0x55ee5e26e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:43.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.584+0000 7f2242000700 1 -- 172.21.15.33:0/921153195 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee5e274ee0 con 0x55ee5e25eb70 2024-05-03T18:41:43.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.584+0000 7f2242000700 1 --2- 172.21.15.33:0/921153195 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ee5e25eb70 0x55ee5e27a170 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f2228000c00 tx=0x7f2228005e90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65f8244df4501963 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:41:43.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.585+0000 7f222e000700 1 -- 172.21.15.33:0/921153195 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f222802d020 con 0x55ee5e25eb70 2024-05-03T18:41:43.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.585+0000 7f222e000700 1 -- 172.21.15.33:0/921153195 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2228011e50 con 0x55ee5e25eb70 2024-05-03T18:41:43.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.585+0000 7f222e000700 1 -- 172.21.15.33:0/921153195 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f222800c780 con 0x55ee5e25eb70 2024-05-03T18:41:43.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.587+0000 7f224e9c5b80 1 -- 172.21.15.33:0/921153195 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ee5e25eb70 msgr2=0x55ee5e27a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:41:43.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.587+0000 7f224e9c5b80 1 --2- 172.21.15.33:0/921153195 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ee5e25eb70 0x55ee5e27a170 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f2228000c00 tx=0x7f2228005e90 comp rx=0 tx=0).stop 2024-05-03T18:41:43.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.588+0000 7f224e9c5b80 1 -- 172.21.15.33:0/921153195 shutdown_connections 2024-05-03T18:41:43.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.588+0000 7f224e9c5b80 1 --2- 172.21.15.33:0/921153195 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ee5e25eb70 0x55ee5e27a170 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:43.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.588+0000 7f224e9c5b80 1 --2- 172.21.15.33:0/921153195 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee5e258de0 0x55ee5e2587c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:43.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.588+0000 7f224e9c5b80 1 --2- 172.21.15.33:0/921153195 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee5e25e250 0x55ee5e26e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:43.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.588+0000 7f224e9c5b80 1 -- 172.21.15.33:0/921153195 >> 172.21.15.33:0/921153195 conn(0x55ee5e17eb60 msgr2=0x55ee5e25c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:41:43.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.588+0000 7f224e9c5b80 1 -- 172.21.15.33:0/921153195 shutdown_connections 2024-05-03T18:41:43.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.588+0000 7f224e9c5b80 1 -- 172.21.15.33:0/921153195 wait complete. 2024-05-03T18:41:43.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.589+0000 7f224e9c5b80 1 -- start start 2024-05-03T18:41:43.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.590+0000 7f224e9c5b80 1 Processor -- start 2024-05-03T18:41:43.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.590+0000 7f224e9c5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee5e258de0 0x55ee5e279b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:43.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.590+0000 7f224e9c5b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ee5e25e250 0x55ee5e27a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:43.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.590+0000 7f224e9c5b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee5e25eb70 0x55ee5e27a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:43.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.590+0000 7f224e9c5b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f223c1b4630 con 0x55ee5e25eb70 2024-05-03T18:41:43.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.590+0000 7f224e9c5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f223c1b5aa0 con 0x55ee5e258de0 2024-05-03T18:41:43.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.590+0000 7f224e9c5b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f223c1b4d00 con 0x55ee5e25e250 2024-05-03T18:41:43.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.590+0000 7f224e9c5b80 10 client.0 _ll_register_callbacks cb 0x55ee5e149c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:41:43.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.591+0000 7f2240c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ee5e25e250 0x55ee5e27a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:43.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.591+0000 7f2240c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ee5e25e250 0x55ee5e27a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:41670/0 (socket says 172.21.15.33:41670) 2024-05-03T18:41:43.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.591+0000 7f2240c00700 1 -- 172.21.15.33:0/1582566147 learned_addr learned my addr 172.21.15.33:0/1582566147 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:41:43.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.591+0000 7f2241600700 1 --2- 172.21.15.33:0/1582566147 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee5e258de0 0x55ee5e279b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:43.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.592+0000 7f2240c00700 1 -- 172.21.15.33:0/1582566147 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee5e258de0 msgr2=0x55ee5e279b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:41:43.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.592+0000 7f2240c00700 1 --2- 172.21.15.33:0/1582566147 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee5e258de0 0x55ee5e279b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:43.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.592+0000 7f2240c00700 1 -- 172.21.15.33:0/1582566147 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee5e25eb70 msgr2=0x55ee5e27a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:41:43.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.592+0000 7f2240c00700 1 --2- 172.21.15.33:0/1582566147 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee5e25eb70 0x55ee5e27a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:43.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.592+0000 7f2240c00700 1 -- 172.21.15.33:0/1582566147 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee5e25e880 con 0x55ee5e25e250 2024-05-03T18:41:43.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.593+0000 7f2240c00700 1 --2- 172.21.15.33:0/1582566147 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ee5e25e250 0x55ee5e27a0c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f22340aaf70 tx=0x7f22340aa840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70eecd50581946db server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:41:43.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.593+0000 7f2223400700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-03T18:41:43.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.593+0000 7f2223400700 1 -- 172.21.15.33:0/1582566147 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22340aafe0 con 0x55ee5e25e250 2024-05-03T18:41:43.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.593+0000 7f2223400700 1 -- 172.21.15.33:0/1582566147 <== mon.1 v2:172.21.15.150:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9518+0+0 (secure 0 0 0) 0x7f22340b3b30 con 0x55ee5e25e250 2024-05-03T18:41:43.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.594+0000 7f224e9c5b80 1 -- 172.21.15.33:0/1582566147 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2228011e50 con 0x55ee5e25e250 2024-05-03T18:41:43.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.594+0000 7f2223400700 1 -- 172.21.15.33:0/1582566147 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f22340af390 con 0x55ee5e25e250 2024-05-03T18:41:43.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.594+0000 7f224e9c5b80 10 client.26521 fetch_fsmap learned FSMap version 21 2024-05-03T18:41:43.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.594+0000 7f224e9c5b80 1 -- 172.21.15.33:0/1582566147 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55ee5e2dd030 con 0x55ee5e25e250 2024-05-03T18:41:43.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.595+0000 7f2223400700 1 -- 172.21.15.33:0/1582566147 <== mon.1 v2:172.21.15.150:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2228011e50 con 0x55ee5e25e250 2024-05-03T18:41:43.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.596+0000 7f2223400700 1 -- 172.21.15.33:0/1582566147 <== mon.1 v2:172.21.15.150:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f2234011070 con 0x55ee5e25e250 2024-05-03T18:41:43.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.596+0000 7f224e9c5b80 10 client.26521 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:41:43.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.597+0000 7f224e9c5b80 10 client.26521 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:41:43.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.597+0000 7f224e9c5b80 20 client.26521 populate_metadata read hostname 'smithi033' 2024-05-03T18:41:43.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.597+0000 7f224e9c5b80 1 --2- 172.21.15.33:0/1582566147 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55ee5e2dd740 0x55ee5e2dfb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:43.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.597+0000 7f224e9c5b80 4 client.26521 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-03T18:41:43.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.597+0000 7f224e9c5b80 1 -- 172.21.15.33:0/1582566147 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ee5e1b0400 con 0x55ee5e2dd740 2024-05-03T18:41:43.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.599+0000 7f2241600700 1 --2- 172.21.15.33:0/1582566147 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55ee5e2dd740 0x55ee5e2dfb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:43.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.600+0000 7f2241600700 1 --2- 172.21.15.33:0/1582566147 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55ee5e2dd740 0x55ee5e2dfb60 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e83a448a71ee6fe1 server_cookie=52c7f1343e3bb73 in_seq=0 out_seq=0 2024-05-03T18:41:43.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.600+0000 7f2223400700 10 client.26521 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:41:43.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.601+0000 7f2223400700 1 -- 172.21.15.33:0/1582566147 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55ee5e1b0400 con 0x55ee5e2dd740 2024-05-03T18:41:43.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.601+0000 7f2223400700 10 client.26521 handle_command_reply: tid=0 2024-05-03T18:41:43.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.601+0000 7f224e9c5b80 1 client.26521 shutdown 2024-05-03T18:41:43.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.603+0000 7f224e9c5b80 1 -- 172.21.15.33:0/1582566147 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ee5e25e250 msgr2=0x55ee5e27a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:41:43.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.603+0000 7f224e9c5b80 1 --2- 172.21.15.33:0/1582566147 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ee5e25e250 0x55ee5e27a0c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f22340aaf70 tx=0x7f22340aa840 comp rx=0 tx=0).stop 2024-05-03T18:41:43.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.604+0000 7f224e9c5b80 1 -- 172.21.15.33:0/1582566147 shutdown_connections 2024-05-03T18:41:43.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.604+0000 7f224e9c5b80 1 --2- 172.21.15.33:0/1582566147 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55ee5e2dd740 0x55ee5e2dfb60 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:43.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.604+0000 7f224e9c5b80 1 --2- 172.21.15.33:0/1582566147 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ee5e25eb70 0x55ee5e27a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:43.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.604+0000 7f224e9c5b80 1 --2- 172.21.15.33:0/1582566147 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee5e258de0 0x55ee5e279b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:43.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.604+0000 7f224e9c5b80 1 --2- 172.21.15.33:0/1582566147 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ee5e25e250 0x55ee5e27a0c0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:43.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.604+0000 7f2223400700 0 client.26521 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:41:43.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.604+0000 7f224e9c5b80 1 -- 172.21.15.33:0/1582566147 >> 172.21.15.33:0/1582566147 conn(0x55ee5e17eb60 msgr2=0x55ee5e25efd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:41:43.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.604+0000 7f224e9c5b80 1 -- 172.21.15.33:0/1582566147 shutdown_connections 2024-05-03T18:41:43.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.605+0000 7f224e9c5b80 1 -- 172.21.15.33:0/1582566147 wait complete. 2024-05-03T18:41:43.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.605+0000 7f224e9c5b80 20 client.26521 trim_cache size 0 max 16384 2024-05-03T18:41:43.608 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-03T18:41:43.608 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-05-03T18:41:43.608 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-05-03T18:41:43.608 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-03T18:41:43.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.607+0000 7f2242a00700 1 -- 172.21.15.33:0/3076000612 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f22240b2b40 msgr2=0x7f22240b5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:41:43.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.607+0000 7f2242a00700 1 --2- 172.21.15.33:0/3076000612 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f22240b2b40 0x7f22240b5000 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2230005950 tx=0x7f22300058e0 comp rx=0 tx=0).stop 2024-05-03T18:41:43.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.607+0000 7f2242a00700 1 -- 172.21.15.33:0/3076000612 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f223c10dab0 msgr2=0x7f223c1a49f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:41:43.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.607+0000 7f2242a00700 1 --2- 172.21.15.33:0/3076000612 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f223c10dab0 0x7f223c1a49f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f223400b070 tx=0x7f2234004e00 comp rx=0 tx=0).stop 2024-05-03T18:41:43.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.607+0000 7f2242a00700 1 -- 172.21.15.33:0/3076000612 shutdown_connections 2024-05-03T18:41:43.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.607+0000 7f2242a00700 1 --2- 172.21.15.33:0/3076000612 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f22240b2b40 0x7f22240b5000 secure :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2230005950 tx=0x7f22300058e0 comp rx=0 tx=0).stop 2024-05-03T18:41:43.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.607+0000 7f2242a00700 1 --2- 172.21.15.33:0/3076000612 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f223c1184a0 0x7f223c1a8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:43.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.607+0000 7f2242a00700 1 --2- 172.21.15.33:0/3076000612 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f223c10a9e0 0x7f223c1a44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:43.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.607+0000 7f2242a00700 1 --2- 172.21.15.33:0/3076000612 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f223c10dab0 0x7f223c1a49f0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:43.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.607+0000 7f2242a00700 1 -- 172.21.15.33:0/3076000612 >> 172.21.15.33:0/3076000612 conn(0x7f223c06e8f0 msgr2=0x7f223c119c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:41:43.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.608+0000 7f2242a00700 1 -- 172.21.15.33:0/3076000612 shutdown_connections 2024-05-03T18:41:43.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:43.608+0000 7f2242a00700 1 -- 172.21.15.33:0/3076000612 wait complete. 2024-05-03T18:41:44.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:41:44 smithi099 ceph-mon[31658]: pgmap v454: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 6.0 MiB/s wr, 504 op/s 2024-05-03T18:41:44.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:41:44 smithi150 ceph-mon[32492]: pgmap v454: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 6.0 MiB/s wr, 504 op/s 2024-05-03T18:41:44.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:41:44 smithi033 ceph-mon[28330]: pgmap v454: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 6.0 MiB/s wr, 504 op/s 2024-05-03T18:41:45.110 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 630.0 s, 549.5 tps, lat 180.908 ms stddev 100.098 2024-05-03T18:41:45.236 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-03T18:41:45.236 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T18:41:45.977 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:41:46.363 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:41:46 smithi033 ceph-mon[28330]: pgmap v455: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 8.1 MiB/s wr, 684 op/s 2024-05-03T18:41:46.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:41:46 smithi150 ceph-mon[32492]: pgmap v455: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 8.1 MiB/s wr, 684 op/s 2024-05-03T18:41:46.548 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:41:46 smithi099 ceph-mon[31658]: pgmap v455: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 8.1 MiB/s wr, 684 op/s 2024-05-03T18:41:48.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:41:48 smithi033 ceph-mon[28330]: pgmap v456: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 6.2 MiB/s wr, 528 op/s 2024-05-03T18:41:48.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:41:48 smithi150 ceph-mon[32492]: pgmap v456: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 6.2 MiB/s wr, 528 op/s 2024-05-03T18:41:48.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:41:48 smithi099 ceph-mon[31658]: pgmap v456: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 6.2 MiB/s wr, 528 op/s 2024-05-03T18:41:50.111 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 635.0 s, 543.6 tps, lat 181.819 ms stddev 83.262 2024-05-03T18:41:50.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:41:50 smithi150 ceph-mon[32492]: pgmap v457: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 5.7 MiB/s wr, 472 op/s 2024-05-03T18:41:50.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:41:50 smithi033 ceph-mon[28330]: pgmap v457: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 5.7 MiB/s wr, 472 op/s 2024-05-03T18:41:50.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:41:50 smithi099 ceph-mon[31658]: pgmap v457: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 5.7 MiB/s wr, 472 op/s 2024-05-03T18:41:51.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:51.096+0000 7f451c800700 1 -- 172.21.15.33:0/3309380358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4514106070 msgr2=0x7f4514112580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:41:51.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:51.096+0000 7f451c800700 1 --2- 172.21.15.33:0/3309380358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4514106070 0x7f4514112580 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f4510009f90 tx=0x7f451000b330 comp rx=0 tx=0).stop 2024-05-03T18:41:51.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:51.097+0000 7f451c800700 1 -- 172.21.15.33:0/3309380358 shutdown_connections 2024-05-03T18:41:51.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:51.097+0000 7f451c800700 1 --2- 172.21.15.33:0/3309380358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4514106070 0x7f4514112580 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:51.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:51.097+0000 7f451c800700 1 --2- 172.21.15.33:0/3309380358 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f451410b6b0 0x7f451410ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:51.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:51.097+0000 7f451c800700 1 --2- 172.21.15.33:0/3309380358 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f45141056b0 0x7f4514105b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:51.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:51.097+0000 7f451c800700 1 -- 172.21.15.33:0/3309380358 >> 172.21.15.33:0/3309380358 conn(0x7f45141011e0 msgr2=0x7f4514103600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:41:51.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:51.097+0000 7f451c800700 1 -- 172.21.15.33:0/3309380358 shutdown_connections 2024-05-03T18:41:51.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:51.097+0000 7f451c800700 1 -- 172.21.15.33:0/3309380358 wait complete. 2024-05-03T18:41:51.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:51.099+0000 7f451c800700 1 Processor -- start 2024-05-03T18:41:51.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:51.099+0000 7f451c800700 1 -- start start 2024-05-03T18:41:51.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:51.099+0000 7f451c800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45141056b0 0x7f4514074b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:51.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:51.099+0000 7f451c800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4514106070 0x7f4514075080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:51.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:51.099+0000 7f451c800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f451410b6b0 0x7f45141116b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:51.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:51.099+0000 7f451c800700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4514079310 con 0x7f451410b6b0 2024-05-03T18:41:51.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:51.099+0000 7f451c800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4514079190 con 0x7f45141056b0 2024-05-03T18:41:51.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:51.099+0000 7f451c800700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f4514079490 con 0x7f4514106070 2024-05-03T18:41:51.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:51.100+0000 7f4519400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4514106070 0x7f4514075080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:51.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:51.100+0000 7f4519400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4514106070 0x7f4514075080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:39998/0 (socket says 172.21.15.33:39998) 2024-05-03T18:41:51.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:51.100+0000 7f4519400700 1 -- 172.21.15.33:0/2970543603 learned_addr learned my addr 172.21.15.33:0/2970543603 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:41:51.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:51.100+0000 7f4519e00700 1 --2- 172.21.15.33:0/2970543603 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45141056b0 0x7f4514074b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:51.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:51.100+0000 7f451a800700 1 --2- 172.21.15.33:0/2970543603 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f451410b6b0 0x7f45141116b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:51.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:51.101+0000 7f4519400700 1 -- 172.21.15.33:0/2970543603 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45141056b0 msgr2=0x7f4514074b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:41:51.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:51.101+0000 7f4519400700 1 --2- 172.21.15.33:0/2970543603 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45141056b0 0x7f4514074b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:51.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:51.101+0000 7f4519400700 1 -- 172.21.15.33:0/2970543603 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f451410b6b0 msgr2=0x7f45141116b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:41:51.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:51.101+0000 7f4519400700 1 --2- 172.21.15.33:0/2970543603 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f451410b6b0 0x7f45141116b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:51.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:51.101+0000 7f4519400700 1 -- 172.21.15.33:0/2970543603 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4514111f80 con 0x7f4514106070 2024-05-03T18:41:51.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:51.101+0000 7f4519e00700 1 --2- 172.21.15.33:0/2970543603 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45141056b0 0x7f4514074b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:41:51.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:51.102+0000 7f4519400700 1 --2- 172.21.15.33:0/2970543603 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4514106070 0x7f4514075080 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f450400b550 tx=0x7f450400b860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:41:51.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:51.102+0000 7f450ea00700 1 -- 172.21.15.33:0/2970543603 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4504015020 con 0x7f4514106070 2024-05-03T18:41:51.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:51.102+0000 7f450ea00700 1 -- 172.21.15.33:0/2970543603 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4504011cc0 con 0x7f4514106070 2024-05-03T18:41:51.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:51.102+0000 7f450ea00700 1 -- 172.21.15.33:0/2970543603 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4504010c10 con 0x7f4514106070 2024-05-03T18:41:51.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:51.102+0000 7f451c800700 1 -- 172.21.15.33:0/2970543603 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4514112270 con 0x7f4514106070 2024-05-03T18:41:51.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:51.102+0000 7f451c800700 1 -- 172.21.15.33:0/2970543603 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45140706a0 con 0x7f4514106070 2024-05-03T18:41:51.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:51.104+0000 7f451c800700 1 -- 172.21.15.33:0/2970543603 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4514106df0 con 0x7f4514106070 2024-05-03T18:41:51.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:51.110+0000 7f450ea00700 1 -- 172.21.15.33:0/2970543603 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f4504047d10 con 0x7f4514106070 2024-05-03T18:41:51.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:51.113+0000 7f450ea00700 1 --2- 172.21.15.33:0/2970543603 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f44fc0b2a70 0x7f44fc0b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:51.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:51.113+0000 7f4519e00700 1 --2- 172.21.15.33:0/2970543603 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f44fc0b2a70 0x7f44fc0b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:51.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:51.114+0000 7f450ea00700 1 -- 172.21.15.33:0/2970543603 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9518+0+0 (secure 0 0 0) 0x7f45040e24b0 con 0x7f4514106070 2024-05-03T18:41:51.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:51.114+0000 7f450ea00700 1 -- 172.21.15.33:0/2970543603 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f450401f020 con 0x7f4514106070 2024-05-03T18:41:51.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:51.115+0000 7f4519e00700 1 --2- 172.21.15.33:0/2970543603 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f44fc0b2a70 0x7f44fc0b4f30 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4508005950 tx=0x7f450800b410 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:41:51.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:51.663+0000 7f451c800700 1 -- 172.21.15.33:0/2970543603 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f451406a190 con 0x7f4514106070 2024-05-03T18:41:51.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:51.664+0000 7f450ea00700 1 -- 172.21.15.33:0/2970543603 <== mon.1 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11801 (secure 0 0 0) 0x7f451406a190 con 0x7f4514106070 2024-05-03T18:41:51.668 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:41:51.668 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1327393503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1327393503},{"type":"v1","addr":"172.21.15.33:6835","nonce":1327393503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/2461472948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":2461472948},{"type":"v1","addr":"172.21.15.33:6837","nonce":2461472948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14559,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/442269281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":442269281},{"type":"v1","addr":"172.21.15.33:6839","nonce":442269281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14571,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/645571756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":645571756},{"type":"v1","addr":"172.21.15.33:6841","nonce":645571756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/203598909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":203598909},{"type":"v1","addr":"172.21.15.150:6833","nonce":203598909}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/4029842493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":4029842493},{"type":"v1","addr":"172.21.15.150:6835","nonce":4029842493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3463664423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3463664423},{"type":"v1","addr":"172.21.15.150:6837","nonce":3463664423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24481,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6839/4051648018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":4051648018},{"type":"v1","addr":"172.21.15.150:6839","nonce":4051648018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34418,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1865105479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1865105479},{"type":"v1","addr":"172.21.15.99:6833","nonce":1865105479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34445,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/1206393914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1206393914},{"type":"v1","addr":"172.21.15.99:6835","nonce":1206393914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-03T18:22:28.837091+0000","modified":"2024-05-03T18:23:49.758398+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34508},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34472":{"gid":34472,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6837/3360970342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3360970342},{"type":"v1","addr":"172.21.15.99:6837","nonce":3360970342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34508":{"gid":34508,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/919907308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":919907308},{"type":"v1","addr":"172.21.15.99:6839","nonce":919907308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34508,"qdb_cluster":[34508]},"id":1}]} 2024-05-03T18:41:51.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:51.671+0000 7f451c800700 1 -- 172.21.15.33:0/2970543603 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f44fc0b2a70 msgr2=0x7f44fc0b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:41:51.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:51.671+0000 7f451c800700 1 --2- 172.21.15.33:0/2970543603 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f44fc0b2a70 0x7f44fc0b4f30 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4508005950 tx=0x7f450800b410 comp rx=0 tx=0).stop 2024-05-03T18:41:51.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:51.671+0000 7f451c800700 1 -- 172.21.15.33:0/2970543603 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4514106070 msgr2=0x7f4514075080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:41:51.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:51.671+0000 7f451c800700 1 --2- 172.21.15.33:0/2970543603 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4514106070 0x7f4514075080 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f450400b550 tx=0x7f450400b860 comp rx=0 tx=0).stop 2024-05-03T18:41:51.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:51.671+0000 7f451c800700 1 -- 172.21.15.33:0/2970543603 shutdown_connections 2024-05-03T18:41:51.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:51.671+0000 7f451c800700 1 --2- 172.21.15.33:0/2970543603 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f451410b6b0 0x7f45141116b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:51.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:51.671+0000 7f451c800700 1 --2- 172.21.15.33:0/2970543603 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f44fc0b2a70 0x7f44fc0b4f30 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:51.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:51.671+0000 7f451c800700 1 --2- 172.21.15.33:0/2970543603 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45141056b0 0x7f4514074b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:51.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:51.671+0000 7f451c800700 1 --2- 172.21.15.33:0/2970543603 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4514106070 0x7f4514075080 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:51.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:51.671+0000 7f451c800700 1 -- 172.21.15.33:0/2970543603 >> 172.21.15.33:0/2970543603 conn(0x7f45141011e0 msgr2=0x7f4514079af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:41:51.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:51.671+0000 7f451c800700 1 -- 172.21.15.33:0/2970543603 shutdown_connections 2024-05-03T18:41:51.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:51.672+0000 7f451c800700 1 -- 172.21.15.33:0/2970543603 wait complete. 2024-05-03T18:41:51.676 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 21 2024-05-03T18:41:52.348 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:41:52 smithi099 ceph-mon[31658]: pgmap v458: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 8.0 MiB/s wr, 680 op/s 2024-05-03T18:41:52.348 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:41:52 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/2970543603' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:41:52.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:41:52 smithi150 ceph-mon[32492]: pgmap v458: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 8.0 MiB/s wr, 680 op/s 2024-05-03T18:41:52.384 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:41:52 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/2970543603' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:41:52.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:41:52 smithi033 ceph-mon[28330]: pgmap v458: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 8.0 MiB/s wr, 680 op/s 2024-05-03T18:41:52.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:41:52 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/2970543603' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:41:54.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:41:54 smithi033 ceph-mon[28330]: pgmap v459: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 4.9 MiB/s wr, 439 op/s 2024-05-03T18:41:54.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:41:54 smithi150 ceph-mon[32492]: pgmap v459: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 4.9 MiB/s wr, 439 op/s 2024-05-03T18:41:54.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:41:54 smithi099 ceph-mon[31658]: pgmap v459: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 4.9 MiB/s wr, 439 op/s 2024-05-03T18:41:55.110 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 640.0 s, 410.4 tps, lat 247.657 ms stddev 274.181 2024-05-03T18:41:55.194 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-03T18:41:55.919 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:41:56.360 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:41:56 smithi033 ceph-mon[28330]: pgmap v460: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 6.7 MiB/s wr, 621 op/s 2024-05-03T18:41:56.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:41:56 smithi150 ceph-mon[32492]: pgmap v460: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 6.7 MiB/s wr, 621 op/s 2024-05-03T18:41:56.549 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:41:56 smithi099 ceph-mon[31658]: pgmap v460: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 6.7 MiB/s wr, 621 op/s 2024-05-03T18:41:58.326 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:41:58 smithi033 ceph-mon[28330]: pgmap v461: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 5.9 MiB/s wr, 522 op/s 2024-05-03T18:41:58.385 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:41:58 smithi150 ceph-mon[32492]: pgmap v461: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 5.9 MiB/s wr, 522 op/s 2024-05-03T18:41:58.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:41:58 smithi099 ceph-mon[31658]: pgmap v461: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 5.9 MiB/s wr, 522 op/s 2024-05-03T18:41:58.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.849+0000 7f915a200700 1 -- 172.21.15.33:0/1503908089 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9154109ad0 msgr2=0x7f9154109f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:41:58.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.850+0000 7f915a200700 1 --2- 172.21.15.33:0/1503908089 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9154109ad0 0x7f9154109f50 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9140009990 tx=0x7f9140009ca0 comp rx=0 tx=0).stop 2024-05-03T18:41:58.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.851+0000 7f915a200700 1 -- 172.21.15.33:0/1503908089 shutdown_connections 2024-05-03T18:41:58.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.851+0000 7f915a200700 1 --2- 172.21.15.33:0/1503908089 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f915410a490 0x7f91541169a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:58.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.851+0000 7f915a200700 1 --2- 172.21.15.33:0/1503908089 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f915410fb60 0x7f915410ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:58.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.851+0000 7f915a200700 1 --2- 172.21.15.33:0/1503908089 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9154109ad0 0x7f9154109f50 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:58.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.851+0000 7f915a200700 1 -- 172.21.15.33:0/1503908089 >> 172.21.15.33:0/1503908089 conn(0x7f9154076c70 msgr2=0x7f9154079090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:41:58.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.852+0000 7f915a200700 1 -- 172.21.15.33:0/1503908089 shutdown_connections 2024-05-03T18:41:58.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.852+0000 7f915a200700 1 -- 172.21.15.33:0/1503908089 wait complete. 2024-05-03T18:41:58.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.854+0000 7f915a200700 1 Processor -- start 2024-05-03T18:41:58.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.855+0000 7f915a200700 1 -- start start 2024-05-03T18:41:58.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.856+0000 7f915a200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9154109ad0 0x7f9154083b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:58.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.857+0000 7f915a200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f915410a490 0x7f915407cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:58.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.857+0000 7f915a200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f915410fb60 0x7f915407d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:58.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.857+0000 7f915a200700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9154119c10 con 0x7f915410fb60 2024-05-03T18:41:58.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.857+0000 7f915a200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9154119a90 con 0x7f915410a490 2024-05-03T18:41:58.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.857+0000 7f915a200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9154119d90 con 0x7f9154109ad0 2024-05-03T18:41:58.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.857+0000 7f9152a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f915410a490 0x7f915407cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:58.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.857+0000 7f9152a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f915410a490 0x7f915407cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:47982/0 (socket says 172.21.15.33:47982) 2024-05-03T18:41:58.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.857+0000 7f9152a00700 1 -- 172.21.15.33:0/1788446393 learned_addr learned my addr 172.21.15.33:0/1788446393 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:41:58.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.858+0000 7f9153400700 1 --2- 172.21.15.33:0/1788446393 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9154109ad0 0x7f9154083b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:58.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.858+0000 7f9153400700 1 -- 172.21.15.33:0/1788446393 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f915410a490 msgr2=0x7f915407cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:41:58.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.858+0000 7f9153400700 1 --2- 172.21.15.33:0/1788446393 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f915410a490 0x7f915407cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:58.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.858+0000 7f9153400700 1 -- 172.21.15.33:0/1788446393 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f915410fb60 msgr2=0x7f915407d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:41:58.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.858+0000 7f9153e00700 1 --2- 172.21.15.33:0/1788446393 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f915410fb60 0x7f915407d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:58.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.859+0000 7f9153400700 1 --2- 172.21.15.33:0/1788446393 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f915410fb60 0x7f915407d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:58.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.859+0000 7f9153400700 1 -- 172.21.15.33:0/1788446393 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f915407d7a0 con 0x7f9154109ad0 2024-05-03T18:41:58.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.859+0000 7f9153e00700 1 --2- 172.21.15.33:0/1788446393 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f915410fb60 0x7f915407d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:41:58.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.860+0000 7f9153400700 1 --2- 172.21.15.33:0/1788446393 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9154109ad0 0x7f9154083b10 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f914400e9f0 tx=0x7f914400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:41:58.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.860+0000 7f914be00700 1 -- 172.21.15.33:0/1788446393 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f914400cbd0 con 0x7f9154109ad0 2024-05-03T18:41:58.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.860+0000 7f914be00700 1 -- 172.21.15.33:0/1788446393 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f91440044b0 con 0x7f9154109ad0 2024-05-03T18:41:58.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.860+0000 7f914be00700 1 -- 172.21.15.33:0/1788446393 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9144005020 con 0x7f9154109ad0 2024-05-03T18:41:58.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.860+0000 7f915a200700 1 -- 172.21.15.33:0/1788446393 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9154081640 con 0x7f9154109ad0 2024-05-03T18:41:58.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.861+0000 7f915a200700 1 -- 172.21.15.33:0/1788446393 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9154081b30 con 0x7f9154109ad0 2024-05-03T18:41:58.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.867+0000 7f914be00700 1 -- 172.21.15.33:0/1788446393 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f91440051c0 con 0x7f9154109ad0 2024-05-03T18:41:58.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.869+0000 7f914be00700 1 --2- 172.21.15.33:0/1788446393 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f91380b2a70 0x7f91380b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:58.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.869+0000 7f914be00700 1 -- 172.21.15.33:0/1788446393 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9518+0+0 (secure 0 0 0) 0x7f91440e2990 con 0x7f9154109ad0 2024-05-03T18:41:58.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.870+0000 7f9152a00700 1 --2- 172.21.15.33:0/1788446393 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f91380b2a70 0x7f91380b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:58.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.871+0000 7f9152a00700 1 --2- 172.21.15.33:0/1788446393 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f91380b2a70 0x7f91380b4f30 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9140005b20 tx=0x7f9140005a90 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:41:58.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.901+0000 7f9164b4eb80 1 Processor -- start 2024-05-03T18:41:58.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.902+0000 7f9164b4eb80 1 -- start start 2024-05-03T18:41:58.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.902+0000 7f9164b4eb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a24ae7d7b0 0x55a24ae7db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:58.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.902+0000 7f9164b4eb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a24ae7e0d0 0x55a24ae78fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:58.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.902+0000 7f9164b4eb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a24ae795d0 0x55a24ae79a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:58.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.902+0000 7f9164b4eb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a24adcba20 con 0x55a24ae795d0 2024-05-03T18:41:58.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.902+0000 7f9164b4eb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55a24ad69760 con 0x55a24ae7e0d0 2024-05-03T18:41:58.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.902+0000 7f9164b4eb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55a24adc79e0 con 0x55a24ae7d7b0 2024-05-03T18:41:58.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.903+0000 7f9153400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a24ae7d7b0 0x55a24ae7db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:58.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.903+0000 7f9153400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a24ae7d7b0 0x55a24ae7db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:52582/0 (socket says 172.21.15.33:52582) 2024-05-03T18:41:58.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.903+0000 7f9153400700 1 -- 172.21.15.33:0/1644587955 learned_addr learned my addr 172.21.15.33:0/1644587955 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:41:58.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.903+0000 7f9153400700 1 -- 172.21.15.33:0/1644587955 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a24ae7e0d0 msgr2=0x55a24ae78fb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-03T18:41:58.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.903+0000 7f9153400700 1 --2- 172.21.15.33:0/1644587955 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a24ae7e0d0 0x55a24ae78fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:58.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.903+0000 7f9153400700 1 -- 172.21.15.33:0/1644587955 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a24ae795d0 msgr2=0x55a24ae79a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:41:58.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.903+0000 7f9153400700 1 --2- 172.21.15.33:0/1644587955 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a24ae795d0 0x55a24ae79a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:58.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.903+0000 7f9153400700 1 -- 172.21.15.33:0/1644587955 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a24ad188d0 con 0x55a24ae7d7b0 2024-05-03T18:41:58.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.904+0000 7f9153400700 1 --2- 172.21.15.33:0/1644587955 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a24ae7d7b0 0x55a24ae7db90 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f91440a97c0 tx=0x7f914400ce00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c05a0f5cafe81c08 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:41:58.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.904+0000 7f9148c00700 1 -- 172.21.15.33:0/1644587955 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f914401c070 con 0x55a24ae7d7b0 2024-05-03T18:41:58.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.905+0000 7f9148c00700 1 -- 172.21.15.33:0/1644587955 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f91440a9f30 con 0x55a24ae7d7b0 2024-05-03T18:41:58.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.906+0000 7f9164b4eb80 1 -- 172.21.15.33:0/1644587955 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a24ae7d7b0 msgr2=0x55a24ae7db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:41:58.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.906+0000 7f9148c00700 1 -- 172.21.15.33:0/1644587955 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91440a9a90 con 0x55a24ae7d7b0 2024-05-03T18:41:58.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.906+0000 7f9164b4eb80 1 --2- 172.21.15.33:0/1644587955 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a24ae7d7b0 0x55a24ae7db90 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f91440a97c0 tx=0x7f914400ce00 comp rx=0 tx=0).stop 2024-05-03T18:41:58.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.906+0000 7f9164b4eb80 1 -- 172.21.15.33:0/1644587955 shutdown_connections 2024-05-03T18:41:58.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.906+0000 7f9164b4eb80 1 --2- 172.21.15.33:0/1644587955 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a24ae795d0 0x55a24ae79a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:58.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.906+0000 7f9164b4eb80 1 --2- 172.21.15.33:0/1644587955 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a24ae7d7b0 0x55a24ae7db90 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:58.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.906+0000 7f9164b4eb80 1 --2- 172.21.15.33:0/1644587955 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a24ae7e0d0 0x55a24ae78fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:58.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.906+0000 7f9164b4eb80 1 -- 172.21.15.33:0/1644587955 >> 172.21.15.33:0/1644587955 conn(0x55a24ada7b60 msgr2=0x55a24ae737f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:41:58.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.906+0000 7f9164b4eb80 1 -- 172.21.15.33:0/1644587955 shutdown_connections 2024-05-03T18:41:58.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.909+0000 7f9164b4eb80 1 -- 172.21.15.33:0/1644587955 wait complete. 2024-05-03T18:41:58.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.910+0000 7f9164b4eb80 1 -- start start 2024-05-03T18:41:58.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.910+0000 7f9164b4eb80 1 Processor -- start 2024-05-03T18:41:58.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.912+0000 7f9164b4eb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a24ae795d0 0x55a24ae84e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:58.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.912+0000 7f9164b4eb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a24ae7d7b0 0x55a24ae82270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:58.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.912+0000 7f9164b4eb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a24ae7e0d0 0x55a24ae827b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:58.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.912+0000 7f9164b4eb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a24ae85ce0 con 0x55a24ae7d7b0 2024-05-03T18:41:58.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.912+0000 7f9164b4eb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55a24ad31b20 con 0x55a24ae795d0 2024-05-03T18:41:58.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.912+0000 7f9164b4eb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55a24ae85e60 con 0x55a24ae7e0d0 2024-05-03T18:41:58.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.912+0000 7f9164b4eb80 10 client.0 _ll_register_callbacks cb 0x55a24ad72c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:41:58.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.913+0000 7f9153e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a24ae7e0d0 0x55a24ae827b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:58.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.913+0000 7f9153400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a24ae795d0 0x55a24ae84e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:58.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.913+0000 7f9153e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a24ae7e0d0 0x55a24ae827b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:52592/0 (socket says 172.21.15.33:52592) 2024-05-03T18:41:58.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.913+0000 7f9153e00700 1 -- 172.21.15.33:0/749535883 learned_addr learned my addr 172.21.15.33:0/749535883 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:41:58.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.913+0000 7f9153e00700 1 -- 172.21.15.33:0/749535883 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a24ae795d0 msgr2=0x55a24ae84e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:41:58.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.913+0000 7f9153e00700 1 --2- 172.21.15.33:0/749535883 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a24ae795d0 0x55a24ae84e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:58.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.913+0000 7f9152a00700 1 --2- 172.21.15.33:0/749535883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a24ae7d7b0 0x55a24ae82270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:58.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.913+0000 7f9153e00700 1 -- 172.21.15.33:0/749535883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a24ae7d7b0 msgr2=0x55a24ae82270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:41:58.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.913+0000 7f9153e00700 1 --2- 172.21.15.33:0/749535883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a24ae7d7b0 0x55a24ae82270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:58.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.913+0000 7f9153e00700 1 -- 172.21.15.33:0/749535883 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a24ad91420 con 0x55a24ae7e0d0 2024-05-03T18:41:58.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.914+0000 7f9153e00700 1 --2- 172.21.15.33:0/749535883 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a24ae7e0d0 0x55a24ae827b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f914c00c690 tx=0x7f914c00ca50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=709be83854e7c596 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:41:58.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.914+0000 7f913d600700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-03T18:41:58.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.915+0000 7f913d600700 1 -- 172.21.15.33:0/749535883 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f914c00e910 con 0x55a24ae7e0d0 2024-05-03T18:41:58.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.915+0000 7f913d600700 1 -- 172.21.15.33:0/749535883 <== mon.1 v2:172.21.15.150:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9518+0+0 (secure 0 0 0) 0x7f914c00eab0 con 0x55a24ae7e0d0 2024-05-03T18:41:58.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.915+0000 7f9164b4eb80 1 -- 172.21.15.33:0/749535883 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91440a9f30 con 0x55a24ae7e0d0 2024-05-03T18:41:58.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.915+0000 7f913d600700 1 -- 172.21.15.33:0/749535883 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f914c00ecb0 con 0x55a24ae7e0d0 2024-05-03T18:41:58.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.916+0000 7f9164b4eb80 10 client.26536 fetch_fsmap learned FSMap version 21 2024-05-03T18:41:58.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.916+0000 7f9164b4eb80 1 -- 172.21.15.33:0/749535883 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55a24ae90590 con 0x55a24ae7e0d0 2024-05-03T18:41:58.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.916+0000 7f913d600700 1 -- 172.21.15.33:0/749535883 <== mon.1 v2:172.21.15.150:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f91440a9f30 con 0x55a24ae7e0d0 2024-05-03T18:41:58.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.917+0000 7f913d600700 1 -- 172.21.15.33:0/749535883 <== mon.1 v2:172.21.15.150:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f914c00d3d0 con 0x55a24ae7e0d0 2024-05-03T18:41:58.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.918+0000 7f9164b4eb80 10 client.26536 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:41:58.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.918+0000 7f9164b4eb80 10 client.26536 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:41:58.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.918+0000 7f9164b4eb80 20 client.26536 populate_metadata read hostname 'smithi033' 2024-05-03T18:41:58.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.918+0000 7f9164b4eb80 1 --2- 172.21.15.33:0/749535883 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55a24ae93020 0x55a24ae95460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:58.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.918+0000 7f9164b4eb80 4 client.26536 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:41:58.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.918+0000 7f9164b4eb80 1 -- 172.21.15.33:0/749535883 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a24ae461d0 con 0x55a24ae93020 2024-05-03T18:41:58.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.919+0000 7f9153400700 1 --2- 172.21.15.33:0/749535883 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55a24ae93020 0x55a24ae95460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:58.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.920+0000 7f9153400700 1 --2- 172.21.15.33:0/749535883 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55a24ae93020 0x55a24ae95460 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2eb69baf72748272 server_cookie=f2f21d691f0d06b0 in_seq=0 out_seq=0 2024-05-03T18:41:58.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.920+0000 7f913d600700 10 client.26536 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:41:58.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.922+0000 7f913d600700 1 -- 172.21.15.33:0/749535883 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a24ae461d0 con 0x55a24ae93020 2024-05-03T18:41:58.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.922+0000 7f913d600700 10 client.26536 handle_command_reply: tid=0 2024-05-03T18:41:58.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.924+0000 7f9164b4eb80 1 client.26536 shutdown 2024-05-03T18:41:58.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.925+0000 7f9164b4eb80 1 -- 172.21.15.33:0/749535883 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a24ae7e0d0 msgr2=0x55a24ae827b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:41:58.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.925+0000 7f9164b4eb80 1 --2- 172.21.15.33:0/749535883 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a24ae7e0d0 0x55a24ae827b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f914c00c690 tx=0x7f914c00ca50 comp rx=0 tx=0).stop 2024-05-03T18:41:58.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.926+0000 7f9164b4eb80 1 -- 172.21.15.33:0/749535883 shutdown_connections 2024-05-03T18:41:58.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.926+0000 7f9164b4eb80 1 --2- 172.21.15.33:0/749535883 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55a24ae93020 0x55a24ae95460 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:58.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.926+0000 7f9164b4eb80 1 --2- 172.21.15.33:0/749535883 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a24ae7e0d0 0x55a24ae827b0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:58.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.926+0000 7f9164b4eb80 1 --2- 172.21.15.33:0/749535883 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a24ae795d0 0x55a24ae84e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:58.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.926+0000 7f9164b4eb80 1 --2- 172.21.15.33:0/749535883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a24ae7d7b0 0x55a24ae82270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:58.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.926+0000 7f913d600700 0 client.26536 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:41:58.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.926+0000 7f9164b4eb80 1 -- 172.21.15.33:0/749535883 >> 172.21.15.33:0/749535883 conn(0x55a24ada7b60 msgr2=0x55a24ae72650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:41:58.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.927+0000 7f9164b4eb80 1 -- 172.21.15.33:0/749535883 shutdown_connections 2024-05-03T18:41:58.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.927+0000 7f9164b4eb80 1 -- 172.21.15.33:0/749535883 wait complete. 2024-05-03T18:41:58.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.928+0000 7f9164b4eb80 20 client.26536 trim_cache size 0 max 16384 2024-05-03T18:41:58.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.957+0000 7f9164b4eb80 1 Processor -- start 2024-05-03T18:41:58.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.957+0000 7f9164b4eb80 1 -- start start 2024-05-03T18:41:58.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.957+0000 7f9164b4eb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a24ada7b60 0x55a24ae73660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:58.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.957+0000 7f9164b4eb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a24ae795d0 0x55a24ae80b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:58.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.958+0000 7f9164b4eb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a24ae7d7b0 0x55a24ae810a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:58.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.958+0000 7f9164b4eb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f91541bc460 con 0x55a24ada7b60 2024-05-03T18:41:58.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.958+0000 7f9164b4eb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f91541becc0 con 0x55a24ae7d7b0 2024-05-03T18:41:58.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.958+0000 7f9164b4eb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f91541bcab0 con 0x55a24ae795d0 2024-05-03T18:41:58.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.958+0000 7f9153400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a24ada7b60 0x55a24ae73660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:58.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.958+0000 7f9152a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a24ae795d0 0x55a24ae80b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:58.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.958+0000 7f9152a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a24ae795d0 0x55a24ae80b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:52594/0 (socket says 172.21.15.33:52594) 2024-05-03T18:41:58.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.958+0000 7f9152a00700 1 -- 172.21.15.33:0/1749985513 learned_addr learned my addr 172.21.15.33:0/1749985513 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:41:58.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.959+0000 7f9153e00700 1 --2- 172.21.15.33:0/1749985513 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a24ae7d7b0 0x55a24ae810a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:58.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.959+0000 7f9152a00700 1 -- 172.21.15.33:0/1749985513 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a24ae7d7b0 msgr2=0x55a24ae810a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:41:58.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.959+0000 7f9152a00700 1 --2- 172.21.15.33:0/1749985513 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a24ae7d7b0 0x55a24ae810a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:58.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.959+0000 7f9152a00700 1 -- 172.21.15.33:0/1749985513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a24ada7b60 msgr2=0x55a24ae73660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:41:58.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.959+0000 7f9152a00700 1 --2- 172.21.15.33:0/1749985513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a24ada7b60 0x55a24ae73660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:58.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.959+0000 7f9152a00700 1 -- 172.21.15.33:0/1749985513 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a24ae7d400 con 0x55a24ae795d0 2024-05-03T18:41:58.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.960+0000 7f9152a00700 1 --2- 172.21.15.33:0/1749985513 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a24ae795d0 0x55a24ae80b60 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f914000b2e0 tx=0x7f914001f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ade2d1dbd4f6fd26 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:41:58.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.960+0000 7f913fe00700 1 -- 172.21.15.33:0/1749985513 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9140003680 con 0x55a24ae795d0 2024-05-03T18:41:58.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.961+0000 7f913fe00700 1 -- 172.21.15.33:0/1749985513 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f914002b020 con 0x55a24ae795d0 2024-05-03T18:41:58.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.962+0000 7f9164b4eb80 1 -- 172.21.15.33:0/1749985513 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a24ae795d0 msgr2=0x55a24ae80b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:41:58.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.962+0000 7f9164b4eb80 1 --2- 172.21.15.33:0/1749985513 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a24ae795d0 0x55a24ae80b60 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f914000b2e0 tx=0x7f914001f000 comp rx=0 tx=0).stop 2024-05-03T18:41:58.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.963+0000 7f9164b4eb80 1 -- 172.21.15.33:0/1749985513 shutdown_connections 2024-05-03T18:41:58.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.963+0000 7f9164b4eb80 1 --2- 172.21.15.33:0/1749985513 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a24ae7d7b0 0x55a24ae810a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:58.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.964+0000 7f9164b4eb80 1 --2- 172.21.15.33:0/1749985513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a24ada7b60 0x55a24ae73660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:58.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.964+0000 7f9164b4eb80 1 --2- 172.21.15.33:0/1749985513 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a24ae795d0 0x55a24ae80b60 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:58.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.964+0000 7f9164b4eb80 1 -- 172.21.15.33:0/1749985513 >> 172.21.15.33:0/1749985513 conn(0x55a24ae83160 msgr2=0x55a24ae82230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:41:58.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.965+0000 7f9164b4eb80 1 -- 172.21.15.33:0/1749985513 shutdown_connections 2024-05-03T18:41:58.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.965+0000 7f9164b4eb80 1 -- 172.21.15.33:0/1749985513 wait complete. 2024-05-03T18:41:58.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.966+0000 7f9164b4eb80 1 -- start start 2024-05-03T18:41:58.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.967+0000 7f9164b4eb80 1 Processor -- start 2024-05-03T18:41:58.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.967+0000 7f9164b4eb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a24ada7b60 0x55a24ae7f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:58.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.968+0000 7f9164b4eb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a24ae795d0 0x55a24ae7f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:58.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.968+0000 7f9164b4eb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a24ae7d7b0 0x55a24ae7fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:58.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.968+0000 7f9164b4eb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f91541bd180 con 0x55a24ae795d0 2024-05-03T18:41:58.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.968+0000 7f9164b4eb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f91541be5f0 con 0x55a24ada7b60 2024-05-03T18:41:58.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.968+0000 7f9164b4eb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f91541bd850 con 0x55a24ae7d7b0 2024-05-03T18:41:58.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.968+0000 7f9164b4eb80 10 client.0 _ll_register_callbacks cb 0x55a24ad72c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:41:58.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.968+0000 7f9153400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a24ada7b60 0x55a24ae7f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:58.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.968+0000 7f9153400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a24ada7b60 0x55a24ae7f2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:48018/0 (socket says 172.21.15.33:48018) 2024-05-03T18:41:58.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.968+0000 7f9153400700 1 -- 172.21.15.33:0/4028238799 learned_addr learned my addr 172.21.15.33:0/4028238799 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:41:58.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.969+0000 7f9153e00700 1 --2- 172.21.15.33:0/4028238799 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a24ae7d7b0 0x55a24ae7fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:58.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.969+0000 7f9153400700 1 -- 172.21.15.33:0/4028238799 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a24ae7d7b0 msgr2=0x55a24ae7fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:41:58.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.969+0000 7f9153400700 1 --2- 172.21.15.33:0/4028238799 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a24ae7d7b0 0x55a24ae7fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:58.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.969+0000 7f9153400700 1 -- 172.21.15.33:0/4028238799 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a24ae795d0 msgr2=0x55a24ae7f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:41:58.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.969+0000 7f9153400700 1 --2- 172.21.15.33:0/4028238799 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a24ae795d0 0x55a24ae7f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:58.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.970+0000 7f9153400700 1 -- 172.21.15.33:0/4028238799 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a24aefcb20 con 0x55a24ada7b60 2024-05-03T18:41:58.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.971+0000 7f9153400700 1 --2- 172.21.15.33:0/4028238799 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a24ada7b60 0x55a24ae7f2e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f91440a9770 tx=0x7f91440a9cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d9b615851bb2f93 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:41:58.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.971+0000 7f913d600700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-03T18:41:58.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.971+0000 7f913d600700 1 -- 172.21.15.33:0/4028238799 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f914401c040 con 0x55a24ada7b60 2024-05-03T18:41:58.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.971+0000 7f9164b4eb80 1 -- 172.21.15.33:0/4028238799 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f914002b020 con 0x55a24ada7b60 2024-05-03T18:41:58.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.971+0000 7f913d600700 1 -- 172.21.15.33:0/4028238799 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9518+0+0 (secure 0 0 0) 0x7f91440056c0 con 0x55a24ada7b60 2024-05-03T18:41:58.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.972+0000 7f913d600700 1 -- 172.21.15.33:0/4028238799 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f91440a9f30 con 0x55a24ada7b60 2024-05-03T18:41:58.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.972+0000 7f9164b4eb80 10 client.36467 fetch_fsmap learned FSMap version 21 2024-05-03T18:41:58.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.972+0000 7f9164b4eb80 1 -- 172.21.15.33:0/4028238799 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55a24aefdde0 con 0x55a24ada7b60 2024-05-03T18:41:58.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.973+0000 7f913d600700 1 -- 172.21.15.33:0/4028238799 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f914002b020 con 0x55a24ada7b60 2024-05-03T18:41:58.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.973+0000 7f913d600700 1 -- 172.21.15.33:0/4028238799 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f91440a8a90 con 0x55a24ada7b60 2024-05-03T18:41:58.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.973+0000 7f9164b4eb80 10 client.36467 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:41:58.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.973+0000 7f9164b4eb80 10 client.36467 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:41:58.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.973+0000 7f9164b4eb80 20 client.36467 populate_metadata read hostname 'smithi033' 2024-05-03T18:41:58.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.973+0000 7f9164b4eb80 1 --2- 172.21.15.33:0/4028238799 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55a24ae7e0d0 0x55a24af00020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:58.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.975+0000 7f9152a00700 1 --2- 172.21.15.33:0/4028238799 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55a24ae7e0d0 0x55a24af00020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:58.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.976+0000 7f9164b4eb80 4 client.36467 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-03T18:41:58.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.976+0000 7f9164b4eb80 1 -- 172.21.15.33:0/4028238799 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a24add9400 con 0x55a24ae7e0d0 2024-05-03T18:41:58.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.979+0000 7f913d600700 10 client.36467 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:41:58.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.978+0000 7f9152a00700 1 --2- 172.21.15.33:0/4028238799 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55a24ae7e0d0 0x55a24af00020 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94ca095be5332cc2 server_cookie=cc90e4efc70c1e0d in_seq=0 out_seq=0 2024-05-03T18:41:58.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.979+0000 7f913d600700 1 -- 172.21.15.33:0/4028238799 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a24add9400 con 0x55a24ae7e0d0 2024-05-03T18:41:58.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.980+0000 7f913d600700 10 client.36467 handle_command_reply: tid=0 2024-05-03T18:41:58.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.980+0000 7f9164b4eb80 1 client.36467 shutdown 2024-05-03T18:41:58.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.980+0000 7f9164b4eb80 1 -- 172.21.15.33:0/4028238799 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a24ada7b60 msgr2=0x55a24ae7f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:41:58.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.980+0000 7f9164b4eb80 1 --2- 172.21.15.33:0/4028238799 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a24ada7b60 0x55a24ae7f2e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f91440a9770 tx=0x7f91440a9cc0 comp rx=0 tx=0).stop 2024-05-03T18:41:58.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.981+0000 7f9164b4eb80 1 -- 172.21.15.33:0/4028238799 shutdown_connections 2024-05-03T18:41:58.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.981+0000 7f9164b4eb80 1 --2- 172.21.15.33:0/4028238799 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55a24ae7e0d0 0x55a24af00020 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:58.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.981+0000 7f9164b4eb80 1 --2- 172.21.15.33:0/4028238799 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a24ae7d7b0 0x55a24ae7fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:58.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.981+0000 7f9164b4eb80 1 --2- 172.21.15.33:0/4028238799 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a24ada7b60 0x55a24ae7f2e0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:58.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.981+0000 7f9164b4eb80 1 --2- 172.21.15.33:0/4028238799 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a24ae795d0 0x55a24ae7f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:58.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.981+0000 7f9164b4eb80 1 -- 172.21.15.33:0/4028238799 >> 172.21.15.33:0/4028238799 conn(0x55a24ae83160 msgr2=0x55a24ae82190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:41:58.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.982+0000 7f913d600700 0 client.36467 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:41:58.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.982+0000 7f9164b4eb80 1 -- 172.21.15.33:0/4028238799 shutdown_connections 2024-05-03T18:41:58.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.982+0000 7f9164b4eb80 1 -- 172.21.15.33:0/4028238799 wait complete. 2024-05-03T18:41:58.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.982+0000 7f9164b4eb80 20 client.36467 trim_cache size 0 max 16384 2024-05-03T18:41:58.983 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-03T18:41:58.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.983+0000 7f915a200700 1 -- 172.21.15.33:0/1788446393 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f91380b2a70 msgr2=0x7f91380b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:41:58.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.983+0000 7f915a200700 1 --2- 172.21.15.33:0/1788446393 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f91380b2a70 0x7f91380b4f30 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9140005b20 tx=0x7f9140005a90 comp rx=0 tx=0).stop 2024-05-03T18:41:58.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.983+0000 7f915a200700 1 -- 172.21.15.33:0/1788446393 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9154109ad0 msgr2=0x7f9154083b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:41:58.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.983+0000 7f915a200700 1 --2- 172.21.15.33:0/1788446393 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9154109ad0 0x7f9154083b10 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f914400e9f0 tx=0x7f914400ed00 comp rx=0 tx=0).stop 2024-05-03T18:41:58.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.984+0000 7f915a200700 1 -- 172.21.15.33:0/1788446393 shutdown_connections 2024-05-03T18:41:58.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.984+0000 7f915a200700 1 --2- 172.21.15.33:0/1788446393 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f915410fb60 0x7f915407d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:58.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.984+0000 7f915a200700 1 --2- 172.21.15.33:0/1788446393 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9154109ad0 0x7f9154083b10 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:58.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.984+0000 7f915a200700 1 --2- 172.21.15.33:0/1788446393 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f91380b2a70 0x7f91380b4f30 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:58.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.984+0000 7f915a200700 1 --2- 172.21.15.33:0/1788446393 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f915410a490 0x7f915407cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:58.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.984+0000 7f915a200700 1 -- 172.21.15.33:0/1788446393 >> 172.21.15.33:0/1788446393 conn(0x7f9154076c70 msgr2=0x7f9154077490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:41:58.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.984+0000 7f915a200700 1 -- 172.21.15.33:0/1788446393 shutdown_connections 2024-05-03T18:41:58.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:41:58.984+0000 7f915a200700 1 -- 172.21.15.33:0/1788446393 wait complete. 2024-05-03T18:42:00.110 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 645.0 s, 570.0 tps, lat 176.305 ms stddev 78.083 2024-05-03T18:42:00.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:42:00 smithi150 ceph-mon[32492]: pgmap v462: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 5.5 MiB/s wr, 472 op/s 2024-05-03T18:42:00.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:42:00 smithi033 ceph-mon[28330]: pgmap v462: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 5.5 MiB/s wr, 472 op/s 2024-05-03T18:42:00.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:42:00 smithi099 ceph-mon[31658]: pgmap v462: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 5.5 MiB/s wr, 472 op/s 2024-05-03T18:42:00.871 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-03T18:42:00.872 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-03T18:42:01.539 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:42:02.378 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:42:02 smithi033 ceph-mon[28330]: pgmap v463: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 7.8 MiB/s wr, 673 op/s 2024-05-03T18:42:02.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:42:02 smithi150 ceph-mon[32492]: pgmap v463: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 7.8 MiB/s wr, 673 op/s 2024-05-03T18:42:02.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:42:02 smithi099 ceph-mon[31658]: pgmap v463: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 7.8 MiB/s wr, 673 op/s 2024-05-03T18:42:04.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:42:04 smithi150 ceph-mon[32492]: pgmap v464: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 5.4 MiB/s wr, 464 op/s 2024-05-03T18:42:04.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:42:04 smithi033 ceph-mon[28330]: pgmap v464: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 5.4 MiB/s wr, 464 op/s 2024-05-03T18:42:04.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:42:04 smithi099 ceph-mon[31658]: pgmap v464: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 5.4 MiB/s wr, 464 op/s 2024-05-03T18:42:05.111 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 650.0 s, 601.6 tps, lat 165.084 ms stddev 104.682 2024-05-03T18:42:06.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:42:06 smithi033 ceph-mon[28330]: pgmap v465: 129 pgs: 129 active+clean; 3.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 7.7 MiB/s wr, 659 op/s 2024-05-03T18:42:06.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:42:06 smithi099 ceph-mon[31658]: pgmap v465: 129 pgs: 129 active+clean; 3.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 7.7 MiB/s wr, 659 op/s 2024-05-03T18:42:06.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:42:06 smithi150 ceph-mon[32492]: pgmap v465: 129 pgs: 129 active+clean; 3.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 7.7 MiB/s wr, 659 op/s 2024-05-03T18:42:08.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:42:08 smithi033 ceph-mon[28330]: pgmap v466: 129 pgs: 129 active+clean; 3.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 6.4 MiB/s wr, 552 op/s 2024-05-03T18:42:08.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:42:08 smithi099 ceph-mon[31658]: pgmap v466: 129 pgs: 129 active+clean; 3.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 6.4 MiB/s wr, 552 op/s 2024-05-03T18:42:08.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.542+0000 7fc2ffe00700 1 -- 172.21.15.33:0/1032169800 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc3001086f0 msgr2=0x7fc3001102c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:42:08.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.542+0000 7fc2ffe00700 1 --2- 172.21.15.33:0/1032169800 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc3001086f0 0x7fc3001102c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc2f800b0a0 tx=0x7fc2f800b3b0 comp rx=0 tx=0).stop 2024-05-03T18:42:08.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.542+0000 7fc2ffe00700 1 -- 172.21.15.33:0/1032169800 shutdown_connections 2024-05-03T18:42:08.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.542+0000 7fc2ffe00700 1 --2- 172.21.15.33:0/1032169800 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc3001086f0 0x7fc3001102c0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:08.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.542+0000 7fc2ffe00700 1 --2- 172.21.15.33:0/1032169800 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc300103610 0x7fc3001039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:08.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.542+0000 7fc2ffe00700 1 --2- 172.21.15.33:0/1032169800 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc300103fc0 0x7fc300108010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:08.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.542+0000 7fc2ffe00700 1 -- 172.21.15.33:0/1032169800 >> 172.21.15.33:0/1032169800 conn(0x7fc3000feec0 msgr2=0x7fc3001012e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:42:08.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.542+0000 7fc2ffe00700 1 -- 172.21.15.33:0/1032169800 shutdown_connections 2024-05-03T18:42:08.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.543+0000 7fc2ffe00700 1 -- 172.21.15.33:0/1032169800 wait complete. 2024-05-03T18:42:08.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.544+0000 7fc2ffe00700 1 Processor -- start 2024-05-03T18:42:08.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.544+0000 7fc2ffe00700 1 -- start start 2024-05-03T18:42:08.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.545+0000 7fc2ffe00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc300103610 0x7fc3001130c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:08.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.545+0000 7fc2ffe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc300103fc0 0x7fc300113600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:08.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.545+0000 7fc2ffe00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc3001086f0 0x7fc3001161f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:08.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.545+0000 7fc2ffe00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc300075ce0 con 0x7fc300103610 2024-05-03T18:42:08.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.545+0000 7fc2ffe00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc300075b60 con 0x7fc300103fc0 2024-05-03T18:42:08.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.545+0000 7fc2ffe00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc300075e60 con 0x7fc3001086f0 2024-05-03T18:42:08.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.546+0000 7fc2ff400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc3001086f0 0x7fc3001161f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:42:08.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.546+0000 7fc2ff400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc3001086f0 0x7fc3001161f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:46064/0 (socket says 172.21.15.33:46064) 2024-05-03T18:42:08.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.546+0000 7fc2ff400700 1 -- 172.21.15.33:0/3148455465 learned_addr learned my addr 172.21.15.33:0/3148455465 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:42:08.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.547+0000 7fc2ff400700 1 -- 172.21.15.33:0/3148455465 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc300103fc0 msgr2=0x7fc300113600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-03T18:42:08.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.547+0000 7fc2ff400700 1 --2- 172.21.15.33:0/3148455465 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc300103fc0 0x7fc300113600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:08.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.547+0000 7fc2ff400700 1 -- 172.21.15.33:0/3148455465 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc300103610 msgr2=0x7fc3001130c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:42:08.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.547+0000 7fc2ff400700 1 --2- 172.21.15.33:0/3148455465 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc300103610 0x7fc3001130c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:08.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.547+0000 7fc2ff400700 1 -- 172.21.15.33:0/3148455465 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc300116900 con 0x7fc3001086f0 2024-05-03T18:42:08.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.548+0000 7fc2ff400700 1 --2- 172.21.15.33:0/3148455465 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc3001086f0 0x7fc3001161f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8006070 tx=0x7fc2f8007770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:42:08.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.549+0000 7fc304a00700 1 -- 172.21.15.33:0/3148455465 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f8021070 con 0x7fc3001086f0 2024-05-03T18:42:08.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.549+0000 7fc304a00700 1 -- 172.21.15.33:0/3148455465 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc2f8007de0 con 0x7fc3001086f0 2024-05-03T18:42:08.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.549+0000 7fc304a00700 1 -- 172.21.15.33:0/3148455465 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f801d450 con 0x7fc3001086f0 2024-05-03T18:42:08.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.549+0000 7fc2ffe00700 1 -- 172.21.15.33:0/3148455465 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc300116b90 con 0x7fc3001086f0 2024-05-03T18:42:08.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.549+0000 7fc2ffe00700 1 -- 172.21.15.33:0/3148455465 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc30019c010 con 0x7fc3001086f0 2024-05-03T18:42:08.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.556+0000 7fc304a00700 1 -- 172.21.15.33:0/3148455465 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fc2f8004140 con 0x7fc3001086f0 2024-05-03T18:42:08.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.558+0000 7fc304a00700 1 --2- 172.21.15.33:0/3148455465 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fc2e40b2a70 0x7fc2e40b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:08.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.559+0000 7fc304a00700 1 -- 172.21.15.33:0/3148455465 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9518+0+0 (secure 0 0 0) 0x7fc2f80e3d90 con 0x7fc3001086f0 2024-05-03T18:42:08.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.559+0000 7fc2fea00700 1 --2- 172.21.15.33:0/3148455465 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fc2e40b2a70 0x7fc2e40b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:42:08.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.604+0000 7fc2fea00700 1 --2- 172.21.15.33:0/3148455465 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fc2e40b2a70 0x7fc2e40b4f30 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc300110290 tx=0x7fc2f0006d20 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:42:08.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.606+0000 7fc3107acb80 1 Processor -- start 2024-05-03T18:42:08.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.607+0000 7fc3107acb80 1 -- start start 2024-05-03T18:42:08.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.607+0000 7fc3107acb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5643b974fc50 0x5643b9750030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:08.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.608+0000 7fc3107acb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5643b9749fb0 0x5643b974a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:08.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.608+0000 7fc3107acb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5643b974a970 0x5643b9756a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:08.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.608+0000 7fc3107acb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5643b969caa0 con 0x5643b974fc50 2024-05-03T18:42:08.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.608+0000 7fc3107acb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5643b963a7e0 con 0x5643b9749fb0 2024-05-03T18:42:08.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.608+0000 7fc3107acb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5643b9698a60 con 0x5643b974a970 2024-05-03T18:42:08.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.608+0000 7fc2fe000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5643b9749fb0 0x5643b974a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:42:08.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.608+0000 7fc2fe000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5643b9749fb0 0x5643b974a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:41800/0 (socket says 172.21.15.33:41800) 2024-05-03T18:42:08.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.608+0000 7fc2fe000700 1 -- 172.21.15.33:0/669059824 learned_addr learned my addr 172.21.15.33:0/669059824 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:42:08.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.608+0000 7fc2fea00700 1 --2- 172.21.15.33:0/669059824 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5643b974fc50 0x5643b9750030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:42:08.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.609+0000 7fc2fea00700 1 -- 172.21.15.33:0/669059824 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5643b974a970 msgr2=0x5643b9756a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:42:08.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.609+0000 7fc2fea00700 1 --2- 172.21.15.33:0/669059824 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5643b974a970 0x5643b9756a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:08.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.609+0000 7fc2fea00700 1 -- 172.21.15.33:0/669059824 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5643b9749fb0 msgr2=0x5643b974a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:42:08.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.609+0000 7fc2fea00700 1 --2- 172.21.15.33:0/669059824 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5643b9749fb0 0x5643b974a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:08.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.609+0000 7fc2fea00700 1 -- 172.21.15.33:0/669059824 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643b95e9950 con 0x5643b974fc50 2024-05-03T18:42:08.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.610+0000 7fc2fea00700 1 --2- 172.21.15.33:0/669059824 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5643b974fc50 0x5643b9750030 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fc2f0015840 tx=0x7fc2f0006f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70ab970c5fe6caee server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:42:08.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.610+0000 7fc2f4c00700 1 -- 172.21.15.33:0/669059824 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f0023070 con 0x5643b974fc50 2024-05-03T18:42:08.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.610+0000 7fc2f4c00700 1 -- 172.21.15.33:0/669059824 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc2f00152e0 con 0x5643b974fc50 2024-05-03T18:42:08.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.610+0000 7fc2f4c00700 1 -- 172.21.15.33:0/669059824 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f001e790 con 0x5643b974fc50 2024-05-03T18:42:08.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.612+0000 7fc3107acb80 1 -- 172.21.15.33:0/669059824 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5643b974fc50 msgr2=0x5643b9750030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:42:08.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.612+0000 7fc3107acb80 1 --2- 172.21.15.33:0/669059824 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5643b974fc50 0x5643b9750030 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fc2f0015840 tx=0x7fc2f0006f90 comp rx=0 tx=0).stop 2024-05-03T18:42:08.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.613+0000 7fc3107acb80 1 -- 172.21.15.33:0/669059824 shutdown_connections 2024-05-03T18:42:08.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.613+0000 7fc3107acb80 1 --2- 172.21.15.33:0/669059824 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5643b974a970 0x5643b9756a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:08.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.613+0000 7fc3107acb80 1 --2- 172.21.15.33:0/669059824 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5643b974fc50 0x5643b9750030 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:08.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.613+0000 7fc3107acb80 1 --2- 172.21.15.33:0/669059824 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5643b9749fb0 0x5643b974a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:08.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.613+0000 7fc3107acb80 1 -- 172.21.15.33:0/669059824 >> 172.21.15.33:0/669059824 conn(0x5643b96c2750 msgr2=0x5643b9744c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:42:08.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.613+0000 7fc3107acb80 1 -- 172.21.15.33:0/669059824 shutdown_connections 2024-05-03T18:42:08.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.613+0000 7fc3107acb80 1 -- 172.21.15.33:0/669059824 wait complete. 2024-05-03T18:42:08.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.614+0000 7fc3107acb80 1 -- start start 2024-05-03T18:42:08.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.615+0000 7fc3107acb80 1 Processor -- start 2024-05-03T18:42:08.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.615+0000 7fc3107acb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5643b9749fb0 0x5643b9756700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:08.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.615+0000 7fc3107acb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5643b974a970 0x5643b9753aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:08.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.615+0000 7fc3107acb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5643b974fc50 0x5643b9753fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:08.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.615+0000 7fc3107acb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5643b9756fb0 con 0x5643b974a970 2024-05-03T18:42:08.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.615+0000 7fc3107acb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5643b9602ba0 con 0x5643b974fc50 2024-05-03T18:42:08.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.615+0000 7fc3107acb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5643b9757130 con 0x5643b9749fb0 2024-05-03T18:42:08.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.615+0000 7fc3107acb80 10 client.0 _ll_register_callbacks cb 0x5643b9643cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:42:08.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.616+0000 7fc2fea00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5643b9749fb0 0x5643b9756700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:42:08.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.616+0000 7fc2fea00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5643b9749fb0 0x5643b9756700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:46078/0 (socket says 172.21.15.33:46078) 2024-05-03T18:42:08.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.616+0000 7fc2fea00700 1 -- 172.21.15.33:0/650253152 learned_addr learned my addr 172.21.15.33:0/650253152 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:42:08.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.617+0000 7fc2ff400700 1 --2- 172.21.15.33:0/650253152 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5643b974fc50 0x5643b9753fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:42:08.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.617+0000 7fc2fe000700 1 --2- 172.21.15.33:0/650253152 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5643b974a970 0x5643b9753aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:42:08.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.618+0000 7fc2ff400700 1 -- 172.21.15.33:0/650253152 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5643b9749fb0 msgr2=0x5643b9756700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:42:08.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.618+0000 7fc2ff400700 1 --2- 172.21.15.33:0/650253152 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5643b9749fb0 0x5643b9756700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:08.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.618+0000 7fc2ff400700 1 -- 172.21.15.33:0/650253152 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5643b974a970 msgr2=0x5643b9753aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:42:08.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.618+0000 7fc2ff400700 1 --2- 172.21.15.33:0/650253152 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5643b974a970 0x5643b9753aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:08.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.618+0000 7fc2ff400700 1 -- 172.21.15.33:0/650253152 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643b96624a0 con 0x5643b974fc50 2024-05-03T18:42:08.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.619+0000 7fc2ff400700 1 --2- 172.21.15.33:0/650253152 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5643b974fc50 0x5643b9753fe0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc2f80e10a0 tx=0x7fc2f80a7200 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4722eda4efb57043 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:42:08.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.619+0000 7fc2e9600700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-03T18:42:08.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.619+0000 7fc2e9600700 1 -- 172.21.15.33:0/650253152 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f802a070 con 0x5643b974fc50 2024-05-03T18:42:08.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.620+0000 7fc2e9600700 1 -- 172.21.15.33:0/650253152 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9518+0+0 (secure 0 0 0) 0x7fc2f80b3600 con 0x5643b974fc50 2024-05-03T18:42:08.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.620+0000 7fc3107acb80 1 -- 172.21.15.33:0/650253152 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2f00152e0 con 0x5643b974fc50 2024-05-03T18:42:08.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.620+0000 7fc2e9600700 1 -- 172.21.15.33:0/650253152 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fc2f80ab3c0 con 0x5643b974fc50 2024-05-03T18:42:08.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.620+0000 7fc3107acb80 10 client.36476 fetch_fsmap learned FSMap version 21 2024-05-03T18:42:08.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.621+0000 7fc3107acb80 1 -- 172.21.15.33:0/650253152 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5643b9761850 con 0x5643b974fc50 2024-05-03T18:42:08.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.621+0000 7fc2e9600700 1 -- 172.21.15.33:0/650253152 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc2f8009cb0 con 0x5643b974fc50 2024-05-03T18:42:08.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.621+0000 7fc2e9600700 1 -- 172.21.15.33:0/650253152 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fc2f8011070 con 0x5643b974fc50 2024-05-03T18:42:08.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.622+0000 7fc3107acb80 10 client.36476 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:42:08.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.622+0000 7fc3107acb80 10 client.36476 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:42:08.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.622+0000 7fc3107acb80 20 client.36476 populate_metadata read hostname 'smithi033' 2024-05-03T18:42:08.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.622+0000 7fc3107acb80 1 --2- 172.21.15.33:0/650253152 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5643b9763e60 0x5643b9766280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:08.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.622+0000 7fc3107acb80 4 client.36476 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:42:08.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.622+0000 7fc3107acb80 1 -- 172.21.15.33:0/650253152 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fc2f00152e0 con 0x5643b9763e60 2024-05-03T18:42:08.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.622+0000 7fc2fea00700 1 --2- 172.21.15.33:0/650253152 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5643b9763e60 0x5643b9766280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:42:08.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.623+0000 7fc2e9600700 10 client.36476 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:42:08.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.626+0000 7fc2fea00700 1 --2- 172.21.15.33:0/650253152 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5643b9763e60 0x5643b9766280 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49774d2648f810f0 server_cookie=f6001056a464caf9 in_seq=0 out_seq=0 2024-05-03T18:42:08.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.627+0000 7fc2e9600700 1 -- 172.21.15.33:0/650253152 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7fc2f00152e0 con 0x5643b9763e60 2024-05-03T18:42:08.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.627+0000 7fc2e9600700 10 client.36476 handle_command_reply: tid=0 2024-05-03T18:42:08.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.628+0000 7fc3107acb80 1 client.36476 shutdown 2024-05-03T18:42:08.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.628+0000 7fc3107acb80 1 -- 172.21.15.33:0/650253152 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5643b974fc50 msgr2=0x5643b9753fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:42:08.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.628+0000 7fc3107acb80 1 --2- 172.21.15.33:0/650253152 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5643b974fc50 0x5643b9753fe0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc2f80e10a0 tx=0x7fc2f80a7200 comp rx=0 tx=0).stop 2024-05-03T18:42:08.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.628+0000 7fc3107acb80 1 -- 172.21.15.33:0/650253152 shutdown_connections 2024-05-03T18:42:08.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.628+0000 7fc3107acb80 1 --2- 172.21.15.33:0/650253152 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5643b9763e60 0x5643b9766280 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:08.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.628+0000 7fc3107acb80 1 --2- 172.21.15.33:0/650253152 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5643b974fc50 0x5643b9753fe0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:08.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.628+0000 7fc3107acb80 1 --2- 172.21.15.33:0/650253152 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5643b9749fb0 0x5643b9756700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:08.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.628+0000 7fc3107acb80 1 --2- 172.21.15.33:0/650253152 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5643b974a970 0x5643b9753aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:08.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.628+0000 7fc3107acb80 1 -- 172.21.15.33:0/650253152 >> 172.21.15.33:0/650253152 conn(0x5643b96c2750 msgr2=0x5643b9748fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:42:08.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.628+0000 7fc2e9600700 0 client.36476 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:42:08.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.629+0000 7fc3107acb80 1 -- 172.21.15.33:0/650253152 shutdown_connections 2024-05-03T18:42:08.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.629+0000 7fc3107acb80 1 -- 172.21.15.33:0/650253152 wait complete. 2024-05-03T18:42:08.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.629+0000 7fc3107acb80 20 client.36476 trim_cache size 0 max 16384 2024-05-03T18:42:08.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:42:08 smithi150 ceph-mon[32492]: pgmap v466: 129 pgs: 129 active+clean; 3.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 6.4 MiB/s wr, 552 op/s 2024-05-03T18:42:08.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.670+0000 7fc3107acb80 1 Processor -- start 2024-05-03T18:42:08.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.671+0000 7fc3107acb80 1 -- start start 2024-05-03T18:42:08.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.671+0000 7fc3107acb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5643b9749fb0 0x5643b9752a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:08.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.671+0000 7fc3107acb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5643b974a970 0x5643b9749830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:08.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.671+0000 7fc3107acb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5643b974fc50 0x5643b9760000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:08.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.671+0000 7fc3107acb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc3001a2df0 con 0x5643b9749fb0 2024-05-03T18:42:08.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.671+0000 7fc3107acb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc3001be740 con 0x5643b974a970 2024-05-03T18:42:08.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.671+0000 7fc3107acb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc3001a3440 con 0x5643b974fc50 2024-05-03T18:42:08.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.672+0000 7fc2ff400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5643b974fc50 0x5643b9760000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:42:08.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.673+0000 7fc2ff400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5643b974fc50 0x5643b9760000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:46086/0 (socket says 172.21.15.33:46086) 2024-05-03T18:42:08.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.673+0000 7fc2ff400700 1 -- 172.21.15.33:0/3441185194 learned_addr learned my addr 172.21.15.33:0/3441185194 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:42:08.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.673+0000 7fc2fe000700 1 --2- 172.21.15.33:0/3441185194 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5643b974a970 0x5643b9749830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:42:08.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.673+0000 7fc2ff400700 1 -- 172.21.15.33:0/3441185194 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5643b974a970 msgr2=0x5643b9749830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:42:08.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.673+0000 7fc2ff400700 1 --2- 172.21.15.33:0/3441185194 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5643b974a970 0x5643b9749830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:08.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.673+0000 7fc2ff400700 1 -- 172.21.15.33:0/3441185194 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5643b9749fb0 msgr2=0x5643b9752a20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-03T18:42:08.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.674+0000 7fc2ff400700 1 --2- 172.21.15.33:0/3441185194 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5643b9749fb0 0x5643b9752a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:08.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.674+0000 7fc2ff400700 1 -- 172.21.15.33:0/3441185194 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643b9756340 con 0x5643b974fc50 2024-05-03T18:42:08.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.676+0000 7fc2ff400700 1 --2- 172.21.15.33:0/3441185194 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5643b974fc50 0x5643b9760000 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc2f80b12a0 tx=0x7fc2f80a6cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=772753ab113072e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:42:08.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.677+0000 7fc2ebe00700 1 -- 172.21.15.33:0/3441185194 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f802a070 con 0x5643b974fc50 2024-05-03T18:42:08.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.677+0000 7fc2ebe00700 1 -- 172.21.15.33:0/3441185194 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc2f80c9020 con 0x5643b974fc50 2024-05-03T18:42:08.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.677+0000 7fc2ebe00700 1 -- 172.21.15.33:0/3441185194 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f80af070 con 0x5643b974fc50 2024-05-03T18:42:08.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.678+0000 7fc3107acb80 1 -- 172.21.15.33:0/3441185194 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5643b974fc50 msgr2=0x5643b9760000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:42:08.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.678+0000 7fc3107acb80 1 --2- 172.21.15.33:0/3441185194 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5643b974fc50 0x5643b9760000 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc2f80b12a0 tx=0x7fc2f80a6cb0 comp rx=0 tx=0).stop 2024-05-03T18:42:08.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.679+0000 7fc3107acb80 1 -- 172.21.15.33:0/3441185194 shutdown_connections 2024-05-03T18:42:08.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.679+0000 7fc3107acb80 1 --2- 172.21.15.33:0/3441185194 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5643b974fc50 0x5643b9760000 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:08.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.679+0000 7fc3107acb80 1 --2- 172.21.15.33:0/3441185194 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5643b9749fb0 0x5643b9752a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:08.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.679+0000 7fc3107acb80 1 --2- 172.21.15.33:0/3441185194 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5643b974a970 0x5643b9749830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:08.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.679+0000 7fc3107acb80 1 -- 172.21.15.33:0/3441185194 >> 172.21.15.33:0/3441185194 conn(0x5643b96c2750 msgr2=0x5643b974d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:42:08.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.679+0000 7fc3107acb80 1 -- 172.21.15.33:0/3441185194 shutdown_connections 2024-05-03T18:42:08.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.680+0000 7fc3107acb80 1 -- 172.21.15.33:0/3441185194 wait complete. 2024-05-03T18:42:08.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.682+0000 7fc3107acb80 1 -- start start 2024-05-03T18:42:08.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.683+0000 7fc3107acb80 1 Processor -- start 2024-05-03T18:42:08.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.684+0000 7fc3107acb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5643b9749fb0 0x5643b97595d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:08.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.685+0000 7fc3107acb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5643b974a970 0x5643b9759b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:08.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.685+0000 7fc3107acb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5643b974fc50 0x5643b97ce6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:08.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.686+0000 7fc3107acb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc3001a3b10 con 0x5643b9749fb0 2024-05-03T18:42:08.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.686+0000 7fc3107acb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc3001be070 con 0x5643b974fc50 2024-05-03T18:42:08.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.686+0000 7fc3107acb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc3001a41e0 con 0x5643b974a970 2024-05-03T18:42:08.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.686+0000 7fc2fe000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5643b974a970 0x5643b9759b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:42:08.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.687+0000 7fc3107acb80 10 client.0 _ll_register_callbacks cb 0x5643b9643cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:42:08.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.687+0000 7fc2fe000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5643b974a970 0x5643b9759b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:46096/0 (socket says 172.21.15.33:46096) 2024-05-03T18:42:08.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.687+0000 7fc2fe000700 1 -- 172.21.15.33:0/2625785460 learned_addr learned my addr 172.21.15.33:0/2625785460 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:42:08.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.687+0000 7fc2fea00700 1 --2- 172.21.15.33:0/2625785460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5643b9749fb0 0x5643b97595d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:42:08.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.687+0000 7fc2ff400700 1 --2- 172.21.15.33:0/2625785460 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5643b974fc50 0x5643b97ce6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:42:08.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.688+0000 7fc2fe000700 1 -- 172.21.15.33:0/2625785460 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5643b974fc50 msgr2=0x5643b97ce6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:42:08.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.688+0000 7fc2fe000700 1 --2- 172.21.15.33:0/2625785460 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5643b974fc50 0x5643b97ce6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:08.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.688+0000 7fc2fe000700 1 -- 172.21.15.33:0/2625785460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5643b9749fb0 msgr2=0x5643b97595d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:42:08.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.688+0000 7fc2fe000700 1 --2- 172.21.15.33:0/2625785460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5643b9749fb0 0x5643b97595d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:08.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.688+0000 7fc2fe000700 1 -- 172.21.15.33:0/2625785460 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643b97cec10 con 0x5643b974a970 2024-05-03T18:42:08.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.688+0000 7fc2ff400700 1 --2- 172.21.15.33:0/2625785460 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5643b974fc50 0x5643b97ce6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:42:08.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.688+0000 7fc2fea00700 1 --2- 172.21.15.33:0/2625785460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5643b9749fb0 0x5643b97595d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:42:08.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.689+0000 7fc2fe000700 1 --2- 172.21.15.33:0/2625785460 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5643b974a970 0x5643b9759b10 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x5643b9755070 tx=0x7fc2ec0039b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97bb9d9b5de488a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:42:08.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.689+0000 7fc2e9600700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-03T18:42:08.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.689+0000 7fc2e9600700 1 -- 172.21.15.33:0/2625785460 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec00bad0 con 0x5643b974a970 2024-05-03T18:42:08.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.689+0000 7fc3107acb80 1 -- 172.21.15.33:0/2625785460 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2f80c9020 con 0x5643b974a970 2024-05-03T18:42:08.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.689+0000 7fc2e9600700 1 -- 172.21.15.33:0/2625785460 <== mon.1 v2:172.21.15.150:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9518+0+0 (secure 0 0 0) 0x7fc2ec00bc70 con 0x5643b974a970 2024-05-03T18:42:08.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.690+0000 7fc2e9600700 1 -- 172.21.15.33:0/2625785460 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fc2ec01b470 con 0x5643b974a970 2024-05-03T18:42:08.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.690+0000 7fc3107acb80 10 client.26554 fetch_fsmap learned FSMap version 21 2024-05-03T18:42:08.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.690+0000 7fc3107acb80 1 -- 172.21.15.33:0/2625785460 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5643b97cff50 con 0x5643b974a970 2024-05-03T18:42:08.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.691+0000 7fc2e9600700 1 -- 172.21.15.33:0/2625785460 <== mon.1 v2:172.21.15.150:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc2f80c9020 con 0x5643b974a970 2024-05-03T18:42:08.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.691+0000 7fc2e9600700 1 -- 172.21.15.33:0/2625785460 <== mon.1 v2:172.21.15.150:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fc2ec012680 con 0x5643b974a970 2024-05-03T18:42:08.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.691+0000 7fc3107acb80 10 client.26554 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:42:08.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.691+0000 7fc3107acb80 10 client.26554 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:42:08.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.691+0000 7fc3107acb80 20 client.26554 populate_metadata read hostname 'smithi033' 2024-05-03T18:42:08.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.691+0000 7fc3107acb80 1 --2- 172.21.15.33:0/2625785460 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5643b97d0620 0x5643b97d2a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:08.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.691+0000 7fc3107acb80 4 client.26554 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-03T18:42:08.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.691+0000 7fc3107acb80 1 -- 172.21.15.33:0/2625785460 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5643b9717070 con 0x5643b97d0620 2024-05-03T18:42:08.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.692+0000 7fc2fea00700 1 --2- 172.21.15.33:0/2625785460 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5643b97d0620 0x5643b97d2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:42:08.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.692+0000 7fc2e9600700 10 client.26554 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:42:08.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.693+0000 7fc2fea00700 1 --2- 172.21.15.33:0/2625785460 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5643b97d0620 0x5643b97d2a40 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bcbc92ad8d943f16 server_cookie=fcb0ba837a8b09d8 in_seq=0 out_seq=0 2024-05-03T18:42:08.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.694+0000 7fc2e9600700 1 -- 172.21.15.33:0/2625785460 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5643b9717070 con 0x5643b97d0620 2024-05-03T18:42:08.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.694+0000 7fc2e9600700 10 client.26554 handle_command_reply: tid=0 2024-05-03T18:42:08.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.695+0000 7fc3107acb80 1 client.26554 shutdown 2024-05-03T18:42:08.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.696+0000 7fc3107acb80 1 -- 172.21.15.33:0/2625785460 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5643b974a970 msgr2=0x5643b9759b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:42:08.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.696+0000 7fc3107acb80 1 --2- 172.21.15.33:0/2625785460 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5643b974a970 0x5643b9759b10 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x5643b9755070 tx=0x7fc2ec0039b0 comp rx=0 tx=0).stop 2024-05-03T18:42:08.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.696+0000 7fc3107acb80 1 -- 172.21.15.33:0/2625785460 shutdown_connections 2024-05-03T18:42:08.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.696+0000 7fc3107acb80 1 --2- 172.21.15.33:0/2625785460 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5643b97d0620 0x5643b97d2a40 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:08.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.696+0000 7fc3107acb80 1 --2- 172.21.15.33:0/2625785460 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5643b974fc50 0x5643b97ce6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:08.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.696+0000 7fc3107acb80 1 --2- 172.21.15.33:0/2625785460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5643b9749fb0 0x5643b97595d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:08.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.696+0000 7fc3107acb80 1 --2- 172.21.15.33:0/2625785460 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5643b974a970 0x5643b9759b10 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:08.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.696+0000 7fc3107acb80 1 -- 172.21.15.33:0/2625785460 >> 172.21.15.33:0/2625785460 conn(0x5643b96c2750 msgr2=0x5643b974df50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:42:08.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.697+0000 7fc2e9600700 0 client.26554 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:42:08.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.699+0000 7fc3107acb80 1 -- 172.21.15.33:0/2625785460 shutdown_connections 2024-05-03T18:42:08.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.699+0000 7fc3107acb80 1 -- 172.21.15.33:0/2625785460 wait complete. 2024-05-03T18:42:08.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.699+0000 7fc3107acb80 20 client.26554 trim_cache size 0 max 16384 2024-05-03T18:42:08.701 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-03T18:42:08.701 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-05-03T18:42:08.701 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "f6a6a320-6cc8-4044-a006-2fa8e1ec3df0", 2024-05-03T18:42:08.701 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-05-03T18:42:08.701 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-03T18:42:08.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.701+0000 7fc2ffe00700 1 -- 172.21.15.33:0/3148455465 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fc2e40b2a70 msgr2=0x7fc2e40b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:42:08.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.701+0000 7fc2ffe00700 1 --2- 172.21.15.33:0/3148455465 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fc2e40b2a70 0x7fc2e40b4f30 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc300110290 tx=0x7fc2f0006d20 comp rx=0 tx=0).stop 2024-05-03T18:42:08.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.701+0000 7fc2ffe00700 1 -- 172.21.15.33:0/3148455465 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc3001086f0 msgr2=0x7fc3001161f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:42:08.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.701+0000 7fc2ffe00700 1 --2- 172.21.15.33:0/3148455465 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc3001086f0 0x7fc3001161f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8006070 tx=0x7fc2f8007770 comp rx=0 tx=0).stop 2024-05-03T18:42:08.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.702+0000 7fc2ffe00700 1 -- 172.21.15.33:0/3148455465 shutdown_connections 2024-05-03T18:42:08.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.702+0000 7fc2ffe00700 1 --2- 172.21.15.33:0/3148455465 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc3001086f0 0x7fc3001161f0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:08.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.702+0000 7fc2ffe00700 1 --2- 172.21.15.33:0/3148455465 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc300103610 0x7fc3001130c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:08.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.702+0000 7fc2ffe00700 1 --2- 172.21.15.33:0/3148455465 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fc2e40b2a70 0x7fc2e40b4f30 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:08.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.702+0000 7fc2ffe00700 1 --2- 172.21.15.33:0/3148455465 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc300103fc0 0x7fc300113600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:08.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.702+0000 7fc2ffe00700 1 -- 172.21.15.33:0/3148455465 >> 172.21.15.33:0/3148455465 conn(0x7fc3000feec0 msgr2=0x7fc300106ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:42:08.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.702+0000 7fc2ffe00700 1 -- 172.21.15.33:0/3148455465 shutdown_connections 2024-05-03T18:42:08.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:08.702+0000 7fc2ffe00700 1 -- 172.21.15.33:0/3148455465 wait complete. 2024-05-03T18:42:09.794 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-03T18:42:10.110 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 655.0 s, 545.6 tps, lat 181.012 ms stddev 69.390 2024-05-03T18:42:10.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:42:10 smithi033 ceph-mon[28330]: pgmap v467: 129 pgs: 129 active+clean; 3.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 5.2 MiB/s wr, 472 op/s 2024-05-03T18:42:10.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:42:10 smithi033 ceph-mon[28330]: scrub queued for path: / 2024-05-03T18:42:10.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:42:10 smithi033 ceph-mon[28330]: scrub summary: idle+waiting paths [/] 2024-05-03T18:42:10.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:42:10 smithi033 ceph-mon[28330]: scrub summary: active paths [/] 2024-05-03T18:42:10.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:42:10 smithi033 ceph-mon[28330]: scrub summary: idle+waiting paths [/] 2024-05-03T18:42:10.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:42:10 smithi099 ceph-mon[31658]: pgmap v467: 129 pgs: 129 active+clean; 3.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 5.2 MiB/s wr, 472 op/s 2024-05-03T18:42:10.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:42:10 smithi099 ceph-mon[31658]: scrub queued for path: / 2024-05-03T18:42:10.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:42:10 smithi099 ceph-mon[31658]: scrub summary: idle+waiting paths [/] 2024-05-03T18:42:10.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:42:10 smithi099 ceph-mon[31658]: scrub summary: active paths [/] 2024-05-03T18:42:10.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:42:10 smithi099 ceph-mon[31658]: scrub summary: idle+waiting paths [/] 2024-05-03T18:42:10.635 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:42:10 smithi150 ceph-mon[32492]: pgmap v467: 129 pgs: 129 active+clean; 3.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 5.2 MiB/s wr, 472 op/s 2024-05-03T18:42:10.635 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:42:10 smithi150 ceph-mon[32492]: scrub queued for path: / 2024-05-03T18:42:10.635 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:42:10 smithi150 ceph-mon[32492]: scrub summary: idle+waiting paths [/] 2024-05-03T18:42:10.635 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:42:10 smithi150 ceph-mon[32492]: scrub summary: active paths [/] 2024-05-03T18:42:10.635 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:42:10 smithi150 ceph-mon[32492]: scrub summary: idle+waiting paths [/] 2024-05-03T18:42:10.689 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:42:12.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:42:12 smithi033 ceph-mon[28330]: pgmap v468: 129 pgs: 129 active+clean; 3.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 7.6 MiB/s wr, 703 op/s 2024-05-03T18:42:12.486 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:42:12 smithi033 ceph-mon[28330]: scrub summary: idle 2024-05-03T18:42:12.486 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:42:12 smithi033 ceph-mon[28330]: osdmap e107: 12 total, 12 up, 12 in 2024-05-03T18:42:12.547 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:42:12 smithi099 ceph-mon[31658]: pgmap v468: 129 pgs: 129 active+clean; 3.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 7.6 MiB/s wr, 703 op/s 2024-05-03T18:42:12.547 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:42:12 smithi099 ceph-mon[31658]: scrub summary: idle 2024-05-03T18:42:12.547 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:42:12 smithi099 ceph-mon[31658]: osdmap e107: 12 total, 12 up, 12 in 2024-05-03T18:42:12.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:42:12 smithi150 ceph-mon[32492]: pgmap v468: 129 pgs: 129 active+clean; 3.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 7.6 MiB/s wr, 703 op/s 2024-05-03T18:42:12.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:42:12 smithi150 ceph-mon[32492]: scrub summary: idle 2024-05-03T18:42:12.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:42:12 smithi150 ceph-mon[32492]: osdmap e107: 12 total, 12 up, 12 in 2024-05-03T18:42:13.547 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:42:13 smithi099 ceph-mon[31658]: pgmap v470: 129 pgs: 129 active+clean; 3.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 6.4 MiB/s wr, 603 op/s 2024-05-03T18:42:13.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:42:13 smithi150 ceph-mon[32492]: pgmap v470: 129 pgs: 129 active+clean; 3.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 6.4 MiB/s wr, 603 op/s 2024-05-03T18:42:13.735 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:42:13 smithi033 ceph-mon[28330]: pgmap v470: 129 pgs: 129 active+clean; 3.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 6.4 MiB/s wr, 603 op/s 2024-05-03T18:42:15.110 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 660.0 s, 433.2 tps, lat 235.002 ms stddev 284.279 2024-05-03T18:42:15.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.348+0000 7f1220c00700 1 -- 172.21.15.33:0/1298974205 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f121c108410 msgr2=0x7f121c1087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:42:15.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.348+0000 7f1220c00700 1 --2- 172.21.15.33:0/1298974205 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f121c108410 0x7f121c1087f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f120c009990 tx=0x7f120c009ca0 comp rx=0 tx=0).stop 2024-05-03T18:42:15.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.351+0000 7f1220c00700 1 -- 172.21.15.33:0/1298974205 shutdown_connections 2024-05-03T18:42:15.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.351+0000 7f1220c00700 1 --2- 172.21.15.33:0/1298974205 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f121c102d60 0x7f121c06b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:15.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.351+0000 7f1220c00700 1 --2- 172.21.15.33:0/1298974205 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f121c108410 0x7f121c1087f0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:15.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.351+0000 7f1220c00700 1 --2- 172.21.15.33:0/1298974205 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f121c1023a0 0x7f121c102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:15.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.351+0000 7f1220c00700 1 -- 172.21.15.33:0/1298974205 >> 172.21.15.33:0/1298974205 conn(0x7f121c079350 msgr2=0x7f121c079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:42:15.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.352+0000 7f1220c00700 1 -- 172.21.15.33:0/1298974205 shutdown_connections 2024-05-03T18:42:15.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.353+0000 7f1220c00700 1 -- 172.21.15.33:0/1298974205 wait complete. 2024-05-03T18:42:15.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.354+0000 7f1220c00700 1 Processor -- start 2024-05-03T18:42:15.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.355+0000 7f1220c00700 1 -- start start 2024-05-03T18:42:15.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.355+0000 7f1220c00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f121c1023a0 0x7f121c115240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:15.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.355+0000 7f1220c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f121c102d60 0x7f121c1122f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:15.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.355+0000 7f1220c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f121c108410 0x7f121c1128b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:15.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.356+0000 7f1220c00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f121c115870 con 0x7f121c1023a0 2024-05-03T18:42:15.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.356+0000 7f1220c00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f121c10f200 con 0x7f121c108410 2024-05-03T18:42:15.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.356+0000 7f1220c00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f121c1159f0 con 0x7f121c102d60 2024-05-03T18:42:15.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.356+0000 7f121a800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f121c108410 0x7f121c1128b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:42:15.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.357+0000 7f121a800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f121c108410 0x7f121c1128b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:41854/0 (socket says 172.21.15.33:41854) 2024-05-03T18:42:15.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.357+0000 7f121a800700 1 -- 172.21.15.33:0/2391411236 learned_addr learned my addr 172.21.15.33:0/2391411236 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:42:15.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.357+0000 7f1219400700 1 --2- 172.21.15.33:0/2391411236 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f121c102d60 0x7f121c1122f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:42:15.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.357+0000 7f1219e00700 1 --2- 172.21.15.33:0/2391411236 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f121c1023a0 0x7f121c115240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:42:15.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.358+0000 7f1219400700 1 -- 172.21.15.33:0/2391411236 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f121c108410 msgr2=0x7f121c1128b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:42:15.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.358+0000 7f1219400700 1 --2- 172.21.15.33:0/2391411236 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f121c108410 0x7f121c1128b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:15.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.358+0000 7f1219400700 1 -- 172.21.15.33:0/2391411236 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f121c1023a0 msgr2=0x7f121c115240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:42:15.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.358+0000 7f1219400700 1 --2- 172.21.15.33:0/2391411236 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f121c1023a0 0x7f121c115240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:15.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.358+0000 7f1219400700 1 -- 172.21.15.33:0/2391411236 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f121c1a1c60 con 0x7f121c102d60 2024-05-03T18:42:15.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.358+0000 7f1219400700 1 --2- 172.21.15.33:0/2391411236 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f121c102d60 0x7f121c1122f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f120800e9f0 tx=0x7f120800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:42:15.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.359+0000 7f1212a00700 1 -- 172.21.15.33:0/2391411236 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f120800c9c0 con 0x7f121c102d60 2024-05-03T18:42:15.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.359+0000 7f1212a00700 1 -- 172.21.15.33:0/2391411236 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1208010430 con 0x7f121c102d60 2024-05-03T18:42:15.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.359+0000 7f1212a00700 1 -- 172.21.15.33:0/2391411236 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1208018680 con 0x7f121c102d60 2024-05-03T18:42:15.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.359+0000 7f1220c00700 1 -- 172.21.15.33:0/2391411236 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f121c1a1f50 con 0x7f121c102d60 2024-05-03T18:42:15.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.359+0000 7f1220c00700 1 -- 172.21.15.33:0/2391411236 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f121c1a2440 con 0x7f121c102d60 2024-05-03T18:42:15.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.368+0000 7f1212a00700 1 -- 172.21.15.33:0/2391411236 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f1208018820 con 0x7f121c102d60 2024-05-03T18:42:15.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.371+0000 7f1212a00700 1 --2- 172.21.15.33:0/2391411236 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f12000b29f0 0x7f12000b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:15.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.371+0000 7f1212a00700 1 -- 172.21.15.33:0/2391411236 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9606+0+0 (secure 0 0 0) 0x7f12080e3c10 con 0x7f121c102d60 2024-05-03T18:42:15.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.371+0000 7f1219e00700 1 --2- 172.21.15.33:0/2391411236 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f12000b29f0 0x7f12000b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:42:15.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.419+0000 7f1219e00700 1 --2- 172.21.15.33:0/2391411236 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f12000b29f0 0x7f12000b4eb0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f120c00b2e0 tx=0x7f120c0058e0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:42:15.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.422+0000 7f122b64db80 1 Processor -- start 2024-05-03T18:42:15.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.422+0000 7f122b64db80 1 -- start start 2024-05-03T18:42:15.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.423+0000 7f122b64db80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562bb0adf7b0 0x562bb0adfb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:15.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.423+0000 7f122b64db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562bb0ae00d0 0x562bb0adafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:15.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.423+0000 7f122b64db80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562bb0adb5d0 0x562bb0adba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:15.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.423+0000 7f122b64db80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x562bb0a2da20 con 0x562bb0adf7b0 2024-05-03T18:42:15.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.423+0000 7f122b64db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x562bb09cb760 con 0x562bb0ae00d0 2024-05-03T18:42:15.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.423+0000 7f122b64db80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x562bb0a299e0 con 0x562bb0adb5d0 2024-05-03T18:42:15.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.424+0000 7f1219400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562bb0ae00d0 0x562bb0adafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:42:15.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.424+0000 7f1219400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562bb0ae00d0 0x562bb0adafb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:41858/0 (socket says 172.21.15.33:41858) 2024-05-03T18:42:15.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.424+0000 7f1219400700 1 -- 172.21.15.33:0/2906354491 learned_addr learned my addr 172.21.15.33:0/2906354491 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:42:15.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.425+0000 7f121a800700 1 --2- 172.21.15.33:0/2906354491 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562bb0adb5d0 0x562bb0adba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:42:15.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.426+0000 7f1219400700 1 -- 172.21.15.33:0/2906354491 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562bb0adb5d0 msgr2=0x562bb0adba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:42:15.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.426+0000 7f1219400700 1 --2- 172.21.15.33:0/2906354491 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562bb0adb5d0 0x562bb0adba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:15.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.426+0000 7f1219400700 1 -- 172.21.15.33:0/2906354491 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562bb0adf7b0 msgr2=0x562bb0adfb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:42:15.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.426+0000 7f1219400700 1 --2- 172.21.15.33:0/2906354491 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562bb0adf7b0 0x562bb0adfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:15.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.426+0000 7f1219400700 1 -- 172.21.15.33:0/2906354491 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562bb097a8d0 con 0x562bb0ae00d0 2024-05-03T18:42:15.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.427+0000 7f1219400700 1 --2- 172.21.15.33:0/2906354491 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562bb0ae00d0 0x562bb0adafb0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f120800e9c0 tx=0x7f12080b31f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=726a397f6e229af6 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:42:15.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.427+0000 7f1207400700 1 -- 172.21.15.33:0/2906354491 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12080a6050 con 0x562bb0ae00d0 2024-05-03T18:42:15.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.428+0000 7f1207400700 1 -- 172.21.15.33:0/2906354491 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f12080b3040 con 0x562bb0ae00d0 2024-05-03T18:42:15.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.429+0000 7f1207400700 1 -- 172.21.15.33:0/2906354491 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12080152d0 con 0x562bb0ae00d0 2024-05-03T18:42:15.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.430+0000 7f122b64db80 1 -- 172.21.15.33:0/2906354491 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562bb0ae00d0 msgr2=0x562bb0adafb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:42:15.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.430+0000 7f122b64db80 1 --2- 172.21.15.33:0/2906354491 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562bb0ae00d0 0x562bb0adafb0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f120800e9c0 tx=0x7f12080b31f0 comp rx=0 tx=0).stop 2024-05-03T18:42:15.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.430+0000 7f122b64db80 1 -- 172.21.15.33:0/2906354491 shutdown_connections 2024-05-03T18:42:15.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.430+0000 7f122b64db80 1 --2- 172.21.15.33:0/2906354491 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562bb0adb5d0 0x562bb0adba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:15.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.430+0000 7f122b64db80 1 --2- 172.21.15.33:0/2906354491 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562bb0adf7b0 0x562bb0adfb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:15.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.430+0000 7f122b64db80 1 --2- 172.21.15.33:0/2906354491 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562bb0ae00d0 0x562bb0adafb0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:15.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.430+0000 7f122b64db80 1 -- 172.21.15.33:0/2906354491 >> 172.21.15.33:0/2906354491 conn(0x562bb0a09b60 msgr2=0x562bb0ad57f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:42:15.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.431+0000 7f122b64db80 1 -- 172.21.15.33:0/2906354491 shutdown_connections 2024-05-03T18:42:15.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.431+0000 7f122b64db80 1 -- 172.21.15.33:0/2906354491 wait complete. 2024-05-03T18:42:15.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.432+0000 7f122b64db80 1 -- start start 2024-05-03T18:42:15.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.433+0000 7f122b64db80 1 Processor -- start 2024-05-03T18:42:15.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.437+0000 7f122b64db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562bb0adb5d0 0x562bb0ae6e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:15.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.437+0000 7f122b64db80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562bb0adf7b0 0x562bb0ae4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:15.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.437+0000 7f122b64db80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562bb0ae00d0 0x562bb0ae47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:15.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.437+0000 7f122b64db80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x562bb0ae7cf0 con 0x562bb0ae00d0 2024-05-03T18:42:15.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.437+0000 7f122b64db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x562bb0993b20 con 0x562bb0adb5d0 2024-05-03T18:42:15.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.437+0000 7f122b64db80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x562bb0ae7e70 con 0x562bb0adf7b0 2024-05-03T18:42:15.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.437+0000 7f122b64db80 10 client.0 _ll_register_callbacks cb 0x562bb09d4c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:42:15.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.438+0000 7f1219e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562bb0adb5d0 0x562bb0ae6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:42:15.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.438+0000 7f1219e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562bb0adb5d0 0x562bb0ae6e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:41870/0 (socket says 172.21.15.33:41870) 2024-05-03T18:42:15.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.438+0000 7f1219e00700 1 -- 172.21.15.33:0/2103357771 learned_addr learned my addr 172.21.15.33:0/2103357771 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:42:15.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.438+0000 7f121a800700 1 --2- 172.21.15.33:0/2103357771 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562bb0ae00d0 0x562bb0ae47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:42:15.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.438+0000 7f1219400700 1 --2- 172.21.15.33:0/2103357771 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562bb0adf7b0 0x562bb0ae4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:42:15.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.439+0000 7f1219e00700 1 -- 172.21.15.33:0/2103357771 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562bb0adf7b0 msgr2=0x562bb0ae4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:42:15.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.439+0000 7f1219e00700 1 --2- 172.21.15.33:0/2103357771 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562bb0adf7b0 0x562bb0ae4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:15.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.440+0000 7f1219e00700 1 -- 172.21.15.33:0/2103357771 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562bb0ae00d0 msgr2=0x562bb0ae47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:42:15.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.440+0000 7f1219e00700 1 --2- 172.21.15.33:0/2103357771 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562bb0ae00d0 0x562bb0ae47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:15.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.440+0000 7f1219e00700 1 -- 172.21.15.33:0/2103357771 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562bb09f3420 con 0x562bb0adb5d0 2024-05-03T18:42:15.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.441+0000 7f1219e00700 1 --2- 172.21.15.33:0/2103357771 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562bb0adb5d0 0x562bb0ae6e50 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f120c015000 tx=0x7f120c00bae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2e5261058b9e4e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:42:15.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.441+0000 7f11fbe00700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-03T18:42:15.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.444+0000 7f11fbe00700 1 -- 172.21.15.33:0/2103357771 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f120c003b50 con 0x562bb0adb5d0 2024-05-03T18:42:15.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.444+0000 7f11fbe00700 1 -- 172.21.15.33:0/2103357771 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9606+0+0 (secure 0 0 0) 0x7f120c003cf0 con 0x562bb0adb5d0 2024-05-03T18:42:15.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.444+0000 7f122b64db80 1 -- 172.21.15.33:0/2103357771 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12080b3040 con 0x562bb0adb5d0 2024-05-03T18:42:15.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.445+0000 7f11fbe00700 1 -- 172.21.15.33:0/2103357771 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f120c0356f0 con 0x562bb0adb5d0 2024-05-03T18:42:15.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.445+0000 7f122b64db80 10 client.36488 fetch_fsmap learned FSMap version 21 2024-05-03T18:42:15.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.445+0000 7f122b64db80 1 -- 172.21.15.33:0/2103357771 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x562bb0b03550 con 0x562bb0adb5d0 2024-05-03T18:42:15.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.447+0000 7f11fbe00700 1 -- 172.21.15.33:0/2103357771 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f12080b3040 con 0x562bb0adb5d0 2024-05-03T18:42:15.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.447+0000 7f11fbe00700 1 -- 172.21.15.33:0/2103357771 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f120c02e070 con 0x562bb0adb5d0 2024-05-03T18:42:15.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.447+0000 7f122b64db80 10 client.36488 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:42:15.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.447+0000 7f122b64db80 10 client.36488 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:42:15.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.447+0000 7f122b64db80 20 client.36488 populate_metadata read hostname 'smithi033' 2024-05-03T18:42:15.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.447+0000 7f122b64db80 1 --2- 172.21.15.33:0/2103357771 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x562bb0b05b90 0x562bb0b07f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:15.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.447+0000 7f122b64db80 4 client.36488 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:42:15.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.447+0000 7f122b64db80 1 -- 172.21.15.33:0/2103357771 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562bb0aa81d0 con 0x562bb0b05b90 2024-05-03T18:42:15.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.448+0000 7f1219400700 1 --2- 172.21.15.33:0/2103357771 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x562bb0b05b90 0x562bb0b07f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:42:15.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.450+0000 7f1219400700 1 --2- 172.21.15.33:0/2103357771 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x562bb0b05b90 0x562bb0b07f80 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=420efeacc4a92db8 server_cookie=26004feca4fdca24 in_seq=0 out_seq=0 2024-05-03T18:42:15.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.450+0000 7f11fbe00700 10 client.36488 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:42:15.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.452+0000 7f11fbe00700 1 -- 172.21.15.33:0/2103357771 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562bb0aa81d0 con 0x562bb0b05b90 2024-05-03T18:42:15.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.452+0000 7f11fbe00700 10 client.36488 handle_command_reply: tid=0 2024-05-03T18:42:15.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.453+0000 7f122b64db80 1 client.36488 shutdown 2024-05-03T18:42:15.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.455+0000 7f122b64db80 1 -- 172.21.15.33:0/2103357771 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562bb0adb5d0 msgr2=0x562bb0ae6e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:42:15.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.455+0000 7f122b64db80 1 --2- 172.21.15.33:0/2103357771 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562bb0adb5d0 0x562bb0ae6e50 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f120c015000 tx=0x7f120c00bae0 comp rx=0 tx=0).stop 2024-05-03T18:42:15.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.455+0000 7f122b64db80 1 -- 172.21.15.33:0/2103357771 shutdown_connections 2024-05-03T18:42:15.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.455+0000 7f122b64db80 1 --2- 172.21.15.33:0/2103357771 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x562bb0b05b90 0x562bb0b07f80 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:15.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.455+0000 7f122b64db80 1 --2- 172.21.15.33:0/2103357771 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562bb0ae00d0 0x562bb0ae47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:15.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.455+0000 7f122b64db80 1 --2- 172.21.15.33:0/2103357771 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562bb0adb5d0 0x562bb0ae6e50 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:15.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.455+0000 7f122b64db80 1 --2- 172.21.15.33:0/2103357771 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562bb0adf7b0 0x562bb0ae4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:15.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.455+0000 7f11fbe00700 0 client.36488 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:42:15.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.455+0000 7f122b64db80 1 -- 172.21.15.33:0/2103357771 >> 172.21.15.33:0/2103357771 conn(0x562bb0a09b60 msgr2=0x562bb0ad4650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:42:15.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.455+0000 7f122b64db80 1 -- 172.21.15.33:0/2103357771 shutdown_connections 2024-05-03T18:42:15.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.455+0000 7f122b64db80 1 -- 172.21.15.33:0/2103357771 wait complete. 2024-05-03T18:42:15.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.456+0000 7f122b64db80 20 client.36488 trim_cache size 0 max 16384 2024-05-03T18:42:15.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.494+0000 7f122b64db80 1 Processor -- start 2024-05-03T18:42:15.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.494+0000 7f122b64db80 1 -- start start 2024-05-03T18:42:15.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.494+0000 7f122b64db80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562bb0adf7b0 0x562bb0ad5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:15.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.494+0000 7f122b64db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562bb0ae00d0 0x562bb0ae4250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:15.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.494+0000 7f122b64db80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562bb0af0dd0 0x562bb0ae4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:15.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.494+0000 7f122b64db80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f121c073000 con 0x562bb0af0dd0 2024-05-03T18:42:15.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.494+0000 7f122b64db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f121c1b65d0 con 0x562bb0ae00d0 2024-05-03T18:42:15.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.494+0000 7f122b64db80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f121c073650 con 0x562bb0adf7b0 2024-05-03T18:42:15.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.495+0000 7f1219e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562bb0adf7b0 0x562bb0ad5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:42:15.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.495+0000 7f1219e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562bb0adf7b0 0x562bb0ad5660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:46138/0 (socket says 172.21.15.33:46138) 2024-05-03T18:42:15.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.495+0000 7f1219e00700 1 -- 172.21.15.33:0/2010524391 learned_addr learned my addr 172.21.15.33:0/2010524391 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:42:15.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.495+0000 7f1219400700 1 --2- 172.21.15.33:0/2010524391 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562bb0ae00d0 0x562bb0ae4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:42:15.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.495+0000 7f121a800700 1 --2- 172.21.15.33:0/2010524391 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562bb0af0dd0 0x562bb0ae4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:42:15.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.496+0000 7f1219e00700 1 -- 172.21.15.33:0/2010524391 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562bb0ae00d0 msgr2=0x562bb0ae4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:42:15.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.496+0000 7f1219e00700 1 --2- 172.21.15.33:0/2010524391 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562bb0ae00d0 0x562bb0ae4250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:15.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.496+0000 7f1219e00700 1 -- 172.21.15.33:0/2010524391 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562bb0af0dd0 msgr2=0x562bb0ae4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:42:15.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.496+0000 7f1219e00700 1 --2- 172.21.15.33:0/2010524391 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562bb0af0dd0 0x562bb0ae4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:15.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.496+0000 7f1219e00700 1 -- 172.21.15.33:0/2010524391 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562bb0adff00 con 0x562bb0adf7b0 2024-05-03T18:42:15.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.496+0000 7f121a800700 1 --2- 172.21.15.33:0/2010524391 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562bb0af0dd0 0x562bb0ae4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:42:15.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.496+0000 7f1219400700 1 --2- 172.21.15.33:0/2010524391 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562bb0ae00d0 0x562bb0ae4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:42:15.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.496+0000 7f1219e00700 1 --2- 172.21.15.33:0/2010524391 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562bb0adf7b0 0x562bb0ad5660 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f120c00fcf0 tx=0x7f120c026630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=242bec1aee84778f server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:42:15.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.497+0000 7f1206a00700 1 -- 172.21.15.33:0/2010524391 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f120c003b50 con 0x562bb0adf7b0 2024-05-03T18:42:15.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.497+0000 7f1206a00700 1 -- 172.21.15.33:0/2010524391 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f120c003cf0 con 0x562bb0adf7b0 2024-05-03T18:42:15.499 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:42:15 smithi033 ceph-mon[28330]: pgmap v471: 129 pgs: 2 active+clean+snaptrim, 122 active+clean, 5 active+clean+snaptrim_wait; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 6.7 MiB/s wr, 640 op/s 2024-05-03T18:42:15.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.497+0000 7f1206a00700 1 -- 172.21.15.33:0/2010524391 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f120c00f460 con 0x562bb0adf7b0 2024-05-03T18:42:15.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.498+0000 7f122b64db80 1 -- 172.21.15.33:0/2010524391 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562bb0adf7b0 msgr2=0x562bb0ad5660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:42:15.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.498+0000 7f122b64db80 1 --2- 172.21.15.33:0/2010524391 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562bb0adf7b0 0x562bb0ad5660 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f120c00fcf0 tx=0x7f120c026630 comp rx=0 tx=0).stop 2024-05-03T18:42:15.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.499+0000 7f122b64db80 1 -- 172.21.15.33:0/2010524391 shutdown_connections 2024-05-03T18:42:15.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.499+0000 7f122b64db80 1 --2- 172.21.15.33:0/2010524391 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562bb0af0dd0 0x562bb0ae4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:15.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.499+0000 7f122b64db80 1 --2- 172.21.15.33:0/2010524391 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562bb0adf7b0 0x562bb0ad5660 secure :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f120c00fcf0 tx=0x7f120c026630 comp rx=0 tx=0).stop 2024-05-03T18:42:15.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.499+0000 7f122b64db80 1 --2- 172.21.15.33:0/2010524391 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562bb0ae00d0 0x562bb0ae4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:15.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.499+0000 7f122b64db80 1 -- 172.21.15.33:0/2010524391 >> 172.21.15.33:0/2010524391 conn(0x562bb0a09b60 msgr2=0x562bb0ae3b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:42:15.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.500+0000 7f122b64db80 1 -- 172.21.15.33:0/2010524391 shutdown_connections 2024-05-03T18:42:15.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.500+0000 7f122b64db80 1 -- 172.21.15.33:0/2010524391 wait complete. 2024-05-03T18:42:15.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.501+0000 7f122b64db80 1 -- start start 2024-05-03T18:42:15.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.501+0000 7f122b64db80 1 Processor -- start 2024-05-03T18:42:15.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.502+0000 7f122b64db80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562bb0ae00d0 0x562bb0aed310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:15.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.502+0000 7f122b64db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562bb0af0dd0 0x562bb0aed850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:15.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.502+0000 7f122b64db80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562bb0ae9740 0x562bb0ae9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:15.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.502+0000 7f122b64db80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f121c073d20 con 0x562bb0ae9740 2024-05-03T18:42:15.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.502+0000 7f122b64db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f121c075190 con 0x562bb0af0dd0 2024-05-03T18:42:15.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.502+0000 7f122b64db80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f121c0743f0 con 0x562bb0ae00d0 2024-05-03T18:42:15.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.502+0000 7f122b64db80 10 client.0 _ll_register_callbacks cb 0x562bb09d4c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:42:15.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.502+0000 7f1219e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562bb0ae00d0 0x562bb0aed310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:42:15.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.502+0000 7f1219e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562bb0ae00d0 0x562bb0aed310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:46152/0 (socket says 172.21.15.33:46152) 2024-05-03T18:42:15.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.502+0000 7f1219e00700 1 -- 172.21.15.33:0/462098895 learned_addr learned my addr 172.21.15.33:0/462098895 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:42:15.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.503+0000 7f1219400700 1 --2- 172.21.15.33:0/462098895 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562bb0af0dd0 0x562bb0aed850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:42:15.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.503+0000 7f121a800700 1 --2- 172.21.15.33:0/462098895 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562bb0ae9740 0x562bb0ae9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:42:15.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.503+0000 7f1219e00700 1 -- 172.21.15.33:0/462098895 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562bb0af0dd0 msgr2=0x562bb0aed850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:42:15.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.503+0000 7f1219e00700 1 --2- 172.21.15.33:0/462098895 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562bb0af0dd0 0x562bb0aed850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:15.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.503+0000 7f1219e00700 1 -- 172.21.15.33:0/462098895 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562bb0ae9740 msgr2=0x562bb0ae9bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:42:15.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.503+0000 7f1219e00700 1 --2- 172.21.15.33:0/462098895 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562bb0ae9740 0x562bb0ae9bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:15.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.503+0000 7f1219e00700 1 -- 172.21.15.33:0/462098895 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562bb0aea400 con 0x562bb0ae00d0 2024-05-03T18:42:15.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.504+0000 7f1219e00700 1 --2- 172.21.15.33:0/462098895 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562bb0ae00d0 0x562bb0aed310 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f120c005010 tx=0x7f120c035420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7bf06c00dac582aa server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:42:15.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.504+0000 7f11fbe00700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-03T18:42:15.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.504+0000 7f11fbe00700 1 -- 172.21.15.33:0/462098895 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f120c035ca0 con 0x562bb0ae00d0 2024-05-03T18:42:15.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.505+0000 7f11fbe00700 1 -- 172.21.15.33:0/462098895 <== mon.1 v2:172.21.15.150:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9606+0+0 (secure 0 0 0) 0x7f120c00f460 con 0x562bb0ae00d0 2024-05-03T18:42:15.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.505+0000 7f122b64db80 1 -- 172.21.15.33:0/462098895 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f120c003cf0 con 0x562bb0ae00d0 2024-05-03T18:42:15.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.505+0000 7f11fbe00700 1 -- 172.21.15.33:0/462098895 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f120c035e40 con 0x562bb0ae00d0 2024-05-03T18:42:15.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.505+0000 7f122b64db80 10 client.26572 fetch_fsmap learned FSMap version 21 2024-05-03T18:42:15.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.505+0000 7f122b64db80 1 -- 172.21.15.33:0/462098895 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x562bb0b713d0 con 0x562bb0ae00d0 2024-05-03T18:42:15.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.506+0000 7f11fbe00700 1 -- 172.21.15.33:0/462098895 <== mon.1 v2:172.21.15.150:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f120c003cf0 con 0x562bb0ae00d0 2024-05-03T18:42:15.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.506+0000 7f11fbe00700 1 -- 172.21.15.33:0/462098895 <== mon.1 v2:172.21.15.150:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f120c03c020 con 0x562bb0ae00d0 2024-05-03T18:42:15.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.506+0000 7f122b64db80 10 client.26572 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:42:15.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.506+0000 7f122b64db80 10 client.26572 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:42:15.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.506+0000 7f122b64db80 20 client.26572 populate_metadata read hostname 'smithi033' 2024-05-03T18:42:15.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.506+0000 7f122b64db80 1 --2- 172.21.15.33:0/462098895 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x562bb0b71960 0x562bb0b73d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:15.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.506+0000 7f122b64db80 4 client.26572 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-03T18:42:15.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.506+0000 7f122b64db80 1 -- 172.21.15.33:0/462098895 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562bb0a3b400 con 0x562bb0b71960 2024-05-03T18:42:15.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.507+0000 7f1219400700 1 --2- 172.21.15.33:0/462098895 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x562bb0b71960 0x562bb0b73d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:42:15.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.509+0000 7f1219400700 1 --2- 172.21.15.33:0/462098895 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x562bb0b71960 0x562bb0b73d50 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c27aa98b7c3abccd server_cookie=3b7496055878ee60 in_seq=0 out_seq=0 2024-05-03T18:42:15.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.509+0000 7f11fbe00700 10 client.26572 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:42:15.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.510+0000 7f11fbe00700 1 -- 172.21.15.33:0/462098895 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x562bb0a3b400 con 0x562bb0b71960 2024-05-03T18:42:15.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.510+0000 7f11fbe00700 10 client.26572 handle_command_reply: tid=0 2024-05-03T18:42:15.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.510+0000 7f122b64db80 1 client.26572 shutdown 2024-05-03T18:42:15.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.512+0000 7f122b64db80 1 -- 172.21.15.33:0/462098895 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562bb0ae00d0 msgr2=0x562bb0aed310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:42:15.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.512+0000 7f122b64db80 1 --2- 172.21.15.33:0/462098895 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562bb0ae00d0 0x562bb0aed310 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f120c005010 tx=0x7f120c035420 comp rx=0 tx=0).stop 2024-05-03T18:42:15.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.513+0000 7f122b64db80 1 -- 172.21.15.33:0/462098895 shutdown_connections 2024-05-03T18:42:15.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.513+0000 7f122b64db80 1 --2- 172.21.15.33:0/462098895 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x562bb0b71960 0x562bb0b73d50 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:15.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.513+0000 7f122b64db80 1 --2- 172.21.15.33:0/462098895 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562bb0ae9740 0x562bb0ae9bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:15.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.513+0000 7f122b64db80 1 --2- 172.21.15.33:0/462098895 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562bb0ae00d0 0x562bb0aed310 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:15.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.513+0000 7f122b64db80 1 --2- 172.21.15.33:0/462098895 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562bb0af0dd0 0x562bb0aed850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:15.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.513+0000 7f11fbe00700 0 client.26572 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:42:15.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.513+0000 7f122b64db80 1 -- 172.21.15.33:0/462098895 >> 172.21.15.33:0/462098895 conn(0x562bb0a09b60 msgr2=0x562bb0af0650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:42:15.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.513+0000 7f122b64db80 1 -- 172.21.15.33:0/462098895 shutdown_connections 2024-05-03T18:42:15.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.513+0000 7f122b64db80 1 -- 172.21.15.33:0/462098895 wait complete. 2024-05-03T18:42:15.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.514+0000 7f122b64db80 20 client.26572 trim_cache size 0 max 16384 2024-05-03T18:42:15.515 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-03T18:42:15.515 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-05-03T18:42:15.515 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-05-03T18:42:15.516 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-03T18:42:15.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.515+0000 7f1220c00700 1 -- 172.21.15.33:0/2391411236 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f12000b29f0 msgr2=0x7f12000b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:42:15.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.515+0000 7f1220c00700 1 --2- 172.21.15.33:0/2391411236 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f12000b29f0 0x7f12000b4eb0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f120c00b2e0 tx=0x7f120c0058e0 comp rx=0 tx=0).stop 2024-05-03T18:42:15.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.515+0000 7f1220c00700 1 -- 172.21.15.33:0/2391411236 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f121c102d60 msgr2=0x7f121c1122f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:42:15.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.515+0000 7f1220c00700 1 --2- 172.21.15.33:0/2391411236 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f121c102d60 0x7f121c1122f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f120800e9f0 tx=0x7f120800ed00 comp rx=0 tx=0).stop 2024-05-03T18:42:15.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.515+0000 7f1220c00700 1 -- 172.21.15.33:0/2391411236 shutdown_connections 2024-05-03T18:42:15.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.515+0000 7f1220c00700 1 --2- 172.21.15.33:0/2391411236 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f12000b29f0 0x7f12000b4eb0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:15.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.515+0000 7f1220c00700 1 --2- 172.21.15.33:0/2391411236 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f121c108410 0x7f121c1128b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:15.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.515+0000 7f1220c00700 1 --2- 172.21.15.33:0/2391411236 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f121c1023a0 0x7f121c115240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:15.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.515+0000 7f1220c00700 1 --2- 172.21.15.33:0/2391411236 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f121c102d60 0x7f121c1122f0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:15.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.515+0000 7f1220c00700 1 -- 172.21.15.33:0/2391411236 >> 172.21.15.33:0/2391411236 conn(0x7f121c079350 msgr2=0x7f121c1014b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:42:15.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.515+0000 7f1220c00700 1 -- 172.21.15.33:0/2391411236 shutdown_connections 2024-05-03T18:42:15.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:15.515+0000 7f1220c00700 1 -- 172.21.15.33:0/2391411236 wait complete. 2024-05-03T18:42:15.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:42:15 smithi099 ceph-mon[31658]: pgmap v471: 129 pgs: 2 active+clean+snaptrim, 122 active+clean, 5 active+clean+snaptrim_wait; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 6.7 MiB/s wr, 640 op/s 2024-05-03T18:42:15.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:42:15 smithi150 ceph-mon[32492]: pgmap v471: 129 pgs: 2 active+clean+snaptrim, 122 active+clean, 5 active+clean+snaptrim_wait; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 6.7 MiB/s wr, 640 op/s 2024-05-03T18:42:16.604 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-03T18:42:16.605 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T18:42:17.405 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:42:17.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:42:17 smithi099 ceph-mon[31658]: pgmap v472: 129 pgs: 2 active+clean+snaptrim, 122 active+clean, 5 active+clean+snaptrim_wait; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 6.1 MiB/s wr, 572 op/s 2024-05-03T18:42:17.587 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:42:17 smithi033 ceph-mon[28330]: pgmap v472: 129 pgs: 2 active+clean+snaptrim, 122 active+clean, 5 active+clean+snaptrim_wait; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 6.1 MiB/s wr, 572 op/s 2024-05-03T18:42:17.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:42:17 smithi150 ceph-mon[32492]: pgmap v472: 129 pgs: 2 active+clean+snaptrim, 122 active+clean, 5 active+clean+snaptrim_wait; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 6.1 MiB/s wr, 572 op/s 2024-05-03T18:42:18.798 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:42:18 smithi099 ceph-mon[31658]: osdmap e108: 12 total, 12 up, 12 in 2024-05-03T18:42:18.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:42:18 smithi150 ceph-mon[32492]: osdmap e108: 12 total, 12 up, 12 in 2024-05-03T18:42:18.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:42:18 smithi033 ceph-mon[28330]: osdmap e108: 12 total, 12 up, 12 in 2024-05-03T18:42:19.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:42:19 smithi099 ceph-mon[31658]: pgmap v474: 129 pgs: 2 active+clean+snaptrim, 122 active+clean, 5 active+clean+snaptrim_wait; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 4.2 MiB/s wr, 376 op/s 2024-05-03T18:42:19.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:42:19 smithi150 ceph-mon[32492]: pgmap v474: 129 pgs: 2 active+clean+snaptrim, 122 active+clean, 5 active+clean+snaptrim_wait; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 4.2 MiB/s wr, 376 op/s 2024-05-03T18:42:19.935 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:42:19 smithi033 ceph-mon[28330]: pgmap v474: 129 pgs: 2 active+clean+snaptrim, 122 active+clean, 5 active+clean+snaptrim_wait; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 4.2 MiB/s wr, 376 op/s 2024-05-03T18:42:20.111 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 665.0 s, 502.6 tps, lat 197.515 ms stddev 126.963 2024-05-03T18:42:20.797 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:42:20 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:42:20.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:42:20 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:42:20.987 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:42:20 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:42:21.735 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:42:21 smithi033 ceph-mon[28330]: pgmap v475: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 7.0 MiB/s wr, 633 op/s 2024-05-03T18:42:21.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:42:21 smithi099 ceph-mon[31658]: pgmap v475: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 7.0 MiB/s wr, 633 op/s 2024-05-03T18:42:21.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:42:21 smithi150 ceph-mon[32492]: pgmap v475: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 7.0 MiB/s wr, 633 op/s 2024-05-03T18:42:22.584 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:42:22 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:42:22.585 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:42:22 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:42:22.585 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:42:22 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:42:22.585 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:42:22 smithi033 ceph-mon[28330]: osdmap e109: 12 total, 12 up, 12 in 2024-05-03T18:42:22.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:42:22 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:42:22.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:42:22 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:42:22.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:42:22 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:42:22.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:42:22 smithi099 ceph-mon[31658]: osdmap e109: 12 total, 12 up, 12 in 2024-05-03T18:42:22.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:42:22 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:42:22.885 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:42:22 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:42:22.885 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:42:22 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:42:22.885 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:42:22 smithi150 ceph-mon[32492]: osdmap e109: 12 total, 12 up, 12 in 2024-05-03T18:42:23.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:42:23 smithi099 ceph-mon[31658]: pgmap v476: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 6.3 MiB/s wr, 571 op/s 2024-05-03T18:42:23.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:42:23 smithi150 ceph-mon[32492]: pgmap v476: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 6.3 MiB/s wr, 571 op/s 2024-05-03T18:42:23.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:42:23 smithi033 ceph-mon[28330]: pgmap v476: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 6.3 MiB/s wr, 571 op/s 2024-05-03T18:42:24.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:24.939+0000 7fcf7fe00700 1 -- 172.21.15.33:0/1269195501 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf7809c5e0 msgr2=0x7fcf7809c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:42:24.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:24.939+0000 7fcf7fe00700 1 --2- 172.21.15.33:0/1269195501 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf7809c5e0 0x7fcf7809c9c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fcf80069f30 tx=0x7fcf800522b0 comp rx=0 tx=0).stop 2024-05-03T18:42:24.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:24.940+0000 7fcf7fe00700 1 -- 172.21.15.33:0/1269195501 shutdown_connections 2024-05-03T18:42:24.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:24.940+0000 7fcf7fe00700 1 --2- 172.21.15.33:0/1269195501 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcf780a1670 0x7fcf780a9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:24.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:24.940+0000 7fcf7fe00700 1 --2- 172.21.15.33:0/1269195501 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf7809c5e0 0x7fcf7809c9c0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:24.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:24.940+0000 7fcf7fe00700 1 --2- 172.21.15.33:0/1269195501 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf7809cf90 0x7fcf780a0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:24.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:24.940+0000 7fcf7fe00700 1 -- 172.21.15.33:0/1269195501 >> 172.21.15.33:0/1269195501 conn(0x7fcf7800c880 msgr2=0x7fcf7809a630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:42:24.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:24.942+0000 7fcf7fe00700 1 -- 172.21.15.33:0/1269195501 shutdown_connections 2024-05-03T18:42:24.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:24.942+0000 7fcf7fe00700 1 -- 172.21.15.33:0/1269195501 wait complete. 2024-05-03T18:42:24.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:24.943+0000 7fcf7fe00700 1 Processor -- start 2024-05-03T18:42:24.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:24.945+0000 7fcf7fe00700 1 -- start start 2024-05-03T18:42:24.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:24.946+0000 7fcf7fe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf7809c5e0 0x7fcf78015f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:24.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:24.946+0000 7fcf7fe00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcf7809cf90 0x7fcf7800eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:24.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:24.946+0000 7fcf7fe00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf780a1670 0x7fcf7800f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:24.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:24.947+0000 7fcf7f400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf780a1670 0x7fcf7800f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:42:24.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:24.947+0000 7fcf7e000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcf7809cf90 0x7fcf7800eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:42:24.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:24.947+0000 7fcf7fe00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcf780ac390 con 0x7fcf780a1670 2024-05-03T18:42:24.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:24.947+0000 7fcf7f400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf780a1670 0x7fcf7800f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52908/0 (socket says 172.21.15.33:52908) 2024-05-03T18:42:24.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:24.947+0000 7fcf7fe00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fcf780ac210 con 0x7fcf7809c5e0 2024-05-03T18:42:24.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:24.947+0000 7fcf7fe00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fcf780ac510 con 0x7fcf7809cf90 2024-05-03T18:42:24.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:24.947+0000 7fcf7e000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcf7809cf90 0x7fcf7800eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:35874/0 (socket says 172.21.15.33:35874) 2024-05-03T18:42:24.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:24.947+0000 7fcf7f400700 1 -- 172.21.15.33:0/1585250058 learned_addr learned my addr 172.21.15.33:0/1585250058 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:42:24.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:24.947+0000 7fcf7ea00700 1 --2- 172.21.15.33:0/1585250058 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf7809c5e0 0x7fcf78015f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:42:24.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:24.948+0000 7fcf7f400700 1 -- 172.21.15.33:0/1585250058 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcf7809cf90 msgr2=0x7fcf7800eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:42:24.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:24.948+0000 7fcf7f400700 1 --2- 172.21.15.33:0/1585250058 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcf7809cf90 0x7fcf7800eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:24.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:24.948+0000 7fcf7f400700 1 -- 172.21.15.33:0/1585250058 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf7809c5e0 msgr2=0x7fcf78015f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:42:24.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:24.948+0000 7fcf7f400700 1 --2- 172.21.15.33:0/1585250058 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf7809c5e0 0x7fcf78015f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:24.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:24.948+0000 7fcf7f400700 1 -- 172.21.15.33:0/1585250058 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf7800fc60 con 0x7fcf780a1670 2024-05-03T18:42:24.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:24.948+0000 7fcf7ea00700 1 --2- 172.21.15.33:0/1585250058 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf7809c5e0 0x7fcf78015f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:42:24.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:24.949+0000 7fcf7f400700 1 --2- 172.21.15.33:0/1585250058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf780a1670 0x7fcf7800f530 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fcf7000eb80 tx=0x7fcf7000ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:42:24.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:24.950+0000 7fcf84a00700 1 -- 172.21.15.33:0/1585250058 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf7000ccf0 con 0x7fcf780a1670 2024-05-03T18:42:24.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:24.950+0000 7fcf84a00700 1 -- 172.21.15.33:0/1585250058 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcf700044c0 con 0x7fcf780a1670 2024-05-03T18:42:24.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:24.950+0000 7fcf7fe00700 1 -- 172.21.15.33:0/1585250058 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf78013a90 con 0x7fcf780a1670 2024-05-03T18:42:24.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:24.950+0000 7fcf7fe00700 1 -- 172.21.15.33:0/1585250058 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf78013fd0 con 0x7fcf780a1670 2024-05-03T18:42:24.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:24.950+0000 7fcf84a00700 1 -- 172.21.15.33:0/1585250058 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf70010430 con 0x7fcf780a1670 2024-05-03T18:42:24.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:24.956+0000 7fcf7fe00700 1 -- 172.21.15.33:0/1585250058 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf60005330 con 0x7fcf780a1670 2024-05-03T18:42:24.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:24.957+0000 7fcf84a00700 1 -- 172.21.15.33:0/1585250058 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fcf70003ee0 con 0x7fcf780a1670 2024-05-03T18:42:24.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:24.960+0000 7fcf84a00700 1 --2- 172.21.15.33:0/1585250058 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fcf640b2bc0 0x7fcf640b5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:24.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:24.960+0000 7fcf84a00700 1 -- 172.21.15.33:0/1585250058 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9502+0+0 (secure 0 0 0) 0x7fcf700e2cd0 con 0x7fcf780a1670 2024-05-03T18:42:24.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:24.961+0000 7fcf7ea00700 1 --2- 172.21.15.33:0/1585250058 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fcf640b2bc0 0x7fcf640b5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:42:24.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:24.967+0000 7fcf7ea00700 1 --2- 172.21.15.33:0/1585250058 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fcf640b2bc0 0x7fcf640b5080 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fcf80069f30 tx=0x7fcf80069ec0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:42:24.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:24.971+0000 7fcf84a00700 1 -- 172.21.15.33:0/1585250058 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fcf700e8020 con 0x7fcf780a1670 2024-05-03T18:42:25.111 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 670.0 s, 405.2 tps, lat 243.233 ms stddev 163.922 2024-05-03T18:42:25.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:25.530+0000 7fcf7fe00700 1 -- 172.21.15.33:0/1585250058 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcf600058f0 con 0x7fcf780a1670 2024-05-03T18:42:25.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:25.535+0000 7fcf84a00700 1 -- 172.21.15.33:0/1585250058 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11801 (secure 0 0 0) 0x7fcf700aa280 con 0x7fcf780a1670 2024-05-03T18:42:25.539 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:42:25.539 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1327393503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1327393503},{"type":"v1","addr":"172.21.15.33:6835","nonce":1327393503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/2461472948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":2461472948},{"type":"v1","addr":"172.21.15.33:6837","nonce":2461472948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14559,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/442269281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":442269281},{"type":"v1","addr":"172.21.15.33:6839","nonce":442269281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14571,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/645571756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":645571756},{"type":"v1","addr":"172.21.15.33:6841","nonce":645571756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/203598909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":203598909},{"type":"v1","addr":"172.21.15.150:6833","nonce":203598909}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/4029842493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":4029842493},{"type":"v1","addr":"172.21.15.150:6835","nonce":4029842493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3463664423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3463664423},{"type":"v1","addr":"172.21.15.150:6837","nonce":3463664423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24481,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6839/4051648018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":4051648018},{"type":"v1","addr":"172.21.15.150:6839","nonce":4051648018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34418,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1865105479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1865105479},{"type":"v1","addr":"172.21.15.99:6833","nonce":1865105479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34445,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/1206393914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1206393914},{"type":"v1","addr":"172.21.15.99:6835","nonce":1206393914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-03T18:22:28.837091+0000","modified":"2024-05-03T18:23:49.758398+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34508},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34472":{"gid":34472,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6837/3360970342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3360970342},{"type":"v1","addr":"172.21.15.99:6837","nonce":3360970342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34508":{"gid":34508,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/919907308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":919907308},{"type":"v1","addr":"172.21.15.99:6839","nonce":919907308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34508,"qdb_cluster":[34508]},"id":1}]} 2024-05-03T18:42:25.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:25.541+0000 7fcf7fe00700 1 -- 172.21.15.33:0/1585250058 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fcf640b2bc0 msgr2=0x7fcf640b5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:42:25.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:25.541+0000 7fcf7fe00700 1 --2- 172.21.15.33:0/1585250058 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fcf640b2bc0 0x7fcf640b5080 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fcf80069f30 tx=0x7fcf80069ec0 comp rx=0 tx=0).stop 2024-05-03T18:42:25.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:25.541+0000 7fcf7fe00700 1 -- 172.21.15.33:0/1585250058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf780a1670 msgr2=0x7fcf7800f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:42:25.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:25.541+0000 7fcf7fe00700 1 --2- 172.21.15.33:0/1585250058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf780a1670 0x7fcf7800f530 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fcf7000eb80 tx=0x7fcf7000ee90 comp rx=0 tx=0).stop 2024-05-03T18:42:25.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:25.541+0000 7fcf7fe00700 1 -- 172.21.15.33:0/1585250058 shutdown_connections 2024-05-03T18:42:25.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:25.541+0000 7fcf7fe00700 1 --2- 172.21.15.33:0/1585250058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf780a1670 0x7fcf7800f530 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:25.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:25.541+0000 7fcf7fe00700 1 --2- 172.21.15.33:0/1585250058 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fcf640b2bc0 0x7fcf640b5080 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:25.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:25.541+0000 7fcf7fe00700 1 --2- 172.21.15.33:0/1585250058 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf7809c5e0 0x7fcf78015f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:25.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:25.541+0000 7fcf7fe00700 1 --2- 172.21.15.33:0/1585250058 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcf7809cf90 0x7fcf7800eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:25.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:25.541+0000 7fcf7fe00700 1 -- 172.21.15.33:0/1585250058 >> 172.21.15.33:0/1585250058 conn(0x7fcf7800c880 msgr2=0x7fcf78009630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:42:25.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:25.544+0000 7fcf7fe00700 1 -- 172.21.15.33:0/1585250058 shutdown_connections 2024-05-03T18:42:25.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:25.544+0000 7fcf7fe00700 1 -- 172.21.15.33:0/1585250058 wait complete. 2024-05-03T18:42:25.548 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 21 2024-05-03T18:42:25.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:42:25 smithi099 ceph-mon[31658]: pgmap v478: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 7.6 MiB/s wr, 669 op/s 2024-05-03T18:42:25.861 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:42:25 smithi033 ceph-mon[28330]: pgmap v478: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 7.6 MiB/s wr, 669 op/s 2024-05-03T18:42:25.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:42:25 smithi150 ceph-mon[32492]: pgmap v478: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 7.6 MiB/s wr, 669 op/s 2024-05-03T18:42:26.735 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:42:26 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/1585250058' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:42:26.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:42:26 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/1585250058' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:42:26.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:42:26 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/1585250058' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:42:27.870 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:42:27 smithi033 ceph-mon[28330]: pgmap v479: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 7.1 MiB/s wr, 635 op/s 2024-05-03T18:42:27.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:42:27 smithi150 ceph-mon[32492]: pgmap v479: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 7.1 MiB/s wr, 635 op/s 2024-05-03T18:42:27.959 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-03T18:42:28.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:42:27 smithi099 ceph-mon[31658]: pgmap v479: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 7.1 MiB/s wr, 635 op/s 2024-05-03T18:42:28.805 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:42:29.847 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:42:29 smithi033 ceph-mon[28330]: pgmap v480: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 6.3 MiB/s wr, 561 op/s 2024-05-03T18:42:29.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:42:29 smithi150 ceph-mon[32492]: pgmap v480: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 6.3 MiB/s wr, 561 op/s 2024-05-03T18:42:30.051 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:42:29 smithi099 ceph-mon[31658]: pgmap v480: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 6.3 MiB/s wr, 561 op/s 2024-05-03T18:42:30.110 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 675.0 s, 428.8 tps, lat 213.176 ms stddev 136.311 2024-05-03T18:42:31.736 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:42:31 smithi033 ceph-mon[28330]: pgmap v481: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 6.1 MiB/s wr, 552 op/s 2024-05-03T18:42:31.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:42:31 smithi150 ceph-mon[32492]: pgmap v481: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 6.1 MiB/s wr, 552 op/s 2024-05-03T18:42:32.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:42:31 smithi099 ceph-mon[31658]: pgmap v481: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 6.1 MiB/s wr, 552 op/s 2024-05-03T18:42:33.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.466+0000 7ff6d9c00700 1 -- 172.21.15.33:0/3346171496 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff6d406bad0 msgr2=0x7ff6d410fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:42:33.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.466+0000 7ff6d9c00700 1 --2- 172.21.15.33:0/3346171496 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff6d406bad0 0x7ff6d410fa10 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff6c0009990 tx=0x7ff6c0009ca0 comp rx=0 tx=0).stop 2024-05-03T18:42:33.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.467+0000 7ff6d9c00700 1 -- 172.21.15.33:0/3346171496 shutdown_connections 2024-05-03T18:42:33.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.467+0000 7ff6d9c00700 1 --2- 172.21.15.33:0/3346171496 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6d410ff50 0x7ff6d4112340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:33.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.467+0000 7ff6d9c00700 1 --2- 172.21.15.33:0/3346171496 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6d406b1b0 0x7ff6d406b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:33.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.467+0000 7ff6d9c00700 1 --2- 172.21.15.33:0/3346171496 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff6d406bad0 0x7ff6d410fa10 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:33.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.467+0000 7ff6d9c00700 1 -- 172.21.15.33:0/3346171496 >> 172.21.15.33:0/3346171496 conn(0x7ff6d4079350 msgr2=0x7ff6d4079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:42:33.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.468+0000 7ff6d9c00700 1 -- 172.21.15.33:0/3346171496 shutdown_connections 2024-05-03T18:42:33.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.468+0000 7ff6d9c00700 1 -- 172.21.15.33:0/3346171496 wait complete. 2024-05-03T18:42:33.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.469+0000 7ff6d9c00700 1 Processor -- start 2024-05-03T18:42:33.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.470+0000 7ff6d9c00700 1 -- start start 2024-05-03T18:42:33.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.471+0000 7ff6d9c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6d406b1b0 0x7ff6d4109120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:33.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.471+0000 7ff6d9c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff6d406bad0 0x7ff6d4109660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:33.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.471+0000 7ff6d9c00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6d410ff50 0x7ff6d41045b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:33.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.472+0000 7ff6d2000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff6d406bad0 0x7ff6d4109660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:42:33.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.472+0000 7ff6d9c00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d4115820 con 0x7ff6d410ff50 2024-05-03T18:42:33.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.472+0000 7ff6d9c00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d41156a0 con 0x7ff6d406b1b0 2024-05-03T18:42:33.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.472+0000 7ff6d9c00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d41159a0 con 0x7ff6d406bad0 2024-05-03T18:42:33.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.472+0000 7ff6d2a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6d406b1b0 0x7ff6d4109120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:42:33.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.472+0000 7ff6d2000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff6d406bad0 0x7ff6d4109660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:47524/0 (socket says 172.21.15.33:47524) 2024-05-03T18:42:33.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.472+0000 7ff6d2000700 1 -- 172.21.15.33:0/3651144195 learned_addr learned my addr 172.21.15.33:0/3651144195 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:42:33.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.472+0000 7ff6d2a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6d406b1b0 0x7ff6d4109120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:51900/0 (socket says 172.21.15.33:51900) 2024-05-03T18:42:33.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.472+0000 7ff6d3400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6d410ff50 0x7ff6d41045b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:42:33.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.474+0000 7ff6d2000700 1 -- 172.21.15.33:0/3651144195 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6d406b1b0 msgr2=0x7ff6d4109120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:42:33.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.474+0000 7ff6d2000700 1 --2- 172.21.15.33:0/3651144195 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6d406b1b0 0x7ff6d4109120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:33.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.474+0000 7ff6d2000700 1 -- 172.21.15.33:0/3651144195 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6d410ff50 msgr2=0x7ff6d41045b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:42:33.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.474+0000 7ff6d2000700 1 --2- 172.21.15.33:0/3651144195 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6d410ff50 0x7ff6d41045b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:33.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.474+0000 7ff6d2000700 1 -- 172.21.15.33:0/3651144195 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6d4104eb0 con 0x7ff6d406bad0 2024-05-03T18:42:33.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.475+0000 7ff6d2000700 1 --2- 172.21.15.33:0/3651144195 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff6d406bad0 0x7ff6d4109660 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff6c00057f0 tx=0x7ff6c0003680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:42:33.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.476+0000 7ff6d2a00700 1 --2- 172.21.15.33:0/3651144195 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6d406b1b0 0x7ff6d4109120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:42:33.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.476+0000 7ff6d3400700 1 --2- 172.21.15.33:0/3651144195 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6d410ff50 0x7ff6d41045b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:42:33.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.477+0000 7ff6cb400700 1 -- 172.21.15.33:0/3651144195 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6c0003b20 con 0x7ff6d406bad0 2024-05-03T18:42:33.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.477+0000 7ff6cb400700 1 -- 172.21.15.33:0/3651144195 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff6c0004150 con 0x7ff6d406bad0 2024-05-03T18:42:33.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.477+0000 7ff6cb400700 1 -- 172.21.15.33:0/3651144195 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6c000fe40 con 0x7ff6d406bad0 2024-05-03T18:42:33.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.477+0000 7ff6d9c00700 1 -- 172.21.15.33:0/3651144195 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6d4105140 con 0x7ff6d406bad0 2024-05-03T18:42:33.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.478+0000 7ff6d9c00700 1 -- 172.21.15.33:0/3651144195 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6d410f010 con 0x7ff6d406bad0 2024-05-03T18:42:33.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.496+0000 7ff6cb400700 1 -- 172.21.15.33:0/3651144195 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7ff6c000f560 con 0x7ff6d406bad0 2024-05-03T18:42:33.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.502+0000 7ff6cb400700 1 --2- 172.21.15.33:0/3651144195 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7ff6b80b2aa0 0x7ff6b80b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:33.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.502+0000 7ff6cb400700 1 -- 172.21.15.33:0/3651144195 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9502+0+0 (secure 0 0 0) 0x7ff6c00e3a40 con 0x7ff6d406bad0 2024-05-03T18:42:33.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.503+0000 7ff6d2a00700 1 --2- 172.21.15.33:0/3651144195 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7ff6b80b2aa0 0x7ff6b80b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:42:33.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.504+0000 7ff6d2a00700 1 --2- 172.21.15.33:0/3651144195 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7ff6b80b2aa0 0x7ff6b80b4f60 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc007fd0 tx=0x7ff6cc00c040 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:42:33.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.510+0000 7ff6e44d2b80 1 Processor -- start 2024-05-03T18:42:33.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.511+0000 7ff6e44d2b80 1 -- start start 2024-05-03T18:42:33.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.512+0000 7ff6e44d2b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff9f1277b0 0x55ff9f127b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:33.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.512+0000 7ff6e44d2b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ff9f1280d0 0x55ff9f122fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:33.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.512+0000 7ff6e44d2b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff9f1235d0 0x55ff9f123a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:33.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.512+0000 7ff6e44d2b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ff9f075a20 con 0x55ff9f1235d0 2024-05-03T18:42:33.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.512+0000 7ff6e44d2b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55ff9f013760 con 0x55ff9f1277b0 2024-05-03T18:42:33.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.512+0000 7ff6e44d2b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55ff9f0719e0 con 0x55ff9f1280d0 2024-05-03T18:42:33.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.512+0000 7ff6d2000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ff9f1280d0 0x55ff9f122fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:42:33.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.513+0000 7ff6d2000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ff9f1280d0 0x55ff9f122fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:47540/0 (socket says 172.21.15.33:47540) 2024-05-03T18:42:33.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.513+0000 7ff6d2000700 1 -- 172.21.15.33:0/3871023596 learned_addr learned my addr 172.21.15.33:0/3871023596 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:42:33.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.513+0000 7ff6d2a00700 1 --2- 172.21.15.33:0/3871023596 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff9f1277b0 0x55ff9f127b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:42:33.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.514+0000 7ff6d3400700 1 --2- 172.21.15.33:0/3871023596 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff9f1235d0 0x55ff9f123a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:42:33.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.514+0000 7ff6d2000700 1 -- 172.21.15.33:0/3871023596 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff9f1277b0 msgr2=0x55ff9f127b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:42:33.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.514+0000 7ff6d2000700 1 --2- 172.21.15.33:0/3871023596 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff9f1277b0 0x55ff9f127b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:33.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.514+0000 7ff6d2000700 1 -- 172.21.15.33:0/3871023596 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff9f1235d0 msgr2=0x55ff9f123a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:42:33.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.514+0000 7ff6d2000700 1 --2- 172.21.15.33:0/3871023596 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff9f1235d0 0x55ff9f123a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:33.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.514+0000 7ff6d2000700 1 -- 172.21.15.33:0/3871023596 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff9efc28d0 con 0x55ff9f1280d0 2024-05-03T18:42:33.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.515+0000 7ff6d2000700 1 --2- 172.21.15.33:0/3871023596 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ff9f1280d0 0x55ff9f122fb0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff6c00e0e20 tx=0x7ff6c00a69d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7c7c9babbfd78b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:42:33.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.517+0000 7ff6bfe00700 1 -- 172.21.15.33:0/3871023596 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6c00aa4f0 con 0x55ff9f1280d0 2024-05-03T18:42:33.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.517+0000 7ff6bfe00700 1 -- 172.21.15.33:0/3871023596 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff6c00ab0e0 con 0x55ff9f1280d0 2024-05-03T18:42:33.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.517+0000 7ff6bfe00700 1 -- 172.21.15.33:0/3871023596 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6c00053a0 con 0x55ff9f1280d0 2024-05-03T18:42:33.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.519+0000 7ff6e44d2b80 1 -- 172.21.15.33:0/3871023596 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ff9f1280d0 msgr2=0x55ff9f122fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:42:33.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.519+0000 7ff6e44d2b80 1 --2- 172.21.15.33:0/3871023596 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ff9f1280d0 0x55ff9f122fb0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff6c00e0e20 tx=0x7ff6c00a69d0 comp rx=0 tx=0).stop 2024-05-03T18:42:33.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.519+0000 7ff6e44d2b80 1 -- 172.21.15.33:0/3871023596 shutdown_connections 2024-05-03T18:42:33.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.520+0000 7ff6e44d2b80 1 --2- 172.21.15.33:0/3871023596 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff9f1235d0 0x55ff9f123a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:33.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.520+0000 7ff6e44d2b80 1 --2- 172.21.15.33:0/3871023596 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff9f1277b0 0x55ff9f127b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:33.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.520+0000 7ff6e44d2b80 1 --2- 172.21.15.33:0/3871023596 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ff9f1280d0 0x55ff9f122fb0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:33.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.520+0000 7ff6e44d2b80 1 -- 172.21.15.33:0/3871023596 >> 172.21.15.33:0/3871023596 conn(0x55ff9f051b60 msgr2=0x55ff9f11d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:42:33.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.520+0000 7ff6e44d2b80 1 -- 172.21.15.33:0/3871023596 shutdown_connections 2024-05-03T18:42:33.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.521+0000 7ff6e44d2b80 1 -- 172.21.15.33:0/3871023596 wait complete. 2024-05-03T18:42:33.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.522+0000 7ff6e44d2b80 1 -- start start 2024-05-03T18:42:33.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.523+0000 7ff6e44d2b80 1 Processor -- start 2024-05-03T18:42:33.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.524+0000 7ff6e44d2b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff9f1235d0 0x55ff9f12ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:33.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.525+0000 7ff6e44d2b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff9f1277b0 0x55ff9f12c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:33.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.525+0000 7ff6d2a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff9f1235d0 0x55ff9f12ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:42:33.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.525+0000 7ff6e44d2b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ff9f1280d0 0x55ff9f12c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:33.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.525+0000 7ff6d2a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff9f1235d0 0x55ff9f12ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:51926/0 (socket says 172.21.15.33:51926) 2024-05-03T18:42:33.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.526+0000 7ff6e44d2b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ff9f140a70 con 0x55ff9f1277b0 2024-05-03T18:42:33.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.526+0000 7ff6e44d2b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55ff9efdbb20 con 0x55ff9f1235d0 2024-05-03T18:42:33.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.526+0000 7ff6e44d2b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55ff9f140bf0 con 0x55ff9f1280d0 2024-05-03T18:42:33.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.526+0000 7ff6d2a00700 1 -- 172.21.15.33:0/3806233024 learned_addr learned my addr 172.21.15.33:0/3806233024 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:42:33.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.526+0000 7ff6d3400700 1 --2- 172.21.15.33:0/3806233024 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ff9f1280d0 0x55ff9f12c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:42:33.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.526+0000 7ff6e44d2b80 10 client.0 _ll_register_callbacks cb 0x55ff9f01cc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:42:33.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.527+0000 7ff6d2a00700 1 -- 172.21.15.33:0/3806233024 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ff9f1280d0 msgr2=0x55ff9f12c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:42:33.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.527+0000 7ff6d2a00700 1 --2- 172.21.15.33:0/3806233024 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ff9f1280d0 0x55ff9f12c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:33.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.527+0000 7ff6d2a00700 1 -- 172.21.15.33:0/3806233024 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff9f1277b0 msgr2=0x55ff9f12c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:42:33.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.527+0000 7ff6d2a00700 1 --2- 172.21.15.33:0/3806233024 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff9f1277b0 0x55ff9f12c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:33.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.527+0000 7ff6d2a00700 1 -- 172.21.15.33:0/3806233024 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff9f03b420 con 0x55ff9f1235d0 2024-05-03T18:42:33.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.527+0000 7ff6d3400700 1 --2- 172.21.15.33:0/3806233024 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ff9f1280d0 0x55ff9f12c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:42:33.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.528+0000 7ff6d2a00700 1 --2- 172.21.15.33:0/3806233024 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff9f1235d0 0x55ff9f12ef10 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc016d80 tx=0x7ff6cc004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=305ce6d084f81ead server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:42:33.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.528+0000 7ff6bcc00700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-03T18:42:33.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.532+0000 7ff6bcc00700 1 -- 172.21.15.33:0/3806233024 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6cc009c20 con 0x55ff9f1235d0 2024-05-03T18:42:33.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.532+0000 7ff6bcc00700 1 -- 172.21.15.33:0/3806233024 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9502+0+0 (secure 0 0 0) 0x7ff6cc009dc0 con 0x55ff9f1235d0 2024-05-03T18:42:33.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.532+0000 7ff6e44d2b80 1 -- 172.21.15.33:0/3806233024 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6c00ab0e0 con 0x55ff9f1235d0 2024-05-03T18:42:33.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.533+0000 7ff6bcc00700 1 -- 172.21.15.33:0/3806233024 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7ff6cc02c450 con 0x55ff9f1235d0 2024-05-03T18:42:33.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.533+0000 7ff6e44d2b80 10 client.36506 fetch_fsmap learned FSMap version 21 2024-05-03T18:42:33.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.533+0000 7ff6e44d2b80 1 -- 172.21.15.33:0/3806233024 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55ff9f14b330 con 0x55ff9f1235d0 2024-05-03T18:42:33.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.534+0000 7ff6bcc00700 1 -- 172.21.15.33:0/3806233024 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff6c00ab0e0 con 0x55ff9f1235d0 2024-05-03T18:42:33.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.534+0000 7ff6bcc00700 1 -- 172.21.15.33:0/3806233024 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7ff6cc0143c0 con 0x55ff9f1235d0 2024-05-03T18:42:33.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.534+0000 7ff6e44d2b80 10 client.36506 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:42:33.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.534+0000 7ff6e44d2b80 10 client.36506 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:42:33.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.534+0000 7ff6e44d2b80 20 client.36506 populate_metadata read hostname 'smithi033' 2024-05-03T18:42:33.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.534+0000 7ff6e44d2b80 1 --2- 172.21.15.33:0/3806233024 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55ff9f14d970 0x55ff9f14fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:33.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.534+0000 7ff6e44d2b80 4 client.36506 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:42:33.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.534+0000 7ff6e44d2b80 1 -- 172.21.15.33:0/3806233024 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ff9f0f01d0 con 0x55ff9f14d970 2024-05-03T18:42:33.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.535+0000 7ff6d2000700 1 --2- 172.21.15.33:0/3806233024 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55ff9f14d970 0x55ff9f14fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:42:33.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.536+0000 7ff6d2000700 1 --2- 172.21.15.33:0/3806233024 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55ff9f14d970 0x55ff9f14fdb0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99da19f914585655 server_cookie=995e5c5a1e8b6a86 in_seq=0 out_seq=0 2024-05-03T18:42:33.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.536+0000 7ff6bcc00700 10 client.36506 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:42:33.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.539+0000 7ff6bcc00700 1 -- 172.21.15.33:0/3806233024 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ff9f0f01d0 con 0x55ff9f14d970 2024-05-03T18:42:33.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.539+0000 7ff6bcc00700 10 client.36506 handle_command_reply: tid=0 2024-05-03T18:42:33.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.539+0000 7ff6e44d2b80 1 client.36506 shutdown 2024-05-03T18:42:33.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.540+0000 7ff6e44d2b80 1 -- 172.21.15.33:0/3806233024 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff9f1235d0 msgr2=0x55ff9f12ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:42:33.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.540+0000 7ff6e44d2b80 1 --2- 172.21.15.33:0/3806233024 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff9f1235d0 0x55ff9f12ef10 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc016d80 tx=0x7ff6cc004ad0 comp rx=0 tx=0).stop 2024-05-03T18:42:33.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.540+0000 7ff6e44d2b80 1 -- 172.21.15.33:0/3806233024 shutdown_connections 2024-05-03T18:42:33.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.540+0000 7ff6e44d2b80 1 --2- 172.21.15.33:0/3806233024 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55ff9f14d970 0x55ff9f14fdb0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:33.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.540+0000 7ff6e44d2b80 1 --2- 172.21.15.33:0/3806233024 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ff9f1280d0 0x55ff9f12c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:33.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.540+0000 7ff6e44d2b80 1 --2- 172.21.15.33:0/3806233024 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff9f1235d0 0x55ff9f12ef10 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:33.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.540+0000 7ff6e44d2b80 1 --2- 172.21.15.33:0/3806233024 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff9f1277b0 0x55ff9f12c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:33.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.540+0000 7ff6e44d2b80 1 -- 172.21.15.33:0/3806233024 >> 172.21.15.33:0/3806233024 conn(0x55ff9f051b60 msgr2=0x55ff9f128e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:42:33.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.541+0000 7ff6bcc00700 0 client.36506 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:42:33.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.541+0000 7ff6e44d2b80 1 -- 172.21.15.33:0/3806233024 shutdown_connections 2024-05-03T18:42:33.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.541+0000 7ff6e44d2b80 1 -- 172.21.15.33:0/3806233024 wait complete. 2024-05-03T18:42:33.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.542+0000 7ff6e44d2b80 20 client.36506 trim_cache size 0 max 16384 2024-05-03T18:42:33.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.572+0000 7ff6e44d2b80 1 Processor -- start 2024-05-03T18:42:33.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.572+0000 7ff6e44d2b80 1 -- start start 2024-05-03T18:42:33.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.572+0000 7ff6e44d2b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff9f1235d0 0x55ff9f12b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:33.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.572+0000 7ff6e44d2b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff9f1277b0 0x55ff9f149ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:33.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.573+0000 7ff6e44d2b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ff9f1280d0 0x55ff9f11d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:33.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.573+0000 7ff6e44d2b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d4072c50 con 0x55ff9f1277b0 2024-05-03T18:42:33.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.573+0000 7ff6e44d2b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d40754b0 con 0x55ff9f1235d0 2024-05-03T18:42:33.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.573+0000 7ff6e44d2b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d40732a0 con 0x55ff9f1280d0 2024-05-03T18:42:33.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.573+0000 7ff6d2000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff9f1277b0 0x55ff9f149ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:42:33.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.573+0000 7ff6d2a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff9f1235d0 0x55ff9f12b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:42:33.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.573+0000 7ff6d2000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff9f1277b0 0x55ff9f149ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54692/0 (socket says 172.21.15.33:54692) 2024-05-03T18:42:33.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.573+0000 7ff6d2000700 1 -- 172.21.15.33:0/2394048713 learned_addr learned my addr 172.21.15.33:0/2394048713 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:42:33.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.575+0000 7ff6d2000700 1 -- 172.21.15.33:0/2394048713 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ff9f1280d0 msgr2=0x55ff9f11d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:42:33.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.575+0000 7ff6d3400700 1 --2- 172.21.15.33:0/2394048713 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ff9f1280d0 0x55ff9f11d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:42:33.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.575+0000 7ff6d2000700 1 --2- 172.21.15.33:0/2394048713 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ff9f1280d0 0x55ff9f11d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:33.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.575+0000 7ff6d2000700 1 -- 172.21.15.33:0/2394048713 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff9f1235d0 msgr2=0x55ff9f12b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:42:33.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.575+0000 7ff6d2000700 1 --2- 172.21.15.33:0/2394048713 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff9f1235d0 0x55ff9f12b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:33.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.575+0000 7ff6d2000700 1 -- 172.21.15.33:0/2394048713 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff9f127400 con 0x55ff9f1277b0 2024-05-03T18:42:33.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.575+0000 7ff6d2a00700 1 --2- 172.21.15.33:0/2394048713 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff9f1235d0 0x55ff9f12b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:42:33.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.575+0000 7ff6d3400700 1 --2- 172.21.15.33:0/2394048713 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ff9f1280d0 0x55ff9f11d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:42:33.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.575+0000 7ff6d2000700 1 --2- 172.21.15.33:0/2394048713 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff9f1277b0 0x55ff9f149ac0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7ff6c00057c0 tx=0x7ff6c00a6930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c820451e5cb5b270 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:42:33.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.577+0000 7ff6bf400700 1 -- 172.21.15.33:0/2394048713 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6c00aa9e0 con 0x55ff9f1277b0 2024-05-03T18:42:33.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.577+0000 7ff6bf400700 1 -- 172.21.15.33:0/2394048713 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff6c00aed90 con 0x55ff9f1277b0 2024-05-03T18:42:33.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.577+0000 7ff6bf400700 1 -- 172.21.15.33:0/2394048713 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6c00b12a0 con 0x55ff9f1277b0 2024-05-03T18:42:33.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.578+0000 7ff6e44d2b80 1 -- 172.21.15.33:0/2394048713 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff9f1277b0 msgr2=0x55ff9f149ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:42:33.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.578+0000 7ff6e44d2b80 1 --2- 172.21.15.33:0/2394048713 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff9f1277b0 0x55ff9f149ac0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7ff6c00057c0 tx=0x7ff6c00a6930 comp rx=0 tx=0).stop 2024-05-03T18:42:33.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.580+0000 7ff6e44d2b80 1 -- 172.21.15.33:0/2394048713 shutdown_connections 2024-05-03T18:42:33.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.580+0000 7ff6e44d2b80 1 --2- 172.21.15.33:0/2394048713 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ff9f1280d0 0x55ff9f11d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:33.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.580+0000 7ff6e44d2b80 1 --2- 172.21.15.33:0/2394048713 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff9f1235d0 0x55ff9f12b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:33.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.580+0000 7ff6e44d2b80 1 --2- 172.21.15.33:0/2394048713 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff9f1277b0 0x55ff9f149ac0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:33.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.580+0000 7ff6e44d2b80 1 -- 172.21.15.33:0/2394048713 >> 172.21.15.33:0/2394048713 conn(0x55ff9f051b60 msgr2=0x55ff9f122c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:42:33.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.580+0000 7ff6e44d2b80 1 -- 172.21.15.33:0/2394048713 shutdown_connections 2024-05-03T18:42:33.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.580+0000 7ff6e44d2b80 1 -- 172.21.15.33:0/2394048713 wait complete. 2024-05-03T18:42:33.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.581+0000 7ff6e44d2b80 1 -- start start 2024-05-03T18:42:33.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.581+0000 7ff6e44d2b80 1 Processor -- start 2024-05-03T18:42:33.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.586+0000 7ff6e44d2b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff9f1235d0 0x55ff9f146300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:33.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.586+0000 7ff6e44d2b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ff9f1277b0 0x55ff9f120860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:33.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.586+0000 7ff6e44d2b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff9f1280d0 0x55ff9f120da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:33.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.586+0000 7ff6d2a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff9f1235d0 0x55ff9f146300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:42:33.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.586+0000 7ff6d2000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ff9f1277b0 0x55ff9f120860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:42:33.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.586+0000 7ff6d2a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff9f1235d0 0x55ff9f146300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:51946/0 (socket says 172.21.15.33:51946) 2024-05-03T18:42:33.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.587+0000 7ff6d2000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ff9f1277b0 0x55ff9f120860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:47582/0 (socket says 172.21.15.33:47582) 2024-05-03T18:42:33.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.587+0000 7ff6d2000700 1 -- 172.21.15.33:0/1343807195 learned_addr learned my addr 172.21.15.33:0/1343807195 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:42:33.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.587+0000 7ff6e44d2b80 1 -- 172.21.15.33:0/1343807195 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d4073970 con 0x55ff9f1280d0 2024-05-03T18:42:33.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.587+0000 7ff6e44d2b80 1 -- 172.21.15.33:0/1343807195 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d4074de0 con 0x55ff9f1235d0 2024-05-03T18:42:33.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.587+0000 7ff6e44d2b80 1 -- 172.21.15.33:0/1343807195 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d4074040 con 0x55ff9f1277b0 2024-05-03T18:42:33.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.587+0000 7ff6e44d2b80 10 client.0 _ll_register_callbacks cb 0x55ff9f01cc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:42:33.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.587+0000 7ff6d3400700 1 --2- 172.21.15.33:0/1343807195 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff9f1280d0 0x55ff9f120da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:42:33.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.587+0000 7ff6d2000700 1 -- 172.21.15.33:0/1343807195 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff9f1235d0 msgr2=0x55ff9f146300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:42:33.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.587+0000 7ff6d2000700 1 --2- 172.21.15.33:0/1343807195 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff9f1235d0 0x55ff9f146300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:33.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.587+0000 7ff6d2000700 1 -- 172.21.15.33:0/1343807195 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff9f1280d0 msgr2=0x55ff9f120da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:42:33.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.587+0000 7ff6d2000700 1 --2- 172.21.15.33:0/1343807195 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff9f1280d0 0x55ff9f120da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:33.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.587+0000 7ff6d2000700 1 -- 172.21.15.33:0/1343807195 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff9f1215e0 con 0x55ff9f1277b0 2024-05-03T18:42:33.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.588+0000 7ff6d2000700 1 --2- 172.21.15.33:0/1343807195 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ff9f1277b0 0x55ff9f120860 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ff6c00aef50 tx=0x7ff6c000f850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2b99805a273869b server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:42:33.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.588+0000 7ff6bcc00700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-03T18:42:33.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.589+0000 7ff6bcc00700 1 -- 172.21.15.33:0/1343807195 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6c008f070 con 0x55ff9f1277b0 2024-05-03T18:42:33.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.589+0000 7ff6bcc00700 1 -- 172.21.15.33:0/1343807195 <== mon.1 v2:172.21.15.150:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9502+0+0 (secure 0 0 0) 0x7ff6c00cd070 con 0x55ff9f1277b0 2024-05-03T18:42:33.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.589+0000 7ff6e44d2b80 1 -- 172.21.15.33:0/1343807195 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ff9f083400 con 0x55ff9f1277b0 2024-05-03T18:42:33.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.595+0000 7ff6bcc00700 1 -- 172.21.15.33:0/1343807195 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7ff6c00cd270 con 0x55ff9f1277b0 2024-05-03T18:42:33.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.595+0000 7ff6e44d2b80 10 client.26593 fetch_fsmap learned FSMap version 21 2024-05-03T18:42:33.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.595+0000 7ff6e44d2b80 1 -- 172.21.15.33:0/1343807195 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55ff9f1b8150 con 0x55ff9f1277b0 2024-05-03T18:42:33.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.598+0000 7ff6bcc00700 1 -- 172.21.15.33:0/1343807195 <== mon.1 v2:172.21.15.150:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55ff9f083400 con 0x55ff9f1277b0 2024-05-03T18:42:33.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.598+0000 7ff6bcc00700 1 -- 172.21.15.33:0/1343807195 <== mon.1 v2:172.21.15.150:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7ff6c00b3320 con 0x55ff9f1277b0 2024-05-03T18:42:33.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.598+0000 7ff6e44d2b80 10 client.26593 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:42:33.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.598+0000 7ff6e44d2b80 10 client.26593 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:42:33.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.598+0000 7ff6e44d2b80 20 client.26593 populate_metadata read hostname 'smithi033' 2024-05-03T18:42:33.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.598+0000 7ff6e44d2b80 1 --2- 172.21.15.33:0/1343807195 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55ff9f1b8480 0x55ff9f1ba8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:33.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.598+0000 7ff6e44d2b80 4 client.26593 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-03T18:42:33.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.598+0000 7ff6e44d2b80 1 -- 172.21.15.33:0/1343807195 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ff9f07d4a0 con 0x55ff9f1b8480 2024-05-03T18:42:33.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.599+0000 7ff6d2a00700 1 --2- 172.21.15.33:0/1343807195 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55ff9f1b8480 0x55ff9f1ba8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:42:33.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.599+0000 7ff6d2a00700 1 --2- 172.21.15.33:0/1343807195 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55ff9f1b8480 0x55ff9f1ba8a0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82a2a429fd0743af server_cookie=e2204cf164b94abf in_seq=0 out_seq=0 2024-05-03T18:42:33.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.599+0000 7ff6bcc00700 10 client.26593 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:42:33.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.600+0000 7ff6bcc00700 1 -- 172.21.15.33:0/1343807195 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ff9f07d4a0 con 0x55ff9f1b8480 2024-05-03T18:42:33.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.600+0000 7ff6bcc00700 10 client.26593 handle_command_reply: tid=0 2024-05-03T18:42:33.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.600+0000 7ff6e44d2b80 1 client.26593 shutdown 2024-05-03T18:42:33.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.600+0000 7ff6e44d2b80 1 -- 172.21.15.33:0/1343807195 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ff9f1277b0 msgr2=0x55ff9f120860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:42:33.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.600+0000 7ff6e44d2b80 1 --2- 172.21.15.33:0/1343807195 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ff9f1277b0 0x55ff9f120860 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ff6c00aef50 tx=0x7ff6c000f850 comp rx=0 tx=0).stop 2024-05-03T18:42:33.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.601+0000 7ff6e44d2b80 1 -- 172.21.15.33:0/1343807195 shutdown_connections 2024-05-03T18:42:33.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.601+0000 7ff6e44d2b80 1 --2- 172.21.15.33:0/1343807195 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55ff9f1b8480 0x55ff9f1ba8a0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:33.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.601+0000 7ff6e44d2b80 1 --2- 172.21.15.33:0/1343807195 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff9f1280d0 0x55ff9f120da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:33.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.601+0000 7ff6e44d2b80 1 --2- 172.21.15.33:0/1343807195 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff9f1235d0 0x55ff9f146300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:33.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.601+0000 7ff6e44d2b80 1 --2- 172.21.15.33:0/1343807195 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ff9f1277b0 0x55ff9f120860 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:33.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.601+0000 7ff6e44d2b80 1 -- 172.21.15.33:0/1343807195 >> 172.21.15.33:0/1343807195 conn(0x55ff9f051b60 msgr2=0x55ff9f1223d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:42:33.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.601+0000 7ff6bcc00700 0 client.26593 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:42:33.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.601+0000 7ff6e44d2b80 1 -- 172.21.15.33:0/1343807195 shutdown_connections 2024-05-03T18:42:33.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.602+0000 7ff6e44d2b80 1 -- 172.21.15.33:0/1343807195 wait complete. 2024-05-03T18:42:33.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.602+0000 7ff6e44d2b80 20 client.26593 trim_cache size 0 max 16384 2024-05-03T18:42:33.609 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-03T18:42:33.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.611+0000 7ff6d9c00700 1 -- 172.21.15.33:0/3651144195 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7ff6b80b2aa0 msgr2=0x7ff6b80b4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:42:33.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.611+0000 7ff6d9c00700 1 --2- 172.21.15.33:0/3651144195 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7ff6b80b2aa0 0x7ff6b80b4f60 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc007fd0 tx=0x7ff6cc00c040 comp rx=0 tx=0).stop 2024-05-03T18:42:33.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.611+0000 7ff6d9c00700 1 -- 172.21.15.33:0/3651144195 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff6d406bad0 msgr2=0x7ff6d4109660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:42:33.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.611+0000 7ff6d9c00700 1 --2- 172.21.15.33:0/3651144195 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff6d406bad0 0x7ff6d4109660 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff6c00057f0 tx=0x7ff6c0003680 comp rx=0 tx=0).stop 2024-05-03T18:42:33.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.612+0000 7ff6d9c00700 1 -- 172.21.15.33:0/3651144195 shutdown_connections 2024-05-03T18:42:33.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.612+0000 7ff6d9c00700 1 --2- 172.21.15.33:0/3651144195 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6d410ff50 0x7ff6d41045b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:33.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.612+0000 7ff6d9c00700 1 --2- 172.21.15.33:0/3651144195 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7ff6b80b2aa0 0x7ff6b80b4f60 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:33.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.612+0000 7ff6d9c00700 1 --2- 172.21.15.33:0/3651144195 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6d406b1b0 0x7ff6d4109120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:33.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.612+0000 7ff6d9c00700 1 --2- 172.21.15.33:0/3651144195 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff6d406bad0 0x7ff6d4109660 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:33.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.612+0000 7ff6d9c00700 1 -- 172.21.15.33:0/3651144195 >> 172.21.15.33:0/3651144195 conn(0x7ff6d4079350 msgr2=0x7ff6d4111480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:42:33.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.612+0000 7ff6d9c00700 1 -- 172.21.15.33:0/3651144195 shutdown_connections 2024-05-03T18:42:33.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:33.612+0000 7ff6d9c00700 1 -- 172.21.15.33:0/3651144195 wait complete. 2024-05-03T18:42:33.735 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:42:33 smithi033 ceph-mon[28330]: pgmap v482: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 6.1 MiB/s wr, 552 op/s 2024-05-03T18:42:33.884 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:42:33 smithi150 ceph-mon[32492]: pgmap v482: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 6.1 MiB/s wr, 552 op/s 2024-05-03T18:42:34.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:42:33 smithi099 ceph-mon[31658]: pgmap v482: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 6.1 MiB/s wr, 552 op/s 2024-05-03T18:42:35.110 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 680.0 s, 381.8 tps, lat 282.876 ms stddev 427.393 2024-05-03T18:42:35.801 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-03T18:42:35.801 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-03T18:42:35.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:42:35 smithi150 ceph-mon[32492]: pgmap v483: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 7.9 MiB/s wr, 708 op/s 2024-05-03T18:42:35.908 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:42:35 smithi033 ceph-mon[28330]: pgmap v483: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 7.9 MiB/s wr, 708 op/s 2024-05-03T18:42:36.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:42:35 smithi099 ceph-mon[31658]: pgmap v483: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 7.9 MiB/s wr, 708 op/s 2024-05-03T18:42:36.577 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:42:37.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:42:37 smithi150 ceph-mon[32492]: pgmap v484: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 5.4 MiB/s wr, 494 op/s 2024-05-03T18:42:37.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:42:37 smithi033 ceph-mon[28330]: pgmap v484: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 5.4 MiB/s wr, 494 op/s 2024-05-03T18:42:38.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:42:37 smithi099 ceph-mon[31658]: pgmap v484: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 5.4 MiB/s wr, 494 op/s 2024-05-03T18:42:39.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.113+0000 7f0c0c800700 1 -- 172.21.15.33:0/3349917701 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0c0406b180 msgr2=0x7f0c0406b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:42:39.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.113+0000 7f0c0c800700 1 --2- 172.21.15.33:0/3349917701 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0c0406b180 0x7f0c0406b600 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0bf8009990 tx=0x7f0bf8009ca0 comp rx=0 tx=0).stop 2024-05-03T18:42:39.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.116+0000 7f0c0c800700 1 -- 172.21.15.33:0/3349917701 shutdown_connections 2024-05-03T18:42:39.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.116+0000 7f0c0c800700 1 --2- 172.21.15.33:0/3349917701 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c0406bb40 0x7f0c04110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:39.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.116+0000 7f0c0c800700 1 --2- 172.21.15.33:0/3349917701 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0c04109510 0x7f0c041098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:39.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.116+0000 7f0c0c800700 1 --2- 172.21.15.33:0/3349917701 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0c0406b180 0x7f0c0406b600 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:39.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.116+0000 7f0c0c800700 1 -- 172.21.15.33:0/3349917701 >> 172.21.15.33:0/3349917701 conn(0x7f0c04079350 msgr2=0x7f0c04079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:42:39.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.116+0000 7f0c0c800700 1 -- 172.21.15.33:0/3349917701 shutdown_connections 2024-05-03T18:42:39.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.116+0000 7f0c0c800700 1 -- 172.21.15.33:0/3349917701 wait complete. 2024-05-03T18:42:39.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.118+0000 7f0c0c800700 1 Processor -- start 2024-05-03T18:42:39.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.118+0000 7f0c0c800700 1 -- start start 2024-05-03T18:42:39.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.119+0000 7f0c0c800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c0406b180 0x7f0c0410a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:39.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.119+0000 7f0c0c800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0c0406bb40 0x7f0c0410a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:39.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.119+0000 7f0c0c800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0c04109510 0x7f0c0410ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:39.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.119+0000 7f0c0c800700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0c041135c0 con 0x7f0c0406b180 2024-05-03T18:42:39.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.119+0000 7f0c0c800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0c04113440 con 0x7f0c04109510 2024-05-03T18:42:39.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.119+0000 7f0c0c800700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f0c04113740 con 0x7f0c0406bb40 2024-05-03T18:42:39.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.120+0000 7f0c0a800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0c04109510 0x7f0c0410ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:42:39.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.120+0000 7f0c0a800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0c04109510 0x7f0c0410ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:38758/0 (socket says 172.21.15.33:38758) 2024-05-03T18:42:39.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.120+0000 7f0c0a800700 1 -- 172.21.15.33:0/1125569229 learned_addr learned my addr 172.21.15.33:0/1125569229 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:42:39.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.120+0000 7f0c09400700 1 --2- 172.21.15.33:0/1125569229 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0c0406bb40 0x7f0c0410a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:42:39.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.121+0000 7f0c09e00700 1 --2- 172.21.15.33:0/1125569229 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c0406b180 0x7f0c0410a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:42:39.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.122+0000 7f0c0a800700 1 -- 172.21.15.33:0/1125569229 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0c0406bb40 msgr2=0x7f0c0410a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:42:39.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.122+0000 7f0c0a800700 1 --2- 172.21.15.33:0/1125569229 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0c0406bb40 0x7f0c0410a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:39.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.122+0000 7f0c0a800700 1 -- 172.21.15.33:0/1125569229 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c0406b180 msgr2=0x7f0c0410a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:42:39.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.122+0000 7f0c0a800700 1 --2- 172.21.15.33:0/1125569229 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c0406b180 0x7f0c0410a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:39.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.122+0000 7f0c0a800700 1 -- 172.21.15.33:0/1125569229 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c0419bb50 con 0x7f0c04109510 2024-05-03T18:42:39.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.122+0000 7f0c09e00700 1 --2- 172.21.15.33:0/1125569229 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c0406b180 0x7f0c0410a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:42:39.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.123+0000 7f0c0a800700 1 --2- 172.21.15.33:0/1125569229 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0c04109510 0x7f0c0410ada0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f0c0000ef60 tx=0x7f0c0000ef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:42:39.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.123+0000 7f0bfea00700 1 -- 172.21.15.33:0/1125569229 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c000092e0 con 0x7f0c04109510 2024-05-03T18:42:39.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.124+0000 7f0bfea00700 1 -- 172.21.15.33:0/1125569229 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0c00004bb0 con 0x7f0c04109510 2024-05-03T18:42:39.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.124+0000 7f0bfea00700 1 -- 172.21.15.33:0/1125569229 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c000092e0 con 0x7f0c04109510 2024-05-03T18:42:39.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.124+0000 7f0c0c800700 1 -- 172.21.15.33:0/1125569229 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c0419bde0 con 0x7f0c04109510 2024-05-03T18:42:39.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.124+0000 7f0c0c800700 1 -- 172.21.15.33:0/1125569229 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c0419c1e0 con 0x7f0c04109510 2024-05-03T18:42:39.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.141+0000 7f0bfea00700 1 -- 172.21.15.33:0/1125569229 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f0c0001b030 con 0x7f0c04109510 2024-05-03T18:42:39.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.143+0000 7f0bfea00700 1 --2- 172.21.15.33:0/1125569229 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f0bec0b2c70 0x7f0bec0b5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:39.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.144+0000 7f0c09e00700 1 --2- 172.21.15.33:0/1125569229 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f0bec0b2c70 0x7f0bec0b5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:42:39.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.144+0000 7f0bfea00700 1 -- 172.21.15.33:0/1125569229 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9502+0+0 (secure 0 0 0) 0x7f0c000e2050 con 0x7f0c04109510 2024-05-03T18:42:39.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.146+0000 7f0c09e00700 1 --2- 172.21.15.33:0/1125569229 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f0bec0b2c70 0x7f0bec0b5130 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0bf4005fd0 tx=0x7f0bf4005db0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:42:39.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.160+0000 7f0c1710ab80 1 Processor -- start 2024-05-03T18:42:39.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.161+0000 7f0c1710ab80 1 -- start start 2024-05-03T18:42:39.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.162+0000 7f0c1710ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5581278889b0 0x558127888d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:39.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.162+0000 7f0c1710ab80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5581278892d0 0x558127882fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:39.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.162+0000 7f0c1710ab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5581278835d0 0x558127883a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:39.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.162+0000 7f0c09e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5581278889b0 0x558127888d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:42:39.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.162+0000 7f0c09e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5581278889b0 0x558127888d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:38774/0 (socket says 172.21.15.33:38774) 2024-05-03T18:42:39.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.162+0000 7f0c1710ab80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5581277d5aa0 con 0x5581278892d0 2024-05-03T18:42:39.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.162+0000 7f0c1710ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5581277737e0 con 0x5581278889b0 2024-05-03T18:42:39.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.162+0000 7f0c1710ab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5581277d1a60 con 0x5581278835d0 2024-05-03T18:42:39.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.162+0000 7f0c09e00700 1 -- 172.21.15.33:0/3110744985 learned_addr learned my addr 172.21.15.33:0/3110744985 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:42:39.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.163+0000 7f0c09400700 1 --2- 172.21.15.33:0/3110744985 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5581278892d0 0x558127882fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:42:39.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.164+0000 7f0c0a800700 1 --2- 172.21.15.33:0/3110744985 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5581278835d0 0x558127883a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:42:39.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.165+0000 7f0c0a800700 1 -- 172.21.15.33:0/3110744985 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5581278889b0 msgr2=0x558127888d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:42:39.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.165+0000 7f0c0a800700 1 --2- 172.21.15.33:0/3110744985 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5581278889b0 0x558127888d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:39.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.165+0000 7f0c0a800700 1 -- 172.21.15.33:0/3110744985 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5581278892d0 msgr2=0x558127882fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:42:39.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.165+0000 7f0c0a800700 1 --2- 172.21.15.33:0/3110744985 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5581278892d0 0x558127882fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:39.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.165+0000 7f0c0a800700 1 -- 172.21.15.33:0/3110744985 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558127722950 con 0x5581278835d0 2024-05-03T18:42:39.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.167+0000 7f0c0a800700 1 --2- 172.21.15.33:0/3110744985 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5581278835d0 0x558127883a50 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f0c000df3d0 tx=0x7f0c000a8bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=719b188fb46844c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:42:39.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.168+0000 7f0bf3400700 1 -- 172.21.15.33:0/3110744985 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c000ad3a0 con 0x5581278835d0 2024-05-03T18:42:39.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.168+0000 7f0bf3400700 1 -- 172.21.15.33:0/3110744985 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0c000b1dc0 con 0x5581278835d0 2024-05-03T18:42:39.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.168+0000 7f0bf3400700 1 -- 172.21.15.33:0/3110744985 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c000b1b90 con 0x5581278835d0 2024-05-03T18:42:39.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.169+0000 7f0c1710ab80 1 -- 172.21.15.33:0/3110744985 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5581278835d0 msgr2=0x558127883a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:42:39.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.169+0000 7f0c1710ab80 1 --2- 172.21.15.33:0/3110744985 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5581278835d0 0x558127883a50 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f0c000df3d0 tx=0x7f0c000a8bd0 comp rx=0 tx=0).stop 2024-05-03T18:42:39.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.169+0000 7f0c1710ab80 1 -- 172.21.15.33:0/3110744985 shutdown_connections 2024-05-03T18:42:39.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.169+0000 7f0c1710ab80 1 --2- 172.21.15.33:0/3110744985 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5581278835d0 0x558127883a50 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:39.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.170+0000 7f0c1710ab80 1 --2- 172.21.15.33:0/3110744985 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5581278889b0 0x558127888d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:39.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.170+0000 7f0c1710ab80 1 --2- 172.21.15.33:0/3110744985 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5581278892d0 0x558127882fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:39.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.170+0000 7f0c1710ab80 1 -- 172.21.15.33:0/3110744985 >> 172.21.15.33:0/3110744985 conn(0x5581277fb750 msgr2=0x55812787d980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:42:39.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.170+0000 7f0c1710ab80 1 -- 172.21.15.33:0/3110744985 shutdown_connections 2024-05-03T18:42:39.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.170+0000 7f0c1710ab80 1 -- 172.21.15.33:0/3110744985 wait complete. 2024-05-03T18:42:39.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.171+0000 7f0c1710ab80 1 -- start start 2024-05-03T18:42:39.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.171+0000 7f0c1710ab80 1 Processor -- start 2024-05-03T18:42:39.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.173+0000 7f0c1710ab80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5581278835d0 0x55812788efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:39.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.173+0000 7f0c1710ab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5581278889b0 0x55812788c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:39.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.173+0000 7f0c1710ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5581278892d0 0x55812788c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:39.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.175+0000 7f0c09e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5581278835d0 0x55812788efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:42:39.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.175+0000 7f0c1710ab80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55812788fd80 con 0x5581278835d0 2024-05-03T18:42:39.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.175+0000 7f0c1710ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55812773bba0 con 0x5581278892d0 2024-05-03T18:42:39.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.175+0000 7f0c1710ab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55812788ff00 con 0x5581278889b0 2024-05-03T18:42:39.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.175+0000 7f0c09e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5581278835d0 0x55812788efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34394/0 (socket says 172.21.15.33:34394) 2024-05-03T18:42:39.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.175+0000 7f0c09e00700 1 -- 172.21.15.33:0/169724860 learned_addr learned my addr 172.21.15.33:0/169724860 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:42:39.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.175+0000 7f0c09400700 1 --2- 172.21.15.33:0/169724860 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5581278889b0 0x55812788c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:42:39.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.176+0000 7f0c1710ab80 10 client.0 _ll_register_callbacks cb 0x55812777ccf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:42:39.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.176+0000 7f0c0a800700 1 --2- 172.21.15.33:0/169724860 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5581278892d0 0x55812788c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:42:39.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.176+0000 7f0c09e00700 1 -- 172.21.15.33:0/169724860 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5581278889b0 msgr2=0x55812788c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:42:39.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.176+0000 7f0c09e00700 1 --2- 172.21.15.33:0/169724860 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5581278889b0 0x55812788c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:39.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.176+0000 7f0c09e00700 1 -- 172.21.15.33:0/169724860 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5581278892d0 msgr2=0x55812788c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:42:39.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.176+0000 7f0c09e00700 1 --2- 172.21.15.33:0/169724860 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5581278892d0 0x55812788c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:39.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.176+0000 7f0c09e00700 1 -- 172.21.15.33:0/169724860 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55812779b4a0 con 0x5581278835d0 2024-05-03T18:42:39.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.178+0000 7f0c09e00700 1 --2- 172.21.15.33:0/169724860 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5581278835d0 0x55812788efd0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f0bf40109e0 tx=0x7f0bf4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f53e8a7d64f60e79 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:42:39.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.178+0000 7f0be7e00700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:42:39.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.179+0000 7f0be7e00700 1 -- 172.21.15.33:0/169724860 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf401dc60 con 0x5581278835d0 2024-05-03T18:42:39.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.179+0000 7f0be7e00700 1 -- 172.21.15.33:0/169724860 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9502+0+0 (secure 0 0 0) 0x7f0bf401de00 con 0x5581278835d0 2024-05-03T18:42:39.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.179+0000 7f0c1710ab80 1 -- 172.21.15.33:0/169724860 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c000b1dc0 con 0x5581278835d0 2024-05-03T18:42:39.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.181+0000 7f0be7e00700 1 -- 172.21.15.33:0/169724860 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f0bf4010e50 con 0x5581278835d0 2024-05-03T18:42:39.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.181+0000 7f0c1710ab80 10 client.16764 fetch_fsmap learned FSMap version 21 2024-05-03T18:42:39.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.181+0000 7f0c1710ab80 1 -- 172.21.15.33:0/169724860 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55812789a5e0 con 0x5581278835d0 2024-05-03T18:42:39.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.181+0000 7f0be7e00700 1 -- 172.21.15.33:0/169724860 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0c000b1dc0 con 0x5581278835d0 2024-05-03T18:42:39.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.181+0000 7f0be7e00700 1 -- 172.21.15.33:0/169724860 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f0bf4005450 con 0x5581278835d0 2024-05-03T18:42:39.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.183+0000 7f0c1710ab80 10 client.16764 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:42:39.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.183+0000 7f0c1710ab80 10 client.16764 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:42:39.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.184+0000 7f0c1710ab80 20 client.16764 populate_metadata read hostname 'smithi033' 2024-05-03T18:42:39.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.184+0000 7f0c1710ab80 1 --2- 172.21.15.33:0/169724860 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55812789cb70 0x55812789efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:39.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.184+0000 7f0c1710ab80 4 client.16764 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:42:39.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.184+0000 7f0c1710ab80 1 -- 172.21.15.33:0/169724860 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558127850070 con 0x55812789cb70 2024-05-03T18:42:39.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.184+0000 7f0c09400700 1 --2- 172.21.15.33:0/169724860 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55812789cb70 0x55812789efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:42:39.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.187+0000 7f0c09400700 1 --2- 172.21.15.33:0/169724860 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55812789cb70 0x55812789efa0 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d140e4e9b1e90f20 server_cookie=402910330f2a1027 in_seq=0 out_seq=0 2024-05-03T18:42:39.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.187+0000 7f0be7e00700 10 client.16764 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:42:39.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.188+0000 7f0be7e00700 1 -- 172.21.15.33:0/169724860 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558127850070 con 0x55812789cb70 2024-05-03T18:42:39.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.188+0000 7f0be7e00700 10 client.16764 handle_command_reply: tid=0 2024-05-03T18:42:39.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.188+0000 7f0c1710ab80 1 client.16764 shutdown 2024-05-03T18:42:39.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.189+0000 7f0c1710ab80 1 -- 172.21.15.33:0/169724860 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5581278835d0 msgr2=0x55812788efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:42:39.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.189+0000 7f0c1710ab80 1 --2- 172.21.15.33:0/169724860 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5581278835d0 0x55812788efd0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f0bf40109e0 tx=0x7f0bf4004750 comp rx=0 tx=0).stop 2024-05-03T18:42:39.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.189+0000 7f0c1710ab80 1 -- 172.21.15.33:0/169724860 shutdown_connections 2024-05-03T18:42:39.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.189+0000 7f0c1710ab80 1 --2- 172.21.15.33:0/169724860 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55812789cb70 0x55812789efa0 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:39.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.189+0000 7f0c1710ab80 1 --2- 172.21.15.33:0/169724860 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5581278892d0 0x55812788c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:39.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.189+0000 7f0c1710ab80 1 --2- 172.21.15.33:0/169724860 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5581278835d0 0x55812788efd0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:39.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.189+0000 7f0be7e00700 0 client.16764 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:42:39.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.190+0000 7f0c1710ab80 1 --2- 172.21.15.33:0/169724860 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5581278889b0 0x55812788c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:39.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.190+0000 7f0c1710ab80 1 -- 172.21.15.33:0/169724860 >> 172.21.15.33:0/169724860 conn(0x5581277fb750 msgr2=0x558127878430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:42:39.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.190+0000 7f0c1710ab80 1 -- 172.21.15.33:0/169724860 shutdown_connections 2024-05-03T18:42:39.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.190+0000 7f0c1710ab80 1 -- 172.21.15.33:0/169724860 wait complete. 2024-05-03T18:42:39.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.190+0000 7f0c1710ab80 20 client.16764 trim_cache size 0 max 16384 2024-05-03T18:42:39.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.240+0000 7f0c1710ab80 1 Processor -- start 2024-05-03T18:42:39.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.241+0000 7f0c1710ab80 1 -- start start 2024-05-03T18:42:39.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.241+0000 7f0c1710ab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5581278835d0 0x558127882fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:39.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.241+0000 7f0c1710ab80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5581278889b0 0x558127898e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:39.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.241+0000 7f0c1710ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5581278892d0 0x55812788b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:39.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.241+0000 7f0c1710ab80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0c041a33b0 con 0x5581278889b0 2024-05-03T18:42:39.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.241+0000 7f0c1710ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0c041bed10 con 0x5581278892d0 2024-05-03T18:42:39.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.241+0000 7f0c1710ab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f0c041a3a00 con 0x5581278835d0 2024-05-03T18:42:39.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.242+0000 7f0c09e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5581278835d0 0x558127882fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:42:39.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.242+0000 7f0c09e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5581278835d0 0x558127882fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:49086/0 (socket says 172.21.15.33:49086) 2024-05-03T18:42:39.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.242+0000 7f0c09e00700 1 -- 172.21.15.33:0/2120558607 learned_addr learned my addr 172.21.15.33:0/2120558607 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:42:39.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.242+0000 7f0c0a800700 1 --2- 172.21.15.33:0/2120558607 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5581278892d0 0x55812788b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:42:39.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.243+0000 7f0c09400700 1 --2- 172.21.15.33:0/2120558607 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5581278889b0 0x558127898e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:42:39.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.244+0000 7f0c0a800700 1 -- 172.21.15.33:0/2120558607 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5581278835d0 msgr2=0x558127882fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:42:39.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.244+0000 7f0c0a800700 1 --2- 172.21.15.33:0/2120558607 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5581278835d0 0x558127882fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:39.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.245+0000 7f0c0a800700 1 -- 172.21.15.33:0/2120558607 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5581278889b0 msgr2=0x558127898e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:42:39.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.245+0000 7f0c0a800700 1 --2- 172.21.15.33:0/2120558607 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5581278889b0 0x558127898e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:39.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.245+0000 7f0c0a800700 1 -- 172.21.15.33:0/2120558607 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55812788ec10 con 0x5581278892d0 2024-05-03T18:42:39.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.246+0000 7f0c0a800700 1 --2- 172.21.15.33:0/2120558607 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5581278892d0 0x55812788b510 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0c000a5070 tx=0x7f0c000a4fc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=386388e13bd1b2d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:42:39.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.246+0000 7f0bf2a00700 1 -- 172.21.15.33:0/2120558607 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c000ad3a0 con 0x5581278892d0 2024-05-03T18:42:39.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.246+0000 7f0bf2a00700 1 -- 172.21.15.33:0/2120558607 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0c0001b320 con 0x5581278892d0 2024-05-03T18:42:39.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.246+0000 7f0bf2a00700 1 -- 172.21.15.33:0/2120558607 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c000b1b90 con 0x5581278892d0 2024-05-03T18:42:39.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.247+0000 7f0c1710ab80 1 -- 172.21.15.33:0/2120558607 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5581278892d0 msgr2=0x55812788b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:42:39.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.247+0000 7f0c1710ab80 1 --2- 172.21.15.33:0/2120558607 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5581278892d0 0x55812788b510 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0c000a5070 tx=0x7f0c000a4fc0 comp rx=0 tx=0).stop 2024-05-03T18:42:39.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.248+0000 7f0c1710ab80 1 -- 172.21.15.33:0/2120558607 shutdown_connections 2024-05-03T18:42:39.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.248+0000 7f0c1710ab80 1 --2- 172.21.15.33:0/2120558607 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5581278892d0 0x55812788b510 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:39.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.248+0000 7f0c1710ab80 1 --2- 172.21.15.33:0/2120558607 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5581278835d0 0x558127882fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:39.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.248+0000 7f0c1710ab80 1 --2- 172.21.15.33:0/2120558607 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5581278889b0 0x558127898e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:39.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.248+0000 7f0c1710ab80 1 -- 172.21.15.33:0/2120558607 >> 172.21.15.33:0/2120558607 conn(0x5581277fb750 msgr2=0x558127889d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:42:39.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.249+0000 7f0c09e00700 1 --2- 172.21.15.33:0/2120558607 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5581278835d0 0x558127882fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:42:39.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.249+0000 7f0c1710ab80 1 -- 172.21.15.33:0/2120558607 shutdown_connections 2024-05-03T18:42:39.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.249+0000 7f0c1710ab80 1 -- 172.21.15.33:0/2120558607 wait complete. 2024-05-03T18:42:39.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.250+0000 7f0c1710ab80 1 -- start start 2024-05-03T18:42:39.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.251+0000 7f0c1710ab80 1 Processor -- start 2024-05-03T18:42:39.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.252+0000 7f0c1710ab80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5581278892d0 0x558127892160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:39.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.253+0000 7f0c1710ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5581278926a0 0x558127892b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:39.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.253+0000 7f0c1710ab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558127905420 0x558127907810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:39.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.253+0000 7f0c1710ab80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0c041a40d0 con 0x5581278892d0 2024-05-03T18:42:39.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.253+0000 7f0c1710ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0c041be640 con 0x5581278926a0 2024-05-03T18:42:39.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.253+0000 7f0c1710ab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f0c041bd8a0 con 0x558127905420 2024-05-03T18:42:39.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.253+0000 7f0c1710ab80 10 client.0 _ll_register_callbacks cb 0x55812777ccf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:42:39.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.253+0000 7f0c09400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5581278926a0 0x558127892b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:42:39.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.253+0000 7f0c09400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5581278926a0 0x558127892b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:38788/0 (socket says 172.21.15.33:38788) 2024-05-03T18:42:39.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.253+0000 7f0c09400700 1 -- 172.21.15.33:0/4122379668 learned_addr learned my addr 172.21.15.33:0/4122379668 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:42:39.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.254+0000 7f0c0a800700 1 --2- 172.21.15.33:0/4122379668 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558127905420 0x558127907810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:42:39.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.254+0000 7f0c09400700 1 -- 172.21.15.33:0/4122379668 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558127905420 msgr2=0x558127907810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:42:39.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.254+0000 7f0c09400700 1 --2- 172.21.15.33:0/4122379668 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558127905420 0x558127907810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:39.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.254+0000 7f0c09400700 1 -- 172.21.15.33:0/4122379668 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5581278892d0 msgr2=0x558127892160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:42:39.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.254+0000 7f0c09400700 1 --2- 172.21.15.33:0/4122379668 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5581278892d0 0x558127892160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:39.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.254+0000 7f0c09400700 1 -- 172.21.15.33:0/4122379668 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558127907f50 con 0x5581278926a0 2024-05-03T18:42:39.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.255+0000 7f0be7e00700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-03T18:42:39.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.255+0000 7f0c09400700 1 --2- 172.21.15.33:0/4122379668 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5581278926a0 0x558127892b20 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0bf800b650 tx=0x7f0bf80055b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f0e5a1bd242c19a server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:42:39.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.255+0000 7f0be7e00700 1 -- 172.21.15.33:0/4122379668 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf8003df0 con 0x5581278926a0 2024-05-03T18:42:39.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.256+0000 7f0be7e00700 1 -- 172.21.15.33:0/4122379668 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9502+0+0 (secure 0 0 0) 0x7f0bf800b6b0 con 0x5581278926a0 2024-05-03T18:42:39.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.256+0000 7f0c1710ab80 1 -- 172.21.15.33:0/4122379668 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c0001b320 con 0x5581278926a0 2024-05-03T18:42:39.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.256+0000 7f0be7e00700 1 -- 172.21.15.33:0/4122379668 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f0bf8039050 con 0x5581278926a0 2024-05-03T18:42:39.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.257+0000 7f0c1710ab80 10 client.36530 fetch_fsmap learned FSMap version 21 2024-05-03T18:42:39.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.257+0000 7f0c1710ab80 1 -- 172.21.15.33:0/4122379668 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x558127909270 con 0x5581278926a0 2024-05-03T18:42:39.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.258+0000 7f0be7e00700 1 -- 172.21.15.33:0/4122379668 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0c0001b320 con 0x5581278926a0 2024-05-03T18:42:39.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.258+0000 7f0be7e00700 1 -- 172.21.15.33:0/4122379668 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f0bf800f870 con 0x5581278926a0 2024-05-03T18:42:39.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.258+0000 7f0c1710ab80 10 client.36530 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:42:39.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.258+0000 7f0c1710ab80 10 client.36530 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:42:39.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.258+0000 7f0c1710ab80 20 client.36530 populate_metadata read hostname 'smithi033' 2024-05-03T18:42:39.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.259+0000 7f0c1710ab80 1 --2- 172.21.15.33:0/4122379668 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x558127909960 0x55812790bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:39.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.259+0000 7f0c1710ab80 4 client.36530 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-03T18:42:39.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.259+0000 7f0c1710ab80 1 -- 172.21.15.33:0/4122379668 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5581277e3480 con 0x558127909960 2024-05-03T18:42:39.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.259+0000 7f0c09e00700 1 --2- 172.21.15.33:0/4122379668 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x558127909960 0x55812790bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:42:39.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.260+0000 7f0c09e00700 1 --2- 172.21.15.33:0/4122379668 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x558127909960 0x55812790bd80 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=639494be5111e74f server_cookie=6010366b108a26b6 in_seq=0 out_seq=0 2024-05-03T18:42:39.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.260+0000 7f0be7e00700 10 client.36530 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:42:39.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.262+0000 7f0be7e00700 1 -- 172.21.15.33:0/4122379668 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5581277e3480 con 0x558127909960 2024-05-03T18:42:39.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.262+0000 7f0be7e00700 10 client.36530 handle_command_reply: tid=0 2024-05-03T18:42:39.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.262+0000 7f0c1710ab80 1 client.36530 shutdown 2024-05-03T18:42:39.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.263+0000 7f0c1710ab80 1 -- 172.21.15.33:0/4122379668 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5581278926a0 msgr2=0x558127892b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:42:39.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.263+0000 7f0c1710ab80 1 --2- 172.21.15.33:0/4122379668 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5581278926a0 0x558127892b20 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0bf800b650 tx=0x7f0bf80055b0 comp rx=0 tx=0).stop 2024-05-03T18:42:39.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.263+0000 7f0c1710ab80 1 -- 172.21.15.33:0/4122379668 shutdown_connections 2024-05-03T18:42:39.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.263+0000 7f0c1710ab80 1 --2- 172.21.15.33:0/4122379668 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x558127909960 0x55812790bd80 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:39.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.263+0000 7f0c1710ab80 1 --2- 172.21.15.33:0/4122379668 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558127905420 0x558127907810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:39.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.263+0000 7f0c1710ab80 1 --2- 172.21.15.33:0/4122379668 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5581278892d0 0x558127892160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:39.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.263+0000 7f0c1710ab80 1 --2- 172.21.15.33:0/4122379668 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5581278926a0 0x558127892b20 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:39.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.263+0000 7f0c1710ab80 1 -- 172.21.15.33:0/4122379668 >> 172.21.15.33:0/4122379668 conn(0x5581277fb750 msgr2=0x55812788b510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:42:39.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.263+0000 7f0be7e00700 0 client.36530 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:42:39.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.263+0000 7f0c1710ab80 1 -- 172.21.15.33:0/4122379668 shutdown_connections 2024-05-03T18:42:39.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.264+0000 7f0c1710ab80 1 -- 172.21.15.33:0/4122379668 wait complete. 2024-05-03T18:42:39.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.264+0000 7f0c1710ab80 20 client.36530 trim_cache size 0 max 16384 2024-05-03T18:42:39.266 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-03T18:42:39.267 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-05-03T18:42:39.267 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "c7ee27db-c834-4f0f-8f28-7736c505ee67", 2024-05-03T18:42:39.267 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-05-03T18:42:39.267 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-03T18:42:39.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.266+0000 7f0c0c800700 1 -- 172.21.15.33:0/1125569229 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f0bec0b2c70 msgr2=0x7f0bec0b5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:42:39.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.266+0000 7f0c0c800700 1 --2- 172.21.15.33:0/1125569229 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f0bec0b2c70 0x7f0bec0b5130 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0bf4005fd0 tx=0x7f0bf4005db0 comp rx=0 tx=0).stop 2024-05-03T18:42:39.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.266+0000 7f0c0c800700 1 -- 172.21.15.33:0/1125569229 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0c04109510 msgr2=0x7f0c0410ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:42:39.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.266+0000 7f0c0c800700 1 --2- 172.21.15.33:0/1125569229 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0c04109510 0x7f0c0410ada0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f0c0000ef60 tx=0x7f0c0000ef90 comp rx=0 tx=0).stop 2024-05-03T18:42:39.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.268+0000 7f0c0c800700 1 -- 172.21.15.33:0/1125569229 shutdown_connections 2024-05-03T18:42:39.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.268+0000 7f0c0c800700 1 --2- 172.21.15.33:0/1125569229 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f0bec0b2c70 0x7f0bec0b5130 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:39.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.268+0000 7f0c0c800700 1 --2- 172.21.15.33:0/1125569229 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0c04109510 0x7f0c0410ada0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:39.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.268+0000 7f0c0c800700 1 --2- 172.21.15.33:0/1125569229 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c0406b180 0x7f0c0410a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:39.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.268+0000 7f0c0c800700 1 --2- 172.21.15.33:0/1125569229 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0c0406bb40 0x7f0c0410a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:39.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.268+0000 7f0c0c800700 1 -- 172.21.15.33:0/1125569229 >> 172.21.15.33:0/1125569229 conn(0x7f0c04079350 msgr2=0x7f0c04100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:42:39.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.268+0000 7f0c0c800700 1 -- 172.21.15.33:0/1125569229 shutdown_connections 2024-05-03T18:42:39.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:39.269+0000 7f0c0c800700 1 -- 172.21.15.33:0/1125569229 wait complete. 2024-05-03T18:42:39.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:42:39 smithi150 ceph-mon[32492]: pgmap v485: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 5.4 MiB/s wr, 491 op/s 2024-05-03T18:42:39.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:42:39 smithi150 ceph-mon[32492]: scrub queued for path: / 2024-05-03T18:42:39.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:42:39 smithi150 ceph-mon[32492]: scrub summary: idle+waiting paths [/] 2024-05-03T18:42:39.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:42:39 smithi150 ceph-mon[32492]: scrub summary: active paths [/] 2024-05-03T18:42:39.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:42:39 smithi033 ceph-mon[28330]: pgmap v485: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 5.4 MiB/s wr, 491 op/s 2024-05-03T18:42:39.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:42:39 smithi033 ceph-mon[28330]: scrub queued for path: / 2024-05-03T18:42:39.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:42:39 smithi033 ceph-mon[28330]: scrub summary: idle+waiting paths [/] 2024-05-03T18:42:39.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:42:39 smithi033 ceph-mon[28330]: scrub summary: active paths [/] 2024-05-03T18:42:40.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:42:39 smithi099 ceph-mon[31658]: pgmap v485: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 5.4 MiB/s wr, 491 op/s 2024-05-03T18:42:40.047 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:42:39 smithi099 ceph-mon[31658]: scrub queued for path: / 2024-05-03T18:42:40.047 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:42:39 smithi099 ceph-mon[31658]: scrub summary: idle+waiting paths [/] 2024-05-03T18:42:40.047 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:42:39 smithi099 ceph-mon[31658]: scrub summary: active paths [/] 2024-05-03T18:42:40.111 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 685.0 s, 361.2 tps, lat 232.435 ms stddev 216.843 2024-05-03T18:42:40.112 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-03T18:42:40.840 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:42:41.735 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:42:41 smithi033 ceph-mon[28330]: pgmap v486: 129 pgs: 129 active+clean; 3.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 7.8 MiB/s wr, 695 op/s 2024-05-03T18:42:41.735 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:42:41 smithi033 ceph-mon[28330]: scrub summary: idle+waiting paths [/] 2024-05-03T18:42:41.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:42:41 smithi150 ceph-mon[32492]: pgmap v486: 129 pgs: 129 active+clean; 3.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 7.8 MiB/s wr, 695 op/s 2024-05-03T18:42:41.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:42:41 smithi150 ceph-mon[32492]: scrub summary: idle+waiting paths [/] 2024-05-03T18:42:42.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:42:41 smithi099 ceph-mon[31658]: pgmap v486: 129 pgs: 129 active+clean; 3.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 7.8 MiB/s wr, 695 op/s 2024-05-03T18:42:42.047 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:42:41 smithi099 ceph-mon[31658]: scrub summary: idle+waiting paths [/] 2024-05-03T18:42:43.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.490+0000 7f43be400700 1 -- 172.21.15.33:0/1393687894 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43b810a540 msgr2=0x7f43b810a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:42:43.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.490+0000 7f43be400700 1 --2- 172.21.15.33:0/1393687894 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43b810a540 0x7f43b810a920 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f43a80099c0 tx=0x7f43a8009cd0 comp rx=0 tx=0).stop 2024-05-03T18:42:43.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.491+0000 7f43be400700 1 -- 172.21.15.33:0/1393687894 shutdown_connections 2024-05-03T18:42:43.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.491+0000 7f43be400700 1 --2- 172.21.15.33:0/1393687894 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43b810ee60 0x7f43b81169f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:43.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.491+0000 7f43be400700 1 --2- 172.21.15.33:0/1393687894 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43b810a540 0x7f43b810a920 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:43.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.491+0000 7f43be400700 1 --2- 172.21.15.33:0/1393687894 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43b810aef0 0x7f43b810e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:43.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.491+0000 7f43be400700 1 -- 172.21.15.33:0/1393687894 >> 172.21.15.33:0/1393687894 conn(0x7f43b8079370 msgr2=0x7f43b807b790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:42:43.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.491+0000 7f43be400700 1 -- 172.21.15.33:0/1393687894 shutdown_connections 2024-05-03T18:42:43.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.492+0000 7f43be400700 1 -- 172.21.15.33:0/1393687894 wait complete. 2024-05-03T18:42:43.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.493+0000 7f43be400700 1 Processor -- start 2024-05-03T18:42:43.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.493+0000 7f43be400700 1 -- start start 2024-05-03T18:42:43.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.494+0000 7f43be400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43b810a540 0x7f43b8074c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:43.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.494+0000 7f43be400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43b810aef0 0x7f43b806dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:43.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.494+0000 7f43be400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43b810ee60 0x7f43b806e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:43.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.494+0000 7f43be400700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f43b8119bc0 con 0x7f43b810ee60 2024-05-03T18:42:43.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.494+0000 7f43be400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f43b8119a40 con 0x7f43b810aef0 2024-05-03T18:42:43.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.494+0000 7f43be400700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f43b8119d40 con 0x7f43b810a540 2024-05-03T18:42:43.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.495+0000 7f43b7400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43b810a540 0x7f43b8074c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:42:43.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.495+0000 7f43b7400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43b810a540 0x7f43b8074c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:49100/0 (socket says 172.21.15.33:49100) 2024-05-03T18:42:43.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.495+0000 7f43b7400700 1 -- 172.21.15.33:0/1875888763 learned_addr learned my addr 172.21.15.33:0/1875888763 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:42:43.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.495+0000 7f43b6a00700 1 --2- 172.21.15.33:0/1875888763 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43b810aef0 0x7f43b806dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:42:43.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.495+0000 7f43b7e00700 1 --2- 172.21.15.33:0/1875888763 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43b810ee60 0x7f43b806e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:42:43.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.496+0000 7f43b7400700 1 -- 172.21.15.33:0/1875888763 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43b810aef0 msgr2=0x7f43b806dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:42:43.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.496+0000 7f43b7400700 1 --2- 172.21.15.33:0/1875888763 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43b810aef0 0x7f43b806dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:43.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.496+0000 7f43b7400700 1 -- 172.21.15.33:0/1875888763 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43b810ee60 msgr2=0x7f43b806e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:42:43.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.496+0000 7f43b7400700 1 --2- 172.21.15.33:0/1875888763 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43b810ee60 0x7f43b806e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:43.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.496+0000 7f43b7400700 1 -- 172.21.15.33:0/1875888763 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43b806e830 con 0x7f43b810a540 2024-05-03T18:42:43.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.496+0000 7f43b7400700 1 --2- 172.21.15.33:0/1875888763 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43b810a540 0x7f43b8074c70 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f43a8006010 tx=0x7f43a8004790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:42:43.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.499+0000 7f43afe00700 1 -- 172.21.15.33:0/1875888763 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a8026070 con 0x7f43b810a540 2024-05-03T18:42:43.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.499+0000 7f43afe00700 1 -- 172.21.15.33:0/1875888763 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f43a8004c90 con 0x7f43b810a540 2024-05-03T18:42:43.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.499+0000 7f43afe00700 1 -- 172.21.15.33:0/1875888763 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a8003d40 con 0x7f43b810a540 2024-05-03T18:42:43.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.499+0000 7f43be400700 1 -- 172.21.15.33:0/1875888763 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43b806eac0 con 0x7f43b810a540 2024-05-03T18:42:43.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.499+0000 7f43be400700 1 -- 172.21.15.33:0/1875888763 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43b8072970 con 0x7f43b810a540 2024-05-03T18:42:43.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.507+0000 7f43c8ca4b80 1 Processor -- start 2024-05-03T18:42:43.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.508+0000 7f43c8ca4b80 1 -- start start 2024-05-03T18:42:43.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.509+0000 7f43c8ca4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561a1b95b930 0x561a1b95bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:43.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.509+0000 7f43afe00700 1 -- 172.21.15.33:0/1875888763 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f43a80050c0 con 0x7f43b810a540 2024-05-03T18:42:43.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.510+0000 7f43b7e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561a1b95b930 0x561a1b95bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:42:43.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.514+0000 7f43c8ca4b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561a1b95c250 0x561a1b955f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:43.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.514+0000 7f43c8ca4b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561a1b956550 0x561a1b9569d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:43.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.514+0000 7f43c8ca4b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x561a1b8a8a20 con 0x561a1b95c250 2024-05-03T18:42:43.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.514+0000 7f43c8ca4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x561a1b846760 con 0x561a1b95b930 2024-05-03T18:42:43.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.514+0000 7f43c8ca4b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x561a1b8a49e0 con 0x561a1b956550 2024-05-03T18:42:43.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.514+0000 7f43b7e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561a1b95b930 0x561a1b95bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:38822/0 (socket says 172.21.15.33:38822) 2024-05-03T18:42:43.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.514+0000 7f43b7e00700 1 -- 172.21.15.33:0/320472606 learned_addr learned my addr 172.21.15.33:0/320472606 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:42:43.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.514+0000 7f43b6a00700 1 --2- 172.21.15.33:0/320472606 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561a1b956550 0x561a1b9569d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:42:43.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.514+0000 7f43afe00700 1 --2- 172.21.15.33:0/1875888763 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f439c0b2af0 0x7f439c0b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:43.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.515+0000 7f43afe00700 1 -- 172.21.15.33:0/1875888763 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9502+0+0 (secure 0 0 0) 0x7f43a80e3270 con 0x7f43b810a540 2024-05-03T18:42:43.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.515+0000 7f43b6a00700 1 -- 172.21.15.33:0/320472606 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561a1b95b930 msgr2=0x561a1b95bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:42:43.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.515+0000 7f43b6a00700 1 --2- 172.21.15.33:0/320472606 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561a1b95b930 0x561a1b95bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:43.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.515+0000 7f43b6a00700 1 -- 172.21.15.33:0/320472606 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561a1b95c250 msgr2=0x561a1b955f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:42:43.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.515+0000 7f43b6a00700 1 --2- 172.21.15.33:0/320472606 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561a1b95c250 0x561a1b955f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:43.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.515+0000 7f43b6a00700 1 -- 172.21.15.33:0/320472606 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a1b7f58d0 con 0x561a1b956550 2024-05-03T18:42:43.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.515+0000 7f43b6a00700 1 --2- 172.21.15.33:0/320472606 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561a1b956550 0x561a1b9569d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f43b806f4c0 tx=0x7f43a400ceb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b34e4164601819aa server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:42:43.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.516+0000 7f43acc00700 1 -- 172.21.15.33:0/320472606 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a400ece0 con 0x561a1b956550 2024-05-03T18:42:43.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.516+0000 7f43acc00700 1 -- 172.21.15.33:0/320472606 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f43a400ee80 con 0x561a1b956550 2024-05-03T18:42:43.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.516+0000 7f43acc00700 1 -- 172.21.15.33:0/320472606 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a401a670 con 0x561a1b956550 2024-05-03T18:42:43.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.517+0000 7f43c8ca4b80 1 -- 172.21.15.33:0/320472606 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561a1b956550 msgr2=0x561a1b9569d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:42:43.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.517+0000 7f43c8ca4b80 1 --2- 172.21.15.33:0/320472606 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561a1b956550 0x561a1b9569d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f43b806f4c0 tx=0x7f43a400ceb0 comp rx=0 tx=0).stop 2024-05-03T18:42:43.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.517+0000 7f43b7e00700 1 --2- 172.21.15.33:0/1875888763 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f439c0b2af0 0x7f439c0b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:42:43.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.518+0000 7f43c8ca4b80 1 -- 172.21.15.33:0/320472606 shutdown_connections 2024-05-03T18:42:43.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.518+0000 7f43c8ca4b80 1 --2- 172.21.15.33:0/320472606 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561a1b956550 0x561a1b9569d0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:43.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.518+0000 7f43c8ca4b80 1 --2- 172.21.15.33:0/320472606 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561a1b95b930 0x561a1b95bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:43.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.518+0000 7f43c8ca4b80 1 --2- 172.21.15.33:0/320472606 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561a1b95c250 0x561a1b955f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:43.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.518+0000 7f43c8ca4b80 1 -- 172.21.15.33:0/320472606 >> 172.21.15.33:0/320472606 conn(0x561a1b8ce6d0 msgr2=0x561a1b950900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:42:43.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.518+0000 7f43c8ca4b80 1 -- 172.21.15.33:0/320472606 shutdown_connections 2024-05-03T18:42:43.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.518+0000 7f43c8ca4b80 1 -- 172.21.15.33:0/320472606 wait complete. 2024-05-03T18:42:43.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.518+0000 7f43c8ca4b80 1 -- start start 2024-05-03T18:42:43.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.519+0000 7f43c8ca4b80 1 Processor -- start 2024-05-03T18:42:43.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.520+0000 7f43c8ca4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561a1b956550 0x561a1b961e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:43.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.520+0000 7f43c8ca4b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561a1b95b930 0x561a1b95f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:43.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.520+0000 7f43c8ca4b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561a1b95c250 0x561a1b95f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:43.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.520+0000 7f43c8ca4b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x561a1b962d00 con 0x561a1b95c250 2024-05-03T18:42:43.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.520+0000 7f43c8ca4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x561a1b80eb20 con 0x561a1b956550 2024-05-03T18:42:43.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.520+0000 7f43c8ca4b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x561a1b962e80 con 0x561a1b95b930 2024-05-03T18:42:43.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.520+0000 7f43c8ca4b80 10 client.0 _ll_register_callbacks cb 0x561a1b84fc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:42:43.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.521+0000 7f43b7e00700 1 --2- 172.21.15.33:0/1875888763 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f439c0b2af0 0x7f439c0b4fb0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f43b0009550 tx=0x7f43b0004ad0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:42:43.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.521+0000 7f43b7400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561a1b956550 0x561a1b961e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:42:43.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.521+0000 7f43b7400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561a1b956550 0x561a1b961e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:38834/0 (socket says 172.21.15.33:38834) 2024-05-03T18:42:43.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.521+0000 7f43b7400700 1 -- 172.21.15.33:0/1907916287 learned_addr learned my addr 172.21.15.33:0/1907916287 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:42:43.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.521+0000 7f43b7400700 1 -- 172.21.15.33:0/1907916287 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561a1b95b930 msgr2=0x561a1b95f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:42:43.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.521+0000 7f43b7400700 1 --2- 172.21.15.33:0/1907916287 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561a1b95b930 0x561a1b95f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:43.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.521+0000 7f43b7400700 1 -- 172.21.15.33:0/1907916287 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561a1b95c250 msgr2=0x561a1b95f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:42:43.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.521+0000 7f43b7400700 1 --2- 172.21.15.33:0/1907916287 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561a1b95c250 0x561a1b95f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:43.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.522+0000 7f43b7400700 1 -- 172.21.15.33:0/1907916287 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a1b86e420 con 0x561a1b956550 2024-05-03T18:42:43.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.522+0000 7f43b7400700 1 --2- 172.21.15.33:0/1907916287 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561a1b956550 0x561a1b961e60 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f43a80a9eb0 tx=0x7f43a80a9bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=edc079003791c002 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:42:43.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.522+0000 7f43a1600700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-03T18:42:43.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.522+0000 7f43a1600700 1 -- 172.21.15.33:0/1907916287 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a80aa1b0 con 0x561a1b956550 2024-05-03T18:42:43.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.523+0000 7f43a1600700 1 -- 172.21.15.33:0/1907916287 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9502+0+0 (secure 0 0 0) 0x7f43a802bd50 con 0x561a1b956550 2024-05-03T18:42:43.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.523+0000 7f43c8ca4b80 1 -- 172.21.15.33:0/1907916287 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561a1b922ff0 con 0x561a1b956550 2024-05-03T18:42:43.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.523+0000 7f43a1600700 1 -- 172.21.15.33:0/1907916287 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f43a80ae560 con 0x561a1b956550 2024-05-03T18:42:43.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.523+0000 7f43c8ca4b80 10 client.36542 fetch_fsmap learned FSMap version 21 2024-05-03T18:42:43.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.523+0000 7f43c8ca4b80 1 -- 172.21.15.33:0/1907916287 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x561a1b96d450 con 0x561a1b956550 2024-05-03T18:42:43.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.524+0000 7f43a1600700 1 -- 172.21.15.33:0/1907916287 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x561a1b922ff0 con 0x561a1b956550 2024-05-03T18:42:43.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.524+0000 7f43a1600700 1 -- 172.21.15.33:0/1907916287 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f43a80e0ce0 con 0x561a1b956550 2024-05-03T18:42:43.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.530+0000 7f43c8ca4b80 10 client.36542 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:42:43.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.530+0000 7f43c8ca4b80 10 client.36542 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:42:43.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.530+0000 7f43c8ca4b80 20 client.36542 populate_metadata read hostname 'smithi033' 2024-05-03T18:42:43.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.530+0000 7f43c8ca4b80 1 --2- 172.21.15.33:0/1907916287 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x561a1b96f920 0x561a1b971d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:43.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.530+0000 7f43c8ca4b80 4 client.36542 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:42:43.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.530+0000 7f43c8ca4b80 1 -- 172.21.15.33:0/1907916287 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561a1b8b6400 con 0x561a1b96f920 2024-05-03T18:42:43.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.532+0000 7f43b6a00700 1 --2- 172.21.15.33:0/1907916287 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x561a1b96f920 0x561a1b971d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:42:43.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.540+0000 7f43b6a00700 1 --2- 172.21.15.33:0/1907916287 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x561a1b96f920 0x561a1b971d40 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a7724a3fe31d663 server_cookie=f18717bff237e762 in_seq=0 out_seq=0 2024-05-03T18:42:43.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.540+0000 7f43a1600700 10 client.36542 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:42:43.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.543+0000 7f43a1600700 1 -- 172.21.15.33:0/1907916287 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561a1b8b6400 con 0x561a1b96f920 2024-05-03T18:42:43.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.543+0000 7f43a1600700 10 client.36542 handle_command_reply: tid=0 2024-05-03T18:42:43.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.543+0000 7f43c8ca4b80 1 client.36542 shutdown 2024-05-03T18:42:43.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.546+0000 7f43c8ca4b80 1 -- 172.21.15.33:0/1907916287 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561a1b956550 msgr2=0x561a1b961e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:42:43.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.546+0000 7f43c8ca4b80 1 --2- 172.21.15.33:0/1907916287 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561a1b956550 0x561a1b961e60 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f43a80a9eb0 tx=0x7f43a80a9bb0 comp rx=0 tx=0).stop 2024-05-03T18:42:43.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.546+0000 7f43c8ca4b80 1 -- 172.21.15.33:0/1907916287 shutdown_connections 2024-05-03T18:42:43.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.546+0000 7f43c8ca4b80 1 --2- 172.21.15.33:0/1907916287 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x561a1b96f920 0x561a1b971d40 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:43.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.546+0000 7f43c8ca4b80 1 --2- 172.21.15.33:0/1907916287 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561a1b95c250 0x561a1b95f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:43.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.546+0000 7f43c8ca4b80 1 --2- 172.21.15.33:0/1907916287 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561a1b956550 0x561a1b961e60 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:43.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.546+0000 7f43c8ca4b80 1 --2- 172.21.15.33:0/1907916287 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561a1b95b930 0x561a1b95f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:43.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.546+0000 7f43c8ca4b80 1 -- 172.21.15.33:0/1907916287 >> 172.21.15.33:0/1907916287 conn(0x561a1b8ce6d0 msgr2=0x561a1b94b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:42:43.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.547+0000 7f43a1600700 0 client.36542 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:42:43.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.547+0000 7f43c8ca4b80 1 -- 172.21.15.33:0/1907916287 shutdown_connections 2024-05-03T18:42:43.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.547+0000 7f43c8ca4b80 1 -- 172.21.15.33:0/1907916287 wait complete. 2024-05-03T18:42:43.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.547+0000 7f43c8ca4b80 20 client.36542 trim_cache size 0 max 16384 2024-05-03T18:42:43.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.591+0000 7f43c8ca4b80 1 Processor -- start 2024-05-03T18:42:43.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.591+0000 7f43c8ca4b80 1 -- start start 2024-05-03T18:42:43.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.592+0000 7f43c8ca4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561a1b956550 0x561a1b955f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:43.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.592+0000 7f43c8ca4b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561a1b95b930 0x561a1b95ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:43.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.592+0000 7f43c8ca4b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561a1b95c250 0x561a1b977650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:43.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.592+0000 7f43c8ca4b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f43b80840a0 con 0x561a1b95c250 2024-05-03T18:42:43.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.592+0000 7f43c8ca4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f43b81c7540 con 0x561a1b956550 2024-05-03T18:42:43.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.592+0000 7f43c8ca4b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f43b8084770 con 0x561a1b95b930 2024-05-03T18:42:43.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.592+0000 7f43b7e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561a1b95c250 0x561a1b977650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:42:43.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.592+0000 7f43b7400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561a1b956550 0x561a1b955f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:42:43.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.592+0000 7f43b7400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561a1b956550 0x561a1b955f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:38838/0 (socket says 172.21.15.33:38838) 2024-05-03T18:42:43.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.592+0000 7f43b7400700 1 -- 172.21.15.33:0/437513856 learned_addr learned my addr 172.21.15.33:0/437513856 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:42:43.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.592+0000 7f43b6a00700 1 --2- 172.21.15.33:0/437513856 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561a1b95b930 0x561a1b95ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:42:43.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.593+0000 7f43b7400700 1 -- 172.21.15.33:0/437513856 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561a1b95b930 msgr2=0x561a1b95ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:42:43.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.593+0000 7f43b7400700 1 --2- 172.21.15.33:0/437513856 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561a1b95b930 0x561a1b95ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:43.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.593+0000 7f43b7400700 1 -- 172.21.15.33:0/437513856 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561a1b95c250 msgr2=0x561a1b977650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:42:43.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.593+0000 7f43b7400700 1 --2- 172.21.15.33:0/437513856 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561a1b95c250 0x561a1b977650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:43.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.593+0000 7f43b7400700 1 -- 172.21.15.33:0/437513856 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a1b95fa50 con 0x561a1b956550 2024-05-03T18:42:43.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.593+0000 7f43b7400700 1 --2- 172.21.15.33:0/437513856 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561a1b956550 0x561a1b955f30 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f43a8005950 tx=0x7f43a80037b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c99d1124272fbfb3 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:42:43.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.594+0000 7f43a3e00700 1 -- 172.21.15.33:0/437513856 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a80ae560 con 0x561a1b956550 2024-05-03T18:42:43.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.594+0000 7f43a3e00700 1 -- 172.21.15.33:0/437513856 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f43a80a67a0 con 0x561a1b956550 2024-05-03T18:42:43.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.594+0000 7f43a3e00700 1 -- 172.21.15.33:0/437513856 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a8069bd0 con 0x561a1b956550 2024-05-03T18:42:43.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.595+0000 7f43c8ca4b80 1 -- 172.21.15.33:0/437513856 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561a1b956550 msgr2=0x561a1b955f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:42:43.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.595+0000 7f43c8ca4b80 1 --2- 172.21.15.33:0/437513856 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561a1b956550 0x561a1b955f30 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f43a8005950 tx=0x7f43a80037b0 comp rx=0 tx=0).stop 2024-05-03T18:42:43.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.598+0000 7f43c8ca4b80 1 -- 172.21.15.33:0/437513856 shutdown_connections 2024-05-03T18:42:43.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.598+0000 7f43c8ca4b80 1 --2- 172.21.15.33:0/437513856 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561a1b95c250 0x561a1b977650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:43.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.598+0000 7f43c8ca4b80 1 --2- 172.21.15.33:0/437513856 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561a1b956550 0x561a1b955f30 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:43.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.598+0000 7f43c8ca4b80 1 --2- 172.21.15.33:0/437513856 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561a1b95b930 0x561a1b95ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:43.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.598+0000 7f43c8ca4b80 1 -- 172.21.15.33:0/437513856 >> 172.21.15.33:0/437513856 conn(0x561a1b8ce6d0 msgr2=0x561a1b95c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:42:43.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.599+0000 7f43c8ca4b80 1 -- 172.21.15.33:0/437513856 shutdown_connections 2024-05-03T18:42:43.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.599+0000 7f43c8ca4b80 1 -- 172.21.15.33:0/437513856 wait complete. 2024-05-03T18:42:43.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.600+0000 7f43c8ca4b80 1 -- start start 2024-05-03T18:42:43.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.600+0000 7f43c8ca4b80 1 Processor -- start 2024-05-03T18:42:43.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.601+0000 7f43c8ca4b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561a1b956550 0x561a1b9771e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:43.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.601+0000 7f43c8ca4b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561a1b95b930 0x561a1b977720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:43.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.601+0000 7f43c8ca4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561a1b95c250 0x561a1b9d9200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:43.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.601+0000 7f43c8ca4b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f43b8084e40 con 0x561a1b95b930 2024-05-03T18:42:43.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.601+0000 7f43c8ca4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f43b80862b0 con 0x561a1b95c250 2024-05-03T18:42:43.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.601+0000 7f43c8ca4b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f43b8085510 con 0x561a1b956550 2024-05-03T18:42:43.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.601+0000 7f43c8ca4b80 10 client.0 _ll_register_callbacks cb 0x561a1b84fc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:42:43.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.602+0000 7f43b7e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561a1b95c250 0x561a1b9d9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:42:43.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.602+0000 7f43b7e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561a1b95c250 0x561a1b9d9200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:38848/0 (socket says 172.21.15.33:38848) 2024-05-03T18:42:43.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.602+0000 7f43b7e00700 1 -- 172.21.15.33:0/2564229039 learned_addr learned my addr 172.21.15.33:0/2564229039 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:42:43.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.602+0000 7f43b7400700 1 --2- 172.21.15.33:0/2564229039 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561a1b956550 0x561a1b9771e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:42:43.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.603+0000 7f43b6a00700 1 --2- 172.21.15.33:0/2564229039 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561a1b95b930 0x561a1b977720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:42:43.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.603+0000 7f43b7400700 1 -- 172.21.15.33:0/2564229039 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561a1b95c250 msgr2=0x561a1b9d9200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:42:43.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.603+0000 7f43b7400700 1 --2- 172.21.15.33:0/2564229039 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561a1b95c250 0x561a1b9d9200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:43.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.603+0000 7f43b7400700 1 -- 172.21.15.33:0/2564229039 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561a1b95b930 msgr2=0x561a1b977720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:42:43.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.603+0000 7f43b7400700 1 --2- 172.21.15.33:0/2564229039 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561a1b95b930 0x561a1b977720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:43.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.604+0000 7f43b7400700 1 -- 172.21.15.33:0/2564229039 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a1b9d9740 con 0x561a1b956550 2024-05-03T18:42:43.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.604+0000 7f43b6a00700 1 --2- 172.21.15.33:0/2564229039 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561a1b95b930 0x561a1b977720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:42:43.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.605+0000 7f43b7400700 1 --2- 172.21.15.33:0/2564229039 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561a1b956550 0x561a1b9771e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f43a802d030 tx=0x7f43a8009cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=126d79bfd8f2c60e server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:42:43.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.605+0000 7f43a1600700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-03T18:42:43.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.605+0000 7f43a1600700 1 -- 172.21.15.33:0/2564229039 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a806d060 con 0x561a1b956550 2024-05-03T18:42:43.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.606+0000 7f43a1600700 1 -- 172.21.15.33:0/2564229039 <== mon.1 v2:172.21.15.150:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9502+0+0 (secure 0 0 0) 0x7f43a8069de0 con 0x561a1b956550 2024-05-03T18:42:43.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.606+0000 7f43c8ca4b80 1 -- 172.21.15.33:0/2564229039 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43a80a67a0 con 0x561a1b956550 2024-05-03T18:42:43.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.607+0000 7f43a1600700 1 -- 172.21.15.33:0/2564229039 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f43a80ae560 con 0x561a1b956550 2024-05-03T18:42:43.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.607+0000 7f43c8ca4b80 10 client.26620 fetch_fsmap learned FSMap version 21 2024-05-03T18:42:43.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.607+0000 7f43c8ca4b80 1 -- 172.21.15.33:0/2564229039 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x561a1b9d9740 con 0x561a1b956550 2024-05-03T18:42:43.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.608+0000 7f43a1600700 1 -- 172.21.15.33:0/2564229039 <== mon.1 v2:172.21.15.150:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f43a80a67a0 con 0x561a1b956550 2024-05-03T18:42:43.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.608+0000 7f43a1600700 1 -- 172.21.15.33:0/2564229039 <== mon.1 v2:172.21.15.150:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f43a80e0ce0 con 0x561a1b956550 2024-05-03T18:42:43.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.610+0000 7f43c8ca4b80 10 client.26620 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:42:43.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.610+0000 7f43c8ca4b80 10 client.26620 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:42:43.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.610+0000 7f43c8ca4b80 20 client.26620 populate_metadata read hostname 'smithi033' 2024-05-03T18:42:43.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.610+0000 7f43c8ca4b80 1 --2- 172.21.15.33:0/2564229039 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x561a1b9dab10 0x561a1b9dcf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:43.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.610+0000 7f43c8ca4b80 4 client.26620 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-03T18:42:43.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.610+0000 7f43c8ca4b80 1 -- 172.21.15.33:0/2564229039 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561a1b8b04a0 con 0x561a1b9dab10 2024-05-03T18:42:43.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.611+0000 7f43b6a00700 1 --2- 172.21.15.33:0/2564229039 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x561a1b9dab10 0x561a1b9dcf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:42:43.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.612+0000 7f43b6a00700 1 --2- 172.21.15.33:0/2564229039 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x561a1b9dab10 0x561a1b9dcf00 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=118b7e4af3f33654 server_cookie=442a3f87d2ba002f in_seq=0 out_seq=0 2024-05-03T18:42:43.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.612+0000 7f43a1600700 10 client.26620 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:42:43.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.613+0000 7f43a1600700 1 -- 172.21.15.33:0/2564229039 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x561a1b8b04a0 con 0x561a1b9dab10 2024-05-03T18:42:43.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.613+0000 7f43a1600700 10 client.26620 handle_command_reply: tid=0 2024-05-03T18:42:43.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.613+0000 7f43c8ca4b80 1 client.26620 shutdown 2024-05-03T18:42:43.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.613+0000 7f43c8ca4b80 1 -- 172.21.15.33:0/2564229039 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561a1b956550 msgr2=0x561a1b9771e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:42:43.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.613+0000 7f43c8ca4b80 1 --2- 172.21.15.33:0/2564229039 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561a1b956550 0x561a1b9771e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f43a802d030 tx=0x7f43a8009cd0 comp rx=0 tx=0).stop 2024-05-03T18:42:43.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.614+0000 7f43c8ca4b80 1 -- 172.21.15.33:0/2564229039 shutdown_connections 2024-05-03T18:42:43.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.614+0000 7f43c8ca4b80 1 --2- 172.21.15.33:0/2564229039 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x561a1b9dab10 0x561a1b9dcf00 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:43.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.614+0000 7f43c8ca4b80 1 --2- 172.21.15.33:0/2564229039 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561a1b95c250 0x561a1b9d9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:43.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.614+0000 7f43c8ca4b80 1 --2- 172.21.15.33:0/2564229039 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561a1b956550 0x561a1b9771e0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:43.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.614+0000 7f43c8ca4b80 1 --2- 172.21.15.33:0/2564229039 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561a1b95b930 0x561a1b977720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:43.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.614+0000 7f43a1600700 0 client.26620 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:42:43.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.614+0000 7f43c8ca4b80 1 -- 172.21.15.33:0/2564229039 >> 172.21.15.33:0/2564229039 conn(0x561a1b8ce6d0 msgr2=0x561a1b94ab20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:42:43.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.615+0000 7f43c8ca4b80 1 -- 172.21.15.33:0/2564229039 shutdown_connections 2024-05-03T18:42:43.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.615+0000 7f43c8ca4b80 1 -- 172.21.15.33:0/2564229039 wait complete. 2024-05-03T18:42:43.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.616+0000 7f43c8ca4b80 20 client.26620 trim_cache size 0 max 16384 2024-05-03T18:42:43.619 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-03T18:42:43.619 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-03T18:42:43.619 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-05-03T18:42:43.620 INFO:teuthology.orchestra.run.smithi033.stdout: "c7ee27db-c834-4f0f-8f28-7736c505ee67": { 2024-05-03T18:42:43.620 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-05-03T18:42:43.620 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "c7ee27db-c834-4f0f-8f28-7736c505ee67", 2024-05-03T18:42:43.620 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-05-03T18:42:43.620 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:42:43.620 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:42:43.620 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-03T18:42:43.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.619+0000 7f43be400700 1 -- 172.21.15.33:0/1875888763 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f439c0b2af0 msgr2=0x7f439c0b4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:42:43.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.619+0000 7f43be400700 1 --2- 172.21.15.33:0/1875888763 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f439c0b2af0 0x7f439c0b4fb0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f43b0009550 tx=0x7f43b0004ad0 comp rx=0 tx=0).stop 2024-05-03T18:42:43.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.619+0000 7f43be400700 1 -- 172.21.15.33:0/1875888763 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43b810a540 msgr2=0x7f43b8074c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:42:43.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.619+0000 7f43be400700 1 --2- 172.21.15.33:0/1875888763 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43b810a540 0x7f43b8074c70 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f43a8006010 tx=0x7f43a8004790 comp rx=0 tx=0).stop 2024-05-03T18:42:43.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.620+0000 7f43be400700 1 -- 172.21.15.33:0/1875888763 shutdown_connections 2024-05-03T18:42:43.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.620+0000 7f43be400700 1 --2- 172.21.15.33:0/1875888763 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43b810ee60 0x7f43b806e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:43.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.620+0000 7f43be400700 1 --2- 172.21.15.33:0/1875888763 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43b810a540 0x7f43b8074c70 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:43.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.620+0000 7f43be400700 1 --2- 172.21.15.33:0/1875888763 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f439c0b2af0 0x7f439c0b4fb0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:43.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.620+0000 7f43be400700 1 --2- 172.21.15.33:0/1875888763 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43b810aef0 0x7f43b806dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:43.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.620+0000 7f43be400700 1 -- 172.21.15.33:0/1875888763 >> 172.21.15.33:0/1875888763 conn(0x7f43b8079370 msgr2=0x7f43b807b760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:42:43.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.620+0000 7f43be400700 1 -- 172.21.15.33:0/1875888763 shutdown_connections 2024-05-03T18:42:43.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:42:43.620+0000 7f43be400700 1 -- 172.21.15.33:0/1875888763 wait complete. 2024-05-03T18:42:43.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:42:43 smithi033 ceph-mon[28330]: pgmap v487: 129 pgs: 129 active+clean; 3.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 5.4 MiB/s wr, 478 op/s 2024-05-03T18:42:44.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:42:43 smithi099 ceph-mon[31658]: pgmap v487: 129 pgs: 129 active+clean; 3.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 5.4 MiB/s wr, 478 op/s 2024-05-03T18:42:44.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:42:43 smithi150 ceph-mon[32492]: pgmap v487: 129 pgs: 129 active+clean; 3.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 5.4 MiB/s wr, 478 op/s 2024-05-03T18:42:44.636 INFO:tasks.cephfs.filesystem:scrub status for tag:c7ee27db-c834-4f0f-8f28-7736c505ee67 - {'path': '/', 'tag': 'c7ee27db-c834-4f0f-8f28-7736c505ee67', 'options': 'recursive,force'} 2024-05-03T18:42:45.111 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 690.0 s, 409.4 tps, lat 286.313 ms stddev 405.781 2024-05-03T18:42:45.934 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:42:45 smithi150 ceph-mon[32492]: pgmap v488: 129 pgs: 129 active+clean; 3.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 8.4 MiB/s wr, 726 op/s 2024-05-03T18:42:45.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:42:45 smithi033 ceph-mon[28330]: pgmap v488: 129 pgs: 129 active+clean; 3.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 8.4 MiB/s wr, 726 op/s 2024-05-03T18:42:46.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:42:45 smithi099 ceph-mon[31658]: pgmap v488: 129 pgs: 129 active+clean; 3.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 8.4 MiB/s wr, 726 op/s 2024-05-03T18:42:46.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:42:46 smithi033 ceph-mon[28330]: scrub summary: idle 2024-05-03T18:42:47.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:42:46 smithi099 ceph-mon[31658]: scrub summary: idle 2024-05-03T18:42:47.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:42:46 smithi150 ceph-mon[32492]: scrub summary: idle 2024-05-03T18:42:47.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:42:47 smithi033 ceph-mon[28330]: pgmap v489: 129 pgs: 129 active+clean; 3.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 5.8 MiB/s wr, 505 op/s 2024-05-03T18:42:48.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:42:47 smithi099 ceph-mon[31658]: pgmap v489: 129 pgs: 129 active+clean; 3.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 5.8 MiB/s wr, 505 op/s 2024-05-03T18:42:48.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:42:47 smithi150 ceph-mon[32492]: pgmap v489: 129 pgs: 129 active+clean; 3.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 5.8 MiB/s wr, 505 op/s 2024-05-03T18:42:49.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:42:49 smithi033 ceph-mon[28330]: pgmap v490: 129 pgs: 129 active+clean; 3.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 6.2 MiB/s wr, 518 op/s 2024-05-03T18:42:50.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:42:49 smithi099 ceph-mon[31658]: pgmap v490: 129 pgs: 129 active+clean; 3.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 6.2 MiB/s wr, 518 op/s 2024-05-03T18:42:50.111 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 695.0 s, 516.4 tps, lat 191.552 ms stddev 116.228 2024-05-03T18:42:50.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:42:49 smithi150 ceph-mon[32492]: pgmap v490: 129 pgs: 129 active+clean; 3.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 6.2 MiB/s wr, 518 op/s 2024-05-03T18:42:51.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:42:51 smithi033 ceph-mon[28330]: pgmap v491: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 8.7 MiB/s wr, 738 op/s 2024-05-03T18:42:52.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:42:51 smithi099 ceph-mon[31658]: pgmap v491: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 8.7 MiB/s wr, 738 op/s 2024-05-03T18:42:52.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:42:51 smithi150 ceph-mon[32492]: pgmap v491: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 8.7 MiB/s wr, 738 op/s 2024-05-03T18:42:53.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:42:53 smithi033 ceph-mon[28330]: pgmap v492: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 6.1 MiB/s wr, 520 op/s 2024-05-03T18:42:54.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:42:53 smithi099 ceph-mon[31658]: pgmap v492: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 6.1 MiB/s wr, 520 op/s 2024-05-03T18:42:54.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:42:53 smithi150 ceph-mon[32492]: pgmap v492: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 6.1 MiB/s wr, 520 op/s 2024-05-03T18:42:55.111 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 700.0 s, 305.4 tps, lat 293.687 ms stddev 253.638 2024-05-03T18:42:55.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:42:55 smithi033 ceph-mon[28330]: pgmap v493: 129 pgs: 129 active+clean; 3.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 8.8 MiB/s wr, 777 op/s 2024-05-03T18:42:56.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:42:55 smithi099 ceph-mon[31658]: pgmap v493: 129 pgs: 129 active+clean; 3.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 8.8 MiB/s wr, 777 op/s 2024-05-03T18:42:56.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:42:55 smithi150 ceph-mon[32492]: pgmap v493: 129 pgs: 129 active+clean; 3.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 8.8 MiB/s wr, 777 op/s 2024-05-03T18:42:57.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:42:57 smithi033 ceph-mon[28330]: pgmap v494: 129 pgs: 129 active+clean; 3.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 6.0 MiB/s wr, 545 op/s 2024-05-03T18:42:58.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:42:57 smithi099 ceph-mon[31658]: pgmap v494: 129 pgs: 129 active+clean; 3.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 6.0 MiB/s wr, 545 op/s 2024-05-03T18:42:58.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:42:57 smithi150 ceph-mon[32492]: pgmap v494: 129 pgs: 129 active+clean; 3.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 6.0 MiB/s wr, 545 op/s 2024-05-03T18:42:59.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:42:59 smithi033 ceph-mon[28330]: pgmap v495: 129 pgs: 129 active+clean; 3.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 6.3 MiB/s wr, 555 op/s 2024-05-03T18:43:00.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:42:59 smithi099 ceph-mon[31658]: pgmap v495: 129 pgs: 129 active+clean; 3.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 6.3 MiB/s wr, 555 op/s 2024-05-03T18:43:00.110 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 705.0 s, 392.6 tps, lat 205.802 ms stddev 219.525 2024-05-03T18:43:00.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:42:59 smithi150 ceph-mon[32492]: pgmap v495: 129 pgs: 129 active+clean; 3.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 6.3 MiB/s wr, 555 op/s 2024-05-03T18:43:01.986 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:43:01 smithi033 ceph-mon[28330]: pgmap v496: 129 pgs: 129 active+clean; 3.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 8.2 MiB/s wr, 737 op/s 2024-05-03T18:43:02.048 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:43:01 smithi099 ceph-mon[31658]: pgmap v496: 129 pgs: 129 active+clean; 3.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 8.2 MiB/s wr, 737 op/s 2024-05-03T18:43:02.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:43:01 smithi150 ceph-mon[32492]: pgmap v496: 129 pgs: 129 active+clean; 3.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 8.2 MiB/s wr, 737 op/s 2024-05-03T18:43:03.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:43:03 smithi033 ceph-mon[28330]: pgmap v497: 129 pgs: 129 active+clean; 3.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 5.6 MiB/s wr, 504 op/s 2024-05-03T18:43:04.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:43:03 smithi099 ceph-mon[31658]: pgmap v497: 129 pgs: 129 active+clean; 3.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 5.6 MiB/s wr, 504 op/s 2024-05-03T18:43:04.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:43:03 smithi150 ceph-mon[32492]: pgmap v497: 129 pgs: 129 active+clean; 3.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 5.6 MiB/s wr, 504 op/s 2024-05-03T18:43:05.111 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 710.0 s, 489.8 tps, lat 268.691 ms stddev 730.704 2024-05-03T18:43:05.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:43:05 smithi033 ceph-mon[28330]: pgmap v498: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 8.4 MiB/s wr, 767 op/s 2024-05-03T18:43:06.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:43:05 smithi099 ceph-mon[31658]: pgmap v498: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 8.4 MiB/s wr, 767 op/s 2024-05-03T18:43:06.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:43:05 smithi150 ceph-mon[32492]: pgmap v498: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 8.4 MiB/s wr, 767 op/s 2024-05-03T18:43:08.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:43:07 smithi099 ceph-mon[31658]: pgmap v499: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 6.0 MiB/s wr, 536 op/s 2024-05-03T18:43:08.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:43:07 smithi099 ceph-mon[31658]: osdmap e110: 12 total, 12 up, 12 in 2024-05-03T18:43:08.134 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:43:07 smithi150 ceph-mon[32492]: pgmap v499: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 6.0 MiB/s wr, 536 op/s 2024-05-03T18:43:08.134 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:43:07 smithi150 ceph-mon[32492]: osdmap e110: 12 total, 12 up, 12 in 2024-05-03T18:43:08.237 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:43:07 smithi033 ceph-mon[28330]: pgmap v499: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 6.0 MiB/s wr, 536 op/s 2024-05-03T18:43:08.238 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:43:07 smithi033 ceph-mon[28330]: osdmap e110: 12 total, 12 up, 12 in 2024-05-03T18:43:10.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:43:09 smithi099 ceph-mon[31658]: pgmap v501: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 6.8 MiB/s wr, 616 op/s 2024-05-03T18:43:10.111 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 715.0 s, 343.6 tps, lat 287.853 ms stddev 330.689 2024-05-03T18:43:10.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:43:09 smithi150 ceph-mon[32492]: pgmap v501: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 6.8 MiB/s wr, 616 op/s 2024-05-03T18:43:10.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:43:09 smithi033 ceph-mon[28330]: pgmap v501: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 6.8 MiB/s wr, 616 op/s 2024-05-03T18:43:11.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:43:11 smithi033 ceph-mon[28330]: pgmap v502: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 3.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 6.9 MiB/s wr, 655 op/s 2024-05-03T18:43:12.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:43:11 smithi099 ceph-mon[31658]: pgmap v502: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 3.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 6.9 MiB/s wr, 655 op/s 2024-05-03T18:43:12.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:43:11 smithi150 ceph-mon[32492]: pgmap v502: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 3.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 6.9 MiB/s wr, 655 op/s 2024-05-03T18:43:13.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:43:13 smithi099 ceph-mon[31658]: pgmap v503: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 3.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 6.9 MiB/s wr, 655 op/s 2024-05-03T18:43:13.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:43:13 smithi099 ceph-mon[31658]: osdmap e111: 12 total, 12 up, 12 in 2024-05-03T18:43:13.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:43:13 smithi150 ceph-mon[32492]: pgmap v503: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 3.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 6.9 MiB/s wr, 655 op/s 2024-05-03T18:43:13.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:43:13 smithi150 ceph-mon[32492]: osdmap e111: 12 total, 12 up, 12 in 2024-05-03T18:43:13.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:43:13 smithi033 ceph-mon[28330]: pgmap v503: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 3.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 6.9 MiB/s wr, 655 op/s 2024-05-03T18:43:13.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:43:13 smithi033 ceph-mon[28330]: osdmap e111: 12 total, 12 up, 12 in 2024-05-03T18:43:14.637 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-03T18:43:15.111 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 720.0 s, 490.2 tps, lat 204.945 ms stddev 159.474 2024-05-03T18:43:15.546 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:43:15.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:43:15 smithi099 ceph-mon[31658]: pgmap v505: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 7.7 MiB/s wr, 684 op/s 2024-05-03T18:43:15.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:43:15 smithi150 ceph-mon[32492]: pgmap v505: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 7.7 MiB/s wr, 684 op/s 2024-05-03T18:43:15.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:43:15 smithi033 ceph-mon[28330]: pgmap v505: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 7.7 MiB/s wr, 684 op/s 2024-05-03T18:43:17.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:43:17 smithi150 ceph-mon[32492]: pgmap v506: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 6.6 MiB/s wr, 596 op/s 2024-05-03T18:43:17.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:43:17 smithi150 ceph-mon[32492]: osdmap e112: 12 total, 12 up, 12 in 2024-05-03T18:43:17.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:43:17 smithi033 ceph-mon[28330]: pgmap v506: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 6.6 MiB/s wr, 596 op/s 2024-05-03T18:43:17.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:43:17 smithi033 ceph-mon[28330]: osdmap e112: 12 total, 12 up, 12 in 2024-05-03T18:43:18.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:43:17 smithi099 ceph-mon[31658]: pgmap v506: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 6.6 MiB/s wr, 596 op/s 2024-05-03T18:43:18.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:43:17 smithi099 ceph-mon[31658]: osdmap e112: 12 total, 12 up, 12 in 2024-05-03T18:43:19.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:43:19 smithi150 ceph-mon[32492]: pgmap v508: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 3.9 MiB/s wr, 338 op/s 2024-05-03T18:43:19.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:43:19 smithi033 ceph-mon[28330]: pgmap v508: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 3.9 MiB/s wr, 338 op/s 2024-05-03T18:43:20.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:43:19 smithi099 ceph-mon[31658]: pgmap v508: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 3.9 MiB/s wr, 338 op/s 2024-05-03T18:43:20.110 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 725.0 s, 466.4 tps, lat 204.322 ms stddev 114.049 2024-05-03T18:43:21.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:43:21 smithi150 ceph-mon[32492]: pgmap v509: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 7.6 MiB/s wr, 682 op/s 2024-05-03T18:43:21.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:43:21 smithi033 ceph-mon[28330]: pgmap v509: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 7.6 MiB/s wr, 682 op/s 2024-05-03T18:43:22.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:43:21 smithi099 ceph-mon[31658]: pgmap v509: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 7.6 MiB/s wr, 682 op/s 2024-05-03T18:43:22.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:21.782+0000 7fc654a00700 1 -- 172.21.15.33:0/2073610567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc65010b6e0 msgr2=0x7fc65010bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:43:22.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:21.782+0000 7fc654a00700 1 --2- 172.21.15.33:0/2073610567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc65010b6e0 0x7fc65010bac0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fc6380098f0 tx=0x7fc638009c00 comp rx=0 tx=0).stop 2024-05-03T18:43:22.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:21.782+0000 7fc654a00700 1 -- 172.21.15.33:0/2073610567 shutdown_connections 2024-05-03T18:43:22.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:21.782+0000 7fc654a00700 1 --2- 172.21.15.33:0/2073610567 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc6501060a0 0x7fc6501125b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:22.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:21.782+0000 7fc654a00700 1 --2- 172.21.15.33:0/2073610567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc65010b6e0 0x7fc65010bac0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:22.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:21.782+0000 7fc654a00700 1 --2- 172.21.15.33:0/2073610567 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc6501056e0 0x7fc650105b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:22.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:21.782+0000 7fc654a00700 1 -- 172.21.15.33:0/2073610567 >> 172.21.15.33:0/2073610567 conn(0x7fc6501011d0 msgr2=0x7fc6501035f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:43:22.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:21.782+0000 7fc654a00700 1 -- 172.21.15.33:0/2073610567 shutdown_connections 2024-05-03T18:43:22.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:21.783+0000 7fc654a00700 1 -- 172.21.15.33:0/2073610567 wait complete. 2024-05-03T18:43:22.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:21.784+0000 7fc654a00700 1 Processor -- start 2024-05-03T18:43:22.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:21.784+0000 7fc654a00700 1 -- start start 2024-05-03T18:43:22.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:21.785+0000 7fc654a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6501056e0 0x7fc6501124f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:43:22.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:21.785+0000 7fc654a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc6501060a0 0x7fc65010f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:43:22.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:21.785+0000 7fc654a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc65010b6e0 0x7fc65010fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:43:22.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:21.786+0000 7fc64de00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6501056e0 0x7fc6501124f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:43:22.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:21.786+0000 7fc64e800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc65010b6e0 0x7fc65010fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:43:22.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:21.786+0000 7fc64e800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc65010b6e0 0x7fc65010fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:56544/0 (socket says 172.21.15.33:56544) 2024-05-03T18:43:22.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:21.786+0000 7fc64e800700 1 -- 172.21.15.33:0/44131374 learned_addr learned my addr 172.21.15.33:0/44131374 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:43:22.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:21.786+0000 7fc64d400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc6501060a0 0x7fc65010f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:43:22.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:21.787+0000 7fc654a00700 1 -- 172.21.15.33:0/44131374 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc6501157d0 con 0x7fc6501056e0 2024-05-03T18:43:22.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:21.787+0000 7fc654a00700 1 -- 172.21.15.33:0/44131374 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc650115650 con 0x7fc65010b6e0 2024-05-03T18:43:22.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:21.787+0000 7fc654a00700 1 -- 172.21.15.33:0/44131374 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc650115950 con 0x7fc6501060a0 2024-05-03T18:43:22.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:21.788+0000 7fc64d400700 1 -- 172.21.15.33:0/44131374 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc65010b6e0 msgr2=0x7fc65010fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:43:22.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:21.788+0000 7fc64d400700 1 --2- 172.21.15.33:0/44131374 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc65010b6e0 0x7fc65010fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:22.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:21.788+0000 7fc64d400700 1 -- 172.21.15.33:0/44131374 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6501056e0 msgr2=0x7fc6501124f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:43:22.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:21.788+0000 7fc64d400700 1 --2- 172.21.15.33:0/44131374 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6501056e0 0x7fc6501124f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:22.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:21.788+0000 7fc64d400700 1 -- 172.21.15.33:0/44131374 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6501a1cb0 con 0x7fc6501060a0 2024-05-03T18:43:22.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:21.789+0000 7fc64de00700 1 --2- 172.21.15.33:0/44131374 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6501056e0 0x7fc6501124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:43:22.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:21.789+0000 7fc64d400700 1 --2- 172.21.15.33:0/44131374 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc6501060a0 0x7fc65010f5a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc64000e9c0 tx=0x7fc64000ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:43:22.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:21.790+0000 7fc646a00700 1 -- 172.21.15.33:0/44131374 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc64000c9c0 con 0x7fc6501060a0 2024-05-03T18:43:22.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:21.790+0000 7fc646a00700 1 -- 172.21.15.33:0/44131374 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc640010430 con 0x7fc6501060a0 2024-05-03T18:43:22.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:21.790+0000 7fc646a00700 1 -- 172.21.15.33:0/44131374 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc640018680 con 0x7fc6501060a0 2024-05-03T18:43:22.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:21.790+0000 7fc654a00700 1 -- 172.21.15.33:0/44131374 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6501a1fa0 con 0x7fc6501060a0 2024-05-03T18:43:22.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:21.790+0000 7fc654a00700 1 -- 172.21.15.33:0/44131374 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6501a23d0 con 0x7fc6501060a0 2024-05-03T18:43:22.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:21.795+0000 7fc646a00700 1 -- 172.21.15.33:0/44131374 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fc64000cb60 con 0x7fc6501060a0 2024-05-03T18:43:22.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:21.798+0000 7fc646a00700 1 --2- 172.21.15.33:0/44131374 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fc6340b2af0 0x7fc6340b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:43:22.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:21.798+0000 7fc646a00700 1 -- 172.21.15.33:0/44131374 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9474+0+0 (secure 0 0 0) 0x7fc6400e2a00 con 0x7fc6501060a0 2024-05-03T18:43:22.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:21.799+0000 7fc64de00700 1 --2- 172.21.15.33:0/44131374 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fc6340b2af0 0x7fc6340b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:43:22.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:21.801+0000 7fc64de00700 1 --2- 172.21.15.33:0/44131374 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fc6340b2af0 0x7fc6340b4fb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc63800ffd0 tx=0x7fc63801a040 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:43:22.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.119+0000 7fc65f472b80 1 Processor -- start 2024-05-03T18:43:22.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.120+0000 7fc65f472b80 1 -- start start 2024-05-03T18:43:22.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.120+0000 7fc65f472b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558414d667b0 0x558414d66b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:43:22.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.120+0000 7fc65f472b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558414d670d0 0x558414d61fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:43:22.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.120+0000 7fc65f472b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558414d625d0 0x558414d62a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:43:22.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.120+0000 7fc65f472b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x558414cb4a20 con 0x558414d625d0 2024-05-03T18:43:22.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.120+0000 7fc65f472b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x558414c52760 con 0x558414d670d0 2024-05-03T18:43:22.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.120+0000 7fc65f472b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x558414cb09e0 con 0x558414d667b0 2024-05-03T18:43:22.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.121+0000 7fc64e800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558414d625d0 0x558414d62a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:43:22.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.121+0000 7fc64de00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558414d667b0 0x558414d66b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:43:22.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.122+0000 7fc64d400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558414d670d0 0x558414d61fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:43:22.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.122+0000 7fc64d400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558414d670d0 0x558414d61fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:56558/0 (socket says 172.21.15.33:56558) 2024-05-03T18:43:22.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.122+0000 7fc64d400700 1 -- 172.21.15.33:0/1523093468 learned_addr learned my addr 172.21.15.33:0/1523093468 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:43:22.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.122+0000 7fc64d400700 1 -- 172.21.15.33:0/1523093468 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558414d667b0 msgr2=0x558414d66b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:43:22.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.122+0000 7fc64d400700 1 --2- 172.21.15.33:0/1523093468 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558414d667b0 0x558414d66b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:22.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.122+0000 7fc64d400700 1 -- 172.21.15.33:0/1523093468 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558414d625d0 msgr2=0x558414d62a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:43:22.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.123+0000 7fc64d400700 1 --2- 172.21.15.33:0/1523093468 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558414d625d0 0x558414d62a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:22.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.123+0000 7fc64d400700 1 -- 172.21.15.33:0/1523093468 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558414c018d0 con 0x558414d670d0 2024-05-03T18:43:22.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.123+0000 7fc64e800700 1 --2- 172.21.15.33:0/1523093468 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558414d625d0 0x558414d62a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:43:22.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.123+0000 7fc64d400700 1 --2- 172.21.15.33:0/1523093468 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558414d670d0 0x558414d61fb0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc6400a9850 tx=0x7fc6400a90b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c08b8d5511b6f18 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:43:22.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.124+0000 7fc63f400700 1 -- 172.21.15.33:0/1523093468 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6400a9fc0 con 0x558414d670d0 2024-05-03T18:43:22.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.124+0000 7fc63f400700 1 -- 172.21.15.33:0/1523093468 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc6400a98c0 con 0x558414d670d0 2024-05-03T18:43:22.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.124+0000 7fc63f400700 1 -- 172.21.15.33:0/1523093468 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6400cc1d0 con 0x558414d670d0 2024-05-03T18:43:22.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.125+0000 7fc65f472b80 1 -- 172.21.15.33:0/1523093468 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558414d670d0 msgr2=0x558414d61fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:43:22.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.125+0000 7fc65f472b80 1 --2- 172.21.15.33:0/1523093468 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558414d670d0 0x558414d61fb0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc6400a9850 tx=0x7fc6400a90b0 comp rx=0 tx=0).stop 2024-05-03T18:43:22.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.125+0000 7fc65f472b80 1 -- 172.21.15.33:0/1523093468 shutdown_connections 2024-05-03T18:43:22.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.125+0000 7fc65f472b80 1 --2- 172.21.15.33:0/1523093468 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558414d625d0 0x558414d62a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:22.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.125+0000 7fc65f472b80 1 --2- 172.21.15.33:0/1523093468 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558414d667b0 0x558414d66b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:22.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.125+0000 7fc65f472b80 1 --2- 172.21.15.33:0/1523093468 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558414d670d0 0x558414d61fb0 secure :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc6400a9850 tx=0x7fc6400a90b0 comp rx=0 tx=0).stop 2024-05-03T18:43:22.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.125+0000 7fc65f472b80 1 -- 172.21.15.33:0/1523093468 >> 172.21.15.33:0/1523093468 conn(0x558414c90b60 msgr2=0x558414d5c7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:43:22.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.125+0000 7fc65f472b80 1 -- 172.21.15.33:0/1523093468 shutdown_connections 2024-05-03T18:43:22.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.125+0000 7fc65f472b80 1 -- 172.21.15.33:0/1523093468 wait complete. 2024-05-03T18:43:22.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.126+0000 7fc65f472b80 1 -- start start 2024-05-03T18:43:22.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.127+0000 7fc65f472b80 1 Processor -- start 2024-05-03T18:43:22.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.129+0000 7fc65f472b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558414d625d0 0x558414d6e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:43:22.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.129+0000 7fc65f472b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558414d667b0 0x558414d6b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:43:22.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.130+0000 7fc65f472b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558414d6b9a0 0x558414d6be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:43:22.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.130+0000 7fc65f472b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x558414d6ebe0 con 0x558414d667b0 2024-05-03T18:43:22.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.130+0000 7fc65f472b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x558414c1ab20 con 0x558414d6b9a0 2024-05-03T18:43:22.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.130+0000 7fc65f472b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x558414d6ed60 con 0x558414d625d0 2024-05-03T18:43:22.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.130+0000 7fc65f472b80 10 client.0 _ll_register_callbacks cb 0x558414c5bc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:43:22.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.131+0000 7fc64de00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558414d625d0 0x558414d6e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:43:22.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.131+0000 7fc64de00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558414d625d0 0x558414d6e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:39396/0 (socket says 172.21.15.33:39396) 2024-05-03T18:43:22.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.131+0000 7fc64de00700 1 -- 172.21.15.33:0/2764263933 learned_addr learned my addr 172.21.15.33:0/2764263933 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:43:22.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.132+0000 7fc64e800700 1 --2- 172.21.15.33:0/2764263933 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558414d6b9a0 0x558414d6be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:43:22.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.132+0000 7fc64de00700 1 -- 172.21.15.33:0/2764263933 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558414d6b9a0 msgr2=0x558414d6be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:43:22.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.132+0000 7fc64de00700 1 --2- 172.21.15.33:0/2764263933 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558414d6b9a0 0x558414d6be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:22.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.132+0000 7fc64de00700 1 -- 172.21.15.33:0/2764263933 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558414d667b0 msgr2=0x558414d6b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:43:22.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.132+0000 7fc64de00700 1 --2- 172.21.15.33:0/2764263933 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558414d667b0 0x558414d6b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:22.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.132+0000 7fc64de00700 1 -- 172.21.15.33:0/2764263933 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558414c7a420 con 0x558414d625d0 2024-05-03T18:43:22.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.133+0000 7fc64de00700 1 --2- 172.21.15.33:0/2764263933 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558414d625d0 0x558414d6e0a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fc6380098f0 tx=0x7fc638009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=79482ad37cc86049 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:43:22.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.133+0000 7fc633e00700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-03T18:43:22.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.135+0000 7fc633e00700 1 -- 172.21.15.33:0/2764263933 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc638003950 con 0x558414d625d0 2024-05-03T18:43:22.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.135+0000 7fc633e00700 1 -- 172.21.15.33:0/2764263933 <== mon.1 v2:172.21.15.150:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9474+0+0 (secure 0 0 0) 0x7fc638003af0 con 0x558414d625d0 2024-05-03T18:43:22.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.136+0000 7fc65f472b80 1 -- 172.21.15.33:0/2764263933 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6400a98c0 con 0x558414d625d0 2024-05-03T18:43:22.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.136+0000 7fc633e00700 1 -- 172.21.15.33:0/2764263933 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fc638003cf0 con 0x558414d625d0 2024-05-03T18:43:22.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.137+0000 7fc65f472b80 10 client.26632 fetch_fsmap learned FSMap version 21 2024-05-03T18:43:22.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.137+0000 7fc65f472b80 1 -- 172.21.15.33:0/2764263933 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x558414d79a10 con 0x558414d625d0 2024-05-03T18:43:22.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.139+0000 7fc633e00700 1 -- 172.21.15.33:0/2764263933 <== mon.1 v2:172.21.15.150:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc6400a98c0 con 0x558414d625d0 2024-05-03T18:43:22.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.139+0000 7fc633e00700 1 -- 172.21.15.33:0/2764263933 <== mon.1 v2:172.21.15.150:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fc638022070 con 0x558414d625d0 2024-05-03T18:43:22.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.140+0000 7fc65f472b80 10 client.26632 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:43:22.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.140+0000 7fc65f472b80 10 client.26632 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:43:22.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.140+0000 7fc65f472b80 20 client.26632 populate_metadata read hostname 'smithi033' 2024-05-03T18:43:22.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.140+0000 7fc65f472b80 1 --2- 172.21.15.33:0/2764263933 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x558414d7bfa0 0x558414d7e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:43:22.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.141+0000 7fc65f472b80 4 client.26632 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:43:22.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.141+0000 7fc65f472b80 1 -- 172.21.15.33:0/2764263933 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558414d2f1d0 con 0x558414d7bfa0 2024-05-03T18:43:22.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.141+0000 7fc64d400700 1 --2- 172.21.15.33:0/2764263933 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x558414d7bfa0 0x558414d7e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:43:22.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.143+0000 7fc64d400700 1 --2- 172.21.15.33:0/2764263933 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x558414d7bfa0 0x558414d7e390 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe46fbf5d3778e12 server_cookie=a481f9f5107d55c1 in_seq=0 out_seq=0 2024-05-03T18:43:22.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.143+0000 7fc633e00700 10 client.26632 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:43:22.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.144+0000 7fc633e00700 1 -- 172.21.15.33:0/2764263933 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558414d2f1d0 con 0x558414d7bfa0 2024-05-03T18:43:22.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.144+0000 7fc633e00700 10 client.26632 handle_command_reply: tid=0 2024-05-03T18:43:22.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.145+0000 7fc65f472b80 1 client.26632 shutdown 2024-05-03T18:43:22.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.146+0000 7fc65f472b80 1 -- 172.21.15.33:0/2764263933 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558414d625d0 msgr2=0x558414d6e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:43:22.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.146+0000 7fc65f472b80 1 --2- 172.21.15.33:0/2764263933 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558414d625d0 0x558414d6e0a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fc6380098f0 tx=0x7fc638009260 comp rx=0 tx=0).stop 2024-05-03T18:43:22.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.146+0000 7fc65f472b80 1 -- 172.21.15.33:0/2764263933 shutdown_connections 2024-05-03T18:43:22.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.146+0000 7fc65f472b80 1 --2- 172.21.15.33:0/2764263933 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x558414d7bfa0 0x558414d7e390 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:22.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.146+0000 7fc65f472b80 1 --2- 172.21.15.33:0/2764263933 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558414d6b9a0 0x558414d6be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:22.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.146+0000 7fc65f472b80 1 --2- 172.21.15.33:0/2764263933 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558414d625d0 0x558414d6e0a0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:22.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.146+0000 7fc65f472b80 1 --2- 172.21.15.33:0/2764263933 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558414d667b0 0x558414d6b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:22.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.146+0000 7fc65f472b80 1 -- 172.21.15.33:0/2764263933 >> 172.21.15.33:0/2764263933 conn(0x558414c90b60 msgr2=0x558414d67e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:43:22.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.146+0000 7fc633e00700 0 client.26632 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:43:22.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.147+0000 7fc65f472b80 1 -- 172.21.15.33:0/2764263933 shutdown_connections 2024-05-03T18:43:22.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.147+0000 7fc65f472b80 1 -- 172.21.15.33:0/2764263933 wait complete. 2024-05-03T18:43:22.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.147+0000 7fc65f472b80 20 client.26632 trim_cache size 0 max 16384 2024-05-03T18:43:22.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.188+0000 7fc65f472b80 1 Processor -- start 2024-05-03T18:43:22.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.188+0000 7fc65f472b80 1 -- start start 2024-05-03T18:43:22.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.188+0000 7fc65f472b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558414d625d0 0x558414d69750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:43:22.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.188+0000 7fc65f472b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558414d667b0 0x558414d5c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:43:22.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.188+0000 7fc65f472b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558414d6b9a0 0x558414d83c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:43:22.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.188+0000 7fc65f472b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc6501bc340 con 0x558414d667b0 2024-05-03T18:43:22.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.188+0000 7fc65f472b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc6501beba0 con 0x558414d6b9a0 2024-05-03T18:43:22.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.188+0000 7fc65f472b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc6501bc990 con 0x558414d625d0 2024-05-03T18:43:22.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.189+0000 7fc64de00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558414d625d0 0x558414d69750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:43:22.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.189+0000 7fc64de00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558414d625d0 0x558414d69750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:39412/0 (socket says 172.21.15.33:39412) 2024-05-03T18:43:22.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.189+0000 7fc64de00700 1 -- 172.21.15.33:0/366192189 learned_addr learned my addr 172.21.15.33:0/366192189 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:43:22.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.189+0000 7fc64d400700 1 --2- 172.21.15.33:0/366192189 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558414d667b0 0x558414d5c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:43:22.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.190+0000 7fc64e800700 1 --2- 172.21.15.33:0/366192189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558414d6b9a0 0x558414d83c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:43:22.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.191+0000 7fc64d400700 1 -- 172.21.15.33:0/366192189 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558414d625d0 msgr2=0x558414d69750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:43:22.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.191+0000 7fc64d400700 1 --2- 172.21.15.33:0/366192189 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558414d625d0 0x558414d69750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:22.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.191+0000 7fc64d400700 1 -- 172.21.15.33:0/366192189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558414d6b9a0 msgr2=0x558414d83c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:43:22.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.191+0000 7fc64d400700 1 --2- 172.21.15.33:0/366192189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558414d6b9a0 0x558414d83c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:22.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.191+0000 7fc64d400700 1 -- 172.21.15.33:0/366192189 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558414d5e4f0 con 0x558414d667b0 2024-05-03T18:43:22.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.191+0000 7fc64d400700 1 --2- 172.21.15.33:0/366192189 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558414d667b0 0x558414d5c660 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x558414d5e710 tx=0x7fc6400e2c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ae20b2ef110a5d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:43:22.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.192+0000 7fc63ea00700 1 -- 172.21.15.33:0/366192189 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6400a9fc0 con 0x558414d667b0 2024-05-03T18:43:22.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.192+0000 7fc63ea00700 1 -- 172.21.15.33:0/366192189 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc6400a98c0 con 0x558414d667b0 2024-05-03T18:43:22.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.192+0000 7fc63ea00700 1 -- 172.21.15.33:0/366192189 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6400a9fc0 con 0x558414d667b0 2024-05-03T18:43:22.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.193+0000 7fc65f472b80 1 -- 172.21.15.33:0/366192189 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558414d667b0 msgr2=0x558414d5c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:43:22.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.193+0000 7fc65f472b80 1 --2- 172.21.15.33:0/366192189 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558414d667b0 0x558414d5c660 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x558414d5e710 tx=0x7fc6400e2c50 comp rx=0 tx=0).stop 2024-05-03T18:43:22.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.194+0000 7fc65f472b80 1 -- 172.21.15.33:0/366192189 shutdown_connections 2024-05-03T18:43:22.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.194+0000 7fc65f472b80 1 --2- 172.21.15.33:0/366192189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558414d6b9a0 0x558414d83c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:22.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.194+0000 7fc65f472b80 1 --2- 172.21.15.33:0/366192189 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558414d625d0 0x558414d69750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:22.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.194+0000 7fc65f472b80 1 --2- 172.21.15.33:0/366192189 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558414d667b0 0x558414d5c660 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:22.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.194+0000 7fc65f472b80 1 -- 172.21.15.33:0/366192189 >> 172.21.15.33:0/366192189 conn(0x558414c90b60 msgr2=0x558414d614d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:43:22.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.194+0000 7fc65f472b80 1 -- 172.21.15.33:0/366192189 shutdown_connections 2024-05-03T18:43:22.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.195+0000 7fc65f472b80 1 -- 172.21.15.33:0/366192189 wait complete. 2024-05-03T18:43:22.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.195+0000 7fc65f472b80 1 -- start start 2024-05-03T18:43:22.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.196+0000 7fc65f472b80 1 Processor -- start 2024-05-03T18:43:22.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.196+0000 7fc65f472b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558414d625d0 0x558414d73f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:43:22.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.196+0000 7fc65f472b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558414d667b0 0x558414d74470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:43:22.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.197+0000 7fc65f472b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558414d6b9a0 0x558414d72370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:43:22.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.197+0000 7fc65f472b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc6501bd060 con 0x558414d6b9a0 2024-05-03T18:43:22.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.197+0000 7fc65f472b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc6501be4d0 con 0x558414d625d0 2024-05-03T18:43:22.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.197+0000 7fc65f472b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc6501bd730 con 0x558414d667b0 2024-05-03T18:43:22.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.198+0000 7fc64de00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558414d625d0 0x558414d73f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:43:22.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.198+0000 7fc64d400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558414d667b0 0x558414d74470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:43:22.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.198+0000 7fc64d400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558414d667b0 0x558414d74470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:39428/0 (socket says 172.21.15.33:39428) 2024-05-03T18:43:22.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.198+0000 7fc64d400700 1 -- 172.21.15.33:0/2955313302 learned_addr learned my addr 172.21.15.33:0/2955313302 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:43:22.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.198+0000 7fc64de00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558414d625d0 0x558414d73f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:56578/0 (socket says 172.21.15.33:56578) 2024-05-03T18:43:22.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.198+0000 7fc64d400700 1 -- 172.21.15.33:0/2955313302 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558414d625d0 msgr2=0x558414d73f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:43:22.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.198+0000 7fc64d400700 1 --2- 172.21.15.33:0/2955313302 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558414d625d0 0x558414d73f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:22.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.198+0000 7fc64d400700 1 -- 172.21.15.33:0/2955313302 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558414d6b9a0 msgr2=0x558414d72370 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-03T18:43:22.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.200+0000 7fc65f472b80 10 client.0 _ll_register_callbacks cb 0x558414c5bc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:43:22.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.200+0000 7fc64d400700 1 --2- 172.21.15.33:0/2955313302 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558414d6b9a0 0x558414d72370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:22.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.200+0000 7fc64d400700 1 -- 172.21.15.33:0/2955313302 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558414d72bb0 con 0x558414d667b0 2024-05-03T18:43:22.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.201+0000 7fc65f472b80 1 -- 172.21.15.33:0/2955313302 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6400a98c0 con 0x558414d667b0 2024-05-03T18:43:22.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.201+0000 7fc64d400700 1 --2- 172.21.15.33:0/2955313302 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558414d667b0 0x558414d74470 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc640000c00 tx=0x7fc6400a94f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=791f131669c1cc7d server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:43:22.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.202+0000 7fc633e00700 10 client.26638 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-03T18:43:22.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.202+0000 7fc633e00700 1 -- 172.21.15.33:0/2955313302 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6400a9fc0 con 0x558414d667b0 2024-05-03T18:43:22.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.202+0000 7fc633e00700 1 -- 172.21.15.33:0/2955313302 <== mon.1 v2:172.21.15.150:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9474+0+0 (secure 0 0 0) 0x7fc6400dffb0 con 0x558414d667b0 2024-05-03T18:43:22.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.202+0000 7fc633e00700 1 -- 172.21.15.33:0/2955313302 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fc6400a9b20 con 0x558414d667b0 2024-05-03T18:43:22.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.202+0000 7fc65f472b80 10 client.26638 fetch_fsmap learned FSMap version 21 2024-05-03T18:43:22.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.202+0000 7fc65f472b80 1 -- 172.21.15.33:0/2955313302 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({fsmap=21}) -- 0x558414de7160 con 0x558414d667b0 2024-05-03T18:43:22.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.203+0000 7fc633e00700 1 -- 172.21.15.33:0/2955313302 <== mon.1 v2:172.21.15.150:3300/0 4 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fc6400b2290 con 0x558414d667b0 2024-05-03T18:43:22.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.203+0000 7fc65f472b80 10 client.26638 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:43:22.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.203+0000 7fc65f472b80 10 client.26638 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:43:22.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.203+0000 7fc65f472b80 20 client.26638 populate_metadata read hostname 'smithi033' 2024-05-03T18:43:22.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.203+0000 7fc65f472b80 1 --2- 172.21.15.33:0/2955313302 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x558414de7160 0x558414de9550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:43:22.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.203+0000 7fc65f472b80 4 client.26638 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-03T18:43:22.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.203+0000 7fc65f472b80 1 -- 172.21.15.33:0/2955313302 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558414cc2400 con 0x558414de7160 2024-05-03T18:43:22.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.204+0000 7fc64de00700 1 --2- 172.21.15.33:0/2955313302 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x558414de7160 0x558414de9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:43:22.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.206+0000 7fc64de00700 1 --2- 172.21.15.33:0/2955313302 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x558414de7160 0x558414de9550 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7d4bab67f0c2602 server_cookie=d36cbb7ec1639912 in_seq=0 out_seq=0 2024-05-03T18:43:22.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.206+0000 7fc633e00700 10 client.26638 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:43:22.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.206+0000 7fc633e00700 1 -- 172.21.15.33:0/2955313302 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x558414cc2400 con 0x558414de7160 2024-05-03T18:43:22.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.206+0000 7fc633e00700 10 client.26638 handle_command_reply: tid=0 2024-05-03T18:43:22.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.207+0000 7fc65f472b80 1 client.26638 shutdown 2024-05-03T18:43:22.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.207+0000 7fc65f472b80 1 -- 172.21.15.33:0/2955313302 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558414d667b0 msgr2=0x558414d74470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:43:22.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.207+0000 7fc65f472b80 1 --2- 172.21.15.33:0/2955313302 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558414d667b0 0x558414d74470 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc640000c00 tx=0x7fc6400a94f0 comp rx=0 tx=0).stop 2024-05-03T18:43:22.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.207+0000 7fc65f472b80 1 -- 172.21.15.33:0/2955313302 shutdown_connections 2024-05-03T18:43:22.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.207+0000 7fc65f472b80 1 --2- 172.21.15.33:0/2955313302 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x558414de7160 0x558414de9550 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:22.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.207+0000 7fc65f472b80 1 --2- 172.21.15.33:0/2955313302 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558414d6b9a0 0x558414d72370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:22.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.207+0000 7fc65f472b80 1 --2- 172.21.15.33:0/2955313302 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558414d625d0 0x558414d73f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:22.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.207+0000 7fc65f472b80 1 --2- 172.21.15.33:0/2955313302 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558414d667b0 0x558414d74470 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:22.210 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-03T18:43:22.210 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-05-03T18:43:22.210 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-05-03T18:43:22.210 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-03T18:43:22.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.207+0000 7fc65f472b80 1 -- 172.21.15.33:0/2955313302 >> 172.21.15.33:0/2955313302 conn(0x558414c90b60 msgr2=0x558414d6cc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:43:22.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.208+0000 7fc633e00700 0 client.26638 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:43:22.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.208+0000 7fc65f472b80 1 -- 172.21.15.33:0/2955313302 shutdown_connections 2024-05-03T18:43:22.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.208+0000 7fc65f472b80 1 -- 172.21.15.33:0/2955313302 wait complete. 2024-05-03T18:43:22.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.208+0000 7fc65f472b80 20 client.26638 trim_cache size 0 max 16384 2024-05-03T18:43:22.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.209+0000 7fc654a00700 1 -- 172.21.15.33:0/44131374 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fc6340b2af0 msgr2=0x7fc6340b4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:43:22.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.209+0000 7fc654a00700 1 --2- 172.21.15.33:0/44131374 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fc6340b2af0 0x7fc6340b4fb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc63800ffd0 tx=0x7fc63801a040 comp rx=0 tx=0).stop 2024-05-03T18:43:22.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.209+0000 7fc654a00700 1 -- 172.21.15.33:0/44131374 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc6501060a0 msgr2=0x7fc65010f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:43:22.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.209+0000 7fc654a00700 1 --2- 172.21.15.33:0/44131374 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc6501060a0 0x7fc65010f5a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc64000e9c0 tx=0x7fc64000ecd0 comp rx=0 tx=0).stop 2024-05-03T18:43:22.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.210+0000 7fc654a00700 1 -- 172.21.15.33:0/44131374 shutdown_connections 2024-05-03T18:43:22.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.210+0000 7fc654a00700 1 --2- 172.21.15.33:0/44131374 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fc6340b2af0 0x7fc6340b4fb0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:22.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.210+0000 7fc654a00700 1 --2- 172.21.15.33:0/44131374 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc65010b6e0 0x7fc65010fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:22.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.210+0000 7fc654a00700 1 --2- 172.21.15.33:0/44131374 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6501056e0 0x7fc6501124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:22.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.210+0000 7fc654a00700 1 --2- 172.21.15.33:0/44131374 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc6501060a0 0x7fc65010f5a0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:22.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.210+0000 7fc654a00700 1 -- 172.21.15.33:0/44131374 >> 172.21.15.33:0/44131374 conn(0x7fc6501011d0 msgr2=0x7fc650101930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:43:22.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.210+0000 7fc654a00700 1 -- 172.21.15.33:0/44131374 shutdown_connections 2024-05-03T18:43:22.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:22.210+0000 7fc654a00700 1 -- 172.21.15.33:0/44131374 wait complete. 2024-05-03T18:43:23.575 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:43:23 smithi033 ceph-mon[28330]: pgmap v510: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 6.2 MiB/s wr, 562 op/s 2024-05-03T18:43:23.575 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:43:23 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:43:23.849 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:43:23 smithi099 ceph-mon[31658]: pgmap v510: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 6.2 MiB/s wr, 562 op/s 2024-05-03T18:43:23.849 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:43:23 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:43:23.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:43:23 smithi150 ceph-mon[32492]: pgmap v510: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 6.2 MiB/s wr, 562 op/s 2024-05-03T18:43:23.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:43:23 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:43:25.111 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 730.0 s, 464.4 tps, lat 223.762 ms stddev 102.111 2024-05-03T18:43:25.264 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-03T18:43:25.264 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T18:43:25.735 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:43:25 smithi033 ceph-mon[28330]: pgmap v511: 129 pgs: 129 active+clean; 3.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 6.3 MiB/s wr, 554 op/s 2024-05-03T18:43:25.735 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:43:25 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:43:25.735 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:43:25 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:43:25.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:43:25 smithi150 ceph-mon[32492]: pgmap v511: 129 pgs: 129 active+clean; 3.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 6.3 MiB/s wr, 554 op/s 2024-05-03T18:43:25.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:43:25 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:43:25.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:43:25 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:43:26.047 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:43:25 smithi099 ceph-mon[31658]: pgmap v511: 129 pgs: 129 active+clean; 3.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 6.3 MiB/s wr, 554 op/s 2024-05-03T18:43:26.047 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:43:25 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:43:26.047 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:43:25 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:43:26.360 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:43:27.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:43:27 smithi150 ceph-mon[32492]: pgmap v512: 129 pgs: 129 active+clean; 3.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 6.5 MiB/s wr, 568 op/s 2024-05-03T18:43:27.986 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:43:27 smithi033 ceph-mon[28330]: pgmap v512: 129 pgs: 129 active+clean; 3.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 6.5 MiB/s wr, 568 op/s 2024-05-03T18:43:28.049 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:43:27 smithi099 ceph-mon[31658]: pgmap v512: 129 pgs: 129 active+clean; 3.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 6.5 MiB/s wr, 568 op/s 2024-05-03T18:43:28.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:43:28 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:43:28.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:43:28 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:43:28.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:43:28 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:43:28.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:43:28 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:43:28.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:43:28 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:43:28.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:43:28 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:43:29.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:43:28 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:43:29.047 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:43:28 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:43:29.047 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:43:28 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:43:29.882 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:43:29 smithi150 ceph-mon[32492]: pgmap v513: 129 pgs: 129 active+clean; 3.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 6.2 MiB/s wr, 546 op/s 2024-05-03T18:43:29.986 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:43:29 smithi033 ceph-mon[28330]: pgmap v513: 129 pgs: 129 active+clean; 3.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 6.2 MiB/s wr, 546 op/s 2024-05-03T18:43:30.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:43:29 smithi099 ceph-mon[31658]: pgmap v513: 129 pgs: 129 active+clean; 3.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 6.2 MiB/s wr, 546 op/s 2024-05-03T18:43:30.111 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 735.0 s, 414.0 tps, lat 243.956 ms stddev 131.714 2024-05-03T18:43:31.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:43:31 smithi150 ceph-mon[32492]: pgmap v514: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 9.7 MiB/s wr, 710 op/s 2024-05-03T18:43:31.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:43:31 smithi033 ceph-mon[28330]: pgmap v514: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 9.7 MiB/s wr, 710 op/s 2024-05-03T18:43:32.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:32.013+0000 7fa9ba000700 1 -- 172.21.15.33:0/283828268 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa9b41083d0 msgr2=0x7fa9b406b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:43:32.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:32.013+0000 7fa9ba000700 1 --2- 172.21.15.33:0/283828268 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa9b41083d0 0x7fa9b406b1b0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa9a400b0a0 tx=0x7fa9a400b3b0 comp rx=0 tx=0).stop 2024-05-03T18:43:32.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:32.013+0000 7fa9ba000700 1 -- 172.21.15.33:0/283828268 shutdown_connections 2024-05-03T18:43:32.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:32.013+0000 7fa9ba000700 1 --2- 172.21.15.33:0/283828268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9b4105ee0 0x7fa9b4106370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:32.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:32.013+0000 7fa9ba000700 1 --2- 172.21.15.33:0/283828268 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa9b41083d0 0x7fa9b406b1b0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:32.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:32.013+0000 7fa9ba000700 1 --2- 172.21.15.33:0/283828268 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9b406b6f0 0x7fa9b406bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:32.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:32.013+0000 7fa9ba000700 1 -- 172.21.15.33:0/283828268 >> 172.21.15.33:0/283828268 conn(0x7fa9b4079350 msgr2=0x7fa9b4079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:43:32.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:32.014+0000 7fa9ba000700 1 -- 172.21.15.33:0/283828268 shutdown_connections 2024-05-03T18:43:32.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:32.014+0000 7fa9ba000700 1 -- 172.21.15.33:0/283828268 wait complete. 2024-05-03T18:43:32.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:32.015+0000 7fa9ba000700 1 Processor -- start 2024-05-03T18:43:32.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:32.016+0000 7fa9ba000700 1 -- start start 2024-05-03T18:43:32.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:32.016+0000 7fa9ba000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9b406b6f0 0x7fa9b41a01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:43:32.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:32.016+0000 7fa9ba000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9b4105ee0 0x7fa9b41a06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:43:32.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:32.016+0000 7fa9ba000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa9b41083d0 0x7fa9b41a4a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:43:32.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:32.016+0000 7fa9ba000700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b4119bb0 con 0x7fa9b4105ee0 2024-05-03T18:43:32.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:32.017+0000 7fa9ba000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b4119a30 con 0x7fa9b406b6f0 2024-05-03T18:43:32.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:32.017+0000 7fa9ba000700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b4119d30 con 0x7fa9b41083d0 2024-05-03T18:43:32.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:32.018+0000 7fa9b3e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa9b41083d0 0x7fa9b41a4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:43:32.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:32.018+0000 7fa9b3400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9b406b6f0 0x7fa9b41a01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:43:32.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:32.018+0000 7fa9b3400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9b406b6f0 0x7fa9b41a01b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:57390/0 (socket says 172.21.15.33:57390) 2024-05-03T18:43:32.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:32.018+0000 7fa9b3400700 1 -- 172.21.15.33:0/192737388 learned_addr learned my addr 172.21.15.33:0/192737388 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:43:32.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:32.019+0000 7fa9aaa00700 1 --2- 172.21.15.33:0/192737388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9b4105ee0 0x7fa9b41a06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:43:32.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:32.019+0000 7fa9b3400700 1 -- 172.21.15.33:0/192737388 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa9b41083d0 msgr2=0x7fa9b41a4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:43:32.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:32.019+0000 7fa9b3400700 1 --2- 172.21.15.33:0/192737388 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa9b41083d0 0x7fa9b41a4a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:32.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:32.019+0000 7fa9b3400700 1 -- 172.21.15.33:0/192737388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9b4105ee0 msgr2=0x7fa9b41a06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:43:32.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:32.019+0000 7fa9b3400700 1 --2- 172.21.15.33:0/192737388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9b4105ee0 0x7fa9b41a06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:32.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:32.019+0000 7fa9b3400700 1 -- 172.21.15.33:0/192737388 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9b41a51c0 con 0x7fa9b406b6f0 2024-05-03T18:43:32.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:32.020+0000 7fa9aaa00700 1 --2- 172.21.15.33:0/192737388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9b4105ee0 0x7fa9b41a06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:43:32.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:32.020+0000 7fa9b3400700 1 --2- 172.21.15.33:0/192737388 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9b406b6f0 0x7fa9b41a01b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa9a4004e20 tx=0x7fa9a4004e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:43:32.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:32.020+0000 7fa9b0c00700 1 -- 172.21.15.33:0/192737388 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a4021070 con 0x7fa9b406b6f0 2024-05-03T18:43:32.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:32.021+0000 7fa9b0c00700 1 -- 172.21.15.33:0/192737388 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa9a4007d00 con 0x7fa9b406b6f0 2024-05-03T18:43:32.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:32.021+0000 7fa9b0c00700 1 -- 172.21.15.33:0/192737388 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a4005260 con 0x7fa9b406b6f0 2024-05-03T18:43:32.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:32.021+0000 7fa9ba000700 1 -- 172.21.15.33:0/192737388 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9b41a5450 con 0x7fa9b406b6f0 2024-05-03T18:43:32.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:32.021+0000 7fa9ba000700 1 -- 172.21.15.33:0/192737388 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9b406cec0 con 0x7fa9b406b6f0 2024-05-03T18:43:32.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:32.022+0000 7fa9ba000700 1 -- 172.21.15.33:0/192737388 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9b41029c0 con 0x7fa9b406b6f0 2024-05-03T18:43:32.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:32.026+0000 7fa9b0c00700 1 -- 172.21.15.33:0/192737388 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fa9a4024070 con 0x7fa9b406b6f0 2024-05-03T18:43:32.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:32.029+0000 7fa9b0c00700 1 --2- 172.21.15.33:0/192737388 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fa9980b2af0 0x7fa9980b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:43:32.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:32.029+0000 7fa9b0c00700 1 -- 172.21.15.33:0/192737388 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9474+0+0 (secure 0 0 0) 0x7fa9a400f020 con 0x7fa9b406b6f0 2024-05-03T18:43:32.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:32.032+0000 7fa9aaa00700 1 --2- 172.21.15.33:0/192737388 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fa9980b2af0 0x7fa9980b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:43:32.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:32.032+0000 7fa9b0c00700 1 -- 172.21.15.33:0/192737388 <== mon.2 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fa9a402b020 con 0x7fa9b406b6f0 2024-05-03T18:43:32.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:32.033+0000 7fa9aaa00700 1 --2- 172.21.15.33:0/192737388 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fa9980b2af0 0x7fa9980b4fb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa9b4079be0 tx=0x7fa99c008040 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:43:32.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:43:31 smithi099 ceph-mon[31658]: pgmap v514: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 9.7 MiB/s wr, 710 op/s 2024-05-03T18:43:32.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:32.531+0000 7fa9ba000700 1 -- 172.21.15.33:0/192737388 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa9b41a1510 con 0x7fa9b406b6f0 2024-05-03T18:43:32.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:32.533+0000 7fa9b0c00700 1 -- 172.21.15.33:0/192737388 <== mon.2 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11801 (secure 0 0 0) 0x7fa9a40abb70 con 0x7fa9b406b6f0 2024-05-03T18:43:32.537 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:43:32.537 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1327393503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1327393503},{"type":"v1","addr":"172.21.15.33:6835","nonce":1327393503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/2461472948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":2461472948},{"type":"v1","addr":"172.21.15.33:6837","nonce":2461472948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14559,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/442269281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":442269281},{"type":"v1","addr":"172.21.15.33:6839","nonce":442269281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14571,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/645571756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":645571756},{"type":"v1","addr":"172.21.15.33:6841","nonce":645571756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/203598909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":203598909},{"type":"v1","addr":"172.21.15.150:6833","nonce":203598909}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/4029842493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":4029842493},{"type":"v1","addr":"172.21.15.150:6835","nonce":4029842493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3463664423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3463664423},{"type":"v1","addr":"172.21.15.150:6837","nonce":3463664423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24481,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6839/4051648018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":4051648018},{"type":"v1","addr":"172.21.15.150:6839","nonce":4051648018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34418,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1865105479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1865105479},{"type":"v1","addr":"172.21.15.99:6833","nonce":1865105479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34445,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/1206393914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1206393914},{"type":"v1","addr":"172.21.15.99:6835","nonce":1206393914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-03T18:22:28.837091+0000","modified":"2024-05-03T18:23:49.758398+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34508},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34472":{"gid":34472,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6837/3360970342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3360970342},{"type":"v1","addr":"172.21.15.99:6837","nonce":3360970342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34508":{"gid":34508,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/919907308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":919907308},{"type":"v1","addr":"172.21.15.99:6839","nonce":919907308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34508,"qdb_cluster":[34508]},"id":1}]} 2024-05-03T18:43:32.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:32.538+0000 7fa9ba000700 1 -- 172.21.15.33:0/192737388 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fa9980b2af0 msgr2=0x7fa9980b4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:43:32.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:32.538+0000 7fa9ba000700 1 --2- 172.21.15.33:0/192737388 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fa9980b2af0 0x7fa9980b4fb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa9b4079be0 tx=0x7fa99c008040 comp rx=0 tx=0).stop 2024-05-03T18:43:32.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:32.538+0000 7fa9ba000700 1 -- 172.21.15.33:0/192737388 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9b406b6f0 msgr2=0x7fa9b41a01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:43:32.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:32.538+0000 7fa9ba000700 1 --2- 172.21.15.33:0/192737388 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9b406b6f0 0x7fa9b41a01b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa9a4004e20 tx=0x7fa9a4004e50 comp rx=0 tx=0).stop 2024-05-03T18:43:32.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:32.538+0000 7fa9ba000700 1 -- 172.21.15.33:0/192737388 shutdown_connections 2024-05-03T18:43:32.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:32.538+0000 7fa9ba000700 1 --2- 172.21.15.33:0/192737388 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa9b41083d0 0x7fa9b41a4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:32.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:32.538+0000 7fa9ba000700 1 --2- 172.21.15.33:0/192737388 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fa9980b2af0 0x7fa9980b4fb0 secure :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa9b4079be0 tx=0x7fa99c008040 comp rx=0 tx=0).stop 2024-05-03T18:43:32.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:32.538+0000 7fa9ba000700 1 --2- 172.21.15.33:0/192737388 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9b406b6f0 0x7fa9b41a01b0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:32.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:32.538+0000 7fa9ba000700 1 --2- 172.21.15.33:0/192737388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9b4105ee0 0x7fa9b41a06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:32.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:32.538+0000 7fa9ba000700 1 -- 172.21.15.33:0/192737388 >> 172.21.15.33:0/192737388 conn(0x7fa9b4079350 msgr2=0x7fa9b4101390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:43:32.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:32.539+0000 7fa9ba000700 1 -- 172.21.15.33:0/192737388 shutdown_connections 2024-05-03T18:43:32.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:32.539+0000 7fa9ba000700 1 -- 172.21.15.33:0/192737388 wait complete. 2024-05-03T18:43:32.543 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 21 2024-05-03T18:43:32.916 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:43:32 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/192737388' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:43:33.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:43:32 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/192737388' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:43:33.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:43:32 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/192737388' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:43:33.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:43:33 smithi033 ceph-mon[28330]: pgmap v515: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 7.3 MiB/s wr, 480 op/s 2024-05-03T18:43:34.047 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:43:33 smithi099 ceph-mon[31658]: pgmap v515: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 7.3 MiB/s wr, 480 op/s 2024-05-03T18:43:34.135 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:43:33 smithi150 ceph-mon[32492]: pgmap v515: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 7.3 MiB/s wr, 480 op/s 2024-05-03T18:43:34.192 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-03T18:43:35.015 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:43:35.111 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 740.0 s, 514.2 tps, lat 193.354 ms stddev 87.697 2024-05-03T18:43:35.735 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:43:35 smithi033 ceph-mon[28330]: pgmap v516: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 11 MiB/s wr, 704 op/s 2024-05-03T18:43:36.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:43:35 smithi099 ceph-mon[31658]: pgmap v516: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 11 MiB/s wr, 704 op/s 2024-05-03T18:43:36.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:43:35 smithi150 ceph-mon[32492]: pgmap v516: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 11 MiB/s wr, 704 op/s 2024-05-03T18:43:37.986 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:43:37 smithi033 ceph-mon[28330]: pgmap v517: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 9.4 MiB/s wr, 522 op/s 2024-05-03T18:43:38.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:43:37 smithi099 ceph-mon[31658]: pgmap v517: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 9.4 MiB/s wr, 522 op/s 2024-05-03T18:43:38.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:43:37 smithi150 ceph-mon[32492]: pgmap v517: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 9.4 MiB/s wr, 522 op/s 2024-05-03T18:43:39.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.279+0000 7f6191400700 1 -- 172.21.15.33:0/1368337628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f618c10fb60 msgr2=0x7f618c10ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:43:39.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.279+0000 7f6191400700 1 --2- 172.21.15.33:0/1368337628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f618c10fb60 0x7f618c10ff40 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f617c0099e0 tx=0x7f617c009cf0 comp rx=0 tx=0).stop 2024-05-03T18:43:39.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.287+0000 7f6191400700 1 -- 172.21.15.33:0/1368337628 shutdown_connections 2024-05-03T18:43:39.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.287+0000 7f6191400700 1 --2- 172.21.15.33:0/1368337628 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f618c10a490 0x7f618c1169a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:39.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.287+0000 7f6191400700 1 --2- 172.21.15.33:0/1368337628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f618c10fb60 0x7f618c10ff40 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:39.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.287+0000 7f6191400700 1 --2- 172.21.15.33:0/1368337628 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f618c109ad0 0x7f618c109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:39.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.287+0000 7f6191400700 1 -- 172.21.15.33:0/1368337628 >> 172.21.15.33:0/1368337628 conn(0x7f618c076c70 msgr2=0x7f618c1082c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:43:39.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.288+0000 7f6191400700 1 -- 172.21.15.33:0/1368337628 shutdown_connections 2024-05-03T18:43:39.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.288+0000 7f6191400700 1 -- 172.21.15.33:0/1368337628 wait complete. 2024-05-03T18:43:39.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.295+0000 7f6191400700 1 Processor -- start 2024-05-03T18:43:39.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.295+0000 7f6191400700 1 -- start start 2024-05-03T18:43:39.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.296+0000 7f6191400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f618c109ad0 0x7f618c0749a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:43:39.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.296+0000 7f6191400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f618c10a490 0x7f618c074ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:43:39.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.296+0000 7f6191400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f618c10fb60 0x7f618c06ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:43:39.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.296+0000 7f6191400700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f618c06d5e0 con 0x7f618c109ad0 2024-05-03T18:43:39.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.296+0000 7f6191400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f618c06d460 con 0x7f618c10a490 2024-05-03T18:43:39.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.296+0000 7f6191400700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f618c06d760 con 0x7f618c10fb60 2024-05-03T18:43:39.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.297+0000 7f6189e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f618c10a490 0x7f618c074ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:43:39.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.297+0000 7f618a800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f618c109ad0 0x7f618c0749a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:43:39.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.298+0000 7f618b200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f618c10fb60 0x7f618c06ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:43:39.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.298+0000 7f618a800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f618c109ad0 0x7f618c0749a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53236/0 (socket says 172.21.15.33:53236) 2024-05-03T18:43:39.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.298+0000 7f618b200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f618c10fb60 0x7f618c06ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:38290/0 (socket says 172.21.15.33:38290) 2024-05-03T18:43:39.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.298+0000 7f618a800700 1 -- 172.21.15.33:0/20855414 learned_addr learned my addr 172.21.15.33:0/20855414 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:43:39.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.300+0000 7f618b200700 1 -- 172.21.15.33:0/20855414 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f618c10a490 msgr2=0x7f618c074ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:43:39.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.300+0000 7f618b200700 1 --2- 172.21.15.33:0/20855414 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f618c10a490 0x7f618c074ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:39.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.300+0000 7f618b200700 1 -- 172.21.15.33:0/20855414 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f618c109ad0 msgr2=0x7f618c0749a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:43:39.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.300+0000 7f618b200700 1 --2- 172.21.15.33:0/20855414 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f618c109ad0 0x7f618c0749a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:39.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.300+0000 7f618b200700 1 -- 172.21.15.33:0/20855414 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f618c06f2b0 con 0x7f618c10fb60 2024-05-03T18:43:39.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.301+0000 7f618b200700 1 --2- 172.21.15.33:0/20855414 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f618c10fb60 0x7f618c06ea70 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f618400ab40 tx=0x7f618400ae50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:43:39.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.303+0000 7f6183400700 1 -- 172.21.15.33:0/20855414 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6184017020 con 0x7f618c10fb60 2024-05-03T18:43:39.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.303+0000 7f6183400700 1 -- 172.21.15.33:0/20855414 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6184004bb0 con 0x7f618c10fb60 2024-05-03T18:43:39.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.303+0000 7f6183400700 1 -- 172.21.15.33:0/20855414 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6184005280 con 0x7f618c10fb60 2024-05-03T18:43:39.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.303+0000 7f6191400700 1 -- 172.21.15.33:0/20855414 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f618c06f5a0 con 0x7f618c10fb60 2024-05-03T18:43:39.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.305+0000 7f6191400700 1 -- 172.21.15.33:0/20855414 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f618c081ef0 con 0x7f618c10fb60 2024-05-03T18:43:39.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.314+0000 7f6183400700 1 -- 172.21.15.33:0/20855414 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f6184041990 con 0x7f618c10fb60 2024-05-03T18:43:39.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.316+0000 7f6183400700 1 --2- 172.21.15.33:0/20855414 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f61700b2bc0 0x7f61700b5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:43:39.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.316+0000 7f6183400700 1 -- 172.21.15.33:0/20855414 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9474+0+0 (secure 0 0 0) 0x7f61840e1a30 con 0x7f618c10fb60 2024-05-03T18:43:39.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.318+0000 7f618a800700 1 --2- 172.21.15.33:0/20855414 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f61700b2bc0 0x7f61700b5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:43:39.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.319+0000 7f618a800700 1 --2- 172.21.15.33:0/20855414 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f61700b2bc0 0x7f61700b5080 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f617c009750 tx=0x7f617c005ad0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:43:39.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.567+0000 7f619bcb1b80 1 Processor -- start 2024-05-03T18:43:39.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.568+0000 7f619bcb1b80 1 -- start start 2024-05-03T18:43:39.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.570+0000 7f619bcb1b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c07f395930 0x55c07f395d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:43:39.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.572+0000 7f619bcb1b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c07f396250 0x55c07f38ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:43:39.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.572+0000 7f619bcb1b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c07f390550 0x55c07f3909d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:43:39.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.572+0000 7f619bcb1b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c07f2e2a20 con 0x55c07f395930 2024-05-03T18:43:39.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.572+0000 7f619bcb1b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55c07f280760 con 0x55c07f396250 2024-05-03T18:43:39.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.572+0000 7f619bcb1b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55c07f2de9e0 con 0x55c07f390550 2024-05-03T18:43:39.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.573+0000 7f6189e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c07f396250 0x55c07f38ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:43:39.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.574+0000 7f6189e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c07f396250 0x55c07f38ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:56440/0 (socket says 172.21.15.33:56440) 2024-05-03T18:43:39.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.574+0000 7f6189e00700 1 -- 172.21.15.33:0/1735334204 learned_addr learned my addr 172.21.15.33:0/1735334204 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:43:39.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.575+0000 7f618b200700 1 --2- 172.21.15.33:0/1735334204 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c07f390550 0x55c07f3909d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:43:39.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.575+0000 7f6189e00700 1 -- 172.21.15.33:0/1735334204 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c07f390550 msgr2=0x55c07f3909d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:43:39.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.575+0000 7f6189e00700 1 --2- 172.21.15.33:0/1735334204 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c07f390550 0x55c07f3909d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:39.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.575+0000 7f6189e00700 1 -- 172.21.15.33:0/1735334204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c07f395930 msgr2=0x55c07f395d10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-03T18:43:39.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.575+0000 7f6189e00700 1 --2- 172.21.15.33:0/1735334204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c07f395930 0x55c07f395d10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:39.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.575+0000 7f6189e00700 1 -- 172.21.15.33:0/1735334204 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c07f22f8d0 con 0x55c07f396250 2024-05-03T18:43:39.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.577+0000 7f6189e00700 1 --2- 172.21.15.33:0/1735334204 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c07f396250 0x55c07f38ff30 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f617800bcf0 tx=0x7f61780103d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5848b659ba56252c server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:43:39.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.578+0000 7f6177e00700 1 -- 172.21.15.33:0/1735334204 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6178010dc0 con 0x55c07f396250 2024-05-03T18:43:39.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.578+0000 7f6177e00700 1 -- 172.21.15.33:0/1735334204 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f61780044c0 con 0x55c07f396250 2024-05-03T18:43:39.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.578+0000 7f6177e00700 1 -- 172.21.15.33:0/1735334204 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6178011440 con 0x55c07f396250 2024-05-03T18:43:39.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.580+0000 7f619bcb1b80 1 -- 172.21.15.33:0/1735334204 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c07f396250 msgr2=0x55c07f38ff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:43:39.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.580+0000 7f619bcb1b80 1 --2- 172.21.15.33:0/1735334204 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c07f396250 0x55c07f38ff30 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f617800bcf0 tx=0x7f61780103d0 comp rx=0 tx=0).stop 2024-05-03T18:43:39.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.581+0000 7f619bcb1b80 1 -- 172.21.15.33:0/1735334204 shutdown_connections 2024-05-03T18:43:39.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.581+0000 7f619bcb1b80 1 --2- 172.21.15.33:0/1735334204 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c07f390550 0x55c07f3909d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:39.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.581+0000 7f619bcb1b80 1 --2- 172.21.15.33:0/1735334204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c07f395930 0x55c07f395d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:39.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.581+0000 7f619bcb1b80 1 --2- 172.21.15.33:0/1735334204 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c07f396250 0x55c07f38ff30 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:39.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.581+0000 7f619bcb1b80 1 -- 172.21.15.33:0/1735334204 >> 172.21.15.33:0/1735334204 conn(0x55c07f3086d0 msgr2=0x55c07f38a900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:43:39.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.582+0000 7f619bcb1b80 1 -- 172.21.15.33:0/1735334204 shutdown_connections 2024-05-03T18:43:39.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.583+0000 7f619bcb1b80 1 -- 172.21.15.33:0/1735334204 wait complete. 2024-05-03T18:43:39.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.583+0000 7f619bcb1b80 1 -- start start 2024-05-03T18:43:39.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.584+0000 7f619bcb1b80 1 Processor -- start 2024-05-03T18:43:39.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.586+0000 7f619bcb1b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c07f390550 0x55c07f39be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:43:39.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.586+0000 7f619bcb1b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c07f395930 0x55c07f399200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:43:39.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.586+0000 7f619bcb1b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c07f396250 0x55c07f399740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:43:39.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.588+0000 7f619bcb1b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c07f39cd00 con 0x55c07f395930 2024-05-03T18:43:39.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.588+0000 7f618a800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c07f390550 0x55c07f39be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:43:39.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.588+0000 7f618a800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c07f390550 0x55c07f39be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:38316/0 (socket says 172.21.15.33:38316) 2024-05-03T18:43:39.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.588+0000 7f618a800700 1 -- 172.21.15.33:0/1087941699 learned_addr learned my addr 172.21.15.33:0/1087941699 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:43:39.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.589+0000 7f618b200700 1 --2- 172.21.15.33:0/1087941699 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c07f396250 0x55c07f399740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:43:39.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.589+0000 7f6189e00700 1 --2- 172.21.15.33:0/1087941699 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c07f395930 0x55c07f399200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:43:39.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.589+0000 7f619bcb1b80 1 -- 172.21.15.33:0/1087941699 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55c07f248b20 con 0x55c07f396250 2024-05-03T18:43:39.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.589+0000 7f619bcb1b80 1 -- 172.21.15.33:0/1087941699 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55c07f39ce80 con 0x55c07f390550 2024-05-03T18:43:39.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.591+0000 7f6189e00700 1 -- 172.21.15.33:0/1087941699 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c07f390550 msgr2=0x55c07f39be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:43:39.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.591+0000 7f6189e00700 1 --2- 172.21.15.33:0/1087941699 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c07f390550 0x55c07f39be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:39.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.592+0000 7f6189e00700 1 -- 172.21.15.33:0/1087941699 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c07f396250 msgr2=0x55c07f399740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:43:39.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.592+0000 7f619bcb1b80 10 client.0 _ll_register_callbacks cb 0x55c07f289c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:43:39.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.592+0000 7f6189e00700 1 --2- 172.21.15.33:0/1087941699 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c07f396250 0x55c07f399740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:39.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.592+0000 7f6189e00700 1 -- 172.21.15.33:0/1087941699 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c07f2a8420 con 0x55c07f395930 2024-05-03T18:43:39.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.593+0000 7f618a800700 1 --2- 172.21.15.33:0/1087941699 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c07f390550 0x55c07f39be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:43:39.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.593+0000 7f618b200700 1 --2- 172.21.15.33:0/1087941699 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c07f396250 0x55c07f399740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:43:39.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.593+0000 7f619bcb1b80 1 -- 172.21.15.33:0/1087941699 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61780044c0 con 0x55c07f395930 2024-05-03T18:43:39.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.595+0000 7f6189e00700 1 --2- 172.21.15.33:0/1087941699 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c07f395930 0x55c07f399200 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f6178000c00 tx=0x7f6178003920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d95720fe6927ece server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:43:39.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.595+0000 7f6174c00700 10 client.16797 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:43:39.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.596+0000 7f6174c00700 1 -- 172.21.15.33:0/1087941699 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6178029060 con 0x55c07f395930 2024-05-03T18:43:39.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.596+0000 7f6174c00700 1 -- 172.21.15.33:0/1087941699 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9474+0+0 (secure 0 0 0) 0x7f6178005240 con 0x55c07f395930 2024-05-03T18:43:39.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.597+0000 7f6174c00700 1 -- 172.21.15.33:0/1087941699 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f6178005480 con 0x55c07f395930 2024-05-03T18:43:39.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.597+0000 7f619bcb1b80 10 client.16797 fetch_fsmap learned FSMap version 21 2024-05-03T18:43:39.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.597+0000 7f619bcb1b80 1 -- 172.21.15.33:0/1087941699 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({fsmap=21}) -- 0x55c07f3a7350 con 0x55c07f395930 2024-05-03T18:43:39.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.598+0000 7f6174c00700 1 -- 172.21.15.33:0/1087941699 <== mon.0 v2:172.21.15.33:3300/0 4 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f6178023030 con 0x55c07f395930 2024-05-03T18:43:39.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.598+0000 7f619bcb1b80 10 client.16797 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:43:39.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.598+0000 7f619bcb1b80 10 client.16797 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:43:39.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.598+0000 7f619bcb1b80 20 client.16797 populate_metadata read hostname 'smithi033' 2024-05-03T18:43:39.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.598+0000 7f619bcb1b80 1 --2- 172.21.15.33:0/1087941699 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55c07f3a9820 0x55c07f3abc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:43:39.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.598+0000 7f619bcb1b80 4 client.16797 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:43:39.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.598+0000 7f619bcb1b80 1 -- 172.21.15.33:0/1087941699 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c07f35cff0 con 0x55c07f3a9820 2024-05-03T18:43:39.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.599+0000 7f618a800700 1 --2- 172.21.15.33:0/1087941699 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55c07f3a9820 0x55c07f3abc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:43:39.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.600+0000 7f6174c00700 10 client.16797 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:43:39.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.600+0000 7f618a800700 1 --2- 172.21.15.33:0/1087941699 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55c07f3a9820 0x55c07f3abc10 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ece3765adf5f935 server_cookie=3044927b8ac83f02 in_seq=0 out_seq=0 2024-05-03T18:43:39.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.602+0000 7f6174c00700 1 -- 172.21.15.33:0/1087941699 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c07f35cff0 con 0x55c07f3a9820 2024-05-03T18:43:39.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.602+0000 7f6174c00700 10 client.16797 handle_command_reply: tid=0 2024-05-03T18:43:39.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.602+0000 7f619bcb1b80 1 client.16797 shutdown 2024-05-03T18:43:39.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.604+0000 7f619bcb1b80 1 -- 172.21.15.33:0/1087941699 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c07f395930 msgr2=0x55c07f399200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:43:39.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.604+0000 7f619bcb1b80 1 --2- 172.21.15.33:0/1087941699 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c07f395930 0x55c07f399200 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f6178000c00 tx=0x7f6178003920 comp rx=0 tx=0).stop 2024-05-03T18:43:39.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.604+0000 7f619bcb1b80 1 -- 172.21.15.33:0/1087941699 shutdown_connections 2024-05-03T18:43:39.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.604+0000 7f619bcb1b80 1 --2- 172.21.15.33:0/1087941699 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55c07f3a9820 0x55c07f3abc10 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:39.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.604+0000 7f619bcb1b80 1 --2- 172.21.15.33:0/1087941699 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c07f396250 0x55c07f399740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:39.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.604+0000 7f619bcb1b80 1 --2- 172.21.15.33:0/1087941699 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c07f390550 0x55c07f39be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:39.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.604+0000 7f619bcb1b80 1 --2- 172.21.15.33:0/1087941699 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c07f395930 0x55c07f399200 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:39.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.604+0000 7f619bcb1b80 1 -- 172.21.15.33:0/1087941699 >> 172.21.15.33:0/1087941699 conn(0x55c07f3086d0 msgr2=0x55c07f3853b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:43:39.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.605+0000 7f6174c00700 0 client.16797 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:43:39.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.605+0000 7f619bcb1b80 1 -- 172.21.15.33:0/1087941699 shutdown_connections 2024-05-03T18:43:39.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.606+0000 7f619bcb1b80 1 -- 172.21.15.33:0/1087941699 wait complete. 2024-05-03T18:43:39.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.607+0000 7f619bcb1b80 20 client.16797 trim_cache size 0 max 16384 2024-05-03T18:43:39.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.638+0000 7f619bcb1b80 1 Processor -- start 2024-05-03T18:43:39.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.638+0000 7f619bcb1b80 1 -- start start 2024-05-03T18:43:39.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.638+0000 7f619bcb1b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c07f390550 0x55c07f396e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:43:39.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.638+0000 7f619bcb1b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c07f395930 0x55c07f3ae850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:43:39.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.638+0000 7f619bcb1b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c07f396250 0x55c07f3aed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:43:39.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.638+0000 7f619bcb1b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f618c1c4960 con 0x55c07f390550 2024-05-03T18:43:39.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.638+0000 7f619bcb1b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f618c1c71c0 con 0x55c07f395930 2024-05-03T18:43:39.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.638+0000 7f619bcb1b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f618c1c4fb0 con 0x55c07f396250 2024-05-03T18:43:39.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.638+0000 7f618a800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c07f390550 0x55c07f396e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:43:39.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.639+0000 7f618a800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c07f390550 0x55c07f396e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53260/0 (socket says 172.21.15.33:53260) 2024-05-03T18:43:39.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.639+0000 7f618a800700 1 -- 172.21.15.33:0/526409061 learned_addr learned my addr 172.21.15.33:0/526409061 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:43:39.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.639+0000 7f6189e00700 1 --2- 172.21.15.33:0/526409061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c07f395930 0x55c07f3ae850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:43:39.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.639+0000 7f618b200700 1 --2- 172.21.15.33:0/526409061 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c07f396250 0x55c07f3aed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:43:39.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.639+0000 7f618a800700 1 -- 172.21.15.33:0/526409061 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c07f396250 msgr2=0x55c07f3aed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:43:39.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.639+0000 7f618a800700 1 --2- 172.21.15.33:0/526409061 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c07f396250 0x55c07f3aed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:39.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.639+0000 7f618a800700 1 -- 172.21.15.33:0/526409061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c07f395930 msgr2=0x55c07f3ae850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:43:39.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.639+0000 7f618a800700 1 --2- 172.21.15.33:0/526409061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c07f395930 0x55c07f3ae850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:39.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.639+0000 7f618a800700 1 -- 172.21.15.33:0/526409061 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c07f38a6b0 con 0x55c07f390550 2024-05-03T18:43:39.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.639+0000 7f618a800700 1 --2- 172.21.15.33:0/526409061 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c07f390550 0x55c07f396e20 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f617c006010 tx=0x7f617c021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca7377941b94acef server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:43:39.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.640+0000 7f6177400700 1 -- 172.21.15.33:0/526409061 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f617c024ae0 con 0x55c07f390550 2024-05-03T18:43:39.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.640+0000 7f6177400700 1 -- 172.21.15.33:0/526409061 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f617c037070 con 0x55c07f390550 2024-05-03T18:43:39.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.640+0000 7f6177400700 1 -- 172.21.15.33:0/526409061 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f617c00f6c0 con 0x55c07f390550 2024-05-03T18:43:39.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.641+0000 7f619bcb1b80 1 -- 172.21.15.33:0/526409061 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c07f390550 msgr2=0x55c07f396e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:43:39.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.641+0000 7f619bcb1b80 1 --2- 172.21.15.33:0/526409061 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c07f390550 0x55c07f396e20 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f617c006010 tx=0x7f617c021000 comp rx=0 tx=0).stop 2024-05-03T18:43:39.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.641+0000 7f619bcb1b80 1 -- 172.21.15.33:0/526409061 shutdown_connections 2024-05-03T18:43:39.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.641+0000 7f619bcb1b80 1 --2- 172.21.15.33:0/526409061 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c07f396250 0x55c07f3aed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:39.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.641+0000 7f619bcb1b80 1 --2- 172.21.15.33:0/526409061 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c07f390550 0x55c07f396e20 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:39.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.641+0000 7f619bcb1b80 1 --2- 172.21.15.33:0/526409061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c07f395930 0x55c07f3ae850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:39.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.641+0000 7f619bcb1b80 1 -- 172.21.15.33:0/526409061 >> 172.21.15.33:0/526409061 conn(0x55c07f3086d0 msgr2=0x55c07f396710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:43:39.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.642+0000 7f619bcb1b80 1 -- 172.21.15.33:0/526409061 shutdown_connections 2024-05-03T18:43:39.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.642+0000 7f619bcb1b80 1 -- 172.21.15.33:0/526409061 wait complete. 2024-05-03T18:43:39.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.642+0000 7f619bcb1b80 1 -- start start 2024-05-03T18:43:39.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.643+0000 7f619bcb1b80 1 Processor -- start 2024-05-03T18:43:39.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.644+0000 7f619bcb1b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c07f395930 0x55c07f3ae790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:43:39.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.644+0000 7f619bcb1b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c07f396250 0x55c07f3aecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:43:39.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.644+0000 7f619bcb1b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c07f3abc70 0x55c07f3ac0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:43:39.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.645+0000 7f6189e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c07f396250 0x55c07f3aecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:43:39.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.645+0000 7f618b200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c07f3abc70 0x55c07f3ac0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:43:39.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.645+0000 7f618a800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c07f395930 0x55c07f3ae790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:43:39.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.645+0000 7f618a800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c07f395930 0x55c07f3ae790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:56464/0 (socket says 172.21.15.33:56464) 2024-05-03T18:43:39.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.645+0000 7f618a800700 1 -- 172.21.15.33:0/832546301 learned_addr learned my addr 172.21.15.33:0/832546301 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:43:39.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.646+0000 7f619bcb1b80 1 -- 172.21.15.33:0/832546301 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f618c1c5680 con 0x55c07f396250 2024-05-03T18:43:39.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.646+0000 7f619bcb1b80 1 -- 172.21.15.33:0/832546301 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f618c1c6af0 con 0x55c07f395930 2024-05-03T18:43:39.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.646+0000 7f619bcb1b80 1 -- 172.21.15.33:0/832546301 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f618c1c5d50 con 0x55c07f3abc70 2024-05-03T18:43:39.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.647+0000 7f619bcb1b80 10 client.0 _ll_register_callbacks cb 0x55c07f289c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:43:39.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.647+0000 7f618a800700 1 -- 172.21.15.33:0/832546301 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c07f3abc70 msgr2=0x55c07f3ac0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:43:39.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.647+0000 7f618a800700 1 --2- 172.21.15.33:0/832546301 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c07f3abc70 0x55c07f3ac0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:39.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.647+0000 7f618a800700 1 -- 172.21.15.33:0/832546301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c07f396250 msgr2=0x55c07f3aecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:43:39.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.647+0000 7f618a800700 1 --2- 172.21.15.33:0/832546301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c07f396250 0x55c07f3aecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:39.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.647+0000 7f618a800700 1 -- 172.21.15.33:0/832546301 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c07f3ac960 con 0x55c07f395930 2024-05-03T18:43:39.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.647+0000 7f6189e00700 1 --2- 172.21.15.33:0/832546301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c07f396250 0x55c07f3aecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:43:39.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.648+0000 7f618a800700 1 --2- 172.21.15.33:0/832546301 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c07f395930 0x55c07f3ae790 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f617c009350 tx=0x7f617c00bfb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73f7d320d6aea9bd server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:43:39.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.648+0000 7f6174c00700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-03T18:43:39.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.649+0000 7f6174c00700 1 -- 172.21.15.33:0/832546301 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f617c024df0 con 0x55c07f395930 2024-05-03T18:43:39.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.649+0000 7f619bcb1b80 1 -- 172.21.15.33:0/832546301 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f617c037070 con 0x55c07f395930 2024-05-03T18:43:39.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.649+0000 7f6174c00700 1 -- 172.21.15.33:0/832546301 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9474+0+0 (secure 0 0 0) 0x7f617c031480 con 0x55c07f395930 2024-05-03T18:43:39.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.650+0000 7f6174c00700 1 -- 172.21.15.33:0/832546301 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f617c031680 con 0x55c07f395930 2024-05-03T18:43:39.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.650+0000 7f619bcb1b80 10 client.36587 fetch_fsmap learned FSMap version 21 2024-05-03T18:43:39.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.650+0000 7f619bcb1b80 1 -- 172.21.15.33:0/832546301 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55c07f413d10 con 0x55c07f395930 2024-05-03T18:43:39.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.650+0000 7f6174c00700 1 -- 172.21.15.33:0/832546301 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f617c037070 con 0x55c07f395930 2024-05-03T18:43:39.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.651+0000 7f6174c00700 1 -- 172.21.15.33:0/832546301 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f617c041030 con 0x55c07f395930 2024-05-03T18:43:39.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.651+0000 7f619bcb1b80 10 client.36587 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:43:39.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.651+0000 7f619bcb1b80 10 client.36587 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:43:39.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.651+0000 7f619bcb1b80 20 client.36587 populate_metadata read hostname 'smithi033' 2024-05-03T18:43:39.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.651+0000 7f619bcb1b80 1 --2- 172.21.15.33:0/832546301 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55c07f4141c0 0x55c07f4165b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:43:39.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.651+0000 7f619bcb1b80 4 client.36587 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-03T18:43:39.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.651+0000 7f619bcb1b80 1 -- 172.21.15.33:0/832546301 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c07f2f0400 con 0x55c07f4141c0 2024-05-03T18:43:39.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.651+0000 7f6189e00700 1 --2- 172.21.15.33:0/832546301 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55c07f4141c0 0x55c07f4165b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:43:39.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.653+0000 7f6189e00700 1 --2- 172.21.15.33:0/832546301 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55c07f4141c0 0x55c07f4165b0 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=154d4aa035880b66 server_cookie=ccd0776e61637405 in_seq=0 out_seq=0 2024-05-03T18:43:39.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.653+0000 7f6174c00700 10 client.36587 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:43:39.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.654+0000 7f6174c00700 1 -- 172.21.15.33:0/832546301 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c07f2f0400 con 0x55c07f4141c0 2024-05-03T18:43:39.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.654+0000 7f6174c00700 10 client.36587 handle_command_reply: tid=0 2024-05-03T18:43:39.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.655+0000 7f619bcb1b80 1 client.36587 shutdown 2024-05-03T18:43:39.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.655+0000 7f619bcb1b80 1 -- 172.21.15.33:0/832546301 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c07f395930 msgr2=0x55c07f3ae790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:43:39.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.655+0000 7f619bcb1b80 1 --2- 172.21.15.33:0/832546301 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c07f395930 0x55c07f3ae790 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f617c009350 tx=0x7f617c00bfb0 comp rx=0 tx=0).stop 2024-05-03T18:43:39.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.655+0000 7f619bcb1b80 1 -- 172.21.15.33:0/832546301 shutdown_connections 2024-05-03T18:43:39.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.655+0000 7f619bcb1b80 1 --2- 172.21.15.33:0/832546301 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55c07f4141c0 0x55c07f4165b0 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:39.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.655+0000 7f619bcb1b80 1 --2- 172.21.15.33:0/832546301 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c07f3abc70 0x55c07f3ac0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:39.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.655+0000 7f619bcb1b80 1 --2- 172.21.15.33:0/832546301 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c07f395930 0x55c07f3ae790 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:39.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.655+0000 7f619bcb1b80 1 --2- 172.21.15.33:0/832546301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c07f396250 0x55c07f3aecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:39.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.655+0000 7f619bcb1b80 1 -- 172.21.15.33:0/832546301 >> 172.21.15.33:0/832546301 conn(0x55c07f3086d0 msgr2=0x55c07f384b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:43:39.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.656+0000 7f6174c00700 0 client.36587 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:43:39.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.656+0000 7f619bcb1b80 1 -- 172.21.15.33:0/832546301 shutdown_connections 2024-05-03T18:43:39.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.656+0000 7f619bcb1b80 1 -- 172.21.15.33:0/832546301 wait complete. 2024-05-03T18:43:39.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.657+0000 7f619bcb1b80 20 client.36587 trim_cache size 0 max 16384 2024-05-03T18:43:39.659 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-03T18:43:39.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.659+0000 7f6191400700 1 -- 172.21.15.33:0/20855414 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f61700b2bc0 msgr2=0x7f61700b5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:43:39.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.659+0000 7f6191400700 1 --2- 172.21.15.33:0/20855414 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f61700b2bc0 0x7f61700b5080 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f617c009750 tx=0x7f617c005ad0 comp rx=0 tx=0).stop 2024-05-03T18:43:39.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.659+0000 7f6191400700 1 -- 172.21.15.33:0/20855414 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f618c10fb60 msgr2=0x7f618c06ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:43:39.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.659+0000 7f6191400700 1 --2- 172.21.15.33:0/20855414 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f618c10fb60 0x7f618c06ea70 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f618400ab40 tx=0x7f618400ae50 comp rx=0 tx=0).stop 2024-05-03T18:43:39.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.659+0000 7f6191400700 1 -- 172.21.15.33:0/20855414 shutdown_connections 2024-05-03T18:43:39.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.659+0000 7f6191400700 1 --2- 172.21.15.33:0/20855414 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f618c10fb60 0x7f618c06ea70 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:39.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.659+0000 7f6191400700 1 --2- 172.21.15.33:0/20855414 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f618c109ad0 0x7f618c0749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:39.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.659+0000 7f6191400700 1 --2- 172.21.15.33:0/20855414 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f61700b2bc0 0x7f61700b5080 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:39.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.659+0000 7f6191400700 1 --2- 172.21.15.33:0/20855414 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f618c10a490 0x7f618c074ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:39.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.659+0000 7f6191400700 1 -- 172.21.15.33:0/20855414 >> 172.21.15.33:0/20855414 conn(0x7f618c076c70 msgr2=0x7f618c0770b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:43:39.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.660+0000 7f6191400700 1 -- 172.21.15.33:0/20855414 shutdown_connections 2024-05-03T18:43:39.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:39.660+0000 7f6191400700 1 -- 172.21.15.33:0/20855414 wait complete. 2024-05-03T18:43:39.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:43:39 smithi033 ceph-mon[28330]: pgmap v518: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 9.6 MiB/s wr, 514 op/s 2024-05-03T18:43:40.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:43:39 smithi099 ceph-mon[31658]: pgmap v518: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 9.6 MiB/s wr, 514 op/s 2024-05-03T18:43:40.111 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 745.0 s, 515.8 tps, lat 195.378 ms stddev 199.121 2024-05-03T18:43:40.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:43:39 smithi150 ceph-mon[32492]: pgmap v518: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 9.6 MiB/s wr, 514 op/s 2024-05-03T18:43:41.961 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:43:41 smithi033 ceph-mon[28330]: pgmap v519: 129 pgs: 129 active+clean; 3.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 737 KiB/s rd, 14 MiB/s wr, 772 op/s 2024-05-03T18:43:42.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:43:41 smithi099 ceph-mon[31658]: pgmap v519: 129 pgs: 129 active+clean; 3.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 737 KiB/s rd, 14 MiB/s wr, 772 op/s 2024-05-03T18:43:42.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:43:41 smithi150 ceph-mon[32492]: pgmap v519: 129 pgs: 129 active+clean; 3.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 737 KiB/s rd, 14 MiB/s wr, 772 op/s 2024-05-03T18:43:42.965 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-03T18:43:42.966 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-03T18:43:43.786 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:43:43.976 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:43:43 smithi033 ceph-mon[28330]: pgmap v520: 129 pgs: 129 active+clean; 3.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 529 KiB/s rd, 9.5 MiB/s wr, 537 op/s 2024-05-03T18:43:44.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:43:43 smithi099 ceph-mon[31658]: pgmap v520: 129 pgs: 129 active+clean; 3.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 529 KiB/s rd, 9.5 MiB/s wr, 537 op/s 2024-05-03T18:43:44.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:43:43 smithi150 ceph-mon[32492]: pgmap v520: 129 pgs: 129 active+clean; 3.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 529 KiB/s rd, 9.5 MiB/s wr, 537 op/s 2024-05-03T18:43:45.110 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 750.0 s, 544.8 tps, lat 181.980 ms stddev 80.911 2024-05-03T18:43:45.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:43:45 smithi033 ceph-mon[28330]: pgmap v521: 129 pgs: 129 active+clean; 3.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 725 KiB/s rd, 13 MiB/s wr, 756 op/s 2024-05-03T18:43:46.047 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:43:45 smithi099 ceph-mon[31658]: pgmap v521: 129 pgs: 129 active+clean; 3.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 725 KiB/s rd, 13 MiB/s wr, 756 op/s 2024-05-03T18:43:46.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:43:45 smithi150 ceph-mon[32492]: pgmap v521: 129 pgs: 129 active+clean; 3.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 725 KiB/s rd, 13 MiB/s wr, 756 op/s 2024-05-03T18:43:47.986 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:43:47 smithi033 ceph-mon[28330]: pgmap v522: 129 pgs: 129 active+clean; 3.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 9.8 MiB/s wr, 559 op/s 2024-05-03T18:43:48.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:43:47 smithi099 ceph-mon[31658]: pgmap v522: 129 pgs: 129 active+clean; 3.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 9.8 MiB/s wr, 559 op/s 2024-05-03T18:43:48.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:43:47 smithi150 ceph-mon[32492]: pgmap v522: 129 pgs: 129 active+clean; 3.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 9.8 MiB/s wr, 559 op/s 2024-05-03T18:43:48.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.691+0000 7f0484c00700 1 -- 172.21.15.33:0/2933504412 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f048006b180 msgr2=0x7f048006b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:43:48.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.691+0000 7f0484c00700 1 --2- 172.21.15.33:0/2933504412 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f048006b180 0x7f048006b600 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f0468009990 tx=0x7f0468009ca0 comp rx=0 tx=0).stop 2024-05-03T18:43:48.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.691+0000 7f0484c00700 1 -- 172.21.15.33:0/2933504412 shutdown_connections 2024-05-03T18:43:48.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.692+0000 7f0484c00700 1 --2- 172.21.15.33:0/2933504412 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f048006bb40 0x7f0480110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:48.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.692+0000 7f0484c00700 1 --2- 172.21.15.33:0/2933504412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0480109510 0x7f04801098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:48.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.693+0000 7f0484c00700 1 --2- 172.21.15.33:0/2933504412 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f048006b180 0x7f048006b600 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:48.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.693+0000 7f0484c00700 1 -- 172.21.15.33:0/2933504412 >> 172.21.15.33:0/2933504412 conn(0x7f0480079350 msgr2=0x7f0480079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:43:48.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.694+0000 7f0484c00700 1 -- 172.21.15.33:0/2933504412 shutdown_connections 2024-05-03T18:43:48.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.694+0000 7f0484c00700 1 -- 172.21.15.33:0/2933504412 wait complete. 2024-05-03T18:43:48.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.698+0000 7f0484c00700 1 Processor -- start 2024-05-03T18:43:48.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.700+0000 7f0484c00700 1 -- start start 2024-05-03T18:43:48.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.701+0000 7f0484c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f048006b180 0x7f048010ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:43:48.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.701+0000 7f0484c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f048006bb40 0x7f048010d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:43:48.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.702+0000 7f0484c00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0480109510 0x7f048010d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:43:48.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.702+0000 7f0484c00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0480103ba0 con 0x7f0480109510 2024-05-03T18:43:48.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.702+0000 7f0484c00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0480103a20 con 0x7f048006bb40 2024-05-03T18:43:48.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.702+0000 7f0484c00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f0480103d20 con 0x7f048006b180 2024-05-03T18:43:48.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.704+0000 7f047e800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0480109510 0x7f048010d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:43:48.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.704+0000 7f047e800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0480109510 0x7f048010d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41094/0 (socket says 172.21.15.33:41094) 2024-05-03T18:43:48.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.704+0000 7f047e800700 1 -- 172.21.15.33:0/689442005 learned_addr learned my addr 172.21.15.33:0/689442005 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:43:48.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.705+0000 7f0477e00700 1 --2- 172.21.15.33:0/689442005 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f048006bb40 0x7f048010d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:43:48.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.705+0000 7f047de00700 1 --2- 172.21.15.33:0/689442005 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f048006b180 0x7f048010ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:43:48.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.707+0000 7f047de00700 1 -- 172.21.15.33:0/689442005 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f048006bb40 msgr2=0x7f048010d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:43:48.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.707+0000 7f047de00700 1 --2- 172.21.15.33:0/689442005 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f048006bb40 0x7f048010d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:48.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.707+0000 7f047de00700 1 -- 172.21.15.33:0/689442005 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0480109510 msgr2=0x7f048010d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:43:48.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.707+0000 7f047de00700 1 --2- 172.21.15.33:0/689442005 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0480109510 0x7f048010d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:48.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.707+0000 7f047de00700 1 -- 172.21.15.33:0/689442005 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f048010dcf0 con 0x7f048006b180 2024-05-03T18:43:48.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.707+0000 7f0477e00700 1 --2- 172.21.15.33:0/689442005 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f048006bb40 0x7f048010d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:43:48.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.712+0000 7f047de00700 1 --2- 172.21.15.33:0/689442005 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f048006b180 0x7f048010ffd0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f047000d950 tx=0x7f047000dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:43:48.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.712+0000 7f0476a00700 1 -- 172.21.15.33:0/689442005 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0470016070 con 0x7f048006b180 2024-05-03T18:43:48.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.713+0000 7f0476a00700 1 -- 172.21.15.33:0/689442005 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f047000fd20 con 0x7f048006b180 2024-05-03T18:43:48.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.713+0000 7f0476a00700 1 -- 172.21.15.33:0/689442005 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f047000e3b0 con 0x7f048006b180 2024-05-03T18:43:48.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.713+0000 7f0484c00700 1 -- 172.21.15.33:0/689442005 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0480073f10 con 0x7f048006b180 2024-05-03T18:43:48.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.713+0000 7f0484c00700 1 -- 172.21.15.33:0/689442005 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04800743d0 con 0x7f048006b180 2024-05-03T18:43:48.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.722+0000 7f0476a00700 1 -- 172.21.15.33:0/689442005 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f0470019070 con 0x7f048006b180 2024-05-03T18:43:48.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.724+0000 7f0476a00700 1 --2- 172.21.15.33:0/689442005 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f04640b2a20 0x7f04640b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:43:48.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.724+0000 7f0476a00700 1 -- 172.21.15.33:0/689442005 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9474+0+0 (secure 0 0 0) 0x7f04700e31b0 con 0x7f048006b180 2024-05-03T18:43:48.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.725+0000 7f0477e00700 1 --2- 172.21.15.33:0/689442005 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f04640b2a20 0x7f04640b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:43:48.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.746+0000 7f0477e00700 1 --2- 172.21.15.33:0/689442005 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f04640b2a20 0x7f04640b4ee0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f04680098e0 tx=0x7f04680058e0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:43:48.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.749+0000 7f048f4a3b80 1 Processor -- start 2024-05-03T18:43:48.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.750+0000 7f048f4a3b80 1 -- start start 2024-05-03T18:43:48.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.753+0000 7f048f4a3b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a4c61429b0 0x55a4c6142d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:43:48.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.754+0000 7f048f4a3b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a4c61432d0 0x55a4c613cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:43:48.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.755+0000 7f048f4a3b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a4c613d5d0 0x55a4c613da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:43:48.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.755+0000 7f048f4a3b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a4c608faa0 con 0x55a4c61429b0 2024-05-03T18:43:48.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.755+0000 7f048f4a3b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55a4c602d7e0 con 0x55a4c613d5d0 2024-05-03T18:43:48.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.755+0000 7f048f4a3b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55a4c608ba60 con 0x55a4c61432d0 2024-05-03T18:43:48.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.755+0000 7f047de00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a4c61429b0 0x55a4c6142d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:43:48.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.756+0000 7f047de00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a4c61429b0 0x55a4c6142d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41096/0 (socket says 172.21.15.33:41096) 2024-05-03T18:43:48.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.756+0000 7f047de00700 1 -- 172.21.15.33:0/747938811 learned_addr learned my addr 172.21.15.33:0/747938811 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:43:48.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.757+0000 7f047e800700 1 --2- 172.21.15.33:0/747938811 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a4c613d5d0 0x55a4c613da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:43:48.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.757+0000 7f0477e00700 1 --2- 172.21.15.33:0/747938811 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a4c61432d0 0x55a4c613cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:43:48.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.759+0000 7f047e800700 1 -- 172.21.15.33:0/747938811 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a4c61432d0 msgr2=0x55a4c613cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:43:48.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.760+0000 7f047e800700 1 --2- 172.21.15.33:0/747938811 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a4c61432d0 0x55a4c613cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:48.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.760+0000 7f047e800700 1 -- 172.21.15.33:0/747938811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a4c61429b0 msgr2=0x55a4c6142d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:43:48.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.760+0000 7f047e800700 1 --2- 172.21.15.33:0/747938811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a4c61429b0 0x55a4c6142d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:48.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.760+0000 7f047e800700 1 -- 172.21.15.33:0/747938811 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4c5fdc950 con 0x55a4c613d5d0 2024-05-03T18:43:48.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.761+0000 7f047e800700 1 --2- 172.21.15.33:0/747938811 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a4c613d5d0 0x55a4c613da50 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0478009e80 tx=0x7f04780091b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=102e77ae74ad3631 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:43:48.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.761+0000 7f046f400700 1 -- 172.21.15.33:0/747938811 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0478004bb0 con 0x55a4c613d5d0 2024-05-03T18:43:48.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.762+0000 7f046f400700 1 -- 172.21.15.33:0/747938811 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0478004d50 con 0x55a4c613d5d0 2024-05-03T18:43:48.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.762+0000 7f046f400700 1 -- 172.21.15.33:0/747938811 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0478005280 con 0x55a4c613d5d0 2024-05-03T18:43:48.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.764+0000 7f048f4a3b80 1 -- 172.21.15.33:0/747938811 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a4c613d5d0 msgr2=0x55a4c613da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:43:48.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.764+0000 7f048f4a3b80 1 --2- 172.21.15.33:0/747938811 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a4c613d5d0 0x55a4c613da50 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0478009e80 tx=0x7f04780091b0 comp rx=0 tx=0).stop 2024-05-03T18:43:48.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.766+0000 7f048f4a3b80 1 -- 172.21.15.33:0/747938811 shutdown_connections 2024-05-03T18:43:48.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.766+0000 7f048f4a3b80 1 --2- 172.21.15.33:0/747938811 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a4c613d5d0 0x55a4c613da50 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:48.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.766+0000 7f048f4a3b80 1 --2- 172.21.15.33:0/747938811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a4c61429b0 0x55a4c6142d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:48.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.766+0000 7f048f4a3b80 1 --2- 172.21.15.33:0/747938811 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a4c61432d0 0x55a4c613cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:48.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.766+0000 7f048f4a3b80 1 -- 172.21.15.33:0/747938811 >> 172.21.15.33:0/747938811 conn(0x55a4c60b5750 msgr2=0x55a4c6137980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:43:48.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.767+0000 7f048f4a3b80 1 -- 172.21.15.33:0/747938811 shutdown_connections 2024-05-03T18:43:48.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.767+0000 7f048f4a3b80 1 -- 172.21.15.33:0/747938811 wait complete. 2024-05-03T18:43:48.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.767+0000 7f048f4a3b80 1 -- start start 2024-05-03T18:43:48.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.768+0000 7f048f4a3b80 1 Processor -- start 2024-05-03T18:43:48.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.770+0000 7f048f4a3b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a4c613d5d0 0x55a4c6148ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:43:48.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.770+0000 7f048f4a3b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a4c61429b0 0x55a4c6146240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:43:48.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.770+0000 7f048f4a3b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a4c61432d0 0x55a4c6146780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:43:48.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.770+0000 7f048f4a3b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a4c6149d80 con 0x55a4c613d5d0 2024-05-03T18:43:48.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.770+0000 7f048f4a3b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55a4c5ff5ba0 con 0x55a4c61432d0 2024-05-03T18:43:48.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.770+0000 7f048f4a3b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55a4c6149f00 con 0x55a4c61429b0 2024-05-03T18:43:48.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.770+0000 7f048f4a3b80 10 client.0 _ll_register_callbacks cb 0x55a4c6036cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:43:48.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.771+0000 7f047de00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a4c613d5d0 0x55a4c6148ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:43:48.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.771+0000 7f0477e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a4c61429b0 0x55a4c6146240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:43:48.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.771+0000 7f047de00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a4c613d5d0 0x55a4c6148ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41100/0 (socket says 172.21.15.33:41100) 2024-05-03T18:43:48.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.771+0000 7f0477e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a4c61429b0 0x55a4c6146240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:33184/0 (socket says 172.21.15.33:33184) 2024-05-03T18:43:48.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.771+0000 7f047de00700 1 -- 172.21.15.33:0/1528518389 learned_addr learned my addr 172.21.15.33:0/1528518389 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:43:48.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.771+0000 7f047e800700 1 --2- 172.21.15.33:0/1528518389 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a4c61432d0 0x55a4c6146780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:43:48.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.772+0000 7f047de00700 1 -- 172.21.15.33:0/1528518389 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a4c61429b0 msgr2=0x55a4c6146240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:43:48.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.772+0000 7f047de00700 1 --2- 172.21.15.33:0/1528518389 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a4c61429b0 0x55a4c6146240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:48.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.772+0000 7f047de00700 1 -- 172.21.15.33:0/1528518389 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a4c61432d0 msgr2=0x55a4c6146780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:43:48.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.772+0000 7f047de00700 1 --2- 172.21.15.33:0/1528518389 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a4c61432d0 0x55a4c6146780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:48.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.772+0000 7f047de00700 1 -- 172.21.15.33:0/1528518389 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4c60554a0 con 0x55a4c613d5d0 2024-05-03T18:43:48.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.772+0000 7f0477e00700 1 --2- 172.21.15.33:0/1528518389 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a4c61429b0 0x55a4c6146240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-05-03T18:43:48.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.773+0000 7f0463e00700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:43:48.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.773+0000 7f047de00700 1 --2- 172.21.15.33:0/1528518389 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a4c613d5d0 0x55a4c6148ea0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f04700e0b40 tx=0x7f04700aa610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbe1001192165b2d server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:43:48.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.773+0000 7f0463e00700 1 -- 172.21.15.33:0/1528518389 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0470020070 con 0x55a4c613d5d0 2024-05-03T18:43:48.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.774+0000 7f048f4a3b80 1 -- 172.21.15.33:0/1528518389 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0478004d50 con 0x55a4c613d5d0 2024-05-03T18:43:48.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.774+0000 7f0463e00700 1 -- 172.21.15.33:0/1528518389 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9474+0+0 (secure 0 0 0) 0x7f04700a66c0 con 0x55a4c613d5d0 2024-05-03T18:43:48.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.774+0000 7f0463e00700 1 -- 172.21.15.33:0/1528518389 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f0470020070 con 0x55a4c613d5d0 2024-05-03T18:43:48.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.774+0000 7f048f4a3b80 10 client.16815 fetch_fsmap learned FSMap version 21 2024-05-03T18:43:48.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.774+0000 7f048f4a3b80 1 -- 172.21.15.33:0/1528518389 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55a4c6154560 con 0x55a4c613d5d0 2024-05-03T18:43:48.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.775+0000 7f0463e00700 1 -- 172.21.15.33:0/1528518389 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0478004d50 con 0x55a4c613d5d0 2024-05-03T18:43:48.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.775+0000 7f0463e00700 1 -- 172.21.15.33:0/1528518389 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f04700a5a40 con 0x55a4c613d5d0 2024-05-03T18:43:48.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.776+0000 7f048f4a3b80 10 client.16815 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:43:48.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.776+0000 7f048f4a3b80 10 client.16815 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:43:48.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.776+0000 7f048f4a3b80 20 client.16815 populate_metadata read hostname 'smithi033' 2024-05-03T18:43:48.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.776+0000 7f048f4a3b80 1 --2- 172.21.15.33:0/1528518389 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55a4c6156af0 0x55a4c6158f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:43:48.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.776+0000 7f048f4a3b80 4 client.16815 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:43:48.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.776+0000 7f048f4a3b80 1 -- 172.21.15.33:0/1528518389 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a4c610a070 con 0x55a4c6156af0 2024-05-03T18:43:48.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.776+0000 7f0477e00700 1 --2- 172.21.15.33:0/1528518389 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55a4c6156af0 0x55a4c6158f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:43:48.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.778+0000 7f0477e00700 1 --2- 172.21.15.33:0/1528518389 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55a4c6156af0 0x55a4c6158f50 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9f0c4421409e0b5 server_cookie=bf7a06074a5e6644 in_seq=0 out_seq=0 2024-05-03T18:43:48.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.778+0000 7f0463e00700 10 client.16815 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:43:48.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.780+0000 7f0463e00700 1 -- 172.21.15.33:0/1528518389 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a4c610a070 con 0x55a4c6156af0 2024-05-03T18:43:48.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.780+0000 7f0463e00700 10 client.16815 handle_command_reply: tid=0 2024-05-03T18:43:48.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.780+0000 7f048f4a3b80 1 client.16815 shutdown 2024-05-03T18:43:48.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.780+0000 7f048f4a3b80 1 -- 172.21.15.33:0/1528518389 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a4c613d5d0 msgr2=0x55a4c6148ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:43:48.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.780+0000 7f048f4a3b80 1 --2- 172.21.15.33:0/1528518389 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a4c613d5d0 0x55a4c6148ea0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f04700e0b40 tx=0x7f04700aa610 comp rx=0 tx=0).stop 2024-05-03T18:43:48.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.780+0000 7f048f4a3b80 1 -- 172.21.15.33:0/1528518389 shutdown_connections 2024-05-03T18:43:48.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.780+0000 7f048f4a3b80 1 --2- 172.21.15.33:0/1528518389 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55a4c6156af0 0x55a4c6158f50 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:48.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.780+0000 7f048f4a3b80 1 --2- 172.21.15.33:0/1528518389 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a4c61432d0 0x55a4c6146780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:48.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.780+0000 7f048f4a3b80 1 --2- 172.21.15.33:0/1528518389 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a4c613d5d0 0x55a4c6148ea0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:48.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.780+0000 7f048f4a3b80 1 --2- 172.21.15.33:0/1528518389 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a4c61429b0 0x55a4c6146240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:48.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.780+0000 7f048f4a3b80 1 -- 172.21.15.33:0/1528518389 >> 172.21.15.33:0/1528518389 conn(0x55a4c60b5750 msgr2=0x55a4c6132430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:43:48.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.780+0000 7f0463e00700 0 client.16815 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:43:48.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.781+0000 7f048f4a3b80 1 -- 172.21.15.33:0/1528518389 shutdown_connections 2024-05-03T18:43:48.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.782+0000 7f048f4a3b80 1 -- 172.21.15.33:0/1528518389 wait complete. 2024-05-03T18:43:48.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.783+0000 7f048f4a3b80 20 client.16815 trim_cache size 0 max 16384 2024-05-03T18:43:48.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.825+0000 7f048f4a3b80 1 Processor -- start 2024-05-03T18:43:48.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.826+0000 7f048f4a3b80 1 -- start start 2024-05-03T18:43:48.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.826+0000 7f048f4a3b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a4c61429b0 0x55a4c613cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:43:48.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.826+0000 7f048f4a3b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a4c61432d0 0x55a4c61486e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:43:48.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.826+0000 7f048f4a3b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a4c6143f40 0x55a4c61322b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:43:48.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.826+0000 7f048f4a3b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f045c00d050 con 0x55a4c61429b0 2024-05-03T18:43:48.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.826+0000 7f048f4a3b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f045c00f930 con 0x55a4c61432d0 2024-05-03T18:43:48.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.826+0000 7f048f4a3b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f045c00d720 con 0x55a4c6143f40 2024-05-03T18:43:48.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.827+0000 7f047e800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a4c6143f40 0x55a4c61322b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:43:48.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.827+0000 7f047e800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a4c6143f40 0x55a4c61322b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:33186/0 (socket says 172.21.15.33:33186) 2024-05-03T18:43:48.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.827+0000 7f047e800700 1 -- 172.21.15.33:0/2770613596 learned_addr learned my addr 172.21.15.33:0/2770613596 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:43:48.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.827+0000 7f0477e00700 1 --2- 172.21.15.33:0/2770613596 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a4c61432d0 0x55a4c61486e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:43:48.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.827+0000 7f047de00700 1 --2- 172.21.15.33:0/2770613596 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a4c61429b0 0x55a4c613cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:43:48.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.828+0000 7f047e800700 1 -- 172.21.15.33:0/2770613596 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a4c61432d0 msgr2=0x55a4c61486e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:43:48.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.828+0000 7f047e800700 1 --2- 172.21.15.33:0/2770613596 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a4c61432d0 0x55a4c61486e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:48.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.828+0000 7f047e800700 1 -- 172.21.15.33:0/2770613596 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a4c61429b0 msgr2=0x55a4c613cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:43:48.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.828+0000 7f047e800700 1 --2- 172.21.15.33:0/2770613596 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a4c61429b0 0x55a4c613cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:48.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.828+0000 7f047e800700 1 -- 172.21.15.33:0/2770613596 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4c6142d90 con 0x55a4c6143f40 2024-05-03T18:43:48.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.829+0000 7f047e800700 1 --2- 172.21.15.33:0/2770613596 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a4c6143f40 0x55a4c61322b0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x55a4c613df90 tx=0x7f04780093f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=743d6a640733ac62 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:43:48.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.829+0000 7f046ea00700 1 -- 172.21.15.33:0/2770613596 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0478015480 con 0x55a4c6143f40 2024-05-03T18:43:48.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.829+0000 7f046ea00700 1 -- 172.21.15.33:0/2770613596 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0478015ab0 con 0x55a4c6143f40 2024-05-03T18:43:48.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.829+0000 7f046ea00700 1 -- 172.21.15.33:0/2770613596 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f047801ea60 con 0x55a4c6143f40 2024-05-03T18:43:48.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.831+0000 7f048f4a3b80 1 -- 172.21.15.33:0/2770613596 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a4c6143f40 msgr2=0x55a4c61322b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:43:48.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.831+0000 7f048f4a3b80 1 --2- 172.21.15.33:0/2770613596 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a4c6143f40 0x55a4c61322b0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x55a4c613df90 tx=0x7f04780093f0 comp rx=0 tx=0).stop 2024-05-03T18:43:48.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.831+0000 7f048f4a3b80 1 -- 172.21.15.33:0/2770613596 shutdown_connections 2024-05-03T18:43:48.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.831+0000 7f048f4a3b80 1 --2- 172.21.15.33:0/2770613596 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a4c6143f40 0x55a4c61322b0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:48.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.831+0000 7f048f4a3b80 1 --2- 172.21.15.33:0/2770613596 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a4c61429b0 0x55a4c613cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:48.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.831+0000 7f048f4a3b80 1 --2- 172.21.15.33:0/2770613596 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a4c61432d0 0x55a4c61486e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:48.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.831+0000 7f048f4a3b80 1 -- 172.21.15.33:0/2770613596 >> 172.21.15.33:0/2770613596 conn(0x55a4c60b5750 msgr2=0x55a4c6131ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:43:48.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.832+0000 7f048f4a3b80 1 -- 172.21.15.33:0/2770613596 shutdown_connections 2024-05-03T18:43:48.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.832+0000 7f048f4a3b80 1 -- 172.21.15.33:0/2770613596 wait complete. 2024-05-03T18:43:48.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.833+0000 7f048f4a3b80 1 -- start start 2024-05-03T18:43:48.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.834+0000 7f048f4a3b80 1 Processor -- start 2024-05-03T18:43:48.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.835+0000 7f048f4a3b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a4c61429b0 0x55a4c614f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:43:48.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.835+0000 7f048f4a3b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a4c61432d0 0x55a4c614c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:43:48.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.836+0000 7f048f4a3b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a4c6143f40 0x55a4c614c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:43:48.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.837+0000 7f047de00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a4c61429b0 0x55a4c614f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:43:48.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.837+0000 7f047de00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a4c61429b0 0x55a4c614f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:33194/0 (socket says 172.21.15.33:33194) 2024-05-03T18:43:48.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.837+0000 7f047de00700 1 -- 172.21.15.33:0/2187801753 learned_addr learned my addr 172.21.15.33:0/2187801753 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:43:48.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.837+0000 7f048f4a3b80 1 -- 172.21.15.33:0/2187801753 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f045c00ddf0 con 0x55a4c61432d0 2024-05-03T18:43:48.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.838+0000 7f048f4a3b80 1 -- 172.21.15.33:0/2187801753 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f045c00f260 con 0x55a4c6143f40 2024-05-03T18:43:48.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.838+0000 7f048f4a3b80 1 -- 172.21.15.33:0/2187801753 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f045c00e4c0 con 0x55a4c61429b0 2024-05-03T18:43:48.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.838+0000 7f048f4a3b80 10 client.0 _ll_register_callbacks cb 0x55a4c6036cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:43:48.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.838+0000 7f0477e00700 1 --2- 172.21.15.33:0/2187801753 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a4c61432d0 0x55a4c614c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:43:48.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.838+0000 7f047e800700 1 --2- 172.21.15.33:0/2187801753 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a4c6143f40 0x55a4c614c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:43:48.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.841+0000 7f0477e00700 1 -- 172.21.15.33:0/2187801753 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a4c61429b0 msgr2=0x55a4c614f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:43:48.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.841+0000 7f0477e00700 1 --2- 172.21.15.33:0/2187801753 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a4c61429b0 0x55a4c614f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:48.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.841+0000 7f0477e00700 1 -- 172.21.15.33:0/2187801753 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a4c6143f40 msgr2=0x55a4c614c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:43:48.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.841+0000 7f0477e00700 1 --2- 172.21.15.33:0/2187801753 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a4c6143f40 0x55a4c614c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:48.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.842+0000 7f0477e00700 1 -- 172.21.15.33:0/2187801753 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4c614cfd0 con 0x55a4c61432d0 2024-05-03T18:43:48.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.842+0000 7f047de00700 1 --2- 172.21.15.33:0/2187801753 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a4c61429b0 0x55a4c614f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:43:48.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.843+0000 7f0477e00700 1 --2- 172.21.15.33:0/2187801753 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a4c61432d0 0x55a4c614c1f0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f0468005950 tx=0x7f0468009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ed51a74f629cd server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:43:48.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.844+0000 7f0463e00700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:43:48.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.845+0000 7f0463e00700 1 -- 172.21.15.33:0/2187801753 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0468003bd0 con 0x55a4c61432d0 2024-05-03T18:43:48.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.845+0000 7f0463e00700 1 -- 172.21.15.33:0/2187801753 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9474+0+0 (secure 0 0 0) 0x7f0468003d70 con 0x55a4c61432d0 2024-05-03T18:43:48.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.845+0000 7f048f4a3b80 1 -- 172.21.15.33:0/2187801753 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0478015ab0 con 0x55a4c61432d0 2024-05-03T18:43:48.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.845+0000 7f0463e00700 1 -- 172.21.15.33:0/2187801753 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f0468031410 con 0x55a4c61432d0 2024-05-03T18:43:48.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.846+0000 7f048f4a3b80 10 client.16821 fetch_fsmap learned FSMap version 21 2024-05-03T18:43:48.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.846+0000 7f048f4a3b80 1 -- 172.21.15.33:0/2187801753 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55a4c61d35f0 con 0x55a4c61432d0 2024-05-03T18:43:48.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.847+0000 7f0463e00700 1 -- 172.21.15.33:0/2187801753 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0478015ab0 con 0x55a4c61432d0 2024-05-03T18:43:48.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.847+0000 7f0463e00700 1 -- 172.21.15.33:0/2187801753 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f046800f8b0 con 0x55a4c61432d0 2024-05-03T18:43:48.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.848+0000 7f048f4a3b80 10 client.16821 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:43:48.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.848+0000 7f048f4a3b80 10 client.16821 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:43:48.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.848+0000 7f048f4a3b80 20 client.16821 populate_metadata read hostname 'smithi033' 2024-05-03T18:43:48.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.848+0000 7f048f4a3b80 1 --2- 172.21.15.33:0/2187801753 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55a4c61d3c60 0x55a4c61d6080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:43:48.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.848+0000 7f048f4a3b80 4 client.16821 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-03T18:43:48.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.848+0000 7f048f4a3b80 1 -- 172.21.15.33:0/2187801753 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a4c609d480 con 0x55a4c61d3c60 2024-05-03T18:43:48.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.849+0000 7f047de00700 1 --2- 172.21.15.33:0/2187801753 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55a4c61d3c60 0x55a4c61d6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:43:48.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.850+0000 7f047de00700 1 --2- 172.21.15.33:0/2187801753 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55a4c61d3c60 0x55a4c61d6080 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3f5965e10f55a6 server_cookie=3d65a14cc0ebc989 in_seq=0 out_seq=0 2024-05-03T18:43:48.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.850+0000 7f0463e00700 10 client.16821 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:43:48.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.854+0000 7f0463e00700 1 -- 172.21.15.33:0/2187801753 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a4c609d480 con 0x55a4c61d3c60 2024-05-03T18:43:48.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.854+0000 7f0463e00700 10 client.16821 handle_command_reply: tid=0 2024-05-03T18:43:48.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.854+0000 7f048f4a3b80 1 client.16821 shutdown 2024-05-03T18:43:48.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.855+0000 7f048f4a3b80 1 -- 172.21.15.33:0/2187801753 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a4c61432d0 msgr2=0x55a4c614c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:43:48.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.855+0000 7f048f4a3b80 1 --2- 172.21.15.33:0/2187801753 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a4c61432d0 0x55a4c614c1f0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f0468005950 tx=0x7f0468009ca0 comp rx=0 tx=0).stop 2024-05-03T18:43:48.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.856+0000 7f048f4a3b80 1 -- 172.21.15.33:0/2187801753 shutdown_connections 2024-05-03T18:43:48.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.856+0000 7f048f4a3b80 1 --2- 172.21.15.33:0/2187801753 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55a4c61d3c60 0x55a4c61d6080 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:48.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.856+0000 7f048f4a3b80 1 --2- 172.21.15.33:0/2187801753 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a4c6143f40 0x55a4c614c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:48.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.856+0000 7f048f4a3b80 1 --2- 172.21.15.33:0/2187801753 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a4c61429b0 0x55a4c614f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:48.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.856+0000 7f048f4a3b80 1 --2- 172.21.15.33:0/2187801753 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a4c61432d0 0x55a4c614c1f0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:48.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.856+0000 7f0463e00700 0 client.16821 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:43:48.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.856+0000 7f048f4a3b80 1 -- 172.21.15.33:0/2187801753 >> 172.21.15.33:0/2187801753 conn(0x55a4c60b5750 msgr2=0x55a4c6132430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:43:48.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.857+0000 7f048f4a3b80 1 -- 172.21.15.33:0/2187801753 shutdown_connections 2024-05-03T18:43:48.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.857+0000 7f048f4a3b80 1 -- 172.21.15.33:0/2187801753 wait complete. 2024-05-03T18:43:48.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.858+0000 7f048f4a3b80 20 client.16821 trim_cache size 0 max 16384 2024-05-03T18:43:48.862 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-03T18:43:48.862 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-05-03T18:43:48.862 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "774bd20d-a401-4729-966a-b6352533c42d", 2024-05-03T18:43:48.862 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-05-03T18:43:48.862 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-03T18:43:48.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.860+0000 7f0484c00700 1 -- 172.21.15.33:0/689442005 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f04640b2a20 msgr2=0x7f04640b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:43:48.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.860+0000 7f0484c00700 1 --2- 172.21.15.33:0/689442005 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f04640b2a20 0x7f04640b4ee0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f04680098e0 tx=0x7f04680058e0 comp rx=0 tx=0).stop 2024-05-03T18:43:48.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.860+0000 7f0484c00700 1 -- 172.21.15.33:0/689442005 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f048006b180 msgr2=0x7f048010ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:43:48.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.860+0000 7f0484c00700 1 --2- 172.21.15.33:0/689442005 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f048006b180 0x7f048010ffd0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f047000d950 tx=0x7f047000dc60 comp rx=0 tx=0).stop 2024-05-03T18:43:48.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.860+0000 7f0484c00700 1 -- 172.21.15.33:0/689442005 shutdown_connections 2024-05-03T18:43:48.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.860+0000 7f0484c00700 1 --2- 172.21.15.33:0/689442005 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0480109510 0x7f048010d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:48.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.860+0000 7f0484c00700 1 --2- 172.21.15.33:0/689442005 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f048006b180 0x7f048010ffd0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:48.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.860+0000 7f0484c00700 1 --2- 172.21.15.33:0/689442005 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f04640b2a20 0x7f04640b4ee0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:48.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.860+0000 7f0484c00700 1 --2- 172.21.15.33:0/689442005 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f048006bb40 0x7f048010d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:48.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.860+0000 7f0484c00700 1 -- 172.21.15.33:0/689442005 >> 172.21.15.33:0/689442005 conn(0x7f0480079350 msgr2=0x7f0480100c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:43:48.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.860+0000 7f0484c00700 1 -- 172.21.15.33:0/689442005 shutdown_connections 2024-05-03T18:43:48.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:48.861+0000 7f0484c00700 1 -- 172.21.15.33:0/689442005 wait complete. 2024-05-03T18:43:49.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:43:49 smithi033 ceph-mon[28330]: pgmap v523: 129 pgs: 129 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 599 KiB/s rd, 9.6 MiB/s wr, 553 op/s 2024-05-03T18:43:49.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:43:49 smithi033 ceph-mon[28330]: scrub queued for path: / 2024-05-03T18:43:49.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:43:49 smithi033 ceph-mon[28330]: scrub summary: idle+waiting paths [/] 2024-05-03T18:43:49.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:43:49 smithi033 ceph-mon[28330]: scrub summary: active paths [/] 2024-05-03T18:43:50.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:43:49 smithi099 ceph-mon[31658]: pgmap v523: 129 pgs: 129 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 599 KiB/s rd, 9.6 MiB/s wr, 553 op/s 2024-05-03T18:43:50.047 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:43:49 smithi099 ceph-mon[31658]: scrub queued for path: / 2024-05-03T18:43:50.047 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:43:49 smithi099 ceph-mon[31658]: scrub summary: idle+waiting paths [/] 2024-05-03T18:43:50.047 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:43:49 smithi099 ceph-mon[31658]: scrub summary: active paths [/] 2024-05-03T18:43:50.111 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 755.0 s, 469.8 tps, lat 215.943 ms stddev 146.102 2024-05-03T18:43:50.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:43:49 smithi150 ceph-mon[32492]: pgmap v523: 129 pgs: 129 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 599 KiB/s rd, 9.6 MiB/s wr, 553 op/s 2024-05-03T18:43:50.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:43:49 smithi150 ceph-mon[32492]: scrub queued for path: / 2024-05-03T18:43:50.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:43:49 smithi150 ceph-mon[32492]: scrub summary: idle+waiting paths [/] 2024-05-03T18:43:50.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:43:49 smithi150 ceph-mon[32492]: scrub summary: active paths [/] 2024-05-03T18:43:50.438 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-03T18:43:51.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:43:50 smithi099 ceph-mon[31658]: scrub summary: idle+waiting paths [/] 2024-05-03T18:43:51.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:43:50 smithi150 ceph-mon[32492]: scrub summary: idle+waiting paths [/] 2024-05-03T18:43:51.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:43:50 smithi033 ceph-mon[28330]: scrub summary: idle+waiting paths [/] 2024-05-03T18:43:51.290 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:43:52.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:43:51 smithi099 ceph-mon[31658]: pgmap v524: 129 pgs: 129 active+clean; 3.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 13 MiB/s wr, 791 op/s 2024-05-03T18:43:52.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:43:51 smithi099 ceph-mon[31658]: scrub summary: idle 2024-05-03T18:43:52.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:43:51 smithi150 ceph-mon[32492]: pgmap v524: 129 pgs: 129 active+clean; 3.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 13 MiB/s wr, 791 op/s 2024-05-03T18:43:52.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:43:51 smithi150 ceph-mon[32492]: scrub summary: idle 2024-05-03T18:43:52.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:43:51 smithi033 ceph-mon[28330]: pgmap v524: 129 pgs: 129 active+clean; 3.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 13 MiB/s wr, 791 op/s 2024-05-03T18:43:52.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:43:51 smithi033 ceph-mon[28330]: scrub summary: idle 2024-05-03T18:43:54.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:43:53 smithi099 ceph-mon[31658]: pgmap v525: 129 pgs: 129 active+clean; 3.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 8.8 MiB/s wr, 519 op/s 2024-05-03T18:43:54.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:43:53 smithi150 ceph-mon[32492]: pgmap v525: 129 pgs: 129 active+clean; 3.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 8.8 MiB/s wr, 519 op/s 2024-05-03T18:43:54.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:43:53 smithi033 ceph-mon[28330]: pgmap v525: 129 pgs: 129 active+clean; 3.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 8.8 MiB/s wr, 519 op/s 2024-05-03T18:43:55.110 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 760.0 s, 549.8 tps, lat 178.465 ms stddev 76.063 2024-05-03T18:43:55.947 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:43:55 smithi033 ceph-mon[28330]: pgmap v526: 129 pgs: 129 active+clean; 3.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 702 KiB/s rd, 13 MiB/s wr, 759 op/s 2024-05-03T18:43:56.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:43:55 smithi099 ceph-mon[31658]: pgmap v526: 129 pgs: 129 active+clean; 3.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 702 KiB/s rd, 13 MiB/s wr, 759 op/s 2024-05-03T18:43:56.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:43:55 smithi150 ceph-mon[32492]: pgmap v526: 129 pgs: 129 active+clean; 3.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 702 KiB/s rd, 13 MiB/s wr, 759 op/s 2024-05-03T18:43:57.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.645+0000 7f69af800700 1 -- 172.21.15.33:0/1290271257 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69a806b6f0 msgr2=0x7f69a806bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:43:57.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.647+0000 7f69a7400700 1 -- 172.21.15.33:0/1290271257 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f699400bab0 con 0x7f69a806b6f0 2024-05-03T18:43:57.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.647+0000 7f69af800700 1 --2- 172.21.15.33:0/1290271257 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69a806b6f0 0x7f69a806bb50 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f6994009990 tx=0x7f6994009ca0 comp rx=0 tx=0).stop 2024-05-03T18:43:57.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.650+0000 7f69af800700 1 -- 172.21.15.33:0/1290271257 shutdown_connections 2024-05-03T18:43:57.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.650+0000 7f69af800700 1 --2- 172.21.15.33:0/1290271257 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f69a8105ee0 0x7f69a8106370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:57.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.650+0000 7f69af800700 1 --2- 172.21.15.33:0/1290271257 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f69a81083d0 0x7f69a806b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:57.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.650+0000 7f69af800700 1 --2- 172.21.15.33:0/1290271257 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69a806b6f0 0x7f69a806bb50 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:57.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.650+0000 7f69af800700 1 -- 172.21.15.33:0/1290271257 >> 172.21.15.33:0/1290271257 conn(0x7f69a8079350 msgr2=0x7f69a8079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:43:57.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.653+0000 7f69af800700 1 -- 172.21.15.33:0/1290271257 shutdown_connections 2024-05-03T18:43:57.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.654+0000 7f69af800700 1 -- 172.21.15.33:0/1290271257 wait complete. 2024-05-03T18:43:57.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.655+0000 7f69af800700 1 Processor -- start 2024-05-03T18:43:57.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.656+0000 7f69af800700 1 -- start start 2024-05-03T18:43:57.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.657+0000 7f69af800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69a806b6f0 0x7f69a8074a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:43:57.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.657+0000 7f69af800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f69a8105ee0 0x7f69a8074f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:43:57.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.657+0000 7f69af800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f69a81083d0 0x7f69a806db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:43:57.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.658+0000 7f69a7e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f69a8105ee0 0x7f69a8074f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:43:57.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.658+0000 7f69a7e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f69a8105ee0 0x7f69a8074f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:38188/0 (socket says 172.21.15.33:38188) 2024-05-03T18:43:57.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.658+0000 7f69a7e00700 1 -- 172.21.15.33:0/2809815502 learned_addr learned my addr 172.21.15.33:0/2809815502 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:43:57.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.658+0000 7f69af800700 1 -- 172.21.15.33:0/2809815502 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f69a8119b80 con 0x7f69a81083d0 2024-05-03T18:43:57.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.658+0000 7f69af800700 1 -- 172.21.15.33:0/2809815502 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f69a8119a00 con 0x7f69a8105ee0 2024-05-03T18:43:57.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.658+0000 7f69af800700 1 -- 172.21.15.33:0/2809815502 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f69a8119d00 con 0x7f69a806b6f0 2024-05-03T18:43:57.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.658+0000 7f69ace00700 1 --2- 172.21.15.33:0/2809815502 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69a806b6f0 0x7f69a8074a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:43:57.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.658+0000 7f69ad800700 1 --2- 172.21.15.33:0/2809815502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f69a81083d0 0x7f69a806db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:43:57.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.659+0000 7f69a7e00700 1 -- 172.21.15.33:0/2809815502 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69a806b6f0 msgr2=0x7f69a8074a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:43:57.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.659+0000 7f69a7e00700 1 --2- 172.21.15.33:0/2809815502 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69a806b6f0 0x7f69a8074a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:57.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.659+0000 7f69a7e00700 1 -- 172.21.15.33:0/2809815502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f69a81083d0 msgr2=0x7f69a806db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:43:57.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.659+0000 7f69a7e00700 1 --2- 172.21.15.33:0/2809815502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f69a81083d0 0x7f69a806db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:57.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.659+0000 7f69a7e00700 1 -- 172.21.15.33:0/2809815502 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69a806e430 con 0x7f69a8105ee0 2024-05-03T18:43:57.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.660+0000 7f69a7e00700 1 --2- 172.21.15.33:0/2809815502 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f69a8105ee0 0x7f69a8074f90 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6994006010 tx=0x7f699400f740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:43:57.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.661+0000 7f69a5600700 1 -- 172.21.15.33:0/2809815502 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6994026070 con 0x7f69a8105ee0 2024-05-03T18:43:57.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.661+0000 7f69a5600700 1 -- 172.21.15.33:0/2809815502 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f699400be40 con 0x7f69a8105ee0 2024-05-03T18:43:57.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.661+0000 7f69a5600700 1 -- 172.21.15.33:0/2809815502 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6994017650 con 0x7f69a8105ee0 2024-05-03T18:43:57.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.662+0000 7f69af800700 1 -- 172.21.15.33:0/2809815502 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69a806e6c0 con 0x7f69a8105ee0 2024-05-03T18:43:57.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.662+0000 7f69af800700 1 -- 172.21.15.33:0/2809815502 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69a8072580 con 0x7f69a8105ee0 2024-05-03T18:43:57.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.671+0000 7f69a5600700 1 -- 172.21.15.33:0/2809815502 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f6994009d10 con 0x7f69a8105ee0 2024-05-03T18:43:57.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.674+0000 7f69a5600700 1 --2- 172.21.15.33:0/2809815502 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f69900b2b30 0x7f69900b4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:43:57.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.674+0000 7f69a5600700 1 -- 172.21.15.33:0/2809815502 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9474+0+0 (secure 0 0 0) 0x7f69940e3760 con 0x7f69a8105ee0 2024-05-03T18:43:57.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.675+0000 7f69ace00700 1 --2- 172.21.15.33:0/2809815502 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f69900b2b30 0x7f69900b4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:43:57.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.677+0000 7f69ace00700 1 --2- 172.21.15.33:0/2809815502 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f69900b2b30 0x7f69900b4ff0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f699c005950 tx=0x7f699c0058e0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:43:57.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.683+0000 7f69ba266b80 1 Processor -- start 2024-05-03T18:43:57.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.683+0000 7f69ba266b80 1 -- start start 2024-05-03T18:43:57.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.684+0000 7f69ba266b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561fd5269930 0x561fd5269d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:43:57.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.684+0000 7f69ba266b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561fd526a250 0x561fd5263f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:43:57.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.684+0000 7f69ba266b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561fd5264550 0x561fd52649d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:43:57.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.684+0000 7f69ba266b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x561fd51b6a20 con 0x561fd526a250 2024-05-03T18:43:57.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.684+0000 7f69ba266b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x561fd5154760 con 0x561fd5269930 2024-05-03T18:43:57.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.684+0000 7f69ba266b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x561fd51b29e0 con 0x561fd5264550 2024-05-03T18:43:57.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.684+0000 7f69a7e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561fd526a250 0x561fd5263f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:43:57.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.685+0000 7f69ad800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561fd5264550 0x561fd52649d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:43:57.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.685+0000 7f69ad800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561fd5264550 0x561fd52649d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:44614/0 (socket says 172.21.15.33:44614) 2024-05-03T18:43:57.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.685+0000 7f69ad800700 1 -- 172.21.15.33:0/3880958050 learned_addr learned my addr 172.21.15.33:0/3880958050 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:43:57.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.686+0000 7f69ace00700 1 --2- 172.21.15.33:0/3880958050 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561fd5269930 0x561fd5269d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:43:57.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.687+0000 7f69ad800700 1 -- 172.21.15.33:0/3880958050 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561fd5269930 msgr2=0x561fd5269d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:43:57.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.687+0000 7f69ad800700 1 --2- 172.21.15.33:0/3880958050 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561fd5269930 0x561fd5269d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:57.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.687+0000 7f69ad800700 1 -- 172.21.15.33:0/3880958050 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561fd526a250 msgr2=0x561fd5263f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:43:57.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.687+0000 7f69ad800700 1 --2- 172.21.15.33:0/3880958050 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561fd526a250 0x561fd5263f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:57.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.687+0000 7f69ad800700 1 -- 172.21.15.33:0/3880958050 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fd51038d0 con 0x561fd5264550 2024-05-03T18:43:57.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.687+0000 7f69ad800700 1 --2- 172.21.15.33:0/3880958050 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561fd5264550 0x561fd52649d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f69a0009fd0 tx=0x7f69a0009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d70b2ad6e043791 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:43:57.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.688+0000 7f699a000700 1 -- 172.21.15.33:0/3880958050 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69a0003c00 con 0x561fd5264550 2024-05-03T18:43:57.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.688+0000 7f699a000700 1 -- 172.21.15.33:0/3880958050 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f69a0019070 con 0x561fd5264550 2024-05-03T18:43:57.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.688+0000 7f699a000700 1 -- 172.21.15.33:0/3880958050 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69a0014670 con 0x561fd5264550 2024-05-03T18:43:57.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.689+0000 7f69ba266b80 1 -- 172.21.15.33:0/3880958050 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561fd5264550 msgr2=0x561fd52649d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:43:57.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.689+0000 7f69ba266b80 1 --2- 172.21.15.33:0/3880958050 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561fd5264550 0x561fd52649d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f69a0009fd0 tx=0x7f69a0009920 comp rx=0 tx=0).stop 2024-05-03T18:43:57.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.690+0000 7f69ba266b80 1 -- 172.21.15.33:0/3880958050 shutdown_connections 2024-05-03T18:43:57.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.690+0000 7f69ba266b80 1 --2- 172.21.15.33:0/3880958050 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561fd5264550 0x561fd52649d0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:57.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.690+0000 7f69ba266b80 1 --2- 172.21.15.33:0/3880958050 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561fd5269930 0x561fd5269d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:57.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.690+0000 7f69ba266b80 1 --2- 172.21.15.33:0/3880958050 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561fd526a250 0x561fd5263f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:57.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.690+0000 7f69ba266b80 1 -- 172.21.15.33:0/3880958050 >> 172.21.15.33:0/3880958050 conn(0x561fd51dc6d0 msgr2=0x561fd525e900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:43:57.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.690+0000 7f69ba266b80 1 -- 172.21.15.33:0/3880958050 shutdown_connections 2024-05-03T18:43:57.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.690+0000 7f69ba266b80 1 -- 172.21.15.33:0/3880958050 wait complete. 2024-05-03T18:43:57.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.691+0000 7f69ba266b80 1 -- start start 2024-05-03T18:43:57.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.691+0000 7f69ba266b80 1 Processor -- start 2024-05-03T18:43:57.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.693+0000 7f69ba266b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561fd5264550 0x561fd526fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:43:57.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.693+0000 7f69ba266b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561fd5269930 0x561fd526d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:43:57.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.693+0000 7f69ba266b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561fd526a250 0x561fd526d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:43:57.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.693+0000 7f69ba266b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x561fd5270d00 con 0x561fd5269930 2024-05-03T18:43:57.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.693+0000 7f69ba266b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x561fd511cb20 con 0x561fd526a250 2024-05-03T18:43:57.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.693+0000 7f69ba266b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x561fd5270e80 con 0x561fd5264550 2024-05-03T18:43:57.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.693+0000 7f69ace00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561fd5264550 0x561fd526fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:43:57.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.694+0000 7f69ace00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561fd5264550 0x561fd526fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:44622/0 (socket says 172.21.15.33:44622) 2024-05-03T18:43:57.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.694+0000 7f69ace00700 1 -- 172.21.15.33:0/1660512927 learned_addr learned my addr 172.21.15.33:0/1660512927 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:43:57.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.694+0000 7f69ace00700 1 -- 172.21.15.33:0/1660512927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561fd526a250 msgr2=0x561fd526d740 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-03T18:43:57.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.694+0000 7f69ace00700 1 --2- 172.21.15.33:0/1660512927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561fd526a250 0x561fd526d740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:57.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.694+0000 7f69ace00700 1 -- 172.21.15.33:0/1660512927 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561fd5269930 msgr2=0x561fd526d200 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-03T18:43:57.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.696+0000 7f69a7e00700 1 --2- 172.21.15.33:0/1660512927 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561fd5269930 0x561fd526d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:43:57.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.696+0000 7f69ace00700 1 --2- 172.21.15.33:0/1660512927 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561fd5269930 0x561fd526d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:57.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.696+0000 7f69ace00700 1 -- 172.21.15.33:0/1660512927 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fd517c420 con 0x561fd5264550 2024-05-03T18:43:57.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.697+0000 7f69ba266b80 10 client.0 _ll_register_callbacks cb 0x561fd515dc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:43:57.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.697+0000 7f69ba266b80 1 -- 172.21.15.33:0/1660512927 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69a0019070 con 0x561fd5264550 2024-05-03T18:43:57.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.697+0000 7f69ace00700 1 --2- 172.21.15.33:0/1660512927 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561fd5264550 0x561fd526fe60 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f699c00ff80 tx=0x7f699c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81a02e186fe5eb71 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:43:57.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.697+0000 7f698ea00700 10 client.26677 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-03T18:43:57.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.699+0000 7f698ea00700 1 -- 172.21.15.33:0/1660512927 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f699c01f630 con 0x561fd5264550 2024-05-03T18:43:57.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.699+0000 7f698ea00700 1 -- 172.21.15.33:0/1660512927 <== mon.1 v2:172.21.15.150:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9474+0+0 (secure 0 0 0) 0x7f699c01f7d0 con 0x561fd5264550 2024-05-03T18:43:57.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.699+0000 7f698ea00700 1 -- 172.21.15.33:0/1660512927 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f699c01f9f0 con 0x561fd5264550 2024-05-03T18:43:57.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.699+0000 7f69ba266b80 10 client.26677 fetch_fsmap learned FSMap version 21 2024-05-03T18:43:57.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.699+0000 7f69ba266b80 1 -- 172.21.15.33:0/1660512927 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({fsmap=21}) -- 0x561fd527b310 con 0x561fd5264550 2024-05-03T18:43:57.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.701+0000 7f698ea00700 1 -- 172.21.15.33:0/1660512927 <== mon.1 v2:172.21.15.150:3300/0 4 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f699c01fb90 con 0x561fd5264550 2024-05-03T18:43:57.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.701+0000 7f69ba266b80 10 client.26677 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:43:57.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.701+0000 7f69ba266b80 10 client.26677 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:43:57.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.701+0000 7f69ba266b80 20 client.26677 populate_metadata read hostname 'smithi033' 2024-05-03T18:43:57.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.701+0000 7f69ba266b80 1 --2- 172.21.15.33:0/1660512927 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x561fd527d840 0x561fd527fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:43:57.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.701+0000 7f69ba266b80 4 client.26677 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:43:57.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.701+0000 7f69ba266b80 1 -- 172.21.15.33:0/1660512927 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561fd5230ff0 con 0x561fd527d840 2024-05-03T18:43:57.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.702+0000 7f69a7e00700 1 --2- 172.21.15.33:0/1660512927 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x561fd527d840 0x561fd527fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:43:57.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.703+0000 7f69a7e00700 1 --2- 172.21.15.33:0/1660512927 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x561fd527d840 0x561fd527fc50 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b9993c551038806 server_cookie=50ac764b92642dac in_seq=0 out_seq=0 2024-05-03T18:43:57.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.703+0000 7f698ea00700 10 client.26677 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:43:57.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.704+0000 7f698ea00700 1 -- 172.21.15.33:0/1660512927 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561fd5230ff0 con 0x561fd527d840 2024-05-03T18:43:57.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.704+0000 7f698ea00700 10 client.26677 handle_command_reply: tid=0 2024-05-03T18:43:57.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.705+0000 7f69ba266b80 1 client.26677 shutdown 2024-05-03T18:43:57.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.706+0000 7f69ba266b80 1 -- 172.21.15.33:0/1660512927 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561fd5264550 msgr2=0x561fd526fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:43:57.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.706+0000 7f69ba266b80 1 --2- 172.21.15.33:0/1660512927 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561fd5264550 0x561fd526fe60 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f699c00ff80 tx=0x7f699c004750 comp rx=0 tx=0).stop 2024-05-03T18:43:57.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.706+0000 7f69ba266b80 1 -- 172.21.15.33:0/1660512927 shutdown_connections 2024-05-03T18:43:57.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.706+0000 7f69ba266b80 1 --2- 172.21.15.33:0/1660512927 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x561fd527d840 0x561fd527fc50 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:57.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.706+0000 7f69ba266b80 1 --2- 172.21.15.33:0/1660512927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561fd526a250 0x561fd526d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:57.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.706+0000 7f69ba266b80 1 --2- 172.21.15.33:0/1660512927 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561fd5264550 0x561fd526fe60 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:57.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.706+0000 7f69ba266b80 1 --2- 172.21.15.33:0/1660512927 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561fd5269930 0x561fd526d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:57.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.706+0000 7f69ba266b80 1 -- 172.21.15.33:0/1660512927 >> 172.21.15.33:0/1660512927 conn(0x561fd51dc6d0 msgr2=0x561fd52593b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:43:57.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.706+0000 7f698ea00700 0 client.26677 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:43:57.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.707+0000 7f69ba266b80 1 -- 172.21.15.33:0/1660512927 shutdown_connections 2024-05-03T18:43:57.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.707+0000 7f69ba266b80 1 -- 172.21.15.33:0/1660512927 wait complete. 2024-05-03T18:43:57.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.707+0000 7f69ba266b80 20 client.26677 trim_cache size 0 max 16384 2024-05-03T18:43:57.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.754+0000 7f69ba266b80 1 Processor -- start 2024-05-03T18:43:57.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.754+0000 7f69ba266b80 1 -- start start 2024-05-03T18:43:57.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.754+0000 7f69ba266b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561fd5264550 0x561fd5263f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:43:57.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.754+0000 7f69ba266b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561fd5269930 0x561fd526c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:43:57.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.754+0000 7f69ba266b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561fd526a250 0x561fd5285520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:43:57.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.754+0000 7f69ba266b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f69a81ab0c0 con 0x561fd5269930 2024-05-03T18:43:57.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.754+0000 7f69ba266b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f69a81ad920 con 0x561fd5264550 2024-05-03T18:43:57.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.754+0000 7f69ba266b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f69a81ab710 con 0x561fd526a250 2024-05-03T18:43:57.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.756+0000 7f69a7e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561fd5269930 0x561fd526c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:43:57.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.756+0000 7f69a7e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561fd5269930 0x561fd526c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48254/0 (socket says 172.21.15.33:48254) 2024-05-03T18:43:57.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.756+0000 7f69a7e00700 1 -- 172.21.15.33:0/4135227658 learned_addr learned my addr 172.21.15.33:0/4135227658 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:43:57.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.756+0000 7f69a7e00700 1 -- 172.21.15.33:0/4135227658 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561fd526a250 msgr2=0x561fd5285520 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-03T18:43:57.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.756+0000 7f69a7e00700 1 --2- 172.21.15.33:0/4135227658 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561fd526a250 0x561fd5285520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:57.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.756+0000 7f69a7e00700 1 -- 172.21.15.33:0/4135227658 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561fd5264550 msgr2=0x561fd5263f30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-03T18:43:57.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.756+0000 7f69a7e00700 1 --2- 172.21.15.33:0/4135227658 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561fd5264550 0x561fd5263f30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:57.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.756+0000 7f69a7e00700 1 -- 172.21.15.33:0/4135227658 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fd5269340 con 0x561fd5269930 2024-05-03T18:43:57.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.757+0000 7f69a7e00700 1 --2- 172.21.15.33:0/4135227658 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561fd5269930 0x561fd526c200 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f6994021cc0 tx=0x7f699400ff10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1864572465d1202 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:43:57.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.759+0000 7f6999600700 1 -- 172.21.15.33:0/4135227658 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69940c3020 con 0x561fd5269930 2024-05-03T18:43:57.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.759+0000 7f6999600700 1 -- 172.21.15.33:0/4135227658 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f69940aa1c0 con 0x561fd5269930 2024-05-03T18:43:57.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.759+0000 7f6999600700 1 -- 172.21.15.33:0/4135227658 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69940b3480 con 0x561fd5269930 2024-05-03T18:43:57.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.760+0000 7f69ba266b80 1 -- 172.21.15.33:0/4135227658 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561fd5269930 msgr2=0x561fd526c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:43:57.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.760+0000 7f69ba266b80 1 --2- 172.21.15.33:0/4135227658 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561fd5269930 0x561fd526c200 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f6994021cc0 tx=0x7f699400ff10 comp rx=0 tx=0).stop 2024-05-03T18:43:57.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.761+0000 7f69ba266b80 1 -- 172.21.15.33:0/4135227658 shutdown_connections 2024-05-03T18:43:57.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.761+0000 7f69ba266b80 1 --2- 172.21.15.33:0/4135227658 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561fd526a250 0x561fd5285520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:57.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.761+0000 7f69ba266b80 1 --2- 172.21.15.33:0/4135227658 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561fd5264550 0x561fd5263f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:57.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.762+0000 7f69ba266b80 1 --2- 172.21.15.33:0/4135227658 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561fd5269930 0x561fd526c200 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:57.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.762+0000 7f69ba266b80 1 -- 172.21.15.33:0/4135227658 >> 172.21.15.33:0/4135227658 conn(0x561fd51dc6d0 msgr2=0x561fd526baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:43:57.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.763+0000 7f69ba266b80 1 -- 172.21.15.33:0/4135227658 shutdown_connections 2024-05-03T18:43:57.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.763+0000 7f69ba266b80 1 -- 172.21.15.33:0/4135227658 wait complete. 2024-05-03T18:43:57.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.764+0000 7f69ba266b80 1 -- start start 2024-05-03T18:43:57.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.764+0000 7f69ba266b80 1 Processor -- start 2024-05-03T18:43:57.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.766+0000 7f69ba266b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561fd5264550 0x561fd52762f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:43:57.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.767+0000 7f69ba266b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561fd5269930 0x561fd5276830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:43:57.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.768+0000 7f69ba266b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561fd526a250 0x561fd52732e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:43:57.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.768+0000 7f69ba266b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f69a81abde0 con 0x561fd5269930 2024-05-03T18:43:57.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.768+0000 7f69ba266b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f69a81ad250 con 0x561fd5264550 2024-05-03T18:43:57.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.768+0000 7f69ba266b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f69a81ac4b0 con 0x561fd526a250 2024-05-03T18:43:57.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.769+0000 7f69ba266b80 10 client.0 _ll_register_callbacks cb 0x561fd526e240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:43:57.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.769+0000 7f69ad800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561fd526a250 0x561fd52732e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:43:57.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.768+0000 7f69ace00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561fd5264550 0x561fd52762f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:43:57.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.769+0000 7f69ad800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561fd526a250 0x561fd52732e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:44642/0 (socket says 172.21.15.33:44642) 2024-05-03T18:43:57.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.769+0000 7f69ad800700 1 -- 172.21.15.33:0/3973878524 learned_addr learned my addr 172.21.15.33:0/3973878524 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:43:57.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.772+0000 7f69ad800700 1 -- 172.21.15.33:0/3973878524 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561fd5264550 msgr2=0x561fd52762f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:43:57.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.772+0000 7f69a7e00700 1 --2- 172.21.15.33:0/3973878524 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561fd5269930 0x561fd5276830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:43:57.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.773+0000 7f69ad800700 1 --2- 172.21.15.33:0/3973878524 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561fd5264550 0x561fd52762f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:57.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.773+0000 7f69ad800700 1 -- 172.21.15.33:0/3973878524 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561fd5269930 msgr2=0x561fd5276830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:43:57.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.773+0000 7f69ad800700 1 --2- 172.21.15.33:0/3973878524 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561fd5269930 0x561fd5276830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:57.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.773+0000 7f69ad800700 1 -- 172.21.15.33:0/3973878524 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fd5273b20 con 0x561fd526a250 2024-05-03T18:43:57.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.775+0000 7f69ace00700 1 --2- 172.21.15.33:0/3973878524 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561fd5264550 0x561fd52762f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:43:57.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.775+0000 7f69ad800700 1 --2- 172.21.15.33:0/3973878524 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561fd526a250 0x561fd52732e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f69a00094d0 tx=0x7f69a0004f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7bb001a8d1bbbcf8 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:43:57.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.775+0000 7f698ea00700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-03T18:43:57.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.777+0000 7f698ea00700 1 -- 172.21.15.33:0/3973878524 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69a00148e0 con 0x561fd526a250 2024-05-03T18:43:57.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.778+0000 7f698ea00700 1 -- 172.21.15.33:0/3973878524 <== mon.1 v2:172.21.15.150:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9474+0+0 (secure 0 0 0) 0x7f69a0014a80 con 0x561fd526a250 2024-05-03T18:43:57.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.780+0000 7f69ba266b80 1 -- 172.21.15.33:0/3973878524 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69940aa1c0 con 0x561fd526a250 2024-05-03T18:43:57.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.804+0000 7f698ea00700 1 -- 172.21.15.33:0/3973878524 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f69a0014ca0 con 0x561fd526a250 2024-05-03T18:43:57.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.804+0000 7f69ba266b80 10 client.26680 fetch_fsmap learned FSMap version 21 2024-05-03T18:43:57.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.804+0000 7f69ba266b80 1 -- 172.21.15.33:0/3973878524 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x561fd52e8d50 con 0x561fd526a250 2024-05-03T18:43:57.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.805+0000 7f698ea00700 1 -- 172.21.15.33:0/3973878524 <== mon.1 v2:172.21.15.150:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f69940aa1c0 con 0x561fd526a250 2024-05-03T18:43:57.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.805+0000 7f698ea00700 1 -- 172.21.15.33:0/3973878524 <== mon.1 v2:172.21.15.150:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f69a0028410 con 0x561fd526a250 2024-05-03T18:43:57.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.805+0000 7f69ba266b80 10 client.26680 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:43:57.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.805+0000 7f69ba266b80 10 client.26680 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:43:57.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.805+0000 7f69ba266b80 20 client.26680 populate_metadata read hostname 'smithi033' 2024-05-03T18:43:57.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.805+0000 7f69ba266b80 1 --2- 172.21.15.33:0/3973878524 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x561fd52e9030 0x561fd52eb420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:43:57.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.805+0000 7f69ba266b80 4 client.26680 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-03T18:43:57.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.805+0000 7f69ba266b80 1 -- 172.21.15.33:0/3973878524 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561fd51c4400 con 0x561fd52e9030 2024-05-03T18:43:57.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.806+0000 7f69ace00700 1 --2- 172.21.15.33:0/3973878524 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x561fd52e9030 0x561fd52eb420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:43:57.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.807+0000 7f69ace00700 1 --2- 172.21.15.33:0/3973878524 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x561fd52e9030 0x561fd52eb420 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a75c33ec8f3bc0b9 server_cookie=2007f5ecd525c004 in_seq=0 out_seq=0 2024-05-03T18:43:57.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.807+0000 7f698ea00700 10 client.26680 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:43:57.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.808+0000 7f698ea00700 1 -- 172.21.15.33:0/3973878524 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x561fd51c4400 con 0x561fd52e9030 2024-05-03T18:43:57.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.808+0000 7f698ea00700 10 client.26680 handle_command_reply: tid=0 2024-05-03T18:43:57.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.808+0000 7f69ba266b80 1 client.26680 shutdown 2024-05-03T18:43:57.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.808+0000 7f69ba266b80 1 -- 172.21.15.33:0/3973878524 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561fd526a250 msgr2=0x561fd52732e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:43:57.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.808+0000 7f69ba266b80 1 --2- 172.21.15.33:0/3973878524 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561fd526a250 0x561fd52732e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f69a00094d0 tx=0x7f69a0004f80 comp rx=0 tx=0).stop 2024-05-03T18:43:57.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.809+0000 7f69ba266b80 1 -- 172.21.15.33:0/3973878524 shutdown_connections 2024-05-03T18:43:57.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.809+0000 7f69ba266b80 1 --2- 172.21.15.33:0/3973878524 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x561fd52e9030 0x561fd52eb420 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:57.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.809+0000 7f69ba266b80 1 --2- 172.21.15.33:0/3973878524 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561fd526a250 0x561fd52732e0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:57.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.809+0000 7f69ba266b80 1 --2- 172.21.15.33:0/3973878524 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561fd5264550 0x561fd52762f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:57.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.809+0000 7f698ea00700 0 client.26680 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:43:57.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.809+0000 7f69ba266b80 1 --2- 172.21.15.33:0/3973878524 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561fd5269930 0x561fd5276830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:57.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.809+0000 7f69ba266b80 1 -- 172.21.15.33:0/3973878524 >> 172.21.15.33:0/3973878524 conn(0x561fd51dc6d0 msgr2=0x561fd52593b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:43:57.814 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-03T18:43:57.815 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-05-03T18:43:57.815 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-05-03T18:43:57.815 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-03T18:43:57.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.811+0000 7f69ba266b80 1 -- 172.21.15.33:0/3973878524 shutdown_connections 2024-05-03T18:43:57.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.811+0000 7f69ba266b80 1 -- 172.21.15.33:0/3973878524 wait complete. 2024-05-03T18:43:57.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.811+0000 7f69ba266b80 20 client.26680 trim_cache size 0 max 16384 2024-05-03T18:43:57.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.813+0000 7f69af800700 1 -- 172.21.15.33:0/2809815502 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f69900b2b30 msgr2=0x7f69900b4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:43:57.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.813+0000 7f69af800700 1 --2- 172.21.15.33:0/2809815502 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f69900b2b30 0x7f69900b4ff0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f699c005950 tx=0x7f699c0058e0 comp rx=0 tx=0).stop 2024-05-03T18:43:57.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.813+0000 7f69af800700 1 -- 172.21.15.33:0/2809815502 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f69a8105ee0 msgr2=0x7f69a8074f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:43:57.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.813+0000 7f69af800700 1 --2- 172.21.15.33:0/2809815502 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f69a8105ee0 0x7f69a8074f90 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6994006010 tx=0x7f699400f740 comp rx=0 tx=0).stop 2024-05-03T18:43:57.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.813+0000 7f69af800700 1 -- 172.21.15.33:0/2809815502 shutdown_connections 2024-05-03T18:43:57.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.813+0000 7f69af800700 1 --2- 172.21.15.33:0/2809815502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f69a81083d0 0x7f69a806db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:57.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.813+0000 7f69af800700 1 --2- 172.21.15.33:0/2809815502 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69a806b6f0 0x7f69a8074a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:57.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.813+0000 7f69af800700 1 --2- 172.21.15.33:0/2809815502 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f69900b2b30 0x7f69900b4ff0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:57.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.813+0000 7f69af800700 1 --2- 172.21.15.33:0/2809815502 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f69a8105ee0 0x7f69a8074f90 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:57.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.813+0000 7f69af800700 1 -- 172.21.15.33:0/2809815502 >> 172.21.15.33:0/2809815502 conn(0x7f69a8079350 msgr2=0x7f69a8104ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:43:57.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.814+0000 7f69af800700 1 -- 172.21.15.33:0/2809815502 shutdown_connections 2024-05-03T18:43:57.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:43:57.814+0000 7f69af800700 1 -- 172.21.15.33:0/2809815502 wait complete. 2024-05-03T18:43:58.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:43:57 smithi099 ceph-mon[31658]: pgmap v527: 129 pgs: 129 active+clean; 3.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 9.0 MiB/s wr, 554 op/s 2024-05-03T18:43:58.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:43:57 smithi150 ceph-mon[32492]: pgmap v527: 129 pgs: 129 active+clean; 3.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 9.0 MiB/s wr, 554 op/s 2024-05-03T18:43:58.193 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:43:57 smithi033 ceph-mon[28330]: pgmap v527: 129 pgs: 129 active+clean; 3.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 9.0 MiB/s wr, 554 op/s 2024-05-03T18:43:59.385 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-03T18:43:59.386 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T18:44:00.111 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 765.0 s, 471.8 tps, lat 212.601 ms stddev 97.938 2024-05-03T18:44:00.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:43:59 smithi150 ceph-mon[32492]: pgmap v528: 129 pgs: 129 active+clean; 3.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 506 KiB/s rd, 8.9 MiB/s wr, 544 op/s 2024-05-03T18:44:00.159 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:44:00.237 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:43:59 smithi033 ceph-mon[28330]: pgmap v528: 129 pgs: 129 active+clean; 3.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 506 KiB/s rd, 8.9 MiB/s wr, 544 op/s 2024-05-03T18:44:00.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:43:59 smithi099 ceph-mon[31658]: pgmap v528: 129 pgs: 129 active+clean; 3.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 506 KiB/s rd, 8.9 MiB/s wr, 544 op/s 2024-05-03T18:44:01.805 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:44:01 smithi033 ceph-mon[28330]: pgmap v529: 129 pgs: 129 active+clean; 3.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 12 MiB/s wr, 759 op/s 2024-05-03T18:44:02.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:44:01 smithi150 ceph-mon[32492]: pgmap v529: 129 pgs: 129 active+clean; 3.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 12 MiB/s wr, 759 op/s 2024-05-03T18:44:02.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:44:01 smithi099 ceph-mon[31658]: pgmap v529: 129 pgs: 129 active+clean; 3.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 12 MiB/s wr, 759 op/s 2024-05-03T18:44:04.133 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:44:03 smithi033 ceph-mon[28330]: pgmap v530: 129 pgs: 129 active+clean; 3.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 7.9 MiB/s wr, 502 op/s 2024-05-03T18:44:04.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:44:03 smithi150 ceph-mon[32492]: pgmap v530: 129 pgs: 129 active+clean; 3.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 7.9 MiB/s wr, 502 op/s 2024-05-03T18:44:04.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:44:03 smithi099 ceph-mon[31658]: pgmap v530: 129 pgs: 129 active+clean; 3.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 7.9 MiB/s wr, 502 op/s 2024-05-03T18:44:05.112 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 770.0 s, 340.0 tps, lat 293.744 ms stddev 347.985 2024-05-03T18:44:06.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:44:05 smithi150 ceph-mon[32492]: pgmap v531: 129 pgs: 129 active+clean; 3.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 10 MiB/s wr, 704 op/s 2024-05-03T18:44:06.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:44:05 smithi033 ceph-mon[28330]: pgmap v531: 129 pgs: 129 active+clean; 3.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 10 MiB/s wr, 704 op/s 2024-05-03T18:44:06.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:44:05 smithi099 ceph-mon[31658]: pgmap v531: 129 pgs: 129 active+clean; 3.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 10 MiB/s wr, 704 op/s 2024-05-03T18:44:06.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:06.518+0000 7f4915e00700 1 -- 172.21.15.33:0/2394650047 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49101056c0 msgr2=0x7f4910105b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:06.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:06.518+0000 7f4915e00700 1 --2- 172.21.15.33:0/2394650047 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49101056c0 0x7f4910105b40 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f48f80098f0 tx=0x7f48f8009c00 comp rx=0 tx=0).stop 2024-05-03T18:44:06.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:06.518+0000 7f4915e00700 1 -- 172.21.15.33:0/2394650047 shutdown_connections 2024-05-03T18:44:06.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:06.518+0000 7f4915e00700 1 --2- 172.21.15.33:0/2394650047 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4910106080 0x7f4910112590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:06.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:06.518+0000 7f4915e00700 1 --2- 172.21.15.33:0/2394650047 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f491010b6c0 0x7f491010baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:06.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:06.518+0000 7f4915e00700 1 --2- 172.21.15.33:0/2394650047 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49101056c0 0x7f4910105b40 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:06.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:06.518+0000 7f4915e00700 1 -- 172.21.15.33:0/2394650047 >> 172.21.15.33:0/2394650047 conn(0x7f49101011d0 msgr2=0x7f49101035f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:44:06.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:06.519+0000 7f4915e00700 1 -- 172.21.15.33:0/2394650047 shutdown_connections 2024-05-03T18:44:06.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:06.519+0000 7f4915e00700 1 -- 172.21.15.33:0/2394650047 wait complete. 2024-05-03T18:44:06.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:06.521+0000 7f4915e00700 1 Processor -- start 2024-05-03T18:44:06.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:06.521+0000 7f4915e00700 1 -- start start 2024-05-03T18:44:06.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:06.523+0000 7f4915e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f49101056c0 0x7f491019bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:06.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:06.523+0000 7f4915e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4910106080 0x7f491019c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:06.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:06.523+0000 7f4915e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f491010b6c0 0x7f49101a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:06.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:06.523+0000 7f4915e00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4910115800 con 0x7f491010b6c0 2024-05-03T18:44:06.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:06.523+0000 7f4915e00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4910115680 con 0x7f4910106080 2024-05-03T18:44:06.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:06.523+0000 7f4915e00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f4910115980 con 0x7f49101056c0 2024-05-03T18:44:06.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:06.524+0000 7f490ea00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f49101056c0 0x7f491019bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:06.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:06.524+0000 7f490ea00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f49101056c0 0x7f491019bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:36024/0 (socket says 172.21.15.33:36024) 2024-05-03T18:44:06.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:06.524+0000 7f490ea00700 1 -- 172.21.15.33:0/4238502718 learned_addr learned my addr 172.21.15.33:0/4238502718 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:44:06.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:06.524+0000 7f490e000700 1 --2- 172.21.15.33:0/4238502718 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4910106080 0x7f491019c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:06.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:06.525+0000 7f490f400700 1 --2- 172.21.15.33:0/4238502718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f491010b6c0 0x7f49101a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:06.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:06.525+0000 7f490e000700 1 -- 172.21.15.33:0/4238502718 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f49101056c0 msgr2=0x7f491019bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:06.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:06.525+0000 7f490e000700 1 --2- 172.21.15.33:0/4238502718 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f49101056c0 0x7f491019bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:06.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:06.525+0000 7f490e000700 1 -- 172.21.15.33:0/4238502718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f491010b6c0 msgr2=0x7f49101a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:06.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:06.525+0000 7f490e000700 1 --2- 172.21.15.33:0/4238502718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f491010b6c0 0x7f49101a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:06.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:06.525+0000 7f490e000700 1 -- 172.21.15.33:0/4238502718 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49101a0d60 con 0x7f4910106080 2024-05-03T18:44:06.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:06.526+0000 7f490ea00700 1 --2- 172.21.15.33:0/4238502718 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f49101056c0 0x7f491019bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:44:06.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:06.526+0000 7f490f400700 1 --2- 172.21.15.33:0/4238502718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f491010b6c0 0x7f49101a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:44:06.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:06.526+0000 7f490e000700 1 --2- 172.21.15.33:0/4238502718 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4910106080 0x7f491019c290 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f48f8009fd0 tx=0x7f48f800fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:44:06.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:06.527+0000 7f4914a00700 1 -- 172.21.15.33:0/4238502718 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f800b790 con 0x7f4910106080 2024-05-03T18:44:06.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:06.527+0000 7f4915e00700 1 -- 172.21.15.33:0/4238502718 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49101a0ff0 con 0x7f4910106080 2024-05-03T18:44:06.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:06.527+0000 7f4915e00700 1 -- 172.21.15.33:0/4238502718 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49101a88d0 con 0x7f4910106080 2024-05-03T18:44:06.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:06.527+0000 7f4914a00700 1 -- 172.21.15.33:0/4238502718 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f48f8009490 con 0x7f4910106080 2024-05-03T18:44:06.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:06.528+0000 7f4914a00700 1 -- 172.21.15.33:0/4238502718 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f8003680 con 0x7f4910106080 2024-05-03T18:44:06.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:06.530+0000 7f4915e00700 1 -- 172.21.15.33:0/4238502718 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48ec005330 con 0x7f4910106080 2024-05-03T18:44:06.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:06.534+0000 7f4914a00700 1 -- 172.21.15.33:0/4238502718 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f48f8005040 con 0x7f4910106080 2024-05-03T18:44:06.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:06.536+0000 7f4914a00700 1 --2- 172.21.15.33:0/4238502718 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f48f40b2a70 0x7f48f40b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:06.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:06.537+0000 7f4914a00700 1 -- 172.21.15.33:0/4238502718 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9474+0+0 (secure 0 0 0) 0x7f48f80e3ba0 con 0x7f4910106080 2024-05-03T18:44:06.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:06.537+0000 7f490ea00700 1 --2- 172.21.15.33:0/4238502718 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f48f40b2a70 0x7f48f40b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:06.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:06.539+0000 7f490ea00700 1 --2- 172.21.15.33:0/4238502718 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f48f40b2a70 0x7f48f40b4f30 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4908009fd0 tx=0x7f490800b040 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:44:06.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:06.550+0000 7f4914a00700 1 -- 172.21.15.33:0/4238502718 <== mon.2 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f48f80e3e20 con 0x7f4910106080 2024-05-03T18:44:07.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:07.108+0000 7f4915e00700 1 -- 172.21.15.33:0/4238502718 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f48ec005620 con 0x7f4910106080 2024-05-03T18:44:07.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:07.109+0000 7f4914a00700 1 -- 172.21.15.33:0/4238502718 <== mon.2 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11801 (secure 0 0 0) 0x7f48f8013030 con 0x7f4910106080 2024-05-03T18:44:07.115 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:44:07.115 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1327393503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1327393503},{"type":"v1","addr":"172.21.15.33:6835","nonce":1327393503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/2461472948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":2461472948},{"type":"v1","addr":"172.21.15.33:6837","nonce":2461472948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14559,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/442269281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":442269281},{"type":"v1","addr":"172.21.15.33:6839","nonce":442269281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14571,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/645571756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":645571756},{"type":"v1","addr":"172.21.15.33:6841","nonce":645571756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/203598909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":203598909},{"type":"v1","addr":"172.21.15.150:6833","nonce":203598909}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/4029842493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":4029842493},{"type":"v1","addr":"172.21.15.150:6835","nonce":4029842493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3463664423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3463664423},{"type":"v1","addr":"172.21.15.150:6837","nonce":3463664423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24481,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6839/4051648018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":4051648018},{"type":"v1","addr":"172.21.15.150:6839","nonce":4051648018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34418,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1865105479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1865105479},{"type":"v1","addr":"172.21.15.99:6833","nonce":1865105479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34445,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/1206393914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1206393914},{"type":"v1","addr":"172.21.15.99:6835","nonce":1206393914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-03T18:22:28.837091+0000","modified":"2024-05-03T18:23:49.758398+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34508},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34472":{"gid":34472,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6837/3360970342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3360970342},{"type":"v1","addr":"172.21.15.99:6837","nonce":3360970342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34508":{"gid":34508,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/919907308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":919907308},{"type":"v1","addr":"172.21.15.99:6839","nonce":919907308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34508,"qdb_cluster":[34508]},"id":1}]} 2024-05-03T18:44:07.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:07.115+0000 7f4915e00700 1 -- 172.21.15.33:0/4238502718 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f48f40b2a70 msgr2=0x7f48f40b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:07.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:07.115+0000 7f4915e00700 1 --2- 172.21.15.33:0/4238502718 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f48f40b2a70 0x7f48f40b4f30 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4908009fd0 tx=0x7f490800b040 comp rx=0 tx=0).stop 2024-05-03T18:44:07.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:07.115+0000 7f4915e00700 1 -- 172.21.15.33:0/4238502718 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4910106080 msgr2=0x7f491019c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:07.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:07.115+0000 7f4915e00700 1 --2- 172.21.15.33:0/4238502718 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4910106080 0x7f491019c290 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f48f8009fd0 tx=0x7f48f800fa00 comp rx=0 tx=0).stop 2024-05-03T18:44:07.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:07.115+0000 7f4915e00700 1 -- 172.21.15.33:0/4238502718 shutdown_connections 2024-05-03T18:44:07.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:07.115+0000 7f4915e00700 1 --2- 172.21.15.33:0/4238502718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f491010b6c0 0x7f49101a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:07.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:07.115+0000 7f4915e00700 1 --2- 172.21.15.33:0/4238502718 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f49101056c0 0x7f491019bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:07.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:07.115+0000 7f4915e00700 1 --2- 172.21.15.33:0/4238502718 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f48f40b2a70 0x7f48f40b4f30 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:07.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:07.115+0000 7f4915e00700 1 --2- 172.21.15.33:0/4238502718 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4910106080 0x7f491019c290 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:07.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:07.115+0000 7f4915e00700 1 -- 172.21.15.33:0/4238502718 >> 172.21.15.33:0/4238502718 conn(0x7f49101011d0 msgr2=0x7f4910101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:44:07.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:07.117+0000 7f4915e00700 1 -- 172.21.15.33:0/4238502718 shutdown_connections 2024-05-03T18:44:07.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:07.117+0000 7f4915e00700 1 -- 172.21.15.33:0/4238502718 wait complete. 2024-05-03T18:44:07.126 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 21 2024-05-03T18:44:07.835 INFO:journalctl@ceph.osd.3.smithi033.stdout:May 03 18:44:07 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-3[49778]: 2024-05-03T18:44:07.573+0000 7f3221200700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 10 after trimming was started 2024-05-03T18:44:08.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:44:07 smithi150 ceph-mon[32492]: pgmap v532: 129 pgs: 129 active+clean; 3.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 8.0 MiB/s wr, 557 op/s 2024-05-03T18:44:08.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:44:07 smithi150 ceph-mon[32492]: osdmap e113: 12 total, 12 up, 12 in 2024-05-03T18:44:08.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:44:07 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/4238502718' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:44:08.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:44:07 smithi033 ceph-mon[28330]: pgmap v532: 129 pgs: 129 active+clean; 3.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 8.0 MiB/s wr, 557 op/s 2024-05-03T18:44:08.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:44:07 smithi033 ceph-mon[28330]: osdmap e113: 12 total, 12 up, 12 in 2024-05-03T18:44:08.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:44:07 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/4238502718' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:44:08.298 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:44:07 smithi099 ceph-mon[31658]: pgmap v532: 129 pgs: 129 active+clean; 3.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 8.0 MiB/s wr, 557 op/s 2024-05-03T18:44:08.298 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:44:07 smithi099 ceph-mon[31658]: osdmap e113: 12 total, 12 up, 12 in 2024-05-03T18:44:08.298 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:44:07 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/4238502718' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:44:09.408 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-03T18:44:10.111 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 775.0 s, 402.2 tps, lat 250.639 ms stddev 224.860 2024-05-03T18:44:10.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:44:09 smithi150 ceph-mon[32492]: pgmap v534: 129 pgs: 129 active+clean; 3.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 9.0 MiB/s wr, 632 op/s 2024-05-03T18:44:10.207 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:44:10.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:44:09 smithi033 ceph-mon[28330]: pgmap v534: 129 pgs: 129 active+clean; 3.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 9.0 MiB/s wr, 632 op/s 2024-05-03T18:44:10.297 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:44:09 smithi099 ceph-mon[31658]: pgmap v534: 129 pgs: 129 active+clean; 3.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 9.0 MiB/s wr, 632 op/s 2024-05-03T18:44:12.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:44:11 smithi150 ceph-mon[32492]: pgmap v535: 129 pgs: 129 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 9.1 MiB/s wr, 609 op/s 2024-05-03T18:44:12.236 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:44:11 smithi033 ceph-mon[28330]: pgmap v535: 129 pgs: 129 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 9.1 MiB/s wr, 609 op/s 2024-05-03T18:44:12.297 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:44:11 smithi099 ceph-mon[31658]: pgmap v535: 129 pgs: 129 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 9.1 MiB/s wr, 609 op/s 2024-05-03T18:44:13.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:44:13 smithi150 ceph-mon[32492]: pgmap v536: 129 pgs: 129 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 9.1 MiB/s wr, 609 op/s 2024-05-03T18:44:13.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:44:13 smithi150 ceph-mon[32492]: osdmap e114: 12 total, 12 up, 12 in 2024-05-03T18:44:13.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:44:13 smithi033 ceph-mon[28330]: pgmap v536: 129 pgs: 129 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 9.1 MiB/s wr, 609 op/s 2024-05-03T18:44:13.986 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:44:13 smithi033 ceph-mon[28330]: osdmap e114: 12 total, 12 up, 12 in 2024-05-03T18:44:14.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:44:13 smithi099 ceph-mon[31658]: pgmap v536: 129 pgs: 129 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 9.1 MiB/s wr, 609 op/s 2024-05-03T18:44:14.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:44:13 smithi099 ceph-mon[31658]: osdmap e114: 12 total, 12 up, 12 in 2024-05-03T18:44:15.111 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 780.0 s, 526.8 tps, lat 190.572 ms stddev 76.699 2024-05-03T18:44:15.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:44:15 smithi150 ceph-mon[32492]: pgmap v538: 129 pgs: 129 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 7.1 MiB/s wr, 499 op/s 2024-05-03T18:44:15.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:44:15 smithi033 ceph-mon[28330]: pgmap v538: 129 pgs: 129 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 7.1 MiB/s wr, 499 op/s 2024-05-03T18:44:16.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.038+0000 7f3a8f400700 1 -- 172.21.15.33:0/667074245 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a7c00b810 con 0x7f3a9010a9e0 2024-05-03T18:44:16.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.038+0000 7f3a97800700 1 -- 172.21.15.33:0/667074245 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3a9010a9e0 msgr2=0x7f3a9010ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:16.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.038+0000 7f3a97800700 1 --2- 172.21.15.33:0/667074245 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3a9010a9e0 0x7f3a9010ae40 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c0098f0 tx=0x7f3a7c009c00 comp rx=0 tx=0).stop 2024-05-03T18:44:16.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.039+0000 7f3a97800700 1 -- 172.21.15.33:0/667074245 shutdown_connections 2024-05-03T18:44:16.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.039+0000 7f3a97800700 1 --2- 172.21.15.33:0/667074245 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a90118530 0x7f3a9011a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:16.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.039+0000 7f3a97800700 1 --2- 172.21.15.33:0/667074245 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a9010dbd0 0x7f3a9010a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:16.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.039+0000 7f3a97800700 1 --2- 172.21.15.33:0/667074245 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3a9010a9e0 0x7f3a9010ae40 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:16.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.039+0000 7f3a97800700 1 -- 172.21.15.33:0/667074245 >> 172.21.15.33:0/667074245 conn(0x7f3a9006e9f0 msgr2=0x7f3a9006ee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:44:16.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.039+0000 7f3a97800700 1 -- 172.21.15.33:0/667074245 shutdown_connections 2024-05-03T18:44:16.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.039+0000 7f3a97800700 1 -- 172.21.15.33:0/667074245 wait complete. 2024-05-03T18:44:16.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.041+0000 7f3a97800700 1 Processor -- start 2024-05-03T18:44:16.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.043+0000 7f3a97800700 1 -- start start 2024-05-03T18:44:16.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.043+0000 7f3a97800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3a9010a9e0 0x7f3a901aa570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:16.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.043+0000 7f3a97800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a9010dbd0 0x7f3a901aaab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:16.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.043+0000 7f3a97800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a90118530 0x7f3a901174e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:16.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.043+0000 7f3a97800700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3a9011dfb0 con 0x7f3a9010dbd0 2024-05-03T18:44:16.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.043+0000 7f3a97800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3a9011de30 con 0x7f3a90118530 2024-05-03T18:44:16.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.043+0000 7f3a97800700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3a9011e130 con 0x7f3a9010a9e0 2024-05-03T18:44:16.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.044+0000 7f3a95800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a90118530 0x7f3a901174e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:16.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.044+0000 7f3a95800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a90118530 0x7f3a901174e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:53544/0 (socket says 172.21.15.33:53544) 2024-05-03T18:44:16.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.044+0000 7f3a95800700 1 -- 172.21.15.33:0/989678827 learned_addr learned my addr 172.21.15.33:0/989678827 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:44:16.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.044+0000 7f3a8fe00700 1 --2- 172.21.15.33:0/989678827 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a9010dbd0 0x7f3a901aaab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:16.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.044+0000 7f3a94e00700 1 --2- 172.21.15.33:0/989678827 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3a9010a9e0 0x7f3a901aa570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:16.047 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:44:15 smithi099 ceph-mon[31658]: pgmap v538: 129 pgs: 129 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 7.1 MiB/s wr, 499 op/s 2024-05-03T18:44:16.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.045+0000 7f3a94e00700 1 -- 172.21.15.33:0/989678827 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a90118530 msgr2=0x7f3a901174e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:16.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.045+0000 7f3a94e00700 1 --2- 172.21.15.33:0/989678827 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a90118530 0x7f3a901174e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:16.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.045+0000 7f3a94e00700 1 -- 172.21.15.33:0/989678827 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a9010dbd0 msgr2=0x7f3a901aaab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:16.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.045+0000 7f3a94e00700 1 --2- 172.21.15.33:0/989678827 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a9010dbd0 0x7f3a901aaab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:16.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.045+0000 7f3a94e00700 1 -- 172.21.15.33:0/989678827 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a90117bb0 con 0x7f3a9010a9e0 2024-05-03T18:44:16.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.045+0000 7f3a94e00700 1 --2- 172.21.15.33:0/989678827 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3a9010a9e0 0x7f3a901aa570 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f3a8400c970 tx=0x7f3a8400cc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:44:16.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.045+0000 7f3a8d600700 1 -- 172.21.15.33:0/989678827 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a84014070 con 0x7f3a9010a9e0 2024-05-03T18:44:16.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.046+0000 7f3a97800700 1 -- 172.21.15.33:0/989678827 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a90117ea0 con 0x7f3a9010a9e0 2024-05-03T18:44:16.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.046+0000 7f3a97800700 1 -- 172.21.15.33:0/989678827 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a90111200 con 0x7f3a9010a9e0 2024-05-03T18:44:16.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.046+0000 7f3a8d600700 1 -- 172.21.15.33:0/989678827 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3a84004440 con 0x7f3a9010a9e0 2024-05-03T18:44:16.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.046+0000 7f3a8d600700 1 -- 172.21.15.33:0/989678827 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a84003ae0 con 0x7f3a9010a9e0 2024-05-03T18:44:16.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.053+0000 7f3a8d600700 1 -- 172.21.15.33:0/989678827 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f3a84017070 con 0x7f3a9010a9e0 2024-05-03T18:44:16.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.057+0000 7f3a8d600700 1 --2- 172.21.15.33:0/989678827 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f3a780b2bf0 0x7f3a780b50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:16.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.057+0000 7f3a8d600700 1 -- 172.21.15.33:0/989678827 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 9518+0+0 (secure 0 0 0) 0x7f3a84003880 con 0x7f3a9010a9e0 2024-05-03T18:44:16.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.058+0000 7f3a8fe00700 1 --2- 172.21.15.33:0/989678827 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f3a780b2bf0 0x7f3a780b50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:16.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.061+0000 7f3a8fe00700 1 --2- 172.21.15.33:0/989678827 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f3a780b2bf0 0x7f3a780b50b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c00b2e0 tx=0x7f3a7c005b80 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:44:16.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.110+0000 7f3aa21d5b80 1 Processor -- start 2024-05-03T18:44:16.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.113+0000 7f3aa21d5b80 1 -- start start 2024-05-03T18:44:16.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.116+0000 7f3aa21d5b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558b0f2f5930 0x558b0f2f5d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:16.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.117+0000 7f3aa21d5b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b0f2f6250 0x558b0f2eff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:16.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.117+0000 7f3aa21d5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558b0f2f0550 0x558b0f2f09d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:16.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.117+0000 7f3aa21d5b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x558b0f242a20 con 0x558b0f2f6250 2024-05-03T18:44:16.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.117+0000 7f3aa21d5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x558b0f1e0760 con 0x558b0f2f0550 2024-05-03T18:44:16.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.117+0000 7f3aa21d5b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x558b0f23e9e0 con 0x558b0f2f5930 2024-05-03T18:44:16.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.117+0000 7f3a94e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558b0f2f5930 0x558b0f2f5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:16.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.117+0000 7f3a94e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558b0f2f5930 0x558b0f2f5d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:41820/0 (socket says 172.21.15.33:41820) 2024-05-03T18:44:16.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.117+0000 7f3a94e00700 1 -- 172.21.15.33:0/2879166163 learned_addr learned my addr 172.21.15.33:0/2879166163 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:44:16.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.117+0000 7f3a8fe00700 1 --2- 172.21.15.33:0/2879166163 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b0f2f6250 0x558b0f2eff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:16.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.118+0000 7f3a94e00700 1 -- 172.21.15.33:0/2879166163 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558b0f2f0550 msgr2=0x558b0f2f09d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-03T18:44:16.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.118+0000 7f3a94e00700 1 --2- 172.21.15.33:0/2879166163 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558b0f2f0550 0x558b0f2f09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:16.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.118+0000 7f3a94e00700 1 -- 172.21.15.33:0/2879166163 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b0f2f6250 msgr2=0x558b0f2eff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:44:16.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.118+0000 7f3a94e00700 1 --2- 172.21.15.33:0/2879166163 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b0f2f6250 0x558b0f2eff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:16.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.118+0000 7f3a94e00700 1 -- 172.21.15.33:0/2879166163 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b0f18f8d0 con 0x558b0f2f5930 2024-05-03T18:44:16.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.118+0000 7f3a94e00700 1 --2- 172.21.15.33:0/2879166163 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558b0f2f5930 0x558b0f2f5d10 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f3a840a8f60 tx=0x7f3a840a8bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d302770a41a43178 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:44:16.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.119+0000 7f3a82000700 1 -- 172.21.15.33:0/2879166163 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a840b1e30 con 0x558b0f2f5930 2024-05-03T18:44:16.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.119+0000 7f3a82000700 1 -- 172.21.15.33:0/2879166163 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3a840a9230 con 0x558b0f2f5930 2024-05-03T18:44:16.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.119+0000 7f3a82000700 1 -- 172.21.15.33:0/2879166163 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a840dfdf0 con 0x558b0f2f5930 2024-05-03T18:44:16.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.120+0000 7f3aa21d5b80 1 -- 172.21.15.33:0/2879166163 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558b0f2f5930 msgr2=0x558b0f2f5d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:16.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.120+0000 7f3aa21d5b80 1 --2- 172.21.15.33:0/2879166163 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558b0f2f5930 0x558b0f2f5d10 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f3a840a8f60 tx=0x7f3a840a8bb0 comp rx=0 tx=0).stop 2024-05-03T18:44:16.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.121+0000 7f3aa21d5b80 1 -- 172.21.15.33:0/2879166163 shutdown_connections 2024-05-03T18:44:16.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.121+0000 7f3aa21d5b80 1 --2- 172.21.15.33:0/2879166163 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558b0f2f0550 0x558b0f2f09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:16.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.121+0000 7f3aa21d5b80 1 --2- 172.21.15.33:0/2879166163 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558b0f2f5930 0x558b0f2f5d10 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:16.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.121+0000 7f3aa21d5b80 1 --2- 172.21.15.33:0/2879166163 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b0f2f6250 0x558b0f2eff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:16.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.121+0000 7f3aa21d5b80 1 -- 172.21.15.33:0/2879166163 >> 172.21.15.33:0/2879166163 conn(0x558b0f2686d0 msgr2=0x558b0f2ea900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:44:16.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.123+0000 7f3aa21d5b80 1 -- 172.21.15.33:0/2879166163 shutdown_connections 2024-05-03T18:44:16.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.123+0000 7f3aa21d5b80 1 -- 172.21.15.33:0/2879166163 wait complete. 2024-05-03T18:44:16.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.124+0000 7f3aa21d5b80 1 -- start start 2024-05-03T18:44:16.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.125+0000 7f3aa21d5b80 1 Processor -- start 2024-05-03T18:44:16.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.126+0000 7f3aa21d5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558b0f2f0550 0x558b0f2fbe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:16.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.126+0000 7f3aa21d5b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b0f2f5930 0x558b0f2f9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:16.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.126+0000 7f3aa21d5b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558b0f2f6250 0x558b0f2f97c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:16.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.126+0000 7f3aa21d5b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x558b0f2fcd00 con 0x558b0f2f5930 2024-05-03T18:44:16.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.126+0000 7f3aa21d5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x558b0f1a8b20 con 0x558b0f2f0550 2024-05-03T18:44:16.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.126+0000 7f3aa21d5b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x558b0f2fce80 con 0x558b0f2f6250 2024-05-03T18:44:16.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.127+0000 7f3aa21d5b80 10 client.0 _ll_register_callbacks cb 0x558b0f1e9c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:44:16.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.128+0000 7f3a95800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558b0f2f6250 0x558b0f2f97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:16.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.128+0000 7f3a94e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558b0f2f0550 0x558b0f2fbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:16.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.128+0000 7f3a95800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558b0f2f6250 0x558b0f2f97c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:41822/0 (socket says 172.21.15.33:41822) 2024-05-03T18:44:16.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.128+0000 7f3a95800700 1 -- 172.21.15.33:0/1068254968 learned_addr learned my addr 172.21.15.33:0/1068254968 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:44:16.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.128+0000 7f3a8fe00700 1 --2- 172.21.15.33:0/1068254968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b0f2f5930 0x558b0f2f9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:16.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.128+0000 7f3a94e00700 1 -- 172.21.15.33:0/1068254968 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558b0f2f6250 msgr2=0x558b0f2f97c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:44:16.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.128+0000 7f3a94e00700 1 --2- 172.21.15.33:0/1068254968 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558b0f2f6250 0x558b0f2f97c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:16.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.128+0000 7f3a94e00700 1 -- 172.21.15.33:0/1068254968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b0f2f5930 msgr2=0x558b0f2f9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:44:16.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.128+0000 7f3a94e00700 1 --2- 172.21.15.33:0/1068254968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b0f2f5930 0x558b0f2f9280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:16.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.128+0000 7f3a94e00700 1 -- 172.21.15.33:0/1068254968 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b0f208420 con 0x558b0f2f0550 2024-05-03T18:44:16.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.129+0000 7f3a94e00700 1 --2- 172.21.15.33:0/1068254968 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558b0f2f0550 0x558b0f2fbe70 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x558b0f2fca30 tx=0x7f3a840a5470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1de10dee4d2f72be server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:44:16.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.129+0000 7f3a76a00700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-03T18:44:16.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.130+0000 7f3a76a00700 1 -- 172.21.15.33:0/1068254968 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a840dfda0 con 0x558b0f2f0550 2024-05-03T18:44:16.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.130+0000 7f3a76a00700 1 -- 172.21.15.33:0/1068254968 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9518+0+0 (secure 0 0 0) 0x7f3a840a8230 con 0x558b0f2f0550 2024-05-03T18:44:16.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.130+0000 7f3aa21d5b80 1 -- 172.21.15.33:0/1068254968 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a840a9230 con 0x558b0f2f0550 2024-05-03T18:44:16.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.130+0000 7f3a76a00700 1 -- 172.21.15.33:0/1068254968 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f3a840a8430 con 0x558b0f2f0550 2024-05-03T18:44:16.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.131+0000 7f3aa21d5b80 10 client.36629 fetch_fsmap learned FSMap version 21 2024-05-03T18:44:16.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.131+0000 7f3aa21d5b80 1 -- 172.21.15.33:0/1068254968 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x558b0f307520 con 0x558b0f2f0550 2024-05-03T18:44:16.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.131+0000 7f3a76a00700 1 -- 172.21.15.33:0/1068254968 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3a840a9230 con 0x558b0f2f0550 2024-05-03T18:44:16.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.132+0000 7f3a76a00700 1 -- 172.21.15.33:0/1068254968 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f3a840a7ca0 con 0x558b0f2f0550 2024-05-03T18:44:16.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.132+0000 7f3aa21d5b80 10 client.36629 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:44:16.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.132+0000 7f3aa21d5b80 10 client.36629 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:44:16.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.132+0000 7f3aa21d5b80 20 client.36629 populate_metadata read hostname 'smithi033' 2024-05-03T18:44:16.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.132+0000 7f3aa21d5b80 1 --2- 172.21.15.33:0/1068254968 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x558b0f309bb0 0x558b0f30bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:16.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.132+0000 7f3aa21d5b80 4 client.36629 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:44:16.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.132+0000 7f3aa21d5b80 1 -- 172.21.15.33:0/1068254968 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558b0f2bcff0 con 0x558b0f309bb0 2024-05-03T18:44:16.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.133+0000 7f3a8fe00700 1 --2- 172.21.15.33:0/1068254968 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x558b0f309bb0 0x558b0f30bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:16.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.134+0000 7f3a76a00700 10 client.36629 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:44:16.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.134+0000 7f3a8fe00700 1 --2- 172.21.15.33:0/1068254968 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x558b0f309bb0 0x558b0f30bff0 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4a5e40cbba9ab50 server_cookie=b4f57760eb0debd9 in_seq=0 out_seq=0 2024-05-03T18:44:16.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.136+0000 7f3a76a00700 1 -- 172.21.15.33:0/1068254968 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558b0f2bcff0 con 0x558b0f309bb0 2024-05-03T18:44:16.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.136+0000 7f3a76a00700 10 client.36629 handle_command_reply: tid=0 2024-05-03T18:44:16.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.136+0000 7f3aa21d5b80 1 client.36629 shutdown 2024-05-03T18:44:16.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.137+0000 7f3aa21d5b80 1 -- 172.21.15.33:0/1068254968 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558b0f2f0550 msgr2=0x558b0f2fbe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:16.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.137+0000 7f3aa21d5b80 1 --2- 172.21.15.33:0/1068254968 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558b0f2f0550 0x558b0f2fbe70 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x558b0f2fca30 tx=0x7f3a840a5470 comp rx=0 tx=0).stop 2024-05-03T18:44:16.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.137+0000 7f3aa21d5b80 1 -- 172.21.15.33:0/1068254968 shutdown_connections 2024-05-03T18:44:16.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.137+0000 7f3aa21d5b80 1 --2- 172.21.15.33:0/1068254968 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x558b0f309bb0 0x558b0f30bff0 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:16.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.137+0000 7f3aa21d5b80 1 --2- 172.21.15.33:0/1068254968 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558b0f2f6250 0x558b0f2f97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:16.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.137+0000 7f3a76a00700 0 client.36629 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:44:16.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.137+0000 7f3aa21d5b80 1 --2- 172.21.15.33:0/1068254968 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558b0f2f0550 0x558b0f2fbe70 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:16.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.137+0000 7f3aa21d5b80 1 --2- 172.21.15.33:0/1068254968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b0f2f5930 0x558b0f2f9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:16.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.137+0000 7f3aa21d5b80 1 -- 172.21.15.33:0/1068254968 >> 172.21.15.33:0/1068254968 conn(0x558b0f2686d0 msgr2=0x558b0f2e53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:44:16.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.138+0000 7f3aa21d5b80 1 -- 172.21.15.33:0/1068254968 shutdown_connections 2024-05-03T18:44:16.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.139+0000 7f3aa21d5b80 1 -- 172.21.15.33:0/1068254968 wait complete. 2024-05-03T18:44:16.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.139+0000 7f3aa21d5b80 20 client.36629 trim_cache size 0 max 16384 2024-05-03T18:44:16.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.169+0000 7f3aa21d5b80 1 Processor -- start 2024-05-03T18:44:16.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.170+0000 7f3aa21d5b80 1 -- start start 2024-05-03T18:44:16.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.170+0000 7f3aa21d5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558b0f2f0550 0x558b0f305df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:16.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.170+0000 7f3aa21d5b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558b0f2f5930 0x558b0f2f97f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:16.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.170+0000 7f3aa21d5b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b0f2f6250 0x558b0f311900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:16.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.170+0000 7f3aa21d5b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3a7000cee0 con 0x558b0f2f6250 2024-05-03T18:44:16.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.170+0000 7f3aa21d5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3a7000f740 con 0x558b0f2f0550 2024-05-03T18:44:16.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.170+0000 7f3aa21d5b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3a7000d530 con 0x558b0f2f5930 2024-05-03T18:44:16.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.171+0000 7f3a8fe00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558b0f2f5930 0x558b0f2f97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:16.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.171+0000 7f3a94e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558b0f2f0550 0x558b0f305df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:16.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.171+0000 7f3a94e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558b0f2f0550 0x558b0f305df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:53582/0 (socket says 172.21.15.33:53582) 2024-05-03T18:44:16.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.171+0000 7f3a94e00700 1 -- 172.21.15.33:0/3658475485 learned_addr learned my addr 172.21.15.33:0/3658475485 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:44:16.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.172+0000 7f3a95800700 1 --2- 172.21.15.33:0/3658475485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b0f2f6250 0x558b0f311900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:16.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.172+0000 7f3a94e00700 1 -- 172.21.15.33:0/3658475485 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558b0f2f5930 msgr2=0x558b0f2f97f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:44:16.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.172+0000 7f3a94e00700 1 --2- 172.21.15.33:0/3658475485 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558b0f2f5930 0x558b0f2f97f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:16.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.172+0000 7f3a94e00700 1 -- 172.21.15.33:0/3658475485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b0f2f6250 msgr2=0x558b0f311900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:44:16.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.172+0000 7f3a94e00700 1 --2- 172.21.15.33:0/3658475485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b0f2f6250 0x558b0f311900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:16.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.173+0000 7f3a94e00700 1 -- 172.21.15.33:0/3658475485 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b0f2f77f0 con 0x558b0f2f0550 2024-05-03T18:44:16.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.174+0000 7f3a94e00700 1 --2- 172.21.15.33:0/3658475485 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558b0f2f0550 0x558b0f305df0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3a840a8f60 tx=0x7f3a840a8ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3dce66fe7c96698 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:44:16.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.174+0000 7f3a81600700 1 -- 172.21.15.33:0/3658475485 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a840a4e90 con 0x558b0f2f0550 2024-05-03T18:44:16.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.175+0000 7f3a81600700 1 -- 172.21.15.33:0/3658475485 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3a840a9230 con 0x558b0f2f0550 2024-05-03T18:44:16.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.175+0000 7f3a81600700 1 -- 172.21.15.33:0/3658475485 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a840a8920 con 0x558b0f2f0550 2024-05-03T18:44:16.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.178+0000 7f3aa21d5b80 1 -- 172.21.15.33:0/3658475485 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558b0f2f0550 msgr2=0x558b0f305df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:16.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.178+0000 7f3aa21d5b80 1 --2- 172.21.15.33:0/3658475485 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558b0f2f0550 0x558b0f305df0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3a840a8f60 tx=0x7f3a840a8ed0 comp rx=0 tx=0).stop 2024-05-03T18:44:16.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.179+0000 7f3aa21d5b80 1 -- 172.21.15.33:0/3658475485 shutdown_connections 2024-05-03T18:44:16.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.179+0000 7f3aa21d5b80 1 --2- 172.21.15.33:0/3658475485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b0f2f6250 0x558b0f311900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:16.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.179+0000 7f3aa21d5b80 1 --2- 172.21.15.33:0/3658475485 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558b0f2f0550 0x558b0f305df0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:16.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.179+0000 7f3aa21d5b80 1 --2- 172.21.15.33:0/3658475485 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558b0f2f5930 0x558b0f2f97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:16.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.179+0000 7f3aa21d5b80 1 -- 172.21.15.33:0/3658475485 >> 172.21.15.33:0/3658475485 conn(0x558b0f2686d0 msgr2=0x558b0f2f7b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:44:16.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.179+0000 7f3aa21d5b80 1 -- 172.21.15.33:0/3658475485 shutdown_connections 2024-05-03T18:44:16.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.179+0000 7f3aa21d5b80 1 -- 172.21.15.33:0/3658475485 wait complete. 2024-05-03T18:44:16.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.180+0000 7f3aa21d5b80 1 -- start start 2024-05-03T18:44:16.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.180+0000 7f3aa21d5b80 1 Processor -- start 2024-05-03T18:44:16.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.181+0000 7f3aa21d5b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b0f2f0550 0x558b0f302170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:16.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.181+0000 7f3aa21d5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558b0f2f5930 0x558b0f3026b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:16.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.182+0000 7f3aa21d5b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558b0f2f6250 0x558b0f3108f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:16.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.182+0000 7f3aa21d5b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3a7000dc00 con 0x558b0f2f0550 2024-05-03T18:44:16.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.182+0000 7f3aa21d5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3a7000f070 con 0x558b0f2f5930 2024-05-03T18:44:16.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.182+0000 7f3aa21d5b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3a7000e2d0 con 0x558b0f2f6250 2024-05-03T18:44:16.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.182+0000 7f3aa21d5b80 10 client.0 _ll_register_callbacks cb 0x558b0f1e9c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:44:16.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.183+0000 7f3a95800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558b0f2f6250 0x558b0f3108f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:16.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.183+0000 7f3a95800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558b0f2f6250 0x558b0f3108f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:41838/0 (socket says 172.21.15.33:41838) 2024-05-03T18:44:16.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.183+0000 7f3a95800700 1 -- 172.21.15.33:0/3261580101 learned_addr learned my addr 172.21.15.33:0/3261580101 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:44:16.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.183+0000 7f3a94e00700 1 --2- 172.21.15.33:0/3261580101 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b0f2f0550 0x558b0f302170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:16.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.183+0000 7f3a8fe00700 1 --2- 172.21.15.33:0/3261580101 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558b0f2f5930 0x558b0f3026b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:16.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.184+0000 7f3a94e00700 1 -- 172.21.15.33:0/3261580101 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558b0f2f6250 msgr2=0x558b0f3108f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:44:16.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.184+0000 7f3a94e00700 1 --2- 172.21.15.33:0/3261580101 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558b0f2f6250 0x558b0f3108f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:16.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.184+0000 7f3a94e00700 1 -- 172.21.15.33:0/3261580101 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558b0f2f5930 msgr2=0x558b0f3026b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:44:16.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.184+0000 7f3a94e00700 1 --2- 172.21.15.33:0/3261580101 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558b0f2f5930 0x558b0f3026b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:16.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.184+0000 7f3a94e00700 1 -- 172.21.15.33:0/3261580101 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b0f310eb0 con 0x558b0f2f0550 2024-05-03T18:44:16.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.184+0000 7f3a8fe00700 1 --2- 172.21.15.33:0/3261580101 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558b0f2f5930 0x558b0f3026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:44:16.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.185+0000 7f3a94e00700 1 --2- 172.21.15.33:0/3261580101 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b0f2f0550 0x558b0f302170 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f3a84004610 tx=0x7f3a8400af70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57633a49098624e server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:44:16.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.186+0000 7f3a76a00700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:44:16.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.186+0000 7f3a76a00700 1 -- 172.21.15.33:0/3261580101 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a840a4e90 con 0x558b0f2f0550 2024-05-03T18:44:16.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.186+0000 7f3a76a00700 1 -- 172.21.15.33:0/3261580101 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9518+0+0 (secure 0 0 0) 0x7f3a840a8230 con 0x558b0f2f0550 2024-05-03T18:44:16.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.186+0000 7f3aa21d5b80 1 -- 172.21.15.33:0/3261580101 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a840a9230 con 0x558b0f2f0550 2024-05-03T18:44:16.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.187+0000 7f3a76a00700 1 -- 172.21.15.33:0/3261580101 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f3a840b1e30 con 0x558b0f2f0550 2024-05-03T18:44:16.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.187+0000 7f3aa21d5b80 10 client.16845 fetch_fsmap learned FSMap version 21 2024-05-03T18:44:16.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.187+0000 7f3aa21d5b80 1 -- 172.21.15.33:0/3261580101 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x558b0f310eb0 con 0x558b0f2f0550 2024-05-03T18:44:16.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.188+0000 7f3a76a00700 1 -- 172.21.15.33:0/3261580101 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3a840a9230 con 0x558b0f2f0550 2024-05-03T18:44:16.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.188+0000 7f3a76a00700 1 -- 172.21.15.33:0/3261580101 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f3a840ae510 con 0x558b0f2f0550 2024-05-03T18:44:16.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.188+0000 7f3aa21d5b80 10 client.16845 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:44:16.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.188+0000 7f3aa21d5b80 10 client.16845 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:44:16.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.188+0000 7f3aa21d5b80 20 client.16845 populate_metadata read hostname 'smithi033' 2024-05-03T18:44:16.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.188+0000 7f3aa21d5b80 1 --2- 172.21.15.33:0/3261580101 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x558b0f373f30 0x558b0f376320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:16.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.188+0000 7f3aa21d5b80 4 client.16845 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-03T18:44:16.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.188+0000 7f3aa21d5b80 1 -- 172.21.15.33:0/3261580101 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558b0f250400 con 0x558b0f373f30 2024-05-03T18:44:16.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.190+0000 7f3a8fe00700 1 --2- 172.21.15.33:0/3261580101 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x558b0f373f30 0x558b0f376320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:16.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.191+0000 7f3a76a00700 10 client.16845 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:44:16.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.191+0000 7f3a8fe00700 1 --2- 172.21.15.33:0/3261580101 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x558b0f373f30 0x558b0f376320 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d348e66de1004f03 server_cookie=d238d57059a9ac25 in_seq=0 out_seq=0 2024-05-03T18:44:16.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.192+0000 7f3a76a00700 1 -- 172.21.15.33:0/3261580101 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558b0f250400 con 0x558b0f373f30 2024-05-03T18:44:16.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.193+0000 7f3a76a00700 10 client.16845 handle_command_reply: tid=0 2024-05-03T18:44:16.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.193+0000 7f3aa21d5b80 1 client.16845 shutdown 2024-05-03T18:44:16.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.193+0000 7f3aa21d5b80 1 -- 172.21.15.33:0/3261580101 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b0f2f0550 msgr2=0x558b0f302170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:16.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.193+0000 7f3aa21d5b80 1 --2- 172.21.15.33:0/3261580101 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b0f2f0550 0x558b0f302170 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f3a84004610 tx=0x7f3a8400af70 comp rx=0 tx=0).stop 2024-05-03T18:44:16.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.194+0000 7f3aa21d5b80 1 -- 172.21.15.33:0/3261580101 shutdown_connections 2024-05-03T18:44:16.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.194+0000 7f3aa21d5b80 1 --2- 172.21.15.33:0/3261580101 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x558b0f373f30 0x558b0f376320 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:16.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.194+0000 7f3aa21d5b80 1 --2- 172.21.15.33:0/3261580101 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558b0f2f6250 0x558b0f3108f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:16.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.194+0000 7f3aa21d5b80 1 --2- 172.21.15.33:0/3261580101 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b0f2f0550 0x558b0f302170 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:16.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.194+0000 7f3aa21d5b80 1 --2- 172.21.15.33:0/3261580101 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558b0f2f5930 0x558b0f3026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:16.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.194+0000 7f3aa21d5b80 1 -- 172.21.15.33:0/3261580101 >> 172.21.15.33:0/3261580101 conn(0x558b0f2686d0 msgr2=0x558b0f2f6630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:44:16.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.195+0000 7f3a76a00700 0 client.16845 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:44:16.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.195+0000 7f3aa21d5b80 1 -- 172.21.15.33:0/3261580101 shutdown_connections 2024-05-03T18:44:16.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.195+0000 7f3aa21d5b80 1 -- 172.21.15.33:0/3261580101 wait complete. 2024-05-03T18:44:16.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.195+0000 7f3aa21d5b80 20 client.16845 trim_cache size 0 max 16384 2024-05-03T18:44:16.197 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-03T18:44:16.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.197+0000 7f3a97800700 1 -- 172.21.15.33:0/989678827 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f3a780b2bf0 msgr2=0x7f3a780b50b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:16.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.197+0000 7f3a97800700 1 --2- 172.21.15.33:0/989678827 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f3a780b2bf0 0x7f3a780b50b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c00b2e0 tx=0x7f3a7c005b80 comp rx=0 tx=0).stop 2024-05-03T18:44:16.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.197+0000 7f3a97800700 1 -- 172.21.15.33:0/989678827 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3a9010a9e0 msgr2=0x7f3a901aa570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:16.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.197+0000 7f3a97800700 1 --2- 172.21.15.33:0/989678827 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3a9010a9e0 0x7f3a901aa570 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f3a8400c970 tx=0x7f3a8400cc80 comp rx=0 tx=0).stop 2024-05-03T18:44:16.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.198+0000 7f3a97800700 1 -- 172.21.15.33:0/989678827 shutdown_connections 2024-05-03T18:44:16.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.198+0000 7f3a97800700 1 --2- 172.21.15.33:0/989678827 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f3a780b2bf0 0x7f3a780b50b0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:16.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.198+0000 7f3a97800700 1 --2- 172.21.15.33:0/989678827 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a90118530 0x7f3a901174e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:16.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.198+0000 7f3a97800700 1 --2- 172.21.15.33:0/989678827 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3a9010a9e0 0x7f3a901aa570 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:16.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.198+0000 7f3a97800700 1 --2- 172.21.15.33:0/989678827 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a9010dbd0 0x7f3a901aaab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:16.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.198+0000 7f3a97800700 1 -- 172.21.15.33:0/989678827 >> 172.21.15.33:0/989678827 conn(0x7f3a9006e9f0 msgr2=0x7f3a90119c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:44:16.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.198+0000 7f3a97800700 1 -- 172.21.15.33:0/989678827 shutdown_connections 2024-05-03T18:44:16.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:16.198+0000 7f3a97800700 1 -- 172.21.15.33:0/989678827 wait complete. 2024-05-03T18:44:17.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:44:17 smithi150 ceph-mon[32492]: pgmap v539: 129 pgs: 129 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 8.7 MiB/s wr, 571 op/s 2024-05-03T18:44:17.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:44:17 smithi033 ceph-mon[28330]: pgmap v539: 129 pgs: 129 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 8.7 MiB/s wr, 571 op/s 2024-05-03T18:44:18.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:44:17 smithi099 ceph-mon[31658]: pgmap v539: 129 pgs: 129 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 8.7 MiB/s wr, 571 op/s 2024-05-03T18:44:19.534 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-03T18:44:19.534 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-03T18:44:19.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:44:19 smithi150 ceph-mon[32492]: pgmap v540: 129 pgs: 129 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 8.2 MiB/s wr, 539 op/s 2024-05-03T18:44:19.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:44:19 smithi033 ceph-mon[28330]: pgmap v540: 129 pgs: 129 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 8.2 MiB/s wr, 539 op/s 2024-05-03T18:44:20.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:44:19 smithi099 ceph-mon[31658]: pgmap v540: 129 pgs: 129 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 8.2 MiB/s wr, 539 op/s 2024-05-03T18:44:20.111 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 785.0 s, 551.0 tps, lat 180.427 ms stddev 73.328 2024-05-03T18:44:20.374 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:44:21.867 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:44:21 smithi033 ceph-mon[28330]: pgmap v541: 129 pgs: 129 active+clean; 3.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 8.4 MiB/s wr, 601 op/s 2024-05-03T18:44:21.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:44:21 smithi150 ceph-mon[32492]: pgmap v541: 129 pgs: 129 active+clean; 3.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 8.4 MiB/s wr, 601 op/s 2024-05-03T18:44:22.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:44:21 smithi099 ceph-mon[31658]: pgmap v541: 129 pgs: 129 active+clean; 3.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 8.4 MiB/s wr, 601 op/s 2024-05-03T18:44:23.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:44:23 smithi150 ceph-mon[32492]: pgmap v542: 129 pgs: 129 active+clean; 3.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 8.4 MiB/s wr, 601 op/s 2024-05-03T18:44:23.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:44:23 smithi033 ceph-mon[28330]: pgmap v542: 129 pgs: 129 active+clean; 3.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 8.4 MiB/s wr, 601 op/s 2024-05-03T18:44:24.047 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:44:23 smithi099 ceph-mon[31658]: pgmap v542: 129 pgs: 129 active+clean; 3.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 8.4 MiB/s wr, 601 op/s 2024-05-03T18:44:25.111 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 790.0 s, 513.6 tps, lat 188.552 ms stddev 110.000 2024-05-03T18:44:25.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.482+0000 7f9076e00700 1 -- 172.21.15.33:0/3888665676 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f907010fea0 msgr2=0x7f9070112290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:25.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.482+0000 7f9076e00700 1 --2- 172.21.15.33:0/3888665676 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f907010fea0 0x7f9070112290 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f9068009990 tx=0x7f9068009ca0 comp rx=0 tx=0).stop 2024-05-03T18:44:25.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.483+0000 7f9076e00700 1 -- 172.21.15.33:0/3888665676 shutdown_connections 2024-05-03T18:44:25.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.483+0000 7f9076e00700 1 --2- 172.21.15.33:0/3888665676 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f907010fea0 0x7f9070112290 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:25.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.483+0000 7f9076e00700 1 --2- 172.21.15.33:0/3888665676 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90700756d0 0x7f9070075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:25.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.483+0000 7f9076e00700 1 --2- 172.21.15.33:0/3888665676 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9070075ff0 0x7f907010f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:25.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.483+0000 7f9076e00700 1 -- 172.21.15.33:0/3888665676 >> 172.21.15.33:0/3888665676 conn(0x7f907007a600 msgr2=0x7f907007aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:44:25.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.486+0000 7f9076e00700 1 -- 172.21.15.33:0/3888665676 shutdown_connections 2024-05-03T18:44:25.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.486+0000 7f9076e00700 1 -- 172.21.15.33:0/3888665676 wait complete. 2024-05-03T18:44:25.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.488+0000 7f9076e00700 1 Processor -- start 2024-05-03T18:44:25.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.489+0000 7f9076e00700 1 -- start start 2024-05-03T18:44:25.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.489+0000 7f9076e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f90700756d0 0x7f907010e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:25.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.489+0000 7f9076e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9070075ff0 0x7f907010e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:25.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.489+0000 7f9076e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f907010fea0 0x7f90701abcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:25.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.489+0000 7f9076e00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9070104540 con 0x7f907010fea0 2024-05-03T18:44:25.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.489+0000 7f9076e00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f90701043c0 con 0x7f90700756d0 2024-05-03T18:44:25.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.489+0000 7f9076e00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f90701046c0 con 0x7f9070075ff0 2024-05-03T18:44:25.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.490+0000 7f9067400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9070075ff0 0x7f907010e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:25.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.490+0000 7f9067400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9070075ff0 0x7f907010e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:41860/0 (socket says 172.21.15.33:41860) 2024-05-03T18:44:25.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.490+0000 7f906fe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f90700756d0 0x7f907010e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:25.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.490+0000 7f9067400700 1 -- 172.21.15.33:0/202664774 learned_addr learned my addr 172.21.15.33:0/202664774 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:44:25.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.491+0000 7f906fe00700 1 -- 172.21.15.33:0/202664774 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9070075ff0 msgr2=0x7f907010e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:25.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.491+0000 7f906fe00700 1 --2- 172.21.15.33:0/202664774 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9070075ff0 0x7f907010e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:25.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.491+0000 7f906fe00700 1 -- 172.21.15.33:0/202664774 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f907010fea0 msgr2=0x7f90701abcc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-03T18:44:25.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.491+0000 7f906fe00700 1 --2- 172.21.15.33:0/202664774 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f907010fea0 0x7f90701abcc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:25.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.491+0000 7f906fe00700 1 -- 172.21.15.33:0/202664774 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90701ac3c0 con 0x7f90700756d0 2024-05-03T18:44:25.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.493+0000 7f906fe00700 1 --2- 172.21.15.33:0/202664774 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f90700756d0 0x7f907010e330 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f906000ada0 tx=0x7f906000c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:44:25.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.494+0000 7f906d600700 1 -- 172.21.15.33:0/202664774 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9060017020 con 0x7f90700756d0 2024-05-03T18:44:25.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.494+0000 7f906d600700 1 -- 172.21.15.33:0/202664774 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f906000ce10 con 0x7f90700756d0 2024-05-03T18:44:25.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.494+0000 7f906d600700 1 -- 172.21.15.33:0/202664774 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9060005280 con 0x7f90700756d0 2024-05-03T18:44:25.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.495+0000 7f9076e00700 1 -- 172.21.15.33:0/202664774 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90701ac6b0 con 0x7f90700756d0 2024-05-03T18:44:25.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.495+0000 7f9076e00700 1 -- 172.21.15.33:0/202664774 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90701acba0 con 0x7f90700756d0 2024-05-03T18:44:25.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.502+0000 7f906d600700 1 -- 172.21.15.33:0/202664774 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f906000c980 con 0x7f90700756d0 2024-05-03T18:44:25.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.504+0000 7f906d600700 1 --2- 172.21.15.33:0/202664774 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f90540b2af0 0x7f90540b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:25.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.505+0000 7f906d600700 1 -- 172.21.15.33:0/202664774 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 9518+0+0 (secure 0 0 0) 0x7f90600e1f20 con 0x7f90700756d0 2024-05-03T18:44:25.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.506+0000 7f9067400700 1 --2- 172.21.15.33:0/202664774 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f90540b2af0 0x7f90540b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:25.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.508+0000 7f9067400700 1 --2- 172.21.15.33:0/202664774 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f90540b2af0 0x7f90540b4fb0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f907010fe70 tx=0x7f9058005f90 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:44:25.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.525+0000 7f908183ab80 1 Processor -- start 2024-05-03T18:44:25.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.525+0000 7f908183ab80 1 -- start start 2024-05-03T18:44:25.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.526+0000 7f908183ab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ab972a49b0 0x55ab972a4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:25.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.526+0000 7f908183ab80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab972a52d0 0x55ab9729efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:25.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.526+0000 7f908183ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab9729f5d0 0x55ab9729fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:25.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.526+0000 7f908183ab80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ab971f1aa0 con 0x55ab972a52d0 2024-05-03T18:44:25.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.526+0000 7f908183ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55ab9718f7e0 con 0x55ab9729f5d0 2024-05-03T18:44:25.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.526+0000 7f908183ab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55ab971eda60 con 0x55ab972a49b0 2024-05-03T18:44:25.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.526+0000 7f9067400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab972a52d0 0x55ab9729efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:25.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.526+0000 7f9067400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab972a52d0 0x55ab9729efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57970/0 (socket says 172.21.15.33:57970) 2024-05-03T18:44:25.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.526+0000 7f9067400700 1 -- 172.21.15.33:0/3859252780 learned_addr learned my addr 172.21.15.33:0/3859252780 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:44:25.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.527+0000 7f906fe00700 1 --2- 172.21.15.33:0/3859252780 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ab972a49b0 0x55ab972a4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:25.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.527+0000 7f9074e00700 1 --2- 172.21.15.33:0/3859252780 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab9729f5d0 0x55ab9729fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:25.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.527+0000 7f906fe00700 1 -- 172.21.15.33:0/3859252780 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab9729f5d0 msgr2=0x55ab9729fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:44:25.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.527+0000 7f906fe00700 1 --2- 172.21.15.33:0/3859252780 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab9729f5d0 0x55ab9729fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:25.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.527+0000 7f906fe00700 1 -- 172.21.15.33:0/3859252780 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab972a52d0 msgr2=0x55ab9729efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:44:25.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.527+0000 7f906fe00700 1 --2- 172.21.15.33:0/3859252780 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab972a52d0 0x55ab9729efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:25.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.527+0000 7f906fe00700 1 -- 172.21.15.33:0/3859252780 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab9713e950 con 0x55ab972a49b0 2024-05-03T18:44:25.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.528+0000 7f906fe00700 1 --2- 172.21.15.33:0/3859252780 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ab972a49b0 0x55ab972a4d90 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f90600df230 tx=0x7f906000cc70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=112724997d0b741e server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:44:25.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.528+0000 7f9065600700 1 -- 172.21.15.33:0/3859252780 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90600a8e50 con 0x55ab972a49b0 2024-05-03T18:44:25.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.528+0000 7f9065600700 1 -- 172.21.15.33:0/3859252780 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f90600ab030 con 0x55ab972a49b0 2024-05-03T18:44:25.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.528+0000 7f9065600700 1 -- 172.21.15.33:0/3859252780 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90600a8e50 con 0x55ab972a49b0 2024-05-03T18:44:25.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.529+0000 7f908183ab80 1 -- 172.21.15.33:0/3859252780 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ab972a49b0 msgr2=0x55ab972a4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:25.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.529+0000 7f908183ab80 1 --2- 172.21.15.33:0/3859252780 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ab972a49b0 0x55ab972a4d90 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f90600df230 tx=0x7f906000cc70 comp rx=0 tx=0).stop 2024-05-03T18:44:25.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.530+0000 7f908183ab80 1 -- 172.21.15.33:0/3859252780 shutdown_connections 2024-05-03T18:44:25.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.530+0000 7f908183ab80 1 --2- 172.21.15.33:0/3859252780 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab9729f5d0 0x55ab9729fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:25.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.530+0000 7f908183ab80 1 --2- 172.21.15.33:0/3859252780 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ab972a49b0 0x55ab972a4d90 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:25.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.530+0000 7f908183ab80 1 --2- 172.21.15.33:0/3859252780 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab972a52d0 0x55ab9729efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:25.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.530+0000 7f908183ab80 1 -- 172.21.15.33:0/3859252780 >> 172.21.15.33:0/3859252780 conn(0x55ab97217750 msgr2=0x55ab97299980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:44:25.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.531+0000 7f908183ab80 1 -- 172.21.15.33:0/3859252780 shutdown_connections 2024-05-03T18:44:25.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.531+0000 7f908183ab80 1 -- 172.21.15.33:0/3859252780 wait complete. 2024-05-03T18:44:25.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.531+0000 7f908183ab80 1 -- start start 2024-05-03T18:44:25.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.532+0000 7f908183ab80 1 Processor -- start 2024-05-03T18:44:25.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.534+0000 7f908183ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab9729f5d0 0x55ab972aaec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:25.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.534+0000 7f908183ab80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab972a49b0 0x55ab972a8260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:25.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.534+0000 7f908183ab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ab972a52d0 0x55ab972a87a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:25.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.535+0000 7f9067400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab972a49b0 0x55ab972a8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:25.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.535+0000 7f9067400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab972a49b0 0x55ab972a8260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57978/0 (socket says 172.21.15.33:57978) 2024-05-03T18:44:25.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.535+0000 7f9067400700 1 -- 172.21.15.33:0/3619590516 learned_addr learned my addr 172.21.15.33:0/3619590516 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:44:25.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.536+0000 7f906fe00700 1 --2- 172.21.15.33:0/3619590516 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab9729f5d0 0x55ab972aaec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:25.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.536+0000 7f908183ab80 1 -- 172.21.15.33:0/3619590516 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ab972abd80 con 0x55ab972a49b0 2024-05-03T18:44:25.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.536+0000 7f908183ab80 1 -- 172.21.15.33:0/3619590516 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55ab97157ba0 con 0x55ab9729f5d0 2024-05-03T18:44:25.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.536+0000 7f908183ab80 1 -- 172.21.15.33:0/3619590516 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55ab972abf00 con 0x55ab972a52d0 2024-05-03T18:44:25.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.536+0000 7f908183ab80 10 client.0 _ll_register_callbacks cb 0x55ab97198cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:44:25.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.537+0000 7f9074e00700 1 --2- 172.21.15.33:0/3619590516 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ab972a52d0 0x55ab972a87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:25.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.538+0000 7f9067400700 1 -- 172.21.15.33:0/3619590516 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ab972a52d0 msgr2=0x55ab972a87a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:44:25.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.538+0000 7f9067400700 1 --2- 172.21.15.33:0/3619590516 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ab972a52d0 0x55ab972a87a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:25.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.538+0000 7f9067400700 1 -- 172.21.15.33:0/3619590516 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab9729f5d0 msgr2=0x55ab972aaec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:44:25.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.538+0000 7f9067400700 1 --2- 172.21.15.33:0/3619590516 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab9729f5d0 0x55ab972aaec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:25.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.538+0000 7f9067400700 1 -- 172.21.15.33:0/3619590516 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab971b74a0 con 0x55ab972a49b0 2024-05-03T18:44:25.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.538+0000 7f906fe00700 1 --2- 172.21.15.33:0/3619590516 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab9729f5d0 0x55ab972aaec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:44:25.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.539+0000 7f9067400700 1 --2- 172.21.15.33:0/3619590516 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab972a49b0 0x55ab972a8260 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f905801da90 tx=0x7f90580047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a286d83ffba5f667 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:44:25.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.539+0000 7f905e000700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:44:25.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.541+0000 7f905e000700 1 -- 172.21.15.33:0/3619590516 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f905802a060 con 0x55ab972a49b0 2024-05-03T18:44:25.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.541+0000 7f905e000700 1 -- 172.21.15.33:0/3619590516 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9518+0+0 (secure 0 0 0) 0x7f9058010840 con 0x55ab972a49b0 2024-05-03T18:44:25.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.541+0000 7f908183ab80 1 -- 172.21.15.33:0/3619590516 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90600ab030 con 0x55ab972a49b0 2024-05-03T18:44:25.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.542+0000 7f905e000700 1 -- 172.21.15.33:0/3619590516 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f9058031050 con 0x55ab972a49b0 2024-05-03T18:44:25.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.542+0000 7f908183ab80 10 client.16851 fetch_fsmap learned FSMap version 21 2024-05-03T18:44:25.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.542+0000 7f908183ab80 1 -- 172.21.15.33:0/3619590516 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55ab972b64c0 con 0x55ab972a49b0 2024-05-03T18:44:25.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.543+0000 7f905e000700 1 -- 172.21.15.33:0/3619590516 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f90600ab030 con 0x55ab972a49b0 2024-05-03T18:44:25.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.543+0000 7f905e000700 1 -- 172.21.15.33:0/3619590516 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f9058024030 con 0x55ab972a49b0 2024-05-03T18:44:25.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.543+0000 7f908183ab80 10 client.16851 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:44:25.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.543+0000 7f908183ab80 10 client.16851 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:44:25.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.543+0000 7f908183ab80 20 client.16851 populate_metadata read hostname 'smithi033' 2024-05-03T18:44:25.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.545+0000 7f908183ab80 1 --2- 172.21.15.33:0/3619590516 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55ab972b8e90 0x55ab972bb310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:25.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.545+0000 7f908183ab80 4 client.16851 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:44:25.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.545+0000 7f908183ab80 1 -- 172.21.15.33:0/3619590516 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ab9726c070 con 0x55ab972b8e90 2024-05-03T18:44:25.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.546+0000 7f906fe00700 1 --2- 172.21.15.33:0/3619590516 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55ab972b8e90 0x55ab972bb310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:25.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.548+0000 7f905e000700 10 client.16851 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:44:25.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.548+0000 7f906fe00700 1 --2- 172.21.15.33:0/3619590516 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55ab972b8e90 0x55ab972bb310 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a86b8c0bb7c9810 server_cookie=e76eabcd87eb916e in_seq=0 out_seq=0 2024-05-03T18:44:25.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.550+0000 7f905e000700 1 -- 172.21.15.33:0/3619590516 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ab9726c070 con 0x55ab972b8e90 2024-05-03T18:44:25.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.550+0000 7f905e000700 10 client.16851 handle_command_reply: tid=0 2024-05-03T18:44:25.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.550+0000 7f908183ab80 1 client.16851 shutdown 2024-05-03T18:44:25.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.551+0000 7f908183ab80 1 -- 172.21.15.33:0/3619590516 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab972a49b0 msgr2=0x55ab972a8260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:25.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.551+0000 7f908183ab80 1 --2- 172.21.15.33:0/3619590516 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab972a49b0 0x55ab972a8260 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f905801da90 tx=0x7f90580047c0 comp rx=0 tx=0).stop 2024-05-03T18:44:25.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.552+0000 7f908183ab80 1 -- 172.21.15.33:0/3619590516 shutdown_connections 2024-05-03T18:44:25.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.552+0000 7f908183ab80 1 --2- 172.21.15.33:0/3619590516 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55ab972b8e90 0x55ab972bb310 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:25.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.552+0000 7f908183ab80 1 --2- 172.21.15.33:0/3619590516 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ab972a52d0 0x55ab972a87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:25.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.552+0000 7f908183ab80 1 --2- 172.21.15.33:0/3619590516 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab9729f5d0 0x55ab972aaec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:25.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.552+0000 7f908183ab80 1 --2- 172.21.15.33:0/3619590516 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab972a49b0 0x55ab972a8260 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:25.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.552+0000 7f908183ab80 1 -- 172.21.15.33:0/3619590516 >> 172.21.15.33:0/3619590516 conn(0x55ab97217750 msgr2=0x55ab97294430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:44:25.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.553+0000 7f905e000700 0 client.16851 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:44:25.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.553+0000 7f908183ab80 1 -- 172.21.15.33:0/3619590516 shutdown_connections 2024-05-03T18:44:25.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.553+0000 7f908183ab80 1 -- 172.21.15.33:0/3619590516 wait complete. 2024-05-03T18:44:25.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.553+0000 7f908183ab80 20 client.16851 trim_cache size 0 max 16384 2024-05-03T18:44:25.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.600+0000 7f908183ab80 1 Processor -- start 2024-05-03T18:44:25.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.601+0000 7f908183ab80 1 -- start start 2024-05-03T18:44:25.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.601+0000 7f908183ab80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab9729f5d0 0x55ab972a56b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:25.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.601+0000 7f908183ab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ab972a49b0 0x55ab972a5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:25.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.601+0000 7f908183ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab972a52d0 0x55ab97293ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:25.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.601+0000 7f908183ab80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f90701b3cb0 con 0x55ab9729f5d0 2024-05-03T18:44:25.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.601+0000 7f908183ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f90701b6510 con 0x55ab972a52d0 2024-05-03T18:44:25.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.601+0000 7f908183ab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f90701b4300 con 0x55ab972a49b0 2024-05-03T18:44:25.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.603+0000 7f9067400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ab972a49b0 0x55ab972a5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:25.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.603+0000 7f9067400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ab972a49b0 0x55ab972a5bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:41876/0 (socket says 172.21.15.33:41876) 2024-05-03T18:44:25.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.603+0000 7f9067400700 1 -- 172.21.15.33:0/2821507406 learned_addr learned my addr 172.21.15.33:0/2821507406 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:44:25.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.603+0000 7f906fe00700 1 --2- 172.21.15.33:0/2821507406 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab9729f5d0 0x55ab972a56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:25.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.603+0000 7f9074e00700 1 --2- 172.21.15.33:0/2821507406 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab972a52d0 0x55ab97293ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:25.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.604+0000 7f9067400700 1 -- 172.21.15.33:0/2821507406 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab972a52d0 msgr2=0x55ab97293ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:44:25.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.604+0000 7f9067400700 1 --2- 172.21.15.33:0/2821507406 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab972a52d0 0x55ab97293ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:25.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.604+0000 7f9067400700 1 -- 172.21.15.33:0/2821507406 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab9729f5d0 msgr2=0x55ab972a56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:44:25.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.604+0000 7f9067400700 1 --2- 172.21.15.33:0/2821507406 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab9729f5d0 0x55ab972a56b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:25.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.604+0000 7f9067400700 1 -- 172.21.15.33:0/2821507406 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab972a5100 con 0x55ab972a49b0 2024-05-03T18:44:25.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.605+0000 7f9067400700 1 --2- 172.21.15.33:0/2821507406 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ab972a49b0 0x55ab972a5bf0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9058005d70 tx=0x7f905801de60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a8e62b3c9ab6365 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:44:25.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.606+0000 7f9064c00700 1 -- 172.21.15.33:0/2821507406 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f905803a070 con 0x55ab972a49b0 2024-05-03T18:44:25.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.606+0000 7f9064c00700 1 -- 172.21.15.33:0/2821507406 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f90580253f0 con 0x55ab972a49b0 2024-05-03T18:44:25.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.606+0000 7f9064c00700 1 -- 172.21.15.33:0/2821507406 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9058016070 con 0x55ab972a49b0 2024-05-03T18:44:25.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.607+0000 7f908183ab80 1 -- 172.21.15.33:0/2821507406 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ab972a49b0 msgr2=0x55ab972a5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:25.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.607+0000 7f908183ab80 1 --2- 172.21.15.33:0/2821507406 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ab972a49b0 0x55ab972a5bf0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9058005d70 tx=0x7f905801de60 comp rx=0 tx=0).stop 2024-05-03T18:44:25.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.609+0000 7f908183ab80 1 -- 172.21.15.33:0/2821507406 shutdown_connections 2024-05-03T18:44:25.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.609+0000 7f908183ab80 1 --2- 172.21.15.33:0/2821507406 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab972a52d0 0x55ab97293ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:25.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.609+0000 7f908183ab80 1 --2- 172.21.15.33:0/2821507406 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab9729f5d0 0x55ab972a56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:25.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.609+0000 7f908183ab80 1 --2- 172.21.15.33:0/2821507406 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ab972a49b0 0x55ab972a5bf0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:25.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.609+0000 7f908183ab80 1 -- 172.21.15.33:0/2821507406 >> 172.21.15.33:0/2821507406 conn(0x55ab97217750 msgr2=0x55ab972a6b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:44:25.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.609+0000 7f908183ab80 1 -- 172.21.15.33:0/2821507406 shutdown_connections 2024-05-03T18:44:25.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.609+0000 7f908183ab80 1 -- 172.21.15.33:0/2821507406 wait complete. 2024-05-03T18:44:25.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.610+0000 7f908183ab80 1 -- start start 2024-05-03T18:44:25.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.610+0000 7f908183ab80 1 Processor -- start 2024-05-03T18:44:25.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.612+0000 7f908183ab80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab9729f5d0 0x55ab972bc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:25.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.613+0000 7f908183ab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab972a49b0 0x55ab972bc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:25.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.613+0000 7f908183ab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ab972a52d0 0x55ab9731e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:25.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.614+0000 7f908183ab80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f90701b49d0 con 0x55ab9729f5d0 2024-05-03T18:44:25.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.614+0000 7f908183ab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f90701b5e40 con 0x55ab972a49b0 2024-05-03T18:44:25.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.614+0000 7f908183ab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f90701b50a0 con 0x55ab972a52d0 2024-05-03T18:44:25.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.614+0000 7f9074e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ab972a52d0 0x55ab9731e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:25.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.614+0000 7f908183ab80 10 client.0 _ll_register_callbacks cb 0x55ab97198cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:44:25.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.615+0000 7f9067400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab972a49b0 0x55ab972bc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:25.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.615+0000 7f9067400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab972a49b0 0x55ab972bc6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:53634/0 (socket says 172.21.15.33:53634) 2024-05-03T18:44:25.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.615+0000 7f9067400700 1 -- 172.21.15.33:0/686091962 learned_addr learned my addr 172.21.15.33:0/686091962 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:44:25.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.615+0000 7f906fe00700 1 --2- 172.21.15.33:0/686091962 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab9729f5d0 0x55ab972bc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:25.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.616+0000 7f9067400700 1 -- 172.21.15.33:0/686091962 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ab972a52d0 msgr2=0x55ab9731e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:44:25.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.616+0000 7f9067400700 1 --2- 172.21.15.33:0/686091962 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ab972a52d0 0x55ab9731e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:25.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.616+0000 7f9067400700 1 -- 172.21.15.33:0/686091962 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab9729f5d0 msgr2=0x55ab972bc1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:44:25.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.616+0000 7f9067400700 1 --2- 172.21.15.33:0/686091962 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab9729f5d0 0x55ab972bc1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:25.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.616+0000 7f9067400700 1 -- 172.21.15.33:0/686091962 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab9731e960 con 0x55ab972a49b0 2024-05-03T18:44:25.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.617+0000 7f906fe00700 1 --2- 172.21.15.33:0/686091962 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab9729f5d0 0x55ab972bc1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:44:25.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.617+0000 7f9067400700 1 --2- 172.21.15.33:0/686091962 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab972a49b0 0x55ab972bc6e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f905801c010 tx=0x7f905800a9e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c67be33cc52fc6a server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:44:25.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.617+0000 7f905e000700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-03T18:44:25.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.620+0000 7f905e000700 1 -- 172.21.15.33:0/686091962 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f905803a070 con 0x55ab972a49b0 2024-05-03T18:44:25.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.620+0000 7f905e000700 1 -- 172.21.15.33:0/686091962 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9518+0+0 (secure 0 0 0) 0x7f9058005030 con 0x55ab972a49b0 2024-05-03T18:44:25.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.621+0000 7f908183ab80 1 -- 172.21.15.33:0/686091962 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90580253f0 con 0x55ab972a49b0 2024-05-03T18:44:25.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.625+0000 7f905e000700 1 -- 172.21.15.33:0/686091962 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f9058044050 con 0x55ab972a49b0 2024-05-03T18:44:25.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.626+0000 7f908183ab80 10 client.36647 fetch_fsmap learned FSMap version 21 2024-05-03T18:44:25.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.626+0000 7f908183ab80 1 -- 172.21.15.33:0/686091962 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55ab97334ce0 con 0x55ab972a49b0 2024-05-03T18:44:25.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.626+0000 7f905e000700 1 -- 172.21.15.33:0/686091962 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f90580253f0 con 0x55ab972a49b0 2024-05-03T18:44:25.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.628+0000 7f905e000700 1 -- 172.21.15.33:0/686091962 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f9058024070 con 0x55ab972a49b0 2024-05-03T18:44:25.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.628+0000 7f908183ab80 10 client.36647 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:44:25.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.628+0000 7f908183ab80 10 client.36647 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:44:25.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.628+0000 7f908183ab80 20 client.36647 populate_metadata read hostname 'smithi033' 2024-05-03T18:44:25.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.628+0000 7f908183ab80 1 --2- 172.21.15.33:0/686091962 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55ab973352a0 0x55ab973376c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:25.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.628+0000 7f908183ab80 4 client.36647 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-03T18:44:25.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.628+0000 7f908183ab80 1 -- 172.21.15.33:0/686091962 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ab971ff480 con 0x55ab973352a0 2024-05-03T18:44:25.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.629+0000 7f906fe00700 1 --2- 172.21.15.33:0/686091962 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55ab973352a0 0x55ab973376c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:25.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.630+0000 7f905e000700 10 client.36647 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:44:25.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.630+0000 7f906fe00700 1 --2- 172.21.15.33:0/686091962 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55ab973352a0 0x55ab973376c0 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92c92e9b61036d8f server_cookie=aaf583740d3f1815 in_seq=0 out_seq=0 2024-05-03T18:44:25.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.633+0000 7f905e000700 1 -- 172.21.15.33:0/686091962 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ab971ff480 con 0x55ab973352a0 2024-05-03T18:44:25.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.633+0000 7f905e000700 10 client.36647 handle_command_reply: tid=0 2024-05-03T18:44:25.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.633+0000 7f908183ab80 1 client.36647 shutdown 2024-05-03T18:44:25.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.633+0000 7f908183ab80 1 -- 172.21.15.33:0/686091962 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab972a49b0 msgr2=0x55ab972bc6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:25.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.633+0000 7f908183ab80 1 --2- 172.21.15.33:0/686091962 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab972a49b0 0x55ab972bc6e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f905801c010 tx=0x7f905800a9e0 comp rx=0 tx=0).stop 2024-05-03T18:44:25.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.633+0000 7f908183ab80 1 -- 172.21.15.33:0/686091962 shutdown_connections 2024-05-03T18:44:25.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.633+0000 7f908183ab80 1 --2- 172.21.15.33:0/686091962 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55ab973352a0 0x55ab973376c0 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:25.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.633+0000 7f905e000700 0 client.36647 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:44:25.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.633+0000 7f908183ab80 1 --2- 172.21.15.33:0/686091962 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ab972a52d0 0x55ab9731e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:25.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.633+0000 7f908183ab80 1 --2- 172.21.15.33:0/686091962 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab9729f5d0 0x55ab972bc1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:25.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.633+0000 7f908183ab80 1 --2- 172.21.15.33:0/686091962 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab972a49b0 0x55ab972bc6e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:25.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.633+0000 7f908183ab80 1 -- 172.21.15.33:0/686091962 >> 172.21.15.33:0/686091962 conn(0x55ab97217750 msgr2=0x55ab972a56b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:44:25.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.634+0000 7f908183ab80 1 -- 172.21.15.33:0/686091962 shutdown_connections 2024-05-03T18:44:25.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.634+0000 7f908183ab80 1 -- 172.21.15.33:0/686091962 wait complete. 2024-05-03T18:44:25.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.635+0000 7f908183ab80 20 client.36647 trim_cache size 0 max 16384 2024-05-03T18:44:25.636 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-03T18:44:25.636 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-05-03T18:44:25.637 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "dafd45c5-1ae5-414d-940f-9eeb29bd5c58", 2024-05-03T18:44:25.637 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-05-03T18:44:25.637 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-03T18:44:25.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.636+0000 7f9076e00700 1 -- 172.21.15.33:0/202664774 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f90540b2af0 msgr2=0x7f90540b4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:25.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.636+0000 7f9076e00700 1 --2- 172.21.15.33:0/202664774 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f90540b2af0 0x7f90540b4fb0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f907010fe70 tx=0x7f9058005f90 comp rx=0 tx=0).stop 2024-05-03T18:44:25.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.636+0000 7f9076e00700 1 -- 172.21.15.33:0/202664774 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f90700756d0 msgr2=0x7f907010e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:25.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.636+0000 7f9076e00700 1 --2- 172.21.15.33:0/202664774 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f90700756d0 0x7f907010e330 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f906000ada0 tx=0x7f906000c330 comp rx=0 tx=0).stop 2024-05-03T18:44:25.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.637+0000 7f9076e00700 1 -- 172.21.15.33:0/202664774 shutdown_connections 2024-05-03T18:44:25.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.637+0000 7f9076e00700 1 --2- 172.21.15.33:0/202664774 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f907010fea0 0x7f90701abcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:25.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.637+0000 7f9076e00700 1 --2- 172.21.15.33:0/202664774 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f90540b2af0 0x7f90540b4fb0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:25.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.637+0000 7f9076e00700 1 --2- 172.21.15.33:0/202664774 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f90700756d0 0x7f907010e330 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:25.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.637+0000 7f9076e00700 1 --2- 172.21.15.33:0/202664774 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9070075ff0 0x7f907010e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:25.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.637+0000 7f9076e00700 1 -- 172.21.15.33:0/202664774 >> 172.21.15.33:0/202664774 conn(0x7f907007a600 msgr2=0x7f9070101ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:44:25.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.637+0000 7f9076e00700 1 -- 172.21.15.33:0/202664774 shutdown_connections 2024-05-03T18:44:25.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:25.637+0000 7f9076e00700 1 -- 172.21.15.33:0/202664774 wait complete. 2024-05-03T18:44:25.737 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:44:25 smithi033 ceph-mon[28330]: pgmap v543: 129 pgs: 129 active+clean; 3.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 602 KiB/s rd, 9.1 MiB/s wr, 661 op/s 2024-05-03T18:44:25.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:44:25 smithi150 ceph-mon[32492]: pgmap v543: 129 pgs: 129 active+clean; 3.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 602 KiB/s rd, 9.1 MiB/s wr, 661 op/s 2024-05-03T18:44:26.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:44:25 smithi099 ceph-mon[31658]: pgmap v543: 129 pgs: 129 active+clean; 3.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 602 KiB/s rd, 9.1 MiB/s wr, 661 op/s 2024-05-03T18:44:26.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:44:26 smithi150 ceph-mon[32492]: scrub queued for path: / 2024-05-03T18:44:26.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:44:26 smithi150 ceph-mon[32492]: scrub summary: idle+waiting paths [/] 2024-05-03T18:44:26.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:44:26 smithi150 ceph-mon[32492]: scrub summary: active paths [/] 2024-05-03T18:44:26.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:44:26 smithi150 ceph-mon[32492]: scrub summary: idle+waiting paths [/] 2024-05-03T18:44:26.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:44:26 smithi150 ceph-mon[32492]: scrub summary: idle 2024-05-03T18:44:26.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:44:26 smithi033 ceph-mon[28330]: scrub queued for path: / 2024-05-03T18:44:26.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:44:26 smithi033 ceph-mon[28330]: scrub summary: idle+waiting paths [/] 2024-05-03T18:44:26.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:44:26 smithi033 ceph-mon[28330]: scrub summary: active paths [/] 2024-05-03T18:44:26.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:44:26 smithi033 ceph-mon[28330]: scrub summary: idle+waiting paths [/] 2024-05-03T18:44:26.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:44:26 smithi033 ceph-mon[28330]: scrub summary: idle 2024-05-03T18:44:27.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:44:26 smithi099 ceph-mon[31658]: scrub queued for path: / 2024-05-03T18:44:27.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:44:26 smithi099 ceph-mon[31658]: scrub summary: idle+waiting paths [/] 2024-05-03T18:44:27.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:44:26 smithi099 ceph-mon[31658]: scrub summary: active paths [/] 2024-05-03T18:44:27.047 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:44:26 smithi099 ceph-mon[31658]: scrub summary: idle+waiting paths [/] 2024-05-03T18:44:27.047 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:44:26 smithi099 ceph-mon[31658]: scrub summary: idle 2024-05-03T18:44:27.735 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:44:27 smithi033 ceph-mon[28330]: pgmap v544: 129 pgs: 129 active+clean; 3.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 636 KiB/s rd, 9.2 MiB/s wr, 655 op/s 2024-05-03T18:44:28.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:44:27 smithi099 ceph-mon[31658]: pgmap v544: 129 pgs: 129 active+clean; 3.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 636 KiB/s rd, 9.2 MiB/s wr, 655 op/s 2024-05-03T18:44:28.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:44:27 smithi150 ceph-mon[32492]: pgmap v544: 129 pgs: 129 active+clean; 3.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 636 KiB/s rd, 9.2 MiB/s wr, 655 op/s 2024-05-03T18:44:28.430 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-03T18:44:29.315 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:44:29.986 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:44:29 smithi033 ceph-mon[28330]: pgmap v545: 129 pgs: 129 active+clean; 3.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 7.1 MiB/s wr, 539 op/s 2024-05-03T18:44:29.986 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:44:29 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:44:30.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:44:29 smithi099 ceph-mon[31658]: pgmap v545: 129 pgs: 129 active+clean; 3.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 7.1 MiB/s wr, 539 op/s 2024-05-03T18:44:30.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:44:29 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:44:30.111 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 795.0 s, 479.4 tps, lat 216.377 ms stddev 111.518 2024-05-03T18:44:30.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:44:29 smithi150 ceph-mon[32492]: pgmap v545: 129 pgs: 129 active+clean; 3.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 7.1 MiB/s wr, 539 op/s 2024-05-03T18:44:30.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:44:29 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:44:30.968 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:44:30 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:44:30.968 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:44:30 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:44:30.968 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:44:30 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:44:30.968 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:44:30 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:44:31.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:44:30 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:44:31.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:44:30 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:44:31.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:44:30 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:44:31.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:44:30 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:44:31.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:44:30 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:44:31.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:44:30 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:44:31.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:44:30 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:44:31.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:44:30 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:44:32.062 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:44:31 smithi099 ceph-mon[31658]: pgmap v546: 129 pgs: 129 active+clean; 3.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 10 MiB/s wr, 778 op/s 2024-05-03T18:44:32.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:44:31 smithi150 ceph-mon[32492]: pgmap v546: 129 pgs: 129 active+clean; 3.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 10 MiB/s wr, 778 op/s 2024-05-03T18:44:32.202 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:44:31 smithi033 ceph-mon[28330]: pgmap v546: 129 pgs: 129 active+clean; 3.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 10 MiB/s wr, 778 op/s 2024-05-03T18:44:33.961 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:44:33 smithi033 ceph-mon[28330]: pgmap v547: 129 pgs: 129 active+clean; 3.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 6.5 MiB/s wr, 513 op/s 2024-05-03T18:44:33.961 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:44:33 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:44:33.961 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:44:33 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:44:33.961 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:44:33 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:44:34.073 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:44:33 smithi099 ceph-mon[31658]: pgmap v547: 129 pgs: 129 active+clean; 3.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 6.5 MiB/s wr, 513 op/s 2024-05-03T18:44:34.073 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:44:33 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:44:34.073 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:44:33 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:44:34.073 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:44:33 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:44:34.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:44:33 smithi150 ceph-mon[32492]: pgmap v547: 129 pgs: 129 active+clean; 3.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 6.5 MiB/s wr, 513 op/s 2024-05-03T18:44:34.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:44:33 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:44:34.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:44:33 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:44:34.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:44:33 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:44:35.111 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 800.0 s, 487.8 tps, lat 203.318 ms stddev 104.441 2024-05-03T18:44:35.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.444+0000 7fab19200700 1 -- 172.21.15.33:0/4254180758 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab1406bb40 msgr2=0x7fab14110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:35.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.444+0000 7fab19200700 1 --2- 172.21.15.33:0/4254180758 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab1406bb40 0x7fab14110330 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fab0c009f90 tx=0x7fab0c00b330 comp rx=0 tx=0).stop 2024-05-03T18:44:35.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.444+0000 7fab19200700 1 -- 172.21.15.33:0/4254180758 shutdown_connections 2024-05-03T18:44:35.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.444+0000 7fab19200700 1 --2- 172.21.15.33:0/4254180758 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab1406bb40 0x7fab14110330 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:35.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.444+0000 7fab19200700 1 --2- 172.21.15.33:0/4254180758 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab14109510 0x7fab141098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:35.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.444+0000 7fab19200700 1 --2- 172.21.15.33:0/4254180758 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab1406b180 0x7fab1406b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:35.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.444+0000 7fab19200700 1 -- 172.21.15.33:0/4254180758 >> 172.21.15.33:0/4254180758 conn(0x7fab14079350 msgr2=0x7fab14079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:44:35.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.445+0000 7fab19200700 1 -- 172.21.15.33:0/4254180758 shutdown_connections 2024-05-03T18:44:35.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.445+0000 7fab19200700 1 -- 172.21.15.33:0/4254180758 wait complete. 2024-05-03T18:44:35.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.447+0000 7fab19200700 1 Processor -- start 2024-05-03T18:44:35.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.447+0000 7fab19200700 1 -- start start 2024-05-03T18:44:35.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.447+0000 7fab19200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab1406b180 0x7fab141a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:35.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.447+0000 7fab19200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab1406bb40 0x7fab141a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:35.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.447+0000 7fab19200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab14109510 0x7fab1410f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:35.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.447+0000 7fab19200700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fab14113520 con 0x7fab1406b180 2024-05-03T18:44:35.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.447+0000 7fab19200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fab141133a0 con 0x7fab14109510 2024-05-03T18:44:35.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.447+0000 7fab19200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fab141136a0 con 0x7fab1406bb40 2024-05-03T18:44:35.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.448+0000 7fab11400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab1406bb40 0x7fab141a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:35.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.448+0000 7fab11400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab1406bb40 0x7fab141a0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:58106/0 (socket says 172.21.15.33:58106) 2024-05-03T18:44:35.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.448+0000 7fab11400700 1 -- 172.21.15.33:0/2175061934 learned_addr learned my addr 172.21.15.33:0/2175061934 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:44:35.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.448+0000 7fab12800700 1 --2- 172.21.15.33:0/2175061934 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab14109510 0x7fab1410f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:35.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.448+0000 7fab11e00700 1 --2- 172.21.15.33:0/2175061934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab1406b180 0x7fab141a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:35.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.449+0000 7fab12800700 1 -- 172.21.15.33:0/2175061934 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab1406bb40 msgr2=0x7fab141a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:35.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.449+0000 7fab12800700 1 --2- 172.21.15.33:0/2175061934 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab1406bb40 0x7fab141a0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:35.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.449+0000 7fab12800700 1 -- 172.21.15.33:0/2175061934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab1406b180 msgr2=0x7fab141a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:35.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.449+0000 7fab12800700 1 --2- 172.21.15.33:0/2175061934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab1406b180 0x7fab141a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:35.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.449+0000 7fab12800700 1 -- 172.21.15.33:0/2175061934 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab1410fe30 con 0x7fab14109510 2024-05-03T18:44:35.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.449+0000 7fab11e00700 1 --2- 172.21.15.33:0/2175061934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab1406b180 0x7fab141a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:44:35.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.450+0000 7fab11400700 1 --2- 172.21.15.33:0/2175061934 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab1406bb40 0x7fab141a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:44:35.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.450+0000 7fab12800700 1 --2- 172.21.15.33:0/2175061934 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab14109510 0x7fab1410f530 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fab0c00b6c0 tx=0x7fab0c009510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:44:35.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.451+0000 7fab0aa00700 1 -- 172.21.15.33:0/2175061934 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab0c021070 con 0x7fab14109510 2024-05-03T18:44:35.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.451+0000 7fab0aa00700 1 -- 172.21.15.33:0/2175061934 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fab0c007ac0 con 0x7fab14109510 2024-05-03T18:44:35.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.451+0000 7fab19200700 1 -- 172.21.15.33:0/2175061934 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab141100c0 con 0x7fab14109510 2024-05-03T18:44:35.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.451+0000 7fab0aa00700 1 -- 172.21.15.33:0/2175061934 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab0c0052a0 con 0x7fab14109510 2024-05-03T18:44:35.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.451+0000 7fab19200700 1 -- 172.21.15.33:0/2175061934 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab1419c240 con 0x7fab14109510 2024-05-03T18:44:35.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.463+0000 7fab0aa00700 1 -- 172.21.15.33:0/2175061934 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fab0c024070 con 0x7fab14109510 2024-05-03T18:44:35.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.466+0000 7fab0aa00700 1 --2- 172.21.15.33:0/2175061934 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7faaf80b2b40 0x7faaf80b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:35.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.466+0000 7fab0aa00700 1 -- 172.21.15.33:0/2175061934 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 9518+0+0 (secure 0 0 0) 0x7fab0c0e4760 con 0x7fab14109510 2024-05-03T18:44:35.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.467+0000 7fab11e00700 1 --2- 172.21.15.33:0/2175061934 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7faaf80b2b40 0x7faaf80b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:35.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.478+0000 7fab11e00700 1 --2- 172.21.15.33:0/2175061934 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7faaf80b2b40 0x7faaf80b5000 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fab0400baa0 tx=0x7fab04005db0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:44:35.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.485+0000 7fab23b76b80 1 Processor -- start 2024-05-03T18:44:35.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.485+0000 7fab23b76b80 1 -- start start 2024-05-03T18:44:35.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.486+0000 7fab23b76b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563cf01247b0 0x563cf0124b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:35.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.486+0000 7fab23b76b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563cf01250d0 0x563cf011ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:35.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.486+0000 7fab23b76b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563cf01205d0 0x563cf0120a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:35.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.486+0000 7fab23b76b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563cf0072a20 con 0x563cf01247b0 2024-05-03T18:44:35.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.486+0000 7fab23b76b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x563cf0010760 con 0x563cf01250d0 2024-05-03T18:44:35.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.486+0000 7fab23b76b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x563cf006e9e0 con 0x563cf01205d0 2024-05-03T18:44:35.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.487+0000 7fab11400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563cf01250d0 0x563cf011ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:35.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.487+0000 7fab12800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563cf01205d0 0x563cf0120a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:35.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.487+0000 7fab12800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563cf01205d0 0x563cf0120a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:58108/0 (socket says 172.21.15.33:58108) 2024-05-03T18:44:35.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.487+0000 7fab12800700 1 -- 172.21.15.33:0/386398230 learned_addr learned my addr 172.21.15.33:0/386398230 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:44:35.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.488+0000 7fab11400700 1 -- 172.21.15.33:0/386398230 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563cf01205d0 msgr2=0x563cf0120a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:44:35.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.488+0000 7fab11400700 1 --2- 172.21.15.33:0/386398230 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563cf01205d0 0x563cf0120a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:35.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.488+0000 7fab11400700 1 -- 172.21.15.33:0/386398230 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563cf01247b0 msgr2=0x563cf0124b90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-03T18:44:35.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.488+0000 7fab11400700 1 --2- 172.21.15.33:0/386398230 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563cf01247b0 0x563cf0124b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:35.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.488+0000 7fab11400700 1 -- 172.21.15.33:0/386398230 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ceffbf8d0 con 0x563cf01250d0 2024-05-03T18:44:35.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.489+0000 7fab11400700 1 --2- 172.21.15.33:0/386398230 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563cf01250d0 0x563cf011ffb0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fab14198e30 tx=0x7faafc011850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee3e2240a980aea8 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:44:35.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.490+0000 7fab03400700 1 -- 172.21.15.33:0/386398230 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faafc004230 con 0x563cf01250d0 2024-05-03T18:44:35.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.490+0000 7fab03400700 1 -- 172.21.15.33:0/386398230 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faafc0043d0 con 0x563cf01250d0 2024-05-03T18:44:35.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.490+0000 7fab03400700 1 -- 172.21.15.33:0/386398230 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faafc0198a0 con 0x563cf01250d0 2024-05-03T18:44:35.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.492+0000 7fab23b76b80 1 -- 172.21.15.33:0/386398230 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563cf01250d0 msgr2=0x563cf011ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:35.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.492+0000 7fab23b76b80 1 --2- 172.21.15.33:0/386398230 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563cf01250d0 0x563cf011ffb0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fab14198e30 tx=0x7faafc011850 comp rx=0 tx=0).stop 2024-05-03T18:44:35.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.492+0000 7fab23b76b80 1 -- 172.21.15.33:0/386398230 shutdown_connections 2024-05-03T18:44:35.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.492+0000 7fab23b76b80 1 --2- 172.21.15.33:0/386398230 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563cf01205d0 0x563cf0120a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:35.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.492+0000 7fab23b76b80 1 --2- 172.21.15.33:0/386398230 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563cf01247b0 0x563cf0124b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:35.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.492+0000 7fab23b76b80 1 --2- 172.21.15.33:0/386398230 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563cf01250d0 0x563cf011ffb0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:35.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.492+0000 7fab23b76b80 1 -- 172.21.15.33:0/386398230 >> 172.21.15.33:0/386398230 conn(0x563cf004eb60 msgr2=0x563cf011a7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:44:35.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.493+0000 7fab23b76b80 1 -- 172.21.15.33:0/386398230 shutdown_connections 2024-05-03T18:44:35.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.493+0000 7fab23b76b80 1 -- 172.21.15.33:0/386398230 wait complete. 2024-05-03T18:44:35.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.494+0000 7fab23b76b80 1 -- start start 2024-05-03T18:44:35.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.494+0000 7fab23b76b80 1 Processor -- start 2024-05-03T18:44:35.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.495+0000 7fab23b76b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563cf01205d0 0x563cf012bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:35.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.495+0000 7fab23b76b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563cf01247b0 0x563cf01292b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:35.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.495+0000 7fab23b76b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563cf01250d0 0x563cf01297f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:35.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.496+0000 7fab11e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563cf01205d0 0x563cf012bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:35.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.497+0000 7fab11e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563cf01205d0 0x563cf012bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:54204/0 (socket says 172.21.15.33:54204) 2024-05-03T18:44:35.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.497+0000 7fab23b76b80 1 -- 172.21.15.33:0/2599270397 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563cf012cca0 con 0x563cf01250d0 2024-05-03T18:44:35.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.497+0000 7fab23b76b80 1 -- 172.21.15.33:0/2599270397 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x563ceffd8b20 con 0x563cf01205d0 2024-05-03T18:44:35.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.497+0000 7fab23b76b80 1 -- 172.21.15.33:0/2599270397 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x563cf012ce20 con 0x563cf01247b0 2024-05-03T18:44:35.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.497+0000 7fab23b76b80 10 client.0 _ll_register_callbacks cb 0x563cf0019c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:44:35.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.497+0000 7fab12800700 1 --2- 172.21.15.33:0/2599270397 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563cf01250d0 0x563cf01297f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:35.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.497+0000 7fab11e00700 1 -- 172.21.15.33:0/2599270397 learned_addr learned my addr 172.21.15.33:0/2599270397 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:44:35.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.497+0000 7fab11400700 1 --2- 172.21.15.33:0/2599270397 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563cf01247b0 0x563cf01292b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:35.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.498+0000 7fab12800700 1 -- 172.21.15.33:0/2599270397 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563cf01247b0 msgr2=0x563cf01292b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:44:35.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.498+0000 7fab12800700 1 --2- 172.21.15.33:0/2599270397 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563cf01247b0 0x563cf01292b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:35.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.498+0000 7fab12800700 1 -- 172.21.15.33:0/2599270397 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563cf01205d0 msgr2=0x563cf012bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:44:35.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.498+0000 7fab12800700 1 --2- 172.21.15.33:0/2599270397 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563cf01205d0 0x563cf012bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:35.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.498+0000 7fab12800700 1 -- 172.21.15.33:0/2599270397 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563cf0038420 con 0x563cf01250d0 2024-05-03T18:44:35.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.499+0000 7faaf7e00700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:44:35.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.499+0000 7fab12800700 1 --2- 172.21.15.33:0/2599270397 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563cf01250d0 0x563cf01297f0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fab0c009f90 tx=0x7fab0c0aae00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd52f2382befbd39 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:44:35.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.499+0000 7faaf7e00700 1 -- 172.21.15.33:0/2599270397 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab0c02b070 con 0x563cf01250d0 2024-05-03T18:44:35.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.500+0000 7fab23b76b80 1 -- 172.21.15.33:0/2599270397 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faafc0043d0 con 0x563cf01250d0 2024-05-03T18:44:35.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.500+0000 7faaf7e00700 1 -- 172.21.15.33:0/2599270397 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9518+0+0 (secure 0 0 0) 0x7fab0c01de00 con 0x563cf01250d0 2024-05-03T18:44:35.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.500+0000 7faaf7e00700 1 -- 172.21.15.33:0/2599270397 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fab0c02b070 con 0x563cf01250d0 2024-05-03T18:44:35.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.500+0000 7fab23b76b80 10 client.16863 fetch_fsmap learned FSMap version 21 2024-05-03T18:44:35.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.500+0000 7fab23b76b80 1 -- 172.21.15.33:0/2599270397 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x563cf0137570 con 0x563cf01250d0 2024-05-03T18:44:35.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.501+0000 7faaf7e00700 1 -- 172.21.15.33:0/2599270397 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faafc0043d0 con 0x563cf01250d0 2024-05-03T18:44:35.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.501+0000 7faaf7e00700 1 -- 172.21.15.33:0/2599270397 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fab0c011070 con 0x563cf01250d0 2024-05-03T18:44:35.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.501+0000 7fab23b76b80 10 client.16863 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:44:35.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.501+0000 7fab23b76b80 10 client.16863 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:44:35.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.502+0000 7fab23b76b80 20 client.16863 populate_metadata read hostname 'smithi033' 2024-05-03T18:44:35.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.502+0000 7fab23b76b80 1 --2- 172.21.15.33:0/2599270397 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x563cf0139b20 0x563cf013bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:35.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.502+0000 7fab23b76b80 4 client.16863 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:44:35.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.502+0000 7fab23b76b80 1 -- 172.21.15.33:0/2599270397 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563cf00ed1d0 con 0x563cf0139b20 2024-05-03T18:44:35.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.503+0000 7fab11e00700 1 --2- 172.21.15.33:0/2599270397 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x563cf0139b20 0x563cf013bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:35.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.504+0000 7fab11e00700 1 --2- 172.21.15.33:0/2599270397 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x563cf0139b20 0x563cf013bf40 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b75174fbac6b8e13 server_cookie=5a66852d97ca6ca0 in_seq=0 out_seq=0 2024-05-03T18:44:35.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.504+0000 7faaf7e00700 10 client.16863 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:44:35.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.505+0000 7faaf7e00700 1 -- 172.21.15.33:0/2599270397 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563cf00ed1d0 con 0x563cf0139b20 2024-05-03T18:44:35.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.505+0000 7faaf7e00700 10 client.16863 handle_command_reply: tid=0 2024-05-03T18:44:35.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.506+0000 7fab23b76b80 1 client.16863 shutdown 2024-05-03T18:44:35.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.506+0000 7fab23b76b80 1 -- 172.21.15.33:0/2599270397 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563cf01250d0 msgr2=0x563cf01297f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:35.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.506+0000 7fab23b76b80 1 --2- 172.21.15.33:0/2599270397 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563cf01250d0 0x563cf01297f0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fab0c009f90 tx=0x7fab0c0aae00 comp rx=0 tx=0).stop 2024-05-03T18:44:35.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.507+0000 7fab23b76b80 1 -- 172.21.15.33:0/2599270397 shutdown_connections 2024-05-03T18:44:35.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.507+0000 7fab23b76b80 1 --2- 172.21.15.33:0/2599270397 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x563cf0139b20 0x563cf013bf40 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:35.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.507+0000 7fab23b76b80 1 --2- 172.21.15.33:0/2599270397 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563cf01250d0 0x563cf01297f0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:35.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.507+0000 7fab23b76b80 1 --2- 172.21.15.33:0/2599270397 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563cf01205d0 0x563cf012bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:35.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.507+0000 7fab23b76b80 1 --2- 172.21.15.33:0/2599270397 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563cf01247b0 0x563cf01292b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:35.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.507+0000 7fab23b76b80 1 -- 172.21.15.33:0/2599270397 >> 172.21.15.33:0/2599270397 conn(0x563cf004eb60 msgr2=0x563cf0125e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:44:35.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.507+0000 7faaf7e00700 0 client.16863 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:44:35.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.508+0000 7fab23b76b80 1 -- 172.21.15.33:0/2599270397 shutdown_connections 2024-05-03T18:44:35.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.508+0000 7fab23b76b80 1 -- 172.21.15.33:0/2599270397 wait complete. 2024-05-03T18:44:35.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.508+0000 7fab23b76b80 20 client.16863 trim_cache size 0 max 16384 2024-05-03T18:44:35.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.550+0000 7fab23b76b80 1 Processor -- start 2024-05-03T18:44:35.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.550+0000 7fab23b76b80 1 -- start start 2024-05-03T18:44:35.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.550+0000 7fab23b76b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563cf01205d0 0x563cf0135dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:35.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.550+0000 7fab23b76b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563cf01247b0 0x563cf011a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:35.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.551+0000 7fab23b76b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563cf01250d0 0x563cf0141810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:35.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.551+0000 7fab23b76b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7faaf000ce10 con 0x563cf01250d0 2024-05-03T18:44:35.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.551+0000 7fab23b76b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7faaf000f670 con 0x563cf01205d0 2024-05-03T18:44:35.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.551+0000 7fab23b76b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7faaf000d460 con 0x563cf01247b0 2024-05-03T18:44:35.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.551+0000 7fab11e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563cf01205d0 0x563cf0135dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:35.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.552+0000 7fab12800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563cf01250d0 0x563cf0141810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:35.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.552+0000 7fab11e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563cf01205d0 0x563cf0135dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:54218/0 (socket says 172.21.15.33:54218) 2024-05-03T18:44:35.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.552+0000 7fab12800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563cf01250d0 0x563cf0141810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56878/0 (socket says 172.21.15.33:56878) 2024-05-03T18:44:35.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.552+0000 7fab11e00700 1 -- 172.21.15.33:0/469702824 learned_addr learned my addr 172.21.15.33:0/469702824 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:44:35.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.552+0000 7fab11400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563cf01247b0 0x563cf011a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:35.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.553+0000 7fab11e00700 1 -- 172.21.15.33:0/469702824 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563cf01247b0 msgr2=0x563cf011a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:44:35.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.553+0000 7fab11e00700 1 --2- 172.21.15.33:0/469702824 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563cf01247b0 0x563cf011a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:35.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.553+0000 7fab11e00700 1 -- 172.21.15.33:0/469702824 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563cf01250d0 msgr2=0x563cf0141810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:44:35.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.553+0000 7fab11e00700 1 --2- 172.21.15.33:0/469702824 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563cf01250d0 0x563cf0141810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:35.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.553+0000 7fab11e00700 1 -- 172.21.15.33:0/469702824 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563cf0124400 con 0x563cf01205d0 2024-05-03T18:44:35.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.554+0000 7fab12800700 1 --2- 172.21.15.33:0/469702824 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563cf01250d0 0x563cf0141810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:44:35.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.554+0000 7fab11e00700 1 --2- 172.21.15.33:0/469702824 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563cf01205d0 0x563cf0135dc0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fab0400e010 tx=0x7fab04013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b690abb6cdcd388 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:44:35.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.555+0000 7fab02a00700 1 -- 172.21.15.33:0/469702824 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab0401ead0 con 0x563cf01205d0 2024-05-03T18:44:35.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.556+0000 7fab02a00700 1 -- 172.21.15.33:0/469702824 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fab0402c070 con 0x563cf01205d0 2024-05-03T18:44:35.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.556+0000 7fab02a00700 1 -- 172.21.15.33:0/469702824 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab0400f680 con 0x563cf01205d0 2024-05-03T18:44:35.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.557+0000 7fab23b76b80 1 -- 172.21.15.33:0/469702824 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563cf01205d0 msgr2=0x563cf0135dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:35.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.557+0000 7fab23b76b80 1 --2- 172.21.15.33:0/469702824 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563cf01205d0 0x563cf0135dc0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fab0400e010 tx=0x7fab04013000 comp rx=0 tx=0).stop 2024-05-03T18:44:35.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.557+0000 7fab23b76b80 1 -- 172.21.15.33:0/469702824 shutdown_connections 2024-05-03T18:44:35.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.558+0000 7fab23b76b80 1 --2- 172.21.15.33:0/469702824 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563cf01250d0 0x563cf0141810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:35.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.558+0000 7fab23b76b80 1 --2- 172.21.15.33:0/469702824 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563cf01205d0 0x563cf0135dc0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:35.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.558+0000 7fab23b76b80 1 --2- 172.21.15.33:0/469702824 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563cf01247b0 0x563cf011a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:35.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.558+0000 7fab23b76b80 1 -- 172.21.15.33:0/469702824 >> 172.21.15.33:0/469702824 conn(0x563cf004eb60 msgr2=0x563cf011fc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:44:35.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.558+0000 7fab23b76b80 1 -- 172.21.15.33:0/469702824 shutdown_connections 2024-05-03T18:44:35.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.559+0000 7fab23b76b80 1 -- 172.21.15.33:0/469702824 wait complete. 2024-05-03T18:44:35.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.560+0000 7fab23b76b80 1 -- start start 2024-05-03T18:44:35.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.560+0000 7fab23b76b80 1 Processor -- start 2024-05-03T18:44:35.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.562+0000 7fab23b76b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563cf01205d0 0x563cf0132150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:35.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.562+0000 7fab23b76b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563cf01247b0 0x563cf0132690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:35.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.562+0000 7fab23b76b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563cf01250d0 0x563cf012d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:35.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.562+0000 7fab23b76b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7faaf000db30 con 0x563cf01205d0 2024-05-03T18:44:35.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.562+0000 7fab23b76b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7faaf000efa0 con 0x563cf01247b0 2024-05-03T18:44:35.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.562+0000 7fab23b76b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7faaf000e200 con 0x563cf01250d0 2024-05-03T18:44:35.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.563+0000 7fab23b76b80 10 client.0 _ll_register_callbacks cb 0x563cf0019c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:44:35.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.563+0000 7fab12800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563cf01250d0 0x563cf012d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:35.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.563+0000 7fab12800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563cf01250d0 0x563cf012d100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:58142/0 (socket says 172.21.15.33:58142) 2024-05-03T18:44:35.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.563+0000 7fab12800700 1 -- 172.21.15.33:0/232752374 learned_addr learned my addr 172.21.15.33:0/232752374 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:44:35.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.563+0000 7fab11400700 1 --2- 172.21.15.33:0/232752374 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563cf01247b0 0x563cf0132690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:35.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.563+0000 7fab11e00700 1 --2- 172.21.15.33:0/232752374 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563cf01205d0 0x563cf0132150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:35.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.565+0000 7fab11e00700 1 -- 172.21.15.33:0/232752374 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563cf01250d0 msgr2=0x563cf012d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:44:35.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.565+0000 7fab11e00700 1 --2- 172.21.15.33:0/232752374 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563cf01250d0 0x563cf012d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:35.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.565+0000 7fab11e00700 1 -- 172.21.15.33:0/232752374 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563cf01247b0 msgr2=0x563cf0132690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:44:35.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.565+0000 7fab11e00700 1 --2- 172.21.15.33:0/232752374 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563cf01247b0 0x563cf0132690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:35.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.565+0000 7fab11e00700 1 -- 172.21.15.33:0/232752374 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563cf012d6c0 con 0x563cf01205d0 2024-05-03T18:44:35.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.565+0000 7fab12800700 1 --2- 172.21.15.33:0/232752374 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563cf01250d0 0x563cf012d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:44:35.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.565+0000 7fab11400700 1 --2- 172.21.15.33:0/232752374 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563cf01247b0 0x563cf0132690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:44:35.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.565+0000 7fab11e00700 1 --2- 172.21.15.33:0/232752374 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563cf01205d0 0x563cf0132150 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fab0400dfe0 tx=0x7fab0401eb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afc6a5a5765eb2 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:44:35.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.565+0000 7faaf7e00700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:44:35.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.567+0000 7faaf7e00700 1 -- 172.21.15.33:0/232752374 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab04035070 con 0x563cf01205d0 2024-05-03T18:44:35.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.567+0000 7faaf7e00700 1 -- 172.21.15.33:0/232752374 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9518+0+0 (secure 0 0 0) 0x7fab040265f0 con 0x563cf01205d0 2024-05-03T18:44:35.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.567+0000 7fab23b76b80 1 -- 172.21.15.33:0/232752374 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab0402c070 con 0x563cf01205d0 2024-05-03T18:44:35.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.568+0000 7faaf7e00700 1 -- 172.21.15.33:0/232752374 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fab040267f0 con 0x563cf01205d0 2024-05-03T18:44:35.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.569+0000 7fab23b76b80 10 client.16869 fetch_fsmap learned FSMap version 21 2024-05-03T18:44:35.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.569+0000 7fab23b76b80 1 -- 172.21.15.33:0/232752374 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x563cf01a4c90 con 0x563cf01205d0 2024-05-03T18:44:35.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.570+0000 7faaf7e00700 1 -- 172.21.15.33:0/232752374 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fab0402c070 con 0x563cf01205d0 2024-05-03T18:44:35.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.570+0000 7faaf7e00700 1 -- 172.21.15.33:0/232752374 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fab04004780 con 0x563cf01205d0 2024-05-03T18:44:35.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.570+0000 7fab23b76b80 10 client.16869 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:44:35.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.570+0000 7fab23b76b80 10 client.16869 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:44:35.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.570+0000 7fab23b76b80 20 client.16869 populate_metadata read hostname 'smithi033' 2024-05-03T18:44:35.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.570+0000 7fab23b76b80 1 --2- 172.21.15.33:0/232752374 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x563cf01a5240 0x563cf01a7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:35.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.570+0000 7fab23b76b80 4 client.16869 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-03T18:44:35.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.570+0000 7fab23b76b80 1 -- 172.21.15.33:0/232752374 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563cf0080400 con 0x563cf01a5240 2024-05-03T18:44:35.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.571+0000 7fab11400700 1 --2- 172.21.15.33:0/232752374 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x563cf01a5240 0x563cf01a7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:35.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.574+0000 7fab11400700 1 --2- 172.21.15.33:0/232752374 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x563cf01a5240 0x563cf01a7660 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3388bc4ddde310b5 server_cookie=5f9e85f842624c03 in_seq=0 out_seq=0 2024-05-03T18:44:35.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.575+0000 7faaf7e00700 10 client.16869 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:44:35.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.576+0000 7faaf7e00700 1 -- 172.21.15.33:0/232752374 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563cf0080400 con 0x563cf01a5240 2024-05-03T18:44:35.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.576+0000 7faaf7e00700 10 client.16869 handle_command_reply: tid=0 2024-05-03T18:44:35.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.576+0000 7fab23b76b80 1 client.16869 shutdown 2024-05-03T18:44:35.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.576+0000 7fab23b76b80 1 -- 172.21.15.33:0/232752374 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563cf01205d0 msgr2=0x563cf0132150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:35.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.576+0000 7fab23b76b80 1 --2- 172.21.15.33:0/232752374 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563cf01205d0 0x563cf0132150 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fab0400dfe0 tx=0x7fab0401eb00 comp rx=0 tx=0).stop 2024-05-03T18:44:35.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.576+0000 7fab23b76b80 1 -- 172.21.15.33:0/232752374 shutdown_connections 2024-05-03T18:44:35.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.576+0000 7fab23b76b80 1 --2- 172.21.15.33:0/232752374 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x563cf01a5240 0x563cf01a7660 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:35.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.576+0000 7fab23b76b80 1 --2- 172.21.15.33:0/232752374 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563cf01250d0 0x563cf012d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:35.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.576+0000 7fab23b76b80 1 --2- 172.21.15.33:0/232752374 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563cf01205d0 0x563cf0132150 secure :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fab0400dfe0 tx=0x7fab0401eb00 comp rx=0 tx=0).stop 2024-05-03T18:44:35.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.576+0000 7fab23b76b80 1 --2- 172.21.15.33:0/232752374 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563cf01247b0 0x563cf0132690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:35.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.576+0000 7fab23b76b80 1 -- 172.21.15.33:0/232752374 >> 172.21.15.33:0/232752374 conn(0x563cf004eb60 msgr2=0x563cf01233f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:44:35.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.576+0000 7faaf7e00700 0 client.16869 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:44:35.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.578+0000 7fab23b76b80 1 -- 172.21.15.33:0/232752374 shutdown_connections 2024-05-03T18:44:35.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.579+0000 7fab23b76b80 1 -- 172.21.15.33:0/232752374 wait complete. 2024-05-03T18:44:35.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.579+0000 7fab23b76b80 20 client.16869 trim_cache size 0 max 16384 2024-05-03T18:44:35.582 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-03T18:44:35.583 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-05-03T18:44:35.583 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-05-03T18:44:35.583 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-03T18:44:35.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.583+0000 7fab19200700 1 -- 172.21.15.33:0/2175061934 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7faaf80b2b40 msgr2=0x7faaf80b5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:35.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.583+0000 7fab19200700 1 --2- 172.21.15.33:0/2175061934 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7faaf80b2b40 0x7faaf80b5000 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fab0400baa0 tx=0x7fab04005db0 comp rx=0 tx=0).stop 2024-05-03T18:44:35.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.583+0000 7fab19200700 1 -- 172.21.15.33:0/2175061934 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab14109510 msgr2=0x7fab1410f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:35.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.583+0000 7fab19200700 1 --2- 172.21.15.33:0/2175061934 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab14109510 0x7fab1410f530 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fab0c00b6c0 tx=0x7fab0c009510 comp rx=0 tx=0).stop 2024-05-03T18:44:35.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.583+0000 7fab19200700 1 -- 172.21.15.33:0/2175061934 shutdown_connections 2024-05-03T18:44:35.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.583+0000 7fab19200700 1 --2- 172.21.15.33:0/2175061934 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7faaf80b2b40 0x7faaf80b5000 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:35.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.583+0000 7fab19200700 1 --2- 172.21.15.33:0/2175061934 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab14109510 0x7fab1410f530 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:35.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.584+0000 7fab19200700 1 --2- 172.21.15.33:0/2175061934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab1406b180 0x7fab141a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:35.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.584+0000 7fab19200700 1 --2- 172.21.15.33:0/2175061934 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab1406bb40 0x7fab141a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:35.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.584+0000 7fab19200700 1 -- 172.21.15.33:0/2175061934 >> 172.21.15.33:0/2175061934 conn(0x7fab14079350 msgr2=0x7fab14100e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:44:35.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.584+0000 7fab19200700 1 -- 172.21.15.33:0/2175061934 shutdown_connections 2024-05-03T18:44:35.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:35.584+0000 7fab19200700 1 -- 172.21.15.33:0/2175061934 wait complete. 2024-05-03T18:44:36.047 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:44:35 smithi099 ceph-mon[31658]: pgmap v548: 129 pgs: 129 active+clean; 3.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 8.6 MiB/s wr, 638 op/s 2024-05-03T18:44:36.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:44:35 smithi150 ceph-mon[32492]: pgmap v548: 129 pgs: 129 active+clean; 3.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 8.6 MiB/s wr, 638 op/s 2024-05-03T18:44:36.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:44:35 smithi033 ceph-mon[28330]: pgmap v548: 129 pgs: 129 active+clean; 3.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 8.6 MiB/s wr, 638 op/s 2024-05-03T18:44:37.602 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-03T18:44:37.602 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T18:44:37.789 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:44:37 smithi033 ceph-mon[28330]: pgmap v549: 129 pgs: 129 active+clean; 3.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 389 KiB/s rd, 7.6 MiB/s wr, 590 op/s 2024-05-03T18:44:38.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:44:37 smithi150 ceph-mon[32492]: pgmap v549: 129 pgs: 129 active+clean; 3.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 389 KiB/s rd, 7.6 MiB/s wr, 590 op/s 2024-05-03T18:44:38.199 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:44:37 smithi099 ceph-mon[31658]: pgmap v549: 129 pgs: 129 active+clean; 3.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 389 KiB/s rd, 7.6 MiB/s wr, 590 op/s 2024-05-03T18:44:38.418 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:44:40.111 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 805.0 s, 436.0 tps, lat 226.870 ms stddev 136.582 2024-05-03T18:44:40.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:44:39 smithi150 ceph-mon[32492]: pgmap v550: 129 pgs: 129 active+clean; 3.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 5.8 MiB/s wr, 461 op/s 2024-05-03T18:44:40.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:44:39 smithi033 ceph-mon[28330]: pgmap v550: 129 pgs: 129 active+clean; 3.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 5.8 MiB/s wr, 461 op/s 2024-05-03T18:44:40.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:44:39 smithi099 ceph-mon[31658]: pgmap v550: 129 pgs: 129 active+clean; 3.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 5.8 MiB/s wr, 461 op/s 2024-05-03T18:44:41.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:41.112+0000 7fc2efa00700 1 -- 172.21.15.33:0/1016744281 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2e8102d60 msgr2=0x7fc2e810f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:41.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:41.112+0000 7fc2efa00700 1 --2- 172.21.15.33:0/1016744281 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2e8102d60 0x7fc2e810f250 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc2d0009990 tx=0x7fc2d0009ca0 comp rx=0 tx=0).stop 2024-05-03T18:44:41.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:41.112+0000 7fc2efa00700 1 -- 172.21.15.33:0/1016744281 shutdown_connections 2024-05-03T18:44:41.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:41.112+0000 7fc2efa00700 1 --2- 172.21.15.33:0/1016744281 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2e8102d60 0x7fc2e810f250 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:41.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:41.112+0000 7fc2efa00700 1 --2- 172.21.15.33:0/1016744281 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc2e8108410 0x7fc2e81087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:41.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:41.112+0000 7fc2efa00700 1 --2- 172.21.15.33:0/1016744281 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2e81023a0 0x7fc2e8102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:41.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:41.112+0000 7fc2efa00700 1 -- 172.21.15.33:0/1016744281 >> 172.21.15.33:0/1016744281 conn(0x7fc2e8079350 msgr2=0x7fc2e8079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:44:41.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:41.113+0000 7fc2efa00700 1 -- 172.21.15.33:0/1016744281 shutdown_connections 2024-05-03T18:44:41.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:41.113+0000 7fc2efa00700 1 -- 172.21.15.33:0/1016744281 wait complete. 2024-05-03T18:44:41.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:41.114+0000 7fc2efa00700 1 Processor -- start 2024-05-03T18:44:41.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:41.115+0000 7fc2efa00700 1 -- start start 2024-05-03T18:44:41.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:41.115+0000 7fc2efa00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2e81023a0 0x7fc2e81a1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:41.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:41.115+0000 7fc2efa00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2e8102d60 0x7fc2e81a1e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:41.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:41.115+0000 7fc2efa00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc2e8108410 0x7fc2e819bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:41.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:41.115+0000 7fc2efa00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc2e8112480 con 0x7fc2e81023a0 2024-05-03T18:44:41.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:41.115+0000 7fc2efa00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc2e8112300 con 0x7fc2e8102d60 2024-05-03T18:44:41.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:41.115+0000 7fc2efa00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc2e8112600 con 0x7fc2e8108410 2024-05-03T18:44:41.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:41.116+0000 7fc2eda00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc2e8108410 0x7fc2e819bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:41.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:41.116+0000 7fc2eda00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc2e8108410 0x7fc2e819bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:43022/0 (socket says 172.21.15.33:43022) 2024-05-03T18:44:41.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:41.116+0000 7fc2eda00700 1 -- 172.21.15.33:0/1649581908 learned_addr learned my addr 172.21.15.33:0/1649581908 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:44:41.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:41.116+0000 7fc2eda00700 1 -- 172.21.15.33:0/1649581908 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2e8102d60 msgr2=0x7fc2e81a1e40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-03T18:44:41.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:41.116+0000 7fc2eda00700 1 --2- 172.21.15.33:0/1649581908 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2e8102d60 0x7fc2e81a1e40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:41.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:41.116+0000 7fc2eda00700 1 -- 172.21.15.33:0/1649581908 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2e81023a0 msgr2=0x7fc2e81a1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:41.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:41.116+0000 7fc2eda00700 1 --2- 172.21.15.33:0/1649581908 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2e81023a0 0x7fc2e81a1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:41.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:41.116+0000 7fc2eda00700 1 -- 172.21.15.33:0/1649581908 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2e819c220 con 0x7fc2e8108410 2024-05-03T18:44:41.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:41.117+0000 7fc2eda00700 1 --2- 172.21.15.33:0/1649581908 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc2e8108410 0x7fc2e819bac0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fc2d00057a0 tx=0x7fc2d0005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:44:41.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:41.117+0000 7fc2e5600700 1 -- 172.21.15.33:0/1649581908 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2d000fb00 con 0x7fc2e8108410 2024-05-03T18:44:41.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:41.117+0000 7fc2efa00700 1 -- 172.21.15.33:0/1649581908 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2e819c4b0 con 0x7fc2e8108410 2024-05-03T18:44:41.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:41.117+0000 7fc2e5600700 1 -- 172.21.15.33:0/1649581908 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc2d000fca0 con 0x7fc2e8108410 2024-05-03T18:44:41.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:41.117+0000 7fc2e5600700 1 -- 172.21.15.33:0/1649581908 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2d0017650 con 0x7fc2e8108410 2024-05-03T18:44:41.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:41.118+0000 7fc2efa00700 1 -- 172.21.15.33:0/1649581908 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2e81a87d0 con 0x7fc2e8108410 2024-05-03T18:44:41.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:41.120+0000 7fc2efa00700 1 -- 172.21.15.33:0/1649581908 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2e806a190 con 0x7fc2e8108410 2024-05-03T18:44:41.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:41.125+0000 7fc2e5600700 1 -- 172.21.15.33:0/1649581908 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fc2d0009d10 con 0x7fc2e8108410 2024-05-03T18:44:41.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:41.128+0000 7fc2e5600700 1 --2- 172.21.15.33:0/1649581908 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fc2cc0b2a70 0x7fc2cc0b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:41.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:41.128+0000 7fc2e5600700 1 -- 172.21.15.33:0/1649581908 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 9518+0+0 (secure 0 0 0) 0x7fc2d00e3970 con 0x7fc2e8108410 2024-05-03T18:44:41.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:41.129+0000 7fc2e5600700 1 -- 172.21.15.33:0/1649581908 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fc2d0013070 con 0x7fc2e8108410 2024-05-03T18:44:41.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:41.129+0000 7fc2ed000700 1 --2- 172.21.15.33:0/1649581908 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fc2cc0b2a70 0x7fc2cc0b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:41.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:41.130+0000 7fc2ed000700 1 --2- 172.21.15.33:0/1649581908 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fc2cc0b2a70 0x7fc2cc0b4f30 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc2e819cc00 tx=0x7fc2e0009250 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:44:41.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:41.678+0000 7fc2efa00700 1 -- 172.21.15.33:0/1649581908 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc2e81a89a0 con 0x7fc2e8108410 2024-05-03T18:44:41.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:41.680+0000 7fc2e5600700 1 -- 172.21.15.33:0/1649581908 <== mon.1 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11801 (secure 0 0 0) 0x7fc2d00aafa0 con 0x7fc2e8108410 2024-05-03T18:44:41.682 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:44:41.683 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1327393503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1327393503},{"type":"v1","addr":"172.21.15.33:6835","nonce":1327393503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/2461472948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":2461472948},{"type":"v1","addr":"172.21.15.33:6837","nonce":2461472948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14559,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/442269281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":442269281},{"type":"v1","addr":"172.21.15.33:6839","nonce":442269281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14571,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/645571756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":645571756},{"type":"v1","addr":"172.21.15.33:6841","nonce":645571756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/203598909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":203598909},{"type":"v1","addr":"172.21.15.150:6833","nonce":203598909}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/4029842493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":4029842493},{"type":"v1","addr":"172.21.15.150:6835","nonce":4029842493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3463664423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3463664423},{"type":"v1","addr":"172.21.15.150:6837","nonce":3463664423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24481,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6839/4051648018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":4051648018},{"type":"v1","addr":"172.21.15.150:6839","nonce":4051648018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34418,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1865105479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1865105479},{"type":"v1","addr":"172.21.15.99:6833","nonce":1865105479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34445,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/1206393914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1206393914},{"type":"v1","addr":"172.21.15.99:6835","nonce":1206393914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-03T18:22:28.837091+0000","modified":"2024-05-03T18:23:49.758398+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34508},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34472":{"gid":34472,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6837/3360970342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3360970342},{"type":"v1","addr":"172.21.15.99:6837","nonce":3360970342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34508":{"gid":34508,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/919907308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":919907308},{"type":"v1","addr":"172.21.15.99:6839","nonce":919907308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34508,"qdb_cluster":[34508]},"id":1}]} 2024-05-03T18:44:41.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:41.689+0000 7fc2efa00700 1 -- 172.21.15.33:0/1649581908 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fc2cc0b2a70 msgr2=0x7fc2cc0b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:41.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:41.689+0000 7fc2efa00700 1 --2- 172.21.15.33:0/1649581908 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fc2cc0b2a70 0x7fc2cc0b4f30 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc2e819cc00 tx=0x7fc2e0009250 comp rx=0 tx=0).stop 2024-05-03T18:44:41.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:41.689+0000 7fc2efa00700 1 -- 172.21.15.33:0/1649581908 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc2e8108410 msgr2=0x7fc2e819bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:41.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:41.689+0000 7fc2efa00700 1 --2- 172.21.15.33:0/1649581908 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc2e8108410 0x7fc2e819bac0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fc2d00057a0 tx=0x7fc2d0005e80 comp rx=0 tx=0).stop 2024-05-03T18:44:41.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:41.689+0000 7fc2efa00700 1 -- 172.21.15.33:0/1649581908 shutdown_connections 2024-05-03T18:44:41.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:41.689+0000 7fc2efa00700 1 --2- 172.21.15.33:0/1649581908 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc2e8108410 0x7fc2e819bac0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:41.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:41.689+0000 7fc2efa00700 1 --2- 172.21.15.33:0/1649581908 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2e81023a0 0x7fc2e81a1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:41.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:41.689+0000 7fc2efa00700 1 --2- 172.21.15.33:0/1649581908 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fc2cc0b2a70 0x7fc2cc0b4f30 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:41.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:41.689+0000 7fc2efa00700 1 --2- 172.21.15.33:0/1649581908 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2e8102d60 0x7fc2e81a1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:41.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:41.689+0000 7fc2efa00700 1 -- 172.21.15.33:0/1649581908 >> 172.21.15.33:0/1649581908 conn(0x7fc2e8079350 msgr2=0x7fc2e8100d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:44:41.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:41.690+0000 7fc2efa00700 1 -- 172.21.15.33:0/1649581908 shutdown_connections 2024-05-03T18:44:41.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:41.690+0000 7fc2efa00700 1 -- 172.21.15.33:0/1649581908 wait complete. 2024-05-03T18:44:41.697 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 21 2024-05-03T18:44:42.134 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:44:41 smithi150 ceph-mon[32492]: pgmap v551: 129 pgs: 129 active+clean; 3.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 8.8 MiB/s wr, 695 op/s 2024-05-03T18:44:42.135 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:44:41 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/1649581908' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:44:42.154 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:44:41 smithi033 ceph-mon[28330]: pgmap v551: 129 pgs: 129 active+clean; 3.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 8.8 MiB/s wr, 695 op/s 2024-05-03T18:44:42.154 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:44:41 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/1649581908' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:44:42.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:44:41 smithi099 ceph-mon[31658]: pgmap v551: 129 pgs: 129 active+clean; 3.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 8.8 MiB/s wr, 695 op/s 2024-05-03T18:44:42.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:44:41 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/1649581908' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:44:42.582 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-03T18:44:43.633 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:44:44.074 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:44:43 smithi033 ceph-mon[28330]: pgmap v552: 129 pgs: 129 active+clean; 3.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 5.9 MiB/s wr, 455 op/s 2024-05-03T18:44:44.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:44:43 smithi150 ceph-mon[32492]: pgmap v552: 129 pgs: 129 active+clean; 3.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 5.9 MiB/s wr, 455 op/s 2024-05-03T18:44:44.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:44:43 smithi099 ceph-mon[31658]: pgmap v552: 129 pgs: 129 active+clean; 3.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 5.9 MiB/s wr, 455 op/s 2024-05-03T18:44:45.111 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 810.0 s, 592.2 tps, lat 172.047 ms stddev 77.937 2024-05-03T18:44:46.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:44:45 smithi150 ceph-mon[32492]: pgmap v553: 129 pgs: 129 active+clean; 3.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 528 KiB/s rd, 7.6 MiB/s wr, 578 op/s 2024-05-03T18:44:46.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.170+0000 7fe004c00700 1 -- 172.21.15.33:0/4081790689 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe000102530 msgr2=0x7fe000102910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:46.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.170+0000 7fe004c00700 1 --2- 172.21.15.33:0/4081790689 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe000102530 0x7fe000102910 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fdfe8009990 tx=0x7fdfe8009ca0 comp rx=0 tx=0).stop 2024-05-03T18:44:46.173 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:44:45 smithi033 ceph-mon[28330]: pgmap v553: 129 pgs: 129 active+clean; 3.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 528 KiB/s rd, 7.6 MiB/s wr, 578 op/s 2024-05-03T18:44:46.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.170+0000 7fe004c00700 1 -- 172.21.15.33:0/4081790689 shutdown_connections 2024-05-03T18:44:46.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.171+0000 7fe004c00700 1 --2- 172.21.15.33:0/4081790689 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe000107610 0x7fe00010f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:46.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.171+0000 7fe004c00700 1 --2- 172.21.15.33:0/4081790689 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe000102530 0x7fe000102910 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:46.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.171+0000 7fe004c00700 1 --2- 172.21.15.33:0/4081790689 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe000102ee0 0x7fe000106f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:46.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.171+0000 7fe004c00700 1 -- 172.21.15.33:0/4081790689 >> 172.21.15.33:0/4081790689 conn(0x7fe0000fddc0 msgr2=0x7fe0001001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:44:46.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.171+0000 7fe004c00700 1 -- 172.21.15.33:0/4081790689 shutdown_connections 2024-05-03T18:44:46.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.171+0000 7fe004c00700 1 -- 172.21.15.33:0/4081790689 wait complete. 2024-05-03T18:44:46.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.173+0000 7fe004c00700 1 Processor -- start 2024-05-03T18:44:46.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.173+0000 7fe004c00700 1 -- start start 2024-05-03T18:44:46.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.173+0000 7fe004c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe000102530 0x7fe000074d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:46.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.174+0000 7fe004c00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe000102ee0 0x7fe00006de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:46.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.174+0000 7fe004c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe000107610 0x7fe00006e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:46.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.174+0000 7fe004c00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe000113500 con 0x7fe000102ee0 2024-05-03T18:44:46.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.174+0000 7fe004c00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe000113380 con 0x7fe000102530 2024-05-03T18:44:46.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.174+0000 7fe004c00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe000113680 con 0x7fe000107610 2024-05-03T18:44:46.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.174+0000 7fdfff400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe000102530 0x7fe000074d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:46.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.174+0000 7fdffea00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe000102ee0 0x7fe00006de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:46.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.174+0000 7fdfff400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe000102530 0x7fe000074d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:59916/0 (socket says 172.21.15.33:59916) 2024-05-03T18:44:46.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.174+0000 7fdfff400700 1 -- 172.21.15.33:0/3542680353 learned_addr learned my addr 172.21.15.33:0/3542680353 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:44:46.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.174+0000 7fdffea00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe000102ee0 0x7fe00006de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49780/0 (socket says 172.21.15.33:49780) 2024-05-03T18:44:46.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.174+0000 7fdfffe00700 1 --2- 172.21.15.33:0/3542680353 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe000107610 0x7fe00006e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:46.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.176+0000 7fdfffe00700 1 -- 172.21.15.33:0/3542680353 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe000102530 msgr2=0x7fe000074d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:46.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.176+0000 7fdfffe00700 1 --2- 172.21.15.33:0/3542680353 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe000102530 0x7fe000074d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:46.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.176+0000 7fdfffe00700 1 -- 172.21.15.33:0/3542680353 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe000102ee0 msgr2=0x7fe00006de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:46.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.176+0000 7fdfffe00700 1 --2- 172.21.15.33:0/3542680353 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe000102ee0 0x7fe00006de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:46.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.176+0000 7fdfffe00700 1 -- 172.21.15.33:0/3542680353 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe00006eab0 con 0x7fe000107610 2024-05-03T18:44:46.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.177+0000 7fdffea00700 1 --2- 172.21.15.33:0/3542680353 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe000102ee0 0x7fe00006de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:44:46.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.177+0000 7fdfff400700 1 --2- 172.21.15.33:0/3542680353 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe000102530 0x7fe000074d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:44:46.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.177+0000 7fdfffe00700 1 --2- 172.21.15.33:0/3542680353 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe000107610 0x7fe00006e380 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fdff800b740 tx=0x7fdff800ba50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:44:46.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.178+0000 7fdff7e00700 1 -- 172.21.15.33:0/3542680353 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff80092e0 con 0x7fe000107610 2024-05-03T18:44:46.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.178+0000 7fdff7e00700 1 -- 172.21.15.33:0/3542680353 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdff8004bb0 con 0x7fe000107610 2024-05-03T18:44:46.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.178+0000 7fe004c00700 1 -- 172.21.15.33:0/3542680353 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe000072890 con 0x7fe000107610 2024-05-03T18:44:46.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.178+0000 7fe004c00700 1 -- 172.21.15.33:0/3542680353 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe000072e20 con 0x7fe000107610 2024-05-03T18:44:46.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.179+0000 7fdff7e00700 1 -- 172.21.15.33:0/3542680353 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff801d670 con 0x7fe000107610 2024-05-03T18:44:46.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.185+0000 7fe010ae2b80 1 Processor -- start 2024-05-03T18:44:46.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.189+0000 7fdff7e00700 1 -- 172.21.15.33:0/3542680353 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fdff8024020 con 0x7fe000107610 2024-05-03T18:44:46.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.191+0000 7fe010ae2b80 1 -- start start 2024-05-03T18:44:46.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.193+0000 7fe010ae2b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561c5d39ebd0 0x561c5d39efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:46.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.193+0000 7fe010ae2b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561c5d398f30 0x561c5d3993b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:46.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.193+0000 7fe010ae2b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561c5d3998f0 0x561c5d3a5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:46.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.193+0000 7fe010ae2b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x561c5d2eba20 con 0x561c5d3998f0 2024-05-03T18:44:46.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.193+0000 7fe010ae2b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x561c5d289760 con 0x561c5d398f30 2024-05-03T18:44:46.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.193+0000 7fe010ae2b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x561c5d2e79e0 con 0x561c5d39ebd0 2024-05-03T18:44:46.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.194+0000 7fdffea00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561c5d39ebd0 0x561c5d39efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:46.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.194+0000 7fdffea00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561c5d39ebd0 0x561c5d39efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:44858/0 (socket says 172.21.15.33:44858) 2024-05-03T18:44:46.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.194+0000 7fdffea00700 1 -- 172.21.15.33:0/378643041 learned_addr learned my addr 172.21.15.33:0/378643041 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:44:46.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.194+0000 7fdfff400700 1 --2- 172.21.15.33:0/378643041 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561c5d398f30 0x561c5d3993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:46.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.195+0000 7fdffea00700 1 --2- 172.21.15.33:0/378643041 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561c5d3998f0 0x561c5d3a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:46.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.195+0000 7fdff7e00700 1 --2- 172.21.15.33:0/3542680353 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fdfe40b2a70 0x7fdfe40b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:46.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.196+0000 7fdff7e00700 1 -- 172.21.15.33:0/3542680353 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 9518+0+0 (secure 0 0 0) 0x7fdff80ea320 con 0x7fe000107610 2024-05-03T18:44:46.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.196+0000 7fdfffe00700 1 --2- 172.21.15.33:0/3542680353 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fdfe40b2a70 0x7fdfe40b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:46.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.196+0000 7fdfff400700 1 -- 172.21.15.33:0/378643041 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561c5d39ebd0 msgr2=0x561c5d39efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:44:46.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.196+0000 7fdfff400700 1 --2- 172.21.15.33:0/378643041 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561c5d39ebd0 0x561c5d39efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:46.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.196+0000 7fdfff400700 1 -- 172.21.15.33:0/378643041 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561c5d3998f0 msgr2=0x561c5d3a5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:44:46.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.196+0000 7fdfff400700 1 --2- 172.21.15.33:0/378643041 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561c5d3998f0 0x561c5d3a5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:46.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.196+0000 7fdfff400700 1 -- 172.21.15.33:0/378643041 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c5d2388d0 con 0x561c5d398f30 2024-05-03T18:44:46.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.197+0000 7fdffea00700 1 --2- 172.21.15.33:0/378643041 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561c5d39ebd0 0x561c5d39efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:44:46.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.198+0000 7fdfffe00700 1 --2- 172.21.15.33:0/3542680353 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fdfe40b2a70 0x7fdfe40b4f30 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fdff8007d30 tx=0x7fdff8000f40 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:44:46.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.198+0000 7fdfff400700 1 --2- 172.21.15.33:0/378643041 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561c5d398f30 0x561c5d3993b0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fdfe8000c00 tx=0x7fdfe800b2a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=976762ac2b633236 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:44:46.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.199+0000 7fdff4c00700 1 -- 172.21.15.33:0/378643041 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe801c330 con 0x561c5d398f30 2024-05-03T18:44:46.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.199+0000 7fdff4c00700 1 -- 172.21.15.33:0/378643041 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdfe801c4d0 con 0x561c5d398f30 2024-05-03T18:44:46.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.199+0000 7fdff4c00700 1 -- 172.21.15.33:0/378643041 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe800bbc0 con 0x561c5d398f30 2024-05-03T18:44:46.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.200+0000 7fe010ae2b80 1 -- 172.21.15.33:0/378643041 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561c5d398f30 msgr2=0x561c5d3993b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:46.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.200+0000 7fe010ae2b80 1 --2- 172.21.15.33:0/378643041 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561c5d398f30 0x561c5d3993b0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fdfe8000c00 tx=0x7fdfe800b2a0 comp rx=0 tx=0).stop 2024-05-03T18:44:46.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.201+0000 7fe010ae2b80 1 -- 172.21.15.33:0/378643041 shutdown_connections 2024-05-03T18:44:46.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.201+0000 7fe010ae2b80 1 --2- 172.21.15.33:0/378643041 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561c5d3998f0 0x561c5d3a5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:46.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.201+0000 7fe010ae2b80 1 --2- 172.21.15.33:0/378643041 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561c5d39ebd0 0x561c5d39efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:46.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.201+0000 7fe010ae2b80 1 --2- 172.21.15.33:0/378643041 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561c5d398f30 0x561c5d3993b0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:46.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.201+0000 7fe010ae2b80 1 -- 172.21.15.33:0/378643041 >> 172.21.15.33:0/378643041 conn(0x561c5d3116d0 msgr2=0x561c5d393bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:44:46.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.201+0000 7fe010ae2b80 1 -- 172.21.15.33:0/378643041 shutdown_connections 2024-05-03T18:44:46.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.201+0000 7fe010ae2b80 1 -- 172.21.15.33:0/378643041 wait complete. 2024-05-03T18:44:46.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.203+0000 7fe010ae2b80 1 -- start start 2024-05-03T18:44:46.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.203+0000 7fe010ae2b80 1 Processor -- start 2024-05-03T18:44:46.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.204+0000 7fe010ae2b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561c5d398f30 0x561c5d3a55a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:46.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.204+0000 7fe010ae2b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561c5d3998f0 0x561c5d3a2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:46.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.205+0000 7fdffea00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561c5d398f30 0x561c5d3a55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:46.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.205+0000 7fe010ae2b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561c5d39ebd0 0x561c5d3a2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:46.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.205+0000 7fdffea00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561c5d398f30 0x561c5d3a55a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:44872/0 (socket says 172.21.15.33:44872) 2024-05-03T18:44:46.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.205+0000 7fe010ae2b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x561c5d3a5ed0 con 0x561c5d3998f0 2024-05-03T18:44:46.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.205+0000 7fe010ae2b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x561c5d251b20 con 0x561c5d39ebd0 2024-05-03T18:44:46.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.205+0000 7fe010ae2b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x561c5d3a6050 con 0x561c5d398f30 2024-05-03T18:44:46.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.205+0000 7fe010ae2b80 10 client.0 _ll_register_callbacks cb 0x561c5d292c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:44:46.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.205+0000 7fdffea00700 1 -- 172.21.15.33:0/3559738365 learned_addr learned my addr 172.21.15.33:0/3559738365 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:44:46.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.206+0000 7fdffea00700 1 --2- 172.21.15.33:0/3559738365 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561c5d39ebd0 0x561c5d3a2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:46.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.207+0000 7fdffea00700 1 -- 172.21.15.33:0/3559738365 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561c5d39ebd0 msgr2=0x561c5d3a2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:44:46.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.207+0000 7fdffea00700 1 --2- 172.21.15.33:0/3559738365 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561c5d39ebd0 0x561c5d3a2e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:46.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.207+0000 7fdffea00700 1 -- 172.21.15.33:0/3559738365 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561c5d3998f0 msgr2=0x561c5d3a2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:44:46.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.207+0000 7fdffea00700 1 --2- 172.21.15.33:0/3559738365 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561c5d3998f0 0x561c5d3a2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:46.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.207+0000 7fdffea00700 1 -- 172.21.15.33:0/3559738365 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c5d2b1420 con 0x561c5d398f30 2024-05-03T18:44:46.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.210+0000 7fdffea00700 1 --2- 172.21.15.33:0/3559738365 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561c5d398f30 0x561c5d3a55a0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fdff0003ca0 tx=0x7fdff0009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5eb176642f1b375 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:44:46.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.211+0000 7fdfed600700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-03T18:44:46.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.211+0000 7fdfed600700 1 -- 172.21.15.33:0/3559738365 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff0003e40 con 0x561c5d398f30 2024-05-03T18:44:46.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.211+0000 7fe010ae2b80 1 -- 172.21.15.33:0/3559738365 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdfe801c4d0 con 0x561c5d398f30 2024-05-03T18:44:46.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.212+0000 7fdfed600700 1 -- 172.21.15.33:0/3559738365 <== mon.1 v2:172.21.15.150:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9518+0+0 (secure 0 0 0) 0x7fdff001a040 con 0x561c5d398f30 2024-05-03T18:44:46.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.212+0000 7fdfed600700 1 -- 172.21.15.33:0/3559738365 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fdff0003e40 con 0x561c5d398f30 2024-05-03T18:44:46.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.212+0000 7fe010ae2b80 10 client.26752 fetch_fsmap learned FSMap version 21 2024-05-03T18:44:46.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.212+0000 7fe010ae2b80 1 -- 172.21.15.33:0/3559738365 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x561c5d3b07d0 con 0x561c5d398f30 2024-05-03T18:44:46.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.213+0000 7fdfed600700 1 -- 172.21.15.33:0/3559738365 <== mon.1 v2:172.21.15.150:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdfe801c4d0 con 0x561c5d398f30 2024-05-03T18:44:46.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.214+0000 7fdfed600700 1 -- 172.21.15.33:0/3559738365 <== mon.1 v2:172.21.15.150:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fdff0020470 con 0x561c5d398f30 2024-05-03T18:44:46.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.214+0000 7fe010ae2b80 10 client.26752 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:44:46.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.214+0000 7fe010ae2b80 10 client.26752 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:44:46.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.214+0000 7fe010ae2b80 20 client.26752 populate_metadata read hostname 'smithi033' 2024-05-03T18:44:46.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.214+0000 7fe010ae2b80 1 --2- 172.21.15.33:0/3559738365 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x561c5d3b2d60 0x561c5d3b5150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:46.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.214+0000 7fe010ae2b80 4 client.26752 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:44:46.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.214+0000 7fe010ae2b80 1 -- 172.21.15.33:0/3559738365 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561c5d365ff0 con 0x561c5d3b2d60 2024-05-03T18:44:46.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.215+0000 7fdfff400700 1 --2- 172.21.15.33:0/3559738365 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x561c5d3b2d60 0x561c5d3b5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:46.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.216+0000 7fdfff400700 1 --2- 172.21.15.33:0/3559738365 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x561c5d3b2d60 0x561c5d3b5150 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b27b3770a50127e server_cookie=8db57990d3acb3d1 in_seq=0 out_seq=0 2024-05-03T18:44:46.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.217+0000 7fdfed600700 10 client.26752 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:44:46.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.218+0000 7fdfed600700 1 -- 172.21.15.33:0/3559738365 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561c5d365ff0 con 0x561c5d3b2d60 2024-05-03T18:44:46.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.218+0000 7fdfed600700 10 client.26752 handle_command_reply: tid=0 2024-05-03T18:44:46.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.218+0000 7fe010ae2b80 1 client.26752 shutdown 2024-05-03T18:44:46.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.220+0000 7fe010ae2b80 1 -- 172.21.15.33:0/3559738365 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561c5d398f30 msgr2=0x561c5d3a55a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:46.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.220+0000 7fe010ae2b80 1 --2- 172.21.15.33:0/3559738365 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561c5d398f30 0x561c5d3a55a0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fdff0003ca0 tx=0x7fdff0009260 comp rx=0 tx=0).stop 2024-05-03T18:44:46.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.220+0000 7fe010ae2b80 1 -- 172.21.15.33:0/3559738365 shutdown_connections 2024-05-03T18:44:46.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.220+0000 7fe010ae2b80 1 --2- 172.21.15.33:0/3559738365 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x561c5d3b2d60 0x561c5d3b5150 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:46.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.220+0000 7fe010ae2b80 1 --2- 172.21.15.33:0/3559738365 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561c5d39ebd0 0x561c5d3a2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:46.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.220+0000 7fe010ae2b80 1 --2- 172.21.15.33:0/3559738365 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561c5d398f30 0x561c5d3a55a0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:46.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.220+0000 7fe010ae2b80 1 --2- 172.21.15.33:0/3559738365 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561c5d3998f0 0x561c5d3a2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:46.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.220+0000 7fdfed600700 0 client.26752 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:44:46.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.220+0000 7fe010ae2b80 1 -- 172.21.15.33:0/3559738365 >> 172.21.15.33:0/3559738365 conn(0x561c5d3116d0 msgr2=0x561c5d397f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:44:46.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.223+0000 7fe010ae2b80 1 -- 172.21.15.33:0/3559738365 shutdown_connections 2024-05-03T18:44:46.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.223+0000 7fe010ae2b80 1 -- 172.21.15.33:0/3559738365 wait complete. 2024-05-03T18:44:46.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.223+0000 7fe010ae2b80 20 client.26752 trim_cache size 0 max 16384 2024-05-03T18:44:46.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.255+0000 7fe010ae2b80 1 Processor -- start 2024-05-03T18:44:46.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.255+0000 7fe010ae2b80 1 -- start start 2024-05-03T18:44:46.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.256+0000 7fe010ae2b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561c5d398f30 0x561c5d3aef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:46.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.256+0000 7fe010ae2b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561c5d3998f0 0x561c5d3baa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:46.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.256+0000 7fe010ae2b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561c5d39ebd0 0x561c5d3bafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:46.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.256+0000 7fe010ae2b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fdfdc00ce50 con 0x561c5d398f30 2024-05-03T18:44:46.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.256+0000 7fe010ae2b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fdfdc00f6b0 con 0x561c5d3998f0 2024-05-03T18:44:46.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.256+0000 7fe010ae2b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fdfdc00d4a0 con 0x561c5d39ebd0 2024-05-03T18:44:46.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.257+0000 7fdfff400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561c5d3998f0 0x561c5d3baa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:46.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.257+0000 7fdfff400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561c5d3998f0 0x561c5d3baa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:59942/0 (socket says 172.21.15.33:59942) 2024-05-03T18:44:46.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.257+0000 7fdfff400700 1 -- 172.21.15.33:0/4041061630 learned_addr learned my addr 172.21.15.33:0/4041061630 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:44:46.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.257+0000 7fdffea00700 1 --2- 172.21.15.33:0/4041061630 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561c5d398f30 0x561c5d3aef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:46.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.257+0000 7fdffea00700 1 --2- 172.21.15.33:0/4041061630 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561c5d39ebd0 0x561c5d3bafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:46.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.257+0000 7fdfff400700 1 -- 172.21.15.33:0/4041061630 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561c5d39ebd0 msgr2=0x561c5d3bafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:44:46.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.257+0000 7fdfff400700 1 --2- 172.21.15.33:0/4041061630 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561c5d39ebd0 0x561c5d3bafa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:46.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.257+0000 7fdfff400700 1 -- 172.21.15.33:0/4041061630 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561c5d398f30 msgr2=0x561c5d3aef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:44:46.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.257+0000 7fdfff400700 1 --2- 172.21.15.33:0/4041061630 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561c5d398f30 0x561c5d3aef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:46.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.257+0000 7fdfff400700 1 -- 172.21.15.33:0/4041061630 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c5d365fd0 con 0x561c5d3998f0 2024-05-03T18:44:46.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.258+0000 7fdfff400700 1 --2- 172.21.15.33:0/4041061630 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561c5d3998f0 0x561c5d3baa60 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fdfe800b790 tx=0x7fdfe800b7c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ebe4eea2567f24 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:44:46.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.259+0000 7fdfefe00700 1 -- 172.21.15.33:0/4041061630 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe80050c0 con 0x561c5d3998f0 2024-05-03T18:44:46.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.259+0000 7fdfefe00700 1 -- 172.21.15.33:0/4041061630 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdfe800b830 con 0x561c5d3998f0 2024-05-03T18:44:46.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.259+0000 7fdfefe00700 1 -- 172.21.15.33:0/4041061630 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe802c580 con 0x561c5d3998f0 2024-05-03T18:44:46.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.260+0000 7fe010ae2b80 1 -- 172.21.15.33:0/4041061630 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561c5d3998f0 msgr2=0x561c5d3baa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:46.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.260+0000 7fe010ae2b80 1 --2- 172.21.15.33:0/4041061630 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561c5d3998f0 0x561c5d3baa60 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fdfe800b790 tx=0x7fdfe800b7c0 comp rx=0 tx=0).stop 2024-05-03T18:44:46.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.260+0000 7fe010ae2b80 1 -- 172.21.15.33:0/4041061630 shutdown_connections 2024-05-03T18:44:46.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.260+0000 7fe010ae2b80 1 --2- 172.21.15.33:0/4041061630 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561c5d39ebd0 0x561c5d3bafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:46.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.260+0000 7fe010ae2b80 1 --2- 172.21.15.33:0/4041061630 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561c5d398f30 0x561c5d3aef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:46.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.260+0000 7fe010ae2b80 1 --2- 172.21.15.33:0/4041061630 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561c5d3998f0 0x561c5d3baa60 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:46.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.260+0000 7fe010ae2b80 1 -- 172.21.15.33:0/4041061630 >> 172.21.15.33:0/4041061630 conn(0x561c5d3116d0 msgr2=0x561c5d3a1250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:44:46.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.261+0000 7fe010ae2b80 1 -- 172.21.15.33:0/4041061630 shutdown_connections 2024-05-03T18:44:46.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.261+0000 7fe010ae2b80 1 -- 172.21.15.33:0/4041061630 wait complete. 2024-05-03T18:44:46.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.261+0000 7fe010ae2b80 1 -- start start 2024-05-03T18:44:46.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.262+0000 7fe010ae2b80 1 Processor -- start 2024-05-03T18:44:46.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.263+0000 7fe010ae2b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561c5d398f30 0x561c5d3ab4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:46.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.263+0000 7fe010ae2b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561c5d3998f0 0x561c5d3aba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:46.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.263+0000 7fe010ae2b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561c5d39ebd0 0x561c5d3b9b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:46.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.263+0000 7fe010ae2b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fdfdc00db70 con 0x561c5d39ebd0 2024-05-03T18:44:46.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.263+0000 7fe010ae2b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fdfdc00efe0 con 0x561c5d398f30 2024-05-03T18:44:46.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.263+0000 7fe010ae2b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fdfdc00e240 con 0x561c5d3998f0 2024-05-03T18:44:46.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.263+0000 7fe010ae2b80 10 client.0 _ll_register_callbacks cb 0x561c5d292c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:44:46.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.263+0000 7fdffea00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561c5d39ebd0 0x561c5d3b9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:46.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.263+0000 7fdffea00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561c5d39ebd0 0x561c5d3b9b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49816/0 (socket says 172.21.15.33:49816) 2024-05-03T18:44:46.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.263+0000 7fdffea00700 1 -- 172.21.15.33:0/3558163294 learned_addr learned my addr 172.21.15.33:0/3558163294 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:44:46.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.264+0000 7fdffea00700 1 --2- 172.21.15.33:0/3558163294 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561c5d398f30 0x561c5d3ab4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:46.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.264+0000 7fdfff400700 1 --2- 172.21.15.33:0/3558163294 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561c5d3998f0 0x561c5d3aba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:46.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.264+0000 7fdffea00700 1 -- 172.21.15.33:0/3558163294 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561c5d3998f0 msgr2=0x561c5d3aba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:44:46.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.264+0000 7fdffea00700 1 --2- 172.21.15.33:0/3558163294 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561c5d3998f0 0x561c5d3aba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:46.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.264+0000 7fdffea00700 1 -- 172.21.15.33:0/3558163294 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561c5d398f30 msgr2=0x561c5d3ab4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:44:46.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.264+0000 7fdffea00700 1 --2- 172.21.15.33:0/3558163294 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561c5d398f30 0x561c5d3ab4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:46.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.264+0000 7fdffea00700 1 -- 172.21.15.33:0/3558163294 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c5d3ba350 con 0x561c5d39ebd0 2024-05-03T18:44:46.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.264+0000 7fdfff400700 1 --2- 172.21.15.33:0/3558163294 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561c5d3998f0 0x561c5d3aba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:44:46.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.265+0000 7fdffea00700 1 --2- 172.21.15.33:0/3558163294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561c5d39ebd0 0x561c5d3b9b40 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fdff0004790 tx=0x7fdff00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a0d2290a07d41d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:44:46.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.265+0000 7fdfed600700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:44:46.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.266+0000 7fdfed600700 1 -- 172.21.15.33:0/3558163294 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff002f840 con 0x561c5d39ebd0 2024-05-03T18:44:46.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.266+0000 7fdfed600700 1 -- 172.21.15.33:0/3558163294 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9518+0+0 (secure 0 0 0) 0x7fdff002f9e0 con 0x561c5d39ebd0 2024-05-03T18:44:46.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.266+0000 7fe010ae2b80 1 -- 172.21.15.33:0/3558163294 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdfe800b830 con 0x561c5d39ebd0 2024-05-03T18:44:46.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.267+0000 7fdfed600700 1 -- 172.21.15.33:0/3558163294 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fdff002fbe0 con 0x561c5d39ebd0 2024-05-03T18:44:46.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.267+0000 7fe010ae2b80 10 client.16887 fetch_fsmap learned FSMap version 21 2024-05-03T18:44:46.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.267+0000 7fe010ae2b80 1 -- 172.21.15.33:0/3558163294 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x561c5d41d210 con 0x561c5d39ebd0 2024-05-03T18:44:46.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.268+0000 7fdfed600700 1 -- 172.21.15.33:0/3558163294 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdfe800b830 con 0x561c5d39ebd0 2024-05-03T18:44:46.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.269+0000 7fdfed600700 1 -- 172.21.15.33:0/3558163294 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fdff0010540 con 0x561c5d39ebd0 2024-05-03T18:44:46.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.269+0000 7fe010ae2b80 10 client.16887 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:44:46.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.269+0000 7fe010ae2b80 10 client.16887 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:44:46.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.269+0000 7fe010ae2b80 20 client.16887 populate_metadata read hostname 'smithi033' 2024-05-03T18:44:46.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.269+0000 7fe010ae2b80 1 --2- 172.21.15.33:0/3558163294 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x561c5d41d8c0 0x561c5d41fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:46.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.269+0000 7fe010ae2b80 4 client.16887 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-03T18:44:46.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.269+0000 7fe010ae2b80 1 -- 172.21.15.33:0/3558163294 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561c5d2f9400 con 0x561c5d41d8c0 2024-05-03T18:44:46.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.270+0000 7fdfff400700 1 --2- 172.21.15.33:0/3558163294 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x561c5d41d8c0 0x561c5d41fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:46.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.272+0000 7fdfff400700 1 --2- 172.21.15.33:0/3558163294 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x561c5d41d8c0 0x561c5d41fce0 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4569e2b73fcc13f server_cookie=10bec8dff7f10394 in_seq=0 out_seq=0 2024-05-03T18:44:46.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.273+0000 7fdfed600700 10 client.16887 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:44:46.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.273+0000 7fdfed600700 1 -- 172.21.15.33:0/3558163294 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561c5d2f9400 con 0x561c5d41d8c0 2024-05-03T18:44:46.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.273+0000 7fdfed600700 10 client.16887 handle_command_reply: tid=0 2024-05-03T18:44:46.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.274+0000 7fe010ae2b80 1 client.16887 shutdown 2024-05-03T18:44:46.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.274+0000 7fe010ae2b80 1 -- 172.21.15.33:0/3558163294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561c5d39ebd0 msgr2=0x561c5d3b9b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:46.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.274+0000 7fe010ae2b80 1 --2- 172.21.15.33:0/3558163294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561c5d39ebd0 0x561c5d3b9b40 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fdff0004790 tx=0x7fdff00047c0 comp rx=0 tx=0).stop 2024-05-03T18:44:46.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.274+0000 7fe010ae2b80 1 -- 172.21.15.33:0/3558163294 shutdown_connections 2024-05-03T18:44:46.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.274+0000 7fe010ae2b80 1 --2- 172.21.15.33:0/3558163294 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x561c5d41d8c0 0x561c5d41fce0 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:46.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.274+0000 7fe010ae2b80 1 --2- 172.21.15.33:0/3558163294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561c5d39ebd0 0x561c5d3b9b40 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:46.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.274+0000 7fe010ae2b80 1 --2- 172.21.15.33:0/3558163294 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561c5d398f30 0x561c5d3ab4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:46.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.274+0000 7fe010ae2b80 1 --2- 172.21.15.33:0/3558163294 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561c5d3998f0 0x561c5d3aba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:46.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.274+0000 7fe010ae2b80 1 -- 172.21.15.33:0/3558163294 >> 172.21.15.33:0/3558163294 conn(0x561c5d3116d0 msgr2=0x561c5d39db10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:44:46.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.274+0000 7fdfed600700 0 client.16887 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:44:46.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.275+0000 7fe010ae2b80 1 -- 172.21.15.33:0/3558163294 shutdown_connections 2024-05-03T18:44:46.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.275+0000 7fe010ae2b80 1 -- 172.21.15.33:0/3558163294 wait complete. 2024-05-03T18:44:46.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.276+0000 7fe010ae2b80 20 client.16887 trim_cache size 0 max 16384 2024-05-03T18:44:46.278 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-03T18:44:46.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.277+0000 7fe004c00700 1 -- 172.21.15.33:0/3542680353 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fdfe40b2a70 msgr2=0x7fdfe40b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:46.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.277+0000 7fe004c00700 1 --2- 172.21.15.33:0/3542680353 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fdfe40b2a70 0x7fdfe40b4f30 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fdff8007d30 tx=0x7fdff8000f40 comp rx=0 tx=0).stop 2024-05-03T18:44:46.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.277+0000 7fe004c00700 1 -- 172.21.15.33:0/3542680353 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe000107610 msgr2=0x7fe00006e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:46.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.277+0000 7fe004c00700 1 --2- 172.21.15.33:0/3542680353 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe000107610 0x7fe00006e380 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fdff800b740 tx=0x7fdff800ba50 comp rx=0 tx=0).stop 2024-05-03T18:44:46.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.277+0000 7fe004c00700 1 -- 172.21.15.33:0/3542680353 shutdown_connections 2024-05-03T18:44:46.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.277+0000 7fe004c00700 1 --2- 172.21.15.33:0/3542680353 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe000107610 0x7fe00006e380 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:46.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.277+0000 7fe004c00700 1 --2- 172.21.15.33:0/3542680353 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fdfe40b2a70 0x7fdfe40b4f30 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:46.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.277+0000 7fe004c00700 1 --2- 172.21.15.33:0/3542680353 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe000102530 0x7fe000074d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:46.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.277+0000 7fe004c00700 1 --2- 172.21.15.33:0/3542680353 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe000102ee0 0x7fe00006de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:46.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.277+0000 7fe004c00700 1 -- 172.21.15.33:0/3542680353 >> 172.21.15.33:0/3542680353 conn(0x7fe0000fddc0 msgr2=0x7fe0000ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:44:46.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.278+0000 7fe004c00700 1 -- 172.21.15.33:0/3542680353 shutdown_connections 2024-05-03T18:44:46.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:46.278+0000 7fe004c00700 1 -- 172.21.15.33:0/3542680353 wait complete. 2024-05-03T18:44:46.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:44:45 smithi099 ceph-mon[31658]: pgmap v553: 129 pgs: 129 active+clean; 3.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 528 KiB/s rd, 7.6 MiB/s wr, 578 op/s 2024-05-03T18:44:48.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:44:47 smithi150 ceph-mon[32492]: pgmap v554: 129 pgs: 129 active+clean; 3.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 7.0 MiB/s wr, 566 op/s 2024-05-03T18:44:48.172 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:44:47 smithi033 ceph-mon[28330]: pgmap v554: 129 pgs: 129 active+clean; 3.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 7.0 MiB/s wr, 566 op/s 2024-05-03T18:44:48.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:44:47 smithi099 ceph-mon[31658]: pgmap v554: 129 pgs: 129 active+clean; 3.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 7.0 MiB/s wr, 566 op/s 2024-05-03T18:44:49.274 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-03T18:44:49.274 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-03T18:44:50.111 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 815.0 s, 577.9 tps, lat 172.577 ms stddev 77.082 2024-05-03T18:44:50.132 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:44:49 smithi150 ceph-mon[32492]: pgmap v555: 129 pgs: 129 active+clean; 3.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 6.1 MiB/s wr, 471 op/s 2024-05-03T18:44:50.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:44:49 smithi033 ceph-mon[28330]: pgmap v555: 129 pgs: 129 active+clean; 3.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 6.1 MiB/s wr, 471 op/s 2024-05-03T18:44:50.297 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:44:49 smithi099 ceph-mon[31658]: pgmap v555: 129 pgs: 129 active+clean; 3.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 6.1 MiB/s wr, 471 op/s 2024-05-03T18:44:50.372 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:44:52.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:44:51 smithi150 ceph-mon[32492]: pgmap v556: 129 pgs: 129 active+clean; 3.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 9.4 MiB/s wr, 713 op/s 2024-05-03T18:44:52.228 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:44:51 smithi033 ceph-mon[28330]: pgmap v556: 129 pgs: 129 active+clean; 3.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 9.4 MiB/s wr, 713 op/s 2024-05-03T18:44:52.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:44:51 smithi099 ceph-mon[31658]: pgmap v556: 129 pgs: 129 active+clean; 3.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 9.4 MiB/s wr, 713 op/s 2024-05-03T18:44:54.134 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:44:53 smithi150 ceph-mon[32492]: pgmap v557: 129 pgs: 129 active+clean; 3.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 6.4 MiB/s wr, 478 op/s 2024-05-03T18:44:54.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:44:53 smithi033 ceph-mon[28330]: pgmap v557: 129 pgs: 129 active+clean; 3.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 6.4 MiB/s wr, 478 op/s 2024-05-03T18:44:54.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:44:53 smithi099 ceph-mon[31658]: pgmap v557: 129 pgs: 129 active+clean; 3.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 6.4 MiB/s wr, 478 op/s 2024-05-03T18:44:55.111 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 820.0 s, 508.2 tps, lat 188.643 ms stddev 123.261 2024-05-03T18:44:56.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:44:55 smithi150 ceph-mon[32492]: pgmap v558: 129 pgs: 129 active+clean; 3.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 8.2 MiB/s wr, 602 op/s 2024-05-03T18:44:56.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:44:55 smithi033 ceph-mon[28330]: pgmap v558: 129 pgs: 129 active+clean; 3.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 8.2 MiB/s wr, 602 op/s 2024-05-03T18:44:56.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:44:55 smithi099 ceph-mon[31658]: pgmap v558: 129 pgs: 129 active+clean; 3.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 8.2 MiB/s wr, 602 op/s 2024-05-03T18:44:56.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.364+0000 7f34a4800700 1 -- 172.21.15.33:0/402064126 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f349c06b180 msgr2=0x7f349c06b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:56.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.364+0000 7f34a4800700 1 --2- 172.21.15.33:0/402064126 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f349c06b180 0x7f349c06b600 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f348c0098f0 tx=0x7f348c009c00 comp rx=0 tx=0).stop 2024-05-03T18:44:56.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.365+0000 7f34a4800700 1 -- 172.21.15.33:0/402064126 shutdown_connections 2024-05-03T18:44:56.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.365+0000 7f34a4800700 1 --2- 172.21.15.33:0/402064126 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f349c06bb40 0x7f349c110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:56.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.365+0000 7f34a4800700 1 --2- 172.21.15.33:0/402064126 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f349c109510 0x7f349c1098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:56.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.365+0000 7f34a4800700 1 --2- 172.21.15.33:0/402064126 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f349c06b180 0x7f349c06b600 secure :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f348c0098f0 tx=0x7f348c009c00 comp rx=0 tx=0).stop 2024-05-03T18:44:56.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.365+0000 7f34a4800700 1 -- 172.21.15.33:0/402064126 >> 172.21.15.33:0/402064126 conn(0x7f349c079350 msgr2=0x7f349c079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:44:56.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.366+0000 7f34a4800700 1 -- 172.21.15.33:0/402064126 shutdown_connections 2024-05-03T18:44:56.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.367+0000 7f34a4800700 1 -- 172.21.15.33:0/402064126 wait complete. 2024-05-03T18:44:56.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.368+0000 7f34a4800700 1 Processor -- start 2024-05-03T18:44:56.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.368+0000 7f34a4800700 1 -- start start 2024-05-03T18:44:56.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.369+0000 7f34a4800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f349c06bb40 0x7f349c102490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:56.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.369+0000 7f34a4800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f349c109510 0x7f349c1029d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:56.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.369+0000 7f34a4800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f349c104120 0x7f349c102f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:56.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.369+0000 7f34a4800700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f349c113570 con 0x7f349c06bb40 2024-05-03T18:44:56.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.369+0000 7f34a4800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f349c1133f0 con 0x7f349c109510 2024-05-03T18:44:56.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.369+0000 7f34a4800700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f349c1136f0 con 0x7f349c104120 2024-05-03T18:44:56.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.369+0000 7f34a1400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f349c109510 0x7f349c1029d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:56.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.370+0000 7f34a2800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f349c104120 0x7f349c102f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:56.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.370+0000 7f34a1400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f349c109510 0x7f349c1029d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:51892/0 (socket says 172.21.15.33:51892) 2024-05-03T18:44:56.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.370+0000 7f34a1400700 1 -- 172.21.15.33:0/3680119359 learned_addr learned my addr 172.21.15.33:0/3680119359 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:44:56.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.370+0000 7f34a2800700 1 -- 172.21.15.33:0/3680119359 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f349c109510 msgr2=0x7f349c1029d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:56.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.370+0000 7f34a2800700 1 --2- 172.21.15.33:0/3680119359 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f349c109510 0x7f349c1029d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:56.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.370+0000 7f34a2800700 1 -- 172.21.15.33:0/3680119359 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f349c06bb40 msgr2=0x7f349c102490 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-03T18:44:56.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.371+0000 7f34a1e00700 1 --2- 172.21.15.33:0/3680119359 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f349c06bb40 0x7f349c102490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:56.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.372+0000 7f34a2800700 1 --2- 172.21.15.33:0/3680119359 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f349c06bb40 0x7f349c102490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:56.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.372+0000 7f34a2800700 1 -- 172.21.15.33:0/3680119359 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f349c1acb30 con 0x7f349c104120 2024-05-03T18:44:56.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.372+0000 7f34a1e00700 1 --2- 172.21.15.33:0/3680119359 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f349c06bb40 0x7f349c102490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:44:56.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.373+0000 7f34a2800700 1 --2- 172.21.15.33:0/3680119359 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f349c104120 0x7f349c102f30 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f3498007f10 tx=0x7f349800d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:44:56.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.374+0000 7f3496a00700 1 -- 172.21.15.33:0/3680119359 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f349800ddd0 con 0x7f349c104120 2024-05-03T18:44:56.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.374+0000 7f3496a00700 1 -- 172.21.15.33:0/3680119359 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3498009e20 con 0x7f349c104120 2024-05-03T18:44:56.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.374+0000 7f3496a00700 1 -- 172.21.15.33:0/3680119359 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3498015400 con 0x7f349c104120 2024-05-03T18:44:56.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.374+0000 7f34a4800700 1 -- 172.21.15.33:0/3680119359 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f349c1ace20 con 0x7f349c104120 2024-05-03T18:44:56.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.374+0000 7f34a4800700 1 -- 172.21.15.33:0/3680119359 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f349c1ad280 con 0x7f349c104120 2024-05-03T18:44:56.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.386+0000 7f3496a00700 1 -- 172.21.15.33:0/3680119359 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f3498005020 con 0x7f349c104120 2024-05-03T18:44:56.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.388+0000 7f3496a00700 1 --2- 172.21.15.33:0/3680119359 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f34840b2b20 0x7f34840b4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:56.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.389+0000 7f3496a00700 1 -- 172.21.15.33:0/3680119359 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 9518+0+0 (secure 0 0 0) 0x7f34980e23c0 con 0x7f349c104120 2024-05-03T18:44:56.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.390+0000 7f34a1e00700 1 --2- 172.21.15.33:0/3680119359 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f34840b2b20 0x7f34840b4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:56.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.391+0000 7f34a1e00700 1 --2- 172.21.15.33:0/3680119359 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f34840b2b20 0x7f34840b4fe0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f3490009b50 tx=0x7f3490009550 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:44:56.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.396+0000 7f34af1b9b80 1 Processor -- start 2024-05-03T18:44:56.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.396+0000 7f34af1b9b80 1 -- start start 2024-05-03T18:44:56.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.397+0000 7f34af1b9b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55baa6a91830 0x55baa6a91c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:56.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.397+0000 7f34af1b9b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55baa6a92150 0x55baa6a8d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:56.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.397+0000 7f34af1b9b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55baa6a8d650 0x55baa6a8dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:56.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.397+0000 7f34af1b9b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55baa69dfaa0 con 0x55baa6a91830 2024-05-03T18:44:56.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.397+0000 7f34af1b9b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55baa697d7e0 con 0x55baa6a92150 2024-05-03T18:44:56.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.397+0000 7f34af1b9b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55baa69dba60 con 0x55baa6a8d650 2024-05-03T18:44:56.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.397+0000 7f34a1e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55baa6a91830 0x55baa6a91c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:56.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.398+0000 7f34a1e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55baa6a91830 0x55baa6a91c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45804/0 (socket says 172.21.15.33:45804) 2024-05-03T18:44:56.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.398+0000 7f34a1e00700 1 -- 172.21.15.33:0/1382780107 learned_addr learned my addr 172.21.15.33:0/1382780107 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:44:56.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.398+0000 7f34a1400700 1 --2- 172.21.15.33:0/1382780107 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55baa6a92150 0x55baa6a8d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:56.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.398+0000 7f34a2800700 1 --2- 172.21.15.33:0/1382780107 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55baa6a8d650 0x55baa6a8dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:56.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.399+0000 7f34a1400700 1 -- 172.21.15.33:0/1382780107 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55baa6a8d650 msgr2=0x55baa6a8dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:44:56.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.399+0000 7f34a1400700 1 --2- 172.21.15.33:0/1382780107 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55baa6a8d650 0x55baa6a8dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:56.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.399+0000 7f34a1400700 1 -- 172.21.15.33:0/1382780107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55baa6a91830 msgr2=0x55baa6a91c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:44:56.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.399+0000 7f34a1400700 1 --2- 172.21.15.33:0/1382780107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55baa6a91830 0x55baa6a91c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:56.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.399+0000 7f34a1400700 1 -- 172.21.15.33:0/1382780107 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55baa692c950 con 0x55baa6a92150 2024-05-03T18:44:56.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.400+0000 7f34a2800700 1 --2- 172.21.15.33:0/1382780107 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55baa6a8d650 0x55baa6a8dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:44:56.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.401+0000 7f34a1400700 1 --2- 172.21.15.33:0/1382780107 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55baa6a92150 0x55baa6a8d030 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f349c19df20 tx=0x7f348c009400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78450dc61a0410d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:44:56.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.401+0000 7f348b400700 1 -- 172.21.15.33:0/1382780107 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f348c010ec0 con 0x55baa6a92150 2024-05-03T18:44:56.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.401+0000 7f348b400700 1 -- 172.21.15.33:0/1382780107 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f348c026070 con 0x55baa6a92150 2024-05-03T18:44:56.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.401+0000 7f348b400700 1 -- 172.21.15.33:0/1382780107 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f348c0038e0 con 0x55baa6a92150 2024-05-03T18:44:56.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.403+0000 7f34af1b9b80 1 -- 172.21.15.33:0/1382780107 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55baa6a92150 msgr2=0x55baa6a8d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:56.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.403+0000 7f34af1b9b80 1 --2- 172.21.15.33:0/1382780107 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55baa6a92150 0x55baa6a8d030 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f349c19df20 tx=0x7f348c009400 comp rx=0 tx=0).stop 2024-05-03T18:44:56.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.403+0000 7f34af1b9b80 1 -- 172.21.15.33:0/1382780107 shutdown_connections 2024-05-03T18:44:56.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.403+0000 7f34af1b9b80 1 --2- 172.21.15.33:0/1382780107 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55baa6a8d650 0x55baa6a8dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:56.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.403+0000 7f34af1b9b80 1 --2- 172.21.15.33:0/1382780107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55baa6a91830 0x55baa6a91c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:56.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.403+0000 7f34af1b9b80 1 --2- 172.21.15.33:0/1382780107 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55baa6a92150 0x55baa6a8d030 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:56.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.403+0000 7f34af1b9b80 1 -- 172.21.15.33:0/1382780107 >> 172.21.15.33:0/1382780107 conn(0x55baa69bbbe0 msgr2=0x55baa6a87870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:44:56.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.404+0000 7f34af1b9b80 1 -- 172.21.15.33:0/1382780107 shutdown_connections 2024-05-03T18:44:56.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.404+0000 7f34af1b9b80 1 -- 172.21.15.33:0/1382780107 wait complete. 2024-05-03T18:44:56.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.405+0000 7f34af1b9b80 1 -- start start 2024-05-03T18:44:56.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.406+0000 7f34af1b9b80 1 Processor -- start 2024-05-03T18:44:56.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.408+0000 7f34af1b9b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55baa6a8d650 0x55baa6a98f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:56.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.408+0000 7f34af1b9b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55baa6a91830 0x55baa6a962a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:56.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.408+0000 7f34af1b9b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55baa6a92150 0x55baa6a967e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:56.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.409+0000 7f34af1b9b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55baa6a99cc0 con 0x55baa6a92150 2024-05-03T18:44:56.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.409+0000 7f34af1b9b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55baa6945ba0 con 0x55baa6a8d650 2024-05-03T18:44:56.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.409+0000 7f34af1b9b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55baa6a99e40 con 0x55baa6a91830 2024-05-03T18:44:56.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.409+0000 7f34af1b9b80 10 client.0 _ll_register_callbacks cb 0x55baa6986cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:44:56.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.409+0000 7f34a1400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55baa6a91830 0x55baa6a962a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:56.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.409+0000 7f34a1400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55baa6a91830 0x55baa6a962a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:45046/0 (socket says 172.21.15.33:45046) 2024-05-03T18:44:56.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.410+0000 7f34a1400700 1 -- 172.21.15.33:0/970122167 learned_addr learned my addr 172.21.15.33:0/970122167 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:44:56.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.410+0000 7f34a1e00700 1 --2- 172.21.15.33:0/970122167 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55baa6a8d650 0x55baa6a98f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:56.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.410+0000 7f34a2800700 1 --2- 172.21.15.33:0/970122167 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55baa6a92150 0x55baa6a967e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:56.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.411+0000 7f34a1400700 1 -- 172.21.15.33:0/970122167 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55baa6a8d650 msgr2=0x55baa6a98f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:44:56.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.411+0000 7f34a1400700 1 --2- 172.21.15.33:0/970122167 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55baa6a8d650 0x55baa6a98f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:56.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.411+0000 7f34a1400700 1 -- 172.21.15.33:0/970122167 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55baa6a92150 msgr2=0x55baa6a967e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:44:56.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.411+0000 7f34a1400700 1 --2- 172.21.15.33:0/970122167 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55baa6a92150 0x55baa6a967e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:56.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.411+0000 7f34a1400700 1 -- 172.21.15.33:0/970122167 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55baa69a54a0 con 0x55baa6a91830 2024-05-03T18:44:56.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.412+0000 7f34a1400700 1 --2- 172.21.15.33:0/970122167 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55baa6a91830 0x55baa6a962a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f348c00f760 tx=0x7f348c00f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ffd3a0473e50fa server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:44:56.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.412+0000 7f347fe00700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-03T18:44:56.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.413+0000 7f347fe00700 1 -- 172.21.15.33:0/970122167 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f348c005450 con 0x55baa6a91830 2024-05-03T18:44:56.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.414+0000 7f347fe00700 1 -- 172.21.15.33:0/970122167 <== mon.1 v2:172.21.15.150:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9518+0+0 (secure 0 0 0) 0x7f348c003680 con 0x55baa6a91830 2024-05-03T18:44:56.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.414+0000 7f34af1b9b80 1 -- 172.21.15.33:0/970122167 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f348c026070 con 0x55baa6a91830 2024-05-03T18:44:56.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.414+0000 7f347fe00700 1 -- 172.21.15.33:0/970122167 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f348c039050 con 0x55baa6a91830 2024-05-03T18:44:56.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.415+0000 7f34af1b9b80 10 client.26767 fetch_fsmap learned FSMap version 21 2024-05-03T18:44:56.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.415+0000 7f34af1b9b80 1 -- 172.21.15.33:0/970122167 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55baa6a99cc0 con 0x55baa6a91830 2024-05-03T18:44:56.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.415+0000 7f347fe00700 1 -- 172.21.15.33:0/970122167 <== mon.1 v2:172.21.15.150:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f348c026070 con 0x55baa6a91830 2024-05-03T18:44:56.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.416+0000 7f347fe00700 1 -- 172.21.15.33:0/970122167 <== mon.1 v2:172.21.15.150:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f348c00ba10 con 0x55baa6a91830 2024-05-03T18:44:56.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.416+0000 7f34af1b9b80 10 client.26767 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:44:56.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.416+0000 7f34af1b9b80 10 client.26767 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:44:56.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.416+0000 7f34af1b9b80 20 client.26767 populate_metadata read hostname 'smithi033' 2024-05-03T18:44:56.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.416+0000 7f34af1b9b80 1 --2- 172.21.15.33:0/970122167 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55baa6aa6760 0x55baa6aa8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:56.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.416+0000 7f34af1b9b80 4 client.26767 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:44:56.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.416+0000 7f34af1b9b80 1 -- 172.21.15.33:0/970122167 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55baa6a5a250 con 0x55baa6aa6760 2024-05-03T18:44:56.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.420+0000 7f34a1e00700 1 --2- 172.21.15.33:0/970122167 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55baa6aa6760 0x55baa6aa8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:56.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.423+0000 7f347fe00700 10 client.26767 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:44:56.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.423+0000 7f34a1e00700 1 --2- 172.21.15.33:0/970122167 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55baa6aa6760 0x55baa6aa8b50 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d5825e996e42095 server_cookie=f5d1ba4d60f95130 in_seq=0 out_seq=0 2024-05-03T18:44:56.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.426+0000 7f347fe00700 1 -- 172.21.15.33:0/970122167 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55baa6a5a250 con 0x55baa6aa6760 2024-05-03T18:44:56.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.426+0000 7f347fe00700 10 client.26767 handle_command_reply: tid=0 2024-05-03T18:44:56.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.426+0000 7f34af1b9b80 1 client.26767 shutdown 2024-05-03T18:44:56.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.427+0000 7f34af1b9b80 1 -- 172.21.15.33:0/970122167 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55baa6a91830 msgr2=0x55baa6a962a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:56.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.427+0000 7f34af1b9b80 1 --2- 172.21.15.33:0/970122167 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55baa6a91830 0x55baa6a962a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f348c00f760 tx=0x7f348c00f790 comp rx=0 tx=0).stop 2024-05-03T18:44:56.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.427+0000 7f34af1b9b80 1 -- 172.21.15.33:0/970122167 shutdown_connections 2024-05-03T18:44:56.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.427+0000 7f34af1b9b80 1 --2- 172.21.15.33:0/970122167 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55baa6aa6760 0x55baa6aa8b50 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:56.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.427+0000 7f34af1b9b80 1 --2- 172.21.15.33:0/970122167 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55baa6a92150 0x55baa6a967e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:56.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.427+0000 7f34af1b9b80 1 --2- 172.21.15.33:0/970122167 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55baa6a8d650 0x55baa6a98f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:56.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.427+0000 7f34af1b9b80 1 --2- 172.21.15.33:0/970122167 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55baa6a91830 0x55baa6a962a0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:56.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.427+0000 7f34af1b9b80 1 -- 172.21.15.33:0/970122167 >> 172.21.15.33:0/970122167 conn(0x55baa69bbbe0 msgr2=0x55baa6a92e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:44:56.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.427+0000 7f347fe00700 0 client.26767 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:44:56.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.427+0000 7f34af1b9b80 1 -- 172.21.15.33:0/970122167 shutdown_connections 2024-05-03T18:44:56.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.428+0000 7f34af1b9b80 1 -- 172.21.15.33:0/970122167 wait complete. 2024-05-03T18:44:56.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.428+0000 7f34af1b9b80 20 client.26767 trim_cache size 0 max 16384 2024-05-03T18:44:56.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.466+0000 7f34af1b9b80 1 Processor -- start 2024-05-03T18:44:56.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.467+0000 7f34af1b9b80 1 -- start start 2024-05-03T18:44:56.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.467+0000 7f34af1b9b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55baa6a8d650 0x55baa6a876e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:56.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.467+0000 7f34af1b9b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55baa6a91830 0x55baa6a96260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:56.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.467+0000 7f34af1b9b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55baa6a92150 0x55baa6a967a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:56.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.467+0000 7f34af1b9b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f349c1b4140 con 0x55baa6a8d650 2024-05-03T18:44:56.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.467+0000 7f34af1b9b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f349c1b69a0 con 0x55baa6a91830 2024-05-03T18:44:56.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.467+0000 7f34af1b9b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f349c1b4790 con 0x55baa6a92150 2024-05-03T18:44:56.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.468+0000 7f34a2800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55baa6a92150 0x55baa6a967a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:56.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.468+0000 7f34a1e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55baa6a8d650 0x55baa6a876e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:56.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.469+0000 7f34a1400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55baa6a91830 0x55baa6a96260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:56.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.469+0000 7f34a1400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55baa6a91830 0x55baa6a96260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:51914/0 (socket says 172.21.15.33:51914) 2024-05-03T18:44:56.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.469+0000 7f34a1400700 1 -- 172.21.15.33:0/543142185 learned_addr learned my addr 172.21.15.33:0/543142185 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:44:56.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.469+0000 7f34a2800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55baa6a92150 0x55baa6a967a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:45054/0 (socket says 172.21.15.33:45054) 2024-05-03T18:44:56.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.470+0000 7f34a1e00700 1 -- 172.21.15.33:0/543142185 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55baa6a92150 msgr2=0x55baa6a967a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:44:56.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.470+0000 7f34a1e00700 1 --2- 172.21.15.33:0/543142185 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55baa6a92150 0x55baa6a967a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:56.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.470+0000 7f34a1e00700 1 -- 172.21.15.33:0/543142185 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55baa6a91830 msgr2=0x55baa6a96260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:44:56.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.470+0000 7f34a1e00700 1 --2- 172.21.15.33:0/543142185 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55baa6a91830 0x55baa6a96260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:56.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.470+0000 7f34a1e00700 1 -- 172.21.15.33:0/543142185 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55baa6a94850 con 0x55baa6a8d650 2024-05-03T18:44:56.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.471+0000 7f34a1400700 1 --2- 172.21.15.33:0/543142185 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55baa6a91830 0x55baa6a96260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:44:56.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.471+0000 7f34a1e00700 1 --2- 172.21.15.33:0/543142185 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55baa6a8d650 0x55baa6a876e0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x55baa697d8d0 tx=0x7f3490014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79d2afea06ed5cd8 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:44:56.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.472+0000 7f348aa00700 1 -- 172.21.15.33:0/543142185 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3490011df0 con 0x55baa6a8d650 2024-05-03T18:44:56.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.473+0000 7f348aa00700 1 -- 172.21.15.33:0/543142185 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3490017040 con 0x55baa6a8d650 2024-05-03T18:44:56.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.473+0000 7f348aa00700 1 -- 172.21.15.33:0/543142185 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3490029a60 con 0x55baa6a8d650 2024-05-03T18:44:56.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.474+0000 7f34af1b9b80 1 -- 172.21.15.33:0/543142185 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55baa6a8d650 msgr2=0x55baa6a876e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:56.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.474+0000 7f34af1b9b80 1 --2- 172.21.15.33:0/543142185 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55baa6a8d650 0x55baa6a876e0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x55baa697d8d0 tx=0x7f3490014000 comp rx=0 tx=0).stop 2024-05-03T18:44:56.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.475+0000 7f34af1b9b80 1 -- 172.21.15.33:0/543142185 shutdown_connections 2024-05-03T18:44:56.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.475+0000 7f34af1b9b80 1 --2- 172.21.15.33:0/543142185 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55baa6a92150 0x55baa6a967a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:56.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.475+0000 7f34af1b9b80 1 --2- 172.21.15.33:0/543142185 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55baa6a8d650 0x55baa6a876e0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:56.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.475+0000 7f34af1b9b80 1 --2- 172.21.15.33:0/543142185 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55baa6a91830 0x55baa6a96260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:56.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.475+0000 7f34af1b9b80 1 -- 172.21.15.33:0/543142185 >> 172.21.15.33:0/543142185 conn(0x55baa69bbbe0 msgr2=0x55baa6a95b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:44:56.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.475+0000 7f34af1b9b80 1 -- 172.21.15.33:0/543142185 shutdown_connections 2024-05-03T18:44:56.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.475+0000 7f34af1b9b80 1 -- 172.21.15.33:0/543142185 wait complete. 2024-05-03T18:44:56.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.476+0000 7f34af1b9b80 1 -- start start 2024-05-03T18:44:56.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.476+0000 7f34af1b9b80 1 Processor -- start 2024-05-03T18:44:56.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.477+0000 7f34af1b9b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55baa6a8d650 0x55baa6a9a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:56.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.477+0000 7f34af1b9b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55baa6a91830 0x55baa6a9a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:56.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.477+0000 7f34af1b9b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55baa6a92150 0x55baa6b10d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:56.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.478+0000 7f34af1b9b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f349c1b4e60 con 0x55baa6a91830 2024-05-03T18:44:56.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.478+0000 7f34af1b9b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f349c1b62d0 con 0x55baa6a8d650 2024-05-03T18:44:56.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.478+0000 7f34af1b9b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f349c1b5530 con 0x55baa6a92150 2024-05-03T18:44:56.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.478+0000 7f34af1b9b80 10 client.0 _ll_register_callbacks cb 0x55baa6986cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:44:56.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.478+0000 7f34a1400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55baa6a91830 0x55baa6a9a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:56.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.479+0000 7f34a2800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55baa6a92150 0x55baa6b10d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:56.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.479+0000 7f34a1e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55baa6a8d650 0x55baa6a9a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:56.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.479+0000 7f34a1400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55baa6a91830 0x55baa6a9a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45830/0 (socket says 172.21.15.33:45830) 2024-05-03T18:44:56.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.479+0000 7f34a1e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55baa6a8d650 0x55baa6a9a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:51924/0 (socket says 172.21.15.33:51924) 2024-05-03T18:44:56.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.479+0000 7f34a1400700 1 -- 172.21.15.33:0/4115894503 learned_addr learned my addr 172.21.15.33:0/4115894503 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:44:56.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.479+0000 7f34a1400700 1 -- 172.21.15.33:0/4115894503 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55baa6a92150 msgr2=0x55baa6b10d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:44:56.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.480+0000 7f34a1400700 1 --2- 172.21.15.33:0/4115894503 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55baa6a92150 0x55baa6b10d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:56.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.480+0000 7f34a1400700 1 -- 172.21.15.33:0/4115894503 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55baa6a8d650 msgr2=0x55baa6a9a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:44:56.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.480+0000 7f34a1400700 1 --2- 172.21.15.33:0/4115894503 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55baa6a8d650 0x55baa6a9a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:56.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.480+0000 7f34a1400700 1 -- 172.21.15.33:0/4115894503 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55baa6b11240 con 0x55baa6a91830 2024-05-03T18:44:56.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.481+0000 7f34a1400700 1 --2- 172.21.15.33:0/4115894503 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55baa6a91830 0x55baa6a9a5d0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f348c00f760 tx=0x7f348c009400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56f74003293bab51 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:44:56.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.484+0000 7f347fe00700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:44:56.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.485+0000 7f347fe00700 1 -- 172.21.15.33:0/4115894503 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f348c00b2c0 con 0x55baa6a91830 2024-05-03T18:44:56.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.485+0000 7f347fe00700 1 -- 172.21.15.33:0/4115894503 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9518+0+0 (secure 0 0 0) 0x7f348c00b610 con 0x55baa6a91830 2024-05-03T18:44:56.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.485+0000 7f34af1b9b80 1 -- 172.21.15.33:0/4115894503 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3490017040 con 0x55baa6a91830 2024-05-03T18:44:56.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.486+0000 7f347fe00700 1 -- 172.21.15.33:0/4115894503 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f348c00b810 con 0x55baa6a91830 2024-05-03T18:44:56.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.486+0000 7f34af1b9b80 10 client.16902 fetch_fsmap learned FSMap version 21 2024-05-03T18:44:56.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.486+0000 7f34af1b9b80 1 -- 172.21.15.33:0/4115894503 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55baa6b12410 con 0x55baa6a91830 2024-05-03T18:44:56.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.487+0000 7f347fe00700 1 -- 172.21.15.33:0/4115894503 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3490017040 con 0x55baa6a91830 2024-05-03T18:44:56.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.488+0000 7f347fe00700 1 -- 172.21.15.33:0/4115894503 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f348c003680 con 0x55baa6a91830 2024-05-03T18:44:56.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.488+0000 7f34af1b9b80 10 client.16902 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:44:56.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.488+0000 7f34af1b9b80 10 client.16902 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:44:56.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.488+0000 7f34af1b9b80 20 client.16902 populate_metadata read hostname 'smithi033' 2024-05-03T18:44:56.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.488+0000 7f34af1b9b80 1 --2- 172.21.15.33:0/4115894503 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55baa6b12a70 0x55baa6b14e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:56.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.488+0000 7f34af1b9b80 4 client.16902 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-03T18:44:56.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.488+0000 7f34af1b9b80 1 -- 172.21.15.33:0/4115894503 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55baa69ed480 con 0x55baa6b12a70 2024-05-03T18:44:56.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.489+0000 7f34a1e00700 1 --2- 172.21.15.33:0/4115894503 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55baa6b12a70 0x55baa6b14e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:56.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.494+0000 7f347fe00700 10 client.16902 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:44:56.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.494+0000 7f34a1e00700 1 --2- 172.21.15.33:0/4115894503 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55baa6b12a70 0x55baa6b14e90 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5fe6592098330f82 server_cookie=1309bde4fcfd6860 in_seq=0 out_seq=0 2024-05-03T18:44:56.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.496+0000 7f347fe00700 1 -- 172.21.15.33:0/4115894503 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55baa69ed480 con 0x55baa6b12a70 2024-05-03T18:44:56.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.496+0000 7f347fe00700 10 client.16902 handle_command_reply: tid=0 2024-05-03T18:44:56.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.496+0000 7f34af1b9b80 1 client.16902 shutdown 2024-05-03T18:44:56.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.496+0000 7f34af1b9b80 1 -- 172.21.15.33:0/4115894503 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55baa6a91830 msgr2=0x55baa6a9a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:56.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.496+0000 7f34af1b9b80 1 --2- 172.21.15.33:0/4115894503 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55baa6a91830 0x55baa6a9a5d0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f348c00f760 tx=0x7f348c009400 comp rx=0 tx=0).stop 2024-05-03T18:44:56.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.497+0000 7f34af1b9b80 1 -- 172.21.15.33:0/4115894503 shutdown_connections 2024-05-03T18:44:56.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.497+0000 7f34af1b9b80 1 --2- 172.21.15.33:0/4115894503 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55baa6b12a70 0x55baa6b14e90 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:56.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.497+0000 7f347fe00700 0 client.16902 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:44:56.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.497+0000 7f34af1b9b80 1 --2- 172.21.15.33:0/4115894503 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55baa6a92150 0x55baa6b10d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:56.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.497+0000 7f34af1b9b80 1 --2- 172.21.15.33:0/4115894503 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55baa6a8d650 0x55baa6a9a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:56.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.497+0000 7f34af1b9b80 1 --2- 172.21.15.33:0/4115894503 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55baa6a91830 0x55baa6a9a5d0 secure :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f348c00f760 tx=0x7f348c009400 comp rx=0 tx=0).stop 2024-05-03T18:44:56.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.497+0000 7f34af1b9b80 1 -- 172.21.15.33:0/4115894503 >> 172.21.15.33:0/4115894503 conn(0x55baa69bbbe0 msgr2=0x55baa6a963b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:44:56.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.497+0000 7f34af1b9b80 1 -- 172.21.15.33:0/4115894503 shutdown_connections 2024-05-03T18:44:56.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.497+0000 7f34af1b9b80 1 -- 172.21.15.33:0/4115894503 wait complete. 2024-05-03T18:44:56.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.497+0000 7f34af1b9b80 20 client.16902 trim_cache size 0 max 16384 2024-05-03T18:44:56.500 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-03T18:44:56.500 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-05-03T18:44:56.500 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "48ed0fce-9416-4316-a6df-f7feda4046af", 2024-05-03T18:44:56.500 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-05-03T18:44:56.500 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-03T18:44:56.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.499+0000 7f34a4800700 1 -- 172.21.15.33:0/3680119359 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f34840b2b20 msgr2=0x7f34840b4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:56.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.499+0000 7f34a4800700 1 --2- 172.21.15.33:0/3680119359 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f34840b2b20 0x7f34840b4fe0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f3490009b50 tx=0x7f3490009550 comp rx=0 tx=0).stop 2024-05-03T18:44:56.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.499+0000 7f34a4800700 1 -- 172.21.15.33:0/3680119359 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f349c104120 msgr2=0x7f349c102f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:56.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.499+0000 7f34a4800700 1 --2- 172.21.15.33:0/3680119359 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f349c104120 0x7f349c102f30 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f3498007f10 tx=0x7f349800d3b0 comp rx=0 tx=0).stop 2024-05-03T18:44:56.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.499+0000 7f34a4800700 1 -- 172.21.15.33:0/3680119359 shutdown_connections 2024-05-03T18:44:56.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.499+0000 7f34a4800700 1 --2- 172.21.15.33:0/3680119359 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f349c104120 0x7f349c102f30 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:56.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.499+0000 7f34a4800700 1 --2- 172.21.15.33:0/3680119359 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f349c06bb40 0x7f349c102490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:56.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.499+0000 7f34a4800700 1 --2- 172.21.15.33:0/3680119359 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f34840b2b20 0x7f34840b4fe0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:56.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.499+0000 7f34a4800700 1 --2- 172.21.15.33:0/3680119359 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f349c109510 0x7f349c1029d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:56.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.499+0000 7f34a4800700 1 -- 172.21.15.33:0/3680119359 >> 172.21.15.33:0/3680119359 conn(0x7f349c079350 msgr2=0x7f349c1014c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:44:56.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.500+0000 7f34a4800700 1 -- 172.21.15.33:0/3680119359 shutdown_connections 2024-05-03T18:44:56.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:44:56.500+0000 7f34a4800700 1 -- 172.21.15.33:0/3680119359 wait complete. 2024-05-03T18:44:58.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:44:57 smithi150 ceph-mon[32492]: pgmap v559: 129 pgs: 129 active+clean; 3.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 7.6 MiB/s wr, 583 op/s 2024-05-03T18:44:58.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:44:57 smithi150 ceph-mon[32492]: scrub queued for path: / 2024-05-03T18:44:58.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:44:57 smithi150 ceph-mon[32492]: scrub summary: idle+waiting paths [/] 2024-05-03T18:44:58.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:44:57 smithi150 ceph-mon[32492]: scrub summary: active paths [/] 2024-05-03T18:44:58.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:44:57 smithi150 ceph-mon[32492]: scrub summary: idle+waiting paths [/] 2024-05-03T18:44:58.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:44:57 smithi033 ceph-mon[28330]: pgmap v559: 129 pgs: 129 active+clean; 3.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 7.6 MiB/s wr, 583 op/s 2024-05-03T18:44:58.236 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:44:57 smithi033 ceph-mon[28330]: scrub queued for path: / 2024-05-03T18:44:58.236 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:44:57 smithi033 ceph-mon[28330]: scrub summary: idle+waiting paths [/] 2024-05-03T18:44:58.236 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:44:57 smithi033 ceph-mon[28330]: scrub summary: active paths [/] 2024-05-03T18:44:58.236 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:44:57 smithi033 ceph-mon[28330]: scrub summary: idle+waiting paths [/] 2024-05-03T18:44:58.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:44:57 smithi099 ceph-mon[31658]: pgmap v559: 129 pgs: 129 active+clean; 3.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 7.6 MiB/s wr, 583 op/s 2024-05-03T18:44:58.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:44:57 smithi099 ceph-mon[31658]: scrub queued for path: / 2024-05-03T18:44:58.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:44:57 smithi099 ceph-mon[31658]: scrub summary: idle+waiting paths [/] 2024-05-03T18:44:58.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:44:57 smithi099 ceph-mon[31658]: scrub summary: active paths [/] 2024-05-03T18:44:58.297 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:44:57 smithi099 ceph-mon[31658]: scrub summary: idle+waiting paths [/] 2024-05-03T18:44:58.435 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-03T18:44:59.167 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:44:59.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:44:59 smithi033 ceph-mon[28330]: pgmap v560: 129 pgs: 129 active+clean; 3.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 6.2 MiB/s wr, 471 op/s 2024-05-03T18:45:00.111 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 825.0 s, 472.8 tps, lat 217.781 ms stddev 176.274 2024-05-03T18:45:00.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:44:59 smithi099 ceph-mon[31658]: pgmap v560: 129 pgs: 129 active+clean; 3.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 6.2 MiB/s wr, 471 op/s 2024-05-03T18:45:00.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:44:59 smithi150 ceph-mon[32492]: pgmap v560: 129 pgs: 129 active+clean; 3.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 6.2 MiB/s wr, 471 op/s 2024-05-03T18:45:02.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:45:01 smithi033 ceph-mon[28330]: pgmap v561: 129 pgs: 129 active+clean; 3.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 455 KiB/s rd, 8.5 MiB/s wr, 662 op/s 2024-05-03T18:45:02.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:45:01 smithi033 ceph-mon[28330]: scrub summary: idle 2024-05-03T18:45:02.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:45:01 smithi099 ceph-mon[31658]: pgmap v561: 129 pgs: 129 active+clean; 3.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 455 KiB/s rd, 8.5 MiB/s wr, 662 op/s 2024-05-03T18:45:02.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:45:01 smithi099 ceph-mon[31658]: scrub summary: idle 2024-05-03T18:45:02.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:45:01 smithi150 ceph-mon[32492]: pgmap v561: 129 pgs: 129 active+clean; 3.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 455 KiB/s rd, 8.5 MiB/s wr, 662 op/s 2024-05-03T18:45:02.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:45:01 smithi150 ceph-mon[32492]: scrub summary: idle 2024-05-03T18:45:03.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.926+0000 7f6af1a00700 1 -- 172.21.15.33:0/2604287392 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6aec0779d0 msgr2=0x7f6aec077db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:03.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.926+0000 7f6af1a00700 1 --2- 172.21.15.33:0/2604287392 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6aec0779d0 0x7f6aec077db0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f6adc0098f0 tx=0x7f6adc009c00 comp rx=0 tx=0).stop 2024-05-03T18:45:03.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.926+0000 7f6af1a00700 1 -- 172.21.15.33:0/2604287392 shutdown_connections 2024-05-03T18:45:03.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.926+0000 7f6af1a00700 1 --2- 172.21.15.33:0/2604287392 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6aec114380 0x7f6aec116770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:03.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.926+0000 7f6af1a00700 1 --2- 172.21.15.33:0/2604287392 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6aec0779d0 0x7f6aec077db0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:03.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.926+0000 7f6af1a00700 1 --2- 172.21.15.33:0/2604287392 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6aec0782f0 0x7f6aec113e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:03.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.926+0000 7f6af1a00700 1 -- 172.21.15.33:0/2604287392 >> 172.21.15.33:0/2604287392 conn(0x7f6aec1011d0 msgr2=0x7f6aec1035f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:45:03.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.926+0000 7f6af1a00700 1 -- 172.21.15.33:0/2604287392 shutdown_connections 2024-05-03T18:45:03.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.927+0000 7f6af1a00700 1 -- 172.21.15.33:0/2604287392 wait complete. 2024-05-03T18:45:03.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.928+0000 7f6af1a00700 1 Processor -- start 2024-05-03T18:45:03.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.929+0000 7f6af1a00700 1 -- start start 2024-05-03T18:45:03.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.929+0000 7f6af1a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6aec0779d0 0x7f6aec1a0160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:03.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.929+0000 7f6af1a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6aec0782f0 0x7f6aec1a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:03.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.930+0000 7f6af1a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6aec114380 0x7f6aec1a4a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:03.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.930+0000 7f6af1a00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6aec119c50 con 0x7f6aec0782f0 2024-05-03T18:45:03.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.930+0000 7f6af1a00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6aec119ad0 con 0x7f6aec0779d0 2024-05-03T18:45:03.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.930+0000 7f6af1a00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6aec119dd0 con 0x7f6aec114380 2024-05-03T18:45:03.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.930+0000 7f6aea000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6aec0782f0 0x7f6aec1a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:03.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.930+0000 7f6aeaa00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6aec0779d0 0x7f6aec1a0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:03.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.931+0000 7f6aeaa00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6aec0779d0 0x7f6aec1a0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:51936/0 (socket says 172.21.15.33:51936) 2024-05-03T18:45:03.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.931+0000 7f6aeaa00700 1 -- 172.21.15.33:0/424446995 learned_addr learned my addr 172.21.15.33:0/424446995 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:45:03.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.931+0000 7f6aeb400700 1 --2- 172.21.15.33:0/424446995 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6aec114380 0x7f6aec1a4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:03.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.931+0000 7f6aeb400700 1 -- 172.21.15.33:0/424446995 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6aec0779d0 msgr2=0x7f6aec1a0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:03.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.931+0000 7f6aeb400700 1 --2- 172.21.15.33:0/424446995 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6aec0779d0 0x7f6aec1a0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:03.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.931+0000 7f6aeb400700 1 -- 172.21.15.33:0/424446995 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6aec0782f0 msgr2=0x7f6aec1a06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:03.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.931+0000 7f6aeb400700 1 --2- 172.21.15.33:0/424446995 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6aec0782f0 0x7f6aec1a06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:03.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.931+0000 7f6aeb400700 1 -- 172.21.15.33:0/424446995 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6aec1a5170 con 0x7f6aec114380 2024-05-03T18:45:03.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.932+0000 7f6aeaa00700 1 --2- 172.21.15.33:0/424446995 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6aec0779d0 0x7f6aec1a0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:45:03.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.932+0000 7f6aea000700 1 --2- 172.21.15.33:0/424446995 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6aec0782f0 0x7f6aec1a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:45:03.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.932+0000 7f6aeb400700 1 --2- 172.21.15.33:0/424446995 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6aec114380 0x7f6aec1a4a30 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f6aec07bcc0 tx=0x7f6ae400ef90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:45:03.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.933+0000 7f6ae3400700 1 -- 172.21.15.33:0/424446995 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae40092e0 con 0x7f6aec114380 2024-05-03T18:45:03.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.933+0000 7f6ae3400700 1 -- 172.21.15.33:0/424446995 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6ae4004bb0 con 0x7f6aec114380 2024-05-03T18:45:03.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.933+0000 7f6ae3400700 1 -- 172.21.15.33:0/424446995 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae4004020 con 0x7f6aec114380 2024-05-03T18:45:03.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.933+0000 7f6af1a00700 1 -- 172.21.15.33:0/424446995 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6aec1a5400 con 0x7f6aec114380 2024-05-03T18:45:03.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.933+0000 7f6af1a00700 1 -- 172.21.15.33:0/424446995 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6aec1acce0 con 0x7f6aec114380 2024-05-03T18:45:03.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.940+0000 7f6ae3400700 1 -- 172.21.15.33:0/424446995 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f6ae401b030 con 0x7f6aec114380 2024-05-03T18:45:03.943 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:45:03 smithi033 ceph-mon[28330]: pgmap v562: 129 pgs: 129 active+clean; 3.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 5.3 MiB/s wr, 419 op/s 2024-05-03T18:45:03.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.946+0000 7f6ae3400700 1 --2- 172.21.15.33:0/424446995 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f6ad00b2aa0 0x7f6ad00b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:03.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.946+0000 7f6ae3400700 1 -- 172.21.15.33:0/424446995 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 9518+0+0 (secure 0 0 0) 0x7f6ae40e2490 con 0x7f6aec114380 2024-05-03T18:45:03.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.947+0000 7f6aeaa00700 1 --2- 172.21.15.33:0/424446995 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f6ad00b2aa0 0x7f6ad00b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:03.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.948+0000 7f6aeaa00700 1 --2- 172.21.15.33:0/424446995 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f6ad00b2aa0 0x7f6ad00b4f60 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6adc011040 tx=0x7f6adc019040 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:45:03.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.953+0000 7f6afc428b80 1 Processor -- start 2024-05-03T18:45:03.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.954+0000 7f6afc428b80 1 -- start start 2024-05-03T18:45:03.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.954+0000 7f6afc428b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557811cc71a0 0x557811cc7580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:03.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.954+0000 7f6afc428b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557811cc7ac0 0x557811ccb500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:03.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.954+0000 7f6afc428b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557811ccba40 0x557811ccbe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:03.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.954+0000 7f6afc428b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x557811c08a20 con 0x557811ccba40 2024-05-03T18:45:03.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.954+0000 7f6afc428b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x557811ba6760 con 0x557811cc71a0 2024-05-03T18:45:03.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.954+0000 7f6afc428b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x557811c049e0 con 0x557811cc7ac0 2024-05-03T18:45:03.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.955+0000 7f6aea000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557811cc7ac0 0x557811ccb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:03.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.955+0000 7f6aea000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557811cc7ac0 0x557811ccb500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:45102/0 (socket says 172.21.15.33:45102) 2024-05-03T18:45:03.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.955+0000 7f6aea000700 1 -- 172.21.15.33:0/1488953747 learned_addr learned my addr 172.21.15.33:0/1488953747 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:45:03.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.955+0000 7f6aeaa00700 1 --2- 172.21.15.33:0/1488953747 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557811cc71a0 0x557811cc7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:03.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.956+0000 7f6aea000700 1 -- 172.21.15.33:0/1488953747 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557811cc71a0 msgr2=0x557811cc7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:45:03.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.956+0000 7f6aea000700 1 --2- 172.21.15.33:0/1488953747 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557811cc71a0 0x557811cc7580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:03.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.956+0000 7f6aea000700 1 -- 172.21.15.33:0/1488953747 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557811ccba40 msgr2=0x557811ccbe80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-03T18:45:03.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.956+0000 7f6aeb400700 1 --2- 172.21.15.33:0/1488953747 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557811ccba40 0x557811ccbe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:03.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.957+0000 7f6aea000700 1 --2- 172.21.15.33:0/1488953747 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557811ccba40 0x557811ccbe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:03.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.957+0000 7f6aea000700 1 -- 172.21.15.33:0/1488953747 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557811b558d0 con 0x557811cc7ac0 2024-05-03T18:45:03.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.959+0000 7f6aea000700 1 --2- 172.21.15.33:0/1488953747 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557811cc7ac0 0x557811ccb500 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f6ad8009c90 tx=0x7f6ad8010370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29c932d6092ce205 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:45:03.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.960+0000 7f6ad7e00700 1 -- 172.21.15.33:0/1488953747 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ad8010d00 con 0x557811cc7ac0 2024-05-03T18:45:03.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.961+0000 7f6ad7e00700 1 -- 172.21.15.33:0/1488953747 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6ad801b910 con 0x557811cc7ac0 2024-05-03T18:45:03.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.962+0000 7f6ad7e00700 1 -- 172.21.15.33:0/1488953747 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ad8012850 con 0x557811cc7ac0 2024-05-03T18:45:03.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.964+0000 7f6afc428b80 1 -- 172.21.15.33:0/1488953747 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557811cc7ac0 msgr2=0x557811ccb500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:03.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.965+0000 7f6afc428b80 1 --2- 172.21.15.33:0/1488953747 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557811cc7ac0 0x557811ccb500 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f6ad8009c90 tx=0x7f6ad8010370 comp rx=0 tx=0).stop 2024-05-03T18:45:03.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.965+0000 7f6afc428b80 1 -- 172.21.15.33:0/1488953747 shutdown_connections 2024-05-03T18:45:03.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.965+0000 7f6afc428b80 1 --2- 172.21.15.33:0/1488953747 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557811ccba40 0x557811ccbe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:03.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.965+0000 7f6afc428b80 1 --2- 172.21.15.33:0/1488953747 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557811cc71a0 0x557811cc7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:03.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.965+0000 7f6afc428b80 1 --2- 172.21.15.33:0/1488953747 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557811cc7ac0 0x557811ccb500 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:03.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.965+0000 7f6afc428b80 1 -- 172.21.15.33:0/1488953747 >> 172.21.15.33:0/1488953747 conn(0x557811be4b60 msgr2=0x557811cb07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:45:03.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.965+0000 7f6afc428b80 1 -- 172.21.15.33:0/1488953747 shutdown_connections 2024-05-03T18:45:03.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.966+0000 7f6afc428b80 1 -- 172.21.15.33:0/1488953747 wait complete. 2024-05-03T18:45:03.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.966+0000 7f6afc428b80 1 -- start start 2024-05-03T18:45:03.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.967+0000 7f6afc428b80 1 Processor -- start 2024-05-03T18:45:03.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.968+0000 7f6afc428b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557811cc71a0 0x557811cd2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:03.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.968+0000 7f6afc428b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557811cc7ac0 0x557811cd02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:03.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.968+0000 7f6afc428b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557811ccba40 0x557811cd07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:03.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.968+0000 7f6afc428b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x557811cb61f0 con 0x557811ccba40 2024-05-03T18:45:03.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.968+0000 7f6afc428b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x557811b6eb20 con 0x557811cc7ac0 2024-05-03T18:45:03.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.968+0000 7f6afc428b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x557811cb6370 con 0x557811cc71a0 2024-05-03T18:45:03.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.968+0000 7f6afc428b80 10 client.0 _ll_register_callbacks cb 0x557811bafc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:45:03.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.969+0000 7f6aeb400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557811ccba40 0x557811cd07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:03.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.970+0000 7f6aeb400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557811ccba40 0x557811cd07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45874/0 (socket says 172.21.15.33:45874) 2024-05-03T18:45:03.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.970+0000 7f6aeb400700 1 -- 172.21.15.33:0/2635143942 learned_addr learned my addr 172.21.15.33:0/2635143942 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:45:03.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.970+0000 7f6aeaa00700 1 --2- 172.21.15.33:0/2635143942 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557811cc71a0 0x557811cd2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:03.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.970+0000 7f6aeb400700 1 -- 172.21.15.33:0/2635143942 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557811cc71a0 msgr2=0x557811cd2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:45:03.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.972+0000 7f6aeb400700 1 --2- 172.21.15.33:0/2635143942 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557811cc71a0 0x557811cd2f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:03.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.972+0000 7f6aeb400700 1 -- 172.21.15.33:0/2635143942 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557811cc7ac0 msgr2=0x557811cd02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:45:03.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.972+0000 7f6aeb400700 1 --2- 172.21.15.33:0/2635143942 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557811cc7ac0 0x557811cd02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:03.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.972+0000 7f6aeb400700 1 -- 172.21.15.33:0/2635143942 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557811bce420 con 0x557811ccba40 2024-05-03T18:45:03.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.975+0000 7f6aeb400700 1 --2- 172.21.15.33:0/2635143942 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557811ccba40 0x557811cd07e0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f6ae40a6690 tx=0x7f6ae40b2040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3349a8d9851612 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:45:03.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.975+0000 7f6ad4c00700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:45:03.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.975+0000 7f6ad4c00700 1 -- 172.21.15.33:0/2635143942 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae4004d60 con 0x557811ccba40 2024-05-03T18:45:03.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.976+0000 7f6afc428b80 1 -- 172.21.15.33:0/2635143942 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ad801b910 con 0x557811ccba40 2024-05-03T18:45:03.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.977+0000 7f6ad4c00700 1 -- 172.21.15.33:0/2635143942 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9518+0+0 (secure 0 0 0) 0x7f6ae40a3b40 con 0x557811ccba40 2024-05-03T18:45:03.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.977+0000 7f6ad4c00700 1 -- 172.21.15.33:0/2635143942 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f6ae40a3d60 con 0x557811ccba40 2024-05-03T18:45:03.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.977+0000 7f6afc428b80 10 client.16908 fetch_fsmap learned FSMap version 21 2024-05-03T18:45:03.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.977+0000 7f6afc428b80 1 -- 172.21.15.33:0/2635143942 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x557811cc0b80 con 0x557811ccba40 2024-05-03T18:45:03.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.978+0000 7f6ad4c00700 1 -- 172.21.15.33:0/2635143942 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6ad801b910 con 0x557811ccba40 2024-05-03T18:45:03.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.979+0000 7f6ad4c00700 1 -- 172.21.15.33:0/2635143942 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f6ae40b0ac0 con 0x557811ccba40 2024-05-03T18:45:03.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.979+0000 7f6afc428b80 10 client.16908 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:45:03.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.979+0000 7f6afc428b80 10 client.16908 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:45:03.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.979+0000 7f6afc428b80 20 client.16908 populate_metadata read hostname 'smithi033' 2024-05-03T18:45:03.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.979+0000 7f6afc428b80 1 --2- 172.21.15.33:0/2635143942 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x557811cc31c0 0x557811cc5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:03.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.979+0000 7f6afc428b80 4 client.16908 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:45:03.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.979+0000 7f6afc428b80 1 -- 172.21.15.33:0/2635143942 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557811c831d0 con 0x557811cc31c0 2024-05-03T18:45:03.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.980+0000 7f6aeaa00700 1 --2- 172.21.15.33:0/2635143942 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x557811cc31c0 0x557811cc5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:03.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.981+0000 7f6aeaa00700 1 --2- 172.21.15.33:0/2635143942 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x557811cc31c0 0x557811cc5600 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=996af2fa1342932d server_cookie=bac38eed8eada5ea in_seq=0 out_seq=0 2024-05-03T18:45:03.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.982+0000 7f6ad4c00700 10 client.16908 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:45:03.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.984+0000 7f6ad4c00700 1 -- 172.21.15.33:0/2635143942 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557811c831d0 con 0x557811cc31c0 2024-05-03T18:45:03.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.984+0000 7f6ad4c00700 10 client.16908 handle_command_reply: tid=0 2024-05-03T18:45:03.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.985+0000 7f6afc428b80 1 client.16908 shutdown 2024-05-03T18:45:03.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.986+0000 7f6afc428b80 1 -- 172.21.15.33:0/2635143942 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557811ccba40 msgr2=0x557811cd07e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:03.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.986+0000 7f6afc428b80 1 --2- 172.21.15.33:0/2635143942 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557811ccba40 0x557811cd07e0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f6ae40a6690 tx=0x7f6ae40b2040 comp rx=0 tx=0).stop 2024-05-03T18:45:03.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.986+0000 7f6afc428b80 1 -- 172.21.15.33:0/2635143942 shutdown_connections 2024-05-03T18:45:03.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.986+0000 7f6afc428b80 1 --2- 172.21.15.33:0/2635143942 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x557811cc31c0 0x557811cc5600 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:03.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.986+0000 7f6afc428b80 1 --2- 172.21.15.33:0/2635143942 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557811ccba40 0x557811cd07e0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:03.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.986+0000 7f6afc428b80 1 --2- 172.21.15.33:0/2635143942 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557811cc71a0 0x557811cd2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:03.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.986+0000 7f6afc428b80 1 --2- 172.21.15.33:0/2635143942 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557811cc7ac0 0x557811cd02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:03.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.986+0000 7f6afc428b80 1 -- 172.21.15.33:0/2635143942 >> 172.21.15.33:0/2635143942 conn(0x557811be4b60 msgr2=0x557811cafee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:45:03.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.986+0000 7f6ad4c00700 0 client.16908 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:45:03.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.987+0000 7f6afc428b80 1 -- 172.21.15.33:0/2635143942 shutdown_connections 2024-05-03T18:45:03.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.988+0000 7f6afc428b80 1 -- 172.21.15.33:0/2635143942 wait complete. 2024-05-03T18:45:03.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:03.988+0000 7f6afc428b80 20 client.16908 trim_cache size 0 max 16384 2024-05-03T18:45:04.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:04.031+0000 7f6afc428b80 1 Processor -- start 2024-05-03T18:45:04.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:04.031+0000 7f6afc428b80 1 -- start start 2024-05-03T18:45:04.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:04.032+0000 7f6afc428b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557811cc71a0 0x557811cbf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:04.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:04.032+0000 7f6afc428b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557811cc7ac0 0x557811cb0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:04.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:04.032+0000 7f6afc428b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557811ccba40 0x557811cb56b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:04.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:04.032+0000 7f6afc428b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6aec073ce0 con 0x557811ccba40 2024-05-03T18:45:04.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:04.032+0000 7f6afc428b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6aec1add20 con 0x557811cc71a0 2024-05-03T18:45:04.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:04.032+0000 7f6afc428b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6aec074330 con 0x557811cc7ac0 2024-05-03T18:45:04.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:04.032+0000 7f6aea000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557811cc7ac0 0x557811cb0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:04.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:04.032+0000 7f6aeaa00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557811cc71a0 0x557811cbf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:04.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:04.032+0000 7f6aea000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557811cc7ac0 0x557811cb0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:45130/0 (socket says 172.21.15.33:45130) 2024-05-03T18:45:04.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:04.032+0000 7f6aea000700 1 -- 172.21.15.33:0/1748028728 learned_addr learned my addr 172.21.15.33:0/1748028728 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:45:04.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:04.032+0000 7f6aeb400700 1 --2- 172.21.15.33:0/1748028728 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557811ccba40 0x557811cb56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:04.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:04.033+0000 7f6aea000700 1 -- 172.21.15.33:0/1748028728 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557811cc71a0 msgr2=0x557811cbf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:45:04.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:04.033+0000 7f6aea000700 1 --2- 172.21.15.33:0/1748028728 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557811cc71a0 0x557811cbf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:04.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:04.033+0000 7f6aea000700 1 -- 172.21.15.33:0/1748028728 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557811ccba40 msgr2=0x557811cb56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:45:04.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:04.033+0000 7f6aea000700 1 --2- 172.21.15.33:0/1748028728 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557811ccba40 0x557811cb56b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:04.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:04.033+0000 7f6aea000700 1 -- 172.21.15.33:0/1748028728 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557811cd0070 con 0x557811cc7ac0 2024-05-03T18:45:04.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:04.033+0000 7f6aea000700 1 --2- 172.21.15.33:0/1748028728 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557811cc7ac0 0x557811cb0660 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x557811cb1450 tx=0x7f6ad8010c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=991af3ae0c7ee2e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:45:04.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:04.034+0000 7f6ad7400700 1 -- 172.21.15.33:0/1748028728 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ad8029070 con 0x557811cc7ac0 2024-05-03T18:45:04.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:04.034+0000 7f6ad7400700 1 -- 172.21.15.33:0/1748028728 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6ad8010dc0 con 0x557811cc7ac0 2024-05-03T18:45:04.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:04.034+0000 7f6ad7400700 1 -- 172.21.15.33:0/1748028728 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ad8004950 con 0x557811cc7ac0 2024-05-03T18:45:04.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:04.035+0000 7f6afc428b80 1 -- 172.21.15.33:0/1748028728 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557811cc7ac0 msgr2=0x557811cb0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:04.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:04.035+0000 7f6afc428b80 1 --2- 172.21.15.33:0/1748028728 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557811cc7ac0 0x557811cb0660 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x557811cb1450 tx=0x7f6ad8010c90 comp rx=0 tx=0).stop 2024-05-03T18:45:04.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:04.035+0000 7f6afc428b80 1 -- 172.21.15.33:0/1748028728 shutdown_connections 2024-05-03T18:45:04.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:04.035+0000 7f6afc428b80 1 --2- 172.21.15.33:0/1748028728 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557811ccba40 0x557811cb56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:04.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:04.035+0000 7f6afc428b80 1 --2- 172.21.15.33:0/1748028728 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557811cc71a0 0x557811cbf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:04.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:04.035+0000 7f6afc428b80 1 --2- 172.21.15.33:0/1748028728 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557811cc7ac0 0x557811cb0660 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:04.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:04.035+0000 7f6afc428b80 1 -- 172.21.15.33:0/1748028728 >> 172.21.15.33:0/1748028728 conn(0x557811be4b60 msgr2=0x557811cb4fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:45:04.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:04.035+0000 7f6afc428b80 1 -- 172.21.15.33:0/1748028728 shutdown_connections 2024-05-03T18:45:04.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:04.035+0000 7f6afc428b80 1 -- 172.21.15.33:0/1748028728 wait complete. 2024-05-03T18:45:04.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:04.036+0000 7f6afc428b80 1 -- start start 2024-05-03T18:45:04.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:04.036+0000 7f6afc428b80 1 Processor -- start 2024-05-03T18:45:04.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:04.037+0000 7f6afc428b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557811cc71a0 0x557811cbb990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:04.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:04.037+0000 7f6afc428b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557811cc7ac0 0x557811cd7f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:04.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:04.037+0000 7f6afc428b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557811ccba40 0x557811cd84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:04.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:04.037+0000 7f6afc428b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6aec074a00 con 0x557811ccba40 2024-05-03T18:45:04.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:04.037+0000 7f6afc428b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6aec1ad650 con 0x557811cc7ac0 2024-05-03T18:45:04.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:04.037+0000 7f6afc428b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6aec0750d0 con 0x557811cc71a0 2024-05-03T18:45:04.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:04.037+0000 7f6afc428b80 10 client.0 _ll_register_callbacks cb 0x557811bafc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:45:04.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:04.037+0000 7f6aeaa00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557811cc71a0 0x557811cbb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:04.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:04.037+0000 7f6aeaa00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557811cc71a0 0x557811cbb990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:45140/0 (socket says 172.21.15.33:45140) 2024-05-03T18:45:04.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:04.037+0000 7f6aeaa00700 1 -- 172.21.15.33:0/4208634459 learned_addr learned my addr 172.21.15.33:0/4208634459 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:45:04.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:04.037+0000 7f6aeb400700 1 --2- 172.21.15.33:0/4208634459 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557811ccba40 0x557811cd84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:04.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:04.037+0000 7f6aea000700 1 --2- 172.21.15.33:0/4208634459 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557811cc7ac0 0x557811cd7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:04.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:04.038+0000 7f6aeaa00700 1 -- 172.21.15.33:0/4208634459 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557811cc7ac0 msgr2=0x557811cd7f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:45:04.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:04.038+0000 7f6aeaa00700 1 --2- 172.21.15.33:0/4208634459 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557811cc7ac0 0x557811cd7f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:04.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:04.038+0000 7f6aeaa00700 1 -- 172.21.15.33:0/4208634459 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557811ccba40 msgr2=0x557811cd84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:45:04.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:04.038+0000 7f6aeaa00700 1 --2- 172.21.15.33:0/4208634459 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557811ccba40 0x557811cd84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:04.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:04.039+0000 7f6aeaa00700 1 -- 172.21.15.33:0/4208634459 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557811cd8b20 con 0x557811cc71a0 2024-05-03T18:45:04.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:04.039+0000 7f6aea000700 1 --2- 172.21.15.33:0/4208634459 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557811cc7ac0 0x557811cd7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:45:04.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:04.040+0000 7f6aeaa00700 1 --2- 172.21.15.33:0/4208634459 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557811cc71a0 0x557811cbb990 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f6adc009fd0 tx=0x7f6adc009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a82654c738791031 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:45:04.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:04.040+0000 7f6ad4c00700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-03T18:45:04.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:04.041+0000 7f6ad4c00700 1 -- 172.21.15.33:0/4208634459 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6adc003880 con 0x557811cc71a0 2024-05-03T18:45:04.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:04.041+0000 7f6ad4c00700 1 -- 172.21.15.33:0/4208634459 <== mon.1 v2:172.21.15.150:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9518+0+0 (secure 0 0 0) 0x7f6adc003a20 con 0x557811cc71a0 2024-05-03T18:45:04.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:04.041+0000 7f6afc428b80 1 -- 172.21.15.33:0/4208634459 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ad8010dc0 con 0x557811cc71a0 2024-05-03T18:45:04.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:04.041+0000 7f6ad4c00700 1 -- 172.21.15.33:0/4208634459 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f6adc074050 con 0x557811cc71a0 2024-05-03T18:45:04.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:04.041+0000 7f6afc428b80 10 client.26791 fetch_fsmap learned FSMap version 21 2024-05-03T18:45:04.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:04.041+0000 7f6afc428b80 1 -- 172.21.15.33:0/4208634459 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x557811d3b440 con 0x557811cc71a0 2024-05-03T18:45:04.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:04.042+0000 7f6ad4c00700 1 -- 172.21.15.33:0/4208634459 <== mon.1 v2:172.21.15.150:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6ad8010dc0 con 0x557811cc71a0 2024-05-03T18:45:04.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:04.043+0000 7f6ad4c00700 1 -- 172.21.15.33:0/4208634459 <== mon.1 v2:172.21.15.150:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f6adc050040 con 0x557811cc71a0 2024-05-03T18:45:04.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:04.043+0000 7f6afc428b80 10 client.26791 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:45:04.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:04.043+0000 7f6afc428b80 10 client.26791 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:45:04.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:04.043+0000 7f6afc428b80 20 client.26791 populate_metadata read hostname 'smithi033' 2024-05-03T18:45:04.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:04.043+0000 7f6afc428b80 1 --2- 172.21.15.33:0/4208634459 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x557811d3bb00 0x557811d3df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:04.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:04.043+0000 7f6afc428b80 4 client.26791 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-03T18:45:04.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:04.043+0000 7f6afc428b80 1 -- 172.21.15.33:0/4208634459 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557811c16400 con 0x557811d3bb00 2024-05-03T18:45:04.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:04.043+0000 7f6aea000700 1 --2- 172.21.15.33:0/4208634459 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x557811d3bb00 0x557811d3df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:04.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:04.044+0000 7f6aea000700 1 --2- 172.21.15.33:0/4208634459 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x557811d3bb00 0x557811d3df20 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9242a1eae0207080 server_cookie=7f8f97b1da23ba8b in_seq=0 out_seq=0 2024-05-03T18:45:04.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:04.044+0000 7f6ad4c00700 10 client.26791 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:45:04.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:04.047+0000 7f6ad4c00700 1 -- 172.21.15.33:0/4208634459 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x557811c16400 con 0x557811d3bb00 2024-05-03T18:45:04.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:04.047+0000 7f6ad4c00700 10 client.26791 handle_command_reply: tid=0 2024-05-03T18:45:04.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:04.047+0000 7f6afc428b80 1 client.26791 shutdown 2024-05-03T18:45:04.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:04.047+0000 7f6afc428b80 1 -- 172.21.15.33:0/4208634459 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557811cc71a0 msgr2=0x557811cbb990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:04.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:04.047+0000 7f6afc428b80 1 --2- 172.21.15.33:0/4208634459 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557811cc71a0 0x557811cbb990 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f6adc009fd0 tx=0x7f6adc009260 comp rx=0 tx=0).stop 2024-05-03T18:45:04.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:04.048+0000 7f6afc428b80 1 -- 172.21.15.33:0/4208634459 shutdown_connections 2024-05-03T18:45:04.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:04.048+0000 7f6afc428b80 1 --2- 172.21.15.33:0/4208634459 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x557811d3bb00 0x557811d3df20 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:04.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:04.048+0000 7f6afc428b80 1 --2- 172.21.15.33:0/4208634459 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557811ccba40 0x557811cd84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:04.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:04.048+0000 7f6afc428b80 1 --2- 172.21.15.33:0/4208634459 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557811cc71a0 0x557811cbb990 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:04.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:04.048+0000 7f6afc428b80 1 --2- 172.21.15.33:0/4208634459 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557811cc7ac0 0x557811cd7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:04.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:04.048+0000 7f6afc428b80 1 -- 172.21.15.33:0/4208634459 >> 172.21.15.33:0/4208634459 conn(0x557811be4b60 msgr2=0x557811cc6690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:45:04.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:04.049+0000 7f6ad4c00700 0 client.26791 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:45:04.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:04.050+0000 7f6afc428b80 1 -- 172.21.15.33:0/4208634459 shutdown_connections 2024-05-03T18:45:04.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:04.050+0000 7f6afc428b80 1 -- 172.21.15.33:0/4208634459 wait complete. 2024-05-03T18:45:04.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:04.050+0000 7f6afc428b80 20 client.26791 trim_cache size 0 max 16384 2024-05-03T18:45:04.052 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-03T18:45:04.052 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-05-03T18:45:04.052 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-05-03T18:45:04.052 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-03T18:45:04.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:04.052+0000 7f6af1a00700 1 -- 172.21.15.33:0/424446995 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f6ad00b2aa0 msgr2=0x7f6ad00b4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:04.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:04.052+0000 7f6af1a00700 1 --2- 172.21.15.33:0/424446995 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f6ad00b2aa0 0x7f6ad00b4f60 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6adc011040 tx=0x7f6adc019040 comp rx=0 tx=0).stop 2024-05-03T18:45:04.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:04.052+0000 7f6af1a00700 1 -- 172.21.15.33:0/424446995 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6aec114380 msgr2=0x7f6aec1a4a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:04.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:04.052+0000 7f6af1a00700 1 --2- 172.21.15.33:0/424446995 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6aec114380 0x7f6aec1a4a30 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f6aec07bcc0 tx=0x7f6ae400ef90 comp rx=0 tx=0).stop 2024-05-03T18:45:04.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:04.052+0000 7f6af1a00700 1 -- 172.21.15.33:0/424446995 shutdown_connections 2024-05-03T18:45:04.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:04.052+0000 7f6af1a00700 1 --2- 172.21.15.33:0/424446995 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6aec114380 0x7f6aec1a4a30 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:04.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:04.052+0000 7f6af1a00700 1 --2- 172.21.15.33:0/424446995 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f6ad00b2aa0 0x7f6ad00b4f60 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:04.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:04.052+0000 7f6af1a00700 1 --2- 172.21.15.33:0/424446995 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6aec0779d0 0x7f6aec1a0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:04.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:04.052+0000 7f6af1a00700 1 --2- 172.21.15.33:0/424446995 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6aec0782f0 0x7f6aec1a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:04.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:04.052+0000 7f6af1a00700 1 -- 172.21.15.33:0/424446995 >> 172.21.15.33:0/424446995 conn(0x7f6aec1011d0 msgr2=0x7f6aec1035c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:45:04.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:04.054+0000 7f6af1a00700 1 -- 172.21.15.33:0/424446995 shutdown_connections 2024-05-03T18:45:04.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:04.054+0000 7f6af1a00700 1 -- 172.21.15.33:0/424446995 wait complete. 2024-05-03T18:45:04.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:45:03 smithi099 ceph-mon[31658]: pgmap v562: 129 pgs: 129 active+clean; 3.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 5.3 MiB/s wr, 419 op/s 2024-05-03T18:45:04.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:45:03 smithi150 ceph-mon[32492]: pgmap v562: 129 pgs: 129 active+clean; 3.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 5.3 MiB/s wr, 419 op/s 2024-05-03T18:45:05.111 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 830.0 s, 509.6 tps, lat 192.829 ms stddev 187.921 2024-05-03T18:45:05.960 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-03T18:45:05.960 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T18:45:06.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:45:05 smithi033 ceph-mon[28330]: pgmap v563: 129 pgs: 129 active+clean; 3.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 6.7 MiB/s wr, 540 op/s 2024-05-03T18:45:06.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:45:05 smithi099 ceph-mon[31658]: pgmap v563: 129 pgs: 129 active+clean; 3.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 6.7 MiB/s wr, 540 op/s 2024-05-03T18:45:06.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:45:05 smithi150 ceph-mon[32492]: pgmap v563: 129 pgs: 129 active+clean; 3.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 6.7 MiB/s wr, 540 op/s 2024-05-03T18:45:06.698 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:45:08.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:45:07 smithi033 ceph-mon[28330]: pgmap v564: 129 pgs: 129 active+clean; 3.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 453 KiB/s rd, 6.1 MiB/s wr, 546 op/s 2024-05-03T18:45:08.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:45:07 smithi033 ceph-mon[28330]: osdmap e115: 12 total, 12 up, 12 in 2024-05-03T18:45:08.297 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:45:07 smithi099 ceph-mon[31658]: pgmap v564: 129 pgs: 129 active+clean; 3.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 453 KiB/s rd, 6.1 MiB/s wr, 546 op/s 2024-05-03T18:45:08.297 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:45:07 smithi099 ceph-mon[31658]: osdmap e115: 12 total, 12 up, 12 in 2024-05-03T18:45:08.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:45:07 smithi150 ceph-mon[32492]: pgmap v564: 129 pgs: 129 active+clean; 3.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 453 KiB/s rd, 6.1 MiB/s wr, 546 op/s 2024-05-03T18:45:08.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:45:07 smithi150 ceph-mon[32492]: osdmap e115: 12 total, 12 up, 12 in 2024-05-03T18:45:10.111 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 835.0 s, 504.0 tps, lat 203.652 ms stddev 103.418 2024-05-03T18:45:10.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:45:09 smithi033 ceph-mon[28330]: pgmap v566: 129 pgs: 129 active+clean; 3.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 6.1 MiB/s wr, 530 op/s 2024-05-03T18:45:10.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:45:09 smithi099 ceph-mon[31658]: pgmap v566: 129 pgs: 129 active+clean; 3.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 6.1 MiB/s wr, 530 op/s 2024-05-03T18:45:10.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:45:09 smithi150 ceph-mon[32492]: pgmap v566: 129 pgs: 129 active+clean; 3.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 6.1 MiB/s wr, 530 op/s 2024-05-03T18:45:11.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:11.802+0000 7f06d7000700 1 -- 172.21.15.33:0/389985134 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f06d0109ad0 msgr2=0x7f06d0109f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:11.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:11.802+0000 7f06d7000700 1 --2- 172.21.15.33:0/389985134 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f06d0109ad0 0x7f06d0109f50 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f06bc009990 tx=0x7f06bc009ca0 comp rx=0 tx=0).stop 2024-05-03T18:45:11.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:11.802+0000 7f06d7000700 1 -- 172.21.15.33:0/389985134 shutdown_connections 2024-05-03T18:45:11.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:11.802+0000 7f06d7000700 1 --2- 172.21.15.33:0/389985134 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f06d010a490 0x7f06d01169a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:11.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:11.802+0000 7f06d7000700 1 --2- 172.21.15.33:0/389985134 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f06d010fb60 0x7f06d010ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:11.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:11.802+0000 7f06d7000700 1 --2- 172.21.15.33:0/389985134 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f06d0109ad0 0x7f06d0109f50 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:11.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:11.802+0000 7f06d7000700 1 -- 172.21.15.33:0/389985134 >> 172.21.15.33:0/389985134 conn(0x7f06d0076c70 msgr2=0x7f06d01082c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:45:11.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:11.802+0000 7f06d7000700 1 -- 172.21.15.33:0/389985134 shutdown_connections 2024-05-03T18:45:11.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:11.802+0000 7f06d7000700 1 -- 172.21.15.33:0/389985134 wait complete. 2024-05-03T18:45:11.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:11.804+0000 7f06d7000700 1 Processor -- start 2024-05-03T18:45:11.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:11.804+0000 7f06d7000700 1 -- start start 2024-05-03T18:45:11.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:11.804+0000 7f06d7000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f06d0109ad0 0x7f06d01a0040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:11.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:11.804+0000 7f06d7000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f06d010a490 0x7f06d01a0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:11.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:11.804+0000 7f06d7000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f06d010fb60 0x7f06d01a4910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:11.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:11.804+0000 7f06d7000700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f06d0119bb0 con 0x7f06d010a490 2024-05-03T18:45:11.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:11.804+0000 7f06d7000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f06d0119a30 con 0x7f06d0109ad0 2024-05-03T18:45:11.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:11.804+0000 7f06d7000700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f06d0119d30 con 0x7f06d010fb60 2024-05-03T18:45:11.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:11.805+0000 7f06cfe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f06d0109ad0 0x7f06d01a0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:11.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:11.805+0000 7f06cfe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f06d0109ad0 0x7f06d01a0040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:35012/0 (socket says 172.21.15.33:35012) 2024-05-03T18:45:11.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:11.805+0000 7f06cfe00700 1 -- 172.21.15.33:0/582590809 learned_addr learned my addr 172.21.15.33:0/582590809 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:45:11.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:11.805+0000 7f06cf400700 1 --2- 172.21.15.33:0/582590809 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f06d010a490 0x7f06d01a0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:11.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:11.805+0000 7f06d5000700 1 --2- 172.21.15.33:0/582590809 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f06d010fb60 0x7f06d01a4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:11.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:11.806+0000 7f06cfe00700 1 -- 172.21.15.33:0/582590809 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f06d010fb60 msgr2=0x7f06d01a4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:11.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:11.806+0000 7f06cfe00700 1 --2- 172.21.15.33:0/582590809 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f06d010fb60 0x7f06d01a4910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:11.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:11.806+0000 7f06cfe00700 1 -- 172.21.15.33:0/582590809 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f06d010a490 msgr2=0x7f06d01a0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:11.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:11.806+0000 7f06cfe00700 1 --2- 172.21.15.33:0/582590809 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f06d010a490 0x7f06d01a0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:11.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:11.806+0000 7f06cfe00700 1 -- 172.21.15.33:0/582590809 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06d01a5050 con 0x7f06d0109ad0 2024-05-03T18:45:11.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:11.807+0000 7f06cfe00700 1 --2- 172.21.15.33:0/582590809 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f06d0109ad0 0x7f06d01a0040 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f06c000c9b0 tx=0x7f06c000ccc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:45:11.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:11.807+0000 7f06ccc00700 1 -- 172.21.15.33:0/582590809 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c0007b00 con 0x7f06d0109ad0 2024-05-03T18:45:11.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:11.807+0000 7f06ccc00700 1 -- 172.21.15.33:0/582590809 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f06c0007ca0 con 0x7f06d0109ad0 2024-05-03T18:45:11.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:11.807+0000 7f06ccc00700 1 -- 172.21.15.33:0/582590809 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c0018660 con 0x7f06d0109ad0 2024-05-03T18:45:11.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:11.808+0000 7f06d7000700 1 -- 172.21.15.33:0/582590809 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06d01a5340 con 0x7f06d0109ad0 2024-05-03T18:45:11.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:11.808+0000 7f06d7000700 1 -- 172.21.15.33:0/582590809 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06d01acbc0 con 0x7f06d0109ad0 2024-05-03T18:45:11.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:11.810+0000 7f06d7000700 1 -- 172.21.15.33:0/582590809 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06d006a190 con 0x7f06d0109ad0 2024-05-03T18:45:11.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:11.813+0000 7f06ccc00700 1 -- 172.21.15.33:0/582590809 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f06c000f450 con 0x7f06d0109ad0 2024-05-03T18:45:11.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:11.816+0000 7f06ccc00700 1 --2- 172.21.15.33:0/582590809 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f06b40b29f0 0x7f06b40b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:11.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:11.816+0000 7f06ccc00700 1 -- 172.21.15.33:0/582590809 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9606+0+0 (secure 0 0 0) 0x7f06c00e2a80 con 0x7f06d0109ad0 2024-05-03T18:45:11.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:11.817+0000 7f06cf400700 1 --2- 172.21.15.33:0/582590809 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f06b40b29f0 0x7f06b40b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:11.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:11.818+0000 7f06cf400700 1 --2- 172.21.15.33:0/582590809 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f06b40b29f0 0x7f06b40b4eb0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f06d0078b70 tx=0x7f06bc00b260 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:45:11.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:11.828+0000 7f06ccc00700 1 -- 172.21.15.33:0/582590809 <== mon.2 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f06c00e4e20 con 0x7f06d0109ad0 2024-05-03T18:45:12.236 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:45:11 smithi033 ceph-mon[28330]: pgmap v567: 129 pgs: 129 active+clean; 3.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 6.2 MiB/s wr, 550 op/s 2024-05-03T18:45:12.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:45:11 smithi099 ceph-mon[31658]: pgmap v567: 129 pgs: 129 active+clean; 3.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 6.2 MiB/s wr, 550 op/s 2024-05-03T18:45:12.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:12.308+0000 7f06d7000700 1 -- 172.21.15.33:0/582590809 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f06d01a13a0 con 0x7f06d0109ad0 2024-05-03T18:45:12.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:12.310+0000 7f06ccc00700 1 -- 172.21.15.33:0/582590809 <== mon.2 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11801 (secure 0 0 0) 0x7f06c0014030 con 0x7f06d0109ad0 2024-05-03T18:45:12.311 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:45:12.314 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1327393503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1327393503},{"type":"v1","addr":"172.21.15.33:6835","nonce":1327393503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/2461472948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":2461472948},{"type":"v1","addr":"172.21.15.33:6837","nonce":2461472948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14559,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/442269281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":442269281},{"type":"v1","addr":"172.21.15.33:6839","nonce":442269281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14571,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/645571756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":645571756},{"type":"v1","addr":"172.21.15.33:6841","nonce":645571756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/203598909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":203598909},{"type":"v1","addr":"172.21.15.150:6833","nonce":203598909}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/4029842493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":4029842493},{"type":"v1","addr":"172.21.15.150:6835","nonce":4029842493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3463664423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3463664423},{"type":"v1","addr":"172.21.15.150:6837","nonce":3463664423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24481,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6839/4051648018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":4051648018},{"type":"v1","addr":"172.21.15.150:6839","nonce":4051648018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34418,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1865105479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1865105479},{"type":"v1","addr":"172.21.15.99:6833","nonce":1865105479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34445,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/1206393914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1206393914},{"type":"v1","addr":"172.21.15.99:6835","nonce":1206393914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-03T18:22:28.837091+0000","modified":"2024-05-03T18:23:49.758398+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34508},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34472":{"gid":34472,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6837/3360970342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3360970342},{"type":"v1","addr":"172.21.15.99:6837","nonce":3360970342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34508":{"gid":34508,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/919907308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":919907308},{"type":"v1","addr":"172.21.15.99:6839","nonce":919907308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34508,"qdb_cluster":[34508]},"id":1}]} 2024-05-03T18:45:12.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:12.313+0000 7f06d7000700 1 -- 172.21.15.33:0/582590809 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f06b40b29f0 msgr2=0x7f06b40b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:12.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:12.313+0000 7f06d7000700 1 --2- 172.21.15.33:0/582590809 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f06b40b29f0 0x7f06b40b4eb0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f06d0078b70 tx=0x7f06bc00b260 comp rx=0 tx=0).stop 2024-05-03T18:45:12.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:12.313+0000 7f06d7000700 1 -- 172.21.15.33:0/582590809 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f06d0109ad0 msgr2=0x7f06d01a0040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:12.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:12.313+0000 7f06d7000700 1 --2- 172.21.15.33:0/582590809 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f06d0109ad0 0x7f06d01a0040 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f06c000c9b0 tx=0x7f06c000ccc0 comp rx=0 tx=0).stop 2024-05-03T18:45:12.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:12.314+0000 7f06d7000700 1 -- 172.21.15.33:0/582590809 shutdown_connections 2024-05-03T18:45:12.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:12.314+0000 7f06d7000700 1 --2- 172.21.15.33:0/582590809 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f06d010fb60 0x7f06d01a4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:12.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:12.314+0000 7f06d7000700 1 --2- 172.21.15.33:0/582590809 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f06b40b29f0 0x7f06b40b4eb0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:12.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:12.314+0000 7f06d7000700 1 --2- 172.21.15.33:0/582590809 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f06d0109ad0 0x7f06d01a0040 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:12.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:12.314+0000 7f06d7000700 1 --2- 172.21.15.33:0/582590809 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f06d010a490 0x7f06d01a0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:12.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:12.314+0000 7f06d7000700 1 -- 172.21.15.33:0/582590809 >> 172.21.15.33:0/582590809 conn(0x7f06d0076c70 msgr2=0x7f06d0077230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:45:12.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:12.315+0000 7f06d7000700 1 -- 172.21.15.33:0/582590809 shutdown_connections 2024-05-03T18:45:12.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:12.316+0000 7f06d7000700 1 -- 172.21.15.33:0/582590809 wait complete. 2024-05-03T18:45:12.319 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 21 2024-05-03T18:45:12.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:45:11 smithi150 ceph-mon[32492]: pgmap v567: 129 pgs: 129 active+clean; 3.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 6.2 MiB/s wr, 550 op/s 2024-05-03T18:45:13.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:45:12 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/582590809' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:45:13.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:45:12 smithi033 ceph-mon[28330]: osdmap e116: 12 total, 12 up, 12 in 2024-05-03T18:45:13.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:45:12 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/582590809' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:45:13.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:45:12 smithi099 ceph-mon[31658]: osdmap e116: 12 total, 12 up, 12 in 2024-05-03T18:45:13.358 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-03T18:45:13.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:45:12 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/582590809' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:45:13.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:45:12 smithi150 ceph-mon[32492]: osdmap e116: 12 total, 12 up, 12 in 2024-05-03T18:45:13.945 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:45:14.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:45:13 smithi033 ceph-mon[28330]: pgmap v568: 129 pgs: 129 active+clean; 3.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 6.2 MiB/s wr, 551 op/s 2024-05-03T18:45:14.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:45:13 smithi099 ceph-mon[31658]: pgmap v568: 129 pgs: 129 active+clean; 3.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 6.2 MiB/s wr, 551 op/s 2024-05-03T18:45:14.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:45:13 smithi150 ceph-mon[32492]: pgmap v568: 129 pgs: 129 active+clean; 3.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 6.2 MiB/s wr, 551 op/s 2024-05-03T18:45:15.111 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 840.0 s, 585.8 tps, lat 170.615 ms stddev 136.163 2024-05-03T18:45:16.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:45:15 smithi033 ceph-mon[28330]: pgmap v570: 129 pgs: 129 active+clean; 3.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 752 KiB/s rd, 6.9 MiB/s wr, 513 op/s 2024-05-03T18:45:16.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:45:15 smithi099 ceph-mon[31658]: pgmap v570: 129 pgs: 129 active+clean; 3.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 752 KiB/s rd, 6.9 MiB/s wr, 513 op/s 2024-05-03T18:45:16.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:45:15 smithi150 ceph-mon[32492]: pgmap v570: 129 pgs: 129 active+clean; 3.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 752 KiB/s rd, 6.9 MiB/s wr, 513 op/s 2024-05-03T18:45:18.236 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:45:17 smithi033 ceph-mon[28330]: pgmap v571: 129 pgs: 129 active+clean; 3.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 783 KiB/s rd, 7.0 MiB/s wr, 543 op/s 2024-05-03T18:45:18.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:45:17 smithi099 ceph-mon[31658]: pgmap v571: 129 pgs: 129 active+clean; 3.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 783 KiB/s rd, 7.0 MiB/s wr, 543 op/s 2024-05-03T18:45:18.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:45:17 smithi150 ceph-mon[32492]: pgmap v571: 129 pgs: 129 active+clean; 3.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 783 KiB/s rd, 7.0 MiB/s wr, 543 op/s 2024-05-03T18:45:18.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.624+0000 7f97ca600700 1 -- 172.21.15.33:0/3139012613 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97c4105690 msgr2=0x7f97c4105b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:18.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.624+0000 7f97ca600700 1 --2- 172.21.15.33:0/3139012613 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97c4105690 0x7f97c4105b10 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f97b4009990 tx=0x7f97b4009ca0 comp rx=0 tx=0).stop 2024-05-03T18:45:18.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.625+0000 7f97ca600700 1 -- 172.21.15.33:0/3139012613 shutdown_connections 2024-05-03T18:45:18.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.625+0000 7f97ca600700 1 --2- 172.21.15.33:0/3139012613 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97c4106050 0x7f97c4112560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:18.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.625+0000 7f97ca600700 1 --2- 172.21.15.33:0/3139012613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97c410b690 0x7f97c410ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:18.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.625+0000 7f97ca600700 1 --2- 172.21.15.33:0/3139012613 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97c4105690 0x7f97c4105b10 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:18.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.625+0000 7f97ca600700 1 -- 172.21.15.33:0/3139012613 >> 172.21.15.33:0/3139012613 conn(0x7f97c4101180 msgr2=0x7f97c41035a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:45:18.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.625+0000 7f97ca600700 1 -- 172.21.15.33:0/3139012613 shutdown_connections 2024-05-03T18:45:18.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.625+0000 7f97ca600700 1 -- 172.21.15.33:0/3139012613 wait complete. 2024-05-03T18:45:18.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.626+0000 7f97ca600700 1 Processor -- start 2024-05-03T18:45:18.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.627+0000 7f97ca600700 1 -- start start 2024-05-03T18:45:18.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.627+0000 7f97ca600700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97c4105690 0x7f97c4074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:18.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.628+0000 7f97ca600700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97c4106050 0x7f97c406ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:18.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.628+0000 7f97ca600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97c410b690 0x7f97c406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:18.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.628+0000 7f97ca600700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f97c41157d0 con 0x7f97c4106050 2024-05-03T18:45:18.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.628+0000 7f97ca600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f97c4115650 con 0x7f97c410b690 2024-05-03T18:45:18.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.628+0000 7f97ca600700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f97c4115950 con 0x7f97c4105690 2024-05-03T18:45:18.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.628+0000 7f97c3400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97c4105690 0x7f97c4074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:18.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.628+0000 7f97c3e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97c410b690 0x7f97c406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:18.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.628+0000 7f97c3e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97c410b690 0x7f97c406e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:54834/0 (socket says 172.21.15.33:54834) 2024-05-03T18:45:18.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.628+0000 7f97c3400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97c4105690 0x7f97c4074cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:37822/0 (socket says 172.21.15.33:37822) 2024-05-03T18:45:18.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.628+0000 7f97c3e00700 1 -- 172.21.15.33:0/3514910646 learned_addr learned my addr 172.21.15.33:0/3514910646 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:45:18.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.628+0000 7f97c2a00700 1 --2- 172.21.15.33:0/3514910646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97c4106050 0x7f97c406ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:18.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.629+0000 7f97c3400700 1 -- 172.21.15.33:0/3514910646 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97c410b690 msgr2=0x7f97c406e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:18.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.629+0000 7f97c3400700 1 --2- 172.21.15.33:0/3514910646 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97c410b690 0x7f97c406e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:18.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.629+0000 7f97c3400700 1 -- 172.21.15.33:0/3514910646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97c4106050 msgr2=0x7f97c406ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:18.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.629+0000 7f97c3400700 1 --2- 172.21.15.33:0/3514910646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97c4106050 0x7f97c406ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:18.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.629+0000 7f97c3400700 1 -- 172.21.15.33:0/3514910646 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97c406e930 con 0x7f97c4105690 2024-05-03T18:45:18.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.629+0000 7f97c3e00700 1 --2- 172.21.15.33:0/3514910646 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97c410b690 0x7f97c406e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:45:18.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.629+0000 7f97c3400700 1 --2- 172.21.15.33:0/3514910646 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97c4105690 0x7f97c4074cd0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f97bc00ef10 tx=0x7f97bc00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:45:18.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.630+0000 7f97bbe00700 1 -- 172.21.15.33:0/3514910646 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97bc00c9c0 con 0x7f97c4105690 2024-05-03T18:45:18.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.630+0000 7f97ca600700 1 -- 172.21.15.33:0/3514910646 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97c406ebc0 con 0x7f97c4105690 2024-05-03T18:45:18.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.630+0000 7f97bbe00700 1 -- 172.21.15.33:0/3514910646 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f97bc019070 con 0x7f97c4105690 2024-05-03T18:45:18.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.630+0000 7f97bbe00700 1 -- 172.21.15.33:0/3514910646 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97bc014480 con 0x7f97c4105690 2024-05-03T18:45:18.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.630+0000 7f97ca600700 1 -- 172.21.15.33:0/3514910646 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97c4072b90 con 0x7f97c4105690 2024-05-03T18:45:18.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.636+0000 7f97bbe00700 1 -- 172.21.15.33:0/3514910646 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f97bc012050 con 0x7f97c4105690 2024-05-03T18:45:18.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.638+0000 7f97bbe00700 1 --2- 172.21.15.33:0/3514910646 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f97a80b2af0 0x7f97a80b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:18.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.639+0000 7f97bbe00700 1 -- 172.21.15.33:0/3514910646 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 9518+0+0 (secure 0 0 0) 0x7f97bc0e2960 con 0x7f97c4105690 2024-05-03T18:45:18.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.639+0000 7f97c2a00700 1 --2- 172.21.15.33:0/3514910646 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f97a80b2af0 0x7f97a80b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:18.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.641+0000 7f97c2a00700 1 --2- 172.21.15.33:0/3514910646 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f97a80b2af0 0x7f97a80b4fb0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f97c406f5b0 tx=0x7f97b4005fb0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:45:18.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.650+0000 7f97d4feab80 1 Processor -- start 2024-05-03T18:45:18.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.650+0000 7f97d4feab80 1 -- start start 2024-05-03T18:45:18.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.650+0000 7f97d4feab80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561336a29930 0x561336a29d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:18.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.650+0000 7f97d4feab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561336a2a250 0x561336a23f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:18.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.650+0000 7f97d4feab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561336a24550 0x561336a249d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:18.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.650+0000 7f97d4feab80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x561336976a20 con 0x561336a29930 2024-05-03T18:45:18.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.650+0000 7f97d4feab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x561336914760 con 0x561336a24550 2024-05-03T18:45:18.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.650+0000 7f97d4feab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5613369729e0 con 0x561336a2a250 2024-05-03T18:45:18.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.651+0000 7f97c3e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561336a24550 0x561336a249d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:18.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.651+0000 7f97c3e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561336a24550 0x561336a249d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:54836/0 (socket says 172.21.15.33:54836) 2024-05-03T18:45:18.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.651+0000 7f97c3400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561336a29930 0x561336a29d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:18.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.652+0000 7f97c3e00700 1 -- 172.21.15.33:0/1369258351 learned_addr learned my addr 172.21.15.33:0/1369258351 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:45:18.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.652+0000 7f97c2a00700 1 --2- 172.21.15.33:0/1369258351 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561336a2a250 0x561336a23f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:18.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.652+0000 7f97c3e00700 1 -- 172.21.15.33:0/1369258351 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561336a2a250 msgr2=0x561336a23f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:45:18.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.652+0000 7f97c3e00700 1 --2- 172.21.15.33:0/1369258351 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561336a2a250 0x561336a23f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:18.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.652+0000 7f97c3e00700 1 -- 172.21.15.33:0/1369258351 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561336a29930 msgr2=0x561336a29d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:45:18.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.652+0000 7f97c3e00700 1 --2- 172.21.15.33:0/1369258351 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561336a29930 0x561336a29d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:18.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.652+0000 7f97c3e00700 1 -- 172.21.15.33:0/1369258351 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613368c38d0 con 0x561336a24550 2024-05-03T18:45:18.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.653+0000 7f97c3e00700 1 --2- 172.21.15.33:0/1369258351 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561336a24550 0x561336a249d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f97ac009f30 tx=0x7f97ac00e4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4daf65b535ecfa7 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:45:18.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.654+0000 7f97b8c00700 1 -- 172.21.15.33:0/1369258351 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97ac017070 con 0x561336a24550 2024-05-03T18:45:18.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.654+0000 7f97b8c00700 1 -- 172.21.15.33:0/1369258351 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f97ac00ece0 con 0x561336a24550 2024-05-03T18:45:18.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.654+0000 7f97b8c00700 1 -- 172.21.15.33:0/1369258351 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97ac012630 con 0x561336a24550 2024-05-03T18:45:18.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.655+0000 7f97d4feab80 1 -- 172.21.15.33:0/1369258351 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561336a24550 msgr2=0x561336a249d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:18.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.655+0000 7f97d4feab80 1 --2- 172.21.15.33:0/1369258351 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561336a24550 0x561336a249d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f97ac009f30 tx=0x7f97ac00e4b0 comp rx=0 tx=0).stop 2024-05-03T18:45:18.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.656+0000 7f97d4feab80 1 -- 172.21.15.33:0/1369258351 shutdown_connections 2024-05-03T18:45:18.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.656+0000 7f97d4feab80 1 --2- 172.21.15.33:0/1369258351 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561336a24550 0x561336a249d0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:18.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.656+0000 7f97d4feab80 1 --2- 172.21.15.33:0/1369258351 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561336a29930 0x561336a29d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:18.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.656+0000 7f97d4feab80 1 --2- 172.21.15.33:0/1369258351 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561336a2a250 0x561336a23f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:18.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.656+0000 7f97d4feab80 1 -- 172.21.15.33:0/1369258351 >> 172.21.15.33:0/1369258351 conn(0x56133699c6d0 msgr2=0x561336a1e900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:45:18.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.656+0000 7f97d4feab80 1 -- 172.21.15.33:0/1369258351 shutdown_connections 2024-05-03T18:45:18.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.657+0000 7f97d4feab80 1 -- 172.21.15.33:0/1369258351 wait complete. 2024-05-03T18:45:18.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.657+0000 7f97d4feab80 1 -- start start 2024-05-03T18:45:18.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.658+0000 7f97d4feab80 1 Processor -- start 2024-05-03T18:45:18.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.658+0000 7f97d4feab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561336a24550 0x561336a2ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:18.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.658+0000 7f97d4feab80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561336a29930 0x561336a2d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:18.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.658+0000 7f97d4feab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561336a2a250 0x561336a2d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:18.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.658+0000 7f97d4feab80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x561336a30d00 con 0x561336a29930 2024-05-03T18:45:18.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.658+0000 7f97d4feab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5613368dcb20 con 0x561336a2a250 2024-05-03T18:45:18.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.658+0000 7f97d4feab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x561336a30e80 con 0x561336a24550 2024-05-03T18:45:18.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.658+0000 7f97d4feab80 10 client.0 _ll_register_callbacks cb 0x56133691dc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:45:18.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.659+0000 7f97c2a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561336a29930 0x561336a2d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:18.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.659+0000 7f97c3e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561336a2a250 0x561336a2d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:18.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.659+0000 7f97c3400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561336a24550 0x561336a2ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:18.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.659+0000 7f97c3e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561336a2a250 0x561336a2d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:54850/0 (socket says 172.21.15.33:54850) 2024-05-03T18:45:18.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.659+0000 7f97c3e00700 1 -- 172.21.15.33:0/4170333711 learned_addr learned my addr 172.21.15.33:0/4170333711 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:45:18.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.660+0000 7f97c3e00700 1 -- 172.21.15.33:0/4170333711 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561336a24550 msgr2=0x561336a2ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:45:18.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.660+0000 7f97c3e00700 1 --2- 172.21.15.33:0/4170333711 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561336a24550 0x561336a2ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:18.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.660+0000 7f97c3e00700 1 -- 172.21.15.33:0/4170333711 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561336a29930 msgr2=0x561336a2d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:45:18.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.660+0000 7f97c3e00700 1 --2- 172.21.15.33:0/4170333711 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561336a29930 0x561336a2d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:18.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.660+0000 7f97c3e00700 1 -- 172.21.15.33:0/4170333711 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56133693c420 con 0x561336a2a250 2024-05-03T18:45:18.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.660+0000 7f97c3e00700 1 --2- 172.21.15.33:0/4170333711 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561336a2a250 0x561336a2d820 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f97ac006eb0 tx=0x7f97ac006ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20b462e342bf6376 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:45:18.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.660+0000 7f97b1600700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-03T18:45:18.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.660+0000 7f97b1600700 1 -- 172.21.15.33:0/4170333711 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97ac023020 con 0x561336a2a250 2024-05-03T18:45:18.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.661+0000 7f97d4feab80 1 -- 172.21.15.33:0/4170333711 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97ac00ece0 con 0x561336a2a250 2024-05-03T18:45:18.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.661+0000 7f97b1600700 1 -- 172.21.15.33:0/4170333711 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9518+0+0 (secure 0 0 0) 0x7f97ac01c860 con 0x561336a2a250 2024-05-03T18:45:18.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.661+0000 7f97b1600700 1 -- 172.21.15.33:0/4170333711 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f97ac02e050 con 0x561336a2a250 2024-05-03T18:45:18.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.661+0000 7f97d4feab80 10 client.36731 fetch_fsmap learned FSMap version 21 2024-05-03T18:45:18.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.661+0000 7f97d4feab80 1 -- 172.21.15.33:0/4170333711 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x561336a3b500 con 0x561336a2a250 2024-05-03T18:45:18.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.662+0000 7f97b1600700 1 -- 172.21.15.33:0/4170333711 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f97ac00ece0 con 0x561336a2a250 2024-05-03T18:45:18.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.662+0000 7f97b1600700 1 -- 172.21.15.33:0/4170333711 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f97ac027a90 con 0x561336a2a250 2024-05-03T18:45:18.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.662+0000 7f97d4feab80 10 client.36731 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:45:18.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.662+0000 7f97d4feab80 10 client.36731 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:45:18.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.662+0000 7f97d4feab80 20 client.36731 populate_metadata read hostname 'smithi033' 2024-05-03T18:45:18.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.662+0000 7f97d4feab80 1 --2- 172.21.15.33:0/4170333711 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x561336a3d9d0 0x561336a3fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:18.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.662+0000 7f97d4feab80 4 client.36731 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:45:18.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.662+0000 7f97d4feab80 1 -- 172.21.15.33:0/4170333711 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5613369f0ff0 con 0x561336a3d9d0 2024-05-03T18:45:18.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.663+0000 7f97c3400700 1 --2- 172.21.15.33:0/4170333711 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x561336a3d9d0 0x561336a3fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:18.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.663+0000 7f97c3400700 1 --2- 172.21.15.33:0/4170333711 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x561336a3d9d0 0x561336a3fdc0 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ddfd82cbe539a72a server_cookie=1d0f1b53a15a41be in_seq=0 out_seq=0 2024-05-03T18:45:18.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.663+0000 7f97b1600700 10 client.36731 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:45:18.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.665+0000 7f97b1600700 1 -- 172.21.15.33:0/4170333711 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5613369f0ff0 con 0x561336a3d9d0 2024-05-03T18:45:18.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.665+0000 7f97b1600700 10 client.36731 handle_command_reply: tid=0 2024-05-03T18:45:18.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.665+0000 7f97d4feab80 1 client.36731 shutdown 2024-05-03T18:45:18.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.665+0000 7f97d4feab80 1 -- 172.21.15.33:0/4170333711 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561336a2a250 msgr2=0x561336a2d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:18.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.665+0000 7f97d4feab80 1 --2- 172.21.15.33:0/4170333711 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561336a2a250 0x561336a2d820 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f97ac006eb0 tx=0x7f97ac006ee0 comp rx=0 tx=0).stop 2024-05-03T18:45:18.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.666+0000 7f97d4feab80 1 -- 172.21.15.33:0/4170333711 shutdown_connections 2024-05-03T18:45:18.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.666+0000 7f97d4feab80 1 --2- 172.21.15.33:0/4170333711 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x561336a3d9d0 0x561336a3fdc0 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:18.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.666+0000 7f97b1600700 0 client.36731 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:45:18.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.666+0000 7f97d4feab80 1 --2- 172.21.15.33:0/4170333711 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561336a2a250 0x561336a2d820 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:18.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.666+0000 7f97d4feab80 1 --2- 172.21.15.33:0/4170333711 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561336a24550 0x561336a2ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:18.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.666+0000 7f97d4feab80 1 --2- 172.21.15.33:0/4170333711 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561336a29930 0x561336a2d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:18.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.666+0000 7f97d4feab80 1 -- 172.21.15.33:0/4170333711 >> 172.21.15.33:0/4170333711 conn(0x56133699c6d0 msgr2=0x561336a193b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:45:18.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.667+0000 7f97d4feab80 1 -- 172.21.15.33:0/4170333711 shutdown_connections 2024-05-03T18:45:18.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.667+0000 7f97d4feab80 1 -- 172.21.15.33:0/4170333711 wait complete. 2024-05-03T18:45:18.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.667+0000 7f97d4feab80 20 client.36731 trim_cache size 0 max 16384 2024-05-03T18:45:18.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.701+0000 7f97d4feab80 1 Processor -- start 2024-05-03T18:45:18.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.701+0000 7f97d4feab80 1 -- start start 2024-05-03T18:45:18.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.702+0000 7f97d4feab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561336a24550 0x561336a23f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:18.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.702+0000 7f97d4feab80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561336a29930 0x561336a39d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:18.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.702+0000 7f97d4feab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561336a2a250 0x561336a2adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:18.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.702+0000 7f97d4feab80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f97c41b3cf0 con 0x561336a29930 2024-05-03T18:45:18.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.702+0000 7f97d4feab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f97c41b6550 con 0x561336a24550 2024-05-03T18:45:18.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.702+0000 7f97d4feab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f97c41b4340 con 0x561336a2a250 2024-05-03T18:45:18.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.703+0000 7f97c2a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561336a29930 0x561336a39d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:18.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.703+0000 7f97c3e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561336a2a250 0x561336a2adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:18.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.703+0000 7f97c3400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561336a24550 0x561336a23f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:18.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.703+0000 7f97c3400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561336a24550 0x561336a23f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:54856/0 (socket says 172.21.15.33:54856) 2024-05-03T18:45:18.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.703+0000 7f97c3400700 1 -- 172.21.15.33:0/2349664122 learned_addr learned my addr 172.21.15.33:0/2349664122 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:45:18.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.704+0000 7f97c3400700 1 -- 172.21.15.33:0/2349664122 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561336a2a250 msgr2=0x561336a2adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:45:18.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.704+0000 7f97c3400700 1 --2- 172.21.15.33:0/2349664122 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561336a2a250 0x561336a2adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:18.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.704+0000 7f97c3400700 1 -- 172.21.15.33:0/2349664122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561336a29930 msgr2=0x561336a39d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:45:18.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.704+0000 7f97c3400700 1 --2- 172.21.15.33:0/2349664122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561336a29930 0x561336a39d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:18.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.704+0000 7f97c3400700 1 -- 172.21.15.33:0/2349664122 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561336a30e70 con 0x561336a24550 2024-05-03T18:45:18.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.705+0000 7f97c3400700 1 --2- 172.21.15.33:0/2349664122 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561336a24550 0x561336a23f30 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f97bc00eee0 tx=0x7f97bc0075b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fcc9b11c47f29d2c server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:45:18.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.706+0000 7f97b3e00700 1 -- 172.21.15.33:0/2349664122 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97bc0a9f00 con 0x561336a24550 2024-05-03T18:45:18.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.706+0000 7f97b3e00700 1 -- 172.21.15.33:0/2349664122 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f97bc0a9800 con 0x561336a24550 2024-05-03T18:45:18.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.706+0000 7f97b3e00700 1 -- 172.21.15.33:0/2349664122 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97bc0dfef0 con 0x561336a24550 2024-05-03T18:45:18.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.710+0000 7f97d4feab80 1 -- 172.21.15.33:0/2349664122 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561336a24550 msgr2=0x561336a23f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:18.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.710+0000 7f97d4feab80 1 --2- 172.21.15.33:0/2349664122 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561336a24550 0x561336a23f30 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f97bc00eee0 tx=0x7f97bc0075b0 comp rx=0 tx=0).stop 2024-05-03T18:45:18.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.711+0000 7f97d4feab80 1 -- 172.21.15.33:0/2349664122 shutdown_connections 2024-05-03T18:45:18.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.711+0000 7f97d4feab80 1 --2- 172.21.15.33:0/2349664122 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561336a2a250 0x561336a2adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:18.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.711+0000 7f97d4feab80 1 --2- 172.21.15.33:0/2349664122 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561336a24550 0x561336a23f30 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:18.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.711+0000 7f97d4feab80 1 --2- 172.21.15.33:0/2349664122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561336a29930 0x561336a39d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:18.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.711+0000 7f97d4feab80 1 -- 172.21.15.33:0/2349664122 >> 172.21.15.33:0/2349664122 conn(0x56133699c6d0 msgr2=0x561336a2a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:45:18.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.712+0000 7f97d4feab80 1 -- 172.21.15.33:0/2349664122 shutdown_connections 2024-05-03T18:45:18.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.712+0000 7f97d4feab80 1 -- 172.21.15.33:0/2349664122 wait complete. 2024-05-03T18:45:18.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.713+0000 7f97d4feab80 1 -- start start 2024-05-03T18:45:18.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.713+0000 7f97d4feab80 1 Processor -- start 2024-05-03T18:45:18.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.714+0000 7f97d4feab80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561336a24550 0x561336a44da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:18.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.714+0000 7f97d4feab80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561336a29930 0x561336a452e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:18.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.714+0000 7f97d4feab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561336a2a250 0x561336a45820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:18.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.714+0000 7f97d4feab80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f97c41b4a10 con 0x561336a24550 2024-05-03T18:45:18.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.714+0000 7f97d4feab80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f97c41b5e80 con 0x561336a29930 2024-05-03T18:45:18.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.714+0000 7f97d4feab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f97c41b50e0 con 0x561336a2a250 2024-05-03T18:45:18.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.714+0000 7f97d4feab80 10 client.0 _ll_register_callbacks cb 0x56133691dc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:45:18.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.715+0000 7f97c2a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561336a29930 0x561336a452e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:18.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.715+0000 7f97c3400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561336a24550 0x561336a44da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:18.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.716+0000 7f97c3400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561336a24550 0x561336a44da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41394/0 (socket says 172.21.15.33:41394) 2024-05-03T18:45:18.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.716+0000 7f97c2a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561336a29930 0x561336a452e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:54866/0 (socket says 172.21.15.33:54866) 2024-05-03T18:45:18.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.716+0000 7f97c2a00700 1 -- 172.21.15.33:0/902561453 learned_addr learned my addr 172.21.15.33:0/902561453 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:45:18.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.717+0000 7f97c3400700 1 -- 172.21.15.33:0/902561453 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561336a2a250 msgr2=0x561336a45820 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-03T18:45:18.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.717+0000 7f97c3400700 1 --2- 172.21.15.33:0/902561453 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561336a2a250 0x561336a45820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:18.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.717+0000 7f97c3400700 1 -- 172.21.15.33:0/902561453 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561336a29930 msgr2=0x561336a452e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:45:18.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.717+0000 7f97c3400700 1 --2- 172.21.15.33:0/902561453 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561336a29930 0x561336a452e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:18.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.717+0000 7f97c3400700 1 -- 172.21.15.33:0/902561453 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561336a2b300 con 0x561336a24550 2024-05-03T18:45:18.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.718+0000 7f97c3400700 1 --2- 172.21.15.33:0/902561453 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561336a24550 0x561336a44da0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f97bc0a5880 tx=0x7f97bc0a9070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e37e3ccbdb9d70cf server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:45:18.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.718+0000 7f97b1600700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:45:18.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.719+0000 7f97b1600700 1 -- 172.21.15.33:0/902561453 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97bc0a9f00 con 0x561336a24550 2024-05-03T18:45:18.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.719+0000 7f97b1600700 1 -- 172.21.15.33:0/902561453 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9518+0+0 (secure 0 0 0) 0x7f97bc0a5df0 con 0x561336a24550 2024-05-03T18:45:18.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.720+0000 7f97d4feab80 1 -- 172.21.15.33:0/902561453 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97bc0a9800 con 0x561336a24550 2024-05-03T18:45:18.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.720+0000 7f97b1600700 1 -- 172.21.15.33:0/902561453 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f97bc0adbb0 con 0x561336a24550 2024-05-03T18:45:18.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.721+0000 7f97d4feab80 10 client.16935 fetch_fsmap learned FSMap version 21 2024-05-03T18:45:18.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.721+0000 7f97d4feab80 1 -- 172.21.15.33:0/902561453 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x561336ab9260 con 0x561336a24550 2024-05-03T18:45:18.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.722+0000 7f97b1600700 1 -- 172.21.15.33:0/902561453 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f97bc0a9800 con 0x561336a24550 2024-05-03T18:45:18.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.722+0000 7f97b1600700 1 -- 172.21.15.33:0/902561453 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f97bc0b21d0 con 0x561336a24550 2024-05-03T18:45:18.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.723+0000 7f97d4feab80 10 client.16935 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:45:18.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.723+0000 7f97d4feab80 10 client.16935 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:45:18.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.723+0000 7f97d4feab80 20 client.16935 populate_metadata read hostname 'smithi033' 2024-05-03T18:45:18.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.723+0000 7f97d4feab80 1 --2- 172.21.15.33:0/902561453 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x561336ab9870 0x561336abbc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:18.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.723+0000 7f97d4feab80 4 client.16935 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-03T18:45:18.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.723+0000 7f97d4feab80 1 -- 172.21.15.33:0/902561453 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561336984400 con 0x561336ab9870 2024-05-03T18:45:18.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.724+0000 7f97c2a00700 1 --2- 172.21.15.33:0/902561453 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x561336ab9870 0x561336abbc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:18.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.725+0000 7f97c2a00700 1 --2- 172.21.15.33:0/902561453 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x561336ab9870 0x561336abbc90 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b14ad0bf887518e7 server_cookie=d78777f01f8be8f1 in_seq=0 out_seq=0 2024-05-03T18:45:18.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.725+0000 7f97b1600700 10 client.16935 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:45:18.731 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-03T18:45:18.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.726+0000 7f97b1600700 1 -- 172.21.15.33:0/902561453 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561336984400 con 0x561336ab9870 2024-05-03T18:45:18.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.726+0000 7f97b1600700 10 client.16935 handle_command_reply: tid=0 2024-05-03T18:45:18.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.726+0000 7f97d4feab80 1 client.16935 shutdown 2024-05-03T18:45:18.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.727+0000 7f97d4feab80 1 -- 172.21.15.33:0/902561453 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561336a24550 msgr2=0x561336a44da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:18.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.727+0000 7f97d4feab80 1 --2- 172.21.15.33:0/902561453 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561336a24550 0x561336a44da0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f97bc0a5880 tx=0x7f97bc0a9070 comp rx=0 tx=0).stop 2024-05-03T18:45:18.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.727+0000 7f97d4feab80 1 -- 172.21.15.33:0/902561453 shutdown_connections 2024-05-03T18:45:18.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.727+0000 7f97d4feab80 1 --2- 172.21.15.33:0/902561453 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x561336ab9870 0x561336abbc90 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:18.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.727+0000 7f97d4feab80 1 --2- 172.21.15.33:0/902561453 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561336a2a250 0x561336a45820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:18.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.727+0000 7f97d4feab80 1 --2- 172.21.15.33:0/902561453 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561336a24550 0x561336a44da0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:18.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.727+0000 7f97d4feab80 1 --2- 172.21.15.33:0/902561453 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561336a29930 0x561336a452e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:18.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.727+0000 7f97d4feab80 1 -- 172.21.15.33:0/902561453 >> 172.21.15.33:0/902561453 conn(0x56133699c6d0 msgr2=0x561336a18b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:45:18.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.727+0000 7f97b1600700 0 client.16935 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:45:18.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.728+0000 7f97d4feab80 1 -- 172.21.15.33:0/902561453 shutdown_connections 2024-05-03T18:45:18.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.728+0000 7f97d4feab80 1 -- 172.21.15.33:0/902561453 wait complete. 2024-05-03T18:45:18.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.728+0000 7f97d4feab80 20 client.16935 trim_cache size 0 max 16384 2024-05-03T18:45:18.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.731+0000 7f97ca600700 1 -- 172.21.15.33:0/3514910646 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f97a80b2af0 msgr2=0x7f97a80b4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:18.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.731+0000 7f97ca600700 1 --2- 172.21.15.33:0/3514910646 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f97a80b2af0 0x7f97a80b4fb0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f97c406f5b0 tx=0x7f97b4005fb0 comp rx=0 tx=0).stop 2024-05-03T18:45:18.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.731+0000 7f97ca600700 1 -- 172.21.15.33:0/3514910646 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97c4105690 msgr2=0x7f97c4074cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:18.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.731+0000 7f97ca600700 1 --2- 172.21.15.33:0/3514910646 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97c4105690 0x7f97c4074cd0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f97bc00ef10 tx=0x7f97bc00c3a0 comp rx=0 tx=0).stop 2024-05-03T18:45:18.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.731+0000 7f97ca600700 1 -- 172.21.15.33:0/3514910646 shutdown_connections 2024-05-03T18:45:18.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.731+0000 7f97ca600700 1 --2- 172.21.15.33:0/3514910646 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f97a80b2af0 0x7f97a80b4fb0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:18.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.731+0000 7f97ca600700 1 --2- 172.21.15.33:0/3514910646 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97c410b690 0x7f97c406e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:18.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.731+0000 7f97ca600700 1 --2- 172.21.15.33:0/3514910646 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97c4105690 0x7f97c4074cd0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:18.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.731+0000 7f97ca600700 1 --2- 172.21.15.33:0/3514910646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97c4106050 0x7f97c406ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:18.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.731+0000 7f97ca600700 1 -- 172.21.15.33:0/3514910646 >> 172.21.15.33:0/3514910646 conn(0x7f97c4101180 msgr2=0x7f97c4101910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:45:18.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.732+0000 7f97ca600700 1 -- 172.21.15.33:0/3514910646 shutdown_connections 2024-05-03T18:45:18.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:18.732+0000 7f97ca600700 1 -- 172.21.15.33:0/3514910646 wait complete. 2024-05-03T18:45:20.111 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 845.0 s, 483.2 tps, lat 206.392 ms stddev 133.865 2024-05-03T18:45:20.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:45:19 smithi033 ceph-mon[28330]: pgmap v572: 129 pgs: 129 active+clean; 3.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 735 KiB/s rd, 6.6 MiB/s wr, 510 op/s 2024-05-03T18:45:20.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:45:19 smithi099 ceph-mon[31658]: pgmap v572: 129 pgs: 129 active+clean; 3.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 735 KiB/s rd, 6.6 MiB/s wr, 510 op/s 2024-05-03T18:45:20.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:45:19 smithi150 ceph-mon[32492]: pgmap v572: 129 pgs: 129 active+clean; 3.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 735 KiB/s rd, 6.6 MiB/s wr, 510 op/s 2024-05-03T18:45:20.697 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-03T18:45:20.698 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-03T18:45:21.528 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:45:22.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:45:21 smithi033 ceph-mon[28330]: pgmap v573: 129 pgs: 129 active+clean; 3.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 755 KiB/s rd, 6.6 MiB/s wr, 520 op/s 2024-05-03T18:45:22.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:45:21 smithi099 ceph-mon[31658]: pgmap v573: 129 pgs: 129 active+clean; 3.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 755 KiB/s rd, 6.6 MiB/s wr, 520 op/s 2024-05-03T18:45:22.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:45:21 smithi150 ceph-mon[32492]: pgmap v573: 129 pgs: 129 active+clean; 3.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 755 KiB/s rd, 6.6 MiB/s wr, 520 op/s 2024-05-03T18:45:24.236 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:45:23 smithi033 ceph-mon[28330]: pgmap v574: 129 pgs: 129 active+clean; 3.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 755 KiB/s rd, 6.6 MiB/s wr, 520 op/s 2024-05-03T18:45:24.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:45:23 smithi099 ceph-mon[31658]: pgmap v574: 129 pgs: 129 active+clean; 3.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 755 KiB/s rd, 6.6 MiB/s wr, 520 op/s 2024-05-03T18:45:24.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:45:23 smithi150 ceph-mon[32492]: pgmap v574: 129 pgs: 129 active+clean; 3.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 755 KiB/s rd, 6.6 MiB/s wr, 520 op/s 2024-05-03T18:45:25.111 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 850.0 s, 564.4 tps, lat 178.255 ms stddev 129.655 2024-05-03T18:45:26.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:45:26 smithi099 ceph-mon[31658]: pgmap v575: 129 pgs: 129 active+clean; 3.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 697 KiB/s rd, 6.8 MiB/s wr, 571 op/s 2024-05-03T18:45:26.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:45:26 smithi150 ceph-mon[32492]: pgmap v575: 129 pgs: 129 active+clean; 3.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 697 KiB/s rd, 6.8 MiB/s wr, 571 op/s 2024-05-03T18:45:26.486 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:45:26 smithi033 ceph-mon[28330]: pgmap v575: 129 pgs: 129 active+clean; 3.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 697 KiB/s rd, 6.8 MiB/s wr, 571 op/s 2024-05-03T18:45:27.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.460+0000 7f7b39c00700 1 -- 172.21.15.33:0/2728742020 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b34104eb0 msgr2=0x7f7b3410d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:27.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.460+0000 7f7b39c00700 1 --2- 172.21.15.33:0/2728742020 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b34104eb0 0x7f7b3410d3a0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f7b20013ef0 tx=0x7f7b200097f0 comp rx=0 tx=0).stop 2024-05-03T18:45:27.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.461+0000 7f7b39c00700 1 -- 172.21.15.33:0/2728742020 shutdown_connections 2024-05-03T18:45:27.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.461+0000 7f7b39c00700 1 --2- 172.21.15.33:0/2728742020 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b3410d9c0 0x7f7b3410fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:27.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.461+0000 7f7b39c00700 1 --2- 172.21.15.33:0/2728742020 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b34104590 0x7f7b34104970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:27.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.461+0000 7f7b39c00700 1 --2- 172.21.15.33:0/2728742020 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b34104eb0 0x7f7b3410d3a0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:27.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.461+0000 7f7b39c00700 1 -- 172.21.15.33:0/2728742020 >> 172.21.15.33:0/2728742020 conn(0x7f7b340fddc0 msgr2=0x7f7b341001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:45:27.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.461+0000 7f7b39c00700 1 -- 172.21.15.33:0/2728742020 shutdown_connections 2024-05-03T18:45:27.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.462+0000 7f7b39c00700 1 -- 172.21.15.33:0/2728742020 wait complete. 2024-05-03T18:45:27.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.464+0000 7f7b39c00700 1 Processor -- start 2024-05-03T18:45:27.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.464+0000 7f7b39c00700 1 -- start start 2024-05-03T18:45:27.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.465+0000 7f7b39c00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b34104590 0x7f7b3419d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:27.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.466+0000 7f7b39c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b34104eb0 0x7f7b3419d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:27.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.466+0000 7f7b39c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b3410d9c0 0x7f7b34197600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:27.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.466+0000 7f7b39c00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7b34111300 con 0x7f7b34104590 2024-05-03T18:45:27.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.466+0000 7f7b39c00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7b34111180 con 0x7f7b34104eb0 2024-05-03T18:45:27.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.466+0000 7f7b39c00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f7b34111480 con 0x7f7b3410d9c0 2024-05-03T18:45:27.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.466+0000 7f7b33e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b34104590 0x7f7b3419d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:27.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.466+0000 7f7b33e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b34104590 0x7f7b3419d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60702/0 (socket says 172.21.15.33:60702) 2024-05-03T18:45:27.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.466+0000 7f7b33e00700 1 -- 172.21.15.33:0/1498976144 learned_addr learned my addr 172.21.15.33:0/1498976144 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:45:27.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.466+0000 7f7b33400700 1 --2- 172.21.15.33:0/1498976144 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b34104eb0 0x7f7b3419d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:27.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.466+0000 7f7b39200700 1 --2- 172.21.15.33:0/1498976144 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b3410d9c0 0x7f7b34197600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:27.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.467+0000 7f7b33e00700 1 -- 172.21.15.33:0/1498976144 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b3410d9c0 msgr2=0x7f7b34197600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:27.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.467+0000 7f7b33e00700 1 --2- 172.21.15.33:0/1498976144 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b3410d9c0 0x7f7b34197600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:27.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.467+0000 7f7b33e00700 1 -- 172.21.15.33:0/1498976144 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b34104eb0 msgr2=0x7f7b3419d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:27.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.467+0000 7f7b33e00700 1 --2- 172.21.15.33:0/1498976144 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b34104eb0 0x7f7b3419d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:27.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.467+0000 7f7b33e00700 1 -- 172.21.15.33:0/1498976144 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b34197d60 con 0x7f7b34104590 2024-05-03T18:45:27.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.467+0000 7f7b39200700 1 --2- 172.21.15.33:0/1498976144 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b3410d9c0 0x7f7b34197600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:45:27.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.467+0000 7f7b33e00700 1 --2- 172.21.15.33:0/1498976144 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b34104590 0x7f7b3419d440 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f7b2400c9b0 tx=0x7f7b2400cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:45:27.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.468+0000 7f7b30c00700 1 -- 172.21.15.33:0/1498976144 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b24007b00 con 0x7f7b34104590 2024-05-03T18:45:27.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.468+0000 7f7b39c00700 1 -- 172.21.15.33:0/1498976144 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b34198050 con 0x7f7b34104590 2024-05-03T18:45:27.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.472+0000 7f7b30c00700 1 -- 172.21.15.33:0/1498976144 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7b24007ca0 con 0x7f7b34104590 2024-05-03T18:45:27.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.472+0000 7f7b30c00700 1 -- 172.21.15.33:0/1498976144 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b24018660 con 0x7f7b34104590 2024-05-03T18:45:27.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.472+0000 7f7b39c00700 1 -- 172.21.15.33:0/1498976144 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b341a4310 con 0x7f7b34104590 2024-05-03T18:45:27.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.484+0000 7f7b30c00700 1 -- 172.21.15.33:0/1498976144 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f7b2400f450 con 0x7f7b34104590 2024-05-03T18:45:27.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.487+0000 7f7b30c00700 1 --2- 172.21.15.33:0/1498976144 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f7b180b2a70 0x7f7b180b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:27.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.487+0000 7f7b30c00700 1 -- 172.21.15.33:0/1498976144 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 9518+0+0 (secure 0 0 0) 0x7f7b240e2a60 con 0x7f7b34104590 2024-05-03T18:45:27.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.487+0000 7f7b33400700 1 --2- 172.21.15.33:0/1498976144 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f7b180b2a70 0x7f7b180b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:27.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.488+0000 7f7b33400700 1 --2- 172.21.15.33:0/1498976144 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f7b180b2a70 0x7f7b180b4f30 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7b3410ee40 tx=0x7f7b20015040 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:45:27.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.499+0000 7f7b45bf8b80 1 Processor -- start 2024-05-03T18:45:27.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.500+0000 7f7b45bf8b80 1 -- start start 2024-05-03T18:45:27.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.501+0000 7f7b45bf8b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56483090fac0 0x56483090fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:27.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.501+0000 7f7b45bf8b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648309103e0 0x56483090a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:27.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.501+0000 7f7b45bf8b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56483090a650 0x56483090aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:27.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.501+0000 7f7b45bf8b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56483085caa0 con 0x56483090a650 2024-05-03T18:45:27.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.501+0000 7f7b45bf8b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5648307fa7e0 con 0x5648309103e0 2024-05-03T18:45:27.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.501+0000 7f7b45bf8b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x564830858a60 con 0x56483090fac0 2024-05-03T18:45:27.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.501+0000 7f7b33e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56483090fac0 0x56483090fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:27.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.501+0000 7f7b33e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56483090fac0 0x56483090fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:53782/0 (socket says 172.21.15.33:53782) 2024-05-03T18:45:27.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.501+0000 7f7b33e00700 1 -- 172.21.15.33:0/1416425905 learned_addr learned my addr 172.21.15.33:0/1416425905 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:45:27.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.501+0000 7f7b33400700 1 --2- 172.21.15.33:0/1416425905 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648309103e0 0x56483090a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:27.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.502+0000 7f7b39200700 1 --2- 172.21.15.33:0/1416425905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56483090a650 0x56483090aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:27.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.502+0000 7f7b33e00700 1 -- 172.21.15.33:0/1416425905 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648309103e0 msgr2=0x56483090a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:45:27.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.502+0000 7f7b33e00700 1 --2- 172.21.15.33:0/1416425905 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648309103e0 0x56483090a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:27.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.503+0000 7f7b33e00700 1 -- 172.21.15.33:0/1416425905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56483090a650 msgr2=0x56483090aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:45:27.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.503+0000 7f7b33e00700 1 --2- 172.21.15.33:0/1416425905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56483090a650 0x56483090aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:27.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.503+0000 7f7b33e00700 1 -- 172.21.15.33:0/1416425905 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648307a9950 con 0x56483090fac0 2024-05-03T18:45:27.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.504+0000 7f7b33e00700 1 --2- 172.21.15.33:0/1416425905 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56483090fac0 0x56483090fea0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f7b240a5980 tx=0x7f7b240a9800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5a29f16f86e2d46 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:45:27.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.505+0000 7f7b29600700 1 -- 172.21.15.33:0/1416425905 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b240aa000 con 0x56483090fac0 2024-05-03T18:45:27.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.506+0000 7f7b29600700 1 -- 172.21.15.33:0/1416425905 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7b240a9900 con 0x56483090fac0 2024-05-03T18:45:27.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.506+0000 7f7b29600700 1 -- 172.21.15.33:0/1416425905 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b240a5b90 con 0x56483090fac0 2024-05-03T18:45:27.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.508+0000 7f7b45bf8b80 1 -- 172.21.15.33:0/1416425905 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56483090fac0 msgr2=0x56483090fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:27.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.508+0000 7f7b45bf8b80 1 --2- 172.21.15.33:0/1416425905 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56483090fac0 0x56483090fea0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f7b240a5980 tx=0x7f7b240a9800 comp rx=0 tx=0).stop 2024-05-03T18:45:27.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.509+0000 7f7b45bf8b80 1 -- 172.21.15.33:0/1416425905 shutdown_connections 2024-05-03T18:45:27.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.509+0000 7f7b45bf8b80 1 --2- 172.21.15.33:0/1416425905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56483090a650 0x56483090aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:27.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.509+0000 7f7b45bf8b80 1 --2- 172.21.15.33:0/1416425905 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56483090fac0 0x56483090fea0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:27.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.509+0000 7f7b45bf8b80 1 --2- 172.21.15.33:0/1416425905 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648309103e0 0x56483090a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:27.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.509+0000 7f7b45bf8b80 1 -- 172.21.15.33:0/1416425905 >> 172.21.15.33:0/1416425905 conn(0x564830838be0 msgr2=0x564830904a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:45:27.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.510+0000 7f7b45bf8b80 1 -- 172.21.15.33:0/1416425905 shutdown_connections 2024-05-03T18:45:27.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.510+0000 7f7b45bf8b80 1 -- 172.21.15.33:0/1416425905 wait complete. 2024-05-03T18:45:27.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.511+0000 7f7b45bf8b80 1 -- start start 2024-05-03T18:45:27.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.511+0000 7f7b45bf8b80 1 Processor -- start 2024-05-03T18:45:27.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.513+0000 7f7b45bf8b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56483090a650 0x564830916160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:27.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.513+0000 7f7b45bf8b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56483090fac0 0x564830913500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:27.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.513+0000 7f7b45bf8b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648309103e0 0x564830913a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:27.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.513+0000 7f7b45bf8b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x564830916df0 con 0x56483090a650 2024-05-03T18:45:27.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.513+0000 7f7b45bf8b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5648307c2ba0 con 0x5648309103e0 2024-05-03T18:45:27.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.513+0000 7f7b45bf8b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x564830916f70 con 0x56483090fac0 2024-05-03T18:45:27.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.513+0000 7f7b45bf8b80 10 client.0 _ll_register_callbacks cb 0x564830803cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:45:27.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.514+0000 7f7b33e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56483090a650 0x564830916160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:27.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.514+0000 7f7b33e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56483090a650 0x564830916160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60720/0 (socket says 172.21.15.33:60720) 2024-05-03T18:45:27.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.514+0000 7f7b33e00700 1 -- 172.21.15.33:0/1452796290 learned_addr learned my addr 172.21.15.33:0/1452796290 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:45:27.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.514+0000 7f7b39200700 1 --2- 172.21.15.33:0/1452796290 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648309103e0 0x564830913a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:27.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.515+0000 7f7b33e00700 1 -- 172.21.15.33:0/1452796290 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56483090fac0 msgr2=0x564830913500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:45:27.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.514+0000 7f7b33400700 1 --2- 172.21.15.33:0/1452796290 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56483090fac0 0x564830913500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:27.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.515+0000 7f7b33e00700 1 --2- 172.21.15.33:0/1452796290 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56483090fac0 0x564830913500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:27.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.515+0000 7f7b33e00700 1 -- 172.21.15.33:0/1452796290 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648309103e0 msgr2=0x564830913a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:45:27.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.515+0000 7f7b33e00700 1 --2- 172.21.15.33:0/1452796290 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648309103e0 0x564830913a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:27.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.515+0000 7f7b33e00700 1 -- 172.21.15.33:0/1452796290 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648308224a0 con 0x56483090a650 2024-05-03T18:45:27.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.515+0000 7f7b33400700 1 --2- 172.21.15.33:0/1452796290 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56483090fac0 0x564830913500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:45:27.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.515+0000 7f7b39200700 1 --2- 172.21.15.33:0/1452796290 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648309103e0 0x564830913a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:45:27.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.516+0000 7f7b33e00700 1 --2- 172.21.15.33:0/1452796290 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56483090a650 0x564830916160 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f7b240a95b0 tx=0x7f7b240a9760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bf92971963cb5e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:45:27.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.516+0000 7f7b1e000700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:45:27.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.516+0000 7f7b1e000700 1 -- 172.21.15.33:0/1452796290 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b240c2070 con 0x56483090a650 2024-05-03T18:45:27.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.516+0000 7f7b1e000700 1 -- 172.21.15.33:0/1452796290 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9518+0+0 (secure 0 0 0) 0x7f7b240a9b60 con 0x56483090a650 2024-05-03T18:45:27.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.522+0000 7f7b45bf8b80 1 -- 172.21.15.33:0/1452796290 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b240a9900 con 0x56483090a650 2024-05-03T18:45:27.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.523+0000 7f7b1e000700 1 -- 172.21.15.33:0/1452796290 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f7b240aa000 con 0x56483090a650 2024-05-03T18:45:27.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.524+0000 7f7b45bf8b80 10 client.16944 fetch_fsmap learned FSMap version 21 2024-05-03T18:45:27.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.524+0000 7f7b45bf8b80 1 -- 172.21.15.33:0/1452796290 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5648309216a0 con 0x56483090a650 2024-05-03T18:45:27.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.526+0000 7f7b1e000700 1 -- 172.21.15.33:0/1452796290 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7b240a9900 con 0x56483090a650 2024-05-03T18:45:27.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.526+0000 7f7b1e000700 1 -- 172.21.15.33:0/1452796290 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f7b240cabb0 con 0x56483090a650 2024-05-03T18:45:27.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.526+0000 7f7b45bf8b80 10 client.16944 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:45:27.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.526+0000 7f7b45bf8b80 10 client.16944 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:45:27.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.526+0000 7f7b45bf8b80 20 client.16944 populate_metadata read hostname 'smithi033' 2024-05-03T18:45:27.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.527+0000 7f7b45bf8b80 1 --2- 172.21.15.33:0/1452796290 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x564830923b70 0x564830925f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:27.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.527+0000 7f7b45bf8b80 4 client.16944 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:45:27.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.527+0000 7f7b45bf8b80 1 -- 172.21.15.33:0/1452796290 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5648308d7250 con 0x564830923b70 2024-05-03T18:45:27.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.527+0000 7f7b33400700 1 --2- 172.21.15.33:0/1452796290 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x564830923b70 0x564830925f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:27.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.529+0000 7f7b1e000700 10 client.16944 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:45:27.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.529+0000 7f7b33400700 1 --2- 172.21.15.33:0/1452796290 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x564830923b70 0x564830925f60 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e4e821caf473698 server_cookie=396b2b5d702c1fb9 in_seq=0 out_seq=0 2024-05-03T18:45:27.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.531+0000 7f7b1e000700 1 -- 172.21.15.33:0/1452796290 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5648308d7250 con 0x564830923b70 2024-05-03T18:45:27.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.531+0000 7f7b1e000700 10 client.16944 handle_command_reply: tid=0 2024-05-03T18:45:27.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.532+0000 7f7b45bf8b80 1 client.16944 shutdown 2024-05-03T18:45:27.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.533+0000 7f7b45bf8b80 1 -- 172.21.15.33:0/1452796290 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56483090a650 msgr2=0x564830916160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:27.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.533+0000 7f7b45bf8b80 1 --2- 172.21.15.33:0/1452796290 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56483090a650 0x564830916160 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f7b240a95b0 tx=0x7f7b240a9760 comp rx=0 tx=0).stop 2024-05-03T18:45:27.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.533+0000 7f7b45bf8b80 1 -- 172.21.15.33:0/1452796290 shutdown_connections 2024-05-03T18:45:27.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.533+0000 7f7b45bf8b80 1 --2- 172.21.15.33:0/1452796290 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x564830923b70 0x564830925f60 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:27.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.533+0000 7f7b45bf8b80 1 --2- 172.21.15.33:0/1452796290 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648309103e0 0x564830913a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:27.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.533+0000 7f7b45bf8b80 1 --2- 172.21.15.33:0/1452796290 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56483090a650 0x564830916160 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:27.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.533+0000 7f7b45bf8b80 1 --2- 172.21.15.33:0/1452796290 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56483090fac0 0x564830913500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:27.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.533+0000 7f7b45bf8b80 1 -- 172.21.15.33:0/1452796290 >> 172.21.15.33:0/1452796290 conn(0x564830838be0 msgr2=0x564830904100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:45:27.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.534+0000 7f7b1e000700 0 client.16944 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:45:27.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.534+0000 7f7b45bf8b80 1 -- 172.21.15.33:0/1452796290 shutdown_connections 2024-05-03T18:45:27.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.534+0000 7f7b45bf8b80 1 -- 172.21.15.33:0/1452796290 wait complete. 2024-05-03T18:45:27.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.535+0000 7f7b45bf8b80 20 client.16944 trim_cache size 0 max 16384 2024-05-03T18:45:27.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.578+0000 7f7b45bf8b80 1 Processor -- start 2024-05-03T18:45:27.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.578+0000 7f7b45bf8b80 1 -- start start 2024-05-03T18:45:27.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.578+0000 7f7b45bf8b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56483090a650 0x56483090a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:27.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.578+0000 7f7b45bf8b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56483090fac0 0x5648309152c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:27.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.578+0000 7f7b45bf8b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5648309103e0 0x564830928f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:27.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.578+0000 7f7b45bf8b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7b341ab310 con 0x56483090a650 2024-05-03T18:45:27.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.578+0000 7f7b45bf8b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7b341adbb0 con 0x56483090fac0 2024-05-03T18:45:27.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.578+0000 7f7b45bf8b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f7b341ab9a0 con 0x5648309103e0 2024-05-03T18:45:27.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.579+0000 7f7b33e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56483090a650 0x56483090a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:27.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.579+0000 7f7b33e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56483090a650 0x56483090a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60724/0 (socket says 172.21.15.33:60724) 2024-05-03T18:45:27.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.579+0000 7f7b33e00700 1 -- 172.21.15.33:0/452527247 learned_addr learned my addr 172.21.15.33:0/452527247 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:45:27.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.580+0000 7f7b33e00700 1 -- 172.21.15.33:0/452527247 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5648309103e0 msgr2=0x564830928f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:45:27.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.580+0000 7f7b33e00700 1 --2- 172.21.15.33:0/452527247 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5648309103e0 0x564830928f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:27.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.580+0000 7f7b33e00700 1 -- 172.21.15.33:0/452527247 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56483090fac0 msgr2=0x5648309152c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:45:27.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.580+0000 7f7b33e00700 1 --2- 172.21.15.33:0/452527247 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56483090fac0 0x5648309152c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:27.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.580+0000 7f7b33e00700 1 -- 172.21.15.33:0/452527247 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564830905520 con 0x56483090a650 2024-05-03T18:45:27.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.580+0000 7f7b33e00700 1 --2- 172.21.15.33:0/452527247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56483090a650 0x56483090a030 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f7b240b2080 tx=0x7f7b240b28e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ee02fe0054fd4d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:45:27.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.581+0000 7f7b28c00700 1 -- 172.21.15.33:0/452527247 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b240aa000 con 0x56483090a650 2024-05-03T18:45:27.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.581+0000 7f7b28c00700 1 -- 172.21.15.33:0/452527247 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7b240cabb0 con 0x56483090a650 2024-05-03T18:45:27.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.581+0000 7f7b28c00700 1 -- 172.21.15.33:0/452527247 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b240b2530 con 0x56483090a650 2024-05-03T18:45:27.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.582+0000 7f7b45bf8b80 1 -- 172.21.15.33:0/452527247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56483090a650 msgr2=0x56483090a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:27.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.582+0000 7f7b45bf8b80 1 --2- 172.21.15.33:0/452527247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56483090a650 0x56483090a030 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f7b240b2080 tx=0x7f7b240b28e0 comp rx=0 tx=0).stop 2024-05-03T18:45:27.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.583+0000 7f7b45bf8b80 1 -- 172.21.15.33:0/452527247 shutdown_connections 2024-05-03T18:45:27.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.583+0000 7f7b45bf8b80 1 --2- 172.21.15.33:0/452527247 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5648309103e0 0x564830928f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:27.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.583+0000 7f7b45bf8b80 1 --2- 172.21.15.33:0/452527247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56483090a650 0x56483090a030 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:27.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.583+0000 7f7b45bf8b80 1 --2- 172.21.15.33:0/452527247 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56483090fac0 0x5648309152c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:27.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.583+0000 7f7b45bf8b80 1 -- 172.21.15.33:0/452527247 >> 172.21.15.33:0/452527247 conn(0x564830838be0 msgr2=0x564830914bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:45:27.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.583+0000 7f7b45bf8b80 1 -- 172.21.15.33:0/452527247 shutdown_connections 2024-05-03T18:45:27.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.583+0000 7f7b45bf8b80 1 -- 172.21.15.33:0/452527247 wait complete. 2024-05-03T18:45:27.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.584+0000 7f7b45bf8b80 1 -- start start 2024-05-03T18:45:27.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.584+0000 7f7b45bf8b80 1 Processor -- start 2024-05-03T18:45:27.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.586+0000 7f7b45bf8b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56483090a650 0x564830927c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:27.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.586+0000 7f7b45bf8b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56483090fac0 0x564830928180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:27.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.586+0000 7f7b45bf8b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5648309103e0 0x5648309286c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:27.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.586+0000 7f7b45bf8b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7b341ac070 con 0x5648309103e0 2024-05-03T18:45:27.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.586+0000 7f7b45bf8b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7b341ad4e0 con 0x56483090a650 2024-05-03T18:45:27.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.586+0000 7f7b45bf8b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f7b341ac740 con 0x56483090fac0 2024-05-03T18:45:27.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.587+0000 7f7b45bf8b80 10 client.0 _ll_register_callbacks cb 0x564830803cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:45:27.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.587+0000 7f7b33e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56483090a650 0x564830927c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:27.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.587+0000 7f7b39200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5648309103e0 0x5648309286c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:27.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.587+0000 7f7b39200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5648309103e0 0x5648309286c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60726/0 (socket says 172.21.15.33:60726) 2024-05-03T18:45:27.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.587+0000 7f7b39200700 1 -- 172.21.15.33:0/1059653844 learned_addr learned my addr 172.21.15.33:0/1059653844 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:45:27.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.587+0000 7f7b33e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56483090a650 0x564830927c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:40412/0 (socket says 172.21.15.33:40412) 2024-05-03T18:45:27.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.588+0000 7f7b39200700 1 -- 172.21.15.33:0/1059653844 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56483090fac0 msgr2=0x564830928180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:45:27.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.588+0000 7f7b39200700 1 --2- 172.21.15.33:0/1059653844 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56483090fac0 0x564830928180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:27.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.588+0000 7f7b39200700 1 -- 172.21.15.33:0/1059653844 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56483090a650 msgr2=0x564830927c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:45:27.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.588+0000 7f7b39200700 1 --2- 172.21.15.33:0/1059653844 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56483090a650 0x564830927c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:27.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.588+0000 7f7b39200700 1 -- 172.21.15.33:0/1059653844 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56483098e120 con 0x5648309103e0 2024-05-03T18:45:27.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.589+0000 7f7b33e00700 1 --2- 172.21.15.33:0/1059653844 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56483090a650 0x564830927c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:45:27.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.589+0000 7f7b39200700 1 --2- 172.21.15.33:0/1059653844 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5648309103e0 0x5648309286c0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c007a70 tx=0x7f7b2c007b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bc5d316c23634fa server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:45:27.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.590+0000 7f7b1e000700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:45:27.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.590+0000 7f7b1e000700 1 -- 172.21.15.33:0/1059653844 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b2c01a030 con 0x5648309103e0 2024-05-03T18:45:27.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.590+0000 7f7b45bf8b80 1 -- 172.21.15.33:0/1059653844 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b240cabb0 con 0x5648309103e0 2024-05-03T18:45:27.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.590+0000 7f7b1e000700 1 -- 172.21.15.33:0/1059653844 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9518+0+0 (secure 0 0 0) 0x7f7b2c004bb0 con 0x5648309103e0 2024-05-03T18:45:27.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.590+0000 7f7b1e000700 1 -- 172.21.15.33:0/1059653844 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f7b2c026050 con 0x5648309103e0 2024-05-03T18:45:27.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.590+0000 7f7b45bf8b80 10 client.16950 fetch_fsmap learned FSMap version 21 2024-05-03T18:45:27.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.590+0000 7f7b45bf8b80 1 -- 172.21.15.33:0/1059653844 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x56483098f510 con 0x5648309103e0 2024-05-03T18:45:27.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.591+0000 7f7b1e000700 1 -- 172.21.15.33:0/1059653844 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7b240cabb0 con 0x5648309103e0 2024-05-03T18:45:27.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.591+0000 7f7b1e000700 1 -- 172.21.15.33:0/1059653844 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f7b2c016930 con 0x5648309103e0 2024-05-03T18:45:27.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.592+0000 7f7b45bf8b80 10 client.16950 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:45:27.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.592+0000 7f7b45bf8b80 10 client.16950 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:45:27.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.592+0000 7f7b45bf8b80 20 client.16950 populate_metadata read hostname 'smithi033' 2024-05-03T18:45:27.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.592+0000 7f7b45bf8b80 1 --2- 172.21.15.33:0/1059653844 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x56483098fb70 0x564830991f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:27.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.592+0000 7f7b45bf8b80 4 client.16950 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-03T18:45:27.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.592+0000 7f7b45bf8b80 1 -- 172.21.15.33:0/1059653844 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56483086a480 con 0x56483098fb70 2024-05-03T18:45:27.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.592+0000 7f7b33e00700 1 --2- 172.21.15.33:0/1059653844 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x56483098fb70 0x564830991f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:27.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.593+0000 7f7b33e00700 1 --2- 172.21.15.33:0/1059653844 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x56483098fb70 0x564830991f60 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3cb1b3167a2f66c server_cookie=8eb866299d98ac6 in_seq=0 out_seq=0 2024-05-03T18:45:27.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.594+0000 7f7b1e000700 10 client.16950 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:45:27.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.598+0000 7f7b1e000700 1 -- 172.21.15.33:0/1059653844 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56483086a480 con 0x56483098fb70 2024-05-03T18:45:27.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.598+0000 7f7b1e000700 10 client.16950 handle_command_reply: tid=0 2024-05-03T18:45:27.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.598+0000 7f7b45bf8b80 1 client.16950 shutdown 2024-05-03T18:45:27.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.599+0000 7f7b45bf8b80 1 -- 172.21.15.33:0/1059653844 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5648309103e0 msgr2=0x5648309286c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:27.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.599+0000 7f7b45bf8b80 1 --2- 172.21.15.33:0/1059653844 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5648309103e0 0x5648309286c0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c007a70 tx=0x7f7b2c007b50 comp rx=0 tx=0).stop 2024-05-03T18:45:27.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.602+0000 7f7b45bf8b80 1 -- 172.21.15.33:0/1059653844 shutdown_connections 2024-05-03T18:45:27.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.602+0000 7f7b45bf8b80 1 --2- 172.21.15.33:0/1059653844 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x56483098fb70 0x564830991f60 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:27.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.602+0000 7f7b45bf8b80 1 --2- 172.21.15.33:0/1059653844 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5648309103e0 0x5648309286c0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:27.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.602+0000 7f7b45bf8b80 1 --2- 172.21.15.33:0/1059653844 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56483090a650 0x564830927c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:27.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.602+0000 7f7b45bf8b80 1 --2- 172.21.15.33:0/1059653844 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56483090fac0 0x564830928180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:27.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.602+0000 7f7b45bf8b80 1 -- 172.21.15.33:0/1059653844 >> 172.21.15.33:0/1059653844 conn(0x564830838be0 msgr2=0x56483090e620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:45:27.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.602+0000 7f7b1e000700 0 client.16950 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:45:27.605 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-03T18:45:27.605 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-05-03T18:45:27.605 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "40ce5b70-cf08-4eb2-95c8-bd3c541332e8", 2024-05-03T18:45:27.605 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-05-03T18:45:27.605 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-03T18:45:27.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.602+0000 7f7b45bf8b80 1 -- 172.21.15.33:0/1059653844 shutdown_connections 2024-05-03T18:45:27.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.603+0000 7f7b45bf8b80 1 -- 172.21.15.33:0/1059653844 wait complete. 2024-05-03T18:45:27.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.603+0000 7f7b45bf8b80 20 client.16950 trim_cache size 0 max 16384 2024-05-03T18:45:27.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.604+0000 7f7b39c00700 1 -- 172.21.15.33:0/1498976144 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f7b180b2a70 msgr2=0x7f7b180b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:27.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.604+0000 7f7b39c00700 1 --2- 172.21.15.33:0/1498976144 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f7b180b2a70 0x7f7b180b4f30 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7b3410ee40 tx=0x7f7b20015040 comp rx=0 tx=0).stop 2024-05-03T18:45:27.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.604+0000 7f7b39c00700 1 -- 172.21.15.33:0/1498976144 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b34104590 msgr2=0x7f7b3419d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:27.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.604+0000 7f7b39c00700 1 --2- 172.21.15.33:0/1498976144 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b34104590 0x7f7b3419d440 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f7b2400c9b0 tx=0x7f7b2400cd70 comp rx=0 tx=0).stop 2024-05-03T18:45:27.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.604+0000 7f7b39c00700 1 -- 172.21.15.33:0/1498976144 shutdown_connections 2024-05-03T18:45:27.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.604+0000 7f7b39c00700 1 --2- 172.21.15.33:0/1498976144 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b3410d9c0 0x7f7b34197600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:27.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.604+0000 7f7b39c00700 1 --2- 172.21.15.33:0/1498976144 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b34104590 0x7f7b3419d440 secure :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f7b2400c9b0 tx=0x7f7b2400cd70 comp rx=0 tx=0).stop 2024-05-03T18:45:27.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.604+0000 7f7b39c00700 1 --2- 172.21.15.33:0/1498976144 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f7b180b2a70 0x7f7b180b4f30 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:27.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.604+0000 7f7b39c00700 1 --2- 172.21.15.33:0/1498976144 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b34104eb0 0x7f7b3419d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:27.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.604+0000 7f7b39c00700 1 -- 172.21.15.33:0/1498976144 >> 172.21.15.33:0/1498976144 conn(0x7f7b340fddc0 msgr2=0x7f7b340febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:45:27.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.609+0000 7f7b39c00700 1 -- 172.21.15.33:0/1498976144 shutdown_connections 2024-05-03T18:45:27.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:27.609+0000 7f7b39c00700 1 -- 172.21.15.33:0/1498976144 wait complete. 2024-05-03T18:45:28.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:45:28 smithi099 ceph-mon[31658]: pgmap v576: 129 pgs: 129 active+clean; 3.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 6.3 MiB/s wr, 555 op/s 2024-05-03T18:45:28.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:45:28 smithi150 ceph-mon[32492]: pgmap v576: 129 pgs: 129 active+clean; 3.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 6.3 MiB/s wr, 555 op/s 2024-05-03T18:45:28.405 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:45:28 smithi033 ceph-mon[28330]: pgmap v576: 129 pgs: 129 active+clean; 3.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 6.3 MiB/s wr, 555 op/s 2024-05-03T18:45:29.301 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:45:29 smithi099 ceph-mon[31658]: scrub queued for path: / 2024-05-03T18:45:29.301 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:45:29 smithi099 ceph-mon[31658]: scrub summary: idle+waiting paths [/] 2024-05-03T18:45:29.301 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:45:29 smithi099 ceph-mon[31658]: scrub summary: active paths [/] 2024-05-03T18:45:29.301 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:45:29 smithi099 ceph-mon[31658]: scrub summary: idle+waiting paths [/] 2024-05-03T18:45:29.331 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:45:29 smithi033 ceph-mon[28330]: scrub queued for path: / 2024-05-03T18:45:29.331 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:45:29 smithi033 ceph-mon[28330]: scrub summary: idle+waiting paths [/] 2024-05-03T18:45:29.331 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:45:29 smithi033 ceph-mon[28330]: scrub summary: active paths [/] 2024-05-03T18:45:29.331 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:45:29 smithi033 ceph-mon[28330]: scrub summary: idle+waiting paths [/] 2024-05-03T18:45:29.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:45:29 smithi150 ceph-mon[32492]: scrub queued for path: / 2024-05-03T18:45:29.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:45:29 smithi150 ceph-mon[32492]: scrub summary: idle+waiting paths [/] 2024-05-03T18:45:29.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:45:29 smithi150 ceph-mon[32492]: scrub summary: active paths [/] 2024-05-03T18:45:29.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:45:29 smithi150 ceph-mon[32492]: scrub summary: idle+waiting paths [/] 2024-05-03T18:45:29.407 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-03T18:45:30.111 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 855.0 s, 553.4 tps, lat 177.562 ms stddev 96.205 2024-05-03T18:45:30.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:45:30 smithi099 ceph-mon[31658]: pgmap v577: 129 pgs: 129 active+clean; 3.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 5.4 MiB/s wr, 472 op/s 2024-05-03T18:45:30.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:45:30 smithi150 ceph-mon[32492]: pgmap v577: 129 pgs: 129 active+clean; 3.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 5.4 MiB/s wr, 472 op/s 2024-05-03T18:45:30.392 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:45:30.487 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:45:30 smithi033 ceph-mon[28330]: pgmap v577: 129 pgs: 129 active+clean; 3.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 5.4 MiB/s wr, 472 op/s 2024-05-03T18:45:32.061 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:45:32 smithi150 ceph-mon[32492]: pgmap v578: 129 pgs: 129 active+clean; 3.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 523 KiB/s rd, 7.9 MiB/s wr, 691 op/s 2024-05-03T18:45:32.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:45:32 smithi099 ceph-mon[31658]: pgmap v578: 129 pgs: 129 active+clean; 3.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 523 KiB/s rd, 7.9 MiB/s wr, 691 op/s 2024-05-03T18:45:32.297 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:45:32 smithi099 ceph-mon[31658]: scrub summary: idle 2024-05-03T18:45:32.386 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:45:32 smithi150 ceph-mon[32492]: scrub summary: idle 2024-05-03T18:45:32.486 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:45:32 smithi033 ceph-mon[28330]: pgmap v578: 129 pgs: 129 active+clean; 3.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 523 KiB/s rd, 7.9 MiB/s wr, 691 op/s 2024-05-03T18:45:32.487 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:45:32 smithi033 ceph-mon[28330]: scrub summary: idle 2024-05-03T18:45:34.384 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:45:34 smithi150 ceph-mon[32492]: pgmap v579: 129 pgs: 129 active+clean; 3.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 5.4 MiB/s wr, 473 op/s 2024-05-03T18:45:34.384 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:45:34 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:45:34.457 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:45:34 smithi033 ceph-mon[28330]: pgmap v579: 129 pgs: 129 active+clean; 3.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 5.4 MiB/s wr, 473 op/s 2024-05-03T18:45:34.457 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:45:34 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:45:34.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:45:34 smithi099 ceph-mon[31658]: pgmap v579: 129 pgs: 129 active+clean; 3.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 5.4 MiB/s wr, 473 op/s 2024-05-03T18:45:34.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:45:34 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:45:35.111 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 860.0 s, 467.3 tps, lat 213.026 ms stddev 143.174 2024-05-03T18:45:35.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.284+0000 7fb2be800700 1 -- 172.21.15.33:0/836166761 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2b8105ee0 msgr2=0x7fb2b8106370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:35.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.284+0000 7fb2be800700 1 --2- 172.21.15.33:0/836166761 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2b8105ee0 0x7fb2b8106370 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb2b00098f0 tx=0x7fb2b0009c00 comp rx=0 tx=0).stop 2024-05-03T18:45:35.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.285+0000 7fb2be800700 1 -- 172.21.15.33:0/836166761 shutdown_connections 2024-05-03T18:45:35.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.285+0000 7fb2be800700 1 --2- 172.21.15.33:0/836166761 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2b8105ee0 0x7fb2b8106370 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:35.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.285+0000 7fb2be800700 1 --2- 172.21.15.33:0/836166761 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2b81083d0 0x7fb2b806b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:35.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.285+0000 7fb2be800700 1 --2- 172.21.15.33:0/836166761 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2b806b6f0 0x7fb2b806bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:35.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.285+0000 7fb2be800700 1 -- 172.21.15.33:0/836166761 >> 172.21.15.33:0/836166761 conn(0x7fb2b8079350 msgr2=0x7fb2b8079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:45:35.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.286+0000 7fb2be800700 1 -- 172.21.15.33:0/836166761 shutdown_connections 2024-05-03T18:45:35.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.287+0000 7fb2be800700 1 -- 172.21.15.33:0/836166761 wait complete. 2024-05-03T18:45:35.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.288+0000 7fb2be800700 1 Processor -- start 2024-05-03T18:45:35.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.289+0000 7fb2be800700 1 -- start start 2024-05-03T18:45:35.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.290+0000 7fb2be800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2b806b6f0 0x7fb2b8074c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:35.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.290+0000 7fb2be800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2b8105ee0 0x7fb2b8075140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:35.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.290+0000 7fb2be800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2b81083d0 0x7fb2b806eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:35.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.290+0000 7fb2b7400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2b806b6f0 0x7fb2b8074c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:35.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.291+0000 7fb2b7400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2b806b6f0 0x7fb2b8074c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:40444/0 (socket says 172.21.15.33:40444) 2024-05-03T18:45:35.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.291+0000 7fb2b7400700 1 -- 172.21.15.33:0/2218871657 learned_addr learned my addr 172.21.15.33:0/2218871657 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:45:35.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.291+0000 7fb2be800700 1 -- 172.21.15.33:0/2218871657 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b806d5e0 con 0x7fb2b8105ee0 2024-05-03T18:45:35.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.291+0000 7fb2be800700 1 -- 172.21.15.33:0/2218871657 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b806d460 con 0x7fb2b806b6f0 2024-05-03T18:45:35.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.291+0000 7fb2be800700 1 -- 172.21.15.33:0/2218871657 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b806d760 con 0x7fb2b81083d0 2024-05-03T18:45:35.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.291+0000 7fb2b7e00700 1 --2- 172.21.15.33:0/2218871657 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2b81083d0 0x7fb2b806eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:35.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.292+0000 7fb2aea00700 1 --2- 172.21.15.33:0/2218871657 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2b8105ee0 0x7fb2b8075140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:35.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.293+0000 7fb2b7400700 1 -- 172.21.15.33:0/2218871657 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2b81083d0 msgr2=0x7fb2b806eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:35.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.293+0000 7fb2b7400700 1 --2- 172.21.15.33:0/2218871657 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2b81083d0 0x7fb2b806eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:35.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.293+0000 7fb2b7400700 1 -- 172.21.15.33:0/2218871657 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2b8105ee0 msgr2=0x7fb2b8075140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:35.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.293+0000 7fb2b7400700 1 --2- 172.21.15.33:0/2218871657 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2b8105ee0 0x7fb2b8075140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:35.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.293+0000 7fb2b7400700 1 -- 172.21.15.33:0/2218871657 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2b806f7b0 con 0x7fb2b806b6f0 2024-05-03T18:45:35.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.295+0000 7fb2b7400700 1 --2- 172.21.15.33:0/2218871657 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2b806b6f0 0x7fb2b8074c00 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb2a8007fa0 tx=0x7fb2a800d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:45:35.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.295+0000 7fb2b4c00700 1 -- 172.21.15.33:0/2218871657 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a801a070 con 0x7fb2b806b6f0 2024-05-03T18:45:35.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.296+0000 7fb2be800700 1 -- 172.21.15.33:0/2218871657 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2b806faa0 con 0x7fb2b806b6f0 2024-05-03T18:45:35.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.296+0000 7fb2be800700 1 -- 172.21.15.33:0/2218871657 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2b81b54d0 con 0x7fb2b806b6f0 2024-05-03T18:45:35.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.296+0000 7fb2b4c00700 1 -- 172.21.15.33:0/2218871657 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb2a800dda0 con 0x7fb2b806b6f0 2024-05-03T18:45:35.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.296+0000 7fb2b4c00700 1 -- 172.21.15.33:0/2218871657 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a8014600 con 0x7fb2b806b6f0 2024-05-03T18:45:35.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.308+0000 7fb2b4c00700 1 -- 172.21.15.33:0/2218871657 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fb2a80147a0 con 0x7fb2b806b6f0 2024-05-03T18:45:35.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.310+0000 7fb2b4c00700 1 --2- 172.21.15.33:0/2218871657 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fb29c0b2b20 0x7fb29c0b4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:35.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.310+0000 7fb2b4c00700 1 -- 172.21.15.33:0/2218871657 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 9518+0+0 (secure 0 0 0) 0x7fb2a80e2120 con 0x7fb2b806b6f0 2024-05-03T18:45:35.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.312+0000 7fb2aea00700 1 --2- 172.21.15.33:0/2218871657 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fb29c0b2b20 0x7fb29c0b4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:35.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.314+0000 7fb2aea00700 1 --2- 172.21.15.33:0/2218871657 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fb29c0b2b20 0x7fb29c0b4fe0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb2b8070440 tx=0x7fb2a0009190 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:45:35.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.563+0000 7fb2c90dcb80 1 Processor -- start 2024-05-03T18:45:35.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.563+0000 7fb2c90dcb80 1 -- start start 2024-05-03T18:45:35.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.564+0000 7fb2c90dcb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560eee335930 0x560eee335d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:35.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.564+0000 7fb2c90dcb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560eee336250 0x560eee32ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:35.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.564+0000 7fb2c90dcb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560eee330550 0x560eee3309d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:35.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.564+0000 7fb2c90dcb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560eee282a20 con 0x560eee336250 2024-05-03T18:45:35.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.564+0000 7fb2c90dcb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x560eee220760 con 0x560eee330550 2024-05-03T18:45:35.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.564+0000 7fb2c90dcb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x560eee27e9e0 con 0x560eee335930 2024-05-03T18:45:35.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.568+0000 7fb2b7400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560eee335930 0x560eee335d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:35.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.568+0000 7fb2aea00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560eee336250 0x560eee32ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:35.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.568+0000 7fb2b7400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560eee335930 0x560eee335d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:53854/0 (socket says 172.21.15.33:53854) 2024-05-03T18:45:35.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.568+0000 7fb2b7400700 1 -- 172.21.15.33:0/3620446292 learned_addr learned my addr 172.21.15.33:0/3620446292 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:45:35.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.568+0000 7fb2b7400700 1 -- 172.21.15.33:0/3620446292 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560eee330550 msgr2=0x560eee3309d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-03T18:45:35.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.568+0000 7fb2b7400700 1 --2- 172.21.15.33:0/3620446292 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560eee330550 0x560eee3309d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:35.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.568+0000 7fb2b7400700 1 -- 172.21.15.33:0/3620446292 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560eee336250 msgr2=0x560eee32ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:45:35.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.569+0000 7fb2b7400700 1 --2- 172.21.15.33:0/3620446292 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560eee336250 0x560eee32ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:35.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.569+0000 7fb2b7400700 1 -- 172.21.15.33:0/3620446292 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560eee1cf8d0 con 0x560eee335930 2024-05-03T18:45:35.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.569+0000 7fb2aea00700 1 --2- 172.21.15.33:0/3620446292 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560eee336250 0x560eee32ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:45:35.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.572+0000 7fb2b7400700 1 --2- 172.21.15.33:0/3620446292 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560eee335930 0x560eee335d10 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb2a80a8f50 tx=0x7fb2a80a8b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa0186ff0c89554a server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:45:35.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.573+0000 7fb2acc00700 1 -- 172.21.15.33:0/3620446292 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a80c2020 con 0x560eee335930 2024-05-03T18:45:35.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.574+0000 7fb2acc00700 1 -- 172.21.15.33:0/3620446292 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb2a80ad370 con 0x560eee335930 2024-05-03T18:45:35.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.574+0000 7fb2acc00700 1 -- 172.21.15.33:0/3620446292 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a80c2020 con 0x560eee335930 2024-05-03T18:45:35.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.576+0000 7fb2c90dcb80 1 -- 172.21.15.33:0/3620446292 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560eee335930 msgr2=0x560eee335d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:35.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.576+0000 7fb2c90dcb80 1 --2- 172.21.15.33:0/3620446292 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560eee335930 0x560eee335d10 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb2a80a8f50 tx=0x7fb2a80a8b50 comp rx=0 tx=0).stop 2024-05-03T18:45:35.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.578+0000 7fb2c90dcb80 1 -- 172.21.15.33:0/3620446292 shutdown_connections 2024-05-03T18:45:35.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.578+0000 7fb2c90dcb80 1 --2- 172.21.15.33:0/3620446292 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560eee330550 0x560eee3309d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:35.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.578+0000 7fb2c90dcb80 1 --2- 172.21.15.33:0/3620446292 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560eee335930 0x560eee335d10 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:35.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.578+0000 7fb2c90dcb80 1 --2- 172.21.15.33:0/3620446292 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560eee336250 0x560eee32ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:35.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.578+0000 7fb2c90dcb80 1 -- 172.21.15.33:0/3620446292 >> 172.21.15.33:0/3620446292 conn(0x560eee2a86d0 msgr2=0x560eee32a900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:45:35.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.579+0000 7fb2c90dcb80 1 -- 172.21.15.33:0/3620446292 shutdown_connections 2024-05-03T18:45:35.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.579+0000 7fb2c90dcb80 1 -- 172.21.15.33:0/3620446292 wait complete. 2024-05-03T18:45:35.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.580+0000 7fb2c90dcb80 1 -- start start 2024-05-03T18:45:35.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.581+0000 7fb2c90dcb80 1 Processor -- start 2024-05-03T18:45:35.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.582+0000 7fb2c90dcb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560eee330550 0x560eee33be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:35.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.582+0000 7fb2c90dcb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560eee335930 0x560eee3391e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:35.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.582+0000 7fb2c90dcb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560eee336250 0x560eee339720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:35.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.582+0000 7fb2c90dcb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560eee33cd00 con 0x560eee336250 2024-05-03T18:45:35.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.582+0000 7fb2c90dcb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x560eee1e8b20 con 0x560eee335930 2024-05-03T18:45:35.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.582+0000 7fb2c90dcb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x560eee33ce80 con 0x560eee330550 2024-05-03T18:45:35.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.582+0000 7fb2c90dcb80 10 client.0 _ll_register_callbacks cb 0x560eee229c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:45:35.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.583+0000 7fb2b7e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560eee336250 0x560eee339720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:35.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.583+0000 7fb2b7400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560eee330550 0x560eee33be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:35.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.583+0000 7fb2b7400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560eee330550 0x560eee33be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:53870/0 (socket says 172.21.15.33:53870) 2024-05-03T18:45:35.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.583+0000 7fb2b7400700 1 -- 172.21.15.33:0/4057046470 learned_addr learned my addr 172.21.15.33:0/4057046470 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:45:35.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.584+0000 7fb2aea00700 1 --2- 172.21.15.33:0/4057046470 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560eee335930 0x560eee3391e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:35.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.584+0000 7fb2b7e00700 1 -- 172.21.15.33:0/4057046470 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560eee330550 msgr2=0x560eee33be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:45:35.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.584+0000 7fb2b7e00700 1 --2- 172.21.15.33:0/4057046470 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560eee330550 0x560eee33be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:35.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.584+0000 7fb2b7e00700 1 -- 172.21.15.33:0/4057046470 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560eee335930 msgr2=0x560eee3391e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:45:35.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.584+0000 7fb2b7e00700 1 --2- 172.21.15.33:0/4057046470 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560eee335930 0x560eee3391e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:35.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.584+0000 7fb2b7e00700 1 -- 172.21.15.33:0/4057046470 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560eee248420 con 0x560eee336250 2024-05-03T18:45:35.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.585+0000 7fb2b7e00700 1 --2- 172.21.15.33:0/4057046470 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560eee336250 0x560eee339720 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fb2b0009230 tx=0x7fb2b000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e0ff5cefb67b3fa server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:45:35.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.585+0000 7fb2a5600700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:45:35.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.586+0000 7fb2a5600700 1 -- 172.21.15.33:0/4057046470 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b000f990 con 0x560eee336250 2024-05-03T18:45:35.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.586+0000 7fb2a5600700 1 -- 172.21.15.33:0/4057046470 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9518+0+0 (secure 0 0 0) 0x7fb2b0005110 con 0x560eee336250 2024-05-03T18:45:35.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.586+0000 7fb2c90dcb80 1 -- 172.21.15.33:0/4057046470 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2a80ad370 con 0x560eee336250 2024-05-03T18:45:35.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.587+0000 7fb2a5600700 1 -- 172.21.15.33:0/4057046470 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fb2b0027050 con 0x560eee336250 2024-05-03T18:45:35.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.587+0000 7fb2c90dcb80 10 client.16962 fetch_fsmap learned FSMap version 21 2024-05-03T18:45:35.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.587+0000 7fb2c90dcb80 1 -- 172.21.15.33:0/4057046470 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x560eee347250 con 0x560eee336250 2024-05-03T18:45:35.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.588+0000 7fb2a5600700 1 -- 172.21.15.33:0/4057046470 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb2a80ad370 con 0x560eee336250 2024-05-03T18:45:35.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.588+0000 7fb2a5600700 1 -- 172.21.15.33:0/4057046470 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fb2b000b650 con 0x560eee336250 2024-05-03T18:45:35.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.588+0000 7fb2c90dcb80 10 client.16962 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:45:35.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.588+0000 7fb2c90dcb80 10 client.16962 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:45:35.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.588+0000 7fb2c90dcb80 20 client.16962 populate_metadata read hostname 'smithi033' 2024-05-03T18:45:35.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.588+0000 7fb2c90dcb80 1 --2- 172.21.15.33:0/4057046470 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x560eee349880 0x560eee34bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:35.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.588+0000 7fb2c90dcb80 4 client.16962 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:45:35.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.588+0000 7fb2c90dcb80 1 -- 172.21.15.33:0/4057046470 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560eee2fcff0 con 0x560eee349880 2024-05-03T18:45:35.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.589+0000 7fb2b7400700 1 --2- 172.21.15.33:0/4057046470 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x560eee349880 0x560eee34bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:35.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.590+0000 7fb2b7400700 1 --2- 172.21.15.33:0/4057046470 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x560eee349880 0x560eee34bca0 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f01ffec5c8b5781 server_cookie=571df7ad8469025 in_seq=0 out_seq=0 2024-05-03T18:45:35.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.590+0000 7fb2a5600700 10 client.16962 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:45:35.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.591+0000 7fb2a5600700 1 -- 172.21.15.33:0/4057046470 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560eee2fcff0 con 0x560eee349880 2024-05-03T18:45:35.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.591+0000 7fb2a5600700 10 client.16962 handle_command_reply: tid=0 2024-05-03T18:45:35.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.592+0000 7fb2c90dcb80 1 client.16962 shutdown 2024-05-03T18:45:35.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.592+0000 7fb2c90dcb80 1 -- 172.21.15.33:0/4057046470 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560eee336250 msgr2=0x560eee339720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:35.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.592+0000 7fb2c90dcb80 1 --2- 172.21.15.33:0/4057046470 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560eee336250 0x560eee339720 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fb2b0009230 tx=0x7fb2b000f690 comp rx=0 tx=0).stop 2024-05-03T18:45:35.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.592+0000 7fb2c90dcb80 1 -- 172.21.15.33:0/4057046470 shutdown_connections 2024-05-03T18:45:35.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.592+0000 7fb2c90dcb80 1 --2- 172.21.15.33:0/4057046470 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x560eee349880 0x560eee34bca0 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:35.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.592+0000 7fb2c90dcb80 1 --2- 172.21.15.33:0/4057046470 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560eee336250 0x560eee339720 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:35.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.592+0000 7fb2c90dcb80 1 --2- 172.21.15.33:0/4057046470 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560eee330550 0x560eee33be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:35.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.592+0000 7fb2c90dcb80 1 --2- 172.21.15.33:0/4057046470 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560eee335930 0x560eee3391e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:35.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.592+0000 7fb2c90dcb80 1 -- 172.21.15.33:0/4057046470 >> 172.21.15.33:0/4057046470 conn(0x560eee2a86d0 msgr2=0x560eee3253b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:45:35.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.593+0000 7fb2a5600700 0 client.16962 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:45:35.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.593+0000 7fb2c90dcb80 1 -- 172.21.15.33:0/4057046470 shutdown_connections 2024-05-03T18:45:35.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.593+0000 7fb2c90dcb80 1 -- 172.21.15.33:0/4057046470 wait complete. 2024-05-03T18:45:35.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.593+0000 7fb2c90dcb80 20 client.16962 trim_cache size 0 max 16384 2024-05-03T18:45:35.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.640+0000 7fb2c90dcb80 1 Processor -- start 2024-05-03T18:45:35.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.641+0000 7fb2c90dcb80 1 -- start start 2024-05-03T18:45:35.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.641+0000 7fb2c90dcb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560eee330550 0x560eee32ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:35.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.641+0000 7fb2c90dcb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560eee335930 0x560eee351530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:35.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.641+0000 7fb2c90dcb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560eee336250 0x560eee351a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:35.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.641+0000 7fb2c90dcb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb29400cfa0 con 0x560eee335930 2024-05-03T18:45:35.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.641+0000 7fb2c90dcb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb29400f800 con 0x560eee330550 2024-05-03T18:45:35.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.641+0000 7fb2c90dcb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb29400d5f0 con 0x560eee336250 2024-05-03T18:45:35.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.642+0000 7fb2b7400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560eee330550 0x560eee32ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:35.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.642+0000 7fb2b7400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560eee330550 0x560eee32ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:40476/0 (socket says 172.21.15.33:40476) 2024-05-03T18:45:35.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.642+0000 7fb2b7400700 1 -- 172.21.15.33:0/110373088 learned_addr learned my addr 172.21.15.33:0/110373088 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:45:35.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.642+0000 7fb2b7e00700 1 --2- 172.21.15.33:0/110373088 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560eee336250 0x560eee351a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:35.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.642+0000 7fb2aea00700 1 --2- 172.21.15.33:0/110373088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560eee335930 0x560eee351530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:35.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.644+0000 7fb2b7400700 1 -- 172.21.15.33:0/110373088 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560eee336250 msgr2=0x560eee351a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:45:35.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.644+0000 7fb2b7400700 1 --2- 172.21.15.33:0/110373088 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560eee336250 0x560eee351a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:35.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.644+0000 7fb2b7400700 1 -- 172.21.15.33:0/110373088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560eee335930 msgr2=0x560eee351530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:45:35.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.644+0000 7fb2b7400700 1 --2- 172.21.15.33:0/110373088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560eee335930 0x560eee351530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:35.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.644+0000 7fb2b7400700 1 -- 172.21.15.33:0/110373088 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560eee339a30 con 0x560eee330550 2024-05-03T18:45:35.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.645+0000 7fb2b7400700 1 --2- 172.21.15.33:0/110373088 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560eee330550 0x560eee32ff30 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb2a80a5530 tx=0x7fb2a80a8ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=baf688d0b3494c62 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:45:35.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.645+0000 7fb2a7e00700 1 -- 172.21.15.33:0/110373088 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a80c2040 con 0x560eee330550 2024-05-03T18:45:35.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.645+0000 7fb2a7e00700 1 -- 172.21.15.33:0/110373088 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb2a80ad370 con 0x560eee330550 2024-05-03T18:45:35.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.645+0000 7fb2a7e00700 1 -- 172.21.15.33:0/110373088 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a80dfb80 con 0x560eee330550 2024-05-03T18:45:35.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.647+0000 7fb2c90dcb80 1 -- 172.21.15.33:0/110373088 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560eee330550 msgr2=0x560eee32ff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:35.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.647+0000 7fb2c90dcb80 1 --2- 172.21.15.33:0/110373088 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560eee330550 0x560eee32ff30 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb2a80a5530 tx=0x7fb2a80a8ec0 comp rx=0 tx=0).stop 2024-05-03T18:45:35.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.648+0000 7fb2c90dcb80 1 -- 172.21.15.33:0/110373088 shutdown_connections 2024-05-03T18:45:35.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.648+0000 7fb2c90dcb80 1 --2- 172.21.15.33:0/110373088 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560eee336250 0x560eee351a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:35.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.648+0000 7fb2c90dcb80 1 --2- 172.21.15.33:0/110373088 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560eee330550 0x560eee32ff30 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:35.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.649+0000 7fb2c90dcb80 1 --2- 172.21.15.33:0/110373088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560eee335930 0x560eee351530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:35.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.649+0000 7fb2c90dcb80 1 -- 172.21.15.33:0/110373088 >> 172.21.15.33:0/110373088 conn(0x560eee2a86d0 msgr2=0x560eee337ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:45:35.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.649+0000 7fb2c90dcb80 1 -- 172.21.15.33:0/110373088 shutdown_connections 2024-05-03T18:45:35.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.650+0000 7fb2c90dcb80 1 -- 172.21.15.33:0/110373088 wait complete. 2024-05-03T18:45:35.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.651+0000 7fb2c90dcb80 1 -- start start 2024-05-03T18:45:35.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.651+0000 7fb2c90dcb80 1 Processor -- start 2024-05-03T18:45:35.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.653+0000 7fb2c90dcb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560eee330550 0x560eee342170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:35.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.653+0000 7fb2c90dcb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560eee335930 0x560eee3426b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:35.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.654+0000 7fb2c90dcb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560eee336250 0x560eee33f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:35.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.654+0000 7fb2b7400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560eee330550 0x560eee342170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:35.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.654+0000 7fb2aea00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560eee335930 0x560eee3426b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:35.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.654+0000 7fb2b7400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560eee330550 0x560eee342170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:53874/0 (socket says 172.21.15.33:53874) 2024-05-03T18:45:35.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.654+0000 7fb2b7400700 1 -- 172.21.15.33:0/2451194210 learned_addr learned my addr 172.21.15.33:0/2451194210 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:45:35.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.655+0000 7fb2c90dcb80 1 -- 172.21.15.33:0/2451194210 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb29400dcc0 con 0x560eee336250 2024-05-03T18:45:35.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.655+0000 7fb2c90dcb80 1 -- 172.21.15.33:0/2451194210 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb29400f130 con 0x560eee335930 2024-05-03T18:45:35.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.655+0000 7fb2c90dcb80 1 -- 172.21.15.33:0/2451194210 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb29400e390 con 0x560eee330550 2024-05-03T18:45:35.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.655+0000 7fb2c90dcb80 10 client.0 _ll_register_callbacks cb 0x560eee229c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:45:35.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.656+0000 7fb2b7e00700 1 --2- 172.21.15.33:0/2451194210 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560eee336250 0x560eee33f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:35.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.657+0000 7fb2b7400700 1 -- 172.21.15.33:0/2451194210 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560eee335930 msgr2=0x560eee3426b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:45:35.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.657+0000 7fb2b7400700 1 --2- 172.21.15.33:0/2451194210 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560eee335930 0x560eee3426b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:35.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.657+0000 7fb2b7400700 1 -- 172.21.15.33:0/2451194210 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560eee336250 msgr2=0x560eee33f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:45:35.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.657+0000 7fb2b7400700 1 --2- 172.21.15.33:0/2451194210 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560eee336250 0x560eee33f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:35.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.657+0000 7fb2b7400700 1 -- 172.21.15.33:0/2451194210 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560eee33f680 con 0x560eee330550 2024-05-03T18:45:35.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.658+0000 7fb2b7400700 1 --2- 172.21.15.33:0/2451194210 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560eee330550 0x560eee342170 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb2a80a8f50 tx=0x7fb2a80074e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b0e39723983a702 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:45:35.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.658+0000 7fb2a5600700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-03T18:45:35.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.660+0000 7fb2a5600700 1 -- 172.21.15.33:0/2451194210 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a8087070 con 0x560eee330550 2024-05-03T18:45:35.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.660+0000 7fb2a5600700 1 -- 172.21.15.33:0/2451194210 <== mon.1 v2:172.21.15.150:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9518+0+0 (secure 0 0 0) 0x7fb2a80a9420 con 0x560eee330550 2024-05-03T18:45:35.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.660+0000 7fb2c90dcb80 1 -- 172.21.15.33:0/2451194210 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2a80ad370 con 0x560eee330550 2024-05-03T18:45:35.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.660+0000 7fb2a5600700 1 -- 172.21.15.33:0/2451194210 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fb2a80b5020 con 0x560eee330550 2024-05-03T18:45:35.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.661+0000 7fb2c90dcb80 10 client.26833 fetch_fsmap learned FSMap version 21 2024-05-03T18:45:35.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.661+0000 7fb2c90dcb80 1 -- 172.21.15.33:0/2451194210 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x560eee3b4920 con 0x560eee330550 2024-05-03T18:45:35.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.662+0000 7fb2a5600700 1 -- 172.21.15.33:0/2451194210 <== mon.1 v2:172.21.15.150:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb2a80ad370 con 0x560eee330550 2024-05-03T18:45:35.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.662+0000 7fb2a5600700 1 -- 172.21.15.33:0/2451194210 <== mon.1 v2:172.21.15.150:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fb2a80ab070 con 0x560eee330550 2024-05-03T18:45:35.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.663+0000 7fb2c90dcb80 10 client.26833 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:45:35.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.663+0000 7fb2c90dcb80 10 client.26833 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:45:35.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.663+0000 7fb2c90dcb80 20 client.26833 populate_metadata read hostname 'smithi033' 2024-05-03T18:45:35.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.663+0000 7fb2c90dcb80 1 --2- 172.21.15.33:0/2451194210 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x560eee3b4b40 0x560eee3b6f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:35.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.663+0000 7fb2c90dcb80 4 client.26833 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-03T18:45:35.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.663+0000 7fb2c90dcb80 1 -- 172.21.15.33:0/2451194210 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560eee290400 con 0x560eee3b4b40 2024-05-03T18:45:35.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.663+0000 7fb2aea00700 1 --2- 172.21.15.33:0/2451194210 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x560eee3b4b40 0x560eee3b6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:35.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.664+0000 7fb2aea00700 1 --2- 172.21.15.33:0/2451194210 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x560eee3b4b40 0x560eee3b6f30 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=caf780d2d27a5507 server_cookie=c4b74b00a86049ea in_seq=0 out_seq=0 2024-05-03T18:45:35.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.664+0000 7fb2a5600700 10 client.26833 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:45:35.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.665+0000 7fb2a5600700 1 -- 172.21.15.33:0/2451194210 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x560eee290400 con 0x560eee3b4b40 2024-05-03T18:45:35.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.665+0000 7fb2a5600700 10 client.26833 handle_command_reply: tid=0 2024-05-03T18:45:35.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.665+0000 7fb2c90dcb80 1 client.26833 shutdown 2024-05-03T18:45:35.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.665+0000 7fb2c90dcb80 1 -- 172.21.15.33:0/2451194210 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560eee330550 msgr2=0x560eee342170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:35.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.665+0000 7fb2c90dcb80 1 --2- 172.21.15.33:0/2451194210 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560eee330550 0x560eee342170 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb2a80a8f50 tx=0x7fb2a80074e0 comp rx=0 tx=0).stop 2024-05-03T18:45:35.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.665+0000 7fb2c90dcb80 1 -- 172.21.15.33:0/2451194210 shutdown_connections 2024-05-03T18:45:35.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.665+0000 7fb2c90dcb80 1 --2- 172.21.15.33:0/2451194210 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x560eee3b4b40 0x560eee3b6f30 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:35.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.666+0000 7fb2a5600700 0 client.26833 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:45:35.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.666+0000 7fb2c90dcb80 1 --2- 172.21.15.33:0/2451194210 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560eee336250 0x560eee33f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:35.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.666+0000 7fb2c90dcb80 1 --2- 172.21.15.33:0/2451194210 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560eee330550 0x560eee342170 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:35.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.666+0000 7fb2c90dcb80 1 --2- 172.21.15.33:0/2451194210 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560eee335930 0x560eee3426b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:35.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.666+0000 7fb2c90dcb80 1 -- 172.21.15.33:0/2451194210 >> 172.21.15.33:0/2451194210 conn(0x560eee2a86d0 msgr2=0x560eee336630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:45:35.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.667+0000 7fb2c90dcb80 1 -- 172.21.15.33:0/2451194210 shutdown_connections 2024-05-03T18:45:35.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.668+0000 7fb2c90dcb80 1 -- 172.21.15.33:0/2451194210 wait complete. 2024-05-03T18:45:35.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.668+0000 7fb2c90dcb80 20 client.26833 trim_cache size 0 max 16384 2024-05-03T18:45:35.671 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-03T18:45:35.671 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-05-03T18:45:35.671 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-05-03T18:45:35.671 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-03T18:45:35.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.669+0000 7fb2be800700 1 -- 172.21.15.33:0/2218871657 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fb29c0b2b20 msgr2=0x7fb29c0b4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:35.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.669+0000 7fb2be800700 1 --2- 172.21.15.33:0/2218871657 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fb29c0b2b20 0x7fb29c0b4fe0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb2b8070440 tx=0x7fb2a0009190 comp rx=0 tx=0).stop 2024-05-03T18:45:35.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.669+0000 7fb2be800700 1 -- 172.21.15.33:0/2218871657 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2b806b6f0 msgr2=0x7fb2b8074c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:35.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.669+0000 7fb2be800700 1 --2- 172.21.15.33:0/2218871657 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2b806b6f0 0x7fb2b8074c00 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb2a8007fa0 tx=0x7fb2a800d3b0 comp rx=0 tx=0).stop 2024-05-03T18:45:35.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.670+0000 7fb2be800700 1 -- 172.21.15.33:0/2218871657 shutdown_connections 2024-05-03T18:45:35.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.670+0000 7fb2be800700 1 --2- 172.21.15.33:0/2218871657 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2b81083d0 0x7fb2b806eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:35.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.670+0000 7fb2be800700 1 --2- 172.21.15.33:0/2218871657 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fb29c0b2b20 0x7fb29c0b4fe0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:35.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.670+0000 7fb2be800700 1 --2- 172.21.15.33:0/2218871657 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2b806b6f0 0x7fb2b8074c00 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:35.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.670+0000 7fb2be800700 1 --2- 172.21.15.33:0/2218871657 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2b8105ee0 0x7fb2b8075140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:35.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.670+0000 7fb2be800700 1 -- 172.21.15.33:0/2218871657 >> 172.21.15.33:0/2218871657 conn(0x7fb2b8079350 msgr2=0x7fb2b8102eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:45:35.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.671+0000 7fb2be800700 1 -- 172.21.15.33:0/2218871657 shutdown_connections 2024-05-03T18:45:35.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:35.671+0000 7fb2be800700 1 -- 172.21.15.33:0/2218871657 wait complete. 2024-05-03T18:45:36.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:45:36 smithi150 ceph-mon[32492]: pgmap v580: 129 pgs: 129 active+clean; 3.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 6.3 MiB/s wr, 572 op/s 2024-05-03T18:45:36.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:45:36 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:45:36.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:45:36 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:45:36.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:45:36 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:45:36.391 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:45:36 smithi033 ceph-mon[28330]: pgmap v580: 129 pgs: 129 active+clean; 3.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 6.3 MiB/s wr, 572 op/s 2024-05-03T18:45:36.391 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:45:36 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:45:36.391 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:45:36 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:45:36.391 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:45:36 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:45:36.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:45:36 smithi099 ceph-mon[31658]: pgmap v580: 129 pgs: 129 active+clean; 3.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 6.3 MiB/s wr, 572 op/s 2024-05-03T18:45:36.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:45:36 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:45:36.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:45:36 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:45:36.547 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:45:36 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:45:37.196 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-03T18:45:37.196 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T18:45:37.959 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:45:38.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:45:38 smithi150 ceph-mon[32492]: pgmap v581: 129 pgs: 129 active+clean; 3.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 6.6 MiB/s wr, 559 op/s 2024-05-03T18:45:38.397 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:45:38 smithi033 ceph-mon[28330]: pgmap v581: 129 pgs: 129 active+clean; 3.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 6.6 MiB/s wr, 559 op/s 2024-05-03T18:45:38.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:45:38 smithi099 ceph-mon[31658]: pgmap v581: 129 pgs: 129 active+clean; 3.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 6.6 MiB/s wr, 559 op/s 2024-05-03T18:45:40.111 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 865.0 s, 581.7 tps, lat 175.730 ms stddev 112.472 2024-05-03T18:45:40.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:45:40 smithi150 ceph-mon[32492]: pgmap v582: 129 pgs: 129 active+clean; 3.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 5.0 MiB/s wr, 429 op/s 2024-05-03T18:45:40.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:45:40 smithi033 ceph-mon[28330]: pgmap v582: 129 pgs: 129 active+clean; 3.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 5.0 MiB/s wr, 429 op/s 2024-05-03T18:45:40.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:45:40 smithi099 ceph-mon[31658]: pgmap v582: 129 pgs: 129 active+clean; 3.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 5.0 MiB/s wr, 429 op/s 2024-05-03T18:45:41.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:41.619+0000 7fac29200700 1 -- 172.21.15.33:0/1959675583 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac24105ee0 msgr2=0x7fac24106370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:41.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:41.619+0000 7fac29200700 1 --2- 172.21.15.33:0/1959675583 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac24105ee0 0x7fac24106370 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fac1c00b0a0 tx=0x7fac1c00b3b0 comp rx=0 tx=0).stop 2024-05-03T18:45:41.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:41.620+0000 7fac29200700 1 -- 172.21.15.33:0/1959675583 shutdown_connections 2024-05-03T18:45:41.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:41.620+0000 7fac29200700 1 --2- 172.21.15.33:0/1959675583 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac24105ee0 0x7fac24106370 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:41.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:41.620+0000 7fac29200700 1 --2- 172.21.15.33:0/1959675583 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac241083d0 0x7fac2406b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:41.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:41.620+0000 7fac29200700 1 --2- 172.21.15.33:0/1959675583 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac2406b6f0 0x7fac2406bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:41.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:41.620+0000 7fac29200700 1 -- 172.21.15.33:0/1959675583 >> 172.21.15.33:0/1959675583 conn(0x7fac24079350 msgr2=0x7fac24079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:45:41.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:41.621+0000 7fac29200700 1 -- 172.21.15.33:0/1959675583 shutdown_connections 2024-05-03T18:45:41.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:41.621+0000 7fac29200700 1 -- 172.21.15.33:0/1959675583 wait complete. 2024-05-03T18:45:41.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:41.623+0000 7fac29200700 1 Processor -- start 2024-05-03T18:45:41.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:41.623+0000 7fac29200700 1 -- start start 2024-05-03T18:45:41.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:41.624+0000 7fac29200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac2406b6f0 0x7fac24108b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:41.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:41.624+0000 7fac29200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac24105ee0 0x7fac241090d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:41.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:41.624+0000 7fac29200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac241083d0 0x7fac2410da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:41.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:41.624+0000 7fac29200700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fac2406d490 con 0x7fac2406b6f0 2024-05-03T18:45:41.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:41.624+0000 7fac29200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fac2406d310 con 0x7fac241083d0 2024-05-03T18:45:41.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:41.624+0000 7fac29200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fac2406d610 con 0x7fac24105ee0 2024-05-03T18:45:41.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:41.625+0000 7fac21400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac24105ee0 0x7fac241090d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:41.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:41.625+0000 7fac22800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac241083d0 0x7fac2410da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:41.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:41.625+0000 7fac21400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac24105ee0 0x7fac241090d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:41292/0 (socket says 172.21.15.33:41292) 2024-05-03T18:45:41.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:41.625+0000 7fac21400700 1 -- 172.21.15.33:0/1625264258 learned_addr learned my addr 172.21.15.33:0/1625264258 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:45:41.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:41.625+0000 7fac22800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac241083d0 0x7fac2410da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:54820/0 (socket says 172.21.15.33:54820) 2024-05-03T18:45:41.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:41.625+0000 7fac21e00700 1 --2- 172.21.15.33:0/1625264258 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac2406b6f0 0x7fac24108b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:41.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:41.626+0000 7fac21400700 1 -- 172.21.15.33:0/1625264258 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac241083d0 msgr2=0x7fac2410da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:41.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:41.626+0000 7fac21400700 1 --2- 172.21.15.33:0/1625264258 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac241083d0 0x7fac2410da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:41.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:41.626+0000 7fac21400700 1 -- 172.21.15.33:0/1625264258 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac2406b6f0 msgr2=0x7fac24108b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:41.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:41.626+0000 7fac21400700 1 --2- 172.21.15.33:0/1625264258 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac2406b6f0 0x7fac24108b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:41.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:41.626+0000 7fac21400700 1 -- 172.21.15.33:0/1625264258 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac2410e150 con 0x7fac24105ee0 2024-05-03T18:45:41.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:41.626+0000 7fac21e00700 1 --2- 172.21.15.33:0/1625264258 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac2406b6f0 0x7fac24108b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:45:41.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:41.627+0000 7fac22800700 1 --2- 172.21.15.33:0/1625264258 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac241083d0 0x7fac2410da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:45:41.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:41.627+0000 7fac21400700 1 --2- 172.21.15.33:0/1625264258 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac24105ee0 0x7fac241090d0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fac1000b530 tx=0x7fac1000b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:45:41.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:41.628+0000 7fac1aa00700 1 -- 172.21.15.33:0/1625264258 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac10015020 con 0x7fac24105ee0 2024-05-03T18:45:41.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:41.628+0000 7fac1aa00700 1 -- 172.21.15.33:0/1625264258 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fac10011cc0 con 0x7fac24105ee0 2024-05-03T18:45:41.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:41.628+0000 7fac1aa00700 1 -- 172.21.15.33:0/1625264258 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac10010bf0 con 0x7fac24105ee0 2024-05-03T18:45:41.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:41.628+0000 7fac29200700 1 -- 172.21.15.33:0/1625264258 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac2410e3e0 con 0x7fac24105ee0 2024-05-03T18:45:41.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:41.629+0000 7fac29200700 1 -- 172.21.15.33:0/1625264258 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac24114ae0 con 0x7fac24105ee0 2024-05-03T18:45:41.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:41.631+0000 7fac29200700 1 -- 172.21.15.33:0/1625264258 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac241029c0 con 0x7fac24105ee0 2024-05-03T18:45:41.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:41.637+0000 7fac1aa00700 1 -- 172.21.15.33:0/1625264258 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fac10018020 con 0x7fac24105ee0 2024-05-03T18:45:41.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:41.645+0000 7fac1aa00700 1 --2- 172.21.15.33:0/1625264258 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fac080b2c00 0x7fac080b50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:41.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:41.645+0000 7fac1aa00700 1 -- 172.21.15.33:0/1625264258 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 9518+0+0 (secure 0 0 0) 0x7fac100e2530 con 0x7fac24105ee0 2024-05-03T18:45:41.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:41.645+0000 7fac1aa00700 1 -- 172.21.15.33:0/1625264258 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fac1001f020 con 0x7fac24105ee0 2024-05-03T18:45:41.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:41.646+0000 7fac21e00700 1 --2- 172.21.15.33:0/1625264258 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fac080b2c00 0x7fac080b50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:41.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:41.649+0000 7fac21e00700 1 --2- 172.21.15.33:0/1625264258 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fac080b2c00 0x7fac080b50c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fac14005fd0 tx=0x7fac14005e30 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:45:42.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:42.175+0000 7fac29200700 1 -- 172.21.15.33:0/1625264258 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fac2406a190 con 0x7fac24105ee0 2024-05-03T18:45:42.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:42.177+0000 7fac1aa00700 1 -- 172.21.15.33:0/1625264258 <== mon.1 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11801 (secure 0 0 0) 0x7fac2406a190 con 0x7fac24105ee0 2024-05-03T18:45:42.180 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:45:42.180 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1327393503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1327393503},{"type":"v1","addr":"172.21.15.33:6835","nonce":1327393503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/2461472948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":2461472948},{"type":"v1","addr":"172.21.15.33:6837","nonce":2461472948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14559,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/442269281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":442269281},{"type":"v1","addr":"172.21.15.33:6839","nonce":442269281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14571,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/645571756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":645571756},{"type":"v1","addr":"172.21.15.33:6841","nonce":645571756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/203598909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":203598909},{"type":"v1","addr":"172.21.15.150:6833","nonce":203598909}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/4029842493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":4029842493},{"type":"v1","addr":"172.21.15.150:6835","nonce":4029842493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3463664423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3463664423},{"type":"v1","addr":"172.21.15.150:6837","nonce":3463664423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24481,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6839/4051648018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":4051648018},{"type":"v1","addr":"172.21.15.150:6839","nonce":4051648018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34418,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1865105479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1865105479},{"type":"v1","addr":"172.21.15.99:6833","nonce":1865105479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34445,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/1206393914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1206393914},{"type":"v1","addr":"172.21.15.99:6835","nonce":1206393914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-03T18:22:28.837091+0000","modified":"2024-05-03T18:23:49.758398+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34508},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34472":{"gid":34472,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6837/3360970342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3360970342},{"type":"v1","addr":"172.21.15.99:6837","nonce":3360970342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34508":{"gid":34508,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/919907308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":919907308},{"type":"v1","addr":"172.21.15.99:6839","nonce":919907308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34508,"qdb_cluster":[34508]},"id":1}]} 2024-05-03T18:45:42.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:42.183+0000 7fac29200700 1 -- 172.21.15.33:0/1625264258 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fac080b2c00 msgr2=0x7fac080b50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:42.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:42.183+0000 7fac29200700 1 --2- 172.21.15.33:0/1625264258 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fac080b2c00 0x7fac080b50c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fac14005fd0 tx=0x7fac14005e30 comp rx=0 tx=0).stop 2024-05-03T18:45:42.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:42.183+0000 7fac29200700 1 -- 172.21.15.33:0/1625264258 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac24105ee0 msgr2=0x7fac241090d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:42.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:42.183+0000 7fac29200700 1 --2- 172.21.15.33:0/1625264258 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac24105ee0 0x7fac241090d0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fac1000b530 tx=0x7fac1000b840 comp rx=0 tx=0).stop 2024-05-03T18:45:42.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:42.183+0000 7fac29200700 1 -- 172.21.15.33:0/1625264258 shutdown_connections 2024-05-03T18:45:42.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:42.183+0000 7fac29200700 1 --2- 172.21.15.33:0/1625264258 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fac080b2c00 0x7fac080b50c0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:42.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:42.183+0000 7fac29200700 1 --2- 172.21.15.33:0/1625264258 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac241083d0 0x7fac2410da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:42.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:42.183+0000 7fac29200700 1 --2- 172.21.15.33:0/1625264258 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac2406b6f0 0x7fac24108b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:42.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:42.183+0000 7fac29200700 1 --2- 172.21.15.33:0/1625264258 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac24105ee0 0x7fac241090d0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:42.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:42.183+0000 7fac29200700 1 -- 172.21.15.33:0/1625264258 >> 172.21.15.33:0/1625264258 conn(0x7fac24079350 msgr2=0x7fac24104ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:45:42.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:42.184+0000 7fac29200700 1 -- 172.21.15.33:0/1625264258 shutdown_connections 2024-05-03T18:45:42.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:42.186+0000 7fac29200700 1 -- 172.21.15.33:0/1625264258 wait complete. 2024-05-03T18:45:42.196 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 21 2024-05-03T18:45:42.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:45:42 smithi150 ceph-mon[32492]: pgmap v583: 129 pgs: 129 active+clean; 3.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 7.1 MiB/s wr, 643 op/s 2024-05-03T18:45:42.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:45:42 smithi033 ceph-mon[28330]: pgmap v583: 129 pgs: 129 active+clean; 3.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 7.1 MiB/s wr, 643 op/s 2024-05-03T18:45:42.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:45:42 smithi099 ceph-mon[31658]: pgmap v583: 129 pgs: 129 active+clean; 3.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 7.1 MiB/s wr, 643 op/s 2024-05-03T18:45:43.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:45:43 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/1625264258' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:45:43.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:45:43 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/1625264258' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:45:43.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:45:43 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/1625264258' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:45:44.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:45:44 smithi033 ceph-mon[28330]: pgmap v584: 129 pgs: 129 active+clean; 3.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 4.6 MiB/s wr, 424 op/s 2024-05-03T18:45:44.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:45:44 smithi150 ceph-mon[32492]: pgmap v584: 129 pgs: 129 active+clean; 3.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 4.6 MiB/s wr, 424 op/s 2024-05-03T18:45:44.400 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-03T18:45:44.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:45:44 smithi099 ceph-mon[31658]: pgmap v584: 129 pgs: 129 active+clean; 3.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 4.6 MiB/s wr, 424 op/s 2024-05-03T18:45:45.111 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 870.0 s, 596.8 tps, lat 166.030 ms stddev 69.248 2024-05-03T18:45:45.213 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:45:46.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:45:46 smithi150 ceph-mon[32492]: pgmap v585: 129 pgs: 129 active+clean; 3.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 5.6 MiB/s wr, 530 op/s 2024-05-03T18:45:46.486 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:45:46 smithi033 ceph-mon[28330]: pgmap v585: 129 pgs: 129 active+clean; 3.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 5.6 MiB/s wr, 530 op/s 2024-05-03T18:45:46.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:45:46 smithi099 ceph-mon[31658]: pgmap v585: 129 pgs: 129 active+clean; 3.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 5.6 MiB/s wr, 530 op/s 2024-05-03T18:45:48.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:45:48 smithi033 ceph-mon[28330]: pgmap v586: 129 pgs: 129 active+clean; 3.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 6.4 MiB/s wr, 550 op/s 2024-05-03T18:45:48.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:45:48 smithi099 ceph-mon[31658]: pgmap v586: 129 pgs: 129 active+clean; 3.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 6.4 MiB/s wr, 550 op/s 2024-05-03T18:45:48.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:45:48 smithi150 ceph-mon[32492]: pgmap v586: 129 pgs: 129 active+clean; 3.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 6.4 MiB/s wr, 550 op/s 2024-05-03T18:45:50.111 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 875.0 s, 602.4 tps, lat 165.237 ms stddev 63.521 2024-05-03T18:45:50.236 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:45:50 smithi033 ceph-mon[28330]: pgmap v587: 129 pgs: 129 active+clean; 3.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 5.0 MiB/s wr, 441 op/s 2024-05-03T18:45:50.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.403+0000 7fcd7a600700 1 -- 172.21.15.33:0/680172373 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcd74105ee0 msgr2=0x7fcd74106370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:50.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.403+0000 7fcd7a600700 1 --2- 172.21.15.33:0/680172373 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcd74105ee0 0x7fcd74106370 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c00b0a0 tx=0x7fcd6c00b3b0 comp rx=0 tx=0).stop 2024-05-03T18:45:50.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.403+0000 7fcd7a600700 1 -- 172.21.15.33:0/680172373 shutdown_connections 2024-05-03T18:45:50.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.403+0000 7fcd7a600700 1 --2- 172.21.15.33:0/680172373 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcd74105ee0 0x7fcd74106370 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:50.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.403+0000 7fcd7a600700 1 --2- 172.21.15.33:0/680172373 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd741083d0 0x7fcd7406b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:50.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.403+0000 7fcd7a600700 1 --2- 172.21.15.33:0/680172373 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd7406b6f0 0x7fcd7406bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:50.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.403+0000 7fcd7a600700 1 -- 172.21.15.33:0/680172373 >> 172.21.15.33:0/680172373 conn(0x7fcd74079350 msgr2=0x7fcd74079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:45:50.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.404+0000 7fcd7a600700 1 -- 172.21.15.33:0/680172373 shutdown_connections 2024-05-03T18:45:50.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.404+0000 7fcd7a600700 1 -- 172.21.15.33:0/680172373 wait complete. 2024-05-03T18:45:50.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.406+0000 7fcd7a600700 1 Processor -- start 2024-05-03T18:45:50.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.407+0000 7fcd7a600700 1 -- start start 2024-05-03T18:45:50.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.408+0000 7fcd7a600700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd7406b6f0 0x7fcd741a0080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:50.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.408+0000 7fcd7a600700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcd74105ee0 0x7fcd741a05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:50.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.408+0000 7fcd7a600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd741083d0 0x7fcd741a4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:50.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.408+0000 7fcd7a600700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcd74119b10 con 0x7fcd7406b6f0 2024-05-03T18:45:50.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.408+0000 7fcd7a600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fcd74119990 con 0x7fcd741083d0 2024-05-03T18:45:50.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.408+0000 7fcd7a600700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fcd74119c90 con 0x7fcd74105ee0 2024-05-03T18:45:50.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.408+0000 7fcd72a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcd74105ee0 0x7fcd741a05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:50.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.408+0000 7fcd72a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcd74105ee0 0x7fcd741a05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:39104/0 (socket says 172.21.15.33:39104) 2024-05-03T18:45:50.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.408+0000 7fcd72a00700 1 -- 172.21.15.33:0/2680870108 learned_addr learned my addr 172.21.15.33:0/2680870108 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:45:50.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.409+0000 7fcd72a00700 1 -- 172.21.15.33:0/2680870108 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd741083d0 msgr2=0x7fcd741a4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:50.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.409+0000 7fcd72a00700 1 --2- 172.21.15.33:0/2680870108 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd741083d0 0x7fcd741a4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:50.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.409+0000 7fcd72a00700 1 -- 172.21.15.33:0/2680870108 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd7406b6f0 msgr2=0x7fcd741a0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:50.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.409+0000 7fcd72a00700 1 --2- 172.21.15.33:0/2680870108 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd7406b6f0 0x7fcd741a0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:50.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.409+0000 7fcd72a00700 1 -- 172.21.15.33:0/2680870108 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd741a5090 con 0x7fcd74105ee0 2024-05-03T18:45:50.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.411+0000 7fcd72a00700 1 --2- 172.21.15.33:0/2680870108 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcd74105ee0 0x7fcd741a05c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fcd6000d980 tx=0x7fcd6000dd40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:45:50.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.412+0000 7fcd6be00700 1 -- 172.21.15.33:0/2680870108 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd60016070 con 0x7fcd74105ee0 2024-05-03T18:45:50.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.412+0000 7fcd6be00700 1 -- 172.21.15.33:0/2680870108 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcd60004020 con 0x7fcd74105ee0 2024-05-03T18:45:50.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.412+0000 7fcd6be00700 1 -- 172.21.15.33:0/2680870108 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd60004c20 con 0x7fcd74105ee0 2024-05-03T18:45:50.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.414+0000 7fcd7a600700 1 -- 172.21.15.33:0/2680870108 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd741a5380 con 0x7fcd74105ee0 2024-05-03T18:45:50.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.414+0000 7fcd7a600700 1 -- 172.21.15.33:0/2680870108 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd741acc00 con 0x7fcd74105ee0 2024-05-03T18:45:50.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.429+0000 7fcd6be00700 1 -- 172.21.15.33:0/2680870108 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fcd6000bb60 con 0x7fcd74105ee0 2024-05-03T18:45:50.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.432+0000 7fcd6be00700 1 --2- 172.21.15.33:0/2680870108 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fcd580b2a70 0x7fcd580b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:50.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.432+0000 7fcd6be00700 1 -- 172.21.15.33:0/2680870108 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 9518+0+0 (secure 0 0 0) 0x7fcd600e2660 con 0x7fcd74105ee0 2024-05-03T18:45:50.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.433+0000 7fcd73400700 1 --2- 172.21.15.33:0/2680870108 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fcd580b2a70 0x7fcd580b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:50.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.436+0000 7fcd73400700 1 --2- 172.21.15.33:0/2680870108 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fcd580b2a70 0x7fcd580b4f30 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fcd64005e90 tx=0x7fcd64005d00 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:45:50.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.444+0000 7fcd84f25b80 1 Processor -- start 2024-05-03T18:45:50.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.444+0000 7fcd84f25b80 1 -- start start 2024-05-03T18:45:50.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.445+0000 7fcd84f25b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2c9bdc7b0 0x55e2c9bdcb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:50.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.445+0000 7fcd84f25b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e2c9bdd0d0 0x55e2c9bd7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:50.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.445+0000 7fcd84f25b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2c9bd85d0 0x55e2c9bd8a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:50.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.445+0000 7fcd84f25b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e2c9b2aa20 con 0x55e2c9bdc7b0 2024-05-03T18:45:50.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.445+0000 7fcd84f25b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55e2c9ac8760 con 0x55e2c9bd85d0 2024-05-03T18:45:50.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.445+0000 7fcd84f25b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55e2c9b269e0 con 0x55e2c9bdd0d0 2024-05-03T18:45:50.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.446+0000 7fcd73e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2c9bd85d0 0x55e2c9bd8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:50.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.446+0000 7fcd73e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2c9bd85d0 0x55e2c9bd8a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:34918/0 (socket says 172.21.15.33:34918) 2024-05-03T18:45:50.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.446+0000 7fcd73e00700 1 -- 172.21.15.33:0/147273726 learned_addr learned my addr 172.21.15.33:0/147273726 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:45:50.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.446+0000 7fcd72a00700 1 --2- 172.21.15.33:0/147273726 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e2c9bdd0d0 0x55e2c9bd7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:50.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.446+0000 7fcd73e00700 1 -- 172.21.15.33:0/147273726 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e2c9bdd0d0 msgr2=0x55e2c9bd7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:45:50.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.446+0000 7fcd73e00700 1 --2- 172.21.15.33:0/147273726 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e2c9bdd0d0 0x55e2c9bd7fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:50.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.446+0000 7fcd73e00700 1 -- 172.21.15.33:0/147273726 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2c9bdc7b0 msgr2=0x55e2c9bdcb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:45:50.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.446+0000 7fcd73e00700 1 --2- 172.21.15.33:0/147273726 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2c9bdc7b0 0x55e2c9bdcb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:50.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.446+0000 7fcd73e00700 1 -- 172.21.15.33:0/147273726 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2c9a778d0 con 0x55e2c9bd85d0 2024-05-03T18:45:50.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.449+0000 7fcd73e00700 1 --2- 172.21.15.33:0/147273726 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2c9bd85d0 0x55e2c9bd8a30 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c006070 tx=0x7fcd6c009d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=810eca14dfb13759 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:45:50.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.450+0000 7fcd68c00700 1 -- 172.21.15.33:0/147273726 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd6c0092d0 con 0x55e2c9bd85d0 2024-05-03T18:45:50.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.450+0000 7fcd68c00700 1 -- 172.21.15.33:0/147273726 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcd6c009470 con 0x55e2c9bd85d0 2024-05-03T18:45:50.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.450+0000 7fcd68c00700 1 -- 172.21.15.33:0/147273726 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd6c01b460 con 0x55e2c9bd85d0 2024-05-03T18:45:50.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.451+0000 7fcd84f25b80 1 -- 172.21.15.33:0/147273726 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2c9bd85d0 msgr2=0x55e2c9bd8a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:50.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.451+0000 7fcd84f25b80 1 --2- 172.21.15.33:0/147273726 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2c9bd85d0 0x55e2c9bd8a30 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c006070 tx=0x7fcd6c009d10 comp rx=0 tx=0).stop 2024-05-03T18:45:50.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.452+0000 7fcd84f25b80 1 -- 172.21.15.33:0/147273726 shutdown_connections 2024-05-03T18:45:50.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.452+0000 7fcd84f25b80 1 --2- 172.21.15.33:0/147273726 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2c9bd85d0 0x55e2c9bd8a30 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:50.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.452+0000 7fcd84f25b80 1 --2- 172.21.15.33:0/147273726 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2c9bdc7b0 0x55e2c9bdcb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:50.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.452+0000 7fcd84f25b80 1 --2- 172.21.15.33:0/147273726 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e2c9bdd0d0 0x55e2c9bd7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:50.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.452+0000 7fcd84f25b80 1 -- 172.21.15.33:0/147273726 >> 172.21.15.33:0/147273726 conn(0x55e2c9b06b60 msgr2=0x55e2c9bd27f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:45:50.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.452+0000 7fcd84f25b80 1 -- 172.21.15.33:0/147273726 shutdown_connections 2024-05-03T18:45:50.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.452+0000 7fcd84f25b80 1 -- 172.21.15.33:0/147273726 wait complete. 2024-05-03T18:45:50.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.453+0000 7fcd84f25b80 1 -- start start 2024-05-03T18:45:50.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.453+0000 7fcd84f25b80 1 Processor -- start 2024-05-03T18:45:50.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.455+0000 7fcd84f25b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2c9bd85d0 0x55e2c9be3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:50.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.455+0000 7fcd84f25b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2c9bdc7b0 0x55e2c9be12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:50.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.455+0000 7fcd84f25b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e2c9bdd0d0 0x55e2c9be1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:50.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.455+0000 7fcd84f25b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e2c9be4c40 con 0x55e2c9bdc7b0 2024-05-03T18:45:50.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.455+0000 7fcd84f25b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55e2c9a90b20 con 0x55e2c9bd85d0 2024-05-03T18:45:50.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.455+0000 7fcd84f25b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55e2c9be4dc0 con 0x55e2c9bdd0d0 2024-05-03T18:45:50.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.455+0000 7fcd84f25b80 10 client.0 _ll_register_callbacks cb 0x55e2c9ad1c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:45:50.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.456+0000 7fcd73e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e2c9bdd0d0 0x55e2c9be1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:50.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.456+0000 7fcd73e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e2c9bdd0d0 0x55e2c9be1830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:39118/0 (socket says 172.21.15.33:39118) 2024-05-03T18:45:50.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.456+0000 7fcd73e00700 1 -- 172.21.15.33:0/3498336381 learned_addr learned my addr 172.21.15.33:0/3498336381 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:45:50.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.456+0000 7fcd73400700 1 --2- 172.21.15.33:0/3498336381 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2c9bd85d0 0x55e2c9be3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:50.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.456+0000 7fcd73e00700 1 -- 172.21.15.33:0/3498336381 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2c9bd85d0 msgr2=0x55e2c9be3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:45:50.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.456+0000 7fcd73e00700 1 --2- 172.21.15.33:0/3498336381 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2c9bd85d0 0x55e2c9be3f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:50.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.456+0000 7fcd73e00700 1 -- 172.21.15.33:0/3498336381 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2c9bdc7b0 msgr2=0x55e2c9be12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:45:50.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.456+0000 7fcd73e00700 1 --2- 172.21.15.33:0/3498336381 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2c9bdc7b0 0x55e2c9be12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:50.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.456+0000 7fcd73e00700 1 -- 172.21.15.33:0/3498336381 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2c9af0420 con 0x55e2c9bdd0d0 2024-05-03T18:45:50.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.457+0000 7fcd73e00700 1 --2- 172.21.15.33:0/3498336381 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e2c9bdd0d0 0x55e2c9be1830 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c00b0a0 tx=0x7fcd6c003db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4787836b05a5fbe server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:45:50.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.457+0000 7fcd5d600700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-03T18:45:50.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.458+0000 7fcd5d600700 1 -- 172.21.15.33:0/3498336381 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd6c007a30 con 0x55e2c9bdd0d0 2024-05-03T18:45:50.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.458+0000 7fcd5d600700 1 -- 172.21.15.33:0/3498336381 <== mon.1 v2:172.21.15.150:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9518+0+0 (secure 0 0 0) 0x7fcd6c007bd0 con 0x55e2c9bdd0d0 2024-05-03T18:45:50.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.459+0000 7fcd84f25b80 1 -- 172.21.15.33:0/3498336381 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd6c009470 con 0x55e2c9bdd0d0 2024-05-03T18:45:50.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.459+0000 7fcd5d600700 1 -- 172.21.15.33:0/3498336381 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fcd6c007dd0 con 0x55e2c9bdd0d0 2024-05-03T18:45:50.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.461+0000 7fcd84f25b80 10 client.26848 fetch_fsmap learned FSMap version 21 2024-05-03T18:45:50.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.461+0000 7fcd84f25b80 1 -- 172.21.15.33:0/3498336381 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55e2c9be4c40 con 0x55e2c9bdd0d0 2024-05-03T18:45:50.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.461+0000 7fcd5d600700 1 -- 172.21.15.33:0/3498336381 <== mon.1 v2:172.21.15.150:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcd6c009470 con 0x55e2c9bdd0d0 2024-05-03T18:45:50.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.462+0000 7fcd5d600700 1 -- 172.21.15.33:0/3498336381 <== mon.1 v2:172.21.15.150:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fcd6c036020 con 0x55e2c9bdd0d0 2024-05-03T18:45:50.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.462+0000 7fcd84f25b80 10 client.26848 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:45:50.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.462+0000 7fcd84f25b80 10 client.26848 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:45:50.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.463+0000 7fcd84f25b80 20 client.26848 populate_metadata read hostname 'smithi033' 2024-05-03T18:45:50.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.463+0000 7fcd84f25b80 1 --2- 172.21.15.33:0/3498336381 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55e2c9c02810 0x55e2c9c04c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:50.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.463+0000 7fcd84f25b80 4 client.26848 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:45:50.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.463+0000 7fcd84f25b80 1 -- 172.21.15.33:0/3498336381 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e2c9ba51d0 con 0x55e2c9c02810 2024-05-03T18:45:50.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.464+0000 7fcd73400700 1 --2- 172.21.15.33:0/3498336381 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55e2c9c02810 0x55e2c9c04c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:50.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.465+0000 7fcd73400700 1 --2- 172.21.15.33:0/3498336381 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55e2c9c02810 0x55e2c9c04c00 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57cffd6c50cf2789 server_cookie=ab58899a5604931c in_seq=0 out_seq=0 2024-05-03T18:45:50.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.466+0000 7fcd5d600700 10 client.26848 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:45:50.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.467+0000 7fcd5d600700 1 -- 172.21.15.33:0/3498336381 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e2c9ba51d0 con 0x55e2c9c02810 2024-05-03T18:45:50.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.467+0000 7fcd5d600700 10 client.26848 handle_command_reply: tid=0 2024-05-03T18:45:50.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.467+0000 7fcd84f25b80 1 client.26848 shutdown 2024-05-03T18:45:50.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.468+0000 7fcd84f25b80 1 -- 172.21.15.33:0/3498336381 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e2c9bdd0d0 msgr2=0x55e2c9be1830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:50.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.468+0000 7fcd84f25b80 1 --2- 172.21.15.33:0/3498336381 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e2c9bdd0d0 0x55e2c9be1830 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c00b0a0 tx=0x7fcd6c003db0 comp rx=0 tx=0).stop 2024-05-03T18:45:50.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.468+0000 7fcd84f25b80 1 -- 172.21.15.33:0/3498336381 shutdown_connections 2024-05-03T18:45:50.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.468+0000 7fcd84f25b80 1 --2- 172.21.15.33:0/3498336381 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55e2c9c02810 0x55e2c9c04c00 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:50.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.468+0000 7fcd5d600700 0 client.26848 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:45:50.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.468+0000 7fcd84f25b80 1 --2- 172.21.15.33:0/3498336381 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e2c9bdd0d0 0x55e2c9be1830 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:50.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.468+0000 7fcd84f25b80 1 --2- 172.21.15.33:0/3498336381 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2c9bd85d0 0x55e2c9be3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:50.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.468+0000 7fcd84f25b80 1 --2- 172.21.15.33:0/3498336381 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2c9bdc7b0 0x55e2c9be12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:50.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.469+0000 7fcd84f25b80 1 -- 172.21.15.33:0/3498336381 >> 172.21.15.33:0/3498336381 conn(0x55e2c9b06b60 msgr2=0x55e2c9bdde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:45:50.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.469+0000 7fcd84f25b80 1 -- 172.21.15.33:0/3498336381 shutdown_connections 2024-05-03T18:45:50.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.469+0000 7fcd84f25b80 1 -- 172.21.15.33:0/3498336381 wait complete. 2024-05-03T18:45:50.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.470+0000 7fcd84f25b80 20 client.26848 trim_cache size 0 max 16384 2024-05-03T18:45:50.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.501+0000 7fcd84f25b80 1 Processor -- start 2024-05-03T18:45:50.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.501+0000 7fcd84f25b80 1 -- start start 2024-05-03T18:45:50.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.501+0000 7fcd84f25b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e2c9bd85d0 0x55e2c9bedd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:50.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.501+0000 7fcd84f25b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2c9bdc7b0 0x55e2c9c0a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:50.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.501+0000 7fcd84f25b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2c9bdd0d0 0x55e2c9c0a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:50.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.501+0000 7fcd84f25b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcd7406a190 con 0x55e2c9bdd0d0 2024-05-03T18:45:50.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.501+0000 7fcd84f25b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fcd741ada50 con 0x55e2c9bdc7b0 2024-05-03T18:45:50.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.501+0000 7fcd84f25b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fcd74110de0 con 0x55e2c9bd85d0 2024-05-03T18:45:50.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.502+0000 7fcd72a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2c9bdc7b0 0x55e2c9c0a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:50.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.503+0000 7fcd73400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e2c9bd85d0 0x55e2c9bedd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:50.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.503+0000 7fcd73400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e2c9bd85d0 0x55e2c9bedd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:39128/0 (socket says 172.21.15.33:39128) 2024-05-03T18:45:50.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.503+0000 7fcd73400700 1 -- 172.21.15.33:0/2196605762 learned_addr learned my addr 172.21.15.33:0/2196605762 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:45:50.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.504+0000 7fcd73e00700 1 --2- 172.21.15.33:0/2196605762 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2c9bdd0d0 0x55e2c9c0a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:50.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.504+0000 7fcd73400700 1 -- 172.21.15.33:0/2196605762 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2c9bdc7b0 msgr2=0x55e2c9c0a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:45:50.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.504+0000 7fcd73400700 1 --2- 172.21.15.33:0/2196605762 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2c9bdc7b0 0x55e2c9c0a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:50.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.505+0000 7fcd73400700 1 -- 172.21.15.33:0/2196605762 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2c9bdd0d0 msgr2=0x55e2c9c0a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:45:50.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.505+0000 7fcd73400700 1 --2- 172.21.15.33:0/2196605762 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2c9bdd0d0 0x55e2c9c0a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:50.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.505+0000 7fcd73400700 1 -- 172.21.15.33:0/2196605762 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2c9be4860 con 0x55e2c9bd85d0 2024-05-03T18:45:50.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.505+0000 7fcd72a00700 1 --2- 172.21.15.33:0/2196605762 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2c9bdc7b0 0x55e2c9c0a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:45:50.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.507+0000 7fcd73400700 1 --2- 172.21.15.33:0/2196605762 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e2c9bd85d0 0x55e2c9bedd20 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fcd6400b840 tx=0x7fcd64010000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=521f4a98c0393610 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:45:50.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.508+0000 7fcd5fe00700 1 -- 172.21.15.33:0/2196605762 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd64010b20 con 0x55e2c9bd85d0 2024-05-03T18:45:50.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.508+0000 7fcd5fe00700 1 -- 172.21.15.33:0/2196605762 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcd64016040 con 0x55e2c9bd85d0 2024-05-03T18:45:50.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.508+0000 7fcd5fe00700 1 -- 172.21.15.33:0/2196605762 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd64004be0 con 0x55e2c9bd85d0 2024-05-03T18:45:50.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.509+0000 7fcd84f25b80 1 -- 172.21.15.33:0/2196605762 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e2c9bd85d0 msgr2=0x55e2c9bedd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:50.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.509+0000 7fcd84f25b80 1 --2- 172.21.15.33:0/2196605762 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e2c9bd85d0 0x55e2c9bedd20 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fcd6400b840 tx=0x7fcd64010000 comp rx=0 tx=0).stop 2024-05-03T18:45:50.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.509+0000 7fcd84f25b80 1 -- 172.21.15.33:0/2196605762 shutdown_connections 2024-05-03T18:45:50.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.509+0000 7fcd84f25b80 1 --2- 172.21.15.33:0/2196605762 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2c9bdd0d0 0x55e2c9c0a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:50.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.509+0000 7fcd84f25b80 1 --2- 172.21.15.33:0/2196605762 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e2c9bd85d0 0x55e2c9bedd20 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:50.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.509+0000 7fcd84f25b80 1 --2- 172.21.15.33:0/2196605762 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2c9bdc7b0 0x55e2c9c0a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:50.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.509+0000 7fcd84f25b80 1 -- 172.21.15.33:0/2196605762 >> 172.21.15.33:0/2196605762 conn(0x55e2c9b06b60 msgr2=0x55e2c9bdb3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:45:50.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.510+0000 7fcd84f25b80 1 -- 172.21.15.33:0/2196605762 shutdown_connections 2024-05-03T18:45:50.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.510+0000 7fcd84f25b80 1 -- 172.21.15.33:0/2196605762 wait complete. 2024-05-03T18:45:50.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.510+0000 7fcd84f25b80 1 -- start start 2024-05-03T18:45:50.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.511+0000 7fcd84f25b80 1 Processor -- start 2024-05-03T18:45:50.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.511+0000 7fcd84f25b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2c9bd85d0 0x55e2c9be0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:50.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.511+0000 7fcd84f25b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e2c9bdc7b0 0x55e2c9be1180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:50.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.512+0000 7fcd84f25b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2c9bdd0d0 0x55e2c9be16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:50.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.512+0000 7fcd84f25b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcd74111490 con 0x55e2c9bd85d0 2024-05-03T18:45:50.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.512+0000 7fcd84f25b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fcd741ad380 con 0x55e2c9bdd0d0 2024-05-03T18:45:50.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.512+0000 7fcd84f25b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fcd74111b60 con 0x55e2c9bdc7b0 2024-05-03T18:45:50.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.512+0000 7fcd84f25b80 10 client.0 _ll_register_callbacks cb 0x55e2c9ad1c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:45:50.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.512+0000 7fcd73e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2c9bdd0d0 0x55e2c9be16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:50.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.512+0000 7fcd73e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2c9bdd0d0 0x55e2c9be16c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:34940/0 (socket says 172.21.15.33:34940) 2024-05-03T18:45:50.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.512+0000 7fcd73e00700 1 -- 172.21.15.33:0/2046556139 learned_addr learned my addr 172.21.15.33:0/2046556139 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:45:50.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.513+0000 7fcd73400700 1 --2- 172.21.15.33:0/2046556139 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2c9bd85d0 0x55e2c9be0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:50.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.513+0000 7fcd72a00700 1 --2- 172.21.15.33:0/2046556139 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e2c9bdc7b0 0x55e2c9be1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:50.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.514+0000 7fcd73e00700 1 -- 172.21.15.33:0/2046556139 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e2c9bdc7b0 msgr2=0x55e2c9be1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:45:50.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.514+0000 7fcd73e00700 1 --2- 172.21.15.33:0/2046556139 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e2c9bdc7b0 0x55e2c9be1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:50.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.514+0000 7fcd73e00700 1 -- 172.21.15.33:0/2046556139 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2c9bd85d0 msgr2=0x55e2c9be0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:45:50.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.514+0000 7fcd73e00700 1 --2- 172.21.15.33:0/2046556139 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2c9bd85d0 0x55e2c9be0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:50.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.514+0000 7fcd73e00700 1 -- 172.21.15.33:0/2046556139 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2c9be8870 con 0x55e2c9bdd0d0 2024-05-03T18:45:50.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.514+0000 7fcd73400700 1 --2- 172.21.15.33:0/2046556139 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2c9bd85d0 0x55e2c9be0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:45:50.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.515+0000 7fcd73e00700 1 --2- 172.21.15.33:0/2046556139 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2c9bdd0d0 0x55e2c9be16c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c024420 tx=0x7fcd6c0021d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67ef9d5228c6228f server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:45:50.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.515+0000 7fcd5d600700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-05-03T18:45:50.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.516+0000 7fcd5d600700 1 -- 172.21.15.33:0/2046556139 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd6c007dd0 con 0x55e2c9bdd0d0 2024-05-03T18:45:50.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.516+0000 7fcd5d600700 1 -- 172.21.15.33:0/2046556139 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9518+0+0 (secure 0 0 0) 0x7fcd6c007bd0 con 0x55e2c9bdd0d0 2024-05-03T18:45:50.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.517+0000 7fcd84f25b80 1 -- 172.21.15.33:0/2046556139 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd64016040 con 0x55e2c9bdd0d0 2024-05-03T18:45:50.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.517+0000 7fcd5d600700 1 -- 172.21.15.33:0/2046556139 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fcd6c011390 con 0x55e2c9bdd0d0 2024-05-03T18:45:50.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.518+0000 7fcd84f25b80 10 client.36782 fetch_fsmap learned FSMap version 21 2024-05-03T18:45:50.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.518+0000 7fcd84f25b80 1 -- 172.21.15.33:0/2046556139 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55e2c9c6d270 con 0x55e2c9bdd0d0 2024-05-03T18:45:50.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.518+0000 7fcd5d600700 1 -- 172.21.15.33:0/2046556139 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcd64016040 con 0x55e2c9bdd0d0 2024-05-03T18:45:50.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.519+0000 7fcd5d600700 1 -- 172.21.15.33:0/2046556139 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fcd6c036020 con 0x55e2c9bdd0d0 2024-05-03T18:45:50.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.519+0000 7fcd84f25b80 10 client.36782 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:45:50.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.519+0000 7fcd84f25b80 10 client.36782 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:45:50.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.519+0000 7fcd84f25b80 20 client.36782 populate_metadata read hostname 'smithi033' 2024-05-03T18:45:50.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.519+0000 7fcd84f25b80 1 --2- 172.21.15.33:0/2046556139 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55e2c9c6d710 0x55e2c9c6fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:50.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.519+0000 7fcd84f25b80 4 client.36782 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-03T18:45:50.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.519+0000 7fcd84f25b80 1 -- 172.21.15.33:0/2046556139 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e2c9b38400 con 0x55e2c9c6d710 2024-05-03T18:45:50.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.520+0000 7fcd73400700 1 --2- 172.21.15.33:0/2046556139 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55e2c9c6d710 0x55e2c9c6fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:50.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.522+0000 7fcd73400700 1 --2- 172.21.15.33:0/2046556139 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55e2c9c6d710 0x55e2c9c6fb30 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55a22ed2bed8e06e server_cookie=cc48ec81de1e459d in_seq=0 out_seq=0 2024-05-03T18:45:50.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.522+0000 7fcd5d600700 10 client.36782 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:45:50.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.522+0000 7fcd5d600700 1 -- 172.21.15.33:0/2046556139 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e2c9b38400 con 0x55e2c9c6d710 2024-05-03T18:45:50.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.522+0000 7fcd5d600700 10 client.36782 handle_command_reply: tid=0 2024-05-03T18:45:50.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.522+0000 7fcd84f25b80 1 client.36782 shutdown 2024-05-03T18:45:50.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.523+0000 7fcd84f25b80 1 -- 172.21.15.33:0/2046556139 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2c9bdd0d0 msgr2=0x55e2c9be16c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:50.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.523+0000 7fcd84f25b80 1 --2- 172.21.15.33:0/2046556139 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2c9bdd0d0 0x55e2c9be16c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c024420 tx=0x7fcd6c0021d0 comp rx=0 tx=0).stop 2024-05-03T18:45:50.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.523+0000 7fcd84f25b80 1 -- 172.21.15.33:0/2046556139 shutdown_connections 2024-05-03T18:45:50.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.523+0000 7fcd84f25b80 1 --2- 172.21.15.33:0/2046556139 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x55e2c9c6d710 0x55e2c9c6fb30 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:50.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.523+0000 7fcd84f25b80 1 --2- 172.21.15.33:0/2046556139 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2c9bdd0d0 0x55e2c9be16c0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:50.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.523+0000 7fcd84f25b80 1 --2- 172.21.15.33:0/2046556139 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2c9bd85d0 0x55e2c9be0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:50.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.523+0000 7fcd84f25b80 1 --2- 172.21.15.33:0/2046556139 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e2c9bdc7b0 0x55e2c9be1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:50.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.523+0000 7fcd84f25b80 1 -- 172.21.15.33:0/2046556139 >> 172.21.15.33:0/2046556139 conn(0x55e2c9b06b60 msgr2=0x55e2c9bd1650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:45:50.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.523+0000 7fcd5d600700 0 client.36782 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:45:50.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.524+0000 7fcd84f25b80 1 -- 172.21.15.33:0/2046556139 shutdown_connections 2024-05-03T18:45:50.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.524+0000 7fcd84f25b80 1 -- 172.21.15.33:0/2046556139 wait complete. 2024-05-03T18:45:50.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.524+0000 7fcd84f25b80 20 client.36782 trim_cache size 0 max 16384 2024-05-03T18:45:50.529 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-03T18:45:50.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.525+0000 7fcd7a600700 1 -- 172.21.15.33:0/2680870108 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fcd580b2a70 msgr2=0x7fcd580b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:50.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.525+0000 7fcd7a600700 1 --2- 172.21.15.33:0/2680870108 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fcd580b2a70 0x7fcd580b4f30 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fcd64005e90 tx=0x7fcd64005d00 comp rx=0 tx=0).stop 2024-05-03T18:45:50.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.525+0000 7fcd7a600700 1 -- 172.21.15.33:0/2680870108 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcd74105ee0 msgr2=0x7fcd741a05c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:50.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.525+0000 7fcd7a600700 1 --2- 172.21.15.33:0/2680870108 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcd74105ee0 0x7fcd741a05c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fcd6000d980 tx=0x7fcd6000dd40 comp rx=0 tx=0).stop 2024-05-03T18:45:50.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.527+0000 7fcd7a600700 1 -- 172.21.15.33:0/2680870108 shutdown_connections 2024-05-03T18:45:50.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.527+0000 7fcd7a600700 1 --2- 172.21.15.33:0/2680870108 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fcd580b2a70 0x7fcd580b4f30 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:50.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.527+0000 7fcd7a600700 1 --2- 172.21.15.33:0/2680870108 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd741083d0 0x7fcd741a4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:50.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.527+0000 7fcd7a600700 1 --2- 172.21.15.33:0/2680870108 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd7406b6f0 0x7fcd741a0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:50.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.527+0000 7fcd7a600700 1 --2- 172.21.15.33:0/2680870108 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcd74105ee0 0x7fcd741a05c0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:50.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.527+0000 7fcd7a600700 1 -- 172.21.15.33:0/2680870108 >> 172.21.15.33:0/2680870108 conn(0x7fcd74079350 msgr2=0x7fcd74104ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:45:50.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.527+0000 7fcd7a600700 1 -- 172.21.15.33:0/2680870108 shutdown_connections 2024-05-03T18:45:50.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:50.527+0000 7fcd7a600700 1 -- 172.21.15.33:0/2680870108 wait complete. 2024-05-03T18:45:50.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:45:50 smithi099 ceph-mon[31658]: pgmap v587: 129 pgs: 129 active+clean; 3.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 5.0 MiB/s wr, 441 op/s 2024-05-03T18:45:50.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:45:50 smithi150 ceph-mon[32492]: pgmap v587: 129 pgs: 129 active+clean; 3.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 5.0 MiB/s wr, 441 op/s 2024-05-03T18:45:52.486 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:45:52 smithi033 ceph-mon[28330]: pgmap v588: 129 pgs: 129 active+clean; 3.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 7.6 MiB/s wr, 686 op/s 2024-05-03T18:45:52.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:45:52 smithi099 ceph-mon[31658]: pgmap v588: 129 pgs: 129 active+clean; 3.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 7.6 MiB/s wr, 686 op/s 2024-05-03T18:45:52.632 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:45:52 smithi150 ceph-mon[32492]: pgmap v588: 129 pgs: 129 active+clean; 3.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 7.6 MiB/s wr, 686 op/s 2024-05-03T18:45:52.964 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-03T18:45:52.964 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-03T18:45:54.161 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:45:54.486 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:45:54 smithi033 ceph-mon[28330]: pgmap v589: 129 pgs: 129 active+clean; 3.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 5.4 MiB/s wr, 472 op/s 2024-05-03T18:45:54.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:45:54 smithi099 ceph-mon[31658]: pgmap v589: 129 pgs: 129 active+clean; 3.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 5.4 MiB/s wr, 472 op/s 2024-05-03T18:45:54.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:45:54 smithi150 ceph-mon[32492]: pgmap v589: 129 pgs: 129 active+clean; 3.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 5.4 MiB/s wr, 472 op/s 2024-05-03T18:45:55.111 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 880.0 s, 458.8 tps, lat 218.045 ms stddev 216.828 2024-05-03T18:45:56.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:45:56 smithi033 ceph-mon[28330]: pgmap v590: 129 pgs: 129 active+clean; 3.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 6.6 MiB/s wr, 586 op/s 2024-05-03T18:45:56.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:45:56 smithi099 ceph-mon[31658]: pgmap v590: 129 pgs: 129 active+clean; 3.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 6.6 MiB/s wr, 586 op/s 2024-05-03T18:45:56.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:45:56 smithi150 ceph-mon[32492]: pgmap v590: 129 pgs: 129 active+clean; 3.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 6.6 MiB/s wr, 586 op/s 2024-05-03T18:45:58.186 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:45:58 smithi033 ceph-mon[28330]: pgmap v591: 129 pgs: 129 active+clean; 3.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 7.1 MiB/s wr, 606 op/s 2024-05-03T18:45:58.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:45:58 smithi099 ceph-mon[31658]: pgmap v591: 129 pgs: 129 active+clean; 3.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 7.1 MiB/s wr, 606 op/s 2024-05-03T18:45:58.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:45:58 smithi150 ceph-mon[32492]: pgmap v591: 129 pgs: 129 active+clean; 3.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 7.1 MiB/s wr, 606 op/s 2024-05-03T18:45:59.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.063+0000 7fa883c00700 1 -- 172.21.15.33:0/3274078702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa87c083ff0 msgr2=0x7fa87c07bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:59.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.063+0000 7fa883c00700 1 --2- 172.21.15.33:0/3274078702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa87c083ff0 0x7fa87c07bde0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fa870009990 tx=0x7fa870009ca0 comp rx=0 tx=0).stop 2024-05-03T18:45:59.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.064+0000 7fa883c00700 1 -- 172.21.15.33:0/3274078702 shutdown_connections 2024-05-03T18:45:59.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.064+0000 7fa883c00700 1 --2- 172.21.15.33:0/3274078702 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa87c11ca00 0x7fa87c11edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:59.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.064+0000 7fa883c00700 1 --2- 172.21.15.33:0/3274078702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa87c083ff0 0x7fa87c07bde0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:59.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.064+0000 7fa883c00700 1 --2- 172.21.15.33:0/3274078702 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa87c07c320 0x7fa87c07c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:59.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.064+0000 7fa883c00700 1 -- 172.21.15.33:0/3274078702 >> 172.21.15.33:0/3274078702 conn(0x7fa87c076c70 msgr2=0x7fa87c1082c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:45:59.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.064+0000 7fa883c00700 1 -- 172.21.15.33:0/3274078702 shutdown_connections 2024-05-03T18:45:59.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.065+0000 7fa883c00700 1 -- 172.21.15.33:0/3274078702 wait complete. 2024-05-03T18:45:59.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.067+0000 7fa883c00700 1 Processor -- start 2024-05-03T18:45:59.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.067+0000 7fa883c00700 1 -- start start 2024-05-03T18:45:59.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.067+0000 7fa883c00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa87c07c320 0x7fa87c1ae850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:59.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.068+0000 7fa883c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa87c11ca00 0x7fa87c1aed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:59.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.068+0000 7fa883c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa87c1a8a10 0x7fa87c1a8e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:59.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.068+0000 7fa883c00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa87c122460 con 0x7fa87c07c320 2024-05-03T18:45:59.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.068+0000 7fa883c00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa87c1222e0 con 0x7fa87c1a8a10 2024-05-03T18:45:59.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.068+0000 7fa883c00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa87c1225e0 con 0x7fa87c11ca00 2024-05-03T18:45:59.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.068+0000 7fa87be00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa87c11ca00 0x7fa87c1aed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:59.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.068+0000 7fa881200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa87c07c320 0x7fa87c1ae850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:59.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.068+0000 7fa87be00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa87c11ca00 0x7fa87c1aed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:57778/0 (socket says 172.21.15.33:57778) 2024-05-03T18:45:59.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.068+0000 7fa87be00700 1 -- 172.21.15.33:0/3208737629 learned_addr learned my addr 172.21.15.33:0/3208737629 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:45:59.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.068+0000 7fa881c00700 1 --2- 172.21.15.33:0/3208737629 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa87c1a8a10 0x7fa87c1a8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:59.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.069+0000 7fa881c00700 1 -- 172.21.15.33:0/3208737629 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa87c11ca00 msgr2=0x7fa87c1aed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:59.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.069+0000 7fa881c00700 1 --2- 172.21.15.33:0/3208737629 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa87c11ca00 0x7fa87c1aed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:59.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.069+0000 7fa881c00700 1 -- 172.21.15.33:0/3208737629 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa87c07c320 msgr2=0x7fa87c1ae850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:59.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.069+0000 7fa881c00700 1 --2- 172.21.15.33:0/3208737629 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa87c07c320 0x7fa87c1ae850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:59.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.069+0000 7fa881c00700 1 -- 172.21.15.33:0/3208737629 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa87c1a9590 con 0x7fa87c1a8a10 2024-05-03T18:45:59.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.069+0000 7fa881c00700 1 --2- 172.21.15.33:0/3208737629 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa87c1a8a10 0x7fa87c1a8e90 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa874007f10 tx=0x7fa87400d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:45:59.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.070+0000 7fa879600700 1 -- 172.21.15.33:0/3208737629 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa87400ddd0 con 0x7fa87c1a8a10 2024-05-03T18:45:59.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.070+0000 7fa879600700 1 -- 172.21.15.33:0/3208737629 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa874009e20 con 0x7fa87c1a8a10 2024-05-03T18:45:59.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.070+0000 7fa879600700 1 -- 172.21.15.33:0/3208737629 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa874015400 con 0x7fa87c1a8a10 2024-05-03T18:45:59.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.070+0000 7fa883c00700 1 -- 172.21.15.33:0/3208737629 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa87c1b5720 con 0x7fa87c1a8a10 2024-05-03T18:45:59.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.071+0000 7fa883c00700 1 -- 172.21.15.33:0/3208737629 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa87c1b5ac0 con 0x7fa87c1a8a10 2024-05-03T18:45:59.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.077+0000 7fa879600700 1 -- 172.21.15.33:0/3208737629 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fa874005020 con 0x7fa87c1a8a10 2024-05-03T18:45:59.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.079+0000 7fa879600700 1 --2- 172.21.15.33:0/3208737629 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fa8640b2c00 0x7fa8640b50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:59.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.079+0000 7fa879600700 1 -- 172.21.15.33:0/3208737629 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 9518+0+0 (secure 0 0 0) 0x7fa8740e2390 con 0x7fa87c1a8a10 2024-05-03T18:45:59.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.080+0000 7fa881200700 1 --2- 172.21.15.33:0/3208737629 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fa8640b2c00 0x7fa8640b50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:59.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.081+0000 7fa881200700 1 --2- 172.21.15.33:0/3208737629 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fa8640b2c00 0x7fa8640b50c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa870006010 tx=0x7fa87001a040 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:45:59.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.118+0000 7fa88e556b80 1 Processor -- start 2024-05-03T18:45:59.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.118+0000 7fa88e556b80 1 -- start start 2024-05-03T18:45:59.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.119+0000 7fa88e556b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561ba22729b0 0x561ba2272d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:59.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.119+0000 7fa88e556b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ba22732d0 0x561ba226cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:59.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.119+0000 7fa88e556b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561ba226d5d0 0x561ba226da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:59.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.119+0000 7fa88e556b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x561ba21bfaa0 con 0x561ba22729b0 2024-05-03T18:45:59.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.119+0000 7fa88e556b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x561ba215d7e0 con 0x561ba22732d0 2024-05-03T18:45:59.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.119+0000 7fa88e556b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x561ba21bba60 con 0x561ba226d5d0 2024-05-03T18:45:59.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.120+0000 7fa881200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561ba22729b0 0x561ba2272d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:59.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.121+0000 7fa881200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561ba22729b0 0x561ba2272d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57458/0 (socket says 172.21.15.33:57458) 2024-05-03T18:45:59.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.121+0000 7fa881200700 1 -- 172.21.15.33:0/2582927030 learned_addr learned my addr 172.21.15.33:0/2582927030 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:45:59.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.121+0000 7fa881c00700 1 --2- 172.21.15.33:0/2582927030 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561ba226d5d0 0x561ba226da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:59.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.122+0000 7fa87be00700 1 --2- 172.21.15.33:0/2582927030 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ba22732d0 0x561ba226cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:59.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.122+0000 7fa881c00700 1 -- 172.21.15.33:0/2582927030 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ba22732d0 msgr2=0x561ba226cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:45:59.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.122+0000 7fa881c00700 1 --2- 172.21.15.33:0/2582927030 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ba22732d0 0x561ba226cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:59.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.122+0000 7fa881c00700 1 -- 172.21.15.33:0/2582927030 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561ba22729b0 msgr2=0x561ba2272d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:45:59.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.122+0000 7fa881c00700 1 --2- 172.21.15.33:0/2582927030 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561ba22729b0 0x561ba2272d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:59.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.122+0000 7fa881c00700 1 -- 172.21.15.33:0/2582927030 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ba210c950 con 0x561ba226d5d0 2024-05-03T18:45:59.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.122+0000 7fa881200700 1 --2- 172.21.15.33:0/2582927030 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561ba22729b0 0x561ba2272d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:45:59.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.123+0000 7fa881c00700 1 --2- 172.21.15.33:0/2582927030 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561ba226d5d0 0x561ba226da50 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fa8740df6a0 tx=0x7fa8740a9930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c667505fb801761b server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:45:59.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.124+0000 7fa86e000700 1 -- 172.21.15.33:0/2582927030 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8740e2150 con 0x561ba226d5d0 2024-05-03T18:45:59.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.124+0000 7fa86e000700 1 -- 172.21.15.33:0/2582927030 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa8740b2090 con 0x561ba226d5d0 2024-05-03T18:45:59.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.124+0000 7fa86e000700 1 -- 172.21.15.33:0/2582927030 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8740ad170 con 0x561ba226d5d0 2024-05-03T18:45:59.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.128+0000 7fa88e556b80 1 -- 172.21.15.33:0/2582927030 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561ba226d5d0 msgr2=0x561ba226da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:59.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.128+0000 7fa88e556b80 1 --2- 172.21.15.33:0/2582927030 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561ba226d5d0 0x561ba226da50 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fa8740df6a0 tx=0x7fa8740a9930 comp rx=0 tx=0).stop 2024-05-03T18:45:59.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.128+0000 7fa88e556b80 1 -- 172.21.15.33:0/2582927030 shutdown_connections 2024-05-03T18:45:59.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.128+0000 7fa88e556b80 1 --2- 172.21.15.33:0/2582927030 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561ba226d5d0 0x561ba226da50 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:59.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.128+0000 7fa88e556b80 1 --2- 172.21.15.33:0/2582927030 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561ba22729b0 0x561ba2272d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:59.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.128+0000 7fa88e556b80 1 --2- 172.21.15.33:0/2582927030 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ba22732d0 0x561ba226cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:59.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.128+0000 7fa88e556b80 1 -- 172.21.15.33:0/2582927030 >> 172.21.15.33:0/2582927030 conn(0x561ba21e5750 msgr2=0x561ba2267980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:45:59.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.129+0000 7fa88e556b80 1 -- 172.21.15.33:0/2582927030 shutdown_connections 2024-05-03T18:45:59.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.129+0000 7fa88e556b80 1 -- 172.21.15.33:0/2582927030 wait complete. 2024-05-03T18:45:59.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.130+0000 7fa88e556b80 1 -- start start 2024-05-03T18:45:59.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.131+0000 7fa88e556b80 1 Processor -- start 2024-05-03T18:45:59.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.132+0000 7fa88e556b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561ba226d5d0 0x561ba2278f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:59.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.133+0000 7fa88e556b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ba22729b0 0x561ba2276310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:59.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.133+0000 7fa88e556b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561ba22732d0 0x561ba2276850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:59.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.134+0000 7fa88e556b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x561ba228ad30 con 0x561ba22732d0 2024-05-03T18:45:59.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.134+0000 7fa88e556b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x561ba2125ba0 con 0x561ba22729b0 2024-05-03T18:45:59.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.134+0000 7fa88e556b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x561ba228aeb0 con 0x561ba226d5d0 2024-05-03T18:45:59.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.134+0000 7fa88e556b80 10 client.0 _ll_register_callbacks cb 0x561ba2166cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:45:59.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.135+0000 7fa881c00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561ba22732d0 0x561ba2276850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:59.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.135+0000 7fa881c00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561ba22732d0 0x561ba2276850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57468/0 (socket says 172.21.15.33:57468) 2024-05-03T18:45:59.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.135+0000 7fa881200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561ba226d5d0 0x561ba2278f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:59.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.135+0000 7fa881c00700 1 -- 172.21.15.33:0/377481080 learned_addr learned my addr 172.21.15.33:0/377481080 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:45:59.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.137+0000 7fa87be00700 1 --2- 172.21.15.33:0/377481080 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ba22729b0 0x561ba2276310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:59.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.137+0000 7fa881200700 1 -- 172.21.15.33:0/377481080 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ba22729b0 msgr2=0x561ba2276310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:45:59.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.137+0000 7fa881200700 1 --2- 172.21.15.33:0/377481080 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ba22729b0 0x561ba2276310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:59.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.137+0000 7fa881200700 1 -- 172.21.15.33:0/377481080 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561ba22732d0 msgr2=0x561ba2276850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:45:59.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.137+0000 7fa881200700 1 --2- 172.21.15.33:0/377481080 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561ba22732d0 0x561ba2276850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:59.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.137+0000 7fa881200700 1 -- 172.21.15.33:0/377481080 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ba21854a0 con 0x561ba226d5d0 2024-05-03T18:45:59.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.137+0000 7fa881c00700 1 --2- 172.21.15.33:0/377481080 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561ba22732d0 0x561ba2276850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:45:59.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.138+0000 7fa881200700 1 --2- 172.21.15.33:0/377481080 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561ba226d5d0 0x561ba2278f70 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fa870000c00 tx=0x7fa870009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c46ec48d769c9362 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:45:59.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.138+0000 7fa862a00700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-03T18:45:59.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.139+0000 7fa862a00700 1 -- 172.21.15.33:0/377481080 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa87000fe60 con 0x561ba226d5d0 2024-05-03T18:45:59.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.139+0000 7fa862a00700 1 -- 172.21.15.33:0/377481080 <== mon.1 v2:172.21.15.150:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9518+0+0 (secure 0 0 0) 0x7fa870023610 con 0x561ba226d5d0 2024-05-03T18:45:59.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.139+0000 7fa88e556b80 1 -- 172.21.15.33:0/377481080 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8740b2090 con 0x561ba226d5d0 2024-05-03T18:45:59.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.139+0000 7fa862a00700 1 -- 172.21.15.33:0/377481080 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fa87003b050 con 0x561ba226d5d0 2024-05-03T18:45:59.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.140+0000 7fa88e556b80 10 client.26863 fetch_fsmap learned FSMap version 21 2024-05-03T18:45:59.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.140+0000 7fa88e556b80 1 -- 172.21.15.33:0/377481080 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x561ba22954f0 con 0x561ba226d5d0 2024-05-03T18:45:59.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.140+0000 7fa862a00700 1 -- 172.21.15.33:0/377481080 <== mon.1 v2:172.21.15.150:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa8740b2090 con 0x561ba226d5d0 2024-05-03T18:45:59.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.140+0000 7fa862a00700 1 -- 172.21.15.33:0/377481080 <== mon.1 v2:172.21.15.150:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fa87002d7e0 con 0x561ba226d5d0 2024-05-03T18:45:59.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.141+0000 7fa88e556b80 10 client.26863 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:45:59.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.141+0000 7fa88e556b80 10 client.26863 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:45:59.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.141+0000 7fa88e556b80 20 client.26863 populate_metadata read hostname 'smithi033' 2024-05-03T18:45:59.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.141+0000 7fa88e556b80 1 --2- 172.21.15.33:0/377481080 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x561ba2297dc0 0x561ba229a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:59.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.141+0000 7fa88e556b80 4 client.26863 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:45:59.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.141+0000 7fa88e556b80 1 -- 172.21.15.33:0/377481080 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561ba223a070 con 0x561ba2297dc0 2024-05-03T18:45:59.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.141+0000 7fa87be00700 1 --2- 172.21.15.33:0/377481080 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x561ba2297dc0 0x561ba229a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:59.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.144+0000 7fa87be00700 1 --2- 172.21.15.33:0/377481080 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x561ba2297dc0 0x561ba229a220 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b7e5cf95c1b4cf2 server_cookie=b79807ee5e041ff4 in_seq=0 out_seq=0 2024-05-03T18:45:59.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.144+0000 7fa862a00700 10 client.26863 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:45:59.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.145+0000 7fa862a00700 1 -- 172.21.15.33:0/377481080 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561ba223a070 con 0x561ba2297dc0 2024-05-03T18:45:59.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.145+0000 7fa862a00700 10 client.26863 handle_command_reply: tid=0 2024-05-03T18:45:59.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.145+0000 7fa88e556b80 1 client.26863 shutdown 2024-05-03T18:45:59.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.146+0000 7fa88e556b80 1 -- 172.21.15.33:0/377481080 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561ba226d5d0 msgr2=0x561ba2278f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:59.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.146+0000 7fa88e556b80 1 --2- 172.21.15.33:0/377481080 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561ba226d5d0 0x561ba2278f70 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fa870000c00 tx=0x7fa870009ca0 comp rx=0 tx=0).stop 2024-05-03T18:45:59.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.146+0000 7fa88e556b80 1 -- 172.21.15.33:0/377481080 shutdown_connections 2024-05-03T18:45:59.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.146+0000 7fa88e556b80 1 --2- 172.21.15.33:0/377481080 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x561ba2297dc0 0x561ba229a220 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:59.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.146+0000 7fa862a00700 0 client.26863 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:45:59.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.146+0000 7fa88e556b80 1 --2- 172.21.15.33:0/377481080 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561ba22732d0 0x561ba2276850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:59.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.146+0000 7fa88e556b80 1 --2- 172.21.15.33:0/377481080 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561ba226d5d0 0x561ba2278f70 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:59.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.146+0000 7fa88e556b80 1 --2- 172.21.15.33:0/377481080 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ba22729b0 0x561ba2276310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:59.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.146+0000 7fa88e556b80 1 -- 172.21.15.33:0/377481080 >> 172.21.15.33:0/377481080 conn(0x561ba21e5750 msgr2=0x561ba2262430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:45:59.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.147+0000 7fa88e556b80 1 -- 172.21.15.33:0/377481080 shutdown_connections 2024-05-03T18:45:59.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.147+0000 7fa88e556b80 1 -- 172.21.15.33:0/377481080 wait complete. 2024-05-03T18:45:59.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.147+0000 7fa88e556b80 20 client.26863 trim_cache size 0 max 16384 2024-05-03T18:45:59.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.195+0000 7fa88e556b80 1 Processor -- start 2024-05-03T18:45:59.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.195+0000 7fa88e556b80 1 -- start start 2024-05-03T18:45:59.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.195+0000 7fa88e556b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ba226d5d0 0x561ba226cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:59.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.196+0000 7fa88e556b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561ba22729b0 0x561ba2293dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:59.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.196+0000 7fa88e556b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561ba22732d0 0x561ba2273d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:59.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.196+0000 7fa88e556b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa87c073ca0 con 0x561ba22729b0 2024-05-03T18:45:59.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.196+0000 7fa88e556b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa87c1b6b90 con 0x561ba226d5d0 2024-05-03T18:45:59.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.196+0000 7fa88e556b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa87c0742f0 con 0x561ba22732d0 2024-05-03T18:45:59.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.196+0000 7fa881200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ba226d5d0 0x561ba226cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:59.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.196+0000 7fa881200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ba226d5d0 0x561ba226cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:51082/0 (socket says 172.21.15.33:51082) 2024-05-03T18:45:59.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.196+0000 7fa881200700 1 -- 172.21.15.33:0/1116424989 learned_addr learned my addr 172.21.15.33:0/1116424989 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:45:59.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.197+0000 7fa881200700 1 -- 172.21.15.33:0/1116424989 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561ba22732d0 msgr2=0x561ba2273d20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-03T18:45:59.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.197+0000 7fa881200700 1 --2- 172.21.15.33:0/1116424989 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561ba22732d0 0x561ba2273d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:59.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.197+0000 7fa881200700 1 -- 172.21.15.33:0/1116424989 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561ba22729b0 msgr2=0x561ba2293dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:45:59.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.197+0000 7fa881200700 1 --2- 172.21.15.33:0/1116424989 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561ba22729b0 0x561ba2293dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:59.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.197+0000 7fa881200700 1 -- 172.21.15.33:0/1116424989 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ba22730d0 con 0x561ba226d5d0 2024-05-03T18:45:59.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.197+0000 7fa881200700 1 --2- 172.21.15.33:0/1116424989 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ba226d5d0 0x561ba226cfb0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fa87002dbe0 tx=0x7fa870005d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96601360a1b2612f server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:45:59.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.201+0000 7fa86d600700 1 -- 172.21.15.33:0/1116424989 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa87000fe60 con 0x561ba226d5d0 2024-05-03T18:45:59.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.202+0000 7fa86d600700 1 -- 172.21.15.33:0/1116424989 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa870044070 con 0x561ba226d5d0 2024-05-03T18:45:59.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.202+0000 7fa86d600700 1 -- 172.21.15.33:0/1116424989 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa870017be0 con 0x561ba226d5d0 2024-05-03T18:45:59.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.203+0000 7fa88e556b80 1 -- 172.21.15.33:0/1116424989 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ba226d5d0 msgr2=0x561ba226cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:59.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.203+0000 7fa88e556b80 1 --2- 172.21.15.33:0/1116424989 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ba226d5d0 0x561ba226cfb0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fa87002dbe0 tx=0x7fa870005d30 comp rx=0 tx=0).stop 2024-05-03T18:45:59.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.203+0000 7fa88e556b80 1 -- 172.21.15.33:0/1116424989 shutdown_connections 2024-05-03T18:45:59.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.203+0000 7fa88e556b80 1 --2- 172.21.15.33:0/1116424989 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561ba22732d0 0x561ba2273d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:59.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.203+0000 7fa88e556b80 1 --2- 172.21.15.33:0/1116424989 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ba226d5d0 0x561ba226cfb0 secure :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fa87002dbe0 tx=0x7fa870005d30 comp rx=0 tx=0).stop 2024-05-03T18:45:59.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.203+0000 7fa88e556b80 1 --2- 172.21.15.33:0/1116424989 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561ba22729b0 0x561ba2293dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:59.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.203+0000 7fa88e556b80 1 -- 172.21.15.33:0/1116424989 >> 172.21.15.33:0/1116424989 conn(0x561ba21e5750 msgr2=0x561ba2262430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:45:59.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.204+0000 7fa88e556b80 1 -- 172.21.15.33:0/1116424989 shutdown_connections 2024-05-03T18:45:59.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.204+0000 7fa88e556b80 1 -- 172.21.15.33:0/1116424989 wait complete. 2024-05-03T18:45:59.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.204+0000 7fa88e556b80 1 -- start start 2024-05-03T18:45:59.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.205+0000 7fa88e556b80 1 Processor -- start 2024-05-03T18:45:59.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.205+0000 7fa88e556b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561ba22729b0 0x561ba228ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:59.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.205+0000 7fa88e556b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ba22732d0 0x561ba22904a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:59.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.205+0000 7fa88e556b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561ba228d4f0 0x561ba228d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:59.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.205+0000 7fa88e556b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa87c0749c0 con 0x561ba22729b0 2024-05-03T18:45:59.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.205+0000 7fa88e556b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa87c1b64c0 con 0x561ba22732d0 2024-05-03T18:45:59.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.205+0000 7fa88e556b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa87c075090 con 0x561ba228d4f0 2024-05-03T18:45:59.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.205+0000 7fa88e556b80 10 client.0 _ll_register_callbacks cb 0x561ba2166cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:45:59.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.206+0000 7fa881200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561ba22729b0 0x561ba228ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:59.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.206+0000 7fa881200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561ba22729b0 0x561ba228ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57488/0 (socket says 172.21.15.33:57488) 2024-05-03T18:45:59.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.206+0000 7fa881200700 1 -- 172.21.15.33:0/3669405505 learned_addr learned my addr 172.21.15.33:0/3669405505 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:45:59.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.206+0000 7fa87be00700 1 --2- 172.21.15.33:0/3669405505 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ba22732d0 0x561ba22904a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:59.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.207+0000 7fa881c00700 1 --2- 172.21.15.33:0/3669405505 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561ba228d4f0 0x561ba228d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:59.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.207+0000 7fa881c00700 1 -- 172.21.15.33:0/3669405505 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ba22732d0 msgr2=0x561ba22904a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:45:59.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.207+0000 7fa881c00700 1 --2- 172.21.15.33:0/3669405505 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ba22732d0 0x561ba22904a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:59.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.207+0000 7fa881c00700 1 -- 172.21.15.33:0/3669405505 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561ba22729b0 msgr2=0x561ba228ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:45:59.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.207+0000 7fa881c00700 1 --2- 172.21.15.33:0/3669405505 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561ba22729b0 0x561ba228ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:59.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.207+0000 7fa881c00700 1 -- 172.21.15.33:0/3669405505 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ba228e020 con 0x561ba228d4f0 2024-05-03T18:45:59.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.208+0000 7fa881c00700 1 --2- 172.21.15.33:0/3669405505 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561ba228d4f0 0x561ba228d970 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fa8740df6a0 tx=0x7fa8740ad700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fdf8592eff70ab3b server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:45:59.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.208+0000 7fa862a00700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-03T18:45:59.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.209+0000 7fa862a00700 1 -- 172.21.15.33:0/3669405505 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8740c8060 con 0x561ba228d4f0 2024-05-03T18:45:59.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.209+0000 7fa862a00700 1 -- 172.21.15.33:0/3669405505 <== mon.1 v2:172.21.15.150:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9518+0+0 (secure 0 0 0) 0x7fa8740ad170 con 0x561ba228d4f0 2024-05-03T18:45:59.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.209+0000 7fa88e556b80 1 -- 172.21.15.33:0/3669405505 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa870044070 con 0x561ba228d4f0 2024-05-03T18:45:59.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.210+0000 7fa862a00700 1 -- 172.21.15.33:0/3669405505 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fa8740b2090 con 0x561ba228d4f0 2024-05-03T18:45:59.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.210+0000 7fa88e556b80 10 client.26866 fetch_fsmap learned FSMap version 21 2024-05-03T18:45:59.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.210+0000 7fa88e556b80 1 -- 172.21.15.33:0/3669405505 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x561ba2304500 con 0x561ba228d4f0 2024-05-03T18:45:59.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.210+0000 7fa862a00700 1 -- 172.21.15.33:0/3669405505 <== mon.1 v2:172.21.15.150:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa870044070 con 0x561ba228d4f0 2024-05-03T18:45:59.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.211+0000 7fa862a00700 1 -- 172.21.15.33:0/3669405505 <== mon.1 v2:172.21.15.150:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fa8740b1c00 con 0x561ba228d4f0 2024-05-03T18:45:59.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.211+0000 7fa88e556b80 10 client.26866 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:45:59.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.211+0000 7fa88e556b80 10 client.26866 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:45:59.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.211+0000 7fa88e556b80 20 client.26866 populate_metadata read hostname 'smithi033' 2024-05-03T18:45:59.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.211+0000 7fa88e556b80 1 --2- 172.21.15.33:0/3669405505 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x561ba23047e0 0x561ba2306bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:59.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.211+0000 7fa88e556b80 4 client.26866 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-03T18:45:59.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.211+0000 7fa88e556b80 1 -- 172.21.15.33:0/3669405505 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561ba21cd480 con 0x561ba23047e0 2024-05-03T18:45:59.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.213+0000 7fa881200700 1 --2- 172.21.15.33:0/3669405505 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x561ba23047e0 0x561ba2306bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:59.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.213+0000 7fa862a00700 10 client.26866 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:45:59.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.214+0000 7fa881200700 1 --2- 172.21.15.33:0/3669405505 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x561ba23047e0 0x561ba2306bd0 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=929376b6d46257cb server_cookie=ee48a3c50cd0ec80 in_seq=0 out_seq=0 2024-05-03T18:45:59.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.216+0000 7fa862a00700 1 -- 172.21.15.33:0/3669405505 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561ba21cd480 con 0x561ba23047e0 2024-05-03T18:45:59.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.216+0000 7fa862a00700 10 client.26866 handle_command_reply: tid=0 2024-05-03T18:45:59.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.216+0000 7fa88e556b80 1 client.26866 shutdown 2024-05-03T18:45:59.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.217+0000 7fa88e556b80 1 -- 172.21.15.33:0/3669405505 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561ba228d4f0 msgr2=0x561ba228d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:59.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.217+0000 7fa88e556b80 1 --2- 172.21.15.33:0/3669405505 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561ba228d4f0 0x561ba228d970 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fa8740df6a0 tx=0x7fa8740ad700 comp rx=0 tx=0).stop 2024-05-03T18:45:59.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.217+0000 7fa88e556b80 1 -- 172.21.15.33:0/3669405505 shutdown_connections 2024-05-03T18:45:59.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.217+0000 7fa88e556b80 1 --2- 172.21.15.33:0/3669405505 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x561ba23047e0 0x561ba2306bd0 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:59.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.217+0000 7fa88e556b80 1 --2- 172.21.15.33:0/3669405505 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x561ba228d4f0 0x561ba228d970 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:59.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.217+0000 7fa88e556b80 1 --2- 172.21.15.33:0/3669405505 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561ba22729b0 0x561ba228ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:59.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.217+0000 7fa88e556b80 1 --2- 172.21.15.33:0/3669405505 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ba22732d0 0x561ba22904a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:59.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.217+0000 7fa88e556b80 1 -- 172.21.15.33:0/3669405505 >> 172.21.15.33:0/3669405505 conn(0x561ba21e5750 msgr2=0x561ba2262430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:45:59.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.217+0000 7fa862a00700 0 client.26866 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:45:59.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.218+0000 7fa88e556b80 1 -- 172.21.15.33:0/3669405505 shutdown_connections 2024-05-03T18:45:59.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.218+0000 7fa88e556b80 1 -- 172.21.15.33:0/3669405505 wait complete. 2024-05-03T18:45:59.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.218+0000 7fa88e556b80 20 client.26866 trim_cache size 0 max 16384 2024-05-03T18:45:59.221 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-03T18:45:59.221 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-05-03T18:45:59.222 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "04c60cf9-800e-4d4a-bb17-e5d9d04a3f98", 2024-05-03T18:45:59.222 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-05-03T18:45:59.222 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-03T18:45:59.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.220+0000 7fa883c00700 1 -- 172.21.15.33:0/3208737629 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fa8640b2c00 msgr2=0x7fa8640b50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:59.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.220+0000 7fa883c00700 1 --2- 172.21.15.33:0/3208737629 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fa8640b2c00 0x7fa8640b50c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa870006010 tx=0x7fa87001a040 comp rx=0 tx=0).stop 2024-05-03T18:45:59.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.220+0000 7fa883c00700 1 -- 172.21.15.33:0/3208737629 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa87c1a8a10 msgr2=0x7fa87c1a8e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:59.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.220+0000 7fa883c00700 1 --2- 172.21.15.33:0/3208737629 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa87c1a8a10 0x7fa87c1a8e90 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa874007f10 tx=0x7fa87400d3b0 comp rx=0 tx=0).stop 2024-05-03T18:45:59.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.220+0000 7fa883c00700 1 -- 172.21.15.33:0/3208737629 shutdown_connections 2024-05-03T18:45:59.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.220+0000 7fa883c00700 1 --2- 172.21.15.33:0/3208737629 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fa8640b2c00 0x7fa8640b50c0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:59.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.220+0000 7fa883c00700 1 --2- 172.21.15.33:0/3208737629 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa87c1a8a10 0x7fa87c1a8e90 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:59.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.220+0000 7fa883c00700 1 --2- 172.21.15.33:0/3208737629 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa87c07c320 0x7fa87c1ae850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:59.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.220+0000 7fa883c00700 1 --2- 172.21.15.33:0/3208737629 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa87c11ca00 0x7fa87c1aed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:59.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.220+0000 7fa883c00700 1 -- 172.21.15.33:0/3208737629 >> 172.21.15.33:0/3208737629 conn(0x7fa87c076c70 msgr2=0x7fa87c077f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:45:59.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.222+0000 7fa883c00700 1 -- 172.21.15.33:0/3208737629 shutdown_connections 2024-05-03T18:45:59.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:45:59.222+0000 7fa883c00700 1 -- 172.21.15.33:0/3208737629 wait complete. 2024-05-03T18:46:00.111 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 885.0 s, 513.6 tps, lat 191.723 ms stddev 134.715 2024-05-03T18:46:00.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:46:00 smithi033 ceph-mon[28330]: pgmap v592: 129 pgs: 129 active+clean; 3.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 5.4 MiB/s wr, 488 op/s 2024-05-03T18:46:00.486 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:46:00 smithi033 ceph-mon[28330]: scrub queued for path: / 2024-05-03T18:46:00.486 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:46:00 smithi033 ceph-mon[28330]: scrub summary: idle+waiting paths [/] 2024-05-03T18:46:00.486 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:46:00 smithi033 ceph-mon[28330]: scrub summary: active paths [/] 2024-05-03T18:46:00.548 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:46:00 smithi099 ceph-mon[31658]: pgmap v592: 129 pgs: 129 active+clean; 3.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 5.4 MiB/s wr, 488 op/s 2024-05-03T18:46:00.548 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:46:00 smithi099 ceph-mon[31658]: scrub queued for path: / 2024-05-03T18:46:00.548 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:46:00 smithi099 ceph-mon[31658]: scrub summary: idle+waiting paths [/] 2024-05-03T18:46:00.549 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:46:00 smithi099 ceph-mon[31658]: scrub summary: active paths [/] 2024-05-03T18:46:00.634 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:46:00 smithi150 ceph-mon[32492]: pgmap v592: 129 pgs: 129 active+clean; 3.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 5.4 MiB/s wr, 488 op/s 2024-05-03T18:46:00.634 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:46:00 smithi150 ceph-mon[32492]: scrub queued for path: / 2024-05-03T18:46:00.634 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:46:00 smithi150 ceph-mon[32492]: scrub summary: idle+waiting paths [/] 2024-05-03T18:46:00.634 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:46:00 smithi150 ceph-mon[32492]: scrub summary: active paths [/] 2024-05-03T18:46:01.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:46:01 smithi033 ceph-mon[28330]: scrub summary: idle+waiting paths [/] 2024-05-03T18:46:01.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:46:01 smithi099 ceph-mon[31658]: scrub summary: idle+waiting paths [/] 2024-05-03T18:46:01.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:46:01 smithi150 ceph-mon[32492]: scrub summary: idle+waiting paths [/] 2024-05-03T18:46:01.813 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-03T18:46:02.486 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:46:02 smithi033 ceph-mon[28330]: pgmap v593: 129 pgs: 129 active+clean; 3.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 7.4 MiB/s wr, 700 op/s 2024-05-03T18:46:02.487 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:46:02 smithi033 ceph-mon[28330]: scrub summary: idle 2024-05-03T18:46:02.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:46:02 smithi099 ceph-mon[31658]: pgmap v593: 129 pgs: 129 active+clean; 3.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 7.4 MiB/s wr, 700 op/s 2024-05-03T18:46:02.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:46:02 smithi099 ceph-mon[31658]: scrub summary: idle 2024-05-03T18:46:02.562 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:46:02.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:46:02 smithi150 ceph-mon[32492]: pgmap v593: 129 pgs: 129 active+clean; 3.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 7.4 MiB/s wr, 700 op/s 2024-05-03T18:46:02.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:46:02 smithi150 ceph-mon[32492]: scrub summary: idle 2024-05-03T18:46:04.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:46:04 smithi033 ceph-mon[28330]: pgmap v594: 129 pgs: 129 active+clean; 3.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 4.8 MiB/s wr, 453 op/s 2024-05-03T18:46:04.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:46:04 smithi099 ceph-mon[31658]: pgmap v594: 129 pgs: 129 active+clean; 3.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 4.8 MiB/s wr, 453 op/s 2024-05-03T18:46:04.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:46:04 smithi150 ceph-mon[32492]: pgmap v594: 129 pgs: 129 active+clean; 3.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 4.8 MiB/s wr, 453 op/s 2024-05-03T18:46:05.111 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 890.0 s, 579.8 tps, lat 172.319 ms stddev 184.795 2024-05-03T18:46:06.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:46:06 smithi099 ceph-mon[31658]: pgmap v595: 129 pgs: 129 active+clean; 3.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 6.4 MiB/s wr, 580 op/s 2024-05-03T18:46:06.636 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:46:06 smithi150 ceph-mon[32492]: pgmap v595: 129 pgs: 129 active+clean; 3.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 6.4 MiB/s wr, 580 op/s 2024-05-03T18:46:06.735 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:46:06 smithi033 ceph-mon[28330]: pgmap v595: 129 pgs: 129 active+clean; 3.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 6.4 MiB/s wr, 580 op/s 2024-05-03T18:46:07.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:46:07 smithi099 ceph-mon[31658]: osdmap e117: 12 total, 12 up, 12 in 2024-05-03T18:46:07.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:46:07 smithi150 ceph-mon[32492]: osdmap e117: 12 total, 12 up, 12 in 2024-05-03T18:46:07.735 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:46:07 smithi033 ceph-mon[28330]: osdmap e117: 12 total, 12 up, 12 in 2024-05-03T18:46:08.486 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:46:08 smithi033 ceph-mon[28330]: pgmap v597: 129 pgs: 129 active+clean; 3.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 5.6 MiB/s wr, 540 op/s 2024-05-03T18:46:08.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:46:08 smithi099 ceph-mon[31658]: pgmap v597: 129 pgs: 129 active+clean; 3.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 5.6 MiB/s wr, 540 op/s 2024-05-03T18:46:08.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:46:08 smithi150 ceph-mon[32492]: pgmap v597: 129 pgs: 129 active+clean; 3.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 5.6 MiB/s wr, 540 op/s 2024-05-03T18:46:08.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.632+0000 7fc66ca00700 1 -- 172.21.15.33:0/3826045546 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc66806b6f0 msgr2=0x7fc66806bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:08.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.632+0000 7fc66ca00700 1 --2- 172.21.15.33:0/3826045546 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc66806b6f0 0x7fc66806bb50 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc650009990 tx=0x7fc650009ca0 comp rx=0 tx=0).stop 2024-05-03T18:46:08.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.632+0000 7fc66ca00700 1 -- 172.21.15.33:0/3826045546 shutdown_connections 2024-05-03T18:46:08.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.632+0000 7fc66ca00700 1 --2- 172.21.15.33:0/3826045546 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc668105ee0 0x7fc668106370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:08.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.632+0000 7fc66ca00700 1 --2- 172.21.15.33:0/3826045546 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6681083d0 0x7fc66806b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:08.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.632+0000 7fc66ca00700 1 --2- 172.21.15.33:0/3826045546 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc66806b6f0 0x7fc66806bb50 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:08.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.632+0000 7fc66ca00700 1 -- 172.21.15.33:0/3826045546 >> 172.21.15.33:0/3826045546 conn(0x7fc668079350 msgr2=0x7fc668079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:46:08.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.632+0000 7fc66ca00700 1 -- 172.21.15.33:0/3826045546 shutdown_connections 2024-05-03T18:46:08.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.633+0000 7fc66ca00700 1 -- 172.21.15.33:0/3826045546 wait complete. 2024-05-03T18:46:08.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.634+0000 7fc66ca00700 1 Processor -- start 2024-05-03T18:46:08.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.634+0000 7fc66ca00700 1 -- start start 2024-05-03T18:46:08.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.635+0000 7fc66ca00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc66806b6f0 0x7fc6681a00f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:08.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.635+0000 7fc66ca00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc668105ee0 0x7fc6681a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:08.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.635+0000 7fc66ca00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc6681083d0 0x7fc6681a4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:08.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.635+0000 7fc66ca00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc668119b80 con 0x7fc668105ee0 2024-05-03T18:46:08.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.635+0000 7fc66ca00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc668119a00 con 0x7fc6681083d0 2024-05-03T18:46:08.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.635+0000 7fc66ca00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc668119d00 con 0x7fc66806b6f0 2024-05-03T18:46:08.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.636+0000 7fc666800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc6681083d0 0x7fc6681a4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:08.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.636+0000 7fc666800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc6681083d0 0x7fc6681a4920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:40842/0 (socket says 172.21.15.33:40842) 2024-05-03T18:46:08.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.636+0000 7fc666800700 1 -- 172.21.15.33:0/491697668 learned_addr learned my addr 172.21.15.33:0/491697668 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:46:08.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.636+0000 7fc65fe00700 1 --2- 172.21.15.33:0/491697668 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc668105ee0 0x7fc6681a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:08.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.636+0000 7fc665e00700 1 --2- 172.21.15.33:0/491697668 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc66806b6f0 0x7fc6681a00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:08.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.637+0000 7fc665e00700 1 -- 172.21.15.33:0/491697668 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc6681083d0 msgr2=0x7fc6681a4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:08.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.637+0000 7fc665e00700 1 --2- 172.21.15.33:0/491697668 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc6681083d0 0x7fc6681a4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:08.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.637+0000 7fc665e00700 1 -- 172.21.15.33:0/491697668 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc668105ee0 msgr2=0x7fc6681a0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:08.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.637+0000 7fc665e00700 1 --2- 172.21.15.33:0/491697668 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc668105ee0 0x7fc6681a0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:08.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.637+0000 7fc665e00700 1 -- 172.21.15.33:0/491697668 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6681a5060 con 0x7fc66806b6f0 2024-05-03T18:46:08.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.637+0000 7fc666800700 1 --2- 172.21.15.33:0/491697668 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc6681083d0 0x7fc6681a4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:46:08.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.638+0000 7fc665e00700 1 --2- 172.21.15.33:0/491697668 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc66806b6f0 0x7fc6681a00f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fc658007ec0 tx=0x7fc65800d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:46:08.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.638+0000 7fc65ea00700 1 -- 172.21.15.33:0/491697668 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc65800ddd0 con 0x7fc66806b6f0 2024-05-03T18:46:08.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.638+0000 7fc66ca00700 1 -- 172.21.15.33:0/491697668 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6681a5350 con 0x7fc66806b6f0 2024-05-03T18:46:08.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.638+0000 7fc66ca00700 1 -- 172.21.15.33:0/491697668 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6681acbd0 con 0x7fc66806b6f0 2024-05-03T18:46:08.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.640+0000 7fc65ea00700 1 -- 172.21.15.33:0/491697668 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc658004bb0 con 0x7fc66806b6f0 2024-05-03T18:46:08.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.640+0000 7fc65ea00700 1 -- 172.21.15.33:0/491697668 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc658004420 con 0x7fc66806b6f0 2024-05-03T18:46:08.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.648+0000 7fc65ea00700 1 -- 172.21.15.33:0/491697668 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fc65801c060 con 0x7fc66806b6f0 2024-05-03T18:46:08.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.651+0000 7fc65ea00700 1 --2- 172.21.15.33:0/491697668 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fc64c0b2aa0 0x7fc64c0b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:08.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.651+0000 7fc65ea00700 1 -- 172.21.15.33:0/491697668 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 9606+0+0 (secure 0 0 0) 0x7fc65801a070 con 0x7fc66806b6f0 2024-05-03T18:46:08.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.652+0000 7fc65fe00700 1 --2- 172.21.15.33:0/491697668 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fc64c0b2aa0 0x7fc64c0b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:08.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.663+0000 7fc65fe00700 1 --2- 172.21.15.33:0/491697668 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fc64c0b2aa0 0x7fc64c0b4f60 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc650006010 tx=0x7fc6500058e0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:46:08.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.666+0000 7fc67733cb80 1 Processor -- start 2024-05-03T18:46:08.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.667+0000 7fc67733cb80 1 -- start start 2024-05-03T18:46:08.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.668+0000 7fc67733cb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5639246317b0 0x563924631b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:08.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.668+0000 7fc67733cb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639246320d0 0x56392462cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:08.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.668+0000 7fc67733cb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56392462d5d0 0x56392462da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:08.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.668+0000 7fc67733cb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56392457fa20 con 0x56392462d5d0 2024-05-03T18:46:08.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.668+0000 7fc67733cb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56392451d760 con 0x5639246317b0 2024-05-03T18:46:08.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.668+0000 7fc67733cb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56392457b9e0 con 0x5639246320d0 2024-05-03T18:46:08.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.669+0000 7fc65fe00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639246320d0 0x56392462cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:08.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.669+0000 7fc65fe00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639246320d0 0x56392462cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:60144/0 (socket says 172.21.15.33:60144) 2024-05-03T18:46:08.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.669+0000 7fc65fe00700 1 -- 172.21.15.33:0/2947144599 learned_addr learned my addr 172.21.15.33:0/2947144599 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:46:08.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.670+0000 7fc666800700 1 --2- 172.21.15.33:0/2947144599 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56392462d5d0 0x56392462da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:08.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.670+0000 7fc665e00700 1 --2- 172.21.15.33:0/2947144599 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5639246317b0 0x563924631b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:08.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.670+0000 7fc65fe00700 1 -- 172.21.15.33:0/2947144599 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5639246317b0 msgr2=0x563924631b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:46:08.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.670+0000 7fc65fe00700 1 --2- 172.21.15.33:0/2947144599 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5639246317b0 0x563924631b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:08.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.670+0000 7fc65fe00700 1 -- 172.21.15.33:0/2947144599 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56392462d5d0 msgr2=0x56392462da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:46:08.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.670+0000 7fc65fe00700 1 --2- 172.21.15.33:0/2947144599 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56392462d5d0 0x56392462da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:08.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.670+0000 7fc65fe00700 1 -- 172.21.15.33:0/2947144599 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639244cc8d0 con 0x5639246320d0 2024-05-03T18:46:08.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.670+0000 7fc666800700 1 --2- 172.21.15.33:0/2947144599 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56392462d5d0 0x56392462da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:46:08.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.671+0000 7fc65fe00700 1 --2- 172.21.15.33:0/2947144599 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639246320d0 0x56392462cfb0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fc650015000 tx=0x7fc65001f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5858a3b96568a157 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:46:08.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.672+0000 7fc657400700 1 -- 172.21.15.33:0/2947144599 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc650003b50 con 0x5639246320d0 2024-05-03T18:46:08.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.672+0000 7fc657400700 1 -- 172.21.15.33:0/2947144599 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc65000bb40 con 0x5639246320d0 2024-05-03T18:46:08.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.672+0000 7fc657400700 1 -- 172.21.15.33:0/2947144599 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc65000fb50 con 0x5639246320d0 2024-05-03T18:46:08.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.673+0000 7fc67733cb80 1 -- 172.21.15.33:0/2947144599 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639246320d0 msgr2=0x56392462cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:08.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.673+0000 7fc67733cb80 1 --2- 172.21.15.33:0/2947144599 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639246320d0 0x56392462cfb0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fc650015000 tx=0x7fc65001f000 comp rx=0 tx=0).stop 2024-05-03T18:46:08.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.674+0000 7fc67733cb80 1 -- 172.21.15.33:0/2947144599 shutdown_connections 2024-05-03T18:46:08.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.674+0000 7fc67733cb80 1 --2- 172.21.15.33:0/2947144599 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56392462d5d0 0x56392462da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:08.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.674+0000 7fc67733cb80 1 --2- 172.21.15.33:0/2947144599 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5639246317b0 0x563924631b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:08.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.674+0000 7fc67733cb80 1 --2- 172.21.15.33:0/2947144599 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639246320d0 0x56392462cfb0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:08.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.674+0000 7fc67733cb80 1 -- 172.21.15.33:0/2947144599 >> 172.21.15.33:0/2947144599 conn(0x56392455bb60 msgr2=0x5639246277f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:46:08.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.674+0000 7fc67733cb80 1 -- 172.21.15.33:0/2947144599 shutdown_connections 2024-05-03T18:46:08.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.675+0000 7fc67733cb80 1 -- 172.21.15.33:0/2947144599 wait complete. 2024-05-03T18:46:08.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.676+0000 7fc67733cb80 1 -- start start 2024-05-03T18:46:08.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.676+0000 7fc67733cb80 1 Processor -- start 2024-05-03T18:46:08.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.677+0000 7fc67733cb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56392462d5d0 0x563924638e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:08.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.677+0000 7fc67733cb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639246317b0 0x5639246361b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:08.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.677+0000 7fc67733cb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639246320d0 0x5639246366f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:08.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.677+0000 7fc67733cb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563924639be0 con 0x5639246320d0 2024-05-03T18:46:08.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.677+0000 7fc67733cb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5639244e5b20 con 0x56392462d5d0 2024-05-03T18:46:08.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.677+0000 7fc67733cb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x563924639d60 con 0x5639246317b0 2024-05-03T18:46:08.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.677+0000 7fc67733cb80 10 client.0 _ll_register_callbacks cb 0x563924526c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:46:08.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.678+0000 7fc65fe00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639246317b0 0x5639246361b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:08.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.678+0000 7fc65fe00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639246317b0 0x5639246361b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:60156/0 (socket says 172.21.15.33:60156) 2024-05-03T18:46:08.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.678+0000 7fc65fe00700 1 -- 172.21.15.33:0/4281029593 learned_addr learned my addr 172.21.15.33:0/4281029593 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:46:08.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.679+0000 7fc666800700 1 --2- 172.21.15.33:0/4281029593 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639246320d0 0x5639246366f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:08.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.679+0000 7fc65fe00700 1 -- 172.21.15.33:0/4281029593 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56392462d5d0 msgr2=0x563924638e10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-03T18:46:08.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.679+0000 7fc65fe00700 1 --2- 172.21.15.33:0/4281029593 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56392462d5d0 0x563924638e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:08.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.679+0000 7fc65fe00700 1 -- 172.21.15.33:0/4281029593 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639246320d0 msgr2=0x5639246366f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:46:08.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.679+0000 7fc65fe00700 1 --2- 172.21.15.33:0/4281029593 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639246320d0 0x5639246366f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:08.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.679+0000 7fc65fe00700 1 -- 172.21.15.33:0/4281029593 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563924545420 con 0x5639246317b0 2024-05-03T18:46:08.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.680+0000 7fc666800700 1 --2- 172.21.15.33:0/4281029593 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639246320d0 0x5639246366f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:46:08.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.680+0000 7fc65fe00700 1 --2- 172.21.15.33:0/4281029593 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639246317b0 0x5639246361b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fc6500098e0 tx=0x7fc650003c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c6ab4727130780d server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:46:08.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.680+0000 7fc64be00700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-05-03T18:46:08.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.681+0000 7fc64be00700 1 -- 172.21.15.33:0/4281029593 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc65003a060 con 0x5639246317b0 2024-05-03T18:46:08.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.681+0000 7fc64be00700 1 -- 172.21.15.33:0/4281029593 <== mon.1 v2:172.21.15.150:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 9606+0+0 (secure 0 0 0) 0x7fc65000f670 con 0x5639246317b0 2024-05-03T18:46:08.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.681+0000 7fc67733cb80 1 -- 172.21.15.33:0/4281029593 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc65000bb40 con 0x5639246317b0 2024-05-03T18:46:08.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.682+0000 7fc64be00700 1 -- 172.21.15.33:0/4281029593 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fc65000f870 con 0x5639246317b0 2024-05-03T18:46:08.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.682+0000 7fc67733cb80 10 client.26878 fetch_fsmap learned FSMap version 21 2024-05-03T18:46:08.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.682+0000 7fc67733cb80 1 -- 172.21.15.33:0/4281029593 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x563924639be0 con 0x5639246317b0 2024-05-03T18:46:08.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.682+0000 7fc64be00700 1 -- 172.21.15.33:0/4281029593 <== mon.1 v2:172.21.15.150:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc65000bb40 con 0x5639246317b0 2024-05-03T18:46:08.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.683+0000 7fc64be00700 1 -- 172.21.15.33:0/4281029593 <== mon.1 v2:172.21.15.150:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fc65002dc00 con 0x5639246317b0 2024-05-03T18:46:08.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.683+0000 7fc67733cb80 10 client.26878 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:46:08.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.683+0000 7fc67733cb80 10 client.26878 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:46:08.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.683+0000 7fc67733cb80 20 client.26878 populate_metadata read hostname 'smithi033' 2024-05-03T18:46:08.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.683+0000 7fc67733cb80 1 --2- 172.21.15.33:0/4281029593 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x563924646620 0x563924648a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:08.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.683+0000 7fc67733cb80 4 client.26878 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:46:08.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.683+0000 7fc67733cb80 1 -- 172.21.15.33:0/4281029593 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5639245fa1d0 con 0x563924646620 2024-05-03T18:46:08.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.685+0000 7fc665e00700 1 --2- 172.21.15.33:0/4281029593 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x563924646620 0x563924648a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:08.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.687+0000 7fc665e00700 1 --2- 172.21.15.33:0/4281029593 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x563924646620 0x563924648a10 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ff3062a1869f766 server_cookie=90ae92c1a791d062 in_seq=0 out_seq=0 2024-05-03T18:46:08.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.687+0000 7fc64be00700 10 client.26878 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:46:08.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.689+0000 7fc64be00700 1 -- 172.21.15.33:0/4281029593 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5639245fa1d0 con 0x563924646620 2024-05-03T18:46:08.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.689+0000 7fc64be00700 10 client.26878 handle_command_reply: tid=0 2024-05-03T18:46:08.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.690+0000 7fc67733cb80 1 client.26878 shutdown 2024-05-03T18:46:08.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.690+0000 7fc67733cb80 1 -- 172.21.15.33:0/4281029593 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639246317b0 msgr2=0x5639246361b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:08.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.690+0000 7fc67733cb80 1 --2- 172.21.15.33:0/4281029593 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639246317b0 0x5639246361b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fc6500098e0 tx=0x7fc650003c80 comp rx=0 tx=0).stop 2024-05-03T18:46:08.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.690+0000 7fc67733cb80 1 -- 172.21.15.33:0/4281029593 shutdown_connections 2024-05-03T18:46:08.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.690+0000 7fc67733cb80 1 --2- 172.21.15.33:0/4281029593 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x563924646620 0x563924648a10 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:08.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.690+0000 7fc67733cb80 1 --2- 172.21.15.33:0/4281029593 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639246320d0 0x5639246366f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:08.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.690+0000 7fc67733cb80 1 --2- 172.21.15.33:0/4281029593 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56392462d5d0 0x563924638e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:08.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.690+0000 7fc67733cb80 1 --2- 172.21.15.33:0/4281029593 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639246317b0 0x5639246361b0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:08.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.690+0000 7fc67733cb80 1 -- 172.21.15.33:0/4281029593 >> 172.21.15.33:0/4281029593 conn(0x56392455bb60 msgr2=0x563924632e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:46:08.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.691+0000 7fc64be00700 0 client.26878 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:46:08.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.691+0000 7fc67733cb80 1 -- 172.21.15.33:0/4281029593 shutdown_connections 2024-05-03T18:46:08.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.692+0000 7fc67733cb80 1 -- 172.21.15.33:0/4281029593 wait complete. 2024-05-03T18:46:08.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.692+0000 7fc67733cb80 20 client.26878 trim_cache size 0 max 16384 2024-05-03T18:46:08.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.731+0000 7fc67733cb80 1 Processor -- start 2024-05-03T18:46:08.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.731+0000 7fc67733cb80 1 -- start start 2024-05-03T18:46:08.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.732+0000 7fc67733cb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56392462d5d0 0x563924627660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:08.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.732+0000 7fc67733cb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639246317b0 0x56392464e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:08.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.732+0000 7fc67733cb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639246320d0 0x56392464e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:08.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.732+0000 7fc67733cb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc64400cf50 con 0x5639246320d0 2024-05-03T18:46:08.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.732+0000 7fc67733cb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc64400f7b0 con 0x56392462d5d0 2024-05-03T18:46:08.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.732+0000 7fc67733cb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc64400d5a0 con 0x5639246317b0 2024-05-03T18:46:08.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.733+0000 7fc65fe00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639246317b0 0x56392464e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:08.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.733+0000 7fc665e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56392462d5d0 0x563924627660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:08.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.734+0000 7fc666800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639246320d0 0x56392464e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:08.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.734+0000 7fc65fe00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639246317b0 0x56392464e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:60158/0 (socket says 172.21.15.33:60158) 2024-05-03T18:46:08.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.734+0000 7fc65fe00700 1 -- 172.21.15.33:0/3984767168 learned_addr learned my addr 172.21.15.33:0/3984767168 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:46:08.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.734+0000 7fc665e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56392462d5d0 0x563924627660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:40878/0 (socket says 172.21.15.33:40878) 2024-05-03T18:46:08.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.735+0000 7fc65fe00700 1 -- 172.21.15.33:0/3984767168 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56392462d5d0 msgr2=0x563924627660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:46:08.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.735+0000 7fc65fe00700 1 --2- 172.21.15.33:0/3984767168 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56392462d5d0 0x563924627660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:08.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.735+0000 7fc65fe00700 1 -- 172.21.15.33:0/3984767168 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639246320d0 msgr2=0x56392464e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:46:08.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.735+0000 7fc65fe00700 1 --2- 172.21.15.33:0/3984767168 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639246320d0 0x56392464e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:08.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.735+0000 7fc65fe00700 1 -- 172.21.15.33:0/3984767168 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563924637090 con 0x5639246317b0 2024-05-03T18:46:08.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.735+0000 7fc665e00700 1 --2- 172.21.15.33:0/3984767168 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56392462d5d0 0x563924627660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:46:08.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.736+0000 7fc65fe00700 1 --2- 172.21.15.33:0/3984767168 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639246317b0 0x56392464e0a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fc65000b2e0 tx=0x7fc650005ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa343b4ced744709 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:46:08.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.737+0000 7fc656a00700 1 -- 172.21.15.33:0/3984767168 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc65003a040 con 0x5639246317b0 2024-05-03T18:46:08.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.737+0000 7fc656a00700 1 -- 172.21.15.33:0/3984767168 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc650003da0 con 0x5639246317b0 2024-05-03T18:46:08.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.737+0000 7fc656a00700 1 -- 172.21.15.33:0/3984767168 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc65002d400 con 0x5639246317b0 2024-05-03T18:46:08.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.738+0000 7fc67733cb80 1 -- 172.21.15.33:0/3984767168 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639246317b0 msgr2=0x56392464e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:08.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.738+0000 7fc67733cb80 1 --2- 172.21.15.33:0/3984767168 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639246317b0 0x56392464e0a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fc65000b2e0 tx=0x7fc650005ab0 comp rx=0 tx=0).stop 2024-05-03T18:46:08.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.739+0000 7fc67733cb80 1 -- 172.21.15.33:0/3984767168 shutdown_connections 2024-05-03T18:46:08.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.739+0000 7fc67733cb80 1 --2- 172.21.15.33:0/3984767168 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639246320d0 0x56392464e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:08.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.739+0000 7fc67733cb80 1 --2- 172.21.15.33:0/3984767168 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56392462d5d0 0x563924627660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:08.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.739+0000 7fc67733cb80 1 --2- 172.21.15.33:0/3984767168 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5639246317b0 0x56392464e0a0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:08.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.739+0000 7fc67733cb80 1 -- 172.21.15.33:0/3984767168 >> 172.21.15.33:0/3984767168 conn(0x56392455bb60 msgr2=0x5639246240c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:46:08.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.740+0000 7fc67733cb80 1 -- 172.21.15.33:0/3984767168 shutdown_connections 2024-05-03T18:46:08.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.740+0000 7fc67733cb80 1 -- 172.21.15.33:0/3984767168 wait complete. 2024-05-03T18:46:08.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.741+0000 7fc67733cb80 1 -- start start 2024-05-03T18:46:08.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.741+0000 7fc67733cb80 1 Processor -- start 2024-05-03T18:46:08.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.743+0000 7fc67733cb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56392462d5d0 0x5639246adc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:08.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.743+0000 7fc67733cb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5639246317b0 0x5639246ae180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:08.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.744+0000 7fc67733cb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639246320d0 0x5639246ae6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:08.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.744+0000 7fc665e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56392462d5d0 0x5639246adc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:08.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.744+0000 7fc665e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56392462d5d0 0x5639246adc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:60162/0 (socket says 172.21.15.33:60162) 2024-05-03T18:46:08.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.744+0000 7fc65fe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5639246317b0 0x5639246ae180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:08.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.744+0000 7fc65fe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5639246317b0 0x5639246ae180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:40894/0 (socket says 172.21.15.33:40894) 2024-05-03T18:46:08.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.744+0000 7fc666800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639246320d0 0x5639246ae6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:08.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.745+0000 7fc666800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639246320d0 0x5639246ae6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35000/0 (socket says 172.21.15.33:35000) 2024-05-03T18:46:08.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.745+0000 7fc665e00700 1 -- 172.21.15.33:0/1774699410 learned_addr learned my addr 172.21.15.33:0/1774699410 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:46:08.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.745+0000 7fc67733cb80 1 -- 172.21.15.33:0/1774699410 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc64400dc70 con 0x5639246320d0 2024-05-03T18:46:08.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.745+0000 7fc67733cb80 1 -- 172.21.15.33:0/1774699410 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc64400f0e0 con 0x5639246317b0 2024-05-03T18:46:08.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.745+0000 7fc67733cb80 1 -- 172.21.15.33:0/1774699410 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc64400e340 con 0x56392462d5d0 2024-05-03T18:46:08.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.746+0000 7fc67733cb80 10 client.0 _ll_register_callbacks cb 0x563924526c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:46:08.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.746+0000 7fc666800700 1 -- 172.21.15.33:0/1774699410 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56392462d5d0 msgr2=0x5639246adc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:46:08.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.746+0000 7fc666800700 1 --2- 172.21.15.33:0/1774699410 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56392462d5d0 0x5639246adc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:08.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.746+0000 7fc666800700 1 -- 172.21.15.33:0/1774699410 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5639246317b0 msgr2=0x5639246ae180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:46:08.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.746+0000 7fc666800700 1 --2- 172.21.15.33:0/1774699410 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5639246317b0 0x5639246ae180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:08.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.746+0000 7fc666800700 1 -- 172.21.15.33:0/1774699410 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56392462abe0 con 0x5639246320d0 2024-05-03T18:46:08.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.747+0000 7fc65fe00700 1 --2- 172.21.15.33:0/1774699410 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5639246317b0 0x5639246ae180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:46:08.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.748+0000 7fc666800700 1 --2- 172.21.15.33:0/1774699410 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639246320d0 0x5639246ae6c0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fc66000f6a0 tx=0x7fc66000fa60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=375a7a47ccbaece4 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:46:08.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.748+0000 7fc64be00700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:46:08.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.749+0000 7fc64be00700 1 -- 172.21.15.33:0/1774699410 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc660020060 con 0x5639246320d0 2024-05-03T18:46:08.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.749+0000 7fc64be00700 1 -- 172.21.15.33:0/1774699410 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 9606+0+0 (secure 0 0 0) 0x7fc66000e880 con 0x5639246320d0 2024-05-03T18:46:08.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.749+0000 7fc67733cb80 1 -- 172.21.15.33:0/1774699410 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc650003da0 con 0x5639246320d0 2024-05-03T18:46:08.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.750+0000 7fc64be00700 1 -- 172.21.15.33:0/1774699410 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fc66000ea80 con 0x5639246320d0 2024-05-03T18:46:08.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.750+0000 7fc67733cb80 10 client.17004 fetch_fsmap learned FSMap version 21 2024-05-03T18:46:08.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.750+0000 7fc67733cb80 1 -- 172.21.15.33:0/1774699410 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x56392463e310 con 0x5639246320d0 2024-05-03T18:46:08.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.751+0000 7fc64be00700 1 -- 172.21.15.33:0/1774699410 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc650003da0 con 0x5639246320d0 2024-05-03T18:46:08.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.751+0000 7fc64be00700 1 -- 172.21.15.33:0/1774699410 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fc66001a070 con 0x5639246320d0 2024-05-03T18:46:08.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.751+0000 7fc67733cb80 10 client.17004 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:46:08.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.751+0000 7fc67733cb80 10 client.17004 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:46:08.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.751+0000 7fc67733cb80 20 client.17004 populate_metadata read hostname 'smithi033' 2024-05-03T18:46:08.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.751+0000 7fc67733cb80 1 --2- 172.21.15.33:0/1774699410 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x56392463e980 0x56392463edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:08.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.751+0000 7fc67733cb80 4 client.17004 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-03T18:46:08.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.751+0000 7fc67733cb80 1 -- 172.21.15.33:0/1774699410 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56392458d400 con 0x56392463e980 2024-05-03T18:46:08.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.753+0000 7fc665e00700 1 --2- 172.21.15.33:0/1774699410 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x56392463e980 0x56392463edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:08.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.754+0000 7fc64be00700 10 client.17004 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:46:08.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.754+0000 7fc665e00700 1 --2- 172.21.15.33:0/1774699410 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x56392463e980 0x56392463edb0 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f939ba0b14e739f5 server_cookie=66bc50040f62dec8 in_seq=0 out_seq=0 2024-05-03T18:46:08.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.754+0000 7fc64be00700 1 -- 172.21.15.33:0/1774699410 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56392458d400 con 0x56392463e980 2024-05-03T18:46:08.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.754+0000 7fc64be00700 10 client.17004 handle_command_reply: tid=0 2024-05-03T18:46:08.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.754+0000 7fc67733cb80 1 client.17004 shutdown 2024-05-03T18:46:08.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.755+0000 7fc67733cb80 1 -- 172.21.15.33:0/1774699410 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639246320d0 msgr2=0x5639246ae6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:08.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.755+0000 7fc67733cb80 1 --2- 172.21.15.33:0/1774699410 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639246320d0 0x5639246ae6c0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fc66000f6a0 tx=0x7fc66000fa60 comp rx=0 tx=0).stop 2024-05-03T18:46:08.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.756+0000 7fc67733cb80 1 -- 172.21.15.33:0/1774699410 shutdown_connections 2024-05-03T18:46:08.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.756+0000 7fc67733cb80 1 --2- 172.21.15.33:0/1774699410 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x56392463e980 0x56392463edb0 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:08.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.756+0000 7fc64be00700 0 client.17004 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:46:08.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.756+0000 7fc67733cb80 1 --2- 172.21.15.33:0/1774699410 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639246320d0 0x5639246ae6c0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:08.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.756+0000 7fc67733cb80 1 --2- 172.21.15.33:0/1774699410 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56392462d5d0 0x5639246adc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:08.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.756+0000 7fc67733cb80 1 --2- 172.21.15.33:0/1774699410 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5639246317b0 0x5639246ae180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:08.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.756+0000 7fc67733cb80 1 -- 172.21.15.33:0/1774699410 >> 172.21.15.33:0/1774699410 conn(0x56392455bb60 msgr2=0x5639246303f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:46:08.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.757+0000 7fc67733cb80 1 -- 172.21.15.33:0/1774699410 shutdown_connections 2024-05-03T18:46:08.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.757+0000 7fc67733cb80 1 -- 172.21.15.33:0/1774699410 wait complete. 2024-05-03T18:46:08.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.757+0000 7fc67733cb80 20 client.17004 trim_cache size 0 max 16384 2024-05-03T18:46:08.760 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-03T18:46:08.761 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-05-03T18:46:08.761 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-05-03T18:46:08.761 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-03T18:46:08.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.759+0000 7fc66ca00700 1 -- 172.21.15.33:0/491697668 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fc64c0b2aa0 msgr2=0x7fc64c0b4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:08.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.759+0000 7fc66ca00700 1 --2- 172.21.15.33:0/491697668 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fc64c0b2aa0 0x7fc64c0b4f60 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc650006010 tx=0x7fc6500058e0 comp rx=0 tx=0).stop 2024-05-03T18:46:08.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.759+0000 7fc66ca00700 1 -- 172.21.15.33:0/491697668 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc66806b6f0 msgr2=0x7fc6681a00f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:08.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.759+0000 7fc66ca00700 1 --2- 172.21.15.33:0/491697668 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc66806b6f0 0x7fc6681a00f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fc658007ec0 tx=0x7fc65800d3b0 comp rx=0 tx=0).stop 2024-05-03T18:46:08.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.759+0000 7fc66ca00700 1 -- 172.21.15.33:0/491697668 shutdown_connections 2024-05-03T18:46:08.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.759+0000 7fc66ca00700 1 --2- 172.21.15.33:0/491697668 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fc64c0b2aa0 0x7fc64c0b4f60 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:08.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.759+0000 7fc66ca00700 1 --2- 172.21.15.33:0/491697668 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc6681083d0 0x7fc6681a4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:08.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.759+0000 7fc66ca00700 1 --2- 172.21.15.33:0/491697668 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc66806b6f0 0x7fc6681a00f0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:08.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.759+0000 7fc66ca00700 1 --2- 172.21.15.33:0/491697668 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc668105ee0 0x7fc6681a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:08.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.759+0000 7fc66ca00700 1 -- 172.21.15.33:0/491697668 >> 172.21.15.33:0/491697668 conn(0x7fc668079350 msgr2=0x7fc668104ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:46:08.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.760+0000 7fc66ca00700 1 -- 172.21.15.33:0/491697668 shutdown_connections 2024-05-03T18:46:08.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:08.760+0000 7fc66ca00700 1 -- 172.21.15.33:0/491697668 wait complete. 2024-05-03T18:46:09.549 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:46:09 smithi099 ceph-mon[31658]: pgmap v598: 129 pgs: 129 active+clean; 3.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 5.6 MiB/s wr, 539 op/s 2024-05-03T18:46:09.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:46:09 smithi150 ceph-mon[32492]: pgmap v598: 129 pgs: 129 active+clean; 3.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 5.6 MiB/s wr, 539 op/s 2024-05-03T18:46:09.735 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:46:09 smithi033 ceph-mon[28330]: pgmap v598: 129 pgs: 129 active+clean; 3.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 5.6 MiB/s wr, 539 op/s 2024-05-03T18:46:10.111 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 895.0 s, 438.8 tps, lat 232.627 ms stddev 287.800 2024-05-03T18:46:11.735 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:46:11 smithi033 ceph-mon[28330]: pgmap v599: 129 pgs: 129 active+clean; 3.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 5.0 MiB/s wr, 473 op/s 2024-05-03T18:46:11.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:46:11 smithi099 ceph-mon[31658]: pgmap v599: 129 pgs: 129 active+clean; 3.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 5.0 MiB/s wr, 473 op/s 2024-05-03T18:46:11.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:46:11 smithi150 ceph-mon[32492]: pgmap v599: 129 pgs: 129 active+clean; 3.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 5.0 MiB/s wr, 473 op/s 2024-05-03T18:46:11.893 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-03T18:46:11.893 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T18:46:12.689 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:46:13.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:46:13 smithi033 ceph-mon[28330]: pgmap v600: 129 pgs: 129 active+clean; 3.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 5.0 MiB/s wr, 473 op/s 2024-05-03T18:46:13.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:46:13 smithi033 ceph-mon[28330]: osdmap e118: 12 total, 12 up, 12 in 2024-05-03T18:46:14.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:46:13 smithi099 ceph-mon[31658]: pgmap v600: 129 pgs: 129 active+clean; 3.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 5.0 MiB/s wr, 473 op/s 2024-05-03T18:46:14.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:46:13 smithi099 ceph-mon[31658]: osdmap e118: 12 total, 12 up, 12 in 2024-05-03T18:46:14.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:46:13 smithi150 ceph-mon[32492]: pgmap v600: 129 pgs: 129 active+clean; 3.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 5.0 MiB/s wr, 473 op/s 2024-05-03T18:46:14.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:46:13 smithi150 ceph-mon[32492]: osdmap e118: 12 total, 12 up, 12 in 2024-05-03T18:46:15.113 INFO:teuthology.orchestra.run.smithi033.stderr:progress: 900.0 s, 523.6 tps, lat 186.832 ms stddev 111.864 2024-05-03T18:46:15.501 INFO:teuthology.orchestra.run.smithi033.stdout:transaction type: 2024-05-03T18:46:15.501 INFO:teuthology.orchestra.run.smithi033.stdout:scaling factor: 32 2024-05-03T18:46:15.502 INFO:teuthology.orchestra.run.smithi033.stdout:query mode: simple 2024-05-03T18:46:15.502 INFO:teuthology.orchestra.run.smithi033.stdout:number of clients: 100 2024-05-03T18:46:15.502 INFO:teuthology.orchestra.run.smithi033.stdout:number of threads: 4 2024-05-03T18:46:15.502 INFO:teuthology.orchestra.run.smithi033.stdout:duration: 900 s 2024-05-03T18:46:15.502 INFO:teuthology.orchestra.run.smithi033.stdout:number of transactions actually processed: 496144 2024-05-03T18:46:15.502 INFO:teuthology.orchestra.run.smithi033.stdout:latency average = 181.268 ms 2024-05-03T18:46:15.502 INFO:teuthology.orchestra.run.smithi033.stdout:latency stddev = 171.773 ms 2024-05-03T18:46:15.502 INFO:teuthology.orchestra.run.smithi033.stdout:tps = 551.032294 (including connections establishing) 2024-05-03T18:46:15.502 INFO:teuthology.orchestra.run.smithi033.stdout:tps = 551.045482 (excluding connections establishing) 2024-05-03T18:46:15.511 DEBUG:teuthology.orchestra.run.smithi033:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl stop postgresql' 2024-05-03T18:46:15.735 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:46:15 smithi033 ceph-mon[28330]: pgmap v602: 129 pgs: 129 active+clean; 3.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 4.6 MiB/s wr, 416 op/s 2024-05-03T18:46:15.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:15.979+0000 7fadd0a00700 1 -- 172.21.15.33:0/3671873308 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fadcc108850 msgr2=0x7fadcc1103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:15.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:15.979+0000 7fadd0a00700 1 --2- 172.21.15.33:0/3671873308 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fadcc108850 0x7fadcc1103d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fadc400b0d0 tx=0x7fadc400b3e0 comp rx=0 tx=0).stop 2024-05-03T18:46:15.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:15.980+0000 7fadd0a00700 1 -- 172.21.15.33:0/3671873308 shutdown_connections 2024-05-03T18:46:15.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:15.980+0000 7fadd0a00700 1 --2- 172.21.15.33:0/3671873308 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fadcc108850 0x7fadcc1103d0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:15.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:15.980+0000 7fadd0a00700 1 --2- 172.21.15.33:0/3671873308 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fadcc103750 0x7fadcc103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:15.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:15.980+0000 7fadd0a00700 1 --2- 172.21.15.33:0/3671873308 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fadcc104100 0x7fadcc108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:15.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:15.980+0000 7fadd0a00700 1 -- 172.21.15.33:0/3671873308 >> 172.21.15.33:0/3671873308 conn(0x7fadcc07a600 msgr2=0x7fadcc07aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:46:15.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:15.981+0000 7fadd0a00700 1 -- 172.21.15.33:0/3671873308 shutdown_connections 2024-05-03T18:46:15.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:15.981+0000 7fadd0a00700 1 -- 172.21.15.33:0/3671873308 wait complete. 2024-05-03T18:46:15.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:15.982+0000 7fadd0a00700 1 Processor -- start 2024-05-03T18:46:15.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:15.983+0000 7fadd0a00700 1 -- start start 2024-05-03T18:46:15.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:15.983+0000 7fadd0a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fadcc103750 0x7fadcc1a1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:15.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:15.983+0000 7fadd0a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fadcc104100 0x7fadcc1a1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:15.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:15.983+0000 7fadd0a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fadcc108850 0x7fadcc19b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:15.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:15.983+0000 7fadd0a00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fadcc1134d0 con 0x7fadcc103750 2024-05-03T18:46:15.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:15.983+0000 7fadd0a00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fadcc113350 con 0x7fadcc104100 2024-05-03T18:46:15.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:15.983+0000 7fadd0a00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fadcc113650 con 0x7fadcc108850 2024-05-03T18:46:15.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:15.984+0000 7fadc9400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fadcc104100 0x7fadcc1a1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:15.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:15.984+0000 7fadca800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fadcc108850 0x7fadcc19b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:15.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:15.984+0000 7fadca800700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fadcc108850 0x7fadcc19b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:32880/0 (socket says 172.21.15.33:32880) 2024-05-03T18:46:15.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:15.984+0000 7fadca800700 1 -- 172.21.15.33:0/1324144097 learned_addr learned my addr 172.21.15.33:0/1324144097 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:46:15.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:15.985+0000 7fadc9e00700 1 --2- 172.21.15.33:0/1324144097 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fadcc103750 0x7fadcc1a1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:15.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:15.986+0000 7fadca800700 1 -- 172.21.15.33:0/1324144097 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fadcc104100 msgr2=0x7fadcc1a1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:15.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:15.986+0000 7fadca800700 1 --2- 172.21.15.33:0/1324144097 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fadcc104100 0x7fadcc1a1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:15.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:15.986+0000 7fadca800700 1 -- 172.21.15.33:0/1324144097 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fadcc103750 msgr2=0x7fadcc1a1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:15.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:15.986+0000 7fadca800700 1 --2- 172.21.15.33:0/1324144097 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fadcc103750 0x7fadcc1a1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:15.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:15.986+0000 7fadca800700 1 -- 172.21.15.33:0/1324144097 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadcc19c100 con 0x7fadcc108850 2024-05-03T18:46:15.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:15.986+0000 7fadc9e00700 1 --2- 172.21.15.33:0/1324144097 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fadcc103750 0x7fadcc1a1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:46:15.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:15.987+0000 7fadca800700 1 --2- 172.21.15.33:0/1324144097 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fadcc108850 0x7fadcc19b9a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fadc4004f90 tx=0x7fadc4009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:46:15.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:15.987+0000 7fadc2a00700 1 -- 172.21.15.33:0/1324144097 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc4020070 con 0x7fadcc108850 2024-05-03T18:46:15.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:15.987+0000 7fadc2a00700 1 -- 172.21.15.33:0/1324144097 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fadc401c440 con 0x7fadcc108850 2024-05-03T18:46:15.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:15.987+0000 7fadc2a00700 1 -- 172.21.15.33:0/1324144097 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc4020070 con 0x7fadcc108850 2024-05-03T18:46:15.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:15.987+0000 7fadd0a00700 1 -- 172.21.15.33:0/1324144097 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadcc19c390 con 0x7fadcc108850 2024-05-03T18:46:15.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:15.988+0000 7fadd0a00700 1 -- 172.21.15.33:0/1324144097 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadcc19c760 con 0x7fadcc108850 2024-05-03T18:46:15.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:15.989+0000 7fadd0a00700 1 -- 172.21.15.33:0/1324144097 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadcc06a190 con 0x7fadcc108850 2024-05-03T18:46:16.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:15.994+0000 7fadc2a00700 1 -- 172.21.15.33:0/1324144097 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fadc4023070 con 0x7fadcc108850 2024-05-03T18:46:16.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:15.997+0000 7fadc2a00700 1 --2- 172.21.15.33:0/1324144097 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fadb00b2bf0 0x7fadb00b50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:16.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:15.997+0000 7fadc2a00700 1 -- 172.21.15.33:0/1324144097 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9518+0+0 (secure 0 0 0) 0x7fadc40e3450 con 0x7fadcc108850 2024-05-03T18:46:16.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:15.999+0000 7fadc9e00700 1 --2- 172.21.15.33:0/1324144097 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fadb00b2bf0 0x7fadb00b50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:16.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:16.000+0000 7fadc9e00700 1 --2- 172.21.15.33:0/1324144097 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fadb00b2bf0 0x7fadb00b50b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fadcc108350 tx=0x7fadb4009260 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:46:16.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:16.000+0000 7fadc2a00700 1 -- 172.21.15.33:0/1324144097 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fadc40aaae0 con 0x7fadcc108850 2024-05-03T18:46:16.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:46:15 smithi099 ceph-mon[31658]: pgmap v602: 129 pgs: 129 active+clean; 3.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 4.6 MiB/s wr, 416 op/s 2024-05-03T18:46:16.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:46:15 smithi150 ceph-mon[32492]: pgmap v602: 129 pgs: 129 active+clean; 3.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 4.6 MiB/s wr, 416 op/s 2024-05-03T18:46:16.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:16.344+0000 7fadd0a00700 1 -- 172.21.15.33:0/1324144097 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fadcc1a8720 con 0x7fadcc108850 2024-05-03T18:46:16.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:16.346+0000 7fadc2a00700 1 -- 172.21.15.33:0/1324144097 <== mon.1 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11801 (secure 0 0 0) 0x7fadc40aa3e0 con 0x7fadcc108850 2024-05-03T18:46:16.348 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:46:16.348 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1327393503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1327393503},{"type":"v1","addr":"172.21.15.33:6835","nonce":1327393503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/2461472948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":2461472948},{"type":"v1","addr":"172.21.15.33:6837","nonce":2461472948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14559,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/442269281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":442269281},{"type":"v1","addr":"172.21.15.33:6839","nonce":442269281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14571,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/645571756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":645571756},{"type":"v1","addr":"172.21.15.33:6841","nonce":645571756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/203598909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":203598909},{"type":"v1","addr":"172.21.15.150:6833","nonce":203598909}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/4029842493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":4029842493},{"type":"v1","addr":"172.21.15.150:6835","nonce":4029842493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3463664423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3463664423},{"type":"v1","addr":"172.21.15.150:6837","nonce":3463664423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24481,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6839/4051648018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":4051648018},{"type":"v1","addr":"172.21.15.150:6839","nonce":4051648018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34418,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1865105479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1865105479},{"type":"v1","addr":"172.21.15.99:6833","nonce":1865105479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34445,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/1206393914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1206393914},{"type":"v1","addr":"172.21.15.99:6835","nonce":1206393914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-03T18:22:28.837091+0000","modified":"2024-05-03T18:23:49.758398+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34508},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34472":{"gid":34472,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6837/3360970342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3360970342},{"type":"v1","addr":"172.21.15.99:6837","nonce":3360970342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34508":{"gid":34508,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/919907308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":919907308},{"type":"v1","addr":"172.21.15.99:6839","nonce":919907308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34508,"qdb_cluster":[34508]},"id":1}]} 2024-05-03T18:46:16.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:16.350+0000 7fadd0a00700 1 -- 172.21.15.33:0/1324144097 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fadb00b2bf0 msgr2=0x7fadb00b50b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:16.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:16.350+0000 7fadd0a00700 1 --2- 172.21.15.33:0/1324144097 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fadb00b2bf0 0x7fadb00b50b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fadcc108350 tx=0x7fadb4009260 comp rx=0 tx=0).stop 2024-05-03T18:46:16.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:16.350+0000 7fadd0a00700 1 -- 172.21.15.33:0/1324144097 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fadcc108850 msgr2=0x7fadcc19b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:16.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:16.350+0000 7fadd0a00700 1 --2- 172.21.15.33:0/1324144097 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fadcc108850 0x7fadcc19b9a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fadc4004f90 tx=0x7fadc4009b80 comp rx=0 tx=0).stop 2024-05-03T18:46:16.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:16.350+0000 7fadd0a00700 1 -- 172.21.15.33:0/1324144097 shutdown_connections 2024-05-03T18:46:16.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:16.350+0000 7fadd0a00700 1 --2- 172.21.15.33:0/1324144097 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fadcc108850 0x7fadcc19b9a0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:16.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:16.350+0000 7fadd0a00700 1 --2- 172.21.15.33:0/1324144097 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fadcc103750 0x7fadcc1a1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:16.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:16.350+0000 7fadd0a00700 1 --2- 172.21.15.33:0/1324144097 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fadb00b2bf0 0x7fadb00b50b0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:16.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:16.350+0000 7fadd0a00700 1 --2- 172.21.15.33:0/1324144097 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fadcc104100 0x7fadcc1a1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:16.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:16.350+0000 7fadd0a00700 1 -- 172.21.15.33:0/1324144097 >> 172.21.15.33:0/1324144097 conn(0x7fadcc07a600 msgr2=0x7fadcc101540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:46:16.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:16.351+0000 7fadd0a00700 1 -- 172.21.15.33:0/1324144097 shutdown_connections 2024-05-03T18:46:16.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:16.351+0000 7fadd0a00700 1 -- 172.21.15.33:0/1324144097 wait complete. 2024-05-03T18:46:16.353 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 21 2024-05-03T18:46:16.651 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:46:16 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/1324144097' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:46:17.034 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:46:16 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/1324144097' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:46:17.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:46:16 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/1324144097' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:46:17.319 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-03T18:46:17.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:46:17 smithi033 ceph-mon[28330]: pgmap v603: 129 pgs: 129 active+clean; 3.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 4.6 MiB/s wr, 448 op/s 2024-05-03T18:46:18.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:46:17 smithi099 ceph-mon[31658]: pgmap v603: 129 pgs: 129 active+clean; 3.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 4.6 MiB/s wr, 448 op/s 2024-05-03T18:46:18.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:46:17 smithi150 ceph-mon[32492]: pgmap v603: 129 pgs: 129 active+clean; 3.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 4.6 MiB/s wr, 448 op/s 2024-05-03T18:46:18.353 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:46:19.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:46:19 smithi033 ceph-mon[28330]: pgmap v604: 129 pgs: 129 active+clean; 3.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 4.6 MiB/s wr, 448 op/s 2024-05-03T18:46:20.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:46:19 smithi099 ceph-mon[31658]: pgmap v604: 129 pgs: 129 active+clean; 3.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 4.6 MiB/s wr, 448 op/s 2024-05-03T18:46:20.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:46:19 smithi150 ceph-mon[32492]: pgmap v604: 129 pgs: 129 active+clean; 3.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 4.6 MiB/s wr, 448 op/s 2024-05-03T18:46:21.689 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:46:21 smithi033 ceph-mon[28330]: pgmap v605: 129 pgs: 129 active+clean; 3.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 7.3 MiB/s wr, 747 op/s 2024-05-03T18:46:22.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:46:21 smithi099 ceph-mon[31658]: pgmap v605: 129 pgs: 129 active+clean; 3.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 7.3 MiB/s wr, 747 op/s 2024-05-03T18:46:22.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:46:21 smithi150 ceph-mon[32492]: pgmap v605: 129 pgs: 129 active+clean; 3.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 7.3 MiB/s wr, 747 op/s 2024-05-03T18:46:23.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:46:23 smithi033 ceph-mon[28330]: pgmap v606: 129 pgs: 129 active+clean; 3.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 7.3 MiB/s wr, 747 op/s 2024-05-03T18:46:24.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:46:23 smithi099 ceph-mon[31658]: pgmap v606: 129 pgs: 129 active+clean; 3.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 7.3 MiB/s wr, 747 op/s 2024-05-03T18:46:24.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:46:23 smithi150 ceph-mon[32492]: pgmap v606: 129 pgs: 129 active+clean; 3.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 7.3 MiB/s wr, 747 op/s 2024-05-03T18:46:25.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.851+0000 7fd72ea00700 1 -- 172.21.15.33:0/4030673006 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd728108410 msgr2=0x7fd7281087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:25.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.851+0000 7fd72ea00700 1 --2- 172.21.15.33:0/4030673006 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd728108410 0x7fd7281087f0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fd718009990 tx=0x7fd718009ca0 comp rx=0 tx=0).stop 2024-05-03T18:46:25.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.851+0000 7fd72ea00700 1 -- 172.21.15.33:0/4030673006 shutdown_connections 2024-05-03T18:46:25.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.851+0000 7fd72ea00700 1 --2- 172.21.15.33:0/4030673006 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd728102d60 0x7fd72810f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:25.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.851+0000 7fd72ea00700 1 --2- 172.21.15.33:0/4030673006 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd728108410 0x7fd7281087f0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:25.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.851+0000 7fd72ea00700 1 --2- 172.21.15.33:0/4030673006 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd7281023a0 0x7fd728102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:25.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.851+0000 7fd72ea00700 1 -- 172.21.15.33:0/4030673006 >> 172.21.15.33:0/4030673006 conn(0x7fd728079350 msgr2=0x7fd728079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:46:25.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.851+0000 7fd72ea00700 1 -- 172.21.15.33:0/4030673006 shutdown_connections 2024-05-03T18:46:25.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.851+0000 7fd72ea00700 1 -- 172.21.15.33:0/4030673006 wait complete. 2024-05-03T18:46:25.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.852+0000 7fd72ea00700 1 Processor -- start 2024-05-03T18:46:25.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.852+0000 7fd72ea00700 1 -- start start 2024-05-03T18:46:25.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.853+0000 7fd72ea00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7281023a0 0x7fd72819f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:25.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.853+0000 7fd72ea00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd728102d60 0x7fd72819fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:25.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.853+0000 7fd72ea00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd728108410 0x7fd728199920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:25.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.853+0000 7fd72ea00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd728112430 con 0x7fd7281023a0 2024-05-03T18:46:25.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.853+0000 7fd72ea00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd7281122b0 con 0x7fd728102d60 2024-05-03T18:46:25.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.853+0000 7fd72ea00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd7281125b0 con 0x7fd728108410 2024-05-03T18:46:25.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.853+0000 7fd727e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7281023a0 0x7fd72819f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:25.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.853+0000 7fd727e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7281023a0 0x7fd72819f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32984/0 (socket says 172.21.15.33:32984) 2024-05-03T18:46:25.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.853+0000 7fd727e00700 1 -- 172.21.15.33:0/1301789729 learned_addr learned my addr 172.21.15.33:0/1301789729 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:46:25.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.854+0000 7fd727400700 1 --2- 172.21.15.33:0/1301789729 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd728102d60 0x7fd72819fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:25.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.854+0000 7fd72ca00700 1 --2- 172.21.15.33:0/1301789729 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd728108410 0x7fd728199920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:25.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.854+0000 7fd727e00700 1 -- 172.21.15.33:0/1301789729 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd728108410 msgr2=0x7fd728199920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:25.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.854+0000 7fd727e00700 1 --2- 172.21.15.33:0/1301789729 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd728108410 0x7fd728199920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:25.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.854+0000 7fd727e00700 1 -- 172.21.15.33:0/1301789729 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd728102d60 msgr2=0x7fd72819fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:25.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.854+0000 7fd727e00700 1 --2- 172.21.15.33:0/1301789729 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd728102d60 0x7fd72819fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:25.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.854+0000 7fd727e00700 1 -- 172.21.15.33:0/1301789729 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd72819a080 con 0x7fd7281023a0 2024-05-03T18:46:25.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.854+0000 7fd727e00700 1 --2- 172.21.15.33:0/1301789729 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7281023a0 0x7fd72819f6c0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fd7180098e0 tx=0x7fd718009550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:46:25.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.855+0000 7fd724c00700 1 -- 172.21.15.33:0/1301789729 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd71800fb00 con 0x7fd7281023a0 2024-05-03T18:46:25.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.855+0000 7fd724c00700 1 -- 172.21.15.33:0/1301789729 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd71800fca0 con 0x7fd7281023a0 2024-05-03T18:46:25.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.855+0000 7fd724c00700 1 -- 172.21.15.33:0/1301789729 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd718017650 con 0x7fd7281023a0 2024-05-03T18:46:25.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.855+0000 7fd72ea00700 1 -- 172.21.15.33:0/1301789729 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd72819a310 con 0x7fd7281023a0 2024-05-03T18:46:25.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.855+0000 7fd72ea00700 1 -- 172.21.15.33:0/1301789729 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd728103fb0 con 0x7fd7281023a0 2024-05-03T18:46:25.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.859+0000 7fd724c00700 1 -- 172.21.15.33:0/1301789729 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fd718009d10 con 0x7fd7281023a0 2024-05-03T18:46:25.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.860+0000 7fd724c00700 1 --2- 172.21.15.33:0/1301789729 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fd70c0b29f0 0x7fd70c0b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:25.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.861+0000 7fd724c00700 1 -- 172.21.15.33:0/1301789729 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9518+0+0 (secure 0 0 0) 0x7fd7180e7c90 con 0x7fd7281023a0 2024-05-03T18:46:25.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.861+0000 7fd727400700 1 --2- 172.21.15.33:0/1301789729 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fd70c0b29f0 0x7fd70c0b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:25.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.862+0000 7fd727400700 1 --2- 172.21.15.33:0/1301789729 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fd70c0b29f0 0x7fd70c0b4eb0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd7280799c0 tx=0x7fd714009260 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:46:25.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.880+0000 7fd739320b80 1 Processor -- start 2024-05-03T18:46:25.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.880+0000 7fd739320b80 1 -- start start 2024-05-03T18:46:25.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.881+0000 7fd739320b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x565212ca77b0 0x565212ca7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:25.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.881+0000 7fd739320b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565212ca80d0 0x565212ca2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:25.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.881+0000 7fd739320b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565212ca35d0 0x565212ca3a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:25.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.881+0000 7fd739320b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x565212bf5a20 con 0x565212ca80d0 2024-05-03T18:46:25.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.881+0000 7fd739320b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x565212b93760 con 0x565212ca35d0 2024-05-03T18:46:25.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.881+0000 7fd739320b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x565212bf19e0 con 0x565212ca77b0 2024-05-03T18:46:25.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.881+0000 7fd727400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565212ca80d0 0x565212ca2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:25.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.881+0000 7fd727400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565212ca80d0 0x565212ca2fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32998/0 (socket says 172.21.15.33:32998) 2024-05-03T18:46:25.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.881+0000 7fd727400700 1 -- 172.21.15.33:0/2077552824 learned_addr learned my addr 172.21.15.33:0/2077552824 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:46:25.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.881+0000 7fd72ca00700 1 --2- 172.21.15.33:0/2077552824 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565212ca35d0 0x565212ca3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:25.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.881+0000 7fd727e00700 1 --2- 172.21.15.33:0/2077552824 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x565212ca77b0 0x565212ca7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:25.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.882+0000 7fd727400700 1 -- 172.21.15.33:0/2077552824 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x565212ca77b0 msgr2=0x565212ca7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:46:25.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.882+0000 7fd727400700 1 --2- 172.21.15.33:0/2077552824 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x565212ca77b0 0x565212ca7b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:25.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.882+0000 7fd727400700 1 -- 172.21.15.33:0/2077552824 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565212ca35d0 msgr2=0x565212ca3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:46:25.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.882+0000 7fd727400700 1 --2- 172.21.15.33:0/2077552824 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565212ca35d0 0x565212ca3a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:25.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.882+0000 7fd727400700 1 -- 172.21.15.33:0/2077552824 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565212b428d0 con 0x565212ca80d0 2024-05-03T18:46:25.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.882+0000 7fd727e00700 1 --2- 172.21.15.33:0/2077552824 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x565212ca77b0 0x565212ca7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:46:25.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.882+0000 7fd72ca00700 1 --2- 172.21.15.33:0/2077552824 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565212ca35d0 0x565212ca3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:46:25.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.882+0000 7fd727400700 1 --2- 172.21.15.33:0/2077552824 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565212ca80d0 0x565212ca2fb0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fd714011e10 tx=0x7fd714014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30ab95925f23e912 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:46:25.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.883+0000 7fd71d600700 1 -- 172.21.15.33:0/2077552824 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd71401f420 con 0x565212ca80d0 2024-05-03T18:46:25.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.883+0000 7fd71d600700 1 -- 172.21.15.33:0/2077552824 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd71401fa50 con 0x565212ca80d0 2024-05-03T18:46:25.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.883+0000 7fd71d600700 1 -- 172.21.15.33:0/2077552824 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd71401da50 con 0x565212ca80d0 2024-05-03T18:46:25.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.884+0000 7fd739320b80 1 -- 172.21.15.33:0/2077552824 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565212ca80d0 msgr2=0x565212ca2fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:25.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.884+0000 7fd739320b80 1 --2- 172.21.15.33:0/2077552824 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565212ca80d0 0x565212ca2fb0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fd714011e10 tx=0x7fd714014000 comp rx=0 tx=0).stop 2024-05-03T18:46:25.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.884+0000 7fd739320b80 1 -- 172.21.15.33:0/2077552824 shutdown_connections 2024-05-03T18:46:25.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.884+0000 7fd739320b80 1 --2- 172.21.15.33:0/2077552824 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565212ca35d0 0x565212ca3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:25.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.884+0000 7fd739320b80 1 --2- 172.21.15.33:0/2077552824 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x565212ca77b0 0x565212ca7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:25.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.884+0000 7fd739320b80 1 --2- 172.21.15.33:0/2077552824 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565212ca80d0 0x565212ca2fb0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:25.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.884+0000 7fd739320b80 1 -- 172.21.15.33:0/2077552824 >> 172.21.15.33:0/2077552824 conn(0x565212bd1b60 msgr2=0x565212c9d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:46:25.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.884+0000 7fd739320b80 1 -- 172.21.15.33:0/2077552824 shutdown_connections 2024-05-03T18:46:25.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.884+0000 7fd739320b80 1 -- 172.21.15.33:0/2077552824 wait complete. 2024-05-03T18:46:25.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.885+0000 7fd739320b80 1 -- start start 2024-05-03T18:46:25.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.885+0000 7fd739320b80 1 Processor -- start 2024-05-03T18:46:25.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.886+0000 7fd739320b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565212ca35d0 0x565212caeea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:25.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.886+0000 7fd739320b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565212ca77b0 0x565212cac2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:25.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.886+0000 7fd739320b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x565212ca80d0 0x565212cac7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:25.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.886+0000 7fd739320b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x565212cafca0 con 0x565212ca35d0 2024-05-03T18:46:25.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.886+0000 7fd739320b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x565212b5bb20 con 0x565212ca77b0 2024-05-03T18:46:25.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.886+0000 7fd739320b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x565212cafe20 con 0x565212ca80d0 2024-05-03T18:46:25.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.886+0000 7fd739320b80 10 client.0 _ll_register_callbacks cb 0x565212b9cc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:46:25.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.886+0000 7fd727e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565212ca35d0 0x565212caeea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:25.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.886+0000 7fd727e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565212ca35d0 0x565212caeea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33014/0 (socket says 172.21.15.33:33014) 2024-05-03T18:46:25.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.886+0000 7fd727e00700 1 -- 172.21.15.33:0/2957614673 learned_addr learned my addr 172.21.15.33:0/2957614673 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:46:25.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.886+0000 7fd727400700 1 --2- 172.21.15.33:0/2957614673 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565212ca77b0 0x565212cac2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:25.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.886+0000 7fd72ca00700 1 --2- 172.21.15.33:0/2957614673 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x565212ca80d0 0x565212cac7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:25.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.887+0000 7fd727e00700 1 -- 172.21.15.33:0/2957614673 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x565212ca80d0 msgr2=0x565212cac7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:46:25.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.887+0000 7fd727e00700 1 --2- 172.21.15.33:0/2957614673 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x565212ca80d0 0x565212cac7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:25.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.887+0000 7fd727e00700 1 -- 172.21.15.33:0/2957614673 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565212ca77b0 msgr2=0x565212cac2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:46:25.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.887+0000 7fd727e00700 1 --2- 172.21.15.33:0/2957614673 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565212ca77b0 0x565212cac2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:25.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.887+0000 7fd727e00700 1 -- 172.21.15.33:0/2957614673 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565212bbb420 con 0x565212ca35d0 2024-05-03T18:46:25.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.887+0000 7fd727e00700 1 --2- 172.21.15.33:0/2957614673 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565212ca35d0 0x565212caeea0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fd7180abf40 tx=0x7fd7180ae4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a89b61d43ffc7d6b server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:46:25.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.887+0000 7fd712000700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:46:25.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.887+0000 7fd712000700 1 -- 172.21.15.33:0/2957614673 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7180ae7e0 con 0x565212ca35d0 2024-05-03T18:46:25.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.887+0000 7fd712000700 1 -- 172.21.15.33:0/2957614673 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9518+0+0 (secure 0 0 0) 0x7fd7180b7820 con 0x565212ca35d0 2024-05-03T18:46:25.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.887+0000 7fd739320b80 1 -- 172.21.15.33:0/2957614673 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd71401fa50 con 0x565212ca35d0 2024-05-03T18:46:25.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.888+0000 7fd712000700 1 -- 172.21.15.33:0/2957614673 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fd7180af3d0 con 0x565212ca35d0 2024-05-03T18:46:25.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.888+0000 7fd739320b80 10 client.17022 fetch_fsmap learned FSMap version 21 2024-05-03T18:46:25.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.888+0000 7fd739320b80 1 -- 172.21.15.33:0/2957614673 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x565212cba4e0 con 0x565212ca35d0 2024-05-03T18:46:25.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.888+0000 7fd712000700 1 -- 172.21.15.33:0/2957614673 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd71401fa50 con 0x565212ca35d0 2024-05-03T18:46:25.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.888+0000 7fd712000700 1 -- 172.21.15.33:0/2957614673 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fd7180050b0 con 0x565212ca35d0 2024-05-03T18:46:25.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.888+0000 7fd739320b80 10 client.17022 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:46:25.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.888+0000 7fd739320b80 10 client.17022 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:46:25.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.888+0000 7fd739320b80 20 client.17022 populate_metadata read hostname 'smithi033' 2024-05-03T18:46:25.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.889+0000 7fd739320b80 1 --2- 172.21.15.33:0/2957614673 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x565212cbcaa0 0x565212cbeec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:25.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.889+0000 7fd739320b80 4 client.17022 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:46:25.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.889+0000 7fd739320b80 1 -- 172.21.15.33:0/2957614673 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565212c701d0 con 0x565212cbcaa0 2024-05-03T18:46:25.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.889+0000 7fd727400700 1 --2- 172.21.15.33:0/2957614673 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x565212cbcaa0 0x565212cbeec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:25.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.890+0000 7fd727400700 1 --2- 172.21.15.33:0/2957614673 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x565212cbcaa0 0x565212cbeec0 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=837f45302336313 server_cookie=cacfbaaa6c94954f in_seq=0 out_seq=0 2024-05-03T18:46:25.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.890+0000 7fd712000700 10 client.17022 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:46:25.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.892+0000 7fd712000700 1 -- 172.21.15.33:0/2957614673 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x565212c701d0 con 0x565212cbcaa0 2024-05-03T18:46:25.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.892+0000 7fd712000700 10 client.17022 handle_command_reply: tid=0 2024-05-03T18:46:25.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.892+0000 7fd739320b80 1 client.17022 shutdown 2024-05-03T18:46:25.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.892+0000 7fd739320b80 1 -- 172.21.15.33:0/2957614673 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565212ca35d0 msgr2=0x565212caeea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:25.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.893+0000 7fd739320b80 1 --2- 172.21.15.33:0/2957614673 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565212ca35d0 0x565212caeea0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fd7180abf40 tx=0x7fd7180ae4f0 comp rx=0 tx=0).stop 2024-05-03T18:46:25.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.893+0000 7fd739320b80 1 -- 172.21.15.33:0/2957614673 shutdown_connections 2024-05-03T18:46:25.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.893+0000 7fd739320b80 1 --2- 172.21.15.33:0/2957614673 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x565212cbcaa0 0x565212cbeec0 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:25.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.893+0000 7fd739320b80 1 --2- 172.21.15.33:0/2957614673 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x565212ca80d0 0x565212cac7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:25.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.893+0000 7fd739320b80 1 --2- 172.21.15.33:0/2957614673 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565212ca35d0 0x565212caeea0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:25.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.893+0000 7fd739320b80 1 --2- 172.21.15.33:0/2957614673 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565212ca77b0 0x565212cac2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:25.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.893+0000 7fd739320b80 1 -- 172.21.15.33:0/2957614673 >> 172.21.15.33:0/2957614673 conn(0x565212bd1b60 msgr2=0x565212ca8e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:46:25.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.893+0000 7fd712000700 0 client.17022 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:46:25.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.893+0000 7fd739320b80 1 -- 172.21.15.33:0/2957614673 shutdown_connections 2024-05-03T18:46:25.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.893+0000 7fd739320b80 1 -- 172.21.15.33:0/2957614673 wait complete. 2024-05-03T18:46:25.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.893+0000 7fd739320b80 20 client.17022 trim_cache size 0 max 16384 2024-05-03T18:46:25.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.912+0000 7fd739320b80 1 Processor -- start 2024-05-03T18:46:25.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.912+0000 7fd739320b80 1 -- start start 2024-05-03T18:46:25.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.913+0000 7fd739320b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x565212ca35d0 0x565212c9f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:25.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.913+0000 7fd739320b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565212ca77b0 0x565212cb8d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:25.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.913+0000 7fd739320b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565212ca80d0 0x565212c9d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:25.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.913+0000 7fd739320b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd7281ab260 con 0x565212ca80d0 2024-05-03T18:46:25.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.913+0000 7fd739320b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd7281adac0 con 0x565212ca77b0 2024-05-03T18:46:25.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.913+0000 7fd739320b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd7281ab8b0 con 0x565212ca35d0 2024-05-03T18:46:25.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.913+0000 7fd72ca00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565212ca80d0 0x565212c9d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:25.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.913+0000 7fd72ca00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565212ca80d0 0x565212c9d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33020/0 (socket says 172.21.15.33:33020) 2024-05-03T18:46:25.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.913+0000 7fd72ca00700 1 -- 172.21.15.33:0/2447337738 learned_addr learned my addr 172.21.15.33:0/2447337738 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:46:25.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.913+0000 7fd727e00700 1 --2- 172.21.15.33:0/2447337738 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x565212ca35d0 0x565212c9f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:25.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.914+0000 7fd727400700 1 --2- 172.21.15.33:0/2447337738 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565212ca77b0 0x565212cb8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:25.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.914+0000 7fd72ca00700 1 -- 172.21.15.33:0/2447337738 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x565212ca35d0 msgr2=0x565212c9f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:46:25.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.914+0000 7fd72ca00700 1 --2- 172.21.15.33:0/2447337738 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x565212ca35d0 0x565212c9f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:25.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.914+0000 7fd72ca00700 1 -- 172.21.15.33:0/2447337738 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565212ca77b0 msgr2=0x565212cb8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:46:25.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.914+0000 7fd72ca00700 1 --2- 172.21.15.33:0/2447337738 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565212ca77b0 0x565212cb8d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:25.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.914+0000 7fd72ca00700 1 -- 172.21.15.33:0/2447337738 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565212cab880 con 0x565212ca80d0 2024-05-03T18:46:25.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.914+0000 7fd72ca00700 1 --2- 172.21.15.33:0/2447337738 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565212ca80d0 0x565212c9d660 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fd720009550 tx=0x7fd72000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aefdd77efaa90866 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:46:25.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.915+0000 7fd71cc00700 1 -- 172.21.15.33:0/2447337738 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd72001a020 con 0x565212ca80d0 2024-05-03T18:46:25.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.915+0000 7fd71cc00700 1 -- 172.21.15.33:0/2447337738 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd7200168c0 con 0x565212ca80d0 2024-05-03T18:46:25.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.915+0000 7fd71cc00700 1 -- 172.21.15.33:0/2447337738 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7200157f0 con 0x565212ca80d0 2024-05-03T18:46:25.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.916+0000 7fd739320b80 1 -- 172.21.15.33:0/2447337738 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565212ca80d0 msgr2=0x565212c9d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:25.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.916+0000 7fd739320b80 1 --2- 172.21.15.33:0/2447337738 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565212ca80d0 0x565212c9d660 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fd720009550 tx=0x7fd72000b040 comp rx=0 tx=0).stop 2024-05-03T18:46:25.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.916+0000 7fd739320b80 1 -- 172.21.15.33:0/2447337738 shutdown_connections 2024-05-03T18:46:25.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.916+0000 7fd739320b80 1 --2- 172.21.15.33:0/2447337738 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565212ca80d0 0x565212c9d660 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:25.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.916+0000 7fd739320b80 1 --2- 172.21.15.33:0/2447337738 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x565212ca35d0 0x565212c9f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:25.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.916+0000 7fd739320b80 1 --2- 172.21.15.33:0/2447337738 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565212ca77b0 0x565212cb8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:25.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.916+0000 7fd739320b80 1 -- 172.21.15.33:0/2447337738 >> 172.21.15.33:0/2447337738 conn(0x565212bd1b60 msgr2=0x565212caabc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:46:25.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.916+0000 7fd739320b80 1 -- 172.21.15.33:0/2447337738 shutdown_connections 2024-05-03T18:46:25.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.916+0000 7fd739320b80 1 -- 172.21.15.33:0/2447337738 wait complete. 2024-05-03T18:46:25.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.916+0000 7fd739320b80 1 -- start start 2024-05-03T18:46:25.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.917+0000 7fd739320b80 1 Processor -- start 2024-05-03T18:46:25.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.917+0000 7fd739320b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565212ca35d0 0x565212ca0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:25.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.917+0000 7fd739320b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565212ca77b0 0x565212ca0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:25.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.917+0000 7fd739320b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x565212ca80d0 0x565212cb40b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:25.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.917+0000 7fd739320b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd7281abf80 con 0x565212ca35d0 2024-05-03T18:46:25.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.917+0000 7fd739320b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd7281ad3f0 con 0x565212ca77b0 2024-05-03T18:46:25.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.917+0000 7fd739320b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd7281ac650 con 0x565212ca80d0 2024-05-03T18:46:25.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.917+0000 7fd739320b80 10 client.0 _ll_register_callbacks cb 0x565212b9cc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:46:25.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.918+0000 7fd727e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565212ca35d0 0x565212ca0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:25.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.918+0000 7fd727e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565212ca35d0 0x565212ca0750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33036/0 (socket says 172.21.15.33:33036) 2024-05-03T18:46:25.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.918+0000 7fd727e00700 1 -- 172.21.15.33:0/3714921805 learned_addr learned my addr 172.21.15.33:0/3714921805 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:46:25.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.918+0000 7fd727400700 1 --2- 172.21.15.33:0/3714921805 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565212ca77b0 0x565212ca0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:25.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.918+0000 7fd72ca00700 1 --2- 172.21.15.33:0/3714921805 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x565212ca80d0 0x565212cb40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:25.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.918+0000 7fd727e00700 1 -- 172.21.15.33:0/3714921805 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x565212ca80d0 msgr2=0x565212cb40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:46:25.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.918+0000 7fd727e00700 1 --2- 172.21.15.33:0/3714921805 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x565212ca80d0 0x565212cb40b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:25.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.918+0000 7fd727e00700 1 -- 172.21.15.33:0/3714921805 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565212ca77b0 msgr2=0x565212ca0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:46:25.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.918+0000 7fd727e00700 1 --2- 172.21.15.33:0/3714921805 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565212ca77b0 0x565212ca0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:25.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.918+0000 7fd727e00700 1 -- 172.21.15.33:0/3714921805 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565212cb4750 con 0x565212ca35d0 2024-05-03T18:46:25.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.919+0000 7fd727e00700 1 --2- 172.21.15.33:0/3714921805 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565212ca35d0 0x565212ca0750 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fd718006010 tx=0x7fd71801ff10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a31fe8287b0a555 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:46:25.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.919+0000 7fd712000700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:46:25.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.919+0000 7fd712000700 1 -- 172.21.15.33:0/3714921805 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7180ae7e0 con 0x565212ca35d0 2024-05-03T18:46:25.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.919+0000 7fd712000700 1 -- 172.21.15.33:0/3714921805 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9518+0+0 (secure 0 0 0) 0x7fd7180b5330 con 0x565212ca35d0 2024-05-03T18:46:25.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.919+0000 7fd739320b80 1 -- 172.21.15.33:0/3714921805 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7200168c0 con 0x565212ca35d0 2024-05-03T18:46:25.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.919+0000 7fd712000700 1 -- 172.21.15.33:0/3714921805 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fd7180aecd0 con 0x565212ca35d0 2024-05-03T18:46:25.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.919+0000 7fd739320b80 10 client.17028 fetch_fsmap learned FSMap version 21 2024-05-03T18:46:25.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.919+0000 7fd739320b80 1 -- 172.21.15.33:0/3714921805 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x565212cb5a30 con 0x565212ca35d0 2024-05-03T18:46:25.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.920+0000 7fd712000700 1 -- 172.21.15.33:0/3714921805 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd7200168c0 con 0x565212ca35d0 2024-05-03T18:46:25.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.920+0000 7fd712000700 1 -- 172.21.15.33:0/3714921805 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fd7180aa640 con 0x565212ca35d0 2024-05-03T18:46:25.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.920+0000 7fd739320b80 10 client.17028 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:46:25.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.920+0000 7fd739320b80 10 client.17028 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:46:25.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.920+0000 7fd739320b80 20 client.17028 populate_metadata read hostname 'smithi033' 2024-05-03T18:46:25.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.920+0000 7fd739320b80 1 --2- 172.21.15.33:0/3714921805 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x565212d275c0 0x565212d299e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:25.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.920+0000 7fd739320b80 4 client.17028 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-03T18:46:25.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.920+0000 7fd739320b80 1 -- 172.21.15.33:0/3714921805 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "damage ls"}) -- 0x565212c03400 con 0x565212d275c0 2024-05-03T18:46:25.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.921+0000 7fd727400700 1 --2- 172.21.15.33:0/3714921805 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x565212d275c0 0x565212d299e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:25.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.922+0000 7fd727400700 1 --2- 172.21.15.33:0/3714921805 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x565212d275c0 0x565212d299e0 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d7c771a16ef1830 server_cookie=98367b9f6104c248 in_seq=0 out_seq=0 2024-05-03T18:46:25.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.922+0000 7fd712000700 10 client.17028 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:46:25.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.923+0000 7fd712000700 1 -- 172.21.15.33:0/3714921805 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x565212c03400 con 0x565212d275c0 2024-05-03T18:46:25.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.923+0000 7fd712000700 10 client.17028 handle_command_reply: tid=0 2024-05-03T18:46:25.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.923+0000 7fd739320b80 1 client.17028 shutdown 2024-05-03T18:46:25.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.923+0000 7fd739320b80 1 -- 172.21.15.33:0/3714921805 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565212ca35d0 msgr2=0x565212ca0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:25.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.923+0000 7fd739320b80 1 --2- 172.21.15.33:0/3714921805 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565212ca35d0 0x565212ca0750 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fd718006010 tx=0x7fd71801ff10 comp rx=0 tx=0).stop 2024-05-03T18:46:25.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.923+0000 7fd739320b80 1 -- 172.21.15.33:0/3714921805 shutdown_connections 2024-05-03T18:46:25.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.923+0000 7fd739320b80 1 --2- 172.21.15.33:0/3714921805 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x565212d275c0 0x565212d299e0 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:25.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.923+0000 7fd739320b80 1 --2- 172.21.15.33:0/3714921805 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x565212ca80d0 0x565212cb40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:25.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.923+0000 7fd739320b80 1 --2- 172.21.15.33:0/3714921805 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565212ca35d0 0x565212ca0750 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:25.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.923+0000 7fd739320b80 1 --2- 172.21.15.33:0/3714921805 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565212ca77b0 0x565212ca0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:25.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.923+0000 7fd712000700 0 client.17028 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:46:25.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.923+0000 7fd739320b80 1 -- 172.21.15.33:0/3714921805 >> 172.21.15.33:0/3714921805 conn(0x565212bd1b60 msgr2=0x565212ca2370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:46:25.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.923+0000 7fd739320b80 1 -- 172.21.15.33:0/3714921805 shutdown_connections 2024-05-03T18:46:25.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.923+0000 7fd739320b80 1 -- 172.21.15.33:0/3714921805 wait complete. 2024-05-03T18:46:25.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.924+0000 7fd739320b80 20 client.17028 trim_cache size 0 max 16384 2024-05-03T18:46:25.926 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-03T18:46:25.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.924+0000 7fd72ea00700 1 -- 172.21.15.33:0/1301789729 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fd70c0b29f0 msgr2=0x7fd70c0b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:25.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.924+0000 7fd72ea00700 1 --2- 172.21.15.33:0/1301789729 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fd70c0b29f0 0x7fd70c0b4eb0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd7280799c0 tx=0x7fd714009260 comp rx=0 tx=0).stop 2024-05-03T18:46:25.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.924+0000 7fd72ea00700 1 -- 172.21.15.33:0/1301789729 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7281023a0 msgr2=0x7fd72819f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:25.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.924+0000 7fd72ea00700 1 --2- 172.21.15.33:0/1301789729 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7281023a0 0x7fd72819f6c0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fd7180098e0 tx=0x7fd718009550 comp rx=0 tx=0).stop 2024-05-03T18:46:25.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.925+0000 7fd72ea00700 1 -- 172.21.15.33:0/1301789729 shutdown_connections 2024-05-03T18:46:25.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.925+0000 7fd72ea00700 1 --2- 172.21.15.33:0/1301789729 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd728108410 0x7fd728199920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:25.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.925+0000 7fd72ea00700 1 --2- 172.21.15.33:0/1301789729 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7281023a0 0x7fd72819f6c0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:25.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.925+0000 7fd72ea00700 1 --2- 172.21.15.33:0/1301789729 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fd70c0b29f0 0x7fd70c0b4eb0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:25.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.925+0000 7fd72ea00700 1 --2- 172.21.15.33:0/1301789729 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd728102d60 0x7fd72819fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:25.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.925+0000 7fd72ea00700 1 -- 172.21.15.33:0/1301789729 >> 172.21.15.33:0/1301789729 conn(0x7fd728079350 msgr2=0x7fd72806b360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:46:25.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.925+0000 7fd72ea00700 1 -- 172.21.15.33:0/1301789729 shutdown_connections 2024-05-03T18:46:25.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:25.925+0000 7fd72ea00700 1 -- 172.21.15.33:0/1301789729 wait complete. 2024-05-03T18:46:25.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:46:25 smithi033 ceph-mon[28330]: pgmap v607: 129 pgs: 129 active+clean; 3.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 7.9 MiB/s wr, 803 op/s 2024-05-03T18:46:26.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:46:25 smithi099 ceph-mon[31658]: pgmap v607: 129 pgs: 129 active+clean; 3.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 7.9 MiB/s wr, 803 op/s 2024-05-03T18:46:26.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:46:25 smithi150 ceph-mon[32492]: pgmap v607: 129 pgs: 129 active+clean; 3.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 7.9 MiB/s wr, 803 op/s 2024-05-03T18:46:27.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:46:27 smithi033 ceph-mon[28330]: pgmap v608: 129 pgs: 129 active+clean; 3.6 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 8.8 MiB/s wr, 928 op/s 2024-05-03T18:46:28.037 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-03T18:46:28.038 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-03T18:46:28.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:46:27 smithi099 ceph-mon[31658]: pgmap v608: 129 pgs: 129 active+clean; 3.6 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 8.8 MiB/s wr, 928 op/s 2024-05-03T18:46:28.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:46:27 smithi150 ceph-mon[32492]: pgmap v608: 129 pgs: 129 active+clean; 3.6 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 8.8 MiB/s wr, 928 op/s 2024-05-03T18:46:28.838 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:46:29.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:46:29 smithi033 ceph-mon[28330]: pgmap v609: 129 pgs: 129 active+clean; 3.6 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 8.1 MiB/s wr, 835 op/s 2024-05-03T18:46:30.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:46:29 smithi099 ceph-mon[31658]: pgmap v609: 129 pgs: 129 active+clean; 3.6 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 8.1 MiB/s wr, 835 op/s 2024-05-03T18:46:30.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:46:29 smithi150 ceph-mon[32492]: pgmap v609: 129 pgs: 129 active+clean; 3.6 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 8.1 MiB/s wr, 835 op/s 2024-05-03T18:46:30.511 DEBUG:teuthology.orchestra.run.smithi033:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-05-03T18:46:31.826 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:46:31 smithi033 ceph-mon[28330]: pgmap v610: 129 pgs: 129 active+clean; 3.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 603 KiB/s rd, 8.4 MiB/s wr, 843 op/s 2024-05-03T18:46:31.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.962+0000 7fec8a600700 1 -- 172.21.15.33:0/3134812532 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec8410b680 msgr2=0x7fec8410ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:31.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.962+0000 7fec8a600700 1 --2- 172.21.15.33:0/3134812532 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec8410b680 0x7fec8410ba60 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fec780099c0 tx=0x7fec78009cd0 comp rx=0 tx=0).stop 2024-05-03T18:46:31.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.962+0000 7fec8a600700 1 -- 172.21.15.33:0/3134812532 shutdown_connections 2024-05-03T18:46:31.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.962+0000 7fec8a600700 1 --2- 172.21.15.33:0/3134812532 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fec84105f90 0x7fec84112450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:31.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.962+0000 7fec8a600700 1 --2- 172.21.15.33:0/3134812532 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec8410b680 0x7fec8410ba60 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:31.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.962+0000 7fec8a600700 1 --2- 172.21.15.33:0/3134812532 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fec841055d0 0x7fec84105a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:31.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.962+0000 7fec8a600700 1 -- 172.21.15.33:0/3134812532 >> 172.21.15.33:0/3134812532 conn(0x7fec84101080 msgr2=0x7fec841034a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:46:31.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.962+0000 7fec8a600700 1 -- 172.21.15.33:0/3134812532 shutdown_connections 2024-05-03T18:46:31.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.962+0000 7fec8a600700 1 -- 172.21.15.33:0/3134812532 wait complete. 2024-05-03T18:46:31.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.964+0000 7fec8a600700 1 Processor -- start 2024-05-03T18:46:31.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.964+0000 7fec8a600700 1 -- start start 2024-05-03T18:46:31.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.964+0000 7fec8a600700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec841055d0 0x7fec841a1770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:31.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.965+0000 7fec8a600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fec84105f90 0x7fec841a1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:31.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.965+0000 7fec8a600700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fec8410b680 0x7fec8419b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:31.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.965+0000 7fec8a600700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fec84115630 con 0x7fec841055d0 2024-05-03T18:46:31.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.965+0000 7fec8a600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fec841154b0 con 0x7fec84105f90 2024-05-03T18:46:31.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.965+0000 7fec8a600700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fec841157b0 con 0x7fec8410b680 2024-05-03T18:46:31.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.965+0000 7fec89200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec841055d0 0x7fec841a1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:31.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.965+0000 7fec89200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec841055d0 0x7fec841a1770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33056/0 (socket says 172.21.15.33:33056) 2024-05-03T18:46:31.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.965+0000 7fec89200700 1 -- 172.21.15.33:0/2615732433 learned_addr learned my addr 172.21.15.33:0/2615732433 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:46:31.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.965+0000 7fec89c00700 1 --2- 172.21.15.33:0/2615732433 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fec8410b680 0x7fec8419b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:31.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.965+0000 7fec83e00700 1 --2- 172.21.15.33:0/2615732433 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fec84105f90 0x7fec841a1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:31.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.966+0000 7fec89200700 1 -- 172.21.15.33:0/2615732433 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fec8410b680 msgr2=0x7fec8419b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:31.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.966+0000 7fec89200700 1 --2- 172.21.15.33:0/2615732433 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fec8410b680 0x7fec8419b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:31.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.966+0000 7fec89200700 1 -- 172.21.15.33:0/2615732433 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fec84105f90 msgr2=0x7fec841a1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:31.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.966+0000 7fec89200700 1 --2- 172.21.15.33:0/2615732433 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fec84105f90 0x7fec841a1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:31.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.966+0000 7fec89200700 1 -- 172.21.15.33:0/2615732433 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec8419c090 con 0x7fec841055d0 2024-05-03T18:46:31.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.966+0000 7fec89200700 1 --2- 172.21.15.33:0/2615732433 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec841055d0 0x7fec841a1770 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fec78000c00 tx=0x7fec780047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:46:31.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.967+0000 7fec81600700 1 -- 172.21.15.33:0/2615732433 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec78004830 con 0x7fec841055d0 2024-05-03T18:46:31.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.967+0000 7fec81600700 1 -- 172.21.15.33:0/2615732433 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fec7800f7f0 con 0x7fec841055d0 2024-05-03T18:46:31.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.967+0000 7fec8a600700 1 -- 172.21.15.33:0/2615732433 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec8419c320 con 0x7fec841055d0 2024-05-03T18:46:31.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.967+0000 7fec81600700 1 -- 172.21.15.33:0/2615732433 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec780053a0 con 0x7fec841055d0 2024-05-03T18:46:31.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.967+0000 7fec8a600700 1 -- 172.21.15.33:0/2615732433 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec841a8640 con 0x7fec841055d0 2024-05-03T18:46:31.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.972+0000 7fec81600700 1 -- 172.21.15.33:0/2615732433 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fec78029060 con 0x7fec841055d0 2024-05-03T18:46:31.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.972+0000 7fec964b5b80 1 Processor -- start 2024-05-03T18:46:31.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.973+0000 7fec964b5b80 1 -- start start 2024-05-03T18:46:31.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.973+0000 7fec964b5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556d871f2ac0 0x556d871f2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:31.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.973+0000 7fec964b5b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d871f33e0 0x556d871ed030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:31.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.973+0000 7fec964b5b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d871ed650 0x556d871edab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:31.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.973+0000 7fec964b5b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556d8713faa0 con 0x556d871ed650 2024-05-03T18:46:31.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.973+0000 7fec964b5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x556d870dd7e0 con 0x556d871f2ac0 2024-05-03T18:46:31.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.973+0000 7fec964b5b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x556d8713ba60 con 0x556d871f33e0 2024-05-03T18:46:31.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.973+0000 7fec83e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d871ed650 0x556d871edab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:31.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.973+0000 7fec83e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d871ed650 0x556d871edab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33058/0 (socket says 172.21.15.33:33058) 2024-05-03T18:46:31.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.973+0000 7fec83e00700 1 -- 172.21.15.33:0/4279228768 learned_addr learned my addr 172.21.15.33:0/4279228768 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:46:31.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.974+0000 7fec89c00700 1 --2- 172.21.15.33:0/4279228768 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556d871f2ac0 0x556d871f2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:31.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.974+0000 7fec89200700 1 --2- 172.21.15.33:0/4279228768 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d871f33e0 0x556d871ed030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:31.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.974+0000 7fec83e00700 1 -- 172.21.15.33:0/4279228768 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d871f33e0 msgr2=0x556d871ed030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:46:31.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.974+0000 7fec83e00700 1 --2- 172.21.15.33:0/4279228768 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d871f33e0 0x556d871ed030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:31.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.974+0000 7fec83e00700 1 -- 172.21.15.33:0/4279228768 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556d871f2ac0 msgr2=0x556d871f2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:46:31.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.974+0000 7fec83e00700 1 --2- 172.21.15.33:0/4279228768 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556d871f2ac0 0x556d871f2ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:31.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.974+0000 7fec83e00700 1 -- 172.21.15.33:0/4279228768 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d8708c950 con 0x556d871ed650 2024-05-03T18:46:31.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.974+0000 7fec81600700 1 --2- 172.21.15.33:0/2615732433 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fec6c0b2a70 0x7fec6c0b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:31.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.974+0000 7fec89200700 1 --2- 172.21.15.33:0/4279228768 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d871f33e0 0x556d871ed030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:46:31.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.975+0000 7fec83e00700 1 --2- 172.21.15.33:0/4279228768 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d871ed650 0x556d871edab0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fec7000bd30 tx=0x7fec700106b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0eadf56d430d6d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:46:31.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.975+0000 7fec81600700 1 -- 172.21.15.33:0/2615732433 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9518+0+0 (secure 0 0 0) 0x7fec780e33f0 con 0x7fec841055d0 2024-05-03T18:46:31.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.975+0000 7fec89200700 1 --2- 172.21.15.33:0/2615732433 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fec6c0b2a70 0x7fec6c0b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:31.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.975+0000 7fec76000700 1 -- 172.21.15.33:0/4279228768 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec70010e10 con 0x556d871ed650 2024-05-03T18:46:31.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.975+0000 7fec76000700 1 -- 172.21.15.33:0/4279228768 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fec700044c0 con 0x556d871ed650 2024-05-03T18:46:31.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.975+0000 7fec76000700 1 -- 172.21.15.33:0/4279228768 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec70011440 con 0x556d871ed650 2024-05-03T18:46:31.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.976+0000 7fec964b5b80 1 -- 172.21.15.33:0/4279228768 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d871ed650 msgr2=0x556d871edab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:31.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.976+0000 7fec964b5b80 1 --2- 172.21.15.33:0/4279228768 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d871ed650 0x556d871edab0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fec7000bd30 tx=0x7fec700106b0 comp rx=0 tx=0).stop 2024-05-03T18:46:31.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.976+0000 7fec89200700 1 --2- 172.21.15.33:0/2615732433 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fec6c0b2a70 0x7fec6c0b4f30 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fec780e0760 tx=0x7fec78009310 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:46:31.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.976+0000 7fec964b5b80 1 -- 172.21.15.33:0/4279228768 shutdown_connections 2024-05-03T18:46:31.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.976+0000 7fec964b5b80 1 --2- 172.21.15.33:0/4279228768 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d871ed650 0x556d871edab0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:31.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.976+0000 7fec964b5b80 1 --2- 172.21.15.33:0/4279228768 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556d871f2ac0 0x556d871f2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:31.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.976+0000 7fec964b5b80 1 --2- 172.21.15.33:0/4279228768 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d871f33e0 0x556d871ed030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:31.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.976+0000 7fec964b5b80 1 -- 172.21.15.33:0/4279228768 >> 172.21.15.33:0/4279228768 conn(0x556d8711bbe0 msgr2=0x556d871e7a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:46:31.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.977+0000 7fec964b5b80 1 -- 172.21.15.33:0/4279228768 shutdown_connections 2024-05-03T18:46:31.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.977+0000 7fec964b5b80 1 -- 172.21.15.33:0/4279228768 wait complete. 2024-05-03T18:46:31.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.977+0000 7fec964b5b80 1 -- start start 2024-05-03T18:46:31.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.977+0000 7fec964b5b80 1 Processor -- start 2024-05-03T18:46:31.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.978+0000 7fec964b5b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d871ed650 0x556d871f9120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:31.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.978+0000 7fec964b5b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d871f2ac0 0x556d871f64c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:31.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.978+0000 7fec964b5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556d871f33e0 0x556d871f6a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:31.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.978+0000 7fec964b5b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556d871f9e50 con 0x556d871f2ac0 2024-05-03T18:46:31.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.978+0000 7fec964b5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x556d870a5ba0 con 0x556d871f33e0 2024-05-03T18:46:31.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.978+0000 7fec964b5b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x556d871f9fd0 con 0x556d871ed650 2024-05-03T18:46:31.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.978+0000 7fec964b5b80 10 client.0 _ll_register_callbacks cb 0x556d870e6cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:46:31.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.979+0000 7fec83e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d871f2ac0 0x556d871f64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:31.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.979+0000 7fec89c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d871ed650 0x556d871f9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:31.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.979+0000 7fec89c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d871ed650 0x556d871f9120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:39738/0 (socket says 172.21.15.33:39738) 2024-05-03T18:46:31.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.979+0000 7fec89c00700 1 -- 172.21.15.33:0/712609013 learned_addr learned my addr 172.21.15.33:0/712609013 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:46:31.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.979+0000 7fec89c00700 1 --2- 172.21.15.33:0/712609013 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556d871f33e0 0x556d871f6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:31.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.979+0000 7fec83e00700 1 -- 172.21.15.33:0/712609013 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d871ed650 msgr2=0x556d871f9120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:46:31.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.979+0000 7fec83e00700 1 --2- 172.21.15.33:0/712609013 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d871ed650 0x556d871f9120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:31.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.979+0000 7fec83e00700 1 -- 172.21.15.33:0/712609013 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556d871f33e0 msgr2=0x556d871f6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:46:31.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.979+0000 7fec83e00700 1 --2- 172.21.15.33:0/712609013 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556d871f33e0 0x556d871f6a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:31.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.979+0000 7fec83e00700 1 -- 172.21.15.33:0/712609013 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d871054a0 con 0x556d871f2ac0 2024-05-03T18:46:31.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.980+0000 7fec83e00700 1 --2- 172.21.15.33:0/712609013 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d871f2ac0 0x556d871f64c0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fec70000c00 tx=0x7fec70005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1090d75a6811e24e server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:46:31.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.980+0000 7fec6aa00700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:46:31.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.980+0000 7fec6aa00700 1 -- 172.21.15.33:0/712609013 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec700055c0 con 0x556d871f2ac0 2024-05-03T18:46:31.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.980+0000 7fec6aa00700 1 -- 172.21.15.33:0/712609013 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9518+0+0 (secure 0 0 0) 0x7fec70004750 con 0x556d871f2ac0 2024-05-03T18:46:31.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.980+0000 7fec964b5b80 1 -- 172.21.15.33:0/712609013 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec700044c0 con 0x556d871f2ac0 2024-05-03T18:46:31.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.981+0000 7fec6aa00700 1 -- 172.21.15.33:0/712609013 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fec70004950 con 0x556d871f2ac0 2024-05-03T18:46:31.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.981+0000 7fec964b5b80 10 client.17040 fetch_fsmap learned FSMap version 21 2024-05-03T18:46:31.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.981+0000 7fec964b5b80 1 -- 172.21.15.33:0/712609013 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x556d872046b0 con 0x556d871f2ac0 2024-05-03T18:46:31.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.981+0000 7fec6aa00700 1 -- 172.21.15.33:0/712609013 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fec700044c0 con 0x556d871f2ac0 2024-05-03T18:46:31.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.981+0000 7fec6aa00700 1 -- 172.21.15.33:0/712609013 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fec70011aa0 con 0x556d871f2ac0 2024-05-03T18:46:31.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.981+0000 7fec964b5b80 10 client.17040 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:46:31.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.981+0000 7fec964b5b80 10 client.17040 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:46:31.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.981+0000 7fec964b5b80 20 client.17040 populate_metadata read hostname 'smithi033' 2024-05-03T18:46:31.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.981+0000 7fec964b5b80 1 --2- 172.21.15.33:0/712609013 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x556d87206c30 0x556d87209020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:31.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.981+0000 7fec964b5b80 4 client.17040 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:46:31.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.981+0000 7fec964b5b80 1 -- 172.21.15.33:0/712609013 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556d871ba250 con 0x556d87206c30 2024-05-03T18:46:31.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.982+0000 7fec89c00700 1 --2- 172.21.15.33:0/712609013 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x556d87206c30 0x556d87209020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:31.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.983+0000 7fec89c00700 1 --2- 172.21.15.33:0/712609013 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x556d87206c30 0x556d87209020 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d62116f854553977 server_cookie=15c8995572736b36 in_seq=0 out_seq=0 2024-05-03T18:46:31.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.983+0000 7fec6aa00700 10 client.17040 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:46:31.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.985+0000 7fec6aa00700 1 -- 172.21.15.33:0/712609013 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556d871ba250 con 0x556d87206c30 2024-05-03T18:46:31.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.985+0000 7fec6aa00700 10 client.17040 handle_command_reply: tid=0 2024-05-03T18:46:31.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.985+0000 7fec964b5b80 1 client.17040 shutdown 2024-05-03T18:46:31.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.985+0000 7fec964b5b80 1 -- 172.21.15.33:0/712609013 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d871f2ac0 msgr2=0x556d871f64c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:31.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.985+0000 7fec964b5b80 1 --2- 172.21.15.33:0/712609013 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d871f2ac0 0x556d871f64c0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fec70000c00 tx=0x7fec70005040 comp rx=0 tx=0).stop 2024-05-03T18:46:31.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.986+0000 7fec964b5b80 1 -- 172.21.15.33:0/712609013 shutdown_connections 2024-05-03T18:46:31.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.986+0000 7fec964b5b80 1 --2- 172.21.15.33:0/712609013 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x556d87206c30 0x556d87209020 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:31.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.986+0000 7fec964b5b80 1 --2- 172.21.15.33:0/712609013 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556d871f33e0 0x556d871f6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:31.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.986+0000 7fec964b5b80 1 --2- 172.21.15.33:0/712609013 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d871ed650 0x556d871f9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:31.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.986+0000 7fec964b5b80 1 --2- 172.21.15.33:0/712609013 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d871f2ac0 0x556d871f64c0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:31.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.986+0000 7fec964b5b80 1 -- 172.21.15.33:0/712609013 >> 172.21.15.33:0/712609013 conn(0x556d8711bbe0 msgr2=0x556d871e7100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:46:31.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.986+0000 7fec6aa00700 0 client.17040 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:46:31.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.986+0000 7fec964b5b80 1 -- 172.21.15.33:0/712609013 shutdown_connections 2024-05-03T18:46:31.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.986+0000 7fec964b5b80 1 -- 172.21.15.33:0/712609013 wait complete. 2024-05-03T18:46:31.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:31.986+0000 7fec964b5b80 20 client.17040 trim_cache size 0 max 16384 2024-05-03T18:46:32.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:32.011+0000 7fec964b5b80 1 Processor -- start 2024-05-03T18:46:32.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:32.011+0000 7fec964b5b80 1 -- start start 2024-05-03T18:46:32.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:32.011+0000 7fec964b5b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d871ed650 0x556d871ed030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:32.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:32.011+0000 7fec964b5b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d871f2ac0 0x556d871f4080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:32.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:32.011+0000 7fec964b5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556d871f33e0 0x556d87202ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:32.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:32.011+0000 7fec964b5b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fec841af370 con 0x556d871ed650 2024-05-03T18:46:32.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:32.011+0000 7fec964b5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fec841b1bd0 con 0x556d871f33e0 2024-05-03T18:46:32.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:32.011+0000 7fec964b5b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fec841af9c0 con 0x556d871f2ac0 2024-05-03T18:46:32.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:32.012+0000 7fec89c00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d871ed650 0x556d871ed030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:32.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:32.012+0000 7fec89c00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d871ed650 0x556d871ed030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33074/0 (socket says 172.21.15.33:33074) 2024-05-03T18:46:32.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:32.012+0000 7fec89c00700 1 -- 172.21.15.33:0/3436260860 learned_addr learned my addr 172.21.15.33:0/3436260860 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:46:32.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:32.012+0000 7fec83e00700 1 --2- 172.21.15.33:0/3436260860 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d871f2ac0 0x556d871f4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:32.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:32.012+0000 7fec89c00700 1 -- 172.21.15.33:0/3436260860 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d871f2ac0 msgr2=0x556d871f4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:46:32.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:32.012+0000 7fec89c00700 1 --2- 172.21.15.33:0/3436260860 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d871f2ac0 0x556d871f4080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:32.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:32.012+0000 7fec89c00700 1 -- 172.21.15.33:0/3436260860 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556d871f33e0 msgr2=0x556d87202ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:46:32.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:32.012+0000 7fec89c00700 1 --2- 172.21.15.33:0/3436260860 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556d871f33e0 0x556d87202ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:32.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:32.012+0000 7fec89c00700 1 -- 172.21.15.33:0/3436260860 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d871f6d80 con 0x556d871ed650 2024-05-03T18:46:32.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:32.012+0000 7fec83e00700 1 --2- 172.21.15.33:0/3436260860 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d871f2ac0 0x556d871f4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:46:32.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:32.013+0000 7fec89c00700 1 --2- 172.21.15.33:0/3436260860 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d871ed650 0x556d871ed030 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fec7c006090 tx=0x7fec7c009940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a41899cc05e91f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:46:32.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:32.013+0000 7fec75600700 1 -- 172.21.15.33:0/3436260860 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec7c0077c0 con 0x556d871ed650 2024-05-03T18:46:32.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:32.013+0000 7fec75600700 1 -- 172.21.15.33:0/3436260860 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fec7c023070 con 0x556d871ed650 2024-05-03T18:46:32.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:32.013+0000 7fec75600700 1 -- 172.21.15.33:0/3436260860 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec7c01bce0 con 0x556d871ed650 2024-05-03T18:46:32.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:32.014+0000 7fec964b5b80 1 -- 172.21.15.33:0/3436260860 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d871ed650 msgr2=0x556d871ed030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:32.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:32.014+0000 7fec964b5b80 1 --2- 172.21.15.33:0/3436260860 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d871ed650 0x556d871ed030 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fec7c006090 tx=0x7fec7c009940 comp rx=0 tx=0).stop 2024-05-03T18:46:32.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:32.014+0000 7fec964b5b80 1 -- 172.21.15.33:0/3436260860 shutdown_connections 2024-05-03T18:46:32.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:32.014+0000 7fec964b5b80 1 --2- 172.21.15.33:0/3436260860 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556d871f33e0 0x556d87202ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:32.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:32.014+0000 7fec964b5b80 1 --2- 172.21.15.33:0/3436260860 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d871ed650 0x556d871ed030 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:32.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:32.014+0000 7fec964b5b80 1 --2- 172.21.15.33:0/3436260860 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d871f2ac0 0x556d871f4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:32.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:32.014+0000 7fec964b5b80 1 -- 172.21.15.33:0/3436260860 >> 172.21.15.33:0/3436260860 conn(0x556d8711bbe0 msgr2=0x556d871e7100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:46:32.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:32.014+0000 7fec964b5b80 1 -- 172.21.15.33:0/3436260860 shutdown_connections 2024-05-03T18:46:32.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:32.014+0000 7fec964b5b80 1 -- 172.21.15.33:0/3436260860 wait complete. 2024-05-03T18:46:32.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:32.015+0000 7fec964b5b80 1 -- start start 2024-05-03T18:46:32.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:32.015+0000 7fec964b5b80 1 Processor -- start 2024-05-03T18:46:32.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:32.016+0000 7fec964b5b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d871ed650 0x556d871ff440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:32.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:32.016+0000 7fec964b5b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d871f2ac0 0x556d871ff980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:32.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:32.016+0000 7fec964b5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556d871f33e0 0x556d871fc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:32.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:32.016+0000 7fec964b5b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fec841b0090 con 0x556d871f2ac0 2024-05-03T18:46:32.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:32.016+0000 7fec964b5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fec841b1500 con 0x556d871f33e0 2024-05-03T18:46:32.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:32.016+0000 7fec964b5b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fec841b0760 con 0x556d871ed650 2024-05-03T18:46:32.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:32.016+0000 7fec964b5b80 10 client.0 _ll_register_callbacks cb 0x556d870e6cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:46:32.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:32.016+0000 7fec83e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d871f2ac0 0x556d871ff980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:32.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:32.016+0000 7fec83e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d871f2ac0 0x556d871ff980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33090/0 (socket says 172.21.15.33:33090) 2024-05-03T18:46:32.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:32.016+0000 7fec83e00700 1 -- 172.21.15.33:0/1742603679 learned_addr learned my addr 172.21.15.33:0/1742603679 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:46:32.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:32.016+0000 7fec89c00700 1 --2- 172.21.15.33:0/1742603679 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d871ed650 0x556d871ff440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:32.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:32.016+0000 7fec89c00700 1 --2- 172.21.15.33:0/1742603679 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556d871f33e0 0x556d871fc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:32.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:32.017+0000 7fec83e00700 1 -- 172.21.15.33:0/1742603679 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d871ed650 msgr2=0x556d871ff440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:46:32.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:32.017+0000 7fec83e00700 1 --2- 172.21.15.33:0/1742603679 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d871ed650 0x556d871ff440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:32.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:32.017+0000 7fec83e00700 1 -- 172.21.15.33:0/1742603679 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556d871f33e0 msgr2=0x556d871fc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:46:32.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:32.017+0000 7fec83e00700 1 --2- 172.21.15.33:0/1742603679 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556d871f33e0 0x556d871fc300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:32.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:32.017+0000 7fec83e00700 1 -- 172.21.15.33:0/1742603679 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d871fcb40 con 0x556d871f2ac0 2024-05-03T18:46:32.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:32.017+0000 7fec89c00700 1 --2- 172.21.15.33:0/1742603679 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556d871f33e0 0x556d871fc300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:46:32.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:32.017+0000 7fec83e00700 1 --2- 172.21.15.33:0/1742603679 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d871f2ac0 0x556d871ff980 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fec7000bc50 tx=0x7fec70005ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c4e4f4a213824da server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:46:32.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:32.017+0000 7fec6aa00700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:46:32.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:32.017+0000 7fec6aa00700 1 -- 172.21.15.33:0/1742603679 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec700055d0 con 0x556d871f2ac0 2024-05-03T18:46:32.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:32.017+0000 7fec6aa00700 1 -- 172.21.15.33:0/1742603679 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9518+0+0 (secure 0 0 0) 0x7fec70004af0 con 0x556d871f2ac0 2024-05-03T18:46:32.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:32.017+0000 7fec964b5b80 1 -- 172.21.15.33:0/1742603679 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec7c023070 con 0x556d871f2ac0 2024-05-03T18:46:32.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:32.018+0000 7fec6aa00700 1 -- 172.21.15.33:0/1742603679 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fec70040050 con 0x556d871f2ac0 2024-05-03T18:46:32.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:32.018+0000 7fec964b5b80 10 client.17046 fetch_fsmap learned FSMap version 21 2024-05-03T18:46:32.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:32.018+0000 7fec964b5b80 1 -- 172.21.15.33:0/1742603679 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x556d87271e50 con 0x556d871f2ac0 2024-05-03T18:46:32.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:32.018+0000 7fec6aa00700 1 -- 172.21.15.33:0/1742603679 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fec7c023070 con 0x556d871f2ac0 2024-05-03T18:46:32.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:32.018+0000 7fec6aa00700 1 -- 172.21.15.33:0/1742603679 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fec700115b0 con 0x556d871f2ac0 2024-05-03T18:46:32.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:32.019+0000 7fec964b5b80 10 client.17046 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:46:32.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:32.019+0000 7fec964b5b80 10 client.17046 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:46:32.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:32.019+0000 7fec964b5b80 20 client.17046 populate_metadata read hostname 'smithi033' 2024-05-03T18:46:32.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:32.019+0000 7fec964b5b80 1 --2- 172.21.15.33:0/1742603679 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x556d87272520 0x556d87274940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:32.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:32.019+0000 7fec964b5b80 4 client.17046 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-03T18:46:32.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:32.019+0000 7fec964b5b80 1 -- 172.21.15.33:0/1742603679 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x556d8714d480 con 0x556d87272520 2024-05-03T18:46:32.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:32.019+0000 7fec89c00700 1 --2- 172.21.15.33:0/1742603679 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x556d87272520 0x556d87274940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:32.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:32.020+0000 7fec89c00700 1 --2- 172.21.15.33:0/1742603679 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x556d87272520 0x556d87274940 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eaf71c265ed999c8 server_cookie=2aff771b27cbee35 in_seq=0 out_seq=0 2024-05-03T18:46:32.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:32.020+0000 7fec6aa00700 10 client.17046 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:46:32.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:32.023+0000 7fec6aa00700 1 -- 172.21.15.33:0/1742603679 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x556d8714d480 con 0x556d87272520 2024-05-03T18:46:32.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:32.023+0000 7fec6aa00700 10 client.17046 handle_command_reply: tid=0 2024-05-03T18:46:32.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:32.023+0000 7fec964b5b80 1 client.17046 shutdown 2024-05-03T18:46:32.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:32.023+0000 7fec964b5b80 1 -- 172.21.15.33:0/1742603679 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d871f2ac0 msgr2=0x556d871ff980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:32.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:32.023+0000 7fec964b5b80 1 --2- 172.21.15.33:0/1742603679 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d871f2ac0 0x556d871ff980 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fec7000bc50 tx=0x7fec70005ce0 comp rx=0 tx=0).stop 2024-05-03T18:46:32.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:32.023+0000 7fec964b5b80 1 -- 172.21.15.33:0/1742603679 shutdown_connections 2024-05-03T18:46:32.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:32.023+0000 7fec964b5b80 1 --2- 172.21.15.33:0/1742603679 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x556d87272520 0x556d87274940 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:32.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:32.024+0000 7fec964b5b80 1 --2- 172.21.15.33:0/1742603679 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556d871f33e0 0x556d871fc300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:32.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:32.024+0000 7fec964b5b80 1 --2- 172.21.15.33:0/1742603679 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556d871ed650 0x556d871ff440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:32.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:32.024+0000 7fec964b5b80 1 --2- 172.21.15.33:0/1742603679 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d871f2ac0 0x556d871ff980 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:32.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:32.024+0000 7fec964b5b80 1 -- 172.21.15.33:0/1742603679 >> 172.21.15.33:0/1742603679 conn(0x556d8711bbe0 msgr2=0x556d871f1620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:46:32.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:32.024+0000 7fec6aa00700 0 client.17046 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:46:32.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:32.024+0000 7fec964b5b80 1 -- 172.21.15.33:0/1742603679 shutdown_connections 2024-05-03T18:46:32.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:32.024+0000 7fec964b5b80 1 -- 172.21.15.33:0/1742603679 wait complete. 2024-05-03T18:46:32.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:32.024+0000 7fec964b5b80 20 client.17046 trim_cache size 0 max 16384 2024-05-03T18:46:32.026 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-03T18:46:32.026 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-05-03T18:46:32.026 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "ecbf00a8-db96-45d6-b515-6fd29b57c5b0", 2024-05-03T18:46:32.027 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-05-03T18:46:32.027 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-03T18:46:32.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:32.025+0000 7fec8a600700 1 -- 172.21.15.33:0/2615732433 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fec6c0b2a70 msgr2=0x7fec6c0b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:32.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:32.025+0000 7fec8a600700 1 --2- 172.21.15.33:0/2615732433 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fec6c0b2a70 0x7fec6c0b4f30 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fec780e0760 tx=0x7fec78009310 comp rx=0 tx=0).stop 2024-05-03T18:46:32.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:32.025+0000 7fec8a600700 1 -- 172.21.15.33:0/2615732433 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec841055d0 msgr2=0x7fec841a1770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:32.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:32.025+0000 7fec8a600700 1 --2- 172.21.15.33:0/2615732433 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec841055d0 0x7fec841a1770 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fec78000c00 tx=0x7fec780047b0 comp rx=0 tx=0).stop 2024-05-03T18:46:32.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:32.025+0000 7fec8a600700 1 -- 172.21.15.33:0/2615732433 shutdown_connections 2024-05-03T18:46:32.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:32.025+0000 7fec8a600700 1 --2- 172.21.15.33:0/2615732433 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fec8410b680 0x7fec8419b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:32.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:32.025+0000 7fec8a600700 1 --2- 172.21.15.33:0/2615732433 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec841055d0 0x7fec841a1770 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:32.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:32.025+0000 7fec8a600700 1 --2- 172.21.15.33:0/2615732433 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fec6c0b2a70 0x7fec6c0b4f30 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:32.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:32.025+0000 7fec8a600700 1 --2- 172.21.15.33:0/2615732433 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fec84105f90 0x7fec841a1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:32.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:32.025+0000 7fec8a600700 1 -- 172.21.15.33:0/2615732433 >> 172.21.15.33:0/2615732433 conn(0x7fec84101080 msgr2=0x7fec84102820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:46:32.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:32.025+0000 7fec8a600700 1 -- 172.21.15.33:0/2615732433 shutdown_connections 2024-05-03T18:46:32.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:32.026+0000 7fec8a600700 1 -- 172.21.15.33:0/2615732433 wait complete. 2024-05-03T18:46:32.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:46:31 smithi099 ceph-mon[31658]: pgmap v610: 129 pgs: 129 active+clean; 3.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 603 KiB/s rd, 8.4 MiB/s wr, 843 op/s 2024-05-03T18:46:32.055 INFO:teuthology.orchestra.run.smithi033.stdout:/tmp/cephfs/postgres/: 2024-05-03T18:46:32.055 INFO:teuthology.orchestra.run.smithi033.stdout:total 2 2024-05-03T18:46:32.055 INFO:teuthology.orchestra.run.smithi033.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1482306596 May 3 18:27 . 2024-05-03T18:46:32.055 INFO:teuthology.orchestra.run.smithi033.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1482306596 May 3 18:27 .. 2024-05-03T18:46:32.055 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1482306596 May 3 18:46 data 2024-05-03T18:46:32.055 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:46:32.055 INFO:teuthology.orchestra.run.smithi033.stdout:/tmp/cephfs/postgres/data: 2024-05-03T18:46:32.055 INFO:teuthology.orchestra.run.smithi033.stdout:total 46 2024-05-03T18:46:32.055 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1482306596 May 3 18:46 . 2024-05-03T18:46:32.055 INFO:teuthology.orchestra.run.smithi033.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1482306596 May 3 18:27 .. 2024-05-03T18:46:32.055 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 575411765 May 3 18:30 base 2024-05-03T18:46:32.056 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 30 May 3 18:28 current_logfiles 2024-05-03T18:46:32.056 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 572520 May 3 18:28 global 2024-05-03T18:46:32.056 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1047 May 3 18:28 log 2024-05-03T18:46:32.056 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 pg_commit_ts 2024-05-03T18:46:32.056 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 pg_dynshmem 2024-05-03T18:46:32.056 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4516 May 3 18:27 pg_hba.conf 2024-05-03T18:46:32.056 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 May 3 18:27 pg_ident.conf 2024-05-03T18:46:32.056 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 May 3 18:46 pg_logical 2024-05-03T18:46:32.056 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 pg_multixact 2024-05-03T18:46:32.056 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 pg_notify 2024-05-03T18:46:32.056 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 pg_replslot 2024-05-03T18:46:32.056 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 pg_serial 2024-05-03T18:46:32.056 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 pg_snapshots 2024-05-03T18:46:32.057 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 14082 May 3 18:46 pg_stat 2024-05-03T18:46:32.057 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:46 pg_stat_tmp 2024-05-03T18:46:32.057 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 155648 May 3 18:45 pg_subtrans 2024-05-03T18:46:32.057 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 pg_tblspc 2024-05-03T18:46:32.057 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 pg_twophase 2024-05-03T18:46:32.057 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 3 18:27 PG_VERSION 2024-05-03T18:46:32.057 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 905969664 May 3 18:46 pg_wal 2024-05-03T18:46:32.057 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 May 3 18:27 pg_xact 2024-05-03T18:46:32.057 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 May 3 18:27 postgresql.auto.conf 2024-05-03T18:46:32.057 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 28082 May 3 18:27 postgresql.conf 2024-05-03T18:46:32.057 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 51 May 3 18:28 postmaster.opts 2024-05-03T18:46:32.057 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:46:32.057 INFO:teuthology.orchestra.run.smithi033.stdout:/tmp/cephfs/postgres/data/base: 2024-05-03T18:46:32.058 INFO:teuthology.orchestra.run.smithi033.stdout:total 3 2024-05-03T18:46:32.058 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 575411765 May 3 18:30 . 2024-05-03T18:46:32.058 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1482306596 May 3 18:46 .. 2024-05-03T18:46:32.058 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7963139 May 3 18:27 1 2024-05-03T18:46:32.058 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7963139 May 3 18:27 13493 2024-05-03T18:46:32.058 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 559485487 May 3 18:35 13494 2024-05-03T18:46:32.058 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:31 pgsql_tmp 2024-05-03T18:46:32.058 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:46:32.058 INFO:teuthology.orchestra.run.smithi033.stdout:/tmp/cephfs/postgres/data/base/1: 2024-05-03T18:46:32.058 INFO:teuthology.orchestra.run.smithi033.stdout:total 7778 2024-05-03T18:46:32.058 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7963139 May 3 18:27 . 2024-05-03T18:46:32.058 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 575411765 May 3 18:30 .. 2024-05-03T18:46:32.058 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 112 2024-05-03T18:46:32.059 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 113 2024-05-03T18:46:32.059 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 3 18:27 1247 2024-05-03T18:46:32.059 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 1247_fsm 2024-05-03T18:46:32.059 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 1247_vm 2024-05-03T18:46:32.059 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 May 3 18:27 1249 2024-05-03T18:46:32.059 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 1249_fsm 2024-05-03T18:46:32.059 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 1249_vm 2024-05-03T18:46:32.059 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 May 3 18:27 1255 2024-05-03T18:46:32.059 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 1255_fsm 2024-05-03T18:46:32.059 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 1255_vm 2024-05-03T18:46:32.059 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 May 3 18:27 1259 2024-05-03T18:46:32.059 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 1259_fsm 2024-05-03T18:46:32.059 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 1259_vm 2024-05-03T18:46:32.059 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 May 3 18:27 13344 2024-05-03T18:46:32.059 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 13344_fsm 2024-05-03T18:46:32.060 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 13344_vm 2024-05-03T18:46:32.060 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 13346 2024-05-03T18:46:32.060 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 13348 2024-05-03T18:46:32.060 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 13349 2024-05-03T18:46:32.060 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 13349_fsm 2024-05-03T18:46:32.060 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 13349_vm 2024-05-03T18:46:32.060 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 13351 2024-05-03T18:46:32.060 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 13353 2024-05-03T18:46:32.060 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 13354 2024-05-03T18:46:32.060 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 13354_fsm 2024-05-03T18:46:32.060 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 13354_vm 2024-05-03T18:46:32.060 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 13356 2024-05-03T18:46:32.060 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 13358 2024-05-03T18:46:32.060 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 13359 2024-05-03T18:46:32.061 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 13359_fsm 2024-05-03T18:46:32.061 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 13359_vm 2024-05-03T18:46:32.061 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 13361 2024-05-03T18:46:32.061 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 13363 2024-05-03T18:46:32.061 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 1417 2024-05-03T18:46:32.061 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 1418 2024-05-03T18:46:32.061 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 174 2024-05-03T18:46:32.061 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 175 2024-05-03T18:46:32.061 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2187 2024-05-03T18:46:32.061 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 2224 2024-05-03T18:46:32.061 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 2328 2024-05-03T18:46:32.061 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 2336 2024-05-03T18:46:32.061 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2337 2024-05-03T18:46:32.061 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2579 2024-05-03T18:46:32.061 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2600 2024-05-03T18:46:32.062 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2600_fsm 2024-05-03T18:46:32.062 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2600_vm 2024-05-03T18:46:32.062 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2601 2024-05-03T18:46:32.062 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2601_fsm 2024-05-03T18:46:32.062 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2601_vm 2024-05-03T18:46:32.062 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 3 18:27 2602 2024-05-03T18:46:32.062 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2602_fsm 2024-05-03T18:46:32.062 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2602_vm 2024-05-03T18:46:32.062 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 3 18:27 2603 2024-05-03T18:46:32.062 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2603_fsm 2024-05-03T18:46:32.062 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2603_vm 2024-05-03T18:46:32.062 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 2604 2024-05-03T18:46:32.062 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2605 2024-05-03T18:46:32.063 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2605_fsm 2024-05-03T18:46:32.063 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2605_vm 2024-05-03T18:46:32.063 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2606 2024-05-03T18:46:32.063 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2606_fsm 2024-05-03T18:46:32.063 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2606_vm 2024-05-03T18:46:32.063 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2607 2024-05-03T18:46:32.063 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2607_fsm 2024-05-03T18:46:32.063 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2607_vm 2024-05-03T18:46:32.063 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 May 3 18:27 2608 2024-05-03T18:46:32.063 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2608_fsm 2024-05-03T18:46:32.063 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2608_vm 2024-05-03T18:46:32.063 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 May 3 18:27 2609 2024-05-03T18:46:32.063 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2609_fsm 2024-05-03T18:46:32.064 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2609_vm 2024-05-03T18:46:32.064 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 3 18:27 2610 2024-05-03T18:46:32.064 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2610_fsm 2024-05-03T18:46:32.064 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2610_vm 2024-05-03T18:46:32.064 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 2611 2024-05-03T18:46:32.064 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2612 2024-05-03T18:46:32.064 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2612_fsm 2024-05-03T18:46:32.064 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2612_vm 2024-05-03T18:46:32.064 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 2613 2024-05-03T18:46:32.064 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2615 2024-05-03T18:46:32.064 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2615_fsm 2024-05-03T18:46:32.064 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2615_vm 2024-05-03T18:46:32.064 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2616 2024-05-03T18:46:32.064 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2616_fsm 2024-05-03T18:46:32.065 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2616_vm 2024-05-03T18:46:32.065 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 May 3 18:27 2617 2024-05-03T18:46:32.065 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2617_fsm 2024-05-03T18:46:32.065 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2617_vm 2024-05-03T18:46:32.065 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 May 3 18:27 2618 2024-05-03T18:46:32.065 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2618_fsm 2024-05-03T18:46:32.065 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2618_vm 2024-05-03T18:46:32.065 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 May 3 18:27 2619 2024-05-03T18:46:32.065 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2619_fsm 2024-05-03T18:46:32.065 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2619_vm 2024-05-03T18:46:32.065 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 2620 2024-05-03T18:46:32.065 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2650 2024-05-03T18:46:32.065 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2651 2024-05-03T18:46:32.065 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2652 2024-05-03T18:46:32.066 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 3 18:27 2653 2024-05-03T18:46:32.066 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 3 18:27 2654 2024-05-03T18:46:32.066 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 3 18:27 2655 2024-05-03T18:46:32.066 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2656 2024-05-03T18:46:32.066 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2657 2024-05-03T18:46:32.066 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 May 3 18:27 2658 2024-05-03T18:46:32.066 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 3 18:27 2659 2024-05-03T18:46:32.066 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2660 2024-05-03T18:46:32.066 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2661 2024-05-03T18:46:32.066 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 3 18:27 2662 2024-05-03T18:46:32.066 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 3 18:27 2663 2024-05-03T18:46:32.066 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2664 2024-05-03T18:46:32.066 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2665 2024-05-03T18:46:32.066 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2666 2024-05-03T18:46:32.066 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2667 2024-05-03T18:46:32.067 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2668 2024-05-03T18:46:32.067 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2669 2024-05-03T18:46:32.067 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2670 2024-05-03T18:46:32.067 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 May 3 18:27 2673 2024-05-03T18:46:32.067 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 May 3 18:27 2674 2024-05-03T18:46:32.067 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 May 3 18:27 2675 2024-05-03T18:46:32.067 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2678 2024-05-03T18:46:32.067 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2679 2024-05-03T18:46:32.067 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2680 2024-05-03T18:46:32.067 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2681 2024-05-03T18:46:32.067 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2682 2024-05-03T18:46:32.067 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2683 2024-05-03T18:46:32.067 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2684 2024-05-03T18:46:32.067 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2685 2024-05-03T18:46:32.067 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2686 2024-05-03T18:46:32.068 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2687 2024-05-03T18:46:32.068 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 3 18:27 2688 2024-05-03T18:46:32.068 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 3 18:27 2689 2024-05-03T18:46:32.068 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 May 3 18:27 2690 2024-05-03T18:46:32.068 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 May 3 18:27 2691 2024-05-03T18:46:32.068 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2692 2024-05-03T18:46:32.068 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2693 2024-05-03T18:46:32.068 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2696 2024-05-03T18:46:32.068 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2699 2024-05-03T18:46:32.068 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2701 2024-05-03T18:46:32.068 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2702 2024-05-03T18:46:32.068 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 3 18:27 2703 2024-05-03T18:46:32.068 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 3 18:27 2704 2024-05-03T18:46:32.068 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2753 2024-05-03T18:46:32.069 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2753_fsm 2024-05-03T18:46:32.069 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2753_vm 2024-05-03T18:46:32.069 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2754 2024-05-03T18:46:32.069 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2755 2024-05-03T18:46:32.069 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 3 18:27 2756 2024-05-03T18:46:32.069 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 3 18:27 2757 2024-05-03T18:46:32.069 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 2830 2024-05-03T18:46:32.069 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2831 2024-05-03T18:46:32.069 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 2832 2024-05-03T18:46:32.069 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2833 2024-05-03T18:46:32.069 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 2834 2024-05-03T18:46:32.069 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2835 2024-05-03T18:46:32.069 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 2836 2024-05-03T18:46:32.069 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2837 2024-05-03T18:46:32.069 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 May 3 18:27 2838 2024-05-03T18:46:32.070 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2838_fsm 2024-05-03T18:46:32.070 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2838_vm 2024-05-03T18:46:32.070 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2839 2024-05-03T18:46:32.070 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2840 2024-05-03T18:46:32.070 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2840_fsm 2024-05-03T18:46:32.070 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2840_vm 2024-05-03T18:46:32.070 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2841 2024-05-03T18:46:32.070 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 2995 2024-05-03T18:46:32.070 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2996 2024-05-03T18:46:32.070 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3079 2024-05-03T18:46:32.070 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 3079_fsm 2024-05-03T18:46:32.070 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3079_vm 2024-05-03T18:46:32.070 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 3080 2024-05-03T18:46:32.070 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 3081 2024-05-03T18:46:32.071 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 3 18:27 3085 2024-05-03T18:46:32.071 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 3118 2024-05-03T18:46:32.071 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3119 2024-05-03T18:46:32.071 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 May 3 18:27 3164 2024-05-03T18:46:32.071 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 3256 2024-05-03T18:46:32.071 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3257 2024-05-03T18:46:32.071 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3258 2024-05-03T18:46:32.071 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 3350 2024-05-03T18:46:32.071 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3351 2024-05-03T18:46:32.071 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3379 2024-05-03T18:46:32.071 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3380 2024-05-03T18:46:32.071 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 3381 2024-05-03T18:46:32.071 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 3394 2024-05-03T18:46:32.071 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 3394_fsm 2024-05-03T18:46:32.071 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3394_vm 2024-05-03T18:46:32.072 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 3395 2024-05-03T18:46:32.072 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 3429 2024-05-03T18:46:32.072 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 3430 2024-05-03T18:46:32.072 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3431 2024-05-03T18:46:32.072 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3433 2024-05-03T18:46:32.072 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 3439 2024-05-03T18:46:32.072 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3440 2024-05-03T18:46:32.072 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 3 18:27 3455 2024-05-03T18:46:32.072 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 May 3 18:27 3456 2024-05-03T18:46:32.072 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 3456_fsm 2024-05-03T18:46:32.072 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3456_vm 2024-05-03T18:46:32.072 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 3466 2024-05-03T18:46:32.072 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3467 2024-05-03T18:46:32.072 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3468 2024-05-03T18:46:32.072 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 3501 2024-05-03T18:46:32.073 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3502 2024-05-03T18:46:32.073 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3503 2024-05-03T18:46:32.073 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3534 2024-05-03T18:46:32.073 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3541 2024-05-03T18:46:32.073 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 3541_fsm 2024-05-03T18:46:32.073 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3541_vm 2024-05-03T18:46:32.073 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 3542 2024-05-03T18:46:32.073 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3574 2024-05-03T18:46:32.073 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3575 2024-05-03T18:46:32.073 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 3576 2024-05-03T18:46:32.073 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 3596 2024-05-03T18:46:32.073 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3597 2024-05-03T18:46:32.073 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 3598 2024-05-03T18:46:32.073 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3599 2024-05-03T18:46:32.073 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3600 2024-05-03T18:46:32.074 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 3600_fsm 2024-05-03T18:46:32.074 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3600_vm 2024-05-03T18:46:32.074 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3601 2024-05-03T18:46:32.074 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 3601_fsm 2024-05-03T18:46:32.074 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3601_vm 2024-05-03T18:46:32.074 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3602 2024-05-03T18:46:32.074 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 3602_fsm 2024-05-03T18:46:32.074 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3602_vm 2024-05-03T18:46:32.074 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 3603 2024-05-03T18:46:32.074 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 3603_fsm 2024-05-03T18:46:32.074 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3603_vm 2024-05-03T18:46:32.074 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 3604 2024-05-03T18:46:32.074 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 3605 2024-05-03T18:46:32.074 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 3606 2024-05-03T18:46:32.075 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 3607 2024-05-03T18:46:32.075 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 3608 2024-05-03T18:46:32.075 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 3 18:27 3609 2024-05-03T18:46:32.075 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 3712 2024-05-03T18:46:32.075 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3764 2024-05-03T18:46:32.075 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 3764_fsm 2024-05-03T18:46:32.075 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3764_vm 2024-05-03T18:46:32.075 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 3766 2024-05-03T18:46:32.075 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 3767 2024-05-03T18:46:32.075 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3997 2024-05-03T18:46:32.075 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4143 2024-05-03T18:46:32.075 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4144 2024-05-03T18:46:32.075 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4145 2024-05-03T18:46:32.075 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4146 2024-05-03T18:46:32.076 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4147 2024-05-03T18:46:32.077 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4148 2024-05-03T18:46:32.077 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4149 2024-05-03T18:46:32.077 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4150 2024-05-03T18:46:32.077 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4151 2024-05-03T18:46:32.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:46:31 smithi150 ceph-mon[32492]: pgmap v610: 129 pgs: 129 active+clean; 3.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 603 KiB/s rd, 8.4 MiB/s wr, 843 op/s 2024-05-03T18:46:32.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:46:32 smithi033 ceph-mon[28330]: scrub queued for path: / 2024-05-03T18:46:32.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:46:32 smithi033 ceph-mon[28330]: scrub summary: idle+waiting paths [/] 2024-05-03T18:46:32.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:46:32 smithi033 ceph-mon[28330]: scrub summary: active paths [/] 2024-05-03T18:46:32.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:46:32 smithi033 ceph-mon[28330]: scrub summary: idle+waiting paths [/] 2024-05-03T18:46:32.979 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-03T18:46:33.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:46:32 smithi099 ceph-mon[31658]: scrub queued for path: / 2024-05-03T18:46:33.047 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:46:32 smithi099 ceph-mon[31658]: scrub summary: idle+waiting paths [/] 2024-05-03T18:46:33.047 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:46:32 smithi099 ceph-mon[31658]: scrub summary: active paths [/] 2024-05-03T18:46:33.047 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:46:32 smithi099 ceph-mon[31658]: scrub summary: idle+waiting paths [/] 2024-05-03T18:46:33.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:46:32 smithi150 ceph-mon[32492]: scrub queued for path: / 2024-05-03T18:46:33.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:46:32 smithi150 ceph-mon[32492]: scrub summary: idle+waiting paths [/] 2024-05-03T18:46:33.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:46:32 smithi150 ceph-mon[32492]: scrub summary: active paths [/] 2024-05-03T18:46:33.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:46:32 smithi150 ceph-mon[32492]: scrub summary: idle+waiting paths [/] 2024-05-03T18:46:33.428 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4152 2024-05-03T18:46:33.429 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4153 2024-05-03T18:46:33.429 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4154 2024-05-03T18:46:33.429 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4155 2024-05-03T18:46:33.429 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4156 2024-05-03T18:46:33.429 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4157 2024-05-03T18:46:33.429 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4158 2024-05-03T18:46:33.429 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4159 2024-05-03T18:46:33.429 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4160 2024-05-03T18:46:33.429 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4161 2024-05-03T18:46:33.429 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4162 2024-05-03T18:46:33.430 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4163 2024-05-03T18:46:33.430 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4164 2024-05-03T18:46:33.430 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4165 2024-05-03T18:46:33.430 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4166 2024-05-03T18:46:33.430 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4167 2024-05-03T18:46:33.430 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4168 2024-05-03T18:46:33.430 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4169 2024-05-03T18:46:33.430 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4170 2024-05-03T18:46:33.430 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4171 2024-05-03T18:46:33.430 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4172 2024-05-03T18:46:33.430 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4173 2024-05-03T18:46:33.430 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4174 2024-05-03T18:46:33.431 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 5002 2024-05-03T18:46:33.431 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 548 2024-05-03T18:46:33.431 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 549 2024-05-03T18:46:33.431 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 6102 2024-05-03T18:46:33.431 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 6104 2024-05-03T18:46:33.431 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 6106 2024-05-03T18:46:33.431 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 6110 2024-05-03T18:46:33.431 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 6111 2024-05-03T18:46:33.431 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 6112 2024-05-03T18:46:33.431 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 6113 2024-05-03T18:46:33.431 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 6117 2024-05-03T18:46:33.431 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 826 2024-05-03T18:46:33.432 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 827 2024-05-03T18:46:33.432 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 828 2024-05-03T18:46:33.432 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 May 3 18:27 pg_filenode.map 2024-05-03T18:46:33.432 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 3 18:27 PG_VERSION 2024-05-03T18:46:33.432 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:46:33.432 INFO:teuthology.orchestra.run.smithi033.stdout:/tmp/cephfs/postgres/data/base/13493: 2024-05-03T18:46:33.432 INFO:teuthology.orchestra.run.smithi033.stdout:total 7778 2024-05-03T18:46:33.432 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7963139 May 3 18:27 . 2024-05-03T18:46:33.432 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 575411765 May 3 18:30 .. 2024-05-03T18:46:33.432 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 112 2024-05-03T18:46:33.432 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 113 2024-05-03T18:46:33.432 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 3 18:27 1247 2024-05-03T18:46:33.432 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 1247_fsm 2024-05-03T18:46:33.433 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 1247_vm 2024-05-03T18:46:33.433 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 May 3 18:27 1249 2024-05-03T18:46:33.433 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 1249_fsm 2024-05-03T18:46:33.433 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 1249_vm 2024-05-03T18:46:33.433 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 May 3 18:27 1255 2024-05-03T18:46:33.433 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 1255_fsm 2024-05-03T18:46:33.433 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 1255_vm 2024-05-03T18:46:33.433 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 May 3 18:27 1259 2024-05-03T18:46:33.433 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 1259_fsm 2024-05-03T18:46:33.433 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 1259_vm 2024-05-03T18:46:33.433 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 May 3 18:27 13344 2024-05-03T18:46:33.433 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 13344_fsm 2024-05-03T18:46:33.433 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 13344_vm 2024-05-03T18:46:33.434 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 13346 2024-05-03T18:46:33.434 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 13348 2024-05-03T18:46:33.434 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 13349 2024-05-03T18:46:33.434 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 13349_fsm 2024-05-03T18:46:33.434 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 13349_vm 2024-05-03T18:46:33.434 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 13351 2024-05-03T18:46:33.434 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 13353 2024-05-03T18:46:33.434 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 13354 2024-05-03T18:46:33.434 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 13354_fsm 2024-05-03T18:46:33.434 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 13354_vm 2024-05-03T18:46:33.434 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 13356 2024-05-03T18:46:33.434 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 13358 2024-05-03T18:46:33.435 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 13359 2024-05-03T18:46:33.435 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 13359_fsm 2024-05-03T18:46:33.435 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 13359_vm 2024-05-03T18:46:33.435 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 13361 2024-05-03T18:46:33.435 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 13363 2024-05-03T18:46:33.435 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 1417 2024-05-03T18:46:33.435 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 1418 2024-05-03T18:46:33.435 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 174 2024-05-03T18:46:33.435 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 175 2024-05-03T18:46:33.435 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2187 2024-05-03T18:46:33.435 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 2224 2024-05-03T18:46:33.435 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 2328 2024-05-03T18:46:33.435 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 2336 2024-05-03T18:46:33.436 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2337 2024-05-03T18:46:33.436 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2579 2024-05-03T18:46:33.436 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2600 2024-05-03T18:46:33.436 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2600_fsm 2024-05-03T18:46:33.436 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2600_vm 2024-05-03T18:46:33.436 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2601 2024-05-03T18:46:33.436 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2601_fsm 2024-05-03T18:46:33.436 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2601_vm 2024-05-03T18:46:33.436 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 3 18:27 2602 2024-05-03T18:46:33.436 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2602_fsm 2024-05-03T18:46:33.436 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2602_vm 2024-05-03T18:46:33.436 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 3 18:27 2603 2024-05-03T18:46:33.437 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2603_fsm 2024-05-03T18:46:33.437 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2603_vm 2024-05-03T18:46:33.437 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 2604 2024-05-03T18:46:33.437 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2605 2024-05-03T18:46:33.437 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2605_fsm 2024-05-03T18:46:33.437 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2605_vm 2024-05-03T18:46:33.437 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2606 2024-05-03T18:46:33.437 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2606_fsm 2024-05-03T18:46:33.437 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2606_vm 2024-05-03T18:46:33.437 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2607 2024-05-03T18:46:33.437 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2607_fsm 2024-05-03T18:46:33.437 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2607_vm 2024-05-03T18:46:33.437 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 May 3 18:27 2608 2024-05-03T18:46:33.438 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2608_fsm 2024-05-03T18:46:33.438 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2608_vm 2024-05-03T18:46:33.438 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 May 3 18:27 2609 2024-05-03T18:46:33.438 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2609_fsm 2024-05-03T18:46:33.438 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2609_vm 2024-05-03T18:46:33.438 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 3 18:27 2610 2024-05-03T18:46:33.438 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2610_fsm 2024-05-03T18:46:33.438 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2610_vm 2024-05-03T18:46:33.438 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 2611 2024-05-03T18:46:33.438 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2612 2024-05-03T18:46:33.438 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2612_fsm 2024-05-03T18:46:33.439 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2612_vm 2024-05-03T18:46:33.439 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 2613 2024-05-03T18:46:33.439 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2615 2024-05-03T18:46:33.439 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2615_fsm 2024-05-03T18:46:33.439 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2615_vm 2024-05-03T18:46:33.439 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2616 2024-05-03T18:46:33.439 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2616_fsm 2024-05-03T18:46:33.439 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2616_vm 2024-05-03T18:46:33.439 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 May 3 18:27 2617 2024-05-03T18:46:33.439 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2617_fsm 2024-05-03T18:46:33.439 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2617_vm 2024-05-03T18:46:33.439 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 May 3 18:27 2618 2024-05-03T18:46:33.440 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2618_fsm 2024-05-03T18:46:33.440 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2618_vm 2024-05-03T18:46:33.440 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 May 3 18:27 2619 2024-05-03T18:46:33.440 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2619_fsm 2024-05-03T18:46:33.440 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2619_vm 2024-05-03T18:46:33.440 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 2620 2024-05-03T18:46:33.440 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2650 2024-05-03T18:46:33.440 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2651 2024-05-03T18:46:33.440 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2652 2024-05-03T18:46:33.440 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 3 18:27 2653 2024-05-03T18:46:33.440 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 3 18:27 2654 2024-05-03T18:46:33.440 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 3 18:27 2655 2024-05-03T18:46:33.440 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2656 2024-05-03T18:46:33.441 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2657 2024-05-03T18:46:33.441 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 May 3 18:27 2658 2024-05-03T18:46:33.441 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 3 18:27 2659 2024-05-03T18:46:33.441 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2660 2024-05-03T18:46:33.441 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2661 2024-05-03T18:46:33.441 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 3 18:27 2662 2024-05-03T18:46:33.441 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 3 18:27 2663 2024-05-03T18:46:33.441 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2664 2024-05-03T18:46:33.441 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2665 2024-05-03T18:46:33.441 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2666 2024-05-03T18:46:33.441 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2667 2024-05-03T18:46:33.441 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2668 2024-05-03T18:46:33.441 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2669 2024-05-03T18:46:33.442 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2670 2024-05-03T18:46:33.442 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 May 3 18:27 2673 2024-05-03T18:46:33.442 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 May 3 18:27 2674 2024-05-03T18:46:33.442 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 May 3 18:27 2675 2024-05-03T18:46:33.442 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2678 2024-05-03T18:46:33.442 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2679 2024-05-03T18:46:33.442 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2680 2024-05-03T18:46:33.442 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2681 2024-05-03T18:46:33.442 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2682 2024-05-03T18:46:33.442 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2683 2024-05-03T18:46:33.442 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2684 2024-05-03T18:46:33.442 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2685 2024-05-03T18:46:33.443 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2686 2024-05-03T18:46:33.443 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2687 2024-05-03T18:46:33.443 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 3 18:27 2688 2024-05-03T18:46:33.443 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 3 18:27 2689 2024-05-03T18:46:33.443 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 May 3 18:27 2690 2024-05-03T18:46:33.443 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 May 3 18:27 2691 2024-05-03T18:46:33.443 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2692 2024-05-03T18:46:33.443 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2693 2024-05-03T18:46:33.443 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2696 2024-05-03T18:46:33.443 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2699 2024-05-03T18:46:33.443 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2701 2024-05-03T18:46:33.443 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2702 2024-05-03T18:46:33.444 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 3 18:27 2703 2024-05-03T18:46:33.444 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 3 18:27 2704 2024-05-03T18:46:33.444 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2753 2024-05-03T18:46:33.444 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2753_fsm 2024-05-03T18:46:33.444 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2753_vm 2024-05-03T18:46:33.444 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2754 2024-05-03T18:46:33.444 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2755 2024-05-03T18:46:33.444 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 3 18:27 2756 2024-05-03T18:46:33.444 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 3 18:27 2757 2024-05-03T18:46:33.444 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 2830 2024-05-03T18:46:33.444 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2831 2024-05-03T18:46:33.445 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 2832 2024-05-03T18:46:33.445 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2833 2024-05-03T18:46:33.445 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 2834 2024-05-03T18:46:33.445 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2835 2024-05-03T18:46:33.445 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 2836 2024-05-03T18:46:33.445 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2837 2024-05-03T18:46:33.445 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 May 3 18:27 2838 2024-05-03T18:46:33.445 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2838_fsm 2024-05-03T18:46:33.445 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2838_vm 2024-05-03T18:46:33.445 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2839 2024-05-03T18:46:33.445 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2840 2024-05-03T18:46:33.445 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2840_fsm 2024-05-03T18:46:33.446 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2840_vm 2024-05-03T18:46:33.446 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2841 2024-05-03T18:46:33.446 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 2995 2024-05-03T18:46:33.446 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2996 2024-05-03T18:46:33.446 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3079 2024-05-03T18:46:33.446 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 3079_fsm 2024-05-03T18:46:33.446 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3079_vm 2024-05-03T18:46:33.446 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 3080 2024-05-03T18:46:33.446 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 3081 2024-05-03T18:46:33.446 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 3 18:27 3085 2024-05-03T18:46:33.446 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 3118 2024-05-03T18:46:33.446 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3119 2024-05-03T18:46:33.446 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 May 3 18:27 3164 2024-05-03T18:46:33.447 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 3256 2024-05-03T18:46:33.447 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3257 2024-05-03T18:46:33.447 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3258 2024-05-03T18:46:33.447 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 3350 2024-05-03T18:46:33.447 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3351 2024-05-03T18:46:33.447 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3379 2024-05-03T18:46:33.447 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3380 2024-05-03T18:46:33.447 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 3381 2024-05-03T18:46:33.447 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 3394 2024-05-03T18:46:33.447 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 3394_fsm 2024-05-03T18:46:33.447 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3394_vm 2024-05-03T18:46:33.447 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 3395 2024-05-03T18:46:33.447 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 3429 2024-05-03T18:46:33.448 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 3430 2024-05-03T18:46:33.448 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3431 2024-05-03T18:46:33.448 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3433 2024-05-03T18:46:33.448 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 3439 2024-05-03T18:46:33.448 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3440 2024-05-03T18:46:33.448 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 3 18:27 3455 2024-05-03T18:46:33.448 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 May 3 18:27 3456 2024-05-03T18:46:33.448 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 3456_fsm 2024-05-03T18:46:33.448 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3456_vm 2024-05-03T18:46:33.448 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 3466 2024-05-03T18:46:33.448 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3467 2024-05-03T18:46:33.448 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3468 2024-05-03T18:46:33.448 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 3501 2024-05-03T18:46:33.449 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3502 2024-05-03T18:46:33.449 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3503 2024-05-03T18:46:33.449 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3534 2024-05-03T18:46:33.449 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3541 2024-05-03T18:46:33.449 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 3541_fsm 2024-05-03T18:46:33.449 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3541_vm 2024-05-03T18:46:33.449 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 3542 2024-05-03T18:46:33.449 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3574 2024-05-03T18:46:33.449 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3575 2024-05-03T18:46:33.449 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 3576 2024-05-03T18:46:33.449 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 3596 2024-05-03T18:46:33.449 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3597 2024-05-03T18:46:33.450 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 3598 2024-05-03T18:46:33.450 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3599 2024-05-03T18:46:33.450 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3600 2024-05-03T18:46:33.450 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 3600_fsm 2024-05-03T18:46:33.450 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3600_vm 2024-05-03T18:46:33.450 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3601 2024-05-03T18:46:33.450 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 3601_fsm 2024-05-03T18:46:33.450 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3601_vm 2024-05-03T18:46:33.450 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3602 2024-05-03T18:46:33.450 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 3602_fsm 2024-05-03T18:46:33.450 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3602_vm 2024-05-03T18:46:33.450 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 3603 2024-05-03T18:46:33.451 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 3603_fsm 2024-05-03T18:46:33.451 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3603_vm 2024-05-03T18:46:33.451 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 3604 2024-05-03T18:46:33.451 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 3605 2024-05-03T18:46:33.451 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 3606 2024-05-03T18:46:33.451 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 3607 2024-05-03T18:46:33.451 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 3608 2024-05-03T18:46:33.451 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 3 18:27 3609 2024-05-03T18:46:33.451 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 3712 2024-05-03T18:46:33.451 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3764 2024-05-03T18:46:33.451 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 3764_fsm 2024-05-03T18:46:33.451 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3764_vm 2024-05-03T18:46:33.452 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 3766 2024-05-03T18:46:33.452 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 3767 2024-05-03T18:46:33.452 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3997 2024-05-03T18:46:33.452 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4143 2024-05-03T18:46:33.452 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4144 2024-05-03T18:46:33.452 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4145 2024-05-03T18:46:33.452 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4146 2024-05-03T18:46:33.452 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4147 2024-05-03T18:46:33.452 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4148 2024-05-03T18:46:33.452 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4149 2024-05-03T18:46:33.452 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4150 2024-05-03T18:46:33.452 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4151 2024-05-03T18:46:33.452 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4152 2024-05-03T18:46:33.453 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4153 2024-05-03T18:46:33.476 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:46:33.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:46:33 smithi033 ceph-mon[28330]: pgmap v611: 129 pgs: 129 active+clean; 3.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 4.6 MiB/s wr, 436 op/s 2024-05-03T18:46:34.047 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:46:33 smithi099 ceph-mon[31658]: pgmap v611: 129 pgs: 129 active+clean; 3.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 4.6 MiB/s wr, 436 op/s 2024-05-03T18:46:34.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:46:33 smithi150 ceph-mon[32492]: pgmap v611: 129 pgs: 129 active+clean; 3.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 4.6 MiB/s wr, 436 op/s 2024-05-03T18:46:35.196 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4154 2024-05-03T18:46:35.196 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4155 2024-05-03T18:46:35.196 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4156 2024-05-03T18:46:35.196 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4157 2024-05-03T18:46:35.196 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4158 2024-05-03T18:46:35.196 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4159 2024-05-03T18:46:35.196 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4160 2024-05-03T18:46:35.196 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4161 2024-05-03T18:46:35.196 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4162 2024-05-03T18:46:35.196 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4163 2024-05-03T18:46:35.197 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4164 2024-05-03T18:46:35.197 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4165 2024-05-03T18:46:35.197 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4166 2024-05-03T18:46:35.197 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4167 2024-05-03T18:46:35.197 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4168 2024-05-03T18:46:35.197 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4169 2024-05-03T18:46:35.197 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4170 2024-05-03T18:46:35.197 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4171 2024-05-03T18:46:35.197 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4172 2024-05-03T18:46:35.197 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4173 2024-05-03T18:46:35.197 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4174 2024-05-03T18:46:35.197 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 5002 2024-05-03T18:46:35.197 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 548 2024-05-03T18:46:35.197 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 549 2024-05-03T18:46:35.197 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 6102 2024-05-03T18:46:35.198 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 6104 2024-05-03T18:46:35.198 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 6106 2024-05-03T18:46:35.198 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 6110 2024-05-03T18:46:35.198 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 6111 2024-05-03T18:46:35.198 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 6112 2024-05-03T18:46:35.198 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 6113 2024-05-03T18:46:35.198 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 6117 2024-05-03T18:46:35.198 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 826 2024-05-03T18:46:35.198 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 827 2024-05-03T18:46:35.198 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 828 2024-05-03T18:46:35.198 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 May 3 18:27 pg_filenode.map 2024-05-03T18:46:35.198 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 3 18:27 PG_VERSION 2024-05-03T18:46:35.198 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:46:35.199 INFO:teuthology.orchestra.run.smithi033.stdout:/tmp/cephfs/postgres/data/base/13494: 2024-05-03T18:46:35.199 INFO:teuthology.orchestra.run.smithi033.stdout:total 546375 2024-05-03T18:46:35.199 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 559485487 May 3 18:35 . 2024-05-03T18:46:35.199 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 575411765 May 3 18:30 .. 2024-05-03T18:46:35.199 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 112 2024-05-03T18:46:35.199 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 113 2024-05-03T18:46:35.199 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 3 18:31 1247 2024-05-03T18:46:35.199 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 1247_fsm 2024-05-03T18:46:35.199 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:31 1247_vm 2024-05-03T18:46:35.199 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 May 3 18:31 1249 2024-05-03T18:46:35.199 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:31 1249_fsm 2024-05-03T18:46:35.199 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:31 1249_vm 2024-05-03T18:46:35.199 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 May 3 18:27 1255 2024-05-03T18:46:35.200 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 1255_fsm 2024-05-03T18:46:35.200 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 1255_vm 2024-05-03T18:46:35.200 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 May 3 18:46 1259 2024-05-03T18:46:35.200 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 1259_fsm 2024-05-03T18:46:35.200 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:31 1259_vm 2024-05-03T18:46:35.200 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 May 3 18:27 13344 2024-05-03T18:46:35.200 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 13344_fsm 2024-05-03T18:46:35.200 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 13344_vm 2024-05-03T18:46:35.200 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 13346 2024-05-03T18:46:35.200 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 13348 2024-05-03T18:46:35.200 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 13349 2024-05-03T18:46:35.200 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 13349_fsm 2024-05-03T18:46:35.200 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 13349_vm 2024-05-03T18:46:35.200 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 13351 2024-05-03T18:46:35.201 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 13353 2024-05-03T18:46:35.201 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 13354 2024-05-03T18:46:35.201 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 13354_fsm 2024-05-03T18:46:35.201 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 13354_vm 2024-05-03T18:46:35.201 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 13356 2024-05-03T18:46:35.201 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 13358 2024-05-03T18:46:35.201 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 13359 2024-05-03T18:46:35.201 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 13359_fsm 2024-05-03T18:46:35.201 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 13359_vm 2024-05-03T18:46:35.201 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 13361 2024-05-03T18:46:35.201 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 13363 2024-05-03T18:46:35.201 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 1417 2024-05-03T18:46:35.201 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 1418 2024-05-03T18:46:35.201 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 443219968 May 3 18:46 16396 2024-05-03T18:46:35.202 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 May 3 18:45 16396_fsm 2024-05-03T18:46:35.202 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:45 16396_vm 2024-05-03T18:46:35.202 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3194880 May 3 18:46 16397 2024-05-03T18:46:35.202 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:46 16397_fsm 2024-05-03T18:46:35.202 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:46 16397_vm 2024-05-03T18:46:35.202 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4153344 May 3 18:46 16399 2024-05-03T18:46:35.202 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:46 16399_fsm 2024-05-03T18:46:35.202 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:46 16399_vm 2024-05-03T18:46:35.202 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:46 16400 2024-05-03T18:46:35.202 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 May 3 18:46 16402 2024-05-03T18:46:35.202 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 71901184 May 3 18:46 16404 2024-05-03T18:46:35.202 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 28540928 May 3 18:46 16406 2024-05-03T18:46:35.202 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:46 16406_fsm 2024-05-03T18:46:35.203 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:43 16406_vm 2024-05-03T18:46:35.203 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 174 2024-05-03T18:46:35.203 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 175 2024-05-03T18:46:35.203 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2187 2024-05-03T18:46:35.203 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 2224 2024-05-03T18:46:35.203 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 2328 2024-05-03T18:46:35.203 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 2336 2024-05-03T18:46:35.203 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2337 2024-05-03T18:46:35.203 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:31 2579 2024-05-03T18:46:35.203 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2600 2024-05-03T18:46:35.203 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2600_fsm 2024-05-03T18:46:35.203 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2600_vm 2024-05-03T18:46:35.203 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2601 2024-05-03T18:46:35.204 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2601_fsm 2024-05-03T18:46:35.204 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2601_vm 2024-05-03T18:46:35.204 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 3 18:27 2602 2024-05-03T18:46:35.204 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2602_fsm 2024-05-03T18:46:35.204 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2602_vm 2024-05-03T18:46:35.204 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 3 18:27 2603 2024-05-03T18:46:35.204 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2603_fsm 2024-05-03T18:46:35.204 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2603_vm 2024-05-03T18:46:35.204 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 2604 2024-05-03T18:46:35.204 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2605 2024-05-03T18:46:35.204 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2605_fsm 2024-05-03T18:46:35.204 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2605_vm 2024-05-03T18:46:35.204 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:31 2606 2024-05-03T18:46:35.204 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2606_fsm 2024-05-03T18:46:35.205 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:31 2606_vm 2024-05-03T18:46:35.205 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2607 2024-05-03T18:46:35.205 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2607_fsm 2024-05-03T18:46:35.205 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2607_vm 2024-05-03T18:46:35.205 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 May 3 18:31 2608 2024-05-03T18:46:35.205 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2608_fsm 2024-05-03T18:46:35.205 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:31 2608_vm 2024-05-03T18:46:35.205 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 May 3 18:27 2609 2024-05-03T18:46:35.205 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2609_fsm 2024-05-03T18:46:35.205 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2609_vm 2024-05-03T18:46:35.205 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 3 18:31 2610 2024-05-03T18:46:35.205 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2610_fsm 2024-05-03T18:46:35.205 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:31 2610_vm 2024-05-03T18:46:35.205 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 2611 2024-05-03T18:46:35.206 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2612 2024-05-03T18:46:35.206 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2612_fsm 2024-05-03T18:46:35.206 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2612_vm 2024-05-03T18:46:35.206 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 2613 2024-05-03T18:46:35.206 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2615 2024-05-03T18:46:35.206 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2615_fsm 2024-05-03T18:46:35.206 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2615_vm 2024-05-03T18:46:35.206 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2616 2024-05-03T18:46:35.206 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2616_fsm 2024-05-03T18:46:35.206 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2616_vm 2024-05-03T18:46:35.206 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 May 3 18:27 2617 2024-05-03T18:46:35.206 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2617_fsm 2024-05-03T18:46:35.206 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2617_vm 2024-05-03T18:46:35.206 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 May 3 18:27 2618 2024-05-03T18:46:35.207 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2618_fsm 2024-05-03T18:46:35.207 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2618_vm 2024-05-03T18:46:35.207 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 172032 May 3 18:46 2619 2024-05-03T18:46:35.207 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:33 2619_fsm 2024-05-03T18:46:35.207 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:31 2619_vm 2024-05-03T18:46:35.207 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 2620 2024-05-03T18:46:35.207 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2650 2024-05-03T18:46:35.207 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2651 2024-05-03T18:46:35.207 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2652 2024-05-03T18:46:35.207 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 3 18:27 2653 2024-05-03T18:46:35.207 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 3 18:27 2654 2024-05-03T18:46:35.207 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 3 18:27 2655 2024-05-03T18:46:35.207 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2656 2024-05-03T18:46:35.207 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2657 2024-05-03T18:46:35.207 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 May 3 18:31 2658 2024-05-03T18:46:35.208 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 3 18:31 2659 2024-05-03T18:46:35.208 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2660 2024-05-03T18:46:35.208 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2661 2024-05-03T18:46:35.208 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 3 18:31 2662 2024-05-03T18:46:35.208 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 3 18:31 2663 2024-05-03T18:46:35.208 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:31 2664 2024-05-03T18:46:35.208 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:31 2665 2024-05-03T18:46:35.208 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:31 2666 2024-05-03T18:46:35.208 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:31 2667 2024-05-03T18:46:35.208 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2668 2024-05-03T18:46:35.208 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2669 2024-05-03T18:46:35.208 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2670 2024-05-03T18:46:35.208 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 May 3 18:31 2673 2024-05-03T18:46:35.208 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 May 3 18:31 2674 2024-05-03T18:46:35.209 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 May 3 18:27 2675 2024-05-03T18:46:35.209 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:31 2678 2024-05-03T18:46:35.209 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:31 2679 2024-05-03T18:46:35.209 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2680 2024-05-03T18:46:35.209 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2681 2024-05-03T18:46:35.209 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2682 2024-05-03T18:46:35.209 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2683 2024-05-03T18:46:35.209 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2684 2024-05-03T18:46:35.209 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2685 2024-05-03T18:46:35.209 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2686 2024-05-03T18:46:35.209 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2687 2024-05-03T18:46:35.210 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 3 18:27 2688 2024-05-03T18:46:35.210 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 3 18:27 2689 2024-05-03T18:46:35.210 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 May 3 18:27 2690 2024-05-03T18:46:35.210 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 May 3 18:27 2691 2024-05-03T18:46:35.210 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2692 2024-05-03T18:46:35.210 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2693 2024-05-03T18:46:35.210 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 3 18:45 2696 2024-05-03T18:46:35.210 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2699 2024-05-03T18:46:35.210 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2701 2024-05-03T18:46:35.210 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2702 2024-05-03T18:46:35.210 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 3 18:31 2703 2024-05-03T18:46:35.210 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 3 18:31 2704 2024-05-03T18:46:35.210 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2753 2024-05-03T18:46:35.210 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2753_fsm 2024-05-03T18:46:35.211 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2753_vm 2024-05-03T18:46:35.211 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2754 2024-05-03T18:46:35.211 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2755 2024-05-03T18:46:35.211 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 3 18:27 2756 2024-05-03T18:46:35.211 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 3 18:27 2757 2024-05-03T18:46:35.211 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 2830 2024-05-03T18:46:35.211 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2831 2024-05-03T18:46:35.211 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 2832 2024-05-03T18:46:35.211 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2833 2024-05-03T18:46:35.211 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 2834 2024-05-03T18:46:35.211 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2835 2024-05-03T18:46:35.211 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 2836 2024-05-03T18:46:35.211 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2837 2024-05-03T18:46:35.212 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 May 3 18:27 2838 2024-05-03T18:46:35.212 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2838_fsm 2024-05-03T18:46:35.212 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2838_vm 2024-05-03T18:46:35.212 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2839 2024-05-03T18:46:35.212 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2840 2024-05-03T18:46:35.212 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2840_fsm 2024-05-03T18:46:35.212 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2840_vm 2024-05-03T18:46:35.212 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2841 2024-05-03T18:46:35.212 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 2995 2024-05-03T18:46:35.212 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2996 2024-05-03T18:46:35.212 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3079 2024-05-03T18:46:35.212 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 3079_fsm 2024-05-03T18:46:35.212 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3079_vm 2024-05-03T18:46:35.212 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 3080 2024-05-03T18:46:35.213 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 3081 2024-05-03T18:46:35.213 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 3 18:27 3085 2024-05-03T18:46:35.213 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 3118 2024-05-03T18:46:35.213 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3119 2024-05-03T18:46:35.213 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 May 3 18:27 3164 2024-05-03T18:46:35.213 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 3256 2024-05-03T18:46:35.213 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3257 2024-05-03T18:46:35.213 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3258 2024-05-03T18:46:35.213 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 3350 2024-05-03T18:46:35.213 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3351 2024-05-03T18:46:35.213 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3379 2024-05-03T18:46:35.213 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3380 2024-05-03T18:46:35.213 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 3381 2024-05-03T18:46:35.213 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 3394 2024-05-03T18:46:35.214 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 3394_fsm 2024-05-03T18:46:35.214 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3394_vm 2024-05-03T18:46:35.214 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 3395 2024-05-03T18:46:35.214 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 3429 2024-05-03T18:46:35.214 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 3430 2024-05-03T18:46:35.214 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3431 2024-05-03T18:46:35.214 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3433 2024-05-03T18:46:35.214 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 3439 2024-05-03T18:46:35.214 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3440 2024-05-03T18:46:35.214 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 3 18:31 3455 2024-05-03T18:46:35.214 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 May 3 18:27 3456 2024-05-03T18:46:35.214 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 3456_fsm 2024-05-03T18:46:35.214 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3456_vm 2024-05-03T18:46:35.215 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 3466 2024-05-03T18:46:35.215 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3467 2024-05-03T18:46:35.215 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3468 2024-05-03T18:46:35.215 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 3501 2024-05-03T18:46:35.215 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3502 2024-05-03T18:46:35.215 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3503 2024-05-03T18:46:35.215 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3534 2024-05-03T18:46:35.215 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3541 2024-05-03T18:46:35.215 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 3541_fsm 2024-05-03T18:46:35.215 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3541_vm 2024-05-03T18:46:35.215 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 3542 2024-05-03T18:46:35.215 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3574 2024-05-03T18:46:35.215 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3575 2024-05-03T18:46:35.215 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 3576 2024-05-03T18:46:35.216 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 3596 2024-05-03T18:46:35.216 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3597 2024-05-03T18:46:35.216 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 3598 2024-05-03T18:46:35.216 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3599 2024-05-03T18:46:35.216 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3600 2024-05-03T18:46:35.216 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 3600_fsm 2024-05-03T18:46:35.216 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3600_vm 2024-05-03T18:46:35.216 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3601 2024-05-03T18:46:35.216 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 3601_fsm 2024-05-03T18:46:35.216 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3601_vm 2024-05-03T18:46:35.216 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3602 2024-05-03T18:46:35.216 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 3602_fsm 2024-05-03T18:46:35.216 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3602_vm 2024-05-03T18:46:35.217 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 3603 2024-05-03T18:46:35.217 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 3603_fsm 2024-05-03T18:46:35.217 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3603_vm 2024-05-03T18:46:35.217 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 3604 2024-05-03T18:46:35.217 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 3605 2024-05-03T18:46:35.217 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 3606 2024-05-03T18:46:35.217 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 3607 2024-05-03T18:46:35.217 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 3608 2024-05-03T18:46:35.217 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 3 18:27 3609 2024-05-03T18:46:35.217 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 3712 2024-05-03T18:46:35.217 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3764 2024-05-03T18:46:35.217 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 3764_fsm 2024-05-03T18:46:35.217 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3764_vm 2024-05-03T18:46:35.217 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 3766 2024-05-03T18:46:35.218 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 3767 2024-05-03T18:46:35.218 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3997 2024-05-03T18:46:35.218 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4143 2024-05-03T18:46:35.218 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4144 2024-05-03T18:46:35.218 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4145 2024-05-03T18:46:35.218 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4146 2024-05-03T18:46:35.218 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4147 2024-05-03T18:46:35.218 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4148 2024-05-03T18:46:35.218 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4149 2024-05-03T18:46:35.218 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4150 2024-05-03T18:46:35.218 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4151 2024-05-03T18:46:35.218 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4152 2024-05-03T18:46:35.218 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4153 2024-05-03T18:46:35.219 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4154 2024-05-03T18:46:35.219 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4155 2024-05-03T18:46:35.219 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4156 2024-05-03T18:46:35.219 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4157 2024-05-03T18:46:35.219 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4158 2024-05-03T18:46:35.219 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4159 2024-05-03T18:46:35.219 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4160 2024-05-03T18:46:35.219 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4161 2024-05-03T18:46:35.219 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4162 2024-05-03T18:46:35.219 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4163 2024-05-03T18:46:35.219 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4164 2024-05-03T18:46:35.219 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4165 2024-05-03T18:46:35.219 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4166 2024-05-03T18:46:35.220 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4167 2024-05-03T18:46:35.220 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4168 2024-05-03T18:46:35.220 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4169 2024-05-03T18:46:35.220 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4170 2024-05-03T18:46:35.220 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4171 2024-05-03T18:46:35.220 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4172 2024-05-03T18:46:35.220 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4173 2024-05-03T18:46:35.220 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4174 2024-05-03T18:46:35.220 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 5002 2024-05-03T18:46:35.220 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 548 2024-05-03T18:46:35.220 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 549 2024-05-03T18:46:35.220 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 6102 2024-05-03T18:46:35.220 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 6104 2024-05-03T18:46:35.220 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 6106 2024-05-03T18:46:35.220 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 6110 2024-05-03T18:46:35.515 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 6111 2024-05-03T18:46:35.515 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 6112 2024-05-03T18:46:35.515 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 6113 2024-05-03T18:46:35.515 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 6117 2024-05-03T18:46:35.515 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 826 2024-05-03T18:46:35.515 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 827 2024-05-03T18:46:35.515 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 828 2024-05-03T18:46:35.516 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 May 3 18:27 pg_filenode.map 2024-05-03T18:46:35.516 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 151596 May 3 18:28 pg_internal.init 2024-05-03T18:46:35.516 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 3 18:27 PG_VERSION 2024-05-03T18:46:35.516 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:46:35.516 INFO:teuthology.orchestra.run.smithi033.stdout:/tmp/cephfs/postgres/data/base/pgsql_tmp: 2024-05-03T18:46:35.516 INFO:teuthology.orchestra.run.smithi033.stdout:total 1 2024-05-03T18:46:35.516 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:31 . 2024-05-03T18:46:35.516 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 575411765 May 3 18:30 .. 2024-05-03T18:46:35.516 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:46:35.516 INFO:teuthology.orchestra.run.smithi033.stdout:/tmp/cephfs/postgres/data/global: 2024-05-03T18:46:35.516 INFO:teuthology.orchestra.run.smithi033.stdout:total 561 2024-05-03T18:46:35.516 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 572520 May 3 18:28 . 2024-05-03T18:46:35.516 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1482306596 May 3 18:46 .. 2024-05-03T18:46:35.516 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 1213 2024-05-03T18:46:35.517 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 1213_fsm 2024-05-03T18:46:35.517 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 1213_vm 2024-05-03T18:46:35.517 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 1214 2024-05-03T18:46:35.517 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 1214_fsm 2024-05-03T18:46:35.517 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 1214_vm 2024-05-03T18:46:35.517 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 1232 2024-05-03T18:46:35.517 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 1233 2024-05-03T18:46:35.517 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 1260 2024-05-03T18:46:35.517 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 1260_fsm 2024-05-03T18:46:35.517 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 1260_vm 2024-05-03T18:46:35.517 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 1261 2024-05-03T18:46:35.517 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 1261_fsm 2024-05-03T18:46:35.517 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 1261_vm 2024-05-03T18:46:35.517 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 1262 2024-05-03T18:46:35.518 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 1262_fsm 2024-05-03T18:46:35.518 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 1262_vm 2024-05-03T18:46:35.518 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2396 2024-05-03T18:46:35.518 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 3 18:27 2396_fsm 2024-05-03T18:46:35.518 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2396_vm 2024-05-03T18:46:35.518 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2397 2024-05-03T18:46:35.518 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2671 2024-05-03T18:46:35.518 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2672 2024-05-03T18:46:35.518 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2676 2024-05-03T18:46:35.518 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2677 2024-05-03T18:46:35.518 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2694 2024-05-03T18:46:35.593 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2695 2024-05-03T18:46:35.593 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2697 2024-05-03T18:46:35.594 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 2698 2024-05-03T18:46:35.594 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 2846 2024-05-03T18:46:35.594 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2847 2024-05-03T18:46:35.594 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 2964 2024-05-03T18:46:35.594 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2965 2024-05-03T18:46:35.594 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 2966 2024-05-03T18:46:35.594 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 2967 2024-05-03T18:46:35.594 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 3592 2024-05-03T18:46:35.594 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 3593 2024-05-03T18:46:35.594 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4060 2024-05-03T18:46:35.594 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4061 2024-05-03T18:46:35.594 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4175 2024-05-03T18:46:35.594 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4176 2024-05-03T18:46:35.594 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4177 2024-05-03T18:46:35.595 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4178 2024-05-03T18:46:35.595 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4181 2024-05-03T18:46:35.595 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4182 2024-05-03T18:46:35.595 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4183 2024-05-03T18:46:35.595 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4184 2024-05-03T18:46:35.595 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 4185 2024-05-03T18:46:35.595 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 4186 2024-05-03T18:46:35.595 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 6000 2024-05-03T18:46:35.595 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 6001 2024-05-03T18:46:35.595 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 6002 2024-05-03T18:46:35.595 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 6100 2024-05-03T18:46:35.595 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 6114 2024-05-03T18:46:35.595 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 6115 2024-05-03T18:46:35.595 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:46 pg_control 2024-05-03T18:46:35.595 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 May 3 18:27 pg_filenode.map 2024-05-03T18:46:35.596 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 23144 May 3 18:28 pg_internal.init 2024-05-03T18:46:35.596 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:46:35.596 INFO:teuthology.orchestra.run.smithi033.stdout:/tmp/cephfs/postgres/data/log: 2024-05-03T18:46:35.596 INFO:teuthology.orchestra.run.smithi033.stdout:total 3 2024-05-03T18:46:35.596 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1047 May 3 18:28 . 2024-05-03T18:46:35.596 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1482306596 May 3 18:46 .. 2024-05-03T18:46:35.596 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1118 May 3 18:46 postgresql-Fri.log 2024-05-03T18:46:35.596 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:46:35.596 INFO:teuthology.orchestra.run.smithi033.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2024-05-03T18:46:35.596 INFO:teuthology.orchestra.run.smithi033.stdout:total 1 2024-05-03T18:46:35.596 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 . 2024-05-03T18:46:35.596 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1482306596 May 3 18:46 .. 2024-05-03T18:46:35.596 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:46:35.596 INFO:teuthology.orchestra.run.smithi033.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2024-05-03T18:46:35.596 INFO:teuthology.orchestra.run.smithi033.stdout:total 1 2024-05-03T18:46:35.597 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 . 2024-05-03T18:46:35.597 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1482306596 May 3 18:46 .. 2024-05-03T18:46:35.597 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:46:35.597 INFO:teuthology.orchestra.run.smithi033.stdout:/tmp/cephfs/postgres/data/pg_logical: 2024-05-03T18:46:35.597 INFO:teuthology.orchestra.run.smithi033.stdout:total 3 2024-05-03T18:46:35.839 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 May 3 18:46 . 2024-05-03T18:46:35.839 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1482306596 May 3 18:46 .. 2024-05-03T18:46:35.839 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 mappings 2024-05-03T18:46:35.839 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 May 3 18:46 replorigin_checkpoint 2024-05-03T18:46:35.839 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 snapshots 2024-05-03T18:46:35.839 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:46:35.839 INFO:teuthology.orchestra.run.smithi033.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2024-05-03T18:46:35.839 INFO:teuthology.orchestra.run.smithi033.stdout:total 1 2024-05-03T18:46:35.839 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 . 2024-05-03T18:46:35.839 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 May 3 18:46 .. 2024-05-03T18:46:35.839 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:46:35.840 INFO:teuthology.orchestra.run.smithi033.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2024-05-03T18:46:35.840 INFO:teuthology.orchestra.run.smithi033.stdout:total 1 2024-05-03T18:46:35.840 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 . 2024-05-03T18:46:35.840 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 May 3 18:46 .. 2024-05-03T18:46:35.840 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:46:35.840 INFO:teuthology.orchestra.run.smithi033.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2024-05-03T18:46:35.840 INFO:teuthology.orchestra.run.smithi033.stdout:total 2 2024-05-03T18:46:35.840 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 . 2024-05-03T18:46:35.840 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1482306596 May 3 18:46 .. 2024-05-03T18:46:35.840 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 members 2024-05-03T18:46:35.840 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 offsets 2024-05-03T18:46:35.840 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:46:35.840 INFO:teuthology.orchestra.run.smithi033.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2024-05-03T18:46:35.840 INFO:teuthology.orchestra.run.smithi033.stdout:total 9 2024-05-03T18:46:35.840 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 . 2024-05-03T18:46:35.841 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 .. 2024-05-03T18:46:35.841 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 0000 2024-05-03T18:46:35.841 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:46:35.841 INFO:teuthology.orchestra.run.smithi033.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2024-05-03T18:46:35.841 INFO:teuthology.orchestra.run.smithi033.stdout:total 9 2024-05-03T18:46:35.841 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:27 . 2024-05-03T18:46:35.841 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 3 18:27 .. 2024-05-03T18:46:35.841 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 3 18:33 0000 2024-05-03T18:46:35.841 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:46:35.841 INFO:teuthology.orchestra.run.smithi033.stdout:/tmp/cephfs/postgres/data/pg_notify: 2024-05-03T18:46:35.841 INFO:teuthology.orchestra.run.smithi033.stdout:total 1 2024-05-03T18:46:35.841 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 . 2024-05-03T18:46:35.841 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1482306596 May 3 18:46 .. 2024-05-03T18:46:35.841 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:46:35.841 INFO:teuthology.orchestra.run.smithi033.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2024-05-03T18:46:35.842 INFO:teuthology.orchestra.run.smithi033.stdout:total 1 2024-05-03T18:46:35.842 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 . 2024-05-03T18:46:35.842 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1482306596 May 3 18:46 .. 2024-05-03T18:46:35.842 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:46:35.842 INFO:teuthology.orchestra.run.smithi033.stdout:/tmp/cephfs/postgres/data/pg_serial: 2024-05-03T18:46:35.842 INFO:teuthology.orchestra.run.smithi033.stdout:total 1 2024-05-03T18:46:35.842 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 . 2024-05-03T18:46:35.842 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1482306596 May 3 18:46 .. 2024-05-03T18:46:35.842 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:46:35.842 INFO:teuthology.orchestra.run.smithi033.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2024-05-03T18:46:35.842 INFO:teuthology.orchestra.run.smithi033.stdout:total 1 2024-05-03T18:46:35.842 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 . 2024-05-03T18:46:35.842 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1482306596 May 3 18:46 .. 2024-05-03T18:46:35.843 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:46:35.843 INFO:teuthology.orchestra.run.smithi033.stdout:/tmp/cephfs/postgres/data/pg_stat: 2024-05-03T18:46:35.843 INFO:teuthology.orchestra.run.smithi033.stdout:total 16 2024-05-03T18:46:35.843 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 14082 May 3 18:46 . 2024-05-03T18:46:35.843 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1482306596 May 3 18:46 .. 2024-05-03T18:46:35.843 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1952 May 3 18:46 db_0.stat 2024-05-03T18:46:35.843 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 10979 May 3 18:46 db_13494.stat 2024-05-03T18:46:35.843 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1151 May 3 18:46 global.stat 2024-05-03T18:46:35.843 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:46:35.843 INFO:teuthology.orchestra.run.smithi033.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2024-05-03T18:46:35.843 INFO:teuthology.orchestra.run.smithi033.stdout:total 1 2024-05-03T18:46:35.843 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:46 . 2024-05-03T18:46:35.843 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1482306596 May 3 18:46 .. 2024-05-03T18:46:35.843 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:46:35.844 INFO:teuthology.orchestra.run.smithi033.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2024-05-03T18:46:35.844 INFO:teuthology.orchestra.run.smithi033.stdout:total 153 2024-05-03T18:46:35.844 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 155648 May 3 18:45 . 2024-05-03T18:46:35.844 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1482306596 May 3 18:46 .. 2024-05-03T18:46:35.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.864+0000 7fbee7e00700 1 -- 172.21.15.33:0/1307456691 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbee006bb40 msgr2=0x7fbee0110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:35.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.864+0000 7fbee7e00700 1 --2- 172.21.15.33:0/1307456691 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbee006bb40 0x7fbee0110330 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fbed800b0a0 tx=0x7fbed800b3b0 comp rx=0 tx=0).stop 2024-05-03T18:46:35.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.865+0000 7fbee7e00700 1 -- 172.21.15.33:0/1307456691 shutdown_connections 2024-05-03T18:46:35.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.865+0000 7fbee7e00700 1 --2- 172.21.15.33:0/1307456691 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbee006bb40 0x7fbee0110330 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:35.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.865+0000 7fbee7e00700 1 --2- 172.21.15.33:0/1307456691 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbee0109510 0x7fbee01098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:35.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.865+0000 7fbee7e00700 1 --2- 172.21.15.33:0/1307456691 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbee006b180 0x7fbee006b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:35.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.865+0000 7fbee7e00700 1 -- 172.21.15.33:0/1307456691 >> 172.21.15.33:0/1307456691 conn(0x7fbee0079350 msgr2=0x7fbee0079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:46:35.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.865+0000 7fbee7e00700 1 -- 172.21.15.33:0/1307456691 shutdown_connections 2024-05-03T18:46:35.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.865+0000 7fbee7e00700 1 -- 172.21.15.33:0/1307456691 wait complete. 2024-05-03T18:46:35.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.867+0000 7fbee7e00700 1 Processor -- start 2024-05-03T18:46:35.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.867+0000 7fbee7e00700 1 -- start start 2024-05-03T18:46:35.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.867+0000 7fbee7e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbee006b180 0x7fbee019f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:35.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.867+0000 7fbee7e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbee006bb40 0x7fbee019fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:35.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.867+0000 7fbee7e00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbee0109510 0x7fbee01998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:35.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.867+0000 7fbee7e00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbee0113520 con 0x7fbee006bb40 2024-05-03T18:46:35.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.867+0000 7fbee7e00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbee01133a0 con 0x7fbee006b180 2024-05-03T18:46:35.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.867+0000 7fbee7e00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fbee01136a0 con 0x7fbee0109510 2024-05-03T18:46:35.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.868+0000 7fbee4a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbee006bb40 0x7fbee019fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:35.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.868+0000 7fbee4a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbee006bb40 0x7fbee019fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55534/0 (socket says 172.21.15.33:55534) 2024-05-03T18:46:35.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.868+0000 7fbee4a00700 1 -- 172.21.15.33:0/2082779431 learned_addr learned my addr 172.21.15.33:0/2082779431 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:46:35.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.868+0000 7fbee5400700 1 --2- 172.21.15.33:0/2082779431 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbee006b180 0x7fbee019f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:35.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.868+0000 7fbee5e00700 1 --2- 172.21.15.33:0/2082779431 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbee0109510 0x7fbee01998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:35.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.868+0000 7fbee4a00700 1 -- 172.21.15.33:0/2082779431 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbee0109510 msgr2=0x7fbee01998a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:35.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.868+0000 7fbee4a00700 1 --2- 172.21.15.33:0/2082779431 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbee0109510 0x7fbee01998a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:35.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.868+0000 7fbee4a00700 1 -- 172.21.15.33:0/2082779431 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbee006b180 msgr2=0x7fbee019f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:35.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.868+0000 7fbee4a00700 1 --2- 172.21.15.33:0/2082779431 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbee006b180 0x7fbee019f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:35.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.868+0000 7fbee4a00700 1 -- 172.21.15.33:0/2082779431 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbee019a000 con 0x7fbee006bb40 2024-05-03T18:46:35.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.868+0000 7fbee5400700 1 --2- 172.21.15.33:0/2082779431 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbee006b180 0x7fbee019f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:46:35.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.869+0000 7fbee4a00700 1 --2- 172.21.15.33:0/2082779431 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbee006bb40 0x7fbee019fc20 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fbecc00d950 tx=0x7fbecc00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:46:35.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.869+0000 7fbede000700 1 -- 172.21.15.33:0/2082779431 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbecc016070 con 0x7fbee006bb40 2024-05-03T18:46:35.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.869+0000 7fbede000700 1 -- 172.21.15.33:0/2082779431 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbecc010460 con 0x7fbee006bb40 2024-05-03T18:46:35.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.869+0000 7fbede000700 1 -- 172.21.15.33:0/2082779431 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbecc00f620 con 0x7fbee006bb40 2024-05-03T18:46:35.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.869+0000 7fbee7e00700 1 -- 172.21.15.33:0/2082779431 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbee019a2f0 con 0x7fbee006bb40 2024-05-03T18:46:35.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.869+0000 7fbee7e00700 1 -- 172.21.15.33:0/2082779431 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbee01a65b0 con 0x7fbee006bb40 2024-05-03T18:46:35.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.873+0000 7fbede000700 1 -- 172.21.15.33:0/2082779431 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fbecc019070 con 0x7fbee006bb40 2024-05-03T18:46:35.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.874+0000 7fbef26afb80 1 Processor -- start 2024-05-03T18:46:35.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.874+0000 7fbef26afb80 1 -- start start 2024-05-03T18:46:35.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.874+0000 7fbede000700 1 --2- 172.21.15.33:0/2082779431 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fbec80b2a70 0x7fbec80b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:35.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.875+0000 7fbede000700 1 -- 172.21.15.33:0/2082779431 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9518+0+0 (secure 0 0 0) 0x7fbecc0e2590 con 0x7fbee006bb40 2024-05-03T18:46:35.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.875+0000 7fbef26afb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555a91d5f930 0x555a91d5fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:35.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.875+0000 7fbef26afb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555a91d60250 0x555a91d59f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:35.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.875+0000 7fbef26afb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555a91d5a550 0x555a91d5a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:35.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.875+0000 7fbef26afb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x555a91caca20 con 0x555a91d5a550 2024-05-03T18:46:35.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.875+0000 7fbef26afb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x555a91c4a760 con 0x555a91d5f930 2024-05-03T18:46:35.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.875+0000 7fbef26afb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x555a91ca89e0 con 0x555a91d60250 2024-05-03T18:46:35.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.875+0000 7fbee5e00700 1 --2- 172.21.15.33:0/2082779431 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fbec80b2a70 0x7fbec80b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:35.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.875+0000 7fbee5e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555a91d5a550 0x555a91d5a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:35.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.875+0000 7fbee5400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555a91d5f930 0x555a91d5fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:35.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.875+0000 7fbee5400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555a91d5f930 0x555a91d5fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.33:42326/0 (socket says 172.21.15.33:42326) 2024-05-03T18:46:35.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.875+0000 7fbee5400700 1 -- 172.21.15.33:0/4026640497 learned_addr learned my addr 172.21.15.33:0/4026640497 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:46:35.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.875+0000 7fbee4a00700 1 --2- 172.21.15.33:0/4026640497 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555a91d60250 0x555a91d59f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:35.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.876+0000 7fbee5e00700 1 -- 172.21.15.33:0/4026640497 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555a91d60250 msgr2=0x555a91d59f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:46:35.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.876+0000 7fbee5e00700 1 --2- 172.21.15.33:0/4026640497 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555a91d60250 0x555a91d59f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:35.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.876+0000 7fbee5e00700 1 -- 172.21.15.33:0/4026640497 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555a91d5f930 msgr2=0x555a91d5fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:46:35.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.876+0000 7fbee5e00700 1 --2- 172.21.15.33:0/4026640497 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555a91d5f930 0x555a91d5fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:35.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.876+0000 7fbee5e00700 1 -- 172.21.15.33:0/4026640497 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a91bf98d0 con 0x555a91d5a550 2024-05-03T18:46:35.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.876+0000 7fbee4a00700 1 --2- 172.21.15.33:0/4026640497 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555a91d60250 0x555a91d59f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:46:35.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.876+0000 7fbee5400700 1 --2- 172.21.15.33:0/4026640497 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555a91d5f930 0x555a91d5fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:46:35.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.876+0000 7fbee5e00700 1 --2- 172.21.15.33:0/2082779431 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fbec80b2a70 0x7fbec80b4f30 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fbed80062a0 tx=0x7fbed800bb90 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:46:35.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.876+0000 7fbee5e00700 1 --2- 172.21.15.33:0/4026640497 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555a91d5a550 0x555a91d5a9d0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fbed800b780 tx=0x7fbed8028000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e58bfaa97bbeef17 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:46:35.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.877+0000 7fbed2a00700 1 -- 172.21.15.33:0/4026640497 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed802f070 con 0x555a91d5a550 2024-05-03T18:46:35.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.877+0000 7fbed2a00700 1 -- 172.21.15.33:0/4026640497 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbed8004d30 con 0x555a91d5a550 2024-05-03T18:46:35.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.877+0000 7fbed2a00700 1 -- 172.21.15.33:0/4026640497 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed801faf0 con 0x555a91d5a550 2024-05-03T18:46:35.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.878+0000 7fbef26afb80 1 -- 172.21.15.33:0/4026640497 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555a91d5a550 msgr2=0x555a91d5a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:35.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.878+0000 7fbef26afb80 1 --2- 172.21.15.33:0/4026640497 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555a91d5a550 0x555a91d5a9d0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fbed800b780 tx=0x7fbed8028000 comp rx=0 tx=0).stop 2024-05-03T18:46:35.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.878+0000 7fbef26afb80 1 -- 172.21.15.33:0/4026640497 shutdown_connections 2024-05-03T18:46:35.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.878+0000 7fbef26afb80 1 --2- 172.21.15.33:0/4026640497 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555a91d5a550 0x555a91d5a9d0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:35.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.878+0000 7fbef26afb80 1 --2- 172.21.15.33:0/4026640497 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555a91d5f930 0x555a91d5fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:35.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.878+0000 7fbef26afb80 1 --2- 172.21.15.33:0/4026640497 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555a91d60250 0x555a91d59f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:35.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.878+0000 7fbef26afb80 1 -- 172.21.15.33:0/4026640497 >> 172.21.15.33:0/4026640497 conn(0x555a91cd26d0 msgr2=0x555a91d54900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:46:35.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.878+0000 7fbef26afb80 1 -- 172.21.15.33:0/4026640497 shutdown_connections 2024-05-03T18:46:35.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.878+0000 7fbef26afb80 1 -- 172.21.15.33:0/4026640497 wait complete. 2024-05-03T18:46:35.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.879+0000 7fbef26afb80 1 -- start start 2024-05-03T18:46:35.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.879+0000 7fbef26afb80 1 Processor -- start 2024-05-03T18:46:35.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.879+0000 7fbef26afb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555a91d5a550 0x555a91d65ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:35.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.879+0000 7fbef26afb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555a91d5f930 0x555a91d632b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:35.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.879+0000 7fbef26afb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555a91d60250 0x555a91d637f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:35.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.879+0000 7fbef26afb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x555a91d6b0b0 con 0x555a91d5a550 2024-05-03T18:46:35.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.879+0000 7fbef26afb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x555a91c12b20 con 0x555a91d5f930 2024-05-03T18:46:35.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.879+0000 7fbef26afb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x555a91d6b230 con 0x555a91d60250 2024-05-03T18:46:35.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.880+0000 7fbef26afb80 10 client.0 _ll_register_callbacks cb 0x555a91c53c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:46:35.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.880+0000 7fbee5400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555a91d5a550 0x555a91d65ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:35.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.880+0000 7fbee5400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555a91d5a550 0x555a91d65ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55552/0 (socket says 172.21.15.33:55552) 2024-05-03T18:46:35.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.880+0000 7fbee5400700 1 -- 172.21.15.33:0/86656029 learned_addr learned my addr 172.21.15.33:0/86656029 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:46:35.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.880+0000 7fbee4a00700 1 --2- 172.21.15.33:0/86656029 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555a91d5f930 0x555a91d632b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:35.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.880+0000 7fbee5e00700 1 --2- 172.21.15.33:0/86656029 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555a91d60250 0x555a91d637f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:35.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.881+0000 7fbee5400700 1 -- 172.21.15.33:0/86656029 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555a91d60250 msgr2=0x555a91d637f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:46:35.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.881+0000 7fbee5400700 1 --2- 172.21.15.33:0/86656029 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555a91d60250 0x555a91d637f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:35.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.881+0000 7fbee5400700 1 -- 172.21.15.33:0/86656029 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555a91d5f930 msgr2=0x555a91d632b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:46:35.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.881+0000 7fbee5400700 1 --2- 172.21.15.33:0/86656029 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555a91d5f930 0x555a91d632b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:35.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.881+0000 7fbee5400700 1 -- 172.21.15.33:0/86656029 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a91c72420 con 0x555a91d5a550 2024-05-03T18:46:35.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.881+0000 7fbee5400700 1 --2- 172.21.15.33:0/86656029 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555a91d5a550 0x555a91d65ec0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fbed4009dc0 tx=0x7fbed4003960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=245ad9837f91d31a server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:46:35.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.881+0000 7fbec7400700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:46:35.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.881+0000 7fbec7400700 1 -- 172.21.15.33:0/86656029 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed4011650 con 0x555a91d5a550 2024-05-03T18:46:35.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.881+0000 7fbec7400700 1 -- 172.21.15.33:0/86656029 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9518+0+0 (secure 0 0 0) 0x7fbed40117f0 con 0x555a91d5a550 2024-05-03T18:46:35.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.881+0000 7fbef26afb80 1 -- 172.21.15.33:0/86656029 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbed8004d30 con 0x555a91d5a550 2024-05-03T18:46:35.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.882+0000 7fbec7400700 1 -- 172.21.15.33:0/86656029 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fbed40119f0 con 0x555a91d5a550 2024-05-03T18:46:35.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.882+0000 7fbef26afb80 10 client.17058 fetch_fsmap learned FSMap version 21 2024-05-03T18:46:35.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.882+0000 7fbef26afb80 1 -- 172.21.15.33:0/86656029 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x555a91d758e0 con 0x555a91d5a550 2024-05-03T18:46:35.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.882+0000 7fbec7400700 1 -- 172.21.15.33:0/86656029 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbed8004d30 con 0x555a91d5a550 2024-05-03T18:46:35.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.882+0000 7fbec7400700 1 -- 172.21.15.33:0/86656029 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fbed401ba70 con 0x555a91d5a550 2024-05-03T18:46:35.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.882+0000 7fbef26afb80 10 client.17058 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:46:35.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.882+0000 7fbef26afb80 10 client.17058 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:46:35.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.882+0000 7fbef26afb80 20 client.17058 populate_metadata read hostname 'smithi033' 2024-05-03T18:46:35.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.883+0000 7fbef26afb80 1 --2- 172.21.15.33:0/86656029 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x555a91d77f20 0x555a91d7a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:35.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.883+0000 7fbef26afb80 4 client.17058 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:46:35.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.883+0000 7fbef26afb80 1 -- 172.21.15.33:0/86656029 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555a91d26ff0 con 0x555a91d77f20 2024-05-03T18:46:35.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.883+0000 7fbee4a00700 1 --2- 172.21.15.33:0/86656029 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x555a91d77f20 0x555a91d7a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:35.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.884+0000 7fbee4a00700 1 --2- 172.21.15.33:0/86656029 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x555a91d77f20 0x555a91d7a340 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83af72add8684145 server_cookie=dc141356e3b68e62 in_seq=0 out_seq=0 2024-05-03T18:46:35.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.884+0000 7fbec7400700 10 client.17058 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:46:35.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.886+0000 7fbec7400700 1 -- 172.21.15.33:0/86656029 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555a91d26ff0 con 0x555a91d77f20 2024-05-03T18:46:35.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.886+0000 7fbec7400700 10 client.17058 handle_command_reply: tid=0 2024-05-03T18:46:35.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.886+0000 7fbef26afb80 1 client.17058 shutdown 2024-05-03T18:46:35.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.886+0000 7fbef26afb80 1 -- 172.21.15.33:0/86656029 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555a91d5a550 msgr2=0x555a91d65ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:35.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.886+0000 7fbef26afb80 1 --2- 172.21.15.33:0/86656029 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555a91d5a550 0x555a91d65ec0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fbed4009dc0 tx=0x7fbed4003960 comp rx=0 tx=0).stop 2024-05-03T18:46:35.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.886+0000 7fbef26afb80 1 -- 172.21.15.33:0/86656029 shutdown_connections 2024-05-03T18:46:35.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.886+0000 7fbef26afb80 1 --2- 172.21.15.33:0/86656029 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x555a91d77f20 0x555a91d7a340 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:35.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.886+0000 7fbef26afb80 1 --2- 172.21.15.33:0/86656029 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555a91d60250 0x555a91d637f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:35.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.886+0000 7fbef26afb80 1 --2- 172.21.15.33:0/86656029 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555a91d5a550 0x555a91d65ec0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:35.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.886+0000 7fbef26afb80 1 --2- 172.21.15.33:0/86656029 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555a91d5f930 0x555a91d632b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:35.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.886+0000 7fbef26afb80 1 -- 172.21.15.33:0/86656029 >> 172.21.15.33:0/86656029 conn(0x555a91cd26d0 msgr2=0x555a91d4f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:46:35.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.886+0000 7fbec7400700 0 client.17058 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:46:35.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.886+0000 7fbef26afb80 1 -- 172.21.15.33:0/86656029 shutdown_connections 2024-05-03T18:46:35.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.886+0000 7fbef26afb80 1 -- 172.21.15.33:0/86656029 wait complete. 2024-05-03T18:46:35.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.887+0000 7fbef26afb80 20 client.17058 trim_cache size 0 max 16384 2024-05-03T18:46:35.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.916+0000 7fbef26afb80 1 Processor -- start 2024-05-03T18:46:35.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.916+0000 7fbef26afb80 1 -- start start 2024-05-03T18:46:35.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.916+0000 7fbef26afb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555a91d5a550 0x555a91d59f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:35.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.916+0000 7fbef26afb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555a91d5f930 0x555a91d74140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:35.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.916+0000 7fbef26afb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555a91d60250 0x555a91d7fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:35.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.916+0000 7fbef26afb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbee01ab2c0 con 0x555a91d5f930 2024-05-03T18:46:35.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.916+0000 7fbef26afb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbee01adb00 con 0x555a91d60250 2024-05-03T18:46:35.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.916+0000 7fbef26afb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fbee01ab910 con 0x555a91d5a550 2024-05-03T18:46:35.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.916+0000 7fbee4a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555a91d5f930 0x555a91d74140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:35.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.917+0000 7fbee4a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555a91d5f930 0x555a91d74140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55564/0 (socket says 172.21.15.33:55564) 2024-05-03T18:46:35.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.917+0000 7fbee4a00700 1 -- 172.21.15.33:0/4058353160 learned_addr learned my addr 172.21.15.33:0/4058353160 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:46:35.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.917+0000 7fbee5400700 1 --2- 172.21.15.33:0/4058353160 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555a91d5a550 0x555a91d59f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:35.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.917+0000 7fbee5e00700 1 --2- 172.21.15.33:0/4058353160 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555a91d60250 0x555a91d7fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:35.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.917+0000 7fbee4a00700 1 -- 172.21.15.33:0/4058353160 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555a91d5a550 msgr2=0x555a91d59f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:46:35.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.917+0000 7fbee4a00700 1 --2- 172.21.15.33:0/4058353160 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555a91d5a550 0x555a91d59f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:35.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.917+0000 7fbee4a00700 1 -- 172.21.15.33:0/4058353160 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555a91d60250 msgr2=0x555a91d7fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:46:35.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.917+0000 7fbee4a00700 1 --2- 172.21.15.33:0/4058353160 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555a91d60250 0x555a91d7fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:35.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.917+0000 7fbee4a00700 1 -- 172.21.15.33:0/4058353160 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a91d60050 con 0x555a91d5f930 2024-05-03T18:46:35.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.917+0000 7fbee5e00700 1 --2- 172.21.15.33:0/4058353160 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555a91d60250 0x555a91d7fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:46:35.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.917+0000 7fbee4a00700 1 --2- 172.21.15.33:0/4058353160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555a91d5f930 0x555a91d74140 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fbecc00b2f0 tx=0x7fbecc0a9c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ad687d4c98708d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:46:35.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.918+0000 7fbed2000700 1 -- 172.21.15.33:0/4058353160 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbecc0c3070 con 0x555a91d5f930 2024-05-03T18:46:35.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.918+0000 7fbed2000700 1 -- 172.21.15.33:0/4058353160 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbecc0a5720 con 0x555a91d5f930 2024-05-03T18:46:35.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.918+0000 7fbed2000700 1 -- 172.21.15.33:0/4058353160 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbecc014030 con 0x555a91d5f930 2024-05-03T18:46:35.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.919+0000 7fbef26afb80 1 -- 172.21.15.33:0/4058353160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555a91d5f930 msgr2=0x555a91d74140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:35.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.919+0000 7fbef26afb80 1 --2- 172.21.15.33:0/4058353160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555a91d5f930 0x555a91d74140 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fbecc00b2f0 tx=0x7fbecc0a9c90 comp rx=0 tx=0).stop 2024-05-03T18:46:35.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.919+0000 7fbef26afb80 1 -- 172.21.15.33:0/4058353160 shutdown_connections 2024-05-03T18:46:35.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.919+0000 7fbef26afb80 1 --2- 172.21.15.33:0/4058353160 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555a91d60250 0x555a91d7fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:35.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.919+0000 7fbef26afb80 1 --2- 172.21.15.33:0/4058353160 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555a91d5a550 0x555a91d59f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:35.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.919+0000 7fbef26afb80 1 --2- 172.21.15.33:0/4058353160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555a91d5f930 0x555a91d74140 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:35.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.919+0000 7fbef26afb80 1 -- 172.21.15.33:0/4058353160 >> 172.21.15.33:0/4058353160 conn(0x555a91cd26d0 msgr2=0x555a91d61ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:46:35.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.919+0000 7fbef26afb80 1 -- 172.21.15.33:0/4058353160 shutdown_connections 2024-05-03T18:46:35.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.919+0000 7fbef26afb80 1 -- 172.21.15.33:0/4058353160 wait complete. 2024-05-03T18:46:35.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.919+0000 7fbef26afb80 1 -- start start 2024-05-03T18:46:35.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.920+0000 7fbef26afb80 1 Processor -- start 2024-05-03T18:46:35.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.920+0000 7fbef26afb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555a91d5a550 0x555a91d7f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:35.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.920+0000 7fbef26afb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555a91d5f930 0x555a91d7f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:35.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.920+0000 7fbef26afb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555a91d60250 0x555a91d7ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:35.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.920+0000 7fbef26afb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbee01abfc0 con 0x555a91d5a550 2024-05-03T18:46:35.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.920+0000 7fbef26afb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbee01ad430 con 0x555a91d60250 2024-05-03T18:46:35.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.920+0000 7fbef26afb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fbee01ac690 con 0x555a91d5f930 2024-05-03T18:46:35.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.920+0000 7fbef26afb80 10 client.0 _ll_register_callbacks cb 0x555a91c53c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:46:35.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.921+0000 7fbee5400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555a91d5a550 0x555a91d7f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:35.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.921+0000 7fbee5400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555a91d5a550 0x555a91d7f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55570/0 (socket says 172.21.15.33:55570) 2024-05-03T18:46:35.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.921+0000 7fbee5400700 1 -- 172.21.15.33:0/1618760244 learned_addr learned my addr 172.21.15.33:0/1618760244 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:46:35.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.921+0000 7fbee5e00700 1 --2- 172.21.15.33:0/1618760244 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555a91d60250 0x555a91d7ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:35.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.921+0000 7fbee5400700 1 -- 172.21.15.33:0/1618760244 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555a91d5f930 msgr2=0x555a91d7f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:46:35.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.921+0000 7fbee5400700 1 --2- 172.21.15.33:0/1618760244 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555a91d5f930 0x555a91d7f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:35.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.921+0000 7fbee5400700 1 -- 172.21.15.33:0/1618760244 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555a91d60250 msgr2=0x555a91d7ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:46:35.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.921+0000 7fbee5400700 1 --2- 172.21.15.33:0/1618760244 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555a91d60250 0x555a91d7ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:35.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.921+0000 7fbee5400700 1 -- 172.21.15.33:0/1618760244 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a91de1870 con 0x555a91d5a550 2024-05-03T18:46:35.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.922+0000 7fbee5400700 1 --2- 172.21.15.33:0/1618760244 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555a91d5a550 0x555a91d7f4b0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fbed40059d0 tx=0x7fbed4011ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6345967bce206045 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:46:35.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.922+0000 7fbec7400700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:46:35.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.922+0000 7fbec7400700 1 -- 172.21.15.33:0/1618760244 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed4005250 con 0x555a91d5a550 2024-05-03T18:46:35.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.922+0000 7fbec7400700 1 -- 172.21.15.33:0/1618760244 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9518+0+0 (secure 0 0 0) 0x7fbed40053f0 con 0x555a91d5a550 2024-05-03T18:46:35.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.922+0000 7fbef26afb80 1 -- 172.21.15.33:0/1618760244 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbecc0a5720 con 0x555a91d5a550 2024-05-03T18:46:35.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.922+0000 7fbec7400700 1 -- 172.21.15.33:0/1618760244 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fbed40055f0 con 0x555a91d5a550 2024-05-03T18:46:35.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.922+0000 7fbef26afb80 10 client.17064 fetch_fsmap learned FSMap version 21 2024-05-03T18:46:35.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.922+0000 7fbef26afb80 1 -- 172.21.15.33:0/1618760244 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x555a91de2b90 con 0x555a91d5a550 2024-05-03T18:46:35.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.923+0000 7fbec7400700 1 -- 172.21.15.33:0/1618760244 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbecc0a5720 con 0x555a91d5a550 2024-05-03T18:46:35.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.923+0000 7fbec7400700 1 -- 172.21.15.33:0/1618760244 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fbed401b420 con 0x555a91d5a550 2024-05-03T18:46:35.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.923+0000 7fbef26afb80 10 client.17064 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:46:35.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.923+0000 7fbef26afb80 10 client.17064 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:46:35.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.923+0000 7fbef26afb80 20 client.17064 populate_metadata read hostname 'smithi033' 2024-05-03T18:46:35.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.923+0000 7fbef26afb80 1 --2- 172.21.15.33:0/1618760244 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x555a91de3320 0x555a91de5740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:35.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.923+0000 7fbef26afb80 4 client.17064 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-03T18:46:35.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.923+0000 7fbef26afb80 1 -- 172.21.15.33:0/1618760244 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555a91cba400 con 0x555a91de3320 2024-05-03T18:46:35.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.924+0000 7fbee4a00700 1 --2- 172.21.15.33:0/1618760244 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x555a91de3320 0x555a91de5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:35.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.925+0000 7fbec7400700 10 client.17064 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:46:35.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.925+0000 7fbee4a00700 1 --2- 172.21.15.33:0/1618760244 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x555a91de3320 0x555a91de5740 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4afb9a21ec9b4757 server_cookie=7151565d60b68c1 in_seq=0 out_seq=0 2024-05-03T18:46:35.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.925+0000 7fbec7400700 1 -- 172.21.15.33:0/1618760244 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x555a91cba400 con 0x555a91de3320 2024-05-03T18:46:35.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.925+0000 7fbec7400700 10 client.17064 handle_command_reply: tid=0 2024-05-03T18:46:35.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.925+0000 7fbef26afb80 1 client.17064 shutdown 2024-05-03T18:46:35.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.926+0000 7fbef26afb80 1 -- 172.21.15.33:0/1618760244 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555a91d5a550 msgr2=0x555a91d7f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:35.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.926+0000 7fbef26afb80 1 --2- 172.21.15.33:0/1618760244 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555a91d5a550 0x555a91d7f4b0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fbed40059d0 tx=0x7fbed4011ef0 comp rx=0 tx=0).stop 2024-05-03T18:46:35.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.926+0000 7fbef26afb80 1 -- 172.21.15.33:0/1618760244 shutdown_connections 2024-05-03T18:46:35.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.926+0000 7fbef26afb80 1 --2- 172.21.15.33:0/1618760244 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x555a91de3320 0x555a91de5740 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:35.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.926+0000 7fbef26afb80 1 --2- 172.21.15.33:0/1618760244 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555a91d60250 0x555a91d7ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:35.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.926+0000 7fbef26afb80 1 --2- 172.21.15.33:0/1618760244 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555a91d5a550 0x555a91d7f4b0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:35.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.926+0000 7fbef26afb80 1 --2- 172.21.15.33:0/1618760244 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555a91d5f930 0x555a91d7f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:35.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.926+0000 7fbec7400700 0 client.17064 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:46:35.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.926+0000 7fbef26afb80 1 -- 172.21.15.33:0/1618760244 >> 172.21.15.33:0/1618760244 conn(0x555a91cd26d0 msgr2=0x555a91d60630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:46:35.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.926+0000 7fbef26afb80 1 -- 172.21.15.33:0/1618760244 shutdown_connections 2024-05-03T18:46:35.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.926+0000 7fbef26afb80 1 -- 172.21.15.33:0/1618760244 wait complete. 2024-05-03T18:46:35.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.926+0000 7fbef26afb80 20 client.17064 trim_cache size 0 max 16384 2024-05-03T18:46:35.930 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-03T18:46:35.931 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-03T18:46:35.931 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-05-03T18:46:35.931 INFO:teuthology.orchestra.run.smithi033.stdout: "ecbf00a8-db96-45d6-b515-6fd29b57c5b0": { 2024-05-03T18:46:35.931 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-05-03T18:46:35.931 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "ecbf00a8-db96-45d6-b515-6fd29b57c5b0", 2024-05-03T18:46:35.931 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-05-03T18:46:35.931 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:46:35.931 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:46:35.931 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-03T18:46:35.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.927+0000 7fbee7e00700 1 -- 172.21.15.33:0/2082779431 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fbec80b2a70 msgr2=0x7fbec80b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:35.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.927+0000 7fbee7e00700 1 --2- 172.21.15.33:0/2082779431 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fbec80b2a70 0x7fbec80b4f30 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fbed80062a0 tx=0x7fbed800bb90 comp rx=0 tx=0).stop 2024-05-03T18:46:35.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.927+0000 7fbee7e00700 1 -- 172.21.15.33:0/2082779431 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbee006bb40 msgr2=0x7fbee019fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:35.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.927+0000 7fbee7e00700 1 --2- 172.21.15.33:0/2082779431 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbee006bb40 0x7fbee019fc20 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fbecc00d950 tx=0x7fbecc00dd10 comp rx=0 tx=0).stop 2024-05-03T18:46:35.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.927+0000 7fbee7e00700 1 -- 172.21.15.33:0/2082779431 shutdown_connections 2024-05-03T18:46:35.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.927+0000 7fbee7e00700 1 --2- 172.21.15.33:0/2082779431 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbee0109510 0x7fbee01998a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:35.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.927+0000 7fbee7e00700 1 --2- 172.21.15.33:0/2082779431 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fbec80b2a70 0x7fbec80b4f30 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:35.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.927+0000 7fbee7e00700 1 --2- 172.21.15.33:0/2082779431 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbee006b180 0x7fbee019f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:35.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.927+0000 7fbee7e00700 1 --2- 172.21.15.33:0/2082779431 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbee006bb40 0x7fbee019fc20 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:35.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.927+0000 7fbee7e00700 1 -- 172.21.15.33:0/2082779431 >> 172.21.15.33:0/2082779431 conn(0x7fbee0079350 msgr2=0x7fbee01010e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:46:35.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.927+0000 7fbee7e00700 1 -- 172.21.15.33:0/2082779431 shutdown_connections 2024-05-03T18:46:35.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:35.928+0000 7fbee7e00700 1 -- 172.21.15.33:0/2082779431 wait complete. 2024-05-03T18:46:35.950 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:46:35 smithi033 ceph-mon[28330]: pgmap v612: 129 pgs: 129 active+clean; 3.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 557 KiB/s rd, 4.7 MiB/s wr, 441 op/s 2024-05-03T18:46:36.020 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 155648 May 3 18:46 0007 2024-05-03T18:46:36.020 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:46:36.020 INFO:teuthology.orchestra.run.smithi033.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2024-05-03T18:46:36.020 INFO:teuthology.orchestra.run.smithi033.stdout:total 1 2024-05-03T18:46:36.020 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 . 2024-05-03T18:46:36.020 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1482306596 May 3 18:46 .. 2024-05-03T18:46:36.020 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:46:36.020 INFO:teuthology.orchestra.run.smithi033.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2024-05-03T18:46:36.020 INFO:teuthology.orchestra.run.smithi033.stdout:total 1 2024-05-03T18:46:36.021 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 . 2024-05-03T18:46:36.021 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1482306596 May 3 18:46 .. 2024-05-03T18:46:36.021 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:46:36.021 INFO:teuthology.orchestra.run.smithi033.stdout:/tmp/cephfs/postgres/data/pg_wal: 2024-05-03T18:46:36.021 INFO:teuthology.orchestra.run.smithi033.stdout:total 884738 2024-05-03T18:46:36.021 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 905969664 May 3 18:46 . 2024-05-03T18:46:36.021 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1482306596 May 3 18:46 .. 2024-05-03T18:46:36.021 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 3 18:46 000000010000000000000077 2024-05-03T18:46:36.021 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 3 18:42 000000010000000000000078 2024-05-03T18:46:36.021 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 3 18:39 000000010000000000000079 2024-05-03T18:46:36.021 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 3 18:38 00000001000000000000007A 2024-05-03T18:46:36.021 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 3 18:40 00000001000000000000007B 2024-05-03T18:46:36.021 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 3 18:38 00000001000000000000007C 2024-05-03T18:46:36.021 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 3 18:38 00000001000000000000007D 2024-05-03T18:46:36.021 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 3 18:41 00000001000000000000007E 2024-05-03T18:46:36.022 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 3 18:38 00000001000000000000007F 2024-05-03T18:46:36.022 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 3 18:40 000000010000000000000080 2024-05-03T18:46:36.022 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 3 18:43 000000010000000000000081 2024-05-03T18:46:36.022 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 3 18:39 000000010000000000000082 2024-05-03T18:46:36.022 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 3 18:38 000000010000000000000083 2024-05-03T18:46:36.022 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 3 18:38 000000010000000000000084 2024-05-03T18:46:36.022 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 3 18:41 000000010000000000000085 2024-05-03T18:46:36.022 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 3 18:39 000000010000000000000086 2024-05-03T18:46:36.022 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 3 18:42 000000010000000000000087 2024-05-03T18:46:36.022 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 3 18:38 000000010000000000000088 2024-05-03T18:46:36.022 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 3 18:39 000000010000000000000089 2024-05-03T18:46:36.022 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 3 18:39 00000001000000000000008A 2024-05-03T18:46:36.022 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 3 18:40 00000001000000000000008B 2024-05-03T18:46:36.022 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 3 18:40 00000001000000000000008C 2024-05-03T18:46:36.023 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 3 18:41 00000001000000000000008D 2024-05-03T18:46:36.023 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 3 18:39 00000001000000000000008E 2024-05-03T18:46:36.023 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 3 18:39 00000001000000000000008F 2024-05-03T18:46:36.023 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 3 18:38 000000010000000000000090 2024-05-03T18:46:36.023 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 3 18:39 000000010000000000000091 2024-05-03T18:46:36.023 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 3 18:43 000000010000000000000092 2024-05-03T18:46:36.042 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 3 18:43 000000010000000000000093 2024-05-03T18:46:36.042 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 3 18:45 000000010000000000000094 2024-05-03T18:46:36.042 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 3 18:46 000000010000000000000095 2024-05-03T18:46:36.042 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 3 18:45 000000010000000000000096 2024-05-03T18:46:36.043 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 3 18:43 000000010000000000000097 2024-05-03T18:46:36.043 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 3 18:44 000000010000000000000098 2024-05-03T18:46:36.043 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 3 18:43 000000010000000000000099 2024-05-03T18:46:36.043 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 3 18:44 00000001000000000000009A 2024-05-03T18:46:36.043 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 3 18:45 00000001000000000000009B 2024-05-03T18:46:36.043 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 3 18:43 00000001000000000000009C 2024-05-03T18:46:36.043 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 3 18:44 00000001000000000000009D 2024-05-03T18:46:36.043 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 3 18:43 00000001000000000000009E 2024-05-03T18:46:36.043 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 3 18:44 00000001000000000000009F 2024-05-03T18:46:36.043 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 3 18:45 0000000100000000000000A0 2024-05-03T18:46:36.043 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 3 18:44 0000000100000000000000A1 2024-05-03T18:46:36.043 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 3 18:44 0000000100000000000000A2 2024-05-03T18:46:36.043 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 3 18:43 0000000100000000000000A3 2024-05-03T18:46:36.044 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 3 18:44 0000000100000000000000A4 2024-05-03T18:46:36.044 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 3 18:44 0000000100000000000000A5 2024-05-03T18:46:36.044 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 3 18:43 0000000100000000000000A6 2024-05-03T18:46:36.044 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 3 18:43 0000000100000000000000A7 2024-05-03T18:46:36.044 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 3 18:44 0000000100000000000000A8 2024-05-03T18:46:36.044 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 3 18:43 0000000100000000000000A9 2024-05-03T18:46:36.044 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 3 18:45 0000000100000000000000AA 2024-05-03T18:46:36.044 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 3 18:45 0000000100000000000000AB 2024-05-03T18:46:36.044 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 3 18:43 0000000100000000000000AC 2024-05-03T18:46:36.044 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 archive_status 2024-05-03T18:46:36.044 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:46:36.044 INFO:teuthology.orchestra.run.smithi033.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2024-05-03T18:46:36.044 INFO:teuthology.orchestra.run.smithi033.stdout:total 1 2024-05-03T18:46:36.045 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 3 18:27 . 2024-05-03T18:46:36.045 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 905969664 May 3 18:46 .. 2024-05-03T18:46:36.045 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:46:36.045 INFO:teuthology.orchestra.run.smithi033.stdout:/tmp/cephfs/postgres/data/pg_xact: 2024-05-03T18:46:36.045 INFO:teuthology.orchestra.run.smithi033.stdout:total 129 2024-05-03T18:46:36.045 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 May 3 18:27 . 2024-05-03T18:46:36.045 INFO:teuthology.orchestra.run.smithi033.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1482306596 May 3 18:46 .. 2024-05-03T18:46:36.045 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 May 3 18:46 0000 2024-05-03T18:46:36.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:46:35 smithi099 ceph-mon[31658]: pgmap v612: 129 pgs: 129 active+clean; 3.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 557 KiB/s rd, 4.7 MiB/s wr, 441 op/s 2024-05-03T18:46:36.052 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-05-03T18:46:36.063 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-05-03T18:46:36.064 INFO:teuthology.task.exec:Executing custom commands... 2024-05-03T18:46:36.064 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi033.front.sepia.ceph.com 2024-05-03T18:46:36.064 DEBUG:teuthology.orchestra.run.smithi033:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-05-03T18:46:36.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:46:35 smithi150 ceph-mon[32492]: pgmap v612: 129 pgs: 129 active+clean; 3.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 557 KiB/s rd, 4.7 MiB/s wr, 441 op/s 2024-05-03T18:46:36.160 DEBUG:teuthology.orchestra.run.smithi033:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-05-03T18:46:36.241 DEBUG:teuthology.orchestra.run.smithi033:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-05-03T18:46:36.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:46:36 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:46:36.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:46:36 smithi033 ceph-mon[28330]: scrub summary: idle 2024-05-03T18:46:37.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.041+0000 7f3f09c00640 1 -- 172.21.15.33:0/3527584097 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f04103350 msgr2=0x7f3f0410f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:37.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.041+0000 7f3f09c00640 1 --2- 172.21.15.33:0/3527584097 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f04103350 0x7f3f0410f860 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f3efc009930 tx=0x7f3efc02a420 comp rx=0 tx=0).stop 2024-05-03T18:46:37.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.041+0000 7f3f09c00640 1 -- 172.21.15.33:0/3527584097 shutdown_connections 2024-05-03T18:46:37.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.041+0000 7f3f09c00640 1 --2- 172.21.15.33:0/3527584097 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f04103350 0x7f3f0410f860 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:37.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.041+0000 7f3f09c00640 1 --2- 172.21.15.33:0/3527584097 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f04102990 0x7f3f04102e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:37.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.041+0000 7f3f09c00640 1 --2- 172.21.15.33:0/3527584097 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f041089a0 0x7f3f04108d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:37.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.041+0000 7f3f09c00640 1 -- 172.21.15.33:0/3527584097 >> 172.21.15.33:0/3527584097 conn(0x7f3f04079e40 msgr2=0x7f3f0407a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:46:37.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.041+0000 7f3f09c00640 1 -- 172.21.15.33:0/3527584097 shutdown_connections 2024-05-03T18:46:37.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.041+0000 7f3f09c00640 1 -- 172.21.15.33:0/3527584097 wait complete. 2024-05-03T18:46:37.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.042+0000 7f3f09c00640 1 Processor -- start 2024-05-03T18:46:37.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.042+0000 7f3f09c00640 1 -- start start 2024-05-03T18:46:37.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.043+0000 7f3f09c00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f04102990 0x7f3f04074cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:37.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.043+0000 7f3f09c00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f04103350 0x7f3f040751f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:37.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.043+0000 7f3f09c00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f041089a0 0x7f3f04075730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:37.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.043+0000 7f3f09c00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3f04112b30 con 0x7f3f04103350 2024-05-03T18:46:37.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.043+0000 7f3f09c00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3f041129b0 con 0x7f3f04102990 2024-05-03T18:46:37.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.043+0000 7f3f09c00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3f04112cb0 con 0x7f3f041089a0 2024-05-03T18:46:37.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.043+0000 7f3f02000640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f04103350 0x7f3f040751f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:37.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.043+0000 7f3f02000640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f04103350 0x7f3f040751f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55580/0 (socket says 172.21.15.33:55580) 2024-05-03T18:46:37.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.043+0000 7f3f02a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f04102990 0x7f3f04074cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:37.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.043+0000 7f3f02000640 1 -- 172.21.15.33:0/2445807286 learned_addr learned my addr 172.21.15.33:0/2445807286 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:46:37.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.043+0000 7f3f03400640 1 --2- 172.21.15.33:0/2445807286 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f041089a0 0x7f3f04075730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:37.047 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:46:36 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:46:37.047 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:46:36 smithi099 ceph-mon[31658]: scrub summary: idle 2024-05-03T18:46:37.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.044+0000 7f3f02000640 1 -- 172.21.15.33:0/2445807286 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f041089a0 msgr2=0x7f3f04075730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:37.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.044+0000 7f3f02000640 1 --2- 172.21.15.33:0/2445807286 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f041089a0 0x7f3f04075730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:37.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.044+0000 7f3f02000640 1 -- 172.21.15.33:0/2445807286 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f04102990 msgr2=0x7f3f04074cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:37.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.044+0000 7f3f02000640 1 --2- 172.21.15.33:0/2445807286 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f04102990 0x7f3f04074cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:37.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.044+0000 7f3f02000640 1 -- 172.21.15.33:0/2445807286 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f041a7530 con 0x7f3f04103350 2024-05-03T18:46:37.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.044+0000 7f3f02000640 1 --2- 172.21.15.33:0/2445807286 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f04103350 0x7f3f040751f0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f3ef000d950 tx=0x7f3ef000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:46:37.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.045+0000 7f3efb400640 1 -- 172.21.15.33:0/2445807286 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef0017070 con 0x7f3f04103350 2024-05-03T18:46:37.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.045+0000 7f3efb400640 1 -- 172.21.15.33:0/2445807286 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3ef0009c30 con 0x7f3f04103350 2024-05-03T18:46:37.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.045+0000 7f3efb400640 1 -- 172.21.15.33:0/2445807286 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef000f620 con 0x7f3f04103350 2024-05-03T18:46:37.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.045+0000 7f3f09c00640 1 -- 172.21.15.33:0/2445807286 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f041a7820 con 0x7f3f04103350 2024-05-03T18:46:37.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.045+0000 7f3f09c00640 1 -- 172.21.15.33:0/2445807286 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f041a7d20 con 0x7f3f04103350 2024-05-03T18:46:37.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.046+0000 7f3f09c00640 1 -- 172.21.15.33:0/2445807286 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f04050740 con 0x7f3f04103350 2024-05-03T18:46:37.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.050+0000 7f3efb400640 1 -- 172.21.15.33:0/2445807286 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f3ef0010460 con 0x7f3f04103350 2024-05-03T18:46:37.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.052+0000 7f3efb400640 1 --2- 172.21.15.33:0/2445807286 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f3ee80b2a90 0x7f3ee80b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:37.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.052+0000 7f3efb400640 1 -- 172.21.15.33:0/2445807286 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9518+0+0 (secure 0 0 0) 0x7f3ef00e2560 con 0x7f3f04103350 2024-05-03T18:46:37.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.053+0000 7f3f02a00640 1 --2- 172.21.15.33:0/2445807286 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f3ee80b2a90 0x7f3ee80b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:37.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.053+0000 7f3f02a00640 1 --2- 172.21.15.33:0/2445807286 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f3ee80b2a90 0x7f3ee80b4f50 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3ef4005fd0 tx=0x7f3ef4009500 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:46:37.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.058+0000 7f3efb400640 1 -- 172.21.15.33:0/2445807286 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f3ef0009de0 con 0x7f3f04103350 2024-05-03T18:46:37.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:46:36 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:46:37.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:46:36 smithi150 ceph-mon[32492]: scrub summary: idle 2024-05-03T18:46:37.195 INFO:tasks.cephfs.filesystem:scrub status for tag:ecbf00a8-db96-45d6-b515-6fd29b57c5b0 - {'path': '/', 'tag': 'ecbf00a8-db96-45d6-b515-6fd29b57c5b0', 'options': 'recursive,force'} 2024-05-03T18:46:37.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.306+0000 7f3f09c00640 1 -- 172.21.15.33:0/2445807286 --> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f3f04062df0 con 0x7f3ee80b2a90 2024-05-03T18:46:37.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.323+0000 7f3efb400640 1 -- 172.21.15.33:0/2445807286 <== mgr.15120 v2:172.21.15.33:6800/3302011241 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+351 (secure 0 0 0) 0x7f3f04062df0 con 0x7f3ee80b2a90 2024-05-03T18:46:37.326 INFO:teuthology.orchestra.run.smithi033.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-05-03T00:00:00", "created": "2024-05-03T18:27:28", "first": "2024-05-03T18:28:01", "last": "2024-05-03T18:46:00", "last_pruned": "2024-05-03T18:46:00", "created_count": 19, "pruned_count": 13, "active": true} 2024-05-03T18:46:37.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.326+0000 7f3f09c00640 1 -- 172.21.15.33:0/2445807286 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f3ee80b2a90 msgr2=0x7f3ee80b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:37.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.326+0000 7f3f09c00640 1 --2- 172.21.15.33:0/2445807286 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f3ee80b2a90 0x7f3ee80b4f50 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3ef4005fd0 tx=0x7f3ef4009500 comp rx=0 tx=0).stop 2024-05-03T18:46:37.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.326+0000 7f3f09c00640 1 -- 172.21.15.33:0/2445807286 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f04103350 msgr2=0x7f3f040751f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:37.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.326+0000 7f3f09c00640 1 --2- 172.21.15.33:0/2445807286 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f04103350 0x7f3f040751f0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f3ef000d950 tx=0x7f3ef000de10 comp rx=0 tx=0).stop 2024-05-03T18:46:37.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.326+0000 7f3f09c00640 1 -- 172.21.15.33:0/2445807286 shutdown_connections 2024-05-03T18:46:37.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.326+0000 7f3f09c00640 1 --2- 172.21.15.33:0/2445807286 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f3ee80b2a90 0x7f3ee80b4f50 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:37.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.326+0000 7f3f09c00640 1 --2- 172.21.15.33:0/2445807286 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f041089a0 0x7f3f04075730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:37.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.326+0000 7f3f09c00640 1 --2- 172.21.15.33:0/2445807286 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f04103350 0x7f3f040751f0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:37.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.326+0000 7f3f09c00640 1 --2- 172.21.15.33:0/2445807286 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f04102990 0x7f3f04074cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:37.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.326+0000 7f3f09c00640 1 -- 172.21.15.33:0/2445807286 >> 172.21.15.33:0/2445807286 conn(0x7f3f04079e40 msgr2=0x7f3f041003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:46:37.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.326+0000 7f3f09c00640 1 -- 172.21.15.33:0/2445807286 shutdown_connections 2024-05-03T18:46:37.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.326+0000 7f3f09c00640 1 -- 172.21.15.33:0/2445807286 wait complete. 2024-05-03T18:46:37.362 DEBUG:teuthology.orchestra.run.smithi033:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-05-03T18:46:37.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.556+0000 7f752e200640 1 -- 172.21.15.33:0/1288233341 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75281057c0 msgr2=0x7f7528109800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:37.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.556+0000 7f752e200640 1 --2- 172.21.15.33:0/1288233341 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75281057c0 0x7f7528109800 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f7514009900 tx=0x7f751402a3f0 comp rx=0 tx=0).stop 2024-05-03T18:46:37.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.556+0000 7f752e200640 1 -- 172.21.15.33:0/1288233341 shutdown_connections 2024-05-03T18:46:37.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.556+0000 7f752e200640 1 --2- 172.21.15.33:0/1288233341 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7528109ee0 0x7f7528111ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:37.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.556+0000 7f752e200640 1 --2- 172.21.15.33:0/1288233341 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75281057c0 0x7f7528109800 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:37.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.556+0000 7f752e200640 1 --2- 172.21.15.33:0/1288233341 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7528104e10 0x7f75281051f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:37.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.557+0000 7f752e200640 1 -- 172.21.15.33:0/1288233341 >> 172.21.15.33:0/1288233341 conn(0x7f75281006b0 msgr2=0x7f7528102ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:46:37.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.557+0000 7f752e200640 1 -- 172.21.15.33:0/1288233341 shutdown_connections 2024-05-03T18:46:37.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.557+0000 7f752e200640 1 -- 172.21.15.33:0/1288233341 wait complete. 2024-05-03T18:46:37.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.558+0000 7f752e200640 1 Processor -- start 2024-05-03T18:46:37.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.558+0000 7f752e200640 1 -- start start 2024-05-03T18:46:37.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.558+0000 7f752e200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7528104e10 0x7f752819b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:37.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.558+0000 7f752e200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7528109ee0 0x7f752819b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:37.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.558+0000 7f752e200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f752819fd50 0x7f75281a0200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:37.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.558+0000 7f752e200640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7528114d40 con 0x7f752819fd50 2024-05-03T18:46:37.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.558+0000 7f752e200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7528114bc0 con 0x7f7528109ee0 2024-05-03T18:46:37.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.558+0000 7f752e200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f7528114ec0 con 0x7f7528104e10 2024-05-03T18:46:37.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.559+0000 7f7527e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f752819fd50 0x7f75281a0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:37.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.559+0000 7f7527e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f752819fd50 0x7f75281a0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55592/0 (socket says 172.21.15.33:55592) 2024-05-03T18:46:37.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.559+0000 7f7527e00640 1 -- 172.21.15.33:0/2384167702 learned_addr learned my addr 172.21.15.33:0/2384167702 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:46:37.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.559+0000 7f7527400640 1 --2- 172.21.15.33:0/2384167702 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7528104e10 0x7f752819b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:37.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.559+0000 7f7526a00640 1 --2- 172.21.15.33:0/2384167702 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7528109ee0 0x7f752819b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:37.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.560+0000 7f7527e00640 1 -- 172.21.15.33:0/2384167702 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7528104e10 msgr2=0x7f752819b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:37.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.560+0000 7f7527e00640 1 --2- 172.21.15.33:0/2384167702 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7528104e10 0x7f752819b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:37.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.560+0000 7f7527e00640 1 -- 172.21.15.33:0/2384167702 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7528109ee0 msgr2=0x7f752819b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:37.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.560+0000 7f7527e00640 1 --2- 172.21.15.33:0/2384167702 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7528109ee0 0x7f752819b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:37.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.560+0000 7f7527e00640 1 -- 172.21.15.33:0/2384167702 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75281a0840 con 0x7f752819fd50 2024-05-03T18:46:37.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.560+0000 7f7526a00640 1 --2- 172.21.15.33:0/2384167702 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7528109ee0 0x7f752819b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:46:37.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.560+0000 7f7527e00640 1 --2- 172.21.15.33:0/2384167702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f752819fd50 0x7f75281a0200 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f752000eaa0 tx=0x7f752000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:46:37.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.560+0000 7f751fe00640 1 -- 172.21.15.33:0/2384167702 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f752000cdb0 con 0x7f752819fd50 2024-05-03T18:46:37.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.560+0000 7f751fe00640 1 -- 172.21.15.33:0/2384167702 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7520005020 con 0x7f752819fd50 2024-05-03T18:46:37.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.560+0000 7f751fe00640 1 -- 172.21.15.33:0/2384167702 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75200116d0 con 0x7f752819fd50 2024-05-03T18:46:37.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.560+0000 7f752e200640 1 -- 172.21.15.33:0/2384167702 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75281a0b30 con 0x7f752819fd50 2024-05-03T18:46:37.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.561+0000 7f752e200640 1 -- 172.21.15.33:0/2384167702 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75281a82a0 con 0x7f752819fd50 2024-05-03T18:46:37.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.562+0000 7f752e200640 1 -- 172.21.15.33:0/2384167702 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7528050740 con 0x7f752819fd50 2024-05-03T18:46:37.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.566+0000 7f751fe00640 1 -- 172.21.15.33:0/2384167702 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f75200051d0 con 0x7f752819fd50 2024-05-03T18:46:37.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.568+0000 7f751fe00640 1 --2- 172.21.15.33:0/2384167702 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f750c0b2a90 0x7f750c0b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:37.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.568+0000 7f751fe00640 1 -- 172.21.15.33:0/2384167702 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9518+0+0 (secure 0 0 0) 0x7f75200e2b60 con 0x7f752819fd50 2024-05-03T18:46:37.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.568+0000 7f7527400640 1 --2- 172.21.15.33:0/2384167702 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f750c0b2a90 0x7f750c0b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:37.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.569+0000 7f7527400640 1 --2- 172.21.15.33:0/2384167702 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f750c0b2a90 0x7f750c0b4f50 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f751800bbe0 tx=0x7f751800b390 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:46:37.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.572+0000 7f751fe00640 1 -- 172.21.15.33:0/2384167702 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f7520016030 con 0x7f752819fd50 2024-05-03T18:46:37.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.818+0000 7f752e200640 1 -- 172.21.15.33:0/2384167702 --> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f7528062df0 con 0x7f750c0b2a90 2024-05-03T18:46:37.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.835+0000 7f751fe00640 1 -- 172.21.15.33:0/2384167702 <== mgr.15120 v2:172.21.15.33:6800/3302011241 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f7528062df0 con 0x7f750c0b2a90 2024-05-03T18:46:37.837 INFO:teuthology.orchestra.run.smithi033.stdout:/ 1m 6m3h 2024-05-03T18:46:37.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.838+0000 7f752e200640 1 -- 172.21.15.33:0/2384167702 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f750c0b2a90 msgr2=0x7f750c0b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:37.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.838+0000 7f752e200640 1 --2- 172.21.15.33:0/2384167702 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f750c0b2a90 0x7f750c0b4f50 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f751800bbe0 tx=0x7f751800b390 comp rx=0 tx=0).stop 2024-05-03T18:46:37.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.838+0000 7f752e200640 1 -- 172.21.15.33:0/2384167702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f752819fd50 msgr2=0x7f75281a0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:37.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.838+0000 7f752e200640 1 --2- 172.21.15.33:0/2384167702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f752819fd50 0x7f75281a0200 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f752000eaa0 tx=0x7f752000ef60 comp rx=0 tx=0).stop 2024-05-03T18:46:37.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.838+0000 7f752e200640 1 -- 172.21.15.33:0/2384167702 shutdown_connections 2024-05-03T18:46:37.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.838+0000 7f752e200640 1 --2- 172.21.15.33:0/2384167702 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f750c0b2a90 0x7f750c0b4f50 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:37.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.838+0000 7f752e200640 1 --2- 172.21.15.33:0/2384167702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f752819fd50 0x7f75281a0200 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:37.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.838+0000 7f752e200640 1 --2- 172.21.15.33:0/2384167702 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7528109ee0 0x7f752819b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:37.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.838+0000 7f752e200640 1 --2- 172.21.15.33:0/2384167702 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7528104e10 0x7f752819b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:37.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.838+0000 7f752e200640 1 -- 172.21.15.33:0/2384167702 >> 172.21.15.33:0/2384167702 conn(0x7f75281006b0 msgr2=0x7f7528100f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:46:37.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.839+0000 7f752e200640 1 -- 172.21.15.33:0/2384167702 shutdown_connections 2024-05-03T18:46:37.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:46:37.839+0000 7f752e200640 1 -- 172.21.15.33:0/2384167702 wait complete. 2024-05-03T18:46:37.860 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-05-03T18:46:37.872 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-05-03T18:46:38.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:46:37 smithi099 ceph-mon[31658]: pgmap v613: 129 pgs: 129 active+clean; 3.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 3.0 MiB/s wr, 281 op/s 2024-05-03T18:46:38.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:46:37 smithi150 ceph-mon[32492]: pgmap v613: 129 pgs: 129 active+clean; 3.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 3.0 MiB/s wr, 281 op/s 2024-05-03T18:46:38.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:46:37 smithi033 ceph-mon[28330]: pgmap v613: 129 pgs: 129 active+clean; 3.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 3.0 MiB/s wr, 281 op/s 2024-05-03T18:46:39.047 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:46:38 smithi099 ceph-mon[31658]: from='client.17070 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:46:39.047 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:46:38 smithi099 ceph-mon[31658]: from='client.17076 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:46:39.047 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:46:38 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:46:39.047 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:46:38 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:46:39.047 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:46:38 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:46:39.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:46:38 smithi150 ceph-mon[32492]: from='client.17070 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:46:39.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:46:38 smithi150 ceph-mon[32492]: from='client.17076 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:46:39.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:46:38 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:46:39.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:46:38 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:46:39.134 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:46:38 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:46:39.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:46:38 smithi033 ceph-mon[28330]: from='client.17070 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:46:39.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:46:38 smithi033 ceph-mon[28330]: from='client.17076 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:46:39.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:46:38 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:46:39.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:46:38 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:46:39.236 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:46:38 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:46:40.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:46:39 smithi150 ceph-mon[32492]: pgmap v614: 129 pgs: 129 active+clean; 3.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 484 KiB/s wr, 14 op/s 2024-05-03T18:46:40.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:46:39 smithi033 ceph-mon[28330]: pgmap v614: 129 pgs: 129 active+clean; 3.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 484 KiB/s wr, 14 op/s 2024-05-03T18:46:40.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:46:39 smithi099 ceph-mon[31658]: pgmap v614: 129 pgs: 129 active+clean; 3.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 484 KiB/s wr, 14 op/s 2024-05-03T18:46:41.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:46:41 smithi033 ceph-mon[28330]: pgmap v615: 129 pgs: 129 active+clean; 3.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 518 KiB/s wr, 17 op/s 2024-05-03T18:46:42.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:46:41 smithi150 ceph-mon[32492]: pgmap v615: 129 pgs: 129 active+clean; 3.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 518 KiB/s wr, 17 op/s 2024-05-03T18:46:42.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:46:41 smithi099 ceph-mon[31658]: pgmap v615: 129 pgs: 129 active+clean; 3.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 518 KiB/s wr, 17 op/s 2024-05-03T18:46:44.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:46:43 smithi150 ceph-mon[32492]: pgmap v616: 129 pgs: 129 active+clean; 3.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 144 KiB/s wr, 9 op/s 2024-05-03T18:46:44.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:46:43 smithi033 ceph-mon[28330]: pgmap v616: 129 pgs: 129 active+clean; 3.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 144 KiB/s wr, 9 op/s 2024-05-03T18:46:44.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:46:43 smithi099 ceph-mon[31658]: pgmap v616: 129 pgs: 129 active+clean; 3.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 144 KiB/s wr, 9 op/s 2024-05-03T18:46:46.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:46:45 smithi150 ceph-mon[32492]: pgmap v617: 129 pgs: 129 active+clean; 3.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 161 KiB/s wr, 10 op/s 2024-05-03T18:46:46.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:46:45 smithi033 ceph-mon[28330]: pgmap v617: 129 pgs: 129 active+clean; 3.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 161 KiB/s wr, 10 op/s 2024-05-03T18:46:46.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:46:45 smithi099 ceph-mon[31658]: pgmap v617: 129 pgs: 129 active+clean; 3.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 161 KiB/s wr, 10 op/s 2024-05-03T18:46:48.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:46:47 smithi150 ceph-mon[32492]: pgmap v618: 129 pgs: 129 active+clean; 3.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 53 KiB/s wr, 6 op/s 2024-05-03T18:46:48.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:46:47 smithi033 ceph-mon[28330]: pgmap v618: 129 pgs: 129 active+clean; 3.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 53 KiB/s wr, 6 op/s 2024-05-03T18:46:48.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:46:47 smithi099 ceph-mon[31658]: pgmap v618: 129 pgs: 129 active+clean; 3.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 53 KiB/s wr, 6 op/s 2024-05-03T18:46:50.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:46:49 smithi150 ceph-mon[32492]: pgmap v619: 129 pgs: 129 active+clean; 3.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 52 KiB/s wr, 5 op/s 2024-05-03T18:46:50.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:46:49 smithi033 ceph-mon[28330]: pgmap v619: 129 pgs: 129 active+clean; 3.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 52 KiB/s wr, 5 op/s 2024-05-03T18:46:50.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:46:49 smithi099 ceph-mon[31658]: pgmap v619: 129 pgs: 129 active+clean; 3.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 52 KiB/s wr, 5 op/s 2024-05-03T18:46:52.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:46:51 smithi150 ceph-mon[32492]: pgmap v620: 129 pgs: 129 active+clean; 3.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 52 KiB/s wr, 5 op/s 2024-05-03T18:46:52.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:46:51 smithi033 ceph-mon[28330]: pgmap v620: 129 pgs: 129 active+clean; 3.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 52 KiB/s wr, 5 op/s 2024-05-03T18:46:52.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:46:51 smithi099 ceph-mon[31658]: pgmap v620: 129 pgs: 129 active+clean; 3.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 52 KiB/s wr, 5 op/s 2024-05-03T18:46:54.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:46:53 smithi033 ceph-mon[28330]: pgmap v621: 129 pgs: 129 active+clean; 3.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 2 op/s 2024-05-03T18:46:54.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:46:53 smithi099 ceph-mon[31658]: pgmap v621: 129 pgs: 129 active+clean; 3.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 2 op/s 2024-05-03T18:46:54.428 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:46:53 smithi150 ceph-mon[32492]: pgmap v621: 129 pgs: 129 active+clean; 3.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 2 op/s 2024-05-03T18:46:56.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:46:55 smithi033 ceph-mon[28330]: pgmap v622: 129 pgs: 129 active+clean; 3.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 28 KiB/s wr, 3 op/s 2024-05-03T18:46:56.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:46:55 smithi099 ceph-mon[31658]: pgmap v622: 129 pgs: 129 active+clean; 3.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 28 KiB/s wr, 3 op/s 2024-05-03T18:46:56.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:46:55 smithi150 ceph-mon[32492]: pgmap v622: 129 pgs: 129 active+clean; 3.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 28 KiB/s wr, 3 op/s 2024-05-03T18:46:58.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:46:57 smithi033 ceph-mon[28330]: pgmap v623: 129 pgs: 129 active+clean; 3.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 2 op/s 2024-05-03T18:46:58.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:46:57 smithi099 ceph-mon[31658]: pgmap v623: 129 pgs: 129 active+clean; 3.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 2 op/s 2024-05-03T18:46:58.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:46:57 smithi150 ceph-mon[32492]: pgmap v623: 129 pgs: 129 active+clean; 3.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 2 op/s 2024-05-03T18:46:59.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:46:59 smithi033 ceph-mon[28330]: pgmap v624: 129 pgs: 129 active+clean; 3.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 2 op/s 2024-05-03T18:47:00.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:46:59 smithi099 ceph-mon[31658]: pgmap v624: 129 pgs: 129 active+clean; 3.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 2 op/s 2024-05-03T18:47:00.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:46:59 smithi150 ceph-mon[32492]: pgmap v624: 129 pgs: 129 active+clean; 3.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 2 op/s 2024-05-03T18:47:02.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:47:01 smithi033 ceph-mon[28330]: pgmap v625: 129 pgs: 129 active+clean; 3.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 2 op/s 2024-05-03T18:47:02.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:47:01 smithi099 ceph-mon[31658]: pgmap v625: 129 pgs: 129 active+clean; 3.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 2 op/s 2024-05-03T18:47:02.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:47:01 smithi150 ceph-mon[32492]: pgmap v625: 129 pgs: 129 active+clean; 3.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 2 op/s 2024-05-03T18:47:04.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:47:03 smithi033 ceph-mon[28330]: pgmap v626: 129 pgs: 129 active+clean; 3.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 2 op/s 2024-05-03T18:47:04.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:47:03 smithi099 ceph-mon[31658]: pgmap v626: 129 pgs: 129 active+clean; 3.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 2 op/s 2024-05-03T18:47:04.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:47:03 smithi150 ceph-mon[32492]: pgmap v626: 129 pgs: 129 active+clean; 3.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 2 op/s 2024-05-03T18:47:06.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:47:05 smithi033 ceph-mon[28330]: pgmap v627: 129 pgs: 129 active+clean; 3.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 3 op/s 2024-05-03T18:47:06.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:47:05 smithi099 ceph-mon[31658]: pgmap v627: 129 pgs: 129 active+clean; 3.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 3 op/s 2024-05-03T18:47:06.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:47:05 smithi150 ceph-mon[32492]: pgmap v627: 129 pgs: 129 active+clean; 3.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 3 op/s 2024-05-03T18:47:07.196 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-03T18:47:07.662 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:47:08.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:47:07 smithi033 ceph-mon[28330]: pgmap v628: 129 pgs: 129 active+clean; 3.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 46 KiB/s wr, 18 op/s 2024-05-03T18:47:08.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:47:07 smithi099 ceph-mon[31658]: pgmap v628: 129 pgs: 129 active+clean; 3.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 46 KiB/s wr, 18 op/s 2024-05-03T18:47:08.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:47:07 smithi150 ceph-mon[32492]: pgmap v628: 129 pgs: 129 active+clean; 3.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 46 KiB/s wr, 18 op/s 2024-05-03T18:47:10.027 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:47:09 smithi033 ceph-mon[28330]: pgmap v629: 129 pgs: 129 active+clean; 3.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 46 KiB/s wr, 19 op/s 2024-05-03T18:47:10.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.104+0000 7f2789a00700 1 -- 172.21.15.33:0/3590336667 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2784104f10 msgr2=0x7f278410d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:10.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.104+0000 7f2789a00700 1 --2- 172.21.15.33:0/3590336667 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2784104f10 0x7f278410d400 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f276c009990 tx=0x7f276c009ca0 comp rx=0 tx=0).stop 2024-05-03T18:47:10.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.104+0000 7f2789a00700 1 -- 172.21.15.33:0/3590336667 shutdown_connections 2024-05-03T18:47:10.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.104+0000 7f2789a00700 1 --2- 172.21.15.33:0/3590336667 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f278410d9d0 0x7f278410fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:10.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.104+0000 7f2789a00700 1 --2- 172.21.15.33:0/3590336667 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f27841045f0 0x7f27841049d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:10.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.104+0000 7f2789a00700 1 --2- 172.21.15.33:0/3590336667 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2784104f10 0x7f278410d400 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:10.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.104+0000 7f2789a00700 1 -- 172.21.15.33:0/3590336667 >> 172.21.15.33:0/3590336667 conn(0x7f27840fddc0 msgr2=0x7f27841001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:47:10.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.104+0000 7f2789a00700 1 -- 172.21.15.33:0/3590336667 shutdown_connections 2024-05-03T18:47:10.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.104+0000 7f2789a00700 1 -- 172.21.15.33:0/3590336667 wait complete. 2024-05-03T18:47:10.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.105+0000 7f2789a00700 1 Processor -- start 2024-05-03T18:47:10.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.105+0000 7f2789a00700 1 -- start start 2024-05-03T18:47:10.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.106+0000 7f2789a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27841045f0 0x7f2784102250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:10.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.106+0000 7f2789a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2784104f10 0x7f2784102790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:10.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.106+0000 7f2789a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f278410d9d0 0x7f2784102cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:10.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.106+0000 7f2789a00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2784113450 con 0x7f27841045f0 2024-05-03T18:47:10.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.106+0000 7f2789a00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f27841132d0 con 0x7f2784104f10 2024-05-03T18:47:10.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.106+0000 7f2789a00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f27841135d0 con 0x7f278410d9d0 2024-05-03T18:47:10.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.106+0000 7f2783e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27841045f0 0x7f2784102250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:10.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.106+0000 7f2783e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27841045f0 0x7f2784102250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36860/0 (socket says 172.21.15.33:36860) 2024-05-03T18:47:10.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.106+0000 7f2783e00700 1 -- 172.21.15.33:0/4150934595 learned_addr learned my addr 172.21.15.33:0/4150934595 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:47:10.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.106+0000 7f277b400700 1 --2- 172.21.15.33:0/4150934595 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2784104f10 0x7f2784102790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:10.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.107+0000 7f2789000700 1 --2- 172.21.15.33:0/4150934595 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f278410d9d0 0x7f2784102cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:10.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.107+0000 7f2783e00700 1 -- 172.21.15.33:0/4150934595 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f278410d9d0 msgr2=0x7f2784102cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:10.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.107+0000 7f2783e00700 1 --2- 172.21.15.33:0/4150934595 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f278410d9d0 0x7f2784102cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:10.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.107+0000 7f2783e00700 1 -- 172.21.15.33:0/4150934595 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2784104f10 msgr2=0x7f2784102790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:10.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.107+0000 7f2783e00700 1 --2- 172.21.15.33:0/4150934595 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2784104f10 0x7f2784102790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:10.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.107+0000 7f2783e00700 1 -- 172.21.15.33:0/4150934595 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27841a7f20 con 0x7f27841045f0 2024-05-03T18:47:10.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.107+0000 7f277b400700 1 --2- 172.21.15.33:0/4150934595 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2784104f10 0x7f2784102790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:47:10.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.107+0000 7f2783e00700 1 --2- 172.21.15.33:0/4150934595 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27841045f0 0x7f2784102250 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f277400d980 tx=0x7f277400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:47:10.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.107+0000 7f2781600700 1 -- 172.21.15.33:0/4150934595 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2774017070 con 0x7f27841045f0 2024-05-03T18:47:10.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.108+0000 7f2789a00700 1 -- 172.21.15.33:0/4150934595 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27841a8210 con 0x7f27841045f0 2024-05-03T18:47:10.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.108+0000 7f2781600700 1 -- 172.21.15.33:0/4150934595 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2774004830 con 0x7f27841045f0 2024-05-03T18:47:10.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.108+0000 7f2781600700 1 -- 172.21.15.33:0/4150934595 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27740053a0 con 0x7f27841045f0 2024-05-03T18:47:10.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.108+0000 7f2789a00700 1 -- 172.21.15.33:0/4150934595 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27841a8750 con 0x7f27841045f0 2024-05-03T18:47:10.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.112+0000 7f2781600700 1 -- 172.21.15.33:0/4150934595 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f27740049e0 con 0x7f27841045f0 2024-05-03T18:47:10.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.113+0000 7f2795a56b80 1 Processor -- start 2024-05-03T18:47:10.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.113+0000 7f2795a56b80 1 -- start start 2024-05-03T18:47:10.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.113+0000 7f2795a56b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587be2d9bd0 0x5587be2d9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:10.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.113+0000 7f2795a56b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587be2d3f30 0x5587be2d43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:10.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.113+0000 7f2795a56b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587be2d48f0 0x5587be2e0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:10.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.113+0000 7f2795a56b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5587be226a20 con 0x5587be2d9bd0 2024-05-03T18:47:10.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.113+0000 7f2795a56b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5587be1c4760 con 0x5587be2d48f0 2024-05-03T18:47:10.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.113+0000 7f2795a56b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5587be2229e0 con 0x5587be2d3f30 2024-05-03T18:47:10.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.114+0000 7f2789000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587be2d9bd0 0x5587be2d9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:10.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.114+0000 7f2789000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587be2d9bd0 0x5587be2d9fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36870/0 (socket says 172.21.15.33:36870) 2024-05-03T18:47:10.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.114+0000 7f2789000700 1 -- 172.21.15.33:0/2968016328 learned_addr learned my addr 172.21.15.33:0/2968016328 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:47:10.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.114+0000 7f2781600700 1 --2- 172.21.15.33:0/4150934595 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f27680b2a70 0x7f27680b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:10.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.114+0000 7f277b400700 1 --2- 172.21.15.33:0/2968016328 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587be2d48f0 0x5587be2e0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:10.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.114+0000 7f2783e00700 1 --2- 172.21.15.33:0/2968016328 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587be2d3f30 0x5587be2d43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:10.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.114+0000 7f2789000700 1 -- 172.21.15.33:0/2968016328 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587be2d3f30 msgr2=0x5587be2d43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:47:10.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.114+0000 7f2789000700 1 --2- 172.21.15.33:0/2968016328 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587be2d3f30 0x5587be2d43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:10.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.114+0000 7f2789000700 1 -- 172.21.15.33:0/2968016328 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587be2d48f0 msgr2=0x5587be2e0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:47:10.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.114+0000 7f2789000700 1 --2- 172.21.15.33:0/2968016328 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587be2d48f0 0x5587be2e0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:10.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.114+0000 7f2789000700 1 -- 172.21.15.33:0/2968016328 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587be1738d0 con 0x5587be2d9bd0 2024-05-03T18:47:10.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.115+0000 7f2781600700 1 -- 172.21.15.33:0/4150934595 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9518+0+0 (secure 0 0 0) 0x7f27740e2980 con 0x7f27841045f0 2024-05-03T18:47:10.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.115+0000 7f2789000700 1 --2- 172.21.15.33:0/4150934595 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f27680b2a70 0x7f27680b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:10.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.115+0000 7f2789000700 1 --2- 172.21.15.33:0/2968016328 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587be2d9bd0 0x5587be2d9fb0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f277c009e80 tx=0x7f277c0077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ac21cf790f17054 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:47:10.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.115+0000 7f2779600700 1 -- 172.21.15.33:0/2968016328 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f277c018d50 con 0x5587be2d9bd0 2024-05-03T18:47:10.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.115+0000 7f2779600700 1 -- 172.21.15.33:0/2968016328 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f277c0238d0 con 0x5587be2d9bd0 2024-05-03T18:47:10.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.115+0000 7f2779600700 1 -- 172.21.15.33:0/2968016328 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f277c017610 con 0x5587be2d9bd0 2024-05-03T18:47:10.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.116+0000 7f2795a56b80 1 -- 172.21.15.33:0/2968016328 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587be2d9bd0 msgr2=0x5587be2d9fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:10.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.116+0000 7f2789000700 1 --2- 172.21.15.33:0/4150934595 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f27680b2a70 0x7f27680b4f30 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f277c020000 tx=0x7f277c01d000 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:47:10.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.116+0000 7f2795a56b80 1 --2- 172.21.15.33:0/2968016328 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587be2d9bd0 0x5587be2d9fb0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f277c009e80 tx=0x7f277c0077e0 comp rx=0 tx=0).stop 2024-05-03T18:47:10.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.116+0000 7f2795a56b80 1 -- 172.21.15.33:0/2968016328 shutdown_connections 2024-05-03T18:47:10.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.116+0000 7f2795a56b80 1 --2- 172.21.15.33:0/2968016328 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587be2d48f0 0x5587be2e0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:10.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.116+0000 7f2795a56b80 1 --2- 172.21.15.33:0/2968016328 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587be2d9bd0 0x5587be2d9fb0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:10.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.116+0000 7f2795a56b80 1 --2- 172.21.15.33:0/2968016328 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587be2d3f30 0x5587be2d43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:10.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.116+0000 7f2795a56b80 1 -- 172.21.15.33:0/2968016328 >> 172.21.15.33:0/2968016328 conn(0x5587be24c6d0 msgr2=0x5587be2cebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:47:10.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.117+0000 7f2795a56b80 1 -- 172.21.15.33:0/2968016328 shutdown_connections 2024-05-03T18:47:10.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.117+0000 7f2795a56b80 1 -- 172.21.15.33:0/2968016328 wait complete. 2024-05-03T18:47:10.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.117+0000 7f2795a56b80 1 -- start start 2024-05-03T18:47:10.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.117+0000 7f2795a56b80 1 Processor -- start 2024-05-03T18:47:10.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.118+0000 7f2795a56b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587be2d3f30 0x5587be2e0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:10.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.118+0000 7f2795a56b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587be2d48f0 0x5587be2dd980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:10.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.118+0000 7f2795a56b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587be2d9bd0 0x5587be2ddec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:10.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.118+0000 7f2795a56b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5587be2e0f30 con 0x5587be2d48f0 2024-05-03T18:47:10.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.118+0000 7f2795a56b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5587be18cb20 con 0x5587be2d3f30 2024-05-03T18:47:10.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.118+0000 7f2795a56b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5587be2e10b0 con 0x5587be2d9bd0 2024-05-03T18:47:10.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.118+0000 7f2795a56b80 10 client.0 _ll_register_callbacks cb 0x5587be1cdc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:47:10.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.118+0000 7f277b400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587be2d48f0 0x5587be2dd980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:10.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.118+0000 7f277b400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587be2d48f0 0x5587be2dd980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36876/0 (socket says 172.21.15.33:36876) 2024-05-03T18:47:10.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.118+0000 7f277b400700 1 -- 172.21.15.33:0/3193095324 learned_addr learned my addr 172.21.15.33:0/3193095324 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:47:10.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.119+0000 7f2783e00700 1 --2- 172.21.15.33:0/3193095324 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587be2d3f30 0x5587be2e0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:10.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.119+0000 7f2789000700 1 --2- 172.21.15.33:0/3193095324 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587be2d9bd0 0x5587be2ddec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:10.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.119+0000 7f277b400700 1 -- 172.21.15.33:0/3193095324 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587be2d9bd0 msgr2=0x5587be2ddec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:47:10.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.119+0000 7f277b400700 1 --2- 172.21.15.33:0/3193095324 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587be2d9bd0 0x5587be2ddec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:10.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.119+0000 7f277b400700 1 -- 172.21.15.33:0/3193095324 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587be2d3f30 msgr2=0x5587be2e0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:47:10.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.119+0000 7f277b400700 1 --2- 172.21.15.33:0/3193095324 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587be2d3f30 0x5587be2e0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:10.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.119+0000 7f277b400700 1 -- 172.21.15.33:0/3193095324 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587be1ec420 con 0x5587be2d48f0 2024-05-03T18:47:10.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.119+0000 7f2789000700 1 --2- 172.21.15.33:0/3193095324 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587be2d9bd0 0x5587be2ddec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:47:10.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.119+0000 7f277b400700 1 --2- 172.21.15.33:0/3193095324 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587be2d48f0 0x5587be2dd980 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f2784105370 tx=0x7f276c009ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50e69c2e64964a86 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:47:10.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.119+0000 7f2772000700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:47:10.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.120+0000 7f2772000700 1 -- 172.21.15.33:0/3193095324 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f276c003bd0 con 0x5587be2d48f0 2024-05-03T18:47:10.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.120+0000 7f2772000700 1 -- 172.21.15.33:0/3193095324 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9518+0+0 (secure 0 0 0) 0x7f276c003d70 con 0x5587be2d48f0 2024-05-03T18:47:10.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.120+0000 7f2795a56b80 1 -- 172.21.15.33:0/3193095324 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f277c0238d0 con 0x5587be2d48f0 2024-05-03T18:47:10.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.120+0000 7f2772000700 1 -- 172.21.15.33:0/3193095324 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f276c031050 con 0x5587be2d48f0 2024-05-03T18:47:10.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.120+0000 7f2795a56b80 10 client.17088 fetch_fsmap learned FSMap version 21 2024-05-03T18:47:10.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.120+0000 7f2795a56b80 1 -- 172.21.15.33:0/3193095324 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5587be2eb740 con 0x5587be2d48f0 2024-05-03T18:47:10.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.120+0000 7f2772000700 1 -- 172.21.15.33:0/3193095324 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f277c0238d0 con 0x5587be2d48f0 2024-05-03T18:47:10.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.121+0000 7f2772000700 1 -- 172.21.15.33:0/3193095324 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f276c023770 con 0x5587be2d48f0 2024-05-03T18:47:10.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.121+0000 7f2795a56b80 10 client.17088 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:47:10.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.121+0000 7f2795a56b80 10 client.17088 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:47:10.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.121+0000 7f2795a56b80 20 client.17088 populate_metadata read hostname 'smithi033' 2024-05-03T18:47:10.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.121+0000 7f2795a56b80 1 --2- 172.21.15.33:0/3193095324 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5587be2eddd0 0x5587be2f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:10.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.121+0000 7f2795a56b80 4 client.17088 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:47:10.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.121+0000 7f2795a56b80 1 -- 172.21.15.33:0/3193095324 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5587be2a0ff0 con 0x5587be2eddd0 2024-05-03T18:47:10.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.122+0000 7f2783e00700 1 --2- 172.21.15.33:0/3193095324 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5587be2eddd0 0x5587be2f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:10.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.122+0000 7f2783e00700 1 --2- 172.21.15.33:0/3193095324 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5587be2eddd0 0x5587be2f0200 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69634a5245bd260c server_cookie=df4cbb5cd012356c in_seq=0 out_seq=0 2024-05-03T18:47:10.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.122+0000 7f2772000700 10 client.17088 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:47:10.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.125+0000 7f2772000700 1 -- 172.21.15.33:0/3193095324 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5587be2a0ff0 con 0x5587be2eddd0 2024-05-03T18:47:10.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.125+0000 7f2772000700 10 client.17088 handle_command_reply: tid=0 2024-05-03T18:47:10.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.125+0000 7f2795a56b80 1 client.17088 shutdown 2024-05-03T18:47:10.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.125+0000 7f2795a56b80 1 -- 172.21.15.33:0/3193095324 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587be2d48f0 msgr2=0x5587be2dd980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:10.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.125+0000 7f2795a56b80 1 --2- 172.21.15.33:0/3193095324 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587be2d48f0 0x5587be2dd980 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f2784105370 tx=0x7f276c009ee0 comp rx=0 tx=0).stop 2024-05-03T18:47:10.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.125+0000 7f2795a56b80 1 -- 172.21.15.33:0/3193095324 shutdown_connections 2024-05-03T18:47:10.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.125+0000 7f2795a56b80 1 --2- 172.21.15.33:0/3193095324 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5587be2eddd0 0x5587be2f0200 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:10.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.125+0000 7f2795a56b80 1 --2- 172.21.15.33:0/3193095324 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587be2d9bd0 0x5587be2ddec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:10.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.125+0000 7f2795a56b80 1 --2- 172.21.15.33:0/3193095324 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587be2d3f30 0x5587be2e0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:10.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.125+0000 7f2795a56b80 1 --2- 172.21.15.33:0/3193095324 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587be2d48f0 0x5587be2dd980 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:10.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.125+0000 7f2772000700 0 client.17088 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:47:10.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.125+0000 7f2795a56b80 1 -- 172.21.15.33:0/3193095324 >> 172.21.15.33:0/3193095324 conn(0x5587be24c6d0 msgr2=0x5587be2d2f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:47:10.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.125+0000 7f2795a56b80 1 -- 172.21.15.33:0/3193095324 shutdown_connections 2024-05-03T18:47:10.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.126+0000 7f2795a56b80 1 -- 172.21.15.33:0/3193095324 wait complete. 2024-05-03T18:47:10.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.126+0000 7f2795a56b80 20 client.17088 trim_cache size 0 max 16384 2024-05-03T18:47:10.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.148+0000 7f2795a56b80 1 Processor -- start 2024-05-03T18:47:10.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.148+0000 7f2795a56b80 1 -- start start 2024-05-03T18:47:10.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.149+0000 7f2795a56b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587be2d3f30 0x5587be2d9250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:10.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.149+0000 7f2795a56b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587be2d48f0 0x5587be2e9f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:10.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.149+0000 7f2795a56b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587be2d9bd0 0x5587be2ced20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:10.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.149+0000 7f2795a56b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f27841af4e0 con 0x5587be2d9bd0 2024-05-03T18:47:10.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.149+0000 7f2795a56b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f27841b1d40 con 0x5587be2d3f30 2024-05-03T18:47:10.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.149+0000 7f2795a56b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f27841afb30 con 0x5587be2d48f0 2024-05-03T18:47:10.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.149+0000 7f2789000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587be2d9bd0 0x5587be2ced20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:10.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.149+0000 7f2789000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587be2d9bd0 0x5587be2ced20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36886/0 (socket says 172.21.15.33:36886) 2024-05-03T18:47:10.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.149+0000 7f2789000700 1 -- 172.21.15.33:0/3819977147 learned_addr learned my addr 172.21.15.33:0/3819977147 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:47:10.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.149+0000 7f2783e00700 1 --2- 172.21.15.33:0/3819977147 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587be2d3f30 0x5587be2d9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:10.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.150+0000 7f277b400700 1 --2- 172.21.15.33:0/3819977147 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587be2d48f0 0x5587be2e9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:10.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.150+0000 7f2789000700 1 -- 172.21.15.33:0/3819977147 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587be2d48f0 msgr2=0x5587be2e9f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:47:10.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.150+0000 7f2789000700 1 --2- 172.21.15.33:0/3819977147 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587be2d48f0 0x5587be2e9f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:10.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.150+0000 7f2789000700 1 -- 172.21.15.33:0/3819977147 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587be2d3f30 msgr2=0x5587be2d9250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:47:10.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.150+0000 7f2789000700 1 --2- 172.21.15.33:0/3819977147 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587be2d3f30 0x5587be2d9250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:10.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.150+0000 7f2789000700 1 -- 172.21.15.33:0/3819977147 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587be2c6ef0 con 0x5587be2d9bd0 2024-05-03T18:47:10.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.150+0000 7f2783e00700 1 --2- 172.21.15.33:0/3819977147 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587be2d3f30 0x5587be2d9250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:47:10.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.150+0000 7f2789000700 1 --2- 172.21.15.33:0/3819977147 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587be2d9bd0 0x5587be2ced20 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f277c0091f0 tx=0x7f277c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b819220793b076b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:47:10.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.151+0000 7f2778c00700 1 -- 172.21.15.33:0/3819977147 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f277c031070 con 0x5587be2d9bd0 2024-05-03T18:47:10.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.151+0000 7f2778c00700 1 -- 172.21.15.33:0/3819977147 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f277c018cd0 con 0x5587be2d9bd0 2024-05-03T18:47:10.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.151+0000 7f2778c00700 1 -- 172.21.15.33:0/3819977147 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f277c028040 con 0x5587be2d9bd0 2024-05-03T18:47:10.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.151+0000 7f2795a56b80 1 -- 172.21.15.33:0/3819977147 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587be2d9bd0 msgr2=0x5587be2ced20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:10.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.151+0000 7f2795a56b80 1 --2- 172.21.15.33:0/3819977147 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587be2d9bd0 0x5587be2ced20 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f277c0091f0 tx=0x7f277c012000 comp rx=0 tx=0).stop 2024-05-03T18:47:10.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.151+0000 7f2795a56b80 1 -- 172.21.15.33:0/3819977147 shutdown_connections 2024-05-03T18:47:10.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.151+0000 7f2795a56b80 1 --2- 172.21.15.33:0/3819977147 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587be2d9bd0 0x5587be2ced20 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:10.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.151+0000 7f2795a56b80 1 --2- 172.21.15.33:0/3819977147 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587be2d3f30 0x5587be2d9250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:10.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.151+0000 7f2795a56b80 1 --2- 172.21.15.33:0/3819977147 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587be2d48f0 0x5587be2e9f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:10.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.151+0000 7f2795a56b80 1 -- 172.21.15.33:0/3819977147 >> 172.21.15.33:0/3819977147 conn(0x5587be24c6d0 msgr2=0x5587be2d8b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:47:10.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.152+0000 7f2795a56b80 1 -- 172.21.15.33:0/3819977147 shutdown_connections 2024-05-03T18:47:10.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.152+0000 7f2795a56b80 1 -- 172.21.15.33:0/3819977147 wait complete. 2024-05-03T18:47:10.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.152+0000 7f2795a56b80 1 -- start start 2024-05-03T18:47:10.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.152+0000 7f2795a56b80 1 Processor -- start 2024-05-03T18:47:10.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.153+0000 7f2795a56b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587be2d3f30 0x5587be2e6300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:10.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.153+0000 7f2795a56b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587be2d48f0 0x5587be2e6840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:10.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.153+0000 7f2795a56b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587be2d9bd0 0x5587be2e32d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:10.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.153+0000 7f2795a56b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f27841b0200 con 0x5587be2d3f30 2024-05-03T18:47:10.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.153+0000 7f2795a56b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f27841b1670 con 0x5587be2d48f0 2024-05-03T18:47:10.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.153+0000 7f2795a56b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f27841b08d0 con 0x5587be2d9bd0 2024-05-03T18:47:10.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.153+0000 7f2795a56b80 10 client.0 _ll_register_callbacks cb 0x5587be1cdc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:47:10.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.153+0000 7f2783e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587be2d3f30 0x5587be2e6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:10.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.153+0000 7f2783e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587be2d3f30 0x5587be2e6300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36898/0 (socket says 172.21.15.33:36898) 2024-05-03T18:47:10.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.153+0000 7f2783e00700 1 -- 172.21.15.33:0/3441383297 learned_addr learned my addr 172.21.15.33:0/3441383297 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:47:10.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.153+0000 7f277b400700 1 --2- 172.21.15.33:0/3441383297 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587be2d48f0 0x5587be2e6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:10.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.153+0000 7f2783e00700 1 -- 172.21.15.33:0/3441383297 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587be2d9bd0 msgr2=0x5587be2e32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:47:10.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.153+0000 7f2789000700 1 --2- 172.21.15.33:0/3441383297 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587be2d9bd0 0x5587be2e32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:10.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.154+0000 7f2783e00700 1 --2- 172.21.15.33:0/3441383297 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587be2d9bd0 0x5587be2e32d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:10.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.154+0000 7f2783e00700 1 -- 172.21.15.33:0/3441383297 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587be2d48f0 msgr2=0x5587be2e6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:47:10.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.154+0000 7f2783e00700 1 --2- 172.21.15.33:0/3441383297 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587be2d48f0 0x5587be2e6840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:10.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.154+0000 7f2783e00700 1 -- 172.21.15.33:0/3441383297 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587be2e3990 con 0x5587be2d3f30 2024-05-03T18:47:10.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.154+0000 7f277b400700 1 --2- 172.21.15.33:0/3441383297 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587be2d48f0 0x5587be2e6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:47:10.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.154+0000 7f2789000700 1 --2- 172.21.15.33:0/3441383297 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587be2d9bd0 0x5587be2e32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:47:10.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.154+0000 7f2783e00700 1 --2- 172.21.15.33:0/3441383297 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587be2d3f30 0x5587be2e6300 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f277400bfd0 tx=0x7f27740e4dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5ec640a8d891255 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:47:10.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.154+0000 7f2772000700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:47:10.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.154+0000 7f2772000700 1 -- 172.21.15.33:0/3441383297 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27740e0110 con 0x5587be2d3f30 2024-05-03T18:47:10.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.154+0000 7f2772000700 1 -- 172.21.15.33:0/3441383297 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9518+0+0 (secure 0 0 0) 0x7f27740a9a80 con 0x5587be2d3f30 2024-05-03T18:47:10.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.154+0000 7f2795a56b80 1 -- 172.21.15.33:0/3441383297 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f277c018cd0 con 0x5587be2d3f30 2024-05-03T18:47:10.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.155+0000 7f2772000700 1 -- 172.21.15.33:0/3441383297 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f27740a9820 con 0x5587be2d3f30 2024-05-03T18:47:10.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.155+0000 7f2795a56b80 10 client.17094 fetch_fsmap learned FSMap version 21 2024-05-03T18:47:10.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.155+0000 7f2795a56b80 1 -- 172.21.15.33:0/3441383297 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5587be3590c0 con 0x5587be2d3f30 2024-05-03T18:47:10.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.155+0000 7f2772000700 1 -- 172.21.15.33:0/3441383297 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f277c018cd0 con 0x5587be2d3f30 2024-05-03T18:47:10.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.155+0000 7f2772000700 1 -- 172.21.15.33:0/3441383297 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f27740b21f0 con 0x5587be2d3f30 2024-05-03T18:47:10.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.155+0000 7f2795a56b80 10 client.17094 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:47:10.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.155+0000 7f2795a56b80 10 client.17094 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:47:10.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.155+0000 7f2795a56b80 20 client.17094 populate_metadata read hostname 'smithi033' 2024-05-03T18:47:10.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.155+0000 7f2795a56b80 1 --2- 172.21.15.33:0/3441383297 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5587be359750 0x5587be35bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:10.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.155+0000 7f2795a56b80 4 client.17094 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-03T18:47:10.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.155+0000 7f2795a56b80 1 -- 172.21.15.33:0/3441383297 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5587be234400 con 0x5587be359750 2024-05-03T18:47:10.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.156+0000 7f277b400700 1 --2- 172.21.15.33:0/3441383297 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5587be359750 0x5587be35bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:10.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.157+0000 7f277b400700 1 --2- 172.21.15.33:0/3441383297 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5587be359750 0x5587be35bb70 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18378732ea3c883c server_cookie=5ba3b4d042da5a5e in_seq=0 out_seq=0 2024-05-03T18:47:10.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.157+0000 7f2772000700 10 client.17094 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:47:10.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.157+0000 7f2772000700 1 -- 172.21.15.33:0/3441383297 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5587be234400 con 0x5587be359750 2024-05-03T18:47:10.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.157+0000 7f2772000700 10 client.17094 handle_command_reply: tid=0 2024-05-03T18:47:10.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.157+0000 7f2795a56b80 1 client.17094 shutdown 2024-05-03T18:47:10.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.157+0000 7f2795a56b80 1 -- 172.21.15.33:0/3441383297 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587be2d3f30 msgr2=0x5587be2e6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:10.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.157+0000 7f2795a56b80 1 --2- 172.21.15.33:0/3441383297 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587be2d3f30 0x5587be2e6300 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f277400bfd0 tx=0x7f27740e4dd0 comp rx=0 tx=0).stop 2024-05-03T18:47:10.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.158+0000 7f2795a56b80 1 -- 172.21.15.33:0/3441383297 shutdown_connections 2024-05-03T18:47:10.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.158+0000 7f2795a56b80 1 --2- 172.21.15.33:0/3441383297 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x5587be359750 0x5587be35bb70 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:10.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.158+0000 7f2795a56b80 1 --2- 172.21.15.33:0/3441383297 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587be2d9bd0 0x5587be2e32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:10.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.158+0000 7f2795a56b80 1 --2- 172.21.15.33:0/3441383297 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587be2d3f30 0x5587be2e6300 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:10.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.158+0000 7f2795a56b80 1 --2- 172.21.15.33:0/3441383297 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587be2d48f0 0x5587be2e6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:10.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.158+0000 7f2795a56b80 1 -- 172.21.15.33:0/3441383297 >> 172.21.15.33:0/3441383297 conn(0x5587be24c6d0 msgr2=0x5587be2d2f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:47:10.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.158+0000 7f2772000700 0 client.17094 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:47:10.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.158+0000 7f2795a56b80 1 -- 172.21.15.33:0/3441383297 shutdown_connections 2024-05-03T18:47:10.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.158+0000 7f2795a56b80 1 -- 172.21.15.33:0/3441383297 wait complete. 2024-05-03T18:47:10.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.158+0000 7f2795a56b80 20 client.17094 trim_cache size 0 max 16384 2024-05-03T18:47:10.163 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-03T18:47:10.163 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-05-03T18:47:10.163 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-05-03T18:47:10.163 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-03T18:47:10.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.159+0000 7f2789a00700 1 -- 172.21.15.33:0/4150934595 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f27680b2a70 msgr2=0x7f27680b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:10.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.159+0000 7f2789a00700 1 --2- 172.21.15.33:0/4150934595 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f27680b2a70 0x7f27680b4f30 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f277c020000 tx=0x7f277c01d000 comp rx=0 tx=0).stop 2024-05-03T18:47:10.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.159+0000 7f2789a00700 1 -- 172.21.15.33:0/4150934595 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27841045f0 msgr2=0x7f2784102250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:10.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.159+0000 7f2789a00700 1 --2- 172.21.15.33:0/4150934595 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27841045f0 0x7f2784102250 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f277400d980 tx=0x7f277400dd40 comp rx=0 tx=0).stop 2024-05-03T18:47:10.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.159+0000 7f2789a00700 1 -- 172.21.15.33:0/4150934595 shutdown_connections 2024-05-03T18:47:10.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.159+0000 7f2789a00700 1 --2- 172.21.15.33:0/4150934595 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f278410d9d0 0x7f2784102cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:10.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.159+0000 7f2789a00700 1 --2- 172.21.15.33:0/4150934595 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27841045f0 0x7f2784102250 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:10.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.159+0000 7f2789a00700 1 --2- 172.21.15.33:0/4150934595 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f27680b2a70 0x7f27680b4f30 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:10.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.159+0000 7f2789a00700 1 --2- 172.21.15.33:0/4150934595 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2784104f10 0x7f2784102790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:10.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.159+0000 7f2789a00700 1 -- 172.21.15.33:0/4150934595 >> 172.21.15.33:0/4150934595 conn(0x7f27840fddc0 msgr2=0x7f27840feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:47:10.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.159+0000 7f2789a00700 1 -- 172.21.15.33:0/4150934595 shutdown_connections 2024-05-03T18:47:10.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:10.159+0000 7f2789a00700 1 -- 172.21.15.33:0/4150934595 wait complete. 2024-05-03T18:47:10.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:47:09 smithi099 ceph-mon[31658]: pgmap v629: 129 pgs: 129 active+clean; 3.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 46 KiB/s wr, 19 op/s 2024-05-03T18:47:10.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:47:09 smithi150 ceph-mon[32492]: pgmap v629: 129 pgs: 129 active+clean; 3.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 46 KiB/s wr, 19 op/s 2024-05-03T18:47:11.172 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-03T18:47:11.172 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T18:47:11.616 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:47:12.022 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:47:11 smithi033 ceph-mon[28330]: pgmap v630: 129 pgs: 129 active+clean; 3.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 46 KiB/s wr, 19 op/s 2024-05-03T18:47:12.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:47:11 smithi099 ceph-mon[31658]: pgmap v630: 129 pgs: 129 active+clean; 3.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 46 KiB/s wr, 19 op/s 2024-05-03T18:47:12.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:47:11 smithi150 ceph-mon[32492]: pgmap v630: 129 pgs: 129 active+clean; 3.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 46 KiB/s wr, 19 op/s 2024-05-03T18:47:13.236 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:47:12 smithi033 ceph-mon[28330]: osdmap e119: 12 total, 12 up, 12 in 2024-05-03T18:47:13.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:47:12 smithi099 ceph-mon[31658]: osdmap e119: 12 total, 12 up, 12 in 2024-05-03T18:47:13.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:47:12 smithi150 ceph-mon[32492]: osdmap e119: 12 total, 12 up, 12 in 2024-05-03T18:47:14.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:47:13 smithi033 ceph-mon[28330]: pgmap v632: 129 pgs: 129 active+clean; 3.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 56 KiB/s wr, 22 op/s 2024-05-03T18:47:14.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:47:13 smithi099 ceph-mon[31658]: pgmap v632: 129 pgs: 129 active+clean; 3.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 56 KiB/s wr, 22 op/s 2024-05-03T18:47:14.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:47:13 smithi150 ceph-mon[32492]: pgmap v632: 129 pgs: 129 active+clean; 3.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 56 KiB/s wr, 22 op/s 2024-05-03T18:47:15.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:15.113+0000 7fb9b7a00700 1 -- 172.21.15.33:0/2756023048 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9b006b180 msgr2=0x7fb9b006b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:15.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:15.113+0000 7fb9b7a00700 1 --2- 172.21.15.33:0/2756023048 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9b006b180 0x7fb9b006b600 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fb998009990 tx=0x7fb998009ca0 comp rx=0 tx=0).stop 2024-05-03T18:47:15.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:15.113+0000 7fb9b7a00700 1 -- 172.21.15.33:0/2756023048 shutdown_connections 2024-05-03T18:47:15.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:15.113+0000 7fb9b7a00700 1 --2- 172.21.15.33:0/2756023048 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb9b006bb40 0x7fb9b0110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:15.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:15.113+0000 7fb9b7a00700 1 --2- 172.21.15.33:0/2756023048 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb9b0109510 0x7fb9b01098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:15.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:15.113+0000 7fb9b7a00700 1 --2- 172.21.15.33:0/2756023048 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9b006b180 0x7fb9b006b600 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:15.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:15.113+0000 7fb9b7a00700 1 -- 172.21.15.33:0/2756023048 >> 172.21.15.33:0/2756023048 conn(0x7fb9b0079350 msgr2=0x7fb9b0079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:47:15.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:15.113+0000 7fb9b7a00700 1 -- 172.21.15.33:0/2756023048 shutdown_connections 2024-05-03T18:47:15.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:15.113+0000 7fb9b7a00700 1 -- 172.21.15.33:0/2756023048 wait complete. 2024-05-03T18:47:15.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:15.115+0000 7fb9b7a00700 1 Processor -- start 2024-05-03T18:47:15.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:15.115+0000 7fb9b7a00700 1 -- start start 2024-05-03T18:47:15.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:15.115+0000 7fb9b7a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb9b006bb40 0x7fb9b019f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:15.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:15.115+0000 7fb9b7a00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb9b0109510 0x7fb9b019fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:15.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:15.115+0000 7fb9b7a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9b0199af0 0x7fb9b0199f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:15.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:15.115+0000 7fb9b7a00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b0113520 con 0x7fb9b0199af0 2024-05-03T18:47:15.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:15.115+0000 7fb9b7a00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b01133a0 con 0x7fb9b006bb40 2024-05-03T18:47:15.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:15.115+0000 7fb9b7a00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b01136a0 con 0x7fb9b0109510 2024-05-03T18:47:15.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:15.116+0000 7fb9b5a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9b0199af0 0x7fb9b0199f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:15.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:15.116+0000 7fb9b5a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9b0199af0 0x7fb9b0199f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36910/0 (socket says 172.21.15.33:36910) 2024-05-03T18:47:15.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:15.116+0000 7fb9b5a00700 1 -- 172.21.15.33:0/1299065051 learned_addr learned my addr 172.21.15.33:0/1299065051 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:47:15.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:15.116+0000 7fb9a7e00700 1 --2- 172.21.15.33:0/1299065051 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb9b0109510 0x7fb9b019fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:15.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:15.116+0000 7fb9b5000700 1 --2- 172.21.15.33:0/1299065051 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb9b006bb40 0x7fb9b019f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:15.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:15.116+0000 7fb9b5a00700 1 -- 172.21.15.33:0/1299065051 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb9b0109510 msgr2=0x7fb9b019fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:15.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:15.116+0000 7fb9b5a00700 1 --2- 172.21.15.33:0/1299065051 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb9b0109510 0x7fb9b019fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:15.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:15.116+0000 7fb9b5a00700 1 -- 172.21.15.33:0/1299065051 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb9b006bb40 msgr2=0x7fb9b019f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:15.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:15.116+0000 7fb9b5a00700 1 --2- 172.21.15.33:0/1299065051 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb9b006bb40 0x7fb9b019f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:15.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:15.116+0000 7fb9b5a00700 1 -- 172.21.15.33:0/1299065051 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9b019a670 con 0x7fb9b0199af0 2024-05-03T18:47:15.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:15.116+0000 7fb9a7e00700 1 --2- 172.21.15.33:0/1299065051 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb9b0109510 0x7fb9b019fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:47:15.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:15.117+0000 7fb9b5a00700 1 --2- 172.21.15.33:0/1299065051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9b0199af0 0x7fb9b0199f70 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fb9a8007d70 tx=0x7fb9a800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:47:15.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:15.117+0000 7fb9ae000700 1 -- 172.21.15.33:0/1299065051 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a8016020 con 0x7fb9b0199af0 2024-05-03T18:47:15.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:15.117+0000 7fb9ae000700 1 -- 172.21.15.33:0/1299065051 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb9a800ad40 con 0x7fb9b0199af0 2024-05-03T18:47:15.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:15.117+0000 7fb9ae000700 1 -- 172.21.15.33:0/1299065051 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a80115e0 con 0x7fb9b0199af0 2024-05-03T18:47:15.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:15.117+0000 7fb9b7a00700 1 -- 172.21.15.33:0/1299065051 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9b01a6800 con 0x7fb9b0199af0 2024-05-03T18:47:15.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:15.118+0000 7fb9b7a00700 1 -- 172.21.15.33:0/1299065051 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9b01a6ba0 con 0x7fb9b0199af0 2024-05-03T18:47:15.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:15.119+0000 7fb9b7a00700 1 -- 172.21.15.33:0/1299065051 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9b006a190 con 0x7fb9b0199af0 2024-05-03T18:47:15.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:15.121+0000 7fb9ae000700 1 -- 172.21.15.33:0/1299065051 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fb9a8019020 con 0x7fb9b0199af0 2024-05-03T18:47:15.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:15.123+0000 7fb9ae000700 1 --2- 172.21.15.33:0/1299065051 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fb9900b2a70 0x7fb9900b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:15.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:15.123+0000 7fb9ae000700 1 -- 172.21.15.33:0/1299065051 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 9606+0+0 (secure 0 0 0) 0x7fb9a80e2d30 con 0x7fb9b0199af0 2024-05-03T18:47:15.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:15.123+0000 7fb9b5000700 1 --2- 172.21.15.33:0/1299065051 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fb9900b2a70 0x7fb9900b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:15.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:15.124+0000 7fb9b5000700 1 --2- 172.21.15.33:0/1299065051 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fb9900b2a70 0x7fb9900b4f30 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb9a0005950 tx=0x7fb9a00058e0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:47:15.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:15.126+0000 7fb9ae000700 1 -- 172.21.15.33:0/1299065051 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fb9a8020020 con 0x7fb9b0199af0 2024-05-03T18:47:15.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:15.423+0000 7fb9b7a00700 1 -- 172.21.15.33:0/1299065051 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb9b002e510 con 0x7fb9b0199af0 2024-05-03T18:47:15.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:15.425+0000 7fb9ae000700 1 -- 172.21.15.33:0/1299065051 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11801 (secure 0 0 0) 0x7fb9a80aa380 con 0x7fb9b0199af0 2024-05-03T18:47:15.426 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:47:15.427 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1327393503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1327393503},{"type":"v1","addr":"172.21.15.33:6835","nonce":1327393503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/2461472948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":2461472948},{"type":"v1","addr":"172.21.15.33:6837","nonce":2461472948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14559,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/442269281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":442269281},{"type":"v1","addr":"172.21.15.33:6839","nonce":442269281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14571,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/645571756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":645571756},{"type":"v1","addr":"172.21.15.33:6841","nonce":645571756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/203598909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":203598909},{"type":"v1","addr":"172.21.15.150:6833","nonce":203598909}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/4029842493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":4029842493},{"type":"v1","addr":"172.21.15.150:6835","nonce":4029842493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3463664423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3463664423},{"type":"v1","addr":"172.21.15.150:6837","nonce":3463664423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24481,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6839/4051648018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":4051648018},{"type":"v1","addr":"172.21.15.150:6839","nonce":4051648018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34418,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1865105479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1865105479},{"type":"v1","addr":"172.21.15.99:6833","nonce":1865105479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34445,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/1206393914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1206393914},{"type":"v1","addr":"172.21.15.99:6835","nonce":1206393914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-03T18:22:28.837091+0000","modified":"2024-05-03T18:23:49.758398+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34508},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34472":{"gid":34472,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6837/3360970342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3360970342},{"type":"v1","addr":"172.21.15.99:6837","nonce":3360970342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34508":{"gid":34508,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/919907308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":919907308},{"type":"v1","addr":"172.21.15.99:6839","nonce":919907308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34508,"qdb_cluster":[34508]},"id":1}]} 2024-05-03T18:47:15.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:15.428+0000 7fb9b7a00700 1 -- 172.21.15.33:0/1299065051 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fb9900b2a70 msgr2=0x7fb9900b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:15.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:15.428+0000 7fb9b7a00700 1 --2- 172.21.15.33:0/1299065051 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fb9900b2a70 0x7fb9900b4f30 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb9a0005950 tx=0x7fb9a00058e0 comp rx=0 tx=0).stop 2024-05-03T18:47:15.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:15.428+0000 7fb9b7a00700 1 -- 172.21.15.33:0/1299065051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9b0199af0 msgr2=0x7fb9b0199f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:15.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:15.428+0000 7fb9b7a00700 1 --2- 172.21.15.33:0/1299065051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9b0199af0 0x7fb9b0199f70 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fb9a8007d70 tx=0x7fb9a800a340 comp rx=0 tx=0).stop 2024-05-03T18:47:15.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:15.428+0000 7fb9b7a00700 1 -- 172.21.15.33:0/1299065051 shutdown_connections 2024-05-03T18:47:15.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:15.428+0000 7fb9b7a00700 1 --2- 172.21.15.33:0/1299065051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9b0199af0 0x7fb9b0199f70 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:15.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:15.428+0000 7fb9b7a00700 1 --2- 172.21.15.33:0/1299065051 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fb9900b2a70 0x7fb9900b4f30 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:15.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:15.428+0000 7fb9b7a00700 1 --2- 172.21.15.33:0/1299065051 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb9b006bb40 0x7fb9b019f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:15.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:15.428+0000 7fb9b7a00700 1 --2- 172.21.15.33:0/1299065051 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb9b0109510 0x7fb9b019fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:15.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:15.428+0000 7fb9b7a00700 1 -- 172.21.15.33:0/1299065051 >> 172.21.15.33:0/1299065051 conn(0x7fb9b0079350 msgr2=0x7fb9b0101320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:47:15.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:15.429+0000 7fb9b7a00700 1 -- 172.21.15.33:0/1299065051 shutdown_connections 2024-05-03T18:47:15.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:15.429+0000 7fb9b7a00700 1 -- 172.21.15.33:0/1299065051 wait complete. 2024-05-03T18:47:15.431 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 21 2024-05-03T18:47:16.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:47:15 smithi099 ceph-mon[31658]: pgmap v633: 129 pgs: 129 active+clean; 3.6 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-05-03T18:47:16.297 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:47:15 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/1299065051' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:47:16.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:47:15 smithi150 ceph-mon[32492]: pgmap v633: 129 pgs: 129 active+clean; 3.6 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-05-03T18:47:16.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:47:15 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/1299065051' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:47:16.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:47:15 smithi033 ceph-mon[28330]: pgmap v633: 129 pgs: 129 active+clean; 3.6 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-05-03T18:47:16.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:47:15 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/1299065051' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:47:16.633 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-03T18:47:17.110 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:47:18.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:47:17 smithi099 ceph-mon[31658]: pgmap v634: 129 pgs: 129 active+clean; 3.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 307 B/s wr, 2 op/s 2024-05-03T18:47:18.297 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:47:17 smithi099 ceph-mon[31658]: osdmap e120: 12 total, 12 up, 12 in 2024-05-03T18:47:18.301 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:47:17 smithi033 ceph-mon[28330]: pgmap v634: 129 pgs: 129 active+clean; 3.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 307 B/s wr, 2 op/s 2024-05-03T18:47:18.301 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:47:17 smithi033 ceph-mon[28330]: osdmap e120: 12 total, 12 up, 12 in 2024-05-03T18:47:18.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:47:17 smithi150 ceph-mon[32492]: pgmap v634: 129 pgs: 129 active+clean; 3.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 307 B/s wr, 2 op/s 2024-05-03T18:47:18.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:47:17 smithi150 ceph-mon[32492]: osdmap e120: 12 total, 12 up, 12 in 2024-05-03T18:47:19.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.379+0000 7f98d4600700 1 -- 172.21.15.33:0/3689135520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98cc106080 msgr2=0x7f98cc112590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:19.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.379+0000 7f98d4600700 1 --2- 172.21.15.33:0/3689135520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98cc106080 0x7f98cc112590 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f98c400b0a0 tx=0x7f98c400b3b0 comp rx=0 tx=0).stop 2024-05-03T18:47:19.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.379+0000 7f98d4600700 1 -- 172.21.15.33:0/3689135520 shutdown_connections 2024-05-03T18:47:19.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.379+0000 7f98d4600700 1 --2- 172.21.15.33:0/3689135520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98cc106080 0x7f98cc112590 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:19.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.379+0000 7f98d4600700 1 --2- 172.21.15.33:0/3689135520 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f98cc10b6c0 0x7f98cc10baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:19.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.379+0000 7f98d4600700 1 --2- 172.21.15.33:0/3689135520 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f98cc1056c0 0x7f98cc105b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:19.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.379+0000 7f98d4600700 1 -- 172.21.15.33:0/3689135520 >> 172.21.15.33:0/3689135520 conn(0x7f98cc1011b0 msgr2=0x7f98cc1035d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:47:19.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.379+0000 7f98d4600700 1 -- 172.21.15.33:0/3689135520 shutdown_connections 2024-05-03T18:47:19.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.379+0000 7f98d4600700 1 -- 172.21.15.33:0/3689135520 wait complete. 2024-05-03T18:47:19.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.381+0000 7f98d4600700 1 Processor -- start 2024-05-03T18:47:19.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.381+0000 7f98d4600700 1 -- start start 2024-05-03T18:47:19.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.381+0000 7f98d4600700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98cc1056c0 0x7f98cc1a17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:19.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.381+0000 7f98d4600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f98cc106080 0x7f98cc1a1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:19.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.381+0000 7f98d4600700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f98cc10b6c0 0x7f98cc19b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:19.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.381+0000 7f98d4600700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f98cc1157b0 con 0x7f98cc1056c0 2024-05-03T18:47:19.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.381+0000 7f98d4600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f98cc115630 con 0x7f98cc106080 2024-05-03T18:47:19.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.381+0000 7f98d4600700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f98cc115930 con 0x7f98cc10b6c0 2024-05-03T18:47:19.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.382+0000 7f98d1c00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98cc1056c0 0x7f98cc1a17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:19.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.382+0000 7f98d1c00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98cc1056c0 0x7f98cc1a17f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53314/0 (socket says 172.21.15.33:53314) 2024-05-03T18:47:19.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.382+0000 7f98d1c00700 1 -- 172.21.15.33:0/2225436242 learned_addr learned my addr 172.21.15.33:0/2225436242 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:47:19.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.382+0000 7f98d2600700 1 --2- 172.21.15.33:0/2225436242 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f98cc10b6c0 0x7f98cc19b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:19.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.382+0000 7f98d1200700 1 --2- 172.21.15.33:0/2225436242 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f98cc106080 0x7f98cc1a1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:19.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.382+0000 7f98d1c00700 1 -- 172.21.15.33:0/2225436242 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f98cc10b6c0 msgr2=0x7f98cc19b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:19.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.382+0000 7f98d1c00700 1 --2- 172.21.15.33:0/2225436242 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f98cc10b6c0 0x7f98cc19b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:19.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.382+0000 7f98d1c00700 1 -- 172.21.15.33:0/2225436242 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f98cc106080 msgr2=0x7f98cc1a1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:19.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.382+0000 7f98d1c00700 1 --2- 172.21.15.33:0/2225436242 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f98cc106080 0x7f98cc1a1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:19.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.382+0000 7f98d1c00700 1 -- 172.21.15.33:0/2225436242 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98cc19c110 con 0x7f98cc1056c0 2024-05-03T18:47:19.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.382+0000 7f98d1200700 1 --2- 172.21.15.33:0/2225436242 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f98cc106080 0x7f98cc1a1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:47:19.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.383+0000 7f98d1c00700 1 --2- 172.21.15.33:0/2225436242 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98cc1056c0 0x7f98cc1a17f0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f98c000c920 tx=0x7f98c000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:47:19.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.383+0000 7f98ca000700 1 -- 172.21.15.33:0/2225436242 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c0007b00 con 0x7f98cc1056c0 2024-05-03T18:47:19.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.383+0000 7f98d4600700 1 -- 172.21.15.33:0/2225436242 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98cc19c400 con 0x7f98cc1056c0 2024-05-03T18:47:19.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.383+0000 7f98ca000700 1 -- 172.21.15.33:0/2225436242 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f98c0004cc0 con 0x7f98cc1056c0 2024-05-03T18:47:19.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.383+0000 7f98ca000700 1 -- 172.21.15.33:0/2225436242 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c00054f0 con 0x7f98cc1056c0 2024-05-03T18:47:19.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.383+0000 7f98d4600700 1 -- 172.21.15.33:0/2225436242 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98cc1a86c0 con 0x7f98cc1056c0 2024-05-03T18:47:19.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.387+0000 7f98ca000700 1 -- 172.21.15.33:0/2225436242 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f98c0007ca0 con 0x7f98cc1056c0 2024-05-03T18:47:19.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.388+0000 7f98ca000700 1 --2- 172.21.15.33:0/2225436242 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f98b40b2a70 0x7f98b40b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:19.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.389+0000 7f98d1200700 1 --2- 172.21.15.33:0/2225436242 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f98b40b2a70 0x7f98b40b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:19.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.388+0000 7f98ca000700 1 -- 172.21.15.33:0/2225436242 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 9518+0+0 (secure 0 0 0) 0x7f98c00e2560 con 0x7f98cc1056c0 2024-05-03T18:47:19.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.389+0000 7f98d1200700 1 --2- 172.21.15.33:0/2225436242 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f98b40b2a70 0x7f98b40b4f30 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f98b8005fd0 tx=0x7f98b8005e20 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:47:19.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.403+0000 7f98defa4b80 1 Processor -- start 2024-05-03T18:47:19.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.403+0000 7f98defa4b80 1 -- start start 2024-05-03T18:47:19.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.403+0000 7f98defa4b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56259f854930 0x56259f854d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:19.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.403+0000 7f98defa4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56259f855250 0x56259f84ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:19.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.403+0000 7f98defa4b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56259f84f550 0x56259f84f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:19.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.403+0000 7f98defa4b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56259f7a1a20 con 0x56259f84f550 2024-05-03T18:47:19.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.403+0000 7f98defa4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56259f73f760 con 0x56259f855250 2024-05-03T18:47:19.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.403+0000 7f98defa4b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56259f79d9e0 con 0x56259f854930 2024-05-03T18:47:19.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.404+0000 7f98d2600700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56259f84f550 0x56259f84f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:19.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.404+0000 7f98d2600700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56259f84f550 0x56259f84f9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53320/0 (socket says 172.21.15.33:53320) 2024-05-03T18:47:19.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.404+0000 7f98d2600700 1 -- 172.21.15.33:0/2195420594 learned_addr learned my addr 172.21.15.33:0/2195420594 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:47:19.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.404+0000 7f98d1c00700 1 --2- 172.21.15.33:0/2195420594 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56259f854930 0x56259f854d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:19.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.404+0000 7f98d1200700 1 --2- 172.21.15.33:0/2195420594 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56259f855250 0x56259f84ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:19.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.404+0000 7f98d2600700 1 -- 172.21.15.33:0/2195420594 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56259f854930 msgr2=0x56259f854d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:47:19.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.404+0000 7f98d2600700 1 --2- 172.21.15.33:0/2195420594 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56259f854930 0x56259f854d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:19.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.404+0000 7f98d2600700 1 -- 172.21.15.33:0/2195420594 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56259f855250 msgr2=0x56259f84ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:47:19.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.404+0000 7f98d2600700 1 --2- 172.21.15.33:0/2195420594 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56259f855250 0x56259f84ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:19.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.404+0000 7f98d2600700 1 -- 172.21.15.33:0/2195420594 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56259f6ee8d0 con 0x56259f84f550 2024-05-03T18:47:19.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.404+0000 7f98d1200700 1 --2- 172.21.15.33:0/2195420594 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56259f855250 0x56259f84ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:47:19.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.405+0000 7f98d2600700 1 --2- 172.21.15.33:0/2195420594 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56259f84f550 0x56259f84f9d0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f98c4009310 tx=0x7f98c4009340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=424a184493d75be5 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:47:19.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.405+0000 7f98bea00700 1 -- 172.21.15.33:0/2195420594 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c4004a50 con 0x56259f84f550 2024-05-03T18:47:19.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.405+0000 7f98bea00700 1 -- 172.21.15.33:0/2195420594 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f98c4004bf0 con 0x56259f84f550 2024-05-03T18:47:19.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.405+0000 7f98bea00700 1 -- 172.21.15.33:0/2195420594 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c401c6d0 con 0x56259f84f550 2024-05-03T18:47:19.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.406+0000 7f98defa4b80 1 -- 172.21.15.33:0/2195420594 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56259f84f550 msgr2=0x56259f84f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:19.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.406+0000 7f98defa4b80 1 --2- 172.21.15.33:0/2195420594 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56259f84f550 0x56259f84f9d0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f98c4009310 tx=0x7f98c4009340 comp rx=0 tx=0).stop 2024-05-03T18:47:19.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.406+0000 7f98defa4b80 1 -- 172.21.15.33:0/2195420594 shutdown_connections 2024-05-03T18:47:19.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.406+0000 7f98defa4b80 1 --2- 172.21.15.33:0/2195420594 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56259f84f550 0x56259f84f9d0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:19.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.406+0000 7f98defa4b80 1 --2- 172.21.15.33:0/2195420594 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56259f854930 0x56259f854d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:19.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.406+0000 7f98defa4b80 1 --2- 172.21.15.33:0/2195420594 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56259f855250 0x56259f84ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:19.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.406+0000 7f98defa4b80 1 -- 172.21.15.33:0/2195420594 >> 172.21.15.33:0/2195420594 conn(0x56259f7c76d0 msgr2=0x56259f849900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:47:19.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.406+0000 7f98defa4b80 1 -- 172.21.15.33:0/2195420594 shutdown_connections 2024-05-03T18:47:19.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.406+0000 7f98defa4b80 1 -- 172.21.15.33:0/2195420594 wait complete. 2024-05-03T18:47:19.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.407+0000 7f98defa4b80 1 -- start start 2024-05-03T18:47:19.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.407+0000 7f98defa4b80 1 Processor -- start 2024-05-03T18:47:19.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.408+0000 7f98defa4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56259f84f550 0x56259f85ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:19.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.408+0000 7f98defa4b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56259f854930 0x56259f8581c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:19.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.408+0000 7f98defa4b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56259f855250 0x56259f858700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:19.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.408+0000 7f98defa4b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56259f85bca0 con 0x56259f854930 2024-05-03T18:47:19.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.408+0000 7f98defa4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56259f707b20 con 0x56259f84f550 2024-05-03T18:47:19.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.408+0000 7f98defa4b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56259f85be20 con 0x56259f855250 2024-05-03T18:47:19.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.408+0000 7f98defa4b80 10 client.0 _ll_register_callbacks cb 0x56259f748c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:47:19.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.408+0000 7f98d1200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56259f854930 0x56259f8581c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:19.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.408+0000 7f98d1200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56259f854930 0x56259f8581c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53322/0 (socket says 172.21.15.33:53322) 2024-05-03T18:47:19.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.408+0000 7f98d1200700 1 -- 172.21.15.33:0/2658882514 learned_addr learned my addr 172.21.15.33:0/2658882514 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:47:19.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.408+0000 7f98d1c00700 1 --2- 172.21.15.33:0/2658882514 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56259f84f550 0x56259f85ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:19.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.408+0000 7f98d2600700 1 --2- 172.21.15.33:0/2658882514 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56259f855250 0x56259f858700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:19.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.409+0000 7f98d1200700 1 -- 172.21.15.33:0/2658882514 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56259f855250 msgr2=0x56259f858700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:47:19.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.409+0000 7f98d1200700 1 --2- 172.21.15.33:0/2658882514 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56259f855250 0x56259f858700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:19.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.409+0000 7f98d1200700 1 -- 172.21.15.33:0/2658882514 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56259f84f550 msgr2=0x56259f85ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:47:19.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.409+0000 7f98d1200700 1 --2- 172.21.15.33:0/2658882514 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56259f84f550 0x56259f85ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:19.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.409+0000 7f98d1200700 1 -- 172.21.15.33:0/2658882514 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56259f767420 con 0x56259f854930 2024-05-03T18:47:19.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.409+0000 7f98d1200700 1 --2- 172.21.15.33:0/2658882514 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56259f854930 0x56259f8581c0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f98b800f6d0 tx=0x7f98b8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=690b3192979bffd7 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:47:19.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.409+0000 7f98b3400700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:47:19.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.409+0000 7f98b3400700 1 -- 172.21.15.33:0/2658882514 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b801c620 con 0x56259f854930 2024-05-03T18:47:19.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.409+0000 7f98b3400700 1 -- 172.21.15.33:0/2658882514 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(120..120 src has 1..120) ==== 9518+0+0 (secure 0 0 0) 0x7f98b801c7c0 con 0x56259f854930 2024-05-03T18:47:19.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.409+0000 7f98defa4b80 1 -- 172.21.15.33:0/2658882514 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98c4004bf0 con 0x56259f854930 2024-05-03T18:47:19.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.410+0000 7f98b3400700 1 -- 172.21.15.33:0/2658882514 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f98b802d050 con 0x56259f854930 2024-05-03T18:47:19.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.410+0000 7f98defa4b80 10 client.17112 fetch_fsmap learned FSMap version 21 2024-05-03T18:47:19.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.410+0000 7f98defa4b80 1 -- 172.21.15.33:0/2658882514 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x56259f866370 con 0x56259f854930 2024-05-03T18:47:19.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.410+0000 7f98b3400700 1 -- 172.21.15.33:0/2658882514 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f98c4004bf0 con 0x56259f854930 2024-05-03T18:47:19.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.410+0000 7f98b3400700 1 -- 172.21.15.33:0/2658882514 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f98b80249d0 con 0x56259f854930 2024-05-03T18:47:19.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.410+0000 7f98defa4b80 10 client.17112 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:47:19.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.410+0000 7f98defa4b80 10 client.17112 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:47:19.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.410+0000 7f98defa4b80 20 client.17112 populate_metadata read hostname 'smithi033' 2024-05-03T18:47:19.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.411+0000 7f98defa4b80 1 --2- 172.21.15.33:0/2658882514 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x56259f8689b0 0x56259f86ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:19.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.411+0000 7f98defa4b80 4 client.17112 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:47:19.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.411+0000 7f98defa4b80 1 -- 172.21.15.33:0/2658882514 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56259f81bff0 con 0x56259f8689b0 2024-05-03T18:47:19.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.411+0000 7f98d1c00700 1 --2- 172.21.15.33:0/2658882514 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x56259f8689b0 0x56259f86ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:19.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.412+0000 7f98d1c00700 1 --2- 172.21.15.33:0/2658882514 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x56259f8689b0 0x56259f86ada0 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a5b496414f328a8 server_cookie=c6815c7b13bb08a9 in_seq=0 out_seq=0 2024-05-03T18:47:19.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.412+0000 7f98b3400700 10 client.17112 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:47:19.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.414+0000 7f98b3400700 1 -- 172.21.15.33:0/2658882514 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56259f81bff0 con 0x56259f8689b0 2024-05-03T18:47:19.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.414+0000 7f98b3400700 10 client.17112 handle_command_reply: tid=0 2024-05-03T18:47:19.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.414+0000 7f98defa4b80 1 client.17112 shutdown 2024-05-03T18:47:19.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.414+0000 7f98defa4b80 1 -- 172.21.15.33:0/2658882514 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56259f854930 msgr2=0x56259f8581c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:19.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.414+0000 7f98defa4b80 1 --2- 172.21.15.33:0/2658882514 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56259f854930 0x56259f8581c0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f98b800f6d0 tx=0x7f98b8004750 comp rx=0 tx=0).stop 2024-05-03T18:47:19.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.414+0000 7f98defa4b80 1 -- 172.21.15.33:0/2658882514 shutdown_connections 2024-05-03T18:47:19.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.414+0000 7f98defa4b80 1 --2- 172.21.15.33:0/2658882514 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x56259f8689b0 0x56259f86ada0 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:19.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.414+0000 7f98defa4b80 1 --2- 172.21.15.33:0/2658882514 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56259f855250 0x56259f858700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:19.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.414+0000 7f98defa4b80 1 --2- 172.21.15.33:0/2658882514 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56259f84f550 0x56259f85ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:19.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.414+0000 7f98defa4b80 1 --2- 172.21.15.33:0/2658882514 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56259f854930 0x56259f8581c0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:19.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.414+0000 7f98defa4b80 1 -- 172.21.15.33:0/2658882514 >> 172.21.15.33:0/2658882514 conn(0x56259f7c76d0 msgr2=0x56259f8443b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:47:19.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.414+0000 7f98b3400700 0 client.17112 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:47:19.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.415+0000 7f98defa4b80 1 -- 172.21.15.33:0/2658882514 shutdown_connections 2024-05-03T18:47:19.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.415+0000 7f98defa4b80 1 -- 172.21.15.33:0/2658882514 wait complete. 2024-05-03T18:47:19.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.415+0000 7f98defa4b80 20 client.17112 trim_cache size 0 max 16384 2024-05-03T18:47:19.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.432+0000 7f98defa4b80 1 Processor -- start 2024-05-03T18:47:19.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.432+0000 7f98defa4b80 1 -- start start 2024-05-03T18:47:19.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.432+0000 7f98defa4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56259f84f550 0x56259f84ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:19.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.432+0000 7f98defa4b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56259f854930 0x56259f858180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:19.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.432+0000 7f98defa4b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56259f855250 0x56259f8586c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:19.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.432+0000 7f98defa4b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f98cc1af6a0 con 0x56259f854930 2024-05-03T18:47:19.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.432+0000 7f98defa4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f98cc1b1f00 con 0x56259f84f550 2024-05-03T18:47:19.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.432+0000 7f98defa4b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f98cc1afcf0 con 0x56259f855250 2024-05-03T18:47:19.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.433+0000 7f98d1200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56259f854930 0x56259f858180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:19.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.433+0000 7f98d1200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56259f854930 0x56259f858180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53330/0 (socket says 172.21.15.33:53330) 2024-05-03T18:47:19.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.433+0000 7f98d1200700 1 -- 172.21.15.33:0/340374659 learned_addr learned my addr 172.21.15.33:0/340374659 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:47:19.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.433+0000 7f98d2600700 1 --2- 172.21.15.33:0/340374659 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56259f855250 0x56259f8586c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:19.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.433+0000 7f98d1c00700 1 --2- 172.21.15.33:0/340374659 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56259f84f550 0x56259f84ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:19.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.433+0000 7f98d1200700 1 -- 172.21.15.33:0/340374659 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56259f855250 msgr2=0x56259f8586c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:47:19.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.433+0000 7f98d1200700 1 --2- 172.21.15.33:0/340374659 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56259f855250 0x56259f8586c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:19.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.433+0000 7f98d1200700 1 -- 172.21.15.33:0/340374659 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56259f84f550 msgr2=0x56259f84ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:47:19.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.433+0000 7f98d1200700 1 --2- 172.21.15.33:0/340374659 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56259f84f550 0x56259f84ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:19.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.433+0000 7f98d1200700 1 -- 172.21.15.33:0/340374659 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56259f858f30 con 0x56259f854930 2024-05-03T18:47:19.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.433+0000 7f98d1200700 1 --2- 172.21.15.33:0/340374659 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56259f854930 0x56259f858180 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f98b8012040 tx=0x7f98b800fb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d71ea5e2ddd2f35 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:47:19.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.434+0000 7f98be000700 1 -- 172.21.15.33:0/340374659 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b8024400 con 0x56259f854930 2024-05-03T18:47:19.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.434+0000 7f98be000700 1 -- 172.21.15.33:0/340374659 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f98b8035070 con 0x56259f854930 2024-05-03T18:47:19.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.434+0000 7f98be000700 1 -- 172.21.15.33:0/340374659 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b803aae0 con 0x56259f854930 2024-05-03T18:47:19.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.434+0000 7f98defa4b80 1 -- 172.21.15.33:0/340374659 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56259f854930 msgr2=0x56259f858180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:19.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.434+0000 7f98defa4b80 1 --2- 172.21.15.33:0/340374659 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56259f854930 0x56259f858180 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f98b8012040 tx=0x7f98b800fb40 comp rx=0 tx=0).stop 2024-05-03T18:47:19.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.435+0000 7f98defa4b80 1 -- 172.21.15.33:0/340374659 shutdown_connections 2024-05-03T18:47:19.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.435+0000 7f98defa4b80 1 --2- 172.21.15.33:0/340374659 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56259f855250 0x56259f8586c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:19.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.435+0000 7f98defa4b80 1 --2- 172.21.15.33:0/340374659 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56259f84f550 0x56259f84ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:19.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.435+0000 7f98defa4b80 1 --2- 172.21.15.33:0/340374659 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56259f854930 0x56259f858180 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:19.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.435+0000 7f98defa4b80 1 -- 172.21.15.33:0/340374659 >> 172.21.15.33:0/340374659 conn(0x56259f7c76d0 msgr2=0x56259f849870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:47:19.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.435+0000 7f98defa4b80 1 -- 172.21.15.33:0/340374659 shutdown_connections 2024-05-03T18:47:19.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.435+0000 7f98defa4b80 1 -- 172.21.15.33:0/340374659 wait complete. 2024-05-03T18:47:19.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.435+0000 7f98defa4b80 1 -- start start 2024-05-03T18:47:19.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.435+0000 7f98defa4b80 1 Processor -- start 2024-05-03T18:47:19.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.436+0000 7f98defa4b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56259f84f550 0x56259f8610b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:19.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.436+0000 7f98defa4b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56259f854930 0x56259f8615f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:19.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.436+0000 7f98defa4b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56259f855250 0x56259f86f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:19.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.436+0000 7f98defa4b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f98cc1b03c0 con 0x56259f84f550 2024-05-03T18:47:19.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.436+0000 7f98defa4b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f98cc1b1830 con 0x56259f855250 2024-05-03T18:47:19.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.436+0000 7f98defa4b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f98cc1b0a90 con 0x56259f854930 2024-05-03T18:47:19.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.436+0000 7f98defa4b80 10 client.0 _ll_register_callbacks cb 0x56259f859200 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:47:19.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.436+0000 7f98d1c00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56259f84f550 0x56259f8610b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:19.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.436+0000 7f98d1c00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56259f84f550 0x56259f8610b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53336/0 (socket says 172.21.15.33:53336) 2024-05-03T18:47:19.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.436+0000 7f98d1c00700 1 -- 172.21.15.33:0/1492511162 learned_addr learned my addr 172.21.15.33:0/1492511162 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:47:19.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.437+0000 7f98d2600700 1 --2- 172.21.15.33:0/1492511162 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56259f855250 0x56259f86f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:19.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.437+0000 7f98d1200700 1 --2- 172.21.15.33:0/1492511162 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56259f854930 0x56259f8615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:19.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.437+0000 7f98d1c00700 1 -- 172.21.15.33:0/1492511162 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56259f854930 msgr2=0x56259f8615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:47:19.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.437+0000 7f98d1c00700 1 --2- 172.21.15.33:0/1492511162 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56259f854930 0x56259f8615f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:19.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.437+0000 7f98d1c00700 1 -- 172.21.15.33:0/1492511162 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56259f855250 msgr2=0x56259f86f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:47:19.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.437+0000 7f98d1c00700 1 --2- 172.21.15.33:0/1492511162 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56259f855250 0x56259f86f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:19.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.437+0000 7f98d1c00700 1 -- 172.21.15.33:0/1492511162 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56259f86fc90 con 0x56259f84f550 2024-05-03T18:47:19.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.437+0000 7f98d1c00700 1 --2- 172.21.15.33:0/1492511162 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56259f84f550 0x56259f8610b0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f98c00a5480 tx=0x7f98c00a8c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd04268602ed27ee server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:47:19.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.437+0000 7f98b3400700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-03T18:47:19.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.437+0000 7f98b3400700 1 -- 172.21.15.33:0/1492511162 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c001d040 con 0x56259f84f550 2024-05-03T18:47:19.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.437+0000 7f98b3400700 1 -- 172.21.15.33:0/1492511162 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(120..120 src has 1..120) ==== 9518+0+0 (secure 0 0 0) 0x7f98c00a9660 con 0x56259f84f550 2024-05-03T18:47:19.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.437+0000 7f98defa4b80 1 -- 172.21.15.33:0/1492511162 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98b8035070 con 0x56259f84f550 2024-05-03T18:47:19.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.438+0000 7f98b3400700 1 -- 172.21.15.33:0/1492511162 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f98c00e2330 con 0x56259f84f550 2024-05-03T18:47:19.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.438+0000 7f98defa4b80 10 client.17118 fetch_fsmap learned FSMap version 21 2024-05-03T18:47:19.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.438+0000 7f98defa4b80 1 -- 172.21.15.33:0/1492511162 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x56259f8d2ba0 con 0x56259f84f550 2024-05-03T18:47:19.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.438+0000 7f98b3400700 1 -- 172.21.15.33:0/1492511162 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f98b8035070 con 0x56259f84f550 2024-05-03T18:47:19.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.438+0000 7f98b3400700 1 -- 172.21.15.33:0/1492511162 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f98c00b1dd0 con 0x56259f84f550 2024-05-03T18:47:19.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.438+0000 7f98defa4b80 10 client.17118 fetch_fsmap finished waiting for FSMap version 21 2024-05-03T18:47:19.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.438+0000 7f98defa4b80 10 client.17118 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-03T18:47:19.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.438+0000 7f98defa4b80 20 client.17118 populate_metadata read hostname 'smithi033' 2024-05-03T18:47:19.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.438+0000 7f98defa4b80 1 --2- 172.21.15.33:0/1492511162 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x56259f8d3200 0x56259f8d5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:19.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.438+0000 7f98defa4b80 4 client.17118 mds_command: new command op to 34508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-03T18:47:19.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.438+0000 7f98defa4b80 1 -- 172.21.15.33:0/1492511162 --> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56259f7af400 con 0x56259f8d3200 2024-05-03T18:47:19.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.439+0000 7f98d1200700 1 --2- 172.21.15.33:0/1492511162 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x56259f8d3200 0x56259f8d5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:19.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.440+0000 7f98d1200700 1 --2- 172.21.15.33:0/1492511162 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x56259f8d3200 0x56259f8d5620 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f42cbc9279714672 server_cookie=a34b46dcb90805ed in_seq=0 out_seq=0 2024-05-03T18:47:19.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.440+0000 7f98b3400700 10 client.17118 ms_handle_connect on v2:172.21.15.99:6838/919907308 2024-05-03T18:47:19.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.440+0000 7f98b3400700 1 -- 172.21.15.33:0/1492511162 <== mds.0 v2:172.21.15.99:6838/919907308 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56259f7af400 con 0x56259f8d3200 2024-05-03T18:47:19.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.440+0000 7f98b3400700 10 client.17118 handle_command_reply: tid=0 2024-05-03T18:47:19.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.440+0000 7f98defa4b80 1 client.17118 shutdown 2024-05-03T18:47:19.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.440+0000 7f98defa4b80 1 -- 172.21.15.33:0/1492511162 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56259f84f550 msgr2=0x56259f8610b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:19.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.440+0000 7f98defa4b80 1 --2- 172.21.15.33:0/1492511162 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56259f84f550 0x56259f8610b0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f98c00a5480 tx=0x7f98c00a8c40 comp rx=0 tx=0).stop 2024-05-03T18:47:19.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.440+0000 7f98defa4b80 1 -- 172.21.15.33:0/1492511162 shutdown_connections 2024-05-03T18:47:19.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.440+0000 7f98defa4b80 1 --2- 172.21.15.33:0/1492511162 >> [v2:172.21.15.99:6838/919907308,v1:172.21.15.99:6839/919907308] conn(0x56259f8d3200 0x56259f8d5620 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:19.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.441+0000 7f98defa4b80 1 --2- 172.21.15.33:0/1492511162 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56259f855250 0x56259f86f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:19.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.441+0000 7f98defa4b80 1 --2- 172.21.15.33:0/1492511162 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56259f84f550 0x56259f8610b0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:19.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.441+0000 7f98defa4b80 1 --2- 172.21.15.33:0/1492511162 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56259f854930 0x56259f8615f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:19.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.441+0000 7f98b3400700 0 client.17118 ms_handle_reset on v2:172.21.15.99:6838/919907308 2024-05-03T18:47:19.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.441+0000 7f98defa4b80 1 -- 172.21.15.33:0/1492511162 >> 172.21.15.33:0/1492511162 conn(0x56259f7c76d0 msgr2=0x56259f858120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:47:19.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.441+0000 7f98defa4b80 1 -- 172.21.15.33:0/1492511162 shutdown_connections 2024-05-03T18:47:19.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.441+0000 7f98defa4b80 1 -- 172.21.15.33:0/1492511162 wait complete. 2024-05-03T18:47:19.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.441+0000 7f98defa4b80 20 client.17118 trim_cache size 0 max 16384 2024-05-03T18:47:19.444 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-03T18:47:19.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.442+0000 7f98d4600700 1 -- 172.21.15.33:0/2225436242 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f98b40b2a70 msgr2=0x7f98b40b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:19.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.442+0000 7f98d4600700 1 --2- 172.21.15.33:0/2225436242 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f98b40b2a70 0x7f98b40b4f30 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f98b8005fd0 tx=0x7f98b8005e20 comp rx=0 tx=0).stop 2024-05-03T18:47:19.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.442+0000 7f98d4600700 1 -- 172.21.15.33:0/2225436242 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98cc1056c0 msgr2=0x7f98cc1a17f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:19.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.442+0000 7f98d4600700 1 --2- 172.21.15.33:0/2225436242 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98cc1056c0 0x7f98cc1a17f0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f98c000c920 tx=0x7f98c000cce0 comp rx=0 tx=0).stop 2024-05-03T18:47:19.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.442+0000 7f98d4600700 1 -- 172.21.15.33:0/2225436242 shutdown_connections 2024-05-03T18:47:19.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.442+0000 7f98d4600700 1 --2- 172.21.15.33:0/2225436242 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f98cc10b6c0 0x7f98cc19b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:19.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.442+0000 7f98d4600700 1 --2- 172.21.15.33:0/2225436242 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98cc1056c0 0x7f98cc1a17f0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:19.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.442+0000 7f98d4600700 1 --2- 172.21.15.33:0/2225436242 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f98b40b2a70 0x7f98b40b4f30 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:19.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.442+0000 7f98d4600700 1 --2- 172.21.15.33:0/2225436242 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f98cc106080 0x7f98cc1a1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:19.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.442+0000 7f98d4600700 1 -- 172.21.15.33:0/2225436242 >> 172.21.15.33:0/2225436242 conn(0x7f98cc1011b0 msgr2=0x7f98cc1017c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:47:19.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.442+0000 7f98d4600700 1 -- 172.21.15.33:0/2225436242 shutdown_connections 2024-05-03T18:47:19.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:19.442+0000 7f98d4600700 1 -- 172.21.15.33:0/2225436242 wait complete. 2024-05-03T18:47:20.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:47:19 smithi099 ceph-mon[31658]: pgmap v636: 129 pgs: 129 active+clean; 3.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 255 B/s wr, 2 op/s 2024-05-03T18:47:20.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:47:19 smithi150 ceph-mon[32492]: pgmap v636: 129 pgs: 129 active+clean; 3.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 255 B/s wr, 2 op/s 2024-05-03T18:47:20.434 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:47:19 smithi033 ceph-mon[28330]: pgmap v636: 129 pgs: 129 active+clean; 3.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 255 B/s wr, 2 op/s 2024-05-03T18:47:20.489 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_9a047fe7599a549042d06a3fb5b6813a203eabea/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_9a047fe7599a549042d06a3fb5b6813a203eabea/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/git.ceph.com_ceph-c_9a047fe7599a549042d06a3fb5b6813a203eabea/qa/tasks/thrasher.py", line 47, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-05-03T18:47:20.489 INFO:tasks.fwd_scrub:done joining 2024-05-03T18:47:20.490 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-05-03T18:47:20.500 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-03T18:47:20.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:20.747+0000 7fcc63e00640 1 -- 172.21.15.33:0/1020833696 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc5c109230 msgr2=0x7fcc5c109610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:20.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:20.747+0000 7fcc63e00640 1 --2- 172.21.15.33:0/1020833696 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc5c109230 0x7fcc5c109610 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fcc50009900 tx=0x7fcc5002a410 comp rx=0 tx=0).stop 2024-05-03T18:47:20.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:20.748+0000 7fcc63e00640 1 -- 172.21.15.33:0/1020833696 shutdown_connections 2024-05-03T18:47:20.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:20.748+0000 7fcc63e00640 1 --2- 172.21.15.33:0/1020833696 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc5c10e300 0x7fcc5c115ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:20.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:20.748+0000 7fcc63e00640 1 --2- 172.21.15.33:0/1020833696 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc5c109be0 0x7fcc5c10dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:20.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:20.748+0000 7fcc63e00640 1 --2- 172.21.15.33:0/1020833696 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc5c109230 0x7fcc5c109610 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:20.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:20.748+0000 7fcc63e00640 1 -- 172.21.15.33:0/1020833696 >> 172.21.15.33:0/1020833696 conn(0x7fcc5c0761b0 msgr2=0x7fcc5c1077d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:47:20.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:20.748+0000 7fcc63e00640 1 -- 172.21.15.33:0/1020833696 shutdown_connections 2024-05-03T18:47:20.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:20.748+0000 7fcc63e00640 1 -- 172.21.15.33:0/1020833696 wait complete. 2024-05-03T18:47:20.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:20.749+0000 7fcc63e00640 1 Processor -- start 2024-05-03T18:47:20.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:20.749+0000 7fcc63e00640 1 -- start start 2024-05-03T18:47:20.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:20.749+0000 7fcc63e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc5c109230 0x7fcc5c1a5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:20.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:20.749+0000 7fcc63e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc5c10e300 0x7fcc5c1a5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:20.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:20.749+0000 7fcc63e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc5c19f5f0 0x7fcc5c19faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:20.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:20.749+0000 7fcc63e00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcc5c1191c0 con 0x7fcc5c19f5f0 2024-05-03T18:47:20.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:20.749+0000 7fcc63e00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fcc5c119040 con 0x7fcc5c109230 2024-05-03T18:47:20.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:20.749+0000 7fcc63e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fcc5c119340 con 0x7fcc5c10e300 2024-05-03T18:47:20.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:20.750+0000 7fcc61e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc5c19f5f0 0x7fcc5c19faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:20.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:20.750+0000 7fcc61e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc5c19f5f0 0x7fcc5c19faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53346/0 (socket says 172.21.15.33:53346) 2024-05-03T18:47:20.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:20.750+0000 7fcc61e00640 1 -- 172.21.15.33:0/2089885351 learned_addr learned my addr 172.21.15.33:0/2089885351 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:47:20.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:20.750+0000 7fcc61400640 1 --2- 172.21.15.33:0/2089885351 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc5c109230 0x7fcc5c1a5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:20.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:20.750+0000 7fcc60a00640 1 --2- 172.21.15.33:0/2089885351 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc5c10e300 0x7fcc5c1a5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:20.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:20.750+0000 7fcc61e00640 1 -- 172.21.15.33:0/2089885351 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc5c10e300 msgr2=0x7fcc5c1a5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:20.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:20.750+0000 7fcc61e00640 1 --2- 172.21.15.33:0/2089885351 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc5c10e300 0x7fcc5c1a5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:20.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:20.750+0000 7fcc61e00640 1 -- 172.21.15.33:0/2089885351 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc5c109230 msgr2=0x7fcc5c1a5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:20.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:20.750+0000 7fcc61e00640 1 --2- 172.21.15.33:0/2089885351 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc5c109230 0x7fcc5c1a5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:20.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:20.750+0000 7fcc61e00640 1 -- 172.21.15.33:0/2089885351 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc5c1a0110 con 0x7fcc5c19f5f0 2024-05-03T18:47:20.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:20.751+0000 7fcc61e00640 1 --2- 172.21.15.33:0/2089885351 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc5c19f5f0 0x7fcc5c19faa0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fcc5400b970 tx=0x7fcc5400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:47:20.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:20.751+0000 7fcc5a000640 1 -- 172.21.15.33:0/2089885351 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc54019070 con 0x7fcc5c19f5f0 2024-05-03T18:47:20.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:20.751+0000 7fcc5a000640 1 -- 172.21.15.33:0/2089885351 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcc54014440 con 0x7fcc5c19f5f0 2024-05-03T18:47:20.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:20.751+0000 7fcc63e00640 1 -- 172.21.15.33:0/2089885351 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc5c1a0400 con 0x7fcc5c19f5f0 2024-05-03T18:47:20.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:20.751+0000 7fcc5a000640 1 -- 172.21.15.33:0/2089885351 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc540075d0 con 0x7fcc5c19f5f0 2024-05-03T18:47:20.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:20.751+0000 7fcc63e00640 1 -- 172.21.15.33:0/2089885351 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc5c1ac6a0 con 0x7fcc5c19f5f0 2024-05-03T18:47:20.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:20.752+0000 7fcc63e00640 1 -- 172.21.15.33:0/2089885351 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc5c069c20 con 0x7fcc5c19f5f0 2024-05-03T18:47:20.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:20.755+0000 7fcc5a000640 1 -- 172.21.15.33:0/2089885351 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fcc5401c070 con 0x7fcc5c19f5f0 2024-05-03T18:47:20.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:20.756+0000 7fcc5a000640 1 --2- 172.21.15.33:0/2089885351 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fcc440b29e0 0x7fcc440b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:20.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:20.756+0000 7fcc5a000640 1 -- 172.21.15.33:0/2089885351 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 9518+0+0 (secure 0 0 0) 0x7fcc540e1f10 con 0x7fcc5c19f5f0 2024-05-03T18:47:20.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:20.757+0000 7fcc61400640 1 --2- 172.21.15.33:0/2089885351 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fcc440b29e0 0x7fcc440b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:20.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:20.758+0000 7fcc61400640 1 --2- 172.21.15.33:0/2089885351 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fcc440b29e0 0x7fcc440b4ea0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fcc50006010 tx=0x7fcc50005a20 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:47:20.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:20.761+0000 7fcc5a000640 1 -- 172.21.15.33:0/2089885351 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fcc540a95c0 con 0x7fcc5c19f5f0 2024-05-03T18:47:20.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:20.982+0000 7fcc63e00640 1 -- 172.21.15.33:0/2089885351 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcc5c109be0 con 0x7fcc5c19f5f0 2024-05-03T18:47:20.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:20.985+0000 7fcc5a000640 1 -- 172.21.15.33:0/2089885351 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v120) ==== 74+0+25275 (secure 0 0 0) 0x7fcc540a8ec0 con 0x7fcc5c19f5f0 2024-05-03T18:47:20.986 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:47:20.987 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":120,"fsid":"5703bba4-0978-11ef-bc97-c7b262605968","created":"2024-05-03T18:11:00.899879+0000","modified":"2024-05-03T18:47:17.657855+0000","last_up_change":"2024-05-03T18:21:00.390173+0000","last_in_change":"2024-05-03T18:20:34.103727+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-03T18:15:37.297738+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-03T18:22:19.875368+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-03T18:22:24.432692+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":35,"snap_epoch":119,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-03T18:22:38.722443+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":35,"snap_epoch":119,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"fdcc4e66-4ba0-4378-a9a4-7d907cd00b4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":637686669},{"type":"v1","addr":"172.21.15.33:6803","nonce":637686669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":637686669},{"type":"v1","addr":"172.21.15.33:6805","nonce":637686669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":637686669},{"type":"v1","addr":"172.21.15.33:6809","nonce":637686669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":637686669},{"type":"v1","addr":"172.21.15.33:6807","nonce":637686669}]},"public_addr":"172.21.15.33:6803/637686669","cluster_addr":"172.21.15.33:6805/637686669","heartbeat_back_addr":"172.21.15.33:6809/637686669","heartbeat_front_addr":"172.21.15.33:6807/637686669","state":["exists","up"]},{"osd":1,"uuid":"cf78cad3-fc24-43f4-8784-21cabea7277c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":1536122841},{"type":"v1","addr":"172.21.15.99:6801","nonce":1536122841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":1536122841},{"type":"v1","addr":"172.21.15.99:6803","nonce":1536122841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":1536122841},{"type":"v1","addr":"172.21.15.99:6807","nonce":1536122841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":1536122841},{"type":"v1","addr":"172.21.15.99:6805","nonce":1536122841}]},"public_addr":"172.21.15.99:6801/1536122841","cluster_addr":"172.21.15.99:6803/1536122841","heartbeat_back_addr":"172.21.15.99:6807/1536122841","heartbeat_front_addr":"172.21.15.99:6805/1536122841","state":["exists","up"]},{"osd":2,"uuid":"59f066d7-626b-410a-8d67-daca6fbbd3ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":1995659622},{"type":"v1","addr":"172.21.15.150:6801","nonce":1995659622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":1995659622},{"type":"v1","addr":"172.21.15.150:6803","nonce":1995659622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":1995659622},{"type":"v1","addr":"172.21.15.150:6807","nonce":1995659622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":1995659622},{"type":"v1","addr":"172.21.15.150:6805","nonce":1995659622}]},"public_addr":"172.21.15.150:6801/1995659622","cluster_addr":"172.21.15.150:6803/1995659622","heartbeat_back_addr":"172.21.15.150:6807/1995659622","heartbeat_front_addr":"172.21.15.150:6805/1995659622","state":["exists","up"]},{"osd":3,"uuid":"d88ed022-4dcb-45fc-9fc6-41ef7c4bf202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":3814942724},{"type":"v1","addr":"172.21.15.33:6811","nonce":3814942724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":3814942724},{"type":"v1","addr":"172.21.15.33:6813","nonce":3814942724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":3814942724},{"type":"v1","addr":"172.21.15.33:6817","nonce":3814942724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":3814942724},{"type":"v1","addr":"172.21.15.33:6815","nonce":3814942724}]},"public_addr":"172.21.15.33:6811/3814942724","cluster_addr":"172.21.15.33:6813/3814942724","heartbeat_back_addr":"172.21.15.33:6817/3814942724","heartbeat_front_addr":"172.21.15.33:6815/3814942724","state":["exists","up"]},{"osd":4,"uuid":"1235e1a6-ff6e-4668-ac1a-073c2a3b7c98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":29081029},{"type":"v1","addr":"172.21.15.99:6809","nonce":29081029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":29081029},{"type":"v1","addr":"172.21.15.99:6811","nonce":29081029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":29081029},{"type":"v1","addr":"172.21.15.99:6815","nonce":29081029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":29081029},{"type":"v1","addr":"172.21.15.99:6813","nonce":29081029}]},"public_addr":"172.21.15.99:6809/29081029","cluster_addr":"172.21.15.99:6811/29081029","heartbeat_back_addr":"172.21.15.99:6815/29081029","heartbeat_front_addr":"172.21.15.99:6813/29081029","state":["exists","up"]},{"osd":5,"uuid":"4b84bff8-d6d7-4258-a0ee-53bc19802585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":3172189533},{"type":"v1","addr":"172.21.15.150:6809","nonce":3172189533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3172189533},{"type":"v1","addr":"172.21.15.150:6811","nonce":3172189533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3172189533},{"type":"v1","addr":"172.21.15.150:6815","nonce":3172189533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3172189533},{"type":"v1","addr":"172.21.15.150:6813","nonce":3172189533}]},"public_addr":"172.21.15.150:6809/3172189533","cluster_addr":"172.21.15.150:6811/3172189533","heartbeat_back_addr":"172.21.15.150:6815/3172189533","heartbeat_front_addr":"172.21.15.150:6813/3172189533","state":["exists","up"]},{"osd":6,"uuid":"51f5fbdd-3794-4c48-9f6c-478f0fa35ac5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":4234541418},{"type":"v1","addr":"172.21.15.33:6819","nonce":4234541418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":4234541418},{"type":"v1","addr":"172.21.15.33:6821","nonce":4234541418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":4234541418},{"type":"v1","addr":"172.21.15.33:6825","nonce":4234541418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":4234541418},{"type":"v1","addr":"172.21.15.33:6823","nonce":4234541418}]},"public_addr":"172.21.15.33:6819/4234541418","cluster_addr":"172.21.15.33:6821/4234541418","heartbeat_back_addr":"172.21.15.33:6825/4234541418","heartbeat_front_addr":"172.21.15.33:6823/4234541418","state":["exists","up"]},{"osd":7,"uuid":"982cc5b3-3163-4505-9bde-a27a8c289457","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":908673090},{"type":"v1","addr":"172.21.15.99:6817","nonce":908673090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":908673090},{"type":"v1","addr":"172.21.15.99:6819","nonce":908673090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":908673090},{"type":"v1","addr":"172.21.15.99:6823","nonce":908673090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":908673090},{"type":"v1","addr":"172.21.15.99:6821","nonce":908673090}]},"public_addr":"172.21.15.99:6817/908673090","cluster_addr":"172.21.15.99:6819/908673090","heartbeat_back_addr":"172.21.15.99:6823/908673090","heartbeat_front_addr":"172.21.15.99:6821/908673090","state":["exists","up"]},{"osd":8,"uuid":"84c9d037-eb84-4eb4-8aa2-e784b703d36c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":1308083602},{"type":"v1","addr":"172.21.15.150:6817","nonce":1308083602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":1308083602},{"type":"v1","addr":"172.21.15.150:6819","nonce":1308083602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":1308083602},{"type":"v1","addr":"172.21.15.150:6823","nonce":1308083602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":1308083602},{"type":"v1","addr":"172.21.15.150:6821","nonce":1308083602}]},"public_addr":"172.21.15.150:6817/1308083602","cluster_addr":"172.21.15.150:6819/1308083602","heartbeat_back_addr":"172.21.15.150:6823/1308083602","heartbeat_front_addr":"172.21.15.150:6821/1308083602","state":["exists","up"]},{"osd":9,"uuid":"a02e965f-0d8f-45e2-9490-337f7f6e3881","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":4133096905},{"type":"v1","addr":"172.21.15.33:6827","nonce":4133096905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":4133096905},{"type":"v1","addr":"172.21.15.33:6829","nonce":4133096905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4133096905},{"type":"v1","addr":"172.21.15.33:6833","nonce":4133096905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":4133096905},{"type":"v1","addr":"172.21.15.33:6831","nonce":4133096905}]},"public_addr":"172.21.15.33:6827/4133096905","cluster_addr":"172.21.15.33:6829/4133096905","heartbeat_back_addr":"172.21.15.33:6833/4133096905","heartbeat_front_addr":"172.21.15.33:6831/4133096905","state":["exists","up"]},{"osd":10,"uuid":"8b29bb96-76e8-45a0-ab19-8d082d755fe9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":935746136},{"type":"v1","addr":"172.21.15.99:6825","nonce":935746136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":935746136},{"type":"v1","addr":"172.21.15.99:6827","nonce":935746136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":935746136},{"type":"v1","addr":"172.21.15.99:6831","nonce":935746136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":935746136},{"type":"v1","addr":"172.21.15.99:6829","nonce":935746136}]},"public_addr":"172.21.15.99:6825/935746136","cluster_addr":"172.21.15.99:6827/935746136","heartbeat_back_addr":"172.21.15.99:6831/935746136","heartbeat_front_addr":"172.21.15.99:6829/935746136","state":["exists","up"]},{"osd":11,"uuid":"07a6a825-c902-4bfd-97cc-cc9074532af5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2069349808},{"type":"v1","addr":"172.21.15.150:6825","nonce":2069349808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2069349808},{"type":"v1","addr":"172.21.15.150:6827","nonce":2069349808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2069349808},{"type":"v1","addr":"172.21.15.150:6831","nonce":2069349808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2069349808},{"type":"v1","addr":"172.21.15.150:6829","nonce":2069349808}]},"public_addr":"172.21.15.150:6825/2069349808","cluster_addr":"172.21.15.150:6827/2069349808","heartbeat_back_addr":"172.21.15.150:6831/2069349808","heartbeat_front_addr":"172.21.15.150:6829/2069349808","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:14:27.454536+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:15:00.538570+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:15:34.125289+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:16:09.853870+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:16:44.928107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:17:19.644037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:17:56.108443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:18:31.731499+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:19:08.062358+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:19:45.204430+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:20:21.013710+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:20:57.653698+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11},{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/4003417322":"2024-05-04T18:27:11.722511+0000","172.21.15.33:6801/4007322071":"2024-05-04T18:11:44.239835+0000","172.21.15.33:6801/2830135294":"2024-05-04T18:12:19.127652+0000","172.21.15.33:0/3767665033":"2024-05-04T18:12:19.127652+0000","172.21.15.33:6800/4007322071":"2024-05-04T18:11:44.239835+0000","172.21.15.33:0/1614640996":"2024-05-04T18:11:44.239835+0000","172.21.15.33:0/1314704208":"2024-05-04T18:11:44.239835+0000","172.21.15.33:0/2014902144":"2024-05-04T18:11:44.239835+0000","172.21.15.33:0/965859483":"2024-05-04T18:12:19.127652+0000","172.21.15.33:6801/3642895501":"2024-05-04T18:27:11.722511+0000","172.21.15.33:6800/2830135294":"2024-05-04T18:12:19.127652+0000","172.21.15.33:0/791712778":"2024-05-04T18:27:11.722511+0000","172.21.15.33:0/3991947997":"2024-05-04T18:12:19.127652+0000","172.21.15.33:0/806149166":"2024-05-04T18:27:11.722511+0000","172.21.15.33:0/1300142939":"2024-05-04T18:27:11.722511+0000","172.21.15.33:6800/3642895501":"2024-05-04T18:27:11.722511+0000","172.21.15.33:0/757614565":"2024-05-04T18:27:11.722511+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":22,"length":1},{"begin":35,"length":1}]},{"pool":4,"snaps":[{"begin":22,"length":1},{"begin":35,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-03T18:47:20.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:20.988+0000 7fcc63e00640 1 -- 172.21.15.33:0/2089885351 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fcc440b29e0 msgr2=0x7fcc440b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:20.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:20.988+0000 7fcc63e00640 1 --2- 172.21.15.33:0/2089885351 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fcc440b29e0 0x7fcc440b4ea0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fcc50006010 tx=0x7fcc50005a20 comp rx=0 tx=0).stop 2024-05-03T18:47:20.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:20.988+0000 7fcc63e00640 1 -- 172.21.15.33:0/2089885351 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc5c19f5f0 msgr2=0x7fcc5c19faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:20.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:20.988+0000 7fcc63e00640 1 --2- 172.21.15.33:0/2089885351 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc5c19f5f0 0x7fcc5c19faa0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fcc5400b970 tx=0x7fcc5400be30 comp rx=0 tx=0).stop 2024-05-03T18:47:20.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:20.988+0000 7fcc63e00640 1 -- 172.21.15.33:0/2089885351 shutdown_connections 2024-05-03T18:47:20.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:20.988+0000 7fcc63e00640 1 --2- 172.21.15.33:0/2089885351 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fcc440b29e0 0x7fcc440b4ea0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:20.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:20.988+0000 7fcc63e00640 1 --2- 172.21.15.33:0/2089885351 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc5c19f5f0 0x7fcc5c19faa0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:20.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:20.988+0000 7fcc63e00640 1 --2- 172.21.15.33:0/2089885351 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc5c10e300 0x7fcc5c1a5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:20.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:20.988+0000 7fcc63e00640 1 --2- 172.21.15.33:0/2089885351 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc5c109230 0x7fcc5c1a5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:20.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:20.988+0000 7fcc63e00640 1 -- 172.21.15.33:0/2089885351 >> 172.21.15.33:0/2089885351 conn(0x7fcc5c0761b0 msgr2=0x7fcc5c108660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:47:20.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:20.988+0000 7fcc63e00640 1 -- 172.21.15.33:0/2089885351 shutdown_connections 2024-05-03T18:47:20.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:20.988+0000 7fcc63e00640 1 -- 172.21.15.33:0/2089885351 wait complete. 2024-05-03T18:47:21.007 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-03T18:15:37.297738+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-03T18:22:19.875368+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-03T18:22:24.432692+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '119', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 35, 'snap_epoch': 119, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-03T18:22:38.722443+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '119', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 35, 'snap_epoch': 119, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-03T18:47:21.008 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-03T18:47:21.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.223+0000 7f9643200640 1 -- 172.21.15.33:0/2127419225 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9644102190 msgr2=0x7f9644106650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:21.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.223+0000 7f9643200640 1 --2- 172.21.15.33:0/2127419225 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9644102190 0x7f9644106650 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f9630009900 tx=0x7f963002a470 comp rx=0 tx=0).stop 2024-05-03T18:47:21.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.223+0000 7f9643200640 1 -- 172.21.15.33:0/2127419225 shutdown_connections 2024-05-03T18:47:21.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.223+0000 7f9643200640 1 --2- 172.21.15.33:0/2127419225 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9644106c70 0x7f9644107100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:21.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.223+0000 7f9643200640 1 --2- 172.21.15.33:0/2127419225 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9644102190 0x7f9644106650 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:21.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.223+0000 7f9643200640 1 --2- 172.21.15.33:0/2127419225 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9644101870 0x7f9644101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:21.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.223+0000 7f9643200640 1 -- 172.21.15.33:0/2127419225 >> 172.21.15.33:0/2127419225 conn(0x7f96440fd2c0 msgr2=0x7f96440ff6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:47:21.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.223+0000 7f9643200640 1 -- 172.21.15.33:0/2127419225 shutdown_connections 2024-05-03T18:47:21.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.223+0000 7f9643200640 1 -- 172.21.15.33:0/2127419225 wait complete. 2024-05-03T18:47:21.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.224+0000 7f9643200640 1 Processor -- start 2024-05-03T18:47:21.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.224+0000 7f9643200640 1 -- start start 2024-05-03T18:47:21.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.225+0000 7f9643200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9644101870 0x7f964419f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:21.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.225+0000 7f9643200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9644102190 0x7f964419fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:21.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.225+0000 7f9643200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9644106c70 0x7f96441a3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:21.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.225+0000 7f9643200640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9644119020 con 0x7f9644106c70 2024-05-03T18:47:21.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.225+0000 7f9643200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9644118ea0 con 0x7f9644101870 2024-05-03T18:47:21.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.225+0000 7f9643200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f96441191a0 con 0x7f9644102190 2024-05-03T18:47:21.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.225+0000 7f9642800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9644106c70 0x7f96441a3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:21.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.225+0000 7f9642800640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9644106c70 0x7f96441a3dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53364/0 (socket says 172.21.15.33:53364) 2024-05-03T18:47:21.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.225+0000 7f9642800640 1 -- 172.21.15.33:0/2414168100 learned_addr learned my addr 172.21.15.33:0/2414168100 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:47:21.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.225+0000 7f9641e00640 1 --2- 172.21.15.33:0/2414168100 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9644101870 0x7f964419f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:21.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.225+0000 7f9641400640 1 --2- 172.21.15.33:0/2414168100 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9644102190 0x7f964419fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:21.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.225+0000 7f9642800640 1 -- 172.21.15.33:0/2414168100 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9644102190 msgr2=0x7f964419fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:21.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.225+0000 7f9642800640 1 --2- 172.21.15.33:0/2414168100 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9644102190 0x7f964419fa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:21.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.225+0000 7f9642800640 1 -- 172.21.15.33:0/2414168100 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9644101870 msgr2=0x7f964419f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:21.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.225+0000 7f9642800640 1 --2- 172.21.15.33:0/2414168100 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9644101870 0x7f964419f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:21.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.225+0000 7f9642800640 1 -- 172.21.15.33:0/2414168100 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96441a44f0 con 0x7f9644106c70 2024-05-03T18:47:21.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.226+0000 7f9642800640 1 --2- 172.21.15.33:0/2414168100 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9644106c70 0x7f96441a3dc0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f963c00cce0 tx=0x7f963c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:47:21.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.226+0000 7f963aa00640 1 -- 172.21.15.33:0/2414168100 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f963c00f450 con 0x7f9644106c70 2024-05-03T18:47:21.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.226+0000 7f963aa00640 1 -- 172.21.15.33:0/2414168100 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f963c00fa80 con 0x7f9644106c70 2024-05-03T18:47:21.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.226+0000 7f9643200640 1 -- 172.21.15.33:0/2414168100 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96441a47e0 con 0x7f9644106c70 2024-05-03T18:47:21.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.226+0000 7f963aa00640 1 -- 172.21.15.33:0/2414168100 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f963c018ad0 con 0x7f9644106c70 2024-05-03T18:47:21.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.226+0000 7f9643200640 1 -- 172.21.15.33:0/2414168100 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96441ac070 con 0x7f9644106c70 2024-05-03T18:47:21.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.227+0000 7f9643200640 1 -- 172.21.15.33:0/2414168100 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f964406b890 con 0x7f9644106c70 2024-05-03T18:47:21.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.230+0000 7f963aa00640 1 -- 172.21.15.33:0/2414168100 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f963c00f5f0 con 0x7f9644106c70 2024-05-03T18:47:21.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.231+0000 7f963aa00640 1 --2- 172.21.15.33:0/2414168100 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f96280b29e0 0x7f96280b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:21.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.232+0000 7f963aa00640 1 -- 172.21.15.33:0/2414168100 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 9518+0+0 (secure 0 0 0) 0x7f963c0e31c0 con 0x7f9644106c70 2024-05-03T18:47:21.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.232+0000 7f9641e00640 1 --2- 172.21.15.33:0/2414168100 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f96280b29e0 0x7f96280b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:21.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.233+0000 7f9641e00640 1 --2- 172.21.15.33:0/2414168100 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f96280b29e0 0x7f96280b4ea0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9634005e10 tx=0x7f9634005da0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:47:21.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.236+0000 7f963aa00640 1 -- 172.21.15.33:0/2414168100 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f963c014030 con 0x7f9644106c70 2024-05-03T18:47:21.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:47:21 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/2089885351' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T18:47:21.305 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:47:20 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/2089885351' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T18:47:21.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:47:21 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/2089885351' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T18:47:21.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.456+0000 7f9643200640 1 -- 172.21.15.33:0/2414168100 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f96441ac2a0 con 0x7f9644106c70 2024-05-03T18:47:21.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.457+0000 7f963aa00640 1 -- 172.21.15.33:0/2414168100 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v120) ==== 93+0+10 (secure 0 0 0) 0x7f963c0aa760 con 0x7f9644106c70 2024-05-03T18:47:21.459 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-05-03T18:47:21.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.460+0000 7f9643200640 1 -- 172.21.15.33:0/2414168100 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f96280b29e0 msgr2=0x7f96280b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:21.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.460+0000 7f9643200640 1 --2- 172.21.15.33:0/2414168100 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f96280b29e0 0x7f96280b4ea0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9634005e10 tx=0x7f9634005da0 comp rx=0 tx=0).stop 2024-05-03T18:47:21.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.460+0000 7f9643200640 1 -- 172.21.15.33:0/2414168100 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9644106c70 msgr2=0x7f96441a3dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:21.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.460+0000 7f9643200640 1 --2- 172.21.15.33:0/2414168100 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9644106c70 0x7f96441a3dc0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f963c00cce0 tx=0x7f963c00a620 comp rx=0 tx=0).stop 2024-05-03T18:47:21.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.461+0000 7f9643200640 1 -- 172.21.15.33:0/2414168100 shutdown_connections 2024-05-03T18:47:21.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.461+0000 7f9643200640 1 --2- 172.21.15.33:0/2414168100 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f96280b29e0 0x7f96280b4ea0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:21.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.461+0000 7f9643200640 1 --2- 172.21.15.33:0/2414168100 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9644106c70 0x7f96441a3dc0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:21.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.461+0000 7f9643200640 1 --2- 172.21.15.33:0/2414168100 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9644102190 0x7f964419fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:21.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.461+0000 7f9643200640 1 --2- 172.21.15.33:0/2414168100 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9644101870 0x7f964419f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:21.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.461+0000 7f9643200640 1 -- 172.21.15.33:0/2414168100 >> 172.21.15.33:0/2414168100 conn(0x7f96440fd2c0 msgr2=0x7f96440fe250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:47:21.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.461+0000 7f9643200640 1 -- 172.21.15.33:0/2414168100 shutdown_connections 2024-05-03T18:47:21.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.461+0000 7f9643200640 1 -- 172.21.15.33:0/2414168100 wait complete. 2024-05-03T18:47:21.482 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-03T18:47:21.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.688+0000 7ff7e3200640 1 -- 172.21.15.33:0/449820742 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7e4102860 msgr2=0x7ff7e4102ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:21.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.688+0000 7ff7e3200640 1 --2- 172.21.15.33:0/449820742 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7e4102860 0x7ff7e4102ce0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc009930 tx=0x7ff7cc02a420 comp rx=0 tx=0).stop 2024-05-03T18:47:21.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.688+0000 7ff7e3200640 1 -- 172.21.15.33:0/449820742 shutdown_connections 2024-05-03T18:47:21.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.688+0000 7ff7e3200640 1 --2- 172.21.15.33:0/449820742 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7e4103220 0x7ff7e410f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:21.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.688+0000 7ff7e3200640 1 --2- 172.21.15.33:0/449820742 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7e4102860 0x7ff7e4102ce0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:21.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.688+0000 7ff7e3200640 1 --2- 172.21.15.33:0/449820742 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7e4108870 0x7ff7e4108c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:21.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.688+0000 7ff7e3200640 1 -- 172.21.15.33:0/449820742 >> 172.21.15.33:0/449820742 conn(0x7ff7e40fe350 msgr2=0x7ff7e4100770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:47:21.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.688+0000 7ff7e3200640 1 -- 172.21.15.33:0/449820742 shutdown_connections 2024-05-03T18:47:21.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.688+0000 7ff7e3200640 1 -- 172.21.15.33:0/449820742 wait complete. 2024-05-03T18:47:21.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.689+0000 7ff7e3200640 1 Processor -- start 2024-05-03T18:47:21.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.689+0000 7ff7e3200640 1 -- start start 2024-05-03T18:47:21.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.690+0000 7ff7e3200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7e4102860 0x7ff7e4198f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:21.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.690+0000 7ff7e3200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7e4103220 0x7ff7e41994d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:21.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.690+0000 7ff7e3200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7e4108870 0x7ff7e419d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:21.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.690+0000 7ff7e3200640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff7e41129f0 con 0x7ff7e4102860 2024-05-03T18:47:21.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.690+0000 7ff7e3200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff7e4112870 con 0x7ff7e4108870 2024-05-03T18:47:21.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.690+0000 7ff7e3200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff7e4112b70 con 0x7ff7e4103220 2024-05-03T18:47:21.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.690+0000 7ff7e1e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7e4102860 0x7ff7e4198f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:21.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.690+0000 7ff7e1e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7e4102860 0x7ff7e4198f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53384/0 (socket says 172.21.15.33:53384) 2024-05-03T18:47:21.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.690+0000 7ff7e1e00640 1 -- 172.21.15.33:0/4103609982 learned_addr learned my addr 172.21.15.33:0/4103609982 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:47:21.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.690+0000 7ff7e2800640 1 --2- 172.21.15.33:0/4103609982 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7e4108870 0x7ff7e419d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:21.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.690+0000 7ff7e1400640 1 --2- 172.21.15.33:0/4103609982 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7e4103220 0x7ff7e41994d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:21.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.691+0000 7ff7e1e00640 1 -- 172.21.15.33:0/4103609982 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7e4103220 msgr2=0x7ff7e41994d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:21.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.691+0000 7ff7e1e00640 1 --2- 172.21.15.33:0/4103609982 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7e4103220 0x7ff7e41994d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:21.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.691+0000 7ff7e1e00640 1 -- 172.21.15.33:0/4103609982 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7e4108870 msgr2=0x7ff7e419d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:21.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.691+0000 7ff7e1e00640 1 --2- 172.21.15.33:0/4103609982 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7e4108870 0x7ff7e419d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:21.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.691+0000 7ff7e1e00640 1 -- 172.21.15.33:0/4103609982 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7e419df80 con 0x7ff7e4102860 2024-05-03T18:47:21.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.691+0000 7ff7e1e00640 1 --2- 172.21.15.33:0/4103609982 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7e4102860 0x7ff7e4198f90 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7ff7d400c920 tx=0x7ff7d400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:47:21.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.691+0000 7ff7daa00640 1 -- 172.21.15.33:0/4103609982 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7d4007c70 con 0x7ff7e4102860 2024-05-03T18:47:21.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.691+0000 7ff7daa00640 1 -- 172.21.15.33:0/4103609982 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff7d4007e10 con 0x7ff7e4102860 2024-05-03T18:47:21.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.691+0000 7ff7e3200640 1 -- 172.21.15.33:0/4103609982 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7e419e270 con 0x7ff7e4102860 2024-05-03T18:47:21.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.691+0000 7ff7daa00640 1 -- 172.21.15.33:0/4103609982 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7d4018710 con 0x7ff7e4102860 2024-05-03T18:47:21.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.692+0000 7ff7e3200640 1 -- 172.21.15.33:0/4103609982 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7e41a5b00 con 0x7ff7e4102860 2024-05-03T18:47:21.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.693+0000 7ff7e3200640 1 -- 172.21.15.33:0/4103609982 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7e4069b40 con 0x7ff7e4102860 2024-05-03T18:47:21.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.695+0000 7ff7daa00640 1 -- 172.21.15.33:0/4103609982 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7ff7d400f450 con 0x7ff7e4102860 2024-05-03T18:47:21.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.697+0000 7ff7daa00640 1 --2- 172.21.15.33:0/4103609982 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7ff7c80b2960 0x7ff7c80b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:21.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.697+0000 7ff7daa00640 1 -- 172.21.15.33:0/4103609982 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 9518+0+0 (secure 0 0 0) 0x7ff7d40e2ec0 con 0x7ff7e4102860 2024-05-03T18:47:21.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.697+0000 7ff7e1400640 1 --2- 172.21.15.33:0/4103609982 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7ff7c80b2960 0x7ff7c80b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:21.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.698+0000 7ff7e1400640 1 --2- 172.21.15.33:0/4103609982 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7ff7c80b2960 0x7ff7c80b4e20 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc0057c0 tx=0x7ff7cc000bc0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:47:21.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.699+0000 7ff7daa00640 1 -- 172.21.15.33:0/4103609982 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7ff7d4014030 con 0x7ff7e4102860 2024-05-03T18:47:21.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.917+0000 7ff7e3200640 1 -- 172.21.15.33:0/4103609982 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff7e410adf0 con 0x7ff7e4102860 2024-05-03T18:47:21.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.918+0000 7ff7daa00640 1 -- 172.21.15.33:0/4103609982 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v120) ==== 104+0+11 (secure 0 0 0) 0x7ff7d40aa4f0 con 0x7ff7e4102860 2024-05-03T18:47:21.919 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-05-03T18:47:21.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.921+0000 7ff7e3200640 1 -- 172.21.15.33:0/4103609982 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7ff7c80b2960 msgr2=0x7ff7c80b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:21.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.921+0000 7ff7e3200640 1 --2- 172.21.15.33:0/4103609982 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7ff7c80b2960 0x7ff7c80b4e20 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc0057c0 tx=0x7ff7cc000bc0 comp rx=0 tx=0).stop 2024-05-03T18:47:21.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.921+0000 7ff7e3200640 1 -- 172.21.15.33:0/4103609982 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7e4102860 msgr2=0x7ff7e4198f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:21.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.921+0000 7ff7e3200640 1 --2- 172.21.15.33:0/4103609982 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7e4102860 0x7ff7e4198f90 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7ff7d400c920 tx=0x7ff7d400cde0 comp rx=0 tx=0).stop 2024-05-03T18:47:21.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.921+0000 7ff7e3200640 1 -- 172.21.15.33:0/4103609982 shutdown_connections 2024-05-03T18:47:21.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.921+0000 7ff7e3200640 1 --2- 172.21.15.33:0/4103609982 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7ff7c80b2960 0x7ff7c80b4e20 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:21.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.921+0000 7ff7e3200640 1 --2- 172.21.15.33:0/4103609982 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7e4108870 0x7ff7e419d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:21.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.921+0000 7ff7e3200640 1 --2- 172.21.15.33:0/4103609982 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7e4103220 0x7ff7e41994d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:21.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.921+0000 7ff7e3200640 1 --2- 172.21.15.33:0/4103609982 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7e4102860 0x7ff7e4198f90 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:21.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.921+0000 7ff7e3200640 1 -- 172.21.15.33:0/4103609982 >> 172.21.15.33:0/4103609982 conn(0x7ff7e40fe350 msgr2=0x7ff7e40feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:47:21.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.921+0000 7ff7e3200640 1 -- 172.21.15.33:0/4103609982 shutdown_connections 2024-05-03T18:47:21.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:21.921+0000 7ff7e3200640 1 -- 172.21.15.33:0/4103609982 wait complete. 2024-05-03T18:47:21.941 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-03T18:47:22.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.155+0000 7fdf58a00640 1 -- 172.21.15.33:0/2385185107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf54074d50 msgr2=0x7fdf54075130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:22.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.155+0000 7fdf58a00640 1 --2- 172.21.15.33:0/2385185107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf54074d50 0x7fdf54075130 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c009900 tx=0x7fdf3c02a480 comp rx=0 tx=0).stop 2024-05-03T18:47:22.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.156+0000 7fdf58a00640 1 -- 172.21.15.33:0/2385185107 shutdown_connections 2024-05-03T18:47:22.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.156+0000 7fdf58a00640 1 --2- 172.21.15.33:0/2385185107 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf54113860 0x7fdf54115c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:22.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.156+0000 7fdf58a00640 1 --2- 172.21.15.33:0/2385185107 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf54075670 0x7fdf54113320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:22.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.156+0000 7fdf58a00640 1 --2- 172.21.15.33:0/2385185107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf54074d50 0x7fdf54075130 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:22.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.156+0000 7fdf58a00640 1 -- 172.21.15.33:0/2385185107 >> 172.21.15.33:0/2385185107 conn(0x7fdf540fe380 msgr2=0x7fdf541007a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:47:22.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.156+0000 7fdf58a00640 1 -- 172.21.15.33:0/2385185107 shutdown_connections 2024-05-03T18:47:22.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.156+0000 7fdf58a00640 1 -- 172.21.15.33:0/2385185107 wait complete. 2024-05-03T18:47:22.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.157+0000 7fdf58a00640 1 Processor -- start 2024-05-03T18:47:22.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.157+0000 7fdf58a00640 1 -- start start 2024-05-03T18:47:22.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.157+0000 7fdf58a00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf54074d50 0x7fdf5419f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:22.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.157+0000 7fdf58a00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf54075670 0x7fdf5419fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:22.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.157+0000 7fdf58a00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf54113860 0x7fdf541a3e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:22.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.157+0000 7fdf58a00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fdf54118fe0 con 0x7fdf54113860 2024-05-03T18:47:22.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.157+0000 7fdf58a00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fdf54118e60 con 0x7fdf54075670 2024-05-03T18:47:22.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.157+0000 7fdf58a00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fdf54119160 con 0x7fdf54074d50 2024-05-03T18:47:22.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.158+0000 7fdf53e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf54113860 0x7fdf541a3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:22.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.158+0000 7fdf53e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf54113860 0x7fdf541a3e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53412/0 (socket says 172.21.15.33:53412) 2024-05-03T18:47:22.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.158+0000 7fdf53e00640 1 -- 172.21.15.33:0/3381723112 learned_addr learned my addr 172.21.15.33:0/3381723112 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:47:22.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.158+0000 7fdf53400640 1 --2- 172.21.15.33:0/3381723112 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf54074d50 0x7fdf5419f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:22.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.158+0000 7fdf52a00640 1 --2- 172.21.15.33:0/3381723112 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf54075670 0x7fdf5419fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:22.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.158+0000 7fdf53e00640 1 -- 172.21.15.33:0/3381723112 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf54074d50 msgr2=0x7fdf5419f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:22.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.158+0000 7fdf53e00640 1 --2- 172.21.15.33:0/3381723112 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf54074d50 0x7fdf5419f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:22.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.158+0000 7fdf53e00640 1 -- 172.21.15.33:0/3381723112 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf54075670 msgr2=0x7fdf5419fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:22.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.158+0000 7fdf53e00640 1 --2- 172.21.15.33:0/3381723112 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf54075670 0x7fdf5419fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:22.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.158+0000 7fdf53e00640 1 -- 172.21.15.33:0/3381723112 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf541a4540 con 0x7fdf54113860 2024-05-03T18:47:22.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.159+0000 7fdf53e00640 1 --2- 172.21.15.33:0/3381723112 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf54113860 0x7fdf541a3e10 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c00c970 tx=0x7fdf4c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:47:22.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.159+0000 7fdf4be00640 1 -- 172.21.15.33:0/3381723112 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf4c007bb0 con 0x7fdf54113860 2024-05-03T18:47:22.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.159+0000 7fdf58a00640 1 -- 172.21.15.33:0/3381723112 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf541a4830 con 0x7fdf54113860 2024-05-03T18:47:22.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.159+0000 7fdf4be00640 1 -- 172.21.15.33:0/3381723112 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdf4c007d50 con 0x7fdf54113860 2024-05-03T18:47:22.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.159+0000 7fdf4be00640 1 -- 172.21.15.33:0/3381723112 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf4c018620 con 0x7fdf54113860 2024-05-03T18:47:22.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.159+0000 7fdf58a00640 1 -- 172.21.15.33:0/3381723112 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf541ac0c0 con 0x7fdf54113860 2024-05-03T18:47:22.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.160+0000 7fdf58a00640 1 -- 172.21.15.33:0/3381723112 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf540696f0 con 0x7fdf54113860 2024-05-03T18:47:22.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.163+0000 7fdf4be00640 1 -- 172.21.15.33:0/3381723112 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fdf4c00f450 con 0x7fdf54113860 2024-05-03T18:47:22.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.164+0000 7fdf4be00640 1 --2- 172.21.15.33:0/3381723112 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fdf380b29e0 0x7fdf380b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:22.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.165+0000 7fdf4be00640 1 -- 172.21.15.33:0/3381723112 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 9518+0+0 (secure 0 0 0) 0x7fdf4c0e2a90 con 0x7fdf54113860 2024-05-03T18:47:22.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.165+0000 7fdf53400640 1 --2- 172.21.15.33:0/3381723112 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fdf380b29e0 0x7fdf380b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:22.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.165+0000 7fdf53400640 1 --2- 172.21.15.33:0/3381723112 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fdf380b29e0 0x7fdf380b4ea0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c006010 tx=0x7fdf3c005a20 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:47:22.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.168+0000 7fdf4be00640 1 -- 172.21.15.33:0/3381723112 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fdf4c014030 con 0x7fdf54113860 2024-05-03T18:47:22.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:47:22 smithi033 ceph-mon[28330]: pgmap v637: 129 pgs: 129 active+clean; 3.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 364 B/s wr, 2 op/s 2024-05-03T18:47:22.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:47:22 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/2414168100' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-03T18:47:22.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:47:22 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/4103609982' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-03T18:47:22.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:47:22 smithi099 ceph-mon[31658]: pgmap v637: 129 pgs: 129 active+clean; 3.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 364 B/s wr, 2 op/s 2024-05-03T18:47:22.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:47:22 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/2414168100' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-03T18:47:22.297 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:47:22 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/4103609982' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-03T18:47:22.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:47:22 smithi150 ceph-mon[32492]: pgmap v637: 129 pgs: 129 active+clean; 3.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 364 B/s wr, 2 op/s 2024-05-03T18:47:22.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:47:22 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/2414168100' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-03T18:47:22.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:47:22 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/4103609982' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-03T18:47:22.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.382+0000 7fdf58a00640 1 -- 172.21.15.33:0/3381723112 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fdf54105d20 con 0x7fdf54113860 2024-05-03T18:47:22.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.383+0000 7fdf4be00640 1 -- 172.21.15.33:0/3381723112 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v120) ==== 100+0+11 (secure 0 0 0) 0x7fdf4c0aa030 con 0x7fdf54113860 2024-05-03T18:47:22.385 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2024-05-03T18:47:22.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.386+0000 7fdf58a00640 1 -- 172.21.15.33:0/3381723112 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fdf380b29e0 msgr2=0x7fdf380b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:22.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.386+0000 7fdf58a00640 1 --2- 172.21.15.33:0/3381723112 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fdf380b29e0 0x7fdf380b4ea0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c006010 tx=0x7fdf3c005a20 comp rx=0 tx=0).stop 2024-05-03T18:47:22.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.386+0000 7fdf58a00640 1 -- 172.21.15.33:0/3381723112 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf54113860 msgr2=0x7fdf541a3e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:22.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.386+0000 7fdf58a00640 1 --2- 172.21.15.33:0/3381723112 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf54113860 0x7fdf541a3e10 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c00c970 tx=0x7fdf4c00ce30 comp rx=0 tx=0).stop 2024-05-03T18:47:22.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.387+0000 7fdf58a00640 1 -- 172.21.15.33:0/3381723112 shutdown_connections 2024-05-03T18:47:22.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.387+0000 7fdf58a00640 1 --2- 172.21.15.33:0/3381723112 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fdf380b29e0 0x7fdf380b4ea0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:22.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.387+0000 7fdf58a00640 1 --2- 172.21.15.33:0/3381723112 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf54113860 0x7fdf541a3e10 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:22.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.387+0000 7fdf58a00640 1 --2- 172.21.15.33:0/3381723112 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf54075670 0x7fdf5419fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:22.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.387+0000 7fdf58a00640 1 --2- 172.21.15.33:0/3381723112 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf54074d50 0x7fdf5419f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:22.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.387+0000 7fdf58a00640 1 -- 172.21.15.33:0/3381723112 >> 172.21.15.33:0/3381723112 conn(0x7fdf540fe380 msgr2=0x7fdf540ffc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:47:22.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.387+0000 7fdf58a00640 1 -- 172.21.15.33:0/3381723112 shutdown_connections 2024-05-03T18:47:22.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.387+0000 7fdf58a00640 1 -- 172.21.15.33:0/3381723112 wait complete. 2024-05-03T18:47:22.408 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-03T18:47:22.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.619+0000 7f928b200640 1 -- 172.21.15.33:0/1343172755 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f928c10f1d0 msgr2=0x7f928c1115c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:22.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.619+0000 7f928b200640 1 --2- 172.21.15.33:0/1343172755 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f928c10f1d0 0x7f928c1115c0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f9284009900 tx=0x7f928402a3f0 comp rx=0 tx=0).stop 2024-05-03T18:47:22.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.620+0000 7f928b200640 1 -- 172.21.15.33:0/1343172755 shutdown_connections 2024-05-03T18:47:22.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.620+0000 7f928b200640 1 --2- 172.21.15.33:0/1343172755 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f928c10f1d0 0x7f928c1115c0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:22.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.620+0000 7f928b200640 1 --2- 172.21.15.33:0/1343172755 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f928c101fd0 0x7f928c10ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:22.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.620+0000 7f928b200640 1 --2- 172.21.15.33:0/1343172755 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f928c101620 0x7f928c101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:22.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.620+0000 7f928b200640 1 -- 172.21.15.33:0/1343172755 >> 172.21.15.33:0/1343172755 conn(0x7f928c0fd230 msgr2=0x7f928c0ff650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:47:22.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.620+0000 7f928b200640 1 -- 172.21.15.33:0/1343172755 shutdown_connections 2024-05-03T18:47:22.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.620+0000 7f928b200640 1 -- 172.21.15.33:0/1343172755 wait complete. 2024-05-03T18:47:22.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.621+0000 7f928b200640 1 Processor -- start 2024-05-03T18:47:22.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.621+0000 7f928b200640 1 -- start start 2024-05-03T18:47:22.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.621+0000 7f928b200640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f928c101620 0x7f928c19b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:22.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.621+0000 7f928b200640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f928c101fd0 0x7f928c19b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:22.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.621+0000 7f928b200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f928c10f1d0 0x7f928c19fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:22.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.621+0000 7f928b200640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f928c114bf0 con 0x7f928c101fd0 2024-05-03T18:47:22.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.621+0000 7f928b200640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f928c114a70 con 0x7f928c10f1d0 2024-05-03T18:47:22.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.621+0000 7f928b200640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f928c114d70 con 0x7f928c101620 2024-05-03T18:47:22.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.622+0000 7f9289400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f928c101fd0 0x7f928c19b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:22.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.622+0000 7f9289400640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f928c101fd0 0x7f928c19b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53442/0 (socket says 172.21.15.33:53442) 2024-05-03T18:47:22.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.622+0000 7f9289400640 1 -- 172.21.15.33:0/2182118647 learned_addr learned my addr 172.21.15.33:0/2182118647 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:47:22.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.622+0000 7f928a800640 1 --2- 172.21.15.33:0/2182118647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f928c10f1d0 0x7f928c19fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:22.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.622+0000 7f9289e00640 1 --2- 172.21.15.33:0/2182118647 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f928c101620 0x7f928c19b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:22.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.622+0000 7f9289400640 1 -- 172.21.15.33:0/2182118647 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f928c101620 msgr2=0x7f928c19b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:22.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.622+0000 7f9289400640 1 --2- 172.21.15.33:0/2182118647 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f928c101620 0x7f928c19b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:22.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.622+0000 7f9289400640 1 -- 172.21.15.33:0/2182118647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f928c10f1d0 msgr2=0x7f928c19fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:22.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.622+0000 7f9289400640 1 --2- 172.21.15.33:0/2182118647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f928c10f1d0 0x7f928c19fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:22.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.622+0000 7f9289400640 1 -- 172.21.15.33:0/2182118647 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f928c1a01a0 con 0x7f928c101fd0 2024-05-03T18:47:22.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.623+0000 7f9289400640 1 --2- 172.21.15.33:0/2182118647 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f928c101fd0 0x7f928c19b6f0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f927400d950 tx=0x7f927400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:47:22.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.623+0000 7f9282a00640 1 -- 172.21.15.33:0/2182118647 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9274017070 con 0x7f928c101fd0 2024-05-03T18:47:22.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.623+0000 7f9282a00640 1 -- 172.21.15.33:0/2182118647 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9274009c30 con 0x7f928c101fd0 2024-05-03T18:47:22.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.623+0000 7f928b200640 1 -- 172.21.15.33:0/2182118647 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f928c1a0490 con 0x7f928c101fd0 2024-05-03T18:47:22.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.623+0000 7f9282a00640 1 -- 172.21.15.33:0/2182118647 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f927400f620 con 0x7f928c101fd0 2024-05-03T18:47:22.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.623+0000 7f928b200640 1 -- 172.21.15.33:0/2182118647 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f928c1a7d20 con 0x7f928c101fd0 2024-05-03T18:47:22.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.624+0000 7f928b200640 1 -- 172.21.15.33:0/2182118647 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f928c0696f0 con 0x7f928c101fd0 2024-05-03T18:47:22.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.627+0000 7f9282a00640 1 -- 172.21.15.33:0/2182118647 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f9274005030 con 0x7f928c101fd0 2024-05-03T18:47:22.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.629+0000 7f9282a00640 1 --2- 172.21.15.33:0/2182118647 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f92700b2960 0x7f92700b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:22.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.629+0000 7f9282a00640 1 -- 172.21.15.33:0/2182118647 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 9518+0+0 (secure 0 0 0) 0x7f92740e2680 con 0x7f928c101fd0 2024-05-03T18:47:22.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.629+0000 7f9289e00640 1 --2- 172.21.15.33:0/2182118647 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f92700b2960 0x7f92700b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:22.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.630+0000 7f9289e00640 1 --2- 172.21.15.33:0/2182118647 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f92700b2960 0x7f92700b4e20 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f927c005fd0 tx=0x7f927c009870 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:47:22.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.631+0000 7f9282a00640 1 -- 172.21.15.33:0/2182118647 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f92740a9c20 con 0x7f928c101fd0 2024-05-03T18:47:22.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.853+0000 7f928b200640 1 -- 172.21.15.33:0/2182118647 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f928c10a2d0 con 0x7f928c101fd0 2024-05-03T18:47:22.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.854+0000 7f9282a00640 1 -- 172.21.15.33:0/2182118647 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v120) ==== 103+0+11 (secure 0 0 0) 0x7f92740e2020 con 0x7f928c101fd0 2024-05-03T18:47:22.855 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-05-03T18:47:22.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.857+0000 7f928b200640 1 -- 172.21.15.33:0/2182118647 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f92700b2960 msgr2=0x7f92700b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:22.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.857+0000 7f928b200640 1 --2- 172.21.15.33:0/2182118647 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f92700b2960 0x7f92700b4e20 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f927c005fd0 tx=0x7f927c009870 comp rx=0 tx=0).stop 2024-05-03T18:47:22.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.857+0000 7f928b200640 1 -- 172.21.15.33:0/2182118647 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f928c101fd0 msgr2=0x7f928c19b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:22.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.857+0000 7f928b200640 1 --2- 172.21.15.33:0/2182118647 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f928c101fd0 0x7f928c19b6f0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f927400d950 tx=0x7f927400de10 comp rx=0 tx=0).stop 2024-05-03T18:47:22.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.857+0000 7f928b200640 1 -- 172.21.15.33:0/2182118647 shutdown_connections 2024-05-03T18:47:22.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.857+0000 7f928b200640 1 --2- 172.21.15.33:0/2182118647 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f92700b2960 0x7f92700b4e20 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:22.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.857+0000 7f928b200640 1 --2- 172.21.15.33:0/2182118647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f928c10f1d0 0x7f928c19fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:22.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.857+0000 7f928b200640 1 --2- 172.21.15.33:0/2182118647 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f928c101fd0 0x7f928c19b6f0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:22.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.857+0000 7f928b200640 1 --2- 172.21.15.33:0/2182118647 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f928c101620 0x7f928c19b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:22.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.857+0000 7f928b200640 1 -- 172.21.15.33:0/2182118647 >> 172.21.15.33:0/2182118647 conn(0x7f928c0fd230 msgr2=0x7f928c0ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:47:22.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.857+0000 7f928b200640 1 -- 172.21.15.33:0/2182118647 shutdown_connections 2024-05-03T18:47:22.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:22.857+0000 7f928b200640 1 -- 172.21.15.33:0/2182118647 wait complete. 2024-05-03T18:47:22.878 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-05-03T18:47:23.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:23.093+0000 7f04f1e00640 1 -- 172.21.15.33:0/2915734869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04ec077800 msgr2=0x7f04ec1132e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:23.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:23.093+0000 7f04f1e00640 1 --2- 172.21.15.33:0/2915734869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04ec077800 0x7f04ec1132e0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f04d4009900 tx=0x7f04d402a3f0 comp rx=0 tx=0).stop 2024-05-03T18:47:23.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:23.094+0000 7f04f1e00640 1 -- 172.21.15.33:0/2915734869 shutdown_connections 2024-05-03T18:47:23.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:23.094+0000 7f04f1e00640 1 --2- 172.21.15.33:0/2915734869 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f04ec113820 0x7f04ec115c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:23.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:23.094+0000 7f04f1e00640 1 --2- 172.21.15.33:0/2915734869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04ec077800 0x7f04ec1132e0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:23.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:23.094+0000 7f04f1e00640 1 --2- 172.21.15.33:0/2915734869 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f04ec076ee0 0x7f04ec0772c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:23.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:23.094+0000 7f04f1e00640 1 -- 172.21.15.33:0/2915734869 >> 172.21.15.33:0/2915734869 conn(0x7f04ec1006b0 msgr2=0x7f04ec102ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:47:23.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:23.094+0000 7f04f1e00640 1 -- 172.21.15.33:0/2915734869 shutdown_connections 2024-05-03T18:47:23.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:23.094+0000 7f04f1e00640 1 -- 172.21.15.33:0/2915734869 wait complete. 2024-05-03T18:47:23.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:23.095+0000 7f04f1e00640 1 Processor -- start 2024-05-03T18:47:23.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:23.095+0000 7f04f1e00640 1 -- start start 2024-05-03T18:47:23.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:23.095+0000 7f04f1e00640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f04ec076ee0 0x7f04ec1a55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:23.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:23.095+0000 7f04f1e00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04ec077800 0x7f04ec1a5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:23.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:23.095+0000 7f04f1e00640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f04ec113820 0x7f04ec19f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:23.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:23.095+0000 7f04f1e00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f04ec119130 con 0x7f04ec077800 2024-05-03T18:47:23.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:23.095+0000 7f04f1e00640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f04ec118fb0 con 0x7f04ec113820 2024-05-03T18:47:23.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:23.095+0000 7f04f1e00640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f04ec1192b0 con 0x7f04ec076ee0 2024-05-03T18:47:23.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:23.095+0000 7f04ea000640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04ec077800 0x7f04ec1a5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:23.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:23.096+0000 7f04ea000640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04ec077800 0x7f04ec1a5ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53458/0 (socket says 172.21.15.33:53458) 2024-05-03T18:47:23.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:23.096+0000 7f04ea000640 1 -- 172.21.15.33:0/975156145 learned_addr learned my addr 172.21.15.33:0/975156145 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:47:23.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:23.096+0000 7f04eaa00640 1 --2- 172.21.15.33:0/975156145 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f04ec076ee0 0x7f04ec1a55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:23.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:23.096+0000 7f04eb400640 1 --2- 172.21.15.33:0/975156145 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f04ec113820 0x7f04ec19f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:23.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:23.096+0000 7f04ea000640 1 -- 172.21.15.33:0/975156145 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f04ec076ee0 msgr2=0x7f04ec1a55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:23.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:23.096+0000 7f04ea000640 1 --2- 172.21.15.33:0/975156145 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f04ec076ee0 0x7f04ec1a55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:23.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:23.096+0000 7f04ea000640 1 -- 172.21.15.33:0/975156145 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f04ec113820 msgr2=0x7f04ec19f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:23.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:23.096+0000 7f04ea000640 1 --2- 172.21.15.33:0/975156145 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f04ec113820 0x7f04ec19f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:23.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:23.096+0000 7f04ea000640 1 -- 172.21.15.33:0/975156145 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04ec19ff20 con 0x7f04ec077800 2024-05-03T18:47:23.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:23.096+0000 7f04ea000640 1 --2- 172.21.15.33:0/975156145 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04ec077800 0x7f04ec1a5ae0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f04d4004770 tx=0x7f04d4003ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:47:23.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:23.097+0000 7f04f0a00640 1 -- 172.21.15.33:0/975156145 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d403f070 con 0x7f04ec077800 2024-05-03T18:47:23.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:23.097+0000 7f04f0a00640 1 -- 172.21.15.33:0/975156145 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f04d403bd00 con 0x7f04ec077800 2024-05-03T18:47:23.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:23.097+0000 7f04f0a00640 1 -- 172.21.15.33:0/975156145 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d4035030 con 0x7f04ec077800 2024-05-03T18:47:23.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:23.097+0000 7f04f1e00640 1 -- 172.21.15.33:0/975156145 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04ec1a01b0 con 0x7f04ec077800 2024-05-03T18:47:23.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:23.097+0000 7f04f1e00640 1 -- 172.21.15.33:0/975156145 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04ec1ac340 con 0x7f04ec077800 2024-05-03T18:47:23.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:23.098+0000 7f04f1e00640 1 -- 172.21.15.33:0/975156145 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04ec078620 con 0x7f04ec077800 2024-05-03T18:47:23.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:23.100+0000 7f04f0a00640 1 -- 172.21.15.33:0/975156145 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f04d4042070 con 0x7f04ec077800 2024-05-03T18:47:23.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:23.102+0000 7f04f0a00640 1 --2- 172.21.15.33:0/975156145 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f04d00b29e0 0x7f04d00b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:23.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:23.102+0000 7f04f0a00640 1 -- 172.21.15.33:0/975156145 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 9518+0+0 (secure 0 0 0) 0x7f04d41032b0 con 0x7f04ec077800 2024-05-03T18:47:23.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:23.102+0000 7f04eaa00640 1 --2- 172.21.15.33:0/975156145 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f04d00b29e0 0x7f04d00b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:23.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:23.104+0000 7f04eaa00640 1 --2- 172.21.15.33:0/975156145 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f04d00b29e0 0x7f04d00b4ea0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f04dc005fd0 tx=0x7f04dc005850 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:47:23.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:23.107+0000 7f04f0a00640 1 -- 172.21.15.33:0/975156145 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f04d40ca850 con 0x7f04ec077800 2024-05-03T18:47:23.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:47:23 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/3381723112' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-03T18:47:23.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:47:23 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/2182118647' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-03T18:47:23.297 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:47:23 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/3381723112' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-03T18:47:23.297 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:47:23 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/2182118647' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-03T18:47:23.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:23.376+0000 7f04f1e00640 1 -- 172.21.15.33:0/975156145 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f04ec10e9b0 con 0x7f04ec077800 2024-05-03T18:47:23.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:47:23 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/3381723112' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-03T18:47:23.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:47:23 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/2182118647' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-03T18:47:23.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:23.427+0000 7f04f0a00640 1 -- 172.21.15.33:0/975156145 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v25) ==== 81+0+816174 (secure 0 0 0) 0x7f04d40ca150 con 0x7f04ec077800 2024-05-03T18:47:23.441 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:47:23.441 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-03T18:47:23.441 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 25, 2024-05-03T18:47:23.441 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.441 INFO:teuthology.orchestra.run.smithi033.stdout: "active_gid": 15120, 2024-05-03T18:47:23.441 INFO:teuthology.orchestra.run.smithi033.stdout: "active_name": "x", 2024-05-03T18:47:23.441 INFO:teuthology.orchestra.run.smithi033.stdout: "active_addrs": { 2024-05-03T18:47:23.441 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-03T18:47:23.441 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:23.441 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-03T18:47:23.441 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6800", 2024-05-03T18:47:23.441 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 3302011241 2024-05-03T18:47:23.442 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.442 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:23.442 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-03T18:47:23.442 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6801", 2024-05-03T18:47:23.442 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 3302011241 2024-05-03T18:47:23.442 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:23.442 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-03T18:47:23.442 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.442 INFO:teuthology.orchestra.run.smithi033.stdout: "active_addr": "172.21.15.33:6801/3302011241", 2024-05-03T18:47:23.442 INFO:teuthology.orchestra.run.smithi033.stdout: "active_change": "2024-05-03T18:27:11.723148+0000", 2024-05-03T18:47:23.442 INFO:teuthology.orchestra.run.smithi033.stdout: "active_mgr_features": 4540701547738038271, 2024-05-03T18:47:23.442 INFO:teuthology.orchestra.run.smithi033.stdout: "available": true, 2024-05-03T18:47:23.442 INFO:teuthology.orchestra.run.smithi033.stdout: "standbys": [ 2024-05-03T18:47:23.442 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:23.443 INFO:teuthology.orchestra.run.smithi033.stdout: "gid": 24892, 2024-05-03T18:47:23.443 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "z", 2024-05-03T18:47:23.443 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr_features": 4540701547738038271, 2024-05-03T18:47:23.443 INFO:teuthology.orchestra.run.smithi033.stdout: "available_modules": [ 2024-05-03T18:47:23.443 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:23.443 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "alerts", 2024-05-03T18:47:23.443 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:23.443 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:23.443 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:23.443 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-05-03T18:47:23.443 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-05-03T18:47:23.443 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-03T18:47:23.443 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.444 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.444 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-05-03T18:47:23.444 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.444 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.444 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.444 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "How frequently to reexamine health status", 2024-05-03T18:47:23.444 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.444 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.444 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.444 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.444 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:23.444 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:23.444 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.445 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.445 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.445 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:23.445 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.445 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.445 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:23.445 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:23.445 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:23.445 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:23.445 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:23.445 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:23.445 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:23.445 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:23.445 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.446 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.446 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.446 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.446 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.446 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:23.446 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:23.446 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.446 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.446 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.446 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.446 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.446 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.447 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.447 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.447 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.447 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.447 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.447 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.447 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:23.447 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:23.447 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.447 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.447 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.447 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:23.447 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.447 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.447 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:23.448 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:23.448 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:23.448 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:23.448 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:23.448 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:23.448 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:23.448 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:23.448 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.448 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.448 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.448 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.448 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.448 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:23.448 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:23.449 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.449 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.449 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.449 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.449 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.449 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.449 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.449 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.449 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.449 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.449 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.449 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.449 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_destination": { 2024-05-03T18:47:23.449 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_destination", 2024-05-03T18:47:23.449 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.450 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.450 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.450 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:23.450 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.450 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.450 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.450 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Email address to send alerts to", 2024-05-03T18:47:23.450 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.450 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.450 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.450 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.450 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_from_name": { 2024-05-03T18:47:23.450 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_from_name", 2024-05-03T18:47:23.450 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.451 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.451 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.451 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "Ceph", 2024-05-03T18:47:23.451 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.451 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.451 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.451 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Email From: name", 2024-05-03T18:47:23.451 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.451 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.451 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.451 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.451 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_host": { 2024-05-03T18:47:23.451 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_host", 2024-05-03T18:47:23.452 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.452 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.452 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.452 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:23.452 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.452 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.452 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.452 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "SMTP server", 2024-05-03T18:47:23.452 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.453 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.453 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.453 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.453 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_password": { 2024-05-03T18:47:23.453 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_password", 2024-05-03T18:47:23.453 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.453 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.453 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.453 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:23.454 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.454 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.454 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.454 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Password to authenticate with", 2024-05-03T18:47:23.454 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.454 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.454 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.454 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.454 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_port": { 2024-05-03T18:47:23.454 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_port", 2024-05-03T18:47:23.454 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:23.454 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.454 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.455 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "465", 2024-05-03T18:47:23.455 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.455 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.455 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.455 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "SMTP port", 2024-05-03T18:47:23.455 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.455 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.455 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.455 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.455 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_sender": { 2024-05-03T18:47:23.455 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_sender", 2024-05-03T18:47:23.455 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.455 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.455 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.455 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:23.456 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.456 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.456 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.456 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "SMTP envelope sender", 2024-05-03T18:47:23.456 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.456 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.456 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.456 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.456 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_ssl": { 2024-05-03T18:47:23.456 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_ssl", 2024-05-03T18:47:23.456 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.456 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.457 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.457 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:23.457 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.457 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.457 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.457 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-03T18:47:23.457 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.457 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.457 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.457 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.457 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_user": { 2024-05-03T18:47:23.458 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_user", 2024-05-03T18:47:23.458 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.458 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.458 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.458 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:23.458 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.458 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.458 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.458 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "User to authenticate as", 2024-05-03T18:47:23.458 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.458 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.458 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.458 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.458 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:23.459 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:23.459 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:23.459 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:23.459 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.459 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:23.459 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.459 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.459 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.459 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.459 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.459 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.459 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.459 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:23.459 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:23.459 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.460 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:23.460 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "balancer", 2024-05-03T18:47:23.460 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:23.460 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:23.460 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:23.460 INFO:teuthology.orchestra.run.smithi033.stdout: "active": { 2024-05-03T18:47:23.460 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "active", 2024-05-03T18:47:23.460 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.460 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.460 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.460 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:23.460 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.460 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.461 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.461 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "automatically balance PGs across cluster", 2024-05-03T18:47:23.461 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.461 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.461 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.461 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.461 INFO:teuthology.orchestra.run.smithi033.stdout: "begin_time": { 2024-05-03T18:47:23.461 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "begin_time", 2024-05-03T18:47:23.461 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.461 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.461 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.461 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0000", 2024-05-03T18:47:23.461 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.462 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.462 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.462 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "beginning time of day to automatically balance", 2024-05-03T18:47:23.462 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-03T18:47:23.462 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.462 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.462 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.462 INFO:teuthology.orchestra.run.smithi033.stdout: "begin_weekday": { 2024-05-03T18:47:23.462 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "begin_weekday", 2024-05-03T18:47:23.462 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-05-03T18:47:23.462 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.462 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.462 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:23.462 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "0", 2024-05-03T18:47:23.463 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "6", 2024-05-03T18:47:23.463 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.463 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-03T18:47:23.463 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-03T18:47:23.463 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.463 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.463 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.463 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_compat_max_iterations": { 2024-05-03T18:47:23.463 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crush_compat_max_iterations", 2024-05-03T18:47:23.463 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-05-03T18:47:23.463 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.463 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.463 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "25", 2024-05-03T18:47:23.463 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1", 2024-05-03T18:47:23.464 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "250", 2024-05-03T18:47:23.464 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.464 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-03T18:47:23.464 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.464 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.464 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.464 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.464 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_compat_metrics": { 2024-05-03T18:47:23.464 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crush_compat_metrics", 2024-05-03T18:47:23.464 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.464 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.464 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.464 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "pgs,objects,bytes", 2024-05-03T18:47:23.464 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.464 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.464 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.465 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-03T18:47:23.465 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-03T18:47:23.465 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.465 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.465 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.465 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_compat_step": { 2024-05-03T18:47:23.465 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crush_compat_step", 2024-05-03T18:47:23.465 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-05-03T18:47:23.465 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.465 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.465 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0.5", 2024-05-03T18:47:23.465 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "0.001", 2024-05-03T18:47:23.465 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "0.999", 2024-05-03T18:47:23.465 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.465 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "aggressiveness of optimization", 2024-05-03T18:47:23.465 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-03T18:47:23.466 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.466 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.466 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.466 INFO:teuthology.orchestra.run.smithi033.stdout: "end_time": { 2024-05-03T18:47:23.466 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "end_time", 2024-05-03T18:47:23.466 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.466 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.466 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.466 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "2359", 2024-05-03T18:47:23.466 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.466 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.466 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.466 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "ending time of day to automatically balance", 2024-05-03T18:47:23.466 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-03T18:47:23.467 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.467 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.467 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.467 INFO:teuthology.orchestra.run.smithi033.stdout: "end_weekday": { 2024-05-03T18:47:23.467 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "end_weekday", 2024-05-03T18:47:23.467 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-05-03T18:47:23.467 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.467 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.467 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:23.467 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "0", 2024-05-03T18:47:23.467 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "6", 2024-05-03T18:47:23.467 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.467 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-03T18:47:23.467 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-03T18:47:23.467 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.468 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.468 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.468 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:23.468 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:23.468 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.468 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.468 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.468 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:23.468 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.468 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.468 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:23.469 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:23.469 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:23.469 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:23.469 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:23.469 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:23.469 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:23.469 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:23.470 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.470 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.470 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.470 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.470 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.470 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:23.470 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:23.470 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.470 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.470 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.470 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.470 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.470 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.471 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.471 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.471 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.471 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.471 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.471 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.471 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:23.471 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:23.471 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.471 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.472 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.472 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:23.472 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.472 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.472 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:23.473 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:23.473 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:23.473 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:23.473 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:23.473 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:23.473 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:23.474 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:23.474 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.474 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.474 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.474 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.474 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.474 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:23.474 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:23.475 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.475 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.475 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.475 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.475 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.475 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.475 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.475 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.476 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.476 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.476 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.476 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.476 INFO:teuthology.orchestra.run.smithi033.stdout: "min_score": { 2024-05-03T18:47:23.476 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "min_score", 2024-05-03T18:47:23.476 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-05-03T18:47:23.476 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.476 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.476 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:23.476 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.477 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.477 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.477 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-03T18:47:23.477 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.477 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.477 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.477 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.477 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": { 2024-05-03T18:47:23.477 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mode", 2024-05-03T18:47:23.477 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.477 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.477 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.477 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "upmap", 2024-05-03T18:47:23.477 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.478 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.478 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:23.478 INFO:teuthology.orchestra.run.smithi033.stdout: "crush-compat", 2024-05-03T18:47:23.478 INFO:teuthology.orchestra.run.smithi033.stdout: "none", 2024-05-03T18:47:23.478 INFO:teuthology.orchestra.run.smithi033.stdout: "read", 2024-05-03T18:47:23.478 INFO:teuthology.orchestra.run.smithi033.stdout: "upmap", 2024-05-03T18:47:23.478 INFO:teuthology.orchestra.run.smithi033.stdout: "upmap-read" 2024-05-03T18:47:23.478 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:23.478 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Balancer mode", 2024-05-03T18:47:23.478 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.478 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.478 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.478 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.479 INFO:teuthology.orchestra.run.smithi033.stdout: "pool_ids": { 2024-05-03T18:47:23.479 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "pool_ids", 2024-05-03T18:47:23.479 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.479 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.479 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.479 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:23.479 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.479 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.479 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.479 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-03T18:47:23.479 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.479 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.479 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.479 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.479 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-05-03T18:47:23.480 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-05-03T18:47:23.480 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-03T18:47:23.480 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.480 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.480 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-05-03T18:47:23.480 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.480 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.480 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.480 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-03T18:47:23.480 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.480 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.480 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.480 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.480 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:23.480 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:23.480 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:23.481 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:23.481 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.481 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:23.481 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.481 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.481 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.481 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.481 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.481 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.481 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.481 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.481 INFO:teuthology.orchestra.run.smithi033.stdout: "upmap_max_deviation": { 2024-05-03T18:47:23.481 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "upmap_max_deviation", 2024-05-03T18:47:23.481 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:23.482 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.482 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.482 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5", 2024-05-03T18:47:23.482 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1", 2024-05-03T18:47:23.482 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.482 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.482 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-03T18:47:23.482 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-03T18:47:23.482 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.482 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.482 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.482 INFO:teuthology.orchestra.run.smithi033.stdout: "upmap_max_optimizations": { 2024-05-03T18:47:23.482 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "upmap_max_optimizations", 2024-05-03T18:47:23.483 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-05-03T18:47:23.483 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.483 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.483 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-05-03T18:47:23.483 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.483 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.483 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.483 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-03T18:47:23.483 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.483 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.483 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.483 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:23.483 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:23.483 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.484 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:23.484 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cephadm", 2024-05-03T18:47:23.484 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:23.484 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:23.484 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:23.484 INFO:teuthology.orchestra.run.smithi033.stdout: "agent_down_multiplier": { 2024-05-03T18:47:23.484 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "agent_down_multiplier", 2024-05-03T18:47:23.484 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-05-03T18:47:23.484 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.484 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.484 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "3.0", 2024-05-03T18:47:23.484 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.484 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.484 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.485 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-03T18:47:23.485 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.485 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.485 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.485 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.485 INFO:teuthology.orchestra.run.smithi033.stdout: "agent_refresh_rate": { 2024-05-03T18:47:23.485 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "agent_refresh_rate", 2024-05-03T18:47:23.485 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-03T18:47:23.485 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.485 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.485 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "20", 2024-05-03T18:47:23.485 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.486 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.486 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.486 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-03T18:47:23.486 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.486 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.486 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.486 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.486 INFO:teuthology.orchestra.run.smithi033.stdout: "agent_starting_port": { 2024-05-03T18:47:23.486 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "agent_starting_port", 2024-05-03T18:47:23.486 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:23.486 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.487 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.487 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "4721", 2024-05-03T18:47:23.487 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.487 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.487 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.487 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-03T18:47:23.487 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.487 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.487 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.487 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.487 INFO:teuthology.orchestra.run.smithi033.stdout: "allow_ptrace": { 2024-05-03T18:47:23.487 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "allow_ptrace", 2024-05-03T18:47:23.488 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.488 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.488 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.488 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.488 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.488 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.488 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.488 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-03T18:47:23.488 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-03T18:47:23.488 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.488 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.488 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.488 INFO:teuthology.orchestra.run.smithi033.stdout: "autotune_interval": { 2024-05-03T18:47:23.488 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "autotune_interval", 2024-05-03T18:47:23.489 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-03T18:47:23.489 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.489 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.489 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-05-03T18:47:23.489 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.489 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.489 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.489 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-03T18:47:23.489 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.489 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.489 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.489 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.489 INFO:teuthology.orchestra.run.smithi033.stdout: "autotune_memory_target_ratio": { 2024-05-03T18:47:23.489 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "autotune_memory_target_ratio", 2024-05-03T18:47:23.489 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-05-03T18:47:23.490 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.490 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.490 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0.7", 2024-05-03T18:47:23.490 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.490 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.490 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.490 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-03T18:47:23.490 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.490 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.490 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.490 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.490 INFO:teuthology.orchestra.run.smithi033.stdout: "cephadm_log_destination": { 2024-05-03T18:47:23.491 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cephadm_log_destination", 2024-05-03T18:47:23.491 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.491 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.491 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.491 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:23.491 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.491 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.491 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:23.491 INFO:teuthology.orchestra.run.smithi033.stdout: "file", 2024-05-03T18:47:23.491 INFO:teuthology.orchestra.run.smithi033.stdout: "file,syslog", 2024-05-03T18:47:23.491 INFO:teuthology.orchestra.run.smithi033.stdout: "syslog" 2024-05-03T18:47:23.491 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:23.491 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-03T18:47:23.491 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.492 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.492 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.492 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.492 INFO:teuthology.orchestra.run.smithi033.stdout: "cgroups_split": { 2024-05-03T18:47:23.492 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cgroups_split", 2024-05-03T18:47:23.492 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.492 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.492 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.492 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:23.492 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.492 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.492 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.492 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-03T18:47:23.492 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.492 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.492 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.493 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.493 INFO:teuthology.orchestra.run.smithi033.stdout: "config_checks_enabled": { 2024-05-03T18:47:23.493 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "config_checks_enabled", 2024-05-03T18:47:23.493 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.493 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.493 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.493 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.493 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.493 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.493 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.493 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-03T18:47:23.493 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.493 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.493 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.493 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.493 INFO:teuthology.orchestra.run.smithi033.stdout: "config_dashboard": { 2024-05-03T18:47:23.493 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "config_dashboard", 2024-05-03T18:47:23.494 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.494 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.494 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.494 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:23.494 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.494 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.494 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.494 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-03T18:47:23.494 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.494 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.494 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.494 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.494 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_alertmanager": { 2024-05-03T18:47:23.494 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_alertmanager", 2024-05-03T18:47:23.494 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.495 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.495 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.495 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-03T18:47:23.495 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.495 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.495 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.495 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Prometheus container image", 2024-05-03T18:47:23.495 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.495 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.495 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.495 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.495 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_base": { 2024-05-03T18:47:23.495 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_base", 2024-05-03T18:47:23.495 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.495 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.495 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.496 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-03T18:47:23.496 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.496 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.496 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.496 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Container image name, without the tag", 2024-05-03T18:47:23.496 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.496 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.496 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.496 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.496 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_elasticsearch": { 2024-05-03T18:47:23.496 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_elasticsearch", 2024-05-03T18:47:23.496 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.496 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.496 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.496 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-03T18:47:23.496 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.496 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.496 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.497 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "elasticsearch container image", 2024-05-03T18:47:23.497 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.497 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.497 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.497 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.497 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_grafana": { 2024-05-03T18:47:23.497 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_grafana", 2024-05-03T18:47:23.497 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.497 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.497 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.497 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-03T18:47:23.497 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.497 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.497 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.497 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Prometheus container image", 2024-05-03T18:47:23.498 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.498 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.498 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.498 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.498 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_haproxy": { 2024-05-03T18:47:23.498 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_haproxy", 2024-05-03T18:47:23.498 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.498 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.498 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.498 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-03T18:47:23.498 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.498 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.498 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.499 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "HAproxy container image", 2024-05-03T18:47:23.499 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.499 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.499 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.499 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.499 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_jaeger_agent": { 2024-05-03T18:47:23.499 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_jaeger_agent", 2024-05-03T18:47:23.499 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.499 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.499 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.499 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-03T18:47:23.499 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.499 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.499 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.499 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Jaeger agent container image", 2024-05-03T18:47:23.500 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.500 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.500 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.500 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.500 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_jaeger_collector": { 2024-05-03T18:47:23.500 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_jaeger_collector", 2024-05-03T18:47:23.500 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.500 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.500 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.500 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-03T18:47:23.500 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.500 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.500 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.500 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Jaeger collector container image", 2024-05-03T18:47:23.500 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.500 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.501 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.501 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.501 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_jaeger_query": { 2024-05-03T18:47:23.501 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_jaeger_query", 2024-05-03T18:47:23.501 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.501 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.501 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.501 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-03T18:47:23.501 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.501 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.501 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.501 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Jaeger query container image", 2024-05-03T18:47:23.501 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.501 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.502 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.502 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.502 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_keepalived": { 2024-05-03T18:47:23.502 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_keepalived", 2024-05-03T18:47:23.502 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.502 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.502 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.502 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-03T18:47:23.502 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.502 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.502 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.502 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Keepalived container image", 2024-05-03T18:47:23.502 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.503 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.503 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.503 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.503 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_loki": { 2024-05-03T18:47:23.503 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_loki", 2024-05-03T18:47:23.503 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.503 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.503 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.503 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-03T18:47:23.503 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.503 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.503 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.503 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Loki container image", 2024-05-03T18:47:23.503 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.503 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.504 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.504 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.504 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_node_exporter": { 2024-05-03T18:47:23.504 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_node_exporter", 2024-05-03T18:47:23.504 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.504 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.504 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.504 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-03T18:47:23.504 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.504 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.504 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.504 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Prometheus container image", 2024-05-03T18:47:23.504 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.504 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.504 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.504 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.505 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_nvmeof": { 2024-05-03T18:47:23.505 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_nvmeof", 2024-05-03T18:47:23.505 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.505 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.505 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.505 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-05-03T18:47:23.505 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.505 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.505 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.505 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Nvme-of container image", 2024-05-03T18:47:23.505 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.505 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.505 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.505 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.505 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_prometheus": { 2024-05-03T18:47:23.505 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_prometheus", 2024-05-03T18:47:23.505 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.506 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.506 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.506 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-03T18:47:23.506 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.506 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.506 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.506 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Prometheus container image", 2024-05-03T18:47:23.506 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.506 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.506 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.506 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.506 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_promtail": { 2024-05-03T18:47:23.506 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_promtail", 2024-05-03T18:47:23.506 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.507 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.507 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.507 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-03T18:47:23.507 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.507 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.507 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.507 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Promtail container image", 2024-05-03T18:47:23.507 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.507 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.507 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.507 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.507 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_samba": { 2024-05-03T18:47:23.507 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_samba", 2024-05-03T18:47:23.507 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.507 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.507 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.507 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-03T18:47:23.508 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.508 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.508 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.508 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Samba/SMB container image", 2024-05-03T18:47:23.508 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.508 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.508 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.508 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.508 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_snmp_gateway": { 2024-05-03T18:47:23.508 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_snmp_gateway", 2024-05-03T18:47:23.508 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.508 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.508 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.508 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-03T18:47:23.508 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.509 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.509 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.509 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "SNMP Gateway container image", 2024-05-03T18:47:23.509 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.509 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.509 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.509 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.509 INFO:teuthology.orchestra.run.smithi033.stdout: "container_init": { 2024-05-03T18:47:23.509 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_init", 2024-05-03T18:47:23.509 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.509 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.509 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.509 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:23.509 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.509 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.509 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.510 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Run podman/docker with `--init`", 2024-05-03T18:47:23.510 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.510 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.510 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.510 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.510 INFO:teuthology.orchestra.run.smithi033.stdout: "daemon_cache_timeout": { 2024-05-03T18:47:23.510 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "daemon_cache_timeout", 2024-05-03T18:47:23.510 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-03T18:47:23.510 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.510 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.510 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-05-03T18:47:23.510 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.510 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.510 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.510 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-03T18:47:23.511 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.511 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.511 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.511 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.511 INFO:teuthology.orchestra.run.smithi033.stdout: "default_cephadm_command_timeout": { 2024-05-03T18:47:23.511 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "default_cephadm_command_timeout", 2024-05-03T18:47:23.511 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:23.511 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.511 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.511 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "900", 2024-05-03T18:47:23.511 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.511 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.511 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.511 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-03T18:47:23.511 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.511 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.512 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.512 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.512 INFO:teuthology.orchestra.run.smithi033.stdout: "default_registry": { 2024-05-03T18:47:23.512 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "default_registry", 2024-05-03T18:47:23.512 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.512 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.512 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.512 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "docker.io", 2024-05-03T18:47:23.512 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.512 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.512 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.512 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-03T18:47:23.512 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.512 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.512 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.512 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.513 INFO:teuthology.orchestra.run.smithi033.stdout: "device_cache_timeout": { 2024-05-03T18:47:23.513 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "device_cache_timeout", 2024-05-03T18:47:23.513 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-03T18:47:23.513 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.513 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.513 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "1800", 2024-05-03T18:47:23.513 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.513 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.513 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.513 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "seconds to cache device inventory", 2024-05-03T18:47:23.513 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.513 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.513 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.513 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.513 INFO:teuthology.orchestra.run.smithi033.stdout: "device_enhanced_scan": { 2024-05-03T18:47:23.513 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "device_enhanced_scan", 2024-05-03T18:47:23.514 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.514 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.514 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.514 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.514 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.514 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.514 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.514 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-03T18:47:23.514 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.514 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.514 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.514 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.514 INFO:teuthology.orchestra.run.smithi033.stdout: "facts_cache_timeout": { 2024-05-03T18:47:23.514 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "facts_cache_timeout", 2024-05-03T18:47:23.514 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-03T18:47:23.515 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.515 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.515 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-05-03T18:47:23.515 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.515 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.515 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.515 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "seconds to cache host facts data", 2024-05-03T18:47:23.515 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.515 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.515 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.515 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.515 INFO:teuthology.orchestra.run.smithi033.stdout: "grafana_dashboards_path": { 2024-05-03T18:47:23.515 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "grafana_dashboards_path", 2024-05-03T18:47:23.515 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.515 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.515 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.516 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-03T18:47:23.516 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.516 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.516 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.516 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-03T18:47:23.516 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.516 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.516 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.516 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.516 INFO:teuthology.orchestra.run.smithi033.stdout: "host_check_interval": { 2024-05-03T18:47:23.516 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "host_check_interval", 2024-05-03T18:47:23.516 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-03T18:47:23.516 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.516 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.516 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-05-03T18:47:23.517 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.517 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.517 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.517 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to perform a host check", 2024-05-03T18:47:23.517 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.517 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.517 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.517 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.517 INFO:teuthology.orchestra.run.smithi033.stdout: "hw_monitoring": { 2024-05-03T18:47:23.517 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "hw_monitoring", 2024-05-03T18:47:23.517 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.517 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.517 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.517 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.517 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.518 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.518 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.518 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-03T18:47:23.518 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.518 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.518 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.518 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.518 INFO:teuthology.orchestra.run.smithi033.stdout: "inventory_list_all": { 2024-05-03T18:47:23.518 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "inventory_list_all", 2024-05-03T18:47:23.518 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.518 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.518 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.518 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.518 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.518 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.519 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.519 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-03T18:47:23.519 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.519 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.519 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.519 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.519 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:23.519 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:23.519 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.519 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.519 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.519 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:23.519 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.519 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.519 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:23.520 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:23.520 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:23.520 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:23.520 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:23.520 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:23.520 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:23.520 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:23.520 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.520 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.520 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.520 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.520 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.520 INFO:teuthology.orchestra.run.smithi033.stdout: "log_refresh_metadata": { 2024-05-03T18:47:23.520 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_refresh_metadata", 2024-05-03T18:47:23.520 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.520 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.521 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.521 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.521 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.521 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.521 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.521 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-03T18:47:23.521 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.521 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.521 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.521 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.521 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:23.521 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:23.521 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.521 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.521 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.521 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:23.522 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.522 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.522 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.522 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-03T18:47:23.522 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.522 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.522 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.522 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.522 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:23.522 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:23.522 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.522 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.522 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.522 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:23.523 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.523 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.523 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:23.523 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:23.523 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:23.523 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:23.523 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:23.523 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:23.523 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:23.523 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:23.523 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.523 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.523 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.523 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.523 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.523 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:23.524 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:23.524 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.524 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.524 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.524 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.524 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.524 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.524 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.524 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.524 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.524 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.524 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.524 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.524 INFO:teuthology.orchestra.run.smithi033.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-03T18:47:23.524 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-03T18:47:23.524 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.525 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.525 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.525 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.525 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.525 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.525 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.525 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-03T18:47:23.525 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.525 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.525 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.525 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.525 INFO:teuthology.orchestra.run.smithi033.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-03T18:47:23.525 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-03T18:47:23.525 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.525 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.525 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.526 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "*", 2024-05-03T18:47:23.526 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.526 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.526 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.526 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-03T18:47:23.526 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.526 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.526 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.526 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.526 INFO:teuthology.orchestra.run.smithi033.stdout: "max_count_per_host": { 2024-05-03T18:47:23.526 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_count_per_host", 2024-05-03T18:47:23.526 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:23.526 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.526 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.526 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-05-03T18:47:23.527 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.527 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.527 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.527 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "max number of daemons per service per host", 2024-05-03T18:47:23.527 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.527 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.527 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.527 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.527 INFO:teuthology.orchestra.run.smithi033.stdout: "max_osd_draining_count": { 2024-05-03T18:47:23.527 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_osd_draining_count", 2024-05-03T18:47:23.527 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:23.527 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.527 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.527 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-05-03T18:47:23.527 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.528 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.528 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.528 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-03T18:47:23.528 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.528 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.528 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.528 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.528 INFO:teuthology.orchestra.run.smithi033.stdout: "migration_current": { 2024-05-03T18:47:23.528 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "migration_current", 2024-05-03T18:47:23.528 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:23.528 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.528 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.528 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:23.528 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.528 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.528 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.528 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "internal - do not modify", 2024-05-03T18:47:23.529 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.529 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.529 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.529 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.529 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": { 2024-05-03T18:47:23.529 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mode", 2024-05-03T18:47:23.529 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.529 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.529 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.529 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "root", 2024-05-03T18:47:23.529 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.529 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.529 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:23.529 INFO:teuthology.orchestra.run.smithi033.stdout: "cephadm-package", 2024-05-03T18:47:23.529 INFO:teuthology.orchestra.run.smithi033.stdout: "root" 2024-05-03T18:47:23.529 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:23.529 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "mode for remote execution of cephadm", 2024-05-03T18:47:23.530 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.530 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.530 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.530 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.530 INFO:teuthology.orchestra.run.smithi033.stdout: "oob_default_addr": { 2024-05-03T18:47:23.530 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "oob_default_addr", 2024-05-03T18:47:23.530 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.530 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.530 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.530 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "169.254.1.1", 2024-05-03T18:47:23.530 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.530 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.530 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.531 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-03T18:47:23.531 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.531 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.531 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.531 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.531 INFO:teuthology.orchestra.run.smithi033.stdout: "prometheus_alerts_path": { 2024-05-03T18:47:23.531 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "prometheus_alerts_path", 2024-05-03T18:47:23.531 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.531 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.531 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.531 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-03T18:47:23.531 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.531 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.531 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.531 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-03T18:47:23.531 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.532 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.532 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.532 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.532 INFO:teuthology.orchestra.run.smithi033.stdout: "registry_insecure": { 2024-05-03T18:47:23.532 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "registry_insecure", 2024-05-03T18:47:23.532 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.532 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.532 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.532 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.532 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.532 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.532 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.532 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-03T18:47:23.532 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.532 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.532 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.533 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.533 INFO:teuthology.orchestra.run.smithi033.stdout: "registry_password": { 2024-05-03T18:47:23.533 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "registry_password", 2024-05-03T18:47:23.533 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.533 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.533 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.533 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:23.533 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.533 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.533 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.533 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-03T18:47:23.533 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.533 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.533 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.534 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.534 INFO:teuthology.orchestra.run.smithi033.stdout: "registry_url": { 2024-05-03T18:47:23.534 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "registry_url", 2024-05-03T18:47:23.534 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.534 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.534 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.534 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:23.534 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.534 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.534 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.534 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-03T18:47:23.534 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.534 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.534 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.535 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.535 INFO:teuthology.orchestra.run.smithi033.stdout: "registry_username": { 2024-05-03T18:47:23.535 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "registry_username", 2024-05-03T18:47:23.535 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.535 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.535 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.535 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:23.535 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.535 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.535 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.535 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-03T18:47:23.535 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.535 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.535 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.536 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.536 INFO:teuthology.orchestra.run.smithi033.stdout: "secure_monitoring_stack": { 2024-05-03T18:47:23.536 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "secure_monitoring_stack", 2024-05-03T18:47:23.536 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.536 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.536 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.536 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.536 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.536 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.536 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.536 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-03T18:47:23.536 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.536 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.536 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.536 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.536 INFO:teuthology.orchestra.run.smithi033.stdout: "service_discovery_port": { 2024-05-03T18:47:23.536 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "service_discovery_port", 2024-05-03T18:47:23.537 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:23.537 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.537 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.537 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8765", 2024-05-03T18:47:23.537 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.537 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.537 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.537 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "cephadm service discovery port", 2024-05-03T18:47:23.537 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.537 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.537 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.537 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.537 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:23.537 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:23.537 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:23.537 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:23.538 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.538 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:23.538 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.538 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.538 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.538 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.538 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.538 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.538 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.538 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.538 INFO:teuthology.orchestra.run.smithi033.stdout: "ssh_config_file": { 2024-05-03T18:47:23.538 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ssh_config_file", 2024-05-03T18:47:23.538 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.538 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.538 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.538 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:23.538 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.539 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.539 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.539 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-03T18:47:23.539 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.539 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.539 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.539 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.539 INFO:teuthology.orchestra.run.smithi033.stdout: "use_agent": { 2024-05-03T18:47:23.539 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "use_agent", 2024-05-03T18:47:23.539 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.539 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.539 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.539 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.539 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.540 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.540 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.540 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-03T18:47:23.540 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.540 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.540 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.540 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.540 INFO:teuthology.orchestra.run.smithi033.stdout: "use_repo_digest": { 2024-05-03T18:47:23.540 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "use_repo_digest", 2024-05-03T18:47:23.540 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.540 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.540 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.540 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:23.540 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.540 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.541 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.541 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-03T18:47:23.541 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.541 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.541 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.541 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.541 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_on_failed_host_check": { 2024-05-03T18:47:23.541 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_on_failed_host_check", 2024-05-03T18:47:23.541 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.541 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.541 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.541 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:23.541 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.541 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.541 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.542 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "raise a health warning if the host check fails", 2024-05-03T18:47:23.542 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.542 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.542 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.542 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.542 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_on_stray_daemons": { 2024-05-03T18:47:23.542 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_on_stray_daemons", 2024-05-03T18:47:23.542 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.542 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.542 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.542 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:23.542 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.542 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.542 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.542 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-03T18:47:23.542 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.543 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.543 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.543 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.543 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_on_stray_hosts": { 2024-05-03T18:47:23.543 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_on_stray_hosts", 2024-05-03T18:47:23.543 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.543 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.543 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.543 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:23.543 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.543 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.543 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.543 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-03T18:47:23.543 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.543 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.543 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.544 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:23.544 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:23.544 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.544 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:23.544 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crash", 2024-05-03T18:47:23.544 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:23.544 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:23.544 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:23.544 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:23.544 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:23.544 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.544 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.544 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.544 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:23.544 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.545 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.545 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:23.545 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:23.545 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:23.545 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:23.545 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:23.545 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:23.545 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:23.545 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:23.545 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.545 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.545 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.545 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.545 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.545 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:23.545 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:23.546 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.546 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.546 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.546 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.546 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.546 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.546 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.546 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.546 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.546 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.546 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.546 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.546 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:23.546 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:23.547 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.547 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.547 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.547 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:23.547 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.547 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.547 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:23.547 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:23.547 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:23.547 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:23.547 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:23.547 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:23.547 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:23.547 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:23.547 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.547 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.548 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.548 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.548 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.548 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:23.548 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:23.548 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.548 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.548 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.548 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.548 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.548 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.548 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.548 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.549 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.549 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.549 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.549 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.549 INFO:teuthology.orchestra.run.smithi033.stdout: "retain_interval": { 2024-05-03T18:47:23.549 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "retain_interval", 2024-05-03T18:47:23.549 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-03T18:47:23.549 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.549 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.549 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "31536000", 2024-05-03T18:47:23.549 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.549 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.549 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.549 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-03T18:47:23.549 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.550 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.550 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.550 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.550 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:23.550 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:23.550 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:23.550 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:23.550 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.550 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:23.550 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.550 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.550 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.550 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.550 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.550 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.551 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.551 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.551 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_recent_interval": { 2024-05-03T18:47:23.551 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_recent_interval", 2024-05-03T18:47:23.551 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-03T18:47:23.551 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.551 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.551 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "1209600", 2024-05-03T18:47:23.551 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.551 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.551 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.551 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-03T18:47:23.551 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.551 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.551 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.552 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:23.552 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:23.552 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.552 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:23.552 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "dashboard", 2024-05-03T18:47:23.552 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:23.552 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:23.552 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:23.552 INFO:teuthology.orchestra.run.smithi033.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-03T18:47:23.552 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-03T18:47:23.552 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:23.552 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.552 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.552 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-05-03T18:47:23.552 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.552 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.552 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.553 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.553 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.553 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.553 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.553 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.553 INFO:teuthology.orchestra.run.smithi033.stdout: "ALERTMANAGER_API_HOST": { 2024-05-03T18:47:23.553 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-03T18:47:23.553 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.553 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.553 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.553 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:23.553 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.553 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.553 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.553 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.553 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.554 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.554 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.554 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.554 INFO:teuthology.orchestra.run.smithi033.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-03T18:47:23.554 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-03T18:47:23.554 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.554 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.554 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.554 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:23.554 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.554 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.554 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.554 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.554 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.555 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.555 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.555 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.555 INFO:teuthology.orchestra.run.smithi033.stdout: "AUDIT_API_ENABLED": { 2024-05-03T18:47:23.555 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "AUDIT_API_ENABLED", 2024-05-03T18:47:23.555 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.555 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.555 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.555 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.555 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.555 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.555 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.555 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.555 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.555 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.555 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.556 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.556 INFO:teuthology.orchestra.run.smithi033.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-03T18:47:23.556 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-03T18:47:23.556 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.556 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.556 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.556 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:23.556 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.556 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.556 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.556 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.556 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.556 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.556 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.556 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.556 INFO:teuthology.orchestra.run.smithi033.stdout: "ENABLE_BROWSABLE_API": { 2024-05-03T18:47:23.557 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-03T18:47:23.557 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.557 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.557 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.557 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:23.557 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.557 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.557 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.557 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.557 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.557 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.557 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.557 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.557 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-03T18:47:23.558 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-03T18:47:23.558 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.558 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.558 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.558 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:23.558 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.558 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.558 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.558 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.558 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.558 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.558 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.558 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.558 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-03T18:47:23.558 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-03T18:47:23.559 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.559 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.559 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.559 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:23.559 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.559 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.559 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.559 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.559 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.559 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.559 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.559 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.559 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-03T18:47:23.559 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-03T18:47:23.559 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.559 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.560 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.560 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:23.560 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.560 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.560 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.560 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.560 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.560 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.560 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.560 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.560 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-03T18:47:23.560 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-03T18:47:23.560 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.560 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.560 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.560 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:23.561 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.561 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.561 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.561 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.561 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.561 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.561 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.561 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.561 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-03T18:47:23.561 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-03T18:47:23.561 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.561 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.561 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.561 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:23.561 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.561 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.562 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.562 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.562 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.562 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.562 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.562 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.562 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-03T18:47:23.562 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-03T18:47:23.562 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.562 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.562 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.562 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:23.562 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.562 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.562 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.563 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.563 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.563 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.563 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.563 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.563 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-03T18:47:23.563 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-03T18:47:23.563 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.563 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.563 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.563 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:23.563 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.563 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.563 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.563 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.563 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.564 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.564 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.564 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.564 INFO:teuthology.orchestra.run.smithi033.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-03T18:47:23.564 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-03T18:47:23.564 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.564 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.564 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.564 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:23.564 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.564 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.564 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.564 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.564 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.565 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.565 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.565 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.565 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_API_PASSWORD": { 2024-05-03T18:47:23.565 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-03T18:47:23.565 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.565 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.565 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.565 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "admin", 2024-05-03T18:47:23.566 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.566 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.566 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.566 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.566 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.566 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.566 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.566 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.566 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-03T18:47:23.566 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-03T18:47:23.566 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.567 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.567 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.567 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:23.567 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.567 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.567 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.567 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.567 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.567 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.567 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.567 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.567 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_API_URL": { 2024-05-03T18:47:23.567 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_API_URL", 2024-05-03T18:47:23.568 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.568 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.568 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.568 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:23.568 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.568 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.568 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.568 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.568 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.568 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.568 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.569 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.569 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_API_USERNAME": { 2024-05-03T18:47:23.569 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-03T18:47:23.569 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.569 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.569 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.569 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "admin", 2024-05-03T18:47:23.570 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.570 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.570 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.570 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.570 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.570 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.570 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.570 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.570 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-03T18:47:23.571 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-03T18:47:23.571 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.571 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.571 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.571 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:23.571 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.571 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.571 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.571 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.571 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.572 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.572 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.572 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.572 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-03T18:47:23.572 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-03T18:47:23.573 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.573 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.573 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.573 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.573 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.573 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.574 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.574 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.574 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.574 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.574 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.575 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.575 INFO:teuthology.orchestra.run.smithi033.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-03T18:47:23.575 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-03T18:47:23.575 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.575 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.575 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.575 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:23.575 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.575 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.575 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.575 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.575 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.576 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.576 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.576 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.576 INFO:teuthology.orchestra.run.smithi033.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-03T18:47:23.576 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-03T18:47:23.576 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.577 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.577 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.577 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:23.577 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.577 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.577 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.577 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.577 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.577 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.577 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.577 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.577 INFO:teuthology.orchestra.run.smithi033.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-03T18:47:23.577 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-03T18:47:23.577 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.577 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.578 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.578 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "[]", 2024-05-03T18:47:23.578 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.578 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.578 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.578 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.578 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.578 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.578 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.578 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.578 INFO:teuthology.orchestra.run.smithi033.stdout: "MULTICLUSTER_CONFIG": { 2024-05-03T18:47:23.578 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-03T18:47:23.578 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.579 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.579 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.579 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "{}", 2024-05-03T18:47:23.579 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.579 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.579 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.579 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.579 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.579 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.579 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.579 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.579 INFO:teuthology.orchestra.run.smithi033.stdout: "PROMETHEUS_API_HOST": { 2024-05-03T18:47:23.579 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-03T18:47:23.579 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.579 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.580 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.580 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:23.580 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.580 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.580 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.580 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.580 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.580 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.580 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.580 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.580 INFO:teuthology.orchestra.run.smithi033.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-03T18:47:23.580 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-03T18:47:23.580 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.580 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.580 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.581 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:23.581 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.581 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.581 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.581 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.581 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.581 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.581 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.581 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.581 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-03T18:47:23.581 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-03T18:47:23.581 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.581 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.581 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.581 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.581 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.581 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.582 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.582 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.582 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.582 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.582 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.582 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.582 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-03T18:47:23.582 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-03T18:47:23.582 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.582 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.582 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.582 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.582 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.583 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.583 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.583 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.583 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.583 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.583 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.583 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.583 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-03T18:47:23.583 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-03T18:47:23.583 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.583 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.583 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.583 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:23.583 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.583 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.584 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.584 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.584 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.584 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.584 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.584 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.584 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-03T18:47:23.584 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-03T18:47:23.584 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.584 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.584 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.584 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:23.584 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.584 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.584 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.584 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.585 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.585 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.585 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.585 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.585 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-03T18:47:23.585 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-03T18:47:23.585 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.585 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.585 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.585 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.585 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.585 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.585 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.585 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.585 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.585 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.586 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.586 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.586 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-03T18:47:23.586 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-03T18:47:23.586 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.586 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.586 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.586 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.586 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.586 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.586 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.586 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.586 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.586 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.586 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.587 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.587 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-03T18:47:23.587 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-03T18:47:23.587 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.587 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.587 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.587 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.587 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.587 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.587 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.587 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.587 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.587 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.587 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.587 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.587 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_ENABLED": { 2024-05-03T18:47:23.588 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-03T18:47:23.588 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.588 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.588 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.588 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:23.588 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.588 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.588 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.588 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.588 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.588 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.588 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.588 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.589 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-03T18:47:23.589 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-03T18:47:23.589 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.589 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.589 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.589 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-03T18:47:23.589 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.589 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.589 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.589 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.589 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.589 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.589 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.589 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.589 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-03T18:47:23.589 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-03T18:47:23.589 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:23.590 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.590 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.590 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-05-03T18:47:23.590 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.590 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.590 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.590 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.590 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.590 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.590 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.590 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.590 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-03T18:47:23.590 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-03T18:47:23.590 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:23.591 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.591 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.591 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8", 2024-05-03T18:47:23.591 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.591 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.591 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.591 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.591 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.591 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.591 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.591 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.591 INFO:teuthology.orchestra.run.smithi033.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-03T18:47:23.591 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-03T18:47:23.591 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:23.591 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.591 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.592 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "45", 2024-05-03T18:47:23.592 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.592 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.592 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.592 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.592 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.592 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.592 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.592 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.592 INFO:teuthology.orchestra.run.smithi033.stdout: "RGW_API_ACCESS_KEY": { 2024-05-03T18:47:23.592 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-03T18:47:23.592 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.592 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.592 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.592 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:23.592 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.592 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.593 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.593 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.593 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.593 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.593 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.593 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.593 INFO:teuthology.orchestra.run.smithi033.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-03T18:47:23.593 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-03T18:47:23.593 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.593 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.593 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.593 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "admin", 2024-05-03T18:47:23.593 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.593 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.593 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.593 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.593 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.594 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.594 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.594 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.594 INFO:teuthology.orchestra.run.smithi033.stdout: "RGW_API_SECRET_KEY": { 2024-05-03T18:47:23.594 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-03T18:47:23.594 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.594 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.594 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.594 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:23.594 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.594 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.594 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.594 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.594 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.594 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.594 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.595 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.595 INFO:teuthology.orchestra.run.smithi033.stdout: "RGW_API_SSL_VERIFY": { 2024-05-03T18:47:23.595 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-03T18:47:23.595 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.595 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.595 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.595 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:23.595 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.595 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.595 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.595 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.595 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.595 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.595 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.596 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.596 INFO:teuthology.orchestra.run.smithi033.stdout: "UNSAFE_TLS_v1_2": { 2024-05-03T18:47:23.596 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-03T18:47:23.596 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.596 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.596 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.596 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.596 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.596 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.596 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.596 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.596 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.596 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.596 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.596 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.596 INFO:teuthology.orchestra.run.smithi033.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-03T18:47:23.596 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-03T18:47:23.597 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:23.597 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.597 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.597 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:23.597 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.597 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.597 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.597 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.597 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.597 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.597 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.597 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.597 INFO:teuthology.orchestra.run.smithi033.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-03T18:47:23.597 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-03T18:47:23.597 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:23.597 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.598 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.598 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-05-03T18:47:23.598 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.598 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.598 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.598 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.598 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.598 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.598 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.598 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.598 INFO:teuthology.orchestra.run.smithi033.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-03T18:47:23.598 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-03T18:47:23.598 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:23.598 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.598 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.599 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5", 2024-05-03T18:47:23.599 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.599 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.599 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.599 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.599 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.599 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.599 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.599 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.599 INFO:teuthology.orchestra.run.smithi033.stdout: "cross_origin_url": { 2024-05-03T18:47:23.599 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cross_origin_url", 2024-05-03T18:47:23.599 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.599 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.599 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.599 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:23.600 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.600 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.600 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.600 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.600 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.600 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.600 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.600 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.600 INFO:teuthology.orchestra.run.smithi033.stdout: "crt_file": { 2024-05-03T18:47:23.600 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crt_file", 2024-05-03T18:47:23.600 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.600 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.600 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.600 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:23.600 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.600 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.601 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.601 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.601 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.601 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.601 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.601 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.601 INFO:teuthology.orchestra.run.smithi033.stdout: "debug": { 2024-05-03T18:47:23.601 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "debug", 2024-05-03T18:47:23.601 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.601 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.601 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.601 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.601 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.601 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.601 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.601 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Enable/disable debug options", 2024-05-03T18:47:23.602 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.602 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.602 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.602 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.602 INFO:teuthology.orchestra.run.smithi033.stdout: "jwt_token_ttl": { 2024-05-03T18:47:23.602 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "jwt_token_ttl", 2024-05-03T18:47:23.602 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:23.602 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.602 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.602 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "28800", 2024-05-03T18:47:23.602 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.602 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.602 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.603 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.603 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.603 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.603 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.603 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.603 INFO:teuthology.orchestra.run.smithi033.stdout: "key_file": { 2024-05-03T18:47:23.603 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "key_file", 2024-05-03T18:47:23.603 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.603 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.603 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.603 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:23.603 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.603 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.603 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.604 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.604 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.604 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.604 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.604 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.604 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:23.604 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:23.604 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.604 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.604 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.604 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:23.604 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.604 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.604 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:23.604 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:23.604 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:23.605 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:23.605 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:23.605 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:23.605 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:23.605 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:23.605 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.605 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.605 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.605 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.605 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.605 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:23.605 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:23.605 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.605 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.605 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.605 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.606 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.606 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.606 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.606 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.606 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.606 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.606 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.606 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.606 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:23.606 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:23.606 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.606 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.606 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.606 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:23.607 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.607 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.607 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:23.607 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:23.607 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:23.607 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:23.607 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:23.607 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:23.607 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:23.607 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:23.607 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.607 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.607 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.608 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.608 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.608 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:23.608 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:23.608 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.608 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.608 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.608 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.608 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.608 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.608 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.608 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.608 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.608 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.608 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.609 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.609 INFO:teuthology.orchestra.run.smithi033.stdout: "motd": { 2024-05-03T18:47:23.609 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "motd", 2024-05-03T18:47:23.609 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.609 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.609 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.609 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:23.609 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.609 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.609 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.609 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "The message of the day", 2024-05-03T18:47:23.609 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.609 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.609 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.609 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.610 INFO:teuthology.orchestra.run.smithi033.stdout: "redirect_resolve_ip_addr": { 2024-05-03T18:47:23.610 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "redirect_resolve_ip_addr", 2024-05-03T18:47:23.610 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.610 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.610 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.610 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.610 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.610 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.610 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.610 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.610 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.610 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.610 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.611 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.611 INFO:teuthology.orchestra.run.smithi033.stdout: "server_addr": { 2024-05-03T18:47:23.611 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_addr", 2024-05-03T18:47:23.611 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.611 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.611 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.611 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "::", 2024-05-03T18:47:23.611 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.611 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.611 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.611 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.611 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.611 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.611 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.611 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.611 INFO:teuthology.orchestra.run.smithi033.stdout: "server_port": { 2024-05-03T18:47:23.612 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_port", 2024-05-03T18:47:23.612 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:23.612 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.612 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.612 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8080", 2024-05-03T18:47:23.612 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.612 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.612 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.612 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.612 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.612 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.612 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.613 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.613 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:23.613 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:23.613 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:23.613 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:23.613 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.613 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:23.613 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.614 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.614 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.614 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.614 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.614 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.614 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.614 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.614 INFO:teuthology.orchestra.run.smithi033.stdout: "ssl": { 2024-05-03T18:47:23.614 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ssl", 2024-05-03T18:47:23.614 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.614 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.614 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.614 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:23.614 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.615 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.615 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.615 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.615 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.615 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.615 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.615 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.615 INFO:teuthology.orchestra.run.smithi033.stdout: "ssl_server_port": { 2024-05-03T18:47:23.615 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ssl_server_port", 2024-05-03T18:47:23.615 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:23.615 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.615 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.615 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8443", 2024-05-03T18:47:23.616 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.616 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.616 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.616 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.616 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.616 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.616 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.616 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.616 INFO:teuthology.orchestra.run.smithi033.stdout: "standby_behaviour": { 2024-05-03T18:47:23.616 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "standby_behaviour", 2024-05-03T18:47:23.616 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.616 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.616 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.616 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "redirect", 2024-05-03T18:47:23.617 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.617 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.617 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:23.617 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:23.617 INFO:teuthology.orchestra.run.smithi033.stdout: "redirect" 2024-05-03T18:47:23.617 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:23.617 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.617 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.617 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.617 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.617 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.617 INFO:teuthology.orchestra.run.smithi033.stdout: "standby_error_status_code": { 2024-05-03T18:47:23.617 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "standby_error_status_code", 2024-05-03T18:47:23.618 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:23.618 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.618 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.618 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "500", 2024-05-03T18:47:23.618 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "400", 2024-05-03T18:47:23.618 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "599", 2024-05-03T18:47:23.618 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.618 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.618 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.618 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.618 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.618 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.618 INFO:teuthology.orchestra.run.smithi033.stdout: "url_prefix": { 2024-05-03T18:47:23.618 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "url_prefix", 2024-05-03T18:47:23.619 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.619 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.619 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.619 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:23.619 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.619 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.619 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.619 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.619 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.619 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.619 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.619 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:23.619 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:23.619 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.619 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:23.619 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "devicehealth", 2024-05-03T18:47:23.619 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:23.620 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:23.620 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:23.620 INFO:teuthology.orchestra.run.smithi033.stdout: "enable_monitoring": { 2024-05-03T18:47:23.620 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "enable_monitoring", 2024-05-03T18:47:23.620 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.620 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.620 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.620 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:23.620 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.620 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.620 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.620 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "monitor device health metrics", 2024-05-03T18:47:23.620 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.620 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.620 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.620 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.620 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:23.621 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:23.621 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.621 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.621 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.621 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:23.621 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.621 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.621 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:23.621 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:23.621 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:23.621 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:23.621 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:23.621 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:23.621 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:23.621 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:23.621 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.622 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.622 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.622 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.622 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.622 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:23.622 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:23.622 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.622 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.622 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.622 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.622 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.622 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.622 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.622 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.622 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.622 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.623 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.623 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.623 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:23.623 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:23.623 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.623 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.623 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.623 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:23.623 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.623 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.623 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:23.623 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:23.623 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:23.623 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:23.623 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:23.623 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:23.624 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:23.624 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:23.624 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.624 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.624 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.624 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.624 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.624 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:23.624 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:23.624 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.624 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.624 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.624 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.624 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.624 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.624 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.624 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.625 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.625 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.625 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.625 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.625 INFO:teuthology.orchestra.run.smithi033.stdout: "mark_out_threshold": { 2024-05-03T18:47:23.625 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mark_out_threshold", 2024-05-03T18:47:23.625 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-03T18:47:23.625 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.625 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.625 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "2419200", 2024-05-03T18:47:23.625 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.625 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.625 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.625 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-03T18:47:23.625 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.626 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.626 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.626 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.626 INFO:teuthology.orchestra.run.smithi033.stdout: "pool_name": { 2024-05-03T18:47:23.626 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "pool_name", 2024-05-03T18:47:23.626 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.626 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.626 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.626 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "device_health_metrics", 2024-05-03T18:47:23.626 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.626 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.626 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.626 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-03T18:47:23.626 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.626 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.626 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.626 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.627 INFO:teuthology.orchestra.run.smithi033.stdout: "retention_period": { 2024-05-03T18:47:23.627 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "retention_period", 2024-05-03T18:47:23.627 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-03T18:47:23.627 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.627 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.627 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "15552000", 2024-05-03T18:47:23.627 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.627 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.627 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.627 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how long to retain device health metrics", 2024-05-03T18:47:23.627 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.627 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.627 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.627 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.627 INFO:teuthology.orchestra.run.smithi033.stdout: "scrape_frequency": { 2024-05-03T18:47:23.627 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "scrape_frequency", 2024-05-03T18:47:23.628 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-03T18:47:23.628 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.628 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.628 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "86400", 2024-05-03T18:47:23.628 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.628 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.628 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.628 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-03T18:47:23.628 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.628 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.628 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.628 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.628 INFO:teuthology.orchestra.run.smithi033.stdout: "self_heal": { 2024-05-03T18:47:23.628 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "self_heal", 2024-05-03T18:47:23.628 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.628 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.628 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.629 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:23.629 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.629 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.629 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.629 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-03T18:47:23.629 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.629 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.629 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.629 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.629 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-05-03T18:47:23.629 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-05-03T18:47:23.629 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-03T18:47:23.629 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.629 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.629 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-05-03T18:47:23.629 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.629 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.630 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.630 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to wake up and check device health", 2024-05-03T18:47:23.630 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.630 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.630 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.630 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.630 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:23.630 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:23.630 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:23.630 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:23.630 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.630 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:23.630 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.630 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.630 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.630 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.631 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.631 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.631 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.631 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.631 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_threshold": { 2024-05-03T18:47:23.631 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_threshold", 2024-05-03T18:47:23.631 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-03T18:47:23.631 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.631 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.631 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "7257600", 2024-05-03T18:47:23.631 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.631 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.631 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.631 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-03T18:47:23.631 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.631 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.631 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.632 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:23.632 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:23.632 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.632 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:23.632 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "diskprediction_local", 2024-05-03T18:47:23.632 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:23.632 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:23.632 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:23.632 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:23.632 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:23.632 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.632 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.632 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.632 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:23.632 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.632 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.633 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:23.633 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:23.633 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:23.633 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:23.633 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:23.633 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:23.633 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:23.633 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:23.633 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.633 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.633 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.633 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.633 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.633 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:23.633 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:23.633 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.634 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.634 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.634 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.634 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.634 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.634 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.634 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.634 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.634 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.634 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.634 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.634 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:23.634 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:23.634 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.635 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.635 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.635 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:23.635 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.635 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.635 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:23.635 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:23.635 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:23.635 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:23.635 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:23.635 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:23.635 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:23.635 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:23.635 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.635 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.635 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.636 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.636 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.636 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:23.636 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:23.636 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.636 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.636 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.636 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.636 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.636 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.636 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.636 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.636 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.636 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.636 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.636 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.637 INFO:teuthology.orchestra.run.smithi033.stdout: "predict_interval": { 2024-05-03T18:47:23.637 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "predict_interval", 2024-05-03T18:47:23.637 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.637 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.637 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.637 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "86400", 2024-05-03T18:47:23.637 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.637 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.637 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.637 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.637 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.637 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.637 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.637 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.637 INFO:teuthology.orchestra.run.smithi033.stdout: "predictor_model": { 2024-05-03T18:47:23.638 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "predictor_model", 2024-05-03T18:47:23.638 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.638 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.638 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.638 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "prophetstor", 2024-05-03T18:47:23.638 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.638 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.638 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.638 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.638 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.638 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.638 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.638 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.638 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-05-03T18:47:23.638 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-05-03T18:47:23.639 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.639 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.639 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.639 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-05-03T18:47:23.639 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.639 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.639 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.639 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.639 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.639 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.639 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.639 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.639 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:23.639 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:23.639 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:23.639 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:23.640 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.640 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:23.640 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.640 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.640 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.640 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.640 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.640 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.640 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.640 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:23.640 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:23.640 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.640 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:23.641 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "influx", 2024-05-03T18:47:23.641 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": false, 2024-05-03T18:47:23.641 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "influxdb python module not found", 2024-05-03T18:47:23.641 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:23.641 INFO:teuthology.orchestra.run.smithi033.stdout: "batch_size": { 2024-05-03T18:47:23.641 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "batch_size", 2024-05-03T18:47:23.641 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:23.641 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.641 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.641 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5000", 2024-05-03T18:47:23.641 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.641 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.641 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.641 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-03T18:47:23.642 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.642 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.642 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.642 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.642 INFO:teuthology.orchestra.run.smithi033.stdout: "database": { 2024-05-03T18:47:23.642 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "database", 2024-05-03T18:47:23.642 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.642 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.642 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.642 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "ceph", 2024-05-03T18:47:23.642 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.642 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.642 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.643 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-03T18:47:23.643 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.643 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.643 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.643 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.643 INFO:teuthology.orchestra.run.smithi033.stdout: "hostname": { 2024-05-03T18:47:23.643 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "hostname", 2024-05-03T18:47:23.643 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.643 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.643 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.643 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:23.643 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.643 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.643 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.643 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "InfluxDB server hostname", 2024-05-03T18:47:23.644 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.644 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.644 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.644 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.644 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-05-03T18:47:23.644 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-05-03T18:47:23.644 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-03T18:47:23.644 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.644 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.644 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "30", 2024-05-03T18:47:23.644 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "5", 2024-05-03T18:47:23.644 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.644 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.644 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-03T18:47:23.645 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.645 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.645 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.645 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.645 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:23.645 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:23.645 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.645 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.645 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.645 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:23.645 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.645 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.645 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:23.646 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:23.646 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:23.646 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:23.646 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:23.646 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:23.646 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:23.646 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:23.646 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.646 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.646 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.646 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.646 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.646 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:23.647 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:23.647 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.647 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.647 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.647 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.647 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.647 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.647 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.647 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.647 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.647 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.647 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.647 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.647 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:23.647 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:23.648 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.648 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.648 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.648 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:23.648 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.648 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.648 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:23.648 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:23.648 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:23.648 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:23.648 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:23.648 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:23.648 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:23.648 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:23.648 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.649 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.649 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.649 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.649 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.649 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:23.649 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:23.649 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.649 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.649 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.649 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.649 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.649 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.649 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.649 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.649 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.649 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.650 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.650 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.650 INFO:teuthology.orchestra.run.smithi033.stdout: "password": { 2024-05-03T18:47:23.650 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "password", 2024-05-03T18:47:23.650 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.650 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.650 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.650 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:23.650 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.650 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.650 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.650 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "password of InfluxDB server user", 2024-05-03T18:47:23.650 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.651 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.651 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.651 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.651 INFO:teuthology.orchestra.run.smithi033.stdout: "port": { 2024-05-03T18:47:23.651 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "port", 2024-05-03T18:47:23.651 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:23.651 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.651 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.651 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8086", 2024-05-03T18:47:23.651 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.651 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.651 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.651 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "InfluxDB server port", 2024-05-03T18:47:23.651 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.651 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.652 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.652 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.652 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:23.652 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:23.652 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:23.652 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:23.652 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.652 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:23.652 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.652 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.652 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.652 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.652 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.652 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.653 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.653 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.653 INFO:teuthology.orchestra.run.smithi033.stdout: "ssl": { 2024-05-03T18:47:23.653 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ssl", 2024-05-03T18:47:23.653 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.653 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.653 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.653 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "false", 2024-05-03T18:47:23.653 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.653 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.653 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.653 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-03T18:47:23.653 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.653 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.653 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.654 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.654 INFO:teuthology.orchestra.run.smithi033.stdout: "threads": { 2024-05-03T18:47:23.654 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "threads", 2024-05-03T18:47:23.654 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:23.654 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.654 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.654 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5", 2024-05-03T18:47:23.654 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1", 2024-05-03T18:47:23.654 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "32", 2024-05-03T18:47:23.654 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.654 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-03T18:47:23.654 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.654 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.654 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.655 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.655 INFO:teuthology.orchestra.run.smithi033.stdout: "username": { 2024-05-03T18:47:23.655 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "username", 2024-05-03T18:47:23.655 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.655 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.655 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.655 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:23.655 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.655 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.656 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.656 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "username of InfluxDB server user", 2024-05-03T18:47:23.656 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.656 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.656 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.656 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.656 INFO:teuthology.orchestra.run.smithi033.stdout: "verify_ssl": { 2024-05-03T18:47:23.656 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "verify_ssl", 2024-05-03T18:47:23.656 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.656 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.656 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.656 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "true", 2024-05-03T18:47:23.656 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.657 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.657 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.657 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-03T18:47:23.657 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.657 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.657 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.657 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:23.657 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:23.657 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.657 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:23.657 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "insights", 2024-05-03T18:47:23.657 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:23.657 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:23.657 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:23.658 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:23.658 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:23.658 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.658 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.658 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.658 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:23.658 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.658 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.658 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:23.658 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:23.658 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:23.658 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:23.658 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:23.659 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:23.659 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:23.659 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:23.659 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.659 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.659 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.659 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.659 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.659 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:23.659 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:23.659 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.659 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.659 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.660 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.660 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.660 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.660 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.660 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.660 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.660 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.660 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.660 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.660 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:23.660 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:23.660 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.660 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.660 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.661 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:23.661 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.661 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.661 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:23.661 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:23.661 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:23.662 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:23.662 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:23.662 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:23.662 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:23.662 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:23.662 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.662 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.662 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.662 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.662 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.662 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:23.662 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:23.662 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.663 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.663 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.663 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.663 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.663 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.663 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.663 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.663 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.663 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.663 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.663 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.664 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:23.664 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:23.664 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:23.664 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:23.664 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.665 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:23.665 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.665 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.665 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.665 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.665 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.665 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.666 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.666 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:23.666 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:23.666 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.666 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:23.666 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "iostat", 2024-05-03T18:47:23.666 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:23.667 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:23.667 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:23.667 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:23.667 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:23.667 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.667 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.667 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.667 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:23.667 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.667 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.667 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:23.668 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:23.668 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:23.668 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:23.668 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:23.668 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:23.668 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:23.668 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:23.668 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.669 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.669 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.669 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.669 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.669 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:23.669 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:23.669 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.669 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.669 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.669 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.669 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.669 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.669 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.670 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.670 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.670 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.670 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.670 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.670 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:23.670 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:23.670 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.670 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.670 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.670 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:23.670 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.670 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.671 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:23.671 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:23.671 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:23.671 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:23.671 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:23.671 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:23.671 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:23.671 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:23.671 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.671 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.671 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.671 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.671 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.671 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:23.671 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:23.671 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.672 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.672 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.672 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.672 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.672 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.672 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.672 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.672 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.672 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.672 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.672 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.672 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:23.673 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:23.673 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:23.673 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:23.673 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.673 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:23.673 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.673 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.673 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.673 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.673 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.673 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.673 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.673 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:23.673 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:23.673 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.674 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:23.674 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "k8sevents", 2024-05-03T18:47:23.674 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:23.674 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:23.674 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:23.674 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_event_retention_days": { 2024-05-03T18:47:23.674 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ceph_event_retention_days", 2024-05-03T18:47:23.674 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:23.674 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.674 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.674 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "7", 2024-05-03T18:47:23.674 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.674 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.675 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.675 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-03T18:47:23.675 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.675 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.675 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.675 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.675 INFO:teuthology.orchestra.run.smithi033.stdout: "config_check_secs": { 2024-05-03T18:47:23.675 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "config_check_secs", 2024-05-03T18:47:23.675 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:23.675 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.675 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.675 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-05-03T18:47:23.675 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "10", 2024-05-03T18:47:23.675 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.675 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.675 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-03T18:47:23.676 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.676 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.676 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.676 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.676 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:23.676 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:23.676 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.676 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.676 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.676 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:23.676 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.676 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.676 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:23.677 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:23.677 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:23.677 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:23.677 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:23.677 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:23.677 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:23.677 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:23.677 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.677 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.677 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.677 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.677 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.677 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:23.677 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:23.677 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.678 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.678 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.678 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.678 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.678 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.678 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.678 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.678 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.678 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.678 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.678 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.678 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:23.678 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:23.679 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.679 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.679 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.679 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:23.679 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.679 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.679 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:23.679 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:23.679 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:23.679 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:23.679 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:23.679 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:23.679 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:23.679 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:23.680 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.680 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.680 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.680 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.680 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.680 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:23.680 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:23.680 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.680 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.680 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.680 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.680 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.680 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.680 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.680 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.680 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.680 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.681 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.681 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.681 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:23.681 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:23.681 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:23.681 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:23.681 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.681 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:23.681 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.681 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.681 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.681 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.681 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.681 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.681 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.682 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:23.682 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:23.682 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.682 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:23.682 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "localpool", 2024-05-03T18:47:23.682 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:23.682 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:23.682 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:23.682 INFO:teuthology.orchestra.run.smithi033.stdout: "failure_domain": { 2024-05-03T18:47:23.682 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "failure_domain", 2024-05-03T18:47:23.682 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.682 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.682 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.682 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "host", 2024-05-03T18:47:23.683 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.683 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.683 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.683 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "failure domain for any created local pool", 2024-05-03T18:47:23.683 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-03T18:47:23.683 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.683 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.683 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.683 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:23.683 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:23.683 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.683 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.683 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.683 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:23.683 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.684 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.684 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:23.684 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:23.684 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:23.684 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:23.684 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:23.684 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:23.684 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:23.684 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:23.684 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.684 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.684 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.684 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.684 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.684 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:23.685 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:23.685 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.685 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.685 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.685 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.685 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.685 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.685 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.685 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.685 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.685 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.685 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.685 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.685 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:23.686 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:23.686 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.686 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.686 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.686 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:23.686 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.686 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.686 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:23.686 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:23.686 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:23.686 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:23.686 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:23.687 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:23.687 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:23.687 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:23.687 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.687 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.687 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.687 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.687 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.687 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:23.687 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:23.687 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.687 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.687 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.687 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.688 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.688 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.688 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.688 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.688 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.688 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.688 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.688 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.688 INFO:teuthology.orchestra.run.smithi033.stdout: "min_size": { 2024-05-03T18:47:23.688 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "min_size", 2024-05-03T18:47:23.688 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:23.688 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.688 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.689 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:23.689 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.689 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.689 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.689 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "default min_size for any created local pool", 2024-05-03T18:47:23.689 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-03T18:47:23.689 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.689 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.689 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.689 INFO:teuthology.orchestra.run.smithi033.stdout: "num_rep": { 2024-05-03T18:47:23.689 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "num_rep", 2024-05-03T18:47:23.689 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:23.689 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.690 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.690 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "3", 2024-05-03T18:47:23.690 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.690 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.690 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.690 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "default replica count for any created local pool", 2024-05-03T18:47:23.690 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.690 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.690 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.690 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.690 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_num": { 2024-05-03T18:47:23.690 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "pg_num", 2024-05-03T18:47:23.690 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:23.690 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.691 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.691 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "128", 2024-05-03T18:47:23.691 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.691 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.691 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.691 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "default pg_num for any created local pool", 2024-05-03T18:47:23.691 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.691 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.691 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.691 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.691 INFO:teuthology.orchestra.run.smithi033.stdout: "prefix": { 2024-05-03T18:47:23.691 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "prefix", 2024-05-03T18:47:23.691 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.691 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.691 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.691 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:23.692 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.692 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.692 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.692 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "name prefix for any created local pool", 2024-05-03T18:47:23.692 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.692 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.692 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.692 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.692 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:23.692 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:23.692 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:23.692 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:23.692 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.692 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:23.692 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.692 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.693 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.693 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.693 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.693 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.693 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.693 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.693 INFO:teuthology.orchestra.run.smithi033.stdout: "subtree": { 2024-05-03T18:47:23.693 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "subtree", 2024-05-03T18:47:23.693 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.693 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.693 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.693 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "rack", 2024-05-03T18:47:23.693 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.693 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.693 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.693 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-03T18:47:23.694 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-03T18:47:23.694 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.694 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.694 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:23.694 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:23.694 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.694 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:23.694 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mds_autoscaler", 2024-05-03T18:47:23.694 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:23.694 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:23.694 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:23.694 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:23.694 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:23.694 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.694 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.695 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.695 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:23.695 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.695 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.695 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:23.695 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:23.695 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:23.695 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:23.695 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:23.695 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:23.695 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:23.695 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:23.695 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.695 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.695 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.696 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.696 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.696 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:23.696 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:23.696 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.696 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.696 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.696 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.696 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.696 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.696 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.696 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.696 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.696 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.696 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.696 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.697 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:23.697 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:23.697 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.697 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.697 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.697 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:23.697 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.697 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.697 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:23.697 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:23.697 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:23.697 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:23.697 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:23.697 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:23.698 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:23.698 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:23.698 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.698 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.698 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.698 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.698 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.698 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:23.698 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:23.698 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.698 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.698 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.698 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.698 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.699 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.699 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.699 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.699 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.699 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.699 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.699 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.699 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:23.699 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:23.699 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:23.699 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:23.699 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.699 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:23.699 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.699 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.699 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.700 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.700 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.700 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.700 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.700 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:23.700 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:23.700 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.700 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:23.700 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mirroring", 2024-05-03T18:47:23.700 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:23.700 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:23.700 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:23.700 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:23.700 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:23.701 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.701 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.701 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.701 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:23.701 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.701 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.701 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:23.701 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:23.701 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:23.701 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:23.701 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:23.701 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:23.701 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:23.701 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:23.701 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.701 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.702 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.702 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.702 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.702 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:23.702 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:23.702 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.702 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.702 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.702 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.702 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.702 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.702 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.702 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.703 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.703 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.703 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.703 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.703 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:23.703 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:23.703 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.703 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.703 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.703 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:23.703 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.703 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.703 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:23.703 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:23.703 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:23.704 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:23.704 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:23.704 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:23.704 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:23.704 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:23.704 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.704 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.704 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.704 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.704 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.704 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:23.704 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:23.704 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.704 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.704 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.705 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.705 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.705 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.705 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.705 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.705 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.705 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.705 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.705 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.705 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:23.705 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:23.705 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:23.705 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:23.705 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.705 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:23.706 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.706 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.706 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.706 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.706 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.706 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.706 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.706 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:23.706 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:23.706 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.706 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:23.706 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "nfs", 2024-05-03T18:47:23.706 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:23.706 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:23.706 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:23.706 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:23.707 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:23.707 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.707 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.707 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.707 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:23.707 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.707 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.707 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:23.707 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:23.707 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:23.707 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:23.707 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:23.707 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:23.707 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:23.708 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:23.708 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.708 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.708 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.708 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.708 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.708 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:23.708 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:23.708 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.708 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.708 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.708 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.708 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.708 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.708 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.709 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.709 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.709 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.709 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.709 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.709 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:23.709 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:23.709 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.709 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.709 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.709 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:23.709 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.709 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.709 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:23.709 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:23.709 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:23.710 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:23.710 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:23.710 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:23.710 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:23.710 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:23.710 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.710 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.710 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.710 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.710 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.710 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:23.710 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:23.710 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.711 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.711 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.711 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.711 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.711 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.711 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.711 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.711 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.711 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.711 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.711 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.711 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:23.711 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:23.711 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:23.711 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:23.711 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.711 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:23.712 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.712 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.712 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.712 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.712 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.712 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.712 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.712 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:23.712 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:23.712 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.712 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:23.712 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "orchestrator", 2024-05-03T18:47:23.712 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:23.712 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:23.712 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:23.712 INFO:teuthology.orchestra.run.smithi033.stdout: "fail_fs": { 2024-05-03T18:47:23.713 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "fail_fs", 2024-05-03T18:47:23.713 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.713 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.713 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.713 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.713 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.713 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.713 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.713 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-03T18:47:23.713 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.713 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.713 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.713 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.713 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:23.713 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:23.714 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.714 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.714 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.714 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:23.714 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.714 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.714 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:23.714 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:23.714 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:23.714 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:23.714 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:23.714 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:23.714 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:23.715 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:23.715 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.715 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.715 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.715 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.715 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.715 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:23.715 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:23.715 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.715 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.715 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.715 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.715 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.715 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.716 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.716 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.716 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.716 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.716 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.716 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.716 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:23.716 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:23.716 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.716 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.716 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.716 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:23.716 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.716 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.716 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:23.717 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:23.717 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:23.717 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:23.717 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:23.717 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:23.717 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:23.717 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:23.717 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.717 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.717 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.717 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.717 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.717 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:23.717 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:23.717 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.717 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.718 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.718 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.718 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.718 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.718 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.718 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.718 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.718 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.718 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.718 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.718 INFO:teuthology.orchestra.run.smithi033.stdout: "orchestrator": { 2024-05-03T18:47:23.718 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "orchestrator", 2024-05-03T18:47:23.719 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.719 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.719 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.719 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:23.719 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.719 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.719 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:23.719 INFO:teuthology.orchestra.run.smithi033.stdout: "cephadm", 2024-05-03T18:47:23.719 INFO:teuthology.orchestra.run.smithi033.stdout: "rook", 2024-05-03T18:47:23.719 INFO:teuthology.orchestra.run.smithi033.stdout: "test_orchestrator" 2024-05-03T18:47:23.719 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:23.719 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Orchestrator backend", 2024-05-03T18:47:23.719 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.719 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.719 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.720 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.720 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:23.720 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:23.720 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:23.720 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:23.720 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.720 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:23.720 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.720 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.720 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.720 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.720 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.720 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.720 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.720 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:23.720 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:23.720 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.721 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:23.721 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "osd_perf_query", 2024-05-03T18:47:23.721 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:23.721 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:23.721 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:23.721 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:23.721 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:23.721 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.721 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.721 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.721 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:23.721 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.721 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.721 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:23.721 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:23.721 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:23.722 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:23.722 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:23.722 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:23.722 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:23.722 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:23.722 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.722 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.722 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.722 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.722 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.722 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:23.722 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:23.722 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.722 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.722 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.722 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.723 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.723 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.723 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.723 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.723 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.723 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.723 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.723 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.723 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:23.723 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:23.723 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.723 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.723 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.723 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:23.723 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.723 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.724 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:23.724 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:23.724 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:23.724 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:23.724 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:23.724 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:23.724 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:23.724 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:23.724 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.724 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.724 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.724 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.724 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.724 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:23.724 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:23.724 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.724 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.725 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.725 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.725 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.725 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.725 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.725 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.725 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.725 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.725 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.725 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.725 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:23.725 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:23.725 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:23.725 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:23.725 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.725 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:23.726 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.726 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.726 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.726 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.726 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.726 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.726 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.726 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:23.726 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:23.726 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.726 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:23.726 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "osd_support", 2024-05-03T18:47:23.726 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:23.726 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:23.726 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:23.726 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:23.726 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:23.727 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.727 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.727 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.727 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:23.727 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.727 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.727 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:23.727 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:23.727 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:23.727 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:23.727 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:23.727 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:23.727 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:23.727 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:23.727 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.727 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.727 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.728 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.728 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.728 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:23.728 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:23.728 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.728 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.728 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.728 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.728 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.728 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.728 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.728 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.728 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.728 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.728 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.729 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.729 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:23.729 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:23.729 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.729 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.729 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.729 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:23.729 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.729 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.729 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:23.729 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:23.729 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:23.729 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:23.729 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:23.729 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:23.729 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:23.730 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:23.730 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.730 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.730 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.730 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.730 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.730 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:23.730 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:23.730 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.730 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.730 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.730 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.730 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.730 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.730 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.731 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.731 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.731 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.731 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.731 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.731 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:23.731 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:23.731 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:23.731 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:23.731 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.731 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:23.731 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.731 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.731 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.731 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.732 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.732 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.732 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.732 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:23.732 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:23.732 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.732 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:23.732 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "pg_autoscaler", 2024-05-03T18:47:23.732 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:23.732 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:23.732 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:23.732 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:23.732 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:23.732 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.732 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.732 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.733 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:23.733 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.733 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.733 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:23.733 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:23.733 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:23.733 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:23.733 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:23.733 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:23.733 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:23.733 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:23.733 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.733 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.733 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.733 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.733 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.734 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:23.734 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:23.734 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.734 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.734 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.734 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.734 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.734 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.734 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.734 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.734 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.734 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.734 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.735 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.735 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:23.735 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:23.735 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.735 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.735 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.735 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:23.735 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.735 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.735 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:23.735 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:23.735 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:23.735 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:23.735 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:23.735 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:23.736 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:23.736 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:23.736 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.736 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.736 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.736 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.736 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.736 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:23.736 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:23.736 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.736 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.736 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.736 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.736 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.736 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.736 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.737 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.737 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.737 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.737 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.737 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.737 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-05-03T18:47:23.737 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-05-03T18:47:23.737 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-03T18:47:23.737 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.737 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.737 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-05-03T18:47:23.737 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.737 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.737 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.737 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.737 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.738 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.738 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.738 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.738 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:23.738 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:23.738 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:23.738 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:23.738 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.738 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:23.738 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.738 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.738 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.738 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.738 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.738 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.739 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.739 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.739 INFO:teuthology.orchestra.run.smithi033.stdout: "threshold": { 2024-05-03T18:47:23.739 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "threshold", 2024-05-03T18:47:23.739 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-05-03T18:47:23.739 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.739 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.739 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "3.0", 2024-05-03T18:47:23.739 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1.0", 2024-05-03T18:47:23.739 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.739 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.739 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "scaling threshold", 2024-05-03T18:47:23.739 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-03T18:47:23.739 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.739 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.739 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:23.740 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:23.740 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.740 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:23.740 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "progress", 2024-05-03T18:47:23.740 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:23.740 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:23.740 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:23.740 INFO:teuthology.orchestra.run.smithi033.stdout: "allow_pg_recovery_event": { 2024-05-03T18:47:23.740 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "allow_pg_recovery_event", 2024-05-03T18:47:23.740 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.740 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.740 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.740 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.740 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.741 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.741 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.741 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-03T18:47:23.741 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.741 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.741 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.741 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.741 INFO:teuthology.orchestra.run.smithi033.stdout: "enabled": { 2024-05-03T18:47:23.741 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "enabled", 2024-05-03T18:47:23.741 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.741 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.741 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.741 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:23.742 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.742 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.742 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.742 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.742 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.742 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.742 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.742 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.742 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:23.742 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:23.742 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.742 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.742 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.742 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:23.742 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.743 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.743 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:23.743 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:23.743 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:23.743 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:23.743 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:23.743 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:23.743 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:23.743 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:23.743 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.743 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.743 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.743 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.743 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.743 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:23.744 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:23.744 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.744 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.744 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.744 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.744 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.744 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.744 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.744 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.744 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.744 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.744 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.744 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.744 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:23.744 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:23.745 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.745 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.745 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.745 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:23.745 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.745 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.745 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:23.745 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:23.745 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:23.745 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:23.745 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:23.745 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:23.745 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:23.745 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:23.745 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.746 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.746 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.746 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.746 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.746 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:23.746 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:23.746 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.746 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.746 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.746 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.746 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.746 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.746 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.747 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.747 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.747 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.747 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.747 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.747 INFO:teuthology.orchestra.run.smithi033.stdout: "max_completed_events": { 2024-05-03T18:47:23.747 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_completed_events", 2024-05-03T18:47:23.747 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:23.747 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.747 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.747 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "50", 2024-05-03T18:47:23.747 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.747 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.747 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.747 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "number of past completed events to remember", 2024-05-03T18:47:23.748 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.748 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.748 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.748 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.748 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-05-03T18:47:23.748 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-05-03T18:47:23.748 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-03T18:47:23.748 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.748 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.748 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5", 2024-05-03T18:47:23.748 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.748 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.748 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.748 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how long the module is going to sleep", 2024-05-03T18:47:23.748 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.749 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.749 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.749 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.749 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:23.749 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:23.749 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:23.749 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:23.749 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.749 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:23.749 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.749 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.749 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.749 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.749 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.749 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.749 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.750 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:23.750 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:23.750 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.750 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:23.750 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "prometheus", 2024-05-03T18:47:23.750 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:23.750 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:23.750 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:23.750 INFO:teuthology.orchestra.run.smithi033.stdout: "cache": { 2024-05-03T18:47:23.750 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cache", 2024-05-03T18:47:23.750 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.750 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.750 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.750 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:23.750 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.750 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.750 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.751 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.751 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.751 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.751 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.751 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.751 INFO:teuthology.orchestra.run.smithi033.stdout: "exclude_perf_counters": { 2024-05-03T18:47:23.751 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "exclude_perf_counters", 2024-05-03T18:47:23.751 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.751 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.751 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.751 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:23.751 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.751 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.751 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.751 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-03T18:47:23.751 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-03T18:47:23.752 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.752 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.752 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.752 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:23.752 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:23.752 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.752 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.752 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.752 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:23.752 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.752 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.752 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:23.752 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:23.753 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:23.753 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:23.753 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:23.753 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:23.753 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:23.753 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:23.753 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.754 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.754 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.754 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.754 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.754 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:23.754 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:23.754 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.754 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.754 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.754 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.754 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.754 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.754 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.754 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.755 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.755 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.755 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.755 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.755 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:23.755 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:23.756 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.756 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.756 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.756 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:23.756 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.756 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.757 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:23.757 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:23.757 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:23.757 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:23.757 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:23.757 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:23.758 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:23.758 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:23.758 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.758 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.758 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.758 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.758 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.758 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:23.758 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:23.758 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.758 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.759 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.759 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.759 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.759 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.759 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.759 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.759 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.759 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.760 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.760 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.760 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_stats_pools": { 2024-05-03T18:47:23.760 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rbd_stats_pools", 2024-05-03T18:47:23.760 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.760 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.760 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.760 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:23.760 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.760 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.760 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.761 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.761 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.761 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.761 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.761 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.761 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-03T18:47:23.761 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-03T18:47:23.762 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:23.762 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.762 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.762 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "300", 2024-05-03T18:47:23.762 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.762 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.762 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.762 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.762 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.762 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.762 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.762 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.762 INFO:teuthology.orchestra.run.smithi033.stdout: "scrape_interval": { 2024-05-03T18:47:23.763 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "scrape_interval", 2024-05-03T18:47:23.763 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-05-03T18:47:23.763 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.763 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.763 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "15.0", 2024-05-03T18:47:23.763 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.763 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.763 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.763 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.763 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.763 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.763 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.763 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.763 INFO:teuthology.orchestra.run.smithi033.stdout: "server_addr": { 2024-05-03T18:47:23.763 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_addr", 2024-05-03T18:47:23.763 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.764 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.764 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.764 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "::", 2024-05-03T18:47:23.764 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.764 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.764 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.764 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-03T18:47:23.764 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.764 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.764 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.764 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.764 INFO:teuthology.orchestra.run.smithi033.stdout: "server_port": { 2024-05-03T18:47:23.764 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_port", 2024-05-03T18:47:23.764 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:23.764 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.765 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.765 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "9283", 2024-05-03T18:47:23.765 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.765 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.765 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.765 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-03T18:47:23.765 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.765 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.765 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.765 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.765 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:23.765 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:23.765 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:23.765 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:23.765 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.766 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:23.766 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.766 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.766 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.766 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.766 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.766 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.766 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.766 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.766 INFO:teuthology.orchestra.run.smithi033.stdout: "stale_cache_strategy": { 2024-05-03T18:47:23.766 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "stale_cache_strategy", 2024-05-03T18:47:23.766 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.766 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.766 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.767 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "log", 2024-05-03T18:47:23.767 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.767 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.767 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.767 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.767 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.767 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.767 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.767 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.767 INFO:teuthology.orchestra.run.smithi033.stdout: "standby_behaviour": { 2024-05-03T18:47:23.767 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "standby_behaviour", 2024-05-03T18:47:23.767 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.767 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.767 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.767 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "default", 2024-05-03T18:47:23.768 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.768 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.768 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:23.768 INFO:teuthology.orchestra.run.smithi033.stdout: "default", 2024-05-03T18:47:23.768 INFO:teuthology.orchestra.run.smithi033.stdout: "error" 2024-05-03T18:47:23.768 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:23.768 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.768 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.768 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.768 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.768 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.768 INFO:teuthology.orchestra.run.smithi033.stdout: "standby_error_status_code": { 2024-05-03T18:47:23.768 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "standby_error_status_code", 2024-05-03T18:47:23.768 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:23.768 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.769 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.769 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "500", 2024-05-03T18:47:23.769 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "400", 2024-05-03T18:47:23.769 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "599", 2024-05-03T18:47:23.769 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.769 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.769 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.769 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.769 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.769 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:23.769 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:23.769 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.770 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:23.770 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rbd_support", 2024-05-03T18:47:23.770 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:23.770 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:23.770 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:23.770 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:23.770 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:23.770 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.770 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.770 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.770 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:23.770 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.770 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.771 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:23.771 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:23.771 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:23.771 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:23.771 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:23.771 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:23.771 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:23.771 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:23.771 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.771 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.771 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.771 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.771 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.771 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:23.771 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:23.772 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.772 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.772 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.772 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.772 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.772 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.772 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.772 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.772 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.772 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.772 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.772 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.772 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:23.772 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:23.772 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.773 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.773 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.773 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:23.773 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.773 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.773 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:23.773 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:23.773 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:23.773 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:23.773 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:23.773 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:23.773 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:23.773 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:23.773 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.774 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.774 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.774 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.774 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.774 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:23.774 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:23.774 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.774 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.774 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.774 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.774 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.774 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.774 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.774 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.774 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.775 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.775 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.775 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.775 INFO:teuthology.orchestra.run.smithi033.stdout: "max_concurrent_snap_create": { 2024-05-03T18:47:23.775 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_concurrent_snap_create", 2024-05-03T18:47:23.775 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:23.775 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.775 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.775 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-05-03T18:47:23.775 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.775 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.775 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.775 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.775 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.775 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.776 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.776 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.776 INFO:teuthology.orchestra.run.smithi033.stdout: "mirror_snapshot_schedule": { 2024-05-03T18:47:23.776 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mirror_snapshot_schedule", 2024-05-03T18:47:23.776 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.776 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.776 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.776 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:23.776 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.776 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.776 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.776 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.776 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.776 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.776 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.777 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.777 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:23.777 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:23.777 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:23.777 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:23.777 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.777 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:23.777 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.777 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.777 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.777 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.777 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.777 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.777 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.777 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.778 INFO:teuthology.orchestra.run.smithi033.stdout: "trash_purge_schedule": { 2024-05-03T18:47:23.778 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "trash_purge_schedule", 2024-05-03T18:47:23.778 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.778 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.778 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.778 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:23.778 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.778 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.778 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.778 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.778 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.778 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.778 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.778 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:23.778 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:23.779 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.779 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:23.779 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "restful", 2024-05-03T18:47:23.779 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:23.779 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:23.779 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:23.779 INFO:teuthology.orchestra.run.smithi033.stdout: "enable_auth": { 2024-05-03T18:47:23.779 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "enable_auth", 2024-05-03T18:47:23.779 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.779 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.779 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.779 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:23.779 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.779 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.779 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.779 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.780 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.780 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.780 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.780 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.780 INFO:teuthology.orchestra.run.smithi033.stdout: "key_file": { 2024-05-03T18:47:23.780 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "key_file", 2024-05-03T18:47:23.780 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.780 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.780 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.780 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:23.780 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.780 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.780 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.780 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.780 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.780 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.780 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.781 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.781 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:23.781 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:23.781 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.781 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.781 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.781 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:23.781 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.781 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.781 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:23.781 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:23.781 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:23.781 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:23.781 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:23.781 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:23.781 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:23.781 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:23.782 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.782 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.782 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.782 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.782 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.782 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:23.782 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:23.782 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.782 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.782 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.782 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.782 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.782 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.782 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.782 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.783 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.783 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.783 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.783 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.783 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:23.783 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:23.783 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.783 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.783 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.783 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:23.783 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.783 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.783 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:23.783 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:23.783 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:23.783 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:23.784 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:23.784 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:23.784 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:23.784 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:23.784 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.784 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.784 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.784 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.784 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.784 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:23.784 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:23.784 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.784 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.784 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.784 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.784 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.785 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.785 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.785 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.785 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.785 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.785 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.785 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.785 INFO:teuthology.orchestra.run.smithi033.stdout: "server_addr": { 2024-05-03T18:47:23.785 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_addr", 2024-05-03T18:47:23.785 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.785 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.785 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.785 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:23.785 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.785 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.785 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.786 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.786 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.786 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.786 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.786 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.786 INFO:teuthology.orchestra.run.smithi033.stdout: "server_port": { 2024-05-03T18:47:23.786 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_port", 2024-05-03T18:47:23.786 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.786 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.786 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.786 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:23.786 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.786 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.787 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.787 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.787 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.787 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.787 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.787 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.787 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:23.787 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:23.787 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:23.787 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:23.787 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.787 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:23.787 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.787 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.787 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.788 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.788 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.788 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.788 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.788 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:23.788 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:23.788 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.788 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:23.788 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rgw", 2024-05-03T18:47:23.788 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:23.788 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:23.788 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:23.788 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:23.788 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:23.788 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.788 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.789 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.789 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:23.789 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.789 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.789 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:23.789 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:23.789 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:23.789 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:23.789 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:23.789 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:23.789 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:23.789 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:23.789 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.790 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.790 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.790 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.790 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.790 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:23.790 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:23.790 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.790 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.790 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.790 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.790 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.790 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.790 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.790 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.791 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.791 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.791 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.791 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.791 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:23.791 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:23.791 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.791 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.791 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.791 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:23.791 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.791 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.791 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:23.791 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:23.791 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:23.791 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:23.792 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:23.792 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:23.792 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:23.792 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:23.792 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.792 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.792 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.792 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.792 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.792 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:23.792 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:23.792 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.792 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.792 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.792 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.792 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.793 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.793 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.793 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.793 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.793 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.793 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.793 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.793 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:23.793 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:23.793 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:23.793 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:23.793 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.793 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:23.793 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.793 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.794 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.794 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.794 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.794 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.794 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.794 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:23.794 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:23.794 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.794 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:23.794 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rook", 2024-05-03T18:47:23.794 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:23.794 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:23.794 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:23.794 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:23.795 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:23.795 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.795 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.795 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.795 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:23.795 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.795 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.795 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:23.795 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:23.795 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:23.795 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:23.795 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:23.795 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:23.795 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:23.795 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:23.796 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.796 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.796 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.796 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.796 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.796 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:23.796 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:23.796 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.796 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.796 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.796 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.796 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.796 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.797 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.797 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.797 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.797 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.797 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.797 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.797 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:23.797 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:23.797 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.797 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.797 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.797 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:23.797 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.797 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.798 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:23.798 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:23.798 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:23.798 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:23.798 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:23.798 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:23.798 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:23.798 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:23.798 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.798 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.798 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.798 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.798 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.798 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:23.799 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:23.799 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.799 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.799 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.799 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.799 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.799 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.799 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.799 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.799 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.799 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.799 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.799 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.799 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:23.799 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:23.800 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:23.800 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:23.800 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.800 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:23.800 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.800 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.800 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.800 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.800 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.800 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.800 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.800 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.800 INFO:teuthology.orchestra.run.smithi033.stdout: "storage_class": { 2024-05-03T18:47:23.800 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "storage_class", 2024-05-03T18:47:23.800 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.801 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.801 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.801 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "local", 2024-05-03T18:47:23.801 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.801 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.801 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.801 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-03T18:47:23.801 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.801 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.801 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.801 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:23.801 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:23.801 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.801 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:23.801 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "selftest", 2024-05-03T18:47:23.802 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:23.802 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:23.802 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:23.802 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:23.802 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:23.802 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.802 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.802 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.802 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:23.802 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.802 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.802 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:23.802 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:23.802 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:23.803 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:23.803 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:23.803 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:23.803 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:23.803 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:23.803 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.803 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.803 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.803 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.803 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.803 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:23.803 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:23.803 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.803 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.804 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.804 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.804 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.804 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.804 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.804 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.804 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.804 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.804 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.804 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.804 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:23.804 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:23.804 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.804 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.804 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.804 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:23.805 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.805 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.805 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:23.805 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:23.805 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:23.805 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:23.805 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:23.805 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:23.805 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:23.805 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:23.805 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.805 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.805 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.805 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.805 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.806 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:23.806 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:23.806 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.806 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.806 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.806 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.806 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.806 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.807 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.807 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.807 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.807 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.807 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.807 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.807 INFO:teuthology.orchestra.run.smithi033.stdout: "roption1": { 2024-05-03T18:47:23.807 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "roption1", 2024-05-03T18:47:23.807 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.807 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.807 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.807 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:23.807 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.807 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.808 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.808 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.808 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.808 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.808 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.808 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.808 INFO:teuthology.orchestra.run.smithi033.stdout: "roption2": { 2024-05-03T18:47:23.808 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "roption2", 2024-05-03T18:47:23.808 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.808 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.809 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.809 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "xyz", 2024-05-03T18:47:23.809 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.809 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.809 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.809 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.809 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.809 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.809 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.809 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.809 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption1": { 2024-05-03T18:47:23.809 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption1", 2024-05-03T18:47:23.809 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.809 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.809 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.810 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:23.810 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.810 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.810 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.810 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.810 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.810 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.810 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.810 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.810 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption2": { 2024-05-03T18:47:23.811 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption2", 2024-05-03T18:47:23.811 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:23.811 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.811 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.811 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:23.811 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.811 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.811 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.812 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.812 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.812 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.812 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.812 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.812 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption3": { 2024-05-03T18:47:23.812 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption3", 2024-05-03T18:47:23.812 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-05-03T18:47:23.812 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.812 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.812 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:23.812 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.813 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.813 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.813 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.813 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.813 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.813 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.813 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.813 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption4": { 2024-05-03T18:47:23.813 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption4", 2024-05-03T18:47:23.813 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.813 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.813 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.813 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:23.813 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.813 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.813 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.814 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.814 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.814 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.814 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.814 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.814 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption5": { 2024-05-03T18:47:23.814 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption5", 2024-05-03T18:47:23.814 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.814 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.814 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.814 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:23.814 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.814 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.815 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.815 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.815 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.815 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.815 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.815 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.815 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption6": { 2024-05-03T18:47:23.815 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption6", 2024-05-03T18:47:23.815 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.815 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.815 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.815 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:23.815 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.816 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.816 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.816 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.816 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.816 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.816 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.816 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.816 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption7": { 2024-05-03T18:47:23.816 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption7", 2024-05-03T18:47:23.816 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:23.816 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.816 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.816 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:23.816 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1", 2024-05-03T18:47:23.816 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "42", 2024-05-03T18:47:23.817 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.817 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.817 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.817 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.817 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.817 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.817 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:23.817 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:23.817 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:23.817 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:23.817 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.817 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:23.817 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.817 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.818 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.818 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.818 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.818 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.818 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.818 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.818 INFO:teuthology.orchestra.run.smithi033.stdout: "testkey": { 2024-05-03T18:47:23.818 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "testkey", 2024-05-03T18:47:23.818 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.818 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.818 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.818 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:23.819 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.819 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.819 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.819 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.819 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.819 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.819 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.819 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.819 INFO:teuthology.orchestra.run.smithi033.stdout: "testlkey": { 2024-05-03T18:47:23.819 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "testlkey", 2024-05-03T18:47:23.819 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.819 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.819 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.819 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:23.819 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.820 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.820 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.820 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.820 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.820 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.820 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.820 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.820 INFO:teuthology.orchestra.run.smithi033.stdout: "testnewline": { 2024-05-03T18:47:23.820 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "testnewline", 2024-05-03T18:47:23.820 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.820 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.820 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.820 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:23.820 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.820 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.820 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.820 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.821 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.821 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.821 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.821 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:23.821 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:23.821 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.821 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:23.821 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smb", 2024-05-03T18:47:23.821 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:23.821 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:23.821 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:23.821 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:23.821 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:23.821 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.821 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.821 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.821 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:23.822 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.822 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.822 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:23.822 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:23.822 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:23.822 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:23.822 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:23.822 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:23.822 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:23.822 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:23.822 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.822 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.822 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.822 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.822 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.823 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:23.823 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:23.823 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.823 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.823 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.823 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.823 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.823 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.823 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.823 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.823 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.823 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.823 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.823 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.823 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:23.823 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:23.823 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.824 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.824 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.824 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:23.824 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.824 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.824 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:23.824 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:23.824 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:23.824 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:23.824 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:23.824 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:23.824 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:23.824 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:23.824 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.824 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.825 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.825 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.825 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.825 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:23.825 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:23.825 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.825 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.825 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.825 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.825 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.825 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.825 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.825 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.825 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.825 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.826 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.826 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.826 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:23.826 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:23.826 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:23.826 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:23.826 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.826 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:23.826 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.826 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.826 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.826 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.826 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.826 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.826 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.827 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.827 INFO:teuthology.orchestra.run.smithi033.stdout: "update_orchestration": { 2024-05-03T18:47:23.827 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "update_orchestration", 2024-05-03T18:47:23.827 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.827 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.827 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.827 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:23.827 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.827 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.827 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.827 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-03T18:47:23.827 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.827 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.827 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.828 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:23.828 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:23.828 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.828 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:23.828 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "snap_schedule", 2024-05-03T18:47:23.828 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:23.828 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:23.828 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:23.828 INFO:teuthology.orchestra.run.smithi033.stdout: "allow_m_granularity": { 2024-05-03T18:47:23.828 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "allow_m_granularity", 2024-05-03T18:47:23.828 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.828 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.828 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.828 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.828 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.828 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.829 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.829 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "allow minute scheduled snapshots", 2024-05-03T18:47:23.829 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.829 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.829 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.829 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.829 INFO:teuthology.orchestra.run.smithi033.stdout: "dump_on_update": { 2024-05-03T18:47:23.829 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "dump_on_update", 2024-05-03T18:47:23.829 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.829 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.829 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.829 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.829 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.829 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.829 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.829 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "dump database to debug log on update", 2024-05-03T18:47:23.830 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.830 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.830 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.830 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.830 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:23.830 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:23.830 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.830 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.830 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.830 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:23.831 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.831 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.831 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:23.831 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:23.831 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:23.831 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:23.831 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:23.831 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:23.831 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:23.831 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:23.831 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.831 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.831 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.831 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.831 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.831 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:23.832 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:23.832 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.832 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.832 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.832 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.832 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.832 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.832 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.832 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.832 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.832 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.832 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.832 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.832 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:23.833 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:23.833 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.833 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.833 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.833 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:23.833 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.833 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.833 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:23.833 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:23.833 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:23.833 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:23.833 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:23.833 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:23.833 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:23.833 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:23.834 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.834 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.834 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.834 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.834 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.834 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:23.834 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:23.834 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.834 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.834 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.834 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.834 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.835 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.835 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.835 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.835 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.835 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.835 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.835 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.835 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:23.835 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:23.835 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:23.835 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:23.835 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.835 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:23.836 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.836 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.836 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.836 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.836 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.836 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.836 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.836 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:23.836 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:23.836 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.836 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:23.837 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "stats", 2024-05-03T18:47:23.837 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:23.837 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:23.837 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:23.837 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:23.837 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:23.837 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.837 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.837 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.837 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:23.837 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.837 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.837 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:23.837 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:23.837 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:23.838 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:23.838 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:23.838 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:23.838 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:23.838 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:23.838 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.838 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.838 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.838 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.838 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.838 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:23.838 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:23.838 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.839 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.839 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.839 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.839 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.839 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.839 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.839 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.839 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.839 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.839 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.839 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.839 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:23.839 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:23.840 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.840 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.840 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.840 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:23.840 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.840 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.840 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:23.840 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:23.840 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:23.840 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:23.840 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:23.840 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:23.840 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:23.840 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:23.840 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.840 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.841 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.841 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.841 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.841 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:23.841 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:23.841 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.841 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.841 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.841 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.841 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.841 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.841 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.842 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.842 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.842 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.842 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.842 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.842 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:23.842 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:23.842 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:23.842 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:23.842 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.842 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:23.842 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.842 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.843 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.843 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.843 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.843 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.843 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.843 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:23.843 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:23.843 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.843 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:23.843 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "status", 2024-05-03T18:47:23.843 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:23.843 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:23.843 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:23.843 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:23.843 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:23.843 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.844 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.844 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.844 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:23.844 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.844 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.844 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:23.844 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:23.844 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:23.845 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:23.845 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:23.845 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:23.845 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:23.845 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:23.845 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.845 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.845 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.845 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.845 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.845 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:23.845 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:23.845 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.845 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.845 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.846 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.846 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.846 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.846 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.846 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.846 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.846 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.846 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.846 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.846 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:23.847 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:23.847 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.847 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.847 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.847 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:23.847 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.848 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.848 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:23.848 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:23.848 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:23.848 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:23.848 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:23.849 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:23.849 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:23.849 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:23.849 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.849 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.849 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.849 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.849 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.849 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:23.849 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:23.849 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.849 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.850 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.850 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.850 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.850 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.850 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.850 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.850 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.850 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.851 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.851 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.851 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:23.851 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:23.851 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:23.851 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:23.851 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.851 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:23.851 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.851 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.851 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.851 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.852 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.852 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.852 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.852 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:23.852 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:23.852 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.852 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:23.852 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "telegraf", 2024-05-03T18:47:23.852 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:23.852 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:23.852 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:23.852 INFO:teuthology.orchestra.run.smithi033.stdout: "address": { 2024-05-03T18:47:23.852 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "address", 2024-05-03T18:47:23.852 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.852 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.852 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.853 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-03T18:47:23.853 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.853 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.853 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.853 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.853 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.853 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.853 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.853 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.853 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-05-03T18:47:23.853 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-05-03T18:47:23.853 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-03T18:47:23.853 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.853 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.853 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "15", 2024-05-03T18:47:23.853 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.854 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.854 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.854 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.854 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.854 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.854 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.854 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.854 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:23.854 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:23.854 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.854 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.854 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.854 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:23.855 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.855 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.855 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:23.855 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:23.855 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:23.855 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:23.855 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:23.855 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:23.855 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:23.855 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:23.855 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.855 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.855 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.855 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.856 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.856 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:23.856 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:23.856 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.856 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.856 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.856 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.856 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.856 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.856 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.856 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.856 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.856 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.857 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.857 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.857 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:23.857 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:23.857 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.857 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.857 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.857 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:23.857 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.857 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.857 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:23.857 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:23.857 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:23.858 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:23.858 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:23.858 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:23.858 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:23.858 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:23.858 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.858 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.858 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.858 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.859 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.859 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:23.859 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:23.859 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.859 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.859 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.859 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.859 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.859 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.859 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.859 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.859 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.859 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.860 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.860 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.860 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:23.860 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:23.860 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:23.860 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:23.860 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.860 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:23.860 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.860 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.860 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.860 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.860 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.860 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.860 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.861 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:23.861 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:23.861 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.861 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:23.861 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "telemetry", 2024-05-03T18:47:23.861 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:23.861 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:23.861 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:23.861 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_basic": { 2024-05-03T18:47:23.861 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_basic", 2024-05-03T18:47:23.861 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.861 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.861 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.862 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:23.862 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.862 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.862 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.862 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-03T18:47:23.862 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.862 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.863 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.863 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.863 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_crash": { 2024-05-03T18:47:23.863 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_crash", 2024-05-03T18:47:23.863 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.863 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.863 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.863 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:23.863 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.863 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.863 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.863 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-03T18:47:23.863 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.863 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.864 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.864 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.864 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_device": { 2024-05-03T18:47:23.864 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_device", 2024-05-03T18:47:23.864 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.864 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.864 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.864 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:23.864 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.864 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.864 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.864 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-03T18:47:23.864 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.864 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.864 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.865 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.865 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_ident": { 2024-05-03T18:47:23.865 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_ident", 2024-05-03T18:47:23.865 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.865 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.865 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.865 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.865 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.865 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.865 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.865 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-03T18:47:23.865 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.865 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.865 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.866 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.866 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_perf": { 2024-05-03T18:47:23.866 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_perf", 2024-05-03T18:47:23.866 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.866 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.866 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.866 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.866 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.866 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.866 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.866 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-03T18:47:23.866 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.866 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.866 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.867 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.867 INFO:teuthology.orchestra.run.smithi033.stdout: "contact": { 2024-05-03T18:47:23.867 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "contact", 2024-05-03T18:47:23.867 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.867 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.867 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.867 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:23.867 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.867 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.867 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.867 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.867 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.868 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.868 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.868 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.868 INFO:teuthology.orchestra.run.smithi033.stdout: "description": { 2024-05-03T18:47:23.868 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "description", 2024-05-03T18:47:23.868 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.868 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.868 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.868 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:23.869 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.869 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.869 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.869 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.869 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.869 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.869 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.869 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.869 INFO:teuthology.orchestra.run.smithi033.stdout: "device_url": { 2024-05-03T18:47:23.869 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "device_url", 2024-05-03T18:47:23.869 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.869 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.870 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.870 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-03T18:47:23.870 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.870 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.870 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.870 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.870 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.870 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.871 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.871 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.871 INFO:teuthology.orchestra.run.smithi033.stdout: "enabled": { 2024-05-03T18:47:23.871 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "enabled", 2024-05-03T18:47:23.871 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.872 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.872 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.872 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.872 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.872 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.872 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.873 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.873 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.873 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.873 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.873 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.873 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-05-03T18:47:23.873 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-05-03T18:47:23.873 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:23.874 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.874 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.874 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "24", 2024-05-03T18:47:23.874 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "8", 2024-05-03T18:47:23.874 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.874 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.874 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.874 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.874 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.874 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.874 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.875 INFO:teuthology.orchestra.run.smithi033.stdout: "last_opt_revision": { 2024-05-03T18:47:23.875 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "last_opt_revision", 2024-05-03T18:47:23.875 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:23.875 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.875 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.875 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "1", 2024-05-03T18:47:23.875 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.876 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.876 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.876 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.876 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.876 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.876 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.876 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.876 INFO:teuthology.orchestra.run.smithi033.stdout: "leaderboard": { 2024-05-03T18:47:23.876 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "leaderboard", 2024-05-03T18:47:23.876 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.876 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.876 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.876 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.876 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.876 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.877 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.877 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.877 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.877 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.877 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.877 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.877 INFO:teuthology.orchestra.run.smithi033.stdout: "leaderboard_description": { 2024-05-03T18:47:23.877 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "leaderboard_description", 2024-05-03T18:47:23.877 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.877 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.877 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.877 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:23.877 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.877 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.878 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.878 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.878 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.878 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.878 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.878 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.878 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:23.878 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:23.878 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.878 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.878 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.878 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:23.879 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.879 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.879 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:23.879 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:23.879 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:23.879 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:23.879 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:23.879 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:23.879 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:23.879 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:23.879 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.879 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.879 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.879 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.880 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.880 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:23.880 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:23.880 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.880 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.880 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.880 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.880 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.880 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.880 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.880 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.880 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.880 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.880 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.880 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.881 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:23.881 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:23.881 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.881 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.881 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.881 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:23.881 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.881 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.881 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:23.881 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:23.881 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:23.881 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:23.881 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:23.881 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:23.881 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:23.882 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:23.882 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.882 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.882 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.882 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.882 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.882 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:23.882 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:23.882 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.882 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.882 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.882 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.882 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.883 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.883 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.883 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.883 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.883 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.883 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.883 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.883 INFO:teuthology.orchestra.run.smithi033.stdout: "organization": { 2024-05-03T18:47:23.883 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "organization", 2024-05-03T18:47:23.883 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.883 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.883 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.883 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:23.883 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.884 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.884 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.884 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.884 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.884 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.884 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.884 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.884 INFO:teuthology.orchestra.run.smithi033.stdout: "proxy": { 2024-05-03T18:47:23.884 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "proxy", 2024-05-03T18:47:23.884 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.884 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.884 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.884 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:23.884 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.884 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.885 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.885 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.885 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.885 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.885 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.885 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.885 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:23.885 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:23.885 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:23.885 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:23.885 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.885 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:23.885 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.885 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.885 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.885 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.886 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.886 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.886 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.886 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.886 INFO:teuthology.orchestra.run.smithi033.stdout: "url": { 2024-05-03T18:47:23.886 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "url", 2024-05-03T18:47:23.886 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.886 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.886 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.886 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-03T18:47:23.886 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.887 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.887 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.887 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.887 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.887 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.887 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.887 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:23.887 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:23.887 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.887 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:23.887 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "test_orchestrator", 2024-05-03T18:47:23.887 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:23.887 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:23.887 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:23.887 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:23.887 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:23.888 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.888 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.888 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.888 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:23.888 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.888 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.888 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:23.888 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:23.888 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:23.888 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:23.888 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:23.888 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:23.888 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:23.888 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:23.888 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.888 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.888 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.889 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.889 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.889 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:23.889 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:23.889 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.889 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.889 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.889 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.889 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.889 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.889 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.889 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.889 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.889 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.889 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.889 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.890 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:23.890 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:23.890 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.890 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.890 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.890 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:23.890 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.890 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.890 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:23.890 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:23.891 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:23.891 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:23.891 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:23.891 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:23.891 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:23.891 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:23.891 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.891 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.891 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.891 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.891 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.891 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:23.891 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:23.891 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.892 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.892 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.892 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.892 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.892 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.892 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.892 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.892 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.892 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.892 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.892 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.892 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:23.892 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:23.892 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:23.892 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:23.893 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.893 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:23.893 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.893 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.893 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.893 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.893 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.893 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.893 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.893 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:23.893 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:23.893 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.893 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:23.893 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "volumes", 2024-05-03T18:47:23.893 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:23.894 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:23.894 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:23.894 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:23.894 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:23.894 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.894 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.894 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.894 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:23.894 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.894 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.894 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:23.894 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:23.894 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:23.894 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:23.895 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:23.895 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:23.895 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:23.895 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:23.895 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.895 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.895 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.895 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.895 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.895 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:23.895 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:23.895 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.895 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.895 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.895 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.896 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.896 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.896 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.896 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.896 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.896 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.896 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.896 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.896 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:23.896 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:23.896 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.896 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.896 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.896 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:23.896 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.897 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.897 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:23.897 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:23.897 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:23.897 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:23.897 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:23.897 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:23.897 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:23.897 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:23.897 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.897 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.897 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.897 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.897 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.897 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:23.898 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:23.898 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.898 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.898 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.898 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.898 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.898 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.898 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.898 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.898 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.898 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.898 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.898 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.899 INFO:teuthology.orchestra.run.smithi033.stdout: "max_concurrent_clones": { 2024-05-03T18:47:23.899 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_concurrent_clones", 2024-05-03T18:47:23.899 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:23.899 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.899 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.899 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "4", 2024-05-03T18:47:23.899 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.899 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.899 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.899 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-03T18:47:23.899 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.899 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.899 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.899 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.900 INFO:teuthology.orchestra.run.smithi033.stdout: "periodic_async_work": { 2024-05-03T18:47:23.900 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "periodic_async_work", 2024-05-03T18:47:23.900 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.900 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.900 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.900 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.900 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.900 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.900 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.900 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Periodically check for async work", 2024-05-03T18:47:23.900 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.900 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.900 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.900 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.900 INFO:teuthology.orchestra.run.smithi033.stdout: "snapshot_clone_delay": { 2024-05-03T18:47:23.901 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "snapshot_clone_delay", 2024-05-03T18:47:23.901 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:23.901 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.901 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.901 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:23.901 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.901 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.901 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.901 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-03T18:47:23.901 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.901 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.901 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.901 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.901 INFO:teuthology.orchestra.run.smithi033.stdout: "snapshot_clone_no_wait": { 2024-05-03T18:47:23.901 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "snapshot_clone_no_wait", 2024-05-03T18:47:23.902 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.902 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.902 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.902 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:23.902 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.902 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.902 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.902 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-03T18:47:23.902 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.902 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.902 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.902 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.903 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:23.903 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:23.903 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:23.903 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:23.903 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.903 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:23.903 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.903 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.903 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.903 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.903 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.903 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.903 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.903 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:23.904 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:23.904 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.904 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:23.904 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "zabbix", 2024-05-03T18:47:23.904 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:23.904 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:23.904 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:23.904 INFO:teuthology.orchestra.run.smithi033.stdout: "discovery_interval": { 2024-05-03T18:47:23.904 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "discovery_interval", 2024-05-03T18:47:23.904 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-05-03T18:47:23.904 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.904 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.904 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "100", 2024-05-03T18:47:23.904 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.904 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.905 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.905 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.905 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.905 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.905 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.905 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.905 INFO:teuthology.orchestra.run.smithi033.stdout: "identifier": { 2024-05-03T18:47:23.905 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "identifier", 2024-05-03T18:47:23.905 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.905 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.905 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.905 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:23.905 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.905 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.905 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.905 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.906 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.906 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.906 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.906 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.906 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-05-03T18:47:23.906 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-05-03T18:47:23.906 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-03T18:47:23.906 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.906 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.906 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-05-03T18:47:23.906 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.906 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.907 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.907 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.907 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.907 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.907 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.907 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.907 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:23.907 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:23.907 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.907 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.907 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.907 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:23.907 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.907 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.907 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:23.907 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:23.908 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:23.908 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:23.908 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:23.908 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:23.908 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:23.908 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:23.908 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.908 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.908 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.908 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.908 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.908 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:23.908 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:23.908 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.908 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.908 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.908 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.909 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.909 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.909 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.909 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.909 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.909 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.909 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.909 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.909 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:23.909 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:23.909 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.909 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.910 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.910 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:23.910 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.910 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.910 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:23.910 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:23.910 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:23.910 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:23.910 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:23.910 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:23.910 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:23.910 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:23.910 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.910 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.911 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.911 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.911 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.911 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:23.911 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:23.911 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.911 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.911 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.911 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.911 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.911 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.911 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.911 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.911 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.911 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.912 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.912 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.912 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:23.912 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:23.912 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:23.912 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:23.912 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.912 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:23.912 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.912 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.912 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.912 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.912 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.912 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.913 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.913 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.913 INFO:teuthology.orchestra.run.smithi033.stdout: "zabbix_host": { 2024-05-03T18:47:23.913 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "zabbix_host", 2024-05-03T18:47:23.913 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.913 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.913 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.913 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:23.913 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.913 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.913 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.913 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.913 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.913 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.913 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.914 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.914 INFO:teuthology.orchestra.run.smithi033.stdout: "zabbix_port": { 2024-05-03T18:47:23.914 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "zabbix_port", 2024-05-03T18:47:23.914 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:23.914 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.914 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.914 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10051", 2024-05-03T18:47:23.914 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.914 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.914 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.914 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.914 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.914 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.915 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.915 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.915 INFO:teuthology.orchestra.run.smithi033.stdout: "zabbix_sender": { 2024-05-03T18:47:23.915 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "zabbix_sender", 2024-05-03T18:47:23.915 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.915 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.915 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.915 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-03T18:47:23.915 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.915 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.915 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.915 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.915 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.915 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.915 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.915 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:23.916 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:23.916 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:23.916 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-03T18:47:23.916 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.916 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:23.916 INFO:teuthology.orchestra.run.smithi033.stdout: "gid": 34889, 2024-05-03T18:47:23.916 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "y", 2024-05-03T18:47:23.916 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr_features": 4540701547738038271, 2024-05-03T18:47:23.916 INFO:teuthology.orchestra.run.smithi033.stdout: "available_modules": [ 2024-05-03T18:47:23.916 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:23.916 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "alerts", 2024-05-03T18:47:23.916 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:23.916 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:23.916 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:23.916 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-05-03T18:47:23.917 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-05-03T18:47:23.917 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-03T18:47:23.917 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.917 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.917 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-05-03T18:47:23.917 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.917 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.917 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.917 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "How frequently to reexamine health status", 2024-05-03T18:47:23.917 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.917 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.917 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.917 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.917 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:23.917 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:23.918 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.918 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.918 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.918 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:23.918 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.918 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.918 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:23.918 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:23.918 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:23.918 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:23.918 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:23.918 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:23.918 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:23.919 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:23.919 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.919 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.919 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.919 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.919 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.919 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:23.919 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:23.919 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.919 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.919 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.919 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.919 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.920 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.920 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.920 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.920 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.920 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.920 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.920 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.920 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:23.920 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:23.920 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.920 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.920 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.920 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:23.920 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.920 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.921 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:23.921 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:23.921 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:23.921 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:23.921 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:23.921 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:23.921 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:23.921 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:23.921 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.921 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.921 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.921 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.921 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.921 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:23.921 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:23.922 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.922 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.922 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.922 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.922 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.922 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.922 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.922 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.922 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.922 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.922 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.922 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.922 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_destination": { 2024-05-03T18:47:23.922 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_destination", 2024-05-03T18:47:23.922 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.923 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.923 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.923 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:23.923 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.923 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.923 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.923 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Email address to send alerts to", 2024-05-03T18:47:23.923 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.923 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.923 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.923 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.923 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_from_name": { 2024-05-03T18:47:23.923 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_from_name", 2024-05-03T18:47:23.923 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.923 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.923 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.924 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "Ceph", 2024-05-03T18:47:23.924 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.924 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.924 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.924 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Email From: name", 2024-05-03T18:47:23.924 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.924 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.924 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.924 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.924 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_host": { 2024-05-03T18:47:23.924 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_host", 2024-05-03T18:47:23.924 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.924 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.924 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.924 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:23.925 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.925 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.925 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.925 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "SMTP server", 2024-05-03T18:47:23.925 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.925 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.925 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.925 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.925 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_password": { 2024-05-03T18:47:23.925 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_password", 2024-05-03T18:47:23.925 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.925 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.925 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.926 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:23.926 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.926 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.926 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.926 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Password to authenticate with", 2024-05-03T18:47:23.926 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.926 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.926 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.926 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.926 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_port": { 2024-05-03T18:47:23.926 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_port", 2024-05-03T18:47:23.926 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:23.926 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.927 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.927 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "465", 2024-05-03T18:47:23.927 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.927 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.927 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.927 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "SMTP port", 2024-05-03T18:47:23.927 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.927 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.927 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.927 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.927 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_sender": { 2024-05-03T18:47:23.927 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_sender", 2024-05-03T18:47:23.927 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.927 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.927 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.927 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:23.928 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.928 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.928 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.928 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "SMTP envelope sender", 2024-05-03T18:47:23.928 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.928 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.928 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.928 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.928 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_ssl": { 2024-05-03T18:47:23.928 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_ssl", 2024-05-03T18:47:23.928 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.928 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.929 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.929 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:23.929 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.929 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.929 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.929 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-03T18:47:23.929 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.929 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.929 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.929 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.929 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_user": { 2024-05-03T18:47:23.929 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_user", 2024-05-03T18:47:23.929 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.929 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.929 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.929 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:23.929 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.930 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.930 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.930 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "User to authenticate as", 2024-05-03T18:47:23.930 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.930 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.930 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.930 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.930 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:23.930 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:23.930 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:23.930 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:23.930 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.930 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:23.930 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.930 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.931 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.931 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.931 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.931 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.931 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.931 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:23.931 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:23.931 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.931 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:23.931 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "balancer", 2024-05-03T18:47:23.931 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:23.931 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:23.931 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:23.931 INFO:teuthology.orchestra.run.smithi033.stdout: "active": { 2024-05-03T18:47:23.931 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "active", 2024-05-03T18:47:23.931 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.931 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.932 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.932 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:23.932 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.932 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.932 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.932 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "automatically balance PGs across cluster", 2024-05-03T18:47:23.932 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.932 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.932 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.932 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.932 INFO:teuthology.orchestra.run.smithi033.stdout: "begin_time": { 2024-05-03T18:47:23.932 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "begin_time", 2024-05-03T18:47:23.932 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.932 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.932 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.932 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0000", 2024-05-03T18:47:23.932 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.933 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.933 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.933 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "beginning time of day to automatically balance", 2024-05-03T18:47:23.933 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-03T18:47:23.933 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.933 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.933 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.933 INFO:teuthology.orchestra.run.smithi033.stdout: "begin_weekday": { 2024-05-03T18:47:23.933 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "begin_weekday", 2024-05-03T18:47:23.933 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-05-03T18:47:23.933 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.933 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.933 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:23.933 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "0", 2024-05-03T18:47:23.933 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "6", 2024-05-03T18:47:23.933 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.933 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-03T18:47:23.934 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-03T18:47:23.934 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.934 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.934 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.934 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_compat_max_iterations": { 2024-05-03T18:47:23.934 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crush_compat_max_iterations", 2024-05-03T18:47:23.934 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-05-03T18:47:23.934 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.934 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.934 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "25", 2024-05-03T18:47:23.934 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1", 2024-05-03T18:47:23.934 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "250", 2024-05-03T18:47:23.934 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.934 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-03T18:47:23.934 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.935 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.935 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.935 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.935 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_compat_metrics": { 2024-05-03T18:47:23.935 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crush_compat_metrics", 2024-05-03T18:47:23.935 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.935 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.935 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.935 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "pgs,objects,bytes", 2024-05-03T18:47:23.935 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.935 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.935 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.935 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-03T18:47:23.935 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-03T18:47:23.935 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.935 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.936 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.936 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_compat_step": { 2024-05-03T18:47:23.936 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crush_compat_step", 2024-05-03T18:47:23.936 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-05-03T18:47:23.936 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.936 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.936 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0.5", 2024-05-03T18:47:23.937 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "0.001", 2024-05-03T18:47:23.937 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "0.999", 2024-05-03T18:47:23.937 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.937 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "aggressiveness of optimization", 2024-05-03T18:47:23.937 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-03T18:47:23.937 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.937 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.937 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.937 INFO:teuthology.orchestra.run.smithi033.stdout: "end_time": { 2024-05-03T18:47:23.937 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "end_time", 2024-05-03T18:47:23.937 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.937 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.937 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.937 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "2359", 2024-05-03T18:47:23.938 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.938 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.938 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.938 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "ending time of day to automatically balance", 2024-05-03T18:47:23.938 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-03T18:47:23.938 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.938 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.938 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.939 INFO:teuthology.orchestra.run.smithi033.stdout: "end_weekday": { 2024-05-03T18:47:23.939 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "end_weekday", 2024-05-03T18:47:23.939 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-05-03T18:47:23.939 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.939 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.940 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:23.940 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "0", 2024-05-03T18:47:23.940 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "6", 2024-05-03T18:47:23.940 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.940 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-03T18:47:23.940 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-03T18:47:23.940 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.940 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.941 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.941 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:23.941 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:23.941 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.941 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.941 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.941 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:23.941 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.941 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.941 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:23.941 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:23.941 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:23.941 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:23.942 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:23.942 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:23.942 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:23.942 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:23.942 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.942 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.942 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.942 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.943 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.943 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:23.943 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:23.943 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.943 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.943 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.943 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.943 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.943 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.944 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.944 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.944 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.944 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.944 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.944 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.944 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:23.944 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:23.944 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.944 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.944 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.944 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:23.944 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.944 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.945 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:23.945 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:23.945 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:23.945 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:23.945 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:23.945 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:23.945 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:23.945 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:23.945 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.945 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.945 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.945 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.945 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.945 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:23.945 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:23.946 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.946 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.946 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.946 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.946 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.946 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.946 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.946 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.946 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.946 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.946 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.946 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.946 INFO:teuthology.orchestra.run.smithi033.stdout: "min_score": { 2024-05-03T18:47:23.947 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "min_score", 2024-05-03T18:47:23.947 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-05-03T18:47:23.947 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.947 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.947 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:23.947 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.947 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.947 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.947 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-03T18:47:23.947 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.947 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.947 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.947 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.947 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": { 2024-05-03T18:47:23.947 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mode", 2024-05-03T18:47:23.948 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.948 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.948 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.948 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "upmap", 2024-05-03T18:47:23.948 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.948 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.948 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:23.948 INFO:teuthology.orchestra.run.smithi033.stdout: "crush-compat", 2024-05-03T18:47:23.948 INFO:teuthology.orchestra.run.smithi033.stdout: "none", 2024-05-03T18:47:23.948 INFO:teuthology.orchestra.run.smithi033.stdout: "read", 2024-05-03T18:47:23.948 INFO:teuthology.orchestra.run.smithi033.stdout: "upmap", 2024-05-03T18:47:23.948 INFO:teuthology.orchestra.run.smithi033.stdout: "upmap-read" 2024-05-03T18:47:23.948 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:23.948 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Balancer mode", 2024-05-03T18:47:23.948 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.949 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.949 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.949 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.949 INFO:teuthology.orchestra.run.smithi033.stdout: "pool_ids": { 2024-05-03T18:47:23.949 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "pool_ids", 2024-05-03T18:47:23.949 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.949 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.949 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.949 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:23.949 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.949 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.949 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.949 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-03T18:47:23.950 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.950 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.950 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.950 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.950 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-05-03T18:47:23.950 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-05-03T18:47:23.950 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-03T18:47:23.950 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.950 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.950 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-05-03T18:47:23.950 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.950 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.950 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.951 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-03T18:47:23.951 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.951 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.951 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.951 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.951 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:23.951 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:23.951 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:23.951 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:23.951 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.951 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:23.951 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.951 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.951 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.952 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.952 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.952 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.952 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.952 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.952 INFO:teuthology.orchestra.run.smithi033.stdout: "upmap_max_deviation": { 2024-05-03T18:47:23.952 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "upmap_max_deviation", 2024-05-03T18:47:23.952 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:23.952 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.952 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.952 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5", 2024-05-03T18:47:23.952 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1", 2024-05-03T18:47:23.952 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.953 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.953 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-03T18:47:23.953 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-03T18:47:23.953 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.953 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.953 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.953 INFO:teuthology.orchestra.run.smithi033.stdout: "upmap_max_optimizations": { 2024-05-03T18:47:23.953 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "upmap_max_optimizations", 2024-05-03T18:47:23.953 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-05-03T18:47:23.953 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.953 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.953 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-05-03T18:47:23.953 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.953 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.954 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.954 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-03T18:47:23.954 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.954 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.954 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.954 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:23.954 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:23.954 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.954 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:23.954 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cephadm", 2024-05-03T18:47:23.954 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:23.954 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:23.955 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:23.955 INFO:teuthology.orchestra.run.smithi033.stdout: "agent_down_multiplier": { 2024-05-03T18:47:23.955 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "agent_down_multiplier", 2024-05-03T18:47:23.955 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-05-03T18:47:23.955 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.955 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.955 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "3.0", 2024-05-03T18:47:23.955 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.955 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.955 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.955 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-03T18:47:23.955 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.955 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.955 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.955 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.956 INFO:teuthology.orchestra.run.smithi033.stdout: "agent_refresh_rate": { 2024-05-03T18:47:23.956 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "agent_refresh_rate", 2024-05-03T18:47:23.956 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-03T18:47:23.956 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.956 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.956 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "20", 2024-05-03T18:47:23.956 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.956 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.956 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.956 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-03T18:47:23.956 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.956 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.956 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.956 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.957 INFO:teuthology.orchestra.run.smithi033.stdout: "agent_starting_port": { 2024-05-03T18:47:23.957 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "agent_starting_port", 2024-05-03T18:47:23.957 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:23.957 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.957 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.957 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "4721", 2024-05-03T18:47:23.957 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.957 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.957 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.957 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-03T18:47:23.957 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.957 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.957 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.957 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.957 INFO:teuthology.orchestra.run.smithi033.stdout: "allow_ptrace": { 2024-05-03T18:47:23.957 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "allow_ptrace", 2024-05-03T18:47:23.958 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.958 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.958 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.958 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.958 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.958 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.958 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.958 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-03T18:47:23.958 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-03T18:47:23.958 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.958 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.958 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.958 INFO:teuthology.orchestra.run.smithi033.stdout: "autotune_interval": { 2024-05-03T18:47:23.958 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "autotune_interval", 2024-05-03T18:47:23.958 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-03T18:47:23.959 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.959 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.959 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-05-03T18:47:23.959 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.959 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.959 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.959 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-03T18:47:23.959 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.959 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.959 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.959 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.960 INFO:teuthology.orchestra.run.smithi033.stdout: "autotune_memory_target_ratio": { 2024-05-03T18:47:23.960 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "autotune_memory_target_ratio", 2024-05-03T18:47:23.960 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-05-03T18:47:23.960 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.960 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.960 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0.7", 2024-05-03T18:47:23.960 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.960 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.960 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.960 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-03T18:47:23.961 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.961 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.961 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.961 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.961 INFO:teuthology.orchestra.run.smithi033.stdout: "cephadm_log_destination": { 2024-05-03T18:47:23.961 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cephadm_log_destination", 2024-05-03T18:47:23.961 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.961 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.961 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.961 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:23.961 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.961 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.961 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:23.961 INFO:teuthology.orchestra.run.smithi033.stdout: "file", 2024-05-03T18:47:23.962 INFO:teuthology.orchestra.run.smithi033.stdout: "file,syslog", 2024-05-03T18:47:23.962 INFO:teuthology.orchestra.run.smithi033.stdout: "syslog" 2024-05-03T18:47:23.962 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:23.962 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-03T18:47:23.962 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.962 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.962 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.962 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.962 INFO:teuthology.orchestra.run.smithi033.stdout: "cgroups_split": { 2024-05-03T18:47:23.962 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cgroups_split", 2024-05-03T18:47:23.962 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.963 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.963 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.963 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:23.963 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.963 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.963 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.963 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-03T18:47:23.963 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.963 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.963 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.964 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.964 INFO:teuthology.orchestra.run.smithi033.stdout: "config_checks_enabled": { 2024-05-03T18:47:23.964 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "config_checks_enabled", 2024-05-03T18:47:23.964 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.964 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.964 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.964 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.964 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.964 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.964 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.964 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-03T18:47:23.964 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.965 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.965 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.965 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.965 INFO:teuthology.orchestra.run.smithi033.stdout: "config_dashboard": { 2024-05-03T18:47:23.965 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "config_dashboard", 2024-05-03T18:47:23.965 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.965 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.965 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.965 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:23.965 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.965 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.965 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.965 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-03T18:47:23.965 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.965 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.966 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.966 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.966 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_alertmanager": { 2024-05-03T18:47:23.966 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_alertmanager", 2024-05-03T18:47:23.966 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.966 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.966 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.966 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-03T18:47:23.966 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.966 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.966 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.966 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Prometheus container image", 2024-05-03T18:47:23.966 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.966 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.966 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.967 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.967 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_base": { 2024-05-03T18:47:23.967 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_base", 2024-05-03T18:47:23.967 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.967 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.967 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.967 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-03T18:47:23.967 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.967 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.967 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.967 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Container image name, without the tag", 2024-05-03T18:47:23.967 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.967 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.967 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.967 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.967 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_elasticsearch": { 2024-05-03T18:47:23.968 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_elasticsearch", 2024-05-03T18:47:23.968 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.968 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.968 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.968 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-03T18:47:23.968 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.968 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.968 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.968 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "elasticsearch container image", 2024-05-03T18:47:23.968 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.968 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.968 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.968 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.968 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_grafana": { 2024-05-03T18:47:23.968 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_grafana", 2024-05-03T18:47:23.969 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.969 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.969 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.969 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-03T18:47:23.969 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.969 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.969 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.969 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Prometheus container image", 2024-05-03T18:47:23.969 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.969 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.969 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.969 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.969 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_haproxy": { 2024-05-03T18:47:23.969 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_haproxy", 2024-05-03T18:47:23.970 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.970 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.970 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.970 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-03T18:47:23.970 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.970 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.970 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.970 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "HAproxy container image", 2024-05-03T18:47:23.970 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.970 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.970 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.970 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.970 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_jaeger_agent": { 2024-05-03T18:47:23.971 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_jaeger_agent", 2024-05-03T18:47:23.971 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.971 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.971 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.971 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-03T18:47:23.971 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.971 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.971 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.971 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Jaeger agent container image", 2024-05-03T18:47:23.971 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.971 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.971 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.971 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.971 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_jaeger_collector": { 2024-05-03T18:47:23.972 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_jaeger_collector", 2024-05-03T18:47:23.972 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.972 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.972 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.972 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-03T18:47:23.972 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.972 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.972 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.972 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Jaeger collector container image", 2024-05-03T18:47:23.972 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.972 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.972 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.972 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.972 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_jaeger_query": { 2024-05-03T18:47:23.972 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_jaeger_query", 2024-05-03T18:47:23.972 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.973 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.973 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.973 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-03T18:47:23.973 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.973 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.973 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.973 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Jaeger query container image", 2024-05-03T18:47:23.973 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.973 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.973 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.973 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.973 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_keepalived": { 2024-05-03T18:47:23.973 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_keepalived", 2024-05-03T18:47:23.973 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.973 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.974 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.974 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-03T18:47:23.974 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.974 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.974 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.974 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Keepalived container image", 2024-05-03T18:47:23.974 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.974 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.974 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.974 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.974 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_loki": { 2024-05-03T18:47:23.974 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_loki", 2024-05-03T18:47:23.974 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.974 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.975 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.975 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-03T18:47:23.975 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.975 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.975 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.975 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Loki container image", 2024-05-03T18:47:23.975 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.975 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.975 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.975 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.975 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_node_exporter": { 2024-05-03T18:47:23.976 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_node_exporter", 2024-05-03T18:47:23.976 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.976 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.976 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.976 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-03T18:47:23.976 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.976 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.976 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.976 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Prometheus container image", 2024-05-03T18:47:23.976 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.976 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.976 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.976 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.976 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_nvmeof": { 2024-05-03T18:47:23.976 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_nvmeof", 2024-05-03T18:47:23.976 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.977 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.977 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.977 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-05-03T18:47:23.977 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.977 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.977 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.977 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Nvme-of container image", 2024-05-03T18:47:23.977 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.977 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.977 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.977 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.977 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_prometheus": { 2024-05-03T18:47:23.977 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_prometheus", 2024-05-03T18:47:23.977 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.977 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.977 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.978 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-03T18:47:23.978 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.978 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.978 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.978 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Prometheus container image", 2024-05-03T18:47:23.978 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.978 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.978 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.978 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.978 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_promtail": { 2024-05-03T18:47:23.978 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_promtail", 2024-05-03T18:47:23.978 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.978 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.978 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.979 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-03T18:47:23.979 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.979 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.979 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.979 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Promtail container image", 2024-05-03T18:47:23.979 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.979 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.979 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.979 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.979 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_samba": { 2024-05-03T18:47:23.979 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_samba", 2024-05-03T18:47:23.979 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.979 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.979 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.979 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-03T18:47:23.979 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.980 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.980 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.980 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Samba/SMB container image", 2024-05-03T18:47:23.980 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.980 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.980 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.980 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.980 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_snmp_gateway": { 2024-05-03T18:47:23.980 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_snmp_gateway", 2024-05-03T18:47:23.980 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.980 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.980 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.980 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-03T18:47:23.980 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.980 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.980 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.981 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "SNMP Gateway container image", 2024-05-03T18:47:23.981 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.981 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.981 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.981 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.981 INFO:teuthology.orchestra.run.smithi033.stdout: "container_init": { 2024-05-03T18:47:23.981 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_init", 2024-05-03T18:47:23.981 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.981 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.981 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.981 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:23.981 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.981 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.981 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.981 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Run podman/docker with `--init`", 2024-05-03T18:47:23.982 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.982 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.982 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.982 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.982 INFO:teuthology.orchestra.run.smithi033.stdout: "daemon_cache_timeout": { 2024-05-03T18:47:23.982 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "daemon_cache_timeout", 2024-05-03T18:47:23.982 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-03T18:47:23.982 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.982 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.982 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-05-03T18:47:23.982 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.983 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.983 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.983 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-03T18:47:23.983 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.983 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.983 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.983 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.983 INFO:teuthology.orchestra.run.smithi033.stdout: "default_cephadm_command_timeout": { 2024-05-03T18:47:23.983 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "default_cephadm_command_timeout", 2024-05-03T18:47:23.984 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:23.984 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.984 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.984 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "900", 2024-05-03T18:47:23.984 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.984 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.984 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.984 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-03T18:47:23.984 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.984 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.984 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.984 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.984 INFO:teuthology.orchestra.run.smithi033.stdout: "default_registry": { 2024-05-03T18:47:23.984 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "default_registry", 2024-05-03T18:47:23.984 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.985 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.985 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.985 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "docker.io", 2024-05-03T18:47:23.985 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.985 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.985 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.985 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-03T18:47:23.985 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.986 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.986 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.986 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.986 INFO:teuthology.orchestra.run.smithi033.stdout: "device_cache_timeout": { 2024-05-03T18:47:23.986 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "device_cache_timeout", 2024-05-03T18:47:23.986 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-03T18:47:23.987 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.987 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.987 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "1800", 2024-05-03T18:47:23.987 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.987 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.988 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.988 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "seconds to cache device inventory", 2024-05-03T18:47:23.988 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.988 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.988 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.988 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.988 INFO:teuthology.orchestra.run.smithi033.stdout: "device_enhanced_scan": { 2024-05-03T18:47:23.988 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "device_enhanced_scan", 2024-05-03T18:47:23.988 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.988 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.989 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.989 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.989 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.989 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.989 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.989 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-03T18:47:23.989 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.989 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.989 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.990 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.990 INFO:teuthology.orchestra.run.smithi033.stdout: "facts_cache_timeout": { 2024-05-03T18:47:23.990 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "facts_cache_timeout", 2024-05-03T18:47:23.990 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-03T18:47:23.990 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.990 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.990 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-05-03T18:47:23.990 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.991 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.991 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.991 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "seconds to cache host facts data", 2024-05-03T18:47:23.991 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.991 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.991 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.991 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.991 INFO:teuthology.orchestra.run.smithi033.stdout: "grafana_dashboards_path": { 2024-05-03T18:47:23.991 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "grafana_dashboards_path", 2024-05-03T18:47:23.991 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.991 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.991 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.991 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-03T18:47:23.991 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.991 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.992 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.992 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-03T18:47:23.992 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.992 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.992 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.992 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.992 INFO:teuthology.orchestra.run.smithi033.stdout: "host_check_interval": { 2024-05-03T18:47:23.992 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "host_check_interval", 2024-05-03T18:47:23.992 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-03T18:47:23.992 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.992 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.992 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-05-03T18:47:23.992 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.992 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.992 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.992 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to perform a host check", 2024-05-03T18:47:23.993 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.993 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.993 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.993 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.993 INFO:teuthology.orchestra.run.smithi033.stdout: "hw_monitoring": { 2024-05-03T18:47:23.993 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "hw_monitoring", 2024-05-03T18:47:23.993 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.993 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.993 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.993 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.993 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.993 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.993 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.993 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-03T18:47:23.994 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.994 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.994 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.994 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.994 INFO:teuthology.orchestra.run.smithi033.stdout: "inventory_list_all": { 2024-05-03T18:47:23.994 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "inventory_list_all", 2024-05-03T18:47:23.994 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.994 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.994 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.994 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.994 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.994 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.994 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.994 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-03T18:47:23.995 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.995 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.995 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.995 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.995 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:23.995 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:23.995 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.995 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.995 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.995 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:23.995 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.995 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.995 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:23.995 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:23.995 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:23.995 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:23.995 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:23.996 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:23.996 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:23.996 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:23.996 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.996 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.996 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.996 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.996 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.996 INFO:teuthology.orchestra.run.smithi033.stdout: "log_refresh_metadata": { 2024-05-03T18:47:23.996 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_refresh_metadata", 2024-05-03T18:47:23.996 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.996 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.996 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.996 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:23.996 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.996 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.997 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.997 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-03T18:47:23.997 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.997 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.997 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.997 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.997 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:23.997 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:23.997 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.997 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.997 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:23.997 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:23.997 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.997 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.997 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:23.997 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-03T18:47:23.998 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.998 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.998 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.998 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.998 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:23.998 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:23.998 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:23.998 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.998 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.998 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:23.998 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:23.998 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:23.998 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:23.998 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:23.998 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:23.998 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:23.999 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:23.999 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:23.999 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:23.999 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:23.999 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:23.999 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:23.999 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:23.999 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:23.999 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:23.999 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:23.999 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:23.999 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:23.999 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:23.999 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:23.999 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.000 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.000 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.000 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.000 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.000 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.000 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.000 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.000 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.000 INFO:teuthology.orchestra.run.smithi033.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-03T18:47:24.000 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-03T18:47:24.000 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.000 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.000 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.000 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.000 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.000 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.000 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.001 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-03T18:47:24.001 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.001 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.001 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.001 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.001 INFO:teuthology.orchestra.run.smithi033.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-03T18:47:24.001 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-03T18:47:24.001 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.001 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.001 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.001 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "*", 2024-05-03T18:47:24.001 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.001 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.001 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.001 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-03T18:47:24.001 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.001 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.002 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.002 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.002 INFO:teuthology.orchestra.run.smithi033.stdout: "max_count_per_host": { 2024-05-03T18:47:24.002 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_count_per_host", 2024-05-03T18:47:24.002 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.002 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.002 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.002 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-05-03T18:47:24.002 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.002 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.002 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.002 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "max number of daemons per service per host", 2024-05-03T18:47:24.002 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.002 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.002 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.002 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.003 INFO:teuthology.orchestra.run.smithi033.stdout: "max_osd_draining_count": { 2024-05-03T18:47:24.003 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_osd_draining_count", 2024-05-03T18:47:24.003 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.003 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.003 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.003 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-05-03T18:47:24.003 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.003 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.003 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.003 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-03T18:47:24.003 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.003 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.003 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.003 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.003 INFO:teuthology.orchestra.run.smithi033.stdout: "migration_current": { 2024-05-03T18:47:24.004 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "migration_current", 2024-05-03T18:47:24.004 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.004 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.004 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.004 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.004 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.004 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.004 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.004 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "internal - do not modify", 2024-05-03T18:47:24.004 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.004 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.004 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.004 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.004 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": { 2024-05-03T18:47:24.005 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mode", 2024-05-03T18:47:24.005 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.005 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.005 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.005 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "root", 2024-05-03T18:47:24.005 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.005 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.005 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.005 INFO:teuthology.orchestra.run.smithi033.stdout: "cephadm-package", 2024-05-03T18:47:24.005 INFO:teuthology.orchestra.run.smithi033.stdout: "root" 2024-05-03T18:47:24.005 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.005 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "mode for remote execution of cephadm", 2024-05-03T18:47:24.005 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.005 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.005 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.005 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.006 INFO:teuthology.orchestra.run.smithi033.stdout: "oob_default_addr": { 2024-05-03T18:47:24.006 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "oob_default_addr", 2024-05-03T18:47:24.006 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.006 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.006 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.006 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "169.254.1.1", 2024-05-03T18:47:24.006 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.006 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.006 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.006 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-03T18:47:24.006 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.006 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.006 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.006 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.006 INFO:teuthology.orchestra.run.smithi033.stdout: "prometheus_alerts_path": { 2024-05-03T18:47:24.007 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "prometheus_alerts_path", 2024-05-03T18:47:24.007 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.007 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.007 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.007 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-03T18:47:24.007 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.007 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.007 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.007 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-03T18:47:24.007 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.007 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.007 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.007 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.007 INFO:teuthology.orchestra.run.smithi033.stdout: "registry_insecure": { 2024-05-03T18:47:24.008 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "registry_insecure", 2024-05-03T18:47:24.008 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.008 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.008 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.008 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.008 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.008 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.008 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.008 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-03T18:47:24.008 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.008 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.008 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.008 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.008 INFO:teuthology.orchestra.run.smithi033.stdout: "registry_password": { 2024-05-03T18:47:24.008 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "registry_password", 2024-05-03T18:47:24.008 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.009 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.009 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.009 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.009 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.009 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.009 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.009 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-03T18:47:24.009 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.009 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.009 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.009 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.009 INFO:teuthology.orchestra.run.smithi033.stdout: "registry_url": { 2024-05-03T18:47:24.009 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "registry_url", 2024-05-03T18:47:24.010 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.010 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.010 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.010 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.010 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.010 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.010 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.010 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-03T18:47:24.010 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.011 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.011 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.011 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.011 INFO:teuthology.orchestra.run.smithi033.stdout: "registry_username": { 2024-05-03T18:47:24.011 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "registry_username", 2024-05-03T18:47:24.011 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.011 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.012 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.012 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.012 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.012 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.012 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.012 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-03T18:47:24.012 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.012 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.013 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.013 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.013 INFO:teuthology.orchestra.run.smithi033.stdout: "secure_monitoring_stack": { 2024-05-03T18:47:24.013 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "secure_monitoring_stack", 2024-05-03T18:47:24.013 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.013 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.014 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.014 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.014 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.014 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.014 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.014 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-03T18:47:24.014 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.015 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.015 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.015 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.015 INFO:teuthology.orchestra.run.smithi033.stdout: "service_discovery_port": { 2024-05-03T18:47:24.015 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "service_discovery_port", 2024-05-03T18:47:24.016 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.016 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.016 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.016 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8765", 2024-05-03T18:47:24.016 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.016 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.016 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.017 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "cephadm service discovery port", 2024-05-03T18:47:24.017 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.017 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.017 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.017 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.017 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:24.018 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:24.018 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.018 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:24.018 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.018 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:24.018 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.019 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.019 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.019 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.019 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.019 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.019 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.019 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.020 INFO:teuthology.orchestra.run.smithi033.stdout: "ssh_config_file": { 2024-05-03T18:47:24.020 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ssh_config_file", 2024-05-03T18:47:24.020 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.020 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.020 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.020 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.020 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.020 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.021 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.021 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-03T18:47:24.021 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.021 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.021 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.021 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.022 INFO:teuthology.orchestra.run.smithi033.stdout: "use_agent": { 2024-05-03T18:47:24.022 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "use_agent", 2024-05-03T18:47:24.022 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.022 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.022 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.022 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.023 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.023 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.023 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.023 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-03T18:47:24.023 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.024 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.024 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.024 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.024 INFO:teuthology.orchestra.run.smithi033.stdout: "use_repo_digest": { 2024-05-03T18:47:24.024 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "use_repo_digest", 2024-05-03T18:47:24.024 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.025 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.025 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.025 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:24.025 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.025 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.025 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.026 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-03T18:47:24.026 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.026 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.026 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.026 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.026 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_on_failed_host_check": { 2024-05-03T18:47:24.027 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_on_failed_host_check", 2024-05-03T18:47:24.027 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.027 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.027 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.027 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:24.027 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.028 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.028 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.028 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "raise a health warning if the host check fails", 2024-05-03T18:47:24.028 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.028 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.028 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.029 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.029 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_on_stray_daemons": { 2024-05-03T18:47:24.029 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_on_stray_daemons", 2024-05-03T18:47:24.029 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.029 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.029 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.030 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:24.030 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.030 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.030 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.030 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-03T18:47:24.030 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.030 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.031 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.031 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.031 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_on_stray_hosts": { 2024-05-03T18:47:24.031 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_on_stray_hosts", 2024-05-03T18:47:24.031 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.031 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.032 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.032 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:24.032 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.032 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.032 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.032 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-03T18:47:24.032 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.033 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.033 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.033 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.033 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.033 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.033 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:24.033 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crash", 2024-05-03T18:47:24.033 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:24.033 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:24.034 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:24.034 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:24.034 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:24.034 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.034 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.034 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.034 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.034 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.035 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.035 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.035 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.035 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.035 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.035 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.035 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.035 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.035 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.036 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.036 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.036 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.036 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.036 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.036 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:24.036 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:24.037 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.037 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.037 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.037 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.037 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.037 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.037 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.037 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.037 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.037 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.037 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.038 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.038 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:24.038 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:24.038 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.038 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.038 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.038 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:24.039 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.039 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.039 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.039 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.039 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.039 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.039 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.039 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.039 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.039 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.039 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.040 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.040 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.040 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.040 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.040 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:24.040 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:24.040 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.040 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.040 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.041 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.041 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.041 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.041 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.041 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.041 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.042 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.042 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.042 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.042 INFO:teuthology.orchestra.run.smithi033.stdout: "retain_interval": { 2024-05-03T18:47:24.042 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "retain_interval", 2024-05-03T18:47:24.043 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-03T18:47:24.043 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.043 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.043 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "31536000", 2024-05-03T18:47:24.043 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.043 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.044 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.044 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-03T18:47:24.044 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.044 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.044 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.044 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.044 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:24.044 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:24.044 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.044 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:24.045 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.045 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:24.045 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.045 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.045 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.045 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.046 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.046 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.046 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.046 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.046 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_recent_interval": { 2024-05-03T18:47:24.046 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_recent_interval", 2024-05-03T18:47:24.046 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-03T18:47:24.046 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.046 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.046 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "1209600", 2024-05-03T18:47:24.046 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.046 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.046 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.047 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-03T18:47:24.047 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.047 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.047 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.047 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.047 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.047 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.047 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:24.047 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "dashboard", 2024-05-03T18:47:24.047 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:24.047 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:24.047 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:24.047 INFO:teuthology.orchestra.run.smithi033.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-03T18:47:24.047 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-03T18:47:24.047 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.048 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.048 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.048 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-05-03T18:47:24.048 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.048 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.048 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.048 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.048 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.048 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.048 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.048 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.048 INFO:teuthology.orchestra.run.smithi033.stdout: "ALERTMANAGER_API_HOST": { 2024-05-03T18:47:24.048 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-03T18:47:24.048 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.048 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.049 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.049 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.049 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.049 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.049 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.049 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.049 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.049 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.049 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.049 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.049 INFO:teuthology.orchestra.run.smithi033.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-03T18:47:24.049 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-03T18:47:24.049 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.049 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.049 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.049 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:24.050 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.050 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.050 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.050 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.050 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.050 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.050 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.050 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.050 INFO:teuthology.orchestra.run.smithi033.stdout: "AUDIT_API_ENABLED": { 2024-05-03T18:47:24.050 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "AUDIT_API_ENABLED", 2024-05-03T18:47:24.050 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.050 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.051 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.051 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.051 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.051 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.051 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.051 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.051 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.051 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.051 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.051 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.051 INFO:teuthology.orchestra.run.smithi033.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-03T18:47:24.051 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-03T18:47:24.051 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.051 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.051 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.052 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:24.052 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.052 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.052 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.052 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.052 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.052 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.052 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.052 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.052 INFO:teuthology.orchestra.run.smithi033.stdout: "ENABLE_BROWSABLE_API": { 2024-05-03T18:47:24.052 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-03T18:47:24.052 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.052 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.052 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.053 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:24.053 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.053 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.053 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.053 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.053 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.053 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.053 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.053 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.053 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-03T18:47:24.053 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-03T18:47:24.053 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.053 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.053 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.053 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:24.054 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.054 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.054 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.054 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.054 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.054 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.054 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.054 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.054 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-03T18:47:24.054 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-03T18:47:24.054 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.054 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.054 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.054 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:24.055 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.055 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.055 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.055 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.055 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.055 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.055 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.055 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.055 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-03T18:47:24.055 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-03T18:47:24.055 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.055 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.055 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.055 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:24.055 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.056 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.056 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.056 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.056 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.056 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.056 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.056 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.056 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-03T18:47:24.056 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-03T18:47:24.056 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.056 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.056 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.056 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:24.056 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.056 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.057 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.057 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.057 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.057 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.057 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.057 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.057 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-03T18:47:24.057 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-03T18:47:24.057 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.057 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.057 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.057 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:24.057 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.057 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.058 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.058 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.058 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.058 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.058 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.058 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.058 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-03T18:47:24.058 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-03T18:47:24.058 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.058 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.058 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.058 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:24.058 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.059 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.059 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.059 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.059 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.059 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.059 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.059 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.059 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-03T18:47:24.059 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-03T18:47:24.059 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.059 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.059 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.059 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:24.059 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.060 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.060 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.060 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.060 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.060 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.060 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.060 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.060 INFO:teuthology.orchestra.run.smithi033.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-03T18:47:24.060 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-03T18:47:24.060 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.060 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.060 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.060 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.060 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.061 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.061 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.061 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.061 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.061 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.061 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.061 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.061 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_API_PASSWORD": { 2024-05-03T18:47:24.062 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-03T18:47:24.062 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.062 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.062 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.062 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "admin", 2024-05-03T18:47:24.062 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.062 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.062 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.062 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.062 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.062 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.062 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.062 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.063 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-03T18:47:24.063 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-03T18:47:24.063 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.063 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.063 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.063 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:24.063 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.063 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.063 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.063 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.063 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.063 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.064 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.064 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.064 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_API_URL": { 2024-05-03T18:47:24.064 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_API_URL", 2024-05-03T18:47:24.064 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.064 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.064 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.064 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.064 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.064 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.064 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.064 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.064 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.064 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.065 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.065 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.065 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_API_USERNAME": { 2024-05-03T18:47:24.065 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-03T18:47:24.065 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.065 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.065 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.065 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "admin", 2024-05-03T18:47:24.065 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.065 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.065 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.065 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.065 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.065 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.066 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.066 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.066 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-03T18:47:24.066 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-03T18:47:24.066 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.066 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.066 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.066 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.066 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.066 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.066 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.066 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.066 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.067 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.067 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.067 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.067 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-03T18:47:24.067 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-03T18:47:24.067 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.067 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.067 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.067 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.067 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.067 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.067 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.067 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.067 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.067 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.068 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.068 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.068 INFO:teuthology.orchestra.run.smithi033.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-03T18:47:24.068 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-03T18:47:24.068 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.068 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.068 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.068 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:24.068 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.068 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.068 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.068 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.068 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.068 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.069 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.069 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.069 INFO:teuthology.orchestra.run.smithi033.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-03T18:47:24.069 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-03T18:47:24.069 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.069 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.069 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.069 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.069 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.069 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.069 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.069 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.069 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.069 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.069 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.069 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.070 INFO:teuthology.orchestra.run.smithi033.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-03T18:47:24.070 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-03T18:47:24.070 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.070 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.070 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.070 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "[]", 2024-05-03T18:47:24.070 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.070 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.070 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.070 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.070 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.070 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.071 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.071 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.071 INFO:teuthology.orchestra.run.smithi033.stdout: "MULTICLUSTER_CONFIG": { 2024-05-03T18:47:24.071 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-03T18:47:24.071 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.071 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.071 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.071 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "{}", 2024-05-03T18:47:24.071 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.071 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.071 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.071 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.071 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.071 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.071 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.071 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.072 INFO:teuthology.orchestra.run.smithi033.stdout: "PROMETHEUS_API_HOST": { 2024-05-03T18:47:24.072 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-03T18:47:24.072 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.072 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.072 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.072 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.072 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.072 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.072 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.072 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.072 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.072 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.072 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.072 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.072 INFO:teuthology.orchestra.run.smithi033.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-03T18:47:24.072 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-03T18:47:24.073 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.073 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.073 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.073 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:24.073 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.073 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.073 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.073 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.073 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.073 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.073 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.073 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.073 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-03T18:47:24.073 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-03T18:47:24.073 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.074 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.074 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.074 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.074 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.074 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.074 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.074 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.074 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.074 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.074 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.074 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.074 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-03T18:47:24.074 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-03T18:47:24.074 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.075 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.075 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.075 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.075 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.075 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.075 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.075 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.075 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.075 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.075 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.075 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.075 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-03T18:47:24.075 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-03T18:47:24.076 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.076 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.076 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.076 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:24.076 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.076 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.076 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.076 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.076 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.076 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.076 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.076 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.076 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-03T18:47:24.076 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-03T18:47:24.077 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.077 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.077 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.077 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:24.077 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.077 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.077 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.077 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.077 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.077 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.077 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.077 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.077 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-03T18:47:24.077 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-03T18:47:24.078 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.078 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.078 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.078 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.078 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.078 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.078 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.078 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.078 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.078 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.078 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.078 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.078 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-03T18:47:24.078 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-03T18:47:24.078 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.079 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.079 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.079 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.079 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.079 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.079 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.079 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.079 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.079 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.079 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.079 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.079 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-03T18:47:24.079 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-03T18:47:24.079 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.079 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.079 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.080 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.080 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.080 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.080 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.080 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.080 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.080 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.080 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.080 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.080 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_ENABLED": { 2024-05-03T18:47:24.080 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-03T18:47:24.080 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.080 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.080 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.080 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:24.080 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.080 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.081 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.081 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.081 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.081 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.081 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.081 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.081 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-03T18:47:24.081 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-03T18:47:24.081 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.081 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.081 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.081 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-03T18:47:24.081 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.081 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.081 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.081 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.082 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.082 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.082 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.082 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.082 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-03T18:47:24.082 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-03T18:47:24.082 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.082 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.082 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.082 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-05-03T18:47:24.082 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.082 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.082 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.082 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.083 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.083 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.083 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.083 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.083 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-03T18:47:24.083 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-03T18:47:24.083 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.083 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.083 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.083 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8", 2024-05-03T18:47:24.083 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.083 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.083 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.083 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.083 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.083 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.083 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.084 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.084 INFO:teuthology.orchestra.run.smithi033.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-03T18:47:24.084 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-03T18:47:24.084 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.084 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.084 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.084 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "45", 2024-05-03T18:47:24.084 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.084 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.084 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.084 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.084 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.084 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.084 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.084 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.084 INFO:teuthology.orchestra.run.smithi033.stdout: "RGW_API_ACCESS_KEY": { 2024-05-03T18:47:24.084 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-03T18:47:24.085 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.085 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.085 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.085 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.085 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.085 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.085 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.085 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.085 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.085 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.085 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.085 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.085 INFO:teuthology.orchestra.run.smithi033.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-03T18:47:24.085 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-03T18:47:24.085 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.085 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.086 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.086 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "admin", 2024-05-03T18:47:24.086 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.086 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.086 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.086 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.086 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.086 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.086 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.086 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.087 INFO:teuthology.orchestra.run.smithi033.stdout: "RGW_API_SECRET_KEY": { 2024-05-03T18:47:24.087 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-03T18:47:24.087 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.087 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.087 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.087 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.087 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.087 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.087 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.087 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.087 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.087 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.088 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.088 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.088 INFO:teuthology.orchestra.run.smithi033.stdout: "RGW_API_SSL_VERIFY": { 2024-05-03T18:47:24.088 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-03T18:47:24.088 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.088 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.088 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.088 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:24.088 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.088 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.088 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.088 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.088 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.089 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.089 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.089 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.089 INFO:teuthology.orchestra.run.smithi033.stdout: "UNSAFE_TLS_v1_2": { 2024-05-03T18:47:24.089 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-03T18:47:24.089 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.089 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.089 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.089 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.089 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.089 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.089 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.089 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.089 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.089 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.090 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.090 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.090 INFO:teuthology.orchestra.run.smithi033.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-03T18:47:24.090 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-03T18:47:24.090 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.090 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.090 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.090 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:24.091 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.091 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.091 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.091 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.091 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.091 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.091 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.091 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.092 INFO:teuthology.orchestra.run.smithi033.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-03T18:47:24.092 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-03T18:47:24.092 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.092 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.092 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.092 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-05-03T18:47:24.092 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.092 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.092 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.092 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.092 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.092 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.092 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.092 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.093 INFO:teuthology.orchestra.run.smithi033.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-03T18:47:24.093 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-03T18:47:24.093 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.093 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.093 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.093 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5", 2024-05-03T18:47:24.093 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.093 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.093 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.093 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.093 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.093 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.093 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.093 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.093 INFO:teuthology.orchestra.run.smithi033.stdout: "cross_origin_url": { 2024-05-03T18:47:24.093 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cross_origin_url", 2024-05-03T18:47:24.094 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.094 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.094 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.094 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.094 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.094 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.094 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.094 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.094 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.094 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.094 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.094 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.094 INFO:teuthology.orchestra.run.smithi033.stdout: "crt_file": { 2024-05-03T18:47:24.095 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crt_file", 2024-05-03T18:47:24.095 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.095 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.095 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.095 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.095 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.095 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.095 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.095 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.095 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.095 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.095 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.095 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.095 INFO:teuthology.orchestra.run.smithi033.stdout: "debug": { 2024-05-03T18:47:24.095 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "debug", 2024-05-03T18:47:24.095 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.096 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.096 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.096 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.096 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.096 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.096 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.096 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Enable/disable debug options", 2024-05-03T18:47:24.096 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.096 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.096 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.096 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.096 INFO:teuthology.orchestra.run.smithi033.stdout: "jwt_token_ttl": { 2024-05-03T18:47:24.096 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "jwt_token_ttl", 2024-05-03T18:47:24.096 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.096 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.096 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.097 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "28800", 2024-05-03T18:47:24.097 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.097 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.097 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.097 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.097 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.097 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.097 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.097 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.097 INFO:teuthology.orchestra.run.smithi033.stdout: "key_file": { 2024-05-03T18:47:24.097 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "key_file", 2024-05-03T18:47:24.097 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.097 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.097 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.097 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.097 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.097 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.098 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.098 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.098 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.098 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.098 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.098 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.098 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:24.098 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:24.098 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.099 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.099 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.099 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.099 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.099 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.099 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.099 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.099 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.099 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.099 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.099 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.100 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.100 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.100 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.100 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.100 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.100 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.100 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.100 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:24.100 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:24.100 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.100 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.100 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.100 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.100 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.100 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.100 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.101 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.101 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.101 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.101 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.101 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.101 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:24.101 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:24.101 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.102 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.102 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.102 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:24.102 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.102 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.102 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.103 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.103 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.103 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.103 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.103 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.103 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.104 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.104 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.104 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.104 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.104 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.104 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.104 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:24.104 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:24.104 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.104 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.105 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.105 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.105 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.105 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.105 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.105 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.105 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.105 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.105 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.105 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.106 INFO:teuthology.orchestra.run.smithi033.stdout: "motd": { 2024-05-03T18:47:24.106 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "motd", 2024-05-03T18:47:24.106 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.106 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.106 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.106 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.106 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.106 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.106 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.107 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "The message of the day", 2024-05-03T18:47:24.107 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.107 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.107 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.107 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.107 INFO:teuthology.orchestra.run.smithi033.stdout: "redirect_resolve_ip_addr": { 2024-05-03T18:47:24.107 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "redirect_resolve_ip_addr", 2024-05-03T18:47:24.107 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.107 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.107 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.107 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.107 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.107 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.107 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.107 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.107 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.107 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.108 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.108 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.108 INFO:teuthology.orchestra.run.smithi033.stdout: "server_addr": { 2024-05-03T18:47:24.108 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_addr", 2024-05-03T18:47:24.108 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.108 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.108 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.108 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "::", 2024-05-03T18:47:24.108 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.108 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.108 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.109 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.109 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.109 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.109 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.109 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.109 INFO:teuthology.orchestra.run.smithi033.stdout: "server_port": { 2024-05-03T18:47:24.109 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_port", 2024-05-03T18:47:24.109 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.109 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.109 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.109 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8080", 2024-05-03T18:47:24.110 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.110 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.110 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.110 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.110 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.110 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.110 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.110 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.111 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:24.111 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:24.111 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.111 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:24.111 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.111 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:24.111 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.111 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.111 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.111 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.112 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.112 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.112 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.112 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.112 INFO:teuthology.orchestra.run.smithi033.stdout: "ssl": { 2024-05-03T18:47:24.112 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ssl", 2024-05-03T18:47:24.112 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.112 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.112 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.113 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:24.113 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.113 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.113 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.113 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.113 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.113 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.113 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.113 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.113 INFO:teuthology.orchestra.run.smithi033.stdout: "ssl_server_port": { 2024-05-03T18:47:24.114 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ssl_server_port", 2024-05-03T18:47:24.114 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.114 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.114 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.114 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8443", 2024-05-03T18:47:24.114 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.114 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.114 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.115 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.115 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.115 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.115 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.115 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.115 INFO:teuthology.orchestra.run.smithi033.stdout: "standby_behaviour": { 2024-05-03T18:47:24.115 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "standby_behaviour", 2024-05-03T18:47:24.115 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.115 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.116 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.116 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "redirect", 2024-05-03T18:47:24.116 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.116 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.116 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.116 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.116 INFO:teuthology.orchestra.run.smithi033.stdout: "redirect" 2024-05-03T18:47:24.116 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.116 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.117 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.117 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.117 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.117 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.117 INFO:teuthology.orchestra.run.smithi033.stdout: "standby_error_status_code": { 2024-05-03T18:47:24.117 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "standby_error_status_code", 2024-05-03T18:47:24.117 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.117 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.118 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.118 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "500", 2024-05-03T18:47:24.118 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "400", 2024-05-03T18:47:24.118 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "599", 2024-05-03T18:47:24.118 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.118 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.118 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.118 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.119 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.119 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.119 INFO:teuthology.orchestra.run.smithi033.stdout: "url_prefix": { 2024-05-03T18:47:24.119 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "url_prefix", 2024-05-03T18:47:24.119 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.119 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.119 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.119 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.119 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.119 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.120 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.120 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.120 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.120 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.120 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.120 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.120 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.120 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.121 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:24.121 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "devicehealth", 2024-05-03T18:47:24.121 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:24.121 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:24.121 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:24.121 INFO:teuthology.orchestra.run.smithi033.stdout: "enable_monitoring": { 2024-05-03T18:47:24.121 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "enable_monitoring", 2024-05-03T18:47:24.121 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.121 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.121 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.122 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:24.122 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.122 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.122 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.122 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "monitor device health metrics", 2024-05-03T18:47:24.122 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.122 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.122 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.122 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.122 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:24.122 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:24.122 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.122 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.122 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.123 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.123 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.123 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.123 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.123 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.123 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.123 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.123 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.123 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.123 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.123 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.123 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.123 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.123 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.123 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.123 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.124 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:24.124 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:24.124 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.124 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.124 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.124 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.124 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.124 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.124 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.124 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.124 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.124 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.124 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.124 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.124 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:24.124 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:24.124 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.125 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.125 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.125 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:24.125 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.125 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.125 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.125 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.125 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.125 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.125 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.125 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.125 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.125 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.125 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.125 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.125 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.125 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.126 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.126 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:24.126 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:24.126 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.126 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.126 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.126 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.126 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.126 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.126 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.126 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.126 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.126 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.126 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.127 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.127 INFO:teuthology.orchestra.run.smithi033.stdout: "mark_out_threshold": { 2024-05-03T18:47:24.127 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mark_out_threshold", 2024-05-03T18:47:24.127 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-03T18:47:24.127 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.127 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.127 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "2419200", 2024-05-03T18:47:24.127 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.127 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.127 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.127 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-03T18:47:24.127 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.127 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.127 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.127 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.127 INFO:teuthology.orchestra.run.smithi033.stdout: "pool_name": { 2024-05-03T18:47:24.128 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "pool_name", 2024-05-03T18:47:24.128 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.128 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.128 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.128 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "device_health_metrics", 2024-05-03T18:47:24.128 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.128 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.128 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.128 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-03T18:47:24.128 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.128 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.128 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.128 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.128 INFO:teuthology.orchestra.run.smithi033.stdout: "retention_period": { 2024-05-03T18:47:24.128 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "retention_period", 2024-05-03T18:47:24.128 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-03T18:47:24.129 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.129 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.129 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "15552000", 2024-05-03T18:47:24.129 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.129 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.129 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.129 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how long to retain device health metrics", 2024-05-03T18:47:24.129 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.129 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.129 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.129 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.129 INFO:teuthology.orchestra.run.smithi033.stdout: "scrape_frequency": { 2024-05-03T18:47:24.129 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "scrape_frequency", 2024-05-03T18:47:24.129 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-03T18:47:24.129 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.129 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.129 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "86400", 2024-05-03T18:47:24.130 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.130 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.130 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.130 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-03T18:47:24.130 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.130 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.130 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.130 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.130 INFO:teuthology.orchestra.run.smithi033.stdout: "self_heal": { 2024-05-03T18:47:24.130 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "self_heal", 2024-05-03T18:47:24.130 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.130 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.130 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.130 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:24.131 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.131 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.131 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.131 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-03T18:47:24.131 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.131 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.131 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.131 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.131 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-05-03T18:47:24.131 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-05-03T18:47:24.131 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-03T18:47:24.131 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.131 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.131 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-05-03T18:47:24.131 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.131 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.132 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.132 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to wake up and check device health", 2024-05-03T18:47:24.132 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.132 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.132 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.132 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.132 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:24.133 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:24.133 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.133 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:24.133 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.133 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:24.133 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.133 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.133 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.133 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.133 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.133 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.133 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.133 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.134 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_threshold": { 2024-05-03T18:47:24.134 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_threshold", 2024-05-03T18:47:24.134 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-03T18:47:24.134 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.134 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.134 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "7257600", 2024-05-03T18:47:24.134 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.134 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.134 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.135 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-03T18:47:24.135 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.135 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.135 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.135 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.135 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.136 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.136 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:24.136 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "diskprediction_local", 2024-05-03T18:47:24.136 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:24.136 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:24.136 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:24.136 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:24.137 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:24.137 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.137 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.137 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.137 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.137 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.137 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.137 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.137 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.137 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.138 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.138 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.138 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.138 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.138 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.138 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.138 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.139 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.139 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.139 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.139 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:24.139 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:24.139 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.139 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.139 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.139 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.139 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.140 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.140 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.140 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.140 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.140 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.140 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.140 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.140 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:24.140 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:24.140 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.140 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.140 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.140 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:24.140 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.140 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.141 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.141 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.141 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.141 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.141 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.141 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.141 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.141 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.141 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.141 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.141 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.141 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.141 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.142 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:24.142 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:24.142 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.142 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.142 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.142 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.142 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.142 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.142 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.142 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.142 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.142 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.142 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.143 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.143 INFO:teuthology.orchestra.run.smithi033.stdout: "predict_interval": { 2024-05-03T18:47:24.143 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "predict_interval", 2024-05-03T18:47:24.143 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.143 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.143 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.143 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "86400", 2024-05-03T18:47:24.143 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.143 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.143 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.143 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.144 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.144 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.144 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.144 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.144 INFO:teuthology.orchestra.run.smithi033.stdout: "predictor_model": { 2024-05-03T18:47:24.144 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "predictor_model", 2024-05-03T18:47:24.144 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.144 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.144 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.144 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "prophetstor", 2024-05-03T18:47:24.144 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.144 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.144 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.144 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.144 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.145 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.145 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.145 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.145 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-05-03T18:47:24.145 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-05-03T18:47:24.145 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.145 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.145 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.145 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-05-03T18:47:24.145 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.145 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.145 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.145 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.145 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.145 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.146 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.146 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.146 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:24.146 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:24.146 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.146 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:24.146 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.146 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:24.146 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.146 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.146 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.146 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.146 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.147 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.147 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.147 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.147 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.147 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.147 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:24.147 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "influx", 2024-05-03T18:47:24.147 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": false, 2024-05-03T18:47:24.147 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "influxdb python module not found", 2024-05-03T18:47:24.147 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:24.147 INFO:teuthology.orchestra.run.smithi033.stdout: "batch_size": { 2024-05-03T18:47:24.147 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "batch_size", 2024-05-03T18:47:24.147 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.147 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.147 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.147 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5000", 2024-05-03T18:47:24.148 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.148 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.148 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.148 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-03T18:47:24.148 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.148 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.148 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.148 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.148 INFO:teuthology.orchestra.run.smithi033.stdout: "database": { 2024-05-03T18:47:24.148 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "database", 2024-05-03T18:47:24.148 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.148 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.148 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.148 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "ceph", 2024-05-03T18:47:24.148 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.148 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.149 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.149 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-03T18:47:24.149 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.149 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.149 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.149 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.149 INFO:teuthology.orchestra.run.smithi033.stdout: "hostname": { 2024-05-03T18:47:24.149 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "hostname", 2024-05-03T18:47:24.149 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.149 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.149 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.149 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.149 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.149 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.150 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.150 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "InfluxDB server hostname", 2024-05-03T18:47:24.150 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.150 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.150 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.150 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.150 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-05-03T18:47:24.150 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-05-03T18:47:24.150 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-03T18:47:24.150 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.150 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.150 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "30", 2024-05-03T18:47:24.150 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "5", 2024-05-03T18:47:24.150 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.151 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.151 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-03T18:47:24.151 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.151 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.151 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.151 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.151 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:24.151 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:24.151 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.151 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.151 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.151 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.151 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.151 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.151 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.152 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.152 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.152 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.152 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.152 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.152 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.152 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.152 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.152 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.152 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.152 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.152 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.152 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:24.152 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:24.152 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.152 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.153 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.153 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.153 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.153 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.153 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.153 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.153 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.153 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.153 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.153 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.153 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:24.153 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:24.153 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.153 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.154 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.154 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:24.154 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.154 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.154 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.154 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.154 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.154 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.154 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.154 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.154 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.154 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.154 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.154 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.155 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.155 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.155 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.155 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:24.155 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:24.155 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.155 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.155 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.155 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.155 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.155 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.155 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.155 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.155 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.155 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.156 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.156 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.156 INFO:teuthology.orchestra.run.smithi033.stdout: "password": { 2024-05-03T18:47:24.156 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "password", 2024-05-03T18:47:24.156 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.156 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.156 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.156 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.156 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.156 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.156 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.156 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "password of InfluxDB server user", 2024-05-03T18:47:24.156 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.156 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.156 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.157 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.157 INFO:teuthology.orchestra.run.smithi033.stdout: "port": { 2024-05-03T18:47:24.157 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "port", 2024-05-03T18:47:24.157 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.157 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.157 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.157 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8086", 2024-05-03T18:47:24.157 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.157 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.157 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.157 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "InfluxDB server port", 2024-05-03T18:47:24.157 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.157 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.157 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.157 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.157 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:24.158 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:24.158 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.158 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:24.158 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.158 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:24.158 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.158 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.158 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.158 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.158 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.158 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.158 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.158 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.159 INFO:teuthology.orchestra.run.smithi033.stdout: "ssl": { 2024-05-03T18:47:24.159 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ssl", 2024-05-03T18:47:24.159 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.159 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.159 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.159 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "false", 2024-05-03T18:47:24.159 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.159 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.159 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.159 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-03T18:47:24.159 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.159 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.159 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.159 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.160 INFO:teuthology.orchestra.run.smithi033.stdout: "threads": { 2024-05-03T18:47:24.160 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "threads", 2024-05-03T18:47:24.160 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.160 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.160 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.160 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5", 2024-05-03T18:47:24.160 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1", 2024-05-03T18:47:24.160 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "32", 2024-05-03T18:47:24.160 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.160 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-03T18:47:24.160 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.160 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.160 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.160 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.161 INFO:teuthology.orchestra.run.smithi033.stdout: "username": { 2024-05-03T18:47:24.161 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "username", 2024-05-03T18:47:24.161 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.161 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.161 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.161 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.161 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.161 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.161 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.161 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "username of InfluxDB server user", 2024-05-03T18:47:24.162 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.162 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.162 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.162 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.162 INFO:teuthology.orchestra.run.smithi033.stdout: "verify_ssl": { 2024-05-03T18:47:24.162 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "verify_ssl", 2024-05-03T18:47:24.162 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.162 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.162 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.163 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "true", 2024-05-03T18:47:24.163 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.163 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.163 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.163 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-03T18:47:24.163 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.163 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.163 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.163 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.163 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.163 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.163 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:24.163 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "insights", 2024-05-03T18:47:24.164 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:24.164 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:24.164 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:24.164 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:24.164 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:24.164 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.164 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.164 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.164 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.164 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.164 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.164 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.164 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.164 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.165 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.165 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.165 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.165 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.165 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.165 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.165 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.165 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.165 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.165 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.165 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:24.165 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:24.165 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.165 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.165 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.166 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.166 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.166 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.166 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.166 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.166 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.166 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.166 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.166 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.166 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:24.166 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:24.166 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.166 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.167 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.167 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:24.167 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.167 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.167 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.167 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.167 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.167 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.167 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.167 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.167 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.167 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.167 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.167 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.167 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.167 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.167 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.168 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:24.168 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:24.168 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.168 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.168 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.168 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.168 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.168 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.168 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.168 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.168 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.168 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.168 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.168 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.168 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:24.168 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:24.169 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.169 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:24.169 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.169 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:24.169 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.169 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.169 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.169 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.169 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.169 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.169 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.169 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.169 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.169 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.169 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:24.170 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "iostat", 2024-05-03T18:47:24.170 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:24.170 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:24.170 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:24.170 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:24.170 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:24.170 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.170 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.170 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.170 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.170 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.171 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.171 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.171 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.171 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.171 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.171 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.171 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.171 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.171 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.171 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.171 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.171 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.171 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.171 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.171 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:24.171 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:24.172 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.172 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.172 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.172 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.172 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.172 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.172 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.172 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.172 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.172 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.172 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.172 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.172 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:24.172 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:24.172 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.172 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.173 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.173 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:24.173 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.173 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.173 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.173 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.173 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.173 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.173 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.173 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.173 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.173 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.173 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.173 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.173 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.173 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.173 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.174 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:24.174 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:24.174 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.174 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.174 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.174 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.174 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.174 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.174 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.174 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.174 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.174 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.174 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.174 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.174 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:24.175 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:24.175 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.175 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:24.175 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.175 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:24.175 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.175 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.175 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.175 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.175 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.175 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.175 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.175 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.175 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.175 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.175 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:24.176 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "k8sevents", 2024-05-03T18:47:24.176 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:24.176 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:24.176 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:24.176 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_event_retention_days": { 2024-05-03T18:47:24.176 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ceph_event_retention_days", 2024-05-03T18:47:24.176 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.176 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.176 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.176 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "7", 2024-05-03T18:47:24.176 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.176 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.176 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.176 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-03T18:47:24.176 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.176 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.176 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.177 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.177 INFO:teuthology.orchestra.run.smithi033.stdout: "config_check_secs": { 2024-05-03T18:47:24.177 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "config_check_secs", 2024-05-03T18:47:24.177 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.177 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.177 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.177 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-05-03T18:47:24.177 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "10", 2024-05-03T18:47:24.177 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.177 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.177 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-03T18:47:24.177 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.177 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.177 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.177 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.177 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:24.178 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:24.178 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.178 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.178 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.178 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.178 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.178 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.178 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.178 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.178 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.178 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.178 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.178 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.178 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.179 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.179 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.179 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.179 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.179 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.179 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.179 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:24.179 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:24.179 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.179 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.179 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.179 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.179 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.179 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.179 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.179 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.179 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.180 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.180 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.180 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.180 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:24.180 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:24.180 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.180 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.180 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.180 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:24.180 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.180 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.180 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.180 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.180 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.180 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.180 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.181 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.181 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.181 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.181 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.181 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.181 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.181 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.181 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.181 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:24.181 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:24.181 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.181 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.181 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.181 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.181 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.181 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.182 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.182 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.182 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.182 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.182 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.182 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.182 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:24.182 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:24.182 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.182 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:24.182 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.182 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:24.182 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.182 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.182 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.183 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.183 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.183 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.183 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.183 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.183 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.183 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.183 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:24.183 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "localpool", 2024-05-03T18:47:24.183 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:24.183 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:24.183 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:24.183 INFO:teuthology.orchestra.run.smithi033.stdout: "failure_domain": { 2024-05-03T18:47:24.183 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "failure_domain", 2024-05-03T18:47:24.183 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.183 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.184 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.184 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "host", 2024-05-03T18:47:24.184 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.184 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.184 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.184 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "failure domain for any created local pool", 2024-05-03T18:47:24.184 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-03T18:47:24.184 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.184 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.184 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.184 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:24.184 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:24.184 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.184 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.184 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.184 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.184 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.185 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.185 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.185 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.185 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.185 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.185 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.185 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.185 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.185 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.185 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.185 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.185 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.185 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.185 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.185 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:24.185 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:24.186 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.186 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.186 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.186 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.186 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.186 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.186 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.186 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.186 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.186 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.186 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.186 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.186 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:24.186 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:24.186 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.187 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.187 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.187 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:24.187 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.187 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.187 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.187 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.187 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.187 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.187 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.187 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.187 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.187 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.187 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.187 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.187 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.188 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.188 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.188 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:24.188 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:24.188 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.188 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.188 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.188 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.188 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.188 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.188 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.188 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.188 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.188 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.188 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.188 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.189 INFO:teuthology.orchestra.run.smithi033.stdout: "min_size": { 2024-05-03T18:47:24.189 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "min_size", 2024-05-03T18:47:24.189 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.189 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.189 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.189 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.189 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.189 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.189 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.189 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "default min_size for any created local pool", 2024-05-03T18:47:24.189 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-03T18:47:24.189 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.189 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.189 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.189 INFO:teuthology.orchestra.run.smithi033.stdout: "num_rep": { 2024-05-03T18:47:24.189 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "num_rep", 2024-05-03T18:47:24.190 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.190 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.190 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.190 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "3", 2024-05-03T18:47:24.190 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.190 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.190 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.190 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "default replica count for any created local pool", 2024-05-03T18:47:24.190 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.190 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.190 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.190 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.190 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_num": { 2024-05-03T18:47:24.190 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "pg_num", 2024-05-03T18:47:24.191 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.191 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.191 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.191 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "128", 2024-05-03T18:47:24.191 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.191 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.191 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.191 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "default pg_num for any created local pool", 2024-05-03T18:47:24.191 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.191 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.191 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.191 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.191 INFO:teuthology.orchestra.run.smithi033.stdout: "prefix": { 2024-05-03T18:47:24.191 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "prefix", 2024-05-03T18:47:24.191 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.191 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.192 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.192 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.192 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.192 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.192 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.192 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "name prefix for any created local pool", 2024-05-03T18:47:24.192 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.192 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.192 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.192 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.192 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:24.192 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:24.192 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.192 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:24.192 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.192 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:24.193 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.193 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.193 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.193 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.193 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.193 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.193 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.193 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.193 INFO:teuthology.orchestra.run.smithi033.stdout: "subtree": { 2024-05-03T18:47:24.193 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "subtree", 2024-05-03T18:47:24.193 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.193 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.193 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.193 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "rack", 2024-05-03T18:47:24.193 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.193 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.193 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.194 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-03T18:47:24.194 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-03T18:47:24.194 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.194 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.194 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.194 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.194 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.194 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:24.194 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mds_autoscaler", 2024-05-03T18:47:24.194 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:24.194 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:24.194 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:24.194 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:24.194 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:24.194 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.194 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.195 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.195 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.195 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.195 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.195 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.195 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.195 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.195 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.195 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.195 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.195 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.195 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.195 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.195 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.195 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.195 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.195 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.196 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:24.196 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:24.196 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.196 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.196 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.196 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.196 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.196 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.196 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.196 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.196 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.196 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.196 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.196 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.197 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:24.197 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:24.197 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.197 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.197 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.197 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:24.197 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.197 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.197 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.197 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.197 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.197 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.197 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.197 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.197 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.197 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.198 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.198 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.198 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.198 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.198 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.198 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:24.198 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:24.198 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.198 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.198 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.198 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.198 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.198 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.198 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.199 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.199 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.199 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.199 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.199 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.199 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:24.199 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:24.199 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.199 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:24.199 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.199 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:24.199 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.199 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.199 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.199 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.200 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.200 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.200 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.200 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.200 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.200 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.200 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:24.200 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mirroring", 2024-05-03T18:47:24.200 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:24.200 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:24.200 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:24.200 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:24.200 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:24.200 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.200 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.201 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.201 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.201 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.201 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.201 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.201 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.201 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.201 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.201 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.201 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.201 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.201 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.201 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.201 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.201 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.202 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.202 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.202 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:24.202 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:24.202 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.202 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.202 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.202 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.202 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.202 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.202 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.202 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.202 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.202 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.202 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.203 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.203 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:24.203 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:24.203 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.203 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.203 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.203 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:24.203 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.203 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.203 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.203 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.203 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.203 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.203 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.203 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.203 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.203 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.204 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.204 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.204 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.204 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.204 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.204 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:24.204 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:24.204 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.204 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.204 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.204 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.204 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.204 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.204 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.204 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.205 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.205 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.205 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.205 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.205 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:24.205 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:24.205 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.205 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:24.205 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.205 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:24.205 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.205 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.205 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.205 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.205 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.205 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.206 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.206 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.206 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.206 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.206 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:24.206 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "nfs", 2024-05-03T18:47:24.206 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:24.206 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:24.206 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:24.206 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:24.206 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:24.206 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.206 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.206 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.206 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.207 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.207 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.207 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.207 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.207 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.207 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.207 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.207 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.207 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.207 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.207 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.207 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.207 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.207 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.207 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.207 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:24.208 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:24.208 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.208 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.208 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.208 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.208 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.208 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.208 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.208 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.208 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.208 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.208 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.208 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.208 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:24.208 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:24.209 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.209 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.209 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.209 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:24.209 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.209 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.209 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.209 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.209 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.209 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.209 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.209 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.209 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.209 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.209 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.209 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.210 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.210 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.210 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.210 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:24.210 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:24.210 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.210 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.210 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.210 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.210 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.210 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.210 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.210 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.210 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.210 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.211 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.211 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.211 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:24.211 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:24.211 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.211 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:24.211 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.211 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:24.211 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.211 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.211 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.211 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.211 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.211 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.211 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.211 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.212 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.212 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.212 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:24.212 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "orchestrator", 2024-05-03T18:47:24.212 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:24.212 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:24.212 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:24.212 INFO:teuthology.orchestra.run.smithi033.stdout: "fail_fs": { 2024-05-03T18:47:24.212 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "fail_fs", 2024-05-03T18:47:24.212 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.212 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.212 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.212 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.212 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.212 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.212 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.213 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-03T18:47:24.213 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.213 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.213 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.213 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.213 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:24.213 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:24.213 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.213 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.213 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.213 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.213 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.213 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.213 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.213 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.213 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.214 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.214 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.214 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.214 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.214 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.214 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.214 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.214 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.214 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.214 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.214 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:24.214 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:24.214 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.214 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.214 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.214 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.215 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.215 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.215 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.215 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.215 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.215 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.215 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.215 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.215 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:24.215 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:24.215 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.215 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.215 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.215 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:24.215 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.215 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.216 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.216 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.216 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.216 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.216 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.216 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.216 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.216 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.216 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.216 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.216 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.216 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.216 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.216 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:24.216 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:24.217 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.217 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.217 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.217 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.217 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.217 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.217 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.217 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.217 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.217 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.217 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.217 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.217 INFO:teuthology.orchestra.run.smithi033.stdout: "orchestrator": { 2024-05-03T18:47:24.217 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "orchestrator", 2024-05-03T18:47:24.217 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.218 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.218 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.218 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.218 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.218 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.218 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.218 INFO:teuthology.orchestra.run.smithi033.stdout: "cephadm", 2024-05-03T18:47:24.218 INFO:teuthology.orchestra.run.smithi033.stdout: "rook", 2024-05-03T18:47:24.218 INFO:teuthology.orchestra.run.smithi033.stdout: "test_orchestrator" 2024-05-03T18:47:24.218 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.218 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Orchestrator backend", 2024-05-03T18:47:24.219 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.219 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.219 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.219 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.219 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:24.219 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:24.219 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.219 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:24.219 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.219 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:24.219 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.219 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.219 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.219 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.220 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.220 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.220 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.220 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.220 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.220 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.220 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:24.220 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "osd_perf_query", 2024-05-03T18:47:24.220 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:24.220 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:24.220 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:24.220 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:24.220 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:24.221 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.221 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.221 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.221 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.221 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.221 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.221 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.221 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.221 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.221 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.221 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.221 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.221 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.222 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.222 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.222 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.222 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.222 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.222 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.222 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:24.222 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:24.222 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.222 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.222 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.222 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.222 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.222 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.223 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.223 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.223 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.223 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.223 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.223 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.223 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:24.223 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:24.223 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.223 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.223 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.224 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:24.224 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.224 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.224 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.224 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.224 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.224 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.224 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.224 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.224 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.224 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.225 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.225 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.225 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.225 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.225 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.225 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:24.225 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:24.225 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.225 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.225 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.225 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.225 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.225 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.225 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.225 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.225 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.226 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.226 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.226 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.226 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:24.226 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:24.227 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.227 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:24.227 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.227 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:24.227 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.227 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.228 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.228 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.228 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.228 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.228 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.228 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.229 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.229 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.229 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:24.229 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "osd_support", 2024-05-03T18:47:24.229 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:24.229 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:24.229 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:24.229 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:24.229 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:24.229 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.229 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.229 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.230 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.230 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.230 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.230 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.230 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.230 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.231 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.231 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.231 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.231 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.231 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.231 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.231 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.231 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.231 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.231 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.231 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:24.231 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:24.231 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.231 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.232 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.232 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.232 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.232 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.232 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.232 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.232 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.232 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.232 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.232 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.232 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:24.232 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:24.232 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.232 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.233 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.233 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:24.233 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.233 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.233 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.233 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.233 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.233 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.233 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.233 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.233 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.233 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.233 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.233 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.233 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.233 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.234 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.234 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:24.234 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:24.234 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.234 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.234 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.234 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.234 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.234 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.234 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.234 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.234 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.234 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.234 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.234 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.235 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:24.235 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:24.235 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.235 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:24.235 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.235 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:24.235 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.235 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.235 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.235 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.235 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.235 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.235 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.235 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.235 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.236 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.236 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:24.236 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "pg_autoscaler", 2024-05-03T18:47:24.236 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:24.236 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:24.236 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:24.236 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:24.236 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:24.236 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.236 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.236 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.236 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.236 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.236 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.236 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.236 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.237 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.237 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.237 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.237 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.237 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.237 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.237 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.237 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.237 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.237 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.237 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.237 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:24.237 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:24.237 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.237 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.237 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.237 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.238 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.238 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.238 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.238 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.238 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.238 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.238 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.238 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.238 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:24.238 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:24.238 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.238 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.238 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.238 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:24.239 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.239 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.239 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.239 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.239 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.239 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.239 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.239 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.239 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.239 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.239 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.239 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.239 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.239 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.239 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.240 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:24.240 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:24.240 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.240 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.240 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.240 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.240 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.240 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.240 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.240 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.240 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.240 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.240 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.240 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.240 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-05-03T18:47:24.241 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-05-03T18:47:24.241 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-03T18:47:24.241 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.241 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.241 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-05-03T18:47:24.241 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.241 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.241 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.241 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.241 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.241 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.241 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.241 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.241 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:24.241 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:24.242 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.242 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:24.242 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.242 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:24.242 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.242 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.242 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.242 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.242 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.242 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.242 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.242 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.243 INFO:teuthology.orchestra.run.smithi033.stdout: "threshold": { 2024-05-03T18:47:24.243 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "threshold", 2024-05-03T18:47:24.243 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-05-03T18:47:24.243 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.243 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.243 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "3.0", 2024-05-03T18:47:24.243 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1.0", 2024-05-03T18:47:24.243 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.243 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.243 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "scaling threshold", 2024-05-03T18:47:24.243 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-03T18:47:24.243 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.243 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.243 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.244 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.244 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.244 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:24.244 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "progress", 2024-05-03T18:47:24.245 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:24.245 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:24.245 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:24.246 INFO:teuthology.orchestra.run.smithi033.stdout: "allow_pg_recovery_event": { 2024-05-03T18:47:24.246 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "allow_pg_recovery_event", 2024-05-03T18:47:24.246 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.246 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.246 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.246 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.246 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.246 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.246 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.246 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-03T18:47:24.246 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.246 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.246 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.246 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.247 INFO:teuthology.orchestra.run.smithi033.stdout: "enabled": { 2024-05-03T18:47:24.247 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "enabled", 2024-05-03T18:47:24.247 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.247 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.247 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.247 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:24.247 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.247 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.247 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.247 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.247 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.247 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.247 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.247 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.247 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:24.248 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:24.248 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.248 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.248 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.248 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.248 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.248 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.248 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.248 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.248 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.248 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.248 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.248 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.248 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.248 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.248 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.249 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.249 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.249 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.249 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.249 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:24.249 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:24.249 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.249 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.249 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.249 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.249 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.249 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.249 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.249 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.250 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.250 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.250 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.250 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.250 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:24.250 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:24.250 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.250 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.250 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.250 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:24.250 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.250 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.250 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.250 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.251 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.251 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.251 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.251 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.251 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.251 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.251 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.251 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.251 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.251 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.251 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.251 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:24.251 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:24.251 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.251 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.251 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.252 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.252 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.252 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.252 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.252 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.252 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.252 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.252 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.252 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.252 INFO:teuthology.orchestra.run.smithi033.stdout: "max_completed_events": { 2024-05-03T18:47:24.252 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_completed_events", 2024-05-03T18:47:24.252 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.252 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.252 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.253 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "50", 2024-05-03T18:47:24.253 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.253 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.253 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.253 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "number of past completed events to remember", 2024-05-03T18:47:24.253 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.253 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.253 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.253 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.253 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-05-03T18:47:24.253 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-05-03T18:47:24.253 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-03T18:47:24.253 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.253 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.254 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5", 2024-05-03T18:47:24.254 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.254 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.254 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.254 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how long the module is going to sleep", 2024-05-03T18:47:24.254 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.254 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.254 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.254 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.254 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:24.254 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:24.254 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.255 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:24.255 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.255 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:24.255 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.255 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.255 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.255 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.255 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.255 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.255 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.255 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.255 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.255 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.255 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:24.255 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "prometheus", 2024-05-03T18:47:24.256 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:24.256 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:24.256 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:24.256 INFO:teuthology.orchestra.run.smithi033.stdout: "cache": { 2024-05-03T18:47:24.256 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cache", 2024-05-03T18:47:24.256 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.256 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.256 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.256 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:24.256 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.256 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.256 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.256 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.256 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.257 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.257 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.257 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.257 INFO:teuthology.orchestra.run.smithi033.stdout: "exclude_perf_counters": { 2024-05-03T18:47:24.257 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "exclude_perf_counters", 2024-05-03T18:47:24.257 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.257 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.257 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.257 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:24.257 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.257 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.257 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.257 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-03T18:47:24.257 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-03T18:47:24.258 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.258 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.258 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.258 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:24.258 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:24.258 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.258 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.258 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.258 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.258 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.258 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.258 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.258 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.258 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.259 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.259 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.259 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.259 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.259 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.259 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.259 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.259 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.259 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.259 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.259 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:24.259 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:24.259 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.260 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.260 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.260 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.260 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.260 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.260 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.260 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.260 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.260 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.260 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.260 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.260 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:24.260 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:24.260 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.261 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.261 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.261 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:24.261 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.261 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.261 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.261 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.261 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.261 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.261 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.262 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.262 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.262 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.262 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.262 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.262 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.262 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.262 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.262 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:24.262 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:24.262 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.263 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.263 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.263 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.263 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.263 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.263 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.263 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.263 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.263 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.263 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.264 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.264 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_stats_pools": { 2024-05-03T18:47:24.264 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rbd_stats_pools", 2024-05-03T18:47:24.264 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.264 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.264 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.264 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.264 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.264 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.264 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.264 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.265 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.265 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.265 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.265 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.265 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-03T18:47:24.265 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-03T18:47:24.265 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.265 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.265 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.265 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "300", 2024-05-03T18:47:24.265 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.265 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.266 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.266 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.266 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.266 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.266 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.266 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.266 INFO:teuthology.orchestra.run.smithi033.stdout: "scrape_interval": { 2024-05-03T18:47:24.266 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "scrape_interval", 2024-05-03T18:47:24.266 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-05-03T18:47:24.266 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.266 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.266 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "15.0", 2024-05-03T18:47:24.266 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.266 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.267 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.267 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.267 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.267 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.267 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.267 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.267 INFO:teuthology.orchestra.run.smithi033.stdout: "server_addr": { 2024-05-03T18:47:24.267 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_addr", 2024-05-03T18:47:24.267 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.267 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.267 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.267 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "::", 2024-05-03T18:47:24.267 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.267 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.267 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.267 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-03T18:47:24.268 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.268 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.268 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.268 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.268 INFO:teuthology.orchestra.run.smithi033.stdout: "server_port": { 2024-05-03T18:47:24.268 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_port", 2024-05-03T18:47:24.268 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.268 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.268 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.268 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "9283", 2024-05-03T18:47:24.268 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.268 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.269 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.269 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-03T18:47:24.269 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.269 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.269 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.269 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.269 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:24.269 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:24.269 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.269 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:24.269 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.269 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:24.269 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.270 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.270 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.270 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.270 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.270 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.270 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.270 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.270 INFO:teuthology.orchestra.run.smithi033.stdout: "stale_cache_strategy": { 2024-05-03T18:47:24.270 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "stale_cache_strategy", 2024-05-03T18:47:24.270 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.270 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.270 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.270 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "log", 2024-05-03T18:47:24.271 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.271 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.271 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.271 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.271 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.271 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.271 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.271 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.271 INFO:teuthology.orchestra.run.smithi033.stdout: "standby_behaviour": { 2024-05-03T18:47:24.271 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "standby_behaviour", 2024-05-03T18:47:24.271 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.271 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.271 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.271 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "default", 2024-05-03T18:47:24.272 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.272 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.272 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.272 INFO:teuthology.orchestra.run.smithi033.stdout: "default", 2024-05-03T18:47:24.272 INFO:teuthology.orchestra.run.smithi033.stdout: "error" 2024-05-03T18:47:24.272 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.272 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.272 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.272 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.272 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.272 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.272 INFO:teuthology.orchestra.run.smithi033.stdout: "standby_error_status_code": { 2024-05-03T18:47:24.272 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "standby_error_status_code", 2024-05-03T18:47:24.272 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.272 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.272 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.273 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "500", 2024-05-03T18:47:24.273 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "400", 2024-05-03T18:47:24.273 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "599", 2024-05-03T18:47:24.273 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.273 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.273 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.273 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.273 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.273 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.273 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.273 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.273 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:24.273 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rbd_support", 2024-05-03T18:47:24.273 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:24.273 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:24.274 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:24.274 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:24.274 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:24.274 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.274 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.274 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.274 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.274 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.274 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.274 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.274 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.274 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.274 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.275 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.275 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.275 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.275 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.275 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.275 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.275 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.275 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.275 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.275 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:24.275 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:24.275 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.275 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.275 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.275 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.275 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.275 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.276 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.276 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.276 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.276 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.276 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.276 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.276 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:24.276 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:24.276 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.276 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.276 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.276 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:24.276 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.276 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.276 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.277 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.277 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.277 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.277 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.277 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.277 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.277 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.277 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.277 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.277 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.277 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.277 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.277 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:24.277 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:24.277 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.278 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.278 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.278 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.278 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.278 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.278 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.278 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.278 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.278 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.278 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.278 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.278 INFO:teuthology.orchestra.run.smithi033.stdout: "max_concurrent_snap_create": { 2024-05-03T18:47:24.278 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_concurrent_snap_create", 2024-05-03T18:47:24.279 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.279 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.279 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.279 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-05-03T18:47:24.279 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.279 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.279 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.279 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.279 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.279 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.279 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.279 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.279 INFO:teuthology.orchestra.run.smithi033.stdout: "mirror_snapshot_schedule": { 2024-05-03T18:47:24.279 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mirror_snapshot_schedule", 2024-05-03T18:47:24.279 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.279 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.279 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.280 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.280 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.280 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.280 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.280 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.280 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.280 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.280 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.280 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.280 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:24.280 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:24.280 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.280 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:24.280 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.280 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:24.280 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.281 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.281 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.281 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.281 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.281 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.281 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.281 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.281 INFO:teuthology.orchestra.run.smithi033.stdout: "trash_purge_schedule": { 2024-05-03T18:47:24.281 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "trash_purge_schedule", 2024-05-03T18:47:24.281 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.281 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.281 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.281 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.281 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.282 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.282 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.282 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.282 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.282 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.282 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.282 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.282 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.282 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.283 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:24.283 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "restful", 2024-05-03T18:47:24.283 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:24.283 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:24.283 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:24.283 INFO:teuthology.orchestra.run.smithi033.stdout: "enable_auth": { 2024-05-03T18:47:24.283 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "enable_auth", 2024-05-03T18:47:24.283 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.284 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.284 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.284 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:24.284 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.284 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.284 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.284 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.285 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.285 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.285 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.285 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.285 INFO:teuthology.orchestra.run.smithi033.stdout: "key_file": { 2024-05-03T18:47:24.285 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "key_file", 2024-05-03T18:47:24.285 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.286 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.286 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.286 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.286 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.286 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.286 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.287 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.287 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.287 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.287 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.287 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.287 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:24.287 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:24.288 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.288 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.288 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.288 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.288 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.288 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.289 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.289 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.289 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.289 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.289 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.289 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.290 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.290 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.290 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.290 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.290 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.290 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.291 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.291 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:24.291 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:24.291 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.291 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.291 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.291 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.291 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.292 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.292 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.292 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.292 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.292 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.292 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.292 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.292 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:24.293 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:24.293 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.293 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.293 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.293 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:24.293 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.293 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.294 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.294 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.294 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.294 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.294 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.294 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.295 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.295 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.295 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.295 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.295 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.295 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.295 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.296 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:24.296 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:24.296 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.296 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.296 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.296 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.296 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.297 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.297 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.297 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.297 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.297 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.297 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.298 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.298 INFO:teuthology.orchestra.run.smithi033.stdout: "server_addr": { 2024-05-03T18:47:24.298 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_addr", 2024-05-03T18:47:24.298 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.298 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.298 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.298 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.299 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.299 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.299 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.299 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.299 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.299 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.300 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.300 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.300 INFO:teuthology.orchestra.run.smithi033.stdout: "server_port": { 2024-05-03T18:47:24.300 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_port", 2024-05-03T18:47:24.300 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.300 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.300 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.301 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.301 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.301 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.301 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.301 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.301 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.301 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.302 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.302 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.302 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:24.302 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:24.302 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.302 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:24.303 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.303 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:24.303 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.303 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.303 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.303 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.303 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.304 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.304 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.304 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.304 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.304 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.304 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:24.304 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rgw", 2024-05-03T18:47:24.304 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:24.304 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:24.305 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:24.305 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:24.305 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:24.305 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.305 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.305 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.305 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.305 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.305 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.306 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.306 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.306 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.306 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.306 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.306 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.306 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.307 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.307 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.307 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.307 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.307 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.307 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.307 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:24.307 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:24.307 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.307 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.307 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.307 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.308 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.308 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.308 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.308 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.308 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.308 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.308 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.308 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.308 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:24.308 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:24.308 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.308 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.308 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.308 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:24.308 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.309 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.309 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.309 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.309 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.309 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.309 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.309 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.309 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.309 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.309 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.309 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.309 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.309 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.309 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.309 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:24.310 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:24.310 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.310 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.310 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.310 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.310 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.310 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.310 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.310 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.310 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.310 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.310 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.311 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.311 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:24.311 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:24.311 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.311 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:24.311 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.311 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:24.311 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.311 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.311 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.311 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.311 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.311 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.311 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.311 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.312 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.312 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.312 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:24.312 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rook", 2024-05-03T18:47:24.312 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:24.312 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:24.312 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:24.312 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:24.312 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:24.312 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.312 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.312 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.312 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.313 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.313 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.313 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.313 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.313 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.313 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.313 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.313 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.313 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.313 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.313 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.313 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.313 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.313 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.313 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.314 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:24.314 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:24.314 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.314 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.314 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.314 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.314 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.314 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.314 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.314 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.314 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.314 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.314 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.315 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.315 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:24.315 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:24.315 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.315 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.315 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.315 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:24.315 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.316 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.316 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.316 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.316 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.316 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.316 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.316 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.316 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.316 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.316 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.316 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.316 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.316 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.316 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.317 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:24.317 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:24.317 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.317 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.317 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.317 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.317 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.317 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.318 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.318 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.318 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.318 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.318 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.318 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.319 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:24.319 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:24.319 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.319 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:24.319 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.319 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:24.320 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.320 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.320 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.320 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.320 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.320 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.320 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.320 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.320 INFO:teuthology.orchestra.run.smithi033.stdout: "storage_class": { 2024-05-03T18:47:24.321 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "storage_class", 2024-05-03T18:47:24.321 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.321 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.321 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.321 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "local", 2024-05-03T18:47:24.321 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.321 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.321 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.321 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-03T18:47:24.322 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.322 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.322 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.322 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.322 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.322 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.322 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:24.322 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "selftest", 2024-05-03T18:47:24.322 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:24.323 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:24.323 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:24.323 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:24.323 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:24.323 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.323 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.323 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.323 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.323 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.323 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.323 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.323 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.323 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.323 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.323 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.324 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.324 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.324 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.324 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.324 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.324 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.324 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.324 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.324 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:24.324 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:24.324 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.324 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.324 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.324 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.324 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.324 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.325 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.325 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.325 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.325 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.325 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.325 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.325 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:24.325 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:24.325 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.325 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.325 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.325 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:24.325 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.325 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.326 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.326 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.326 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.326 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.326 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.326 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.326 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.326 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.326 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.326 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.326 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.326 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.326 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.326 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:24.326 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:24.327 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.327 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.327 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.327 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.327 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.327 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.327 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.327 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.327 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.327 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.327 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.327 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.327 INFO:teuthology.orchestra.run.smithi033.stdout: "roption1": { 2024-05-03T18:47:24.327 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "roption1", 2024-05-03T18:47:24.327 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.327 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.328 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.328 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.328 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.328 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.328 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.328 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.328 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.328 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.328 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.328 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.328 INFO:teuthology.orchestra.run.smithi033.stdout: "roption2": { 2024-05-03T18:47:24.328 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "roption2", 2024-05-03T18:47:24.328 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.328 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.328 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.328 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "xyz", 2024-05-03T18:47:24.329 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.329 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.329 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.329 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.329 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.329 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.329 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.329 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.329 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption1": { 2024-05-03T18:47:24.329 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption1", 2024-05-03T18:47:24.329 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.329 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.329 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.329 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.330 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.330 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.330 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.330 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.330 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.330 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.330 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.330 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.330 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption2": { 2024-05-03T18:47:24.330 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption2", 2024-05-03T18:47:24.330 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.330 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.331 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.331 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.331 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.331 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.331 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.331 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.331 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.331 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.331 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.331 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.331 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption3": { 2024-05-03T18:47:24.331 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption3", 2024-05-03T18:47:24.331 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-05-03T18:47:24.331 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.331 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.332 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.332 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.332 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.332 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.332 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.332 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.332 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.332 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.332 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.332 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption4": { 2024-05-03T18:47:24.332 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption4", 2024-05-03T18:47:24.332 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.332 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.332 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.333 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.333 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.333 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.333 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.333 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.333 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.333 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.333 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.333 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.333 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption5": { 2024-05-03T18:47:24.333 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption5", 2024-05-03T18:47:24.333 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.333 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.333 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.334 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.334 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.334 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.334 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.334 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.334 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.334 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.334 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.334 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.334 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption6": { 2024-05-03T18:47:24.334 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption6", 2024-05-03T18:47:24.334 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.334 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.335 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.335 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:24.335 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.335 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.335 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.335 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.335 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.335 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.335 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.335 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.335 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption7": { 2024-05-03T18:47:24.335 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption7", 2024-05-03T18:47:24.335 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.335 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.336 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.336 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.336 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1", 2024-05-03T18:47:24.336 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "42", 2024-05-03T18:47:24.336 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.336 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.336 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.336 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.336 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.336 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.336 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:24.336 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:24.336 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.336 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:24.336 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.337 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:24.337 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.337 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.337 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.337 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.337 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.337 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.337 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.337 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.337 INFO:teuthology.orchestra.run.smithi033.stdout: "testkey": { 2024-05-03T18:47:24.337 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "testkey", 2024-05-03T18:47:24.337 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.337 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.337 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.337 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.338 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.338 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.338 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.338 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.338 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.338 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.338 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.338 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.338 INFO:teuthology.orchestra.run.smithi033.stdout: "testlkey": { 2024-05-03T18:47:24.338 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "testlkey", 2024-05-03T18:47:24.338 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.338 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.339 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.339 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.339 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.339 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.339 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.339 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.339 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.339 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.339 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.339 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.339 INFO:teuthology.orchestra.run.smithi033.stdout: "testnewline": { 2024-05-03T18:47:24.339 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "testnewline", 2024-05-03T18:47:24.339 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.339 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.339 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.340 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.340 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.340 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.340 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.340 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.340 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.340 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.340 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.340 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.340 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.340 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.340 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:24.340 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smb", 2024-05-03T18:47:24.340 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:24.340 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:24.341 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:24.341 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:24.341 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:24.341 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.341 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.341 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.341 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.341 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.341 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.341 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.341 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.341 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.341 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.341 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.341 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.342 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.342 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.342 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.342 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.342 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.342 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.342 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.342 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:24.342 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:24.342 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.343 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.343 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.343 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.343 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.343 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.343 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.343 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.343 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.343 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.343 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.343 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.343 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:24.343 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:24.343 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.343 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.344 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.344 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:24.344 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.344 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.344 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.344 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.344 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.344 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.344 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.344 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.344 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.344 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.344 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.345 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.345 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.345 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.345 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.345 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:24.345 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:24.345 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.345 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.345 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.345 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.345 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.345 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.345 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.345 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.346 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.346 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.346 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.346 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.346 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:24.346 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:24.346 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.346 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:24.346 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.346 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:24.346 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.346 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.346 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.347 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.347 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.347 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.347 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.347 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.347 INFO:teuthology.orchestra.run.smithi033.stdout: "update_orchestration": { 2024-05-03T18:47:24.347 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "update_orchestration", 2024-05-03T18:47:24.347 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.347 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.347 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.347 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:24.347 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.347 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.347 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.348 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-03T18:47:24.348 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.348 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.348 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.348 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.348 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.348 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.348 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:24.348 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "snap_schedule", 2024-05-03T18:47:24.348 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:24.348 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:24.348 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:24.348 INFO:teuthology.orchestra.run.smithi033.stdout: "allow_m_granularity": { 2024-05-03T18:47:24.348 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "allow_m_granularity", 2024-05-03T18:47:24.348 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.348 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.349 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.349 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.349 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.349 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.349 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.349 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "allow minute scheduled snapshots", 2024-05-03T18:47:24.349 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.349 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.349 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.349 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.349 INFO:teuthology.orchestra.run.smithi033.stdout: "dump_on_update": { 2024-05-03T18:47:24.349 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "dump_on_update", 2024-05-03T18:47:24.349 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.349 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.349 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.350 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.350 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.350 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.350 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.350 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "dump database to debug log on update", 2024-05-03T18:47:24.350 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.350 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.350 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.350 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.350 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:24.350 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:24.350 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.350 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.350 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.351 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.351 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.351 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.351 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.351 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.351 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.351 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.351 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.351 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.351 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.351 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.351 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.351 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.351 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.351 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.352 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.352 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:24.352 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:24.352 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.352 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.352 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.352 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.352 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.352 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.352 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.352 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.352 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.352 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.352 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.353 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.353 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:24.353 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:24.353 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.353 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.353 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.353 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:24.353 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.353 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.353 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.353 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.353 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.353 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.353 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.353 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.354 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.354 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.354 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.354 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.354 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.354 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.354 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.354 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:24.354 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:24.354 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.354 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.354 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.354 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.354 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.355 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.355 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.355 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.355 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.355 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.355 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.355 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.355 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:24.355 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:24.355 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.355 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:24.355 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.355 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:24.355 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.355 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.356 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.356 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.356 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.356 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.356 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.356 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.356 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.356 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.356 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:24.356 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "stats", 2024-05-03T18:47:24.356 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:24.356 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:24.356 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:24.356 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:24.356 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:24.357 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.357 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.357 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.357 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.357 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.357 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.357 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.357 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.357 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.357 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.357 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.357 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.357 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.357 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.358 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.358 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.358 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.358 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.358 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.358 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:24.358 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:24.358 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.359 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.359 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.359 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.359 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.359 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.359 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.359 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.359 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.360 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.360 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.360 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.360 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:24.360 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:24.360 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.360 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.360 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.361 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:24.361 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.361 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.361 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.361 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.361 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.361 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.362 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.362 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.362 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.362 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.362 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.362 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.362 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.363 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.363 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.363 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:24.363 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:24.363 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.363 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.363 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.363 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.363 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.363 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.363 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.364 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.364 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.364 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.364 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.364 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.364 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:24.364 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:24.364 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.364 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:24.364 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.364 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:24.364 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.364 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.364 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.365 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.365 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.365 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.365 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.365 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.365 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.365 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.365 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:24.365 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "status", 2024-05-03T18:47:24.365 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:24.365 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:24.365 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:24.365 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:24.365 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:24.365 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.366 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.366 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.366 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.366 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.366 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.366 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.366 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.366 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.366 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.366 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.366 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.366 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.366 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.366 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.366 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.367 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.367 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.367 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.367 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:24.367 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:24.367 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.367 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.367 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.367 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.367 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.367 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.367 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.367 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.367 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.367 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.368 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.368 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.368 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:24.368 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:24.368 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.368 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.368 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.368 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:24.368 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.368 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.368 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.368 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.368 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.368 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.368 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.369 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.369 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.369 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.369 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.369 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.369 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.369 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.369 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.369 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:24.369 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:24.369 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.369 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.369 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.369 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.370 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.370 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.370 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.370 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.370 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.370 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.370 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.370 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.370 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:24.370 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:24.370 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.370 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:24.370 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.371 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:24.371 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.371 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.371 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.371 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.371 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.371 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.371 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.371 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.371 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.371 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.371 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:24.371 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "telegraf", 2024-05-03T18:47:24.371 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:24.372 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:24.372 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:24.372 INFO:teuthology.orchestra.run.smithi033.stdout: "address": { 2024-05-03T18:47:24.372 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "address", 2024-05-03T18:47:24.372 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.372 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.372 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.372 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-03T18:47:24.372 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.372 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.372 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.372 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.372 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.372 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.372 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.373 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.373 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-05-03T18:47:24.373 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-05-03T18:47:24.373 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-03T18:47:24.373 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.373 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.373 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "15", 2024-05-03T18:47:24.373 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.373 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.373 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.373 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.373 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.373 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.373 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.374 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.374 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:24.374 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:24.374 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.374 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.374 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.374 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.374 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.374 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.374 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.374 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.374 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.374 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.375 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.375 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.375 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.375 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.375 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.375 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.375 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.375 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.375 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.375 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:24.375 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:24.375 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.375 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.375 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.376 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.376 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.376 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.376 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.376 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.376 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.376 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.376 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.376 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.376 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:24.376 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:24.376 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.376 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.376 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.376 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:24.376 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.377 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.377 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.377 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.377 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.377 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.377 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.377 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.377 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.377 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.377 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.377 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.377 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.377 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.377 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.377 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:24.378 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:24.378 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.378 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.378 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.378 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.378 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.378 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.378 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.378 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.378 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.378 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.378 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.378 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.378 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:24.379 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:24.379 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.379 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:24.379 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.379 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:24.379 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.379 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.379 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.379 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.379 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.379 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.379 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.379 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.379 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.379 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.380 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:24.380 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "telemetry", 2024-05-03T18:47:24.380 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:24.380 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:24.380 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:24.380 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_basic": { 2024-05-03T18:47:24.380 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_basic", 2024-05-03T18:47:24.380 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.380 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.380 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.380 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:24.380 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.380 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.380 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.380 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-03T18:47:24.380 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.381 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.381 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.381 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.381 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_crash": { 2024-05-03T18:47:24.381 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_crash", 2024-05-03T18:47:24.381 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.381 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.381 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.381 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:24.381 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.381 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.381 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.381 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-03T18:47:24.381 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.381 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.381 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.382 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.382 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_device": { 2024-05-03T18:47:24.382 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_device", 2024-05-03T18:47:24.382 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.382 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.382 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.382 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:24.382 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.382 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.382 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.382 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-03T18:47:24.382 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.382 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.382 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.382 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.383 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_ident": { 2024-05-03T18:47:24.383 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_ident", 2024-05-03T18:47:24.383 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.383 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.383 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.383 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.383 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.383 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.383 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.383 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-03T18:47:24.383 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.383 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.383 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.383 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.383 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_perf": { 2024-05-03T18:47:24.384 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_perf", 2024-05-03T18:47:24.384 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.384 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.384 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.384 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.384 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.384 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.384 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.384 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-03T18:47:24.384 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.384 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.384 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.384 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.384 INFO:teuthology.orchestra.run.smithi033.stdout: "contact": { 2024-05-03T18:47:24.384 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "contact", 2024-05-03T18:47:24.384 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.385 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.385 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.385 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.385 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.385 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.385 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.385 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.385 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.385 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.385 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.385 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.385 INFO:teuthology.orchestra.run.smithi033.stdout: "description": { 2024-05-03T18:47:24.385 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "description", 2024-05-03T18:47:24.385 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.385 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.385 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.386 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.386 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.386 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.386 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.386 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.386 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.386 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.386 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.386 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.386 INFO:teuthology.orchestra.run.smithi033.stdout: "device_url": { 2024-05-03T18:47:24.386 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "device_url", 2024-05-03T18:47:24.386 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.386 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.386 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.386 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-03T18:47:24.387 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.387 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.387 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.387 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.387 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.387 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.387 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.387 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.387 INFO:teuthology.orchestra.run.smithi033.stdout: "enabled": { 2024-05-03T18:47:24.387 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "enabled", 2024-05-03T18:47:24.387 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.387 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.387 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.387 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.387 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.387 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.388 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.388 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.388 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.388 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.388 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.388 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.388 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-05-03T18:47:24.388 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-05-03T18:47:24.388 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.388 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.388 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.388 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "24", 2024-05-03T18:47:24.388 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "8", 2024-05-03T18:47:24.388 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.388 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.388 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.389 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.389 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.389 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.389 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.389 INFO:teuthology.orchestra.run.smithi033.stdout: "last_opt_revision": { 2024-05-03T18:47:24.389 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "last_opt_revision", 2024-05-03T18:47:24.389 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.389 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.389 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.389 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "1", 2024-05-03T18:47:24.389 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.389 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.389 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.389 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.389 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.389 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.390 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.390 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.390 INFO:teuthology.orchestra.run.smithi033.stdout: "leaderboard": { 2024-05-03T18:47:24.390 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "leaderboard", 2024-05-03T18:47:24.390 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.390 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.390 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.390 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.390 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.390 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.390 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.390 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.390 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.390 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.390 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.390 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.390 INFO:teuthology.orchestra.run.smithi033.stdout: "leaderboard_description": { 2024-05-03T18:47:24.391 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "leaderboard_description", 2024-05-03T18:47:24.391 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.391 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.391 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.391 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.391 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.391 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.391 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.391 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.391 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.391 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.391 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.391 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.391 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:24.391 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:24.391 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.392 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.392 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.392 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.392 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.392 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.392 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.392 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.392 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.392 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.392 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.392 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.392 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.392 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.392 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.392 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.392 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.393 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.393 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.393 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:24.393 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:24.393 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.393 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.393 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.393 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.393 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.393 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.393 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.393 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.393 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.393 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.393 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.393 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.393 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:24.394 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:24.394 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.394 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.394 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.394 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:24.394 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.394 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.394 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.394 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.394 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.394 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.394 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.394 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.394 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.394 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.395 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.395 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.395 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.395 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.395 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.395 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:24.395 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:24.395 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.395 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.395 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.395 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.395 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.395 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.395 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.395 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.396 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.396 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.396 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.396 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.396 INFO:teuthology.orchestra.run.smithi033.stdout: "organization": { 2024-05-03T18:47:24.396 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "organization", 2024-05-03T18:47:24.396 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.396 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.396 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.396 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.396 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.396 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.396 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.396 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.396 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.396 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.397 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.397 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.397 INFO:teuthology.orchestra.run.smithi033.stdout: "proxy": { 2024-05-03T18:47:24.397 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "proxy", 2024-05-03T18:47:24.397 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.397 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.397 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.397 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.397 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.397 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.397 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.397 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.397 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.397 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.397 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.397 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.397 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:24.398 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:24.398 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.398 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:24.398 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.398 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:24.398 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.398 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.398 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.398 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.398 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.398 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.398 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.398 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.398 INFO:teuthology.orchestra.run.smithi033.stdout: "url": { 2024-05-03T18:47:24.398 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "url", 2024-05-03T18:47:24.398 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.399 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.399 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.399 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-03T18:47:24.399 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.399 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.399 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.399 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.399 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.399 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.399 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.399 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.399 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.399 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.399 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:24.399 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "test_orchestrator", 2024-05-03T18:47:24.399 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:24.399 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:24.400 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:24.400 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:24.400 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:24.400 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.400 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.400 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.400 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.400 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.400 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.400 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.400 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.400 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.400 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.400 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.400 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.400 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.401 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.401 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.401 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.401 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.401 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.401 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.401 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:24.401 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:24.401 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.401 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.401 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.401 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.401 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.401 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.401 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.401 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.402 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.402 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.402 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.402 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.402 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:24.402 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:24.402 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.402 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.402 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.402 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:24.402 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.402 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.402 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.402 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.402 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.403 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.403 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.403 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.403 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.403 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.403 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.403 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.403 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.403 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.403 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.403 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:24.403 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:24.403 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.403 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.403 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.404 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.404 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.404 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.404 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.404 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.404 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.404 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.404 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.404 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.404 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:24.404 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:24.404 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.404 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:24.404 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.404 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:24.404 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.405 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.405 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.405 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.405 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.405 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.405 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.405 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.405 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.405 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.405 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:24.405 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "volumes", 2024-05-03T18:47:24.405 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:24.405 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:24.405 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:24.405 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:24.405 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:24.406 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.406 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.406 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.406 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.406 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.406 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.406 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.406 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.406 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.406 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.406 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.406 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.406 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.406 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.407 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.407 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.407 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.407 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.407 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.407 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:24.407 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:24.407 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.407 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.407 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.407 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.407 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.407 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.407 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.407 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.408 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.408 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.408 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.408 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.408 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:24.408 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:24.408 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.408 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.408 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.408 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:24.408 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.408 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.408 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.408 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.409 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.409 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.409 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.409 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.409 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.409 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.409 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.409 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.409 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.409 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.409 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.409 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:24.409 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:24.409 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.410 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.410 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.410 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.410 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.410 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.410 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.410 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.410 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.410 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.410 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.410 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.410 INFO:teuthology.orchestra.run.smithi033.stdout: "max_concurrent_clones": { 2024-05-03T18:47:24.410 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_concurrent_clones", 2024-05-03T18:47:24.410 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.411 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.411 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.411 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "4", 2024-05-03T18:47:24.411 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.411 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.411 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.411 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-03T18:47:24.411 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.411 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.411 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.411 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.411 INFO:teuthology.orchestra.run.smithi033.stdout: "periodic_async_work": { 2024-05-03T18:47:24.411 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "periodic_async_work", 2024-05-03T18:47:24.411 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.411 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.412 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.412 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.412 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.412 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.412 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.412 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Periodically check for async work", 2024-05-03T18:47:24.412 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.412 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.412 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.412 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.412 INFO:teuthology.orchestra.run.smithi033.stdout: "snapshot_clone_delay": { 2024-05-03T18:47:24.412 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "snapshot_clone_delay", 2024-05-03T18:47:24.412 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.412 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.412 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.413 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:24.413 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.413 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.413 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.413 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-03T18:47:24.413 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.413 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.413 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.413 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.413 INFO:teuthology.orchestra.run.smithi033.stdout: "snapshot_clone_no_wait": { 2024-05-03T18:47:24.413 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "snapshot_clone_no_wait", 2024-05-03T18:47:24.413 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.413 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.413 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.413 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:24.414 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.414 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.414 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.414 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-03T18:47:24.414 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.414 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.414 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.414 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.414 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:24.414 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:24.414 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.414 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:24.414 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.414 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:24.414 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.415 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.415 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.415 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.415 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.415 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.415 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.415 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.415 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.415 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.415 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:24.415 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "zabbix", 2024-05-03T18:47:24.415 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:24.415 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:24.415 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:24.415 INFO:teuthology.orchestra.run.smithi033.stdout: "discovery_interval": { 2024-05-03T18:47:24.415 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "discovery_interval", 2024-05-03T18:47:24.416 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-05-03T18:47:24.416 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.416 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.416 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "100", 2024-05-03T18:47:24.416 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.416 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.416 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.416 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.416 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.416 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.416 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.416 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.416 INFO:teuthology.orchestra.run.smithi033.stdout: "identifier": { 2024-05-03T18:47:24.416 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "identifier", 2024-05-03T18:47:24.416 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.416 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.417 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.417 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.417 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.417 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.417 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.417 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.417 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.417 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.417 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.417 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.417 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-05-03T18:47:24.417 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-05-03T18:47:24.417 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-03T18:47:24.417 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.417 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.417 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-05-03T18:47:24.418 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.418 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.418 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.418 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.418 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.418 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.418 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.418 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.418 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:24.418 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:24.418 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.418 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.418 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.418 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.418 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.418 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.419 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.419 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.419 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.419 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.419 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.419 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.419 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.419 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.419 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.419 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.419 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.419 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.419 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.419 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:24.419 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:24.419 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.419 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.420 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.420 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.420 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.420 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.420 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.420 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.420 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.420 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.420 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.420 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.420 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:24.420 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:24.420 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.420 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.420 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.420 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:24.421 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.421 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.421 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.421 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.421 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.421 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.421 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.421 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.421 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.421 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.421 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.421 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.421 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.421 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.421 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.422 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:24.422 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:24.422 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.422 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.422 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.422 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.422 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.422 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.422 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.422 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.422 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.422 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.422 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.422 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.422 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:24.423 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:24.423 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.423 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:24.423 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.423 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:24.423 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.423 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.423 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.423 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.423 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.423 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.423 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.423 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.423 INFO:teuthology.orchestra.run.smithi033.stdout: "zabbix_host": { 2024-05-03T18:47:24.423 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "zabbix_host", 2024-05-03T18:47:24.423 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.424 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.424 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.424 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.424 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.424 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.424 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.424 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.424 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.424 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.424 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.424 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.424 INFO:teuthology.orchestra.run.smithi033.stdout: "zabbix_port": { 2024-05-03T18:47:24.424 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "zabbix_port", 2024-05-03T18:47:24.424 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.424 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.424 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.425 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10051", 2024-05-03T18:47:24.425 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.425 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.425 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.425 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.425 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.425 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.425 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.425 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.425 INFO:teuthology.orchestra.run.smithi033.stdout: "zabbix_sender": { 2024-05-03T18:47:24.425 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "zabbix_sender", 2024-05-03T18:47:24.425 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.425 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.425 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.425 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-03T18:47:24.425 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.426 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.426 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.426 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.426 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.426 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.426 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.426 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.426 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.426 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.426 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-03T18:47:24.426 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.426 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.426 INFO:teuthology.orchestra.run.smithi033.stdout: "modules": [ 2024-05-03T18:47:24.426 INFO:teuthology.orchestra.run.smithi033.stdout: "cephadm", 2024-05-03T18:47:24.427 INFO:teuthology.orchestra.run.smithi033.stdout: "dashboard", 2024-05-03T18:47:24.427 INFO:teuthology.orchestra.run.smithi033.stdout: "iostat", 2024-05-03T18:47:24.427 INFO:teuthology.orchestra.run.smithi033.stdout: "nfs", 2024-05-03T18:47:24.427 INFO:teuthology.orchestra.run.smithi033.stdout: "restful", 2024-05-03T18:47:24.427 INFO:teuthology.orchestra.run.smithi033.stdout: "snap_schedule" 2024-05-03T18:47:24.427 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.427 INFO:teuthology.orchestra.run.smithi033.stdout: "available_modules": [ 2024-05-03T18:47:24.427 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:24.427 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "alerts", 2024-05-03T18:47:24.427 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:24.427 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:24.427 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:24.427 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-05-03T18:47:24.427 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-05-03T18:47:24.427 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-03T18:47:24.427 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.428 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.428 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-05-03T18:47:24.428 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.428 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.428 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.428 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "How frequently to reexamine health status", 2024-05-03T18:47:24.428 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.428 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.428 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.428 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.428 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:24.428 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:24.428 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.428 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.429 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.429 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.429 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.429 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.429 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.429 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.429 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.429 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.429 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.429 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.429 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.429 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.429 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.429 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.430 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.430 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.430 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.430 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:24.430 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:24.430 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.430 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.430 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.430 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.430 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.430 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.430 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.430 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.430 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.431 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.431 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.431 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.431 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:24.431 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:24.431 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.431 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.431 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.431 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:24.431 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.431 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.431 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.431 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.431 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.431 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.431 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.432 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.432 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.432 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.432 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.432 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.432 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.432 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.432 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.432 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:24.432 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:24.432 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.432 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.432 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.432 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.433 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.433 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.433 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.433 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.433 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.433 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.433 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.433 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.433 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_destination": { 2024-05-03T18:47:24.433 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_destination", 2024-05-03T18:47:24.433 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.433 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.433 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.433 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.433 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.433 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.434 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.434 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Email address to send alerts to", 2024-05-03T18:47:24.434 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.434 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.434 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.434 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.434 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_from_name": { 2024-05-03T18:47:24.434 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_from_name", 2024-05-03T18:47:24.434 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.434 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.434 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.434 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "Ceph", 2024-05-03T18:47:24.434 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.434 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.434 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.435 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Email From: name", 2024-05-03T18:47:24.435 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.435 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.435 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.435 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.435 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_host": { 2024-05-03T18:47:24.435 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_host", 2024-05-03T18:47:24.435 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.435 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.435 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.435 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.435 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.435 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.436 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.436 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "SMTP server", 2024-05-03T18:47:24.436 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.436 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.436 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.436 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.436 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_password": { 2024-05-03T18:47:24.436 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_password", 2024-05-03T18:47:24.436 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.436 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.436 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.436 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.436 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.436 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.437 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.437 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Password to authenticate with", 2024-05-03T18:47:24.437 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.437 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.437 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.437 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.437 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_port": { 2024-05-03T18:47:24.437 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_port", 2024-05-03T18:47:24.437 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.437 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.437 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.437 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "465", 2024-05-03T18:47:24.437 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.437 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.438 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.438 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "SMTP port", 2024-05-03T18:47:24.438 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.438 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.438 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.438 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.438 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_sender": { 2024-05-03T18:47:24.438 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_sender", 2024-05-03T18:47:24.438 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.438 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.438 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.438 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.438 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.439 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.439 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.439 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "SMTP envelope sender", 2024-05-03T18:47:24.439 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.439 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.439 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.439 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.439 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_ssl": { 2024-05-03T18:47:24.439 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_ssl", 2024-05-03T18:47:24.439 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.439 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.439 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.439 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:24.439 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.439 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.440 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.440 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-03T18:47:24.440 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.440 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.440 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.440 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.440 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_user": { 2024-05-03T18:47:24.440 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_user", 2024-05-03T18:47:24.440 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.440 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.440 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.440 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.440 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.440 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.440 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.440 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "User to authenticate as", 2024-05-03T18:47:24.441 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.441 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.441 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.441 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.441 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:24.441 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:24.441 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.441 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:24.441 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.441 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:24.441 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.441 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.441 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.441 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.441 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.442 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.442 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.442 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.442 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.442 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.442 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:24.442 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "balancer", 2024-05-03T18:47:24.442 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:24.442 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:24.442 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:24.442 INFO:teuthology.orchestra.run.smithi033.stdout: "active": { 2024-05-03T18:47:24.442 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "active", 2024-05-03T18:47:24.442 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.442 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.443 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.443 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:24.443 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.443 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.443 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.443 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "automatically balance PGs across cluster", 2024-05-03T18:47:24.443 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.443 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.443 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.443 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.443 INFO:teuthology.orchestra.run.smithi033.stdout: "begin_time": { 2024-05-03T18:47:24.443 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "begin_time", 2024-05-03T18:47:24.443 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.443 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.443 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.443 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0000", 2024-05-03T18:47:24.444 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.444 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.444 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.444 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "beginning time of day to automatically balance", 2024-05-03T18:47:24.444 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-03T18:47:24.444 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.444 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.444 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.444 INFO:teuthology.orchestra.run.smithi033.stdout: "begin_weekday": { 2024-05-03T18:47:24.444 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "begin_weekday", 2024-05-03T18:47:24.444 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-05-03T18:47:24.444 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.444 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.444 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:24.444 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "0", 2024-05-03T18:47:24.445 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "6", 2024-05-03T18:47:24.445 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.445 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-03T18:47:24.445 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-03T18:47:24.445 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.445 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.445 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.445 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_compat_max_iterations": { 2024-05-03T18:47:24.445 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crush_compat_max_iterations", 2024-05-03T18:47:24.445 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-05-03T18:47:24.445 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.445 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.445 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "25", 2024-05-03T18:47:24.445 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1", 2024-05-03T18:47:24.446 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "250", 2024-05-03T18:47:24.446 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.446 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-03T18:47:24.446 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.446 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.446 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.446 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.446 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_compat_metrics": { 2024-05-03T18:47:24.446 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crush_compat_metrics", 2024-05-03T18:47:24.446 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.446 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.446 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.446 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "pgs,objects,bytes", 2024-05-03T18:47:24.447 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.447 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.447 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.447 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-03T18:47:24.447 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-03T18:47:24.447 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.447 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.447 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.447 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_compat_step": { 2024-05-03T18:47:24.447 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crush_compat_step", 2024-05-03T18:47:24.447 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-05-03T18:47:24.447 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.447 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.447 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0.5", 2024-05-03T18:47:24.447 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "0.001", 2024-05-03T18:47:24.448 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "0.999", 2024-05-03T18:47:24.448 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.448 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "aggressiveness of optimization", 2024-05-03T18:47:24.448 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-03T18:47:24.448 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.448 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.448 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.448 INFO:teuthology.orchestra.run.smithi033.stdout: "end_time": { 2024-05-03T18:47:24.448 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "end_time", 2024-05-03T18:47:24.448 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.448 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.448 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.448 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "2359", 2024-05-03T18:47:24.448 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.448 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.449 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.449 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "ending time of day to automatically balance", 2024-05-03T18:47:24.449 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-03T18:47:24.449 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.450 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.450 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.450 INFO:teuthology.orchestra.run.smithi033.stdout: "end_weekday": { 2024-05-03T18:47:24.450 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "end_weekday", 2024-05-03T18:47:24.450 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-05-03T18:47:24.450 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.450 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.450 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:24.450 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "0", 2024-05-03T18:47:24.451 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "6", 2024-05-03T18:47:24.451 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.451 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-03T18:47:24.451 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-03T18:47:24.451 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.451 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.451 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.451 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:24.451 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:24.451 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.451 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.451 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.451 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.451 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.452 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.452 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.452 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.452 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.452 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.452 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.452 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.452 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.452 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.452 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.452 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.452 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.452 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.452 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.452 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:24.453 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:24.453 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.453 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.453 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.453 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.453 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.453 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.453 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.453 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.453 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.453 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.453 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.453 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.454 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:24.454 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:24.454 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.454 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.454 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.454 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:24.454 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.454 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.454 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.454 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.454 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.454 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.454 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.455 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.455 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.455 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.455 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.455 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.455 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.455 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.455 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.455 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:24.455 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:24.455 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.455 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.455 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.455 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.455 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.456 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.456 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.456 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.456 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.456 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.456 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.456 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.456 INFO:teuthology.orchestra.run.smithi033.stdout: "min_score": { 2024-05-03T18:47:24.456 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "min_score", 2024-05-03T18:47:24.456 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-05-03T18:47:24.456 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.456 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.456 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:24.456 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.456 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.457 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.457 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-03T18:47:24.457 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.457 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.457 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.457 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.457 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": { 2024-05-03T18:47:24.457 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mode", 2024-05-03T18:47:24.457 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.457 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.457 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.457 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "upmap", 2024-05-03T18:47:24.457 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.457 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.457 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.458 INFO:teuthology.orchestra.run.smithi033.stdout: "crush-compat", 2024-05-03T18:47:24.458 INFO:teuthology.orchestra.run.smithi033.stdout: "none", 2024-05-03T18:47:24.458 INFO:teuthology.orchestra.run.smithi033.stdout: "read", 2024-05-03T18:47:24.458 INFO:teuthology.orchestra.run.smithi033.stdout: "upmap", 2024-05-03T18:47:24.458 INFO:teuthology.orchestra.run.smithi033.stdout: "upmap-read" 2024-05-03T18:47:24.458 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.458 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Balancer mode", 2024-05-03T18:47:24.458 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.458 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.458 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.458 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.458 INFO:teuthology.orchestra.run.smithi033.stdout: "pool_ids": { 2024-05-03T18:47:24.458 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "pool_ids", 2024-05-03T18:47:24.459 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.459 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.459 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.459 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.459 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.459 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.459 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.459 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-03T18:47:24.459 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.459 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.459 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.459 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.459 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-05-03T18:47:24.459 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-05-03T18:47:24.459 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-03T18:47:24.459 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.459 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.460 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-05-03T18:47:24.460 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.460 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.460 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.460 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-03T18:47:24.460 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.460 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.460 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.460 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.460 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:24.460 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:24.460 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.460 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:24.460 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.460 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:24.461 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.461 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.461 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.461 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.461 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.461 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.461 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.461 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.461 INFO:teuthology.orchestra.run.smithi033.stdout: "upmap_max_deviation": { 2024-05-03T18:47:24.461 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "upmap_max_deviation", 2024-05-03T18:47:24.461 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.461 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.461 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.461 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5", 2024-05-03T18:47:24.462 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1", 2024-05-03T18:47:24.462 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.462 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.462 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-03T18:47:24.462 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-03T18:47:24.462 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.462 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.462 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.462 INFO:teuthology.orchestra.run.smithi033.stdout: "upmap_max_optimizations": { 2024-05-03T18:47:24.462 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "upmap_max_optimizations", 2024-05-03T18:47:24.462 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-05-03T18:47:24.462 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.462 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.462 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-05-03T18:47:24.462 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.463 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.463 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.463 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-03T18:47:24.463 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.463 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.463 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.463 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.463 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.463 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.463 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:24.463 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cephadm", 2024-05-03T18:47:24.463 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:24.463 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:24.463 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:24.463 INFO:teuthology.orchestra.run.smithi033.stdout: "agent_down_multiplier": { 2024-05-03T18:47:24.463 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "agent_down_multiplier", 2024-05-03T18:47:24.464 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-05-03T18:47:24.464 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.464 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.464 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "3.0", 2024-05-03T18:47:24.464 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.464 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.464 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.464 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-03T18:47:24.464 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.464 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.464 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.464 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.464 INFO:teuthology.orchestra.run.smithi033.stdout: "agent_refresh_rate": { 2024-05-03T18:47:24.464 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "agent_refresh_rate", 2024-05-03T18:47:24.464 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-03T18:47:24.464 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.464 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.465 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "20", 2024-05-03T18:47:24.465 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.465 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.465 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.465 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-03T18:47:24.465 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.465 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.465 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.465 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.465 INFO:teuthology.orchestra.run.smithi033.stdout: "agent_starting_port": { 2024-05-03T18:47:24.465 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "agent_starting_port", 2024-05-03T18:47:24.465 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.465 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.465 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.465 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "4721", 2024-05-03T18:47:24.466 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.466 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.466 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.466 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-03T18:47:24.466 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.466 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.466 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.466 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.466 INFO:teuthology.orchestra.run.smithi033.stdout: "allow_ptrace": { 2024-05-03T18:47:24.466 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "allow_ptrace", 2024-05-03T18:47:24.466 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.466 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.466 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.466 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.467 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.467 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.467 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.467 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-03T18:47:24.467 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-03T18:47:24.467 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.467 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.467 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.467 INFO:teuthology.orchestra.run.smithi033.stdout: "autotune_interval": { 2024-05-03T18:47:24.467 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "autotune_interval", 2024-05-03T18:47:24.467 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-03T18:47:24.467 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.467 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.467 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-05-03T18:47:24.468 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.468 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.468 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.468 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-03T18:47:24.468 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.468 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.468 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.468 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.468 INFO:teuthology.orchestra.run.smithi033.stdout: "autotune_memory_target_ratio": { 2024-05-03T18:47:24.468 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "autotune_memory_target_ratio", 2024-05-03T18:47:24.468 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-05-03T18:47:24.468 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.468 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.468 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0.7", 2024-05-03T18:47:24.469 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.469 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.469 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.469 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-03T18:47:24.469 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.469 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.469 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.469 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.469 INFO:teuthology.orchestra.run.smithi033.stdout: "cephadm_log_destination": { 2024-05-03T18:47:24.469 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cephadm_log_destination", 2024-05-03T18:47:24.469 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.469 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.469 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.469 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.470 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.470 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.470 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.470 INFO:teuthology.orchestra.run.smithi033.stdout: "file", 2024-05-03T18:47:24.470 INFO:teuthology.orchestra.run.smithi033.stdout: "file,syslog", 2024-05-03T18:47:24.470 INFO:teuthology.orchestra.run.smithi033.stdout: "syslog" 2024-05-03T18:47:24.470 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.470 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-03T18:47:24.470 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.470 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.470 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.470 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.470 INFO:teuthology.orchestra.run.smithi033.stdout: "cgroups_split": { 2024-05-03T18:47:24.471 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cgroups_split", 2024-05-03T18:47:24.471 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.471 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.471 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.471 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:24.471 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.471 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.471 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.471 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-03T18:47:24.471 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.471 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.471 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.471 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.471 INFO:teuthology.orchestra.run.smithi033.stdout: "config_checks_enabled": { 2024-05-03T18:47:24.472 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "config_checks_enabled", 2024-05-03T18:47:24.472 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.472 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.472 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.472 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.472 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.472 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.472 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.472 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-03T18:47:24.472 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.472 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.472 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.472 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.472 INFO:teuthology.orchestra.run.smithi033.stdout: "config_dashboard": { 2024-05-03T18:47:24.472 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "config_dashboard", 2024-05-03T18:47:24.472 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.473 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.473 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.473 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:24.473 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.473 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.473 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.473 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-03T18:47:24.473 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.473 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.473 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.473 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.473 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_alertmanager": { 2024-05-03T18:47:24.473 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_alertmanager", 2024-05-03T18:47:24.473 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.473 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.474 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.474 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-03T18:47:24.474 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.474 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.474 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.474 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Prometheus container image", 2024-05-03T18:47:24.474 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.474 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.474 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.474 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.474 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_base": { 2024-05-03T18:47:24.474 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_base", 2024-05-03T18:47:24.474 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.474 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.475 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.475 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-03T18:47:24.475 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.475 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.475 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.475 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Container image name, without the tag", 2024-05-03T18:47:24.475 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.475 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.475 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.475 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.475 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_elasticsearch": { 2024-05-03T18:47:24.475 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_elasticsearch", 2024-05-03T18:47:24.475 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.475 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.476 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.476 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-03T18:47:24.476 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.476 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.476 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.476 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "elasticsearch container image", 2024-05-03T18:47:24.476 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.476 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.476 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.476 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.476 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_grafana": { 2024-05-03T18:47:24.476 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_grafana", 2024-05-03T18:47:24.476 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.476 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.476 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.476 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-03T18:47:24.477 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.477 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.477 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.477 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Prometheus container image", 2024-05-03T18:47:24.477 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.477 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.477 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.477 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.477 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_haproxy": { 2024-05-03T18:47:24.477 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_haproxy", 2024-05-03T18:47:24.477 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.477 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.477 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.477 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-03T18:47:24.477 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.477 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.478 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.478 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "HAproxy container image", 2024-05-03T18:47:24.478 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.478 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.478 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.478 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.478 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_jaeger_agent": { 2024-05-03T18:47:24.478 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_jaeger_agent", 2024-05-03T18:47:24.478 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.478 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.478 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.478 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-03T18:47:24.478 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.478 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.479 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.479 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Jaeger agent container image", 2024-05-03T18:47:24.479 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.479 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.479 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.479 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.479 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_jaeger_collector": { 2024-05-03T18:47:24.479 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_jaeger_collector", 2024-05-03T18:47:24.479 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.479 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.479 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.479 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-03T18:47:24.479 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.479 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.479 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.480 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Jaeger collector container image", 2024-05-03T18:47:24.480 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.480 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.480 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.480 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.480 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_jaeger_query": { 2024-05-03T18:47:24.480 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_jaeger_query", 2024-05-03T18:47:24.480 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.480 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.480 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.480 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-03T18:47:24.480 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.480 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.480 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.480 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Jaeger query container image", 2024-05-03T18:47:24.481 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.481 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.481 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.481 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.481 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_keepalived": { 2024-05-03T18:47:24.481 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_keepalived", 2024-05-03T18:47:24.481 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.481 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.481 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.481 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-03T18:47:24.481 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.481 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.481 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.481 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Keepalived container image", 2024-05-03T18:47:24.482 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.482 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.482 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.482 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.482 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_loki": { 2024-05-03T18:47:24.482 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_loki", 2024-05-03T18:47:24.482 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.482 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.482 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.482 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-03T18:47:24.482 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.482 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.482 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.482 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Loki container image", 2024-05-03T18:47:24.483 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.483 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.483 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.483 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.483 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_node_exporter": { 2024-05-03T18:47:24.483 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_node_exporter", 2024-05-03T18:47:24.483 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.483 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.483 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.483 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-03T18:47:24.483 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.483 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.483 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.483 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Prometheus container image", 2024-05-03T18:47:24.483 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.483 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.484 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.484 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.484 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_nvmeof": { 2024-05-03T18:47:24.484 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_nvmeof", 2024-05-03T18:47:24.484 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.484 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.484 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.484 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-05-03T18:47:24.484 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.484 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.484 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.484 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Nvme-of container image", 2024-05-03T18:47:24.484 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.484 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.484 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.485 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.485 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_prometheus": { 2024-05-03T18:47:24.485 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_prometheus", 2024-05-03T18:47:24.485 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.485 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.485 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.485 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-03T18:47:24.485 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.485 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.485 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.485 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Prometheus container image", 2024-05-03T18:47:24.485 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.485 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.485 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.486 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.486 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_promtail": { 2024-05-03T18:47:24.486 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_promtail", 2024-05-03T18:47:24.486 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.486 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.486 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.486 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-03T18:47:24.486 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.486 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.486 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.486 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Promtail container image", 2024-05-03T18:47:24.486 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.486 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.487 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.487 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.487 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_samba": { 2024-05-03T18:47:24.487 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_samba", 2024-05-03T18:47:24.487 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.487 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.487 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.487 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-03T18:47:24.487 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.487 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.487 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.487 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Samba/SMB container image", 2024-05-03T18:47:24.487 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.487 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.487 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.488 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.488 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_snmp_gateway": { 2024-05-03T18:47:24.488 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_snmp_gateway", 2024-05-03T18:47:24.488 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.488 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.488 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.488 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-03T18:47:24.488 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.488 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.488 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.488 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "SNMP Gateway container image", 2024-05-03T18:47:24.488 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.488 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.488 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.488 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.489 INFO:teuthology.orchestra.run.smithi033.stdout: "container_init": { 2024-05-03T18:47:24.489 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_init", 2024-05-03T18:47:24.489 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.489 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.489 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.489 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:24.489 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.489 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.489 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.489 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Run podman/docker with `--init`", 2024-05-03T18:47:24.489 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.489 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.489 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.489 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.489 INFO:teuthology.orchestra.run.smithi033.stdout: "daemon_cache_timeout": { 2024-05-03T18:47:24.489 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "daemon_cache_timeout", 2024-05-03T18:47:24.490 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-03T18:47:24.490 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.490 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.490 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-05-03T18:47:24.490 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.490 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.490 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.490 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-03T18:47:24.490 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.490 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.490 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.490 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.490 INFO:teuthology.orchestra.run.smithi033.stdout: "default_cephadm_command_timeout": { 2024-05-03T18:47:24.490 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "default_cephadm_command_timeout", 2024-05-03T18:47:24.491 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.491 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.491 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.491 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "900", 2024-05-03T18:47:24.491 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.491 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.491 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.491 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-03T18:47:24.491 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.491 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.491 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.491 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.491 INFO:teuthology.orchestra.run.smithi033.stdout: "default_registry": { 2024-05-03T18:47:24.491 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "default_registry", 2024-05-03T18:47:24.492 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.492 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.492 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.492 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "docker.io", 2024-05-03T18:47:24.492 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.492 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.492 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.492 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-03T18:47:24.492 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.492 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.492 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.492 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.492 INFO:teuthology.orchestra.run.smithi033.stdout: "device_cache_timeout": { 2024-05-03T18:47:24.492 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "device_cache_timeout", 2024-05-03T18:47:24.492 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-03T18:47:24.493 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.493 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.493 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "1800", 2024-05-03T18:47:24.493 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.493 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.493 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.493 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "seconds to cache device inventory", 2024-05-03T18:47:24.493 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.493 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.493 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.493 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.493 INFO:teuthology.orchestra.run.smithi033.stdout: "device_enhanced_scan": { 2024-05-03T18:47:24.493 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "device_enhanced_scan", 2024-05-03T18:47:24.493 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.493 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.493 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.494 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.494 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.494 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.494 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.494 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-03T18:47:24.494 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.494 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.494 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.494 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.494 INFO:teuthology.orchestra.run.smithi033.stdout: "facts_cache_timeout": { 2024-05-03T18:47:24.494 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "facts_cache_timeout", 2024-05-03T18:47:24.494 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-03T18:47:24.494 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.494 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.494 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-05-03T18:47:24.495 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.495 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.495 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.495 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "seconds to cache host facts data", 2024-05-03T18:47:24.495 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.495 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.495 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.495 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.495 INFO:teuthology.orchestra.run.smithi033.stdout: "grafana_dashboards_path": { 2024-05-03T18:47:24.495 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "grafana_dashboards_path", 2024-05-03T18:47:24.495 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.495 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.495 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.495 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-03T18:47:24.495 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.495 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.496 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.496 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-03T18:47:24.496 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.496 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.496 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.496 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.496 INFO:teuthology.orchestra.run.smithi033.stdout: "host_check_interval": { 2024-05-03T18:47:24.496 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "host_check_interval", 2024-05-03T18:47:24.496 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-03T18:47:24.496 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.496 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.496 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-05-03T18:47:24.496 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.496 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.496 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.497 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to perform a host check", 2024-05-03T18:47:24.497 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.497 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.497 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.497 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.497 INFO:teuthology.orchestra.run.smithi033.stdout: "hw_monitoring": { 2024-05-03T18:47:24.497 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "hw_monitoring", 2024-05-03T18:47:24.497 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.497 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.497 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.497 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.497 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.497 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.497 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.497 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-03T18:47:24.498 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.498 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.498 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.498 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.498 INFO:teuthology.orchestra.run.smithi033.stdout: "inventory_list_all": { 2024-05-03T18:47:24.498 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "inventory_list_all", 2024-05-03T18:47:24.498 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.498 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.498 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.498 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.498 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.498 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.498 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.498 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-03T18:47:24.498 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.499 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.499 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.499 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.499 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:24.499 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:24.499 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.499 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.499 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.499 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.499 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.499 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.499 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.499 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.499 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.500 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.500 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.500 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.500 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.500 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.500 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.500 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.500 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.500 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.500 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.500 INFO:teuthology.orchestra.run.smithi033.stdout: "log_refresh_metadata": { 2024-05-03T18:47:24.500 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_refresh_metadata", 2024-05-03T18:47:24.500 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.500 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.501 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.501 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.501 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.501 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.501 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.501 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-03T18:47:24.501 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.501 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.501 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.501 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.501 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:24.501 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:24.501 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.501 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.501 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.501 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:24.502 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.502 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.502 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.502 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-03T18:47:24.502 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.502 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.502 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.502 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.502 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:24.502 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:24.502 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.502 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.502 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.503 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:24.503 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.503 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.503 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.503 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.503 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.503 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.503 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.503 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.503 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.503 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.503 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.503 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.503 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.503 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.503 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.504 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:24.504 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:24.504 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.504 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.504 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.504 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.504 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.504 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.504 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.504 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.504 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.504 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.504 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.504 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.505 INFO:teuthology.orchestra.run.smithi033.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-03T18:47:24.505 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-03T18:47:24.505 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.505 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.505 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.505 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.505 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.505 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.505 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.505 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-03T18:47:24.505 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.505 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.505 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.505 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.505 INFO:teuthology.orchestra.run.smithi033.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-03T18:47:24.505 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-03T18:47:24.506 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.506 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.506 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.506 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "*", 2024-05-03T18:47:24.506 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.506 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.506 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.506 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-03T18:47:24.506 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.506 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.506 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.506 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.506 INFO:teuthology.orchestra.run.smithi033.stdout: "max_count_per_host": { 2024-05-03T18:47:24.506 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_count_per_host", 2024-05-03T18:47:24.507 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.507 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.507 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.507 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-05-03T18:47:24.507 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.507 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.507 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.507 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "max number of daemons per service per host", 2024-05-03T18:47:24.507 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.507 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.507 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.507 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.507 INFO:teuthology.orchestra.run.smithi033.stdout: "max_osd_draining_count": { 2024-05-03T18:47:24.508 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_osd_draining_count", 2024-05-03T18:47:24.508 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.508 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.508 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.508 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-05-03T18:47:24.508 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.508 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.508 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.508 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-03T18:47:24.508 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.508 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.508 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.508 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.508 INFO:teuthology.orchestra.run.smithi033.stdout: "migration_current": { 2024-05-03T18:47:24.509 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "migration_current", 2024-05-03T18:47:24.509 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.509 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.509 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.509 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.509 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.509 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.509 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.509 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "internal - do not modify", 2024-05-03T18:47:24.509 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.509 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.509 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.509 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.510 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": { 2024-05-03T18:47:24.510 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mode", 2024-05-03T18:47:24.510 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.510 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.510 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.510 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "root", 2024-05-03T18:47:24.510 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.510 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.510 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.510 INFO:teuthology.orchestra.run.smithi033.stdout: "cephadm-package", 2024-05-03T18:47:24.510 INFO:teuthology.orchestra.run.smithi033.stdout: "root" 2024-05-03T18:47:24.511 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.511 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "mode for remote execution of cephadm", 2024-05-03T18:47:24.511 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.511 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.511 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.511 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.511 INFO:teuthology.orchestra.run.smithi033.stdout: "oob_default_addr": { 2024-05-03T18:47:24.511 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "oob_default_addr", 2024-05-03T18:47:24.511 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.511 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.511 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.511 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "169.254.1.1", 2024-05-03T18:47:24.512 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.512 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.512 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.512 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-03T18:47:24.512 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.512 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.512 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.512 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.512 INFO:teuthology.orchestra.run.smithi033.stdout: "prometheus_alerts_path": { 2024-05-03T18:47:24.512 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "prometheus_alerts_path", 2024-05-03T18:47:24.512 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.512 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.513 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.513 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-03T18:47:24.513 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.513 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.513 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.513 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-03T18:47:24.513 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.513 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.513 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.513 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.513 INFO:teuthology.orchestra.run.smithi033.stdout: "registry_insecure": { 2024-05-03T18:47:24.514 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "registry_insecure", 2024-05-03T18:47:24.514 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.514 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.514 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.514 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.514 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.514 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.514 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.514 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-03T18:47:24.514 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.514 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.514 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.514 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.515 INFO:teuthology.orchestra.run.smithi033.stdout: "registry_password": { 2024-05-03T18:47:24.515 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "registry_password", 2024-05-03T18:47:24.515 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.515 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.515 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.515 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.515 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.515 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.515 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.515 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-03T18:47:24.515 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.515 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.515 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.515 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.516 INFO:teuthology.orchestra.run.smithi033.stdout: "registry_url": { 2024-05-03T18:47:24.516 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "registry_url", 2024-05-03T18:47:24.516 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.516 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.516 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.516 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.516 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.516 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.516 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.516 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-03T18:47:24.516 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.516 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.516 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.516 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.516 INFO:teuthology.orchestra.run.smithi033.stdout: "registry_username": { 2024-05-03T18:47:24.517 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "registry_username", 2024-05-03T18:47:24.517 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.517 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.517 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.517 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.517 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.517 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.517 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.517 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-03T18:47:24.517 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.517 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.517 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.517 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.517 INFO:teuthology.orchestra.run.smithi033.stdout: "secure_monitoring_stack": { 2024-05-03T18:47:24.518 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "secure_monitoring_stack", 2024-05-03T18:47:24.518 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.518 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.518 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.518 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.518 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.518 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.518 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.518 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-03T18:47:24.518 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.518 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.518 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.518 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.519 INFO:teuthology.orchestra.run.smithi033.stdout: "service_discovery_port": { 2024-05-03T18:47:24.519 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "service_discovery_port", 2024-05-03T18:47:24.519 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.519 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.519 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.519 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8765", 2024-05-03T18:47:24.519 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.519 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.519 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.519 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "cephadm service discovery port", 2024-05-03T18:47:24.519 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.519 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.520 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.520 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.520 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:24.520 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:24.520 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.520 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:24.520 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.520 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:24.520 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.520 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.520 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.520 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.520 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.520 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.520 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.521 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.521 INFO:teuthology.orchestra.run.smithi033.stdout: "ssh_config_file": { 2024-05-03T18:47:24.521 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ssh_config_file", 2024-05-03T18:47:24.521 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.521 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.521 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.521 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.521 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.521 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.521 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.521 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-03T18:47:24.521 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.521 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.521 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.521 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.521 INFO:teuthology.orchestra.run.smithi033.stdout: "use_agent": { 2024-05-03T18:47:24.522 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "use_agent", 2024-05-03T18:47:24.522 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.522 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.522 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.522 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.522 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.522 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.522 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.522 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-03T18:47:24.522 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.522 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.523 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.523 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.523 INFO:teuthology.orchestra.run.smithi033.stdout: "use_repo_digest": { 2024-05-03T18:47:24.523 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "use_repo_digest", 2024-05-03T18:47:24.523 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.523 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.523 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.523 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:24.524 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.524 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.524 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.524 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-03T18:47:24.524 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.524 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.524 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.525 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.525 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_on_failed_host_check": { 2024-05-03T18:47:24.525 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_on_failed_host_check", 2024-05-03T18:47:24.525 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.525 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.525 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.525 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:24.525 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.526 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.526 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.526 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "raise a health warning if the host check fails", 2024-05-03T18:47:24.526 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.526 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.526 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.526 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.527 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_on_stray_daemons": { 2024-05-03T18:47:24.527 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_on_stray_daemons", 2024-05-03T18:47:24.527 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.527 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.527 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.527 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:24.527 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.527 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.528 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.528 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-03T18:47:24.528 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.528 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.528 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.528 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.528 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_on_stray_hosts": { 2024-05-03T18:47:24.528 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_on_stray_hosts", 2024-05-03T18:47:24.528 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.528 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.528 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.528 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:24.528 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.528 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.529 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.529 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-03T18:47:24.529 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.529 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.529 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.529 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.529 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.529 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.529 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:24.529 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crash", 2024-05-03T18:47:24.529 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:24.529 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:24.529 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:24.529 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:24.529 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:24.530 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.530 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.530 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.530 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.530 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.530 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.530 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.530 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.530 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.530 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.530 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.530 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.530 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.530 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.531 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.531 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.531 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.531 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.531 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.531 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:24.531 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:24.531 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.531 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.531 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.531 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.531 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.531 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.531 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.531 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.531 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.532 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.532 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.532 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.532 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:24.532 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:24.532 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.532 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.532 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.532 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:24.532 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.532 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.532 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.532 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.532 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.532 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.532 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.532 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.533 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.533 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.533 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.533 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.533 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.533 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.533 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.533 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:24.533 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:24.533 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.533 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.533 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.533 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.533 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.533 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.533 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.534 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.534 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.534 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.534 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.534 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.534 INFO:teuthology.orchestra.run.smithi033.stdout: "retain_interval": { 2024-05-03T18:47:24.534 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "retain_interval", 2024-05-03T18:47:24.534 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-03T18:47:24.534 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.534 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.534 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "31536000", 2024-05-03T18:47:24.534 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.534 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.534 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.535 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-03T18:47:24.535 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.535 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.535 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.535 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.535 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:24.535 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:24.535 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.535 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:24.535 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.535 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:24.535 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.535 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.535 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.535 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.535 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.535 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.536 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.536 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.536 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_recent_interval": { 2024-05-03T18:47:24.536 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_recent_interval", 2024-05-03T18:47:24.536 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-03T18:47:24.536 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.536 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.536 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "1209600", 2024-05-03T18:47:24.536 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.536 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.536 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.536 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-03T18:47:24.536 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.536 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.537 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.537 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.537 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.537 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.537 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:24.537 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "dashboard", 2024-05-03T18:47:24.537 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:24.537 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:24.537 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:24.537 INFO:teuthology.orchestra.run.smithi033.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-03T18:47:24.537 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-03T18:47:24.537 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.537 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.537 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.537 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-05-03T18:47:24.537 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.538 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.538 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.538 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.538 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.538 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.538 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.538 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.538 INFO:teuthology.orchestra.run.smithi033.stdout: "ALERTMANAGER_API_HOST": { 2024-05-03T18:47:24.538 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-03T18:47:24.538 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.538 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.538 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.538 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.538 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.539 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.539 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.539 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.539 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.539 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.539 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.539 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.539 INFO:teuthology.orchestra.run.smithi033.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-03T18:47:24.539 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-03T18:47:24.539 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.539 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.539 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.539 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:24.540 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.540 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.540 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.540 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.540 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.540 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.540 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.540 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.540 INFO:teuthology.orchestra.run.smithi033.stdout: "AUDIT_API_ENABLED": { 2024-05-03T18:47:24.540 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "AUDIT_API_ENABLED", 2024-05-03T18:47:24.540 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.540 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.540 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.540 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.541 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.541 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.541 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.541 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.541 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.541 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.541 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.541 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.541 INFO:teuthology.orchestra.run.smithi033.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-03T18:47:24.541 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-03T18:47:24.541 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.541 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.541 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.542 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:24.542 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.542 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.542 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.542 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.542 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.542 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.542 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.542 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.542 INFO:teuthology.orchestra.run.smithi033.stdout: "ENABLE_BROWSABLE_API": { 2024-05-03T18:47:24.543 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-03T18:47:24.543 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.543 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.543 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.543 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:24.543 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.543 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.543 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.543 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.543 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.543 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.543 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.543 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.543 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-03T18:47:24.543 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-03T18:47:24.544 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.544 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.544 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.544 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:24.544 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.544 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.544 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.544 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.544 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.544 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.544 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.544 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.544 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-03T18:47:24.544 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-03T18:47:24.545 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.545 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.545 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.545 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:24.545 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.545 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.545 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.545 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.545 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.545 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.545 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.545 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.545 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-03T18:47:24.545 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-03T18:47:24.546 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.546 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.546 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.546 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:24.546 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.546 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.546 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.546 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.546 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.546 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.546 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.546 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.546 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-03T18:47:24.547 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-03T18:47:24.547 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.547 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.547 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.547 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:24.547 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.547 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.547 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.547 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.547 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.547 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.547 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.547 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.547 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-03T18:47:24.548 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-03T18:47:24.548 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.548 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.548 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.548 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:24.548 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.548 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.548 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.548 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.548 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.548 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.548 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.548 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.548 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-03T18:47:24.549 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-03T18:47:24.549 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.549 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.549 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.549 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:24.549 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.549 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.549 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.549 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.549 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.549 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.549 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.549 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.550 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-03T18:47:24.550 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-03T18:47:24.550 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.550 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.550 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.550 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:24.550 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.550 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.550 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.550 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.550 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.550 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.550 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.551 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.551 INFO:teuthology.orchestra.run.smithi033.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-03T18:47:24.551 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-03T18:47:24.551 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.551 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.551 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.551 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.551 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.551 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.551 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.551 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.551 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.551 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.551 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.552 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.552 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_API_PASSWORD": { 2024-05-03T18:47:24.552 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-03T18:47:24.552 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.552 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.552 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.552 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "admin", 2024-05-03T18:47:24.552 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.552 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.552 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.552 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.552 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.552 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.552 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.552 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.552 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-03T18:47:24.553 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-03T18:47:24.553 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.553 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.553 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.553 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:24.553 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.553 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.553 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.553 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.553 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.553 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.553 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.553 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.553 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_API_URL": { 2024-05-03T18:47:24.553 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_API_URL", 2024-05-03T18:47:24.554 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.554 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.554 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.554 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.554 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.554 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.554 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.554 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.554 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.554 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.554 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.554 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.554 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_API_USERNAME": { 2024-05-03T18:47:24.554 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-03T18:47:24.554 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.555 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.555 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.555 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "admin", 2024-05-03T18:47:24.555 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.555 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.555 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.555 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.555 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.555 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.555 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.555 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.555 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-03T18:47:24.555 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-03T18:47:24.555 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.555 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.555 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.556 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.556 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.556 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.556 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.556 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.556 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.556 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.556 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.556 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.556 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-03T18:47:24.556 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-03T18:47:24.556 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.556 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.556 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.556 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.556 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.556 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.557 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.557 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.557 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.557 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.557 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.557 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.557 INFO:teuthology.orchestra.run.smithi033.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-03T18:47:24.557 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-03T18:47:24.557 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.557 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.557 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.557 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:24.557 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.557 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.557 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.557 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.558 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.558 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.558 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.558 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.558 INFO:teuthology.orchestra.run.smithi033.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-03T18:47:24.558 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-03T18:47:24.558 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.558 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.558 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.558 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.558 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.558 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.558 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.558 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.559 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.559 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.559 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.559 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.559 INFO:teuthology.orchestra.run.smithi033.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-03T18:47:24.559 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-03T18:47:24.559 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.559 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.559 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.559 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "[]", 2024-05-03T18:47:24.559 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.559 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.559 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.559 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.559 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.559 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.560 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.560 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.560 INFO:teuthology.orchestra.run.smithi033.stdout: "MULTICLUSTER_CONFIG": { 2024-05-03T18:47:24.560 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-03T18:47:24.560 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.560 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.560 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.560 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "{}", 2024-05-03T18:47:24.560 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.560 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.560 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.560 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.560 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.560 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.561 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.561 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.561 INFO:teuthology.orchestra.run.smithi033.stdout: "PROMETHEUS_API_HOST": { 2024-05-03T18:47:24.561 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-03T18:47:24.561 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.561 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.562 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.562 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.562 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.562 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.562 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.562 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.562 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.562 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.562 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.562 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.562 INFO:teuthology.orchestra.run.smithi033.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-03T18:47:24.562 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-03T18:47:24.562 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.563 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.563 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.563 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:24.563 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.563 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.563 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.563 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.563 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.563 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.563 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.563 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.563 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-03T18:47:24.563 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-03T18:47:24.563 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.563 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.564 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.564 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.564 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.564 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.564 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.564 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.564 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.564 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.564 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.564 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.564 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-03T18:47:24.564 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-03T18:47:24.564 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.564 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.564 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.565 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.565 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.565 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.565 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.565 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.565 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.565 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.565 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.565 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.565 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-03T18:47:24.565 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-03T18:47:24.565 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.566 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.566 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.566 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:24.566 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.566 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.566 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.566 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.566 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.566 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.566 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.566 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.566 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-03T18:47:24.566 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-03T18:47:24.567 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.567 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.567 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.567 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:24.567 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.567 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.567 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.567 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.567 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.567 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.567 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.567 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.567 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-03T18:47:24.567 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-03T18:47:24.568 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.568 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.568 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.568 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.568 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.568 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.568 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.568 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.568 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.568 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.568 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.569 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.569 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-03T18:47:24.569 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-03T18:47:24.569 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.569 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.569 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.569 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.569 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.569 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.569 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.569 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.569 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.570 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.570 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.570 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.570 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-03T18:47:24.570 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-03T18:47:24.570 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.570 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.570 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.570 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.570 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.570 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.571 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.571 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.571 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.571 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.571 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.571 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.571 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_ENABLED": { 2024-05-03T18:47:24.571 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-03T18:47:24.571 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.571 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.571 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.571 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:24.571 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.572 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.572 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.572 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.572 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.572 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.572 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.572 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.572 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-03T18:47:24.572 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-03T18:47:24.572 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.572 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.572 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.572 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-03T18:47:24.572 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.572 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.572 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.572 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.573 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.573 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.573 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.573 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.573 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-03T18:47:24.573 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-03T18:47:24.573 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.573 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.573 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.573 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-05-03T18:47:24.573 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.573 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.573 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.573 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.573 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.573 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.574 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.574 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.574 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-03T18:47:24.574 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-03T18:47:24.574 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.574 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.574 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.574 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8", 2024-05-03T18:47:24.574 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.574 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.574 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.574 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.574 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.574 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.575 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.575 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.575 INFO:teuthology.orchestra.run.smithi033.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-03T18:47:24.575 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-03T18:47:24.575 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.575 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.575 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.575 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "45", 2024-05-03T18:47:24.575 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.575 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.575 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.575 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.575 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.575 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.576 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.576 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.576 INFO:teuthology.orchestra.run.smithi033.stdout: "RGW_API_ACCESS_KEY": { 2024-05-03T18:47:24.576 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-03T18:47:24.576 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.576 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.576 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.576 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.577 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.577 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.577 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.577 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.577 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.577 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.577 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.577 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.577 INFO:teuthology.orchestra.run.smithi033.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-03T18:47:24.577 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-03T18:47:24.577 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.577 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.577 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.577 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "admin", 2024-05-03T18:47:24.577 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.577 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.578 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.578 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.578 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.578 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.578 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.578 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.578 INFO:teuthology.orchestra.run.smithi033.stdout: "RGW_API_SECRET_KEY": { 2024-05-03T18:47:24.578 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-03T18:47:24.578 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.578 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.579 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.579 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.579 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.579 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.579 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.579 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.579 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.580 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.580 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.580 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.580 INFO:teuthology.orchestra.run.smithi033.stdout: "RGW_API_SSL_VERIFY": { 2024-05-03T18:47:24.580 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-03T18:47:24.580 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.580 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.581 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.581 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:24.581 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.581 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.581 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.581 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.581 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.581 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.581 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.581 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.581 INFO:teuthology.orchestra.run.smithi033.stdout: "UNSAFE_TLS_v1_2": { 2024-05-03T18:47:24.581 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-03T18:47:24.582 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.582 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.582 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.582 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.582 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.582 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.582 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.582 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.582 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.582 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.583 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.583 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.583 INFO:teuthology.orchestra.run.smithi033.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-03T18:47:24.583 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-03T18:47:24.583 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.583 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.583 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.583 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:24.583 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.583 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.584 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.584 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.584 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.584 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.584 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.584 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.584 INFO:teuthology.orchestra.run.smithi033.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-03T18:47:24.584 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-03T18:47:24.584 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.584 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.584 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.584 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-05-03T18:47:24.584 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.584 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.584 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.584 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.585 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.585 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.585 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.585 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.585 INFO:teuthology.orchestra.run.smithi033.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-03T18:47:24.585 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-03T18:47:24.585 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.585 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.585 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.585 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5", 2024-05-03T18:47:24.585 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.585 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.585 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.585 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.585 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.585 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.586 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.586 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.586 INFO:teuthology.orchestra.run.smithi033.stdout: "cross_origin_url": { 2024-05-03T18:47:24.586 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cross_origin_url", 2024-05-03T18:47:24.586 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.586 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.586 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.586 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.586 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.586 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.586 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.586 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.586 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.586 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.586 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.587 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.587 INFO:teuthology.orchestra.run.smithi033.stdout: "crt_file": { 2024-05-03T18:47:24.587 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crt_file", 2024-05-03T18:47:24.587 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.587 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.587 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.587 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.587 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.587 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.587 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.587 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.587 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.587 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.587 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.587 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.587 INFO:teuthology.orchestra.run.smithi033.stdout: "debug": { 2024-05-03T18:47:24.588 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "debug", 2024-05-03T18:47:24.588 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.588 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.588 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.588 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.588 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.588 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.588 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.588 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Enable/disable debug options", 2024-05-03T18:47:24.588 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.588 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.588 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.588 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.588 INFO:teuthology.orchestra.run.smithi033.stdout: "jwt_token_ttl": { 2024-05-03T18:47:24.588 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "jwt_token_ttl", 2024-05-03T18:47:24.588 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.588 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.589 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.589 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "28800", 2024-05-03T18:47:24.589 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.589 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.589 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.589 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.589 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.589 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.589 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.589 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.589 INFO:teuthology.orchestra.run.smithi033.stdout: "key_file": { 2024-05-03T18:47:24.589 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "key_file", 2024-05-03T18:47:24.590 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.590 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.590 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.590 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.590 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.590 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.590 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.590 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.590 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.590 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.590 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.590 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.590 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:24.590 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:24.591 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.591 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.591 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.591 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.591 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.591 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.591 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.591 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.591 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.591 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.591 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.591 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.592 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.592 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.592 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.592 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.592 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.592 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.592 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.592 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:24.592 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:24.592 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.592 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.593 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.593 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.593 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.593 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.593 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.593 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.593 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.593 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.593 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.593 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.593 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:24.593 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:24.593 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.594 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.594 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.594 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:24.594 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.594 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.594 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.594 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.594 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.594 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.594 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.594 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.594 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.594 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.594 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.595 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.595 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.595 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.595 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.595 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:24.595 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:24.595 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.596 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.596 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.596 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.596 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.596 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.596 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.597 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.597 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.597 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.597 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.597 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.597 INFO:teuthology.orchestra.run.smithi033.stdout: "motd": { 2024-05-03T18:47:24.597 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "motd", 2024-05-03T18:47:24.598 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.598 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.598 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.598 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.598 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.598 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.598 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.598 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "The message of the day", 2024-05-03T18:47:24.598 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.598 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.598 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.598 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.598 INFO:teuthology.orchestra.run.smithi033.stdout: "redirect_resolve_ip_addr": { 2024-05-03T18:47:24.599 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "redirect_resolve_ip_addr", 2024-05-03T18:47:24.599 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.599 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.599 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.599 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.599 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.599 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.599 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.600 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.600 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.600 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.600 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.600 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.600 INFO:teuthology.orchestra.run.smithi033.stdout: "server_addr": { 2024-05-03T18:47:24.600 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_addr", 2024-05-03T18:47:24.600 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.600 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.600 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.600 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "::", 2024-05-03T18:47:24.600 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.600 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.600 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.600 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.600 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.601 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.601 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.601 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.601 INFO:teuthology.orchestra.run.smithi033.stdout: "server_port": { 2024-05-03T18:47:24.601 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_port", 2024-05-03T18:47:24.601 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.601 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.601 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.601 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8080", 2024-05-03T18:47:24.601 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.601 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.601 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.601 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.601 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.601 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.601 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.602 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.602 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:24.602 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:24.602 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.602 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:24.602 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.602 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:24.602 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.602 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.602 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.602 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.602 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.602 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.602 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.602 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.602 INFO:teuthology.orchestra.run.smithi033.stdout: "ssl": { 2024-05-03T18:47:24.603 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ssl", 2024-05-03T18:47:24.603 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.603 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.603 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.603 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:24.603 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.603 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.603 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.603 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.603 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.603 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.603 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.603 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.603 INFO:teuthology.orchestra.run.smithi033.stdout: "ssl_server_port": { 2024-05-03T18:47:24.603 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ssl_server_port", 2024-05-03T18:47:24.603 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.604 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.604 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.604 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8443", 2024-05-03T18:47:24.604 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.604 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.604 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.604 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.604 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.604 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.604 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.604 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.604 INFO:teuthology.orchestra.run.smithi033.stdout: "standby_behaviour": { 2024-05-03T18:47:24.604 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "standby_behaviour", 2024-05-03T18:47:24.604 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.604 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.604 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.604 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "redirect", 2024-05-03T18:47:24.605 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.605 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.605 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.605 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.605 INFO:teuthology.orchestra.run.smithi033.stdout: "redirect" 2024-05-03T18:47:24.605 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.605 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.605 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.605 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.605 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.605 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.605 INFO:teuthology.orchestra.run.smithi033.stdout: "standby_error_status_code": { 2024-05-03T18:47:24.605 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "standby_error_status_code", 2024-05-03T18:47:24.605 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.605 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.605 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.605 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "500", 2024-05-03T18:47:24.606 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "400", 2024-05-03T18:47:24.606 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "599", 2024-05-03T18:47:24.606 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.606 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.606 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.606 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.606 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.606 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.606 INFO:teuthology.orchestra.run.smithi033.stdout: "url_prefix": { 2024-05-03T18:47:24.606 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "url_prefix", 2024-05-03T18:47:24.606 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.606 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.606 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.606 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.606 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.606 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.606 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.607 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.607 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.607 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.607 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.607 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.607 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.607 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.607 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:24.607 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "devicehealth", 2024-05-03T18:47:24.607 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:24.607 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:24.607 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:24.607 INFO:teuthology.orchestra.run.smithi033.stdout: "enable_monitoring": { 2024-05-03T18:47:24.607 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "enable_monitoring", 2024-05-03T18:47:24.607 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.607 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.607 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.608 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:24.608 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.608 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.608 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.608 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "monitor device health metrics", 2024-05-03T18:47:24.608 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.608 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.608 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.608 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.608 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:24.608 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:24.608 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.608 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.608 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.608 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.608 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.609 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.609 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.609 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.609 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.609 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.609 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.609 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.609 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.609 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.609 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.609 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.609 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.609 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.609 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.609 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:24.609 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:24.610 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.610 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.610 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.610 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.610 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.610 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.610 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.610 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.610 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.610 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.610 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.611 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.611 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:24.611 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:24.611 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.611 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.611 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.611 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:24.611 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.611 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.611 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.611 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.611 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.611 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.611 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.612 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.612 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.612 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.612 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.612 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.612 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.612 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.612 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.612 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:24.612 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:24.612 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.612 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.612 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.612 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.612 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.612 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.613 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.613 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.613 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.613 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.613 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.613 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.613 INFO:teuthology.orchestra.run.smithi033.stdout: "mark_out_threshold": { 2024-05-03T18:47:24.613 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mark_out_threshold", 2024-05-03T18:47:24.613 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-03T18:47:24.613 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.613 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.613 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "2419200", 2024-05-03T18:47:24.613 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.613 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.613 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.613 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-03T18:47:24.614 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.614 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.614 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.614 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.614 INFO:teuthology.orchestra.run.smithi033.stdout: "pool_name": { 2024-05-03T18:47:24.614 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "pool_name", 2024-05-03T18:47:24.614 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.614 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.614 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.614 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "device_health_metrics", 2024-05-03T18:47:24.614 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.614 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.614 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.614 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-03T18:47:24.614 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.615 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.615 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.615 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.615 INFO:teuthology.orchestra.run.smithi033.stdout: "retention_period": { 2024-05-03T18:47:24.615 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "retention_period", 2024-05-03T18:47:24.615 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-03T18:47:24.615 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.615 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.615 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "15552000", 2024-05-03T18:47:24.615 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.615 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.615 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.615 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how long to retain device health metrics", 2024-05-03T18:47:24.615 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.615 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.616 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.616 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.616 INFO:teuthology.orchestra.run.smithi033.stdout: "scrape_frequency": { 2024-05-03T18:47:24.616 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "scrape_frequency", 2024-05-03T18:47:24.616 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-03T18:47:24.616 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.616 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.616 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "86400", 2024-05-03T18:47:24.616 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.616 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.616 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.616 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-03T18:47:24.616 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.616 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.616 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.616 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.617 INFO:teuthology.orchestra.run.smithi033.stdout: "self_heal": { 2024-05-03T18:47:24.617 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "self_heal", 2024-05-03T18:47:24.617 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.617 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.617 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.617 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:24.617 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.617 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.617 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.617 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-03T18:47:24.617 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.617 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.617 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.617 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.617 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-05-03T18:47:24.617 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-05-03T18:47:24.618 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-03T18:47:24.618 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.618 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.618 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-05-03T18:47:24.618 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.618 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.618 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.618 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to wake up and check device health", 2024-05-03T18:47:24.618 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.618 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.618 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.618 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.618 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:24.619 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:24.619 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.619 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:24.619 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.619 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:24.619 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.619 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.619 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.619 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.619 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.619 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.619 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.619 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.619 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_threshold": { 2024-05-03T18:47:24.619 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_threshold", 2024-05-03T18:47:24.619 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-03T18:47:24.620 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.620 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.620 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "7257600", 2024-05-03T18:47:24.620 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.620 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.620 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.620 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-03T18:47:24.620 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.620 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.620 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.620 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.620 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.620 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.620 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:24.621 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "diskprediction_local", 2024-05-03T18:47:24.621 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:24.621 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:24.621 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:24.621 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:24.621 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:24.621 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.621 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.621 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.621 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.621 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.621 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.621 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.621 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.621 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.621 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.621 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.622 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.622 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.622 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.622 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.622 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.622 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.622 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.622 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.622 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:24.622 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:24.622 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.622 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.622 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.622 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.622 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.623 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.623 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.623 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.623 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.623 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.623 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.623 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.623 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:24.623 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:24.623 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.623 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.623 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.623 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:24.623 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.623 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.623 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.623 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.624 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.624 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.624 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.624 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.624 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.624 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.624 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.624 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.624 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.624 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.624 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.624 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:24.624 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:24.624 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.624 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.624 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.625 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.625 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.625 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.625 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.625 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.625 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.625 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.625 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.625 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.625 INFO:teuthology.orchestra.run.smithi033.stdout: "predict_interval": { 2024-05-03T18:47:24.625 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "predict_interval", 2024-05-03T18:47:24.625 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.625 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.625 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.625 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "86400", 2024-05-03T18:47:24.625 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.626 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.626 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.626 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.626 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.626 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.626 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.626 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.626 INFO:teuthology.orchestra.run.smithi033.stdout: "predictor_model": { 2024-05-03T18:47:24.626 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "predictor_model", 2024-05-03T18:47:24.626 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.626 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.626 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.626 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "prophetstor", 2024-05-03T18:47:24.626 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.626 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.627 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.627 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.627 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.627 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.627 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.627 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.627 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-05-03T18:47:24.627 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-05-03T18:47:24.627 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.627 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.627 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.627 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-05-03T18:47:24.627 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.627 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.627 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.627 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.628 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.628 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.628 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.628 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.628 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:24.628 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:24.628 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.628 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:24.628 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.628 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:24.628 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.628 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.628 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.628 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.628 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.628 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.629 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.629 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.629 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.629 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.629 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:24.629 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "influx", 2024-05-03T18:47:24.629 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": false, 2024-05-03T18:47:24.629 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "influxdb python module not found", 2024-05-03T18:47:24.629 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:24.629 INFO:teuthology.orchestra.run.smithi033.stdout: "batch_size": { 2024-05-03T18:47:24.629 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "batch_size", 2024-05-03T18:47:24.629 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.629 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.629 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.629 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5000", 2024-05-03T18:47:24.630 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.630 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.630 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.630 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-03T18:47:24.630 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.630 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.630 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.630 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.630 INFO:teuthology.orchestra.run.smithi033.stdout: "database": { 2024-05-03T18:47:24.630 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "database", 2024-05-03T18:47:24.630 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.630 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.630 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.630 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "ceph", 2024-05-03T18:47:24.630 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.631 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.631 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.631 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-03T18:47:24.631 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.631 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.631 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.631 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.631 INFO:teuthology.orchestra.run.smithi033.stdout: "hostname": { 2024-05-03T18:47:24.631 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "hostname", 2024-05-03T18:47:24.631 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.631 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.631 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.631 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.631 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.631 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.631 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.632 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "InfluxDB server hostname", 2024-05-03T18:47:24.632 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.632 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.632 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.632 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.632 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-05-03T18:47:24.632 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-05-03T18:47:24.632 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-03T18:47:24.632 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.632 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.632 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "30", 2024-05-03T18:47:24.632 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "5", 2024-05-03T18:47:24.632 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.632 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.632 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-03T18:47:24.632 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.633 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.633 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.633 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.633 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:24.633 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:24.633 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.633 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.633 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.633 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.633 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.633 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.633 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.633 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.633 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.633 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.633 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.633 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.634 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.634 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.634 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.634 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.634 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.634 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.634 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.634 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:24.634 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:24.634 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.634 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.634 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.634 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.634 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.635 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.635 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.635 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.635 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.635 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.635 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.635 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.635 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:24.635 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:24.635 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.635 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.635 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.635 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:24.635 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.635 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.635 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.636 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.636 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.636 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.636 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.636 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.636 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.636 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.636 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.636 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.636 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.636 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.636 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.636 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:24.636 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:24.636 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.637 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.637 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.637 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.637 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.637 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.637 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.637 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.637 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.637 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.637 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.637 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.637 INFO:teuthology.orchestra.run.smithi033.stdout: "password": { 2024-05-03T18:47:24.637 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "password", 2024-05-03T18:47:24.637 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.637 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.638 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.638 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.638 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.638 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.638 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.638 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "password of InfluxDB server user", 2024-05-03T18:47:24.638 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.638 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.638 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.638 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.638 INFO:teuthology.orchestra.run.smithi033.stdout: "port": { 2024-05-03T18:47:24.638 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "port", 2024-05-03T18:47:24.638 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.638 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.639 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.639 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8086", 2024-05-03T18:47:24.639 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.639 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.639 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.639 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "InfluxDB server port", 2024-05-03T18:47:24.639 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.639 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.639 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.639 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.639 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:24.639 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:24.639 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.639 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:24.639 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.640 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:24.640 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.640 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.640 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.640 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.640 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.640 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.640 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.640 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.640 INFO:teuthology.orchestra.run.smithi033.stdout: "ssl": { 2024-05-03T18:47:24.640 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ssl", 2024-05-03T18:47:24.640 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.640 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.640 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.641 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "false", 2024-05-03T18:47:24.641 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.641 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.641 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.641 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-03T18:47:24.641 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.641 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.641 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.641 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.641 INFO:teuthology.orchestra.run.smithi033.stdout: "threads": { 2024-05-03T18:47:24.641 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "threads", 2024-05-03T18:47:24.641 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.641 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.642 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.642 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5", 2024-05-03T18:47:24.642 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1", 2024-05-03T18:47:24.642 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "32", 2024-05-03T18:47:24.642 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.642 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-03T18:47:24.642 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.642 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.642 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.642 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.642 INFO:teuthology.orchestra.run.smithi033.stdout: "username": { 2024-05-03T18:47:24.642 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "username", 2024-05-03T18:47:24.643 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.643 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.643 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.643 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.643 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.643 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.643 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.643 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "username of InfluxDB server user", 2024-05-03T18:47:24.643 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.643 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.643 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.644 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.644 INFO:teuthology.orchestra.run.smithi033.stdout: "verify_ssl": { 2024-05-03T18:47:24.644 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "verify_ssl", 2024-05-03T18:47:24.644 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.644 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.644 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.644 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "true", 2024-05-03T18:47:24.644 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.644 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.644 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.644 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-03T18:47:24.644 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.644 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.644 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.644 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.645 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.645 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.645 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:24.645 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "insights", 2024-05-03T18:47:24.645 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:24.645 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:24.645 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:24.645 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:24.645 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:24.645 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.645 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.645 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.645 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.645 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.645 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.646 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.646 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.646 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.646 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.646 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.646 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.646 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.646 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.646 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.646 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.646 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.646 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.646 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.646 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:24.647 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:24.647 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.647 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.647 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.647 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.647 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.647 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.647 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.647 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.647 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.647 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.647 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.647 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.647 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:24.647 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:24.648 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.648 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.648 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.648 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:24.648 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.648 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.648 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.648 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.648 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.648 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.648 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.648 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.648 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.648 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.648 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.649 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.649 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.649 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.649 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.649 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:24.649 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:24.649 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.649 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.649 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.649 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.649 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.649 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.649 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.649 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.649 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.649 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.650 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.650 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.650 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:24.650 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:24.650 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.650 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:24.650 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.650 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:24.650 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.650 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.650 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.650 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.650 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.651 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.651 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.651 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.651 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.651 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.651 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:24.651 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "iostat", 2024-05-03T18:47:24.651 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:24.651 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:24.651 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:24.651 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:24.651 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:24.651 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.651 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.651 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.652 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.652 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.652 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.652 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.652 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.652 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.652 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.652 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.652 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.652 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.652 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.652 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.652 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.652 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.652 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.652 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.652 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:24.653 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:24.653 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.653 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.653 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.653 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.653 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.653 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.653 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.653 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.653 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.653 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.653 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.653 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.653 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:24.654 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:24.654 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.654 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.654 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.654 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:24.654 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.654 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.654 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.654 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.654 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.654 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.654 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.654 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.654 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.655 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.655 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.655 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.655 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.655 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.655 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.655 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:24.655 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:24.655 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.655 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.655 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.655 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.656 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.656 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.656 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.656 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.656 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.656 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.656 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.656 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.656 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:24.656 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:24.656 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.656 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:24.656 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.656 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:24.657 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.657 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.657 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.657 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.657 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.657 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.657 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.657 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.657 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.657 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.657 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:24.657 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "k8sevents", 2024-05-03T18:47:24.657 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:24.657 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:24.658 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:24.658 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_event_retention_days": { 2024-05-03T18:47:24.658 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ceph_event_retention_days", 2024-05-03T18:47:24.658 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.658 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.658 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.658 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "7", 2024-05-03T18:47:24.658 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.658 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.658 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.658 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-03T18:47:24.658 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.658 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.658 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.658 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.659 INFO:teuthology.orchestra.run.smithi033.stdout: "config_check_secs": { 2024-05-03T18:47:24.659 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "config_check_secs", 2024-05-03T18:47:24.659 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.659 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.659 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.659 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-05-03T18:47:24.659 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "10", 2024-05-03T18:47:24.659 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.659 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.659 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-03T18:47:24.659 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.659 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.659 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.659 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.659 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:24.660 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:24.660 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.660 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.660 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.660 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.660 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.660 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.660 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.660 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.660 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.660 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.660 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.660 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.660 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.660 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.660 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.661 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.661 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.661 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.661 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.661 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:24.661 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:24.661 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.661 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.661 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.661 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.661 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.661 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.661 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.661 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.662 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.662 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.662 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.662 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.662 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:24.662 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:24.662 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.663 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.663 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.663 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:24.663 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.663 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.663 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.663 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.663 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.663 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.663 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.663 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.663 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.663 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.663 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.663 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.663 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.664 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.664 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.664 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:24.664 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:24.664 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.664 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.664 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.664 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.664 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.664 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.664 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.664 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.664 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.664 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.664 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.665 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.665 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:24.665 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:24.665 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.665 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:24.665 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.665 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:24.665 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.665 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.665 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.665 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.665 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.665 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.665 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.665 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.666 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.666 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.666 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:24.666 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "localpool", 2024-05-03T18:47:24.666 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:24.666 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:24.666 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:24.666 INFO:teuthology.orchestra.run.smithi033.stdout: "failure_domain": { 2024-05-03T18:47:24.666 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "failure_domain", 2024-05-03T18:47:24.666 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.666 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.666 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.666 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "host", 2024-05-03T18:47:24.666 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.666 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.666 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.667 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "failure domain for any created local pool", 2024-05-03T18:47:24.667 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-03T18:47:24.667 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.667 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.667 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.667 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:24.667 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:24.667 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.667 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.667 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.667 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.667 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.668 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.668 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.668 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.668 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.668 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.668 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.668 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.668 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.668 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.668 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.668 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.668 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.668 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.669 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.669 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:24.669 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:24.669 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.669 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.669 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.669 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.669 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.669 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.669 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.669 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.669 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.669 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.669 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.669 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.669 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:24.670 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:24.670 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.670 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.670 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.670 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:24.670 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.670 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.670 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.670 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.670 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.670 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.671 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.671 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.671 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.671 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.671 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.671 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.671 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.671 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.671 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.671 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:24.671 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:24.671 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.671 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.672 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.672 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.672 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.672 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.672 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.672 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.672 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.672 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.672 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.672 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.672 INFO:teuthology.orchestra.run.smithi033.stdout: "min_size": { 2024-05-03T18:47:24.672 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "min_size", 2024-05-03T18:47:24.672 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.672 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.672 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.672 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.673 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.673 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.673 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.673 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "default min_size for any created local pool", 2024-05-03T18:47:24.673 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-03T18:47:24.673 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.673 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.673 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.673 INFO:teuthology.orchestra.run.smithi033.stdout: "num_rep": { 2024-05-03T18:47:24.673 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "num_rep", 2024-05-03T18:47:24.673 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.673 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.673 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.673 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "3", 2024-05-03T18:47:24.674 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.674 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.674 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.674 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "default replica count for any created local pool", 2024-05-03T18:47:24.674 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.674 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.674 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.674 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.674 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_num": { 2024-05-03T18:47:24.674 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "pg_num", 2024-05-03T18:47:24.674 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.674 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.674 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.674 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "128", 2024-05-03T18:47:24.674 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.675 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.675 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.675 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "default pg_num for any created local pool", 2024-05-03T18:47:24.675 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.675 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.675 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.675 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.675 INFO:teuthology.orchestra.run.smithi033.stdout: "prefix": { 2024-05-03T18:47:24.675 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "prefix", 2024-05-03T18:47:24.675 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.675 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.675 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.675 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.675 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.675 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.676 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.676 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "name prefix for any created local pool", 2024-05-03T18:47:24.676 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.676 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.676 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.676 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.676 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:24.676 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:24.676 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.676 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:24.676 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.676 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:24.676 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.676 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.677 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.677 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.677 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.677 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.677 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.677 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.677 INFO:teuthology.orchestra.run.smithi033.stdout: "subtree": { 2024-05-03T18:47:24.677 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "subtree", 2024-05-03T18:47:24.677 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.677 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.677 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.677 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "rack", 2024-05-03T18:47:24.677 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.677 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.677 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.677 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-03T18:47:24.677 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-03T18:47:24.678 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.678 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.678 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.678 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.678 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.678 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:24.678 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mds_autoscaler", 2024-05-03T18:47:24.678 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:24.678 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:24.678 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:24.678 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:24.678 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:24.678 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.679 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.679 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.679 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.679 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.679 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.679 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.679 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.679 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.679 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.679 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.679 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.679 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.679 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.679 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.679 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.680 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.680 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.680 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.680 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:24.680 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:24.680 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.680 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.680 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.680 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.680 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.680 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.680 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.681 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.681 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.681 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.681 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.681 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.681 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:24.681 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:24.681 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.681 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.681 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.681 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:24.681 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.681 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.682 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.682 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.682 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.682 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.682 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.682 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.682 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.682 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.682 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.682 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.682 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.682 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.682 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.682 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:24.682 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:24.682 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.683 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.683 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.683 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.683 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.683 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.683 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.683 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.683 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.683 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.683 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.683 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.683 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:24.683 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:24.683 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.683 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:24.683 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.684 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:24.684 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.684 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.684 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.684 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.684 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.684 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.684 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.684 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.684 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.684 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.684 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:24.684 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mirroring", 2024-05-03T18:47:24.684 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:24.684 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:24.684 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:24.685 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:24.685 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:24.685 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.685 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.685 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.685 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.685 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.685 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.685 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.685 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.685 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.685 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.685 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.685 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.685 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.686 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.686 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.686 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.686 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.686 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.686 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.686 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:24.686 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:24.686 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.686 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.686 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.686 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.686 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.687 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.687 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.687 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.687 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.687 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.687 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.687 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.687 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:24.687 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:24.687 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.687 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.687 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.687 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:24.687 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.688 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.688 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.688 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.688 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.688 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.688 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.688 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.688 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.688 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.688 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.688 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.688 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.688 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.688 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.688 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:24.688 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:24.688 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.689 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.689 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.689 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.689 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.689 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.689 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.689 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.689 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.689 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.689 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.689 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.689 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:24.689 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:24.689 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.689 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:24.690 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.690 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:24.690 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.690 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.690 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.690 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.690 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.690 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.690 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.690 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.690 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.690 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.690 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:24.690 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "nfs", 2024-05-03T18:47:24.691 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:24.691 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:24.691 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:24.691 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:24.691 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:24.691 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.691 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.691 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.691 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.691 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.691 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.691 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.691 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.691 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.691 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.691 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.692 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.692 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.692 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.692 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.692 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.692 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.692 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.692 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.692 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:24.692 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:24.692 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.692 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.692 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.692 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.692 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.693 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.693 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.693 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.693 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.693 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.693 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.693 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.693 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:24.693 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:24.693 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.693 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.693 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.693 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:24.693 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.693 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.693 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.694 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.694 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.694 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.694 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.694 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.694 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.694 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.694 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.694 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.694 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.694 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.694 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.694 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:24.694 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:24.694 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.694 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.695 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.695 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.695 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.695 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.695 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.695 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.695 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.695 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.695 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.695 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.695 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:24.695 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:24.695 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.695 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:24.695 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.696 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:24.696 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.696 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.696 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.696 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.696 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.696 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.696 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.696 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.696 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.696 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.696 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:24.696 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "orchestrator", 2024-05-03T18:47:24.696 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:24.696 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:24.696 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:24.697 INFO:teuthology.orchestra.run.smithi033.stdout: "fail_fs": { 2024-05-03T18:47:24.697 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "fail_fs", 2024-05-03T18:47:24.697 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.697 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.697 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.697 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.697 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.697 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.697 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.697 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-03T18:47:24.697 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.697 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.697 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.697 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.697 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:24.697 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:24.698 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.698 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.698 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.698 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.698 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.698 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.698 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.698 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.698 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.698 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.698 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.698 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.698 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.699 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.699 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.699 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.699 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.699 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.699 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.699 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:24.699 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:24.699 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.699 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.699 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.699 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.699 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.699 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.699 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.699 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.700 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.700 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.700 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.700 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.700 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:24.700 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:24.700 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.700 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.700 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.700 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:24.700 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.700 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.700 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.700 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.700 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.700 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.701 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.701 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.701 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.701 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.701 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.701 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.701 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.701 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.701 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.701 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:24.701 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:24.701 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.701 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.701 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.701 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.702 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.702 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.702 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.702 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.702 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.702 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.702 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.702 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.702 INFO:teuthology.orchestra.run.smithi033.stdout: "orchestrator": { 2024-05-03T18:47:24.702 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "orchestrator", 2024-05-03T18:47:24.702 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.702 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.702 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.702 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.703 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.703 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.703 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.703 INFO:teuthology.orchestra.run.smithi033.stdout: "cephadm", 2024-05-03T18:47:24.703 INFO:teuthology.orchestra.run.smithi033.stdout: "rook", 2024-05-03T18:47:24.703 INFO:teuthology.orchestra.run.smithi033.stdout: "test_orchestrator" 2024-05-03T18:47:24.703 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.703 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Orchestrator backend", 2024-05-03T18:47:24.703 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.703 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.703 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.703 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.703 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:24.703 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:24.703 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.704 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:24.704 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.704 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:24.704 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.704 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.704 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.704 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.704 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.704 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.704 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.704 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.704 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.704 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.704 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:24.704 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "osd_perf_query", 2024-05-03T18:47:24.704 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:24.705 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:24.705 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:24.705 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:24.705 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:24.705 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.705 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.705 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.705 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.705 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.705 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.705 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.705 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.705 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.705 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.705 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.705 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.706 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.706 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.706 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.706 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.706 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.706 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.706 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.706 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:24.706 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:24.706 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.706 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.706 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.706 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.706 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.706 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.707 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.707 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.707 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.707 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.707 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.707 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.707 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:24.707 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:24.707 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.707 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.707 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.707 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:24.707 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.707 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.708 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.708 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.708 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.708 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.708 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.708 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.708 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.708 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.708 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.708 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.708 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.708 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.708 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.708 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:24.708 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:24.709 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.709 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.709 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.709 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.709 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.709 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.709 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.709 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.709 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.709 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.709 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.709 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.709 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:24.709 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:24.710 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.710 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:24.710 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.710 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:24.710 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.710 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.710 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.710 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.710 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.710 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.710 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.710 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.710 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.710 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.710 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:24.711 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "osd_support", 2024-05-03T18:47:24.711 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:24.711 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:24.711 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:24.711 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:24.711 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:24.711 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.711 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.711 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.711 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.711 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.711 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.711 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.711 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.712 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.712 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.712 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.712 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.712 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.712 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.712 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.712 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.712 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.712 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.712 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.712 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:24.712 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:24.712 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.712 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.713 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.713 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.713 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.713 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.713 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.713 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.713 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.713 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.713 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.713 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.713 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:24.713 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:24.713 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.713 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.713 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.714 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:24.714 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.714 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.714 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.714 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.714 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.714 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.714 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.714 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.714 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.714 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.714 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.714 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.714 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.714 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.715 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.715 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:24.715 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:24.715 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.715 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.715 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.715 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.715 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.715 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.715 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.715 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.715 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.715 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.715 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.716 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.716 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:24.716 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:24.716 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.716 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:24.716 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.716 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:24.716 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.716 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.716 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.716 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.716 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.716 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.716 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.717 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.717 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.717 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.717 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:24.717 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "pg_autoscaler", 2024-05-03T18:47:24.717 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:24.717 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:24.717 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:24.717 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:24.717 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:24.717 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.717 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.717 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.717 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.717 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.718 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.718 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.718 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.718 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.718 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.718 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.718 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.718 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.718 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.718 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.718 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.718 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.718 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.718 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.719 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:24.719 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:24.719 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.719 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.719 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.719 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.719 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.719 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.719 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.719 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.719 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.719 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.719 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.719 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.719 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:24.720 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:24.720 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.720 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.720 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.720 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:24.720 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.720 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.720 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.720 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.720 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.720 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.720 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.720 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.720 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.720 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.721 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.721 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.721 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.721 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.721 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.721 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:24.721 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:24.721 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.721 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.721 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.721 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.721 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.721 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.721 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.722 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.722 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.722 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.722 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.722 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.722 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-05-03T18:47:24.722 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-05-03T18:47:24.722 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-03T18:47:24.722 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.722 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.722 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-05-03T18:47:24.722 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.722 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.722 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.723 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.723 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.723 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.723 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.723 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.723 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:24.723 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:24.723 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.723 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:24.723 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.723 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:24.723 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.723 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.723 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.724 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.724 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.724 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.724 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.724 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.724 INFO:teuthology.orchestra.run.smithi033.stdout: "threshold": { 2024-05-03T18:47:24.724 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "threshold", 2024-05-03T18:47:24.724 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-05-03T18:47:24.724 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.724 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.724 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "3.0", 2024-05-03T18:47:24.724 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1.0", 2024-05-03T18:47:24.724 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.724 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.725 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "scaling threshold", 2024-05-03T18:47:24.725 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-03T18:47:24.725 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.725 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.725 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.725 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.725 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.725 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:24.725 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "progress", 2024-05-03T18:47:24.725 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:24.725 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:24.725 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:24.726 INFO:teuthology.orchestra.run.smithi033.stdout: "allow_pg_recovery_event": { 2024-05-03T18:47:24.726 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "allow_pg_recovery_event", 2024-05-03T18:47:24.726 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.726 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.726 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.726 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.726 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.726 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.726 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.726 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-03T18:47:24.726 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.726 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.727 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.727 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.727 INFO:teuthology.orchestra.run.smithi033.stdout: "enabled": { 2024-05-03T18:47:24.727 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "enabled", 2024-05-03T18:47:24.727 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.727 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.727 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.727 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:24.727 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.727 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.727 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.727 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.727 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.727 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.728 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.728 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.728 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:24.728 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:24.728 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.728 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.728 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.728 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.728 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.728 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.728 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.728 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.728 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.728 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.728 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.728 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.729 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.729 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.729 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.729 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.729 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.729 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.729 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.729 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:24.729 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:24.729 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.729 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.729 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.729 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.729 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.729 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.729 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.730 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.730 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.730 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.730 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.730 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.730 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:24.730 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:24.730 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.730 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.730 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.730 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:24.730 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.730 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.730 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.731 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.731 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.731 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.731 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.731 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.731 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.731 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.731 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.731 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.731 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.731 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.731 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.731 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:24.731 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:24.731 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.732 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.732 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.732 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.732 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.732 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.732 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.732 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.732 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.732 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.732 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.732 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.732 INFO:teuthology.orchestra.run.smithi033.stdout: "max_completed_events": { 2024-05-03T18:47:24.732 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_completed_events", 2024-05-03T18:47:24.732 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.732 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.733 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.733 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "50", 2024-05-03T18:47:24.733 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.733 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.733 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.733 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "number of past completed events to remember", 2024-05-03T18:47:24.733 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.733 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.733 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.733 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.733 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-05-03T18:47:24.733 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-05-03T18:47:24.733 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-03T18:47:24.733 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.733 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.734 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5", 2024-05-03T18:47:24.734 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.734 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.734 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.734 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how long the module is going to sleep", 2024-05-03T18:47:24.734 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.734 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.734 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.734 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.734 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:24.734 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:24.734 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.734 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:24.734 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.735 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:24.735 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.735 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.735 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.735 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.735 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.735 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.735 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.735 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.735 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.735 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.735 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:24.735 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "prometheus", 2024-05-03T18:47:24.735 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:24.735 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:24.736 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:24.736 INFO:teuthology.orchestra.run.smithi033.stdout: "cache": { 2024-05-03T18:47:24.736 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cache", 2024-05-03T18:47:24.736 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.736 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.736 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.736 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:24.736 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.736 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.736 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.736 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.736 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.736 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.736 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.737 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.737 INFO:teuthology.orchestra.run.smithi033.stdout: "exclude_perf_counters": { 2024-05-03T18:47:24.737 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "exclude_perf_counters", 2024-05-03T18:47:24.737 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.737 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.737 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.737 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:24.737 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.737 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.737 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.737 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-03T18:47:24.737 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-03T18:47:24.737 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.737 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.738 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.738 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:24.738 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:24.738 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.738 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.738 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.738 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.738 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.738 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.738 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.738 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.738 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.738 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.738 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.739 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.739 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.739 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.739 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.739 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.739 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.739 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.739 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.739 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:24.739 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:24.739 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.739 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.739 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.739 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.740 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.740 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.740 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.740 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.740 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.740 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.740 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.740 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.740 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:24.740 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:24.740 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.740 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.740 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.740 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:24.741 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.741 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.741 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.741 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.741 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.741 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.741 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.741 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.741 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.741 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.741 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.741 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.741 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.741 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.741 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.742 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:24.742 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:24.742 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.742 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.742 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.742 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.742 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.742 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.742 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.742 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.742 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.742 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.742 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.742 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.743 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_stats_pools": { 2024-05-03T18:47:24.743 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rbd_stats_pools", 2024-05-03T18:47:24.743 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.743 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.743 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.743 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.743 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.743 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.743 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.743 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.743 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.743 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.743 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.743 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.743 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-03T18:47:24.743 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-03T18:47:24.744 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.744 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.744 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.744 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "300", 2024-05-03T18:47:24.744 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.744 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.744 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.744 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.744 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.744 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.744 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.744 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.744 INFO:teuthology.orchestra.run.smithi033.stdout: "scrape_interval": { 2024-05-03T18:47:24.744 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "scrape_interval", 2024-05-03T18:47:24.744 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-05-03T18:47:24.744 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.745 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.745 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "15.0", 2024-05-03T18:47:24.745 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.745 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.745 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.745 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.745 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.745 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.745 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.745 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.745 INFO:teuthology.orchestra.run.smithi033.stdout: "server_addr": { 2024-05-03T18:47:24.745 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_addr", 2024-05-03T18:47:24.745 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.745 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.745 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.745 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "::", 2024-05-03T18:47:24.745 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.746 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.746 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.746 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-03T18:47:24.746 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.746 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.746 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.746 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.746 INFO:teuthology.orchestra.run.smithi033.stdout: "server_port": { 2024-05-03T18:47:24.746 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_port", 2024-05-03T18:47:24.746 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.746 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.746 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.746 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "9283", 2024-05-03T18:47:24.746 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.746 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.747 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.747 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-03T18:47:24.747 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.747 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.747 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.747 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.747 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:24.747 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:24.747 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.747 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:24.747 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.747 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:24.747 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.747 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.747 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.747 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.748 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.748 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.748 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.748 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.748 INFO:teuthology.orchestra.run.smithi033.stdout: "stale_cache_strategy": { 2024-05-03T18:47:24.748 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "stale_cache_strategy", 2024-05-03T18:47:24.748 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.748 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.748 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.748 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "log", 2024-05-03T18:47:24.748 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.748 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.748 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.748 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.748 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.748 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.749 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.749 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.749 INFO:teuthology.orchestra.run.smithi033.stdout: "standby_behaviour": { 2024-05-03T18:47:24.749 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "standby_behaviour", 2024-05-03T18:47:24.749 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.749 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.749 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.749 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "default", 2024-05-03T18:47:24.749 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.749 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.749 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.749 INFO:teuthology.orchestra.run.smithi033.stdout: "default", 2024-05-03T18:47:24.749 INFO:teuthology.orchestra.run.smithi033.stdout: "error" 2024-05-03T18:47:24.749 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.749 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.749 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.749 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.750 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.750 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.750 INFO:teuthology.orchestra.run.smithi033.stdout: "standby_error_status_code": { 2024-05-03T18:47:24.750 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "standby_error_status_code", 2024-05-03T18:47:24.750 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.750 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.750 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.750 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "500", 2024-05-03T18:47:24.750 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "400", 2024-05-03T18:47:24.750 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "599", 2024-05-03T18:47:24.750 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.750 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.750 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.750 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.750 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.750 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.751 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.751 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.751 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:24.751 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rbd_support", 2024-05-03T18:47:24.751 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:24.751 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:24.751 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:24.751 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:24.751 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:24.751 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.751 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.751 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.751 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.751 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.751 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.751 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.752 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.752 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.752 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.752 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.752 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.752 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.752 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.752 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.752 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.752 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.752 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.752 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.752 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:24.752 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:24.752 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.752 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.753 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.753 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.753 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.753 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.753 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.753 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.753 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.753 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.753 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.753 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.753 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:24.753 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:24.753 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.753 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.753 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.753 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:24.753 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.754 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.754 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.754 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.754 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.754 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.754 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.754 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.754 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.754 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.754 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.754 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.754 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.754 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.754 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.755 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:24.755 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:24.755 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.755 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.755 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.755 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.755 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.755 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.755 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.755 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.755 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.755 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.755 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.755 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.755 INFO:teuthology.orchestra.run.smithi033.stdout: "max_concurrent_snap_create": { 2024-05-03T18:47:24.756 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_concurrent_snap_create", 2024-05-03T18:47:24.756 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.756 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.756 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.756 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-05-03T18:47:24.756 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.756 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.756 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.756 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.756 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.756 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.756 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.756 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.756 INFO:teuthology.orchestra.run.smithi033.stdout: "mirror_snapshot_schedule": { 2024-05-03T18:47:24.756 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mirror_snapshot_schedule", 2024-05-03T18:47:24.757 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.757 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.757 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.757 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.757 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.757 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.757 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.757 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.757 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.757 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.757 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.757 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.757 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:24.757 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:24.757 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.758 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:24.758 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.758 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:24.758 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.758 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.758 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.758 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.758 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.758 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.758 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.758 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.758 INFO:teuthology.orchestra.run.smithi033.stdout: "trash_purge_schedule": { 2024-05-03T18:47:24.758 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "trash_purge_schedule", 2024-05-03T18:47:24.758 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.759 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.759 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.759 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.759 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.759 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.759 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.759 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.759 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.759 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.759 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.759 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.759 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.759 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.759 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:24.759 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "restful", 2024-05-03T18:47:24.760 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:24.760 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:24.760 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:24.760 INFO:teuthology.orchestra.run.smithi033.stdout: "enable_auth": { 2024-05-03T18:47:24.760 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "enable_auth", 2024-05-03T18:47:24.760 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.760 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.760 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.760 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:24.760 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.760 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.760 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.760 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.760 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.760 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.760 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.761 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.761 INFO:teuthology.orchestra.run.smithi033.stdout: "key_file": { 2024-05-03T18:47:24.761 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "key_file", 2024-05-03T18:47:24.761 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.761 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.761 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.761 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.761 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.761 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.761 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.761 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.761 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.761 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.761 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.762 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.762 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:24.762 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:24.762 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.762 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.762 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.762 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.762 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.762 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.762 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.762 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.762 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.762 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.763 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.763 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.763 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.763 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.763 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.763 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.763 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.763 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.763 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.763 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:24.763 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:24.763 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.763 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.763 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.763 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.764 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.764 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.764 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.764 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.764 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.764 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.764 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.764 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.764 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:24.764 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:24.764 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.764 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.764 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.764 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:24.765 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.765 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.765 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.765 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.765 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.765 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.765 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.765 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.765 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.765 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.765 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.765 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.765 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.765 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.765 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.766 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:24.766 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:24.766 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.766 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.766 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.766 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.766 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.766 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.766 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.766 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.766 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.766 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.766 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.766 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.767 INFO:teuthology.orchestra.run.smithi033.stdout: "server_addr": { 2024-05-03T18:47:24.767 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_addr", 2024-05-03T18:47:24.767 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.767 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.767 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.767 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.767 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.767 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.767 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.767 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.767 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.767 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.767 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.767 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.767 INFO:teuthology.orchestra.run.smithi033.stdout: "server_port": { 2024-05-03T18:47:24.768 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_port", 2024-05-03T18:47:24.768 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.768 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.768 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.768 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.768 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.768 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.768 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.768 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.768 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.768 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.768 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.768 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.768 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:24.768 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:24.769 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.769 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:24.769 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.769 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:24.769 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.769 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.769 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.769 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.769 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.769 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.769 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.769 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.769 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.769 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.770 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:24.770 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rgw", 2024-05-03T18:47:24.770 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:24.770 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:24.770 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:24.770 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:24.770 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:24.770 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.770 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.770 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.770 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.770 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.770 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.770 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.771 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.771 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.771 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.771 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.771 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.771 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.771 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.771 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.771 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.771 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.771 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.771 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.771 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:24.771 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:24.772 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.772 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.772 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.772 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.772 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.772 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.772 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.772 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.772 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.772 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.772 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.772 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.772 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:24.773 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:24.773 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.773 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.773 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.773 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:24.773 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.773 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.773 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.773 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.773 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.773 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.773 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.773 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.773 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.773 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.773 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.773 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.774 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.774 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.774 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.774 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:24.774 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:24.774 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.774 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.774 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.774 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.774 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.774 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.774 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.774 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.774 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.774 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.775 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.775 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.775 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:24.775 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:24.775 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.775 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:24.775 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.775 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:24.775 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.775 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.775 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.775 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.775 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.775 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.775 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.775 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.775 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.776 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.776 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:24.776 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rook", 2024-05-03T18:47:24.776 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:24.776 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:24.776 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:24.776 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:24.776 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:24.776 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.776 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.776 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.776 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.776 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.776 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.776 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.776 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.777 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.777 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.777 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.777 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.777 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.777 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.777 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.777 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.777 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.777 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.777 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.777 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:24.777 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:24.777 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.777 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.778 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.778 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.778 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.778 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.778 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.778 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.778 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.778 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.778 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.778 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.778 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:24.778 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:24.778 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.778 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.778 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.779 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:24.779 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.779 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.779 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.779 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.779 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.779 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.779 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.779 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.779 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.779 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.779 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.779 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.779 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.779 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.779 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.779 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:24.780 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:24.780 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.780 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.780 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.780 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.780 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.780 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.780 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.780 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.780 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.780 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.780 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.780 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.780 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:24.780 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:24.780 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.781 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:24.781 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.781 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:24.781 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.781 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.781 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.781 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.781 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.781 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.781 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.781 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.781 INFO:teuthology.orchestra.run.smithi033.stdout: "storage_class": { 2024-05-03T18:47:24.781 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "storage_class", 2024-05-03T18:47:24.781 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.781 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.781 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.781 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "local", 2024-05-03T18:47:24.782 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.782 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.782 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.782 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-03T18:47:24.782 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.782 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.782 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.782 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.782 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.782 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.782 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:24.782 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "selftest", 2024-05-03T18:47:24.782 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:24.782 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:24.782 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:24.783 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:24.783 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:24.783 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.783 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.783 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.783 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.783 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.783 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.783 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.783 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.783 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.783 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.783 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.783 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.783 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.783 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.783 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.784 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.784 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.784 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.784 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.784 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:24.784 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:24.784 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.784 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.784 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.784 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.784 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.784 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.784 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.784 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.784 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.784 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.785 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.785 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.785 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:24.785 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:24.785 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.785 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.785 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.785 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:24.785 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.785 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.785 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.785 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.785 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.785 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.785 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.785 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.785 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.786 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.786 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.786 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.786 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.786 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.786 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.786 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:24.786 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:24.786 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.786 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.786 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.786 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.786 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.786 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.786 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.787 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.787 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.787 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.787 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.787 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.787 INFO:teuthology.orchestra.run.smithi033.stdout: "roption1": { 2024-05-03T18:47:24.787 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "roption1", 2024-05-03T18:47:24.787 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.787 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.787 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.787 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.787 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.787 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.787 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.787 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.787 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.788 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.788 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.788 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.788 INFO:teuthology.orchestra.run.smithi033.stdout: "roption2": { 2024-05-03T18:47:24.788 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "roption2", 2024-05-03T18:47:24.788 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.788 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.788 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.788 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "xyz", 2024-05-03T18:47:24.788 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.788 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.788 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.788 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.788 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.788 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.788 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.788 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.789 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption1": { 2024-05-03T18:47:24.789 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption1", 2024-05-03T18:47:24.789 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.789 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.789 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.789 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.789 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.789 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.789 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.789 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.789 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.789 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.789 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.789 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.789 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption2": { 2024-05-03T18:47:24.789 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption2", 2024-05-03T18:47:24.789 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.790 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.790 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.790 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.790 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.790 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.790 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.790 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.790 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.790 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.790 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.790 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.790 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption3": { 2024-05-03T18:47:24.790 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption3", 2024-05-03T18:47:24.790 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-05-03T18:47:24.790 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.791 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.791 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.791 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.791 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.791 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.791 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.791 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.791 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.791 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.791 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.791 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption4": { 2024-05-03T18:47:24.791 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption4", 2024-05-03T18:47:24.791 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.791 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.791 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.791 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.792 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.792 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.792 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.792 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.792 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.792 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.792 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.792 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.792 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption5": { 2024-05-03T18:47:24.792 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption5", 2024-05-03T18:47:24.792 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.792 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.792 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.792 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.792 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.792 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.793 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.793 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.793 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.793 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.793 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.793 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.793 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption6": { 2024-05-03T18:47:24.793 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption6", 2024-05-03T18:47:24.793 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.793 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.793 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.793 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:24.793 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.793 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.793 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.793 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.794 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.794 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.794 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.794 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.794 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption7": { 2024-05-03T18:47:24.794 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption7", 2024-05-03T18:47:24.794 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.794 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.794 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.794 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.794 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1", 2024-05-03T18:47:24.794 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "42", 2024-05-03T18:47:24.794 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.794 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.794 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.794 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.795 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.795 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.795 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:24.795 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:24.795 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.795 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:24.795 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.795 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:24.795 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.795 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.795 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.795 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.795 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.795 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.795 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.795 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.795 INFO:teuthology.orchestra.run.smithi033.stdout: "testkey": { 2024-05-03T18:47:24.796 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "testkey", 2024-05-03T18:47:24.796 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.796 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.796 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.796 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.796 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.796 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.796 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.796 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.796 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.796 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.796 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.796 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.796 INFO:teuthology.orchestra.run.smithi033.stdout: "testlkey": { 2024-05-03T18:47:24.796 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "testlkey", 2024-05-03T18:47:24.796 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.796 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.797 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.797 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.797 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.797 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.797 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.797 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.797 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.797 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.797 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.797 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.797 INFO:teuthology.orchestra.run.smithi033.stdout: "testnewline": { 2024-05-03T18:47:24.797 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "testnewline", 2024-05-03T18:47:24.797 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.797 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.797 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.797 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.797 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.798 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.798 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.798 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.798 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.798 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.798 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.798 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.798 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.798 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.798 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:24.798 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smb", 2024-05-03T18:47:24.798 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:24.798 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:24.798 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:24.799 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:24.799 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:24.799 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.799 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.799 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.799 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.799 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.799 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.799 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.799 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.799 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.799 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.799 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.799 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.799 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.800 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.800 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.800 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.800 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.800 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.800 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.800 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:24.800 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:24.800 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.800 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.800 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.800 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.800 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.800 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.800 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.801 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.801 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.801 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.801 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.801 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.801 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:24.801 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:24.801 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.801 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.801 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.801 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:24.801 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.801 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.801 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.801 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.801 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.802 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.802 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.802 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.802 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.802 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.802 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.802 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.802 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.802 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.802 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.802 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:24.802 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:24.802 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.802 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.803 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.803 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.803 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.803 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.803 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.803 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.803 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.803 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.803 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.803 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.803 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:24.803 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:24.803 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.803 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:24.804 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.804 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:24.804 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.804 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.804 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.804 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.804 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.804 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.804 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.804 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.804 INFO:teuthology.orchestra.run.smithi033.stdout: "update_orchestration": { 2024-05-03T18:47:24.804 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "update_orchestration", 2024-05-03T18:47:24.804 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.804 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.805 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.805 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:24.805 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.805 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.805 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.805 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-03T18:47:24.805 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.805 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.805 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.805 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.805 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.805 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.805 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:24.805 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "snap_schedule", 2024-05-03T18:47:24.805 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:24.805 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:24.805 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:24.806 INFO:teuthology.orchestra.run.smithi033.stdout: "allow_m_granularity": { 2024-05-03T18:47:24.806 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "allow_m_granularity", 2024-05-03T18:47:24.806 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.806 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.806 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.806 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.806 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.806 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.806 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.806 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "allow minute scheduled snapshots", 2024-05-03T18:47:24.806 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.806 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.806 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.806 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.806 INFO:teuthology.orchestra.run.smithi033.stdout: "dump_on_update": { 2024-05-03T18:47:24.806 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "dump_on_update", 2024-05-03T18:47:24.807 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.807 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.807 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.807 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.807 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.807 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.807 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.807 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "dump database to debug log on update", 2024-05-03T18:47:24.807 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.807 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.807 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.807 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.807 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:24.807 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:24.807 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.807 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.807 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.808 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.808 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.808 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.808 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.808 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.808 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.808 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.808 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.808 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.808 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.808 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.808 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.808 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.808 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.808 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.808 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.808 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:24.809 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:24.809 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.809 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.809 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.809 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.809 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.809 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.809 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.809 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.809 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.809 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.809 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.809 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.809 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:24.809 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:24.809 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.809 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.810 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.810 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:24.810 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.810 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.810 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.810 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.810 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.810 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.810 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.810 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.810 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.810 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.810 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.810 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.810 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.810 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.811 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.811 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:24.811 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:24.811 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.811 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.811 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.811 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.811 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.811 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.811 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.811 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.811 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.811 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.811 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.811 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.811 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:24.812 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:24.812 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.812 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:24.812 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.812 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:24.812 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.812 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.812 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.812 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.812 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.812 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.812 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.812 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.812 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.812 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.812 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:24.813 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "stats", 2024-05-03T18:47:24.813 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:24.813 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:24.813 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:24.813 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:24.813 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:24.813 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.813 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.813 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.813 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.813 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.813 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.813 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.813 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.813 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.813 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.814 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.814 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.814 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.814 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.814 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.814 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.814 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.814 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.814 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.814 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:24.814 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:24.814 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.814 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.814 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.814 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.815 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.815 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.815 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.815 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.815 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.815 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.815 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.815 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.815 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:24.815 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:24.815 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.815 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.815 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.815 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:24.815 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.815 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.816 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.816 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.816 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.816 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.816 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.816 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.816 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.816 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.816 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.816 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.816 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.816 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.816 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.816 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:24.816 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:24.817 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.817 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.817 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.817 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.817 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.817 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.817 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.817 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.817 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.817 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.817 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.817 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.817 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:24.817 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:24.817 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.817 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:24.818 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.818 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:24.818 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.818 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.818 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.818 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.818 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.818 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.818 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.818 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.818 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.818 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.818 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:24.818 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "status", 2024-05-03T18:47:24.818 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:24.818 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:24.819 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:24.819 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:24.819 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:24.819 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.819 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.819 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.819 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.819 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.819 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.819 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.819 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.819 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.819 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.819 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.819 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.819 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.820 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.820 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.820 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.820 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.820 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.820 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.820 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:24.820 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:24.820 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.820 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.820 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.820 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.820 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.820 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.820 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.820 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.820 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.821 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.821 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.821 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.821 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:24.821 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:24.821 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.821 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.821 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.821 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:24.821 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.821 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.821 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.821 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.821 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.821 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.821 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.821 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.822 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.822 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.822 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.822 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.822 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.822 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.822 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.822 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:24.822 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:24.822 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.822 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.822 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.822 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.822 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.822 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.822 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.823 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.823 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.823 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.823 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.823 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.823 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:24.823 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:24.823 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.823 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:24.823 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.823 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:24.823 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.823 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.823 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.823 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.823 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.824 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.824 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.824 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.824 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.824 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.824 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:24.824 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "telegraf", 2024-05-03T18:47:24.824 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:24.824 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:24.824 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:24.824 INFO:teuthology.orchestra.run.smithi033.stdout: "address": { 2024-05-03T18:47:24.824 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "address", 2024-05-03T18:47:24.824 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.824 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.824 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.825 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-03T18:47:24.825 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.825 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.825 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.825 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.825 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.825 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.825 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.825 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.825 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-05-03T18:47:24.825 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-05-03T18:47:24.825 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-03T18:47:24.825 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.825 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.825 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "15", 2024-05-03T18:47:24.825 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.825 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.826 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.826 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.826 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.826 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.826 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.826 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.826 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:24.826 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:24.826 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.826 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.826 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.826 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.826 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.826 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.826 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.826 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.827 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.827 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.827 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.827 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.827 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.827 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.827 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.827 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.827 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.827 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.827 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.827 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:24.827 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:24.827 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.827 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.828 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.828 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.828 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.828 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.828 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.828 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.828 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.828 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.828 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.828 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.828 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:24.828 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:24.828 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.828 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.828 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.828 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:24.829 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.829 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.829 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.829 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.829 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.829 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.829 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.829 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.829 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.829 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.829 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.829 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.829 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.829 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.829 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.829 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:24.830 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:24.830 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.830 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.830 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.830 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.830 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.830 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.830 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.830 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.830 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.830 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.830 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.830 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.830 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:24.830 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:24.831 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.831 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:24.831 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.831 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:24.831 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.831 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.831 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.831 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.831 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.831 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.831 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.831 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.831 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.831 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.831 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:24.831 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "telemetry", 2024-05-03T18:47:24.832 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:24.832 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:24.832 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:24.832 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_basic": { 2024-05-03T18:47:24.832 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_basic", 2024-05-03T18:47:24.832 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.832 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.832 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.832 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:24.832 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.832 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.832 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.832 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-03T18:47:24.832 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.832 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.832 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.832 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.833 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_crash": { 2024-05-03T18:47:24.833 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_crash", 2024-05-03T18:47:24.833 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.833 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.833 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.833 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:24.833 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.833 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.833 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.833 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-03T18:47:24.833 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.833 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.833 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.833 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.833 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_device": { 2024-05-03T18:47:24.833 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_device", 2024-05-03T18:47:24.834 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.834 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.834 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.834 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:24.834 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.834 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.834 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.834 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-03T18:47:24.834 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.834 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.834 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.834 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.834 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_ident": { 2024-05-03T18:47:24.834 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_ident", 2024-05-03T18:47:24.834 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.834 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.835 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.835 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.835 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.835 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.835 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.835 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-03T18:47:24.835 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.835 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.835 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.835 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.835 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_perf": { 2024-05-03T18:47:24.835 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_perf", 2024-05-03T18:47:24.835 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.835 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.835 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.835 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.836 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.836 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.836 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.836 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-03T18:47:24.836 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.836 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.836 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.836 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.836 INFO:teuthology.orchestra.run.smithi033.stdout: "contact": { 2024-05-03T18:47:24.836 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "contact", 2024-05-03T18:47:24.836 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.836 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.836 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.836 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.836 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.836 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.836 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.837 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.837 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.837 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.837 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.837 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.837 INFO:teuthology.orchestra.run.smithi033.stdout: "description": { 2024-05-03T18:47:24.837 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "description", 2024-05-03T18:47:24.837 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.837 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.837 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.837 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.837 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.837 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.837 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.837 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.837 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.837 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.838 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.838 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.838 INFO:teuthology.orchestra.run.smithi033.stdout: "device_url": { 2024-05-03T18:47:24.838 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "device_url", 2024-05-03T18:47:24.838 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.838 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.838 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.838 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-03T18:47:24.838 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.838 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.838 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.838 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.838 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.838 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.838 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.839 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.839 INFO:teuthology.orchestra.run.smithi033.stdout: "enabled": { 2024-05-03T18:47:24.839 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "enabled", 2024-05-03T18:47:24.839 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.839 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.839 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.839 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.839 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.839 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.839 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.839 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.839 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.839 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.839 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.839 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.839 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-05-03T18:47:24.840 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-05-03T18:47:24.840 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.840 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.840 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.840 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "24", 2024-05-03T18:47:24.840 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "8", 2024-05-03T18:47:24.840 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.840 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.840 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.840 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.840 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.840 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.840 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.840 INFO:teuthology.orchestra.run.smithi033.stdout: "last_opt_revision": { 2024-05-03T18:47:24.840 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "last_opt_revision", 2024-05-03T18:47:24.841 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.841 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.841 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.841 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "1", 2024-05-03T18:47:24.841 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.841 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.841 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.841 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.841 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.841 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.841 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.841 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.841 INFO:teuthology.orchestra.run.smithi033.stdout: "leaderboard": { 2024-05-03T18:47:24.841 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "leaderboard", 2024-05-03T18:47:24.841 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.841 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.842 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.842 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.842 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.842 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.842 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.842 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.842 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.842 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.842 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.842 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.842 INFO:teuthology.orchestra.run.smithi033.stdout: "leaderboard_description": { 2024-05-03T18:47:24.842 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "leaderboard_description", 2024-05-03T18:47:24.842 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.842 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.842 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.843 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.843 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.843 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.843 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.843 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.843 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.843 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.843 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.843 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.843 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:24.843 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:24.843 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.843 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.843 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.843 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.843 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.844 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.844 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.844 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.844 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.844 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.844 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.844 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.844 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.844 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.844 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.844 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.844 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.844 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.844 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.844 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:24.845 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:24.845 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.845 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.845 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.845 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.845 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.845 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.845 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.845 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.845 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.845 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.845 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.845 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.845 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:24.845 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:24.845 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.846 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.846 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.846 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:24.846 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.846 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.846 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.846 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.846 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.846 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.846 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.846 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.846 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.846 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.846 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.846 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.846 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.847 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.847 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.847 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:24.847 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:24.847 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.847 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.847 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.847 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.847 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.847 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.847 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.847 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.847 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.847 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.847 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.847 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.848 INFO:teuthology.orchestra.run.smithi033.stdout: "organization": { 2024-05-03T18:47:24.848 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "organization", 2024-05-03T18:47:24.848 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.848 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.848 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.848 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.848 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.848 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.848 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.848 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.848 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.848 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.848 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.848 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.848 INFO:teuthology.orchestra.run.smithi033.stdout: "proxy": { 2024-05-03T18:47:24.848 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "proxy", 2024-05-03T18:47:24.849 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.849 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.849 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.849 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.849 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.849 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.849 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.849 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.849 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.849 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.849 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.849 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.849 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:24.849 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:24.849 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.849 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:24.849 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.850 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:24.850 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.850 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.850 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.850 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.850 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.850 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.850 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.850 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.850 INFO:teuthology.orchestra.run.smithi033.stdout: "url": { 2024-05-03T18:47:24.850 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "url", 2024-05-03T18:47:24.850 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.850 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.850 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.850 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-03T18:47:24.851 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.851 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.851 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.851 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.851 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.851 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.851 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.851 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.851 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.851 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.851 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:24.851 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "test_orchestrator", 2024-05-03T18:47:24.851 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:24.851 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:24.851 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:24.851 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:24.851 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:24.852 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.852 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.852 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.852 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.852 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.852 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.852 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.852 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.852 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.852 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.852 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.852 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.852 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.852 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.852 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.852 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.852 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.853 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.853 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.853 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:24.853 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:24.853 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.853 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.853 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.853 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.853 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.853 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.853 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.853 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.853 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.853 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.853 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.853 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.853 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:24.854 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:24.854 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.854 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.854 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.854 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:24.854 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.854 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.854 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.854 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.854 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.854 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.854 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.854 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.854 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.854 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.854 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.855 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.855 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.855 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.855 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.855 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:24.855 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:24.855 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.855 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.855 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.855 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.855 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.855 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.855 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.855 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.855 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.855 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.855 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.856 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.856 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:24.856 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:24.856 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.856 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:24.856 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.856 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:24.856 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.856 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.856 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.856 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.856 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.856 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.856 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.856 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.856 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.856 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.857 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:24.857 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "volumes", 2024-05-03T18:47:24.857 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:24.857 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:24.857 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:24.857 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:24.857 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:24.857 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.857 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.857 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.857 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.857 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.857 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.857 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.857 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.857 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.857 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.858 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.858 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.858 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.858 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.858 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.858 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.858 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.858 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.858 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.858 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:24.858 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:24.858 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.858 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.858 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.858 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.858 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.859 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.859 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.859 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.859 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.859 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.859 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.859 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.859 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:24.859 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:24.859 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.859 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.859 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.859 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:24.859 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.859 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.859 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.860 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.860 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.860 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.860 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.860 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.860 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.860 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.860 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.860 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.860 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.860 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.860 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.860 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:24.860 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:24.860 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.860 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.861 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.861 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.861 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.861 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.861 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.861 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.861 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.861 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.861 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.861 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.861 INFO:teuthology.orchestra.run.smithi033.stdout: "max_concurrent_clones": { 2024-05-03T18:47:24.861 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_concurrent_clones", 2024-05-03T18:47:24.861 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.861 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.861 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.862 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "4", 2024-05-03T18:47:24.862 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.862 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.862 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.862 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-03T18:47:24.862 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.862 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.862 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.862 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.862 INFO:teuthology.orchestra.run.smithi033.stdout: "periodic_async_work": { 2024-05-03T18:47:24.862 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "periodic_async_work", 2024-05-03T18:47:24.862 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.862 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.863 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.863 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.863 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.863 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.863 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.863 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Periodically check for async work", 2024-05-03T18:47:24.863 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.863 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.863 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.863 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.863 INFO:teuthology.orchestra.run.smithi033.stdout: "snapshot_clone_delay": { 2024-05-03T18:47:24.863 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "snapshot_clone_delay", 2024-05-03T18:47:24.863 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.863 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.863 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.864 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:24.864 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.864 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.864 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.864 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-03T18:47:24.864 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.864 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.864 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.864 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.864 INFO:teuthology.orchestra.run.smithi033.stdout: "snapshot_clone_no_wait": { 2024-05-03T18:47:24.864 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "snapshot_clone_no_wait", 2024-05-03T18:47:24.864 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.864 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.864 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.864 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-03T18:47:24.865 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.865 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.865 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.865 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-03T18:47:24.865 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.865 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.865 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.865 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.865 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:24.865 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:24.865 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.865 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:24.865 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.865 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:24.866 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.866 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.866 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.866 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.866 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.866 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.866 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.866 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.866 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.866 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.866 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:24.866 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "zabbix", 2024-05-03T18:47:24.866 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-03T18:47:24.866 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-03T18:47:24.866 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-03T18:47:24.867 INFO:teuthology.orchestra.run.smithi033.stdout: "discovery_interval": { 2024-05-03T18:47:24.867 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "discovery_interval", 2024-05-03T18:47:24.867 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-05-03T18:47:24.867 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.867 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.867 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "100", 2024-05-03T18:47:24.867 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.867 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.867 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.867 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.867 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.867 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.867 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.867 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.867 INFO:teuthology.orchestra.run.smithi033.stdout: "identifier": { 2024-05-03T18:47:24.867 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "identifier", 2024-05-03T18:47:24.868 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.868 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.868 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.868 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.868 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.868 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.868 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.868 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.868 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.868 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.868 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.868 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.868 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-05-03T18:47:24.868 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-05-03T18:47:24.868 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-03T18:47:24.869 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.869 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.869 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-05-03T18:47:24.869 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.869 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.869 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.869 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.869 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.869 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.869 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.869 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.869 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-03T18:47:24.869 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-03T18:47:24.869 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.869 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.870 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.870 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.870 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.870 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.870 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.870 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.870 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.870 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.870 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.870 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.870 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.870 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.870 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.870 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.871 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.871 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.871 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.871 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-03T18:47:24.871 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-03T18:47:24.871 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.871 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.871 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.871 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.871 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.871 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.871 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.871 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.871 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.871 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.872 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.872 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.872 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-03T18:47:24.872 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-03T18:47:24.872 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.872 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.872 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.872 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-03T18:47:24.872 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.872 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.872 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-03T18:47:24.872 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-03T18:47:24.872 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-03T18:47:24.872 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-03T18:47:24.872 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-03T18:47:24.873 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-03T18:47:24.873 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-03T18:47:24.873 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.873 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.873 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.873 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.873 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.873 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.873 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-03T18:47:24.873 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-03T18:47:24.873 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-03T18:47:24.873 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.873 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.873 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-03T18:47:24.873 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.873 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.874 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.874 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.874 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.874 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.874 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.874 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.874 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-03T18:47:24.874 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-03T18:47:24.874 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.874 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-03T18:47:24.874 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-03T18:47:24.874 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-03T18:47:24.874 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.874 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.874 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.875 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.875 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.875 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.875 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.875 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.875 INFO:teuthology.orchestra.run.smithi033.stdout: "zabbix_host": { 2024-05-03T18:47:24.875 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "zabbix_host", 2024-05-03T18:47:24.875 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.875 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.875 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.875 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-03T18:47:24.875 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.875 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.875 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.875 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.875 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.876 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.876 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.876 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.876 INFO:teuthology.orchestra.run.smithi033.stdout: "zabbix_port": { 2024-05-03T18:47:24.876 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "zabbix_port", 2024-05-03T18:47:24.876 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-03T18:47:24.876 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.876 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.876 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10051", 2024-05-03T18:47:24.876 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.876 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.876 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.876 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.876 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.876 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.876 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.876 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.877 INFO:teuthology.orchestra.run.smithi033.stdout: "zabbix_sender": { 2024-05-03T18:47:24.877 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "zabbix_sender", 2024-05-03T18:47:24.877 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-03T18:47:24.877 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-03T18:47:24.877 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-03T18:47:24.877 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-03T18:47:24.877 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-03T18:47:24.877 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-03T18:47:24.877 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-03T18:47:24.877 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-03T18:47:24.877 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-03T18:47:24.877 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-03T18:47:24.877 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-03T18:47:24.877 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.877 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.877 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.877 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.878 INFO:teuthology.orchestra.run.smithi033.stdout: "services": { 2024-05-03T18:47:24.878 INFO:teuthology.orchestra.run.smithi033.stdout: "dashboard": "https://172.21.15.33:8443/" 2024-05-03T18:47:24.878 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.878 INFO:teuthology.orchestra.run.smithi033.stdout: "always_on_modules": { 2024-05-03T18:47:24.878 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus": [ 2024-05-03T18:47:24.878 INFO:teuthology.orchestra.run.smithi033.stdout: "balancer", 2024-05-03T18:47:24.878 INFO:teuthology.orchestra.run.smithi033.stdout: "crash", 2024-05-03T18:47:24.878 INFO:teuthology.orchestra.run.smithi033.stdout: "devicehealth", 2024-05-03T18:47:24.878 INFO:teuthology.orchestra.run.smithi033.stdout: "orchestrator", 2024-05-03T18:47:24.878 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_autoscaler", 2024-05-03T18:47:24.878 INFO:teuthology.orchestra.run.smithi033.stdout: "progress", 2024-05-03T18:47:24.878 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_support", 2024-05-03T18:47:24.878 INFO:teuthology.orchestra.run.smithi033.stdout: "status", 2024-05-03T18:47:24.878 INFO:teuthology.orchestra.run.smithi033.stdout: "telemetry", 2024-05-03T18:47:24.878 INFO:teuthology.orchestra.run.smithi033.stdout: "volumes" 2024-05-03T18:47:24.878 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.879 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific": [ 2024-05-03T18:47:24.879 INFO:teuthology.orchestra.run.smithi033.stdout: "balancer", 2024-05-03T18:47:24.879 INFO:teuthology.orchestra.run.smithi033.stdout: "crash", 2024-05-03T18:47:24.879 INFO:teuthology.orchestra.run.smithi033.stdout: "devicehealth", 2024-05-03T18:47:24.879 INFO:teuthology.orchestra.run.smithi033.stdout: "orchestrator", 2024-05-03T18:47:24.879 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_autoscaler", 2024-05-03T18:47:24.879 INFO:teuthology.orchestra.run.smithi033.stdout: "progress", 2024-05-03T18:47:24.879 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_support", 2024-05-03T18:47:24.879 INFO:teuthology.orchestra.run.smithi033.stdout: "status", 2024-05-03T18:47:24.879 INFO:teuthology.orchestra.run.smithi033.stdout: "telemetry", 2024-05-03T18:47:24.879 INFO:teuthology.orchestra.run.smithi033.stdout: "volumes" 2024-05-03T18:47:24.879 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.879 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy": [ 2024-05-03T18:47:24.879 INFO:teuthology.orchestra.run.smithi033.stdout: "balancer", 2024-05-03T18:47:24.879 INFO:teuthology.orchestra.run.smithi033.stdout: "crash", 2024-05-03T18:47:24.879 INFO:teuthology.orchestra.run.smithi033.stdout: "devicehealth", 2024-05-03T18:47:24.880 INFO:teuthology.orchestra.run.smithi033.stdout: "orchestrator", 2024-05-03T18:47:24.880 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_autoscaler", 2024-05-03T18:47:24.880 INFO:teuthology.orchestra.run.smithi033.stdout: "progress", 2024-05-03T18:47:24.880 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_support", 2024-05-03T18:47:24.880 INFO:teuthology.orchestra.run.smithi033.stdout: "status", 2024-05-03T18:47:24.880 INFO:teuthology.orchestra.run.smithi033.stdout: "telemetry", 2024-05-03T18:47:24.880 INFO:teuthology.orchestra.run.smithi033.stdout: "volumes" 2024-05-03T18:47:24.880 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.880 INFO:teuthology.orchestra.run.smithi033.stdout: "reef": [ 2024-05-03T18:47:24.880 INFO:teuthology.orchestra.run.smithi033.stdout: "balancer", 2024-05-03T18:47:24.880 INFO:teuthology.orchestra.run.smithi033.stdout: "crash", 2024-05-03T18:47:24.880 INFO:teuthology.orchestra.run.smithi033.stdout: "devicehealth", 2024-05-03T18:47:24.880 INFO:teuthology.orchestra.run.smithi033.stdout: "orchestrator", 2024-05-03T18:47:24.880 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_autoscaler", 2024-05-03T18:47:24.880 INFO:teuthology.orchestra.run.smithi033.stdout: "progress", 2024-05-03T18:47:24.881 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_support", 2024-05-03T18:47:24.881 INFO:teuthology.orchestra.run.smithi033.stdout: "status", 2024-05-03T18:47:24.881 INFO:teuthology.orchestra.run.smithi033.stdout: "telemetry", 2024-05-03T18:47:24.881 INFO:teuthology.orchestra.run.smithi033.stdout: "volumes" 2024-05-03T18:47:24.881 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-03T18:47:24.881 INFO:teuthology.orchestra.run.smithi033.stdout: "squid": [ 2024-05-03T18:47:24.881 INFO:teuthology.orchestra.run.smithi033.stdout: "balancer", 2024-05-03T18:47:24.881 INFO:teuthology.orchestra.run.smithi033.stdout: "crash", 2024-05-03T18:47:24.881 INFO:teuthology.orchestra.run.smithi033.stdout: "devicehealth", 2024-05-03T18:47:24.881 INFO:teuthology.orchestra.run.smithi033.stdout: "orchestrator", 2024-05-03T18:47:24.881 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_autoscaler", 2024-05-03T18:47:24.881 INFO:teuthology.orchestra.run.smithi033.stdout: "progress", 2024-05-03T18:47:24.881 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_support", 2024-05-03T18:47:24.881 INFO:teuthology.orchestra.run.smithi033.stdout: "status", 2024-05-03T18:47:24.881 INFO:teuthology.orchestra.run.smithi033.stdout: "telemetry", 2024-05-03T18:47:24.881 INFO:teuthology.orchestra.run.smithi033.stdout: "volumes" 2024-05-03T18:47:24.881 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-03T18:47:24.882 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.882 INFO:teuthology.orchestra.run.smithi033.stdout: "last_failure_osd_epoch": 90, 2024-05-03T18:47:24.882 INFO:teuthology.orchestra.run.smithi033.stdout: "active_clients": [ 2024-05-03T18:47:24.882 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:24.882 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "devicehealth", 2024-05-03T18:47:24.882 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-03T18:47:24.882 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:24.882 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-03T18:47:24.882 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:0", 2024-05-03T18:47:24.882 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 1697418511 2024-05-03T18:47:24.882 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.882 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-03T18:47:24.882 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.882 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:24.882 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "libcephsqlite", 2024-05-03T18:47:24.883 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-03T18:47:24.883 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:24.883 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-03T18:47:24.883 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:0", 2024-05-03T18:47:24.883 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 3679424304 2024-05-03T18:47:24.883 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.883 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-03T18:47:24.883 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.883 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:24.883 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rbd_support", 2024-05-03T18:47:24.883 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-03T18:47:24.883 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:24.883 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-03T18:47:24.883 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:0", 2024-05-03T18:47:24.883 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 2368164624 2024-05-03T18:47:24.883 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.884 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-03T18:47:24.884 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.884 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:24.884 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "snap_schedule", 2024-05-03T18:47:24.884 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-03T18:47:24.884 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:24.884 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-03T18:47:24.884 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:0", 2024-05-03T18:47:24.884 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 3702454328 2024-05-03T18:47:24.884 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.884 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-03T18:47:24.884 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.884 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:24.884 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "snap_schedule", 2024-05-03T18:47:24.884 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-03T18:47:24.884 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:24.885 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-03T18:47:24.885 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:0", 2024-05-03T18:47:24.885 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 1851013003 2024-05-03T18:47:24.885 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.885 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-03T18:47:24.885 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:24.885 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:24.885 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "volumes", 2024-05-03T18:47:24.885 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-03T18:47:24.885 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-03T18:47:24.885 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-03T18:47:24.885 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:0", 2024-05-03T18:47:24.885 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 899621660 2024-05-03T18:47:24.885 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.885 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-03T18:47:24.885 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:24.885 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-03T18:47:24.886 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-03T18:47:24.886 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:47:24 smithi150 ceph-mon[32492]: pgmap v638: 129 pgs: 129 active+clean; 3.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 307 B/s wr, 2 op/s 2024-05-03T18:47:24.886 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:47:24 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/975156145' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-03T18:47:24.887 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:47:24 smithi099 ceph-mon[31658]: pgmap v638: 129 pgs: 129 active+clean; 3.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 307 B/s wr, 2 op/s 2024-05-03T18:47:24.887 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:47:24 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/975156145' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-03T18:47:24.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:23.436+0000 7f04f1e00640 1 -- 172.21.15.33:0/975156145 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f04d00b29e0 msgr2=0x7f04d00b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:24.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:23.436+0000 7f04f1e00640 1 --2- 172.21.15.33:0/975156145 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f04d00b29e0 0x7f04d00b4ea0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f04dc005fd0 tx=0x7f04dc005850 comp rx=0 tx=0).stop 2024-05-03T18:47:24.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:23.436+0000 7f04f1e00640 1 -- 172.21.15.33:0/975156145 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04ec077800 msgr2=0x7f04ec1a5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:24.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:23.436+0000 7f04f1e00640 1 --2- 172.21.15.33:0/975156145 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04ec077800 0x7f04ec1a5ae0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f04d4004770 tx=0x7f04d4003ca0 comp rx=0 tx=0).stop 2024-05-03T18:47:24.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:23.437+0000 7f04f1e00640 1 -- 172.21.15.33:0/975156145 shutdown_connections 2024-05-03T18:47:24.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:23.437+0000 7f04f1e00640 1 --2- 172.21.15.33:0/975156145 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f04d00b29e0 0x7f04d00b4ea0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:24.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:23.437+0000 7f04f1e00640 1 --2- 172.21.15.33:0/975156145 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f04ec113820 0x7f04ec19f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:24.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:23.437+0000 7f04f1e00640 1 --2- 172.21.15.33:0/975156145 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04ec077800 0x7f04ec1a5ae0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:24.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:23.437+0000 7f04f1e00640 1 --2- 172.21.15.33:0/975156145 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f04ec076ee0 0x7f04ec1a55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:24.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:23.437+0000 7f04f1e00640 1 -- 172.21.15.33:0/975156145 >> 172.21.15.33:0/975156145 conn(0x7f04ec1006b0 msgr2=0x7f04ec115020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:47:24.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:23.437+0000 7f04f1e00640 1 -- 172.21.15.33:0/975156145 shutdown_connections 2024-05-03T18:47:24.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:47:23.437+0000 7f04f1e00640 1 -- 172.21.15.33:0/975156145 wait complete. 2024-05-03T18:47:24.927 DEBUG:tasks.check_counter:Getting stats from a 2024-05-03T18:47:24.928 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph daemon mds.a perf dump 2024-05-03T18:47:24.928 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:47:24 smithi033 ceph-mon[28330]: pgmap v638: 129 pgs: 129 active+clean; 3.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 307 B/s wr, 2 op/s 2024-05-03T18:47:24.928 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:47:24 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/975156145' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-03T18:47:25.415 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:47:26.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:47:26 smithi033 ceph-mon[28330]: pgmap v639: 129 pgs: 129 active+clean; 3.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-05-03T18:47:26.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:47:26 smithi099 ceph-mon[31658]: pgmap v639: 129 pgs: 129 active+clean; 3.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-05-03T18:47:26.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:47:26 smithi150 ceph-mon[32492]: pgmap v639: 129 pgs: 129 active+clean; 3.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-05-03T18:47:27.817 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-03T18:47:27.817 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-0": { 2024-05-03T18:47:27.817 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 385, 2024-05-03T18:47:27.817 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 376, 2024-05-03T18:47:27.817 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 1607776, 2024-05-03T18:47:27.817 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 232096, 2024-05-03T18:47:27.817 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 5, 2024-05-03T18:47:27.817 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 0, 2024-05-03T18:47:27.817 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.146663151, 2024-05-03T18:47:27.817 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.063045200, 2024-05-03T18:47:27.817 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.068762632, 2024-05-03T18:47:27.817 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.017280060, 2024-05-03T18:47:27.818 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-05-03T18:47:27.818 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 374, 2024-05-03T18:47:27.818 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.061124860, 2024-05-03T18:47:27.818 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000163435 2024-05-03T18:47:27.818 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:27.818 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-05-03T18:47:27.818 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-03T18:47:27.818 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-03T18:47:27.818 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-03T18:47:27.818 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:27.818 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 1607776, 2024-05-03T18:47:27.818 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 232096 2024-05-03T18:47:27.818 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:27.818 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-1": { 2024-05-03T18:47:27.819 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 39, 2024-05-03T18:47:27.819 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 342, 2024-05-03T18:47:27.819 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 163856, 2024-05-03T18:47:27.819 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 311344, 2024-05-03T18:47:27.819 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 7, 2024-05-03T18:47:27.819 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 3, 2024-05-03T18:47:27.819 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.079898514, 2024-05-03T18:47:27.819 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.056246062, 2024-05-03T18:47:27.819 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.017141379, 2024-05-03T18:47:27.819 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.001296560, 2024-05-03T18:47:27.819 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-05-03T18:47:27.819 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 340, 2024-05-03T18:47:27.819 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.075338902, 2024-05-03T18:47:27.819 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000221585 2024-05-03T18:47:27.819 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:27.820 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-05-03T18:47:27.820 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-03T18:47:27.820 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-03T18:47:27.820 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-03T18:47:27.820 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:27.820 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 163856, 2024-05-03T18:47:27.820 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 311344 2024-05-03T18:47:27.820 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:27.820 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-2": { 2024-05-03T18:47:27.820 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 0, 2024-05-03T18:47:27.820 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 3, 2024-05-03T18:47:27.820 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 0, 2024-05-03T18:47:27.820 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 0, 2024-05-03T18:47:27.820 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 3, 2024-05-03T18:47:27.820 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 1, 2024-05-03T18:47:27.821 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.002069393, 2024-05-03T18:47:27.821 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.000000000, 2024-05-03T18:47:27.821 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.001803781, 2024-05-03T18:47:27.821 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-03T18:47:27.821 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-05-03T18:47:27.821 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-03T18:47:27.821 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-03T18:47:27.821 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-03T18:47:27.821 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:27.821 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-05-03T18:47:27.821 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-03T18:47:27.821 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-03T18:47:27.821 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-03T18:47:27.821 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:27.821 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-03T18:47:27.822 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 0 2024-05-03T18:47:27.822 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:27.822 INFO:teuthology.orchestra.run.smithi033.stdout: "cct": { 2024-05-03T18:47:27.822 INFO:teuthology.orchestra.run.smithi033.stdout: "total_workers": 0, 2024-05-03T18:47:27.822 INFO:teuthology.orchestra.run.smithi033.stdout: "unhealthy_workers": 0 2024-05-03T18:47:27.822 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:27.822 INFO:teuthology.orchestra.run.smithi033.stdout: "mempool": { 2024-05-03T18:47:27.822 INFO:teuthology.orchestra.run.smithi033.stdout: "bloom_filter_bytes": 0, 2024-05-03T18:47:27.822 INFO:teuthology.orchestra.run.smithi033.stdout: "bloom_filter_items": 0, 2024-05-03T18:47:27.822 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_alloc_bytes": 0, 2024-05-03T18:47:27.822 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_alloc_items": 0, 2024-05-03T18:47:27.822 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_data_bytes": 0, 2024-05-03T18:47:27.822 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_data_items": 0, 2024-05-03T18:47:27.822 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-03T18:47:27.822 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_onode_items": 0, 2024-05-03T18:47:27.823 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-03T18:47:27.823 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_meta_items": 0, 2024-05-03T18:47:27.823 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_other_bytes": 0, 2024-05-03T18:47:27.823 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_other_items": 0, 2024-05-03T18:47:27.823 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-03T18:47:27.823 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_buffer_items": 0, 2024-05-03T18:47:27.823 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_extent_bytes": 0, 2024-05-03T18:47:27.823 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_extent_items": 0, 2024-05-03T18:47:27.823 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_blob_bytes": 0, 2024-05-03T18:47:27.823 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_blob_items": 0, 2024-05-03T18:47:27.823 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-03T18:47:27.823 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_shared_blob_items": 0, 2024-05-03T18:47:27.823 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-03T18:47:27.823 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_inline_bl_items": 0, 2024-05-03T18:47:27.823 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_fsck_bytes": 0, 2024-05-03T18:47:27.824 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_fsck_items": 0, 2024-05-03T18:47:27.824 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_txc_bytes": 0, 2024-05-03T18:47:27.824 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_txc_items": 0, 2024-05-03T18:47:27.824 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-03T18:47:27.824 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_deferred_items": 0, 2024-05-03T18:47:27.824 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_bytes": 0, 2024-05-03T18:47:27.824 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_items": 0, 2024-05-03T18:47:27.824 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_bytes": 0, 2024-05-03T18:47:27.824 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_items": 0, 2024-05-03T18:47:27.824 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_reader_bytes": 0, 2024-05-03T18:47:27.824 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_reader_items": 0, 2024-05-03T18:47:27.824 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_writer_bytes": 0, 2024-05-03T18:47:27.824 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_writer_items": 0, 2024-05-03T18:47:27.824 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_anon_bytes": 149324, 2024-05-03T18:47:27.824 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_anon_items": 31, 2024-05-03T18:47:27.824 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_meta_bytes": 0, 2024-05-03T18:47:27.825 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_meta_items": 0, 2024-05-03T18:47:27.825 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_bytes": 0, 2024-05-03T18:47:27.825 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_items": 0, 2024-05-03T18:47:27.825 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_mapbl_bytes": 0, 2024-05-03T18:47:27.825 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_mapbl_items": 0, 2024-05-03T18:47:27.825 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pglog_bytes": 0, 2024-05-03T18:47:27.825 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pglog_items": 0, 2024-05-03T18:47:27.825 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_bytes": 0, 2024-05-03T18:47:27.825 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_items": 0, 2024-05-03T18:47:27.825 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_mapping_bytes": 0, 2024-05-03T18:47:27.825 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_mapping_items": 0, 2024-05-03T18:47:27.825 INFO:teuthology.orchestra.run.smithi033.stdout: "pgmap_bytes": 0, 2024-05-03T18:47:27.825 INFO:teuthology.orchestra.run.smithi033.stdout: "pgmap_items": 0, 2024-05-03T18:47:27.825 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_co_bytes": 928, 2024-05-03T18:47:27.825 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_co_items": 2, 2024-05-03T18:47:27.826 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_1_bytes": 0, 2024-05-03T18:47:27.826 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_1_items": 0, 2024-05-03T18:47:27.826 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_2_bytes": 0, 2024-05-03T18:47:27.826 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_2_items": 0 2024-05-03T18:47:27.826 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:27.826 INFO:teuthology.orchestra.run.smithi033.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-03T18:47:27.826 INFO:teuthology.orchestra.run.smithi033.stdout: "val": 0, 2024-05-03T18:47:27.826 INFO:teuthology.orchestra.run.smithi033.stdout: "max": 104857600, 2024-05-03T18:47:27.826 INFO:teuthology.orchestra.run.smithi033.stdout: "get_started": 0, 2024-05-03T18:47:27.826 INFO:teuthology.orchestra.run.smithi033.stdout: "get": 420, 2024-05-03T18:47:27.826 INFO:teuthology.orchestra.run.smithi033.stdout: "get_sum": 1709342, 2024-05-03T18:47:27.826 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_fail": 0, 2024-05-03T18:47:27.826 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_success": 420, 2024-05-03T18:47:27.826 INFO:teuthology.orchestra.run.smithi033.stdout: "take": 0, 2024-05-03T18:47:27.827 INFO:teuthology.orchestra.run.smithi033.stdout: "take_sum": 0, 2024-05-03T18:47:27.827 INFO:teuthology.orchestra.run.smithi033.stdout: "put": 420, 2024-05-03T18:47:27.827 INFO:teuthology.orchestra.run.smithi033.stdout: "put_sum": 1709342, 2024-05-03T18:47:27.827 INFO:teuthology.orchestra.run.smithi033.stdout: "wait": { 2024-05-03T18:47:27.827 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-03T18:47:27.827 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-03T18:47:27.827 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-03T18:47:27.827 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:27.827 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:27.827 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-03T18:47:28.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:47:28 smithi033 ceph-mon[28330]: pgmap v640: 129 pgs: 129 active+clean; 3.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-05-03T18:47:28.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:47:28 smithi099 ceph-mon[31658]: pgmap v640: 129 pgs: 129 active+clean; 3.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-05-03T18:47:28.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:47:28 smithi150 ceph-mon[32492]: pgmap v640: 129 pgs: 129 active+clean; 3.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-05-03T18:47:28.732 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2024-05-03T18:47:28.732 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2024-05-03T18:47:28.732 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2024-05-03T18:47:28.732 DEBUG:tasks.check_counter:Getting stats from d 2024-05-03T18:47:28.733 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph daemon mds.d perf dump 2024-05-03T18:47:29.216 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:47:30.337 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:47:30 smithi033 ceph-mon[28330]: pgmap v641: 129 pgs: 129 active+clean; 3.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 95 B/s wr, 2 op/s 2024-05-03T18:47:30.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:47:30 smithi150 ceph-mon[32492]: pgmap v641: 129 pgs: 129 active+clean; 3.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 95 B/s wr, 2 op/s 2024-05-03T18:47:30.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:47:30 smithi099 ceph-mon[31658]: pgmap v641: 129 pgs: 129 active+clean; 3.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 95 B/s wr, 2 op/s 2024-05-03T18:47:31.763 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-03T18:47:31.764 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-0": { 2024-05-03T18:47:31.764 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 4, 2024-05-03T18:47:31.764 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 4, 2024-05-03T18:47:31.764 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 2912, 2024-05-03T18:47:31.764 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 448, 2024-05-03T18:47:31.764 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 4, 2024-05-03T18:47:31.764 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 0, 2024-05-03T18:47:31.764 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.004600164, 2024-05-03T18:47:31.764 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.000084789, 2024-05-03T18:47:31.764 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.001325012, 2024-05-03T18:47:31.764 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-03T18:47:31.764 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-05-03T18:47:31.765 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 3, 2024-05-03T18:47:31.765 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.001664543, 2024-05-03T18:47:31.765 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000554847 2024-05-03T18:47:31.765 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:31.765 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-05-03T18:47:31.765 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-03T18:47:31.765 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-03T18:47:31.765 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-03T18:47:31.765 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:31.765 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 2912, 2024-05-03T18:47:31.765 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 448 2024-05-03T18:47:31.765 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:31.765 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-1": { 2024-05-03T18:47:31.765 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 423, 2024-05-03T18:47:31.765 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 413, 2024-05-03T18:47:31.766 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 1618832, 2024-05-03T18:47:31.766 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 255776, 2024-05-03T18:47:31.766 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 2, 2024-05-03T18:47:31.766 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 1, 2024-05-03T18:47:31.766 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.172658309, 2024-05-03T18:47:31.766 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.072995902, 2024-05-03T18:47:31.766 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.091818964, 2024-05-03T18:47:31.766 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.022866537, 2024-05-03T18:47:31.766 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-05-03T18:47:31.766 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 412, 2024-05-03T18:47:31.766 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.069548006, 2024-05-03T18:47:31.766 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000168805 2024-05-03T18:47:31.766 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:31.766 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-05-03T18:47:31.767 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-03T18:47:31.767 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-03T18:47:31.767 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-03T18:47:31.767 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:31.767 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 1618832, 2024-05-03T18:47:31.767 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 255776 2024-05-03T18:47:31.767 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:31.767 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-2": { 2024-05-03T18:47:31.767 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 3, 2024-05-03T18:47:31.767 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 316, 2024-05-03T18:47:31.767 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 480, 2024-05-03T18:47:31.767 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 291488, 2024-05-03T18:47:31.767 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 6, 2024-05-03T18:47:31.767 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 3, 2024-05-03T18:47:31.767 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.063592554, 2024-05-03T18:47:31.768 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.051740617, 2024-05-03T18:47:31.768 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.004659615, 2024-05-03T18:47:31.768 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-03T18:47:31.768 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-05-03T18:47:31.768 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 313, 2024-05-03T18:47:31.768 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.079029692, 2024-05-03T18:47:31.768 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000252491 2024-05-03T18:47:31.768 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:31.768 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-05-03T18:47:31.768 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-03T18:47:31.768 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-03T18:47:31.768 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-03T18:47:31.768 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:31.768 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 480, 2024-05-03T18:47:31.768 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 291488 2024-05-03T18:47:31.769 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:31.769 INFO:teuthology.orchestra.run.smithi033.stdout: "cct": { 2024-05-03T18:47:31.769 INFO:teuthology.orchestra.run.smithi033.stdout: "total_workers": 0, 2024-05-03T18:47:31.769 INFO:teuthology.orchestra.run.smithi033.stdout: "unhealthy_workers": 0 2024-05-03T18:47:31.769 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:31.769 INFO:teuthology.orchestra.run.smithi033.stdout: "mempool": { 2024-05-03T18:47:31.769 INFO:teuthology.orchestra.run.smithi033.stdout: "bloom_filter_bytes": 0, 2024-05-03T18:47:31.769 INFO:teuthology.orchestra.run.smithi033.stdout: "bloom_filter_items": 0, 2024-05-03T18:47:31.769 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_alloc_bytes": 0, 2024-05-03T18:47:31.769 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_alloc_items": 0, 2024-05-03T18:47:31.769 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_data_bytes": 0, 2024-05-03T18:47:31.769 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_data_items": 0, 2024-05-03T18:47:31.769 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-03T18:47:31.769 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_onode_items": 0, 2024-05-03T18:47:31.769 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-03T18:47:31.769 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_meta_items": 0, 2024-05-03T18:47:31.769 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_other_bytes": 0, 2024-05-03T18:47:31.770 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_other_items": 0, 2024-05-03T18:47:31.770 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-03T18:47:31.770 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_buffer_items": 0, 2024-05-03T18:47:31.770 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_extent_bytes": 0, 2024-05-03T18:47:31.770 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_extent_items": 0, 2024-05-03T18:47:31.770 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_blob_bytes": 0, 2024-05-03T18:47:31.770 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_blob_items": 0, 2024-05-03T18:47:31.770 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-03T18:47:31.770 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_shared_blob_items": 0, 2024-05-03T18:47:31.770 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-03T18:47:31.770 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_inline_bl_items": 0, 2024-05-03T18:47:31.770 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_fsck_bytes": 0, 2024-05-03T18:47:31.770 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_fsck_items": 0, 2024-05-03T18:47:31.770 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_txc_bytes": 0, 2024-05-03T18:47:31.770 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_txc_items": 0, 2024-05-03T18:47:31.771 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-03T18:47:31.771 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_deferred_items": 0, 2024-05-03T18:47:31.771 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_bytes": 0, 2024-05-03T18:47:31.771 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_items": 0, 2024-05-03T18:47:31.771 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_bytes": 0, 2024-05-03T18:47:31.771 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_items": 0, 2024-05-03T18:47:31.771 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_reader_bytes": 0, 2024-05-03T18:47:31.771 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_reader_items": 0, 2024-05-03T18:47:31.771 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_writer_bytes": 0, 2024-05-03T18:47:31.771 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_writer_items": 0, 2024-05-03T18:47:31.771 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_anon_bytes": 148714, 2024-05-03T18:47:31.771 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_anon_items": 30, 2024-05-03T18:47:31.771 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_meta_bytes": 0, 2024-05-03T18:47:31.771 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_meta_items": 0, 2024-05-03T18:47:31.771 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_bytes": 0, 2024-05-03T18:47:31.772 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_items": 0, 2024-05-03T18:47:31.772 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_mapbl_bytes": 0, 2024-05-03T18:47:31.772 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_mapbl_items": 0, 2024-05-03T18:47:31.772 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pglog_bytes": 0, 2024-05-03T18:47:31.772 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pglog_items": 0, 2024-05-03T18:47:31.772 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_bytes": 0, 2024-05-03T18:47:31.772 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_items": 0, 2024-05-03T18:47:31.772 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_mapping_bytes": 0, 2024-05-03T18:47:31.772 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_mapping_items": 0, 2024-05-03T18:47:31.772 INFO:teuthology.orchestra.run.smithi033.stdout: "pgmap_bytes": 0, 2024-05-03T18:47:31.772 INFO:teuthology.orchestra.run.smithi033.stdout: "pgmap_items": 0, 2024-05-03T18:47:31.772 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_co_bytes": 928, 2024-05-03T18:47:31.772 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_co_items": 2, 2024-05-03T18:47:31.772 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_1_bytes": 0, 2024-05-03T18:47:31.772 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_1_items": 0, 2024-05-03T18:47:31.772 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_2_bytes": 0, 2024-05-03T18:47:31.773 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_2_items": 0 2024-05-03T18:47:31.773 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:31.773 INFO:teuthology.orchestra.run.smithi033.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-03T18:47:31.773 INFO:teuthology.orchestra.run.smithi033.stdout: "val": 0, 2024-05-03T18:47:31.773 INFO:teuthology.orchestra.run.smithi033.stdout: "max": 104857600, 2024-05-03T18:47:31.773 INFO:teuthology.orchestra.run.smithi033.stdout: "get_started": 0, 2024-05-03T18:47:31.773 INFO:teuthology.orchestra.run.smithi033.stdout: "get": 426, 2024-05-03T18:47:31.773 INFO:teuthology.orchestra.run.smithi033.stdout: "get_sum": 1559071, 2024-05-03T18:47:31.773 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_fail": 0, 2024-05-03T18:47:31.773 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_success": 426, 2024-05-03T18:47:31.773 INFO:teuthology.orchestra.run.smithi033.stdout: "take": 0, 2024-05-03T18:47:31.773 INFO:teuthology.orchestra.run.smithi033.stdout: "take_sum": 0, 2024-05-03T18:47:31.773 INFO:teuthology.orchestra.run.smithi033.stdout: "put": 426, 2024-05-03T18:47:31.773 INFO:teuthology.orchestra.run.smithi033.stdout: "put_sum": 1559071, 2024-05-03T18:47:31.773 INFO:teuthology.orchestra.run.smithi033.stdout: "wait": { 2024-05-03T18:47:31.773 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-03T18:47:31.774 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-03T18:47:31.774 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-03T18:47:31.774 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:31.774 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:31.774 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-03T18:47:32.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:47:32 smithi150 ceph-mon[32492]: pgmap v642: 129 pgs: 129 active+clean; 3.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 2 op/s 2024-05-03T18:47:32.426 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:47:32 smithi033 ceph-mon[28330]: pgmap v642: 129 pgs: 129 active+clean; 3.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 2 op/s 2024-05-03T18:47:32.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:47:32 smithi099 ceph-mon[31658]: pgmap v642: 129 pgs: 129 active+clean; 3.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 2 op/s 2024-05-03T18:47:32.569 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-05-03T18:47:32.569 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-05-03T18:47:32.569 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-05-03T18:47:32.569 DEBUG:tasks.check_counter:Getting stats from g 2024-05-03T18:47:32.569 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph daemon mds.g perf dump 2024-05-03T18:47:33.063 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:47:34.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:47:34 smithi150 ceph-mon[32492]: pgmap v643: 129 pgs: 129 active+clean; 3.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:47:34.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:47:34 smithi033 ceph-mon[28330]: pgmap v643: 129 pgs: 129 active+clean; 3.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:47:34.547 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:47:34 smithi099 ceph-mon[31658]: pgmap v643: 129 pgs: 129 active+clean; 3.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:47:35.394 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-03T18:47:35.394 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-0": { 2024-05-03T18:47:35.394 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 4, 2024-05-03T18:47:35.395 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 4, 2024-05-03T18:47:35.395 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 2912, 2024-05-03T18:47:35.395 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 448, 2024-05-03T18:47:35.395 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 4, 2024-05-03T18:47:35.395 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 0, 2024-05-03T18:47:35.395 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.004739339, 2024-05-03T18:47:35.395 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.000078817, 2024-05-03T18:47:35.395 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.001329989, 2024-05-03T18:47:35.395 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-03T18:47:35.395 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-05-03T18:47:35.395 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 3, 2024-05-03T18:47:35.395 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.001800157, 2024-05-03T18:47:35.395 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000600052 2024-05-03T18:47:35.395 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:35.396 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-05-03T18:47:35.396 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-03T18:47:35.396 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-03T18:47:35.396 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-03T18:47:35.396 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:35.396 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 2912, 2024-05-03T18:47:35.396 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 448 2024-05-03T18:47:35.396 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:35.396 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-1": { 2024-05-03T18:47:35.396 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 419, 2024-05-03T18:47:35.396 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 409, 2024-05-03T18:47:35.396 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 1617808, 2024-05-03T18:47:35.396 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 252736, 2024-05-03T18:47:35.396 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 2, 2024-05-03T18:47:35.396 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 1, 2024-05-03T18:47:35.397 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.168574896, 2024-05-03T18:47:35.397 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.077953009, 2024-05-03T18:47:35.397 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.085729072, 2024-05-03T18:47:35.397 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.019418282, 2024-05-03T18:47:35.397 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-05-03T18:47:35.397 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 408, 2024-05-03T18:47:35.397 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.066391175, 2024-05-03T18:47:35.397 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000162723 2024-05-03T18:47:35.397 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:35.397 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-05-03T18:47:35.397 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-03T18:47:35.397 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-03T18:47:35.397 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-03T18:47:35.397 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:35.397 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 1617808, 2024-05-03T18:47:35.397 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 252736 2024-05-03T18:47:35.398 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:35.398 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-2": { 2024-05-03T18:47:35.398 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 2, 2024-05-03T18:47:35.398 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 311, 2024-05-03T18:47:35.398 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 320, 2024-05-03T18:47:35.398 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 211808, 2024-05-03T18:47:35.398 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 5, 2024-05-03T18:47:35.398 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 3, 2024-05-03T18:47:35.398 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.058311643, 2024-05-03T18:47:35.398 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.047371320, 2024-05-03T18:47:35.398 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.003103987, 2024-05-03T18:47:35.398 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-03T18:47:35.399 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-05-03T18:47:35.399 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 308, 2024-05-03T18:47:35.399 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.071189686, 2024-05-03T18:47:35.399 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000231135 2024-05-03T18:47:35.399 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:35.399 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-05-03T18:47:35.399 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-03T18:47:35.399 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-03T18:47:35.399 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-03T18:47:35.399 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:35.399 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-03T18:47:35.399 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 211808 2024-05-03T18:47:35.399 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:35.400 INFO:teuthology.orchestra.run.smithi033.stdout: "cct": { 2024-05-03T18:47:35.400 INFO:teuthology.orchestra.run.smithi033.stdout: "total_workers": 0, 2024-05-03T18:47:35.400 INFO:teuthology.orchestra.run.smithi033.stdout: "unhealthy_workers": 0 2024-05-03T18:47:35.400 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:35.400 INFO:teuthology.orchestra.run.smithi033.stdout: "mempool": { 2024-05-03T18:47:35.400 INFO:teuthology.orchestra.run.smithi033.stdout: "bloom_filter_bytes": 0, 2024-05-03T18:47:35.400 INFO:teuthology.orchestra.run.smithi033.stdout: "bloom_filter_items": 0, 2024-05-03T18:47:35.400 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_alloc_bytes": 0, 2024-05-03T18:47:35.400 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_alloc_items": 0, 2024-05-03T18:47:35.400 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_data_bytes": 0, 2024-05-03T18:47:35.400 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_data_items": 0, 2024-05-03T18:47:35.400 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-03T18:47:35.400 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_onode_items": 0, 2024-05-03T18:47:35.400 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-03T18:47:35.400 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_meta_items": 0, 2024-05-03T18:47:35.401 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_other_bytes": 0, 2024-05-03T18:47:35.401 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_other_items": 0, 2024-05-03T18:47:35.401 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-03T18:47:35.401 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_buffer_items": 0, 2024-05-03T18:47:35.401 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_extent_bytes": 0, 2024-05-03T18:47:35.401 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_extent_items": 0, 2024-05-03T18:47:35.401 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_blob_bytes": 0, 2024-05-03T18:47:35.401 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_blob_items": 0, 2024-05-03T18:47:35.401 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-03T18:47:35.401 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_shared_blob_items": 0, 2024-05-03T18:47:35.401 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-03T18:47:35.401 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_inline_bl_items": 0, 2024-05-03T18:47:35.401 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_fsck_bytes": 0, 2024-05-03T18:47:35.401 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_fsck_items": 0, 2024-05-03T18:47:35.401 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_txc_bytes": 0, 2024-05-03T18:47:35.401 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_txc_items": 0, 2024-05-03T18:47:35.402 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-03T18:47:35.402 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_deferred_items": 0, 2024-05-03T18:47:35.402 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_bytes": 0, 2024-05-03T18:47:35.402 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_items": 0, 2024-05-03T18:47:35.402 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_bytes": 0, 2024-05-03T18:47:35.402 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_items": 0, 2024-05-03T18:47:35.402 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_reader_bytes": 0, 2024-05-03T18:47:35.402 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_reader_items": 0, 2024-05-03T18:47:35.402 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_writer_bytes": 0, 2024-05-03T18:47:35.402 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_writer_items": 0, 2024-05-03T18:47:35.402 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_anon_bytes": 148714, 2024-05-03T18:47:35.402 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_anon_items": 30, 2024-05-03T18:47:35.402 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_meta_bytes": 0, 2024-05-03T18:47:35.402 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_meta_items": 0, 2024-05-03T18:47:35.403 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_bytes": 0, 2024-05-03T18:47:35.403 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_items": 0, 2024-05-03T18:47:35.403 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_mapbl_bytes": 0, 2024-05-03T18:47:35.403 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_mapbl_items": 0, 2024-05-03T18:47:35.403 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pglog_bytes": 0, 2024-05-03T18:47:35.403 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pglog_items": 0, 2024-05-03T18:47:35.403 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_bytes": 0, 2024-05-03T18:47:35.403 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_items": 0, 2024-05-03T18:47:35.403 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_mapping_bytes": 0, 2024-05-03T18:47:35.403 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_mapping_items": 0, 2024-05-03T18:47:35.403 INFO:teuthology.orchestra.run.smithi033.stdout: "pgmap_bytes": 0, 2024-05-03T18:47:35.403 INFO:teuthology.orchestra.run.smithi033.stdout: "pgmap_items": 0, 2024-05-03T18:47:35.403 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_co_bytes": 928, 2024-05-03T18:47:35.403 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_co_items": 2, 2024-05-03T18:47:35.403 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_1_bytes": 0, 2024-05-03T18:47:35.403 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_1_items": 0, 2024-05-03T18:47:35.403 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_2_bytes": 0, 2024-05-03T18:47:35.404 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_2_items": 0 2024-05-03T18:47:35.404 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:35.404 INFO:teuthology.orchestra.run.smithi033.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-03T18:47:35.404 INFO:teuthology.orchestra.run.smithi033.stdout: "val": 0, 2024-05-03T18:47:35.404 INFO:teuthology.orchestra.run.smithi033.stdout: "max": 104857600, 2024-05-03T18:47:35.404 INFO:teuthology.orchestra.run.smithi033.stdout: "get_started": 0, 2024-05-03T18:47:35.404 INFO:teuthology.orchestra.run.smithi033.stdout: "get": 421, 2024-05-03T18:47:35.404 INFO:teuthology.orchestra.run.smithi033.stdout: "get_sum": 1558588, 2024-05-03T18:47:35.404 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_fail": 0, 2024-05-03T18:47:35.404 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_success": 421, 2024-05-03T18:47:35.404 INFO:teuthology.orchestra.run.smithi033.stdout: "take": 0, 2024-05-03T18:47:35.404 INFO:teuthology.orchestra.run.smithi033.stdout: "take_sum": 0, 2024-05-03T18:47:35.404 INFO:teuthology.orchestra.run.smithi033.stdout: "put": 421, 2024-05-03T18:47:35.404 INFO:teuthology.orchestra.run.smithi033.stdout: "put_sum": 1558588, 2024-05-03T18:47:35.404 INFO:teuthology.orchestra.run.smithi033.stdout: "wait": { 2024-05-03T18:47:35.404 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-03T18:47:35.405 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-03T18:47:35.405 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-03T18:47:35.405 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:35.405 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:35.405 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-03T18:47:36.196 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:47:36 smithi033 ceph-mon[28330]: pgmap v644: 129 pgs: 129 active+clean; 3.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:47:36.266 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2024-05-03T18:47:36.266 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2024-05-03T18:47:36.266 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2024-05-03T18:47:36.266 DEBUG:tasks.check_counter:Getting stats from j 2024-05-03T18:47:36.266 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph daemon mds.j perf dump 2024-05-03T18:47:36.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:47:36 smithi150 ceph-mon[32492]: pgmap v644: 129 pgs: 129 active+clean; 3.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:47:36.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:47:36 smithi099 ceph-mon[31658]: pgmap v644: 129 pgs: 129 active+clean; 3.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:47:36.749 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:47:38.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:47:38 smithi033 ceph-mon[28330]: pgmap v645: 129 pgs: 129 active+clean; 3.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:47:38.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:47:38 smithi150 ceph-mon[32492]: pgmap v645: 129 pgs: 129 active+clean; 3.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:47:38.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:47:38 smithi099 ceph-mon[31658]: pgmap v645: 129 pgs: 129 active+clean; 3.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:47:39.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:47:39 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:47:39.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:47:39 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:47:39.397 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-03T18:47:39.398 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-0": { 2024-05-03T18:47:39.398 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 432, 2024-05-03T18:47:39.398 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 420, 2024-05-03T18:47:39.398 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 1623168, 2024-05-03T18:47:39.398 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 258944, 2024-05-03T18:47:39.398 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 4, 2024-05-03T18:47:39.398 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 0, 2024-05-03T18:47:39.398 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.183066954, 2024-05-03T18:47:39.398 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.083433316, 2024-05-03T18:47:39.398 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.084436356, 2024-05-03T18:47:39.399 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.022888149, 2024-05-03T18:47:39.399 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-05-03T18:47:39.399 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 420, 2024-05-03T18:47:39.399 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.081623152, 2024-05-03T18:47:39.399 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000194340 2024-05-03T18:47:39.399 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:39.399 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-05-03T18:47:39.399 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-03T18:47:39.399 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-03T18:47:39.399 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-03T18:47:39.399 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:39.399 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 1623168, 2024-05-03T18:47:39.399 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 258944 2024-05-03T18:47:39.399 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:39.399 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-1": { 2024-05-03T18:47:39.400 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 3, 2024-05-03T18:47:39.400 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 319, 2024-05-03T18:47:39.400 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 480, 2024-05-03T18:47:39.400 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 292016, 2024-05-03T18:47:39.400 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 6, 2024-05-03T18:47:39.400 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 3, 2024-05-03T18:47:39.400 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.056306121, 2024-05-03T18:47:39.400 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.044400196, 2024-05-03T18:47:39.400 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.004203764, 2024-05-03T18:47:39.400 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-03T18:47:39.400 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-05-03T18:47:39.400 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 316, 2024-05-03T18:47:39.400 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.084282194, 2024-05-03T18:47:39.400 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000266715 2024-05-03T18:47:39.400 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:39.401 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-05-03T18:47:39.401 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-03T18:47:39.401 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-03T18:47:39.401 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-03T18:47:39.401 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:39.401 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 480, 2024-05-03T18:47:39.401 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 292016 2024-05-03T18:47:39.401 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:39.401 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-2": { 2024-05-03T18:47:39.401 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 0, 2024-05-03T18:47:39.401 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 2, 2024-05-03T18:47:39.401 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 0, 2024-05-03T18:47:39.401 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 0, 2024-05-03T18:47:39.401 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 2, 2024-05-03T18:47:39.401 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 1, 2024-05-03T18:47:39.401 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.000903909, 2024-05-03T18:47:39.402 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.000000000, 2024-05-03T18:47:39.402 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.000695970, 2024-05-03T18:47:39.402 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-03T18:47:39.402 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-05-03T18:47:39.402 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-03T18:47:39.402 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-03T18:47:39.402 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-03T18:47:39.402 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:39.402 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-05-03T18:47:39.402 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-03T18:47:39.402 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-03T18:47:39.402 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-03T18:47:39.402 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:39.402 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-03T18:47:39.403 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 0 2024-05-03T18:47:39.403 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:39.403 INFO:teuthology.orchestra.run.smithi033.stdout: "cct": { 2024-05-03T18:47:39.403 INFO:teuthology.orchestra.run.smithi033.stdout: "total_workers": 0, 2024-05-03T18:47:39.403 INFO:teuthology.orchestra.run.smithi033.stdout: "unhealthy_workers": 0 2024-05-03T18:47:39.403 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:39.403 INFO:teuthology.orchestra.run.smithi033.stdout: "mempool": { 2024-05-03T18:47:39.403 INFO:teuthology.orchestra.run.smithi033.stdout: "bloom_filter_bytes": 0, 2024-05-03T18:47:39.403 INFO:teuthology.orchestra.run.smithi033.stdout: "bloom_filter_items": 0, 2024-05-03T18:47:39.403 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_alloc_bytes": 0, 2024-05-03T18:47:39.403 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_alloc_items": 0, 2024-05-03T18:47:39.403 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_data_bytes": 0, 2024-05-03T18:47:39.403 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_data_items": 0, 2024-05-03T18:47:39.403 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-03T18:47:39.403 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_onode_items": 0, 2024-05-03T18:47:39.403 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-03T18:47:39.404 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_meta_items": 0, 2024-05-03T18:47:39.404 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_other_bytes": 0, 2024-05-03T18:47:39.404 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_other_items": 0, 2024-05-03T18:47:39.404 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-03T18:47:39.404 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_buffer_items": 0, 2024-05-03T18:47:39.404 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_extent_bytes": 0, 2024-05-03T18:47:39.404 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_extent_items": 0, 2024-05-03T18:47:39.404 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_blob_bytes": 0, 2024-05-03T18:47:39.404 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_blob_items": 0, 2024-05-03T18:47:39.404 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-03T18:47:39.404 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_shared_blob_items": 0, 2024-05-03T18:47:39.404 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-03T18:47:39.404 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_inline_bl_items": 0, 2024-05-03T18:47:39.404 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_fsck_bytes": 0, 2024-05-03T18:47:39.404 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_fsck_items": 0, 2024-05-03T18:47:39.405 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_txc_bytes": 0, 2024-05-03T18:47:39.405 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_txc_items": 0, 2024-05-03T18:47:39.405 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-03T18:47:39.405 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_deferred_items": 0, 2024-05-03T18:47:39.405 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_bytes": 0, 2024-05-03T18:47:39.405 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_items": 0, 2024-05-03T18:47:39.405 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_bytes": 0, 2024-05-03T18:47:39.405 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_items": 0, 2024-05-03T18:47:39.405 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_reader_bytes": 0, 2024-05-03T18:47:39.405 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_reader_items": 0, 2024-05-03T18:47:39.405 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_writer_bytes": 0, 2024-05-03T18:47:39.405 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_writer_items": 0, 2024-05-03T18:47:39.405 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_anon_bytes": 148731, 2024-05-03T18:47:39.405 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_anon_items": 31, 2024-05-03T18:47:39.405 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_meta_bytes": 0, 2024-05-03T18:47:39.406 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_meta_items": 0, 2024-05-03T18:47:39.406 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_bytes": 0, 2024-05-03T18:47:39.406 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_items": 0, 2024-05-03T18:47:39.406 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_mapbl_bytes": 0, 2024-05-03T18:47:39.406 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_mapbl_items": 0, 2024-05-03T18:47:39.406 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pglog_bytes": 0, 2024-05-03T18:47:39.406 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pglog_items": 0, 2024-05-03T18:47:39.406 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_bytes": 0, 2024-05-03T18:47:39.406 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_items": 0, 2024-05-03T18:47:39.406 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_mapping_bytes": 0, 2024-05-03T18:47:39.406 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_mapping_items": 0, 2024-05-03T18:47:39.406 INFO:teuthology.orchestra.run.smithi033.stdout: "pgmap_bytes": 0, 2024-05-03T18:47:39.406 INFO:teuthology.orchestra.run.smithi033.stdout: "pgmap_items": 0, 2024-05-03T18:47:39.406 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_co_bytes": 928, 2024-05-03T18:47:39.406 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_co_items": 2, 2024-05-03T18:47:39.407 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_1_bytes": 0, 2024-05-03T18:47:39.407 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_1_items": 0, 2024-05-03T18:47:39.407 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_2_bytes": 0, 2024-05-03T18:47:39.407 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_2_items": 0 2024-05-03T18:47:39.407 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-03T18:47:39.407 INFO:teuthology.orchestra.run.smithi033.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-03T18:47:39.407 INFO:teuthology.orchestra.run.smithi033.stdout: "val": 0, 2024-05-03T18:47:39.407 INFO:teuthology.orchestra.run.smithi033.stdout: "max": 104857600, 2024-05-03T18:47:39.407 INFO:teuthology.orchestra.run.smithi033.stdout: "get_started": 0, 2024-05-03T18:47:39.407 INFO:teuthology.orchestra.run.smithi033.stdout: "get": 431, 2024-05-03T18:47:39.407 INFO:teuthology.orchestra.run.smithi033.stdout: "get_sum": 1559785, 2024-05-03T18:47:39.407 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_fail": 0, 2024-05-03T18:47:39.407 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_success": 431, 2024-05-03T18:47:39.408 INFO:teuthology.orchestra.run.smithi033.stdout: "take": 0, 2024-05-03T18:47:39.408 INFO:teuthology.orchestra.run.smithi033.stdout: "take_sum": 0, 2024-05-03T18:47:39.408 INFO:teuthology.orchestra.run.smithi033.stdout: "put": 431, 2024-05-03T18:47:39.408 INFO:teuthology.orchestra.run.smithi033.stdout: "put_sum": 1559785, 2024-05-03T18:47:39.408 INFO:teuthology.orchestra.run.smithi033.stdout: "wait": { 2024-05-03T18:47:39.408 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-03T18:47:39.408 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-03T18:47:39.408 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-03T18:47:39.408 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:39.408 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-03T18:47:39.408 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-03T18:47:39.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:47:39 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:47:40.372 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:47:40 smithi033 ceph-mon[28330]: pgmap v646: 129 pgs: 129 active+clean; 3.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:47:40.372 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:47:40 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:47:40.372 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:47:40 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:47:40.372 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:47:40 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:47:40.376 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-05-03T18:47:40.377 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-05-03T18:47:40.377 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-05-03T18:47:40.377 DEBUG:tasks.check_counter:Getting stats from b 2024-05-03T18:47:40.377 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph daemon mds.b perf dump 2024-05-03T18:47:40.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:47:40 smithi150 ceph-mon[32492]: pgmap v646: 129 pgs: 129 active+clean; 3.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:47:40.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:47:40 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:47:40.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:47:40 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:47:40.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:47:40 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:47:40.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:47:40 smithi099 ceph-mon[31658]: pgmap v646: 129 pgs: 129 active+clean; 3.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:47:40.547 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:47:40 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:47:40.547 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:47:40 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:47:40.547 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:47:40 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:47:41.043 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.b/config 2024-05-03T18:47:42.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:47:42 smithi150 ceph-mon[32492]: pgmap v647: 129 pgs: 129 active+clean; 3.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 2 op/s 2024-05-03T18:47:42.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:47:42 smithi033 ceph-mon[28330]: pgmap v647: 129 pgs: 129 active+clean; 3.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 2 op/s 2024-05-03T18:47:42.547 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:47:42 smithi099 ceph-mon[31658]: pgmap v647: 129 pgs: 129 active+clean; 3.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 2 op/s 2024-05-03T18:47:43.944 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-05-03T18:47:43.944 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-0": { 2024-05-03T18:47:43.945 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 40455, 2024-05-03T18:47:43.945 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 38378, 2024-05-03T18:47:43.945 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 12799630, 2024-05-03T18:47:43.945 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 99675066, 2024-05-03T18:47:43.945 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 26, 2024-05-03T18:47:43.945 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 5, 2024-05-03T18:47:43.945 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 7.295142645, 2024-05-03T18:47:43.945 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 2.954301047, 2024-05-03T18:47:43.945 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 4.209066317, 2024-05-03T18:47:43.945 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.784765572, 2024-05-03T18:47:43.945 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2024-05-03T18:47:43.945 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 38376, 2024-05-03T18:47:43.945 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 4.351156993, 2024-05-03T18:47:43.945 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000113382 2024-05-03T18:47:43.946 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:43.946 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2024-05-03T18:47:43.946 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 54452, 2024-05-03T18:47:43.946 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.025448021, 2024-05-03T18:47:43.946 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000467 2024-05-03T18:47:43.946 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:43.946 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 630944, 2024-05-03T18:47:43.946 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 203856 2024-05-03T18:47:43.946 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:43.946 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-1": { 2024-05-03T18:47:43.946 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 16080, 2024-05-03T18:47:43.946 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 16054, 2024-05-03T18:47:43.946 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 4095242, 2024-05-03T18:47:43.946 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 105422198, 2024-05-03T18:47:43.947 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 232, 2024-05-03T18:47:43.947 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 8, 2024-05-03T18:47:43.947 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 2.284303703, 2024-05-03T18:47:43.947 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.828024216, 2024-05-03T18:47:43.947 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.663424309, 2024-05-03T18:47:43.947 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.585830090, 2024-05-03T18:47:43.947 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2024-05-03T18:47:43.947 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 16052, 2024-05-03T18:47:43.947 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 1.430838707, 2024-05-03T18:47:43.947 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000089137 2024-05-03T18:47:43.947 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:43.947 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2024-05-03T18:47:43.947 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 326, 2024-05-03T18:47:43.947 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000080437, 2024-05-03T18:47:43.947 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000246 2024-05-03T18:47:43.948 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:43.948 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 46208, 2024-05-03T18:47:43.948 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 159280 2024-05-03T18:47:43.948 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:43.948 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-2": { 2024-05-03T18:47:43.948 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 17843, 2024-05-03T18:47:43.948 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 17817, 2024-05-03T18:47:43.948 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 6158692, 2024-05-03T18:47:43.948 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 24431357, 2024-05-03T18:47:43.948 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 11, 2024-05-03T18:47:43.948 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 5, 2024-05-03T18:47:43.948 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 2.378110533, 2024-05-03T18:47:43.948 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.927804159, 2024-05-03T18:47:43.948 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.647257943, 2024-05-03T18:47:43.948 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.657366329, 2024-05-03T18:47:43.949 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2024-05-03T18:47:43.949 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 17816, 2024-05-03T18:47:43.949 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 1.197113760, 2024-05-03T18:47:43.949 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000067193 2024-05-03T18:47:43.949 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:43.949 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2024-05-03T18:47:43.949 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 446, 2024-05-03T18:47:43.949 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000156286, 2024-05-03T18:47:43.949 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000350 2024-05-03T18:47:43.949 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:43.949 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 1840272, 2024-05-03T18:47:43.949 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 851296 2024-05-03T18:47:43.949 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:43.949 INFO:teuthology.orchestra.run.smithi099.stdout: "cct": { 2024-05-03T18:47:43.949 INFO:teuthology.orchestra.run.smithi099.stdout: "total_workers": 1, 2024-05-03T18:47:43.949 INFO:teuthology.orchestra.run.smithi099.stdout: "unhealthy_workers": 0 2024-05-03T18:47:43.950 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:43.950 INFO:teuthology.orchestra.run.smithi099.stdout: "finisher-MDSRank": { 2024-05-03T18:47:43.950 INFO:teuthology.orchestra.run.smithi099.stdout: "queue_len": 0, 2024-05-03T18:47:43.950 INFO:teuthology.orchestra.run.smithi099.stdout: "complete_latency": { 2024-05-03T18:47:43.950 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 37752, 2024-05-03T18:47:43.950 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 22.814532377, 2024-05-03T18:47:43.950 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000604326 2024-05-03T18:47:43.950 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-03T18:47:43.950 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:43.950 INFO:teuthology.orchestra.run.smithi099.stdout: "finisher-PurgeQueue": { 2024-05-03T18:47:43.950 INFO:teuthology.orchestra.run.smithi099.stdout: "queue_len": 0, 2024-05-03T18:47:43.950 INFO:teuthology.orchestra.run.smithi099.stdout: "complete_latency": { 2024-05-03T18:47:43.950 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 1049, 2024-05-03T18:47:43.950 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.134008742, 2024-05-03T18:47:43.951 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000127749 2024-05-03T18:47:43.951 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-03T18:47:43.951 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:43.951 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": { 2024-05-03T18:47:43.951 INFO:teuthology.orchestra.run.smithi099.stdout: "request": 8966, 2024-05-03T18:47:43.951 INFO:teuthology.orchestra.run.smithi099.stdout: "reply": 8966, 2024-05-03T18:47:43.951 INFO:teuthology.orchestra.run.smithi099.stdout: "reply_latency": { 2024-05-03T18:47:43.951 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 8966, 2024-05-03T18:47:43.951 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 23.920120380, 2024-05-03T18:47:43.951 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.002667869 2024-05-03T18:47:43.951 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:43.951 INFO:teuthology.orchestra.run.smithi099.stdout: "slow_reply": 0, 2024-05-03T18:47:43.951 INFO:teuthology.orchestra.run.smithi099.stdout: "forward": 0, 2024-05-03T18:47:43.951 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_fetch_complete": 0, 2024-05-03T18:47:43.951 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_fetch_keys": 0, 2024-05-03T18:47:43.952 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_commit": 113, 2024-05-03T18:47:43.952 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_split": 12, 2024-05-03T18:47:43.952 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_merge": 0, 2024-05-03T18:47:43.952 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes": 3013, 2024-05-03T18:47:43.952 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_top": 398, 2024-05-03T18:47:43.952 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_bottom": 2615, 2024-05-03T18:47:43.952 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_pin_tail": 0, 2024-05-03T18:47:43.952 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_pinned": 2444, 2024-05-03T18:47:43.952 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_expired": 0, 2024-05-03T18:47:43.952 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_with_caps": 1045, 2024-05-03T18:47:43.952 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": 1046, 2024-05-03T18:47:43.952 INFO:teuthology.orchestra.run.smithi099.stdout: "subtrees": 2, 2024-05-03T18:47:43.952 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse": 14081, 2024-05-03T18:47:43.952 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_hit": 12893, 2024-05-03T18:47:43.952 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_forward": 0, 2024-05-03T18:47:43.953 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_discover": 0, 2024-05-03T18:47:43.953 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_dir_fetch": 0, 2024-05-03T18:47:43.953 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_remote_ino": 0, 2024-05-03T18:47:43.953 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_lock": 19, 2024-05-03T18:47:43.953 INFO:teuthology.orchestra.run.smithi099.stdout: "load_cent": 0, 2024-05-03T18:47:43.953 INFO:teuthology.orchestra.run.smithi099.stdout: "q": 0, 2024-05-03T18:47:43.953 INFO:teuthology.orchestra.run.smithi099.stdout: "exported": 0, 2024-05-03T18:47:43.953 INFO:teuthology.orchestra.run.smithi099.stdout: "exported_inodes": 0, 2024-05-03T18:47:43.953 INFO:teuthology.orchestra.run.smithi099.stdout: "imported": 0, 2024-05-03T18:47:43.953 INFO:teuthology.orchestra.run.smithi099.stdout: "imported_inodes": 0, 2024-05-03T18:47:43.953 INFO:teuthology.orchestra.run.smithi099.stdout: "openino_dir_fetch": 0, 2024-05-03T18:47:43.953 INFO:teuthology.orchestra.run.smithi099.stdout: "openino_backtrace_fetch": 36612, 2024-05-03T18:47:43.953 INFO:teuthology.orchestra.run.smithi099.stdout: "openino_peer_discover": 0, 2024-05-03T18:47:43.954 INFO:teuthology.orchestra.run.smithi099.stdout: "root_rfiles": 1015, 2024-05-03T18:47:43.954 INFO:teuthology.orchestra.run.smithi099.stdout: "root_rbytes": 1482306901, 2024-05-03T18:47:43.954 INFO:teuthology.orchestra.run.smithi099.stdout: "root_rsnaps": 6, 2024-05-03T18:47:43.954 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_backtrace_fetch": 36612, 2024-05-03T18:47:43.954 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_set_tag": 0, 2024-05-03T18:47:43.954 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_backtrace_repaired": 0, 2024-05-03T18:47:43.954 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_inotable_repaired": 0, 2024-05-03T18:47:43.954 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_dir_inodes": 1291, 2024-05-03T18:47:43.954 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_dir_base_inodes": 37, 2024-05-03T18:47:43.954 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_dirfrag_rstats": 1254, 2024-05-03T18:47:43.954 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_file_inodes": 35321, 2024-05-03T18:47:43.954 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_inode_file_caps": 0, 2024-05-03T18:47:43.954 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_revoke": 1502, 2024-05-03T18:47:43.955 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_grant": 3222, 2024-05-03T18:47:43.955 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_trunc": 0, 2024-05-03T18:47:43.955 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-03T18:47:43.955 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-03T18:47:43.955 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_caps": 18386, 2024-05-03T18:47:43.955 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_caps_dirty": 12300, 2024-05-03T18:47:43.955 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_cap_release": 79, 2024-05-03T18:47:43.955 INFO:teuthology.orchestra.run.smithi099.stdout: "process_request_cap_release": 2834 2024-05-03T18:47:43.955 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:43.955 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_cache": { 2024-05-03T18:47:43.955 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_update": 0, 2024-05-03T18:47:43.955 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_update_receipt": 0, 2024-05-03T18:47:43.955 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_try_discover": 0, 2024-05-03T18:47:43.956 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_send_discover": 0, 2024-05-03T18:47:43.956 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_handle_discover": 0, 2024-05-03T18:47:43.956 INFO:teuthology.orchestra.run.smithi099.stdout: "num_strays": 1, 2024-05-03T18:47:43.956 INFO:teuthology.orchestra.run.smithi099.stdout: "num_strays_delayed": 0, 2024-05-03T18:47:43.956 INFO:teuthology.orchestra.run.smithi099.stdout: "num_strays_enqueuing": 0, 2024-05-03T18:47:43.956 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_created": 439, 2024-05-03T18:47:43.956 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_enqueued": 266, 2024-05-03T18:47:43.956 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_reintegrated": 172, 2024-05-03T18:47:43.956 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_migrated": 0, 2024-05-03T18:47:43.956 INFO:teuthology.orchestra.run.smithi099.stdout: "num_recovering_processing": 0, 2024-05-03T18:47:43.956 INFO:teuthology.orchestra.run.smithi099.stdout: "num_recovering_enqueued": 0, 2024-05-03T18:47:43.956 INFO:teuthology.orchestra.run.smithi099.stdout: "num_recovering_prioritized": 0, 2024-05-03T18:47:43.956 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_started": 2, 2024-05-03T18:47:43.956 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_completed": 2, 2024-05-03T18:47:43.956 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_quiesce_path": 0, 2024-05-03T18:47:43.957 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_quiesce_inode": 0, 2024-05-03T18:47:43.957 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_enqueue_scrub": 37, 2024-05-03T18:47:43.957 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_exportdir": 0, 2024-05-03T18:47:43.957 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_flush": 0, 2024-05-03T18:47:43.957 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_fragmentdir": 12, 2024-05-03T18:47:43.957 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_fragstats": 0, 2024-05-03T18:47:43.957 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_inodestats": 0 2024-05-03T18:47:43.957 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:43.957 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_log": { 2024-05-03T18:47:43.957 INFO:teuthology.orchestra.run.smithi099.stdout: "evlrg": 0, 2024-05-03T18:47:43.957 INFO:teuthology.orchestra.run.smithi099.stdout: "evadd": 21255, 2024-05-03T18:47:43.957 INFO:teuthology.orchestra.run.smithi099.stdout: "evex": 0, 2024-05-03T18:47:43.957 INFO:teuthology.orchestra.run.smithi099.stdout: "evtrm": 0, 2024-05-03T18:47:43.957 INFO:teuthology.orchestra.run.smithi099.stdout: "ev": 21255, 2024-05-03T18:47:43.957 INFO:teuthology.orchestra.run.smithi099.stdout: "evexg": 0, 2024-05-03T18:47:43.958 INFO:teuthology.orchestra.run.smithi099.stdout: "evexd": 0, 2024-05-03T18:47:43.958 INFO:teuthology.orchestra.run.smithi099.stdout: "segadd": 56, 2024-05-03T18:47:43.958 INFO:teuthology.orchestra.run.smithi099.stdout: "segex": 0, 2024-05-03T18:47:43.958 INFO:teuthology.orchestra.run.smithi099.stdout: "segtrm": 0, 2024-05-03T18:47:43.958 INFO:teuthology.orchestra.run.smithi099.stdout: "seg": 56, 2024-05-03T18:47:43.958 INFO:teuthology.orchestra.run.smithi099.stdout: "segmjr": 3, 2024-05-03T18:47:43.958 INFO:teuthology.orchestra.run.smithi099.stdout: "segexg": 0, 2024-05-03T18:47:43.958 INFO:teuthology.orchestra.run.smithi099.stdout: "segexd": 0, 2024-05-03T18:47:43.958 INFO:teuthology.orchestra.run.smithi099.stdout: "expos": 4194304, 2024-05-03T18:47:43.958 INFO:teuthology.orchestra.run.smithi099.stdout: "wrpos": 216161932, 2024-05-03T18:47:43.958 INFO:teuthology.orchestra.run.smithi099.stdout: "rdpos": 0, 2024-05-03T18:47:43.958 INFO:teuthology.orchestra.run.smithi099.stdout: "jlat": { 2024-05-03T18:47:43.958 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 4188, 2024-05-03T18:47:43.958 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 52.932497543, 2024-05-03T18:47:43.958 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.012639087 2024-05-03T18:47:43.959 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:43.959 INFO:teuthology.orchestra.run.smithi099.stdout: "replayed": 0 2024-05-03T18:47:43.959 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:43.959 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_mem": { 2024-05-03T18:47:43.959 INFO:teuthology.orchestra.run.smithi099.stdout: "ino": 2703, 2024-05-03T18:47:43.959 INFO:teuthology.orchestra.run.smithi099.stdout: "ino+": 3006, 2024-05-03T18:47:43.959 INFO:teuthology.orchestra.run.smithi099.stdout: "ino-": 303, 2024-05-03T18:47:43.959 INFO:teuthology.orchestra.run.smithi099.stdout: "dir": 132, 2024-05-03T18:47:43.959 INFO:teuthology.orchestra.run.smithi099.stdout: "dir+": 145, 2024-05-03T18:47:43.959 INFO:teuthology.orchestra.run.smithi099.stdout: "dir-": 13, 2024-05-03T18:47:43.959 INFO:teuthology.orchestra.run.smithi099.stdout: "dn": 3013, 2024-05-03T18:47:43.959 INFO:teuthology.orchestra.run.smithi099.stdout: "dn+": 3377, 2024-05-03T18:47:43.959 INFO:teuthology.orchestra.run.smithi099.stdout: "dn-": 364, 2024-05-03T18:47:43.959 INFO:teuthology.orchestra.run.smithi099.stdout: "cap": 1046, 2024-05-03T18:47:43.959 INFO:teuthology.orchestra.run.smithi099.stdout: "cap+": 1329, 2024-05-03T18:47:43.959 INFO:teuthology.orchestra.run.smithi099.stdout: "cap-": 283, 2024-05-03T18:47:43.960 INFO:teuthology.orchestra.run.smithi099.stdout: "rss": 78024, 2024-05-03T18:47:43.960 INFO:teuthology.orchestra.run.smithi099.stdout: "heap": 231800 2024-05-03T18:47:43.960 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:43.960 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_server": { 2024-05-03T18:47:43.960 INFO:teuthology.orchestra.run.smithi099.stdout: "dispatch_client_request": 9916, 2024-05-03T18:47:43.960 INFO:teuthology.orchestra.run.smithi099.stdout: "dispatch_server_request": 0, 2024-05-03T18:47:43.960 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_request": 8966, 2024-05-03T18:47:43.960 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_session": 337, 2024-05-03T18:47:43.960 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_peer_request": 0, 2024-05-03T18:47:43.960 INFO:teuthology.orchestra.run.smithi099.stdout: "req_create_latency": { 2024-05-03T18:47:43.960 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 1280, 2024-05-03T18:47:43.960 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 5.874382681, 2024-05-03T18:47:43.960 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.004589361 2024-05-03T18:47:43.960 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:43.960 INFO:teuthology.orchestra.run.smithi099.stdout: "req_getattr_latency": { 2024-05-03T18:47:43.961 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 670, 2024-05-03T18:47:43.961 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.508923533, 2024-05-03T18:47:43.961 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000759587 2024-05-03T18:47:43.961 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:43.961 INFO:teuthology.orchestra.run.smithi099.stdout: "req_getfilelock_latency": { 2024-05-03T18:47:43.961 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-05-03T18:47:43.961 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-05-03T18:47:43.961 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-05-03T18:47:43.961 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:43.961 INFO:teuthology.orchestra.run.smithi099.stdout: "req_link_latency": { 2024-05-03T18:47:43.961 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 172, 2024-05-03T18:47:43.961 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.822352516, 2024-05-03T18:47:43.961 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.004781119 2024-05-03T18:47:43.961 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:43.961 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookup_latency": { 2024-05-03T18:47:43.962 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 3698, 2024-05-03T18:47:43.962 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 4.705097787, 2024-05-03T18:47:43.962 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.001272335 2024-05-03T18:47:43.962 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:43.962 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookuphash_latency": { 2024-05-03T18:47:43.962 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-05-03T18:47:43.962 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-05-03T18:47:43.962 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-05-03T18:47:43.962 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:43.962 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupino_latency": { 2024-05-03T18:47:43.962 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-05-03T18:47:43.962 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-05-03T18:47:43.962 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-05-03T18:47:43.962 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:43.963 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupname_latency": { 2024-05-03T18:47:43.963 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-05-03T18:47:43.963 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-05-03T18:47:43.963 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-05-03T18:47:43.963 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:43.963 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupparent_latency": { 2024-05-03T18:47:43.963 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-05-03T18:47:43.963 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-05-03T18:47:43.963 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-05-03T18:47:43.963 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:43.963 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupsnap_latency": { 2024-05-03T18:47:43.963 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-05-03T18:47:43.963 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-05-03T18:47:43.963 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-05-03T18:47:43.963 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:43.964 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lssnap_latency": { 2024-05-03T18:47:43.964 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 20, 2024-05-03T18:47:43.964 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.011305483, 2024-05-03T18:47:43.964 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000565274 2024-05-03T18:47:43.964 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:43.964 INFO:teuthology.orchestra.run.smithi099.stdout: "req_mkdir_latency": { 2024-05-03T18:47:43.964 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 37, 2024-05-03T18:47:43.965 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.137857976, 2024-05-03T18:47:43.965 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.003725891 2024-05-03T18:47:43.965 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:43.965 INFO:teuthology.orchestra.run.smithi099.stdout: "req_mknod_latency": { 2024-05-03T18:47:43.965 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-05-03T18:47:43.965 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-05-03T18:47:43.965 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-05-03T18:47:43.965 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:43.965 INFO:teuthology.orchestra.run.smithi099.stdout: "req_mksnap_latency": { 2024-05-03T18:47:43.965 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 20, 2024-05-03T18:47:43.965 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.733314025, 2024-05-03T18:47:43.965 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.036665701 2024-05-03T18:47:43.965 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:43.965 INFO:teuthology.orchestra.run.smithi099.stdout: "req_open_latency": { 2024-05-03T18:47:43.966 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 2266, 2024-05-03T18:47:43.966 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 4.837535243, 2024-05-03T18:47:43.966 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.002134834 2024-05-03T18:47:43.966 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:43.966 INFO:teuthology.orchestra.run.smithi099.stdout: "req_readdir_latency": { 2024-05-03T18:47:43.966 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 154, 2024-05-03T18:47:43.966 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.490409438, 2024-05-03T18:47:43.966 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.003184476 2024-05-03T18:47:43.966 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:43.966 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rename_latency": { 2024-05-03T18:47:43.966 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 422, 2024-05-03T18:47:43.966 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 4.768785404, 2024-05-03T18:47:43.966 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.011300439 2024-05-03T18:47:43.966 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:43.967 INFO:teuthology.orchestra.run.smithi099.stdout: "req_renamesnap_latency": { 2024-05-03T18:47:43.967 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-05-03T18:47:43.967 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-05-03T18:47:43.967 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-05-03T18:47:43.967 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:43.967 INFO:teuthology.orchestra.run.smithi099.stdout: "req_snapdiff_latency": { 2024-05-03T18:47:43.967 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-05-03T18:47:43.967 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-05-03T18:47:43.967 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-05-03T18:47:43.967 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:43.967 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rmdir_latency": { 2024-05-03T18:47:43.967 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 2, 2024-05-03T18:47:43.967 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.015763001, 2024-05-03T18:47:43.967 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.007881500 2024-05-03T18:47:43.967 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:43.968 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rmsnap_latency": { 2024-05-03T18:47:43.968 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 14, 2024-05-03T18:47:43.968 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.232383484, 2024-05-03T18:47:43.968 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.016598820 2024-05-03T18:47:43.968 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:43.968 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rmxattr_latency": { 2024-05-03T18:47:43.968 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-05-03T18:47:43.968 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-05-03T18:47:43.968 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-05-03T18:47:43.968 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:43.968 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setattr_latency": { 2024-05-03T18:47:43.968 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 2, 2024-05-03T18:47:43.968 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.002096857, 2024-05-03T18:47:43.968 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.001048428 2024-05-03T18:47:43.969 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:43.969 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setdirlayout_latency": { 2024-05-03T18:47:43.969 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-05-03T18:47:43.969 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-05-03T18:47:43.969 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-05-03T18:47:43.969 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:43.969 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setfilelock_latency": { 2024-05-03T18:47:43.969 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-05-03T18:47:43.969 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-05-03T18:47:43.969 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-05-03T18:47:43.969 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:43.969 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setlayout_latency": { 2024-05-03T18:47:43.969 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-05-03T18:47:43.970 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-05-03T18:47:43.970 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-05-03T18:47:43.970 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:43.970 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setxattr_latency": { 2024-05-03T18:47:43.970 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 8, 2024-05-03T18:47:43.970 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.052864832, 2024-05-03T18:47:43.970 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.006608104 2024-05-03T18:47:43.970 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:43.970 INFO:teuthology.orchestra.run.smithi099.stdout: "req_symlink_latency": { 2024-05-03T18:47:43.970 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-05-03T18:47:43.970 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-05-03T18:47:43.970 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-05-03T18:47:43.970 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:43.970 INFO:teuthology.orchestra.run.smithi099.stdout: "req_unlink_latency": { 2024-05-03T18:47:43.971 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 201, 2024-05-03T18:47:43.971 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.727048120, 2024-05-03T18:47:43.971 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.003617154 2024-05-03T18:47:43.971 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:43.971 INFO:teuthology.orchestra.run.smithi099.stdout: "cap_revoke_eviction": 0, 2024-05-03T18:47:43.971 INFO:teuthology.orchestra.run.smithi099.stdout: "cap_acquisition_throttle": 0, 2024-05-03T18:47:43.971 INFO:teuthology.orchestra.run.smithi099.stdout: "req_getvxattr_latency": { 2024-05-03T18:47:43.971 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-05-03T18:47:43.971 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-05-03T18:47:43.971 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-05-03T18:47:43.971 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-03T18:47:43.971 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:43.971 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_sessions": { 2024-05-03T18:47:43.971 INFO:teuthology.orchestra.run.smithi099.stdout: "session_count": 2, 2024-05-03T18:47:43.972 INFO:teuthology.orchestra.run.smithi099.stdout: "session_add": 6, 2024-05-03T18:47:43.972 INFO:teuthology.orchestra.run.smithi099.stdout: "session_remove": 4, 2024-05-03T18:47:43.972 INFO:teuthology.orchestra.run.smithi099.stdout: "sessions_open": 2, 2024-05-03T18:47:43.972 INFO:teuthology.orchestra.run.smithi099.stdout: "sessions_stale": 0, 2024-05-03T18:47:43.972 INFO:teuthology.orchestra.run.smithi099.stdout: "total_load": 1368, 2024-05-03T18:47:43.972 INFO:teuthology.orchestra.run.smithi099.stdout: "average_load": 684, 2024-05-03T18:47:43.972 INFO:teuthology.orchestra.run.smithi099.stdout: "avg_session_uptime": 1206, 2024-05-03T18:47:43.972 INFO:teuthology.orchestra.run.smithi099.stdout: "mdthresh_evicted": 0 2024-05-03T18:47:43.972 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:43.973 INFO:teuthology.orchestra.run.smithi099.stdout: "mempool": { 2024-05-03T18:47:43.973 INFO:teuthology.orchestra.run.smithi099.stdout: "bloom_filter_bytes": 0, 2024-05-03T18:47:43.973 INFO:teuthology.orchestra.run.smithi099.stdout: "bloom_filter_items": 0, 2024-05-03T18:47:43.973 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_alloc_bytes": 0, 2024-05-03T18:47:43.973 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_alloc_items": 0, 2024-05-03T18:47:43.973 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_data_bytes": 0, 2024-05-03T18:47:43.973 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_data_items": 0, 2024-05-03T18:47:43.973 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-03T18:47:43.973 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_onode_items": 0, 2024-05-03T18:47:43.973 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-03T18:47:43.973 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_meta_items": 0, 2024-05-03T18:47:43.973 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_other_bytes": 0, 2024-05-03T18:47:43.973 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_other_items": 0, 2024-05-03T18:47:43.973 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-03T18:47:43.974 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_buffer_items": 0, 2024-05-03T18:47:43.974 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_extent_bytes": 0, 2024-05-03T18:47:43.974 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_extent_items": 0, 2024-05-03T18:47:43.974 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_blob_bytes": 0, 2024-05-03T18:47:43.974 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_blob_items": 0, 2024-05-03T18:47:43.974 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-03T18:47:43.974 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_shared_blob_items": 0, 2024-05-03T18:47:43.974 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-03T18:47:43.974 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_inline_bl_items": 0, 2024-05-03T18:47:43.974 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_fsck_bytes": 0, 2024-05-03T18:47:43.974 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_fsck_items": 0, 2024-05-03T18:47:43.974 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_txc_bytes": 0, 2024-05-03T18:47:43.974 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_txc_items": 0, 2024-05-03T18:47:43.975 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-03T18:47:43.975 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_deferred_items": 0, 2024-05-03T18:47:43.975 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_bytes": 0, 2024-05-03T18:47:43.975 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_items": 0, 2024-05-03T18:47:43.975 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_bytes": 0, 2024-05-03T18:47:43.975 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_items": 0, 2024-05-03T18:47:43.975 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_reader_bytes": 0, 2024-05-03T18:47:43.975 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_reader_items": 0, 2024-05-03T18:47:43.975 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_writer_bytes": 0, 2024-05-03T18:47:43.975 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_writer_items": 0, 2024-05-03T18:47:43.975 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_anon_bytes": 749619, 2024-05-03T18:47:43.975 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_anon_items": 232, 2024-05-03T18:47:43.975 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_meta_bytes": 0, 2024-05-03T18:47:43.975 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_meta_items": 0, 2024-05-03T18:47:43.975 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_bytes": 0, 2024-05-03T18:47:43.976 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_items": 0, 2024-05-03T18:47:43.976 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_mapbl_bytes": 0, 2024-05-03T18:47:43.976 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_mapbl_items": 0, 2024-05-03T18:47:43.976 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_pglog_bytes": 0, 2024-05-03T18:47:43.976 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_pglog_items": 0, 2024-05-03T18:47:43.976 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_bytes": 9648, 2024-05-03T18:47:43.976 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_items": 202, 2024-05-03T18:47:43.976 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_mapping_bytes": 0, 2024-05-03T18:47:43.976 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_mapping_items": 0, 2024-05-03T18:47:43.976 INFO:teuthology.orchestra.run.smithi099.stdout: "pgmap_bytes": 0, 2024-05-03T18:47:43.976 INFO:teuthology.orchestra.run.smithi099.stdout: "pgmap_items": 0, 2024-05-03T18:47:43.976 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_co_bytes": 8970996, 2024-05-03T18:47:43.976 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_co_items": 75485, 2024-05-03T18:47:43.976 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_1_bytes": 0, 2024-05-03T18:47:43.977 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_1_items": 0, 2024-05-03T18:47:43.977 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_2_bytes": 0, 2024-05-03T18:47:43.977 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_2_items": 0 2024-05-03T18:47:43.977 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:43.977 INFO:teuthology.orchestra.run.smithi099.stdout: "objecter": { 2024-05-03T18:47:43.977 INFO:teuthology.orchestra.run.smithi099.stdout: "op_active": 0, 2024-05-03T18:47:43.977 INFO:teuthology.orchestra.run.smithi099.stdout: "op_laggy": 0, 2024-05-03T18:47:43.977 INFO:teuthology.orchestra.run.smithi099.stdout: "op_send": 43137, 2024-05-03T18:47:43.977 INFO:teuthology.orchestra.run.smithi099.stdout: "op_send_bytes": 213530560, 2024-05-03T18:47:43.977 INFO:teuthology.orchestra.run.smithi099.stdout: "op_resend": 37, 2024-05-03T18:47:43.977 INFO:teuthology.orchestra.run.smithi099.stdout: "op_reply": 43137, 2024-05-03T18:47:43.977 INFO:teuthology.orchestra.run.smithi099.stdout: "op_latency": { 2024-05-03T18:47:43.977 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 43137, 2024-05-03T18:47:43.977 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 148.217565651, 2024-05-03T18:47:43.978 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.003435972 2024-05-03T18:47:43.978 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:43.978 INFO:teuthology.orchestra.run.smithi099.stdout: "op_inflight": 0, 2024-05-03T18:47:43.978 INFO:teuthology.orchestra.run.smithi099.stdout: "oplen_avg": { 2024-05-03T18:47:43.978 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 43137, 2024-05-03T18:47:43.978 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 43892 2024-05-03T18:47:43.978 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:43.978 INFO:teuthology.orchestra.run.smithi099.stdout: "op": 43137, 2024-05-03T18:47:43.978 INFO:teuthology.orchestra.run.smithi099.stdout: "op_r": 36831, 2024-05-03T18:47:43.978 INFO:teuthology.orchestra.run.smithi099.stdout: "op_w": 6306, 2024-05-03T18:47:43.978 INFO:teuthology.orchestra.run.smithi099.stdout: "op_rmw": 0, 2024-05-03T18:47:43.978 INFO:teuthology.orchestra.run.smithi099.stdout: "op_pg": 0, 2024-05-03T18:47:43.978 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_stat": 2, 2024-05-03T18:47:43.978 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_create": 11, 2024-05-03T18:47:43.979 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_read": 180, 2024-05-03T18:47:43.979 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_write": 4331, 2024-05-03T18:47:43.979 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_writefull": 182, 2024-05-03T18:47:43.979 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_writesame": 0, 2024-05-03T18:47:43.979 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_append": 0, 2024-05-03T18:47:43.979 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_zero": 0, 2024-05-03T18:47:43.979 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_truncate": 5, 2024-05-03T18:47:43.979 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_delete": 1138, 2024-05-03T18:47:43.979 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_mapext": 0, 2024-05-03T18:47:43.979 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_sparse_read": 0, 2024-05-03T18:47:43.979 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_clonerange": 0, 2024-05-03T18:47:43.979 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_getxattr": 36649, 2024-05-03T18:47:43.979 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_setxattr": 22, 2024-05-03T18:47:43.980 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_cmpxattr": 0, 2024-05-03T18:47:43.980 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_rmxattr": 0, 2024-05-03T18:47:43.980 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_resetxattrs": 0, 2024-05-03T18:47:43.980 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_call": 0, 2024-05-03T18:47:43.980 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_watch": 0, 2024-05-03T18:47:43.980 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_notify": 0, 2024-05-03T18:47:43.980 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_src_cmpxattr": 0, 2024-05-03T18:47:43.980 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_pgls": 0, 2024-05-03T18:47:43.980 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_pgls_filter": 0, 2024-05-03T18:47:43.980 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_other": 0, 2024-05-03T18:47:43.980 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_active": 0, 2024-05-03T18:47:43.980 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_send": 0, 2024-05-03T18:47:43.980 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_resend": 0, 2024-05-03T18:47:43.981 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_ping": 0, 2024-05-03T18:47:43.981 INFO:teuthology.orchestra.run.smithi099.stdout: "poolop_active": 0, 2024-05-03T18:47:43.981 INFO:teuthology.orchestra.run.smithi099.stdout: "poolop_send": 0, 2024-05-03T18:47:43.981 INFO:teuthology.orchestra.run.smithi099.stdout: "poolop_resend": 0, 2024-05-03T18:47:43.981 INFO:teuthology.orchestra.run.smithi099.stdout: "poolstat_active": 0, 2024-05-03T18:47:43.981 INFO:teuthology.orchestra.run.smithi099.stdout: "poolstat_send": 0, 2024-05-03T18:47:43.981 INFO:teuthology.orchestra.run.smithi099.stdout: "poolstat_resend": 0, 2024-05-03T18:47:43.981 INFO:teuthology.orchestra.run.smithi099.stdout: "statfs_active": 0, 2024-05-03T18:47:43.981 INFO:teuthology.orchestra.run.smithi099.stdout: "statfs_send": 0, 2024-05-03T18:47:43.981 INFO:teuthology.orchestra.run.smithi099.stdout: "statfs_resend": 0, 2024-05-03T18:47:43.981 INFO:teuthology.orchestra.run.smithi099.stdout: "command_active": 0, 2024-05-03T18:47:43.981 INFO:teuthology.orchestra.run.smithi099.stdout: "command_send": 0, 2024-05-03T18:47:43.981 INFO:teuthology.orchestra.run.smithi099.stdout: "command_resend": 0, 2024-05-03T18:47:43.981 INFO:teuthology.orchestra.run.smithi099.stdout: "map_epoch": 120, 2024-05-03T18:47:43.982 INFO:teuthology.orchestra.run.smithi099.stdout: "map_full": 0, 2024-05-03T18:47:43.982 INFO:teuthology.orchestra.run.smithi099.stdout: "map_inc": 42, 2024-05-03T18:47:43.982 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_sessions": 12, 2024-05-03T18:47:43.982 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_session_open": 12, 2024-05-03T18:47:43.982 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_session_close": 0, 2024-05-03T18:47:43.982 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_laggy": 0, 2024-05-03T18:47:43.982 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_wr": 1107, 2024-05-03T18:47:43.982 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_rd": 0, 2024-05-03T18:47:43.982 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_del": 265 2024-05-03T18:47:43.982 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:43.982 INFO:teuthology.orchestra.run.smithi099.stdout: "oft": { 2024-05-03T18:47:43.982 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_objs": 1, 2024-05-03T18:47:43.982 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_kv_pairs": 0, 2024-05-03T18:47:43.983 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_updates": 0, 2024-05-03T18:47:43.983 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_removes": 0 2024-05-03T18:47:43.983 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:43.983 INFO:teuthology.orchestra.run.smithi099.stdout: "osd-slow-ops": { 2024-05-03T18:47:43.983 INFO:teuthology.orchestra.run.smithi099.stdout: "slow_ops_count": 1 2024-05-03T18:47:43.983 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:43.983 INFO:teuthology.orchestra.run.smithi099.stdout: "purge_queue": { 2024-05-03T18:47:43.983 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing_ops": 0, 2024-05-03T18:47:43.983 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing_ops_high_water": 38, 2024-05-03T18:47:43.983 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing": 0, 2024-05-03T18:47:43.983 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing_high_water": 9, 2024-05-03T18:47:43.983 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executed_ops": 569, 2024-05-03T18:47:43.983 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executed": 266, 2024-05-03T18:47:43.983 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_item_in_journal": 0 2024-05-03T18:47:43.983 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:43.984 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-03T18:47:43.984 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-05-03T18:47:43.984 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 104857600, 2024-05-03T18:47:43.984 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-05-03T18:47:43.984 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 74374, 2024-05-03T18:47:43.984 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 16241459, 2024-05-03T18:47:43.984 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-05-03T18:47:43.984 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 74374, 2024-05-03T18:47:43.984 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 0, 2024-05-03T18:47:43.984 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 0, 2024-05-03T18:47:43.984 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 74374, 2024-05-03T18:47:43.984 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 16241459, 2024-05-03T18:47:43.984 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-05-03T18:47:43.984 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-05-03T18:47:43.984 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-05-03T18:47:43.985 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-05-03T18:47:43.985 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-03T18:47:43.985 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:43.985 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-objecter_bytes": { 2024-05-03T18:47:43.985 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-05-03T18:47:43.985 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 104857600, 2024-05-03T18:47:43.985 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-05-03T18:47:43.985 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 0, 2024-05-03T18:47:43.985 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 0, 2024-05-03T18:47:43.985 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-05-03T18:47:43.985 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 0, 2024-05-03T18:47:43.985 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 43137, 2024-05-03T18:47:43.985 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 213560706, 2024-05-03T18:47:43.985 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 43137, 2024-05-03T18:47:43.985 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 213560706, 2024-05-03T18:47:43.986 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-05-03T18:47:43.986 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-05-03T18:47:43.986 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-05-03T18:47:43.986 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-05-03T18:47:43.986 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-03T18:47:43.986 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:43.986 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-objecter_ops": { 2024-05-03T18:47:43.986 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-05-03T18:47:43.986 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 1024, 2024-05-03T18:47:43.986 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-05-03T18:47:43.986 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 0, 2024-05-03T18:47:43.986 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 0, 2024-05-03T18:47:43.986 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-05-03T18:47:43.987 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 0, 2024-05-03T18:47:43.987 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 43137, 2024-05-03T18:47:43.987 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 43137, 2024-05-03T18:47:43.987 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 43137, 2024-05-03T18:47:43.987 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 43137, 2024-05-03T18:47:43.987 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-05-03T18:47:43.987 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-05-03T18:47:43.987 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-05-03T18:47:43.987 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-05-03T18:47:43.987 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-03T18:47:43.987 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:43.987 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-write_buf_throttle": { 2024-05-03T18:47:43.987 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-05-03T18:47:43.988 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 3758096384, 2024-05-03T18:47:43.988 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-05-03T18:47:43.988 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 266, 2024-05-03T18:47:43.988 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 30146, 2024-05-03T18:47:43.988 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-05-03T18:47:43.988 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 266, 2024-05-03T18:47:43.988 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 0, 2024-05-03T18:47:43.988 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 0, 2024-05-03T18:47:43.988 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 143, 2024-05-03T18:47:43.988 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 30146, 2024-05-03T18:47:43.988 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-05-03T18:47:43.988 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-05-03T18:47:43.988 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-05-03T18:47:43.988 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-05-03T18:47:43.988 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-03T18:47:43.989 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:43.989 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-write_buf_throttle-0x5633198ee500": { 2024-05-03T18:47:43.989 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-05-03T18:47:43.989 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 3758096384, 2024-05-03T18:47:43.989 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-05-03T18:47:43.989 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 21255, 2024-05-03T18:47:43.989 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 211967628, 2024-05-03T18:47:43.989 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-05-03T18:47:43.989 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 21255, 2024-05-03T18:47:43.989 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 0, 2024-05-03T18:47:43.989 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 0, 2024-05-03T18:47:43.989 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 4188, 2024-05-03T18:47:43.989 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 211967628, 2024-05-03T18:47:43.989 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-05-03T18:47:43.990 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-05-03T18:47:43.990 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-05-03T18:47:43.990 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-05-03T18:47:43.990 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-03T18:47:43.990 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-03T18:47:43.990 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-05-03T18:47:44.297 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:47:44 smithi099 ceph-mon[31658]: pgmap v648: 129 pgs: 129 active+clean; 3.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:47:44.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:47:44 smithi150 ceph-mon[32492]: pgmap v648: 129 pgs: 129 active+clean; 3.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:47:44.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:47:44 smithi033 ceph-mon[28330]: pgmap v648: 129 pgs: 129 active+clean; 3.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:47:45.316 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-05-03T18:47:45.317 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=20 2024-05-03T18:47:45.317 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=14 2024-05-03T18:47:45.317 DEBUG:tasks.check_counter:Getting stats from e 2024-05-03T18:47:45.318 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph daemon mds.e perf dump 2024-05-03T18:47:45.746 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.b/config 2024-05-03T18:47:46.270 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:47:46 smithi099 ceph-mon[31658]: pgmap v649: 129 pgs: 129 active+clean; 3.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:47:46.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:47:46 smithi150 ceph-mon[32492]: pgmap v649: 129 pgs: 129 active+clean; 3.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:47:46.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:47:46 smithi033 ceph-mon[28330]: pgmap v649: 129 pgs: 129 active+clean; 3.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:47:48.381 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:47:48 smithi099 ceph-mon[31658]: pgmap v650: 129 pgs: 129 active+clean; 3.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:47:48.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:47:48 smithi150 ceph-mon[32492]: pgmap v650: 129 pgs: 129 active+clean; 3.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:47:48.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:47:48 smithi033 ceph-mon[28330]: pgmap v650: 129 pgs: 129 active+clean; 3.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:47:48.770 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-05-03T18:47:48.770 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-0": { 2024-05-03T18:47:48.770 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 0, 2024-05-03T18:47:48.770 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 2, 2024-05-03T18:47:48.770 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 0, 2024-05-03T18:47:48.770 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 0, 2024-05-03T18:47:48.771 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 4, 2024-05-03T18:47:48.771 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 0, 2024-05-03T18:47:48.771 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.003981240, 2024-05-03T18:47:48.771 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.000000000, 2024-05-03T18:47:48.771 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.000772790, 2024-05-03T18:47:48.771 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-03T18:47:48.771 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2024-05-03T18:47:48.771 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-05-03T18:47:48.771 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-05-03T18:47:48.771 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-05-03T18:47:48.771 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:48.771 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2024-05-03T18:47:48.771 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-05-03T18:47:48.771 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-05-03T18:47:48.771 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-05-03T18:47:48.772 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:48.772 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-03T18:47:48.772 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 0 2024-05-03T18:47:48.772 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:48.772 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-1": { 2024-05-03T18:47:48.772 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 3, 2024-05-03T18:47:48.772 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 320, 2024-05-03T18:47:48.772 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 480, 2024-05-03T18:47:48.772 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 194272, 2024-05-03T18:47:48.772 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 6, 2024-05-03T18:47:48.772 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 3, 2024-05-03T18:47:48.772 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.040956937, 2024-05-03T18:47:48.772 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.025576643, 2024-05-03T18:47:48.772 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.007687421, 2024-05-03T18:47:48.772 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-03T18:47:48.772 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2024-05-03T18:47:48.773 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 317, 2024-05-03T18:47:48.773 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.052835960, 2024-05-03T18:47:48.773 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000166674 2024-05-03T18:47:48.773 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:48.773 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2024-05-03T18:47:48.773 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-05-03T18:47:48.773 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-05-03T18:47:48.773 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-05-03T18:47:48.773 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:48.773 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 480, 2024-05-03T18:47:48.773 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 194272 2024-05-03T18:47:48.773 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:48.773 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-2": { 2024-05-03T18:47:48.773 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 432, 2024-05-03T18:47:48.773 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 420, 2024-05-03T18:47:48.773 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 1623168, 2024-05-03T18:47:48.774 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 258944, 2024-05-03T18:47:48.774 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 2, 2024-05-03T18:47:48.774 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 1, 2024-05-03T18:47:48.774 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.169751519, 2024-05-03T18:47:48.774 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.060335226, 2024-05-03T18:47:48.774 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.117364760, 2024-05-03T18:47:48.774 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.019635233, 2024-05-03T18:47:48.774 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2024-05-03T18:47:48.774 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 420, 2024-05-03T18:47:48.774 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.046889432, 2024-05-03T18:47:48.774 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000111641 2024-05-03T18:47:48.774 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:48.774 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2024-05-03T18:47:48.775 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-05-03T18:47:48.775 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-05-03T18:47:48.775 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-05-03T18:47:48.775 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:48.775 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 1623168, 2024-05-03T18:47:48.775 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 258944 2024-05-03T18:47:48.775 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:48.775 INFO:teuthology.orchestra.run.smithi099.stdout: "cct": { 2024-05-03T18:47:48.775 INFO:teuthology.orchestra.run.smithi099.stdout: "total_workers": 0, 2024-05-03T18:47:48.775 INFO:teuthology.orchestra.run.smithi099.stdout: "unhealthy_workers": 0 2024-05-03T18:47:48.775 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:48.775 INFO:teuthology.orchestra.run.smithi099.stdout: "mempool": { 2024-05-03T18:47:48.775 INFO:teuthology.orchestra.run.smithi099.stdout: "bloom_filter_bytes": 0, 2024-05-03T18:47:48.775 INFO:teuthology.orchestra.run.smithi099.stdout: "bloom_filter_items": 0, 2024-05-03T18:47:48.775 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_alloc_bytes": 0, 2024-05-03T18:47:48.775 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_alloc_items": 0, 2024-05-03T18:47:48.776 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_data_bytes": 0, 2024-05-03T18:47:48.776 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_data_items": 0, 2024-05-03T18:47:48.776 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-03T18:47:48.776 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_onode_items": 0, 2024-05-03T18:47:48.776 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-03T18:47:48.776 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_meta_items": 0, 2024-05-03T18:47:48.776 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_other_bytes": 0, 2024-05-03T18:47:48.776 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_other_items": 0, 2024-05-03T18:47:48.776 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-03T18:47:48.776 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_buffer_items": 0, 2024-05-03T18:47:48.776 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_extent_bytes": 0, 2024-05-03T18:47:48.776 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_extent_items": 0, 2024-05-03T18:47:48.776 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_blob_bytes": 0, 2024-05-03T18:47:48.776 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_blob_items": 0, 2024-05-03T18:47:48.776 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-03T18:47:48.776 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_shared_blob_items": 0, 2024-05-03T18:47:48.777 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-03T18:47:48.777 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_inline_bl_items": 0, 2024-05-03T18:47:48.777 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_fsck_bytes": 0, 2024-05-03T18:47:48.777 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_fsck_items": 0, 2024-05-03T18:47:48.777 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_txc_bytes": 0, 2024-05-03T18:47:48.777 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_txc_items": 0, 2024-05-03T18:47:48.777 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-03T18:47:48.777 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_deferred_items": 0, 2024-05-03T18:47:48.777 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_bytes": 0, 2024-05-03T18:47:48.777 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_items": 0, 2024-05-03T18:47:48.777 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_bytes": 0, 2024-05-03T18:47:48.777 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_items": 0, 2024-05-03T18:47:48.777 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_reader_bytes": 0, 2024-05-03T18:47:48.777 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_reader_items": 0, 2024-05-03T18:47:48.777 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_writer_bytes": 0, 2024-05-03T18:47:48.778 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_writer_items": 0, 2024-05-03T18:47:48.778 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_anon_bytes": 148731, 2024-05-03T18:47:48.778 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_anon_items": 31, 2024-05-03T18:47:48.778 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_meta_bytes": 0, 2024-05-03T18:47:48.778 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_meta_items": 0, 2024-05-03T18:47:48.778 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_bytes": 0, 2024-05-03T18:47:48.778 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_items": 0, 2024-05-03T18:47:48.778 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_mapbl_bytes": 0, 2024-05-03T18:47:48.778 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_mapbl_items": 0, 2024-05-03T18:47:48.778 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_pglog_bytes": 0, 2024-05-03T18:47:48.778 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_pglog_items": 0, 2024-05-03T18:47:48.778 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_bytes": 0, 2024-05-03T18:47:48.778 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_items": 0, 2024-05-03T18:47:48.778 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_mapping_bytes": 0, 2024-05-03T18:47:48.779 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_mapping_items": 0, 2024-05-03T18:47:48.779 INFO:teuthology.orchestra.run.smithi099.stdout: "pgmap_bytes": 0, 2024-05-03T18:47:48.779 INFO:teuthology.orchestra.run.smithi099.stdout: "pgmap_items": 0, 2024-05-03T18:47:48.779 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_co_bytes": 928, 2024-05-03T18:47:48.779 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_co_items": 2, 2024-05-03T18:47:48.779 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_1_bytes": 0, 2024-05-03T18:47:48.779 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_1_items": 0, 2024-05-03T18:47:48.779 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_2_bytes": 0, 2024-05-03T18:47:48.779 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_2_items": 0 2024-05-03T18:47:48.779 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:48.779 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-03T18:47:48.779 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-05-03T18:47:48.779 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 104857600, 2024-05-03T18:47:48.779 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-05-03T18:47:48.779 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 431, 2024-05-03T18:47:48.779 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 1559785, 2024-05-03T18:47:48.780 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-05-03T18:47:48.780 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 431, 2024-05-03T18:47:48.780 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 0, 2024-05-03T18:47:48.780 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 0, 2024-05-03T18:47:48.780 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 431, 2024-05-03T18:47:48.780 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 1559785, 2024-05-03T18:47:48.780 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-05-03T18:47:48.780 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-05-03T18:47:48.780 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-05-03T18:47:48.780 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-05-03T18:47:48.780 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-03T18:47:48.780 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-03T18:47:48.780 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-05-03T18:47:49.706 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.e 2024-05-03T18:47:49.706 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.e 2024-05-03T18:47:49.706 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.e 2024-05-03T18:47:49.706 DEBUG:tasks.check_counter:Getting stats from h 2024-05-03T18:47:49.706 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph daemon mds.h perf dump 2024-05-03T18:47:50.232 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.b/config 2024-05-03T18:47:50.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:47:50 smithi150 ceph-mon[32492]: pgmap v651: 129 pgs: 129 active+clean; 3.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:47:50.386 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:47:50 smithi099 ceph-mon[31658]: pgmap v651: 129 pgs: 129 active+clean; 3.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:47:50.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:47:50 smithi033 ceph-mon[28330]: pgmap v651: 129 pgs: 129 active+clean; 3.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:47:52.290 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:47:52 smithi099 ceph-mon[31658]: pgmap v652: 129 pgs: 129 active+clean; 3.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 2 op/s 2024-05-03T18:47:52.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:47:52 smithi150 ceph-mon[32492]: pgmap v652: 129 pgs: 129 active+clean; 3.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 2 op/s 2024-05-03T18:47:52.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:47:52 smithi033 ceph-mon[28330]: pgmap v652: 129 pgs: 129 active+clean; 3.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 2 op/s 2024-05-03T18:47:52.631 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-05-03T18:47:52.632 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-0": { 2024-05-03T18:47:52.632 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 3071, 2024-05-03T18:47:52.632 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 3071, 2024-05-03T18:47:52.632 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 87331292, 2024-05-03T18:47:52.632 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 975764, 2024-05-03T18:47:52.632 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 8, 2024-05-03T18:47:52.632 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 4, 2024-05-03T18:47:52.632 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.838313900, 2024-05-03T18:47:52.632 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.260311820, 2024-05-03T18:47:52.632 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.357145187, 2024-05-03T18:47:52.633 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.163879202, 2024-05-03T18:47:52.633 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2024-05-03T18:47:52.633 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 3070, 2024-05-03T18:47:52.633 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.276856203, 2024-05-03T18:47:52.633 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000090181 2024-05-03T18:47:52.633 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:52.633 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2024-05-03T18:47:52.633 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-05-03T18:47:52.633 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-05-03T18:47:52.633 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-05-03T18:47:52.633 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:52.633 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 2912, 2024-05-03T18:47:52.633 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 448 2024-05-03T18:47:52.634 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:52.634 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-1": { 2024-05-03T18:47:52.634 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 1048, 2024-05-03T18:47:52.634 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 1362, 2024-05-03T18:47:52.634 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 75992148, 2024-05-03T18:47:52.634 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 962454, 2024-05-03T18:47:52.634 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 9, 2024-05-03T18:47:52.634 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 7, 2024-05-03T18:47:52.634 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.433211810, 2024-05-03T18:47:52.634 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.090167977, 2024-05-03T18:47:52.634 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.260837019, 2024-05-03T18:47:52.634 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.056262856, 2024-05-03T18:47:52.634 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2024-05-03T18:47:52.635 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 1359, 2024-05-03T18:47:52.635 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.139837779, 2024-05-03T18:47:52.635 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000102897 2024-05-03T18:47:52.635 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:52.635 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2024-05-03T18:47:52.635 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-05-03T18:47:52.635 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-05-03T18:47:52.635 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-05-03T18:47:52.635 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:52.635 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-03T18:47:52.635 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 629760 2024-05-03T18:47:52.635 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:52.635 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-2": { 2024-05-03T18:47:52.635 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 1376, 2024-05-03T18:47:52.636 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 1365, 2024-05-03T18:47:52.636 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 52288589, 2024-05-03T18:47:52.636 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 553397, 2024-05-03T18:47:52.636 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 5, 2024-05-03T18:47:52.636 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 4, 2024-05-03T18:47:52.636 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.505156110, 2024-05-03T18:47:52.636 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.117376895, 2024-05-03T18:47:52.636 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.320120613, 2024-05-03T18:47:52.636 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.073316629, 2024-05-03T18:47:52.636 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2024-05-03T18:47:52.636 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 1364, 2024-05-03T18:47:52.636 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.132231636, 2024-05-03T18:47:52.636 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000096944 2024-05-03T18:47:52.636 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:52.637 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2024-05-03T18:47:52.637 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-05-03T18:47:52.637 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-05-03T18:47:52.637 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-05-03T18:47:52.637 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:52.637 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 1661504, 2024-05-03T18:47:52.637 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 264016 2024-05-03T18:47:52.637 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:52.637 INFO:teuthology.orchestra.run.smithi099.stdout: "cct": { 2024-05-03T18:47:52.637 INFO:teuthology.orchestra.run.smithi099.stdout: "total_workers": 1, 2024-05-03T18:47:52.637 INFO:teuthology.orchestra.run.smithi099.stdout: "unhealthy_workers": 0 2024-05-03T18:47:52.637 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:52.637 INFO:teuthology.orchestra.run.smithi099.stdout: "finisher-MDSRank": { 2024-05-03T18:47:52.637 INFO:teuthology.orchestra.run.smithi099.stdout: "queue_len": 0, 2024-05-03T18:47:52.637 INFO:teuthology.orchestra.run.smithi099.stdout: "complete_latency": { 2024-05-03T18:47:52.638 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 8341, 2024-05-03T18:47:52.638 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.412373721, 2024-05-03T18:47:52.638 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000049439 2024-05-03T18:47:52.638 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-03T18:47:52.638 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:52.638 INFO:teuthology.orchestra.run.smithi099.stdout: "finisher-PurgeQueue": { 2024-05-03T18:47:52.638 INFO:teuthology.orchestra.run.smithi099.stdout: "queue_len": 0, 2024-05-03T18:47:52.638 INFO:teuthology.orchestra.run.smithi099.stdout: "complete_latency": { 2024-05-03T18:47:52.638 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-05-03T18:47:52.638 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-05-03T18:47:52.638 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-05-03T18:47:52.638 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-03T18:47:52.638 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:52.639 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": { 2024-05-03T18:47:52.639 INFO:teuthology.orchestra.run.smithi099.stdout: "request": 0, 2024-05-03T18:47:52.639 INFO:teuthology.orchestra.run.smithi099.stdout: "reply": 0, 2024-05-03T18:47:52.639 INFO:teuthology.orchestra.run.smithi099.stdout: "reply_latency": { 2024-05-03T18:47:52.639 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-05-03T18:47:52.639 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-05-03T18:47:52.639 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-05-03T18:47:52.639 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:52.639 INFO:teuthology.orchestra.run.smithi099.stdout: "slow_reply": 0, 2024-05-03T18:47:52.639 INFO:teuthology.orchestra.run.smithi099.stdout: "forward": 0, 2024-05-03T18:47:52.639 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_fetch_complete": 0, 2024-05-03T18:47:52.639 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_fetch_keys": 0, 2024-05-03T18:47:52.639 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_commit": 0, 2024-05-03T18:47:52.639 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_split": 0, 2024-05-03T18:47:52.639 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_merge": 0, 2024-05-03T18:47:52.640 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes": 3373, 2024-05-03T18:47:52.640 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_top": 0, 2024-05-03T18:47:52.640 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_bottom": 3373, 2024-05-03T18:47:52.640 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_pin_tail": 0, 2024-05-03T18:47:52.640 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_pinned": 3373, 2024-05-03T18:47:52.640 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_expired": 0, 2024-05-03T18:47:52.640 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_with_caps": 0, 2024-05-03T18:47:52.640 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": 0, 2024-05-03T18:47:52.640 INFO:teuthology.orchestra.run.smithi099.stdout: "subtrees": 2, 2024-05-03T18:47:52.640 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse": 0, 2024-05-03T18:47:52.640 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_hit": 0, 2024-05-03T18:47:52.640 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_forward": 0, 2024-05-03T18:47:52.640 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_discover": 0, 2024-05-03T18:47:52.640 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_dir_fetch": 0, 2024-05-03T18:47:52.640 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_remote_ino": 0, 2024-05-03T18:47:52.641 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_lock": 0, 2024-05-03T18:47:52.641 INFO:teuthology.orchestra.run.smithi099.stdout: "load_cent": 0, 2024-05-03T18:47:52.641 INFO:teuthology.orchestra.run.smithi099.stdout: "q": 0, 2024-05-03T18:47:52.641 INFO:teuthology.orchestra.run.smithi099.stdout: "exported": 0, 2024-05-03T18:47:52.641 INFO:teuthology.orchestra.run.smithi099.stdout: "exported_inodes": 0, 2024-05-03T18:47:52.641 INFO:teuthology.orchestra.run.smithi099.stdout: "imported": 0, 2024-05-03T18:47:52.641 INFO:teuthology.orchestra.run.smithi099.stdout: "imported_inodes": 0, 2024-05-03T18:47:52.641 INFO:teuthology.orchestra.run.smithi099.stdout: "openino_dir_fetch": 0, 2024-05-03T18:47:52.641 INFO:teuthology.orchestra.run.smithi099.stdout: "openino_backtrace_fetch": 0, 2024-05-03T18:47:52.641 INFO:teuthology.orchestra.run.smithi099.stdout: "openino_peer_discover": 0, 2024-05-03T18:47:52.641 INFO:teuthology.orchestra.run.smithi099.stdout: "root_rfiles": 1015, 2024-05-03T18:47:52.641 INFO:teuthology.orchestra.run.smithi099.stdout: "root_rbytes": 1482306901, 2024-05-03T18:47:52.641 INFO:teuthology.orchestra.run.smithi099.stdout: "root_rsnaps": 6, 2024-05-03T18:47:52.642 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_backtrace_fetch": 0, 2024-05-03T18:47:52.642 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_set_tag": 0, 2024-05-03T18:47:52.642 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_backtrace_repaired": 0, 2024-05-03T18:47:52.642 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_inotable_repaired": 0, 2024-05-03T18:47:52.642 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_dir_inodes": 0, 2024-05-03T18:47:52.642 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_dir_base_inodes": 0, 2024-05-03T18:47:52.642 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_dirfrag_rstats": 0, 2024-05-03T18:47:52.642 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_file_inodes": 0, 2024-05-03T18:47:52.642 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_inode_file_caps": 0, 2024-05-03T18:47:52.642 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_revoke": 0, 2024-05-03T18:47:52.642 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_grant": 0, 2024-05-03T18:47:52.642 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_trunc": 0, 2024-05-03T18:47:52.642 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-03T18:47:52.643 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-03T18:47:52.643 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_caps": 0, 2024-05-03T18:47:52.643 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_caps_dirty": 0, 2024-05-03T18:47:52.643 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_cap_release": 0, 2024-05-03T18:47:52.643 INFO:teuthology.orchestra.run.smithi099.stdout: "process_request_cap_release": 0 2024-05-03T18:47:52.643 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:52.643 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_cache": { 2024-05-03T18:47:52.643 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_update": 0, 2024-05-03T18:47:52.643 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_update_receipt": 0, 2024-05-03T18:47:52.643 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_try_discover": 0, 2024-05-03T18:47:52.643 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_send_discover": 0, 2024-05-03T18:47:52.643 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_handle_discover": 0, 2024-05-03T18:47:52.643 INFO:teuthology.orchestra.run.smithi099.stdout: "num_strays": 0, 2024-05-03T18:47:52.643 INFO:teuthology.orchestra.run.smithi099.stdout: "num_strays_delayed": 0, 2024-05-03T18:47:52.644 INFO:teuthology.orchestra.run.smithi099.stdout: "num_strays_enqueuing": 0, 2024-05-03T18:47:52.644 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_created": 0, 2024-05-03T18:47:52.644 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_enqueued": 0, 2024-05-03T18:47:52.644 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_reintegrated": 0, 2024-05-03T18:47:52.644 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_migrated": 0, 2024-05-03T18:47:52.644 INFO:teuthology.orchestra.run.smithi099.stdout: "num_recovering_processing": 0, 2024-05-03T18:47:52.644 INFO:teuthology.orchestra.run.smithi099.stdout: "num_recovering_enqueued": 0, 2024-05-03T18:47:52.644 INFO:teuthology.orchestra.run.smithi099.stdout: "num_recovering_prioritized": 0, 2024-05-03T18:47:52.644 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_started": 0, 2024-05-03T18:47:52.644 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_completed": 0, 2024-05-03T18:47:52.644 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_quiesce_path": 0, 2024-05-03T18:47:52.644 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_quiesce_inode": 0, 2024-05-03T18:47:52.644 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_enqueue_scrub": 0, 2024-05-03T18:47:52.644 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_exportdir": 0, 2024-05-03T18:47:52.645 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_flush": 0, 2024-05-03T18:47:52.645 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_fragmentdir": 0, 2024-05-03T18:47:52.645 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_fragstats": 0, 2024-05-03T18:47:52.645 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_inodestats": 0 2024-05-03T18:47:52.645 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:52.645 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_log": { 2024-05-03T18:47:52.645 INFO:teuthology.orchestra.run.smithi099.stdout: "evlrg": 0, 2024-05-03T18:47:52.645 INFO:teuthology.orchestra.run.smithi099.stdout: "evadd": 0, 2024-05-03T18:47:52.645 INFO:teuthology.orchestra.run.smithi099.stdout: "evex": 0, 2024-05-03T18:47:52.645 INFO:teuthology.orchestra.run.smithi099.stdout: "evtrm": 0, 2024-05-03T18:47:52.645 INFO:teuthology.orchestra.run.smithi099.stdout: "ev": 21255, 2024-05-03T18:47:52.645 INFO:teuthology.orchestra.run.smithi099.stdout: "evexg": 0, 2024-05-03T18:47:52.645 INFO:teuthology.orchestra.run.smithi099.stdout: "evexd": 0, 2024-05-03T18:47:52.645 INFO:teuthology.orchestra.run.smithi099.stdout: "segadd": 0, 2024-05-03T18:47:52.645 INFO:teuthology.orchestra.run.smithi099.stdout: "segex": 0, 2024-05-03T18:47:52.645 INFO:teuthology.orchestra.run.smithi099.stdout: "segtrm": 0, 2024-05-03T18:47:52.646 INFO:teuthology.orchestra.run.smithi099.stdout: "seg": 56, 2024-05-03T18:47:52.646 INFO:teuthology.orchestra.run.smithi099.stdout: "segmjr": 3, 2024-05-03T18:47:52.646 INFO:teuthology.orchestra.run.smithi099.stdout: "segexg": 0, 2024-05-03T18:47:52.646 INFO:teuthology.orchestra.run.smithi099.stdout: "segexd": 0, 2024-05-03T18:47:52.646 INFO:teuthology.orchestra.run.smithi099.stdout: "expos": 4194304, 2024-05-03T18:47:52.646 INFO:teuthology.orchestra.run.smithi099.stdout: "wrpos": 216161932, 2024-05-03T18:47:52.646 INFO:teuthology.orchestra.run.smithi099.stdout: "rdpos": 216161796, 2024-05-03T18:47:52.646 INFO:teuthology.orchestra.run.smithi099.stdout: "jlat": { 2024-05-03T18:47:52.646 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-05-03T18:47:52.646 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-05-03T18:47:52.646 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-05-03T18:47:52.646 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:52.646 INFO:teuthology.orchestra.run.smithi099.stdout: "replayed": 21255 2024-05-03T18:47:52.646 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:52.647 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_mem": { 2024-05-03T18:47:52.647 INFO:teuthology.orchestra.run.smithi099.stdout: "ino": 2703, 2024-05-03T18:47:52.647 INFO:teuthology.orchestra.run.smithi099.stdout: "ino+": 2969, 2024-05-03T18:47:52.647 INFO:teuthology.orchestra.run.smithi099.stdout: "ino-": 266, 2024-05-03T18:47:52.647 INFO:teuthology.orchestra.run.smithi099.stdout: "dir": 132, 2024-05-03T18:47:52.647 INFO:teuthology.orchestra.run.smithi099.stdout: "dir+": 145, 2024-05-03T18:47:52.647 INFO:teuthology.orchestra.run.smithi099.stdout: "dir-": 13, 2024-05-03T18:47:52.647 INFO:teuthology.orchestra.run.smithi099.stdout: "dn": 3373, 2024-05-03T18:47:52.647 INFO:teuthology.orchestra.run.smithi099.stdout: "dn+": 3375, 2024-05-03T18:47:52.647 INFO:teuthology.orchestra.run.smithi099.stdout: "dn-": 2, 2024-05-03T18:47:52.647 INFO:teuthology.orchestra.run.smithi099.stdout: "cap": 0, 2024-05-03T18:47:52.647 INFO:teuthology.orchestra.run.smithi099.stdout: "cap+": 0, 2024-05-03T18:47:52.647 INFO:teuthology.orchestra.run.smithi099.stdout: "cap-": 0, 2024-05-03T18:47:52.647 INFO:teuthology.orchestra.run.smithi099.stdout: "rss": 64052, 2024-05-03T18:47:52.647 INFO:teuthology.orchestra.run.smithi099.stdout: "heap": 215416 2024-05-03T18:47:52.648 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:52.648 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_server": { 2024-05-03T18:47:52.648 INFO:teuthology.orchestra.run.smithi099.stdout: "dispatch_client_request": 0, 2024-05-03T18:47:52.648 INFO:teuthology.orchestra.run.smithi099.stdout: "dispatch_server_request": 0, 2024-05-03T18:47:52.648 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_request": 0, 2024-05-03T18:47:52.648 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_session": 0, 2024-05-03T18:47:52.648 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_peer_request": 0, 2024-05-03T18:47:52.648 INFO:teuthology.orchestra.run.smithi099.stdout: "req_create_latency": { 2024-05-03T18:47:52.648 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-05-03T18:47:52.648 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-05-03T18:47:52.648 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-05-03T18:47:52.648 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:52.648 INFO:teuthology.orchestra.run.smithi099.stdout: "req_getattr_latency": { 2024-05-03T18:47:52.649 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-05-03T18:47:52.649 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-05-03T18:47:52.649 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-05-03T18:47:52.649 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:52.649 INFO:teuthology.orchestra.run.smithi099.stdout: "req_getfilelock_latency": { 2024-05-03T18:47:52.649 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-05-03T18:47:52.649 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-05-03T18:47:52.649 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-05-03T18:47:52.649 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:52.649 INFO:teuthology.orchestra.run.smithi099.stdout: "req_link_latency": { 2024-05-03T18:47:52.649 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-05-03T18:47:52.649 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-05-03T18:47:52.649 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-05-03T18:47:52.649 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:52.649 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookup_latency": { 2024-05-03T18:47:52.650 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-05-03T18:47:52.650 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-05-03T18:47:52.650 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-05-03T18:47:52.650 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:52.650 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookuphash_latency": { 2024-05-03T18:47:52.650 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-05-03T18:47:52.650 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-05-03T18:47:52.650 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-05-03T18:47:52.650 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:52.650 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupino_latency": { 2024-05-03T18:47:52.650 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-05-03T18:47:52.650 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-05-03T18:47:52.650 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-05-03T18:47:52.650 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:52.651 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupname_latency": { 2024-05-03T18:47:52.651 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-05-03T18:47:52.651 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-05-03T18:47:52.651 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-05-03T18:47:52.651 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:52.651 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupparent_latency": { 2024-05-03T18:47:52.651 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-05-03T18:47:52.651 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-05-03T18:47:52.651 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-05-03T18:47:52.651 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:52.651 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupsnap_latency": { 2024-05-03T18:47:52.651 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-05-03T18:47:52.651 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-05-03T18:47:52.651 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-05-03T18:47:52.651 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:52.651 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lssnap_latency": { 2024-05-03T18:47:52.652 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-05-03T18:47:52.652 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-05-03T18:47:52.652 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-05-03T18:47:52.652 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:52.652 INFO:teuthology.orchestra.run.smithi099.stdout: "req_mkdir_latency": { 2024-05-03T18:47:52.652 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-05-03T18:47:52.653 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-05-03T18:47:52.653 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-05-03T18:47:52.653 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:52.653 INFO:teuthology.orchestra.run.smithi099.stdout: "req_mknod_latency": { 2024-05-03T18:47:52.653 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-05-03T18:47:52.653 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-05-03T18:47:52.653 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-05-03T18:47:52.653 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:52.653 INFO:teuthology.orchestra.run.smithi099.stdout: "req_mksnap_latency": { 2024-05-03T18:47:52.653 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-05-03T18:47:52.653 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-05-03T18:47:52.653 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-05-03T18:47:52.653 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:52.653 INFO:teuthology.orchestra.run.smithi099.stdout: "req_open_latency": { 2024-05-03T18:47:52.654 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-05-03T18:47:52.654 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-05-03T18:47:52.654 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-05-03T18:47:52.654 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:52.654 INFO:teuthology.orchestra.run.smithi099.stdout: "req_readdir_latency": { 2024-05-03T18:47:52.654 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-05-03T18:47:52.654 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-05-03T18:47:52.654 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-05-03T18:47:52.654 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:52.654 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rename_latency": { 2024-05-03T18:47:52.654 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-05-03T18:47:52.654 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-05-03T18:47:52.654 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-05-03T18:47:52.655 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:52.655 INFO:teuthology.orchestra.run.smithi099.stdout: "req_renamesnap_latency": { 2024-05-03T18:47:52.655 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-05-03T18:47:52.655 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-05-03T18:47:52.655 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-05-03T18:47:52.655 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:52.655 INFO:teuthology.orchestra.run.smithi099.stdout: "req_snapdiff_latency": { 2024-05-03T18:47:52.655 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-05-03T18:47:52.655 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-05-03T18:47:52.655 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-05-03T18:47:52.655 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:52.655 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rmdir_latency": { 2024-05-03T18:47:52.655 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-05-03T18:47:52.655 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-05-03T18:47:52.655 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-05-03T18:47:52.656 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:52.656 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rmsnap_latency": { 2024-05-03T18:47:52.656 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-05-03T18:47:52.656 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-05-03T18:47:52.656 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-05-03T18:47:52.656 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:52.656 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rmxattr_latency": { 2024-05-03T18:47:52.656 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-05-03T18:47:52.656 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-05-03T18:47:52.656 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-05-03T18:47:52.656 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:52.656 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setattr_latency": { 2024-05-03T18:47:52.656 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-05-03T18:47:52.656 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-05-03T18:47:52.657 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-05-03T18:47:52.657 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:52.657 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setdirlayout_latency": { 2024-05-03T18:47:52.657 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-05-03T18:47:52.657 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-05-03T18:47:52.657 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-05-03T18:47:52.657 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:52.657 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setfilelock_latency": { 2024-05-03T18:47:52.657 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-05-03T18:47:52.657 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-05-03T18:47:52.657 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-05-03T18:47:52.657 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:52.657 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setlayout_latency": { 2024-05-03T18:47:52.658 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-05-03T18:47:52.658 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-05-03T18:47:52.658 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-05-03T18:47:52.658 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:52.658 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setxattr_latency": { 2024-05-03T18:47:52.658 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-05-03T18:47:52.658 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-05-03T18:47:52.658 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-05-03T18:47:52.658 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:52.658 INFO:teuthology.orchestra.run.smithi099.stdout: "req_symlink_latency": { 2024-05-03T18:47:52.658 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-05-03T18:47:52.658 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-05-03T18:47:52.658 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-05-03T18:47:52.659 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:52.659 INFO:teuthology.orchestra.run.smithi099.stdout: "req_unlink_latency": { 2024-05-03T18:47:52.659 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-05-03T18:47:52.659 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-05-03T18:47:52.659 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-05-03T18:47:52.659 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:52.659 INFO:teuthology.orchestra.run.smithi099.stdout: "cap_revoke_eviction": 0, 2024-05-03T18:47:52.659 INFO:teuthology.orchestra.run.smithi099.stdout: "cap_acquisition_throttle": 0, 2024-05-03T18:47:52.659 INFO:teuthology.orchestra.run.smithi099.stdout: "req_getvxattr_latency": { 2024-05-03T18:47:52.659 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-05-03T18:47:52.659 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-05-03T18:47:52.659 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-05-03T18:47:52.659 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-03T18:47:52.659 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:52.660 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_sessions": { 2024-05-03T18:47:52.660 INFO:teuthology.orchestra.run.smithi099.stdout: "session_count": 2, 2024-05-03T18:47:52.660 INFO:teuthology.orchestra.run.smithi099.stdout: "session_add": 6, 2024-05-03T18:47:52.660 INFO:teuthology.orchestra.run.smithi099.stdout: "session_remove": 4, 2024-05-03T18:47:52.660 INFO:teuthology.orchestra.run.smithi099.stdout: "sessions_open": 4, 2024-05-03T18:47:52.660 INFO:teuthology.orchestra.run.smithi099.stdout: "sessions_stale": 0, 2024-05-03T18:47:52.660 INFO:teuthology.orchestra.run.smithi099.stdout: "total_load": 0, 2024-05-03T18:47:52.660 INFO:teuthology.orchestra.run.smithi099.stdout: "average_load": 0, 2024-05-03T18:47:52.660 INFO:teuthology.orchestra.run.smithi099.stdout: "avg_session_uptime": 7088, 2024-05-03T18:47:52.660 INFO:teuthology.orchestra.run.smithi099.stdout: "mdthresh_evicted": 0 2024-05-03T18:47:52.660 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:52.660 INFO:teuthology.orchestra.run.smithi099.stdout: "mempool": { 2024-05-03T18:47:52.660 INFO:teuthology.orchestra.run.smithi099.stdout: "bloom_filter_bytes": 0, 2024-05-03T18:47:52.660 INFO:teuthology.orchestra.run.smithi099.stdout: "bloom_filter_items": 0, 2024-05-03T18:47:52.661 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_alloc_bytes": 0, 2024-05-03T18:47:52.661 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_alloc_items": 0, 2024-05-03T18:47:52.661 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_data_bytes": 0, 2024-05-03T18:47:52.661 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_data_items": 0, 2024-05-03T18:47:52.661 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-03T18:47:52.661 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_onode_items": 0, 2024-05-03T18:47:52.661 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-03T18:47:52.661 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_meta_items": 0, 2024-05-03T18:47:52.661 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_other_bytes": 0, 2024-05-03T18:47:52.661 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_other_items": 0, 2024-05-03T18:47:52.661 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-03T18:47:52.661 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_buffer_items": 0, 2024-05-03T18:47:52.661 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_extent_bytes": 0, 2024-05-03T18:47:52.661 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_extent_items": 0, 2024-05-03T18:47:52.662 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_blob_bytes": 0, 2024-05-03T18:47:52.662 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_blob_items": 0, 2024-05-03T18:47:52.662 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-03T18:47:52.662 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_shared_blob_items": 0, 2024-05-03T18:47:52.662 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-03T18:47:52.662 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_inline_bl_items": 0, 2024-05-03T18:47:52.662 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_fsck_bytes": 0, 2024-05-03T18:47:52.662 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_fsck_items": 0, 2024-05-03T18:47:52.662 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_txc_bytes": 0, 2024-05-03T18:47:52.662 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_txc_items": 0, 2024-05-03T18:47:52.662 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-03T18:47:52.662 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_deferred_items": 0, 2024-05-03T18:47:52.662 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_bytes": 0, 2024-05-03T18:47:52.663 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_items": 0, 2024-05-03T18:47:52.663 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_bytes": 0, 2024-05-03T18:47:52.663 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_items": 0, 2024-05-03T18:47:52.663 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_reader_bytes": 0, 2024-05-03T18:47:52.663 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_reader_items": 0, 2024-05-03T18:47:52.663 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_writer_bytes": 0, 2024-05-03T18:47:52.663 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_writer_items": 0, 2024-05-03T18:47:52.663 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_anon_bytes": 241817, 2024-05-03T18:47:52.663 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_anon_items": 85, 2024-05-03T18:47:52.663 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_meta_bytes": 0, 2024-05-03T18:47:52.663 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_meta_items": 0, 2024-05-03T18:47:52.663 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_bytes": 0, 2024-05-03T18:47:52.663 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_items": 0, 2024-05-03T18:47:52.663 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_mapbl_bytes": 0, 2024-05-03T18:47:52.663 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_mapbl_items": 0, 2024-05-03T18:47:52.664 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_pglog_bytes": 0, 2024-05-03T18:47:52.664 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_pglog_items": 0, 2024-05-03T18:47:52.664 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_bytes": 9648, 2024-05-03T18:47:52.664 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_items": 202, 2024-05-03T18:47:52.664 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_mapping_bytes": 0, 2024-05-03T18:47:52.664 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_mapping_items": 0, 2024-05-03T18:47:52.664 INFO:teuthology.orchestra.run.smithi099.stdout: "pgmap_bytes": 0, 2024-05-03T18:47:52.664 INFO:teuthology.orchestra.run.smithi099.stdout: "pgmap_items": 0, 2024-05-03T18:47:52.664 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_co_bytes": 8699317, 2024-05-03T18:47:52.664 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_co_items": 59328, 2024-05-03T18:47:52.664 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_1_bytes": 0, 2024-05-03T18:47:52.664 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_1_items": 0, 2024-05-03T18:47:52.664 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_2_bytes": 0, 2024-05-03T18:47:52.664 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_2_items": 0 2024-05-03T18:47:52.664 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:52.665 INFO:teuthology.orchestra.run.smithi099.stdout: "objecter": { 2024-05-03T18:47:52.665 INFO:teuthology.orchestra.run.smithi099.stdout: "op_active": 0, 2024-05-03T18:47:52.665 INFO:teuthology.orchestra.run.smithi099.stdout: "op_laggy": 0, 2024-05-03T18:47:52.665 INFO:teuthology.orchestra.run.smithi099.stdout: "op_send": 5023, 2024-05-03T18:47:52.665 INFO:teuthology.orchestra.run.smithi099.stdout: "op_send_bytes": 26, 2024-05-03T18:47:52.665 INFO:teuthology.orchestra.run.smithi099.stdout: "op_resend": 4, 2024-05-03T18:47:52.665 INFO:teuthology.orchestra.run.smithi099.stdout: "op_reply": 5023, 2024-05-03T18:47:52.665 INFO:teuthology.orchestra.run.smithi099.stdout: "op_latency": { 2024-05-03T18:47:52.665 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 5023, 2024-05-03T18:47:52.665 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 8.782796866, 2024-05-03T18:47:52.665 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.001748516 2024-05-03T18:47:52.665 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:52.665 INFO:teuthology.orchestra.run.smithi099.stdout: "op_inflight": 0, 2024-05-03T18:47:52.665 INFO:teuthology.orchestra.run.smithi099.stdout: "oplen_avg": { 2024-05-03T18:47:52.666 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 5023, 2024-05-03T18:47:52.666 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 5024 2024-05-03T18:47:52.666 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:52.666 INFO:teuthology.orchestra.run.smithi099.stdout: "op": 5023, 2024-05-03T18:47:52.666 INFO:teuthology.orchestra.run.smithi099.stdout: "op_r": 5023, 2024-05-03T18:47:52.666 INFO:teuthology.orchestra.run.smithi099.stdout: "op_w": 0, 2024-05-03T18:47:52.666 INFO:teuthology.orchestra.run.smithi099.stdout: "op_rmw": 0, 2024-05-03T18:47:52.666 INFO:teuthology.orchestra.run.smithi099.stdout: "op_pg": 0, 2024-05-03T18:47:52.666 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_stat": 2931, 2024-05-03T18:47:52.666 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_create": 0, 2024-05-03T18:47:52.666 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_read": 2089, 2024-05-03T18:47:52.666 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_write": 0, 2024-05-03T18:47:52.666 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_writefull": 0, 2024-05-03T18:47:52.667 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_writesame": 0, 2024-05-03T18:47:52.667 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_append": 0, 2024-05-03T18:47:52.667 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_zero": 0, 2024-05-03T18:47:52.667 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_truncate": 0, 2024-05-03T18:47:52.667 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_delete": 0, 2024-05-03T18:47:52.667 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_mapext": 0, 2024-05-03T18:47:52.667 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_sparse_read": 0, 2024-05-03T18:47:52.667 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_clonerange": 0, 2024-05-03T18:47:52.667 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_getxattr": 2, 2024-05-03T18:47:52.670 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_setxattr": 0, 2024-05-03T18:47:52.671 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_cmpxattr": 0, 2024-05-03T18:47:52.671 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_rmxattr": 0, 2024-05-03T18:47:52.671 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_resetxattrs": 0, 2024-05-03T18:47:52.671 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_call": 0, 2024-05-03T18:47:52.671 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_watch": 0, 2024-05-03T18:47:52.671 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_notify": 0, 2024-05-03T18:47:52.671 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_src_cmpxattr": 0, 2024-05-03T18:47:52.671 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_pgls": 0, 2024-05-03T18:47:52.671 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_pgls_filter": 0, 2024-05-03T18:47:52.671 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_other": 0, 2024-05-03T18:47:52.671 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_active": 0, 2024-05-03T18:47:52.671 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_send": 0, 2024-05-03T18:47:52.671 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_resend": 0, 2024-05-03T18:47:52.672 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_ping": 0, 2024-05-03T18:47:52.672 INFO:teuthology.orchestra.run.smithi099.stdout: "poolop_active": 0, 2024-05-03T18:47:52.672 INFO:teuthology.orchestra.run.smithi099.stdout: "poolop_send": 0, 2024-05-03T18:47:52.672 INFO:teuthology.orchestra.run.smithi099.stdout: "poolop_resend": 0, 2024-05-03T18:47:52.672 INFO:teuthology.orchestra.run.smithi099.stdout: "poolstat_active": 0, 2024-05-03T18:47:52.672 INFO:teuthology.orchestra.run.smithi099.stdout: "poolstat_send": 0, 2024-05-03T18:47:52.672 INFO:teuthology.orchestra.run.smithi099.stdout: "poolstat_resend": 0, 2024-05-03T18:47:52.672 INFO:teuthology.orchestra.run.smithi099.stdout: "statfs_active": 0, 2024-05-03T18:47:52.672 INFO:teuthology.orchestra.run.smithi099.stdout: "statfs_send": 0, 2024-05-03T18:47:52.672 INFO:teuthology.orchestra.run.smithi099.stdout: "statfs_resend": 0, 2024-05-03T18:47:52.672 INFO:teuthology.orchestra.run.smithi099.stdout: "command_active": 0, 2024-05-03T18:47:52.672 INFO:teuthology.orchestra.run.smithi099.stdout: "command_send": 0, 2024-05-03T18:47:52.672 INFO:teuthology.orchestra.run.smithi099.stdout: "command_resend": 0, 2024-05-03T18:47:52.672 INFO:teuthology.orchestra.run.smithi099.stdout: "map_epoch": 120, 2024-05-03T18:47:52.673 INFO:teuthology.orchestra.run.smithi099.stdout: "map_full": 0, 2024-05-03T18:47:52.673 INFO:teuthology.orchestra.run.smithi099.stdout: "map_inc": 33, 2024-05-03T18:47:52.673 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_sessions": 11, 2024-05-03T18:47:52.673 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_session_open": 11, 2024-05-03T18:47:52.673 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_session_close": 0, 2024-05-03T18:47:52.673 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_laggy": 0, 2024-05-03T18:47:52.673 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_wr": 0, 2024-05-03T18:47:52.673 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_rd": 2, 2024-05-03T18:47:52.673 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_del": 0 2024-05-03T18:47:52.673 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:52.673 INFO:teuthology.orchestra.run.smithi099.stdout: "oft": { 2024-05-03T18:47:52.673 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_objs": 0, 2024-05-03T18:47:52.673 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_kv_pairs": 0, 2024-05-03T18:47:52.673 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_updates": 0, 2024-05-03T18:47:52.673 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_removes": 0 2024-05-03T18:47:52.674 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:52.674 INFO:teuthology.orchestra.run.smithi099.stdout: "osd-slow-ops": { 2024-05-03T18:47:52.674 INFO:teuthology.orchestra.run.smithi099.stdout: "slow_ops_count": 0 2024-05-03T18:47:52.674 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:52.674 INFO:teuthology.orchestra.run.smithi099.stdout: "purge_queue": { 2024-05-03T18:47:52.674 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing_ops": 0, 2024-05-03T18:47:52.674 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing_ops_high_water": 0, 2024-05-03T18:47:52.674 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing": 0, 2024-05-03T18:47:52.674 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing_high_water": 0, 2024-05-03T18:47:52.674 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executed_ops": 0, 2024-05-03T18:47:52.674 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executed": 0, 2024-05-03T18:47:52.674 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_item_in_journal": 0 2024-05-03T18:47:52.674 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:52.675 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-03T18:47:52.675 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-05-03T18:47:52.675 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 104857600, 2024-05-03T18:47:52.675 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-05-03T18:47:52.675 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 5491, 2024-05-03T18:47:52.675 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 215091045, 2024-05-03T18:47:52.675 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-05-03T18:47:52.675 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 5491, 2024-05-03T18:47:52.675 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 0, 2024-05-03T18:47:52.675 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 0, 2024-05-03T18:47:52.675 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 5491, 2024-05-03T18:47:52.675 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 215091045, 2024-05-03T18:47:52.675 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-05-03T18:47:52.675 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-05-03T18:47:52.675 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-05-03T18:47:52.675 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-05-03T18:47:52.676 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-03T18:47:52.676 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:52.676 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-objecter_bytes": { 2024-05-03T18:47:52.676 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-05-03T18:47:52.676 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 104857600, 2024-05-03T18:47:52.676 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-05-03T18:47:52.676 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 0, 2024-05-03T18:47:52.676 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 0, 2024-05-03T18:47:52.676 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-05-03T18:47:52.676 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 0, 2024-05-03T18:47:52.676 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 5023, 2024-05-03T18:47:52.676 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 212550497, 2024-05-03T18:47:52.676 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 5023, 2024-05-03T18:47:52.676 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 212550497, 2024-05-03T18:47:52.676 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-05-03T18:47:52.677 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-05-03T18:47:52.677 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-05-03T18:47:52.677 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-05-03T18:47:52.677 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-03T18:47:52.677 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:52.677 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-objecter_ops": { 2024-05-03T18:47:52.677 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-05-03T18:47:52.677 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 1024, 2024-05-03T18:47:52.677 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-05-03T18:47:52.677 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 0, 2024-05-03T18:47:52.677 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 0, 2024-05-03T18:47:52.677 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-05-03T18:47:52.677 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 0, 2024-05-03T18:47:52.677 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 5023, 2024-05-03T18:47:52.677 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 5023, 2024-05-03T18:47:52.678 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 5023, 2024-05-03T18:47:52.678 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 5023, 2024-05-03T18:47:52.678 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-05-03T18:47:52.678 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-05-03T18:47:52.678 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-05-03T18:47:52.678 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-05-03T18:47:52.678 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-03T18:47:52.678 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:52.678 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-write_buf_throttle": { 2024-05-03T18:47:52.678 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-05-03T18:47:52.678 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 3758096384, 2024-05-03T18:47:52.678 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-05-03T18:47:52.678 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 0, 2024-05-03T18:47:52.679 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 0, 2024-05-03T18:47:52.679 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-05-03T18:47:52.679 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 0, 2024-05-03T18:47:52.679 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 0, 2024-05-03T18:47:52.679 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 0, 2024-05-03T18:47:52.679 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 0, 2024-05-03T18:47:52.679 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 0, 2024-05-03T18:47:52.679 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-05-03T18:47:52.679 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-05-03T18:47:52.679 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-05-03T18:47:52.679 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-05-03T18:47:52.679 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-03T18:47:52.679 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:52.679 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-write_buf_throttle-0x5561fdf06780": { 2024-05-03T18:47:52.680 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-05-03T18:47:52.680 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 3758096384, 2024-05-03T18:47:52.680 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-05-03T18:47:52.680 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 0, 2024-05-03T18:47:52.680 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 0, 2024-05-03T18:47:52.680 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-05-03T18:47:52.680 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 0, 2024-05-03T18:47:52.680 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 0, 2024-05-03T18:47:52.680 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 0, 2024-05-03T18:47:52.680 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 0, 2024-05-03T18:47:52.680 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 0, 2024-05-03T18:47:52.680 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-05-03T18:47:52.680 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-05-03T18:47:52.680 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-05-03T18:47:52.681 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-05-03T18:47:52.681 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-03T18:47:52.681 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-03T18:47:52.681 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-05-03T18:47:53.726 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-05-03T18:47:53.726 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-05-03T18:47:53.726 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-05-03T18:47:53.726 DEBUG:tasks.check_counter:Getting stats from k 2024-05-03T18:47:53.726 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph daemon mds.k perf dump 2024-05-03T18:47:54.196 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.b/config 2024-05-03T18:47:54.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:47:54 smithi150 ceph-mon[32492]: pgmap v653: 129 pgs: 129 active+clean; 3.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:47:54.429 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:47:54 smithi099 ceph-mon[31658]: pgmap v653: 129 pgs: 129 active+clean; 3.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:47:54.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:47:54 smithi033 ceph-mon[28330]: pgmap v653: 129 pgs: 129 active+clean; 3.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:47:56.202 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:47:56 smithi099 ceph-mon[31658]: pgmap v654: 129 pgs: 129 active+clean; 3.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:47:56.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:47:56 smithi033 ceph-mon[28330]: pgmap v654: 129 pgs: 129 active+clean; 3.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:47:56.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:47:56 smithi150 ceph-mon[32492]: pgmap v654: 129 pgs: 129 active+clean; 3.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:47:56.648 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-05-03T18:47:56.649 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-0": { 2024-05-03T18:47:56.649 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 433, 2024-05-03T18:47:56.649 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 423, 2024-05-03T18:47:56.649 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 1621616, 2024-05-03T18:47:56.649 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 261696, 2024-05-03T18:47:56.649 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 4, 2024-05-03T18:47:56.649 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 0, 2024-05-03T18:47:56.649 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.169959502, 2024-05-03T18:47:56.649 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.059077143, 2024-05-03T18:47:56.650 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.115253685, 2024-05-03T18:47:56.650 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.019360987, 2024-05-03T18:47:56.650 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2024-05-03T18:47:56.650 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 422, 2024-05-03T18:47:56.650 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.048866614, 2024-05-03T18:47:56.650 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000115797 2024-05-03T18:47:56.650 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:56.650 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2024-05-03T18:47:56.650 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-05-03T18:47:56.650 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-05-03T18:47:56.650 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-05-03T18:47:56.650 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:56.651 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 1621616, 2024-05-03T18:47:56.651 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 261696 2024-05-03T18:47:56.651 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:56.651 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-1": { 2024-05-03T18:47:56.651 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 6, 2024-05-03T18:47:56.651 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 325, 2024-05-03T18:47:56.651 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 3232, 2024-05-03T18:47:56.651 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 181728, 2024-05-03T18:47:56.651 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 5, 2024-05-03T18:47:56.651 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 3, 2024-05-03T18:47:56.651 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.041294782, 2024-05-03T18:47:56.651 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.027738148, 2024-05-03T18:47:56.651 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.005093840, 2024-05-03T18:47:56.651 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-03T18:47:56.652 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2024-05-03T18:47:56.652 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 323, 2024-05-03T18:47:56.652 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.046816529, 2024-05-03T18:47:56.652 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000144942 2024-05-03T18:47:56.652 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:56.652 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2024-05-03T18:47:56.652 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-05-03T18:47:56.652 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-05-03T18:47:56.652 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-05-03T18:47:56.652 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:56.652 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 3232, 2024-05-03T18:47:56.652 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 181728 2024-05-03T18:47:56.652 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:56.652 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-2": { 2024-05-03T18:47:56.652 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 0, 2024-05-03T18:47:56.653 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 2, 2024-05-03T18:47:56.653 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 0, 2024-05-03T18:47:56.653 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 0, 2024-05-03T18:47:56.653 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 2, 2024-05-03T18:47:56.653 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 1, 2024-05-03T18:47:56.653 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.000698232, 2024-05-03T18:47:56.653 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.000000000, 2024-05-03T18:47:56.653 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.000508228, 2024-05-03T18:47:56.653 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-03T18:47:56.653 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2024-05-03T18:47:56.653 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-05-03T18:47:56.653 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-05-03T18:47:56.653 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-05-03T18:47:56.653 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:56.654 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2024-05-03T18:47:56.654 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-05-03T18:47:56.654 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-05-03T18:47:56.654 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-05-03T18:47:56.654 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:56.654 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-03T18:47:56.654 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 0 2024-05-03T18:47:56.654 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:56.654 INFO:teuthology.orchestra.run.smithi099.stdout: "cct": { 2024-05-03T18:47:56.654 INFO:teuthology.orchestra.run.smithi099.stdout: "total_workers": 0, 2024-05-03T18:47:56.654 INFO:teuthology.orchestra.run.smithi099.stdout: "unhealthy_workers": 0 2024-05-03T18:47:56.654 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:56.654 INFO:teuthology.orchestra.run.smithi099.stdout: "mempool": { 2024-05-03T18:47:56.654 INFO:teuthology.orchestra.run.smithi099.stdout: "bloom_filter_bytes": 0, 2024-05-03T18:47:56.655 INFO:teuthology.orchestra.run.smithi099.stdout: "bloom_filter_items": 0, 2024-05-03T18:47:56.655 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_alloc_bytes": 0, 2024-05-03T18:47:56.655 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_alloc_items": 0, 2024-05-03T18:47:56.655 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_data_bytes": 0, 2024-05-03T18:47:56.655 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_data_items": 0, 2024-05-03T18:47:56.655 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-03T18:47:56.655 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_onode_items": 0, 2024-05-03T18:47:56.655 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-03T18:47:56.655 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_meta_items": 0, 2024-05-03T18:47:56.655 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_other_bytes": 0, 2024-05-03T18:47:56.655 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_other_items": 0, 2024-05-03T18:47:56.655 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-03T18:47:56.655 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_buffer_items": 0, 2024-05-03T18:47:56.655 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_extent_bytes": 0, 2024-05-03T18:47:56.655 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_extent_items": 0, 2024-05-03T18:47:56.656 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_blob_bytes": 0, 2024-05-03T18:47:56.656 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_blob_items": 0, 2024-05-03T18:47:56.656 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-03T18:47:56.656 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_shared_blob_items": 0, 2024-05-03T18:47:56.656 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-03T18:47:56.656 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_inline_bl_items": 0, 2024-05-03T18:47:56.656 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_fsck_bytes": 0, 2024-05-03T18:47:56.656 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_fsck_items": 0, 2024-05-03T18:47:56.656 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_txc_bytes": 0, 2024-05-03T18:47:56.656 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_txc_items": 0, 2024-05-03T18:47:56.656 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-03T18:47:56.657 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_deferred_items": 0, 2024-05-03T18:47:56.657 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_bytes": 0, 2024-05-03T18:47:56.657 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_items": 0, 2024-05-03T18:47:56.657 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_bytes": 0, 2024-05-03T18:47:56.657 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_items": 0, 2024-05-03T18:47:56.657 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_reader_bytes": 0, 2024-05-03T18:47:56.657 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_reader_items": 0, 2024-05-03T18:47:56.657 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_writer_bytes": 0, 2024-05-03T18:47:56.657 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_writer_items": 0, 2024-05-03T18:47:56.657 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_anon_bytes": 148704, 2024-05-03T18:47:56.657 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_anon_items": 29, 2024-05-03T18:47:56.657 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_meta_bytes": 0, 2024-05-03T18:47:56.657 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_meta_items": 0, 2024-05-03T18:47:56.657 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_bytes": 0, 2024-05-03T18:47:56.657 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_items": 0, 2024-05-03T18:47:56.658 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_mapbl_bytes": 0, 2024-05-03T18:47:56.658 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_mapbl_items": 0, 2024-05-03T18:47:56.658 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_pglog_bytes": 0, 2024-05-03T18:47:56.658 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_pglog_items": 0, 2024-05-03T18:47:56.658 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_bytes": 0, 2024-05-03T18:47:56.658 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_items": 0, 2024-05-03T18:47:56.658 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_mapping_bytes": 0, 2024-05-03T18:47:56.658 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_mapping_items": 0, 2024-05-03T18:47:56.658 INFO:teuthology.orchestra.run.smithi099.stdout: "pgmap_bytes": 0, 2024-05-03T18:47:56.658 INFO:teuthology.orchestra.run.smithi099.stdout: "pgmap_items": 0, 2024-05-03T18:47:56.658 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_co_bytes": 928, 2024-05-03T18:47:56.658 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_co_items": 2, 2024-05-03T18:47:56.658 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_1_bytes": 0, 2024-05-03T18:47:56.658 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_1_items": 0, 2024-05-03T18:47:56.659 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_2_bytes": 0, 2024-05-03T18:47:56.659 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_2_items": 0 2024-05-03T18:47:56.659 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-05-03T18:47:56.659 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-03T18:47:56.659 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-05-03T18:47:56.659 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 104857600, 2024-05-03T18:47:56.659 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-05-03T18:47:56.659 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 435, 2024-05-03T18:47:56.659 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 1560408, 2024-05-03T18:47:56.659 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-05-03T18:47:56.659 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 435, 2024-05-03T18:47:56.659 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 0, 2024-05-03T18:47:56.659 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 0, 2024-05-03T18:47:56.659 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 435, 2024-05-03T18:47:56.660 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 1560408, 2024-05-03T18:47:56.660 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-05-03T18:47:56.660 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-05-03T18:47:56.660 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-05-03T18:47:56.660 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-05-03T18:47:56.660 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-03T18:47:56.660 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-05-03T18:47:56.660 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-05-03T18:47:57.684 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2024-05-03T18:47:57.684 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2024-05-03T18:47:57.684 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2024-05-03T18:47:57.684 DEBUG:tasks.check_counter:Getting stats from c 2024-05-03T18:47:57.684 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph daemon mds.c perf dump 2024-05-03T18:47:58.292 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.c/config 2024-05-03T18:47:58.443 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:47:58 smithi150 ceph-mon[32492]: pgmap v655: 129 pgs: 129 active+clean; 3.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:47:58.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:47:58 smithi033 ceph-mon[28330]: pgmap v655: 129 pgs: 129 active+clean; 3.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:47:58.547 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:47:58 smithi099 ceph-mon[31658]: pgmap v655: 129 pgs: 129 active+clean; 3.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:48:00.428 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:48:00 smithi150 ceph-mon[32492]: pgmap v656: 129 pgs: 129 active+clean; 3.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:48:00.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:48:00 smithi033 ceph-mon[28330]: pgmap v656: 129 pgs: 129 active+clean; 3.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:48:00.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:48:00 smithi099 ceph-mon[31658]: pgmap v656: 129 pgs: 129 active+clean; 3.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-03T18:48:01.222 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-05-03T18:48:01.222 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-0": { 2024-05-03T18:48:01.222 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 4, 2024-05-03T18:48:01.222 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 4, 2024-05-03T18:48:01.222 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 2912, 2024-05-03T18:48:01.222 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 448, 2024-05-03T18:48:01.222 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 4, 2024-05-03T18:48:01.222 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 0, 2024-05-03T18:48:01.222 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.004344994, 2024-05-03T18:48:01.222 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.000080587, 2024-05-03T18:48:01.223 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.001283524, 2024-05-03T18:48:01.223 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-03T18:48:01.223 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-05-03T18:48:01.223 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 3, 2024-05-03T18:48:01.223 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.001672292, 2024-05-03T18:48:01.223 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000557430 2024-05-03T18:48:01.223 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-03T18:48:01.223 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-05-03T18:48:01.223 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-03T18:48:01.223 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-03T18:48:01.223 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-03T18:48:01.223 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-03T18:48:01.223 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 2912, 2024-05-03T18:48:01.223 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 448 2024-05-03T18:48:01.223 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-03T18:48:01.224 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-1": { 2024-05-03T18:48:01.224 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 429, 2024-05-03T18:48:01.224 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 419, 2024-05-03T18:48:01.224 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 1620528, 2024-05-03T18:48:01.224 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 259136, 2024-05-03T18:48:01.224 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 2, 2024-05-03T18:48:01.224 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 1, 2024-05-03T18:48:01.224 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.155238904, 2024-05-03T18:48:01.224 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.056706947, 2024-05-03T18:48:01.224 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.104111604, 2024-05-03T18:48:01.224 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.017964646, 2024-05-03T18:48:01.224 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-05-03T18:48:01.224 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 418, 2024-05-03T18:48:01.224 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.057343010, 2024-05-03T18:48:01.224 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000137184 2024-05-03T18:48:01.224 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-03T18:48:01.224 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-05-03T18:48:01.225 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-03T18:48:01.225 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-03T18:48:01.225 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-03T18:48:01.225 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-03T18:48:01.225 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 1620528, 2024-05-03T18:48:01.225 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 259136 2024-05-03T18:48:01.225 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-03T18:48:01.225 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-2": { 2024-05-03T18:48:01.225 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 3, 2024-05-03T18:48:01.225 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 320, 2024-05-03T18:48:01.225 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 480, 2024-05-03T18:48:01.225 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 226912, 2024-05-03T18:48:01.225 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 6, 2024-05-03T18:48:01.225 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 3, 2024-05-03T18:48:01.225 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.036980846, 2024-05-03T18:48:01.225 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.024394437, 2024-05-03T18:48:01.226 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.005298725, 2024-05-03T18:48:01.226 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-03T18:48:01.226 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-05-03T18:48:01.226 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 317, 2024-05-03T18:48:01.226 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.047289895, 2024-05-03T18:48:01.226 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000149179 2024-05-03T18:48:01.226 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-03T18:48:01.226 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-05-03T18:48:01.226 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-03T18:48:01.226 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-03T18:48:01.226 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-03T18:48:01.226 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-03T18:48:01.226 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 480, 2024-05-03T18:48:01.226 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 226912 2024-05-03T18:48:01.227 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-03T18:48:01.227 INFO:teuthology.orchestra.run.smithi150.stdout: "cct": { 2024-05-03T18:48:01.227 INFO:teuthology.orchestra.run.smithi150.stdout: "total_workers": 0, 2024-05-03T18:48:01.227 INFO:teuthology.orchestra.run.smithi150.stdout: "unhealthy_workers": 0 2024-05-03T18:48:01.227 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-03T18:48:01.227 INFO:teuthology.orchestra.run.smithi150.stdout: "mempool": { 2024-05-03T18:48:01.227 INFO:teuthology.orchestra.run.smithi150.stdout: "bloom_filter_bytes": 0, 2024-05-03T18:48:01.227 INFO:teuthology.orchestra.run.smithi150.stdout: "bloom_filter_items": 0, 2024-05-03T18:48:01.227 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_alloc_bytes": 0, 2024-05-03T18:48:01.227 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_alloc_items": 0, 2024-05-03T18:48:01.227 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_data_bytes": 0, 2024-05-03T18:48:01.227 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_data_items": 0, 2024-05-03T18:48:01.227 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-03T18:48:01.227 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_onode_items": 0, 2024-05-03T18:48:01.227 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-03T18:48:01.228 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_meta_items": 0, 2024-05-03T18:48:01.228 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_other_bytes": 0, 2024-05-03T18:48:01.228 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_other_items": 0, 2024-05-03T18:48:01.228 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-03T18:48:01.228 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_buffer_items": 0, 2024-05-03T18:48:01.228 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_extent_bytes": 0, 2024-05-03T18:48:01.228 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_extent_items": 0, 2024-05-03T18:48:01.228 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_blob_bytes": 0, 2024-05-03T18:48:01.228 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_blob_items": 0, 2024-05-03T18:48:01.228 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-03T18:48:01.228 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_shared_blob_items": 0, 2024-05-03T18:48:01.228 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-03T18:48:01.229 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_inline_bl_items": 0, 2024-05-03T18:48:01.229 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_bytes": 0, 2024-05-03T18:48:01.229 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_items": 0, 2024-05-03T18:48:01.229 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_txc_bytes": 0, 2024-05-03T18:48:01.229 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_txc_items": 0, 2024-05-03T18:48:01.229 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-03T18:48:01.229 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_deferred_items": 0, 2024-05-03T18:48:01.229 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_bytes": 0, 2024-05-03T18:48:01.229 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_items": 0, 2024-05-03T18:48:01.229 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_bytes": 0, 2024-05-03T18:48:01.229 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_items": 0, 2024-05-03T18:48:01.229 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_reader_bytes": 0, 2024-05-03T18:48:01.229 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_reader_items": 0, 2024-05-03T18:48:01.229 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_writer_bytes": 0, 2024-05-03T18:48:01.230 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_writer_items": 0, 2024-05-03T18:48:01.230 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_anon_bytes": 148731, 2024-05-03T18:48:01.230 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_anon_items": 31, 2024-05-03T18:48:01.230 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_meta_bytes": 0, 2024-05-03T18:48:01.230 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_meta_items": 0, 2024-05-03T18:48:01.230 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_bytes": 0, 2024-05-03T18:48:01.230 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_items": 0, 2024-05-03T18:48:01.230 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mapbl_bytes": 0, 2024-05-03T18:48:01.230 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mapbl_items": 0, 2024-05-03T18:48:01.230 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pglog_bytes": 0, 2024-05-03T18:48:01.230 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pglog_items": 0, 2024-05-03T18:48:01.230 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_bytes": 0, 2024-05-03T18:48:01.230 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_items": 0, 2024-05-03T18:48:01.230 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_mapping_bytes": 0, 2024-05-03T18:48:01.231 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_mapping_items": 0, 2024-05-03T18:48:01.231 INFO:teuthology.orchestra.run.smithi150.stdout: "pgmap_bytes": 0, 2024-05-03T18:48:01.231 INFO:teuthology.orchestra.run.smithi150.stdout: "pgmap_items": 0, 2024-05-03T18:48:01.231 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_co_bytes": 928, 2024-05-03T18:48:01.231 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_co_items": 2, 2024-05-03T18:48:01.231 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_1_bytes": 0, 2024-05-03T18:48:01.231 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_1_items": 0, 2024-05-03T18:48:01.231 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_2_bytes": 0, 2024-05-03T18:48:01.231 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_2_items": 0 2024-05-03T18:48:01.231 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-03T18:48:01.231 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-03T18:48:01.231 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-05-03T18:48:01.231 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 104857600, 2024-05-03T18:48:01.232 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-05-03T18:48:01.232 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 432, 2024-05-03T18:48:01.232 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 1559915, 2024-05-03T18:48:01.232 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-05-03T18:48:01.232 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 432, 2024-05-03T18:48:01.232 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 0, 2024-05-03T18:48:01.232 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 0, 2024-05-03T18:48:01.232 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 432, 2024-05-03T18:48:01.232 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 1559915, 2024-05-03T18:48:01.232 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-05-03T18:48:01.232 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-03T18:48:01.232 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-03T18:48:01.232 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-03T18:48:01.232 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-03T18:48:01.232 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-03T18:48:01.233 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-05-03T18:48:02.211 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.c 2024-05-03T18:48:02.211 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.c 2024-05-03T18:48:02.211 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.c 2024-05-03T18:48:02.211 DEBUG:tasks.check_counter:Getting stats from f 2024-05-03T18:48:02.212 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph daemon mds.f perf dump 2024-05-03T18:48:02.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:48:02 smithi033 ceph-mon[28330]: pgmap v657: 129 pgs: 129 active+clean; 3.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 2 op/s 2024-05-03T18:48:02.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:48:02 smithi150 ceph-mon[32492]: pgmap v657: 129 pgs: 129 active+clean; 3.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 2 op/s 2024-05-03T18:48:02.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:48:02 smithi099 ceph-mon[31658]: pgmap v657: 129 pgs: 129 active+clean; 3.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 2 op/s 2024-05-03T18:48:02.657 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.c/config 2024-05-03T18:48:04.442 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:48:04 smithi150 ceph-mon[32492]: pgmap v658: 129 pgs: 129 active+clean; 3.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:48:04.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:48:04 smithi033 ceph-mon[28330]: pgmap v658: 129 pgs: 129 active+clean; 3.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:48:04.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:48:04 smithi099 ceph-mon[31658]: pgmap v658: 129 pgs: 129 active+clean; 3.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-05-03T18:48:04.980 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-05-03T18:48:04.980 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-0": { 2024-05-03T18:48:04.980 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 431, 2024-05-03T18:48:04.980 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 421, 2024-05-03T18:48:04.980 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 1621072, 2024-05-03T18:48:04.980 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 260416, 2024-05-03T18:48:04.980 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 4, 2024-05-03T18:48:04.980 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 0, 2024-05-03T18:48:04.980 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.160153978, 2024-05-03T18:48:04.980 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.061088106, 2024-05-03T18:48:04.981 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.096776763, 2024-05-03T18:48:04.981 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.019143296, 2024-05-03T18:48:04.981 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-05-03T18:48:04.981 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 420, 2024-05-03T18:48:04.981 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.043958327, 2024-05-03T18:48:04.981 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000104662 2024-05-03T18:48:04.981 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-03T18:48:04.981 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-05-03T18:48:04.981 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-03T18:48:04.981 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-03T18:48:04.981 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-03T18:48:04.981 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-03T18:48:04.981 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 1621072, 2024-05-03T18:48:04.982 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 260416 2024-05-03T18:48:04.982 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-03T18:48:04.982 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-1": { 2024-05-03T18:48:04.982 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 7, 2024-05-03T18:48:04.982 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 324, 2024-05-03T18:48:04.982 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 3392, 2024-05-03T18:48:04.982 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 227712, 2024-05-03T18:48:04.982 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 6, 2024-05-03T18:48:04.982 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 3, 2024-05-03T18:48:04.982 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.038743867, 2024-05-03T18:48:04.982 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.023019387, 2024-05-03T18:48:04.982 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.006495219, 2024-05-03T18:48:04.982 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-03T18:48:04.982 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-05-03T18:48:04.983 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 322, 2024-05-03T18:48:04.983 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.060997759, 2024-05-03T18:48:04.983 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000189434 2024-05-03T18:48:04.983 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-03T18:48:04.983 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-05-03T18:48:04.983 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-03T18:48:04.983 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-03T18:48:04.983 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-03T18:48:04.983 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-03T18:48:04.983 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 3392, 2024-05-03T18:48:04.983 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 227712 2024-05-03T18:48:04.983 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-03T18:48:04.983 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-2": { 2024-05-03T18:48:04.983 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 0, 2024-05-03T18:48:04.983 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 2, 2024-05-03T18:48:04.984 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 0, 2024-05-03T18:48:04.984 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 0, 2024-05-03T18:48:04.984 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 2, 2024-05-03T18:48:04.984 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 1, 2024-05-03T18:48:04.984 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.000736084, 2024-05-03T18:48:04.984 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.000000000, 2024-05-03T18:48:04.984 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.000514743, 2024-05-03T18:48:04.984 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-03T18:48:04.984 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-05-03T18:48:04.984 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-03T18:48:04.984 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-03T18:48:04.984 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-03T18:48:04.984 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-03T18:48:04.985 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-05-03T18:48:04.985 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-03T18:48:04.985 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-03T18:48:04.985 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-03T18:48:04.985 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-03T18:48:04.985 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-03T18:48:04.985 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 0 2024-05-03T18:48:04.985 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-03T18:48:04.985 INFO:teuthology.orchestra.run.smithi150.stdout: "cct": { 2024-05-03T18:48:04.985 INFO:teuthology.orchestra.run.smithi150.stdout: "total_workers": 0, 2024-05-03T18:48:04.985 INFO:teuthology.orchestra.run.smithi150.stdout: "unhealthy_workers": 0 2024-05-03T18:48:04.985 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-03T18:48:04.985 INFO:teuthology.orchestra.run.smithi150.stdout: "mempool": { 2024-05-03T18:48:04.985 INFO:teuthology.orchestra.run.smithi150.stdout: "bloom_filter_bytes": 0, 2024-05-03T18:48:04.985 INFO:teuthology.orchestra.run.smithi150.stdout: "bloom_filter_items": 0, 2024-05-03T18:48:04.986 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_alloc_bytes": 0, 2024-05-03T18:48:04.986 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_alloc_items": 0, 2024-05-03T18:48:04.986 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_data_bytes": 0, 2024-05-03T18:48:04.986 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_data_items": 0, 2024-05-03T18:48:04.986 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-03T18:48:04.986 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_onode_items": 0, 2024-05-03T18:48:04.986 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-03T18:48:04.986 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_meta_items": 0, 2024-05-03T18:48:04.986 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_other_bytes": 0, 2024-05-03T18:48:04.986 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_other_items": 0, 2024-05-03T18:48:04.986 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-03T18:48:04.986 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_buffer_items": 0, 2024-05-03T18:48:04.986 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_extent_bytes": 0, 2024-05-03T18:48:04.986 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_extent_items": 0, 2024-05-03T18:48:04.987 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_blob_bytes": 0, 2024-05-03T18:48:04.987 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_blob_items": 0, 2024-05-03T18:48:04.987 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-03T18:48:04.987 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_shared_blob_items": 0, 2024-05-03T18:48:04.987 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-03T18:48:04.987 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_inline_bl_items": 0, 2024-05-03T18:48:04.987 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_bytes": 0, 2024-05-03T18:48:04.987 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_items": 0, 2024-05-03T18:48:04.987 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_txc_bytes": 0, 2024-05-03T18:48:04.987 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_txc_items": 0, 2024-05-03T18:48:04.987 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-03T18:48:04.987 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_deferred_items": 0, 2024-05-03T18:48:04.987 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_bytes": 0, 2024-05-03T18:48:04.987 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_items": 0, 2024-05-03T18:48:04.987 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_bytes": 0, 2024-05-03T18:48:04.988 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_items": 0, 2024-05-03T18:48:04.988 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_reader_bytes": 0, 2024-05-03T18:48:04.988 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_reader_items": 0, 2024-05-03T18:48:04.988 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_writer_bytes": 0, 2024-05-03T18:48:04.988 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_writer_items": 0, 2024-05-03T18:48:04.988 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_anon_bytes": 148744, 2024-05-03T18:48:04.988 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_anon_items": 30, 2024-05-03T18:48:04.988 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_meta_bytes": 0, 2024-05-03T18:48:04.988 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_meta_items": 0, 2024-05-03T18:48:04.988 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_bytes": 0, 2024-05-03T18:48:04.988 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_items": 0, 2024-05-03T18:48:04.988 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mapbl_bytes": 0, 2024-05-03T18:48:04.988 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mapbl_items": 0, 2024-05-03T18:48:04.988 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pglog_bytes": 0, 2024-05-03T18:48:04.988 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pglog_items": 0, 2024-05-03T18:48:04.988 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_bytes": 0, 2024-05-03T18:48:04.989 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_items": 0, 2024-05-03T18:48:04.989 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_mapping_bytes": 0, 2024-05-03T18:48:04.989 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_mapping_items": 0, 2024-05-03T18:48:04.989 INFO:teuthology.orchestra.run.smithi150.stdout: "pgmap_bytes": 0, 2024-05-03T18:48:04.989 INFO:teuthology.orchestra.run.smithi150.stdout: "pgmap_items": 0, 2024-05-03T18:48:04.989 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_co_bytes": 928, 2024-05-03T18:48:04.989 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_co_items": 2, 2024-05-03T18:48:04.989 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_1_bytes": 0, 2024-05-03T18:48:04.989 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_1_items": 0, 2024-05-03T18:48:04.989 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_2_bytes": 0, 2024-05-03T18:48:04.989 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_2_items": 0 2024-05-03T18:48:04.989 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-03T18:48:04.989 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-03T18:48:04.989 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-05-03T18:48:04.989 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 104857600, 2024-05-03T18:48:04.990 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-05-03T18:48:04.990 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 434, 2024-05-03T18:48:04.990 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 1560175, 2024-05-03T18:48:04.990 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-05-03T18:48:04.990 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 434, 2024-05-03T18:48:04.990 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 0, 2024-05-03T18:48:04.990 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 0, 2024-05-03T18:48:04.990 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 434, 2024-05-03T18:48:04.990 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 1560175, 2024-05-03T18:48:04.990 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-05-03T18:48:04.990 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-03T18:48:04.990 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-03T18:48:04.991 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-03T18:48:04.991 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-03T18:48:04.991 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-03T18:48:04.991 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-05-03T18:48:05.805 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.f 2024-05-03T18:48:05.805 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.f 2024-05-03T18:48:05.805 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.f 2024-05-03T18:48:05.805 DEBUG:tasks.check_counter:Getting stats from i 2024-05-03T18:48:05.805 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph daemon mds.i perf dump 2024-05-03T18:48:06.349 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.c/config 2024-05-03T18:48:06.486 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:48:06 smithi033 ceph-mon[28330]: pgmap v659: 129 pgs: 129 active+clean; 3.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-05-03T18:48:06.532 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:48:06 smithi150 ceph-mon[32492]: pgmap v659: 129 pgs: 129 active+clean; 3.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-05-03T18:48:06.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:48:06 smithi099 ceph-mon[31658]: pgmap v659: 129 pgs: 129 active+clean; 3.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-05-03T18:48:07.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:48:07 smithi033 ceph-mon[28330]: osdmap e121: 12 total, 12 up, 12 in 2024-05-03T18:48:07.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:48:07 smithi099 ceph-mon[31658]: osdmap e121: 12 total, 12 up, 12 in 2024-05-03T18:48:07.587 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:48:07 smithi150 ceph-mon[32492]: osdmap e121: 12 total, 12 up, 12 in 2024-05-03T18:48:08.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:48:08 smithi033 ceph-mon[28330]: pgmap v661: 129 pgs: 129 active+clean; 3.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 22 op/s 2024-05-03T18:48:08.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:48:08 smithi099 ceph-mon[31658]: pgmap v661: 129 pgs: 129 active+clean; 3.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 22 op/s 2024-05-03T18:48:08.556 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:48:08 smithi150 ceph-mon[32492]: pgmap v661: 129 pgs: 129 active+clean; 3.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 22 op/s 2024-05-03T18:48:09.681 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-05-03T18:48:09.682 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-0": { 2024-05-03T18:48:09.682 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 429, 2024-05-03T18:48:09.682 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 419, 2024-05-03T18:48:09.682 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 1620528, 2024-05-03T18:48:09.682 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 259136, 2024-05-03T18:48:09.682 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 4, 2024-05-03T18:48:09.682 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 0, 2024-05-03T18:48:09.682 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.161332047, 2024-05-03T18:48:09.682 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.061488788, 2024-05-03T18:48:09.682 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.099153129, 2024-05-03T18:48:09.682 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.018365633, 2024-05-03T18:48:09.683 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-05-03T18:48:09.683 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 418, 2024-05-03T18:48:09.683 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.046273159, 2024-05-03T18:48:09.683 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000110701 2024-05-03T18:48:09.683 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-03T18:48:09.683 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-05-03T18:48:09.683 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-03T18:48:09.683 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-03T18:48:09.683 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-03T18:48:09.683 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-03T18:48:09.683 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 1620528, 2024-05-03T18:48:09.683 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 259136 2024-05-03T18:48:09.683 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-03T18:48:09.683 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-1": { 2024-05-03T18:48:09.683 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 2, 2024-05-03T18:48:09.684 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 319, 2024-05-03T18:48:09.684 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 320, 2024-05-03T18:48:09.684 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 147936, 2024-05-03T18:48:09.684 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 5, 2024-05-03T18:48:09.684 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 3, 2024-05-03T18:48:09.684 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.039185547, 2024-05-03T18:48:09.684 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.025296931, 2024-05-03T18:48:09.684 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.006766775, 2024-05-03T18:48:09.684 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-03T18:48:09.684 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-05-03T18:48:09.684 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 316, 2024-05-03T18:48:09.684 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.040770561, 2024-05-03T18:48:09.684 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000129020 2024-05-03T18:48:09.684 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-03T18:48:09.684 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-05-03T18:48:09.684 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-03T18:48:09.685 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-03T18:48:09.685 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-03T18:48:09.685 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-03T18:48:09.685 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-03T18:48:09.685 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 147936 2024-05-03T18:48:09.685 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-03T18:48:09.685 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-2": { 2024-05-03T18:48:09.685 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 4, 2024-05-03T18:48:09.685 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 4, 2024-05-03T18:48:09.685 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 2912, 2024-05-03T18:48:09.685 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 448, 2024-05-03T18:48:09.685 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 2, 2024-05-03T18:48:09.685 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 1, 2024-05-03T18:48:09.685 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.001568386, 2024-05-03T18:48:09.686 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.000094022, 2024-05-03T18:48:09.686 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.001383677, 2024-05-03T18:48:09.686 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-03T18:48:09.686 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-05-03T18:48:09.686 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 3, 2024-05-03T18:48:09.686 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.001777004, 2024-05-03T18:48:09.686 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000592334 2024-05-03T18:48:09.686 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-03T18:48:09.686 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-05-03T18:48:09.686 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-03T18:48:09.686 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-03T18:48:09.686 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-03T18:48:09.686 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-03T18:48:09.686 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 2912, 2024-05-03T18:48:09.687 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 448 2024-05-03T18:48:09.687 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-03T18:48:09.687 INFO:teuthology.orchestra.run.smithi150.stdout: "cct": { 2024-05-03T18:48:09.687 INFO:teuthology.orchestra.run.smithi150.stdout: "total_workers": 0, 2024-05-03T18:48:09.687 INFO:teuthology.orchestra.run.smithi150.stdout: "unhealthy_workers": 0 2024-05-03T18:48:09.687 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-03T18:48:09.687 INFO:teuthology.orchestra.run.smithi150.stdout: "mempool": { 2024-05-03T18:48:09.687 INFO:teuthology.orchestra.run.smithi150.stdout: "bloom_filter_bytes": 0, 2024-05-03T18:48:09.687 INFO:teuthology.orchestra.run.smithi150.stdout: "bloom_filter_items": 0, 2024-05-03T18:48:09.687 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_alloc_bytes": 0, 2024-05-03T18:48:09.687 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_alloc_items": 0, 2024-05-03T18:48:09.687 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_data_bytes": 0, 2024-05-03T18:48:09.687 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_data_items": 0, 2024-05-03T18:48:09.687 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-03T18:48:09.687 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_onode_items": 0, 2024-05-03T18:48:09.688 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-03T18:48:09.688 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_meta_items": 0, 2024-05-03T18:48:09.688 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_other_bytes": 0, 2024-05-03T18:48:09.688 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_other_items": 0, 2024-05-03T18:48:09.688 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-03T18:48:09.688 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_buffer_items": 0, 2024-05-03T18:48:09.688 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_extent_bytes": 0, 2024-05-03T18:48:09.688 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_extent_items": 0, 2024-05-03T18:48:09.688 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_blob_bytes": 0, 2024-05-03T18:48:09.688 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_blob_items": 0, 2024-05-03T18:48:09.688 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-03T18:48:09.688 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_shared_blob_items": 0, 2024-05-03T18:48:09.688 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-03T18:48:09.688 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_inline_bl_items": 0, 2024-05-03T18:48:09.688 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_bytes": 0, 2024-05-03T18:48:09.689 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_items": 0, 2024-05-03T18:48:09.689 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_txc_bytes": 0, 2024-05-03T18:48:09.689 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_txc_items": 0, 2024-05-03T18:48:09.689 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-03T18:48:09.689 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_deferred_items": 0, 2024-05-03T18:48:09.689 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_bytes": 0, 2024-05-03T18:48:09.689 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_items": 0, 2024-05-03T18:48:09.689 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_bytes": 0, 2024-05-03T18:48:09.689 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_items": 0, 2024-05-03T18:48:09.689 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_reader_bytes": 0, 2024-05-03T18:48:09.689 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_reader_items": 0, 2024-05-03T18:48:09.689 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_writer_bytes": 0, 2024-05-03T18:48:09.689 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_writer_items": 0, 2024-05-03T18:48:09.689 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_anon_bytes": 148794, 2024-05-03T18:48:09.689 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_anon_items": 32, 2024-05-03T18:48:09.689 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_meta_bytes": 0, 2024-05-03T18:48:09.690 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_meta_items": 0, 2024-05-03T18:48:09.690 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_bytes": 0, 2024-05-03T18:48:09.690 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_items": 0, 2024-05-03T18:48:09.690 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mapbl_bytes": 0, 2024-05-03T18:48:09.690 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mapbl_items": 0, 2024-05-03T18:48:09.690 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pglog_bytes": 0, 2024-05-03T18:48:09.690 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pglog_items": 0, 2024-05-03T18:48:09.690 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_bytes": 0, 2024-05-03T18:48:09.690 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_items": 0, 2024-05-03T18:48:09.690 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_mapping_bytes": 0, 2024-05-03T18:48:09.690 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_mapping_items": 0, 2024-05-03T18:48:09.690 INFO:teuthology.orchestra.run.smithi150.stdout: "pgmap_bytes": 0, 2024-05-03T18:48:09.690 INFO:teuthology.orchestra.run.smithi150.stdout: "pgmap_items": 0, 2024-05-03T18:48:09.690 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_co_bytes": 928, 2024-05-03T18:48:09.691 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_co_items": 2, 2024-05-03T18:48:09.691 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_1_bytes": 0, 2024-05-03T18:48:09.691 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_1_items": 0, 2024-05-03T18:48:09.691 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_2_bytes": 0, 2024-05-03T18:48:09.691 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_2_items": 0 2024-05-03T18:48:09.691 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-03T18:48:09.691 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-03T18:48:09.691 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-05-03T18:48:09.691 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 104857600, 2024-05-03T18:48:09.691 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-05-03T18:48:09.691 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 431, 2024-05-03T18:48:09.691 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 1559888, 2024-05-03T18:48:09.691 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-05-03T18:48:09.691 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 431, 2024-05-03T18:48:09.692 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 0, 2024-05-03T18:48:09.692 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 0, 2024-05-03T18:48:09.692 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 431, 2024-05-03T18:48:09.692 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 1559888, 2024-05-03T18:48:09.692 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-05-03T18:48:09.692 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-03T18:48:09.692 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-03T18:48:09.692 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-03T18:48:09.692 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-03T18:48:09.692 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-03T18:48:09.692 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-05-03T18:48:10.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:48:10 smithi150 ceph-mon[32492]: pgmap v662: 129 pgs: 129 active+clean; 3.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 22 op/s 2024-05-03T18:48:10.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:48:10 smithi033 ceph-mon[28330]: pgmap v662: 129 pgs: 129 active+clean; 3.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 22 op/s 2024-05-03T18:48:10.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:48:10 smithi099 ceph-mon[31658]: pgmap v662: 129 pgs: 129 active+clean; 3.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 22 op/s 2024-05-03T18:48:10.567 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.i 2024-05-03T18:48:10.567 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.i 2024-05-03T18:48:10.567 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.i 2024-05-03T18:48:10.567 DEBUG:tasks.check_counter:Getting stats from l 2024-05-03T18:48:10.568 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph daemon mds.l perf dump 2024-05-03T18:48:11.030 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.c/config 2024-05-03T18:48:12.383 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:48:12 smithi150 ceph-mon[32492]: pgmap v663: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 22 op/s 2024-05-03T18:48:12.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:48:12 smithi033 ceph-mon[28330]: pgmap v663: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 22 op/s 2024-05-03T18:48:12.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:48:12 smithi099 ceph-mon[31658]: pgmap v663: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 22 op/s 2024-05-03T18:48:13.377 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-05-03T18:48:13.377 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-0": { 2024-05-03T18:48:13.377 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 4, 2024-05-03T18:48:13.377 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 4, 2024-05-03T18:48:13.377 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 2912, 2024-05-03T18:48:13.377 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 448, 2024-05-03T18:48:13.377 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 4, 2024-05-03T18:48:13.377 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 0, 2024-05-03T18:48:13.378 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.004818663, 2024-05-03T18:48:13.378 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.000090319, 2024-05-03T18:48:13.378 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.001302828, 2024-05-03T18:48:13.378 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-03T18:48:13.378 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-05-03T18:48:13.378 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 3, 2024-05-03T18:48:13.378 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.001525499, 2024-05-03T18:48:13.378 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000508499 2024-05-03T18:48:13.378 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-03T18:48:13.378 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-05-03T18:48:13.378 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-03T18:48:13.378 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-03T18:48:13.379 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-03T18:48:13.379 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-03T18:48:13.379 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 2912, 2024-05-03T18:48:13.379 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 448 2024-05-03T18:48:13.379 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-03T18:48:13.379 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-1": { 2024-05-03T18:48:13.379 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 439, 2024-05-03T18:48:13.379 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 429, 2024-05-03T18:48:13.379 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 1623248, 2024-05-03T18:48:13.379 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 265536, 2024-05-03T18:48:13.380 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 2, 2024-05-03T18:48:13.380 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 1, 2024-05-03T18:48:13.380 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.157678650, 2024-05-03T18:48:13.380 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.057556087, 2024-05-03T18:48:13.380 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.103716579, 2024-05-03T18:48:13.380 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.019577805, 2024-05-03T18:48:13.380 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-05-03T18:48:13.380 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 428, 2024-05-03T18:48:13.380 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.044640850, 2024-05-03T18:48:13.380 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000104301 2024-05-03T18:48:13.380 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-03T18:48:13.380 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-05-03T18:48:13.381 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-03T18:48:13.381 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-03T18:48:13.381 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-03T18:48:13.381 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-03T18:48:13.381 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 1623248, 2024-05-03T18:48:13.381 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 265536 2024-05-03T18:48:13.381 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-03T18:48:13.381 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-2": { 2024-05-03T18:48:13.381 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 3, 2024-05-03T18:48:13.381 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 328, 2024-05-03T18:48:13.381 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 480, 2024-05-03T18:48:13.381 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 228320, 2024-05-03T18:48:13.382 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 6, 2024-05-03T18:48:13.382 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 3, 2024-05-03T18:48:13.382 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.040902640, 2024-05-03T18:48:13.382 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.026450047, 2024-05-03T18:48:13.382 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.006906446, 2024-05-03T18:48:13.382 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-03T18:48:13.382 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-05-03T18:48:13.382 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 325, 2024-05-03T18:48:13.382 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.063991224, 2024-05-03T18:48:13.382 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000196896 2024-05-03T18:48:13.382 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-03T18:48:13.383 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-05-03T18:48:13.383 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-03T18:48:13.383 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-03T18:48:13.383 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-03T18:48:13.383 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-03T18:48:13.383 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 480, 2024-05-03T18:48:13.383 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 228320 2024-05-03T18:48:13.383 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-03T18:48:13.383 INFO:teuthology.orchestra.run.smithi150.stdout: "cct": { 2024-05-03T18:48:13.383 INFO:teuthology.orchestra.run.smithi150.stdout: "total_workers": 0, 2024-05-03T18:48:13.383 INFO:teuthology.orchestra.run.smithi150.stdout: "unhealthy_workers": 0 2024-05-03T18:48:13.383 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-03T18:48:13.383 INFO:teuthology.orchestra.run.smithi150.stdout: "mempool": { 2024-05-03T18:48:13.384 INFO:teuthology.orchestra.run.smithi150.stdout: "bloom_filter_bytes": 0, 2024-05-03T18:48:13.384 INFO:teuthology.orchestra.run.smithi150.stdout: "bloom_filter_items": 0, 2024-05-03T18:48:13.384 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_alloc_bytes": 0, 2024-05-03T18:48:13.384 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_alloc_items": 0, 2024-05-03T18:48:13.384 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_data_bytes": 0, 2024-05-03T18:48:13.384 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_data_items": 0, 2024-05-03T18:48:13.384 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-03T18:48:13.384 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_onode_items": 0, 2024-05-03T18:48:13.384 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-03T18:48:13.384 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_meta_items": 0, 2024-05-03T18:48:13.384 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_other_bytes": 0, 2024-05-03T18:48:13.384 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_other_items": 0, 2024-05-03T18:48:13.384 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-03T18:48:13.385 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_buffer_items": 0, 2024-05-03T18:48:13.385 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_extent_bytes": 0, 2024-05-03T18:48:13.385 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_extent_items": 0, 2024-05-03T18:48:13.385 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_blob_bytes": 0, 2024-05-03T18:48:13.385 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_blob_items": 0, 2024-05-03T18:48:13.385 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-03T18:48:13.385 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_shared_blob_items": 0, 2024-05-03T18:48:13.385 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-03T18:48:13.385 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_inline_bl_items": 0, 2024-05-03T18:48:13.385 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_bytes": 0, 2024-05-03T18:48:13.385 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_items": 0, 2024-05-03T18:48:13.386 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_txc_bytes": 0, 2024-05-03T18:48:13.386 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_txc_items": 0, 2024-05-03T18:48:13.386 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-03T18:48:13.386 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_deferred_items": 0, 2024-05-03T18:48:13.386 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_bytes": 0, 2024-05-03T18:48:13.386 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_items": 0, 2024-05-03T18:48:13.386 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_bytes": 0, 2024-05-03T18:48:13.386 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_items": 0, 2024-05-03T18:48:13.386 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_reader_bytes": 0, 2024-05-03T18:48:13.386 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_reader_items": 0, 2024-05-03T18:48:13.386 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_writer_bytes": 0, 2024-05-03T18:48:13.386 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_writer_items": 0, 2024-05-03T18:48:13.387 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_anon_bytes": 148692, 2024-05-03T18:48:13.387 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_anon_items": 30, 2024-05-03T18:48:13.387 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_meta_bytes": 0, 2024-05-03T18:48:13.387 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_meta_items": 0, 2024-05-03T18:48:13.387 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_bytes": 0, 2024-05-03T18:48:13.387 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_items": 0, 2024-05-03T18:48:13.387 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mapbl_bytes": 0, 2024-05-03T18:48:13.387 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mapbl_items": 0, 2024-05-03T18:48:13.387 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pglog_bytes": 0, 2024-05-03T18:48:13.387 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pglog_items": 0, 2024-05-03T18:48:13.387 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_bytes": 0, 2024-05-03T18:48:13.388 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_items": 0, 2024-05-03T18:48:13.388 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_mapping_bytes": 0, 2024-05-03T18:48:13.388 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_mapping_items": 0, 2024-05-03T18:48:13.388 INFO:teuthology.orchestra.run.smithi150.stdout: "pgmap_bytes": 0, 2024-05-03T18:48:13.388 INFO:teuthology.orchestra.run.smithi150.stdout: "pgmap_items": 0, 2024-05-03T18:48:13.388 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_co_bytes": 928, 2024-05-03T18:48:13.388 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_co_items": 2, 2024-05-03T18:48:13.388 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_1_bytes": 0, 2024-05-03T18:48:13.388 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_1_items": 0, 2024-05-03T18:48:13.388 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_2_bytes": 0, 2024-05-03T18:48:13.388 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_2_items": 0 2024-05-03T18:48:13.388 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-05-03T18:48:13.388 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-03T18:48:13.389 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-05-03T18:48:13.389 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 104857600, 2024-05-03T18:48:13.389 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-05-03T18:48:13.389 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 442, 2024-05-03T18:48:13.389 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 1561215, 2024-05-03T18:48:13.389 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-05-03T18:48:13.389 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 442, 2024-05-03T18:48:13.389 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 0, 2024-05-03T18:48:13.389 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 0, 2024-05-03T18:48:13.389 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 442, 2024-05-03T18:48:13.389 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 1561215, 2024-05-03T18:48:13.389 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-05-03T18:48:13.389 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-05-03T18:48:13.389 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-05-03T18:48:13.390 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-05-03T18:48:13.390 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-03T18:48:13.390 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-05-03T18:48:13.390 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-05-03T18:48:13.866 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:48:13 smithi150 ceph-mon[32492]: pgmap v664: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 22 op/s 2024-05-03T18:48:13.866 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:48:13 smithi150 ceph-mon[32492]: osdmap e122: 12 total, 12 up, 12 in 2024-05-03T18:48:13.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:48:13 smithi033 ceph-mon[28330]: pgmap v664: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 22 op/s 2024-05-03T18:48:13.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:48:13 smithi033 ceph-mon[28330]: osdmap e122: 12 total, 12 up, 12 in 2024-05-03T18:48:14.047 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:48:13 smithi099 ceph-mon[31658]: pgmap v664: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 22 op/s 2024-05-03T18:48:14.047 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:48:13 smithi099 ceph-mon[31658]: osdmap e122: 12 total, 12 up, 12 in 2024-05-03T18:48:14.259 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2024-05-03T18:48:14.259 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2024-05-03T18:48:14.259 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2024-05-03T18:48:14.259 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-05-03T18:48:14.271 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-05-03T18:48:14.272 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-03T18:48:14.272 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-03T18:48:14.349 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-03T18:48:14.350 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-03T18:48:14.406 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph osd blocklist ls 2024-05-03T18:48:14.906 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:48:15.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:48:15 smithi033 ceph-mon[28330]: pgmap v666: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 249 B/s wr, 2 op/s 2024-05-03T18:48:16.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:48:15 smithi099 ceph-mon[31658]: pgmap v666: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 249 B/s wr, 2 op/s 2024-05-03T18:48:16.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:48:15 smithi150 ceph-mon[32492]: pgmap v666: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 249 B/s wr, 2 op/s 2024-05-03T18:48:17.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:17.750+0000 7f938be00700 1 -- 172.21.15.33:0/3517577673 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f938c109460 msgr2=0x7f938c109840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:48:17.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:17.750+0000 7f938be00700 1 --2- 172.21.15.33:0/3517577673 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f938c109460 0x7f938c109840 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f937c009990 tx=0x7f937c009ca0 comp rx=0 tx=0).stop 2024-05-03T18:48:17.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:17.751+0000 7f938be00700 1 -- 172.21.15.33:0/3517577673 shutdown_connections 2024-05-03T18:48:17.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:17.751+0000 7f938be00700 1 --2- 172.21.15.33:0/3517577673 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f938c103d90 0x7f938c1102a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:48:17.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:17.751+0000 7f938be00700 1 --2- 172.21.15.33:0/3517577673 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f938c109460 0x7f938c109840 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:48:17.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:17.751+0000 7f938be00700 1 --2- 172.21.15.33:0/3517577673 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f938c1033d0 0x7f938c103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:48:17.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:17.751+0000 7f938be00700 1 -- 172.21.15.33:0/3517577673 >> 172.21.15.33:0/3517577673 conn(0x7f938c0fef20 msgr2=0x7f938c101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:48:17.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:17.751+0000 7f938be00700 1 -- 172.21.15.33:0/3517577673 shutdown_connections 2024-05-03T18:48:17.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:17.751+0000 7f938be00700 1 -- 172.21.15.33:0/3517577673 wait complete. 2024-05-03T18:48:17.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:17.752+0000 7f938be00700 1 Processor -- start 2024-05-03T18:48:17.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:17.752+0000 7f938be00700 1 -- start start 2024-05-03T18:48:17.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:17.753+0000 7f938be00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f938c1033d0 0x7f938c10e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:48:17.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:17.753+0000 7f938be00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f938c103d90 0x7f938c10eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:48:17.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:17.753+0000 7f938be00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f938c109460 0x7f938c1a4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:48:17.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:17.753+0000 7f938be00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f938c113490 con 0x7f938c103d90 2024-05-03T18:48:17.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:17.753+0000 7f938be00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f938c113310 con 0x7f938c109460 2024-05-03T18:48:17.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:17.753+0000 7f938be00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f938c113610 con 0x7f938c1033d0 2024-05-03T18:48:17.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:17.753+0000 7f9382000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f938c103d90 0x7f938c10eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:48:17.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:17.753+0000 7f9382000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f938c103d90 0x7f938c10eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34186/0 (socket says 172.21.15.33:34186) 2024-05-03T18:48:17.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:17.753+0000 7f9382000700 1 -- 172.21.15.33:0/3573376245 learned_addr learned my addr 172.21.15.33:0/3573376245 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:48:17.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:17.753+0000 7f938aa00700 1 --2- 172.21.15.33:0/3573376245 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f938c1033d0 0x7f938c10e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:48:17.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:17.753+0000 7f938b400700 1 --2- 172.21.15.33:0/3573376245 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f938c109460 0x7f938c1a4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:48:17.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:17.754+0000 7f9382000700 1 -- 172.21.15.33:0/3573376245 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f938c1033d0 msgr2=0x7f938c10e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:48:17.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:17.754+0000 7f9382000700 1 --2- 172.21.15.33:0/3573376245 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f938c1033d0 0x7f938c10e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:48:17.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:17.754+0000 7f9382000700 1 -- 172.21.15.33:0/3573376245 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f938c109460 msgr2=0x7f938c1a4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:48:17.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:17.754+0000 7f9382000700 1 --2- 172.21.15.33:0/3573376245 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f938c109460 0x7f938c1a4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:48:17.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:17.754+0000 7f9382000700 1 -- 172.21.15.33:0/3573376245 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f938c1a5020 con 0x7f938c103d90 2024-05-03T18:48:17.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:17.754+0000 7f9382000700 1 --2- 172.21.15.33:0/3573376245 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f938c103d90 0x7f938c10eac0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f937400d980 tx=0x7f937400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:48:17.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:17.754+0000 7f9383e00700 1 -- 172.21.15.33:0/3573376245 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9374017070 con 0x7f938c103d90 2024-05-03T18:48:17.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:17.754+0000 7f9383e00700 1 -- 172.21.15.33:0/3573376245 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9374005020 con 0x7f938c103d90 2024-05-03T18:48:17.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:17.754+0000 7f938be00700 1 -- 172.21.15.33:0/3573376245 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f938c1a5310 con 0x7f938c103d90 2024-05-03T18:48:17.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:17.755+0000 7f9383e00700 1 -- 172.21.15.33:0/3573376245 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9374009ab0 con 0x7f938c103d90 2024-05-03T18:48:17.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:17.755+0000 7f938be00700 1 -- 172.21.15.33:0/3573376245 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f938c1acb90 con 0x7f938c103d90 2024-05-03T18:48:17.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:17.756+0000 7f938be00700 1 -- 172.21.15.33:0/3573376245 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f938c06a190 con 0x7f938c103d90 2024-05-03T18:48:17.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:17.760+0000 7f9383e00700 1 -- 172.21.15.33:0/3573376245 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f93740051d0 con 0x7f938c103d90 2024-05-03T18:48:17.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:17.762+0000 7f9383e00700 1 --2- 172.21.15.33:0/3573376245 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f93700b2a70 0x7f93700b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:48:17.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:17.762+0000 7f9383e00700 1 -- 172.21.15.33:0/3573376245 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 9518+0+0 (secure 0 0 0) 0x7f93740e2d30 con 0x7f938c103d90 2024-05-03T18:48:17.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:17.762+0000 7f938aa00700 1 --2- 172.21.15.33:0/3573376245 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f93700b2a70 0x7f93700b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:48:17.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:17.763+0000 7f938aa00700 1 --2- 172.21.15.33:0/3573376245 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f93700b2a70 0x7f93700b4f30 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f937c003760 tx=0x7f937c005fd0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:48:17.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:17.765+0000 7f9383e00700 1 -- 172.21.15.33:0/3573376245 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f9374020020 con 0x7f938c103d90 2024-05-03T18:48:17.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:48:17 smithi033 ceph-mon[28330]: pgmap v667: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-05-03T18:48:18.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:18.025+0000 7f938be00700 1 -- 172.21.15.33:0/3573376245 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f938c1acd60 con 0x7f938c103d90 2024-05-03T18:48:18.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:18.026+0000 7f9383e00700 1 -- 172.21.15.33:0/3573376245 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 17 entries v122) ==== 81+0+1000 (secure 0 0 0) 0x7f93740aa350 con 0x7f938c103d90 2024-05-03T18:48:18.028 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/4003417322 2024-05-04T18:27:11.722511+0000 2024-05-03T18:48:18.028 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6801/4007322071 2024-05-04T18:11:44.239835+0000 2024-05-03T18:48:18.028 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6801/2830135294 2024-05-04T18:12:19.127652+0000 2024-05-03T18:48:18.028 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/3767665033 2024-05-04T18:12:19.127652+0000 2024-05-03T18:48:18.028 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6800/4007322071 2024-05-04T18:11:44.239835+0000 2024-05-03T18:48:18.028 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/1614640996 2024-05-04T18:11:44.239835+0000 2024-05-03T18:48:18.028 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/1314704208 2024-05-04T18:11:44.239835+0000 2024-05-03T18:48:18.029 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/2014902144 2024-05-04T18:11:44.239835+0000 2024-05-03T18:48:18.029 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/965859483 2024-05-04T18:12:19.127652+0000 2024-05-03T18:48:18.029 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6801/3642895501 2024-05-04T18:27:11.722511+0000 2024-05-03T18:48:18.029 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6800/2830135294 2024-05-04T18:12:19.127652+0000 2024-05-03T18:48:18.029 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/791712778 2024-05-04T18:27:11.722511+0000 2024-05-03T18:48:18.029 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/3991947997 2024-05-04T18:12:19.127652+0000 2024-05-03T18:48:18.029 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/806149166 2024-05-04T18:27:11.722511+0000 2024-05-03T18:48:18.029 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/1300142939 2024-05-04T18:27:11.722511+0000 2024-05-03T18:48:18.029 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6800/3642895501 2024-05-04T18:27:11.722511+0000 2024-05-03T18:48:18.029 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/757614565 2024-05-04T18:27:11.722511+0000 2024-05-03T18:48:18.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:18.030+0000 7f938be00700 1 -- 172.21.15.33:0/3573376245 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f93700b2a70 msgr2=0x7f93700b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:48:18.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:18.030+0000 7f938be00700 1 --2- 172.21.15.33:0/3573376245 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f93700b2a70 0x7f93700b4f30 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f937c003760 tx=0x7f937c005fd0 comp rx=0 tx=0).stop 2024-05-03T18:48:18.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:18.030+0000 7f938be00700 1 -- 172.21.15.33:0/3573376245 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f938c103d90 msgr2=0x7f938c10eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:48:18.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:18.030+0000 7f938be00700 1 --2- 172.21.15.33:0/3573376245 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f938c103d90 0x7f938c10eac0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f937400d980 tx=0x7f937400dc90 comp rx=0 tx=0).stop 2024-05-03T18:48:18.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:18.030+0000 7f938be00700 1 -- 172.21.15.33:0/3573376245 shutdown_connections 2024-05-03T18:48:18.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:18.030+0000 7f938be00700 1 --2- 172.21.15.33:0/3573376245 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f93700b2a70 0x7f93700b4f30 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:48:18.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:18.030+0000 7f938be00700 1 --2- 172.21.15.33:0/3573376245 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f938c109460 0x7f938c1a4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:48:18.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:18.030+0000 7f938be00700 1 --2- 172.21.15.33:0/3573376245 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f938c1033d0 0x7f938c10e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:48:18.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:18.030+0000 7f938be00700 1 --2- 172.21.15.33:0/3573376245 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f938c103d90 0x7f938c10eac0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:48:18.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:18.030+0000 7f938be00700 1 -- 172.21.15.33:0/3573376245 >> 172.21.15.33:0/3573376245 conn(0x7f938c0fef20 msgr2=0x7f938c0756d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:48:18.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:18.030+0000 7f938be00700 1 -- 172.21.15.33:0/3573376245 shutdown_connections 2024-05-03T18:48:18.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:18.031+0000 7f938be00700 1 -- 172.21.15.33:0/3573376245 wait complete. 2024-05-03T18:48:18.033 INFO:teuthology.orchestra.run.smithi033.stderr:listed 17 entries 2024-05-03T18:48:18.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:48:17 smithi099 ceph-mon[31658]: pgmap v667: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-05-03T18:48:18.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:48:17 smithi150 ceph-mon[32492]: pgmap v667: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-05-03T18:48:18.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:48:18 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/3573376245' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-03T18:48:19.035 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:48:18 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/3573376245' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-03T18:48:19.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:48:18 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/3573376245' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-03T18:48:19.173 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-03T18:48:19.173 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-03T18:48:19.231 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph osd blocklist ls 2024-05-03T18:48:19.677 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:48:19.982 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:48:19 smithi150 ceph-mon[32492]: pgmap v668: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-05-03T18:48:19.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:48:19 smithi033 ceph-mon[28330]: pgmap v668: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-05-03T18:48:20.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:48:19 smithi099 ceph-mon[31658]: pgmap v668: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-05-03T18:48:21.909 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:48:21 smithi033 ceph-mon[28330]: pgmap v669: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-05-03T18:48:22.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:48:21 smithi099 ceph-mon[31658]: pgmap v669: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-05-03T18:48:22.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:22.071+0000 7efefd200700 1 -- 172.21.15.33:0/1779993061 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efef810fb60 msgr2=0x7efef810ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:48:22.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:22.071+0000 7efefd200700 1 --2- 172.21.15.33:0/1779993061 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efef810fb60 0x7efef810ff40 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7efee8009990 tx=0x7efee8009ca0 comp rx=0 tx=0).stop 2024-05-03T18:48:22.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:22.071+0000 7efefd200700 1 -- 172.21.15.33:0/1779993061 shutdown_connections 2024-05-03T18:48:22.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:22.071+0000 7efefd200700 1 --2- 172.21.15.33:0/1779993061 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efef810a490 0x7efef81169a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:48:22.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:22.072+0000 7efefd200700 1 --2- 172.21.15.33:0/1779993061 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efef810fb60 0x7efef810ff40 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:48:22.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:22.072+0000 7efefd200700 1 --2- 172.21.15.33:0/1779993061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efef8109ad0 0x7efef8109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:48:22.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:22.072+0000 7efefd200700 1 -- 172.21.15.33:0/1779993061 >> 172.21.15.33:0/1779993061 conn(0x7efef8076c70 msgr2=0x7efef81082c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:48:22.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:22.072+0000 7efefd200700 1 -- 172.21.15.33:0/1779993061 shutdown_connections 2024-05-03T18:48:22.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:22.072+0000 7efefd200700 1 -- 172.21.15.33:0/1779993061 wait complete. 2024-05-03T18:48:22.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:22.073+0000 7efefd200700 1 Processor -- start 2024-05-03T18:48:22.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:22.073+0000 7efefd200700 1 -- start start 2024-05-03T18:48:22.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:22.074+0000 7efefd200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efef8109ad0 0x7efef81a00c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:48:22.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:22.074+0000 7efefd200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efef810a490 0x7efef81a0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:48:22.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:22.074+0000 7efefd200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efef810fb60 0x7efef81a4990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:48:22.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:22.074+0000 7efefd200700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7efef8119bd0 con 0x7efef810a490 2024-05-03T18:48:22.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:22.074+0000 7efefd200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7efef8119a50 con 0x7efef810fb60 2024-05-03T18:48:22.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:22.074+0000 7efefd200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7efef8119d50 con 0x7efef8109ad0 2024-05-03T18:48:22.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:22.074+0000 7efef5400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efef810a490 0x7efef81a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:48:22.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:22.074+0000 7efef5400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efef810a490 0x7efef81a0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34204/0 (socket says 172.21.15.33:34204) 2024-05-03T18:48:22.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:22.074+0000 7efef5400700 1 -- 172.21.15.33:0/2351521371 learned_addr learned my addr 172.21.15.33:0/2351521371 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:48:22.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:22.074+0000 7efef5e00700 1 --2- 172.21.15.33:0/2351521371 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efef8109ad0 0x7efef81a00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:48:22.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:22.075+0000 7efef6800700 1 --2- 172.21.15.33:0/2351521371 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efef810fb60 0x7efef81a4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:48:22.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:22.075+0000 7efef5400700 1 -- 172.21.15.33:0/2351521371 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efef8109ad0 msgr2=0x7efef81a00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:48:22.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:22.075+0000 7efef5400700 1 --2- 172.21.15.33:0/2351521371 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efef8109ad0 0x7efef81a00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:48:22.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:22.075+0000 7efef5400700 1 -- 172.21.15.33:0/2351521371 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efef810fb60 msgr2=0x7efef81a4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:48:22.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:22.075+0000 7efef5400700 1 --2- 172.21.15.33:0/2351521371 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efef810fb60 0x7efef81a4990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:48:22.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:22.075+0000 7efef5400700 1 -- 172.21.15.33:0/2351521371 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efef81a50d0 con 0x7efef810a490 2024-05-03T18:48:22.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:22.075+0000 7efef6800700 1 --2- 172.21.15.33:0/2351521371 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efef810fb60 0x7efef81a4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:48:22.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:22.075+0000 7efef5400700 1 --2- 172.21.15.33:0/2351521371 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efef810a490 0x7efef81a0600 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7efee400d950 tx=0x7efee400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:48:22.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:22.075+0000 7efeeea00700 1 -- 172.21.15.33:0/2351521371 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee4016070 con 0x7efef810a490 2024-05-03T18:48:22.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:22.075+0000 7efefd200700 1 -- 172.21.15.33:0/2351521371 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efef81a53c0 con 0x7efef810a490 2024-05-03T18:48:22.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:22.075+0000 7efeeea00700 1 -- 172.21.15.33:0/2351521371 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efee4010460 con 0x7efef810a490 2024-05-03T18:48:22.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:22.075+0000 7efeeea00700 1 -- 172.21.15.33:0/2351521371 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee400f620 con 0x7efef810a490 2024-05-03T18:48:22.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:22.076+0000 7efefd200700 1 -- 172.21.15.33:0/2351521371 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efef81acc40 con 0x7efef810a490 2024-05-03T18:48:22.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:22.077+0000 7efefd200700 1 -- 172.21.15.33:0/2351521371 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efef806a190 con 0x7efef810a490 2024-05-03T18:48:22.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:22.080+0000 7efeeea00700 1 -- 172.21.15.33:0/2351521371 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7efee4019070 con 0x7efef810a490 2024-05-03T18:48:22.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:22.081+0000 7efeeea00700 1 --2- 172.21.15.33:0/2351521371 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7efedc0b29f0 0x7efedc0b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:48:22.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:22.082+0000 7efeeea00700 1 -- 172.21.15.33:0/2351521371 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 9518+0+0 (secure 0 0 0) 0x7efee40e2670 con 0x7efef810a490 2024-05-03T18:48:22.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:22.082+0000 7efef5e00700 1 --2- 172.21.15.33:0/2351521371 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7efedc0b29f0 0x7efedc0b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:48:22.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:22.083+0000 7efef5e00700 1 --2- 172.21.15.33:0/2351521371 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7efedc0b29f0 0x7efedc0b4eb0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7efee800b5c0 tx=0x7efee801a040 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:48:22.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:22.083+0000 7efeeea00700 1 -- 172.21.15.33:0/2351521371 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7efee40a9c10 con 0x7efef810a490 2024-05-03T18:48:22.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:48:21 smithi150 ceph-mon[32492]: pgmap v669: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-05-03T18:48:22.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:22.345+0000 7efefd200700 1 -- 172.21.15.33:0/2351521371 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7efef802e510 con 0x7efef810a490 2024-05-03T18:48:22.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:22.346+0000 7efeeea00700 1 -- 172.21.15.33:0/2351521371 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 17 entries v122) ==== 81+0+1000 (secure 0 0 0) 0x7efee40a9510 con 0x7efef810a490 2024-05-03T18:48:22.347 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/4003417322 2024-05-04T18:27:11.722511+0000 2024-05-03T18:48:22.347 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6801/4007322071 2024-05-04T18:11:44.239835+0000 2024-05-03T18:48:22.348 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6801/2830135294 2024-05-04T18:12:19.127652+0000 2024-05-03T18:48:22.348 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/3767665033 2024-05-04T18:12:19.127652+0000 2024-05-03T18:48:22.348 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6800/4007322071 2024-05-04T18:11:44.239835+0000 2024-05-03T18:48:22.348 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/1614640996 2024-05-04T18:11:44.239835+0000 2024-05-03T18:48:22.348 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/1314704208 2024-05-04T18:11:44.239835+0000 2024-05-03T18:48:22.348 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/2014902144 2024-05-04T18:11:44.239835+0000 2024-05-03T18:48:22.348 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/965859483 2024-05-04T18:12:19.127652+0000 2024-05-03T18:48:22.348 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6801/3642895501 2024-05-04T18:27:11.722511+0000 2024-05-03T18:48:22.348 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6800/2830135294 2024-05-04T18:12:19.127652+0000 2024-05-03T18:48:22.348 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/791712778 2024-05-04T18:27:11.722511+0000 2024-05-03T18:48:22.348 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/3991947997 2024-05-04T18:12:19.127652+0000 2024-05-03T18:48:22.348 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/806149166 2024-05-04T18:27:11.722511+0000 2024-05-03T18:48:22.348 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/1300142939 2024-05-04T18:27:11.722511+0000 2024-05-03T18:48:22.348 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6800/3642895501 2024-05-04T18:27:11.722511+0000 2024-05-03T18:48:22.349 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/757614565 2024-05-04T18:27:11.722511+0000 2024-05-03T18:48:22.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:22.349+0000 7efefd200700 1 -- 172.21.15.33:0/2351521371 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7efedc0b29f0 msgr2=0x7efedc0b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:48:22.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:22.349+0000 7efefd200700 1 --2- 172.21.15.33:0/2351521371 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7efedc0b29f0 0x7efedc0b4eb0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7efee800b5c0 tx=0x7efee801a040 comp rx=0 tx=0).stop 2024-05-03T18:48:22.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:22.349+0000 7efefd200700 1 -- 172.21.15.33:0/2351521371 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efef810a490 msgr2=0x7efef81a0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:48:22.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:22.349+0000 7efefd200700 1 --2- 172.21.15.33:0/2351521371 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efef810a490 0x7efef81a0600 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7efee400d950 tx=0x7efee400dd10 comp rx=0 tx=0).stop 2024-05-03T18:48:22.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:22.349+0000 7efefd200700 1 -- 172.21.15.33:0/2351521371 shutdown_connections 2024-05-03T18:48:22.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:22.349+0000 7efefd200700 1 --2- 172.21.15.33:0/2351521371 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7efedc0b29f0 0x7efedc0b4eb0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:48:22.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:22.349+0000 7efefd200700 1 --2- 172.21.15.33:0/2351521371 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efef810fb60 0x7efef81a4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:48:22.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:22.349+0000 7efefd200700 1 --2- 172.21.15.33:0/2351521371 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efef8109ad0 0x7efef81a00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:48:22.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:22.349+0000 7efefd200700 1 --2- 172.21.15.33:0/2351521371 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efef810a490 0x7efef81a0600 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:48:22.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:22.349+0000 7efefd200700 1 -- 172.21.15.33:0/2351521371 >> 172.21.15.33:0/2351521371 conn(0x7efef8076c70 msgr2=0x7efef8077290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:48:22.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:22.350+0000 7efefd200700 1 -- 172.21.15.33:0/2351521371 shutdown_connections 2024-05-03T18:48:22.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:22.350+0000 7efefd200700 1 -- 172.21.15.33:0/2351521371 wait complete. 2024-05-03T18:48:22.352 INFO:teuthology.orchestra.run.smithi033.stderr:listed 17 entries 2024-05-03T18:48:23.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:48:22 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/2351521371' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-03T18:48:23.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:48:22 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/2351521371' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-03T18:48:23.183 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:48:22 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/2351521371' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-03T18:48:23.261 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi033.front.sepia.ceph.com... 2024-05-03T18:48:23.261 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T18:48:23.262 DEBUG:teuthology.orchestra.run.smithi033:> sudo fusermount -u /tmp/cephfs 2024-05-03T18:48:23.387 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi033.stderr:ceph-fuse[86974]: fuse finished with error 0 and tester_r 0 2024-05-03T18:48:23.638 INFO:teuthology.orchestra.run:waiting for 300 2024-05-03T18:48:23.986 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:48:23 smithi033 ceph-mon[28330]: pgmap v670: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-05-03T18:48:24.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:48:23 smithi099 ceph-mon[31658]: pgmap v670: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-05-03T18:48:24.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:48:23 smithi150 ceph-mon[32492]: pgmap v670: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-05-03T18:48:26.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:48:25 smithi099 ceph-mon[31658]: pgmap v671: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 8.5 KiB/s wr, 2 op/s 2024-05-03T18:48:26.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:48:25 smithi150 ceph-mon[32492]: pgmap v671: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 8.5 KiB/s wr, 2 op/s 2024-05-03T18:48:26.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:48:25 smithi033 ceph-mon[28330]: pgmap v671: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 8.5 KiB/s wr, 2 op/s 2024-05-03T18:48:28.038 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:48:27 smithi033 ceph-mon[28330]: pgmap v672: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 8.2 KiB/s wr, 1 op/s 2024-05-03T18:48:28.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:48:27 smithi099 ceph-mon[31658]: pgmap v672: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 8.2 KiB/s wr, 1 op/s 2024-05-03T18:48:28.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:48:27 smithi150 ceph-mon[32492]: pgmap v672: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 8.2 KiB/s wr, 1 op/s 2024-05-03T18:48:29.740 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi033.front.sepia.ceph.com 2024-05-03T18:48:29.741 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T18:48:29.741 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest && exec rmdir -- /tmp/cephfs) 2024-05-03T18:48:29.820 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T18:48:29.820 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2024-05-03T18:48:29.918 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--tmp-cephfs (id: 0) 2024-05-03T18:48:29.919 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T18:48:29.919 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns delete ceph-ns--tmp-cephfs 2024-05-03T18:48:29.998 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T18:48:29.998 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link delete ceph-brx 2024-05-03T18:48:30.015 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:48:29 smithi033 ceph-mon[28330]: pgmap v673: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 8.2 KiB/s wr, 2 op/s 2024-05-03T18:48:30.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:48:29 smithi099 ceph-mon[31658]: pgmap v673: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 8.2 KiB/s wr, 2 op/s 2024-05-03T18:48:30.103 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-05-03T18:48:30.115 INFO:tasks.cephadm:Teardown begin 2024-05-03T18:48:30.115 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-03T18:48:30.116 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T18:48:30.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:48:29 smithi150 ceph-mon[32492]: pgmap v673: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 8.2 KiB/s wr, 2 op/s 2024-05-03T18:48:30.575 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:48:31.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:48:31 smithi033 ceph-mon[28330]: pgmap v674: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 8.2 KiB/s wr, 2 op/s 2024-05-03T18:48:32.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:48:31 smithi099 ceph-mon[31658]: pgmap v674: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 8.2 KiB/s wr, 2 op/s 2024-05-03T18:48:32.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:48:31 smithi150 ceph-mon[32492]: pgmap v674: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 8.2 KiB/s wr, 2 op/s 2024-05-03T18:48:32.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:32.971+0000 7fc696400700 1 -- 172.21.15.33:0/4172309411 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6900756d0 msgr2=0x7fc690075ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:48:32.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:32.971+0000 7fc696400700 1 --2- 172.21.15.33:0/4172309411 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6900756d0 0x7fc690075ab0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fc6800098f0 tx=0x7fc680009c00 comp rx=0 tx=0).stop 2024-05-03T18:48:32.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:32.971+0000 7fc696400700 1 -- 172.21.15.33:0/4172309411 shutdown_connections 2024-05-03T18:48:32.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:32.971+0000 7fc696400700 1 --2- 172.21.15.33:0/4172309411 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc69010fea0 0x7fc690112290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:48:32.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:32.971+0000 7fc696400700 1 --2- 172.21.15.33:0/4172309411 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6900756d0 0x7fc690075ab0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:48:32.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:32.971+0000 7fc696400700 1 --2- 172.21.15.33:0/4172309411 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc690075ff0 0x7fc69010f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:48:32.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:32.971+0000 7fc696400700 1 -- 172.21.15.33:0/4172309411 >> 172.21.15.33:0/4172309411 conn(0x7fc69007a600 msgr2=0x7fc69007aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:48:32.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:32.971+0000 7fc696400700 1 -- 172.21.15.33:0/4172309411 shutdown_connections 2024-05-03T18:48:32.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:32.971+0000 7fc696400700 1 -- 172.21.15.33:0/4172309411 wait complete. 2024-05-03T18:48:32.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:32.972+0000 7fc696400700 1 Processor -- start 2024-05-03T18:48:32.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:32.972+0000 7fc696400700 1 -- start start 2024-05-03T18:48:32.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:32.973+0000 7fc696400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6900756d0 0x7fc6901a1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:48:32.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:32.973+0000 7fc696400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc690075ff0 0x7fc6901a1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:48:32.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:32.973+0000 7fc696400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc69010fea0 0x7fc69019bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:48:32.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:32.973+0000 7fc696400700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc690115720 con 0x7fc6900756d0 2024-05-03T18:48:32.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:32.973+0000 7fc696400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc6901155a0 con 0x7fc69010fea0 2024-05-03T18:48:32.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:32.973+0000 7fc696400700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc6901158a0 con 0x7fc690075ff0 2024-05-03T18:48:32.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:32.973+0000 7fc68f400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6900756d0 0x7fc6901a1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:48:32.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:32.973+0000 7fc68f400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6900756d0 0x7fc6901a1a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53068/0 (socket says 172.21.15.33:53068) 2024-05-03T18:48:32.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:32.973+0000 7fc68f400700 1 -- 172.21.15.33:0/2435371188 learned_addr learned my addr 172.21.15.33:0/2435371188 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:48:32.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:32.974+0000 7fc687e00700 1 --2- 172.21.15.33:0/2435371188 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc690075ff0 0x7fc6901a1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:48:32.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:32.974+0000 7fc68fe00700 1 --2- 172.21.15.33:0/2435371188 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc69010fea0 0x7fc69019bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:48:32.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:32.974+0000 7fc68f400700 1 -- 172.21.15.33:0/2435371188 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc690075ff0 msgr2=0x7fc6901a1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:48:32.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:32.974+0000 7fc68f400700 1 --2- 172.21.15.33:0/2435371188 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc690075ff0 0x7fc6901a1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:48:32.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:32.974+0000 7fc68f400700 1 -- 172.21.15.33:0/2435371188 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc69010fea0 msgr2=0x7fc69019bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:48:32.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:32.974+0000 7fc68f400700 1 --2- 172.21.15.33:0/2435371188 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc69010fea0 0x7fc69019bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:48:32.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:32.974+0000 7fc68f400700 1 -- 172.21.15.33:0/2435371188 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc69019c410 con 0x7fc6900756d0 2024-05-03T18:48:32.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:32.974+0000 7fc68f400700 1 --2- 172.21.15.33:0/2435371188 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6900756d0 0x7fc6901a1a60 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fc680005950 tx=0x7fc680010fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:48:32.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:32.974+0000 7fc68cc00700 1 -- 172.21.15.33:0/2435371188 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc68000b790 con 0x7fc6900756d0 2024-05-03T18:48:32.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:32.975+0000 7fc696400700 1 -- 172.21.15.33:0/2435371188 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc69019c6a0 con 0x7fc6900756d0 2024-05-03T18:48:32.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:32.975+0000 7fc68cc00700 1 -- 172.21.15.33:0/2435371188 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc6800054d0 con 0x7fc6900756d0 2024-05-03T18:48:32.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:32.975+0000 7fc68cc00700 1 -- 172.21.15.33:0/2435371188 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc680003ae0 con 0x7fc6900756d0 2024-05-03T18:48:32.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:32.975+0000 7fc696400700 1 -- 172.21.15.33:0/2435371188 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6901a8810 con 0x7fc6900756d0 2024-05-03T18:48:32.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:32.976+0000 7fc696400700 1 -- 172.21.15.33:0/2435371188 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc690076dc0 con 0x7fc6900756d0 2024-05-03T18:48:32.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:32.978+0000 7fc68cc00700 1 -- 172.21.15.33:0/2435371188 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fc680003c80 con 0x7fc6900756d0 2024-05-03T18:48:32.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:32.980+0000 7fc68cc00700 1 --2- 172.21.15.33:0/2435371188 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fc6740b29f0 0x7fc6740b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:48:32.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:32.980+0000 7fc68cc00700 1 -- 172.21.15.33:0/2435371188 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 9518+0+0 (secure 0 0 0) 0x7fc6800e3c30 con 0x7fc6900756d0 2024-05-03T18:48:32.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:32.980+0000 7fc687e00700 1 --2- 172.21.15.33:0/2435371188 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fc6740b29f0 0x7fc6740b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:48:32.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:32.981+0000 7fc687e00700 1 --2- 172.21.15.33:0/2435371188 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fc6740b29f0 0x7fc6740b4eb0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc69019d280 tx=0x7fc678009310 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:48:32.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:32.982+0000 7fc68cc00700 1 -- 172.21.15.33:0/2435371188 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fc68005fd70 con 0x7fc6900756d0 2024-05-03T18:48:33.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:33.272+0000 7fc696400700 1 -- 172.21.15.33:0/2435371188 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc69002e510 con 0x7fc6900756d0 2024-05-03T18:48:33.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:33.274+0000 7fc68cc00700 1 -- 172.21.15.33:0/2435371188 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11801 (secure 0 0 0) 0x7fc680013030 con 0x7fc6900756d0 2024-05-03T18:48:33.275 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:48:33.276 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1327393503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1327393503},{"type":"v1","addr":"172.21.15.33:6835","nonce":1327393503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/2461472948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":2461472948},{"type":"v1","addr":"172.21.15.33:6837","nonce":2461472948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14559,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/442269281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":442269281},{"type":"v1","addr":"172.21.15.33:6839","nonce":442269281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14571,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/645571756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":645571756},{"type":"v1","addr":"172.21.15.33:6841","nonce":645571756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/203598909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":203598909},{"type":"v1","addr":"172.21.15.150:6833","nonce":203598909}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/4029842493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":4029842493},{"type":"v1","addr":"172.21.15.150:6835","nonce":4029842493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3463664423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3463664423},{"type":"v1","addr":"172.21.15.150:6837","nonce":3463664423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24481,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6839/4051648018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":4051648018},{"type":"v1","addr":"172.21.15.150:6839","nonce":4051648018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34418,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1865105479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1865105479},{"type":"v1","addr":"172.21.15.99:6833","nonce":1865105479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34445,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/1206393914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1206393914},{"type":"v1","addr":"172.21.15.99:6835","nonce":1206393914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-03T18:22:28.837091+0000","modified":"2024-05-03T18:23:49.758398+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34508},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34472":{"gid":34472,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6837/3360970342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3360970342},{"type":"v1","addr":"172.21.15.99:6837","nonce":3360970342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34508":{"gid":34508,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/919907308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":919907308},{"type":"v1","addr":"172.21.15.99:6839","nonce":919907308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34508,"qdb_cluster":[34508]},"id":1}]} 2024-05-03T18:48:33.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:33.277+0000 7fc696400700 1 -- 172.21.15.33:0/2435371188 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fc6740b29f0 msgr2=0x7fc6740b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:48:33.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:33.277+0000 7fc696400700 1 --2- 172.21.15.33:0/2435371188 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fc6740b29f0 0x7fc6740b4eb0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc69019d280 tx=0x7fc678009310 comp rx=0 tx=0).stop 2024-05-03T18:48:33.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:33.277+0000 7fc696400700 1 -- 172.21.15.33:0/2435371188 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6900756d0 msgr2=0x7fc6901a1a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:48:33.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:33.277+0000 7fc696400700 1 --2- 172.21.15.33:0/2435371188 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6900756d0 0x7fc6901a1a60 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fc680005950 tx=0x7fc680010fe0 comp rx=0 tx=0).stop 2024-05-03T18:48:33.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:33.277+0000 7fc696400700 1 -- 172.21.15.33:0/2435371188 shutdown_connections 2024-05-03T18:48:33.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:33.277+0000 7fc696400700 1 --2- 172.21.15.33:0/2435371188 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fc6740b29f0 0x7fc6740b4eb0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:48:33.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:33.277+0000 7fc696400700 1 --2- 172.21.15.33:0/2435371188 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc69010fea0 0x7fc69019bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:48:33.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:33.277+0000 7fc696400700 1 --2- 172.21.15.33:0/2435371188 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6900756d0 0x7fc6901a1a60 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:48:33.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:33.277+0000 7fc696400700 1 --2- 172.21.15.33:0/2435371188 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc690075ff0 0x7fc6901a1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:48:33.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:33.277+0000 7fc696400700 1 -- 172.21.15.33:0/2435371188 >> 172.21.15.33:0/2435371188 conn(0x7fc69007a600 msgr2=0x7fc6901007d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:48:33.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:33.278+0000 7fc696400700 1 -- 172.21.15.33:0/2435371188 shutdown_connections 2024-05-03T18:48:33.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:33.278+0000 7fc696400700 1 -- 172.21.15.33:0/2435371188 wait complete. 2024-05-03T18:48:33.280 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 21 2024-05-03T18:48:33.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:48:33 smithi033 ceph-mon[28330]: pgmap v675: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 8.2 KiB/s wr, 2 op/s 2024-05-03T18:48:33.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:48:33 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/2435371188' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:48:34.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:48:33 smithi099 ceph-mon[31658]: pgmap v675: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 8.2 KiB/s wr, 2 op/s 2024-05-03T18:48:34.047 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:48:33 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/2435371188' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:48:34.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:48:33 smithi150 ceph-mon[32492]: pgmap v675: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 8.2 KiB/s wr, 2 op/s 2024-05-03T18:48:34.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:48:33 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/2435371188' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:48:34.244 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T18:48:34.732 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:48:36.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:48:35 smithi099 ceph-mon[31658]: pgmap v676: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 3 op/s 2024-05-03T18:48:36.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:48:35 smithi150 ceph-mon[32492]: pgmap v676: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 3 op/s 2024-05-03T18:48:36.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:48:35 smithi033 ceph-mon[28330]: pgmap v676: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 3 op/s 2024-05-03T18:48:37.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:37.168+0000 7fcedd600700 1 -- 172.21.15.33:0/1247790966 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fced8103610 msgr2=0x7fced81039f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:48:37.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:37.168+0000 7fcedd600700 1 --2- 172.21.15.33:0/1247790966 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fced8103610 0x7fced81039f0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fcec80099c0 tx=0x7fcec8009cd0 comp rx=0 tx=0).stop 2024-05-03T18:48:37.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:37.169+0000 7fcedd600700 1 -- 172.21.15.33:0/1247790966 shutdown_connections 2024-05-03T18:48:37.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:37.169+0000 7fcedd600700 1 --2- 172.21.15.33:0/1247790966 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fced81086f0 0x7fced81102c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:48:37.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:37.169+0000 7fcedd600700 1 --2- 172.21.15.33:0/1247790966 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fced8103610 0x7fced81039f0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:48:37.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:37.169+0000 7fcedd600700 1 --2- 172.21.15.33:0/1247790966 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fced8103fc0 0x7fced8108010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:48:37.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:37.169+0000 7fcedd600700 1 -- 172.21.15.33:0/1247790966 >> 172.21.15.33:0/1247790966 conn(0x7fced80feec0 msgr2=0x7fced81012e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:48:37.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:37.169+0000 7fcedd600700 1 -- 172.21.15.33:0/1247790966 shutdown_connections 2024-05-03T18:48:37.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:37.169+0000 7fcedd600700 1 -- 172.21.15.33:0/1247790966 wait complete. 2024-05-03T18:48:37.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:37.170+0000 7fcedd600700 1 Processor -- start 2024-05-03T18:48:37.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:37.170+0000 7fcedd600700 1 -- start start 2024-05-03T18:48:37.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:37.170+0000 7fcedd600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fced8103610 0x7fced819f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:48:37.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:37.170+0000 7fcedd600700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fced8103fc0 0x7fced819fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:48:37.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:37.170+0000 7fcedd600700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fced81086f0 0x7fced8199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:48:37.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:37.170+0000 7fcedd600700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fced81134b0 con 0x7fced8103fc0 2024-05-03T18:48:37.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:37.170+0000 7fcedd600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fced8113330 con 0x7fced8103610 2024-05-03T18:48:37.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:37.170+0000 7fcedd600700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fced8113630 con 0x7fced81086f0 2024-05-03T18:48:37.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:37.171+0000 7fced7400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fced8103fc0 0x7fced819fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:48:37.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:37.171+0000 7fced7400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fced8103fc0 0x7fced819fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37340/0 (socket says 172.21.15.33:37340) 2024-05-03T18:48:37.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:37.171+0000 7fced7400700 1 -- 172.21.15.33:0/254380485 learned_addr learned my addr 172.21.15.33:0/254380485 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:48:37.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:37.171+0000 7fced7e00700 1 --2- 172.21.15.33:0/254380485 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fced8103610 0x7fced819f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:48:37.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:37.171+0000 7fcedcc00700 1 --2- 172.21.15.33:0/254380485 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fced81086f0 0x7fced8199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:48:37.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:37.171+0000 7fced7400700 1 -- 172.21.15.33:0/254380485 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fced81086f0 msgr2=0x7fced8199850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:48:37.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:37.171+0000 7fced7400700 1 --2- 172.21.15.33:0/254380485 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fced81086f0 0x7fced8199850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:48:37.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:37.171+0000 7fced7400700 1 -- 172.21.15.33:0/254380485 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fced8103610 msgr2=0x7fced819f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:48:37.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:37.171+0000 7fced7400700 1 --2- 172.21.15.33:0/254380485 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fced8103610 0x7fced819f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:48:37.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:37.171+0000 7fced7400700 1 -- 172.21.15.33:0/254380485 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fced819a030 con 0x7fced8103fc0 2024-05-03T18:48:37.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:37.172+0000 7fced7400700 1 --2- 172.21.15.33:0/254380485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fced8103fc0 0x7fced819fbc0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fcec400d980 tx=0x7fcec400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:48:37.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:37.172+0000 7fced4c00700 1 -- 172.21.15.33:0/254380485 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec4017070 con 0x7fced8103fc0 2024-05-03T18:48:37.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:37.172+0000 7fced4c00700 1 -- 172.21.15.33:0/254380485 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcec4010460 con 0x7fced8103fc0 2024-05-03T18:48:37.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:37.172+0000 7fced4c00700 1 -- 172.21.15.33:0/254380485 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec400de60 con 0x7fced8103fc0 2024-05-03T18:48:37.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:37.172+0000 7fcedd600700 1 -- 172.21.15.33:0/254380485 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fced819a320 con 0x7fced8103fc0 2024-05-03T18:48:37.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:37.172+0000 7fcedd600700 1 -- 172.21.15.33:0/254380485 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fced81a64e0 con 0x7fced8103fc0 2024-05-03T18:48:37.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:37.173+0000 7fcedd600700 1 -- 172.21.15.33:0/254380485 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fced8104af0 con 0x7fced8103fc0 2024-05-03T18:48:37.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:37.176+0000 7fced4c00700 1 -- 172.21.15.33:0/254380485 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fcec4010610 con 0x7fced8103fc0 2024-05-03T18:48:37.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:37.177+0000 7fced4c00700 1 --2- 172.21.15.33:0/254380485 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fcebc0b2a70 0x7fcebc0b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:48:37.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:37.178+0000 7fced4c00700 1 -- 172.21.15.33:0/254380485 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 9518+0+0 (secure 0 0 0) 0x7fcec40e24f0 con 0x7fced8103fc0 2024-05-03T18:48:37.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:37.178+0000 7fced7e00700 1 --2- 172.21.15.33:0/254380485 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fcebc0b2a70 0x7fcebc0b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:48:37.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:37.179+0000 7fced7e00700 1 --2- 172.21.15.33:0/254380485 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fcebc0b2a70 0x7fcebc0b4f30 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fcec8000c00 tx=0x7fcec801a040 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:48:37.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:37.180+0000 7fced4c00700 1 -- 172.21.15.33:0/254380485 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fcec40a9b10 con 0x7fced8103fc0 2024-05-03T18:48:37.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:37.470+0000 7fcedd600700 1 -- 172.21.15.33:0/254380485 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fced802e510 con 0x7fced8103fc0 2024-05-03T18:48:37.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:37.472+0000 7fced4c00700 1 -- 172.21.15.33:0/254380485 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11801 (secure 0 0 0) 0x7fcec40a9410 con 0x7fced8103fc0 2024-05-03T18:48:37.474 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:48:37.474 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/1327393503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":1327393503},{"type":"v1","addr":"172.21.15.33:6835","nonce":1327393503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/2461472948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":2461472948},{"type":"v1","addr":"172.21.15.33:6837","nonce":2461472948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14559,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/442269281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":442269281},{"type":"v1","addr":"172.21.15.33:6839","nonce":442269281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14571,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/645571756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":645571756},{"type":"v1","addr":"172.21.15.33:6841","nonce":645571756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/203598909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":203598909},{"type":"v1","addr":"172.21.15.150:6833","nonce":203598909}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/4029842493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":4029842493},{"type":"v1","addr":"172.21.15.150:6835","nonce":4029842493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3463664423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3463664423},{"type":"v1","addr":"172.21.15.150:6837","nonce":3463664423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24481,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6839/4051648018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":4051648018},{"type":"v1","addr":"172.21.15.150:6839","nonce":4051648018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34418,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1865105479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1865105479},{"type":"v1","addr":"172.21.15.99:6833","nonce":1865105479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34445,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/1206393914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1206393914},{"type":"v1","addr":"172.21.15.99:6835","nonce":1206393914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-03T18:22:28.837091+0000","modified":"2024-05-03T18:23:49.758398+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34508},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34472":{"gid":34472,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6837/3360970342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3360970342},{"type":"v1","addr":"172.21.15.99:6837","nonce":3360970342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34508":{"gid":34508,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/919907308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":919907308},{"type":"v1","addr":"172.21.15.99:6839","nonce":919907308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34508,"qdb_cluster":[34508]},"id":1}]} 2024-05-03T18:48:37.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:37.475+0000 7fcedd600700 1 -- 172.21.15.33:0/254380485 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fcebc0b2a70 msgr2=0x7fcebc0b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:48:37.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:37.475+0000 7fcedd600700 1 --2- 172.21.15.33:0/254380485 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fcebc0b2a70 0x7fcebc0b4f30 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fcec8000c00 tx=0x7fcec801a040 comp rx=0 tx=0).stop 2024-05-03T18:48:37.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:37.475+0000 7fcedd600700 1 -- 172.21.15.33:0/254380485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fced8103fc0 msgr2=0x7fced819fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:48:37.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:37.475+0000 7fcedd600700 1 --2- 172.21.15.33:0/254380485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fced8103fc0 0x7fced819fbc0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fcec400d980 tx=0x7fcec400dd40 comp rx=0 tx=0).stop 2024-05-03T18:48:37.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:37.475+0000 7fcedd600700 1 -- 172.21.15.33:0/254380485 shutdown_connections 2024-05-03T18:48:37.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:37.475+0000 7fcedd600700 1 --2- 172.21.15.33:0/254380485 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fced81086f0 0x7fced8199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:48:37.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:37.475+0000 7fcedd600700 1 --2- 172.21.15.33:0/254380485 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fcebc0b2a70 0x7fcebc0b4f30 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:48:37.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:37.475+0000 7fcedd600700 1 --2- 172.21.15.33:0/254380485 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fced8103610 0x7fced819f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:48:37.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:37.475+0000 7fcedd600700 1 --2- 172.21.15.33:0/254380485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fced8103fc0 0x7fced819fbc0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:48:37.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:37.475+0000 7fcedd600700 1 -- 172.21.15.33:0/254380485 >> 172.21.15.33:0/254380485 conn(0x7fced80feec0 msgr2=0x7fced8100220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:48:37.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:37.476+0000 7fcedd600700 1 -- 172.21.15.33:0/254380485 shutdown_connections 2024-05-03T18:48:37.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:37.476+0000 7fcedd600700 1 -- 172.21.15.33:0/254380485 wait complete. 2024-05-03T18:48:37.479 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 21 2024-05-03T18:48:38.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:48:37 smithi099 ceph-mon[31658]: pgmap v677: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.1 KiB/s wr, 2 op/s 2024-05-03T18:48:38.047 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:48:37 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/254380485' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:48:38.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:48:37 smithi150 ceph-mon[32492]: pgmap v677: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.1 KiB/s wr, 2 op/s 2024-05-03T18:48:38.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:48:37 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/254380485' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:48:38.139 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:48:37 smithi033 ceph-mon[28330]: pgmap v677: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.1 KiB/s wr, 2 op/s 2024-05-03T18:48:38.139 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:48:37 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/254380485' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:48:38.216 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-03T18:48:38.687 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:48:39.879 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:48:39 smithi033 ceph-mon[28330]: pgmap v678: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.1 KiB/s wr, 2 op/s 2024-05-03T18:48:40.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:48:39 smithi099 ceph-mon[31658]: pgmap v678: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.1 KiB/s wr, 2 op/s 2024-05-03T18:48:40.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:48:39 smithi150 ceph-mon[32492]: pgmap v678: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.1 KiB/s wr, 2 op/s 2024-05-03T18:48:41.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:41.068+0000 7f7a21600700 1 -- 172.21.15.33:0/690281677 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a1c06c590 msgr2=0x7f7a1c110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:48:41.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:41.068+0000 7f7a21600700 1 --2- 172.21.15.33:0/690281677 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a1c06c590 0x7f7a1c110330 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f7a1400b0a0 tx=0x7f7a1400b3b0 comp rx=0 tx=0).stop 2024-05-03T18:48:41.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:41.068+0000 7f7a21600700 1 -- 172.21.15.33:0/690281677 shutdown_connections 2024-05-03T18:48:41.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:41.068+0000 7f7a21600700 1 --2- 172.21.15.33:0/690281677 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a1c06c590 0x7f7a1c110330 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:48:41.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:41.068+0000 7f7a21600700 1 --2- 172.21.15.33:0/690281677 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7a1c109510 0x7f7a1c1098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:48:41.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:41.068+0000 7f7a21600700 1 --2- 172.21.15.33:0/690281677 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7a1c06bbd0 0x7f7a1c06c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:48:41.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:41.068+0000 7f7a21600700 1 -- 172.21.15.33:0/690281677 >> 172.21.15.33:0/690281677 conn(0x7f7a1c079350 msgr2=0x7f7a1c079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:48:41.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:41.068+0000 7f7a21600700 1 -- 172.21.15.33:0/690281677 shutdown_connections 2024-05-03T18:48:41.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:41.068+0000 7f7a21600700 1 -- 172.21.15.33:0/690281677 wait complete. 2024-05-03T18:48:41.072 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:48:40 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:48:41.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:41.069+0000 7f7a21600700 1 Processor -- start 2024-05-03T18:48:41.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:41.070+0000 7f7a21600700 1 -- start start 2024-05-03T18:48:41.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:41.070+0000 7f7a21600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7a1c06bbd0 0x7f7a1c1a18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:48:41.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:41.070+0000 7f7a21600700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7a1c06c590 0x7f7a1c1a1e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:48:41.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:41.070+0000 7f7a21600700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a1c109510 0x7f7a1c19baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:48:41.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:41.070+0000 7f7a21600700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7a1c113540 con 0x7f7a1c109510 2024-05-03T18:48:41.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:41.070+0000 7f7a21600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7a1c1133c0 con 0x7f7a1c06bbd0 2024-05-03T18:48:41.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:41.070+0000 7f7a21600700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f7a1c1136c0 con 0x7f7a1c06c590 2024-05-03T18:48:41.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:41.071+0000 7f7a1a000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7a1c06c590 0x7f7a1c1a1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:48:41.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:41.071+0000 7f7a1a000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7a1c06c590 0x7f7a1c1a1e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.33:34880/0 (socket says 172.21.15.33:34880) 2024-05-03T18:48:41.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:41.071+0000 7f7a1a000700 1 -- 172.21.15.33:0/379860323 learned_addr learned my addr 172.21.15.33:0/379860323 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:48:41.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:41.071+0000 7f7a1aa00700 1 --2- 172.21.15.33:0/379860323 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7a1c06bbd0 0x7f7a1c1a18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:48:41.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:41.071+0000 7f7a1b400700 1 --2- 172.21.15.33:0/379860323 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a1c109510 0x7f7a1c19baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:48:41.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:41.071+0000 7f7a1a000700 1 -- 172.21.15.33:0/379860323 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7a1c06bbd0 msgr2=0x7f7a1c1a18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:48:41.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:41.071+0000 7f7a1a000700 1 --2- 172.21.15.33:0/379860323 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7a1c06bbd0 0x7f7a1c1a18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:48:41.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:41.071+0000 7f7a1a000700 1 -- 172.21.15.33:0/379860323 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a1c109510 msgr2=0x7f7a1c19baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:48:41.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:41.071+0000 7f7a1a000700 1 --2- 172.21.15.33:0/379860323 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a1c109510 0x7f7a1c19baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:48:41.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:41.071+0000 7f7a1a000700 1 -- 172.21.15.33:0/379860323 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a1c19c200 con 0x7f7a1c06c590 2024-05-03T18:48:41.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:41.072+0000 7f7a1a000700 1 --2- 172.21.15.33:0/379860323 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7a1c06c590 0x7f7a1c1a1e20 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f7a0800e9c0 tx=0x7f7a0800ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:48:41.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:41.072+0000 7f7a13400700 1 -- 172.21.15.33:0/379860323 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a0800cbd0 con 0x7f7a1c06c590 2024-05-03T18:48:41.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:41.072+0000 7f7a21600700 1 -- 172.21.15.33:0/379860323 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a1c19c490 con 0x7f7a1c06c590 2024-05-03T18:48:41.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:41.072+0000 7f7a13400700 1 -- 172.21.15.33:0/379860323 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7a0800ee60 con 0x7f7a1c06c590 2024-05-03T18:48:41.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:41.072+0000 7f7a13400700 1 -- 172.21.15.33:0/379860323 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a08005440 con 0x7f7a1c06c590 2024-05-03T18:48:41.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:41.073+0000 7f7a21600700 1 -- 172.21.15.33:0/379860323 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a1c1a87b0 con 0x7f7a1c06c590 2024-05-03T18:48:41.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:41.074+0000 7f7a21600700 1 -- 172.21.15.33:0/379860323 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a1c06a190 con 0x7f7a1c06c590 2024-05-03T18:48:41.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:41.078+0000 7f7a13400700 1 -- 172.21.15.33:0/379860323 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f7a08004750 con 0x7f7a1c06c590 2024-05-03T18:48:41.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:41.079+0000 7f7a13400700 1 --2- 172.21.15.33:0/379860323 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f7a000b2a70 0x7f7a000b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:48:41.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:41.080+0000 7f7a13400700 1 -- 172.21.15.33:0/379860323 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 9518+0+0 (secure 0 0 0) 0x7f7a080e2ea0 con 0x7f7a1c06c590 2024-05-03T18:48:41.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:41.080+0000 7f7a1aa00700 1 --2- 172.21.15.33:0/379860323 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f7a000b2a70 0x7f7a000b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:48:41.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:41.081+0000 7f7a1aa00700 1 --2- 172.21.15.33:0/379860323 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f7a000b2a70 0x7f7a000b4f30 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c19cbe0 tx=0x7f7a0c0073d0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:48:41.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:41.095+0000 7f7a13400700 1 -- 172.21.15.33:0/379860323 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f7a08054d70 con 0x7f7a1c06c590 2024-05-03T18:48:41.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:48:40 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:48:41.296 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:48:40 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:48:41.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:41.354+0000 7f7a21600700 1 -- 172.21.15.33:0/379860323 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7a1c1a89e0 con 0x7f7a1c06c590 2024-05-03T18:48:41.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:41.359+0000 7f7a13400700 1 -- 172.21.15.33:0/379860323 <== mon.1 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v122) ==== 74+0+25275 (secure 0 0 0) 0x7f7a080aa440 con 0x7f7a1c06c590 2024-05-03T18:48:41.363 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-03T18:48:41.363 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":122,"fsid":"5703bba4-0978-11ef-bc97-c7b262605968","created":"2024-05-03T18:11:00.899879+0000","modified":"2024-05-03T18:48:12.680310+0000","last_up_change":"2024-05-03T18:21:00.390173+0000","last_in_change":"2024-05-03T18:20:34.103727+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-03T18:15:37.297738+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-03T18:22:19.875368+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-03T18:22:24.432692+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":37,"snap_epoch":121,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-03T18:22:38.722443+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":37,"snap_epoch":121,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"fdcc4e66-4ba0-4378-a9a4-7d907cd00b4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":637686669},{"type":"v1","addr":"172.21.15.33:6803","nonce":637686669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":637686669},{"type":"v1","addr":"172.21.15.33:6805","nonce":637686669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":637686669},{"type":"v1","addr":"172.21.15.33:6809","nonce":637686669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":637686669},{"type":"v1","addr":"172.21.15.33:6807","nonce":637686669}]},"public_addr":"172.21.15.33:6803/637686669","cluster_addr":"172.21.15.33:6805/637686669","heartbeat_back_addr":"172.21.15.33:6809/637686669","heartbeat_front_addr":"172.21.15.33:6807/637686669","state":["exists","up"]},{"osd":1,"uuid":"cf78cad3-fc24-43f4-8784-21cabea7277c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":1536122841},{"type":"v1","addr":"172.21.15.99:6801","nonce":1536122841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":1536122841},{"type":"v1","addr":"172.21.15.99:6803","nonce":1536122841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":1536122841},{"type":"v1","addr":"172.21.15.99:6807","nonce":1536122841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":1536122841},{"type":"v1","addr":"172.21.15.99:6805","nonce":1536122841}]},"public_addr":"172.21.15.99:6801/1536122841","cluster_addr":"172.21.15.99:6803/1536122841","heartbeat_back_addr":"172.21.15.99:6807/1536122841","heartbeat_front_addr":"172.21.15.99:6805/1536122841","state":["exists","up"]},{"osd":2,"uuid":"59f066d7-626b-410a-8d67-daca6fbbd3ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":1995659622},{"type":"v1","addr":"172.21.15.150:6801","nonce":1995659622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":1995659622},{"type":"v1","addr":"172.21.15.150:6803","nonce":1995659622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":1995659622},{"type":"v1","addr":"172.21.15.150:6807","nonce":1995659622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":1995659622},{"type":"v1","addr":"172.21.15.150:6805","nonce":1995659622}]},"public_addr":"172.21.15.150:6801/1995659622","cluster_addr":"172.21.15.150:6803/1995659622","heartbeat_back_addr":"172.21.15.150:6807/1995659622","heartbeat_front_addr":"172.21.15.150:6805/1995659622","state":["exists","up"]},{"osd":3,"uuid":"d88ed022-4dcb-45fc-9fc6-41ef7c4bf202","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":3814942724},{"type":"v1","addr":"172.21.15.33:6811","nonce":3814942724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":3814942724},{"type":"v1","addr":"172.21.15.33:6813","nonce":3814942724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":3814942724},{"type":"v1","addr":"172.21.15.33:6817","nonce":3814942724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":3814942724},{"type":"v1","addr":"172.21.15.33:6815","nonce":3814942724}]},"public_addr":"172.21.15.33:6811/3814942724","cluster_addr":"172.21.15.33:6813/3814942724","heartbeat_back_addr":"172.21.15.33:6817/3814942724","heartbeat_front_addr":"172.21.15.33:6815/3814942724","state":["exists","up"]},{"osd":4,"uuid":"1235e1a6-ff6e-4668-ac1a-073c2a3b7c98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":29081029},{"type":"v1","addr":"172.21.15.99:6809","nonce":29081029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":29081029},{"type":"v1","addr":"172.21.15.99:6811","nonce":29081029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":29081029},{"type":"v1","addr":"172.21.15.99:6815","nonce":29081029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":29081029},{"type":"v1","addr":"172.21.15.99:6813","nonce":29081029}]},"public_addr":"172.21.15.99:6809/29081029","cluster_addr":"172.21.15.99:6811/29081029","heartbeat_back_addr":"172.21.15.99:6815/29081029","heartbeat_front_addr":"172.21.15.99:6813/29081029","state":["exists","up"]},{"osd":5,"uuid":"4b84bff8-d6d7-4258-a0ee-53bc19802585","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":3172189533},{"type":"v1","addr":"172.21.15.150:6809","nonce":3172189533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3172189533},{"type":"v1","addr":"172.21.15.150:6811","nonce":3172189533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3172189533},{"type":"v1","addr":"172.21.15.150:6815","nonce":3172189533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3172189533},{"type":"v1","addr":"172.21.15.150:6813","nonce":3172189533}]},"public_addr":"172.21.15.150:6809/3172189533","cluster_addr":"172.21.15.150:6811/3172189533","heartbeat_back_addr":"172.21.15.150:6815/3172189533","heartbeat_front_addr":"172.21.15.150:6813/3172189533","state":["exists","up"]},{"osd":6,"uuid":"51f5fbdd-3794-4c48-9f6c-478f0fa35ac5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":4234541418},{"type":"v1","addr":"172.21.15.33:6819","nonce":4234541418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":4234541418},{"type":"v1","addr":"172.21.15.33:6821","nonce":4234541418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":4234541418},{"type":"v1","addr":"172.21.15.33:6825","nonce":4234541418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":4234541418},{"type":"v1","addr":"172.21.15.33:6823","nonce":4234541418}]},"public_addr":"172.21.15.33:6819/4234541418","cluster_addr":"172.21.15.33:6821/4234541418","heartbeat_back_addr":"172.21.15.33:6825/4234541418","heartbeat_front_addr":"172.21.15.33:6823/4234541418","state":["exists","up"]},{"osd":7,"uuid":"982cc5b3-3163-4505-9bde-a27a8c289457","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":908673090},{"type":"v1","addr":"172.21.15.99:6817","nonce":908673090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":908673090},{"type":"v1","addr":"172.21.15.99:6819","nonce":908673090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":908673090},{"type":"v1","addr":"172.21.15.99:6823","nonce":908673090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":908673090},{"type":"v1","addr":"172.21.15.99:6821","nonce":908673090}]},"public_addr":"172.21.15.99:6817/908673090","cluster_addr":"172.21.15.99:6819/908673090","heartbeat_back_addr":"172.21.15.99:6823/908673090","heartbeat_front_addr":"172.21.15.99:6821/908673090","state":["exists","up"]},{"osd":8,"uuid":"84c9d037-eb84-4eb4-8aa2-e784b703d36c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":1308083602},{"type":"v1","addr":"172.21.15.150:6817","nonce":1308083602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":1308083602},{"type":"v1","addr":"172.21.15.150:6819","nonce":1308083602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":1308083602},{"type":"v1","addr":"172.21.15.150:6823","nonce":1308083602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":1308083602},{"type":"v1","addr":"172.21.15.150:6821","nonce":1308083602}]},"public_addr":"172.21.15.150:6817/1308083602","cluster_addr":"172.21.15.150:6819/1308083602","heartbeat_back_addr":"172.21.15.150:6823/1308083602","heartbeat_front_addr":"172.21.15.150:6821/1308083602","state":["exists","up"]},{"osd":9,"uuid":"a02e965f-0d8f-45e2-9490-337f7f6e3881","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":4133096905},{"type":"v1","addr":"172.21.15.33:6827","nonce":4133096905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":4133096905},{"type":"v1","addr":"172.21.15.33:6829","nonce":4133096905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4133096905},{"type":"v1","addr":"172.21.15.33:6833","nonce":4133096905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":4133096905},{"type":"v1","addr":"172.21.15.33:6831","nonce":4133096905}]},"public_addr":"172.21.15.33:6827/4133096905","cluster_addr":"172.21.15.33:6829/4133096905","heartbeat_back_addr":"172.21.15.33:6833/4133096905","heartbeat_front_addr":"172.21.15.33:6831/4133096905","state":["exists","up"]},{"osd":10,"uuid":"8b29bb96-76e8-45a0-ab19-8d082d755fe9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":935746136},{"type":"v1","addr":"172.21.15.99:6825","nonce":935746136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":935746136},{"type":"v1","addr":"172.21.15.99:6827","nonce":935746136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":935746136},{"type":"v1","addr":"172.21.15.99:6831","nonce":935746136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":935746136},{"type":"v1","addr":"172.21.15.99:6829","nonce":935746136}]},"public_addr":"172.21.15.99:6825/935746136","cluster_addr":"172.21.15.99:6827/935746136","heartbeat_back_addr":"172.21.15.99:6831/935746136","heartbeat_front_addr":"172.21.15.99:6829/935746136","state":["exists","up"]},{"osd":11,"uuid":"07a6a825-c902-4bfd-97cc-cc9074532af5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":2069349808},{"type":"v1","addr":"172.21.15.150:6825","nonce":2069349808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":2069349808},{"type":"v1","addr":"172.21.15.150:6827","nonce":2069349808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":2069349808},{"type":"v1","addr":"172.21.15.150:6831","nonce":2069349808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":2069349808},{"type":"v1","addr":"172.21.15.150:6829","nonce":2069349808}]},"public_addr":"172.21.15.150:6825/2069349808","cluster_addr":"172.21.15.150:6827/2069349808","heartbeat_back_addr":"172.21.15.150:6831/2069349808","heartbeat_front_addr":"172.21.15.150:6829/2069349808","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:14:27.454536+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:15:00.538570+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:15:34.125289+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:16:09.853870+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:16:44.928107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:17:19.644037+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:17:56.108443+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:18:31.731499+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:19:08.062358+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:19:45.204430+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:20:21.013710+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:20:57.653698+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11},{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/4003417322":"2024-05-04T18:27:11.722511+0000","172.21.15.33:6801/4007322071":"2024-05-04T18:11:44.239835+0000","172.21.15.33:6801/2830135294":"2024-05-04T18:12:19.127652+0000","172.21.15.33:0/3767665033":"2024-05-04T18:12:19.127652+0000","172.21.15.33:6800/4007322071":"2024-05-04T18:11:44.239835+0000","172.21.15.33:0/1614640996":"2024-05-04T18:11:44.239835+0000","172.21.15.33:0/1314704208":"2024-05-04T18:11:44.239835+0000","172.21.15.33:0/2014902144":"2024-05-04T18:11:44.239835+0000","172.21.15.33:0/965859483":"2024-05-04T18:12:19.127652+0000","172.21.15.33:6801/3642895501":"2024-05-04T18:27:11.722511+0000","172.21.15.33:6800/2830135294":"2024-05-04T18:12:19.127652+0000","172.21.15.33:0/791712778":"2024-05-04T18:27:11.722511+0000","172.21.15.33:0/3991947997":"2024-05-04T18:12:19.127652+0000","172.21.15.33:0/806149166":"2024-05-04T18:27:11.722511+0000","172.21.15.33:0/1300142939":"2024-05-04T18:27:11.722511+0000","172.21.15.33:6800/3642895501":"2024-05-04T18:27:11.722511+0000","172.21.15.33:0/757614565":"2024-05-04T18:27:11.722511+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":24,"length":1},{"begin":37,"length":1}]},{"pool":4,"snaps":[{"begin":24,"length":1},{"begin":37,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-03T18:48:41.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:41.362+0000 7f7a21600700 1 -- 172.21.15.33:0/379860323 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f7a000b2a70 msgr2=0x7f7a000b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:48:41.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:41.362+0000 7f7a21600700 1 --2- 172.21.15.33:0/379860323 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f7a000b2a70 0x7f7a000b4f30 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c19cbe0 tx=0x7f7a0c0073d0 comp rx=0 tx=0).stop 2024-05-03T18:48:41.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:41.362+0000 7f7a21600700 1 -- 172.21.15.33:0/379860323 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7a1c06c590 msgr2=0x7f7a1c1a1e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:48:41.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:41.362+0000 7f7a21600700 1 --2- 172.21.15.33:0/379860323 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7a1c06c590 0x7f7a1c1a1e20 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f7a0800e9c0 tx=0x7f7a0800ecd0 comp rx=0 tx=0).stop 2024-05-03T18:48:41.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:41.362+0000 7f7a21600700 1 -- 172.21.15.33:0/379860323 shutdown_connections 2024-05-03T18:48:41.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:41.362+0000 7f7a21600700 1 --2- 172.21.15.33:0/379860323 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a1c109510 0x7f7a1c19baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:48:41.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:41.362+0000 7f7a21600700 1 --2- 172.21.15.33:0/379860323 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f7a000b2a70 0x7f7a000b4f30 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:48:41.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:41.362+0000 7f7a21600700 1 --2- 172.21.15.33:0/379860323 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7a1c06bbd0 0x7f7a1c1a18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:48:41.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:41.362+0000 7f7a21600700 1 --2- 172.21.15.33:0/379860323 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7a1c06c590 0x7f7a1c1a1e20 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:48:41.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:41.362+0000 7f7a21600700 1 -- 172.21.15.33:0/379860323 >> 172.21.15.33:0/379860323 conn(0x7f7a1c079350 msgr2=0x7f7a1c10e660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:48:41.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:41.362+0000 7f7a21600700 1 -- 172.21.15.33:0/379860323 shutdown_connections 2024-05-03T18:48:41.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:41.362+0000 7f7a21600700 1 -- 172.21.15.33:0/379860323 wait complete. 2024-05-03T18:48:42.000 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:48:41 smithi033 ceph-mon[28330]: pgmap v679: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.0 KiB/s wr, 2 op/s 2024-05-03T18:48:42.000 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:48:41 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/379860323' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T18:48:42.133 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:48:41 smithi150 ceph-mon[32492]: pgmap v679: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.0 KiB/s wr, 2 op/s 2024-05-03T18:48:42.134 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:48:41 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/379860323' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T18:48:42.138 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:48:41 smithi099 ceph-mon[31658]: pgmap v679: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.0 KiB/s wr, 2 op/s 2024-05-03T18:48:42.138 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:48:41 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/379860323' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T18:48:42.793 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph fs fail cephfs 2024-05-03T18:48:43.235 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:48:44.547 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:48:44 smithi099 ceph-mon[31658]: pgmap v680: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 7.0 KiB/s wr, 1 op/s 2024-05-03T18:48:44.547 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:48:44 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:48:44.547 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:48:44 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:48:44.547 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:48:44 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:48:44.547 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:48:44 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:48:44.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:48:44 smithi150 ceph-mon[32492]: pgmap v680: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 7.0 KiB/s wr, 1 op/s 2024-05-03T18:48:44.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:48:44 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:48:44.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:48:44 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:48:44.634 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:48:44 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:48:44.634 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:48:44 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:48:44.735 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:48:44 smithi033 ceph-mon[28330]: pgmap v680: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 7.0 KiB/s wr, 1 op/s 2024-05-03T18:48:44.735 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:48:44 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:48:44.735 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:48:44 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:48:44.735 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:48:44 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:48:44.735 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:48:44 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:48:45.417 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:48:45 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:48:45.417 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:48:45 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:48:45.417 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:48:45 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:48:45.417 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:48:45 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:48:45.417 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:48:45 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:48:45.417 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:48:45 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:48:45.417 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:48:45 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:48:45.417 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:48:45 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:48:45.418 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:48:45 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:48:45.418 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:48:45 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:48:45.547 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:48:45 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:48:45.547 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:48:45 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:48:45.547 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:48:45 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:48:45.547 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:48:45 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:48:45.547 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:48:45 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:48:45.547 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:48:45 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:48:45.547 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:48:45 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:48:45.547 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:48:45 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:48:45.547 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:48:45 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:48:45.548 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:48:45 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:48:45.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:45.587+0000 7f5d38400700 1 -- 172.21.15.33:0/1605159029 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d3010a490 msgr2=0x7f5d301169a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:48:45.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:45.587+0000 7f5d38400700 1 --2- 172.21.15.33:0/1605159029 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d3010a490 0x7f5d301169a0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f5d2800b0a0 tx=0x7f5d2800b3b0 comp rx=0 tx=0).stop 2024-05-03T18:48:45.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:45.587+0000 7f5d38400700 1 -- 172.21.15.33:0/1605159029 shutdown_connections 2024-05-03T18:48:45.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:45.587+0000 7f5d38400700 1 --2- 172.21.15.33:0/1605159029 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d3010a490 0x7f5d301169a0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:48:45.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:45.587+0000 7f5d38400700 1 --2- 172.21.15.33:0/1605159029 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d3010fb60 0x7f5d3010ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:48:45.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:45.587+0000 7f5d38400700 1 --2- 172.21.15.33:0/1605159029 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d30109ad0 0x7f5d30109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:48:45.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:45.587+0000 7f5d38400700 1 -- 172.21.15.33:0/1605159029 >> 172.21.15.33:0/1605159029 conn(0x7f5d30076c70 msgr2=0x7f5d301082c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:48:45.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:45.588+0000 7f5d38400700 1 -- 172.21.15.33:0/1605159029 shutdown_connections 2024-05-03T18:48:45.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:45.588+0000 7f5d38400700 1 -- 172.21.15.33:0/1605159029 wait complete. 2024-05-03T18:48:45.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:45.589+0000 7f5d38400700 1 Processor -- start 2024-05-03T18:48:45.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:45.589+0000 7f5d38400700 1 -- start start 2024-05-03T18:48:45.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:45.589+0000 7f5d38400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d30109ad0 0x7f5d3019fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:48:45.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:45.589+0000 7f5d38400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d3010a490 0x7f5d301a0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:48:45.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:45.589+0000 7f5d38400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d3010fb60 0x7f5d301a48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:48:45.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:45.589+0000 7f5d38400700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5d30119bc0 con 0x7f5d30109ad0 2024-05-03T18:48:45.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:45.589+0000 7f5d38400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5d30119a40 con 0x7f5d3010a490 2024-05-03T18:48:45.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:45.589+0000 7f5d38400700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f5d30119d40 con 0x7f5d3010fb60 2024-05-03T18:48:45.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:45.589+0000 7f5d35a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d30109ad0 0x7f5d3019fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:48:45.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:45.589+0000 7f5d35a00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d30109ad0 0x7f5d3019fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37376/0 (socket says 172.21.15.33:37376) 2024-05-03T18:48:45.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:45.590+0000 7f5d35a00700 1 -- 172.21.15.33:0/1909603254 learned_addr learned my addr 172.21.15.33:0/1909603254 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:48:45.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:45.590+0000 7f5d35000700 1 --2- 172.21.15.33:0/1909603254 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d3010a490 0x7f5d301a0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:48:45.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:45.590+0000 7f5d36400700 1 --2- 172.21.15.33:0/1909603254 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d3010fb60 0x7f5d301a48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:48:45.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:45.590+0000 7f5d35a00700 1 -- 172.21.15.33:0/1909603254 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d3010fb60 msgr2=0x7f5d301a48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:48:45.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:45.590+0000 7f5d35a00700 1 --2- 172.21.15.33:0/1909603254 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d3010fb60 0x7f5d301a48c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:48:45.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:45.590+0000 7f5d35a00700 1 -- 172.21.15.33:0/1909603254 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d3010a490 msgr2=0x7f5d301a0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:48:45.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:45.590+0000 7f5d35a00700 1 --2- 172.21.15.33:0/1909603254 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d3010a490 0x7f5d301a0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:48:45.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:45.590+0000 7f5d35a00700 1 -- 172.21.15.33:0/1909603254 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d301a5000 con 0x7f5d30109ad0 2024-05-03T18:48:45.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:45.590+0000 7f5d35a00700 1 --2- 172.21.15.33:0/1909603254 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d30109ad0 0x7f5d3019fff0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f5d2400b530 tx=0x7f5d2400b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:48:45.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:45.590+0000 7f5d2e000700 1 -- 172.21.15.33:0/1909603254 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d24015020 con 0x7f5d30109ad0 2024-05-03T18:48:45.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:45.591+0000 7f5d2e000700 1 -- 172.21.15.33:0/1909603254 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5d24011cc0 con 0x7f5d30109ad0 2024-05-03T18:48:45.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:45.591+0000 7f5d38400700 1 -- 172.21.15.33:0/1909603254 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d301a5290 con 0x7f5d30109ad0 2024-05-03T18:48:45.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:45.591+0000 7f5d2e000700 1 -- 172.21.15.33:0/1909603254 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d24010d30 con 0x7f5d30109ad0 2024-05-03T18:48:45.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:45.591+0000 7f5d38400700 1 -- 172.21.15.33:0/1909603254 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d301acb70 con 0x7f5d30109ad0 2024-05-03T18:48:45.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:45.592+0000 7f5d38400700 1 -- 172.21.15.33:0/1909603254 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d3006a190 con 0x7f5d30109ad0 2024-05-03T18:48:45.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:45.594+0000 7f5d2e000700 1 -- 172.21.15.33:0/1909603254 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f5d24018020 con 0x7f5d30109ad0 2024-05-03T18:48:45.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:45.596+0000 7f5d2e000700 1 --2- 172.21.15.33:0/1909603254 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f5d180b29f0 0x7f5d180b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:48:45.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:45.596+0000 7f5d2e000700 1 -- 172.21.15.33:0/1909603254 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 9518+0+0 (secure 0 0 0) 0x7f5d240e21e0 con 0x7f5d30109ad0 2024-05-03T18:48:45.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:45.596+0000 7f5d35000700 1 --2- 172.21.15.33:0/1909603254 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f5d180b29f0 0x7f5d180b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:48:45.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:45.597+0000 7f5d35000700 1 --2- 172.21.15.33:0/1909603254 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f5d180b29f0 0x7f5d180b4eb0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c005ea0 tx=0x7f5d1c005e30 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:48:45.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:45.600+0000 7f5d2e000700 1 -- 172.21.15.33:0/1909603254 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f5d2401f020 con 0x7f5d30109ad0 2024-05-03T18:48:45.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:48:45 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:48:45.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:48:45 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:48:45.634 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:48:45 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:48:45.634 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:48:45 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:48:45.634 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:48:45 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:48:45.634 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:48:45 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:48:45.634 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:48:45 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:48:45.634 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:48:45 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:48:45.634 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:48:45 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:48:45.634 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:48:45 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:48:45.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:45.905+0000 7f5d38400700 1 -- 172.21.15.33:0/1909603254 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f5d3002e510 con 0x7f5d30109ad0 2024-05-03T18:48:45.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:45.924+0000 7f5d2e000700 1 -- 172.21.15.33:0/1909603254 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v22) ==== 161+0+0 (secure 0 0 0) 0x7f5d240a9890 con 0x7f5d30109ad0 2024-05-03T18:48:45.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:45.929+0000 7f5d38400700 1 -- 172.21.15.33:0/1909603254 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f5d180b29f0 msgr2=0x7f5d180b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:48:45.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:45.929+0000 7f5d38400700 1 --2- 172.21.15.33:0/1909603254 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f5d180b29f0 0x7f5d180b4eb0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c005ea0 tx=0x7f5d1c005e30 comp rx=0 tx=0).stop 2024-05-03T18:48:45.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:45.929+0000 7f5d38400700 1 -- 172.21.15.33:0/1909603254 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d30109ad0 msgr2=0x7f5d3019fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:48:45.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:45.929+0000 7f5d38400700 1 --2- 172.21.15.33:0/1909603254 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d30109ad0 0x7f5d3019fff0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f5d2400b530 tx=0x7f5d2400b8f0 comp rx=0 tx=0).stop 2024-05-03T18:48:45.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:45.930+0000 7f5d38400700 1 -- 172.21.15.33:0/1909603254 shutdown_connections 2024-05-03T18:48:45.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:45.930+0000 7f5d38400700 1 --2- 172.21.15.33:0/1909603254 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d3010fb60 0x7f5d301a48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:48:45.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:45.930+0000 7f5d38400700 1 --2- 172.21.15.33:0/1909603254 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d30109ad0 0x7f5d3019fff0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:48:45.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:45.930+0000 7f5d38400700 1 --2- 172.21.15.33:0/1909603254 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f5d180b29f0 0x7f5d180b4eb0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:48:45.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:45.930+0000 7f5d38400700 1 --2- 172.21.15.33:0/1909603254 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d3010a490 0x7f5d301a0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:48:45.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:45.930+0000 7f5d38400700 1 -- 172.21.15.33:0/1909603254 >> 172.21.15.33:0/1909603254 conn(0x7f5d30076c70 msgr2=0x7f5d300771b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:48:45.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:45.930+0000 7f5d38400700 1 -- 172.21.15.33:0/1909603254 shutdown_connections 2024-05-03T18:48:45.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:45.930+0000 7f5d38400700 1 -- 172.21.15.33:0/1909603254 wait complete. 2024-05-03T18:48:45.933 INFO:teuthology.orchestra.run.smithi033.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-03T18:48:46.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:48:45 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mon-a[28325]: 2024-05-03T18:48:45.921+0000 7fb6fea00700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-03T18:48:46.547 INFO:journalctl@ceph.mds.b.smithi099.stdout:May 03 18:48:46 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-b[62001]: ignoring --setuser ceph since I am not root 2024-05-03T18:48:46.547 INFO:journalctl@ceph.mds.b.smithi099.stdout:May 03 18:48:46 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-b[62001]: ignoring --setgroup ceph since I am not root 2024-05-03T18:48:46.547 INFO:journalctl@ceph.mds.b.smithi099.stdout:May 03 18:48:46 smithi099 ceph-mds[62010]: -- 172.21.15.99:0/3333447293 <== mon.2 v2:172.21.15.99:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cc03e9da40 con 0x55cc03e70800 2024-05-03T18:48:46.548 INFO:journalctl@ceph.mds.b.smithi099.stdout:May 03 18:48:46 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-b[62001]: starting mds.b at 2024-05-03T18:48:46.548 INFO:journalctl@ceph.mds.h.smithi099.stdout:May 03 18:48:46 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-h[61696]: ignoring --setuser ceph since I am not root 2024-05-03T18:48:46.548 INFO:journalctl@ceph.mds.h.smithi099.stdout:May 03 18:48:46 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-h[61696]: ignoring --setgroup ceph since I am not root 2024-05-03T18:48:46.548 INFO:journalctl@ceph.mds.h.smithi099.stdout:May 03 18:48:46 smithi099 ceph-mds[61700]: -- 172.21.15.99:0/1862584637 <== mon.2 v2:172.21.15.99:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ae84cd9a40 con 0x55ae84cacc00 2024-05-03T18:48:46.548 INFO:journalctl@ceph.mds.h.smithi099.stdout:May 03 18:48:46 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-h[61696]: starting mds.h at 2024-05-03T18:48:46.548 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:48:46 smithi099 ceph-mon[31658]: pgmap v681: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.0 KiB/s wr, 2 op/s 2024-05-03T18:48:46.548 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:48:46 smithi099 ceph-mon[31658]: Adjusting osd_memory_target on smithi150 to 176.4M 2024-05-03T18:48:46.549 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:48:46 smithi099 ceph-mon[31658]: Unable to set osd_memory_target on smithi150 to 185064857: error parsing value: Value '185064857' is below minimum 939524096 2024-05-03T18:48:46.549 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:48:46 smithi099 ceph-mon[31658]: Adjusting osd_memory_target on smithi099 to 176.4M 2024-05-03T18:48:46.549 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:48:46 smithi099 ceph-mon[31658]: Unable to set osd_memory_target on smithi099 to 185040486: error parsing value: Value '185040486' is below minimum 939524096 2024-05-03T18:48:46.549 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:48:46 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/1909603254' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-03T18:48:46.549 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:48:46 smithi099 ceph-mon[31658]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-03T18:48:46.549 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:48:46 smithi099 ceph-mon[31658]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-03T18:48:46.549 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:48:46 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/1909603254' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-03T18:48:46.549 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:48:46 smithi099 ceph-mon[31658]: osdmap e123: 12 total, 12 up, 12 in 2024-05-03T18:48:46.549 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:48:46 smithi099 ceph-mon[31658]: fsmap cephfs:0/1 10 up:standby, 1 failed 2024-05-03T18:48:46.549 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:48:46 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:48:46.549 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:48:46 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:48:46.549 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:48:46 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:48:46.549 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:48:46 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:48:46.550 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:48:46 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:48:46.550 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:48:46 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:48:46.550 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:48:46 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:48:46.634 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:48:46 smithi150 ceph-mon[32492]: pgmap v681: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.0 KiB/s wr, 2 op/s 2024-05-03T18:48:46.634 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:48:46 smithi150 ceph-mon[32492]: Adjusting osd_memory_target on smithi150 to 176.4M 2024-05-03T18:48:46.634 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:48:46 smithi150 ceph-mon[32492]: Unable to set osd_memory_target on smithi150 to 185064857: error parsing value: Value '185064857' is below minimum 939524096 2024-05-03T18:48:46.634 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:48:46 smithi150 ceph-mon[32492]: Adjusting osd_memory_target on smithi099 to 176.4M 2024-05-03T18:48:46.634 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:48:46 smithi150 ceph-mon[32492]: Unable to set osd_memory_target on smithi099 to 185040486: error parsing value: Value '185040486' is below minimum 939524096 2024-05-03T18:48:46.634 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:48:46 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/1909603254' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-03T18:48:46.634 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:48:46 smithi150 ceph-mon[32492]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-03T18:48:46.634 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:48:46 smithi150 ceph-mon[32492]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-03T18:48:46.634 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:48:46 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/1909603254' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-03T18:48:46.634 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:48:46 smithi150 ceph-mon[32492]: osdmap e123: 12 total, 12 up, 12 in 2024-05-03T18:48:46.635 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:48:46 smithi150 ceph-mon[32492]: fsmap cephfs:0/1 10 up:standby, 1 failed 2024-05-03T18:48:46.635 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:48:46 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:48:46.635 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:48:46 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:48:46.635 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:48:46 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:48:46.635 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:48:46 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:48:46.635 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:48:46 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:48:46.635 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:48:46 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:48:46.635 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:48:46 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:48:46.672 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:48:46 smithi033 ceph-mon[28330]: pgmap v681: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.0 KiB/s wr, 2 op/s 2024-05-03T18:48:46.672 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:48:46 smithi033 ceph-mon[28330]: Adjusting osd_memory_target on smithi150 to 176.4M 2024-05-03T18:48:46.672 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:48:46 smithi033 ceph-mon[28330]: Unable to set osd_memory_target on smithi150 to 185064857: error parsing value: Value '185064857' is below minimum 939524096 2024-05-03T18:48:46.673 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:48:46 smithi033 ceph-mon[28330]: Adjusting osd_memory_target on smithi099 to 176.4M 2024-05-03T18:48:46.673 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:48:46 smithi033 ceph-mon[28330]: Unable to set osd_memory_target on smithi099 to 185040486: error parsing value: Value '185040486' is below minimum 939524096 2024-05-03T18:48:46.673 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:48:46 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/1909603254' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-03T18:48:46.673 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:48:46 smithi033 ceph-mon[28330]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-03T18:48:46.673 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:48:46 smithi033 ceph-mon[28330]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-03T18:48:46.673 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:48:46 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/1909603254' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-03T18:48:46.673 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:48:46 smithi033 ceph-mon[28330]: osdmap e123: 12 total, 12 up, 12 in 2024-05-03T18:48:46.673 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:48:46 smithi033 ceph-mon[28330]: fsmap cephfs:0/1 10 up:standby, 1 failed 2024-05-03T18:48:46.673 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:48:46 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:48:46.673 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:48:46 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:48:46.673 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:48:46 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:48:46.673 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:48:46 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:48:46.673 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:48:46 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:48:46.673 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:48:46 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:48:46.673 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:48:46 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' 2024-05-03T18:48:46.726 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-05-03T18:48:47.192 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:48:47.547 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:48:47 smithi099 ceph-mon[31658]: Adjusting osd_memory_target on smithi033 to 177.5M 2024-05-03T18:48:47.547 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:48:47 smithi099 ceph-mon[31658]: Unable to set osd_memory_target on smithi033 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-05-03T18:48:47.547 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:48:47 smithi099 ceph-mon[31658]: mds.? [v2:172.21.15.99:6836/3871019131,v1:172.21.15.99:6837/3871019131] up:boot 2024-05-03T18:48:47.547 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:48:47 smithi099 ceph-mon[31658]: mds.? [v2:172.21.15.99:6838/3101493528,v1:172.21.15.99:6839/3101493528] up:boot 2024-05-03T18:48:47.547 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:48:47 smithi099 ceph-mon[31658]: fsmap cephfs:0/1 12 up:standby, 1 failed 2024-05-03T18:48:47.547 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:48:47 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-03T18:48:47.547 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:48:47 smithi099 ceph-mon[31658]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-03T18:48:47.557 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:48:47 smithi033 ceph-mon[28330]: Adjusting osd_memory_target on smithi033 to 177.5M 2024-05-03T18:48:47.557 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:48:47 smithi033 ceph-mon[28330]: Unable to set osd_memory_target on smithi033 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-05-03T18:48:47.557 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:48:47 smithi033 ceph-mon[28330]: mds.? [v2:172.21.15.99:6836/3871019131,v1:172.21.15.99:6837/3871019131] up:boot 2024-05-03T18:48:47.557 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:48:47 smithi033 ceph-mon[28330]: mds.? [v2:172.21.15.99:6838/3101493528,v1:172.21.15.99:6839/3101493528] up:boot 2024-05-03T18:48:47.557 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:48:47 smithi033 ceph-mon[28330]: fsmap cephfs:0/1 12 up:standby, 1 failed 2024-05-03T18:48:47.557 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:48:47 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-03T18:48:47.557 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:48:47 smithi033 ceph-mon[28330]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-03T18:48:47.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:48:47 smithi150 ceph-mon[32492]: Adjusting osd_memory_target on smithi033 to 177.5M 2024-05-03T18:48:47.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:48:47 smithi150 ceph-mon[32492]: Unable to set osd_memory_target on smithi033 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-05-03T18:48:47.634 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:48:47 smithi150 ceph-mon[32492]: mds.? [v2:172.21.15.99:6836/3871019131,v1:172.21.15.99:6837/3871019131] up:boot 2024-05-03T18:48:47.634 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:48:47 smithi150 ceph-mon[32492]: mds.? [v2:172.21.15.99:6838/3101493528,v1:172.21.15.99:6839/3101493528] up:boot 2024-05-03T18:48:47.634 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:48:47 smithi150 ceph-mon[32492]: fsmap cephfs:0/1 12 up:standby, 1 failed 2024-05-03T18:48:47.634 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:48:47 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-03T18:48:47.634 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:48:47 smithi150 ceph-mon[32492]: from='mgr.15120 172.21.15.33:0/1541401618' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-03T18:48:48.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:48:48 smithi099 ceph-mon[31658]: pgmap v683: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-05-03T18:48:48.632 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:48:48 smithi033 ceph-mon[28330]: pgmap v683: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-05-03T18:48:48.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:48:48 smithi150 ceph-mon[32492]: pgmap v683: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-05-03T18:48:49.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:49.496+0000 7f099a200700 1 -- 172.21.15.33:0/2216983144 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09941056c0 msgr2=0x7f0994105b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:48:49.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:49.496+0000 7f099a200700 1 --2- 172.21.15.33:0/2216983144 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09941056c0 0x7f0994105b40 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f0980009990 tx=0x7f0980009ca0 comp rx=0 tx=0).stop 2024-05-03T18:48:49.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:49.496+0000 7f099a200700 1 -- 172.21.15.33:0/2216983144 shutdown_connections 2024-05-03T18:48:49.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:49.496+0000 7f099a200700 1 --2- 172.21.15.33:0/2216983144 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0994106080 0x7f0994112590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:48:49.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:49.496+0000 7f099a200700 1 --2- 172.21.15.33:0/2216983144 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f099410b6c0 0x7f099410baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:48:49.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:49.496+0000 7f099a200700 1 --2- 172.21.15.33:0/2216983144 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09941056c0 0x7f0994105b40 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:48:49.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:49.496+0000 7f099a200700 1 -- 172.21.15.33:0/2216983144 >> 172.21.15.33:0/2216983144 conn(0x7f09941011b0 msgr2=0x7f09941035d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:48:49.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:49.497+0000 7f099a200700 1 -- 172.21.15.33:0/2216983144 shutdown_connections 2024-05-03T18:48:49.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:49.497+0000 7f099a200700 1 -- 172.21.15.33:0/2216983144 wait complete. 2024-05-03T18:48:49.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:49.498+0000 7f099a200700 1 Processor -- start 2024-05-03T18:48:49.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:49.498+0000 7f099a200700 1 -- start start 2024-05-03T18:48:49.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:49.499+0000 7f099a200700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09941056c0 0x7f09941a18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:48:49.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:49.499+0000 7f099a200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0994106080 0x7f09941a1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:48:49.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:49.499+0000 7f099a200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f099410b6c0 0x7f099419bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:48:49.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:49.499+0000 7f099a200700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f09941157b0 con 0x7f099410b6c0 2024-05-03T18:48:49.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:49.499+0000 7f099a200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0994115630 con 0x7f0994106080 2024-05-03T18:48:49.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:49.499+0000 7f099a200700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f0994115930 con 0x7f09941056c0 2024-05-03T18:48:49.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:49.499+0000 7f0993e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f099410b6c0 0x7f099419bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:48:49.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:49.499+0000 7f0993e00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f099410b6c0 0x7f099419bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56262/0 (socket says 172.21.15.33:56262) 2024-05-03T18:48:49.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:49.499+0000 7f0993e00700 1 -- 172.21.15.33:0/2720914456 learned_addr learned my addr 172.21.15.33:0/2720914456 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:48:49.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:49.499+0000 7f0992a00700 1 --2- 172.21.15.33:0/2720914456 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0994106080 0x7f09941a1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:48:49.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:49.499+0000 7f0993400700 1 --2- 172.21.15.33:0/2720914456 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09941056c0 0x7f09941a18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:48:49.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:49.500+0000 7f0993e00700 1 -- 172.21.15.33:0/2720914456 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09941056c0 msgr2=0x7f09941a18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:48:49.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:49.500+0000 7f0993e00700 1 --2- 172.21.15.33:0/2720914456 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09941056c0 0x7f09941a18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:48:49.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:49.500+0000 7f0993e00700 1 -- 172.21.15.33:0/2720914456 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0994106080 msgr2=0x7f09941a1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:48:49.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:49.500+0000 7f0993e00700 1 --2- 172.21.15.33:0/2720914456 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0994106080 0x7f09941a1e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:48:49.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:49.500+0000 7f0993e00700 1 -- 172.21.15.33:0/2720914456 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f099419c210 con 0x7f099410b6c0 2024-05-03T18:48:49.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:49.500+0000 7f0993e00700 1 --2- 172.21.15.33:0/2720914456 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f099410b6c0 0x7f099419bab0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f098c007ee0 tx=0x7f098c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:48:49.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:49.500+0000 7f098be00700 1 -- 172.21.15.33:0/2720914456 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f098c00dbd0 con 0x7f099410b6c0 2024-05-03T18:48:49.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:49.500+0000 7f098be00700 1 -- 172.21.15.33:0/2720914456 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f098c00dd70 con 0x7f099410b6c0 2024-05-03T18:48:49.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:49.500+0000 7f098be00700 1 -- 172.21.15.33:0/2720914456 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f098c0058d0 con 0x7f099410b6c0 2024-05-03T18:48:49.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:49.500+0000 7f099a200700 1 -- 172.21.15.33:0/2720914456 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f099419c500 con 0x7f099410b6c0 2024-05-03T18:48:49.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:49.500+0000 7f099a200700 1 -- 172.21.15.33:0/2720914456 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09941a87c0 con 0x7f099410b6c0 2024-05-03T18:48:49.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:49.502+0000 7f099a200700 1 -- 172.21.15.33:0/2720914456 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f099406a190 con 0x7f099410b6c0 2024-05-03T18:48:49.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:49.504+0000 7f098be00700 1 -- 172.21.15.33:0/2720914456 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f098c01a020 con 0x7f099410b6c0 2024-05-03T18:48:49.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:49.505+0000 7f098be00700 1 --2- 172.21.15.33:0/2720914456 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f09780b29f0 0x7f09780b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:48:49.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:49.506+0000 7f098be00700 1 -- 172.21.15.33:0/2720914456 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 9516+0+0 (secure 0 0 0) 0x7f098c0e22b0 con 0x7f099410b6c0 2024-05-03T18:48:49.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:49.506+0000 7f0993400700 1 --2- 172.21.15.33:0/2720914456 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f09780b29f0 0x7f09780b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:48:49.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:49.506+0000 7f0993400700 1 --2- 172.21.15.33:0/2720914456 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f09780b29f0 0x7f09780b4eb0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f099419cbf0 tx=0x7f098400b4d0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:48:49.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:49.510+0000 7f098be00700 1 -- 172.21.15.33:0/2720914456 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f098c010070 con 0x7f099410b6c0 2024-05-03T18:48:49.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:49.804+0000 7f099a200700 1 -- 172.21.15.33:0/2720914456 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f099402e510 con 0x7f099410b6c0 2024-05-03T18:48:50.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:50.285+0000 7f098be00700 1 -- 172.21.15.33:0/2720914456 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v24) ==== 104+0+0 (secure 0 0 0) 0x7f098c0a9850 con 0x7f099410b6c0 2024-05-03T18:48:50.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:50.289+0000 7f099a200700 1 -- 172.21.15.33:0/2720914456 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f09780b29f0 msgr2=0x7f09780b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:48:50.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:50.289+0000 7f099a200700 1 --2- 172.21.15.33:0/2720914456 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f09780b29f0 0x7f09780b4eb0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f099419cbf0 tx=0x7f098400b4d0 comp rx=0 tx=0).stop 2024-05-03T18:48:50.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:50.289+0000 7f099a200700 1 -- 172.21.15.33:0/2720914456 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f099410b6c0 msgr2=0x7f099419bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:48:50.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:50.289+0000 7f099a200700 1 --2- 172.21.15.33:0/2720914456 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f099410b6c0 0x7f099419bab0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f098c007ee0 tx=0x7f098c00d3b0 comp rx=0 tx=0).stop 2024-05-03T18:48:50.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:50.289+0000 7f099a200700 1 -- 172.21.15.33:0/2720914456 shutdown_connections 2024-05-03T18:48:50.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:50.289+0000 7f099a200700 1 --2- 172.21.15.33:0/2720914456 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f099410b6c0 0x7f099419bab0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:48:50.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:50.289+0000 7f099a200700 1 --2- 172.21.15.33:0/2720914456 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09941056c0 0x7f09941a18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:48:50.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:50.289+0000 7f099a200700 1 --2- 172.21.15.33:0/2720914456 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f09780b29f0 0x7f09780b4eb0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:48:50.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:50.289+0000 7f099a200700 1 --2- 172.21.15.33:0/2720914456 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0994106080 0x7f09941a1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:48:50.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:50.289+0000 7f099a200700 1 -- 172.21.15.33:0/2720914456 >> 172.21.15.33:0/2720914456 conn(0x7f09941011b0 msgr2=0x7f0994101ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:48:50.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:50.290+0000 7f099a200700 1 -- 172.21.15.33:0/2720914456 shutdown_connections 2024-05-03T18:48:50.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:50.290+0000 7f099a200700 1 -- 172.21.15.33:0/2720914456 wait complete. 2024-05-03T18:48:50.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:48:50 smithi150 ceph-mon[32492]: pgmap v684: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 818 B/s rd, 1 op/s 2024-05-03T18:48:50.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:48:50 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/2720914456' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-03T18:48:50.680 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:48:50 smithi033 ceph-mon[28330]: pgmap v684: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 818 B/s rd, 1 op/s 2024-05-03T18:48:50.681 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:48:50 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/2720914456' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-03T18:48:50.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:48:50 smithi099 ceph-mon[31658]: pgmap v684: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 818 B/s rd, 1 op/s 2024-05-03T18:48:50.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:48:50 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/2720914456' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-03T18:48:51.100 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-03T18:48:51.588 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:48:51.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:48:51 smithi150 ceph-mon[32492]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-03T18:48:51.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:48:51 smithi150 ceph-mon[32492]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-03T18:48:51.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:48:51 smithi150 ceph-mon[32492]: Cluster is now healthy 2024-05-03T18:48:51.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:48:51 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/2720914456' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-03T18:48:51.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:48:51 smithi150 ceph-mon[32492]: fsmap 12 up:standby 2024-05-03T18:48:51.634 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:48:51 smithi150 ceph-mon[32492]: pgmap v685: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 818 B/s rd, 1 op/s 2024-05-03T18:48:51.735 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:48:51 smithi033 ceph-mon[28330]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-03T18:48:51.735 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:48:51 smithi033 ceph-mon[28330]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-03T18:48:51.735 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:48:51 smithi033 ceph-mon[28330]: Cluster is now healthy 2024-05-03T18:48:51.735 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:48:51 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/2720914456' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-03T18:48:51.735 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:48:51 smithi033 ceph-mon[28330]: fsmap 12 up:standby 2024-05-03T18:48:51.735 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:48:51 smithi033 ceph-mon[28330]: pgmap v685: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 818 B/s rd, 1 op/s 2024-05-03T18:48:51.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:48:51 smithi099 ceph-mon[31658]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-03T18:48:51.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:48:51 smithi099 ceph-mon[31658]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-03T18:48:51.797 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:48:51 smithi099 ceph-mon[31658]: Cluster is now healthy 2024-05-03T18:48:51.797 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:48:51 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/2720914456' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-03T18:48:51.797 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:48:51 smithi099 ceph-mon[31658]: fsmap 12 up:standby 2024-05-03T18:48:51.797 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:48:51 smithi099 ceph-mon[31658]: pgmap v685: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 818 B/s rd, 1 op/s 2024-05-03T18:48:53.520 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:48:53 smithi033 ceph-mon[28330]: pgmap v686: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 818 B/s rd, 1 op/s 2024-05-03T18:48:53.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:48:53 smithi099 ceph-mon[31658]: pgmap v686: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 818 B/s rd, 1 op/s 2024-05-03T18:48:53.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:53.813+0000 7f38f3c00700 1 -- 172.21.15.33:0/1014471847 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38ec108850 msgr2=0x7f38ec1103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:48:53.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:53.813+0000 7f38f3c00700 1 --2- 172.21.15.33:0/1014471847 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38ec108850 0x7f38ec1103d0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f38e400b0d0 tx=0x7f38e400b3e0 comp rx=0 tx=0).stop 2024-05-03T18:48:53.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:53.813+0000 7f38f3c00700 1 -- 172.21.15.33:0/1014471847 shutdown_connections 2024-05-03T18:48:53.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:53.813+0000 7f38f3c00700 1 --2- 172.21.15.33:0/1014471847 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38ec108850 0x7f38ec1103d0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:48:53.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:53.813+0000 7f38f3c00700 1 --2- 172.21.15.33:0/1014471847 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38ec103750 0x7f38ec103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:48:53.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:53.813+0000 7f38f3c00700 1 --2- 172.21.15.33:0/1014471847 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38ec104100 0x7f38ec108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:48:53.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:53.813+0000 7f38f3c00700 1 -- 172.21.15.33:0/1014471847 >> 172.21.15.33:0/1014471847 conn(0x7f38ec07a600 msgr2=0x7f38ec07aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:48:53.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:53.813+0000 7f38f3c00700 1 -- 172.21.15.33:0/1014471847 shutdown_connections 2024-05-03T18:48:53.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:53.814+0000 7f38f3c00700 1 -- 172.21.15.33:0/1014471847 wait complete. 2024-05-03T18:48:53.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:53.815+0000 7f38f3c00700 1 Processor -- start 2024-05-03T18:48:53.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:53.816+0000 7f38f3c00700 1 -- start start 2024-05-03T18:48:53.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:53.816+0000 7f38f3c00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38ec103750 0x7f38ec19f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:48:53.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:53.816+0000 7f38f3c00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38ec104100 0x7f38ec19fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:48:53.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:53.816+0000 7f38f3c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38ec108850 0x7f38ec1996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:48:53.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:53.816+0000 7f38f3c00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f38ec113520 con 0x7f38ec104100 2024-05-03T18:48:53.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:53.816+0000 7f38f3c00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f38ec1133a0 con 0x7f38ec108850 2024-05-03T18:48:53.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:53.816+0000 7f38f3c00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f38ec1136a0 con 0x7f38ec103750 2024-05-03T18:48:53.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:53.816+0000 7f38ebe00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38ec104100 0x7f38ec19fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:48:53.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:53.817+0000 7f38ebe00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38ec104100 0x7f38ec19fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56278/0 (socket says 172.21.15.33:56278) 2024-05-03T18:48:53.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:53.817+0000 7f38ebe00700 1 -- 172.21.15.33:0/69146491 learned_addr learned my addr 172.21.15.33:0/69146491 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:48:53.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:53.817+0000 7f38f1200700 1 --2- 172.21.15.33:0/69146491 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38ec103750 0x7f38ec19f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:48:53.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:53.817+0000 7f38f1c00700 1 --2- 172.21.15.33:0/69146491 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38ec108850 0x7f38ec1996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:48:53.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:53.817+0000 7f38ebe00700 1 -- 172.21.15.33:0/69146491 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38ec103750 msgr2=0x7f38ec19f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:48:53.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:53.817+0000 7f38ebe00700 1 --2- 172.21.15.33:0/69146491 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38ec103750 0x7f38ec19f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:48:53.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:53.817+0000 7f38ebe00700 1 -- 172.21.15.33:0/69146491 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38ec108850 msgr2=0x7f38ec1996c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:48:53.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:53.817+0000 7f38ebe00700 1 --2- 172.21.15.33:0/69146491 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38ec108850 0x7f38ec1996c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:48:53.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:53.817+0000 7f38ebe00700 1 -- 172.21.15.33:0/69146491 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38ec199fa0 con 0x7f38ec104100 2024-05-03T18:48:53.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:53.818+0000 7f38ebe00700 1 --2- 172.21.15.33:0/69146491 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38ec104100 0x7f38ec19fb30 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f38d800c9b0 tx=0x7f38d800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:48:53.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:53.818+0000 7f38e9600700 1 -- 172.21.15.33:0/69146491 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38d8007b00 con 0x7f38ec104100 2024-05-03T18:48:53.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:53.818+0000 7f38e9600700 1 -- 172.21.15.33:0/69146491 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f38d8007ca0 con 0x7f38ec104100 2024-05-03T18:48:53.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:53.818+0000 7f38e9600700 1 -- 172.21.15.33:0/69146491 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38d8018660 con 0x7f38ec104100 2024-05-03T18:48:53.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:53.818+0000 7f38f3c00700 1 -- 172.21.15.33:0/69146491 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38ec19a290 con 0x7f38ec104100 2024-05-03T18:48:53.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:53.818+0000 7f38f3c00700 1 -- 172.21.15.33:0/69146491 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38ec1a6570 con 0x7f38ec104100 2024-05-03T18:48:53.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:53.819+0000 7f38f3c00700 1 -- 172.21.15.33:0/69146491 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38ec104be0 con 0x7f38ec104100 2024-05-03T18:48:53.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:53.822+0000 7f38e9600700 1 -- 172.21.15.33:0/69146491 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f38d800f450 con 0x7f38ec104100 2024-05-03T18:48:53.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:53.824+0000 7f38e9600700 1 --2- 172.21.15.33:0/69146491 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f38d40b2a70 0x7f38d40b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:48:53.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:53.824+0000 7f38e9600700 1 -- 172.21.15.33:0/69146491 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 9516+0+0 (secure 0 0 0) 0x7f38d80e29f0 con 0x7f38ec104100 2024-05-03T18:48:53.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:53.824+0000 7f38f1200700 1 --2- 172.21.15.33:0/69146491 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f38d40b2a70 0x7f38d40b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:48:53.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:53.825+0000 7f38f1200700 1 --2- 172.21.15.33:0/69146491 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f38d40b2a70 0x7f38d40b4f30 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f38e000ac50 tx=0x7f38e000a380 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:48:53.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:53.829+0000 7f38e9600700 1 -- 172.21.15.33:0/69146491 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f38d8014030 con 0x7f38ec104100 2024-05-03T18:48:53.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:48:53 smithi150 ceph-mon[32492]: pgmap v686: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 818 B/s rd, 1 op/s 2024-05-03T18:48:54.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:54.068+0000 7f38f3c00700 1 -- 172.21.15.33:0/69146491 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f38ec1a6740 con 0x7f38ec104100 2024-05-03T18:48:54.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:54.485+0000 7f38e9600700 1 -- 172.21.15.33:0/69146491 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v124) ==== 181+0+0 (secure 0 0 0) 0x7f38d80a9f90 con 0x7f38ec104100 2024-05-03T18:48:54.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:54.491+0000 7f38f3c00700 1 -- 172.21.15.33:0/69146491 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f38d40b2a70 msgr2=0x7f38d40b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:48:54.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:54.491+0000 7f38f3c00700 1 --2- 172.21.15.33:0/69146491 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f38d40b2a70 0x7f38d40b4f30 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f38e000ac50 tx=0x7f38e000a380 comp rx=0 tx=0).stop 2024-05-03T18:48:54.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:54.491+0000 7f38f3c00700 1 -- 172.21.15.33:0/69146491 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38ec104100 msgr2=0x7f38ec19fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:48:54.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:54.491+0000 7f38f3c00700 1 --2- 172.21.15.33:0/69146491 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38ec104100 0x7f38ec19fb30 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f38d800c9b0 tx=0x7f38d800ccc0 comp rx=0 tx=0).stop 2024-05-03T18:48:54.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:54.491+0000 7f38f3c00700 1 -- 172.21.15.33:0/69146491 shutdown_connections 2024-05-03T18:48:54.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:54.491+0000 7f38f3c00700 1 --2- 172.21.15.33:0/69146491 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f38d40b2a70 0x7f38d40b4f30 secure :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f38e000ac50 tx=0x7f38e000a380 comp rx=0 tx=0).stop 2024-05-03T18:48:54.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:54.491+0000 7f38f3c00700 1 --2- 172.21.15.33:0/69146491 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38ec108850 0x7f38ec1996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:48:54.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:54.491+0000 7f38f3c00700 1 --2- 172.21.15.33:0/69146491 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38ec103750 0x7f38ec19f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:48:54.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:54.491+0000 7f38f3c00700 1 --2- 172.21.15.33:0/69146491 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38ec104100 0x7f38ec19fb30 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:48:54.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:54.491+0000 7f38f3c00700 1 -- 172.21.15.33:0/69146491 >> 172.21.15.33:0/69146491 conn(0x7f38ec07a600 msgr2=0x7f38ec101420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:48:54.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:54.493+0000 7f38f3c00700 1 -- 172.21.15.33:0/69146491 shutdown_connections 2024-05-03T18:48:54.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:54.493+0000 7f38f3c00700 1 -- 172.21.15.33:0/69146491 wait complete. 2024-05-03T18:48:54.500 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_metadata' removed 2024-05-03T18:48:54.735 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:48:54 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/69146491' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-03T18:48:54.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:48:54 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/69146491' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-03T18:48:54.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:48:54 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/69146491' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-03T18:48:55.447 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-03T18:48:55.735 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:48:55 smithi033 ceph-mon[28330]: pgmap v687: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 307 B/s rd, 0 op/s 2024-05-03T18:48:55.735 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:48:55 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/69146491' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-03T18:48:55.735 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:48:55 smithi033 ceph-mon[28330]: osdmap e124: 12 total, 12 up, 12 in 2024-05-03T18:48:55.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:48:55 smithi099 ceph-mon[31658]: pgmap v687: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 307 B/s rd, 0 op/s 2024-05-03T18:48:55.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:48:55 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/69146491' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-03T18:48:55.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:48:55 smithi099 ceph-mon[31658]: osdmap e124: 12 total, 12 up, 12 in 2024-05-03T18:48:55.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:48:55 smithi150 ceph-mon[32492]: pgmap v687: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 307 B/s rd, 0 op/s 2024-05-03T18:48:55.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:48:55 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/69146491' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-03T18:48:55.884 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:48:55 smithi150 ceph-mon[32492]: osdmap e124: 12 total, 12 up, 12 in 2024-05-03T18:48:55.910 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:48:57.666 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:48:57 smithi033 ceph-mon[28330]: pgmap v689: 97 pgs: 97 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:48:57.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:48:57 smithi099 ceph-mon[31658]: pgmap v689: 97 pgs: 97 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:48:57.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:48:57 smithi150 ceph-mon[32492]: pgmap v689: 97 pgs: 97 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:48:58.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:58.247+0000 7fad95400700 1 -- 172.21.15.33:0/1227705766 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad901056c0 msgr2=0x7fad90105b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:48:58.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:58.247+0000 7fad95400700 1 --2- 172.21.15.33:0/1227705766 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad901056c0 0x7fad90105b40 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fad7c009990 tx=0x7fad7c009ca0 comp rx=0 tx=0).stop 2024-05-03T18:48:58.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:58.247+0000 7fad95400700 1 -- 172.21.15.33:0/1227705766 shutdown_connections 2024-05-03T18:48:58.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:58.247+0000 7fad95400700 1 --2- 172.21.15.33:0/1227705766 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad90106080 0x7fad90112590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:48:58.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:58.247+0000 7fad95400700 1 --2- 172.21.15.33:0/1227705766 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fad9010b6c0 0x7fad9010baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:48:58.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:58.247+0000 7fad95400700 1 --2- 172.21.15.33:0/1227705766 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad901056c0 0x7fad90105b40 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:48:58.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:58.247+0000 7fad95400700 1 -- 172.21.15.33:0/1227705766 >> 172.21.15.33:0/1227705766 conn(0x7fad901011d0 msgr2=0x7fad901035f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:48:58.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:58.247+0000 7fad95400700 1 -- 172.21.15.33:0/1227705766 shutdown_connections 2024-05-03T18:48:58.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:58.247+0000 7fad95400700 1 -- 172.21.15.33:0/1227705766 wait complete. 2024-05-03T18:48:58.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:58.248+0000 7fad95400700 1 Processor -- start 2024-05-03T18:48:58.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:58.249+0000 7fad95400700 1 -- start start 2024-05-03T18:48:58.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:58.249+0000 7fad95400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fad901056c0 0x7fad901a1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:48:58.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:58.249+0000 7fad95400700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad90106080 0x7fad901a1e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:48:58.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:58.249+0000 7fad95400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad9010b6c0 0x7fad9019bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:48:58.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:58.249+0000 7fad95400700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fad901157b0 con 0x7fad9010b6c0 2024-05-03T18:48:58.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:58.249+0000 7fad95400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fad90115630 con 0x7fad901056c0 2024-05-03T18:48:58.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:58.249+0000 7fad95400700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fad90115930 con 0x7fad90106080 2024-05-03T18:48:58.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:58.249+0000 7fad8f200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad9010b6c0 0x7fad9019bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:48:58.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:58.249+0000 7fad8f200700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad9010b6c0 0x7fad9019bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48168/0 (socket says 172.21.15.33:48168) 2024-05-03T18:48:58.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:58.249+0000 7fad8f200700 1 -- 172.21.15.33:0/3528261698 learned_addr learned my addr 172.21.15.33:0/3528261698 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:48:58.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:58.250+0000 7fad8e800700 1 --2- 172.21.15.33:0/3528261698 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fad901056c0 0x7fad901a1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:48:58.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:58.250+0000 7fad8de00700 1 --2- 172.21.15.33:0/3528261698 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad90106080 0x7fad901a1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:48:58.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:58.250+0000 7fad8f200700 1 -- 172.21.15.33:0/3528261698 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad90106080 msgr2=0x7fad901a1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:48:58.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:58.250+0000 7fad8f200700 1 --2- 172.21.15.33:0/3528261698 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad90106080 0x7fad901a1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:48:58.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:58.250+0000 7fad8f200700 1 -- 172.21.15.33:0/3528261698 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fad901056c0 msgr2=0x7fad901a1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:48:58.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:58.250+0000 7fad8f200700 1 --2- 172.21.15.33:0/3528261698 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fad901056c0 0x7fad901a1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:48:58.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:58.250+0000 7fad8f200700 1 -- 172.21.15.33:0/3528261698 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad9019c290 con 0x7fad9010b6c0 2024-05-03T18:48:58.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:58.250+0000 7fad8f200700 1 --2- 172.21.15.33:0/3528261698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad9010b6c0 0x7fad9019bb30 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fad88007d70 tx=0x7fad8800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:48:58.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:58.251+0000 7fad87400700 1 -- 172.21.15.33:0/3528261698 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad88016020 con 0x7fad9010b6c0 2024-05-03T18:48:58.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:58.251+0000 7fad87400700 1 -- 172.21.15.33:0/3528261698 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fad8800ad40 con 0x7fad9010b6c0 2024-05-03T18:48:58.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:58.251+0000 7fad87400700 1 -- 172.21.15.33:0/3528261698 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad880115e0 con 0x7fad9010b6c0 2024-05-03T18:48:58.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:58.251+0000 7fad95400700 1 -- 172.21.15.33:0/3528261698 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad9019c580 con 0x7fad9010b6c0 2024-05-03T18:48:58.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:58.251+0000 7fad95400700 1 -- 172.21.15.33:0/3528261698 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad901a87f0 con 0x7fad9010b6c0 2024-05-03T18:48:58.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:58.252+0000 7fad95400700 1 -- 172.21.15.33:0/3528261698 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad9006a190 con 0x7fad9010b6c0 2024-05-03T18:48:58.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:58.254+0000 7fad87400700 1 -- 172.21.15.33:0/3528261698 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fad88019020 con 0x7fad9010b6c0 2024-05-03T18:48:58.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:58.256+0000 7fad87400700 1 --2- 172.21.15.33:0/3528261698 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fad740b2a70 0x7fad740b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:48:58.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:58.256+0000 7fad87400700 1 -- 172.21.15.33:0/3528261698 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 9020+0+0 (secure 0 0 0) 0x7fad880e2670 con 0x7fad9010b6c0 2024-05-03T18:48:58.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:58.256+0000 7fad8e800700 1 --2- 172.21.15.33:0/3528261698 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fad740b2a70 0x7fad740b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:48:58.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:58.257+0000 7fad8e800700 1 --2- 172.21.15.33:0/3528261698 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fad740b2a70 0x7fad740b4f30 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fad80009be0 tx=0x7fad80009260 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:48:58.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:58.261+0000 7fad87400700 1 -- 172.21.15.33:0/3528261698 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fad88020020 con 0x7fad9010b6c0 2024-05-03T18:48:58.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:58.503+0000 7fad95400700 1 -- 172.21.15.33:0/3528261698 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fad9019d000 con 0x7fad9010b6c0 2024-05-03T18:48:59.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:59.511+0000 7fad87400700 1 -- 172.21.15.33:0/3528261698 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v125) ==== 169+0+0 (secure 0 0 0) 0x7fad9019d000 con 0x7fad9010b6c0 2024-05-03T18:48:59.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:59.514+0000 7fad95400700 1 -- 172.21.15.33:0/3528261698 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fad740b2a70 msgr2=0x7fad740b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:48:59.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:59.514+0000 7fad95400700 1 --2- 172.21.15.33:0/3528261698 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fad740b2a70 0x7fad740b4f30 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fad80009be0 tx=0x7fad80009260 comp rx=0 tx=0).stop 2024-05-03T18:48:59.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:59.515+0000 7fad95400700 1 -- 172.21.15.33:0/3528261698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad9010b6c0 msgr2=0x7fad9019bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:48:59.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:59.515+0000 7fad95400700 1 --2- 172.21.15.33:0/3528261698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad9010b6c0 0x7fad9019bb30 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fad88007d70 tx=0x7fad8800a340 comp rx=0 tx=0).stop 2024-05-03T18:48:59.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:59.516+0000 7fad95400700 1 -- 172.21.15.33:0/3528261698 shutdown_connections 2024-05-03T18:48:59.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:59.516+0000 7fad95400700 1 --2- 172.21.15.33:0/3528261698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad9010b6c0 0x7fad9019bb30 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:48:59.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:59.516+0000 7fad95400700 1 --2- 172.21.15.33:0/3528261698 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7fad740b2a70 0x7fad740b4f30 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:48:59.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:59.516+0000 7fad95400700 1 --2- 172.21.15.33:0/3528261698 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fad901056c0 0x7fad901a1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:48:59.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:59.516+0000 7fad95400700 1 --2- 172.21.15.33:0/3528261698 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad90106080 0x7fad901a1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:48:59.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:59.516+0000 7fad95400700 1 -- 172.21.15.33:0/3528261698 >> 172.21.15.33:0/3528261698 conn(0x7fad901011d0 msgr2=0x7fad901019a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:48:59.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:59.518+0000 7fad95400700 1 -- 172.21.15.33:0/3528261698 shutdown_connections 2024-05-03T18:48:59.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:48:59.518+0000 7fad95400700 1 -- 172.21.15.33:0/3528261698 wait complete. 2024-05-03T18:48:59.523 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data' removed 2024-05-03T18:48:59.764 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:48:59 smithi033 ceph-mon[28330]: pgmap v690: 97 pgs: 97 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:48:59.764 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:48:59 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/3528261698' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-03T18:48:59.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:48:59 smithi099 ceph-mon[31658]: pgmap v690: 97 pgs: 97 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:48:59.797 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:48:59 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/3528261698' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-03T18:48:59.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:48:59 smithi150 ceph-mon[32492]: pgmap v690: 97 pgs: 97 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:48:59.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:48:59 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/3528261698' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-03T18:49:00.419 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid 5703bba4-0978-11ef-bc97-c7b262605968 -- ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-05-03T18:49:00.735 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:49:00 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/3528261698' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-03T18:49:00.735 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:49:00 smithi033 ceph-mon[28330]: osdmap e125: 12 total, 12 up, 12 in 2024-05-03T18:49:00.735 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:49:00 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:49:00.385+0000 7f51bc400700 -1 client.15168: SimpleRADOSStriper: lock: snap_db_v0.db: lock failed: (2) No such file or directory 2024-05-03T18:49:00.735 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:49:00 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:49:00.389+0000 7f51bc400700 -1 client.15168: SimpleRADOSStriper: lock: snap_db_v0.db: lock failed: (2) No such file or directory 2024-05-03T18:49:00.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:49:00 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/3528261698' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-03T18:49:00.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:49:00 smithi099 ceph-mon[31658]: osdmap e125: 12 total, 12 up, 12 in 2024-05-03T18:49:00.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:49:00 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/3528261698' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-03T18:49:00.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:49:00 smithi150 ceph-mon[32492]: osdmap e125: 12 total, 12 up, 12 in 2024-05-03T18:49:00.930 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:49:01.735 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:49:01 smithi033 ceph-mon[28330]: pgmap v692: 33 pgs: 33 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:49:01.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:49:01 smithi099 ceph-mon[31658]: pgmap v692: 33 pgs: 33 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:49:01.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:49:01 smithi150 ceph-mon[32492]: pgmap v692: 33 pgs: 33 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:49:03.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:49:03.390+0000 7f6f4a000700 1 -- 172.21.15.33:0/645918795 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f4410b6c0 msgr2=0x7f6f4410baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:49:03.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:49:03.390+0000 7f6f4a000700 1 --2- 172.21.15.33:0/645918795 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f4410b6c0 0x7f6f4410baa0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f6f34009990 tx=0x7f6f34009ca0 comp rx=0 tx=0).stop 2024-05-03T18:49:03.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:49:03.390+0000 7f6f4a000700 1 -- 172.21.15.33:0/645918795 shutdown_connections 2024-05-03T18:49:03.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:49:03.390+0000 7f6f4a000700 1 --2- 172.21.15.33:0/645918795 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6f44106080 0x7f6f44112590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:49:03.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:49:03.390+0000 7f6f4a000700 1 --2- 172.21.15.33:0/645918795 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f4410b6c0 0x7f6f4410baa0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:49:03.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:49:03.390+0000 7f6f4a000700 1 --2- 172.21.15.33:0/645918795 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f441056c0 0x7f6f44105b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:49:03.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:49:03.390+0000 7f6f4a000700 1 -- 172.21.15.33:0/645918795 >> 172.21.15.33:0/645918795 conn(0x7f6f44101170 msgr2=0x7f6f44103590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:49:03.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:49:03.390+0000 7f6f4a000700 1 -- 172.21.15.33:0/645918795 shutdown_connections 2024-05-03T18:49:03.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:49:03.390+0000 7f6f4a000700 1 -- 172.21.15.33:0/645918795 wait complete. 2024-05-03T18:49:03.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:49:03.391+0000 7f6f4a000700 1 Processor -- start 2024-05-03T18:49:03.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:49:03.392+0000 7f6f4a000700 1 -- start start 2024-05-03T18:49:03.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:49:03.392+0000 7f6f4a000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6f441056c0 0x7f6f441a18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:49:03.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:49:03.392+0000 7f6f4a000700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f44106080 0x7f6f441a1de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:49:03.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:49:03.392+0000 7f6f4a000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f4410b6c0 0x7f6f4419ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:49:03.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:49:03.392+0000 7f6f4a000700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6f441157c0 con 0x7f6f44106080 2024-05-03T18:49:03.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:49:03.392+0000 7f6f4a000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6f44115640 con 0x7f6f4410b6c0 2024-05-03T18:49:03.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:49:03.392+0000 7f6f4a000700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6f44115940 con 0x7f6f441056c0 2024-05-03T18:49:03.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:49:03.392+0000 7f6f3aa00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f44106080 0x7f6f441a1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:49:03.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:49:03.393+0000 7f6f3aa00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f44106080 0x7f6f441a1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48190/0 (socket says 172.21.15.33:48190) 2024-05-03T18:49:03.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:49:03.393+0000 7f6f3aa00700 1 -- 172.21.15.33:0/3070420595 learned_addr learned my addr 172.21.15.33:0/3070420595 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-03T18:49:03.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:49:03.393+0000 7f6f43400700 1 --2- 172.21.15.33:0/3070420595 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6f441056c0 0x7f6f441a18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:49:03.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:49:03.393+0000 7f6f43e00700 1 --2- 172.21.15.33:0/3070420595 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f4410b6c0 0x7f6f4419ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:49:03.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:49:03.393+0000 7f6f3aa00700 1 -- 172.21.15.33:0/3070420595 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6f441056c0 msgr2=0x7f6f441a18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:49:03.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:49:03.393+0000 7f6f3aa00700 1 --2- 172.21.15.33:0/3070420595 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6f441056c0 0x7f6f441a18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:49:03.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:49:03.393+0000 7f6f3aa00700 1 -- 172.21.15.33:0/3070420595 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f4410b6c0 msgr2=0x7f6f4419ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:49:03.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:49:03.393+0000 7f6f3aa00700 1 --2- 172.21.15.33:0/3070420595 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f4410b6c0 0x7f6f4419ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:49:03.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:49:03.393+0000 7f6f3aa00700 1 -- 172.21.15.33:0/3070420595 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f4419c1c0 con 0x7f6f44106080 2024-05-03T18:49:03.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:49:03.394+0000 7f6f3aa00700 1 --2- 172.21.15.33:0/3070420595 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f44106080 0x7f6f441a1de0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c007e20 tx=0x7f6f2c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:49:03.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:49:03.394+0000 7f6f40c00700 1 -- 172.21.15.33:0/3070420595 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f2c016020 con 0x7f6f44106080 2024-05-03T18:49:03.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:49:03.394+0000 7f6f4a000700 1 -- 172.21.15.33:0/3070420595 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f4419c4b0 con 0x7f6f44106080 2024-05-03T18:49:03.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:49:03.394+0000 7f6f40c00700 1 -- 172.21.15.33:0/3070420595 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6f2c00ad40 con 0x7f6f44106080 2024-05-03T18:49:03.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:49:03.394+0000 7f6f40c00700 1 -- 172.21.15.33:0/3070420595 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f2c0115e0 con 0x7f6f44106080 2024-05-03T18:49:03.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:49:03.394+0000 7f6f4a000700 1 -- 172.21.15.33:0/3070420595 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f441a8770 con 0x7f6f44106080 2024-05-03T18:49:03.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:49:03.395+0000 7f6f4a000700 1 -- 172.21.15.33:0/3070420595 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f4406a190 con 0x7f6f44106080 2024-05-03T18:49:03.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:49:03.398+0000 7f6f40c00700 1 -- 172.21.15.33:0/3070420595 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f6f2c019020 con 0x7f6f44106080 2024-05-03T18:49:03.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:49:03.399+0000 7f6f40c00700 1 --2- 172.21.15.33:0/3070420595 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f6f280b29f0 0x7f6f280b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:49:03.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:49:03.399+0000 7f6f40c00700 1 -- 172.21.15.33:0/3070420595 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 8432+0+0 (secure 0 0 0) 0x7f6f2c0e26d0 con 0x7f6f44106080 2024-05-03T18:49:03.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:49:03.400+0000 7f6f43400700 1 --2- 172.21.15.33:0/3070420595 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f6f280b29f0 0x7f6f280b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:49:03.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:49:03.400+0000 7f6f43400700 1 --2- 172.21.15.33:0/3070420595 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f6f280b29f0 0x7f6f280b4eb0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6f34003760 tx=0x7f6f3401a040 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:49:03.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:49:03.402+0000 7f6f40c00700 1 -- 172.21.15.33:0/3070420595 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f6f2c020070 con 0x7f6f44106080 2024-05-03T18:49:03.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:49:03.641+0000 7f6f4a000700 1 -- 172.21.15.33:0/3070420595 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f6f4402e510 con 0x7f6f44106080 2024-05-03T18:49:03.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:49:03 smithi099 ceph-mon[31658]: pgmap v693: 33 pgs: 33 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:49:03.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:49:03 smithi150 ceph-mon[32492]: pgmap v693: 33 pgs: 33 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:49:03.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:49:03 smithi033 ceph-mon[28330]: pgmap v693: 33 pgs: 33 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:49:04.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:49:04.551+0000 7f6f40c00700 1 -- 172.21.15.33:0/3070420595 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v126) ==== 178+0+0 (secure 0 0 0) 0x7f6f2c0aa1e0 con 0x7f6f44106080 2024-05-03T18:49:04.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:49:04.556+0000 7f6f4a000700 1 -- 172.21.15.33:0/3070420595 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f6f280b29f0 msgr2=0x7f6f280b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:49:04.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:49:04.556+0000 7f6f4a000700 1 --2- 172.21.15.33:0/3070420595 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f6f280b29f0 0x7f6f280b4eb0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6f34003760 tx=0x7f6f3401a040 comp rx=0 tx=0).stop 2024-05-03T18:49:04.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:49:04.556+0000 7f6f4a000700 1 -- 172.21.15.33:0/3070420595 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f44106080 msgr2=0x7f6f441a1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:49:04.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:49:04.556+0000 7f6f4a000700 1 --2- 172.21.15.33:0/3070420595 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f44106080 0x7f6f441a1de0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c007e20 tx=0x7f6f2c00a340 comp rx=0 tx=0).stop 2024-05-03T18:49:04.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:49:04.559+0000 7f6f4a000700 1 -- 172.21.15.33:0/3070420595 shutdown_connections 2024-05-03T18:49:04.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:49:04.559+0000 7f6f4a000700 1 --2- 172.21.15.33:0/3070420595 >> [v2:172.21.15.33:6800/3302011241,v1:172.21.15.33:6801/3302011241] conn(0x7f6f280b29f0 0x7f6f280b4eb0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:49:04.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:49:04.559+0000 7f6f4a000700 1 --2- 172.21.15.33:0/3070420595 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f4410b6c0 0x7f6f4419ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:49:04.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:49:04.559+0000 7f6f4a000700 1 --2- 172.21.15.33:0/3070420595 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6f441056c0 0x7f6f441a18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:49:04.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:49:04.559+0000 7f6f4a000700 1 --2- 172.21.15.33:0/3070420595 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f44106080 0x7f6f441a1de0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:49:04.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:49:04.559+0000 7f6f4a000700 1 -- 172.21.15.33:0/3070420595 >> 172.21.15.33:0/3070420595 conn(0x7f6f44101170 msgr2=0x7f6f441018f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:49:04.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:49:04.560+0000 7f6f4a000700 1 -- 172.21.15.33:0/3070420595 shutdown_connections 2024-05-03T18:49:04.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-03T18:49:04.560+0000 7f6f4a000700 1 -- 172.21.15.33:0/3070420595 wait complete. 2024-05-03T18:49:04.564 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data_ec' removed 2024-05-03T18:49:04.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:49:04 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/3070420595' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-03T18:49:04.883 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:49:04 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/3070420595' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-03T18:49:04.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:49:04 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/3070420595' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-03T18:49:05.531 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-03T18:49:05.652 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-03T18:49:05.735 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:49:05 smithi033 ceph-mon[28330]: pgmap v694: 33 pgs: 33 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:49:05.735 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:49:05 smithi033 ceph-mon[28330]: from='client.? 172.21.15.33:0/3070420595' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-05-03T18:49:05.735 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:49:05 smithi033 ceph-mon[28330]: osdmap e126: 12 total, 12 up, 12 in 2024-05-03T18:49:05.770 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-03T18:49:05.862 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:49:05 smithi150 ceph-mon[32492]: pgmap v694: 33 pgs: 33 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:49:05.862 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:49:05 smithi150 ceph-mon[32492]: from='client.? 172.21.15.33:0/3070420595' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-05-03T18:49:05.862 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:49:05 smithi150 ceph-mon[32492]: osdmap e126: 12 total, 12 up, 12 in 2024-05-03T18:49:05.880 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-05-03T18:49:05.881 DEBUG:teuthology.orchestra.run.smithi033:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-03T18:49:05.968 DEBUG:teuthology.orchestra.run.smithi099:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-03T18:49:06.047 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:49:05 smithi099 ceph-mon[31658]: pgmap v694: 33 pgs: 33 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:49:06.047 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:49:05 smithi099 ceph-mon[31658]: from='client.? 172.21.15.33:0/3070420595' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-05-03T18:49:06.047 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:49:05 smithi099 ceph-mon[31658]: osdmap e126: 12 total, 12 up, 12 in 2024-05-03T18:49:06.051 DEBUG:teuthology.orchestra.run.smithi150:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-03T18:49:06.136 INFO:tasks.cephadm:Stopping all daemons... 2024-05-03T18:49:06.136 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-05-03T18:49:06.136 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-5703bba4-0978-11ef-bc97-c7b262605968@mon.a 2024-05-03T18:49:06.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:49:06 smithi033 systemd[1]: Stopping Ceph mon.a for 5703bba4-0978-11ef-bc97-c7b262605968... 2024-05-03T18:49:06.987 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:49:06 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mon-a[28325]: 2024-05-03T18:49:06.700+0000 7fb705800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T18:49:06.987 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:49:06 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mon-a[28325]: 2024-05-03T18:49:06.700+0000 7fb705800700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-05-03T18:49:06.987 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:49:06 smithi033 podman[126695]: 2024-05-03 18:49:06.775159207 +0000 UTC m=+0.216516093 container died b6180ce9a1aba09fc279a712a70e56313100bb29eaf8e9bc94fb031b5a595714 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mon-a, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True) 2024-05-03T18:49:07.364 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:49:07 smithi033 podman[126695]: 2024-05-03 18:49:07.362964506 +0000 UTC m=+0.804321361 container cleanup b6180ce9a1aba09fc279a712a70e56313100bb29eaf8e9bc94fb031b5a595714 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-03T18:49:07.649 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:49:07 smithi033 bash[126695]: ceph-5703bba4-0978-11ef-bc97-c7b262605968-mon-a 2024-05-03T18:49:07.907 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 03 18:49:07 smithi033 podman[126707]: 2024-05-03 18:49:07.648244362 +0000 UTC m=+0.857374766 container remove b6180ce9a1aba09fc279a712a70e56313100bb29eaf8e9bc94fb031b5a595714 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux ) 2024-05-03T18:49:07.939 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5703bba4-0978-11ef-bc97-c7b262605968@mon.a.service' 2024-05-03T18:49:08.124 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T18:49:08.125 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-05-03T18:49:08.125 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-05-03T18:49:08.125 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-5703bba4-0978-11ef-bc97-c7b262605968@mon.b 2024-05-03T18:49:08.546 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:49:08 smithi099 systemd[1]: Stopping Ceph mon.b for 5703bba4-0978-11ef-bc97-c7b262605968... 2024-05-03T18:49:09.047 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:49:08 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mon-b[31654]: 2024-05-03T18:49:08.769+0000 7f9d89600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T18:49:09.047 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:49:08 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mon-b[31654]: 2024-05-03T18:49:08.769+0000 7f9d89600700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-05-03T18:49:09.047 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:49:08 smithi099 podman[70652]: 2024-05-03 18:49:08.921618633 +0000 UTC m=+0.310964783 container died 7fd86560ad990d432f5e47fe88649ddb1df88194a39d20914ddf475590fe59fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mon-b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.build-date=20240430, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-03T18:49:09.549 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:49:09 smithi099 podman[70652]: 2024-05-03 18:49:09.477979535 +0000 UTC m=+0.867325695 container cleanup 7fd86560ad990d432f5e47fe88649ddb1df88194a39d20914ddf475590fe59fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mon-b, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-05-03T18:49:09.549 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:49:09 smithi099 bash[70652]: ceph-5703bba4-0978-11ef-bc97-c7b262605968-mon-b 2024-05-03T18:49:10.046 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:49:09 smithi099 podman[70667]: 2024-05-03 18:49:09.729195524 +0000 UTC m=+0.786563863 container remove 7fd86560ad990d432f5e47fe88649ddb1df88194a39d20914ddf475590fe59fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mon-b, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-05-03T18:49:10.081 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5703bba4-0978-11ef-bc97-c7b262605968@mon.b.service' 2024-05-03T18:49:10.547 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:49:10 smithi099 systemd[1]: ceph-5703bba4-0978-11ef-bc97-c7b262605968@mon.b.service: Deactivated successfully. 2024-05-03T18:49:10.547 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:49:10 smithi099 systemd[1]: Stopped Ceph mon.b for 5703bba4-0978-11ef-bc97-c7b262605968. 2024-05-03T18:49:10.547 INFO:journalctl@ceph.mon.b.smithi099.stdout:May 03 18:49:10 smithi099 systemd[1]: ceph-5703bba4-0978-11ef-bc97-c7b262605968@mon.b.service: Consumed 40.802s CPU time. 2024-05-03T18:49:10.968 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T18:49:10.969 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-05-03T18:49:10.969 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-05-03T18:49:10.969 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-5703bba4-0978-11ef-bc97-c7b262605968@mon.c 2024-05-03T18:49:11.384 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:49:11 smithi150 systemd[1]: Stopping Ceph mon.c for 5703bba4-0978-11ef-bc97-c7b262605968... 2024-05-03T18:49:13.134 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:49:12 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mon-c[32479]: 2024-05-03T18:49:12.754+0000 7fefb0a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T18:49:13.134 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:49:12 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mon-c[32479]: 2024-05-03T18:49:12.754+0000 7fefb0a00700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-05-03T18:49:13.134 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:49:12 smithi150 podman[71457]: 2024-05-03 18:49:12.935129131 +0000 UTC m=+0.345639194 container died a45cbbe8663d93fc5e0abb73897e4890ed593f639efa684967df7d9567772571 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mon-c, org.label-schema.build-date=20240430, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-05-03T18:49:13.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:49:13 smithi150 podman[71457]: 2024-05-03 18:49:13.415489514 +0000 UTC m=+0.825999596 container cleanup a45cbbe8663d93fc5e0abb73897e4890ed593f639efa684967df7d9567772571 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mon-c, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-05-03T18:49:13.633 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:49:13 smithi150 bash[71457]: ceph-5703bba4-0978-11ef-bc97-c7b262605968-mon-c 2024-05-03T18:49:14.116 INFO:journalctl@ceph.mon.c.smithi150.stdout:May 03 18:49:13 smithi150 podman[71471]: 2024-05-03 18:49:13.666958648 +0000 UTC m=+0.718364083 container remove a45cbbe8663d93fc5e0abb73897e4890ed593f639efa684967df7d9567772571 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mon-c, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-03T18:49:14.156 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5703bba4-0978-11ef-bc97-c7b262605968@mon.c.service' 2024-05-03T18:49:14.277 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T18:49:14.277 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-05-03T18:49:14.278 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-05-03T18:49:14.278 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-5703bba4-0978-11ef-bc97-c7b262605968@mgr.x 2024-05-03T18:49:14.735 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:49:14 smithi033 systemd[1]: Stopping Ceph mgr.x for 5703bba4-0978-11ef-bc97-c7b262605968... 2024-05-03T18:49:15.235 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:49:14 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:49:14.900+0000 7f528fe00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T18:49:15.236 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:49:14 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x[28572]: 2024-05-03T18:49:14.900+0000 7f528fe00700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-05-03T18:49:15.236 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:49:14 smithi033 podman[126810]: 2024-05-03 18:49:14.99319438 +0000 UTC m=+0.232712393 container died baf6dc704ff570c8ae8af8003fafac25f0d00e15950cade55fec01f17c100d56 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240430, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-03T18:49:18.140 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:49:17 smithi033 podman[126810]: 2024-05-03 18:49:17.85333106 +0000 UTC m=+3.092849046 container cleanup baf6dc704ff570c8ae8af8003fafac25f0d00e15950cade55fec01f17c100d56 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS) 2024-05-03T18:49:18.141 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:49:17 smithi033 bash[126810]: ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x 2024-05-03T18:49:18.485 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 03 18:49:18 smithi033 podman[126824]: 2024-05-03 18:49:18.138532712 +0000 UTC m=+3.126876080 container remove baf6dc704ff570c8ae8af8003fafac25f0d00e15950cade55fec01f17c100d56 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-x, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, io.buildah.version=1.33.5) 2024-05-03T18:49:18.649 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5703bba4-0978-11ef-bc97-c7b262605968@mgr.x.service' 2024-05-03T18:49:18.765 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T18:49:18.765 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-05-03T18:49:18.765 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-05-03T18:49:18.766 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-5703bba4-0978-11ef-bc97-c7b262605968@mgr.y 2024-05-03T18:49:19.296 INFO:journalctl@ceph.mgr.y.smithi099.stdout:May 03 18:49:18 smithi099 systemd[1]: Stopping Ceph mgr.y for 5703bba4-0978-11ef-bc97-c7b262605968... 2024-05-03T18:49:19.796 INFO:journalctl@ceph.mgr.y.smithi099.stdout:May 03 18:49:19 smithi099 podman[70770]: 2024-05-03 18:49:19.439403651 +0000 UTC m=+0.228763101 container died 028f4cdf6284139251cf532de484f4338ba682fa738ca67d359627135bcbc7cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-05-03T18:49:20.252 INFO:journalctl@ceph.mgr.y.smithi099.stdout:May 03 18:49:19 smithi099 podman[70770]: 2024-05-03 18:49:19.966350546 +0000 UTC m=+0.755709969 container cleanup 028f4cdf6284139251cf532de484f4338ba682fa738ca67d359627135bcbc7cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-y, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-05-03T18:49:20.253 INFO:journalctl@ceph.mgr.y.smithi099.stdout:May 03 18:49:19 smithi099 bash[70770]: ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-y 2024-05-03T18:49:20.546 INFO:journalctl@ceph.mgr.y.smithi099.stdout:May 03 18:49:20 smithi099 podman[70785]: 2024-05-03 18:49:20.25118826 +0000 UTC m=+0.799643448 container remove 028f4cdf6284139251cf532de484f4338ba682fa738ca67d359627135bcbc7cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-y, org.label-schema.build-date=20240430, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0) 2024-05-03T18:49:20.546 INFO:journalctl@ceph.mgr.y.smithi099.stdout:May 03 18:49:20 smithi099 systemd[1]: ceph-5703bba4-0978-11ef-bc97-c7b262605968@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-05-03T18:49:20.738 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5703bba4-0978-11ef-bc97-c7b262605968@mgr.y.service' 2024-05-03T18:49:20.836 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T18:49:20.836 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-05-03T18:49:20.837 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-05-03T18:49:20.837 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-5703bba4-0978-11ef-bc97-c7b262605968@mgr.z 2024-05-03T18:49:21.384 INFO:journalctl@ceph.mgr.z.smithi150.stdout:May 03 18:49:20 smithi150 systemd[1]: Stopping Ceph mgr.z for 5703bba4-0978-11ef-bc97-c7b262605968... 2024-05-03T18:49:21.803 INFO:journalctl@ceph.mgr.z.smithi150.stdout:May 03 18:49:21 smithi150 podman[71591]: 2024-05-03 18:49:21.534882502 +0000 UTC m=+0.259965017 container died aa8e47cd8dde7fb55087c2c58b1e8296b5a03f8e0c48d945d25be78d1bd495d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-z, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-03T18:49:22.133 INFO:journalctl@ceph.mgr.z.smithi150.stdout:May 03 18:49:21 smithi150 podman[71591]: 2024-05-03 18:49:21.979977719 +0000 UTC m=+0.705060177 container cleanup aa8e47cd8dde7fb55087c2c58b1e8296b5a03f8e0c48d945d25be78d1bd495d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-z, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-05-03T18:49:22.133 INFO:journalctl@ceph.mgr.z.smithi150.stdout:May 03 18:49:21 smithi150 bash[71591]: ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-z 2024-05-03T18:49:22.633 INFO:journalctl@ceph.mgr.z.smithi150.stdout:May 03 18:49:22 smithi150 podman[71605]: 2024-05-03 18:49:22.223379551 +0000 UTC m=+0.670753356 container remove aa8e47cd8dde7fb55087c2c58b1e8296b5a03f8e0c48d945d25be78d1bd495d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mgr-z, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-05-03T18:49:22.634 INFO:journalctl@ceph.mgr.z.smithi150.stdout:May 03 18:49:22 smithi150 systemd[1]: ceph-5703bba4-0978-11ef-bc97-c7b262605968@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-05-03T18:49:22.695 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5703bba4-0978-11ef-bc97-c7b262605968@mgr.z.service' 2024-05-03T18:49:22.798 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T18:49:22.804 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-05-03T18:49:22.804 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-05-03T18:49:22.804 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-5703bba4-0978-11ef-bc97-c7b262605968@osd.0 2024-05-03T18:49:23.235 INFO:journalctl@ceph.osd.0.smithi033.stdout:May 03 18:49:22 smithi033 systemd[1]: Stopping Ceph osd.0 for 5703bba4-0978-11ef-bc97-c7b262605968... 2024-05-03T18:49:23.736 INFO:journalctl@ceph.osd.0.smithi033.stdout:May 03 18:49:23 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-0[42993]: 2024-05-03T18:49:23.391+0000 7f45a9000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T18:49:23.736 INFO:journalctl@ceph.osd.0.smithi033.stdout:May 03 18:49:23 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-0[42993]: 2024-05-03T18:49:23.391+0000 7f45a9000700 -1 osd.0 126 *** Got signal Terminated *** 2024-05-03T18:49:23.736 INFO:journalctl@ceph.osd.0.smithi033.stdout:May 03 18:49:23 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-0[42993]: 2024-05-03T18:49:23.391+0000 7f45a9000700 -1 osd.0 126 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-03T18:49:29.850 INFO:journalctl@ceph.osd.0.smithi033.stdout:May 03 18:49:29 smithi033 podman[126930]: 2024-05-03 18:49:29.510759574 +0000 UTC m=+6.254368381 container died 9fc394357a1cbb94e0dd4521eb31178215f2ec05278b126de2c629ebf9b7ca1e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-0, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-05-03T18:49:30.107 INFO:journalctl@ceph.osd.0.smithi033.stdout:May 03 18:49:30 smithi033 podman[126930]: 2024-05-03 18:49:30.075516697 +0000 UTC m=+6.819125520 container cleanup 9fc394357a1cbb94e0dd4521eb31178215f2ec05278b126de2c629ebf9b7ca1e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-0, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-05-03T18:49:30.107 INFO:journalctl@ceph.osd.0.smithi033.stdout:May 03 18:49:30 smithi033 bash[126930]: ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-0 2024-05-03T18:49:30.485 INFO:journalctl@ceph.osd.0.smithi033.stdout:May 03 18:49:30 smithi033 podman[126950]: 2024-05-03 18:49:30.352796579 +0000 UTC m=+0.826624224 container remove 9fc394357a1cbb94e0dd4521eb31178215f2ec05278b126de2c629ebf9b7ca1e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-0, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2) 2024-05-03T18:49:31.485 INFO:journalctl@ceph.osd.0.smithi033.stdout:May 03 18:49:31 smithi033 podman[127035]: 2024-05-03 18:49:31.090722808 +0000 UTC m=+0.038530453 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T18:49:31.485 INFO:journalctl@ceph.osd.0.smithi033.stdout:May 03 18:49:31 smithi033 podman[127035]: 2024-05-03 18:49:31.212086354 +0000 UTC m=+0.159894029 container create 819db4a27e91cbfb85b06e859a4071f2ce860337425ec36c2f06e3989a14334a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-05-03T18:49:31.765 INFO:journalctl@ceph.osd.0.smithi033.stdout:May 03 18:49:31 smithi033 podman[127035]: 2024-05-03 18:49:31.499060825 +0000 UTC m=+0.446868557 container init 819db4a27e91cbfb85b06e859a4071f2ce860337425ec36c2f06e3989a14334a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430) 2024-05-03T18:49:31.765 INFO:journalctl@ceph.osd.0.smithi033.stdout:May 03 18:49:31 smithi033 podman[127035]: 2024-05-03 18:49:31.509754662 +0000 UTC m=+0.457562341 container start 819db4a27e91cbfb85b06e859a4071f2ce860337425ec36c2f06e3989a14334a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True) 2024-05-03T18:49:31.765 INFO:journalctl@ceph.osd.0.smithi033.stdout:May 03 18:49:31 smithi033 podman[127035]: 2024-05-03 18:49:31.631625956 +0000 UTC m=+0.579433601 container attach 819db4a27e91cbfb85b06e859a4071f2ce860337425ec36c2f06e3989a14334a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240430, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-05-03T18:49:32.735 INFO:journalctl@ceph.osd.0.smithi033.stdout:May 03 18:49:32 smithi033 podman[127035]: 2024-05-03 18:49:32.439102401 +0000 UTC m=+1.386910044 container died 819db4a27e91cbfb85b06e859a4071f2ce860337425ec36c2f06e3989a14334a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-05-03T18:49:33.313 INFO:journalctl@ceph.osd.0.smithi033.stdout:May 03 18:49:33 smithi033 podman[127072]: 2024-05-03 18:49:33.261467896 +0000 UTC m=+0.806219746 container remove 819db4a27e91cbfb85b06e859a4071f2ce860337425ec36c2f06e3989a14334a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-0-deactivate, ceph=True, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-03T18:49:33.313 INFO:journalctl@ceph.osd.0.smithi033.stdout:May 03 18:49:33 smithi033 systemd[1]: ceph-5703bba4-0978-11ef-bc97-c7b262605968@osd.0.service: Deactivated successfully. 2024-05-03T18:49:33.313 INFO:journalctl@ceph.osd.0.smithi033.stdout:May 03 18:49:33 smithi033 systemd[1]: Stopped Ceph osd.0 for 5703bba4-0978-11ef-bc97-c7b262605968. 2024-05-03T18:49:33.313 INFO:journalctl@ceph.osd.0.smithi033.stdout:May 03 18:49:33 smithi033 systemd[1]: ceph-5703bba4-0978-11ef-bc97-c7b262605968@osd.0.service: Consumed 10min 27.463s CPU time. 2024-05-03T18:49:33.317 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5703bba4-0978-11ef-bc97-c7b262605968@osd.0.service' 2024-05-03T18:49:33.432 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T18:49:33.432 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-05-03T18:49:33.432 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-05-03T18:49:33.432 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-5703bba4-0978-11ef-bc97-c7b262605968@osd.1 2024-05-03T18:49:33.796 INFO:journalctl@ceph.osd.1.smithi099.stdout:May 03 18:49:33 smithi099 systemd[1]: Stopping Ceph osd.1 for 5703bba4-0978-11ef-bc97-c7b262605968... 2024-05-03T18:49:34.297 INFO:journalctl@ceph.osd.1.smithi099.stdout:May 03 18:49:34 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-1[36254]: 2024-05-03T18:49:34.008+0000 7fe350c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T18:49:34.297 INFO:journalctl@ceph.osd.1.smithi099.stdout:May 03 18:49:34 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-1[36254]: 2024-05-03T18:49:34.008+0000 7fe350c00700 -1 osd.1 126 *** Got signal Terminated *** 2024-05-03T18:49:34.297 INFO:journalctl@ceph.osd.1.smithi099.stdout:May 03 18:49:34 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-1[36254]: 2024-05-03T18:49:34.008+0000 7fe350c00700 -1 osd.1 126 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-03T18:49:40.297 INFO:journalctl@ceph.osd.1.smithi099.stdout:May 03 18:49:40 smithi099 podman[70894]: 2024-05-03 18:49:40.029356886 +0000 UTC m=+6.153653014 container died 1ee7cabe9359414363dcebbb2aba9d45c0cce0f0f8c041bed4c98dd51d08de13 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-1, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, io.buildah.version=1.33.5) 2024-05-03T18:49:40.763 INFO:journalctl@ceph.osd.1.smithi099.stdout:May 03 18:49:40 smithi099 podman[70894]: 2024-05-03 18:49:40.527476692 +0000 UTC m=+6.651772860 container cleanup 1ee7cabe9359414363dcebbb2aba9d45c0cce0f0f8c041bed4c98dd51d08de13 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-1, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-05-03T18:49:40.764 INFO:journalctl@ceph.osd.1.smithi099.stdout:May 03 18:49:40 smithi099 bash[70894]: ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-1 2024-05-03T18:49:41.046 INFO:journalctl@ceph.osd.1.smithi099.stdout:May 03 18:49:40 smithi099 podman[70925]: 2024-05-03 18:49:40.76207612 +0000 UTC m=+0.708544940 container remove 1ee7cabe9359414363dcebbb2aba9d45c0cce0f0f8c041bed4c98dd51d08de13 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-1, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, maintainer=Guillaume Abrioux ) 2024-05-03T18:49:41.918 INFO:journalctl@ceph.osd.1.smithi099.stdout:May 03 18:49:41 smithi099 podman[71012]: 2024-05-03 18:49:41.506808733 +0000 UTC m=+0.035281484 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T18:49:41.919 INFO:journalctl@ceph.osd.1.smithi099.stdout:May 03 18:49:41 smithi099 podman[71012]: 2024-05-03 18:49:41.621367052 +0000 UTC m=+0.149839831 container create bc5f69cf143a17056284814999cc4097a0b905c26edcb6b165a97a895b708414 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240430, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-05-03T18:49:41.919 INFO:journalctl@ceph.osd.1.smithi099.stdout:May 03 18:49:41 smithi099 podman[71012]: 2024-05-03 18:49:41.916987292 +0000 UTC m=+0.445460112 container init bc5f69cf143a17056284814999cc4097a0b905c26edcb6b165a97a895b708414 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-1-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-05-03T18:49:42.297 INFO:journalctl@ceph.osd.1.smithi099.stdout:May 03 18:49:41 smithi099 podman[71012]: 2024-05-03 18:49:41.92831425 +0000 UTC m=+0.456786996 container start bc5f69cf143a17056284814999cc4097a0b905c26edcb6b165a97a895b708414 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True) 2024-05-03T18:49:42.297 INFO:journalctl@ceph.osd.1.smithi099.stdout:May 03 18:49:42 smithi099 podman[71012]: 2024-05-03 18:49:42.099080694 +0000 UTC m=+0.627553492 container attach bc5f69cf143a17056284814999cc4097a0b905c26edcb6b165a97a895b708414 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-03T18:49:43.502 INFO:journalctl@ceph.osd.1.smithi099.stdout:May 03 18:49:43 smithi099 podman[71012]: 2024-05-03 18:49:43.082400122 +0000 UTC m=+1.610872904 container died bc5f69cf143a17056284814999cc4097a0b905c26edcb6b165a97a895b708414 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-1-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.build-date=20240430, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-05-03T18:49:43.954 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5703bba4-0978-11ef-bc97-c7b262605968@osd.1.service' 2024-05-03T18:49:44.061 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T18:49:44.062 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-05-03T18:49:44.062 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-05-03T18:49:44.062 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-5703bba4-0978-11ef-bc97-c7b262605968@osd.2 2024-05-03T18:49:44.633 INFO:journalctl@ceph.osd.2.smithi150.stdout:May 03 18:49:44 smithi150 systemd[1]: Stopping Ceph osd.2 for 5703bba4-0978-11ef-bc97-c7b262605968... 2024-05-03T18:49:45.133 INFO:journalctl@ceph.osd.2.smithi150.stdout:May 03 18:49:44 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-2[37659]: 2024-05-03T18:49:44.682+0000 7f5e3d800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T18:49:45.134 INFO:journalctl@ceph.osd.2.smithi150.stdout:May 03 18:49:44 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-2[37659]: 2024-05-03T18:49:44.682+0000 7f5e3d800700 -1 osd.2 126 *** Got signal Terminated *** 2024-05-03T18:49:45.134 INFO:journalctl@ceph.osd.2.smithi150.stdout:May 03 18:49:44 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-2[37659]: 2024-05-03T18:49:44.682+0000 7f5e3d800700 -1 osd.2 126 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-03T18:49:50.633 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:49:50 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:49:50.273+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:25.480913+0000 front 2024-05-03T18:49:25.480403+0000 (oldest deadline 2024-05-03T18:49:50.180152+0000) 2024-05-03T18:49:51.076 INFO:journalctl@ceph.osd.2.smithi150.stdout:May 03 18:49:50 smithi150 podman[71715]: 2024-05-03 18:49:50.782699025 +0000 UTC m=+6.292860035 container died c200ebbca765e4249b98cf4c26f57f604c8612a342cc8805b858c6718e13f74e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.build-date=20240430, ceph=True, io.buildah.version=1.33.5) 2024-05-03T18:49:51.076 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:49:50 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:49:50.881+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:24.877883+0000 front 2024-05-03T18:49:24.877625+0000 (oldest deadline 2024-05-03T18:49:50.777881+0000) 2024-05-03T18:49:51.383 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:49:51 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:49:51.240+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:25.480913+0000 front 2024-05-03T18:49:25.480403+0000 (oldest deadline 2024-05-03T18:49:50.180152+0000) 2024-05-03T18:49:51.383 INFO:journalctl@ceph.osd.2.smithi150.stdout:May 03 18:49:51 smithi150 podman[71715]: 2024-05-03 18:49:51.225890266 +0000 UTC m=+6.736051331 container cleanup c200ebbca765e4249b98cf4c26f57f604c8612a342cc8805b858c6718e13f74e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-2, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS) 2024-05-03T18:49:51.383 INFO:journalctl@ceph.osd.2.smithi150.stdout:May 03 18:49:51 smithi150 bash[71715]: ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-2 2024-05-03T18:49:51.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:49:51 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:49:51.206+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:26.972061+0000 front 2024-05-03T18:49:26.972448+0000 (oldest deadline 2024-05-03T18:49:50.472344+0000) 2024-05-03T18:49:51.866 INFO:journalctl@ceph.osd.2.smithi150.stdout:May 03 18:49:51 smithi150 podman[71733]: 2024-05-03 18:49:51.519425378 +0000 UTC m=+0.721178732 container remove c200ebbca765e4249b98cf4c26f57f604c8612a342cc8805b858c6718e13f74e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.build-date=20240430) 2024-05-03T18:49:52.133 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:49:51 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:49:51.864+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:24.877883+0000 front 2024-05-03T18:49:24.877625+0000 (oldest deadline 2024-05-03T18:49:50.777881+0000) 2024-05-03T18:49:52.546 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:49:52 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:49:52.214+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:26.972061+0000 front 2024-05-03T18:49:26.972448+0000 (oldest deadline 2024-05-03T18:49:50.472344+0000) 2024-05-03T18:49:52.577 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:49:52 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:49:52.192+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:25.480913+0000 front 2024-05-03T18:49:25.480403+0000 (oldest deadline 2024-05-03T18:49:50.180152+0000) 2024-05-03T18:49:52.578 INFO:journalctl@ceph.osd.2.smithi150.stdout:May 03 18:49:52 smithi150 podman[71820]: 2024-05-03 18:49:52.274628199 +0000 UTC m=+0.036410870 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T18:49:52.578 INFO:journalctl@ceph.osd.2.smithi150.stdout:May 03 18:49:52 smithi150 podman[71820]: 2024-05-03 18:49:52.412186497 +0000 UTC m=+0.173969205 container create 2fa347c7ecebc29a757adb6e7f15c6eab969e5d0afb74d48d34ee5896ad757e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0) 2024-05-03T18:49:52.841 INFO:journalctl@ceph.osd.2.smithi150.stdout:May 03 18:49:52 smithi150 podman[71820]: 2024-05-03 18:49:52.674141422 +0000 UTC m=+0.435924150 container init 2fa347c7ecebc29a757adb6e7f15c6eab969e5d0afb74d48d34ee5896ad757e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True) 2024-05-03T18:49:52.842 INFO:journalctl@ceph.osd.2.smithi150.stdout:May 03 18:49:52 smithi150 podman[71820]: 2024-05-03 18:49:52.684815681 +0000 UTC m=+0.446598415 container start 2fa347c7ecebc29a757adb6e7f15c6eab969e5d0afb74d48d34ee5896ad757e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430) 2024-05-03T18:49:52.842 INFO:journalctl@ceph.osd.2.smithi150.stdout:May 03 18:49:52 smithi150 podman[71820]: 2024-05-03 18:49:52.83980119 +0000 UTC m=+0.601583958 container attach 2fa347c7ecebc29a757adb6e7f15c6eab969e5d0afb74d48d34ee5896ad757e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240430) 2024-05-03T18:49:53.133 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:49:52 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:49:52.913+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:24.877883+0000 front 2024-05-03T18:49:24.877625+0000 (oldest deadline 2024-05-03T18:49:50.777881+0000) 2024-05-03T18:49:53.485 INFO:journalctl@ceph.osd.3.smithi033.stdout:May 03 18:49:53 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-3[49778]: 2024-05-03T18:49:53.003+0000 7f3239800700 -1 osd.3 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:27.631499+0000 front 2024-05-03T18:49:27.631693+0000 (oldest deadline 2024-05-03T18:49:52.331642+0000) 2024-05-03T18:49:53.546 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:49:53 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:49:53.211+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:26.972061+0000 front 2024-05-03T18:49:26.972448+0000 (oldest deadline 2024-05-03T18:49:50.472344+0000) 2024-05-03T18:49:53.626 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:49:53 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:49:53.201+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:25.480913+0000 front 2024-05-03T18:49:25.480403+0000 (oldest deadline 2024-05-03T18:49:50.180152+0000) 2024-05-03T18:49:53.883 INFO:journalctl@ceph.osd.5.smithi150.stdout:May 03 18:49:53 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-5[44561]: 2024-05-03T18:49:53.625+0000 7fdfa7000700 -1 osd.5 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:28.083853+0000 front 2024-05-03T18:49:28.084464+0000 (oldest deadline 2024-05-03T18:49:52.685392+0000) 2024-05-03T18:49:53.883 INFO:journalctl@ceph.osd.2.smithi150.stdout:May 03 18:49:53 smithi150 podman[71820]: 2024-05-03 18:49:53.660205192 +0000 UTC m=+1.421987890 container died 2fa347c7ecebc29a757adb6e7f15c6eab969e5d0afb74d48d34ee5896ad757e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-03T18:49:53.985 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:49:53 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:49:53.507+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:27.608012+0000 front 2024-05-03T18:49:27.608078+0000 (oldest deadline 2024-05-03T18:49:53.508290+0000) 2024-05-03T18:49:54.219 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:49:53 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:49:53.891+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:24.877883+0000 front 2024-05-03T18:49:24.877625+0000 (oldest deadline 2024-05-03T18:49:50.777881+0000) 2024-05-03T18:49:54.383 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5703bba4-0978-11ef-bc97-c7b262605968@osd.2.service' 2024-05-03T18:49:54.484 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:49:54 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:49:54.218+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:25.480913+0000 front 2024-05-03T18:49:25.480403+0000 (oldest deadline 2024-05-03T18:49:50.180152+0000) 2024-05-03T18:49:54.485 INFO:journalctl@ceph.osd.3.smithi033.stdout:May 03 18:49:54 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-3[49778]: 2024-05-03T18:49:54.018+0000 7f3239800700 -1 osd.3 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:27.631499+0000 front 2024-05-03T18:49:27.631693+0000 (oldest deadline 2024-05-03T18:49:52.331642+0000) 2024-05-03T18:49:54.489 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T18:49:54.489 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-05-03T18:49:54.489 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-05-03T18:49:54.490 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-5703bba4-0978-11ef-bc97-c7b262605968@osd.3 2024-05-03T18:49:54.546 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:49:54 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:49:54.208+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:26.972061+0000 front 2024-05-03T18:49:26.972448+0000 (oldest deadline 2024-05-03T18:49:50.472344+0000) 2024-05-03T18:49:54.883 INFO:journalctl@ceph.osd.5.smithi150.stdout:May 03 18:49:54 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-5[44561]: 2024-05-03T18:49:54.596+0000 7fdfa7000700 -1 osd.5 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:28.083853+0000 front 2024-05-03T18:49:28.084464+0000 (oldest deadline 2024-05-03T18:49:52.685392+0000) 2024-05-03T18:49:54.985 INFO:journalctl@ceph.osd.3.smithi033.stdout:May 03 18:49:54 smithi033 systemd[1]: Stopping Ceph osd.3 for 5703bba4-0978-11ef-bc97-c7b262605968... 2024-05-03T18:49:54.985 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:49:54 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:49:54.535+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:27.608012+0000 front 2024-05-03T18:49:27.608078+0000 (oldest deadline 2024-05-03T18:49:53.508290+0000) 2024-05-03T18:49:55.251 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:49:54 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:49:54.918+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:24.877883+0000 front 2024-05-03T18:49:24.877625+0000 (oldest deadline 2024-05-03T18:49:50.777881+0000) 2024-05-03T18:49:55.485 INFO:journalctl@ceph.osd.3.smithi033.stdout:May 03 18:49:55 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-3[49778]: 2024-05-03T18:49:55.041+0000 7f3239800700 -1 osd.3 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:27.631499+0000 front 2024-05-03T18:49:27.631693+0000 (oldest deadline 2024-05-03T18:49:52.331642+0000) 2024-05-03T18:49:55.485 INFO:journalctl@ceph.osd.3.smithi033.stdout:May 03 18:49:55 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-3[49778]: 2024-05-03T18:49:55.108+0000 7f323e000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T18:49:55.485 INFO:journalctl@ceph.osd.3.smithi033.stdout:May 03 18:49:55 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-3[49778]: 2024-05-03T18:49:55.108+0000 7f323e000700 -1 osd.3 126 *** Got signal Terminated *** 2024-05-03T18:49:55.485 INFO:journalctl@ceph.osd.3.smithi033.stdout:May 03 18:49:55 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-3[49778]: 2024-05-03T18:49:55.108+0000 7f323e000700 -1 osd.3 126 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-03T18:49:55.546 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:49:55 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:49:55.211+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:26.972061+0000 front 2024-05-03T18:49:26.972448+0000 (oldest deadline 2024-05-03T18:49:50.472344+0000) 2024-05-03T18:49:55.567 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:49:55 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:49:55.250+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:25.480913+0000 front 2024-05-03T18:49:25.480403+0000 (oldest deadline 2024-05-03T18:49:50.180152+0000) 2024-05-03T18:49:55.871 INFO:journalctl@ceph.osd.5.smithi150.stdout:May 03 18:49:55 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-5[44561]: 2024-05-03T18:49:55.566+0000 7fdfa7000700 -1 osd.5 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:28.083853+0000 front 2024-05-03T18:49:28.084464+0000 (oldest deadline 2024-05-03T18:49:52.685392+0000) 2024-05-03T18:49:55.985 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:49:55 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:49:55.491+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:27.608012+0000 front 2024-05-03T18:49:27.608078+0000 (oldest deadline 2024-05-03T18:49:53.508290+0000) 2024-05-03T18:49:56.133 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:49:55 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:49:55.870+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:24.877883+0000 front 2024-05-03T18:49:24.877625+0000 (oldest deadline 2024-05-03T18:49:50.777881+0000) 2024-05-03T18:49:56.486 INFO:journalctl@ceph.osd.3.smithi033.stdout:May 03 18:49:56 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-3[49778]: 2024-05-03T18:49:56.031+0000 7f3239800700 -1 osd.3 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:27.631499+0000 front 2024-05-03T18:49:27.631693+0000 (oldest deadline 2024-05-03T18:49:52.331642+0000) 2024-05-03T18:49:56.521 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:49:56 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:49:56.246+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:25.480913+0000 front 2024-05-03T18:49:25.480403+0000 (oldest deadline 2024-05-03T18:49:50.180152+0000) 2024-05-03T18:49:56.546 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:49:56 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:49:56.196+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:26.972061+0000 front 2024-05-03T18:49:26.972448+0000 (oldest deadline 2024-05-03T18:49:50.472344+0000) 2024-05-03T18:49:56.861 INFO:journalctl@ceph.osd.5.smithi150.stdout:May 03 18:49:56 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-5[44561]: 2024-05-03T18:49:56.520+0000 7fdfa7000700 -1 osd.5 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:28.083853+0000 front 2024-05-03T18:49:28.084464+0000 (oldest deadline 2024-05-03T18:49:52.685392+0000) 2024-05-03T18:49:56.985 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:49:56 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:49:56.527+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:27.608012+0000 front 2024-05-03T18:49:27.608078+0000 (oldest deadline 2024-05-03T18:49:53.508290+0000) 2024-05-03T18:49:57.133 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:49:56 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:49:56.860+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:24.877883+0000 front 2024-05-03T18:49:24.877625+0000 (oldest deadline 2024-05-03T18:49:50.777881+0000) 2024-05-03T18:49:57.481 INFO:journalctl@ceph.osd.3.smithi033.stdout:May 03 18:49:57 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-3[49778]: 2024-05-03T18:49:57.024+0000 7f3239800700 -1 osd.3 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:27.631499+0000 front 2024-05-03T18:49:27.631693+0000 (oldest deadline 2024-05-03T18:49:52.331642+0000) 2024-05-03T18:49:57.514 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:49:57 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:49:57.205+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:25.480913+0000 front 2024-05-03T18:49:25.480403+0000 (oldest deadline 2024-05-03T18:49:50.180152+0000) 2024-05-03T18:49:57.546 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:49:57 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:49:57.239+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:26.972061+0000 front 2024-05-03T18:49:26.972448+0000 (oldest deadline 2024-05-03T18:49:50.472344+0000) 2024-05-03T18:49:57.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:49:57 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:49:57.478+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:27.608012+0000 front 2024-05-03T18:49:27.608078+0000 (oldest deadline 2024-05-03T18:49:53.508290+0000) 2024-05-03T18:49:57.832 INFO:journalctl@ceph.osd.5.smithi150.stdout:May 03 18:49:57 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-5[44561]: 2024-05-03T18:49:57.512+0000 7fdfa7000700 -1 osd.5 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:28.083853+0000 front 2024-05-03T18:49:28.084464+0000 (oldest deadline 2024-05-03T18:49:52.685392+0000) 2024-05-03T18:49:58.133 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:49:57 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:49:57.831+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:24.877883+0000 front 2024-05-03T18:49:24.877625+0000 (oldest deadline 2024-05-03T18:49:50.777881+0000) 2024-05-03T18:49:58.466 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:49:58 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:49:58.180+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:25.480913+0000 front 2024-05-03T18:49:25.480403+0000 (oldest deadline 2024-05-03T18:49:50.180152+0000) 2024-05-03T18:49:58.485 INFO:journalctl@ceph.osd.3.smithi033.stdout:May 03 18:49:58 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-3[49778]: 2024-05-03T18:49:58.009+0000 7f3239800700 -1 osd.3 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:27.631499+0000 front 2024-05-03T18:49:27.631693+0000 (oldest deadline 2024-05-03T18:49:52.331642+0000) 2024-05-03T18:49:58.546 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:49:58 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:49:58.284+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:26.972061+0000 front 2024-05-03T18:49:26.972448+0000 (oldest deadline 2024-05-03T18:49:50.472344+0000) 2024-05-03T18:49:58.785 INFO:journalctl@ceph.osd.5.smithi150.stdout:May 03 18:49:58 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-5[44561]: 2024-05-03T18:49:58.464+0000 7fdfa7000700 -1 osd.5 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:28.083853+0000 front 2024-05-03T18:49:28.084464+0000 (oldest deadline 2024-05-03T18:49:52.685392+0000) 2024-05-03T18:49:58.985 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:49:58 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:49:58.483+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:27.608012+0000 front 2024-05-03T18:49:27.608078+0000 (oldest deadline 2024-05-03T18:49:53.508290+0000) 2024-05-03T18:49:59.133 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:49:58 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:49:58.782+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:24.877883+0000 front 2024-05-03T18:49:24.877625+0000 (oldest deadline 2024-05-03T18:49:50.777881+0000) 2024-05-03T18:49:59.470 INFO:journalctl@ceph.osd.3.smithi033.stdout:May 03 18:49:59 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-3[49778]: 2024-05-03T18:49:59.028+0000 7f3239800700 -1 osd.3 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:27.631499+0000 front 2024-05-03T18:49:27.631693+0000 (oldest deadline 2024-05-03T18:49:52.331642+0000) 2024-05-03T18:49:59.508 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:49:59 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:49:59.199+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:25.480913+0000 front 2024-05-03T18:49:25.480403+0000 (oldest deadline 2024-05-03T18:49:50.180152+0000) 2024-05-03T18:49:59.546 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:49:59 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:49:59.257+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:26.972061+0000 front 2024-05-03T18:49:26.972448+0000 (oldest deadline 2024-05-03T18:49:50.472344+0000) 2024-05-03T18:49:59.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:49:59 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:49:59.465+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:27.608012+0000 front 2024-05-03T18:49:27.608078+0000 (oldest deadline 2024-05-03T18:49:53.508290+0000) 2024-05-03T18:49:59.883 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:49:59 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:49:59.750+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:24.877883+0000 front 2024-05-03T18:49:24.877625+0000 (oldest deadline 2024-05-03T18:49:50.777881+0000) 2024-05-03T18:49:59.883 INFO:journalctl@ceph.osd.5.smithi150.stdout:May 03 18:49:59 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-5[44561]: 2024-05-03T18:49:59.506+0000 7fdfa7000700 -1 osd.5 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:28.083853+0000 front 2024-05-03T18:49:28.084464+0000 (oldest deadline 2024-05-03T18:49:52.685392+0000) 2024-05-03T18:50:00.320 INFO:journalctl@ceph.osd.3.smithi033.stdout:May 03 18:50:00 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-3[49778]: 2024-05-03T18:50:00.037+0000 7f3239800700 -1 osd.3 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:27.631499+0000 front 2024-05-03T18:49:27.631693+0000 (oldest deadline 2024-05-03T18:49:52.331642+0000) 2024-05-03T18:50:00.516 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:00 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:00.157+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:25.480913+0000 front 2024-05-03T18:49:25.480403+0000 (oldest deadline 2024-05-03T18:49:50.180152+0000) 2024-05-03T18:50:00.546 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:00 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:00.208+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:26.972061+0000 front 2024-05-03T18:49:26.972448+0000 (oldest deadline 2024-05-03T18:49:50.472344+0000) 2024-05-03T18:50:00.726 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:00 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:00.447+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:27.608012+0000 front 2024-05-03T18:49:27.608078+0000 (oldest deadline 2024-05-03T18:49:53.508290+0000) 2024-05-03T18:50:00.883 INFO:journalctl@ceph.osd.5.smithi150.stdout:May 03 18:50:00 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-5[44561]: 2024-05-03T18:50:00.513+0000 7fdfa7000700 -1 osd.5 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:28.083853+0000 front 2024-05-03T18:49:28.084464+0000 (oldest deadline 2024-05-03T18:49:52.685392+0000) 2024-05-03T18:50:00.883 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:00 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:00.718+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:24.877883+0000 front 2024-05-03T18:49:24.877625+0000 (oldest deadline 2024-05-03T18:49:50.777881+0000) 2024-05-03T18:50:01.512 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:01 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:01.160+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:25.480913+0000 front 2024-05-03T18:49:25.480403+0000 (oldest deadline 2024-05-03T18:49:50.180152+0000) 2024-05-03T18:50:01.546 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:01 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:01.188+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:26.972061+0000 front 2024-05-03T18:49:26.972448+0000 (oldest deadline 2024-05-03T18:49:50.472344+0000) 2024-05-03T18:50:01.696 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:01 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:01.425+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:27.608012+0000 front 2024-05-03T18:49:27.608078+0000 (oldest deadline 2024-05-03T18:49:53.508290+0000) 2024-05-03T18:50:01.697 INFO:journalctl@ceph.osd.3.smithi033.stdout:May 03 18:50:01 smithi033 podman[127153]: 2024-05-03 18:50:01.361355814 +0000 UTC m=+6.391500634 container died a18ce44dad59b54a59d43fa0d4cb39ff38fb130810acc788e1d3dfa49c30d31a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-3, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-05-03T18:50:01.883 INFO:journalctl@ceph.osd.5.smithi150.stdout:May 03 18:50:01 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-5[44561]: 2024-05-03T18:50:01.510+0000 7fdfa7000700 -1 osd.5 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:28.083853+0000 front 2024-05-03T18:49:28.084464+0000 (oldest deadline 2024-05-03T18:49:52.685392+0000) 2024-05-03T18:50:01.883 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:01 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:01.689+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:24.877883+0000 front 2024-05-03T18:49:24.877625+0000 (oldest deadline 2024-05-03T18:49:50.777881+0000) 2024-05-03T18:50:01.951 INFO:journalctl@ceph.osd.3.smithi033.stdout:May 03 18:50:01 smithi033 podman[127153]: 2024-05-03 18:50:01.921209946 +0000 UTC m=+6.951354794 container cleanup a18ce44dad59b54a59d43fa0d4cb39ff38fb130810acc788e1d3dfa49c30d31a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-3, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-05-03T18:50:01.951 INFO:journalctl@ceph.osd.3.smithi033.stdout:May 03 18:50:01 smithi033 bash[127153]: ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-3 2024-05-03T18:50:02.464 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:02 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:02.197+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:25.480913+0000 front 2024-05-03T18:49:25.480403+0000 (oldest deadline 2024-05-03T18:49:50.180152+0000) 2024-05-03T18:50:02.485 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:02 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:02.463+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:27.608012+0000 front 2024-05-03T18:49:27.608078+0000 (oldest deadline 2024-05-03T18:49:53.508290+0000) 2024-05-03T18:50:02.485 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:02 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:02.463+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:38.811037+0000 front 2024-05-03T18:49:38.811219+0000 (oldest deadline 2024-05-03T18:50:01.611951+0000) 2024-05-03T18:50:02.485 INFO:journalctl@ceph.osd.3.smithi033.stdout:May 03 18:50:02 smithi033 podman[127183]: 2024-05-03 18:50:02.218833636 +0000 UTC m=+0.834800318 container remove a18ce44dad59b54a59d43fa0d4cb39ff38fb130810acc788e1d3dfa49c30d31a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-3, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-05-03T18:50:02.546 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:02 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:02.203+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:26.972061+0000 front 2024-05-03T18:49:26.972448+0000 (oldest deadline 2024-05-03T18:49:50.472344+0000) 2024-05-03T18:50:02.546 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:02 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:02.203+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:37.476081+0000 front 2024-05-03T18:49:37.476172+0000 (oldest deadline 2024-05-03T18:50:01.576192+0000) 2024-05-03T18:50:02.547 INFO:journalctl@ceph.osd.4.smithi099.stdout:May 03 18:50:02 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-4[43040]: 2024-05-03T18:50:02.205+0000 7fc4bf600700 -1 osd.4 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:34.388247+0000 front 2024-05-03T18:49:34.388354+0000 (oldest deadline 2024-05-03T18:50:01.989026+0000) 2024-05-03T18:50:02.715 INFO:journalctl@ceph.osd.5.smithi150.stdout:May 03 18:50:02 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-5[44561]: 2024-05-03T18:50:02.462+0000 7fdfa7000700 -1 osd.5 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:28.083853+0000 front 2024-05-03T18:49:28.084464+0000 (oldest deadline 2024-05-03T18:49:52.685392+0000) 2024-05-03T18:50:03.046 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:02 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7[49675]: 2024-05-03T18:50:02.728+0000 7ff8ace00700 -1 osd.7 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:36.774719+0000 front 2024-05-03T18:49:36.774769+0000 (oldest deadline 2024-05-03T18:50:02.674865+0000) 2024-05-03T18:50:03.133 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:02 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:02.713+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:24.877883+0000 front 2024-05-03T18:49:24.877625+0000 (oldest deadline 2024-05-03T18:49:50.777881+0000) 2024-05-03T18:50:03.350 INFO:journalctl@ceph.osd.3.smithi033.stdout:May 03 18:50:03 smithi033 podman[127277]: 2024-05-03 18:50:03.034954288 +0000 UTC m=+0.038932284 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T18:50:03.350 INFO:journalctl@ceph.osd.3.smithi033.stdout:May 03 18:50:03 smithi033 podman[127277]: 2024-05-03 18:50:03.165786634 +0000 UTC m=+0.169764645 container create bb78fd058547c7b2378aad7bfbb75bcf9ddd5075a4877c542fa25339fd5dac79 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-05-03T18:50:03.351 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 03 18:50:03 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-6[56391]: 2024-05-03T18:50:03.085+0000 7f3ff2200700 -1 osd.6 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:37.490057+0000 front 2024-05-03T18:49:37.490102+0000 (oldest deadline 2024-05-03T18:50:02.789772+0000) 2024-05-03T18:50:03.502 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:03 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:03.203+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:25.480913+0000 front 2024-05-03T18:49:25.480403+0000 (oldest deadline 2024-05-03T18:49:50.180152+0000) 2024-05-03T18:50:03.546 INFO:journalctl@ceph.osd.4.smithi099.stdout:May 03 18:50:03 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-4[43040]: 2024-05-03T18:50:03.199+0000 7fc4bf600700 -1 osd.4 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:34.388247+0000 front 2024-05-03T18:49:34.388354+0000 (oldest deadline 2024-05-03T18:50:01.989026+0000) 2024-05-03T18:50:03.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:03 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:03.220+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:26.972061+0000 front 2024-05-03T18:49:26.972448+0000 (oldest deadline 2024-05-03T18:49:50.472344+0000) 2024-05-03T18:50:03.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:03 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:03.220+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:37.476081+0000 front 2024-05-03T18:49:37.476172+0000 (oldest deadline 2024-05-03T18:50:01.576192+0000) 2024-05-03T18:50:03.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:03 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:03.439+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:27.608012+0000 front 2024-05-03T18:49:27.608078+0000 (oldest deadline 2024-05-03T18:49:53.508290+0000) 2024-05-03T18:50:03.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:03 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:03.439+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:38.811037+0000 front 2024-05-03T18:49:38.811219+0000 (oldest deadline 2024-05-03T18:50:01.611951+0000) 2024-05-03T18:50:03.735 INFO:journalctl@ceph.osd.3.smithi033.stdout:May 03 18:50:03 smithi033 podman[127277]: 2024-05-03 18:50:03.477759994 +0000 UTC m=+0.481738049 container init bb78fd058547c7b2378aad7bfbb75bcf9ddd5075a4877c542fa25339fd5dac79 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-03T18:50:03.735 INFO:journalctl@ceph.osd.3.smithi033.stdout:May 03 18:50:03 smithi033 podman[127277]: 2024-05-03 18:50:03.490763 +0000 UTC m=+0.494741017 container start bb78fd058547c7b2378aad7bfbb75bcf9ddd5075a4877c542fa25339fd5dac79 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-3-deactivate, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-05-03T18:50:03.735 INFO:journalctl@ceph.osd.3.smithi033.stdout:May 03 18:50:03 smithi033 podman[127277]: 2024-05-03 18:50:03.602041931 +0000 UTC m=+0.606019946 container attach bb78fd058547c7b2378aad7bfbb75bcf9ddd5075a4877c542fa25339fd5dac79 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-05-03T18:50:03.759 INFO:journalctl@ceph.osd.5.smithi150.stdout:May 03 18:50:03 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-5[44561]: 2024-05-03T18:50:03.500+0000 7fdfa7000700 -1 osd.5 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:28.083853+0000 front 2024-05-03T18:49:28.084464+0000 (oldest deadline 2024-05-03T18:49:52.685392+0000) 2024-05-03T18:50:04.046 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:03 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7[49675]: 2024-05-03T18:50:03.756+0000 7ff8ace00700 -1 osd.7 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:36.774719+0000 front 2024-05-03T18:49:36.774769+0000 (oldest deadline 2024-05-03T18:50:02.674865+0000) 2024-05-03T18:50:04.133 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:03 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:03.757+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:24.877883+0000 front 2024-05-03T18:49:24.877625+0000 (oldest deadline 2024-05-03T18:49:50.777881+0000) 2024-05-03T18:50:04.235 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 03 18:50:04 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-6[56391]: 2024-05-03T18:50:04.094+0000 7f3ff2200700 -1 osd.6 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:37.490057+0000 front 2024-05-03T18:49:37.490102+0000 (oldest deadline 2024-05-03T18:50:02.789772+0000) 2024-05-03T18:50:04.235 INFO:journalctl@ceph.osd.3.smithi033.stdout:May 03 18:50:03 smithi033 podman[127277]: 2024-05-03 18:50:03.959068652 +0000 UTC m=+0.963046697 container died bb78fd058547c7b2378aad7bfbb75bcf9ddd5075a4877c542fa25339fd5dac79 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-3-deactivate, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-05-03T18:50:04.440 INFO:journalctl@ceph.osd.4.smithi099.stdout:May 03 18:50:04 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-4[43040]: 2024-05-03T18:50:04.241+0000 7fc4bf600700 -1 osd.4 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:34.388247+0000 front 2024-05-03T18:49:34.388354+0000 (oldest deadline 2024-05-03T18:50:01.989026+0000) 2024-05-03T18:50:04.440 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:04 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:04.210+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:26.972061+0000 front 2024-05-03T18:49:26.972448+0000 (oldest deadline 2024-05-03T18:49:50.472344+0000) 2024-05-03T18:50:04.440 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:04 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:04.210+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:37.476081+0000 front 2024-05-03T18:49:37.476172+0000 (oldest deadline 2024-05-03T18:50:01.576192+0000) 2024-05-03T18:50:04.509 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:04 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:04.249+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:25.480913+0000 front 2024-05-03T18:49:25.480403+0000 (oldest deadline 2024-05-03T18:49:50.180152+0000) 2024-05-03T18:50:04.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:04 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:04.457+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:27.608012+0000 front 2024-05-03T18:49:27.608078+0000 (oldest deadline 2024-05-03T18:49:53.508290+0000) 2024-05-03T18:50:04.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:04 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:04.457+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:38.811037+0000 front 2024-05-03T18:49:38.811219+0000 (oldest deadline 2024-05-03T18:50:01.611951+0000) 2024-05-03T18:50:04.853 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5703bba4-0978-11ef-bc97-c7b262605968@osd.3.service' 2024-05-03T18:50:04.883 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:04 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:04.744+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:24.877883+0000 front 2024-05-03T18:49:24.877625+0000 (oldest deadline 2024-05-03T18:49:50.777881+0000) 2024-05-03T18:50:04.883 INFO:journalctl@ceph.osd.5.smithi150.stdout:May 03 18:50:04 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-5[44561]: 2024-05-03T18:50:04.507+0000 7fdfa7000700 -1 osd.5 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:28.083853+0000 front 2024-05-03T18:49:28.084464+0000 (oldest deadline 2024-05-03T18:49:52.685392+0000) 2024-05-03T18:50:04.883 INFO:journalctl@ceph.osd.5.smithi150.stdout:May 03 18:50:04 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-5[44561]: 2024-05-03T18:50:04.507+0000 7fdfa7000700 -1 osd.5 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:38.586865+0000 front 2024-05-03T18:49:38.586927+0000 (oldest deadline 2024-05-03T18:50:04.487124+0000) 2024-05-03T18:50:04.975 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T18:50:04.975 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-05-03T18:50:04.975 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-05-03T18:50:04.975 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-5703bba4-0978-11ef-bc97-c7b262605968@osd.4 2024-05-03T18:50:05.046 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:04 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7[49675]: 2024-05-03T18:50:04.736+0000 7ff8ace00700 -1 osd.7 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:36.774719+0000 front 2024-05-03T18:49:36.774769+0000 (oldest deadline 2024-05-03T18:50:02.674865+0000) 2024-05-03T18:50:05.235 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 03 18:50:05 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-6[56391]: 2024-05-03T18:50:05.049+0000 7f3ff2200700 -1 osd.6 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:37.490057+0000 front 2024-05-03T18:49:37.490102+0000 (oldest deadline 2024-05-03T18:50:02.789772+0000) 2024-05-03T18:50:05.535 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:05 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:05.270+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:25.480913+0000 front 2024-05-03T18:49:25.480403+0000 (oldest deadline 2024-05-03T18:49:50.180152+0000) 2024-05-03T18:50:05.546 INFO:journalctl@ceph.osd.4.smithi099.stdout:May 03 18:50:05 smithi099 systemd[1]: Stopping Ceph osd.4 for 5703bba4-0978-11ef-bc97-c7b262605968... 2024-05-03T18:50:05.547 INFO:journalctl@ceph.osd.4.smithi099.stdout:May 03 18:50:05 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-4[43040]: 2024-05-03T18:50:05.222+0000 7fc4bf600700 -1 osd.4 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:34.388247+0000 front 2024-05-03T18:49:34.388354+0000 (oldest deadline 2024-05-03T18:50:01.989026+0000) 2024-05-03T18:50:05.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:05 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:05.195+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:26.972061+0000 front 2024-05-03T18:49:26.972448+0000 (oldest deadline 2024-05-03T18:49:50.472344+0000) 2024-05-03T18:50:05.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:05 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:05.195+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:37.476081+0000 front 2024-05-03T18:49:37.476172+0000 (oldest deadline 2024-05-03T18:50:01.576192+0000) 2024-05-03T18:50:05.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:05 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:05.409+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:27.608012+0000 front 2024-05-03T18:49:27.608078+0000 (oldest deadline 2024-05-03T18:49:53.508290+0000) 2024-05-03T18:50:05.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:05 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:05.409+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:38.811037+0000 front 2024-05-03T18:49:38.811219+0000 (oldest deadline 2024-05-03T18:50:01.611951+0000) 2024-05-03T18:50:05.786 INFO:journalctl@ceph.osd.5.smithi150.stdout:May 03 18:50:05 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-5[44561]: 2024-05-03T18:50:05.532+0000 7fdfa7000700 -1 osd.5 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:28.083853+0000 front 2024-05-03T18:49:28.084464+0000 (oldest deadline 2024-05-03T18:49:52.685392+0000) 2024-05-03T18:50:05.786 INFO:journalctl@ceph.osd.5.smithi150.stdout:May 03 18:50:05 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-5[44561]: 2024-05-03T18:50:05.532+0000 7fdfa7000700 -1 osd.5 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:38.586865+0000 front 2024-05-03T18:49:38.586927+0000 (oldest deadline 2024-05-03T18:50:04.487124+0000) 2024-05-03T18:50:06.047 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:05 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7[49675]: 2024-05-03T18:50:05.712+0000 7ff8ace00700 -1 osd.7 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:36.774719+0000 front 2024-05-03T18:49:36.774769+0000 (oldest deadline 2024-05-03T18:50:02.674865+0000) 2024-05-03T18:50:06.047 INFO:journalctl@ceph.osd.4.smithi099.stdout:May 03 18:50:05 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-4[43040]: 2024-05-03T18:50:05.558+0000 7fc4c3e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T18:50:06.047 INFO:journalctl@ceph.osd.4.smithi099.stdout:May 03 18:50:05 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-4[43040]: 2024-05-03T18:50:05.558+0000 7fc4c3e00700 -1 osd.4 126 *** Got signal Terminated *** 2024-05-03T18:50:06.047 INFO:journalctl@ceph.osd.4.smithi099.stdout:May 03 18:50:05 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-4[43040]: 2024-05-03T18:50:05.558+0000 7fc4c3e00700 -1 osd.4 126 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-03T18:50:06.133 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:05 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:05.783+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:24.877883+0000 front 2024-05-03T18:49:24.877625+0000 (oldest deadline 2024-05-03T18:49:50.777881+0000) 2024-05-03T18:50:06.424 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 03 18:50:06 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-6[56391]: 2024-05-03T18:50:06.049+0000 7f3ff2200700 -1 osd.6 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:37.490057+0000 front 2024-05-03T18:49:37.490102+0000 (oldest deadline 2024-05-03T18:50:02.789772+0000) 2024-05-03T18:50:06.537 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:06 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:06.270+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:25.480913+0000 front 2024-05-03T18:49:25.480403+0000 (oldest deadline 2024-05-03T18:49:50.180152+0000) 2024-05-03T18:50:06.537 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:06 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:06.270+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:34.882184+0000 front 2024-05-03T18:49:34.882683+0000 (oldest deadline 2024-05-03T18:50:05.483250+0000) 2024-05-03T18:50:06.546 INFO:journalctl@ceph.osd.4.smithi099.stdout:May 03 18:50:06 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-4[43040]: 2024-05-03T18:50:06.180+0000 7fc4bf600700 -1 osd.4 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:34.388247+0000 front 2024-05-03T18:49:34.388354+0000 (oldest deadline 2024-05-03T18:50:01.989026+0000) 2024-05-03T18:50:06.546 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:06 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:06.188+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:26.972061+0000 front 2024-05-03T18:49:26.972448+0000 (oldest deadline 2024-05-03T18:49:50.472344+0000) 2024-05-03T18:50:06.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:06 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:06.188+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:37.476081+0000 front 2024-05-03T18:49:37.476172+0000 (oldest deadline 2024-05-03T18:50:01.576192+0000) 2024-05-03T18:50:06.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:06 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:06.422+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:27.608012+0000 front 2024-05-03T18:49:27.608078+0000 (oldest deadline 2024-05-03T18:49:53.508290+0000) 2024-05-03T18:50:06.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:06 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:06.422+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:38.811037+0000 front 2024-05-03T18:49:38.811219+0000 (oldest deadline 2024-05-03T18:50:01.611951+0000) 2024-05-03T18:50:06.883 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:06 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:06.783+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:24.877883+0000 front 2024-05-03T18:49:24.877625+0000 (oldest deadline 2024-05-03T18:49:50.777881+0000) 2024-05-03T18:50:06.883 INFO:journalctl@ceph.osd.5.smithi150.stdout:May 03 18:50:06 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-5[44561]: 2024-05-03T18:50:06.534+0000 7fdfa7000700 -1 osd.5 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:28.083853+0000 front 2024-05-03T18:49:28.084464+0000 (oldest deadline 2024-05-03T18:49:52.685392+0000) 2024-05-03T18:50:06.883 INFO:journalctl@ceph.osd.5.smithi150.stdout:May 03 18:50:06 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-5[44561]: 2024-05-03T18:50:06.534+0000 7fdfa7000700 -1 osd.5 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:38.586865+0000 front 2024-05-03T18:49:38.586927+0000 (oldest deadline 2024-05-03T18:50:04.487124+0000) 2024-05-03T18:50:07.046 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:06 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7[49675]: 2024-05-03T18:50:06.753+0000 7ff8ace00700 -1 osd.7 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:36.774719+0000 front 2024-05-03T18:49:36.774769+0000 (oldest deadline 2024-05-03T18:50:02.674865+0000) 2024-05-03T18:50:07.440 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 03 18:50:07 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-6[56391]: 2024-05-03T18:50:07.092+0000 7f3ff2200700 -1 osd.6 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:37.490057+0000 front 2024-05-03T18:49:37.490102+0000 (oldest deadline 2024-05-03T18:50:02.789772+0000) 2024-05-03T18:50:07.546 INFO:journalctl@ceph.osd.4.smithi099.stdout:May 03 18:50:07 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-4[43040]: 2024-05-03T18:50:07.145+0000 7fc4bf600700 -1 osd.4 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:34.388247+0000 front 2024-05-03T18:49:34.388354+0000 (oldest deadline 2024-05-03T18:50:01.989026+0000) 2024-05-03T18:50:07.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:07 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:07.142+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:26.972061+0000 front 2024-05-03T18:49:26.972448+0000 (oldest deadline 2024-05-03T18:49:50.472344+0000) 2024-05-03T18:50:07.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:07 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:07.142+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:37.476081+0000 front 2024-05-03T18:49:37.476172+0000 (oldest deadline 2024-05-03T18:50:01.576192+0000) 2024-05-03T18:50:07.563 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:07 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:07.260+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:25.480913+0000 front 2024-05-03T18:49:25.480403+0000 (oldest deadline 2024-05-03T18:49:50.180152+0000) 2024-05-03T18:50:07.563 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:07 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:07.260+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:34.882184+0000 front 2024-05-03T18:49:34.882683+0000 (oldest deadline 2024-05-03T18:50:05.483250+0000) 2024-05-03T18:50:07.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:07 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:07.439+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:27.608012+0000 front 2024-05-03T18:49:27.608078+0000 (oldest deadline 2024-05-03T18:49:53.508290+0000) 2024-05-03T18:50:07.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:07 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:07.439+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:38.811037+0000 front 2024-05-03T18:49:38.811219+0000 (oldest deadline 2024-05-03T18:50:01.611951+0000) 2024-05-03T18:50:07.883 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:07 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:07.804+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:24.877883+0000 front 2024-05-03T18:49:24.877625+0000 (oldest deadline 2024-05-03T18:49:50.777881+0000) 2024-05-03T18:50:07.883 INFO:journalctl@ceph.osd.5.smithi150.stdout:May 03 18:50:07 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-5[44561]: 2024-05-03T18:50:07.561+0000 7fdfa7000700 -1 osd.5 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:28.083853+0000 front 2024-05-03T18:49:28.084464+0000 (oldest deadline 2024-05-03T18:49:52.685392+0000) 2024-05-03T18:50:07.883 INFO:journalctl@ceph.osd.5.smithi150.stdout:May 03 18:50:07 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-5[44561]: 2024-05-03T18:50:07.561+0000 7fdfa7000700 -1 osd.5 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:38.586865+0000 front 2024-05-03T18:49:38.586927+0000 (oldest deadline 2024-05-03T18:50:04.487124+0000) 2024-05-03T18:50:08.046 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:07 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7[49675]: 2024-05-03T18:50:07.794+0000 7ff8ace00700 -1 osd.7 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:36.774719+0000 front 2024-05-03T18:49:36.774769+0000 (oldest deadline 2024-05-03T18:50:02.674865+0000) 2024-05-03T18:50:08.453 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 03 18:50:08 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-6[56391]: 2024-05-03T18:50:08.057+0000 7f3ff2200700 -1 osd.6 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:37.490057+0000 front 2024-05-03T18:49:37.490102+0000 (oldest deadline 2024-05-03T18:50:02.789772+0000) 2024-05-03T18:50:08.517 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:08 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:08.226+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:25.480913+0000 front 2024-05-03T18:49:25.480403+0000 (oldest deadline 2024-05-03T18:49:50.180152+0000) 2024-05-03T18:50:08.517 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:08 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:08.226+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:34.882184+0000 front 2024-05-03T18:49:34.882683+0000 (oldest deadline 2024-05-03T18:50:05.483250+0000) 2024-05-03T18:50:08.546 INFO:journalctl@ceph.osd.4.smithi099.stdout:May 03 18:50:08 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-4[43040]: 2024-05-03T18:50:08.124+0000 7fc4bf600700 -1 osd.4 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:34.388247+0000 front 2024-05-03T18:49:34.388354+0000 (oldest deadline 2024-05-03T18:50:01.989026+0000) 2024-05-03T18:50:08.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:08 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:08.156+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:26.972061+0000 front 2024-05-03T18:49:26.972448+0000 (oldest deadline 2024-05-03T18:49:50.472344+0000) 2024-05-03T18:50:08.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:08 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:08.156+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:37.476081+0000 front 2024-05-03T18:49:37.476172+0000 (oldest deadline 2024-05-03T18:50:01.576192+0000) 2024-05-03T18:50:08.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:08 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:08.451+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:27.608012+0000 front 2024-05-03T18:49:27.608078+0000 (oldest deadline 2024-05-03T18:49:53.508290+0000) 2024-05-03T18:50:08.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:08 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:08.451+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:38.811037+0000 front 2024-05-03T18:49:38.811219+0000 (oldest deadline 2024-05-03T18:50:01.611951+0000) 2024-05-03T18:50:08.806 INFO:journalctl@ceph.osd.5.smithi150.stdout:May 03 18:50:08 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-5[44561]: 2024-05-03T18:50:08.514+0000 7fdfa7000700 -1 osd.5 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:28.083853+0000 front 2024-05-03T18:49:28.084464+0000 (oldest deadline 2024-05-03T18:49:52.685392+0000) 2024-05-03T18:50:08.806 INFO:journalctl@ceph.osd.5.smithi150.stdout:May 03 18:50:08 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-5[44561]: 2024-05-03T18:50:08.514+0000 7fdfa7000700 -1 osd.5 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:38.586865+0000 front 2024-05-03T18:49:38.586927+0000 (oldest deadline 2024-05-03T18:50:04.487124+0000) 2024-05-03T18:50:09.133 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:08 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:08.804+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:24.877883+0000 front 2024-05-03T18:49:24.877625+0000 (oldest deadline 2024-05-03T18:49:50.777881+0000) 2024-05-03T18:50:09.149 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:08 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7[49675]: 2024-05-03T18:50:08.797+0000 7ff8ace00700 -1 osd.7 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:36.774719+0000 front 2024-05-03T18:49:36.774769+0000 (oldest deadline 2024-05-03T18:50:02.674865+0000) 2024-05-03T18:50:09.485 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 03 18:50:09 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-6[56391]: 2024-05-03T18:50:09.097+0000 7f3ff2200700 -1 osd.6 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:37.490057+0000 front 2024-05-03T18:49:37.490102+0000 (oldest deadline 2024-05-03T18:50:02.789772+0000) 2024-05-03T18:50:09.503 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:09 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:09.193+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:25.480913+0000 front 2024-05-03T18:49:25.480403+0000 (oldest deadline 2024-05-03T18:49:50.180152+0000) 2024-05-03T18:50:09.503 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:09 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:09.193+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:34.882184+0000 front 2024-05-03T18:49:34.882683+0000 (oldest deadline 2024-05-03T18:50:05.483250+0000) 2024-05-03T18:50:09.546 INFO:journalctl@ceph.osd.4.smithi099.stdout:May 03 18:50:09 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-4[43040]: 2024-05-03T18:50:09.154+0000 7fc4bf600700 -1 osd.4 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:34.388247+0000 front 2024-05-03T18:49:34.388354+0000 (oldest deadline 2024-05-03T18:50:01.989026+0000) 2024-05-03T18:50:09.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:09 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:09.147+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:26.972061+0000 front 2024-05-03T18:49:26.972448+0000 (oldest deadline 2024-05-03T18:49:50.472344+0000) 2024-05-03T18:50:09.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:09 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:09.147+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:37.476081+0000 front 2024-05-03T18:49:37.476172+0000 (oldest deadline 2024-05-03T18:50:01.576192+0000) 2024-05-03T18:50:09.821 INFO:journalctl@ceph.osd.5.smithi150.stdout:May 03 18:50:09 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-5[44561]: 2024-05-03T18:50:09.500+0000 7fdfa7000700 -1 osd.5 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:28.083853+0000 front 2024-05-03T18:49:28.084464+0000 (oldest deadline 2024-05-03T18:49:52.685392+0000) 2024-05-03T18:50:09.821 INFO:journalctl@ceph.osd.5.smithi150.stdout:May 03 18:50:09 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-5[44561]: 2024-05-03T18:50:09.500+0000 7fdfa7000700 -1 osd.5 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:38.586865+0000 front 2024-05-03T18:49:38.586927+0000 (oldest deadline 2024-05-03T18:50:04.487124+0000) 2024-05-03T18:50:09.985 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:09 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:09.486+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:27.608012+0000 front 2024-05-03T18:49:27.608078+0000 (oldest deadline 2024-05-03T18:49:53.508290+0000) 2024-05-03T18:50:09.985 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:09 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:09.486+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:38.811037+0000 front 2024-05-03T18:49:38.811219+0000 (oldest deadline 2024-05-03T18:50:01.611951+0000) 2024-05-03T18:50:10.129 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:09 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7[49675]: 2024-05-03T18:50:09.807+0000 7ff8ace00700 -1 osd.7 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:36.774719+0000 front 2024-05-03T18:49:36.774769+0000 (oldest deadline 2024-05-03T18:50:02.674865+0000) 2024-05-03T18:50:10.133 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:09 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:09.819+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:24.877883+0000 front 2024-05-03T18:49:24.877625+0000 (oldest deadline 2024-05-03T18:49:50.777881+0000) 2024-05-03T18:50:10.448 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 03 18:50:10 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-6[56391]: 2024-05-03T18:50:10.091+0000 7f3ff2200700 -1 osd.6 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:37.490057+0000 front 2024-05-03T18:49:37.490102+0000 (oldest deadline 2024-05-03T18:50:02.789772+0000) 2024-05-03T18:50:10.545 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:10 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:10.191+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:25.480913+0000 front 2024-05-03T18:49:25.480403+0000 (oldest deadline 2024-05-03T18:49:50.180152+0000) 2024-05-03T18:50:10.545 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:10 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:10.191+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:34.882184+0000 front 2024-05-03T18:49:34.882683+0000 (oldest deadline 2024-05-03T18:50:05.483250+0000) 2024-05-03T18:50:10.546 INFO:journalctl@ceph.osd.4.smithi099.stdout:May 03 18:50:10 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-4[43040]: 2024-05-03T18:50:10.128+0000 7fc4bf600700 -1 osd.4 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:34.388247+0000 front 2024-05-03T18:49:34.388354+0000 (oldest deadline 2024-05-03T18:50:01.989026+0000) 2024-05-03T18:50:10.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:10 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:10.195+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:26.972061+0000 front 2024-05-03T18:49:26.972448+0000 (oldest deadline 2024-05-03T18:49:50.472344+0000) 2024-05-03T18:50:10.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:10 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:10.195+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:37.476081+0000 front 2024-05-03T18:49:37.476172+0000 (oldest deadline 2024-05-03T18:50:01.576192+0000) 2024-05-03T18:50:10.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:10 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:10.446+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:27.608012+0000 front 2024-05-03T18:49:27.608078+0000 (oldest deadline 2024-05-03T18:49:53.508290+0000) 2024-05-03T18:50:10.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:10 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:10.446+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:38.811037+0000 front 2024-05-03T18:49:38.811219+0000 (oldest deadline 2024-05-03T18:50:01.611951+0000) 2024-05-03T18:50:10.824 INFO:journalctl@ceph.osd.5.smithi150.stdout:May 03 18:50:10 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-5[44561]: 2024-05-03T18:50:10.543+0000 7fdfa7000700 -1 osd.5 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:28.083853+0000 front 2024-05-03T18:49:28.084464+0000 (oldest deadline 2024-05-03T18:49:52.685392+0000) 2024-05-03T18:50:10.824 INFO:journalctl@ceph.osd.5.smithi150.stdout:May 03 18:50:10 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-5[44561]: 2024-05-03T18:50:10.543+0000 7fdfa7000700 -1 osd.5 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:38.586865+0000 front 2024-05-03T18:49:38.586927+0000 (oldest deadline 2024-05-03T18:50:04.487124+0000) 2024-05-03T18:50:11.046 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:10 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7[49675]: 2024-05-03T18:50:10.775+0000 7ff8ace00700 -1 osd.7 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:36.774719+0000 front 2024-05-03T18:49:36.774769+0000 (oldest deadline 2024-05-03T18:50:02.674865+0000) 2024-05-03T18:50:11.133 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:10 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:10.822+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:24.877883+0000 front 2024-05-03T18:49:24.877625+0000 (oldest deadline 2024-05-03T18:49:50.777881+0000) 2024-05-03T18:50:11.457 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 03 18:50:11 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-6[56391]: 2024-05-03T18:50:11.051+0000 7f3ff2200700 -1 osd.6 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:37.490057+0000 front 2024-05-03T18:49:37.490102+0000 (oldest deadline 2024-05-03T18:50:02.789772+0000) 2024-05-03T18:50:11.502 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:11 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:11.217+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:25.480913+0000 front 2024-05-03T18:49:25.480403+0000 (oldest deadline 2024-05-03T18:49:50.180152+0000) 2024-05-03T18:50:11.502 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:11 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:11.217+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:34.882184+0000 front 2024-05-03T18:49:34.882683+0000 (oldest deadline 2024-05-03T18:50:05.483250+0000) 2024-05-03T18:50:11.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:11 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:11.202+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:26.972061+0000 front 2024-05-03T18:49:26.972448+0000 (oldest deadline 2024-05-03T18:49:50.472344+0000) 2024-05-03T18:50:11.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:11 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:11.202+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:37.476081+0000 front 2024-05-03T18:49:37.476172+0000 (oldest deadline 2024-05-03T18:50:01.576192+0000) 2024-05-03T18:50:11.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:11 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:11.202+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:49.079592+0000 front 2024-05-03T18:49:49.080117+0000 (oldest deadline 2024-05-03T18:50:10.779802+0000) 2024-05-03T18:50:11.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:11 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:11.455+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:27.608012+0000 front 2024-05-03T18:49:27.608078+0000 (oldest deadline 2024-05-03T18:49:53.508290+0000) 2024-05-03T18:50:11.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:11 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:11.455+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:38.811037+0000 front 2024-05-03T18:49:38.811219+0000 (oldest deadline 2024-05-03T18:50:01.611951+0000) 2024-05-03T18:50:11.863 INFO:journalctl@ceph.osd.5.smithi150.stdout:May 03 18:50:11 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-5[44561]: 2024-05-03T18:50:11.500+0000 7fdfa7000700 -1 osd.5 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:28.083853+0000 front 2024-05-03T18:49:28.084464+0000 (oldest deadline 2024-05-03T18:49:52.685392+0000) 2024-05-03T18:50:11.864 INFO:journalctl@ceph.osd.5.smithi150.stdout:May 03 18:50:11 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-5[44561]: 2024-05-03T18:50:11.500+0000 7fdfa7000700 -1 osd.5 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:38.586865+0000 front 2024-05-03T18:49:38.586927+0000 (oldest deadline 2024-05-03T18:50:04.487124+0000) 2024-05-03T18:50:12.046 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:11 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7[49675]: 2024-05-03T18:50:11.745+0000 7ff8ace00700 -1 osd.7 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:36.774719+0000 front 2024-05-03T18:49:36.774769+0000 (oldest deadline 2024-05-03T18:50:02.674865+0000) 2024-05-03T18:50:12.047 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:11 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7[49675]: 2024-05-03T18:50:11.745+0000 7ff8ace00700 -1 osd.7 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:47.376623+0000 front 2024-05-03T18:49:47.376700+0000 (oldest deadline 2024-05-03T18:50:10.876637+0000) 2024-05-03T18:50:12.047 INFO:journalctl@ceph.osd.4.smithi099.stdout:May 03 18:50:11 smithi099 podman[71128]: 2024-05-03 18:50:11.752453783 +0000 UTC m=+6.377292014 container stop 64303dee81d8df0b389c84c4f2f6222a245c52ea88d6c4f34bcda3f05dadbe7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-05-03T18:50:12.047 INFO:journalctl@ceph.osd.4.smithi099.stdout:May 03 18:50:11 smithi099 podman[71128]: 2024-05-03 18:50:11.781141306 +0000 UTC m=+6.405979570 container died 64303dee81d8df0b389c84c4f2f6222a245c52ea88d6c4f34bcda3f05dadbe7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-4, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.vendor=CentOS) 2024-05-03T18:50:12.133 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:11 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:11.862+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:24.877883+0000 front 2024-05-03T18:49:24.877625+0000 (oldest deadline 2024-05-03T18:49:50.777881+0000) 2024-05-03T18:50:12.133 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:11 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:11.862+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:47.685409+0000 front 2024-05-03T18:49:47.685527+0000 (oldest deadline 2024-05-03T18:50:11.785436+0000) 2024-05-03T18:50:12.432 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 03 18:50:12 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-6[56391]: 2024-05-03T18:50:12.004+0000 7f3ff2200700 -1 osd.6 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:37.490057+0000 front 2024-05-03T18:49:37.490102+0000 (oldest deadline 2024-05-03T18:50:02.789772+0000) 2024-05-03T18:50:12.523 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:12 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:12.206+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:26.972061+0000 front 2024-05-03T18:49:26.972448+0000 (oldest deadline 2024-05-03T18:49:50.472344+0000) 2024-05-03T18:50:12.523 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:12 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:12.206+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:37.476081+0000 front 2024-05-03T18:49:37.476172+0000 (oldest deadline 2024-05-03T18:50:01.576192+0000) 2024-05-03T18:50:12.523 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:12 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:12.206+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:49.079592+0000 front 2024-05-03T18:49:49.080117+0000 (oldest deadline 2024-05-03T18:50:10.779802+0000) 2024-05-03T18:50:12.524 INFO:journalctl@ceph.osd.4.smithi099.stdout:May 03 18:50:12 smithi099 podman[71128]: 2024-05-03 18:50:12.254096014 +0000 UTC m=+6.878934238 container cleanup 64303dee81d8df0b389c84c4f2f6222a245c52ea88d6c4f34bcda3f05dadbe7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-4, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.schema-version=1.0) 2024-05-03T18:50:12.524 INFO:journalctl@ceph.osd.4.smithi099.stdout:May 03 18:50:12 smithi099 bash[71128]: ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-4 2024-05-03T18:50:12.633 INFO:journalctl@ceph.osd.5.smithi150.stdout:May 03 18:50:12 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-5[44561]: 2024-05-03T18:50:12.461+0000 7fdfa7000700 -1 osd.5 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:28.083853+0000 front 2024-05-03T18:49:28.084464+0000 (oldest deadline 2024-05-03T18:49:52.685392+0000) 2024-05-03T18:50:12.633 INFO:journalctl@ceph.osd.5.smithi150.stdout:May 03 18:50:12 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-5[44561]: 2024-05-03T18:50:12.461+0000 7fdfa7000700 -1 osd.5 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:38.586865+0000 front 2024-05-03T18:49:38.586927+0000 (oldest deadline 2024-05-03T18:50:04.487124+0000) 2024-05-03T18:50:12.634 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:12 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:12.259+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:25.480913+0000 front 2024-05-03T18:49:25.480403+0000 (oldest deadline 2024-05-03T18:49:50.180152+0000) 2024-05-03T18:50:12.634 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:12 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:12.259+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:34.882184+0000 front 2024-05-03T18:49:34.882683+0000 (oldest deadline 2024-05-03T18:50:05.483250+0000) 2024-05-03T18:50:12.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:12 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:12.430+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:27.608012+0000 front 2024-05-03T18:49:27.608078+0000 (oldest deadline 2024-05-03T18:49:53.508290+0000) 2024-05-03T18:50:12.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:12 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:12.430+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:38.811037+0000 front 2024-05-03T18:49:38.811219+0000 (oldest deadline 2024-05-03T18:50:01.611951+0000) 2024-05-03T18:50:12.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:12 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:12.430+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:46.215926+0000 front 2024-05-03T18:49:46.216076+0000 (oldest deadline 2024-05-03T18:50:12.116128+0000) 2024-05-03T18:50:12.796 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:12 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7[49675]: 2024-05-03T18:50:12.732+0000 7ff8ace00700 -1 osd.7 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:36.774719+0000 front 2024-05-03T18:49:36.774769+0000 (oldest deadline 2024-05-03T18:50:02.674865+0000) 2024-05-03T18:50:12.797 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:12 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7[49675]: 2024-05-03T18:50:12.732+0000 7ff8ace00700 -1 osd.7 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:47.376623+0000 front 2024-05-03T18:49:47.376700+0000 (oldest deadline 2024-05-03T18:50:10.876637+0000) 2024-05-03T18:50:12.797 INFO:journalctl@ceph.osd.4.smithi099.stdout:May 03 18:50:12 smithi099 podman[71149]: 2024-05-03 18:50:12.521781902 +0000 UTC m=+0.731336572 container remove 64303dee81d8df0b389c84c4f2f6222a245c52ea88d6c4f34bcda3f05dadbe7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-4, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-05-03T18:50:13.133 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:12 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:12.825+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:24.877883+0000 front 2024-05-03T18:49:24.877625+0000 (oldest deadline 2024-05-03T18:49:50.777881+0000) 2024-05-03T18:50:13.133 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:12 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:12.825+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:47.685409+0000 front 2024-05-03T18:49:47.685527+0000 (oldest deadline 2024-05-03T18:50:11.785436+0000) 2024-05-03T18:50:13.426 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 03 18:50:12 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-6[56391]: 2024-05-03T18:50:12.991+0000 7f3ff2200700 -1 osd.6 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:37.490057+0000 front 2024-05-03T18:49:37.490102+0000 (oldest deadline 2024-05-03T18:50:02.789772+0000) 2024-05-03T18:50:13.426 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 03 18:50:12 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-6[56391]: 2024-05-03T18:50:12.991+0000 7f3ff2200700 -1 osd.6 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:48.692256+0000 front 2024-05-03T18:49:48.692122+0000 (oldest deadline 2024-05-03T18:50:12.792315+0000) 2024-05-03T18:50:13.535 INFO:journalctl@ceph.osd.4.smithi099.stdout:May 03 18:50:13 smithi099 podman[71239]: 2024-05-03 18:50:13.235315622 +0000 UTC m=+0.035962352 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T18:50:13.535 INFO:journalctl@ceph.osd.4.smithi099.stdout:May 03 18:50:13 smithi099 podman[71239]: 2024-05-03 18:50:13.364424948 +0000 UTC m=+0.165071701 container create 1a39bc752292499a966fef3c789e5719d286d4d4318a562b89717793abc736cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-03T18:50:13.535 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:13 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:13.221+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:26.972061+0000 front 2024-05-03T18:49:26.972448+0000 (oldest deadline 2024-05-03T18:49:50.472344+0000) 2024-05-03T18:50:13.535 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:13 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:13.221+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:37.476081+0000 front 2024-05-03T18:49:37.476172+0000 (oldest deadline 2024-05-03T18:50:01.576192+0000) 2024-05-03T18:50:13.535 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:13 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:13.221+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:49.079592+0000 front 2024-05-03T18:49:49.080117+0000 (oldest deadline 2024-05-03T18:50:10.779802+0000) 2024-05-03T18:50:13.633 INFO:journalctl@ceph.osd.5.smithi150.stdout:May 03 18:50:13 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-5[44561]: 2024-05-03T18:50:13.450+0000 7fdfa7000700 -1 osd.5 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:28.083853+0000 front 2024-05-03T18:49:28.084464+0000 (oldest deadline 2024-05-03T18:49:52.685392+0000) 2024-05-03T18:50:13.633 INFO:journalctl@ceph.osd.5.smithi150.stdout:May 03 18:50:13 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-5[44561]: 2024-05-03T18:50:13.450+0000 7fdfa7000700 -1 osd.5 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:38.586865+0000 front 2024-05-03T18:49:38.586927+0000 (oldest deadline 2024-05-03T18:50:04.487124+0000) 2024-05-03T18:50:13.633 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:13 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:13.289+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:25.480913+0000 front 2024-05-03T18:49:25.480403+0000 (oldest deadline 2024-05-03T18:49:50.180152+0000) 2024-05-03T18:50:13.634 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:13 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:13.289+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:34.882184+0000 front 2024-05-03T18:49:34.882683+0000 (oldest deadline 2024-05-03T18:50:05.483250+0000) 2024-05-03T18:50:13.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:13 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:13.440+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:27.608012+0000 front 2024-05-03T18:49:27.608078+0000 (oldest deadline 2024-05-03T18:49:53.508290+0000) 2024-05-03T18:50:13.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:13 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:13.440+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:38.811037+0000 front 2024-05-03T18:49:38.811219+0000 (oldest deadline 2024-05-03T18:50:01.611951+0000) 2024-05-03T18:50:13.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:13 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:13.440+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:46.215926+0000 front 2024-05-03T18:49:46.216076+0000 (oldest deadline 2024-05-03T18:50:12.116128+0000) 2024-05-03T18:50:13.796 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:13 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7[49675]: 2024-05-03T18:50:13.748+0000 7ff8ace00700 -1 osd.7 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:36.774719+0000 front 2024-05-03T18:49:36.774769+0000 (oldest deadline 2024-05-03T18:50:02.674865+0000) 2024-05-03T18:50:13.797 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:13 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7[49675]: 2024-05-03T18:50:13.748+0000 7ff8ace00700 -1 osd.7 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:47.376623+0000 front 2024-05-03T18:49:47.376700+0000 (oldest deadline 2024-05-03T18:50:10.876637+0000) 2024-05-03T18:50:13.797 INFO:journalctl@ceph.osd.4.smithi099.stdout:May 03 18:50:13 smithi099 podman[71239]: 2024-05-03 18:50:13.69320698 +0000 UTC m=+0.493853740 container init 1a39bc752292499a966fef3c789e5719d286d4d4318a562b89717793abc736cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-05-03T18:50:13.797 INFO:journalctl@ceph.osd.4.smithi099.stdout:May 03 18:50:13 smithi099 podman[71239]: 2024-05-03 18:50:13.70412721 +0000 UTC m=+0.504773953 container start 1a39bc752292499a966fef3c789e5719d286d4d4318a562b89717793abc736cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-05-03T18:50:14.133 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:13 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:13.820+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:24.877883+0000 front 2024-05-03T18:49:24.877625+0000 (oldest deadline 2024-05-03T18:49:50.777881+0000) 2024-05-03T18:50:14.133 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:13 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:13.820+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:47.685409+0000 front 2024-05-03T18:49:47.685527+0000 (oldest deadline 2024-05-03T18:50:11.785436+0000) 2024-05-03T18:50:14.161 INFO:journalctl@ceph.osd.4.smithi099.stdout:May 03 18:50:13 smithi099 podman[71239]: 2024-05-03 18:50:13.80863308 +0000 UTC m=+0.609279880 container attach 1a39bc752292499a966fef3c789e5719d286d4d4318a562b89717793abc736cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-05-03T18:50:14.235 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 03 18:50:13 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-6[56391]: 2024-05-03T18:50:13.980+0000 7f3ff2200700 -1 osd.6 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:37.490057+0000 front 2024-05-03T18:49:37.490102+0000 (oldest deadline 2024-05-03T18:50:02.789772+0000) 2024-05-03T18:50:14.235 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 03 18:50:13 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-6[56391]: 2024-05-03T18:50:13.980+0000 7f3ff2200700 -1 osd.6 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:48.692256+0000 front 2024-05-03T18:49:48.692122+0000 (oldest deadline 2024-05-03T18:50:12.792315+0000) 2024-05-03T18:50:14.546 INFO:journalctl@ceph.osd.4.smithi099.stdout:May 03 18:50:14 smithi099 podman[71239]: 2024-05-03 18:50:14.160788364 +0000 UTC m=+0.961435154 container died 1a39bc752292499a966fef3c789e5719d286d4d4318a562b89717793abc736cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True) 2024-05-03T18:50:14.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:14 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:14.251+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:26.972061+0000 front 2024-05-03T18:49:26.972448+0000 (oldest deadline 2024-05-03T18:49:50.472344+0000) 2024-05-03T18:50:14.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:14 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:14.251+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:37.476081+0000 front 2024-05-03T18:49:37.476172+0000 (oldest deadline 2024-05-03T18:50:01.576192+0000) 2024-05-03T18:50:14.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:14 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:14.251+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:49.079592+0000 front 2024-05-03T18:49:49.080117+0000 (oldest deadline 2024-05-03T18:50:10.779802+0000) 2024-05-03T18:50:14.633 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:14 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:14.279+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:25.480913+0000 front 2024-05-03T18:49:25.480403+0000 (oldest deadline 2024-05-03T18:49:50.180152+0000) 2024-05-03T18:50:14.633 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:14 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:14.279+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:34.882184+0000 front 2024-05-03T18:49:34.882683+0000 (oldest deadline 2024-05-03T18:50:05.483250+0000) 2024-05-03T18:50:14.633 INFO:journalctl@ceph.osd.5.smithi150.stdout:May 03 18:50:14 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-5[44561]: 2024-05-03T18:50:14.432+0000 7fdfa7000700 -1 osd.5 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:28.083853+0000 front 2024-05-03T18:49:28.084464+0000 (oldest deadline 2024-05-03T18:49:52.685392+0000) 2024-05-03T18:50:14.634 INFO:journalctl@ceph.osd.5.smithi150.stdout:May 03 18:50:14 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-5[44561]: 2024-05-03T18:50:14.432+0000 7fdfa7000700 -1 osd.5 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:38.586865+0000 front 2024-05-03T18:49:38.586927+0000 (oldest deadline 2024-05-03T18:50:04.487124+0000) 2024-05-03T18:50:14.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:14 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:14.398+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:27.608012+0000 front 2024-05-03T18:49:27.608078+0000 (oldest deadline 2024-05-03T18:49:53.508290+0000) 2024-05-03T18:50:14.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:14 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:14.398+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:38.811037+0000 front 2024-05-03T18:49:38.811219+0000 (oldest deadline 2024-05-03T18:50:01.611951+0000) 2024-05-03T18:50:14.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:14 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:14.398+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:46.215926+0000 front 2024-05-03T18:49:46.216076+0000 (oldest deadline 2024-05-03T18:50:12.116128+0000) 2024-05-03T18:50:14.964 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:14 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7[49675]: 2024-05-03T18:50:14.721+0000 7ff8ace00700 -1 osd.7 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:36.774719+0000 front 2024-05-03T18:49:36.774769+0000 (oldest deadline 2024-05-03T18:50:02.674865+0000) 2024-05-03T18:50:14.964 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:14 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7[49675]: 2024-05-03T18:50:14.721+0000 7ff8ace00700 -1 osd.7 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:47.376623+0000 front 2024-05-03T18:49:47.376700+0000 (oldest deadline 2024-05-03T18:50:10.876637+0000) 2024-05-03T18:50:15.025 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5703bba4-0978-11ef-bc97-c7b262605968@osd.4.service' 2024-05-03T18:50:15.133 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:14 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:14.860+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:24.877883+0000 front 2024-05-03T18:49:24.877625+0000 (oldest deadline 2024-05-03T18:49:50.777881+0000) 2024-05-03T18:50:15.133 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:14 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:14.860+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:47.685409+0000 front 2024-05-03T18:49:47.685527+0000 (oldest deadline 2024-05-03T18:50:11.785436+0000) 2024-05-03T18:50:15.139 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T18:50:15.139 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-05-03T18:50:15.139 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-05-03T18:50:15.139 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-5703bba4-0978-11ef-bc97-c7b262605968@osd.5 2024-05-03T18:50:15.235 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 03 18:50:14 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-6[56391]: 2024-05-03T18:50:14.977+0000 7f3ff2200700 -1 osd.6 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:37.490057+0000 front 2024-05-03T18:49:37.490102+0000 (oldest deadline 2024-05-03T18:50:02.789772+0000) 2024-05-03T18:50:15.235 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 03 18:50:14 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-6[56391]: 2024-05-03T18:50:14.977+0000 7f3ff2200700 -1 osd.6 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:48.692256+0000 front 2024-05-03T18:49:48.692122+0000 (oldest deadline 2024-05-03T18:50:12.792315+0000) 2024-05-03T18:50:15.546 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:15 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:15.220+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:26.972061+0000 front 2024-05-03T18:49:26.972448+0000 (oldest deadline 2024-05-03T18:49:50.472344+0000) 2024-05-03T18:50:15.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:15 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:15.220+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:37.476081+0000 front 2024-05-03T18:49:37.476172+0000 (oldest deadline 2024-05-03T18:50:01.576192+0000) 2024-05-03T18:50:15.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:15 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:15.220+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:49.079592+0000 front 2024-05-03T18:49:49.080117+0000 (oldest deadline 2024-05-03T18:50:10.779802+0000) 2024-05-03T18:50:15.633 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:15 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:15.279+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:25.480913+0000 front 2024-05-03T18:49:25.480403+0000 (oldest deadline 2024-05-03T18:49:50.180152+0000) 2024-05-03T18:50:15.633 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:15 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:15.279+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:34.882184+0000 front 2024-05-03T18:49:34.882683+0000 (oldest deadline 2024-05-03T18:50:05.483250+0000) 2024-05-03T18:50:15.633 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:15 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:15.279+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:45.484981+0000 front 2024-05-03T18:49:45.484933+0000 (oldest deadline 2024-05-03T18:50:14.885581+0000) 2024-05-03T18:50:15.634 INFO:journalctl@ceph.osd.5.smithi150.stdout:May 03 18:50:15 smithi150 systemd[1]: Stopping Ceph osd.5 for 5703bba4-0978-11ef-bc97-c7b262605968... 2024-05-03T18:50:15.634 INFO:journalctl@ceph.osd.5.smithi150.stdout:May 03 18:50:15 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-5[44561]: 2024-05-03T18:50:15.416+0000 7fdfa7000700 -1 osd.5 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:28.083853+0000 front 2024-05-03T18:49:28.084464+0000 (oldest deadline 2024-05-03T18:49:52.685392+0000) 2024-05-03T18:50:15.634 INFO:journalctl@ceph.osd.5.smithi150.stdout:May 03 18:50:15 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-5[44561]: 2024-05-03T18:50:15.416+0000 7fdfa7000700 -1 osd.5 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:38.586865+0000 front 2024-05-03T18:49:38.586927+0000 (oldest deadline 2024-05-03T18:50:04.487124+0000) 2024-05-03T18:50:15.634 INFO:journalctl@ceph.osd.5.smithi150.stdout:May 03 18:50:15 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-5[44561]: 2024-05-03T18:50:15.416+0000 7fdfa7000700 -1 osd.5 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:49.188983+0000 front 2024-05-03T18:49:49.189367+0000 (oldest deadline 2024-05-03T18:50:15.089043+0000) 2024-05-03T18:50:15.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:15 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:15.358+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:27.608012+0000 front 2024-05-03T18:49:27.608078+0000 (oldest deadline 2024-05-03T18:49:53.508290+0000) 2024-05-03T18:50:15.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:15 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:15.358+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:38.811037+0000 front 2024-05-03T18:49:38.811219+0000 (oldest deadline 2024-05-03T18:50:01.611951+0000) 2024-05-03T18:50:15.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:15 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:15.358+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:46.215926+0000 front 2024-05-03T18:49:46.216076+0000 (oldest deadline 2024-05-03T18:50:12.116128+0000) 2024-05-03T18:50:16.046 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:15 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7[49675]: 2024-05-03T18:50:15.723+0000 7ff8ace00700 -1 osd.7 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:36.774719+0000 front 2024-05-03T18:49:36.774769+0000 (oldest deadline 2024-05-03T18:50:02.674865+0000) 2024-05-03T18:50:16.047 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:15 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7[49675]: 2024-05-03T18:50:15.723+0000 7ff8ace00700 -1 osd.7 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:47.376623+0000 front 2024-05-03T18:49:47.376700+0000 (oldest deadline 2024-05-03T18:50:10.876637+0000) 2024-05-03T18:50:16.133 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:15 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:15.860+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:24.877883+0000 front 2024-05-03T18:49:24.877625+0000 (oldest deadline 2024-05-03T18:49:50.777881+0000) 2024-05-03T18:50:16.133 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:15 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:15.860+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:47.685409+0000 front 2024-05-03T18:49:47.685527+0000 (oldest deadline 2024-05-03T18:50:11.785436+0000) 2024-05-03T18:50:16.134 INFO:journalctl@ceph.osd.5.smithi150.stdout:May 03 18:50:15 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-5[44561]: 2024-05-03T18:50:15.681+0000 7fdfb0800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T18:50:16.134 INFO:journalctl@ceph.osd.5.smithi150.stdout:May 03 18:50:15 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-5[44561]: 2024-05-03T18:50:15.681+0000 7fdfb0800700 -1 osd.5 126 *** Got signal Terminated *** 2024-05-03T18:50:16.134 INFO:journalctl@ceph.osd.5.smithi150.stdout:May 03 18:50:15 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-5[44561]: 2024-05-03T18:50:15.681+0000 7fdfb0800700 -1 osd.5 126 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-03T18:50:16.355 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 03 18:50:15 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-6[56391]: 2024-05-03T18:50:15.997+0000 7f3ff2200700 -1 osd.6 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:37.490057+0000 front 2024-05-03T18:49:37.490102+0000 (oldest deadline 2024-05-03T18:50:02.789772+0000) 2024-05-03T18:50:16.355 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 03 18:50:15 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-6[56391]: 2024-05-03T18:50:15.997+0000 7f3ff2200700 -1 osd.6 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:48.692256+0000 front 2024-05-03T18:49:48.692122+0000 (oldest deadline 2024-05-03T18:50:12.792315+0000) 2024-05-03T18:50:16.546 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:16 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:16.235+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:26.972061+0000 front 2024-05-03T18:49:26.972448+0000 (oldest deadline 2024-05-03T18:49:50.472344+0000) 2024-05-03T18:50:16.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:16 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:16.235+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:37.476081+0000 front 2024-05-03T18:49:37.476172+0000 (oldest deadline 2024-05-03T18:50:01.576192+0000) 2024-05-03T18:50:16.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:16 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:16.235+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:49.079592+0000 front 2024-05-03T18:49:49.080117+0000 (oldest deadline 2024-05-03T18:50:10.779802+0000) 2024-05-03T18:50:16.633 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:16 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:16.291+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:25.480913+0000 front 2024-05-03T18:49:25.480403+0000 (oldest deadline 2024-05-03T18:49:50.180152+0000) 2024-05-03T18:50:16.633 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:16 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:16.291+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:34.882184+0000 front 2024-05-03T18:49:34.882683+0000 (oldest deadline 2024-05-03T18:50:05.483250+0000) 2024-05-03T18:50:16.633 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:16 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:16.291+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:45.484981+0000 front 2024-05-03T18:49:45.484933+0000 (oldest deadline 2024-05-03T18:50:14.885581+0000) 2024-05-03T18:50:16.634 INFO:journalctl@ceph.osd.5.smithi150.stdout:May 03 18:50:16 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-5[44561]: 2024-05-03T18:50:16.461+0000 7fdfa7000700 -1 osd.5 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:28.083853+0000 front 2024-05-03T18:49:28.084464+0000 (oldest deadline 2024-05-03T18:49:52.685392+0000) 2024-05-03T18:50:16.634 INFO:journalctl@ceph.osd.5.smithi150.stdout:May 03 18:50:16 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-5[44561]: 2024-05-03T18:50:16.461+0000 7fdfa7000700 -1 osd.5 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:38.586865+0000 front 2024-05-03T18:49:38.586927+0000 (oldest deadline 2024-05-03T18:50:04.487124+0000) 2024-05-03T18:50:16.634 INFO:journalctl@ceph.osd.5.smithi150.stdout:May 03 18:50:16 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-5[44561]: 2024-05-03T18:50:16.461+0000 7fdfa7000700 -1 osd.5 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:49.188983+0000 front 2024-05-03T18:49:49.189367+0000 (oldest deadline 2024-05-03T18:50:15.089043+0000) 2024-05-03T18:50:16.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:16 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:16.354+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:27.608012+0000 front 2024-05-03T18:49:27.608078+0000 (oldest deadline 2024-05-03T18:49:53.508290+0000) 2024-05-03T18:50:16.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:16 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:16.354+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:38.811037+0000 front 2024-05-03T18:49:38.811219+0000 (oldest deadline 2024-05-03T18:50:01.611951+0000) 2024-05-03T18:50:16.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:16 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:16.354+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:46.215926+0000 front 2024-05-03T18:49:46.216076+0000 (oldest deadline 2024-05-03T18:50:12.116128+0000) 2024-05-03T18:50:17.046 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:16 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7[49675]: 2024-05-03T18:50:16.769+0000 7ff8ace00700 -1 osd.7 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:36.774719+0000 front 2024-05-03T18:49:36.774769+0000 (oldest deadline 2024-05-03T18:50:02.674865+0000) 2024-05-03T18:50:17.047 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:16 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7[49675]: 2024-05-03T18:50:16.769+0000 7ff8ace00700 -1 osd.7 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:47.376623+0000 front 2024-05-03T18:49:47.376700+0000 (oldest deadline 2024-05-03T18:50:10.876637+0000) 2024-05-03T18:50:17.133 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:16 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:16.880+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:24.877883+0000 front 2024-05-03T18:49:24.877625+0000 (oldest deadline 2024-05-03T18:49:50.777881+0000) 2024-05-03T18:50:17.133 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:16 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:16.880+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:47.685409+0000 front 2024-05-03T18:49:47.685527+0000 (oldest deadline 2024-05-03T18:50:11.785436+0000) 2024-05-03T18:50:17.235 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 03 18:50:16 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-6[56391]: 2024-05-03T18:50:16.949+0000 7f3ff2200700 -1 osd.6 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:37.490057+0000 front 2024-05-03T18:49:37.490102+0000 (oldest deadline 2024-05-03T18:50:02.789772+0000) 2024-05-03T18:50:17.235 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 03 18:50:16 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-6[56391]: 2024-05-03T18:50:16.949+0000 7f3ff2200700 -1 osd.6 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:48.692256+0000 front 2024-05-03T18:49:48.692122+0000 (oldest deadline 2024-05-03T18:50:12.792315+0000) 2024-05-03T18:50:17.546 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:17 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:17.250+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:26.972061+0000 front 2024-05-03T18:49:26.972448+0000 (oldest deadline 2024-05-03T18:49:50.472344+0000) 2024-05-03T18:50:17.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:17 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:17.250+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:37.476081+0000 front 2024-05-03T18:49:37.476172+0000 (oldest deadline 2024-05-03T18:50:01.576192+0000) 2024-05-03T18:50:17.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:17 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:17.250+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:49.079592+0000 front 2024-05-03T18:49:49.080117+0000 (oldest deadline 2024-05-03T18:50:10.779802+0000) 2024-05-03T18:50:17.633 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:17 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:17.327+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:25.480913+0000 front 2024-05-03T18:49:25.480403+0000 (oldest deadline 2024-05-03T18:49:50.180152+0000) 2024-05-03T18:50:17.633 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:17 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:17.327+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:34.882184+0000 front 2024-05-03T18:49:34.882683+0000 (oldest deadline 2024-05-03T18:50:05.483250+0000) 2024-05-03T18:50:17.633 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:17 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:17.327+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:45.484981+0000 front 2024-05-03T18:49:45.484933+0000 (oldest deadline 2024-05-03T18:50:14.885581+0000) 2024-05-03T18:50:17.634 INFO:journalctl@ceph.osd.5.smithi150.stdout:May 03 18:50:17 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-5[44561]: 2024-05-03T18:50:17.422+0000 7fdfa7000700 -1 osd.5 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:28.083853+0000 front 2024-05-03T18:49:28.084464+0000 (oldest deadline 2024-05-03T18:49:52.685392+0000) 2024-05-03T18:50:17.634 INFO:journalctl@ceph.osd.5.smithi150.stdout:May 03 18:50:17 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-5[44561]: 2024-05-03T18:50:17.422+0000 7fdfa7000700 -1 osd.5 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:38.586865+0000 front 2024-05-03T18:49:38.586927+0000 (oldest deadline 2024-05-03T18:50:04.487124+0000) 2024-05-03T18:50:17.634 INFO:journalctl@ceph.osd.5.smithi150.stdout:May 03 18:50:17 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-5[44561]: 2024-05-03T18:50:17.422+0000 7fdfa7000700 -1 osd.5 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:49.188983+0000 front 2024-05-03T18:49:49.189367+0000 (oldest deadline 2024-05-03T18:50:15.089043+0000) 2024-05-03T18:50:17.734 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:17 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:17.398+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:27.608012+0000 front 2024-05-03T18:49:27.608078+0000 (oldest deadline 2024-05-03T18:49:53.508290+0000) 2024-05-03T18:50:17.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:17 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:17.398+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:38.811037+0000 front 2024-05-03T18:49:38.811219+0000 (oldest deadline 2024-05-03T18:50:01.611951+0000) 2024-05-03T18:50:17.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:17 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:17.398+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:46.215926+0000 front 2024-05-03T18:49:46.216076+0000 (oldest deadline 2024-05-03T18:50:12.116128+0000) 2024-05-03T18:50:18.047 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:17 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7[49675]: 2024-05-03T18:50:17.788+0000 7ff8ace00700 -1 osd.7 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:36.774719+0000 front 2024-05-03T18:49:36.774769+0000 (oldest deadline 2024-05-03T18:50:02.674865+0000) 2024-05-03T18:50:18.047 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:17 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7[49675]: 2024-05-03T18:50:17.788+0000 7ff8ace00700 -1 osd.7 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:47.376623+0000 front 2024-05-03T18:49:47.376700+0000 (oldest deadline 2024-05-03T18:50:10.876637+0000) 2024-05-03T18:50:18.235 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 03 18:50:17 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-6[56391]: 2024-05-03T18:50:17.926+0000 7f3ff2200700 -1 osd.6 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:37.490057+0000 front 2024-05-03T18:49:37.490102+0000 (oldest deadline 2024-05-03T18:50:02.789772+0000) 2024-05-03T18:50:18.235 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 03 18:50:17 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-6[56391]: 2024-05-03T18:50:17.926+0000 7f3ff2200700 -1 osd.6 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:48.692256+0000 front 2024-05-03T18:49:48.692122+0000 (oldest deadline 2024-05-03T18:50:12.792315+0000) 2024-05-03T18:50:18.301 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:17 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:17.906+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:24.877883+0000 front 2024-05-03T18:49:24.877625+0000 (oldest deadline 2024-05-03T18:49:50.777881+0000) 2024-05-03T18:50:18.301 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:17 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:17.906+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:47.685409+0000 front 2024-05-03T18:49:47.685527+0000 (oldest deadline 2024-05-03T18:50:11.785436+0000) 2024-05-03T18:50:18.546 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:18 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:18.267+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:26.972061+0000 front 2024-05-03T18:49:26.972448+0000 (oldest deadline 2024-05-03T18:49:50.472344+0000) 2024-05-03T18:50:18.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:18 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:18.267+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:37.476081+0000 front 2024-05-03T18:49:37.476172+0000 (oldest deadline 2024-05-03T18:50:01.576192+0000) 2024-05-03T18:50:18.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:18 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:18.267+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:49.079592+0000 front 2024-05-03T18:49:49.080117+0000 (oldest deadline 2024-05-03T18:50:10.779802+0000) 2024-05-03T18:50:18.633 INFO:journalctl@ceph.osd.5.smithi150.stdout:May 03 18:50:18 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-5[44561]: 2024-05-03T18:50:18.404+0000 7fdfa7000700 -1 osd.5 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:28.083853+0000 front 2024-05-03T18:49:28.084464+0000 (oldest deadline 2024-05-03T18:49:52.685392+0000) 2024-05-03T18:50:18.633 INFO:journalctl@ceph.osd.5.smithi150.stdout:May 03 18:50:18 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-5[44561]: 2024-05-03T18:50:18.404+0000 7fdfa7000700 -1 osd.5 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:38.586865+0000 front 2024-05-03T18:49:38.586927+0000 (oldest deadline 2024-05-03T18:50:04.487124+0000) 2024-05-03T18:50:18.633 INFO:journalctl@ceph.osd.5.smithi150.stdout:May 03 18:50:18 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-5[44561]: 2024-05-03T18:50:18.404+0000 7fdfa7000700 -1 osd.5 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:49.188983+0000 front 2024-05-03T18:49:49.189367+0000 (oldest deadline 2024-05-03T18:50:15.089043+0000) 2024-05-03T18:50:18.634 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:18 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:18.298+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:25.480913+0000 front 2024-05-03T18:49:25.480403+0000 (oldest deadline 2024-05-03T18:49:50.180152+0000) 2024-05-03T18:50:18.634 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:18 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:18.298+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:34.882184+0000 front 2024-05-03T18:49:34.882683+0000 (oldest deadline 2024-05-03T18:50:05.483250+0000) 2024-05-03T18:50:18.634 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:18 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:18.298+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:45.484981+0000 front 2024-05-03T18:49:45.484933+0000 (oldest deadline 2024-05-03T18:50:14.885581+0000) 2024-05-03T18:50:18.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:18 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:18.370+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:27.608012+0000 front 2024-05-03T18:49:27.608078+0000 (oldest deadline 2024-05-03T18:49:53.508290+0000) 2024-05-03T18:50:18.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:18 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:18.370+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:38.811037+0000 front 2024-05-03T18:49:38.811219+0000 (oldest deadline 2024-05-03T18:50:01.611951+0000) 2024-05-03T18:50:18.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:18 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:18.370+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:46.215926+0000 front 2024-05-03T18:49:46.216076+0000 (oldest deadline 2024-05-03T18:50:12.116128+0000) 2024-05-03T18:50:19.046 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:18 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7[49675]: 2024-05-03T18:50:18.752+0000 7ff8ace00700 -1 osd.7 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:36.774719+0000 front 2024-05-03T18:49:36.774769+0000 (oldest deadline 2024-05-03T18:50:02.674865+0000) 2024-05-03T18:50:19.047 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:18 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7[49675]: 2024-05-03T18:50:18.752+0000 7ff8ace00700 -1 osd.7 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:47.376623+0000 front 2024-05-03T18:49:47.376700+0000 (oldest deadline 2024-05-03T18:50:10.876637+0000) 2024-05-03T18:50:19.235 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 03 18:50:18 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-6[56391]: 2024-05-03T18:50:18.893+0000 7f3ff2200700 -1 osd.6 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:37.490057+0000 front 2024-05-03T18:49:37.490102+0000 (oldest deadline 2024-05-03T18:50:02.789772+0000) 2024-05-03T18:50:19.235 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 03 18:50:18 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-6[56391]: 2024-05-03T18:50:18.893+0000 7f3ff2200700 -1 osd.6 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:48.692256+0000 front 2024-05-03T18:49:48.692122+0000 (oldest deadline 2024-05-03T18:50:12.792315+0000) 2024-05-03T18:50:19.253 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:18 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:18.945+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:24.877883+0000 front 2024-05-03T18:49:24.877625+0000 (oldest deadline 2024-05-03T18:49:50.777881+0000) 2024-05-03T18:50:19.253 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:18 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:18.945+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:47.685409+0000 front 2024-05-03T18:49:47.685527+0000 (oldest deadline 2024-05-03T18:50:11.785436+0000) 2024-05-03T18:50:19.633 INFO:journalctl@ceph.osd.5.smithi150.stdout:May 03 18:50:19 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-5[44561]: 2024-05-03T18:50:19.365+0000 7fdfa7000700 -1 osd.5 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:28.083853+0000 front 2024-05-03T18:49:28.084464+0000 (oldest deadline 2024-05-03T18:49:52.685392+0000) 2024-05-03T18:50:19.633 INFO:journalctl@ceph.osd.5.smithi150.stdout:May 03 18:50:19 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-5[44561]: 2024-05-03T18:50:19.366+0000 7fdfa7000700 -1 osd.5 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:38.586865+0000 front 2024-05-03T18:49:38.586927+0000 (oldest deadline 2024-05-03T18:50:04.487124+0000) 2024-05-03T18:50:19.633 INFO:journalctl@ceph.osd.5.smithi150.stdout:May 03 18:50:19 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-5[44561]: 2024-05-03T18:50:19.366+0000 7fdfa7000700 -1 osd.5 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:49.188983+0000 front 2024-05-03T18:49:49.189367+0000 (oldest deadline 2024-05-03T18:50:15.089043+0000) 2024-05-03T18:50:19.634 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:19 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:19.251+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:25.480913+0000 front 2024-05-03T18:49:25.480403+0000 (oldest deadline 2024-05-03T18:49:50.180152+0000) 2024-05-03T18:50:19.634 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:19 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:19.251+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:34.882184+0000 front 2024-05-03T18:49:34.882683+0000 (oldest deadline 2024-05-03T18:50:05.483250+0000) 2024-05-03T18:50:19.634 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:19 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:19.251+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:45.484981+0000 front 2024-05-03T18:49:45.484933+0000 (oldest deadline 2024-05-03T18:50:14.885581+0000) 2024-05-03T18:50:19.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:19 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:19.336+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:27.608012+0000 front 2024-05-03T18:49:27.608078+0000 (oldest deadline 2024-05-03T18:49:53.508290+0000) 2024-05-03T18:50:19.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:19 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:19.336+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:38.811037+0000 front 2024-05-03T18:49:38.811219+0000 (oldest deadline 2024-05-03T18:50:01.611951+0000) 2024-05-03T18:50:19.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:19 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:19.336+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:46.215926+0000 front 2024-05-03T18:49:46.216076+0000 (oldest deadline 2024-05-03T18:50:12.116128+0000) 2024-05-03T18:50:19.771 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:19 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:19.315+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:26.972061+0000 front 2024-05-03T18:49:26.972448+0000 (oldest deadline 2024-05-03T18:49:50.472344+0000) 2024-05-03T18:50:19.771 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:19 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:19.315+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:37.476081+0000 front 2024-05-03T18:49:37.476172+0000 (oldest deadline 2024-05-03T18:50:01.576192+0000) 2024-05-03T18:50:19.771 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:19 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:19.315+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:49.079592+0000 front 2024-05-03T18:49:49.080117+0000 (oldest deadline 2024-05-03T18:50:10.779802+0000) 2024-05-03T18:50:20.046 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:19 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7[49675]: 2024-05-03T18:50:19.769+0000 7ff8ace00700 -1 osd.7 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:36.774719+0000 front 2024-05-03T18:49:36.774769+0000 (oldest deadline 2024-05-03T18:50:02.674865+0000) 2024-05-03T18:50:20.046 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:19 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7[49675]: 2024-05-03T18:50:19.769+0000 7ff8ace00700 -1 osd.7 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:47.376623+0000 front 2024-05-03T18:49:47.376700+0000 (oldest deadline 2024-05-03T18:50:10.876637+0000) 2024-05-03T18:50:20.235 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 03 18:50:19 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-6[56391]: 2024-05-03T18:50:19.848+0000 7f3ff2200700 -1 osd.6 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:37.490057+0000 front 2024-05-03T18:49:37.490102+0000 (oldest deadline 2024-05-03T18:50:02.789772+0000) 2024-05-03T18:50:20.235 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 03 18:50:19 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-6[56391]: 2024-05-03T18:50:19.848+0000 7f3ff2200700 -1 osd.6 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:48.692256+0000 front 2024-05-03T18:49:48.692122+0000 (oldest deadline 2024-05-03T18:50:12.792315+0000) 2024-05-03T18:50:20.342 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:19 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:19.978+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:24.877883+0000 front 2024-05-03T18:49:24.877625+0000 (oldest deadline 2024-05-03T18:49:50.777881+0000) 2024-05-03T18:50:20.342 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:19 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:19.978+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:47.685409+0000 front 2024-05-03T18:49:47.685527+0000 (oldest deadline 2024-05-03T18:50:11.785436+0000) 2024-05-03T18:50:20.343 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:20 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:20.210+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:25.480913+0000 front 2024-05-03T18:49:25.480403+0000 (oldest deadline 2024-05-03T18:49:50.180152+0000) 2024-05-03T18:50:20.343 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:20 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:20.210+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:34.882184+0000 front 2024-05-03T18:49:34.882683+0000 (oldest deadline 2024-05-03T18:50:05.483250+0000) 2024-05-03T18:50:20.343 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:20 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:20.210+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:45.484981+0000 front 2024-05-03T18:49:45.484933+0000 (oldest deadline 2024-05-03T18:50:14.885581+0000) 2024-05-03T18:50:20.633 INFO:journalctl@ceph.osd.5.smithi150.stdout:May 03 18:50:20 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-5[44561]: 2024-05-03T18:50:20.340+0000 7fdfa7000700 -1 osd.5 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:28.083853+0000 front 2024-05-03T18:49:28.084464+0000 (oldest deadline 2024-05-03T18:49:52.685392+0000) 2024-05-03T18:50:20.633 INFO:journalctl@ceph.osd.5.smithi150.stdout:May 03 18:50:20 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-5[44561]: 2024-05-03T18:50:20.340+0000 7fdfa7000700 -1 osd.5 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:38.586865+0000 front 2024-05-03T18:49:38.586927+0000 (oldest deadline 2024-05-03T18:50:04.487124+0000) 2024-05-03T18:50:20.633 INFO:journalctl@ceph.osd.5.smithi150.stdout:May 03 18:50:20 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-5[44561]: 2024-05-03T18:50:20.340+0000 7fdfa7000700 -1 osd.5 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:49.188983+0000 front 2024-05-03T18:49:49.189367+0000 (oldest deadline 2024-05-03T18:50:15.089043+0000) 2024-05-03T18:50:20.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:20 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:20.346+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:27.608012+0000 front 2024-05-03T18:49:27.608078+0000 (oldest deadline 2024-05-03T18:49:53.508290+0000) 2024-05-03T18:50:20.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:20 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:20.346+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:38.811037+0000 front 2024-05-03T18:49:38.811219+0000 (oldest deadline 2024-05-03T18:50:01.611951+0000) 2024-05-03T18:50:20.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:20 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:20.346+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:46.215926+0000 front 2024-05-03T18:49:46.216076+0000 (oldest deadline 2024-05-03T18:50:12.116128+0000) 2024-05-03T18:50:20.752 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:20 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:20.343+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:26.972061+0000 front 2024-05-03T18:49:26.972448+0000 (oldest deadline 2024-05-03T18:49:50.472344+0000) 2024-05-03T18:50:20.752 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:20 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:20.343+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:37.476081+0000 front 2024-05-03T18:49:37.476172+0000 (oldest deadline 2024-05-03T18:50:01.576192+0000) 2024-05-03T18:50:20.752 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:20 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:20.343+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:49.079592+0000 front 2024-05-03T18:49:49.080117+0000 (oldest deadline 2024-05-03T18:50:10.779802+0000) 2024-05-03T18:50:21.046 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:20 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7[49675]: 2024-05-03T18:50:20.749+0000 7ff8ace00700 -1 osd.7 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:36.774719+0000 front 2024-05-03T18:49:36.774769+0000 (oldest deadline 2024-05-03T18:50:02.674865+0000) 2024-05-03T18:50:21.046 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:20 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7[49675]: 2024-05-03T18:50:20.749+0000 7ff8ace00700 -1 osd.7 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:47.376623+0000 front 2024-05-03T18:49:47.376700+0000 (oldest deadline 2024-05-03T18:50:10.876637+0000) 2024-05-03T18:50:21.235 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 03 18:50:20 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-6[56391]: 2024-05-03T18:50:20.884+0000 7f3ff2200700 -1 osd.6 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:37.490057+0000 front 2024-05-03T18:49:37.490102+0000 (oldest deadline 2024-05-03T18:50:02.789772+0000) 2024-05-03T18:50:21.235 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 03 18:50:20 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-6[56391]: 2024-05-03T18:50:20.884+0000 7f3ff2200700 -1 osd.6 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:48.692256+0000 front 2024-05-03T18:49:48.692122+0000 (oldest deadline 2024-05-03T18:50:12.792315+0000) 2024-05-03T18:50:21.383 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:21 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:21.182+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:25.480913+0000 front 2024-05-03T18:49:25.480403+0000 (oldest deadline 2024-05-03T18:49:50.180152+0000) 2024-05-03T18:50:21.383 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:21 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:21.182+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:34.882184+0000 front 2024-05-03T18:49:34.882683+0000 (oldest deadline 2024-05-03T18:50:05.483250+0000) 2024-05-03T18:50:21.383 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:21 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:21.182+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:45.484981+0000 front 2024-05-03T18:49:45.484933+0000 (oldest deadline 2024-05-03T18:50:14.885581+0000) 2024-05-03T18:50:21.384 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:20 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:20.985+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:24.877883+0000 front 2024-05-03T18:49:24.877625+0000 (oldest deadline 2024-05-03T18:49:50.777881+0000) 2024-05-03T18:50:21.384 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:20 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:20.985+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:47.685409+0000 front 2024-05-03T18:49:47.685527+0000 (oldest deadline 2024-05-03T18:50:11.785436+0000) 2024-05-03T18:50:21.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:21 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:21.357+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:27.608012+0000 front 2024-05-03T18:49:27.608078+0000 (oldest deadline 2024-05-03T18:49:53.508290+0000) 2024-05-03T18:50:21.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:21 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:21.357+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:38.811037+0000 front 2024-05-03T18:49:38.811219+0000 (oldest deadline 2024-05-03T18:50:01.611951+0000) 2024-05-03T18:50:21.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:21 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:21.357+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:46.215926+0000 front 2024-05-03T18:49:46.216076+0000 (oldest deadline 2024-05-03T18:50:12.116128+0000) 2024-05-03T18:50:21.796 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:21 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:21.377+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:26.972061+0000 front 2024-05-03T18:49:26.972448+0000 (oldest deadline 2024-05-03T18:49:50.472344+0000) 2024-05-03T18:50:21.796 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:21 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:21.377+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:37.476081+0000 front 2024-05-03T18:49:37.476172+0000 (oldest deadline 2024-05-03T18:50:01.576192+0000) 2024-05-03T18:50:21.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:21 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:21.377+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:49.079592+0000 front 2024-05-03T18:49:49.080117+0000 (oldest deadline 2024-05-03T18:50:10.779802+0000) 2024-05-03T18:50:22.059 INFO:journalctl@ceph.osd.5.smithi150.stdout:May 03 18:50:22 smithi150 podman[71925]: 2024-05-03 18:50:22.007744968 +0000 UTC m=+6.485142020 container died 87bc6b436276a9c4a4efd6c2fe8b7f13f0b5a3ce6423ac5fa504f3e320da6579 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-5, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240430) 2024-05-03T18:50:22.059 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:21 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:21.949+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:24.877883+0000 front 2024-05-03T18:49:24.877625+0000 (oldest deadline 2024-05-03T18:49:50.777881+0000) 2024-05-03T18:50:22.059 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:21 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:21.949+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:47.685409+0000 front 2024-05-03T18:49:47.685527+0000 (oldest deadline 2024-05-03T18:50:11.785436+0000) 2024-05-03T18:50:22.235 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 03 18:50:21 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-6[56391]: 2024-05-03T18:50:21.903+0000 7f3ff2200700 -1 osd.6 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:37.490057+0000 front 2024-05-03T18:49:37.490102+0000 (oldest deadline 2024-05-03T18:50:02.789772+0000) 2024-05-03T18:50:22.236 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 03 18:50:21 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-6[56391]: 2024-05-03T18:50:21.903+0000 7f3ff2200700 -1 osd.6 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:48.692256+0000 front 2024-05-03T18:49:48.692122+0000 (oldest deadline 2024-05-03T18:50:12.792315+0000) 2024-05-03T18:50:22.297 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:21 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7[49675]: 2024-05-03T18:50:21.795+0000 7ff8ace00700 -1 osd.7 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:36.774719+0000 front 2024-05-03T18:49:36.774769+0000 (oldest deadline 2024-05-03T18:50:02.674865+0000) 2024-05-03T18:50:22.297 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:21 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7[49675]: 2024-05-03T18:50:21.795+0000 7ff8ace00700 -1 osd.7 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:47.376623+0000 front 2024-05-03T18:49:47.376700+0000 (oldest deadline 2024-05-03T18:50:10.876637+0000) 2024-05-03T18:50:22.329 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:22 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:22.137+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:25.480913+0000 front 2024-05-03T18:49:25.480403+0000 (oldest deadline 2024-05-03T18:49:50.180152+0000) 2024-05-03T18:50:22.329 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:22 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:22.137+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:34.882184+0000 front 2024-05-03T18:49:34.882683+0000 (oldest deadline 2024-05-03T18:50:05.483250+0000) 2024-05-03T18:50:22.329 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:22 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:22.137+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:45.484981+0000 front 2024-05-03T18:49:45.484933+0000 (oldest deadline 2024-05-03T18:50:14.885581+0000) 2024-05-03T18:50:22.633 INFO:journalctl@ceph.osd.5.smithi150.stdout:May 03 18:50:22 smithi150 podman[71925]: 2024-05-03 18:50:22.473187874 +0000 UTC m=+6.950584970 container cleanup 87bc6b436276a9c4a4efd6c2fe8b7f13f0b5a3ce6423ac5fa504f3e320da6579 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-05-03T18:50:22.633 INFO:journalctl@ceph.osd.5.smithi150.stdout:May 03 18:50:22 smithi150 bash[71925]: ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-5 2024-05-03T18:50:22.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:22 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:22.386+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:27.608012+0000 front 2024-05-03T18:49:27.608078+0000 (oldest deadline 2024-05-03T18:49:53.508290+0000) 2024-05-03T18:50:22.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:22 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:22.386+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:38.811037+0000 front 2024-05-03T18:49:38.811219+0000 (oldest deadline 2024-05-03T18:50:01.611951+0000) 2024-05-03T18:50:22.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:22 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:22.386+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:46.215926+0000 front 2024-05-03T18:49:46.216076+0000 (oldest deadline 2024-05-03T18:50:12.116128+0000) 2024-05-03T18:50:22.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:22 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:22.386+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:56.719409+0000 front 2024-05-03T18:49:56.719531+0000 (oldest deadline 2024-05-03T18:50:21.419652+0000) 2024-05-03T18:50:22.787 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:22 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:22.333+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:26.972061+0000 front 2024-05-03T18:49:26.972448+0000 (oldest deadline 2024-05-03T18:49:50.472344+0000) 2024-05-03T18:50:22.787 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:22 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:22.333+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:37.476081+0000 front 2024-05-03T18:49:37.476172+0000 (oldest deadline 2024-05-03T18:50:01.576192+0000) 2024-05-03T18:50:22.787 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:22 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:22.333+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:49.079592+0000 front 2024-05-03T18:49:49.080117+0000 (oldest deadline 2024-05-03T18:50:10.779802+0000) 2024-05-03T18:50:22.787 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:22 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:22.333+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:59.985863+0000 front 2024-05-03T18:49:59.986111+0000 (oldest deadline 2024-05-03T18:50:22.285990+0000) 2024-05-03T18:50:22.941 INFO:journalctl@ceph.osd.5.smithi150.stdout:May 03 18:50:22 smithi150 podman[71950]: 2024-05-03 18:50:22.691011704 +0000 UTC m=+0.661534454 container remove 87bc6b436276a9c4a4efd6c2fe8b7f13f0b5a3ce6423ac5fa504f3e320da6579 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.build-date=20240430, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True) 2024-05-03T18:50:23.046 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:22 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7[49675]: 2024-05-03T18:50:22.784+0000 7ff8ace00700 -1 osd.7 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:36.774719+0000 front 2024-05-03T18:49:36.774769+0000 (oldest deadline 2024-05-03T18:50:02.674865+0000) 2024-05-03T18:50:23.046 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:22 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7[49675]: 2024-05-03T18:50:22.784+0000 7ff8ace00700 -1 osd.7 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:47.376623+0000 front 2024-05-03T18:49:47.376700+0000 (oldest deadline 2024-05-03T18:50:10.876637+0000) 2024-05-03T18:50:23.235 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 03 18:50:22 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-6[56391]: 2024-05-03T18:50:22.936+0000 7f3ff2200700 -1 osd.6 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:37.490057+0000 front 2024-05-03T18:49:37.490102+0000 (oldest deadline 2024-05-03T18:50:02.789772+0000) 2024-05-03T18:50:23.235 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 03 18:50:22 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-6[56391]: 2024-05-03T18:50:22.936+0000 7f3ff2200700 -1 osd.6 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:48.692256+0000 front 2024-05-03T18:49:48.692122+0000 (oldest deadline 2024-05-03T18:50:12.792315+0000) 2024-05-03T18:50:23.235 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 03 18:50:22 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-6[56391]: 2024-05-03T18:50:22.936+0000 7f3ff2200700 -1 osd.6 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:56.294541+0000 front 2024-05-03T18:49:56.294785+0000 (oldest deadline 2024-05-03T18:50:22.194699+0000) 2024-05-03T18:50:23.383 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:22 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:22.943+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:24.877883+0000 front 2024-05-03T18:49:24.877625+0000 (oldest deadline 2024-05-03T18:49:50.777881+0000) 2024-05-03T18:50:23.383 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:22 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:22.943+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:47.685409+0000 front 2024-05-03T18:49:47.685527+0000 (oldest deadline 2024-05-03T18:50:11.785436+0000) 2024-05-03T18:50:23.383 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:23 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:23.129+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:25.480913+0000 front 2024-05-03T18:49:25.480403+0000 (oldest deadline 2024-05-03T18:49:50.180152+0000) 2024-05-03T18:50:23.384 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:23 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:23.129+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:34.882184+0000 front 2024-05-03T18:49:34.882683+0000 (oldest deadline 2024-05-03T18:50:05.483250+0000) 2024-05-03T18:50:23.384 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:23 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:23.129+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:45.484981+0000 front 2024-05-03T18:49:45.484933+0000 (oldest deadline 2024-05-03T18:50:14.885581+0000) 2024-05-03T18:50:23.618 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:23 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:23.311+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:26.972061+0000 front 2024-05-03T18:49:26.972448+0000 (oldest deadline 2024-05-03T18:49:50.472344+0000) 2024-05-03T18:50:23.618 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:23 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:23.311+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:37.476081+0000 front 2024-05-03T18:49:37.476172+0000 (oldest deadline 2024-05-03T18:50:01.576192+0000) 2024-05-03T18:50:23.618 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:23 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:23.311+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:49.079592+0000 front 2024-05-03T18:49:49.080117+0000 (oldest deadline 2024-05-03T18:50:10.779802+0000) 2024-05-03T18:50:23.618 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:23 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:23.311+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:59.985863+0000 front 2024-05-03T18:49:59.986111+0000 (oldest deadline 2024-05-03T18:50:22.285990+0000) 2024-05-03T18:50:23.687 INFO:journalctl@ceph.osd.5.smithi150.stdout:May 03 18:50:23 smithi150 podman[72048]: 2024-05-03 18:50:23.344292727 +0000 UTC m=+0.046884079 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T18:50:23.687 INFO:journalctl@ceph.osd.5.smithi150.stdout:May 03 18:50:23 smithi150 podman[72048]: 2024-05-03 18:50:23.492195152 +0000 UTC m=+0.194786507 container create f7e7667b8441605d8cb7a4ce18ca13302df1f31e095ffd872940e244cfcfd51a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-5-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe) 2024-05-03T18:50:23.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:23 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:23.339+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:27.608012+0000 front 2024-05-03T18:49:27.608078+0000 (oldest deadline 2024-05-03T18:49:53.508290+0000) 2024-05-03T18:50:23.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:23 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:23.339+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:38.811037+0000 front 2024-05-03T18:49:38.811219+0000 (oldest deadline 2024-05-03T18:50:01.611951+0000) 2024-05-03T18:50:23.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:23 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:23.339+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:46.215926+0000 front 2024-05-03T18:49:46.216076+0000 (oldest deadline 2024-05-03T18:50:12.116128+0000) 2024-05-03T18:50:23.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:23 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:23.339+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:56.719409+0000 front 2024-05-03T18:49:56.719531+0000 (oldest deadline 2024-05-03T18:50:21.419652+0000) 2024-05-03T18:50:24.046 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:23 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7[49675]: 2024-05-03T18:50:23.828+0000 7ff8ace00700 -1 osd.7 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:36.774719+0000 front 2024-05-03T18:49:36.774769+0000 (oldest deadline 2024-05-03T18:50:02.674865+0000) 2024-05-03T18:50:24.047 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:23 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7[49675]: 2024-05-03T18:50:23.828+0000 7ff8ace00700 -1 osd.7 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:47.376623+0000 front 2024-05-03T18:49:47.376700+0000 (oldest deadline 2024-05-03T18:50:10.876637+0000) 2024-05-03T18:50:24.047 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:23 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7[49675]: 2024-05-03T18:50:23.828+0000 7ff8ace00700 -1 osd.7 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:59.080150+0000 front 2024-05-03T18:49:59.080046+0000 (oldest deadline 2024-05-03T18:50:23.179826+0000) 2024-05-03T18:50:24.133 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:23 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:23.975+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:24.877883+0000 front 2024-05-03T18:49:24.877625+0000 (oldest deadline 2024-05-03T18:49:50.777881+0000) 2024-05-03T18:50:24.133 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:23 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:23.976+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:47.685409+0000 front 2024-05-03T18:49:47.685527+0000 (oldest deadline 2024-05-03T18:50:11.785436+0000) 2024-05-03T18:50:24.133 INFO:journalctl@ceph.osd.5.smithi150.stdout:May 03 18:50:23 smithi150 podman[72048]: 2024-05-03 18:50:23.787684393 +0000 UTC m=+0.490275754 container init f7e7667b8441605d8cb7a4ce18ca13302df1f31e095ffd872940e244cfcfd51a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-03T18:50:24.134 INFO:journalctl@ceph.osd.5.smithi150.stdout:May 03 18:50:23 smithi150 podman[72048]: 2024-05-03 18:50:23.798878676 +0000 UTC m=+0.501470041 container start f7e7667b8441605d8cb7a4ce18ca13302df1f31e095ffd872940e244cfcfd51a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-5-deactivate, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-05-03T18:50:24.134 INFO:journalctl@ceph.osd.5.smithi150.stdout:May 03 18:50:23 smithi150 podman[72048]: 2024-05-03 18:50:23.9027276 +0000 UTC m=+0.605319042 container attach f7e7667b8441605d8cb7a4ce18ca13302df1f31e095ffd872940e244cfcfd51a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-03T18:50:24.235 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 03 18:50:23 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-6[56391]: 2024-05-03T18:50:23.925+0000 7f3ff2200700 -1 osd.6 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:37.490057+0000 front 2024-05-03T18:49:37.490102+0000 (oldest deadline 2024-05-03T18:50:02.789772+0000) 2024-05-03T18:50:24.235 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 03 18:50:23 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-6[56391]: 2024-05-03T18:50:23.925+0000 7f3ff2200700 -1 osd.6 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:48.692256+0000 front 2024-05-03T18:49:48.692122+0000 (oldest deadline 2024-05-03T18:50:12.792315+0000) 2024-05-03T18:50:24.235 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 03 18:50:23 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-6[56391]: 2024-05-03T18:50:23.925+0000 7f3ff2200700 -1 osd.6 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:56.294541+0000 front 2024-05-03T18:49:56.294785+0000 (oldest deadline 2024-05-03T18:50:22.194699+0000) 2024-05-03T18:50:24.569 INFO:journalctl@ceph.osd.5.smithi150.stdout:May 03 18:50:24 smithi150 podman[72048]: 2024-05-03 18:50:24.23588078 +0000 UTC m=+0.938472182 container died f7e7667b8441605d8cb7a4ce18ca13302df1f31e095ffd872940e244cfcfd51a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240430) 2024-05-03T18:50:24.570 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:24 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:24.140+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:25.480913+0000 front 2024-05-03T18:49:25.480403+0000 (oldest deadline 2024-05-03T18:49:50.180152+0000) 2024-05-03T18:50:24.570 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:24 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:24.140+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:34.882184+0000 front 2024-05-03T18:49:34.882683+0000 (oldest deadline 2024-05-03T18:50:05.483250+0000) 2024-05-03T18:50:24.570 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:24 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:24.140+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:45.484981+0000 front 2024-05-03T18:49:45.484933+0000 (oldest deadline 2024-05-03T18:50:14.885581+0000) 2024-05-03T18:50:24.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:24 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:24.296+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:27.608012+0000 front 2024-05-03T18:49:27.608078+0000 (oldest deadline 2024-05-03T18:49:53.508290+0000) 2024-05-03T18:50:24.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:24 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:24.296+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:38.811037+0000 front 2024-05-03T18:49:38.811219+0000 (oldest deadline 2024-05-03T18:50:01.611951+0000) 2024-05-03T18:50:24.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:24 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:24.296+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:46.215926+0000 front 2024-05-03T18:49:46.216076+0000 (oldest deadline 2024-05-03T18:50:12.116128+0000) 2024-05-03T18:50:24.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:24 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:24.296+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:56.719409+0000 front 2024-05-03T18:49:56.719531+0000 (oldest deadline 2024-05-03T18:50:21.419652+0000) 2024-05-03T18:50:24.796 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:24 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:24.359+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:26.972061+0000 front 2024-05-03T18:49:26.972448+0000 (oldest deadline 2024-05-03T18:49:50.472344+0000) 2024-05-03T18:50:24.796 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:24 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:24.359+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:37.476081+0000 front 2024-05-03T18:49:37.476172+0000 (oldest deadline 2024-05-03T18:50:01.576192+0000) 2024-05-03T18:50:24.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:24 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:24.359+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:49.079592+0000 front 2024-05-03T18:49:49.080117+0000 (oldest deadline 2024-05-03T18:50:10.779802+0000) 2024-05-03T18:50:24.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:24 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:24.359+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:59.985863+0000 front 2024-05-03T18:49:59.986111+0000 (oldest deadline 2024-05-03T18:50:22.285990+0000) 2024-05-03T18:50:24.902 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5703bba4-0978-11ef-bc97-c7b262605968@osd.5.service' 2024-05-03T18:50:25.006 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T18:50:25.006 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-05-03T18:50:25.007 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-05-03T18:50:25.007 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-5703bba4-0978-11ef-bc97-c7b262605968@osd.6 2024-05-03T18:50:25.133 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:24 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:24.943+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:24.877883+0000 front 2024-05-03T18:49:24.877625+0000 (oldest deadline 2024-05-03T18:49:50.777881+0000) 2024-05-03T18:50:25.134 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:24 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:24.943+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:47.685409+0000 front 2024-05-03T18:49:47.685527+0000 (oldest deadline 2024-05-03T18:50:11.785436+0000) 2024-05-03T18:50:25.134 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:24 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:24.943+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:57.687614+0000 front 2024-05-03T18:49:57.687504+0000 (oldest deadline 2024-05-03T18:50:24.688928+0000) 2024-05-03T18:50:25.235 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 03 18:50:24 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-6[56391]: 2024-05-03T18:50:24.934+0000 7f3ff2200700 -1 osd.6 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:37.490057+0000 front 2024-05-03T18:49:37.490102+0000 (oldest deadline 2024-05-03T18:50:02.789772+0000) 2024-05-03T18:50:25.236 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 03 18:50:24 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-6[56391]: 2024-05-03T18:50:24.934+0000 7f3ff2200700 -1 osd.6 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:48.692256+0000 front 2024-05-03T18:49:48.692122+0000 (oldest deadline 2024-05-03T18:50:12.792315+0000) 2024-05-03T18:50:25.236 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 03 18:50:24 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-6[56391]: 2024-05-03T18:50:24.934+0000 7f3ff2200700 -1 osd.6 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:56.294541+0000 front 2024-05-03T18:49:56.294785+0000 (oldest deadline 2024-05-03T18:50:22.194699+0000) 2024-05-03T18:50:25.236 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 03 18:50:25 smithi033 systemd[1]: Stopping Ceph osd.6 for 5703bba4-0978-11ef-bc97-c7b262605968... 2024-05-03T18:50:25.296 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:24 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7[49675]: 2024-05-03T18:50:24.834+0000 7ff8ace00700 -1 osd.7 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:36.774719+0000 front 2024-05-03T18:49:36.774769+0000 (oldest deadline 2024-05-03T18:50:02.674865+0000) 2024-05-03T18:50:25.297 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:24 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7[49675]: 2024-05-03T18:50:24.834+0000 7ff8ace00700 -1 osd.7 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:47.376623+0000 front 2024-05-03T18:49:47.376700+0000 (oldest deadline 2024-05-03T18:50:10.876637+0000) 2024-05-03T18:50:25.297 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:24 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7[49675]: 2024-05-03T18:50:24.834+0000 7ff8ace00700 -1 osd.7 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:59.080150+0000 front 2024-05-03T18:49:59.080046+0000 (oldest deadline 2024-05-03T18:50:23.179826+0000) 2024-05-03T18:50:25.626 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:25 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:25.328+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:27.608012+0000 front 2024-05-03T18:49:27.608078+0000 (oldest deadline 2024-05-03T18:49:53.508290+0000) 2024-05-03T18:50:25.626 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:25 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:25.328+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:38.811037+0000 front 2024-05-03T18:49:38.811219+0000 (oldest deadline 2024-05-03T18:50:01.611951+0000) 2024-05-03T18:50:25.626 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:25 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:25.328+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:46.215926+0000 front 2024-05-03T18:49:46.216076+0000 (oldest deadline 2024-05-03T18:50:12.116128+0000) 2024-05-03T18:50:25.626 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:25 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:25.328+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:56.719409+0000 front 2024-05-03T18:49:56.719531+0000 (oldest deadline 2024-05-03T18:50:21.419652+0000) 2024-05-03T18:50:25.633 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:25 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:25.186+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:25.480913+0000 front 2024-05-03T18:49:25.480403+0000 (oldest deadline 2024-05-03T18:49:50.180152+0000) 2024-05-03T18:50:25.633 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:25 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:25.186+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:34.882184+0000 front 2024-05-03T18:49:34.882683+0000 (oldest deadline 2024-05-03T18:50:05.483250+0000) 2024-05-03T18:50:25.633 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:25 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:25.186+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:45.484981+0000 front 2024-05-03T18:49:45.484933+0000 (oldest deadline 2024-05-03T18:50:14.885581+0000) 2024-05-03T18:50:25.787 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:25 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:25.325+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:26.972061+0000 front 2024-05-03T18:49:26.972448+0000 (oldest deadline 2024-05-03T18:49:50.472344+0000) 2024-05-03T18:50:25.787 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:25 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:25.325+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:37.476081+0000 front 2024-05-03T18:49:37.476172+0000 (oldest deadline 2024-05-03T18:50:01.576192+0000) 2024-05-03T18:50:25.788 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:25 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:25.325+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:49.079592+0000 front 2024-05-03T18:49:49.080117+0000 (oldest deadline 2024-05-03T18:50:10.779802+0000) 2024-05-03T18:50:25.788 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:25 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:25.325+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:59.985863+0000 front 2024-05-03T18:49:59.986111+0000 (oldest deadline 2024-05-03T18:50:22.285990+0000) 2024-05-03T18:50:25.887 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 03 18:50:25 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-6[56391]: 2024-05-03T18:50:25.624+0000 7f3ff6a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T18:50:25.887 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 03 18:50:25 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-6[56391]: 2024-05-03T18:50:25.624+0000 7f3ff6a00700 -1 osd.6 126 *** Got signal Terminated *** 2024-05-03T18:50:25.887 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 03 18:50:25 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-6[56391]: 2024-05-03T18:50:25.624+0000 7f3ff6a00700 -1 osd.6 126 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-03T18:50:25.887 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 03 18:50:25 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-6[56391]: 2024-05-03T18:50:25.885+0000 7f3ff2200700 -1 osd.6 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:37.490057+0000 front 2024-05-03T18:49:37.490102+0000 (oldest deadline 2024-05-03T18:50:02.789772+0000) 2024-05-03T18:50:25.887 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 03 18:50:25 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-6[56391]: 2024-05-03T18:50:25.885+0000 7f3ff2200700 -1 osd.6 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:48.692256+0000 front 2024-05-03T18:49:48.692122+0000 (oldest deadline 2024-05-03T18:50:12.792315+0000) 2024-05-03T18:50:25.887 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 03 18:50:25 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-6[56391]: 2024-05-03T18:50:25.885+0000 7f3ff2200700 -1 osd.6 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:56.294541+0000 front 2024-05-03T18:49:56.294785+0000 (oldest deadline 2024-05-03T18:50:22.194699+0000) 2024-05-03T18:50:26.046 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:25 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7[49675]: 2024-05-03T18:50:25.785+0000 7ff8ace00700 -1 osd.7 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:36.774719+0000 front 2024-05-03T18:49:36.774769+0000 (oldest deadline 2024-05-03T18:50:02.674865+0000) 2024-05-03T18:50:26.047 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:25 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7[49675]: 2024-05-03T18:50:25.785+0000 7ff8ace00700 -1 osd.7 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:47.376623+0000 front 2024-05-03T18:49:47.376700+0000 (oldest deadline 2024-05-03T18:50:10.876637+0000) 2024-05-03T18:50:26.047 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:25 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7[49675]: 2024-05-03T18:50:25.785+0000 7ff8ace00700 -1 osd.7 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:59.080150+0000 front 2024-05-03T18:49:59.080046+0000 (oldest deadline 2024-05-03T18:50:23.179826+0000) 2024-05-03T18:50:26.166 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:25 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:25.909+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:24.877883+0000 front 2024-05-03T18:49:24.877625+0000 (oldest deadline 2024-05-03T18:49:50.777881+0000) 2024-05-03T18:50:26.167 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:25 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:25.909+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:47.685409+0000 front 2024-05-03T18:49:47.685527+0000 (oldest deadline 2024-05-03T18:50:11.785436+0000) 2024-05-03T18:50:26.167 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:25 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:25.909+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:57.687614+0000 front 2024-05-03T18:49:57.687504+0000 (oldest deadline 2024-05-03T18:50:24.688928+0000) 2024-05-03T18:50:26.633 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:26 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:26.164+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:25.480913+0000 front 2024-05-03T18:49:25.480403+0000 (oldest deadline 2024-05-03T18:49:50.180152+0000) 2024-05-03T18:50:26.633 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:26 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:26.164+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:34.882184+0000 front 2024-05-03T18:49:34.882683+0000 (oldest deadline 2024-05-03T18:50:05.483250+0000) 2024-05-03T18:50:26.633 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:26 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:26.164+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:45.484981+0000 front 2024-05-03T18:49:45.484933+0000 (oldest deadline 2024-05-03T18:50:14.885581+0000) 2024-05-03T18:50:26.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:26 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:26.306+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:27.608012+0000 front 2024-05-03T18:49:27.608078+0000 (oldest deadline 2024-05-03T18:49:53.508290+0000) 2024-05-03T18:50:26.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:26 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:26.306+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:38.811037+0000 front 2024-05-03T18:49:38.811219+0000 (oldest deadline 2024-05-03T18:50:01.611951+0000) 2024-05-03T18:50:26.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:26 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:26.306+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:46.215926+0000 front 2024-05-03T18:49:46.216076+0000 (oldest deadline 2024-05-03T18:50:12.116128+0000) 2024-05-03T18:50:26.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:26 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:26.306+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:56.719409+0000 front 2024-05-03T18:49:56.719531+0000 (oldest deadline 2024-05-03T18:50:21.419652+0000) 2024-05-03T18:50:26.796 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:26 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:26.362+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:26.972061+0000 front 2024-05-03T18:49:26.972448+0000 (oldest deadline 2024-05-03T18:49:50.472344+0000) 2024-05-03T18:50:26.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:26 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:26.362+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:37.476081+0000 front 2024-05-03T18:49:37.476172+0000 (oldest deadline 2024-05-03T18:50:01.576192+0000) 2024-05-03T18:50:26.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:26 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:26.362+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:49.079592+0000 front 2024-05-03T18:49:49.080117+0000 (oldest deadline 2024-05-03T18:50:10.779802+0000) 2024-05-03T18:50:26.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:26 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:26.362+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:59.985863+0000 front 2024-05-03T18:49:59.986111+0000 (oldest deadline 2024-05-03T18:50:22.285990+0000) 2024-05-03T18:50:27.212 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:26 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:26.955+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:24.877883+0000 front 2024-05-03T18:49:24.877625+0000 (oldest deadline 2024-05-03T18:49:50.777881+0000) 2024-05-03T18:50:27.213 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:26 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:26.955+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:47.685409+0000 front 2024-05-03T18:49:47.685527+0000 (oldest deadline 2024-05-03T18:50:11.785436+0000) 2024-05-03T18:50:27.213 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:26 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:26.955+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:57.687614+0000 front 2024-05-03T18:49:57.687504+0000 (oldest deadline 2024-05-03T18:50:24.688928+0000) 2024-05-03T18:50:27.235 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 03 18:50:26 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-6[56391]: 2024-05-03T18:50:26.882+0000 7f3ff2200700 -1 osd.6 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:37.490057+0000 front 2024-05-03T18:49:37.490102+0000 (oldest deadline 2024-05-03T18:50:02.789772+0000) 2024-05-03T18:50:27.235 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 03 18:50:26 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-6[56391]: 2024-05-03T18:50:26.882+0000 7f3ff2200700 -1 osd.6 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:48.692256+0000 front 2024-05-03T18:49:48.692122+0000 (oldest deadline 2024-05-03T18:50:12.792315+0000) 2024-05-03T18:50:27.235 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 03 18:50:26 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-6[56391]: 2024-05-03T18:50:26.882+0000 7f3ff2200700 -1 osd.6 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:56.294541+0000 front 2024-05-03T18:49:56.294785+0000 (oldest deadline 2024-05-03T18:50:22.194699+0000) 2024-05-03T18:50:27.297 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:26 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7[49675]: 2024-05-03T18:50:26.796+0000 7ff8ace00700 -1 osd.7 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:36.774719+0000 front 2024-05-03T18:49:36.774769+0000 (oldest deadline 2024-05-03T18:50:02.674865+0000) 2024-05-03T18:50:27.297 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:26 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7[49675]: 2024-05-03T18:50:26.796+0000 7ff8ace00700 -1 osd.7 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:47.376623+0000 front 2024-05-03T18:49:47.376700+0000 (oldest deadline 2024-05-03T18:50:10.876637+0000) 2024-05-03T18:50:27.297 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:26 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7[49675]: 2024-05-03T18:50:26.796+0000 7ff8ace00700 -1 osd.7 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:59.080150+0000 front 2024-05-03T18:49:59.080046+0000 (oldest deadline 2024-05-03T18:50:23.179826+0000) 2024-05-03T18:50:27.633 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:27 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:27.210+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:25.480913+0000 front 2024-05-03T18:49:25.480403+0000 (oldest deadline 2024-05-03T18:49:50.180152+0000) 2024-05-03T18:50:27.633 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:27 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:27.210+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:34.882184+0000 front 2024-05-03T18:49:34.882683+0000 (oldest deadline 2024-05-03T18:50:05.483250+0000) 2024-05-03T18:50:27.633 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:27 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:27.210+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:45.484981+0000 front 2024-05-03T18:49:45.484933+0000 (oldest deadline 2024-05-03T18:50:14.885581+0000) 2024-05-03T18:50:27.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:27 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:27.294+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:27.608012+0000 front 2024-05-03T18:49:27.608078+0000 (oldest deadline 2024-05-03T18:49:53.508290+0000) 2024-05-03T18:50:27.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:27 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:27.294+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:38.811037+0000 front 2024-05-03T18:49:38.811219+0000 (oldest deadline 2024-05-03T18:50:01.611951+0000) 2024-05-03T18:50:27.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:27 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:27.294+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:46.215926+0000 front 2024-05-03T18:49:46.216076+0000 (oldest deadline 2024-05-03T18:50:12.116128+0000) 2024-05-03T18:50:27.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:27 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:27.294+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:56.719409+0000 front 2024-05-03T18:49:56.719531+0000 (oldest deadline 2024-05-03T18:50:21.419652+0000) 2024-05-03T18:50:27.796 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:27 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:27.338+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:26.972061+0000 front 2024-05-03T18:49:26.972448+0000 (oldest deadline 2024-05-03T18:49:50.472344+0000) 2024-05-03T18:50:27.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:27 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:27.338+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:37.476081+0000 front 2024-05-03T18:49:37.476172+0000 (oldest deadline 2024-05-03T18:50:01.576192+0000) 2024-05-03T18:50:27.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:27 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:27.338+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:49.079592+0000 front 2024-05-03T18:49:49.080117+0000 (oldest deadline 2024-05-03T18:50:10.779802+0000) 2024-05-03T18:50:27.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:27 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:27.338+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:59.985863+0000 front 2024-05-03T18:49:59.986111+0000 (oldest deadline 2024-05-03T18:50:22.285990+0000) 2024-05-03T18:50:28.235 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 03 18:50:27 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-6[56391]: 2024-05-03T18:50:27.848+0000 7f3ff2200700 -1 osd.6 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:37.490057+0000 front 2024-05-03T18:49:37.490102+0000 (oldest deadline 2024-05-03T18:50:02.789772+0000) 2024-05-03T18:50:28.235 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 03 18:50:27 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-6[56391]: 2024-05-03T18:50:27.848+0000 7f3ff2200700 -1 osd.6 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:48.692256+0000 front 2024-05-03T18:49:48.692122+0000 (oldest deadline 2024-05-03T18:50:12.792315+0000) 2024-05-03T18:50:28.235 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 03 18:50:27 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-6[56391]: 2024-05-03T18:50:27.848+0000 7f3ff2200700 -1 osd.6 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:56.294541+0000 front 2024-05-03T18:49:56.294785+0000 (oldest deadline 2024-05-03T18:50:22.194699+0000) 2024-05-03T18:50:28.294 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:27 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7[49675]: 2024-05-03T18:50:27.811+0000 7ff8ace00700 -1 osd.7 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:36.774719+0000 front 2024-05-03T18:49:36.774769+0000 (oldest deadline 2024-05-03T18:50:02.674865+0000) 2024-05-03T18:50:28.294 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:27 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7[49675]: 2024-05-03T18:50:27.811+0000 7ff8ace00700 -1 osd.7 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:47.376623+0000 front 2024-05-03T18:49:47.376700+0000 (oldest deadline 2024-05-03T18:50:10.876637+0000) 2024-05-03T18:50:28.294 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:27 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7[49675]: 2024-05-03T18:50:27.811+0000 7ff8ace00700 -1 osd.7 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:59.080150+0000 front 2024-05-03T18:49:59.080046+0000 (oldest deadline 2024-05-03T18:50:23.179826+0000) 2024-05-03T18:50:28.383 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:28 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:28.200+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:25.480913+0000 front 2024-05-03T18:49:25.480403+0000 (oldest deadline 2024-05-03T18:49:50.180152+0000) 2024-05-03T18:50:28.383 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:28 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:28.200+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:34.882184+0000 front 2024-05-03T18:49:34.882683+0000 (oldest deadline 2024-05-03T18:50:05.483250+0000) 2024-05-03T18:50:28.383 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:28 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:28.200+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:45.484981+0000 front 2024-05-03T18:49:45.484933+0000 (oldest deadline 2024-05-03T18:50:14.885581+0000) 2024-05-03T18:50:28.384 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:27 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:27.992+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:24.877883+0000 front 2024-05-03T18:49:24.877625+0000 (oldest deadline 2024-05-03T18:49:50.777881+0000) 2024-05-03T18:50:28.384 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:27 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:27.992+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:47.685409+0000 front 2024-05-03T18:49:47.685527+0000 (oldest deadline 2024-05-03T18:50:11.785436+0000) 2024-05-03T18:50:28.384 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:27 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:27.992+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:57.687614+0000 front 2024-05-03T18:49:57.687504+0000 (oldest deadline 2024-05-03T18:50:24.688928+0000) 2024-05-03T18:50:28.546 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:28 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:28.292+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:26.972061+0000 front 2024-05-03T18:49:26.972448+0000 (oldest deadline 2024-05-03T18:49:50.472344+0000) 2024-05-03T18:50:28.546 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:28 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:28.292+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:37.476081+0000 front 2024-05-03T18:49:37.476172+0000 (oldest deadline 2024-05-03T18:50:01.576192+0000) 2024-05-03T18:50:28.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:28 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:28.292+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:49.079592+0000 front 2024-05-03T18:49:49.080117+0000 (oldest deadline 2024-05-03T18:50:10.779802+0000) 2024-05-03T18:50:28.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:28 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:28.292+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:59.985863+0000 front 2024-05-03T18:49:59.986111+0000 (oldest deadline 2024-05-03T18:50:22.285990+0000) 2024-05-03T18:50:28.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:28 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:28.320+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:27.608012+0000 front 2024-05-03T18:49:27.608078+0000 (oldest deadline 2024-05-03T18:49:53.508290+0000) 2024-05-03T18:50:28.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:28 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:28.320+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:38.811037+0000 front 2024-05-03T18:49:38.811219+0000 (oldest deadline 2024-05-03T18:50:01.611951+0000) 2024-05-03T18:50:28.737 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:28 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:28.320+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:46.215926+0000 front 2024-05-03T18:49:46.216076+0000 (oldest deadline 2024-05-03T18:50:12.116128+0000) 2024-05-03T18:50:28.737 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:28 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:28.320+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:56.719409+0000 front 2024-05-03T18:49:56.719531+0000 (oldest deadline 2024-05-03T18:50:21.419652+0000) 2024-05-03T18:50:29.235 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 03 18:50:28 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-6[56391]: 2024-05-03T18:50:28.845+0000 7f3ff2200700 -1 osd.6 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:37.490057+0000 front 2024-05-03T18:49:37.490102+0000 (oldest deadline 2024-05-03T18:50:02.789772+0000) 2024-05-03T18:50:29.235 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 03 18:50:28 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-6[56391]: 2024-05-03T18:50:28.845+0000 7f3ff2200700 -1 osd.6 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:48.692256+0000 front 2024-05-03T18:49:48.692122+0000 (oldest deadline 2024-05-03T18:50:12.792315+0000) 2024-05-03T18:50:29.235 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 03 18:50:28 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-6[56391]: 2024-05-03T18:50:28.845+0000 7f3ff2200700 -1 osd.6 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:56.294541+0000 front 2024-05-03T18:49:56.294785+0000 (oldest deadline 2024-05-03T18:50:22.194699+0000) 2024-05-03T18:50:29.296 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:28 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7[49675]: 2024-05-03T18:50:28.842+0000 7ff8ace00700 -1 osd.7 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:36.774719+0000 front 2024-05-03T18:49:36.774769+0000 (oldest deadline 2024-05-03T18:50:02.674865+0000) 2024-05-03T18:50:29.297 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:28 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7[49675]: 2024-05-03T18:50:28.842+0000 7ff8ace00700 -1 osd.7 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:47.376623+0000 front 2024-05-03T18:49:47.376700+0000 (oldest deadline 2024-05-03T18:50:10.876637+0000) 2024-05-03T18:50:29.297 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:28 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7[49675]: 2024-05-03T18:50:28.842+0000 7ff8ace00700 -1 osd.7 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:59.080150+0000 front 2024-05-03T18:49:59.080046+0000 (oldest deadline 2024-05-03T18:50:23.179826+0000) 2024-05-03T18:50:29.383 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:29 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:29.022+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:24.877883+0000 front 2024-05-03T18:49:24.877625+0000 (oldest deadline 2024-05-03T18:49:50.777881+0000) 2024-05-03T18:50:29.383 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:29 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:29.022+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:47.685409+0000 front 2024-05-03T18:49:47.685527+0000 (oldest deadline 2024-05-03T18:50:11.785436+0000) 2024-05-03T18:50:29.384 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:29 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:29.022+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:57.687614+0000 front 2024-05-03T18:49:57.687504+0000 (oldest deadline 2024-05-03T18:50:24.688928+0000) 2024-05-03T18:50:29.384 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:29 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:29.242+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:25.480913+0000 front 2024-05-03T18:49:25.480403+0000 (oldest deadline 2024-05-03T18:49:50.180152+0000) 2024-05-03T18:50:29.384 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:29 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:29.242+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:34.882184+0000 front 2024-05-03T18:49:34.882683+0000 (oldest deadline 2024-05-03T18:50:05.483250+0000) 2024-05-03T18:50:29.384 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:29 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:29.242+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:45.484981+0000 front 2024-05-03T18:49:45.484933+0000 (oldest deadline 2024-05-03T18:50:14.885581+0000) 2024-05-03T18:50:29.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:29 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:29.310+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:27.608012+0000 front 2024-05-03T18:49:27.608078+0000 (oldest deadline 2024-05-03T18:49:53.508290+0000) 2024-05-03T18:50:29.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:29 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:29.310+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:38.811037+0000 front 2024-05-03T18:49:38.811219+0000 (oldest deadline 2024-05-03T18:50:01.611951+0000) 2024-05-03T18:50:29.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:29 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:29.310+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:46.215926+0000 front 2024-05-03T18:49:46.216076+0000 (oldest deadline 2024-05-03T18:50:12.116128+0000) 2024-05-03T18:50:29.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:29 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:29.310+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:56.719409+0000 front 2024-05-03T18:49:56.719531+0000 (oldest deadline 2024-05-03T18:50:21.419652+0000) 2024-05-03T18:50:29.796 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:29 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:29.305+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:26.972061+0000 front 2024-05-03T18:49:26.972448+0000 (oldest deadline 2024-05-03T18:49:50.472344+0000) 2024-05-03T18:50:29.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:29 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:29.305+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:37.476081+0000 front 2024-05-03T18:49:37.476172+0000 (oldest deadline 2024-05-03T18:50:01.576192+0000) 2024-05-03T18:50:29.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:29 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:29.305+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:49.079592+0000 front 2024-05-03T18:49:49.080117+0000 (oldest deadline 2024-05-03T18:50:10.779802+0000) 2024-05-03T18:50:29.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:29 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:29.305+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:59.985863+0000 front 2024-05-03T18:49:59.986111+0000 (oldest deadline 2024-05-03T18:50:22.285990+0000) 2024-05-03T18:50:30.235 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 03 18:50:29 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-6[56391]: 2024-05-03T18:50:29.828+0000 7f3ff2200700 -1 osd.6 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:37.490057+0000 front 2024-05-03T18:49:37.490102+0000 (oldest deadline 2024-05-03T18:50:02.789772+0000) 2024-05-03T18:50:30.235 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 03 18:50:29 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-6[56391]: 2024-05-03T18:50:29.828+0000 7f3ff2200700 -1 osd.6 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:48.692256+0000 front 2024-05-03T18:49:48.692122+0000 (oldest deadline 2024-05-03T18:50:12.792315+0000) 2024-05-03T18:50:30.235 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 03 18:50:29 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-6[56391]: 2024-05-03T18:50:29.828+0000 7f3ff2200700 -1 osd.6 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:56.294541+0000 front 2024-05-03T18:49:56.294785+0000 (oldest deadline 2024-05-03T18:50:22.194699+0000) 2024-05-03T18:50:30.296 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:29 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7[49675]: 2024-05-03T18:50:29.811+0000 7ff8ace00700 -1 osd.7 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:36.774719+0000 front 2024-05-03T18:49:36.774769+0000 (oldest deadline 2024-05-03T18:50:02.674865+0000) 2024-05-03T18:50:30.297 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:29 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7[49675]: 2024-05-03T18:50:29.811+0000 7ff8ace00700 -1 osd.7 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:47.376623+0000 front 2024-05-03T18:49:47.376700+0000 (oldest deadline 2024-05-03T18:50:10.876637+0000) 2024-05-03T18:50:30.297 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:29 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7[49675]: 2024-05-03T18:50:29.811+0000 7ff8ace00700 -1 osd.7 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:59.080150+0000 front 2024-05-03T18:49:59.080046+0000 (oldest deadline 2024-05-03T18:50:23.179826+0000) 2024-05-03T18:50:30.383 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:30 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:30.020+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:24.877883+0000 front 2024-05-03T18:49:24.877625+0000 (oldest deadline 2024-05-03T18:49:50.777881+0000) 2024-05-03T18:50:30.383 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:30 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:30.020+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:47.685409+0000 front 2024-05-03T18:49:47.685527+0000 (oldest deadline 2024-05-03T18:50:11.785436+0000) 2024-05-03T18:50:30.384 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:30 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:30.020+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:57.687614+0000 front 2024-05-03T18:49:57.687504+0000 (oldest deadline 2024-05-03T18:50:24.688928+0000) 2024-05-03T18:50:30.384 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:30 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:30.223+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:25.480913+0000 front 2024-05-03T18:49:25.480403+0000 (oldest deadline 2024-05-03T18:49:50.180152+0000) 2024-05-03T18:50:30.384 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:30 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:30.223+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:34.882184+0000 front 2024-05-03T18:49:34.882683+0000 (oldest deadline 2024-05-03T18:50:05.483250+0000) 2024-05-03T18:50:30.384 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:30 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:30.223+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:45.484981+0000 front 2024-05-03T18:49:45.484933+0000 (oldest deadline 2024-05-03T18:50:14.885581+0000) 2024-05-03T18:50:30.643 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:30 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:30.339+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:27.608012+0000 front 2024-05-03T18:49:27.608078+0000 (oldest deadline 2024-05-03T18:49:53.508290+0000) 2024-05-03T18:50:30.643 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:30 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:30.339+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:38.811037+0000 front 2024-05-03T18:49:38.811219+0000 (oldest deadline 2024-05-03T18:50:01.611951+0000) 2024-05-03T18:50:30.643 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:30 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:30.339+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:46.215926+0000 front 2024-05-03T18:49:46.216076+0000 (oldest deadline 2024-05-03T18:50:12.116128+0000) 2024-05-03T18:50:30.643 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:30 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:30.339+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:56.719409+0000 front 2024-05-03T18:49:56.719531+0000 (oldest deadline 2024-05-03T18:50:21.419652+0000) 2024-05-03T18:50:30.796 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:30 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:30.301+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:26.972061+0000 front 2024-05-03T18:49:26.972448+0000 (oldest deadline 2024-05-03T18:49:50.472344+0000) 2024-05-03T18:50:30.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:30 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:30.301+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:37.476081+0000 front 2024-05-03T18:49:37.476172+0000 (oldest deadline 2024-05-03T18:50:01.576192+0000) 2024-05-03T18:50:30.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:30 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:30.301+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:49.079592+0000 front 2024-05-03T18:49:49.080117+0000 (oldest deadline 2024-05-03T18:50:10.779802+0000) 2024-05-03T18:50:30.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:30 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:30.301+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:59.985863+0000 front 2024-05-03T18:49:59.986111+0000 (oldest deadline 2024-05-03T18:50:22.285990+0000) 2024-05-03T18:50:31.297 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:30 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7[49675]: 2024-05-03T18:50:30.836+0000 7ff8ace00700 -1 osd.7 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:36.774719+0000 front 2024-05-03T18:49:36.774769+0000 (oldest deadline 2024-05-03T18:50:02.674865+0000) 2024-05-03T18:50:31.297 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:30 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7[49675]: 2024-05-03T18:50:30.836+0000 7ff8ace00700 -1 osd.7 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:47.376623+0000 front 2024-05-03T18:49:47.376700+0000 (oldest deadline 2024-05-03T18:50:10.876637+0000) 2024-05-03T18:50:31.297 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:30 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7[49675]: 2024-05-03T18:50:30.836+0000 7ff8ace00700 -1 osd.7 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:59.080150+0000 front 2024-05-03T18:49:59.080046+0000 (oldest deadline 2024-05-03T18:50:23.179826+0000) 2024-05-03T18:50:31.383 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:30 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:30.987+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:24.877883+0000 front 2024-05-03T18:49:24.877625+0000 (oldest deadline 2024-05-03T18:49:50.777881+0000) 2024-05-03T18:50:31.383 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:30 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:30.987+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:47.685409+0000 front 2024-05-03T18:49:47.685527+0000 (oldest deadline 2024-05-03T18:50:11.785436+0000) 2024-05-03T18:50:31.383 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:30 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:30.988+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:57.687614+0000 front 2024-05-03T18:49:57.687504+0000 (oldest deadline 2024-05-03T18:50:24.688928+0000) 2024-05-03T18:50:31.384 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:31 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:31.237+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:25.480913+0000 front 2024-05-03T18:49:25.480403+0000 (oldest deadline 2024-05-03T18:49:50.180152+0000) 2024-05-03T18:50:31.384 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:31 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:31.237+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:34.882184+0000 front 2024-05-03T18:49:34.882683+0000 (oldest deadline 2024-05-03T18:50:05.483250+0000) 2024-05-03T18:50:31.384 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:31 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:31.237+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:45.484981+0000 front 2024-05-03T18:49:45.484933+0000 (oldest deadline 2024-05-03T18:50:14.885581+0000) 2024-05-03T18:50:31.734 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:31 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:31.323+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:27.608012+0000 front 2024-05-03T18:49:27.608078+0000 (oldest deadline 2024-05-03T18:49:53.508290+0000) 2024-05-03T18:50:31.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:31 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:31.323+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:38.811037+0000 front 2024-05-03T18:49:38.811219+0000 (oldest deadline 2024-05-03T18:50:01.611951+0000) 2024-05-03T18:50:31.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:31 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:31.323+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:46.215926+0000 front 2024-05-03T18:49:46.216076+0000 (oldest deadline 2024-05-03T18:50:12.116128+0000) 2024-05-03T18:50:31.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:31 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:31.323+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:56.719409+0000 front 2024-05-03T18:49:56.719531+0000 (oldest deadline 2024-05-03T18:50:21.419652+0000) 2024-05-03T18:50:31.796 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:31 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:31.349+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:26.972061+0000 front 2024-05-03T18:49:26.972448+0000 (oldest deadline 2024-05-03T18:49:50.472344+0000) 2024-05-03T18:50:31.796 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:31 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:31.349+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:37.476081+0000 front 2024-05-03T18:49:37.476172+0000 (oldest deadline 2024-05-03T18:50:01.576192+0000) 2024-05-03T18:50:31.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:31 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:31.349+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:49.079592+0000 front 2024-05-03T18:49:49.080117+0000 (oldest deadline 2024-05-03T18:50:10.779802+0000) 2024-05-03T18:50:31.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:31 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:31.349+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:59.985863+0000 front 2024-05-03T18:49:59.986111+0000 (oldest deadline 2024-05-03T18:50:22.285990+0000) 2024-05-03T18:50:32.115 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 03 18:50:31 smithi033 podman[127383]: 2024-05-03 18:50:31.773982219 +0000 UTC m=+6.307386160 container died 01db5bde5c9b76caf244e755b22354617a26cf4309f90a1dd91c43f4f34e62ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-6, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-03T18:50:32.296 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:31 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7[49675]: 2024-05-03T18:50:31.855+0000 7ff8ace00700 -1 osd.7 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:36.774719+0000 front 2024-05-03T18:49:36.774769+0000 (oldest deadline 2024-05-03T18:50:02.674865+0000) 2024-05-03T18:50:32.297 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:31 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7[49675]: 2024-05-03T18:50:31.855+0000 7ff8ace00700 -1 osd.7 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:47.376623+0000 front 2024-05-03T18:49:47.376700+0000 (oldest deadline 2024-05-03T18:50:10.876637+0000) 2024-05-03T18:50:32.297 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:31 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7[49675]: 2024-05-03T18:50:31.855+0000 7ff8ace00700 -1 osd.7 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:59.080150+0000 front 2024-05-03T18:49:59.080046+0000 (oldest deadline 2024-05-03T18:50:23.179826+0000) 2024-05-03T18:50:32.383 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:32 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:32.036+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:24.877883+0000 front 2024-05-03T18:49:24.877625+0000 (oldest deadline 2024-05-03T18:49:50.777881+0000) 2024-05-03T18:50:32.383 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:32 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:32.036+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:47.685409+0000 front 2024-05-03T18:49:47.685527+0000 (oldest deadline 2024-05-03T18:50:11.785436+0000) 2024-05-03T18:50:32.384 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:32 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:32.036+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:57.687614+0000 front 2024-05-03T18:49:57.687504+0000 (oldest deadline 2024-05-03T18:50:24.688928+0000) 2024-05-03T18:50:32.384 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:32 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:32.197+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:25.480913+0000 front 2024-05-03T18:49:25.480403+0000 (oldest deadline 2024-05-03T18:49:50.180152+0000) 2024-05-03T18:50:32.384 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:32 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:32.197+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:34.882184+0000 front 2024-05-03T18:49:34.882683+0000 (oldest deadline 2024-05-03T18:50:05.483250+0000) 2024-05-03T18:50:32.384 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:32 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:32.197+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:45.484981+0000 front 2024-05-03T18:49:45.484933+0000 (oldest deadline 2024-05-03T18:50:14.885581+0000) 2024-05-03T18:50:32.384 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:32 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:32.197+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:08.290430+0000 front 2024-05-03T18:50:08.290390+0000 (oldest deadline 2024-05-03T18:50:31.790398+0000) 2024-05-03T18:50:32.485 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 03 18:50:32 smithi033 podman[127383]: 2024-05-03 18:50:32.33354387 +0000 UTC m=+6.866947804 container cleanup 01db5bde5c9b76caf244e755b22354617a26cf4309f90a1dd91c43f4f34e62ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-6, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-03T18:50:32.486 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 03 18:50:32 smithi033 bash[127383]: ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-6 2024-05-03T18:50:32.486 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:32 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:32.281+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:27.608012+0000 front 2024-05-03T18:49:27.608078+0000 (oldest deadline 2024-05-03T18:49:53.508290+0000) 2024-05-03T18:50:32.486 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:32 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:32.281+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:38.811037+0000 front 2024-05-03T18:49:38.811219+0000 (oldest deadline 2024-05-03T18:50:01.611951+0000) 2024-05-03T18:50:32.486 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:32 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:32.281+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:46.215926+0000 front 2024-05-03T18:49:46.216076+0000 (oldest deadline 2024-05-03T18:50:12.116128+0000) 2024-05-03T18:50:32.486 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:32 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:32.281+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:56.719409+0000 front 2024-05-03T18:49:56.719531+0000 (oldest deadline 2024-05-03T18:50:21.419652+0000) 2024-05-03T18:50:32.796 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:32 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:32.396+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:26.972061+0000 front 2024-05-03T18:49:26.972448+0000 (oldest deadline 2024-05-03T18:49:50.472344+0000) 2024-05-03T18:50:32.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:32 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:32.396+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:37.476081+0000 front 2024-05-03T18:49:37.476172+0000 (oldest deadline 2024-05-03T18:50:01.576192+0000) 2024-05-03T18:50:32.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:32 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:32.396+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:49.079592+0000 front 2024-05-03T18:49:49.080117+0000 (oldest deadline 2024-05-03T18:50:10.779802+0000) 2024-05-03T18:50:32.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:32 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:32.396+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:59.985863+0000 front 2024-05-03T18:49:59.986111+0000 (oldest deadline 2024-05-03T18:50:22.285990+0000) 2024-05-03T18:50:32.985 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 03 18:50:32 smithi033 podman[127413]: 2024-05-03 18:50:32.618330952 +0000 UTC m=+0.807028222 container remove 01db5bde5c9b76caf244e755b22354617a26cf4309f90a1dd91c43f4f34e62ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-6, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-05-03T18:50:33.297 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:32 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7[49675]: 2024-05-03T18:50:32.848+0000 7ff8ace00700 -1 osd.7 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:36.774719+0000 front 2024-05-03T18:49:36.774769+0000 (oldest deadline 2024-05-03T18:50:02.674865+0000) 2024-05-03T18:50:33.297 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:32 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7[49675]: 2024-05-03T18:50:32.848+0000 7ff8ace00700 -1 osd.7 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:47.376623+0000 front 2024-05-03T18:49:47.376700+0000 (oldest deadline 2024-05-03T18:50:10.876637+0000) 2024-05-03T18:50:33.297 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:32 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7[49675]: 2024-05-03T18:50:32.848+0000 7ff8ace00700 -1 osd.7 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:59.080150+0000 front 2024-05-03T18:49:59.080046+0000 (oldest deadline 2024-05-03T18:50:23.179826+0000) 2024-05-03T18:50:33.297 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:32 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7[49675]: 2024-05-03T18:50:32.848+0000 7ff8ace00700 -1 osd.7 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:07.282208+0000 front 2024-05-03T18:50:07.282301+0000 (oldest deadline 2024-05-03T18:50:31.982398+0000) 2024-05-03T18:50:33.383 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:33 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:33.047+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:24.877883+0000 front 2024-05-03T18:49:24.877625+0000 (oldest deadline 2024-05-03T18:49:50.777881+0000) 2024-05-03T18:50:33.383 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:33 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:33.047+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:47.685409+0000 front 2024-05-03T18:49:47.685527+0000 (oldest deadline 2024-05-03T18:50:11.785436+0000) 2024-05-03T18:50:33.383 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:33 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:33.047+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:57.687614+0000 front 2024-05-03T18:49:57.687504+0000 (oldest deadline 2024-05-03T18:50:24.688928+0000) 2024-05-03T18:50:33.384 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:33 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:33.194+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:25.480913+0000 front 2024-05-03T18:49:25.480403+0000 (oldest deadline 2024-05-03T18:49:50.180152+0000) 2024-05-03T18:50:33.384 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:33 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:33.194+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:34.882184+0000 front 2024-05-03T18:49:34.882683+0000 (oldest deadline 2024-05-03T18:50:05.483250+0000) 2024-05-03T18:50:33.384 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:33 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:33.194+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:45.484981+0000 front 2024-05-03T18:49:45.484933+0000 (oldest deadline 2024-05-03T18:50:14.885581+0000) 2024-05-03T18:50:33.384 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:33 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:33.194+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:08.290430+0000 front 2024-05-03T18:50:08.290390+0000 (oldest deadline 2024-05-03T18:50:31.790398+0000) 2024-05-03T18:50:33.671 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:33 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:33.426+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:26.972061+0000 front 2024-05-03T18:49:26.972448+0000 (oldest deadline 2024-05-03T18:49:50.472344+0000) 2024-05-03T18:50:33.671 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:33 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:33.426+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:37.476081+0000 front 2024-05-03T18:49:37.476172+0000 (oldest deadline 2024-05-03T18:50:01.576192+0000) 2024-05-03T18:50:33.671 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:33 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:33.426+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:49.079592+0000 front 2024-05-03T18:49:49.080117+0000 (oldest deadline 2024-05-03T18:50:10.779802+0000) 2024-05-03T18:50:33.671 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:33 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:33.426+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:59.985863+0000 front 2024-05-03T18:49:59.986111+0000 (oldest deadline 2024-05-03T18:50:22.285990+0000) 2024-05-03T18:50:33.690 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:33 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:33.273+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:27.608012+0000 front 2024-05-03T18:49:27.608078+0000 (oldest deadline 2024-05-03T18:49:53.508290+0000) 2024-05-03T18:50:33.690 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:33 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:33.273+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:38.811037+0000 front 2024-05-03T18:49:38.811219+0000 (oldest deadline 2024-05-03T18:50:01.611951+0000) 2024-05-03T18:50:33.690 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:33 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:33.273+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:46.215926+0000 front 2024-05-03T18:49:46.216076+0000 (oldest deadline 2024-05-03T18:50:12.116128+0000) 2024-05-03T18:50:33.690 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:33 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:33.273+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:56.719409+0000 front 2024-05-03T18:49:56.719531+0000 (oldest deadline 2024-05-03T18:50:21.419652+0000) 2024-05-03T18:50:33.690 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 03 18:50:33 smithi033 podman[127504]: 2024-05-03 18:50:33.316449641 +0000 UTC m=+0.036996940 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T18:50:33.690 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 03 18:50:33 smithi033 podman[127504]: 2024-05-03 18:50:33.519152511 +0000 UTC m=+0.239699835 container create 396a248168a4469142054c0c7b154d25573eaded6d9c8a7f19eb3f74845fbd50 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-6-deactivate, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-03T18:50:33.964 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 03 18:50:33 smithi033 podman[127504]: 2024-05-03 18:50:33.856226703 +0000 UTC m=+0.576774034 container init 396a248168a4469142054c0c7b154d25573eaded6d9c8a7f19eb3f74845fbd50 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-6-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.license=GPLv2) 2024-05-03T18:50:33.965 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 03 18:50:33 smithi033 podman[127504]: 2024-05-03 18:50:33.868977825 +0000 UTC m=+0.589525179 container start 396a248168a4469142054c0c7b154d25573eaded6d9c8a7f19eb3f74845fbd50 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-03T18:50:33.965 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 03 18:50:33 smithi033 podman[127504]: 2024-05-03 18:50:33.963236995 +0000 UTC m=+0.683784348 container attach 396a248168a4469142054c0c7b154d25573eaded6d9c8a7f19eb3f74845fbd50 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-05-03T18:50:33.992 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:33 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7[49675]: 2024-05-03T18:50:33.857+0000 7ff8ace00700 -1 osd.7 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:36.774719+0000 front 2024-05-03T18:49:36.774769+0000 (oldest deadline 2024-05-03T18:50:02.674865+0000) 2024-05-03T18:50:33.992 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:33 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7[49675]: 2024-05-03T18:50:33.857+0000 7ff8ace00700 -1 osd.7 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:47.376623+0000 front 2024-05-03T18:49:47.376700+0000 (oldest deadline 2024-05-03T18:50:10.876637+0000) 2024-05-03T18:50:33.992 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:33 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7[49675]: 2024-05-03T18:50:33.857+0000 7ff8ace00700 -1 osd.7 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:59.080150+0000 front 2024-05-03T18:49:59.080046+0000 (oldest deadline 2024-05-03T18:50:23.179826+0000) 2024-05-03T18:50:33.992 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:33 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7[49675]: 2024-05-03T18:50:33.857+0000 7ff8ace00700 -1 osd.7 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:07.282208+0000 front 2024-05-03T18:50:07.282301+0000 (oldest deadline 2024-05-03T18:50:31.982398+0000) 2024-05-03T18:50:34.317 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:34 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:34.047+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:24.877883+0000 front 2024-05-03T18:49:24.877625+0000 (oldest deadline 2024-05-03T18:49:50.777881+0000) 2024-05-03T18:50:34.317 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:34 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:34.047+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:47.685409+0000 front 2024-05-03T18:49:47.685527+0000 (oldest deadline 2024-05-03T18:50:11.785436+0000) 2024-05-03T18:50:34.318 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:34 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:34.047+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:57.687614+0000 front 2024-05-03T18:49:57.687504+0000 (oldest deadline 2024-05-03T18:50:24.688928+0000) 2024-05-03T18:50:34.318 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:34 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:34.187+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:25.480913+0000 front 2024-05-03T18:49:25.480403+0000 (oldest deadline 2024-05-03T18:49:50.180152+0000) 2024-05-03T18:50:34.318 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:34 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:34.187+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:34.882184+0000 front 2024-05-03T18:49:34.882683+0000 (oldest deadline 2024-05-03T18:50:05.483250+0000) 2024-05-03T18:50:34.318 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:34 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:34.187+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:45.484981+0000 front 2024-05-03T18:49:45.484933+0000 (oldest deadline 2024-05-03T18:50:14.885581+0000) 2024-05-03T18:50:34.318 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:34 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:34.187+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:08.290430+0000 front 2024-05-03T18:50:08.290390+0000 (oldest deadline 2024-05-03T18:50:31.790398+0000) 2024-05-03T18:50:34.735 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 03 18:50:34 smithi033 podman[127504]: 2024-05-03 18:50:34.343447296 +0000 UTC m=+1.063994626 container died 396a248168a4469142054c0c7b154d25573eaded6d9c8a7f19eb3f74845fbd50 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.build-date=20240430, maintainer=Guillaume Abrioux , ceph=True) 2024-05-03T18:50:34.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:34 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:34.265+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:27.608012+0000 front 2024-05-03T18:49:27.608078+0000 (oldest deadline 2024-05-03T18:49:53.508290+0000) 2024-05-03T18:50:34.736 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:34 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:34.265+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:38.811037+0000 front 2024-05-03T18:49:38.811219+0000 (oldest deadline 2024-05-03T18:50:01.611951+0000) 2024-05-03T18:50:34.736 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:34 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:34.265+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:46.215926+0000 front 2024-05-03T18:49:46.216076+0000 (oldest deadline 2024-05-03T18:50:12.116128+0000) 2024-05-03T18:50:34.736 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:34 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:34.265+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:56.719409+0000 front 2024-05-03T18:49:56.719531+0000 (oldest deadline 2024-05-03T18:50:21.419652+0000) 2024-05-03T18:50:34.736 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:34 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:34.265+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:08.223552+0000 front 2024-05-03T18:50:08.224016+0000 (oldest deadline 2024-05-03T18:50:33.925215+0000) 2024-05-03T18:50:34.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:34 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:34.437+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:26.972061+0000 front 2024-05-03T18:49:26.972448+0000 (oldest deadline 2024-05-03T18:49:50.472344+0000) 2024-05-03T18:50:34.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:34 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:34.437+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:37.476081+0000 front 2024-05-03T18:49:37.476172+0000 (oldest deadline 2024-05-03T18:50:01.576192+0000) 2024-05-03T18:50:34.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:34 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:34.437+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:49.079592+0000 front 2024-05-03T18:49:49.080117+0000 (oldest deadline 2024-05-03T18:50:10.779802+0000) 2024-05-03T18:50:34.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:34 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:34.437+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:59.985863+0000 front 2024-05-03T18:49:59.986111+0000 (oldest deadline 2024-05-03T18:50:22.285990+0000) 2024-05-03T18:50:35.297 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:34 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7[49675]: 2024-05-03T18:50:34.890+0000 7ff8ace00700 -1 osd.7 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:36.774719+0000 front 2024-05-03T18:49:36.774769+0000 (oldest deadline 2024-05-03T18:50:02.674865+0000) 2024-05-03T18:50:35.297 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:34 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7[49675]: 2024-05-03T18:50:34.890+0000 7ff8ace00700 -1 osd.7 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:47.376623+0000 front 2024-05-03T18:49:47.376700+0000 (oldest deadline 2024-05-03T18:50:10.876637+0000) 2024-05-03T18:50:35.297 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:34 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7[49675]: 2024-05-03T18:50:34.890+0000 7ff8ace00700 -1 osd.7 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:59.080150+0000 front 2024-05-03T18:49:59.080046+0000 (oldest deadline 2024-05-03T18:50:23.179826+0000) 2024-05-03T18:50:35.297 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:34 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7[49675]: 2024-05-03T18:50:34.890+0000 7ff8ace00700 -1 osd.7 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:07.282208+0000 front 2024-05-03T18:50:07.282301+0000 (oldest deadline 2024-05-03T18:50:31.982398+0000) 2024-05-03T18:50:35.383 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:35 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:35.157+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:25.480913+0000 front 2024-05-03T18:49:25.480403+0000 (oldest deadline 2024-05-03T18:49:50.180152+0000) 2024-05-03T18:50:35.383 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:35 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:35.157+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:34.882184+0000 front 2024-05-03T18:49:34.882683+0000 (oldest deadline 2024-05-03T18:50:05.483250+0000) 2024-05-03T18:50:35.383 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:35 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:35.157+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:45.484981+0000 front 2024-05-03T18:49:45.484933+0000 (oldest deadline 2024-05-03T18:50:14.885581+0000) 2024-05-03T18:50:35.384 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:35 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:35.157+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:08.290430+0000 front 2024-05-03T18:50:08.290390+0000 (oldest deadline 2024-05-03T18:50:31.790398+0000) 2024-05-03T18:50:35.384 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:35 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:35.006+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:24.877883+0000 front 2024-05-03T18:49:24.877625+0000 (oldest deadline 2024-05-03T18:49:50.777881+0000) 2024-05-03T18:50:35.384 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:35 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:35.006+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:47.685409+0000 front 2024-05-03T18:49:47.685527+0000 (oldest deadline 2024-05-03T18:50:11.785436+0000) 2024-05-03T18:50:35.384 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:35 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:35.006+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:57.687614+0000 front 2024-05-03T18:49:57.687504+0000 (oldest deadline 2024-05-03T18:50:24.688928+0000) 2024-05-03T18:50:35.384 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:35 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:35.006+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:08.191069+0000 front 2024-05-03T18:50:08.190956+0000 (oldest deadline 2024-05-03T18:50:34.091182+0000) 2024-05-03T18:50:35.451 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5703bba4-0978-11ef-bc97-c7b262605968@osd.6.service' 2024-05-03T18:50:35.567 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:35 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:35.304+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:27.608012+0000 front 2024-05-03T18:49:27.608078+0000 (oldest deadline 2024-05-03T18:49:53.508290+0000) 2024-05-03T18:50:35.567 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:35 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:35.304+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:38.811037+0000 front 2024-05-03T18:49:38.811219+0000 (oldest deadline 2024-05-03T18:50:01.611951+0000) 2024-05-03T18:50:35.567 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:35 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:35.304+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:46.215926+0000 front 2024-05-03T18:49:46.216076+0000 (oldest deadline 2024-05-03T18:50:12.116128+0000) 2024-05-03T18:50:35.567 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:35 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:35.304+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:56.719409+0000 front 2024-05-03T18:49:56.719531+0000 (oldest deadline 2024-05-03T18:50:21.419652+0000) 2024-05-03T18:50:35.567 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:35 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:35.304+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:08.223552+0000 front 2024-05-03T18:50:08.224016+0000 (oldest deadline 2024-05-03T18:50:33.925215+0000) 2024-05-03T18:50:35.571 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T18:50:35.572 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-05-03T18:50:35.572 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-05-03T18:50:35.572 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-5703bba4-0978-11ef-bc97-c7b262605968@osd.7 2024-05-03T18:50:35.651 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:35 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:35.390+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:26.972061+0000 front 2024-05-03T18:49:26.972448+0000 (oldest deadline 2024-05-03T18:49:50.472344+0000) 2024-05-03T18:50:35.651 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:35 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:35.390+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:37.476081+0000 front 2024-05-03T18:49:37.476172+0000 (oldest deadline 2024-05-03T18:50:01.576192+0000) 2024-05-03T18:50:35.651 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:35 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:35.390+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:49.079592+0000 front 2024-05-03T18:49:49.080117+0000 (oldest deadline 2024-05-03T18:50:10.779802+0000) 2024-05-03T18:50:35.651 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:35 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:35.390+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:59.985863+0000 front 2024-05-03T18:49:59.986111+0000 (oldest deadline 2024-05-03T18:50:22.285990+0000) 2024-05-03T18:50:36.046 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:35 smithi099 systemd[1]: Stopping Ceph osd.7 for 5703bba4-0978-11ef-bc97-c7b262605968... 2024-05-03T18:50:36.047 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:35 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7[49675]: 2024-05-03T18:50:35.873+0000 7ff8ace00700 -1 osd.7 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:36.774719+0000 front 2024-05-03T18:49:36.774769+0000 (oldest deadline 2024-05-03T18:50:02.674865+0000) 2024-05-03T18:50:36.047 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:35 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7[49675]: 2024-05-03T18:50:35.873+0000 7ff8ace00700 -1 osd.7 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:47.376623+0000 front 2024-05-03T18:49:47.376700+0000 (oldest deadline 2024-05-03T18:50:10.876637+0000) 2024-05-03T18:50:36.047 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:35 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7[49675]: 2024-05-03T18:50:35.873+0000 7ff8ace00700 -1 osd.7 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:59.080150+0000 front 2024-05-03T18:49:59.080046+0000 (oldest deadline 2024-05-03T18:50:23.179826+0000) 2024-05-03T18:50:36.047 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:35 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7[49675]: 2024-05-03T18:50:35.873+0000 7ff8ace00700 -1 osd.7 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:07.282208+0000 front 2024-05-03T18:50:07.282301+0000 (oldest deadline 2024-05-03T18:50:31.982398+0000) 2024-05-03T18:50:36.383 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:36 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:36.030+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:24.877883+0000 front 2024-05-03T18:49:24.877625+0000 (oldest deadline 2024-05-03T18:49:50.777881+0000) 2024-05-03T18:50:36.383 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:36 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:36.030+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:47.685409+0000 front 2024-05-03T18:49:47.685527+0000 (oldest deadline 2024-05-03T18:50:11.785436+0000) 2024-05-03T18:50:36.384 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:36 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:36.030+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:57.687614+0000 front 2024-05-03T18:49:57.687504+0000 (oldest deadline 2024-05-03T18:50:24.688928+0000) 2024-05-03T18:50:36.384 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:36 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:36.030+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:08.191069+0000 front 2024-05-03T18:50:08.190956+0000 (oldest deadline 2024-05-03T18:50:34.091182+0000) 2024-05-03T18:50:36.384 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:36 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:36.180+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:25.480913+0000 front 2024-05-03T18:49:25.480403+0000 (oldest deadline 2024-05-03T18:49:50.180152+0000) 2024-05-03T18:50:36.384 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:36 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:36.180+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:34.882184+0000 front 2024-05-03T18:49:34.882683+0000 (oldest deadline 2024-05-03T18:50:05.483250+0000) 2024-05-03T18:50:36.384 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:36 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:36.180+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:45.484981+0000 front 2024-05-03T18:49:45.484933+0000 (oldest deadline 2024-05-03T18:50:14.885581+0000) 2024-05-03T18:50:36.384 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:36 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:36.180+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:08.290430+0000 front 2024-05-03T18:50:08.290390+0000 (oldest deadline 2024-05-03T18:50:31.790398+0000) 2024-05-03T18:50:36.411 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:36 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7[49675]: 2024-05-03T18:50:36.153+0000 7ff8b6600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T18:50:36.411 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:36 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7[49675]: 2024-05-03T18:50:36.153+0000 7ff8b6600700 -1 osd.7 126 *** Got signal Terminated *** 2024-05-03T18:50:36.411 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:36 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7[49675]: 2024-05-03T18:50:36.153+0000 7ff8b6600700 -1 osd.7 126 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-03T18:50:36.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:36 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:36.311+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:27.608012+0000 front 2024-05-03T18:49:27.608078+0000 (oldest deadline 2024-05-03T18:49:53.508290+0000) 2024-05-03T18:50:36.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:36 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:36.311+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:38.811037+0000 front 2024-05-03T18:49:38.811219+0000 (oldest deadline 2024-05-03T18:50:01.611951+0000) 2024-05-03T18:50:36.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:36 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:36.311+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:46.215926+0000 front 2024-05-03T18:49:46.216076+0000 (oldest deadline 2024-05-03T18:50:12.116128+0000) 2024-05-03T18:50:36.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:36 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:36.311+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:56.719409+0000 front 2024-05-03T18:49:56.719531+0000 (oldest deadline 2024-05-03T18:50:21.419652+0000) 2024-05-03T18:50:36.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:36 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:36.311+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:08.223552+0000 front 2024-05-03T18:50:08.224016+0000 (oldest deadline 2024-05-03T18:50:33.925215+0000) 2024-05-03T18:50:36.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:36 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:36.409+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:26.972061+0000 front 2024-05-03T18:49:26.972448+0000 (oldest deadline 2024-05-03T18:49:50.472344+0000) 2024-05-03T18:50:36.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:36 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:36.409+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:37.476081+0000 front 2024-05-03T18:49:37.476172+0000 (oldest deadline 2024-05-03T18:50:01.576192+0000) 2024-05-03T18:50:36.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:36 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:36.409+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:49.079592+0000 front 2024-05-03T18:49:49.080117+0000 (oldest deadline 2024-05-03T18:50:10.779802+0000) 2024-05-03T18:50:36.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:36 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:36.409+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:59.985863+0000 front 2024-05-03T18:49:59.986111+0000 (oldest deadline 2024-05-03T18:50:22.285990+0000) 2024-05-03T18:50:37.297 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:36 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7[49675]: 2024-05-03T18:50:36.869+0000 7ff8ace00700 -1 osd.7 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:36.774719+0000 front 2024-05-03T18:49:36.774769+0000 (oldest deadline 2024-05-03T18:50:02.674865+0000) 2024-05-03T18:50:37.297 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:36 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7[49675]: 2024-05-03T18:50:36.869+0000 7ff8ace00700 -1 osd.7 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:47.376623+0000 front 2024-05-03T18:49:47.376700+0000 (oldest deadline 2024-05-03T18:50:10.876637+0000) 2024-05-03T18:50:37.297 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:36 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7[49675]: 2024-05-03T18:50:36.869+0000 7ff8ace00700 -1 osd.7 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:59.080150+0000 front 2024-05-03T18:49:59.080046+0000 (oldest deadline 2024-05-03T18:50:23.179826+0000) 2024-05-03T18:50:37.297 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:36 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7[49675]: 2024-05-03T18:50:36.869+0000 7ff8ace00700 -1 osd.7 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:07.282208+0000 front 2024-05-03T18:50:07.282301+0000 (oldest deadline 2024-05-03T18:50:31.982398+0000) 2024-05-03T18:50:37.383 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:37 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:37.003+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:24.877883+0000 front 2024-05-03T18:49:24.877625+0000 (oldest deadline 2024-05-03T18:49:50.777881+0000) 2024-05-03T18:50:37.383 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:37 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:37.003+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:47.685409+0000 front 2024-05-03T18:49:47.685527+0000 (oldest deadline 2024-05-03T18:50:11.785436+0000) 2024-05-03T18:50:37.383 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:37 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:37.003+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:57.687614+0000 front 2024-05-03T18:49:57.687504+0000 (oldest deadline 2024-05-03T18:50:24.688928+0000) 2024-05-03T18:50:37.384 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:37 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:37.003+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:08.191069+0000 front 2024-05-03T18:50:08.190956+0000 (oldest deadline 2024-05-03T18:50:34.091182+0000) 2024-05-03T18:50:37.384 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:37 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:37.229+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:25.480913+0000 front 2024-05-03T18:49:25.480403+0000 (oldest deadline 2024-05-03T18:49:50.180152+0000) 2024-05-03T18:50:37.384 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:37 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:37.229+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:34.882184+0000 front 2024-05-03T18:49:34.882683+0000 (oldest deadline 2024-05-03T18:50:05.483250+0000) 2024-05-03T18:50:37.384 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:37 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:37.229+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:45.484981+0000 front 2024-05-03T18:49:45.484933+0000 (oldest deadline 2024-05-03T18:50:14.885581+0000) 2024-05-03T18:50:37.384 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:37 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:37.229+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:08.290430+0000 front 2024-05-03T18:50:08.290390+0000 (oldest deadline 2024-05-03T18:50:31.790398+0000) 2024-05-03T18:50:37.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:37 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:37.295+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:27.608012+0000 front 2024-05-03T18:49:27.608078+0000 (oldest deadline 2024-05-03T18:49:53.508290+0000) 2024-05-03T18:50:37.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:37 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:37.295+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:38.811037+0000 front 2024-05-03T18:49:38.811219+0000 (oldest deadline 2024-05-03T18:50:01.611951+0000) 2024-05-03T18:50:37.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:37 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:37.295+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:46.215926+0000 front 2024-05-03T18:49:46.216076+0000 (oldest deadline 2024-05-03T18:50:12.116128+0000) 2024-05-03T18:50:37.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:37 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:37.295+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:56.719409+0000 front 2024-05-03T18:49:56.719531+0000 (oldest deadline 2024-05-03T18:50:21.419652+0000) 2024-05-03T18:50:37.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:37 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:37.295+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:08.223552+0000 front 2024-05-03T18:50:08.224016+0000 (oldest deadline 2024-05-03T18:50:33.925215+0000) 2024-05-03T18:50:37.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:37 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:37.392+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:26.972061+0000 front 2024-05-03T18:49:26.972448+0000 (oldest deadline 2024-05-03T18:49:50.472344+0000) 2024-05-03T18:50:37.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:37 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:37.392+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:37.476081+0000 front 2024-05-03T18:49:37.476172+0000 (oldest deadline 2024-05-03T18:50:01.576192+0000) 2024-05-03T18:50:37.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:37 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:37.392+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:49.079592+0000 front 2024-05-03T18:49:49.080117+0000 (oldest deadline 2024-05-03T18:50:10.779802+0000) 2024-05-03T18:50:37.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:37 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:37.392+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:59.985863+0000 front 2024-05-03T18:49:59.986111+0000 (oldest deadline 2024-05-03T18:50:22.285990+0000) 2024-05-03T18:50:37.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:37 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:37.392+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:05.688218+0000 front 2024-05-03T18:50:05.688162+0000 (oldest deadline 2024-05-03T18:50:36.889315+0000) 2024-05-03T18:50:38.296 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:37 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7[49675]: 2024-05-03T18:50:37.891+0000 7ff8ace00700 -1 osd.7 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:36.774719+0000 front 2024-05-03T18:49:36.774769+0000 (oldest deadline 2024-05-03T18:50:02.674865+0000) 2024-05-03T18:50:38.297 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:37 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7[49675]: 2024-05-03T18:50:37.891+0000 7ff8ace00700 -1 osd.7 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:47.376623+0000 front 2024-05-03T18:49:47.376700+0000 (oldest deadline 2024-05-03T18:50:10.876637+0000) 2024-05-03T18:50:38.297 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:37 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7[49675]: 2024-05-03T18:50:37.891+0000 7ff8ace00700 -1 osd.7 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:59.080150+0000 front 2024-05-03T18:49:59.080046+0000 (oldest deadline 2024-05-03T18:50:23.179826+0000) 2024-05-03T18:50:38.297 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:37 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7[49675]: 2024-05-03T18:50:37.891+0000 7ff8ace00700 -1 osd.7 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:07.282208+0000 front 2024-05-03T18:50:07.282301+0000 (oldest deadline 2024-05-03T18:50:31.982398+0000) 2024-05-03T18:50:38.383 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:38 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:38.001+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:24.877883+0000 front 2024-05-03T18:49:24.877625+0000 (oldest deadline 2024-05-03T18:49:50.777881+0000) 2024-05-03T18:50:38.383 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:38 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:38.001+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:47.685409+0000 front 2024-05-03T18:49:47.685527+0000 (oldest deadline 2024-05-03T18:50:11.785436+0000) 2024-05-03T18:50:38.383 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:38 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:38.001+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:57.687614+0000 front 2024-05-03T18:49:57.687504+0000 (oldest deadline 2024-05-03T18:50:24.688928+0000) 2024-05-03T18:50:38.384 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:38 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:38.001+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:08.191069+0000 front 2024-05-03T18:50:08.190956+0000 (oldest deadline 2024-05-03T18:50:34.091182+0000) 2024-05-03T18:50:38.384 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:38 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:38.189+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:25.480913+0000 front 2024-05-03T18:49:25.480403+0000 (oldest deadline 2024-05-03T18:49:50.180152+0000) 2024-05-03T18:50:38.384 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:38 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:38.189+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:34.882184+0000 front 2024-05-03T18:49:34.882683+0000 (oldest deadline 2024-05-03T18:50:05.483250+0000) 2024-05-03T18:50:38.384 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:38 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:38.189+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:45.484981+0000 front 2024-05-03T18:49:45.484933+0000 (oldest deadline 2024-05-03T18:50:14.885581+0000) 2024-05-03T18:50:38.384 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:38 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:38.189+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:08.290430+0000 front 2024-05-03T18:50:08.290390+0000 (oldest deadline 2024-05-03T18:50:31.790398+0000) 2024-05-03T18:50:38.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:38 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:38.342+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:27.608012+0000 front 2024-05-03T18:49:27.608078+0000 (oldest deadline 2024-05-03T18:49:53.508290+0000) 2024-05-03T18:50:38.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:38 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:38.342+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:38.811037+0000 front 2024-05-03T18:49:38.811219+0000 (oldest deadline 2024-05-03T18:50:01.611951+0000) 2024-05-03T18:50:38.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:38 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:38.342+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:46.215926+0000 front 2024-05-03T18:49:46.216076+0000 (oldest deadline 2024-05-03T18:50:12.116128+0000) 2024-05-03T18:50:38.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:38 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:38.342+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:56.719409+0000 front 2024-05-03T18:49:56.719531+0000 (oldest deadline 2024-05-03T18:50:21.419652+0000) 2024-05-03T18:50:38.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:38 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:38.342+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:08.223552+0000 front 2024-05-03T18:50:08.224016+0000 (oldest deadline 2024-05-03T18:50:33.925215+0000) 2024-05-03T18:50:38.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:38 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:38.372+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:26.972061+0000 front 2024-05-03T18:49:26.972448+0000 (oldest deadline 2024-05-03T18:49:50.472344+0000) 2024-05-03T18:50:38.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:38 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:38.372+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:37.476081+0000 front 2024-05-03T18:49:37.476172+0000 (oldest deadline 2024-05-03T18:50:01.576192+0000) 2024-05-03T18:50:38.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:38 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:38.372+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:49.079592+0000 front 2024-05-03T18:49:49.080117+0000 (oldest deadline 2024-05-03T18:50:10.779802+0000) 2024-05-03T18:50:38.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:38 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:38.372+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:59.985863+0000 front 2024-05-03T18:49:59.986111+0000 (oldest deadline 2024-05-03T18:50:22.285990+0000) 2024-05-03T18:50:38.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:38 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:38.372+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:05.688218+0000 front 2024-05-03T18:50:05.688162+0000 (oldest deadline 2024-05-03T18:50:36.889315+0000) 2024-05-03T18:50:39.297 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:38 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7[49675]: 2024-05-03T18:50:38.872+0000 7ff8ace00700 -1 osd.7 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:36.774719+0000 front 2024-05-03T18:49:36.774769+0000 (oldest deadline 2024-05-03T18:50:02.674865+0000) 2024-05-03T18:50:39.297 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:38 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7[49675]: 2024-05-03T18:50:38.872+0000 7ff8ace00700 -1 osd.7 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:47.376623+0000 front 2024-05-03T18:49:47.376700+0000 (oldest deadline 2024-05-03T18:50:10.876637+0000) 2024-05-03T18:50:39.297 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:38 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7[49675]: 2024-05-03T18:50:38.872+0000 7ff8ace00700 -1 osd.7 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:59.080150+0000 front 2024-05-03T18:49:59.080046+0000 (oldest deadline 2024-05-03T18:50:23.179826+0000) 2024-05-03T18:50:39.297 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:38 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7[49675]: 2024-05-03T18:50:38.872+0000 7ff8ace00700 -1 osd.7 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:07.282208+0000 front 2024-05-03T18:50:07.282301+0000 (oldest deadline 2024-05-03T18:50:31.982398+0000) 2024-05-03T18:50:39.383 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:39 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:39.032+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:24.877883+0000 front 2024-05-03T18:49:24.877625+0000 (oldest deadline 2024-05-03T18:49:50.777881+0000) 2024-05-03T18:50:39.384 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:39 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:39.032+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:47.685409+0000 front 2024-05-03T18:49:47.685527+0000 (oldest deadline 2024-05-03T18:50:11.785436+0000) 2024-05-03T18:50:39.384 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:39 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:39.032+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:57.687614+0000 front 2024-05-03T18:49:57.687504+0000 (oldest deadline 2024-05-03T18:50:24.688928+0000) 2024-05-03T18:50:39.384 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:39 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:39.032+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:08.191069+0000 front 2024-05-03T18:50:08.190956+0000 (oldest deadline 2024-05-03T18:50:34.091182+0000) 2024-05-03T18:50:39.384 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:39 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:39.239+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:25.480913+0000 front 2024-05-03T18:49:25.480403+0000 (oldest deadline 2024-05-03T18:49:50.180152+0000) 2024-05-03T18:50:39.384 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:39 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:39.239+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:34.882184+0000 front 2024-05-03T18:49:34.882683+0000 (oldest deadline 2024-05-03T18:50:05.483250+0000) 2024-05-03T18:50:39.384 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:39 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:39.239+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:45.484981+0000 front 2024-05-03T18:49:45.484933+0000 (oldest deadline 2024-05-03T18:50:14.885581+0000) 2024-05-03T18:50:39.384 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:39 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:39.239+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:08.290430+0000 front 2024-05-03T18:50:08.290390+0000 (oldest deadline 2024-05-03T18:50:31.790398+0000) 2024-05-03T18:50:39.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:39 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:39.365+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:27.608012+0000 front 2024-05-03T18:49:27.608078+0000 (oldest deadline 2024-05-03T18:49:53.508290+0000) 2024-05-03T18:50:39.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:39 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:39.365+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:38.811037+0000 front 2024-05-03T18:49:38.811219+0000 (oldest deadline 2024-05-03T18:50:01.611951+0000) 2024-05-03T18:50:39.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:39 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:39.365+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:46.215926+0000 front 2024-05-03T18:49:46.216076+0000 (oldest deadline 2024-05-03T18:50:12.116128+0000) 2024-05-03T18:50:39.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:39 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:39.365+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:56.719409+0000 front 2024-05-03T18:49:56.719531+0000 (oldest deadline 2024-05-03T18:50:21.419652+0000) 2024-05-03T18:50:39.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:39 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:39.365+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:08.223552+0000 front 2024-05-03T18:50:08.224016+0000 (oldest deadline 2024-05-03T18:50:33.925215+0000) 2024-05-03T18:50:39.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:39 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:39.373+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:26.972061+0000 front 2024-05-03T18:49:26.972448+0000 (oldest deadline 2024-05-03T18:49:50.472344+0000) 2024-05-03T18:50:39.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:39 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:39.373+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:37.476081+0000 front 2024-05-03T18:49:37.476172+0000 (oldest deadline 2024-05-03T18:50:01.576192+0000) 2024-05-03T18:50:39.798 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:39 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:39.373+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:49.079592+0000 front 2024-05-03T18:49:49.080117+0000 (oldest deadline 2024-05-03T18:50:10.779802+0000) 2024-05-03T18:50:39.798 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:39 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:39.373+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:59.985863+0000 front 2024-05-03T18:49:59.986111+0000 (oldest deadline 2024-05-03T18:50:22.285990+0000) 2024-05-03T18:50:39.798 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:39 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:39.373+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:05.688218+0000 front 2024-05-03T18:50:05.688162+0000 (oldest deadline 2024-05-03T18:50:36.889315+0000) 2024-05-03T18:50:40.297 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:39 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7[49675]: 2024-05-03T18:50:39.892+0000 7ff8ace00700 -1 osd.7 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:36.774719+0000 front 2024-05-03T18:49:36.774769+0000 (oldest deadline 2024-05-03T18:50:02.674865+0000) 2024-05-03T18:50:40.297 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:39 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7[49675]: 2024-05-03T18:50:39.892+0000 7ff8ace00700 -1 osd.7 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:47.376623+0000 front 2024-05-03T18:49:47.376700+0000 (oldest deadline 2024-05-03T18:50:10.876637+0000) 2024-05-03T18:50:40.297 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:39 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7[49675]: 2024-05-03T18:50:39.892+0000 7ff8ace00700 -1 osd.7 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:59.080150+0000 front 2024-05-03T18:49:59.080046+0000 (oldest deadline 2024-05-03T18:50:23.179826+0000) 2024-05-03T18:50:40.297 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:39 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7[49675]: 2024-05-03T18:50:39.892+0000 7ff8ace00700 -1 osd.7 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:07.282208+0000 front 2024-05-03T18:50:07.282301+0000 (oldest deadline 2024-05-03T18:50:31.982398+0000) 2024-05-03T18:50:40.383 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:40 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:40.063+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:24.877883+0000 front 2024-05-03T18:49:24.877625+0000 (oldest deadline 2024-05-03T18:49:50.777881+0000) 2024-05-03T18:50:40.383 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:40 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:40.063+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:47.685409+0000 front 2024-05-03T18:49:47.685527+0000 (oldest deadline 2024-05-03T18:50:11.785436+0000) 2024-05-03T18:50:40.384 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:40 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:40.063+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:57.687614+0000 front 2024-05-03T18:49:57.687504+0000 (oldest deadline 2024-05-03T18:50:24.688928+0000) 2024-05-03T18:50:40.384 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:40 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:40.063+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:08.191069+0000 front 2024-05-03T18:50:08.190956+0000 (oldest deadline 2024-05-03T18:50:34.091182+0000) 2024-05-03T18:50:40.384 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:40 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:40.195+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:25.480913+0000 front 2024-05-03T18:49:25.480403+0000 (oldest deadline 2024-05-03T18:49:50.180152+0000) 2024-05-03T18:50:40.384 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:40 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:40.195+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:34.882184+0000 front 2024-05-03T18:49:34.882683+0000 (oldest deadline 2024-05-03T18:50:05.483250+0000) 2024-05-03T18:50:40.384 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:40 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:40.195+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:45.484981+0000 front 2024-05-03T18:49:45.484933+0000 (oldest deadline 2024-05-03T18:50:14.885581+0000) 2024-05-03T18:50:40.384 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:40 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:40.195+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:08.290430+0000 front 2024-05-03T18:50:08.290390+0000 (oldest deadline 2024-05-03T18:50:31.790398+0000) 2024-05-03T18:50:40.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:40 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:40.336+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:27.608012+0000 front 2024-05-03T18:49:27.608078+0000 (oldest deadline 2024-05-03T18:49:53.508290+0000) 2024-05-03T18:50:40.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:40 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:40.336+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:38.811037+0000 front 2024-05-03T18:49:38.811219+0000 (oldest deadline 2024-05-03T18:50:01.611951+0000) 2024-05-03T18:50:40.736 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:40 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:40.336+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:46.215926+0000 front 2024-05-03T18:49:46.216076+0000 (oldest deadline 2024-05-03T18:50:12.116128+0000) 2024-05-03T18:50:40.736 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:40 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:40.336+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:56.719409+0000 front 2024-05-03T18:49:56.719531+0000 (oldest deadline 2024-05-03T18:50:21.419652+0000) 2024-05-03T18:50:40.736 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:40 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:40.336+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:08.223552+0000 front 2024-05-03T18:50:08.224016+0000 (oldest deadline 2024-05-03T18:50:33.925215+0000) 2024-05-03T18:50:40.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:40 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:40.360+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:26.972061+0000 front 2024-05-03T18:49:26.972448+0000 (oldest deadline 2024-05-03T18:49:50.472344+0000) 2024-05-03T18:50:40.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:40 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:40.360+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:37.476081+0000 front 2024-05-03T18:49:37.476172+0000 (oldest deadline 2024-05-03T18:50:01.576192+0000) 2024-05-03T18:50:40.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:40 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:40.360+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:49.079592+0000 front 2024-05-03T18:49:49.080117+0000 (oldest deadline 2024-05-03T18:50:10.779802+0000) 2024-05-03T18:50:40.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:40 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:40.360+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:59.985863+0000 front 2024-05-03T18:49:59.986111+0000 (oldest deadline 2024-05-03T18:50:22.285990+0000) 2024-05-03T18:50:40.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:40 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:40.360+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:05.688218+0000 front 2024-05-03T18:50:05.688162+0000 (oldest deadline 2024-05-03T18:50:36.889315+0000) 2024-05-03T18:50:41.297 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:40 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7[49675]: 2024-05-03T18:50:40.891+0000 7ff8ace00700 -1 osd.7 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:36.774719+0000 front 2024-05-03T18:49:36.774769+0000 (oldest deadline 2024-05-03T18:50:02.674865+0000) 2024-05-03T18:50:41.297 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:40 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7[49675]: 2024-05-03T18:50:40.891+0000 7ff8ace00700 -1 osd.7 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:47.376623+0000 front 2024-05-03T18:49:47.376700+0000 (oldest deadline 2024-05-03T18:50:10.876637+0000) 2024-05-03T18:50:41.297 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:40 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7[49675]: 2024-05-03T18:50:40.891+0000 7ff8ace00700 -1 osd.7 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:59.080150+0000 front 2024-05-03T18:49:59.080046+0000 (oldest deadline 2024-05-03T18:50:23.179826+0000) 2024-05-03T18:50:41.297 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:40 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7[49675]: 2024-05-03T18:50:40.891+0000 7ff8ace00700 -1 osd.7 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:07.282208+0000 front 2024-05-03T18:50:07.282301+0000 (oldest deadline 2024-05-03T18:50:31.982398+0000) 2024-05-03T18:50:41.383 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:41 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:41.062+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:24.877883+0000 front 2024-05-03T18:49:24.877625+0000 (oldest deadline 2024-05-03T18:49:50.777881+0000) 2024-05-03T18:50:41.384 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:41 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:41.062+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:47.685409+0000 front 2024-05-03T18:49:47.685527+0000 (oldest deadline 2024-05-03T18:50:11.785436+0000) 2024-05-03T18:50:41.384 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:41 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:41.062+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:57.687614+0000 front 2024-05-03T18:49:57.687504+0000 (oldest deadline 2024-05-03T18:50:24.688928+0000) 2024-05-03T18:50:41.384 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:41 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:41.062+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:08.191069+0000 front 2024-05-03T18:50:08.190956+0000 (oldest deadline 2024-05-03T18:50:34.091182+0000) 2024-05-03T18:50:41.384 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:41 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:41.178+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:25.480913+0000 front 2024-05-03T18:49:25.480403+0000 (oldest deadline 2024-05-03T18:49:50.180152+0000) 2024-05-03T18:50:41.384 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:41 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:41.178+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:34.882184+0000 front 2024-05-03T18:49:34.882683+0000 (oldest deadline 2024-05-03T18:50:05.483250+0000) 2024-05-03T18:50:41.384 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:41 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:41.178+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:45.484981+0000 front 2024-05-03T18:49:45.484933+0000 (oldest deadline 2024-05-03T18:50:14.885581+0000) 2024-05-03T18:50:41.385 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:41 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:41.178+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:08.290430+0000 front 2024-05-03T18:50:08.290390+0000 (oldest deadline 2024-05-03T18:50:31.790398+0000) 2024-05-03T18:50:41.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:41 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:41.336+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:27.608012+0000 front 2024-05-03T18:49:27.608078+0000 (oldest deadline 2024-05-03T18:49:53.508290+0000) 2024-05-03T18:50:41.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:41 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:41.336+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:38.811037+0000 front 2024-05-03T18:49:38.811219+0000 (oldest deadline 2024-05-03T18:50:01.611951+0000) 2024-05-03T18:50:41.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:41 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:41.336+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:46.215926+0000 front 2024-05-03T18:49:46.216076+0000 (oldest deadline 2024-05-03T18:50:12.116128+0000) 2024-05-03T18:50:41.736 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:41 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:41.336+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:56.719409+0000 front 2024-05-03T18:49:56.719531+0000 (oldest deadline 2024-05-03T18:50:21.419652+0000) 2024-05-03T18:50:41.736 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:41 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:41.336+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:08.223552+0000 front 2024-05-03T18:50:08.224016+0000 (oldest deadline 2024-05-03T18:50:33.925215+0000) 2024-05-03T18:50:41.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:41 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:41.314+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:26.972061+0000 front 2024-05-03T18:49:26.972448+0000 (oldest deadline 2024-05-03T18:49:50.472344+0000) 2024-05-03T18:50:41.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:41 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:41.314+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:37.476081+0000 front 2024-05-03T18:49:37.476172+0000 (oldest deadline 2024-05-03T18:50:01.576192+0000) 2024-05-03T18:50:41.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:41 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:41.314+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:49.079592+0000 front 2024-05-03T18:49:49.080117+0000 (oldest deadline 2024-05-03T18:50:10.779802+0000) 2024-05-03T18:50:41.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:41 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:41.314+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:59.985863+0000 front 2024-05-03T18:49:59.986111+0000 (oldest deadline 2024-05-03T18:50:22.285990+0000) 2024-05-03T18:50:41.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:41 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:41.314+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:05.688218+0000 front 2024-05-03T18:50:05.688162+0000 (oldest deadline 2024-05-03T18:50:36.889315+0000) 2024-05-03T18:50:42.383 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:42 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:42.103+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:24.877883+0000 front 2024-05-03T18:49:24.877625+0000 (oldest deadline 2024-05-03T18:49:50.777881+0000) 2024-05-03T18:50:42.383 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:42 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:42.103+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:47.685409+0000 front 2024-05-03T18:49:47.685527+0000 (oldest deadline 2024-05-03T18:50:11.785436+0000) 2024-05-03T18:50:42.383 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:42 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:42.103+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:57.687614+0000 front 2024-05-03T18:49:57.687504+0000 (oldest deadline 2024-05-03T18:50:24.688928+0000) 2024-05-03T18:50:42.383 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:42 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:42.103+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:08.191069+0000 front 2024-05-03T18:50:08.190956+0000 (oldest deadline 2024-05-03T18:50:34.091182+0000) 2024-05-03T18:50:42.384 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:42 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:42.160+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:25.480913+0000 front 2024-05-03T18:49:25.480403+0000 (oldest deadline 2024-05-03T18:49:50.180152+0000) 2024-05-03T18:50:42.384 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:42 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:42.160+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:34.882184+0000 front 2024-05-03T18:49:34.882683+0000 (oldest deadline 2024-05-03T18:50:05.483250+0000) 2024-05-03T18:50:42.384 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:42 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:42.160+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:45.484981+0000 front 2024-05-03T18:49:45.484933+0000 (oldest deadline 2024-05-03T18:50:14.885581+0000) 2024-05-03T18:50:42.384 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:42 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:42.160+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:08.290430+0000 front 2024-05-03T18:50:08.290390+0000 (oldest deadline 2024-05-03T18:50:31.790398+0000) 2024-05-03T18:50:42.708 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:42 smithi099 podman[71355]: 2024-05-03 18:50:42.390493575 +0000 UTC m=+6.392611459 container died 90cd82296c0ed800b144d2f0fd14ecad924a1b409b1c6f0a74985f83e67fb497 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-03T18:50:42.708 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:42 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:42.325+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:26.972061+0000 front 2024-05-03T18:49:26.972448+0000 (oldest deadline 2024-05-03T18:49:50.472344+0000) 2024-05-03T18:50:42.708 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:42 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:42.325+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:37.476081+0000 front 2024-05-03T18:49:37.476172+0000 (oldest deadline 2024-05-03T18:50:01.576192+0000) 2024-05-03T18:50:42.708 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:42 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:42.325+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:49.079592+0000 front 2024-05-03T18:49:49.080117+0000 (oldest deadline 2024-05-03T18:50:10.779802+0000) 2024-05-03T18:50:42.708 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:42 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:42.325+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:59.985863+0000 front 2024-05-03T18:49:59.986111+0000 (oldest deadline 2024-05-03T18:50:22.285990+0000) 2024-05-03T18:50:42.708 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:42 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:42.325+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:05.688218+0000 front 2024-05-03T18:50:05.688162+0000 (oldest deadline 2024-05-03T18:50:36.889315+0000) 2024-05-03T18:50:42.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:42 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:42.353+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:27.608012+0000 front 2024-05-03T18:49:27.608078+0000 (oldest deadline 2024-05-03T18:49:53.508290+0000) 2024-05-03T18:50:42.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:42 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:42.353+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:38.811037+0000 front 2024-05-03T18:49:38.811219+0000 (oldest deadline 2024-05-03T18:50:01.611951+0000) 2024-05-03T18:50:42.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:42 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:42.353+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:46.215926+0000 front 2024-05-03T18:49:46.216076+0000 (oldest deadline 2024-05-03T18:50:12.116128+0000) 2024-05-03T18:50:42.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:42 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:42.353+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:56.719409+0000 front 2024-05-03T18:49:56.719531+0000 (oldest deadline 2024-05-03T18:50:21.419652+0000) 2024-05-03T18:50:42.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:42 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:42.353+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:08.223552+0000 front 2024-05-03T18:50:08.224016+0000 (oldest deadline 2024-05-03T18:50:33.925215+0000) 2024-05-03T18:50:43.047 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:42 smithi099 podman[71355]: 2024-05-03 18:50:42.895059479 +0000 UTC m=+6.897177397 container cleanup 90cd82296c0ed800b144d2f0fd14ecad924a1b409b1c6f0a74985f83e67fb497 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430, GIT_CLEAN=True) 2024-05-03T18:50:43.047 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:42 smithi099 bash[71355]: ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7 2024-05-03T18:50:43.383 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:43 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:43.158+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:25.480913+0000 front 2024-05-03T18:49:25.480403+0000 (oldest deadline 2024-05-03T18:49:50.180152+0000) 2024-05-03T18:50:43.383 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:43 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:43.158+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:34.882184+0000 front 2024-05-03T18:49:34.882683+0000 (oldest deadline 2024-05-03T18:50:05.483250+0000) 2024-05-03T18:50:43.383 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:43 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:43.158+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:45.484981+0000 front 2024-05-03T18:49:45.484933+0000 (oldest deadline 2024-05-03T18:50:14.885581+0000) 2024-05-03T18:50:43.384 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:43 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:43.158+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:08.290430+0000 front 2024-05-03T18:50:08.290390+0000 (oldest deadline 2024-05-03T18:50:31.790398+0000) 2024-05-03T18:50:43.384 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:43 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:43.084+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:24.877883+0000 front 2024-05-03T18:49:24.877625+0000 (oldest deadline 2024-05-03T18:49:50.777881+0000) 2024-05-03T18:50:43.384 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:43 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:43.084+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:47.685409+0000 front 2024-05-03T18:49:47.685527+0000 (oldest deadline 2024-05-03T18:50:11.785436+0000) 2024-05-03T18:50:43.384 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:43 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:43.084+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:57.687614+0000 front 2024-05-03T18:49:57.687504+0000 (oldest deadline 2024-05-03T18:50:24.688928+0000) 2024-05-03T18:50:43.384 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:43 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:43.084+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:08.191069+0000 front 2024-05-03T18:50:08.190956+0000 (oldest deadline 2024-05-03T18:50:34.091182+0000) 2024-05-03T18:50:43.384 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:43 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:43.084+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2024-05-03T18:50:18.193738+0000 front 2024-05-03T18:50:18.193572+0000 (oldest deadline 2024-05-03T18:50:42.893222+0000) 2024-05-03T18:50:43.546 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:43 smithi099 podman[71373]: 2024-05-03 18:50:43.206605246 +0000 UTC m=+0.785043764 container remove 90cd82296c0ed800b144d2f0fd14ecad924a1b409b1c6f0a74985f83e67fb497 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-05-03T18:50:43.546 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:43 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:43.279+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:26.972061+0000 front 2024-05-03T18:49:26.972448+0000 (oldest deadline 2024-05-03T18:49:50.472344+0000) 2024-05-03T18:50:43.546 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:43 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:43.279+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:37.476081+0000 front 2024-05-03T18:49:37.476172+0000 (oldest deadline 2024-05-03T18:50:01.576192+0000) 2024-05-03T18:50:43.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:43 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:43.279+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:49.079592+0000 front 2024-05-03T18:49:49.080117+0000 (oldest deadline 2024-05-03T18:50:10.779802+0000) 2024-05-03T18:50:43.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:43 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:43.279+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:59.985863+0000 front 2024-05-03T18:49:59.986111+0000 (oldest deadline 2024-05-03T18:50:22.285990+0000) 2024-05-03T18:50:43.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:43 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:43.279+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:05.688218+0000 front 2024-05-03T18:50:05.688162+0000 (oldest deadline 2024-05-03T18:50:36.889315+0000) 2024-05-03T18:50:43.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:43 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:43.378+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:27.608012+0000 front 2024-05-03T18:49:27.608078+0000 (oldest deadline 2024-05-03T18:49:53.508290+0000) 2024-05-03T18:50:43.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:43 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:43.378+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:38.811037+0000 front 2024-05-03T18:49:38.811219+0000 (oldest deadline 2024-05-03T18:50:01.611951+0000) 2024-05-03T18:50:43.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:43 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:43.378+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:46.215926+0000 front 2024-05-03T18:49:46.216076+0000 (oldest deadline 2024-05-03T18:50:12.116128+0000) 2024-05-03T18:50:43.736 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:43 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:43.378+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:56.719409+0000 front 2024-05-03T18:49:56.719531+0000 (oldest deadline 2024-05-03T18:50:21.419652+0000) 2024-05-03T18:50:43.736 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:43 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:43.378+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:08.223552+0000 front 2024-05-03T18:50:08.224016+0000 (oldest deadline 2024-05-03T18:50:33.925215+0000) 2024-05-03T18:50:43.736 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:43 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:43.378+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2024-05-03T18:50:20.327897+0000 front 2024-05-03T18:50:20.327850+0000 (oldest deadline 2024-05-03T18:50:43.227764+0000) 2024-05-03T18:50:44.116 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:43 smithi099 podman[71461]: 2024-05-03 18:50:43.712319414 +0000 UTC m=+0.035089506 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T18:50:44.116 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:43 smithi099 podman[71461]: 2024-05-03 18:50:43.860571033 +0000 UTC m=+0.183341143 container create 1b1d5cd239265232a12778554e1633cca64a219bce90035cb0aef2351373d952 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-05-03T18:50:44.116 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:44 smithi099 podman[71461]: 2024-05-03 18:50:44.113505187 +0000 UTC m=+0.436275310 container init 1b1d5cd239265232a12778554e1633cca64a219bce90035cb0aef2351373d952 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-03T18:50:44.383 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:44 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:44.035+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:24.877883+0000 front 2024-05-03T18:49:24.877625+0000 (oldest deadline 2024-05-03T18:49:50.777881+0000) 2024-05-03T18:50:44.384 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:44 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:44.035+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:47.685409+0000 front 2024-05-03T18:49:47.685527+0000 (oldest deadline 2024-05-03T18:50:11.785436+0000) 2024-05-03T18:50:44.384 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:44 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:44.035+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:57.687614+0000 front 2024-05-03T18:49:57.687504+0000 (oldest deadline 2024-05-03T18:50:24.688928+0000) 2024-05-03T18:50:44.384 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:44 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:44.035+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:08.191069+0000 front 2024-05-03T18:50:08.190956+0000 (oldest deadline 2024-05-03T18:50:34.091182+0000) 2024-05-03T18:50:44.384 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:44 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:44.035+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2024-05-03T18:50:18.193738+0000 front 2024-05-03T18:50:18.193572+0000 (oldest deadline 2024-05-03T18:50:42.893222+0000) 2024-05-03T18:50:44.384 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:44 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:44.184+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:25.480913+0000 front 2024-05-03T18:49:25.480403+0000 (oldest deadline 2024-05-03T18:49:50.180152+0000) 2024-05-03T18:50:44.384 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:44 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:44.184+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:34.882184+0000 front 2024-05-03T18:49:34.882683+0000 (oldest deadline 2024-05-03T18:50:05.483250+0000) 2024-05-03T18:50:44.384 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:44 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:44.184+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:45.484981+0000 front 2024-05-03T18:49:45.484933+0000 (oldest deadline 2024-05-03T18:50:14.885581+0000) 2024-05-03T18:50:44.384 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:44 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:44.184+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:08.290430+0000 front 2024-05-03T18:50:08.290390+0000 (oldest deadline 2024-05-03T18:50:31.790398+0000) 2024-05-03T18:50:44.384 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:44 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:44.184+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2024-05-03T18:50:18.193786+0000 front 2024-05-03T18:50:18.193853+0000 (oldest deadline 2024-05-03T18:50:44.093727+0000) 2024-05-03T18:50:44.546 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:44 smithi099 podman[71461]: 2024-05-03 18:50:44.126498053 +0000 UTC m=+0.449268139 container start 1b1d5cd239265232a12778554e1633cca64a219bce90035cb0aef2351373d952 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe) 2024-05-03T18:50:44.546 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:44 smithi099 podman[71461]: 2024-05-03 18:50:44.24331076 +0000 UTC m=+0.566080932 container attach 1b1d5cd239265232a12778554e1633cca64a219bce90035cb0aef2351373d952 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7-deactivate, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-03T18:50:44.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:44 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:44.254+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:26.972061+0000 front 2024-05-03T18:49:26.972448+0000 (oldest deadline 2024-05-03T18:49:50.472344+0000) 2024-05-03T18:50:44.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:44 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:44.254+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:37.476081+0000 front 2024-05-03T18:49:37.476172+0000 (oldest deadline 2024-05-03T18:50:01.576192+0000) 2024-05-03T18:50:44.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:44 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:44.254+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:49.079592+0000 front 2024-05-03T18:49:49.080117+0000 (oldest deadline 2024-05-03T18:50:10.779802+0000) 2024-05-03T18:50:44.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:44 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:44.254+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:59.985863+0000 front 2024-05-03T18:49:59.986111+0000 (oldest deadline 2024-05-03T18:50:22.285990+0000) 2024-05-03T18:50:44.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:44 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:44.254+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:05.688218+0000 front 2024-05-03T18:50:05.688162+0000 (oldest deadline 2024-05-03T18:50:36.889315+0000) 2024-05-03T18:50:44.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:44 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:44.388+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:27.608012+0000 front 2024-05-03T18:49:27.608078+0000 (oldest deadline 2024-05-03T18:49:53.508290+0000) 2024-05-03T18:50:44.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:44 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:44.388+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:38.811037+0000 front 2024-05-03T18:49:38.811219+0000 (oldest deadline 2024-05-03T18:50:01.611951+0000) 2024-05-03T18:50:44.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:44 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:44.388+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:46.215926+0000 front 2024-05-03T18:49:46.216076+0000 (oldest deadline 2024-05-03T18:50:12.116128+0000) 2024-05-03T18:50:44.736 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:44 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:44.388+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:56.719409+0000 front 2024-05-03T18:49:56.719531+0000 (oldest deadline 2024-05-03T18:50:21.419652+0000) 2024-05-03T18:50:44.736 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:44 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:44.388+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:08.223552+0000 front 2024-05-03T18:50:08.224016+0000 (oldest deadline 2024-05-03T18:50:33.925215+0000) 2024-05-03T18:50:44.736 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:44 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:44.388+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2024-05-03T18:50:20.327897+0000 front 2024-05-03T18:50:20.327850+0000 (oldest deadline 2024-05-03T18:50:43.227764+0000) 2024-05-03T18:50:44.934 INFO:journalctl@ceph.osd.7.smithi099.stdout:May 03 18:50:44 smithi099 podman[71461]: 2024-05-03 18:50:44.579681121 +0000 UTC m=+0.902451323 container died 1b1d5cd239265232a12778554e1633cca64a219bce90035cb0aef2351373d952 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-05-03T18:50:45.337 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5703bba4-0978-11ef-bc97-c7b262605968@osd.7.service' 2024-05-03T18:50:45.383 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:45 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:45.001+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:24.877883+0000 front 2024-05-03T18:49:24.877625+0000 (oldest deadline 2024-05-03T18:49:50.777881+0000) 2024-05-03T18:50:45.384 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:45 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:45.001+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:47.685409+0000 front 2024-05-03T18:49:47.685527+0000 (oldest deadline 2024-05-03T18:50:11.785436+0000) 2024-05-03T18:50:45.384 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:45 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:45.001+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:57.687614+0000 front 2024-05-03T18:49:57.687504+0000 (oldest deadline 2024-05-03T18:50:24.688928+0000) 2024-05-03T18:50:45.384 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:45 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:45.001+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:08.191069+0000 front 2024-05-03T18:50:08.190956+0000 (oldest deadline 2024-05-03T18:50:34.091182+0000) 2024-05-03T18:50:45.384 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:45 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:45.001+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2024-05-03T18:50:18.193738+0000 front 2024-05-03T18:50:18.193572+0000 (oldest deadline 2024-05-03T18:50:42.893222+0000) 2024-05-03T18:50:45.384 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:45 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:45.207+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:25.480913+0000 front 2024-05-03T18:49:25.480403+0000 (oldest deadline 2024-05-03T18:49:50.180152+0000) 2024-05-03T18:50:45.384 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:45 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:45.208+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:34.882184+0000 front 2024-05-03T18:49:34.882683+0000 (oldest deadline 2024-05-03T18:50:05.483250+0000) 2024-05-03T18:50:45.384 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:45 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:45.208+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:45.484981+0000 front 2024-05-03T18:49:45.484933+0000 (oldest deadline 2024-05-03T18:50:14.885581+0000) 2024-05-03T18:50:45.384 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:45 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:45.208+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:08.290430+0000 front 2024-05-03T18:50:08.290390+0000 (oldest deadline 2024-05-03T18:50:31.790398+0000) 2024-05-03T18:50:45.384 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:45 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:45.208+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2024-05-03T18:50:18.193786+0000 front 2024-05-03T18:50:18.193853+0000 (oldest deadline 2024-05-03T18:50:44.093727+0000) 2024-05-03T18:50:45.441 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T18:50:45.441 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-05-03T18:50:45.441 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-05-03T18:50:45.442 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-5703bba4-0978-11ef-bc97-c7b262605968@osd.8 2024-05-03T18:50:45.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:45 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:45.258+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:26.972061+0000 front 2024-05-03T18:49:26.972448+0000 (oldest deadline 2024-05-03T18:49:50.472344+0000) 2024-05-03T18:50:45.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:45 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:45.258+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:37.476081+0000 front 2024-05-03T18:49:37.476172+0000 (oldest deadline 2024-05-03T18:50:01.576192+0000) 2024-05-03T18:50:45.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:45 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:45.258+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:49.079592+0000 front 2024-05-03T18:49:49.080117+0000 (oldest deadline 2024-05-03T18:50:10.779802+0000) 2024-05-03T18:50:45.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:45 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:45.258+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:59.985863+0000 front 2024-05-03T18:49:59.986111+0000 (oldest deadline 2024-05-03T18:50:22.285990+0000) 2024-05-03T18:50:45.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:45 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:45.258+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:05.688218+0000 front 2024-05-03T18:50:05.688162+0000 (oldest deadline 2024-05-03T18:50:36.889315+0000) 2024-05-03T18:50:45.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:45 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:45.405+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:27.608012+0000 front 2024-05-03T18:49:27.608078+0000 (oldest deadline 2024-05-03T18:49:53.508290+0000) 2024-05-03T18:50:45.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:45 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:45.405+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:38.811037+0000 front 2024-05-03T18:49:38.811219+0000 (oldest deadline 2024-05-03T18:50:01.611951+0000) 2024-05-03T18:50:45.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:45 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:45.405+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:46.215926+0000 front 2024-05-03T18:49:46.216076+0000 (oldest deadline 2024-05-03T18:50:12.116128+0000) 2024-05-03T18:50:45.736 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:45 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:45.405+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:56.719409+0000 front 2024-05-03T18:49:56.719531+0000 (oldest deadline 2024-05-03T18:50:21.419652+0000) 2024-05-03T18:50:45.736 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:45 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:45.405+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:08.223552+0000 front 2024-05-03T18:50:08.224016+0000 (oldest deadline 2024-05-03T18:50:33.925215+0000) 2024-05-03T18:50:45.736 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:45 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:45.405+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2024-05-03T18:50:20.327897+0000 front 2024-05-03T18:50:20.327850+0000 (oldest deadline 2024-05-03T18:50:43.227764+0000) 2024-05-03T18:50:45.883 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:45 smithi150 systemd[1]: Stopping Ceph osd.8 for 5703bba4-0978-11ef-bc97-c7b262605968... 2024-05-03T18:50:46.384 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:46 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:46.197+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:25.480913+0000 front 2024-05-03T18:49:25.480403+0000 (oldest deadline 2024-05-03T18:49:50.180152+0000) 2024-05-03T18:50:46.384 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:46 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:46.197+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:34.882184+0000 front 2024-05-03T18:49:34.882683+0000 (oldest deadline 2024-05-03T18:50:05.483250+0000) 2024-05-03T18:50:46.384 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:46 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:46.197+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:45.484981+0000 front 2024-05-03T18:49:45.484933+0000 (oldest deadline 2024-05-03T18:50:14.885581+0000) 2024-05-03T18:50:46.384 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:46 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:46.197+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:08.290430+0000 front 2024-05-03T18:50:08.290390+0000 (oldest deadline 2024-05-03T18:50:31.790398+0000) 2024-05-03T18:50:46.384 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:46 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:46.197+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2024-05-03T18:50:18.193786+0000 front 2024-05-03T18:50:18.193853+0000 (oldest deadline 2024-05-03T18:50:44.093727+0000) 2024-05-03T18:50:46.384 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:45 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:45.987+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:24.877883+0000 front 2024-05-03T18:49:24.877625+0000 (oldest deadline 2024-05-03T18:49:50.777881+0000) 2024-05-03T18:50:46.384 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:45 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:45.987+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:47.685409+0000 front 2024-05-03T18:49:47.685527+0000 (oldest deadline 2024-05-03T18:50:11.785436+0000) 2024-05-03T18:50:46.384 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:45 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:45.987+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:57.687614+0000 front 2024-05-03T18:49:57.687504+0000 (oldest deadline 2024-05-03T18:50:24.688928+0000) 2024-05-03T18:50:46.385 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:45 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:45.987+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:08.191069+0000 front 2024-05-03T18:50:08.190956+0000 (oldest deadline 2024-05-03T18:50:34.091182+0000) 2024-05-03T18:50:46.385 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:45 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:45.987+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2024-05-03T18:50:18.193738+0000 front 2024-05-03T18:50:18.193572+0000 (oldest deadline 2024-05-03T18:50:42.893222+0000) 2024-05-03T18:50:46.385 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:46 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:46.025+0000 7f0d8fc00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T18:50:46.385 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:46 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:46.025+0000 7f0d8fc00700 -1 osd.8 126 *** Got signal Terminated *** 2024-05-03T18:50:46.385 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:46 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:46.025+0000 7f0d8fc00700 -1 osd.8 126 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-03T18:50:46.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:46 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:46.262+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:26.972061+0000 front 2024-05-03T18:49:26.972448+0000 (oldest deadline 2024-05-03T18:49:50.472344+0000) 2024-05-03T18:50:46.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:46 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:46.262+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:37.476081+0000 front 2024-05-03T18:49:37.476172+0000 (oldest deadline 2024-05-03T18:50:01.576192+0000) 2024-05-03T18:50:46.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:46 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:46.262+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:49.079592+0000 front 2024-05-03T18:49:49.080117+0000 (oldest deadline 2024-05-03T18:50:10.779802+0000) 2024-05-03T18:50:46.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:46 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:46.262+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:59.985863+0000 front 2024-05-03T18:49:59.986111+0000 (oldest deadline 2024-05-03T18:50:22.285990+0000) 2024-05-03T18:50:46.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:46 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:46.262+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:05.688218+0000 front 2024-05-03T18:50:05.688162+0000 (oldest deadline 2024-05-03T18:50:36.889315+0000) 2024-05-03T18:50:46.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:46 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:46.262+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2024-05-03T18:50:17.391461+0000 front 2024-05-03T18:50:17.391830+0000 (oldest deadline 2024-05-03T18:50:46.192419+0000) 2024-05-03T18:50:46.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:46 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:46.364+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:27.608012+0000 front 2024-05-03T18:49:27.608078+0000 (oldest deadline 2024-05-03T18:49:53.508290+0000) 2024-05-03T18:50:46.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:46 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:46.364+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:38.811037+0000 front 2024-05-03T18:49:38.811219+0000 (oldest deadline 2024-05-03T18:50:01.611951+0000) 2024-05-03T18:50:46.736 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:46 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:46.364+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:46.215926+0000 front 2024-05-03T18:49:46.216076+0000 (oldest deadline 2024-05-03T18:50:12.116128+0000) 2024-05-03T18:50:46.736 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:46 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:46.364+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:56.719409+0000 front 2024-05-03T18:49:56.719531+0000 (oldest deadline 2024-05-03T18:50:21.419652+0000) 2024-05-03T18:50:46.736 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:46 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:46.364+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:08.223552+0000 front 2024-05-03T18:50:08.224016+0000 (oldest deadline 2024-05-03T18:50:33.925215+0000) 2024-05-03T18:50:46.736 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:46 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:46.364+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2024-05-03T18:50:20.327897+0000 front 2024-05-03T18:50:20.327850+0000 (oldest deadline 2024-05-03T18:50:43.227764+0000) 2024-05-03T18:50:47.200 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:46 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:46.941+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:24.877883+0000 front 2024-05-03T18:49:24.877625+0000 (oldest deadline 2024-05-03T18:49:50.777881+0000) 2024-05-03T18:50:47.201 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:46 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:46.941+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:47.685409+0000 front 2024-05-03T18:49:47.685527+0000 (oldest deadline 2024-05-03T18:50:11.785436+0000) 2024-05-03T18:50:47.201 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:46 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:46.941+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:57.687614+0000 front 2024-05-03T18:49:57.687504+0000 (oldest deadline 2024-05-03T18:50:24.688928+0000) 2024-05-03T18:50:47.201 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:46 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:46.941+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:08.191069+0000 front 2024-05-03T18:50:08.190956+0000 (oldest deadline 2024-05-03T18:50:34.091182+0000) 2024-05-03T18:50:47.201 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:46 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:46.941+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2024-05-03T18:50:18.193738+0000 front 2024-05-03T18:50:18.193572+0000 (oldest deadline 2024-05-03T18:50:42.893222+0000) 2024-05-03T18:50:47.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:47 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:47.227+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:26.972061+0000 front 2024-05-03T18:49:26.972448+0000 (oldest deadline 2024-05-03T18:49:50.472344+0000) 2024-05-03T18:50:47.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:47 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:47.227+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:37.476081+0000 front 2024-05-03T18:49:37.476172+0000 (oldest deadline 2024-05-03T18:50:01.576192+0000) 2024-05-03T18:50:47.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:47 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:47.227+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:49.079592+0000 front 2024-05-03T18:49:49.080117+0000 (oldest deadline 2024-05-03T18:50:10.779802+0000) 2024-05-03T18:50:47.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:47 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:47.227+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:59.985863+0000 front 2024-05-03T18:49:59.986111+0000 (oldest deadline 2024-05-03T18:50:22.285990+0000) 2024-05-03T18:50:47.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:47 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:47.227+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:05.688218+0000 front 2024-05-03T18:50:05.688162+0000 (oldest deadline 2024-05-03T18:50:36.889315+0000) 2024-05-03T18:50:47.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:47 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:47.227+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2024-05-03T18:50:17.391461+0000 front 2024-05-03T18:50:17.391830+0000 (oldest deadline 2024-05-03T18:50:46.192419+0000) 2024-05-03T18:50:47.633 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:47 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:47.199+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:25.480913+0000 front 2024-05-03T18:49:25.480403+0000 (oldest deadline 2024-05-03T18:49:50.180152+0000) 2024-05-03T18:50:47.633 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:47 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:47.199+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:34.882184+0000 front 2024-05-03T18:49:34.882683+0000 (oldest deadline 2024-05-03T18:50:05.483250+0000) 2024-05-03T18:50:47.633 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:47 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:47.199+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:45.484981+0000 front 2024-05-03T18:49:45.484933+0000 (oldest deadline 2024-05-03T18:50:14.885581+0000) 2024-05-03T18:50:47.634 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:47 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:47.199+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:08.290430+0000 front 2024-05-03T18:50:08.290390+0000 (oldest deadline 2024-05-03T18:50:31.790398+0000) 2024-05-03T18:50:47.634 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:47 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:47.199+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2024-05-03T18:50:18.193786+0000 front 2024-05-03T18:50:18.193853+0000 (oldest deadline 2024-05-03T18:50:44.093727+0000) 2024-05-03T18:50:47.736 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:47 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:47.404+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:27.608012+0000 front 2024-05-03T18:49:27.608078+0000 (oldest deadline 2024-05-03T18:49:53.508290+0000) 2024-05-03T18:50:47.736 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:47 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:47.404+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:38.811037+0000 front 2024-05-03T18:49:38.811219+0000 (oldest deadline 2024-05-03T18:50:01.611951+0000) 2024-05-03T18:50:47.736 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:47 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:47.404+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:46.215926+0000 front 2024-05-03T18:49:46.216076+0000 (oldest deadline 2024-05-03T18:50:12.116128+0000) 2024-05-03T18:50:47.736 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:47 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:47.404+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:56.719409+0000 front 2024-05-03T18:49:56.719531+0000 (oldest deadline 2024-05-03T18:50:21.419652+0000) 2024-05-03T18:50:47.737 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:47 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:47.404+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:08.223552+0000 front 2024-05-03T18:50:08.224016+0000 (oldest deadline 2024-05-03T18:50:33.925215+0000) 2024-05-03T18:50:47.737 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:47 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:47.404+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2024-05-03T18:50:20.327897+0000 front 2024-05-03T18:50:20.327850+0000 (oldest deadline 2024-05-03T18:50:43.227764+0000) 2024-05-03T18:50:48.244 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:47 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:47.981+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:24.877883+0000 front 2024-05-03T18:49:24.877625+0000 (oldest deadline 2024-05-03T18:49:50.777881+0000) 2024-05-03T18:50:48.244 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:47 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:47.981+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:47.685409+0000 front 2024-05-03T18:49:47.685527+0000 (oldest deadline 2024-05-03T18:50:11.785436+0000) 2024-05-03T18:50:48.245 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:47 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:47.981+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:57.687614+0000 front 2024-05-03T18:49:57.687504+0000 (oldest deadline 2024-05-03T18:50:24.688928+0000) 2024-05-03T18:50:48.245 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:47 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:47.981+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:08.191069+0000 front 2024-05-03T18:50:08.190956+0000 (oldest deadline 2024-05-03T18:50:34.091182+0000) 2024-05-03T18:50:48.245 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:47 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:47.981+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2024-05-03T18:50:18.193738+0000 front 2024-05-03T18:50:18.193572+0000 (oldest deadline 2024-05-03T18:50:42.893222+0000) 2024-05-03T18:50:48.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:48 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:48.219+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:26.972061+0000 front 2024-05-03T18:49:26.972448+0000 (oldest deadline 2024-05-03T18:49:50.472344+0000) 2024-05-03T18:50:48.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:48 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:48.219+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:37.476081+0000 front 2024-05-03T18:49:37.476172+0000 (oldest deadline 2024-05-03T18:50:01.576192+0000) 2024-05-03T18:50:48.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:48 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:48.219+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:49.079592+0000 front 2024-05-03T18:49:49.080117+0000 (oldest deadline 2024-05-03T18:50:10.779802+0000) 2024-05-03T18:50:48.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:48 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:48.219+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:59.985863+0000 front 2024-05-03T18:49:59.986111+0000 (oldest deadline 2024-05-03T18:50:22.285990+0000) 2024-05-03T18:50:48.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:48 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:48.219+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:05.688218+0000 front 2024-05-03T18:50:05.688162+0000 (oldest deadline 2024-05-03T18:50:36.889315+0000) 2024-05-03T18:50:48.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:48 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:48.219+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2024-05-03T18:50:17.391461+0000 front 2024-05-03T18:50:17.391830+0000 (oldest deadline 2024-05-03T18:50:46.192419+0000) 2024-05-03T18:50:48.633 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:48 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:48.243+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:25.480913+0000 front 2024-05-03T18:49:25.480403+0000 (oldest deadline 2024-05-03T18:49:50.180152+0000) 2024-05-03T18:50:48.633 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:48 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:48.243+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:34.882184+0000 front 2024-05-03T18:49:34.882683+0000 (oldest deadline 2024-05-03T18:50:05.483250+0000) 2024-05-03T18:50:48.633 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:48 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:48.243+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:45.484981+0000 front 2024-05-03T18:49:45.484933+0000 (oldest deadline 2024-05-03T18:50:14.885581+0000) 2024-05-03T18:50:48.634 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:48 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:48.243+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:08.290430+0000 front 2024-05-03T18:50:08.290390+0000 (oldest deadline 2024-05-03T18:50:31.790398+0000) 2024-05-03T18:50:48.634 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:48 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:48.243+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2024-05-03T18:50:18.193786+0000 front 2024-05-03T18:50:18.193853+0000 (oldest deadline 2024-05-03T18:50:44.093727+0000) 2024-05-03T18:50:48.736 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:48 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:48.445+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:27.608012+0000 front 2024-05-03T18:49:27.608078+0000 (oldest deadline 2024-05-03T18:49:53.508290+0000) 2024-05-03T18:50:48.736 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:48 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:48.445+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:38.811037+0000 front 2024-05-03T18:49:38.811219+0000 (oldest deadline 2024-05-03T18:50:01.611951+0000) 2024-05-03T18:50:48.737 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:48 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:48.445+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:46.215926+0000 front 2024-05-03T18:49:46.216076+0000 (oldest deadline 2024-05-03T18:50:12.116128+0000) 2024-05-03T18:50:48.737 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:48 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:48.445+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:56.719409+0000 front 2024-05-03T18:49:56.719531+0000 (oldest deadline 2024-05-03T18:50:21.419652+0000) 2024-05-03T18:50:48.737 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:48 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:48.445+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:08.223552+0000 front 2024-05-03T18:50:08.224016+0000 (oldest deadline 2024-05-03T18:50:33.925215+0000) 2024-05-03T18:50:48.737 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:48 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:48.445+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2024-05-03T18:50:20.327897+0000 front 2024-05-03T18:50:20.327850+0000 (oldest deadline 2024-05-03T18:50:43.227764+0000) 2024-05-03T18:50:49.383 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:49 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:49.014+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:24.877883+0000 front 2024-05-03T18:49:24.877625+0000 (oldest deadline 2024-05-03T18:49:50.777881+0000) 2024-05-03T18:50:49.384 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:49 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:49.014+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:47.685409+0000 front 2024-05-03T18:49:47.685527+0000 (oldest deadline 2024-05-03T18:50:11.785436+0000) 2024-05-03T18:50:49.384 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:49 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:49.014+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:57.687614+0000 front 2024-05-03T18:49:57.687504+0000 (oldest deadline 2024-05-03T18:50:24.688928+0000) 2024-05-03T18:50:49.384 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:49 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:49.014+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:08.191069+0000 front 2024-05-03T18:50:08.190956+0000 (oldest deadline 2024-05-03T18:50:34.091182+0000) 2024-05-03T18:50:49.384 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:49 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:49.014+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2024-05-03T18:50:18.193738+0000 front 2024-05-03T18:50:18.193572+0000 (oldest deadline 2024-05-03T18:50:42.893222+0000) 2024-05-03T18:50:49.384 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:49 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:49.226+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:25.480913+0000 front 2024-05-03T18:49:25.480403+0000 (oldest deadline 2024-05-03T18:49:50.180152+0000) 2024-05-03T18:50:49.384 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:49 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:49.226+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:34.882184+0000 front 2024-05-03T18:49:34.882683+0000 (oldest deadline 2024-05-03T18:50:05.483250+0000) 2024-05-03T18:50:49.384 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:49 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:49.226+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:45.484981+0000 front 2024-05-03T18:49:45.484933+0000 (oldest deadline 2024-05-03T18:50:14.885581+0000) 2024-05-03T18:50:49.384 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:49 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:49.226+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:08.290430+0000 front 2024-05-03T18:50:08.290390+0000 (oldest deadline 2024-05-03T18:50:31.790398+0000) 2024-05-03T18:50:49.384 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:49 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:49.226+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2024-05-03T18:50:18.193786+0000 front 2024-05-03T18:50:18.193853+0000 (oldest deadline 2024-05-03T18:50:44.093727+0000) 2024-05-03T18:50:49.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:49 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:49.170+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:26.972061+0000 front 2024-05-03T18:49:26.972448+0000 (oldest deadline 2024-05-03T18:49:50.472344+0000) 2024-05-03T18:50:49.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:49 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:49.170+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:37.476081+0000 front 2024-05-03T18:49:37.476172+0000 (oldest deadline 2024-05-03T18:50:01.576192+0000) 2024-05-03T18:50:49.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:49 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:49.170+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:49.079592+0000 front 2024-05-03T18:49:49.080117+0000 (oldest deadline 2024-05-03T18:50:10.779802+0000) 2024-05-03T18:50:49.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:49 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:49.170+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:59.985863+0000 front 2024-05-03T18:49:59.986111+0000 (oldest deadline 2024-05-03T18:50:22.285990+0000) 2024-05-03T18:50:49.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:49 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:49.170+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:05.688218+0000 front 2024-05-03T18:50:05.688162+0000 (oldest deadline 2024-05-03T18:50:36.889315+0000) 2024-05-03T18:50:49.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:49 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:49.170+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2024-05-03T18:50:17.391461+0000 front 2024-05-03T18:50:17.391830+0000 (oldest deadline 2024-05-03T18:50:46.192419+0000) 2024-05-03T18:50:49.985 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:49 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:49.488+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:27.608012+0000 front 2024-05-03T18:49:27.608078+0000 (oldest deadline 2024-05-03T18:49:53.508290+0000) 2024-05-03T18:50:49.985 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:49 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:49.488+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:38.811037+0000 front 2024-05-03T18:49:38.811219+0000 (oldest deadline 2024-05-03T18:50:01.611951+0000) 2024-05-03T18:50:49.986 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:49 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:49.488+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:46.215926+0000 front 2024-05-03T18:49:46.216076+0000 (oldest deadline 2024-05-03T18:50:12.116128+0000) 2024-05-03T18:50:49.986 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:49 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:49.488+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:56.719409+0000 front 2024-05-03T18:49:56.719531+0000 (oldest deadline 2024-05-03T18:50:21.419652+0000) 2024-05-03T18:50:49.986 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:49 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:49.488+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:08.223552+0000 front 2024-05-03T18:50:08.224016+0000 (oldest deadline 2024-05-03T18:50:33.925215+0000) 2024-05-03T18:50:49.986 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:49 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:49.488+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2024-05-03T18:50:20.327897+0000 front 2024-05-03T18:50:20.327850+0000 (oldest deadline 2024-05-03T18:50:43.227764+0000) 2024-05-03T18:50:50.384 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:50 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:50.184+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:25.480913+0000 front 2024-05-03T18:49:25.480403+0000 (oldest deadline 2024-05-03T18:49:50.180152+0000) 2024-05-03T18:50:50.384 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:50 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:50.184+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:34.882184+0000 front 2024-05-03T18:49:34.882683+0000 (oldest deadline 2024-05-03T18:50:05.483250+0000) 2024-05-03T18:50:50.384 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:50 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:50.184+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:45.484981+0000 front 2024-05-03T18:49:45.484933+0000 (oldest deadline 2024-05-03T18:50:14.885581+0000) 2024-05-03T18:50:50.384 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:50 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:50.184+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:08.290430+0000 front 2024-05-03T18:50:08.290390+0000 (oldest deadline 2024-05-03T18:50:31.790398+0000) 2024-05-03T18:50:50.384 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:50 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:50.184+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2024-05-03T18:50:18.193786+0000 front 2024-05-03T18:50:18.193853+0000 (oldest deadline 2024-05-03T18:50:44.093727+0000) 2024-05-03T18:50:50.385 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:50 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:50.010+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:24.877883+0000 front 2024-05-03T18:49:24.877625+0000 (oldest deadline 2024-05-03T18:49:50.777881+0000) 2024-05-03T18:50:50.385 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:50 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:50.010+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:47.685409+0000 front 2024-05-03T18:49:47.685527+0000 (oldest deadline 2024-05-03T18:50:11.785436+0000) 2024-05-03T18:50:50.385 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:50 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:50.010+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:57.687614+0000 front 2024-05-03T18:49:57.687504+0000 (oldest deadline 2024-05-03T18:50:24.688928+0000) 2024-05-03T18:50:50.385 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:50 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:50.010+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:08.191069+0000 front 2024-05-03T18:50:08.190956+0000 (oldest deadline 2024-05-03T18:50:34.091182+0000) 2024-05-03T18:50:50.385 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:50 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8[51079]: 2024-05-03T18:50:50.010+0000 7f0d8b400700 -1 osd.8 126 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2024-05-03T18:50:18.193738+0000 front 2024-05-03T18:50:18.193572+0000 (oldest deadline 2024-05-03T18:50:42.893222+0000) 2024-05-03T18:50:50.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:50 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:50.161+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:26.972061+0000 front 2024-05-03T18:49:26.972448+0000 (oldest deadline 2024-05-03T18:49:50.472344+0000) 2024-05-03T18:50:50.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:50 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:50.161+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:37.476081+0000 front 2024-05-03T18:49:37.476172+0000 (oldest deadline 2024-05-03T18:50:01.576192+0000) 2024-05-03T18:50:50.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:50 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:50.161+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:49.079592+0000 front 2024-05-03T18:49:49.080117+0000 (oldest deadline 2024-05-03T18:50:10.779802+0000) 2024-05-03T18:50:50.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:50 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:50.161+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:59.985863+0000 front 2024-05-03T18:49:59.986111+0000 (oldest deadline 2024-05-03T18:50:22.285990+0000) 2024-05-03T18:50:50.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:50 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:50.161+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:05.688218+0000 front 2024-05-03T18:50:05.688162+0000 (oldest deadline 2024-05-03T18:50:36.889315+0000) 2024-05-03T18:50:50.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:50 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:50.161+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2024-05-03T18:50:17.391461+0000 front 2024-05-03T18:50:17.391830+0000 (oldest deadline 2024-05-03T18:50:46.192419+0000) 2024-05-03T18:50:50.985 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:50 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:50.505+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:27.608012+0000 front 2024-05-03T18:49:27.608078+0000 (oldest deadline 2024-05-03T18:49:53.508290+0000) 2024-05-03T18:50:50.985 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:50 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:50.505+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:38.811037+0000 front 2024-05-03T18:49:38.811219+0000 (oldest deadline 2024-05-03T18:50:01.611951+0000) 2024-05-03T18:50:50.985 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:50 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:50.505+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:46.215926+0000 front 2024-05-03T18:49:46.216076+0000 (oldest deadline 2024-05-03T18:50:12.116128+0000) 2024-05-03T18:50:50.985 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:50 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:50.505+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:56.719409+0000 front 2024-05-03T18:49:56.719531+0000 (oldest deadline 2024-05-03T18:50:21.419652+0000) 2024-05-03T18:50:50.986 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:50 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:50.505+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:08.223552+0000 front 2024-05-03T18:50:08.224016+0000 (oldest deadline 2024-05-03T18:50:33.925215+0000) 2024-05-03T18:50:50.986 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:50 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:50.505+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2024-05-03T18:50:20.327897+0000 front 2024-05-03T18:50:20.327850+0000 (oldest deadline 2024-05-03T18:50:43.227764+0000) 2024-05-03T18:50:51.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:51 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:51.197+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:26.972061+0000 front 2024-05-03T18:49:26.972448+0000 (oldest deadline 2024-05-03T18:49:50.472344+0000) 2024-05-03T18:50:51.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:51 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:51.197+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:37.476081+0000 front 2024-05-03T18:49:37.476172+0000 (oldest deadline 2024-05-03T18:50:01.576192+0000) 2024-05-03T18:50:51.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:51 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:51.197+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:49.079592+0000 front 2024-05-03T18:49:49.080117+0000 (oldest deadline 2024-05-03T18:50:10.779802+0000) 2024-05-03T18:50:51.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:51 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:51.197+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:59.985863+0000 front 2024-05-03T18:49:59.986111+0000 (oldest deadline 2024-05-03T18:50:22.285990+0000) 2024-05-03T18:50:51.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:51 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:51.197+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:05.688218+0000 front 2024-05-03T18:50:05.688162+0000 (oldest deadline 2024-05-03T18:50:36.889315+0000) 2024-05-03T18:50:51.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:51 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:51.197+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2024-05-03T18:50:17.391461+0000 front 2024-05-03T18:50:17.391830+0000 (oldest deadline 2024-05-03T18:50:46.192419+0000) 2024-05-03T18:50:51.633 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:51 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:51.232+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:25.480913+0000 front 2024-05-03T18:49:25.480403+0000 (oldest deadline 2024-05-03T18:49:50.180152+0000) 2024-05-03T18:50:51.634 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:51 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:51.232+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:34.882184+0000 front 2024-05-03T18:49:34.882683+0000 (oldest deadline 2024-05-03T18:50:05.483250+0000) 2024-05-03T18:50:51.634 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:51 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:51.232+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:45.484981+0000 front 2024-05-03T18:49:45.484933+0000 (oldest deadline 2024-05-03T18:50:14.885581+0000) 2024-05-03T18:50:51.634 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:51 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:51.232+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:08.290430+0000 front 2024-05-03T18:50:08.290390+0000 (oldest deadline 2024-05-03T18:50:31.790398+0000) 2024-05-03T18:50:51.634 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:51 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:51.232+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2024-05-03T18:50:18.193786+0000 front 2024-05-03T18:50:18.193853+0000 (oldest deadline 2024-05-03T18:50:44.093727+0000) 2024-05-03T18:50:51.985 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:51 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:51.550+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:27.608012+0000 front 2024-05-03T18:49:27.608078+0000 (oldest deadline 2024-05-03T18:49:53.508290+0000) 2024-05-03T18:50:51.985 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:51 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:51.550+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:38.811037+0000 front 2024-05-03T18:49:38.811219+0000 (oldest deadline 2024-05-03T18:50:01.611951+0000) 2024-05-03T18:50:51.985 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:51 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:51.550+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:46.215926+0000 front 2024-05-03T18:49:46.216076+0000 (oldest deadline 2024-05-03T18:50:12.116128+0000) 2024-05-03T18:50:51.985 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:51 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:51.550+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:56.719409+0000 front 2024-05-03T18:49:56.719531+0000 (oldest deadline 2024-05-03T18:50:21.419652+0000) 2024-05-03T18:50:51.985 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:51 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:51.550+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:08.223552+0000 front 2024-05-03T18:50:08.224016+0000 (oldest deadline 2024-05-03T18:50:33.925215+0000) 2024-05-03T18:50:51.986 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:51 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:51.550+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2024-05-03T18:50:20.327897+0000 front 2024-05-03T18:50:20.327850+0000 (oldest deadline 2024-05-03T18:50:43.227764+0000) 2024-05-03T18:50:52.255 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:52 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:52.209+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:25.480913+0000 front 2024-05-03T18:49:25.480403+0000 (oldest deadline 2024-05-03T18:49:50.180152+0000) 2024-05-03T18:50:52.255 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:52 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:52.209+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:34.882184+0000 front 2024-05-03T18:49:34.882683+0000 (oldest deadline 2024-05-03T18:50:05.483250+0000) 2024-05-03T18:50:52.255 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:52 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:52.209+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:45.484981+0000 front 2024-05-03T18:49:45.484933+0000 (oldest deadline 2024-05-03T18:50:14.885581+0000) 2024-05-03T18:50:52.255 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:52 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:52.209+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:08.290430+0000 front 2024-05-03T18:50:08.290390+0000 (oldest deadline 2024-05-03T18:50:31.790398+0000) 2024-05-03T18:50:52.255 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:52 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:52.209+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2024-05-03T18:50:18.193786+0000 front 2024-05-03T18:50:18.193853+0000 (oldest deadline 2024-05-03T18:50:44.093727+0000) 2024-05-03T18:50:52.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:52 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:52.205+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:26.972061+0000 front 2024-05-03T18:49:26.972448+0000 (oldest deadline 2024-05-03T18:49:50.472344+0000) 2024-05-03T18:50:52.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:52 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:52.205+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:37.476081+0000 front 2024-05-03T18:49:37.476172+0000 (oldest deadline 2024-05-03T18:50:01.576192+0000) 2024-05-03T18:50:52.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:52 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:52.205+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:49.079592+0000 front 2024-05-03T18:49:49.080117+0000 (oldest deadline 2024-05-03T18:50:10.779802+0000) 2024-05-03T18:50:52.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:52 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:52.205+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:59.985863+0000 front 2024-05-03T18:49:59.986111+0000 (oldest deadline 2024-05-03T18:50:22.285990+0000) 2024-05-03T18:50:52.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:52 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:52.205+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:05.688218+0000 front 2024-05-03T18:50:05.688162+0000 (oldest deadline 2024-05-03T18:50:36.889315+0000) 2024-05-03T18:50:52.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:52 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:52.205+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2024-05-03T18:50:17.391461+0000 front 2024-05-03T18:50:17.391830+0000 (oldest deadline 2024-05-03T18:50:46.192419+0000) 2024-05-03T18:50:52.549 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:52 smithi150 podman[72156]: 2024-05-03 18:50:52.254185889 +0000 UTC m=+6.346371177 container died 16783a1beef8a193c2db551480026260f8f8877c119ef9d460eb3a77616f4102 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-05-03T18:50:52.985 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:52 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:52.600+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:27.608012+0000 front 2024-05-03T18:49:27.608078+0000 (oldest deadline 2024-05-03T18:49:53.508290+0000) 2024-05-03T18:50:52.985 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:52 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:52.600+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:38.811037+0000 front 2024-05-03T18:49:38.811219+0000 (oldest deadline 2024-05-03T18:50:01.611951+0000) 2024-05-03T18:50:52.985 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:52 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:52.600+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:46.215926+0000 front 2024-05-03T18:49:46.216076+0000 (oldest deadline 2024-05-03T18:50:12.116128+0000) 2024-05-03T18:50:52.986 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:52 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:52.600+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:56.719409+0000 front 2024-05-03T18:49:56.719531+0000 (oldest deadline 2024-05-03T18:50:21.419652+0000) 2024-05-03T18:50:52.986 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:52 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:52.600+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:08.223552+0000 front 2024-05-03T18:50:08.224016+0000 (oldest deadline 2024-05-03T18:50:33.925215+0000) 2024-05-03T18:50:52.986 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:52 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:52.600+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2024-05-03T18:50:20.327897+0000 front 2024-05-03T18:50:20.327850+0000 (oldest deadline 2024-05-03T18:50:43.227764+0000) 2024-05-03T18:50:53.133 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:52 smithi150 podman[72156]: 2024-05-03 18:50:52.809992969 +0000 UTC m=+6.902178318 container cleanup 16783a1beef8a193c2db551480026260f8f8877c119ef9d460eb3a77616f4102 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-03T18:50:53.133 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:52 smithi150 bash[72156]: ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8 2024-05-03T18:50:53.133 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:53 smithi150 podman[72184]: 2024-05-03 18:50:53.052821844 +0000 UTC m=+0.781911159 container remove 16783a1beef8a193c2db551480026260f8f8877c119ef9d460eb3a77616f4102 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-05-03T18:50:53.546 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:53 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:53.226+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:26.972061+0000 front 2024-05-03T18:49:26.972448+0000 (oldest deadline 2024-05-03T18:49:50.472344+0000) 2024-05-03T18:50:53.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:53 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:53.226+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:37.476081+0000 front 2024-05-03T18:49:37.476172+0000 (oldest deadline 2024-05-03T18:50:01.576192+0000) 2024-05-03T18:50:53.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:53 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:53.226+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:49.079592+0000 front 2024-05-03T18:49:49.080117+0000 (oldest deadline 2024-05-03T18:50:10.779802+0000) 2024-05-03T18:50:53.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:53 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:53.226+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:59.985863+0000 front 2024-05-03T18:49:59.986111+0000 (oldest deadline 2024-05-03T18:50:22.285990+0000) 2024-05-03T18:50:53.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:53 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:53.226+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:05.688218+0000 front 2024-05-03T18:50:05.688162+0000 (oldest deadline 2024-05-03T18:50:36.889315+0000) 2024-05-03T18:50:53.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:53 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:53.226+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2024-05-03T18:50:17.391461+0000 front 2024-05-03T18:50:17.391830+0000 (oldest deadline 2024-05-03T18:50:46.192419+0000) 2024-05-03T18:50:53.633 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:53 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:53.185+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:25.480913+0000 front 2024-05-03T18:49:25.480403+0000 (oldest deadline 2024-05-03T18:49:50.180152+0000) 2024-05-03T18:50:53.633 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:53 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:53.185+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:34.882184+0000 front 2024-05-03T18:49:34.882683+0000 (oldest deadline 2024-05-03T18:50:05.483250+0000) 2024-05-03T18:50:53.633 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:53 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:53.185+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:45.484981+0000 front 2024-05-03T18:49:45.484933+0000 (oldest deadline 2024-05-03T18:50:14.885581+0000) 2024-05-03T18:50:53.633 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:53 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:53.185+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:08.290430+0000 front 2024-05-03T18:50:08.290390+0000 (oldest deadline 2024-05-03T18:50:31.790398+0000) 2024-05-03T18:50:53.633 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:53 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:53.185+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2024-05-03T18:50:18.193786+0000 front 2024-05-03T18:50:18.193853+0000 (oldest deadline 2024-05-03T18:50:44.093727+0000) 2024-05-03T18:50:53.985 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:53 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:53.578+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:27.608012+0000 front 2024-05-03T18:49:27.608078+0000 (oldest deadline 2024-05-03T18:49:53.508290+0000) 2024-05-03T18:50:53.985 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:53 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:53.578+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:38.811037+0000 front 2024-05-03T18:49:38.811219+0000 (oldest deadline 2024-05-03T18:50:01.611951+0000) 2024-05-03T18:50:53.985 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:53 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:53.578+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:46.215926+0000 front 2024-05-03T18:49:46.216076+0000 (oldest deadline 2024-05-03T18:50:12.116128+0000) 2024-05-03T18:50:53.985 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:53 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:53.578+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:56.719409+0000 front 2024-05-03T18:49:56.719531+0000 (oldest deadline 2024-05-03T18:50:21.419652+0000) 2024-05-03T18:50:53.986 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:53 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:53.578+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:08.223552+0000 front 2024-05-03T18:50:08.224016+0000 (oldest deadline 2024-05-03T18:50:33.925215+0000) 2024-05-03T18:50:53.986 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:53 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:53.578+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2024-05-03T18:50:20.327897+0000 front 2024-05-03T18:50:20.327850+0000 (oldest deadline 2024-05-03T18:50:43.227764+0000) 2024-05-03T18:50:54.133 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:53 smithi150 podman[72269]: 2024-05-03 18:50:53.762021189 +0000 UTC m=+0.033396230 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T18:50:54.133 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:53 smithi150 podman[72269]: 2024-05-03 18:50:53.887384993 +0000 UTC m=+0.158760075 container create 57f805b881a2c2f5c37c4d371f4cca90eee8d26b1b8179a049e9e8bb4e2e8615 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240430, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-05-03T18:50:54.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:54 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:54.264+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:26.972061+0000 front 2024-05-03T18:49:26.972448+0000 (oldest deadline 2024-05-03T18:49:50.472344+0000) 2024-05-03T18:50:54.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:54 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:54.264+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:37.476081+0000 front 2024-05-03T18:49:37.476172+0000 (oldest deadline 2024-05-03T18:50:01.576192+0000) 2024-05-03T18:50:54.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:54 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:54.264+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:49.079592+0000 front 2024-05-03T18:49:49.080117+0000 (oldest deadline 2024-05-03T18:50:10.779802+0000) 2024-05-03T18:50:54.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:54 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:54.265+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:59.985863+0000 front 2024-05-03T18:49:59.986111+0000 (oldest deadline 2024-05-03T18:50:22.285990+0000) 2024-05-03T18:50:54.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:54 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:54.265+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:05.688218+0000 front 2024-05-03T18:50:05.688162+0000 (oldest deadline 2024-05-03T18:50:36.889315+0000) 2024-05-03T18:50:54.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:54 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:54.265+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2024-05-03T18:50:17.391461+0000 front 2024-05-03T18:50:17.391830+0000 (oldest deadline 2024-05-03T18:50:46.192419+0000) 2024-05-03T18:50:54.628 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:54 smithi150 podman[72269]: 2024-05-03 18:50:54.157528788 +0000 UTC m=+0.428903939 container init 57f805b881a2c2f5c37c4d371f4cca90eee8d26b1b8179a049e9e8bb4e2e8615 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe) 2024-05-03T18:50:54.628 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:54 smithi150 podman[72269]: 2024-05-03 18:50:54.170293448 +0000 UTC m=+0.441668529 container start 57f805b881a2c2f5c37c4d371f4cca90eee8d26b1b8179a049e9e8bb4e2e8615 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-05-03T18:50:54.628 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:54 smithi150 podman[72269]: 2024-05-03 18:50:54.231277966 +0000 UTC m=+0.502653032 container attach 57f805b881a2c2f5c37c4d371f4cca90eee8d26b1b8179a049e9e8bb4e2e8615 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240430, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-05-03T18:50:54.628 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:54 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:54.209+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:25.480913+0000 front 2024-05-03T18:49:25.480403+0000 (oldest deadline 2024-05-03T18:49:50.180152+0000) 2024-05-03T18:50:54.628 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:54 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:54.209+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:34.882184+0000 front 2024-05-03T18:49:34.882683+0000 (oldest deadline 2024-05-03T18:50:05.483250+0000) 2024-05-03T18:50:54.628 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:54 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:54.209+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:45.484981+0000 front 2024-05-03T18:49:45.484933+0000 (oldest deadline 2024-05-03T18:50:14.885581+0000) 2024-05-03T18:50:54.628 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:54 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:54.209+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:08.290430+0000 front 2024-05-03T18:50:08.290390+0000 (oldest deadline 2024-05-03T18:50:31.790398+0000) 2024-05-03T18:50:54.628 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:54 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:54.209+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2024-05-03T18:50:18.193786+0000 front 2024-05-03T18:50:18.193853+0000 (oldest deadline 2024-05-03T18:50:44.093727+0000) 2024-05-03T18:50:54.883 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:54 smithi150 podman[72269]: 2024-05-03 18:50:54.627253153 +0000 UTC m=+0.898628271 container died 57f805b881a2c2f5c37c4d371f4cca90eee8d26b1b8179a049e9e8bb4e2e8615 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8-deactivate, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240430, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-03T18:50:54.985 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:54 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:54.556+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:27.608012+0000 front 2024-05-03T18:49:27.608078+0000 (oldest deadline 2024-05-03T18:49:53.508290+0000) 2024-05-03T18:50:54.985 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:54 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:54.556+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:38.811037+0000 front 2024-05-03T18:49:38.811219+0000 (oldest deadline 2024-05-03T18:50:01.611951+0000) 2024-05-03T18:50:54.985 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:54 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:54.556+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:46.215926+0000 front 2024-05-03T18:49:46.216076+0000 (oldest deadline 2024-05-03T18:50:12.116128+0000) 2024-05-03T18:50:54.985 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:54 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:54.556+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:56.719409+0000 front 2024-05-03T18:49:56.719531+0000 (oldest deadline 2024-05-03T18:50:21.419652+0000) 2024-05-03T18:50:54.985 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:54 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:54.556+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:08.223552+0000 front 2024-05-03T18:50:08.224016+0000 (oldest deadline 2024-05-03T18:50:33.925215+0000) 2024-05-03T18:50:54.986 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:54 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:54.556+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2024-05-03T18:50:20.327897+0000 front 2024-05-03T18:50:20.327850+0000 (oldest deadline 2024-05-03T18:50:43.227764+0000) 2024-05-03T18:50:55.380 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5703bba4-0978-11ef-bc97-c7b262605968@osd.8.service' 2024-05-03T18:50:55.600 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:55 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:55.262+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:26.972061+0000 front 2024-05-03T18:49:26.972448+0000 (oldest deadline 2024-05-03T18:49:50.472344+0000) 2024-05-03T18:50:55.600 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:55 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:55.262+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:37.476081+0000 front 2024-05-03T18:49:37.476172+0000 (oldest deadline 2024-05-03T18:50:01.576192+0000) 2024-05-03T18:50:55.600 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:55 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:55.262+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:49.079592+0000 front 2024-05-03T18:49:49.080117+0000 (oldest deadline 2024-05-03T18:50:10.779802+0000) 2024-05-03T18:50:55.600 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:55 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:55.262+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:59.985863+0000 front 2024-05-03T18:49:59.986111+0000 (oldest deadline 2024-05-03T18:50:22.285990+0000) 2024-05-03T18:50:55.601 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:55 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:55.262+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:05.688218+0000 front 2024-05-03T18:50:05.688162+0000 (oldest deadline 2024-05-03T18:50:36.889315+0000) 2024-05-03T18:50:55.601 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:55 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:55.262+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2024-05-03T18:50:17.391461+0000 front 2024-05-03T18:50:17.391830+0000 (oldest deadline 2024-05-03T18:50:46.192419+0000) 2024-05-03T18:50:55.633 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:55 smithi150 podman[72295]: 2024-05-03 18:50:55.334332608 +0000 UTC m=+0.687876809 container remove 57f805b881a2c2f5c37c4d371f4cca90eee8d26b1b8179a049e9e8bb4e2e8615 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-8-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-03T18:50:55.633 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:55 smithi150 systemd[1]: ceph-5703bba4-0978-11ef-bc97-c7b262605968@osd.8.service: Deactivated successfully. 2024-05-03T18:50:55.633 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:55 smithi150 systemd[1]: Stopped Ceph osd.8 for 5703bba4-0978-11ef-bc97-c7b262605968. 2024-05-03T18:50:55.633 INFO:journalctl@ceph.osd.8.smithi150.stdout:May 03 18:50:55 smithi150 systemd[1]: ceph-5703bba4-0978-11ef-bc97-c7b262605968@osd.8.service: Consumed 13min 7.412s CPU time. 2024-05-03T18:50:55.633 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:55 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:55.245+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:25.480913+0000 front 2024-05-03T18:49:25.480403+0000 (oldest deadline 2024-05-03T18:49:50.180152+0000) 2024-05-03T18:50:55.634 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:55 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:55.245+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:34.882184+0000 front 2024-05-03T18:49:34.882683+0000 (oldest deadline 2024-05-03T18:50:05.483250+0000) 2024-05-03T18:50:55.634 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:55 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:55.245+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:45.484981+0000 front 2024-05-03T18:49:45.484933+0000 (oldest deadline 2024-05-03T18:50:14.885581+0000) 2024-05-03T18:50:55.634 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:55 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:55.245+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:08.290430+0000 front 2024-05-03T18:50:08.290390+0000 (oldest deadline 2024-05-03T18:50:31.790398+0000) 2024-05-03T18:50:55.634 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:55 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:55.245+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2024-05-03T18:50:18.193786+0000 front 2024-05-03T18:50:18.193853+0000 (oldest deadline 2024-05-03T18:50:44.093727+0000) 2024-05-03T18:50:55.714 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T18:50:55.714 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-05-03T18:50:55.715 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-05-03T18:50:55.715 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-5703bba4-0978-11ef-bc97-c7b262605968@osd.9 2024-05-03T18:50:55.986 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:55 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:55.600+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:27.608012+0000 front 2024-05-03T18:49:27.608078+0000 (oldest deadline 2024-05-03T18:49:53.508290+0000) 2024-05-03T18:50:55.986 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:55 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:55.600+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:38.811037+0000 front 2024-05-03T18:49:38.811219+0000 (oldest deadline 2024-05-03T18:50:01.611951+0000) 2024-05-03T18:50:55.986 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:55 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:55.600+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:46.215926+0000 front 2024-05-03T18:49:46.216076+0000 (oldest deadline 2024-05-03T18:50:12.116128+0000) 2024-05-03T18:50:55.986 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:55 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:55.600+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:56.719409+0000 front 2024-05-03T18:49:56.719531+0000 (oldest deadline 2024-05-03T18:50:21.419652+0000) 2024-05-03T18:50:55.986 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:55 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:55.600+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:08.223552+0000 front 2024-05-03T18:50:08.224016+0000 (oldest deadline 2024-05-03T18:50:33.925215+0000) 2024-05-03T18:50:55.986 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:55 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:55.600+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2024-05-03T18:50:20.327897+0000 front 2024-05-03T18:50:20.327850+0000 (oldest deadline 2024-05-03T18:50:43.227764+0000) 2024-05-03T18:50:55.986 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:55 smithi033 systemd[1]: Stopping Ceph osd.9 for 5703bba4-0978-11ef-bc97-c7b262605968... 2024-05-03T18:50:56.634 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:56 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:56.273+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:25.480913+0000 front 2024-05-03T18:49:25.480403+0000 (oldest deadline 2024-05-03T18:49:50.180152+0000) 2024-05-03T18:50:56.634 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:56 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:56.273+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:34.882184+0000 front 2024-05-03T18:49:34.882683+0000 (oldest deadline 2024-05-03T18:50:05.483250+0000) 2024-05-03T18:50:56.634 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:56 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:56.273+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:45.484981+0000 front 2024-05-03T18:49:45.484933+0000 (oldest deadline 2024-05-03T18:50:14.885581+0000) 2024-05-03T18:50:56.634 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:56 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:56.273+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:08.290430+0000 front 2024-05-03T18:50:08.290390+0000 (oldest deadline 2024-05-03T18:50:31.790398+0000) 2024-05-03T18:50:56.634 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:56 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:56.273+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2024-05-03T18:50:18.193786+0000 front 2024-05-03T18:50:18.193853+0000 (oldest deadline 2024-05-03T18:50:44.093727+0000) 2024-05-03T18:50:56.634 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:56 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:56.273+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-05-03T18:50:29.995411+0000 front 2024-05-03T18:50:29.995188+0000 (oldest deadline 2024-05-03T18:50:55.294685+0000) 2024-05-03T18:50:56.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:56 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:56.325+0000 7f0fd9a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T18:50:56.736 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:56 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:56.325+0000 7f0fd9a00700 -1 osd.9 126 *** Got signal Terminated *** 2024-05-03T18:50:56.736 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:56 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:56.325+0000 7f0fd9a00700 -1 osd.9 126 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-03T18:50:56.736 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:56 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:56.571+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:27.608012+0000 front 2024-05-03T18:49:27.608078+0000 (oldest deadline 2024-05-03T18:49:53.508290+0000) 2024-05-03T18:50:56.736 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:56 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:56.571+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:38.811037+0000 front 2024-05-03T18:49:38.811219+0000 (oldest deadline 2024-05-03T18:50:01.611951+0000) 2024-05-03T18:50:56.736 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:56 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:56.571+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:46.215926+0000 front 2024-05-03T18:49:46.216076+0000 (oldest deadline 2024-05-03T18:50:12.116128+0000) 2024-05-03T18:50:56.736 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:56 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:56.571+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:56.719409+0000 front 2024-05-03T18:49:56.719531+0000 (oldest deadline 2024-05-03T18:50:21.419652+0000) 2024-05-03T18:50:56.736 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:56 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:56.571+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:08.223552+0000 front 2024-05-03T18:50:08.224016+0000 (oldest deadline 2024-05-03T18:50:33.925215+0000) 2024-05-03T18:50:56.736 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:56 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:56.571+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2024-05-03T18:50:20.327897+0000 front 2024-05-03T18:50:20.327850+0000 (oldest deadline 2024-05-03T18:50:43.227764+0000) 2024-05-03T18:50:56.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:56 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:56.296+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:26.972061+0000 front 2024-05-03T18:49:26.972448+0000 (oldest deadline 2024-05-03T18:49:50.472344+0000) 2024-05-03T18:50:56.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:56 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:56.296+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:37.476081+0000 front 2024-05-03T18:49:37.476172+0000 (oldest deadline 2024-05-03T18:50:01.576192+0000) 2024-05-03T18:50:56.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:56 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:56.296+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:49.079592+0000 front 2024-05-03T18:49:49.080117+0000 (oldest deadline 2024-05-03T18:50:10.779802+0000) 2024-05-03T18:50:56.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:56 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:56.296+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:59.985863+0000 front 2024-05-03T18:49:59.986111+0000 (oldest deadline 2024-05-03T18:50:22.285990+0000) 2024-05-03T18:50:56.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:56 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:56.296+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:05.688218+0000 front 2024-05-03T18:50:05.688162+0000 (oldest deadline 2024-05-03T18:50:36.889315+0000) 2024-05-03T18:50:56.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:56 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:56.296+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2024-05-03T18:50:17.391461+0000 front 2024-05-03T18:50:17.391830+0000 (oldest deadline 2024-05-03T18:50:46.192419+0000) 2024-05-03T18:50:56.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:56 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:56.296+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-05-03T18:50:26.193991+0000 front 2024-05-03T18:50:26.194137+0000 (oldest deadline 2024-05-03T18:50:56.194110+0000) 2024-05-03T18:50:57.634 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:57 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:57.310+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:25.480913+0000 front 2024-05-03T18:49:25.480403+0000 (oldest deadline 2024-05-03T18:49:50.180152+0000) 2024-05-03T18:50:57.634 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:57 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:57.310+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:34.882184+0000 front 2024-05-03T18:49:34.882683+0000 (oldest deadline 2024-05-03T18:50:05.483250+0000) 2024-05-03T18:50:57.634 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:57 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:57.310+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:45.484981+0000 front 2024-05-03T18:49:45.484933+0000 (oldest deadline 2024-05-03T18:50:14.885581+0000) 2024-05-03T18:50:57.634 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:57 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:57.310+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:08.290430+0000 front 2024-05-03T18:50:08.290390+0000 (oldest deadline 2024-05-03T18:50:31.790398+0000) 2024-05-03T18:50:57.634 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:57 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:57.310+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2024-05-03T18:50:18.193786+0000 front 2024-05-03T18:50:18.193853+0000 (oldest deadline 2024-05-03T18:50:44.093727+0000) 2024-05-03T18:50:57.634 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:57 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:57.310+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-05-03T18:50:29.995411+0000 front 2024-05-03T18:50:29.995188+0000 (oldest deadline 2024-05-03T18:50:55.294685+0000) 2024-05-03T18:50:57.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:57 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:57.345+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:26.972061+0000 front 2024-05-03T18:49:26.972448+0000 (oldest deadline 2024-05-03T18:49:50.472344+0000) 2024-05-03T18:50:57.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:57 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:57.345+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:37.476081+0000 front 2024-05-03T18:49:37.476172+0000 (oldest deadline 2024-05-03T18:50:01.576192+0000) 2024-05-03T18:50:57.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:57 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:57.345+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:49.079592+0000 front 2024-05-03T18:49:49.080117+0000 (oldest deadline 2024-05-03T18:50:10.779802+0000) 2024-05-03T18:50:57.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:57 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:57.345+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:59.985863+0000 front 2024-05-03T18:49:59.986111+0000 (oldest deadline 2024-05-03T18:50:22.285990+0000) 2024-05-03T18:50:57.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:57 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:57.345+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:05.688218+0000 front 2024-05-03T18:50:05.688162+0000 (oldest deadline 2024-05-03T18:50:36.889315+0000) 2024-05-03T18:50:57.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:57 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:57.345+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2024-05-03T18:50:17.391461+0000 front 2024-05-03T18:50:17.391830+0000 (oldest deadline 2024-05-03T18:50:46.192419+0000) 2024-05-03T18:50:57.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:57 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:57.345+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-05-03T18:50:26.193991+0000 front 2024-05-03T18:50:26.194137+0000 (oldest deadline 2024-05-03T18:50:56.194110+0000) 2024-05-03T18:50:57.985 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:57 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:57.556+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:27.608012+0000 front 2024-05-03T18:49:27.608078+0000 (oldest deadline 2024-05-03T18:49:53.508290+0000) 2024-05-03T18:50:57.985 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:57 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:57.556+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:38.811037+0000 front 2024-05-03T18:49:38.811219+0000 (oldest deadline 2024-05-03T18:50:01.611951+0000) 2024-05-03T18:50:57.986 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:57 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:57.556+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:46.215926+0000 front 2024-05-03T18:49:46.216076+0000 (oldest deadline 2024-05-03T18:50:12.116128+0000) 2024-05-03T18:50:57.986 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:57 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:57.556+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:56.719409+0000 front 2024-05-03T18:49:56.719531+0000 (oldest deadline 2024-05-03T18:50:21.419652+0000) 2024-05-03T18:50:57.986 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:57 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:57.556+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:08.223552+0000 front 2024-05-03T18:50:08.224016+0000 (oldest deadline 2024-05-03T18:50:33.925215+0000) 2024-05-03T18:50:57.986 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:57 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:57.556+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2024-05-03T18:50:20.327897+0000 front 2024-05-03T18:50:20.327850+0000 (oldest deadline 2024-05-03T18:50:43.227764+0000) 2024-05-03T18:50:58.633 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:58 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:58.316+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:25.480913+0000 front 2024-05-03T18:49:25.480403+0000 (oldest deadline 2024-05-03T18:49:50.180152+0000) 2024-05-03T18:50:58.634 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:58 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:58.316+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:34.882184+0000 front 2024-05-03T18:49:34.882683+0000 (oldest deadline 2024-05-03T18:50:05.483250+0000) 2024-05-03T18:50:58.634 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:58 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:58.316+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:45.484981+0000 front 2024-05-03T18:49:45.484933+0000 (oldest deadline 2024-05-03T18:50:14.885581+0000) 2024-05-03T18:50:58.634 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:58 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:58.316+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:08.290430+0000 front 2024-05-03T18:50:08.290390+0000 (oldest deadline 2024-05-03T18:50:31.790398+0000) 2024-05-03T18:50:58.634 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:58 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:58.316+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2024-05-03T18:50:18.193786+0000 front 2024-05-03T18:50:18.193853+0000 (oldest deadline 2024-05-03T18:50:44.093727+0000) 2024-05-03T18:50:58.634 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:58 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:58.316+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-05-03T18:50:29.995411+0000 front 2024-05-03T18:50:29.995188+0000 (oldest deadline 2024-05-03T18:50:55.294685+0000) 2024-05-03T18:50:58.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:58 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:58.324+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:26.972061+0000 front 2024-05-03T18:49:26.972448+0000 (oldest deadline 2024-05-03T18:49:50.472344+0000) 2024-05-03T18:50:58.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:58 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:58.324+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:37.476081+0000 front 2024-05-03T18:49:37.476172+0000 (oldest deadline 2024-05-03T18:50:01.576192+0000) 2024-05-03T18:50:58.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:58 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:58.324+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:49.079592+0000 front 2024-05-03T18:49:49.080117+0000 (oldest deadline 2024-05-03T18:50:10.779802+0000) 2024-05-03T18:50:58.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:58 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:58.324+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:59.985863+0000 front 2024-05-03T18:49:59.986111+0000 (oldest deadline 2024-05-03T18:50:22.285990+0000) 2024-05-03T18:50:58.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:58 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:58.324+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:05.688218+0000 front 2024-05-03T18:50:05.688162+0000 (oldest deadline 2024-05-03T18:50:36.889315+0000) 2024-05-03T18:50:58.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:58 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:58.324+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2024-05-03T18:50:17.391461+0000 front 2024-05-03T18:50:17.391830+0000 (oldest deadline 2024-05-03T18:50:46.192419+0000) 2024-05-03T18:50:58.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:58 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:58.324+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-05-03T18:50:26.193991+0000 front 2024-05-03T18:50:26.194137+0000 (oldest deadline 2024-05-03T18:50:56.194110+0000) 2024-05-03T18:50:58.985 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:58 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:58.574+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:27.608012+0000 front 2024-05-03T18:49:27.608078+0000 (oldest deadline 2024-05-03T18:49:53.508290+0000) 2024-05-03T18:50:58.985 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:58 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:58.574+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:38.811037+0000 front 2024-05-03T18:49:38.811219+0000 (oldest deadline 2024-05-03T18:50:01.611951+0000) 2024-05-03T18:50:58.986 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:58 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:58.574+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:46.215926+0000 front 2024-05-03T18:49:46.216076+0000 (oldest deadline 2024-05-03T18:50:12.116128+0000) 2024-05-03T18:50:58.986 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:58 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:58.574+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:56.719409+0000 front 2024-05-03T18:49:56.719531+0000 (oldest deadline 2024-05-03T18:50:21.419652+0000) 2024-05-03T18:50:58.986 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:58 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:58.574+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:08.223552+0000 front 2024-05-03T18:50:08.224016+0000 (oldest deadline 2024-05-03T18:50:33.925215+0000) 2024-05-03T18:50:58.986 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:58 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:58.574+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2024-05-03T18:50:20.327897+0000 front 2024-05-03T18:50:20.327850+0000 (oldest deadline 2024-05-03T18:50:43.227764+0000) 2024-05-03T18:50:59.634 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:59 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:59.285+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:25.480913+0000 front 2024-05-03T18:49:25.480403+0000 (oldest deadline 2024-05-03T18:49:50.180152+0000) 2024-05-03T18:50:59.634 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:59 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:59.285+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:34.882184+0000 front 2024-05-03T18:49:34.882683+0000 (oldest deadline 2024-05-03T18:50:05.483250+0000) 2024-05-03T18:50:59.634 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:59 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:59.285+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:45.484981+0000 front 2024-05-03T18:49:45.484933+0000 (oldest deadline 2024-05-03T18:50:14.885581+0000) 2024-05-03T18:50:59.634 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:59 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:59.285+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:08.290430+0000 front 2024-05-03T18:50:08.290390+0000 (oldest deadline 2024-05-03T18:50:31.790398+0000) 2024-05-03T18:50:59.634 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:59 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:59.285+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2024-05-03T18:50:18.193786+0000 front 2024-05-03T18:50:18.193853+0000 (oldest deadline 2024-05-03T18:50:44.093727+0000) 2024-05-03T18:50:59.634 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:50:59 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:50:59.285+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-05-03T18:50:29.995411+0000 front 2024-05-03T18:50:29.995188+0000 (oldest deadline 2024-05-03T18:50:55.294685+0000) 2024-05-03T18:50:59.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:59 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:59.341+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:26.972061+0000 front 2024-05-03T18:49:26.972448+0000 (oldest deadline 2024-05-03T18:49:50.472344+0000) 2024-05-03T18:50:59.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:59 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:59.341+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:37.476081+0000 front 2024-05-03T18:49:37.476172+0000 (oldest deadline 2024-05-03T18:50:01.576192+0000) 2024-05-03T18:50:59.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:59 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:59.341+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:49.079592+0000 front 2024-05-03T18:49:49.080117+0000 (oldest deadline 2024-05-03T18:50:10.779802+0000) 2024-05-03T18:50:59.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:59 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:59.341+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:59.985863+0000 front 2024-05-03T18:49:59.986111+0000 (oldest deadline 2024-05-03T18:50:22.285990+0000) 2024-05-03T18:50:59.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:59 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:59.341+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:05.688218+0000 front 2024-05-03T18:50:05.688162+0000 (oldest deadline 2024-05-03T18:50:36.889315+0000) 2024-05-03T18:50:59.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:59 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:59.341+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2024-05-03T18:50:17.391461+0000 front 2024-05-03T18:50:17.391830+0000 (oldest deadline 2024-05-03T18:50:46.192419+0000) 2024-05-03T18:50:59.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:50:59 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:50:59.341+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-05-03T18:50:26.193991+0000 front 2024-05-03T18:50:26.194137+0000 (oldest deadline 2024-05-03T18:50:56.194110+0000) 2024-05-03T18:50:59.887 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:59 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:59.593+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:27.608012+0000 front 2024-05-03T18:49:27.608078+0000 (oldest deadline 2024-05-03T18:49:53.508290+0000) 2024-05-03T18:50:59.887 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:59 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:59.593+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:38.811037+0000 front 2024-05-03T18:49:38.811219+0000 (oldest deadline 2024-05-03T18:50:01.611951+0000) 2024-05-03T18:50:59.887 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:59 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:59.593+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:46.215926+0000 front 2024-05-03T18:49:46.216076+0000 (oldest deadline 2024-05-03T18:50:12.116128+0000) 2024-05-03T18:50:59.888 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:59 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:59.593+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:56.719409+0000 front 2024-05-03T18:49:56.719531+0000 (oldest deadline 2024-05-03T18:50:21.419652+0000) 2024-05-03T18:50:59.888 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:59 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:59.593+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:08.223552+0000 front 2024-05-03T18:50:08.224016+0000 (oldest deadline 2024-05-03T18:50:33.925215+0000) 2024-05-03T18:50:59.888 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:50:59 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:50:59.593+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2024-05-03T18:50:20.327897+0000 front 2024-05-03T18:50:20.327850+0000 (oldest deadline 2024-05-03T18:50:43.227764+0000) 2024-05-03T18:51:00.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:51:00 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:51:00.292+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:26.972061+0000 front 2024-05-03T18:49:26.972448+0000 (oldest deadline 2024-05-03T18:49:50.472344+0000) 2024-05-03T18:51:00.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:51:00 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:51:00.292+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:37.476081+0000 front 2024-05-03T18:49:37.476172+0000 (oldest deadline 2024-05-03T18:50:01.576192+0000) 2024-05-03T18:51:00.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:51:00 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:51:00.292+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:49.079592+0000 front 2024-05-03T18:49:49.080117+0000 (oldest deadline 2024-05-03T18:50:10.779802+0000) 2024-05-03T18:51:00.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:51:00 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:51:00.292+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:59.985863+0000 front 2024-05-03T18:49:59.986111+0000 (oldest deadline 2024-05-03T18:50:22.285990+0000) 2024-05-03T18:51:00.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:51:00 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:51:00.292+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:05.688218+0000 front 2024-05-03T18:50:05.688162+0000 (oldest deadline 2024-05-03T18:50:36.889315+0000) 2024-05-03T18:51:00.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:51:00 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:51:00.292+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2024-05-03T18:50:17.391461+0000 front 2024-05-03T18:50:17.391830+0000 (oldest deadline 2024-05-03T18:50:46.192419+0000) 2024-05-03T18:51:00.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:51:00 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:51:00.292+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-05-03T18:50:26.193991+0000 front 2024-05-03T18:50:26.194137+0000 (oldest deadline 2024-05-03T18:50:56.194110+0000) 2024-05-03T18:51:00.633 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:00 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:00.277+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:25.480913+0000 front 2024-05-03T18:49:25.480403+0000 (oldest deadline 2024-05-03T18:49:50.180152+0000) 2024-05-03T18:51:00.633 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:00 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:00.277+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:34.882184+0000 front 2024-05-03T18:49:34.882683+0000 (oldest deadline 2024-05-03T18:50:05.483250+0000) 2024-05-03T18:51:00.634 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:00 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:00.277+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:45.484981+0000 front 2024-05-03T18:49:45.484933+0000 (oldest deadline 2024-05-03T18:50:14.885581+0000) 2024-05-03T18:51:00.634 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:00 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:00.277+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:08.290430+0000 front 2024-05-03T18:50:08.290390+0000 (oldest deadline 2024-05-03T18:50:31.790398+0000) 2024-05-03T18:51:00.634 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:00 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:00.277+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2024-05-03T18:50:18.193786+0000 front 2024-05-03T18:50:18.193853+0000 (oldest deadline 2024-05-03T18:50:44.093727+0000) 2024-05-03T18:51:00.634 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:00 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:00.277+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-05-03T18:50:29.995411+0000 front 2024-05-03T18:50:29.995188+0000 (oldest deadline 2024-05-03T18:50:55.294685+0000) 2024-05-03T18:51:00.914 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:51:00 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:51:00.557+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:27.608012+0000 front 2024-05-03T18:49:27.608078+0000 (oldest deadline 2024-05-03T18:49:53.508290+0000) 2024-05-03T18:51:00.914 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:51:00 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:51:00.557+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:38.811037+0000 front 2024-05-03T18:49:38.811219+0000 (oldest deadline 2024-05-03T18:50:01.611951+0000) 2024-05-03T18:51:00.915 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:51:00 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:51:00.557+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:46.215926+0000 front 2024-05-03T18:49:46.216076+0000 (oldest deadline 2024-05-03T18:50:12.116128+0000) 2024-05-03T18:51:00.915 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:51:00 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:51:00.557+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:56.719409+0000 front 2024-05-03T18:49:56.719531+0000 (oldest deadline 2024-05-03T18:50:21.419652+0000) 2024-05-03T18:51:00.915 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:51:00 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:51:00.557+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:08.223552+0000 front 2024-05-03T18:50:08.224016+0000 (oldest deadline 2024-05-03T18:50:33.925215+0000) 2024-05-03T18:51:00.915 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:51:00 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9[63220]: 2024-05-03T18:51:00.557+0000 7f0fd0200700 -1 osd.9 126 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2024-05-03T18:50:20.327897+0000 front 2024-05-03T18:50:20.327850+0000 (oldest deadline 2024-05-03T18:50:43.227764+0000) 2024-05-03T18:51:01.633 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:01 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:01.304+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:25.480913+0000 front 2024-05-03T18:49:25.480403+0000 (oldest deadline 2024-05-03T18:49:50.180152+0000) 2024-05-03T18:51:01.634 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:01 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:01.304+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:34.882184+0000 front 2024-05-03T18:49:34.882683+0000 (oldest deadline 2024-05-03T18:50:05.483250+0000) 2024-05-03T18:51:01.634 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:01 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:01.304+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:45.484981+0000 front 2024-05-03T18:49:45.484933+0000 (oldest deadline 2024-05-03T18:50:14.885581+0000) 2024-05-03T18:51:01.634 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:01 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:01.304+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:08.290430+0000 front 2024-05-03T18:50:08.290390+0000 (oldest deadline 2024-05-03T18:50:31.790398+0000) 2024-05-03T18:51:01.634 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:01 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:01.304+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2024-05-03T18:50:18.193786+0000 front 2024-05-03T18:50:18.193853+0000 (oldest deadline 2024-05-03T18:50:44.093727+0000) 2024-05-03T18:51:01.634 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:01 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:01.304+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-05-03T18:50:29.995411+0000 front 2024-05-03T18:50:29.995188+0000 (oldest deadline 2024-05-03T18:50:55.294685+0000) 2024-05-03T18:51:01.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:51:01 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:51:01.300+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:26.972061+0000 front 2024-05-03T18:49:26.972448+0000 (oldest deadline 2024-05-03T18:49:50.472344+0000) 2024-05-03T18:51:01.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:51:01 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:51:01.300+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:37.476081+0000 front 2024-05-03T18:49:37.476172+0000 (oldest deadline 2024-05-03T18:50:01.576192+0000) 2024-05-03T18:51:01.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:51:01 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:51:01.300+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:49.079592+0000 front 2024-05-03T18:49:49.080117+0000 (oldest deadline 2024-05-03T18:50:10.779802+0000) 2024-05-03T18:51:01.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:51:01 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:51:01.300+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:59.985863+0000 front 2024-05-03T18:49:59.986111+0000 (oldest deadline 2024-05-03T18:50:22.285990+0000) 2024-05-03T18:51:01.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:51:01 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:51:01.300+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:05.688218+0000 front 2024-05-03T18:50:05.688162+0000 (oldest deadline 2024-05-03T18:50:36.889315+0000) 2024-05-03T18:51:01.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:51:01 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:51:01.300+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2024-05-03T18:50:17.391461+0000 front 2024-05-03T18:50:17.391830+0000 (oldest deadline 2024-05-03T18:50:46.192419+0000) 2024-05-03T18:51:01.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:51:01 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:51:01.300+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-05-03T18:50:26.193991+0000 front 2024-05-03T18:50:26.194137+0000 (oldest deadline 2024-05-03T18:50:56.194110+0000) 2024-05-03T18:51:02.633 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:02 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:02.343+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:25.480913+0000 front 2024-05-03T18:49:25.480403+0000 (oldest deadline 2024-05-03T18:49:50.180152+0000) 2024-05-03T18:51:02.634 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:02 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:02.343+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:34.882184+0000 front 2024-05-03T18:49:34.882683+0000 (oldest deadline 2024-05-03T18:50:05.483250+0000) 2024-05-03T18:51:02.634 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:02 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:02.343+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:45.484981+0000 front 2024-05-03T18:49:45.484933+0000 (oldest deadline 2024-05-03T18:50:14.885581+0000) 2024-05-03T18:51:02.634 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:02 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:02.343+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:08.290430+0000 front 2024-05-03T18:50:08.290390+0000 (oldest deadline 2024-05-03T18:50:31.790398+0000) 2024-05-03T18:51:02.634 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:02 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:02.343+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2024-05-03T18:50:18.193786+0000 front 2024-05-03T18:50:18.193853+0000 (oldest deadline 2024-05-03T18:50:44.093727+0000) 2024-05-03T18:51:02.634 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:02 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:02.343+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-05-03T18:50:29.995411+0000 front 2024-05-03T18:50:29.995188+0000 (oldest deadline 2024-05-03T18:50:55.294685+0000) 2024-05-03T18:51:02.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:51:02 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:51:02.349+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:26.972061+0000 front 2024-05-03T18:49:26.972448+0000 (oldest deadline 2024-05-03T18:49:50.472344+0000) 2024-05-03T18:51:02.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:51:02 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:51:02.349+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:37.476081+0000 front 2024-05-03T18:49:37.476172+0000 (oldest deadline 2024-05-03T18:50:01.576192+0000) 2024-05-03T18:51:02.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:51:02 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:51:02.349+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:49.079592+0000 front 2024-05-03T18:49:49.080117+0000 (oldest deadline 2024-05-03T18:50:10.779802+0000) 2024-05-03T18:51:02.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:51:02 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:51:02.349+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:59.985863+0000 front 2024-05-03T18:49:59.986111+0000 (oldest deadline 2024-05-03T18:50:22.285990+0000) 2024-05-03T18:51:02.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:51:02 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:51:02.349+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:05.688218+0000 front 2024-05-03T18:50:05.688162+0000 (oldest deadline 2024-05-03T18:50:36.889315+0000) 2024-05-03T18:51:02.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:51:02 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:51:02.349+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2024-05-03T18:50:17.391461+0000 front 2024-05-03T18:50:17.391830+0000 (oldest deadline 2024-05-03T18:50:46.192419+0000) 2024-05-03T18:51:02.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:51:02 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:51:02.349+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-05-03T18:50:26.193991+0000 front 2024-05-03T18:50:26.194137+0000 (oldest deadline 2024-05-03T18:50:56.194110+0000) 2024-05-03T18:51:02.982 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:51:02 smithi033 podman[127609]: 2024-05-03 18:51:02.700807908 +0000 UTC m=+6.519524309 container died 2b58646c996a2c980623cb22c4d89a87ec39cb2e67a214fbb34d115ed5583703 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-05-03T18:51:03.233 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:51:03 smithi033 podman[127609]: 2024-05-03 18:51:03.223098519 +0000 UTC m=+7.041814927 container cleanup 2b58646c996a2c980623cb22c4d89a87ec39cb2e67a214fbb34d115ed5583703 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-05-03T18:51:03.234 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:51:03 smithi033 bash[127609]: ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9 2024-05-03T18:51:03.485 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:51:03 smithi033 podman[127641]: 2024-05-03 18:51:03.436481139 +0000 UTC m=+0.719164266 container remove 2b58646c996a2c980623cb22c4d89a87ec39cb2e67a214fbb34d115ed5583703 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9, org.label-schema.build-date=20240430, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe) 2024-05-03T18:51:03.633 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:03 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:03.363+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:25.480913+0000 front 2024-05-03T18:49:25.480403+0000 (oldest deadline 2024-05-03T18:49:50.180152+0000) 2024-05-03T18:51:03.634 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:03 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:03.363+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:34.882184+0000 front 2024-05-03T18:49:34.882683+0000 (oldest deadline 2024-05-03T18:50:05.483250+0000) 2024-05-03T18:51:03.634 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:03 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:03.363+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:45.484981+0000 front 2024-05-03T18:49:45.484933+0000 (oldest deadline 2024-05-03T18:50:14.885581+0000) 2024-05-03T18:51:03.634 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:03 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:03.363+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:08.290430+0000 front 2024-05-03T18:50:08.290390+0000 (oldest deadline 2024-05-03T18:50:31.790398+0000) 2024-05-03T18:51:03.634 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:03 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:03.363+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2024-05-03T18:50:18.193786+0000 front 2024-05-03T18:50:18.193853+0000 (oldest deadline 2024-05-03T18:50:44.093727+0000) 2024-05-03T18:51:03.634 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:03 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:03.363+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-05-03T18:50:29.995411+0000 front 2024-05-03T18:50:29.995188+0000 (oldest deadline 2024-05-03T18:50:55.294685+0000) 2024-05-03T18:51:03.699 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:51:03 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:51:03.356+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:26.972061+0000 front 2024-05-03T18:49:26.972448+0000 (oldest deadline 2024-05-03T18:49:50.472344+0000) 2024-05-03T18:51:03.699 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:51:03 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:51:03.356+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:37.476081+0000 front 2024-05-03T18:49:37.476172+0000 (oldest deadline 2024-05-03T18:50:01.576192+0000) 2024-05-03T18:51:03.699 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:51:03 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:51:03.356+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:49.079592+0000 front 2024-05-03T18:49:49.080117+0000 (oldest deadline 2024-05-03T18:50:10.779802+0000) 2024-05-03T18:51:03.699 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:51:03 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:51:03.356+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:59.985863+0000 front 2024-05-03T18:49:59.986111+0000 (oldest deadline 2024-05-03T18:50:22.285990+0000) 2024-05-03T18:51:03.699 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:51:03 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:51:03.356+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:05.688218+0000 front 2024-05-03T18:50:05.688162+0000 (oldest deadline 2024-05-03T18:50:36.889315+0000) 2024-05-03T18:51:03.699 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:51:03 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:51:03.356+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2024-05-03T18:50:17.391461+0000 front 2024-05-03T18:50:17.391830+0000 (oldest deadline 2024-05-03T18:50:46.192419+0000) 2024-05-03T18:51:03.699 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:51:03 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:51:03.356+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-05-03T18:50:26.193991+0000 front 2024-05-03T18:50:26.194137+0000 (oldest deadline 2024-05-03T18:50:56.194110+0000) 2024-05-03T18:51:04.609 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:51:04 smithi033 podman[127728]: 2024-05-03 18:51:04.20276948 +0000 UTC m=+0.037320385 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T18:51:04.610 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:51:04 smithi033 podman[127728]: 2024-05-03 18:51:04.353971885 +0000 UTC m=+0.188522815 container create b9969e77670e996918db32ac4022011da48aba133372783fe56f4ae1662bbd4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9-deactivate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-03T18:51:04.610 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:51:04 smithi033 podman[127728]: 2024-05-03 18:51:04.607935403 +0000 UTC m=+0.442486340 container init b9969e77670e996918db32ac4022011da48aba133372783fe56f4ae1662bbd4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-03T18:51:04.633 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:04 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:04.337+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:25.480913+0000 front 2024-05-03T18:49:25.480403+0000 (oldest deadline 2024-05-03T18:49:50.180152+0000) 2024-05-03T18:51:04.634 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:04 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:04.337+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:34.882184+0000 front 2024-05-03T18:49:34.882683+0000 (oldest deadline 2024-05-03T18:50:05.483250+0000) 2024-05-03T18:51:04.634 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:04 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:04.337+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:45.484981+0000 front 2024-05-03T18:49:45.484933+0000 (oldest deadline 2024-05-03T18:50:14.885581+0000) 2024-05-03T18:51:04.634 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:04 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:04.337+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:08.290430+0000 front 2024-05-03T18:50:08.290390+0000 (oldest deadline 2024-05-03T18:50:31.790398+0000) 2024-05-03T18:51:04.634 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:04 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:04.337+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2024-05-03T18:50:18.193786+0000 front 2024-05-03T18:50:18.193853+0000 (oldest deadline 2024-05-03T18:50:44.093727+0000) 2024-05-03T18:51:04.634 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:04 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:04.337+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-05-03T18:50:29.995411+0000 front 2024-05-03T18:50:29.995188+0000 (oldest deadline 2024-05-03T18:50:55.294685+0000) 2024-05-03T18:51:04.634 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:04 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:04.337+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6820 osd.7 since back 2024-05-03T18:50:40.596548+0000 front 2024-05-03T18:50:40.596453+0000 (oldest deadline 2024-05-03T18:51:03.397253+0000) 2024-05-03T18:51:04.736 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:51:04 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:51:04.385+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:26.972061+0000 front 2024-05-03T18:49:26.972448+0000 (oldest deadline 2024-05-03T18:49:50.472344+0000) 2024-05-03T18:51:04.736 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:51:04 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:51:04.385+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:37.476081+0000 front 2024-05-03T18:49:37.476172+0000 (oldest deadline 2024-05-03T18:50:01.576192+0000) 2024-05-03T18:51:04.736 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:51:04 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:51:04.385+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:49.079592+0000 front 2024-05-03T18:49:49.080117+0000 (oldest deadline 2024-05-03T18:50:10.779802+0000) 2024-05-03T18:51:04.736 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:51:04 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:51:04.385+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:59.985863+0000 front 2024-05-03T18:49:59.986111+0000 (oldest deadline 2024-05-03T18:50:22.285990+0000) 2024-05-03T18:51:04.737 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:51:04 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:51:04.385+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:05.688218+0000 front 2024-05-03T18:50:05.688162+0000 (oldest deadline 2024-05-03T18:50:36.889315+0000) 2024-05-03T18:51:04.737 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:51:04 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:51:04.385+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2024-05-03T18:50:17.391461+0000 front 2024-05-03T18:50:17.391830+0000 (oldest deadline 2024-05-03T18:50:46.192419+0000) 2024-05-03T18:51:04.737 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:51:04 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:51:04.385+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-05-03T18:50:26.193991+0000 front 2024-05-03T18:50:26.194137+0000 (oldest deadline 2024-05-03T18:50:56.194110+0000) 2024-05-03T18:51:04.985 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:51:04 smithi033 podman[127728]: 2024-05-03 18:51:04.620054127 +0000 UTC m=+0.454605087 container start b9969e77670e996918db32ac4022011da48aba133372783fe56f4ae1662bbd4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True) 2024-05-03T18:51:04.985 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:51:04 smithi033 podman[127728]: 2024-05-03 18:51:04.697973755 +0000 UTC m=+0.532524712 container attach b9969e77670e996918db32ac4022011da48aba133372783fe56f4ae1662bbd4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430) 2024-05-03T18:51:05.455 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 03 18:51:05 smithi033 podman[127728]: 2024-05-03 18:51:05.095345728 +0000 UTC m=+0.929896629 container died b9969e77670e996918db32ac4022011da48aba133372783fe56f4ae1662bbd4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-9-deactivate, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-03T18:51:05.633 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:05 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:05.379+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:25.480913+0000 front 2024-05-03T18:49:25.480403+0000 (oldest deadline 2024-05-03T18:49:50.180152+0000) 2024-05-03T18:51:05.634 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:05 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:05.379+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:34.882184+0000 front 2024-05-03T18:49:34.882683+0000 (oldest deadline 2024-05-03T18:50:05.483250+0000) 2024-05-03T18:51:05.634 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:05 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:05.379+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:45.484981+0000 front 2024-05-03T18:49:45.484933+0000 (oldest deadline 2024-05-03T18:50:14.885581+0000) 2024-05-03T18:51:05.634 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:05 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:05.379+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:08.290430+0000 front 2024-05-03T18:50:08.290390+0000 (oldest deadline 2024-05-03T18:50:31.790398+0000) 2024-05-03T18:51:05.634 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:05 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:05.379+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2024-05-03T18:50:18.193786+0000 front 2024-05-03T18:50:18.193853+0000 (oldest deadline 2024-05-03T18:50:44.093727+0000) 2024-05-03T18:51:05.634 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:05 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:05.379+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-05-03T18:50:29.995411+0000 front 2024-05-03T18:50:29.995188+0000 (oldest deadline 2024-05-03T18:50:55.294685+0000) 2024-05-03T18:51:05.634 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:05 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:05.379+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6820 osd.7 since back 2024-05-03T18:50:40.596548+0000 front 2024-05-03T18:50:40.596453+0000 (oldest deadline 2024-05-03T18:51:03.397253+0000) 2024-05-03T18:51:05.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:51:05 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:51:05.393+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:26.972061+0000 front 2024-05-03T18:49:26.972448+0000 (oldest deadline 2024-05-03T18:49:50.472344+0000) 2024-05-03T18:51:05.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:51:05 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:51:05.393+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:37.476081+0000 front 2024-05-03T18:49:37.476172+0000 (oldest deadline 2024-05-03T18:50:01.576192+0000) 2024-05-03T18:51:05.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:51:05 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:51:05.393+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:49.079592+0000 front 2024-05-03T18:49:49.080117+0000 (oldest deadline 2024-05-03T18:50:10.779802+0000) 2024-05-03T18:51:05.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:51:05 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:51:05.393+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:59.985863+0000 front 2024-05-03T18:49:59.986111+0000 (oldest deadline 2024-05-03T18:50:22.285990+0000) 2024-05-03T18:51:05.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:51:05 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:51:05.393+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:05.688218+0000 front 2024-05-03T18:50:05.688162+0000 (oldest deadline 2024-05-03T18:50:36.889315+0000) 2024-05-03T18:51:05.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:51:05 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:51:05.393+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2024-05-03T18:50:17.391461+0000 front 2024-05-03T18:50:17.391830+0000 (oldest deadline 2024-05-03T18:50:46.192419+0000) 2024-05-03T18:51:05.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:51:05 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:51:05.393+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-05-03T18:50:26.193991+0000 front 2024-05-03T18:50:26.194137+0000 (oldest deadline 2024-05-03T18:50:56.194110+0000) 2024-05-03T18:51:05.916 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5703bba4-0978-11ef-bc97-c7b262605968@osd.9.service' 2024-05-03T18:51:06.042 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T18:51:06.042 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-05-03T18:51:06.043 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-05-03T18:51:06.043 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-5703bba4-0978-11ef-bc97-c7b262605968@osd.10 2024-05-03T18:51:06.363 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:51:06 smithi099 systemd[1]: Stopping Ceph osd.10 for 5703bba4-0978-11ef-bc97-c7b262605968... 2024-05-03T18:51:06.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:51:06 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:51:06.361+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:26.972061+0000 front 2024-05-03T18:49:26.972448+0000 (oldest deadline 2024-05-03T18:49:50.472344+0000) 2024-05-03T18:51:06.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:51:06 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:51:06.361+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:37.476081+0000 front 2024-05-03T18:49:37.476172+0000 (oldest deadline 2024-05-03T18:50:01.576192+0000) 2024-05-03T18:51:06.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:51:06 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:51:06.361+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:49.079592+0000 front 2024-05-03T18:49:49.080117+0000 (oldest deadline 2024-05-03T18:50:10.779802+0000) 2024-05-03T18:51:06.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:51:06 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:51:06.361+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:59.985863+0000 front 2024-05-03T18:49:59.986111+0000 (oldest deadline 2024-05-03T18:50:22.285990+0000) 2024-05-03T18:51:06.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:51:06 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:51:06.361+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:05.688218+0000 front 2024-05-03T18:50:05.688162+0000 (oldest deadline 2024-05-03T18:50:36.889315+0000) 2024-05-03T18:51:06.798 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:51:06 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:51:06.361+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2024-05-03T18:50:17.391461+0000 front 2024-05-03T18:50:17.391830+0000 (oldest deadline 2024-05-03T18:50:46.192419+0000) 2024-05-03T18:51:06.798 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:51:06 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:51:06.361+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-05-03T18:50:26.193991+0000 front 2024-05-03T18:50:26.194137+0000 (oldest deadline 2024-05-03T18:50:56.194110+0000) 2024-05-03T18:51:06.798 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:51:06 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:51:06.553+0000 7f2654c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T18:51:06.798 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:51:06 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:51:06.553+0000 7f2654c00700 -1 osd.10 126 *** Got signal Terminated *** 2024-05-03T18:51:06.798 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:51:06 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:51:06.553+0000 7f2654c00700 -1 osd.10 126 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-03T18:51:06.883 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:06 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:06.401+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:25.480913+0000 front 2024-05-03T18:49:25.480403+0000 (oldest deadline 2024-05-03T18:49:50.180152+0000) 2024-05-03T18:51:06.884 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:06 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:06.401+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:34.882184+0000 front 2024-05-03T18:49:34.882683+0000 (oldest deadline 2024-05-03T18:50:05.483250+0000) 2024-05-03T18:51:06.884 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:06 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:06.401+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:45.484981+0000 front 2024-05-03T18:49:45.484933+0000 (oldest deadline 2024-05-03T18:50:14.885581+0000) 2024-05-03T18:51:06.884 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:06 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:06.401+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:08.290430+0000 front 2024-05-03T18:50:08.290390+0000 (oldest deadline 2024-05-03T18:50:31.790398+0000) 2024-05-03T18:51:06.884 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:06 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:06.401+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2024-05-03T18:50:18.193786+0000 front 2024-05-03T18:50:18.193853+0000 (oldest deadline 2024-05-03T18:50:44.093727+0000) 2024-05-03T18:51:06.884 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:06 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:06.401+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-05-03T18:50:29.995411+0000 front 2024-05-03T18:50:29.995188+0000 (oldest deadline 2024-05-03T18:50:55.294685+0000) 2024-05-03T18:51:06.884 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:06 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:06.401+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6820 osd.7 since back 2024-05-03T18:50:40.596548+0000 front 2024-05-03T18:50:40.596453+0000 (oldest deadline 2024-05-03T18:51:03.397253+0000) 2024-05-03T18:51:07.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:51:07 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:51:07.342+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:26.972061+0000 front 2024-05-03T18:49:26.972448+0000 (oldest deadline 2024-05-03T18:49:50.472344+0000) 2024-05-03T18:51:07.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:51:07 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:51:07.342+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:37.476081+0000 front 2024-05-03T18:49:37.476172+0000 (oldest deadline 2024-05-03T18:50:01.576192+0000) 2024-05-03T18:51:07.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:51:07 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:51:07.342+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:49.079592+0000 front 2024-05-03T18:49:49.080117+0000 (oldest deadline 2024-05-03T18:50:10.779802+0000) 2024-05-03T18:51:07.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:51:07 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:51:07.342+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:59.985863+0000 front 2024-05-03T18:49:59.986111+0000 (oldest deadline 2024-05-03T18:50:22.285990+0000) 2024-05-03T18:51:07.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:51:07 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:51:07.342+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:05.688218+0000 front 2024-05-03T18:50:05.688162+0000 (oldest deadline 2024-05-03T18:50:36.889315+0000) 2024-05-03T18:51:07.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:51:07 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:51:07.342+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2024-05-03T18:50:17.391461+0000 front 2024-05-03T18:50:17.391830+0000 (oldest deadline 2024-05-03T18:50:46.192419+0000) 2024-05-03T18:51:07.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:51:07 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:51:07.342+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-05-03T18:50:26.193991+0000 front 2024-05-03T18:50:26.194137+0000 (oldest deadline 2024-05-03T18:50:56.194110+0000) 2024-05-03T18:51:07.883 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:07 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:07.394+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:25.480913+0000 front 2024-05-03T18:49:25.480403+0000 (oldest deadline 2024-05-03T18:49:50.180152+0000) 2024-05-03T18:51:07.884 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:07 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:07.394+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:34.882184+0000 front 2024-05-03T18:49:34.882683+0000 (oldest deadline 2024-05-03T18:50:05.483250+0000) 2024-05-03T18:51:07.884 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:07 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:07.394+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:45.484981+0000 front 2024-05-03T18:49:45.484933+0000 (oldest deadline 2024-05-03T18:50:14.885581+0000) 2024-05-03T18:51:07.884 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:07 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:07.394+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:08.290430+0000 front 2024-05-03T18:50:08.290390+0000 (oldest deadline 2024-05-03T18:50:31.790398+0000) 2024-05-03T18:51:07.884 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:07 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:07.394+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2024-05-03T18:50:18.193786+0000 front 2024-05-03T18:50:18.193853+0000 (oldest deadline 2024-05-03T18:50:44.093727+0000) 2024-05-03T18:51:07.884 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:07 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:07.394+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-05-03T18:50:29.995411+0000 front 2024-05-03T18:50:29.995188+0000 (oldest deadline 2024-05-03T18:50:55.294685+0000) 2024-05-03T18:51:07.884 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:07 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:07.394+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6820 osd.7 since back 2024-05-03T18:50:40.596548+0000 front 2024-05-03T18:50:40.596453+0000 (oldest deadline 2024-05-03T18:51:03.397253+0000) 2024-05-03T18:51:08.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:51:08 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:51:08.344+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:26.972061+0000 front 2024-05-03T18:49:26.972448+0000 (oldest deadline 2024-05-03T18:49:50.472344+0000) 2024-05-03T18:51:08.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:51:08 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:51:08.344+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:37.476081+0000 front 2024-05-03T18:49:37.476172+0000 (oldest deadline 2024-05-03T18:50:01.576192+0000) 2024-05-03T18:51:08.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:51:08 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:51:08.344+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:49.079592+0000 front 2024-05-03T18:49:49.080117+0000 (oldest deadline 2024-05-03T18:50:10.779802+0000) 2024-05-03T18:51:08.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:51:08 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:51:08.344+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:59.985863+0000 front 2024-05-03T18:49:59.986111+0000 (oldest deadline 2024-05-03T18:50:22.285990+0000) 2024-05-03T18:51:08.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:51:08 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:51:08.344+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:05.688218+0000 front 2024-05-03T18:50:05.688162+0000 (oldest deadline 2024-05-03T18:50:36.889315+0000) 2024-05-03T18:51:08.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:51:08 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:51:08.344+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2024-05-03T18:50:17.391461+0000 front 2024-05-03T18:50:17.391830+0000 (oldest deadline 2024-05-03T18:50:46.192419+0000) 2024-05-03T18:51:08.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:51:08 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:51:08.344+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-05-03T18:50:26.193991+0000 front 2024-05-03T18:50:26.194137+0000 (oldest deadline 2024-05-03T18:50:56.194110+0000) 2024-05-03T18:51:08.883 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:08 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:08.391+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:25.480913+0000 front 2024-05-03T18:49:25.480403+0000 (oldest deadline 2024-05-03T18:49:50.180152+0000) 2024-05-03T18:51:08.884 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:08 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:08.391+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:34.882184+0000 front 2024-05-03T18:49:34.882683+0000 (oldest deadline 2024-05-03T18:50:05.483250+0000) 2024-05-03T18:51:08.884 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:08 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:08.391+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:45.484981+0000 front 2024-05-03T18:49:45.484933+0000 (oldest deadline 2024-05-03T18:50:14.885581+0000) 2024-05-03T18:51:08.884 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:08 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:08.391+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:08.290430+0000 front 2024-05-03T18:50:08.290390+0000 (oldest deadline 2024-05-03T18:50:31.790398+0000) 2024-05-03T18:51:08.884 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:08 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:08.391+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2024-05-03T18:50:18.193786+0000 front 2024-05-03T18:50:18.193853+0000 (oldest deadline 2024-05-03T18:50:44.093727+0000) 2024-05-03T18:51:08.884 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:08 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:08.391+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-05-03T18:50:29.995411+0000 front 2024-05-03T18:50:29.995188+0000 (oldest deadline 2024-05-03T18:50:55.294685+0000) 2024-05-03T18:51:08.884 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:08 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:08.391+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6820 osd.7 since back 2024-05-03T18:50:40.596548+0000 front 2024-05-03T18:50:40.596453+0000 (oldest deadline 2024-05-03T18:51:03.397253+0000) 2024-05-03T18:51:09.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:51:09 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:51:09.388+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:26.972061+0000 front 2024-05-03T18:49:26.972448+0000 (oldest deadline 2024-05-03T18:49:50.472344+0000) 2024-05-03T18:51:09.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:51:09 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:51:09.388+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:37.476081+0000 front 2024-05-03T18:49:37.476172+0000 (oldest deadline 2024-05-03T18:50:01.576192+0000) 2024-05-03T18:51:09.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:51:09 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:51:09.388+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:49.079592+0000 front 2024-05-03T18:49:49.080117+0000 (oldest deadline 2024-05-03T18:50:10.779802+0000) 2024-05-03T18:51:09.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:51:09 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:51:09.388+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:59.985863+0000 front 2024-05-03T18:49:59.986111+0000 (oldest deadline 2024-05-03T18:50:22.285990+0000) 2024-05-03T18:51:09.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:51:09 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:51:09.388+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:05.688218+0000 front 2024-05-03T18:50:05.688162+0000 (oldest deadline 2024-05-03T18:50:36.889315+0000) 2024-05-03T18:51:09.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:51:09 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:51:09.388+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2024-05-03T18:50:17.391461+0000 front 2024-05-03T18:50:17.391830+0000 (oldest deadline 2024-05-03T18:50:46.192419+0000) 2024-05-03T18:51:09.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:51:09 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:51:09.388+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-05-03T18:50:26.193991+0000 front 2024-05-03T18:50:26.194137+0000 (oldest deadline 2024-05-03T18:50:56.194110+0000) 2024-05-03T18:51:09.883 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:09 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:09.431+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:25.480913+0000 front 2024-05-03T18:49:25.480403+0000 (oldest deadline 2024-05-03T18:49:50.180152+0000) 2024-05-03T18:51:09.883 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:09 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:09.431+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:34.882184+0000 front 2024-05-03T18:49:34.882683+0000 (oldest deadline 2024-05-03T18:50:05.483250+0000) 2024-05-03T18:51:09.884 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:09 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:09.431+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:45.484981+0000 front 2024-05-03T18:49:45.484933+0000 (oldest deadline 2024-05-03T18:50:14.885581+0000) 2024-05-03T18:51:09.884 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:09 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:09.431+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:08.290430+0000 front 2024-05-03T18:50:08.290390+0000 (oldest deadline 2024-05-03T18:50:31.790398+0000) 2024-05-03T18:51:09.884 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:09 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:09.431+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2024-05-03T18:50:18.193786+0000 front 2024-05-03T18:50:18.193853+0000 (oldest deadline 2024-05-03T18:50:44.093727+0000) 2024-05-03T18:51:09.884 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:09 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:09.431+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-05-03T18:50:29.995411+0000 front 2024-05-03T18:50:29.995188+0000 (oldest deadline 2024-05-03T18:50:55.294685+0000) 2024-05-03T18:51:09.884 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:09 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:09.431+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6820 osd.7 since back 2024-05-03T18:50:40.596548+0000 front 2024-05-03T18:50:40.596453+0000 (oldest deadline 2024-05-03T18:51:03.397253+0000) 2024-05-03T18:51:10.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:51:10 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:51:10.384+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:26.972061+0000 front 2024-05-03T18:49:26.972448+0000 (oldest deadline 2024-05-03T18:49:50.472344+0000) 2024-05-03T18:51:10.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:51:10 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:51:10.384+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:37.476081+0000 front 2024-05-03T18:49:37.476172+0000 (oldest deadline 2024-05-03T18:50:01.576192+0000) 2024-05-03T18:51:10.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:51:10 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:51:10.384+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:49.079592+0000 front 2024-05-03T18:49:49.080117+0000 (oldest deadline 2024-05-03T18:50:10.779802+0000) 2024-05-03T18:51:10.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:51:10 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:51:10.384+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:59.985863+0000 front 2024-05-03T18:49:59.986111+0000 (oldest deadline 2024-05-03T18:50:22.285990+0000) 2024-05-03T18:51:10.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:51:10 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:51:10.384+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:05.688218+0000 front 2024-05-03T18:50:05.688162+0000 (oldest deadline 2024-05-03T18:50:36.889315+0000) 2024-05-03T18:51:10.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:51:10 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:51:10.384+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2024-05-03T18:50:17.391461+0000 front 2024-05-03T18:50:17.391830+0000 (oldest deadline 2024-05-03T18:50:46.192419+0000) 2024-05-03T18:51:10.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:51:10 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:51:10.384+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-05-03T18:50:26.193991+0000 front 2024-05-03T18:50:26.194137+0000 (oldest deadline 2024-05-03T18:50:56.194110+0000) 2024-05-03T18:51:10.883 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:10 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:10.419+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:25.480913+0000 front 2024-05-03T18:49:25.480403+0000 (oldest deadline 2024-05-03T18:49:50.180152+0000) 2024-05-03T18:51:10.884 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:10 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:10.419+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:34.882184+0000 front 2024-05-03T18:49:34.882683+0000 (oldest deadline 2024-05-03T18:50:05.483250+0000) 2024-05-03T18:51:10.884 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:10 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:10.419+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:45.484981+0000 front 2024-05-03T18:49:45.484933+0000 (oldest deadline 2024-05-03T18:50:14.885581+0000) 2024-05-03T18:51:10.884 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:10 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:10.419+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:08.290430+0000 front 2024-05-03T18:50:08.290390+0000 (oldest deadline 2024-05-03T18:50:31.790398+0000) 2024-05-03T18:51:10.884 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:10 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:10.419+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2024-05-03T18:50:18.193786+0000 front 2024-05-03T18:50:18.193853+0000 (oldest deadline 2024-05-03T18:50:44.093727+0000) 2024-05-03T18:51:10.884 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:10 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:10.419+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-05-03T18:50:29.995411+0000 front 2024-05-03T18:50:29.995188+0000 (oldest deadline 2024-05-03T18:50:55.294685+0000) 2024-05-03T18:51:10.884 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:10 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:10.419+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6820 osd.7 since back 2024-05-03T18:50:40.596548+0000 front 2024-05-03T18:50:40.596453+0000 (oldest deadline 2024-05-03T18:51:03.397253+0000) 2024-05-03T18:51:11.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:51:11 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:51:11.383+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:26.972061+0000 front 2024-05-03T18:49:26.972448+0000 (oldest deadline 2024-05-03T18:49:50.472344+0000) 2024-05-03T18:51:11.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:51:11 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:51:11.383+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:37.476081+0000 front 2024-05-03T18:49:37.476172+0000 (oldest deadline 2024-05-03T18:50:01.576192+0000) 2024-05-03T18:51:11.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:51:11 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:51:11.383+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:49.079592+0000 front 2024-05-03T18:49:49.080117+0000 (oldest deadline 2024-05-03T18:50:10.779802+0000) 2024-05-03T18:51:11.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:51:11 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:51:11.383+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-03T18:49:59.985863+0000 front 2024-05-03T18:49:59.986111+0000 (oldest deadline 2024-05-03T18:50:22.285990+0000) 2024-05-03T18:51:11.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:51:11 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:51:11.383+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:05.688218+0000 front 2024-05-03T18:50:05.688162+0000 (oldest deadline 2024-05-03T18:50:36.889315+0000) 2024-05-03T18:51:11.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:51:11 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:51:11.383+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2024-05-03T18:50:17.391461+0000 front 2024-05-03T18:50:17.391830+0000 (oldest deadline 2024-05-03T18:50:46.192419+0000) 2024-05-03T18:51:11.797 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:51:11 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10[56378]: 2024-05-03T18:51:11.383+0000 7f2650400700 -1 osd.10 126 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-05-03T18:50:26.193991+0000 front 2024-05-03T18:50:26.194137+0000 (oldest deadline 2024-05-03T18:50:56.194110+0000) 2024-05-03T18:51:11.883 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:11 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:11.439+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:25.480913+0000 front 2024-05-03T18:49:25.480403+0000 (oldest deadline 2024-05-03T18:49:50.180152+0000) 2024-05-03T18:51:11.884 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:11 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:11.439+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:34.882184+0000 front 2024-05-03T18:49:34.882683+0000 (oldest deadline 2024-05-03T18:50:05.483250+0000) 2024-05-03T18:51:11.884 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:11 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:11.439+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:45.484981+0000 front 2024-05-03T18:49:45.484933+0000 (oldest deadline 2024-05-03T18:50:14.885581+0000) 2024-05-03T18:51:11.884 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:11 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:11.439+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:08.290430+0000 front 2024-05-03T18:50:08.290390+0000 (oldest deadline 2024-05-03T18:50:31.790398+0000) 2024-05-03T18:51:11.884 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:11 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:11.439+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2024-05-03T18:50:18.193786+0000 front 2024-05-03T18:50:18.193853+0000 (oldest deadline 2024-05-03T18:50:44.093727+0000) 2024-05-03T18:51:11.884 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:11 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:11.439+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-05-03T18:50:29.995411+0000 front 2024-05-03T18:50:29.995188+0000 (oldest deadline 2024-05-03T18:50:55.294685+0000) 2024-05-03T18:51:11.884 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:11 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:11.439+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6820 osd.7 since back 2024-05-03T18:50:40.596548+0000 front 2024-05-03T18:50:40.596453+0000 (oldest deadline 2024-05-03T18:51:03.397253+0000) 2024-05-03T18:51:12.710 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:51:12 smithi099 podman[71578]: 2024-05-03 18:51:12.401588415 +0000 UTC m=+6.006752098 container died 4a52b3693de22d3a99726b32af92aca8e799cfae11c85c87660580e0bb5541b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240430) 2024-05-03T18:51:12.884 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:12 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:12.433+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:25.480913+0000 front 2024-05-03T18:49:25.480403+0000 (oldest deadline 2024-05-03T18:49:50.180152+0000) 2024-05-03T18:51:12.884 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:12 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:12.433+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:34.882184+0000 front 2024-05-03T18:49:34.882683+0000 (oldest deadline 2024-05-03T18:50:05.483250+0000) 2024-05-03T18:51:12.884 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:12 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:12.433+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:45.484981+0000 front 2024-05-03T18:49:45.484933+0000 (oldest deadline 2024-05-03T18:50:14.885581+0000) 2024-05-03T18:51:12.884 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:12 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:12.433+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:08.290430+0000 front 2024-05-03T18:50:08.290390+0000 (oldest deadline 2024-05-03T18:50:31.790398+0000) 2024-05-03T18:51:12.884 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:12 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:12.433+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2024-05-03T18:50:18.193786+0000 front 2024-05-03T18:50:18.193853+0000 (oldest deadline 2024-05-03T18:50:44.093727+0000) 2024-05-03T18:51:12.884 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:12 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:12.433+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-05-03T18:50:29.995411+0000 front 2024-05-03T18:50:29.995188+0000 (oldest deadline 2024-05-03T18:50:55.294685+0000) 2024-05-03T18:51:12.884 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:12 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:12.433+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6820 osd.7 since back 2024-05-03T18:50:40.596548+0000 front 2024-05-03T18:50:40.596453+0000 (oldest deadline 2024-05-03T18:51:03.397253+0000) 2024-05-03T18:51:13.046 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:51:12 smithi099 podman[71578]: 2024-05-03 18:51:12.879549392 +0000 UTC m=+6.484713085 container cleanup 4a52b3693de22d3a99726b32af92aca8e799cfae11c85c87660580e0bb5541b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-03T18:51:13.046 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:51:12 smithi099 bash[71578]: ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10 2024-05-03T18:51:13.546 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:51:13 smithi099 podman[71598]: 2024-05-03 18:51:13.189214463 +0000 UTC m=+0.764870208 container remove 4a52b3693de22d3a99726b32af92aca8e799cfae11c85c87660580e0bb5541b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240430, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-05-03T18:51:13.883 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:13 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:13.408+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:25.480913+0000 front 2024-05-03T18:49:25.480403+0000 (oldest deadline 2024-05-03T18:49:50.180152+0000) 2024-05-03T18:51:13.884 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:13 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:13.408+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:34.882184+0000 front 2024-05-03T18:49:34.882683+0000 (oldest deadline 2024-05-03T18:50:05.483250+0000) 2024-05-03T18:51:13.884 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:13 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:13.408+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:45.484981+0000 front 2024-05-03T18:49:45.484933+0000 (oldest deadline 2024-05-03T18:50:14.885581+0000) 2024-05-03T18:51:13.884 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:13 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:13.408+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:08.290430+0000 front 2024-05-03T18:50:08.290390+0000 (oldest deadline 2024-05-03T18:50:31.790398+0000) 2024-05-03T18:51:13.884 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:13 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:13.408+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2024-05-03T18:50:18.193786+0000 front 2024-05-03T18:50:18.193853+0000 (oldest deadline 2024-05-03T18:50:44.093727+0000) 2024-05-03T18:51:13.884 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:13 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:13.408+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-05-03T18:50:29.995411+0000 front 2024-05-03T18:50:29.995188+0000 (oldest deadline 2024-05-03T18:50:55.294685+0000) 2024-05-03T18:51:13.884 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:13 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:13.408+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6820 osd.7 since back 2024-05-03T18:50:40.596548+0000 front 2024-05-03T18:50:40.596453+0000 (oldest deadline 2024-05-03T18:51:03.397253+0000) 2024-05-03T18:51:14.111 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:51:13 smithi099 podman[71688]: 2024-05-03 18:51:13.717332964 +0000 UTC m=+0.034921068 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T18:51:14.112 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:51:13 smithi099 podman[71688]: 2024-05-03 18:51:13.856392418 +0000 UTC m=+0.173980490 container create 678c6ee24216ed3df4bd89779f376c1fab4cfe0c154bedcbc41e06fc30004e03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10-deactivate, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD) 2024-05-03T18:51:14.112 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:51:14 smithi099 podman[71688]: 2024-05-03 18:51:14.110129748 +0000 UTC m=+0.427717903 container init 678c6ee24216ed3df4bd89779f376c1fab4cfe0c154bedcbc41e06fc30004e03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10-deactivate, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-05-03T18:51:14.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:51:14 smithi099 podman[71688]: 2024-05-03 18:51:14.122688614 +0000 UTC m=+0.440276731 container start 678c6ee24216ed3df4bd89779f376c1fab4cfe0c154bedcbc41e06fc30004e03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-03T18:51:14.547 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:51:14 smithi099 podman[71688]: 2024-05-03 18:51:14.267164627 +0000 UTC m=+0.584752719 container attach 678c6ee24216ed3df4bd89779f376c1fab4cfe0c154bedcbc41e06fc30004e03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-03T18:51:14.884 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:14 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:14.409+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:25.480913+0000 front 2024-05-03T18:49:25.480403+0000 (oldest deadline 2024-05-03T18:49:50.180152+0000) 2024-05-03T18:51:14.884 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:14 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:14.409+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:34.882184+0000 front 2024-05-03T18:49:34.882683+0000 (oldest deadline 2024-05-03T18:50:05.483250+0000) 2024-05-03T18:51:14.884 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:14 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:14.409+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:45.484981+0000 front 2024-05-03T18:49:45.484933+0000 (oldest deadline 2024-05-03T18:50:14.885581+0000) 2024-05-03T18:51:14.884 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:14 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:14.409+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:08.290430+0000 front 2024-05-03T18:50:08.290390+0000 (oldest deadline 2024-05-03T18:50:31.790398+0000) 2024-05-03T18:51:14.884 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:14 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:14.409+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2024-05-03T18:50:18.193786+0000 front 2024-05-03T18:50:18.193853+0000 (oldest deadline 2024-05-03T18:50:44.093727+0000) 2024-05-03T18:51:14.884 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:14 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:14.409+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-05-03T18:50:29.995411+0000 front 2024-05-03T18:50:29.995188+0000 (oldest deadline 2024-05-03T18:50:55.294685+0000) 2024-05-03T18:51:14.884 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:14 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:14.409+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6820 osd.7 since back 2024-05-03T18:50:40.596548+0000 front 2024-05-03T18:50:40.596453+0000 (oldest deadline 2024-05-03T18:51:03.397253+0000) 2024-05-03T18:51:14.899 INFO:journalctl@ceph.osd.10.smithi099.stdout:May 03 18:51:14 smithi099 podman[71688]: 2024-05-03 18:51:14.559079744 +0000 UTC m=+0.876667862 container died 678c6ee24216ed3df4bd89779f376c1fab4cfe0c154bedcbc41e06fc30004e03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-10-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-03T18:51:15.266 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5703bba4-0978-11ef-bc97-c7b262605968@osd.10.service' 2024-05-03T18:51:15.372 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T18:51:15.372 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-05-03T18:51:15.373 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-05-03T18:51:15.373 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-5703bba4-0978-11ef-bc97-c7b262605968@osd.11 2024-05-03T18:51:15.884 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:15 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:15.438+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:25.480913+0000 front 2024-05-03T18:49:25.480403+0000 (oldest deadline 2024-05-03T18:49:50.180152+0000) 2024-05-03T18:51:15.884 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:15 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:15.438+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:34.882184+0000 front 2024-05-03T18:49:34.882683+0000 (oldest deadline 2024-05-03T18:50:05.483250+0000) 2024-05-03T18:51:15.884 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:15 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:15.438+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:45.484981+0000 front 2024-05-03T18:49:45.484933+0000 (oldest deadline 2024-05-03T18:50:14.885581+0000) 2024-05-03T18:51:15.884 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:15 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:15.438+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:08.290430+0000 front 2024-05-03T18:50:08.290390+0000 (oldest deadline 2024-05-03T18:50:31.790398+0000) 2024-05-03T18:51:15.884 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:15 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:15.438+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2024-05-03T18:50:18.193786+0000 front 2024-05-03T18:50:18.193853+0000 (oldest deadline 2024-05-03T18:50:44.093727+0000) 2024-05-03T18:51:15.884 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:15 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:15.438+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-05-03T18:50:29.995411+0000 front 2024-05-03T18:50:29.995188+0000 (oldest deadline 2024-05-03T18:50:55.294685+0000) 2024-05-03T18:51:15.884 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:15 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:15.438+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6820 osd.7 since back 2024-05-03T18:50:40.596548+0000 front 2024-05-03T18:50:40.596453+0000 (oldest deadline 2024-05-03T18:51:03.397253+0000) 2024-05-03T18:51:15.884 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:15 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:15.438+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6820 osd.8 since back 2024-05-03T18:50:49.199857+0000 front 2024-05-03T18:50:49.199949+0000 (oldest deadline 2024-05-03T18:51:14.499655+0000) 2024-05-03T18:51:15.885 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:15 smithi150 systemd[1]: Stopping Ceph osd.11 for 5703bba4-0978-11ef-bc97-c7b262605968... 2024-05-03T18:51:16.383 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:15 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:15.920+0000 7f4591a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T18:51:16.383 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:15 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:15.920+0000 7f4591a00700 -1 osd.11 126 *** Got signal Terminated *** 2024-05-03T18:51:16.383 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:15 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:15.920+0000 7f4591a00700 -1 osd.11 126 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-03T18:51:16.883 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:16 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:16.425+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:25.480913+0000 front 2024-05-03T18:49:25.480403+0000 (oldest deadline 2024-05-03T18:49:50.180152+0000) 2024-05-03T18:51:16.884 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:16 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:16.425+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:34.882184+0000 front 2024-05-03T18:49:34.882683+0000 (oldest deadline 2024-05-03T18:50:05.483250+0000) 2024-05-03T18:51:16.884 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:16 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:16.425+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:45.484981+0000 front 2024-05-03T18:49:45.484933+0000 (oldest deadline 2024-05-03T18:50:14.885581+0000) 2024-05-03T18:51:16.884 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:16 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:16.425+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:08.290430+0000 front 2024-05-03T18:50:08.290390+0000 (oldest deadline 2024-05-03T18:50:31.790398+0000) 2024-05-03T18:51:16.884 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:16 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:16.425+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2024-05-03T18:50:18.193786+0000 front 2024-05-03T18:50:18.193853+0000 (oldest deadline 2024-05-03T18:50:44.093727+0000) 2024-05-03T18:51:16.884 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:16 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:16.425+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-05-03T18:50:29.995411+0000 front 2024-05-03T18:50:29.995188+0000 (oldest deadline 2024-05-03T18:50:55.294685+0000) 2024-05-03T18:51:16.884 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:16 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:16.425+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6820 osd.7 since back 2024-05-03T18:50:40.596548+0000 front 2024-05-03T18:50:40.596453+0000 (oldest deadline 2024-05-03T18:51:03.397253+0000) 2024-05-03T18:51:16.884 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:16 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:16.425+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6820 osd.8 since back 2024-05-03T18:50:49.199857+0000 front 2024-05-03T18:50:49.199949+0000 (oldest deadline 2024-05-03T18:51:14.499655+0000) 2024-05-03T18:51:17.633 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:17 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:17.380+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:25.480913+0000 front 2024-05-03T18:49:25.480403+0000 (oldest deadline 2024-05-03T18:49:50.180152+0000) 2024-05-03T18:51:17.634 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:17 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:17.380+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:34.882184+0000 front 2024-05-03T18:49:34.882683+0000 (oldest deadline 2024-05-03T18:50:05.483250+0000) 2024-05-03T18:51:17.634 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:17 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:17.380+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:45.484981+0000 front 2024-05-03T18:49:45.484933+0000 (oldest deadline 2024-05-03T18:50:14.885581+0000) 2024-05-03T18:51:17.634 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:17 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:17.380+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:08.290430+0000 front 2024-05-03T18:50:08.290390+0000 (oldest deadline 2024-05-03T18:50:31.790398+0000) 2024-05-03T18:51:17.634 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:17 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:17.380+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2024-05-03T18:50:18.193786+0000 front 2024-05-03T18:50:18.193853+0000 (oldest deadline 2024-05-03T18:50:44.093727+0000) 2024-05-03T18:51:17.634 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:17 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:17.380+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-05-03T18:50:29.995411+0000 front 2024-05-03T18:50:29.995188+0000 (oldest deadline 2024-05-03T18:50:55.294685+0000) 2024-05-03T18:51:17.634 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:17 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:17.380+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6820 osd.7 since back 2024-05-03T18:50:40.596548+0000 front 2024-05-03T18:50:40.596453+0000 (oldest deadline 2024-05-03T18:51:03.397253+0000) 2024-05-03T18:51:17.634 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:17 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:17.380+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6820 osd.8 since back 2024-05-03T18:50:49.199857+0000 front 2024-05-03T18:50:49.199949+0000 (oldest deadline 2024-05-03T18:51:14.499655+0000) 2024-05-03T18:51:18.883 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:18 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:18.429+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:25.480913+0000 front 2024-05-03T18:49:25.480403+0000 (oldest deadline 2024-05-03T18:49:50.180152+0000) 2024-05-03T18:51:18.884 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:18 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:18.429+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:34.882184+0000 front 2024-05-03T18:49:34.882683+0000 (oldest deadline 2024-05-03T18:50:05.483250+0000) 2024-05-03T18:51:18.884 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:18 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:18.429+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:45.484981+0000 front 2024-05-03T18:49:45.484933+0000 (oldest deadline 2024-05-03T18:50:14.885581+0000) 2024-05-03T18:51:18.884 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:18 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:18.429+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:08.290430+0000 front 2024-05-03T18:50:08.290390+0000 (oldest deadline 2024-05-03T18:50:31.790398+0000) 2024-05-03T18:51:18.884 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:18 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:18.429+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2024-05-03T18:50:18.193786+0000 front 2024-05-03T18:50:18.193853+0000 (oldest deadline 2024-05-03T18:50:44.093727+0000) 2024-05-03T18:51:18.884 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:18 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:18.429+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-05-03T18:50:29.995411+0000 front 2024-05-03T18:50:29.995188+0000 (oldest deadline 2024-05-03T18:50:55.294685+0000) 2024-05-03T18:51:18.884 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:18 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:18.429+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6820 osd.7 since back 2024-05-03T18:50:40.596548+0000 front 2024-05-03T18:50:40.596453+0000 (oldest deadline 2024-05-03T18:51:03.397253+0000) 2024-05-03T18:51:18.884 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:18 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:18.429+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6820 osd.8 since back 2024-05-03T18:50:49.199857+0000 front 2024-05-03T18:50:49.199949+0000 (oldest deadline 2024-05-03T18:51:14.499655+0000) 2024-05-03T18:51:19.884 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:19 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:19.420+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:25.480913+0000 front 2024-05-03T18:49:25.480403+0000 (oldest deadline 2024-05-03T18:49:50.180152+0000) 2024-05-03T18:51:19.884 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:19 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:19.420+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:34.882184+0000 front 2024-05-03T18:49:34.882683+0000 (oldest deadline 2024-05-03T18:50:05.483250+0000) 2024-05-03T18:51:19.884 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:19 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:19.420+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:45.484981+0000 front 2024-05-03T18:49:45.484933+0000 (oldest deadline 2024-05-03T18:50:14.885581+0000) 2024-05-03T18:51:19.884 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:19 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:19.420+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:08.290430+0000 front 2024-05-03T18:50:08.290390+0000 (oldest deadline 2024-05-03T18:50:31.790398+0000) 2024-05-03T18:51:19.884 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:19 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:19.420+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2024-05-03T18:50:18.193786+0000 front 2024-05-03T18:50:18.193853+0000 (oldest deadline 2024-05-03T18:50:44.093727+0000) 2024-05-03T18:51:19.884 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:19 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:19.420+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-05-03T18:50:29.995411+0000 front 2024-05-03T18:50:29.995188+0000 (oldest deadline 2024-05-03T18:50:55.294685+0000) 2024-05-03T18:51:19.884 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:19 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:19.420+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6820 osd.7 since back 2024-05-03T18:50:40.596548+0000 front 2024-05-03T18:50:40.596453+0000 (oldest deadline 2024-05-03T18:51:03.397253+0000) 2024-05-03T18:51:19.884 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:19 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:19.420+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6820 osd.8 since back 2024-05-03T18:50:49.199857+0000 front 2024-05-03T18:50:49.199949+0000 (oldest deadline 2024-05-03T18:51:14.499655+0000) 2024-05-03T18:51:20.884 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:20 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:20.449+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-03T18:49:25.480913+0000 front 2024-05-03T18:49:25.480403+0000 (oldest deadline 2024-05-03T18:49:50.180152+0000) 2024-05-03T18:51:20.884 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:20 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:20.449+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-05-03T18:49:34.882184+0000 front 2024-05-03T18:49:34.882683+0000 (oldest deadline 2024-05-03T18:50:05.483250+0000) 2024-05-03T18:51:20.884 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:20 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:20.449+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2024-05-03T18:49:45.484981+0000 front 2024-05-03T18:49:45.484933+0000 (oldest deadline 2024-05-03T18:50:14.885581+0000) 2024-05-03T18:51:20.884 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:20 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:20.450+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-05-03T18:50:08.290430+0000 front 2024-05-03T18:50:08.290390+0000 (oldest deadline 2024-05-03T18:50:31.790398+0000) 2024-05-03T18:51:20.884 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:20 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:20.450+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2024-05-03T18:50:18.193786+0000 front 2024-05-03T18:50:18.193853+0000 (oldest deadline 2024-05-03T18:50:44.093727+0000) 2024-05-03T18:51:20.884 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:20 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:20.450+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-05-03T18:50:29.995411+0000 front 2024-05-03T18:50:29.995188+0000 (oldest deadline 2024-05-03T18:50:55.294685+0000) 2024-05-03T18:51:20.884 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:20 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:20.450+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.99:6820 osd.7 since back 2024-05-03T18:50:40.596548+0000 front 2024-05-03T18:50:40.596453+0000 (oldest deadline 2024-05-03T18:51:03.397253+0000) 2024-05-03T18:51:20.884 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:20 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11[58180]: 2024-05-03T18:51:20.450+0000 7f4588200700 -1 osd.11 126 heartbeat_check: no reply from 172.21.15.150:6820 osd.8 since back 2024-05-03T18:50:49.199857+0000 front 2024-05-03T18:50:49.199949+0000 (oldest deadline 2024-05-03T18:51:14.499655+0000) 2024-05-03T18:51:22.254 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:21 smithi150 podman[72375]: 2024-05-03 18:51:21.959454861 +0000 UTC m=+6.171820900 container died d6f67eb3ce89703ac4c5e4a7fd4fcb56610269581265715408507d75d23637fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe) 2024-05-03T18:51:22.624 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:22 smithi150 podman[72375]: 2024-05-03 18:51:22.380569465 +0000 UTC m=+6.592935497 container cleanup d6f67eb3ce89703ac4c5e4a7fd4fcb56610269581265715408507d75d23637fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-05-03T18:51:22.625 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:22 smithi150 bash[72375]: ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11 2024-05-03T18:51:22.883 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:22 smithi150 podman[72395]: 2024-05-03 18:51:22.622978603 +0000 UTC m=+0.647945560 container remove d6f67eb3ce89703ac4c5e4a7fd4fcb56610269581265715408507d75d23637fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-05-03T18:51:23.686 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:23 smithi150 podman[72480]: 2024-05-03 18:51:23.316132252 +0000 UTC m=+0.034808581 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T18:51:23.686 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:23 smithi150 podman[72480]: 2024-05-03 18:51:23.415493901 +0000 UTC m=+0.134170266 container create 832e22a2f0863dc82edb962e6e0828e0fa319c4f252f5a2c3223226786e17c1e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-05-03T18:51:23.686 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:23 smithi150 podman[72480]: 2024-05-03 18:51:23.685108658 +0000 UTC m=+0.403785033 container init 832e22a2f0863dc82edb962e6e0828e0fa319c4f252f5a2c3223226786e17c1e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, ceph=True, GIT_BRANCH=HEAD) 2024-05-03T18:51:24.133 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:23 smithi150 podman[72480]: 2024-05-03 18:51:23.696950383 +0000 UTC m=+0.415626719 container start 832e22a2f0863dc82edb962e6e0828e0fa319c4f252f5a2c3223226786e17c1e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11-deactivate, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe) 2024-05-03T18:51:24.133 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:23 smithi150 podman[72480]: 2024-05-03 18:51:23.775970119 +0000 UTC m=+0.494646544 container attach 832e22a2f0863dc82edb962e6e0828e0fa319c4f252f5a2c3223226786e17c1e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240430, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-05-03T18:51:24.509 INFO:journalctl@ceph.osd.11.smithi150.stdout:May 03 18:51:24 smithi150 podman[72506]: 2024-05-03 18:51:24.213940458 +0000 UTC m=+0.035908264 container died 832e22a2f0863dc82edb962e6e0828e0fa319c4f252f5a2c3223226786e17c1e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.schema-version=1.0) 2024-05-03T18:51:24.892 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5703bba4-0978-11ef-bc97-c7b262605968@osd.11.service' 2024-05-03T18:51:24.983 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T18:51:24.983 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-05-03T18:51:24.983 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-05-03T18:51:24.984 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-5703bba4-0978-11ef-bc97-c7b262605968@mds.a 2024-05-03T18:51:25.485 INFO:journalctl@ceph.mds.a.smithi033.stdout:May 03 18:51:25 smithi033 systemd[1]: Stopping Ceph mds.a for 5703bba4-0978-11ef-bc97-c7b262605968... 2024-05-03T18:51:25.985 INFO:journalctl@ceph.mds.a.smithi033.stdout:May 03 18:51:25 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-a[70753]: 2024-05-03T18:51:25.542+0000 7fa3f6000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T18:51:25.985 INFO:journalctl@ceph.mds.a.smithi033.stdout:May 03 18:51:25 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-a[70753]: 2024-05-03T18:51:25.542+0000 7fa3f6000700 -1 mds.a *** got signal Terminated *** 2024-05-03T18:51:29.891 INFO:journalctl@ceph.mds.a.smithi033.stdout:May 03 18:51:29 smithi033 podman[127832]: 2024-05-03 18:51:29.605731863 +0000 UTC m=+4.189886942 container died dfbdd4791c709740fc8ffca5a634f24c2d64210b6b485d97633e117649e025cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-a, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430) 2024-05-03T18:51:30.237 INFO:journalctl@ceph.mds.a.smithi033.stdout:May 03 18:51:30 smithi033 podman[127832]: 2024-05-03 18:51:30.084117068 +0000 UTC m=+4.668272140 container cleanup dfbdd4791c709740fc8ffca5a634f24c2d64210b6b485d97633e117649e025cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-a, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-03T18:51:30.237 INFO:journalctl@ceph.mds.a.smithi033.stdout:May 03 18:51:30 smithi033 bash[127832]: ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-a 2024-05-03T18:51:30.735 INFO:journalctl@ceph.mds.a.smithi033.stdout:May 03 18:51:30 smithi033 podman[127846]: 2024-05-03 18:51:30.277205531 +0000 UTC m=+0.657989804 container remove dfbdd4791c709740fc8ffca5a634f24c2d64210b6b485d97633e117649e025cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-a, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-03T18:51:30.777 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5703bba4-0978-11ef-bc97-c7b262605968@mds.a.service' 2024-05-03T18:51:30.893 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T18:51:30.894 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-05-03T18:51:30.894 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-05-03T18:51:30.894 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-5703bba4-0978-11ef-bc97-c7b262605968@mds.d 2024-05-03T18:51:31.467 INFO:journalctl@ceph.mds.d.smithi033.stdout:May 03 18:51:31 smithi033 systemd[1]: Stopping Ceph mds.d for 5703bba4-0978-11ef-bc97-c7b262605968... 2024-05-03T18:51:31.735 INFO:journalctl@ceph.mds.d.smithi033.stdout:May 03 18:51:31 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-d[69361]: 2024-05-03T18:51:31.553+0000 7f129ac00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T18:51:31.735 INFO:journalctl@ceph.mds.d.smithi033.stdout:May 03 18:51:31 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-d[69361]: 2024-05-03T18:51:31.553+0000 7f129ac00700 -1 mds.d *** got signal Terminated *** 2024-05-03T18:51:35.871 INFO:journalctl@ceph.mds.d.smithi033.stdout:May 03 18:51:35 smithi033 podman[127952]: 2024-05-03 18:51:35.577808046 +0000 UTC m=+4.209516121 container died 8821c4cab9ce53c050e666561c3fa9eace584d67264b0cfea3aa99a8310f73f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-d, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-05-03T18:51:36.124 INFO:journalctl@ceph.mds.d.smithi033.stdout:May 03 18:51:36 smithi033 podman[127952]: 2024-05-03 18:51:36.080126728 +0000 UTC m=+4.711834810 container cleanup 8821c4cab9ce53c050e666561c3fa9eace584d67264b0cfea3aa99a8310f73f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-d, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240430) 2024-05-03T18:51:36.124 INFO:journalctl@ceph.mds.d.smithi033.stdout:May 03 18:51:36 smithi033 bash[127952]: ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-d 2024-05-03T18:51:36.735 INFO:journalctl@ceph.mds.d.smithi033.stdout:May 03 18:51:36 smithi033 podman[127977]: 2024-05-03 18:51:36.398203203 +0000 UTC m=+0.801779026 container remove 8821c4cab9ce53c050e666561c3fa9eace584d67264b0cfea3aa99a8310f73f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-d, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240430, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-05-03T18:51:36.956 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5703bba4-0978-11ef-bc97-c7b262605968@mds.d.service' 2024-05-03T18:51:37.081 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T18:51:37.081 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-05-03T18:51:37.081 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-05-03T18:51:37.081 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-5703bba4-0978-11ef-bc97-c7b262605968@mds.g 2024-05-03T18:51:37.485 INFO:journalctl@ceph.mds.g.smithi033.stdout:May 03 18:51:37 smithi033 systemd[1]: Stopping Ceph mds.g for 5703bba4-0978-11ef-bc97-c7b262605968... 2024-05-03T18:51:38.235 INFO:journalctl@ceph.mds.g.smithi033.stdout:May 03 18:51:37 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-g[70355]: 2024-05-03T18:51:37.774+0000 7f5483a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T18:51:38.235 INFO:journalctl@ceph.mds.g.smithi033.stdout:May 03 18:51:37 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-g[70355]: 2024-05-03T18:51:37.774+0000 7f5483a00700 -1 mds.g *** got signal Terminated *** 2024-05-03T18:51:41.140 INFO:journalctl@ceph.mds.g.smithi033.stdout:May 03 18:51:40 smithi033 podman[128084]: 2024-05-03 18:51:40.875782234 +0000 UTC m=+3.286799763 container died 518816162c381fd21b9339ea93ca2e88f33cdab2567beaf91724e80cd5042207 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-g, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-05-03T18:51:41.468 INFO:journalctl@ceph.mds.g.smithi033.stdout:May 03 18:51:41 smithi033 podman[128084]: 2024-05-03 18:51:41.250106785 +0000 UTC m=+3.661124307 container cleanup 518816162c381fd21b9339ea93ca2e88f33cdab2567beaf91724e80cd5042207 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-g, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True) 2024-05-03T18:51:41.469 INFO:journalctl@ceph.mds.g.smithi033.stdout:May 03 18:51:41 smithi033 bash[128084]: ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-g 2024-05-03T18:51:41.735 INFO:journalctl@ceph.mds.g.smithi033.stdout:May 03 18:51:41 smithi033 podman[128098]: 2024-05-03 18:51:41.467461248 +0000 UTC m=+0.576049033 container remove 518816162c381fd21b9339ea93ca2e88f33cdab2567beaf91724e80cd5042207 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-03T18:51:41.968 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5703bba4-0978-11ef-bc97-c7b262605968@mds.g.service' 2024-05-03T18:51:42.082 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T18:51:42.082 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-05-03T18:51:42.082 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-05-03T18:51:42.082 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-5703bba4-0978-11ef-bc97-c7b262605968@mds.j 2024-05-03T18:51:42.721 INFO:journalctl@ceph.mds.j.smithi033.stdout:May 03 18:51:42 smithi033 systemd[1]: Stopping Ceph mds.j for 5703bba4-0978-11ef-bc97-c7b262605968... 2024-05-03T18:51:42.985 INFO:journalctl@ceph.mds.j.smithi033.stdout:May 03 18:51:42 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-j[68901]: 2024-05-03T18:51:42.718+0000 7fcb77800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T18:51:42.985 INFO:journalctl@ceph.mds.j.smithi033.stdout:May 03 18:51:42 smithi033 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-j[68901]: 2024-05-03T18:51:42.718+0000 7fcb77800700 -1 mds.j *** got signal Terminated *** 2024-05-03T18:51:46.934 INFO:journalctl@ceph.mds.j.smithi033.stdout:May 03 18:51:46 smithi033 podman[128204]: 2024-05-03 18:51:46.570163563 +0000 UTC m=+3.981523619 container died 85f42a639f4287f6a7d09f2ffac894fd02536cdf06f89e7cefaca1c3f22ef141 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-j, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-05-03T18:51:47.185 INFO:journalctl@ceph.mds.j.smithi033.stdout:May 03 18:51:47 smithi033 podman[128204]: 2024-05-03 18:51:47.053400004 +0000 UTC m=+4.464760088 container cleanup 85f42a639f4287f6a7d09f2ffac894fd02536cdf06f89e7cefaca1c3f22ef141 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-j, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux ) 2024-05-03T18:51:47.185 INFO:journalctl@ceph.mds.j.smithi033.stdout:May 03 18:51:47 smithi033 bash[128204]: ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-j 2024-05-03T18:51:47.485 INFO:journalctl@ceph.mds.j.smithi033.stdout:May 03 18:51:47 smithi033 podman[128218]: 2024-05-03 18:51:47.295988846 +0000 UTC m=+0.695180205 container remove 85f42a639f4287f6a7d09f2ffac894fd02536cdf06f89e7cefaca1c3f22ef141 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-j, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240430, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-05-03T18:51:47.858 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5703bba4-0978-11ef-bc97-c7b262605968@mds.j.service' 2024-05-03T18:51:47.971 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T18:51:47.971 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-05-03T18:51:47.972 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-05-03T18:51:47.972 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-5703bba4-0978-11ef-bc97-c7b262605968@mds.b 2024-05-03T18:51:48.296 INFO:journalctl@ceph.mds.b.smithi099.stdout:May 03 18:51:48 smithi099 systemd[1]: Stopping Ceph mds.b for 5703bba4-0978-11ef-bc97-c7b262605968... 2024-05-03T18:51:49.047 INFO:journalctl@ceph.mds.b.smithi099.stdout:May 03 18:51:48 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-b[62001]: 2024-05-03T18:51:48.551+0000 7f94c1200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T18:51:49.047 INFO:journalctl@ceph.mds.b.smithi099.stdout:May 03 18:51:48 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-b[62001]: 2024-05-03T18:51:48.551+0000 7f94c1200700 -1 mds.b *** got signal Terminated *** 2024-05-03T18:51:50.382 INFO:journalctl@ceph.mds.b.smithi099.stdout:May 03 18:51:50 smithi099 podman[71793]: 2024-05-03 18:51:50.108173876 +0000 UTC m=+1.727958356 container died 0372eed14b6b57538b9264c724ef39d4aa2e0051106cf2c623813ba5c9275660 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_CLEAN=True) 2024-05-03T18:51:50.710 INFO:journalctl@ceph.mds.b.smithi099.stdout:May 03 18:51:50 smithi099 podman[71793]: 2024-05-03 18:51:50.500176328 +0000 UTC m=+2.119960801 container cleanup 0372eed14b6b57538b9264c724ef39d4aa2e0051106cf2c623813ba5c9275660 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-b, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.vendor=CentOS) 2024-05-03T18:51:50.711 INFO:journalctl@ceph.mds.b.smithi099.stdout:May 03 18:51:50 smithi099 bash[71793]: ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-b 2024-05-03T18:51:51.046 INFO:journalctl@ceph.mds.b.smithi099.stdout:May 03 18:51:50 smithi099 podman[71807]: 2024-05-03 18:51:50.70942047 +0000 UTC m=+0.571115106 container remove 0372eed14b6b57538b9264c724ef39d4aa2e0051106cf2c623813ba5c9275660 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-b, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-05-03T18:51:51.164 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5703bba4-0978-11ef-bc97-c7b262605968@mds.b.service' 2024-05-03T18:51:51.275 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T18:51:51.275 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-05-03T18:51:51.275 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-05-03T18:51:51.276 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-5703bba4-0978-11ef-bc97-c7b262605968@mds.e 2024-05-03T18:51:51.546 INFO:journalctl@ceph.mds.e.smithi099.stdout:May 03 18:51:51 smithi099 systemd[1]: Stopping Ceph mds.e for 5703bba4-0978-11ef-bc97-c7b262605968... 2024-05-03T18:51:52.296 INFO:journalctl@ceph.mds.e.smithi099.stdout:May 03 18:51:51 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-e[61378]: 2024-05-03T18:51:51.833+0000 7f040fe00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T18:51:52.296 INFO:journalctl@ceph.mds.e.smithi099.stdout:May 03 18:51:51 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-e[61378]: 2024-05-03T18:51:51.833+0000 7f040fe00700 -1 mds.e *** got signal Terminated *** 2024-05-03T18:51:56.235 INFO:journalctl@ceph.mds.e.smithi099.stdout:May 03 18:51:55 smithi099 podman[71913]: 2024-05-03 18:51:55.85769694 +0000 UTC m=+4.143158468 container died 827d8341c3f940ba3d37850c1c5f770aec9dad2bba5082ec65679cac9d706b67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-e, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-03T18:51:56.546 INFO:journalctl@ceph.mds.e.smithi099.stdout:May 03 18:51:56 smithi099 podman[71913]: 2024-05-03 18:51:56.352969394 +0000 UTC m=+4.638430934 container cleanup 827d8341c3f940ba3d37850c1c5f770aec9dad2bba5082ec65679cac9d706b67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-e, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-03T18:51:56.547 INFO:journalctl@ceph.mds.e.smithi099.stdout:May 03 18:51:56 smithi099 bash[71913]: ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-e 2024-05-03T18:51:57.046 INFO:journalctl@ceph.mds.e.smithi099.stdout:May 03 18:51:56 smithi099 podman[71927]: 2024-05-03 18:51:56.55382693 +0000 UTC m=+0.683013061 container remove 827d8341c3f940ba3d37850c1c5f770aec9dad2bba5082ec65679cac9d706b67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-e, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-05-03T18:51:57.090 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5703bba4-0978-11ef-bc97-c7b262605968@mds.e.service' 2024-05-03T18:51:57.196 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T18:51:57.197 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-05-03T18:51:57.197 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-05-03T18:51:57.197 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-5703bba4-0978-11ef-bc97-c7b262605968@mds.h 2024-05-03T18:51:57.546 INFO:journalctl@ceph.mds.h.smithi099.stdout:May 03 18:51:57 smithi099 systemd[1]: Stopping Ceph mds.h for 5703bba4-0978-11ef-bc97-c7b262605968... 2024-05-03T18:51:58.047 INFO:journalctl@ceph.mds.h.smithi099.stdout:May 03 18:51:57 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-h[61696]: 2024-05-03T18:51:57.728+0000 7f20ce200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T18:51:58.047 INFO:journalctl@ceph.mds.h.smithi099.stdout:May 03 18:51:57 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-h[61696]: 2024-05-03T18:51:57.728+0000 7f20ce200700 -1 mds.h *** got signal Terminated *** 2024-05-03T18:52:02.047 INFO:journalctl@ceph.mds.h.smithi099.stdout:May 03 18:52:01 smithi099 podman[72032]: 2024-05-03 18:52:01.752777815 +0000 UTC m=+4.155583085 container died 08471d7f10c42253437c204aa85aeedec31e2160dc9a58d43e8bd4efdd3dd604 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-h, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-05-03T18:52:02.432 INFO:journalctl@ceph.mds.h.smithi099.stdout:May 03 18:52:02 smithi099 podman[72032]: 2024-05-03 18:52:02.180467343 +0000 UTC m=+4.583272606 container cleanup 08471d7f10c42253437c204aa85aeedec31e2160dc9a58d43e8bd4efdd3dd604 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-h, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-05-03T18:52:02.432 INFO:journalctl@ceph.mds.h.smithi099.stdout:May 03 18:52:02 smithi099 bash[72032]: ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-h 2024-05-03T18:52:02.796 INFO:journalctl@ceph.mds.h.smithi099.stdout:May 03 18:52:02 smithi099 podman[72046]: 2024-05-03 18:52:02.431167016 +0000 UTC m=+0.663833217 container remove 08471d7f10c42253437c204aa85aeedec31e2160dc9a58d43e8bd4efdd3dd604 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-h, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe) 2024-05-03T18:52:02.955 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5703bba4-0978-11ef-bc97-c7b262605968@mds.h.service' 2024-05-03T18:52:03.051 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T18:52:03.051 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-05-03T18:52:03.051 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-05-03T18:52:03.051 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-5703bba4-0978-11ef-bc97-c7b262605968@mds.k 2024-05-03T18:52:03.546 INFO:journalctl@ceph.mds.k.smithi099.stdout:May 03 18:52:03 smithi099 systemd[1]: Stopping Ceph mds.k for 5703bba4-0978-11ef-bc97-c7b262605968... 2024-05-03T18:52:04.046 INFO:journalctl@ceph.mds.k.smithi099.stdout:May 03 18:52:03 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-k[61071]: 2024-05-03T18:52:03.698+0000 7f19c5400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T18:52:04.047 INFO:journalctl@ceph.mds.k.smithi099.stdout:May 03 18:52:03 smithi099 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-k[61071]: 2024-05-03T18:52:03.698+0000 7f19c5400700 -1 mds.k *** got signal Terminated *** 2024-05-03T18:52:05.622 INFO:journalctl@ceph.mds.k.smithi099.stdout:May 03 18:52:05 smithi099 podman[72153]: 2024-05-03 18:52:05.312620847 +0000 UTC m=+1.744298758 container died d1fe17b0ec713c4b2e822c2725a8cbf48f010058e8ee785d5be0decd7430fb84 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-k, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-05-03T18:52:06.001 INFO:journalctl@ceph.mds.k.smithi099.stdout:May 03 18:52:05 smithi099 podman[72153]: 2024-05-03 18:52:05.741403408 +0000 UTC m=+2.173081312 container cleanup d1fe17b0ec713c4b2e822c2725a8cbf48f010058e8ee785d5be0decd7430fb84 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-k, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-05-03T18:52:06.002 INFO:journalctl@ceph.mds.k.smithi099.stdout:May 03 18:52:05 smithi099 bash[72153]: ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-k 2024-05-03T18:52:06.296 INFO:journalctl@ceph.mds.k.smithi099.stdout:May 03 18:52:06 smithi099 podman[72167]: 2024-05-03 18:52:06.00017049 +0000 UTC m=+0.673543543 container remove d1fe17b0ec713c4b2e822c2725a8cbf48f010058e8ee785d5be0decd7430fb84 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-k, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430) 2024-05-03T18:52:06.529 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5703bba4-0978-11ef-bc97-c7b262605968@mds.k.service' 2024-05-03T18:52:06.632 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T18:52:06.633 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-05-03T18:52:06.633 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-05-03T18:52:06.633 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-5703bba4-0978-11ef-bc97-c7b262605968@mds.c 2024-05-03T18:52:07.133 INFO:journalctl@ceph.mds.c.smithi150.stdout:May 03 18:52:06 smithi150 systemd[1]: Stopping Ceph mds.c for 5703bba4-0978-11ef-bc97-c7b262605968... 2024-05-03T18:52:07.633 INFO:journalctl@ceph.mds.c.smithi150.stdout:May 03 18:52:07 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-c[63205]: 2024-05-03T18:52:07.185+0000 7f3cf0c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T18:52:07.633 INFO:journalctl@ceph.mds.c.smithi150.stdout:May 03 18:52:07 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-c[63205]: 2024-05-03T18:52:07.185+0000 7f3cf0c00700 -1 mds.c *** got signal Terminated *** 2024-05-03T18:52:08.809 INFO:journalctl@ceph.mds.c.smithi150.stdout:May 03 18:52:08 smithi150 podman[72597]: 2024-05-03 18:52:08.533550672 +0000 UTC m=+1.473101107 container died 78f83371523d8fb123ac087ca9e9098de90b62ea27db5232d7ace81f9ef1cd26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-c, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True) 2024-05-03T18:52:09.122 INFO:journalctl@ceph.mds.c.smithi150.stdout:May 03 18:52:08 smithi150 podman[72597]: 2024-05-03 18:52:08.953432005 +0000 UTC m=+1.892982433 container cleanup 78f83371523d8fb123ac087ca9e9098de90b62ea27db5232d7ace81f9ef1cd26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-05-03T18:52:09.122 INFO:journalctl@ceph.mds.c.smithi150.stdout:May 03 18:52:08 smithi150 bash[72597]: ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-c 2024-05-03T18:52:09.383 INFO:journalctl@ceph.mds.c.smithi150.stdout:May 03 18:52:09 smithi150 podman[72620]: 2024-05-03 18:52:09.120870813 +0000 UTC m=+0.570510175 container remove 78f83371523d8fb123ac087ca9e9098de90b62ea27db5232d7ace81f9ef1cd26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-c, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-03T18:52:09.551 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5703bba4-0978-11ef-bc97-c7b262605968@mds.c.service' 2024-05-03T18:52:09.657 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T18:52:09.657 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-05-03T18:52:09.657 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-05-03T18:52:09.657 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-5703bba4-0978-11ef-bc97-c7b262605968@mds.f 2024-05-03T18:52:10.133 INFO:journalctl@ceph.mds.f.smithi150.stdout:May 03 18:52:09 smithi150 systemd[1]: Stopping Ceph mds.f for 5703bba4-0978-11ef-bc97-c7b262605968... 2024-05-03T18:52:10.633 INFO:journalctl@ceph.mds.f.smithi150.stdout:May 03 18:52:10 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-f[62902]: 2024-05-03T18:52:10.257+0000 7f68d6200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T18:52:10.633 INFO:journalctl@ceph.mds.f.smithi150.stdout:May 03 18:52:10 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-f[62902]: 2024-05-03T18:52:10.257+0000 7f68d6200700 -1 mds.f *** got signal Terminated *** 2024-05-03T18:52:12.441 INFO:journalctl@ceph.mds.f.smithi150.stdout:May 03 18:52:12 smithi150 podman[72728]: 2024-05-03 18:52:12.180997425 +0000 UTC m=+2.047363925 container died 2c9f0e84d96e01405d8920e5af7ad3d1a70ae61c9c78d1ad9eb312ee74701218 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-f, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe) 2024-05-03T18:52:12.751 INFO:journalctl@ceph.mds.f.smithi150.stdout:May 03 18:52:12 smithi150 podman[72728]: 2024-05-03 18:52:12.557391382 +0000 UTC m=+2.423757881 container cleanup 2c9f0e84d96e01405d8920e5af7ad3d1a70ae61c9c78d1ad9eb312ee74701218 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-f, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-05-03T18:52:12.751 INFO:journalctl@ceph.mds.f.smithi150.stdout:May 03 18:52:12 smithi150 bash[72728]: ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-f 2024-05-03T18:52:13.133 INFO:journalctl@ceph.mds.f.smithi150.stdout:May 03 18:52:12 smithi150 podman[72741]: 2024-05-03 18:52:12.749731231 +0000 UTC m=+0.555868312 container remove 2c9f0e84d96e01405d8920e5af7ad3d1a70ae61c9c78d1ad9eb312ee74701218 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-f, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-05-03T18:52:13.287 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5703bba4-0978-11ef-bc97-c7b262605968@mds.f.service' 2024-05-03T18:52:13.389 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T18:52:13.390 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-05-03T18:52:13.390 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-05-03T18:52:13.390 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-5703bba4-0978-11ef-bc97-c7b262605968@mds.i 2024-05-03T18:52:13.883 INFO:journalctl@ceph.mds.i.smithi150.stdout:May 03 18:52:13 smithi150 systemd[1]: Stopping Ceph mds.i for 5703bba4-0978-11ef-bc97-c7b262605968... 2024-05-03T18:52:14.383 INFO:journalctl@ceph.mds.i.smithi150.stdout:May 03 18:52:13 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-i[63508]: 2024-05-03T18:52:13.965+0000 7f8dc3c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T18:52:14.383 INFO:journalctl@ceph.mds.i.smithi150.stdout:May 03 18:52:13 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-i[63508]: 2024-05-03T18:52:13.965+0000 7f8dc3c00700 -1 mds.i *** got signal Terminated *** 2024-05-03T18:52:16.844 INFO:journalctl@ceph.mds.i.smithi150.stdout:May 03 18:52:16 smithi150 podman[72851]: 2024-05-03 18:52:16.535628958 +0000 UTC m=+2.701161217 container died ed43a8474889db2d7cff52192388905ab8c90a19c7549b8e315b37ece40cecd1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-i, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-03T18:52:17.133 INFO:journalctl@ceph.mds.i.smithi150.stdout:May 03 18:52:16 smithi150 podman[72851]: 2024-05-03 18:52:16.943668797 +0000 UTC m=+3.109201059 container cleanup ed43a8474889db2d7cff52192388905ab8c90a19c7549b8e315b37ece40cecd1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430) 2024-05-03T18:52:17.133 INFO:journalctl@ceph.mds.i.smithi150.stdout:May 03 18:52:16 smithi150 bash[72851]: ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-i 2024-05-03T18:52:17.633 INFO:journalctl@ceph.mds.i.smithi150.stdout:May 03 18:52:17 smithi150 podman[72865]: 2024-05-03 18:52:17.227959466 +0000 UTC m=+0.683812819 container remove ed43a8474889db2d7cff52192388905ab8c90a19c7549b8e315b37ece40cecd1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-i, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True) 2024-05-03T18:52:17.684 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5703bba4-0978-11ef-bc97-c7b262605968@mds.i.service' 2024-05-03T18:52:17.782 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T18:52:17.782 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-05-03T18:52:17.782 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-05-03T18:52:17.782 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-5703bba4-0978-11ef-bc97-c7b262605968@mds.l 2024-05-03T18:52:18.133 INFO:journalctl@ceph.mds.l.smithi150.stdout:May 03 18:52:17 smithi150 systemd[1]: Stopping Ceph mds.l for 5703bba4-0978-11ef-bc97-c7b262605968... 2024-05-03T18:52:18.633 INFO:journalctl@ceph.mds.l.smithi150.stdout:May 03 18:52:18 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-l[62585]: 2024-05-03T18:52:18.336+0000 7f3f1a000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T18:52:18.633 INFO:journalctl@ceph.mds.l.smithi150.stdout:May 03 18:52:18 smithi150 ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-l[62585]: 2024-05-03T18:52:18.336+0000 7f3f1a000700 -1 mds.l *** got signal Terminated *** 2024-05-03T18:52:21.496 INFO:journalctl@ceph.mds.l.smithi150.stdout:May 03 18:52:21 smithi150 podman[72972]: 2024-05-03 18:52:21.229848511 +0000 UTC m=+3.017985952 container died 14f4381f26949366e4605c59850eb394b5053417ed59d9ec0dee92482c658472 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-l, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5) 2024-05-03T18:52:21.782 INFO:journalctl@ceph.mds.l.smithi150.stdout:May 03 18:52:21 smithi150 podman[72972]: 2024-05-03 18:52:21.613731714 +0000 UTC m=+3.401869145 container cleanup 14f4381f26949366e4605c59850eb394b5053417ed59d9ec0dee92482c658472 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-l, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-05-03T18:52:21.782 INFO:journalctl@ceph.mds.l.smithi150.stdout:May 03 18:52:21 smithi150 bash[72972]: ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-l 2024-05-03T18:52:22.133 INFO:journalctl@ceph.mds.l.smithi150.stdout:May 03 18:52:21 smithi150 podman[72986]: 2024-05-03 18:52:21.781234078 +0000 UTC m=+0.540274919 container remove 14f4381f26949366e4605c59850eb394b5053417ed59d9ec0dee92482c658472 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-5703bba4-0978-11ef-bc97-c7b262605968-mds-l, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-05-03T18:52:22.254 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5703bba4-0978-11ef-bc97-c7b262605968@mds.l.service' 2024-05-03T18:52:22.350 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T18:52:22.350 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-05-03T18:52:22.350 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5703bba4-0978-11ef-bc97-c7b262605968 --force --keep-logs 2024-05-03T18:52:22.689 INFO:teuthology.orchestra.run.smithi033.stdout:Deleting cluster with fsid: 5703bba4-0978-11ef-bc97-c7b262605968 2024-05-03T18:52:27.939 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5703bba4-0978-11ef-bc97-c7b262605968 --force --keep-logs 2024-05-03T18:52:28.263 INFO:teuthology.orchestra.run.smithi099.stdout:Deleting cluster with fsid: 5703bba4-0978-11ef-bc97-c7b262605968 2024-05-03T18:52:33.246 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5703bba4-0978-11ef-bc97-c7b262605968 --force --keep-logs 2024-05-03T18:52:33.552 INFO:teuthology.orchestra.run.smithi150.stdout:Deleting cluster with fsid: 5703bba4-0978-11ef-bc97-c7b262605968 2024-05-03T18:52:38.683 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-03T18:52:38.773 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-03T18:52:38.859 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-03T18:52:38.934 INFO:tasks.cephadm:Archiving crash dumps... 2024-05-03T18:52:38.935 DEBUG:teuthology.misc:Transferring archived files from smithi033:/var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-05-03_13:56:19-fs-wip-rishabh-testing-20240501.193033-testing-default-smithi/7687421/remote/smithi033/crash 2024-05-03T18:52:38.936 DEBUG:teuthology.orchestra.run.smithi033:> sudo tar c -f - -C /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/crash -- . 2024-05-03T18:52:39.076 INFO:teuthology.orchestra.run.smithi033.stderr:tar: /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/crash: Cannot open: No such file or directory 2024-05-03T18:52:39.076 INFO:teuthology.orchestra.run.smithi033.stderr:tar: Error is not recoverable: exiting now 2024-05-03T18:52:39.083 DEBUG:teuthology.misc:Transferring archived files from smithi099:/var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-05-03_13:56:19-fs-wip-rishabh-testing-20240501.193033-testing-default-smithi/7687421/remote/smithi099/crash 2024-05-03T18:52:39.084 DEBUG:teuthology.orchestra.run.smithi099:> sudo tar c -f - -C /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/crash -- . 2024-05-03T18:52:39.184 INFO:teuthology.orchestra.run.smithi099.stderr:tar: /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/crash: Cannot open: No such file or directory 2024-05-03T18:52:39.184 INFO:teuthology.orchestra.run.smithi099.stderr:tar: Error is not recoverable: exiting now 2024-05-03T18:52:39.188 DEBUG:teuthology.misc:Transferring archived files from smithi150:/var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-05-03_13:56:19-fs-wip-rishabh-testing-20240501.193033-testing-default-smithi/7687421/remote/smithi150/crash 2024-05-03T18:52:39.189 DEBUG:teuthology.orchestra.run.smithi150:> sudo tar c -f - -C /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/crash -- . 2024-05-03T18:52:39.281 INFO:teuthology.orchestra.run.smithi150.stderr:tar: /var/lib/ceph/5703bba4-0978-11ef-bc97-c7b262605968/crash: Cannot open: No such file or directory 2024-05-03T18:52:39.281 INFO:teuthology.orchestra.run.smithi150.stderr:tar: Error is not recoverable: exiting now 2024-05-03T18:52:39.284 INFO:tasks.cephadm:Checking cluster log for badness... 2024-05-03T18:52:39.285 DEBUG:teuthology.orchestra.run.smithi033:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-05-03T18:52:39.469 INFO:tasks.cephadm:Compressing logs... 2024-05-03T18:52:39.469 DEBUG:teuthology.orchestra.run.smithi033:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-03T18:52:39.474 DEBUG:teuthology.orchestra.run.smithi099:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-03T18:52:39.479 DEBUG:teuthology.orchestra.run.smithi150:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-03T18:52:39.562 INFO:teuthology.orchestra.run.smithi099.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose --: No such file or directory 2024-05-03T18:52:39.562 INFO:teuthology.orchestra.run.smithi099.stderr: /var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph.log 2024-05-03T18:52:39.562 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph.audit.log 2024-05-03T18:52:39.563 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph-osd.7.log 2024-05-03T18:52:39.564 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph-mds.b.log 2024-05-03T18:52:39.565 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph-osd.4.log 2024-05-03T18:52:39.566 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph-osd.1.log 2024-05-03T18:52:39.567 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph-mgr.y.log 2024-05-03T18:52:39.568 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph-mds.h.log 2024-05-03T18:52:39.569 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph-osd.10.log 2024-05-03T18:52:39.570 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph.cephadm.log 2024-05-03T18:52:39.571 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph-mds.e.log 2024-05-03T18:52:39.571 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph-volume.log 2024-05-03T18:52:39.572 INFO:teuthology.orchestra.run.smithi033.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-05-03T18:52:39.572 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.log 2024-05-03T18:52:39.573 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph-mon.b.log 2024-05-03T18:52:39.573 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph-mds.k.log 2024-05-03T18:52:39.574 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-03T18:52:39.576 INFO:teuthology.orchestra.run.smithi150.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph.log 2024-05-03T18:52:39.577 INFO:teuthology.orchestra.run.smithi150.stderr:: No such file or directory 2024-05-03T18:52:39.577 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph-osd.2.log 2024-05-03T18:52:39.578 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph-mds.f.log 2024-05-03T18:52:39.579 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph-mds.k.log: /var/log/ceph/cephadm.log: 94.3% -- replaced with /var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph-mds.e.log.gz 2024-05-03T18:52:39.579 INFO:teuthology.orchestra.run.smithi099.stderr: 92.5% -- replaced with /var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph-mgr.y.log.gz 2024-05-03T18:52:39.580 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph.audit.log 2024-05-03T18:52:39.580 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph.log 2024-05-03T18:52:39.581 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph-osd.5.log 2024-05-03T18:52:39.582 INFO:teuthology.orchestra.run.smithi099.stderr: 94.4% -- replaced with /var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph-mds.k.log.gz 2024-05-03T18:52:39.582 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.0.log: gzip -5 --verbose -- /var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph.audit.log 2024-05-03T18:52:39.583 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph-mds.c.log 2024-05-03T18:52:39.583 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph-mon.c.log 2024-05-03T18:52:39.584 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph-mds.g.log 2024-05-03T18:52:39.584 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph-mgr.z.log 2024-05-03T18:52:39.585 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5/var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph.audit.log: --verbose -- /var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph-mds.j.log 2024-05-03T18:52:39.586 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph-mds.i.log 2024-05-03T18:52:39.586 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph-osd.3.log 2024-05-03T18:52:39.587 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph.cephadm.log 2024-05-03T18:52:39.587 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph-mgr.z.log: 94.4% -- replaced with /var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph-mds.f.log.gz 2024-05-03T18:52:39.588 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph-osd.11.log 2024-05-03T18:52:39.588 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph-mds.j.log: 87.7% -- replaced with /var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph.log.gz 2024-05-03T18:52:39.588 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph-osd.0.log 2024-05-03T18:52:39.589 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph-mgr.x.log 2024-05-03T18:52:39.590 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph-osd.8.log 2024-05-03T18:52:39.590 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph-volume.log 2024-05-03T18:52:39.591 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph-mon.a.log 2024-05-03T18:52:39.591 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph-mds.l.log 2024-05-03T18:52:39.592 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph-osd.6.log 2024-05-03T18:52:39.592 INFO:teuthology.orchestra.run.smithi033.stderr: 94.4% -- replaced with /var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph-mds.g.log.gz 2024-05-03T18:52:39.593 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph.cephadm.log 2024-05-03T18:52:39.594 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-03T18:52:39.594 INFO:teuthology.orchestra.run.smithi150.stderr: 94.3% -- replaced with /var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph-mds.c.log.gz 2024-05-03T18:52:39.594 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph-mds.l.log: /var/log/ceph/cephadm.log: 94.5% -- replaced with /var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph-mds.i.log.gz 2024-05-03T18:52:39.595 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph-mds.a.log 2024-05-03T18:52:39.595 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph-volume.log 2024-05-03T18:52:39.596 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph-mds.d.log 2024-05-03T18:52:39.597 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph-volume.log: 94.4% -- replaced with /var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph-mds.j.log.gz 2024-05-03T18:52:39.598 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph-osd.9.log 2024-05-03T18:52:39.598 INFO:teuthology.orchestra.run.smithi150.stderr: 92.7% -- replaced with /var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph-mgr.z.log.gz 2024-05-03T18:52:39.599 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-03T18:52:39.599 INFO:teuthology.orchestra.run.smithi150.stderr: 94.4% -- replaced with /var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph-mds.l.log.gz 2024-05-03T18:52:39.600 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-05-03T18:52:39.603 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/ceph-client.admin.log: 94.2% -- replaced with /var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph-mds.a.log.gz 2024-05-03T18:52:39.604 INFO:teuthology.orchestra.run.smithi033.stderr: 94.3% -- replaced with /var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph-mds.d.log.gz 2024-05-03T18:52:39.622 INFO:teuthology.orchestra.run.smithi033.stderr: 92.6% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-05-03T18:52:39.649 INFO:teuthology.orchestra.run.smithi150.stderr: 84.0% -- replaced with /var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph.cephadm.log.gz 2024-05-03T18:52:39.657 INFO:teuthology.orchestra.run.smithi033.stderr: 84.3% -- replaced with /var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph.cephadm.log.gz 2024-05-03T18:52:39.657 INFO:teuthology.orchestra.run.smithi099.stderr: 84.1% -- replaced with /var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph.cephadm.log.gz 2024-05-03T18:52:39.729 INFO:teuthology.orchestra.run.smithi033.stderr: 90.2% -- replaced with /var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph.audit.log.gz 2024-05-03T18:52:39.806 INFO:teuthology.orchestra.run.smithi033.stderr: 91.0% -- replaced with /var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph-volume.log.gz 2024-05-03T18:52:39.817 INFO:teuthology.orchestra.run.smithi099.stderr: 90.9% -- replaced with /var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph-volume.log.gz 2024-05-03T18:52:39.845 INFO:teuthology.orchestra.run.smithi099.stderr: 87.7% -- replaced with /var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph.log.gz 2024-05-03T18:52:39.961 INFO:teuthology.orchestra.run.smithi033.stderr: 94.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-03T18:52:39.971 INFO:teuthology.orchestra.run.smithi150.stderr: 87.7% -- replaced with /var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph.log.gz 2024-05-03T18:52:39.971 INFO:teuthology.orchestra.run.smithi150.stderr: 92.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-03T18:52:39.971 INFO:teuthology.orchestra.run.smithi150.stderr: 90.4% -- replaced with /var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph.audit.log.gz 2024-05-03T18:52:40.006 INFO:teuthology.orchestra.run.smithi150.stderr: 90.9% -- replaced with /var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph-volume.log.gz 2024-05-03T18:52:40.560 INFO:teuthology.orchestra.run.smithi099.stderr: 91.9% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-03T18:52:40.856 INFO:teuthology.orchestra.run.smithi099.stderr: 90.4% -- replaced with /var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph.audit.log.gz 2024-05-03T18:52:42.406 INFO:teuthology.orchestra.run.smithi150.stderr: 91.7% -- replaced with /var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph-mon.c.log.gz 2024-05-03T18:52:42.493 INFO:teuthology.orchestra.run.smithi099.stderr: 91.7% -- replaced with /var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph-mon.b.log.gz 2024-05-03T18:52:43.352 INFO:teuthology.orchestra.run.smithi033.stderr: 90.2% -- replaced with /var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph-mgr.x.log.gz 2024-05-03T18:52:49.106 INFO:teuthology.orchestra.run.smithi099.stderr: 95.3% -- replaced with /var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph-mds.h.log.gz 2024-05-03T18:52:53.386 INFO:teuthology.orchestra.run.smithi033.stderr: 90.3% -- replaced with /var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph-mon.a.log.gz 2024-05-03T18:54:05.180 INFO:teuthology.orchestra.run.smithi099.stderr: 94.1% -- replaced with /var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph-mds.b.log.gz 2024-05-03T18:56:35.885 INFO:teuthology.orchestra.run.smithi099.stderr: 91.0% -- replaced with /var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph-osd.10.log.gz 2024-05-03T18:58:02.834 INFO:teuthology.orchestra.run.smithi099.stderr: 91.1% -- replaced with /var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph-osd.1.log.gz 2024-05-03T18:59:01.926 INFO:teuthology.orchestra.run.smithi099.stderr: 90.9% -- replaced with /var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph-osd.7.log.gz 2024-05-03T18:59:10.492 INFO:teuthology.orchestra.run.smithi099.stderr: 91.0% -- replaced with /var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph-osd.4.log.gz 2024-05-03T18:59:10.510 INFO:teuthology.orchestra.run.smithi099.stderr: 2024-05-03T18:59:10.510 INFO:teuthology.orchestra.run.smithi099.stderr:real 6m30.973s 2024-05-03T18:59:10.510 INFO:teuthology.orchestra.run.smithi099.stderr:user 9m51.364s 2024-05-03T18:59:10.510 INFO:teuthology.orchestra.run.smithi099.stderr:sys 1m4.427s 2024-05-03T19:03:41.137 INFO:teuthology.orchestra.run.smithi150.stderr: 91.0% -- replaced with /var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph-osd.8.log.gz 2024-05-03T19:03:41.998 INFO:teuthology.orchestra.run.smithi150.stderr: 91.0% -- replaced with /var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph-osd.11.log.gz 2024-05-03T19:04:27.937 INFO:teuthology.orchestra.run.smithi150.stderr: 91.0% -- replaced with /var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph-osd.2.log.gz 2024-05-03T19:04:58.899 INFO:teuthology.orchestra.run.smithi150.stderr: 90.9% -- replaced with /var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph-osd.5.log.gz 2024-05-03T19:04:58.906 INFO:teuthology.orchestra.run.smithi150.stderr: 2024-05-03T19:04:58.906 INFO:teuthology.orchestra.run.smithi150.stderr:real 12m19.380s 2024-05-03T19:04:58.906 INFO:teuthology.orchestra.run.smithi150.stderr:user 15m10.413s 2024-05-03T19:04:58.906 INFO:teuthology.orchestra.run.smithi150.stderr:sys 1m43.263s 2024-05-03T19:08:23.412 INFO:teuthology.orchestra.run.smithi033.stderr: 91.0% -- replaced with /var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph-osd.0.log.gz 2024-05-03T19:09:30.320 INFO:teuthology.orchestra.run.smithi033.stderr: 94.2% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2024-05-03T19:09:56.490 INFO:teuthology.orchestra.run.smithi033.stderr: 90.9% -- replaced with /var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph-osd.6.log.gz 2024-05-03T19:10:35.569 INFO:teuthology.orchestra.run.smithi033.stderr: 90.9% -- replaced with /var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph-osd.3.log.gz 2024-05-03T19:11:33.725 INFO:teuthology.orchestra.run.smithi033.stderr: 90.9% -- replaced with /var/log/ceph/5703bba4-0978-11ef-bc97-c7b262605968/ceph-osd.9.log.gz 2024-05-03T19:11:33.729 INFO:teuthology.orchestra.run.smithi033.stderr: 2024-05-03T19:11:33.729 INFO:teuthology.orchestra.run.smithi033.stderr:real 18m54.214s 2024-05-03T19:11:33.729 INFO:teuthology.orchestra.run.smithi033.stderr:user 21m38.465s 2024-05-03T19:11:33.730 INFO:teuthology.orchestra.run.smithi033.stderr:sys 2m39.448s 2024-05-03T19:11:33.731 INFO:tasks.cephadm:Archiving logs... 2024-05-03T19:11:33.732 DEBUG:teuthology.misc:Transferring archived files from smithi033:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-05-03_13:56:19-fs-wip-rishabh-testing-20240501.193033-testing-default-smithi/7687421/remote/smithi033/log 2024-05-03T19:11:33.733 DEBUG:teuthology.orchestra.run.smithi033:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-03T19:12:55.553 DEBUG:teuthology.misc:Transferring archived files from smithi099:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-05-03_13:56:19-fs-wip-rishabh-testing-20240501.193033-testing-default-smithi/7687421/remote/smithi099/log 2024-05-03T19:12:55.554 DEBUG:teuthology.orchestra.run.smithi099:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-03T19:13:51.920 DEBUG:teuthology.misc:Transferring archived files from smithi150:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-05-03_13:56:19-fs-wip-rishabh-testing-20240501.193033-testing-default-smithi/7687421/remote/smithi150/log 2024-05-03T19:13:51.921 DEBUG:teuthology.orchestra.run.smithi150:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-03T19:14:59.149 INFO:tasks.cephadm:Removing cluster... 2024-05-03T19:14:59.150 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5703bba4-0978-11ef-bc97-c7b262605968 --force 2024-05-03T19:14:59.532 INFO:teuthology.orchestra.run.smithi033.stdout:Deleting cluster with fsid: 5703bba4-0978-11ef-bc97-c7b262605968 2024-05-03T19:15:01.484 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5703bba4-0978-11ef-bc97-c7b262605968 --force 2024-05-03T19:15:02.037 INFO:teuthology.orchestra.run.smithi099.stdout:Deleting cluster with fsid: 5703bba4-0978-11ef-bc97-c7b262605968 2024-05-03T19:15:03.838 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5703bba4-0978-11ef-bc97-c7b262605968 --force 2024-05-03T19:15:04.419 INFO:teuthology.orchestra.run.smithi150.stdout:Deleting cluster with fsid: 5703bba4-0978-11ef-bc97-c7b262605968 2024-05-03T19:15:06.470 INFO:tasks.cephadm:Removing cephadm ... 2024-05-03T19:15:06.470 DEBUG:teuthology.orchestra.run.smithi033:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-03T19:15:06.539 DEBUG:teuthology.orchestra.run.smithi099:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-03T19:15:06.598 DEBUG:teuthology.orchestra.run.smithi150:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-03T19:15:06.652 INFO:tasks.cephadm:Teardown complete 2024-05-03T19:15:06.653 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-03T19:15:06.665 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-03T19:15:06.665 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-03T19:15:06.673 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-03T19:15:06.681 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-03T19:15:06.850 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-03T19:15:06.861 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-03T19:15:06.862 DEBUG:teuthology.orchestra.run.smithi033:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-03T19:15:06.870 DEBUG:teuthology.orchestra.run.smithi099:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-03T19:15:06.877 DEBUG:teuthology.orchestra.run.smithi150:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-03T19:15:06.920 INFO:teuthology.orchestra.run.smithi033.stderr:bash: line 1: ntpq: command not found 2024-05-03T19:15:06.924 INFO:teuthology.orchestra.run.smithi099.stderr:bash: line 1: ntpq: command not found 2024-05-03T19:15:06.933 INFO:teuthology.orchestra.run.smithi150.stderr:bash: line 1: ntpq: command not found 2024-05-03T19:15:07.020 INFO:teuthology.orchestra.run.smithi033.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-03T19:15:07.020 INFO:teuthology.orchestra.run.smithi033.stdout:=============================================================================== 2024-05-03T19:15:07.021 INFO:teuthology.orchestra.run.smithi033.stdout:^+ hv01.front.sepia.ceph.com 3 6 377 35 +1314us[+1308us] +/- 40ms 2024-05-03T19:15:07.021 INFO:teuthology.orchestra.run.smithi033.stdout:^* hv02.front.sepia.ceph.com 3 6 377 29 -1027us[-1033us] +/- 50ms 2024-05-03T19:15:07.021 INFO:teuthology.orchestra.run.smithi033.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 30 -1210us[-1217us] +/- 88ms 2024-05-03T19:15:07.021 INFO:teuthology.orchestra.run.smithi033.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-05-03T19:15:07.035 INFO:teuthology.orchestra.run.smithi099.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-03T19:15:07.035 INFO:teuthology.orchestra.run.smithi099.stdout:=============================================================================== 2024-05-03T19:15:07.035 INFO:teuthology.orchestra.run.smithi099.stdout:^* hv01.front.sepia.ceph.com 3 6 377 34 +1280us[+1282us] +/- 40ms 2024-05-03T19:15:07.035 INFO:teuthology.orchestra.run.smithi099.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 33 -1069us[-1069us] +/- 50ms 2024-05-03T19:15:07.035 INFO:teuthology.orchestra.run.smithi099.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 34 -1272us[-1270us] +/- 88ms 2024-05-03T19:15:07.035 INFO:teuthology.orchestra.run.smithi099.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-05-03T19:15:07.064 INFO:teuthology.orchestra.run.smithi150.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-03T19:15:07.064 INFO:teuthology.orchestra.run.smithi150.stdout:=============================================================================== 2024-05-03T19:15:07.065 INFO:teuthology.orchestra.run.smithi150.stdout:^+ hv01.front.sepia.ceph.com 3 6 377 37 +1380us[+1380us] +/- 40ms 2024-05-03T19:15:07.065 INFO:teuthology.orchestra.run.smithi150.stdout:^* hv02.front.sepia.ceph.com 3 6 377 37 -908us[ -844us] +/- 50ms 2024-05-03T19:15:07.065 INFO:teuthology.orchestra.run.smithi150.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 34 -1142us[-1142us] +/- 88ms 2024-05-03T19:15:07.065 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-05-03T19:15:07.069 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-03T19:15:07.079 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-03T19:15:07.079 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-03T19:15:07.110 DEBUG:teuthology.orchestra.run.smithi033:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-03T19:15:07.474 DEBUG:teuthology.orchestra.run.smithi099:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-03T19:15:07.729 DEBUG:teuthology.orchestra.run.smithi150:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-03T19:15:07.963 DEBUG:teuthology.orchestra.run.smithi033:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-03T19:15:08.059 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-03T19:15:08.060 DEBUG:teuthology.orchestra.run.smithi099:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-03T19:15:08.155 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-03T19:15:08.156 DEBUG:teuthology.orchestra.run.smithi150:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-03T19:15:08.251 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-03T19:15:08.252 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-03T19:15:08.263 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-03T19:15:08.318 INFO:teuthology.task.internal:Duration was 4395.710734 seconds 2024-05-03T19:15:08.319 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-03T19:15:08.386 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-03T19:15:08.386 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-03T19:15:08.394 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-03T19:15:08.402 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-03T19:15:08.550 INFO:teuthology.orchestra.run.smithi033.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-03T19:15:08.558 INFO:teuthology.orchestra.run.smithi150.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-03T19:15:08.566 INFO:teuthology.orchestra.run.smithi099.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-03T19:15:08.989 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-03T19:15:08.989 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi033.front.sepia.ceph.com 2024-05-03T19:15:08.989 DEBUG:teuthology.orchestra.run.smithi033:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-03T19:15:09.085 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi099.front.sepia.ceph.com 2024-05-03T19:15:09.086 DEBUG:teuthology.orchestra.run.smithi099:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-03T19:15:09.165 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi150.front.sepia.ceph.com 2024-05-03T19:15:09.165 DEBUG:teuthology.orchestra.run.smithi150:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-03T19:15:09.240 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-03T19:15:09.241 DEBUG:teuthology.orchestra.run.smithi033:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-03T19:15:09.249 DEBUG:teuthology.orchestra.run.smithi099:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-03T19:15:09.256 DEBUG:teuthology.orchestra.run.smithi150:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-03T19:15:09.341 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-03T19:15:09.341 DEBUG:teuthology.orchestra.run.smithi033:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-03T19:15:09.346 DEBUG:teuthology.orchestra.run.smithi099:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-03T19:15:09.352 DEBUG:teuthology.orchestra.run.smithi150:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-03T19:15:10.764 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-03T19:15:10.776 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-03T19:15:10.776 DEBUG:teuthology.orchestra.run.smithi033:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-03T19:15:10.867 DEBUG:teuthology.orchestra.run.smithi099:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-03T19:15:10.959 DEBUG:teuthology.orchestra.run.smithi150:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-03T19:15:11.050 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-03T19:15:11.062 DEBUG:teuthology.orchestra.run.smithi033:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-03T19:15:11.070 DEBUG:teuthology.orchestra.run.smithi099:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-03T19:15:11.078 DEBUG:teuthology.orchestra.run.smithi150:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-03T19:15:11.173 INFO:teuthology.orchestra.run.smithi033.stdout:kernel.core_pattern = core 2024-05-03T19:15:11.185 INFO:teuthology.orchestra.run.smithi099.stdout:kernel.core_pattern = core 2024-05-03T19:15:11.195 INFO:teuthology.orchestra.run.smithi150.stdout:kernel.core_pattern = core 2024-05-03T19:15:11.245 DEBUG:teuthology.orchestra.run.smithi033:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-03T19:15:11.298 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-03T19:15:11.299 DEBUG:teuthology.orchestra.run.smithi099:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-03T19:15:11.354 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-03T19:15:11.355 DEBUG:teuthology.orchestra.run.smithi150:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-03T19:15:11.409 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-03T19:15:11.409 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-03T19:15:11.421 INFO:teuthology.task.internal:Transferring archived files... 2024-05-03T19:15:11.422 DEBUG:teuthology.misc:Transferring archived files from smithi033:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-05-03_13:56:19-fs-wip-rishabh-testing-20240501.193033-testing-default-smithi/7687421/remote/smithi033 2024-05-03T19:15:11.422 DEBUG:teuthology.orchestra.run.smithi033:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-03T19:15:11.560 DEBUG:teuthology.misc:Transferring archived files from smithi099:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-05-03_13:56:19-fs-wip-rishabh-testing-20240501.193033-testing-default-smithi/7687421/remote/smithi099 2024-05-03T19:15:11.560 DEBUG:teuthology.orchestra.run.smithi099:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-03T19:15:11.660 DEBUG:teuthology.misc:Transferring archived files from smithi150:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-05-03_13:56:19-fs-wip-rishabh-testing-20240501.193033-testing-default-smithi/7687421/remote/smithi150 2024-05-03T19:15:11.661 DEBUG:teuthology.orchestra.run.smithi150:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-03T19:15:11.760 INFO:teuthology.task.internal:Removing archive directory... 2024-05-03T19:15:11.760 DEBUG:teuthology.orchestra.run.smithi033:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-03T19:15:11.768 DEBUG:teuthology.orchestra.run.smithi099:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-03T19:15:11.776 DEBUG:teuthology.orchestra.run.smithi150:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-03T19:15:11.832 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-03T19:15:11.846 INFO:teuthology.task.internal:Not uploading archives. 2024-05-03T19:15:11.846 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-03T19:15:11.903 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-03T19:15:11.903 DEBUG:teuthology.orchestra.run.smithi033:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-03T19:15:11.911 DEBUG:teuthology.orchestra.run.smithi099:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-03T19:15:11.918 DEBUG:teuthology.orchestra.run.smithi150:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-03T19:15:11.965 INFO:teuthology.orchestra.run.smithi033.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 3 19:15 /home/ubuntu/cephtest 2024-05-03T19:15:11.967 INFO:teuthology.orchestra.run.smithi099.stdout: 393949 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 3 19:15 /home/ubuntu/cephtest 2024-05-03T19:15:11.977 INFO:teuthology.orchestra.run.smithi150.stdout: 393251 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 3 19:15 /home/ubuntu/cephtest 2024-05-03T19:15:11.980 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-03T19:15:11.992 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-03T19:15:12.089 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-quiesce/no 6-workunit/postgres}} duration: 4395.710733890533 flavor: default mon.a-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 mon.b-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 mon.c-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 owner: scheduled_rishabh@teuthology success: true 2024-05-03T19:15:12.090 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-03T19:15:12.167 INFO:teuthology.run:pass